AdsDax were one of the first companies to launch our application on the Hedera Hashgraph mainnet when it launched during last year’s Open Access event. If you have been following the news and announcements since Open Access, you may be familiar with the work that we are doing (see How we add transactions at scale, How we flew past 10 million transactions and our previous blog post about how we achieve this here).
It is fair to say that we are heavy users of the network. When we were initially approached about exploring the Hedera Consensus Service, it's also fair to say that we were excited. How excited? Since we launched our tracking topic on the testnet on the 24th January, we’ve tracked over 45m event messages using the Consensus Service.
In order to give more clarity into both what we are doing and the thought processes behind our choices, we’ve created a demo application using the Hedera Go SDK that showcases the end-to-end process as events are tracked, sent to the Hedera network and then collected after being processed by the Consensus Service.
It is important to note that the demo is a some-what simplified version of our production logic, which is necessary in order to reduce the amount of dependencies a user would have to install when trying to run the application.
The demo itself includes a readme file that should give the user some more technical information about what is happening in the application, and some pointers towards which areas you might need to change in order to make your own applications.
As it stands, the only requirements to run the application is that the user must have at least version 1.13 of Go installed, which is a requirement stemming from the Hedera Go SDK itself.
Broadly speaking the demo application consists of three main segments: the “demo.env” file which contains configuration values, the “demo.html” file which gets rendered in the browser and the “main.go” file which contains the Go logic to run a web-server and interact with the Hedera network.
The demo application is supplied with default values in the “demo.env” file for use on the testnet. If you wanted to use your own testnet account or change the demo to run on the mainnet, you can replace the values as required. One noteworthy feature to be aware of is that if you want to create a new topic, clearing the “TOPIC_ID”, “TOPIC_ADMIN_KEY” and “TOPIC_SUBMIT_KEY” values in the config will force a new topic to be generated the next time the application is run.
The “demo.html” file is a fairly simple web-page, which listens for events being dispatched as a video is viewed, sending those events to the Go web-server using the “/track” route, and retrieving those messages from the web-server using the “/retrieve” route. All of the information is displayed in the log viewer as it happens, so the user can follow in real time as events are tracked and pass through consensus.
Whilst the “/track” and “/retrieve” routes are used to pass message information back to the browser to display to the user, all of the tracking logic itself happens in the “main.go” file. Broadly speaking, the demo follows the same process as the live AdsDax platform, which looks like this:
The AdsDax platform can use separate groups within our message queue to handle the flow of how events and messages are processed (in the above image, the orange, purple and blue blocks are handled by separate groups).
In the demo application, in order to avoid having to make the user install and manage a queue service we handle this process slightly differently making use of different routes within the web-server.
A Topic subscriber is started just before we start the web-server, which ensures we don’t miss any events. Events are tracked, reformatted and sent to the Consensus Service via the “/track” route. As the subscriber learns of new messages, it adds the additional consensus information to the message, decrypts the private payload and stores it in an event store, which is repeatedly checked in order to return the information via the “/retrieve” route.
In both the demo application and our live platform, we make use of hybrid encrypted messages, where the message data has both human-readable and non-readable sections. These “private” sections are encrypted using AES-256 encryption using 32-byte encryption key, with each advertiser in our system using their own unique key. This ensures that Advertiser’s sensitive information remains secure, and prevents anybody from scraping all of our Topic information to try and gather information about our users and their browsing habits.
Whilst both ourselves and our advertising partners see the need for increased transparency in the advertising eco-system, being fully transparent with all event data has several issues which include:
As users of the Hedera Cryptocurrency Service, you might expect that there is fairly little for us to gain from switching to the Consensus Service. The speed and throughput are practically the same, as are the costs if you exclude the extra costs associated with having larger messages.
One massive benefit is the increased message size available when using the Consensus Service. Mixed with the high transaction throughput, we can finally track our full event data on a 1:1 basis in real time, with the added benefit of it being signed off by the ledger giving us consensus timestamps and fair-ordering, with aBFT-assured immutability and finality.
Another benefit of the Consensus Service is that it decouples the tracking element of our operations from the payment element, which is very important for some of our advertisers. This is not only due to the sensitivity of price information, but another issue is that the legal standing of crypto-currency transfers in some regions is still questionable. For instance, at the time of writing there is still an on-going court case in India concerning the Reserve Bank of India’s stance on the legality of owning, possessing and transferring crypto-currencies and crypto-assets.
Being able to separate these concerns and approach large markets without the threat or concern surrounding legal issues is one area where we feel the Consensus Service really excels when compared to the existing Cryptocurrency Service.
Another great benefit of the Consensus Service is that it isn’t a case of an either / or system when compared to the Cryptocurrency Service, as they can actually be used in a symbiotic manner, with the Consensus Service being used to augment the Cryptocurrency Service rather than replace it.
For example, we make a lot of payments with multiple accounts on the same transfer, but it’s not immediately apparent who those accounts belong to or why those payments are being made. Now, we could use the Consensus Service to post an itemised invoice to a topic containing all of our financial records, optionally encrypted to prevent it being humanly-readable, and then use that message ID as the memo for our Cryptocurrency payment. This then avoids the need to use external services such as IPFS, and keeps all of your referenced information within the Hedera eco-system.
Whilst we are excited for the future of the Consensus Service, the truth is we have only begun scratching the surface of what is possible. We are continuing our discussions with advertisers and publishers to ensure we are offering the most transparent advertising marketplace possible, whilst still respecting each party's needs and requirements are met.
One step we are taking towards this is by continuing our on-going relationship with the DragonGlass team by offering our Advertising partners the ability to stream all of their advert events in real-time without having to set up their own mirror nodes or even installing any of the Hedera SDKs.
We look forward to using HCS on the mainnet soon, and seeing what other exciting and innovative uses that the rest of the Hedera community has found for using the Consensus Service.