Unstoppable Data for Unstoppable Apps: A DATAcoin Whitepaper Draft

There’s a revolution brewing in computing infrastructure. The winning architecture will be decentralized apps (Ðapps): Front-ends served to the browser from decentralized storage, and back-ends implemented as smart contracts living on the blockchain.

This stack has many desirable properties: It makes applications naturally resistant to distributed denial-of-service attacks, it adds transparency and security by keeping an untamperable transaction log, it has zero reliance on commercial services, and it enables easy peer-to-peer transfers of value in every application.

But what serves this new breed of applications with data? How to make an instant messaging Ðapp capable of delivering a million messages per second? What about a stock trading Ðapp? Vehicle fleet tracking? Factory automation? Air traffic control?

At first sight, blockchains as decentralized databases seem perfect for data distribution. (We’ve even demonstrated the promise of this pattern in a previous blog post.) It’s a powerful concept: write a piece of data to a smart contract, and all nodes in the network get a copy. Front-ends can watch the contract data on a local node, and update accordingly. Unstoppable! Ok, data delivery is a bit slow due to mining delay, but hey, still unstoppable!

Unfortunately, it’s also unscalable. My granny’s smart fridge produces more data than any current public blockchain can reasonably handle. Transaction capacities are low, and writing any more than a miniscule amount of data into the chain incurs prohibitive gas costs.

If only there was a real-time, low-latency, scalable, peer-to-peer data transport layer sitting on top of the blockchain to handle the heavy lifting! Ðapps as well as centralized apps could then be served with unstoppable data by the network. The most important events — or aggregates calculated from the raw data, as in this post — could still be delivered to the underlying blockchain. The chain would also be perfect for supporting the operation of the data network by facilitating incentivization and permissioning.

My granny lives on a small pension, but she’s pretty clever as far as money matters go. What if the data network were powered by a usage token called DATAcoin, and we added native support for data monetization? Now my granny could offer her smartwatch heart rate data feed for sale, and if there’s enough grannies out there, the pharmaceuticals would certainly be interested. Or I could subscribe to her heart rate, just to know she’s ok. She’s made a smart contract to incentivize her doctor to keep her alive for another 100 years, but still!

For months, we’ve been developing the idea of this network and its related ecosystems. We have converged to the concept of DATAcoin and a technology stack that could become the global real-time data backbone for decentralized apps. We humbly present a whitepaper draft on “Unstoppable Data for Unstoppable Apps”.

As always, we would really like your feedback. Please come and talk to us on Slack if you have ideas, comments, or suggestions. You can also reach us on Twitter.

Streamr brings U.S. stocks to Ethereum blockchain

What are the future implications of massively distributed, decentralised databases enabled by blockchains? Will it some day be possible to have a significant share of new data generated in the world – data from IoT, social media, stock exchanges and so on – pass through a blockchain-powered system for superior persistence, security, and peer-to-peer distribution? Data could be easily available, transparent, proven, and not controlled by giant corporations.

While present-day blockchains aren’t nearly scalable enough to ingest big data, we at Streamr are working on ways to make the above vision happen sooner rather than later. Stay tuned for more information about that in the near future. Meanwhile, the concept of real-time data flowing into a decentralised system can already be easily demonstrated with the existing Streamr platform and its built-in Ethereum integration.

For this week’s demo, we decided to inject the U.S. stock market into the blockchain. Well, not the whole market just yet, we’ll start with the stocks in the S&P 500 index. For now, direct input of data into the blockchain, combined with current blockchain scalability means that the idea can only be explored on a miniature scale, to avoid spamming the Ethereum blockchain and paying excessive gas costs.

In the demo, you can see a process that ingests streaming trade data from NASDAQ, aggregates the data into 15 minute OHLC (open, high, low, close) “bars” by stock symbol, and submits those bars to a smart contract deployed on the Rinkeby testnet. Data for all the stocks are reported in a single transaction, in order to avoid spamming hundreds of transactions every 15 minutes.

What effectively happens is that everyone running a full Ethereum node will automatically have fast and verifiable local access to the published data points. A smart contract holds the data, and it can be queried by anyone to retrieve the events posted earlier (to display a chart, for example).

Here’s the live canvas (full screen, or open in editor):

The smart contract is at 0xcc85a5f3ecc0ae62023e35faeb9c72fea25d75d6, and its source code can be seen on the canvas inside the StockTicker contract. Here’s an example bit of web3 code for querying the Apple stock price history and watching for new events:

// Symbol to watch
var symbol = 'AAPL'

// Contract address and abi
var address = '0xcc85a5f3ecc0ae62023e35faeb9c72fea25d75d6'
var abi = [{"constant":false,"inputs":[{"name":"symbol","type":"bytes32[]"},{"name":"open","type":"int256[]"},{"name":"high","type":"int256[]"},{"name":"low","type":"int256[]"},{"name":"close","type":"int256[]"},{"name":"time","type":"uint256"}],"name":"setPrices","outputs":[],"payable":false,"type":"function"},{"inputs":[],"payable":false,"type":"constructor"},{"anonymous":false,"inputs":[{"indexed":true,"name":"symbolIdx","type":"bytes32"},{"indexed":false,"name":"symbol","type":"string"},{"indexed":false,"name":"time","type":"uint256"},{"indexed":false,"name":"open","type":"int256"},{"indexed":false,"name":"high","type":"int256"},{"indexed":false,"name":"low","type":"int256"},{"indexed":false,"name":"close","type":"int256"}],"name":"Price","type":"event"}];

	 	// Convert symbol to hex bytes32
		symbolIdx: web3.fromUtf8(symbol).padEnd(66, '0')
		// Fetch complete history of events
		fromBlock: 0,
		toBlock: 'latest'
	function(error, event) {
			// Convert integer BigNumbers back to double-precision primitives
			symbol: event.args.symbol,
			time: new Date(event.args.time.toNumber()),
			open: event.args.open.toNumber() / 10000,
			high: event.args.high.toNumber() / 10000,
			low: event.args.low.toNumber() / 10000,
			close: event.args.low.toNumber() / 10000

Note that we originally used a string field to index the events, but ran into this bug when trying to query the events in web3. We worked around the issue by using a bytes32 representation of the symbol string for the purposes of this demo.

While this example demonstrates the idea of decentralised streaming data delivery in the future, we’re currently working with Oraclize to enable smart contracts to obtain data from Streamr securely on a per-request basis, and on many different blockchains. This will be a topic for an upcoming blog post.

Overall, we begin to see hints of how a decentralized system for storing and transmitting data might work in practice. And of course, there is much more that can be done: we have not even scratched the surface of Ethereum’s other protocols, nor have we explored Polkadot Parachains or other multi-chain scaling schemes.

Decentralised data feeds are a new and exciting frontier, and we could not be more thrilled that the state of working technology already allows us to get this far! As always, we invite anyone who is as data-crazy as we are, to join us on our Slack, and on Twitter.