Golem + Streamr = ♥

A fascinating blog post written last year by Golem Factory CTO Piotr Janiuk describes an imaginary application called Hyperbrick, which allows users to combine microservices into workflows. By happy coincidence, the Streamr Engine and Streamr Editor, as presented in our DATAcoin white paper draft, is remarkably close to Golem’s example of an ideal application in their ecosystem. This blog post is the direct result of having our minds blown while considering the overall implications of this space and its emerging tech.

A quick reminder:

  • In Streamr, “workflows” are called Canvases; they consist of modules that operate on incoming data.
  • The Streamr Engine that powers it all can easily be made to run inside a Docker container, which is natively supported by Golem.

Right now, the Streamr Engine (i.e. the component which processes our data feeds) runs on more or less any computer, most often in a cloud environment. However, running in the cloud means that a centralized commercial entity like Amazon or Microsoft controls the server hardware, network, storage, and price of computation. They are also able to see who is doing what quite plainly.

In our vision, the data transport from sources to Streamr Engine nodes running on Golem would happen via the Streamr Network. This is a peer-to-peer network which ensures the data gets delivered and that it can be trusted. With Golem, the Streamr Engine, and the Streamr Network working together, computation in real-time analytics use cases could be completely decentralized, and the pricing would be determined transparently by the market. Security, privacy, and quite possibly pricing, would be vastly improved.

The current prototypes of Golem support tasks of finite computation. This works well for large, batch-oriented computation such as 3D rendering. However, for more fine-grained computation such as hosting microservices or stateful apps, support for long-running tasks/processes is required. The same goes for more obvious matters such as access to networking.

Happily, these features are indeed on the Golem roadmap. When available, these features will allow Golem to host processes such as the Streamr Engine, potentially processing hundreds of thousands of incoming requests or data points per second on a single node, and many millions on a cluster of Golem-Streamr-nodes.

As best we can tell, Streamr on Golem could bring about incredible use cases in the domains of algorithmic trading, asset tracking, anomaly detection, predictive analytics, and much more. Needless to say, we look forward to begin experimenting with Golem, and to the world of decentralized distributed compute in general. Three cheers to the #truecloud!

Questions and comments about this post and Streamr in general are appreciated! Join us on Slack, and of course feel free to follow us on Twitter as well.

Unstoppable Data for Unstoppable Apps: A DATAcoin Whitepaper Draft

There’s a revolution brewing in computing infrastructure. The winning architecture will be decentralized apps (Ðapps): Front-ends served to the browser from decentralized storage, and back-ends implemented as smart contracts living on the blockchain.

This stack has many desirable properties: It makes applications naturally resistant to distributed denial-of-service attacks, it adds transparency and security by keeping an untamperable transaction log, it has zero reliance on commercial services, and it enables easy peer-to-peer transfers of value in every application.

But what serves this new breed of applications with data? How to make an instant messaging Ðapp capable of delivering a million messages per second? What about a stock trading Ðapp? Vehicle fleet tracking? Factory automation? Air traffic control?

At first sight, blockchains as decentralized databases seem perfect for data distribution. (We’ve even demonstrated the promise of this pattern in a previous blog post.) It’s a powerful concept: write a piece of data to a smart contract, and all nodes in the network get a copy. Front-ends can watch the contract data on a local node, and update accordingly. Unstoppable! Ok, data delivery is a bit slow due to mining delay, but hey, still unstoppable!

Unfortunately, it’s also unscalable. My granny’s smart fridge produces more data than any current public blockchain can reasonably handle. Transaction capacities are low, and writing any more than a miniscule amount of data into the chain incurs prohibitive gas costs.

If only there was a real-time, low-latency, scalable, peer-to-peer data transport layer sitting on top of the blockchain to handle the heavy lifting! Ðapps as well as centralized apps could then be served with unstoppable data by the network. The most important events — or aggregates calculated from the raw data, as in this post — could still be delivered to the underlying blockchain. The chain would also be perfect for supporting the operation of the data network by facilitating incentivization and permissioning.

My granny lives on a small pension, but she’s pretty clever as far as money matters go. What if the data network were powered by a usage token called DATAcoin, and we added native support for data monetization? Now my granny could offer her smartwatch heart rate data feed for sale, and if there’s enough grannies out there, the pharmaceuticals would certainly be interested. Or I could subscribe to her heart rate, just to know she’s ok. She’s made a smart contract to incentivize her doctor to keep her alive for another 100 years, but still!

For months, we’ve been developing the idea of this network and its related ecosystems. We have converged to the concept of DATAcoin and a technology stack that could become the global real-time data backbone for decentralized apps. We humbly present a whitepaper draft on “Unstoppable Data for Unstoppable Apps”.

As always, we would really like your feedback. Please come and talk to us on Slack if you have ideas, comments, or suggestions. You can also reach us on Twitter.