Summary

Decentralized cloud services have long been regarded as one of the most crucial real-world applications of blockchain technology. The integration of Web3 and cloud services presents an exciting narrative, further enhanced by the incorporation of AI. However, achieving instant consensus for computations in a purely asynchronous environment like cloud services poses significant technical challenges for DFINITY IC. Arweave AO breaks free from the limitations of traditional blockchains by not directly handling computations or consensus. Instead, it employs an economic model and lazy verification to guarantee the accuracy of computation results, offering a novel approach to decentralized computing.

Basic Introduction

The cloud is an IT environment that abstracts, aggregates, and shares scalable resources across an entire network. These resources include computing power, storage, and network bandwidth. The main technical features include virtualization technology, distributed resource management, and parallel execution technology. Compared to traditional IT environments based on physical machines, the cloud is more cost-effective, scalable, and easier to manage.

Cloud services, cloud computing, and cloud storage are sub-concepts derived from the broader concept of the cloud. Cloud computing refers to providing computing services through the cloud, cloud storage refers to providing storage services through the cloud, and cloud services encompass various IT services provided through the cloud, including both computing and storage services. It is worth noting that cloud computing and cloud services are often considered synonymous in many contexts, but this article distinguishes between the two.

Currently, the cloud services market is dominated by several major players. Globally, Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure account for over 60% of the market. Domestically, Alibaba Cloud, Tencent Cloud, and Huawei Cloud hold about 60% of the market in China, demonstrating a significant Matthew effect.

Decentralized cloud services, on the other hand, disrupt the current cloud service model through blockchain technology. They rely on market forces to determine the value and allocation of computing power, storage space, and bandwidth. This not only achieves more efficient and direct resource allocation but also breaks the monopoly of cloud service giants. Since all underlying resources are provided by users, if the economic structure of a decentralized cloud service network becomes unsuitable, users can choose to leave the network or join other decentralized cloud service networks. This limits the ability of decentralized cloud service providers to engage in harmful practices.

The Path of Decentralized Cloud Service Development

Decentralized computation is not mentioned here because the author believes it to be a false proposition. Firstly, smart contracts in blockchain are inherently responsible for decentralized computation. Secondly, for complex computation tasks, decentralization implies a waste of computational power. One should not pursue decentralization for its own sake; verifiable distributed computation might be a better choice, which the author will explain later.

Compared to instantaneous computation, persistent storage is clearly an excellent application scenario for decentralization. It allows data to be stored and distributed in a decentralized network, making it immutable and resistant to censorship. In fact, decentralized storage has already become one of the most successful applications of blockchain in non-financial scenarios.

In 2014, Protocol Labs proposed IPFS, the InterPlanetary File System, opening the door to decentralized storage on a cosmic scale. IPFS implemented content-based file addressing, preventing the same file from being stored multiple times and significantly saving storage space. It also uses a P2P network, allowing data to be concurrently downloaded from multiple nodes, greatly reducing bandwidth costs. This made IPFS the underlying protocol for many decentralized storage projects, with Filecoin being the most representative.

In July 2017, Protocol Labs, the developer of IPFS, announced the establishment of the Filecoin project. Filecoin is an incentive application layer and blockchain public chain system on top of the IPFS network. It adopts a hybrid consensus mechanism, primarily based on Expected Consensus (EC), and supplemented by Proof of Spacetime (PoSt) and Proof of Replication (PoRep). Filecoin’s vision is to incentivize a vast number of nodes worldwide to provide storage and retrieval services, promoting the widespread use of the IPFS file storage and transmission protocol.

Also in July 2017, Stroj Labs founded Storj, almost concurrently with Filecoin. Storj positions itself as enterprise-level storage service, leaning more towards commercialization, directly competing with AWS’s S3 service. However, its architecture is pseudo-decentralized. Although it began issuing tokens in 2018, the management of its metadata, block production, rewards, and penalties are all handled by satellite nodes, which are currently maintained only by the project team. Although the future plan includes decentralizing the satellite nodes, the problem seems challenging to solve at the moment. As a result, its current state still resembles traditional storage cloaked in the guise of blockchain.

In June 2018, the Arweave mainnet launched. Arweave is not an incentive layer based on IPFS, but instead integrates data storage and incentives, with a focus on enabling permanent data storage and access. Arweave doesn’t require miners to preserve all block records, but encourages miners to save as many blocks as possible, especially those with less redundancy, through Simplified Proofs of Random Access (SPoRA). Miners who do this have a higher probability of receiving mining rewards. Essentially, Arweave ensures data is copied as much as possible to increase the reliability of data storage through a game theory approach.

In May 2021, the Internet Computer (IC), developed by the DFINITY Foundation, went live. IC is the first complete decentralized cloud service, acclaimed as the third-generation blockchain architecture. Through innovative blockchain technology, IC can provide unlimited scalability and rapid transaction processing, and it can handle HTTP requests to support large-scale decentralized applications, ranging from social media platforms, open-source project hosting services to various enterprise-level applications, and even large AI models. Theoretically, any existing internet service can be rebuilt on IC as a decentralized version.

In November 2022, the emergence of ChatGPT marked a milestone in strong artificial intelligence and triggered a new round of artificial intelligence craze. AI emerging companies have sprung up like mushrooms after a rain. With the iterative upgrade of generative large models, the demand and cost of computing power have increased exponentially. As a result, the track of decentralized computing power has been born, which is committed to using shared computing power to reduce the training of large models for AI emerging enterprises. costs, weakening the unfair competition caused by cloud service giants taking advantage of their dominant position.

Well-known projects in the decentralized computational power race include io.net, Render, Akash, Gensyn, etc. Although they can also be categorized under decentralized cloud services, their core protocol lies in building a computational power market and incentivizing computational power providers. This more closely aligns with the definition of DePIN (Decentralized Physical Infrastructure Networks).

In February 2024, Arweave officially launched the super parallel computer AO*, becoming the second complete decentralized cloud service after DFINITY’s IC. The journey of decentralized cloud service development continues.

Arweave AO: Separation of Computation and Consensus

Both DFINITY IC and Arweave AO are complete decentralized cloud services, with significant similarities. Firstly, functionally, they both support the decentralization of large-scale internet services and the introduction of large AI models to run in blockchain smart contracts. Secondly, their architectures are both designed based on the Actor model. An Actor is a basic unit of a concurrent computing model in computer science. Adopting the Actor model is suitable for building high-concurrency, distributed, fault-tolerant systems, which is also the origin of Arweave AO’s name.

The main difference between the two is the data storage layer, execution layer and consensus layer.

• Data storage layer: The smart contract on DFINITY IC is called Canister. Canister has its own dedicated container (similar to Docker). The data of each Canister is encapsulated in its own independent container, and the data cannot be seen by the outside world. Details: Internal data can only be accessed through the interface provided by Canister; Arweave AO is based on Arweave, and the data is stored in Arweave and is public to the outside world;
• Execution layer: The virtual machine of DFINITY IC is WASM. The Canister code will be compiled into a WASM module to be deployed and run on the IC. It only supports standards such as WebAssembly system interface; Arweave AO is more flexible, as long as it follows the AO protocol standard, Any virtual machine can be used, including EVM, WASM, Move VM, etc. The current AOS contract development language officially built by Arweave is Lua;
• Consensus layer: DFINITY IC’s subnet adopts a variant of BFT (Byzantine Fault Tolerance, Byzantine Fault Tolerance) consensus, and subnets are verified through Chain-Key technology; Arweave AO is based on SCP (Storage-based Consensus Paradigm) , is a storage-based consensus paradigm that emphasizes that consensus occurs at the storage layer and uses Arweave for immutable storage to ensure security and verifiability.

From the above comparison, it is apparent that DFINITY IC still adheres to the standard blockchain paradigm, whereas Arweave AO seems less like a typical blockchain, as it does not even have a consensus mechanism. So, how does it ensure that different nodes agree on the computation results?

The answer is that Arweave AO cannot guarantee consistent computation results, its computation results do not generate any proofs (such as a Merkle tree), but Arweave’s immutable storage is verifiable. The separation of computation and consensus is the most ingenious part of Arweave AO’s design.

Arweave stores the holographic data of AO and every thread on AO. Anyone can recover AO and any thread on AO through this holographic data. This is actually the core idea of SCP, that is, as long as the storage is immutable, all transactions on it are traceable, so no matter where the application is computed, the same result will be obtained.

Once the verifiability issue is resolved, AO’s economic model can be used to encourage everyone to provide correct computation results. This is similar to the margin mechanism of Chainlink nodes in DON. Nodes need to pledge tokens first when joining the AO network. When nodes provide correct computation results, they are incentivized, and when they provide incorrect results, they are penalized.

Conclusion

Decentralized cloud services have long been regarded as one of the most crucial real-world applications of blockchain technology. The combination of Web3 and cloud services not only presents an exciting narrative direction but also seamlessly integrates with AI, further enhancing its potential.

Decentralized cloud services have evolved over a decade, starting with IPFS. Technically speaking, the underlying storage and computation protocols have been perfected. From a market perspective, there is ample demand for resource-sharing networks, primarily driven by computational power. However, despite these developments, there is a noticeable lack of comprehensive solutions, and those that do exist are far from perfect.

Before its launch, DFINITY’s IC was considered the AWS of Web3, crowned as the third-generation blockchain architecture. Unfortunately, it encountered difficulties upon launch and has only recently started to recover. Aside from market factors, achieving instant consensus for computations in a purely asynchronous environment, such as cloud services, requires high hardware resources. This requirement impacts the degree of decentralization, undermining the significance of the blockchain’s consensus mechanism.

Arweave AO breaks free from the limitations of traditional blockchains by not directly handling computations or consensus. Instead, it employs an economic model and lazy verification to guarantee the accuracy of computation results, offering a novel approach to decentralized computing. However, the verifiable computation implemented based on SCP is supervised and verified off-chain, theoretically posing the risk of insufficient off-chain supervision. While its applicability in large-scale computation scenarios still needs market validation, the future prospects are promising.

statement:

1. This article is reprinted from [permadao], the original title is “The Evolutionary History of Decentralized Cloud Services: From DFINITY IC to Arweave AO”, copyright Attribution to original author[Pignard ], if you have any objection to the reprint, please contact Gate Learn Team, the team will handle it as soon as possible according to relevant procedures.

2. Disclaimer: The views and opinions expressed in this article represent only the author’s personal views and do not constitute any investment advice.

3. Other language versions of the article are translated by the Gate Learn team, not mentioned in Gate.io, the translated article may not be reproduced, distributed or plagiarized.