In-depth Analysis of Dark Pools: Privacy Research on Transparent Blockchains

This article provides a detailed introduction to various types of dark pools, including their working principles, privacy architecture, order matching, liquidity sources, MEV, and anti-censorship measures. By incorporating privacy-enhancing technologies such as zero-knowledge proofs, dark pools aim to provide a high level of security while protecting transaction privacy. Additionally, the article discusses issues related to regulatory compliance and transaction visibility, as well as explores different business models within dark pools.

Before delving into the inner workings of blockchain-based dark pools, let’s provide some context on how dark pools originated, why they still operate, and how much they change the game.

The Dark Pool of the 1960s

In 1969, when computers were still large enough to occupy a room or two and traders communicated orders by shouting on the trading floor, institutional investors sought a more efficient method to buy and sell stocks without causing a chain reaction in the market. Jerome Pustilnik revolutionized Wall Street by establishing Instinet, the pioneer of electronic trading. Through Instinet, institutional investors could submit orders, which would then be matched and executed. However, for a platform like Instinet to succeed, it required significant trading volume from both buyers and sellers. What made Instinet particularly appealing to institutional investors was the promise of confidentiality. By keeping their identities and orders hidden from other participants and the broader market, Instinet allowed the “big guys” to operate anonymously. This not only prevented their trades from impacting the market, but also reduced the risk of being front-run by other traders.

As of 2022, more than 60 dark pools have been registered with the SEC. Some are exchange-operated, such as the NYSE or NASDAQ, while others are broker-dealer operated, like MS Pool by Morgan Stanley or SigmaX by Goldman Sachs. Additionally, there are independently operated dark pools, like Liquidnet or MatchNow.

The problem with operator-run dark pools is that operators may have more incentive to engage in misconduct rather than operate in a compliant manner. It can be summarized in a simple equation: Profit from Corruption > Cost of Corruption. Operators can potentially earn more profits than the penalties they would have to pay. For instance, in 2018, the SEC fined Citi Group $12 million for misleading investors about their dark pool operations, as they leaked confidential order information to High-Frequency Traders. These traders executed orders worth over $9 billion against Citi’s customers, resulting in significant profits for them.

Traditional dark pool users are at risk of being manipulated by the operators, which is a significant drawback of the broken trust model. Dark pool operators have paid over $340 million in penalties since 2011 to settle allegations, although this amount is relatively small compared to the profits they may have made. Blockchain-based dark pools aim to eliminate the need to trust potentially malicious operators. However, there are still problems.

Mixers, Pools, and Everything In Between

Blockchains were originally designed to be completely transparent. While this promotes accountability, it is a double-edged sword. If you received a salary payment on-chain, anyone with your wallet address could see the amount you were paid in the past and the amount you will be paid in the future.

DEXs (Decentralized Exchanges) and wallets can be tracked by wallet tracking and copy trading platforms. These platforms go a step further by enabling anonymous users to copy successful traders, which can complicate their trading strategies. Additionally, placing large orders on DEXs can expose them to front-running by individuals monitoring public mempools.

Before discussing dark pools, it’s important to differentiate between mixers and dark pools. Mixers are a type of dark pools. Mixers, such as Tornado Cash, mix tokens to eliminate connections between wallets and assets, making it challenging to trace the origin of funds. In contrast, dark pools not only sever connections between wallets and tokens but also facilitate trading among users without disclosing any information about the involved parties.

Now that we have examined the origins and motivations of traditional dark pools and differentiated a mixer from a dark pool, let’s explore blockchain-based dark pools. These dark pools incorporate privacy into their fundamental architecture through Account/Balance models and PETs (Privacy Enhancing Technologies) like Zero Knowledge, MPC, and FHE.

As of the time of writing this ariticle, projects like Portal Gate, Renegade, and Tristero are still under development. On the other hand, privacy protocols like Panther and Railgun are already operational, facilitating private decentralized finance (DeFi) transactions. These transactions not only include exchanges but also operations such as staking, providing liquidity, and borrowing.

Privacy Architecture

Balance Models

Blockchains are state machines that consist of accounts and transactions. As accounts engage in transactions, each state transition adds a new set of updates to the chain. The balances of the accounts are also adjusted accordingly. Balance models are various approaches used by blockchains to track and manage user account balances. For example, Ethereum, Solana, and Polygon utilize the Account model, while Bitcoin, Zcash, and Monero employ the UTXO model. These models differ in their methods of managing and representing the state of a blockchain.

The Account model, similar to a bank account, represents the current state as a collection of addresses and their respective balances. In this model, when you make a payment, you send the exact amount without any change left over, and no new balances are created. However, the UTXO (Unspent Transaction Output) model operates differently. When you make a payment in this model, your entire balance is transferred, effectively “burning” your existing balance. The remaining value you own is then added to a new balance. The UTXO model displays the current state as a graph that includes all the spent and unspent outputs of transactions.

To illustrate, consider the following scenario:

Alice’s wallet contains two UTXOs, one with 6 coins and another with 5 coins, while Bob’s wallet only has one UTXO containing 5 coins. When Alice sends 5 coins to Bob, she combines her two UTXOs into a single transaction, making it difficult for observers to determine the source of the 5 coins she sent to Bob and the destination of the remaining balance in her UTXOs. This privacy protection mechanism effectively conceals the flow of funds and ensures the privacy of the transaction participants.

A newly generated UTXO does not have a known owner and requires advanced chain analysis to be linked to a single user. However, with the account model, there is fungibility, which encourages address reuse and makes it easy to track the transaction history for a single user.

With the UTXO model, it is also difficult to double-spend. Each UTXO has a unique nullifier derived from itself. When a transaction is verified, meaning when the UTXO is spent, its nullifier is also spent. This prevents the transaction from being spent again.

Different versions of the UTXO model have been utilized in various Layer 1 (L1) and Layer 2 (L2) solutions that prioritize privacy. Aleo employs the Record model, which is a modified version of the UTXO model. Aztec and Polygon Miden, on the other hand, utilize a hybrid approach, combining elements of both the UTXO and Account models.

The UTXO model used by Zcash heavily inspired the UTXO model used in dark pools. Various protocols have modified it to support multiple assets and accounts. Protocols such as Portal Gate, Penumbra, Railgun, and Renegade utilize a note-based mechanism where each UTXO is represented as a note. These notes, similar to UTXOs, contain information about the asset, order, and value, but are only visible to the owner/user. The notes are stored within Merkle trees that maintain the state of the chain, known as the State Commitment Tree. These notes can be created through internal transfers or settlements within the dark pool, or through a fee payment made to a relayer. Renegade employs enshrined MPC to match UTXOs (orders) and update private state. It is also possible to store multiple notes within a single note.

With Panther Protocol, each asset or account is represented as a zAsset or a zNFT. The UTXO model operates through a Merkle tree called the “Bus”, which processes UTXOs in batches. Each batch can contain up to 64 UTXOs, optimizing cost efficiency for users. The processing of UTXOs is handled by “Bus operators”, who are incentivized by users with rewards denominated in $ZKP. For executing individual UTXOs, Panther uses a “Taxi” Merkle tree. Additionally, to support multi-chain UTXOs, there is a “Ferry” Merkle Tree.

Proof Systems

This is where one of the primary PETs (Privacy Enhancing Technologies) comes into play. Zero Knowledge Proofs (ZKPs) are used to prove the possession of correct information without revealing it. In the context of dark pools, ZKPs can be issued to prove that a user has sufficient funds and permissions to place orders, have them match and settle, all while keeping these details hidden from dark pool operators, fellow traders, and external third parties.

Proof systems are cryptographic constructs designed to generate proofs. They come in various types, each producing proofs of different sizes, requiring varying timeframes, and consuming different computational resources for both generation and verification. In this document, we will explore two prominent proof systems: Groth16 and UltraPlonk. These proof systems have found applications in the dark pools we have discussed.

Panther, Penumbra, and Railgun use the Groth16 proof system to generate their proofs. Groth16 is preferred because it produces proofs of consistent size, which can impact transaction size, gas costs, and throughput. In contrast, Portal Gate and Renegade employ the UltraPlonk proof system.

Both Groth16 and UltraPlonK require a trusted set-up, where a group of trusted parties collaborate to establish a common set of parameters. These parameters are used to verify proofs in a way that ensures the verifier does not need to trust the prover, and vice versa. It is important to note that a trusted setup implies trust assumptions about the involved parties. Secure Multi-Party Computation can be used to prevent any single party from gaining access to the underlying setup structure. Groth16 relies on inputs from multiple parties for its trusted setup, while UltraPlonk only requires the involvement of a single party.

Internal Operations of Dark Pools

In the following sections, we will explore the inner operations of dark pools. We will discuss how they preserve privacy during trades, source liquidity, prevent MEV extraction, exhibit censorship resistance, and ensure compliance.

Wallet Settings

In most cases, you will need to create new, specialized wallets to interact with dark pools. Railgun has a third-party wallet provider called Railway Wallet that enables you to create an encrypted EOA. This EOA allows you to hold a balance of shielded tokens/NFTs and privately call any public smart contract you desire through a Relay Adapt mechanism.

With Renegade, a wallet is equivalent to a UTXO (Unspent Transaction Output) that possesses private keys and can sign transactions. To ensure the integrity of the funds, you must generate a new UTXO and provide a commitment using a ZK (Zero-Knowledge) proof. This proof verifies that the new UTXO is indeed new and that the funds it contains have not been previously spent. Please note that most operations such as depositing, placing orders, and trading will invalidate the existing wallet, necessitating the creation of a new one to prevent any instances of double spending. The user experience (UX) remains consistent with other Ethereum wallets.

To use Penumbra, you need to create a new wallet, just like you would when starting with any other Cosmos chain. Once the wallet is created, it can be funded through an IBC transfer.

For Panther and Portal Gate, a KYC process is in place. While an Ethereum wallet can be used, the user must verify their passport through the KYC process for individuals in order to interact with the protocol. Business entities need to go through KYB and share their business details. The verification is conducted off-chain by a registered compliance provider. These credentials expire periodically and need to be renewed through the verification process.

Order Discovery and Matching

When it comes to order discovery and matching, general decentralized exchanges (DEXs) like Uniswap and Curve take an Automated Market Maker (AMM) approach, where orders are matched against a liquidity pool. However, this method is not the best choice for privacy. When orders are matched against a public liquidity pool, the order details become publicly visible on the blockchain. This is not ideal for traders dealing in large volumes, as it may reveal their intentions to other market participants who could take advantage of that information. Some dark pools, which may still match orders against liquidity pools or use other external public contracts for order matching, utilize relayers as a privacy measure. These relayers act as a Chinese wall between traders who require privacy and the external liquidity sources. In the next section, we will explore how dark pools source liquidity while maintaining privacy.

Currently, traders have already deposited funds into their wallets in the dark pools. Now, let’s discuss what happens when an order is placed. Some dark pools use a Peer-to-Peer approach for order matching, while others use a Peer-to-Pool approach with PETs such as Zero-Knowledge or Multi-party Computation. In the Peer-to-Peer approach, orders are crossed between traders to achieve better execution without slippage. This is because if orders were matched with a liquidity pool, the price of the pool could change between the time the order is placed and when it is executed. When orders are crossed between traders, they are executed at the agreed-upon price between them.

With Renegade, a Peer-to-Peer approach is taken. When a trader submits an order, they also choose a relayer who can match orders for a fee. The trader generates a ZK Proof called “VALID COMMITMENT” to prove ownership of their wallet and its orders. A “handshake” is then sent out to the network to match with handshakes of other traders. A handshake consists of the VALID COMMITMENT ZK Proof, hashed order details, a nullifier, and key pairs. Once a trader’s handshake matches with another trader’s handshake, Multi-party Computation is initiated. Renegade uses collaborative ZK-SNARKs to prove that the trader’s orders truly match with the counterparty’s orders. This ensures that the order details remain hidden from other parties during the order matching process. Once orders are matched and the required proofs are provided, a new wallet is created. The swapped tokens are deposited into this new wallet after verifying that it is indeed new to prevent double-spending. The user experience remains the same as using any Ethereum wallet.

With Portal Gate, when a trader submits an order, an API node picks it up, encrypts it, generates a ZK proof, and relays the order to the Book. The Book is a set of off-chain nodes that collect orders and run an order-matching algorithm inside a Fully-Homomorphically Encrypted (FHE) environment. This means that the encrypted orders do not need to be decrypted to be matched. Once the orders are matched, the sum result of all the matched orders is shown, while no details of individual orders being matched are revealed. This again helps preserve privacy.

Railgun is a system that enables traders to place swap orders. When a trader places swap orders through Railgun, a smart contract called Adapt Module performs multiple actions. First, it takes the trader’s specified private balance from their wallet and unshields it. The Adapt Module also verifies that the order (UTXO) has not already been nullified or spent. Next, the 0x API swaps the assets across aggregated DEX liquidity, searching for the best rates. After the swap, the Adapt Module shields the swapped assets back into the private balance, ensuring that their activity or addresses are not revealed. This workflow can also be applied to other types of DeFi transactions.

Panther Protocol operates similarly to other protocols by utilizing a module called Zswap. This module connects Panther’s MASP (Multi-Asset Shielded Pool) with other DeFi protocols. The MASP is not exactly a “pool,” but rather a collection of “append-only” Merkle trees. Each leaf in the trees represents a commitment to a UTXO, which represents an IOU of an asset deposited into the MASP. When a trader creates an order, Zswap gathers quotes from other DeFi protocols, allowing the trader to choose the best option. Once an order is placed, Zswap generates an encrypted time-based escrow contract. This contract enables private swaps between users without disclosing any details. After the assets are swapped, the user receives an IOU of the token in the form of a ZAsset (ZNFT) from Panther’s MASP.

Penumbra’s swap module, also known as Zswap, operates in a distinct manner. When a trader creates an order, the asset in their private balance is burned, and the encrypted order values are sent to validators. A fee is paid to process the order. To record the event, the user receives a “swap” NFT. This NFT is uniquely identified by various parameters, including the traded assets, fee, input amounts, and cryptographic keys. Validators consolidate order inputs from multiple traders into a single batch, which is then executed against concentrated liquidity positions. Upon execution, the outputs are distributed to the traders proportionally based on their inputs. While individual data remains private, aggregate data is exposed when validators combine orders for execution against concentrated liquidity positions.

Finding Liquidity

Dark pools such as Renegade and Portal Gate facilitate anonymous trading by directly matching buy and sell orders between users. However, there is no guarantee that there will always be available liquidity on the other side of an order. On the other hand, general decentralized exchanges (DEXs) offer almost guaranteed liquidity for orders, but at the expense of privacy and price impact. Privacy is compromised because anyone can observe your buying or selling activity. Price impact is a concern because guaranteed liquidity comes with a premium, which you have to pay along with your order. In contrast, dark pools keep order information hidden both before and after the trade.

Renegade follows this design philosophy that involves matching order flow directly without the involvement of intermediary liquidity providers. In situations where there is a lack of liquidity, a mechanism called “Indication of Interests” allows traders to disclose certain details of their order to relayers that match orders. These details, such as price, size, asset, and order type, are securely proven through zero knowledge. If traders choose to reveal these details, other traders can view these orders on the order book, potentially resulting in faster execution of these orders.

Panther Protocol is more than just a dark pool. It offers a range of other DeFi activities that can be performed privately. To get started, a user deposits assets into a Multi-Asset Shielded Pool, where the deposited assets are represented as ZkAssets (ZkNFT). These ZkAssets function as IOU tokens, similar to stETH obtained when staking Ethereum. DeFi Adaptors, known as plug-ins, establish private connections between the Multi-Asset Shielded Pool (MASP) and existing DeFi protocols. Adaptors like Zswap and Ztrade enable connectivity between the MASP and DEXs such as Uniswap, Quickswap, Curve, and more.

Similar to Panther or Railgun, Penumbra serves as a platform for various DeFi activities. When users initiate an LP (Liquidity Provider) position within Penumbra, they essentially create their own customized AMM (Automated Market Maker). As a result, there are thousands of concentrated liquidity positions, each with its own user-defined fees. Although this may lead to fragmented liquidity, Zswap / DEX Engine addresses this issue by considering and synthesizing all these individual liquidity positions into a unified AMM. Consequently, incoming trades can be routed across the entire liquidity graph.

Portal Gate is an anonymous AMM-DEX where Automaton is used to provide liquidity as a backup to the Book. The relayer operates an order book to match users’ trades. If the book is unable to facilitate trades due to insufficient liquidity, the orders are directed to Automaton. Liquidity in Automaton will be bootstrapped in the same way as any other AMM.

Railgun’s Railway DEX utilizes the 0xAPI developed by 0x. This implies that orders from the Railway DEX are directed towards finding the best price for execution through the 0xAPI DEX aggregator. Once a trading route is determined, the wallet generates proof to enable the use of funds in the trader’s balance for swapping and protecting the incoming tokens from 0xAPI to the trader’s balance.

Business Models

Tradfi dark pools are like exclusive clubs where individuals with large balances trade. These individuals pay membership fees to the dark pool, allowing them to place orders anonymously and receive relevant counter-order flow. Trading fees in dark pools are generally lower than traditional financial exchanges. Additionally, tradfi dark pools may act as brokers, purchasing securities from sellers at a lower price and selling them to buyers at a higher price.

In the crypto space, dark pools function differently. They transition from an “operator” focused model to a “facilitator” role. Their revenue generation relies on two main components: protocol fees on successfully matched orders and a portion of the relayer fees paid by traders to handle transactions. Some of the relayer fees may also go to the relay operator. Traders also have the option to run their own relayers for enhanced privacy and to avoid paying relayer fees.

Decentralized dark pools in crypto may face a “cold start” problem. Unlike DEXs that trade against a liquidity pool, most dark pools have users trade against each other by matching buy and sell orders. It is crucial for such a system to have traders on both sides of the market placing orders. It is also important to note that dark pools in tradfi were hosted by multi-billion dollar banks like Goldman Sachs and exchanges like NYSE. This way, supplying order flow to their users would have been relatively easier. There are also standalone dark pools like Liquidnet who had estimated that they would require at least 100 buy-side firms to be live on their platform since day to reach critical mass but ended up launching with 38 firms. Liquidnet is now among the top 3 largest dark pools in the world.

Centralized exchanges, on average, have 10x more volume than decentralized exchanges due to their larger user base, attracted by better user experience and regulatory compliance. This increased user base leads to a greater supply to meet the demand. However, liquidity for over-the-counter (OTC) deals is generally more difficult to find.

In the crypto industry, most OTC deals are facilitated through OTC desks, escrow smart contracts, and even Telegram groups in some cases. Unfortunately, scams are common. It is relatively easy for scammers to impersonate others on Telegram, manipulate messages, and alter deal details. Dark pools offer a solution to this issue. While they may not guarantee immediate settlement for illiquid tokens or non-fungible tokens (NFTs), they provide a higher level of security and privacy. Additionally, customers of OTC desks who use dark pools do not have to pay a broker’s fee.

Volume is crucial for the effective functioning of dark pools. It aids in the efficient matching of orders and the preservation of trader anonymity. The greater the number of anonymized funds in the pool or the more orders in the order book, the more challenging it becomes to connect depositors and traders to their assets. This is because there are more potential depositors to attempt to link assets to. When it comes to shielding assets, it is also important to consider the type of token. Shielding common stablecoins like USDC or DAI will provide greater anonymity compared to shielding obscure meme tokens with only a few depositors.

MEV and Censorship Resistance

Most decentralized exchanges (DEXs) operating on public blockchains, due to their open and transparent nature, enable the existence of Miner Extractable Value (MEV). Order details on DEXs such as Uniswap or Curve are not kept confidential. This openness allows searchers and builders to rearrange transactions in bundles and blocks in order to extract MEV.

Privacy is important but it is a subjective matter. Tornado Cash has been effective in enabling individuals to conduct their financial activities on-chain anonymously. However, being open-source and freely accessible, it has also been used by criminals to launder stolen funds. The Lazarus North Korean hacker group, for instance, has managed to steal nearly a billion dollars ($958 million) to date. The Office of Foreign Asset Control (OFAC) maintains a list of specially designated nationals and blocked individuals, which includes malicious actors like Lazarus and even the smart contract address of Tornado Cash for non-compliance. Non-compliant protocols face a high risk of having their transactions excluded from on-chain blocks, as the top 5 out of 6 largest block builders are OFAC compliant. This raises concerns about censorship resistance. The issue extends beyond builders, as validators and relayers can also choose to omit transactions or blocks.

Dark pools allow order details to remain hidden from third parties, including validators. This is achieved by issuing Zero-Knowledge Proofs (ZKPs) for each order, eliminating the need to reveal details for order matching or settlement. This architecture can provide resistance against Miner Extractable Value (MEV) and censorship, as long as the transaction workflow remains within the encrypted environment of a dark pool. However, if public contracts are invoked or a shared sequencer is used, MEV-capturing opportunities may arise again.

Compliance

Whether you’re operating within a protocol or as an individual user, the last thing you would want is to have your transactions left hanging, unprocessed within blocks. This can risk the success of your state transition or even attract unwanted attention from regulatory authorities like the SEC. Unfortunately, these concerns became a harsh reality for the founders and developers of Tornado Cash, who were arrested on money-laundering charges. Additionally, the TC front-end is considered illegal or defunct in many countries and is not supported by several CEXs (centralized exchanges). However, it is worth noting that the TC smart contracts themselves are still operational and can be utilized by individuals with some technical knowledge.

In order to avoid such situations, dark pools and other privacy-focused protocols have devised methods that enable users to maintain compliance based on their own discretion.

Renegade allows traders to select a set of counterparties to trade with. Individual traders may only require basic AML/sanction checks on their counterparties, while institutions may need KYB/KYC checks done on their counterparties. The logic for selecting counterparties and performing compliance checks using ZKPs can be established prior to the start of the MPC.

Portal Gate and Panther Protocol collaborate with compliance oracles, who in turn collaborate with traditional compliance service providers, to perform off-chain KYC/KYB verification of user accounts. Once a user is verified, they receive a ZK proof of compliance, enabling their wallet to function with the dark pool protocol. The verification process needs to be performed periodically.

With Penumbra, users can demonstrate compliance off-chain by using ‘transaction perspectives’ to disclose their activity, including the selective disclosure of the provenance of funds. Similarly, with Railgun, users have “viewing keys” that enable them to view transactions and balances from a specific block time range, from X to Y.

Railgun utilizes a concept called “Proof of Innocence”, which is discussed in the Privacy Pools 2.0 paper developed by Chainway. Proof of Innocence enables users to demonstrate the legitimacy of their transactions without disclosing any transaction details. Merkel trees serve as a collective representation of all previous UTXOs (transactions and balances). This allows for the determination of whether a specific balance was included in a particular transaction set. However, in order to prove that a certain balance was NOT part of a specific transaction set, a Sparse Merkle tree is required, which keeps track of NULL values. With this proof, it is possible to verify that the balance was not included in the specified transaction set. To accomplish this, Recursive SNARKs (Succinct Non-interactive Arguments of Knowledge) are employed. They provide evidence that the chain of proofs, starting from the initial deposit and leading to the eventual withdrawal, was accurately calculated. This serves as proof that a user’s balance was not part of the specified transaction set.

Persistent Thoughts

Dark pool users rely on public prices as a benchmark for their trades. However, the prices of assets on public exchanges may not accurately reflect movements happening within dark pools. Prior to 2014, price discovery was a one-sided relationship between dark pools and public exchanges. To address this issue, FINRA and the SEC implemented an initiative that mandated the public disclosure of dark pool trading information on a security-by-security basis. However, there was a delay of 2-4 weeks in publishing this information after settlement. It is still uncertain whether similar public disclosure standards will be applied to crypto dark pools in the future to enhance price discovery for all participants in the market.

Pure Play Dark Pools will need to generate order flow on both sides of the market to achieve critical mass. We might witness a migration of OTC deals to dark pools due to their enhanced security, privacy, and cost-efficiency compared to OTC desks and Telegram groups. However, when private DEXs interact with public liquidity pools or external smart contracts, conducting simulations based on leaked information could potentially expose certain trades and traders.

Disclaimer：

1. This article is reprinted from [吴说区块链]. All copyrights belong to the original author [Muhammad Yusuf， Delphi Digital]. If there are objections to this reprint, please contact the Gate Learn team, and they will handle it promptly.
2. Liability Disclaimer: The views and opinions expressed in this article are solely those of the author and do not constitute any investment advice.
3. Translations of the article into other languages are done by the Gate Learn team. Unless mentioned, copying, distributing, or plagiarizing the translated articles is prohibited.