Local Informants and Verification Theatre: Reflections on Cross-Chain Proofs between Distributed Ledger Networks

Introduction and summary

This article discusses some foundational questions in cross-network interoperation between distributed ledger networks. Following a preamble setting out some terms, it delves into a pair of case studies showing how “proof of action” is assembled on one network and verified on another, a core mechanism for reliable interop. The case of Corda, a private permissioned network, is contrasted with that of Ethereum, a public blockchain. Reconciling the two cases has been an essential part of the Harmonia project, which aims to demonstrate a reliable asset swap mechanism between Corda and Ethereum networks. The difficulties and trade-offs involved are quite subtle, and the discussion of the case studies is fairly technical, but some general observations and conclusions are presented at the end of the article.

The argument made here is that the goal of verification without trust, relying solely on “proof” that an event has occurred on a remote network, is sometimes unfeasible, and attempts to realise it may result in “verification theatre” that accumulates cryptographic evidence while masking unprovable assumptions about the provenance and correctness of the information relied upon. Instead, when needing to evaluate network-local knowledge, it may be preferable to make use of the attestations of trusted “local informants”, reasoning carefully and explicitly about where trust is placed.

Preamble: Consensus, Trust and Proof

A distributed ledger can usefully be thought of as a shared recital of things that have definitely happened, where what it means for something to have definitely happened is that there is binding agreement among concerned parties that it has done so. It is a collective endeavour in which the goal is to produce and maintain a consensus reality.

The facts recorded in such a ledger are typically not externally empirically verifiable: there may be no real-world correlate, outside the ledger, to the fact that a party has a certain number of tokens of a particular denomination to their name. Instead, it is held to be true because an agreed history of token issuances and transfers entails that it is true. For example: A, to whom we attribute the authority to issue tokens, issued 1000 of them to B, who transferred 500 of them to C, who transferred 200 of them to D and therefore has 300 remaining.

Real-world facts can enter the ledger in the form of attestations. If an issuance of 1000 tokens is backed by a quantity of gold held in a physical vault somewhere, then this fact can be represented by the cryptographic signatures of trusted parties who attest to it. Typically these parties are trusted because they also hold some real-world liability for the validity of the facts to which they attest. For example, there may be a legally binding contract that says that if a custodian cryptographically signs to say that they hold certain real-world assets in custody, then they are responsible for releasing those assets when agreed conditions are met and will incur financial and reputational penalties if they find that they are unable to do so.

The consensus mechanisms of distributed ledger technologies are usually designed so that properties of the ledger are provable by a combination of cryptographic and other probabilistic means. By “probabilistic” I mean that “if X, then it is very highly unlikely that not-Y” is treated as a good enough stand-in for “if X, then necessarily Y”. Cryptographic proofs are themselves probabilistic: for example, “if you can produce a hash preimage of a committed hash of sufficient cryptographic strength, then it is very highly unlikely that you guessed or calculated that preimage as opposed to being (or having had the preimage disclosed to you by) the party who initially committed the hash”. When (as happens from time to time) a cryptographic scheme is broken, proofs founded upon it cease to function as proofs.

Sometimes, as in proof-of-stake consensus, “very highly unlikely” has a social and economic component as well as a strictly numerically probabilistic one. It’s possible that a quorum of validators with a combined stake sufficient to finalise a transaction block will all decide to give false reports, but the risk to each individual party of losing their stake, and the difficulty of co-ordinating mass-falsification, are such that honest behaviour is overwhelmingly more strongly incentivised than dishonest behaviour. Game theoretic reasoning about incentives is central to the design of public blockchain technologies, which rely on being able to treat “this would never happen” as an acceptable stand-in for “this cannot happen”.

The purpose of this preamble is to remind the reader that distributed ledgers are social technologies with computational scaffolding. The role of the scaffolding is to enhance confidence by lowering risk: to provide not absolute but “good enough” guarantees that rules have been followed. The problem we are concerned with in “blockchain interop” is how to transfer confidence across not only technological but also social (legal/regulatory, institutional, geographic and so on) boundaries. What guarantees can we provide when information leaves the circle of peers bound by common protocols and incentives?

Case Study 1: Consensus Transcripts in IBC

The concept of a “consensus transcript” is introduced in the specification of IBC (Inter-Blockchain Communication), a protocol for reliably passing messages between blockchain networks by means of relayers which observe provable properties of one network’s ledger and transmit remotely verifiable proofs of these properties to the other. These proofs have two aspects, which I will call proof of outcome and proof of consensus. Proof of outcome is proof that a transaction will, if finalised, have caused a specific thing to have happened on the originating network, for example “proof that the outcome of transaction X is to transfer 100 tokens from address A to address B”. Proof of consensus is proof that this outcome “definitely happened” in the sense that the originating network reached consensus on the transaction in question. A consensus transcript is a summary of the consensus process of the source network sufficient to convince an off-network verifier that consensus was reached.

On an Ethereum network, proof of outcome is achieved in IBC by demonstrating that a value representing a commitment is present in a contract’s storage, and demonstrating that a given IBC message is committed to in this value. Changes to a contract’s storage are recorded as part of each transaction, so if we can prove that a transaction writes the value in question to storage, and that the transaction is included in a given transaction block, then we have a proof that the block materialises that value. All of this can be demonstrated using hashes and Merkle inclusion proofs.

An alternative method, followed for example by the Corda client in Harmonia, is to use transaction event log records, which are recorded in the transaction receipts stored in the transaction receipt trie of each block. Essentially the same methodology applies in this case, except that event log records are stored in the clear rather than as commitment values. By contrast, IBC’s approach enables IBC messages to be validated by the target chain while remaining opaque to the source network.

Proof of consensus is achieved by having a “light consensus client” (i.e. one that can observe consensus but does not necessarily take part in establishing it) on the source chain construct “cheaply-verifiable consensus transcripts” which demonstrate that the steps required to reach consensus on the source chain were followed. For example, in a network where consensus was reached through proof-of-authority, it it might gather the signatures of a quorum of a fixed pool of validators on the root hash of the transaction block. In this case, for the transcript to be verifiable, the receiving chain would have to know the identities of the validators in question, and trust their signatures as evidence that a transaction block had been finalised.

Here there is already some coupling between the two chains: one must know something about the authorities which act as arbiters of consensus on the other. Furthermore, what is the basis of trust in those authorities? Do they assume legal liability to the interested parties on the receiving chain for only signing valid and finalised transaction blocks? It may be that parties on the receiving chain trust them on the basis that they are already trusted as validators on the source chain, and what matters is that consensus has been reached on that chain. However, nothing technically prevents the validators on the source chain from colluding to manufacture a false consensus transcript (i.e. signing a spurious transaction block) for the purpose of deceiving someone on a remote chain, while providing true information to their peers on the source chain. The light consensus client might check with those peers to see what the view of the source network was, ensuring that it agreed with the transcript, but in that case the light consensus client itself must be brought into the circle of trust.

The general point here is that if proof of consensus bottoms out in verifying trusted signatures, then the overall proof of action (proof of outcome plus proof of consensus) also depends on trust. It is trivial for a party who wishes to counterfeit an action to construct and sign a fake transaction and to build a Merkle tree ostensibly showing its inclusion in a block of transactions (the other transactions ostensibly represented in the tree being, in reality, randomised hashes). Only the proof of consensus establishes this action as having “definitely happened” (“my network agrees that I have transferred the funds I said I transferred, therefore it is true”), and this proof is only as strong as our trust in the authorities underwriting it.

Proof of consensus in a Proof of Stake network

In the case of a proof-of-stake Ethereum network, matters are considerably more complicated. The pool of potential validators is extremely large (>500,000), and whether a signing address is truly a validator address depends on whether it has staked a certain amount of ETH, a fact which can only be checked by referring to the ledger itself. The finality of a transaction depends on a block chain history including the block containing that transaction having been accepted by a pseudo-randomly-selected committee of validators. In the case of diverging histories, the history validated by the committee with the highest aggregate stake wins. while validators who have backed the wrong horse are penalised and lose some of their stake) In short, a very large amount of information local to the network is involved, and there is no direct way to construct a cheaply verifiable transcript of the consensus process.

Outside of “interop” scenarios this problem tends not to arise, because instead of consulting an offline transcript of a remote chain’s consensus process a local client will simply query a beacon node to discover the status of a transaction. Assuming the integrity of the beacon node (or quorum of carefully chosen beacon nodes, to mitigate the risk of an individual node’s having been subverted), we can observe that a transaction has been finalised and move on to the next action. But this option is not available to IBC, for example, because the assumption is that the remote chain cannot contact the source chain: it must be able to perform its consensus verification offline.

One option is to use the sync committee, a rotating pool of validators which receive a small reward for continuously signing the block header that is the new head of the chain at each slot. Once again the receiving chain must know which addresses belong to this pool of validators. An online client might do this by consulting its record of previous validated blocks, in which the committee members for future blocks are advertised, or by making a live request to the network. However, an offline transcript must also be able to refer to a history of previously validated blocks, up to at least a trusted snapshot, and verify that each block’s advertised validators have validated the next block up all the way up to the block of interest.

We are still in the position of having to trust the first committee of validators not to sign a spurious block setting up a sequence of spurious committees signing spurious blocks, but there is at least the possibility of checking the consistency of block histories presented across multiple consensus transcripts stemming from the same snapshot. Alternatively, a sync contract on the receiving chain could be set up to track the history of sync committees, and then referred to as a local authority on who the validators are for a block at any given time. This would bring us back to the “proof of authority” situation, except that the validator set would be continually rotating, increasing the difficulty of co-ordinating collusion and making it feasible for the receiving chain to trust the source chain’s validators purely on the basis that they were trusted by the source chain.

Case Study 2: Proof of outcome in Corda, a private network

In the case of Corda, proof of consensus is a relatively straightforward matter. Transactions are finalised by being observed and signed by a notary, which marks off the input states of each finalised transaction as consumed to prevent double-spends of transaction inputs. However, a non-validating notary performs no other validation on the transaction it notarises; indeed, the details of the transaction are “torn off” by being replaced in a Merkle tree by nodes containing only their hashes.

This is an aspect of Corda’s privacy model, which means that no central party has oversight over all transactions. It means however that validation of transactions is left to the parties involved, and subsequent parties checking the history of new transaction input states along the “backchain” of previous transactions which generated those states as outputs. This peer validation means that while I can get a notary to finalise an invalid transaction illegitimately awarding myself a large number of tokens, I cannot spend those tokens with another party because my invalid transaction will fail validation during their checking of the backchain of the transaction consuming those tokens as inputs.

Proof of outcome is accordingly more difficult to come by, since even if the receiving chain can decipher a serialised Corda transaction it cannot rely on the proof of consensus also to act as proof that the transaction was valid according to peers on the source network. There is the further problem that the representation of the transaction itself is somewhat opaque to off-network validators Both the class definitions defining the data structures serialised into the transaction and the contract validation method are expressed in the Java bytecode of the source CorDapp. A peer on the CorDapp’s application network, sharing the same code, can readily deserialise a Corda transaction and validate it, but a Solidity contract on a remote network cannot readily (or inexpensively) do the same.

The mechanism used in Harmonia is as follows. A peer on a Corda network can produce a “draft” transaction which would require their signature to be valid, and forward it minus that signature to another peer for validation. The validating peer can then use the contract logic to check the validity of the draft, together with the validity of all transactions along the backchains of its input states. If everything checks out, the validating peer can attest that the notary’s signature on the draft transaction’s hash, together with the signature of the author of the draft, will constitute both proof of outcome (the transaction does what the author says it does) and proof of consensus. It is then up to the author to obtain both of these signatures (i.e. by signing and notarising the draft transaction) to complete a proof of action.

A remote chain looking to consume this proof must trust the validating peer, since there is no easy way of determining the outcome of a presented Corda transaction outside of its originating Corda network. To repeat: full validation of a Corda UTXO transaction requires not only the ability to decode the serialised transaction data, but also the ability to run the contract validation code not only for that transaction but also for all transactions in its backchain. Rather than strongly coupling the two networks and onerously reproducing the contract verification logic of one on the other, we may choose to make use of trust relationships instead. Much as we might use the sync committees in Ethereum to provide an assurance that consensus has been reached without needing to present a complete transcript of the full consensus process, we instead make use of a trusted local informant to avoid needing to construct and verify a complete transcript of the validation process containing all the evidence that a validating Corda peer would need to see.

If we are given the hash of a draft transaction, signed by a local informant which has reviewed the draft, together with the public keys of the author and the notary which will finalised the transaction, then provided we trust the local informant we can treat this information as a proof of outcome for the transaction hash, with the signatures of the author and notary then supplying the proof of consensus and completing the proof of action. Notably, this means we can carry out a transaction on the target network which commits to a given draft transaction for which we have an attested outcome (first phase in the two-phase execution of an asset transfer) and requires the corresponding proof of consensus to authorise a subsequent transaction (second phase, completing the transfer).

Ego and Alter Ego

Who should take on the role of the local informant, especially given the relationship noted earlier between trust and liability? In the case of an asset swap, the identities of the parties on the two network are aligned in terms of interest: we say that the receiver on network B is the “alter ego” of the sender on network A, and vice versa. In terms of incentives this means that I benefit if my alter ego benefits (for example, if payment for an asset I am selling on my network is received by my wallet on the payment network); in terms of trust, it means that I trust my alter ego to act for me on their network, and they trust me to instruct them to do so (for example, as the receiver of the asset I instruct my alter ego on the payment network to make a payment to the seller’s wallet, and they act on my instruction).

Given these trust relationships, the role of proof in the asset swap is to provide unilateral authority to act if the conditions set by a counterparty have been met. I do not need proof that my alter ego has sent payment, only confirmation, since my alter ego is trusted by me as a source of information about their own actions. However, the seller of the asset may place the asset into a locked state from which it can be unilaterally obtained by me, without further authorisation from them, provided I meet their condition that I present proof that payment was sent. My own trust in my alter ego is not sufficient to meet this condition, because the seller does not trust my alter ego. Conversely, because I want assurance that the seller cannot default on sending me the asset once I have sent payment, I cannot rely on an agreement that the seller will transfer the asset to me if their alter ego confirms to them that payment has been received: I want to be able to take the asset automatically if I have proof of payment in hand.

Supposing that the network on which the asset is to be transferred is a Corda network, and the payment network is an Ethereum network, the asset swap “pattern” looks like this:

• The seller on the Corda network prepares a draft transaction that would place the asset into a lock from which the buyer could automatically obtain it on presentation of proof of payment.
• The buyer on the Corda network validates the draft transaction, and instructs the payer on the Ethereum network to move payment into a “committed” state. The contract on the Ethereum network ensures that the payee will be able to transfer payment from this committed state to their own wallet if they can prove that the draft transaction was finalised, i.e. that the asset has been locked such that the buyer will be able to obtain it if they can show proof of payment.
• The seller signs and notarises the draft transaction, completing a proof of action on the Corda network that the payee can use on the Ethereum network to transfer payment to themselves.
• The payee uses the proof of action to take payment. The payer observes that payment has been taken and constructs a proof of action showing that transaction receipts containing event log messages giving evidence of the payment were connected to a finalised Ethereum transaction block.
• The buyer uses the proof of payment to obtain the asset from the lock on the Corda network.

Key to this pattern is that the trust relationships on which we rely in taking the attestation of a local informant as proof of outcome for a Corda transaction are already established in the commercial and operational relationships between the parties to the swap.

Observations and conclusions

Based on these case studies, we make the following observations:

• Knowing that something in particular (proof of outcome) has definitely happened (proof of consensus) on a distributed ledger often requires situationally enriched knowledge of that ledger’s history, protocols, network structure, smart contract rules, consensus mechanisms and so on.
• Local participants in a distributed ledger typically have either direct access to this situationally enriched knowledge or mechanisms for reliably gathering from their peers, since they need it to verify smart contract rules and contribute to consensus processes.
• Situationally enriched knowledge can be difficult to transmit and verify outside of the bounds of the network. For example, a “cheaply-verifiable consensus transcript” such as that required by IBC can be difficult to construct for proof-of-stake Ethereum: once we start pulling on the threads of consensus, we may find that we end up having to buy the whole rug.
• We can use attestation to shorten the path to proof, so that branches of a proof terminate in the signature of a trusted local informant rather than having to spread out into the depths of the originating network.
• Use of the sync committees in Ethereum to construct more lightweight consensus transcripts is one example of this principle in action; validation of draft transactions by Corda network peers is another.

Given that it is often practically necessary to use trusted local informants to reduce the dependence of an interoperating network on situationally enriched local knowledge, it is worth weighing the value of closely inspecting another network’s transaction data, or tracking the membership of its validator set, against the cost of faithfully reflecting the full contextual detail of one network within another. Is there a trust relationship we can leverage to reduce the weight and complexity of evidence that needs to be examined to reach confidence that an action has been taken?

This question becomes particularly acute when Corda networks are involved, since the use of non-validating notaries means that proof of finality does not automatically count, as it does on Ethereum, as proof of validty. When we observe an event log record in the transaction receipt of a finalised Ethereum transaction, we know that the network’s validators agreed that the contract rules for the contract governing that transaction were followed. Provided the event would only have been emitted, by the rules of the contract, if a certain outcome was reached, we can treat the emission of the event by a finalised transaction as proof of outcome. The same inference is not available to us for a Corda transaction notarised without validation, which makes IBC-style interoperation unfeasible unless we are prepared to make the strong assumption that no party in a Corda network will ever (accidentally or deliberately) notarise an invalid transaction. The detailed checking of “proofs” which lack a crucial component needed for their integrity is both costly and foolhardy: a form of verification theatre which may placate a casual observer but will not deter an attacker.

The sync committee example shows that a trust relationship need not be in a single party holding the entire liability for giving an accurate report of network-local facts. A federation of witnesses with rotating membership is a strong alternative to a centralised authority. Alternatively, as in the asset swap example discussed, the necessary trust relationships between parties may be already implied in the roles they play within the orchestration of an interoperability pattern. In each case, the interplay of proof, trust and liability must be carefully considered.