Introduction to the Ethereum blockchain
By: Roberto Infante
Tags: Merkle Patricia Trie, Ethereum Blockchain, Ethereum, Blockchain, Merkle Tree

In the previous article, Transaction lifecycle on the Ethereum blockchain, extracted from chapter 1 of my book Building Ethereum Dapps, I walked you through a typical Ethereum blockchain transaction. While doing so, I introduced various terms you might not be entirely familiar with, such as cryptographic hash, blockchain, mining, consensus, Proof of Work (PoW), Merkle tree, Merkle-Patricia trie and you might want to learn more. In this article I will clarify these concepts.

Cryptographic hash

I’ll start by explaining what a cryptographic hash is. A hash function is any function that can map data of arbitrary size to data of fixed size. The fixed size data is called hash or digest. To give an example, you can design a hash function so that it always generates a 64-bit hash from a file or string of any size. Whether its size is 10 KB or 10 GB, a 64-bit hash will get generated, as illustrated in figure 2.

Figure 2. A hash function produces a hash of a fixed size (64 bits in this example) given an input of any size.

A cryptographic hash function is a hash function that has five additional properties:

· It’s deterministic. The same input will always generate the same hash.

· It’s quick to compute.

· It’s a one-way function, unfeasible to invert. This means that the only way to deduce the original data from its hash would be to try, by brute force, to obtain the same hash by applying the function to an enormous number of input data sets.

· It should be almost impossible to obtain the same hash from two different sets of input data. Although a low chance exists that two inputs might produce the same hash, it’s impossible to determine them a priori, without applying the function to an enormous number of inputs, as suggested in the previous point.

· A slight change in the input data should produce substantially different hash values. Consequently, also because of what I said in the previous point, unless you’re applying the cryptographic hash function to the same input, you won’t be able to intentionally get the same hash or even a close one.

You will see shortly why cryptographic functions are useful when it come to blockchains.

Blockchain

What’s the blockchain? A blockchain is a distributed database that holds records called blocks. Figure 3 illustrates the structure of a typical blockchain.

Figure 3. A blockchain is a sequence of blocks, each containing a sequence number, a timestamp, and a list of transactions, each individually digitally signed. Each block also references the cryptographic hash of the previous block.

A block includes a list of transactions, which are digitally signed to prove their provenance. Most blockchains digitally sign transactions with an elliptic curve digital signature algorithm (ECDSA), based on elliptic-curve cryptography. Each block contains a timestamp and a link to a previous block based on its cryptographic hash. In this way, a blockchain block holds both a reference to the latest set of transactions (the hash of the previous block) and indirectly to the full history of all the transactions that have been stored on it since its inception (because each block is linked to its predecessor).

This structure guarantees transactions can’t be tampered with or modified. A transaction recorded in a block can’t be altered retroactively because, to modify it, the hash of the block containing it would have to be regenerated, and this wouldn’t match the existing one already referenced by subsequent blocks, as shown in figure 4.

Figure 4. An attempt at altering the contents of a block, for example its transactions, won’t be successful: the new hash generated from the altered block details won’t match the original block’s hash already directly referenced in the next block and indirectly referenced in the subsequent blocks.

The blockchain structure I’ve described is, in fact, a simplified version of real-world blockchain data structures such as the Merkle tree used by Bitcoin or the modified Merkle-Patricia trie used in Ethereum. I will describe them to you shortly.

A blockchain is managed autonomously through a peer-to-peer (P2P) network that facilitates fault tolerance and decentralized consensus by processing all transactions independently on each node. Given these characteristics, blockchains are particularly suitable for recording permanently the history of events. This is useful for identity management, transaction processing, and provenance tracking, to name a few use cases.

Mining

To encourage the P2P network supporting the Ethereum blockchain to process its transactions continuously, active processing nodes, also called mining nodes or miners, are rewarded for the computational resources provided, and indirectly to cover the associated electricity costs, through the consensus mechanism. Every few seconds, one successful miner is entitled to generate and keep a certain number of tokens of the cryptocurrency supported by the platform. Such cryptocurrency has economic value, as it can be used to purchase services on the network, but it also can be exchanged for conventional currencies, such as dollars, yen, euros, and so forth. In the case of the Bitcoin blockchain, they’ll be given several Bitcoin tokens, worth around $2,000 each at the time of writing. The tokens given by the Ethereum blockchain are called Ether, and they’re worth around $200 each at the time of writing. Let’s now look at how the consensus mechanism works.

Consensus

Consensus is, as I mentioned earlier, the mechanism by which participant nodes of the network agree on the outcome of a transaction. Consensus is also distributed, because it’s determined by many participants, and trustless, because the participants don’t need to trust each other. In fact, consensus isn’t reached on individual transactions but on new blockchain blocks. Each participant verifies independently that a new block is valid and, if satisfied, propagates it further to the rest of the network. What happens in practice is that if most participants have accepted the block as valid and it has propagated successfully throughout the network, miners will use such a block as the latest valid block, and the rest of the blockchain will be built on it. If a malicious miner appended an incorrect block to the blockchain and it propagated to its peer nodes, these nodes would reject the new block, and the malicious chain would die out immediately. The same would happen if a full node tried to modify a block before propagating it to its peers. As you can see, the key step of the consensus mechanism is the verification of the latest block by a participant node. After verifying the digital signature of the individual transactions present on a block, a participant node verifies that the hash of the block is valid. Such hash is produced by miners according to an agreed protocol. The earlier versions of Ethereum used an algorithm called Ethash, based on a Proof of Work protocol. Future versions will be based on a Full of Stake protocol called Casper.

Proof of Work

As you saw earlier, the content of a block can be summarized with a cryptographic hash calculated on its metadata and transactions data, and also on an additional piece of data of a fixed length, such as 32 bits, called nonce. The objective of the Proof of Work (PoW) protocol is that miners must find a nonce such that the hash generated fits a certain constraint, for instance, having many leading zeros. Constraining a 64-bit unsigned integer hash to have 13 leading zeros when represented in hexadecimal format, as in the example of figure 5, reduces the number of valid hashes from the theoretical maximum number of 18,446,744,073,709,551,615 to 4,095.

Figure 5. Proof of Work: generation of an unsuccessful and a successful block hash

Because of the properties of hash functions you saw earlier, the only way a miner can find such a nonce is by trying many possible values until the constraint on the hash has been met. In the example I just gave, every such attempt will only have a roughly 0.00000000000002% chance of being successful. When a satisfactory hash has been found, the miner is entitled to append the new block being processed to the blockchain and claim the token reward. As you can understand, this way of producing a valid hash is CPU-intensive, energy-consuming, and, consequently, economically expensive. The main reason for such an expensive algorithm is to dissuade malicious blockchain participants from appending new incorrect blocks or modifying pre-existing blocks and making them look like genuine blocks. The amount of energy (and money) necessary to perform such actions would make them unviable.

The Merkle tree and the Merkle root

The blockchain structure I’ve shown in the previous diagrams is a simplified representation of a real one. Generally, a miner places in the block two parts: a header and a body, as shown in figure 6. The body contains all the transactions included in the block. The header contains the block metadata you saw earlier, such as the block number, timestamp, previous block hash, and PoW nonce. It also contains the Merkle root of the transactions Merkle tree that the miner calculates.

Figure 6. The structure of a block, including a header containing metadata, such as the block number, timestamp, previous block hash, and Merkle root of the transactions Merkle tree, and a body containing the transactions collection

The transactions Merkle tree, as shown in figure 7, is a tree structure built as follows:

· The block’s transactions are placed at the bottom of the tree, arranged in pairs.

· Each transaction is hashed, and each of these hashes becomes a leaf of the Merkle tree.

· A hash is calculated for each pair of contiguous hashes.

· The hashing of contiguous hashes is repeated until only two hashes remain. The hash of these two final hashes is the Merkle root.

Figure 7. A Merkle tree. Individual transactions are located at the bottom; the tree’s leaves are the hashes of the individual transactions; and the next row up is made of the hashes of the tree’s leaves. The top row, which is the hash of the hashes below, ends the tree: this is the Merkle root.

The Merkle root is therefore a single hash summarizing all of the transactions contained in the block in a way that guarantees their integrity. The advantage of having the Merkle root on the block header is that a client can synchronize the blockchain in a faster way by retrieving the block headers, rather than the entire transaction history, from the network peers. This is generally called light synchronization.

The modified Merkle-Patricia trie and the Ethereum blockchain

The Ethereum blockchain improves the design further. First of all, transactions are hashed in a more compact and efficient (yet still cryptographically authenticated) structure called a Merkle-Patricia-trie. Secondly, the block header (generated as usual by the miner) also contains, in addition to the Merkle-Patricia root of the transactions, the Merkle-Patricia root of the receipts, which are the transaction outputs, and the Merkle-Patricia root of the current blockchain state, as shown in figure 8.

Figure 8. An Ethereum improved block header. The header of a block of the Ethereum blockchain contains the root of the transactions Merkle-Patricia trie, which is a more compact and efficient structure than a Merkle tree. In addition, it contains the Merkle-Patricia root of receipts (which are the transactions effects) and the blockchain state.

As Vitalik Buterin explained in his blog post “Merkling in Ethereum”, with these three Merkle-Patricia tries, a client can efficiently check, in a verifiable way, the following:

· Whether a certain transaction is included in a certain block

· What the output of a transaction would be

· Whether an account exists

· What the balance of an account is

Detailed block validation

Now that you have learnt a lot more about blockchain structures, let’s revisit in more detail what happens during block validation which I briefly explained in step 6 of figure 1.

When a full node receives a new block, the transactions contained in the body are processed as follows:

The transactions are arranged in a transaction Merkle-Patricia trie specific to the new block. Transactions are executed on the node. This action generates transaction outputs (also known as receipts), which are arranged in a receipts Merkle-Patricia trie specific to the new block. It also alters the global state trie, of which only one instance exists on each node. If the roots of the new transaction trie, receipt trie, and modified state trie match the corresponding ones in the header, the block is considered validated. Then the new and altered tries are stored on the full node in a respective key-value store based on LevelDB, an open source NoSQL database developed by Google.

Note the following in figure 9:

· The transaction store contains a transaction trie per block, and each trie is immutable. The key of this store is the transaction hash (keccak 256-bit hash).

· The receipts store contains a transaction trie per block, and each trie is immutable. The key of this store is the hash of the receipts of a transaction (keccak 256-bit hash).

· The state store contains a single state trie that represents the latest global state and is updated each time a new block is appended to the blockchain. The state trie is account-centric, so the key of this store is the account address (160 bytes).