Package: py-overlay-evm
py-overlay-evmEthereum VM written in Python with minimal external dependencies and aimed at a mock execution of smart contracts.RationaleThis project grew from what was essentially an exercise to understand how smart contracts work on the Ethereum blockchain and its clones. One way to gain understanding is to program an EVM from scratch, without referencing existing code, and that is precisely what I did. The codebase includes an implementation for the Ethereum bytecodes, including the corresponding gas calculations, own Keccak-256 implementation, as well as a Solidity wrapper for conveniently calling functions in Solidity contracts.As of now, the implementation of the EVM is incomplete, and the API is still likely to change. I reckon the code is good enough to play around with, but it is certainly not ready for production (for instance, there is zero test coverage). The examples below demonstrate what can be done now.FeaturesMinimal external dependencies:numpy, requests.Functional style state changes:each call to a smart contract returns a new view of the blockchain with the respective changes implemented, the initial view from which the call started remains available and unchanged.Overlay architecture:data is read from the (public) node if these data have not been previously accessed, further reads are cached, and any changes are kept in an overlay in memory.Gas calculations:gas usage is calculated exactly (work in progress).ExamplesThe following examples all use the Ethereum blockchain.Example I (manual contract calls)frompy_overlay_evm.rpcimportNodefrompy_overlay_evm.evmimportChain,executefrompy_overlay_evm.keccakimportkeccak# Initialize a link to a public node## During initialization, the current block number is noted, and all the# subsequent requests to the chain are done for the same block number. This# behaviour ensures consistency but it also means that all the subsequent# requests must be done within a short time interval, because public nodes do# not return data for somewhat older blocks.## For testing purposes, we can use the public nodes from the Flashbots project.url="https://rpc.flashbots.net"chain=Chain(Node(url,verbose=True))# We consider a Wrapped Ether contract for this example## Addresses are always encoded as integers.weth=int("0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2",16)# We will be calling the contract from a mock addresscaller=int("0xabababababababababababababababababababab",16)# We are going to call the `name()` method on the contract. The method accepts# zero arguments. According to the Solidity ABI, we need to compute the Keccak# signature of "name()", take its first 4 bytes and pass it as the byte input# to the contract.data=keccak(b'name()')[:4]rslt=execute(chain=chain,caller=caller,address=weth,value=0,data=data,trace=True,)# `rslt.data` contains the string "Wrapped Ether" encoded according to the# Solidity ABIprint(rslt.data)# prints an equivalent of:# bytearray(b'# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x20# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0d# Wrapped Ether\x00\x00\x00# \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00# ')# `rslt.chain` contains the new state of the blockchain after a successful# execution of a contract. In this case, given that the method `name()` is# read-only, `rslt.chain` will be equivalent to `chain`.# `rslt.gas` reports the exact gas used by the transaction. Please note that# gas calculations have not been extensively tested and could well be# erroneous.print(rslt.gas)# prints "24174"# Finally, `rslt.trace` contains the trace of the contract execution.print(len(rslt.trace))# prints "228"foropinrslt.trace:print(op)# prints:# 0x60 push1() -> 0x60, gas: 3# 0x60 push1() -> 0x40, gas: 3# 0x52 mstore(0x40, 0x60), gas: 12# 0x60 push1() -> 0x4, gas: 3# 0x36 calldatasize() -> 0x4, gas: 2# ...# 0x80 dup1() -> 0xa0, gas: 3# 0x91 swap2(), gas: 3# 0x03 sub(0x100, 0xa0) -> 0x60, gas: 3# 0x90 swap1(), gas: 3# 0xf3 op_return(0xa0, 0x60), gas: 0# In this case there are no external calls to other contracts, but if there are# such calls, their traces also get recorded.Example II (Solidity wrapper)For conveniently calling Solidity contracts, there is a wrapper that handles the encoding of the call signature as well as the translation of the data from Python types to Solidity binary format and back. Here, we repeat the first example but using the Solidity wrapper.frompy_overlay_evm.rpcimportNodefrompy_overlay_evm.evmimportChain,execute,mkcallfrompy_overlay_evm.solidityimportsolidity,string# Initialize a link to a public nodeurl="https://rpc.flashbots.net"chain=Chain(Node(url,verbose=True))# We consider a Wrapped Ether contract for this exampleweth=int("0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2",16)# We will be calling the contract from a mock addresscaller=int("0xabababababababababababababababababababab",16)# To use the Solidity wrapper, we simply define a Python function with the same# name as the Solidity function and we type annotate it using Solidity type# names. The body of the function does not matter, so we'll just use `pass`.@soliditydefname()->string:pass# A wrapped function accepts a partial call to `execute()` as the first# parameter, and the remaining parameters are simply the parameters for the# Solidity function, in this case none.rslt=name(lambdadata:execute(chain,caller,weth,0,data,trace=True),)print(rslt.value)# prints "Wrapped Ether"# To simplify making a partial call to `execute()` there is also `mkcall()`:rslt=name(mkcall(chain,caller,weth,0,trace=True),)Example III (Uniswap V3 router)This is a longer example that walks through a non-trivial operation, namely depositing mock WETH coins to a mock account, and then changing them into USDT coins via a Uniswap V3 router. This example also demonstrates more extensive usage of the Solidity wrapper.fromdataclassesimportdataclassfrompy_overlay_evm.evmimportChain,mkcall,save_tracefrompy_overlay_evm.rpcimportNodefrompy_overlay_evm.solidityimportsolidity,addressfrompy_overlay_evm.solidityimportuint24,uint160,uint256## Firstly, we define all the Solidity functions that we will need@soliditydefdeposit():pass@soliditydefbalanceOf(address:address)->uint256:pass@soliditydefapprove(guy:address,wad:uint256,)->None:pass# A dataclass can be used if a Solidity function wants a structure as a# parameter. The dataclass can have an arbitrary name, because the name does# not form a part of the function's signature.@dataclassclassSwapParams:tokenIn:addresstokenOut:addressfee:uint24recipient:addressdeadline:uint256amountIn:uint256amountOutMinimum:uint256sqrtPriceLimitX96:uint160@soliditydefexactInputSingle(params:SwapParams)->uint256:pass## Secondly, we define all the required addresses# Tokensweth=int("0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2",16)usdt=int("0xdac17f958d2ee523a2206206994597c13d831ec7",16)# Uniswap V2 routerrouter=int("0xe592427a0aece92de3edee1f18e0157c05861564",16)# Mock holder addressholder=int("0xabababababababababababababababababababab",16)## Finally, we prepare mock WETH and change them into USDT# Initialize a link to a public nodeurl="https://rpc.flashbots.net"chain=Chain(Node(url,verbose=True))# Create an unlimited supply of ETH at the mock address :3chain[holder].balance=(1<<256)-1# Deposit 100 ETH to WETHamount=100*10**18rslt1=deposit(mkcall(chain,holder,weth,amount),)# Check that the deposit operation was successfulx=balanceOf(mkcall(rslt1.chain,holder,weth,0),address=holder,).valueprint(f'WETH balance:{x/10**18}')# Approve withdrawal of 100 WETHrslt2=approve(mkcall(rslt1.chain,holder,weth,0),guy=router,wad=amount,)# We use the block's timestamp plus 60 seconds as a deadlinedeadline=chain.node.block_timestamp()+60# Exchange 100 WETH to USDTrslt3=exactInputSingle(mkcall(rslt2.chain,holder,router,0,trace=True),params=SwapParams(tokenIn=weth,tokenOut=usdt,fee=500,# 0.05%recipient=holder,deadline=deadline,amountIn=amount,amountOutMinimum=0,sqrtPriceLimitX96=0,),)print(f'USDT amount out:{rslt3.value/10**6:.2f}')# prints "USDT amount out: 256474.98" at the time or writing# The trace is large (8,402 operations at the time of writing, but that will# differ depending on how many ticks get traversed), so we save it to a file for# viewing in an external editorsave_trace(rslt3.trace,'swap.trace')# Check final WETH balancex=balanceOf(mkcall(rslt3.chain,holder,weth,0),address=holder,).valueprint(f'WETH balance:{x/10**18:.2f}')# prints "0.00"# Check final USDT balancex=balanceOf(mkcall(rslt3.chain,holder,usdt,0),address=holder,).valueprint(f'USDT balance:{x/10**6:.2f}')# prints "USDT balance: 256474.98" at the time or writingRoadmapIn case I happen to have time to further work on this project, the overall roadmap is as follows:Gas calculationsPython packageFull coverage of EVM bytecodesTests for EVM bytecodesFull coverage of Solidity datatypesTests for Solidity datatypesDocumentation