Back to the future!! Testing Solidity through time with Brownie.
By: Adrian Guerrera
Tags: Smart Contracts, Pytest, Testing, Ethereum, Solidity

Time to activate pytest

All you have to do is add a .py file in the /tests directory.

That’s it.

If you’re familiar with pytests, you can create a conftest.py file which we’ll keep all our deployed contracts and fixtures. Fixtures are reusable objects we’ll be calling each time we go to the future, and back.

from brownie import accounts

import pytest # Use pytest fixture so we can reuse this

@pytest.fixture(scope='module', autouse=True)

def test_token(Token):

name = 'Test Token'

symbol = 'TEST'

decimals = 18

mintable = True

initial_supply = 0

owner = accounts[0] # Deploy test_token

test_token = Token.deploy(owner

, symbol, name

, decimals

, initial_supply

, mintable

, {'from': accounts[0]}) return test_token

Here we deployed a contract “Token” that is frozen in time, importantly the scope set to ‘module’ allows you to run tests, but after the function finishes, resets the contract state before the next test.

@pytest.fixture(scope='module', autouse=True)

def frames_crowdsale(Crowdsale, test_token):



# Time when you run the test

start_date = rpc.time()

end_date = startDate + 60*60*24

owner = accounts[0] # Deploy crowdsale

crowd_sale = Crowdsale.deploy(test_token

, start_date

, end_date

, {"from": owner})



# Let the crowdsale mint

tx = frame_token.setMinter(

crowd_sale

, {"from": owner}) # Test if successful

assert 'MinterAdded' in tx.events. return crowd_sale

That’s kinda cool, we set up two contracts in Brownie, but when will we see some time travel?!

Well before we get started, we have just activated pytests, deployed two contracts in a fixture, added the crowd sale as the minter and asserted there indeed was an AddedMinter event. We can now run the tests and check the code coverage - which is pretty - with the following lines.

brownie test -v

Sample Brownie test output — Not representative of above pseudo code.

brownie test -coverage

Brownie shows you code test coverage — I cut the greenest section out for this post.

Setting a start and end time

So, after 30 years, how much HEX will I get?

Previously I would have either test a tiny time window. Or play with Geth and Ganache settings, write out a long perfectly curated script of test transactions to run, avoiding any prior txns errors, restart the node and do it all over again as I develop… I have even heard of someone - deploying to Ropsten, waiting 24 hours and testing again - checking it actually finalized - before deploying to main-net.

“ok it’s been 24 hours lets make sure it finalised”

Still, better than no testing at all - but far from ideal. Instead, let’s do a test in Brownie, going forwards in time, then going back in time & run a second test.

#/tests/test_crowdsale.py # Test 1

def test_finalise(crowd_sale)

with reverts():

# error: Crowdsale not yet finished

tx = crowd_sale.finalise()



# Travel 25hr into the future!

rpc.sleep(60*60*25)



tx = crowd_sale.finalise({'from':owner})

# Check if finished

assert crowd_sale.finalised() == True # Test 2 - Reset back in time

def test_buy_eth(crowd_sale, test_token):

tx = accounts[3].transfer(crowd_sale, "10 ether") # Can still buy tokens!!!

assert 'Purchased' in tx.events

assert test_token.balanceOf( accounts[3]) == 10 * 10 **18

There are in fact a number of ways to use Brownie to skip through time. https://eth-brownie.readthedocs.io/en/stable/core-rpc.htm

After my initial excitement in Gitter, Ben Hauser, the lead developer of Brownie goes to me…

“You should try it in stateful testing”

Stateful testing is a gamechanger! Essentially you set up some rules, set up some truths — called invariants — and let it run until it breaks. You don't have to map out specific user interactions, it tries a number of combinations. It’s fantastic for testing a bunch of edge cases, finding dust, you name it. But it really gets into its own world when you include time travel, as a rule.

https://eth-brownie.readthedocs.io/en/stable/tests-hypothesis-stateful.html

Invariants at 1.21 gigawatts! Great Scott!

As we approach 1.21 Gigawatts, all you need to gather in the pseudo code below is that there are 4 rules for the contract, and one truth that should hold, called an invariant. The truth being, how much the contract should hold.

#/tests/test_crowdsale.py import brownie

from brownie.test import strategy class StateMachine: eth = strategy('uint256', max_value = "10 ether")

sleep = strategy('uint256', max_value = 60000)

addr = strategy('address') def __init__(cls, accounts, contract):

# Initialise the contract

cls.accounts = accounts

cls.contract = contract def setup(self):

# Reset each run

self.contributed_usd = 0

self.hardcap_usd = 3000000 def rule_1_buyEth(self,eth):

# Purchase in ETH calcs def rule_2_buyUsd(self,usd):

# Purchase in dollars def rule_3_sleep(self,sleep):

rpc.sleep(st_sleep) def rule_4_final(self, addr):

self.contract.finalised() def invariant(self):

# compare the contract amount with the local state

assert self.contract.contributedUsd() == self.contributed_usd



In the reference implementation I go through the logic for each of the rules. I share the rule_3_sleep() as it is simple, yet a powerful addition to the testing. What this means is, as the state machine is going through all the possible combinations, one option is for it to skip ahead in time, eventually ending the crowdsale. If there is some combination of buying, skipping through time or trying to finalize early that causes an error, it will show the exact sequence of steps taken and accounts used to recreate the error.

Then next step would be to wrap the state_machine in a function, just as you would in a regular test, and let hypothesis do its magic!

#/tests/test_crowdsale.py def test_state(crowd_sale, accounts, state_machine): # This is how many steps and examples you want to try

settings = {"stateful_step_count": 20

, "max_examples": 50} state_machine( StateMachine

, accounts[0:3]

, crowd_sale

, settings=settings)

It will then start to process a bunch of permutations, mixing different rules and checking if it broke the invariant. The moment I first got it running it had a bug and showed a step by step, each transaction how it broke the invariant.

You can skip property tests and only run stateful tests with >brownie test— stateful true

Another cool example of a state machine using hypothesis is this: Solving the 3l jug puzzle from Die Hard 3.

Combining both property based testing and state based testing is crucial for complex code that deals with state change over time. Now 30% better with Brownie, pytest, hypothesis and time travel!

State based testing is a powerful tool essential for developers to think about as part of their testing/CI, as well as for preparing for, and running security audits on smart contract code.

If you’re going to deploy on mainnet, please do some testing and share your test suite before locking up other peoples money. Users will both have increased confidence in your contracts, and best case, developers will highlight errors in your test suite.