#gigablocktestnet

Measuring maximum sustained transaction
throughput on a global network of Bitcoin nodes
4 November 2017

Andrea Suisani,1 Andrew Clifford,1 Andrew Stone,1 Erik Beijnoff,1Peter Rizun,1 Peter Tschipper,1 Alexandra Fedorova,2 Chen Feng,2Victoria Lemieux,2 Stefan Matthews3
1Bitcoin Unlimited, 2 University of British Columbia, 3 nChain

Motivation- Transaction volume
was growing
exponentially- Hitting the “1 MB
block size limit” put alid on growth- Fees have increased
and confirmation
times have become
unreliable- We want to raise thelimit but there arescaling concerns

1 MB

1 MB

1 MBTransactions

1 MB

Motivation- Transaction volume
was growing
exponentially- Hitting the “1 MBblock size limit” put alid on growth- Fees have increased
and confirmation
times have become
unreliable- We want to raise thelimit but there arescaling concerns

1 MB

1 MB

1 MB

$3

1 MB

Scaling concerns
USERS

NETWORK NODES- Simplified payment verification (SPV)
technology is highly scalable- Users can:+ Be there own banks+ Verify their own transactions+ Send payments to any other user
- 4 billion people already have access totechnology to facilitate this (“dumb phone”+ SMS text message)- Network nodes must validate every transaction- 4 billion users x 1 transaction per day =50,000 tx/sec- Network nodes are needed for:+ Mining new blocks+ Serving Merkle-branch proofs to SPV wallets+ Archiving historical blocks+ Some businesses (e.g., payment processing)+ Research/development

Scaling concerns
USERS

NETWORK NODES- Simplified payment verification (SPV)
technology is highly scalable- Users can:+ Be there own banks+ Verify their own transactions+ Send payments to any other user
- 4 billion people already have access totechnology to facilitate this (“dumb phone”+ SMS text message)- Network nodes must validate every transaction- 4 billion users x 1 transaction per day =50,000 tx/sec- Network nodes are needed for:+ Mining new blocks+ Serving Merkle-branch proofs to SPV wallets+ Archiving historical blocks+ Some businesses (e.g., payment processing)+ Research/development

We wanted to measure the maximum sustained throughput of a
global network of bitcoin nodes to see how close we are to achieving
this, and then to identify bottlenecks.

Gigablock Testnet (October 2017 – 18 nodes)

Gigablock Testnet (October 2017 – 18 nodes)

Reference spec

4-core CPU

16 GB RAM

30 Mbps

SSD storage

4-6 miners

12 generators

CPU mining

Python scripts
2-in/2-out TXs

Ramp tests

Ramp tests

Ramp tests

Ramp tests

Mempool acceptancekeeps up

Bottleneck #1: 100 tx/s
Mempool acceptancecannot keep up

Ramp tests

18% coherent(slow propagation)

97% coherent

99% coherent

What caused the bottleneck?
• It wasn’t the CPU• 25% of a 4-core machine at 100tx/s• It was the single-threaded ofthe mempool acceptancecode• Andrew Stone parallelized
mempool acceptance• Now can achieve over 500
tx/sec sustained• Peaks over 10,000 tx/s onstrongest nodes

What caused the bottleneck?
• It wasn’t the CPU• 25% of a 4-core machine at 100tx/s• It was the single-threaded ofthe mempool acceptancecode• Andrew Stone parallelized
mempool acceptance• Now can achieve over 500
tx/sec sustained• Peaks over 10,000 tx/s onstrongest nodes

What caused the bottleneck?
• It wasn’t the CPU• 25% of a 4-core machine at 100tx/s• It was the single-threaded ofthe mempool acceptancecode• Andrew Stone parallelized
mempool acceptance• Now can achieve over 500
tx/sec sustained• Peaks over 10,000 tx/s onstrongest nodes

What caused the bottleneck?
• It wasn’t the CPU• 25% of a 4-core machine at 100tx/s• It was the single-threaded ofthe mempool acceptancecode• Andrew Stone parallelized
mempool acceptance• Now can achieve over 500
tx/sec sustained• Peaks over 10,000 tx/s onstrongest nodes

What caused the bottleneck?
• It wasn’t the CPU• 25% of a 4-core machine at 100tx/s• It was the single-threaded
mempool acceptance codepath• Andrew Stone parallelized
mempool acceptance• Now can achieve over 1,000tx/sec sustained• Bursts over 10,000 tx/s onstrongest nodes

Ramp tests with bottleneck removed

Ramp tests with bottleneck removed

Ramp tests with bottleneck removed

Xthin blockpropagation

Xthin blockpropagation
Linear model:
Propagation time

Block size

Empty-blocktime

Propagation
impedance

Least-squares best fit:

Bottleneck: propagation timecommensurate with 10 min
block time

Xthin blockpropagation
10 minute block time

Linear model:
Propagation time

Block size

Empty-blocktime

Propagation
impedance

Least-squares best fit:

Side note: propagation time does
not depend strongly on network BW

Regressions, interpolations & extrapolations
Regression
coefficient

100 tx/sec(mempoolbottleneck)2000 tx/sec(Visa level)

50,000 tx/sec(global adoption)

CPU

0.01 cores / (tx/sec)

1 core20 cores500 cores

Network

0.03 Mbps / (tx/sec)

3 Mbps60 Mbps

1.5 Gbps

MemoryTBD in Experiment #2: UTXO stress test
Disk IO

Regressions, interpolations & extrapolations
Regression
coefficient

100 tx/sec(mempoolbottleneck)2000 tx/sec(Visa level)

50,000 tx/sec(global adoption)

CPU

0.01 cores / (tx/sec)

1 core20 cores500 cores

Network

0.03 Mbps / (tx/sec)

3 Mbps60 Mbps

1.5 Gbps

MemoryBottlenecks were neither the protocol nor the infrastructure.
The bottlenecks were in the implementation of the protocol.

TBD in Experiment #2: UTXO stress test
Disk IO

My hunch: we can achieve Visa level with 4-core/16GB
machines with better implementations.

Transaction
Processing
Architecture

Locking
Strategy

Simultaneity

#gigablocktestnetThank you!
Funding provided byCode at https://github.com/gandrewstone/BitcoinUnlimited"giga_perf" branch

GTI contact person
peter.rizun@gmail.com