Introduction to the Bitcoin Network Protocol using Python and TCP Sockets
Tags: bitcoin, python, blockchain, protocol
Published: 2019-10-30 07:02:47

Blockchain technology is built around consensus algorithms which allow distributed nodes to share a common ledger. A fundamental dependency of these algorithms is a common network protocol to enable communication between participating nodes. Today, let's write a Python program from scratch to interact with a real Bitcoin node.

This post will assume you're familiar with the fundamentals of blockchain technology. If you aren't, I would recommend checking out the [Bitcoin White Paper](https://bitcoin.org/bitcoin.pdf) by Satoshi Nakamoto.

Bitcoin nodes communicate with each other using the TCP protocol. Nodes will typically listen on port number 8333. For a detailed description of the bitcoin network protocol [check out this resource](https://en.bitcoin.it/wiki/Protocol_documentation).

Today, we are going to write a Python program to connect to a Bitcoin node and fetch the details of a specific transaction. Here is a diagram of the message flow that will be developed.

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/4b8fg9fpn3iyq871rxa1.png)

Before we start coding our program, we must make one point clear. Interacting with a Bitcoin node using raw TCP sockets is reinventing the wheel. This has already been done by python packages such as [python-bitcoinlib](https://github.com/petertodd/python-bitcoinlib).

If you want to write sophisticated applications you should definitely use the correct tool for the job. With that said though, programming with TCP sockets is a great way to improve your low level understanding of a network protocol.

To begin, let's import the dependencies our program will require.

```python
#!/usr/bin/env python

# Filename:                     bitcoin-network-tutorial.py
# Command to run the program:   python bitcoin-network-tutorial.py

# Import dependencies
import socket
import time
import random
import struct
import hashlib
import binascii
```

Let's now define the methods required for constructing the "version" request message. 

```python
# Binary encode the sub-version
def create_sub_version():
    sub_version = "/Satoshi:0.7.2/"
    return b'\x0F' + sub_version.encode()

# Binary encode the network addresses
def create_network_address(ip_address, port):
    network_address = struct.pack('>8s16sH', b'\x01', 
        bytearray.fromhex("00000000000000000000ffff") + socket.inet_aton(ip_address), port)
    return(network_address)

# Create the TCP request object
def create_message(magic, command, payload):
    checksum = hashlib.sha256(hashlib.sha256(payload).digest()).digest()[0:4]
    return(struct.pack('L12sL4s', magic, command.encode(), len(payload), checksum) + payload)

# Create the "version" request payload
def create_payload_version(peer_ip_address):
    version = 60002
    services = 1
    timestamp = int(time.time())
    addr_local = create_network_address("127.0.0.1", 8333)
    addr_peer = create_network_address(peer_ip_address, 8333)
    nonce = random.getrandbits(64)
    start_height = 0
    payload = struct.pack('<LQQ26s26sQ16sL', version, services, timestamp, addr_peer,
                          addr_local, nonce, create_sub_version(), start_height)
    return(payload)
```

The [struct](https://docs.python.org/2/library/struct.html) module is used for packing binary data. The [hashlib](https://docs.python.org/3/library/hashlib.html) module is used for generating message checksums. For a full understanding of the code, you'll need to cross reference the data encoding with the [protocol documentation.](https://en.bitcoin.it/wiki/Protocol_documentation)

Next, let's add a method for creating the "verack" request message. The verack command name is derived from "version acknowledge".

```python
# Create the "verack" request message
def create_message_verack():
    return bytearray.fromhex("f9beb4d976657261636b000000000000000000005df6e0e2")
```

With the mandatory messages out of the way, we may now create our "getdata" method for retrieving the details of a specific transaction.

```python
# Create the "getdata" request payload
def create_payload_getdata(tx_id):
    count = 1
    type = 1
    hash = bytearray.fromhex(tx_id)
    payload = struct.pack('<bb32s', count, type, hash)
    return(payload)
```

Please note that not all nodes will be able to return arbitrary transaction data; some will prune their history to save disk space.

We'll also create a method for printing TCP data to the terminal.

```python
# Print request/response data
def print_response(command, request_data, response_data):
    print("")
    print("Command: " + command)
    print("Request:")
    print(binascii.hexlify(request_data))
    print("Response:")
    print(binascii.hexlify(response_data))
```

We may now add our main method which will connect to a bitcoin node and execute the desired message flow.

```python
if __name__ == '__main__':
    # Set constants
    magic_value = 0xd9b4bef9
    tx_id = "fc57704eff327aecfadb2cf3774edc919ba69aba624b836461ce2be9c00a0c20"
    peer_ip_address = '104.199.184.15'
    peer_tcp_port = 8333
    buffer_size = 1024

    # Create Request Objects
    version_payload = create_payload_version(peer_ip_address)
    version_message = create_message(magic_value, 'version', version_payload)
    verack_message = create_message_verack()
    getdata_payload = create_payload_getdata(tx_id)
    getdata_message = create_message(magic_value, 'getdata', getdata_payload)

    # Establish TCP Connection
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((peer_ip_address, peer_tcp_port))

    # Send message "version"
    s.send(version_message)
    response_data = s.recv(buffer_size)
    print_response("version", version_message, response_data)

    # Send message "verack"
    s.send(verack_message)
    response_data = s.recv(buffer_size)
    print_response("verack", verack_message, response_data)

    # Send message "getdata"
    s.send(getdata_message)
    response_data = s.recv(buffer_size)
    print_response("getdata", getdata_message, response_data)

    # Close the TCP connection
    s.close()
```

We found the IP address of the node using [Bitnodes.](https://bitnodes.earn.com/nodes/) Details of the transaction we elected to query can be found on a [block explorer.](https://www.blockchain.com/btc/tx/fc57704eff327aecfadb2cf3774edc919ba69aba624b836461ce2be9c00a0c20)

Execute the program on a terminal with the command `python bitcoin-network-tutorial.py`. A sample output is provided below.

```
Command: version
Request:
b'f9beb4d976657273696f6e000000000064000000f4de76b762ea00000100000000000000c8c6ae5d00000000010000000000000000000000000000000000ffff68c7b80f208d010000000000000000000000000000000000ffff7f000001208d0f2f736a397699b60f2f5361746f7368693a302e372e322f00000000'
Response:
b'f9beb4d976657273696f6e000000000066000000fe4aee167f1101000d04000000000000c2c6ae5d00000000010000000000000000000000000000000000ffff68c7b80f208d0d040000000000000000000000000000000000000000000000000b63185e17ebcdb3102f5361746f7368693a302e31382e302fbc29090001'

Command: verack
Request:
b'f9beb4d976657261636b000000000000000000005df6e0e2'
Response:
b'f9beb4d976657261636b000000000000000000005df6e0e2'

Command: getdata
Request:
b'f9beb4d9676574646174610000000000220000007b00a9b50101fc57704eff327aecfadb2cf3774edc919ba69aba624b836461ce2be9c00a0c20'
Response:
b'f9beb4d9616c65727400000000000000c0000000d2f50d9ef9beb4d9616c65727400000000000000a80000001bf9aaea60010000000000000000000000ffffff7f00000000ffffff7ffeffff7f01ffffff7f00000000ffffff7f00ffffff7f002f555247454e543a20416c657274206b657920636f6d70726f6d697365642c2075706772616465207265717569726564004630440220653febd6410f470f6bae11cad19c48413becb1ac2c17f908fd0fd53bdc3abd5202206d0e9c96fe88d4a0f01ed9dedae2b6f9e00da94cad0fecaae66ecf689bf71b50'
```

That concludes the tutorial! Stay tuned for more.

#### ULTRA CONFIG GENERATOR

Have you heard of [Ultra Config Generator](/ultra-config-generator/)? If you haven't, I highly recommend you check it out.

We designed the product to allow network engineers to generate and automate network configuration in a highly flexible, efficient and elegant manner. Our customers love the application and I hope that you will too.

Take care until next time!

Ultra Config