Getting started with libp2p in Go
Tags: go, distributedsystems, p2p, libp2p
Published: 2021-02-03 16:20:48

This post gives a brief introduction to developing peer-to-peer applications using libp2p using the Go programming language.

# **Table of Contents**

- [Introduction](#introduction)
  - [What is libp2p?](#what-is-libp2p)
  - [What are peer-to-peer network applications?](#what-are-peertopeer-network-applications)
- [Coding the Node](#coding-the-node)
  - [Creating libp2p hosts](#creating-libp2p-hosts)
  - [Connecting to the node (from another node)](#connecting-to-the-node-from-another-node)
     - [Multiaddress](#multiaddress)
     - [Node ID](#node-id)
     - [Connecting the nodes](#connecting-the-nodes)
- [Sending and Receiving Data](#sending-and-receiving-data)
- [Finding Additional Peers](#finding-additional-peers)
- [Complete Code](#complete-code)
- [Conclusion](#conclusion)

# **Introduction**

This section explains the concepts we are going to see in this post.

### **What is libp2p?**

From the [libp2p docs](https://docs.libp2p.io/introduction/what-is-libp2p/): 

> libp2p is a modular system of protocols, specifications and libraries that enable the development of peer-to-peer network applications.

### **What are peer-to-peer network applications?**

A pure peer-to-peer network application is one in which 
> the machines connected to it act like both as clients and servers, thus sharing their own hardware resources to make the network function.

Instead of clients and servers, machines connected peer-to-peer networks are usually called "nodes".

# **Coding the Node**

### **Creating libp2p hosts**

The code below simply creates a new libp2p host with default options.

```go
package main

import (
	"context"
	"fmt"
	"os"
	"os/signal"
	"syscall"

	"github.com/libp2p/go-libp2p"
)

func main() {
	ctx := context.Background()

	host, err := libp2p.New(ctx)
	if err != nil {
		panic(err)
	}
	defer host.Close()

	fmt.Println(host.Addrs())

	sigCh := make(chan os.Signal)
	signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
	<-sigCh
}
```

Running the code I got the following output:

```
[/ip6/2804:d45:3613:5400:4b34:ed8f:df00:5055/tcp/43937 /ip6/::1/tcp/43937 /ip4/192.168.1.68/tcp/45559 /ip4/127.0.0.1/tcp/45559]
```

We are able to see that libp2p automatically chose IPv4 and IPv6 addresses on all interfaces for the host to listen for connections. By doing this, our node can now act as a server for others to connect to. 

If those address strings look odd, don't worry. We'll dive deeper into node addressing in the next section as we'll need it to connect nodes.

### **Connecting to the node (from another node)**

Before we can connect to the node from the previous section, let's see how node addressing works in libp2p. We'll explore 2 concepts required to connect to a libp2p node: `multiaddr` and node IDs.

---
#### **Multiaddress**

libp2p does a lot to work on top of different network transports (i.e. the technology used to send and receive bits on the wire). That requires a flexible addressing scheme.

The address we saw in the output of the node execution are encoded using `multiaddr` (see the [spec](https://github.com/multiformats/multiaddr)). `multiaddr` allows the encoding of many protocols on top of each other along with their addressing information.

Let's dissect the output of the previous section's node execution:

```
/ip4/127.0.0.1/tcp/45559
```

There are two protocols encoded in this `multiaddr` string: `/ip4/127.0.0.1` which tells us to use the `127.0.0.1` address of the IPv4 protocol and `/tcp/45559` which is telling us to layer (on top of IP) the TCP protocol on port 45559.

#### **Node ID**

libp2p defines the `/p2p` protocol and the addressing part of its `multiaddr` string is the ID of the node we want to connect to. That means the address of a node would be something like:

```
/ip4/127.0.0.1/tcp/3000/p2p/NODE_ID
```

Where `NODE_ID` is the node's ID.

Nodes need to generate a cryptographic key pair in order to secure the connections with other nodes (or peers). 

The node's ID is simply a [multihash](https://github.com/multiformats/multihash) of its public key. 

*That way (besides identifying different nodes) IDs are unique, can be made permanent and provide a way for other nodes to verify the public key sent by another node.*

---

#### **Connecting the nodes**

With all that said, we can get back to writing the code to connect two nodes.

First we'll print the host's addresses and ID:

```go
fmt.Println("Addresses:", host.Addrs())
fmt.Println("ID:", host.ID())
```

Starting the node again we get:

```
Addresses: [/ip4/192.168.1.68/tcp/44511 /ip4/127.0.0.1/tcp/44511 /ip6/2804:d45:3613:5400:4b34:ed8f:df00:5055/tcp/46471 /ip6/::1/tcp/46471]
ID: Qmdfuscj69bwzza5nyC1RCMRkV1aoYjQq2nvDYqUYG8Zoq
```

So the p2p address string for this node would be (I'll be using the IPv4 address):

```
/ip4/127.0.0.1/tcp/44511/p2p/Qmdfuscj69bwzza5nyC1RCMRkV1aoYjQq2nvDYqUYG8Zoq
```

In order to connect to other nodes we can extend our code to accept a peer address as argument and some connection logic:

```go
package main

import (
	"context"
	"flag"
	"fmt"
	"os"
	"os/signal"
	"syscall"

	"github.com/libp2p/go-libp2p"
	"github.com/libp2p/go-libp2p-core/peer"
	"github.com/multiformats/go-multiaddr"
)

func main() {
	// Add -peer-address flag
	peerAddr := flag.String("peer-address", "", "peer address")
	flag.Parse()

	// Create the libp2p host.
	//
	// Note that we are explicitly passing the listen address and restricting it to IPv4 over the
	// loopback interface (127.0.0.1).
	//
	// Setting the TCP port as 0 makes libp2p choose an available port for us.
	// You could, of course, specify one if you like.
	host, err := libp2p.New(context.Background(), libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"))
	if err != nil {
		panic(err)
	}
	defer host.Close()

	// Print this node's addresses and ID
	fmt.Println("Addresses:", host.Addrs())
	fmt.Println("ID:", host.ID())

	// If we received a peer address, we should connect to it.
	if *peerAddr != "" {
		// Parse the multiaddr string.
		peerMA, err := multiaddr.NewMultiaddr(*peerAddr)
		if err != nil {
			panic(err)
		}
		peerAddrInfo, err := peer.AddrInfoFromP2pAddr(peerMA)
		if err != nil {
			panic(err)
		}

		// Connect to the node at the given address.
		if err := host.Connect(context.Background(), *peerAddrInfo); err != nil {
			panic(err)
		}
		fmt.Println("Connected to", peerAddrInfo.String())
	}

	sigCh := make(chan os.Signal)
	signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
	<-sigCh
}
```

# **Sending and Receiving Data**

When we want to send and receive data directly from other peers, we can use a libp2p stream.

Let's make the nodes start a counter for each new connection (inbound and outbound) and send it through a stream every second. At the same time, nodes will keep reading the counters sent on that same stream.

First, we create a function to write data to the stream:

```go
func writeCounter(s network.Stream) {
	var counter uint64

	for {
		<-time.After(time.Second)
		counter++

		err := binary.Write(s, binary.BigEndian, counter)
		if err != nil {
			panic(err)
		}
	}
}
```

After that, we create a function to read data from the stream:

```go
func readCounter(s network.Stream) {
	for {
		var counter uint64

		err := binary.Read(s, binary.BigEndian, &counter)
		if err != nil {
			panic(err)
		}

		fmt.Printf("Received %d from %s\n", counter, s.ID())
	}
}
```

Then we modify the code to do 2 additional things:

- Setup a stream handler using the `SetStreamHandler` function (the handler function is called every time a peer opens a stream)
- Create a new stream using the `NewStream` function after connecting to a peer

After we create the host instance we can set up the stream handler function with the following code:

```go
// This gets called every time a peer connects 
// and opens a stream to this node.
host.SetStreamHandler(protocolID, func(s network.Stream) {
	go writeCounter(s)
	go readCounter(s)
})
```

After we connect to a peer we can open a new stream by doing:

```go
s, err := host.NewStream(
	context.Background(), 
	peerAddrInfo.ID, 
	protocolID,
)
if err != nil {
	panic(err)
}

go writeCounter(s)
go readCounter(s)
```

# **Finding Additional Peers**

Peer-to-peer networks do not require a central server for machines to make a connection. All that is required is the address of one of the nodes in the network. 

But what happens if that node goes offline? We lose our connection.

To prevent that from happening, we want to find and remember the address of additional peers in the network. 

Each node in the network will maintain a list of peers they know. Each node will also announce to the peers they know their own addresses in order to be found by others.

As the last step in this post, let's implement peer discovery.

First, we need a new type to define a method to be called when the discovery service finds peers. 

```go
type discoveryNotifee struct{}

func (n *discoveryNotifee) HandlePeerFound(peerInfo peer.AddrInfo) {
	fmt.Println("found peer", peerInfo.String())
}

```

`HandlePeerFound` will be called by the discovery service every time a peer is found (even if we already knew it).

Next, we create an instance of the discovery service. In this example we are using the mDNS protocol which tries to find peers in the local network.

```go
discoveryService, err := discovery.NewMdnsService(
	context.Background(),
	host,
	time.Second,
	discoveryNamespace,
	)
if err != nil {
	panic(err)
}
defer discoveryService.Close()

discoveryService.RegisterNotifee(&discoveryNotifee{})
```

After we add this piece of code we should be able to start nodes that can connect to other nodes directly and start sending them the counter values. The node will also be periodically searching for peers in the local network and printing their IDs and address.

# **Complete code**

This is the complete final code we developed in this post:

```go
package main

import (
	"context"
	"encoding/binary"
	"flag"
	"fmt"
	"os"
	"os/signal"
	"syscall"
	"time"

	"github.com/libp2p/go-libp2p"
	"github.com/libp2p/go-libp2p-core/host"
	"github.com/libp2p/go-libp2p-core/network"
	"github.com/libp2p/go-libp2p-core/peer"
	"github.com/libp2p/go-libp2p/p2p/discovery"
	"github.com/multiformats/go-multiaddr"
)

const protocolID = "/example/1.0.0"
const discoveryNamespace = "example"

func main() {
	// Add -peer-address flag
	peerAddr := flag.String("peer-address", "", "peer address")
	flag.Parse()

	// Create the libp2p host.
	//
	// Note that we are explicitly passing the listen address and restricting it to IPv4 over the
	// loopback interface (127.0.0.1).
	//
	// Setting the TCP port as 0 makes libp2p choose an available port for us.
	// You could, of course, specify one if you like.
	host, err := libp2p.New(context.Background(), libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"))
	if err != nil {
		panic(err)
	}
	defer host.Close()

	// Print this node's addresses and ID
	fmt.Println("Addresses:", host.Addrs())
	fmt.Println("ID:", host.ID())

	// Setup a stream handler.
	//
	// This gets called every time a peer connects and opens a stream to this node.
	host.SetStreamHandler(protocolID, func(s network.Stream) {
		go writeCounter(s)
		go readCounter(s)
	})

	// Setup peer discovery.
	discoveryService, err := discovery.NewMdnsService(
		context.Background(),
		host,
		time.Second,
		discoveryNamespace,
	)
	if err != nil {
		panic(err)
	}
	defer discoveryService.Close()

	discoveryService.RegisterNotifee(&discoveryNotifee{h: host})

	// If we received a peer address, we should connect to it.
	if *peerAddr != "" {
		// Parse the multiaddr string.
		peerMA, err := multiaddr.NewMultiaddr(*peerAddr)
		if err != nil {
			panic(err)
		}
		peerAddrInfo, err := peer.AddrInfoFromP2pAddr(peerMA)
		if err != nil {
			panic(err)
		}

		// Connect to the node at the given address.
		if err := host.Connect(context.Background(), *peerAddrInfo); err != nil {
			panic(err)
		}
		fmt.Println("Connected to", peerAddrInfo.String())

		// Open a stream with the given peer.
		s, err := host.NewStream(context.Background(), peerAddrInfo.ID, protocolID)
		if err != nil {
			panic(err)
		}

		// Start the write and read threads.
		go writeCounter(s)
		go readCounter(s)
	}

	sigCh := make(chan os.Signal)
	signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
	<-sigCh
}

func writeCounter(s network.Stream) {
	var counter uint64

	for {
		<-time.After(time.Second)
		counter++

		err := binary.Write(s, binary.BigEndian, counter)
		if err != nil {
			panic(err)
		}
	}
}

func readCounter(s network.Stream) {
	for {
		var counter uint64

		err := binary.Read(s, binary.BigEndian, &counter)
		if err != nil {
			panic(err)
		}

		fmt.Printf("Received %d from %s\n", counter, s.ID())
	}
}

type discoveryNotifee struct {
	h host.Host
}

func (n *discoveryNotifee) HandlePeerFound(peerInfo peer.AddrInfo) {
	fmt.Println("found peer", peerInfo.String())
}

```

# **Conclusion**

We developed a basic example to show various features of peer-to-peer networking (direct connections, data streams and peer discovery). I believe these can give you a taste of the types of applications that can be built using this kind of architecture (well-known examples include BitTorrent and the InterPlanetary File System).

And finally, I hope you liked the content and that this gets you a bit more interested in getting started in the peer-to-peer networks world.