Building a Bitcoin P2P Network Analyzer with Golang and Fiber Framework
Published: 2023-06-07 21:20:31

In the world of Bitcoin, understanding the dynamics of the peer-to-peer (P2P) network is crucial. Monitoring node information, connection metrics, and P2P traffic can provide valuable insights into the network's health, performance, and behavior. In this article, we will explore how to build a Bitcoin P2P network analyzer using Golang and the Fiber framework. We'll cover fetching node information, tracking connection metrics, and exposing relevant information through APIs.

## Prerequisites
To follow along with this tutorial,the necessary dependencies installed:
- Golang 1.17 or later
- SQLite
- Btcd ( a bitcoin node implementation written in golang )
- A running testnet bitcoin node daemon 

Additionally, a basic understanding of Bitcoin, P2P networks, and the Golang Fiber framework will be helpful.

Setting Up the Project:

1. Open a new folder and create a new Go module for our project:

```
go mod init bitcoin-p2p-analyzer

```

2. Install the required dependencies:

```
go get -u github.com/gofiber/fiber/v2
go get -u github.com/lncm/lnd-rpc/v0.10.0/lnrpc
go get -u github.com/btcsuite/btcd
go get -u gorm.io/gorm
go get -u github.com/joho/godotenv

```

I would be using this folder format during the course of this tutorial:

```
📁 app
📁 bitcoin
📁 controllers
📁 db
📁 lightning
📁 services
📁 utils
go.mod
go.sum
main.go 
.env

```
The `app` directory will house the configuration and initialization of the fiber application which will run in the `main.go`. The `bitcoin` and `lightning` folder will contain configuration of our bitcoin and lightning clients respectively. The controllers will contain function handlers for our api endpoints, while the `db` will hold our connection to our sqlite database using gorm. The services and utils fodler will contain helper methods for our application. 
It is important to note that all our implementation will focus on the bitcoin test network(testnet).

## Node Information Monitoring

The first step in our network analyzer is to fetch and monitor node information such as user agent, version, and services. We can use the Lightning Network Daemon (LND) client and Bitcoin Client to retrieve this information. Let us setup our Lightning Client:

```golang
// lightning/client.go


package lightning

import (
	"context"
	"encoding/hex"
	"fmt"
	"io/ioutil"
	"log"
	"os/user"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"gopkg.in/macaroon.v2"

	"github.com/lncm/lnd-rpc/v0.10.0/lnrpc"
)

type rpcCreds map[string]string

func (m rpcCreds) RequireTransportSecurity() bool { return true }
func (m rpcCreds) GetRequestMetadata(_ context.Context, _ ...string) (map[string]string, error) {
	return m, nil
}
func newCreds(bytes []byte) rpcCreds {
	creds := make(map[string]string)
	creds["macaroon"] = hex.EncodeToString(bytes)
	return creds
}

func getClient(hostname string, port int, tlsFile, macaroonFile string) lnrpc.LightningClient {
	macaroonBytes, err := ioutil.ReadFile(macaroonFile)
	if err != nil {
		panic(fmt.Sprintln("Cannot read macaroon file", err))
	}

	mac := &macaroon.Macaroon{}
	if err = mac.UnmarshalBinary(macaroonBytes); err != nil {
		panic(fmt.Sprintln("Cannot unmarshal macaroon", err))
	}

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	transportCredentials, err := credentials.NewClientTLSFromFile(tlsFile, hostname)
	if err != nil {
		panic(err)
	}

	fullHostname := fmt.Sprintf("%s:%d", hostname, port)

	connection, err := grpc.DialContext(ctx, fullHostname, []grpc.DialOption{
		grpc.WithBlock(),
		grpc.WithTransportCredentials(transportCredentials),
		grpc.WithPerRPCCredentials(newCreds(macaroonBytes)),
	}...)
	if err != nil {
		panic(fmt.Errorf("unable to connect to %s: %w", fullHostname, err))
	}

	return lnrpc.NewLightningClient(connection)
}

func Client() lnrpc.LightningClient {
	usr, err := user.Current()
	if err != nil {
		log.Fatal(err)
	}
	homeDir := usr.HomeDir
	lndDir := fmt.Sprintf("%s/app_container/lightning", homeDir)
	var (
		hostname     = "localhost"
		port         = 10009
		tlsFile      = fmt.Sprintf("%s/tls.cert", lndDir)
		macaroonFile = fmt.Sprintf("%s/data/chain/bitcoin/testnet/admin.macaroon", lndDir)
	)

	client := getClient(hostname, port, tlsFile, macaroonFile)

	return client
}


```
The Bitcoin Client:

```golang

package bitcoin

import (
	"log"

	"bitcoin-p2p-analyzer/utils"
	"github.com/btcsuite/btcd/rpcclient"
)

func Client() *rpcclient.Client {

	// Connect to a running Bitcoin Core node via RPC
	connCfg := &rpcclient.ConnConfig{
		Host:         utils.GetEnv("BTC_HOST"),
		User:         utils.GetEnv("BTC_USER"),
		Pass:         utils.GetEnv("BTC_PASS"),
		HTTPPostMode: true,
		DisableTLS:   true,
	}

	client, err := rpcclient.New(connCfg, nil)
	if err != nil {
		log.Fatal("Error connecting to bitcoind:", err)
	}
	// Get the current block count
	blockCount, err := client.GetBlockCount()
	if err != nil {
		log.Println("Error getting block count:", err)

	}

	log.Println("Current block count:", blockCount)
	return client
}

```



Then we go ahead and create our function in the `services` directory that displays our node information:

```golang
// services/lightning.go

package services

import (
	"context"
	"log"

	"bitcoin-p2p-analyzer/lightning"
	"github.com/lncm/lnd-rpc/v0.10.0/lnrpc"
)

type LNodeMetrics struct {
	PubKey      string `json:"pub_key"`
	UserAgent   string `json:"user_agent"`
	Alias       string `json:"alias"`
	NetCapacity int    `json:"network_capacity"`
}

func GetNodeInfo() *LNodeMetrics {

	client := lightning.Client()

	infoReq := &lnrpc.GetInfoRequest{}

	info, err := client.GetInfo(context.Background(), infoReq)

	if err != nil {
		log.Fatalf("Error getting node info: %v", err)
	}

	moreInfo, _ := client.GetNetworkInfo(context.Background(), &lnrpc.NetworkInfoRequest{})

	result := &LNodeMetrics{
		UserAgent:   info.Version,
		Alias:       info.Alias,
		NetCapacity: int(moreInfo.TotalNetworkCapacity),
		PubKey:      info.IdentityPubkey,
	}

	return result
}

```

Bitcoin Node Info:

```golang

// services/bitcoin.go

package services

import (
	"log"
	"math"

	"bitcoin-p2p-analyzer/bitcoin"
	"github.com/btcsuite/btcd/chaincfg/chainhash"
)

type NodeMetrics struct {
	Difficulty    float64     `json:"difficulty"`
	Version       interface{} `json:"version"`
	Chain         string      `json:"chain"`
	Blocks        int32       `json:"no_of_blocks"`
	BestBlockHash string      `json:"bestblockhash"`
	UserAgent     interface{} `json:"user_agent"`
	HashRate      float64     `json:"hash_rate"`
}

func GetInfo() *NodeMetrics {

	client := bitcoin.Client()

	defer client.Shutdown()

	info, err := client.GetBlockChainInfo()

	if err != nil {
		log.Println(err)
	}

	networkInfo, _ := client.GetNetworkInfo()

	lastBlockHash, err := chainhash.NewHashFromStr(info.BestBlockHash)
	if err != nil {
		log.Println(err)
	}

	lastBlock, err := client.GetBlock(lastBlockHash)
	if err != nil {
		log.Println(err)
	}

	timeToFindBlock := lastBlock.Header.Timestamp.Unix() - int64(lastBlock.Header.PrevBlock[len(lastBlock.Header.PrevBlock)-1])
	hashrate := float64(info.Difficulty) / (float64(timeToFindBlock) * math.Pow(2, 32))

	metrics := &NodeMetrics{
		Difficulty:    info.Difficulty,
		Version:       networkInfo.Version,
		Chain:         info.Chain,
		Blocks:        info.Blocks,
		BestBlockHash: info.BestBlockHash,
		UserAgent:     networkInfo.SubVersion,
		HashRate:      hashrate,
	}

	return metrics

}


```

Then we create an api function handler in our controllers folder:

```golang
// controllers/controller.go
package controllers

import (
	"bitcoin-p2p-analyzer/services"
	"github.com/gofiber/fiber/v2"
)

type Response struct {
	Success bool        `json:"success"`
	Data    interface{} `json:"data"`
}

func GetMetrics(c *fiber.Ctx) error {

	type NodeResponse struct {
		Lightning interface{} `json:"lightning"`
		Bitcoin   interface{} `json:"bitcoin"`
	}

	bitcoin := services.GetInfo()
	lightning := services.GetNodeInfo()

	response := &NodeResponse{
		Bitcoin:   bitcoin,
		Lightning: lightning,
	}

	return c.Status(fiber.StatusOK).JSON(response)
}


```





Then we update the `app` directory:

```golang
// app/app.go

package app

import (
	"bitcoin-p2p-analyzer/controllers"
	"github.com/gofiber/fiber/v2"
	"github.com/gofiber/fiber/v2/middleware/cors"
)

func App() *fiber.App {
	app := fiber.New()

	app.Use(cors.New())

	app.Use(cors.New(cors.Config{
		AllowOrigins: "*",
		AllowHeaders: "Origin, Content-Type, Accept",
	}))

	app.Get("/", func(c *fiber.Ctx) error {
		return c.SendString("Hello, World!")
	})

	app.Get("/node-info", controllers.GetMetrics)
	return app
}

```
update the main.go:

```golang
// main.go

package main

import (
	"log"

	"bitcoin-p2p-analyzer/app"
)

func main() {

	err := app.App().Listen("0.0.0.0:1700")

	if err != nil {
		log.Fatal(err)
	}

}

```
Save the file and run the app in your terminal:

```bash
go run main.go
```
Open your postman on the following address `http://127.0.0.1:1700/node-info`

## Connection Metrics Monitoring
Next, we'll focus on tracking connection metrics, such as uptime and churn, for both on-premises servers and EC2 instances running Bitcoin Core (btcd). We would be implementing in a time-series manner where by we can compare the metrics overtime and to get insights on the connection performance. We would use goroutines to scrape the metrics every 3 minutes and save in a sqlite database. We'll use the btcd and LND clients to fetch the necessary information. Here's an example code snippet:

```golang 
// db/database.go

package db

import (
	"log"

	"gorm.io/driver/sqlite"
	"gorm.io/gorm"
)

func DB() *gorm.DB {
	db, err := gorm.Open(sqlite.Open("metrics.db"), &gorm.Config{})
	if err != nil {
		log.Fatalf("Failed to open database: %v", err)
	}

	return db

}
```

```golang
// services/metrics.go
package services

import (
	"context"
	"log"
	"time"

	"bitcoin-p2p-analyzer/bitcoin"
	"bitcoin-p2p-analyzer/db"
	"bitcoin-p2p-analyzer/lightning"
	"bitcoin-p2p-analyzer/utils"
	"github.com/lncm/lnd-rpc/v0.10.0/lnrpc"
)

func ConnectionMetrics() {

	db := db.DB()

	// Create the metrics table if it doesn't already exist
	db.AutoMigrate(&utils.ConnectionMetrics{})

	// Get Bitcoin Client
	bitcoin := bitcoin.Client()

	defer bitcoin.Shutdown()

	// Get Lightning Client
	lnd := lightning.Client()

	for {
		//Get Bitcoin Peer Info

		peerInfo, err := bitcoin.GetPeerInfo()

		if err != nil {
			log.Printf("Failed to fetch btcd peer info: %v", err)
			continue
		}

		infoReq := &lnrpc.GetInfoRequest{}

		lndInfo, err := lnd.GetInfo(context.Background(), infoReq)

		if err != nil {
			log.Printf("Failed to fetch lnd info: %v", err)
			continue
		}

		// Calculate the incoming and outgoing bandwidth for the btcd node
		var btcdBandwidthIn, btcdBandwidthOut uint64
		for _, peer := range peerInfo {
			btcdBandwidthIn += peer.BytesRecv
			btcdBandwidthOut += peer.BytesSent
		}

		metrics := &utils.ConnectionMetrics{
			Timestamp:           time.Now(),
			NumBTCPeers:         int32(len(peerInfo)),
			NumLNDPeers:         int32(lndInfo.NumPeers),
			NumActiveChannels:   int32(lndInfo.NumActiveChannels),
			NumPendingChannels:  int32(lndInfo.NumPendingChannels),
			NumInactiveChannels: int32(lndInfo.NumInactiveChannels),
			BtcdBandwidthIn:     btcdBandwidthIn,
			BtcdBandwidthOut:    btcdBandwidthOut,
			BlockHeight:         int64(lndInfo.BlockHeight),
			BlockHash:           lndInfo.BlockHash,
			BestHeaderAge:       lndInfo.BestHeaderTimestamp,
			SyncedToChain:       lndInfo.SyncedToChain,
		}

		db.Create(&metrics)

		// Wait for 3 minute before fetching the next set of connection metrics
		time.Sleep(time.Minute * 3)
	}

}

```
Fetching Metrics:
To retrieve the stored connection metrics, we can create an API endpoint using the Fiber framework:

```golang
...
//services/metrics.go

func FetchMetrics() []utils.ConnectionMetrics {

	var allMetrics []utils.ConnectionMetrics

	db := db.DB()

	//fetch all metrics
	if err := db.Find(&allMetrics).Error; err != nil {
		log.Fatal(err)
	}

	return allMetrics

}
```

```golang
// controllers/controller.go
...

func GetConnMetrics(c *fiber.Ctx) error {

	metrics := services.FetchMetrics()

	response := &Response{
		Success: true,
		Data:    metrics,
	}

	return c.Status(fiber.StatusOK).JSON(response)
}

```

Then you update the `app.go` with the required endpoints:

```golang

// app/app.go
...
app.Get("/conn-metrics", controllers.GetConnMetrics)

go services.ConnectionMetrics() // run the function as a goroutine

```
Open your postman on `http://127.0.0.1:1700/conn-metrics` and test after 3 minutes.

## Conclusion
In this tutorial, we explored how to build a Bitcoin P2P network analyzer using Golang and the Fiber framework. We learned how to fetch node information, track connection metrics, and expose the relevant information through an API endpoint. By monitoring node information, connection metrics, and P2P traffic, we can gain valuable insights into the behaviour, performance, and bandwidth usage of the Bitcoin network. For example, analysing node information allows us to identify the distribution of different node implementations, such as Bitcoin Core, btcd, or other client software versions. This insight can help us understand the level of network diversity and the adoption rate of software updates within the Bitcoin ecosystem.

Connection metrics provide visibility into the connectivity and network topology of the Bitcoin network. By monitoring connection counts and peer relationships, we can identify well-connected nodes, influential network participants, and potential bottlenecks. This information helps us evaluate the resilience and decentralisation of the network, and it can guide decisions related to optimising node connections for better performance and redundancy.

P2P traffic analysis allows us to examine the flow of data within the Bitcoin network. By studying message types, transaction propagation, and block dissemination, we can gain insights into the efficiency and effectiveness of the network's information dissemination protocols. This analysis can reveal potential delays or inefficiencies in block propagation, highlighting areas for optimisation to enhance the overall network throughput and reduce confirmation times.

Furthermore, by extending the functionality of a Bitcoin P2P network analyser, we can unlock even more valuable insights. For instance, incorporating transaction analysis capabilities can enable the identification of transaction patterns, fee dynamics, and network congestion levels. By visualising these metrics, we can better understand the factors influencing transaction confirmation times and fee market dynamics.

Additionally, integrating data from Lightning Network nodes and channels can provide insights into the growth, liquidity distribution, and routing efficiency of the Layer 2 network. This information can help evaluate the scalability and usability of the Lightning Network, as well as identify areas for improvement.

By leveraging the data collected and analyzed through a Bitcoin network monitoring tool, researchers, developers, and network participants can make informed decisions to enhance the security, efficiency, and scalability of the Bitcoin network.

Note: This article provides a high-level overview of the implementation process. Please refer to the official documentation and relevant libraries' documentation for detailed instructions and best practices.

Happy coding and exploring the fascinating world of Bitcoin P2P networks!