Implement a load balancer in Golang
Tags: go, loadbalancer, roundrobin
Published: 2022-01-01 15:00:22

# Overview
This article is a translation of [Golangでロードバランサーを実装する](https://bmf-tech.com/posts/Golang%E3%81%A7%E3%83%AD%E3%83%BC%E3%83%89%E3%83%90%E3%83%A9%E3%83%B3%E3%82%B5%E3%83%BC%E3%82%92%E5%AE%9F%E8%A3%85%E3%81%99%E3%82%8B)

This article is the 24th day article of [Makuake Advent Calendar 2021](https://adventar.org/calendars/6822).

It is a story of trying to make your own load balancer with Golang that distributes the load with round robin.

# What is a load balancer
A load balancer is a server that has a load balancing function that distributes requests to multiple servers.

![スクリーンショット 2022-01-01 23 05 20](https://user-images.githubusercontent.com/13291041/147852643-0d5a6fab-1d8f-4d60-81f6-cf95091ca974.png)

It is a kind of reverse proxy that increases the availability of services.

There are two main types of load balancers. 

An L7 load balancer that load balances at the application layer and an L4 load balancer that load balances at the transport layer.

In addition to load balancing, the load balancer has the functions of persistence (session maintenance) and health check.

# Type of load balancing
There are different types of load balancing, static and dynamic.

A typical static method is Round Robin, which distributes requests evenly.

A typical dynamic method is Least Connection, which distributes requests to the server with the least number of unprocessed requests.

# Types of persistence
Persistence is a function for maintaining a session between multiple servers to which the load balancer is distributed.

There are two main methods, Source address affinity persistence and Cookie persistence.

Source address affinity persistence is a method to fix the destination server by looking at the source IP address.

Cookie persistence is a method that issues a cookie to maintain a session, looks at the cookie, and fixes the server to which it is distributed.

# Health check type
Health check is a function that the load balancer checks the operating status of the distribution destination server.

An active health check method that checks the health of the load balancer to the server to which it is distributed, and a method that monitors the response to requests from clients.

Active checks can be classified into L3 check, L4 check, and L7 check depending on the protocol used.

# Implementation
Implement the L4 load balancer as a package.

The type of load balancing is round robin, and the health check corresponds to active check and passive check respectively.

Persistence does not support.

The code implemented this time can be found at [github.com/bmf-san/godon](https://github.com/bmf-san/godon).


# Implement a reverse proxy
A load balancer is a type of reverse proxy. Let's start by implementing a simple reverse proxy.

Golang can be easily implemented by using `httputil`.

```golang
package godon

import (
	"log"
	"net/http"
	"net/http/httputil"
)


func Serve() {
	director := func(request *http.Request) {
		request.URL.Scheme = "http"
		request.URL.Host = ":8081"
	}

	rp := &httputil.ReverseProxy{
		Director: director,
	}

	s := http.Server{
		Addr:    ":8080",
		Handler: rp,
	}

	if err := s.ListenAndServe(); err != nil {
		log.Fatal(err.Error())
	}
}
```

I won't explain it here, but I think it's a good idea to read [pkg.go.dev/net/http/httputil#ReverseProxy](https://pkg.go.dev/net/http/httputil#ReverseProxy) carefully.

# Implementation of Config
Since it is a simple load balancer, it does not have complicated settings, but I will implement a setting function that reads the settings from json.

```json
{
	"proxy": {
		"port": "8080"
	},
	"backends": [
		{
			"url": "http://localhost:8081/"
		},
		{
			"url": "http://localhost:8082/"
		},
		{
			"url": "http://localhost:8083/"
		},
		{
			"url": "http://localhost:8084/"
		}
	]
}
```

```golang
// ...

// Config is a configuration.
type Config struct {
	Proxy    Proxy     `json:"proxy"`
	Backends []Backend `json:"backends"`
}

// Proxy is a reverse proxy, and means load balancer.
type Proxy struct {
	Port string `json:"port"`
}

// Backend is servers which load balancer is transferred.
type Backend struct {
	URL    string `json:"url"`
	IsDead bool
	mu     sync.RWMutex
}

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
	// ...
	
	data, err := ioutil.ReadFile("./config.json")
	if err != nil {
		log.Fatal(err.Error())
	}
	json.Unmarshal(data, &cfg)

	// ...
}
```

# Implementation of round robin
Next, we will implement round robin.

All you have to do is distribute the requests evenly to the backend server, and implement it regardless of whether the backend server is alive or dead.


```golang
// ...

var mu sync.Mutex
var idx int = 0

// lbHandler is a handler for loadbalancing
func lbHandler(w http.ResponseWriter, r *http.Request) {
	maxLen := len(cfg.Backends)
	// Round Robin
	mu.Lock()
	currentBackend := cfg.Backends[idx%maxLen]
	targetURL, err := url.Parse(cfg.Backends[idx%maxLen].URL)
	if err != nil {
		log.Fatal(err.Error())
	}
	idx++
	mu.Unlock()
	reverseProxy := httputil.NewSingleHostReverseProxy(targetURL)
	reverseProxy.ServeHTTP(w, r)
}

// ...

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
	data, err := ioutil.ReadFile("./config.json")
	if err != nil {
		log.Fatal(err.Error())
	}
	json.Unmarshal(data, &cfg)

	s := http.Server{
		Addr:    ":" + cfg.Proxy.Port,
		Handler: http.HandlerFunc(lbHandler),
	}
	if err = s.ListenAndServe(); err != nil {
		log.Fatal(err.Error())
	}
}
```

The reason for using `sync.Mutex` is to avoid race conditions caused by multiple Goroutines accessing variables.

You can check the race condition by removing `sync.Mutex`, starting the server with` go run -race server.go`, and requesting from multiple terminals at the same time.

# Implementation of active check
In the implementation so far, the load balancer has the logic to forward the request even to the abnormal backend.

In a real use case, you don't want the request to be forwarded to the anomalous backend, so detect the anomalous backend and try to deviate from the destination.

```golang
// Backend is servers which load balancer is transferred.
type Backend struct {
	URL    string `json:"url"`
	IsDead bool
	mu     sync.RWMutex
}

// SetDead updates the value of IsDead in Backend.
func (backend *Backend) SetDead(b bool) {
	backend.mu.Lock()
	backend.IsDead = b
	backend.mu.Unlock()
}

// GetIsDead returns the value of IsDead in Backend.
func (backend *Backend) GetIsDead() bool {
	backend.mu.RLock()
	isAlive := backend.IsDead
	backend.mu.RUnlock()
	return isAlive
}

var mu sync.Mutex
var idx int = 0

// lbHandler is a handler for loadbalancing
func lbHandler(w http.ResponseWriter, r *http.Request) {
	maxLen := len(cfg.Backends)
	// Round Robin
	mu.Lock()
	currentBackend := cfg.Backends[idx%maxLen]
	if currentBackend.GetIsDead() {
		idx++
	}
	targetURL, err := url.Parse(cfg.Backends[idx%maxLen].URL)
	if err != nil {
		log.Fatal(err.Error())
	}
	idx++
	mu.Unlock()
	reverseProxy := httputil.NewSingleHostReverseProxy(targetURL)
	reverseProxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, e error) {
		// NOTE: It is better to implement retry.
		log.Printf("%v is dead.", targetURL)
		currentBackend.SetDead(true)
		lbHandler(w, r)
	}
	reverseProxy.ServeHTTP(w, r)
}

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
	data, err := ioutil.ReadFile("./config.json")
	if err != nil {
		log.Fatal(err.Error())
	}
	json.Unmarshal(data, &cfg)

	s := http.Server{
		Addr:    ":" + cfg.Proxy.Port,
		Handler: http.HandlerFunc(lbHandler),
	}
	if err = s.ListenAndServe(); err != nil {
		log.Fatal(err.Error())
	}
}
```

Implements a `ErrorHandler` that is called when the load balancer detects an error when forwarding a request to the backend.

In `ErrorHandler`, the backend that does not return a response normally is flagged so that the load balancer forwards the request again.

The load balancer is adjusting its logic so that it does not forward requests to the flagged backend.

# Implementation of passive check
Finally, we will implement a passive check.

Passive checks only monitor the response of the backend server at intervals.

The backend where the anomaly is detected is flagged as it was during the active check.

Below is all the code that has implemented the passive check.

```golang
package godon

import (
	"encoding/json"
	"io/ioutil"
	"log"
	"net"
	"net/http"
	"net/http/httputil"
	"net/url"
	"sync"
	"time"
)

// Config is a configuration.
type Config struct {
	Proxy    Proxy     `json:"proxy"`
	Backends []Backend `json:"backends"`
}

// Proxy is a reverse proxy, and means load balancer.
type Proxy struct {
	Port string `json:"port"`
}

// Backend is servers which load balancer is transferred.
type Backend struct {
	URL    string `json:"url"`
	IsDead bool
	mu     sync.RWMutex
}

// SetDead updates the value of IsDead in Backend.
func (backend *Backend) SetDead(b bool) {
	backend.mu.Lock()
	backend.IsDead = b
	backend.mu.Unlock()
}

// GetIsDead returns the value of IsDead in Backend.
func (backend *Backend) GetIsDead() bool {
	backend.mu.RLock()
	isAlive := backend.IsDead
	backend.mu.RUnlock()
	return isAlive
}

var mu sync.Mutex
var idx int = 0

// lbHandler is a handler for loadbalancing
func lbHandler(w http.ResponseWriter, r *http.Request) {
	maxLen := len(cfg.Backends)
	// Round Robin
	mu.Lock()
	currentBackend := cfg.Backends[idx%maxLen]
	if currentBackend.GetIsDead() {
		idx++
	}
	targetURL, err := url.Parse(cfg.Backends[idx%maxLen].URL)
	if err != nil {
		log.Fatal(err.Error())
	}
	idx++
	mu.Unlock()
	reverseProxy := httputil.NewSingleHostReverseProxy(targetURL)
	reverseProxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, e error) {
		// NOTE: It is better to implement retry.
		log.Printf("%v is dead.", targetURL)
		currentBackend.SetDead(true)
		lbHandler(w, r)
	}
	reverseProxy.ServeHTTP(w, r)
}

// pingBackend checks if the backend is alive.
func isAlive(url *url.URL) bool {
	conn, err := net.DialTimeout("tcp", url.Host, time.Minute*1)
	if err != nil {
		log.Printf("Unreachable to %v, error:", url.Host, err.Error())
		return false
	}
	defer conn.Close()
	return true
}

// healthCheck is a function for healthcheck
func healthCheck() {
	t := time.NewTicker(time.Minute * 1)
	for {
		select {
		case <-t.C:
			for _, backend := range cfg.Backends {
				pingURL, err := url.Parse(backend.URL)
				if err != nil {
					log.Fatal(err.Error())
				}
				isAlive := isAlive(pingURL)
				backend.SetDead(!isAlive)
				msg := "ok"
				if !isAlive {
					msg = "dead"
				}
				log.Printf("%v checked %v by healthcheck", backend.URL, msg)
			}
		}
	}

}

var cfg Config

// Serve serves a loadbalancer.
func Serve() {
	data, err := ioutil.ReadFile("./config.json")
	if err != nil {
		log.Fatal(err.Error())
	}
	json.Unmarshal(data, &cfg)

	go healthCheck()

	s := http.Server{
		Addr:    ":" + cfg.Proxy.Port,
		Handler: http.HandlerFunc(lbHandler),
	}
	if err = s.ListenAndServe(); err != nil {
		log.Fatal(err.Error())
	}
}
```

# Impressions
I haven't been able to implement retries or support persistence, but I think you've found that Golang makes it relatively easy to implement a load balancer.

# References
- [qiita.com - Goでリバースプロキシつくるときにつかえる net/http/httputil.ReverseProxy の紹介](https://qiita.com/convto/items/64e8f090198a4cf7a4fc)
- [kasvith.me - Let's Create a Simple Load Balancer With Go](https://kasvith.me/posts/lets-create-a-simple-lb-go/)
- [dev.to - Build Load Balancer in Go](https://dev.to/b0r/build-load-balancer-in-go-1oo7)
- [en.wikipedia.org - Load_balancing](https://en.wikipedia.org/wiki/Load_balancing_(computing)#Others)
- [www.infraexpert.com - ロードバランサをはじめから](https://www.infraexpert.com/study/study24.html)
- [www.opensquare.co.jp - Module 5 – パーシステンス（Persistence）](https://www.opensquare.co.jp/lmfile/support/document/TraningG/Module-5.pdf)
- [ascii.jp - 知っておきたいロードバランサーの基礎技術](https://ascii.jp/elem/000/000/506/506272/)
- [www.f5.com - ヘルスチェック](https://www.f5.com/ja_jp/services/resources/glossary/health-check)
- [www.rworks.jp - ロードバランサー（LB）とは？仕組みやDNSラウンドロビンとの違いについて解説](https://www.rworks.jp/system/system-column/sys-entry/16305/)
- [docs.nginx.com - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#choosing-a-load-balancing-method)
- [medium.com - Running multiple HTTP servers in Go](https://medium.com/rungo/running-multiple-http-servers-in-go-d15300f4e59f)
- [news.mynavi.jp - ロードバランサーの基本的な役割についてあらためておさらい](https://news.mynavi.jp/techplus/article/load_balancer-1/)
- [github.com - yyyar/gobetween](https://github.com/yyyar/gobetween)
- [github.com - kasvith/simplelb](https://github.com/kasvith/simplelb)
- [github.com - arjunmahishi/loadbalancer-in-go](https://github.com/arjunmahishi/loadbalancer-in-go)
- [github.com - arbazsiddiqui/anabranch](https://github.com/arbazsiddiqui/anabranch)