Middleware...for the Client?
Tags: go, middleware
Published: 2021-02-08 10:34:57

---
title: Middleware...for the Client?
published: true
description: Using the middleware concept with Go HTTP Client
cover_image: 
tags: Go, Middleware
---

HTTP middleware is a common concept on the server side, with multiple packages and ecosystems.  These providing common functionality like logging, tracing, authentication, routing, and more.  

At the heart of server-side middleware is a simple interface in the "net/http" package of the standard library:
```go
type Handler interface {
	ServeHTTP(ResponseWriter, *Request)
}
```
Middleware signatures then wrap the handler, to call the next in the chain:
```go
func Middleware(h http.Handler) http.Handler {
    return http.HandleFunc(func(w http.ResponseWriter, r *http.Request) {
        // before processing actions
        h.ServeHTTP(w,r) // process request  
        // after processing actions
    })
}
```

## Client side

For every concern on the server side, a similar concern exists on client side.  Authentication, tracing, metrics, headers, etc end up duplicated for each request.  

```go 
req := http.NewRequest(http.GET, "https://google.com", nil)
// Set standard headers
req.Header.Set("Accept", "application/json")
req.Header.Set("Api-Key", "<some api key>")
req.SetBasicAuth(user, pass)
// Metrics injection
ctx := httptrace.WithClientTrace(context.Background(), newTrace())
// Set timeout
ctx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
// Execute
req = req.WithContext(ctx)
resp, err := http.Do(req)
```

As we look through the request building code, nothing between `NewRequest` and `http.Do` are specific to the endpoint being called.  That means it's more likely to be forgotten or make the logic less readable because it's cluttered with boilerplate.

## Middleware to the rescue

Server side has a single, small interface that is used for loading middleware.  Client side has a similar, but less known, interface [http.RoundTripper](https://pkg.go.dev/net/http#RoundTripper):
```go
type RoundTripper interface {
	RoundTrip(*Request) (*Response, error)
}
```
Using this interface in a similar manner to `http.Handler` gives us the freedom to pull headers and other standard request building operations out of the business logic code. 

Let's see how it works.

### Basic Auth

```go
type BasicAuth struct {
    User string
    Pass string
    Next http.RoundTripper
}

func (b BasicAuth) RoundTrip(r *http.Request) (*http.Response, error) {
    r.SetBasicAuth(b.User, b.Pass)
    return b.Next.RoundTrip(r)
}

client := http.Client{
    Transport: BasicAuth{User: "username", Pass: "password", Next: http.DefaultTransport},
}
```

Now, every request sent by `client` will have the basic auth header set.  

### Chaining Middleware

Now that we have our first middleware, let's make it easier to chain multiple.
```go
type BasicAuth struct {
    User string
    Pass string
    Next http.RoundTripper
}

func (b BasicAuth) RoundTrip(r *http.Request) (*http.Response, error) {
    r.SetBasicAuth(b.User, b.Pass)
    return b.Next.RoundTrip(r)
}

func (b BasicAuth) Register(rt http.RoundTripper) http.RoundTripper {
    b.Next = rt
    return b
}

auth := BasicAuth{User: "username", Pass: "password"}
client := http.Client{
    Transport: auth.Register(http.DefaultTransport),
}
```

## Functional Middleware

While a struct with a method works well for a middleware like basic auth, it's cumbersome for standalone middleware actions.  Setting `req.Header.Set("Accept", "application/json")` doesn't have a need for it. 

Here we can take a lesson from `http.HandlerFunc` on the server side to create our own `RoundTripFunc`:
```go
type RoundTripFunc func(r *http.Request) (*http.Response, error)
func (f RoundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) {
    return f(r)
}
```
I understand if this looks like a strange method definition.  We're defining a method _on a function_ which implements the interface.

With that defined, our accept json middleware is as simple as:
```go
func AcceptJSON(rt http.RoundTripper) http.RoundTripper {
    return RoundTripFunc(func(r *http.Request) (*http.Response, error) {
        r.Header.Set("Accept", "application/json")
        return rt.RoundTrip(r)
    })
}

client := &http.Client{
    Transport: AcceptJSON(http.DefaultTransport),
}
```

### Basic Auth part deux

Revisiting the basic auth example above, the functional approach can remove the need for a defined struct entirely.
```go
func BasicAuth(rt http.RoundTripper, user, pass string) http.RoundTripper {
    return RoundTripFunc(func(r *http.Request) (*http.Response, error) {
        r.SetBasicAuth(user, pass)
        return rt.RoundTrip(r)
    })
}

client := http.Client{
    Transport: BasicAuth(http.DefaultTransport, "username", "password"),
}
```

One level deeper moves the configuration away from the registration.  Note the parallels to the struct we defined earlier:
```go
// Configure the auth, return the registration func
func BasicAuth(user, pass string) func(rt http.RoundTripper) http.RoundTripper {
    // BasicAuth.Register
    return func(rt http.RoundTripper) http.RoundTripper {
        // BasicAuth.RoundTrip
        return RoundTripFunc(func(r *http.Request) (*http.Response, error) {
            r.SetBasicAuth(user, pass)
            return rt.RoundTrip(r)
        })
    }
}

auth := BasicAuth("username", "password")
client2 := http.Client{
    Transport: auth(http.DefaultTransport),
}
```
This is especially useful to reduce boilerplate when integrating on a REST API, whether it's internal or external.  Similarly helpful when building SDKs, keeping the endpoint methods limited to the logic that is specific the endpoint being called. 

## Example Time
We've gone through all of the parts, let's see how it works together.
```go
package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"time"
)

func main() {
	auth := BasicAuth("username", "password")
	client := http.Client{
		Transport: Timing(auth(LogHeaders(http.DefaultTransport))),
	}
	resp, err := client.Get("https://google.com")
	if err != nil {
		fmt.Println("error: ", err)
		return
	}
	fmt.Println("status: ", resp.Status)
	buf, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		fmt.Println("body error: ", err)
		return
	}
	fmt.Println("response body size: ", len(buf))
}

type RoundTripFunc func(r *http.Request) (*http.Response, error)

func (f RoundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) {
	return f(r)
}

func BasicAuth(user, pass string) func(rt http.RoundTripper) http.RoundTripper {
	return func(rt http.RoundTripper) http.RoundTripper {
		return RoundTripFunc(func(r *http.Request) (*http.Response, error) {
			r.SetBasicAuth(user, pass)
			return rt.RoundTrip(r)
		})
	}
}

func LogHeaders(rt http.RoundTripper) http.RoundTripper {
	return RoundTripFunc(func(r *http.Request) (*http.Response, error) {
		fmt.Println("URL:", r.URL, "headers: ", r.Header)
		return rt.RoundTrip(r)
	})
}

func Timing(rt http.RoundTripper) http.RoundTripper {
	return RoundTripFunc(func(r *http.Request) (*http.Response, error) {
		defer func(t time.Time) {
			fmt.Printf("request to %q took %v\n", r.URL, time.Since(t))
		}(time.Now())
		return rt.RoundTrip(r)
	})
}
```

Output:
```
URL: https://google.com headers:  map[Authorization:[Basic dXNlcm5hbWU6cGFzc3dvcmQ=]]
request to "https://google.com" took 215.103005ms
URL: https://www.google.com/ headers:  map[Authorization:[Basic dXNlcm5hbWU6cGFzc3dvcmQ=] Referer:[https://google.com]]
request to "https://www.google.com/" took 156.457554ms
status:  200 OK
response body size:  14219
```

As you can see, the middleware were executed on the original `https://google.com` and the redirect `http://google.com/` request.