Practical unit-testing web client in Go
Tags: go, code, testing
Published: 2021-05-28 11:24:29

---
title: Practical unit-testing web client in Go
published: true
canonical_url: https://devopsian.net/posts/practical-test-web-client-go/
description: Practical web client unit-testing in Go by mocking the server side, with examples from my first open-source project libvault: https://github.com/canidam/libvault
tags: golang, code, testing
keywords: ["golang", "unit-test", "test webserver example", "test http example", "httptest", "practical testing example with go"]
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xh5jnt9k9ixqopg09ske.jpg
---
I've started a Go project, a lightweight Hashicorp Vault[^1] client with no dependencies, and a simple API (for the user). Part of the reason I use no 3rd party modules is, I want to better understand Go internals, structure, and improve my skills. In this post, I'll conduct a *practical* example of how I ended up testing it.

The the project's name is **[libvault](https://github.com/canidam/libvault)**, and it's my *first open-source* project.

Vault is a secret manager service with web API and CLI. Applications can communicate with it through HTTP. I find the CLI easy to use, however, the official Vault library is a bit more complicated; it felt like a swiss-army knife when all I needed was just a simple kitchen knife. I decided to implement a light version, that covers basic functionality while maintaining a simple API.

> I'm not going to cover the basics in this post. The intention of it is to be *practical* and provide *real-life, working* examples. The opinions here are mine. It worked for me, still, it doesn't mean it would work for you.

*I removed ALL the error handling from the code for brevity. Please make sure you handle your errors.*

The Go standard library provides a really good testing package. You can manage without external frameworks or 3rd party packages. In my scenario, I need to test a web client that I wrote. The technique I found useful is *mock testing.* 

> Mock testing is an approach to unit testing that lets you make assertions about how the code under test is interacting with other system modules. In mock testing, the dependencies are replaced with objects that simulate the behaviour of the real ones. ... Such a service can be replaced with a mock object. ~ Wikipedia

I had to choose which component to mock:

1. **Client**
2. **Webserver**

At a high level, mocking the client means creating a new struct that implements the *interface* you are testing (mocking the interface). Then, provide your mocking client to the code under test. A good library with examples is [testify](https://github.com/stretchr/testify#mock-package).

I didn't find it useful for my use case, as I need to mock the server-side. I prefer not to modify any code on the client if I can. This would result in more **reliable tests** for my package. So, I've chosen to go with the second option. Read on for how.

## The httptest package

Go is very friendly to web services; it has a [utility package (httptest)](https://golang.org/pkg/net/http/httptest/) for testing an http server. You can simply start a webserver in your *testing code*.

First I had to fetch **an exact response** from a **real webserver** (Vault server), then I could easily mock it using this package. For example, when querying the `/v1/auth/approle/login` endpoint, the response looks like this:

```json
{
  "request_id": "de7c8097-1a38-50a6-b971-fe1836840e45",
  "lease_id": "",
  "renewable": false,
  ...
  ...
}
```

Now that I have the content, I find it easier to save it to a file (rather than put it inside the code). Go `testing` package has another cool feature:

>The go tool will ignore a directory named "testdata", making it available
to hold ancillary data needed by the tests.[^2]

I created a directory named `testdata` inside my project and placed the JSON content in a file - *approleExample.json*.

## Start the mocking server

I'll start with code, followed by an explanation:

```go
package main

import (
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
)

func TestClientLogin(t *testing.T*) {
	mux := http.NewServeMux()
	ts := httptest.NewServer(mux)
	defer ts.Close()
	
	mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
		// request validation logic
		...
		// read json response
		jsonPayload, _ := ioutil.ReadFile("testdata/approleExample.json")
		w.Header().Set("Content-Type", "application/json")
		w.WriteHeader(200)
		w.Write(jsonPayload)
	})
	
	// initalize new client pointing to the testing server
	client, err := NewClient(ts.URL)
	if err != nil {
		// error handling
	}
	
	// test logic
}
```

If you're familiar with the Go http package, this code is pretty self-explanatory. This is another advantage of using the standard library - *you have one less thing to learn* when you want to contribute.

1. **`mux`** is of `ServeMux` type. Which is *an HTTP request multiplexer*. It matches the URL of each incoming request (`/v1/auth/approle/login`) against a list of registered patterns and calls the handler for the pattern that most closely matches the URL (our mux.HandleFunc function body).

2. **`ts`** is of `Server` type. A Server is an HTTP server listening on a system-chosen port on the local loopback interface, for use in end-to-end HTTP tests. Note that I use `httptest.NewServer` to initialize it.

3. **`mux.HandleFunc(..)`** defines a *path* and an *handler (a function)* to call. The content inside describes our server's response.

4. **`client, err := NewClient(ts.URL)`** creates a new Client (my Vault client), providing it the test server URL to work with.

The elegance of this pattern is, *every* test case has its test webserver with all the relevant configurations. The mocked content, the test logic, etc. are all *implemented inside the test itself.*
This really makes life easier when *debugging, reviewing a test case logic* or coverage.

We can improve it further, make the code more clear and concise. This code includes some boilerplate: creating the mux, the server, and reading the json content. We just need to refactor these elements out (to a `setup()` function, and `readJson(path string)`, for example). Then call this `setup()` function for every test case. I'll leave that to the reader to decide.

## Summary

There are numerous articles about the importance of software testing. *Go makes it easier*. You **should** always write tests for your packages; it has so many advantages to just skip it. However, many people do that and I can assume the reason, which is it doesn't provide any additional functionality to your code. *Don't be one of these people.*

Go standard library provides great tools, and you should use them. Personally, I prefer it over other dependencies.

In this post, I gave a practical example of how you can unit-test a web client by *mocking* it.

The takeaways:

1. **Always** write tests. They are too valuable to give up and very easy to do with Go.
2. **Mock** the part your code connects with, don't mock *your* code. If you write a server, mock the client, and vice-versa. It makes your tests much more reliable.
   1. If you mock a server, get a *real* server response and save it to a file
   2. Do that for every API you would like to mock
3. Use **`testdata`** directory to hold your ancillary data needed by your tests

In the next article, I will cover how to test with a TLS (HTTPS) web server and self-signed certificates, adding more examples. You can also find more examples in the source code.

Feedback and comments are welcome.

[^1]: https://www.vaultproject.io/
[^2]: https://golang.org/pkg/cmd/go/internal/test/