Web Development in Go: Middleware, Templating, Databases & Beyond
Tags: go
Published: 2021-01-28 00:21:58


*This article was originally written by [Ayooluwa Isaiah][1] on the [Honeybadger Developer Blog][2].* 

In the [previous article](https://www.honeybadger.io/blog/go-web-services/) in this series, we had an extensive discussion on the Go
`net/http` package and how it can be used for production-ready web applications. We focused mostly on the routing aspect and other quirks and features of the
`http.ServeMux` type.

This article will close out the discussion on `ServeMux` by demonstrating how middleware functions can be implemented with the default router and introducing other standard library packages that are sure to come in handy when developing web services with Go.

## Middleware in Go

The practice of setting up shared functionality that needs to run for many or all HTTP requests is called _middleware_. Some operations, such as authentication, logging, and cookie validation, are often implemented as middleware functions, which act on a request independently before or after the regular route handlers.

To implement middleware in Go, you need to make sure you have a type that satisfies the [http.Handler](http://golang.org/pkg/net/http/#Handler) interface. Ordinarily, this means that you need to attach a method with the signature
`ServeHTTP(http.ResponseWriter, *http.Request)` to the type. When using this method, any type will satisfy the `http.Handler` interface.

Here's a simple example:

```go
package main

import "net/http"

type helloHandler struct {
	name string
}

func (h helloHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	w.Write([]byte("Hello " + h.name))
}

func main() {
	mux := http.NewServeMux()

	helloJohn := helloHandler{name: "John"}
	mux.Handle("/john", helloJohn)
	http.ListenAndServe(":8080", mux)
}
```

Any request sent to the `/john` route will be passed straight to the
`helloHandler.ServeHTTP` method. You can observe this in action by starting the server and heading to http://localhost:8080/john.

Having to add the `ServeHTTP` method to a custom type every time you want to implement an `http.Handler` would be quite tedious, so the `net/http` package provides the `http.HandlerFunc` type, which allows the use of ordinary functions as HTTP handlers.

All you need to do is ensure that your function has the following signature:
`func(http.ResponseWriter, *http.Request)`; then, convert it to the
`http.HandlerFunc` type.

```go
package main

import "net/http"

func helloJohnHandler(w http.ResponseWriter, r *http.Request) {
	w.Write([]byte("Hello John"))
}

func main() {
	mux := http.NewServeMux()
	mux.Handle("/john", http.HandlerFunc(helloJohnHandler))
	http.ListenAndServe(":8080", mux)
}
```

You can even replace the `mux.Handle` line in the `main` function above with `mux.HandleFunc` and pass the function to it directly. We used this pattern exclusively in the previous article.

```go
func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("/john", helloJohnHandler)
	http.ListenAndServe(":8080", mux)
}
```

At this point, the name is hardcoded into the string, unlike before when we were able to set the name in the `main` function before calling the handler. To remove this limitation, we can put our handler logic into a closure, as shown below:

```go
package main

import "net/http"

func helloHandler(name string) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("Hello " + name))
	})
}

func main() {
	mux := http.NewServeMux()
	mux.Handle("/john", helloHandler("John"))
	http.ListenAndServe(":8080", mux)
}
```

The `helloHandler` function itself does not satisfy the `http.Handler` interface, but it creates and returns an anonymous function that does. This function closes over the `name` parameter, which means it can access it whenever it is called. At this point, the `helloHandler` function can be reused for as many different names as necessary.

So, what does all this have to do with middleware? Well, creating a middleware function is done in the same way as we've seen above. Instead of passing a string to the closure (as in the example), we could pass the next handler in the chain as an argument.

Here's the complete pattern:

```go
func middleware(next http.Handler) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    // Middleware logic goes here...
    next.ServeHTTP(w, r)
  })
}
```

The `middleware` function above accepts a handler and returns a handler. Notice how we're able to make the anonymous function satisfy the `http.Handler` interface by casting it to an `http.HandlerFunc` type. At the end of the anonymous function, control is transferred to the `next` handler by invoking the
`ServeHTTP()` method. If you need to pass values between handlers, such as the ID of an authenticated user, you can use the `http.Request.Context()` method introduced in Go 1.7.

Let's write a middleware function that simply demonstrates this pattern. This function adds a property called `requestTime` to the request object, which is subsequently utilized by `helloHandler` to display the timestamp of a request.

```go
package main

import (
	"context"
	"net/http"
	"time"
)

func requestTime(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		ctx = context.WithValue(ctx, "requestTime", time.Now().Format(time.RFC3339))
		r = r.WithContext(ctx)
		next.ServeHTTP(w, r)
	})
}

func helloHandler(name string) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		responseText := "<h1>Hello " + name + "</h1>"

		if requestTime := r.Context().Value("requestTime"); requestTime != nil {
			if str, ok := requestTime.(string); ok {
				responseText = responseText + "\n<small>Generated at: " + str + "</small>"
			}
		}
		w.Write([]byte(responseText))
	})
}

func main() {
	mux := http.NewServeMux()
	mux.Handle("/john", requestTime(helloHandler("John")))
	http.ListenAndServe(":8080", mux)
}
```

![Screenshot of middleware in action](https://www.honeybadger.io/images/blog/posts/ruby-to-go-4/2hVR3dx.png)

Since our middleware function accepts and returns an `http.Handler` type, it is
possible to create an infinite chain of middleware functions nested inside each
other.

For example,

```go
mux := http.NewServeMux()
mux.Handle("/", middleware1(middleware2(appHandler)))
```

You can use a library like [Alice](https://github.com/justinas/alice) to
transform the above construct to a more readable form such as:

```go
alice.New(middleware1, middleware2).Then(appHandler)
```

## Templating

Although the use of templates has waned with the advent of single-page applications, it remains an important aspect of a complete web development solution.

Go provides two packages for all your templating needs: `text/template` and `html/template`. Both of them have the same interface, but the latter will do some encoding behind the scenes to guard against code injection exploits.

Although Go templates aren't the most expressive out there, they get the job done just fine and can be used for production applications. In fact, it's what Hugo, the popular static site generator, bases its templating system on.

Let's take a quick look at how the `html/template` package may be used to send HTML output as a response to a web request.

### Creating a template

Create an `index.html` file in the same directory as your `main.go` file and add the following code to the file:

```html
<ul>
  {{ range .TodoItems }}
  <li>{{ . }}</li>
  {{ end }}
</ul>
```

Next, add the following code to your `main.go` file:

```go
package main

import (
	"html/template"
	"log"
	"os"
)

func main() {
	t, err := template.ParseFiles("index.html")
	if err != nil {
		log.Fatal(err)
	}

	todos := []string{"Watch TV", "Do homework", "Play games", "Read"}

	err = t.Execute(os.Stdout, todos)
	if err != nil {
		log.Fatal(err)
	}
}
```

If you execute the above program with `go run main.go`. You should see the following output:

```html
<ul>
  <li>Watch TV</li>
  <li>Do homework</li>
  <li>Play games</li>
  <li>Read</li>
</ul>
```

Congratulations! You just created your first Go template. Here's a short explanation of the syntax we used in the template file:

- Go uses double braces (`{{` and `}}`) to delimit data evaluation and control structures (known as _actions_) in templates.
- The `range` action is how we're able to iterate over data structures, such as slices.
- `.` represents the current context. In the `range` action, the current context is the slice of `todos`. Inside the block, `{{ . }}` refers to each element in the slice.

In the `main.go` file, the `template.ParseFiles` method is used to create a new template from one or more files. This template is subsequently executed using the `template.Execute` method; it takes an `io.Writer` and the data, which will be applied to the template.

In the above example, the template is executed to the standard output, but we can execute it to any destination, as long as it satisfies the `io.Writer` interface. For example, if you want to return the output as part of a web request, all you need to do is execute the template to the `ResponseWriter` interface, as shown below.

```go
package main

import (
	"html/template"
	"log"
	"net/http"
)

func main() {
	t, err := template.ParseFiles("index.html")
	if err != nil {
		log.Fatal(err)
	}

	todos := []string{"Watch TV", "Do homework", "Play games", "Read"}

	http.HandleFunc("/todos", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/html")
		err = t.Execute(w, todos)
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
		}
	})
	http.ListenAndServe(":8080", nil)
}
```

![Image of HTML output rendered in Chrome](https://www.honeybadger.io/images/blog/posts/ruby-to-go-4/6tyxOCq.png)

This section is only meant to be a quick intro to Go’s template packages. Make sure to check out the documentation for the
[text/template](https://golang.org/pkg/text/template/) and
[html/template](https://golang.org/pkg/html/template/) if you’re interested in more complex use cases.

If you're not a fan of how Go does its templating, alternatives exist, such as the [Plush library](https://github.com/gobuffalo/plush).

## Working with JSON

If you need to work with JSON objects, you will be pleased to hear that Go's standard library includes everything you need to parse and encode JSON through the `encoding/json` package.

### Default types

When encoding or decoding a JSON object in Go, the following types are used:

- `bool` for JSON booleans,
- `float64` for JSON numbers,
- `string` for JSON strings,
- `nil` for JSON null,
- `map[string]interface{}` for JSON objects, and
- `[]interface{}` for JSON arrays.

### Encoding

To encode a data structure as JSON, the `json.Marshal` function is used. Here's an example:

```go
package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	FirstName string
	LastName  string
	Age       int
	email     string
}

func main() {
	p := Person{
		FirstName: "Abraham",
		LastName:  "Freeman",
		Age:       100,
		email:     "abraham.freeman@hey.com",
	}

	json, err := json.Marshal(p)
	if err != nil {
		fmt.Println(err)
		return
	}

	fmt.Println(string(json))
}
```

In the above program, we have a `Person` struct with four different fields. In the `main` function, an instance of `Person` is created with all the fields initialized. The `json.Marshal` method is then used to convert the `p` structure to JSON. This method returns a slice of bytes or an error, which we have to handle before accessing the JSON data.

To convert a slice of bytes to a string in Go, we need to perform type conversion, as demonstrated above. Running [this program](https://play.golang.org/p/0fDZI2IvI7Q) will produce the following
output:

```plaintext
{"FirstName":"Abraham","LastName":"Freeman","Age":100}
```

As you can see, we get a valid JSON object that can be used in any way we want. Note that the `email` field is left out of the result. This is because it is not exported from the `Person` object by virtue of starting with a lowercase letter.

By default, Go uses the same property names in the struct as field names in the resulting JSON object. However, this can be changed through the use of struct field tags.

```go
type Person struct {
	FirstName string `json:"first_name"`
	LastName  string `json:"last_name"`
	Age       int    `json:"age"`
	email     string `json:"email"`
}
```

The struct field tags above specify that the JSON encoder should map the `FirstName` property in the struct to a `first_name` field in the JSON object and so on. This change in the previous example produces the following output:

```plaintext
{"first_name":"Abraham","last_name":"Freeman","age":100}
```

### Decoding

The `json.Unmarshal` function is used for decoding a JSON object into a Go struct. It has the following signature:

```go
func Unmarshal(data []byte, v interface{}) error
```

It accepts a byte slice of JSON data and a place to store the decoded data. If the decoding is successful, the error returned will be `nil`.

Assuming we have the following JSON object,

```go
json := "{"first_name":"John","last_name":"Smith","age":35, "place_of_birth": "London", gender:"male"}"
```

We can decode it to an instance of the `Person` struct, as shown below:

```go
func main() {
	b := `{"first_name":"John","last_name":"Smith","age":35, "place_of_birth": "London", "gender":"male", "email": "john.smith@hmail.com"}`
	var p Person
	err := json.Unmarshal([]byte(b), &p)
	if err != nil {
		fmt.Println(err)
		return
	}

	fmt.Printf("%+v\n", p)
}
```

And you get the following output:

```plaintext
{FirstName:John LastName:Smith Age:35 email:}
```

`Unmarshal` only decodes fields that are found in the destination type. In this case, `place_of_birth` and `gender` are ignored since they do not map to any struct field in `Person`. This behavior can be leveraged to pick only a few specific fields out of a large JSON object. As before, unexported fields in the destination struct are unaffected even if they have a corresponding field in the JSON object. That's why `email` remains an empty string in the output even though it is present in the JSON object.

## Databases

The `database/sql` package provides a generic interface around SQL (or SQL-like) databases. It must be used in conjunction with a database driver, such as the ones listed [here](https://github.com/golang/go/wiki/SQLDrivers). When importing a database driver, you need to prefix it with an underscore `_` to initialize
it.

For example, here's how to use the [MySQL driver](https://github.com/go-sql-driver/mysql) package with `database/sql`:

```go
import (
	"database/sql"
	_ "github.com/go-sql-driver/mysql"
)
```

Under the hood, the driver registers itself as being available to the `database/sql` package, but it won't be used directly in our code. This helps us reduce dependency on a specific driver so that it can be easily swapped out for another with minimal effort.

### Opening a database connection

To access a database, you need to create a `sql.DB` object, as shown below:

```go
func main() {
	db, err := sql.Open("mysql", "user:password@tcp(127.0.0.1:3306)/hello")
	if err != nil {
		log.Fatal(err)
	}
}
```

The `sql.Open` method prepares the database abstraction for later use. It does not establish a connection to the database or validate the connection parameters. If you want to ensure that the database is available and accessible immediately, use the `db.Ping()` method:

```go
err = db.Ping()
if err != nil {
  log.Fatal(err)
}
```

### Closing a database connection

To close a database connection, you can use `db.Close()`. Normally, you want to `defer` the closing of the database until the function that opened the database connection ends, usually the `main` function:

```go
func main() {
	db, err := sql.Open("mysql", "user:password@tcp(127.0.0.1:3306)/hello")
	if err != nil {
		log.Fatal(err)
	}
  defer db.Close()
}
```

The `sql.DB` object is designed to be long-lived, so you should not open and close it frequently. If you do, you may experience problems, such as poor reuse and sharing of connections, running out of available network resources, or sporadic failures. It's best to pass the `sql.DB` method around or make it available globally and only close it when the program is done accessing that datastore.

### Fetching data from the database

Querying a table can be done in three steps. First, call `db.Query()`. Then, iterate over the rows. Finally, use `rows.Scan()` to extract each row into variables. Here's an example:

```go
var (
	id int
	name string
)

rows, err := db.Query("select id, name from users where id = ?", 1)
if err != nil {
	log.Fatal(err)
}

defer rows.Close()

for rows.Next() {
	err := rows.Scan(&id, &name)
	if err != nil {
		log.Fatal(err)
	}

	log.Println(id, name)
}

err = rows.Err()
if err != nil {
	log.Fatal(err)
}
```

If a query returns a single row, you can use the `db.QueryRow` method instead of `db.Query` and avoid some of the lengthy boilerplate code in the previous code snippet:

```go
var (
	id int
	name string
)

err = db.QueryRow("select id, name from users where id = ?", 1).Scan(&id, &name)
if err != nil {
	log.Fatal(err)
}

fmt.Println(id, name)
```

### NoSQL databases

Go also has good support for NoSQL databases, such as Redis, MongoDB, Cassandra, and the like, but it does not provide a standard interface for working with them. You'll have to rely entirely on the driver package for the specific database. Some examples are listed below.

- https://github.com/go-redis/redis (Redis driver).
- https://github.com/mongodb/mongo-go-driver (MongoDB driver).
- https://github.com/gocql/gocql (Cassandra driver).
- https://github.com/Shopify/sarama (Apache Kafka driver)

## Wrapping up

In this article, we discussed some essential aspects of building web
applications with Go. You should now be able to understand why many Go programmers swear by the standard library. It's very comprehensive and provides most of the tools necessary for a production-ready service.

If you require clarification on anything we've covered here, please send me a message on [Twitter](https://twitter.com/ayisaiah). In the next and final article in this series, we'll discuss the `go` tool and how to use it to tackle common tasks in the course of developing with Go.

Thanks for reading, and happy coding!
[1]: https://www.honeybadger.io/blog/ruby-to-go-4/#authorDetails
[2]: https://www.honeybadger.io/blog/ruby-to-go-4/machine-learning-for-web-developers/