Writing server with http package. Understanding Mux and Handler concept by writing own RESTful Mux.
Tags: go, router, mux, webdev
Published: 2022-07-19 19:07:53


All examples of code can be found [here](https://github.com/kliukovkin/go_services_cookbook/tree/main/mux_and_handler).

In this article, we will try to understand two crucial Go concepts - `mux` and `Handler`. But before we start writing any code let us firstly understand what we need to run a simple web service. 

- First of all, we need a **server** itself that will run on some port, listening for the requests and providing responses for those requests.
- The next thing we need is a **router**. This entity is responsible for routing requests to corresponding handlers. In the Go world, servemux is actually an analog of a router.
- The last thing we need is a **handler**. It is responsible for processing a request, executing the business logic of your application, and providing a response for it.

### **Implementing a Handler Interface**

Let’s start our journey with pure Go code, no libraries or frameworks. To run a server, we need to implement Handler interface:

```go
type Handler interface{
   ServeHTTP(ResponseWriter, *Request)
}
```

To achieve that we need to create an empty struct and provide a method for it:

```go
package main

import (
	"fmt"
	"net/http"
)

type handler struct{} 

func (t *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) { 
	fmt.Fprintf(w, "ping %s\n", r.URL.Query().Get("name"))
}

func main() {
	h := &handler{} //1
	http.Handle("/", h) //3
	http.ListenAndServe(":8000", nil) //4

}
```

1. We creating an empty struct that implements `http.Handler` interface
2.  `http.Handle` will register our `handler` for a given pattern, in our case, it is **“/”**. Why pattern and not URI? Because under the hood when your server is running and getting any request - it will find the closest pattern to the request path and dispatch the request to the corresponding handler. That means that if you will try to call '[http://localhost:8000/some/other/path?value=foo](http://localhost:8000/?value=foo)' it will still be dispatched to our registered handler, even if it is registered under the **“/”** pattern.
3. On the last line with `http.ListenAndServe` we starting server on the port 8000. Keep in mind the second argument, which is nil for now, but we will consider it in detail in a few moments.

Let us check how it works using curl:

```bash
❯ curl "localhost:8000?name=foo"
ping foo
```

### **Using handler function**

The next example looks very similar to the previous one, but it is a little bit shorter and easier to develop. We don’t need to implement any interface, just to create a function with has a similar signature as `Handler.ServeHTTP`

```go
package main

import ( 
	"fmt"
	"net/http" 
)

func handler(w http.ResponseWriter, r *http.Request) { 
	fmt.Fprintf(w, "ping %s\n", r.URL.Query().Get("name"))
}

func main() {
	http.HandleFunc("/", handler) 
	http.ListenAndServe(":8000", nil) 
}
```

Notice that under the hood it is just syntax sugar for avoiding the creation of Handler instances on each pattern. `HandleFunc` is an adapter that converts it to a struct with `serveHTTP` method. So instead of this:

```go
type root struct{} 
func (t *root) ServeHTTP(w http.ResponseWriter, r *http.Request) {...}

type home struct{} 
func (t *home) ServeHTTP(w http.ResponseWriter, r *http.Request) {...}

type login struct{} 
func (t *login) ServeHTTP(w http.ResponseWriter, r *http.Request) {...}
//...
http.Handle("/", root)
http.Handle("/home", home)
http.Handle("/login", login)
```

We can just use this approach:

```go
func root(w http.ResponseWriter, r *http.Request) {...}
func home(w http.ResponseWriter, r *http.Request) {...}
func login(w http.ResponseWriter, r *http.Request) {...}
...
http.HandleFunc("/", root)
http.HandleFunc("/home", home)
http.HandleFunc("/login", login)
```

### Creating own ServeMux

Remember we passed `nil` to `http.ListenAndServe`? Well, under the hood http package will use default ServeMux and bind handlers to it with `http.Handle` and `http.HandleFunc` . In production, it is not a good pattern to use default serveMux because it is a global variable thus any package can access it and register a new router or something worse. So let’s create our own serveMux. To do this we will use `http.NewServeMux` function. It returns an instance of ServeMux which also got `Handle` and `HandleFunc` methods.

```go
mux := http.NewServeMux()
mux.HandleFunc("/", handlerFunc)
http.ListenAndServe(":8000", mux)
```

One interesting thing here is that our `mux` is a Handler too. Signature of `http.ListenAndServe` waiting for a Handler as a second argument and after receiving a request our HTTP server will call `serveHTTP` method of our mux which in turn call `serveHTTP` method of registered handlers. 

So, it is not obligate to provide a mux from `http.NewServeMux()` . To understand this let’s create our own instance of a router.

```go
package custom_router

import (
	"fmt"
	"net/http"
)

type router struct{}

func (r *router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	switch req.URL.Path {
	case "/foo":
		fmt.Fprint(w, "here is /foo")
	case "/bar":
		fmt.Fprint(w, "here is /bar")
	case "/baz":
		fmt.Fprint(w, "here is /baz")
	default:
		http.Error(w, "404 Not Found", 404)
	}
}

func main() {
	var r router
	http.ListenAndServe(":8000", &r)
}
```

And let’s check this:

```bash
❯ curl "localhost:8000/foo"
here is /foo
```

Keep in mind that servemux, provided by Go will process each request in a separate goroutine. You may try to implement a custom router acting like a Go mux using goroutines per request.

### Creating own server

What will happen if a customer will call our endpoint which needs to get info from the DB, but DB is not responding for a long time? Will the customer wait for the response? If so, probably it is not so user-friendly API. So, to avoid this situation and provide a response after some time of waiting we may instantiate `http.Server`, which has `ListenAndServe` function. In production, we often need to tune our server, e.g. provide a non-standard logger or set timeouts. 

```go
srv := &http.Server{
   Addr:":8000",
   Handler: mux,
	 // ReadTimeout is the maximum duration for reading the entire
   // request, including the body.
	 ReadTimeout: 5 * time.Second,
	 // WriteTimeout is the maximum duration before timing out
   // writes of the response.
	 WriteTimeout: 2 * time.Second,
}
srv.ListenAndServe()
```

Behaviour of timeouts might seems not obvious. What will happen when the timeout will be exceeded? Will the request be forced to quit or responded to? Let’s create a handler that will sleep for 2 seconds and take a look at how our server will behave with WriteTimeout for 1 second.

```go
func handler(w http.ResponseWriter, r *http.Request) {
	time.Sleep(2 * time.Second)
	fmt.Fprintf(w, "ping %s\n", r.URL.Query().Get("name"))
}

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("/", handler)
	srv := &http.Server{
		Addr:    ":8000",
		Handler: mux,
		WriteTimeout: 1 * time.Second,
	}
	srv.ListenAndServe()
}
```

Now let’s call `curl` with the help of the `time` util to measure how much time will take our request.

```bash
❯ time curl "http://localhost:8000?name=foo"
curl: (52) Empty reply from server
curl "http://localhost:8000?name=foo"  0.00s user 0.01s system 0% cpu 2.022 total
```

We see no reply from the server and request took 2 seconds instead of 1. This is because timeouts are just a mechanism that restricts specific actions after timeout was exceeded. In our case, writing anything to the response was restricted after 1 second has passed. I provided 2 links to awesome articles at the end.

And still, we have an open question: how to force our handler to quit after some period of time?

To achieve that we can simply use `http` method `TimeoutHandler`:

```go
func TimeoutHandler(h Handler, dt time.Duration, msg string) Handler
```

Let’s rewrite our example with timeout handler. Don’t forget to increase time.sleep and timeout for +1 sec each, otherwise there will be still no response:

```go
func handler(w http.ResponseWriter, r *http.Request) {
	time.Sleep(3 * time.Second)
	fmt.Fprintf(w, "ping %s\n", r.URL.Query().Get("name"))
}

func main() {
	mux := http.NewServeMux()
	mux.Handle("/", http.TimeoutHandler(http.HandlerFunc(handler), time.Second * 1, "Timeout"))
	srv := &http.Server{
		Addr:    ":8000",
		Handler: mux,
		WriteTimeout: 2 * time.Second,
	}
	srv.ListenAndServe()
}
```

Now our curl works exactly as we expected:

```bash
❯ time curl "http://localhost:8000?name=foo"
Timeoutcurl "http://localhost:8000?name=foo"  0.01s user 0.01s system 1% cpu 1.022 total
```

### RESTful routing

Servemux provided by Go hasn’t some convenient way to support HTTP methods. Of course we always can achieve the same result using `*http.Request`, which contains all required information about the response, including HTTP method:

```go
package main

import"net/http"

func createUser(w http.ResponseWriter, r *http.Request) {
	if r.Method != http.MethodPost {
		http.Error(w, "Method Not Allowed", 405)
	}
	w.Write([]byte("New user has been created"))
}

func main() {
   mux := http.NewServeMux()
   mux.HandleFunc("/users", createUser)
   http.ListenAndServe(":3000", mux)
}

```

### Custom RESTful router

Now let’s try to make something more interesting. We need our router to be able to register handlers using `handleFunc` method which will accept 3 parameters: 

- `method string,`
- `pattern string,`
- `f func(w http.ResponseWriter, req *http.Request)`

To achieve that we need to write a small bunch of code :)

Let’s start with types. First we need a router itself. It should have a map that will store registered URL pattern(e.g. `/users` ) and all information (or rules) that we want to apply to it:

```go
type urlPattern string

type router struct {
	routes map[urlPattern]routeRules
}

func New() *router {
	return &router{routes: make(map[urlPattern]routeRules)}
}
```

Next let’s define what is a routeRules. In case of REST we want to store registered HTTP methods and related handlers:

```go
type httpMethod string

type routeRules struct {
	methods map[httpMethod]http.Handler
}
```

Now we want our router to has a `HandleFunc` method:

```go
/*
	method - string, e.g. POST, GET, PUT
	pattern - URL path for which we want to register a handler
	f - handler 
*/
func (r *router) HandleFunc(method httpMethod, pattern urlPattern, f func(w http.ResponseWriter, req *http.Request)) {
	rules, exists := r.routes[pattern]
	if !exists {
		rules = routeRules{methods: make(map[httpMethod]http.Handler)}
		r.routes[pattern] = rules
	}
	rules.methods[method] = http.HandlerFunc(f)
}
```

The last thing we need is that our router should implement `Handler` interface. So we need to implement `ServeHTTP(w http.ResponseWriter, req *http.Request)` method:

```go
func (r *router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	// first we will try to find a registered URL pattern
	foundPattern, exists := r.routes[urlPattern(req.URL.Path)]
	if !exists {
		http.NotFound(w, req)
		return
	}
	// next we will try to check if such HTTP method was registered
	handler, exists := foundPattern.methods[httpMethod(req.Method)]
	if !exists {
		notAllowed(w, req, foundPattern)
		return
	}
	// finally we will call registered handler
	handler.ServeHTTP(w, req)
}

// small helper method
func notAllowed(w http.ResponseWriter, req *http.Request, r routeRules) {
	methods := make([]string, 1)
	for k := range r.methods {
		methods = append(methods, string(k))
	}
	w.Header().Set("Allow", strings.Join(methods, " "))
	http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
}
```

And that’s it. Now let’s register simple handler:

```go
func handler(w http.ResponseWriter, req *http.Request) {
	w.Write([]byte("hello"))
}

func main() {
	r := New()
	r.HandleFunc(http.MethodGet, "/test", handler)
	http.ListenAndServe(":8000", r)
}
```

And check how it works:

```bash
❯ curl -X GET -i "http://localhost:8000/test"
HTTP/1.1 200 OK
Date: Wed, 13 Jul 2022 14:24:43 GMT
Content-Length: 5
Content-Type: text/plain; charset=utf-8

hello
```

```bash
❯ curl -X POST -i "http://localhost:8000/test"
HTTP/1.1 405 Method Not Allowed
Allow: GET
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Wed, 13 Jul 2022 14:24:14 GMT
Content-Length: 19

Method Not Allowed
```

Awesome. We just wrote a router that may easily register a handler for specific HTTP method. The full code looks like:

```go
package main

import (
	"net/http"
	"strings"
)

type httpMethod string
type urlPattern string

type routeRules struct {
	methods map[httpMethod]http.Handler
}

type router struct {
	routes map[urlPattern]routeRules
}

func (r *router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	foundRoute, exists := r.routes[urlPattern(req.URL.Path)]
	if !exists {
		http.NotFound(w, req)
		return
	}
	handler, exists := foundRoute.methods[httpMethod(req.Method)]
	if !exists {
		notAllowed(w, req, foundRoute)
		return
	}
	handler.ServeHTTP(w, req)
}

func (r *router) HandleFunc(method httpMethod, pattern urlPattern, f func(w http.ResponseWriter, req *http.Request)) {
	rules, exists := r.routes[pattern]
	if !exists {
		rules = routeRules{methods: make(map[httpMethod]http.Handler)}
		r.routes[pattern] = rules
	}
	rules.methods[method] = http.HandlerFunc(f)
}

func notAllowed(w http.ResponseWriter, req *http.Request, r routeRules) {
	methods := make([]string, 1)
	for k := range r.methods {
		methods = append(methods, string(k))
	}
	w.Header().Set("Allow", strings.Join(methods, " "))
	http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
}

func New() *router {
	return &router{routes: make(map[urlPattern]routeRules)}
}

func handler(w http.ResponseWriter, req *http.Request) {
	w.Write([]byte("hello"))
}

func main() {
	r := New()
	r.HandleFunc(http.MethodGet, "/test", handler)
	http.ListenAndServe(":8000", r)
}
```

Why not use it in prod? Well, because there are several libraries that offer you such features and a bunch more! Restrictions by host header, handling paths with path parameters, query parameters, pattern matching(we implement only exact equals), and a lot more.

## Gorilla mux

One of the most popular libraries for that is [Gorilla/mux](https://github.com/gorilla/mux). 

```bash
❯ go get "github.com/gorilla/mux"
```

Here is a simple example of a GET handler.

```go
func handler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello World!")
}

func main() {
	r := mux.NewRouter()
	r.HandleFunc("/test", handler).Methods("GET")
	http.ListenAndServe(":8000", r)
}
```

Let’s check this endpoint and see the result:

```bash
❯ curl -X GET "http://localhost:8000/test"
Hello World!
```

And if we try to send the POST method we will get 405:

```bash
❯ curl -X POST -i "http://localhost:8000/test"
HTTP/1.1 405 Method Not Allowed
Date: Wed, 13 Jul 2022 12:54:22 GMT
Content-Length: 0
```

Gorilla mux:

[https://github.com/gorilla/mux](https://github.com/gorilla/mux)

Timeouts articles:

[https://ieftimov.com/posts/make-resilient-golang-net-http-servers-using-timeouts-deadlines-context-cancellation/](https://ieftimov.com/posts/make-resilient-golang-net-http-servers-using-timeouts-deadlines-context-cancellation/)

[https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/)