Building Microservices in Go: Graceful Shutdown
Tags: go, microservices
Published: 2021-07-29 21:23:02

When building any _long-term running_ process, like a webserver or a program importing data, we should consider providing a way to gracefully shut it down, the idea behind this is to provide a way to exit the process cleanly, to clean up resources and to properly cancel that said running process.

The steps to support `Graceful Shutdown` in Go consist of two steps:

1. Listen for OS signals, and
1. Handle those signals.

In Go those signals are provided by the [`os/signal` package](https://pkg.go.dev/os/signal). Starting with [Go 1.16](https://mariocarrion.com/2021/02/17/what-is-new-in-go-1-16.html) the way I like to implement _Graceful Shutdown_ is by using `os/signal.NotifyContext`, this function provides an idiomatic way to propagate cancellation when using goroutines, which is usually the case when dealing with long-term running processes.

Keep in mind that depending on how our `main` package is implemented you may need to refactor it, having the `main` function to reach the following objectives:

1. Call a `Parse` function, if needed, like `flag.Parse()`, and
1. Call a `run`-like function.

The `run` function is the one orchestrating all the different types, initializing everything, connecting all the dots and perhaps using explicit _Dependency Injection_, and more importantly it may run a few goroutines to implement the call to `signal.NotifyContext` that in the end is going to handle the logic for implementing _Graceful Shutdown_.

Let's look at some concrete examples.

## Using `signal.NotifyContext`

Starting in Go 1.16 `signal.NotifyContext` is the way I like to recommend when handling signals, this replaces the previous way where a channel was required.

For example having the same `main()`:

```go
func main() {
	errC := run()
	if err := <-errC; err != nil {
		fmt.Println("error", err)
	}
	fmt.Println("exiting...")
}
```

When using `signal.Notify`:

```go
func run() <-chan error {
	errC := make(chan error, 1)

	sc := make(chan os.Signal, 1)

	signal.Notify(sc,
		os.Interrupt,
		syscall.SIGTERM,
		syscall.SIGQUIT)

	go func() {
		defer close(errC)

		fmt.Println("waiting for signal...")

		<-sc

		fmt.Println("signal received")
	}()

	return errC
}
```

And when using `signal.NotifyContext`:

```go
func run() <-chan error {
	errC := make(chan error, 1)

	ctx, stop := signal.NotifyContext(context.Background(),
		os.Interrupt,
		syscall.SIGTERM,
		syscall.SIGQUIT)

	go func() {
		defer func() {
			stop()
			close(errC)
		}()

		fmt.Println("waiting for signal...")

		<-ctx.Done()

		fmt.Println("signal received")
	}()

	return errC
}
```

In practice both of them work achieve the same goal because both of them are meant to listen to signals, however the biggest difference is that `signal.NotifyContext` provides a context `ctx` that could be used for creating more complex propagation rules (like timeouts for example) that we can use to cancel other goroutines, instead of doing more work manually.

## Implementing Graceful Shutdown in HTTP Servers

> The code used for this post is [available on Github](https://github.com/MarioCarrion/todo-api-microservice-example/tree/76509a58cabf95bb8329da4e8bf41986885ba814).

Included in the standard library, in `net/http`, Go includes its own HTTP Server in [`net/http.Server`](https://pkg.go.dev/net/http#Server), this server defines a method called [`Shutdown`](https://pkg.go.dev/net/http#Server.Shutdown) meant to be called when the server is supposed to exit and it shutdowns the server gracefully.

If we use the snippet we defined above we can write our code to handle _Graceful Shutdown_ for HTTP servers in [the following way](https://github.com/MarioCarrion/todo-api-microservice-example/blob/76509a58cabf95bb8329da4e8bf41986885ba814/cmd/rest-server/main.go#L117-L150):

```go
	// ... other code initializing things used by this HTTP server

	go func() {
		<-ctx.Done()

		fmt.Println("Shutdown signal received")

		ctxTimeout, cancel := context.WithTimeout(context.Background(), 5*time.Second)

		defer func() {
			stop()
			cancel()
			close(errC)
		}()

		srv.SetKeepAlivesEnabled(false)

		if err := srv.Shutdown(ctxTimeout); err != nil {
			errC <- err
		}

		fmt.Println("Shutdown completed")
	}()

	go func() {
		fmt.Println("Listening and serving")

		// "ListenAndServe always returns a non-nil error. After Shutdown or Close, the returned error is
		// ErrServerClosed."
		if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
			errC <- err
		}
	}()

	return errC
```

## Conclusion

The goal of implementing _Graceful Shutdowns_ is to allow defining some clean-up steps when dealing with a long-running process, in cases where perhaps we need to commit some database transactions, remove some used files or maybe trigger an event to indicate some other process should take over the subsequent events.