The Go Microservice Toolkit. For many years, I considered myself a…
By: George Francis Jr
Tags: Grpc, Go, Rest, Microservices

For many years, I considered myself a language-agnostic software developer, as I always prioritized mastering the fundamentals and learning new concepts over “playing favorites” when it comes to programming languages. Over the course of my 15-year career, I’ve written thousands of lines of code across many languages (e.g., Java, Scala, Go, etc.). It wasn’t until I mastered Go that I realized: picking the right language matters. I became a true loyalist; today, it is easily my favorite language. Its simplicity and elegance, coupled with its powerful concurrency paradigm, make it perfect for the next generation of distributed services.

As an ode to my love for the language, I’ve developed a toolkit to help other developers looking to use Go to power microservices.

1. REST + gRPC: Creating a Perfect Marriage

Microservices are usually backed by HTTP or RPC frameworks such as REST and gRPC.

REST comes with the familiarity of an entity-oriented design — a design approach that is a building block of the HTTP protocol. CRUD (Create Read Update Delete) operations define an entity’s set of behaviors. REST APIs use a subset of HTTP methods to perform CRUD operations on an entity that is commonly represented/serialized as JSON.

gRPC is a high-performance RPC framework (as background, RPC APIs allow developers to access distributed procedures — or methods — that are syntactically indistinguishable from centralized ones, hiding the complexity of data serialization/transmission over a network). It offers client, server, and bi-directional streaming.

Under the hood, gRPC uses HTTP/2 (for transport) and protocol buffers (for efficient serialization) to achieve superior performance to that of REST+JSON. It offers first-class support for code generation. The protobuf compiler generates client and server code which promotes rapid application development and reduces the level of effort required to ship a new service.

By marrying REST + gRPC, we can create distributed services that are high-performing, offering clients dual modes of access while preserving the benefits of an entity-oriented design approach.

Below is an example of the above, in which, we begin by defining a gRPC service to place orders in an entity-oriented manner using the protobuf spec. With order as our entity, we need to define RPC methods that correspond to the CRUD operations the service will support. We will add an additional RPC method List to support listing/filtering of existing orders.

order.proto

Next we compile our order.proto using protoc with the necessary Go options.

Compiling order.proto

Running the command above will generate two files: order.pb.go and order_grpc.pb.go . order.pb.go contains a struct for every protobuf message type defined in order.proto.

Order struct (generated code)

order_grpc.pb.go provides client/server code for interacting with the order service. Included in this file is OrderServiceServer — the interface translation of OrderService (for the purpose of the marriage analogy, think of it as the master of ceremonies).

OrderServiceServer interface (generated code)

To get the gRPC server up and running, we will need to implement the OrderServiceServer interface. For this exercise, we can use UnimplementedOrderServiceServer (the bare bones implementation provided in the generated code).

UnimplementedOrderServiceServer (generated code)

The RegisterOrderServiceServer (generated code) method accepts a grpc.Server and the OrderServiceServer interface; this method wraps a grpc.Server around an implementation of our order service interface and must be called before the server’s Serve() method is called. See the below example.

Initializing a gRPC server

With this step, the gRPC order service is up with just a few lines of code. The final step is to develop a REST server. By injecting the OrderServiceServer interface into the REST server, we can make this “marriage” official.

An example REST server that embeds an order service interface

Lastly, update the main method and marry REST + gRPC.

Unifying REST + gRPC servers using a service interface

Now, both gRPC and REST servers are up and running using the same order service implementation. Note that there are several optimizations we can make to the code snippets above as it relates to error handling, concurrency, readability, etc. Make a mental note to resolve those issues later.

As demonstrated above, the gRPC framework provides rich protobuf tooling that promotes rapid application development enabling developers to generate client/server code, including a service interface that can be used to marry gRPC to REST or other HTTP APIs.

2. Concurrency — Goroutines & Channels

Goroutines are functions that execute concurrently with other functions. We can think of them as background processes that do not block the current thread of execution. Behind the scenes, these lightweight threads are multiplexed onto one or more (many:1) OS threads . This allows a Go program to handle millions of goroutines where as the number of futures Java could handle would be limited by the number of available OS threads (as Java threads are 1:1 with OS threads). The caveat to this performance benefit is that Go threads share memory space and access to this memory space must be synchronized (this should sound familiar to Java developers). This is where channels come to rescue us from a free fall into race condition and deadlock hell.

Channels are primitive typed conduits (you can think of them as mailboxes) that allow goroutines to share data back and forth securely without mutexes (locks). A channel read/write blocks the current execution thread until the sender or receiver is ready.

Below are a few common tasks goroutines could be useful for.

Application tasks : running web servers, db connection pools, daemons, API polling, data processing queues

: running web servers, db connection pools, daemons, API polling, data processing queues Request/event tasks : handling incoming HTTP requests, executing expensive subtasks (ex. multiple network calls) to fulfill a request, publishing a new message to Kafka

: handling incoming HTTP requests, executing expensive subtasks (ex. multiple network calls) to fulfill a request, publishing a new message to Kafka Fire & Forget tasks: logging, alerting, metrics

A web server is an application level process that usually has a start / serve method that blocks the current execution thread until the server has finished serving requests. If you are curious how Go’s HTTP server handles requests, checkout the source code (tl;dr, a goroutine is spawned for every incoming HTTP request).

Since grpcServer.Serve() and restServer.Start() are both blocking calls, only one of those calls can be executed in main ’s execution thread. The other must be executed in the background. The start / serve methods of our REST & gRPC servers also return errors that we need to gracefully handle. (A quick tip on this: wrap each server in a struct that exposes an error channel. Call the start/serve method in a goroutine that writes the error to the error channel. This allows us to use select to wait for multiple channel operations to complete).

The below code demonstrates how to optimize the REST & gRPC servers for background processing and channel-based error propagation.

RestServer refactor

GrpcServer

Remember to treat your Go app as an entity. Often times, devs can write solid service level code and then litter their main methods with lots of conditional log.Fatal() statements and other logic that is hard to understand.

Consider creating a struct for the app that encompasses configs, servers, and other application level dependencies. Although Go provides the ability to create multiple init functions, avoid using init when you can. Init functions have a few drawbacks, including that they have empty returns. Specifically, the Go runtime looks for package level functions that have following signature

This means you cannot return values from the init function. If you attempt to initialize a variable and an error occurs you may be forced to panic, exit the application, or write recover logic. Init functions can make the code harder to understand. Instead, explore creating your own custom constructor-like function that creates a new app, performs all the necessary app initialization, and returns the app. If an error can occur in your app initialization process, simply change the return signature of your function to return an app instance and an error.

Below is an optimized version of main which creates a struct for the app, uses select to listen for errors from the REST & gRPC servers, and handles app start/shutdown (including OS termination signals).

main refactor

Before we create or update an order we need to get a pre-authorization approval for the payment method, and we should confirm that the items to be purchased are in stock. Assume that these subtasks can error (fail or timeout) and can be executed independently. There are a few options for handling request level concurrency. We could use standard goroutines and channels, but maybe there is a better option.

Waitgroups allow us to kick off a collection of goroutines and wait for them to finish. A waitGroup could work but this comes with the responsibility of managing the waitGroup counter. ErrGroups are well suited to execute a collection of related subtasks. An errGroup is composed of a collection of goroutines that execute subtasks and handle error propagation. An errGroup waits (blocks) until all subtasks are completed.

Use Context for incoming and outgoing server requests. Context allows request scoped values, deadlines, and cancellation signals to be propagated across clients and servers. Context has a Done() channel that enables goroutines to be notified when the Context has been cancelled, allowing them to exit early and free up system resources. When errgroup.WithContext() is used, the derived context is cancelled on the first encountered subtask error or the first time wait() returns.

In the below example, validateOrder creates an errGroup that spawns off 2 concurrent subtasks, one task to preAuthorizePayment and another task to checkInventory and confirm whether or not all items are in stock. The functions called in both subtasks accept a Context and are able to return early in the event a context cancellation (or request timeout) occurs.

Most warehouses (and fulfillment centers) have order management systems to enable efficient and cost-effective order fulfillment. Similarly, managing concurrency is essential to maintaining the quality of an application. The below example uses a waitgroup and channels to limit the number of orders our warehouse can process at a time.

Waitgroup w/rating limiting

3. Effective Unit Testing

In the early part of my career (the Java days), unit testing reminded me of the portion of vegetables my mom would always put on my dinner plate. As a child, I always ate the good stuff first and then secretly shoveled the veggies into the trash. In other words, unit testing left a bad taste in my mouth. This was primarily because it required teams to get up to speed with new mocking frameworks, creating a steep learning curve with a framework that was often hard to grasp. Not to mention, these mocking frameworks relied on reflection — and as Rob Pike once said, reflection is never clear.

Fortunately, Go, however, has changed my outlook on unit testing. Here are a few testing tips I have learned along the way.

Use pure functions instead of methods. One of the easiest units of code to test is a pure function. A pure function is deterministic and requires no initialization to test. Methods are functions defined on a type (ex. struct). In order to test a method, its parent type must be initialized. See below.

Method vs Pure Function (example)

Create functional dependencies. Any external dependency (DB, web service call, event producer, etc) that a function needs to perform a task can be injected into the function as a parameter. Functions that have embedded dependencies are difficult to test. Developers usually get around this code smell by using a testing framework that has the ability to change (mock) the values of external dependencies at runtime (via reflection). If you take another look at the validateOrder function (in one of the code snippets above), you may notice that the external dependencies preAuthorizePayment and verifyInventory are embedded. This function would be difficult to test. Since Go supports first-class functions — we can fix this by turning validateOrder into a higher order function.

Here is an example test case to tie things together.

Mocking frameworks are useful when used as tools and not crutches. Even though we can mock external dependencies without our 3rd party friends, these frameworks still provide value for the tedious parts of unit testing such as performing test assertions.

when used as tools and not crutches. Even though we can mock external dependencies without our 3rd party friends, these frameworks still provide value for the tedious parts of unit testing such as performing test assertions. Be friendly to your teammates. As Rob Pike said “Clear is better than clever.” I always encourage developers to write code with an audience in mind. Clear code is simple to write, easy to test, and should be easy for developers (and non-developers) to understand.

Enjoy your journey to mastering Go. Leave your comments below. Feel free to reach out to me at george@imprasia.com