Managing Shared Components in Go Microservices with Fx
Tags: go, microservices, infrastructure
Published: 2020-04-12 12:27:39

---
title: Managing Shared Components in Go Microservices with Fx
published: true
description: How to effectively manage shared code across your services
tags: go, golang, microservices, microservice, infrastructure
series: Dependency Injection in Go using Fx
cover_image: https://i0.wp.com/pmihaylov.com/wp-content/uploads/2020/04/rectangular-white-building-under-blue-sky-1544944-scaled.jpg
canonical_url: https://pmihaylov.com/shared-components-go-microservices
---
In many companies nowadays, microservices is the de facto way of handling service architecture.

Some do it out of necessity as their application has reached a scale where the monolith is a bottleneck. Others, simply like being onboard the hype train.

Whatever the scenario, the decision is often backed by the classical case for adopting microservices, which every junior dev studies extensively before their system design interview.

What gets often neglected, however, is the problems which come with such an approach.
Each of these problems usually demands a sophisticated solution, which raises system complexity.

One such problem is how to reuse the shared infrastructure components in your microservices. Each of your services will probably have a distinct business logic, but it will also come with a big baggage of infrastructure code.

These components usually don't change too much between your services - healthchecks, monitoring configs, logging, standard service configurations, etc.

Fortunately, there is a very elegant solution for this problem for your Go services, which utilises the <a href="https://github.com/uber-go/fx">Fx Framework</a>. It helps you by automatically managing your dependencies, but it can do much more than that as I'll show you in the upcoming sections.

In this article, I will show you how to effectively extract your components into reusable &amp; independent modules which can easily be shared across your Go services.

<!--more-->
<h2>Make Sure You've Covered the Basics First</h2>
I've already covered the basics of Dependency Injection Frameworks and Fx in <a href="https://pmihaylov.com/dependency-injection-go-fx/">this article</a>. In it, I've shared how to extract your code into separate functions, which can be automatically wired using the framework.

In this article, I will continue from where the previous one ended. If you want to get the full picture of what we've been doing, I suggest going through that article first.

Alternatively, if you're already familiar with DI frameworks and are only interested in the problem outlined by this article, then get familiar with the code <a href="https://github.com/preslavmihaylov/fxappexample/tree/v2-fx-example">here.</a> We will use that as a baseline, which we'll refactor.

Supplement your code walkthrough with the linked article, which explains some of the primitives used there.
<h2>What We've Got So Far</h2>
We're starting from <a href="https://github.com/preslavmihaylov/fxappexample/tree/v2-fx-example">this small web application</a>, which simply displays a "Hello World" message when you access <a href="http://localhost:8080">localhost:8080</a>.

The application has some infrastructure code already. What I refer to as infra code is (in this case) reading the standard config file, initialising the logger &amp; serve mux.

It is that part of the application which isn't related to its business logic but is necessary for managing your service properly.

The codebase we're starting from is already in good shape as we've abstracted away the wiring code necessary to wire up your components and their various dependencies.

However, we still have everything bundled in <code>main.go</code>, instead of it being organised into distinct components.

This is what we'll focus on in this article.

As a side-note, have in mind that the code in this example application is purposefully kept simple in order to be easier to digest &amp; follow along.
In production microservices, the components will be much more hairy and contain much more details.

Nevertheless, this article will cover all the concepts you'll need in order to apply them to your production microservices.

This is a trade-off I've made in order to balance readability &amp; completeness.
<h2>Refactoring our Application into Fx Modules</h2>
In order to standardise how components are being shared, we will bundle a component's provided functions into <strong>modules</strong>.

Our modules are logical units which encapsulate the functions provided to Fx.

I will walk you through how to do that in the next sections.

If you're not interested in the process of getting there, but simply want to look at the final result, check out the final code <a href="https://github.com/preslavmihaylov/fxappexample/tree/v3-fx-modules">here</a>.
<h3>Creating our Configuration Module - configfx</h3>
We'll start by refactoring this piece of code, which provides our configuration-handling code to fx.

It contains a bunch of yaml struct definitions, used to parse our standard configuration:

{% gist https://gist.github.com/preslavmihaylov/444ea031a0193247309a38fe4cfee238.js %}

And it continues with the actual provider function which reads the config, parses its contents and is finally provided to fx:

{% gist https://gist.github.com/preslavmihaylov/0774a336dce8cb4d23387764e051f92d.js %}

To decouple this from our <code>main.go</code> file, create a new package called <code>configfx</code> and a go file <code>configfx/configfx.go</code> and move the function there:

{% gist https://gist.github.com/preslavmihaylov/6b47c8664cfbbe3be1a653d81d1bcd07.js %}

Now, in the same file, we will bundle the provided function in an <b>Fx module</b> like this:

{% gist https://gist.github.com/preslavmihaylov/28529625a61a3368b3948ba42a5d3337.js %}

Finally, import the new package in main.go, delete the old code and provide our brand new module:
{% gist https://gist.github.com/preslavmihaylov/dd776bcef83926e267b5f2135d1ffabf.js %}

This final step of bundling the ProvideConfig function in an exported Module variable seems redundant, but it helps in two ways:
<ul>
 	<li>In reality, modules often provide more than one function.For example, in this case, we could easily provide the separate configuration sections in different provider functions.This avoids client code having to import our entire configuration when it only needs one of all the sections.
I haven't done that here to keep the code simple &amp; easier to follow.</li>
 	<li>It makes code more readable and standard by following the convention of exporting a single <strong>Module</strong> variable in all components</li>
</ul>
This covers the configuration section. The rest of the transformations are pretty similar.
<h3>Creating our Logger Module - loggerfx</h3>
This step is pretty similar to the previous one, we'll take our logger initialisation code from <code>main.go</code>:

{% gist https://gist.github.com/preslavmihaylov/d6edb55b33f37bb9e3b66161d4ac10a5.js %}

And extract it into its separate <code>loggerfx</code> package in <code>loggerfx/loggerfx.go</code>:
{% gist https://gist.github.com/preslavmihaylov/51a85bbe4c0fc2f3cb0d1c85bc3dc14f.js %}

Finally, provide the new module in <code>main.go</code>:
{% gist https://gist.github.com/preslavmihaylov/a7faa961d1fa88a0e40f2309e86fbf15.js %}

This transformation was pretty similar to the previous one, so there is nothing too interesting to discuss here.

Let's move on to the more interesting cases.
<h3>Creating our Http Module - httpfx</h3>
This transformation is very simple, but holds the potential to plug-in very interesting things.

Take the provided <strong>http.NewServeMux</strong> function from <code>main.go</code>:
{% gist https://gist.github.com/preslavmihaylov/cbc0ea0577f82b1bdb0cecbdb12fe7dc.js %}

And extract it into a <code>httpfx</code> package in <code>httpfx/httpfx.go</code>:
{% gist https://gist.github.com/preslavmihaylov/7e5722cc336211f50827dd1b947533e7.js %}

This was probably the simplest transformation, but it provides the most interesting opportunities.

In this module, you can plug-in any behaviour, which is standard to your http microservices without the consent of any of your other components.

For example, let's say you have a set of standard http endpoints which should be enabled by default in all your microservices.

A great example is the <code>/health</code> endpoint which should unconditionally return <code>200 OK</code>.

This is your health check endpoint which should be present on all your services.
Perhaps some of your dashboards invoke it for all your services to monitor if your service is alive.

It would be very tedious to make the service developers have to write that code every time.

What's more, you're taking a risk by relying on your developers to do that in each service.
Doing it manually for each service is error-prone.

This is just one example, you could use this package for.

There are many more - adding a standard auth middleware, adding a standard logger for all HTTP requests, providing a similarly configured http client, etc.
<h3>Connecting The Dots - bundlefx</h3>
By extracting all our providers into separate Fx modules, we've achieved a very lean <code>main.go</code> file:
{% gist https://gist.github.com/preslavmihaylov/272c12a8c97fbc5281db5237a8762a0d.js %}

This is already a great place to be, but we can put one final touch to this application, by bundling all the infra components into a single package, called <code>bundlefx</code>:
{% gist https://gist.github.com/preslavmihaylov/9f9547f1cb66d43cad8be89b7ddd9b26.js %}

After we do this, our <code>main.go</code> file is pretty slim:
{% gist https://gist.github.com/preslavmihaylov/8971f10a8e700e16b93b98fa1c71c8c3.js %}

Notice how even the hooks can be extracted in a separate independent module. This allows you to entirely disregard how your server is being boot up or ran.

All you need to do is register the routes you need. <code>bundlefx</code> will take care of the rest.
<h2>How to Leverage This In Your Microservices</h2>
Notice how all our infra code is now in a single module, which we can extract into a separate repository &amp; import is as a library via <code>go modules</code>.

Using this approach, you can now supply this package to all service developers in your team. They simply provide it to Fx and get all the standard machinery your service needs.

What's more, their service source code will now comprise only of that service's business logic, while supporting the necessary infra for your specific environment.

If your needs grow in the future and you need e.g. a new standard endpoint, which you want to ship to all services, simply update the <code>bundlefx</code> library, ask service developers to update the library to latest version and voila.

There are some caveats to look out for, though.

Since you're hiding all the infra details from your service developers, they won't know what options they have available.

This is why, you should make sure that all your standard packages are very well documented, by providing a reference guide to exported objects &amp; examples on how to use them.

Additionally, some might be afraid that bundling all your infra in a single package like this will make your services unnecessarily heavy-loaded as they might not need all of the standard packages <code>bundlefx</code> supports.

This, fortunately, is not the case as Fx only provides those objects which you need. For example, if you never request a <code>*zap.Logger</code> in any of your components, Fx will never invoke the functions in the <code>loggerfx</code> package.
<h2>Conclusion</h2>
Structuring your services into Fx modules enables you to create reusable &amp; independent modules, which can be easily composed together &amp; reused throughout your microservices.

This is especially useful for extracting common infra code into a separate package, reusing it across your services seamlessly.

Doing this, will enable your team's service developers to focus on their business logic, without spending extra development time on wiring infra code across your microservices.

It will also enable you to independently evolve your platform's infrastructure without affecting the existing service code.