Load config from file & environment variables in Golang with Viper
Tags: go, tutorial, beginners, webdev
Published: 2020-11-02 16:45:29

When developing and deploying a backend web application, we usually have to use different configurations for different environments such as development, testing, staging, and production.

Today we will learn how to use [Viper](https://github.com/spf13/viper) to load configurations from file or environment variables.

{% youtube n5p8HkO6bnE %}

Here's:

- Link to the [full series playlist on Youtube](https://bit.ly/backendmaster)
- And its [Github repository](https://github.com/techschool/simplebank)

## Why file and environment variables

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2bh5o1lqfxkzapnzpl4s.png)

Reading values from file allows us to easily specify default configuration for local development and testing.

While reading values from environment variables will help us override the default settings when deploying our application to staging or production using docker containers.

## Why Viper

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/lfgy1v72y2agoib2j4ge.png)

[Viper](https://github.com/spf13/viper) is a very popular Golang package for this purpose.

- It can find, load, and unmarshal values from a config file.
- It supports many types of files, such as JSON, TOML, YAML, ENV, or INI.
- It can also read values from environment variables or command-line flags.
- It gives us the ability to set or override default values.
- Moreover, if you prefer to store your settings in a remote system such as [Etcd](https://etcd.io/) or [Consul](https://www.consul.io/), then you can use viper to read data from them directly.
- It works for both unencrypted and encrypted values.
- One more interesting thing about Viper is, it can watch for changes in the config file, and notify the application about it.
- We can also use viper to save any config modification we made to the file.

A lot of useful features, right?

## What will we do

In the current code of our simple bank project, we’re hard-coding some constants for the `dbDriver`, `dbSource` in the `main_test.go` file, and also one more constant for the `serverAddress` in the `main.go` file.

```go
const (
    dbDriver      = "postgres"
    dbSource      = "postgresql://root:secret@localhost:5432/simple_bank?sslmode=disable"
    serverAddress = "0.0.0.0:8080"
)
```

So in this tutorial, we will learn how to use Viper to read these configurations from file and environment variables.

## Install Viper

OK, let’s start by installing Viper! Open your browser and search for `golang viper`.

Then open its [Github page](https://github.com/spf13/viper). Scroll down a bit, copy this `go get` command and run it in the terminal to install the package:

```bash
❯ go get github.com/spf13/viper
```

After this, in the `go.mod` file of [our project](https://github.com/techschool/simplebank), we can see Viper has been added as a dependency.

## Create config file

Now I’m gonna create a new file `app.env` to store our config values for development. Then let’s copy these variables from `main.go` file and paste them to this config file. Since we’re using dot env format, we must change the way we declare these variables. It should be similar to how we declare environment variables:

- Each variable should be declared on a separate line.
- The variable's name is uppercase and its words are separated by an underscore.
- The variable value is followed by the name after an equal symbol.

```env
DB_DRIVER=postgres
DB_SOURCE=postgresql://root:secret@localhost:5432/simple_bank?sslmode=disable
SERVER_ADDRESS=0.0.0.0:8080
```

And that’s it! This `app.env` file is now containing the default configurations for our local development environment. Next, we will use Viper to load this config file.

## Load config file

Let’s create a new file `config.go` inside the `util` package.

Then declare a new type `Config` struct in this file. This `Config` struct will hold all configuration variables of the application that we read from file or environment variables. For now, we only have 3 variables:

- First, the `DBDriver` of type string,
- Second, the `DBSource`, also of type string.
- And finally the `ServerAddress` of type string as well.

```go
type Config struct {
    DBDriver      string `mapstructure:"DB_DRIVER"`
    DBSource      string `mapstructure:"DB_SOURCE"`
    ServerAddress string `mapstructure:"SERVER_ADDRESS"`
}
```

In order to get the value of the variables and store them in this struct, we need to use the unmarshaling feature of Viper. Viper uses the [mapstructure](https://github.com/mitchellh/mapstructure) package under the hood for unmarshaling values, so we use the `mapstructure` tags to specify the name of each config field.

In our case, we must use the exact name of each variable as being declared in the `app.env`. For example, the `DBDriver`’s tag name should be `DB_DRIVER`, the `DBSource`’s tag name should be `DB_SOURCE`, and similar for the `ServerAddress`, should be `SERVER_ADDRESS`.

OK, next I’m gonna define a new function `LoadConfig()`, which takes a `path` as input, and returns a `config` object or an `error`. This function will read configurations from a config file inside the path if it exists, or override their values with environment variables if they’re provided.

```go
func LoadConfig(path string) (config Config, err error) {
    viper.AddConfigPath(path)
    viper.SetConfigName("app")
    viper.SetConfigType("env")

    ...
}
```

First, we call `viper.AddConfigPath()` to tell Viper the location of the config file. In this case, the location is given by the input path argument.

Next, we call `viper.SetConfigName()` to tell Viper to look for a config file with a specific name. Our config file is `app.env`, so its name is `app`.

We also tell Viper the type of the config file, which is `env` in this case, by calling `viper.SetConfigFile()` and pass in `env`. You can also use JSON, XML or any other format here if you want, just make sure your config file has the correct format and extension.

Now, besides reading configurations from file, we also want viper to read values from environment variables. So we call `viper.AutomaticEnv()` to tell viper to automatically override values that it has read from config file with the values of the corresponding environment variables if they exist.

```go
// LoadConfig reads configuration from file or environment variables.
func LoadConfig(path string) (config Config, err error) {
    viper.AddConfigPath(path)
    viper.SetConfigName("app")
    viper.SetConfigType("env")

    viper.AutomaticEnv()

    err = viper.ReadInConfig()
    if err != nil {
        return
    }

    err = viper.Unmarshal(&config)
    return
}
```

After that, we call `viper.ReadInConfig()` to start reading config values. If `error` is not `nil`, then we simply return it.

Otherwise, we call `viper.Unmarshal()` to unmarshals the values into the target `config` object. And finally just return the `config` object and any error if it occurs.

So basically the load config function is completed. Now we can use it in the `main.go` file.

## Use LoadConfig in the main function

Let’s remove all of the previous hardcoded values. Then in the `main()` function, let’s call `util.LoadConfig()` and pass in `"."` here, which means the current folder because our config file `app.env` is in the same location as this `main.go` file.

If there’s an error, then we just write a fatal log saying cannot load configuration. Else, we just change these variables to `config.DBDriver`, `config.DBSource`, and `config.ServerAdress`.

```go
func main() {
    config, err := util.LoadConfig(".")
    if err != nil {
        log.Fatal("cannot load config:", err)
    }

    conn, err := sql.Open(config.DBDriver, config.DBSource)
    if err != nil {
        log.Fatal("cannot connect to db:", err)
    }

    store := db.NewStore(conn)
    server := api.NewServer(store)

    err = server.Start(config.ServerAddress)
    if err != nil {
        log.Fatal("cannot start server:", err)
    }
}
```

And we’re done! Let’s try to run this server.

```bash
❯ make server
go run main.go
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /accounts                 --> github.com/techschool/simplebank/api.(*Server).createAccount-fm (3 handlers)
[GIN-debug] GET    /accounts/:id             --> github.com/techschool/simplebank/api.(*Server).getAccount-fm (3 handlers)
[GIN-debug] GET    /accounts                 --> github.com/techschool/simplebank/api.(*Server).listAccount-fm (3 handlers)
[GIN-debug] Listening and serving HTTP on 0.0.0.0:8080
```

It’s working! The server is listening on `localhost` port `8080`, just like what we specified in the `app.env` file.

Let’s open Postman and send an API request. I’m gonna call the list accounts API.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/litml0g2ppnh6s52v6dh.png)

Oops, we’ve got a `500 Internal Server Error`. It’s because our web server cannot connect to Postgres database on port `5432`. Let’s open the terminal and check if Postgres is running or not:

```bash
❯ docker ps
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/mxselwizllznarp2qmd9.png)

There are no containers running. If we run `docker ps -a`, we can see that Postgres container has been exited. So we have to start it by running:

```bash
❯ docker start postgres12
```

OK, now the database is up and running. Let’s go back to Postman and send the request again.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/jalhvyywwcpodx5p0s7e.png)

This time, it’s successful. We’ve got a list of accounts here. So our code to load configurations from file is working well.

Let’s try overriding those configurations with environment variables. I will set the `SERVER_ADDRESS` variable to `localhost` port `8081` before calling `make server`.

```bash
❯ SERVER_ADDRESS=0.0.0.0:8081 make server
go run main.go
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /accounts                 --> github.com/techschool/simplebank/api.(*Server).createAccount-fm (3 handlers)
[GIN-debug] GET    /accounts/:id             --> github.com/techschool/simplebank/api.(*Server).getAccount-fm (3 handlers)
[GIN-debug] GET    /accounts                 --> github.com/techschool/simplebank/api.(*Server).listAccount-fm (3 handlers)
[GIN-debug] Listening and serving HTTP on 0.0.0.0:8081
```

Here we can see that the server is now listening on port `8081` instead of `8080` as before. Now if we try to send the same API request in Postman to port `8080`, we will get a connection refused error:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/68002w5oekob1ns7qdkh.png)

Only when we change this port to `8081`, then the request will be successful:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/bejjeuf5mnvh7a7vesz4.png)

So we can conclude that Viper has successfully overridden the values it read from the config file with environment variables. That’s very convenient when we want to deploy the application to different environments such as staging or production in the future.

## Use LoadConfig in the test

Now before we finish, let’s update the `main_test.go` file to use the new `LoadConfig()` function.

First, remove all of the hard-coded constants. Then in the `TestMain()` function, we call `util.LoadConfig()`.

But this time, the `main_test.go` file is inside the `db/sqlc` folder, while the `app.env` config file is at the root of the repository, so we must pass in this relative path: `"../.."`. These 2 dots `..` basically mean go to the parent folder.

```go
func TestMain(m *testing.M) {
    config, err := util.LoadConfig("../..")
    if err != nil {
        log.Fatal("cannot load config:", err)
    }

    testDB, err = sql.Open(config.DBDriver, config.DBSource)
    if err != nil {
        log.Fatal("cannot connect to db:", err)
    }

    testQueries = New(testDB)

    os.Exit(m.Run())
}

```

If the error is not `nil`, then we just write a fatal log. Otherwise, we should change these 2 values to `config.DBDriver` and `config.DBSource`.

And that’s it! We’re done. Let’s run the whole package test.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/pite0yu09jpnuj4qjx6i.png)

All passed! And that wraps up this lecture about reading configuration from file and environment variables.

I highly recommend you to check out the documentation and try some other interesting features of viper such as [live watching](https://github.com/spf13/viper#watching-and-re-reading-config-files), or [reading from remote system](https://github.com/spf13/viper#remote-keyvalue-store-support).

Happy coding and I will see you in the next lecture!

---

If you like the article, please [subscribe to our Youtube channel](http://bit.ly/techschoolguru) and [follow us on Twitter](https://twitter.com/techschoolguru) for more tutorials in the future.

---
If you want to join me on my current amazing team at Voodoo, check out [our job openings here](https://bit.ly/voodoojobs). Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.