Golang Web API: Project configuration management using Viper
Tags: go, programming, gin, webdev
Published: 2023-10-15 13:57:13

In this section, I create separate config files (yaml) for the development environment and docker environment, then make a config package to read configuration from yaml files.
To see this session project you can visit [Session 4 GitHub repo URL](https://github.com/naeemaei/golang-clean-web-api/tree/Session04-Viper).

### Create config files
In the first step, we need to make three config files for the docker, production, and development environments in `src/config` path:

#### 1. config-development.yml
```yaml
server:
  internalPort: 5005
  externalPort: 5005
  runMode: debug
logger:
  filePath: ../logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: localhost
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: localhost
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 1440
  refreshTokenExpireDuration: 60

```

#### 2. config-docker.yml
```yml
server:
  internalPort: 5000
  externalPort: 0
  runMode: release
logger:
  filePath: /app/logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: postgres_container
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: redis_container
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 60
  refreshTokenExpireDuration: 60
```

#### 3. config-production.yml
```yml
server:
  internalPort: 5010
  externalPort: 5010
  runMode: release
logger:
  filePath: logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: localhost
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: localhost
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 1440
  refreshTokenExpireDuration: 60

```

#### 4. Config structure in go
All configuration of the project must be placed in `config-{env}.yml` files. For now, server, logger, postgres, cors, and Redis configurations are placed in this file and will be completed later. 
Now it's time to start making `config.go` file to the central point of the project environment the configuration management. this file is placed in `src/config` directory. First, we need to create structs corresponding to the config sections:
```go
type Config struct {
	Server   ServerConfig
	Postgres PostgresConfig
	Redis    RedisConfig
	Password PasswordConfig
	Cors     CorsConfig
	Logger   LoggerConfig
	Otp      OtpConfig
	JWT      JWTConfig
}

type ServerConfig struct {
	InternalPort    string
	ExternalPort    string
	RunMode string
}

type LoggerConfig struct {
	FilePath string
	Encoding string
	Level    string
	Logger   string
}

type PostgresConfig struct {
	Host            string
	Port            string
	User            string
	Password        string
	DbName          string
	SSLMode         string
	MaxIdleConns    int
	MaxOpenConns    int
	ConnMaxLifetime time.Duration
}

type RedisConfig struct {
	Host               string
	Port               string
	Password           string
	Db                 string
	DialTimeout        time.Duration
	ReadTimeout        time.Duration
	WriteTimeout       time.Duration
	IdleCheckFrequency time.Duration
	PoolSize           int
	PoolTimeout        time.Duration
}

type PasswordConfig struct {
	IncludeChars     bool
	IncludeDigits    bool
	MinLength        int
	MaxLength        int
	IncludeUppercase bool
	IncludeLowercase bool
}

type CorsConfig struct {
	AllowOrigins string
}

type OtpConfig struct {
	ExpireTime time.Duration
	Digits     int
	Limiter    time.Duration
}

type JWTConfig struct {
	AccessTokenExpireDuration  time.Duration
	RefreshTokenExpireDuration time.Duration
	Secret                     string
	RefreshSecret              string
}
```

#### 5. Config functions
In this project, we use [viper](https://github.com/spf13/viper) package to manage configurations. So first we should install the Viper package `go get -u github.com/spf13/viper`. Then run `go mod tidy` command to ensure that all imports are satisfied.
At this stage, we need some functions

##### 5.1 Get config file path
this function returns a file path based environment.
```go
func getConfigPath(env string) string {
	if env == "docker" {
		return "/app/config/config-docker"
	} else if env == "production" {
		return "/config/config-production"
	} else {
		return "/config/config-development"
	}
}
```

##### 5.2 Load config file with Viper
After the config file was found, we needed to load the config file with `Viper` based specified file type. 
```go
func LoadConfig(filename string, fileType string) (*viper.Viper, error) {
	v := viper.New()
	v.SetConfigType(fileType)
	v.SetConfigName(filename)
	v.AddConfigPath(".")
	v.AutomaticEnv()

	err := v.ReadInConfig()
	if err != nil {
		log.Printf("Unable to read config: %v", err)
		if _, ok := err.(viper.ConfigFileNotFoundError); ok {
			return nil, errors.New("config file not found")
		}
		return nil, err
	}
	return v, nil
}
```

##### 5.3 Parse Viper config
In this section, we should Unmarshal Viper config to the custom `Config` type declared in section 4. If this instruction was successful means the file is parsed and can be used.
```go
func ParseConfig(v *viper.Viper) (*Config, error) {
	var cfg Config
	err := v.Unmarshal(&cfg)
	if err != nil {
		log.Printf("Unable to parse config: %v", err)
		return nil, err
	}
	return &cfg, nil
}
```

##### 5.4 Get config file from other layers
Finally, we write a function that is exposed to other layers to using config struct. This function returns a pointer of the config struct. Furthermore, some swagger port settings are handled in this function and I will explain about it in the next sections.
```go
func GetConfig() *Config {
	cfgPath := getConfigPath(os.Getenv("APP_ENV"))
	v, err := LoadConfig(cfgPath, "yml")
	if err != nil {
		log.Fatalf("Error in load config %v", err)
	}

	cfg, err := ParseConfig(v)
	envPort := os.Getenv("PORT")
	if envPort != ""{
		cfg.Server.ExternalPort = envPort
		log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
	}else{
		cfg.Server.ExternalPort = cfg.Server.InternalPort
		log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
	}
	if err != nil {
		log.Fatalf("Error in parse config %v", err)
	}

	return cfg
}
```

This is the final version of the `config.go` file:
```go
package config

import (
	"errors"
	"log"
	"os"
	"time"

	"github.com/spf13/viper"
)

type Config struct {
	Server   ServerConfig
	Postgres PostgresConfig
	Redis    RedisConfig
	Password PasswordConfig
	Cors     CorsConfig
	Logger   LoggerConfig
	Otp      OtpConfig
	JWT      JWTConfig
}

type ServerConfig struct {
	InternalPort    string
	ExternalPort    string
	RunMode string
}

type LoggerConfig struct {
	FilePath string
	Encoding string
	Level    string
	Logger   string
}

type PostgresConfig struct {
	Host            string
	Port            string
	User            string
	Password        string
	DbName          string
	SSLMode         string
	MaxIdleConns    int
	MaxOpenConns    int
	ConnMaxLifetime time.Duration
}

type RedisConfig struct {
	Host               string
	Port               string
	Password           string
	Db                 string
	DialTimeout        time.Duration
	ReadTimeout        time.Duration
	WriteTimeout       time.Duration
	IdleCheckFrequency time.Duration
	PoolSize           int
	PoolTimeout        time.Duration
}

type PasswordConfig struct {
	IncludeChars     bool
	IncludeDigits    bool
	MinLength        int
	MaxLength        int
	IncludeUppercase bool
	IncludeLowercase bool
}

type CorsConfig struct {
	AllowOrigins string
}

type OtpConfig struct {
	ExpireTime time.Duration
	Digits     int
	Limiter    time.Duration
}

type JWTConfig struct {
	AccessTokenExpireDuration  time.Duration
	RefreshTokenExpireDuration time.Duration
	Secret                     string
	RefreshSecret              string
}

func GetConfig() *Config {
	cfgPath := getConfigPath(os.Getenv("APP_ENV"))
	v, err := LoadConfig(cfgPath, "yml")
	if err != nil {
		log.Fatalf("Error in load config %v", err)
	}

	cfg, err := ParseConfig(v)
	envPort := os.Getenv("PORT")
	if envPort != ""{
		cfg.Server.ExternalPort = envPort
		log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
	}else{
		cfg.Server.ExternalPort = cfg.Server.InternalPort
		log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
	}
	if err != nil {
		log.Fatalf("Error in parse config %v", err)
	}

	return cfg
}

func ParseConfig(v *viper.Viper) (*Config, error) {
	var cfg Config
	err := v.Unmarshal(&cfg)
	if err != nil {
		log.Printf("Unable to parse config: %v", err)
		return nil, err
	}
	return &cfg, nil
}
func LoadConfig(filename string, fileType string) (*viper.Viper, error) {
	v := viper.New()
	v.SetConfigType(fileType)
	v.SetConfigName(filename)
	v.AddConfigPath(".")
	v.AutomaticEnv()

	err := v.ReadInConfig()
	if err != nil {
		log.Printf("Unable to read config: %v", err)
		if _, ok := err.(viper.ConfigFileNotFoundError); ok {
			return nil, errors.New("config file not found")
		}
		return nil, err
	}
	return v, nil
}

func getConfigPath(env string) string {
	if env == "docker" {
		return "/app/config/config-docker"
	} else if env == "production" {
		return "/config/config-production"
	} else {
		return "/config/config-development"
	}
}

```

In the end, I want to load the config from the config file and read the webserver port from the config instead of hard-coding.
For example, I made some changes in `src/api/api.go` file. First append `cfg := config.GetConfig()` to start of `InitServer` function and then load the webserver port from config `r.Run(fmt.Sprintf(":%s", cfg.Server.Port))`.
This is the final version of `api.go` file:
```go
package api

import (
	"fmt"

	"github.com/gin-gonic/gin"
	"github.com/naeemaei/golang-clean-web-api/api/routers"
	"github.com/naeemaei/golang-clean-web-api/config" // Added
)


func InitServer(){
	cfg := config.GetConfig()  // Added
	r := gin.New()
	r.Use(gin.Logger(), gin.Recovery())

	v1 := r.Group("/api/v1/")
    {
        health := v1.Group("/health")
        routers.Health(health)
    }

	if err := r.Run(fmt.Sprintf(":%s", cfg.Server.InternalPort)); err != nil {
		panic(err)
	} // Added
}
```
Now we can run the project with the `go run cmd/main.go` command from `src` directory.

![Project execution result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zai6r6d2hvu39d25f5h0.png)
In the next article, I will make a simple endpoint and explain how to validate the requests and how works validators in gin.

[Project until here: Configuration management usingViper](https://github.com/naeemaei/golang-clean-web-api/tree/Session04-Viper)