How to Design a Basic Logging System in Your Go Application
By: Adeshina H. Hassan
Tags: Programming, Logging, Tracing, Go, Golang

Best Practices for Designing and Storing Golang Logs (Datadog, 2019):

Avoid declaring ‘goroutines’ for logging

There are two reasons to avoid creating your own goroutine s to handle writing logs. First, it can lead to concurrency issues, as duplicates of the logger would attempt to access the same io.Writer . Second, logging libraries usually start goroutine s themselves, managing any concurrency issues internally, and starting your own goroutine s will only interfere.

Write your logs to a file

As you collect logs from your application, it’s recommended you write them to a local file directly (not to a remote file directly). You need to ensure your logs are always available locally and not lost in the network.

Implement a standard logging interface

In a team of developers/engineers, it’s probable to see members using different attribute names to describe the same thing. This can lead to an unresolvable inconsistency in your logs and make it drastic to correlate logs that should form part of the same picture. For example, two developers might log the same error — a missing client name — when handling an upload in different ways. Implementing a standard logging interface is the only way to avoid such a disaster.

Let's look at a practical example. The below struct defined the objects/events we wish to log. Of course, we can add more events to the list:

type LogEvent struct {

id int

message string

}

These variables store our log messages as new events:

var (

invalidArgMessage = LogEvent{1, “Invalid arg: %s”} invalidArgValueMessage = LogEvent{2, “Invalid value for argument: %s: %v”} missingArgMessage = LogEvent{3, “Missing arg: %s”}

)

Standard error messages and init():

// Standard “InvalidArg error messagefunc (m *MainLogger) InvalidArg(argumentName string){

m.Errorf(invalidArgMessage.message, argumentName)

} // Standard “InvalidArgValue error message

func (m *MainLogger) InvalidArgValue(argumentName string, argumentValue string){

m.Errorf(invalidArgValueMessage.message, argumentName, argumentValue)

} // “Standard “MissingArg error message”

func (m *MainLogger) MissingArg(argumentName string){

m.Errorf(missingArgMessage.message, argumentName)

} // MainLogger enforces specific log message formats (we are using logrus, but you are free to use other libraries) type MainLogger struct {

*logrus.Logger

} func init() {

// Log as JSON instead of the default ASCII formatter.

logrus.SetFormatter(&logrus.JSONFormatter{})

}

Creating a constructor/initializer to return our MainLogger struct:

func NewLogger() *MainLogger {

f, err := os.OpenFile(“logrus.log”, os.O_APPEND|os.O_CREATE|os.O_RDWR, 0666) if err != nil {

fmt.Printf(“error opening file: %v”, err)

} var baseLogger = logrus.New() var standardLogger = &MainLogger{baseLogger} standardLogger.Formatter = &logrus.JSONFormatter{} mw := io.MultiWriter(os.Stdout, f) standardLogger.SetOutput(mw) return standardLogger

}

We’re done designing the basic standard interface and can start calling the methods in our Go files as:

var standardLogger = logger.NewLogger() // call method when an argument is missing

standardLogger.MissingArg(“imgModel.ImageData”) // call method when an invalid argument is supplied

standardLogger.InvalidArg(“imgModel.ImageData”)

4. Centralize Golang logs

Some Go applications are deployed across a cluster of hosts.

When you need to check/investigate your logs in those hosts, you’d be compelled to SSH into each one of them. This isn’t efficient in any way!

A scalable and efficient solution is to pass a log from your local files to a central platform where you can then process the logs as you deem required.

There are many ways to achieve this; one way is to use the Golang syslog package to forward logs from throughout your infrastructure to a single syslog server. Another way is to use a log management solution. Datadog and OpenTracing, for example, can tail your log files and forward logs to a central platform for processing and analysis.