Logging in your API
Tags: programming, backend, api, java
Published: 2023-02-22 08:19:46

## Logging in your API

This article will highlight some methods and best practices you can adopt in your service for proper logging and error tracing.

## Table of content

- [What is logging](#what-is-logging)
- [Choosing your log framework](#choosing-your-log-framework)
- [What and when to log](#what-and-when-to-log)
- [Try and catch?](#try-and-catch)
    - [Exception Handling](#exception-handling)
    - [Logging Exception For Some Weakly-Typed Language](#logging-exception-for-some-weakly-typed-language)
    - [Almost say no to exception message](#almost-say-no-to-eception-message)
    - [Logging your exception](#logging-your-exception)
- [Data Safety](#data-safety)
- [Cloud Logging](#cloud-logging)
- [Glossary](#glossary)
- [References](#references)

## What is logging

As copied from Wikipedia Logging is the act of keeping a log of events that occur in a computer system, such as problems, errors or just information on current operations. These events may occur in the operating system or in other software.

## Choosing your log framework

Most API frameworks like spring, .NET, laravel comes equiped with 
robust logging packages that you can use in your project. The list 
below highlights some logging framework you can use in your services.

- Java -> [Logback](https://logback.qos.ch/), [Log4j2](https://logging.apache.org/log4j/), [JDK (Java Util Logging)](https://docs.oracle.com/en/java/javase/12/docs/api/java.logging/java/util/logging/package-summary.html), [Slf4j](https://www.slf4j.org/), e.t.c.
- C# -> [Built in Logger](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/logging/), [Log4Net](https://logging.apache.org/log4net/), [Serilog](https://serilog.net/), [NLog](https://nlog-project.org/), e.t.c.
- PHP -> [Built in Logger](https://www.loggly.com/ultimate-guide/php-logging-basics/), [Monolog](https://seldaek.github.io/monolog/), [Analog](https://github.com/jbroadway/analog), [KLogger](https://github.com/katzgrau/KLogger), [Log4PHP](https://logging.apache.org/log4php/)
- C/C++ -> [liblogax](https://exoticlibraries.github.io/liblogax/), [Boost.Log](https://www.boost.org/doc/libs/1_64_0/libs/log/doc/html/index.html), [log4c](https://github.com/bmanojlovic/log4c), e.t.c.
- NodeJS -> [Pino](https://getpino.io/), [Winston](https://github.com/winstonjs/winston), [Bunyan](https://github.com/trentm/node-bunyan), [Npmlog](https://github.com/npm/npmlog), e.t.c.


## What and when to log

When it come to loggin no information is too much to be logged and 
there is no specific place to place a log just ensure that the logs 
are properly classified. 

There are 6 basic log classification

- TRACE
- DEBUG
- INFO
- WARN
- ERROR
- FATAL

When logging basic information that does not change on each call it can be classified as INFO log, if a logic or operation is likely going to cause an error in the service or a non fatal error occur then it can be classified as WARN log. For operations that cause an error that 
breaks or stop the flow of an operation then it can be classified as ERROR log. 

In the case of FATAL log the service is expected to end or shutdown, 
a situation when a FATAL log is required is when the memory of the 
system is completely used or corrupt a shutdown is expected cause it 
will be almost impossible for the service to continuing function 
properly after that.


## Try and catch?

Exception plays an important part in logging as the error in the 
exception catch block is logged to give the user an idea of what error 
occur and possibly why it occur. 

In strongly type language like Java, C# the try and catch logic 
is robust enough to allow handling specific exceptions but in weakly 
typed language like Javascript, Python, developers can become lazy in 
using the language provided type check to determine what error is thrown 
which in turn encorages generic and un-helpful logging. Then there is 
languages like PHP that allows both strong and weak typing in special 
case like try...catch block, for such languages always go for the strong 
typing feature for safety and better error tracing.


### Exception Handling

If you have a function or method that perform multiple operations, the 
try and catch block should not be at the highlevel of function rather it should be at the last logic level such that when the exception is logged 
it can easily be traced to the logic point. 

An example of this is a method that accept a POST body, converts the body json string to object then store the object in the database.

**BAD**
```java
public ResponseEntity<?> store(@RequestBody String body) {
    try {
        User user = mapper.readValue(body, User.class); // can throw JsonException
        // some long operation ...
        repository.save(user); // can throw ConstraintException
        // some other long operation ...
        return new ResponseEntity<>(HttpStatus.OK);
    } catch (Exception ex) {
        return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
    }
}
```

**GOOD**
```java
public ResponseEntity<?> store(@RequestBody String body) {
    User user;
    try {
        User user = mapper.readValue(body, User.class); // can throw JsonException
        // some long operation ...
        repository.save(user); // can throw ConstraintException
        // some other long operation ...
        return new ResponseEntity<>(HttpStatus.OK);
    } catch (JsonException ex) {
        log.error("Error occur while processing body payload", ex);
        return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
    } catch (ConstraintException ex) {
        log.error("Error occur while persisting to database", ex);
        return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
    }
    return new ResponseEntity<>(HttpStatus.OK);
}
```

**EFFICIENT**
```java
public ResponseEntity<?> store(@RequestBody String body) {
    User user;
    try {
        user = mapper.readValue(body, User.class); 
    } catch (JsonException ex) {
        log.error("Error occur while processing body payload", ex);
        return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
    }
    // some long operation ...
    try {
        repository.save(user);
    } catch (ConstraintException ex) {
        log.error("Error occur while persisting to database", ex);
        return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
    }
    // some other long operation ...
    return new ResponseEntity<>(HttpStatus.OK);
}
```

Now let analyse the trinity above, take note of the long operation 
comment.

- Bad: This prevent logging which error actually occur though it 
will catch any error even even those that we did not imagine occuring 
at the long operation blocks. It makes the service not debuggable as it will almost be difficult to trace the error down do where it actually occurs. And also a custom message cannot be returned to the 
caller.
- Good: In the good block we take account of the two types of error that can occur but we have our other code logic (long operation) into tha try..catch block which is not neccessary.
- Efficient: In this code block we only put a try catch around each 
logic which will spare the other logics (long operation) from being 
caught in the try..catch block.

### Logging Exception For Some Weakly Typed Language

In languages that does not provide specific exception catching, it is 
best practice to check what exception occur so as to produce proper log.

Take for example the code block below in Javascript and PHP, handles 
general error that can occur in a try catch block.

- Javascript
```js
try {
    // operation 1 can throw JsonParseException
    // operation 2 can throw DatabaseException
} catch (err) {
    console.error("Error occur", err);
    return response.status(500);
}
```

- PHP
```php
try {
    # operation 1 can throw JsonParseException
    # operation 2 can throw DatabaseException
} catch ($e) {
    echo "Error occur: " .  $e->getMessage();
    return response->status(500);
}
```

The issue with the exception catch above is that, two type of exception is likely to occur but both of them are handled in the catch block and 
if a response is expected from the caller a generic response is giving 
which is not likely to be helpful to the caller. Therefore it important 
to check the type/instance of that error to properly log the error and 
send meaning ful response to caller.

- Javascript
```js
try {
    // operation 1 can throw JsonParseException
    // operation 2 can throw DatabaseException
} catch (err) {
    if (err instanceof JsonParseException) {
        console.error("Error while parsing Json body", err);
        return response.status(400);
    } else if (err instanceof DatabaseException) {
        console.error("Unable to persist to database", err);
        return response.status(500);
    }
    console.error("Unknown error occur", err);
    return response.status(500); 
}
```

- PHP
```php
try {
    # operation 1 can throw JsonParseException
    # operation 2 can throw DatabaseException
} catch ($e) {
    if ($e instanceof JsonParseException) {
        console.error("Error while parsing Json body", $e);
        return response->status(400);
    } else if ($e instanceof DatabaseException) {
        console.error("Unable to persist to database", $e);
        return response->status(500);
    }
    console.error("Unknown error occur", $e);
    return response->status(500); 
}
```

In strongly typed language like Java you can handle the two exceptions 
in seperate catch blocks. And there is no need for the general 
exception case since it unlikely to occur.

```java
try {
    // operation 1 can throw JsonParseException
    // operation 2 can throw DatabaseException
} catch (JsonParseException ex) {
    log.error(ex.getMessage(), ex);
    return response.status(400);
} catch (DatabaseException ex) {
    log.error(ex.getMessage(), ex);
    return response.status(500);
}
```

### Almost say no to eception message

When it comes to logging the exception's `.getMessage()`, or `.message` 
method or fields are almost useless as a properper logging needs to 
throughly output where error occur, why it occur and possible what 
caused the error to happen. The message value of exceptions in this 
case does not carry all the informations rather it only returns 
what error occurs. 

Hence it proper to log the entire stack trace of the exception.

### Logging your exception

If the framework or logging framework used in your project provides 
methods to log stack trace then you should use that instead of printing 
the stack trace directly to your console. This allow the log 
framework to categorize the exception and possibly sanitize the stack 
trace before printing to the console.

For example In java spring rather than using `ex.printStacktrace()`, 
invoke the error method in any of the provided log implementation e.g.

```java
Logger log; // = any log implementation

try {
    //...
} catch (DatabaseException ex) {
    // bad
    ex.printStackTrace();
    // good
    log.error(ex.getMessage(), ex);
}

```


## Data Safety

When it comes to logging it is important to be careful of the data 
being sent to the log service or printed in the console. Information 
like secrets, passwords should **NEVER** be logged rather these type 
of information can be printed out as `***` or in an encrypted value 
that only the developer can decrypt the actual value.

A good example is a simple User class in spring if there is a need to 
print out the class field ensure the secrets/password fields are 
hidden or encrypted in the log.

```java
class User {
    String username;
    String password;

    @Override
    public String toString() {
        return "[User { username=" + username  +", password=******}]";
    }
}
```

## Cloud Logging

Most API frameworks come equipped with robust logging framework which 
you can take advantage of when implementing logging in your service. 
An example is the spring framework which have the cloud logging 
feature to enable tracing log and fails between multiple services.

### Configuring Spring Sleuth

For Spring Framework the cloud logging is provided through the 
spring cloud package. Detailed walkthrough and version of spring 
cloud that matches your spring or spring boot version can be seen 
[here](https://spring.io/projects/spring-cloud)

## Glossary

- Strongly Typed: A strongly typed programming language is one in which each type of data, such as integers, characters, hexadecimals and packed decimals, is predefined as part of the programming language.

- Stack trace: This is the collection of stack records which stores the 
procedure or method call information up until the stack trace is 
invoked. Information stored in stack trace are the file name, line 
number, column number e.t.c.

- Weakly Typed: 


## References

- https://spring.io/projects/spring-cloud
- https://enlabsoftware.com/development/top-logging-frameworks-for-net-applications-and-the-best-configuration-tips.html
- https://www.loggly.com/ultimate-guide/php-logging-libraries/
- https://github.com/exoticlibraries/liblogax