Traceability in distributed services part II
By: Borja Sanchez Yuste
Tags: Spring Boot, Logback, Sleuth, Logstash

In the latest article, we viewed how to transport the value of any headers through our service so that we could have a unique identifier ‘xTraceId’ in all our services so that we could trace a request and link the logs that are generated in the different micro services that are part of our system.

In this post, we are going to integrate these changes into our server application so that we can see how the value of the xTraceId header is the same in the different services that our request passes through.

Then we will see how to modify the logback configuration to generate the traces in Json format and finally we will include the generation of traces for our own application.

All these changes will be based on the spring Sleuth library.

You can download the source code of the projects in the following Github link

Integration of changes in the server

Remember that all the classes described in the previous article have been generated in a library, this will allow us to include the described functionality in all our microservices.

Then we will generate a new ‘logger-server’ application and include in our pom file, the reference to the new library



<properties>

<spring-cloud.version>Hoxton.SR8</spring-cloud.version>

</properties>

<dependencies>

<dependency>

<groupId>com.jichu20</groupId>

<artifactId>logger-lib</artifactId>

<version>0.0.1-SNAPSHOT</version>

</dependency>

</dependencies>

Modifying the client …

We will modify the client service so that the requests point to the new service.

Creating the server

In the server service, we will create a controller and a service.

@controller

@service

The next step will be to create a client capable of invoking a service through spring’s RestTemplate connector. Sleuth, allows you to use different connectors, but in this article we will focus on RestTemplate.

It is important that the RestTemplate component used is a bean injected in our context, that is, we cannot use the `new RestTemplate ()` statement since sleuth would not have the capacity to inject the necessary interceptors or filters, for this we generate a bean in the context of type RestTemplate

Next we will execute a request against the client to view the traces in the two services

This first test will generate the following request

As we can see, in the headers sent, the B3 headers are included with the corresponding information.

Generating new headers

Now is the time to generate new headers, specifically we will generate the `xTraceId` header whose value will be a random UUID.

In the first time we will generate a new filter capable of identifying if the header has been filled in from a client and therefore we propagate its value or if, on the contrary, it is not informed and we will have to generate a new value for it.

Now the request we make will be like this:

And the response of our service will also return the information.

Pay special attention to the sentence.

MDC.put(Constant.X_TRACE_ID, xTraceId);

This allows us to have the information when writing the logs of our application.

Configuring logback

It is the moment to modify our logback configuration for the generation of logs in Json format, later we can use other processes to send these logs to centralized systems.

To make out this configuration we will use the logsthas library, which provides different appenders, layouts, … for logback. For this we will add the dependency.

<dependency>

<groupId>net.logstash.logback</groupId>

<artifactId>logstash-logback-encoder</artifactId>

<version>6.4</version>

</dependency>

Now we will add a new appender to our configuration

When we run our application, we can see that the traces now have a json format

Extra

Converters in the appender

In our appender, we can add converters, for example, we may want to get the date in nanosecond format.

For this we will generate a new class with the name `NanoSecondsConverter.java` with the following code

And we will modify the appender to include a new field that contains the date in nano seconds format.

<! — custom converters → <conversionRule conversionWord=”nanos” converterClass=”com.jichu20.loggerlib.converter.NanoSecondsConverter” />

... ... ...

“creationDateNano”: “#asLong{%-19nanos}”,

... ... ...

Interceptors

We also have the possibility of adding interceptors to our restTemplate so that it shows both the requests made and the responses received.

To do this, we will include the `RequestResponseLoggingInterceptor.java` class in our project and we will modify the definition of the restTemplate bean as follows.

We also have the possibility of adding interceptors to our restTemplate so that it shows both the requests made and the responses received.

To do this, we will include the `RequestResponseLoggingInterceptor.java` class in our project and we will modify the definition of the restTemplate bean as follows.

Request

Response

{

"creationDate": 1608033542219,

"level": "INFO",

"message": "============================response begin==========================================

Status code: 202 ACCEPTED

Status text:

Headers: [xTraceId:\"0407a46e-7c55–4613-a264-b2a320255c77\", Content-Type:\"application/json\", Transfer-Encoding:\"chunked\", Date:\"Tue, 15 Dec 2020 11:59:02 GMT\", Keep-Alive:\"timeout=60\", Connection:\"keep-alive\"]

Response body: {\"name\":\"elbarcodelpirata\",\"resume\":\"resume of the book\",\"author\":\"Robert Louis Stevenson\",\"numPAges\":2}=======================response end=================================================",

"spanId": "",

"traceId": "0407a46e-7c55–4613-a264-b2a320255c77",

"properties": {

"app-name": "client-logger",

"class": "c.j.l.i.RequestResponseLoggingInterceptor",

"thread": "http-nio-8080-exec-1"

},

"creationDateNano": 1608033542219000000

}

Source

Github link