Spring Boot Microservices — Implementing Distributed Tracing
By: Lal Verma
Tags: Spring Boot, Microservice Architecture, Programming, Distributed Tracing, Microservices

Spring Boot Microservices — Implementing Distributed Tracing

In this article, we will learn an important patten in Microservices Architecture — Distributed Tracing. We will create a sample implementation based on Spring Boot, Spring Cloud Sleuth and Zipkin. This is the fourth part of our series — Spring Boot Microservices — Learning through examples.

Photo by Marten Bjork on Unsplash

What is Distributed Tracing?

Logging is an important aspect of software development. It not only helps in troubleshooting the issues but also help us in understanding the behavior of our softwares. With multiple microservices, logging becomes a multifold challenge.

Consider the example we discussed in our previous exercise — Developing Service Discovery. In this exercise, Our Shopping Cart Service needs to call Product Catalog Service to get the product details. The request is distributed across multiple services (2 in our case). If each of the services has multiple instances running, request will follow a path determined at run time.

The picture below shows just the 9 user requests getting distributed across 6 instances and it has already started looking complex. Imagine a situation when millions of transactions are getting processed with multiple participating services.

Distributed Transactions — An illustration

If we try to trouble shoot an issue in this environment, our monolith style logging will be as useful as nothing. The Session, Thread, or the Http Request references do not help much when the request is getting processed across multiple services. We will be lost, digging the logs.

We must have a unique request/transaction reference which should be passed across all the calls, to all the dependent microservices, in the call chain. This unique reference is usually referred as Correlation-id or Trace Id.

The Trace Id is created when the request hits the very first service in the chain. For subsequent calls, already existing Trace Id is passed along, typically as an http header attribute. This pattern is called — Distributed Tracing.

Trace Id Propagation — An Illustration

Now that we understood the need of Distributed Tracing, lets see how we can implement this pattern.

How are we implementing Distributed Tracing?

Our sample implementation consists of two parts

Implementing Tracing — In this part we will focus on pattern implementation to the core including Trace Id generation, passing it along the service calls and including it in the logs. We will be using Spring Cloud Sleuth library to achieve this. Enabling Tracing System — In this part we will focus on how the traces are collected and visualized to get better insights into the service interactions. This reduces time in triage by contextualizing errors and delays. We will be using Zipkin to implement this.

Other than this we will also discuss patterns like “log aggregation” and “centralized logging” as these patterns are related and further enhances the trouble-shooting capabilities.

Implementing Tracing

We already developed our Product Catalog Service and Shopping Cart Service in our previous exercise. We do not need to revisit the exercise, but a brief recall is needed to set the grounds for further discussion.

Our Product Catalog Service is responsible to manage the product and provides api to create, update, delete and read the product (source code can be referred here).

Our Shopping Cart Service is responsible to manage the Shopping Cart. We will be focussing on addItem method of this service (Source code can be referred here).

Both these services are based on Spring Boot. We will be updating these services to introduce the Trace Id.

Step 1: Lets add a dependency of spring-cloud-starter-sleuth to the Product Catalog Service’s pom.xml

<project>

... <dependencies>

...

<dependency>

<groupId>org.springframework.cloud</groupId>

<artifactId>spring-cloud-starter-sleuth</artifactId>

</dependency>

</dependencies>

... </project>

Step 2: Update getProductDetails method of ProductCatalogService.java and add some sample log statements. We are using slf4j to get the Logger instance.

@RestController

public class ProductCatalogService { public class ProductCatalogService { @Autowired

private MongoTemplate mongoTemplate; private MongoTemplate mongoTemplate; private Logger logger = LoggerFactory.getLogger(ProductCatalogService.class);

public Product getProductDetails(

logger.info("get product details - process started");

Product product = mongoTemplate.findById(id, Product.class);

if (product != null)

logger.info("get product details - product found");

else

logger.info("get product details - product not found");

@GetMapping ("/product/{id}")public Product getProductDetails( @PathVariable String id) {Product product = mongoTemplate.findById(id, Product.class);if (product != null)else return product;

} }

Step 3: Update src/main/resources/application.properties to use the log level as TRACE and to include the service name as product_catalog.

spring.application.name=product_catalog spring.data.mongodb.uri=mongodb+srv://xxx-user:xxx-pwd@cluster0.nrsv6.gcp.mongodb.net/ecommerce logging.level.org.springframework.web.servlet.DispatcherServlet=TRACE

Thats all we need, to add Trace Id in our requests and logs. Spring Cloud Sleuth is a layer over Brave, which is a distributed tracing instrumentation library. We do not need to write any custom code to create or propagate the trace contexts as it is done automatically by Brave. It typically intercepts the requests to do it. It also configures the logging context to include Trace Id and other variables.