Microservices — Discovering Service Mesh
By: Lal Verma
Tags: Istio, Service Mesh, Microservices, Spring Boot, Software Engineering

Microservices — Discovering Service Mesh

Service interactions in the microservices world deal with many non-functional concerns — service discovery, load balancing, fault tolerance, etc. Service Mesh provides the platform to manage these concerns in a more efficient and cleaner way. In this article, we will understand the framework in more detail along with a sample implementation based on Istio and Spring Boot. This is the 9th part of our learning series on Spring Boot Microservices.

Photo by Ricardo Gomez Angel on Unsplash

Why do we need Service Mesh?

The unique proposition of Service Mesh does not lie in “what it offers” instead “how it achieves it”. It solves the problems, originated in the microservices architecture, with a different and mature perspective. It offers the platform where the non-functional concerns related to service interactions, are managed more efficiently. It ensures these operational concerns are not coupled together with the business logic, as was the case with earlier solutions.

We already discussed multiple microservice patterns as part of our spring-boot learning series including Service Discovery, Load Balancing, API Gateway, Circuit Breaker, and many others. Before moving further, I assume, you have a basic understanding of these patterns. We will be discussing and referring to them throughout this article. If you do not have the background on them, it will be difficult to understand “What Service Mesh offers”. You can check out our learning series to get insight into these patterns if needed.

If we rewind our previous exercises, we will find that the non-functional concerns are tightly coupled with the application logic. For instance, in our service discovery exercise, we implemented the load balancing on the client service side. In our circuit breaker exercise, we implemented the decorators, again on the client service end. These solutions work fine except that they restrict software maintenance, both from infrastructure and business perspectives. Different technologies, standards, and design approaches across the multiple microservices teams create a diversified set of implementations for the same set of problems. This creates a much bigger problem to solve.

Assume the scenario, where we have to implement TLS Certificates across all the “service to service communications”. If different teams start working on this, this will become a long-lasting exercise. It's primarily due to the fact that the operation logic is bundled together with the application logic. This increases the implementation complexity multifold. Teams will be busier in resolving the inconsistencies rather than focusing on their core business logic.

The primary objective of Service Mesh is to segregate the non-functional concerns, primarily dealing with connecting, securing, and monitoring services, from the application code. With the help of a separate infrastructure layer, we can enable the non-functional features with almost zero impact on the existing services.

Consider the case of our e-commerce system. We have a Product Catalog Service responsible for product management and a Product Inventory Service responsible for product inventory management. If our portal is interested in getting the product details the call will look as shown in the figure above. With the traditional approach, the logic of Service Discovery and Circuit Breaker will be implemented along with the application logic of Product Catalog Service. Service Mesh promises to separate this out.

Enabling operational concerns with the help of a separate layer improves the overall maintainability of the system significantly. Also, the changes can be managed more effectively as different teams can focus on different concerns. Development teams can focus on the business logic whereas the DevOps teams can focus on implementing the infrastructure concerns.

How Service-Mesh Works?

There are multiple service mesh technologies including Linkerd, Istio, Consul, AWS, and many more. More or less they work on the same architecture based on the proxies. Each of the business services is associated with one proxy each. So in our case Product Catalog Service will have one proxy and the Product Inventory Service will have another. The proxies reside alongside the services and this is the reason they are termed as — sidecar proxies.

All the sidecar proxies reside in the data plane. They intercept all the calls to and from the service and enable the operational functionalities through it. The list of operational features is long. Few examples could be automatic load balancing, routing, retries, fail-overs, access controls, rate limits, automatic metrics, logs, tracing, etc. Most of these features operate at the request level.

For instance, if Product Catalog Service makes an HTTP call to Product Inventory Service, the sidecar proxy on the Product Catalog end can load balance the call intelligently across all the instances of Product Inventory Service. It can retry the request if it fails. Similarly, the sidecar proxy on the Product Inventory Service side can reject the call if it’s not allowed, or is over the rate limit.

Another important component in Service Mesh is the control plane which helps in coordinating the behavior of proxies and provides API to manipulate and measure the mesh. It’s responsible for managing the sidecar proxies, ingress/egress gateways, service registry, certificates, and other management aspects.

Now that we understand the Service Mesh framework to some extent, let's see how it works on the ground. We will be using Istio for our sample implementation which is the leading service mesh framework. It's an open-source technology supported by Redhat, Google Cloud, IBM Cloud, Pivotal, Apigee, and other technology leaders.

Sample Implementation

Istio is designed to be platform-independent and supports services deployed over Kubernetes, Consul, or Virtual Machines. We will be using Kubernetes as the underline deployment platform. If you are new to Kubernetes, I suggest getting a basic understanding of it. You can visit my article on Working with Kubernetes for a high-level overview of this topic. For a detailed overview, you can visit its official website.

Istio provides the platform to enable multiple features in the areas of traffic management, security, and observability. With this exercise, we will focus on enabling Service Discovery and Circuit Breaker patterns along with a flavor of API Gateway. I have already covered Service Discovery using Netflix Eureka in one of the previous exercises. Similarly, I have captured the Circuit Breaker pattern based on Resilience4j and API Gateway in separate articles. In each of these exercises, the implementation of the patterns is coupled to the application logic, significantly. In this exercise, we will implement these patterns independently, out of the services code.

Service Mesh — Sample Implementation

We will be implementing these patterns in the context of two Spring Boot based microservices — Product Catalog Service and Product Inventory Service. Assuming an external client is interested in getting the product details which include product availability as well, we will see how the API Gateway, Service Discovery, and Circuit Breaker patterns are implemented in this call. We will cover the exercise with the help of the following sections —

Setting up

Installing Kubernetes — We will be installing Minikube which can run a single-node Kubernetes cluster and best suited for learning purposes. I am using a Debian machine and used the following commands to download and install the minikube package. You can get the installation instructions for other platforms at https://kubernetes.io/docs/tasks/tools/install-minikube/.



$ curl -LO ###### downloading package$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb ###### installing package

$ sudo dpkg -i minikube_latest_amd64.deb ###### starting kubernetes cluster

$ minikube start

Installing Docker — We will not be able to start the Kubernetes cluster as it needs an underline virtualization technology such as containers or virtual machines to function. We will be using the most popular container option here — Docker. The following command helps in installing Docker on my machine. You can check other installation options at https://docs.docker.com/engine/install/.

# installing through convenience scripts for testing purpose

$ curl -fsSL https://get.docker.com -o get-docker.sh

$ sudo sh get-docker.sh # enabling current user to run docker

$ sudo usermod -aG docker $current_user && newgrp docker

Now that we have the Docker installed you can run minikube start to start the Kubernetes cluster.

In this exercise, we will be building container images for our spring-boot based services. If you are new to this topic, you can get a crash course on this here.

Installing Kubectl — This is the command-line tool to access Kubernetes APIs. We will be using it to manage service deployments.

# installing kubectl on Debian

sudo apt-get install -y kubectl

Installing Istio — Here comes our primary framework. We have already installed its pre-requisites, so its installation should be smooth.



$ curl -L ######### downloading latest release of istio$ curl -L https://istio.io/downloadIstio | sh - ######### including istio on path

$ export PATH=$PWD/bin:$PATH



######### installing istio in demo mode

$ istioctl install --set profile=demo ######### instructing istio to automatically inject envoy sidecar proxies

$ kubectl label namespace default istio-injection=enabled

By running the above commands, a lot of things have happened behind the scenes—

We are ready to create and run containerized services with the help of Docker. We are ready with our Kubernetes Cluster. Istio has installed ingress and egress gateways to control the incoming and outgoing traffic. Istio is ready to deploy its side-car proxies.

Deploying Services, Enabling Service Discovery

It's time to deploy our services. Let's get the microservices code from our Github repository —

This will get the code across all the samples. For the purpose of this exercise, we will be dealing with the samples present in the directory spring-boot/istio-example . Before we jump into creating the container images for our services, run the command $ eval $(minikube docker-env) to use the docker environment available with minikube. This will ensure that all the local images are stored in this environment and referred correctly during runtime.

Let's create a container image for the Product Inventory Service. Change your working directory to spring-boot/istio-example/product_inventory . Dockerfile is already available for this service —

###### product inventory service ##### FROM adoptopenjdk:11-jre-hotspot as builder

ARG JAR_FILE=target/product_inventory-0.0.1-SNAPSHOT.jar

ADD ${JAR_FILE} app.jar

ENTRYPOINT ["java","-jar","/app.jar"]

Run the following command to build the docker image. This will update the local docker daemon respectively.

##### building container image for product inventory service

$ docker build -t product-inventory:v0.0.1 .

We need to create deployment and service configurations for our Product Inventory Service. Configurations are already available in the root directory of the service — deployment-def.yaml and service-def.yaml

deployment-def.yaml — creating basic deployment configuration based on the container image product-inventory:v0.0.1

apiVersion: apps/v1

kind: Deployment

metadata:

name: product-inventory

namespace: default

labels:

app: product-inventory

spec:

replicas: 1

selector:

matchLabels:

app: product-inventory

template:

metadata:

labels:

app: product-inventory

spec:

containers:

- name: product-inventory

image: 'product-inventory:v0.0.1'

imagePullPolicy: Never

ports:

- containerPort: 8080

service-def.yaml — exposing product inventory service accessible inside the Kubernetes cluster.

apiVersion: v1

kind: Service

metadata:

labels:

app: product-inventory

service: product-inventory

name: product-inventory

namespace: default

spec:

ports:

- port: 8080

name: http

selector:

app: product-inventory

type: ClusterIP

Run the following commands to apply these configurations —

$ kubectl apply -f deployment-def.yaml

$ kubectl apply -f service-def.yaml

The above commands have done quite a few things. Our Product Inventory Service is deployed and is accessible in the cluster. Istio has also installed a side-car proxy for this service. If you see the container pods by running kubectl get pods you will see something like this.

product inventory service — running pod

In the READY column, it displays how many containers the pod is running. If we investigate the running pod with the command kubectl describe pod product-inventory-db9686d7d-7xsz5 , we will see that the pod has two containers — product-inventory & istio-proxy

This means along with our Product Inventory Service, Istio has already installed the side-car proxy. This proxy has the capability to intercept all the incoming and outgoing requests of the Service.

Also by applying service-def.yaml we have instructed Istio to register the service in the service registry with the name product-inventory . Even if we create 10 instances of this service, we can communicate to it by referring just the DNS name — product-inventory . Service Discovery and Load Balancing will continue to happen behind the scenes.

Similar to the Product Inventory Service lets configure and deploy Product Catalog Service

##### creating docker image for product-catalog service

$ cd istio-example/product_catalog

$ docker build -t product-catalog:v0.0.1 . ##### deploying product catalog service

$ kubectl apply -f deployment-def.yaml

$ kubectl apply -f service-def.yaml

With this, our Product Catalog Service is up and running. Let's take a quick look at how the service is calling the Product Inventory Service. Open the code for ProductCatalogService.java and check the getProductDetails API.

//get product details api

public Product getProductDetails(

Product product = mongoTemplate.findById(id, Product.class);

ProductInventory productInventory = restTemplate.getForObject(" @GetMapping ("/product/{id}")public Product getProductDetails( @PathVariable String id) {Product product = mongoTemplate.findById(id, Product.class);ProductInventory productInventory = restTemplate.getForObject(" http://product-inventory:8080/inventory/ " + id, ProductInventory.class); product.setProductInventory(productInventory); return product;

}

In this, it's referring to Product Inventory Service with the DNS name — product-inventory . We have enabled service discovery for both our services and we should be able to make the call to getProductDetails API. But wait! To do this, we must enable access to Product Catalog Service from outside our cluster. And this will be done with the help of Gateway configuration.

Implementing Gateway

This is relatively simpler. You can find the gateway configuration in istio-example root directory with the name gateway-config.yaml .

apiVersion: networking.istio.io/v1alpha3

kind: Gateway

metadata:

name: product-catalog-gateway

spec:

selector:

istio: ingressgateway

servers:

- port:

number: 80

name: http

protocol: HTTP

hosts:

- '*' apiVersion: networking.istio.io/v1alpha3

kind: VirtualService

metadata:

name: product-catalog

spec:

hosts:

- '*'

gateways:

- product-catalog-gateway

http:

- match:

- uri:

prefix: /product

route:

- destination:

host: product-catalog

port:

number: 8080

We are defining a routing rule here instructing to forward all the requests starting with /product to the Product Catalog Service. Apply this configuration by running kubectl apply -f gateway-config.yaml .

Our basic API gateway is ready with the above configuration. We can use it for multiple purposes but let's stick to our basic need. Run the following commands to get the service URL for Product Catalog Service

##### identifying service host and port

$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')

$ export INGRESS_HOST=$(minikube ip)

$ export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT

$ echo $GATEWAY_URL ## returns something like 192.168.49.2:30682 ##### minikube tunnel facilitate creating a network route

$ minikube tunnel &

Now that we have the URL to access the service, let's run curl to access the product details for test-product-123. This should return the product details successfully.

$ curl http://192.168.49.2:30682/product/test-product-123

Congratulations, you have successfully implemented API Gateway and Service Discovery for our services.

Implementing Circuit Breaker

Implementing the Circuit Breaker is also easy to configure. You can find the configuration in the root directory — circuit-breaker-config.yaml .

apiVersion: networking.istio.io/v1alpha3

kind: DestinationRule

metadata:

name: product-inventory

spec:

host: product-inventory

trafficPolicy:

connectionPool:

tcp:

maxConnections: 1

http:

http1MaxPendingRequests: 1

maxRequestsPerConnection: 1

outlierDetection:

consecutiveErrors: 1

interval: 1s

baseEjectionTime: 3m

maxEjectionPercent: 100

With this, we are applying the configuration on top of the Product Inventory Service based on DestinationRule. By setting maxConnections: 1 and http1MaxPendingRequests: 1 we are instructing to activate the circuit breaker if it receives requests from more than one connection at the same time. The circuit breaker will remain in an open state and will keep rejecting the requests till it starts receiving the requests from one connection only.

You can use any performance testing tool to validate this behavior. You can also use the bundled tool called folio under the directory — sample-client . More details on this tool can be found here.

Next Steps

We successfully implemented API Gateway, Service Discovery, and Circuit Breaker with the help of Istio. We did not update our services to implement these infrastructure concerns. Instead, we used the separate infrastructure layer provided by Istio to enable them. We provided the configurations and Istio did the rest of the magic.

We can use Istio to configure other traffic management aspects including request routing, fault injection, request timeouts, and ingress/egress policies. We can use its security layer to configure TLS certificates, authentication, and authorization. Observability is another important area of offering from Istio. We can use it to enable service monitoring, logs, distributed tracing, telemetry, and other visualizations. Check out more on its features at the Istio website.

The solution provided by Service Mesh looks clean but as usual, it has some side effects too. It adds additional hops for each call in the form of side-car proxies. In our case, one hop is added at Product Catalog Service and another at Product Inventory Service. The additional hops in the form of proxies need additional resources — CPU and memory. As the microservices increase, this can increase the resource overhead to a good extent. We must keep a watch on this!

Additionally, each of the microservices patterns, when implemented through Service Mesh, does limit the control and capabilities over the operational features to some extent. I am sure this concern will subside with time, as the technology matures. For now, this is one of the best approaches to manage your service to service interactions.