What the heck is a Service Mesh, anyway?
Tags: devops, microservices, kubernetes, tutorial
Published: 2023-03-23 10:38:38

Software applications can be thought as an ever-evolving organism. As the number of active users for your apps increase, at some point you're going to eventually consider the problem of scale.

One of the solutions that gradually evolved over the years to manage the challenges of scale were **microservices**. The design principle of a micro-service based architecture is to treat the application as a collection of individual, loosely-coupled services. This offers several advantages over monoliths-faster development, better deployment, fault tolerance, security and more to name a few.

But designing a microservice-based architecture comes with its own set up challenges. For instance:

* How do you manage and visualize **traffic flows** between microservices?
* How do you keep the data flow between microservices **secure**?
* **Observability** - how do  measure the internal states of a particular service?
* **Monitoring** and **tracing** of network requests.

For this, you need something called **Service Mesh**. But first, let's first look at how a typical cluster is implemented using Kubernetes.

## A conceptual container orchestration setup

Here is a diagram of a typical cluster in Kubernetes:

![Diagram showing a typical cluster of microservices](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z0m3flmflrw05dhcbo83.jpg)

Here, *microservice A* could for instance, use the services of *microservice B*, which in-turn could rely on another *microservice C*. A typical architecture built using Kubernetes would involve the Pod A making a **network request** to Pod B, and in-turn would make a request to Pod C.

A container orchestration tool like Kubernetes helps you manage and handle the pods, but it has little control over the actual data or **network flow** between pods. What's going on at the *network connections* when each individual pod makes a request to another pod?

In a cluster with dozens or even hundreds of pods, you will need visibility on the *interconnections* between the pods themselves. Here is where a service mesh comes into play.

## What is a Service Mesh?

A Service Mesh is an extra layer of abstraction that you add to your Cluster of services, in-order to efficiently manage and monitor them.

For instance, if you're using Kubernetes, you can think of a service mesh as this layer of software that resides *underneath* your kubernetes pods. This software takes over control over every network request made between pods in the entire cluster.

![Conceptual diagram of a service mesh](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i2t8v3infdurbgfyp6hs.jpg)

### Wait, but why though?

The service mesh software intercepts each incoming network request, and re-routes it to the desired container. In-doing so, it can implement some function or some *mesh logic*, that uses the network request and do some pretty interesting stuff. For instance:

* Measure the **time** taken for the network request to complete.
* Collate **network errors** that occur in any part of the cluster.
* Implement **security** - for instance, all network calls must be  encrypted.
* **Re-route requests** based on a given  criteria or when certain conditions are met.
* **Visualize** and manage network traffic

Without a service mesh, it would be pretty difficult to implement the above.

Indeed, the most useful features of a service mesh involves **telemetry**. Telemetry involves monitoring the health and security of microservices, measuring their performance, and collecting useful statistics such as the memory overhead, CPU utilization and more.

Another feature of a service mesh is **tracing**. This would allow you to visualize the network flow and trace the **chain of requests** within the context of a particular feature or service.The service mesh could  gather such useful information that would enable for faster debugging and bug resolution, or even run a time series analysis to predict future outages of a particular service.

## Istio

Istio is a Kubernetes native solution, used by several large scale corporations like Google, IBM and Microsoft. It has support for K8s and also virtual machines.

There are many different ways by which a service mesh could be implemented. Here's how Istio implements it:

![Service mesh implemented using Istio](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p69pj2gik2yl7qydq9fa.jpg)

For each of the pods in the cluster, Istio injects its own container, also known as a **[proxy](https://en.wikipedia.org/wiki/Proxy_pattern)**. Each individual proxy takes over and handles all incoming and outgoing network requests for that pod. This proxy contains the mesh logic to further process the network request for tracing, telemetry, and re-direct it to the target container proxy.

Istio is designed to be loosely coupled. The individual containers have no knowledge about the proxy injected by Istio, and neither do they need to contain any code for traffic management, tracing or other features provided by Istio. The individual pods continue to run and make network calls as though they were directly communicating to other pods.

This design of service mesh implemented by Istio is also known as the **[Sidecar pattern](https://www.oreilly.com/library/view/designing-distributed-systems/9781491983638/ch02.html)**.

## Other implementations

Similar to Istio, there are other popular implementations as well:

* **[Linkerd](https://linkerd.io/)**: Claims to be the lightest and simpler than any other service mesh implementation. It's K8s only. Has its own proxy written in Rust.
* **[Consul](https://www.consul.io/)**: A full-feature service management system. Runs as a Daemon on every K8s node and works with Envoy.

Here's an [article](https://platform9.com/blog/kubernetes-service-mesh-a-comparison-of-istio-linkerd-and-consul/) that compares these implementations.

## Istio under the hood

Istio actually assembles together several third party components in-order to construct a service-mesh for your cluster.

![Istio architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xac1hgekd1x29p3ab79s.png)

### Data plane

Istio uses **[Envoy](https://www.envoyproxy.io/)**, an open-source cloud-native service proxy under the hood for the Proxy containers within your pods. These proxies constitute the data plane layer of Istio.

### Control plane

The control plane consists of majorly 3 components:

**Galley**: If you're using Kubernetes, Galley reads your Kubernetes YAML file and transforms it into a format that Istio understands. It can also work with other orchestration frameworks such as [Mesos](http://mesos.apache.org/) and Consul.

**Pilot**: This component takes the configurations and then converts them into a format that Envoy understands, in-order to deploy the proxies.

**Citadel**: Responsible for certificate management and enabling TLS / SSL across your entire cluster.

> Since [Istio version 1.5](https://istio.io/latest/blog/2020/istiod/), the above components have been combined into a single binary called `istiod`. 

*****

At Egen, we've put together a video in-order to understand fully well how Istio works, along with use-cases showing what you can do with it. We also show you a real-world example of a service not working as intended, and how Istio can help you narrow down the problem and apply an appropriate fix.

{% embed https://www.youtube.com/watch?v=IvgE_PZui3M %}
**Thanks for reading**

We host several such meetups discussing scalability, distributed systems, application engineering and more. If you're interested in learning these topics, do give us a follow on [collective.ac](http://collective.ac) to get notified for future events.

*****

## Resources

* [Does your organization need Istio](https://egen.solutions/videos/role-of-service-mesh-in-kubernetes-explained-do-you-need-istio/)?
* [What is Istio Service Mesh](https://www.youtube.com/watch?v=z6WjVJ1XelY)?
* [Istio and Service Mesh - simply explained in 15 minutes](https://www.youtube.com/watch?v=16fgzklcF7Y)