Introduction to Istio Traffic Management
By: George Novack
Tags: Kubernetes, Istio, Service Mesh, Microservices

Introduction to Istio Traffic Management

Traffic Routing with Istio by Example

What is Istio?

The continued adoption of microservices architectures and the move toward complex distributed applications composed of decoupled components that communicate via APIs introduces various new challenges for developers. While these types of applications help us to encapsulate, reuse, and efficiently scale discrete components of business logic, they also require a lot more network traffic between services, which makes the need for reliable, secure, and observable channels of communication all the more important.

This is where Istio comes in. Istio is an open-source Service Mesh that helps to simplify the communication between microservices in distributed applications. The core features of Istio generally fall into one of three categories:

Traffic Management: Istio’s Traffic Management capabilities include support for resilience patterns like Retry Policies and Circuit Breakers, as well as request routing capabilities to support scenarios like A/B Testing and Canary Deployments. The Traffic Management components of Istio are what we are going to focus on primarily in the article.

Istio’s Traffic Management capabilities include support for resilience patterns like Retry Policies and Circuit Breakers, as well as request routing capabilities to support scenarios like A/B Testing and Canary Deployments. The Traffic Management components of Istio are what we are going to focus on primarily in the article. Observability: When applications are running in the service mesh, Istio provides out-of-the-box Metrics, Traces, and Logs to provide engineers with observability of service health and behavior. We will see shortly how Istio is able to generate this telemetry without any additional work from the application developers.

When applications are running in the service mesh, Istio provides out-of-the-box Metrics, Traces, and Logs to provide engineers with observability of service health and behavior. We will see shortly how Istio is able to generate this telemetry without any additional work from the application developers. Security: By default, Istio provides encrypted communication channels between microservices in our application, as well as support for various types of authentication and authorizations policies that can be applied to different components of our application.

Note: While Istio is a platform-independent technology, we will be running Istio on Kubernetes for the purposes of the examples below. This writing assumes a basic understanding of how Kubernetes works.

How does Istio Work?

At a high level, Istio works by deploying a proxy alongside each service in your application. This Envoy Proxy, which is often referred to as a sidecar, intercepts all traffic to and from the service, generates telemetry for this traffic, and can be configured to enable the various Authentication, Authorization, and Traffic Management functions described above.

The two diagrams below illustrate the difference between service communication without Istio and the Envoy Proxies (top), and communication within an Isito service mesh, where traffic is routed through the proxies (bottom).

Basic communication between two services without Istio.

Communication between two services in an Isito service mesh.

Note: In Kubernetes terms, the Proxy and its corresponding Service (e.g. the Proxy on the left and Service A above) are two containers running in the same pod.

Istio Components

One of the great things about Istio is that it is highly configurable; however, this flexibility means that there are several different components that we need to understand in order to configure traffic routing for our services.

Here is a bird’s eye view of all of the components involved in a single request to a service within an Istio service mesh running on Kubernetes.

Overview of Istio Components

Let’s break down each component involved in this request flow:

Load Balancer: This is an external load balancer that exposes a public IP address from which external traffic is routed into our Kubernetes cluster. This is not specific to Istio, this kind of load balancer is provisioned by your cloud provider (e.g. AWS or GCP) any time a service of type LoadBalancer is deployed to you cluster. If you’re running Kubernetes on-prem, there are options such as MetalLB that allow you to provision your own external load balancer.

This is an external load balancer that exposes a public IP address from which external traffic is routed into our Kubernetes cluster. This is not specific to Istio, this kind of load balancer is provisioned by your cloud provider (e.g. AWS or GCP) any time a service of type is deployed to you cluster. If you’re running Kubernetes on-prem, there are options such as MetalLB that allow you to provision your own external load balancer. Gateway Proxy: This is a Kubernetes service of type LoadBalancer that we can configure to customize how traffic entering the cluster is routed to our services. By default, Istio deploys a Gateway Proxy called istio-ingressgateway in the istio-system namespace (we’ll use this in our examples applications), but you can also deploy and configure your own Gateway Proxy if needed.

This is a Kubernetes service of type that we can configure to customize how traffic entering the cluster is routed to our services. By default, Istio deploys a Gateway Proxy called in the namespace (we’ll use this in our examples applications), but you can also deploy and configure your own Gateway Proxy if needed. Gateway Configuration: This is one of the more confusing Istio resources we’ll encounter, as it is often called simply called a Gateway in Istio documentation and can easily be confused with the Gateway Proxy described above. In fact, when we create one of these resources (which we will do shortly), we provide a value of Gateway to the kind property of the resource; however, I’ve chosen to call this resource a Gateway Configuration in the above diagram, as I feel that better describes its function. The Gateway Configuration configures the Gateway Proxy, specifying which ports are exposed and which protocols can be used by ingress traffic. The Gateway Configuration operates only on properties of OSI layers 4–6. You can’t configure application-layer routing rules here (this is what Virtual Services are for).

This is one of the more confusing Istio resources we’ll encounter, as it is often called simply called a in Istio documentation and can easily be confused with the Gateway Proxy described above. In fact, when we create one of these resources (which we will do shortly), we provide a value of to the kind property of the resource; however, I’ve chosen to call this resource a Gateway Configuration in the above diagram, as I feel that better describes its function. The Gateway Configuration configures the Gateway Proxy, specifying which ports are exposed and which protocols can be used by ingress traffic. The Gateway Configuration operates only on properties of OSI layers 4–6. You can’t configure application-layer routing rules here (this is what Virtual Services are for). Virtual Service: A Virtual Service defines a set of request routing rules that can be used to distribute traffic to different destinations in the service mesh. Specifically, Virtual Services define application-layer traffic routing rules, meaning that HTTP requests can be routed to different destinations based on properties like URI, request method, and headers. Similar to Gateway resources discussed above, VirtualService resources are not standalone services running on their own set of pods, instead they are simply configuration that is applied to the proxies in the mesh that actually accept and send requests. Virtual Services can be applied either to the Gateway Proxy, or to the sidecar Envoy proxies that run alongside the services for your application that are running in the mesh.

A Virtual Service defines a set of request routing rules that can be used to distribute traffic to different destinations in the service mesh. Specifically, Virtual Services define application-layer traffic routing rules, meaning that HTTP requests can be routed to different destinations based on properties like URI, request method, and headers. Similar to resources discussed above, resources are not standalone services running on their own set of pods, instead they are simply configuration that is applied to the proxies in the mesh that actually accept and send requests. Virtual Services can be applied either to the Gateway Proxy, or to the sidecar Envoy proxies that run alongside the services for your application that are running in the mesh. Destination Rule: Destination Rules define routing policies applied to traffic that has already been routed to a particular service. Additionally, we can use Destination Rules to define service subsets, which allow us to group the instances of our service by version, giving us the ability to route traffic intelligently between multiple active versions of a service without changing anything in our service code.

Destination Rules define routing policies applied to traffic that has already been routed to a particular service. Additionally, we can use Destination Rules to define service subsets, which allow us to group the instances of our service by version, giving us the ability to route traffic intelligently between multiple active versions of a service without changing anything in our service code. MyApplication Sidecar: This is the Envoy Proxy that runs alongside each instance of your deployed service. Traffic to and from your service is intercepted by this proxy.

This is the Envoy Proxy that runs alongside each instance of your deployed service. Traffic to and from your service is intercepted by this proxy. MyApplication Service: This is our application, deployed to a Kubernetes cluster as a standard Kubernetes service. Instances of this service run in pods alongside instances of the MyApplication Sidecar proxy.

This is a lot to wrap your head around, so if it doesn’t completely make sense yet, it’s okay. Hopefully the purpose of each of these Istio components will become clearer when we start to see them in action with some example services and traffic routing configurations.

Istio Traffic Management in Action

To follow along with these examples, you’ll need a running Kubernetes cluster. I’ll be using GKE on Google Cloud, but the equivalent offerings on AWS and Azure will work fine as well.

Downloading Istio

Run the following command to download Istio:

curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.7.3 TARGET_ARCH=x86_64 sh -

This will download, among other things, the istioctl command line tool which we will use to install Istio on our Kubernetes cluster.

Note: I’m using version 1.7.3 in these examples, but the latest version of Istio is likely higher by the time you read this.

Installing Istio on Kubernetes

Run this command to install Istio on your cluster using the demo configuration profile:

istio-1.7.3/bin/istioctl install --set profile=demo

Note: The path to istioctl may be slightly different on your machine if you’re using a different version of Istio.

Sample Application

The application that we will be running in the service mesh to illustrate Istio’s traffic routing capabilities will be a super simple blog website using Node.js and Express.

Our website will simply display a static html page with some example text.

Here is what the site will look like when we navigate to it (I know, pretty spectacular):

Next, we’ll need to containerize this website in order to run it on our Kubernetes cluster. Here is a Dockerfile defining a container image for the application:

Creating the Istio Components

Now let’s start creating the Istio components described earlier.

Example 1: Simple Routing

The first scenario we will demonstrate will be as simple as it gets: routing requests directly to the blog site. For this example, we will need a Gateway Configuration and a Virtual Service. We’ll define these components in a YAML manifest file:

Let’s break this down:

The first resource defined in this file (lines 1–14) is the Gateway Configuration, as specified by kind: Gateway

By using the selector app: istio-ingressgateway we’re saying that this Gateway Configuration should be applied to the istio-ingressgateway gateway proxy. If you wanted to use your own gateway proxy, you would modify this line to match your own labels.

we’re saying that this Gateway Configuration should be applied to the gateway proxy. If you wanted to use your own gateway proxy, you would modify this line to match your own labels. The server field contains the meat of the Gateway Configuration. Here we specify a list of Server Specifications, which is essentially an open port, an expected protocol, and a set of hosts that can accessed through this port. If we wanted to set up TLS or Mutual TLS (MTLS), we would configure that at the Server Specification level as well. For this simple example, we’re just exposing port 80 for HTTP traffic, and stating that all hosts are accessible through this gateway by using the wildcard "*"

field contains the meat of the Gateway Configuration. Here we specify a list of Server Specifications, which is essentially an open port, an expected protocol, and a set of hosts that can accessed through this port. If we wanted to set up TLS or Mutual TLS (MTLS), we would configure that at the Server Specification level as well. For this simple example, we’re just exposing port 80 for HTTP traffic, and stating that all hosts are accessible through this gateway by using the wildcard The second resource defined is the Virtual Service (lines 16–35).

We provide fakeblog.com in the hosts field to specify that this Virtual Service handles traffic going to the fakeblog.com domain (on my local machine, I’ve mapped the IP address of my Kubernetes cluster’s external load balancer to the domain name fakeblog.com. You could also use the public IP address instead of a domain name, or you could use the wildcard "*" to allow this Virtual Service to handle traffic meant for any host).

in the field to specify that this Virtual Service handles traffic going to the fakeblog.com domain (on my local machine, I’ve mapped the IP address of my Kubernetes cluster’s external load balancer to the domain name fakeblog.com. You could also use the public IP address instead of a domain name, or you could use the wildcard to allow this Virtual Service to handle traffic meant for any host). In the gateways field, we’re using the name of the Gateway Configuration defined in this same file. This applies the Virtual Service’s routing rules to traffic entering the mesh through this gateway.

field, we’re using the name of the Gateway Configuration defined in this same file. This applies the Virtual Service’s routing rules to traffic entering the mesh through this gateway. The http field is where we specify our Virtual Service’s routing logic. Lines 26–28 define a match condition for the following routing actions, namely that the URI begins with /myblog . Lines 29–30 describe a URI rewrite rule that replaces the URI with / (this is done so that, when the request is ultimately sent to the blog website, it will be sent to the root of the site). Lines 31–35 contain the action to be taken if the match condition is met and after the rewrite rule has been applied: the request will be routed to a service named myblog on port 80.

With our Istio routing configuration created, we now need to actually deploy the blog website to our Kubernetes cluster. Since this next YAML file contains only standard Kubernetes resources (nothing Istio-specific), we won’t go into a line-by-line breakdown, but there are a few points worth noting.

Notice the name given to the service on line 4 above. This must match the value given to the host field (line 35 in the previous file) in the Virtual Service definition.

given to the service on line 4 above. This must match the value given to the field (line 35 in the previous file) in the Virtual Service definition. On line 32, the image we’re using for this service is gnovack/my-blog:v1 . This is a public Docker image repository on Docker Hub to which I pushed the blog website image defined in the Dockerfile from earlier.

For convenience, I’ve created both of the above YAML files in the same directory like so:

1_simple_access/

├── myblog-gateway.yaml

└── myblog.yaml

Assuming you’ve followed this pattern, you can create all of the resources for this example on your cluster with the following command:

kubectl apply -f 1_simple_access/

Now, we can go check out our underwhelming blog website by navigating to http://fakeblog.com/myblog (assuming the domain name fakeblog.com has been mapped to the public IP address of the Kubernetes cluster external load balancer, if not, you can simply use the public IP).

So, this proves that we can access the blog website through Istio’s components, but that isn’t really all that exciting. Let’s take a look at something a little more interesting in the next example.

Example 2: Weighted Routing

In this example, we’re going to deploy a second version of the blog website, then use Istio’s traffic management capabilities to perform weighted traffic routing between both versions.

First, we’ll need to create a new version of the blog website. To keep it simple, all we’re going to change is the header color; version 2 will have a blue header.

Next, we’ll push a container image for the new website version to the same Docker image repository used previously, meaning we will now have both a v1 and v2 version of the image.

Now let’s reconfigure the Istio components created earlier to enable weighted routing. We’re going to run both versions of the blog website (red & blue) in the Kubernetes cluster, then use Istio to route 75% of requests to the original version, and 25% to the new version.

Since the Istio Gateway Configuration is just concerned with exposing ports and protocols, we won’t need to make any changes to it to support the weighted routing scenario; however, we will need to modify the Virtual Service created earlier.

Here is the new Virtual Service definition for our weighted routing setup:

A lot of this (lines 1–16) is the same as before. We’re still using the same hostname, Gateway Configuration, match condition, and URI rewrite rule. The difference is in the route field, that is, the action taken if the match condition is met. Whereas before we were simply routing the request to the myblog service on port 80, now we have two different destinations defined. They both use port 80, and both ultimately send the request to the myblog service (lines 20 & 26), but they are distinguished by the values of the subset and weight fields. Weight is fairly self-evident; this is the proportion of total traffic that will be sent to that particular destination (75% goes to the first destination and 25% goes to the second destination in the example above). But what is a subset?

Subset is a property of Istio Destination Rules. We mentioned Destination Rules earlier, but didn’t need to use them for the first example. Destination Rules allow us to create routing policies to apply to traffic bound for a particular service, and to logically group instances of that service into subsets.

Let’s define a Destination Rule now that groups the instances of our service into v1 and v2 subsets based on whether they are running version 1 (red) or version 2 (blue) of the blog website.

This Destination Rule should be pretty easy to make sense of. We’re giving it a name (line 4), specifying that it applies to the myblog service (line 6), and then defining two subsets (v1 & v2; these names must match the value of the subset field in the Virtual Service) of our blog service (lines 7–13). The real question is, “How does this Destination Rule know which instances of the blog service are running version 1 and which are running version 2?”. This is accomplished by the labels field in each subset. The Destination Rule will group particular pods running the myblog service into it’s subsets based on the labels assigned to those pods. For example, any pod with the label version:v1 will be considered part of the v1 subset.

So now the last thing to do is to redeploy the blog website, this time with some pods running version 1, and other pods running version 2, ensuring that the appropriate labels are assigned to each.

Here is the same definition from earlier for the myblog service, although this time with a separate deployment for each version of the website (notice that the deployments assign different labels and make use of different container images).

Now, if we deploy the resources to the cluster and navigate back our blog site at http://fakeblog.com/myblog we will see, after refreshing the page several times, that we are shown the red blog page about 75% of the time, and the blue blog page the other 25%.

By using Istio’s routing capabilities, we were able to set up a simple A/B test where a certain portion of our customers are shown a new experience, without making any modification to our application code to enable this. If the new version isn’t a success, we can easily modify the routing configuration to send all requests back to the original version, or if we feel we need more data on the new version, we can just as easily modify the weights in our Virtual Service to make this happen.

Example 3: Routing with Request Parameters

Another common scenario where we can leverage Istio Traffic Management is when we want to expose a new version of our service to only a specific set of users, such as internal users or beta testers. Let’s look at one way to accomplish this using Istio.

Instead of simply routing a specified percentage of traffic to each version of our service, Istio also allows us to configure routing based on properties of each HTTP request. In this example, we’ll modify our Virtual Service to route traffic based on the value of a query string parameter, specifically if the user query parameter has a value of internal, we will route the request to version 2, otherwise the request will go to version 1 (I chose to use query string parameters since these are easy modify directly in the browser, but you can also use other properties like URI or headers. A full list of properties that can used to match a request to a destination can be found in the Istio reference docs: HTTPMatchRequest).

Let’s see how to do this with our updated Virtual Service definition:

As you can see, we now have 2 different match conditions. When multiple match conditions are provided, they are evaluated sequentially, and the first route for which the request satisfies the match condition is chosen. For the Virtual Service above, this means that the request is first evaluated against the first match condition (i.e. does the URI begin with /myblog AND does the request have a user query string parameter with a value of internal?). If the request satisfies these conditions, then it is routed to version 2, of not, then it is checked against the next match condition (and so on, if there are more than 2 match conditions).

If we redeploy the Virtual Service with this definition, we will be able to reach the new version of the service by navigating to http://fakeblog.com/myblog?user=internal; all other requests will continue to be routed to the old version. In a real-world scenario, you might add this parameter programmatically based on on the IP address of the user, or based on some kind of user identifier if your application is one that requires users to log in.

Conclusion

We’ve looked at just a few of the myriad use cases where Istio Traffic Management can be leveraged to simplify the development and deployment of microservices. Specifically, we saw how to intelligently route between different versions of a service by using either weights or request parameters to support processes like A/B testing or beta releases; and, most importantly, we accomplished this by simply configuring and reconfiguring our Istio components rather than creating (and probably later removing) custom logic in our application to support these scenarios.

We didn’t have a chance to look into many of the great features that Istio offers (maybe next time), such as Fault Injection, Retry Policies, and Mirroring, but hopefully this article helped establish an understanding of the foundational concepts of Istio that can help you start exploring its many other capabilities and components on your own.

Thanks for reading. Feel free to reach out with any questions or comments.

Source Code

All of the code used in the examples can be found here: https://github.com/gnovack/istio-routing

References