Autoscaling spring boot applications running on K8s cluster in AWS
By: Raghuram Gururajan
Tags: Spring Boot, Kubernetes, AWS, Docker, Spring

Photo by Dominik Lückmann on Unsplash

In this article, we will understand how we can create a simple spring boot micro service and deploy on Kubernetes cluster running on AWS and autoscale the application. After reading this article you will be able to deploy your own micro service on AWS and autoscale based on predefined parameters.

You might be wondering if we are able to deploy the Spring boot micro service on EC2 VM’s and individually scale these instances do we still need containers? If we want to achieve true scaling, low cost, faster deployment and consistency then containers are a must for micro services. Individual applications running on containers can be scaled independently of each other.

Time to deploy and start application on containers is much faster compared to bare metal machines or VM’s. Overall, containers start up time is in seconds, also container orchestration takes care of better utilization of resources. One of the most popular tool used for container orchestration is Kubernetes. It provides platform for container orchestration along with other useful services including container creation, scale-up/down , rollbacks, load balancing and service discovery.

Basics:

Containers are an abstraction at the app layer that packages code and dependencies together as below. By containerizing the application platform and its dependencies alone, differences in OS distributions and underlying infrastructure are abstracted away. Virtual machines are abstraction of physical hardware turning one server into many servers. Containers by contrast, isolate application’s execution environment from one another, but share the same underlying OS kernel.

Most of our web applications consist of several components like web server, database, cache etc and containers make it possible to compose these services into functional units. Each of these units can be treated as a separate container and can be maintained, updated and modified independently of others. Lightweight and portable containers make it easier for us to build and maintain micro services based applications.

What is Docker ?

Docker is an open source project that was launched in 2013 to popularize the container technology and its a platform that helps developers build light weight and portable software containers.

Docker consist of a docker engine that is a core component and it has a server and client, communication between the two is via Rest API. The server contains a docker daemon called dockerd which can create and manage containers. The Rest API is used to instruct docker daemon what action to perform. The client is a CLI(command line interface) that is used to enter docker commands

The main components of docker are:

1)Docker client and server

2)Docker image

3)Docker registry

4)Docker container

Docker Client and Docker Server

Docker client(Command Line Interface-CLI) is the primary way to interact with docker. When the user types command such as docker pull command, the client sends an instruction to the docker daemon to perform the operation by interacting with other components(image, container and registry). The communication between the Docker Client and Docker host is through REST API.

Docker daemon is actually a server that interacts with operating system and performs services. It constantly listens to the Docker API requests issued by client to see if it needs to service any requests and manages docker objects like images, containers, networks and volumes.

Docker image

Docker image can be considered as a template that contains instructions for docker container. The template is defined in a language called YAML (Yet Another Markup Language). Docker image is built using YAML and stored as a file in the docker registry and can be treated as a portable file that contains specification of which components the container will run. Each image consist of several key layers and each layer depends on the layer below it. Image layers are created by executing each command in the docker file.

From the above docker image we can see that there are four layers of instructs

From : Base image to build our image

: Base image to build our image Pull : Adds file from docker repository

: Adds file from docker repository Run : Builds your container

: Builds your container Cmd: Specifies which command to run within the container

Docker Registry

Docker registry is a collection of docker images and it’s a repository to host various types of images. Docker Hub is one of publicly accessible docker registry that contain official docker images from open source projects and software vendors and also unofficial images from general public. You can provide a tag name with each image so that it is easy for people to find and share them in registry. You can also create your own registry for your use internally. The main commands used for interaction with registry are PUSH and PULL. PULL command retrieves the docker image from registry and PUSH command is used to push a latest created image to registry.

Docker Container

Docker container is a runnable instance of container image. It is portable and can be considered as an executable package of application and dependencies together. A container is defined by its image and also the configuration options and by default container is well isolated from other containers and the host machine. A container can be started by a simple command using docker RUN as below

docker run ubuntu

If the image is not available locally then as the first step the image is downloaded from registry and an instance of container will be created from the image and it becomes available for your execution.

What is Kubernetes ?

Containers are lightweight, flexible and faster but with increased number of containers and management of the same if we don’t take right actions to manage them it will affect the availability of the system.

Kubernetes provides the platform to automate activities like container creation, deployment, replacement and scale-up/down of containers, rollbacks etc. It also provides other useful services for micro services like Service Discovery, Load Balancing and Fault Tolerance etc. The good part is with Kubernetes we don’t need any additional framework to implement these additional services and they are provided out of box for us to leverage the same.

Pods

Pods are basic building blocks of Kubernetes. To reduce coupling the best practice is to use single container pods. If we need two instances of our service running, we will need to have two instances of pods running.

Controllers

Controller consist of individual process that monitor the state of cluster and continuously aim in driving the current state towards the provided desired state of the cluster. Example of controllers are Deployment , ReplicaSet and NodeController.

Services

Services add a layer of abstraction over the pods. The abstraction defines a policy through which pods are accessed. Services also load balance request across pods.

Volumes

Containers are ephemeral and once the container crashes , all its data is lost. Volumes help in restoring container to the previous known state. Volumes provide storage for the container and they can be attached to any pod.

ConfigMaps and Secrets

ConfigMaps and Secrets allow users to configure applications externally. Allows passing sensitive information without hardcoding the same inside the container.

Deployments

Deployment helps in transitioning current environment to the desired state in a controlled manner. Deploys replication controller or replica set which further deploys the pods.

StatefulSet

StatefulSet is the ReplicaSet equivalent for stateful application. Maintains uniqueness of each pod across different deployments and scaling operations.

Basic Architecture of Kubernetes Cluster

As you can see above the basic architecture building blocks of kubernetes consist of a Master Node and one or more Worker Nodes. The primary components and their role are as below

Controller Manager

This is the primary component of the Master node that runs controller processes related to replication, endpoints, node and service account. Controller manager ensures that it monitors and responds to situation when nodes go down or ensuring pod replication etc.

Scheduler

Scheduler is responsible for allocating a node when a pod is created.

API Server

API server is the interface for Kubernetes control plane. All interactions have to go through API Server. External and internal components in Kubernetes cluster interact through API Server.

Kubelet

Kubelet is an agent that runs on each node and ensures that the containers inside the pod are running.

Container Runtime

Container runtime is the implementation of runtime engine responsible for running containers. Kubernetes supports different container runtimes like Docker, CRI-O, Container etc.

Kube-Proxy

Kube-Proxy is a component that allows the external access of internally deployed applications running on pod. Kube proxy configures the network rules to allow application access

Different modes for configuring network rules include

Userspace

IPtables

IPVS

ClusterStore

Cluster store is a distributed store used to store all data that is used to manage the cluster. ETCD is a distributed key-value cluster store used in Kubernetes. When we have multiple nodes and multiple masters in the cluster, etc maintains all the information of the nodes in the cluster in the distributed store.

What happens when you create a pod in Kubernetes ?

Lets explore under the hood what happens when you try to run the below command to create a pod using kubectl

kubectl run web - image=nginx - replicas=3

Step1 : After running the above command Kubectl generates a manifest describing the deployment

Step 2 : The manifest is sent to Kubernetes API Server

Step 3 : Kubernetes API server validates the manifest and persist to etcd

Step 4: Controller manager has a controller for deployment that notices that new deployment has been created and it creates a replica set based on deployment.

Step 5: ReplicaSet controller creates POD objects are stores them in etc but there still details to be added to the POD object like the node details etc . They are in PENDING state since they are not scheduled on any node

Step 6 : Scheduler notices that there are PENDING pod objects and tries to schedule these PODS on nodes that have resources available in the cluster

Step 7: The PODS now are now actually created on the nodes and their status updated is updated with node information and RUNNING state in etc and we have successfully completed our workflow.

Kubernetes in AWS

The below diagram shows a typical Kubernetes running on a typical two node cluster in AWS.

Core concepts in AWS Eco System

VPC

Amazon Virtual Private Cloud (VPC) is a private subsection of AWS that lets you provision a logically isolated section of the Amazon Web Services where you can launch AWS resources in a virtual network that you define. You have full control over who has access to the AWS resources that you place inside your VPC.

Route 53

Amazon Route 53 is a highly available and scalable Domain Name Service (DNS). Route 53 translates the domain name into IP addresses to direct traffic to your website.

Internet Gateway

Internet Gateway is a logical connection between Amazon VPC and the internet. An internet gateway has two main goals

a)Provide target in your VPC route tables for internet-routable traffic

b)Perform network address translation for instances that have been assigned public IP addresses.

ELB

AWS Elastic Load Balancer automatically distributes incoming application traffic across multiple Amazon EC2 instances. It helps you achieve fault tolerance in your application by providing the required amount of load balancing capacity needed to route application traffic. Different types of load balancer that AWS provides are

Network Load Balancer (TCP/SSL)

Application Load Balancer (HTTP/HTTPS)

Network Access Control List

A network access control list is an optional layer of security for your VPC that acts as a firewall controlling traffic in and out of one or more subnets

Subnet

A subnet is a sub-section of a network which includes all computers in a specific location.

Availability Zone

Any AWS resource that we launch must be placed in a VPC subnet. Any given subnet must be located in an availability zone. You can utilize multiple availability zone to create redundancy in your architecture. This is what allows for High Availability and Fault Tolerant systems.

Security Group

Security groups are similar to NACLs, they allow or deny traffic. However security groups are found only at the instance level. Security group acts like a virtual firewall that controls the traffic to one or more instances. We add rules to each security group that allow traffic to or from its associated instances. You can modify the rules for a security group at anytime, the new rules are automatically applied to instances within the group.

Spring boot Application(WeatherAPI service)

Now let us take a look at our spring boot application. Its a simple web application that returns you the current weather given a city and country as a parameter. Lets take a look at the structure of the web application

The main classes of our application are

WeatherApiDemoApplication

This is the main class to bootstrap or launch a spring application. This class automatically creates the application context from class path, scans the configuration classes and launches the application.

WeatherService

This class contains all the business logic w.r.t to getting the current weather info. This class also interfaces with api.openweathermap.org and retrieves the details of weather based on city, country etc.

CurrentWeather

This class represents the model that holds the weather related info and this model is populated with weather related info by WeatherService and returned back to WeatherController. CurrentWeather model contains attributes like city, country, temperature, wind speed etc

WeatherController

WeatherController is the class that implements the RESTFul web services for our weather api. It takes care of mapping the request data to the define request handler method. WeatherController integrates with WeatherService. To get the weather data based on input parameters and once the response body is generated it converts into necessary format and returns back to the user.

The default city and country is San Jose and USA respectively. A sample invocation of the API would return something as below.

curl -X GET http://localhost:8080/current-weather?city=Georgia&country=USA

Output:

{"description":"Clouds","temperature":11.7,"feelsLike":10.98,"windSpeed":0.94,"country":"USA","city":"Georgia"}

The core dependencies of our application are

Spring boot starter for web/tomcat

Micrometer

Actuator

Sample pom.xml dependencies

<dependencies>

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-web</artifactId>

</dependency>



<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-tomcat</artifactId>

<scope>provided</scope>

</dependency>

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-test</artifactId>

<scope>test</scope>

</dependency>

<dependency>

<groupId>io.micrometer</groupId>

<artifactId>micrometer-registry-prometheus</artifactId>

</dependency>



<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-actuator</artifactId>

</dependency>

</dependencies>

Actuator:

Spring boot actuator provides secured endpoints for monitoring and managing your application. Actuator is mainly used to expose operational information (health, metrics, info, dump, env) about your running application .

You can find more information about Actuator here

Prometheus:

Prometheus pulls metrics(keys/values) from application and stores the data as time-series, allowing users to query data and alert in a real time fashion. Prometheus will hit targets to collect metrics, aggregate data, show data or alert if some threshold is met.

As you can see above there are three components of a prometheus server

Metrics retrieval component ( To pull metrics from multiple servers/machines)

Metrics storage component ( Stores metrics data in a time series fashion)

HttpServer ( provides API to expose metrics to the outside world for querying)

Simple example of metrics displayed on prometheus endpoint for our weather application. As you can see below we are able to track the number of requests for our endpoint /health and /current-weather .

Lets build and push our spring boot weather app to docker hub and we can pull it later on our Kubernetes pods when creating a deployment

The docker file used for my weather app is very simple as below

FROM tomcat:8.0.51-jre8-alpine EXPOSE 8080 RUN rm -rf /usr/local/tomcat/webapps/* COPY target/*.war /usr/local/tomcat/webapps/ROOT.war CMD [“catalina.sh”,”run”]

I am also using a dockerfile-maven-plugin from Spotify to build docker image of the weather app The plugin configuration is as below. The plugin will build a docker image as part of

mvn clean install

The main plugin section in pom.xml looks as below

<plugins>

<plugin>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-maven-plugin</artifactId>

</plugin>

<plugin>

<groupId>com.spotify</groupId>

<artifactId>dockerfile-maven-plugin</artifactId>

<version>1.4.10</version>

<executions>

<execution>

<id>default</id>

<goals>

<goal>build</goal>

</goals>

</execution>

</executions>

<configuration>

<repository>raghuramg24/${project.name}</repository>

<tag>${project.version}</tag>

<skipDockerInfo>true</skipDockerInfo>

</configuration>

</plugin>

</plugins>

Also make sure that you are configuring your repository name and also make sure that you are doing docker push after the image is built to docker hub.

Auto scaling applications in Kubernetes

There are two basic principles when it comes to scaling applications

a) When there is increase in traffic or number of requests, the application should automatically scale up to meet the demand

b)When there is decrease in traffic the application should automatically scale down to not waste resources

Auto scaling options in Kubernetes

Kubernetes provides the below options for autoscaling our applications

Horizontal Pod Autoscaler

Vertical Pod Autoscaler

Cluster Autoscaler

Horizontal Pod Autoscaler auto scales the number of replicas in the system. Vertical Pod Autoscaler increases and decreases the compute resources of container. Cluster Autoscaler scales the number of nodes in a cluster.

Each of these types of auto scaling components are used for different use cases and in this article we will be discussing about Horizontal Pod Autoscaler. Horizontal Pod Autoscaler can monitor specific metrics of your application continuously and adjust the number of replicas to meet the demand.The number of replicas created is based on user provided target value for the metric.

Metrics Registry is the component where prometheus is integrated with Pod AutoScaler to help with metric collection and exposing the same to POD autoscaler.

Now we are going to look at how we can set up a simple Kubernetes cluster on AWS and deploy our application. After ensuring our application is successfully running in the Kubernetes pods we will look at auto scaling.

HorizontalPODAutoscaler can scale the number of pods based on memory, CPU or any other custom metrics. The custom metrics are pulled using Custom Metrics API. We will have only one instance of our Spring boot application deployed on a POD initially and all the metrics related to the same will be obtained by prometheus by querying the end point /actuator/prometheus. Subsequently based on the config setting on the HorizontalPODScaler and the metric value it will scale up the pod if the value of metric exceeds the assumed value.

Set up of Kubernetes cluster in AWS

The steps below indicate the different steps in setting the AWS instances and deploying our application on Kubernetes and auto scaling the same.Lets go through the different steps in detail now.

1.Creating VPC:

As a first step, let us create a VPC with CIDR as 10.0.0.0/16. For each AWS resource that we create we need to associate the tag “kubernetes.io/cluster/kubernetes” with the value “owned”. This tag will be used by K8s for auto discovering AWS resources related to Kubernetes stack.

2.Enable the DNS hostnames

3.Create Subnet in the above VPC

Lets create subnet in the VPC(kubernetes-cluster-vpc) that we created above. Create it with CIDR 10.0.0.0/24 and add our tag “kubernetes.io/cluster/kubernetes” with value “owned”

4.Auto Assign IP

After the subnet is created lets enable the auto assign IP settings

5.Internet Gateway

In order for us to have the traffic routed from subnet into the Internet we need Internet Gateway. Lets also associate the Internet Gateway to our VPC(kubernetes-cluster-vpc) created above.

Add tag “kubernetes.io/cluster/kubernetes” with value “owned”

6.Route table

A route table contains a set of rules, called routes that is used to determine how to direct the traffic from the gateway or subnet. Each subnet in the VPC must be associated with a route table, which controls the routing for the subnet. Each route in a route table specifies a destination and a target and that determines which way the network packets need to go to get to their destination.

Create a route table and associate with the above VPC(kubernetes-cluster-VPC) and also add our tag

Edit the routes after the route table is created and add 0.0.0.0/0 (CIDR notation equivalent to 0.0.0.0 with a mask of 0.0.0.0), because if you make a binary comparison between the destination IP and the mask 0.0.0.0, it matches any IP address including the 0.0.0.0 IP itself.

Attach this route table to the subnet(kubernetes-cluster-net) that we created earlier

7.IAM Roles

In order to get Kubernetes working with AWS we need to set up roles/policies in IAM for master and worker nodes. Lets see how to create them.

IAM Master Role/policy:

We need to first define IAM master policy by going to IAM > Create policy from AWS console and click on JSON tab as below. You can refer to the policy for control plane here and copy the JSON into the create policy JSON tab

The JSON will look something like below

{

"Version":"2012–10–17",

"Statement":[

{

"Effect":"Allow",

"Action":[

"autoscaling:DescribeAutoScalingGroups",

"autoscaling:DescribeLaunchConfigurations",

"autoscaling:DescribeTags",

"ec2:DescribeInstances",

"ec2:DescribeRegions",

"ec2:DescribeRouteTables",

"ec2:DescribeSecurityGroups",

"ec2:DescribeSubnets",

"ec2:DescribeVolumes",

"ec2:CreateSecurityGroup",

"ec2:CreateTags",

"ec2:CreateVolume",

"ec2:ModifyInstanceAttribute",

"ec2:ModifyVolume",

"ec2:AttachVolume",

"ec2:AuthorizeSecurityGroupIngress",

"ec2:CreateRoute",

"ec2:DeleteRoute",

"ec2:DeleteSecurityGroup",

"ec2:DeleteVolume",

"ec2:DetachVolume",

"ec2:RevokeSecurityGroupIngress",

"ec2:DescribeVpcs",

"elasticloadbalancing:AddTags",

"elasticloadbalancing:AttachLoadBalancerToSubnets",

"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",

"elasticloadbalancing:CreateLoadBalancer",

"elasticloadbalancing:CreateLoadBalancerPolicy",

"elasticloadbalancing:CreateLoadBalancerListeners",

"elasticloadbalancing:ConfigureHealthCheck",

"elasticloadbalancing:DeleteLoadBalancer",

"elasticloadbalancing:DeleteLoadBalancerListeners",

"elasticloadbalancing:DescribeLoadBalancers",

"elasticloadbalancing:DescribeLoadBalancerAttributes",

"elasticloadbalancing:DetachLoadBalancerFromSubnets",

"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",

"elasticloadbalancing:ModifyLoadBalancerAttributes",

"elasticloadbalancing:RegisterInstancesWithLoadBalancer",

"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",

"elasticloadbalancing:AddTags",

"elasticloadbalancing:CreateListener",

"elasticloadbalancing:CreateTargetGroup",

"elasticloadbalancing:DeleteListener",

"elasticloadbalancing:DeleteTargetGroup",

"elasticloadbalancing:DescribeListeners",

"elasticloadbalancing:DescribeLoadBalancerPolicies",

"elasticloadbalancing:DescribeTargetGroups",

"elasticloadbalancing:DescribeTargetHealth",

"elasticloadbalancing:ModifyListener",

"elasticloadbalancing:ModifyTargetGroup",

"elasticloadbalancing:RegisterTargets",

"elasticloadbalancing:DeregisterTargets",

"elasticloadbalancing:SetLoadBalancerPoliciesOfListener",

"iam:CreateServiceLinkedRole",

"kms:DescribeKey"

],

"Resource":[

"*"

]

}

]

}

You can give a name for the policy like kubernetes-iam-master-policy and save the same

Now create a IAM role for the above created policy as below

Also add the tags as below

Now create a policy for the worker node as below

You can look at this example here and copy the details for Node policy for worker and create a worker policy

Now lets create a role with the above created policy with tag kubernetes.io/cluster/kubernetes

8.Creating Ec2 instances

Create two Ec2 instances one for master and one for worker with corresponding roles. Make sure that we are associating the above created subnet (kubernetes-cluster-net), VPC (kubernetes-cluster-VPC) with the instances. We have also make sure that we have the tag kubernetes.io/cluster/kubernetes added to the instances. I will be using Ubuntu Server 18.04 LTS image for creating EC2 instances .

We have to select instance type that has at least 2 CPU’s when installing Kubernetes master and therefore I will be using t2.medium for both master and worker.

Create security group

Lets create security group called kubernetes-security-group to allow all traffic to the instance

Create Key pair

Before you launch the instance AWS will prompt you to create keys that will help you to access the instance in future. Store the keys in a folder where you access at a later point for installing kubernetes. Make sure that the permissions on the keypair file is changed to 600.

Steps for worker node

Follow the above steps for worker node also and do remember to selected worker role and the security group kubernetes-security-group created earlier. Also do remember to select the key created earlier before launching the instance.

9.Installing Kubernetes

Perform the below steps on both the instances of Ec2

Update packages list and installed packages

apt update && apt -y upgrade

Adding Docker and Kubernetes repositories

Update and install docker ,kubelet,kubeadm,kubectl

apt install -y docker-ce kubelet kubeadm kubectl

Get full qualified hostname and set it as hostname

Set the hostname as FQDN

hostnamectl set-hostname ip-10–0–0–25.ec2.internal

Verify the hostname

ip-10–0–0–25.ec2.internal

Initialization of Master Node

Create a master yaml with the below configuration

apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration networking: serviceSubnet: “10.100.0.0/16” podSubnet: “10.244.0.0/16” apiServer: extraArgs: cloud-provider: “aws” controllerManager: extraArgs: cloud-provider: “aws”

Initialize the Custer with the above config using the command

kubeadm init — config master.yaml

After you run the command successfully, you should notice the output as below

Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 10.0.0.35:6443 — token v2w4nw.zss3ev6ur7jpcsqy \ — discovery-token-ca-cert-hash sha256:33979f52b7e6fa5e549ee192d17c3cc0b5b5b98639ced6036f83a9938934c436

Make a note of the hash,token and the api server endpoint (i.e 10.0.0.35:6443) we may need all of them when setting up the worker

As the next step run the above commands on your master node , these commands were listed as part of your initial output of kubeadm command

mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config

Kubernetes Networking

In Kubernetes, every Pod has a real IP address and each Pod communicates with other Pods using that IP address. Within the same node direct pod-to-pod communication is possible without requiring any kind of proxy or address translation. But when we have PODS that need communication across different nodes we need CNI (Container Network Interface)

Container Networking Interface, is basically an external module with a well defined interface that can be called by kubernetes to execute actions to provide networking functionality. There are different flavours of CNI implementation that are available in the market and Flannel is one of them and we will be using Flannel in our article.

Installing Flannel:

Execute the below command on the master node to install Flannel network.

You should see output similar to one below

podsecuritypolicy.policy/psp.flannel.unprivileged created clusterrole.rbac.authorization.k8s.io/flannel created clusterrolebinding.rbac.authorization.k8s.io/flannel created serviceaccount/flannel created configmap/kube-flannel-cfg created daemonset.apps/kube-flannel-ds created

Lets check the status of node to make sure that our master node has a ready status by executing the below command

kubectl get nodes

Initialization of Worker node

On the worker node that you have created make sure to use the below YAML file to initialize the worker with the token, hash and the api server endpoint generated above in the master node during kubeadm initialization.

apiVersion: kubeadm.k8s.io/v1beta1

kind: JoinConfiguration

discovery:

bootstrapToken:

token: "v2w4nw.zss3ev6ur7jpcsqy"

apiServerEndpoint: "10.0.0.35:6443"

caCertHashes:- "sha256:33979f52b7e6fa5e549ee192d17c3cc0b5b5b98639ced6036f83a9938934c436"

nodeRegistration:

name: ip-10–0–0–147.ec2.internal

kubeletExtraArgs:

cloud-provider: aws

Run the below command to join the node to the cluster

kubeadm join — config=worker.yaml

You should see the output as below

You can verify if your worker node has joined by executing the below command on master

kubectl get nodes

You should be able to see two nodes in the cluster now

10.Installing Prometheus

Prerequisites:(Master)

Install Curl:

sudo apt install curl

Install Helm

curl -sSL https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo add stable https://charts.helm.sh/stable/

Update the repo

helm repo update

Install prometheus through helm

helm install prometheus prometheus-community/prometheus

After installing prometheus ensure that you have all your pods in a running state by executing the below command

kubectl get pods — all-namespaces

Deploying Spring boot application on Kubernetes

Let us deploy our spring boot application on Kubernetes using weatherapidemo.yaml similar to the one below. Prior to execution of this step awe need to ensure that our image weatherapidemo:0.0.1-SNAPSHOT is pushed to the docker hub repository

apiVersion: apps/v1

kind: Deployment

metadata:

name: weatherapidemo

spec:

selector:

matchLabels:

app: weatherapidemo

template:

metadata:

annotations:

prometheus.io/path: /actuator/prometheus

prometheus.io/scrape: "true"

prometheus.io/port: "8080"

labels:

app: weatherapidemo

spec:

containers:

- name: weatherapidemo

image: raghuramg24/weatherapidemo:0.0.1-SNAPSHOT

ports:

- containerPort: 8080

You can create deployment resource using the below command for our weatherapidemo

kubectl create -f weatherapidemo.yaml

To verify if the deployment is successful we can execute the below command

kubectl get deployments

You can see output similar to the above if the deployment is successful. We should be able to see our deployment weatherapidemo in the list.

Check if there are corresponding pods created with weatherapidemo corresponding to the above deployment by executing

kubectl get pods — all-namespaces

As you can see from the above list that our pod with weatherapidemo container is up and running

After successfully executing the above command and creating the adapter try to find out the IP address for our weatherapi demo and try executing couple of times the below command to make sure you are getting the output. Please note that you need to replace the IP address (10.244.1.6) with your own pod IP address.

curl — location — request GET 'http://10.244.1.6:8080/current-weather?city=Milpitas&country=USA'

Install Prometheus adapter for capturing metrics from our application

As we saw from our above Spring boot application weatherapidemo that it exposes the metrics like http_server_requests_seconds_count for each of the Rest API that the application exposes. Prometheus adapter pulls metrics from the Prometheus instance and exposes them as the Custom Metrics API. In this step, you will have to provide configuration for pulling a custom metric exposed by the Spring Boot Actuator. Let’s define a custom rule to pull the desired metric from Prometheus.

prometheus:



url:

port: 80

path: "" prometheus:url: http://prometheus-server.default.svc port: 80path: "" rules:

default: true

custom:

- seriesQuery: '{__name__=~"^http_server_requests_seconds_.*"}'

resources:

overrides:

kubernetes_namespace:

resource: namespace

kubernetes_pod_name:

resource: pod

name:

matches: "^http_server_requests_seconds_count(.*)"

as: "http_server_requests_seconds_count_sum"

metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>,uri=~"/current-weather.*"}) by (<<.GroupBy>>)

Note that the URI contains the endpoint for which we are capturing metrics as “/current-weather”.

Lets execute the below command to install the prometheus adapter as below

helm install -f weatherapiconfig.yaml prometheus-adapter prometheus-community/prometheus-adapter

After executing the above command u can try executing the below url to see if you are able to see the values captured and returned as part of your endpoint

kubectl get — raw /apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/*/http_server_requests_seconds_count_sum

Your output should look like something as below. Note that I have hit the endpoint /current-weather 3 times and therefore it shows “3” in the value section.

{“kind”:”MetricValueList”,”apiVersion”:”custom.metrics.k8s.io/v1beta1",”metadata”:{“selfLink”:”/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/%2A/http_server_requests_seconds_count_sum”},”items”:[{“describedObject”:{“kind”:”Pod”,”namespace”:”default”,”name”:”weatherapidemo-7d8d5f9d6b-kr8hb”,”apiVersion”:”/v1"},”metricName”:”http_server_requests_seconds_count_sum”,”timestamp”:”2020–12–26T00:45:38Z”,”value”:”3",”selector”:null}]}

Install Horizontal POD autoscaler of Kubernetes

This is the final step that will help us in auto scaling our spring boot application. HorizontalPodAutoscaler is capable to autoscaling the number of instances of our PODs based on the metric name and average value provided. A sample autoscaler created for our weatherapidemo would like below.

apiVersion: autoscaling/v2beta2

kind: HorizontalPodAutoscaler

metadata:

name: weatherapidemo

namespace: default

spec:

scaleTargetRef:

apiVersion: apps/v1

kind: Deployment

name: weatherapidemo

minReplicas: 1

maxReplicas: 10

metrics:

- type: Pods

pods:

metric:

name: http_server_requests_seconds_count_sum

target:

type: AverageValue

averageValue: 6

As you can see from the above HorizontalPodAutoScaler spec details we have the minimumReplicas set to 1 and maximumReplicas set to 10. This config indicates that as part of scale up there needs to be at least 1 minimum pod created and a max of 10 pods created. The average value indicates the threshold for the number of request i.e when the number request per second goes beyond 6 then the pods for weatherapidemo are autoscaled. Lets see the current number of pods for weatherapidemo using the below command

kubectl get pods — all-namespaces | grep “weatherapidemo”

You should be seeing only one instance of weatherapidemo running

Create the pod autoscaler by running the below command

kubectl apply -f podautoscaler.yaml

Try to execute the CURL command multiple times one after another and then you start to see that your PODS are getting created automatically as below

curl — location — request GET ‘http://<podipaddress>:8080/current-weather?city=Milpitas&country=USA&appId=888438ee801053da344ce7bdc04d8e42’

Autoscaling is working!!!!

Lets also try to create a service type LoadBalancer so that you can hit this request outside of this cluster and you can see them autoscaling

kind: Service

apiVersion: v1

metadata:

name: weatherapidemo

spec:

type: LoadBalancer

selector:

app: weatherapidemo

ports:

- name: http

protocol: TCP

# ELB's port

port: 80

You can see that once the above service is created there is a corresponding external load balancer also created in AWS and you can notice your instance id also part of the load balancer

Lets see the steps involved in redirecting the external traffic to the POD using ELB

NodePort exposes a service externally to the cluster by means of the target nodes IP address and the NodePort

TargetPort is the port on which the service will send requests to, that your pod will be listening on

Container Port exposes the Kubernetes service on the specified port within the cluster. Other pods within the cluster can communicate with this server on the specified port

Now we can also hit the AWS ELB url and try to auto scale our weatherapidemo pods

curl — location — request GET ‘http://a37966cad0aa0428fb59b06a790525f6-2126778632.us-east-1.elb.amazonaws.com:8080/current-weather?city=Milpitas&country=USA&appId=888438ee801053da344ce7bdc04d8e42'

And after sending subsequent requests one after another and once we reach the threshold of 6 we can start to see the PODS getting auto created as below

We have successfully completed setting up of kubernetes cluster on AWS, deploying our application and autoscaling the same.