Build a highly scalable dashboard that runs on Kubernetes
By: Arnold Lutsch
Tags: Docker, Flask, Dash, Kubernetes, Python3

Deployment Process in Kubernetes

Since this is my first implementation in this environment, I will give you a step-by-step guide to what I have done. I hope that the explanations help you to get a good start on the topics! Here we go… :):

First, I applied to the Google Cloud Platform for a free trial subscription. (Any other platform running Kubernetes can be used as well) Create a Kubernetes Cluster

Go to “Kubernetes Engine” → “Create Cluster” → choose “Your first cluster”

Create a Kubernetes cluster

Change the name in the parameters to dashboard-web, you can apply the remaining settings. Then click on Create to build the Kubernetes cluster called dashboard-web.

Now a cluster with one node is created on which up to 110 Pods can be created.

Create a Kubernetes Cluster over REST or command line

Hint: The cluster configuration you made is stored and provided as script and can be executed via a REST-API or via the command line. Thus it is possible to automate a cluster creation. Here’s an example for command line code you can use in the Cloud Shell:

gcloud beta container --project "<YOUR-PROJECT>" clusters create "dashboard-web" --zone "us-central1-a" --no-enable-basic-auth --cluster-version "1.15.4-gke.22" --machine-type "g1-small" --image-type "COS" --disk-type "pd-standard" --disk-size "30" --scopes "https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring","https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly","https://www.googleapis.com/auth/trace.append" --num-nodes "1" --no-enable-cloud-logging --no-enable-cloud-monitoring --enable-ip-alias --network "projects/<YOUR-PROJECT>/global/networks/default" --subnetwork "projects/<YOUR-PROJECT>/regions/us-central1/subnetworks/default" --default-max-pods-per-node "110" --addons HorizontalPodAutoscaling,HttpLoadBalancing --enable-autoupgrade --enable-autorepair

3. Containerize a Python App

To get the dashboard - which was developed in Python - running in a docker container, 3 files are needed:

index.py (file containing the python code of the dashboard)

(file containing the python code of the dashboard) Dockerfile (build images automatically by reading the instructions from this file)

(build images automatically by reading the instructions from this file) requirements.txt (contains a list of the dependencies that the application needs to run)

4. Create a Docker Image and push it to google container registry

Start the Cloud Shell and then launch the editor.

Cloud Shell Editor

I have created my dashboard structure within the Explorer provided by the Cloud Shell. You’ll find the complete code in my git repository.

After the files have been copied into the cloud shell editor, the Docker image can be created. For this, you have to change to the dashboard directory.

cd dashboard

From this directory, you can run the bash script BuildPushDockerImage.sh,

bash BuildPushDockerImage.sh

which executes the following commands in sequence:

Build Docker Image and push it to Google Container Registry

The Docker image is now built based on the 3 files (index.py, Dockerfile, requirements.txt) and is available in the Google Container Registry.

Google Container Registry

5. Creating a Deployment, that creates a ReplicaSet to bring up three “dashboard-app” Pods

To start the deployment the following bash script can be executed:

bash DeployDashboardPod.sh

which executes the following commands in sequence:

Deployment with three Pods

The first step is to ensure that we are allowed to run kubectl for our project. Kubectl is a command line interface for running commands against our Kubernetes cluster.

After that we create our “dashboard-app” Pods using the DashboardPod.yaml template.

Deployment of dashboard-app to bring up 3 Pods

After the Pods have been created we can check them in detail by using the following command:

kubectl get pods -o wide

Output of the command

There are 3 Pods in deployment (Status = ContainerCreating), so let’s wait a few more seconds and execute the command again:

Output after executing the command again

We see that the containers are now in status Ready (1/1) and in Status=Running. In addition, they have each been assigned an internal IP address. The maximum number of pods we can run on this node is 110 as described above, which means we could create 107 more containers on this node if the load on the dashboard increases. If we would need more containers, we could order a larger cluster with more nodes, each running 110 pods. This approach gives us the possibility to realize a highly scalable environment.

6. Create a service object that exposes an external IP address to allow access to the dashboard through an Internet Browser

To be able to access the dashboard-app from the web, we create a service object that exposes the deployment. The “type=LoadBalancer” creates a Load Balancer in the current cluster and assign a fixed external IP Address to the service. The Service runs on port 80 and connects to the containers on port 8080.

To expose the deployment the following bash script can be executed:

CreateExtWebService.sh

which executes the following command:

Display information about the dashboard-app service:

kubectl get service dashboard-app

Output of the command

The service “dashboard-app” can now be accessed via an external IP address.

To get more service information, you can execute the following command:

kubectl describe services dashboard-app

Output of the command

Make a note of the external IP address ( LoadBalancer Ingress ) exposed by the dashboard-app service. In this example, the external IP address is 35.202.15.18. In addition you can see that the service has several endpoints: 10.44.0.10:8080,10.44.0.11:8080,10.44.0.12:8080. These are internal addresses of the pods that are running the dashboard-app application.

You can use the external IP address ( LoadBalancer Ingress ) to access the dashboard-app application. Since the application runs in the “/dash” directory, the call in this case is made as follows:

http://<LoadBalancer Ingress IP>/dash/

The dashboard is now displayed in the Internet Browser:

Dashboard-app which was called via the External IP

The dashboard is structured into four areas: