Deploy Any AI/ML Application On Kubernetes: A Step-by-Step Guide!
Tags: kubernetes, beginners, tutorial, ai
Published: 2023-10-02 05:02:17

In today's fast-paced technological landscape, deploying AI/ML applications efficiently and effectively is paramount. [Kubernetes](https://kubernetes.io/), a powerful open-source platform, has emerged as a leading solution for managing and scaling containerized applications, ensuring they run seamlessly across various environments. 

In this comprehensive guide, we will walk you through the step-by-step process of deploying any AI/ML application on Kubernetes. From containerising your application to setting up your Kubernetes cluster to deploying your AI/ML applications, this guide covers it. 

#### Let's embark on this learning adventure together!

## Why Deploy GenAI Applications On Kubernetes?
Deploying AI/ML applications on Kubernetes provides a robust solution for managing complex AI/ML workloads. _One of the primary benefits is scalability_. Kubernetes can automatically scale the infrastructure, accommodating varying workloads efficiently, ensuring that resources are allocated effectively based on demand. This auto-scaling feature is crucial for handling large computations involved in AI/ML tasks. 

![AI/ML deployment on Kubernetes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pqvyga1e6hn5ik3t8rk5.png)

Additionally, Kubernetes supports multi-cloud and hybrid-cloud environments, offering flexibility and avoiding vendor lock-in. It provides a consistent and unified environment for development, testing, and deployment, enhancing the collaboration between data scientists and engineers. 

Kubernetes also ensures high availability and fault tolerance, automatically replacing or rescheduling containers that fail, ensuring the reliability and robustness of AI/ML applications. Furthermore, it simplifies many operational aspects, including updates and rollbacks, allowing teams to focus more on building AI/ML models rather than managing infrastructure.

## Prerequisites 
- [Docker Desktop](https://www.docker.com/products/docker-desktop/) installed for containerizing our application. 
- [Minikube](https://minikube.sigs.k8s.io/docs/start/) installed. We will use Minikube to create a single node cluster.
- Download and install [Node.js](https://nodejs.org/en/download) as the sample application is a Node.js application.
- Sign up for a [free SingleStore database cloud account](https://www.singlestore.com/cloud-trial/?utm_campaign=singlestore-for-ai&utm_medium=referral&utm_source=pavan&utm_term=pavancicd) to integrate our application with a database. 

### Tutorial
Hope you have GitHub account. 

First, we will clone the already available [openai-quickstart-node repository](https://github.com/pavanbelagatti/openai-quickstart-node.git) to our local.

```
git clone https://github.com/pavanbelagatti/openai-quickstart-node.git
```

Let's navigate into the project directory

```
cd openai-quickstart-node
```
Install the project requirements and dependencies

```
npm install
```
Create a `.env` file and add your OpenAI API Key.

```
touch .env
```
In your .env file, add the OpenAI Key as an environment variable as shown below.

```
OPENAI_API_KEY=<Add Your OpenAI API Key>
```

Run the application using the below command.

```
npm run dev
```
You should see the application on local 3000.

![application on local](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0ht9ib21mwx10ceicbfl.png)

#### Let's write a Dockerfile for our application to containerize it.

```
touch Dockerfile
```
Add the following Dockerfile instructions in it.

```
# Use the official Node.js image as a parent image
FROM node:14-alpine as build

# Set the working directory in the Docker container
WORKDIR /app

# Copy the package.json and package-lock.json files into the container at /app
COPY package*.json ./

# Install the dependencies
RUN npm install

# Copy the local files into the container at /app
COPY . .

# Build the application
RUN npm run build

# Start from a smaller image to reduce image size
FROM node:14-alpine as run

# Set the working directory in the Docker container
WORKDIR /app

# Copy over dependencies
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/.next ./.next
COPY --from=build /app/public ./public
COPY --from=build /app/package*.json ./

# Expose port 3000 for the app to be accessible externally
EXPOSE 3000

# Command to run the application
CMD ["npm", "start"]
```
Let's build, tag and push this to our DockerHub.

```
docker build -t <image name> .
```
**Note**: I am naming my image as `generativeai-node-app`

Let's launch our container.
The image is built! It’s time to launch the Docker container with assigned ports using the following command:

```
docker run -p 3002:3002 generativeai-node-app
```

Let's build it again with our DockerHub credentails.

```
docker build -t <your dockerhub username>/<image name> .
```

Let's push the image to DockerHub.
```
docker push <your dockerhub username>/<image name>
```

You can confirm if the image is pushed by going to your DockerHub.
![docker image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4rzgsyk9b3epx17eljkz.png)

#### Deploy and expose our application on Kubernetes
To deploy and expose application, we need two yaml files.
`deployment.yaml` and `service.yaml` files.

One file contains the instructions of deployment and the other one for service exposure.

Let's see our `deployment.yaml` file first.

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: genai-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: genai-app
  template:
    metadata:
      labels:
        app: genai-app
    spec:
      containers:
        - name: genai-app
          image: pavansa/generativeai-node-app:latest
          ports:
            - containerPort: 3000
```

The below is our `service.yaml` file.

```
apiVersion: v1
kind: Service
metadata:
  name: genai-app-service
spec:
  selector:
    app: genai-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000
  type: LoadBalancer
```

We are using Minikube to create a single node Kubernetes cluster and we will use it to deploy our application.

Start the Minikube using the below command.

```
minikube start
```
This is the output you should see.
![minikube started](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u66awe0n3rensy8f08e9.png)

**Note**: Keep your Docker Desktop running and enable Kubernetes in it. Below image is just for your reference. 

![Docker desktop reference](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cjyge7bfeflj7ikzvbc9.png)

Let's apply our deployment file using the below command.

```
kubectl apply -f deployment.yaml
```

Similarly, apply our service yaml file.

```
kubectl apply -f service.yaml
```
Let's check the status of our Pods first using the below command.

```
kubectl get pods
```
You should see the desired output as below.
![pods](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rszcf7mjdf1f10v6w8vg.png)

Let's check the deployment status of our application to see if the desired pods are running as expected.

```
kubectl get deployment
```

![deployment status](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/49kksc3zsstocsd6wska.png)

Let's check the service status of our application.

```
kubectl get service
```

![service](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/31emk97al8378znmrsrm.png)

Let's see if we can expose our application and access it.

```
minikube service <add your application service name> --url
```
You should see the output as below and will be able to access your application.

![app running](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8g825gae0sanabq2xpq6.png)

You can see the url link to go and access our application. 

_**Congratulations!**_ We containerized our application, deployed and exposed it using Kubernetes. 

## Integrating a Database for Our Application
After successfully deploying and exposing your AI/ML application on Kubernetes, you might need a robust and scalable database to handle your application data. SingleStore is a high-performance, scalable SQL database that is well-suited for AI/ML applications. In this section, we will guide you through the process of integrating SingleStore database into your Kubernetes-deployed application.

You need a free [SingleStore cloud account](https://www.singlestore.com/cloud-trial/?utm_campaign=singlestore-for-ai&utm_medium=referral&utm_source=pavan&utm_term=pavancicd).

- Create a workspace and then create a database and table suitable for your application.

Go to the SQL editor as shown in the below image
![SQL editor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a9c991bwa9yg9a56q235.png)

#### Create a new database using the following SQL statement.

```
-- create a database
CREATE DATABASE <database name>;
```

Next, switch to the new database using the USE command

```
USE <database name>;
```

Then, create a table in the new database with the desired schema.

```
-- create a table
CREATE TABLE <table name> (
);
```

You can paste these SQL commands in the SQL Editor, highlight them, and then click the Run button

You can find the whole process of creating a database, tables and feeding information in tables in the tutorial below.

{% embed https://dev.to/pavanbelagatti/getting-started-with-sql-a-beginners-hands-on-guide-46p1 %}

#### Update Kubernetes Deployment Configuration:
If your SingleStore database is running outside the Kubernetes cluster, update your application’s Kubernetes deployment configuration to allow connections to the SingleStore database.

```
apiVersion: apps/v1
kind: Deployment
...
spec:
  containers:
    - name: genai-app
      ...
      env:
        - name: DB_HOST
          value: "<Your SingleStore DB Host>"
        - name: DB_PORT
          value: "<Your SingleStore DB Port>"
        ...

```
#### Redeploy Your Application:
Apply the updated Kubernetes deployment configuration to redeploy your application with SingleStore integration.

```
kubectl apply -f deployment.yaml
```

#### Verify the Integration:
After redeployment, verify that your application is successfully connected to the SingleStore database and is performing database operations as expected.

By following these steps, you have successfully integrated SingleStore database into your Kubernetes-deployed AI/ML application, providing a robust and scalable solution for managing your application data.

#### Conclusion
Congratulations on successfully navigating through the comprehensive steps to deploy an AI/ML application on Kubernetes! This guide has walked you through each essential phase, from containerizing your application to deploying and exposing it on Kubernetes.

As you continue to explore and enhance your AI/ML deployments, consider integrating a high-performance database like SingleStore for managing your application data seamlessly. SingleStore offers scalability, speed, and efficiency, ensuring your AI/ML applications run optimally with a robust database backend.