Google Cloud Kubernetes in 10 Minutes
Tags: kubernetes, devops, productivity, tutorial
Published: 2019-11-14 16:28:30

In the beginning Google created Kubernetes. “Let it be open source,” Google said, and the sources opened. And Google saw it was good. All kidding aside, if anyone knows how to run Kubernetes, it’s Google.  

In this hands-on post, we’ll learn continuously deliver a demo application to the Google Kubernetes Engine using Semaphore CI/CD. By the end of this read, you’ll have a better understanding of how Kubernetes works,  and, even better, a continuous delivery pipeline to play with.

## How Do Deployments Work in Kubernetes?

A Kubernetes deployment is like one of those Russian dolls. The application lives inside a Docker container, which is inside a pod, which takes part in the deployment.

A pod is a group of Docker containers running on the same node and sharing resources. Pods are ephemeral—they are meant to be started and stopped as needed. To get a stable public IP address, Kubernetes provides a load balancing service that forwards incoming requests to the pods.

The most straightforward way to define a deployment is to write a manifest like the one I present below.

First, we have the **deployment** resource, which holds and controls the pods. Deployments have a `name` and a `spec`, which defines the final desired state:

- Replicas: how many pods to create. Set the number to match the number of nodes in your cluster. For instance, I’m using three pods, so I’ll **change** it to `replicas: 3`.
- spec.containers: Defines the docker image running in the pods. We're going to upload the image to a Google private registry and pull it from there.
- Labels: they are key-value mappings that we stick on to pods. We can then use `matchLabels` to relate the deployment with the pods.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: semaphore-demo-nodejs-k8s-server
spec:
  replicas: 3
  selector:
    matchLabels:
      app: semaphore-demo-nodejs-k8s-server
  template:
    metadata:
      labels:
        app: semaphore-demo-nodejs-k8s-server
    spec:
      containers:
        - name: semaphore-demo-nodejs-k8s-server
          image: gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:$SEMAPHORE_WORKFLOW_ID
          env:
            - name: NODE_ENV
              value: "production"
```
The final piece of the manifest is the **service**. A load balancer service exposes a stable public IP for our users to connect to. We tell Kubernetes that the service will serve the pods labeled as  `app: semaphore-demo-nodejs-k8s-server`.  

```yaml
kind: Service
metadata:
  name: semaphore-demo-nodejs-k8s-lb
spec:
  selector:
    app: semaphore-demo-nodejs-k8s-server
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 3001
```
Before we can use this manifest, we need two things:

- Push a Docker image to Google private repository.
- Send the manifest to the cluster.

We’ll take care of both in the next section. But first, let’s create the necessary services on Google cloud.

## CI/CD and Kubernetes

We’ll use Semaphore to run our [Continuous Integration and Delivery](https://semaphoreci.com/cicd) workflow:

- Continuous Integration: the tiniest error can bring down a site and crash an application. We’ll put the code through a Continuous Integration pipeline that can weed out the bugs before they creep into our deployment.
- Dockerize: generates Docker images for each update so that we can track the exact version that is running in production, and we can rollback or forward in seconds.
- Deploy: 100% automated deployment to Google Kubernetes. No human intervention means more reliable and frequent releases.

Since I wish to focus on the Kubernetes deployment, I’ll skip the Continuous Integration section altogether.  If you are curious and would like to examine in detail how it works, you can find about it in the [full demo](https://semaphoreci.com/blog/continuous-delivery-google-kubernetes) tutorial in Semaphore blog.

### Gettings Things Ready

You’ll need to sign up few services: [Google Cloud Platform](https://cloud.google.com) will be our cloud provider; also [GitHub](https://github.com) for the application code and [Semaphore](https://semaphoreci.com) for the CI/CD. I recommend installing the [semaphore cli](https://docs.semaphoreci.com/article/53-sem-reference) for a quick setup.

Go to your Google Cloud Platform and:

1. Create a project. By default, Google assigns a random name, but you can change it using the **Edit** button. I prefer using something more descriptive like `semaphore-demo-nodejs-k8s`.
2. Go to IAM and create a [Service account](https://console.cloud.google.com/iam-admin/serviceaccounts). The account should be *Owner* of the project. Once created, create and download the **key file** in JSON format.
3. In the [Kubernetes Engine](https://console.cloud.google.com/kubernetes), create a cluster named `semaphore-demo-nodejs-k8s-server`. You may choose how many nodes and the size of each machine. Three nodes are enough to get a taste of Kubernetes. The smallest machine will do for this demo.
4. Go to the [SQL](https://console.cloud.google.com/sql/instances) console, and create a PostgreSQL database in the same region as the cluster. Enable the *Private IP* network. You can also enable the Public IP and whitelist yourself to connect remotely. Take note of the IP that Google assigned to your db.
5. Create a database named `demo` and a username called `demouser`.

I know it’s a lot of work.  The good news is that you only have to do it once.  

Finally, fork the [demo](https://github.com/semaphoreci-demos/semaphore-demo-nodejs-k8s).

{% github semaphoreci-demos/semaphore-demo-nodejs-k8s %}

Clone it, and add it to Semaphore:

```bash
$ cd semaphore-demo-nodejs-k8s
$ sem init
```

The application implements an API endpoint. It is written in TypeScript with the [Nest.js](https://nestjs.com/) framework.

### Dockerize Pipeline

This [pipeline](https://semaphoreci.com/continuous-integration) prepares the Docker image, which is then pushed into Google’s Private [Container Registry](https://cloud.google.com/container-registry/).

![Promotion and Docker Build](https://thepracticaldev.s3.amazonaws.com/i/xx6plzdvmcy5kpr78d5n.png)

Shall we see how it works? Open `.semaphore/docker-build.yml`.

At the start of the file, we have a `name` for the pipeline and the `agent`. The agent tells Semaphore which of the available [machines](https://docs.semaphoreci.com/article/20-machine-types) and OS runs the jobs:

``` yaml
version: v1.0
name: Docker build server
agent:
  machine:
    type: e1-standard-2
    os_image: ubuntu1804
```
*Blocks* and *Jobs* organize the [pipeline](https://semaphoreci.com/blog/cicd-pipeline) execution order. Blocks are executed sequentially, one after the other. Jobs within a block are executed in parallel. If any command in a job fails, the pipeline stops. 

Here is the Docker Build block:

``` yaml


blocks:
  - name: Build
    task:
      secrets:
        - name: gcr-secret
        - name: production-env

      prologue:
        commands:
          - gcloud auth activate-service-account --key-file=.secrets.gcp.json
          - gcloud auth configure-docker -q
          - gcloud config set project $GCP_PROJECT_ID
          - gcloud config set compute/zone $GCP_PROJECT_DEFAULT_ZONE
          - checkout

      jobs:
      - name: Docker build
        commands:
          - cp /home/semaphore/ormconfig.production.json ormconfig.json
          - cp /home/semaphore/production.env production.env

          - docker pull "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:latest" || true
          - docker build --cache-from "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:latest" -t "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:$SEMAPHORE_WORKFLOW_ID" .
          - docker images
          - docker push "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:$SEMAPHORE_WORKFLOW_ID"
```

The `prologue` is executed before each job. Here, it sets up [gcloud](https://cloud.google.com/sdk/gcloud/) to work with the project. First, we need to get authorized using `gcloud auth`. Next, we configure gcloud to work as a docker helper, so we can use the private registry. Finally, we set the active project and zones for the session.

[Checkout](https://docs.semaphoreci.com/article/54-toolbox-reference#checkout) clones the repository.

The build job copies some configuration files inside the Docker image and pushes it to the registry.
To tag the image, we use `$SEMAPHORE_WORKFLOW_ID`, which is guaranteed to be unique for every workflow.

At this point, you may be wondering where those config files and environment variables came from. They were imported with the `secrets` keyword. Semaphore [secrets](https://docs.semaphoreci.com/article/66-environment-variables-and-secrets) mechanism allows us to store sensitive data outside the repository securely. We'll create the secrets in a moment.

Once this pipeline is read, we can link it up to the next one  with a [promotion](https://docs.semaphoreci.com/article/67-deploying-with-promotions):

``` yaml
promotions:
  - name: Deploy server to Kubernetes
    pipeline_file: deploy-k8s.yml
    auto_promote_on:
      - result: passed
```

### Secrets and Environment Files

We need to pass the database username and password to the server. We’ll use two files for this, both have more or less the same variables:

  - environment: a regular bash file with environment variables.
  - ormconfig: a config file for [TypeORM](https://typeorm.io/), the database ORM for our project.

Since the files have sensitive information, we shouldn’t check them into GitHub. Instead, we upload them as Secrets to Semaphore. [Secrets](https://docs.semaphoreci.com/article/66-environment-variables-and-secrets) in Semaphore are automatically encrypted and made available when requested to your jobs.

Copy the provided sample configs outside your repository, for instance to your `/tmp` directory:

``` bash
$ cp ormconfig.sample.json /tmp/ormconfig.production.json
$ cp sample.env /tmp/production.env
```

Edit `ormconfig.production.json`. Replace the `host` and `password` values with your database IP address and the password for your `demouser`. The first part of the file should look like:

``` json
{
  "type": "postgres",
  "host": "YOUR_DB_IP",
  "port": 5432,
  "username": "demouser",
  "password": "YOUR_DB_PASSWORD",
  "database": "demo",

  . . .
```

Edit `production.env`. Set `NODE_ENV=production`, leave `PORT` unmodified and change the `DB_HOST`, `DB_PASSWORD` and `DB_PORT` as appropriate:

``` bash
NODE_ENV=production
PORT=3001
URL_PREFIX=v1/api
DATABASE_HOST=YOUR_DB_IP
DATABASE_USER=demouser
DATABASE_PASSWORD=YOUR_DB_PASSWORD
DATABASE_DBNAME=demo
DATABASE_PORT=5432
```
Upload both files to Semaphore as a secret called `production-env`:

```bash
$ sem create secret production-env \
    -f /tmp/production.env:/home/semaphore/production.env \
    -f /tmp/ormconfig.production.json:/home/semaphore/ormconfig.production.json
```
We have to create a second secret to store the Google-related information and the service account JSON key:

```bash
$ sem create secret gcr-secret \
   -e GCP_PROJECT_ID=semaphore-demo-nodejs-k8s \
   -e GCP_PROJECT_DEFAULT_ZONE=YOUR_REGION \
   -f YOUR_GCP_ACCESS_KEY_FILE.json:/home/semaphore/.secrets.gcp.json
```

### Deployment Pipeline

With a Docker image on hand, we are ready to run it on our Kubernetes cluster.

Take a look at the deployment pipeline at `.semaphore/deploy-k8s.yml`. It’s made of 2 blocks, each has one job.

![Deploy Pipeline and Promotion](https://thepracticaldev.s3.amazonaws.com/i/uzzr2mp6sg7bwggewn72.png)

Most of the `gcloud` commands in the prologue we’ve seen, the only new guy here is `gcloud container`, this one retrieves the Kubernetes config file for the following `kubectl` commands.

With `envsubst` we expand in-place the environment variables. The result is a file that should be plain YAML. The last thing remaining is using [kubectl apply](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply) to send the manifest to our cluster:

``` yaml
blocks:
  - name: Deploy to Kubernetes
    task:
      secrets:
        - name: gcr-secret

      env_vars:
        - name: CLUSTER_NAME
          value: semaphore-demo-nodejs-k8s-server

      prologue:
        commands:
          - gcloud auth activate-service-account --key-file=.secrets.gcp.json
          - gcloud auth configure-docker -q
          - gcloud config set project $GCP_PROJECT_ID
          - gcloud config set compute/zone $GCP_PROJECT_DEFAULT_ZONE
          - gcloud container clusters get-credentials $CLUSTER_NAME --zone $GCP_PROJECT_DEFAULT_ZONE --project $GCP_PROJECT_ID
          - checkout

      jobs:
      - name: Deploy
        commands:
          - cat deployment.yml | envsubst | tee deployment.yml
          - kubectl apply -f deployment.yml
```

At this point, we’re almost done. To mark that this image is the one running on the cluster, we have the last block that tags the image as `latest`. Here are the relevant commands:

``` yaml
        - docker pull "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:$SEMAPHORE_WORKFLOW_ID"
        - docker tag "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:$SEMAPHORE_WORKFLOW_ID" "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:latest"
        - docker push "gcr.io/$GCP_PROJECT_ID/semaphore-demo-nodejs-k8s-server:latest"
```

Ready to do a trial run? Push the modifications and watch the pipelines go.

``` bash
$ git add deployment.yml
$ git add .semaphore/*
$ git commit -m "first deployment”
$ git push origin master
```

You can check the progress of the jobs from your Semaphore account. Wait a few minutes until all the pipelines are done. Hopefully, everything is green, and we can check the cluster state now. 

Here’s the easiest way of connecting to the cluster from your Google Cloud Console, go to:

- Kubernetes Engine > Clusters > Select your cluster > **Connect** button

You’ll get an in-browser terminal that is connected to the project. 

Let’s check the cluster. First, check the pods:

```bash
$ kubectl get pods
NAME                                                READY   STATUS    RESTARTS   AGE
semaphore-demo-nodejs-k8s-server-6b95cf5dfd-hgrmn   1/1     Running   0          97s
semaphore-demo-nodejs-k8s-server-6b95cf5dfd-jgc9p   1/1     Running   0          112s
semaphore-demo-nodejs-k8s-server-6b95cf5dfd-r29gc   1/1     Running   0          105s
```

Each pod has been assigned a different name; all of them are running a copy of our application.

Next, let’s check the deployment:

```bash
$ kubectl get deployment
NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
semaphore-demo-nodejs-k8s-server   3/3     3            3           12m
```

The deployment is controlling the 3 pods. We are told the 3 pods are available and up to date.

Finally, let’s check the service:

```bash
$ kubectl get service
NAME                                  TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)        AGE
kubernetes                            ClusterIP      10.36.0.1     <none>          443/TCP        71m
semaphore-demo-nodejs-k8s-server-lb   LoadBalancer   10.36.9.180   104.154.96.87   80:32217/TCP   12m
```

Ignore the ClusterIP service, that one was before the deployment. We can connect to the application using the EXTERNAL-IP of the LoadBalancer service. 

Let’s tests the application with curl:

```bash
$ curl -w "\n" -X POST -d \
  "username=jimmyh&firstName=Johnny&lastName=Hendrix&age=30&description=Burn the guitar" \
  http://YOUR_EXTERNAL_IP/v1/api/users
{
    "username": "jimmyh",
    "description": "Burn the guitar",
    "age": "30",
    "firstName": "Johnny",
    "lastName": "Hendrix",
    "id": 1,
    "createdAt": "2019-08-05T20:45:48.287Z",
    "updatedAt": "2019-08-05T20:45:48.287Z"
}
```

The API endpoint accepts `GET`, `POST` and `DELETE` requests:

```bash
$ curl -w "\n" http://YOUR_EXTERNAL_IP/v1/api/users/1
{
    "id": 1,
    "username": "jimmyh",
    "description": "Burn the guitar",
    "firstName": "Johnny",
    "lastName": "Hendrix",
    "age": 30,
    "createdAt": "2019-08-05T20:45:48.287Z",
    "updatedAt": "2019-08-05T20:45:48.287Z"
}
```

## Conclusion

It's been a long ride, but hopefully a smooth one. Now you know how to build a CI/CD pipeline for Google Cloud Kubernetes Engine. 

Some ideas play with:

  - Create a staging cluster.
  - Build a development container and run tests inside it.
  - Enhance your manifest with [rolling updates]().

Using AWS instead of Google? Check out this tutorial:

  {% link https://dev.to/semaphore/continuous-integration-and-delivery-to-aws-kubernetes-12d6 %}

If you wish to learn more about how Semaphore can work Kubernetes and Docker check out these:

  - [CI/CD for Microservices on DigitalOcean Kubernetes](https://semaphoreci.com/blog/cicd-microservices-digitalocean-kubernetes)
  - [Working with Docker Images](https://docs.semaphoreci.com/article/78-working-with-docker-images)

Did you find the post useful? Hit those ❤️ and 🦄, follow me or leave a comment below.

Interested in CI/CD and Kubernetes? We’re working on a free ebook, [sign up](http://bit.ly/2KsBHI2), to receive it as soon as it’s published.

Thanks for reading!