Develop with Kubernetes Locally with Tilt

In this article, I wanted to try **Tilt** for local software development targeting Kubernetes on a Mac.

The general idea is that, when targeting Kubernetes, making the development environment look and behave like production and also keep the process productive and reliable is not easy. And there are a couple of projects that aims to make the workflow less painful.

I’m new to the subject but, from the little I know and researched, these are the features those tools may provide:

- Detection of changes to the source-code and automation of deployment to Kubernetes
- Some mechanism to avoid rebuilding images and restarting containers
- Easy to replicate environments
- Mechanisms to aid testing and debugging
- File synchronization, port forwarding and even some web UI to support development

# Options and Choice

These are the options I’ve found with the corresponding number of starts on GitHub. **Skaffold** is developed by Google and is certainly the most well-known. I don’t know enough to evaluate them right now, but I felt each tool has a particular focus and approaches. Example: one aims almost exclusively to make development more productive, other aims to be more comprehensive.

| Project | ⭐ |
| --- | --- |
| [Skaffold](https://github.com/GoogleContainerTools/skaffold) | 12.2k |
| [Tilt](https://github.com/tilt-dev/tilt) | 4.9k |
| [Tye](https://github.com/dotnet/tye) | 3.9k |
| [DevSpace](https://github.com/loft-sh/devspace) | 2.6k |
| [Garden](https://github.com/garden-io/garden) | 2.3k |
| [Okteto](https://github.com/okteto/okteto) | 1.9k |
| [Bridge to Kubernetes](https://github.com/microsoft/mindaro) | 259 |
| [CodeReady](https://developers.redhat.com/products/codeready-workspaces/overview) | - |

From the options here, I decided to take a deeper look at Tilt. I didn’t test the other options, so I don’t have a strong reasoning for the choice. It just that, at first glance, Tilt and its documentation seemed to be simpler. And it was easier to imagine its goals from the phrases in the docs:

> Run `tilt up` to work in a complete dev environment configured for your team.
> 

> Think `docker build && kubectl apply` or `docker-compose up`.
> 

# Goal

My idea is to setup a local development environment targeting Kubernetes with Tilt on a Mac. I will use the demo app [**RealWorld**](https://github.com/gothinkster/realworld) that I covered in a previous [article](https://qiita.com/dennistanaka/items/78585b6bda374be98aad) for this example. Also, I plan to reuse the Kubernetes manifest files I created for that article. For a local Kubernetes cluster, I’m going to use [**Minikube**](https://minikube.sigs.k8s.io/docs/start/).

# Steps

## Requirements

- Install Docker Desktop
- Install Minikube: [https://minikube.sigs.k8s.io/docs/start/](https://minikube.sigs.k8s.io/docs/start/)
- Install kubectl: [https://kubernetes.io/docs/tasks/tools/#kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)
- Install Tilt: [https://docs.tilt.dev/](https://docs.tilt.dev/)
- Start Minikube:

```bash
$ minikube start
```

## More Preparation

The project structure for this tutorial is:

- realworld-example-app
    - deploy
    - node-express-realworld-example-app
    - react-redux-realworld-example-app

### deploy

For the `deploy` folder, please use the Kubernetes manifest files in my repository below. These are the files I used in a previous article, but some changes were necessary mainly due to changes in the project structure as well as differences in the available resources since I’ve prepared a separate VM at that opportunity with more resources than what I have this time.

[https://github.com/dennistanaka/tilt-dev-env](https://github.com/dennistanaka/tilt-dev-env)

### node-express-realworld-example-app

For the backend, let’s clone the corresponding RealWorld project and create a `Dockerfile` for it:

```bash
$ git clone git@github.com:gothinkster/node-express-realworld-example-app.git
$ cd node-express-realworld-example-app
$ touch Dockerfile
```

Contents for the `Dockerfile`:

```docker
FROM node:lts-alpine

WORKDIR /usr/src/app

COPY node-express-realworld-example-app/package.json ./

RUN npm install

COPY node-express-realworld-example-app/. .

EXPOSE 3000
CMD [ "npm", "start" ]
```

Compared to my previous articles, I’ve just changed the paths to match the required project structure.

I’ve also changed the line below in `package.json`:

```jsx
"start": "node ./app.js",
```

to:

```jsx
"start": "nodemon ./app.js",
```

This is a workaround to make our backend application update in the case of changes to the source-code. I need to figure out a better way to do this, but opted for this change for now since I didn’t want to touch the applications themselves much as it’s not the focus of this article.

### react-redux-realworld-example-app

Let’s do the same steps for the frontend:

```bash
$ git clone git@github.com:gothinkster/react-redux-realworld-example-app.git
$ cd react-redux-realworld-example-app
$ touch Dockerfile
```

`Dockerfile`:

```docker
FROM node:lts-alpine

WORKDIR /usr/src/app

COPY react-redux-realworld-example-app/package.json ./

RUN npm install

COPY react-redux-realworld-example-app/. .

EXPOSE 4100
CMD [ "npm", "start" ]
```

I’ve changed the line below in `src/agent.js`:

```jsx
const API_ROOT = 'https://conduit.productionready.io/api';
```

to:

```jsx
const API_ROOT = 'http://localhost/api';
```

This is because the frontend project uses a live API server by default to make it more convenient to test without the backend running locally. But, in our case, that’s exactly what we want to test.

## Secret

The backend uses a configuration that requires the creation of a secret:

```bash
$ kubectl create secret generic backend-secrets --from-file=backend_secret=deploy/secrets/backend_secret.txt
```

## Tilt

Now, let’s go to the root folder and run:

```bash
$ tilt up
Tilt started on http://localhost:10350/
v0.23.3, built 2021-12-10

(space) to open the browser
...
```

Press `space` to open the **Tilt IU in the** browser and it will complain `Tiltfile` found. Create a `Tiltfile` and save. Now, it will complain that `No resources found`.

This file is the Tilt configuration and this is written in [Starlark](https://docs.bazel.build/versions/main/skylark/language.html), a simplified dialect of Python. Here, we describe the build process and the Kubernetes manifests we want to apply to our cluster.

Here is the complete file I wrote for this example:

```python
version_settings(constraint='>=0.22.2')

k8s_yaml(['deploy/db-pv-claim.yaml', 'deploy/pv.yaml', 'deploy/db-deployment.yaml', 'deploy/db-service.yaml'])

docker_build(
    'conduit-express-backend',
    context='.',
    dockerfile='./node-express-realworld-example-app/Dockerfile',
    only=['./node-express-realworld-example-app/'],
    live_update=[
        sync('./node-express-realworld-example-app/', '/usr/src/app/'),
        run(
            'npm install',
            trigger=['./node-express-realworld-example-app/package.json']
        )
    ]
)

k8s_yaml(['deploy/backend-config.yaml', 'deploy/backend-deployment.yaml', 'deploy/backend-service.yaml'])

docker_build(
  'conduit-react-frontend',
  context='.',
  dockerfile='./react-redux-realworld-example-app/Dockerfile',
  only=['./react-redux-realworld-example-app/'],
  live_update=[
      sync('./react-redux-realworld-example-app/', '/usr/src/app/'),
      run(
          'npm install',
          trigger=['./react-redux-realworld-example-app/package.json']
      )
  ]
)

k8s_yaml(['deploy/frontend-deployment.yaml', 'deploy/frontend-service.yaml'])

k8s_yaml('deploy/ingress.yaml')
```

The [Tiltfile API](https://docs.tilt.dev/api.html) function [k8s_yaml](https://docs.tilt.dev/api.html#api.k8s_yaml) is very self-explanatory and registers Kubernetes objects we want to deploy. [docker_build](https://docs.tilt.dev/api.html#api.docker_build) as the name suggests, describes how the images need to be built.

The `live_update` configuration is particularly important as it specifies what folders need to be synced between your local machine and the corresponding container. Changes to the files will make Tilt copy them to the container.

The `run` configuration specifies the commands that will be run when certain files are changed. In this case, changes to `package.json` will trigger the `npm install` command.

Please add the contents above and save. If you switch to the browser windows, you can see Tilt processing the changes through the Tilt UI. This is very convenient and allow us to have a general view of what’s going on behind the scenes.

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/509832/e76f17c8-3978-0066-54b4-d8777b705caf.png)

## Accessing the Application

We need one more step to access our application and we will install Minikube’s ingress addon to allow us to access our application ingress. First, run the command below:

```bash
$ minikube addons enable ingress
```

And then:

```bash
$ minikube tunnel
```

We need to provide our password in this step. From the examples I read, the command outputs some information but, in my case, this didn’t happen for some reason. Nothing happened after I typed my password and it looked like the command hanged. But, the application behave as expected.

Just access [http://localhost/](http://localhost/) to check the application.

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/509832/545ea486-6309-c56d-4d21-bc8f62b7cc35.png)

## Control Loop

What Tilt calls the [Control Loop](https://docs.tilt.dev/controlloop.html) is the mechanism that makes Tilt respond to our changes and run the most current code and configuration.

You can try for yourself and see that any changes to the `TiltFile`, Kubernetes manifests, `Dockerfile` or changes to the application source-code will make Tilt process these changes to update the cluster up-to-date.

# Conclusion

I hope this article can be of some help. I myself still know very little about the subject, so I’d like to explore Tilt more to understand its limitations as well as try the other alternatives to see what they do differently.

# References

- [https://docs.tilt.dev/tutorial/index.html](https://docs.tilt.dev/tutorial/index.html)
- [https://docs.tilt.dev/tiltfile_authoring.html](https://docs.tilt.dev/tiltfile_authoring.html)
- [https://docs.tilt.dev/choosing_clusters.html#minikube](https://docs.tilt.dev/choosing_clusters.html#minikube)
- [https://minikube.sigs.k8s.io/docs/handbook/accessing/#using-minikube-tunnel](https://minikube.sigs.k8s.io/docs/handbook/accessing/#using-minikube-tunnel)