Kubernetes-based development with Devspace
Tags: kubernetes, devspace, devops, microservices
Published: 2021-10-15 16:48:36

# Kubernetes-based development with Devspace

Modern applications base more and more on micro-services. Splitting large applications into smaller pieces makes the whole more maintainable and easier to develop. However, instead of developing a big monolith, we work on a bunch of tiny applications, making it more challenging to debug and deploy the whole system. Luckily, there are many tools out there to help us out. An interesting comparison of some of them can be found [here](https://kubevious.io/blog/post/kubernetes-development-tools-comparison-skaffold-vs-devSpace-vs-draft-vs-codeready-vs-bridge). In what follows, we want to see how easy it is to do Kubernetes-based development with [devspace](https://devspace.sh).

## A micro-services application

Suppose we are developing a micro-services application, for example an e-shop. In essence, our e-shop consists of a frontend application that communicates with a backend through an API. For the sake of simplicity, let's say that our backend looks like this:

<img src="https://hidora.io/wp-content/uploads/2021/05/baseline.png" alt="baseline" width=500>

User management is handled by the `iam-service`. Orders are processed via the `message-queue`. Most of our backend's business logic is packed in serverless functions served by the `faas`. Our application's state is held in our `database`. Finally, for some good reasons (e.g. the ease of testing setup), we are developing our software in a [monorepo](https://www.atlassian.com/git/tutorials/monorepos).

With time, our micro-services application will necessarily contain a lot of business logic that will be packed in even more micro-service code or serverless functions. For example, we might need a connector service between our `message-queue` and our `faas`, or an assets service with some logic to add new assets in a controlled way. A very convenient way to host our micro-services is to dockerize them and let Kubernetes orchestrate them.

Typically, our IAM service is a third-party like [keycloak](https://www.keycloak.org/) or [fusionauth](https://fusionauth.io/) which we can easily deploy on Kubernetes by means of a [helm chart](https://github.com/FusionAuth/charts). [Helm](https://helm.sh/) is a very practical package manager for Kubernetes. For example, a typical fusionauth deployment would look like something along these lines:

```bash
helm repo add fusionauth https://fusionauth.github.io/charts
helm install fusionauth --create-namespace fusionauth/fusionauth --namespace auth \
    --set database.protocol=postgresql \
    --set database.user=<username> \
    --set database.password=<password> \
    --set database.host=<hostname> \
    --set database.port=5432 \
    --set database.name=<database-name> \
    --set database.root.user=<root-user> \
    --set database.root.password=<root-password> \
    --set app.runtimeMode=production \
    --set search.engine=database
```

Our message queue is probably [redismq](https://redislabs.com/solutions/use-cases/messaging/), [rabbitmq](https://www.rabbitmq.com/) or [kubemq](https://kubemq.io/), for which we also easily find helm charts. 

Then come our own custom services for which we need to write our own Kubernetes resources (deployments, services, ingresses, etc.). Finally, we can write some kind of script to install all the necessary helm charts and apply our Kubernetes resources.

Because our software deals with sensitive data and makes our business, we need to be careful when deploying a new release. Therefore, we want to somehow test it before we release it, which is very easy to do on Kubernetes clusters. Indeed, we can imagine we have two environments, one for testing and one for production. The testing (or staging) environment would be synchronized with our software repository's `main` branch while the production environment would be the pendant of our repo's `production` branch. We develop on the `main` branch and, as soon as the Q&A is satisfied with the software pushed there, we push it to `production`.

We are now in the complicated situation where we want to develop our software on a development machine, test it somehow on an almost productive environment, and release it to a production environment. That leads us to three different build and deployment procedures. On a development machine, we surely want to interact with a short-lived database. Moreover, login credentials to our microservices (like the assets service) should be trivial. On staging, we might want to grant unprotected access to some of our services, for the sake of debugging. On production, we want to secure and hide as much as possible. 

Finally, if our development environment was close to the production environment, we would minimize the amount of surprises following a deployment to staging or production, which would increase our productivity.

## Enter devspace

[Devspace](https://devspace.sh) is a cli tool that allows automation of both the build and the deployment of container images. In addition, that tool might as well replace our makefile or docker-compose configurations and provides us with the ability to do Kubernetes-based development. Because of the latter ability, let's assume we have set up a small cluster on our development machine. In one click, you can have Jelastic set up that development cluster for you

<img src="https://hidora.io/wp-content/uploads/2021/05/marketplace.png">

through a very simple interface

<img src="https://hidora.io/wp-content/uploads/2021/05/k8s-jelastic.png">

Or you can manually set up your own [kind](https://kind.sigs.k8s.io/), [minikube](https://minikube.sigs.k8s.io/), or [docker for desktop](https://docs.docker.com/desktop/kubernetes/) cluster.

The easiest way to install devspace (not on your Kubernetes cluster, on a remote machine from which you develop your code!) is to do
```bash
npm install -g devspace 
```

Then, depending on our use-case, we might run
```bash
devspace init
```
and follow the instructions. In our particular case, we want to build 

- our API
- a bunch of custom micro-services

That we do with the following configuration:
```yaml
version: v1beta10
vars:
- name: SOME_IMPORTANT_VARIABLE
  source: env
  default: the-important-value
images:
  my-custom-service:
    image: my-repo/my-custom-service
    tags:
    - ${DEVSPACE_RANDOM}
    dockerfile: ./my-custom-service/Dockerfile
    context: .
    build:
      docker:
        options:
          target: app
          buildArgs:
            SOME_IMPORTANT_VARIABLE: ${SOME_IMPORTANT_VARIABLE}
  api:
    image: my-repo/api
    tags:
    - ${DEVSPACE_RANDOM}
    dockerfile: ./api/Dockerfile
    context: .
```
The above configuration defines how to build our API and our micro-services. When they are pushed to their docker registry, both docker images will have the same random tag (defined by the built-in variable `DEVSPACE_RANDOM`). Instead of using a docker daemon, we can also choose to use custom build commands or [kaniko](https://github.com/GoogleContainerTools/kaniko). We can use environment variables, like `SOME_IMPORTANT_VARIABLE` and provide the usual options to build docker images. 

Next, we want to deploy

- our API
- our custom micro-services
- various third-party services (iam, message queue, faas, assets)

In order to take care of that, we complete the previous configuration with the following snippet:
```yaml
deployments:
# for the custom service, we have regular k8s manifests
- name: my-custom-service
  kubectl:
    manifests:
    - my-custom-service/manifest.yaml
# for the api, we have written a helm chart
- name: api
  helm:
    chart:
      name: api/chart
    values:
      image: my-repo/api
      postgres:
        database: my-database
        hostname: postgres
        username: my-username
        password: my-password
# the database service is a 3rd party
- name: postgres
  helm:
    chart:
      name: postgresql
      repo: https://charts.bitnami.com/bitnami
    values:
      postgresqlDatabase: my-database
      postgresqlUsername: my-username
      postgresqlPassword: my-password
# the iam service is a 3rd party
- name: iam-service
  helm:
    chart:
      name: fusionauth/fusionauth
    values:
      database:
        protocol: postgresql
        user: iam-user
        password: iam-password
        host: postgres
        name: iam-database
          user: root-db-username
          password: root-db-password
      search:
        engine: database
```
The first deployment, `my-custom-service`, amounts to
```bash
kubectl apply -f my-custom-service/manifest.yaml
```
The second deployment, `api`, is a regular helm installation. Instead of writing our own helm chart, we could have used the built-in [component charts](https://devspace.sh/component-chart/docs/introduction) which offer a compromise between defining our own helm charts and keeping our Kubernetes resources configuration simple. With our current devspace configuration in place, we can start our development environment:
```bash
devspace dev
```
That command builds our docker images and deploys our software to our development Kubernetes cluster's `default` namespace. We are now in a situation where we can develop our code on our development machine and push it to our development Kubernetes cluster. With either [hot reloading](https://devspace.sh/cli/docs/configuration/development/file-synchronization) or [auto-reloading](https://devspace.sh/cli/docs/configuration/development/auto-reloading), we can even fix our code and the result is automatically propagated to our cluster.

## Deploy to multiple environments

Now we have a setup that works for development. We are not very far away from our staging environment setup. First, our docker images need to be tagged following the pattern `<my-repo>/<my-service>:staging-<commit-short-sha>`. Second, our staging environment bases on external database and IAM services. Consequently, we don't want to deploy them on staging and we need to adapt the services that depend on them. In devspace, we can define [profiles](https://devspace.sh/cli/docs/configuration/profiles/basics). Until now, our configuration has no reference to any profile, therefore it is the `development` profile. We can define the `staging` profile, let it base on the `development` profile and adapt it as we've just described. To do that, let's add the following configuration to our `devspace.yaml`:
```yaml
profiles:
- name: staging
  patches:
  # images -> adapt tag
  - op: replace
    path: /images/0=${DEVSPACE_RANDOM}
    value:
    - staging-${DEVSPACE_GIT_COMMIT}
  # postgres -> remove, we have an external database
  - op: remove
    path: /deployments/name=postgres
  # iam service -> remove, we have an external iam service
  - op: remove
    path: /deployments/name=iam-service
  # api 
  # -> we need an ingress
  - op: replace
    path: /deployments/name=api/helm/values/ingress
    value:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: nginx-cert
        cert-manager.io/cluster-issuer: letsencrypt-prod
      hosts:
      - host: api-staging.my-staging-domain.com
        paths:
        - /
      tls:
      - secretName: api-tls
        hosts:
        - api-staging.my-staging-domain.com
  # -> we need up-to-date database accesses
  - op: replace
    path: /deployments/name=api/helm/values/postgres
    value:
        database: my-external-database
        hostname: my-external-database-hostname
        username: my-external-username
        password: my-external-password
  # my-custom-service -> nothing to do
```

We can of course follow the same philosophy coupled with the concept of [parent profiles](https://devspace.sh/cli/docs/configuration/profiles/parents) to define our `production` profile. Then, building and deploying to staging or production is as simple as
```yaml
devspace deploy -p staging
devspace deploy -p production
```
Obviously, remotely debugging those profiles is also possible. 

## We've only scratched the surface...

Many more features are available, like custom commands definition, port-(reverse-)forwarding, file synchronization, container log streaming, etc., which you can read about [here](https://devspace.sh/cli/docs/configuration/development/basics). Wisely used in CI / CD pipelines, devspace can drastically simplify the way you release your software.