Deploying a Kubernetes application with ArgoCD and Gitlab CI
Tags: kubernetes, argocd, gitlab, scaleway
Published: 2021-01-18 16:21:37

Hello there !

In the [part 4][part4], we deployed the Scaleway Infrastructure with GitLab and Terraform. 

In this last part, we will deploy a simple application using ArgoCD and Gitlab.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xfvehcav80tiswl9kcog.jpg)

# Plan

* Building and publishing our Docker image using Gitlab and Google Cloud Build.
* Configuring and deploying our Docker image to Kubernetes using Gitlab and ArgoCD.

# Docker image

As mentioned in the [part 2][part2], we need to build the docker image after each git tag. Once the new docker image is published we edit the Kubernetes manifests using Kustomize and an env repo pipeline is triggered from the `app-repo` pipeline.

That mechanism is described in the `demo-app` repo pipeline:

{% details .gitlab-ci.yaml %}

```yaml
stages:
  - publish

publish docker image:
  stage: publish
  image: 
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script: 
    - eval $(ssh-agent -s)
    - echo "$GITLAB_SSH_KEY" | tr -d '\r' | ssh-add -
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
    - ssh -T git@gitlab.com

    - git config --global user.name "${GITLAB_USER_NAME}"
    - git config --global user.email "${GITLAB_USER_EMAIL}"
    - git config --global push.followTags true

    - IMAGE_TAG=$CI_COMMIT_TAG-$CI_COMMIT_SHORT_SHA
    - cd src
    - gcloud config set project ${GCP_PROJECT_ID}
    - gcloud builds submit . --tag=eu.gcr.io/${GCP_PROJECT_ID}/demo:$IMAGE_TAG --project ${GCP_PROJECT_ID} --gcs-log-dir=gs://${GCP_PROJECT_ID}_cloudbuild/logs
    - git clone "git@gitlab.com:stack-labs/internal/sandbox/chabanerefes/meetup/demo-env.git"
    - cd demo-env/envs/dev
    - kustomize edit set image eu.gcr.io/${GCP_PROJECT_ID}/demo:$IMAGE_TAG
    - cd ../..
    - git add .
    - 'git commit -m "ci: update image to $IMAGE_TAG"'
    - git tag -a -m "New release available - $CI_COMMIT_TAG" $CI_COMMIT_TAG
    - git push -o ci.skip
  tags:
    - k8s-dev-runner
  only:
    refs:
      - tags
    changes:
      - src/**/*
```
{% enddetails %}

Share the specific runner `k8s-dev-runner` created previously with this project. You will need `Maintainer` permission in Gitlab.

Now you can run the Gitlab pipeline with the following Gitlab CI/CD Variables:

```shell
GCP_PROJECT_ID=$GCP_PROJECT_ID
```

# Deployment

In the `demo-env` repo we have our Kubernetes manifests. Each time the docker image version is edited and a git tag is created, the Gitlab pipeline is executed.

The following files describe the Kubernetes manifests to deploy:

{% details base/demo-deployment.yaml %}

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  labels:
    app: demo
spec:
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      labels:
        app: demo
    spec:
      containers:
      - image: eu.gcr.io/<GCP_PROJECT_ID>/demo:v0.1.0
        name: demo
        ports:
        - containerPort: 8080
```
{% enddetails %}

{% details base/demo-svc.yaml %}

```yaml
apiVersion: v1
kind: Service
metadata:
  name: demo
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: demo
```
{% enddetails %}

{% details base/kustomization.yaml %}

```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# Adds namespace to all resources.
namespace: app-dev
resources:
- demo-deployment.yaml
- demo-svc.yaml
```
{% enddetails %}

{% details envs/dev/kustomization.yaml %}

```yaml
namespace: app-dev
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
```
{% enddetails %}

The ArgoCD application is also defined in a yaml file:

{% details envs/dev/application.yaml %}

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: app-demo-dev
  namespace: argocd
spec:
  project: demo-dev
  source:
    repoURL: <GIT_REPOSITORY_URL>
    targetRevision: HEAD
    path: envs/dev
  destination:
    server: <SW_KAPSULE_CLUSTER_URL>
    namespace: app-dev
  syncPolicy:
    automated:
      prune: true
```
{% enddetails %}

In the gitlab pipeline we have two stages `init` and `deploy`. 

* The first stage adds Kapsule cluster in ArgoCD and initializes the dev environment in Kubernetes.
* The second stage creates and configures the application in ArgoCD.
* I added a third stage for manual sync on ArgoCD.

{% details envs/dev/application.yaml %}

```yaml
stages:
  - init
  - deploy
  - sync

# Get Scaleway credentials from Vault
before_script:
    - export VAULT_TOKEN="$(gcloud secrets versions access latest --secret=vault-token --project ${GCP_PROJECT_ID})"
    - export SCW_SECRET_KEY="$(vault kv get -field=key scaleway/project/${SW_PROJECT_NAME}/credentials/secret)"
    - export AROGOCD_TOKEN="$(gcloud secrets versions access latest --secret=argocd-token --project ${GCP_PROJECT_ID})"

init sw k8s project 🔬:
  stage: init
  when: manual
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    # Connect to GCP GKE DevOps Cluster
    - gcloud container clusters get-credentials gke-cluster-devops --zone europe-west1-b --project ${GCP_PROJECT_ID}
    # Connect to scaleway
    - scw init secret-key=$SCW_SECRET_KEY
    # Get kubeconfig from sw kapsule cluster
    - scw k8s kubeconfig get $(scw k8s cluster list | grep kapsule-cluster-dev-demo | awk '{ print $1 }') region=fr-par > kapsule_config
    # Register kapsule cluster on Argocd
    - export KUBECONFIG=~/.kube/config:$(pwd)/kapsule_config
    - argocd cluster add admin@kapsuleclusterdevdemo --name kapsule-cluster-dev-demo --kubeconfig kapsule_config --auth-token=${AROGOCD_TOKEN} --server ${ARGOCD_ADDR} --grpc-web 
    # Create namespace on kapsule cluster
    - kubectl config use-context admin@kapsuleclusterdevdemo
    - kubectl create namespace app-dev || echo 'namespace app-dev already exists'
    # To access GCR service, create the json key file and associate it with the service account
    - gcloud iam service-accounts keys create sw-gcr-auth-ro.json --iam-account=sw-gcr-auth-ro@${GCP_PROJECT_ID}.iam.gserviceaccount.com
    - export GCR_SECRET_NAME=gcp-gcr-auth-ro
    - |
      kubectl create secret docker-registry $GCR_SECRET_NAME -n app-dev \
      --docker-server=https://eu.gcr.io \
      --docker-username=_json_key \
      --docker-email=ci@<MY_COMPANY>.com \
      --docker-password="$(cat sw-gcr-auth-ro.json)" || echo 'secret $GCR_SECRET_NAME already exists'
    - |
      kubectl patch serviceaccount default -n app-dev \
      -p "{\"imagePullSecrets\": [{\"name\": \"$GCR_SECRET_NAME\"}]}"
  tags:
    - k8s-dev-runner
  only:
    - master 

deploy sw k8s project 🚀:
  stage: deploy
  when: manual
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    # Connect to scaleway
    - scw init secret-key=$SCW_SECRET_KEY
    # Get sw kapsule cluster url
    - export SW_KAPSULE_CLUSTER_URL=$(scw k8s cluster get $(scw k8s cluster list | grep kapsule-cluster-dev-demo | awk '{ print $1 }') | grep ClusterURL | awk '{ print $2 }' | tr -d '\r')
    - cd envs/dev
    - sed -i "s,<SW_KAPSULE_CLUSTER_URL>,$SW_KAPSULE_CLUSTER_URL,g;s,<GIT_REPOSITORY_URL>,$CI_PROJECT_URL.git,g" application.yaml
    # Connecto to gcp gke devops cluster
    - gcloud container clusters get-credentials gke-cluster-devops --zone europe-west1-b --project ${GCP_PROJECT_ID}
    # Create ArgoCD project
    - argocd proj create demo-dev -d $SW_KAPSULE_CLUSTER_URL,app-dev -s $CI_PROJECT_URL.git --auth-token=${AROGOCD_TOKEN} --server ${ARGOCD_ADDR} --grpc-web 
    # Create ArgoCD application
    - kubectl apply -n argocd -f application.yaml
  tags:
    - k8s-dev-runner
  only:
    - master

sync dev 🔨:
  stage: sync
  when: manual
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    - argocd app sync app-demo-dev
  tags:
    - k8s-dev-runner
  only:
    - master
```
{% enddetails %}

Share the specific runner `k8s-dev-runner` created previously with this project. You will need `Maintainer` permission in Gitlab.

Now you can run the Gitlab pipeline with the following Gitlab CI/CD Variables:

```shell
GCP_PROJECT_ID=$GCP_PROJECT_ID
SW_PROJECT_NAME=$SW_PROJECT_NAME
ARGOCD_ADDR=$ARGOCD_ADDR
VAULT_ADDR=$VAULT_ADDR
ENV=dev
```

That's it!

# Final Words

[The source code is available on Gitlab][demo-repo].

The demonstration is also available (but in 🇫🇷) on Youtube  at [32:12][meetup].

{% youtube XyIkg9aFS1k %}

If you have any questions or feedback, please feel free to leave a comment.

Otherwise, I hope I have helped you answer some of the hard questions about building a multi-cloud between an European cloud and an American cloud leader.

By the way, do not hesitate to share with peers 😊

Thanks for reading!

[part2]: https://dev.to/stack-labs/how-to-build-step-by-step-a-multi-cloud-infrastructure-with-google-cloud-and-scaleway-elements-part-2-2j37
[part4]: https://dev.to/stack-labs/how-to-build-step-by-step-a-multi-cloud-infrastructure-with-google-cloud-and-scaleway-elements-part-4-196o
[demo-repo]: https://gitlab.com/Chabane87/multi-cloud-google-cloud-scaleway-playground/-/tree/master/
[meetup]: https://www.youtube.com/watch?v=XyIkg9aFS1k&feature=youtu.be