Deploying Scaleway Elements Kubernetes Kapsule using Terraform and Gitlab CI
Tags: googlecloud, scaleway, terraform, gitlab
Published: 2021-01-18 11:36:43

Hello there !

In the [part 3][part3], we built our DevOps platform in Google Cloud with GitLab and Kubernetes. We also configured Vault and ArgoCD.

In this part 4, we will deploy the Scaleway Infrastructure using Terraform and Gitlab.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/gnz7vz5yd5bq2helneyx.jpg)

# Plan

* Creating the Kapsule Cluster.
* Writing the Gitlab pipeline.

# Infrastructure as code

## Kapsule Cluster

[Kapsule] is a fully-managed Kubernetes service of Scaleway Elements offering a free managed control plane, high availability and auto-scaling.

The following terraform:

* Creates Kapsule cluster.
* Creates a nodepool.

{% details plan/kapsule.tf %}

```terraform
resource "scaleway_k8s_cluster_beta" "k8s-cluster-demo" {
  name = "kapsule-cluster-${var.env}-demo"
  description = "K8S Demo ${var.env} Cluster"
  version = "1.19.4"
  cni = "calico"
  enable_dashboard = true
  ingress = "nginx"
  tags = [var.env, "demo"]

  autoscaler_config {
    disable_scale_down = false
    scale_down_delay_after_add = "5m"
    estimator = "binpacking"
    expander = "random"
    ignore_daemonsets_utilization = true
    balance_similar_node_groups = true
    expendable_pods_priority_cutoff = -5
  }
}

resource "scaleway_k8s_pool_beta" "k8s-pool-demo" {
  cluster_id = scaleway_k8s_cluster_beta.k8s-cluster-demo.id
  name = "kapsule-pool-${var.env}-demo"
  node_type = "DEV1-M"
  size = 3
  autoscaling = true
  autohealing = true
  min_size = 1
  max_size = 5
}

```
{% enddetails %}

## Other

{% details plan/provider.tf %}

```terraform
provider "scaleway" {
  zone            = var.zone
  region          = var.region
}
```
{% enddetails %}

{% details plan/backend.tf %}

```terraform
terraform {
  backend "gcs" {
  }
}
```
{% enddetails %}

{% details plan/variables.tf %}

```terraform
variable "zone" {
  type = string
}

variable "region" {
  type = string
}

variable "env" {
  type = string
}
```
{% enddetails %}

{% details envs/dev/terraform.tfvars %}

```terraform
zone             = "fr-par-1"
region           = "fr-par"
env              = "dev"
```
{% enddetails %}

# Automation

Let's automate our deployment with Gitlab.

The Gitlab pipeline defines two jobs:

* Initializing the infrastructure and terraform state.
* Deploying the infrastructure with `terraform apply`. 

Before each stage:

* We get the vault token from Google Secret Manager.
* Getting Scaleway credentials from Vault.

{% details .gitlab-ci.yaml %}

```yaml
stages:
  - init
  - deploy

# Get Scaleway credentials from Vault
before_script:
    - export VAULT_TOKEN="$(gcloud secrets versions access latest --secret=vault-token --project ${GCP_PROJECT_ID})"
    - export SCW_ACCESS_KEY="$(vault kv get -field=key scaleway/project/${SW_PROJECT_NAME}/credentials/access)"
    - export SCW_SECRET_KEY="$(vault kv get -field=key scaleway/project/${SW_PROJECT_NAME}/credentials/secret)"
    - export SCW_DEFAULT_ORGANIZATION_ID="$(vault kv get -field=id scaleway/project/${SW_PROJECT_NAME}/config)"

init sw infrastructure 🔬:
  stage: init
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    - cd envs/dev
    # Init terraform
    - |
       terraform init \
        -backend-config="bucket=bucket-${GCP_PROJECT_ID}-sw-gcp-terraform-backend" \
        -backend-config="prefix=scaleway/terraform/state" \
        ../../plan/
  artifacts:
    paths:
      - envs/dev/.terraform
  tags:
    - k8s-dev-runner
  only:
    - master 

deploy sw infrastructure 🚀:
  stage: deploy
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    - cd envs/dev
    # Deploy sw ressources
    - terraform apply -auto-approve ../../plan/ 
  tags:
    - k8s-dev-runner
  only:
    - master 
```
{% enddetails %}

Share the specific runner `k8s-dev-runner` created previously with this project. You will need `Maintainer` permission in Gitlab.

Now you can run the Gitlab pipeline with the following Gitlab CI/CD Variables:

```shell
GCP_PROJECT_ID=$GCP_PROJECT_ID
SW_PROJECT_NAME=$SW_PROJECT_NAME
VAULT_ADDR=$VAULT_ADDR
ENV=dev
```

# Conclusion

In the [last part][part5] we will see how to build, publish and deploy Docker images from Gitlab CI to Kubernetes using GitOps practices.

[part3]: https://dev.to/stack-labs/how-to-build-step-by-step-a-multi-cloud-infrastructure-with-google-cloud-and-scaleway-elements-part-3-5fg9
[part5]: https://dev.to/stack-labs/how-to-build-step-by-step-a-multi-cloud-infrastructure-with-google-cloud-and-scaleway-elements-part-5-1350
[kapsule]: https://www.scaleway.com/en/kubernetes-kapsule/