Gitlab CI/CD with CSI PowerMax
Tags: kubernetes, csi, devops, powermax
Published: 2020-07-06 09:06:40

---
title: Gitlab CI/CD with CSI PowerMax
published: true
date: 2020-05-29 07:00:00 UTC
tags: #kubernetes #csi #devops #powermax
canonical_url: https://storage-chaos.io/gitlab-powermax.html
---

# TL; DR

Watch the [basic deployment](https://youtu.be/dfKPWqKMuGk) & [snapshot-based deployment](https://youtu.be/6sClYeToXRg) videos on Youtube and check the [.gitlab-ci-cd.yaml](https://gitlab.com/coulof/todos) on Gitlab.

# The premise

For the first release of the [CSI Driver for PowerMax](https://github.com/dell/csi-powermax) we wanted to show how the PV dynamic provisioning and snapshot capabilities.

To present a realistic scenario, we used Gitlab CI/CD, its [Kubernetes runner](https://docs.gitlab.com/ee/user/project/clusters/add_remove_clusters.html#adding-and-removing-kubernetes-clusters), and the CSI Driver off course.

The application itself is a fork of the [VueJS example app TODO](https://vuejs.org/v2/examples/todomvc.html), which we modified to use [Sinatra as an API provider](https://gitlab.com/coulof/todos/-/blob/master/server.rb) and SQLite to store the TODOs.

# The implementation

The concept is:

- the master branch corresponds to the latest image and is the production environment
- anytime we push a new branch to GitLab we:    
  - build the image
  - take a snapshot of PV from production
  - create an [environment](https://docs.gitlab.com/ee/ci/environments/) to access the new app
- new commits on the branch will keep using their own environment with an independent PV
- on branch merge:    
  - the dedicated environment and related PV are deleted
  - the production is redeployed with the latest image

Most of the magic on the storage layer happens in the [PVC](https://gitlab.com/coulof/todos/-/blob/master/deploy/todos/templates/pvc.yaml), [Snap](https://gitlab.com/coulof/todos/-/blob/master/deploy/todos/templates/snap.yaml) definitions, and with the Helm variables.

we can see that, if the branch is the latest, we deploy a dedicated volume (i.e. only the first time). For every other branches we start from a snapshot taken at the time of the first branch creation.

Under the scene, we will have two independent volumes in PowerMax. For more deep-dive on PowerMax SnapVX (i.e. PowerMax local replicas) you can check that [white paper](https://www.dellemc.com/resources/en-us/asset/white-papers/products/storage/h13697-dell-emc-powermax-vmax-all-flash-timefinder-snapvx-local-replication.pdf).

<noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/25d38cc5fc58fa16df71f2e87dfcb840.js?file=pvc.yaml"> </script>

To avoid, the storage box to be bloated by the project, we also defined a resource Quota on the namespace.

<noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/25d38cc5fc58fa16df71f2e87dfcb840.js?file=quota.yaml"> </script>
* * *

**Limitations**

> The current version of the CSI driver (v1.2) the snapshot API is v1alpha1 and not compatible with Kubernetes v1.17 and beyond.
> 
> A snapshot is only accessible from the [same namespace](https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-cis-volume-snapshot-beta/#importing-an-existing-volume-snapshot-with-kubernetes) and cannot restore a volume on a different namespace.

## Other tips

One of the [tricks](https://gitlab.com/coulof/todos/-/blob/master/.gitlab-ci.yml#L41) is is to put the Gitlab variable `CI_COMMIT_SHORT_SHA` in the helm template ; that way, we make sure it is re-proceed and therefore redeployed with the latest build by Helm.

Finally, to save some time in building the images, I used [local gems](https://gitlab.com/coulof/todos/-/blob/master/Dockerfile#L11).

# Videos

For a live demo, check the videos here:

<iframe height="560" src="https://www.youtube.com/embed/dfKPWqKMuGk" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe><iframe height="560" src="https://www.youtube.com/embed/6sClYeToXRg" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>