Efficient DevSecOps Pipelines in a Cloud Native World - free workshop
Tags: gitlab, devsecops, workshop, cicd
Published: 2021-12-10 16:37:58

---
title: Efficient DevSecOps Pipelines in a Cloud Native World - free workshop
published: true
description: Practice pipeline efficiency with GitLab in this 10h+ free workshop
tags: gitlab,devsecops,workshop,cicd
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c9j8fftw1s2aoa5cwft4.png
---

Getting started with CI/CD has become more easy with the [DevOps platform era](https://about.gitlab.com/blog/2021/08/03/welcome-to-the-devops-platform-era/). GitLab provides a new pipeline editor to help with onboarding, and allow to prevent common configuration mistakes.

Pipelines and their definitions are always changing, and a work in progress similar to source code development. It is easy to create a long lasting pipeline which blocks reviews from fast feedback, or waste ten thousands of dollars of cloud compute resources.

The overall question is - how to create efficient DevSecOps pipelines, and make use of all available options: Configuration, resources, CI/CD infrastructure, Security and Observability. 

Based on the talk I had created after creating the CI/CD Pipeline Efficiency docs after a CI monitoring webinar ... 

* [Continuous LifeCycle 2021: Efficient DevSecOps Pipelines in a Cloud Native World](https://docs.google.com/presentation/d/1nq7Q4WMv6rQc6WFJCRqjtgtgQfgtKTtwSFw4IXXuRsA/edit)
* [AWS UG Nuremberg: Efficient DevSecOps Pipelines](https://docs.google.com/presentation/d/1NwKIJ-PG9xfV7CVGJC3N1Cn7KhPaFTwMhVBi98WAMDI/edit)
* [cdCon: Dev loves CI/CD: Efficient Sec and Ops Pipelines](https://www.youtube.com/watch?v=p2B0COC1u64)
* [Tweakers Dev Summit: Devs love CI/CD: Efficient DevSecOps Pipelines](https://docs.google.com/presentation/d/1jZYHIXjL6CDBQVblDvxUvBFl8Gbz4LTx5yCx-iwRsqg/edit) 
* [2. German Connect Day: Efficient Sec and Ops Pipelines](https://docs.google.com/presentation/d/1EiBoaTOFx-bE2HkNT2v1ucXbAnwedc9v26ElLk9_fks/edit)
* [IT-Tage 365: Devs ❤️ CI/CD: Efficient Sec and Ops Pipelines](https://docs.google.com/presentation/d/1emmdaQmqunV8fIlfLMB2gk6JHtVljKQ4slUaLBMrPIQ/edit)
* [Open Source Automation Days: Developers love CI/CD: The Sec and Ops Sequel](https://www.youtube.com/watch?v=QA2KeufcPdQ)
* [Webinar: Identify, analyze, action! Deep monitoring with CI](https://docs.google.com/presentation/d/1ONwIIzRB7GWX-WOSziIIv8fz1ngqv77HO1yVfRooOHM/edit)

... I have now built a 10+ hours workshop for Open Source Automation Days 2021 with old and new best practices for everyone to learn and benefit. I've added new topics focussing on auto-scaling and cloud native deployments, whilst enriching the configuration examples with many hidden gems. 

Yesterday, a discussion about a great blog post ["Let's make faster GitLab CI/CD pipelines"](https://blog.nimbleways.com/let-s-make-faster-gitlab-ci-cd-pipelines/) on [Hacker News](https://news.ycombinator.com/item?id=29502999) inspired me to share the workshop and now blog about it.  You can find all workshops in the [GitLab Developer Evangelism handbook](https://about.gitlab.com/handbook/marketing/community-relations/developer-evangelism/projects/#workshops).

You can learn async at your own pace, with the [slides](https://docs.google.com/presentation/d/12ifd_w7G492FHRaS9CXAXOGky20pEQuV-Qox8V4Rq8s/edit) providing the exercises and the solutions in a [workshop project](https://gitlab.com/gitlab-de/workshops/ci-cd-pipeline-efficiency-workshop). Note that there is one exercise which needs preparations for auto-scaling which is described in the slides. 

The following topics will be practiced:

- Introduction: CI/CD meets Dev, Sec and Ops
- CI/CD: Terminology and first steps
  - Pipeline Editor 
- Analyse & Identify
  - Learn using the [GitLab CI Pipeline Exporter](https://github.com/mvisonneau/gitlab-ci-pipelines-exporter/tree/main/examples/quickstart) to monitor the exercise project throughout the workshop.
- Efficiency actions
  - Config Efficiency: CI/CD Variables in variables, job templates (YAML anchors, extends), includes (local, remote), rules and conditions (if, dynamic variables, conditional includes), `!reference` tags (script, rules), maintain own CI/CD templates (include templates, override config values), parent-child pipelines, multi project pipelines, better error messages to fix failures fast 
  - Resource Use Efficiency: Identification, max pipeline duration analysis, fail fast with stages grouping, fail fast with async needs, analyse blocking stages pipeline (solution with needs), matrix builds for parallel execution (pratice: combine matrix and `extends`, combine matrix and `!reference`), `extends` merge strategies (with and without `!reference`)
  - CI/CD Infrastructure Efficiency: Optimization ideas, custom build images, optimize builds with C++ as example, GitLab runner resource analysis (sharing, tags, external dependencies, Kubernetes), local runner exercise, resource groups, storage usage analysis, caching (Python dependency exercise, including `when:always` on failed jobs) 
  - Auto-scaling: Overview, AWS auto-scaling with GitLab Runner with Terraform, insights into [Spot Runners on AWS Graviton](https://about.gitlab.com/blog/2021/08/17/100-runners-in-less-than-10mins-and-less-than-10-clicks/)
- Group discussion
  - Deployment Strategies: IaC, GitOps, Terraform, Kubernetes, registries
  - Security: Secrets in CI/CD variables, Hashicorp Vault, secrets scanning, vulnerability scanning
  - Observability: CI/CD Runner monitoring, SLOs, quality gates, CI/CD Tracing
  - More efficiency ideas: Auto DevOps, Fast vs Resources, Conclusion and tips

Take your time with learning, the workshop has a lot of topics inside. Let me know how it goes, and join https://forum.gitlab.com/ for questions and feedback (please tag @dnsmichi). 🤗 

[Start exploring now](https://docs.google.com/presentation/d/12ifd_w7G492FHRaS9CXAXOGky20pEQuV-Qox8V4Rq8s/edit)

Everyone Can Contribute - learned a new gem from this workshop? Blog about it and share on social tagging [@dnsmichi](https://twitter.com/dnsmichi) [@gitlab](https://twitter.com/gitlab)! 💡 


_Blog originally published on https://dnsmichi.at/2021/12/10/efficient-devsecops-pipelines-in-a-cloud-native-world-free-workshop/_