Package: gitlab-job-guard
gitlab-job-guardGuard Gitlab CI pipeline jobs from multiple parallel executions.$PRIVATE_TOKEN="$GITLAB_API_TOKEN"gitlab-job-guard
$my-unguarded-deployment-task--to=productiongitlab-job-guardwill  block  if  it  detects  other  pipelines  running  for
the  current  project   to  avoid  multiple  pipelines  from   clobbering  up  a
deployment/environment. This is  especially a problem when  jobs are distributed
and picked up by multiple gitlab runners.While gitlab willauto-cancel redundant, pending pipelinesfor the same branch
by  default -  this  is not  the  case for  multiple  pipelines fromdifferent
branchestargeting  a particular deployment/environment.  Gitlab has no  way to
detect  or control  these user-defined  branch-to-environment mappings  and this
means  environments  can  easily  be  left  in  an  unsafe/broken  state.  (e.g.terraform applyoransible, etc from different pipelines running at the same
time).gitlab-job-guarduses  the Gitlab API  to determine if existing  pipelines are
scheduled and to backoff-and-retry until it is  safe to proceed even if jobs are
distributed  across multiple  runners.  Conflicts are  detected by  user-defined
matches on pipeline ref names (branch, tag, etc) and/or pipeline status.UsageThe  simplest  use-case   would  likely  be  placinggitlab-job-guardin  abefore_scriptsection in yourgitlab-ci.ymlto protect all jobs (though this
can slow things down and be unfriendly to your local gitlab-ops).$GITLAB_API_TOKENis aPersonal Access
Tokenwith at minimumapiscope.before_script:-PRIVATE_TOKEN="$GITLAB_API_TOKEN" gitlab-job-guardThough often,  most pipelines only  need to guard  the critical jobs  that share
common state/data (i.e. a provisioning/deployment job, an artifact build/release
job, etc).deploy-production:stage:deployscript:-PRIVATE_TOKEN="$GITLAB_API_TOKEN" gitlab-job-guard-my-unguarded-deployment-task --to=productione.g To guard something critical like aterraformjob running for atag.provision-infrastructure:stage:provisionscript:-export PRIVATE_TOKEN="$GITLAB_API_TOKEN"-gitlab-job-guard --guard-ref-regex='^v[0-9\.]+'# Regex matches tags-terraform plan  ...-terraform apply ...only:-tagsOther usages--guard-ref-regextakes a regular expression to detect conflicts on other
pipelines whose ref name (branch, tag) is a (partial) match. This allows for
guarding of jobs in the complex use-cases.To hold jobs for a collisions on pattern matches on the branch/tag names.gitlab-job-guard-c=^master$# Match branch names matching 'master' exactlygitlab-job-guard-c=^(master|dev(elop)?)$# Match any of the mainline branchesgitlab-job-guard-c=^(feature|release|hotfix)/# Match any gitflow transient branch prefixesgitlab-job-guard-c=^[0-9]\-# Match branch names beginning with a number# and dash ignoring all other text.# e.g. a gitlab branch made from an issuegitlab-job-guard-c=^v?[\d.]+$# Match (semver) tags like v1.0.9, 2.0gitlab-job-guard-c=^environment/# Match any environment deployments?gitlab-job-guard-c=^environment/dc1.+# Match environment deployments to DC1?gitlab-job-guard-c="$CI_BUILD_REF_NAME"# Match current branch name (partially).# i.e. 'master' matches 'feature/master-document'gitlab-job-guard-c="^$CI_BUILD_REF_NAME$"# Match current branch name (exactly).# i.e. 'master' does not match 'master-deployment'gitlab-job-guard-c='.+'-s='running|pending'# Match any pipeline in running or pending stateTo hold a job for a collision on part of the ref name (e.g. on branch prefix
such asfeature/orhotfix/orrelease/, etca lagitflow).# Assuming current pipeline is for the 'feature/foo' branch and that# CI_BUILD_REF_NAME=feature/foo# Extract the branch prefixCI_BUILD_REF_PREFIX=$(echo"$CI_BUILD_REF_NAME"|sed-r's@(.+/)(.+)@\1@')# CI_BUILD_REF_PREFIX now contains 'feature/'# This will now block if there are any other pipelines running or pending# for feature/ branches.gitlab-job-guard-c="^$CI_BUILD_REF_PREFIX"-s='running|pending'Other argumentsTypicallygitlab-job-guardreads  in  the  following  environment  variables
to  compose   its  arguments.   NOTE:  environment  variables   take  precedence
over   arguments  passed   at  the   CLI  and   so  only   theuser-specifiedarguments/envvars  need  to be  set.  Setting  these asprotected  environment
variablesavoids
leaking sensitive information and needing to duplicate them for every call.envvar              |  argument             | source
-------------------   ---------------------   ----------------
GUARD_REF_REGEX     | --guard-ref-regex     | user specified
GUARD_STATUS_REGEX  | --guard-status-regex  | user specified
PRIVATE_TOKEN       | --private-token       | user specified
GUARD_TIMEOUT       | --guard-timeout       | user specified
CI_PROJECT_ID       | --ci-project-id       | runner environment
CI_BUILD_REF_NAME   | --ci-build-ref-name   | runner environment
CI_API_V4_URL       | --ci-api-v4-url       | runner environment
CI_PIPELINE_ID      | --ci-pipeline-id      | runner environmentSeegitlab-job-guard -hfor the full argument set and current/default values.Behaviourgitlab-job-guardwill backoff-and-retry  until it  is  safe to  proceed or  a
timeout is  reached. An exit code  of0indicates no  other pipelines conflict
and that the job  can continue. On reaching the timeout (1  hourby default to
align with the default timeout for gitlab jobs), it will return an error to fail
the job. The timeout in seconds can be set via-w/--guard-timeout.If  a  fail-fast  approach  is   needed,  the-x/--no-waitflag  will  causegitlab-job-guardto  exit immediately  on conflicts.  (Though, this  means the
pipeline run will have to be restarted some other way or abandoned).TODOFor  long-running pipelines,  this solution  can have  subtle consequences  with
growing  queues  and  increased  contention and  unpredictability  as  to  which
pipeline is  the first-past-the-post. An  older pipeline taking  precedence over
newer  commits if  often  not  desired and  newer  pipelines  always winning  is
probably desired.Handle existing conflicting pipelines - cancel them or give-way.Narrow down conflicts to jobs (CI_JOB_NAME) or stages (CI_JOB_STAGE)
so that other parts of the pipelines that do not share state are allowed to
run freely.