Package: gitlab-multi-group-runner
GitLab Multi Group RunnerOverviewUntil the creation of this README (July 2021), GitLab lacks the feature of assigning a CI runner to multiple groups (seeissue #23722 in the GitLab code repository). This project
circumvents this limitation by assigning runners to all projects of given groups with the GitLab API.This tool can be used as a command line program from any machine which has network access to your GitLab instance.
Alternatively, you setup it as a custom GitLab runner which is triggered from a configuration repository.This tool needs administrator access to your GitLab instance.InstallationInstall as a standalone command line toolgitlab-multi-group-runneris available on PyPI for Python 3.5+ and can be installed withpip:python3-mpipinstallgitlab-multi-group-runnerpipwill create an executablegitlab-multi-group-runner.If you use Arch Linux or one of its derivatives, you can also installgitlab-multi-group-runnerfrom theAUR:yay-Spython-gitlab-multi-group-runnerYou also find self-contained executables for 64-bit Linux distributions and macOS High Sierra and newer on thereleases
page. They are created withPyInstallerand only require glibc >= 2.17 on Linux (should be fine on any recent Linux
system).Install as a custom GitLab runnerThis is the recommended setup.You can follow these steps:Create a new empty GitLab repository which will contain the configuration file.Install GitLab runner on a dedicated host as described in theGitLab runner
documentation(of course a virtual machine or an LXC is also sufficient).Go to the webpage of your fresh GitLab repository and navigate toSettings->CI/CD->Runnersand copy theregistration tokenfrom theSpecific runnerssection. DisableShared runners.Run theregistercommand on the GitLab runner hostgitlab-runnerregisterand enter these values:FieldValueGitLab instance URLURL to your GitLab instanceRegistration tokenCopied beforeDescriptionmulti-group-configuration-runnerTagsEmptyExecutorcustomClone the GitHub repositoryhttps://github.com/sciapp/gitlab-multi-group-runneronto your GitLab runner host and run
the installation:gitclonehttps://github.com/sciapp/gitlab-multi-group-runnercdgitlab-multi-group-runner
makeinstallBy default, themake installcommand will install a custom GitLab runner driver to/opt/multi-group-runner-driver.
You can passPREFIXto the make command to change the installation destination.Edit/etc/gitlab-runner/config.toml:Configure thebuilds_dirandcache_dirsettings:[[runners]]builds_dir="/home/gitlab-runner/builds"cache_dir="/home/gitlab-runner/cache"Configure therunstage:[runners.custom]run_exec="/opt/multi-group-runner-driver/bin/run_stage"run_args=["-f","/opt/multi-group-runner-driver/etc/gitlab_multi_group_runnerrc.yml"]Create a configuration file/opt/multi-group-runner-driver/etc/gitlab_multi_group_runnerrc.yml. See the sectionConfigurationfor more details.Restart the GitLab runner service:sudosystemctlrestartgitlab-runnerCreate a configuration filemulti-group-runner-config.ymlin the top level directory of the GitLab repository. SeeConfigurationfor more details.Create a CI configuration file.gitlab-ci.ymlnext tomulti-group-runner-config.ymlwith the content:stages:-apply-configapply-config:stage:apply-configscript:noopThis minimal configuration file is needed to trigger a CI pipeline when new commits are pushed to the repository and
to activate the custom CI runner.(Optional) Go to the webpage of your fresh GitLab repository and navigate toCI/CD->Schedulesand create a
repeating event to configure new projects automatically in a defined interval.When the CI pipeline is triggered, the configuration file in the repository is read and the runner will reconfigure the
runners in the configured groups and projects. You should see something likeRunning on multi-group-configuration-runner...
[INFO] Disable shared runners in project "mygroup/myproject"
[INFO] Enable runner "intel-docker", (id: 10, tags: ["docker", "intel"]) in project "mygroup/myproject"
[...]in your CI job log.Configurationgitlab-multi-group-runneruses two configuration files:The first configuration file must be located on the same machine as the executablegitlab-multi-group-runner. It
contains parameters which are needed for the GitLab API access and specifies which runners and groups are allowed to
be configured and which configuration repositories are accepted (see next step). This is an example configuration file
(can also be printed withgitlab-multi-group-runner --print-example-config):general:disable_shared_runners:truegitlab:auth_token:xxxxxxxxxxxxxxxxxxxxurl:https://mygitlab.comrunners:-allowed_projects_rules:one_member_of:-my-group-name-my-user-nameids:-1-3config_repo:branch:masterpath:administration/my-multi-group-runnersSome notes:disable_shared_runnersspecifies if shared runners will be disabled inallrepositories which are reconfigured
by this tool. Set it tofalse, to not touch shared runners.Theauth_tokenmust be a token for the administrator account withapiandread_repositoryaccess. Login asrootand go toPreferences->Access Tokensto generate a new token.allowed_projects_rulesis a set of rules to identify projects which are allowed to be configured. Currently, only
the ruleone_member_ofis supported. The value is a list of groups and users from which it least one user must be
a member of the project which shall be configured.Example: Iffoois given, then all projects in the groupfooare allowed to be configured. If another
projectbarhas a member of groupfoo, the projectbarcan be configured as well.idsis a list of runner ids which are allowed to be assigned to the projects defined byallowed_projects_rules.
TheAdmin Areaof your GitLab instance contains aRunnerssection which lists all runners and their ids.Important: Only specific runners can be used (so group runners must be re-registered as specific runners first).
It does not matter if specific runners are locked or not.config_repospecifies the GitLab repository which contains the second configuration file. The configuration
repository specifies concretely which runners are assigned to which groups and projects.Note thatallowed_projects_rules,idsandconfig_repoform a list item of therunnerslist. You can specify
as much triples as you need to describe which configuration repositories can configure which combinations of
runners, groups and projects.As noted before, the second configuration part is located in a Git repository on your GitLab instance. It must be
located in the top level root and must be namedmulti-group-runner-config.yml. This is an example configuration (can
also be printed withgitlab-multi-group-runner --print-example-repo-config):runners:-groups_and_projects:-mygroup-myusername/myprojectids:-1-3Therunnerssection is a list of group/project and runner combinations. It configures which runners will be assigned
to which concrete projects and groups.UsageUsage of the standalone command line toolRungitlab-multi-group-runnerwith a local configuration file and a configuration repository path:gitlab-multi-group-runner-fmy_config.ymladministration/my-multi-group-runnersgitlab-multi-group-runnerwill read both configuration files, check if the settings inadministration/my-multi-group-runnersare allowed and make the appropriate GitLab API calls to add the runners to the
given projects.You can run with the--allparameter to fetch all configuration repositories which are defined inmy_config.yml.Usage as a custom GitLab runnerPush a new commit to your configuration repository and wait for the CI pipeline to complete. That's it!ContributingPlease openan issue on GitHubif you experience bugs
or miss features. Please consider to send a pull request if you can spend time on fixing the issue yourself. This
project usespre-committo ensure code quality and a consistent code style. Runmakegit-hooks-installto install all linters as Git hooks in your local clone ofgitlab-multi-group-runner.