Package: gitlab-runner-tart-driver
Gitlab Runner Custom Tart DriverGitlab Runner Custom Tart DriverAboutSetupGitLab Runner ConfigurationGitLab CISupported Environment VariablesPrivate OCI RegistriesConfiguring the SSH CredentialsAdvanced ConfigurationGitLab Runner InstallationConcurrencyHost Cache and Builds directoriesVolume MountsAuto Host Resource DistributionCustomshellCustompull_policyCLICLI Parameters forconfig.tomlCommandconfigCommandprepareCommandrunCommandcleanupAboutTartis a virtualization toolset to build, run and manage macOS and Linux virtual machines on Apple Silicon. Tart is using Apple’s native Virtualization.Framework that was developed along with architecting the first M1 chip. This seamless integration between hardware and software ensures smooth performance without any drawbacks.For storing virtual machine images Tart integrates withOCI-compatible container registries. This allows you to work with virtual machines as you used to with Docker containers.This Custom GitLab Runner Extension integratesTartsemlessly into the GitLabCI ecosystem with similar functionality as the welldocker-runnerExecutorTartDockerClean build environment for every build✓✓Reuse previous clone if it exists✓✓Runner file system access protected✓✓Migrate runner machine?✓Zero-configuration support for concurrent builds✓✓Complicated build environments✗✓Debugging build problemseasymediumSetupATTENTIONThis setup will only work on an Apple Silicon based machineTo setup thegitlab-runner-tart-driveryou will need to go through the following stepsInstallHomebrewInstallpython3InstalltartInstallgitlab-runnerInstallgitlab-runner-tart-driverConfigure and register acustomgitlab-runnerwith your GitLab InstanceUpdateconfig.tomlTo installHomebrewsimply execute the command/bin/bash-c"$(curl-fsSLhttps://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"Next we need to installtartby executingbrewinstallcirruslabs/cli/tartWe can verify to virtualization functionality by pulling an image for the officialtart managed imagestartcloneghcr.io/cirruslabs/macos-ventura-base:latestventura-base
tartrunventura-baseAfter the image has been downloaded you will see a window come up with a virtualized OSX Ventura runningTo run thegitlab-runner-tart-driverwe will need a validpython3installation as well as thegitlab-runnerbrewinstallpython3gitlab-runnerFinally we will need to installgitlab-runner-tart-driver.pipinstallgitlab-runner-tart-driverGitLab Runner ConfigurationFollow the guideInstall GitLab Runner on macOS.Registeryour GitLab Runner and use thecustomexecutorgitlab-runnerregisterInstall the service withgitlab-runner installATTENTIONThe GitLab Runner currently fails to properly setup the LaunchAgent. The workaround described can be omitted as soon the theBughas been resolved# Fix missing log path issuesudomkdir-p/usr/local/var/log
sudochmoda+rw/usr/local/var/log

gitlab-runneruninstall
gitlab-runnerinstall
gitlab-runnerstartConfigure theconfig.tomlfor the tart custom driverTheconfig.tomlshould be located at~/.gitlab-runner/config.toml. Open it and add the following contentconfig.toml[runners.custom]config_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"config_args=[ "config", '-x', '/opt/homebrew/bin/tart' ]config_exec_timeout=200prepare_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"prepare_args=[ "prepare", '-x', '/opt/homebrew/bin/tart' ]prepare_exec_timeout=200run_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"run_args=[ "run", '-x', '/opt/homebrew/bin/tart' ]cleanup_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"cleanup_args=[ "cleanup", '-x', '/opt/homebrew/bin/tart' ]cleanup_exec_timeout=200Now restart the gitlab-runner withgitlab-runner restartGitLab CIOne of the great advantages in usingTartis that it gives almost all functionality to we are used from the standarddockerexecutors allowing us to write GitLabCI piplines that do not need any or minimal adaptions for OSX.ATTENTION:The driver will always automatically login to your GitLab Registry usingCI_REGISTRY_USER,CI_REGISTRY_PASSWORDandCI_REGISTRYstages:-buildrun-on-tart:image:ghcr.io/cirruslabs/macos-ventura-base:lateststage:-buildtags:-tartbefore_script:-brew install gitlab-runner# you will need to make sure `gitlab-runner` executable is available to be able to upload artifactsscript:-echo "This is brought to you by tart" >> artifact.txtartifacts:paths:-artifact.txtSupported Environment VariablesNameDefaultDescriptionTART_REGISTRY_USERNAMEUsername to login to private OCI registryTART_REGISTRY_PASSWORDPassword to login to private OCI registryTART_REGISTRYPrivate OCI registryTART_PULL_POLICYif-not-presentdefine how runners pull tart images from registries. Options arealways,if-not-present,neverTART_SSH_USERNAMEadminUsername to use to login to VMTART_SSH_PASSWORDadminPassword to use to login to VMTART_EXECUTOR_HEADLESStrueDon't open a UI window.TART_EXECUTOR_SOFTNET_ENABLEDfalseUse software networking instead of the default shared (NAT) networkingTART_EXECUTOR_VNC_ENABLEDfalseUse screen sharing instead of the built-in UI. Note that Remote Login option should be enabled inside the VM.TART_EXECUTOR_INSTALL_GITLAB_RUNNERfalseInstall a GitLabRunner into the VM to ensure full CI/CD functionalityTART_EXECUTOR_SHELL/bin/zshThe shell that should be used when running commandsTART_EXECUTOR_TIMEOUT60Timeout fortart ipto respond with a valid IPTART_EXECUTOR_DISPLAY1920x1200Display resolution in formatWxHPrivate OCI RegistriesOftentimes you might want to provide your own OCI-compliant images created withPackerand the officialTart Builder. If you push your images to a private registry you will need to provide the credentials for thegitlab-runner-tart-driverto login there first.You can provide a default login registry using theCLIparameters (seeCommandprepare) but also provide the credentials from with each job variable definitionstages:-buildvariables:TART_REGISTRY_USERNAME:myuser@myregistry.comTART_REGISTRY_PASSWORD:<some_secret>TART_REGISTRY:private.registry.iojob1:image:private.registry.io/tart/xcode14:lateststage:-buildtags:-tartbefore_script:-brew install gitlab-runner# you will need to make sure `gitlab-runner` executable is available to be able to upload artifactsscript:-echo "This is brought to you by tart" >> artifact.txtartifacts:paths:-artifact.txtjob2:image:private.other.io/tart/xcode14:latestvariables:# override the OCI login information for this job onlyTART_REGISTRY_USERNAME:myuser@other.comTART_REGISTRY_PASSWORD:<some_secret>TART_REGISTRY:private.other.iostage:-buildtags:-tartbefore_script:-brew install gitlab-runner# you will need to make sure `gitlab-runner` executable is available to be able to upload artifactsscript:-echo "This is brought to you by tart" >> artifact.txtartifacts:paths:-artifact.txtConfiguring the SSH CredentialsTheTart managed imagescome with a default useradminand passwordadminwhich will be used to establish an ssh connection. If you decide to provide your own images, you might opt for a different user and password. You can provide the default user and password using theconfig.tomlwithin therunsection using--default-ssh-userand--default-ssh-passwordor provide it from within your jobjob2:image:private.other.io/tart/xcode14:latestvariables:# override the OCI login information for this job onlyTART_REGISTRY_USERNAME:myuser@other.comTART_REGISTRY_PASSWORD:<some_secret>TART_REGISTRY:private.other.ioTART_SSH_USERNAME:gitlabTART_SSH_PASSWORD:gitlabstage:-buildtags:-tartbefore_script:-brew install gitlab-runner# you will need to make sure `gitlab-runner` executable is available to be able to upload artifactsscript:-echo "This is brought to you by tart" >> artifact.txtartifacts:paths:-artifact.txtAdvanced ConfigurationGitLab Runner InstallationTo have the full GitLabCI functionality in your pipelines like the capabilties to upload artifacts, you will need toensurethat thegitlab-runneris available within your VM. Thepreparescript can help you to ensure the presence and even install a specific version for you.
Use--install-gitlab-runnerto tell the driver to check for a valid runner installation. Per default, if an installation is found that version is used. If you want to ensure that you have always thelatestor a specific version you can use--force-install-gitlab-runnerand--gitlab-runner-versionto indicate your concrete setup.The installation will follow these simple steps to ensure the propergitlab-runnersetup.Check if gitlab-runner is already installed by trying to execute itIf the runner is not present or force install is enabledCheck ifcurlcan be used to download and install the version specified to/usr/local/binCheck ifwgetcan be used to download and install the version specified to/usr/local/binCheck ifbrewcan be used to install the version specifiedFail if none of the tools is availableConcurrencyCurrentlytartonly supports two executions of VMs at the same time. This limits the scalability of the solution on a single host. Please ensure you are setting theconcurrentsetting to a maximum of2in yourconfig.toml.concurrent=2# <-- ATTENTION: ensure not sure go higher than '2'check_interval=0shutdown_timeout=0[session_server]session_timeout=1800[[runners]]name="macos-tart-driver1"url="https://gitlab.com/"id=11111111token="XXXXXXXXXXXX"token_obtained_at=2023-04-03T07:21:42Ztoken_expires_at=0001-01-01T00:00:00Zexecutor="custom"[runners.cache]MaxUploadedArchiveSize=0[runners.custom]config_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"config_args=[ "config" ]prepare_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"prepare_args=[ "prepare", '-x', '/opt/homebrew/bin/tart', '--concurrency', '2', '--auto-resources' ]run_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"run_args=[ "run", '-x', '/opt/homebrew/bin/tart' ]cleanup_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"cleanup_args=[ "cleanup", '-x', '/opt/homebrew/bin/tart' ]Host Cache and Builds directoriesWithtartit is possible to mount local directories into the VM. Mounting local directories brings a number of benefits like up tp 30% higher IO performance as well as a correct caching mechanism.
To enable Host-local caching and builds, simply pass--builds-dirand--cache-dirto thepreparecommandprepare_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"prepare_args=[ "prepare", '-x', '/opt/homebrew/bin/tart', '--concurrency', '2', '--auto-resources', '--cache-dir', '/Users/gitlab/gitlab-runner/cache', '--builds-dir', '/Users/gitlab/gitlab-runner/builds']Volume MountsJust like withdockerthegitlab-runner-tart-driverallows you to mount any arbitrary host path into your VM. This can be especially useful if you have to mount runner-local resources like shared test-data or additional caches into the executor. You can follow the standard docker syntax forvolume mounts--volume /my/local/dir:/opt/remote/dir. Additionally you can also pass theroflag to make the mountread-onlyi.e.--volume /my/local/dir:/opt/remote/dir:ro.[runners.custom]config_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"config_args=[ "config" ]prepare_exec="/opt/homebrew/bin/gitlab-runner-tart-driver"prepare_args=[ "prepare",'-x', '/opt/homebrew/bin/tart','--concurrency', '2','--auto-resources','--volume /data/models/:/opt/data/models:ro']Auto Host Resource Distributiontartimages come with a pre-defined number of CPUs and Memory allocation. Typical numbers for default images arecpu_count=4andmemory=8192. With the concurrency limitation of two VMs/images running at the same time this might not utilize your host system completely.
Per default,--auto-resourcesis enable for thegitlab-runner-tart-driverwhich will split the host resources equally to the VMs defined by--concurrency. The default concurrency is1and therefore will assign all host resources to the VM.ATTENTIONmake sure yourgitlab-runnerconcurrency setting is the same as the--concurrencyparameter you are passing to thepreparecommandCustomshellYou can use a custom shell to execute your commands. The default ist/bin/zshseeCommandrunCustompull_policyYou can use a custompull_policy. The default policy isif-not-present. UseTART_PULL_POLICYto override the default pull policyCLICLI Parameters forconfig.tomlThegitlab-runner-tart-drivergives a number of advanced configuration options. Usegitlab-runner-tart-driver [stage] --helpto get a full list of options that you can pass to the exectuable using yourconfig.tomlfile.Usage: gitlab-runner-tart-driver [OPTIONS] COMMAND [ARGS]...

Options:
  -v, --version  Show the version and exit.
  --help         Show this message and exit.

Commands:
  cleanup  Command to greet a user.
  config   Implementation of the CONFIG stage of the Custom Executor.
  prepare  Prepare the environment and start the tart VM.
  run      Run commands.CommandconfigUsage: gitlab-runner-tart-driver config [OPTIONS]

  Implementation of the CONFIG stage of the Custom Executor. Details on how to
  use this command can be found at
  https://docs.gitlab.com/runner/executors/custom.html#config.

Options:
  --help                      Show this message and exit.CommandprepareUsage: gitlab-runner-tart-driver prepare [OPTIONS]

  Prepare the environment and start the tart VM.

Options:
  --cpu INTEGER                   Number of CPUs associated to VM
  --memory INTEGER                VM memory size in megabytes associated to VM
  --display TEXT                  VM display resolution in a format of
                                  <width>x<height>. For example, 1200x800
  --auto-resources / --no-auto-resources
                                  If enabled, the driver will divide system
                                  resources equally to the concurrent VMs.
  --concurrency INTEGER           Number of concurrent processes that are
                                  supported. ATTENTION tart currently only
                                  support two concurrent VMs
  --cache-dir TEXT                Caching dir to be used.
  --builds-dir TEXT               Path to the builds directory.
  --timeout INTEGER               Timeout in seconds for the VM to be
                                  reachable via SSH.
  --volume TEXT                   Volume mount definition with docker syntax.
                                  <host_dir>:<vm_dir>[:ro]
  --install-gitlab-runner         Will install the gitlab-runner if not
                                  present.
  --force-install-gitlab-runner   This will force the installation of the
                                  GitLab Runner independent of a previously
                                  installed version
  --gitlab-runner-version TEXT    The version of the GitLab Runner to be
                                  installed. Example '15.11.0'
  -x, --tart-executable TEXT      Path to the tart executable.
  --help                          Show this message and exit.CommandrunUsage: gitlab-runner-tart-driver run [OPTIONS] SCRIPT STAGE

  Run commands.

Options:
  --timeout INTEGER            SSH connection timeout in seconds
  -x, --tart-executable TEXT   Path to the tart executable.
  --shell TEXT                 Path to the shell to be used for commands over
                               ssh.
  --help                       Show this message and exit.CommandcleanupUsage: gitlab-runner-tart-driver cleanup [OPTIONS]

  Command to greet a user.

Options:
  -x, --tart-executable TEXT  Path to the tart executable.
  --help                      Show this message and exit.