Package: mirror-tool
mirror-toolA tool for maintaining Git subtree mirrors.mirror-toolInstallationUsagemirror-tool validate-configmirror-tool update-localmirror-tool updatemirror-tool promotemirror-tool gitlab-ci-ymlConfigurationJinja contextLicenseInstallationInstall themirror-toolpackage from PyPI.pip install mirror-toolAlternatively, runmirror-tooldirectly as a container image.Note that you must expose the git repo you want to manage as a directory to the
container as the current working directory, as well as setting any needed
environment variables if GitLab features are enabled. The details of how to do
this are beyond the scope of this README, but an example command may look like:podman run \
  -v $PWD:/workdir:Z -w /workdir \
  quay.io/rmcgover/mirror-tool:latest \
  update-localUsagemirror-toolhas the following subcommands. Please install the tool and
run with--helpfor complete documentation on the available commands and
options.mirror-tool validate-configVerify that.mirror-tool.yamlin the current directory, or a specified
configuration file, is valid.Exits with a 0 exit code if and only if a valid config file was found.mirror-tool update-localFor each mirror defined in the config file, create a subtree merge commit
updating that mirror.By default, this will not create any commits if there are no changes to be made.
It can be forced to create a commit by using the--allow-emptyargument.mirror-tool updatePerform the same updates asupdate-local, but also push the commit(s) to any
configured remote targets.Currently, GitLab is the only supported target. See the configuration reference
below for more information about GitLab integration.When using this command to push to GitLab, it is recommended to run it from
within a GitLab CI/CD pipeline. The command will use predefined environment
variables in the CI environment to determine how to connect to GitLab.
If used in other contexts, it will be necessary to explicitly set many
environment variables.mirror-tool promoteFor any merge requests previously created byupdate, create additional
merge request(s) to promote the same changes to other branch(es) as defined
in config.This command can be used to implement a multi-tiered deployment/update workflow,
for example:Whenever mirrored repos change, create an MR updating them (viamirror-tool update), targetingtestingbranch.Perform some pre- or post-merge testing on that MR by some means (outside the
scope of mirror-tool).After the MR is submitted totestingbranch, create a new MR promoting the
same changes tostablebranch (viamirror-tool promote).The command only operates on changes previously created viamirror-tool update.Likeupdate, GitLab is currently the only supported target for this command.mirror-tool gitlab-ci-ymlGenerates a.gitlab-ci.ymlsnippet with a recommended configuration for integrating
mirror-tool into a GitLab pipeline.It is recommended to put the generated config into its own YAML file and use theincludekeyword to include that file.The output of this command is influenced by the.mirror-tool.yamlconfig.
When changing configuration elements relating to GitLab, it is a good idea to
re-run this command.Configurationmirror-toolrequires a configuration file. By convention, this should
be placed at.mirror-tool.yamlat the top level of your superproject
repository.The following example demonstrates the available configuration options.# Define the repositories to mirror.mirror:-url:https://github.com/org/repo1ref:refs/heads/masterdir:repo1-url:https://github.com/org/repo2ref:refs/heads/maindir:repo2# Git configuration to be applied when mirror-tool creates commits.# Any arbitrary config can be set, but this is most commonly needed# just to set the name/email on merge commits.git_config:user.name:"mirror-tool"user.email:"noreply@example.com"# Message for generated commits.# This is a Jinja template.commitmsg:|-Merge {{commits[0].revision_abbrev}} to {{mirror.dir}}{{commits|length}} commit(s) are being merged.{% for commit in commits %}- {{ commit.revision_abbrev }} {{ commit.subject }}{%- endfor %}# Configures the GitLab merge request integration.gitlab_merge:# If enabled, the update command will create/update a GitLab# merge request whenever a mirrored repo is updated.enabled:true# Token used to authenticate with GitLab.## Currently, this token must always be of the format '$SOME_VARIABLE',# and the token will be accessed from that environment variable at# runtime. If running from a GitLab CI/CD pipeline, this should be# set as a protected CI variable.token:$GITLAB_MIRROR_TOKEN# Source branch used for merge requests.# WARNING: update will do force pushes to this branch!src:latest# Target branch used for merge requests.# The following example supposes that the target branch is used# to perform some kind of deployment.dest:deploy# Title for the merge request.# This is a Jinja template.title:"Deploychanges[{{datetime_day}}]"# Any desired labels to add onto the merge request.labels:-deploy# Description for the merge request.# This is a Jinja template.description:|-Automated update of dependencies generated by{{ env.CI_JOB_URL }}.Submitting this merge request will trigger a deployment.# Comment(s) to be added when a merge request is created or updated.# Can be used to ping reviewers.# If omitted, comments won't be added.# These are Jinja templates.comment:create:"@some-team:pleasereviewandsubmit."update:"@some-team:mergerequesthasbeenupdated,pleasere-review."# Configures GitLab promotion between branches.# A list of (src, dest) branch pairs with other config.# Most config has the same meaning as in gitlab_merge.gitlab_promote:-src:stagedest:prodtitle:"Promotefromstagetoprod[{{datetime_day}}]"token:$GITLAB_MIRROR_TOKENlabels:-promotedescription:|-Automated promotion of {{ src_mr.web_url }} to prod.Jinja contextSome configuration elements are described above asJinja templates.
The following variables are available for use within the templates:env(dict)All environment variables at the timemirror-toolis invoked.If running in GitLab CI/CD, can be used to access theCI/CD variables.Example:{{ env.CI_JOB_URL }}=>https://gitlab.example.com/someteam/somerepo/-/jobs/6366493datetime_iso8601(str)Current UTC date/time, in ISO8601 format, with seconds precision.Example:2022-05-10T05:28:26Zdatetime_minute(str)Current UTC date/time, with minutes precision.Example:2022-05-10 05:28datetime_day(str)Current UTC date.Example:2022-05-10datetime_week(str)Current UTC year and week of year.Example:2022wk19for week 19 of 2022.updates(list[UpdateInfo])(update only)In most Jinja contexts for theupdateandupdate-localcommands,
this is a list of objects of the following form:UpdateInfo(mirror=Mirror(url="https://github.com/rohanpm/mirror-tool",ref="refs/heads/main",dir="mirror-tool"),# Objects representing commits included in the update, starting# with the most recent.## If there is a large number of commits being handled, some may be# elided from this list.commits=[Commit(revision="472b7797518b963f8ab381c39858c18b2b784c2e",revision_abbrev="472b779",author_name="Rohan McGovern",author_email="rohan@mcgovern.id.au",author_email_local="rohan",author_datetime=datetime.datetime(2022,5,26,0,24,37),committer_name="Rohan McGovern",committer_email="rohan@mcgovern.id.au",committer_email_local="rohan",committer_datetime=datetime.datetime(2022,5,26,0,37,37),subject="Raise test coverage to 100%",body="Do this, that and some other\nthings as well.",url="https://github.com/rohanpm/mirror-tool/commit/472b7797518b963f8ab381c39858c18b2b784c2e",),...,],# Total number of commits in the update (may be more than len(commits)# if some were elided).commit_count=4,# Number of commits omitted from 'commits' object.# For example, if an update pulled 200 commits, only the most recent 20# may appear in 'commits', and this value will be set to 180.commit_elided_count=0,)In the Jinja context forcommitmsg, as only a single update is being processed,updatesis not defined.  Instead, all of the fields shown above underUpdateInfoare directly included onto the context.src_mr(dict)(promote only)A merge request object which is now being promoted; i.e. a merge request
previously created by mirror-tool and submitted to one branch, and now
being promoted by mirror-tool to another branch.The format of this object can be found in theGitLab API docs.Only available for thepromotecommand.LicenseThis program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.