Package: gerrit_mq
A merge commit serializer for special branches.Purposegerrit-mqsolves a particular problem in continuous integeration
whereby two separate changes are syntactically conflict free but semantically in
conflict.Here’s a motivating (though contrived) example. Let’s say that our code
repository has the following:foo.py:def bar(x):
  return x + 10foo_test.py:import unittest

import foo

class TestFoo(unittest.TestCase):
  def test_foo(self):
    self.assertGreater(foo.bar(0), 5)

if __name__ == '__main__':
  unittest.main()Now let’s say that Alice decies to improve this loose test with some more
meaningful bounds and makes the following change tofoo_test.py:class TestFoo(unittest.TestCase):
    def test_foo(self):
      self.assertEqual(foo.bar(0), 10)And let’s say that Bob makes the following change tofoo.py:def bar(x):
  return x + 6If we have designed our continuous integration infrastructure with pre-submit
build-and-test (on, say, jenkins or buildbot), then both of these changes will
pass pre-submit no problem. However, the two changes are logically incompatible.
If both changes are merged the unit test will begin to fail. One way to deal
with this issue is after-the-fact. We can add a post-merge job to our continuous
integration server to give us a heads-up every time something like this happens.
However, as your team scales, this becomes impractical for two reasons:As the submit rate goes up, the frequency of this occurance will go upOnce a breaking change is in, then all pre-submit jobs will fail. Your
CI pre-submit job becomes an ignorable signal (if advisory) or the entire
pipeline is frozen until someone can fix the build (if manditory).gerrit-mqattempts to solve this problem by re-executing the
build-and-tests checks on each merge in serial order. It ensures that no
conflicts like that illustrated above ever get merged. In the above scenario,
whichever merge was queued first will pass, and the second will fail.How it worksThere are two components to the merge queue:The merge daemon: polls gerrit for new merge requests and verifies/merges
themThe web frontend: displays the current queue, past job summary table, and
job output streamsWhengerrit-mqpolls gerrit it looks for new “merge requests”. A merge
request is any change which isCode-Review:+2andMerge-Request:+1.
TheMerge-Requestlabel is not a gerrit built-in and so must be added. For
example you might add the following toall_projects.config:[access "refs/heads/*"]
...
  label-Merge-Queue = -1..+1 group TestGroupBecause the queue is maintained in gerrit, when a merge failsgerrit-mqwill
add aMerge-Queue:-1label. In order to allow re-request of a merge,gerrit-mqresolves theMerge-Queuelabel score as “The highest score
after the latest -1”.Usageusage: gerrit-mq [-h] [-c CONFIG_PATH] [-l {debug,info,warning,error}] CMD ...

Entry point / launcher for gerrit-mq components.

positional arguments:
  CMD
    webfront            Start the merge-queue master service.
    get-next            Retrieve the next merge request.
    get-queue           Retrieve the currently cached queue in json format
    daemon              Execute the daemon process.
    render-templates    Render jinja2 templates into full html files.
    migrate-database    Migrate a database from one schema to another
    sync-account-table  Fetch account table from gerrit and store locally
    gzip-old-logs       Gzip files in an old log directory
    poll-gerrit         Hit gerrit REST and read off the current queue of
                        merge requests. Write that to a json file.

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_PATH, --config-path CONFIG_PATH
                        path to config file
  -l {debug,info,warning,error}, --log-level {debug,info,warning,error}Configurationgerrit-mqtakes a configuration file as input. The configuration file is
python and will beexec(). See the example configuration insamples/mqconfig.pywhich contains comments describing what each option
means.gerrit-mqsupports multiple logical “queues”. Each queue is defined by:which gerrit project the queue applies toa pattern used to match against branch namesa unique name for the queuea dictionary describing the environment of subprocess callsa list of commands to execute to verify the merge request, if any exits with
non-zero exit code then verification failsa flag indicating whether or not to finally merge using the gerrit rest API
(you set this to false if the last command in your list of commands does
the actual merge)This allows you to configure different verification steps for different
projects or different branches. It also allows you to specify a common queue
for a pattern of branches. For instance,release-candidate/\d{4}-\d{2}-\d{2}will match branches likerelease-candidate/2018-01-14release-candidate/2018-02-12All jobs from a single queue are built/verified in the same git working tree.
This means that (unless you otherwise specify) the merge queue will generally
execute an incremental build. You can, of course, remove the build tree as your
first step to get a clean build every time.ExecutionStart the daemon with:gerrit-mq --config config.py daemonStart the webfront withgerrit-mq --config config.py webfrontThe webfront only servesJSON. Use:gerrit-mq render-templates <outdir>to create the html document root for the webfront views.The directorysamples/contains an example nginx configuration andsystemdunit files for the webfront and daemon. These all presume that
the system has a usermergequeue, the config file is at/home/mergequeue/config.pyand the html document root is at/home/mergequeue/pages.Init SystemIf you’d like to rungerrit-mqon startup in ubuntu, there are samplesystemdunit files in thesamples/directory.Test setupThere is a script to create a docker image with gerrit configured for two
users. Just execute:python -Bm gerrit_mq.test.gerrit_docker buildto create the docker image and then:python -Bm gerrit_mq.test.gerrit_docker start --debugto start the container (--debugputs it in the foreground).Once it’s started openhttp://localhost:8081in a browser and use the
“Become” link to become one of the test users. Then add your public key
to that user.Start the webfront and the nginx forward proxy:python -Bm gerrit_mq --config gerrit_mq/test/mqconfig.py webfront
python -Bm gerrit_mq.test --config gerrit_mq/test/mqconfig.py start-nginxAnd check it out athttp://localhost:8080.Now start the daemon with:python -Bm gerrit_mq --config gerrit_mq/test/mqconfig.py daemonAdd your public key to the mergequeue user on gerrit
TODO(josh): plumb –identity through the daemon config and use the testing keyYou can submit multiple jobs for testing with:python -Bm gerrit_mq.test --config gerrit_mq/test/mqconfig.py \
    create-reviews --approve --queue 5You can manually clone the test repo with:git clone ssh://test1@localhost:29418/mq_testGet the commit hook with:curl --insecure -Lo .git/hooks/commit-msg http://localhost:8081/tools/hooks/commit-msg
chmod +x .git/hooks/commit-msgCheckout a feature branch:git checkout -b feature_001Make a change:cat > file_a.txt
Hello world

git add -A
git commit
git push -u origin
git push origin HEAD:refs/for/masterNotes on testingGerrit 2.8.11 only offersdiff-hellman-group1-shaas an exchange method,
which unfortunately OpenSSH (client) disables by default. To run tests againsts
this gerrit version in the docker container you’ll need to add the following to
your~/.ssh/configHost localhost
  KexAlgorithms +diffie-hellman-group1-sha1
  StrictHostKeyChecking no
  UserKnownHostsFile=/dev/nullCopy the commit message hook from the server using:curl -Lo .git/hooks/commit-msg http://review.example.com/tools/hooks/commit-msgThis will append a random changeID to the change message.Put the change out for review with:git push origin HEAD:refs/for/masterCreate test commits for coalesced merge:python -m gerrit_mq.test -c test/config.py create-reviews --approve --queue --repo-path /tmp/mq_test --branch build pass-fail P P P P F P P P P