Repository: buildbot/buildbot

Issue:
Gitpoller: command 'rev-parse' failed with 'unknown revision or path not in the working tree'
Buildbot version: 3.4.0

I am getting this error when I was setting up the change_source to a private git repo:
```
buildbot.changes.gitpoller.GitError: command ['rev-parse', 'refs/buildbot/%2Fvar%2Fbuildbot%2Fabc/my_branch'] in /var/buildbot/master/gitpoller-work on repourl /var/buildbot/abc failed with exit code 128: fatal: ambiguous argument 'refs/buildbot/%2Fvar%2Fbuildbot%2Fabc/my_branch': unknown revision or path not in the working tree.
        Use '--' to separate paths from revisions, like this:
        'git <command> [<revision>...] -- [<file>...]
```

What is puzzling is that when I `cd` into the `/var/buildbot/master/gitpoller-work` directory and run the exact same command `git rev-parse refs/buildbot/%2Fvar%2Fbuildbot%2Fabc/my_branch` every ~3 seconds. Sometimes it prints the SHA on the stdout, sometimes return `unknown revision...`.

I am wondering if there is a race condition between `git fetch` command and `git rev-parse` command? How do I get the Git poller setup properly?

Here's the code to setup the Gitpoller on my `master.cfg`:
```python
c["change_source"].append(changes.GitPoller(
    repourl='/var/buildbot/abc',
    branches=['my_branch',],
    pollInterval = 10,
    pollAtLaunch = True,
    buildPushesWithNoCommits=True,
))
```


Discussion:
Something similar is affecting me, using buildbot 3.6.1:
~~~
c['change_source'].append(
    changes.GitPoller(
        repourl='git://10.0.0.4/linuxcnc-upstream.git',
        branches=True,
        pollInterval=10
    )
)
~~~

I get this in the log every time the gitpoller runs:
~~~
2022-12-16 08:43:25-0700 [-] gitpoller: processing changes from "git://10.0.0.4/linuxcnc-upstream.git"
2022-12-16 08:43:25-0700 [-] trying to poll branch refs/heads/2.8 of git://10.0.0.4/linuxcnc-upstream.git
        Traceback (most recent call last):
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 857, in _runCallbacks
            current.result = callback(  # type: ignore[misc]
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1750, in gotResult
            current_context.run(_inlineCallbacks, r, gen, status)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
            result = current_context.run(
          File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
        --- <exception caught here> ---
          File "/usr/lib/python3/dist-packages/buildbot/changes/gitpoller.py", line 265, in poll
            rev = yield self._dovccmd(
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
            result = current_context.run(
          File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/usr/lib/python3/dist-packages/buildbot/changes/gitpoller.py", line 449, in _dovccmd
            stdout = yield self._dovccmdImpl(command, args, path, None)
          File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
            result = current_context.run(gen.send, result)
          File "/usr/lib/python3/dist-packages/buildbot/changes/gitpoller.py", line 479, in _dovccmdImpl
            raise GitError(f'command {full_args} in {path} on repourl {self.repourl} failed '
        buildbot.changes.gitpoller.GitError: command ['rev-parse', 'refs/buildbot/git%3A%2F%2F10.0.0.4%2Flinuxcnc-upstream.git/2.8'] in /var/lib/buildbot/masters/linuxcnc/gitpoller-work on repourl git://10.0.0.4/linuxcnc-upstream.git failed with exit code 128: fatal: ambiguous argument 'refs/buildbot/git%3A%2F%2F10.0.0.4%2Flinuxcnc-upstream.git/2.8': unknown revision or path not in the working tree.
        Use '--' to separate paths from revisions, like this:
        'git <command> [<revision>...] -- [<file>...]'
~~~

I get one copy of that traceback for each branch in the repo.

Cloning the repourl by hand works fine.  

Like @antonysigma reported, running `git rev-parse` on the ref by hand in the gitpoller-work dir works intermittently.

I tried stopping my buildbot, deleting the gitpoller-work dir, and restarting the buildbot.  The first poll works fine, the second poll and all after it fail in this way.
I've just updated from an old version of Buildbot and hit this issue.  Is there any planned fix or should I just roll back?
> I am wondering if there is a race condition between git fetch command and git rev-parse command? How do I get the Git poller setup properly?

Sharing a mitigation to resolve the issue... I found that if I increased the `pollInterval` from 10 seconds to 10 minutes or more, the issue resolves on its own.

Perhaps the `GitPoller`, by design, assumes the HTTP/SSH response latency from the external git server is less than the value set in `pollInterval` argument.
So after rolling back to my Buildbot revision I'm still seeing this error but only in the test instance I'm running.  Our production instance doesn't seem to have any of these rev-parse errors.  No idea what the difference is yet.

While investigating I managed to see a rev-parse error within the gitpoller-work directory which seemed to be caused by all refs being deleted and re-populated.  Does the GitPoller start from a clean work folder each time?  Or should it be bombing out early if the previous invocation is still running?
So I've managed to replicate what's going on with a pure Git command.  After lots of `log.msg()` printing, I noticed that the refs in `refs/buildbot/<escaped_url>/*` were disappearing every other time the `git fetch` command was called.  I'm running Git 2.18.0.

So as an example, running the following command consecutively will fail every other try.

```
$ git fetch https://<bitbucket_url>.git +<branch>:refs/buildbot/<escaped_rul>/<branch>; git rev-parse refs/buildbot/<escaped_url>/<branch>
```

This doesn't happen with Git 1.8.3.1 which is the only other version I have on my system.  When it fails I get the following:

```
 - [deleted]           (none)     -> refs/buildbot/<escaped_url>/<branch>
```
Ergh, this is caused because I have a global config of `fetch.prune=true`.  Setting it to false and the fetch behaves correctly.  I wonder if the GitPoller should be updates to check this.

Resolving patch:
diff --git a/master/buildbot/changes/gitpoller.py b/master/buildbot/changes/gitpoller.py
--- a/master/buildbot/changes/gitpoller.py
+++ b/master/buildbot/changes/gitpoller.py
@@ -32,6 +32,7 @@
 from buildbot.util.git import GitServiceAuth
 from buildbot.util.git import check_ssh_config
 from buildbot.util.state import StateMixin
+from buildbot.util.twisted import async_to_deferred
 
 if TYPE_CHECKING:
     from typing import Callable
@@ -250,25 +251,28 @@ def describe(self):
 
         return str
 
-    def _getBranches(self):
-        d = self._dovccmd('ls-remote', ['--refs', self.repourl])
+    @async_to_deferred
+    async def _get_refs(self, refs: list[str] | None = None) -> list[str]:
+        rows: str = await self._dovccmd(
+            'ls-remote', ['--refs', self.repourl] + (refs if refs is not None else [])
+        )
 
-        @d.addCallback
-        def parseRemote(rows):
-            branches = []
-            for row in rows.splitlines():
-                if '\t' not in row:
-                    # Not a useful line
-                    continue
-                _, ref = row.split("\t")
-                branches.append(ref)
-            return branches
+        branches: list[str] = []
+        for row in rows.splitlines():
+            if '\t' not in row:
+                # Not a useful line
+                continue
+            _, ref = row.split("\t")
+            branches.append(ref)
 
-        return d
+        return branches
 
-    def _headsFilter(self, branch):
-        """Filter out remote references that don't begin with 'refs/heads'."""
-        return branch.startswith("refs/heads/")
+    @staticmethod
+    def _trim_prefix(value: str, prefix: str) -> str:
+        """Remove prefix from value."""
+        if value.startswith(prefix):
+            return value[len(prefix) :]
+        return value
 
     def _removeHeads(self, branch):
         """Remove 'refs/heads/' prefix from remote references."""
@@ -278,7 +282,7 @@ def _removeHeads(self, branch):
 
     def _trackerBranch(self, branch):
         url = urlquote(self.repourl, '').replace('~', '%7E')
-        return f"refs/buildbot/{url}/{self._removeHeads(branch)}"
+        return f"refs/buildbot/{url}/{self._trim_prefix(branch, 'refs/')}"
 
     def poll_should_exit(self):
         # A single gitpoller loop may take a while on a loaded master, which would block
@@ -295,34 +299,40 @@ def poll(self):
             log.msg(e.args[0])
             return
 
-        branches = self.branches if self.branches else []
-        remote_refs = yield self._getBranches()
+        refs: list[str] = []
+        trim_ref_head = False
+        if callable(self.branches):
+            # Get all refs and let callback filter them
+            remote_refs = yield self._get_refs()
+            refs = [b for b in remote_refs if self.branches(b)]
+        elif self.branches is True:
+            # Get all branch refs
+            refs = yield self._get_refs(["refs/heads/*"])
+        elif self.branches:
+            refs = yield self._get_refs([f"refs/heads/{b}" for b in self.branches])
+            trim_ref_head = True
+
+        # Nothing to fetch and process.
+        if not refs:
+            return
 
         if self.poll_should_exit():
             return
 
-        if branches is True or callable(branches):
-            if callable(self.branches):
-                branches = [b for b in remote_refs if self.branches(b)]
-            else:
-                branches = [b for b in remote_refs if self._headsFilter(b)]
-        elif branches and remote_refs:
-            remote_branches = [self._removeHeads(b) for b in remote_refs]
-            branches = sorted(list(set(branches) & set(remote_branches)))
-
-        refspecs = [
-            f'+{self._removeHeads(branch)}:{self._trackerBranch(branch)}' for branch in branches
-        ]
+        refspecs = [f'+{ref}:{self._trackerBranch(ref)}' for ref in refs]
 
         try:
-            yield self._dovccmd('fetch', ['--progress', self.repourl] + refspecs, path=self.workdir)
+            yield self._dovccmd(
+                'fetch', ['--progress', self.repourl] + refspecs + ['--'], path=self.workdir
+            )
         except GitError as e:
             log.msg(e.args[0])
             return
 
         revs = {}
         log.msg(f'gitpoller: processing changes from "{self.repourl}"')
-        for branch in branches:
+        for ref in refs:
+            branch = ref if not trim_ref_head else self._trim_prefix(ref, 'refs/heads/')
             try:
                 if self.poll_should_exit():  # pragma: no cover
                     # Note that we still want to update the last known revisions for the branches
@@ -330,10 +340,10 @@ def poll(self):
                     break
 
                 rev = yield self._dovccmd(
-                    'rev-parse', [self._trackerBranch(branch)], path=self.workdir
+                    'rev-parse', [self._trackerBranch(ref), '--'], path=self.workdir
                 )
-                revs[branch] = bytes2unicode(rev, self.encoding)
-                yield self._process_changes(revs[branch], branch)
+                revs[branch] = rev
+                yield self._process_changes(rev, branch)
             except Exception:
                 log.err(_why=f"trying to poll branch {branch} of {self.repourl}")
 


Test patch:
diff --git a/master/buildbot/test/unit/changes/test_gitpoller.py b/master/buildbot/test/unit/changes/test_gitpoller.py
--- a/master/buildbot/test/unit/changes/test_gitpoller.py
+++ b/master/buildbot/test/unit/changes/test_gitpoller.py
@@ -224,20 +224,26 @@ def test_poll_initial(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\n'),
@@ -261,9 +267,13 @@ def test_poll_initial_poller_not_running(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
         )
 
         self.poller.doPoll.running = False
@@ -284,46 +294,44 @@ def test_poll_failInit(self):
         d.addCallback(lambda _: self.assert_all_commands_ran())
         return d
 
-    def test_poll_failFetch(self):
+    @defer.inlineCallbacks
+    def test_poll_branch_do_not_exist(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]),
-            ExpectMasterShell([
-                'git',
-                'fetch',
-                '--progress',
-                self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
-            ])
-            .workdir(self.POLLER_WORKDIR)
-            .exit(1),
+            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL, 'refs/heads/master']),
         )
 
         self.poller.doPoll.running = True
-        d = self.assertFailure(self.poller.poll(), EnvironmentError)
-        d.addCallback(lambda _: self.assert_all_commands_ran())
-        return d
+        yield self.poller.poll()
+
+        self.assert_all_commands_ran()
 
     @defer.inlineCallbacks
     def test_poll_failRevParse(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .exit(1),
@@ -341,20 +349,26 @@ def test_poll_failLog(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -419,22 +433,28 @@ def test_poll_nothingNew(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'no interesting output'),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -468,31 +488,42 @@ def test_poll_multipleBranches_initial(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2\t'
-                b'refs/heads/release\n'
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+                'refs/heads/release',
+                'refs/heads/not_on_remote',
+            ]).stdout(
                 b'4423cdbcbb89c14e50dd5f4152415afd686c5241\t'
                 b'refs/heads/master\n'
+                b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2\t'
+                b'refs/heads/release\n'
             ),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
-                '+release:refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '+refs/heads/release:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2'),
@@ -517,24 +548,33 @@ def test_poll_multipleBranches(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2\t'
-                b'refs/heads/release\n'
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+                'refs/heads/release',
+            ]).stdout(
                 b'4423cdbcbb89c14e50dd5f4152415afd686c5241\t'
                 b'refs/heads/master\n'
+                b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2\t'
+                b'refs/heads/release\n'
             ),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
-                '+release:refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '+refs/heads/release:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -558,7 +598,8 @@ def test_poll_multipleBranches(self):
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2'),
@@ -680,20 +721,26 @@ def test_poll_multipleBranches_buildPushesWithNoCommits_default(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/release',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+release:refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                '+refs/heads/release:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -729,20 +776,26 @@ def test_poll_multipleBranches_buildPushesWithNoCommits_true(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/release',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+release:refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                '+refs/heads/release:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -827,20 +880,26 @@ def test_poll_multipleBranches_buildPushesWithNoCommits_true_fast_forward(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/release',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+release:refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                '+refs/heads/release:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -927,20 +986,26 @@ def test_poll_multipleBranches_buildPushesWithNoCommits_true_not_tip(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/release',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/release\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+release:refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                '+refs/heads/release:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -1025,7 +1090,7 @@ def test_poll_allBranches_single(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
+            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL, 'refs/heads/*']).stdout(
                 b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
             ),
             ExpectMasterShell([
@@ -1033,12 +1098,14 @@ def test_poll_allBranches_single(self):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -1131,22 +1198,28 @@ def test_poll_noChanges(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'no interesting output'),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -1177,7 +1250,7 @@ def test_poll_allBranches_multiple(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
+            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL, 'refs/heads/*']).stdout(
                 b'\n'.join([
                     b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master',
                     b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2\trefs/heads/release',
@@ -1188,13 +1261,15 @@ def test_poll_allBranches_multiple(self):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
-                '+release:refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '+refs/heads/release:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -1218,7 +1293,8 @@ def test_poll_allBranches_multiple(self):
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/release',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/release',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2'),
@@ -1322,12 +1398,14 @@ def test_poll_callableFilteredBranches(self):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -1432,12 +1510,14 @@ def test_poll_branchFilter(self):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+refs/pull/410/head:refs/buildbot/' + self.REPOURL_QUOTED + '/refs/pull/410/head',
+                '+refs/pull/410/head:refs/buildbot/' + self.REPOURL_QUOTED + '/pull/410/head',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/refs/pull/410/head',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/pull/410/head',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'9118f4ab71963d23d02d4bdc54876ac8bf05acf2'),
@@ -1523,22 +1603,28 @@ def test_poll_old(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                'refs/heads/master',
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'no interesting output'),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -1646,7 +1732,7 @@ def test_poll_callableCategory(self):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 1.7.5\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
+            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL, 'refs/heads/*']).stdout(
                 b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
             ),
             ExpectMasterShell([
@@ -1654,12 +1740,14 @@ def test_poll_callableCategory(self):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\n'),
@@ -1810,7 +1898,8 @@ def test_poll_initial_2_10(self, write_local_file_mock, temp_dir_mock):
                 'ls-remote',
                 '--refs',
                 self.REPOURL,
-            ]),
+                "refs/heads/master",
+            ]).stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 '-c',
@@ -1818,12 +1907,14 @@ def test_poll_initial_2_10(self, write_local_file_mock, temp_dir_mock):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\n'),
@@ -1858,22 +1949,28 @@ def test_poll_initial_2_3(self, write_local_file_mock, temp_dir_mock):
         self.expect_commands(
             ExpectMasterShell(['git', '--version']).stdout(b'git version 2.3.0\n'),
             ExpectMasterShell(['git', 'init', '--bare', self.POLLER_WORKDIR]),
-            ExpectMasterShell(['git', 'ls-remote', '--refs', self.REPOURL]).stdout(
-                b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'
-            ),
+            ExpectMasterShell([
+                'git',
+                'ls-remote',
+                '--refs',
+                self.REPOURL,
+                "refs/heads/master",
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .env({'GIT_SSH_COMMAND': f'ssh -o "BatchMode=yes" -i "{key_path}"'}),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\n'),
@@ -1916,7 +2013,8 @@ def test_poll_failFetch_git_2_10(self, write_local_file_mock, temp_dir_mock):
                 'ls-remote',
                 '--refs',
                 self.REPOURL,
-            ]),
+                "refs/heads/master",
+            ]).stdout(b'4423cdbcbb89c14e50dd5f4152415afd686c5241\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 '-c',
@@ -1924,7 +2022,8 @@ def test_poll_failFetch_git_2_10(self, write_local_file_mock, temp_dir_mock):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .exit(1),
@@ -1967,7 +2066,8 @@ def test_poll_initial_2_10(self, write_local_file_mock, temp_dir_mock):
                 'ls-remote',
                 '--refs',
                 self.REPOURL,
-            ]),
+                "refs/heads/master",
+            ]).stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 '-c',
@@ -1976,12 +2076,14 @@ def test_poll_initial_2_10(self, write_local_file_mock, temp_dir_mock):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\n'),
@@ -2042,7 +2144,8 @@ def test_poll_initial_2_10(self, write_local_file_mock, temp_dir_mock):
                 'ls-remote',
                 '--refs',
                 self.REPOURL,
-            ]),
+                'refs/heads/master',
+            ]).stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\trefs/heads/master\n'),
             ExpectMasterShell([
                 'git',
                 '-c',
@@ -2051,12 +2154,14 @@ def test_poll_initial_2_10(self, write_local_file_mock, temp_dir_mock):
                 'fetch',
                 '--progress',
                 self.REPOURL,
-                '+master:refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                '+refs/heads/master:refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ]).workdir(self.POLLER_WORKDIR),
             ExpectMasterShell([
                 'git',
                 'rev-parse',
-                'refs/buildbot/' + self.REPOURL_QUOTED + '/master',
+                'refs/buildbot/' + self.REPOURL_QUOTED + '/heads/master',
+                '--',
             ])
             .workdir(self.POLLER_WORKDIR)
             .stdout(b'bf0b01df6d00ae8d1ffa0b2e2acbe642a6cd35d5\n'),