Repository: buildbot/buildbot

Issue:
GerritChangeSource marks comment-added events as duplicated
Hi,

We recently upgraded our master from 2.0.0 to 2.8.4.  After the upgrade, all the comment-added gerrit events seem to be classified as duplicated.  This message is being printed when a comment is added (we use gerrit comments to signal CI)

    "gerrit: duplicate change event {} by {}"

Looks like the changes to GerritChangeSourceBase.addChange broke this.  In particular, the stampdict passed to findOrCreateId looks to be missing a few fields for the duplication detection to work correctly.  I suspect that change-merged events have a similar problem, but i didnt check for sure.

As a workaround I reverted the entire gerritchangesource.py to 2.0.0 level, which works for us.


Discussion:
@gerard Thanks for the bug report. Could you explain your use case a little bit? How do you configure the GerritChangeSource? I assume you subclass it and add an additional method for the `comment-added` events which creates builds depending on the comment text. Is it something like this in your setup?
There is no subclassing.  I just have "comment-added" in my handled_events array.  My understanding is that this is documented behaviour.

        gerrit_listener = changes.GerritChangeSource(
            gerrit_server,
            gerrit_user,
            handled_events=[
                "patchset-created",
                "ref-updated",
                "change-merged",
                "comment-added"
            ]
        )

Hi yes i'm also getting this issue too. I'm about to be away so I don't have time to give details right now, but in terms of replication it occurs when we had legacy code do an addChange call (haven't refactored it but it compiled so assumed it was okay), we triggered a Gerrit patch, it would succeed to queue up (the build would fail for our own reasons), we would trigger that same patch again for various reasons (no change to patchset commit hash), and it would not queue up with the following error in twistd:

 gerrit: duplicate change event ssh://buildbot@gerritserver.com:12345/project -- master:49cb...a5df7 by ServerGerritChangeSource

This is my buildbot version:
$ buildbot --version
Buildbot version: 2.9.2
Twisted version: 20.3.0

Resolving patch:
diff --git a/master/buildbot/changes/gerritchangesource.py b/master/buildbot/changes/gerritchangesource.py
--- a/master/buildbot/changes/gerritchangesource.py
+++ b/master/buildbot/changes/gerritchangesource.py
@@ -163,7 +163,7 @@ def flatten(properties, base, event):
         return func(properties, event)
 
     @defer.inlineCallbacks
-    def addChange(self, chdict):
+    def addChange(self, event_type, chdict):
         stampdict = {
             "branch": chdict["branch"],
             "revision": chdict["revision"],
@@ -177,7 +177,7 @@ def addChange(self, chdict):
         stampid, found_existing = yield(
              self.master.db.sourcestamps.findOrCreateId(**stampdict))
 
-        if found_existing:
+        if found_existing and event_type in ("patchset-created", "ref-updated"):
             if self.debug or True:
                 eventstr = "{}/{} -- {}:{}".format(
                     self.gitBaseURL, chdict["project"], chdict["branch"],
@@ -236,7 +236,7 @@ def addChangeFromEvent(self, properties, event):
                 patchset=event["patchSet"]["number"]
             )
 
-        yield self.addChange({
+        yield self.addChange(event['type'], {
             'author': _gerrit_user_to_author(event_change["owner"]),
             'project': util.bytes2unicode(event_change["project"]),
             'repository': "{}/{}".format(


Test patch:
diff --git a/master/buildbot/test/unit/changes/test_gerritchangesource.py b/master/buildbot/test/unit/changes/test_gerritchangesource.py
--- a/master/buildbot/test/unit/changes/test_gerritchangesource.py
+++ b/master/buildbot/test/unit/changes/test_gerritchangesource.py
@@ -199,6 +199,39 @@ def test_duplicate_events_ignored(self):
         )))
         self.assertEqual(len(self.master.data.updates.changesAdded), 1)
 
+    @defer.inlineCallbacks
+    def test_duplicate_non_source_events_not_ignored(self):
+        s = self.newChangeSource('somehost', 'someuser',
+                                 handled_events=['patchset-created', 'ref-updated',
+                                                 'change-merged', 'comment-added'])
+        yield s.lineReceived(json.dumps({
+            'type': "comment-added",
+            'change': {
+                'branch': "br",
+                'project': "pr",
+                'number': "4321",
+                'owner': {'name': "Dustin", 'email': "dustin@mozilla.com"},
+                'url': "http://buildbot.net",
+                'subject': "fix 1234"
+            },
+            'patchSet': {'revision': "abcdef", 'number': "12"}
+        }))
+        self.assertEqual(len(self.master.data.updates.changesAdded), 1)
+
+        yield s.lineReceived(json.dumps({
+            'type': "comment-added",
+            'change': {
+                'branch': "br",
+                'project': "pr",
+                'number': "4321",
+                'owner': {'name': "Dustin", 'email': "dustin@mozilla.com"},
+                'url': "http://buildbot.net",
+                'subject': "fix 1234"
+            },
+            'patchSet': {'revision': "abcdef", 'number': "12"}
+        }))
+        self.assertEqual(len(self.master.data.updates.changesAdded), 2)
+
     @defer.inlineCallbacks
     def test_malformed_events_ignored(self):
         s = self.newChangeSource('somehost', 'someuser')