Repository: fedora-infra/bodhi

Issue:
Update was not auto-pushed to stable after 7 days
[This update](https://bodhi.fedoraproject.org/updates/FEDORA-2020-189a42d65a) should have been pushed to stable after 7 days, but wasn't. I'm not sure if this was a one time issue.

It received +2 karma, the threshold was +3. So it received feedback, but not enough to reach the threshold. Maybe it's related to that?


Discussion:
[Here](https://bodhi.fedoraproject.org/updates/FEDORA-2020-d5cd2e2fa6)'s a similar update that received 2 karma, sent me a notification that it can be submitted for stable (even though limit is +3), and then never submitted it automatically when 7 days came around.
[Here](https://bodhi.fedoraproject.org/updates/FEDORA-2021-0691392fdd)'s another update that says it can be pushed after 2 (not 3) karma, then never auto-pushed after 3 days.
@nirik said in the infra ticket that the logs said:
> FEDORA-2021-0691392fdd has already the comment that it can be pushed to stable - bailing

And yes, the [second check](https://github.com/fedora-infra/bodhi/blob/348c3072a5c83106fad573bc35d168b27d2f7b34/bodhi/server/tasks/approve_testing.py#L69) of `approve_testing` does look for an existing comment from Bodhi. Automatic pushing to stable occurs [later in this function](https://github.com/fedora-infra/bodhi/blob/348c3072a5c83106fad573bc35d168b27d2f7b34/bodhi/server/tasks/approve_testing.py#L96). This means that Bodhi will never automatically push if it's left a comment for the packager due to some other condition than the automatic one.

Resolving patch:
diff --git a/bodhi/server/models.py b/bodhi/server/models.py
--- a/bodhi/server/models.py
+++ b/bodhi/server/models.py
@@ -3820,8 +3820,7 @@ def has_stable_comment(self):
         """
         for comment in self.comments_since_karma_reset:
             if comment.user.name == 'bodhi' and \
-               comment.text.startswith('This update ') and \
-               'can be pushed to stable now if the maintainer wishes' in comment.text:
+               comment.text == config.get('testing_approval_msg'):
                 return True
         return False
 


Test patch:
diff --git a/bodhi/server/tasks/approve_testing.py b/bodhi/server/tasks/approve_testing.py
--- a/bodhi/server/tasks/approve_testing.py
+++ b/bodhi/server/tasks/approve_testing.py
@@ -65,12 +65,6 @@ def approve_update(update: Update, db: Session):
         # skip it
         log.info(f"{update.alias} doesn't have mandatory days in testing - bailing")
         return
-    # If this update was already commented, skip it
-    if update.has_stable_comment:
-        log.info(
-            f"{update.alias} has already the comment that it can be pushed to stable - "
-            "bailing")
-        return
     # If updates have reached the testing threshold, say something! Keep in mind
     # that we don't care about karma here, because autokarma updates get their request set
     # to stable by the Update.comment() workflow when they hit the required threshold. Thus,
@@ -83,6 +77,11 @@ def approve_update(update: Update, db: Session):
     # If the update is going to be pushed automatically to stable, do not
     # double comment that the maintainer can push it manually (#3846)
     if not update.autotime or update.days_in_testing < update.stable_days:
+        # If this update was already commented, skip it
+        if update.has_stable_comment:
+            log.info(f"{update.alias} has already the comment that it can be pushed to stable - "
+                     "bailing")
+            return
         # Only send email notification about the update reaching
         # testing approval on releases composed by bodhi
         update.comment(
@@ -91,9 +90,12 @@ def approve_update(update: Update, db: Session):
             author='bodhi',
             email_notification=update.release.composed_by_bodhi
         )
-    notifications.publish(update_schemas.UpdateRequirementsMetStableV1.from_dict(
-        dict(update=update)))
+        notifications.publish(update_schemas.UpdateRequirementsMetStableV1.from_dict(
+            dict(update=update)))
     if update.autotime and update.days_in_testing >= update.stable_days:
+        if not update.has_stable_comment:
+            notifications.publish(update_schemas.UpdateRequirementsMetStableV1.from_dict(
+                dict(update=update)))
         log.info(f"Automatically marking {update.alias} as stable")
         # For now only rawhide update can be created using side tag
         # Do not add the release.pending_stable_tag if the update
diff --git a/bodhi/tests/server/tasks/test_approve_testing.py b/bodhi/tests/server/tasks/test_approve_testing.py
--- a/bodhi/tests/server/tasks/test_approve_testing.py
+++ b/bodhi/tests/server/tasks/test_approve_testing.py
@@ -502,7 +502,8 @@ def test_autotime_update_meeting_test_requirements_gets_pushed(self):
         self.db.info['messages'] = []
         self.db.commit()
 
-        with fml_testing.mock_sends(api.Message, api.Message):
+        with fml_testing.mock_sends(update_schemas.UpdateRequirementsMetStableV1,
+                                    update_schemas.UpdateRequestStableV1):
             approve_testing_main()
 
         assert update.request == models.UpdateRequest.stable
@@ -895,3 +896,83 @@ def test_update_conflicting_build_not_pushed(self, build_creation_time,
         assert cmnts[0].text == "This update cannot be pushed to stable. "\
             "These builds bodhi-2.0-1.fc17 have a more recent build in koji's "\
             f"{update.release.stable_tag} tag."
+
+    def test_autotime_update_gets_pushed_dont_send_duplicated_notification(self):
+        """
+        Ensure not emitting UpdateRequirementsMetStable notification when an autotime update
+        gets pushed to stable if that was emitted before.
+
+        When an update reaches its mandatory_days_in_testing threshold, the
+        UpdateRequirementsMetStableV1 notification is sent along with a comment that informs
+        the maintainer they can manually push the update to stable. The update IS NOT
+        automatically pushed now.
+        Then, when the update reaches its stable_days threshold, it is automatically pushed,
+        but we don't want to emit UpdateRequirementsMetStable a second time.
+        """
+        update = self.db.query(models.Update).all()[0]
+        update.autokarma = False
+        update.autotime = True
+        update.request = None
+        update.stable_karma = 10
+        update.stable_days = 10
+        update.date_testing = datetime.utcnow() - timedelta(days=7)
+        update.status = models.UpdateStatus.testing
+        # Clear pending messages
+        self.db.info['messages'] = []
+        self.db.commit()
+
+        assert not update.has_stable_comment
+
+        with fml_testing.mock_sends(update_schemas.UpdateRequirementsMetStableV1):
+            approve_testing_main()
+
+        assert update.has_stable_comment
+        assert update.request == None
+        assert update.status == models.UpdateStatus.testing
+
+        update.date_testing = datetime.utcnow() - timedelta(days=10)
+        # Clear pending messages
+        self.db.info['messages'] = []
+        self.db.commit()
+
+        with fml_testing.mock_sends(update_schemas.UpdateRequestStableV1):
+            approve_testing_main()
+
+        assert update.request == models.UpdateRequest.stable
+
+    def test_autotime_update_with_stable_comment_set_stable_on_branched(self):
+        """
+        Ensure update is pushed to stable on releases not composed by Bodhi if
+        the update already has a stable comment.
+        """
+        update = self.db.query(models.Update).all()[0]
+        update.autokarma = False
+        update.autotime = True
+        update.request = None
+        update.stable_karma = 10
+        update.stable_days = 10
+        update.release.composed_by_bodhi = False
+        update.date_testing = datetime.utcnow() - timedelta(days=7)
+        update.status = models.UpdateStatus.testing
+        # Clear pending messages
+        self.db.info['messages'] = []
+        self.db.commit()
+
+        assert not update.has_stable_comment
+
+        with fml_testing.mock_sends(update_schemas.UpdateRequirementsMetStableV1):
+            approve_testing_main()
+
+        assert update.has_stable_comment
+        assert update.request == None
+        assert update.status == models.UpdateStatus.testing
+
+        update.date_testing = datetime.utcnow() - timedelta(days=10)
+        # Clear pending messages
+        self.db.info['messages'] = []
+        self.db.commit()
+
+        # No further notifications emitted
+        approve_testing_main()
+
+        assert update.status == models.UpdateStatus.stable
diff --git a/docs/user/testing.rst b/docs/user/testing.rst
--- a/docs/user/testing.rst
+++ b/docs/user/testing.rst
@@ -35,4 +35,30 @@ updates. If this is enabled, the update's web page will display the current stat
 gating decision.
 
 
+Stable push
+===========
+
+An update can be pushed to stable repository either manually or automatically, through Bodhi's
+`autotime` or `autokarma` features.
+
+For becoming pushable to stable an update must fullfill some criteria set by the associated
+release. Tipically, each release has a `mandatory_days_in_testing` and a `critpath_min_karma`
+values that define the threshold after that the associated updates can be pushed to stable.
+The `mandatory_days_in_testing` define the minimum amount of days each update must spend in
+testing repository. The `critpath_min_karma` is the minimum karma that each update must reach.
+An update can be manually pushed to stable by its submitter whatever threshold is reached first.
+For example, if a release has a `critpath_min_karma` of +2, an update which reaches a +2 karma
+**can be pushed to stable even if it hasn't reached the `mandatory_days_in_testing`**.
+
+When submitting an update, the user can enable the `autotime` and the `autokarma` features and
+set the related values `stable_days` and `stable_karma` for that specific update.
+The `stable_days` value define the minimum amount of days each update must spend in
+testing repository before the autopush is performed. It must be equal or greater than the release
+`mandatory_days_in_testing`. The `stable_karma` is the minimum karma that each update must reach
+before the autopush is performed. It must be equal or greater than the release `critpath_min_karma`.
+For example, if a release has a `mandatory_days_in_testing` of 7, an update which has `autotime`
+enabled and `stable_days` set to 10 will be pushable after 7 days **by manual action**, but the
+autopush will be performed **after 3 more days**.
+
+
 .. _Greenwave: https://pagure.io/greenwave