Repository: fedora-infra/bodhi

Issue:
don't close comments when update is pushed stable
I have noticed that people can no longer comment on updates once they went stable [1]. Please revert this change. Future people comments are very important for QA. I see quite frequently that people post comments like "but this has broken XYZ!" on stable updates. Some people even share a bugzilla link this way, so that it's easy for people affected to participate in the bug discussion. Other people don't report this to bugzilla, but posting this feedback to Bodhi alerts others (like me) to go and investigate. We can't follow the whole bugzilla, but when there are several angry comments in Bodhi on an already stable bug, that's easy to notice and act on that. Please re-allow comments on stable updates. Thank you.

[1] Example: https://bodhi.fedoraproject.org/updates/FEDORA-2019-103a594d07


Discussion:
I have no preference either way here, but here is the initial RFE with the request:

https://github.com/fedora-infra/bodhi/issues/2050


I can understand @jasontibbitts 's motivation. However, I think the disadvantages here outweigh the advantages. Pushing something broken stable is not uncommon. I then debug the culprit package, and then look into Bodhi at the relevant update page. Often, there are already people complaining. Sometimes there is already a bug link. If there is not, I file the bug, and then put it as a comment into Bodhi, so that anyone affected, who did the same steps as I did (found out the culprit package, looked into Bodhi) is informed and can participate in the bug report. I think this also informs the package maintainer a bit better, because it can highlight a very recent regression compared to the usual avalanche of bugzilla bug reports (which often are not recent regressions). This is all lost when comments are closed.

I'd rather have Bodhi to display a warning of sorts (as proposed in #2050) that says e.g. "This update has already been pushed stable and the process can't be reversed. If you have found a problem related to this very update, please report it into Bugzilla(link) and then optionally share the link here."
+1, it is important to allow comments "on both ends": before the update has gone to testing and after the update has gone to stable.
And now I have a broken Firefox that was pushed stable in mere 12 hours and I can't even comment on the update:
https://bodhi.fedoraproject.org/updates/FEDORA-2019-492e5a2d98

**Please don't close comment section on recent updates. This is critical!**

If the comment section needs to be closed, do it at least after a month. Thank you.
@jasontibbitts do you want to chime in since you requested that behavior in #2050?
Just to add the reference to the discussion I started on fedora-devel. 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/75OK5QJ7EER3PQTCMLTVOVGGP3COB6N6/
Just for reference, I also filed a bug, when I was hit by the issue and I think it would have been nice if one could have linked this back to bodhi. So that anyone could have found the relevant information.
https://bugzilla.redhat.com/show_bug.cgi?id=1773172
How about allowing the comments, but if the update is stable they cannot +1 or -1 and the UI has some note that those comments were made after the update was pushed to stable and are only informational?
How about actually allowing unpushing stable updates (either entirely or back to testing) and letting users use `dnf distro-sync` to get the previous stable update?
As a packager I would like to manage bugs in the appropriate place (Bugzilla) rather than by Bodhi comments... so -1 for reopening comments on stable bugs.

What about a button which permits users to mark the update as introducing a regression with a link to a BZ ticket?
Something like this mokup:
![Screenshot_20191207_174945](https://user-images.githubusercontent.com/17218209/70378148-753e8900-191d-11ea-8f29-32399aca3665.png)
![Screenshot_20191207_181232](https://user-images.githubusercontent.com/17218209/70378151-7a033d00-191d-11ea-8b58-9f88bad13bd8.png)

By adding a new `regression` state (or whatever you would call that) we can also have a nice overview in Bodhi of how many and what updates are broken for each release.
> What about a button which permits users to mark the update as introducing a regression with a link to a BZ ticket?

... or just allow comments? You can't assume that reporters always are confident that something is a bug. The proposed "regression" field solves one very specific case, but not many others: that another update is a follow-up, or that you shouldn't install this update if you use some package, or just "thank you" or anything else that doesn't fit that narrow scope.
> How about actually allowing unpushing stable updates (either entirely or back to testing) and letting users use `dnf distro-sync` to get the previous stable update?

This is impractical for many reasons, including, but not limited to: 
* few people likely use distro-sync
* it would remove urgency to push a fixed package for some maintainers (oh, it was only stable for a day or two, it's fine)
* it could have legal implications, or at least require adjustments since we shipped it, but when it disappears by being untagged we no longer ship the srpm for it. 
* an unknown number of other packages could have built against it and will break on the older version since once things are in stable they are added to the buildroot. 
* It would cause confusion in users (I have the latest version, I ran dnf update!
and I am sure other things... 

People who don't use `distro-sync` won't notice any difference compared to now. But if people run into known regressions that have already lead to the update getting unpushed, they can explicitly `distro-sync` them away. So I don't see how it would make anything worse compared to the status quo.

As for the legal implications, the SRPMs also "disappear" when the update gets replaced by a newer one, so I don't see how the downgrading case would be any different.
> ... or just allow comments? You can't assume that reporters always are confident that something is a bug. The proposed "regression" field solves one very specific case, but not many others: that another update is a follow-up, or that you shouldn't install this update if you use some package, or just "thank you" or anything else that doesn't fit that narrow scope.

I used the word "regression", but I meant to simply give users a way to flag the update as "problematic".
If I understand well, you want to post a comment to warn the maintainer about a problem with the update and add a link to a BZ ticket.
A flag would be more effective and it can be seen at a glance. Also this would force a lazy user to open that BZ ticket and not rely on having the maintainer monitor Bodhi comments by email (which I do not, for example...). Following users can then find the link to the BZ ticket and add comments there.



> an unknown number of other packages could have built against it and will break on the older version since once things are in stable they are added to the buildroot.

This is indeed the reason we should avoid unpushing updates once they're in stable.
> I used the word "regression", but I meant to simply give users a way to flag the update as "problematic".

... but then sometimes the maintainer will want to add a comment saying "no, this is not problematic, the reporter was mistaken". Things that users and maintainers may say in comments are more than just a single bit.
@AdamWill has also provided good justification for allowing comments post-push in the devel thread:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/75OK5QJ7EER3PQTCMLTVOVGGP3COB6N6/

I believe it's safe to say that QA department asks for reverting the change and allowing comments.

Some package maintainers would like to prevent users from reporting issues into Bodhi and direct them to Bugzilla, which I completely understand. But the current solution breaks too many other workflows and is likely to achieve very little positive. The difference will not be a bug report in Bodhi vs a bug report in Bugzilla, but likely a bug report in Bodhi or no bug report at all. It would be much better to educate people with a good UI, that explains them that all bug reports need to go to Bugzilla or they will likely get ignored, rather than shut down a communication channel completely (for all use cases).

The mockup from @mattiaverga is also quite restrictive, because it doesn't allow people to discuss an issue that is not clear cut. For example, "Ever since I updated this, I have a feeling that the UI is slower. Does anybody also see this?" is a prime example for a good Bodhi discussion. If you file this into Bugzilla, it's 1on1 conversation with the maintainer. In Bodhi, you have several interested users already in the discussion. Of course even better might be to use a mailing list, but the Bodhi > Bugzilla comparison is clear here (and once the issue is clarified, only then a new bug report in Bugzilla gets filed).
And here we go again! Firefox is now completely broken (not rendering ANY pages) on many machines. In F30 update, I could point people to my bug report, because fortunately it hasn't been pushed yet:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-2713adc57f
But for F31, I can't tell people to help debug the issue in the bugzilla, because the comments are closed:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-732a5913b6
And so we lose important feedback, we lose understanding of the gravity of the problem, we lose understanding of how many people with which hardware are affected.

**Can we please revert the change ASAP and THEN discuss what the perfect solution is?**

This has been a nuclear bomb for QA processes. If you (Bodhi maintainers) prefer not to make this decision, I'm happy to file a FESCo ticket and let them decide, just let me know. But I need something to happen now. Thanks!

Resolving patch:
diff --git a/bodhi/server/services/comments.py b/bodhi/server/services/comments.py
--- a/bodhi/server/services/comments.py
+++ b/bodhi/server/services/comments.py
@@ -33,7 +33,6 @@
     validate_update_owner,
     validate_ignore_user,
     validate_comment_id,
-    validate_comments_open,
     validate_username,
     validate_bug_feedback,
     validate_testcase_feedback,
@@ -194,7 +193,6 @@ def query_comments(request):
                    validate_update,
                    validate_bug_feedback,
                    validate_testcase_feedback,
-                   validate_comments_open,
                ))
 def new_comment(request):
     """
diff --git a/bodhi/server/validators.py b/bodhi/server/validators.py
--- a/bodhi/server/validators.py
+++ b/bodhi/server/validators.py
@@ -916,23 +916,6 @@ def _conditionally_get_update(request):
     return update
 
 
-@postschema_validator
-def validate_comments_open(request, **kwargs):
-    """
-    Ensure that a comment cannot be added to a stable, pushed update.
-
-    Args:
-        request (pyramid.request.Request): The current request.
-        kwargs (dict): The kwargs of the related service definition. Unused.
-    """
-    update = _conditionally_get_update(request)
-
-    if update.status == UpdateStatus.stable and update.pushed:
-        request.errors.add("body", "comment",
-                           "Comments are Closed on this update. It has been "
-                           "pushed to stable")
-
-
 @postschema_validator
 def validate_bug_feedback(request, **kwargs):
     """


Test patch:
diff --git a/bodhi/tests/server/services/test_comments.py b/bodhi/tests/server/services/test_comments.py
--- a/bodhi/tests/server/services/test_comments.py
+++ b/bodhi/tests/server/services/test_comments.py
@@ -651,32 +651,3 @@ def test_comment_on_locked_update_no_threshhold_action(self):
         # Ensure that the request did not change .. don't trigger something.
         self.assertEqual(up.status.value, UpdateStatus.testing.value)
         self.assertEqual(up.request.value, UpdateRequest.stable.value)
-
-    def test_no_comment_on_stable_pushed_update(self):
-        """ Make sure you cannot comment on stable, pushed updates """
-        up = self.db.query(Build).filter_by(nvr=up2).one().update
-        up.status = UpdateStatus.stable
-        up.request = None
-        up.pushed = True
-        self.assertEqual(len(up.comments), 0)
-        self.assertEqual(up.karma, 0)
-        self.db.info['messages'] = []
-        self.db.flush()
-
-        # check the webui deosnt show the comment form
-        htmlres = self.app.get(f'/updates/{up.alias}', headers={'Accept': 'text/html'})
-        self.assertIn("Comments and Feedback are Closed", htmlres.text)
-        self.assertIn("this update has been pushed to stable", htmlres.text)
-        self.assertNotIn('<form id="new_comment"', htmlres.text)
-
-        # check that we can't actually submit a comment
-        comment = self.make_comment(up2, karma=1)
-        with fml_testing.mock_sends():
-            res = self.app.post_json('/comments/', comment, status=400)
-
-        print(res)
-        self.assertEqual(res.json_body['errors'][0]['description'],
-                         "Comments are Closed on this update. It has been pushed to stable")
-        up = self.db.query(Build).filter_by(nvr=up2).one().update
-        self.assertEqual(len(up.comments), 0)
-        self.assertEqual(up.karma, 0)