Repository: fedora-infra/bodhi

Issue:
taskotron results don't show up for all tested architectures
We have recently done a few changes in how taskotron tests are scheduled and reported. We now run tests on both `x86_64` and `armhfp` builds/repos, and we often execute this separately (two runs), instead all in a single run. This was to increase tests reliability and allow future scaling. As a consequence, results shown in Bodhi got a bit out of sync, and don't work as expected. I believe this affects just `dist.rpmdeplint`, `dist.abicheck` and `dist.rpmlint`, where these changes have been performed.  For these tests, Bodhi shows up the very last report in ResultsDB, regardless of architecture. That means you sometimes see `dist.rpmdeplint` result for `x86_64`, and sometimes for `armhfp`, based on which one was executed last. The same applies to the other two tests. The ideal state would of course be to show results for all (both) architectures.

I was the one writing the taskotron querying code in the past, but I didn't have time to delve deep into it again, so I'm operating mostly from memory (sorry if I get something wrong). As a reference, I'll use:
https://bodhi.fedoraproject.org/updates/FEDORA-2018-25ccbfaf95

Seeing which queries Bodhi javascript makes, I think this could be solved pretty easily by adjusting this request:
```
latest?item=$item&type=koji_build&testcases:like=dist.*
```
so in this particular example this one:
https://taskotron.fedoraproject.org/resultsdb_api//api/v2.0/results/latest?item=vim-8.1.035-1.fc28&type=koji_build&testcases:like=dist.*

into two separate requests with a specified architecture:
```
latest?item=$item&type=koji_build&testcases:like=dist.*&arch=x86_64
latest?item=$item&type=koji_build&testcases:like=dist.*&arch=armhfp
```
i.e. these two:
https://taskotron.fedoraproject.org/resultsdb_api//api/v2.0/results/latest?item=vim-8.1.035-1.fc28&type=koji_build&testcases:like=dist.*&arch=x86_64
https://taskotron.fedoraproject.org/resultsdb_api//api/v2.0/results/latest?item=vim-8.1.035-1.fc28&type=koji_build&testcases:like=dist.*&arch=armhfp

And then making sure the results show up properly in the web UI (sorted, etc). Please note that some of the results will be part of both queries (tests which still run on both architectures in a single run, for different reasons, report the result marked with both arches - that includes `dist.rpmgrill` and `dist.python-versions`, iirc), so you'll need to deduplicate those. Should be simple since all results have an unique ID.

I'm not sure if the queries performed by the server side of Bodhi (python, not javascript) also need to be adjusted, or whether all of that code has been moved to greenwave.

@bowlofeggs If you prefer, @lukassparrow was willing to look into this.


Discussion:

Resolving patch:
diff --git a/bodhi/server/util.py b/bodhi/server/util.py
--- a/bodhi/server/util.py
+++ b/bodhi/server/util.py
@@ -530,43 +530,6 @@ def status2html(context, status):
             "</span>")
 
 
-def test_gating_status2html(context, status, reason=None):
-    """
-    Render the given TestGatingStatus as a span, with an optional p tag with a reason.
-
-    Convert the specified status into a html string used to present the
-    test gating status in a human friendly way.
-
-    Args:
-        context (mako.runtime.Context): Unused.
-        status (bodhi.server.models.TestGatingStatus): The test gating status
-        reason (unicode): A short text explains why the test gating is not passed.
-
-    Returns:
-        unicode: A human friendly HTML label of the test gating status
-
-    """
-    if status:
-        label_class = {
-            'Ignored': 'success',
-            'Running': 'warning',
-            'Passed': 'success',
-            'Failed': 'danger',
-            'Queued': 'info',
-            'Waiting': 'info'
-        }[status.description]
-        description = status.description
-    else:
-        # status could be None when test_gating.required was not set to True
-        description = 'not running'
-        label_class = 'primary'
-    html = u'<span class="label label-%s">Tests %s</span>' % (label_class, description)
-    from bodhi.server.models import TestGatingStatus
-    if status not in [None, TestGatingStatus.ignored, TestGatingStatus.passed] and reason:
-        html += u'<p><a class="gating-summary" href="#automatedtests">%s</a></p>' % reason
-    return html
-
-
 def greenwave_unsatisfied_requirements_html(context, update):
     """
     Return an html representation of the given update's greenwave_unsatisfied_requirements field.


Test patch:
diff --git a/bodhi/tests/server/services/test_updates.py b/bodhi/tests/server/services/test_updates.py
--- a/bodhi/tests/server/services/test_updates.py
+++ b/bodhi/tests/server/services/test_updates.py
@@ -4972,84 +4972,6 @@ def test_urgent_update_bypass_batched(self, publish, *args):
         up = self.db.query(Update).filter_by(title=resp.json['title']).one()
         self.assertEquals(up.request, UpdateRequest.stable)
 
-    @mock.patch(**mock_taskotron_results)
-    @mock.patch(**mock_valid_requirements)
-    def test_alert_not_displayed_if_no_summary_present(self, publish, *args):
-        """ Ensure that the alert is not display if there are not summary
-        status (returned normaly by greenwave) set in the database for this
-        update.
-        """
-        nvr = u'bodhi-2.1-1.fc17'
-
-        up_data = self.get_update(nvr)
-        up_data['csrf_token'] = self.app.get('/csrf').json_body['csrf_token']
-        self.app.post_json('/updates/', up_data)
-
-        build = self.db.query(RpmBuild).filter_by(nvr=nvr).one()
-        build.update.test_gating_status = TestGatingStatus.failed
-        self.assertEqual(build.update.request, UpdateRequest.testing)
-
-        with mock.patch.dict(self.app.app.registry.settings, {'test_gating.required': True}):
-            resp = self.app.get('/updates/%s' % nvr,
-                                headers={'Accept': 'text/html'})
-            self.assertIn('text/html', resp.headers['Content-Type'])
-            self.assertIn(nvr, resp)
-            self.assertNotIn(
-                '<div class="alert alert-danger">The update can not be pushed:',
-                resp)
-
-    @mock.patch(**mock_taskotron_results)
-    @mock.patch(**mock_valid_requirements)
-    def test_alert_notdisplayed_if_summary_present_but_gating_off(
-            self, publish, *args):
-        """ Ensure that the alert is not being displayed even if there is a
-        summary status (returned normaly by greenwave) set in the database
-        for this update, because the configuration is set otherwise.
-        """
-        nvr = u'bodhi-2.1-1.fc17'
-
-        up_data = self.get_update(nvr)
-        up_data['csrf_token'] = self.app.get('/csrf').json_body['csrf_token']
-        self.app.post_json('/updates/', up_data)
-
-        build = self.db.query(RpmBuild).filter_by(nvr=nvr).one()
-        build.update.test_gating_status = TestGatingStatus.failed
-        build.update.greenwave_summary_string = 'Because Randy said so'
-        self.assertEqual(build.update.request, UpdateRequest.testing)
-
-        with mock.patch.dict(self.app.app.registry.settings, {'test_gating.required': False}):
-            resp = self.app.get('/updates/%s' % nvr,
-                                headers={'Accept': 'text/html'})
-            self.assertIn('text/html', resp.headers['Content-Type'])
-            self.assertIn(nvr, resp)
-            self.assertNotIn(
-                '<div class="alert alert-danger">The update can not be pushed: \' + summary ', resp)
-
-    @mock.patch(**mock_taskotron_results)
-    @mock.patch(**mock_valid_requirements)
-    def test_alert_displayed_if_summary_present(self, publish, *args):
-        """ Ensure that the alert is displayed if there is a summary status
-        (returned normaly by greenwave) set in the database for this update.
-        """
-        nvr = u'bodhi-2.1-1.fc17'
-
-        up_data = self.get_update(nvr)
-        up_data['csrf_token'] = self.app.get('/csrf').json_body['csrf_token']
-        self.app.post_json('/updates/', up_data)
-
-        build = self.db.query(RpmBuild).filter_by(nvr=nvr).one()
-        build.update.test_gating_status = TestGatingStatus.failed
-        build.update.greenwave_summary_string = 'Because Randy said so'
-        self.assertEqual(build.update.request, UpdateRequest.testing)
-
-        with mock.patch.dict(self.app.app.registry.settings, {'test_gating.required': True}):
-            resp = self.app.get('/updates/%s' % nvr,
-                                headers={'Accept': 'text/html'})
-            self.assertIn('text/html', resp.headers['Content-Type'])
-            self.assertIn(nvr, resp)
-            self.assertIn(
-                '<div class="alert alert-danger">The update can not be pushed: \' + summary ', resp)
-
 
 class TestWaiveTestResults(BaseTestCase):
     """
diff --git a/bodhi/tests/server/test_util.py b/bodhi/tests/server/test_util.py
--- a/bodhi/tests/server/test_util.py
+++ b/bodhi/tests/server/test_util.py
@@ -1218,44 +1218,6 @@ def test_splitter(self):
         splitspacestring = util.splitter("build-0.1 build-0.2")
         self.assertEqual(splitspacestring, ['build-0.1', 'build-0.2'])
 
-    def test_test_gating_status2html_ignored(self):
-        """ Test the test_gating_status2html method with a status: Ignored. """
-        output = util.test_gating_status2html(None, TestGatingStatus.ignored)
-        assert output == '<span class="label label-success">Tests Ignored</span>'
-
-    def test_test_gating_status2html_running(self):
-        """ Test the test_gating_status2html method with a status: Running. """
-        output = util.test_gating_status2html(None, TestGatingStatus.running)
-        assert output == '<span class="label label-warning">Tests Running</span>'
-
-    def test_test_gating_status2html_passed(self):
-        """ Test the test_gating_status2html method with a status: Passed. """
-        output = util.test_gating_status2html(None, TestGatingStatus.passed)
-        assert output == '<span class="label label-success">Tests Passed</span>'
-
-    def test_test_gating_status2html_failed(self):
-        """ Test the test_gating_status2html method with a status: Failed. """
-        output = util.test_gating_status2html(None, TestGatingStatus.failed, '1 of 3 tests failed')
-        expected = ('<span class="label label-danger">Tests Failed</span>'
-                    '<p><a class="gating-summary" href="#automatedtests">'
-                    '1 of 3 tests failed</a></p>')
-        assert output == expected
-
-    def test_test_gating_status2html_queued(self):
-        """ Test the test_gating_status2html method with a status: Queued. """
-        output = util.test_gating_status2html(None, TestGatingStatus.queued)
-        assert output == '<span class="label label-info">Tests Queued</span>'
-
-    def test_test_gating_status2html_waiting(self):
-        """ Test the test_gating_status2html method with a status: Waiting. """
-        output = util.test_gating_status2html(None, TestGatingStatus.waiting)
-        assert output == '<span class="label label-info">Tests Waiting</span>'
-
-    def test_test_gating_status2html_missing(self):
-        """ Test the test_gating_status2html method with a status: None. """
-        output = util.test_gating_status2html(None, None)
-        assert output == '<span class="label label-primary">Tests not running</span>'
-
     @mock.patch('bodhi.server.util.requests.get')
     @mock.patch('bodhi.server.util.log.exception')
     def test_taskotron_results_non_200(self, log_exception, mock_get):