Repository: oppia/oppia

Issue:
Flaky backend test (shard 2)
This is shard is currently failing on develop too. Its fails sometimes, passes the othertime.

Fail Runs on my PRs:
https://github.com/oppia/oppia/runs/2659831696?check_suite_focus=true
https://github.com/oppia/oppia/runs/2659745309?check_suite_focus=true
https://github.com/oppia/oppia/runs/2657293249?check_suite_focus=true

Fail runs on develop:
https://github.com/oppia/oppia/runs/2635029355?check_suite_focus=true
https://github.com/oppia/oppia/runs/2632306837?check_suite_focus=true
https://github.com/oppia/oppia/runs/2664130560?check_suite_focus=true

Screenshot of one of my PRs in which the tests pass after a merge with develop. I had merged with develop just a couple of hours before so the only change in that merge was Chirs's PR fixing some e2e test flake.
(A couple JS files having changes)

![image](https://user-images.githubusercontent.com/23002114/119506527-a0c06800-bd8b-11eb-8b1c-d6a0574bff5e.png)

Flaky backend test (shard 2)
This is shard is currently failing on develop too. Its fails sometimes, passes the othertime.

Fail Runs on my PRs:
https://github.com/oppia/oppia/runs/2659831696?check_suite_focus=true
https://github.com/oppia/oppia/runs/2659745309?check_suite_focus=true
https://github.com/oppia/oppia/runs/2657293249?check_suite_focus=true

Fail runs on develop:
https://github.com/oppia/oppia/runs/2635029355?check_suite_focus=true
https://github.com/oppia/oppia/runs/2632306837?check_suite_focus=true
https://github.com/oppia/oppia/runs/2664130560?check_suite_focus=true

Screenshot of one of my PRs in which the tests pass after a merge with develop. I had merged with develop just a couple of hours before so the only change in that merge was Chirs's PR fixing some e2e test flake.
(A couple JS files having changes)

![image](https://user-images.githubusercontent.com/23002114/119506527-a0c06800-bd8b-11eb-8b1c-d6a0574bff5e.png)



Discussion:
@U8NWXD for now I have added it under the automated QA team since I am not sure which team will be picking this up. Feel free to change it.
@chris7716 could you PTAL? The error log refers to DecoratorForUpdatingSuggestionTests.
Also /cc @kevintab95 @sagangwee since this is mostly contributor-dashboard related.
Ack - I will add a PR fixing this ASAP. Thanks.
Hi @chris7716 -- any updates? Maybe you could link the debugging doc here, too.
Hi, I was afk today and will address your comments in the doc https://docs.google.com/document/d/1z2AFo_1SpWzKk5DHEWcxI0AzJjMXs-U9Z8S7_dI6FJM/edit?ts=60ad827a#heading=h.5ce1aah842zc
@chris7716 This needs to be fixed asap, it's creating a big problem for developers (I just wanted hr. fixing this test issue, I assumed it's failing because of changes I've made.) and we never expected the backend test to be flaky! The general procedure is to revert the PR which is introducing flakiness in tests.

Do you have any updates on this? 
> Hi, I was afk today and will address your comments in the doc https://docs.google.com/document/d/1z2AFo_1SpWzKk5DHEWcxI0AzJjMXs-U9Z8S7_dI6FJM/edit?ts=60ad827a#heading=h.5ce1aah842zc

@seanlip Can you please check the comment/replies in this doc^?
I already did. What do you want me to check exactly? I thought we have a plan of action for this already.
@U8NWXD for now I have added it under the automated QA team since I am not sure which team will be picking this up. Feel free to change it.
@chris7716 could you PTAL? The error log refers to DecoratorForUpdatingSuggestionTests.
Also /cc @kevintab95 @sagangwee since this is mostly contributor-dashboard related.
Ack - I will add a PR fixing this ASAP. Thanks.
Hi @chris7716 -- any updates? Maybe you could link the debugging doc here, too.
Hi, I was afk today and will address your comments in the doc https://docs.google.com/document/d/1z2AFo_1SpWzKk5DHEWcxI0AzJjMXs-U9Z8S7_dI6FJM/edit?ts=60ad827a#heading=h.5ce1aah842zc
@chris7716 This needs to be fixed asap, it's creating a big problem for developers (I just wanted hr. fixing this test issue, I assumed it's failing because of changes I've made.) and we never expected the backend test to be flaky! The general procedure is to revert the PR which is introducing flakiness in tests.

Do you have any updates on this? 
> Hi, I was afk today and will address your comments in the doc https://docs.google.com/document/d/1z2AFo_1SpWzKk5DHEWcxI0AzJjMXs-U9Z8S7_dI6FJM/edit?ts=60ad827a#heading=h.5ce1aah842zc

@seanlip Can you please check the comment/replies in this doc^?
I already did. What do you want me to check exactly? I thought we have a plan of action for this already.

Resolving patch:
diff --git a/core/controllers/acl_decorators.py b/core/controllers/acl_decorators.py
--- a/core/controllers/acl_decorators.py
+++ b/core/controllers/acl_decorators.py
@@ -2992,7 +2992,8 @@ def test_can_update_suggestion(
 
         if suggestion.author_id == self.user_id:
             raise base.UserFacingExceptions.UnauthorizedUserException(
-                'You are not allowed to update suggestions that you created.')
+                'The user, %s is not allowed to update self-created'
+                'suggestions.' % (user_services.get_username(self.user_id)))
 
         if suggestion.suggestion_type not in (
                 feconf.CONTRIBUTOR_DASHBOARD_SUGGESTION_TYPES):


Test patch:
diff --git a/core/controllers/acl_decorators_test.py b/core/controllers/acl_decorators_test.py
--- a/core/controllers/acl_decorators_test.py
+++ b/core/controllers/acl_decorators_test.py
@@ -3595,7 +3595,6 @@ def setUp(self):
             debug=feconf.DEBUG,
         ))
 
-        self.login(self.author_email)
         exploration = (
             self.save_new_linear_exp_with_state_names_and_interactions(
                 self.exploration_id, self.author_id, [
@@ -3662,19 +3661,24 @@ def setUp(self):
             },
             'change to state 1')
 
-        translation_suggestion = suggestion_services.query_suggestions(
-            [('author_id', self.author_id),
-             ('target_id', self.exploration_id)])[0]
-        question_suggestion = suggestion_services.query_suggestions(
-            [('author_id', self.author_id),
-             ('target_id', 'skill_123')])[0]
-        edit_state_suggestion = suggestion_services.query_suggestions(
-            [('author_id', self.author_id),
-             ('target_id', self.exploration_id)])[1]
+        translation_suggestions = suggestion_services.get_submitted_suggestions(
+            self.author_id, feconf.SUGGESTION_TYPE_TRANSLATE_CONTENT)
+        question_suggestions = suggestion_services.get_submitted_suggestions(
+            self.author_id, feconf.SUGGESTION_TYPE_ADD_QUESTION)
+        edit_state_suggestions = suggestion_services.get_submitted_suggestions(
+            self.author_id, feconf.SUGGESTION_TYPE_EDIT_STATE_CONTENT)
+
+        self.assertEqual(len(translation_suggestions), 1)
+        self.assertEqual(len(question_suggestions), 1)
+        self.assertEqual(len(edit_state_suggestions), 1)
+
+        translation_suggestion = translation_suggestions[0]
+        question_suggestion = question_suggestions[0]
+        edit_state_suggestion = edit_state_suggestions[0]
+
         self.translation_suggestion_id = translation_suggestion.suggestion_id
         self.question_suggestion_id = question_suggestion.suggestion_id
         self.edit_state_suggestion_id = edit_state_suggestion.suggestion_id
-        self.logout()
 
     def test_authors_cannot_update_suggestion_that_they_created(self):
         self.login(self.author_email)
@@ -3684,7 +3688,8 @@ def test_authors_cannot_update_suggestion_that_they_created(self):
                 expected_status_int=401)
         self.assertEqual(
             response['error'],
-            'You are not allowed to update suggestions that you created.')
+            'The user, %s is not allowed to update self-created'
+            'suggestions.' % self.author_username)
         self.logout()
 
     def test_admin_can_update_any_given_translation_suggestion(self):