Repository: translate/pootle

Issue:
Translations are attributed as 'reviews' in last action
Example: if one goes to https://translate.evernote.com/id/web_evernote/translate/resources/notes/onboarding/Jot_down_ideas.enex.master.po#unit=5382977 they will see that the unit was translated (there was no review, according to the log):

![screenclip 5](https://cloud.githubusercontent.com/assets/1728158/14256433/f0f967b0-fa4d-11e5-93e9-78847ea124ad.png)

TM results also suggest that there was a 100% match.

Now if one goes to extended stats and searches for this unit, they will see this:

![screenclip 6](https://cloud.githubusercontent.com/assets/1728158/14256507/3417e238-fa4e-11e5-93c1-3b17319de131.png)

Indeed the unit was translated with 100% similarity. My understanding is that the translator got to this unit, and at this moment Pootle pre-populated the translation with a fuzzy value from the TM. Translator just removed the "Needs action" flag and hit "Submit".

So far so good. But when one opens the browse UI and sees the last action for this unit, it shows this:

![screenclip 7](https://cloud.githubusercontent.com/assets/1728158/14256622/bbb53c18-fa4e-11e5-881c-d97ded5b7c0d.png)

And the action here is "reviewed", not "translated", which is misleading.

I believe this is a regression. Since the last time we updated our translation server was March, 17. Something likely happened before this date that affected the rendering of the last action snippet.



Discussion:
When we add translation we create a couple of submissions (for `STATE` and `TARGET` fields).
This is very likely that we can create last action message based on `STATE` submission field but we have to use `TARGET` field instead.
See https://github.com/translate/pootle/blob/a064d54b69d7fb344c04accf12876845850116eb/pootle/apps/pootle_statistics/models.py#L311-L313.

When the stats cache retrieves the latest submission, it currently [determines the order by `creation_time` + `pk`](https://github.com/translate/pootle/pull/4025), and since [the change to the state is recorded after the change to the target](https://github.com/translate/pootle/blob/a064d54b69d7fb344c04accf12876845850116eb/pootle/apps/pootle_store/models.py#L1586-L1597), the latest submission information refers to the state change and therefore it interprets it as a review.

After diving deep through git log, I can confirm [this has always been the case](https://github.com/translate/pootle/commit/68fc7235ab230212138b7c26c8222c447a1f4bf7#diff-0038950c975bb50b22ceffd7badd6276R1408) and hence it's not a regression but a bug. 
Still, it's an unknown why the code wasn't adjusted at the time [the comment linked above](https://github.com/translate/pootle/blob/a064d54b69d7fb344c04accf12876845850116eb/pootle/apps/pootle_statistics/models.py#L311-L313) was added. We definitely need to fix this as well as improve the test coverage when it comes to submissions and recorded events.

So summing up, this has always been an issue but it could as well get more noticeable recently after the change in ordering.

I think we also missed this as the exposure of Last Activity on the browse page seems to require really big screen real estate to be able to see it.  So test coverage in this area would be really important.


Resolving patch:
diff --git a/pootle/apps/pootle_store/forms.py b/pootle/apps/pootle_store/forms.py
--- a/pootle/apps/pootle_store/forms.py
+++ b/pootle/apps/pootle_store/forms.py
@@ -267,17 +267,25 @@ class Meta(object):
         def __init__(self, *args, **kwargs):
             self.request = kwargs.pop('request', None)
             super(UnitForm, self).__init__(*args, **kwargs)
-            self.updated_fields = []
+            self._updated_fields = []
 
             self.fields['target_f'].widget.attrs['data-translation-aid'] = \
                 self['target_f'].value()
 
+        @property
+        def updated_fields(self):
+            order_dict = {
+                SubmissionFields.STATE: 0,
+                SubmissionFields.TARGET: 1,
+            }
+            return sorted(self._updated_fields, key=lambda x: order_dict[x[0]])
+
         def clean_target_f(self):
             value = self.cleaned_data['target_f']
 
             if self.instance.target.strings != multistring(value or [u'']):
                 self.instance._target_updated = True
-                self.updated_fields.append((SubmissionFields.TARGET,
+                self._updated_fields.append((SubmissionFields.TARGET,
                                             to_db(self.instance.target),
                                             to_db(value)))
 
@@ -347,7 +355,7 @@ def clean(self):
 
             if old_state not in [new_state, OBSOLETE]:
                 self.instance._state_updated = True
-                self.updated_fields.append((SubmissionFields.STATE,
+                self._updated_fields.append((SubmissionFields.STATE,
                                             old_state, new_state))
 
                 self.cleaned_data['state'] = new_state
diff --git a/pootle/apps/pootle_store/models.py b/pootle/apps/pootle_store/models.py
--- a/pootle/apps/pootle_store/models.py
+++ b/pootle/apps/pootle_store/models.py
@@ -12,6 +12,8 @@
 import os
 from hashlib import md5
 
+from collections import OrderedDict
+
 from translate.filters.decorators import Category
 from translate.storage import base
 
@@ -1138,11 +1140,11 @@ def accept_suggestion(self, suggestion, translation_project, reviewer):
         suggestion.review_time = current_time
         suggestion.save()
 
-        create_subs = {}
-        create_subs[SubmissionFields.TARGET] = [old_target, self.target]
+        create_subs = OrderedDict()
         if old_state != self.state:
             create_subs[SubmissionFields.STATE] = [old_state, self.state]
             self.store.mark_dirty(CachedMethods.WORDCOUNT_STATS)
+        create_subs[SubmissionFields.TARGET] = [old_target, self.target]
 
         for field in create_subs:
             kwargs = {
@@ -1576,14 +1578,7 @@ def record_submissions(self, unit, old_target, old_state, current_time,
         being available in `unit`. Let's look into replacing such members with
         something saner (#3895).
         """
-        create_subs = {}
-
-        # FIXME: extreme implicit hazard
-        if unit._target_updated:
-            create_subs[SubmissionFields.TARGET] = [
-                old_target,
-                unit.target_f,
-            ]
+        create_subs = OrderedDict()
 
         # FIXME: extreme implicit hazard
         if unit._state_updated:
@@ -1592,6 +1587,13 @@ def record_submissions(self, unit, old_target, old_state, current_time,
                 unit.state,
             ]
 
+        # FIXME: extreme implicit hazard
+        if unit._target_updated:
+            create_subs[SubmissionFields.TARGET] = [
+                old_target,
+                unit.target_f,
+            ]
+
         # FIXME: extreme implicit hazard
         if unit._comment_updated:
             create_subs[SubmissionFields.COMMENT] = [
diff --git a/pootle/apps/pootle_store/unit/timeline.py b/pootle/apps/pootle_store/unit/timeline.py
--- a/pootle/apps/pootle_store/unit/timeline.py
+++ b/pootle/apps/pootle_store/unit/timeline.py
@@ -303,10 +303,16 @@ def get_grouped_entries(self):
                 ]
             ])
         )
+
+        # Target field timeline entry should go first
+        def target_field_should_be_first(x):
+            return 0 if x["field"] == SubmissionFields.TARGET else 1
+
         # Group by submitter id and creation_time because
         # different submissions can have same creation time
         for key, values in grouped_timeline:
             entry_group = {'entries': []}
+            values = sorted(values, key=target_field_should_be_first)
             for item in values:
                 if "submitter" not in entry_group:
                     entry_group['submitter'] = DisplayUser(


Test patch:
diff --git a/tests/models/submission.py b/tests/models/submission.py
--- a/tests/models/submission.py
+++ b/tests/models/submission.py
@@ -11,9 +11,12 @@
 from django.utils import timezone
 
 from pytest_pootle.factories import SubmissionFactory
+from pytest_pootle.utils import create_store
 
+from pootle_app.models.permissions import check_permission
 from pootle_statistics.models import (Submission, SubmissionFields,
                                       SubmissionTypes)
+from pootle_store.models import Suggestion, Unit
 from pootle_store.util import TRANSLATED, UNTRANSLATED
 
 
@@ -121,3 +124,79 @@ def test_needs_scorelog():
         new_value=u'',
     )
     assert submission.needs_scorelog()
+
+
+@pytest.mark.django_db
+def test_update_submission_ordering():
+    unit = Unit.objects.filter(state=UNTRANSLATED).first()
+    unit.markfuzzy()
+    unit.target = "Fuzzy Translation for " + unit.source_f
+    unit.save()
+
+    store = create_store(
+        unit.store.pootle_path,
+        "0",
+        [(unit.source_f, "Translation for " + unit.source_f)]
+    )
+    unit.store.update(store)
+    submission_field = Submission.objects.filter(unit=unit).latest().field
+    assert submission_field == SubmissionFields.TARGET
+
+
+@pytest.mark.django_db
+def test_new_translation_submission_ordering(client, request_users, settings):
+    unit = Unit.objects.filter(state=UNTRANSLATED).first()
+    settings.POOTLE_CAPTCHA_ENABLED = False
+    user = request_users["user"]
+    if user.username != "nobody":
+        client.login(
+            username=user.username,
+            password=request_users["password"])
+
+    url = '/xhr/units/%d/' % unit.id
+
+    response = client.post(
+        url,
+        {
+            'state': False,
+            'target_f_0': "Translation for " + unit.source_f,
+        },
+        HTTP_X_REQUESTED_WITH='XMLHttpRequest'
+    )
+
+    if check_permission('translate', response.wsgi_request):
+        assert response.status_code == 200
+        submission_field = Submission.objects.filter(unit=unit).latest().field
+        assert submission_field == SubmissionFields.TARGET
+    else:
+        assert response.status_code == 403
+
+
+@pytest.mark.django_db
+def test_accept_sugg_submission_ordering(client, request_users, settings):
+    """Tests suggestion can be accepted with a comment."""
+    settings.POOTLE_CAPTCHA_ENABLED = False
+    unit = Unit.objects.filter(suggestion__state='pending',
+                               state=UNTRANSLATED)[0]
+    unit.markfuzzy()
+    unit.target = "Fuzzy Translation for " + unit.source_f
+    unit.save()
+    sugg = Suggestion.objects.filter(unit=unit, state='pending')[0]
+    user = request_users["user"]
+    if user.username != "nobody":
+        client.login(
+            username=user.username,
+            password=request_users["password"])
+
+    url = '/xhr/units/%d/suggestions/%d/' % (unit.id, sugg.id)
+    response = client.post(
+        url,
+        HTTP_X_REQUESTED_WITH='XMLHttpRequest'
+    )
+
+    if check_permission('review', response.wsgi_request):
+        assert response.status_code == 200
+        submission_field = Submission.objects.filter(unit=unit).latest().field
+        assert submission_field == SubmissionFields.TARGET
+    else:
+        assert response.status_code == 403