internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: translate/pootle

Issue:
Improve plurals UI for suggested translations
There are no plural form labels in current plural suggestions.
We need to reuse React component `UnitSource` to display those plural form labels.

Also changes in plural suggestions are not highlighted properly. It checks changes between first plural form value only.

![image](https://cloud.githubusercontent.com/assets/682845/18625885/294b1844-7e5a-11e6-8e7d-af4980cc8729.png)



Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_store/views.py b/pootle/apps/pootle_store/views.py
--- a/pootle/apps/pootle_store/views.py
+++ b/pootle/apps/pootle_store/views.py
@@ -427,6 +427,7 @@ def get_context_data(self, *args, **kwargs):
             self.store.priority
             if 'virtualfolder' in settings.INSTALLED_APPS
             else None)
+        suggestions = self.object.get_suggestions()
         return {
             'unit': self.object,
             'form': self.get_unit_edit_form(),
@@ -458,6 +459,9 @@ def get_context_data(self, *args, **kwargs):
             'target_nplurals': self.get_target_nplurals(),
             'has_plurals': self.object.hasplural(),
             'filetype': self.object.store.filetype.name,
+            'suggestions': suggestions,
+            'suggestions_dict': {x.id: dict(id=x.id, target=x.target.strings)
+                                 for x in suggestions},
         }
 
     def get_response_data(self, context):


Test patch:
diff --git a/tests/views/get_edit_unit.py b/tests/views/get_edit_unit.py
--- a/tests/views/get_edit_unit.py
+++ b/tests/views/get_edit_unit.py
@@ -43,6 +43,8 @@ def test_get_edit_unit(project0_nongnu, get_edit_unit, client,
     altsrcs = {x.id: x.data for x in altsrcs}
     sources = {altsrcs[x]['language_code']: altsrcs[x]['target'] for x in altsrcs}
     sources[src_lang.code] = unit.source
+    suggestions_dict = {x.id: dict(id=x.id, target=x.target.strings)
+                        for x in unit.get_suggestions()}
 
     assert result["is_obsolete"] is False
     assert result["sources"] == sources
@@ -55,6 +57,7 @@ def test_get_edit_unit(project0_nongnu, get_edit_unit, client,
     assert response.context["language"] == language
     assert response.context["source_language"] == src_lang
     assert response.context["altsrcs"] == altsrcs
+    assert response.context["suggestions_dict"] == suggestions_dict
 
     assert response.context["cantranslate"] == check_user_permission(
         user, "translate", directory)
Repository: translate/pootle

Issue:
Suggestions count on stats is wrong
See https://mozilla.locamotion.org/cy/ where it says 1 pending suggestion, but clicking on the link says that there are no results.


Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_store/utils.py b/pootle/apps/pootle_store/utils.py
--- a/pootle/apps/pootle_store/utils.py
+++ b/pootle/apps/pootle_store/utils.py
@@ -15,6 +15,7 @@
 
 from pootle.core.delegate import site
 from pootle.core.mail import send_mail
+from pootle.core.signals import update_data
 from pootle.i18n.gettext import ugettext as _
 from pootle_comment.forms import UnsecuredCommentForm
 from pootle_statistics.models import (
@@ -166,6 +167,7 @@ def accept_suggestion(self, suggestion):
         unit.save()
 
     def reject_suggestion(self, suggestion):
+        store = suggestion.unit.store
         suggestion.state = SuggestionStates.REJECTED
         suggestion.review_time = timezone.now()
         suggestion.reviewer = self.reviewer
@@ -176,6 +178,8 @@ def reject_suggestion(self, suggestion):
             self.reviewer,
             creation_time=suggestion.review_time).save()
 
+        update_data.send(store.__class__, instance=store)
+
     def accept_suggestions(self):
         for suggestion in self.suggestions:
             self.accept_suggestion(suggestion)


Test patch:
Repository: translate/pootle

Issue:
Forever loading after captcha
At the first loading of poolte, i mean after this

```
2016-07-22 11:38:22,805 INFO Loading custom settings from '/var/www/translate.tamrielunlimited.it/pootle.conf'...
2016-07-22 11:38:23,146 INFO Using Python PO
2016-07-22 11:38:24,357 INFO Loading custom settings from '/var/www/translate.tamrielunlimited.it/pootle.conf'...
2016-07-22 11:38:24,700 INFO Using Python PO
```

If an anonymous user make a suggestion, he has this problem, it keeps loading forever after the captcha... the only way to resolve this is reloading the page (??several times) and then it works... i think it's the captcha, cause the registered members have not this problem...

I found out this problem few days ago, but i wanted to test with some friends and they had all the same problem after the captcha... when they inserted it and then reloaded the page (someone more than one time), it worked without any other problem...

this is the test as video, check it out to understand: https://dl.dropboxusercontent.com/u/89718989/Screenshot/pootle/pootle-suggestion.mp4



Discussion:
I think it's not the captcha causing the problem, but the guest user itself... tried with `POOTLE_CAPTCHA_ENABLED = False` and still loading...


Resolving patch:
diff --git a/pootle/apps/pootle_store/views.py b/pootle/apps/pootle_store/views.py
--- a/pootle/apps/pootle_store/views.py
+++ b/pootle/apps/pootle_store/views.py
@@ -597,7 +597,8 @@ def suggest(request, unit, **kwargs_):
                 similarity=form.cleaned_data['similarity'],
                 mt_similarity=form.cleaned_data['mt_similarity'])
 
-            json['user_score'] = request.user.public_score
+            if not request.user.is_anonymous:
+                json['user_score'] = request.user.public_score
 
         return JsonResponse(json)
 


Test patch:
diff --git a/tests/views/unit.py b/tests/views/unit.py
--- a/tests/views/unit.py
+++ b/tests/views/unit.py
@@ -282,3 +282,30 @@ def test_submit_unit_plural(client, unit_plural, request_users, settings):
 
     else:
         assert response.status_code == 403
+
+
+@pytest.mark.django_db
+def test_add_suggestion(client, request_users, settings):
+    """Tests translation can be applied after suggestion is accepted."""
+    settings.POOTLE_CAPTCHA_ENABLED = False
+    user = request_users["user"]
+    if user.username != "nobody":
+        client.login(
+            username=user.username,
+            password=request_users["password"])
+
+    unit = Unit.objects.all().first()
+    url = '/xhr/units/%d/suggestions' % unit.id
+    target = "%s TEST SUGGESTION" % unit.source
+    response = client.post(
+        url,
+        {
+            'target_f_0': target,
+        },
+        HTTP_X_REQUESTED_WITH='XMLHttpRequest'
+    )
+
+    assert response.status_code == 200
+    changed = Unit.objects.get(id=unit.id)
+    suggestion = changed.get_suggestions().order_by('id').last()
+    assert suggestion.target == multistring(target)
Repository: translate/pootle

Issue:
Directory hashes are not expired when directories are added/removed
if you add or remove a directory it should expire hashes on all parents and related - ie /projects/x directories


Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_revision/receivers.py b/pootle/apps/pootle_revision/receivers.py
--- a/pootle/apps/pootle_revision/receivers.py
+++ b/pootle/apps/pootle_revision/receivers.py
@@ -6,10 +6,11 @@
 # or later license. See the LICENSE file for a copy of the license and the
 # AUTHORS file for copyright and authorship information.
 
-from django.db.models.signals import post_save
+from django.db.models.signals import post_save, pre_delete
 from django.dispatch import receiver
 
 from pootle.core.delegate import revision_updater
+from pootle_app.models import Directory
 from pootle_data.models import StoreData
 from pootle_store.models import Store
 
@@ -18,3 +19,17 @@
 def handle_storedata_save(**kwargs):
     revision_updater.get(Store)(
         context=kwargs["instance"].store).update(keys=["stats", "checks"])
+
+
+@receiver(post_save, sender=Directory)
+def handle_directory_save(**kwargs):
+    if kwargs.get("created"):
+        return
+    revision_updater.get(Directory)(
+        context=kwargs["instance"]).update(keys=["stats", "checks"])
+
+
+@receiver(pre_delete, sender=Directory)
+def handle_directory_delete(**kwargs):
+    revision_updater.get(Directory)(
+        context=kwargs["instance"].parent).update(keys=["stats", "checks"])


Test patch:
diff --git a/tests/pootle_app/__init__.py b/tests/pootle_app/__init__.py
new file mode 100644
diff --git a/tests/pootle_app/models.py b/tests/pootle_app/models.py
new file mode 100644
--- /dev/null
+++ b/tests/pootle_app/models.py
@@ -0,0 +1,35 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) Pootle contributors.
+#
+# This file is a part of the Pootle project. It is distributed under the GPL3
+# or later license. See the LICENSE file for a copy of the license and the
+# AUTHORS file for copyright and authorship information.
+
+import pytest
+
+from pootle.core.delegate import revision
+from pootle_app.models import Directory
+
+
+@pytest.mark.django_db
+def test_directory_lifecycle_revision(subdir0, tp0, project0, language0):
+    target = subdir0.child_dirs.create(name="foo")
+    rev = revision.get(Directory)(target)
+    # no cache_key on creation
+    assert not rev.get(key="stats")
+    target.save()
+    key = rev.get(key="stats")
+    assert key
+    assert key == revision.get(Directory)(subdir0).get(key="stats")
+    assert key == revision.get(Directory)(tp0.directory).get(key="stats")
+    assert key == revision.get(Directory)(tp0.directory.parent).get(key="stats")
+    assert key == revision.get(Directory)(language0.directory).get(key="stats")
+
+    target.delete()
+    new_key = revision.get(Directory)(subdir0).get(key="stats")
+    assert new_key
+    assert new_key != key
+    assert new_key == revision.get(Directory)(tp0.directory).get(key="stats")
+    assert new_key == revision.get(Directory)(tp0.directory.parent).get(key="stats")
+    assert new_key == revision.get(Directory)(language0.directory).get(key="stats")
Repository: translate/pootle

Issue:
Make comment submit_date tz aware
atm comments generate

```
RuntimeWarning: DateTimeField Comment.submit_date received a naive datetime
```



Discussion:
I'm not sure if its possible to audit these and try globally block anything that isn't TZ aware? Feels like Pootle as a whole should just be very locale aware.

the problem is that its a runtime warning - we probs dont want anything to fail that late in play if it doesnt really have to

...i was thinking aloud there probably - i was thinking of turning the warning to error

quite a few of these are picked up in the test env actually - ive removed a few - we could make those error at least


Resolving patch:
diff --git a/pootle/apps/pootle_comment/forms.py b/pootle/apps/pootle_comment/forms.py
--- a/pootle/apps/pootle_comment/forms.py
+++ b/pootle/apps/pootle_comment/forms.py
@@ -14,6 +14,8 @@
 
 from django_comments.forms import CommentForm as DjCommentForm
 
+from pootle.core.utils.timezone import make_aware
+
 from .delegate import comment_should_not_be_saved
 from .exceptions import CommentNotSaved
 from .signals import comment_was_saved
@@ -55,7 +57,7 @@ def clean(self):
     def save(self):
         comment = self.comment
         comment.user = self.cleaned_data["user"]
-        comment.submit_date = datetime.now()
+        comment.submit_date = make_aware(datetime.now())
         comment.save()
         comment_was_saved.send(
             sender=comment.__class__,
diff --git a/pootle/apps/pootle_store/models.py b/pootle/apps/pootle_store/models.py
--- a/pootle/apps/pootle_store/models.py
+++ b/pootle/apps/pootle_store/models.py
@@ -901,7 +901,7 @@ def toggle_qualitycheck(self, check_id, false_positive, user):
         else:
             sub_type = SubmissionTypes.UNMUTE_CHECK
 
-        sub = Submission(creation_time=timezone.now(),
+        sub = Submission(creation_time=make_aware(timezone.now()),
                          translation_project=self.store.translation_project,
                          submitter=user, field=SubmissionFields.NONE,
                          unit=self, store=self.store, type=sub_type,


Test patch:
diff --git a/tests/models/scorelog.py b/tests/models/scorelog.py
--- a/tests/models/scorelog.py
+++ b/tests/models/scorelog.py
@@ -8,8 +8,6 @@
 
 import pytest
 
-from datetime import datetime
-
 from pytest_pootle.factories import ScoreLogFactory, SubmissionFactory
 
 from pootle_statistics.models import (ScoreLog, SubmissionTypes, SubmissionFields,
@@ -38,7 +36,6 @@ def test_record_submission(member, submission_type):
         'mt_similarity': 0,
         'submitter': member,
         'translation_project': store.translation_project,
-        'creation_time': datetime.now(),
     }
 
     sub = SubmissionFactory(**submission_params)
Repository: translate/pootle

Issue:
Priority column is missing
Since the column reordering we've lost the priority column in the vfolders table


Discussion:

Resolving patch:
diff --git a/pootle/apps/virtualfolder/views.py b/pootle/apps/virtualfolder/views.py
--- a/pootle/apps/virtualfolder/views.py
+++ b/pootle/apps/virtualfolder/views.py
@@ -122,7 +122,7 @@ class VFoldersDataView(object):
     _table_fields = (
         'name', 'progress', 'activity',
         'total', 'need-translation',
-        'suggestions', 'critical')
+        'suggestions', 'critical', 'priority')
 
     def __init__(self, context, user, has_admin_access=False):
         self.context = context


Test patch:
diff --git a/tests/vfolders/vf_data.py b/tests/vfolders/vf_data.py
--- a/tests/vfolders/vf_data.py
+++ b/tests/vfolders/vf_data.py
@@ -37,6 +37,7 @@ def test_vfolder_data_view(tp0, request_users):
         in vf_data.all_stats.items()]
     for i, row in enumerate(vf_data.table_items):
         assert rows[i] == row
+    assert "priority" in vf_data.table_fields
     expected_table_data = dict(
         children=dict(
             id='vfolders',
Repository: translate/pootle

Issue:
Stats are wrong for children where the child name is repeated in the childs descendants
this is causing stats to foo


Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_data/directory_data.py b/pootle/apps/pootle_data/directory_data.py
--- a/pootle/apps/pootle_data/directory_data.py
+++ b/pootle/apps/pootle_data/directory_data.py
@@ -54,6 +54,5 @@ def get_children_stats(self, qs):
         return children
 
     def get_root_child_path(self, child):
-        return (
-            child["store__parent__tp_path"].replace(
-                self.context.tp_path, "").split("/")[0])
+        return child["store__parent__tp_path"][
+            len(self.context.tp_path):].split("/")[0]


Test patch:
Repository: translate/pootle

Issue:
Uploading downloaded store to other pootle_path fails to update stats
if you download a store and then upload it elsewhere in the tree it doesnt update the directory stats correctly


Discussion:

Resolving patch:
diff --git a/pootle/apps/import_export/utils.py b/pootle/apps/import_export/utils.py
--- a/pootle/apps/import_export/utils.py
+++ b/pootle/apps/import_export/utils.py
@@ -42,8 +42,8 @@ def import_file(f, user=None):
     rev = int(rev)
 
     try:
-        store = Store.objects.get_or_create(pootle_path=pootle_path)[0]
-    except Exception as e:
+        store = Store.objects.get(pootle_path=pootle_path)
+    except Store.DoesNotExist as e:
         raise FileImportError(_("Could not create '%s'. Missing "
                                 "Project/Language? (%s)", (f.name, e)))
 
diff --git a/pootle/apps/import_export/views.py b/pootle/apps/import_export/views.py
--- a/pootle/apps/import_export/views.py
+++ b/pootle/apps/import_export/views.py
@@ -111,8 +111,7 @@ def handle_upload_form(request, tp):
                             with zf.open(path, "r") as f:
                                 import_file(f, user=uploader)
                 else:
-                    # It is necessary to seek to the beginning because
-                    # is_zipfile fucks the file, and thus cannot be read.
+                    # is_zipfile consumes the file buffer
                     django_file.seek(0)
                     import_file(django_file, user=uploader)
             except Exception as e:
diff --git a/pootle/apps/pootle_translationproject/views.py b/pootle/apps/pootle_translationproject/views.py
--- a/pootle/apps/pootle_translationproject/views.py
+++ b/pootle/apps/pootle_translationproject/views.py
@@ -244,8 +244,9 @@ class TPBrowseBaseView(PootleBrowseView):
     template_extends = 'translation_projects/base.html'
 
     def get_context_data(self, *args, **kwargs):
+        upload_widget = self.get_upload_widget()
         ctx = super(TPBrowseBaseView, self).get_context_data(*args, **kwargs)
-        ctx.update(self.get_upload_widget())
+        ctx.update(upload_widget)
         ctx.update(
             {'parent': get_parent(self.object)})
         return ctx


Test patch:
Repository: translate/pootle

Issue:
Silence dev checks in the admin UI as well
There's the following section in the Pootle dev config:

``` Python
# Silence the DEBUG check on dev servers
SILENCED_SYSTEM_CHECKS = [
    'pootle.W004',  # python-levenstein not installed
    'pootle.W005',  # DEBUG = True
    'pootle.W010',  # DEFAULT_FROM_EMAIL has default setting
    'pootle.W011',  # POOTLE_CONTACT_EMAIL has default setting
]
```

When running Pootle, I still get these notifications in the admin UI among others:

> `/!\` DEBUG mode is on. Do not do this in production! Set DEBUG = False in Pootle settings
> 
> `/!\`  POOTLE_CONTACT_EMAIL is using the following default setting 'info@YOUR_DOMAIN.com'. POOTLE_CONTACT_EMAIL is the address that will receive messages sent by the contact form.
> 
> `/!\` DEFAULT_FROM_EMAIL is using the following default setting 'webmaster@localhost'. DEFAULT_FROM_EMAIL is used in all outgoing Pootle email. Don't forget to review your mail server settings.

I think it might make sense to silence them consistently not only in the console, but in the admin UI as well.



Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_app/views/admin/dashboard.py b/pootle/apps/pootle_app/views/admin/dashboard.py
--- a/pootle/apps/pootle_app/views/admin/dashboard.py
+++ b/pootle/apps/pootle_app/views/admin/dashboard.py
@@ -82,7 +82,7 @@ def rq_stats():
 def checks():
     from django.core.checks.registry import registry
 
-    return registry.run_checks()
+    return [e for e in registry.run_checks() if not e.is_silenced()]
 
 
 @admin_required


Test patch:
Repository: translate/pootle

Issue:
Downloading store in a zip file with no revision lacks revision headers
not sure if this is a dupe or related to other PO-headers bugs - but it seems like stores with no unit revision dont get headers - this causes re-uploading to fail


Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_store/store/serialize.py b/pootle/apps/pootle_store/store/serialize.py
--- a/pootle/apps/pootle_store/store/serialize.py
+++ b/pootle/apps/pootle_store/store/serialize.py
@@ -50,8 +50,9 @@ def tostring(self):
         if hasattr(store, "updateheader"):
             # FIXME We need those headers on import
             # However some formats just don't support setting metadata
+            max_unit_revision = self.max_unit_revision or 0
             store.updateheader(add=True, X_Pootle_Path=self.pootle_path)
-            store.updateheader(add=True, X_Pootle_Revision=self.max_unit_revision)
+            store.updateheader(add=True, X_Pootle_Revision=max_unit_revision)
         return str(store)
 
     def pipeline(self, data):


Test patch:
diff --git a/tests/import_export/export.py b/tests/import_export/export.py
new file mode 100644
Repository: translate/pootle

Issue:
Lost timeSince data for suggestions



Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_misc/templatetags/locale.py b/pootle/apps/pootle_misc/templatetags/locale.py
--- a/pootle/apps/pootle_misc/templatetags/locale.py
+++ b/pootle/apps/pootle_misc/templatetags/locale.py
@@ -6,11 +6,14 @@
 # or later license. See the LICENSE file for a copy of the license and the
 # AUTHORS file for copyright and authorship information.
 
+import calendar
+
 from django import template
 from django.utils.formats import get_format
 from django.utils.translation import trans_real
 
 from pootle.core.utils import dateformat
+from pootle.local.dates import timesince
 
 
 register = template.Library()
@@ -40,6 +43,11 @@ def do_dateformat(value, use_format='c'):
     return dateformat.format(value, use_format)
 
 
+@register.filter(name='relative_datetime_format')
+def do_relative_datetime_format(value):
+    return timesince(calendar.timegm(value.timetuple()))
+
+
 @register.simple_tag
 def locale_align():
     """Returns current locale's default alignment."""


Test patch:
Repository: translate/pootle

Issue:
Stats table shows no zero counts
This can be seen in the following screenshot:

![no-zero-counts](https://cloud.githubusercontent.com/assets/939880/21317886/55bed600-c607-11e6-8fb6-44099121b6af.png)



Discussion:

Resolving patch:
diff --git a/pootle/core/views/display.py b/pootle/core/views/display.py
--- a/pootle/core/views/display.py
+++ b/pootle/core/views/display.py
@@ -157,7 +157,7 @@ def make_display_stat(d, keys=["total", "critical", "incomplete",
                                    "suggestions", "fuzzy", "untranslated"]):
         assert isinstance(d, dict)
         for k in keys:
-            if d.get(k):
+            if k in d:
                 d[k + '_display'] = formatter.number(d[k])
 
     @cached_property


Test patch:
Repository: translate/pootle

Issue:
update_stores' last updated date doesn't tickle up to project overview/language list
When updating against templates, the /projects/projectname/ listing doesn't reflect the **last update**, **unless** the update affected a file in the **toplevel** dir.

Within a language overview (/lang/projectname), changes deep in a directory hierarchy will also affect the  parent directory's last-change date.

using pootle 2.8.0b5 (TDF)

screenshots to clarify. overview lists last update as e.g. 3 weeks ago:
![bildschirmfoto von 2016-12-22 16 40 21](https://cloud.githubusercontent.com/assets/477936/21430954/acb2d028-c865-11e6-84ec-0784563abba2.png)

drilling down to the language reveals that the files in xmlsecurity actually had been updated only 8 hours ago (in fact xmlsecurity/uiconfig/ui.po)
![bildschirmfoto von 2016-12-22 16 41 26](https://cloud.githubusercontent.com/assets/477936/21431071/27d3ab2e-c866-11e6-8350-4690fe89cc03.png)

(also sorting by the last updated is not working properly, goes from 10months to 8 hours, to 3weeks…)   


Discussion:

Resolving patch:
diff --git a/pootle/apps/pootle_data/project_data.py b/pootle/apps/pootle_data/project_data.py
--- a/pootle/apps/pootle_data/project_data.py
+++ b/pootle/apps/pootle_data/project_data.py
@@ -6,6 +6,8 @@
 # or later license. See the LICENSE file for a copy of the license and the
 # AUTHORS file for copyright and authorship information.
 
+from pootle.core.delegate import revision
+
 from .utils import RelatedStoresDataTool, RelatedTPsDataTool
 
 
@@ -17,6 +19,11 @@ class ProjectDataTool(RelatedTPsDataTool):
     def filter_data(self, qs):
         return qs.filter(tp__project=self.context)
 
+    @property
+    def rev_cache_key(self):
+        return revision.get(
+            self.context.__class__)(self.context.directory).get(key="stats")
+
 
 class ProjectResourceDataTool(RelatedStoresDataTool):
     group_by = ("store__translation_project__language__code", )


Test patch:
diff --git a/tests/pootle_data/data_tool.py b/tests/pootle_data/data_tool.py
--- a/tests/pootle_data/data_tool.py
+++ b/tests/pootle_data/data_tool.py
@@ -237,7 +237,7 @@ def test_data_cache_keys(language0, project0, subdir0, vfolder0):
         '%s.%s.%s'
         % (project0.data_tool.cache_key_name,
            project0.code,
-           revision.get(Project)(project0).get(key="stats"))
+           revision.get(Project)(project0.directory).get(key="stats"))
         == project0.data_tool.cache_key)
     # directory
     assert subdir0.data_tool.ns == "pootle.data"