internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: carpentries/amy

Issue:
Fix fake_database.py script
As it was reported by @maneesha, the script is broken.


Discussion:

Resolving patch:
diff --git a/amy/workshops/management/commands/fake_database.py b/amy/workshops/management/commands/fake_database.py
--- a/amy/workshops/management/commands/fake_database.py
+++ b/amy/workshops/management/commands/fake_database.py
@@ -214,7 +214,8 @@ def fake_training_request(self, person_or_None):
             state = 'a'
             person = person_or_None
 
-        occupation = choice(ProfileUpdateRequest.OCCUPATION_CHOICES)[0]
+        occupation = choice(ProfileUpdateRequest._meta.get_field('occupation')
+                                                      .choices)[0]
         training_completion_agreement = randbool(0.5)
         req = TrainingRequest.objects.create(
             state=state,
@@ -250,7 +251,8 @@ def fake_training_request(self, person_or_None):
                 TrainingRequest.MAX_TRAVELLING_FREQUENCY_CHOICES)[0],
             max_travelling_frequency_other='',
             reason=self.faker.text(),
-            comment=self.faker.text() if randbool(0.3) else '',
+            # there's no such field as `comment` anymore
+            # comment=self.faker.text() if randbool(0.3) else '',
             training_completion_agreement=training_completion_agreement,
             workshop_teaching_agreement=randbool(0.5) if training_completion_agreement else False,
         )
@@ -289,8 +291,8 @@ def fake_person(self, *, is_instructor, is_trainer=False):
             # assume that the username is provided by the person
             username = social_username
 
-        github = social_username if randbool(0.5) else None
-        twitter = social_username if randbool(0.5) else None
+        github = social_username
+        twitter = social_username
         url = self.faker.url() if randbool(0.5) else ''
 
         person = Person.objects.create(
@@ -587,7 +589,9 @@ def fake_profile_update_requests(self, count=20):
                 airport_iata=(p.airport.iata
                               if randbool(0.9) and p.airport is not None
                               else choice(Airport.objects.all()).iata),
-                occupation=choice(ProfileUpdateRequest.OCCUPATION_CHOICES)[0],
+                occupation=choice(ProfileUpdateRequest._meta
+                                                      .get_field('occupation')
+                                                      .choices)[0],
                 occupation_other='',
                 github=p.github or '',
                 twitter=p.twitter or '',


Test patch:
Repository: carpentries/amy

Issue:
Include badge & badge date in member view 
In the member view [such as this](https://amy.software-carpentry.org/fiscal/membership/121/), we see a list of instructor training seats with the columns Event, Person, and Task.

Can we also have two more columns in this view: Badge (SWC, DC, and/or LC) and Date Badged?

Task appears to be a concatenation of Event, Person, and Role. If Role is always going to be Learner, we can remove this column to make room for the new columns suggested above.


Discussion:

Resolving patch:
diff --git a/amy/fiscal/views.py b/amy/fiscal/views.py
--- a/amy/fiscal/views.py
+++ b/amy/fiscal/views.py
@@ -34,6 +34,8 @@
     Organization,
     Membership,
     Sponsorship,
+    Task,
+    Award,
 )
 from workshops.util import (
     OnlyForAdminsMixin,
@@ -126,11 +128,16 @@ class AllMemberships(OnlyForAdminsMixin, AMYListView):
 
 
 class MembershipDetails(OnlyForAdminsMixin, AMYDetailView):
-    queryset = (
-        Membership.objects
-                  .select_related('organization')
-                  .prefetch_related('task_set')
-    )
+    prefetch_awards = Prefetch('person__award_set',
+                               queryset=Award.objects.select_related('badge'))
+    queryset = Membership.objects.select_related('organization') \
+        .prefetch_related(
+            Prefetch(
+                'task_set',
+                queryset=Task.objects.select_related('event', 'person')
+                                     .prefetch_related(prefetch_awards)
+            )
+        )
     context_object_name = 'membership'
     template_name = 'fiscal/membership.html'
     pk_url_kwarg = 'membership_id'


Test patch:
Repository: carpentries/amy

Issue:
"Dutch Flemish" should be "Dutch/Flemish"



Discussion:
@steltenpower - where is this listed?

Languages were included in AMY through a script. See https://github.com/swcarpentry/amy/blob/develop/data/registry.json for language source, and migrations https://github.com/swcarpentry/amy/blob/develop/workshops/migrations/0097_auto_20160519_0739.py and https://github.com/swcarpentry/amy/blob/develop/workshops/migrations/0139_fix_language_names.py for a way to parse & load the data into DB.

The space in `Dutch Flemish` is result of concatenation a description list; this issue might apply to other languages as well, e.g. Spanish. As a solution, we can do a migration to re-import languages and fix the names.

Resolving patch:
diff --git a/amy/workshops/migrations/0178_auto_20190213_1640.py b/amy/workshops/migrations/0178_auto_20190213_1640.py
new file mode 100644
--- /dev/null
+++ b/amy/workshops/migrations/0178_auto_20190213_1640.py
@@ -0,0 +1,47 @@
+# Generated by Django 2.1.2 on 2019-02-13 16:40
+
+from django.db import migrations
+
+import json
+
+
+def update_language_names(apps, schema_editor):
+    """`registry.json` in some cases provides multiple descriptions for the
+    same language. This will update the names of these languages in the
+    database with proper punctuation between description choices.
+
+    For example: language coded as "nl" is described as ["Dutch", "Flemish"].
+    These are separate but similar languages. They used to be named
+    "Dutch Flemish" in database, but that is changed here to "Dutch, Flemish".
+
+    See https://github.com/swcarpentry/amy/issues/1298 for more info.
+
+    Additionally, the registry was updated with 2018-08 version, and so the
+    database is updated with other languages as well."""
+
+    Language = apps.get_model('workshops', 'Language')
+
+    # read list of languages
+    languages_json = json.load(open('data/registry.json', encoding='utf-8'))
+
+    for language in languages_json:
+        if language['Type'] == 'language' and len(language['Subtag']) <= 2:
+            # update language (found by subtag) with a new name,
+            # or create if it doesn't exist yet
+            Language.objects.update_or_create(
+                subtag=language['Subtag'],
+                defaults=dict(
+                    name=', '.join(language['Description']),
+                ),
+            )
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('workshops', '0177_auto_20190130_0815'),
+    ]
+
+    operations = [
+        migrations.RunPython(update_language_names),
+    ]


Test patch:
Repository: carpentries/amy

Issue:
Freeze top navbar
Top navigation bar should freeze so it stays visible even when scrolling down screen.


Discussion:
I just found out that there's some bug with our CSS compression mechanism, that prevents the top navbar from sticking.

Resolving patch:
diff --git a/amy/trainings/filters.py b/amy/trainings/filters.py
--- a/amy/trainings/filters.py
+++ b/amy/trainings/filters.py
@@ -1,10 +1,10 @@
 import re
 
-from dal import autocomplete
 import django_filters
 from django.db.models import Q
 from django.forms import widgets
 
+from workshops.fields import ModelSelect2
 from workshops.filters import (
     AMYFilterSet,
     NamesOrderingFilter,
@@ -128,7 +128,7 @@ class TraineeFilter(AMYFilterSet):
         queryset=Event.objects.ttt(),
         method=filter_trainees_by_training,
         label='Training',
-        widget=autocomplete.ModelSelect2(
+        widget=ModelSelect2(
             url='ttt-event-lookup',
             attrs=SIDEBAR_DAL_WIDTH,
         ),
diff --git a/config/settings.py b/config/settings.py
--- a/config/settings.py
+++ b/config/settings.py
@@ -22,6 +22,8 @@
     # OS environment variables take precedence over variables from .env
     env.read_env(str(ROOT_DIR.path('.env')))
 
+CONTINUOUS_INTEGRATION = env.bool('CONTINUOUS_INTEGRATION', default=False)
+
 # GENERAL
 # -----------------------------------------------------------------------------
 # https://docs.djangoproject.com/en/dev/ref/settings/#debug
@@ -284,9 +286,14 @@
 # DJANGO-COMPRESSOR
 # -----------------------------------------------------------------------------
 # https://django-compressor.readthedocs.io/en/stable/settings/
+COMPRESS_ENABLED = not CONTINUOUS_INTEGRATION
+
 COMPRESS_CSS_FILTERS = [
     'compressor.filters.css_default.CssAbsoluteFilter',
-    'compressor.filters.cssmin.CSSCompressorFilter',
+    'compressor.filters.cssmin.rCSSMinFilter',
+]
+COMPRESS_JS_FILTERS = [
+    'compressor.filters.jsmin.JSMinFilter',
 ]
 
 # MEDIA


Test patch:
Repository: carpentries/amy

Issue:
Narrow drop down in assigning admin to event
When a user tries to assign an event from the event view or request view, the drop down is super narrow.  Edit - I think this happens when no admin has yet been assigned; drop down is wide enough if a user has been assigned and you try to change it.

![image](https://user-images.githubusercontent.com/829690/65254408-41945580-daca-11e9-9472-b1547f5609e6.png)



Discussion:

Resolving patch:
diff --git a/amy/workshops/forms.py b/amy/workshops/forms.py
--- a/amy/workshops/forms.py
+++ b/amy/workshops/forms.py
@@ -873,7 +873,10 @@ class AdminLookupForm(forms.Form):
         label='Administrator',
         required=True,
         queryset=Person.objects.all(),
-        widget=ModelSelect2Widget(data_view='admin-lookup')
+        widget=ModelSelect2Widget(
+            data_view='admin-lookup',
+            attrs=SELECT2_SIDEBAR,
+        ),
     )
 
     helper = BootstrapHelper(add_cancel_button=False)


Test patch:
Repository: carpentries/amy

Issue:
"Results could not be loaded" when trying to assign admin to event or request
When trying to assign an admin to an event, the drop down does not load, and instead it says "The results could not be loaded"

This happens inconsistently.  Trying again will sometimes make the drop down list load.



![image](https://user-images.githubusercontent.com/829690/65254768-ce3f1380-daca-11e9-93f9-2067326b42f3.png)



Discussion:
Does this only happen to assign admin window?
Looks like this is happening in other views where you try to look up a name.  For example, assigning admin on the event view itself:

![image](https://user-images.githubusercontent.com/829690/65328417-474e7180-db84-11e9-8078-e9baf34a36cd.png)

And assigning tasks in an event:

![image](https://user-images.githubusercontent.com/829690/65328356-24bc5880-db84-11e9-9521-a1a1bba5f8c9.png)
Comments from @Talishask  

AMY issue tied to the latest update - Hey @maneesha, I have been experiencing issues with name recognition when adding information into AMY. This issue seems to be pretty inconsistent in what it looks like, however this does have a significant impact on my ability to enter workshops. I have mainly been observing:
- it is is taking AMY longer than usual to pull up matches
- sometimes it doesn't pull up anything at all when I know there is a match in the system.
- Once you backspace in the input field it has an even more difficult time matching.
- There is a weird phenomenon when the matching works great just as it did before however only on even characters. So, if for example I am trying to pull up 'Talisha Sutton-Kennedy" the field would only show matches on the bolded characters(spaces included): T **a** l **i** s **h** a **_** S **u** t **t** o **n** - **K** e **n** n **e** d **y** 
For now it looks like the erratic behavior came from django-select2 using cache database that we didn't explicitly created.

For tests I added such cache table on production server (that's only place where the error would occur) and it seems fine now. I'm waiting for more tests from @Talishask and @maneesha.

Resolving patch:
diff --git a/config/settings.py b/config/settings.py
--- a/config/settings.py
+++ b/config/settings.py
@@ -247,6 +247,20 @@
     },
 ]
 
+# CACHE
+# -----------------------------------------------------------------------------
+# https://docs.djangoproject.com/en/2.2/topics/cache/#database-caching
+CACHES = {
+    'default': {
+        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
+        'LOCATION': 'cache_default',
+    },
+    'select2': {
+        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
+        'LOCATION': 'cache_select2',
+    }
+}
+
 # MIDDLEWARE
 # -----------------------------------------------------------------------------
 # https://docs.djangoproject.com/en/dev/ref/settings/#middleware
@@ -521,3 +535,4 @@
 SELECT2_JS = ''  # don't load JS on it's own - we're loading it in `base.html`
 SELECT2_CSS = ''  # the same for CSS
 SELECT2_I18N = 'select2/js/i18n'
+SELECT2_CACHE_BACKEND = 'select2'


Test patch:
Repository: carpentries/amy

Issue:
"Forgotten" fields in Workshop Request -> Self Organised Submission migration
These fields aren't migrated:
* `created_at`
* `last_updated_at`
* `state`
* `country`
* `assigned_to`
* `event`

https://github.com/carpentries/amy/blob/develop/amy/extrequests/migrations/0006_migrate_workshop_requests_to_selforg_submissions.py


Discussion:

Resolving patch:
diff --git a/amy/extrequests/migrations/0006_migrate_workshop_requests_to_selforg_submissions.py b/amy/extrequests/migrations/0006_migrate_workshop_requests_to_selforg_submissions.py
--- a/amy/extrequests/migrations/0006_migrate_workshop_requests_to_selforg_submissions.py
+++ b/amy/extrequests/migrations/0006_migrate_workshop_requests_to_selforg_submissions.py
@@ -14,6 +14,12 @@ def migrate(apps, schema_editor):
 
     for wr in self_org_wr:
         sos = SelfOrganizedSubmission.objects.create(
+            created_at=wr.created_at,
+            last_updated_at=wr.last_updated_at,
+            state=wr.state,
+            assigned_to=wr.assigned_to,
+            event=wr.event,
+
             personal=wr.personal,
             family=wr.family,
             email=wr.email,
@@ -27,6 +33,7 @@ def migrate(apps, schema_editor):
             workshop_url=wr.self_organized_github,
             workshop_types_other='',
             workshop_types_other_explain='',
+            country=wr.country,
             language=wr.language,
 
             public_event=wr.public_event,


Test patch:
Repository: carpentries/amy

Issue:
Event form: improper validation of curricula and selected tags
`EventForm.clean_curricula` does try to validate selected curricula (corresponding tags must exist), but doesn't work with Mix & Match curriculum.


Discussion:

Resolving patch:
diff --git a/amy/workshops/forms.py b/amy/workshops/forms.py
--- a/amy/workshops/forms.py
+++ b/amy/workshops/forms.py
@@ -557,10 +557,12 @@ def clean_curricula(self):
         tags = self.cleaned_data['tags']
 
         try:
-            expected_tags = [
-                c.slug.split("-")[0].upper() for c in curricula
-                if c.active and not c.unknown
-            ]
+            expected_tags = []
+            for c in curricula:
+                if c.active and c.carpentry:
+                    expected_tags.append(c.carpentry)
+                elif c.active and c.mix_match:
+                    expected_tags.append('Circuits')
         except (ValueError, TypeError):
             expected_tags = []
 


Test patch:
diff --git a/amy/workshops/tests/test_event.py b/amy/workshops/tests/test_event.py
--- a/amy/workshops/tests/test_event.py
+++ b/amy/workshops/tests/test_event.py
@@ -737,6 +737,34 @@ def test_curricula_and_tags_validation(self):
         form = EventForm(data)
         self.assertNotIn('curricula', form.errors)
 
+    def test_curricula_circuits_tag(self):
+        """Ensure validation of `curricula` and `tags` fields."""
+        # missing tags
+        data = {
+            'slug': '2018-10-28-curriculum',
+            'host': self.org_alpha.pk,
+            # there has to be some tag
+            'tags': [Tag.objects.get(name='DC').pk],
+            'curricula': [
+                Curriculum.objects.get(slug='swc-python').pk,
+                Curriculum.objects.get(mix_match=True).pk,
+            ],
+        }
+        form = EventForm(data)
+        # we're missing SWC and Circuits
+        self.assertIn('curricula', form.errors)
+
+        # try adding SWC tag
+        data['tags'].append(Tag.objects.get(name='SWC').pk)
+        form = EventForm(data)
+        self.assertIn('curricula', form.errors)
+        # now we're missing only circuits
+
+        # try adding Circuits tag
+        data['tags'].append(Tag.objects.get(name='Circuits').pk)
+        form = EventForm(data)
+        self.assertNotIn('curricula', form.errors)
+
 
 class TestEventMerging(TestBase):
     def setUp(self):
Repository: carpentries/amy

Issue:
Wrong permission name for `WorkshopInquiry*` views
* `WorkshopInquiryChange`
* `WorkshopInquirySetState`
* `workshopinquiry_accept_event`

... all use wrong permission name (`extrequests.change_workshopinquiry` instead of `extrequests.change_workshopinquiryrequest`).


Discussion:

Resolving patch:
diff --git a/amy/extrequests/views.py b/amy/extrequests/views.py
--- a/amy/extrequests/views.py
+++ b/amy/extrequests/views.py
@@ -236,7 +236,7 @@ def get_context_data(self, **kwargs):
 
 class WorkshopInquiryChange(OnlyForAdminsMixin, PermissionRequiredMixin,
                             AMYUpdateView):
-    permission_required = 'extrequests.change_workshopinquiry'
+    permission_required = 'extrequests.change_workshopinquiryrequest'
     model = WorkshopInquiryRequest
     pk_url_kwarg = 'inquiry_id'
     form_class = WorkshopInquiryRequestAdminForm
@@ -244,7 +244,7 @@ class WorkshopInquiryChange(OnlyForAdminsMixin, PermissionRequiredMixin,
 
 
 class WorkshopInquirySetState(OnlyForAdminsMixin, ChangeRequestStateView):
-    permission_required = 'extrequests.change_workshopinquiry'
+    permission_required = 'extrequests.change_workshopinquiryrequest'
     model = WorkshopInquiryRequest
     pk_url_kwarg = 'inquiry_id'
     state_url_kwarg = 'state'
@@ -252,7 +252,7 @@ class WorkshopInquirySetState(OnlyForAdminsMixin, ChangeRequestStateView):
 
 
 @admin_required
-@permission_required(['extrequests.change_workshopinquiry',
+@permission_required(['extrequests.change_workshopinquiryrequest',
                       'workshops.add_event'],
                      raise_exception=True)
 def workshopinquiry_accept_event(request, inquiry_id):


Test patch:
Repository: carpentries/amy

Issue:
Database is locked: queries sent massively by Django-Compressor (cache)
For some reason, Django-Compressor sends out over 100 queries for each request, all of them check database cache table. This probably renders AMY unstable for multiple concurrent requests from users.


Discussion:
Loading just the dashboard when compression is enabled:

![obraz](https://user-images.githubusercontent.com/72821/71588940-8dbc5080-2b23-11ea-89b2-d96515524674.png)


Resolving patch:
diff --git a/config/settings.py b/config/settings.py
--- a/config/settings.py
+++ b/config/settings.py
@@ -258,6 +258,12 @@
     'select2': {
         'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
         'LOCATION': 'cache_select2',
+    },
+    'compressor': {
+        # TODO: in future, either switch to offline compression or
+        #       install Memcached
+        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
+        # 'LOCATION': 'templates-media',  # required if >1 LocMemCache used
     }
 }
 
@@ -301,13 +307,23 @@
 # https://django-compressor.readthedocs.io/en/stable/settings/
 COMPRESS_ENABLED = not CONTINUOUS_INTEGRATION
 
-COMPRESS_CSS_FILTERS = [
-    'compressor.filters.css_default.CssAbsoluteFilter',
-    'compressor.filters.cssmin.rCSSMinFilter',
-]
-COMPRESS_JS_FILTERS = [
-    'compressor.filters.jsmin.JSMinFilter',
-]
+COMPRESS_FILTERS = {
+    'css': [
+        'compressor.filters.css_default.CssAbsoluteFilter',
+        'compressor.filters.cssmin.rCSSMinFilter',
+    ],
+    'js': [
+        'compressor.filters.jsmin.JSMinFilter',
+    ]
+}
+
+# DB-backend cache is very slow when we're using SQLite. Therefore it is
+# advised to use a different cache, like MemCached. By default, AMY sets
+# LocMemCache for Django-Compressor. It has some drawbacks, biggest one
+# this cache being available only for one process. Therefore, for more
+# processes Django-Compressor will start regenerating the cache.
+# TODO: switch to Redis / Memcached once they're in place.
+COMPRESS_CACHE_BACKEND = 'compressor'
 
 # MEDIA
 # -----------------------------------------------------------------------------


Test patch:
Repository: carpentries/amy

Issue:
500 Server error when 'manual attendance' field is not set
Having 'Manual attendance' field unset causes a 500 Server Error. I tried leaving it unset for workshops where the attendance number is not known yet. It defaults to '0' - not sure if this is the desired behaviour (to have attendance at 0). If yes, disregard the issue.


Discussion:
Thanks @anenadic for reporting this. I saw bug emails from AMY but didn't know what caused them.

In this case, this is a bug, because the form should always require numeric manual attendance, event if it's 0 by default.

// EDIT: note: suggested solution is to either change `blank=True` to `blank=False` in `Event.manual_attendance` or add `required=True` for `EventForm`'s field. The former seems better.

Resolving patch:
diff --git a/amy/workshops/forms.py b/amy/workshops/forms.py
--- a/amy/workshops/forms.py
+++ b/amy/workshops/forms.py
@@ -575,6 +575,11 @@ def clean_curricula(self):
 
         return curricula
 
+    def clean_manual_attendance(self):
+        """Regression: #1608 - fix 500 server error when field is cleared."""
+        manual_attendance = self.cleaned_data['manual_attendance'] or 0
+        return manual_attendance
+
     def save(self, *args, **kwargs):
         res = super().save(*args, **kwargs)
 


Test patch:
diff --git a/amy/workshops/tests/test_event.py b/amy/workshops/tests/test_event.py
--- a/amy/workshops/tests/test_event.py
+++ b/amy/workshops/tests/test_event.py
@@ -567,6 +567,24 @@ def test_negative_manual_attendance(self):
         f = EventForm(data)
         self.assertTrue(f.is_valid())
 
+    def test_empty_manual_attendance(self):
+        """Ensure we don't get 500 server error when field is left with empty
+        value.
+
+        This is a regression test for
+        https://github.com/swcarpentry/amy/issues/1608."""
+
+        data = {
+            'slug': '2016-06-30-test-event',
+            'host': self.test_host.id,
+            'tags': [self.test_tag.id],
+            'manual_attendance': '',
+        }
+        f = EventForm(data)
+        self.assertTrue(f.is_valid())
+        event = f.save()
+        self.assertEqual(event.manual_attendance, 0)
+
     def test_number_of_attendees_increasing(self):
         """Ensure event.attendance gets bigger after adding new learners."""
         event = Event.objects.get(slug='test_event_0')