internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: carpentries/amy

Issue:
Strip spaces from personal/middle/family names during import
I just removed leading and trailing spaces from the personal, middle, and family names of about 50 people. My guess is that these came in during import from CSV; we should strip these. We should also check that multiple internal spaces are compressed to one (e.g., "Luc  Andre" becomes "Luc Andre" with only one internal space).



Discussion:
But there is a `strip` in the code: https://github.com/swcarpentry/amy/blob/develop/workshops/util.py#L85

Hm - there were a bunch of names in the database with leading and 
trailing spaces, which I've now removed by hand.

Were these spaces in consecutive records?

Mmm... don't know.  Can you pull up the change log from the AMY database 
and look for all the people I touched this afternoon?

Looking at https://amy.software-carpentry.org/workshops/log/, for example last 4 people you changed:
- https://amy.software-carpentry.org/workshops/person/10383
- https://amy.software-carpentry.org/workshops/person/10382
- https://amy.software-carpentry.org/workshops/person/10352
- https://amy.software-carpentry.org/workshops/person/10351

It seems that IDs are relatively close to each other, but looking at creation timestamp I wouldn't say they were introduced with Bulk Upload feature.

Maybe @maneesha can comment on that since she entered these people in?

Cheers


Resolving patch:
diff --git a/workshops/models.py b/workshops/models.py
--- a/workshops/models.py
+++ b/workshops/models.py
@@ -194,8 +194,10 @@ def is_staff(self):
     def save(self, *args, **kwargs):
         # save empty string as NULL to the database - otherwise there are
         # issues with UNIQUE constraint failing
-        self.middle = self.middle or None
-        self.email = self.email or None
+        self.personal = self.personal.strip()
+        self.family = self.family.strip()
+        self.middle = self.middle.strip() if self.middle else None
+        self.email = self.email.strip() if self.email else None
         self.gender = self.gender or None
         self.airport = self.airport or None
         self.github = self.github or None
@@ -325,6 +327,9 @@ class ProfileUpdateRequest(models.Model):
 
     def save(self, *args, **kwargs):
         """Save nullable char fields as empty strings."""
+        self.personal = self.personal.strip()
+        self.family = self.family.strip()
+        self.email = self.email.strip()
         self.gender = self.gender or ''
         self.occupation = self.occupation or ''
         return super().save(*args, **kwargs)


Test patch:
Repository: carpentries/amy

Issue:
List of people who taught at events of specific type
Usecase: Tracy wants to grab list of people who taught at DC workshops, so that she knows who is experienced DC instructor.



Discussion:
Filter the "Events" display by tag?

@gvwilson and you'll end up with list of events, not instructors.

> @gvwilson and you'll end up with list of events, not instructors.

But we can get the instructors from those events, right?

@maneesha yes, but it requires you to open each workshop and look at the list of tasks.

Right - we'll probably need to filter people by award ("has instructor 
badge") and then by properties of associated events.

@gvwilson right now it's possible to filter people by their badges on All Persons page. I'm going to add "filter by type of workshop where person taught", which is not the same - some people have 'instructor' task for event but are actually missing the badge.


Resolving patch:
diff --git a/workshops/filters.py b/workshops/filters.py
--- a/workshops/filters.py
+++ b/workshops/filters.py
@@ -5,7 +5,17 @@
 import django_filters
 from django_countries import Countries
 
-from workshops.models import Event, Host, Person, Task, Airport, EventRequest
+from workshops.models import (
+    Event,
+    Host,
+    Person,
+    Task,
+    Airport,
+    EventRequest,
+    Tag,
+    Role,
+    Task,
+)
 
 EMPTY_SELECTION = (None, '---------')
 
@@ -124,11 +134,31 @@ class Meta:
         order_by = ['fullname', '-fullname', 'domain', '-domain', ]
 
 
+def filter_taught_workshops(queryset, values):
+    """Limit Persons to only instructors from events with specific tags.
+
+    This needs to be in a separate function because django-filters doesn't
+    support `action` parameter as supposed, ie. with
+    `action='filter_taught_workshops'` it doesn't call the method; instead it
+    tries calling a string, which results in error."""
+    if not values:
+        return queryset
+
+    return queryset.filter(task__role__name='instructor') \
+                   .filter(task__event__tags__in=values) \
+                   .distinct()
+
+
 class PersonFilter(django_filters.FilterSet):
+    taught_workshops = django_filters.ModelMultipleChoiceFilter(
+        queryset=Tag.objects.all(), label='Taught at workshops of type',
+        action=filter_taught_workshops,
+    )
+
     class Meta:
         model = Person
         fields = [
-            'badges',
+            'badges', 'taught_workshops',
         ]
         order_by = ["lastname", "-lastname", "firstname", "-firstname",
                     "email", "-email"]


Test patch:
Repository: carpentries/amy

Issue:
Remove slider widget from attendance count & invoice amount
Just let it accept a inputted number.
I've never used it, and actually had it cause problems when my scroll wheel activates it and changes the value.



Discussion:
@maneesha Can you show me a screenshot? I have never seen such thing in AMY, so it may be browser-dependend.

Sorry... meant "spinner" not "slider"  

![image](https://cloud.githubusercontent.com/assets/829690/10823404/fab4fe24-7e31-11e5-991b-5e11673d6ca1.png)

@maneesha is this still an issue?

Unfortunately & probably, it's not possible to switch only one field to non-HTML5 mode – I think it's "all or nothing". But I'll check.

@maneesha ping on this issue

See the image in my comment on Oct 29.  The spinner (up/down arrows) mean that if I hit my mouse scroll wheel while in that field, I change the value. 
I'd like to disable the spinner so the only way to enter a value in any numeric field (attendance, admin fee, etc) is to key it in.


Resolving patch:
diff --git a/workshops/forms.py b/workshops/forms.py
--- a/workshops/forms.py
+++ b/workshops/forms.py
@@ -2,7 +2,7 @@
 
 from django import forms
 from django.core.validators import RegexValidator
-from django.forms import HiddenInput, CheckboxSelectMultiple
+from django.forms import HiddenInput, CheckboxSelectMultiple, TextInput
 
 from captcha.fields import ReCaptchaField
 from crispy_forms.helper import FormHelper
@@ -258,7 +258,7 @@ class EventForm(forms.ModelForm):
     )
 
     admin_fee = forms.DecimalField(min_value=0, decimal_places=2,
-                                   required=False)
+                                   required=False, widget=TextInput)
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
@@ -311,6 +311,12 @@ class Meta:
         # WARNING: don't change put any fields between 'country' and
         #          'longitude' that don't relate to the venue of the event
 
+        widgets = {
+            'attendance': TextInput,
+            'latitude': TextInput,
+            'longitude': TextInput,
+        }
+
     class Media:
         # thanks to this, {{ form.media }} in the template will generate
         # a <link href=""> (for CSS files) or <script src=""> (for JS files)


Test patch:
Repository: carpentries/amy

Issue:
API: don't return todos with unknown start
This breaks the timeline.



Discussion:
ref #618 


Resolving patch:
diff --git a/api/views.py b/api/views.py
--- a/api/views.py
+++ b/api/views.py
@@ -162,4 +162,5 @@ def get_queryset(self):
         """Return current TODOs for currently logged in user."""
         return TodoItem.objects.user(self.request.user) \
                                .incomplete() \
+                               .exclude(due=None) \
                                .select_related('event')


Test patch:
Repository: carpentries/amy

Issue:
Remove instructor training from debriefing list



Discussion:

Resolving patch:
diff --git a/workshops/views.py b/workshops/views.py
--- a/workshops/views.py
+++ b/workshops/views.py
@@ -1483,6 +1483,8 @@ def debrief(request):
 
     start_date = end_date = None
 
+    tags = Tag.objects.filter(name__in=['stalled', 'unresponsive'])
+
     if request.method == 'POST':
         form = DebriefForm(request.POST)
         if form.is_valid():
@@ -1493,7 +1495,8 @@ def debrief(request):
                 event__end__lte=end_date,
                 role__name='instructor',
                 person__may_contact=True,
-            ).order_by('event', 'person', 'role')
+            ).exclude(event__tags=tags).order_by('event', 'person', 'role') \
+             .select_related('person', 'event', 'role')
 
     else:
         # if a GET (or any other method) we'll create a blank form


Test patch:
Repository: carpentries/amy

Issue:
Survey links field should be Survey results 
Rename "Survey links" to "Survey results"

Survey _links_ are automatically generated using workshop slug.
The _results_ are what we need to store because those have to be manually generated.



Discussion:

Resolving patch:
diff --git a/workshops/forms.py b/workshops/forms.py
--- a/workshops/forms.py
+++ b/workshops/forms.py
@@ -291,7 +291,7 @@ def __init__(self, *args, **kwargs):
             Div, css_class='panel panel-default')
         # add <div class='panel-heading'>Venue details</div> inside "div.panel"
         self.helper.layout[id_learners_pre].insert(
-            0, Div(HTML('Survey links'), css_class='panel-heading'))
+            0, Div(HTML('Survey results'), css_class='panel-heading'))
 
     def clean_slug(self):
         # Ensure slug is not an integer value for Event.get_by_ident


Test patch:
Repository: carpentries/amy

Issue:
Consistent language for invoicing options 
Change 
Not invoiced for historical reasons 
and 
Not invoiced because of membership 

to:

Not applicable for historical reasons 
and 
Not applicable because of membership 



Discussion:

Resolving patch:
diff --git a/workshops/migrations/0073_auto_20160127_0944.py b/workshops/migrations/0073_auto_20160127_0944.py
new file mode 100644
--- /dev/null
+++ b/workshops/migrations/0073_auto_20160127_0944.py
@@ -0,0 +1,35 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import models, migrations
+import django.core.validators
+
+
+def switch_invoice_ni_to_na(apps, schema_editor):
+    """Switch 'not invoiced' to 'not applicable' in events' invoice status."""
+    Event = apps.get_model('workshops', 'Event')
+    Event.objects.filter(invoice_status='ni-historic') \
+                 .update(invoice_status='na-historic')
+    Event.objects.filter(invoice_status='ni-member') \
+                 .update(invoice_status='na-member')
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('workshops', '0072_underscore_usernames_fixed_migration'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='event',
+            name='invoice_status',
+            field=models.CharField(choices=[('unknown', 'Unknown'), ('invoiced', 'Invoiced'), ('not-invoiced', 'Not invoiced'), ('na-historic', 'Not applicable for historical reasons'), ('na-member', 'Not applicable because of membership'), ('na-self-org', 'Not applicable because self-organized'), ('na-waiver', 'Not applicable because waiver granted'), ('na-other', 'Not applicable because other arrangements made')], blank=True, max_length=40, verbose_name='Invoice status', default='unknown'),
+        ),
+        migrations.AlterField(
+            model_name='person',
+            name='username',
+            field=models.CharField(validators=[django.core.validators.RegexValidator('^[\\w\\-_]+$', flags=256)], unique=True, max_length=40),
+        ),
+        migrations.RunPython(switch_invoice_ni_to_na),
+    ]
diff --git a/workshops/models.py b/workshops/models.py
--- a/workshops/models.py
+++ b/workshops/models.py
@@ -608,8 +608,8 @@ class Event(AssignmentMixin, models.Model):
         ('unknown', 'Unknown'),
         ('invoiced', 'Invoiced'),
         ('not-invoiced', 'Not invoiced'),
-        ('ni-historic', 'Not invoiced for historical reasons'),
-        ('ni-member', 'Not invoiced because of membership'),
+        ('na-historic', 'Not applicable for historical reasons'),
+        ('na-member', 'Not applicable because of membership'),
         ('na-self-org', 'Not applicable because self-organized'),
         ('na-waiver', 'Not applicable because waiver granted'),
         ('na-other', 'Not applicable because other arrangements made'),


Test patch:
Repository: carpentries/amy

Issue:
Assets management
Usually AMY has some changes to CSS from time to time, and also the dependencies change, but I always forget to change the links in order to refresh browser cache.

The trick is to use `?something` (e.g. `?v=4`) at the end of asset link:

``` html
<link rel="stylesheet" href="/static/amy.css?v=4" />
```

And after each update to the assets, just change the query to something unique (e.g. `?v=5`).

The problem with this approach lies on the human side: I simply forget to bump the `?v` param.
Probably there's an application/plugin for Django that:
- compresses / concatenates all the assets (AMY dashboard has 11 assets, we'd get down to 2: one CSS, one JS)
- adds a single link to template sources with query param generated automatically.



Discussion:

Resolving patch:
diff --git a/amy/settings.py b/amy/settings.py
--- a/amy/settings.py
+++ b/amy/settings.py
@@ -131,6 +131,7 @@
     'rest_framework',
     'api',
     'captcha',
+    'compressor',
 )
 
 CRISPY_TEMPLATE_PACK = 'bootstrap3'
@@ -216,6 +217,11 @@
 STATICFILES_DIRS = (
     os.path.join(BASE_DIR, 'bower_components'),
 )
+STATICFILES_FINDERS = [
+    'django.contrib.staticfiles.finders.FileSystemFinder',
+    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
+    'compressor.finders.CompressorFinder',
+]
 
 # if "next" (or "?next") variable is not set when logging in, redirect to
 # workshops


Test patch:
Repository: carpentries/amy

Issue:
"Export badges" should include award dates in data.



Discussion:

Resolving patch:
diff --git a/api/serializers.py b/api/serializers.py
--- a/api/serializers.py
+++ b/api/serializers.py
@@ -13,6 +13,15 @@
 )
 
 
+class AwardPersonSerializer(serializers.ModelSerializer):
+    name = serializers.CharField(source='person.get_full_name')
+    user = serializers.CharField(source='person.username')
+
+    class Meta:
+        model = Award
+        fields = ('name', 'user', 'awarded')
+
+
 class PersonUsernameSerializer(serializers.ModelSerializer):
     name = serializers.CharField(source='get_full_name')
     user = serializers.CharField(source='username')
@@ -31,7 +40,7 @@ class Meta:
 
 
 class ExportBadgesSerializer(serializers.ModelSerializer):
-    persons = PersonUsernameSerializer(many=True, source='person_set')
+    persons = AwardPersonSerializer(many=True, source='award_set')
 
     class Meta:
         model = Badge
diff --git a/api/views.py b/api/views.py
--- a/api/views.py
+++ b/api/views.py
@@ -110,7 +110,7 @@ class ExportBadgesView(ListAPIView):
     permission_classes = (IsAuthenticatedOrReadOnly, )
     paginator = None  # disable pagination
 
-    queryset = Badge.objects.prefetch_related('person_set')
+    queryset = Badge.objects.prefetch_related('award_set')
     serializer_class = ExportBadgesSerializer
 
 


Test patch:
diff --git a/api/test/test_export.py b/api/test/test_export.py
--- a/api/test/test_export.py
+++ b/api/test/test_export.py
@@ -39,6 +39,8 @@ class TestExportingBadges(BaseExportingTest):
     def setUp(self):
         super().setUp()
 
+        today = datetime.date.today()
+
         # set up two badges, one with users, one without any
         self.badge1 = Badge.objects.create(name='badge1', title='Badge1',
                                            criteria='')
@@ -49,18 +51,20 @@ def setUp(self):
         self.user2 = Person.objects.create_user('user2', 'User2', 'Name',
                                                 'user2@name.org')
         Award.objects.create(person=self.user1, badge=self.badge1,
-                             awarded=datetime.date.today())
+                             awarded=today)
         Award.objects.create(person=self.user2, badge=self.badge1,
-                             awarded=datetime.date.today())
+                             awarded=today)
 
         # make sure we *do* get empty badges
         self.expecting = [
             {
                 'name': 'badge1',
                 'persons': [
-                    {'name': 'User1 Name', 'user': 'user1'},
-                    {'name': 'User2 Name', 'user': 'user2'},
-                ]
+                    {'name': 'User1 Name', 'user': 'user1',
+                     'awarded': '{:%Y-%m-%d}'.format(today)},
+                    {'name': 'User2 Name', 'user': 'user2',
+                     'awarded': '{:%Y-%m-%d}'.format(today)},
+                ],
             },
             {
                 'name': 'badge2',
Repository: carpentries/amy

Issue:
Add CSV support for API
All API calls that deliver JSON and YAML should also deliver CSV.



Discussion:
https://github.com/mjumbewu/django-rest-framework-csv


Resolving patch:
diff --git a/api/views.py b/api/views.py
--- a/api/views.py
+++ b/api/views.py
@@ -14,8 +14,10 @@
 from rest_framework.renderers import BrowsableAPIRenderer, JSONRenderer
 from rest_framework.response import Response
 from rest_framework.reverse import reverse
+from rest_framework.settings import api_settings
 from rest_framework.views import APIView
 from rest_framework.viewsets import ViewSet
+from rest_framework_csv.renderers import CSVRenderer
 
 from workshops.models import (
     Badge,
@@ -129,6 +131,8 @@ class ExportMembersView(ListAPIView):
     permission_classes = (IsAuthenticated, )
     paginator = None  # disable pagination
 
+    renderer_classes = api_settings.DEFAULT_RENDERER_CLASSES + [CSVRenderer, ]
+
     serializer_class = PersonNameEmailUsernameSerializer
 
     def get_queryset(self):


Test patch:
Repository: carpentries/amy

Issue:
No reverse match for rest_framework namespace
The error for a very strange reason shows when accessing these URLs:
https://github.com/swcarpentry/amy/blob/develop/api/urls.py#L57

I wasn't able to get rid of it; it's not being used at all, so maybe it should be removed…?



Discussion:
No idea...


Resolving patch:
diff --git a/api/urls.py b/api/urls.py
--- a/api/urls.py
+++ b/api/urls.py
@@ -52,10 +52,4 @@
     url('^', include(todos_router.urls)),
 ]
 
-# for login-logout functionality
-urlpatterns += [
-    url(r'^api-auth/',
-        include('rest_framework.urls', namespace='rest_framework')),
-]
-
 urlpatterns = format_suffix_patterns(urlpatterns)  # allow to specify format


Test patch:
Repository: carpentries/amy

Issue:
Tokenize person lookup to work with "name last_name"
`lookups.PersonLookup` won't show 'Piotr Banaszkiewicz' for 'Piotr Ban' input – because it only looks up `personal` or `family` or `email`…



Discussion:

Resolving patch:
diff --git a/workshops/lookups.py b/workshops/lookups.py
--- a/workshops/lookups.py
+++ b/workshops/lookups.py
@@ -1,3 +1,7 @@
+from functools import reduce
+import operator
+import re
+
 from django.contrib.auth.models import Group
 from django.db.models import Q
 
@@ -33,6 +37,32 @@ class PersonLookup(ModelLookup):
         'username__icontains'
     )
 
+    def get_query(self, request, term):
+        """Override this method to allow for additional lookup method: """
+        # original code from selectable.base.ModelLookup.get_query:
+        qs = self.get_queryset()
+        if term:
+            search_filters = []
+            if self.search_fields:
+                for field in self.search_fields:
+                    search_filters.append(Q(**{field: term}))
+
+            # tokenizing part
+            tokens = re.split('\s+', term)
+            if len(tokens) == 2:
+                name1, name2 = tokens
+                complex_q = (
+                    Q(personal__icontains=name1) & Q(family__icontains=name2)
+                ) | (
+                    Q(personal__icontains=name2) & Q(family__icontains=name1)
+                )
+                search_filters.append(complex_q)
+
+            # this is brilliant: it applies OR to all search filters
+            qs = qs.filter(reduce(operator.or_, search_filters))
+
+        return qs
+
 
 @login_required
 class AdminLookup(ModelLookup):


Test patch:
Repository: carpentries/amy

Issue:
Autogenerate username
On this page
https://amy.software-carpentry.org/workshops/persons/add/

Can the username be automatically generated (just as it is for bulk upload persons)?



Discussion:

Resolving patch:
diff --git a/workshops/forms.py b/workshops/forms.py
--- a/workshops/forms.py
+++ b/workshops/forms.py
@@ -397,6 +397,14 @@ class Meta:
                   'domains']
 
 
+class PersonCreateForm(PersonForm):
+    class Meta(PersonForm.Meta):
+        # remove 'username' field as it's being populated after form save
+        # in the `views.PersonCreate.form_valid`
+        fields = PersonForm.Meta.fields
+        fields.remove('username')
+
+
 class PersonPermissionsForm(forms.ModelForm):
     class Meta:
         model = Person
diff --git a/workshops/views.py b/workshops/views.py
--- a/workshops/views.py
+++ b/workshops/views.py
@@ -62,7 +62,7 @@
     AdminLookupForm, ProfileUpdateRequestFormNoCaptcha, MembershipForm,
     TodoFormSet, EventsSelectionForm, EventsMergeForm, InvoiceRequestForm,
     InvoiceRequestUpdateForm, EventSubmitForm, EventSubmitFormNoCaptcha,
-    PersonsMergeForm,
+    PersonsMergeForm, PersonCreateForm,
 )
 from workshops.util import (
     upload_person_task_csv,  verify_upload_person_task,
@@ -80,6 +80,7 @@
     failed_to_delete,
     assign,
     merge_objects,
+    create_username,
 )
 
 from workshops.filters import (
@@ -633,7 +634,7 @@ class PersonCreate(LoginRequiredMixin, PermissionRequiredMixin,
                    CreateViewContext):
     permission_required = 'workshops.add_person'
     model = Person
-    form_class = PersonForm
+    form_class = PersonCreateForm
     template_name = 'workshops/generic_form.html'
 
     def form_valid(self, form):
@@ -643,6 +644,10 @@ def form_valid(self, form):
         See more here: http://stackoverflow.com/a/15745652"""
         self.object = form.save(commit=False)  # don't save M2M fields
 
+        self.object.username = create_username(
+            personal=form.cleaned_data['personal'],
+            family=form.cleaned_data['family'])
+
         # Need to save that object because of commit=False previously.
         # This doesn't save our troublesome M2M field.
         self.object.save()


Test patch:
diff --git a/workshops/test/test_person.py b/workshops/test/test_person.py
--- a/workshops/test/test_person.py
+++ b/workshops/test/test_person.py
@@ -348,6 +348,19 @@ def test_person_username_validation(self):
         )
         person.clean_fields(exclude=['password'])
 
+    def test_new_person_auto_username(self):
+        """Ensure after adding a new person, they're automatically assigned
+        a unique username."""
+        url = reverse('person_add')
+        data = {
+            'personal': 'Albert',
+            'family': 'Einstein',
+            'gender': 'U',
+        }
+        self.client.post(url, data)
+        Person.objects.get(personal='Albert', family='Einstein',
+                           username='einstein_albert')
+
 
 class TestPersonPassword(TestBase):
     """Separate tests for testing password setting.