Repository: DDMAL/CantusDB

Issue:
adding editors
1) All the editors to add are displaying in one long list, more-ore-less alphabetical (some weird Anonymour User accounts at the top, and lowercase/special letters after the Zs.) It might be nice if you could start typing your name instead of having to scroll past hundreds of others.

2) What does this actually do? I tried adding a few editors to a source, and it doesn't seem to display anywhere who has already been added as an editor. 


Discussion:
Where is this? I'm guessing that this is in the source edit page; In the Admin panel, there is such a searchable list:
![Screen Shot 2022-12-02 at 2 35 18 PM](https://user-images.githubusercontent.com/58090591/205373213-7e9a7fb9-b10c-4c43-9394-3ebd784816af.png)

If you add new "Current Editors" in the admin panel, I don't believe they visibly appear anywhere (if you add "Other editors", however, they should appear listed as indexers in the sidebar on the source detail page), but it should give that user access to modify the chants contained within the source (and perhaps the source description too, I can't remember off the top of my head). I think this difference is because any old user can be a "current editor", whereas "other editors" are meant to be indexers, with publicly visible user pages.

If you're indeed looking at the source edit page, I agree that it is weird that there are only two lists of users, "current editors" (which should contain all users) and "melodies entered by" (which should be indexer-specific). If this is in imitation of OldCantus, it's an odd design choice. I'm tempted to simply get rid of those fields, and perhaps provide a link to the admin panel....
Yes, I was looking at the source edit page (which doesn't seem to have "other editors" as an option.) In OldCantus there are lists of users and indexers for a great many possible roles, so it's strange that only two lists are available here; it's also strange that you have these two lists, but they don't seem to have any real function. Presumably the intent is to make it so a source creator can assign other people to work with them on something, but in that case they would probably want to know who is already assigned! Maybe the admin panel is a better solution...

Also, I thought indexers and users were being harmonized into the same role--am I missing something? Shouldn't both lists be the same?
> Also, I thought indexers and users were being harmonized into the same role--am I missing something? Shouldn't both lists be the same?

So both Indexers and Users are now represented in the database as `User` objects, but that doesn't necessarily mean they have the same "role" (depending on what you mean by "role")...

As far as I recall, users can have edit access to a source, or proofread access to a source (or neither), and this is the key variable in dictating whether or not a user can access certain edit/proofread views for chants within that source.

Users also have lists of sources they have proofread/indexed/added melodies for/etc., which is only relevant if they are marked as an indexer. This affects which sources are displayed on a user's detail page (see, e.g. http://206.12.88.113/user/613#sources [This source list is only present on staging, and not currently on production]), but not who has access to which pages.
Ok, so when you say that the list for "Added melodies" should be indexer-specific, you mean it should only show the subset of users that have already been assigned edit access to that particular source?  Or should only list the 88 people currently under /indexers, as opposed to the 223 under /users? 
As far as I understand, any user can be added as an "added melodies"er for a source, whether or not they have edit/proofread access for it. We don't expect any non-indexer users to be added as "added melodies"ers for sources, but I think that if they were, it would show up on their user detail page as one of their "added melodies" sources. I think that in the side panel on Source Detail pages, only users who are indexers will be shown as having added melodies for the source.

That said, this is just my best guess as to what NewCantus currently does - all this goes to show that there's a fair bit of ambiguity between users-as-users and users-as-indexers. Maybe we should do some thinking as to how to clearly indicate these roles and what meanings these roles have (e.g. if a user is a "current editor" for a source, that means something meaningful for permissions, but if they're an "other editor" for a source, that's just an indicator that they did some indexerly work on a source but has no implications for permissions.) (Also, come to think of it, I have no idea what we currently do with granting-a-user-access-to-proofread-chants-in-a-source vs. this-is-an-indexer-who-did-proofreading-for-the-source...)

A good first step would be me going through the code, along with testing the site, to figure out exactly what's going on and to document it. This whole issue is a bit of a can of worms now that we're digging into it, and having things clearly described will be quite useful once people start interacting in earnest with user roles and indexer labels. I'll add this to my list of tasks for after the break.
This issue has also been mentioned by Debra, who writes: 
> "Not a huge issue at the moment, but is there an easy way for me to find my name in the list of "current editors"?
>
> I'm creating a new test source, and I'm assuming the list of "Current editors" is taken from anyone who has an account. Could it be alphabetical?
I suppose it's probably more work, but could it be one of those fields that completes as you type? (I start with "D", then "De" ... and it limits my choices as I go.)"

also:

> "Also in the "Create Source" (http://206.12.88.113/source-create/), we are missing some fields for authorship / editorship.
> 
> We need all of the "Inventoried by", "Full Texts Entered by", "Proofreaders", and "Other Editors" fields." 
handling these cases via the admin panel rather than the source-create page is a possible solution to both the need to credit editors in certain roles and the need to search editors--though since I currently can't access this panel, I don't have a sense of how workable a solution it would be. 

Resolving patch:
diff --git a/django/cantusdb_project/main_app/forms.py b/django/cantusdb_project/main_app/forms.py
--- a/django/cantusdb_project/main_app/forms.py
+++ b/django/cantusdb_project/main_app/forms.py
@@ -28,6 +28,7 @@
 from django.contrib.admin.widgets import (
     FilteredSelectMultiple,
 )
+from dal import autocomplete
 
 # ModelForm allows to build a form directly from a model
 # see https://docs.djangoproject.com/en/3.0/topics/forms/modelforms/
@@ -196,6 +197,10 @@ class Meta:
             "cursus",
             "current_editors",
             "melodies_entered_by",
+            "inventoried_by",
+            "full_text_entered_by",
+            "proofreaders",
+            "other_editors",
             "complete_inventory",
             "summary",
             "description",
@@ -218,6 +223,25 @@ class Meta:
             "fragmentarium_id": TextInputWidget(),
             "dact_id": TextInputWidget(),
             "indexing_notes": TextAreaWidget(),
+            "current_editors": autocomplete.ModelSelect2Multiple(
+                url="current-editors-autocomplete"
+            ),
+            "melodies_entered_by": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "century": autocomplete.ModelSelect2Multiple(url="century-autocomplete"),
+            "inventoried_by": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "full_text_entered_by": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "proofreaders": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "other_editors": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
         }
 
     rism_siglum = forms.ModelChoiceField(
@@ -243,35 +267,6 @@ class Meta:
     full_source.widget.attrs.update(
         {"class": "form-control custom-select custom-select-sm"}
     )
-
-    century = forms.ModelMultipleChoiceField(
-        queryset=Century.objects.all().order_by("name"), required=False
-    )
-    century.widget.attrs.update(
-        {"class": "form-control custom-select custom-select-sm"}
-    )
-
-    current_editors = forms.ModelMultipleChoiceField(
-        queryset=get_user_model()
-        .objects.filter(
-            Q(groups__name="project manager")
-            | Q(groups__name="editor")
-            | Q(groups__name="contributor")
-        )
-        .order_by("last_name"),
-        required=False,
-    )
-    current_editors.widget.attrs.update(
-        {"class": "form-control custom-select custom-select-sm"}
-    )
-
-    melodies_entered_by = forms.ModelMultipleChoiceField(
-        queryset=get_user_model().objects.all().order_by("full_name"), required=False
-    )
-    melodies_entered_by.widget.attrs.update(
-        {"class": "form-control custom-select custom-select-sm"}
-    )
-
     TRUE_FALSE_CHOICES_INVEN = ((True, "Complete"), (False, "Incomplete"))
 
     complete_inventory = forms.ChoiceField(
@@ -503,8 +498,6 @@ class Meta:
             "date",
             "century",
             "cursus",
-            "current_editors",
-            "melodies_entered_by",
             "complete_inventory",
             "summary",
             "description",
@@ -513,6 +506,12 @@ class Meta:
             "fragmentarium_id",
             "dact_id",
             "indexing_notes",
+            "current_editors",
+            "melodies_entered_by",
+            "inventoried_by",
+            "full_text_entered_by",
+            "proofreaders",
+            "other_editors",
         ]
         widgets = {
             "title": TextInputWidget(),
@@ -527,6 +526,25 @@ class Meta:
             "fragmentarium_id": TextInputWidget(),
             "dact_id": TextInputWidget(),
             "indexing_notes": TextAreaWidget(),
+            "current_editors": autocomplete.ModelSelect2Multiple(
+                url="current-editors-autocomplete"
+            ),
+            "melodies_entered_by": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "century": autocomplete.ModelSelect2Multiple(url="century-autocomplete"),
+            "inventoried_by": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "full_text_entered_by": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "proofreaders": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
+            "other_editors": autocomplete.ModelSelect2Multiple(
+                url="all-users-autocomplete"
+            ),
         }
 
     provenance = forms.ModelChoiceField(
@@ -536,13 +554,6 @@ class Meta:
         {"class": "form-control custom-select custom-select-sm"}
     )  # adds styling
 
-    century = forms.ModelMultipleChoiceField(
-        queryset=Century.objects.all().order_by("name"), required=False
-    )
-    century.widget.attrs.update(
-        {"class": "form-control custom-select custom-select-sm"}
-    )
-
     CHOICES_FULL_SOURCE = (
         (None, "None"),
         (True, "Full source"),
@@ -561,27 +572,6 @@ class Meta:
     cursus = forms.ChoiceField(choices=CHOICES_CURSUS, required=False)
     cursus.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
-    current_editors = forms.ModelMultipleChoiceField(
-        queryset=get_user_model()
-        .objects.filter(
-            Q(groups__name="project manager")
-            | Q(groups__name="editor")
-            | Q(groups__name="contributor")
-        )
-        .order_by("last_name"),
-        required=False,
-    )
-    current_editors.widget.attrs.update(
-        {"class": "form-control custom-select custom-select-sm"}
-    )
-
-    melodies_entered_by = forms.ModelMultipleChoiceField(
-        queryset=get_user_model().objects.all().order_by("full_name"), required=False
-    )
-    melodies_entered_by.widget.attrs.update(
-        {"class": "form-control custom-select custom-select-sm"}
-    )
-
     CHOICES_COMPLETE_INV = (
         (True, "complete inventory"),
         (False, "partial inventory"),
diff --git a/django/cantusdb_project/main_app/urls.py b/django/cantusdb_project/main_app/urls.py
--- a/django/cantusdb_project/main_app/urls.py
+++ b/django/cantusdb_project/main_app/urls.py
@@ -63,6 +63,11 @@
     UserListView,
     UserSourceListView,
 )
+from main_app.views.views import (
+    CurrentEditorsAutocomplete,
+    AllUsersAutocomplete,
+    CenturyAutocomplete,
+)
 
 urlpatterns = [
     path("__debug__/", include(debug_toolbar.urls)),
@@ -440,6 +445,21 @@
         views.redirect_documents,
         name="redirect-how-to-manuscript-descriptions",
     ),
+    path(
+        "current-editors-autocomplete/",
+        CurrentEditorsAutocomplete.as_view(),
+        name="current-editors-autocomplete",
+    ),
+    path(
+        "all-users-autocomplete/",
+        AllUsersAutocomplete.as_view(),
+        name="all-users-autocomplete",
+    ),
+    path(
+        "century-autocomplete/",
+        CenturyAutocomplete.as_view(),
+        name="century-autocomplete",
+    ),
 ]
 
 handler404 = "main_app.views.views.handle404"
diff --git a/django/cantusdb_project/main_app/views/views.py b/django/cantusdb_project/main_app/views/views.py
--- a/django/cantusdb_project/main_app/views/views.py
+++ b/django/cantusdb_project/main_app/views/views.py
@@ -31,6 +31,8 @@
 from django.core.paginator import Paginator
 from django.templatetags.static import static
 from django.contrib.flatpages.models import FlatPage
+from dal import autocomplete
+from django.db.models import Q
 
 
 @login_required
@@ -949,3 +951,45 @@ def redirect_documents(request) -> HttpResponse:
         print(old_path)
         raise Http404
     return redirect(new_path)
+
+
+class CurrentEditorsAutocomplete(autocomplete.Select2QuerySetView):
+    def get_queryset(self):
+        if not self.request.user.is_authenticated:
+            return get_user_model().objects.none()
+        qs = (
+            get_user_model()
+            .objects.filter(
+                Q(groups__name="project manager")
+                | Q(groups__name="editor")
+                | Q(groups__name="contributor")
+            )
+            .order_by("full_name")
+        )
+        if self.q:
+            qs = qs.filter(
+                Q(full_name__istartswith=self.q) | Q(email__istartswith=self.q)
+            )
+        return qs
+
+
+class AllUsersAutocomplete(autocomplete.Select2QuerySetView):
+    def get_queryset(self):
+        if not self.request.user.is_authenticated:
+            return get_user_model().objects.none()
+        qs = get_user_model().objects.all().order_by("full_name")
+        if self.q:
+            qs = qs.filter(
+                Q(full_name__istartswith=self.q) | Q(email__istartswith=self.q)
+            )
+        return qs
+
+
+class CenturyAutocomplete(autocomplete.Select2QuerySetView):
+    def get_queryset(self):
+        if not self.request.user.is_authenticated:
+            return Century.objects.none()
+        qs = Century.objects.all().order_by("name")
+        if self.q:
+            qs = qs.filter(name__istartswith=self.q)
+        return qs


Test patch:
diff --git a/django/cantusdb_project/main_app/tests/test_views.py b/django/cantusdb_project/main_app/tests/test_views.py
--- a/django/cantusdb_project/main_app/tests/test_views.py
+++ b/django/cantusdb_project/main_app/tests/test_views.py
@@ -5355,3 +5355,64 @@ def test_chant_selected_model(self):
         )
         self.assertContains(response, "Test Chant", html=True)
         self.assertNotContains(response, "Test Source", html=True)
+
+
+class AutocompleteViewsTest(TestCase):
+    def setUp(self):
+        self.user = get_user_model().objects.create(email="test@test.com")
+        self.user.set_password("pass")
+        self.user.save()
+        self.client = Client()
+        self.client.login(email="test@test.com", password="pass")
+
+    @classmethod
+    def setUpTestData(cls):
+        Group.objects.create(name="editor")
+        u1 = get_user_model().objects.create(email="hello@test.com")
+        u2 = get_user_model().objects.create(full_name="Lucas")
+        editor = Group.objects.get(name="editor")
+        editor.user_set.add(u1)
+
+        for i in range(10):
+            make_fake_century()
+
+    def test_current_editors_autocomplete(self):
+        response = self.client.get(reverse("current-editors-autocomplete"))
+        self.assertEqual(response.status_code, 200)
+
+        data = response.json()
+        self.assertEqual(len(data["results"]), 1)
+
+    def test_all_users_autocomplete(self):
+        response = self.client.get(reverse("all-users-autocomplete"))
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(len(data["results"]), 3)
+
+        response2 = self.client.get(reverse("all-users-autocomplete"), {"q": "L"})
+        self.assertEqual(response2.status_code, 200)
+        data = response2.json()
+        self.assertEqual(len(data["results"]), 1)
+
+    def test_century_autocomplete(self):
+        response = self.client.get(reverse("century-autocomplete"))
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(len(data["results"]), 10)
+
+    def test_non_authenticated_user(self):
+        self.client.logout()
+        response = self.client.get(reverse("current-editors-autocomplete"))
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(len(data["results"]), 0)
+
+        response = self.client.get(reverse("all-users-autocomplete"))
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(len(data["results"]), 0)
+
+        response = self.client.get(reverse("century-autocomplete"))
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(len(data["results"]), 0)