internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: DDMAL/CantusDB

Issue:
source-create: full source / fragment dropdown
On the source-create page on NewCantus, the two options in the Full Source / Fragment dropdown are "Full" and "Fragment". In OldCantus, the two options are "Full source" and "Fragment or Fragmented". NewCantus should be updated to match OldCantus.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/forms.py b/django/cantusdb_project/main_app/forms.py
--- a/django/cantusdb_project/main_app/forms.py
+++ b/django/cantusdb_project/main_app/forms.py
@@ -193,8 +193,8 @@ class Meta:
     provenance.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
     TRUE_FALSE_CHOICES_SOURCE = (
-        (True, "Full"), 
-        (False, "Fragment")
+        (True, "Full source"), 
+        (False, "Fragment or Fragmented")
     )
 
     full_source = forms.ChoiceField(
@@ -393,22 +393,22 @@ class Meta:
     provenance = forms.ModelChoiceField(
         queryset=Provenance.objects.all().order_by("name"), required=False
     )
-    provenance.widget.attrs.update({"class": "form-control custom-select custom-select-sm"}) # what does this do? - this adds styling - TODO remove these comments
+    provenance.widget.attrs.update({"class": "form-control custom-select custom-select-sm"}) # adds styling
     
     century = forms.ModelMultipleChoiceField(
         queryset=Century.objects.all().order_by("name"), required = False
     )
-    century.widget.attrs.update({"class": "form-control custom-select custom-select-sm"}) # what does this do?
+    century.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
     CHOICES_FULL_SOURCE = (
         (None, "None"),
         (True, "Full source"),
-        (False, "Fragment or Fragmented"),
+        (False, "Fragment or Fragmented"),        
     )
     full_source = forms.ChoiceField(
         choices=CHOICES_FULL_SOURCE, required=False
     )
-    full_source.widget.attrs.update({"class": "form-control custom-select custom-select-sm"}) # what does this do?
+    full_source.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
     CHOICES_CURSUS = (
         (None, "None"),
@@ -418,7 +418,7 @@ class Meta:
     cursus = forms.ChoiceField(
         choices=CHOICES_CURSUS, required=False
     )
-    cursus.widget.attrs.update({"class": "form-control custom-select custom-select-sm"}) # what does this do?
+    cursus.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
     current_editors = forms.ModelMultipleChoiceField(
         queryset=get_user_model().objects.filter(
@@ -440,7 +440,7 @@ class Meta:
     complete_inventory = forms.ChoiceField(
         choices=CHOICES_COMPLETE_INV, required=False
     )
-    complete_inventory.widget.attrs.update({"class": "form-control custom-select custom-select-sm"}) # what does this do?
+    complete_inventory.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
 class SequenceEditForm(forms.ModelForm):
     class Meta:


Test patch:
Repository: DDMAL/CantusDB

Issue:
csv view: AttributeError sometimes appears
(draft - more details coming soon)

the csv view (e.g. http://206.12.93.196/csv/704571 - arrived at via the CSV export link on http://206.12.93.196/source/704571) sometimes raises an attribute error, evidently when a source does not have a `segment`.


Discussion:
A source should always have a `segment`. It is either "Cantus Database" or "Sequence Database". It's a foreign key field. In cases where a source doesn't have a `segment`, it is probably a test source that we created. 
Desired behaviour: when creating a source, assign it to "Cantus Database" by default.

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/views.py b/django/cantusdb_project/main_app/views/views.py
--- a/django/cantusdb_project/main_app/views/views.py
+++ b/django/cantusdb_project/main_app/views/views.py
@@ -167,8 +167,8 @@ def csv_export(request, source_id):
         source = Source.objects.get(id=source_id)
     except:
         raise Http404("This source does not exist")
-    # "4064" is the segment id of the sequence DB, sources in that segment has sequences instead of chants
-    if source.segment.id == 4064:
+    # "4064" is the segment id of the sequence DB, sources in that segment have sequences instead of chants
+    if source.segment and source.segment.id == 4064:
         entries = source.sequence_set.order_by("id")
     else:
         entries = source.chant_set.order_by("id").select_related(


Test patch:
Repository: DDMAL/CantusDB

Issue:
Sources should automatically have segments
From #257:

> A source should always have a segment. It is either "Cantus Database" or "Sequence Database". It's a foreign key field. In cases where a source doesn't have a segment, it is probably a test source that we created.
> Desired behaviour: when creating a source, assign it to "Cantus Database" by default.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/models/source.py b/django/cantusdb_project/main_app/models/source.py
--- a/django/cantusdb_project/main_app/models/source.py
+++ b/django/cantusdb_project/main_app/models/source.py
@@ -1,5 +1,5 @@
 from django.db import models
-from main_app.models import BaseModel
+from main_app.models import BaseModel, Segment
 from django.contrib.auth import get_user_model
 
 
@@ -109,4 +109,8 @@ def __str__(self):
         string = '{t} ({i})'.format(t=self.title, i=self.id)
         return string
 
-    
\ No newline at end of file
+    def save(self, *args, **kwargs):
+        # when creating a source, assign it to "Cantus Database" by default
+        cantus_db_segment = Segment.objects.get(name="CANTUS Database")
+        self.segment = cantus_db_segment
+        super().save(*args, **kwargs)


Test patch:
Repository: DDMAL/CantusDB

Issue:
Assign a specific user to multiple sources in Django admin
In the user-edit page in the Django admin interface, we already have a selector that allows for multi-selecting sources and assigning the user to them. We need to make the selector box wider so that the source titles are not clipped. 

This issue is related to issue #216 , the relationship between the User model and Source model should go both ways.  


Discussion:
doing a bit of poking around, there does not appear to be an easy way to adjust the width of the select box. And currently, the boxes get narrower when the window gets narrower, which is behavior we probably want to conserve I imagine. 

Suggested workaround: change the string representation of Source objects such that the source's ID precedes its name. The ID is not that long to begin with, and it's unique, which would eliminate the bunch-of-identical-"Budapest, Egyetemi Könyvtár (Univ"-incipits phenomenon currently observed. And if a user needs more information on a source, they can mouse over it to reveal its full name.
@jacobdgm this functionality is for Debras to assign sources to users, right? And you're proposing the numerical source ID, e.g. "Graz, Universitätsbibliothek, Ms. 211; Antiphonale "Codex Albensis"" becomes "653927"?
It seems to me like a Debra would have to scroll down through a whole lot of potential sources to find the ID for each one of the sources she is trying to assign, mousing over each one, or take down which ones she wants and find their more or less random numbers in the list one by one; either way it would be sort of annoying. Is the space wide enough to fit the RISM siglum (in this case A-Gu 211)?  
Currently, sources are represented along the lines of `Graz, Universitätsbibliothek, Ms. 211; Antiphonale "Codex Albensis" (653927) `. I was suggesting that we represent them along the lines of `653927 - Graz, Universitätsbibliothek, Ms. 211; Antiphonale "Codex Albensis"`

The space is definitely wide enough for a RISM siglum.

Using the RISM siglum—since it's short, and (I believe) unique—to begin the string representation seems like a much better approach. Something along the lines of `A-Gu 211: Graz, Universitätsbibliothek, Ms. 211; Antiphonale "Codex Albensis" (653927)` would give all the information anyone would need, while providing a unique identifier up front. Good suggestion!

Resolving patch:
diff --git a/django/cantusdb_project/main_app/models/source.py b/django/cantusdb_project/main_app/models/source.py
--- a/django/cantusdb_project/main_app/models/source.py
+++ b/django/cantusdb_project/main_app/models/source.py
@@ -106,7 +106,7 @@ def number_of_melodies(self) -> int:
         return self.chant_set.filter(volpiano__isnull=False).count()
 
     def __str__(self):
-        string = '{t} ({i})'.format(t=self.title, i=self.id)
+        string = '[{s}] {t} ({i})'.format(s=self.rism_siglum, t=self.title, i=self.id)
         return string
 
     def save(self, *args, **kwargs):


Test patch:
Repository: DDMAL/CantusDB

Issue:
Add "position" as a search field for chant-search
We want to be able to search for entries in the Position field. It's a specific type of chant that our user is needing to isolate, but I think with the combination of search terms already on that page plus "Position" they should be able to get what they need.

TLDR: Add `position` as a search field on the chant-search page.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -365,6 +365,7 @@ class ChantSearchView(ListView):
         ``genre``: Filters by Genre of Chant
         ``cantus_id``: Filters by the Cantus ID field of Chant
         ``mode``: Filters by mode of Chant
+        ``position``: Filters by position of chant
         ``melodies``: Filters Chant by whether or not it contains a melody in
                       Volpiano form. Valid values are "true" or "false".
         ``feast``: Filters by Feast of Chant
@@ -429,6 +430,9 @@ def get_queryset(self) -> QuerySet:
             if self.request.GET.get("mode"):
                 mode = self.request.GET.get("mode")
                 q_obj_filter &= Q(mode=mode)
+            if self.request.GET.get("position"):
+                position = self.request.GET.get("position")
+                q_obj_filter &= Q(position=position)
             if self.request.GET.get("melodies") in ["true", "false"]:
                 melodies = self.request.GET.get("melodies")
                 if melodies == "true":


Test patch:
Repository: DDMAL/CantusDB

Issue:
Missing `migrations` folder with `__init__.py` file in `articles` app



Discussion:
This file should be synced to Github, so that the `articles` app will be included automatically in the first `makemigrations` after cloning. 

Resolving patch:
diff --git a/django/cantusdb_project/articles/migrations/__init__.py b/django/cantusdb_project/articles/migrations/__init__.py
new file mode 100644


Test patch:
Repository: DDMAL/CantusDB

Issue:
Representation of Notation objects
When I recently tried to edit a source, I was presented with an error message, and found that I was missing several required fields, including this one: ![Screen Shot 2022-09-06 at 08 59 56](https://user-images.githubusercontent.com/58090591/188641630-13f87954-0f92-405f-a561-effe3075a3db.png)
Notation objects are currently pretty inscrutable. They should be represented such that at least their `name` property is visible.

Larger question: why do we have notation objects at all? Currently, the notation model has only one property: `name`. Could this information in Source objects not be more simply represented by a CharField? Is using Notation objects simply the way things were done in OldCantus? Are we using them to ensure standardization among multiple Sources?


Discussion:
Displaying the `name` property can be done by adding a `__str__()` to the model. I'm not sure why notation is a separate model either... It only has one property, true, but it was acting as a foreign key on the old Cantus I think. 

Another possible reason for keeping it a separate model is that we want a "controlled-vocabulary" in defining the notations. Currently, source creators who want to specify a notation must choose from a given list (of notation objects pre-defined by experts). If notation becomes a CharField, the source creator will manually type in the notation name, which may introduce typos or inconsistency with other sources of the same notation – the same notation may be called different things by different people. 
okay, so the current implementation is probably a combination of that's-how-it-was-done-in-OldCantus and standardization/controlled-vocabulary. Fair enough.

I'll push a change with an updated `__str__` presently.

Resolving patch:
diff --git a/django/cantusdb_project/main_app/models/notation.py b/django/cantusdb_project/main_app/models/notation.py
--- a/django/cantusdb_project/main_app/models/notation.py
+++ b/django/cantusdb_project/main_app/models/notation.py
@@ -4,3 +4,5 @@
 
 class Notation(BaseModel):
     name = models.CharField(max_length=63)
+    def __str__(self):
+        return f"{self.name} ({self.id})"
\ No newline at end of file


Test patch:
Repository: DDMAL/CantusDB

Issue:
"Sequence" disappears for value 0
For fragments, we sometimes have the "Sequence" value as 0, meaning it is an incomplete item that begins on the previous (missing) page. This doesn't show up in New Cantus though:
![image](https://user-images.githubusercontent.com/67451875/204898397-56c6be48-6512-43c0-8d7a-0a1c6275315c.png)
 cf oldCantus:
![image](https://user-images.githubusercontent.com/67451875/204898679-19ad2e72-fc67-4f22-92e1-d80798ecc37a.png)


I also think it would be better if the folio/sequence items displayed next to each other, as they do in OldCantus. Typically you need both bits of information at once.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -1071,7 +1071,7 @@ def get_initial(self):
             }
         latest_folio = latest_chant.folio if latest_chant.folio else "001r"
         latest_feast = latest_chant.feast.id if latest_chant.feast else ""
-        latest_seq = latest_chant.sequence_number if latest_chant.sequence_number else 0
+        latest_seq = latest_chant.sequence_number if latest_chant.sequence_number is not None else 0
         latest_image = latest_chant.image_link if latest_chant.image_link else ""
         return {
             "folio": latest_folio,


Test patch:
Repository: DDMAL/CantusDB

Issue:
Full Inventory Page, Bower segment: sequences out of order
discovered on staging, http://206.12.88.113/index/?source=123906. Seems to be caused by some folios not having leading zeroes. Sequences, as far as I understand, can be ordered by `s_sequence` alone, rather than by `folio` and then by `c_sequence` (sequences don't have a c_sequence - only chants do)


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -5,7 +5,7 @@
 import threading
 from django.contrib import messages
 from django.contrib.postgres.search import SearchQuery, SearchRank
-from django.db.models import F, Q, QuerySet
+from django.db.models import F, Q, QuerySet, Value
 from django.shortcuts import get_object_or_404
 from django.urls import reverse
 from django.views.generic import (
@@ -1389,9 +1389,13 @@ def get_context_data(self, **kwargs):
 
         # 4064 is the id for the sequence database
         if source.segment.id == 4064:
-            queryset = source.sequence_set.order_by("folio", "s_sequence")
+            queryset = source.sequence_set.annotate(
+                record_type=Value("sequence")
+            ).order_by("s_sequence")
         else:
-            queryset = source.chant_set.order_by("folio", "c_sequence")
+            queryset = source.chant_set.annotate(
+                record_type=Value("chant")
+            ).order_by("folio", "c_sequence")
 
         context["source"] = source
         context["chants"] = queryset


Test patch:
Repository: DDMAL/CantusDB

Issue:
no option to edit from "view all chants"
![image](https://user-images.githubusercontent.com/67451875/204901241-a0137979-1c29-453b-b0cb-7d6c84df3741.png)
http://206.12.93.196/chants/?source=674961

c.f. 
![image](https://user-images.githubusercontent.com/67451875/204902270-d21537f7-8921-48fc-9652-232865a3e213.png)
https://cantus.uwaterloo.ca/chants?source=123654

Note that on OldCantus an edit done from this link takes you back to "view all chants" when you are done with it, which I personally think is pretty handy.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -573,6 +573,9 @@ def get_feast_selector_options(source, folios):
         source = Source.objects.get(id=source_id)
         context["source"] = source
 
+        user = self.request.user
+        context["user_can_edit_chant"] = user_can_edit_chants_in_source(user, source)
+
         chants_in_source = source.chant_set
         if chants_in_source.count() == 0:
             # these are needed in the selectors and hyperlinks on the right side of the page
@@ -1696,8 +1699,14 @@ def form_valid(self, form):
             return super().form_invalid(form)
 
     def get_success_url(self):
-        # stay on the same page after save
-        return self.request.get_full_path()
+        # Take user back to chant list page if that is the referrer page
+        # `ref` url parameter is used to indicate referring page
+        next_url = self.request.GET.get("ref")
+        if next_url == "chant-list":
+            return self.request.POST.get("referrer")
+        else:
+            # stay on the same page after save
+            return self.request.get_full_path()
 
 
 class ChantProofreadView(SourceEditChantsView):


Test patch:
Repository: DDMAL/CantusDB

Issue:
Chant Search Manuscript page not properly filtering by keyword
On Staging, filtering by keyword seems not to work for the Chant Search Manuscript page. Searching for chants that [start with](http://206.12.88.113/chant-search-ms/123660?op=starts_with&keyword=quat&office=&genre=&cantus_id=&mode=&feast=&position=&melodies=) `quat` in A-KR VI/258, we get all 1822 chants contained in the MS, few/none of which begin with the string `quat`. Searching for chants that [contain](http://206.12.88.113/chant-search-ms/123660?op=contains&keyword=quat&office=&genre=&cantus_id=&mode=&feast=&position=&melodies=) the same string, we also get all 1822 chants.

This is not a problem on Production (e.g. http://206.12.93.196/chant-search-ms/123660?op=starts_with&keyword=quat&office=&genre=&cantus_id=&mode=&feast=&position=&melodies=)


Discussion:
This needs to be fixed before we merge #549, and in order to be confident that we've not broken anything new in solving this, we need to unpick #607.

Resolving patch:
diff --git a/django/cantusdb_project/main_app/models/source.py b/django/cantusdb_project/main_app/models/source.py
--- a/django/cantusdb_project/main_app/models/source.py
+++ b/django/cantusdb_project/main_app/models/source.py
@@ -120,7 +120,7 @@ class Source(BaseModel):
     number_of_melodies = models.IntegerField(blank=True, null=True)
 
     def __str__(self):
-        string = '[{s}] {t} ({i})'.format(s=self.siglum, t=self.title, i=self.id)
+        string = "[{s}] {t} ({i})".format(s=self.siglum, t=self.title, i=self.id)
         return string
 
     def save(self, *args, **kwargs):
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -1064,7 +1064,7 @@ def get_queryset(self) -> QuerySet:
                 )
                 incipit_filter = Q(incipit__istartswith=keyword)
             keyword_filter = ms_spelling_filter | std_spelling_filter | incipit_filter
-            queryset.filter(keyword_filter)
+            queryset = queryset.filter(keyword_filter)
         # ordering with the folio string gives wrong order
         # old cantus is also not strictly ordered by folio (there are outliers)
         # so we order by id for now, which is the order that the chants are entered into the DB


Test patch:
Repository: DDMAL/CantusDB

Issue:
CI Input tool pop-up should go away
Currently after you choose a chant using the "Input tool" the window stays there, making it hard to tell if you selected the chant or not. In OldCantus it goes away after the information is in the edit form. 
CI input tool should also send text
After selecting a chant using the CI input tool, the edit form should get not only the selected Cantus ID, but also the full text from CantusIndex. Currently this is left blank, so you have to copy it from elsewhere. 


Discussion:
This is the CI Search View, correct, i.e. the one accessed via "Search ID" on the Chant Create page?

I don't immediately know how to do this, but it strikes me as something that should not be difficult to set up.
Huh - I'd expect it to populate the genre etc. as well. Something is broken here, surely!

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -1290,8 +1290,10 @@ class CISearchView(TemplateView):
 
     def get_context_data(self, **kwargs):
         MAX_PAGE_NUMBER_CI = 5
-
         context = super().get_context_data(**kwargs)
+        context["genres"] = list(
+            Genre.objects.all().order_by("name").values("id", "name")
+        )
         search_term = kwargs["search_term"]
         search_term = search_term.replace(" ", "+")  # for multiple keywords
         # Create empty list for the 3 types of info


Test patch:
diff --git a/django/cantusdb_project/main_app/tests/test_views.py b/django/cantusdb_project/main_app/tests/test_views.py
--- a/django/cantusdb_project/main_app/tests/test_views.py
+++ b/django/cantusdb_project/main_app/tests/test_views.py
@@ -3665,6 +3665,7 @@ def test_context_returned(self):
         fake_search_term = faker.word()
         response = self.client.get(f"/ci-search/{fake_search_term}")
         self.assertTrue("results" in response.context)
+        self.assertTrue("genres" in response.context)
 
 
 class CsvExportTest(TestCase):
Repository: DDMAL/CantusDB

Issue:
Chant Edit page should redirect to Chant Detail page upon submission of form under certain conditions
If the user got to the Chant Edit page by clicking on a link on the Chant Detail page, they should be redirected to the Chant Detail page when they press "Save".


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -1673,13 +1673,13 @@ def form_valid(self, form):
             return super().form_invalid(form)
 
     def get_success_url(self):
-        # Take user back to chant list page if that is the referrer page
+        # Take user back to the referring page
         # `ref` url parameter is used to indicate referring page
         next_url = self.request.GET.get("ref")
-        if next_url == "chant-list":
+        if next_url:
             return self.request.POST.get("referrer")
         else:
-            # stay on the same page after save
+            # ref not found, stay on the same page after save
             return self.request.get_full_path()
 
 


Test patch: