internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: DDMAL/CantusDB

Issue:
Something has broken in displaying concordances on the Chant Detail Page
![Screenshot 2023-05-25 at 10 36 26 AM](https://github.com/DDMAL/CantusDB/assets/58090591/952960a3-73ee-40e8-93cd-a982b721e295)

This is on the `develop` branch


Discussion:
For what it's worth the concordances tool doesn't work on OldCantus right now either (it has an 
under maintenance" message and a somewhat frustrating link to the Index that doesn't really lead to that chant's ID page e.g. https://cantusindex.org/ci/006129?c=1) .
This seems to be a problem just on our `develop` branch - it was only occurring locally, and when I switched to the `staging` branch, things started working fine again...

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -353,7 +353,7 @@ def get_context_data(self, **kwargs):
                 },
             ]
             context["concordances_databases"] = [
-                d["results_count"]
+                d
                 for d in concordances_databases
                 if "results_count" in d and d["results_count"] > 0
             ]


Test patch:
Repository: DDMAL/CantusDB

Issue:
edit chant--fields out of order
On OldCantus all the fields on a chant's edit page are in the same order as they are when viewing the chant. 
![image](https://user-images.githubusercontent.com/67451875/211906941-ee0a6f87-7612-4b96-a8f0-40366c0c2265.png)


Not so in NewCantus, where the text is above the fields with the folio, sequence, etc. 

![image](https://user-images.githubusercontent.com/67451875/211906385-fb9b9ee9-442c-479d-84ff-ee7020a81f47.png)



Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/forms.py b/django/cantusdb_project/main_app/forms.py
--- a/django/cantusdb_project/main_app/forms.py
+++ b/django/cantusdb_project/main_app/forms.py
@@ -277,6 +277,7 @@ class Meta:
             "extra",
             "image_link",
             "indexing_notes",
+            "addendum",
         ]
         widgets = {
             # manuscript_full_text_std_spelling: defined below (required)
@@ -298,6 +299,7 @@ class Meta:
             "extra": TextInputWidget(),
             "image_link": TextInputWidget(),
             "indexing_notes": TextAreaWidget(),
+            "addendum": TextInputWidget(),
         }
 
     manuscript_full_text_std_spelling = forms.CharField(


Test patch:
Repository: DDMAL/CantusDB

Issue:
chants on chant search page out of order
yes, it's a source to which chants have been added haphazardly, however it's not clear as to why 004r should be before 001r on http://206.12.88.113/chants/?source=699578 ...

![Screen Shot 2023-01-13 at 10 51 43 AM](https://user-images.githubusercontent.com/58090591/212361890-7459dc3e-adfc-4b41-8764-18b5efe2dd44.png)



Discussion:
on the same page, it would also be worth looking into where the siglum is coming from, as the chant I just created doesn't have one...
![Screen Shot 2023-01-13 at 10 53 42 AM](https://user-images.githubusercontent.com/58090591/212362290-71e38b7a-4732-43b9-a9b5-258843615143.png)



interesting. It's doing the same for me, but this seems to be a new development. Must be something caused by a recent change.
![image](https://user-images.githubusercontent.com/67451875/212366517-1eb38e11-ba70-483e-b880-63db212c5e05.png)

and, as pointed out up top, the chants seems to show up in the order added, not by folio 
![image](https://user-images.githubusercontent.com/67451875/212367256-c09ec327-75e9-49ed-90f7-44100631dcc6.png)
The "Full index" has them correctly ordered--not a case of an invisible leading zero or something.
![image](https://user-images.githubusercontent.com/67451875/212367403-79d418aa-90af-4da1-9350-69b617a73eb1.png)


Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -530,7 +530,7 @@ def get_queryset(self):
                 | Q(incipit__icontains=search_text)
                 | Q(manuscript_full_text__icontains=search_text)
             )
-        return chants.order_by("id")
+        return chants.order_by("folio", "c_sequence")
 
     def get_context_data(self, **kwargs):
         def get_feast_selector_options(source, folios):


Test patch:
Repository: DDMAL/CantusDB

Issue:
CSV export endpoint returns 403 Error on unpublished source
On NewCantus, accessing the `/csv/` endpoint on an unpublished source returns a 403 error. On OldCantus, this endpoint can be accessed on an unpublished source (at least an unpublished source that I have edit access to). 

For a user with edit access to a source, it probably makes sense to return a more appropriate error, hide the link to the csv export on the Source Detail page (and still return the 403 error at the endpoint, which at least makes more sense if the endpoint is accessed programmatically), or make the csv available to logged-in users with edit access to a source.


Discussion:
I'll make this available to logged-in users. Good catch.

Resolving patch:
diff --git a/django/cantusdb_project/main_app/urls.py b/django/cantusdb_project/main_app/urls.py
--- a/django/cantusdb_project/main_app/urls.py
+++ b/django/cantusdb_project/main_app/urls.py
@@ -296,10 +296,15 @@
         name="items-count",
     ),
     path(
-        "csv/<str:source_id>",
+        "source/<str:source_id>/csv/",
         views.csv_export,
         name="csv-export",
     ),
+    path(
+        "sites/default/files/csv/<str:source_id>.csv",
+        views.csv_export_redirect_from_old_path,
+        name="csv-export-old-path",
+    ),
     path(
         "ajax/concordance/<str:cantus_id>",
         views.ajax_concordance_list,
diff --git a/django/cantusdb_project/main_app/views/views.py b/django/cantusdb_project/main_app/views/views.py
--- a/django/cantusdb_project/main_app/views/views.py
+++ b/django/cantusdb_project/main_app/views/views.py
@@ -1,7 +1,7 @@
 import csv
 from django.http.response import JsonResponse
 from django.http import HttpResponse, HttpResponseNotFound
-from django.shortcuts import render
+from django.shortcuts import render, redirect
 from django.urls.base import reverse
 from main_app.models import (
     Century,
@@ -175,7 +175,9 @@ def csv_export(request, source_id):
     except:
         raise Http404("This source does not exist")
 
-    if not source.published:
+    display_unpublished = request.user.is_authenticated
+
+    if (not source.published) and (not display_unpublished):
         raise PermissionDenied
 
     # "4064" is the segment id of the sequence DB, sources in that segment have sequences instead of chants
@@ -252,6 +254,10 @@ def csv_export(request, source_id):
     return response
 
 
+def csv_export_redirect_from_old_path(request, source_id):
+    return redirect(reverse("csv-export", args=[source_id]))
+
+
 def contact(request):
     """
     Function-based view that renders the contact page ``contact``


Test patch:
Repository: DDMAL/CantusDB

Issue:
`/articles` has different url and does not show article previews
On OldCantus, `/news` is a valid url and returns a page showing a list of articles with titles and article details/previews.

On NewCantus, `/news` (http://206.12.88.113/news) returns a 404Error and the resulting list of articles at `/articles` does not contain previews of article contents. 

Additionally, the page on OldCantus paginates the list of articles, while on NewCantus it does not (very possible it has this behaviour programmed but just doesn't have enough articles...)

OldCantus:
<img width="960" alt="image" src="https://github.com/DDMAL/CantusDB/assets/11023634/153c9a31-5ea1-4e8a-a646-cde3bd7982f7">


NewCantus:

<img width="923" alt="image" src="https://github.com/DDMAL/CantusDB/assets/11023634/0b59df5a-b195-49d1-9c77-6cb920f23958">




Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/articles/urls.py b/django/cantusdb_project/articles/urls.py
--- a/django/cantusdb_project/articles/urls.py
+++ b/django/cantusdb_project/articles/urls.py
@@ -1,8 +1,12 @@
 from django.urls import path
-from articles.views import ArticleDetailView
-from articles.views import ArticleListView
+from articles.views import (
+    ArticleDetailView,
+    ArticleListView,
+    article_list_redirect_from_old_path,
+)
 
 urlpatterns = [
     path("articles/", ArticleListView.as_view(), name="article-list"),
     path("article/<int:pk>", ArticleDetailView.as_view(), name="article-detail"),
+    path("news/", article_list_redirect_from_old_path),
 ]
diff --git a/django/cantusdb_project/articles/views.py b/django/cantusdb_project/articles/views.py
--- a/django/cantusdb_project/articles/views.py
+++ b/django/cantusdb_project/articles/views.py
@@ -1,4 +1,5 @@
-from django.shortcuts import render
+from django.shortcuts import render, redirect
+from django.urls.base import reverse
 from django.views.generic import DetailView, ListView
 from articles.models import Article
 
@@ -12,6 +13,10 @@ class ArticleDetailView(DetailView):
 class ArticleListView(ListView):
     model = Article
     queryset = Article.objects.order_by("-date_created")
-    paginate_by = 100
+    paginate_by = 10
     context_object_name = "articles"
     template_name = "article_list.html"
+
+
+def article_list_redirect_from_old_path(request):
+    return redirect(reverse("article-list"))


Test patch:
Repository: DDMAL/CantusDB

Issue:
`IndexError` in syllabization and alignment
I just found this poking around on staging...I don't know how many chants this might be true for: `/chant/396856`.

<img width="1414" alt="image" src="https://github.com/DDMAL/CantusDB/assets/11023634/a9c12f04-ae46-49ed-a4b1-30379b4c6258">

I navigated to this page by following a link on the chant search page, specifically: `/chant-search/?op=contains&order=has_melody&sort=asc`



Discussion:
Strange. That chant seems to break regardless of how you access it (e.g. directly from the inventory of the source); it works fine in OldCantus, for what it's worth. 
Same breakage happens to the  next item in your search results, http://206.12.88.113/chant/521098. Not sure what they have in common...
for both of these chants, `chant.volpiano` is `\n` (i.e. just a line break character) lol

This should not be too hard to fix. Nice catch!

Resolving patch:
diff --git a/django/cantusdb_project/align_text_mel.py b/django/cantusdb_project/align_text_mel.py
--- a/django/cantusdb_project/align_text_mel.py
+++ b/django/cantusdb_project/align_text_mel.py
@@ -84,6 +84,12 @@ def syllabize_text(text, pre_syllabized=False):
 
 
 def syllabize_melody(volpiano):
+    # there exist several chants in the database whose volpiano is just `\n` -
+    # if a chant essentially has no volpiano, bail out early.
+    volpiano = volpiano.strip()
+    if not volpiano:
+        return []
+
     # the clef in volpiano should be 1--- with three dashes, if missing any dash, insert it
     if volpiano[1] != "-":
         volpiano = volpiano[:1] + "-" + volpiano[1:]


Test patch:
diff --git a/django/cantusdb_project/main_app/tests/test_views.py b/django/cantusdb_project/main_app/tests/test_views.py
--- a/django/cantusdb_project/main_app/tests/test_views.py
+++ b/django/cantusdb_project/main_app/tests/test_views.py
@@ -3508,7 +3508,7 @@ def test_json_sources_format(self):
         self.assertEqual(sample_item_keys, ["csv"])
 
         # the single value should be a link in form `cantusdatabase.com/csv/{source.id}`
-        expected_substring = f"/csv/{sample_id}"
+        expected_substring = f"source/{sample_id}/csv"
         sample_item_value = list(sample_item.values())[0]
         self.assertIn(expected_substring, sample_item_value)
 
@@ -3673,8 +3673,6 @@ def test_url(self):
         source = make_fake_source(published=True)
         response_1 = self.client.get(reverse("csv-export", args=[source.id]))
         self.assertEqual(response_1.status_code, 200)
-        response_2 = self.client.get(f"/csv/{source.id}")
-        self.assertEqual(response_2.status_code, 200)
 
     def test_content(self):
         NUM_CHANTS = 5
Repository: DDMAL/CantusDB

Issue:
Ensure that dummy users (i.e. old indexers) can't log in.
Currently, on NewCantus we have created dummy users for people who had an Indexer object but no User account on OldCantus. These dummy users have randomly generated email addresses and no passwords, the rationale being that a user with no password will not be able to log in.

A comment on [this stackoverflow topic](https://stackoverflow.com/questions/35690022/django-create-new-user-without-password) suggests that this is not strictly the case. It would probably be safer to set these dummy users to have [has_useable_password = False](https://docs.djangoproject.com/en/4.2/ref/contrib/auth/#django.contrib.auth.models.User.has_usable_password) if it is not already set up this way.


Discussion:
assigning @lucasmarchd01 because it would probably make sense to do this while running the data synchronization scripts.

Resolving patch:
diff --git a/django/cantusdb_project/main_app/management/commands/sync_indexers.py b/django/cantusdb_project/main_app/management/commands/sync_indexers.py
--- a/django/cantusdb_project/main_app/management/commands/sync_indexers.py
+++ b/django/cantusdb_project/main_app/management/commands/sync_indexers.py
@@ -69,12 +69,9 @@ def get_new_indexer(indexer_id):
             full_name=indexer_full_name,
             # assign random email to dummy users
             email=f"{faker.lexify('????????')}@fakeemail.com",
-            # leave the password empty for dummy users
-            # the password can't be empty in login form, so they can't log in
-            password="",
             old_indexer_id=indexer_id,
             is_indexer=True,
-        )
+        ).set_unusable_password()  # Set unusable password so the user can't log in or access reset password page
 
 
 class Command(BaseCommand):


Test patch:
Repository: DDMAL/CantusDB

Issue:
Hide "Edit chants (Fulltext & Volpiano editor)" link from my sources sidebar on flatpages
To prevent encountering a 404 error, we should hide the link from the "My Sources" sidebar on flatpages when the corresponding source has no chants.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/templatetags/helper_tags.py b/django/cantusdb_project/main_app/templatetags/helper_tags.py
--- a/django/cantusdb_project/main_app/templatetags/helper_tags.py
+++ b/django/cantusdb_project/main_app/templatetags/helper_tags.py
@@ -73,7 +73,10 @@ def make_links_for_source(source):
         link_with_siglum = make_source_detail_link_with_siglum(source)
         link_with_title = make_source_detail_link_with_title(source)
         add_new_chants_link = make_add_new_chants_link(source)
-        edit_chants_link = make_edit_chants_link(source)
+        if source.chant_set.exists():
+            edit_chants_link = make_edit_chants_link(source)
+        else:
+            edit_chants_link = ""
         template = """{sigl}<br>
         <small>
             <b>{title}</b><br>


Test patch:
Repository: DDMAL/CantusDB

Issue:
Links to unpublished sources should not appear on Provenance detail pages
Example: visit http://206.12.93.196/provenance/3665 (while logged out), click on first link. We get a 403 Forbidden error, since the source is unpublished.

Unpublished sources should not be listed on the Provenance Detail page.

Credit to @zhannaklimanova and her link checker script for catching this bug!


Discussion:
just out of curiosity--how complicated an exercise would it be to have a page like this one list the *names* of all the sources, but only have the published ones be *links*?
> just out of curiosity--how complicated an exercise would it be to have a page like this one list the _names_ of all the sources, but only have the published ones be _links_?

This would be easy!
oooo. I may confer about this...

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/views.py b/django/cantusdb_project/main_app/views/views.py
--- a/django/cantusdb_project/main_app/views/views.py
+++ b/django/cantusdb_project/main_app/views/views.py
@@ -648,7 +648,7 @@ def redirect_node_url(request, pk: int) -> HttpResponse:
     # we will manually create (unpublished) dummy objects in the database to ensure that all subqequent objects created will have IDs above this number.
     if pk >= 1_000_000:
         raise Http404("Invalid ID for /node/ path.")
-    
+
     # chant, source, sequence, article
     possible_types = [
         (Chant, "chant-detail"),
@@ -670,7 +670,6 @@ def redirect_node_url(request, pk: int) -> HttpResponse:
     raise Http404("No record found matching the /node/ query.")
 
 
-
 # used to determine whether record of specific type (chant, source, sequence, article) exists for a given pk
 def record_exists(rec_type: BaseModel, pk: int) -> bool:
     try:
@@ -707,6 +706,6 @@ def redirect_indexer(request, pk: int) -> HttpResponse:
     """
     user_id = get_user_id_from_old_indexer_id(pk)
     if get_user_id_from_old_indexer_id(pk) is not None:
-        return redirect('user-detail', user_id)
-    
+        return redirect("user-detail", user_id)
+
     raise Http404("No indexer found matching the query.")


Test patch:
Repository: DDMAL/CantusDB

Issue:
assigning-sources-to-users process could be smoother
I recently tried to add myself as an editor for a test source on staging. I went into the admin area, selected my user account, and tried to add CDN-Test2020 to my sources:

So far so good...:
![Screenshot 2023-06-19 at 4 17 36 PM](https://github.com/DDMAL/CantusDB/assets/58090591/badfa6ac-7256-4c9c-b384-24c1d41fee42)

Oh dear...:
![Screenshot 2023-06-19 at 4 17 26 PM](https://github.com/DDMAL/CantusDB/assets/58090591/69407ac3-6aa2-418b-b4d3-588480c14b98)

Let me try finding the source in the admin area, and then add myself to the source that way! oh... oh dear...:
![Screenshot 2023-06-19 at 4 18 09 PM](https://github.com/DDMAL/CantusDB/assets/58090591/c25aca10-b04d-4848-8558-0d1d0d62de53)

Admittedly, there exists a faster way to get to the source page—source-detail -> click "manage source editors", which takes you to the admin area for the source in question, where you can find a filterable list of users—but these two pages of the admin area could certainly be improved by the ability to filter.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/admin.py b/django/cantusdb_project/main_app/admin.py
--- a/django/cantusdb_project/main_app/admin.py
+++ b/django/cantusdb_project/main_app/admin.py
@@ -69,6 +69,11 @@ class SequenceAdmin(BaseModelAdmin):
 
 
 class SourceAdmin(BaseModelAdmin):
+    # These search fields are also available on the user-source inline relationship in the user admin page
+    search_fields = (
+        "siglum",
+        "title",
+    )
     # from the Django docs:
     # Adding a ManyToManyField to this list will instead use a nifty unobtrusive JavaScript “filter” interface
     # that allows searching within the options. The unselected and selected options appear in two boxes side by side.
diff --git a/django/cantusdb_project/users/admin.py b/django/cantusdb_project/users/admin.py
--- a/django/cantusdb_project/users/admin.py
+++ b/django/cantusdb_project/users/admin.py
@@ -9,6 +9,7 @@
 # this will allow us to assign sources to users in the User admin page
 class SourceInline(admin.TabularInline):
     model = Source.current_editors.through
+    raw_id_fields = ["source"]
 
 
 class UserAdmin(BaseUserAdmin):


Test patch:
Repository: DDMAL/CantusDB

Issue:
Source Edit Page: sidebar: source navigation links not displayed
breaking this issue out from #483. 

OldCantus - https://cantus.uwaterloo.ca/node/711311/edit:
![Screen Shot 2022-12-16 at 2 23 09 PM](https://user-images.githubusercontent.com/58090591/208173403-56dcb32a-ec87-4639-88cf-d9bbc74dc192.png)

NewCantus - http://206.12.88.113/edit-source/123653:
![Screen Shot 2022-12-16 at 2 23 47 PM](https://user-images.githubusercontent.com/58090591/208173695-676814b9-4306-4ffc-b187-ab8f9f624914.png)


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/source.py b/django/cantusdb_project/main_app/views/source.py
--- a/django/cantusdb_project/main_app/views/source.py
+++ b/django/cantusdb_project/main_app/views/source.py
@@ -211,7 +211,9 @@ def form_valid(self, form):
         return HttpResponseRedirect(self.get_success_url())
 
 
-class SourceEditView(LoginRequiredMixin, UserPassesTestMixin, UpdateView):
+class SourceEditView(
+    LoginRequiredMixin, UserPassesTestMixin, UpdateView, SourceDetailView
+):
     template_name = "source_edit.html"
     model = Source
     form_class = SourceEditForm


Test patch:
Repository: DDMAL/CantusDB

Issue:
Admin Area: Cannot delete Source that contains chants
When trying to delete a source that contains chants on the source admin page, we get a source DoesNotExist error. The source delete function works fine for sources that do not contain any chants, so I believe this has to do with deleting the associated chants after the source has already been deleted.


Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/signals.py b/django/cantusdb_project/main_app/signals.py
--- a/django/cantusdb_project/main_app/signals.py
+++ b/django/cantusdb_project/main_app/signals.py
@@ -12,6 +12,7 @@
 from main_app.models import Chant
 from main_app.models import Sequence
 from main_app.models import Feast
+from main_app.models import Source
 
 
 @receiver(post_save, sender=Chant)
@@ -68,7 +69,11 @@ def update_source_chant_count(instance):
     Called in on_chant_save(), on_chant_delete(), on_sequence_save() and on_sequence_delete()
     """
 
-    source = instance.source
+    # When a source is deleted (which in turn calls on_chant_delete() on all of its chants) instance.source does not exist
+    try:
+        source = instance.source
+    except Source.DoesNotExist:
+        source = None
     if source is not None:
         source.number_of_chants = source.chant_set.count() + source.sequence_set.count()
         source.save()
@@ -79,7 +84,12 @@ def update_source_melody_count(instance):
 
     Called in on_chant_save() and on_chant_delete()
     """
-    source = instance.source
+
+    # When a source is deleted (which in turn calls on_chant_delete() on all of its chants) instance.source does not exist
+    try:
+        source = instance.source
+    except Source.DoesNotExist:
+        source = None
     if source is not None:
         source.number_of_melodies = source.chant_set.filter(
             volpiano__isnull=False


Test patch:
Repository: DDMAL/CantusDB

Issue:
"my sources"" page side panel
http://206.12.88.113/my-sources/ has a "created by" side panel. This is
1. not paginated
3. includes all the sources and not just the created ones (so it's both extra- long and also redundant).
Staging:
<img width="1106" alt="image" src="https://github.com/DDMAL/CantusDB/assets/67451875/3d11789e-6027-4358-8595-328e95e89d7b">
on production this only has the sources created on production, so it only has one source (I'm assuming the others will show up once we sort out the "Created by" info from OldCantus?
<img width="1160" alt="image" src="https://github.com/DDMAL/CantusDB/assets/67451875/f6e98d78-0f66-421c-aad9-2ede47400d88">
On OldCantus it looks like this:

<img width="981" alt="image" src="https://github.com/DDMAL/CantusDB/assets/67451875/15f4b995-d930-4645-9ca4-3befce6a868d">




Discussion:

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/user.py b/django/cantusdb_project/main_app/views/user.py
--- a/django/cantusdb_project/main_app/views/user.py
+++ b/django/cantusdb_project/main_app/views/user.py
@@ -104,7 +104,7 @@ def get_context_data(self, **kwargs):
             .order_by("-date_created")
             .distinct()
         )
-        paginator = Paginator(user_created_sources, 10)
+        paginator = Paginator(user_created_sources, 6)
         page_number = self.request.GET.get("page2")
         page_obj = paginator.get_page(page_number)
 


Test patch: