Repository: DDMAL/CantusDB

Issue:
"Select feast" dropdown repeats feasts
On the dropdown to "Select a feast" on a given source (in this case https://cantusdatabase.org/source/637892)  many feasts will repeat for no obvious reason--even though all the chants for one feast are in a continuous block, the list acts as if they are separate. On OldCantus these would show as part of one feast. Comparison:

OldCantus:
<img width="229" alt="image" src="https://github.com/DDMAL/CantusDB/assets/67451875/20abb0d9-33df-48d8-a6f5-eedc5be5684b">

NewCantus:
<img width="390" alt="image" src="https://github.com/DDMAL/CantusDB/assets/67451875/ac76f4b6-80b4-4ad8-8338-d6b95d7f3142">




Discussion:
Looking at this selector in this source, it's odd that the duplicates always seem to occur in alternation with duplicates from another feast. Weird.
was going to make an issue about this today but realized I already had! 
<img width="784" alt="image" src="https://github.com/DDMAL/CantusDB/assets/67451875/8c4387e7-6bb5-41f9-a155-a8094b6775b9">
There doesn't seem to be much pattern to it--if there's multiple feasts on the page they sort of show up in random order, sometimes repeating, sometimes not.  it *feels* like the problem is that some sort of information about the sequence number is not incorporated into the group-by-feast function, but I don't really know.
e.g. 133v actually has 4 "De Tobia" and 5 "De Esther" chants; if these were randomly sorted as (say) TEETTETEE, and then the adjacent ones sharing a feast condensed to one entry, you'd get a list that looks like the above. 
this continues to be both annoying/frustrating to use and extremely silly to look at. 
Often I want to know what, e.g. the 130 feasts in St-Gall 390 are and exactly what order they go in, in addition to navigating straight to them; a dropdown with 484(!) entries in only-sort-of-order is not useful for this, and scrolling through three times as many feasts as necessary in order to find some specific feast is aggravating. 
personally find this higher priority than #1089, which doesn't bother me nearly as much or as often...but maybe my need to look at feasts is more than most!


I agree with @annamorphism. Bumping the priority from medium to high - this is a place where the site isn't working, in a way that makes it clear that us developers have messed something up, in a place that's easy to get to.
@annamorphism I'm also noticing that when you select a feast from the dropdown, you get results that include every chant in the source associated with that feast, not just the chants associated with that feast on the specific folio. This feels like false advertising (why would you give me both `001r - Feast A` and `001v - Feast A` if selecting either one will give me the same thing).

Should the results of clicking a drop down give you only the chants of that feast on that folio? Or should we just do away with the folio numbers together and have the feast dropdown give you all the chants in the source associated with that particular feast?

To see what I mean, go to [this source](https://cantusdatabase.org/source/637892/chants/) and select `005r - Thomae de Aquino` and then select `012v - Thomae de Aquino` and see that you get the exact same results.
I see what you mean--even in cases where there might actually be two copies of the feast in the manuscript (maybe there's one on 99r and a revised version added in on 152v), even the old interface took you to a page with both versions one after the other, never the "revised folio 152v version" by itself. 
However I can imagine that if you happen to read somewhere that this manuscript has a certain feast on 152v, it might be easier to find it in the dropdown if the folio number is included rather than reading down the whole list of headings with no page information. (There are of course other ways of finding feasts in a manuscript! But I do find myself navigating this way.)
So to summarize: keep the same behaviour as it is now when you select an item from the dropdown (show all of the chants with the given feast regardless of folio) but fix that fact that we have all of these duplicated options in the dropdown....
For now anyway, yes--this would duplicate the OldCantus behaviour. Having it take you to only the chants for feast X starting on some folio Y seems like a potentially interesting feature, so if that is a good way to implement it then it's a bonus--but if it's non-trivial then it's probably worth thinking a bit further about the pros/cons.

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -1,6 +1,13 @@
-import requests
 import urllib
-import json
+from collections import Counter
+from typing import Optional, Iterator, cast
+
+from volpiano_display_utilities.cantus_text_syllabification import (
+    syllabify_text,
+    flatten_syllabified_text,
+)
+from volpiano_display_utilities.text_volpiano_alignment import align_text_and_volpiano
+
 from django.contrib import messages
 from django.db.models import Q, QuerySet
 from django.shortcuts import get_object_or_404
@@ -14,6 +21,11 @@
     UpdateView,
 )
 from django.core.exceptions import PermissionDenied
+from django.contrib.auth.mixins import LoginRequiredMixin
+from django.http import Http404
+
+from django.contrib.auth.mixins import UserPassesTestMixin
+
 from main_app.forms import (
     ChantCreateForm,
     ChantEditForm,
@@ -27,23 +39,12 @@
     Sequence,
     Office,
 )
-from django.contrib.auth.mixins import LoginRequiredMixin
-from django.http import Http404
-from collections import Counter
-from django.contrib.auth.mixins import UserPassesTestMixin
-from typing import Optional
-from requests.exceptions import SSLError, Timeout, ConnectionError
-from requests import Response
 from main_app.permissions import (
     user_can_edit_chants_in_source,
     user_can_proofread_chant,
     user_can_view_chant,
 )
-from volpiano_display_utilities.cantus_text_syllabification import (
-    syllabify_text,
-    flatten_syllabified_text,
-)
-from volpiano_display_utilities.text_volpiano_alignment import align_text_and_volpiano
+
 from cantusindex import get_suggested_chants, get_suggested_fulltext
 
 CHANT_SEARCH_TEMPLATE_VALUES: tuple[str, ...] = (
@@ -75,9 +76,7 @@
 )
 
 
-def get_feast_selector_options(
-    source: Source, folios: list[str]
-) -> list[tuple[str, Feast]]:
+def get_feast_selector_options(source: Source) -> list[tuple[str, int, str]]:
     """Generate folio-feast pairs as options for the feast selector
 
     Going through all chants in the source, folio by folio,
@@ -85,50 +84,22 @@ def get_feast_selector_options(
 
     Args:
         source (Source object): The source that the user is browsing in.
-        folios (list of strs): A list of folios in the source.
 
     Returns:
         list of tuples: A list of folios and Feast objects, to be unpacked in template.
     """
-    # the two lists to be zipped
-    feast_selector_feasts: list[Feast] = []
-    feast_selector_folios: list[str] = []
-    # get all chants in the source, select those that have a feast
-    chants_in_source: QuerySet[Chant] = (
+    folios_feasts_iter: Iterator[tuple[Optional[str], int, str]] = (
         source.chant_set.exclude(feast=None)
         .order_by("folio", "c_sequence")
-        .select_related("feast")
+        .values_list("folio", "feast_id", "feast__name")
+        .iterator()
     )
-    # initialize the feast selector options with the first chant in the source that has a feast
-    first_feast_chant: Optional[Feast] = chants_in_source.first()
-    if not first_feast_chant:
-        # if none of the chants in this source has a feast, return an empty list
-        return []
-    # if there is at least one chant that has a feast
-    current_feast: Feast = first_feast_chant.feast
-    feast_selector_feasts.append(current_feast)
-    current_folio: str = first_feast_chant.folio
-    feast_selector_folios.append(current_folio)
-
-    chants_by_folio: QuerySet[Chant] = chants_in_source.filter(
-        folio__in=folios
-    ).order_by("folio")
-    for chant in chants_by_folio:
-        if chant.feast != current_feast:
-            # if the feast changes, add the new feast and the corresponding folio to the lists
-            feast_selector_feasts.append(chant.feast)
-            feast_selector_folios.append(chant.folio)
-            # update the current_feast to track future changes
-            current_feast = chant.feast
-    # as the two lists will always be of the same length, no need for zip,
-    # just naively combine them
-    # if we use zip, the returned generator will be exhausted in rendering templates,
-    # making it hard to test the returned value
-    folios_with_feasts: list[tuple[str, Feast]] = [
-        (feast_selector_folios[i], feast_selector_feasts[i])
-        for i in range(len(feast_selector_folios))
-    ]
-    return folios_with_feasts
+    # Cast because we know, by restrictions on chant create form, that
+    # folio won't be None
+    folios_feasts_list = cast(list[tuple[str, int, str]], list(folios_feasts_iter))
+    # De-dupe query set while maintaining order
+    deduped_folios_feasts_lists = list(dict.fromkeys(folios_feasts_list))
+    return deduped_folios_feasts_lists
 
 
 class ChantDetailView(DetailView):
@@ -1064,7 +1035,7 @@ def get_chants_with_folios(chants_in_feast):
         )
         context["folios"] = folios
         # the options for the feast selector on the right, same as the source detail page
-        context["feasts_with_folios"] = get_feast_selector_options(source, folios)
+        context["feasts_with_folios"] = get_feast_selector_options(source)
 
         if self.request.GET.get("feast"):
             # if there is a "feast" query parameter, it means the user has chosen a specific feast
diff --git a/django/cantusdb_project/main_app/views/source.py b/django/cantusdb_project/main_app/views/source.py
--- a/django/cantusdb_project/main_app/views/source.py
+++ b/django/cantusdb_project/main_app/views/source.py
@@ -156,7 +156,7 @@ def get_context_data(self, **kwargs):
             )
 
         # the options for the feast selector on the right, same as the source detail page
-        context["feasts_with_folios"] = get_feast_selector_options(source, folios)
+        context["feasts_with_folios"] = get_feast_selector_options(source)
         return context
 
 
@@ -191,7 +191,7 @@ def get_context_data(self, **kwargs):
             )
             context["folios"] = folios
             # the options for the feast selector on the right, only chant sources have this
-            context["feasts_with_folios"] = get_feast_selector_options(source, folios)
+            context["feasts_with_folios"] = get_feast_selector_options(source)
 
         context["user_can_edit_chants"] = user_can_edit_chants_in_source(user, source)
         context["user_can_edit_source"] = user_can_edit_source(user, source)
@@ -404,7 +404,7 @@ def get_context_data(self, **kwargs):
             )
             context["folios"] = folios
             # the options for the feast selector on the right, only chant sources have this
-            context["feasts_with_folios"] = get_feast_selector_options(source, folios)
+            context["feasts_with_folios"] = get_feast_selector_options(source)
         return context
 
     def test_func(self):


Test patch:
diff --git a/django/cantusdb_project/main_app/tests/test_views.py b/django/cantusdb_project/main_app/tests/test_views.py
--- a/django/cantusdb_project/main_app/tests/test_views.py
+++ b/django/cantusdb_project/main_app/tests/test_views.py
@@ -706,7 +706,11 @@ def test_context_feasts_with_folios(self):
         response = self.client.get(reverse("browse-chants", args=[source.id]))
         # context "feasts_with_folios" is a list of tuples
         # it records the folios where the feast changes
-        expected_result = [("001r", feast_1), ("001v", feast_2), ("002r", feast_1)]
+        expected_result = [
+            ("001r", feast_1.id, feast_1.name),
+            ("001v", feast_2.id, feast_2.name),
+            ("002r", feast_1.id, feast_1.name),
+        ]
         self.assertEqual(response.context["feasts_with_folios"], expected_result)
 
     def test_redirect_with_source_parameter(self):
@@ -4367,7 +4371,11 @@ def test_context_feasts_with_folios(self):
         response = self.client.get(reverse("source-detail", args=[source.id]))
         # context "feasts_with_folios" is a list of tuples
         # it records the folios where the feast changes
-        expected_result = [("001r", feast_1), ("001v", feast_2), ("002r", feast_1)]
+        expected_result = [
+            ("001r", feast_1.id, feast_1.name),
+            ("001v", feast_2.id, feast_2.name),
+            ("002r", feast_1.id, feast_1.name),
+        ]
         self.assertEqual(response.context["feasts_with_folios"], expected_result)
 
     def test_context_sequences(self):