Repository: DDMAL/CantusDB

Issue:
Chant Search: keyword/cantus_id field should automatically be populated when searching from search bar
The way the search chants page is set up makes it not obvious (in my opinion) what it is you've actually searched for, which is floating off in the upper right while the "keyword" field is blank.

![image](https://user-images.githubusercontent.com/67451875/211895450-63081608-6d7a-49d6-8968-8f4fba2bab7e.png)

On OldCantus it's a little clearer what it is you were looking for, because it's right by the "Full Text Search" words. Or maybe that's just where I'm used to looking for it.

![image](https://user-images.githubusercontent.com/67451875/211895790-a459ad9e-e925-4820-b674-f691b93f36db.png)


search bar: results preview inconsistent with results upon pressing enter
When using the search bar in the top-right corner of most pages, results appear as the user types. These results appear to search for the search term anywhere in the incipit. But when the user presses enter and is taken to the "Search Chants" page, they get a bunch of results that begin with the search term.

The search bar preview replicates the behavior of OldCantus. But we made a conscious decision to change the behavior of the Search Chants page on NewCantus when arriving via the search bar, to return results where the search term occurs at the beginning of one of the text fields. Having changed one, we should probably update the other.

This discrepancy can lead to some counterintuitive, bug-seeming results: if you type "ysterium" in the search box, you get lots of "mysterium"-related results in the preview, but when you press enter, you get no results on the Search Chants page.
Chant Search: keyword/cantus_id field should automatically be populated when searching from search bar
The way the search chants page is set up makes it not obvious (in my opinion) what it is you've actually searched for, which is floating off in the upper right while the "keyword" field is blank.

![image](https://user-images.githubusercontent.com/67451875/211895450-63081608-6d7a-49d6-8968-8f4fba2bab7e.png)

On OldCantus it's a little clearer what it is you were looking for, because it's right by the "Full Text Search" words. Or maybe that's just where I'm used to looking for it.

![image](https://user-images.githubusercontent.com/67451875/211895790-a459ad9e-e925-4820-b674-f691b93f36db.png)




Discussion:
Ah! this is another one that would be solved by getting rid of the `?search-bar` query parameter and replacing it with `?keyword` (see #516)
How exactly do we want this to work?

Currently, in `?search-bar=<search_term>`, if `search_term` contains any non-letter characters, we search for all chants/sequences whose `cantus_id` is equal to `search_term` and return them - this is exactly the same behavior as `?cantus_id=<search_term>`.

However, in `?search-bar=<search_term>`, if `search_term` contains only letters, we search for all chants/sequences whose standard-spelling fulltext begins with `search_term`. This is different from `?keyword=<search_term>&operation=starts_with`, which searches standard-spelling fulltext, ms-spelling fulltext, and incipit.

I feel that in an ideal world, whenever a person makes a search from the search bar, we would use `?keyword=` or `?cantus_id=` as appropriate, and not use a `?search_bar` query parameter at all (this would also have the effect of the search term appearing in the corresponding field in the form). If we implement it this way, _are we happy with searching all three text fields for the search term rather than searching just the standard-spelling field as we currently do?_

A further question: currently, in the `ajax_search_bar` function (which displays search results when you type into the search bar without pressing enter), we only search chants' incipits. We also search only for chants, and not for sequences. This means that if a user types something into the search bar and sees a certain collection of chants displayed, they might get an entirely different set of search results once they press enter. _Is this desired/acceptable behavior?_
My initial thought is that searching all three text fields is probably an improvement over the current behaviour, since it means people don't have to mentally "standardize" the spelling to get results. I'll think about whether there's any cases where this would be a bad thing. 

I'll have to think about the Ajax search bar. My instinct is that the difference in results is not a problem, though.
One consideration: My guess is that the Ajax search bar searches only the incipit in order for it to return results quickly (and that OldCantus searches the same field for the same reason), and if we searched multiple fields here, the user would need to wait longer before results are displayed.
I was thinking the same--that there's a trade-off between completeness and getting some results quickly. I don't think that's too weird for the user (it's sort of like an autocomplete for chant...)
Notes for future fixing of this: Django has a built in `get_initial` method that could be useful here.
Currently on OldCantus, no matter what you search for in the search bar, it ends up in the "incipit" field on the main form. This means that if you search for a Cantus ID and press `enter`, OldCantus will search for the ID in the incipit field. So NewCantus's system is already better than OldCantus's, in that you can see a list of chants with matching Cantus IDs when you search for something involving numbers!

But it would certainly be even better if we auto-populated the keyword/cantusID field (depending on the presence of numbers) when a user uses the search bar.

Assigning a label of "enhancement" for this reason.

Ah! this is another one that would be solved by getting rid of the `?search-bar` query parameter and replacing it with `?keyword` (see #516)
How exactly do we want this to work?

Currently, in `?search-bar=<search_term>`, if `search_term` contains any non-letter characters, we search for all chants/sequences whose `cantus_id` is equal to `search_term` and return them - this is exactly the same behavior as `?cantus_id=<search_term>`.

However, in `?search-bar=<search_term>`, if `search_term` contains only letters, we search for all chants/sequences whose standard-spelling fulltext begins with `search_term`. This is different from `?keyword=<search_term>&operation=starts_with`, which searches standard-spelling fulltext, ms-spelling fulltext, and incipit.

I feel that in an ideal world, whenever a person makes a search from the search bar, we would use `?keyword=` or `?cantus_id=` as appropriate, and not use a `?search_bar` query parameter at all (this would also have the effect of the search term appearing in the corresponding field in the form). If we implement it this way, _are we happy with searching all three text fields for the search term rather than searching just the standard-spelling field as we currently do?_

A further question: currently, in the `ajax_search_bar` function (which displays search results when you type into the search bar without pressing enter), we only search chants' incipits. We also search only for chants, and not for sequences. This means that if a user types something into the search bar and sees a certain collection of chants displayed, they might get an entirely different set of search results once they press enter. _Is this desired/acceptable behavior?_
My initial thought is that searching all three text fields is probably an improvement over the current behaviour, since it means people don't have to mentally "standardize" the spelling to get results. I'll think about whether there's any cases where this would be a bad thing. 

I'll have to think about the Ajax search bar. My instinct is that the difference in results is not a problem, though.
One consideration: My guess is that the Ajax search bar searches only the incipit in order for it to return results quickly (and that OldCantus searches the same field for the same reason), and if we searched multiple fields here, the user would need to wait longer before results are displayed.
I was thinking the same--that there's a trade-off between completeness and getting some results quickly. I don't think that's too weird for the user (it's sort of like an autocomplete for chant...)
Notes for future fixing of this: Django has a built in `get_initial` method that could be useful here.
Currently on OldCantus, no matter what you search for in the search bar, it ends up in the "incipit" field on the main form. This means that if you search for a Cantus ID and press `enter`, OldCantus will search for the ID in the incipit field. So NewCantus's system is already better than OldCantus's, in that you can see a list of chants with matching Cantus IDs when you search for something involving numbers!

But it would certainly be even better if we auto-populated the keyword/cantusID field (depending on the presence of numbers) when a user uses the search bar.

Assigning a label of "enhancement" for this reason.

Resolving patch:
diff --git a/django/cantusdb_project/main_app/views/chant.py b/django/cantusdb_project/main_app/views/chant.py
--- a/django/cantusdb_project/main_app/views/chant.py
+++ b/django/cantusdb_project/main_app/views/chant.py
@@ -792,7 +792,7 @@ def get_queryset(self) -> QuerySet:
             else:
                 # if search bar is doing Cantus ID search
                 cantus_id = self.request.GET.get("search_bar")
-                q_obj_filter &= Q(cantus_id=cantus_id)
+                q_obj_filter &= Q(cantus_id__icontains=cantus_id)
                 chant_set = chant_set.filter(q_obj_filter).values(
                     *CHANT_SEARCH_TEMPLATE_VALUES
                 )
diff --git a/django/cantusdb_project/main_app/views/views.py b/django/cantusdb_project/main_app/views/views.py
--- a/django/cantusdb_project/main_app/views/views.py
+++ b/django/cantusdb_project/main_app/views/views.py
@@ -404,12 +404,12 @@ def ajax_search_bar(request, search_term):
     # load only the first seven chants
     CHANT_CNT = 7
 
-    if any(char.isdigit() for char in search_term):
+    if not search_term.replace(" ", "").isalpha():
         # if the search term contains at least one digit, assume user is searching by Cantus ID
         chants = Chant.objects.filter(cantus_id__istartswith=search_term).order_by("id")
     else:
         # if the search term does not contain any digits, assume user is searching by incipit
-        chants = Chant.objects.filter(incipit__icontains=search_term).order_by("id")
+        chants = Chant.objects.filter(incipit__istartswith=search_term).order_by("id")
 
     display_unpublished = request.user.is_authenticated
     if not display_unpublished:


Test patch: