Repository: internetarchive/openlibrary

Issue:
Search results page fails to render if search string includes special characters
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

### Evidence / Screenshot (if possible)
<img width="1050" alt="Screen Shot 2023-03-08 at 2 56 35 PM" src="https://user-images.githubusercontent.com/82178/223893720-68b534ce-5bf8-4b2c-b601-92ef76a4a2d4.png">

### Relevant url?

https://openlibrary.org/search?q=Cornell+%2777%3A+The+Music%2C+the+Myth+and+the+Magnificence+of+the+Grateful+Dead+Show+at+Barton+Hall&mode=everything

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
@cdrini Hi, I just pushed a fix for this issue, do let me know if I missed out on anything!
This is an issue with the library we're using to parse solr queries. I've created issues there:

- https://github.com/jurismarches/luqum/issues/86
- https://github.com/jurismarches/luqum/issues/87

Their community did in the past fix another issue I reported: https://github.com/jurismarches/luqum/issues/79 . Maybe you could try to fix these issues on that repo @JaydenTeoh and open a PR? Using their commit https://github.com/jurismarches/luqum/commit/0cc77673145aa6033fc0bc031d7702a20071df33 as a template? 

In the meantime, we can try adding these to the escape flow. Currently all user-submitted queries pass through this function:

- https://github.com/internetarchive/openlibrary/blob/14f8029ff74eb34c25c5cf7262f5662d655347fc/openlibrary/plugins/worksearch/schemes/__init__.py#L66

This function will try to parse the query, and if parsing errors, try to escape every character to convert the query to a plain solr string with no special features. This is done by this function:

https://github.com/internetarchive/openlibrary/blob/ed47d577da57d16fbdc36f39e4e82110fdfe0665/openlibrary/solr/query_utils.py#L120-L140

We likely want to expand this function to include the single quote.

Resolving patch:
diff --git a/openlibrary/solr/query_utils.py b/openlibrary/solr/query_utils.py
--- a/openlibrary/solr/query_utils.py
+++ b/openlibrary/solr/query_utils.py
@@ -131,10 +131,12 @@ def fully_escape_query(query: str) -> str:
     'x\\\\:\\\\[A TO Z\\\\}'
     >>> fully_escape_query('foo AND bar')
     'foo and bar'
+    >>> fully_escape_query("foo's bar")
+    "foo\\\\'s bar"
     """
     escaped = query
     # Escape special characters
-    escaped = re.sub(r'[\[\]\(\)\{\}:"\-+?~^/\\,]', r'\\\g<0>', escaped)
+    escaped = re.sub(r'[\[\]\(\)\{\}:"\-+?~^/\\,\']', r'\\\g<0>', escaped)
     # Remove boolean operators by making them lowercase
     escaped = re.sub(r'AND|OR|NOT', lambda _1: _1.group(0).lower(), escaped)
     return escaped


Test patch: