Repository: internetarchive/openlibrary

Issue:
strip underscores from the isbn endpoint
<!-- IMPORTANT: Before posting, be sure to redact or remove sensitive data, such as passwords, secret keys, session cookies, etc. -->

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->
Like this: https://openlibrary.org/isbn/_9781554042951_
That would save a decent amount of annoyance (glares at Amazon where double clicking isbns always seems to include extra spaces) if they could be stripped automatically. (including ones between numbers b/c of isbns with spaces).

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
It might make sense to just strip everything that isn't a valid isbn character 0-9 and X/x?

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@scottbarnes 




Discussion:
Hey! Can I work on this :)
@QuantuM410 go for it!
@QuantuM410 are you still working on this?
@RayBB Yep! Will make a PR soon
@QuantuM410, hopefully I am not mentioning this too late, but it would be good to look at [isbnlib](https://github.com/xlcnd/isbnlib) if you have not already done so. E.g.:
```python
In [1]: from isbnlib import canonical

In [2]: canonical("_9781.554!04--29%51_")
Out[2]: '9781554042951'
```

That said, I'm not entirely sure how some of these characters, including spaces, will look once accessed in Python. I would hope you have access to the unquoted string, but I'm not actually sure. If you run into issues, please report back. :)

Also, ordinarily we wouldn't want to rely on a third party library, but in this case, Open Library [already uses it](https://github.com/internetarchive/openlibrary/blob/44e250f89a710ced051a1d974a843f6c3c81dab7/requirements.txt#L16).
> @QuantuM410, hopefully I am not mentioning this too late, but it would be good to look at [isbnlib](https://github.com/xlcnd/isbnlib) if you have not already done so. E.g.:
> 
> ```python
> In [1]: from isbnlib import canonical
> 
> In [2]: canonical("_9781.554!04--29%51_")
> Out[2]: '9781554042951'
> ```
> 
> That said, I'm not entirely sure how some of these characters, including spaces, will look once accessed in Python. I would hope you have access to the unquoted string, but I'm not actually sure. If you run into issues, please report back. :)
> 
> Also, ordinarily we wouldn't want to rely on a third party library, but in this case, Open Library [already uses it](https://github.com/internetarchive/openlibrary/blob/44e250f89a710ced051a1d974a843f6c3c81dab7/requirements.txt#L16).

@scottbarnes 
I added a line here ```isbn = canonical(isbn)``` in the GET method of isbn_lookup class. Can you verify if this is right. Furthermore, how may I test this issue? 
https://github.com/internetarchive/openlibrary/blob/44e250f89a710ced051a1d974a843f6c3c81dab7/openlibrary/plugins/openlibrary/code.py#L481-L508
@QuantuM410, this looks like a good start. You can simply put something like, `print(f"ISBN is: {isbn}", flush=True)` to see the output in the Docker console, assuming you're running Docker in the foreground. That way you can see what the ISBN is becoming, when you visit http://localhost:8080/isbn/whatever

One thing to look out for is the regex for `path`: `path = r'/(?:isbn|ISBN)/([0-9xX-]+)'`. The `path` value is used in two places, but `canonical` won't modify that. But it _probably_ makes sense to show people what was actually searched for when, e.g., `return render.notfound(web.ctx.path, create=False)` is run.

You can visit, for example, http://localhost:8080/isbn/059035342-X_.json or http://localhost:8080/isbn/059035342-X_.json to see what I mean. Perhaps this should simply return that `/isbn/059035342X` was tried, rather than what was literally entered.
> @QuantuM410, this looks like a good start. You can simply put something like, `print(f"ISBN is: {isbn}", flush=True)` to see the output in the Docker console, assuming you're running Docker in the foreground. That way you can see what the ISBN is becoming, when you visit http://localhost:8080/isbn/whatever
> 
> One thing to look out for is the regex for `path`: `path = r'/(?:isbn|ISBN)/([0-9xX-]+)'`. The `path` value is used in two places, but `canonical` won't modify that. But it _probably_ makes sense to show people what was actually searched for when, e.g., `return render.notfound(web.ctx.path, create=False)` is run.
> 
> You can visit, for example, http://localhost:8080/isbn/059035342-X_.json or http://localhost:8080/isbn/059035342-X_.json to see what I mean. Perhaps this should simply return that `/isbn/059035342X` was tried, rather than what was literally entered.

Hey @scottbarnes, its still a bit confusing while testing the isbns. If the isbn is something like ```_9781554042951_```, the ```path``` doesn't match the pattern here
https://github.com/internetarchive/openlibrary/blob/44e250f89a710ced051a1d974a843f6c3c81dab7/openlibrary/plugins/openlibrary/code.py#L482
 and doesnt go through the ```isbn_lookup``` GET method. 
Furthermore to display the path which was searched for, can we do something like ```web.ctx.path = f'/isbn/{isbn}' ```?

@QuantuM410, sorry, I think I wasn't super clear there and I took it for granted that the regex will need updating, perhaps with something as simple as `r'/(?:isbn|ISBN)/(.{10,})'` to just match anything 10 or more characters.

As for what to do with the `path` value, originally I was thinking we'd probably want to show people what was actually searched for (e.g. the output from `canoncial()`), but `canoncial()` returns an empty string from an ISBN that can't have a canonical form. So if someone tries to access `/isbn/9999999999999999999999999`, it's probably more confusing to show `/isbn/` doesn't exist than it is to show `/isbn/9999999999999999999999999` doesn't exist.

I'm certainly open to arguments otherwise, but maybe all that needs to be done is to update `path` to match 10 or more characters of any type, then use `isbn = canoncial(isbn)` as you had done originally. Unless you feel strongly otherwise, let's just keep it minimal.

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/code.py b/openlibrary/plugins/openlibrary/code.py
--- a/openlibrary/plugins/openlibrary/code.py
+++ b/openlibrary/plugins/openlibrary/code.py
@@ -35,7 +35,7 @@
 
 from openlibrary.core import cache
 from openlibrary.core.vendors import create_edition_from_amazon_metadata
-from openlibrary.utils.isbn import isbn_13_to_isbn_10, isbn_10_to_isbn_13
+from openlibrary.utils.isbn import isbn_13_to_isbn_10, isbn_10_to_isbn_13, canonical
 from openlibrary.core.models import Edition
 from openlibrary.core.lending import get_availability
 import openlibrary.core.stats
@@ -479,11 +479,11 @@ def remove_high_priority(query: str) -> str:
 
 
 class isbn_lookup(delegate.page):
-    path = r'/(?:isbn|ISBN)/([0-9xX-]+)'
+    path = r'/(?:isbn|ISBN)/(.{10,})'
 
     def GET(self, isbn):
         input = web.input(high_priority=False)
-
+        isbn = canonical(isbn)
         high_priority = input.get("high_priority") == "true"
         if "high_priority" in web.ctx.env.get('QUERY_STRING'):
             web.ctx.env['QUERY_STRING'] = remove_high_priority(


Test patch: