Repository: internetarchive/openlibrary

Issue:
"Random Book" option isn't
The "random book" feature on the openlibrary.org site appears to not function as originally intended, and instead is only returning one of a small number of results for books with "available" or "lending" in their textual metadata (title, description, etc).

### Evidence / Screenshot (if possible)

![image](https://user-images.githubusercontent.com/344766/107159376-68b02900-6944-11eb-9821-de30501f6b68.png)

### Relevant url

https://openlibrary.org/random

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to https://openlibrary.org/random repeatedly

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: The resulting book pages seem to repeat quite frequently, and all have "lending" or "available" in the book title / subject / description.
* Expected: An actual random assortment of books from across the corpus.

### Details

- **Logged in (Y/N)?** N
- **Browser type/version?** Chrome / curl
- **Operating system?**  Windows / Linux
- **Environment (prod/dev/local)?** prod

### Related files

This functionality appears to be implemented here:
https://github.com/internetarchive/openlibrary/blob/8575acb2543143d9b7e2bd0f387dccaa4541835e/openlibrary/core/lending.py#L207

It looks like it is relying on the advanced search page to be able to provide a random selection from the corpus when invoked in a certain way, and it seems like the search implementation likely changed since this functionality was originally implemented, such that search is no longer honoring the flags this function is trying to send, and instead interpreting them as free text search terms (hence the bias towards "available" and "lending" topics in the non-random book result).

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
Hi... Just browsing around looking for a first good issue.
I'm not a python dev, but after taking a peek under the hood it looks like line 211 might hold some answers:
`               "+AND+(lending___available_to_borrow OR lending___available_to_browse)"`
Is it possible that these are being interpreted as the actual query string, causing the key words "lending", "available" "borrow" and "browse" to be interpreted as the search query?
@ayust thank you for submitting this issue, you're right, there appears to be an issue!

@pixeljava thanks for looking into this! You may be on to something -- feel encouraged to modify the actual archive.org query to see if changes result in the correct behavior <3

Removing Open Library from the equation, there does indeed appear to be something strange / problematic with the query itself (upstream):

http://archive.org/advancedsearch.php?q=_exists_:openlibrary_work+AND+(lending___available_to_borrow%20OR%20lending___available_to_browse)&fl=identifier,openlibrary_edition&output=json&rows=1&sort[]=random

Like @pixeljava mentions, it looks like the query is being expanded into something relatively strange w/ using `title` fields:

```
query: "_exists_:openlibrary_work AND ( (((title:lending___available_to_borrow^100 OR salients:lending___available_to_borrow^50 OR subject:lending___available_to_borrow^25 OR description:lending___available_to_borrow^15 OR collection:lending___available_to_borrow^10 OR language:lending___available_to_borrow^10 OR text:lending___available_to_borrow^1) AND !indexflag:(uncurated OR favorites OR users))^10000 OR ((title:lending___available_to_borrow^100 OR salients:lending___available_to_borrow^50 OR subject:lending___available_to_borrow^25 OR description:lending___available_to_borrow^15 OR collection:lending___available_to_borrow^10 OR language:lending___available_to_borrow^10 OR text:lending___available_to_borrow^1) AND indexflag:(uncurated OR favorites OR users))^0.00001) OR (((title:lending___available_to_browse^100 OR salients:lending___available_to_browse^50 OR subject:lending___available_to_browse^25 OR description:lending___available_to_browse^15 OR collection:lending___available_to_browse^10 OR language:lending___available_to_browse^10 OR text:lending___available_to_browse^1) AND !indexflag:(uncurated OR favorites OR users))^10000 OR ((title:lending___available_to_browse^100 OR salients:lending___available_to_browse^50 OR subject:lending___available_to_browse^25 OR description:lending___available_to_browse^15 OR collection:lending___available_to_browse^10 OR language:lending___available_to_browse^10 OR text:lending___available_to_browse^1) AND indexflag:(uncurated OR favorites OR users))^0.00001) )",
```

cc: @ximm can you see based on this url / query why the same set of books may be returned across requests? Is it a problem with cache and perhaps all we need to do is e.g. add `&reCache=1` to the url?

EDIT: `reCache=1` doesn't appear to resolve the problem.
@pixeljava thank you, you're totally right, I believe in the query, we need to update the call to:

https://archive.org/advancedsearch.php?q=_exists_:openlibrary_work+AND+(lending___available_to_borrow:true%20OR%20lending___available_to_browse:true)&fl=identifier,openlibrary_edition&output=json&rows=1&sort[]=random

namely adding `:true` after `lending___available_to_browse` and `lending___available_to_borrow`.

We can refer to the correct example here:
https://github.com/internetarchive/openlibrary/blob/8575acb2543143d9b7e2bd0f387dccaa4541835e/openlibrary/core/lending.py#L140

Do you mind opening up a PR? :tada:
Can I follow the above given instructions and open up a PR ?

Yes, please do. I'm not a Python developer and I'd feel bad changing something without being able to thoroughly test it first.

Resolving patch:
diff --git a/openlibrary/core/lending.py b/openlibrary/core/lending.py
--- a/openlibrary/core/lending.py
+++ b/openlibrary/core/lending.py
@@ -208,7 +208,7 @@ def get_random_available_ia_edition():
     """uses archive advancedsearch to raise a random book"""
     try:
         url = ("http://%s/advancedsearch.php?q=_exists_:openlibrary_work"
-               "+AND+(lending___available_to_borrow OR lending___available_to_browse)"
+               "+AND+(lending___available_to_borrow:true OR lending___available_to_browse:true)"
                "&fl=identifier,openlibrary_edition"
                "&output=json&rows=1&sort[]=random" % (config_bookreader_host))
         response = requests.get(url, timeout=config_http_request_timeout)


Test patch: