Repository: internetarchive/openlibrary

Issue:
Reading log JSON view errors when too many work IDs passed to solr
### Problem

Since the last run of the new redirect resolution script, the JSON view of my already-read shelf is generating an error. The script fixed a number of redirects, and I haven't made any manual changes to the shelf since, suggesting that the script might have written some bad data.

My other shelves are sending JSON correctly, so it's not a general API issue.

#### Relevant URL
https://openlibrary.org/people/mheiman9541/books/already-read.json?debug=true

### Reproducing the bug

_No response_

### Context

- Browser (Chrome, Safari, Firefox, etc): Firefox
- OS (Windows, Mac, etc): Mac
- Logged in (Y/N): Y
- Environment (prod, dev, local): prod


### ⚠️ Experienced contributors only

### Proposal & constraints

### Related files

### Stakeholders
@mekarpeles @mheiman @cdrini 



Discussion:
Update from @cdrini : Solr returned an http 414 URI too long. The script fixed so many issues that we're now sending too many work IDs to solr. We're currently making a GET request to solr's /get endpoint, but I wonder if there's a way we can make a POST request and put the work keys in the request body. That should avoid any URI length errors.
The way it currently looks, Solr innately supports POST requests. The bodies of the requests will have to be formatted in the manner of 
```
{
   "params": {
         "q": "field:value",
         "rows": 10,
         "wt": "json"
     }
}
``` 
As such, I think this should be fairly doable. 



Resolving patch:
diff --git a/openlibrary/core/bookshelves.py b/openlibrary/core/bookshelves.py
--- a/openlibrary/core/bookshelves.py
+++ b/openlibrary/core/bookshelves.py
@@ -176,6 +176,24 @@ def count_total_books_logged_by_user(
             ).values()
         )
 
+    @classmethod
+    def count_user_books_on_shelf(
+        cls,
+        username: str,
+        bookshelf_id: int,
+    ) -> int:
+        result = db.get_db().query(
+            """
+            SELECT count(*) from bookshelves_books
+            WHERE bookshelf_id=$bookshelf_id AND username=$username
+            """,
+            vars={
+                'bookshelf_id': bookshelf_id,
+                'username': username,
+            },
+        )
+        return result[0].count if result else 0
+
     @classmethod
     def count_total_books_logged_by_user_per_shelf(
         cls, username: str, bookshelf_ids: list[str] | None = None
diff --git a/openlibrary/plugins/upstream/mybooks.py b/openlibrary/plugins/upstream/mybooks.py
--- a/openlibrary/plugins/upstream/mybooks.py
+++ b/openlibrary/plugins/upstream/mybooks.py
@@ -1,7 +1,7 @@
 import json
 import web
 
-from typing import Final, Literal
+from typing import Final, Literal, cast
 
 from infogami import config
 from infogami.utils import delegate
@@ -297,11 +297,12 @@ class public_my_books_json(delegate.page):
     encoding = "json"
 
     def GET(self, username, key='want-to-read'):
-        i = web.input(page=1, limit=5000, q="")
+        i = web.input(page=1, limit=100, q="")
+        key = cast(ReadingLog.READING_LOG_KEYS, key.lower())
         if len(i.q) < 3:
             i.q = ""
         page = safeint(i.page, 1)
-        limit = safeint(i.limit, 5000)
+        limit = safeint(i.limit, 100)
         # check if user's reading log is public
         user = web.ctx.site.get('/people/%s' % username)
         if not user:
@@ -317,7 +318,7 @@ def GET(self, username, key='want-to-read'):
             and logged_in_user.key.split('/')[-1] == username
         ):
             readlog = ReadingLog(user=user)
-            books = readlog.get_works(key.lower(), page, limit, q=i.q).docs
+            books = readlog.get_works(key, page, limit, q=i.q).docs
             records_json = [
                 {
                     'work': {
@@ -342,8 +343,20 @@ def GET(self, username, key='want-to-read'):
                 }
                 for w in books
             ]
+
+            if page == 1 and len(records_json) < limit:
+                num_found = len(records_json)
+            else:
+                num_found = readlog.count_shelf(key)
+
             return delegate.RawText(
-                json.dumps({'page': page, 'reading_log_entries': records_json}),
+                json.dumps(
+                    {
+                        'page': page,
+                        'numFound': num_found,
+                        'reading_log_entries': records_json,
+                    }
+                ),
                 content_type="application/json",
             )
         else:
@@ -443,6 +456,11 @@ class ReadingLog:
     # Constants
     PRESET_SHELVES = Literal["Want to Read", "Already Read", "Currently Reading"]
     READING_LOG_KEYS = Literal["want-to-read", "already-read", "currently-reading"]
+    READING_LOG_KEY_TO_SHELF: dict[READING_LOG_KEYS, PRESET_SHELVES] = {
+        "want-to-read": "Want to Read",
+        "already-read": "Already Read",
+        "currently-reading": "Currently Reading",
+    }
 
     def __init__(self, user=None):
         self.user = user or accounts.get_current_user()
@@ -487,6 +505,12 @@ def reading_log_counts(self):
             ),
         }
 
+    def count_shelf(self, key: READING_LOG_KEYS) -> int:
+        username = self.user.get_username()
+        assert username
+        shelf_id = Bookshelves.PRESET_BOOKSHELVES[self.READING_LOG_KEY_TO_SHELF[key]]
+        return Bookshelves.count_user_books_on_shelf(username, shelf_id)
+
     def get_works(
         self,
         key: READING_LOG_KEYS,
@@ -503,16 +527,8 @@ def get_works(
 
         See LoggedBooksData for specifics on what's returned.
         """
-        if key == "want-to-read":
-            shelf = "Want to Read"
-        elif key == "already-read":
-            shelf = "Already Read"
-        elif key == "currently-reading":
-            shelf = "Currently Reading"
-        else:
-            raise ValueError(
-                "key must be want-to-read, already-read, or currently-reading"
-            )
+        shelf = self.READING_LOG_KEY_TO_SHELF[key]
+
         # Mypy is unhappy about the sort argument not being a literal string.
         # Although this doesn't satisfy Mypy, at least make sure sort is either
         # "created asc" or "created desc"


Test patch: