internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: internetarchive/openlibrary

Issue:
Change /partials endpoint to /partials.json to get gzip support
Currently the /partials endpoint is not gzipped. This is a performance issue. Our nginx automatically gzips .json URLs, so since this is also a JSON endpoint, let's change it to `/partials.json` and update all references in the code. 

Revealed by Google Lighthouse/pagespeed insights:

![image](https://user-images.githubusercontent.com/6251786/214691547-20e14666-a0ca-43ca-9a7b-5527ae470cda.png)
https://pagespeed.web.dev/report?url=https%3A%2F%2Fopenlibrary.org%2Fbooks%2FOL23153983M%2FAlgebra

Steps:
- Find all references to /partials and change it to /partials.json

### Stakeholders
@cclauss @jimchamp 


Discussion:
@cdrini Hello, I want to work on this issue and contribute.. Please assign me this issue. Thanks

@Mayuresh0072 please go ahead and create a pull request.  When multiple contributors create pull requests to fix the same issue then it is almost always a learning experience and often the merging of ideas is more powerful than a single pull request,
You are absolutely right. Actually, Solving issues is part of college assignment. So, It would be very helpful for me if you assign me, so that i can show my progress. Thanks
Could I participate in this issue? this is my first time in open source. 
Apologies @Blueclaus13 -- it seems @bkochendorfer has submitted a PR for this one, however we would love to work with you on one of these projects if you're interested in teaming up on an issue!

https://github.com/internetarchive/openlibrary/issues?q=is%3Aopen+is%3Aissue+label%3A%22Good+First+Issue%22+label%3A%22Needs%3A+Help%22
It looks like the `/partials` and `/lists/partials` endpoints are already `.json`, but not being compressed due to the `Content-Type` header not starting with `text/` (**Edit**: This has been fixed.)

The remaining `/merges/partials` and `/reading-goal/partials` endpoints are not returning `json`, and should be updated.  Any JS functions that hit these endpoints also need to be updated.
The `/merges/partials` endpoint was removed some time ago.  Only the `/reading-goal/partials` endpoint remains.  This is used to return a rendered yearly reading goal progress bar whenever somebody sets a reading goal from their My Books page.

Resolving patch:
diff --git a/openlibrary/plugins/upstream/checkins.py b/openlibrary/plugins/upstream/checkins.py
--- a/openlibrary/plugins/upstream/checkins.py
+++ b/openlibrary/plugins/upstream/checkins.py
@@ -267,13 +267,15 @@ def calc_progress(cls, books_read, goal):
 
 class ui_partials(delegate.page):
     path = '/reading-goal/partials'
+    encoding = 'json'
 
     def GET(self):
         i = web.input(year=None)
         year = i.year or datetime.now().year
         goal = get_reading_goals(year=year)
         component = render_template('check_ins/reading_goal_progress', [goal])
-        return delegate.RawText(component)
+        partials = {"partials": str(component)}
+        return delegate.RawText(json.dumps(partials))
 
 
 def setup():


Test patch:
Repository: internetarchive/openlibrary

Issue:
`providers/isbndb.py` crashes when the `'subjects'` value is a literal `None`
<!-- IMPORTANT: Before posting, be sure to redact or remove sensitive data, such as passwords, secret keys, session cookies, etc. -->
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

### Evidence / Screenshot (if possible)

    
Sometimes the `'subjects'` key in the ISBNdb dump have literal `None` values, which would cause an error even if doing something like `data.get('subjects', [])`, because it would return `None` instead of `[]`.
    
E.g., consider the following record:
```python
{
    'isbn': '0134878868',
    'msrp': '0.00',
    'title': 'Learn Adobe Premiere Pro Cc For Video Communication',
    'isbn13': '9780134878867',
    'authors': ['Dockery, Joe (author.)'],
    'binding': 'Electronic Resource',
    'language': 'en',
    'subjects': None,
}
```
This would crash with:
```python
        ^^^^^^^^^^^^^^^^^^^
  File "/openlibrary/scripts/providers/isbndb.py", line 73, in __init__
    self.subjects = [
                    ^
TypeError: 'NoneType' object is not iterable
```

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@mekarpeles 


Discussion:

Resolving patch:
diff --git a/scripts/providers/isbndb.py b/scripts/providers/isbndb.py
--- a/scripts/providers/isbndb.py
+++ b/scripts/providers/isbndb.py
@@ -71,7 +71,7 @@ def __init__(self, data: dict[str, Any]):
         self.languages = self._get_languages(data)
         self.source_records = [self.source_id]
         self.subjects = [
-            subject.capitalize() for subject in data.get('subjects', '') if subject
+            subject.capitalize() for subject in (data.get('subjects') or []) if subject
         ]
         self.binding = data.get('binding', '')
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Fix My Notes
<!-- IMPORTANT: Before posting, be sure to redact or remove sensitive data, such as passwords, secret keys, session cookies, etc. -->
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

Going to page 2 of notes breaks because the page is a string instead of an int.

### Evidence / Screenshot (if possible)

![openlibrary org_people_mekBot_books_notes_page=2 debug=true](https://github.com/internetarchive/openlibrary/assets/978325/71f02dc3-434f-45d7-8815-4e35f7c03a87)

### Relevant url?
https://openlibrary.org/people/mekBot/books/notes?page=2&debug=true

<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
Hey @mekarpeles! I would like to work on this issue. Do I need to create a draft proposal first or can I directly link a PR?
Hey, i saw it was like a one liner fix so i just did it . See if ther is need of changes in it.
Possibly related: #8698 
Should probably check page two of every paginated My Books page in order to confirm that this isn't happening elsewhere.
Okay I did check that and there was same error in my reviews, I did fix that too. 

Resolving patch:
diff --git a/openlibrary/plugins/upstream/mybooks.py b/openlibrary/plugins/upstream/mybooks.py
--- a/openlibrary/plugins/upstream/mybooks.py
+++ b/openlibrary/plugins/upstream/mybooks.py
@@ -100,7 +100,7 @@ def GET(self, username):
         if mb.is_my_page:
             docs = PatronBooknotes(mb.user).get_notes(page=int(i.page))
             template = render['account/notes'](
-                docs, mb.user, mb.counts['notes'], page=i.page
+                docs, mb.user, mb.counts['notes'], page=int(i.page)
             )
             return mb.render(header_title=_("Notes"), template=template)
         raise web.seeother(mb.user.key)


Test patch:
Repository: internetarchive/openlibrary

Issue:
sort books by most recently added/updated
<!-- What issue does this PR close? -->
Closes #7526 

<!-- What does this PR achieve? [feature|hotfix|fix|refactor] -->
Books carousel is now sorted by most recently added.
Most recently added refers to the "updated" field from the API schema. Reason for doing so is because since the original updated field is same as created field (when book is first added to mybooks bookshelf), it is safe to compare using the "updated field". Therefore, when a user changes a book from "Want to Read" to "Already Read", the book added to the "Already Read" shelf will be at the front of the carousel list.

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@mekarpeles Lead
@bicolino34 Created issue

Sort books on My Books carousels (by most recent)
My books page carousels always show books that are the oldest added to Reading Log. I think it would be better to randomise books shown here. For example, in Want to Read carousel patron might see a book that he added a long time ago, but forgot about it, and might decide to read it. Same goes for Already Read carousel

![зображення](https://user-images.githubusercontent.com/58372007/218318034-6de93728-c0c9-4d02-87fe-18b4f073ac53.png)


### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:
Thank you! I think this gets us most of the way there.
During a previous design call, the community tentatively decided to sort by most recently added -- I have a PR I'll be submitting soon.
@mekarpeles Hi, I'm Sid, one of the students who listened to ur sharing. Your sharing today was truly valuable, and I am grateful for it. I'd love to get involved by working on this issue. Can u kindly points out which part of the code this is related to so I can get started? 
@mekarpeles I have made a PR for this issue. Please check on it and let me know if I missed out on anything https://github.com/internetarchive/openlibrary/pull/7572

Resolving patch:
diff --git a/openlibrary/plugins/upstream/mybooks.py b/openlibrary/plugins/upstream/mybooks.py
--- a/openlibrary/plugins/upstream/mybooks.py
+++ b/openlibrary/plugins/upstream/mybooks.py
@@ -57,7 +57,7 @@ def render_template(self, mb):
                 loans.docs.append(book)
 
         if mb.me or mb.is_public:
-            params = {'sort': 'created', 'limit': 6, 'sort_order': 'asc', 'page': 1}
+            params = {'sort': 'created', 'limit': 6, 'sort_order': 'desc', 'page': 1}
             want_to_read = mb.readlog.get_works(key='want-to-read', **params)
             currently_reading = mb.readlog.get_works(key='currently-reading', **params)
             already_read = mb.readlog.get_works(key='already-read', **params)


Test patch:
Repository: internetarchive/openlibrary

Issue:
error on user page: `error in processing template: ValueError: Invalid seed`
### Problem

```
/openlibrary/openlibrary/templates/lists/snippet.html: error in processing template: ValueError: Invalid seed: Thing(site=<infogami.infobase.client.Site object at 0x7f3053b88050>, key=/works/OL5159916W, data={'location': '/works/OL276798W', 'key': '/works/OL5159916W', 'type': Thing(site=<infogami.infobase.client.Site object at 0x7f3053b88050>, key=/type/redirect, data=None, revision=None), 'latest_revision': 9, 'revision': 9, 'created': datetime.datetime(2009, 12, 10, 8, 51, 6, 587414), 'last_modified': datetime.datetime(2021, 4, 14, 13, 29, 27, 506)}, revision=None) (falling back to default template)
```

#### Evidence / Screenshot
<img width="1113" alt="image" src="https://github.com/internetarchive/openlibrary/assets/921217/77f77d6b-c91f-4841-a4c7-020ce297e32b">


#### Relevant URL
https://openlibrary.org/people/clh7cb

### Reproducing the bug

1. Go to the page

* Expected behavior: no error
* Actual behavior: there is an error


### Context

- Browser (Chrome, Safari, Firefox, etc): FF
- OS (Windows, Mac, etc): Mac
- Logged in (Y/N): Y
- Environment (prod, dev, local): prod


### ⚠️ Experienced contributors only

### Proposal & constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
@scottbarnes and I did a preliminary analysis ; this seems to be a sporadic error.

This is the underlying error:
![image](https://github.com/internetarchive/openlibrary/assets/6251786/f593d6bd-7293-4bcf-8df7-77b7e5ede0e7)


Our current hypothesis is that this only errors when a cache miss happens, and the user's list contains a redirect.

It seems like under these conditions, the seed is returned as a `client.Thing` instead of the openlibrary subclass of `client.Thing`, which is what is checked for on line 403 in the image above.

Resolving patch:
diff --git a/openlibrary/core/lists/model.py b/openlibrary/core/lists/model.py
--- a/openlibrary/core/lists/model.py
+++ b/openlibrary/core/lists/model.py
@@ -399,7 +399,9 @@ def __init__(
     def from_db(list: List, seed: Thing | SeedSubjectString) -> 'Seed':
         if isinstance(seed, str):
             return Seed(list, seed)
-        elif isinstance(seed, Thing):
+        # If there is a cache miss, `seed` is a client.Thing.
+        # See https://github.com/internetarchive/openlibrary/issues/8882#issuecomment-1983844076
+        elif isinstance(seed, Thing | client.Thing):
             if seed.key is None:
                 return Seed(list, cast(AnnotatedSeed, seed._data))
             else:


Test patch:
Repository: internetarchive/openlibrary

Issue:
Remove fixed subject line from contact form
The contact form has a drop down for choosing a topic. When patrons submit this form, the submissions all go to the same inbox. The selected topics are usually incorrect, but also make it more difficult to manage the inbox due to most messages looking the same.

### Describe the problem that you'd like solved
Let the patron populate the subject line OR auto-populate the subject line from the message.

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/support.py b/openlibrary/plugins/openlibrary/support.py
--- a/openlibrary/plugins/openlibrary/support.py
+++ b/openlibrary/plugins/openlibrary/support.py
@@ -32,11 +32,12 @@ def POST(self):
         patron_name = form.get("name", "")
         email = form.get("email", "")
         topic = form.get("topic", "")
+        subject_line = form.get('subject', '')
         description = form.get("question", "")
         url = form.get("url", "")
         user = accounts.get_current_user()
         useragent = web.ctx.env.get("HTTP_USER_AGENT", "")
-        if not all([email, topic, description]):
+        if not all([email, description]):
             return ""
 
         hashed_ip = hashlib.md5(web.ctx.ip.encode('utf-8')).hexdigest()
@@ -59,7 +60,7 @@ def POST(self):
         else:
             assignee = default_assignees.get("default", "openlibrary@archive.org")
         stats.increment("ol.support.all")
-        subject = "Support case *%s*" % topic
+        subject = "Support case *%s*" % self.prepare_subject_line(subject_line)
 
         url = web.ctx.home + url
         displayname = user and user.get_name() or ""
@@ -73,6 +74,14 @@ def POST(self):
         )
         return render_template("email/case_created", assignee)
 
+    def prepare_subject_line(self, subject, max_length=60):
+        if not subject:
+            return '[no subject]'
+        if len(subject) <= max_length:
+            return subject
+
+        return subject[:max_length]
+
 
 def sendmail(from_address, to_address, subject, message):
     if config.get('dummy_sendmail'):
@@ -95,7 +104,7 @@ def sendmail(from_address, to_address, subject, message):
 
 A new support case has been filed by %(displayname)s <%(email)s>.
 
-Topic: %(topic)s
+Subject: %(subject_line)s
 URL: %(url)s
 User-Agent: %(useragent)s
 OL-username: %(username)s


Test patch:
Repository: internetarchive/openlibrary

Issue:
Restrict read of patron preferences to logged in patron
### Problem

There isn't a significant security implication given how we use settings today, but in the interest of patron privacy, we should make account preferences only accessible by the logged in patron.

#### Relevant URL
e.g. https://openlibrary.org/people/mekBot/preferences

### Reproducing the bug

1. Go to ...
2. Do ...

* Expected behavior:
* Actual behavior:


### Context

- Browser (Chrome, Safari, Firefox, etc):
- OS (Windows, Mac, etc):
- Logged in (Y/N): Y
- Environment (prod, dev, local): prod


### ⚠️ Experienced contributors only

### Proposal & constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
I'm thinking that this should also include the `.json` and `.yml` preference requests.

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/code.py b/openlibrary/plugins/openlibrary/code.py
--- a/openlibrary/plugins/openlibrary/code.py
+++ b/openlibrary/plugins/openlibrary/code.py
@@ -46,6 +46,7 @@
 delegate.app.add_processor(processors.ReadableUrlProcessor())
 delegate.app.add_processor(processors.ProfileProcessor())
 delegate.app.add_processor(processors.CORSProcessor(cors_prefixes={'/api/'}))
+delegate.app.add_processor(processors.PreferenceProcessor())
 
 try:
     from infogami.plugins.api import code as api
diff --git a/openlibrary/plugins/openlibrary/processors.py b/openlibrary/plugins/openlibrary/processors.py
--- a/openlibrary/plugins/openlibrary/processors.py
+++ b/openlibrary/plugins/openlibrary/processors.py
@@ -1,8 +1,10 @@
 """web.py application processors for Open Library.
 """
 
+import re
 import web
 
+from openlibrary.accounts import get_current_user
 from openlibrary.core.processors import ReadableUrlProcessor
 
 from openlibrary.core import helpers as h
@@ -86,6 +88,27 @@ def add_cors_headers(self):
         web.header("Access-Control-Max-Age", 3600 * 24)  # one day
 
 
+class PreferenceProcessor:
+    """Processor to handle unauthorized patron preferece reads"""
+
+    def __init__(self):
+        self.pref_pattern = re.compile(r'^\/people\/([^/]+)\/preferences(.json|.yml)?$')
+
+    def __call__(self, handler):
+        if self.pref_pattern.match(web.ctx.path):
+            user = get_current_user()
+            if not user:
+                # Must be logged in to see preferences
+                raise web.Unauthorized()
+
+            username = web.ctx.path.split('/')[2]
+            if username != user.get_username() and not user.is_admin():
+                # Can only view preferences if page owner or admin
+                raise web.Forbidden()
+
+        return handler()
+
+
 if __name__ == "__main__":
     import doctest
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Navigating to a private reading log shelf results in an error page
### Problem

When visiting any private reading log page, an error page is rendered.

#### Evidence / Screenshot
![image](https://github.com/internetarchive/openlibrary/assets/28732543/5f0b2d25-1cd4-4962-afab-a5724b65fab4)

#### Relevant URL(s)
eg https://openlibrary.org/people/jachamp/books/currently-reading

### Reproducing the bug

1. Navigate to a reading log page of a patron who has made their reading log private

* Expected behavior: I am redirected to the patron's profile page
* Actual behavior: I am confronted by an error page


### Context

- Browser (Chrome, Safari, Firefox, etc): N/A
- OS (Windows, Mac, etc): N/A
- Logged in (Y/N): N/A
- Environment (prod, dev, local): prod


### Notes from this Issue's Lead

#### Proposal & constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

#### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

#### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/mybooks.py b/openlibrary/plugins/upstream/mybooks.py
--- a/openlibrary/plugins/upstream/mybooks.py
+++ b/openlibrary/plugins/upstream/mybooks.py
@@ -232,18 +232,19 @@ class mybooks_readinglog(delegate.page):
 
     def GET(self, username, key='want-to-read'):
         mb = MyBooksTemplate(username, key)
-        KEYS_TITLES = {
-            'currently-reading': _(
-                "Currently Reading (%(count)d)", count=mb.counts['currently-reading']
-            ),
-            'want-to-read': _(
-                "Want to Read (%(count)d)", count=mb.counts['want-to-read']
-            ),
-            'already-read': _(
-                "Already Read (%(count)d)", count=mb.counts['already-read']
-            ),
-        }
         if mb.is_my_page or mb.is_public:
+            KEYS_TITLES = {
+                'currently-reading': _(
+                    "Currently Reading (%(count)d)",
+                    count=mb.counts['currently-reading'],
+                ),
+                'want-to-read': _(
+                    "Want to Read (%(count)d)", count=mb.counts['want-to-read']
+                ),
+                'already-read': _(
+                    "Already Read (%(count)d)", count=mb.counts['already-read']
+                ),
+            }
             template = self.render_template(mb)
             return mb.render(header_title=KEYS_TITLES[key], template=template)
         raise web.seeother(mb.user.key)
@@ -402,7 +403,7 @@ def __init__(self, username, key):
         self.counts = (
             self.readlog.reading_log_counts
             if (self.is_my_page or self.is_public)
-            else []
+            else {}
         )
 
         self.reading_goals = []


Test patch: