internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: internetarchive/openlibrary

Issue:
"`Nothing` type not JSON serializable" errors occurring when fetching list data from `/seeds.json`
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->
When a list containing Infogami's `Nothing` type is fetched using `/seeds.json`, a TypeError is thrown because `Nothing` is not JSON serializable.

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->
https://openlibrary.org/people/tmanarl/lists/OL197952L/seeds.json?debug=true

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Navigate to the link listed above.

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: An error page is displayed.
* Expected: A JSON-encoded list should be displayed

### Details

- **Logged in (Y/N)?** N/A
- **Browser type/version?** N/A
- **Operating system?** N/A
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
`helpers.py` contains a `json_encode` function that serializes `Nothing` types.  This can be used instead of the vanilla JSON encoder.

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@libjenner



Discussion:

Resolving patch:
diff --git a/openlibrary/core/formats.py b/openlibrary/core/formats.py
--- a/openlibrary/core/formats.py
+++ b/openlibrary/core/formats.py
@@ -3,6 +3,8 @@
 import json
 import yaml
 
+from openlibrary.core.helpers import NothingEncoder
+
 __all__ = ["load_yaml", "dump_yaml"]
 
 
@@ -25,7 +27,7 @@ def load(text, format):
 
 def dump(data, format):
     if format == "json":
-        return json.dumps(data)
+        return json.dumps(data, cls=NothingEncoder)
     elif format == "yml":
         return dump_yaml(data)
     else:
diff --git a/openlibrary/core/helpers.py b/openlibrary/core/helpers.py
--- a/openlibrary/core/helpers.py
+++ b/openlibrary/core/helpers.py
@@ -107,8 +107,24 @@ def get_nofollow(name, event):
     return stream.render()
 
 
+class NothingEncoder(json.JSONEncoder):
+    def default(self, obj):
+        """Custom JSON encoder for handling Nothing values.
+
+        Returns None if a value is a Nothing object. Otherwise,
+        encode values using the default JSON encoder.
+        """
+        if isinstance(obj, Nothing):
+            return None
+        return super().default(obj)
+
+
 def json_encode(d, **kw):
-    """Same as json.dumps."""
+    """Calls json.dumps on the given data d.
+    If d is a Nothing object, passes an empty list to json.dumps.
+
+    Returns the json.dumps results.
+    """
     return json.dumps([] if isinstance(d, Nothing) else d, **kw)
 
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Admin Sponsorship Dashboard broken: civi related?
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->
Divide by zero issue (likely failing to fetch sponsorship data) at https://openlibrary.org/admin/sponsorship

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->

Admin Sponsorship Dashboard broken: civi related?
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->
Divide by zero issue (likely failing to fetch sponsorship data) at https://openlibrary.org/admin/sponsorship

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
Can I work on this Issue @mekarpeles ?
Can I work on this Issue @mekarpeles ?

Resolving patch:
diff --git a/openlibrary/core/sponsorships.py b/openlibrary/core/sponsorships.py
--- a/openlibrary/core/sponsorships.py
+++ b/openlibrary/core/sponsorships.py
@@ -287,7 +287,7 @@ def get_sponsored_books():
             'collection',
             'isbn',
         ],
-        params={'page': 1, 'rows': 1000, 'scope': 'all'},
+        params={'page': 1, 'rows': 1000},
         config={'general': {'secure': False}},
     )
     return [


Test patch:
Repository: internetarchive/openlibrary

Issue:
Include full url in "Report a problem" emails
### Describe the problem that you'd like solved
Currently the "Report a problem" form sends the user's URL, but doesn't include any parameters. So staff only sees e.g. `/search` instead of `/search?q=harry+potter`. This makes it harder to debug issues.

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
Currently, the `Report a Problem` link in the footer includes the path, but not the parameters!

Related code:
- The link in the footer: https://github.com/internetarchive/openlibrary/blob/8a1e17358c107e665bc653b689c3f50fb329f2c2/openlibrary/templates/lib/nav_foot.html#L44
- The spot this parameter is passed to the support form: https://github.com/internetarchive/openlibrary/blob/8a1e17358c107e665bc653b689c3f50fb329f2c2/openlibrary/templates/support.html#L54-L56
- The form's post endpoint: https://github.com/internetarchive/openlibrary/blob/39962157c12465d050e43458444aba9a59661c8d/openlibrary/plugins/openlibrary/support.py#L65

### Stakeholders
@seabelis 


Discussion:
@cdrini I could give this a try. Can I work on it?


Resolving patch:
diff --git a/openlibrary/plugins/upstream/utils.py b/openlibrary/plugins/upstream/utils.py
--- a/openlibrary/plugins/upstream/utils.py
+++ b/openlibrary/plugins/upstream/utils.py
@@ -937,6 +937,7 @@ class Request:
     path = property(lambda self: web.ctx.path)
     home = property(lambda self: web.ctx.home)
     domain = property(lambda self: web.ctx.host)
+    fullpath = property(lambda self: web.ctx.fullpath)
 
     @property
     def canonical_url(self):


Test patch:
Repository: internetarchive/openlibrary

Issue:
Fix Reading Log json endpoint for "Already Read"
<!-- What issue does this PR close? -->
Closes #6165 

<!-- What does this PR achieve? [feature|hotfix|fix|refactor] -->


### Technical
<!-- What should be noted about the implementation? -->

### Testing
<!-- Steps for reviewer to reproduce/verify what this PR does/fixes. -->

### Screenshot
<!-- If this PR touches UI, please post evidence (screenshots) of it behaving correctly. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->


<!-- Attribution Disclaimer: By proposing this pull request, I affirm to have made a best-effort and exercised my discretion to make sure relevant sections of this code which substantially leverage code suggestions, code generation, or code snippets from sources (e.g. Stack Overflow, GitHub) have been annotated with basic attribution so reviewers & contributors may have confidence and access to the correct context to evaluate and use this code. -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/mybooks.py b/openlibrary/plugins/upstream/mybooks.py
--- a/openlibrary/plugins/upstream/mybooks.py
+++ b/openlibrary/plugins/upstream/mybooks.py
@@ -60,9 +60,14 @@ def GET(self, username, key='want-to-read'):
                     'work': {
                         'title': w.get('title'),
                         'key': w.key,
-                        'author_keys': [a.author.key for a in w.get('authors', [])],
+                        'author_keys': [
+                            a.author.get("key")
+                            for a in w.get('authors', []) if a.author
+                        ],
                         'author_names': [
-                            str(a.author.name) for a in w.get('authors', [])
+                            str(a.author.name)
+                            for a in w.get('authors', [])
+                            if type(a.author) is not str
                         ],
                         'first_publish_year': w.first_publish_year or None,
                         'lending_edition_s': (


Test patch:
Repository: internetarchive/openlibrary

Issue:
ISBN only imports have empty string Library of congress classification set
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->


Example:
https://openlibrary.org/books/OL28864152M/Emerging_Issues_in_Life-Cycle_Management

In the edition JSON:
```
"lc_classifications": [
""
],
```
UI has an empty 
**Classifications**
Library of Congress

section.

**Expectation:**
If there is no Library of Congress classification date, the field will not exist, and there will be no **Classifications** section on the UI.
ISBN only imports have empty string Library of congress classification set
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->


Example:
https://openlibrary.org/books/OL28864152M/Emerging_Issues_in_Life-Cycle_Management

In the edition JSON:
```
"lc_classifications": [
""
],
```
UI has an empty 
**Classifications**
Library of Congress

section.

**Expectation:**
If there is no Library of Congress classification date, the field will not exist, and there will be no **Classifications** section on the UI.


Discussion:

Resolving patch:
diff --git a/openlibrary/catalog/add_book/__init__.py b/openlibrary/catalog/add_book/__init__.py
--- a/openlibrary/catalog/add_book/__init__.py
+++ b/openlibrary/catalog/add_book/__init__.py
@@ -808,10 +808,11 @@ def load(rec, account_key=None):
         'local_id',
         'lccn',
         'lc_classifications',
+        'oclc_numbers',
         'source_records',
     ]
     for f in edition_fields:
-        if f not in rec:
+        if f not in rec or not rec[f]:
             continue
         # ensure values is a list
         values = rec[f] if isinstance(rec[f], list) else [rec[f]]
@@ -824,6 +825,15 @@ def load(rec, account_key=None):
         if to_add:
             need_edition_save = True
 
+    # Add new identifiers
+    if 'identifiers' in rec:
+        identifiers = defaultdict(set, e.get('identifiers', {}))
+        for k, vals in rec.identifiers:
+            identifiers[k].update(vals)
+        if e.get('identifiers') != identifiers:
+            e['identifiers'] = {k: list(vals) for k, vals in identifiers.items()}
+            need_edition_save = True
+
     edits = []
     reply = {
         'success': True,


Test patch:
Repository: internetarchive/openlibrary

Issue:
Missing "Preview" & "Checked Out" btns 
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

Borrowable books are missing preview buttons, likely related to last edit of LoanStatus page.

https://openlibrary.org/books/OL35676065M/Harry_Potter_and_the_Philosopher's_Stone

Also, "Checked Out" button not showing up, perhaps related to removing "Not In Library" button

### Evidence / Screenshot (if possible)

![openlibrary org_works_OL2231866W_The_Omega_Rx_Zone](https://user-images.githubusercontent.com/978325/158632329-53fa94db-4f27-485c-8580-f745c29d9fe1.png)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/core/lending.py b/openlibrary/core/lending.py
--- a/openlibrary/core/lending.py
+++ b/openlibrary/core/lending.py
@@ -337,16 +337,14 @@ def get_availability(key, ids):
         return {}
 
     def update_availability_schema_to_v2(v1_resp, ocaid):
+        """This functionattempts to take the output of e.g. Bulk Availability
+        API and add/infer attributes which are missing (but are
+        present on Ground Truth API)
+        """
+        # TODO: Make less brittle; maybe add simplelists/copy counts to Bulk Availability
         v1_resp['identifier'] = ocaid
         v1_resp['is_restricted'] = v1_resp['status'] != 'open'
-        v1_resp['is_printdisabled'] = v1_resp.get('is_printdisabled', False)
-        v1_resp['is_lendable'] = v1_resp['status'] == 'borrow_available'
-        v1_resp['is_readable'] = v1_resp['status'] == 'open'
-        # TODO: Make less brittle; maybe add simplelists/copy counts to IA availability
-        # endpoint
-        v1_resp['is_browseable'] = (
-            v1_resp['is_lendable'] and v1_resp['status'] == 'error'
-        )
+        v1_resp['is_browseable'] = v1_resp.get('available_to_browse', False)
         # For debugging
         v1_resp['__src__'] = 'core.models.lending.get_availability'
         return v1_resp


Test patch:
Repository: internetarchive/openlibrary

Issue:
Remove unnecessary patron loan and waiting list queries
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->
Whenever a logged in patron visits a book page, API calls are made to archive.org/services/openlibrary.php in order to fetch the patron's current loan and wait list status (one call is required for each).  These calls require the patron's Internet Archive username, which is prefixed with an `@`.

Today, four calls are made when the books page is loaded.  The two extra calls are made using an obsolete form of the patron's username (the username is prefixed with `ol:`, not `@`).  These calls do nothing useful, and should be removed.

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->
Any book page.

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. While logged in, visit a book page.
2. Check the logs for waiting list queries and loan queries.

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: Log entries for four calls exist.
* Expected: Log entries exist for only two calls.

### Details

- **Logged in (Y/N)?** Y
- **Browser type/version?** N/A
- **Operating system?** N/A
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
Identify where the unneeded calls are being made, and remove these calls.

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/core/lending.py b/openlibrary/core/lending.py
--- a/openlibrary/core/lending.py
+++ b/openlibrary/core/lending.py
@@ -563,7 +563,6 @@ def get_loans_of_user(user_key):
     loandata = web.ctx.site.store.values(type='/type/loan', name='user', value=user_key)
     loans = [Loan(d) for d in loandata] + (
         _get_ia_loans_of_user(account.itemname)
-        + _get_ia_loans_of_user(userkey2userid(user_key))
     )
     return loans
 
diff --git a/openlibrary/core/waitinglist.py b/openlibrary/core/waitinglist.py
--- a/openlibrary/core/waitinglist.py
+++ b/openlibrary/core/waitinglist.py
@@ -129,9 +129,7 @@ def find(cls, user_key, identifier, itemname=None):
         if not itemname:
             account = OpenLibraryAccount.get(key=user_key)
             itemname = account.itemname
-        result = cls.query(userid=itemname, identifier=identifier) or cls.query(
-            userid=lending.userkey2userid(user_key), identifier=identifier
-        )
+        result = cls.query(userid=itemname, identifier=identifier)
         if result:
             return result[0]
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Tags are deleted/lost with line break
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->
It's too easy to delete existing tags or lose new tags. If there is a line break in the tags, everything after the line break is deleted or not saved.
### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->
See people tags: A new person was pasted in which created a line break. All the other tags were deleted.
https://openlibrary.org/works/OL540258W/Signet_Classic_Book_of_American_Short_Stories?_compare=Compare&b=9&a=8&m=diff
### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...any work
2. Do ... add new tags before existing tags, followed by a line break OR add a line break after existing tags, add new tags and save.

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: tags are lost
* Expected: line breaks should be ignored and tags should be retained or saved

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
Is this for subject tags? (can't check because of maintenance mode)
Yeah; converting it a textarea (which we did recently) allows newlines, but the system ignores everything after a newline char somehow
I'm happy to take a look at this.

Resolving patch:
diff --git a/openlibrary/plugins/upstream/addbook.py b/openlibrary/plugins/upstream/addbook.py
--- a/openlibrary/plugins/upstream/addbook.py
+++ b/openlibrary/plugins/upstream/addbook.py
@@ -723,7 +723,7 @@ def read_subject(subjects):
             """
             if not subjects:
                 return
-            f = six.StringIO(subjects)
+            f = six.StringIO(subjects.replace('\r\n',''))
             dedup = set()
             for s in next(csv.reader(f, dialect='excel', skipinitialspace=True)):
                 if s.lower() not in dedup:


Test patch:
Repository: internetarchive/openlibrary

Issue:
Remove any collection starting with `fav-*` from Solr
We don't use them and they will take up even more storage once editions are in solr.

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:

Resolving patch:
diff --git a/openlibrary/solr/update_work.py b/openlibrary/solr/update_work.py
--- a/openlibrary/solr/update_work.py
+++ b/openlibrary/solr/update_work.py
@@ -768,7 +768,13 @@ def get_ia_sorting_key(ed: dict) -> tuple[AvailabilityEnum, str]:
             if best_availability == AvailabilityEnum.PUBLIC:
                 ebook_info['public_scan_b'] = True
 
-            all_collection = uniq(c for e in ia_eds for c in e.get('ia_collection', []))
+            all_collection = sorted(uniq(
+                c
+                for e in ia_eds
+                for c in e.get('ia_collection', [])
+                # Exclude fav-* collections because they're not useful to us.
+                if not c.startswith('fav-')
+            ))
             if all_collection:
                 ebook_info['ia_collection_s'] = ';'.join(all_collection)
 


Test patch:
diff --git a/openlibrary/tests/solr/test_update_work.py b/openlibrary/tests/solr/test_update_work.py
--- a/openlibrary/tests/solr/test_update_work.py
+++ b/openlibrary/tests/solr/test_update_work.py
@@ -754,6 +754,24 @@ def test_goog_deprioritized(self):
         ]
         assert SolrProcessor.get_ebook_info(editions)['ia'] == ["foobarblah", "foobargoog"]
 
+    def test_excludes_fav_ia_collections(self):
+        doc = {}
+        editions = [
+            {
+                "key": "/books/OL789M",
+                "ocaid": "foobargoog",
+                "ia_collection": ['americanlibraries', 'fav-foobar'],
+            },
+            {
+                "key": "/books/OL789M",
+                "ocaid": "foobarblah",
+                "ia_collection": ['fav-bluebar', 'blah'],
+            },
+        ]
+
+        doc = SolrProcessor.get_ebook_info(editions)
+        assert doc['ia_collection_s'] == "americanlibraries;blah"
+
 
 class TestSolrUpdate:
     def sample_response_200(self):