internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: internetarchive/openlibrary

Issue:
Non-ASCII character rendering broken on author page
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

This author https://openlibrary.org/authors/OL8115327A/Robert_Wi_347_niewski
has a name that renders as `Robert Wi&#347;niewski` on the author page
but, correctly, as `Robert Wiśniewski` on the pages of his works e.g. https://openlibrary.org/works/OL21191421W

### Evidence / Screenshot (if possible)
![Screen Shot 2022-04-10 at 10 40 15 PM](https://user-images.githubusercontent.com/82178/162656086-3a7ef02f-7816-4aed-9805-611dd144feb3.png)

### Relevant url?
See above

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected: Correct rendering is used on all pages where the name appears

### Details

- **Logged in (Y/N)?** Y
- **Browser type/version?** Chrome Version 100.0.4896.75 (Official Build) (x86_64)
- **Operating system?** Mac OS
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/utils.py b/openlibrary/plugins/upstream/utils.py
--- a/openlibrary/plugins/upstream/utils.py
+++ b/openlibrary/plugins/upstream/utils.py
@@ -1,10 +1,11 @@
-from typing import List, Union, Tuple, Any
+from typing import Iterable, List, Union, Tuple, Any
 
 import web
 import json
 import babel
 import babel.core
 import babel.dates
+from babel.lists import format_list
 from collections import defaultdict
 import re
 import random
@@ -210,6 +211,13 @@ def list_recent_pages(path, limit=100, offset=0):
     return web.ctx.site.get_many(web.ctx.site.things(q))
 
 
+@public
+def commify_list(items: Iterable[Any]):
+    # If the list item is a template/html element, we strip it
+    # so that there is no space before the comma.
+    return format_list([str(x).strip() for x in items], locale=web.ctx.lang)
+
+
 @public
 def json_encode(d):
     return json.dumps(d)


Test patch:
Repository: internetarchive/openlibrary

Issue:
api.py: vendors.search_amazon() is a stub function
```
def search_amazon(title='', author=''):
    pass
```
https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/openlibrary/api.py#L400-L425 -->
* https://github.com/internetarchive/openlibrary/blob/master/openlibrary/core/vendors.py#L283

<!-- What issue does this PR close? -->
See #2342
Closes #

<!-- What does this PR achieve? [feature|hotfix|fix|refactor] -->


### Technical
<!-- What should be noted about the implementation? -->

### Testing
<!-- Steps for reviewer to reproduce/verify what this PR does/fixes. -->

### Screenshot
<!-- If this PR touches UI, please post evidence (screenshots) of it behaving correctly. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->


<!-- Attribution Disclaimer: By proposing this pull request, I affirm to have made a best-effort and exercised my discretion to make sure relevant sections of this code which substantially leverage code suggestions, code generation, or code snippets from sources (e.g. Stack Overflow, GitHub) have been annotated with basic attribution so reviewers & contributors may have confidence and access to the correct context to evaluate and use this code. -->



Discussion:
TODO (cclauss): Remove `amazon_search_api()`

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/api.py b/openlibrary/plugins/openlibrary/api.py
--- a/openlibrary/plugins/openlibrary/api.py
+++ b/openlibrary/plugins/openlibrary/api.py
@@ -25,9 +25,8 @@
 from openlibrary.core.models import Booknotes, Work
 from openlibrary.core.sponsorships import qualifies_for_sponsorship
 from openlibrary.core.vendors import (
-    get_amazon_metadata,
     create_edition_from_amazon_metadata,
-    search_amazon,
+    get_amazon_metadata,
     get_betterworldbooks_metadata,
 )
 
@@ -397,34 +396,6 @@ def get_works_data(self, author, limit, offset):
         return {"links": links, "size": size, "entries": works}
 
 
-class amazon_search_api(delegate.page):
-    """Librarian + admin only endpoint to check for books
-    available on Amazon via the Product Advertising API
-    ItemSearch operation.
-
-    https://docs.aws.amazon.com/AWSECommerceService/latest/DG/ItemSearch.html
-
-    Currently experimental to explore what data is available to affiliates.
-
-    :return: JSON {"results": []} containing Amazon product metadata
-             for items matching the title and author search parameters.
-    :rtype: str
-    """
-
-    path = '/_tools/amazon_search'
-
-    @jsonapi
-    def GET(self):
-        user = accounts.get_current_user()
-        if not (user and (user.is_admin() or user.is_librarian())):
-            return web.HTTPError('403 Forbidden')
-        i = web.input(title='', author='')
-        if not (i.author or i.title):
-            return json.dumps({'error': 'author or title required'})
-        results = search_amazon(title=i.title, author=i.author)
-        return json.dumps(results)
-
-
 class sponsorship_eligibility_check(delegate.page):
     path = r'/sponsorship/eligibility/(.*)'
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
"Unable to render this page" while accessing a specific work
Every time I attempt to access OL20465177W, I get a type error and the page fails to load. 

### Evidence / Screenshot (if possible)

![image](https://user-images.githubusercontent.com/16920817/163726481-266adb85-e226-482f-89e6-77be708674d3.png)

### Relevant url?
https://openlibrary.org/works/OL20465177W/Diary_of_a_Wimpy_Kid

### Details

- **I am logged in**
- **Using Firefox 99.0.1 (64-bit)**
- **Windows 10**



Discussion:
https://openlibrary.org/works/OL20465177W/Diary_of_a_Wimpy_Kid?debug=true

```
/openlibrary/openlibrary/core/lending.py in get_groundtruth_availability
    try:
        response = requests.post(url + params, data=s3_keys)
        response.raise_for_status()
    except requests.HTTPError:
        pass  # TODO: Handle unexpected responses from the availability server.
    data = response.json().get('lending_status')
    # For debugging
    data['__src__'] = 'core.models.lending.get_groundtruth_availability
```

Error handling is TBD


Resolving patch:
diff --git a/openlibrary/core/lending.py b/openlibrary/core/lending.py
--- a/openlibrary/core/lending.py
+++ b/openlibrary/core/lending.py
@@ -239,7 +239,7 @@ def get_groundtruth_availability(ocaid, s3_keys=None):
         response.raise_for_status()
     except requests.HTTPError:
         pass  # TODO: Handle unexpected responses from the availability server.
-    data = response.json().get('lending_status')
+    data = response.json().get('lending_status', {})
     # For debugging
     data['__src__'] = 'core.models.lending.get_groundtruth_availability'
     return data


Test patch:
Repository: internetarchive/openlibrary

Issue:
Combine Loans and Waitlist pages
We currently have loans and waitlists as separate items in the My Books sidebar:

![image](https://user-images.githubusercontent.com/6251786/163466932-cb7430b2-2489-4283-a931-771fafd7cf45.png)

As we add more items to this sidebar (Eg #6427 ), this it's getting a little messy. "Waitlist" isn't as useful anymore for most patrons, so lets combine Loans and Waitlist:

![image](https://user-images.githubusercontent.com/6251786/163469613-8871d6cc-232d-4f1a-8a11-4eafa29011fe.png)

### Proposal & Constraints
- [ ] /account/loans should render the waitlist items
- [ ] /account/waitlist should redirect to /account/loans
- [ ] Remove the "Waitlist" link from the sidebar
- [ ] Rename "Checked out" to "Loans"

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
@jimchamp 



Discussion:
Hi @tangym27 ! Would you be interested in this one after working on Read more? Also before #6427 :)
Hi @cdrini! Yes, this sounds great. Thank you! 

Resolving patch:
diff --git a/openlibrary/plugins/upstream/account.py b/openlibrary/plugins/upstream/account.py
--- a/openlibrary/plugins/upstream/account.py
+++ b/openlibrary/plugins/upstream/account.py
@@ -858,13 +858,8 @@ def GET(self):
 class account_waitlist(delegate.page):
     path = "/account/waitlist"
 
-    @require_login
     def GET(self):
-        user = accounts.get_current_user()
-        username = user['key'].split('/')[-1]
-
-        return MyBooksTemplate(username, 'waitlist').render()
-
+        raise web.seeother("/account/loans")
 
 # Disabling be cause it prevents account_my_books_redirect from working
 # for some reason. The purpose of this class is to not show the "Create" link for


Test patch:
Repository: internetarchive/openlibrary

Issue:
Create patron role that blocks editing without blocking the entire account
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
Occasionally a valid patron account is abusive (vandalising records). An account type that blocks edits without blocking loans, lists, etc would be useful.
### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->
A usergroup or account designation that will allow a patron to use Open Library, but not edit book records.
### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->
A usergroup or account designation that will allow a patron to use Open Library, but not edit book records.

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:

Resolving patch:
diff --git a/openlibrary/core/models.py b/openlibrary/core/models.py
--- a/openlibrary/core/models.py
+++ b/openlibrary/core/models.py
@@ -758,6 +758,9 @@ def is_beta_tester(self):
     def has_librarian_tools(self):
         return self.is_usergroup_member('/usergroup/librarian-tools')
 
+    def is_read_only(self):
+        return self.is_usergroup_member('/usergroup/read-only')
+
     def get_lists(self, seed=None, limit=100, offset=0, sort=True):
         """Returns all the lists of this user.
 
diff --git a/openlibrary/plugins/upstream/spamcheck.py b/openlibrary/plugins/upstream/spamcheck.py
--- a/openlibrary/plugins/upstream/spamcheck.py
+++ b/openlibrary/plugins/upstream/spamcheck.py
@@ -35,6 +35,8 @@ def is_spam(i=None, allow_privileged_edits=False):
         # Allow admins and librarians to make edits:
         if allow_privileged_edits and (user.is_admin() or user.is_librarian()):
             return False
+        if user.is_read_only():
+            return True
         # Prevent deleted users from making edits:
         if user.type.key == '/type/delete':
             return True


Test patch:
Repository: internetarchive/openlibrary

Issue:
[WIP] refactor: Reformat the openlibrary/ directory with black
Subtask of #4776 

Run `black` on ***only*** `openlibrary/` directory:
```console
> black --skip-string-normalization openlibrary/
```

**TODO:**
- [x] Review the initial pass by `black`
- [ ] List out any formatting which does not seem proper
- [ ] Ignore this commit in `.git-blame-ignore-revs` similar to https://github.com/internetarchive/infogami/pull/169


<!-- What issue does this PR close? -->
Closes #

<!-- What does this PR achieve? [feature|hotfix|fix|refactor] -->

### Technical
<!-- What should be noted about the implementation? -->

### Testing
<!-- Steps for reviewer to reproduce/verify what this PR does/fixes. -->

### Screenshot
<!-- If this PR touches UI, please post evidence (screenshots) of it behaving correctly. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@cclauss @cdrini 


Discussion:
`--max-line-length=1195`
> openlibrary/utils/olcompress.py:10:89: E501 line too long (1195 > 88 characters)
Hey, @cdrini Can you please take a look at why the tests are failing? https://github.com/internetarchive/openlibrary/pull/5079/checks?check_run_id=2498124543#step:8:124

This function was added by you so you would know more about it :)

Resolving patch:
diff --git a/openlibrary/plugins/upstream/code.py b/openlibrary/plugins/upstream/code.py
--- a/openlibrary/plugins/upstream/code.py
+++ b/openlibrary/plugins/upstream/code.py
@@ -352,7 +352,7 @@ def setup():
     addbook.setup()
     covers.setup()
     merge_authors.setup()
-    #merge_works.setup() # ILE code
+    # merge_works.setup() # ILE code
     edits.setup()
 
     from openlibrary.plugins.upstream import data, jsdef


Test patch:
Repository: internetarchive/openlibrary

Issue:
Add Google Sign in support
Use an iframe based approach to allow login into open library using a google account, similar to archive.org


Discussion:

Resolving patch:
diff --git a/openlibrary/accounts/model.py b/openlibrary/accounts/model.py
--- a/openlibrary/accounts/model.py
+++ b/openlibrary/accounts/model.py
@@ -1,6 +1,7 @@
 """
 
 """
+import secrets
 import time
 import datetime
 import hashlib
@@ -793,7 +794,10 @@ def audit_accounts(
         r = InternetArchiveAccount.s3auth(s3_access_key, s3_secret_key)
         if not r.get('authorized', False):
             return {'error': 'invalid_s3keys'}
-        ia_login = {'success': True}
+        ia_login = {
+            'success': True,
+            'values': {'access': s3_access_key, 'secret': s3_secret_key},
+        }
         email = r['username']
     else:
         if not valid_email(email):
@@ -848,13 +852,13 @@ def audit_accounts(
         # lending.config_ia_auth_only is enabled, we need to create
         # and link it.
         if not ol_account:
-            if not password:
-                raise {'error': 'link_attempt_requires_password'}
             try:
                 ol_account = OpenLibraryAccount.create(
                     ia_account.itemname,
                     email,
-                    password,
+                    # since switching to IA creds, OL password
+                    # not used; make challenging random
+                    secrets.token_urlsafe(32),
                     displayname=ia_account.screenname,
                     verified=True,
                     retries=5,


Test patch:
Repository: internetarchive/openlibrary

Issue:
Change `librarian-work-merge` usergroup to more appropriate name
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->
Today, members of `/usergroup/librarian-work-merge` are able to merge works via the recently released merge works UI.  In the near future, we plan to allow this group to perform other potentially risky tasks that could result in data loss if performed incorrectly.

Since the scope of the usergroup is growing beyond merging works, it makes sense to give this group a more appropriate name.

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
Change `/usergroup/librarian-work-merge` to `/usergroup/super-librarians`.  Ensure that all references to the group are updated in the codebase.

Once the changes have been made, use `pg_dump` to generate a new version of https://github.com/internetarchive/openlibrary/blob/master/scripts/dev-instance/dev_db.pg_dump .If this file doesn't contain entries for the new usergroup, local dev instances will likely fail.

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/code.py b/openlibrary/plugins/upstream/code.py
--- a/openlibrary/plugins/upstream/code.py
+++ b/openlibrary/plugins/upstream/code.py
@@ -105,7 +105,7 @@ def GET(self):
         user = web.ctx.site.get_user()
         has_access = user and (
             (user.is_admin() or user.is_librarian())
-            and user.is_usergroup_member('/usergroup/librarian-work-merge')
+            and user.is_usergroup_member('/usergroup/super-librarians')
         )
         if not has_access:
             raise web.HTTPError('403 Forbidden')


Test patch:
Repository: internetarchive/openlibrary

Issue:
`/merges` entries should be displayed in descending order
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->
Today, entries in all `/merges` views are displayed in order of `created` timestamp, ascending.  They should instead by ordered by `created` timestamp, __descending__.

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->
https://openlibrary.org/merges

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to any `/merges` view.
2. Note the order of the merge requests in the table.

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: Entries are listed from oldest to newest.
* Expected: Entries are listed from newest to oldest.

### Details

- **Logged in (Y/N)?**N
- **Browser type/version?**Any
- **Operating system?** Any
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
When we fetch the requests for the `/merges` table, we should pass `order='created desc'` to the `CommunityEditsQueue.get_requests()` call.

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->
`/merges` controller is in [/openlibrary/plugins/upstream/edits.py](https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/upstream/edits.py)

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
Hi! So the issue we're facing here is that the requests at https://openlibrary.org/merges is listed from oldest to newest but we want it to be from newest to oldest right?
Yes, that's precisely the problem!
Have you tried order='created desc'? 
Which database are you using for this program? MySQL or PostgreSQL? We want to sort by date & time in descending order which should be okay to fix.
> Have you tried order='created desc'?

I'm sure that will work.  Would you like to open a pull request with that change?

We use Postgres for our production and local dev environment.  If we do use MySQL, it's for our Python unit tests only.

Resolving patch:
diff --git a/openlibrary/plugins/upstream/edits.py b/openlibrary/plugins/upstream/edits.py
--- a/openlibrary/plugins/upstream/edits.py
+++ b/openlibrary/plugins/upstream/edits.py
@@ -93,7 +93,7 @@ def GET(self):
             mode=i.mode,
             submitter=i.submitter,
             reviewer=i.reviewer,
-            order='created',
+            order='created desc',
         ).list()
         enriched_requests = self.enrich(merge_requests)
         total_found = CommunityEditsQueue.get_counts_by_mode(


Test patch:
Repository: internetarchive/openlibrary

Issue:
librarian merge queue fixes
Closes #6807

- allows flexible sorting with ?order=asc or desc -- piggy backs on #6785 
- adds total counts to Open and Closed
- removes "All"
- fixes bug where page? persists when switching modes -- fixes **half** of #6782 (i.e. mode part, not submitter!)


<!-- What does this PR achieve? [feature|hotfix|fix|refactor] -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@jimchamp 

<!-- Attribution Disclaimer: By proposing this pull request, I affirm to have made a best-effort and exercised my discretion to make sure relevant sections of this code which substantially leverage code suggestions, code generation, or code snippets from sources (e.g. Stack Overflow, GitHub) have been annotated with basic attribution so reviewers & contributors may have confidence and access to the correct context to evaluate and use this code. -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/edits.py b/openlibrary/plugins/upstream/edits.py
--- a/openlibrary/plugins/upstream/edits.py
+++ b/openlibrary/plugins/upstream/edits.py
@@ -96,19 +96,26 @@ def response(status='ok', **kwargs):
                 )
 
     def GET(self):
-        i = web.input(page=1, limit=25, mode="open", submitter=None, reviewer=None)
+        i = web.input(
+            page=1, limit=25, mode="open", submitter=None, reviewer=None, order='desc'
+        )
         merge_requests = CommunityEditsQueue.get_requests(
             page=int(i.page),
             limit=int(i.limit),
             mode=i.mode,
             submitter=i.submitter,
             reviewer=i.reviewer,
-            order='created desc',
+            order=f'created {i.order}',
         ).list()
 
-        total_found = CommunityEditsQueue.get_counts_by_mode(
-            mode=i.mode, submitter=i.submitter, reviewer=i.reviewer
-        )
+        total_found = {
+            "open": CommunityEditsQueue.get_counts_by_mode(
+                mode='open', submitter=i.submitter, reviewer=i.reviewer
+            ),
+            "closed": CommunityEditsQueue.get_counts_by_mode(
+                mode='closed', submitter=i.submitter, reviewer=i.reviewer
+            ),
+        }
         return render_template(
             'merge_queue/merge_queue',
             total_found,


Test patch: