internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: internetarchive/openlibrary

Issue:
No `Cache-Control` headers on IA CDN requests
### Problem

These two JS load on every page: https://openlibrary.org/cdn/archive.org/donate.js and. https://openlibrary.org/cdn/archive.org/analytics.js

However, they are not cached like the rest of the JS.
This causes a lot (2 seconds) delay when on a very slow connection (only a few 100ms on a decent connection).


Their headers:
```
Content-Encoding: gzip
Content-Type: text/javascript
Date: Wed, 20 Mar 2024 11:48:52 GMT
Referrer-Policy: no-referrer-when-downgrade
Server: nginx/1.18.0 (Ubuntu)
Vary: Accept-Encoding
X-Ol-Stats: ""
```

Headers for all other requests:
```
Cache-Control: max-age=315360000
Content-Encoding: gzip
Content-Type: application/javascript
Date: Wed, 20 Mar 2024 11:03:17 GMT
Etag: W/"65e78c3f-69215"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Last-Modified: Tue, 05 Mar 2024 21:18:55 GMT
Referrer-Policy: no-referrer-when-downgrade
Server: nginx/1.18.0 (Ubuntu)
```

#### Evidence / Screenshot
<img width="1511" alt="image" src="https://github.com/internetarchive/openlibrary/assets/921217/79644e7c-160c-44f4-ac3d-1c795665b1f0">


#### Relevant URL(s)


### Reproducing the bug

_No response_

### Context

_No response_

### Notes from this Issue's Lead

#### Proposal & constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
Add cache control headers.

Probably should also add `defer` or `async` to their script tags so they don't block loading when they're not cached but this could be a different issue.
https://javascript.info/script-async-defer


#### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->
https://github.com/internetarchive/openlibrary/blob/45ed08186221233c04f1859e3b8a703152b01539/openlibrary/plugins/openlibrary/code.py#L425-L430

https://github.com/internetarchive/openlibrary/blob/45ed08186221233c04f1859e3b8a703152b01539/openlibrary/templates/site/footer.html#L41-L42

https://github.com/internetarchive/openlibrary/blob/45ed08186221233c04f1859e3b8a703152b01539/openlibrary/plugins/upstream/utils.py#L1468-L1469

#### Stakeholders
<!-- @ tag stakeholders of this bug -->
PS: I don't have bandwidth to take this right now.


Discussion:
Hey! I would like to work on this issue :)
@QuantuM410 go for it!

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/code.py b/openlibrary/plugins/openlibrary/code.py
--- a/openlibrary/plugins/openlibrary/code.py
+++ b/openlibrary/plugins/openlibrary/code.py
@@ -427,6 +427,7 @@ class ia_js_cdn(delegate.page):
 
     def GET(self, filename):
         web.header('Content-Type', 'text/javascript')
+        web.header("Cache-Control", "max-age=%d" % (24 * 3600))
         return web.ok(fetch_ia_js(filename))
 
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Support different seeds for random.hourly sort
These carousels are all sorted by random.hourly, but we want them to have a different random subset!

![image](https://github.com/internetarchive/openlibrary/assets/6251786/408dcd86-f96a-4534-bacf-46f437f9f444)


### Proposal & Constraints
Expand `random.hourly` sorting to support a custom seed like `random`

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
@RayBB 


Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/worksearch/schemes/__init__.py b/openlibrary/plugins/worksearch/schemes/__init__.py
--- a/openlibrary/plugins/worksearch/schemes/__init__.py
+++ b/openlibrary/plugins/worksearch/schemes/__init__.py
@@ -44,17 +44,27 @@ def process_user_sort(self, user_sort: str) -> str:
         >>> scheme.process_user_sort('random')
         'random_1 asc'
         >>> scheme.process_user_sort('random_custom_seed')
-        'random_custom_seed asc'
+        'random_1_custom_seed asc'
         >>> scheme.process_user_sort('random_custom_seed desc')
-        'random_custom_seed desc'
+        'random_1_custom_seed desc'
         >>> scheme.process_user_sort('random_custom_seed asc')
-        'random_custom_seed asc'
+        'random_1_custom_seed asc'
         """
 
-        def process_individual_sort(sort: str):
-            if sort.startswith('random_'):
+        def process_individual_sort(sort: str) -> str:
+            if sort.startswith(('random_', 'random.hourly_', 'random.daily_')):
                 # Allow custom randoms; so anything random_* is allowed
-                return sort if ' ' in sort else f'{sort} asc'
+                # Also Allow custom time randoms to allow carousels with overlapping
+                # books to have a fresh ordering when on the same collection
+                sort_order: str | None = None
+                if ' ' in sort:
+                    sort, sort_order = sort.split(' ', 1)
+                random_type, random_seed = sort.split('_', 1)
+                solr_sort = self.sorts[random_type]
+                solr_sort_str = solr_sort() if callable(solr_sort) else solr_sort
+                solr_sort_field, solr_sort_order = solr_sort_str.split(' ', 1)
+                sort_order = sort_order or solr_sort_order
+                return f'{solr_sort_field}_{random_seed} {sort_order}'
             else:
                 solr_sort = self.sorts[sort]
                 return solr_sort() if callable(solr_sort) else solr_sort


Test patch:
Repository: internetarchive/openlibrary

Issue:
Error message is not displayed on Google sign-in failure
### Problem

Whenever a Google sign-in attempt fails, the `/account/login` page is reloaded and no error message is rendered.

#### Evidence / Screenshot

#### Relevant URL(s)


### Reproducing the bug

1. Go to `/account/login`
2. Do attempt to log in with a Google account (preferably one with broken IA<->OL linkage)

* Expected behavior: On failure, a useful error message is displayed
* Actual behavior: The page simply reloads, with no indication that anything went wrong


### Context

- Browser (Chrome, Safari, Firefox, etc): Any
- OS (Windows, Mac, etc): Any
- Logged in (Y/N): N
- Environment (prod, dev, local): prod


### Notes from this Issue's Lead

#### Proposal & constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
Update client-side third-party authentication code to display an error whenever Google sign-in fails.

#### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

#### Stakeholders
<!-- @ tag stakeholders of this bug -->
@mekarpeles @seabelis 



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/account.py b/openlibrary/plugins/upstream/account.py
--- a/openlibrary/plugins/upstream/account.py
+++ b/openlibrary/plugins/upstream/account.py
@@ -341,7 +341,11 @@ def POST(self):
             )
             error = audit.get('error')
             if error:
-                raise olib.code.BadRequest(error)
+                resp = {
+                    'error': error,
+                    'errorDisplayString': LOGIN_ERRORS[error],
+                }
+                raise olib.code.BadRequest(json.dumps(resp))
             expires = 3600 * 24 * 365 if remember.lower() == 'true' else ""
             web.setcookie(config.login_cookie_name, web.ctx.conn.get_auth_token())
             if audit.get('ia_email'):


Test patch:
Repository: internetarchive/openlibrary

Issue:
Reading log JSON view errors when too many work IDs passed to solr
### Problem

Since the last run of the new redirect resolution script, the JSON view of my already-read shelf is generating an error. The script fixed a number of redirects, and I haven't made any manual changes to the shelf since, suggesting that the script might have written some bad data.

My other shelves are sending JSON correctly, so it's not a general API issue.

#### Relevant URL
https://openlibrary.org/people/mheiman9541/books/already-read.json?debug=true

### Reproducing the bug

_No response_

### Context

- Browser (Chrome, Safari, Firefox, etc): Firefox
- OS (Windows, Mac, etc): Mac
- Logged in (Y/N): Y
- Environment (prod, dev, local): prod


### ⚠️ Experienced contributors only

### Proposal & constraints

### Related files

### Stakeholders
@mekarpeles @mheiman @cdrini 



Discussion:
Update from @cdrini : Solr returned an http 414 URI too long. The script fixed so many issues that we're now sending too many work IDs to solr. We're currently making a GET request to solr's /get endpoint, but I wonder if there's a way we can make a POST request and put the work keys in the request body. That should avoid any URI length errors.
The way it currently looks, Solr innately supports POST requests. The bodies of the requests will have to be formatted in the manner of 
```
{
   "params": {
         "q": "field:value",
         "rows": 10,
         "wt": "json"
     }
}
``` 
As such, I think this should be fairly doable. 



Resolving patch:
diff --git a/openlibrary/utils/solr.py b/openlibrary/utils/solr.py
--- a/openlibrary/utils/solr.py
+++ b/openlibrary/utils/solr.py
@@ -19,6 +19,9 @@
 
 class Solr:
     def __init__(self, base_url):
+        """
+        :param base_url: The base url of the solr server/collection. E.g. http://localhost:8983/solr/openlibrary
+        """
         self.base_url = base_url
         self.host = urlsplit(self.base_url)[1]
         self.session = requests.Session()
@@ -59,9 +62,9 @@ def get_many(
         if not keys:
             return []
         logger.info(f"solr /get: {keys}, {fields}")
-        resp = self.session.get(
+        resp = self.session.post(
             f"{self.base_url}/get",
-            params={
+            data={
                 'ids': ','.join(keys),
                 **({'fl': ','.join(fields)} if fields else {}),
             },


Test patch:
Repository: internetarchive/openlibrary

Issue:
Remove unused `LOGIN_ERRORS`
### Problem

Not a problem _per se_, but two entries in the [`LOGIN_ERRORS` dictionary](https://github.com/internetarchive/openlibrary/blob/3e7918bfa36b83ebc1438c76c2627945c1f6f721/openlibrary/plugins/upstream/account.py#L60-L76) are no longer referenced in the codebase.

The last `wrong_ia_account` reference was removed in #8980.  `ia_login_only` is also not referenced in the code.

#### Evidence / Screenshot

#### Relevant URL(s)


### Reproducing the bug

1. Go to ...
2. Do ...

* Expected behavior:
* Actual behavior:


### Context

- Browser (Chrome, Safari, Firefox, etc):
- OS (Windows, Mac, etc):
- Logged in (Y/N): Y
- Environment (prod, dev, local): prod


### Notes from this Issue's Lead

#### Proposal & constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

Remove the two unused `LOGIN_ERROR` entries.

#### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

https://github.com/internetarchive/openlibrary/blob/3e7918bfa36b83ebc1438c76c2627945c1f6f721/openlibrary/plugins/upstream/account.py

#### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/account.py b/openlibrary/plugins/upstream/account.py
--- a/openlibrary/plugins/upstream/account.py
+++ b/openlibrary/plugins/upstream/account.py
@@ -69,10 +69,8 @@
     "missing_fields": "Please fill out all fields and try again",
     "email_registered": "This email is already registered",
     "username_registered": "This username is already registered",
-    "ia_login_only": "Sorry, you must use your Internet Archive email and password to log in",
     "max_retries_exceeded": "A problem occurred and we were unable to log you in.",
     "invalid_s3keys": "Login attempted with invalid Internet Archive s3 credentials.",
-    "wrong_ia_account": "An Open Library account with this email is already linked to a different Internet Archive account. Please contact info@openlibrary.org.",
 }
 
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Bingbot getting stuck on shuffle links
The "Shuffle" link we show patrons is causing some bots to get stuck crawling the site. We already label it as `rel="nofollow"`, which is supposed to prevent them from doing this, but some (eg BingBot) appear not to respect this flag. So let's hide it entirely for bots.

This wastes our crawl budgets as well as provides un-useful strain on our servers.


Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/code.py b/openlibrary/plugins/openlibrary/code.py
--- a/openlibrary/plugins/openlibrary/code.py
+++ b/openlibrary/plugins/openlibrary/code.py
@@ -1090,6 +1090,16 @@ def is_bot():
         'bingbot',
         'mj12bot',
         'yoozbotadsbot',
+        'ahrefsbot',
+        'amazonbot',
+        'applebot',
+        'bingbot',
+        'brightbot',
+        'gptbot',
+        'petalbot',
+        'semanticscholarbot',
+        'yandex.com/bots',
+        'icc-crawler',
     ]
     if not web.ctx.env.get('HTTP_USER_AGENT'):
         return True


Test patch:
Repository: internetarchive/openlibrary

Issue:
Edition Editing: language autocomplete is slow
### Problem

According to [this](https://internetarchive.slack.com/archives/C0119PRDV46/p1713446825373169?thread_ts=1713436300.354359&cid=C0119PRDV46) thread, it is very slow:

<img width="455" alt="Screenshot 2024-04-22 at 5 32 04 AM" src="https://github.com/internetarchive/openlibrary/assets/978325/54575542-e9a8-4452-a12a-5ed262897196">

#### Evidence / Screenshot

#### Relevant URL(s)
https://openlibrary.org/books/OL24938286M/Pacific_Vortex!/edit

### Reproducing the bug

1. Go to a work edit page
2. Try to add a language

* Expected behavior: Fast
* Actual behavior: Slow


### Context

Other keywords: dropdown, pulldown

### Notes from this Issue's Lead

#### Proposal & constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

#### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

#### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/worksearch/autocomplete.py b/openlibrary/plugins/worksearch/autocomplete.py
--- a/openlibrary/plugins/worksearch/autocomplete.py
+++ b/openlibrary/plugins/worksearch/autocomplete.py
@@ -96,6 +96,7 @@ class languages_autocomplete(delegate.page):
     def GET(self):
         i = web.input(q="", limit=5)
         i.limit = safeint(i.limit, 5)
+        web.header("Cache-Control", "max-age=%d" % (24 * 3600))
         return to_json(
             list(itertools.islice(utils.autocomplete_languages(i.q), i.limit))
         )


Test patch:
Repository: internetarchive/openlibrary

Issue:
Update Github Action "Recent Comment Slack Bot" to mark issues as Needs: Response 
<!-- IMPORTANT: Before posting, be sure to redact or remove sensitive data, such as passwords, secret keys, session cookies, etc. -->

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->

```
def add_label_to_issues(ids):
    owner = 'internetarchive'
    repo = 'openlibrary'
    labels_url = f"https://api.github.com/repos/{owner}/{repo}/issues/labels"
    payload = {"labels": ["Needs: Response"]}
    for issue_number in ids:
        issue_labels_url = f"https://api.github.com/repos/{owner}/{repo}/issues/{issue_number}/labels"
        response = requests.post(issue_labels_url, json=payload, headers=github_headers)

if __name__ == "__main__":
    add_label_to_issues([9054, 9065])
```

to https://github.com/internetarchive/openlibrary/blob/master/scripts/gh_scripts/issue_comment_bot.py#L255

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:

Resolving patch:
diff --git a/scripts/gh_scripts/issue_comment_bot.py b/scripts/gh_scripts/issue_comment_bot.py
--- a/scripts/gh_scripts/issue_comment_bot.py
+++ b/scripts/gh_scripts/issue_comment_bot.py
@@ -245,6 +245,16 @@ def time_since(hours):
     return since, since.strftime('%Y-%m-%dT%H:%M:%S')
 
 
+def add_label_to_issues(issues):
+    for issue in issues:
+        issue_labels_url = f"https://api.github.com/repos/internetarchive/openlibrary/issues/{issue['number']}/labels"
+        response = requests.post(
+            issue_labels_url,
+            json={"labels": ["Needs: Response"]},
+            headers=github_headers,
+        )
+
+
 def start_job(args: argparse.Namespace):
     """
     Starts the new comment digest job.
@@ -253,6 +263,7 @@ def start_job(args: argparse.Namespace):
     issues = fetch_issues(date_string)
 
     filtered_issues = filter_issues(issues, since)
+    add_label_to_issues(filtered_issues)
     publish_digest(filtered_issues, args.channel, args.slack_token, args.hours)
     print('Digest posted to Slack.')
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Cleanup use of `is_usergroup_member('/usergroup/librarians')`
<!-- IMPORTANT: Before posting, be sure to redact or remove sensitive data, such as passwords, secret keys, session cookies, etc. -->

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->
We currently still use: `is_usergroup_member('/usergroup/librarians')` in a few places.
We should probably refactor them to use `is_librarian()` instead like we do in most places.

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->
<img width="1171" alt="image" src="https://github.com/internetarchive/openlibrary/assets/921217/94e93422-4737-4ce6-be5d-0d883ee413dc">


### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:
Hey @RayBB! I can make a quick PR, can I get assigned 
@QuantuM410 you're assigned. Please be sure to test the code before the PR and provide links to the effected pages.

Resolving patch:
diff --git a/openlibrary/plugins/admin/code.py b/openlibrary/plugins/admin/code.py
--- a/openlibrary/plugins/admin/code.py
+++ b/openlibrary/plugins/admin/code.py
@@ -80,7 +80,7 @@ def handle(self, cls, args=(), librarians=False):
         else:
             if (
                 context.user
-                and context.user.is_usergroup_member('/usergroup/librarians')
+                and context.user.is_librarian()
                 and web.ctx.path == '/admin/solr'
             ):
                 return m(*args)
diff --git a/openlibrary/plugins/upstream/merge_authors.py b/openlibrary/plugins/upstream/merge_authors.py
--- a/openlibrary/plugins/upstream/merge_authors.py
+++ b/openlibrary/plugins/upstream/merge_authors.py
@@ -288,9 +288,7 @@ def POST(self):
         can_merge = user and (
             user.is_admin() or user.is_usergroup_member('/usergroup/super-librarians')
         )
-        can_request_merge = not can_merge and (
-            user and user.is_usergroup_member('/usergroup/librarians')
-        )
+        can_request_merge = not can_merge and (user and user.is_librarian())
 
         # filter bad keys
         keys = self.filter_authors(keys)


Test patch:
Repository: internetarchive/openlibrary

Issue:
Update Github Action "Recent Comment Slack Bot" to mark issues as Needs: Response 
<!-- IMPORTANT: Before posting, be sure to redact or remove sensitive data, such as passwords, secret keys, session cookies, etc. -->

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->

```
def add_label_to_issues(ids):
    owner = 'internetarchive'
    repo = 'openlibrary'
    labels_url = f"https://api.github.com/repos/{owner}/{repo}/issues/labels"
    payload = {"labels": ["Needs: Response"]}
    for issue_number in ids:
        issue_labels_url = f"https://api.github.com/repos/{owner}/{repo}/issues/{issue_number}/labels"
        response = requests.post(issue_labels_url, json=payload, headers=github_headers)

if __name__ == "__main__":
    add_label_to_issues([9054, 9065])
```

to https://github.com/internetarchive/openlibrary/blob/master/scripts/gh_scripts/issue_comment_bot.py#L255

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:

Resolving patch:
diff --git a/scripts/gh_scripts/issue_comment_bot.py b/scripts/gh_scripts/issue_comment_bot.py
--- a/scripts/gh_scripts/issue_comment_bot.py
+++ b/scripts/gh_scripts/issue_comment_bot.py
@@ -127,6 +127,7 @@ def filter_issues(issues: list, since: datetime):
                 lead_label = find_lead_label(i.get('labels', []))
                 results.append(
                     {
+                        'number': i['number'],
                         'comment_url': last_comment['html_url'],
                         'commenter': last_commenter,
                         'issue_title': i['title'],


Test patch: