internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: internetarchive/openlibrary

Issue:
Python 3.10 : Threading related deprecations
### Proposal & Constraints

Ref : python/cpython#25174

Camelcase aliases have been deprecated and raise deprecation warning.

getName -> access name directly
currentThread -> current_thread

### Related files

```
openlibrary/openlibrary/core/cache.py:            self.active_threads.pop(threading.currentThread().getName(), None)
```

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/core/cache.py b/openlibrary/core/cache.py
--- a/openlibrary/core/cache.py
+++ b/openlibrary/core/cache.py
@@ -140,7 +140,7 @@ def _update_async_worker(self, *args, **kw):
             self.update(*args, **kw)
         finally:
             # Remove current thread from active threads
-            self.active_threads.pop(threading.currentThread().getName(), None)
+            self.active_threads.pop(threading.current_thread().name, None)
 
             # remove the flag
             self.memcache.delete(key)


Test patch:
Repository: internetarchive/openlibrary

Issue:
Export list in json format does not work
The option to export list in json format does not work.

### Evidence / Screenshot
![screenshot](https://user-images.githubusercontent.com/7125952/119910099-ab7b2880-bf56-11eb-8bbc-2d34ed37c602.png)

### Steps to Reproduce
1. Go to any list (e.g. [Rif War](https://openlibrary.org/people/dcapillae/lists/OL197248L))
2. Select option to export as json 

* Actual: You receive an error message like this: "Hmm... Sorry. There seems to be a problem with what you were just looking at. We've noted the error 2021-05-27/234105967423 and will look into it as soon as possible. Head for home?".

* Expected: Get a data file with the list books in json format.

### Details

- **Logged in (Y/N)?** Y
- **Browser type/version?** Mozilla Firefox 78.9.0esr (32-bit)
- **Operating system?** Debian 9
- **Environment (prod/dev/local)?** prod


Discussion:
We can see the error by going directly to this url (w/ `debug=true`)
https://openlibrary.org/people/dcapillae/lists/OL197248L/Rif_War/export?format=json&debug=true

![openlibrary org_people_dcapillae_lists_OL197248L_Rif_War_export_format=json debug=true](https://user-images.githubusercontent.com/978325/120380403-f938bd80-c2d5-11eb-9502-3eef48e58ae4.png)

@dherbst this was possibly an accidental regression based on changes to `plugins/openlibrary/list.py` -- do you mind taking a look?
Specifically I think it was this one: f9de97d . Actually this looks super quick to fix :P I'll open a PR.

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/lists.py b/openlibrary/plugins/openlibrary/lists.py
--- a/openlibrary/plugins/openlibrary/lists.py
+++ b/openlibrary/plugins/openlibrary/lists.py
@@ -1,5 +1,6 @@
 """Lists implementaion.
 """
+import json
 import random
 import web
 
@@ -473,7 +474,7 @@ def GET(self, key):
         elif format == "json":
             data = {"editions": self.get_editions(lst, raw=True)}
             web.header("Content-Type", "application/json")
-            return delegate.RawText(formats.dump_json(data))
+            return delegate.RawText(json.dumps(data))
         elif format == "yaml":
             data = {"editions": self.get_editions(lst, raw=True)}
             web.header("Content-Type", "application/yaml")


Test patch:
Repository: internetarchive/openlibrary

Issue:
staging/testing appearing in Google search!
These developer-only endpoints should not be appearing in Google! They likely cause SEO issues or just send our users to the wrong place.

### Evidence / Screenshot (if possible)

### Relevant url?
- Staging: https://www.google.com/search?q=site%3Astaging.openlibrary.org (1760 results)
- Testing: https://www.google.com/search?q=site%3Atesting.openlibrary.org (2100 results)

### Proposal & Constraints
@RayBB recommends: 

> FYI if you want to have this pages removed from Google you need to serve a noindex for a while before serving the robots.txt
>
> Counter intuitively, if you just add robots.txt it'll keep those pages indexed longer.
> 
> https://developers.google.com/search/docs/advanced/crawling/block-indexing
> https://www.reddit.com/r/TechSEO/comments/mcc98l/googles_unusual_handling_of_robots_noindex/
> 
> Considering it looks like google has 1,650 staging and 2,340 testing urls it may be a good idea to do the noindex first. Especially since those urls are slower and that could cause further SEO issues.

So: 
- [x] #5406 Add `<meta name="robots" content="noindex">` to head when `web.ctx.host != 'openlibrary.org'`
- [x] Wait until the number of search results goes down
- [ ]  #5397 Update robots.txt

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
@RayBB 



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/code.py b/openlibrary/plugins/openlibrary/code.py
--- a/openlibrary/plugins/openlibrary/code.py
+++ b/openlibrary/plugins/openlibrary/code.py
@@ -345,7 +345,9 @@ class robotstxt(delegate.page):
     def GET(self):
         web.header('Content-Type', 'text/plain')
         try:
-            robots_file = 'norobots.txt' if 'dev' in infogami.config.features else 'robots.txt'
+            is_dev = ('dev' in infogami.config.features or
+                      web.ctx.host != 'openlibrary.org')
+            robots_file = 'norobots.txt' if is_dev else 'robots.txt'
             data = open('static/' + robots_file).read()
             raise web.HTTPError('200 OK', {}, data)
         except IOError:


Test patch:
Repository: internetarchive/openlibrary

Issue:
Refactor i18n update command to specify a language
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->
When `./scripts/i18n-messages update` is executed, it updates the `.po` files for all languages.  Each updated files should be reviewed for unwanted changes (particularly changes flagged with the `fuzzy` keyword).  Most translators will be working with a single `.po` file, and it's a bit unreasonable for them to review unrelated files.

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
Update the i18n update script, allowing translators to pass a language code when executing the script.  When a language code is passed, only the corresponding `.po` file will be updated.

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@cdrini 




Discussion:

Resolving patch:
diff --git a/openlibrary/i18n/__init__.py b/openlibrary/i18n/__init__.py
--- a/openlibrary/i18n/__init__.py
+++ b/openlibrary/i18n/__init__.py
@@ -83,19 +83,26 @@ def extract_messages(dirs: List[str]):
 
     print('wrote template to', path)
 
-def compile_translations():
-    for locale in get_locales():
+
+def compile_translations(locales: List[str]):
+    locales_to_update = locales or get_locales()
+
+    for locale in locales_to_update:
         po_path = os.path.join(root, locale, 'messages.po')
         mo_path = os.path.join(root, locale, 'messages.mo')
 
         if os.path.exists(po_path):
             _compile_translation(po_path, mo_path)
 
-def update_translations():
+
+def update_translations(locales: List[str]):
+    locales_to_update = locales or get_locales()
+    print(f"Updating {locales_to_update}")
+
     pot_path = os.path.join(root, 'messages.pot')
     template = read_po(open(pot_path, 'rb'))
 
-    for locale in get_locales():
+    for locale in locales_to_update:
         po_path = os.path.join(root, locale, 'messages.po')
         mo_path = os.path.join(root, locale, 'messages.mo')
 
@@ -107,8 +114,10 @@ def update_translations():
             write_po(f, catalog)
             f.close()
             print('updated', po_path)
+        else:
+            print(f"ERROR: {po_path} does not exist...")
 
-    compile_translations()
+    compile_translations(locales_to_update)
 
 
 def generate_po(args):


Test patch:
Repository: internetarchive/openlibrary

Issue:
Create `i18n-messages validate` script to check for presence of fuzzy flags
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->
When `i18n-messages update` is executed, some translations in `.po` files will have a `fuzzy` annotation.  This means that the translation may not be completely accurate, and should be manually approved by a translator.  The last step of the `i18n-messages update` flow compiles all `.po` files into `.mo` files, regardless of the presence of fuzzy translations.

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
Add validation step to i18n flow, in between the update and compile steps.  Validation step will read each `.po` file, searching for lines that begin with `#, fuzzy`.  If any are found in a `.po` file, that file should not be compiled.

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->
This is a sub-task of #5134.
### Stakeholders
<!-- @ tag stakeholders of this bug -->
@cdrini




Discussion:

Resolving patch:
diff --git a/openlibrary/i18n/__init__.py b/openlibrary/i18n/__init__.py
--- a/openlibrary/i18n/__init__.py
+++ b/openlibrary/i18n/__init__.py
@@ -30,6 +30,50 @@ def _compile_translation(po, mo):
         raise e
 
 
+def _validate_catalog(catalog, locale):
+    validation_errors = []
+    for message in catalog:
+        if message.fuzzy:
+            if message.lineno:
+                validation_errors.append(
+                    f'openlibrary/i18n/{locale}/messages.po:{message.lineno}:'
+                    f' "{message.string}" is fuzzy.'
+                )
+            else:
+                validation_errors.append(
+                    '  File is fuzzy.  Remove line containing "#, fuzzy" found near '
+                    'the beginning of the file.'
+                )
+
+    if validation_errors:
+        print("Validation failed...")
+        print("Please correct the following errors before proceeding:")
+        for e in validation_errors:
+            print(e)
+
+    return len(validation_errors) == 0
+
+
+def validate_translations(args):
+    if args:
+        locale = args[0]
+        po_path = os.path.join(root, locale, 'messages.po')
+
+        if os.path.exists(po_path):
+            catalog = read_po(open(po_path, 'rb'))
+            is_valid = _validate_catalog(catalog, locale)
+
+            if is_valid:
+                print(f'Translations for locale "{locale}" are valid!')
+            return is_valid
+        else:
+            print(f'Portable object file for locale "{locale}" does not exist.')
+            return False
+    else:
+        print('Must include locale code when executing validate.')
+        return False
+
+
 def get_locales():
     return [
         d


Test patch:
Repository: internetarchive/openlibrary

Issue:
Prevent duplicate default carousel keys
<!-- What issue does this PR close? -->
Closes #

<!-- What does this PR achieve? [feature|hotfix|fix|refactor] -->
When multiple carousels on a single page share the same key, they tend to be buggy in different ways.  The most visible bug causes a carousel to render without its "next" and "previous" buttons.  In some cases, carousels with the same key will fetch more books using another carousel's query string.

This PR seeks to avoid these bugs by attempting to create a unique key for all carousels created without a key.  It doesn't absolutely prevent duplicate keys on a single page, but should greatly reduce the amount that can occur.

### Technical
<!-- What should be noted about the implementation? -->
In cases where carousels are created without a key, the absolute value of the carousel's hashed query string is used as the key.  This approach has some limitations:
1. Multiple carousels using the same query will have the same generated key.
2. My understanding is that keys are used for site analytics.  Randomly generated keys provide little insight into what carousels are receiving attention from patrons.
3. By default, Python's `hash` function will return different values for strings between sessions.  This further complicates the analytics issue as the same carousel will have different default keys over time.
4. While highly unlikely, two different query strings can produce the same hashed value.  The odds are higher in this case as the absolute value of the hash is being used as the key.

### Testing
<!-- Steps for reviewer to reproduce/verify what this PR does/fixes. -->
1. Add two or more carousels to a single page using the QueryCarousel macro.  Ensure that each carousel is created with no key and different query strings.
2. Inspect the carousel elements, specifically the divs with class `carousel`.  These will also have a class name that contains the key (`carousel-${key}`). Ensure that the keys are different for each carousel.

### Screenshot
<!-- If this PR touches UI, please post evidence (screenshots) of it behaving correctly. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@libjenner 
@mekarpeles 



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/code.py b/openlibrary/plugins/openlibrary/code.py
--- a/openlibrary/plugins/openlibrary/code.py
+++ b/openlibrary/plugins/openlibrary/code.py
@@ -892,6 +892,7 @@ def setup_template_globals():
         'sorted': sorted,
         'zip': zip,
         'tuple': tuple,
+        'hash': hash,
         'urlquote': web.urlquote,
         'isbn_13_to_isbn_10': isbn_13_to_isbn_10,
         'isbn_10_to_isbn_13': isbn_10_to_isbn_13,
diff --git a/openlibrary/plugins/upstream/utils.py b/openlibrary/plugins/upstream/utils.py
--- a/openlibrary/plugins/upstream/utils.py
+++ b/openlibrary/plugins/upstream/utils.py
@@ -873,7 +873,6 @@ def setup():
         'sum': sum,
         'get_donation_include': get_donation_include,
         'websafe': web.websafe,
-        'hash': hash
     })
 
     from openlibrary.core import helpers as h


Test patch:
Repository: internetarchive/openlibrary

Issue:
Delete old solr configs/etc.
Now that we've been stably on solr 8 for a while, remove all the old solr 3 configs/etc. ~Blocked by #4337~


Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/worksearch/code.py b/openlibrary/plugins/worksearch/code.py
--- a/openlibrary/plugins/worksearch/code.py
+++ b/openlibrary/plugins/worksearch/code.py
@@ -436,12 +436,8 @@ def run_solr_query(param=None, rows=100, page=1, sort=None, spellcheck_count=Non
         if use_dismax:
             params.append(('q', ' '.join(q_list)))
             params.append(('defType', 'dismax'))
-            if config.plugin_worksearch.get('solr8'):
-                params.append(('qf', 'text title^20 author_name^20'))
-                params.append(('bf', 'min(100,edition_count)'))
-            else:
-                params.append(('qf', 'text title^5 author_name^5'))
-                params.append(('bf', 'sqrt(edition_count)^10'))
+            params.append(('qf', 'text title^20 author_name^20'))
+            params.append(('bf', 'min(100,edition_count)'))
         else:
             params.append(('q', ' '.join(q_list + ['_val_:"sqrt(edition_count)"^10'])))
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Image uploader does not recognise uploaded file
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->
As of today (8-09-2021) the image uploader does not recognise that an image has been selected and uploaded. Instead, it displays "Please provide an image URL" after hitting submit.

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...any edition
2. Do ...upload an image as a cover and submit.

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: "Please provide an image URL"
* Expected: Image should be added as cover.

### Details

- **Logged in (Y/N)?** y
- **Browser type/version?** Chrome Version 92.0.4515.159 (Official Build) (x86_64)
- **Operating system?** MacOS
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/covers.py b/openlibrary/plugins/upstream/covers.py
--- a/openlibrary/plugins/upstream/covers.py
+++ b/openlibrary/plugins/upstream/covers.py
@@ -54,7 +54,7 @@ def upload(self, key, i):
         else:
             data = None
 
-        if i.url and i.url.strip() == "http://":
+        if i.url and i.url.strip() == "https://":
             i.url = ""
 
         user = accounts.get_current_user()


Test patch:
Repository: internetarchive/openlibrary

Issue:
"add-cover" edits in Recent Changes table should display Cover as comment (for review)
It would be very useful to review new covers uploaded to watch for spam.

So, in entries where add-cover is used, like all these - http://openlibrary.org/recentchanges/add-cover - let's start displaying the actual cover as the comment.



Discussion:
Just like they display here: http://openlibrary.org/works/OL1248930W/A_history_of_England_from_the_coming_of_the_English_to_1918?m=diff&b=3

Can't finish it in July. Moving it to August.

Moved to september milestone.

Hi, can I work on this issue? 
Sure! We'd love that @kryptc!
@kryptc do let us know in case we can help you out if you're stuck
Sorry for the delay, I was having some trouble installing docker on my system 
Hi, I had a question regarding where the file corresponding to the add-cover section is. In the recentchanges directory in templates, there is no add-cover directory. Where do I make the changes?
@kryptc I think you need to ask these kind of questions on Slack or Gitter. There you will find the help you needed.
@kryptc can we send you an invitation to our slack channel?
Hey, Can I work on this issue?

Resolving patch:
diff --git a/openlibrary/plugins/upstream/covers.py b/openlibrary/plugins/upstream/covers.py
--- a/openlibrary/plugins/upstream/covers.py
+++ b/openlibrary/plugins/upstream/covers.py
@@ -11,7 +11,8 @@
 from infogami.utils.view import safeint
 from openlibrary import accounts
 from openlibrary.plugins.upstream.models import Image
-from openlibrary.plugins.upstream.utils import get_coverstore_url, render_template
+from openlibrary.plugins.upstream.utils import (
+    get_coverstore_url, get_coverstore_public_url, render_template)
 
 logger = getLogger("openlibrary.plugins.upstream.covers")
 
@@ -83,7 +84,9 @@ def upload(self, key, i):
 
     def save(self, book, coverid, url=None):
         book.covers = [coverid] + [cover.id for cover in book.get_covers()]
-        book._save("Added new cover", action="add-cover", data={"url": url})
+        book._save('%s/b/id/%s-S.jpg' % (
+            get_coverstore_public_url(), coverid),
+            action="add-cover", data={"url": url})
 
 
 class add_work_cover(add_cover):


Test patch:
Repository: internetarchive/openlibrary

Issue:
Create permission group for work merge
Create permission group for work merge so we can deploy work merge UI.

The Work Merge UI is a little no fences on the edge of cliffs, so we don't want to open it up to all librarians. That's one of the blockers for merging the pr (#3408 ). If we can 

- [ ] Add permission group to prod
- [ ] Add @seabelis @libjenner GLBW @mekarpeles @cdrini @jimchamp  to the new group
- [ ] Add permission group to dev instance
- [ ] Update #3408 to limit ui to the new perm group


Discussion:
We do have a candidate group of a few existing librarians who are consistent, detail-oriented, conscientious, asking when they're unsure, and who are also currently overwhelming us (me) with merge requests. I feel like a blocker for the great work they could do. Are the steps in this issue also going to accommodate giving them these powers in the near future? 
For those users not in the permitted group, there should be some easy means for proposing the work mergers they may not do themselves.

Resolving patch:
diff --git a/openlibrary/core/models.py b/openlibrary/core/models.py
--- a/openlibrary/core/models.py
+++ b/openlibrary/core/models.py
@@ -712,6 +712,9 @@ def in_sponsorship_beta(self):
     def is_beta_tester(self):
         return self.is_usergroup_member('/usergroup/beta-testers')
 
+    def has_librarian_tools(self):
+        return self.is_usergroup_member('/usergroup/librarian-tools')
+
     def get_lists(self, seed=None, limit=100, offset=0, sort=True):
         """Returns all the lists of this user.
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Open Library login issue after visiting google
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

If http referer, Logging in erroneously redirects away from OpenLibrary 

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: 
* Expected: 

### Details

- **Logged in (Y/N)?** 
- **Browser type/version?** 
- **Operating system?** 
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
Related to #4277 and https://github.com/internetarchive/openlibrary/commit/585a764051cfc722025c12c4343a433083ed8686
Let's try -- `if not 'openlibrary' in referer: referer = None`?
We reproduced incognito, go to google.com, search for `openlibrary login` click the first link, login, you are brought to Google (??? no profit).
Data point:  I could not reproduce this in Firefox.  Was able to in Chrome, though.

Resolving patch:
diff --git a/openlibrary/plugins/upstream/account.py b/openlibrary/plugins/upstream/account.py
--- a/openlibrary/plugins/upstream/account.py
+++ b/openlibrary/plugins/upstream/account.py
@@ -318,8 +318,8 @@ def render_error(self, error_key, i):
 
     def GET(self):
         referer = web.ctx.env.get('HTTP_REFERER', '/')
-        # Don't set referer on user activation
-        if 'archive.org' in referer:
+        # Don't set referer if request is from offsite
+        if 'openlibrary.org' not in referer:
             referer = None
         i = web.input(redirect=referer)
         f = forms.Login()


Test patch: