internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: internetarchive/openlibrary

Issue:
Add pager to trending pages
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

<!-- Which suggestions or requirements should be considered for how feature needs to appear or be implemented? -->

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->





Discussion:

Resolving patch:
diff --git a/openlibrary/views/loanstats.py b/openlibrary/views/loanstats.py
--- a/openlibrary/views/loanstats.py
+++ b/openlibrary/views/loanstats.py
@@ -158,19 +158,25 @@ def get_cached_activity_stream(limit):
 class activity_stream(app.view):
     path = "/trending(/?.*)"
 
-    def GET(self, page=''):
-        if not page:
+    def GET(self, mode=''):
+        i = web.input(page=1)
+        page = i.page
+        if not mode:
             raise web.seeother("/trending/now")
-        page = page[1:]  # remove slash
+        mode = mode[1:]  # remove slash
         limit = 20
-        if page == "now":
-            logged_books = Bookshelves.fetch(get_activity_stream(limit=limit))
+        if mode == "now":
+            logged_books = Bookshelves.fetch(
+                get_activity_stream(limit=limit, page=page)
+            )
         else:
             shelf_id = None  # optional; get from web.input()?
             logged_books = Bookshelves.fetch(
-                cached_get_most_logged_books(since_days=SINCE_DAYS[page], limit=limit)
+                cached_get_most_logged_books(
+                    since_days=SINCE_DAYS[mode], limit=limit, page=page
+                )
             )
-        return app.render_template("trending", logged_books=logged_books, mode=page)
+        return app.render_template("trending", logged_books=logged_books, mode=mode)
 
 
 class readinglog_stats(app.view):


Test patch:
Repository: internetarchive/openlibrary

Issue:
Solr updater not picking up edits to books with IA ids
Eg: https://openlibrary.org/works/OL5717624W/Ttfn is not showing up in the results https://openlibrary.org/search?q=key%3A%2Fworks%2FOL5717624W+AND+subject%3A%22collectionID%3AAlpineChallenge%22&mode=everything when it should. Even if it is force reindexed via https://openlibrary.org/admin/solr , or when edited. (eg it should show a lowercase title on https://openlibrary.org/search?mode=everything&q=key%3A%2Fworks%2FOL5717624W )

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
Tmanral



Discussion:

Resolving patch:
diff --git a/openlibrary/solr/update_work.py b/openlibrary/solr/update_work.py
--- a/openlibrary/solr/update_work.py
+++ b/openlibrary/solr/update_work.py
@@ -612,6 +612,11 @@ def get_edition_ddcs(ed: dict):
 
         d |= self.get_ebook_info(editions, ia_metadata)
 
+        # See https://github.com/internetarchive/openlibrary/issues/6836
+        # This was half-implemented
+        if 'ia_collection' in d and not get_solr_next():
+            del d['ia_collection']
+
         return d
 
     @staticmethod


Test patch:
Repository: internetarchive/openlibrary

Issue:
Investigate/gather feedback on current mobile UX issues
End goal is to have proposals we can convert into actionable issues for the epic.


Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/openlibrary/code.py b/openlibrary/plugins/openlibrary/code.py
--- a/openlibrary/plugins/openlibrary/code.py
+++ b/openlibrary/plugins/openlibrary/code.py
@@ -1094,6 +1094,7 @@ def setup_template_globals():
             'isbn_10_to_isbn_13': isbn_10_to_isbn_13,
             'NEWLINE': '\n',
             'random': random.Random(),
+            'choose_random_from': random.choice,
             'get_lang': lambda: web.ctx.lang,
             'ceil': math.ceil,
             'get_best_edition': get_best_edition,


Test patch:
Repository: internetarchive/openlibrary

Issue:
Sorting buttons in Reading log don't work as intended
When you press sort by "Date Added (oldest)" page refreshes and sorting is applied but "Date Added (oldest)" button doesn't turn green and doesn't become unclickable, and "Date Added (newest)" button doesn't become clickable

### Evidence / Screenshot (if possible)
On this scrrencshot books are sorted by oldest
![image](https://user-images.githubusercontent.com/58372007/185414834-58610de2-e5f4-43a3-8b91-238b5195f81d.png)

### Relevant url?
<!-- `https://openlibrary.org/...` -->

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ... Reading log
2. Do ... press sort by "Date Added (oldest)"

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?** Y
- **Browser type/version?** Firefox 103 (64-bit)
- **Operating system?** Linux Mint and Windows 8.1
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:
I see the issue. If you change the sort param in the URL then it filters the results as per the filter applied but there is no effect on the buttons. They remain as is. Checking it locally, Hopefull will figure out the issue.

Resolving patch:
diff --git a/openlibrary/plugins/upstream/mybooks.py b/openlibrary/plugins/upstream/mybooks.py
--- a/openlibrary/plugins/upstream/mybooks.py
+++ b/openlibrary/plugins/upstream/mybooks.py
@@ -248,6 +248,7 @@ def render(self, page=1, sort='desc', list=None):
                 lists=self.lists,
                 public=is_public,
                 owners_page=is_logged_in_user,
+                sort_order=sort,
             )
 
         raise web.seeother(self.user.key)


Test patch:
Repository: internetarchive/openlibrary

Issue:
Sort records by `last_modified` date instead of `created` date P2



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/edits.py b/openlibrary/plugins/upstream/edits.py
--- a/openlibrary/plugins/upstream/edits.py
+++ b/openlibrary/plugins/upstream/edits.py
@@ -41,7 +41,7 @@ def GET(self):
             mode=i.mode,
             submitter=i.submitter,
             reviewer=i.reviewer,
-            order=f'created {i.order}',
+            order=f'updated {i.order}',
         ).list()
 
         total_found = {


Test patch:
Repository: internetarchive/openlibrary

Issue:
Bulk Redirects Test Run & Cron
See: https://github.com/internetarchive/openlibrary/issues/6667#issuecomment-1186275514_

Doing a team review of a few more examples prior to running resolve redirects bulk code over a larger batch of records (and discussing how it should ultimately be run; e.g. cron?)


Discussion:

Resolving patch:
diff --git a/openlibrary/core/models.py b/openlibrary/core/models.py
--- a/openlibrary/core/models.py
+++ b/openlibrary/core/models.py
@@ -646,7 +646,7 @@ def resolve_redirects_bulk(
         start_offset=0,
         grace_period_days=7,
         cutoff_date=datetime.datetime(year=2017, month=1, day=1),
-        test=True,
+        test=False,
     ):
         """
         batch_size - how many records to fetch per batch
diff --git a/scripts/update_stale_work_references.py b/scripts/update_stale_work_references.py
new file mode 100644
--- /dev/null
+++ b/scripts/update_stale_work_references.py
@@ -0,0 +1,25 @@
+"""
+PYTHONPATH=. python ./scripts/update_stale_work_references.py /olsystem/etc/openlibrary.yml
+"""
+
+import web
+import infogami
+from infogami import config  # noqa: F401
+from openlibrary.config import load_config
+from openlibrary.core.models import Work
+from scripts.solr_builder.solr_builder.fn_to_cli import FnToCLI
+import datetime
+
+
+def main(ol_config: str, start_offset=0, days=31):
+    load_config(ol_config)
+    infogami._setup()
+    cutoff_date = datetime.datetime.today() - datetime.timedelta(days=days)
+    Work.resolve_redirects_bulk(
+        start_offset=start_offset,
+        cutoff_date=cutoff_date,
+    )
+
+
+if __name__ == '__main__':
+    FnToCLI(main).run()


Test patch:
Repository: internetarchive/openlibrary

Issue:
Pagination broken on trending pages
```
<class 'TypeError'> at /trending/daily
unsupported operand type(s) for -: 'str' and 'int'
```

### Evidence / Screenshot (if possible)
![image](https://user-images.githubusercontent.com/6251786/191132271-7a46ae90-02a9-4b52-90d2-80ba97417122.png)

### Relevant url?
- eg https://openlibrary.org/trending/daily?page=2&debug=true
- Sentry (staff only): https://sentry.archive.org/organizations/ia-ux/issues/45801/?project=7&query=is%3Aunresolved

### Details

- **Logged in (Y/N)?** Y 
- **Browser type/version?** FF104
- **Operating system?** Win10
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/core/bookshelves.py b/openlibrary/core/bookshelves.py
--- a/openlibrary/core/bookshelves.py
+++ b/openlibrary/core/bookshelves.py
@@ -88,6 +88,7 @@ def most_logged_books(
         users. This query is limited to a specific shelf_id (e.g. 1
         for "Want to Read").
         """
+        page = int(page or 1)
         offset = (page - 1) * limit
         oldb = db.get_db()
         where = 'WHERE bookshelf_id' + ('=$shelf_id' if shelf_id else ' IS NOT NULL ')


Test patch:
Repository: internetarchive/openlibrary

Issue:
openlibrary/core/lending.py: Fix duplicate enties in a dict
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

### Evidence / Screenshot (if possible)

### Relevant url?
<!-- `https://openlibrary.org/...` -->
There are duplicate keys in the definition of a dict at:
https://github.com/internetarchive/openlibrary/blob/master/openlibrary/core/lending.py#L724-L728

`resource_type` is added twice and `resource_id` is added twice.  It is difficult to determine if the first two duplicate keys should be removed or if the second two should.

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected:

### Details

- **Logged in (Y/N)?**
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/core/lending.py b/openlibrary/core/lending.py
--- a/openlibrary/core/lending.py
+++ b/openlibrary/core/lending.py
@@ -721,11 +721,9 @@ def new(identifier, resource_type, user_key, book_key=None):
                 'ocaid': identifier,
                 'expiry': expiry,
                 'uuid': _uuid,
-                'resource_type': 'bookreader',  # noqa: F601
-                'resource_id': 'bookreader:%s' % identifier,  # noqa: F601
                 'loaned_at': loaned_at,
-                'resource_type': resource_type,  # noqa: F601
-                'resource_id': resource_id,  # noqa: F601
+                'resource_type': resource_type,
+                'resource_id': resource_id,
                 'loan_link': loan_link,
             }
         )


Test patch:
Repository: internetarchive/openlibrary

Issue:
Empty field for weight unit results in crash upon saving 
In certain instances, items are imported with an entry in the weight field, but no weight unit (grams, kilos, ounces, pounds) selected.

This will lead to a crash when one attempts to edit and save that item without setting a weight unit.

### Evidence / Screenshot (if possibl
![Screenshot_20221012_190916](https://user-images.githubusercontent.com/36878944/195406212-ebac4f3d-bc5d-4136-aafe-b5aa978c8914.png)

Also, see error report 2022-10-12/170923872798.

### Relevant url?
https://openlibrary.org/works/OL27130932W/

### Steps to Reproduce
1. Edit the work OL27130932W as usual (it works with any work that has an entry in weight but no weight unit).
2. Save the work without selecting a weight unit.
3. Error message, work is not saved.

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: Error message, work not saved
* Expected: Work saved or notice that one needs to select a measurement if the weight field is not empty.

### Details

- **Logged in (Y/N)?**
Y

- **Browser type/version?**
Firefox 104-2

- **Operating system?**
Suse Tumbleweed

- **Environment (prod/dev/local)?** prod
prod

### Proposal & Constraints
The work should either save or tell me how to solve the problem.

### Related files


### Stakeholders




Discussion:
I would like to work on this.

Resolving patch:
diff --git a/openlibrary/plugins/upstream/addbook.py b/openlibrary/plugins/upstream/addbook.py
--- a/openlibrary/plugins/upstream/addbook.py
+++ b/openlibrary/plugins/upstream/addbook.py
@@ -717,11 +717,13 @@ def process_edition(self, edition):
 
         edition = trim_doc(edition)
 
-        if list(edition.get('physical_dimensions', [])) == ['units']:
-            edition.physical_dimensions = None
+        if physical_dimensions := (edition.get('physical_dimensions', [])):
+            if list(physical_dimensions) == ['units']:
+                edition.physical_dimensions = None
 
-        if list(edition.get('weight', [])) == ['units']:
-            edition.weight = None
+        if weight := (edition.get('weight', [])):
+            if list(weight) == ['units']:
+                edition.weight = None
 
         for k in ['roles', 'identifiers', 'classifications']:
             edition[k] = edition.get(k) or []


Test patch:
Repository: internetarchive/openlibrary

Issue:
Author pages should make use of matching edition data
Author pages should make use of edition-aware search to show works in user's language.

E.g. https://openlibrary.org/authors/OL2162284A/Stephen_King?lang=fr Should show French titles.


Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/worksearch/code.py b/openlibrary/plugins/worksearch/code.py
--- a/openlibrary/plugins/worksearch/code.py
+++ b/openlibrary/plugins/worksearch/code.py
@@ -483,13 +483,17 @@ def works_by_author(
                 "time_facet",
             ]
         ),
+        fields=WorkSearchScheme.default_fetched_fields | {'editions'},
         extra_params=[
             ('fq', f'author_key:{akey}'),
             ('facet.limit', 25),
         ],
     )
 
-    result.docs = add_availability([get_doc(doc) for doc in result.docs])
+    result.docs = [get_doc(doc) for doc in result.docs]
+    add_availability(
+        [(work.get('editions') or [None])[0] or work for work in result.docs]
+    )
     return result
 
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Fix Amazon book-cover imports (data mapping?)
When we import books from Amazon, it seems as if the book cover is either not making it in the import queue, is using the wrong field name, or the importer itself is not succeeding to follow the schema and add the cover.

I'd recommend investigating the following files / areas (in ~order):
* https://github.com/internetarchive/openlibrary-client/blob/master/olclient/schemata/import.schema.json#L112-L119
* https://github.com/internetarchive/openlibrary/blob/master/openlibrary/core/vendors.py#L384
* https://github.com/internetarchive/openlibrary/blob/master/openlibrary/core/vendors.py#L180
* https://github.com/internetarchive/openlibrary/blob/master/openlibrary/core/vendors.py#L244-L252
* https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/importapi/code.py#L144
* https://github.com/internetarchive/openlibrary/blob/master/openlibrary/catalog/add_book/__init__.py#L683

I'd recommend testing by issuing a `POST` from the browser's developer tools (while logged in) directly to the /api/import pipeline to see if what we're queuing in the batch importer succeeds.


Discussion:

Resolving patch:
diff --git a/openlibrary/core/vendors.py b/openlibrary/core/vendors.py
--- a/openlibrary/core/vendors.py
+++ b/openlibrary/core/vendors.py
@@ -240,7 +240,7 @@ def serialize(product: Any) -> dict:
                 and item_info.title
                 and getattr(item_info.title, 'display_value')
             ),
-            'covers': (
+            'cover': (
                 images.primary.large.url
                 if images
                 and images.primary


Test patch:
Repository: internetarchive/openlibrary

Issue:
Update Progress Widget to allow clearing yearly reading goal



Discussion:

Resolving patch:
diff --git a/openlibrary/core/yearly_reading_goals.py b/openlibrary/core/yearly_reading_goals.py
--- a/openlibrary/core/yearly_reading_goals.py
+++ b/openlibrary/core/yearly_reading_goals.py
@@ -100,3 +100,15 @@ def delete_by_username(cls, username):
         data = {'username': username}
 
         return oldb.delete(cls.TABLENAME, where=where, vars=data)
+
+    @classmethod
+    def delete_by_username_and_year(cls, username, year):
+        oldb = db.get_db()
+
+        data = {
+            'username': username,
+            'year': year,
+        }
+        where = 'username=$username AND year=$year'
+
+        return oldb.delete(cls.TABLENAME, where=where, vars=data)
diff --git a/openlibrary/plugins/upstream/checkins.py b/openlibrary/plugins/upstream/checkins.py
--- a/openlibrary/plugins/upstream/checkins.py
+++ b/openlibrary/plugins/upstream/checkins.py
@@ -180,7 +180,12 @@ def POST(self):
 
         goal = int(i.goal)
 
-        if not goal or goal < 0:
+        if i.is_update:
+            if goal < 0:
+                raise web.badrequest(
+                    'Reading goal update must be 0 or a positive integer'
+                )
+        elif not goal or goal < 1:
             raise web.badrequest('Reading goal must be a positive integer')
 
         if i.is_update and not i.year:
@@ -192,7 +197,12 @@ def POST(self):
         current_year = i.year or datetime.now().year
 
         if i.is_update:
-            YearlyReadingGoals.update_target(username, i.year, goal)
+            if goal == 0:
+                # Delete goal if "0" was submitted:
+                YearlyReadingGoals.delete_by_username_and_year(username, i.year)
+            else:
+                # Update goal normally:
+                YearlyReadingGoals.update_target(username, i.year, goal)
         else:
             YearlyReadingGoals.create(username, current_year, goal)
 


Test patch:
Repository: internetarchive/openlibrary

Issue:
Reading Log: Don't auto hide/remove works on shelf change (until refresh)



Discussion:

Resolving patch:
diff --git a/openlibrary/core/bookshelves_events.py b/openlibrary/core/bookshelves_events.py
--- a/openlibrary/core/bookshelves_events.py
+++ b/openlibrary/core/bookshelves_events.py
@@ -197,3 +197,15 @@ def delete_by_username(cls, username):
         where_vars = {'username': username}
 
         return oldb.delete(cls.TABLENAME, where=where_clause, vars=where_vars)
+
+    @classmethod
+    def delete_by_username_and_work(cls, username, work_id):
+        oldb = db.get_db()
+
+        where_clause = 'username=$username AND work_id=$work_id'
+        data = {
+            'username': username,
+            'work_id': work_id,
+        }
+
+        return oldb.delete(cls.TABLENAME, where=where_clause, vars=data)
diff --git a/openlibrary/plugins/openlibrary/api.py b/openlibrary/plugins/openlibrary/api.py
--- a/openlibrary/plugins/openlibrary/api.py
+++ b/openlibrary/plugins/openlibrary/api.py
@@ -21,6 +21,7 @@
 from openlibrary.plugins.worksearch.subjects import get_subject
 from openlibrary.accounts.model import OpenLibraryAccount
 from openlibrary.core import ia, db, models, lending, helpers as h
+from openlibrary.core.bookshelves_events import BookshelvesEvents
 from openlibrary.core.observations import Observations, get_observation_metrics
 from openlibrary.core.models import Booknotes, Work
 from openlibrary.core.sponsorships import qualifies_for_sponsorship
@@ -324,6 +325,7 @@ def POST(self, work_id):
             work_bookshelf = Bookshelves.remove(
                 username=username, work_id=work_id, bookshelf_id=current_status
             )
+            BookshelvesEvents.delete_by_username_and_work(username, work_id)
 
         else:
             edition_id = int(i.edition_id.split('/')[2][2:-1]) if i.edition_id else None


Test patch:
Repository: internetarchive/openlibrary

Issue:
Author search should boost results that exactly match user search
Subtask of #7035

### Stakeholders
@tfmorris 


Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/worksearch/code.py b/openlibrary/plugins/worksearch/code.py
--- a/openlibrary/plugins/worksearch/code.py
+++ b/openlibrary/plugins/worksearch/code.py
@@ -610,7 +610,6 @@ def get_results(self, q, offset=0, limit=100, fields='*'):
             offset=offset,
             rows=limit,
             fields=fields,
-            sort='work_count desc',
         )
 
         return resp
diff --git a/openlibrary/plugins/worksearch/schemes/authors.py b/openlibrary/plugins/worksearch/schemes/authors.py
--- a/openlibrary/plugins/worksearch/schemes/authors.py
+++ b/openlibrary/plugins/worksearch/schemes/authors.py
@@ -52,4 +52,6 @@ def q_to_solr_params(
             ('q.op', 'AND'),
             ('defType', 'edismax'),
             ('qf', 'name alternate_names'),
+            ('pf', 'name^10 alternate_names^10'),
+            ('bf', 'min(work_count,20)'),
         ]


Test patch: