Repository: internetarchive/openlibrary

Issue:
Create endpoint for Archive.org to update OCAIDs in OL records
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

### Describe the problem that you'd like solved
<!-- A clear and concise description of what you want to happen. -->
Today, there is no way for Internet Archive to notify Open Library of important changes to records.  If data becomes out-of-sync between the two platforms, Open Library records must be updated and synced by a human.  Before this is done, somebody has to first notice that the data is out-of-sync.  This is not ideal...

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->
Edited proposal:
Create a new POST handler and endpoint for syncing OCAID changes.  Sync requests must contain S3 credentials, edition `olid`, the original `ocaid`, and either `is_dark` if the item has been made dark, or `new_ocaid` if the OCAID has been changed.

### Additional context
<!-- Add any other context or screenshots about the feature request here. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->
@mekarpeles 




Discussion:
I had proposed something a bit different further into the Slack thread:

> what if archive.org sent OL a general “re-sync this item” request, with OL determining which values had changed since the last sync? the only additional special information archive.org would have to send would be in the rename case, where it would include something like `old_id={identifier}`, so OL would know which prior values to compare. (the “made dark” case needs no special indication, as OL will find out that the item is dark when it tries to fetch its metadata.)

This situation is different from initial imports because archive.org will be sending the request only for items that have already been previously imported (but have now been changed in some meaningful way), and archive.org will know the key for the corresponding OL record.

That's why I'm suggesting a distinct "re-sync" request, with OL comparing the current archive.org info with the prior sync, and making updates as needed. Could we set up something where archive.org sends, for example,  `ocaid=goodytwoshoes00newyiala&olid=OL7095326M` (or the equivalent in JSON) and OL compares what it previously had and makes updates as needed? It may find that the archive.org item's ISBN or other metadata has changed; if the item has been darked, it will find `is_dark: true` in the item's metadata record; if the item has been renamed, we could include an `old_ocaid=...` value in the request.

Does that sound feasible?
How about something like this @hbromley?
https://github.com/internetarchive/openlibrary/issues/7543#issuecomment-1431908921
That is quite different from what I proposed in my previous comment. Is that approach not feasible?
Sorry for the delay (and confusion) @hbromley.  The new endpoint that you are describing is totally feasible, and I'll start working on it today.

Resolving patch:
diff --git a/openlibrary/plugins/admin/code.py b/openlibrary/plugins/admin/code.py
--- a/openlibrary/plugins/admin/code.py
+++ b/openlibrary/plugins/admin/code.py
@@ -239,6 +239,80 @@ def GET(self):
         return delegate.RawText(json.dumps(data), content_type="application/json")
 
 
+class sync_ia_ol(delegate.page):
+    path = '/ia/sync'
+    encoding = 'json'
+
+    def POST(self):
+        # Authenticate request:
+        s3_access_key = web.ctx.env.get('HTTP_X_S3_ACCESS', '')
+        s3_secret_key = web.ctx.env.get('HTTP_X_S3_SECRET', '')
+
+        if not self.is_authorized(s3_access_key, s3_secret_key):
+            raise web.unauthorized()
+
+        # Validate input
+        i = json.loads(web.data())
+
+        if not self.validate_input(i):
+            raise web.badrequest('Missing required fields')
+
+        # Find record using OLID (raise 404 if not found)
+        edition_key = f'/books/{i.get("olid")}'
+        edition = web.ctx.site.get(edition_key)
+
+        if not edition:
+            raise web.notfound()
+
+        # Update record
+        match i.get('action', ''):
+            case 'remove':
+                self.remove_ocaid(edition)
+            case 'modify':
+                self.modify_ocaid(edition, i.get('ocaid'))
+            case '_':
+                raise web.badrequest('Unknown action')
+
+        return delegate.RawText(json.dumps({"status": "ok"}))
+
+    def is_authorized(self, access_key, secret_key):
+        """Returns True if account is authorized to make changes to records."""
+        auth = accounts.InternetArchiveAccount.s3auth(access_key, secret_key)
+
+        if not auth.get('username', ''):
+            return False
+
+        acct = accounts.OpenLibraryAccount.get(email=auth.get('username'))
+        user = acct.get_user() if acct else None
+
+        if not user or (user and not user.is_usergroup_member('/usergroup/ia')):
+            return False
+
+        return True
+
+    def validate_input(self, i):
+        """Returns True if the request is valid.
+        All requests must have an olid and an action.  If the action is
+        'modify', the request must also include 'ocaid'.
+        """
+        action = i.get('action', '')
+        return 'olid' in i and (
+            action == 'remove' or (action == 'modify' and 'ocaid' in i)
+        )
+
+    def remove_ocaid(self, edition):
+        """Deletes OCAID from given edition"""
+        data = edition.dict()
+        del data['ocaid']
+        web.ctx.site.save(data, 'Remove OCAID: Item no longer available to borrow.')
+
+    def modify_ocaid(self, edition, new_ocaid):
+        """Adds the given new_ocaid to an edition."""
+        data = edition.dict()
+        data['ocaid'] = new_ocaid
+        web.ctx.site.save(data, 'Update OCAID')
+
+
 class people_view:
     def GET(self, key):
         account = accounts.find(username=key) or accounts.find(email=key)
diff --git a/openlibrary/plugins/openlibrary/api.py b/openlibrary/plugins/openlibrary/api.py
--- a/openlibrary/plugins/openlibrary/api.py
+++ b/openlibrary/plugins/openlibrary/api.py
@@ -16,8 +16,8 @@
 from infogami.utils.view import add_flash_message
 from openlibrary import accounts
 from openlibrary.plugins.openlibrary.code import can_write
-from openlibrary.utils.isbn import isbn_10_to_isbn_13, normalize_isbn
 from openlibrary.utils import extract_numeric_id_from_olid
+from openlibrary.utils.isbn import isbn_10_to_isbn_13, normalize_isbn
 from openlibrary.plugins.worksearch.subjects import get_subject
 from openlibrary.accounts.model import OpenLibraryAccount
 from openlibrary.core import ia, db, models, lending, helpers as h


Test patch: