Repository: internetarchive/openlibrary

Issue:
Privacy issue - leaking IP addresses
The edit history shows that not only are IP addresses being tracked, but they are being retained and made publically available through an unsecured API


### Evidence / Screenshot (if possible)
![Screen Shot 2022-09-30 at 5 18 32 PM](https://user-images.githubusercontent.com/82178/193357712-85165c9d-1a56-4b3a-a1d6-f281c38ba8f3.png)

### Relevant url?
https://openlibrary.org/authors/OL117575A.json?m=history

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. Go to ...
2. Do ...

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual:
* Expected: Privacy should be respected

### Details

- **Logged in (Y/N)?** No
- **Browser type/version?**
- **Operating system?**
- **Environment (prod/dev/local)?** prod
<!-- If not sure, put prod -->

### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
@mekarpeles 



Discussion:
@tfmorris I am looking into this as we speak. Nginx should be anonymizing any IP coming through at time of SSL termination:

`log_format iacombined '$remote_addr_ipscrub $host $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time';`

cc: @samuel-archive 

Investigating to understand the source and coverage of any issues.
Most immediately, I think it may make sense for us to prevent IP from being returned as a public field. cc: @cdrini @jimchamp 
After research, it looks like this IP data is historical (pre 2014) for patrons making edits on the website. Anything beyond this time should use anonymization or use Open Library's `207.` IP (which isn't great either, probably a bug, but here we are).

We think another case may be where e.g. bots or google colab notebooks are used.

**Action:** First and foremost, as a path forward we are going to remove IPs from the APIs for non-admins

Thanks for investigating. I didn't notice that the recent IP addresses were from the IA block. The one that I spot checked resolved to a European ISP. I'm not sure if old data is grandfathered under the European privacy laws, but removing all IP addresses sounds like the safe thing to do.
@tfmorris as a first step, I'd like to just prevent these addresses from being returned with the metadata.

The `types` are being registered in openlibrary/core/models.py -- the underlying logic is being handled by infogami.

Any suggestions on how we may be able to remove this data from the API endpoint response (without changing the code of infogami/infobase?)

Presumably we can edit `?m=history` code

Maybe this is `get_history_preview`?

In plugins/upstream/code.py there's a `class edit(core.edit)` which overrided the behavior of m=edit, maybe we can do something similar for the response of m=history?

catalog/utils/query.py get_mc looks related
there's a `history` class here: https://github.com/internetarchive/openlibrary/blob/39962157c12465d050e43458444aba9a59661c8d/openlibrary/plugins/upstream/recentchanges.py#L187-L196

This is what happend when `m=history` (ie mode is history)

That overrides the infogami defaults here:
https://github.com/internetarchive/infogami/blob/master/infogami/core/code.py#L186-L195

Here is the infogami implementation for .json:

https://github.com/internetarchive/infogami/blob/master/infogami/plugins/api/code.py#L238-L248
Ok, I figured it out -- there's probably a more efficient way to hit the changes API directly, but every time I did I was getting json decode errors for timestamps and the fields were slightly different.

Advice + feedback welcome for the code review

```
diff --git a/openlibrary/plugins/upstream/code.py b/openlibrary/plugins/upstream/code.py
index 125ccb501..ece511360 100644
--- a/openlibrary/plugins/upstream/code.py
+++ b/openlibrary/plugins/upstream/code.py
@@ -11,6 +11,11 @@ import web
 
 from infogami import config
 from infogami.core import code as core
+from infogami.plugins.api.code import (
+    jsonapi, make_query,
+    history as history_api,
+    request as infogami_request
+)
 from infogami.infobase import client
 from infogami.utils import delegate, app, types
 from infogami.utils.view import public, safeint, render
@@ -24,7 +29,7 @@ from openlibrary.plugins.upstream import spamcheck
 from openlibrary.plugins.upstream import merge_authors
 from openlibrary.plugins.upstream import edits
 from openlibrary.plugins.upstream import borrow, recentchanges  # TODO: unused imports?
-from openlibrary.plugins.upstream.utils import render_component
+from openlibrary.plugins.upstream.utils import render_component, get_changes
 
 if not config.get('coverstore_url'):
     config.coverstore_url = "https://covers.openlibrary.org"  # type: ignore[attr-defined]
@@ -42,6 +47,23 @@ class static(delegate.page):
         raise web.seeother(host + '/static' + web.ctx.path)
 
 
+class history(history_api):
+    """Overwrite ?m=history to remove IP"""
+    encoding = "json"
+
+    @jsonapi
+    def GET(self, path):
+        query = make_query(
+            web.input(), required_keys=['author', 'offset', 'limit']
+        )
+        query['key'] = path
+        query['sort'] = '-created'
+        history = json.loads(infogami_request('/versions', data=dict(query=json.dumps(query))))
+        for i, row in enumerate(history):
+            history[i].pop("ip")
+        return json.dumps(history)
+
+
 class edit(core.edit):
     """Overwrite ?m=edit behaviour for author, book, work, and people pages."""
 ```

Resolving patch:
diff --git a/openlibrary/plugins/upstream/code.py b/openlibrary/plugins/upstream/code.py
--- a/openlibrary/plugins/upstream/code.py
+++ b/openlibrary/plugins/upstream/code.py
@@ -11,6 +11,9 @@
 
 from infogami import config
 from infogami.core import code as core
+from infogami.plugins.api.code import jsonapi, make_query
+from infogami.plugins.api.code import request as infogami_request
+
 from infogami.infobase import client
 from infogami.utils import delegate, app, types
 from infogami.utils.view import public, safeint, render
@@ -42,6 +45,25 @@ def GET(self):
         raise web.seeother(host + '/static' + web.ctx.path)
 
 
+class history(delegate.mode):
+    """Overwrite ?m=history to remove IP"""
+
+    encoding = "json"
+
+    @jsonapi
+    def GET(self, path):
+        query = make_query(web.input(), required_keys=['author', 'offset', 'limit'])
+        query['key'] = path
+        query['sort'] = '-created'
+        # Possibly use infogami.plugins.upstream.utils get_changes to avoid json load/dump?
+        history = json.loads(
+            infogami_request('/versions', data=dict(query=json.dumps(query)))
+        )
+        for i, row in enumerate(history):
+            history[i].pop("ip")
+        return json.dumps(history)
+
+
 class edit(core.edit):
     """Overwrite ?m=edit behaviour for author, book, work, and people pages."""
 


Test patch: