Repository: matrix-org/synapse

Issue:
Device IP & UA can get overwritten by Synapse on matrix.org
Right now, my iPhone (Obfuscate) claims to have no IP, and a UA of Synapse/1.6.0rc1 despite me having used it fairly recently.  It looks like we have a bug causing the devices info to be overwritten by Synapse somehow when Synapse auths on your behalf (perhaps replication traffic between workers?)  This is dangerous as it stops the user being able to see the actual IP they last connected from.

```
matrix=# select * from matrix.devices where user_id='@matthew:matrix.org';
       user_id       | device_id  |    display_name    |   last_seen   |       ip       |                                                        user_agent                                                         | hidden 
---------------------+------------+--------------------+---------------+----------------+---------------------------------------------------------------------------------------------------------------------------+--------
 @matthew:matrix.org | GFCFEVCZBR | Obfuscate          | 1574302825180 | -              | Synapse/1.6.0rc1 (b=matrix-org-hotfixes,2c59eb368)                                                                        | f
```


Discussion:
Should probably be a release blocker?
This seems to be a long-standing problem:

```
          user_id           | device_id  |                 display_name                  |   last_seen   | ip |                        user_agent                        | hidden 
----------------------------+------------+-----------------------------------------------+---------------+----+----------------------------------------------------------+--------
 @xxxxxxxxxxxxxx:matrix.org | XXXXXXXXXX | https://riot.im/develop/ via Chrome on Linux  | 1559559762800 | -  | Synapse/0.99.5.2 (b=matrix-org-hotfixes,9f5048c19,dirty) | f
 @xxxxxxxxxxxxxx:matrix.org | YYYYYYYYYY | https://riot.im/develop/ via Chrome on Linux  | 1558442587658 | -  | Synapse/0.99.4rc1 (b=HEAD,9feee29d7)                     | f
 @xxxxxxxxxxxxxx:matrix.org | ZZZZZZZZZZ | Riot Desktop on Linux                         | 1563532662959 | -  | Synapse/1.1.0 (b=matrix-org-hotfixes,4102cb220)          | f
 @xxxxxxxxxxxxxx:matrix.org | WWWWWWWWWW | https://riot.im/develop/ via Firefox on Linux | 1563369636440 | -  | Synapse/1.1.0 (b=matrix-org-hotfixes,529970732)          | f
```
Running the same query on other people's servers that have always been monolith has come up with nothing. Looks like this indeed might be a replication problem.
With E2EE by default on the horizon, the fact that people's device lists on matrix.org are full of spurious devices with no IP and a UA of "Synapse" doesn't encourage them to keep good device hygiene.

It looks an awful lot like worker replication traffic is showing up in the devices table - I'm assuming this could be a fairly easy fix; would it be possible to get it on the maintenance radar @neilisfragile?
The `-` as an IP means there was no X-Forwarded-For header. This is probably from e.g. the proxied key upload requests done by the frontend proxy not setting a `X-Forwarded-For` header on its request to the master. The code lives in `synapse/app/generic_worker.py`. Its probably also worth searching for other situations where we proxy requests and `Authorization` headers/access tokens (note I doubt replication requests in `synapse/replication` are affected by this as they don't auth as clients)

Resolving patch:
diff --git a/synapse/app/generic_worker.py b/synapse/app/generic_worker.py
--- a/synapse/app/generic_worker.py
+++ b/synapse/app/generic_worker.py
@@ -21,7 +21,7 @@
 
 from typing_extensions import ContextManager
 
-from twisted.internet import defer, reactor
+from twisted.internet import address, defer, reactor
 
 import synapse
 import synapse.events
@@ -206,10 +206,30 @@ async def on_POST(self, request, device_id):
 
         if body:
             # They're actually trying to upload something, proxy to main synapse.
-            # Pass through the auth headers, if any, in case the access token
-            # is there.
-            auth_headers = request.requestHeaders.getRawHeaders(b"Authorization", [])
-            headers = {"Authorization": auth_headers}
+
+            # Proxy headers from the original request, such as the auth headers
+            # (in case the access token is there) and the original IP /
+            # User-Agent of the request.
+            headers = {
+                header: request.requestHeaders.getRawHeaders(header, [])
+                for header in (b"Authorization", b"User-Agent")
+            }
+            # Add the previous hop the the X-Forwarded-For header.
+            x_forwarded_for = request.requestHeaders.getRawHeaders(
+                b"X-Forwarded-For", []
+            )
+            if isinstance(request.client, (address.IPv4Address, address.IPv6Address)):
+                previous_host = request.client.host.encode("ascii")
+                # If the header exists, add to the comma-separated list of the first
+                # instance of the header. Otherwise, generate a new header.
+                if x_forwarded_for:
+                    x_forwarded_for = [
+                        x_forwarded_for[0] + b", " + previous_host
+                    ] + x_forwarded_for[1:]
+                else:
+                    x_forwarded_for = [previous_host]
+            headers[b"X-Forwarded-For"] = x_forwarded_for
+
             try:
                 result = await self.http_client.post_json_get_json(
                     self.main_uri + request.uri.decode("ascii"), body, headers=headers


Test patch: