Repository: matrix-org/synapse

Issue:
Incorrect HTTP Status returned on unknown/invalid access token
As per [latest stable](https://matrix.org/docs/spec/client_server/r0.2.0.html#client-authentication) and [unstable](https://matrix.org/speculator/spec/HEAD/client_server/unstable.html#client-authentication) spec, `401` must be returned but `403` is returned instead.

This leads to further spec breakage down the line for endpoints that are supposed to return 403 for another error, like (not complete list):
- `/_matrix/client/r0/rooms/{roomId}/state/{eventType}/{stateKey}`
- `/_matrix/client/r0/rooms/{roomId}/state/{eventType}`
- `/_matrix/client/r0/rooms/{roomId}/state`
- `/_matrix/client/r0/rooms/{roomId}/members`
- `/_matrix/client/r0/rooms/{roomId}/joined_members`
- `/_matrix/client/r0/rooms/{roomId}/messages`
- `/_matrix/client/r0/rooms/{roomId}/invite`
- etc.

On some of those endpoints, the client would be expected to behave differently, depending if the token is invalid or if authorization to perform the call was lacking (invalidate session & re-login VS give up specific action).


Discussion:
I believe I've found a [comment](https://github.com/matrix-org/synapse/blob/master/synapse/server.py#L208) in the code which refers to this:

> Matrix spec makes no reference to what HTTP status code is returned,
but the V1 API uses 403 where it means 401, and the webclient
relies on this behaviour, so V1 gets its own copy of the auth
with backwards compat behaviour.

I'm not sure what should be done, but I assume this is the reason for the bug.
I was curious, and as far as I can tell, this has been fixed:

    $ curl -v 'https://matrix.org/_matrix/client/r0/account/whoami?access_token=192837918'
    [...]
    < HTTP/2 401

Same if the token is missing.
It's still not.

Without access token:
```
$ curl -vd '{}' 'https://matrix.org/_matrix/client/r0/rooms/!dummy:matrix.org/invite'
[...]
< HTTP/1.1 403 Forbidden
[...]
{
    "errcode": "M_MISSING_TOKEN",
    "error": "Missing access token."
}
```

```
$ curl -v 'https://matrix.org/_matrix/client/r0/rooms/!dummy:matrix.org/members'
[...]
< HTTP/1.1 403 Forbidden
[...]
{
    "errcode": "M_MISSING_TOKEN",
    "error": "Missing access token."
}
```

---

With invalid access token:
```
$ curl -vd '{}' 'https://matrix.org/_matrix/client/r0/rooms/!dummy:matrix.org/invite?access_token=1'
[...]
< HTTP/1.1 403 Forbidden
[...]
{
    "errcode": "M_UNKNOWN_TOKEN",
    "error": "Unrecognised access token."
}
```

```
$ curl -v 'https://matrix.org/_matrix/client/r0/rooms/!dummy:matrix.org/members?access_token=1'
[...]
< HTTP/1.1 403 Forbidden
[...]
{
    "errcode": "M_UNKNOWN_TOKEN",
    "error": "Unrecognised access token."
}
```

So here's what I've been able to discover about this issue and why it only affects some endpoints:

There used to be two versions: v1 and v2_alpha, which were later merged into r0, however they are still separate in the synapse code. Any endpoints that are identical to the original v1 spec are served by the v1 code path (which returns 403), while anything else is served by the v2_alpha code path (which returns 401). This causes the mix between the two behaviours.

The solution would be to simply move the v1 endpoints to the new behaviour. This might possibly break the old angular client, which is deprecated anyway.
i suspect when we fixed this on the v2 servlets we forgot to fix the v1 servlets, and it'd be fine to apply the same fix there (and hope that no clients are relying on the incorrect behaviour....)

Resolving patch:
diff --git a/synapse/rest/client/v1/base.py b/synapse/rest/client/v1/base.py
--- a/synapse/rest/client/v1/base.py
+++ b/synapse/rest/client/v1/base.py
@@ -52,6 +52,10 @@ class ClientV1RestServlet(RestServlet):
     """A base Synapse REST Servlet for the client version 1 API.
     """
 
+    # This subclass was presumably created to allow the auth for the v1
+    # protocol version to be different, however this behaviour was removed.
+    # it may no longer be necessary
+
     def __init__(self, hs):
         """
         Args:
@@ -59,5 +63,5 @@ def __init__(self, hs):
         """
         self.hs = hs
         self.builder_factory = hs.get_event_builder_factory()
-        self.auth = hs.get_v1auth()
+        self.auth = hs.get_auth()
         self.txns = HttpTransactionCache(hs.get_clock())
diff --git a/synapse/rest/client/v1/pusher.py b/synapse/rest/client/v1/pusher.py
--- a/synapse/rest/client/v1/pusher.py
+++ b/synapse/rest/client/v1/pusher.py
@@ -150,7 +150,7 @@ def __init__(self, hs):
         super(RestServlet, self).__init__()
         self.hs = hs
         self.notifier = hs.get_notifier()
-        self.auth = hs.get_v1auth()
+        self.auth = hs.get_auth()
         self.pusher_pool = self.hs.get_pusherpool()
 
     @defer.inlineCallbacks
diff --git a/synapse/server.py b/synapse/server.py
--- a/synapse/server.py
+++ b/synapse/server.py
@@ -105,7 +105,6 @@ def build_DEPENDENCY(self)
         'federation_client',
         'federation_server',
         'handlers',
-        'v1auth',
         'auth',
         'state_handler',
         'state_resolution_handler',
@@ -225,15 +224,6 @@ def build_http_client_context_factory(self):
     def build_simple_http_client(self):
         return SimpleHttpClient(self)
 
-    def build_v1auth(self):
-        orf = Auth(self)
-        # Matrix spec makes no reference to what HTTP status code is returned,
-        # but the V1 API uses 403 where it means 401, and the webclient
-        # relies on this behaviour, so V1 gets its own copy of the auth
-        # with backwards compat behaviour.
-        orf.TOKEN_NOT_FOUND_HTTP_STATUS = 403
-        return orf
-
     def build_state_handler(self):
         return StateHandler(self)
 


Test patch:
diff --git a/tests/rest/client/v1/test_events.py b/tests/rest/client/v1/test_events.py
--- a/tests/rest/client/v1/test_events.py
+++ b/tests/rest/client/v1/test_events.py
@@ -148,11 +148,16 @@ def tearDown(self):
 
     @defer.inlineCallbacks
     def test_stream_basic_permissions(self):
-        # invalid token, expect 403
+        # invalid token, expect 401
+        # note: this is in violation of the original v1 spec, which expected
+        # 403. However, since the v1 spec no longer exists and the v1
+        # implementation is now part of the r0 implementation, the newer
+        # behaviour is used instead to be consistent with the r0 spec.
+        # see issue #2602
         (code, response) = yield self.mock_resource.trigger_get(
             "/events?access_token=%s" % ("invalid" + self.token, )
         )
-        self.assertEquals(403, code, msg=str(response))
+        self.assertEquals(401, code, msg=str(response))
 
         # valid token, expect content
         (code, response) = yield self.mock_resource.trigger_get(
diff --git a/tests/rest/client/v1/test_profile.py b/tests/rest/client/v1/test_profile.py
--- a/tests/rest/client/v1/test_profile.py
+++ b/tests/rest/client/v1/test_profile.py
@@ -52,7 +52,7 @@ def setUp(self):
         def _get_user_by_req(request=None, allow_guest=False):
             return synapse.types.create_requester(myid)
 
-        hs.get_v1auth().get_user_by_req = _get_user_by_req
+        hs.get_auth().get_user_by_req = _get_user_by_req
 
         profile.register_servlets(hs, self.mock_resource)
 
diff --git a/tests/rest/client/v1/test_rooms.py b/tests/rest/client/v1/test_rooms.py
--- a/tests/rest/client/v1/test_rooms.py
+++ b/tests/rest/client/v1/test_rooms.py
@@ -60,7 +60,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "token_id": 1,
                 "is_guest": False,
             }
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
@@ -70,7 +70,7 @@ def _insert_client_ip(*args, **kwargs):
 
         synapse.rest.client.v1.room.register_servlets(hs, self.mock_resource)
 
-        self.auth = hs.get_v1auth()
+        self.auth = hs.get_auth()
 
         # create some rooms under the name rmcreator_id
         self.uncreated_rmid = "!aa:test"
@@ -425,7 +425,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "token_id": 1,
                 "is_guest": False,
             }
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
@@ -507,7 +507,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "token_id": 1,
                 "is_guest": False,
             }
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
@@ -597,7 +597,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "is_guest": False,
             }
 
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
@@ -711,7 +711,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "token_id": 1,
                 "is_guest": False,
             }
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
@@ -843,7 +843,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "token_id": 1,
                 "is_guest": False,
             }
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
@@ -945,7 +945,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "token_id": 1,
                 "is_guest": False,
             }
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
@@ -1017,7 +1017,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "token_id": 1,
                 "is_guest": False,
             }
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)
diff --git a/tests/rest/client/v1/test_typing.py b/tests/rest/client/v1/test_typing.py
--- a/tests/rest/client/v1/test_typing.py
+++ b/tests/rest/client/v1/test_typing.py
@@ -68,7 +68,7 @@ def get_user_by_access_token(token=None, allow_guest=False):
                 "is_guest": False,
             }
 
-        hs.get_v1auth().get_user_by_access_token = get_user_by_access_token
+        hs.get_auth().get_user_by_access_token = get_user_by_access_token
 
         def _insert_client_ip(*args, **kwargs):
             return defer.succeed(None)