Repository: hyperledger/aries-cloudagent-python

Issue:
Websocket webhook not working with --admin-api-key enabled
Hi all,

I'm using the websocket webhook to receive changes from aca-py, e.g. new connection invitations and I pretty much like this feature :-)

Unfortunately after activating ```--admin-api-key```, websocket can't connect any more. Reason: the ```server.py``` middleware code correctly checks for the http header ```x-api-key```
Problem is, that with (builtin) Javascript ```WebSocket()``` there is no way to set custom http-headers. Only way (I found) is when switching to "polling" protocol, which is the fallback and probably not the reason why to use websockets in the beginning.

I think we somehow should enable the admin-api-key also for websockets and I see 2 options at the moment:

1. Allow query params as an additional method to transfer ```x-api-key``` to aca-py.
One-liner change on the server in [server.py](https://github.com/hyperledger/aries-cloudagent-python/blob/master/aries_cloudagent/admin/server.py#L169)
```
header_admin_api_key = request.headers.get("x-api-key") or request.query.get("x-api-key")
```
Client:
```
new WebSocket('wss://localhost:10000/ws?x-api-key=12345
```
pro: Rather simple fix on both sides
con: URL with the key in the browser cache and server logs? (Still, client can decide not to use this feature...)


2. Send a message with the api-key to the server after websocket connection is established.
Pro: Security?
Con:
Need to define message.
Keep state of connection and only publish messages when allowed to do so.
Maybe more likely to accidentally publish data on websocket connections with wrong state (before api-key message) when implementing new features?

3. any other ideas?

Thanks for your help,
Matthias





Discussion:
@andrewwhitehead @nrempel -- please take a look and help out with this one.

Thanks!
@swcurran I now created a pull request for option 1. Anything I can help with to get this merged?
Hi @matgnt, sorry about the delay.

Your implementation looks good as far as I can tell. However, I agree about the security concern with passing the api key in the url which is generally bad practice.

I think the best solution will be to open the connection and for the server to wait for an api key message. This isn't as easy as adding a query parameter though.

That being said, if you are using the builtin Javascript `WebSocket` class, does that mean you are connecting from a browser? If that's the case, then the API key will always be insecure. Most non-browser implementations of a websocket client _do_ support sending headers with a websocket connection request.

For example:

https://godoc.org/golang.org/x/net/websocket#Config
https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientSession.ws_connect
Hi @nrempel 

thanks for the feedback! Yes, I'm using `WebSocket` from the browser.

I now created a [commit](https://github.com/boschresearch/aries-cloudagent-python/commit/d2dca06248362f15755e8d0145f76d3500eceb58) with a basic option 2 implementation - still with rough edges ;-)

It basically does the following things:
1. Exception for the /ws path from middleware logic that closes the connection if no api key is given in header
2. Add a property api_key_authenticated to the websocket / BasicMessageQueue
3. Only send messages to websocket if it is authenticated

I also added a simple html page to check the console output.

Todos:
- Still allow http headers (non-browser websocket implementations)
- message structure for passing the apiKey
- ...

@nrempel What do you think? Is this going to the right direction? Thanks for your feedback,
Matthias

@matgnt so I think we should not support connections from a web browser by design. In any scenario, if a controller connects to the agent from the browser, the API key will be exposed to the end-user. Whether by a network call to retrieve the key or having the key hard-coded in the HTML or Javascript, it will always be exposed.

Can you share a bit more about what you are trying to accomplish?


@nrempel Good that you ask, yes, the api key is exposed to the user.

It started as a system like a "developer playground" where developers can easily spawn new agents and connect to them to e.g. issue credentials and "play" with SSI. Only thing missing is notifications when, e.g. new invitations are in the queue.

I see different options to solve this problem:
1. poll the agent (from the frontend)
2. create a controller that does the notification through websockets to the frontend
3. run the agent in insecure mode (which allows everyone to connect).
4. using websockets with api key to get notifications from the agent directly. Only the user who created the agent is able to connect with its "private" / individual api key - of course, the system that started the agent also knows the api key.

I thought number 4 would be a good idea. I thought the user who knows the api key is the "owner" of the agent. It's like a hosting service for agents and you authenticate by knowing the api key.


I've created a new pull request #528 with a message structure to send the api key and test cases for both cases: authentication via header and via websocket message content.

Let me know if I can help with anything.
Hi @matgnt, we do not support connecting to the agent from a browser by design since connecting from a browser is inherently insecure. No matter which method we use to solve the connection problem, the API key will still be exposed to the end-user.

In the case of a developer playground, I recommend running the agents in insecure mode.

Resolving patch:
diff --git a/aries_cloudagent/admin/server.py b/aries_cloudagent/admin/server.py
--- a/aries_cloudagent/admin/server.py
+++ b/aries_cloudagent/admin/server.py
@@ -135,6 +135,10 @@ def __init__(
             task_queue: An optional task queue for handlers
         """
         self.app = None
+        self.admin_api_key = context.settings.get("admin.admin_api_key")
+        self.admin_insecure_mode = bool(
+            context.settings.get("admin.admin_insecure_mode")
+        )
         self.host = host
         self.port = port
         self.conductor_stats = conductor_stats
@@ -156,26 +160,29 @@ async def make_application(self) -> web.Application:
 
         middlewares = [validation_middleware]
 
-        admin_api_key = self.context.settings.get("admin.admin_api_key")
-        admin_insecure_mode = self.context.settings.get("admin.admin_insecure_mode")
-
         # admin-token and admin-token are mutually exclusive and required.
         # This should be enforced during parameter parsing but to be sure,
         # we check here.
-        assert admin_insecure_mode or admin_api_key
-        assert not (admin_insecure_mode and admin_api_key)
+        assert self.admin_insecure_mode ^ bool(self.admin_api_key)
+
+        def is_unprotected_path(path: str):
+            return path in [
+                "/api/doc",
+                "/api/docs/swagger.json",
+                "/favicon.ico",
+                "/ws",  # ws handler checks authentication
+            ] or path.startswith("/static/swagger/")
 
         # If admin_api_key is None, then admin_insecure_mode must be set so
         # we can safely enable the admin server with no security
-        if admin_api_key:
+        if self.admin_api_key:
 
             @web.middleware
             async def check_token(request, handler):
                 header_admin_api_key = request.headers.get("x-api-key")
-                if not header_admin_api_key:
-                    raise web.HTTPUnauthorized()
+                valid_key = self.admin_api_key == header_admin_api_key
 
-                if admin_api_key == header_admin_api_key:
+                if valid_key or is_unprotected_path(request.path):
                     return await handler(request)
                 else:
                     raise web.HTTPUnauthorized()
@@ -283,6 +290,12 @@ async def stop(self) -> None:
 
     async def on_startup(self, app: web.Application):
         """Perform webserver startup actions."""
+        if self.admin_api_key:
+            swagger = app["swagger_dict"]
+            swagger["securityDefinitions"] = {
+                "ApiKeyHeader": {"type": "apiKey", "in": "header", "name": "X-API-KEY"}
+            }
+            swagger["security"] = [{"ApiKeyHeader": []}]
 
     @docs(tags=["server"], summary="Fetch the list of loaded plugins")
     @response_schema(AdminModulesSchema(), 200)
@@ -355,12 +368,21 @@ async def websocket_handler(self, request):
         queue = BasicMessageQueue()
         loop = asyncio.get_event_loop()
 
+        if self.admin_insecure_mode:
+            # open to send websocket messages without api key auth
+            queue.authenticated = True
+        else:
+            header_admin_api_key = request.headers.get("x-api-key")
+            # authenticated via http header?
+            queue.authenticated = header_admin_api_key == self.admin_api_key
+
         try:
             self.websocket_queues[socket_id] = queue
             await queue.enqueue(
                 {
                     "topic": "settings",
                     "payload": {
+                        "authenticated": queue.authenticated,
                         "label": self.context.settings.get("default_label"),
                         "endpoint": self.context.settings.get("default_endpoint"),
                         "no_receive_invites": self.context.settings.get(
@@ -372,7 +394,7 @@ async def websocket_handler(self, request):
             )
 
             closed = False
-            receive = loop.create_task(ws.receive())
+            receive = loop.create_task(ws.receive_json())
             send = loop.create_task(queue.dequeue(timeout=5.0))
 
             while not closed:
@@ -384,9 +406,22 @@ async def websocket_handler(self, request):
                         closed = True
 
                     if receive.done():
-                        # ignored
                         if not closed:
-                            receive = loop.create_task(ws.receive())
+                            msg_received = None
+                            msg_api_key = None
+                            try:
+                                # this call can re-raise exeptions from inside the task
+                                msg_received = receive.result()
+                                msg_api_key = msg_received.get("x-api-key")
+                            except Exception:
+                                LOGGER.exception(
+                                    "Exception in websocket receiving task:"
+                                )
+                            if self.admin_api_key and self.admin_api_key == msg_api_key:
+                                # authenticated via websocket message
+                                queue.authenticated = True
+
+                            receive = loop.create_task(ws.receive_json())
 
                     if send.done():
                         try:
@@ -397,7 +432,10 @@ async def websocket_handler(self, request):
                         if msg is None:
                             # we send fake pings because the JS client
                             # can't detect real ones
-                            msg = {"topic": "ping"}
+                            msg = {
+                                "topic": "ping",
+                                "authenticated": queue.authenticated,
+                            }
                         if not closed:
                             if msg:
                                 await ws.send_json(msg)
@@ -441,4 +479,5 @@ async def send_webhook(self, topic: str, payload: dict):
                     )
 
         for queue in self.websocket_queues.values():
-            await queue.enqueue({"topic": topic, "payload": payload})
+            if queue.authenticated or topic in ("ping", "settings"):
+                await queue.enqueue({"topic": topic, "payload": payload})
diff --git a/aries_cloudagent/transport/queue/basic.py b/aries_cloudagent/transport/queue/basic.py
--- a/aries_cloudagent/transport/queue/basic.py
+++ b/aries_cloudagent/transport/queue/basic.py
@@ -14,6 +14,7 @@ def __init__(self):
         self.queue = self.make_queue()
         self.logger = logging.getLogger(__name__)
         self.stop_event = asyncio.Event()
+        self.authenticated = False
 
     def make_queue(self):
         """Create the queue instance."""


Test patch:
diff --git a/aries_cloudagent/admin/tests/test_admin_server.py b/aries_cloudagent/admin/tests/test_admin_server.py
--- a/aries_cloudagent/admin/tests/test_admin_server.py
+++ b/aries_cloudagent/admin/tests/test_admin_server.py
@@ -218,6 +218,35 @@ async def test_status_secure(self):
         result = await resp.json()
         assert isinstance(result, dict)
 
+    @unittest_run_loop
+    async def test_websocket_with_api_key_message(self):
+        async with self.client.ws_connect("/ws") as ws:
+            result = await ws.receive_json()
+            assert result["topic"] == "settings"
+
+            ping1 = await ws.receive_json()
+            assert ping1["topic"] == "ping"
+            assert ping1["authenticated"] == False
+
+            await ws.send_json({"dummy": ""})
+            ping2 = await ws.receive_json()
+            assert ping2["authenticated"] == False
+
+            await ws.send_json({"x-api-key": self.TEST_API_KEY})
+            ping3 = await ws.receive_json()
+            assert ping3["authenticated"] == True
+
+    @unittest_run_loop
+    async def test_websocket_with_api_key_header(self):
+        async with self.client.ws_connect(
+            "/ws", headers={"x-api-key": self.TEST_API_KEY}
+        ) as ws:
+            result = await ws.receive_json()
+            assert result["topic"] == "settings"
+
+            ping1 = await ws.receive_json()
+            assert ping1["authenticated"] == True
+
 
 class TestAdminServerWebhook(AioHTTPTestCase):
     async def setUpAsync(self):