Repository: encode/starlette

Issue:
Error in app startup event is not logged and does not prevent startup
Related #431 #177  and mentioned here https://github.com/encode/uvicorn/issues/333#issuecomment-484885300

Scenario:
I have a 'startup' event handler:
```python
@app.on_event('startup')
async def startup():
    initialise()
```
and I am running under uvicorn for local dev just via the:
```python
if __name__ == '__main__':
    uvicorn.run(app, host='0.0.0.0', port=8000)
```

If there is an uncaught exception in the startup event handler, the exception will be swallowed (not logged) and uvicorn will continue to startup.

The only anomaly seen in the logs is that uvicorn will report:
```
ASGI 'lifespan' protocol appears unsupported
```

I would like to see:
- the exception and traceback logged
- process aborted, since we haven't fully started up yet




Discussion:
@anentropic I also encountered this issue before. I was able to have uvicorn show the traceback by passing `lifespan="on"` to `uvicorn.run()` — `lifespan` defaults to `"auto"` which makes uvicorn show the message you listed, see [on.py-L46](https://github.com/encode/uvicorn/blob/master/uvicorn/lifespan/on.py#L46).

Also, in `"auto"` mode uvicorn doesn't abort the process, see [on.py-L29](https://github.com/encode/uvicorn/blob/master/uvicorn/lifespan/on.py#L29).

Perhaps this is more of a uvicorn issue than a Starlette one?
@florimondmanca thank you, yes that does what I want now:

```python
ERROR    uvicorn                          Exception in 'lifespan' protocol

Traceback (most recent call last):
  File "/Users/anentropic/.virtualenvs/myproject-vPNUpldR/lib/python3.7/site-packages/uvicorn/lifespan/on.py", line 44, in main
    await app(scope, self.receive, self.send)
  File "/Users/anentropic/.virtualenvs/myproject-vPNUpldR/lib/python3.7/site-packages/uvicorn/middleware/asgi2.py", line 7, in __call__
    await instance(receive, send)
  File "/Users/anentropic/.virtualenvs/myproject-vPNUpldR/lib/python3.7/site-packages/starlette/routing.py", line 478, in asgi
    await self.startup()
  File "/Users/anentropic/.virtualenvs/myproject-vPNUpldR/lib/python3.7/site-packages/starlette/routing.py", line 461, in startup
    await handler()
  File "src/myproject/website/main.py", line 50, in startup
    raise ValueError('Startup Error')
ValueError: Startup Error
ERROR    uvicorn                          Application startup failed. Exiting.
```

So maybe that's enough.

[ASGI docs](https://asgi.readthedocs.io/en/latest/specs/lifespan.html#scope) say:

> If an exception is raised when calling the application callable with a `lifespan.startup` message or a scope with type `lifespan` the server must continue but not send any lifespan events.
>
> This allows for compatibility with applications that do not support the lifespan protocol. If you want to log an error that occurred during lifespan startup and prevent the server from starting, then send back `lifespan.startup.failed` instead.

I am not totally clear about the distribution of responsibilities between Uvicorn, Starlette and my app with regards to 'lifespan' events.

It seems that in `lifespan="auto"` mode Uvicorn is expecting either Starlette or my app to catch the exception (and send a `lifespan.startup.failed` event if we want to prevent startup)... so the uncaught exception is treated as a sign that the app does not support lifespan events: Uvicorn logs _"'lifespan' protocol appears unsupported"_ and continues startup, as per ASGI spec. This makes sense now.

In `lifespan="on"` mode the _"Exception in 'lifespan' protocol"_ error logged suggests to me that it is still the responsibility of either Starlette or my app to catch the exception and send a `lifespan.startup.failed` event... but in this case we have explicitly told Uvicorn that we support the lifespan protocol, so in presence of an uncaught error Uvicorn does not continue with startup but exits, logging that we violated the protocol.

The end result is what I want though so I will go with this for now.
Thanks for this writeup! It clarifies a few things about how lifespan responsibility is distributed between the server and the app. :+1:

So, the bug is that since Starlette *does* support the lifespan protocol, it should handle exceptions and send back the `lifespan.startup.failed` event in case of an error, right?

Resolving patch:
diff --git a/starlette/routing.py b/starlette/routing.py
--- a/starlette/routing.py
+++ b/starlette/routing.py
@@ -1,6 +1,7 @@
 import asyncio
 import inspect
 import re
+import traceback
 import typing
 from enum import Enum
 
@@ -469,9 +470,15 @@ async def shutdown(self) -> None:
     async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
         message = await receive()
         assert message["type"] == "lifespan.startup"
-        await self.startup()
-        await send({"type": "lifespan.startup.complete"})
 
+        try:
+            await self.startup()
+        except BaseException:
+            msg = traceback.format_exc()
+            await send({"type": "lifespan.startup.failed", "message": msg})
+            raise
+
+        await send({"type": "lifespan.startup.complete"})
         message = await receive()
         assert message["type"] == "lifespan.shutdown"
         await self.shutdown()
diff --git a/starlette/testclient.py b/starlette/testclient.py
--- a/starlette/testclient.py
+++ b/starlette/testclient.py
@@ -463,9 +463,14 @@ async def lifespan(self) -> None:
     async def wait_startup(self) -> None:
         await self.receive_queue.put({"type": "lifespan.startup"})
         message = await self.send_queue.get()
-        if message is None:
-            self.task.result()
-        assert message["type"] == "lifespan.startup.complete"
+        assert message["type"] in (
+            "lifespan.startup.complete",
+            "lifespan.startup.failed",
+        )
+        if message["type"] == "lifespan.startup.failed":
+            message = await self.send_queue.get()
+            if message is None:
+                self.task.result()
 
     async def wait_shutdown(self) -> None:
         await self.receive_queue.put({"type": "lifespan.shutdown"})


Test patch:
diff --git a/tests/middleware/test_lifespan.py b/tests/middleware/test_lifespan.py
--- a/tests/middleware/test_lifespan.py
+++ b/tests/middleware/test_lifespan.py
@@ -42,11 +42,22 @@ def test_raise_on_startup():
     def run_startup():
         raise RuntimeError()
 
-    app = Router(routes=[Lifespan(on_startup=run_startup)])
+    router = Router(routes=[Lifespan(on_startup=run_startup)])
 
+    async def app(scope, receive, send):
+        async def _send(message):
+            nonlocal startup_failed
+            if message["type"] == "lifespan.startup.failed":
+                startup_failed = True
+            return await send(message)
+
+        await router(scope, receive, _send)
+
+    startup_failed = False
     with pytest.raises(RuntimeError):
         with TestClient(app):
             pass  # pragma: nocover
+    assert startup_failed
 
 
 def test_raise_on_shutdown():
diff --git a/tests/test_testclient.py b/tests/test_testclient.py
--- a/tests/test_testclient.py
+++ b/tests/test_testclient.py
@@ -1,4 +1,5 @@
 import asyncio
+
 import pytest
 
 from starlette.applications import Starlette