Repository: aio-libs/aiohttp

Issue:
Cleanup doesn't run when exception occurs in startup.
When an exception occurs in startup, the cleanup is never run.

Expected behaviour: Any cleanup steps added should be run, especially when using the cleanup-context.

e.g.
Task A = Setup DB as cleanup context.
Task B = Setup a remote connection as cleanup context. This throws an exception.

Actual behaviour:
  Task A is successfully setup.
  Task B creates an exception.
  No cleanup happens.

Expected behaviour:
  Task A is successfully setup.
  Task B creates an exception.
  Task A cleanup happens.


Discussion:
Please use cleanup_ctx instead
@asvetlov As I mentioned in that PR, and in the description of this issue, I am using a cleanup context.

If an exception occurs in another part of the startup, the cleanup context does not get cleaned up, despite having been successfully initiated.
Thanks for the clarification.
Please let me keep the issue open.
Would you provide a reproducible example to save my time?
This is a minimal example of what I mean. Expected behaviour is that "CLEANUP" is printed after the exception occurs.

```
from aiohttp import web

async def cleanup_ctx(app):
    print("SETUP")
    yield
    print("CLEANUP")

async def an_error(app):
    raise Exception()
    yield

def init_func(argv):
    app = web.Application()
    app.cleanup_ctx.append(cleanup_ctx)
    app.cleanup_ctx.append(an_error)
    return app
```

The reason this is such a problem in my use case, is that the cleanup_ctx is actually launching a process which listens on a specific port. The cleanup part will shut down that process. If the cleanup is not run, then devtools restarts the server and spawns a new process which then fails because the old process is still bound to the port, meaning that the application breaks and I have to shutdown everything and restart devtools, which is a massive pain when trying to develop another part of the application with its own startup steps.
This would also need a small change to devtools in order to ensure the cleanup is run, which I've suggested here: https://github.com/aio-libs/aiohttp-devtools/pull/266/files

But, it won't do anything unless aiohttp ensures that the cleanup call actually does something at this stage.

Resolving patch:
diff --git a/aiohttp/web_app.py b/aiohttp/web_app.py
--- a/aiohttp/web_app.py
+++ b/aiohttp/web_app.py
@@ -318,7 +318,11 @@ async def cleanup(self) -> None:
 
         Should be called after shutdown()
         """
-        await self.on_cleanup.send(self)
+        if self.on_cleanup.frozen:
+            await self.on_cleanup.send(self)
+        else:
+            # If an exception occurs in startup, ensure cleanup contexts are completed.
+            await self._cleanup_ctx._on_cleanup(self)
 
     def _prepare_middleware(self) -> Iterator[_Middleware]:
         yield from reversed(self._middlewares)
diff --git a/aiohttp/web_runner.py b/aiohttp/web_runner.py
--- a/aiohttp/web_runner.py
+++ b/aiohttp/web_runner.py
@@ -288,10 +288,6 @@ async def shutdown(self) -> None:
     async def cleanup(self) -> None:
         loop = asyncio.get_event_loop()
 
-        if self._server is None:
-            # no started yet, do nothing
-            return
-
         # The loop over sites is intentional, an exception on gather()
         # leaves self._sites in unpredictable state.
         # The loop guarantees that a site is either deleted on success or


Test patch:
diff --git a/tests/test_web_app.py b/tests/test_web_app.py
--- a/tests/test_web_app.py
+++ b/tests/test_web_app.py
@@ -250,6 +250,32 @@ async def inner(app):
     assert out == ["pre_1", "pre_2", "pre_3", "post_3", "post_2", "post_1"]
 
 
+async def test_cleanup_ctx_cleanup_after_exception() -> None:
+    app = web.Application()
+    ctx_state = None
+
+    async def success_ctx(app):
+        nonlocal ctx_state
+        ctx_state = "START"
+        yield
+        ctx_state = "CLEAN"
+
+    async def fail_ctx(app):
+        raise Exception()
+        yield
+
+    app.cleanup_ctx.append(success_ctx)
+    app.cleanup_ctx.append(fail_ctx)
+    runner = web.AppRunner(app)
+    try:
+        with pytest.raises(Exception):
+            await runner.setup()
+    finally:
+        await runner.cleanup()
+
+    assert ctx_state == "CLEAN"
+
+
 async def test_cleanup_ctx_exception_on_cleanup_multiple() -> None:
     app = web.Application()
     out = []