Repository: aio-libs/aiohttp

Issue:
web._cancel_all_tasks causes spurious errors on Python <3.7
## Long story short

Python 3.5's `asyncio.Task.all_tasks` differs from Python 3.7's `asyncio.all_tasks` in that it [includes completed tasks](https://github.com/python/cpython/blob/ddc613f20394eba25f2420f6d2fa62b277c95058/Lib/asyncio/tasks.py#L42). However, the `all_tasks` helper uses them interchangeably.

This means that when you shutdown a server started with `web.run_app` on Python 3.5 or 3.6, failed requests are rethrown as exceptions in [`web._cancel_all_tasks`](https://github.com/aio-libs/aiohttp/blob/7a324fd46ff7dc9bb0bb1bc5afb326e04cf7cef0/aiohttp/web.py#L353), whether they were handled or not.

## Steps to reproduce
1. `python3.5 -c 'from aiohttp import web; web.run_app(web.Application())'`
2. visit http://localhost:8080
3. ctrl+c the server

## Expected behaviour

```
```

## Actual behaviour
```
unhandled exception during asyncio.run() shutdown
task: <Task finished coro=<Application._handle() done, defined at /usr/lib/python3.5/site-packages/aiohttp/web_app.py:405> exception=<HTTPNotFound Not Found eof>
  File "/usr/lib/python3.5/site-packages/aiohttp/web_protocol.py", line 406, in start
    resp = await task
  File "/usr/lib/python3.5/asyncio/futures.py", line 380, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 304, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/usr/lib/python3.5/site-packages/aiohttp/web_app.py", line 435, in _handle
    resp = await handler(request)
  File "/usr/lib/python3.5/site-packages/aiohttp/web_urldispatcher.py", line 848, in _handle
    raise self._http_exception
aiohttp.web_exceptions.HTTPNotFound: Not Found
```
web._cancel_all_tasks causes spurious errors on Python <3.7
## Long story short

Python 3.5's `asyncio.Task.all_tasks` differs from Python 3.7's `asyncio.all_tasks` in that it [includes completed tasks](https://github.com/python/cpython/blob/ddc613f20394eba25f2420f6d2fa62b277c95058/Lib/asyncio/tasks.py#L42). However, the `all_tasks` helper uses them interchangeably.

This means that when you shutdown a server started with `web.run_app` on Python 3.5 or 3.6, failed requests are rethrown as exceptions in [`web._cancel_all_tasks`](https://github.com/aio-libs/aiohttp/blob/7a324fd46ff7dc9bb0bb1bc5afb326e04cf7cef0/aiohttp/web.py#L353), whether they were handled or not.

## Steps to reproduce
1. `python3.5 -c 'from aiohttp import web; web.run_app(web.Application())'`
2. visit http://localhost:8080
3. ctrl+c the server

## Expected behaviour

```
```

## Actual behaviour
```
unhandled exception during asyncio.run() shutdown
task: <Task finished coro=<Application._handle() done, defined at /usr/lib/python3.5/site-packages/aiohttp/web_app.py:405> exception=<HTTPNotFound Not Found eof>
  File "/usr/lib/python3.5/site-packages/aiohttp/web_protocol.py", line 406, in start
    resp = await task
  File "/usr/lib/python3.5/asyncio/futures.py", line 380, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 304, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/usr/lib/python3.5/site-packages/aiohttp/web_app.py", line 435, in _handle
    resp = await handler(request)
  File "/usr/lib/python3.5/site-packages/aiohttp/web_urldispatcher.py", line 848, in _handle
    raise self._http_exception
aiohttp.web_exceptions.HTTPNotFound: Not Found
```


Discussion:
[GitMate.io](https://gitmate.io) thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are https://github.com/aio-libs/aiohttp/issues/3242 (Error), https://github.com/aio-libs/aiohttp/issues/1745 (Call https website through proxy will cause error), https://github.com/aio-libs/aiohttp/issues/816 (Logging error), https://github.com/aio-libs/aiohttp/issues/1839 (aiohttp webserver with python 3.7 broken), and https://github.com/aio-libs/aiohttp/issues/429 (Websocket error).
Thanks for the report.
I've committed the difference into asyncio but forgot about the behavior change on working on aiohttp.
[GitMate.io](https://gitmate.io) thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are https://github.com/aio-libs/aiohttp/issues/3242 (Error), https://github.com/aio-libs/aiohttp/issues/1745 (Call https website through proxy will cause error), https://github.com/aio-libs/aiohttp/issues/816 (Logging error), https://github.com/aio-libs/aiohttp/issues/1839 (aiohttp webserver with python 3.7 broken), and https://github.com/aio-libs/aiohttp/issues/429 (Websocket error).
Thanks for the report.
I've committed the difference into asyncio but forgot about the behavior change on working on aiohttp.

Resolving patch:
diff --git a/aiohttp/helpers.py b/aiohttp/helpers.py
--- a/aiohttp/helpers.py
+++ b/aiohttp/helpers.py
@@ -63,10 +63,15 @@
     from typing_extensions import ContextManager
 
 
-all_tasks = asyncio.Task.all_tasks
+def all_tasks(
+        loop: Optional[asyncio.AbstractEventLoop] = None
+) -> Set['asyncio.Task[Any]']:
+    tasks = list(asyncio.Task.all_tasks(loop))  # type: ignore
+    return {t for t in tasks if not t.done()}
+
 
 if PY_37:
-    all_tasks = getattr(asyncio, 'all_tasks')  # use the trick to cheat mypy
+    all_tasks = getattr(asyncio, 'all_tasks')  # noqa
 
 
 _T = TypeVar('_T')


Test patch: