Repository: encode/uvicorn

Issue:
Can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE
Related to #111 
#1636 may be a solution 

## Reproducing

### 1. server code

```python
import asyncio
from fastapi import FastAPI, Request

app = FastAPI()

# !!!middleware is required to reproduce this!!!
@app.middleware("http")
async def add_process_time_header(request: Request, call_next):
    response = await call_next(request)
    return response

@app.post(
    "/test",
)
async def test(query: dict) -> dict:
    await asyncio.sleep(1)
    return {"foo": "bar"}
```

### 2. locust client

```python
# client.py
from locust import HttpUser, task

class WebsiteUser(HttpUser):
    @task
    def inference(self):
        rtn = self.client.post(
            "/test",
            headers={"accept": "application/json"},
            json={"query": "test"},
        )
```

### 3. call

```bash
locust -f client.py  -u 10 -r 10 --headless -t 10min -H http://localhost:8000
```

It will take 30s to reproduce this issue

## Traceback logs

```
Exception in callback H11Protocol.timeout_keep_alive_handler()
handle: <TimerHandle when=3544453.523044797 H11Protocol.timeout_keep_alive_handler()>
Traceback (most recent call last):
  File "/home/r.yang/miniconda3/envs/py37/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/home/r.yang/miniconda3/envs/py37/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 360, in timeout_keep_alive_handler
    self.conn.send(event)
  File "/home/r.yang/miniconda3/envs/py37/lib/python3.7/site-packages/h11/_connection.py", line 510, in send
    data_list = self.send_with_data_passthrough(event)
  File "/home/r.yang/miniconda3/envs/py37/lib/python3.7/site-packages/h11/_connection.py", line 535, in send_with_data_passthrough
    self._process_event(self.our_role, event)
  File "/home/r.yang/miniconda3/envs/py37/lib/python3.7/site-packages/h11/_connection.py", line 272, in _process_event
    self._cstate.process_event(role, type(event), server_switch_event)
  File "/home/r.yang/miniconda3/envs/py37/lib/python3.7/site-packages/h11/_state.py", line 289, in process_event
    self._fire_event_triggered_transitions(role, _event_type)
  File "/home/r.yang/miniconda3/envs/py37/lib/python3.7/site-packages/h11/_state.py", line 311, in _fire_event_triggered_transitions
    ) from None
h11._util.LocalProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE
```


<!-- POLAR PLEDGE BADGE START -->
> [!IMPORTANT]
> - We're using [Polar.sh](https://polar.sh/encode) so you can upvote and help fund this issue.
> - We receive the funding once the issue is completed & confirmed by you.
> - Thank you in advance for helping prioritize & fund our backlog.

<a href="https://polar.sh/encode/uvicorn/issues/1637">
<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://polar.sh/api/github/encode/uvicorn/issues/1637/pledge.svg?darkmode=1">
  <img alt="Fund with Polar" src="https://polar.sh/api/github/encode/uvicorn/issues/1637/pledge.svg">
</picture>
</a>
<!-- POLAR PLEDGE BADGE END -->



Discussion:
## >>> here is more information <<<

### how to start the server

```bash
uvicorn app:app --reload --host 0.0.0.0 --timeout-keep-alive 1
```

### versions

```
python==3.7
uvicorn==0.18.3
fastapi==0.81.0
h11==0.13.0
starlette==0.19.1
```
I cannot reproduce it. :eyes:
For other people encountering such issue - This might be caused when your server has CPU starvation.

So each client, making a request, expect a keep alive that is configured by uvicorn to 5 by default. That keepalive, if the server process is starved, wont happen on time and thus, client will bail the connection with an execute of "RemoteDisconnect" while the server will show a traceback as mentioned on the PR description.

The solution here, would be to allow more CPU for the process or find the bottleneck on the server process to allow it sending the keepalive on time. (or, course, increase the default keepalive on the server from 5 to whatever is needed, but i guess, that would mask other issues)

Resolving patch:
diff --git a/uvicorn/protocols/http/h11_impl.py b/uvicorn/protocols/http/h11_impl.py
--- a/uvicorn/protocols/http/h11_impl.py
+++ b/uvicorn/protocols/http/h11_impl.py
@@ -236,6 +236,14 @@ def handle_events(self) -> None:
                 else:
                     app = self.app
 
+                # When starting to process a request, disable the keep-alive
+                # timeout. Normally we disable this when receiving data from
+                # client and set back when finishing processing its request.
+                # However, for pipelined requests processing finishes after
+                # already receiving the next request and thus the timer may
+                # be set here, which we don't want.
+                self._unset_keepalive_if_required()
+
                 self.cycle = RequestResponseCycle(
                     scope=self.scope,
                     conn=self.conn,
diff --git a/uvicorn/protocols/http/httptools_impl.py b/uvicorn/protocols/http/httptools_impl.py
--- a/uvicorn/protocols/http/httptools_impl.py
+++ b/uvicorn/protocols/http/httptools_impl.py
@@ -326,22 +326,22 @@ def on_response_complete(self) -> None:
         if self.transport.is_closing():
             return
 
-        # Set a short Keep-Alive timeout.
         self._unset_keepalive_if_required()
 
-        self.timeout_keep_alive_task = self.loop.call_later(
-            self.timeout_keep_alive, self.timeout_keep_alive_handler
-        )
-
         # Unpause data reads if needed.
         self.flow.resume_reading()
 
-        # Unblock any pipelined events.
+        # Unblock any pipelined events. If there are none, arm the
+        # Keep-Alive timeout instead.
         if self.pipeline:
             cycle, app = self.pipeline.pop()
             task = self.loop.create_task(cycle.run_asgi(app))
             task.add_done_callback(self.tasks.discard)
             self.tasks.add(task)
+        else:
+            self.timeout_keep_alive_task = self.loop.call_later(
+                self.timeout_keep_alive, self.timeout_keep_alive_handler
+            )
 
     def shutdown(self) -> None:
         """


Test patch:
diff --git a/tests/protocols/test_http.py b/tests/protocols/test_http.py
--- a/tests/protocols/test_http.py
+++ b/tests/protocols/test_http.py
@@ -176,6 +176,20 @@ def set_protocol(self, protocol):
         pass
 
 
+class MockTimerHandle:
+    def __init__(self, loop_later_list, delay, callback, args):
+        self.loop_later_list = loop_later_list
+        self.delay = delay
+        self.callback = callback
+        self.args = args
+        self.cancelled = False
+
+    def cancel(self):
+        if not self.cancelled:
+            self.cancelled = True
+            self.loop_later_list.remove(self)
+
+
 class MockLoop:
     def __init__(self):
         self._tasks = []
@@ -186,18 +200,20 @@ def create_task(self, coroutine):
         return MockTask()
 
     def call_later(self, delay, callback, *args):
-        self._later.insert(0, (delay, callback, args))
+        handle = MockTimerHandle(self._later, delay, callback, args)
+        self._later.insert(0, handle)
+        return handle
 
     async def run_one(self):
         return await self._tasks.pop()
 
     def run_later(self, with_delay):
         later = []
-        for delay, callback, args in self._later:
-            if with_delay >= delay:
-                callback(*args)
+        for timer_handle in self._later:
+            if with_delay >= timer_handle.delay:
+                timer_handle.callback(*timer_handle.args)
             else:
-                later.append((delay, callback, args))
+                later.append(timer_handle)
         self._later = later
 
 
@@ -315,6 +331,32 @@ async def test_keepalive_timeout(http_protocol_cls: HTTPProtocol):
     assert protocol.transport.is_closing()
 
 
+@pytest.mark.anyio
+async def test_keepalive_timeout_with_pipelined_requests(
+    http_protocol_cls: HTTPProtocol,
+):
+    app = Response("Hello, world", media_type="text/plain")
+
+    protocol = get_connected_protocol(app, http_protocol_cls)
+    protocol.data_received(SIMPLE_GET_REQUEST)
+    protocol.data_received(SIMPLE_GET_REQUEST)
+
+    # After processing the first request, the keep-alive task should be
+    # disabled because the second request is not responded yet.
+    await protocol.loop.run_one()
+    assert b"HTTP/1.1 200 OK" in protocol.transport.buffer
+    assert b"Hello, world" in protocol.transport.buffer
+    assert protocol.timeout_keep_alive_task is None
+
+    # Process the second request and ensure that the keep-alive task
+    # has been enabled again as the connection is now idle.
+    protocol.transport.clear_buffer()
+    await protocol.loop.run_one()
+    assert b"HTTP/1.1 200 OK" in protocol.transport.buffer
+    assert b"Hello, world" in protocol.transport.buffer
+    assert protocol.timeout_keep_alive_task is not None
+
+
 @pytest.mark.anyio
 async def test_close(http_protocol_cls: HTTPProtocol):
     app = Response(b"", status_code=204, headers={"connection": "close"})