Repository: DataDog/dd-trace-py

Issue:
UnicodeDecodeError for invalid headers in ASGI middleware
When the ASGI middleware receives headers with invalid encoding, for example as the result of somebody fuzzing an endpoint as is the case here, it introduces a Unicode decode error into the HTTP pipeline. It shouldn't be doing this.

### Which version of dd-trace-py are you using?

0.46.0

### Which version of the libraries are you using?

What I believe are the potentially relevant libraries are listed below.

```
datadog==0.40.1
...
ddtrace==0.46.0
...
fastapi==0.63.0
...
httpcore==0.12.3
httptools==0.1.1
...
starlette==0.13.6
...
uvicorn==0.13.4
uvloop==0.15.2
```

### How can we reproduce your problem?

```python
from ddtrace.contrib.asgi.middleware import _extract_headers


def test_ddtrace_invalid_header_name_encoding():
    assert _extract_headers({"headers": [((bytes.fromhex("c0")), "test")]}) == {}


def test_ddtrace_invalid_header_value_encoding():
    assert _extract_headers({"headers": [("test", (bytes.fromhex("c0")))]}) == {}
```

### What is the result that you get?

The middleware blows up with a `UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 0: invalid start byte` exception.

### What is the result that you expected?

This is trace middleware so it should not introduce exceptions into request processing. In the case of a header it can't decode then I'd expect it to either:

1. Omit the invalid header from the result and continue attempting to decode the remainder, or
2. Substitute a sentinel value for the unprocessable header to indicate an error

I've suggested (1) in the repro above, as that seems like it'd be the least likely to cause any problems.


Discussion:

Resolving patch:
diff --git a/ddtrace/contrib/asgi/middleware.py b/ddtrace/contrib/asgi/middleware.py
--- a/ddtrace/contrib/asgi/middleware.py
+++ b/ddtrace/contrib/asgi/middleware.py
@@ -85,11 +85,15 @@ async def __call__(self, scope, receive, send):
         if scope["type"] != "http":
             return await self.app(scope, receive, send)
 
-        headers = _extract_headers(scope)
-
-        trace_utils.activate_distributed_headers(
-            self.tracer, int_config=self.integration_config, request_headers=headers
-        )
+        try:
+            headers = _extract_headers(scope)
+        except Exception:
+            log.warning("failed to decode headers for distributed tracing", exc_info=True)
+            headers = {}
+        else:
+            trace_utils.activate_distributed_headers(
+                self.tracer, int_config=self.integration_config, request_headers=headers
+            )
 
         resource = "{} {}".format(scope["method"], scope["path"])
 


Test patch:
diff --git a/tests/contrib/asgi/test_asgi.py b/tests/contrib/asgi/test_asgi.py
--- a/tests/contrib/asgi/test_asgi.py
+++ b/tests/contrib/asgi/test_asgi.py
@@ -336,3 +336,29 @@ async def test_multiple_requests(tracer, test_spans):
     assert r2_span.get_tag("http.method") == "GET"
     assert r2_span.get_tag("http.url") == "http://testserver/"
     assert r2_span.get_tag("http.query.string") == "sleep=true"
+
+
+@pytest.mark.asyncio
+async def test_bad_headers(scope, tracer, test_spans):
+    """
+    When headers can't be decoded
+        The middleware should not raise
+    """
+
+    app = TraceMiddleware(basic_app, tracer=tracer)
+    headers = [(bytes.fromhex("c0"), "test")]
+    scope["headers"] = headers
+
+    instance = ApplicationCommunicator(app, scope)
+    await instance.send_input({"type": "http.request", "body": b""})
+    response_start = await instance.receive_output(1)
+    assert response_start == {
+        "type": "http.response.start",
+        "status": 200,
+        "headers": [[b"Content-Type", b"text/plain"]],
+    }
+    response_body = await instance.receive_output(1)
+    assert response_body == {
+        "type": "http.response.body",
+        "body": b"*",
+    }