Repository: elastic/apm-agent-python

Issue:
when capture_body is false, flask capture_body should not capture body at all
A code base I contribute to currently requires reading the `environ['wsgi.input']` stream directly before flask can get to it. When latest Elastic APM 5.5.2 is installed, this stream is empty when the app gets to it. This is because `environ['wsgi.input']` is a read once stream and does not support seek. When the Elastic APM capture_body logic runs, it first reads the stream before deciding if it will include it in the APM report. In my app, this flag is set to the default value of false.

Suggested fix: guard the body capturing logic with the capture_body flag; that is do not take the branch, set the body to '[REDACTED]' right away. 

The capture_body logic I'm referring to is here:

https://github.com/elastic/apm-agent-python/blob/master/elasticapm/contrib/flask/utils.py#L48-L66

Further, I figured out that going back to version 5.2.3 does not cause the wsgi.input stream to be consumed so this change occurred in 5.3.0.

**To Reproduce**

app.py
```
import flask
from flask import request
from elasticapm.contrib.flask import ElasticAPM
from werkzeug.wsgi import get_content_length


app = flask.Flask(__name__)
# Prevent ElasticAPM from contacting APM Server (which is not running on my dev machine)
app.config['ELASTIC_APM'] = {
    'DISABLE_SEND': True,
    'CENTRAL_CONFIG': False,
}
ElasticAPM(app)


@app.route('/', methods=["POST"])
def empty_input():
    length = get_content_length(request.environ)
    if length == 0:
        return "Request Content-Length is 0, you must provide a body", 400
    foo = request.environ['wsgi.input'].read(length)
    read_len = len(foo)
    if read_len != length and read_len == 0:
        return "Empty environ wsgi.input body\n", 500
    else:
        return "OK\n"
```

```
$ gunicorn app:app # using gunicorn because the flask server hangs on the read call for this scenario
```

```
$ curl -d key=foo 127.0.0.1:8000
Empty environ wsgi.input body
```

**Environment**

- OS: Docker image python:3.7-slim
- Python version: 3.7.6
- Framework and version: Flask 1.1.1
- APM Server version: 6.2.4
- Agent version: 5.5.2


**Additional context**

Add any other context about the problem here.

- Agent config options: All default.
- `requirements.txt`:
```
Flask==1.1.1
elastic-apm==5.5.2
blinker==1.4
gunicorn==20.0.4
```
when capture_body is false, flask capture_body should not capture body at all
A code base I contribute to currently requires reading the `environ['wsgi.input']` stream directly before flask can get to it. When latest Elastic APM 5.5.2 is installed, this stream is empty when the app gets to it. This is because `environ['wsgi.input']` is a read once stream and does not support seek. When the Elastic APM capture_body logic runs, it first reads the stream before deciding if it will include it in the APM report. In my app, this flag is set to the default value of false.

Suggested fix: guard the body capturing logic with the capture_body flag; that is do not take the branch, set the body to '[REDACTED]' right away. 

The capture_body logic I'm referring to is here:

https://github.com/elastic/apm-agent-python/blob/master/elasticapm/contrib/flask/utils.py#L48-L66

Further, I figured out that going back to version 5.2.3 does not cause the wsgi.input stream to be consumed so this change occurred in 5.3.0.

**To Reproduce**

app.py
```
import flask
from flask import request
from elasticapm.contrib.flask import ElasticAPM
from werkzeug.wsgi import get_content_length


app = flask.Flask(__name__)
# Prevent ElasticAPM from contacting APM Server (which is not running on my dev machine)
app.config['ELASTIC_APM'] = {
    'DISABLE_SEND': True,
    'CENTRAL_CONFIG': False,
}
ElasticAPM(app)


@app.route('/', methods=["POST"])
def empty_input():
    length = get_content_length(request.environ)
    if length == 0:
        return "Request Content-Length is 0, you must provide a body", 400
    foo = request.environ['wsgi.input'].read(length)
    read_len = len(foo)
    if read_len != length and read_len == 0:
        return "Empty environ wsgi.input body\n", 500
    else:
        return "OK\n"
```

```
$ gunicorn app:app # using gunicorn because the flask server hangs on the read call for this scenario
```

```
$ curl -d key=foo 127.0.0.1:8000
Empty environ wsgi.input body
```

**Environment**

- OS: Docker image python:3.7-slim
- Python version: 3.7.6
- Framework and version: Flask 1.1.1
- APM Server version: 6.2.4
- Agent version: 5.5.2


**Additional context**

Add any other context about the problem here.

- Agent config options: All default.
- `requirements.txt`:
```
Flask==1.1.1
elastic-apm==5.5.2
blinker==1.4
gunicorn==20.0.4
```


Discussion:

Resolving patch:
diff --git a/elasticapm/contrib/django/client.py b/elasticapm/contrib/django/client.py
--- a/elasticapm/contrib/django/client.py
+++ b/elasticapm/contrib/django/client.py
@@ -131,21 +131,25 @@ def get_data_from_request(self, request, event_type):
             result["headers"] = request_headers
 
         if request.method in constants.HTTP_WITH_BODY:
-            content_type = request.META.get("CONTENT_TYPE")
-            if content_type == "application/x-www-form-urlencoded":
-                data = compat.multidict_to_dict(request.POST)
-            elif content_type and content_type.startswith("multipart/form-data"):
-                data = compat.multidict_to_dict(request.POST)
-                if request.FILES:
-                    data["_files"] = {field: file.name for field, file in compat.iteritems(request.FILES)}
-            else:
-                try:
-                    data = request.body
-                except Exception as e:
-                    self.logger.debug("Can't capture request body: %s", compat.text_type(e))
-                    data = "<unavailable>"
             capture_body = self.config.capture_body in ("all", event_type)
-            result["body"] = data if (capture_body or not data) else "[REDACTED]"
+            if not capture_body:
+                result["body"] = "[REDACTED]"
+            else:
+                content_type = request.META.get("CONTENT_TYPE")
+                if content_type == "application/x-www-form-urlencoded":
+                    data = compat.multidict_to_dict(request.POST)
+                elif content_type and content_type.startswith("multipart/form-data"):
+                    data = compat.multidict_to_dict(request.POST)
+                    if request.FILES:
+                        data["_files"] = {field: file.name for field, file in compat.iteritems(request.FILES)}
+                else:
+                    try:
+                        data = request.body
+                    except Exception as e:
+                        self.logger.debug("Can't capture request body: %s", compat.text_type(e))
+                        data = "<unavailable>"
+                if data is not None:
+                    result["body"] = data
 
         if hasattr(request, "get_raw_uri"):
             # added in Django 1.9
diff --git a/elasticapm/contrib/flask/utils.py b/elasticapm/contrib/flask/utils.py
--- a/elasticapm/contrib/flask/utils.py
+++ b/elasticapm/contrib/flask/utils.py
@@ -46,24 +46,27 @@ def get_data_from_request(request, config, event_type):
     if config.capture_headers:
         result["headers"] = dict(get_headers(request.environ))
     if request.method in constants.HTTP_WITH_BODY:
-        body = None
-        if request.content_type == "application/x-www-form-urlencoded":
-            body = compat.multidict_to_dict(request.form)
-        elif request.content_type and request.content_type.startswith("multipart/form-data"):
-            body = compat.multidict_to_dict(request.form)
-            if request.files:
-                body["_files"] = {
-                    field: val[0].filename if len(val) == 1 else [f.filename for f in val]
-                    for field, val in compat.iterlists(request.files)
-                }
+        if config.capture_body not in ("all", event_type):
+            result["body"] = "[REDACTED]"
         else:
-            try:
-                body = request.get_data(as_text=True)
-            except ClientDisconnected:
-                pass
+            body = None
+            if request.content_type == "application/x-www-form-urlencoded":
+                body = compat.multidict_to_dict(request.form)
+            elif request.content_type and request.content_type.startswith("multipart/form-data"):
+                body = compat.multidict_to_dict(request.form)
+                if request.files:
+                    body["_files"] = {
+                        field: val[0].filename if len(val) == 1 else [f.filename for f in val]
+                        for field, val in compat.iterlists(request.files)
+                    }
+            else:
+                try:
+                    body = request.get_data(as_text=True)
+                except ClientDisconnected:
+                    pass
 
-        if body is not None:
-            result["body"] = body if config.capture_body in ("all", event_type) else "[REDACTED]"
+            if body is not None:
+                result["body"] = body
 
     result["url"] = get_url_dict(request.url)
     return result
diff --git a/elasticapm/contrib/starlette/utils.py b/elasticapm/contrib/starlette/utils.py
--- a/elasticapm/contrib/starlette/utils.py
+++ b/elasticapm/contrib/starlette/utils.py
@@ -58,17 +58,20 @@ async def get_data_from_request(request: Request, config: Config, event_type: st
         result["headers"] = dict(request.headers)
 
     if request.method in constants.HTTP_WITH_BODY:
-        body = None
-        try:
-            body = await get_body(request)
-            if request.headers.get("content-type") == "application/x-www-form-urlencoded":
-                body = await query_params_to_dict(body)
-            else:
-                body = json.loads(body)
-        except Exception:
-            pass
-        if body is not None:
-            result["body"] = body if config.capture_body in ("all", event_type) else "[REDACTED]"
+        if config.capture_body not in ("all", event_type):
+            result["body"] = "[REDACTED]"
+        else:
+            body = None
+            try:
+                body = await get_body(request)
+                if request.headers.get("content-type") == "application/x-www-form-urlencoded":
+                    body = await query_params_to_dict(body)
+                else:
+                    body = json.loads(body)
+            except Exception:
+                pass
+            if body is not None:
+                result["body"] = body
 
     result["url"] = get_url_dict(str(request.url))
 


Test patch:
diff --git a/tests/contrib/django/django_tests.py b/tests/contrib/django/django_tests.py
--- a/tests/contrib/django/django_tests.py
+++ b/tests/contrib/django/django_tests.py
@@ -613,6 +613,7 @@ def test_post_raw_data(django_elasticapm_client):
         assert request["body"] == "[REDACTED]"
 
 
+@pytest.mark.parametrize("django_elasticapm_client", [{"capture_body": "errors"}], indirect=True)
 def test_post_read_error_logging(django_elasticapm_client, caplog, rf):
     request = rf.post("/test", data="{}", content_type="application/json")
 
@@ -1426,8 +1427,10 @@ def test_capture_empty_body(client, django_elasticapm_client):
     with pytest.raises(MyException):
         client.post(reverse("elasticapm-raise-exc"), data={})
     error = django_elasticapm_client.events[ERROR][0]
-    # body should be empty no matter if we capture it or not
-    assert error["context"]["request"]["body"] == {}
+    if django_elasticapm_client.config.capture_body not in ("error", "all"):
+        assert error["context"]["request"]["body"] == "[REDACTED]"
+    else:
+        assert error["context"]["request"]["body"] == {}
 
 
 @pytest.mark.parametrize(