Repository: zulip/zulip

Issue:
Slack webhook: do not read body twice
The code in   https://github.com/zulip/zulip/blob/c8849f8fe30bf96c227044b94ee5c269b166a1b9/zerver/webhooks/slack_incoming/view.py#L26-L35 unfortunately cannot work as documented, since `request.body` has already been parsed.  We will need to not use `@has_request_variables` and instead look at the content-type to determine how to parse the body, since it can only be read once.

As is, this results in the following, if `payload` is not provided:
```
RawPostDataException: You cannot access body after reading from request's data stream
  File "zerver/decorator.py", line 334, in _wrapped_func_arguments
    return view_func(request, user_profile, *args, **kwargs)
  File "zerver/lib/request.py", line 390, in _wrapped_view_func
    return view_func(request, *args, **kwargs)
  File "zerver/webhooks/slack_incoming/view.py", line 33, in api_slack_incoming_webhook
    payload = orjson.loads(request.body)
  File "django/http/request.py", line 328, in body
    raise RawPostDataException("You cannot access body after reading from request's data stream")

You cannot access body after reading from request's data stream
```


Discussion:
Hello @zulip/server-integrations members, this issue was labeled with the "area: integrations" label, so you may want to check it out!

<!-- areaLabelAddition -->
I can't replicate the issue. What is the exact payload that causes the error?
Unfortunately, this was an error observed from an exception in production -- and I don't have the trace anymore.  I suspect this is triggerable with a `application/x-www-form-urlencoded` body which _doesn't_ set the `body` parameter.  Did you give that a shot?
Ah, found the trace.

## Query string

Key | Value
-- | --
`api_key` | [redacted]
`stream` | [redacted]
`topic` | [redacted]

## Body
(this is presumably the parsed form of the `multipart/form-data`)
<table>
<tr>
<th> Key </th> <th> Value </th>
</tr>
<tr>
<td><code>content</code></td>
<td>

```
Message:

From:
```

</td></tr></table>

## Headers

Key | value
-- | --
`Accept-Encoding` | `gzip, deflate`
`Content-Length` | `178`
`Content-Type` | `multipart/form-data; boundary=--------------------------780381918857007658710229`

-----

Sadly, I don't have any more "raw" version of the request than that, but hopefully that helps.
I have tried sending requests using `application/x-www-form-urlencoded` and `multipart/form-data`, with and without files, but with no luck.
Given that this doesn't reproduce, the original traceback is likely caused by some non-Slack software being used to send a notification into Zulip with this webhook.
Yeah.  I think we can close this unless it recurs.
This just recurred, also clearly from a (probably manual) test message.

I can replicate with:
```
https -v --multipart POST https://some-hostname.zulipchat.com/api/v1/external/slack_incoming api_key==redacted stream==general topic==foo
```
...which produces:
```
POST /api/v1/external/slack_incoming?api_key=redacted&stream=general&topic=foo HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 38
Content-Type: multipart/form-data; boundary=14537ad40cab4c77b6699c9c0fa9c82f
Host: some-hostname.zulipchat.com

--14537ad40cab4c77b6699c9c0fa9c82f--
```
Hi guys, I am interested in picking this up, however I don't know how to replicate this issue in my local vagrant setup. Can someone please guide me on that?
I believe https://github.com/zulip/zulip/issues/19056#issuecomment-1040863023 contains a replication recipe. (You'll presumably want to adjust the hostname/etc. to target your development environment, look at http://localhost:9991/api for some useful details). 
For reference, the `https` tool which I show using, can be installed via `pip install httpie`, [among other ways](https://httpie.io/docs/cli/installation).

Resolving patch:
diff --git a/zerver/webhooks/slack_incoming/view.py b/zerver/webhooks/slack_incoming/view.py
--- a/zerver/webhooks/slack_incoming/view.py
+++ b/zerver/webhooks/slack_incoming/view.py
@@ -7,10 +7,9 @@
 from django.utils.translation import gettext as _
 
 from zerver.decorator import webhook_view
-from zerver.lib.exceptions import InvalidJSONError
-from zerver.lib.request import REQ, has_request_variables
+from zerver.lib.exceptions import InvalidJSONError, JsonableError
+from zerver.lib.request import REQ, RequestVariableMissingError, has_request_variables
 from zerver.lib.response import json_success
-from zerver.lib.validator import check_dict
 from zerver.lib.webhooks.common import check_send_webhook_message
 from zerver.models import UserProfile
 
@@ -21,19 +20,28 @@ def api_slack_incoming_webhook(
     request: HttpRequest,
     user_profile: UserProfile,
     user_specified_topic: Optional[str] = REQ("topic", default=None),
-    payload: Optional[Dict[str, Any]] = REQ("payload", json_validator=check_dict(), default=None),
 ) -> HttpResponse:
 
     # Slack accepts webhook payloads as payload="encoded json" as
     # application/x-www-form-urlencoded, as well as in the body as
-    # application/json. We use has_request_variables to try to get
-    # the form encoded version, and parse the body out ourselves if
-    # # we were given JSON.
-    if payload is None:
+    # application/json.
+    if request.content_type == "application/json":
         try:
-            payload = orjson.loads(request.body)
-        except orjson.JSONDecodeError:  # nocoverage
-            raise InvalidJSONError(_("Malformed JSON"))
+            val = request.body.decode(request.encoding or "utf-8")
+        except UnicodeDecodeError:  # nocoverage
+            raise JsonableError(_("Malformed payload"))
+    else:
+        req_var = "payload"
+        if req_var in request.POST:
+            val = request.POST[req_var]
+        elif req_var in request.GET:  # nocoverage
+            val = request.GET[req_var]
+        else:
+            raise RequestVariableMissingError(req_var)
+    try:
+        payload = orjson.loads(val)
+    except orjson.JSONDecodeError:  # nocoverage
+        raise InvalidJSONError(_("Malformed JSON"))
 
     if user_specified_topic is None and "channel" in payload:
         user_specified_topic = re.sub("^[@#]", "", payload["channel"])


Test patch:
diff --git a/zerver/webhooks/slack_incoming/tests.py b/zerver/webhooks/slack_incoming/tests.py
--- a/zerver/webhooks/slack_incoming/tests.py
+++ b/zerver/webhooks/slack_incoming/tests.py
@@ -37,6 +37,15 @@ def test_message_as_www_urlencoded(self) -> None:
             content_type="application/x-www-form-urlencoded",
         )
 
+    def test_message_without_payload(self) -> None:
+        self.url = self.build_webhook_url()
+        result = self.client_post(
+            self.url,
+            None,  # type: ignore[arg-type] # we need to simulate a http request that doesn't send any payload
+            content_type="multipart/form-data; boundary=14537ad40cab4c77b6699c9c0fa9c82f",
+        )
+        self.assert_json_error(result, "Missing 'payload' argument")
+
     def test_message_with_actions(self) -> None:
         expected_topic = "C1H9RESGL"
         expected_message = """