Repository: zulip/zulip

Issue:
Update Slack integration to use `check_send_webhook_message`.
The current Slack integration documented at https://zulip.com/integrations/doc/slack was never updated to use `check_send_webhook_message` as part of its implementation (see [`zerver/webhooks/slack/view.py`](https://github.com/zulip/zulip/blob/main/zerver/webhooks/slack/view.py)).

How to handle the `channels_map_to_topics` parameter implemented in this Slack webhook probably merits a discussion in the [# integrations](https://chat.zulip.org/#narrow/stream/127-integrations) stream on chat.zulip.org.

CZO thread: https://chat.zulip.org/#narrow/stream/9-issues/topic/slack.20webhook.20bug/near/1655623



Discussion:
Hello @zulip/server-integrations members, this issue was labeled with the "area: integrations" label, so you may want to check it out!

<!-- areaLabelAddition -->

@zulipbot claim
Hello @Rahulr2101, it looks like you've currently claimed 1 issue in this repository. We encourage new contributors to focus their efforts on at most 1 issue at a time, so please complete your work on your other claimed issues before trying to claim this issue again.

We look forward to your valuable contributions!

@zulipbot claim
Welcome to Zulip, @AlexiusTatius! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

- Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
- [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

Hey @laurynmm,
I've been working on the code changes for the assigned Slack integration issue and have been using the [commit](https://github.com/zulip/zulip/commit/a4979410f983e7aaa10f4833a9c302110bea685c)
as a reference for making the necessary changes. However, before proceeding with the changes, I wanted to test the existing behavior on my local server.
I successfully spun up the Zulip server locally using this [guide]( https://zulip.readthedocs.io/en/latest/development/setup-recommended.html#windows-10-or-11 ), but when I attempt to introduce an "outgoing webhook" for my Zulip app from Slack, the local Zulip app is not receiving any messages from Slack. On the other hand, when I set up an outgoing hook from Slack to the actual Zulip server (chat.zulip.org), the webhook functions as expected.
For your reference, here is the outgoing hook API key configuration I have set up for my local Zulip instance:
![outgoing_Webhooks](https://github.com/zulip/zulip/assets/96935985/134bc335-bb9a-4f86-bf43-573ea438a62c)

The above configuration is expected to publish the message to the Zulip server hosted on localhost at port 9991 in the test channel. However, it's not working as expected. Could you please suggest a possible way to test the changes locally? Your assistance would greatly help in progressing with the issue.
Additionally, I attempted to refer to the documentation for Zulip-Slack integration at this [link](https://zulip.com/integrations/doc/slack). Unfortunately, no text is getting rendered on this link now, whereas I distinctly remember it was rendering properly two days ago. Can you help me with this as well.
Thank you for your attention to these matters. Looking forward to your guidance and support.

@AlexiusTatius I recommend asking for help in the [Zulip development community](https://zulip.com/development-community/), where more people will see your question.
Sure, I'll reach out in the zulip community 
Thank you!
@AlexiusTatius - Just checking in to see if you'd made progress on this issue and will be putting up a PR soon as this is one of the issues being tracked for the upcoming 8.0 release.
@laurynmm - Yes I've been working on the issue and will soon put up the PR. Could you please provide information on the expected release date for the 8.0 version?
You can follow the [# release management stream](https://chat.zulip.org/#narrow/stream/438-release-management) on chat.zulip.org for up-to-date information.
Hey @laurynmm,

Apologies for my delayed response—I've been swamped with semester exams at university. Regarding the migration of Slack from `check_send_stream_message` to `check_send_webhook_message` in the Integrations Developer Panel, I've run into an issue during testing. The error "missing ‘user_name’ argument" is coming up. You can check out the screenshot of the error here.
![Integration Testing](https://github.com/zulip/zulip/assets/96935985/e2c14000-bd1f-44e7-81e5-e714ec816525)

I suspect there's an issue with the fixture body, seemingly created for a Slack team associated with gsoc 2017. I also tried creating a new webhook without specifying a URL. Should I define the URL where the outgoing webhook should send the data?
![outgoing_webhook_without_url](https://github.com/zulip/zulip/assets/96935985/02faf4d5-3bda-4dfa-9adc-122d8cbe8ae2)


Even after creating my fixture using the token from the outgoing webhook (g5uxoQHojGenqdJVlJ54xLBL), some terms like team_id=T0001, team_domain=example, user_name=steve seem to default. I've figured out my channel_id and user_id, but I'm unsure about channel_name, user_name, text, team_id. Any suggestions?

![outgoing_webhook_config](https://github.com/zulip/zulip/assets/96935985/02a7eda3-d8cb-4f5c-8150-5d67d305cd00)
 
I went through the CZO thread https://chat.zulip.org/#narrow/stream/9-issues/topic/slack.20webhook.20bug/near/1655623
and have a basic grasp of the highlighted issue.

Could you suggest some ideas on managing the `channels_map_to_topics` variable to prevent the creation of a new topic if the "topic" variable is mentioned in the URL?

Thanks for your help!
Mehtab

@AlexiusTatius You have been unassigned from this issue because you have not made any updates for over 14 days. Please feel free to reclaim the issue if you decide to pick up again. Thanks!

Yes, I've been working on the issue. Very sorry for the delay because my semester exams were holding me back. 
@zulipbot claim
@laurynmm I have solved the issue
I migrated the slack webhook from `check_send_stream_message ` to `check_send_webhook_message`.

I checked it on integration dev panel, now whenever the client mentions a new topic, the message gets delivered to that particular topic. 
For example, if I now mention the `topic name` as `topic_1` then zulip would display it as:
![1_solved_topic](https://github.com/zulip/zulip/assets/96935985/352c8ac6-7c9c-4c5b-aa57-f8028ba97b6f)

If we don’t specify the specify the topic then zulip would simply pick up the channel name of the slack.
![2_solved_topic](https://github.com/zulip/zulip/assets/96935985/253123ec-fa44-46f5-b010-075ed9ec8efc)
But there is only one thing that I wanted to ask, that was: 
When I was testing the changes on the integration developer panel, the query parameters which I added in the URL of the API endpoint, were added manually. 
What I mean is, when I specify the `Bot`, the `Integration`, the `Fixture`, the `stream` and the `Topic` I get the following `URL` generated:

![3_solved_topic](https://github.com/zulip/zulip/assets/96935985/cf79fe00-dbad-4fd4-992f-918734e2acc8)

I added the following query parameters manually `&user_name=mehtabsinghcse&text=web_bruh+1st+Text+sent&channel_name=zulip_web_local_host`
Only then the slack webhook works properly, else it shows error of 
`Result: (400) Missing ‘user_name’ argument`
One thing to note is that, when I send a message to the `test here` stream of `chat.zulip.org`. from slack using the outgoing webhook the message gets sent successfully.
![4_solved_topic](https://github.com/zulip/zulip/assets/96935985/b1d4703c-27b2-4d74-8786-899993f68c68)

The above screenshot is from `chat.zulip.org`

I believe that the changes I made would work for the production as well (as evident by the above screenshot), so there would be no need to alter the query parameters for the production.

I wanted to ask whether I should fix the query parameters for the integration testing as well or shall I raise the pull request?
@AlexiusTatius - I feel like you should open a pull request with your changes as it's hard to understand what the issue you're still running into at this point from the information above. Looking at https://github.com/zulip/zulip/blob/main/zerver/webhooks/slack/view.py, the `user_name` and `text` parameters are used to construct the message content and are likely expected in the payload from Slack.
@laurynmm I've raised the pull request
> Hey @laurynmm, I've been working on the code changes for the assigned Slack integration issue and have been using the [commit](https://github.com/zulip/zulip/commit/a4979410f983e7aaa10f4833a9c302110bea685c) as a reference for making the necessary changes. However, before proceeding with the changes, I wanted to test the existing behavior on my local server. I successfully spun up the Zulip server locally using this [guide](https://zulip.readthedocs.io/en/latest/development/setup-recommended.html#windows-10-or-11), but when I attempt to introduce an "outgoing webhook" for my Zulip app from Slack, the local Zulip app is not receiving any messages from Slack. On the other hand, when I set up an outgoing hook from Slack to the actual Zulip server (chat.zulip.org), the webhook functions as expected. For your reference, here is the outgoing hook API key configuration I have set up for my local Zulip instance: ![outgoing_Webhooks](https://private-user-images.githubusercontent.com/96935985/282293760-134bc335-bb9a-4f86-bf43-573ea438a62c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDM0Mjk5OTksIm5iZiI6MTcwMzQyOTY5OSwicGF0aCI6Ii85NjkzNTk4NS8yODIyOTM3NjAtMTM0YmMzMzUtYmI5YS00Zjg2LWJmNDMtNTczZWE0MzhhNjJjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFJV05KWUFYNENTVkVINTNBJTJGMjAyMzEyMjQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjMxMjI0VDE0NTQ1OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTg2NTQ3ZGM4YzBjNjJlZDRjYzE3ZjFiMGIwOTA2NWFlOWNiMDIzOGEzNzQ0MjVjMTRhNjAyZjA5YTc3ZjU1ZDAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.8cA0XFvN3aTBZekvjfKeK26f6UWSaq81Glpzl6zqUjY)
> 
> The above configuration is expected to publish the message to the Zulip server hosted on localhost at port 9991 in the test channel. However, it's not working as expected. Could you please suggest a possible way to test the changes locally? Your assistance would greatly help in progressing with the issue. Additionally, I attempted to refer to the documentation for Zulip-Slack integration at this [link](https://zulip.com/integrations/doc/slack). Unfortunately, no text is getting rendered on this link now, whereas I distinctly remember it was rendering properly two days ago. Can you help me with this as well. Thank you for your attention to these matters. Looking forward to your guidance and support.

Hello @AlexiusTatius , I have been facing the same issue while working on some other issues for quite a while. It would be really great if you could tell me how did you resolve this issue. 

Thanks.
@shashank-23002 
Yeah sure!
Basically, the slack outgoing webhook cannot be given a local address, 
"localhost:9991", because `localhost:9991` is a URL that is only going to resolve to computer on your computer, and on Slack's computer, it resolves to their own server's port 9991.

So, if your issue is related testing webhooks by capturing payloads, you should use the `integration dev panel` of zulip.
When you host your website you shall get this below link:
http://localhost:9991/devtools/integrations/ 
for testing the webhook integrations.
https://zulip.com/api/incoming-webhooks-overview
Checkout the above link and related documentation for how to test webhooks by capturing payloads.
@zulipbot abandon

@zulipbot claim
Welcome to Zulip, @Sadaf-A! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

- Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
- [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

@Sadaf-A Thank you for finding this issue. But I've already, solved and raised a pull request for this issue and it is waiting to get reviewed. Can you look for another issue? 
You can review my pull request and suggest changes too. Thanks :)
> @Sadaf-A Thank you for finding this issue. But I've already, solved and raised a pull request for this issue and it is waiting to get reviewed. Can you look for another issue? 
> You can review my pull request and suggest changes too. Thanks :)

I'm sorry you abandoned it so I thought it's up for contribution
@zulipbot abandon
> > @Sadaf-A Thank you for finding this issue. But I've already, solved and raised a pull request for this issue and it is waiting to get reviewed. Can you look for another issue?
> > You can review my pull request and suggest changes too. Thanks :)
> 
> I'm sorry you abandoned it so I thought it's up for contribution

No problem :)

Resolving patch:
diff --git a/zerver/webhooks/slack/view.py b/zerver/webhooks/slack/view.py
--- a/zerver/webhooks/slack/view.py
+++ b/zerver/webhooks/slack/view.py
@@ -1,13 +1,14 @@
+from typing import Optional
+
 from django.http import HttpRequest
 from django.http.response import HttpResponse
 from django.utils.translation import gettext as _
 
-from zerver.actions.message_send import check_send_stream_message
 from zerver.decorator import webhook_view
 from zerver.lib.exceptions import JsonableError
-from zerver.lib.request import RequestNotes
 from zerver.lib.response import json_success
 from zerver.lib.typed_endpoint import typed_endpoint
+from zerver.lib.webhooks.common import check_send_webhook_message
 from zerver.models import UserProfile
 
 ZULIP_MESSAGE_TEMPLATE = "**{message_sender}**: {text}"
@@ -23,20 +24,39 @@ def api_slack_webhook(
     user_name: str,
     text: str,
     channel_name: str,
-    stream: str = "slack",
-    channels_map_to_topics: str = "1",
+    channels_map_to_topics: Optional[str] = None,
 ) -> HttpResponse:
-    if channels_map_to_topics not in VALID_OPTIONS.values():
-        raise JsonableError(_("Error: channels_map_to_topics parameter other than 0 or 1"))
+    content = ZULIP_MESSAGE_TEMPLATE.format(message_sender=user_name, text=text)
+    topic_name = "Message from Slack"
 
-    if channels_map_to_topics == VALID_OPTIONS["SHOULD_BE_MAPPED"]:
+    if channels_map_to_topics is None:
+        check_send_webhook_message(
+            request,
+            user_profile,
+            topic_name,
+            content,
+        )
+    elif channels_map_to_topics == VALID_OPTIONS["SHOULD_BE_MAPPED"]:
+        # If the webhook URL has a user_specified_topic,
+        # then this topic-channel mapping will not be used.
         topic_name = f"channel: {channel_name}"
+        check_send_webhook_message(
+            request,
+            user_profile,
+            topic_name,
+            content,
+        )
+    elif channels_map_to_topics == VALID_OPTIONS["SHOULD_NOT_BE_MAPPED"]:
+        # This stream-channel mapping will be used even if
+        # there is a stream specified in the webhook URL.
+        check_send_webhook_message(
+            request,
+            user_profile,
+            topic_name,
+            content,
+            stream=channel_name,
+        )
     else:
-        stream = channel_name
-        topic_name = _("Message from Slack")
+        raise JsonableError(_("Error: channels_map_to_topics parameter other than 0 or 1"))
 
-    content = ZULIP_MESSAGE_TEMPLATE.format(message_sender=user_name, text=text)
-    client = RequestNotes.get_notes(request).client
-    assert client is not None
-    check_send_stream_message(user_profile, client, stream, topic_name, content)
     return json_success(request)


Test patch:
diff --git a/zerver/webhooks/slack/tests.py b/zerver/webhooks/slack/tests.py
--- a/zerver/webhooks/slack/tests.py
+++ b/zerver/webhooks/slack/tests.py
@@ -2,31 +2,71 @@
 
 from zerver.lib.test_classes import WebhookTestCase
 
+EXPECTED_TOPIC = "Message from Slack"
+EXPECTED_MESSAGE = "**slack_user**: test"
+
 
 class SlackWebhookTests(WebhookTestCase):
     STREAM_NAME = "slack"
     URL_TEMPLATE = "/api/v1/external/slack?stream={stream}&api_key={api_key}"
     WEBHOOK_DIR_NAME = "slack"
 
-    def test_slack_channel_to_topic(self) -> None:
+    def test_slack_only_stream_parameter(self) -> None:
+        self.check_webhook(
+            "message_info",
+            EXPECTED_TOPIC,
+            EXPECTED_MESSAGE,
+            content_type="application/x-www-form-urlencoded",
+        )
+
+    def test_slack_with_user_specified_topic(self) -> None:
+        self.url = self.build_webhook_url(topic="test")
+        expected_topic_name = "test"
+        self.check_webhook(
+            "message_info",
+            expected_topic_name,
+            EXPECTED_MESSAGE,
+            content_type="application/x-www-form-urlencoded",
+        )
+
+    def test_slack_channels_map_to_topics_true(self) -> None:
+        self.url = self.build_webhook_url(channels_map_to_topics="1")
         expected_topic_name = "channel: general"
-        expected_message = "**slack_user**: test"
         self.check_webhook(
             "message_info",
             expected_topic_name,
-            expected_message,
+            EXPECTED_MESSAGE,
+            content_type="application/x-www-form-urlencoded",
+        )
+
+    def test_slack_channels_map_to_topics_true_and_user_specified_topic(self) -> None:
+        self.url = self.build_webhook_url(topic="test", channels_map_to_topics="1")
+        expected_topic_name = "test"
+        self.check_webhook(
+            "message_info",
+            expected_topic_name,
+            EXPECTED_MESSAGE,
+            content_type="application/x-www-form-urlencoded",
+        )
+
+    def test_slack_channels_map_to_topics_false(self) -> None:
+        self.STREAM_NAME = "general"
+        self.url = self.build_webhook_url(channels_map_to_topics="0")
+        self.check_webhook(
+            "message_info",
+            EXPECTED_TOPIC,
+            EXPECTED_MESSAGE,
             content_type="application/x-www-form-urlencoded",
         )
 
-    def test_slack_channel_to_stream(self) -> None:
+    def test_slack_channels_map_to_topics_false_and_user_specified_topic(self) -> None:
         self.STREAM_NAME = "general"
-        self.url = "{}{}".format(self.url, "&channels_map_to_topics=0")
-        expected_topic_name = "Message from Slack"
-        expected_message = "**slack_user**: test"
+        self.url = self.build_webhook_url(topic="test", channels_map_to_topics="0")
+        expected_topic_name = "test"
         self.check_webhook(
             "message_info",
             expected_topic_name,
-            expected_message,
+            EXPECTED_MESSAGE,
             content_type="application/x-www-form-urlencoded",
         )
 
@@ -50,7 +90,7 @@ def test_missing_data_text(self) -> None:
 
     def test_invalid_channels_map_to_topics(self) -> None:
         payload = self.get_body("message_info")
-        url = "{}{}".format(self.url, "&channels_map_to_topics=abc")
+        url = self.build_webhook_url(channels_map_to_topics="abc")
         result = self.client_post(url, payload, content_type="application/x-www-form-urlencoded")
         self.assert_json_error(result, "Error: channels_map_to_topics parameter other than 0 or 1")