Repository: zulip/zulip

Issue:
Create incoming webhook integration for Patreon
Create an [incoming webhook](https://zulip.com/api/incoming-webhooks-overview) integration for Patreon. The main use case is getting notifications when new patrons sign up.

Patreon webhook documentation: https://docs.patreon.com/#webhooks

Now that [Zulip has a Patreon page](https://www.patreon.com/zulip), it would be great to be able to monitor new sign-ups!


Discussion:
Hello @zulip/server-integrations members, this issue was labeled with the "area: integrations" label, so you may want to check it out!

<!-- areaLabelAddition -->
i would be willing to work on this

would this feature be used for internal use?
@Sick-codes It will be used as an integration on all Zulip services.
@zulipbot claim

@drrosa You have been unassigned from this issue because you have not made any updates for over 14 days. Please feel free to reclaim the issue if you decide to pick up again. Thanks!

you can use this [link](https://www.patreon.com/portal/registration/register-webhooks) to create webhook on patreon when you have created your account.

(someone who has a patreon account should handle the issue. )
You will need to `@zulipbot abandon` so somebody else can claim it. @ritikchauhan-01 
@zulipbot abandon
I have been following the repository for some time now, and feel like this would be a good first issue. Would it be possible for me to work on this without properly claiming it? I wouldn't want to discourage anyone else from fixing it if it could be done faster. I would just like to throw my hat in the ring.
I tried setting up a basic [JSON integration](https://zulip.com/integrations/doc/json) to get sample data for this issue, and it seems that the Patreon outgoing webhook setup has a problem. In particular, it convert the URL to all-caps, which breaks it for us.

I think the next step is probably to contact Patreon support about this, which I can do and post an update here once I hear back.
Support request submitted.

In the meantime, I sent sample data from all the non-deprecated event types here: https://webhook.site/#!/a6ed673f-a5c6-4c64-b292-13c71d0523f0/8f5f510b-a1f2-4145-a44b-037ceb15b2c3/1
@zulipbot claim
Welcome to Zulip, @sudipto! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:
* Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
* [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!
I have created the webhook integration for Patreon using the webhook.site data posted here by @alya .
Please check it once here: https://imgur.com/a/oHVDP8V
I am open to suggestions if any. Also I haven't tested this with Patreon website. I don't have a patreon so if anyone can help me test that out.
@sudipto have you created a PR for this? 
@adambirds not yet. I am new to opensource so a little hesitant in creating the PR.
@sudipto in case you haven't found it yet, https://zulip.readthedocs.io/en/latest/git/using.html is a helpful reference.
Patreon's support response on the URL capitalization issue (after a clarification):

> I understand your report here and will share it with our engineers. 

Hello, I'm new to open source and i'm interested in solving this issue. Is this issue still open?

@prah23 please let us know if you plan on finishing this so if not @akshatakulkarni25 can pick the work up instead.
@adambirds @akshatakulkarni25 this is blocked on clarification from Patreon. The current actual webhook payload's structure is not reflected on the test webhook payloads they provide. I would be updating #22561 as clarification is provided.

CZO discussion: https://chat.zulip.org/#narrow/stream/127-integrations/topic/Patreon.20Integration.
Hello @alya , can I contribute to this issue, Is it open for me to work upon
@zulipbot claim
Welcome to Zulip, @komala777! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

- Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
- [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

Is this issue still open ?
If an issue is open, its still open.
@zulipbot claim
Welcome to Zulip, @sachinsingh3107! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

- Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
- [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

@zulipbot abandon
@zulipbot claim
**ERROR:** Unexpected response from GitHub API.
@zulipbot claim
**ERROR:** Unexpected response from GitHub API.
I would like to work on this issue

On Wed, Sep 27, 2023 at 8:41 PM zulipbot ***@***.***> wrote:

> *ERROR:* Unexpected response from GitHub API.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/zulip/zulip/issues/18321#issuecomment-1737974053>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AIFAQETJ6CYCOWO2Y36ETWDX4R6NVANCNFSM4326YI7A>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>

not interested. been assigned to a different one


On Wed, Sep 27, 2023 at 8:42 PM Fahd Agodzo ***@***.***> wrote:

> I would like to work on this issue
>
> On Wed, Sep 27, 2023 at 8:41 PM zulipbot ***@***.***> wrote:
>
>> *ERROR:* Unexpected response from GitHub API.
>>
>> —
>> Reply to this email directly, view it on GitHub
>> <https://github.com/zulip/zulip/issues/18321#issuecomment-1737974053>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/AIFAQETJ6CYCOWO2Y36ETWDX4R6NVANCNFSM4326YI7A>
>> .
>> You are receiving this because you commented.Message ID:
>> ***@***.***>
>>
>

@zulipbot claim
@zulipbot claim

Resolving patch:
diff --git a/zerver/lib/integrations.py b/zerver/lib/integrations.py
--- a/zerver/lib/integrations.py
+++ b/zerver/lib/integrations.py
@@ -441,6 +441,7 @@ def __init__(self, name: str, *args: Any, **kwargs: Any) -> None:
     WebhookIntegration("opsgenie", ["meta-integration", "monitoring"]),
     WebhookIntegration("pagerduty", ["monitoring"], display_name="PagerDuty"),
     WebhookIntegration("papertrail", ["monitoring"]),
+    WebhookIntegration("patreon", ["financial"], display_name="Patreon"),
     WebhookIntegration("pingdom", ["monitoring"]),
     WebhookIntegration("pivotal", ["project-management"], display_name="Pivotal Tracker"),
     WebhookIntegration("radarr", ["entertainment"], display_name="Radarr"),
@@ -777,6 +778,7 @@ def __init__(self, name: str, *args: Any, **kwargs: Any) -> None:
     "opsgenie": [ScreenshotConfig("addrecipient.json", image_name="000.png")],
     "pagerduty": [ScreenshotConfig("trigger_v2.json")],
     "papertrail": [ScreenshotConfig("short_post.json", payload_as_query_param=True)],
+    "patreon": [ScreenshotConfig("members_pledge_create.json")],
     "pingdom": [ScreenshotConfig("http_up_to_down.json", image_name="002.png")],
     "pivotal": [ScreenshotConfig("v5_type_changed.json")],
     "radarr": [ScreenshotConfig("radarr_movie_grabbed.json")],
diff --git a/zerver/webhooks/patreon/__init__.py b/zerver/webhooks/patreon/__init__.py
new file mode 100644
diff --git a/zerver/webhooks/patreon/view.py b/zerver/webhooks/patreon/view.py
new file mode 100644
--- /dev/null
+++ b/zerver/webhooks/patreon/view.py
@@ -0,0 +1,190 @@
+from typing import Callable, Dict, Optional
+
+from django.http import HttpRequest, HttpResponse
+
+from zerver.decorator import webhook_view
+from zerver.lib.exceptions import UnsupportedWebhookEventTypeError
+from zerver.lib.response import json_success
+from zerver.lib.typed_endpoint import JsonBodyPayload, typed_endpoint
+from zerver.lib.validator import WildValue, check_int, check_none_or, check_string
+from zerver.lib.webhooks.common import (
+    check_send_webhook_message,
+    validate_extract_webhook_http_header,
+)
+from zerver.models import UserProfile
+
+
+# The events for this integration contain the ":" character, which is not appropriate in a
+# filename and requires us to deviate from the common `get_http_headers_from_filename` method
+# from zerver.lib.webhooks.common.
+def get_custom_http_headers_from_filename(http_header_key: str) -> Callable[[str], Dict[str, str]]:
+    def fixture_to_headers(filename: str) -> Dict[str, str]:
+        event_type = filename.replace("_", ":")
+        return {http_header_key: event_type}
+
+    return fixture_to_headers
+
+
+fixture_to_headers = get_custom_http_headers_from_filename("HTTP_X_PATREON_EVENT")
+
+
+def get_members_create_body(payload: WildValue) -> Optional[str]:
+    last_charge_status = get_last_charge_status(payload)
+    patron_status = get_patron_status(payload)
+    # null values indicate the member has never pledged
+    if last_charge_status is None and patron_status is None:
+        template = "{user_name} has joined as a member!"
+        return template.format(
+            user_name=get_user_name(payload),
+        ).rstrip()
+    return None
+
+
+def get_members_update_body(payload: WildValue) -> Optional[str]:
+    last_charge_status = get_last_charge_status(payload)
+    patron_status = get_patron_status(payload)
+    if last_charge_status in ("Paid", None) and patron_status in ("active_patron", "former_patron"):
+        template = "{user_name}'s membership has been updated to {patron_status}."
+        return template.format(
+            user_name=get_user_name(payload),
+            patron_status=str(patron_status).replace("_", " "),
+        ).rstrip()
+    return None
+
+
+def get_members_delete_body(payload: WildValue) -> Optional[str]:
+    last_charge_status = get_last_charge_status(payload)
+    patron_status = get_patron_status(payload)
+    # null value indicates the member has never pledged
+    if last_charge_status in ("Paid", None) and patron_status != "declined_patron":
+        template = "{user_name}'s membership has ended."
+        return template.format(
+            user_name=get_user_name(payload),
+        ).rstrip()
+    return None
+
+
+def get_members_pledge_create_body(payload: WildValue) -> Optional[str]:
+    last_charge_status = get_last_charge_status(payload)
+    pledge_amount = get_pledge_amount(payload)
+    # The only successful charge status is "Paid". null if not yet charged.
+    if last_charge_status in ("Paid", None) and pledge_amount > 0:
+        template = "{user_name} has pledged ${pledge_amount:.2f} per {pay_per_name}. :tada:\nTotal number of patrons: {patron_count}"
+        return template.format(
+            user_name=get_user_name(payload),
+            pledge_amount=pledge_amount,
+            pay_per_name=get_pay_per_name(payload),
+            patron_count=get_patron_count(payload),
+        ).rstrip()
+    return None
+
+
+def get_members_pledge_update_body(payload: WildValue) -> Optional[str]:
+    last_charge_status = get_last_charge_status(payload)
+    pledge_amount = get_pledge_amount(payload)
+    # The only successful charge status is "Paid". null if not yet charged.
+    if last_charge_status in ("Paid", None) and pledge_amount > 0:
+        template = "{user_name} has updated their pledge to ${pledge_amount:.2f} per {pay_per_name}. :gear:"
+        return template.format(
+            user_name=get_user_name(payload),
+            pledge_amount=pledge_amount,
+            pay_per_name=get_pay_per_name(payload),
+        ).rstrip()
+    return None
+
+
+def get_members_pledge_delete_body(payload: WildValue) -> Optional[str]:
+    last_charge_status = get_last_charge_status(payload)
+    if last_charge_status in ("Paid", "Deleted", None):
+        template = "{user_name}'s pledge has been cancelled. :cross_mark:\nTotal number of patrons: {patron_count}"
+        return template.format(
+            user_name=get_user_name(payload),
+            patron_count=get_patron_count(payload),
+        ).rstrip()
+    return None
+
+
+def get_last_charge_status(payload: WildValue) -> Optional[str]:
+    return payload["data"]["attributes"]["last_charge_status"].tame(check_none_or(check_string))
+
+
+def get_patron_status(payload: WildValue) -> Optional[str]:
+    return payload["data"]["attributes"]["patron_status"].tame(check_none_or(check_string))
+
+
+def get_user_name(payload: WildValue) -> str:
+    return payload["data"]["attributes"]["full_name"].tame(check_string)
+
+
+def get_pledge_amount(payload: WildValue) -> float:
+    return payload["data"]["attributes"]["currently_entitled_amount_cents"].tame(check_int) / 100
+
+
+def get_patron_count(payload: WildValue) -> int:
+    return payload["included"][0]["attributes"]["patron_count"].tame(check_int)
+
+
+def get_pay_per_name(payload: WildValue) -> str:
+    return payload["included"][0]["attributes"]["pay_per_name"].tame(check_string)
+
+
+EVENT_FUNCTION_MAPPER: Dict[str, Callable[[WildValue], Optional[str]]] = {
+    "members:create": get_members_create_body,
+    "members:update": get_members_update_body,
+    "members:delete": get_members_delete_body,
+    "members:pledge:create": get_members_pledge_create_body,
+    "members:pledge:update": get_members_pledge_update_body,
+    "members:pledge:delete": get_members_pledge_delete_body,
+}
+
+# deprecated events
+IGNORED_EVENTS = [
+    "pledges:create",
+    "pledges:update",
+    "pledges:delete",
+]
+
+ALL_EVENT_TYPES = list(EVENT_FUNCTION_MAPPER.keys())
+
+
+@webhook_view("Patreon", all_event_types=ALL_EVENT_TYPES)
+@typed_endpoint
+def api_patreon_webhook(
+    request: HttpRequest,
+    user_profile: UserProfile,
+    *,
+    payload: JsonBodyPayload[WildValue],
+) -> HttpResponse:
+    header_event = validate_extract_webhook_http_header(request, "X-Patreon-Event", "Patreon")
+
+    event_name = get_zulip_event_name(header_event, payload)
+    if event_name is None:
+        # See IGNORED_EVENTS.
+        return json_success(request)
+    topic = "membership notifications"
+
+    body_function = EVENT_FUNCTION_MAPPER[event_name]
+    body = body_function(payload)
+
+    if body is None:
+        # None for payloads that are valid,
+        # but where we intentionally do not send a message.
+        return json_success(request)
+
+    check_send_webhook_message(request, user_profile, topic, body, event_name)
+    return json_success(request)
+
+
+def get_zulip_event_name(
+    header_event: str,
+    payload: WildValue,
+) -> Optional[str]:
+    """
+    Usually, we return an event name that is a key in EVENT_FUNCTION_MAPPER.
+    We return None for an event that we know we don't want to handle.
+    """
+    if header_event in EVENT_FUNCTION_MAPPER:
+        return header_event
+    elif header_event in IGNORED_EVENTS:
+        return None
+    raise UnsupportedWebhookEventTypeError(header_event)


Test patch:
diff --git a/zerver/webhooks/patreon/tests.py b/zerver/webhooks/patreon/tests.py
new file mode 100644
--- /dev/null
+++ b/zerver/webhooks/patreon/tests.py
@@ -0,0 +1,105 @@
+from unittest.mock import patch
+
+import orjson
+
+from zerver.lib.test_classes import WebhookTestCase
+
+EXPECTED_TOPIC = "membership notifications"
+IGNORED_EVENTS = [
+    "pledges:create",
+    "pledges:update",
+    "pledges:delete",
+]
+
+
+class PatreonHookTests(WebhookTestCase):
+    STREAM_NAME = "Patreon"
+    URL_TEMPLATE = "/api/v1/external/patreon?&api_key={api_key}&stream={stream}"
+    WEBHOOK_DIR_NAME = "patreon"
+
+    def test_patreon_members_create(self) -> None:
+        expected_message = "Kopi has joined as a member!"
+        self.check_webhook(
+            "members_create",
+            EXPECTED_TOPIC,
+            expected_message,
+        )
+
+    def test_patreon_members_update(self) -> None:
+        expected_message = "Kopi's membership has been updated to active patron."
+        self.check_webhook(
+            "members_update",
+            EXPECTED_TOPIC,
+            expected_message,
+        )
+
+    def test_patreon_members_delete(self) -> None:
+        expected_message = "Kopi's membership has ended."
+        self.check_webhook(
+            "members_delete",
+            EXPECTED_TOPIC,
+            expected_message,
+        )
+
+    def test_patreon_members_pledge_create(self) -> None:
+        expected_message = "Kopi has pledged $5.00 per month. :tada:\nTotal number of patrons: 5"
+        self.check_webhook(
+            "members_pledge_create",
+            EXPECTED_TOPIC,
+            expected_message,
+        )
+
+    def test_patreon_members_pledge_update(self) -> None:
+        expected_message = "Kopi has updated their pledge to $10.00 per month. :gear:"
+        self.check_webhook(
+            "members_pledge_update",
+            EXPECTED_TOPIC,
+            expected_message,
+        )
+
+    def test_patreon_members_pledge_delete(self) -> None:
+        expected_message = (
+            "Kopi's pledge has been cancelled. :cross_mark:\nTotal number of patrons: 4"
+        )
+        self.check_webhook(
+            "members_pledge_delete",
+            EXPECTED_TOPIC,
+            expected_message,
+        )
+
+    def test_ignored_events(self) -> None:
+        # The payload for these events never gets looked at in the
+        # webhook itself; it only needs to be valid JSON.
+        payload = "{}"
+
+        for event in IGNORED_EVENTS:
+            self.verify_post_is_ignored(payload, event)
+
+    def test_ignored_payloads(self) -> None:
+        payload = orjson.loads(self.get_body("members_create"))
+        payload["data"]["attributes"]["last_charge_status"] = "Declined"
+        payload["data"]["attributes"]["patron_status"] = "declined_patron"
+
+        event_types = [
+            "members:create",
+            "members:update",
+            "members:delete",
+            "members:pledge:create",
+            "members:pledge:update",
+            "members:pledge:delete",
+        ]
+
+        for event in event_types:
+            self.verify_post_is_ignored(orjson.dumps(payload).decode(), event)
+
+    def verify_post_is_ignored(self, payload: str, http_x_patreon_event: str) -> None:
+        with patch("zerver.webhooks.patreon.view.check_send_webhook_message") as m:
+            result = self.client_post(
+                self.url,
+                payload,
+                HTTP_X_PATREON_EVENT=http_x_patreon_event,
+                content_type="application/json",
+            )
+        if http_x_patreon_event in IGNORED_EVENTS:
+            self.assertFalse(m.called)
+        self.assert_json_success(result)