Repository: zulip/zulip

Issue:
Create incoming webhook integration for Open Collective
Create an [incoming webhook](https://zulip.com/api/incoming-webhooks-overview) integration for Open Collective. The main use case is getting notifications when new financial contributors sign up.

Open Collective integrations documentation: https://docs.opencollective.com/help/collectives/integrations

Now that [Zulip has an Open Collective page](https://opencollective.com/zulip), it would be great to be able to monitor new sign-ups!


Discussion:
Hello @zulip/server-integrations members, this issue was labeled with the "area: integrations" label, so you may want to check it out!

<!-- areaLabelAddition -->
Hello @alya, you claimed this issue to work on it, but this issue and any referenced pull requests haven't been updated for 10 days. Are you still working on this issue?

If so, please update this issue by leaving a comment on this issue to let me know that you're still working on it. Otherwise, I'll automatically remove you from this issue in 4 days.

If you've decided to work on something else, simply comment `@zulipbot abandon` so that someone else can claim it and continue from where you left off.

Thank you for your valuable contributions to Zulip!

<!-- inactiveWarning -->
I made a simple JSON integration that pulls sample data. @adambirds handing it over to you. :)
Mind if I take a stab at this? I am fine if I am not the assignee and my work becomes redundant, I just see it as a good task for me to make my first contribution.
@zulipbot claim
Welcome to Zulip, @snailgirl2612! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:
* Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
* [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!
@alya 
I'm trying to create a fixture for this.
How can I access the JSON integration that you've made? 

@zulipbot claim

Welcome to Zulip, @thanasisath31! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:
* Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
* [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!
I'm not planning on working on this further, so happy to have someone else pick it up! CC @adambirds who may have been looking into it as well.

Here are sample payloads from the json integration.

A one-time $1 donation I made to get sample data triggered both of these:
```
{
  "createdAt": "2021-04-30T19:04:52.962Z",
  "id": 1121190,
  "type": "collective.transaction.created",
  "CollectiveId": 30312,
  "data": {
    "transaction": {
      "amount": 100,
      "currency": "USD",
      "formattedAmount": "$1.00",
      "formattedAmountWithInterval": "$1.00"
    },
    "fromCollective": {
      "id": 251827,
      "type": "USER",
      "slug": "alya-abbott",
      "name": "Alya Abbott",
      "twitterHandle": null,
      "githubHandle": null,
      "image": "https://opencollective-production.s3.us-west-1.amazonaws.com/ba943750-9722-11eb-8df5-4deb989f8775.jpeg"
    }
  }
}

{
  "createdAt": "2021-04-30T19:04:52.991Z",
  "id": 1121191,
  "type": "collective.member.created",
  "CollectiveId": 30312,
  "data": {
    "member": {
      "role": "BACKER",
      "description": null,
      "since": "2021-04-30T19:04:52.976Z",
      "memberCollective": {
        "id": 251827,
        "type": "USER",
        "slug": "alya-abbott",
        "name": "Alya Abbott",
        "company": null,
        "website": null,
        "twitterHandle": null,
        "githubHandle": null,
        "description": null,
        "previewImage": "https://res.cloudinary.com/opencollective/image/fetch/c_thumb,g_face,h_48,r_max,w_48,bo_3px_solid_white/c_thumb,h_48,r_max,w_48,bo_2px_solid_rgb:66C71A/e_trim/f_jpg/https%3A%2F%2Fopencollective-production.s3.us-west-1.amazonaws.com%2Fba943750-9722-11eb-8df5-4deb989f8775.jpeg"
      }
    },
    "order": {
      "id": 168627,
      "totalAmount": 100,
      "currency": "USD",
      "description": "Financial contribution to Zulip",
      "interval": null,
      "createdAt": "2021-04-30T19:04:47.348Z",
      "quantity": 1,
      "formattedAmount": "$1.00",
      "formattedAmountWithInterval": "$1.00"
    }
  }
}
```

An incognito monthly donation I created looks like this (also 2 events):
```
{
  "createdAt": "2021-04-30T19:20:04.346Z",
  "id": 1121199,
  "type": "collective.member.created",
  "CollectiveId": 30312,
  "data": {
    "member": {
      "role": "BACKER",
      "description": null,
      "since": "2021-04-30T19:20:04.314Z",
      "memberCollective": {
        "type": "USER",
        "name": "Incognito",
        "previewImage": null
      }
    },
    "order": {
      "id": 168629,
      "totalAmount": 100,
      "currency": "USD",
      "description": "Monthly financial contribution to Zulip",
      "interval": "month",
      "createdAt": "2021-04-30T19:19:58.647Z",
      "quantity": 1,
      "formattedAmount": "$1.00",
      "formattedAmountWithInterval": "$1.00 / month"
    }
  }
}

{
  "createdAt": "2021-04-30T19:20:04.294Z",
  "id": 1121198,
  "type": "collective.transaction.created",
  "CollectiveId": 30312,
  "data": {
    "transaction": {
      "amount": 100,
      "currency": "USD",
      "formattedAmount": "$1.00",
      "formattedAmountWithInterval": "$1.00"
    },
    "fromCollective": {
      "id": 262738,
      "type": "USER",
      "slug": "incognito-b4a9278e",
      "name": "Incognito",
      "twitterHandle": null,
      "githubHandle": null
    }
  }
}
```

@zulipbot unclaim
@zulipbot claim

@zulipbot claim
@zulipbot unclaim

@zulipbot  claim 

@zulipbot claim

@timabbott  @alya 

Hello, i have made an integration using only one fixture which was generated by an other Opencollective account i made in order to get sample fixtures. 
When i made a one dollar one time donation and by setting the OpenCollective webhook trigger on new Members i got this response (basically is event 2 alya got my making the same type of donation)
 `{
  "createdAt": "2021-06-21T17:25:24.344Z",
  "id": 1242511,
  "type": "collective.member.created",
  "CollectiveId": 281290,
  "data": {
    "member": {
      "role": "BACKER",
      "description": null,
      "since": "2021-06-21T17:25:24.332Z",
      "memberCollective": {
        "id": 280138,
        "type": "USER",
        "slug": "leyteris-kyriazanos",
        "name": "Λευτέρης Κυριαζάνος",
        "company": null,
        "website": null,
        "twitterHandle": null,
        "githubHandle": null,
        "description": null,
        "previewImage": null
      }
    },
    "order": {
      "id": 183773,
      "totalAmount": 100,
      "currency": "EUR",
      "description": "Financial contribution to Test-Webhooks",
      "interval": null,
      "createdAt": "2021-06-21T17:25:17.968Z",
      "quantity": 1,
      "formattedAmount": "€1.00",
      "formattedAmountWithInterval": "€1.00"
    }
  }
}`

i have made a simple integration that looks like this. 

![Screenshot 2021-06-23 at 9 27 10 PM](https://user-images.githubusercontent.com/43907486/123150555-29a1f080-d46b-11eb-8357-1a0100cab44d.jpg)

Problem is that i don't have all the possible generated fixtures that can be created by openCollective when a new member registers. For example i don't have the fixture created when someone becomes a backer or sponsor. And i can't seem to find templates anywhere. Any advice on this?

I think you should submit an integration that ignores all events other than the ones we have fixtures for above.  At least 90% of the value is in supporting just that event type.  It's unclear to me whether it's worth supporting other event types at all, but we can do that audit once we have a basic integration merged.
@timabbott Ok thank you, that's clear! 

Are the below considered 4 different fixtures? When i made a donation only one event was triggered (not 2). 
Can the template vary regarding the same type of donation or there can be both at the same time? The 2 events triggered 
confused me to be honest. 

> I'm not planning on working on this further, so happy to have someone else pick it up! CC @adambirds who may have been looking into it as well.
> 
> Here are sample payloads from the json integration.
> 
> A one-time $1 donation I made to get sample data triggered both of these:
> 
> ```
> {
>   "createdAt": "2021-04-30T19:04:52.962Z",
>   "id": 1121190,
>   "type": "collective.transaction.created",
>   "CollectiveId": 30312,
>   "data": {
>     "transaction": {
>       "amount": 100,
>       "currency": "USD",
>       "formattedAmount": "$1.00",
>       "formattedAmountWithInterval": "$1.00"
>     },
>     "fromCollective": {
>       "id": 251827,
>       "type": "USER",
>       "slug": "alya-abbott",
>       "name": "Alya Abbott",
>       "twitterHandle": null,
>       "githubHandle": null,
>       "image": "https://opencollective-production.s3.us-west-1.amazonaws.com/ba943750-9722-11eb-8df5-4deb989f8775.jpeg"
>     }
>   }
> }
> 
> {
>   "createdAt": "2021-04-30T19:04:52.991Z",
>   "id": 1121191,
>   "type": "collective.member.created",
>   "CollectiveId": 30312,
>   "data": {
>     "member": {
>       "role": "BACKER",
>       "description": null,
>       "since": "2021-04-30T19:04:52.976Z",
>       "memberCollective": {
>         "id": 251827,
>         "type": "USER",
>         "slug": "alya-abbott",
>         "name": "Alya Abbott",
>         "company": null,
>         "website": null,
>         "twitterHandle": null,
>         "githubHandle": null,
>         "description": null,
>         "previewImage": "https://res.cloudinary.com/opencollective/image/fetch/c_thumb,g_face,h_48,r_max,w_48,bo_3px_solid_white/c_thumb,h_48,r_max,w_48,bo_2px_solid_rgb:66C71A/e_trim/f_jpg/https%3A%2F%2Fopencollective-production.s3.us-west-1.amazonaws.com%2Fba943750-9722-11eb-8df5-4deb989f8775.jpeg"
>       }
>     },
>     "order": {
>       "id": 168627,
>       "totalAmount": 100,
>       "currency": "USD",
>       "description": "Financial contribution to Zulip",
>       "interval": null,
>       "createdAt": "2021-04-30T19:04:47.348Z",
>       "quantity": 1,
>       "formattedAmount": "$1.00",
>       "formattedAmountWithInterval": "$1.00"
>     }
>   }
> }
> ```
> 
> An incognito monthly donation I created looks like this (also 2 events):
> 
> ```
> {
>   "createdAt": "2021-04-30T19:20:04.346Z",
>   "id": 1121199,
>   "type": "collective.member.created",
>   "CollectiveId": 30312,
>   "data": {
>     "member": {
>       "role": "BACKER",
>       "description": null,
>       "since": "2021-04-30T19:20:04.314Z",
>       "memberCollective": {
>         "type": "USER",
>         "name": "Incognito",
>         "previewImage": null
>       }
>     },
>     "order": {
>       "id": 168629,
>       "totalAmount": 100,
>       "currency": "USD",
>       "description": "Monthly financial contribution to Zulip",
>       "interval": "month",
>       "createdAt": "2021-04-30T19:19:58.647Z",
>       "quantity": 1,
>       "formattedAmount": "$1.00",
>       "formattedAmountWithInterval": "$1.00 / month"
>     }
>   }
> }
> 
> {
>   "createdAt": "2021-04-30T19:20:04.294Z",
>   "id": 1121198,
>   "type": "collective.transaction.created",
>   "CollectiveId": 30312,
>   "data": {
>     "transaction": {
>       "amount": 100,
>       "currency": "USD",
>       "formattedAmount": "$1.00",
>       "formattedAmountWithInterval": "$1.00"
>     },
>     "fromCollective": {
>       "id": 262738,
>       "type": "USER",
>       "slug": "incognito-b4a9278e",
>       "name": "Incognito",
>       "twitterHandle": null,
>       "githubHandle": null
>     }
>   }
> }
> ```


I suspect the difference is that you get both events for a new person, but if an existing person does a new donation, you only et the `transaction.created` event.  I would consider just implementing this:
* `collective.transaction.created` should send a notification about the new transaction including the financial details. (aka line 2 of your screenshot, but with the name mentioned).

(I'm not sure there's anything the Zulip integration needs to do with "new member").
@timabbott Open Collective has multiple options as to when to trigger the webhook. 
![Screenshot 2021-06-24 at 6 42 03 PM](https://user-images.githubusercontent.com/43907486/123292652-ee132f00-d51b-11eb-8bea-ca3768476cf1.jpg)

I suppose then i should change "New Member" to "New Transaction" and get a basic fixture for that


Resolving patch:
diff --git a/zerver/lib/integrations.py b/zerver/lib/integrations.py
--- a/zerver/lib/integrations.py
+++ b/zerver/lib/integrations.py
@@ -423,6 +423,9 @@ def __init__(self, name: str, *args: Any, **kwargs: Any) -> None:
         stream_name="opbeat",
         function="zerver.webhooks.opbeat.view.api_opbeat_webhook",
     ),
+    WebhookIntegration(
+        "opencollective", ["communication"], display_name="Open Collective Incoming Webhook"
+    ),
     WebhookIntegration("opsgenie", ["meta-integration", "monitoring"]),
     WebhookIntegration("pagerduty", ["monitoring"], display_name="PagerDuty"),
     WebhookIntegration("papertrail", ["monitoring"]),
@@ -761,6 +764,7 @@ def __init__(self, name: str, *args: Any, **kwargs: Any) -> None:
         ScreenshotConfig("incident_closed.json", "003.png"),
     ],
     "opbeat": [ScreenshotConfig("error_reopen.json")],
+    "opencollective": [ScreenshotConfig("one_time_donation.json")],
     "opsgenie": [ScreenshotConfig("addrecipient.json", image_name="000.png")],
     "pagerduty": [ScreenshotConfig("trigger_v2.json")],
     "papertrail": [ScreenshotConfig("short_post.json", payload_as_query_param=True)],
diff --git a/zerver/webhooks/opencollective/__init__.py b/zerver/webhooks/opencollective/__init__.py
new file mode 100644
diff --git a/zerver/webhooks/opencollective/view.py b/zerver/webhooks/opencollective/view.py
new file mode 100644
--- /dev/null
+++ b/zerver/webhooks/opencollective/view.py
@@ -0,0 +1,47 @@
+from typing import Any, Dict
+
+from django.http import HttpRequest, HttpResponse
+
+from zerver.decorator import webhook_view
+from zerver.lib.request import REQ, has_request_variables
+from zerver.lib.response import json_success
+from zerver.lib.webhooks.common import check_send_webhook_message
+from zerver.models import UserProfile
+
+MEMBER_NAME_TEMPLATE = "{name}"
+AMOUNT_TEMPLATE = "{amount}"
+
+
+@webhook_view("OpenCollective")
+@has_request_variables
+def api_opencollective_webhook(
+    request: HttpRequest,
+    user_profile: UserProfile,
+    payload: Dict[str, Any] = REQ(argument_type="body"),
+) -> HttpResponse:
+
+    name = get_name(payload)
+    amount = get_amount(payload)
+
+    # construct the body of the message
+    body = ""
+
+    if name == "Incognito":  # Incognito donation
+        body = f"An **Incognito** member donated **{amount}**! :tada:"
+    else:  # non - Incognito donation
+        body = f"@_**{name}** donated **{amount}**! :tada:"
+
+    topic = "New Member"
+
+    # send the message
+    check_send_webhook_message(request, user_profile, topic, body)
+
+    return json_success()
+
+
+def get_name(payload: Dict[str, Any]) -> str:
+    return MEMBER_NAME_TEMPLATE.format(name=payload["data"]["member"]["memberCollective"]["name"])
+
+
+def get_amount(payload: Dict[str, Any]) -> str:
+    return AMOUNT_TEMPLATE.format(amount=payload["data"]["order"]["formattedAmount"])


Test patch:
diff --git a/zerver/webhooks/opencollective/tests.py b/zerver/webhooks/opencollective/tests.py
new file mode 100644
--- /dev/null
+++ b/zerver/webhooks/opencollective/tests.py
@@ -0,0 +1,31 @@
+from zerver.lib.test_classes import WebhookTestCase
+
+
+class OpenCollectiveHookTests(WebhookTestCase):
+    STREAM_NAME = "test"
+    URL_TEMPLATE = "/api/v1/external/opencollective?&api_key={api_key}&stream={stream}"
+    WEBHOOK_DIR_NAME = "opencollective"
+
+    # Note: Include a test function per each distinct message condition your integration supports
+    def test_one_time_donation(self) -> None:  # test one time donation
+        expected_topic = "New Member"
+        expected_message = "@_**Λευτέρης Κυριαζάνος** donated **€1.00**! :tada:"
+
+        self.check_webhook(
+            "one_time_donation",
+            expected_topic,
+            expected_message,
+            content_type="application/x-www-form-urlencoded",
+        )
+
+    def test_one_time_incognito_donation(self) -> None:  # test one time incognito donation
+        expected_topic = "New Member"
+        expected_message = "An **Incognito** member donated **$1.00**! :tada:"
+
+        # use fixture named helloworld_hello
+        self.check_webhook(
+            "one_time_incognito_donation",
+            expected_topic,
+            expected_message,
+            content_type="application/x-www-form-urlencoded",
+        )