Repository: zulip/zulip

Issue:
Pivotal integration exception
Hi, 
I've added Pivotal integration and from time to time I receive those two e-mails when working in Pivotal:
I'm running ubuntu 20.04

If you need more information, I'd be happy to help.

```
Logger django.request, from module django.utils.log line 224:
Error generated by PivotalMessenger <pivotal-bot@***> (Member) on *** deployment

No stack trace available

Deployed code:
- git: None
- ZULIP_VERSION: 4.2


Request info:
- path: /api/v1/external/pivotal
- POST: {}
- REMOTE_ADDR: "35.184.18.147"
- QUERY_STRING: "api_key=******&stream=******&topic=******"
- SERVER_NAME: ""
```

```
Logger zerver.middleware.json_error_handler, from module zerver.middleware line 450:
Error generated by PivotalMessenger <pivotal-bot@***> (Member) on *** deployment

Traceback (most recent call last):
  File "/usr/lib/python3.8/xml/etree/ElementTree.py", line 1693, in feed
    self.parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./zerver/webhooks/pivotal/view.py", line 172, in api_pivotal_webhook
    subject, content = api_pivotal_webhook_v3(request, user_profile)
  File "./zerver/webhooks/pivotal/view.py", line 19, in api_pivotal_webhook_v3
    payload = xml_fromstring(request.body)
  File "/srv/zulip-venv-cache/9d0f5ac272f4e644b222ed65b0b5a996616a215f/zulip-py3-venv/lib/python3.8/site-packages/defusedxml/common.py", line 131, in fromstring
    parser.feed(text)
  File "/usr/lib/python3.8/xml/etree/ElementTree.py", line 1695, in feed
    self._raiseerror(v)
  File "/usr/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
    raise err
  File "<string>", line None
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/zulip-venv-cache/9d0f5ac272f4e644b222ed65b0b5a996616a215f/zulip-py3-venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/srv/zulip-venv-cache/9d0f5ac272f4e644b222ed65b0b5a996616a215f/zulip-py3-venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "./zerver/lib/request.py", line 390, in _wrapped_view_func
    return view_func(request, *args, **kwargs)
  File "./zerver/decorator.py", line 354, in _wrapped_func_arguments
    raise err
  File "./zerver/decorator.py", line 334, in _wrapped_func_arguments
    return view_func(request, user_profile, *args, **kwargs)
  File "./zerver/lib/request.py", line 390, in _wrapped_view_func
    return view_func(request, *args, **kwargs)
  File "./zerver/webhooks/pivotal/view.py", line 175, in api_pivotal_webhook
    subject, content = api_pivotal_webhook_v5(request, user_profile)
  File "./zerver/webhooks/pivotal/view.py", line 87, in api_pivotal_webhook_v5
    story_url = primary_resources["url"]
KeyError: 'url'


Deployed code:
- git: None
- ZULIP_VERSION: 4.2


Request info:
- path: /api/v1/external/pivotal
- POST: {}
- REMOTE_ADDR: "35.184.18.147"
- QUERY_STRING: "api_key=******&stream=******&topic=******"
- SERVER_NAME: ""


```


Discussion:
Thanks for the report @vrozkovec!  Error payloads from webhooks should be logged to `/var/log/zulip/webhooks_errors.log`; can you capture that to figure out what event type is causing the exception?  

(Sharing the whole payload, potentially after anonymizing any data with `example.com` type values, is ideal for being able to fix it quickly, but even just knowing the event type makes it a lot easier to reproduce)
Hello @zulip/server-integrations members, this issue was labeled with the "area: integrations" label, so you may want to check it out!

<!-- areaLabelAddition -->
Hi @timabbott, here are few last payloads: 
[log.txt](https://github.com/zulip/zulip/files/6536711/log.txt)

@vrozkovec thanks!  That should make it possible for someone to investigate and fix this.
@zulipbot claim
@vrozkovec are you wanting to alert on this event or not? As we have a list of a bunch of unsupported events that we just ignore, we can just add to this if you aren't specifically wanting to alert on this event.
As the content of that payload seems pretty useless to be fair.

Resolving patch:
diff --git a/zerver/webhooks/pivotal/view.py b/zerver/webhooks/pivotal/view.py
--- a/zerver/webhooks/pivotal/view.py
+++ b/zerver/webhooks/pivotal/view.py
@@ -72,6 +72,7 @@ def get_text(attrs: List[str]) -> str:
     "story_delete_activity",
     "story_move_into_project_activity",
     "epic_update_activity",
+    "label_create_activity",
 ]
 
 


Test patch: