Repository: dj-stripe/dj-stripe

Issue:
Djstripe is suppressing backtraces that would otherwise be shown by the logging framework
Example logging config:

```
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
    },
    'handlers': {
        'file': {
            'level': 'ERROR',
            'class': 'logging.FileHandler',
            'filename': os.path.join(BASE_DIR, 'logs/django.log'),
            'formatter': 'verbose',
        },
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
            'formatter': 'verbose',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file', 'mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
    },
}
```

Normally it would print an error and a backtrace, but it can't because djstripe is explicitly returning a 500 rather than raising the exception again!

I initially suggested that return line in https://github.com/dj-stripe/dj-stripe/blob/master/djstripe/views.py#L39 should be replaced with `raise trigger.exception`. Though I now realised that it's actually just a string as it's being processed in https://github.com/dj-stripe/dj-stripe/blob/master/djstripe/models/webhooks.py#L101
The correct solution would actually to remove the "return trigger.exception" from views.py, and reraise the exception in the code block I linked to in L101. Serializing it is fine, but also raise it again. Let django handle it.

Thoughts?


Discussion:
Alternatively would adding `logger.exception()` at https://github.com/dj-stripe/dj-stripe/blob/master/djstripe/models/webhooks.py#L101 do the job?
No idea... Haven't looked at that subsystem since I created the above
config. It's easy to check though, just try the above config and see if it
triggers an email with a backtrace? Maybe you don't even need the config,
but just try the debug server and check the output?

Sorry, writing from my phone. Really no idea. :/

On Tue, 19 Feb 2019, 18:38 John Carter <notifications@github.com wrote:

> Alternatively would adding logger.exception() at
> https://github.com/dj-stripe/dj-stripe/blob/master/djstripe/models/webhooks.py#L101
> do the job?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/dj-stripe/dj-stripe/issues/833#issuecomment-465256877>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAGofq54oo3Q-706R0bX_D7uIC6vI3PBks5vPES9gaJpZM4bCh6X>
> .
>

I took a look here: https://docs.djangoproject.com/en/2.1/topics/logging/
and it looks like logger.exception() will do the trick.
I don't disagree with logging these exceptions as discussed, but just checking - are you not able to see the saved exceptions in webhook admin?
I do, which is a good point for the next time I need to report an issue (instead of rerunning the hooks manually), though being able to view them doesn't solve this issue. Djstripe is messing with how Django handles exceptions and is therefore breaking the logging system that can be used (as in my case) for email reporting, but I assume also for other integrations such as Sentry.

Why are you against just raising the exception again and let Django handle that? logger.exception() may work, though it also may not, and to be honest, I don't have time to properly test it.
I don't know why it was implemented this way - one advantage is that it keeps a record of the input data with the exception, which does make it a lot easier to debug in my experience.

But I think there's no harm in also logging it at the point it occurs.
I have no problem with logging it to the model. That looks like a good idea and I'm all for it. What I'm saying is that in addition I think the event should be raised again to Django after it has been recorded so Django will just handle it however it sees fit.
Alternatively, if logger.exception() works correctly, that's also a solution, though I have no idea if it will.
I rewrote the exception handling code at some point. Before I did so, it also was swallowing exceptions. I honestly don't remember why I didn't change that behaviour.

I'm +1 on re-raising.

Resolving patch:
diff --git a/djstripe/models/webhooks.py b/djstripe/models/webhooks.py
--- a/djstripe/models/webhooks.py
+++ b/djstripe/models/webhooks.py
@@ -107,6 +107,9 @@ def from_request(cls, request):
 			webhook_processing_error.send(
 				sender=WebhookEventTrigger, exception=e, data=getattr(e, "http_body", "")
 			)
+
+			# re-raise the exception so Django sees it
+			raise e
 		finally:
 			obj.save()
 
diff --git a/djstripe/views.py b/djstripe/views.py
--- a/djstripe/views.py
+++ b/djstripe/views.py
@@ -3,7 +3,7 @@
 """
 import logging
 
-from django.http import HttpResponse, HttpResponseBadRequest, HttpResponseServerError
+from django.http import HttpResponse, HttpResponseBadRequest
 from django.utils.decorators import method_decorator
 from django.views.decorators.csrf import csrf_exempt
 from django.views.generic import View
@@ -34,10 +34,6 @@ def post(self, request):
 
 		trigger = WebhookEventTrigger.from_request(request)
 
-		if trigger.exception:
-			# An exception happened, return 500
-			return HttpResponseServerError()
-
 		if trigger.is_test_event:
 			# Since we don't do signature verification, we have to skip trigger.valid
 			return HttpResponse("Test webhook successfully received!")


Test patch:
diff --git a/tests/test_webhooks.py b/tests/test_webhooks.py
--- a/tests/test_webhooks.py
+++ b/tests/test_webhooks.py
@@ -156,6 +156,29 @@ def test_webhook_no_remote_addr(self):
 		event_trigger = WebhookEventTrigger.objects.first()
 		self.assertEqual(event_trigger.remote_ip, "0.0.0.0")
 
+	@patch("djstripe.models.WebhookEventTrigger.validate", return_value=True)
+	@patch("djstripe.models.WebhookEventTrigger.process")
+	def test_webhook_reraise_exception(
+		self, webhook_event_process_mock, webhook_event_validate_mock
+	):
+		class ProcessException(Exception):
+			pass
+
+		exception_message = "process fail"
+
+		webhook_event_process_mock.side_effect = ProcessException(exception_message)
+
+		self.assertEqual(WebhookEventTrigger.objects.count(), 0)
+
+		fake_event = deepcopy(FAKE_EVENT_TRANSFER_CREATED)
+
+		with self.assertRaisesMessage(ProcessException, exception_message):
+			self._send_event(fake_event)
+
+		self.assertEqual(WebhookEventTrigger.objects.count(), 1)
+		event_trigger = WebhookEventTrigger.objects.first()
+		self.assertEqual(event_trigger.exception, exception_message)
+
 	@patch.object(
 		djstripe_settings, "WEBHOOK_EVENT_CALLBACK", return_value=mock_webhook_handler
 	)