Repository: jazzband/django-debug-toolbar

Issue:
Middleware breaks raising APIException from ViewSet
When raising a subclass of `APIException` (for example `ValidationError` or `ParseError`) from a `ViewSet`, I get a very cryptic error message that suggests that some sort function is broken. When I disable the middleware, everything works as expected.

Minimal code example:

```
# views.py
class MyViewSet(viewsets.GenericViewSet):
    ...

    @action(methods=['post'], detail=False, url_path='bulk-create')
    def bulk_create(self, request, *args, **kwargs):
        # Some assertion about data structure        
        if True:
            raise ParseError("Expected the data to be in list format.")

        # Actually process incoming data
        ...
```

Error log:

```
Traceback (most recent call last):
2022-05-17T12:23:45.366216553Z   File "/opt/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
2022-05-17T12:23:45.366221178Z     response = get_response(request)
2022-05-17T12:23:45.366223345Z   File "/opt/venv/lib/python3.9/site-packages/debug_toolbar/middleware.py", line 67, in __call__
2022-05-17T12:23:45.366225470Z     panel.generate_stats(request, response)
2022-05-17T12:23:45.366238220Z   File "/opt/venv/lib/python3.9/site-packages/debug_toolbar/panels/request.py", line 30, in generate_stats
2022-05-17T12:23:45.366240470Z     "post": get_sorted_request_variable(request.POST),
2022-05-17T12:23:45.366346803Z   File "/opt/venv/lib/python3.9/site-packages/debug_toolbar/utils.py", line 227, in get_sorted_request_variable
2022-05-17T12:23:45.366352511Z     return [(k, variable.getlist(k)) for k in sorted(variable)]
2022-05-17T12:23:45.366354803Z TypeError: '<' not supported between instances of 'dict' and 'dict'
```

I have also posted this question on StackOverflow: https://stackoverflow.com/questions/72274329/cannot-raise-apiexception-from-django-viewset

Thank you for your help in advance.


Discussion:
I think we had a problem like this in the past. We should probably add a fallback somewhere if sorting the data in `request.POST` fails instead of crashing and burning. It's definitely unexpected that `request.POST` contains anything else than string keys and string values though.

Are you using django-rest-framework or something similar by chance? Does DRF modify Django's standard `request.POST` object?
> Are you using django-rest-framework or something similar by chance? Does DRF modify Django's standard `request.POST` object?

Yes, I am using DRF version 3.12.4.

I just logged `request.POST` and only got `<QueryDict: {}>`. Does that help?

Can you log/print `variable` for the exception flow? Both QueryDict and a `dict` can have sorted called on them. However, calling `sorted([{"a": "b"}, {"c": "d"}])` will raise the error you get. And in your SO question, you made a statement about `request.data` being a list or not.
@matthiask it sounds like somehow `request.POST` isn't actually a dictionary for this case.
@tim-schilling Yes, it seems like that. I'm confused by `<QueryDict: {}>` though if it isn't actually a `QueryDict` instance.

@FinnGu What are you POSTing exactly? A JSON document with a list as its outer element? DRF seems to set `request.POST` (see https://github.com/encode/django-rest-framework/blob/3.12.4/rest_framework/request.py) (well, `request._post` but that's basically the same) but only if the content type of the request matches a form media type, and `application/json` shouldn't trigger this behavior. Maybe you're POSTing a JSON document with a form media type? I have a hard time imaging what goes wrong, but the exact request you're sending might help with constructing a test case for django-debug-toolbar. I wouldn't want DDT to produce meaningful output if the request is in fact not well formed but DDT shouldn't crash.
@matthiask Here is the JSON, I am POSTing with `Content-Type: application/json`: 

```
[
	{
		"shop_reference_id": "891",
		"date": "2021-09-01T20:49:01.0000Z",
		"donations": [
			{
				"amount": "1.23",
				"cooperation": 14,
				"donor_type": "PERSONAL"
			},
			{
				"amount": "0.10",
				"cooperation": 14,
				"donor_type": "BUSINESS"
			}
		]
	},
	{
		"shop_reference_id": "1343",
		"date": "2021-11-20T21:03:22.0000Z",
		"donations":[
			{
				"amount": "0.15",
				"cooperation": 14,
				"donor_type": "BUSINESS"
			}
		]
	}
]
```

I just tested your assumption and you are correct. It is enough to pass an array with any kind of object in it, to crash the DDT. Is it bad practice to use arrays in RESTful design? Because it is valid JSON.
@tim-schilling Which variable did you want me to print? I am confused.


Resolving patch:
diff --git a/debug_toolbar/panels/request.py b/debug_toolbar/panels/request.py
--- a/debug_toolbar/panels/request.py
+++ b/debug_toolbar/panels/request.py
@@ -61,9 +61,11 @@ def generate_stats(self, request, response):
         if hasattr(request, "session"):
             self.record_stats(
                 {
-                    "session": [
-                        (k, request.session.get(k))
-                        for k in sorted(request.session.keys())
-                    ]
+                    "session": {
+                        "list": [
+                            (k, request.session.get(k))
+                            for k in sorted(request.session.keys())
+                        ]
+                    }
                 }
             )
diff --git a/debug_toolbar/utils.py b/debug_toolbar/utils.py
--- a/debug_toolbar/utils.py
+++ b/debug_toolbar/utils.py
@@ -231,12 +231,16 @@ def getframeinfo(frame, context=1):
 
 def get_sorted_request_variable(variable):
     """
-    Get a sorted list of variables from the request data.
+    Get a data structure for showing a sorted list of variables from the
+    request data.
     """
-    if isinstance(variable, dict):
-        return [(k, variable.get(k)) for k in sorted(variable)]
-    else:
-        return [(k, variable.getlist(k)) for k in sorted(variable)]
+    try:
+        if isinstance(variable, dict):
+            return {"list": [(k, variable.get(k)) for k in sorted(variable)]}
+        else:
+            return {"list": [(k, variable.getlist(k)) for k in sorted(variable)]}
+    except TypeError:
+        return {"raw": variable}
 
 
 def get_stack(context=1):


Test patch:
diff --git a/tests/panels/test_request.py b/tests/panels/test_request.py
--- a/tests/panels/test_request.py
+++ b/tests/panels/test_request.py
@@ -85,6 +85,19 @@ def test_dict_for_request_in_method_post(self):
         self.assertIn("foo", content)
         self.assertIn("bar", content)
 
+    def test_list_for_request_in_method_post(self):
+        """
+        Verify that the toolbar doesn't crash if request.POST contains unexpected data.
+
+        See https://github.com/jazzband/django-debug-toolbar/issues/1621
+        """
+        self.request.POST = [{"a": 1}, {"b": 2}]
+        response = self.panel.process_request(self.request)
+        self.panel.generate_stats(self.request, response)
+        # ensure the panel POST request data is processed correctly.
+        content = self.panel.content
+        self.assertIn("[{&#x27;a&#x27;: 1}, {&#x27;b&#x27;: 2}]", content)
+
     def test_namespaced_url(self):
         self.request.path = "/admin/login/"
         response = self.panel.process_request(self.request)