Repository: jazzband/django-debug-toolbar

Issue:
Error when recording SQL queries that are byte-strings
When a byte-string SQL query is run, e.g. those produced by `psycopg2.execute_values`, I get the error `TypeError: startswith first arg must be bytes or a tuple of bytes, not str`. 

This seems to be a result of line 142 in `django-debug-toolbar/debug_toolbar/panels/sql/tracking.py`: `'is_select': sql.lower().strip().startswith('select')`
since 'select' is unicode. 

Flask-debugtoolbar [addressed this](https://github.com/mgood/flask-debugtoolbar/commit/70488fc14a387cfcae6c48811ea4b6ce68564fef) with `prefix = b'select' if isinstance(statement, bytes) else 'select'` - would that work in this case?


Discussion:
The DB API 2.0 PEP (https://www.python.org/dev/peps/pep-0249/) does not mandate a type for the SQL operation -- whether it's bytes or str isn't defined (which isn't surprising at all since the PEP was added at [least 17 years ago](https://github.com/python/peps/commit/8f8671dac38b1ed8950d0318a32618881eff1982)). 

Still, I'm a bit reluctant because I fear breakage throughout; is this really all that's required to support byte string queries? Should we even support byte string queries at all?
I'm OK with that change.
Great. @ahardjasa do you want to submit a pull request?
Same problem for ``Django`` raw SQL queries such as:

```
with connection.cursor() as cursor:
            cursor.execute(
                sql.SQL(
                    '''
                    SELECT json_build_object(
                        'data', array_agg(r)
                    )
                    FROM (
                        SELECT
                        a,
                        {}
                        FROM b
                        WHERE c=%s
                        ORDER BY a DESC
                        LIMIT %s
                    ) r
                    '''
                ).format(var),
                [x, 5]
            )
            result = cursor.fetchall()
```
This appears to be a problem for me as well.  I'm not sure if this should be broken off into a different issue ticket?

I'm storing binary image thumbnails, and it works fine when I am not running django debug toolbar, but as soon as I enable DDT, a bit of chaos occurs.

With it enabled, Django can not store, read, or manipulate the binary fields.  Everything else works fine, but the binary fields do not work.
Is this still a problem? Looking around I found this test (in `tests/panels/test_sql.py`) and it seems to catch this use case:

```python
def test_non_ascii_query(self):
        ...
        # non-ASCII bytes parameters
        list(Binary.objects.filter(field__in=["café".encode()]))
        self.assertEqual(len(self.panel._queries), 3)
        ...
```
@Caiofcas Thank you! If I'm reading this issue correctly it is about the whole SQL string being bytes which is different from the test where (only) a parameter is bytes.
I just got this with the latest version, 4.1.0. [This line](https://github.com/psycopg/psycopg2/blob/9f020124f83ed04d0d213eca7735555f618fcd47/lib/extras.py#L1299) in `psycopg2.extras.execute_values` causes it to execute a query of type `bytes`.

utting this in a view alone will raise the issue when the toolbar is active:

```python
from django.db import connection

with connection.cursor() as cursor:
    cursor.execute(b'SELECT 1')
```
@Lucidiot It seems nobody is actively working on this right now. Does the proposed fix in the initial report make everything work as it should? It would be very helpful if you could check this and maybe submit a pull request with a small test case. Thanks in advance!
@Lucidiot I could also potentially take a look in the next couple of days if you don't have time to get a PR together.

Resolving patch:
diff --git a/debug_toolbar/panels/sql/tracking.py b/debug_toolbar/panels/sql/tracking.py
--- a/debug_toolbar/panels/sql/tracking.py
+++ b/debug_toolbar/panels/sql/tracking.py
@@ -180,7 +180,10 @@ def _record(self, method, sql, params):
             # Sql might be an object (such as psycopg Composed).
             # For logging purposes, make sure it's str.
             if vendor == "postgresql" and not isinstance(sql, str):
-                sql = sql.as_string(conn)
+                if isinstance(sql, bytes):
+                    sql = sql.decode("utf-8")
+                else:
+                    sql = sql.as_string(conn)
             else:
                 sql = str(sql)
 


Test patch:
diff --git a/tests/panels/test_sql.py b/tests/panels/test_sql.py
--- a/tests/panels/test_sql.py
+++ b/tests/panels/test_sql.py
@@ -158,6 +158,17 @@ def test_non_ascii_query(self):
         # ensure the panel renders correctly
         self.assertIn("café", self.panel.content)
 
+    @unittest.skipUnless(
+        connection.vendor == "postgresql", "Test valid only on PostgreSQL"
+    )
+    def test_bytes_query(self):
+        self.assertEqual(len(self.panel._queries), 0)
+
+        with connection.cursor() as cursor:
+            cursor.execute(b"SELECT 1")
+
+        self.assertEqual(len(self.panel._queries), 1)
+
     def test_param_conversion(self):
         self.assertEqual(len(self.panel._queries), 0)