Repository: adamchainz/django-mysql

Issue:
DB index is not used on some queries
### Summary: 

django-mysql allows to perform a query on JSON encapsulated fields. To prevent performance issues, we created an index as explained [here](https://mysqlserverteam.com/indexing-json-documents-via-virtual-columns/).
So we expect the index to be used when running this query:
```
TableDoc.objects.filter(field__encaspulated_field="value")
```
But it does not, as you can see below (the sql query is the one generated by django):
```
EXPLAIN SELECT * FROM `tabledoc` 
WHERE JSON_EXTRACT(`tabledoc`.`field`, '$.encaspulated_field') = (CAST('\"value\"' AS JSON));
```
id|	select_type|	table|	partitions|	type|possible_keys|	key|	key_len|	ref|	rows|	filtered|	Extra
----- | ----- | ----- | ----- | ----- | ----- | ----- | ----- |  ----- | ----- | ----- | ----- |  
1|SIMPLE	|tabledoc	|NULL	|ALL|	encaspulated_field|	NULL|	NULL|	NULL|	71763|	10.00|	Using where; Using filesort

**When modifying the query to use naively the value without JSON CAST, the index is well used.**
```
EXPLAIN SELECT * FROM `tabledoc` 
WHERE JSON_EXTRACT(`tabledoc`.`field`, '$.encaspulated_field') = "value");
```
id|	select_type|	table|	partitions|	type|possible_keys|	key|	key_len|	ref|	rows|	filtered|	Extra
----- | ----- | ----- | ----- | ----- | ----- | ----- | ----- |  ----- | ----- | ----- | ----- |  
1|SIMPLE	|tabledoc	|NULL	|ALL|	encaspulated_field|	NULL|	NULL|	NULL|	1|	10.00|	Using where; Using filesort

I'm not sure if it is a bug and if it has been solved on more recent django/django-mysql versions but I would appreciate your feedbacks.

Thanks :)

### Configuration

Django Version: *Django 1.9.1*

Database and version used: *MySQL 5.7*

Version: *Django-MySQL 1.1.0*



Discussion:
I'd consider this a performance bug. I don't believe anything has changed on this in later versions of django-mysql, but there isn't a reason not to upgrade - 2.0.0 was only triggered by two small backwards-incompatible changes that probably don't affect you: https://github.com/adamchainz/django-mysql/blob/master/HISTORY.rst#200-2017-05-28 . Also I hope you find time to upgrade to the latest Django 1.9.x at least, there are lots of security fixes you're missing out on!

As for solving this, perhaps we can improve the search behaviour to not always insert `CAST` on the right hand side for expressions that resolve in simple values. Would you be interested in looking into this?
Hi @adamchainz,
I gave a short shot by forking the repository and change some stuff in the `JSONValue` class but it was not enough... I will try harder to find a solution. Thank you.
@enono78 do you want to come back to this?

Resolving patch:
diff --git a/src/django_mysql/models/lookups.py b/src/django_mysql/models/lookups.py
--- a/src/django_mysql/models/lookups.py
+++ b/src/django_mysql/models/lookups.py
@@ -2,7 +2,7 @@
 from collections.abc import Sequence
 
 import django
-from django.db.models import CharField, Lookup, Transform
+from django.db.models import CharField, Lookup, Transform, Value
 from django.db.models.lookups import (
     BuiltinLookup,
     Exact,
@@ -53,7 +53,10 @@ class JSONLookupMixin:
         def get_prep_lookup(self):
             value = self.rhs
             if not hasattr(value, "resolve_expression") and value is not None:
-                return JSONValue(value)
+                if isinstance(value, (str, int, float)) and not isinstance(value, bool):
+                    return Value(value)
+                else:
+                    return JSONValue(value)
             return super().get_prep_lookup()
 
     else:
@@ -61,7 +64,10 @@ def get_prep_lookup(self):
         def get_prep_lookup(self):
             value = self.rhs
             if not hasattr(value, "_prepare") and value is not None:
-                return JSONValue(value)
+                if isinstance(value, (str, int, float)) and not isinstance(value, bool):
+                    return Value(value)
+                else:
+                    return JSONValue(value)
             return super().get_prep_lookup()
 
 


Test patch:
diff --git a/tests/testapp/models.py b/tests/testapp/models.py
--- a/tests/testapp/models.py
+++ b/tests/testapp/models.py
@@ -280,6 +280,8 @@ class JSONModel(Model):
     ) and connection._nodb_connection.mysql_version >= (5, 7):
         attrs = JSONField(null=True)
 
+    name = CharField(max_length=3)
+
     def __unicode__(self):
         return str(json.dumps(self.attrs))
 
diff --git a/tests/testapp/test_jsonfield.py b/tests/testapp/test_jsonfield.py
--- a/tests/testapp/test_jsonfield.py
+++ b/tests/testapp/test_jsonfield.py
@@ -132,11 +132,14 @@ def setUp(self):
         super().setUp()
         JSONModel.objects.bulk_create(
             [
-                JSONModel(attrs={"a": "b"}),
-                JSONModel(attrs=1337),
+                JSONModel(attrs={"a": "b"}, name="b"),
+                JSONModel(attrs=1337, name="x"),
                 JSONModel(attrs=["an", "array"]),
                 JSONModel(attrs=None),
-                JSONModel(attrs="foo"),
+                JSONModel(attrs="foo", name="foo"),
+                JSONModel(attrs=True),
+                JSONModel(attrs=False),
+                JSONModel(attrs=0.5),
             ]
         )
         self.objs = list(JSONModel.objects.all().order_by("id"))
@@ -144,15 +147,23 @@ def setUp(self):
     def test_equal(self):
         assert list(JSONModel.objects.filter(attrs={"a": "b"})) == [self.objs[0]]
 
-    def test_equal_value(self):
+    def test_equal_int(self):
         assert list(JSONModel.objects.filter(attrs=1337)) == [self.objs[1]]
 
+    def test_equal_float(self):
+        assert list(JSONModel.objects.filter(attrs=0.5)) == [self.objs[7]]
+        assert list(JSONModel.objects.filter(attrs=0.501)) == []
+
     def test_equal_string(self):
         assert list(JSONModel.objects.filter(attrs="foo")) == [self.objs[4]]
 
     def test_equal_array(self):
         assert list(JSONModel.objects.filter(attrs=["an", "array"])) == [self.objs[2]]
 
+    def test_equal_bool(self):
+        assert list(JSONModel.objects.filter(attrs=True)) == [self.objs[5]]
+        assert list(JSONModel.objects.filter(attrs=False)) == [self.objs[6]]
+
     def test_equal_no_match(self):
         assert list(JSONModel.objects.filter(attrs={"c": "z"})) == []
 
@@ -162,7 +173,12 @@ def test_equal_F_attrs(self):
             self.objs[1],
             self.objs[2],
             self.objs[4],
+            self.objs[5],
+            self.objs[6],
+            self.objs[7],
         ]
+        assert list(JSONModel.objects.filter(attrs=F("name"))) == [self.objs[4]]
+        assert list(JSONModel.objects.filter(attrs__a=F("name"))) == [self.objs[0]]
 
     def test_isnull_True(self):
         assert list(JSONModel.objects.filter(attrs__isnull=True)) == [self.objs[3]]
@@ -173,6 +189,9 @@ def test_isnull_False(self):
             self.objs[1],
             self.objs[2],
             self.objs[4],
+            self.objs[5],
+            self.objs[6],
+            self.objs[7],
         ]
 
     def test_range_broken(self):
@@ -507,7 +526,8 @@ class TestSerialization(JSONFieldTestCase):
     test_data = """[
         {
             "fields": {
-                "attrs": {"a": "b", "c": null}
+                "attrs": {"a": "b", "c": null},
+                "name": ""
             },
             "model": "testapp.jsonmodel",
             "pk": null