Repository: Kinto/kinto

Issue:
Crash on dev ota server when requesting /records?min_target.version=53
```
$ http GET https://kinto-ota.dev.mozaws.net/v1/buckets/build-hub/collections/archives/records\?min_target.version\=53`

HTTP/1.1 503 Service Unavailable
Access-Control-Expose-Headers: Total-Records, Next-Page, Expires, Last-Modified, ETag, Content-Length, Pragma, Alert, Cache-Control, Retry-After, Backoff
Connection: keep-alive
Content-Length: 151
Content-Type: application/json
Date: Fri, 05 May 2017 15:13:28 GMT
Retry-After: 3
Server: nginx
X-Content-Type-Options: nosniff

{
    "code": 503, 
    "errno": 201, 
    "error": "Service Unavailable", 
    "message": "Service temporary unavailable due to overloading or maintenance, please retry later."
}
```



Discussion:
```
raise exceptions.BackendError(original=e) from e\nkinto.core.storage.exceptions.BackendError: DataError: (psycopg2.DataError) invalid input syntax for type numeric: "53.0b4"
```
Work around: put quotes around `53`:  `/records?min_target.version="53"`
Is that a feature we support or a side effect?
> Is that a feature we support or a side effect?

What?

We want to use numeric comparison if both field and filter values are numeric, textual otherwise.
I was talking about the fact of adding quotes in the querystring.
It is not properly documented, so I guess we can classify as side effect :))

Do you think that something we should support?
Refs #1216 and #1215. I think we're trying too hard to be clever with an ad-hoc string query syntax and IMO we should leave the current syntax in place ("simple" syntax, where anything that can be converted to a number is a number and everything else is a string) and add a robust syntax that can encode any value as well as its type. I'd propose JSON (so `/records?min_target.version=53` for number, `/records?min_target.version="53"` for string, `/records?min_target.version=null` for null, etc.).

For comparison order, one reason I don't want to be too clever here is that I'm afraid of inventing an inconsistent order. E.g. 2.0 < 10 but 10 < "2.0b". Having a robust syntax doesn't completely avoid this problem because data in different records can be of different types, and we will have to order them somehow (if we get e.g. `_sort=different_typed_field`). How do we handle that? I don't think throwing an exception in this case is necessarily a bad thing, but if we have to invent an ordering, I think "all ints are less than all strings" [i.e. the [Python2.7 comparison rules](https://docs.python.org/2/library/stdtypes.html)] is fine.



Resolving patch:
diff --git a/kinto/core/utils.py b/kinto/core/utils.py
--- a/kinto/core/utils.py
+++ b/kinto/core/utils.py
@@ -1,4 +1,3 @@
-import ast
 import collections
 import hashlib
 import hmac
@@ -11,7 +10,7 @@
 from urllib.parse import unquote
 from enum import Enum
 
-import ujson as json  # NOQA
+import ujson as json
 
 try:
     import sqlalchemy
@@ -114,14 +113,13 @@ def native_value(value):
     :returns: the value coerced to python type
     """
     if isinstance(value, str):
-        if value.lower() in ['on', 'true', 'yes']:
-            value = True
-        elif value.lower() in ['off', 'false', 'no']:
-            value = False
         try:
-            return ast.literal_eval(value)
-        except (TypeError, ValueError, SyntaxError):
-            pass
+            value = json.loads(value)
+        except ValueError:
+            try:
+                value = json.loads('"{}"'.format(value))
+            except ValueError:
+                return value
     return value
 
 


Test patch:
diff --git a/tests/core/test_utils.py b/tests/core/test_utils.py
--- a/tests/core/test_utils.py
+++ b/tests/core/test_utils.py
@@ -33,9 +33,24 @@ class NativeValueTest(unittest.TestCase):
     def test_simple_string(self):
         self.assertEqual(native_value('value'), 'value')
 
+    def test_defined_string(self):
+        self.assertEqual(native_value('"value"'), 'value')
+
+    def test_null_value(self):
+        self.assertEqual(native_value('null'), None)
+
+    def test_defined_null_as_text_value(self):
+        self.assertEqual(native_value('"null"'), 'null')
+
     def test_integer(self):
         self.assertEqual(native_value('7'), 7)
 
+    def test_defined_integer_as_text_value(self):
+        self.assertEqual(native_value('"7"'), '7')
+
+    def test_defined_simple_quote_string_as_text_value(self):
+        self.assertEqual(native_value("'7'"), "'7'")
+
     def test_zero_and_one_coerce_to_integers(self):
         self.assertEqual(native_value('1'), 1)
         self.assertEqual(native_value('0'), 0)
@@ -44,12 +59,12 @@ def test_float(self):
         self.assertEqual(native_value('3.14'), 3.14)
 
     def test_true_values(self):
-        true_strings = ['True', 'on', 'true', 'yes']
+        true_strings = ['true']
         true_values = [native_value(s) for s in true_strings]
         self.assertTrue(all(true_values))
 
     def test_false_values(self):
-        false_strings = ['False', 'off', 'false', 'no']
+        false_strings = ['false']
         false_values = [native_value(s) for s in false_strings]
         self.assertFalse(any(false_values))