Repository: Kinto/kinto

Issue:
Optimize use of jsonschema.validate()
It could be interesting to investigate if the fastjsonschema lib would give us better perfs :)

I marked this as easy-pick since everything happens in `schema_validation.py`

https://github.com/horejsek/python-fastjsonschema/


Discussion:
Hey, @leplatrem I am interested in taking up this issue. I am a newbie to open source contribution. 
Hi @anshvyas !

First, you can try to setup the project development environment locally:
https://kinto.readthedocs.io/en/stable/community.html#hack

Once you have that, you can try to execute the test suite related to schema validation:

The following command should be successful:
`$ py.test tests/test_views_schema_record.py`

Then you introduce some changes and verify that the tests still pass :) 

Don't wait for your code to be perfect/done before submitting a pull-request though. 
Hey @leplatrem ,
On it. 
Hey @leplatrem,
The command `$ py.test tests/test_views_schema_record.py` runs successfully without any errors but the command `$ make tests` show certain failing test related to MemcachedCacheTest. How do I proceed with this?
That probably means you don't have `memcached` running on your machine. But it's fine as long as you don't touch the `kinto.core.cache.memcached` code.
Thanks, @glasserc.
Can you explain to me how the schema_validation.py gets executed in the following line
`        resp = self.app.put_json(
            COLLECTION_URL, {"data": {"schema": newschema}}, headers=self.headers, status=400
        )`

> Can you explain to me how the schema_validation.py gets executed in the following line

In this particular case, you will be executing the validation of a collection creation/modification (in `kinto/views/collections.py`)

Thanks @leplatrem That was helpful in understanding the flow.
I suggest we close this issue because switching to `python-fastjsonschema` is probably NOT the right package to use.

Take a look at https://www.peterbe.com/plog/jsonschema-validate-10x-faster-in-python
The point of that blog post is that if you **reuse** an instance, with the same schema, for multiple validations you get a 10x speed boost with good old `python-jsonschema` and potentially even more so with `python-fastjsonschema`. 
BUT, if you look at the benchmark numbers and compare:

```python
def f1(qs):
    for build in qs:
        validate(build.build, SCHEMA)

def f4(qs):
    for build in qs:
        fastjsonschema.validate(SCHEMA, build.build)
```
this becomes a "fair" comparison where you *don't* reuse an instance. In this case the numbers point to that `python-jsonschema` is faster:

```
FUNCTION: f1 Used 3 times
    BEST   1247.9ms
    MEDIAN 1309.0ms
    MEAN   1330.0ms
    STDEV  94.5ms

FUNCTION: f4 Used 3 times
    BEST   2032.3ms
    MEDIAN 2033.4ms
    MEAN   2143.9ms
    STDEV  192.3ms
```

Also, `python-jsonschema` looks like a very healthy project. It's mature. I base that one the fact that it [been around since 2012](https://pypi.org/project/jsonschema/#history). It's an active project with ~2,000 stars on [GitHub](https://github.com/Julian/jsonschema). `python-fastjsonschema` on the other hand is very immature in comparison and has much fewer eyeballs on it. 


Leave it to @leplatrem to decide. 

More interestingly would be to file a new issue that measures how often the JSON Schema validation happens on a busy environment and if it'd be possible to use something like a module level cache of reusable instances. E.g something like this:

```python
_json_schema_validators = {}

def validate(record, schema):
    # return validate(record, schema) # Old way
    schema_hash = hash(schema)
    if schema_hash not in _json_schema_validators:
        _json_schema_validators[schema_hash] = create_json_validator(schema)
    return _json_schema_validators[schema_hash].validate(record)
```
> More interestingly would be to file a new issue that measures how often the JSON Schema validation happens on a busy environment

The JSON schema validation is called everytime a record is modified in a collection that has a schema.

>  if it'd be possible to use something like a module level cache of reusable instances.

Sounds like a good idea!

**edit**: And thanks a lot Peter for your inputs!

Hi @leplatrem, 

It may be a way for `/batch` subrequests to reuse one instance for validation - but that would propably require some changes at Record `process_record` method or invoking it outside of Record and pre-fail before request reaches `process_record` method. WDYT ?

Resolving patch:
diff --git a/kinto/schema_validation.py b/kinto/schema_validation.py
--- a/kinto/schema_validation.py
+++ b/kinto/schema_validation.py
@@ -1,5 +1,6 @@
 import colander
-from jsonschema import ValidationError, SchemaError, RefResolutionError, validate
+from jsonschema import ValidationError, SchemaError, RefResolutionError
+from jsonschema.validators import validator_for
 
 try:  # pragma: no cover
     from jsonschema import Draft7Validator as DraftValidator
@@ -39,6 +40,34 @@ def check_schema(data):
         raise ValidationError(message)
 
 
+# Module level global that stores a version of every possible schema (as a <class 'dict'>)
+# turned into a jsonschema instance (as <class 'jsonschema.validators.Validator'>).
+_schema_cache = {}
+
+
+def validate(data, schema):
+    """Raise a ValidationError or a RefResolutionError if the data doesn't validate
+    with the given schema.
+
+    Note that this function is just a "wrapper" on `jsonschema.validate()` but with
+    some memoization based on the schema for better repeat performance.
+    """
+    # Because the schema is a dict, it can't be used as a hash key so it needs to be
+    # "transformed" to something that is hashable. The quickest solution is to convert
+    # it to a string.
+    # Note that the order of the dict will determine the string it becomes. The solution
+    # to that would a canonical serializer like `json.dumps(..., sort_keys=True)` but it's
+    # overkill since the assumption is that the schema is very unlikely to be exactly
+    # the same but different order.
+    cache_key = str(schema)
+    if cache_key not in _schema_cache:
+        # This is essentially what the `jsonschema.validate()` shortcut function does.
+        cls = validator_for(schema)
+        cls.check_schema(schema)
+        _schema_cache[cache_key] = cls(schema)
+    return _schema_cache[cache_key].validate(data)
+
+
 def validate_schema(data, schema, ignore_fields=[]):
     required_fields = [f for f in schema.get("required", []) if f not in ignore_fields]
     # jsonschema doesn't accept 'required': [] yet.


Test patch: