Repository: bokeh/bokeh

Issue:
Turn off self.validate checks during update
When running a Bokeh server with streaming updates in my application I often find that bokeh takes up 30% of the CPU time, about half of which is consumed by the following self.validate checks:

```diff
diff --git a/bokeh/core/property/bases.py b/bokeh/core/property/bases.py
index d28aa2b..a50ed9e 100644
--- a/bokeh/core/property/bases.py
+++ b/bokeh/core/property/bases.py
@@ -260,7 +260,7 @@ class Property(PropertyDescriptorFactory):
 
         '''
         try:
-            self.validate(value, False)
+            # self.validate(value, False)
         except ValueError as e:
             return False
         else:
@@ -275,7 +275,7 @@ class Property(PropertyDescriptorFactory):
 
     def prepare_value(self, obj_or_cls, name, value):
         try:
-            self.validate(value)
+            # self.validate(value)
         except ValueError as e:
             for tp, converter in self.alternatives:
                 if tp.is_valid(value):
```

I would like some way to disable these, preferably without clobbering with other users' Bokeh experience (my bokeh server application is often embedded within other Jupyter sessions, where the user may use Bokeh on their own).

So some questions:

1.  Does the current environment variable solution with `BOKEH_VALIDATE_DOC=0` disable these checks?
2.  Is there some way I can turn them off programmatically,rather than tweaking an environment variable?  
3.  Is there some safe way that I can mark all of the properties that I care about as being safe and not needing validation, while not affecting other users' experience?

Some thoughts:

1.  Can we construct a function that propagates through a figure, CDS, or doc and marks all properties as not needing to be validated?
2.  Can we temporarily set a global before and after calling update/stream/patch that signals bokeh to not validate during that period but unset it directly afterwards?

    ```python
    with bokeh.no_validate:
        source.update(...)
    ```

Follows on from https://github.com/bokeh/bokeh/issues/6042
Turn off self.validate checks during update
When running a Bokeh server with streaming updates in my application I often find that bokeh takes up 30% of the CPU time, about half of which is consumed by the following self.validate checks:

```diff
diff --git a/bokeh/core/property/bases.py b/bokeh/core/property/bases.py
index d28aa2b..a50ed9e 100644
--- a/bokeh/core/property/bases.py
+++ b/bokeh/core/property/bases.py
@@ -260,7 +260,7 @@ class Property(PropertyDescriptorFactory):
 
         '''
         try:
-            self.validate(value, False)
+            # self.validate(value, False)
         except ValueError as e:
             return False
         else:
@@ -275,7 +275,7 @@ class Property(PropertyDescriptorFactory):
 
     def prepare_value(self, obj_or_cls, name, value):
         try:
-            self.validate(value)
+            # self.validate(value)
         except ValueError as e:
             for tp, converter in self.alternatives:
                 if tp.is_valid(value):
```

I would like some way to disable these, preferably without clobbering with other users' Bokeh experience (my bokeh server application is often embedded within other Jupyter sessions, where the user may use Bokeh on their own).

So some questions:

1.  Does the current environment variable solution with `BOKEH_VALIDATE_DOC=0` disable these checks?
2.  Is there some way I can turn them off programmatically,rather than tweaking an environment variable?  
3.  Is there some safe way that I can mark all of the properties that I care about as being safe and not needing validation, while not affecting other users' experience?

Some thoughts:

1.  Can we construct a function that propagates through a figure, CDS, or doc and marks all properties as not needing to be validated?
2.  Can we temporarily set a global before and after calling update/stream/patch that signals bokeh to not validate during that period but unset it directly afterwards?

    ```python
    with bokeh.no_validate:
        source.update(...)
    ```

Follows on from https://github.com/bokeh/bokeh/issues/6042


Discussion:
1) No, `BOKEH_VALIDATE_DOC` only affect document validation checks at serialization time, not property type validation 

2) There is a way to turn off document validation checks programmatically, there is not a way to turn of property type validation checks. 

3) No, but only vacuously, because there is currently no way at all to mark any properties as not needing validation

Regarding possible implementations, I guess it boils down to how fine grained we would like to be. Options:

* mark individual properties, definitely tedious without some helper API

* mark code blocks with context managers, could be OK but not my pref

* mark entire update functions with a decorator, i.e.

    ```
    @without_property_validation
    def update(attr, old, new):
        # do stuff without validation
    ```

My personal preference is the decorator, but I am open to arguments why other approaches are better or more preferable.  
The decorator would be fine with me if that would work on the Bokeh side.  As a performance-conscious Bokeh user I'm pretty willing to do stuff on my end to make this happen.
@mrocklin great I am happy to collaborate on this. Ideally, a PR would add:

* an implementation of the decorator (I think you have a handle on how this would look) We currently have `from bokeh.document import without_document_lock` so offhandf I'd just suggest also putting `without_property_validation` in `bokeh.document` as well
* docstring for the decorator
* new subsection in "Running a Bokeh Server" users guide chapter
* a test that verifies the operation of the decorator, offhand probably via mocking the validate function somehow

Optionally we might want to update one or two of the examples to demonstrate the decorator. 

I'd say put up a PR with whatever subset of that you are free to finish and I will take it from there. 

Is it OK to communicate the "don't validate" message through some global variable that the result of the decorator mutates while calling?  Or is there some more clever way?
I think it's ok to start with that. Or if you want to wrap the global state in some API function that'd be ok. I suppose that has the added benefit of being accessible to e.g. someone wanting to speed up standalone plot generation in a flask app
1) No, `BOKEH_VALIDATE_DOC` only affect document validation checks at serialization time, not property type validation 

2) There is a way to turn off document validation checks programmatically, there is not a way to turn of property type validation checks. 

3) No, but only vacuously, because there is currently no way at all to mark any properties as not needing validation

Regarding possible implementations, I guess it boils down to how fine grained we would like to be. Options:

* mark individual properties, definitely tedious without some helper API

* mark code blocks with context managers, could be OK but not my pref

* mark entire update functions with a decorator, i.e.

    ```
    @without_property_validation
    def update(attr, old, new):
        # do stuff without validation
    ```

My personal preference is the decorator, but I am open to arguments why other approaches are better or more preferable.  
The decorator would be fine with me if that would work on the Bokeh side.  As a performance-conscious Bokeh user I'm pretty willing to do stuff on my end to make this happen.
@mrocklin great I am happy to collaborate on this. Ideally, a PR would add:

* an implementation of the decorator (I think you have a handle on how this would look) We currently have `from bokeh.document import without_document_lock` so offhandf I'd just suggest also putting `without_property_validation` in `bokeh.document` as well
* docstring for the decorator
* new subsection in "Running a Bokeh Server" users guide chapter
* a test that verifies the operation of the decorator, offhand probably via mocking the validate function somehow

Optionally we might want to update one or two of the examples to demonstrate the decorator. 

I'd say put up a PR with whatever subset of that you are free to finish and I will take it from there. 

Is it OK to communicate the "don't validate" message through some global variable that the result of the decorator mutates while calling?  Or is there some more clever way?
I think it's ok to start with that. Or if you want to wrap the global state in some API function that'd be ok. I suppose that has the added benefit of being accessible to e.g. someone wanting to speed up standalone plot generation in a flask app

Resolving patch:
diff --git a/bokeh/core/properties.py b/bokeh/core/properties.py
--- a/bokeh/core/properties.py
+++ b/bokeh/core/properties.py
@@ -81,13 +81,26 @@ class SomeModel(Model):
 .. autofunction:: field
 .. autofunction:: value
 
-
 Special Properties
 ------------------
 
 .. autoclass:: Include
 .. autoclass:: Override
 
+Validation Control
+------------------
+
+By default, Bokeh properties perform type validation on values. This helps to
+ensure the consistency of any data exchanged between Python and JavaScript, as
+well as provide detailed and immediate feedback to users if they attempt to
+set values of the wrong type. However, these type checks incur some overhead.
+In some cases it may be desirable to turn off validation in specific places,
+or even entirely, in order to boost performance. The following API is available
+to control when type validation occurs.
+
+.. autoclass:: validate
+.. autofunction:: without_property_validation
+
 '''
 from __future__ import absolute_import, print_function
 
@@ -99,6 +112,7 @@ class SomeModel(Model):
 from copy import copy
 import datetime
 import dateutil.parser
+from functools import wraps
 from importlib import import_module
 from io import BytesIO
 import numbers
@@ -2128,6 +2142,62 @@ def make_descriptors(self, base_name):
 
         return descriptors
 
+#------------------------------------------------------------------------------
+# Validation Control
+#------------------------------------------------------------------------------
+
+class validate(object):
+    ''' Control validation of bokeh properties
+
+    This can be used as a context manager, or as a normal callable
+
+    Args:
+        value (bool) : Whether validation should occur or not
+
+    Example:
+        .. code-block:: python
+
+            with validate(False):  # do no validate while within this block
+                pass
+
+            validate(False)  # don't validate ever
+
+    See Also:
+        :func:`~bokeh.core.property.bases.validation_on`: check the state of validation
+
+        :func:`~bokeh.core.properties.without_property_validation`: function decorator
+
+    '''
+    def __init__(self, value):
+        self.old = Property._should_validate
+        Property._should_validate = value
+
+    def __enter__(self):
+        pass
+
+    def __exit__(self, typ, value, traceback):
+        Property._should_validate = self.old
+
+
+def without_property_validation(input_function):
+    ''' Turn off property validation during update callbacks
+
+    Example:
+        .. code-block:: python
+
+            @without_property_validation
+            def update(attr, old, new):
+                # do things without validation
+
+    See Also:
+        :class:`~bokeh.core.properties.validate`: context mangager for more fine-grained control
+
+    '''
+    @wraps(input_function)
+    def func(*args, **kwargs):
+        with validate(False):
+            return input_function(*args, **kwargs)
+    return func
 
 # Everything below is just to update the module docstring
 _all_props = set(x for x in globals().values() if isinstance(x, type) and issubclass(x, Property))
diff --git a/bokeh/core/property/bases.py b/bokeh/core/property/bases.py
--- a/bokeh/core/property/bases.py
+++ b/bokeh/core/property/bases.py
@@ -53,6 +53,9 @@ class Property(PropertyDescriptorFactory):
 
     '''
 
+    # This class attribute is controlled by external helper API for validation
+    _should_validate = True
+
     def __init__(self, default=None, help=None, serialized=True, readonly=False):
         # This is how the descriptor is created in the class declaration.
         self._serialized = False if readonly else serialized
@@ -260,7 +263,8 @@ def is_valid(self, value):
 
         '''
         try:
-            self.validate(value, False)
+            if validation_on():
+                self.validate(value, False)
         except ValueError as e:
             return False
         else:
@@ -275,7 +279,8 @@ def wrap(cls, value):
 
     def prepare_value(self, obj_or_cls, name, value):
         try:
-            self.validate(value)
+            if validation_on():
+                self.validate(value)
         except ValueError as e:
             for tp, converter in self.alternatives:
                 if tp.is_valid(value):
@@ -427,3 +432,12 @@ class ContainerProperty(ParameterizedProperty):
     def _may_have_unstable_default(self):
         # all containers are mutable, so the default can be modified
         return True
+
+def validation_on():
+    ''' Check if property validation is currently active
+
+    Returns:
+        bool
+
+    '''
+    return Property._should_validate


Test patch:
diff --git a/bokeh/core/property/tests/test_bases.py b/bokeh/core/property/tests/test_bases.py
--- a/bokeh/core/property/tests/test_bases.py
+++ b/bokeh/core/property/tests/test_bases.py
@@ -147,3 +147,13 @@ def test_property_matches_dicts_with_index_values(capsys, pd):
     assert p.matches(d1.index, 10) is False
     out, err = capsys.readouterr()
     assert err == ""
+
+def test_validation_on():
+    assert pb.Property._should_validate == True
+    assert pb.validation_on()
+
+    pb.Property._should_validate = False
+    assert not pb.validation_on()
+
+    pb.Property._should_validate = True
+    assert pb.validation_on()
diff --git a/bokeh/core/tests/test_properties.py b/bokeh/core/tests/test_properties.py
--- a/bokeh/core/tests/test_properties.py
+++ b/bokeh/core/tests/test_properties.py
@@ -17,15 +17,46 @@
     DistanceSpec, FontSize, FontSizeSpec, Override, Include, MinMaxBounds,
     DataDistanceSpec, ScreenDistanceSpec, ColumnData, UnitsSpec, Image)
 
+from bokeh.core.property.bases import validation_on
 from bokeh.core.property.containers import PropertyValueColumnData, PropertyValueDict, PropertyValueList
 
 from bokeh.core.has_props import HasProps
 
 from bokeh.models import Plot
 
+import bokeh.core.properties as bcp
 
 SPECS = (NumberSpec, ColorSpec, AngleSpec, StringSpec, DistanceSpec, FontSizeSpec, DataDistanceSpec, ScreenDistanceSpec)
 
+class TestValidationControl(object):
+    def test_validate(self):
+        assert validation_on()
+        with bcp.validate(False):
+            assert not validation_on()
+        assert validation_on()
+
+        with bcp.validate(False):
+            assert not validation_on()
+            with bcp.validate(True):
+                assert validation_on()
+            assert not validation_on()
+        assert validation_on()
+
+        bcp.validate(False)
+        assert not validation_on()
+        bcp.validate(True)
+        assert validation_on()
+
+
+    def test_without_property_validation(self):
+        @bcp.without_property_validation
+        def f():
+            assert not validation_on()
+
+        assert validation_on()
+        f()
+        assert validation_on()
+
 class TestValidateDetailDefault(object):
 
     def test_Angle(self):