Repository: bokeh/bokeh

Issue:
Modifying datasources broken
In bokeh master and presumably any rc/dev releases cut in the last few days, modifying a CDS is broken completely. This is because https://github.com/bokeh/bokeh/pull/6374 introduced a change in the way bokeh properties check whether an old and new value has changed. The approach there guards against checks on arrays, but since the ``.data`` property on a CDS is not itself an array it ends up doing a simple equality between the dictionary of arrays, which raises this error:

>ValueError�: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

This occurs for any change to a ``ColumnDataSource.data``, which means that any server or notebook examples that modify a CDS in python are currently broken.


Discussion:
@philippjfr can you post the entire stack trace
The only code path before was `old==new`. Now there is still that code path, but also another to handle a special case (i.e. at least of the args is an array). I have to conclude then that the error is coming from inside `array_equal`, or else it would have shown up before (in the same `old==new` code path that has always been there) that's why a full stack trace is important. 
I can only presume that those instance checks are wrong for whatever reason, but without code to reproduce this and learn what `new` and `old` are, I can only speculate.
sliders app reproduces, I will look into it now
Minimal example in the notebook since a server app doesn't give me the full stacktrace:

```python
from bokeh.plotting import figure, output_notebook, show, ColumnDataSource
import numpy as np
from bokeh.plotting import ColumnDataSource

cds = ColumnDataSource(data={'x': np.array([1, 2, 3, 4, 5]), 'y': np.array([6, 7, 2, 4, 5])})

# output to static HTML file
output_notebook()

p = figure(plot_width=400, plot_height=400)

# add a circle renderer with a size, color, and alpha
p.circle(source=cds, x='x', y='y', size=20, color="navy", alpha=0.5)

# show the results
show(p)

cds.data['x'] = np.arange(5)
```

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-9b5e79fa2ec7> in <module>()
----> 1 cds.data['x'] = np.arange(5)

/Users/philippjfr/bokeh/bokeh/core/property/containers.pyc in wrapper(self, *args, **kwargs)
     75         old = self._saved_copy()
     76         result = func(self, *args, **kwargs)
---> 77         self._notify_owners(old)
     78         return result
     79     wrapper.__doc__ = "Container method ``%s`` instrumented to notify property owners" % func.__name__

/Users/philippjfr/bokeh/bokeh/core/property/containers.pyc in _notify_owners(self, old, hint)
    102     def _notify_owners(self, old, hint=None):
    103         for (owner, prop) in self._owners:
--> 104             prop._notify_mutated(owner, old, hint)
    105 
    106     def _saved_copy(self):

/Users/philippjfr/bokeh/bokeh/core/property/descriptors.pyc in _notify_mutated(self, obj, old, hint)
    813         value = self.property.prepare_value(obj, self.name, value)
    814 
--> 815         self._real_set(obj, old, value, hint)
    816 
    817     def _trigger(self, obj, old, value, hint=None, setter=None):

/Users/philippjfr/bokeh/bokeh/core/property/descriptors.pyc in _real_set(self, obj, old, value, hint, setter)
    747         # As of Bokeh 0.11.1, all hinted events modify in place. However this
    748         # may need refining later if this assumption changes.
--> 749         unchanged = self.property.matches(value, old) and (hint is None)
    750         if unchanged:
    751             return

/Users/philippjfr/bokeh/bokeh/core/property/bases.pyc in matches(self, new, old)
    180             return np.array_equal(new, old)
    181 
--> 182         return new == old
    183 
    184     def from_json(self, json, models=None):

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Actually, before there was also a general exception to catch `ValueErrors` that was removed, and is probably the problem. It's a cheap way to solve this I think but I will put it back for now. 
Here is a succinct elucidation of the probelm:
```
In [11]: d1 = dict(foo=np.arange(10))

In [12]: d2 = dict(foo=np.arange(10))

In [13]: d1 == d2
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-13-4919d8a01ac8> in <module>()
----> 1 d1 == d2

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```
To really solve this we should compare containers more carefully, I will just put back the try block for now

```
ipdb> p new
{'y': array([6, 7, 2, 4, 5]), 'x': array([0, 1, 2, 3, 4])}
ipdb> p old
{'y': array([6, 7, 2, 4, 5]), 'x': array([1, 2, 3, 4, 5])}
ipdb>
```
Yeah, `ValueError` was there for nested data structures with arrays.
I doubt this is actually a solvable problem, because you would have to change `__eq__` for all containers or preferably make numpy not mess with the definition of equality.
@mattpap this seems to work:
```
        if isinstance(new, np.ndarray) or isinstance(old, np.ndarray):
            return np.array_equal(new, old)

        if isinstance(new, dict) and isinstance(old, dict):
            return list(new.keys()) == list(old.keys()) and \
                all(self.matches(*args) for args in zip(new.values(), old.values()))

        return new == old
```

Could still add the `try` block as a failsafe but this would give better match results for a common case.

What about lists of arrays, etc.? I would just restore the original and consider a better solution for next release.
@mattpap those will just return no-match (as before) but "dict of arrays" is actually very common. I am proposing this:
```
        if isinstance(new, np.ndarray) or isinstance(old, np.ndarray):
            return np.array_equal(new, old)

        # this handles the special but common case where there is a dict with numpy
        # arrays as values (e.g. the .data property of a ColumnDataSource)
        if isinstance(new, dict) and isinstance(old, dict):
            return list(new.keys()) == list(old.keys()) and \
                all(self.matches(*args) for args in zip(new.values(), old.values()))

        try:
            return new == old

        # if the comparison fails for some reason, just punt and return no-match
        except ValueError:
            return False
```
with some tests added 
OK.

Resolving patch:
diff --git a/bokeh/core/property/bases.py b/bokeh/core/property/bases.py
--- a/bokeh/core/property/bases.py
+++ b/bokeh/core/property/bases.py
@@ -178,7 +178,19 @@ def matches(self, new, old):
         if isinstance(new, np.ndarray) or isinstance(old, np.ndarray):
             return np.array_equal(new, old)
 
-        return new == old
+        # this handles the special but common case where there is a dict with numpy
+        # arrays as values (e.g. the .data property of a ColumnDataSource)
+        if isinstance(new, dict) and isinstance(old, dict):
+            if set(new.keys()) != set(old.keys()):
+                return False
+            return all(self.matches(new[k], old[k]) for k in new)
+
+        try:
+            return new == old
+
+        # if the comparison fails for some reason, just punt and return no-match
+        except ValueError:
+            return False
 
     def from_json(self, json, models=None):
         ''' Convert from JSON-compatible values into a value for this property.


Test patch:
diff --git a/bokeh/core/property/tests/test_bases.py b/bokeh/core/property/tests/test_bases.py
--- a/bokeh/core/property/tests/test_bases.py
+++ b/bokeh/core/property/tests/test_bases.py
@@ -45,6 +45,14 @@ def raise_(ex):
         p.prepare_value(hp, "foo", 10)
         assert str(e) == "bad True name, 10"
 
+def test_property_matches_basic_types(capsys):
+    p = pb.Property()
+    for x in [1, 1.2, "a", np.arange(4), None, False, True, {}, []]:
+        assert p.matches(x, x) is True
+        assert p.matches(x, "junk") is False
+    out, err = capsys.readouterr()
+    assert err == ""
+
 def test_property_matches_compatible_arrays(capsys):
     p = pb.Property()
     a = np.arange(5)
@@ -65,3 +73,35 @@ def test_property_matches_incompatible_arrays(capsys):
     out, err = capsys.readouterr()
     # no way to suppress FutureWarning in this case
     # assert err == ""
+
+def test_property_matches_dicts_with_array_values(capsys):
+    p = pb.Property()
+    d1 = dict(foo=np.arange(10))
+    d2 = dict(foo=np.arange(10))
+
+    assert p.matches(d1, d1) is True
+    assert p.matches(d1, d2) is True
+
+    # XXX not sure if this is preferable to have match, or not
+    assert p.matches(d1, dict(foo=list(range(10)))) is True
+
+    assert p.matches(d1, dict(foo=np.arange(11))) is False
+    assert p.matches(d1, dict(bar=np.arange(10))) is False
+    assert p.matches(d1, dict(bar=10)) is False
+    out, err = capsys.readouterr()
+    assert err == ""
+
+def test_property_matches_non_dict_containers_with_array_false(capsys):
+    p = pb.Property()
+    d1 = [np.arange(10)]
+    d2 = [np.arange(10)]
+    assert p.matches(d1, d1) is True  # because object identity
+    assert p.matches(d1, d2) is False
+
+    t1 = (np.arange(10),)
+    t2 = (np.arange(10),)
+    assert p.matches(t1, t1) is True  # because object identity
+    assert p.matches(t1, t2) is False
+
+    out, err = capsys.readouterr()
+    assert err == ""