Repository: bokeh/bokeh

Issue:
Graphs with list values as attributes fail
### Software versions
python version: Python 3.6.8 :: Anaconda, Inc.
jupyter version: 4.4.0
NetworkX version: 2.2
bokeh version: 1.0.4

### Description of expected behavior and the observed behavior
When assigning any iterator as a node attribute to the root node in networkx, transforming it to ColumnDataSource and then trying to render it using bokehjs will cause bokehjs to crash with an uncought exception -> "Uncaught TypeError: Cannot read property '0' of undefined".

Assigning node attributes to any other node will work, only root node will cause it to crash.
It is a regression and likely introduced by PR #8340. Specifically bokehjs/src/lib/core/util/serialization.ts:209

I am not really a frontend person so my typescript/javascript skills are lacking, but my guess is that 
if stmt on serialization.ts:200 protects every node except the root node.
```typescript
if (v.length == 0 || !(isObject(v[0]) || isArray(v[0]))) {
    return [v, []]
}
```
And then for the root node, as we are decoding every attribute:
```typescript
for (const obj of v) {
    const [arr, shape] = isArray(obj) ? decode_traverse_data(obj, buffers) :
                                        process_array(obj as NDArray, buffers);
    arrays.push(arr)
    shapes.push(shape)
  }
```
We fail to parse obj since its not an an Array, so we call process_array(..) which fails to identify it as anything parseable and returns 'undefined', which causes the [arr, shape] tuple assignment to fail.

***As mentioned, I am not really a frontend person and dont have the energy to setup build environment to do testing and PR, but for anyone with their environment already setup I assume this is a trivial fix.***

#### Complete, minimal, self-contained example code that reproduces the issue
Using the networkx integration example from the documentation, but adding some iterator to first node:
https://bokeh.pydata.org/en/latest/docs/user_guide/graph.html#networkx-integration

```python
import bokeh
from bokeh.io import show, output_notebook
from bokeh.plotting import figure
from bokeh.models.graphs import from_networkx
from bokeh.models import Plot, Range1d, Circle, HoverTool, BoxZoomTool, ResetTool
from bokeh.palettes import Spectral4
import networkx as nx

output_notebook()

# Prepare Data
G = nx.karate_club_graph()

# Adding iterator element to node 0 will cause exception
# Adding it to any other element will work
G.nodes[0]['some_attr'] = ['a1', 'a2', 'a3']

# Show with Bokeh
plot = Plot(plot_width=400, plot_height=400,
            x_range=Range1d(-1.1, 1.1), y_range=Range1d(-1.1, 1.1))
plot.title.text = "Graph Interaction Demonstration"

node_hover_tool = HoverTool(tooltips=[("index", "@index"), ("club", "@club")])
plot.add_tools(node_hover_tool, BoxZoomTool(), ResetTool())

graph_renderer = from_networkx(G, nx.spring_layout, scale=1, center=(0, 0))
graph_renderer.node_renderer.glyph = Circle(size=15, fill_color=Spectral4[0])
plot.renderers.append(graph_renderer)

show(plot)
```

### Stack traceback and/or browser JavaScript console output
```
Uncaught TypeError: Cannot read property '0' of undefined
    at v (VM55 bokeh-1.0.4.min.js:formatted:4572)
    at Object.u.decode_column_data (VM55 bokeh-1.0.4.min.js:formatted:4636)
    at l.initialize (VM55 bokeh-1.0.4.min.js:formatted:21071)
    at l.c.finalize (VM55 bokeh-1.0.4.min.js:formatted:1142)
    at VM55 bokeh-1.0.4.min.js:formatted:5899
    at _ (VM55 bokeh-1.0.4.min.js:formatted:5874)
    at _ (VM55 bokeh-1.0.4.min.js:formatted:5872)
    at _ (VM55 bokeh-1.0.4.min.js:formatted:5872)
    at _ (VM55 bokeh-1.0.4.min.js:formatted:5879)
    at _ (VM55 bokeh-1.0.4.min.js:formatted:5872)
```
In the screendump below, I believe the crash happens at bokehjs/src/lib/core/util/serialization.ts:209
where `v(s, e)` is `decode_traverse_data(..)` and `f(s, e)` is `process_array(..)`

![screenshot from 2019-02-22 11-11-28](https://user-images.githubusercontent.com/1316057/53235674-b3223780-3692-11e9-8d22-a015c0de619d.png)




Discussion:
@jderehag thank you for the detailed and thorough report
So the problem is not actually with setting any attribute on the root node, it's specifically with setting the attribute to be a *list*. E.g. the following code functions correctly:
```
G.nodes[0]['some_attr'] = 'foo' 
```
And the reason is because setting one node to a list breaks BokehJS's assumption that any array with a subarray will have all subarrays (i.e. be a full "2d" array). This assumption is true in all other circumstances that I can think of. I think it is simply that  case that we did not consider that node/edge attributes could be non-scalar. 

In more detail, the Python code for adding attributes does this:
```
for attr_key in node_attr_keys:
            node_dict[attr_key] = [node_attr[attr_key] if attr_key in node_attr.keys() else None
                                   for _, node_attr
                                   in graph.nodes(data=True)]
```
Which results in this on the JS side:
<img width="558" alt="screen shot 2019-02-22 at 09 44 21" src="https://user-images.githubusercontent.com/1078448/53260494-794b3400-3686-11e9-8f0a-638fce7e96d5.png">

To determine whether we are dealing with a "2d" array, BokehJS speculatively probes the first element of a column to see if it is a scalar or not, and thats what leads to trouble, as this is not a "2d" array. 

I'm not sure how best to handle this. One option would be to explicitly not support non-scalar node attributes, i.e. throwing an exception Python side with code like the OP. This would preserve the current assumption on the BokehJS side. It might also be possible to have `process_array` do something different/better with `null` inputs (or have the code that normally calls it do something better), at the expense of making the handling logic less uniform. 

cc @jsignell do you have any thoughts? 
@jderehag As a workaround in the mean time perhaps 
```
G.nodes[0]['some_attr'] = ",".join(data)
```
would work for you, if you are just using the data to drive a hover display
Sure, thats one possible approach. Alternatly simply removing those attributes..

As for the solution, I would lean towards skipping those unparseable attributes, with possibly a warning (not stopping or raising an exception).

In my case, the original graph is fairly large and definetly not uniform over the nodes, so forcing the user (i.e me) to clean the dataset just to be able to plot it does not feel really consistant with how you handle missing attributes in other cases. For instance for the hover tooltip, if you use an attribute that is missing on some nodes it simply prints "???" in those cases.
I don't have much familiarity with network graphing. Is it common to have non-scalar node attributes? I guess if it is not common or strange in some way to do that, then I'd lean towards specifying that attributes can't be iterables. 
Well, non-scalar values are definetly common (even the karate-club dataset which is the canonical example) contains strings as attributes. I wouldnt dare answer how common iterables are though.

But to me, it kind of doesnt matter. In networkx you **can** have any object as an attribute, therefore why should bokeh impose additional restrictions? Of course, not everything can be indexed and/or visualized, but you should atleast not crash.
So, it's also possible to "normalize" the graph data, i.e. if there are any non-scalar values, make the missing values empty lists instead of `None`. This works well and is pretty simple to implement. However it will fall over badly in cases where a an attribute *mixed* scalar and non-scalar data. I guess I would be amenable to punting on that case, but letting things work as long as *all* the values for a graph attr were lists/tuples (or all not lists/tuples)
@jderehag Here is a minimal diff that does what I describe above:
```
diff --git a/bokeh/models/graphs.py b/bokeh/models/graphs.py
index efd46c59e..b4a8ec877 100644
--- a/bokeh/models/graphs.py
+++ b/bokeh/models/graphs.py
@@ -103,9 +103,12 @@ def from_networkx(graph, layout_function, **kwargs):
         node_attr_keys = list(set(node_attr_keys))

         for attr_key in node_attr_keys:
-            node_dict[attr_key] = [node_attr[attr_key] if attr_key in node_attr.keys() else None
-                                   for _, node_attr
-                                   in graph.nodes(data=True)]
+            values = [node_attr[attr_key] if attr_key in node_attr.keys() else None
+                      for _, node_attr in graph.nodes(data=True)]
+
+            values = _handle_sublists(values)
+
+            node_dict[attr_key] = values

         if 'index' in node_attr_keys:
             from warnings import warn
@@ -121,9 +124,12 @@ def from_networkx(graph, layout_function, **kwargs):
         edge_attr_keys = list(set(edge_attr_keys))

         for attr_key in edge_attr_keys:
-            edge_dict[attr_key] = [edge_attr[attr_key] if attr_key in edge_attr.keys() else None
-                                   for _, _, edge_attr
-                                   in graph.edges(data=True)]
+            values = [edge_attr[attr_key] if attr_key in edge_attr.keys() else None
+                      for _, _, edge_attr in graph.edges(data=True)]
+
+            values = _handle_sublists(values)
+
+            edge_dict[attr_key] = values

         if 'start' in edge_attr_keys or 'end' in edge_attr_keys:
             from warnings import warn
@@ -200,6 +206,14 @@ class EdgesAndLinkedNodes(GraphHitTestPolicy):
 # Private API
 #-----------------------------------------------------------------------------

+def _handle_sublists(values):
+    # if any of the items is non-scalar, they all must be
+    if any(isinstance(x, (list, tuple)) for x in values):
+        if not all(isinstance(x, (list, tuple)) for x in values if x is not None):
+            raise RuntimeError("Can't mix")
+        return [[] if x is None else x for x in values]
+    return values
+
 #-----------------------------------------------------------------------------
 # Code
 #-----------------------------------------------------------------------------
``` 
Would you be interested in picking this up and running it to completion? It needs a few simple tests and a better exception message.  

With that, your code (updated to show `some_attr` in tooltip) works:

<img width="425" alt="screen shot 2019-03-07 at 10 47 15 am" src="https://user-images.githubusercontent.com/1078448/53981936-84f81b00-40c8-11e9-9073-708411220137.png">
<img width="423" alt="screen shot 2019-03-07 at 10 47 20 am" src="https://user-images.githubusercontent.com/1078448/53981955-89bccf00-40c8-11e9-99d8-0becbd294e9d.png">



Resolving patch:
diff --git a/bokeh/models/graphs.py b/bokeh/models/graphs.py
--- a/bokeh/models/graphs.py
+++ b/bokeh/models/graphs.py
@@ -74,6 +74,8 @@ def from_networkx(graph, layout_function, **kwargs):
         layout function. Any keyword arguments will be passed to the
         layout function.
 
+        Only two dimensional layouts are supported.
+
         Args:
             graph (networkx.Graph) : a networkx graph to render
             layout_function (function or dict) : a networkx layout function or mapping of node keys to positions.
@@ -82,13 +84,18 @@ def from_networkx(graph, layout_function, **kwargs):
         Returns:
             instance (GraphRenderer)
 
-        .. warning::
-            Only two dimensional layouts are currently supported.
+        .. note::
+            Node and edge attributes may be lists or tuples. However, a given
+            attribute must either have *all* lists or tuple values, or *all*
+            scalar values, for nodes or edges it is defined on.
 
         .. warning::
             Node attributes labeled 'index' and edge attributes labeled 'start' or 'end' are ignored.
             If you want to convert these attributes, please re-label them to other names.
 
+        Raises:
+            ValueError
+
         '''
 
         # inline import to prevent circular imports
@@ -103,9 +110,12 @@ def from_networkx(graph, layout_function, **kwargs):
         node_attr_keys = list(set(node_attr_keys))
 
         for attr_key in node_attr_keys:
-            node_dict[attr_key] = [node_attr[attr_key] if attr_key in node_attr.keys() else None
-                                   for _, node_attr
-                                   in graph.nodes(data=True)]
+            values = [node_attr[attr_key] if attr_key in node_attr.keys() else None
+                      for _, node_attr in graph.nodes(data=True)]
+
+            values = _handle_sublists(values)
+
+            node_dict[attr_key] = values
 
         if 'index' in node_attr_keys:
             from warnings import warn
@@ -121,9 +131,12 @@ def from_networkx(graph, layout_function, **kwargs):
         edge_attr_keys = list(set(edge_attr_keys))
 
         for attr_key in edge_attr_keys:
-            edge_dict[attr_key] = [edge_attr[attr_key] if attr_key in edge_attr.keys() else None
-                                   for _, _, edge_attr
-                                   in graph.edges(data=True)]
+            values = [edge_attr[attr_key] if attr_key in edge_attr.keys() else None
+                      for _, _, edge_attr in graph.edges(data=True)]
+
+            values = _handle_sublists(values)
+
+            edge_dict[attr_key] = values
 
         if 'start' in edge_attr_keys or 'end' in edge_attr_keys:
             from warnings import warn
@@ -200,6 +213,14 @@ class EdgesAndLinkedNodes(GraphHitTestPolicy):
 # Private API
 #-----------------------------------------------------------------------------
 
+def _handle_sublists(values):
+    # if any of the items is non-scalar, they all must be
+    if any(isinstance(x, (list, tuple)) for x in values):
+        if not all(isinstance(x, (list, tuple)) for x in values if x is not None):
+            raise ValueError("Can't mix scalar and non-scalar values for graph attributes")
+        return [[] if x is None else list(x) for x in values]
+    return values
+
 #-----------------------------------------------------------------------------
 # Code
 #-----------------------------------------------------------------------------


Test patch:
diff --git a/bokeh/models/tests/test_graphs.py b/bokeh/models/tests/test_graphs.py
--- a/bokeh/models/tests/test_graphs.py
+++ b/bokeh/models/tests/test_graphs.py
@@ -65,7 +65,7 @@ def test_from_networkx_method_with_kwargs():
     assert set(gl.keys()) == set([0,1,2,3])
     assert_allclose(gl[0], np.array([2., 0.]), atol=1e-7)
 
-def test_from_networkx_with_attributes():
+def test_from_networkx_with_scalar_attributes():
     G = nx.Graph()
     G.add_nodes_from([(0, {"attr_1": "a", "attr_2": 10}),
                       (1, {"attr_1": "b"}),
@@ -84,6 +84,42 @@ def test_from_networkx_with_attributes():
     assert renderer.edge_renderer.data_source.data["attr_1"] == ["A", "B"]
     assert renderer.edge_renderer.data_source.data["attr_2"] == [None, 10]
 
+@pytest.mark.parametrize('typ', [list, tuple])
+def test_from_networkx_with_sequence_attributes(typ):
+    G = nx.Graph()
+    G.add_nodes_from([(0, {"attr_1": typ([1, 2]), "attr_2": 10}),
+                      (1, {}),
+                      (2, {"attr_1": typ([3]), "attr_2": 30})])
+    G.add_edges_from([(0, 1, {"attr_1": typ([1, 11])}),
+                      (0, 2, {"attr_1": typ([2, 22]), "attr_2": 10})])
+
+    renderer = from_networkx(G, nx.circular_layout)
+
+    assert renderer.node_renderer.data_source.data["index"] == [0, 1, 2]
+    assert renderer.node_renderer.data_source.data["attr_1"] == [[1, 2], [], [3]]
+    assert renderer.node_renderer.data_source.data["attr_2"] == [10, None, 30]
+
+    assert renderer.edge_renderer.data_source.data["start"] == [0, 0]
+    assert renderer.edge_renderer.data_source.data["end"] == [1, 2]
+    assert renderer.edge_renderer.data_source.data["attr_1"] == [[1, 11], [2, 22]]
+    assert renderer.edge_renderer.data_source.data["attr_2"] == [None, 10]
+
+def test_from_networkx_errors_with_mixed_attributes():
+    G = nx.Graph()
+    G.add_nodes_from([(0, {"attr_1": [1, 2], "attr_2": 10}),
+                      (1, {}),
+                      (2, {"attr_1": 3, "attr_2": 30})])
+
+    with pytest.raises(ValueError):
+        from_networkx(G, nx.circular_layout)
+
+    G = nx.Graph()
+    G.add_edges_from([(0, 1, {"attr_1": [1, 11]}),
+                      (0, 2, {"attr_1": 2, "attr_2": 10})])
+
+    with pytest.raises(ValueError):
+        from_networkx(G, nx.circular_layout)
+
 def test_from_networkx_with_bad_attributes():
     G = nx.Graph()
     G.add_nodes_from([(0, {"index": "a", "attr_1": 10}),