Repository: bokeh/bokeh

Issue:
bokeh TimeSeries chokes on NaN values
TimeSeries generates an Exception if presented a TimeSeries containing a NaN at position 0. NaNs elsewhere are tolerated.

See the following snippet extracted/adapted from my code
# -Python code

import pandas as pd, numpy as np
from bokeh.plotting import figure, show
from bokeh.io import output_notebook
from bokeh.charts import TimeSeries

Series=pd.Series(index=idx,data=np.random.randn(1000)).cumsum()
p=TimeSeries(Series, index=idx)
# -That works nicely

show(p)
# -now let's add a NaN in the middle

Series[100]=np.nan
p=TimeSeries(Series, index=idx)
# -That works nicely too

show(p)
# -now let's add a NaN in the beginning

Series[0]=np.nan
p=TimeSeries(Series, index=idx)
# -Kaboom

show(p)



Discussion:
Bokeh 0.9.2, numpy 1.9.2, pandas 0.16.2

This seems to come down to the fact that the charts are constructing a y-range `Range1d` by taking the min and max of the y-values, and `min([np.nan, ..]) == max([np.nan, ..]) == np.nan` (and only when `nan` is the first entry in the list. edit: you know what i mean by nan equality :smile:). It affects most of the charts, for example:

``` python
from bokeh.plotting import show, figure
from bokeh.charts import Scatter
show(Scatter({"a": [(1.0, np.nan), (2.0, 2.0)]}))
```

The lower-level glyph plotting functions seem not to be affected by this.

Probably the easiest way to fix this is to use `np.nanmin/np.nanmax` everywhere in these range constructions instead of `min/max`.

@birdsarah @bryevdv Based on [BEP-3:Charts-interface](https://github.com/bokeh/bokeh/wiki/BEP-3:-Charts-interface) and issues like https://github.com/bokeh/bokeh/issues/2288, should this wait for refactors or should I go ahead and submit a hotfix PR? My thought would be to wrap the `Range1d(start=min(min() for ..) - 0.1 * , end=max(max() for ..) + 0.1 * )` calls in `charts` into a static constructor for `Range1d`.

Also in general it'd be nice to have fail-immediately validation that a `bokeh.properties.Float` cannot be `np.nan`, given that it later can't be serialized to json? There seem to be plenty of comments around in favor of more informative and direct error messages for issues like this.

pinging @fpliger @rothnic who are working in chart refactor...

This is something that should really be fixed on the new charts re-work.. 

@azjps I'm +1 if you want to submit a hotfix PR. It shouldn't take too much time and it's a valuable fix anyway.. :+1: 

Also, yes being more informative around validation is definitely a priority. :-)

We need to figure out `NaN` in general. I guess I have lost the thread of things recently. We used to be able to handle NaNs at least in some (possibly "nonstandard" but seemingly commonly supported)  way. Is this no longer the case? We have to be able to deal with them, though. If the only robust way is to ship some kind of sidecar "masked array", or the indices of `NaN` values explicitly, we can do that. The upcoming column data store on the new document structure is probably a good time/place to re-visit things. I am already planning on associating more metadata (type, and shape, e.g.) with each column, we could add other optional things like `NaN`  or `NA` masks as well. Inching closer to a "JS dataframe"...

> We need to figure out NaN in general.

I completely agree, although I think the thing discussed specifically here could be solved with a quick Pr until a more encompassing resolution of the general problem is in place...


Resolving patch:
diff --git a/bokeh/charts/glyphs.py b/bokeh/charts/glyphs.py
--- a/bokeh/charts/glyphs.py
+++ b/bokeh/charts/glyphs.py
@@ -74,19 +74,37 @@ def _build_label_array(self, props, value):
 
     @property
     def x_max(self):
-        return max(self.source.data['x_values'])
+        # TODO(fpliger): since CompositeGlyphs are not exposed in general we
+        #                should expect to always have a Series but in case
+        #                it's not we just use the default min/max instead
+        #                of just failing. When/If we end up exposing
+        #                CompositeGlyphs we should consider making this
+        #                more robust (either enforcing data or checking)
+        try:
+            return self.source.data['x_values'].max()
+        except AttributeError:
+            return max(self.source.data['x_values'])
 
     @property
     def x_min(self):
-        return min(self.source.data['x_values'])
+        try:
+            return self.source.data['x_values'].min()
+        except AttributeError:
+            return min(self.source.data['x_values'])
 
     @property
     def y_max(self):
-        return max(self.source.data['y_values'])
+        try:
+            return self.source.data['y_values'].max()
+        except AttributeError:
+            return max(self.source.data['y_values'])
 
     @property
     def y_min(self):
-        return min(self.source.data['y_values'])
+        try:
+            return self.source.data['y_values'].min()
+        except AttributeError:
+            return min(self.source.data['y_values'])
 
 
 class PointGlyph(XyGlyph):


Test patch:
diff --git a/bokeh/charts/tests/test_comp_glyphs.py b/bokeh/charts/tests/test_comp_glyphs.py
--- a/bokeh/charts/tests/test_comp_glyphs.py
+++ b/bokeh/charts/tests/test_comp_glyphs.py
@@ -1,5 +1,6 @@
-from bokeh.charts.glyphs import AreaGlyph
+import numpy as np
 import pandas as pd
+from bokeh.charts.glyphs import AreaGlyph, LineGlyph, PointGlyph, StepGlyph
 
 
 def test_area_base_values(test_data):
@@ -10,4 +11,42 @@ def test_area_base_values(test_data):
     ag = AreaGlyph(x=x, y=y)
 
     assert ag.source.data['y_values'][0][0] == 0
-    assert ag.source.data['y_values'][0][-1] == 0
\ No newline at end of file
+    assert ag.source.data['y_values'][0][-1] == 0
+
+
+def test_xyglyph_xy_range():
+    def check_bounds(xyg, xmin=0, xmax=4, ymin=1, ymax=5):
+        assert xyg.x_min == xmin
+        assert xyg.x_max == xmax
+        assert xyg.y_min == ymin
+        assert xyg.y_max == ymax
+
+    for Glyph in [LineGlyph, PointGlyph]:
+        x = pd.Series([0, 1, 2, 3, 4])
+        y = pd.Series([5, 4, 3, 2, 1])
+        xyg = Glyph(x=x, y=y)
+        check_bounds(xyg)
+
+        x[1] = x[2] = np.nan
+        xyg = Glyph(x=x, y=y)
+        check_bounds(xyg)
+
+        x[0] = np.nan
+        xyg = Glyph(x=x, y=y)
+        check_bounds(xyg, xmin=3)
+
+        x[4] = np.nan
+        xyg = Glyph(x=x, y=y)
+        check_bounds(xyg, xmin=3, xmax=3)
+
+        y[1] = y[2] = np.nan
+        xyg = Glyph(x=x, y=y)
+        check_bounds(xyg, xmin=3, xmax=3)
+
+        y[0] = np.nan
+        xyg = Glyph(x=x, y=y)
+        check_bounds(xyg, xmin=3, xmax=3, ymax=2)
+
+        y[4] = np.nan
+        xyg = Glyph(x=x, y=y)
+        check_bounds(xyg, xmin=3, xmax=3, ymax=2, ymin=2)