Repository: bokeh/bokeh

Issue:
ValueError: NaTType does not support timetuple
See http://stackoverflow.com/questions/18212247/convert-text-dates-to-dates-and-then-keeping-the-na-values
See https://github.com/pydata/pandas/issues/9513
See https://github.com/pydata/pandas/issues/12976

bokeh 0.11.1
pandas 0.18.0

pandas uses "pseudo-native" NaT for missing datetime64 values. These can't be jsonified at the moment. Arguably it's a `pandas` compatibility issue, but something that can be easily addressed.

core\json_encoder.py: 44

For desired features, please provide:
- Treat them as None or NaN

Please assign this issue to me.

```
In [78]: isinstance(pd.Timestamp(np.nan), datetime.datetime)
Out[78]: True

In [80]: pd.Timestamp('2015-04-16').timetuple()
Out[80]: time.struct_time(tm_year=2015, tm_mon=4, tm_mday=16, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=106, tm_isdst=-1)

In [81]: pd.Timestamp(np.nan).timetuple()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-81-bf30011ea8d1> in <module>()
----> 1 pd.Timestamp(np.nan).timetuple()

pandas\tslib.pyx in pandas.tslib._make_error_func.f (pandas\tslib.c:14736)()

ValueError: NaTType does not support timetuple
```



Discussion:
@rs2 thanks for the issue, I can only assign people from @bokeh/dev but certainly if you would like to work on this a PR would be very appreciated

Has this been addressed? I'm coming across the same issue:

"ValueError: NaTType does not support timetuple".

How do I get around this issue? Is there an option to ignore NaT's?
@eddy-ojb Yes, just filter `pd.NaT` values using `pd.isnull`.

Resolving patch:
diff --git a/bokeh/util/serialization.py b/bokeh/util/serialization.py
--- a/bokeh/util/serialization.py
+++ b/bokeh/util/serialization.py
@@ -53,6 +53,7 @@
     DATETIME_TYPES.add(_pd_timestamp)
     DATETIME_TYPES.add(pd.Timedelta)
     DATETIME_TYPES.add(pd.Period)
+    DATETIME_TYPES.add(type(pd.NaT))
 
 NP_EPOCH = np.datetime64(0, 'ms')
 NP_MS_DELTA = np.timedelta64(1, 'ms')
@@ -117,6 +118,10 @@ def convert_datetime_type(obj):
         float : milliseconds
 
     '''
+    # Pandas NaT
+    if pd and obj is pd.NaT:
+        return np.nan
+
     # Pandas Period
     if pd and isinstance(obj, pd.Period):
         return obj.to_timestamp().value / 10**6.0


Test patch:
diff --git a/bokeh/util/tests/test_serialization.py b/bokeh/util/tests/test_serialization.py
--- a/bokeh/util/tests/test_serialization.py
+++ b/bokeh/util/tests/test_serialization.py
@@ -43,7 +43,7 @@ def test_datetime_types(pd):
     if pd is None:
         assert len(bus.DATETIME_TYPES) == 4
     else:
-        assert len(bus.DATETIME_TYPES) == 7
+        assert len(bus.DATETIME_TYPES) == 8
 
 def test_is_timedelta_type_non_pandas_types():
     assert bus.is_timedelta_type(datetime.timedelta(3000))
@@ -68,6 +68,7 @@ def test_is_datetime_type_non_pandas_types():
 def test_is_datetime_type_pandas_types(pd):
     assert bus.is_datetime_type(bus._pd_timestamp(3000000))
     assert bus.is_datetime_type(pd.Period('1900', 'A-DEC'))
+    assert bus.is_datetime_type(pd.NaT)
 
 def test_convert_datetime_type_non_pandas_types():
     assert bus.convert_datetime_type(datetime.datetime(2018, 1, 3, 15, 37, 59, 922452)) == 1514993879922.452
@@ -81,6 +82,7 @@ def test_convert_datetime_type_pandas_types(pd):
     assert bus.convert_datetime_type(bus._pd_timestamp(3000000)) == 3.0
     assert bus.convert_datetime_type(pd.Period('1900', 'A-DEC')) == -2208988800000.0
     assert bus.convert_datetime_type(pd.Period('1900', 'A-DEC')) == bus.convert_datetime_type(np.datetime64("1900-01-01"))
+    assert np.isnan(bus.convert_datetime_type(pd.NaT))
 
 @pytest.mark.parametrize('obj', [[1,2], (1,2), dict(), set(), 10.2, "foo"])
 @pytest.mark.unit