Repository: pandas-dev/pandas

Issue:
merge_asof(): cannot use tolerance flag when the index is a TimedeltaIndex
#### Code Sample

```python
import pandas as pd 
import numpy as np

print(
    """
    \nPandas merge_asof() bug:
    
    \tUnimplemented error?
    \tcannot use tolerance flag when my index is a timedelta (not a timestamp)
    \tjust documenting so I can try to add this functionality
        
    """)

print(f"pandas version: {pd.__version__}")
print(f"numpy version: {np.__version__}")

delta_300 = pd.timedelta_range(start='0 minutes', freq='3333334 N', periods=301, name='Time')
delta_120 = pd.timedelta_range(start='0 minutes', freq='8333334 N', periods=301, name='Time')

df_300hz = pd.DataFrame({'my300hz_data': np.arange(301)}, index=delta_300)
df_120hz = pd.DataFrame({'my120hz_data': np.arange(301)}, index=delta_120)

print(df_300hz)
print(df_120hz) 

### this throws error "pandas.errors.MergeError: key must be integer, timestamp or float"
merged = pd.merge_asof(df_120hz, df_300hz, on='Time', direction='nearest', tolerance=pd.Timedelta('15 ms'))

### The line below works, but output is not what I want
# merged = pd.merge_asof(df_120hz, df_300hz, on='Time', direction='nearest')

merged.set_index('Time', inplace=True)
print(merged)
```

#### Problem description

I need to see NaNs when I merge and there is a gap in my data, without begin able to use the tolerance flag my data gets smoothed.

Current work around, convert all my TimedeltaIndex's to a time stamp. Since I do not have a date for this data I am using unix time. This feels bulky since I am going to drop the date when I save the file anyway.

#### Expected Output

[301 rows x 1 columns]
                 my120hz_data  my300hz_data
Time                                       
00:00:00                    0             0
00:00:00.008333             1             2
00:00:00.016666             2             5
00:00:00.025000             3             7
00:00:00.033333             4            10
...                       ...           ...
00:00:02.466666           296           NaN
00:00:02.475000           297           NaN
00:00:02.483333           298           NaN
00:00:02.491666           299           NaN
00:00:02.500000           300           NaN

#### Output of ``pd.show_versions()``

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.3.final.0
python-bits      : 64
OS               : Linux
OS-release       : 4.19.0-5-amd64
machine          : x86_64
processor        : 
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 0.25.0
numpy            : 1.17.0
pytz             : 2019.1
dateutil         : 2.8.0
pip              : 18.1
setuptools       : 40.8.0
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : None
scipy            : 1.3.0
sqlalchemy       : None
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None


Discussion:
`merge_asof` should have raised a `NotImplementedError` with `TimedeltaIndex`, but I think it's reasonable to support it for that data type. 
I think adding an elif statement to handle timedeltaIndex then verify tolerance is positive gives the result I desire.

```
            elif is_timedelta64_dtype(lt):
                if not isinstance(self.tolerance, Timedelta):
                    raise MergeError(msg)
                if self.tolerance < Timedelta(0):
                    raise MergeError("tolerance must be positive")
```

line 1665 merge.py

This does give the results I desire, so not unimplemented?
I haven't tried to break it though.

also add "is_timedelta64_dtype" to `from pandas.core.dtypes.common import ()`

or use `is_datetimelike()`  or `is_datetime_or_timedelta_dtype()` for if check?
Still works after I add gaps to my 300Hz data.

```
import pandas as pd 
import numpy as np

print(
    """
    \nPandas merge_asof() bug:
    
    \tUnimplemented error?
    \tcannot use tolerance flag when my index is a timedelta (not a timestamp)
    \tjust documenting so I can try to add this functionality
        
    """)

print(f"pandas version: {pd.show_versions()}")
print(f"numpy version: {np.__version__}")

delta_300 = pd.timedelta_range(start='0 minutes', freq='3333334 N', periods=301, name='Time')
delta_120 = pd.timedelta_range(start='0 minutes', freq='8333334 N', periods=301, name='Time')

df_300hz = pd.DataFrame({'my300hz_data': np.arange(301)}, index=delta_300)
df_120hz = pd.DataFrame({'my120hz_data': np.arange(301)}, index=delta_120)

df_300hz.drop(df_300hz.index[10:110], inplace=True)

print(df_300hz)
print(df_120hz) 

merged = pd.merge_asof(df_120hz, df_300hz, on='Time', direction='nearest', tolerance=pd.Timedelta('5 ms'))

merged.set_index('Time', inplace=True)
print(merged.to_string())
```
Right, you can just add the `is_timedelta64_dtype ` check here:

https://github.com/pandas-dev/pandas/blob/0fd888c8b318d4041b0ae53c64715eca0d345dd3/pandas/core/reshape/merge.py#L1638

PR's welcome!

Resolving patch:
diff --git a/pandas/core/reshape/merge.py b/pandas/core/reshape/merge.py
--- a/pandas/core/reshape/merge.py
+++ b/pandas/core/reshape/merge.py
@@ -22,7 +22,6 @@
     is_bool,
     is_bool_dtype,
     is_categorical_dtype,
-    is_datetime64_dtype,
     is_datetime64tz_dtype,
     is_datetimelike,
     is_dtype_equal,
@@ -1635,7 +1634,7 @@ def _get_merge_keys(self):
                 )
             )
 
-            if is_datetime64_dtype(lt) or is_datetime64tz_dtype(lt):
+            if is_datetimelike(lt):
                 if not isinstance(self.tolerance, Timedelta):
                     raise MergeError(msg)
                 if self.tolerance < Timedelta(0):


Test patch:
diff --git a/pandas/tests/reshape/merge/test_merge_asof.py b/pandas/tests/reshape/merge/test_merge_asof.py
--- a/pandas/tests/reshape/merge/test_merge_asof.py
+++ b/pandas/tests/reshape/merge/test_merge_asof.py
@@ -1,3 +1,5 @@
+import datetime
+
 import numpy as np
 import pytest
 import pytz
@@ -588,14 +590,23 @@ def test_non_sorted(self):
         # ok, though has dupes
         merge_asof(trades, self.quotes, on="time", by="ticker")
 
-    def test_tolerance(self):
+    @pytest.mark.parametrize(
+        "tolerance",
+        [
+            Timedelta("1day"),
+            pytest.param(
+                datetime.timedelta(days=1),
+                marks=pytest.mark.xfail(reason="not implemented", strict=True),
+            ),
+        ],
+        ids=["pd.Timedelta", "datetime.timedelta"],
+    )
+    def test_tolerance(self, tolerance):
 
         trades = self.trades
         quotes = self.quotes
 
-        result = merge_asof(
-            trades, quotes, on="time", by="ticker", tolerance=Timedelta("1day")
-        )
+        result = merge_asof(trades, quotes, on="time", by="ticker", tolerance=tolerance)
         expected = self.tolerance
         assert_frame_equal(result, expected)
 
@@ -1246,3 +1257,39 @@ def test_by_mixed_tz_aware(self):
         )
         expected["value_y"] = np.array([np.nan], dtype=object)
         assert_frame_equal(result, expected)
+
+    def test_timedelta_tolerance_nearest(self):
+        # GH 27642
+
+        left = pd.DataFrame(
+            list(zip([0, 5, 10, 15, 20, 25], [0, 1, 2, 3, 4, 5])),
+            columns=["time", "left"],
+        )
+
+        left["time"] = pd.to_timedelta(left["time"], "ms")
+
+        right = pd.DataFrame(
+            list(zip([0, 3, 9, 12, 15, 18], [0, 1, 2, 3, 4, 5])),
+            columns=["time", "right"],
+        )
+
+        right["time"] = pd.to_timedelta(right["time"], "ms")
+
+        expected = pd.DataFrame(
+            list(
+                zip(
+                    [0, 5, 10, 15, 20, 25],
+                    [0, 1, 2, 3, 4, 5],
+                    [0, np.nan, 2, 4, np.nan, np.nan],
+                )
+            ),
+            columns=["time", "left", "right"],
+        )
+
+        expected["time"] = pd.to_timedelta(expected["time"], "ms")
+
+        result = pd.merge_asof(
+            left, right, on="time", tolerance=Timedelta("1ms"), direction="nearest"
+        )
+
+        assert_frame_equal(result, expected)