Repository: pandas-dev/pandas
Pull request: implement truediv, rtruediv directly in TimedeltaArray; tests
Follow-up to #23642, implements `__rdiv__` in TimedeltaArray, plus missing tests.

After this will be a PR that does the same for floordiv+rfloordiv.

- [x] closes #22631
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Fixes:
`tdi / np.timedelta64('NaT')`
` tdi / tdi` (#22631)
`tdi / object_dtyped`
`object_dtyped / tdi`

Problem statement:
BUG: TDI / TDI should work
```
In [1]: pd.__version__
Out[1]: '0.20.3'

In [2]: i = pd.timedelta_range('1 day', periods=3)

In [3]: i / i
TypeError: cannot evaluate a numeric op with a non-numeric dtype
```

master
```
In [1]: pd.__version__
Out[1]: '0.24.0.dev0+550.g70c9003c9'

In [2]: i = pd.timedelta_range('1 day', periods=3)

In [3]: i / i
TypeError: unsupported operand type(s) for /: 'TimedeltaIndex' and 'TimedeltaIndex'

In [4]: i / 2
Out[4]: TimedeltaIndex(['0 days 12:00:00', '1 days 00:00:00', '1 days 12:00:00'], dtype='timedelta64[ns]', freq='12H')

In [5]: i / pd.Timedelta('1 day')
Out[5]: Float64Index([1.0, 2.0, 3.0], dtype='float64')

# conversion then op works
In [8]: i.astype('i8') / i.astype('i8')
Out[8]: Float64Index([1.0, 1.0, 1.0], dtype='float64')


```

Hints from issue discussion:
cc @jbrockmendel 

I really thought this DID work in 0.20.3, though my example is not reproducible ATM.

of course this works for a scalar TD or numeric on the rhs.

Is there a reason we didn't allow TDI / TDI though?
cc @joshuastorck 
> Is there a reason we didn't allow TDI / TDI though?

Not that I can think of.  In trying this myself found #22632.

Commit message: more checks and test cases

Patch to review:
diff --git a/pandas/core/arrays/timedeltas.py b/pandas/core/arrays/timedeltas.py
index d1e6d979b554c..58b1f929b0774 100644
--- a/pandas/core/arrays/timedeltas.py
+++ b/pandas/core/arrays/timedeltas.py
@@ -7,7 +7,7 @@
 
 import numpy as np
 
-from pandas._libs import tslibs
+from pandas._libs import lib, tslibs
 from pandas._libs.tslibs import NaT, Timedelta, Timestamp, iNaT
 from pandas._libs.tslibs.fields import get_timedelta_field
 from pandas._libs.tslibs.timedeltas import (
@@ -165,6 +165,7 @@ def _simple_new(cls, values, freq=None, dtype=_TD_DTYPE):
     def __new__(cls, values, freq=None, dtype=_TD_DTYPE):
 
         freq, freq_infer = dtl.maybe_infer_freq(freq)
+        values, inferred_freq = sequence_to_td64ns(values)
 
         values = np.array(values, copy=False)
         if values.dtype == np.object_:
@@ -324,12 +325,106 @@ def _evaluate_with_timedelta_like(self, other, op):
 
     __mul__ = _wrap_tdi_op(operator.mul)
     __rmul__ = __mul__
-    __truediv__ = _wrap_tdi_op(operator.truediv)
     __floordiv__ = _wrap_tdi_op(operator.floordiv)
     __rfloordiv__ = _wrap_tdi_op(ops.rfloordiv)
 
+    def __truediv__(self, other):
+        # TODO: Decimals?
+        other = lib.item_from_zerodim(other)
+
+        if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
+            return NotImplemented
+
+        if isinstance(other, (timedelta, np.timedelta64, Tick)):
+            other = Timedelta(other)
+            if other is NaT:
+                # specifically timedelta64-NaT
+                result = np.empty(self.shape, dtype=np.float64)
+                result.fill(np.nan)
+                return result
+
+            # otherwise, dispatch to Timedelta implementation
+            return self._data / other
+
+        elif lib.is_scalar(other):
+            # assume it is numeric
+            result = self._data / other
+            freq = None
+            if self.freq is not None:
+                # Tick division is not implemented, so operate on Timedelta
+                freq = self.freq.delta / other
+            return type(self)(result, freq=freq)
+
+        if not hasattr(other, "dtype"):
+            # e.g. list, tuple
+            other = np.array(other)
+
+        if len(other) != len(self):
+            raise ValueError("Cannot divide vectors with unequal lengths")
+
+        elif is_timedelta64_dtype(other):
+            # let numpy handle it
+            return self._data / other
+
+        elif is_object_dtype(other):
+            result = [self[n] / other[n] for n in range(len(self))]
+            result = np.array(result)
+            if lib.infer_dtype(result) == 'timedelta':
+                # TODO: keep inferred freq?
+                result, _ = sequence_to_td64ns(result)
+                return type(self)(result)
+
+            return np.array(result)
+
+        else:
+            result = self._data / other
+            return type(self)(result)
+
+    def __rtruediv__(self, other):
+        other = lib.item_from_zerodim(other)
+
+        if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
+            return NotImplemented
+
+        if isinstance(other, (timedelta, np.timedelta64, Tick)):
+            other = Timedelta(other)
+            if other is NaT:
+                # specifically timedelta64-NaT
+                result = np.empty(self.shape, dtype=np.float64)
+                result.fill(np.nan)
+                return result
+
+            # otherwise, dispatch to Timedelta implementation
+            return other / self._data
+
+        elif lib.is_scalar(other):
+            raise TypeError("Cannot divide {typ} by {cls}"
+                            .format(typ=type(other).__name__,
+                                    cls=type(self).__name__))
+
+        if not hasattr(other, "dtype"):
+            # e.g. list, tuple
+            other = np.array(other)
+
+        if len(other) != len(self):
+            raise ValueError("Cannot divide vectors with unequal lengths")
+
+        elif is_timedelta64_dtype(other):
+            # let numpy handle it
+            return other / self._data
+
+        elif is_object_dtype(other):
+            result = [other[n] / self[n] for n in range(len(self))]
+            return np.array(result)
+
+        else:
+            # object-dtype *may* be OK here, but generally this will raise
+            result = other / self._data
+            return type(self)(result)
+
     if compat.PY2:
         __div__ = __truediv__
+        __rdiv__ = __rtruediv__
 
     # Note: TimedeltaIndex overrides this in call to cls._add_numeric_methods
     def __neg__(self):
diff --git a/pandas/core/indexes/timedeltas.py b/pandas/core/indexes/timedeltas.py
index 8f50b40a20738..ddf49271fada2 100644
--- a/pandas/core/indexes/timedeltas.py
+++ b/pandas/core/indexes/timedeltas.py
@@ -229,11 +229,8 @@ def _format_native_types(self, na_rep=u'NaT', date_format=None, **kwargs):
 
     __mul__ = Index.__mul__
     __rmul__ = Index.__rmul__
-    __truediv__ = Index.__truediv__
     __floordiv__ = Index.__floordiv__
     __rfloordiv__ = Index.__rfloordiv__
-    if compat.PY2:
-        __div__ = Index.__div__
 
     days = wrap_field_accessor(TimedeltaArray.days)
     seconds = wrap_field_accessor(TimedeltaArray.seconds)
@@ -242,6 +239,38 @@ def _format_native_types(self, na_rep=u'NaT', date_format=None, **kwargs):
 
     total_seconds = wrap_array_method(TimedeltaArray.total_seconds, True)
 
+    @classmethod
+    def _override_div_mod_methods(cls):
+        """
+        Define __div__, __mod__, etc by wrapping TimedeltaArray implementations
+        rather than the versions defined by _add_numeric_methods.  Because
+        _add_numeric_methods is called after class creation, we have to define
+        these methods even later.
+        """
+
+        def __truediv__(self, other):
+            oth = other
+            if isinstance(other, Index):
+                # TimedeltaArray defers, so we need to unwrap
+                oth = other._values
+            result = TimedeltaArray.__truediv__(self, oth)
+            return wrap_arithmetic_op(self, other, result)
+
+        def __rtruediv__(self, other):
+            oth = other
+            if isinstance(other, Index):
+                # TimedeltaArray defers, so we need to unwrap
+                oth = other._values
+            result = TimedeltaArray.__rtruediv__(self, oth)
+            return wrap_arithmetic_op(self, other, result)
+
+        cls.__truediv__ = __truediv__
+        cls.__rtruediv__ = __rtruediv__
+
+        if compat.PY2:
+            cls.__div__ = __truediv__
+            cls.__rdiv__ = __rtruediv__
+
     # -------------------------------------------------------------------
 
     @Appender(_index_shared_docs['astype'])
@@ -642,6 +671,7 @@ def delete(self, loc):
 TimedeltaIndex._add_numeric_methods()
 TimedeltaIndex._add_logical_methods_disabled()
 TimedeltaIndex._add_datetimelike_methods()
+TimedeltaIndex._override_div_mod_methods()
 
 
 def _is_convertible_to_index(other):
diff --git a/pandas/tests/arithmetic/test_timedelta64.py b/pandas/tests/arithmetic/test_timedelta64.py
index 58c7216f0eece..cfafe4c7c1d8d 100644
--- a/pandas/tests/arithmetic/test_timedelta64.py
+++ b/pandas/tests/arithmetic/test_timedelta64.py
@@ -1115,7 +1115,7 @@ def test_tdi_rmul_arraylike(self, other, box_with_array):
         tm.assert_equal(commute, expected)
 
     # ------------------------------------------------------------------
-    # __div__
+    # __div__, __rdiv__
 
     def test_td64arr_div_nat_invalid(self, box_with_array):
         # don't allow division by NaT (maybe could in the future)
@@ -1123,6 +1123,23 @@ def test_td64arr_div_nat_invalid(self, box_with_array):
         rng = tm.box_expected(rng, box_with_array)
         with pytest.raises(TypeError):
             rng / pd.NaT
+        with pytest.raises(TypeError):
+            pd.NaT / rng
+
+    def test_td64arr_div_td64nat(self, box_with_array):
+        rng = timedelta_range('1 days', '10 days',)
+        rng = tm.box_expected(rng, box_with_array)
+
+        other = np.timedelta64('NaT')
+
+        expected = np.array([np.nan] * 10)
+        expected = tm.box_expected(expected, box_with_array)
+
+        result = rng / other
+        tm.assert_equal(result, expected)
+
+        result = other / rng
+        tm.assert_equal(result, expected)
 
     def test_td64arr_div_int(self, box_with_array):
         idx = TimedeltaIndex(np.arange(5, dtype='int64'))
@@ -1131,7 +1148,10 @@ def test_td64arr_div_int(self, box_with_array):
         result = idx / 1
         tm.assert_equal(result, idx)
 
-    def test_tdi_div_tdlike_scalar(self, two_hours, box_with_array):
+        with pytest.raises(TypeError):
+            1 / idx
+
+    def test_td64arr_div_tdlike_scalar(self, two_hours, box_with_array):
         # GH#20088, GH#22163 ensure DataFrame returns correct dtype
         rng = timedelta_range('1 days', '10 days', name='foo')
         expected = pd.Float64Index((np.arange(10) + 1) * 12, name='foo')
@@ -1142,7 +1162,12 @@ def test_tdi_div_tdlike_scalar(self, two_hours, box_with_array):
         result = rng / two_hours
         tm.assert_equal(result, expected)
 
-    def test_tdi_div_tdlike_scalar_with_nat(self, two_hours, box_with_array):
+        result = two_hours / rng
+        expected = 1 / expected
+        tm.assert_equal(result, expected)
+
+    def test_td64arr_div_tdlike_scalar_with_nat(self, two_hours,
+                                                box_with_array):
         rng = TimedeltaIndex(['1 days', pd.NaT, '2 days'], name='foo')
         expected = pd.Float64Index([12, np.nan, 24], name='foo')
 
@@ -1152,6 +1177,58 @@ def test_tdi_div_tdlike_scalar_with_nat(self, two_hours, box_with_array):
         result = rng / two_hours
         tm.assert_equal(result, expected)
 
+        result = two_hours / rng
+        expected = 1 / expected
+        tm.assert_equal(result, expected)
+
+    def test_td64arr_div_td64_ndarray(self, box_with_array):
+        # GH#22631
+        rng = TimedeltaIndex(['1 days', pd.NaT, '2 days'])
+        expected = pd.Float64Index([12, np.nan, 24])
+
+        rng = tm.box_expected(rng, box_with_array)
+        expected = tm.box_expected(expected, box_with_array)
+
+        other = np.array([2, 4, 2], dtype='m8[h]')
+        result = rng / other
+        tm.assert_equal(result, expected)
+
+        result = rng / tm.box_expected(other, box_with_array)
+        tm.assert_equal(result, expected)
+
+        result = rng / other.astype(object)
+        tm.assert_equal(result, expected)
+
+        result = rng / list(other)
+        tm.assert_equal(result, expected)
+
+        # reversed op
+        expected = 1 / expected
+        result = other / rng
+        tm.assert_equal(result, expected)
+
+        result = tm.box_expected(other, box_with_array) / rng
+        tm.assert_equal(result, expected)
+
+        result = other.astype(object) / rng
+        tm.assert_equal(result, expected)
+
+        result = list(other) / rng
+        tm.assert_equal(result, expected)
+
+    def test_tdarr_div_length_mismatch(self, box_with_array):
+        rng = TimedeltaIndex(['1 days', pd.NaT, '2 days'])
+        mismatched = [1, 2, 3, 4]
+
+        rng = tm.box_expected(rng, box_with_array)
+        for obj in [mismatched, mismatched[:2]]:
+            # one shorter, one longer
+            for other in [obj, np.array(obj), pd.Index(obj)]:
+                with pytest.raises(ValueError):
+                    rng / other
+                with pytest.raises(ValueError):
+                    other / rng
+
     # ------------------------------------------------------------------
     # __floordiv__, __rfloordiv__
 
@@ -1203,6 +1280,9 @@ def test_td64arr_floordiv_int(self, box_with_array):
         result = idx // 1
         tm.assert_equal(result, idx)
 
+        with pytest.raises(TypeError):
+            1 // idx
+
     def test_td64arr_floordiv_tdlike_scalar(self, two_hours, box_with_array):
         tdi = timedelta_range('1 days', '10 days', name='foo')
         expected = pd.Int64Index((np.arange(10) + 1) * 12, name='foo')
@@ -1309,6 +1389,9 @@ def test_td64arr_div_numeric_scalar(self, box_with_array, two):
         result = tdser / two
         tm.assert_equal(result, expected)
 
+        with pytest.raises(TypeError):
+            two / tdser
+
     @pytest.mark.parametrize('dtype', ['int64', 'int32', 'int16',
                                        'uint64', 'uint32', 'uint16', 'uint8',
                                        'float64', 'float32', 'float16'])
@@ -1361,6 +1444,14 @@ def test_td64arr_div_numeric_array(self, box_with_array, vector, dtype):
         with pytest.raises(TypeError):
             vector / tdser
 
+        if not isinstance(vector, pd.Index):
+            # Index.__rdiv__ won't try to operate elementwise, just raises
+            result = tdser / vector.astype(object)
+            tm.assert_equal(result, expected)
+
+        with pytest.raises(TypeError):
+            vector.astype(object) / tdser
+
     @pytest.mark.parametrize('names', [(None, None, None),
                                        ('Egon', 'Venkman', None),
                                        ('NCC1701D', 'NCC1701D', 'NCC1701D')])
@@ -1391,20 +1482,25 @@ def test_td64arr_mul_int_series(self, box_df_fail, names):
     @pytest.mark.parametrize('names', [(None, None, None),
                                        ('Egon', 'Venkman', None),
                                        ('NCC1701D', 'NCC1701D', 'NCC1701D')])
-    def test_float_series_rdiv_td64arr(self, box, names):
+    def test_float_series_rdiv_td64arr(self, box_with_array, names):
         # GH#19042 test for correct name attachment
         # TODO: the direct operation TimedeltaIndex / Series still
         # needs to be fixed.
+        box = box_with_array
         tdi = TimedeltaIndex(['0days', '1day', '2days', '3days', '4days'],
                              name=names[0])
         ser = Series([1.5, 3, 4.5, 6, 7.5], dtype=np.float64, name=names[1])
 
+        xname = names[2] if box is not tm.to_array else names[1]
         expected = Series([tdi[n] / ser[n] for n in range(len(ser))],
                           dtype='timedelta64[ns]',
-                          name=names[2])
+                          name=xname)
+
+        xbox = box
+        if box in [pd.Index, tm.to_array] and type(ser) is Series:
+            xbox = Series
 
         tdi = tm.box_expected(tdi, box)
-        xbox = Series if (box is pd.Index and type(ser) is Series) else box
         expected = tm.box_expected(expected, xbox)
 
         result = ser.__rdiv__(tdi)


Code review comments:
diff_hunk: @@ -324,12 +325,106 @@ def _evaluate_with_timedelta_like(self, other, op):
 
     __mul__ = _wrap_tdi_op(operator.mul)
     __rmul__ = __mul__
-    __truediv__ = _wrap_tdi_op(operator.truediv)
     __floordiv__ = _wrap_tdi_op(operator.floordiv)
     __rfloordiv__ = _wrap_tdi_op(ops.rfloordiv)
 
+    def __truediv__(self, other):
+        # TODO: Decimals?
+        other = lib.item_from_zerodim(other)
+
+        if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
+            return NotImplemented
+
+        if isinstance(other, (timedelta, np.timedelta64, Tick)):
+            other = Timedelta(other)
+            if other is NaT:
+                # specifically timedelta64-NaT
+                result = np.empty(self.shape, dtype=np.float64)
+                result.fill(np.nan)
+                return result
+
+            # otherwise, dispatch to Timedelta implementation
+            return self._data / other
+
+        elif lib.is_scalar(other):
+            # assume it is numeric
+            result = self._data / other
+            freq = None
+            if self.freq is not None:
+                # Tick division is not implemented, so operate on Timedelta
+                freq = self.freq.delta / other
+            return type(self)(result, freq=freq)
+
+        if not hasattr(other, "dtype"):
+            # e.g. list, tuple
+            other = np.array(other)
+
+        if len(other) != len(self):
+            raise ValueError("Cannot divide vectors with unequal lengths")
+
+        elif is_timedelta64_dtype(other):
+            # let numpy handle it
+            return self._data / other
+
+        elif is_object_dtype(other):
+            result = [self[n] / other[n] for n in range(len(self))]
+            result = np.array(result)
+            if lib.infer_dtype(result) == 'timedelta':
+                # TODO: keep inferred freq?
+                result, _ = sequence_to_td64ns(result)
+                return type(self)(result)
+
+            return np.array(result)
+
+        else:
+            result = self._data / other
+            return type(self)(result)
+
+    def __rtruediv__(self, other):
+        other = lib.item_from_zerodim(other)
+
+        if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
+            return NotImplemented
+
+        if isinstance(other, (timedelta, np.timedelta64, Tick)):
+            other = Timedelta(other)
+            if other is NaT:
+                # specifically timedelta64-NaT
+                result = np.empty(self.shape, dtype=np.float64)
+                result.fill(np.nan)
+                return result
+
+            # otherwise, dispatch to Timedelta implementation
+            return other / self._data
+
+        elif lib.is_scalar(other):
+            raise TypeError("Cannot divide {typ} by {cls}"
+                            .format(typ=type(other).__name__,
+                                    cls=type(self).__name__))
+
+        if not hasattr(other, "dtype"):
+            # e.g. list, tuple
+            other = np.array(other)
+
+        if len(other) != len(self):
+            raise ValueError("Cannot divide vectors with unequal lengths")
+
+        elif is_timedelta64_dtype(other):
+            # let numpy handle it
+            return other / self._data
+
+        elif is_object_dtype(other):
+            result = [other[n] / self[n] for n in range(len(self))]
+            return np.array(result)
+
+        else:
+            # object-dtype *may* be OK here, but generally this will raise
+            result = other / self._data
+            return type(self)(result)
path: pandas/core/arrays/timedeltas.py
text: @user1:
according to codecov, this line is not covered

@author:
Thanks, I'll take a look.

@author:
This is reached in `test_td64arr_div_numeric_array`.  No idea whats going on with codecov.

@user1:
I think what is being tested there, is that `numeric_array / TDA` raises? So that will raise the error on the line above, and the return line is still not reached.

@author:
You're right.  Changed to raise directly.

diff_hunk: @@ -242,6 +239,38 @@ def _format_native_types(self, na_rep=u'NaT', date_format=None, **kwargs):
 
     total_seconds = wrap_array_method(TimedeltaArray.total_seconds, True)
 
+    @classmethod
+    def _override_div_mod_methods(cls):
+        """
+        Define __div__, __mod__, etc by wrapping TimedeltaArray implementations
+        rather than the versions defined by _add_numeric_methods.  Because
+        _add_numeric_methods is called after class creation, we have to define
+        these methods even later.
+        """
+
+        def __truediv__(self, other):
+            oth = other
+            if isinstance(other, Index):
+                # TimedeltaArray defers, so we need to unwrap
+                oth = other._values
+            result = TimedeltaArray.__truediv__(self, oth)
path: pandas/core/indexes/timedeltas.py
text: @user1:
Can we do here something like `result = self._values / oth` instead of calling the dunder method? (I don't really know the difference in practice, but it looks a bit cleaner)

@author:
I'll double-check.  Usually my reasoning for using the dunder methods is to preserve `NotImplemented` semantics.

diff_hunk: @@ -242,6 +239,38 @@ def _format_native_types(self, na_rep=u'NaT', date_format=None, **kwargs):
 
     total_seconds = wrap_array_method(TimedeltaArray.total_seconds, True)
 
+    @classmethod
+    def _override_div_mod_methods(cls):
path: pandas/core/indexes/timedeltas.py
text: @user1:
why are you defining new class level api here? if you need it here, wouldn't you also need this for base classes? this seems somewhat arbitrary 

@author:
private --> not API
not used by any other classes --> define just for class that uses it

The alternative is to define the methods directly in the class and then back in `Index._add_numeric_methods_binary` special-case to avoid re-defining the methods for `TimedeltaIndex`.  I actually kind of like this, since a couple of PRs from now we'll be able to stop calling `_add_numeric_methods_binary` for `TimedeltaIndex` at all.

@user1:
well, again, rather than create extra helper functions, just put it where everything else is. if you are then going to remove it in the future that is orthognal. the problem with adding things like this is it is just not obvious to anyone else reading this code why this is here and not in the standard location.

@author:
OK, I'll put it in base.Index I guess

diff_hunk: @@ -324,12 +325,106 @@ def _evaluate_with_timedelta_like(self, other, op):
 
     __mul__ = _wrap_tdi_op(operator.mul)
     __rmul__ = __mul__
-    __truediv__ = _wrap_tdi_op(operator.truediv)
     __floordiv__ = _wrap_tdi_op(operator.floordiv)
     __rfloordiv__ = _wrap_tdi_op(ops.rfloordiv)
 
+    def __truediv__(self, other):
+        # TODO: Decimals?
path: pandas/core/arrays/timedeltas.py
text: @user1:
Nit: I'm not sure I understand what this `TODO` means.

@author:
will clarify both here and there.  question is whether we want to go out of our way to operate against `decimal.Decimal` objects

@user2:
no

@user3:
Yep, agreed