Repository: pandas-dev/pandas
Pull request: COMPAT: Return NotImplemented for subclassing
This changes index ops to check the *type* of the argument in index
ops, rather than just the dtype. This lets index subclasses take control
of binary ops when they know better what the result should be.

Closes https://github.com/pandas-dev/pandas/issues/31109

cc @jbrockmendel, I've fixed this as close to the root of the problem as I could, but perhaps this case should be checked elsewhere? As written currently, it only affects add & sub between an Index and a datelike index, but perhaps it should be done more generally?

Problem statement:
Return NotImpelmented for arith ops with object-dtype Index subclasses
This change in 1.0 impacted xarray

```python
 In [22]: import xarray as xr

In [23]: import pandas as pd

In [24]: idx = pd.timedelta_range("1D", periods=5, freq="D")

In [25]: a = xr.cftime_range("2000", periods=5)

In [26]: idx + a
/Users/taugspurger/sandbox/pandas/pandas/core/arrays/datetimelike.py:1204: PerformanceWarning: Adding/subtracting array of DateOffsets to TimedeltaArray not vectorized
  PerformanceWarning,
Out[26]:
Index([2000-01-02 00:00:00, 2000-01-04 00:00:00, 2000-01-06 00:00:00,
       2000-01-08 00:00:00, 2000-01-10 00:00:00],
      dtype='object')

In [27]: a + idx
Out[27]:
CFTimeIndex([2000-01-02 00:00:00, 2000-01-04 00:00:00, 2000-01-06 00:00:00,
             2000-01-08 00:00:00, 2000-01-10 00:00:00],
            dtype='object')
```

cc @jbrockmendel @dcherian. Brock suggested checking `type(other)`, and returning NotImplemented for index subclasses we don't know about, to give the subclass a chance to take over the op.

Hints from issue discussion:
Should we try to get a short-term fix in for 1.0?
Yep, I think so.

Commit message: COMPAT: Return NotImplemented for subclassing

This changes index ops to check the *type* of the argument in index
ops, rather than just the dtype. This lets index subclasses take control
of binary ops when they know better what the result should be.

Closes https://github.com/pandas-dev/pandas/issues/31109

Patch to review:
diff --git a/pandas/core/arrays/datetimelike.py b/pandas/core/arrays/datetimelike.py
index 70637026c278d..7f5f0652c67e0 100644
--- a/pandas/core/arrays/datetimelike.py
+++ b/pandas/core/arrays/datetimelike.py
@@ -1299,8 +1299,16 @@ def __add__(self, other):
             # TimedeltaIndex, ndarray[timedelta64]
             result = self._add_delta(other)
         elif is_object_dtype(other):
-            # e.g. Array/Index of DateOffset objects
-            result = self._addsub_object_array(other, operator.add)
+            from pandas import Index
+
+            if type(other) is Index:
+                # e.g. Array/Index of DateOffset objects
+                # We deliberately check for other being exactly an Index, rather
+                # than a subclass, to let Index subclasses take control of binary
+                # operations. See https://github.com/pandas-dev/pandas/issues/31109
+                result = self._addsub_object_array(other, operator.add)
+            else:
+                return NotImplemented
         elif is_datetime64_dtype(other) or is_datetime64tz_dtype(other):
             # DatetimeIndex, ndarray[datetime64]
             return self._add_datetime_arraylike(other)
@@ -1354,8 +1362,16 @@ def __sub__(self, other):
             # TimedeltaIndex, ndarray[timedelta64]
             result = self._add_delta(-other)
         elif is_object_dtype(other):
-            # e.g. Array/Index of DateOffset objects
-            result = self._addsub_object_array(other, operator.sub)
+            from pandas import Index
+
+            if type(other) is Index:
+                # e.g. Array/Index of DateOffset objects
+                # We deliberately check for other being exactly an Index, rather
+                # than a subclass, to let Index subclasses take control of binary
+                # operations. See https://github.com/pandas-dev/pandas/issues/31109
+                result = self._addsub_object_array(other, operator.sub)
+            else:
+                return NotImplemented
         elif is_datetime64_dtype(other) or is_datetime64tz_dtype(other):
             # DatetimeIndex, ndarray[datetime64]
             result = self._sub_datetime_arraylike(other)
diff --git a/pandas/tests/arithmetic/test_object.py b/pandas/tests/arithmetic/test_object.py
index c0d3c9d4977bd..5bf694e574dc2 100644
--- a/pandas/tests/arithmetic/test_object.py
+++ b/pandas/tests/arithmetic/test_object.py
@@ -1,6 +1,7 @@
 # Arithmetic tests for DataFrame/Series/Index/Array classes that should
 # behave identically.
 # Specifically for object dtype
+import datetime
 from decimal import Decimal
 import operator
 
@@ -323,3 +324,35 @@ def test_rsub_object(self):
 
         with pytest.raises(TypeError):
             np.array([True, pd.Timestamp.now()]) - index
+
+    def test_index_ops_defer_to_unknown_subclasses(self):
+        # https://github.com/pandas-dev/pandas/issues/31109
+        class MyIndex(pd.Index):
+
+            _calls: int
+
+            @classmethod
+            def _simple_new(cls, values, name=None, dtype=None):
+                result = object.__new__(cls)
+                result._data = values
+                result._index_data = values
+                result._name = name
+                result._calls = 0
+
+                return result._reset_identity()
+
+            def __add__(self, other):
+                self._calls += 1
+                return self._simple_new(self._index_data + other.to_list())
+
+            def __radd__(self, other):
+                return self.__add__(other)
+
+        values = np.array(
+            [datetime.date(2000, 1, 1), datetime.date(2000, 1, 2)], dtype=object
+        )
+        a = MyIndex._simple_new(values)
+        b = pd.timedelta_range("1D", periods=2, freq="D")
+        result = b + a
+        assert isinstance(result, MyIndex)
+        assert a._calls == 1


Code review comments:
diff_hunk: @@ -1299,8 +1299,16 @@ def __add__(self, other):
             # TimedeltaIndex, ndarray[timedelta64]
             result = self._add_delta(other)
         elif is_object_dtype(other):
-            # e.g. Array/Index of DateOffset objects
-            result = self._addsub_object_array(other, operator.add)
+            from pandas import Index
+
+            if type(other) is Index:
path: pandas/core/arrays/datetimelike.py
text: @user1:
i think we also want ndarray here.  so we want to exclude something like `isinstance(other, ABCIndex) and type(other) is not Index`