COMPAT: Return NotImplemented for subclassing
State: closed

This changes index ops to check the *type* of the argument in index
ops, rather than just the dtype. This lets index subclasses take control
of binary ops when they know better what the result should be.

Closes https://github.com/pandas-dev/pandas/issues/31109

cc @jbrockmendel, I've fixed this as close to the root of the problem as I could, but perhaps this case should be checked elsewhere? As written currently, it only affects add & sub between an Index and a datelike index, but perhaps it should be done more generally?

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
Mmm, this is the wrong place to fix it. We've already unboxed the array in `other` by the time we get to my changes... Will take another crack.
LGTM, azure error looks unrelated
Just a regression on master.
k merge on green
Moving the conversation in https://github.com/pandas-dev/pandas/pull/31136/files/35549e681845d8505250dda00a9c22cff915f67f#diff-8ccfea8b354e9d2156ebc3150f69a18b out of inline.

Python has a behavior I hadn't realized before. Typically, for `A + B`, that calls `A.__add__(B)`, and calls `B.__radd__(A)` if needed. However, if `B` is a subclass of A, then `B.__radd__(A)` is called first.

```python
In [8]: class A:
   ...:     def __add__(self, other):
   ...:         print('a-add')
   ...:         return 0
   ...:     def __radd__(self, other):
   ...:         print('a-radd')
   ...:         return 1
   ...:

In [9]: class B:
   ...:     def __add__(self, other):
   ...:         print('b-add')
   ...:         return 2
   ...:     def __radd__(self, other):
   ...:         print("b-radd")
   ...:         return 3
   ...:

In [10]: A() + B()
a-add
Out[10]: 0

In [11]: class B(A):  # subclass
    ...:     def __add__(self, other):
    ...:         print('b-add')
    ...:         return 2
    ...:     def __radd__(self, other):
    ...:         print("b-radd")
    ...:         return 3
    ...:

In [12]: A() + B()
b-radd
Out[12]: 3
```

This means that an object dtype Index *would* have deferred to an Index subclass, just implicitly since Python does that. So numeric is the only behavior change we need to consider.
> So numeric is the only behavior change we need to consider.

Numeric shouldn't be relevant for the CFDatetimeIndex case of interest right?  I think the fix here should stay narrow and targeted at that for 1.0.
@jorisvandenbossche can you comment on https://github.com/pandas-dev/pandas/pull/31136#issuecomment-578155951? Does it address your concerns in https://github.com/pandas-dev/pandas/pull/31136#discussion_r368708696?
Merging later today if there aren't any objections.
> This means that an object dtype Index would have deferred to an Index subclass, just implicitly since Python does that. So numeric is the only behavior change we need to consider.

You mean because if you subclass `Index`, ops with Index class (i.e. object dtype) already get dispatched fine by python's mechanism. But for ops with numeric indices, the Index subclass is not a subclass of the numeric Index in that case, so therefore we need to handle it specifically there?
> But for ops with numeric indices, the Index subclass is not a subclass of the numeric Index in that case[...]

All this is correct. As an example, if you have

```python
class MyIndex(Index):
    pass
```

Then `Int64Index() + MyIndex()` will use `Int64Index.__add__` since MyIndex isn't a subclass of Int64Index.

> [...], so therefore we need to handle it specifically there?

This is where I think we disagree. In general, I don't think that all our Index subclasses can / should be doing these checks.

Ideally in the future we have an ExtensionIndex API and CFTimeIndex is an instance of that. But for now I think we should just restore the 0.25 behavior.
> > [...], so therefore we need to handle it specifically there?
>
> This is where I think we disagree. In general, I don't think that all our Index subclasses can / should be doing these checks.

Ah, sorry, I thought this was what you were doing in this PR. So now you do the check not in general for numeric index, but only for EA-based indexes (to have it for datetime-like indexes) ?

> So now you do the check not in general for numeric index, but only for EA-based indexes

Anything using `make_wrapped_arith_op`, which in practice is just the Datetimelike indexes.
OK, I was just going from the fact it being in `pandas/core/indexes/extension.py`. 

Anyway, I am fine with focusing on getting the compatibility with 0.25. 
And you are right that longer term the better way is to allow storing general ExtensionArrays in the Index.
@Meeseeksdev backport to 1.0.x