BUG: __getitem__ raise blank KeyError for IntervalIndex and missing keys
State: closed

- [x] closes #27365
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Without ``tolist`` error would look like ``KeyError: array([6])``

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Interval
description: Interval data type

Comments:
Tracking this down, it looks like the KeyError raised in Loc._validate_read_indexer has the well-written version of the message, and it looks like it is special-casing Categoricalndex for no apparent reason (CategoricalIndex and IntervalIndex being the only two Index subclasses that do the check this PR is looking at).

Not yet totally sure, but it _may_ be that we should always have this exception be raised in _validate_read_indexer
@jbrockmendel I was thinking along the same line. One problem is, that we arrive in ``_get_listlike_indexer`` sometimes through ``_getitem_iterable`` where ``raise_missing`` ist set to ``False``. This leads to supressed KeyErrors in some cases when not raising in IntervalIndex implementation. If we could change this to ``True``, we do not need the raise above. But comment says will stop working in the future, so I thought we can not change this now.
> But comment says will stop working in the future, so I thought we can not change this now.

Can you track down when these comments got added?  I think its likely that they are out of date.
@phofl i came to the conclusion that the future-behavior you mentioned was supposed to have already been implemented, so those comments should be removed/updated.

moreover, if we just remove IntervalIndex._convert_list_indexer altogether (and disable it from being called from CategoricalIndex._convert_list_indexer, we end up falling though to raise `raise KeyError(f"None of [{key}] are in the [{axis_name}]")` in `Loc._validate_read_indexer`, like we would for any other Index.

The remaining trouble is that the exception message isn't quite right for IntervalIndex bc it isnt a matter of the key not being present, but the key not being contained in any of the intervals.
@jbrockmendel ok great. Will remove the code paths and update this pr with it. Should come to this tomorrow evening
Removing

``IntervalIndex._convert_list_indexer``

completly does not work out of the box.

```
ser = Series(np.arange(5), IntervalIndex.from_breaks(np.arange(6)))
result = s.loc[[1.5, 2.5, 3.5]]
```

where result has index
``idx=Index([1.5,2.5,3.5])`` instead of intervals.

Additionally, if disable calling it from ``CategoricalIndex._convert_list_indexer``

```
df = DataFrame({"A": range(10)})
s = pd.cut(df.A, 5)
df["B"] = s
df = df.set_index("B")
result = df.loc[[4]]
```

raises ``{KeyError}'a list-indexer must only include values that are in the categories'`` which is probably wrong because we have an interval containing 4?
> completly does not work out of the box.

Thanks for giving it a try.  I may have been overly optimistic about reducing code here.
cc @jbrockmendel 
Looked a bit more into this and got a follow up question.

```
df = DataFrame({"A": range(10)})
s = pd.cut(df.A, 5)
df["B"] = s
df = df.set_index("B")
result = df.loc[[4]]
```

This raises because 

```
Int64Index([4], dtype='int64').difference(IntervalIndex([(-0.009, 1.8], (1.8, 3.6], (3.6, 5.4], (5.4, 7.2], (7.2, 9.0]],  closed='right', dtype='interval[float64]')
```

returns
```
Int64Index([4], dtype='int64')
```
 I am not that familiar with interval support in this operation, but this seems at least odd. I would have expected one of the following:

- Raising because incompatible
- Empty return

Is my assumption correct? Then fixing this would bring us one step closer to removing this as you proposed
> This raises because

This example doesn't raise for me on master.  Do you mean in this branch?

> I am not that familiar with interval support in this operation, but this seems at least odd. I would have expected one of the following:

indexing on a CategoricalIndex is difficult, and I think not entirely consistent (pretty sure there's an issue about this).

IntervalIndex.get_loc/get_indexer behavior is pretty much unconnected to the `.difference` operation (though that would make sense for e.g. 2 `Int64Index`es).  So im not sure i understand the question.
Sorry, my question was rubbish without context. 

If we remove 

```
     if self.categories._defer_to_indexing:
            indexer = self.categories._convert_list_indexer(keyarr)
            return Index(self.codes).get_indexer_for(indexer)
```

from ``_convert_list_indexer`` in ``category.py`` as discussed above, my example raises. In this case it gets piped through to the code snippet posted above with the ``differences`` return.
> from _convert_list_indexer in category.py as discussed above, my example raises. In this case it gets piped through to the code snippet posted above with the differences return.

yah lets ignore this as orthogonal to this PR.  can revisit later if we get ambitious about code simplification.
this looks good, can you merge master @phofl if we want to open an issue about the actual error message ok by me.
thanks @phofl very nice, keep em coming (as you have been!)
> if we want to open an issue about the actual error message ok by me.

#25996 already exists for key error messages in general.
Thanks very much, its fun working with you all