ENH: add IntervalIndex.get_loc_exact to look for exact matches only
State: closed

- [x] closes #19349
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This PR add a 'exact' option to the parameter ``method`` on ``IntervalIndex.get_loc`` for accepting exact matches only.

In addition to it's direct advantage, it will later make it possible to make a PR to allow ``IntervalIndex.get_indexer`` calls with the other index having non-Interval values, and hence allow mixing mix Interval with non-intervals in ``pd.Index(..., dtype=object)`` instances. This would fix the remaining issue with #19021.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: API Design
name: Interval
description: Interval data type

Comments:
There's a private method that looks like it's supposed to do this:

https://github.com/pandas-dev/pandas/blob/9872d6757e5117dce070981141cee562f675694e/pandas/core/indexes/interval.py#L870-L880

It doesn't appear to be working at all though:
```python
In [2]: ii = pd.interval_range(0, 4)

In [3]: ii[0]
Out[3]: Interval(0, 1, closed='right')

In [4]: ii.get_loc(ii[0])
Out[4]: 0

In [5]: ii._get_loc_only_exact_matches(ii[0])
---------------------------------------------------------------------------
KeyError: Interval(0, 1, closed='right')
```
The issue appears to be that `Index(self._multiindex.values)` creates an index of tuples, not intervals, so passing an `Interval` object will always raise a `KeyError`.  Likewise, due to the first line, passing anything other than an `Interval` object will also raise a `KeyError`.
```python
In [6]: pd.Index(ii._multiindex.values)
Out[6]: Index([(0, 1), (1, 2), (2, 3), (3, 4)], dtype='object')
```
Might be best to implement this in `_get_loc_only_exact_matches` instead?  It looks like `IntervalIndex.get_slice_bound` (inherited from `Index`) relies on  `_get_loc_only_exact_matches`.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19353?src=pr&el=h1) Report
> Merging [#19353](https://codecov.io/gh/pandas-dev/pandas/pull/19353?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/324379ce75269aa6bced90ecf3edb692539a2742?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `83.33%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/19353/graphs/tree.svg?height=150&width=650&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/19353?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #19353      +/-   ##
==========================================
- Coverage   91.61%   91.59%   -0.03%     
==========================================
  Files         150      150              
  Lines       48807    48817      +10     
==========================================
- Hits        44716    44712       -4     
- Misses       4091     4105      +14
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.96% <83.33%> (-0.03%)` | :arrow_down: |
| #single | `41.72% <16.66%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/19353?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/19353/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.38% <50%> (-0.06%)` | :arrow_down: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/19353/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `92.33% <90%> (-0.07%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/19353/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.22% <0%> (-1.74%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19353?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19353?src=pr&el=footer). Last update [324379c...9045cc9](https://codecov.io/gh/pandas-dev/pandas/pull/19353?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

yes, rather than adding kwargs to ``.get_loc`` much better to fix the private method here.
I can look into the private method.

@jreback , do you mean you are negative wrt. ``method='exact'`` in the public interface? I think it will be useful to have in many cases, e.g.

```python
>>> ii = pd.interva_range(0, 3)
>>> ii.get_loc(pd.Interval(1,3))
slice(1, 3, None)
>>> ii.get_loc(pd.Interval(1,3), method='exact')
KeyError: ...
```

Above, we've answered two different questions, both reasonable to ask, IMO.
In ``_get_loc_only_exact_matches`` I've converted the line ``return Index(self._multiindex.values).get_loc(key)`` to ``return super(IntervalIndex, self)._engine.get_loc(key)``. Another option would be to do ``self.astype(object).get_loc(key)``, which is more readable, but slower, presumedly because of the need to cast.

The super method is faster, around 13 ms, while the astype method is around 75 ms, so I've chosen the super method.

Also, ``_get_loc_only_exact_matches`` only worked for unique indices, while ``get_loc`` should work with non-unique indices, so some changes to accomodate non-unique values.
Hello @topper-123! Thanks for updating the PR.









Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on February 11, 2018 at 16:12 Hours UTC
@topper-123 we are not going to change the ``.get_loc`` api. it is very simple and needs to stay that way. As I said I am not above exposing the private function to make an exact match.
@jreback , adding this *keeps* things simple IMO: If you look at the doc string for ``pd.Index`.get_loc``, the ``method`` part is:
```
method : {None, 'pad'/'ffill', 'backfill'/'bfill', 'nearest'}, optional
    * default: exact matches only.
    * ...
```

It's logical that ``IntervalIndex.get_loc`` should be able to revert back to exact matches only, which is the default for all other index types.
I've added a ``get_loc_exact`` method instead of using ``method='exact'`` in ``get_loc``.
I think there are some caching issues when trying to use `super(IntervalIndex, self)._engine`.  Since both `IntervalIndex._engine` and `Index._engine` are decorated with `@cache_readonly`, it looks like you can't actually create both, which can lead to some weird errors depending on which `_engine` gets created first.

As an example, when an `IntervalIndex` is not unique, `get_loc` uses `self._engine`.  In such a scenario, you can't successfully use both `get_loc` and `get_loc_exact`, as whichever method you call second will fail.

Using  `get_loc` first causes `get_loc_exact` to fail:
```python
In [1]: import pandas as pd

In [2]: ii = pd.IntervalIndex.from_tuples([(0, 1), (1, 2), (0, 1)])

In [3]: ii.get_loc(pd.Interval(0, 1))
Out[3]: array([0, 2], dtype=int64)

In [4]: ii.get_loc_exact(pd.Interval(0, 1))
---------------------------------------------------------------------------
TypeError: No matching signature found
```

Using  `get_loc_exact` first causes `get_loc` to fail:
```python
In [1]: import pandas as pd

In [2]: ii = pd.IntervalIndex.from_tuples([(0, 1), (1, 2), (0, 1)])

In [3]: ii.get_loc_exact(pd.Interval(0, 1))
Out[3]: array([ True, False,  True], dtype=bool)

In [4]: ii.get_loc(pd.Interval(0, 1))
---------------------------------------------------------------------------
AttributeError: 'pandas._libs.index.ObjectEngine' object has no attribute 'get_loc_interval'
```

Not sure if there's a simple workaround here or not.
Nice catch.

I've changed the call to ``return self.astype(object).get_loc(key)`` and added a test for the condition you pointed out.

A shame, as the super method would have been faster if your issue would not have surfaced.
All concerns have been addessed, AFAICS.
green.
Is it really needed to add a method to all Index objects only for use in `IntervalIndex` ? 
Our public API is already huge, not sure this is needed to add even more. We have other examples where Index subclasses have specific type-related methods that are not available for other Index classes.
I was just going over some of the tests in `test_interval_new.py`, and noticed the following comment in `test_get_loc_interval`, which indicates that the desired future behavior for `get_loc` is to only return exact matches:

https://github.com/pandas-dev/pandas/blob/324379ce75269aa6bced90ecf3edb692539a2742/pandas/tests/indexes/interval/test_interval_new.py#L24-L31

Is this really the desired future behavior?  If so, should we try putting the `get_loc_exact` behavior in this PR directly into the `Interval` cases for `get_loc`?  Couldn't find any comments specifically about this after doing a quick search through the issue/pr that added the new test.
@jschendel Yes, that was also my first thought when I saw this PR, and wanted to say: let's change `get_loc` to be exact as we discussed that we want it this way. 
However, the comment you cite above is when it is passed an *interval*, then it is required to be exact (currently you can also pass a 'sub-interval'). But for non-interval scalars it still works as well, so eg `df.loc[1]` will still work to select all rows with intervalindex where the interval contains 1. So for scalars it does not need to be exact (at least, that is what we discussed then).
So it is the test below the one you quoted (`test_get_loc_scalar`): https://github.com/pandas-dev/pandas/blob/324379ce75269aa6bced90ecf3edb692539a2742/pandas/tests/indexes/interval/test_interval_new.py#L57-L58
@jorisvandenbossche : Yes, maybe it wasn't clear from my comment, but I was suggesting just modifying the `Interval` specific behavior of `get_loc` to force exactness.  Since we're already writing the logic to do this, I thought it might not be that much additional work to just incorporate it in `get_loc` as well, which could lead to some simplifications for `get_loc_exact`.  After looking over the code a bit more though, it might not be as straightforward as I initially thought.
> modifying the Interval specific behavior of get_loc to force exactness.

Ah, yes, I missed that. But that's a good idea