Fix IntervalDtype Bugs and Inconsistencies
State: closed

- [X] closes #18980
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry

Summary:
- `IntervalDtype(*)` now returns `True` when compared against `'interval'` and `IntervalDtype()` regardless of subtype
- `IntervalDtype(*)` now returns `'interval'` regardless of subtype
   - `str(IntervalDtype(*))` still displays subtype information, e.g. `'interval[int64]'`
- Cleaned up miscellaneous tests related to `IntervalDtype`
  

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: API Design
name: Interval
description: Interval data type

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18997?src=pr&el=h1) Report
> Merging [#18997](https://codecov.io/gh/pandas-dev/pandas/pull/18997?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d5194e5692a3327e05e9d0f2544e2512708921f3?src=pr&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18997/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18997?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18997      +/-   ##
==========================================
+ Coverage   91.51%   91.54%   +0.03%     
==========================================
  Files         148      148              
  Lines       48780    48777       -3     
==========================================
+ Hits        44639    44652      +13     
+ Misses       4141     4125      -16
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.91% <100%> (+0.03%)` | :arrow_up: |
| #single | `41.6% <27.27%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18997?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/dtypes/dtypes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18997/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2R0eXBlcy5weQ==) | `96.11% <100%> (+0.79%)` | :arrow_up: |
| [pandas/core/dtypes/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/18997/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2NvbW1vbi5weQ==) | `94.54% <0%> (+0.24%)` | :arrow_up: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18997/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `66.95% <0%> (+1.73%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18997?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18997?src=pr&el=footer). Last update [d5194e5...bd9805e](https://codecov.io/gh/pandas-dev/pandas/pull/18997?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

is there utility in having an IntervalIndex that works on objects / categorical types generally? (as opposed to numeric [float,int,datetime]).
> is there utility in having an IntervalIndex that works on objects / categorical types generally?

I've been wondering the same thing, and have been going back and forth.  It seems like we'd need to put in a fair number of guardrails for dtypes that aren't continuous (or continuous enough) where things like `mid` and `length` aren't guaranteed to be defined.  I've been asked in other PR's to handle such edge cases, e.g. https://github.com/pandas-dev/pandas/pull/18805#discussion_r157347554.  I'm not sure I'd classify that as _demand_, so much as reviewers just trying to ensure that all corners are covered.

As a source of comparison, postgres [`range` types](https://www.postgresql.org/docs/current/static/rangetypes.html) only provide numeric support by default.  However, postgres also makes it fairly easy to extend this to non-numeric types, e.g.
```sql
CREATE TYPE inetrange AS RANGE (
    SUBTYPE = inet
);
``` 
would provide support for intervals of IP addresses.  I don't think there'd be such easy extensions in our case if we only provided numeric support?

In terms of use cases, everything that I'd use intervals for would be covered by numeric.  I can imagine some scenarios where non-numeric could be useful though.  For example, intervals with string endpoints essentially allows for prefix level searches:
```python
In [2]: iv = pd.Interval('ab', 'bas')

In [3]: 'abracadabra' in iv
Out[3]: True

In [4]: 'bar' in iv
Out[4]: True

In [5]: 'bass' in iv
Out[5]: False
```
Some quick searching turns up a [postgres extension](https://github.com/dimitri/prefix) that appears do this.  The real world use case mentioned there is telephony applications.  I could maybe see similar prefix stuff being done in a biology/bioinformatics context, but really only have passing knowledge of that area, so could be wrong.

I could see applications for ordered categoricals with categories along the lines of low/medium/high (but more categories), where you'd have logical intervals to partition by.  Many problems could probably be solved via alternative approaches though, e.g. `groupby` logic.  Not sure if there'd be cases where `Interval` support would be far and away a superior solution.  You could probably even solve such problems without explicit `Categorical` support by mapping label <--> integer, creating integer intervals, and mapping back as appropriate, though kind of seems like reinventing the `Categorical` wheel.

All that being said, I don't know how prevalent these use cases would actually be.  Could very well not be worth the effort to support such operations.
cc @zfrenchee any thoughts on the object / categorical (non-numeric) case for intervals?
@jreback @jschendel I can't think of a use case for categorical intervals in my work, but that doesn't mean it can't exist. The telephony / ip address idea seems like it would be reasonable. 

The prefix search is interesting, but I imagine if you wanted a prefix tree, you wouldn't create a multi-intervalindex df (though it might work), but instead use a prefix-tree package. 

I might be cautious with permitting IntervalIndex of arbitrary subtype. Having a complex type system is nice when it works, but I wouldn't want to have to debug issues people brought me involving multi-IntervalIndexes of exotic subtypes. Not sure that's a good reason though. 
I agree. @jschendel if you wouldn't mind in this (or maybe better other) PR. to limit IntervalIndex dtypes that it will accept. (e.g. I think all except object, categorical). Then this PR becomes much simpler.
Updated to remove code for unsupported dtypes now that #19022 is merged.  Also removed parsing `'interval[]'` for `IntervalDtype` construction.
lgtm. @TomAugspurger if you'd have a look.