Repository: pandas-dev/pandas
Pull request: BUG: fix IntervalIndex for pivot table raise type error 
Close (#25814). When use IntervalIndex for pivot table, raise
TypeError: cannot determine next label for type <class 'str'> 
- [x] closes #25814
- [x] tets added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
Cannot use categorical IntervalIndex as index when creating pivot_table
#### Code Sample, a copy-pastable example if possible

```python
df = pd.DataFrame({"A":[1, 1, 2, 2], "B":[1,2,3,4]})
df['C'] = pd.cut(df['A'],[0,2])
df.pivot_table(index='C', values='B', aggfunc=np.sum)

```
#### Problem description
Cannot use IntervalIndex as index when creating  pivot_table. It works fine on 0.23.4.

C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\interval.py in _get_prev_label(label)
     75     else:
     76         raise TypeError('cannot determine next label for type {typ!r}'
---> 77                         .format(typ=type(label)))
     78 
     79 

TypeError: cannot determine next label for type <class 'str'>




#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.7.1.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: zh_CN.GBK@cjknarrow
LOCALE: None.None

pandas: 0.24.2
pytest: 4.0.2
pip: 18.1
setuptools: 40.6.3
Cython: 0.29.2
numpy: 1.15.4
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: 1.8.2
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.7
blosc: None
bottleneck: 1.2.1
tables: 3.4.4
numexpr: 2.6.8
feather: None
matplotlib: 3.0.2
openpyxl: 2.5.12
xlrd: 1.2.0
xlwt: 1.3.0
xlsxwriter: 1.1.2
lxml.etree: 4.2.5
bs4: 4.6.3
html5lib: 1.0.1
sqlalchemy: 1.2.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: Nonetput of ``pd.show_versions()`` here below this line]

</details>

Hints from issue discussion:
Thanks for the report.  I can confirm that this is broken on master:

```python
In [1]: import numpy as np; import pandas as pd; pd.__version__
Out[1]: '0.25.0.dev0+294.gfbe252333'

In [2]: df = pd.DataFrame({"A":[1, 1, 2, 2], "B":[1,2,3,4]}) 
   ...: df['C'] = pd.cut(df['A'],[0,2]) 
   ...: df
Out[2]: 
   A  B       C
0  1  1  (0, 2]
1  1  2  (0, 2]
2  2  3  (0, 2]
3  2  4  (0, 2]

In [3]: df.pivot_table(index='C', values='B', aggfunc=np.sum)
---------------------------------------------------------------------------
TypeError: cannot determine next label for type <class 'str'>
```

Interestingly, this works when `df['C']` is a plain `IntervalArray`:
```python
In [4]: df['C'] = pd.arrays.IntervalArray([pd.Interval(0, 2)] * 4)
   ...: df
Out[4]: 
   A  B       C
0  1  1  (0, 2]
1  1  2  (0, 2]
2  2  3  (0, 2]
3  2  4  (0, 2]

In [5]: df.pivot_table(index='C', values='B', aggfunc=np.sum)
Out[5]: 
         B
C         
(0, 2]  10
```

I can also confirm that this worked on 0.23.4:
```python
In [1]: import numpy as np; import pandas as pd; pd.__version__
Out[1]: '0.23.4'

In [2]: df = pd.DataFrame({"A":[1, 1, 2, 2], "B":[1,2,3,4]}) 
   ...: df['C'] = pd.cut(df['A'],[0,2]) 
   ...: df
Out[2]: 
   A  B       C
0  1  1  (0, 2]
1  1  2  (0, 2]
2  2  3  (0, 2]
3  2  4  (0, 2]

In [3]: df.pivot_table(index='C', values='B', aggfunc=np.sum)
Out[3]: 
         B
C         
(0, 2]  10
```

Commit message: update whatsnews's reshaping section

Patch to review:
diff --git a/doc/source/whatsnew/v0.24.2.rst b/doc/source/whatsnew/v0.24.2.rst
index c3b442e2352bb..e1e087f5600f0 100644
--- a/doc/source/whatsnew/v0.24.2.rst
+++ b/doc/source/whatsnew/v0.24.2.rst
@@ -59,6 +59,7 @@ Bug Fixes
 
 - Bug in :meth:`~pandas.core.groupby.GroupBy.transform` where applying a function to a timezone aware column would return a timezone naive result (:issue:`24198`)
 - Bug in :func:`DataFrame.join` when joining on a timezone aware :class:`DatetimeIndex` (:issue:`23931`)
+- Bug in :func:`DataFrame.pivot_table` where use :class:`IntervalIndex` as pivot index would raise TypeError (:issue:`25814`)
 
 **Visualization**
 
diff --git a/pandas/core/arrays/categorical.py b/pandas/core/arrays/categorical.py
index dc77599444505..c079b860bb924 100644
--- a/pandas/core/arrays/categorical.py
+++ b/pandas/core/arrays/categorical.py
@@ -181,7 +181,7 @@ def contains(cat, key, container):
     #  can't be in container either.
     try:
         loc = cat.categories.get_loc(key)
-    except KeyError:
+    except (KeyError, TypeError):
         return False
 
     # loc is the location of key in categories, but also the *value*
diff --git a/pandas/tests/reshape/test_pivot.py b/pandas/tests/reshape/test_pivot.py
index cc91bef525eff..f1f277135785d 100644
--- a/pandas/tests/reshape/test_pivot.py
+++ b/pandas/tests/reshape/test_pivot.py
@@ -19,6 +19,18 @@ def dropna(request):
     return request.param
 
 
+@pytest.fixture(
+    params=[
+        pd.Categorical([pd.Interval(0, 1, 'right')] * 4),
+        pd.Categorical([pd.Interval(0, 1, 'left')] * 4),
+        pd.Categorical([pd.Interval(low, high, 'right')
+                        for low, high in zip(range(0, 3), range(1, 4))]),
+        pd.Categorical([pd.Interval(low, high, 'left')
+                        for low, high in zip(range(0, 3), range(1, 4))])])
+def interval_values(request):
+    return request.param
+
+
 class TestPivotTable:
 
     def setup_method(self, method):
@@ -198,6 +210,18 @@ def test_pivot_with_non_observable_dropna(self, dropna):
 
         tm.assert_frame_equal(result, expected)
 
+    def test_pivot_with_interval_index(self, interval_values, dropna):
+        # GH 25814
+        df = pd.DataFrame(
+            {'A': interval_values,
+             'B': [1] * interval_values.size})
+        result = df.pivot_table(index='A', values='B', dropna=dropna)
+        expected = pd.DataFrame(
+            {'B': [1]},
+            index=pd.Index(interval_values.unique(),
+                           name='A'))
+        tm.assert_frame_equal(result, expected)
+
     def test_pass_array(self):
         result = self.data.pivot_table(
             'D', index=self.data.A, columns=self.data.C)


Code review comments:
diff_hunk: @@ -198,6 +210,18 @@ def test_pivot_with_non_observable_dropna(self, dropna):
 
         tm.assert_frame_equal(result, expected)
 
+    def test_pivot_with_interval_index(self, interval_values, dropna):
+        # GH 25814
+        df = pd.DataFrame(
+            {'A': interval_values,
+             'B': [1] * interval_values.size})
+        result = df.pivot_table(index='A', values='B', dropna=dropna)
+        expected = pd.DataFrame(
+            {'B': [1]},
+            index=pd.Index(interval_values.unique(),
path: pandas/tests/reshape/test_pivot.py
text: @user1:
Note that `DataFrame` and `Index` are both already imported, so you can omit the `pd.`

diff_hunk: @@ -19,6 +19,18 @@ def dropna(request):
     return request.param
 
 
+@pytest.fixture(
+    params=[
+        pd.Categorical([pd.Interval(0, 1, 'right')] * 4),
+        pd.Categorical([pd.Interval(0, 1, 'left')] * 4),
+        pd.Categorical([pd.Interval(low, high, 'right')
+                        for low, high in zip(range(0, 3), range(1, 4))]),
+        pd.Categorical([pd.Interval(low, high, 'left')
+                        for low, high in zip(range(0, 3), range(1, 4))])])
+def interval_values(request):
path: pandas/tests/reshape/test_pivot.py
text: @user2:
A bit cleaner implementation that tests all variants of closed:
```python
@user1.fixture(params=[([0] * 4, [1] * 4), (range(0, 3), range(1, 4))])
def interval_values(request, closed):
    left, right = request.param
    return Categorical(pd.IntervalIndex.from_arrays(left, right, closed))
```

diff_hunk: @@ -198,6 +210,18 @@ def test_pivot_with_non_observable_dropna(self, dropna):
 
         tm.assert_frame_equal(result, expected)
 
+    def test_pivot_with_interval_index(self, interval_values, dropna):
+        # GH 25814
+        df = pd.DataFrame(
+            {'A': interval_values,
+             'B': [1] * interval_values.size})
path: pandas/tests/reshape/test_pivot.py
text: @user1:
You don't need to match size here, can just do:
```python
df = DataFrame({'A': interval_values, 'B': 1})
```