Repository: pandas-dev/pandas

Issue:
Categorical.from_codes does not allow categories to be a CategoricalIndex
#### Code Sample, a copy-pastable example if possible

`Categorical.from_codes` requires `categories` to be specified, with documented type "index-like" but passing a `CategoricalIndex` raises an error.

```python
>>> pd.Categorical.from_codes([1, 2], pd.CategoricalIndex(['Hello', 'world', '!', '**'])
/Users/joel/anaconda3/envs/scipy3k/lib/python3.5/site-packages/pandas/core/categorical.py in get_values(self)
   1276         if is_datetimelike(self.categories):
   1277             return self.categories.take(self._codes, fill_value=np.nan)
-> 1278         return np.array(self)
   1279
   1280     def check_for_ordered(self, op):

ValueError: object __array__ method not producing an array
```

#### Problem description

It seems that being able to construct a categorical column with an existing `CategoricalIndex` would be a useful way to share `CategoricalIndex` int-string mappings. This would seem to be what `from_codes` aims to facilitate, but requires reconstructing the `CategoricalIndex`, which seems unnecessary to me.

The documentation also says index-like which seems underspecified if a CategoricalIndex is not accepted. At least, the error is not sufficiently informative.
Categorical.from_codes does not allow categories to be a CategoricalIndex
#### Code Sample, a copy-pastable example if possible

`Categorical.from_codes` requires `categories` to be specified, with documented type "index-like" but passing a `CategoricalIndex` raises an error.

```python
>>> pd.Categorical.from_codes([1, 2], pd.CategoricalIndex(['Hello', 'world', '!', '**'])
/Users/joel/anaconda3/envs/scipy3k/lib/python3.5/site-packages/pandas/core/categorical.py in get_values(self)
   1276         if is_datetimelike(self.categories):
   1277             return self.categories.take(self._codes, fill_value=np.nan)
-> 1278         return np.array(self)
   1279
   1280     def check_for_ordered(self, op):

ValueError: object __array__ method not producing an array
```

#### Problem description

It seems that being able to construct a categorical column with an existing `CategoricalIndex` would be a useful way to share `CategoricalIndex` int-string mappings. This would seem to be what `from_codes` aims to facilitate, but requires reconstructing the `CategoricalIndex`, which seems unnecessary to me.

The documentation also says index-like which seems underspecified if a CategoricalIndex is not accepted. At least, the error is not sufficiently informative.


Discussion:
That seems like a bug, as I don't think there is a reason why a CategoricalIndex shouldn't be allowed.

But, it also seems that the Categorical *is* constructed, but just so in an invalid way that printing it errors:

```
In [46]: res = pd.Categorical.from_codes([1, 2], pd.CategoricalIndex(['Hello', 'world', '!', '**']))

In [47]: res.codes
Out[47]: array([1, 2], dtype=int8)

In [48]: res.categories
Out[48]: CategoricalIndex(['Hello', 'world', '!', '**'], categories=['!', '**', 'Hello', 'world'], ordered=False, dtype='category')

In [49]: repr(res)
...
ValueError: object __array__ method not producing an array
```

This is certainly a bug.
Further, we have the exact same issue when in the normal constructor `pd.Categorical`, and also with passing a `Categorical` (instead of CategoricalIndex) as categories. 
Of course we could fix the bug in the repr code, but I would say that we should not allow the categories to be a CategoricalIndex (in the sense: we should convert a passes Categorical/CategoricalIndex under the hood to a normal Index).
ahh, I misread the error. thanks for clarifying

That seems like a bug, as I don't think there is a reason why a CategoricalIndex shouldn't be allowed.

But, it also seems that the Categorical *is* constructed, but just so in an invalid way that printing it errors:

```
In [46]: res = pd.Categorical.from_codes([1, 2], pd.CategoricalIndex(['Hello', 'world', '!', '**']))

In [47]: res.codes
Out[47]: array([1, 2], dtype=int8)

In [48]: res.categories
Out[48]: CategoricalIndex(['Hello', 'world', '!', '**'], categories=['!', '**', 'Hello', 'world'], ordered=False, dtype='category')

In [49]: repr(res)
...
ValueError: object __array__ method not producing an array
```

This is certainly a bug.
Further, we have the exact same issue when in the normal constructor `pd.Categorical`, and also with passing a `Categorical` (instead of CategoricalIndex) as categories. 
Of course we could fix the bug in the repr code, but I would say that we should not allow the categories to be a CategoricalIndex (in the sense: we should convert a passes Categorical/CategoricalIndex under the hood to a normal Index).
ahh, I misread the error. thanks for clarifying


Resolving patch:
diff --git a/pandas/core/dtypes/dtypes.py b/pandas/core/dtypes/dtypes.py
--- a/pandas/core/dtypes/dtypes.py
+++ b/pandas/core/dtypes/dtypes.py
@@ -3,7 +3,7 @@
 import re
 import numpy as np
 from pandas import compat
-from pandas.core.dtypes.generic import ABCIndexClass
+from pandas.core.dtypes.generic import ABCIndexClass, ABCCategoricalIndex
 
 
 class ExtensionDtype(object):
@@ -170,16 +170,16 @@ def _from_categorical_dtype(cls, dtype, categories=None, ordered=None):
         return cls(categories, ordered)
 
     def _finalize(self, categories, ordered, fastpath=False):
-        from pandas.core.indexes.base import Index
 
         if ordered is None:
             ordered = False
+        else:
+            self._validate_ordered(ordered)
 
         if categories is not None:
-            categories = Index(categories, tupleize_cols=False)
-            # validation
-            self._validate_categories(categories, fastpath=fastpath)
-            self._validate_ordered(ordered)
+            categories = self._validate_categories(categories,
+                                                   fastpath=fastpath)
+
         self._categories = categories
         self._ordered = ordered
 
@@ -316,7 +316,7 @@ def _validate_categories(categories, fastpath=False):
         from pandas import Index
 
         if not isinstance(categories, ABCIndexClass):
-            categories = Index(categories)
+            categories = Index(categories, tupleize_cols=False)
 
         if not fastpath:
 
@@ -326,6 +326,9 @@ def _validate_categories(categories, fastpath=False):
             if not categories.is_unique:
                 raise ValueError('Categorical categories must be unique')
 
+        if isinstance(categories, ABCCategoricalIndex):
+            categories = categories.categories
+
         return categories
 
     @property


Test patch:
diff --git a/pandas/tests/dtypes/test_dtypes.py b/pandas/tests/dtypes/test_dtypes.py
--- a/pandas/tests/dtypes/test_dtypes.py
+++ b/pandas/tests/dtypes/test_dtypes.py
@@ -6,7 +6,8 @@
 
 import numpy as np
 import pandas as pd
-from pandas import Series, Categorical, IntervalIndex, date_range
+from pandas import (
+    Series, Categorical, CategoricalIndex, IntervalIndex, date_range)
 
 from pandas.core.dtypes.dtypes import (
     DatetimeTZDtype, PeriodDtype,
@@ -657,3 +658,10 @@ def test_str_vs_repr(self):
         # Py2 will have unicode prefixes
         pat = r"CategoricalDtype\(categories=\[.*\], ordered=False\)"
         assert re.match(pat, repr(c1))
+
+    def test_categorical_categories(self):
+        # GH17884
+        c1 = CategoricalDtype(Categorical(['a', 'b']))
+        tm.assert_index_equal(c1.categories, pd.Index(['a', 'b']))
+        c1 = CategoricalDtype(CategoricalIndex(['a', 'b']))
+        tm.assert_index_equal(c1.categories, pd.Index(['a', 'b']))
diff --git a/pandas/tests/test_categorical.py b/pandas/tests/test_categorical.py
--- a/pandas/tests/test_categorical.py
+++ b/pandas/tests/test_categorical.py
@@ -519,6 +519,18 @@ def test_contructor_from_categorical_string(self):
         result = Categorical(values, categories=['a', 'b', 'c'], ordered=True)
         tm.assert_categorical_equal(result, expected)
 
+    def test_constructor_with_categorical_categories(self):
+        # GH17884
+        expected = Categorical(['a', 'b'], categories=['a', 'b', 'c'])
+
+        result = Categorical(
+            ['a', 'b'], categories=Categorical(['a', 'b', 'c']))
+        tm.assert_categorical_equal(result, expected)
+
+        result = Categorical(
+            ['a', 'b'], categories=CategoricalIndex(['a', 'b', 'c']))
+        tm.assert_categorical_equal(result, expected)
+
     def test_from_codes(self):
 
         # too few categories
@@ -560,6 +572,22 @@ def f():
             codes = np.random.choice([0, 1], 5, p=[0.9, 0.1])
             pd.Categorical.from_codes(codes, categories=["train", "test"])
 
+    def test_from_codes_with_categorical_categories(self):
+        # GH17884
+        expected = Categorical(['a', 'b'], categories=['a', 'b', 'c'])
+
+        result = Categorical.from_codes(
+            [0, 1], categories=Categorical(['a', 'b', 'c']))
+        tm.assert_categorical_equal(result, expected)
+
+        result = Categorical.from_codes(
+            [0, 1], categories=CategoricalIndex(['a', 'b', 'c']))
+        tm.assert_categorical_equal(result, expected)
+
+        # non-unique Categorical still raises
+        with pytest.raises(ValueError):
+            Categorical.from_codes([0, 1], Categorical(['a', 'b', 'a']))
+
     @pytest.mark.parametrize('dtype', [None, 'category'])
     def test_from_inferred_categories(self, dtype):
         cats = ['a', 'b']