Repository: pandas-dev/pandas

Issue:
CategoricalIndex reidex duplicates values
From [SO question](https://stackoverflow.com/q/51228928/2901002):

**Sample**:

    np.random.seed(57)

    idx = pd.CategoricalIndex(['low'] * 3 + ['hi'] * 3)
    dfb = pd.DataFrame(np.random.rand(6, 3), columns=list('abc'), index=idx)
    print (dfb)
                a         b         c
    low  0.087350  0.230477  0.411061
    low  0.310783  0.565956  0.545064
    low  0.807099  0.918155  0.522091
    hi   0.424687  0.071804  0.898529
    hi   0.420514  0.582170  0.214154
    hi   0.447486  0.467864  0.100637

Round incorrectly explode rows:
    
    print (dfb.round(3))
             a      b      c
    low  0.087  0.230  0.411
    low  0.311  0.566  0.545
    low  0.807  0.918  0.522
    low  0.087  0.230  0.411
    low  0.311  0.566  0.545
    low  0.807  0.918  0.522
    low  0.087  0.230  0.411
    low  0.311  0.566  0.545
    low  0.807  0.918  0.522
    hi   0.425  0.072  0.899
    hi   0.421  0.582  0.214
    hi   0.447  0.468  0.101
    hi   0.425  0.072  0.899
    hi   0.421  0.582  0.214
    hi   0.447  0.468  0.101
    hi   0.425  0.072  0.899
    hi   0.421  0.582  0.214
    hi   0.447  0.468  0.101

Expected output:

    print (dfb.round(3))
             a      b      c
    low  0.087  0.230  0.411
    low  0.311  0.566  0.545
    low  0.807  0.918  0.522
    hi   0.425  0.072  0.899
    hi   0.421  0.582  0.214
    hi   0.447  0.468  0.101

---

    print (pd.show_versions())
    
    INSTALLED VERSIONS
    ------------------
    commit: None
    python: 3.6.4.final.0
    python-bits: 64
    OS: Windows
    OS-release: 7
    machine: AMD64
    processor: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
    byteorder: little
    LC_ALL: None
    LANG: en
    LOCALE: None.None
    
    pandas: 0.23.1
    pytest: 3.3.2
    pip: 9.0.1
    setuptools: 39.2.0
    Cython: 0.27.3
    numpy: 1.14.3
    scipy: 1.0.0
    pyarrow: None
    xarray: None
    IPython: 6.2.1
    sphinx: 1.6.6
    patsy: 0.5.0
    dateutil: 2.7.3
    pytz: 2018.4
    blosc: None
    bottleneck: 1.2.1
    tables: 3.4.2
    numexpr: 2.6.4
    feather: None
    matplotlib: 2.2.2
    openpyxl: 2.4.10
    xlrd: 1.1.0
    xlwt: 1.3.0
    xlsxwriter: 1.0.2
    lxml: 4.1.1
    bs4: 4.6.0
    html5lib: 1.0.1
    sqlalchemy: 1.2.1
    pymysql: None
    psycopg2: None
    jinja2: 2.10
    s3fs: None
    fastparquet: None
    pandas_gbq: None
    pandas_datareader: None
    None

    



Discussion:
Seems to be an issue with the DataFrame constructor.


```python
In [11]: idx = pd.CategoricalIndex(['low', 'low', 'hi', 'hi'])

In [12]: pd.DataFrame(pd.DataFrame({"A": [1, 2, 3, 4]}, index=idx), index=idx)
Out[12]:
     A
low  1
low  2
low  1
low  2
hi   3
hi   4
hi   3
hi   4
```

Would welcome further investigate from others!
The problem seems to be with the `reindex` method:
```python
>>> idx = pd.CategoricalIndex(['low', 'low', 'hi', 'hi'])
>>> idx.reindex(idx)[0].values
[low, low, low, low, hi, hi, hi, hi]
Categories (2, object): [hi, low]
```
**Update**:
Problem is here:
https://github.com/pandas-dev/pandas/blob/bdb61688aa80a6ec9bdb6832946dd83ab4b4c9c8/pandas/core/indexes/category.py#L589-L602

changing line 601 to:
```python
indexer, missing = self._engine.get_indexer_non_unique(np.unique(codes))
```
helps and doesn't break tests, but I'm not sure that it's correct
Thanks for investigating further.

We *probably* don't want to `unique` the codes, as that can be expensive... Let me see if there's a simpler way.

Hmm, `Index.reindex` with duplicates seems strange :/ Will have to come back to this later. Other's are welcome to continue investigating of course :)

Resolving patch:
diff --git a/pandas/core/indexes/category.py b/pandas/core/indexes/category.py
--- a/pandas/core/indexes/category.py
+++ b/pandas/core/indexes/category.py
@@ -532,12 +532,16 @@ def reindex(self, target, method=None, level=None, limit=None,
 
         target = ibase.ensure_index(target)
 
-        if not is_categorical_dtype(target) and not target.is_unique:
-            raise ValueError("cannot reindex with a non-unique indexer")
+        if self.equals(target):
+            indexer = None
+            missing = []
+        else:
+            if not target.is_unique:
+                raise ValueError("cannot reindex with a non-unique indexer")
 
-        indexer, missing = self.get_indexer_non_unique(np.array(target))
+            indexer, missing = self.get_indexer_non_unique(np.array(target))
 
-        if len(self.codes):
+        if len(self.codes) and indexer is not None:
             new_target = self.take(indexer)
         else:
             new_target = target


Test patch:
diff --git a/pandas/tests/frame/test_analytics.py b/pandas/tests/frame/test_analytics.py
--- a/pandas/tests/frame/test_analytics.py
+++ b/pandas/tests/frame/test_analytics.py
@@ -1805,6 +1805,21 @@ def test_built_in_round(self):
             {'col1': [1., 2., 3.], 'col2': [1., 2., 3.]})
         tm.assert_frame_equal(round(df), expected_rounded)
 
+    def test_round_nonunique_categorical(self):
+        # See GH21809
+        idx = pd.CategoricalIndex(['low'] * 3 + ['hi'] * 3)
+        df = pd.DataFrame(np.random.rand(6, 3), columns=list('abc'))
+
+        expected = df.round(3)
+        expected.index = idx
+
+        df_categorical = df.copy().set_index(idx)
+        assert df_categorical.shape == (6, 3)
+        result = df_categorical.round(3)
+        assert result.shape == (6, 3)
+
+        tm.assert_frame_equal(result, expected)
+
     def test_pct_change(self):
         # GH 11150
         pnl = DataFrame([np.arange(0, 40, 10), np.arange(0, 40, 10), np.arange(
diff --git a/pandas/tests/indexes/test_category.py b/pandas/tests/indexes/test_category.py
--- a/pandas/tests/indexes/test_category.py
+++ b/pandas/tests/indexes/test_category.py
@@ -540,6 +540,17 @@ def test_reindex_dtype(self):
         tm.assert_numpy_array_equal(indexer,
                                     np.array([0, 3, 2], dtype=np.intp))
 
+    def test_reindex_duplicate_target(self):
+        # See GH23963
+        c = CategoricalIndex(['a', 'b', 'c', 'a'],
+                             categories=['a', 'b', 'c', 'd'])
+        with pytest.raises(ValueError, match='non-unique indexer'):
+            c.reindex(['a', 'a', 'c'])
+
+        with pytest.raises(ValueError, match='non-unique indexer'):
+            c.reindex(CategoricalIndex(['a', 'a', 'c'],
+                                       categories=['a', 'b', 'c', 'd']))
+
     def test_reindex_empty_index(self):
         # See GH16770
         c = CategoricalIndex([])