BUG: reindex would throw when a categorical index was empty #16770
State: closed

closes #16770 
tests added
passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff`` (On Windows, ``git diff upstream/master -u -- "*.py" | flake8 --diff`` might work as an alternative.)
whatsnew entry (0.20.3 release)


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=h1) Report
> Merging [#16820](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/9e55af2552664267ce27c7fc6c932acb0651c259?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16820/graphs/tree.svg?width=650&src=pr&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16820      +/-   ##
==========================================
+ Coverage   90.94%   90.94%   +<.01%     
==========================================
  Files         161      161              
  Lines       49272    49275       +3     
==========================================
+ Hits        44811    44814       +3     
  Misses       4461     4461
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.71% <100%> (ø)` | :arrow_up: |
| #single | `40.17% <50%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9jYXRlZ29yeS5weQ==) | `98.2% <100%> (+0.01%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=footer). Last update [9e55af2...3ab37e6](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=h1) Report
> Merging [#16820](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/34210ac4d8c61ec4d695baba24d84bd7a1826af4?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16820/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16820      +/-   ##
==========================================
- Coverage   90.99%   90.98%   -0.02%     
==========================================
  Files         161      161              
  Lines       49294    49296       +2     
==========================================
- Hits        44857    44850       -7     
- Misses       4437     4446       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.75% <100%> (ø)` | :arrow_up: |
| #single | `40.19% <66.66%> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9jYXRlZ29yeS5weQ==) | `98.5% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.76% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=footer). Last update [34210ac...0e2d315](https://codecov.io/gh/pandas-dev/pandas/pull/16820?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Justification: can't check for len > 0 within take at a lower level than this as expected behaviour of take is to throw if you pass an index out of range.  So have to check before this is called.  Converting the indexer into a np array is needed to ensure the correct return type of the function -- currently this is of an unexpected Index type as _reindex_non_unique -> get_index_non_unique is returning as an Index rather than the documented numpy array.  Suspect is not ok to change this type at a lower level (although that would be correct if it was new code) as it would break user code (and certainly break existing tests although those could easily be changed).
also, pls add some general tests in ``tests/indexes/test_base.py`` which exercise this comparison for allindexers

e.g. something like

```
def get_indexer_consistency(self):

       for name, index in self.indices.items():
             expected = index.get_indexer(index[0:2])
             result, _ = index.get_indexer_non_unique(index[0:2])
             tm.assert_numpy_array_equal(result, expected)
```

@ri938 pls remove the change in indexer from here and (separate) PR of #16819 
Maybe should be in 0.21.0 release(?) as fixing this bug lets code run to a point where the bug from issue https://github.com/pandas-dev/pandas/pull/16826 is hit causing the return type of reindex to be an Index rather than numpy array. This causes test to fail.  Since that issue is to be fixed for 0.21 release, this should prob go afterwards?
ok, so this is dependent on #16826 that is fine. We'll merge that one first.
ok merged other. rebase and ping on green.
looks like you are picking up other changes.

```
git rebase -i upstream/master
git push <yourbrancch> <yourfork> -f
```
Ready to merge?
Should be good now.
thanks!