PERF: building MultiIndex with categorical levels
State: closed

- [x] closes #22044
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

```python
df = pd.DataFrame({
    'a': np.arange(1_000_000, dtype=np.int32),
    'b': np.arange(1_000_000, dtype=np.int64),
    'c': np.arange(1_000_000, dtype=float),
}).astype({'a': 'category', 'b': 'category'})

%timeit df.set_index(['a', 'b'])
```
On my machine, this takes ~20ms. The current implem takes 140ms.

Any suggestion for tests ?

Labels:
name: Performance
description: Memory or execution speed performance
name: MultiIndex
name: Categorical
description: Categorical Data Type

Comments:
> Any suggestion for tests ?

For performance related items you can create a benchmark in `asv_bench/benchmarks` probably in the multiindex_object.py module for this (though categorical.py and ctors.py maybe work to dependent on how you build test)
@WillAyd 
I can't think of any possible downside in performance since this is just avoiding the rebuild of the codes (which are just 0..N-1 by definition) through lookup. 

I was thinking more in terms of possible regressions. I'm not sure what I should really test or worry for. But I couldn't find any test using a MultiIndex with at least a CategoricalIndex level...
Hello @0x0L! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-06-08 19:56:51 UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=h1) Report
> Merging [#26721](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/3937fbc5fa7d4228c65e863a24d69fa4b7cf4ec7?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26721/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26721      +/-   ##
==========================================
- Coverage   91.78%   91.77%   -0.01%     
==========================================
  Files         174      174              
  Lines       50703    50703              
==========================================
- Hits        46538    46534       -4     
- Misses       4165     4169       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.37% <100%> (ø)` | :arrow_up: |
| #single | `41.81% <100%> (-0.09%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/26721/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.92% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26721/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26721/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.88% <0%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=footer). Last update [3937fbc...59a3c67](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=h1) Report
> Merging [#26721](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/3937fbc5fa7d4228c65e863a24d69fa4b7cf4ec7?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26721/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26721      +/-   ##
==========================================
- Coverage   91.78%   91.77%   -0.01%     
==========================================
  Files         174      174              
  Lines       50703    50703              
==========================================
- Hits        46538    46534       -4     
- Misses       4165     4169       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.37% <100%> (ø)` | :arrow_up: |
| #single | `41.81% <100%> (-0.09%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/26721/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.92% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26721/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26721/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.88% <0%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=footer). Last update [3937fbc...59a3c67](https://codecov.io/gh/pandas-dev/pandas/pull/26721?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

thanks @0x0L very nice!