BUG-19214 int categoricals are formatted as ints
State: closed

- [X] closes #19214
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry

This PR alters the `ExtensionArrayFormatter` such that if it receives a categorical series with only integers and `NaN`, it will print the integers as integers instead of converting to floats.

```
>>> # previous behaviour
>>> pd.Series([1,2, np.nan], dtype="object").astype("category")
0    1.0
1    2.0
2    NaN
dtype: category
Categories (2, int64): [1, 2]
>>> # new behaviour
>>> pd.Series([1,2, np.nan], dtype="object").astype("category")
0      1
1      2
2    NaN
dtype: category
Categories (2, int64): [1, 2]
```

Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Categorical
description: Categorical Data Type

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=h1) Report
> Merging [#24494](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/1ebfd8a9fda12cd68c3e068b777c7a0a2470dfaf?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24494/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24494      +/-   ##
==========================================
+ Coverage   92.31%   92.31%   +<.01%     
==========================================
  Files         166      166              
  Lines       52412    52414       +2     
==========================================
+ Hits        48382    48384       +2     
  Misses       4030     4030
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.73% <100%> (ø)` | :arrow_up: |
| #single | `43.05% <50%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `97.98% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=footer). Last update [1ebfd8a...a2593f0](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=h1) Report
> Merging [#24494](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cf922304a16b1d4539b841f56ab2a83496a80384?src=pr&el=desc) will **increase** coverage by `0.05%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24494/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24494      +/-   ##
==========================================
+ Coverage   92.32%   92.38%   +0.05%     
==========================================
  Files         166      166              
  Lines       52440    52379      -61     
==========================================
- Hits        48417    48388      -29     
+ Misses       4023     3991      -32
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.8% <ø> (+0.05%)` | :arrow_up: |
| #single | `43.01% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.43% <ø> (-0.05%)` | :arrow_down: |
| [pandas/core/internals/construction.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2NvbnN0cnVjdGlvbi5weQ==) | `95.93% <0%> (-0.73%)` | :arrow_down: |
| [pandas/core/nanops.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvbmFub3BzLnB5) | `94.36% <0%> (-0.55%)` | :arrow_down: |
| [pandas/core/dtypes/concat.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2NvbmNhdC5weQ==) | `96.6% <0%> (-0.46%)` | :arrow_down: |
| [pandas/core/internals/concat.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2NvbmNhdC5weQ==) | `96.45% <0%> (-0.39%)` | :arrow_down: |
| [pandas/core/algorithms.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYWxnb3JpdGhtcy5weQ==) | `94.77% <0%> (-0.34%)` | :arrow_down: |
| [pandas/core/indexes/period.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9wZXJpb2QucHk=) | `92.06% <0%> (-0.32%)` | :arrow_down: |
| [pandas/core/indexes/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy90aW1lZGVsdGFzLnB5) | `90.22% <0%> (-0.26%)` | :arrow_down: |
| [pandas/core/arrays/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lcy5weQ==) | `97.84% <0%> (-0.08%)` | :arrow_down: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.29% <0%> (-0.07%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/pandas-dev/pandas/pull/24494/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=footer). Last update [cf92230...7cc02c1](https://codecov.io/gh/pandas-dev/pandas/pull/24494?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jreback I looked into it and the `CategoricalFormatter` is used for printing categoricals like this:
```
[1.0, 2.0, NaN]
Categories (2, int64): [1, 2]
```
instead of like a series. The PR now modifies the `CategoricalFormatter` in addition to the `ExtensionArrayFormatter`.
@TomAugspurger any objections here?
> To me, the change in dtype for concat is the larger change. Just from the release notes it looks like the formatting change is larger.

I don't think this is that big of a deal, do you want a sub-section for this?
@TomAugspurger you think need to highlite the change more?
The formatting change seems fine. I'm more concerned about the breaking
change to get_values
with no deprecation warning. Is there any way to deprecate the behavior, or
would that require to
many internal changes so that the warning doesn't show up when printing the
categorical?

On Wed, Jan 2, 2019 at 6:57 PM Jeff Reback <notifications@github.com> wrote:

> @TomAugspurger <https://github.com/TomAugspurger> you think need to
> highlite the change more?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/24494#issuecomment-451031339>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIvMndIhl1YZHo1Iye0ANfYxd0P88ks5u_VV8gaJpZM4Zk2sS>
> .
>

> I'm more concerned about the breaking
change to get_values
with no deprecation warning. Is there any way to deprecate the behavior, or
would that require to

This is a very small case to be honest (and our current behavior for concatting any non-same types, so this is a bug IMHO). 

@JustinZhengBC can you make a small sub-section about the changes in Categorical int-like concat, showing previous and current behavior. I think that is fine.

@TomAugspurger better?
@TomAugspurger ?
thanks @JustinZhengBC keep em coming!