BUG: #12815 Always use np.nan for missing values of object dtypes
State: closed

- [x] closes #12815
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Handles the issue of unstacked object columns filling missing values with `None` instead of `np.nan` by modifying `pd.core.dtypes.cast.maybe_promote()` to use a `fill_value` of `np.nan` (or `pd.NaT`) when the original `fill_value` is `None`.

Let me know if you'd like me to clarify anything!

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=h1) Report
> Merging [#18313](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c2590b346a7f8e9c0e2e7efa1d5c7318f73fa982?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18313/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18313      +/-   ##
==========================================
- Coverage   91.39%   91.37%   -0.02%     
==========================================
  Files         164      164              
  Lines       49854    49855       +1     
==========================================
- Hits        45566    45557       -9     
- Misses       4288     4298      +10
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.18% <100%> (-0.01%)` | :arrow_down: |
| #single | `39.44% <0%> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `88.19% <100%> (+0.02%)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.38% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=footer). Last update [c2590b3...8ec39ee](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=h1) Report
> Merging [#18313](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c8684239e7803a38a64c2bc85e23a3b87af35e9f?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18313/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18313      +/-   ##
==========================================
- Coverage   91.38%   91.36%   -0.02%     
==========================================
  Files         164      164              
  Lines       49796    49798       +2     
==========================================
- Hits        45507    45500       -7     
- Misses       4289     4298       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.17% <100%> (ø)` | :arrow_up: |
| #single | `39.55% <0%> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `88.19% <100%> (+0.02%)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.54% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=footer). Last update [c868423...5f15554](https://codecov.io/gh/pandas-dev/pandas/pull/18313?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jreback Changes made.
trivial change. push & ping when green.
note if you are interested in cleaning some things up :>

``maybe_promote`` I think will pretty much ignore its fill value for almost all dtypes. So the question is, when is it *not* ignored, can we safely remove it (or really just make the default ``fill_value=None``, and then when its not ``None`` actually do something.

Not sure of the answer here.
@jreback Green. I'll open an issue for the `maybe_promote` changes.
thanks @mattayes