Allows for merging of SparseDataFrames, and fixes __array__ interface
State: closed

- [x] closes #13665 and #14167  and #19506
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This is still an in progress thing... since this is pretty complected code that I'm trying to unravel.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Sparse
description: Sparse Data Type

Comments:
Hello @hexgnu! Thanks for updating the PR.

























Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on February 12, 2018 at 05:07 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19488?src=pr&el=h1) Report
> Merging [#19488](https://codecov.io/gh/pandas-dev/pandas/pull/19488?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cdfce2b0ad99f7faad57cc5247cf33aab5725bed?src=pr&el=desc) will **decrease** coverage by `0.19%`.
> The diff coverage is `96.87%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/19488/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/19488?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master   #19488     +/-   ##
=========================================
- Coverage   91.82%   91.62%   -0.2%     
=========================================
  Files         152      150      -2     
  Lines       49248    48833    -415     
=========================================
- Hits        45222    44744    -478     
- Misses       4026     4089     +63
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90% <96.87%> (-0.22%)` | :arrow_down: |
| #single | `41.72% <28.12%> (-0.18%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/19488?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.44% <100%> (-0.25%)` | :arrow_down: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.28% <100%> (+0.02%)` | :arrow_up: |
| [pandas/core/algorithms.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYWxnb3JpdGhtcy5weQ==) | `94.16% <100%> (-0.2%)` | :arrow_down: |
| [pandas/core/sparse/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL3Nlcmllcy5weQ==) | `95.31% <100%> (+0.1%)` | :arrow_up: |
| [pandas/core/sparse/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2ZyYW1lLnB5) | `95.28% <100%> (+0.46%)` | :arrow_up: |
| [pandas/core/sparse/array.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2FycmF5LnB5) | `91.62% <100%> (+0.3%)` | :arrow_up: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `95.04% <92.85%> (-0.5%)` | :arrow_down: |
| [pandas/tseries/plotting.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvcGxvdHRpbmcucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/core/dtypes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Jhc2UucHk=) | `47.61% <0%> (-44.28%)` | :arrow_down: |
| [pandas/plotting/\_compat.py](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb21wYXQucHk=) | `62% <0%> (-28.91%)` | :arrow_down: |
| ... and [69 more](https://codecov.io/gh/pandas-dev/pandas/pull/19488/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19488?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/19488?src=pr&el=footer). Last update [cdfce2b...353171b](https://codecov.io/gh/pandas-dev/pandas/pull/19488?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Also wanted to note that I would have liked to keep PR's separate (one for each bug) but honestly this code was all tied together.
This is ready for a review now that the tests pass. @TomAugspurger thanks :)

Do you want me to squash commits on here?
No need to squash commits.

Looks like there's a merge conflict in the whatsnew though, if you want to fix that and push again. Taking a look now.
@hexgnu  `SparseArray.__array__` has been fixed to return a dense ndarray with all the values.

Is there anything remaining in this PR that would be useful?
@TomAugspurger I don't think there's anything else we should merge in. Let's just close this PR :) Thanks for all your work!