Deprecate SparseDataFrame and SparseSeries
State: closed

Closes https://github.com/pandas-dev/pandas/issues/19239

This currently includes the changes from https://github.com/pandas-dev/pandas/pull/25682, which I think is mergeable.

I think this would be good to have for 0.25.0. I think it's close, but I may not have time to push this across the finish line. Anyone interested in finishing it off?

Labels:
name: Sparse
description: Sparse Data Type
name: Deprecate
description: Functionality to remove in pandas

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=h1) Report
> Merging [#26137](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/612c24433b8486eb6b6b0013354dd6d6a296c100?src=pr&el=desc) will **decrease** coverage by `0.57%`.
> The diff coverage is `30.43%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26137/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26137      +/-   ##
==========================================
- Coverage   41.31%   40.73%   -0.58%     
==========================================
  Files         174      175       +1     
  Lines       50749    52432    +1683     
==========================================
+ Hits        20968    21360     +392     
- Misses      29781    31072    +1291
```

| Flag | Coverage Δ | |
|---|---|---|
| #single | `40.73% <30.43%> (-0.58%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `49.31% <100%> (+0.1%)` | :arrow_up: |
| [pandas/core/sparse/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL3Nlcmllcy5weQ==) | `44.24% <100%> (+0.49%)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `34.57% <100%> (-0.12%)` | :arrow_down: |
| [pandas/core/arrays/sparse.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3NwYXJzZS5weQ==) | `38.9% <21.05%> (+0.01%)` | :arrow_up: |
| [pandas/core/sparse/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2ZyYW1lLnB5) | `28.76% <60%> (+0.49%)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-48.69%)` | :arrow_down: |
| [pandas/core/arrays/array\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2FycmF5Xy5weQ==) | `15.55% <0%> (-22.23%)` | :arrow_down: |
| [pandas/core/sorting.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc29ydGluZy5weQ==) | `22.22% <0%> (-4.12%)` | :arrow_down: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `30.27% <0%> (-4.03%)` | :arrow_down: |
| [pandas/core/indexes/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy90aW1lZGVsdGFzLnB5) | `44.38% <0%> (-3.07%)` | :arrow_down: |
| ... and [81 more](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=footer). Last update [612c244...836d19b](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=h1) Report
> Merging [#26137](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/7f0a7435e38b366eddb59036291bd3e166dac694?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26137/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26137      +/-   ##
==========================================
- Coverage   91.77%   91.76%   -0.01%     
==========================================
  Files         174      174              
  Lines       50639    50646       +7     
==========================================
+ Hits        46473    46476       +3     
- Misses       4166     4170       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.3% <100%> (ø)` | :arrow_up: |
| #single | `41.69% <45.45%> (-0.09%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `93.61% <ø> (ø)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97% <ø> (-0.12%)` | :arrow_down: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `93.57% <ø> (ø)` | :arrow_up: |
| [pandas/core/sparse/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL3Nlcmllcy5weQ==) | `93.24% <100%> (+0.06%)` | :arrow_up: |
| [pandas/core/arrays/sparse.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3NwYXJzZS5weQ==) | `93.07% <100%> (ø)` | :arrow_up: |
| [pandas/core/sparse/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2ZyYW1lLnB5) | `95.65% <100%> (+0.01%)` | :arrow_up: |
| [pandas/core/sparse/scipy\_sparse.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL3NjaXB5X3NwYXJzZS5weQ==) | `100% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26137/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=footer). Last update [7f0a743...12d8d83](https://codecov.io/gh/pandas-dev/pandas/pull/26137?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Note: we still have some warnings leaking through on some of the CI jobs (just not numpydev). Trying to track those down.
I think I got all the warnings... I added a global `filterwarnings` to our setup.cfg https://github.com/pandas-dev/pandas/pull/26137/files#diff-380c6a8ebbbce17d55d50ef17d3cf906. This proved helpful in tracking them down. Are people OK with keeping that there? As an aside, I couldn't get the syntax for "raise on all warnings from pandas" to work. In theory `error:::pandas[.*]` should do it, but that was still elevating warnings from other packages.
will have a look soon
I've updated the doc examples to all use Series[sparse], rather than SparseSeries. I've just left a note that the sparse subclasses are deprecated.
https://github.com/pandas-dev/pandas/pull/26137/commits/c5fa3fb984dd3d999c7909f5d3bd8ff8ec521533 also has a change to Series.sparse.from_coo. Previously that was using SparseSeries internally, and so a warning was being raised. I (lazily) applied the warnings filter to the class so it was being ignored in the test.
thanks @TomAugspurger 

Thanks. I'll make a PR with these doc updates.

On Wed, May 29, 2019 at 8:51 AM Simon Hawkins <notifications@github.com>
wrote:

> *@simonjayhawkins* commented on this pull request.
> ------------------------------
>
> In pandas/core/sparse/series.py
> <https://github.com/pandas-dev/pandas/pull/26137#discussion_r288577643>:
>
> >  class SparseSeries(Series):
>      """Data structure for labeled, sparse floating point data
>
> +    .. deprectaed:: 0.25.0
>
> typo
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/26137?email_source=notifications&email_token=AAKAOITMLOLJFTRQDF54URLPX2C5BA5CNFSM4HG7LDP2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB2ABB5A#pullrequestreview-243273972>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIS7PLLTJDVHORMM4YDPX2C5BANCNFSM4HG7LDPQ>
> .
>