Repository: pandas-dev/pandas
Pull request: TST: GH28813 test .diff() on Sparse dtype
- [x] closes #28813 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Add a quick test to confirm that .diff() function handles sparse dataframes the same way as regular dataframes. 

First time making a pandas PR so comments, advice, critiques are encouraged.

Problem statement:
Diff on sparse dtype
Correct behaviour without Sparse[int]:

```python
a = [[0, 1], [1, 0]]
print(pd.DataFrame(a).diff())
```
yields:
```
     0    1
0  NaN  NaN
1  1.0 -1.0
```

Current behaviour (doesn't work):
```python
a = [[0, 1], [1, 0]]
print(pd.DataFrame(a, dtype='Sparse[int]').diff())
```
yields:

```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-34-3cc99aac7c69> in <module>
      3 print(pd.DataFrame(a).diff())
      4 
----> 5 print(pd.DataFrame(a, dtype='Sparse[int]').diff())

~/.local/lib/python3.6/site-packages/pandas/core/frame.py in diff(self, periods, axis)
   6592         """
   6593         bm_axis = self._get_block_manager_axis(axis)
-> 6594         new_data = self._data.diff(n=periods, axis=bm_axis)
   6595         return self._constructor(new_data)
   6596 

~/.local/lib/python3.6/site-packages/pandas/core/internals/managers.py in diff(self, **kwargs)
    564 
    565     def diff(self, **kwargs):
--> 566         return self.apply("diff", **kwargs)
    567 
    568     def interpolate(self, **kwargs):

~/.local/lib/python3.6/site-packages/pandas/core/internals/managers.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
    436                     kwargs[k] = obj.reindex(b_items, axis=axis, copy=align_copy)
    437 
--> 438             applied = getattr(b, f)(**kwargs)
    439             result_blocks = _extend_blocks(applied, result_blocks)
    440 

~/.local/lib/python3.6/site-packages/pandas/core/internals/blocks.py in diff(self, n, axis)
   1328     def diff(self, n, axis=1):
   1329         """ return block for the diff of the values """
-> 1330         new_values = algos.diff(self.values, n, axis=axis)
   1331         return [self.make_block(values=new_values)]
   1332 

~/.local/lib/python3.6/site-packages/pandas/core/algorithms.py in diff(arr, n, axis)
   1943 
   1944     na_indexer = [slice(None)] * arr.ndim
-> 1945     na_indexer[axis] = slice(None, n) if n >= 0 else slice(n, None)
   1946     out_arr[tuple(na_indexer)] = na
   1947 

IndexError: list assignment index out of range```

Hints from issue discussion:
This looks to work on master. Could use a test

```
In [25]: a = [[0, 1], [1, 0]]
    ...: print(pd.DataFrame(a, dtype='Sparse[int]').diff())
     0    1
0  NaN  NaN
1  1.0 -1.0

In [26]: pd.DataFrame(a, dtype='Sparse[int]').diff().dtypes
Out[26]:
0    Sparse[float64, 0.0]
1    Sparse[float64, 0.0]
dtype: object

In [27]: pd.__version__
Out[27]: '1.1.0.dev0+1361.g77a0f19c5'
```
take

Commit message: TST: GH28813 test .diff() on Sparse dtype

Patch to review:
diff --git a/pandas/tests/frame/methods/test_diff.py b/pandas/tests/frame/methods/test_diff.py
index 6a9248e1cba1e..4ab0e80b73d90 100644
--- a/pandas/tests/frame/methods/test_diff.py
+++ b/pandas/tests/frame/methods/test_diff.py
@@ -36,6 +36,14 @@ def test_diff(self, datetime_frame):
         ).astype("float64")
         tm.assert_frame_equal(result, expected)
 
+        # Result should be the same for sparse df, see GH28813
+        arr = [[0, 1], [1, 0]]
+        normal = pd.DataFrame(arr)
+        sparse = pd.DataFrame(arr, dtype='Sparse[int]')
+        # we don't check dtype because one is sparse and the other isn't
+        tm.assert_frame_equal(normal.diff(), sparse.diff(), check_dtype=False)
+
+
     @pytest.mark.parametrize("tz", [None, "UTC"])
     def test_diff_datetime_axis0(self, tz):
         # GH#18578


Code review comments:
diff_hunk: @@ -36,6 +36,14 @@ def test_diff(self, datetime_frame):
         ).astype("float64")
         tm.assert_frame_equal(result, expected)
 
+        # Result should be the same for sparse df, see GH28813
+        arr = [[0, 1], [1, 0]]
+        normal = pd.DataFrame(arr)
+        sparse = pd.DataFrame(arr, dtype='Sparse[int]')
+        # we don't check dtype because one is sparse and the other isn't
+        tm.assert_frame_equal(normal.diff(), sparse.diff(), check_dtype=False)
path: pandas/tests/frame/methods/test_diff.py
text: @user1:
Ideally we want to create the expected DataFrame with just the constructor:

```
result = sparse.diff()
expected = pd.DataFrame(...)
tm.assert_frame_equal(result, expected)
```