Repository: pandas-dev/pandas

Issue:
SparseArray compatibility issues in e.g. sklearn. Series.shape and SparseArray.shape output not consistent.
#### Code Sample

```python
import pandas as pd
import numpy as np
spdf = pd.DataFrame(np.random.rand(5,5)>0.7).astype(float).to_sparse(fill_value=0)
spdf[1].shape[0] == spdf[1].values.shape[0]
```

#### Problem description
Using packages that internally translate pandas dataframes to arrays (like sklearn) requires vectors and matrices to be specific sizes. Using sparse data matrices in pandas produces unpredictable results as internals of sklearn will translate pandas to SparseArray arrays and check sizes. The size of `spdf[1].values.shape` is not guaranteed to have the expected shape output.

Also depending on the order of columns selection (after or before applying values) gives different results as the output of `spdf.values` is a `numpy.ndarray`.

#### Expected Output
I would expect the last statement in the code above to return True for all vectors in spdf.
I'm not sure this is a bug or intended feature but it struck me as weird as well as inconvenient when working with different libraries.
I encountered this running sklearn.linear_model.Lasso with a sparse target vector.

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.6.0-040600-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.0
pytest: None
pip: 10.0.1
setuptools: 39.1.0
Cython: None
numpy: 1.14.2
scipy: 1.0.1
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: None
patsy: 0.5.0
dateutil: 2.7.2
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.2.2
openpyxl: 2.4.9
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml: 4.1.1
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.1.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>



Discussion:
Seems like we inherit shape from `ndarray`. Should be enough to override it with

```python
@property
def shape(self):
    return (len(self),)
```

Interested in making a PR?
@Xparx. Let me know if you're going to work on this. If not, I would be interested in picking this ticket up.
@nprad. Feel free to pick this up. 

Resolving patch:
diff --git a/pandas/core/sparse/array.py b/pandas/core/sparse/array.py
--- a/pandas/core/sparse/array.py
+++ b/pandas/core/sparse/array.py
@@ -290,6 +290,7 @@ def __reduce__(self):
         """Necessary for making this object picklable"""
         object_state = list(np.ndarray.__reduce__(self))
         subclass_state = self.fill_value, self.sp_index
+        object_state[2] = self.sp_values.__reduce__()[2]
         object_state[2] = (object_state[2], subclass_state)
         return tuple(object_state)
 
@@ -339,6 +340,10 @@ def values(self):
         output.put(int_index.indices, self)
         return output
 
+    @property
+    def shape(self):
+        return (len(self),)
+
     @property
     def sp_values(self):
         # caching not an option, leaks memory


Test patch:
diff --git a/pandas/tests/sparse/test_array.py b/pandas/tests/sparse/test_array.py
--- a/pandas/tests/sparse/test_array.py
+++ b/pandas/tests/sparse/test_array.py
@@ -454,6 +454,17 @@ def test_values_asarray(self):
         assert_almost_equal(self.arr.to_dense(), self.arr_data)
         assert_almost_equal(self.arr.sp_values, np.asarray(self.arr))
 
+    @pytest.mark.parametrize('data,shape,dtype', [
+        ([0, 0, 0, 0, 0], (5,), None),
+        ([], (0,), None),
+        ([0], (1,), None),
+        (['A', 'A', np.nan, 'B'], (4,), np.object)
+    ])
+    def test_shape(self, data, shape, dtype):
+        # GH 21126
+        out = SparseArray(data, dtype=dtype)
+        assert out.shape == shape
+
     def test_to_dense(self):
         vals = np.array([1, np.nan, np.nan, 3, np.nan])
         res = SparseArray(vals).to_dense()