Repository: pandas-dev/pandas
Pull request: BUG: pandas.DataFrame().stack() raise an error, while expected is empty
- [ ] closes #36113 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Problem statement:
BUG: pandas.DataFrame().stack() raise an error, while expected is empty
- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this bug exists on the latest version of pandas.

- [ ] (optional) I have confirmed this bug exists on the master branch of pandas.

---

#### Code Sample, a copy-pastable example

```python
pandas.DataFrame().stack()
pandas.DataFrame().stack(dropna = True)
```

#### Problem description

I expect that stack() of empty dataframe will be empty dataframe and not an exception

Hints from issue discussion:
Thanks for the report. I think the expected output should be an empty Series rather than an empty DataFrame. What do you think?
@TomAugspurger 
I expect a dataframe, because if input is not empty, then the method returns a dataframe
Doesn't DataFrame.stack return a Series? Or do I misunderstand?
From the code on ``stack`` method:

```
from pandas.core.reshape.reshape import stack, stack_multiple

if isinstance(level, (tuple, list)):
    return stack_multiple(self, level, dropna=dropna)
else:
    return stack(self, level, dropna=dropna)
```
By default ``level=-1``.
Meanwhile ``pandas.core.reshape.reshape.stack`` returns a Series.
@TomAugspurger 
All of my use cases were about multiindex, so .stack() returns a dataframe for my cases.
But I checked the documentation and it returns a Series if it's just one level.
So let's use the logic @ivanovmg  described

Commit message: simply fixes and parameterize tests

Patch to review:
diff --git a/doc/source/whatsnew/v1.2.0.rst b/doc/source/whatsnew/v1.2.0.rst
index 9a778acba4764..d28bbdedec8e8 100644
--- a/doc/source/whatsnew/v1.2.0.rst
+++ b/doc/source/whatsnew/v1.2.0.rst
@@ -322,7 +322,7 @@ Reshaping
 - Bug in :meth:`DataFrame.pivot_table` with ``aggfunc='count'`` or ``aggfunc='sum'`` returning ``NaN`` for missing categories when pivoted on a ``Categorical``. Now returning ``0`` (:issue:`31422`)
 - Bug in :func:`union_indexes` where input index names are not preserved in some cases. Affects :func:`concat` and :class:`DataFrame` constructor (:issue:`13475`)
 - Bug in func :meth:`crosstab` when using multiple columns with ``margins=True`` and ``normalize=True`` (:issue:`35144`)
--
+- Bug in :meth:`DataFrame.stack` for empty DataFrame (:issue:`36113`)
 
 Sparse
 ^^^^^^
diff --git a/pandas/core/reshape/reshape.py b/pandas/core/reshape/reshape.py
index e81dd8f0c735c..3f482064cdea2 100644
--- a/pandas/core/reshape/reshape.py
+++ b/pandas/core/reshape/reshape.py
@@ -513,7 +513,7 @@ def factorize(index):
             verify_integrity=False,
         )
 
-    if frame._is_homogeneous_type:
+    if not frame.empty and frame._is_homogeneous_type:
         # For homogeneous EAs, frame._values will coerce to object. So
         # we concatenate instead.
         dtypes = list(frame.dtypes._values)
diff --git a/pandas/tests/frame/test_reshape.py b/pandas/tests/frame/test_reshape.py
index d80ebaa09b6a8..e86731e5aa599 100644
--- a/pandas/tests/frame/test_reshape.py
+++ b/pandas/tests/frame/test_reshape.py
@@ -1273,6 +1273,23 @@ def test_stack_timezone_aware_values():
     tm.assert_series_equal(result, expected)
 
 
+@pytest.mark.parametrize("dropna", [True, False])
+def test_stack_empty_frame(dropna):
+    # GH 36113
+    expected = Series(index=MultiIndex([[], []], [[], []]), dtype=np.float64)
+    result = DataFrame().stack(dropna=dropna)
+    tm.assert_series_equal(result, expected)
+
+
+@pytest.mark.parametrize("dropna", [True, False])
+@pytest.mark.parametrize("fill_value", [None, 0])
+def test_stack_unstack_empty_frame(dropna, fill_value):
+    # GH 36113
+    expected = DataFrame()
+    result = DataFrame().stack(dropna=dropna).unstack(fill_value=fill_value)
+    tm.assert_frame_equal(result, expected)
+
+
 def test_unstacking_multi_index_df():
     # see gh-30740
     df = DataFrame(


Code review comments:
diff_hunk: @@ -1273,6 +1273,23 @@ def test_stack_timezone_aware_values():
     tm.assert_series_equal(result, expected)
 
 
+@pytest.mark.parametrize("dropna", [True, False])
+def test_stack_empty_frame(dropna):
+    # GH 36113
+    expected = Series(index=MultiIndex([[], []], [[], []]), dtype=np.float64)
path: pandas/tests/frame/test_reshape.py
text: @user2:
even though this is nice now that it returns a series that can now unstack easily, it seems actually pretty fragile.

e.g. if we try to unstack with a serires of no or 1 levels it will fail with different errors.

I think it might be more appropriate to actually raise a nice exception rather than return something like this.

cc @user1 

@author:
@user2 I agree. Looking at the [guide](https://github.com/pandas-dev/pandas/wiki/Choosing-Exceptions-to-Raise), it seems that a ValueError would do with a proper message? Is there any pandas-provided exception that is more appropriate?

@user2:
yeah that's fine, with a nicely workded message :->

```
In [2]: pd.Series().unstack()                                                                                                                                    
/nas/local/simscripts/jreback/.nest/pandas/bin/ipython:1: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
  #!/usr/bin/python2.7
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-0ecdefda3bf9> in <module>
----> 1 pd.Series().unstack()

~/pandas-dev/pandas/core/series.py in unstack(self, level, fill_value)
   3910         from pandas.core.reshape.reshape import unstack
   3911 
-> 3912         return unstack(self, level, fill_value)
   3913 
   3914     # ----------------------------------------------------------------------

~/pandas-dev/pandas/core/reshape/reshape.py in unstack(obj, level, fill_value)
    421             return _unstack_extension_series(obj, level, fill_value)
    422         unstacker = _Unstacker(
--> 423             obj.index, level=level, constructor=obj._constructor_expanddim
    424         )
    425         return unstacker.get_result(

~/pandas-dev/pandas/core/reshape/reshape.py in __init__(self, index, level, constructor)
     88         self.constructor = constructor
     89 
---> 90         self.index = index.remove_unused_levels()
     91 
     92         self.level = self.index._get_level_number(level)

AttributeError: 'Index' object has no attribute 'remove_unused_levels'

In [3]: pd.DataFrame().unstack()                                                                                                                                 
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-3-841b2b512233> in <module>
----> 1 pd.DataFrame().unstack()

~/pandas-dev/pandas/core/frame.py in unstack(self, level, fill_value)
   7197         from pandas.core.reshape.reshape import unstack
   7198 
-> 7199         return unstack(self, level, fill_value)
   7200 
   7201     @user3(

~/pandas-dev/pandas/core/reshape/reshape.py in unstack(obj, level, fill_value)
    416             return _unstack_frame(obj, level, fill_value=fill_value)
    417         else:
--> 418             return obj.T.stack(dropna=False)
    419     else:
    420         if is_extension_array_dtype(obj.dtype):

~/pandas-dev/pandas/core/frame.py in stack(self, level, dropna)
   7055             return stack_multiple(self, level, dropna=dropna)
   7056         else:
-> 7057             return stack(self, level, dropna=dropna)
   7058 
   7059     def explode(

~/pandas-dev/pandas/core/reshape/reshape.py in stack(frame, level, dropna)
    518         # we concatenate instead.
    519         dtypes = list(frame.dtypes._values)
--> 520         dtype = dtypes[0]
    521 
    522         if is_extension_array_dtype(dtype):

IndexError: list index out of range

In [4]:                                                                                                                                                          
```

these should both give helpful ValueError messages (e.g. you are covering the 2nd, but if you can test the first as well)

@user1:
At least for Dask, and I think other cases, having empty inputs behave identically to non-empty inputs is ideal.

So IIUC for DataFrame.stack the requirements are that

1. The result type is a Series
2. The result index is a MultiIndex with the product of the original index and columns

```python
In [50]: c = pd.DataFrame(1, index=[0, 1], columns=['a', 'b'])

In [51]: c.stack()
Out[51]:
0  a    1
   b    1
1  a    1
   b    1
dtype: int64
```

So for an empty dataframe with columns, that would perhaps be the following (which looks right on master)

```python
In [65]: b = pd.DataFrame(columns=['a', 'b'])

In [66]: b.stack().index.levels
Out[66]: FrozenList([[], ['a', 'b']])
```

On master, an empty DataFrame with an index but no columns raises with the IndexError noted in the issue. But I think the MultiIndex sho7uld be something like

```python
In [69]: a = pd.DataFrame(index=[0, 1])

In [70]: a.stack().index
# In [76]: pd.MultiIndex([[0, 1], []], [[], []])
Out[76]: MultiIndex([], )
```

And for an empty DataFrame with no index / columns, I think the result would be an empty multiindex.

Does that make sense? Then I think that we can make a sensible `.unstack()`.

@user2:
ok i am fine with this too (@user1 suggestion), but if in this PR we can update the other error message will be great.

@author:
@user1 @user2, I did not know we can create empty DataFrame with non-empty index/columns. In the two cases above, stack unstack round-trip actually failed.
`a.stack().unstack().index -> Int64Index([], dtype='int64')`
and 
`b.stack().unstack().columns -> Index([], dtype='object')`
we lose the index values in both cases, Iet me look into this as well...