Repository: pandas-dev/pandas

Issue:
DOC: get_dummies behaviour with dummy_na = True is counter-intuitive / incorrect when no NaN values present
### Pandas version checks

- [X] I have checked that the issue still exists on the latest versions of the docs on `main` [here](https://pandas.pydata.org/docs/dev/)


### Location of the documentation

https://pandas.pydata.org/docs/reference/api/pandas.get_dummies.html

### Documentation problem

The [docs](https://pandas.pydata.org/docs/reference/api/pandas.get_dummies.html) currently read for this function:

> Add a column to indicate NaNs, if False NaNs are ignored.

However, when no NaN values are present, a useless constant NaN indicator column is still added:

```python
>>> df = pd.DataFrame(pd.Series([0, 1], dtype="int64"))
>>> df
   0
0  0
1  1
>>> pd.get_dummies(df, columns=[0], dummy_na=True, drop_first=True)
   0_1.0  0_nan
0  False  False
1   True  False
>>> pd.get_dummies(df, columns=[0], dummy_na=True)
   0_0.0  0_1.0  0_nan
0   True  False  False
1  False   True  False
```

This is arguably quite unexpected behaviour, as constant columns do not contain information except in some very rare cases and for specific custom models. 

I.e. for almost any kind of model this column will be ignored (but then annoyingly clutter e.g. `.feature_importances` variables and also perhaps needlessly increase compute times for algorithms that scale significantly with the number of features, but which may not have methods for ignoring constant input features). For data with a lot of binary features, and pipelines or models which also might do e.g. conversion to floating-point dtypes, all these useless extra constant features could also significantly increase memory requirements (especially in multiprocessing contexts). 

I imagine the intended design decision is that if you do e.g. `df1, df2 = train_test_split(df)`, and it ends up such that `df1` doesn't have any NaN values for some feature `"f"`, but `df2` does, then at least with the current implementation, the user can be ensured the following does not raise an AssertionError:

```python
dummies1 = pd.get_dummies(df1, ["f"], dummy_na=True)
dummies2 = pd.get_dummies(df2, ["f"], dummy_na=True)
assert dummies1.columns.tolist() == dummies2.columns.tolist()
```

But still, that is a pretty strange use-case, as dummification should generally happen right at the beginning on the full data. 

In my opinion the default behaviour should to _**only** add a NaN indicator column if NaN values are actually present..._ . I actually would consider this to be an implementation or design bug, for like 99% of use-cases. But at bare minimum this undesirable and unexpected behaviour should be documented with some reasoning. 

### Suggested fix for documentation

Just change the docs to:

> Add a column to indicate NaNs. If True, A NaN indicator column will be added even if no NaN values are present [insert reasoning here]. If False, NaNs are ignored [actually "ignored" is extremely unclear too, as per [this issue](https://github.com/pandas-dev/pandas/issues/15923)].

Discussion:
saldanhad: Hi @mroeschke, RFR for this and the pending PR on `functools`. Thanks!

Patch:
diff --git a/pandas/core/groupby/ops.py b/pandas/core/groupby/ops.py
index a82e77140d..b32119a2dd 100644
--- a/pandas/core/groupby/ops.py
+++ b/pandas/core/groupby/ops.py
@@ -867,7 +867,7 @@ class BaseGrouper:
             names=names,
             verify_integrity=False,
         )
-        if not consistent_sorting:
+        if not consistent_sorting and len(ob_index) > 0:
             # Sort by the levels where the corresponding sort argument is True
             n_levels = len(sorts)
             drop_levels = [
diff --git a/pandas/core/reshape/encoding.py b/pandas/core/reshape/encoding.py
index c397c1c256..33ff182f5b 100644
--- a/pandas/core/reshape/encoding.py
+++ b/pandas/core/reshape/encoding.py
@@ -68,7 +68,8 @@ def get_dummies(
         If appending prefix, separator/delimiter to use. Or pass a
         list or dictionary as with `prefix`.
     dummy_na : bool, default False
-        Add a column to indicate NaNs, if False NaNs are ignored.
+        If True, a NaN indicator column will be added even if no NaN values are present.
+        If False, NA values are encoded as all zero.
     columns : list-like, default None
         Column names in the DataFrame to be encoded.
         If `columns` is None then all the columns with


Test patch:
diff --git a/pandas/tests/groupby/methods/test_value_counts.py b/pandas/tests/groupby/methods/test_value_counts.py
index 8f3022fbe5..8ca6593a19 100644
--- a/pandas/tests/groupby/methods/test_value_counts.py
+++ b/pandas/tests/groupby/methods/test_value_counts.py
@@ -1219,3 +1219,25 @@ def test_value_counts_sort_categorical(sort, vc_sort, normalize):
     expected = expected.take(taker)
 
     tm.assert_series_equal(result, expected)
+
+
+@pytest.mark.parametrize("groupby_sort", [True, False])
+def test_value_counts_all_na(sort, dropna, groupby_sort):
+    # GH#59989
+    df = DataFrame({"a": [2, 1, 1], "b": np.nan})
+    gb = df.groupby("a", sort=groupby_sort)
+    result = gb.value_counts(sort=sort, dropna=dropna)
+
+    kwargs = {"levels": [[1, 2], [np.nan]], "names": ["a", "b"]}
+    if dropna:
+        data = []
+        index = MultiIndex(codes=[[], []], **kwargs)
+    elif not groupby_sort and not sort:
+        data = [1, 2]
+        index = MultiIndex(codes=[[1, 0], [0, 0]], **kwargs)
+    else:
+        data = [2, 1]
+        index = MultiIndex(codes=[[0, 1], [0, 0]], **kwargs)
+    expected = Series(data, index=index, dtype="int64", name="count")
+
+    tm.assert_series_equal(result, expected)