Repository: pandas-dev/pandas (python)

Issue:
BUG: `DataFrameGroupBy.groups` fails when Categorical indexer contains NaNs and `dropna=False`
### Pandas version checks

- [x] I have checked that this issue has not already been reported.

- [x] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [x] I have confirmed this bug exists on the [main branch](https://pandas.pydata.org/docs/dev/getting_started/install.html#installing-the-development-version-of-pandas) of pandas.


### Reproducible Example

```python
>>> df = DataFrame(
...         {
...             "cat": Categorical(["a", np.nan, "a"], categories=["a", "b", "d"]),
...             "vals": [1, 2, 3],
...         }
...     )
>>> g = df.groupby("cat", observed=True, dropna=False)
>>> result = g.groups
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/workspaces/pandas/pandas/core/groupby/groupby.py", line 569, in groups
    return self._grouper.groups
  File "properties.pyx", line 36, in pandas._libs.properties.CachedProperty.__get__
  File "/workspaces/pandas/pandas/core/groupby/ops.py", line 710, in groups
    return self.groupings[0].groups
  File "properties.pyx", line 36, in pandas._libs.properties.CachedProperty.__get__
  File "/workspaces/pandas/pandas/core/groupby/grouper.py", line 711, in groups
    return codes, uniques
  File "/workspaces/pandas/pandas/core/arrays/categorical.py", line 745, in from_codes
    dtype = CategoricalDtype._from_values_or_dtype(
  File "/workspaces/pandas/pandas/core/dtypes/dtypes.py", line 347, in _from_values_or_dtype
    dtype = CategoricalDtype(categories, ordered)
  File "/workspaces/pandas/pandas/core/dtypes/dtypes.py", line 230, in __init__
    self._finalize(categories, ordered, fastpath=False)
  File "/workspaces/pandas/pandas/core/dtypes/dtypes.py", line 387, in _finalize
    categories = self.validate_categories(categories, fastpath=fastpath)
  File "/workspaces/pandas/pandas/core/dtypes/dtypes.py", line 585, in validate_categories
    raise ValueError("Categorical categories cannot be null")
ValueError: Categorical categories cannot be null
>>>
```

### Issue Description

When using `df.groupby(cat, dropna=False).groups`, we encounter a `ValueError`. This is counter-intuitive, as grouping operations work without an issue.

```python
>>> df = DataFrame(
...         {
...             "cat": Categorical(["a", np.nan, "a"], categories=["a", "b", "d"]),
...             "vals": [1, 2, 3],
...         }
...     )
>>> g = df.groupby("cat", observed=True, dropna=False)
>>> g.sum()
     vals
cat      
a       4
NaN     2
>>> g.sum().index
CategoricalIndex(['a', nan], categories=['a', 'b', 'd'], ordered=False, dtype='category', name='cat')
```


### Expected Behavior

`.groups` should return a dictionary which includes the NaN as the last entry.

### Installed Versions

<details>

INSTALLED VERSIONS
------------------
commit                : 41131a14324ababc5c81f194de3d9a239d120f27
python                : 3.10.8
python-bits           : 64
OS                    : Linux
OS-release            : 5.15.167.4-microsoft-standard-WSL2
Version               : #1 SMP Tue Nov 5 00:21:55 UTC 2024
machine               : x86_64
processor             : 
byteorder             : little
LC_ALL                : None
LANG                  : C.UTF-8
LOCALE                : en_US.UTF-8

pandas                : 3.0.0.dev0+2085.g41131a1432
numpy                 : 2.2.5
dateutil              : 2.9.0.post0
pip                   : 25.0.1
Cython                : 3.0.12
sphinx                : 8.1.3
IPython               : 8.35.0
adbc-driver-postgresql: None
adbc-driver-sqlite    : None
bs4                   : 4.13.4
blosc                 : None
bottleneck            : 1.4.2
fastparquet           : 2024.11.0
fsspec                : 2025.3.2
html5lib              : 1.1
hypothesis            : 6.131.8
gcsfs                 : 2025.3.2
jinja2                : 3.1.6
lxml.etree            : 5.4.0
matplotlib            : 3.10.1
numba                 : 0.61.2
numexpr               : 2.10.2
odfpy                 : None
openpyxl              : 3.1.5
psycopg2              : 2.9.10
pymysql               : 1.4.6
pyarrow               : 19.0.1
pyreadstat            : 1.2.8
pytest                : 8.3.5
python-calamine       : None
pytz                  : 2025.2
pyxlsb                : 1.0.10
s3fs                  : 2025.3.2
scipy                 : 1.15.2
sqlalchemy            : 2.0.40
tables                : 3.10.1
tabulate              : 0.9.0
xarray                : 2024.9.0
xlrd                  : 2.0.1
xlsxwriter            : 3.2.3
zstandard             : 0.23.0
tzdata                : 2025.2
qtpy                  : None
pyqt5                 : None

</details>


Pull request:
BUG: groupby.groups with NA categories fails
- [x] closes #61356 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

There is a slight code duplication here, but we don't need to rely on Cateorical's codes because we can just directly use groupby's. We also can't use `groupby` to implement `Index.groupby` because the former only works in the case where the `values` are exhaustive.

Patch:
diff --git a/doc/source/whatsnew/v3.0.0.rst b/doc/source/whatsnew/v3.0.0.rst
index fe2886a022..5223b36006 100644
--- a/doc/source/whatsnew/v3.0.0.rst
+++ b/doc/source/whatsnew/v3.0.0.rst
@@ -806,6 +806,7 @@ Groupby/resample/rolling
 ^^^^^^^^^^^^^^^^^^^^^^^^
 - Bug in :meth:`.DataFrameGroupBy.__len__` and :meth:`.SeriesGroupBy.__len__` would raise when the grouping contained NA values and ``dropna=False`` (:issue:`58644`)
 - Bug in :meth:`.DataFrameGroupBy.any` that returned True for groups where all Timedelta values are NaT. (:issue:`59712`)
+- Bug in :meth:`.DataFrameGroupBy.groups` and :meth:`.SeriesGroupBy.groups` would fail when the groups were :class:`Categorical` with an NA value (:issue:`61356`)
 - Bug in :meth:`.DataFrameGroupBy.groups` and :meth:`.SeriesGroupby.groups` that would not respect groupby argument ``dropna`` (:issue:`55919`)
 - Bug in :meth:`.DataFrameGroupBy.median` where nat values gave an incorrect result. (:issue:`57926`)
 - Bug in :meth:`.DataFrameGroupBy.quantile` when ``interpolation="nearest"`` is inconsistent with :meth:`DataFrame.quantile` (:issue:`47942`)
diff --git a/pandas/core/groupby/grouper.py b/pandas/core/groupby/grouper.py
index c9d874fc08..f8e92b7e26 100644
--- a/pandas/core/groupby/grouper.py
+++ b/pandas/core/groupby/grouper.py
@@ -12,11 +12,16 @@ from typing import (
 
 import numpy as np
 
+from pandas._libs import (
+    algos as libalgos,
+)
 from pandas._libs.tslibs import OutOfBoundsDatetime
 from pandas.errors import InvalidIndexError
 from pandas.util._decorators import cache_readonly
 
 from pandas.core.dtypes.common import (
+    ensure_int64,
+    ensure_platform_int,
     is_list_like,
     is_scalar,
 )
@@ -38,7 +43,10 @@ from pandas.core.indexes.api import (
 )
 from pandas.core.series import Series
 
-from pandas.io.formats.printing import pprint_thing
+from pandas.io.formats.printing import (
+    PrettyDict,
+    pprint_thing,
+)
 
 if TYPE_CHECKING:
     from collections.abc import (
@@ -668,8 +676,14 @@ class Grouping:
     def groups(self) -> dict[Hashable, Index]:
         codes, uniques = self._codes_and_uniques
         uniques = Index._with_infer(uniques, name=self.name)
-        cats = Categorical.from_codes(codes, uniques, validate=False)
-        return self._index.groupby(cats)
+
+        r, counts = libalgos.groupsort_indexer(ensure_platform_int(codes), len(uniques))
+        counts = ensure_int64(counts).cumsum()
+        _result = (r[start:end] for start, end in zip(counts, counts[1:]))
+        # map to the label
+        result = {k: self._index.take(v) for k, v in zip(uniques, _result)}
+
+        return PrettyDict(result)
 
     @property
     def observed_grouping(self) -> Grouping:
diff --git a/pandas/core/reshape/encoding.py b/pandas/core/reshape/encoding.py
index 54704b274b..ad4a5db441 100644
--- a/pandas/core/reshape/encoding.py
+++ b/pandas/core/reshape/encoding.py
@@ -60,13 +60,15 @@ def get_dummies(
     data : array-like, Series, or DataFrame
         Data of which to get dummy indicators.
     prefix : str, list of str, or dict of str, default None
-        String to append DataFrame column names.
+        A string to be prepended to DataFrame column names.
         Pass a list with length equal to the number of columns
         when calling get_dummies on a DataFrame. Alternatively, `prefix`
         can be a dictionary mapping column names to prefixes.
-    prefix_sep : str, default '_'
-        If appending prefix, separator/delimiter to use. Or pass a
-        list or dictionary as with `prefix`.
+    prefix_sep : str, list of str, or dict of str, default '_'
+        Should you choose to prepend DataFrame column names with a prefix, this
+        is the separator/delimiter to use between the two. Alternatively,
+        `prefix_sep` can be a list with length equal to the number of columns,
+        or a dictionary mapping column names to separators.
     dummy_na : bool, default False
         If True, a NaN indicator column will be added even if no NaN values are present.
         If False, NA values are encoded as all zero.
diff --git a/pandas/core/sample.py b/pandas/core/sample.py
index 5b1c4b6a33..4f12563e3c 100644
--- a/pandas/core/sample.py
+++ b/pandas/core/sample.py
@@ -123,7 +123,7 @@ def sample(
     random_state: np.random.RandomState | np.random.Generator,
 ) -> np.ndarray:
     """
-    Randomly sample `size` indices in `np.arange(obj_len)`
+    Randomly sample `size` indices in `np.arange(obj_len)`.
 
     Parameters
     ----------


Test patch:
diff --git a/pandas/tests/groupby/test_categorical.py b/pandas/tests/groupby/test_categorical.py
index e49be8c00b..cae3013642 100644
--- a/pandas/tests/groupby/test_categorical.py
+++ b/pandas/tests/groupby/test_categorical.py
@@ -506,6 +506,23 @@ def test_observed_groups(observed):
     tm.assert_dict_equal(result, expected)
 
 
+def test_groups_na_category(dropna, observed):
+    # https://github.com/pandas-dev/pandas/issues/61356
+    df = DataFrame(
+        {"cat": Categorical(["a", np.nan, "a"], categories=list("adb"))},
+        index=list("xyz"),
+    )
+    g = df.groupby("cat", observed=observed, dropna=dropna)
+
+    result = g.groups
+    expected = {"a": Index(["x", "z"])}
+    if not dropna:
+        expected |= {np.nan: Index(["y"])}
+    if not observed:
+        expected |= {"b": Index([]), "d": Index([])}
+    tm.assert_dict_equal(result, expected)
+
+
 @pytest.mark.parametrize(
     "keys, expected_values, expected_index_levels",
     [