Repository: scverse/scanpy

Issue:
sc.get.aggregate looses values
### Please make sure these conditions are met

- [X] I have checked that this issue has not already been reported.
- [X] I have confirmed this bug exists on the latest version of scanpy.
- [ ] (optional) I have confirmed this bug exists on the main branch of scanpy.

### What happened?

I'm working with the `sc.get.aggregate` function from the latest release candidate. In certain combinations of `groupby` variables, some valuese get lost. 

I wasn't able to make a minimal reproducible example, but I obfuscated the `obs` table of my real data and can share it here: 
https://www.dropbox.com/scl/fi/jsbrb2ulki7mmih2242kc/adata_aggregate_bug.h5ad?rlkey=qczuaf2v5vlwb00zyuxmzjkix&dl=1




### Minimal code sample

```python
>>> test_adata = sc.read_h5ad("adata_aggregate_bug.h5ad")

>>> test_adata.obs["patient_id"].nunique()
69

>>> test_adata.obs.isnull().sum()
patient_id           0
timepoint            0
external_batch_id    0
dtype: int64

>>> pb = sc.get.aggregate(
      test_adata,
      by=[
          "patient_id",
          "timepoint",
          "external_batch_id",
      ],
      func="mean",
  )
  pb.obs["patient_id"].nunique()

15

>>> pb = sc.get.aggregate(
      test_adata,
      by=[
          "patient_id",
          "external_batch_id",
      ],
      func="mean",
  )
  pb.obs["patient_id"].nunique()

69

>>> pb = sc.get.aggregate(
      test_adata,
      by=[
          "patient_id",
          "timepoint",
      ],
      func="mean",
  )
  pb.obs["patient_id"].nunique()

69
```


### Error output

```pytb
So only if using all three variables, some patient IDs are lost. I don't see why this would be happening.
```


### Versions

<details>

```
Package                   Version         Editable project location
------------------------- --------------- -------------------------------------------------------------------------------------------------------------------------
aiohttp                   3.9.3
aiosignal                 1.3.1
anndata                   0.10.5.post1
anyio                     4.3.0
appdirs                   1.4.4
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
array_api_compat          1.5
arrow                     1.3.0
asciitree                 0.3.3
asttokens                 2.4.1
async-lru                 2.0.4
async-timeout             4.0.3
attrs                     23.2.0
Babel                     2.14.0
beautifulsoup4            4.12.3
bleach                    6.1.0
bokeh                     3.3.4
branca                    0.7.1
Brotli                    1.1.0
cached-property           1.5.2
cachetools                5.3.3
certifi                   2024.2.2
cffi                      1.16.0
charset-normalizer        3.3.2
click                     8.1.7
click-plugins             1.1.1
cligj                     0.7.2
cloudpickle               3.0.0
colorama                  0.4.6
colorcet                  3.1.0
comm                      0.2.1
confluent-kafka           1.9.2
contourpy                 1.2.0
cubinlinker               0.3.0
cucim                     24.2.0
cuda-python               11.8.3
cudf                      24.2.2
cudf_kafka                24.2.2
cugraph                   24.2.0
cuml                      24.2.0
cuproj                    24.2.0
cupy                      12.2.0
cuspatial                 24.2.0
custreamz                 24.2.2
cuxfilter                 24.2.0
cycler                    0.12.1
cytoolz                   0.12.3
dask                      2024.1.1
dask-cuda                 24.2.0
dask-cudf                 24.2.2
datashader                0.16.0
debugpy                   1.8.1
decorator                 5.1.1
decoupler                 1.6.0
defusedxml                0.7.1
distributed               2024.1.1
docrep                    0.3.2
entrypoints               0.4
et-xmlfile                1.1.0
exceptiongroup            1.2.0
executing                 2.0.1
fa2                       0.3.5
fasteners                 0.19
fastjsonschema            2.19.1
fastrlock                 0.8.2
fcsparser                 0.2.8
filelock                  3.13.1
fiona                     1.9.5
folium                    0.16.0
fonttools                 4.49.0
fqdn                      1.5.1
frozenlist                1.4.1
fsspec                    2024.2.0
GDAL                      3.8.1
gdown                     5.1.0
geopandas                 0.14.3
h11                       0.14.0
h2                        4.1.0
h5py                      3.10.0
harmonypy                 0.0.9
holoviews                 1.18.3
hpack                     4.0.0
httpcore                  1.0.4
httpx                     0.27.0
hyperframe                6.0.1
idna                      3.6
igraph                    0.11.4
imagecodecs               2024.1.1
imageio                   2.34.0
importlib_metadata        7.0.2
importlib_resources       6.1.3
inflect                   7.0.0
ipykernel                 6.29.3
ipylab                    1.0.0
ipython                   8.22.2
ipywidgets                8.1.2
isoduration               20.11.0
jedi                      0.19.1
Jinja2                    3.1.3
joblib                    1.3.2
json5                     0.9.22
jsonpointer               2.4
jsonschema                4.21.1
jsonschema-specifications 2023.12.1
jupyter_client            8.6.0
jupyter_core              5.7.1
jupyter-events            0.9.0
jupyter-lsp               2.2.4
jupyter_server            2.13.0
jupyter_server_proxy      4.1.0
jupyter_server_terminals  0.5.2
jupyterlab                4.1.4
jupyterlab_pygments       0.3.0
jupyterlab_server         2.25.3
jupyterlab_widgets        3.0.10
kiwisolver                1.4.5
lamin_utils               0.13.0
lazy_loader               0.3
legacy-api-wrap           1.4
leidenalg                 0.10.2
linkify-it-py             2.0.3
llvmlite                  0.42.0
locket                    1.0.0
louvain                   0.8.1
lz4                       4.3.3
mapclassify               2.6.1
Markdown                  3.5.2
markdown-it-py            3.0.0
MarkupSafe                2.1.5
matplotlib                3.8.3
matplotlib-inline         0.1.6
mdit-py-plugins           0.4.0
mdurl                     0.1.2
mistune                   3.0.2
msgpack                   1.0.7
mudata                    0.2.3
multidict                 6.0.5
multipledispatch          0.6.0
munkres                   1.1.4
muon                      0.1.5
natsort                   8.4.0
nbclient                  0.8.0
nbconvert                 7.16.2
nbformat                  5.9.2
nbproject                 0.10.1
nest_asyncio              1.6.0
networkx                  3.2.1
notebook                  7.1.1
notebook_shim             0.2.4
numba                     0.59.0
numcodecs                 0.12.1
numpy                     1.24.4
nvtx                      0.2.10
omnipath                  1.0.8
openpyxl                  3.1.2
orjson                    3.9.15
overrides                 7.7.0
packaging                 24.0
pandas                    1.5.3
pandocfilters             1.5.0
panel                     1.3.8
param                     2.0.2
parso                     0.8.3
partd                     1.4.1
patsy                     0.5.6
pexpect                   4.9.0
pickleshare               0.7.5
pillow                    10.2.0
pip                       24.0
pkgutil_resolve_name      1.3.10
platformdirs              4.2.0
pooch                     1.8.1
prometheus_client         0.20.0
prompt-toolkit            3.0.42
protobuf                  4.25.3
psutil                    5.9.8
ptxcompiler               0.8.1
ptyprocess                0.7.0
pure-eval                 0.2.2
pyarrow                   14.0.2
pyarrow-hotfix            0.6
pycparser                 2.21
pyct                      0.5.0
pydantic                  1.10.14
pyee                      8.1.0
Pygments                  2.17.2
pylibcugraph              24.2.0
pylibraft                 24.2.0
pynndescent               0.5.11
pynvml                    11.4.1
pyparsing                 3.1.2
pyppeteer                 1.0.2
pyproj                    3.6.1
PySocks                   1.7.1
python-dateutil           2.9.0
python-json-logger        2.0.7
pytometry                 0.1.4
pytz                      2024.1
pyviz_comms               3.0.1
PyWavelets                1.4.1
PyYAML                    6.0.1
pyzmq                     25.1.2
raft-dask                 24.2.0
rapids_singlecell         0.9.6
readfcs                   1.1.7
referencing               0.33.0
requests                  2.31.0
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rich                      13.7.1
rmm                       24.2.0
rpds-py                   0.18.0
Rtree                     1.2.0
scanpy                    1.10.0rc2
scikit-image              0.22.0
scikit-learn              1.4.1.post1
scikit-misc               0.3.1
scipy                     1.12.0
seaborn                   0.13.2
Send2Trash                1.8.2
session-info              1.0.0
setuptools                69.1.1
shapely                   2.0.3
simpervisor               1.0.0
single_cell_helper        0.0.1          
six                       1.16.0
sniffio                   1.3.1
sortedcontainers          2.4.0
soupsieve                 2.5
stack-data                0.6.2
statsmodels               0.14.1
stdlib-list               0.10.0
streamz                   0.6.4
tblib                     3.0.0
terminado                 0.18.0
texttable                 1.7.0
threadpoolctl             3.3.0
tifffile                  2024.2.12
tinycss2                  1.2.1
tomli                     2.0.1
toolz                     0.12.1
tornado                   6.4
tqdm                      4.66.2
traitlets                 5.14.1
treelite                  4.0.0
types-python-dateutil     2.8.19.20240311
typing_extensions         4.10.0
typing-utils              0.1.0
uc-micro-py               1.0.3
ucx-py                    0.36.0
umap-learn                0.5.5
unicodedata2              15.1.0
uri-template              1.3.0
urllib3                   1.26.18
wcwidth                   0.2.13
webcolors                 1.13
webencodings              0.5.1
websocket-client          1.7.0
websockets                10.4
wget                      3.2
wheel                     0.42.0
widgetsnbextension        4.0.10
wrapt                     1.16.0
xarray                    2024.2.0
xgboost                   2.0.3
xyzservices               2023.10.1
yarl                      1.9.4
zarr                      2.17.1
zict                      3.0.0
zipp                      3.17.0
```

</details>


Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/scverse/scanpy/pull/2965?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scverse) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 73.14%. Comparing base [(`65f567e`)](https://app.codecov.io/gh/scverse/scanpy/commit/65f567e6905afbbf673ecf715ad0f796794ce9cb?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scverse) to head [(`f00e045`)](https://app.codecov.io/gh/scverse/scanpy/pull/2965?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scverse).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #2965      +/-   ##
==========================================
- Coverage   75.49%   73.14%   -2.35%     
==========================================
  Files         116      116              
  Lines       12911    12875      -36     
==========================================
- Hits         9747     9418     -329     
- Misses       3164     3457     +293     
```


| [Files](https://app.codecov.io/gh/scverse/scanpy/pull/2965?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scverse) | Coverage Δ | |
|---|---|---|
| [scanpy/get/\_aggregated.py](https://app.codecov.io/gh/scverse/scanpy/pull/2965?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scverse#diff-c2NhbnB5L2dldC9fYWdncmVnYXRlZC5weQ==) | `94.73% <100.00%> (ø)` | |

... and [27 files with indirect coverage changes](https://app.codecov.io/gh/scverse/scanpy/pull/2965/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scverse)

</details>

Patch:
diff --git a/docs/release-notes/1.10.1.md b/docs/release-notes/1.10.1.md
new file mode 100644
index 00000000..282d2297
--- /dev/null
+++ b/docs/release-notes/1.10.1.md
@@ -0,0 +1,13 @@
+### 1.10.1 {small}`the future`
+
+
+```{rubric} Docs
+```
+
+```{rubric} Bug fixes
+```
+
+* Fix `aggregate` when aggregating by more than two groups {pr}`2965` {smaller}`I Virshup`
+
+```{rubric} Performance
+```
diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md
index 0dd4e2a0..ce582c46 100644
--- a/docs/release-notes/index.md
+++ b/docs/release-notes/index.md
@@ -9,6 +9,9 @@
 
 ## Version 1.10
 
+```{include} /release-notes/1.10.1.md
+```
+
 ```{include} /release-notes/1.10.0.md
 ```
 
diff --git a/scanpy/get/_aggregated.py b/scanpy/get/_aggregated.py
index 5530059c..e68053bb 100644
--- a/scanpy/get/_aggregated.py
+++ b/scanpy/get/_aggregated.py
@@ -377,7 +377,7 @@ def _combine_categories(
 
     # Calculating result codes
     factors = np.ones(len(cols) + 1, dtype=np.int32)  # First factor needs to be 1
-    np.cumsum(n_categories[::-1], out=factors[1:])
+    np.cumprod(n_categories[::-1], out=factors[1:])
     factors = factors[:-1][::-1]
 
     code_array = np.zeros((len(cols), df.shape[0]), dtype=np.int32)


Test patch:
diff --git a/scanpy/tests/test_aggregated.py b/scanpy/tests/test_aggregated.py
index 99d36947..312f4ba5 100644
--- a/scanpy/tests/test_aggregated.py
+++ b/scanpy/tests/test_aggregated.py
@@ -460,3 +460,19 @@ def test_dispatch_not_implemented():
     adata = sc.datasets.blobs()
     with pytest.raises(NotImplementedError):
         sc.get.aggregate(adata.X, adata.obs["blobs"], "sum")
+
+
+def test_factors():
+    from itertools import product
+
+    obs = pd.DataFrame(
+        product(range(5), range(5), range(5), range(5)), columns=list("abcd")
+    )
+    obs.index = [f"cell_{i:04d}" for i in range(obs.shape[0])]
+    adata = ad.AnnData(
+        X=np.arange(obs.shape[0]).reshape(-1, 1),
+        obs=obs,
+    )
+
+    res = sc.get.aggregate(adata, by=["a", "b", "c", "d"], func="sum")
+    np.testing.assert_equal(res.layers["sum"], adata.X)