Repository: scverse/spatialdata

Issue:
Concatenating Multiple Spatialdata objects: Regions in the AnnData object and `cells_region` do not match.
Hi,

If you select multiple different regions on a Merscope you get multiple different folders as output.

I am trying to combine them into a single object like this:

```
import os
import spatialdata as sd
from spatialdata_io import merscope

def read_regions(top_level_path):
    regions = []
    for folder in os.listdir(top_level_path):
        if os.path.isdir(os.path.join(top_level_path, folder)):
            region = merscope(
                os.path.join(top_level_path, folder)
            )
            regions.append(region)

    combined = sd.concatenate(regions, join="outer")
    combined.table.uns = regions[0].uns.copy()

    return combined
```

This yields the error for the concatenate step: 
```

.../spatialdata/models/models.py:652) return data

ValueError: Regions in the AnnData object and `cells_region` do not match.
```

It is a little bit unclear to me what is going wrong and what role `region_key` and `instance_key` play here.

Discussion:
@AmosFong1: Did you find a solution? I am also having this issue. @pakiessling @LucaMarconato @LucaMarconato: @AmosFong1 @pakiessling is the problem still open after https://github.com/scverse/spatialdata/pull/720? (available with `spatialdata`>=0.2.4).

If yes, could you please make a reproducible example using the `blobs` dataset?
```python
from spatialdata import blobs
```
alternatively, could you point to some public datasets to reproduce this? Thank you.@pakiessling: ```
spatialdata-io-0.1.7.dev77+ga5dfb92
spatialdata-0.2.7.dev16+g7bce868
```
Everything working nicely, I also enjoy the automatic suffix addition 🌈 @AmosFong1: @LucaMarconato update to `spatial data-0.2.6` and `spatialdata-io-0.1.6` did not resolve my issue but from reviewing #349   and testing with the [concatenate.py](https://github.com/scverse/spatialdata/blob/main/src/spatialdata/_core/concatenate.py) script, I was able to figure out a few issues leading to this error message. 

The first issue is at lines 237-239:
```
region, region_key, _ = get_table_keys(table)
table.obs[region_key] = (table.obs[region_key].astype("str") + f"-{suffix}").astype("category")
table.uns[TableModel.ATTRS_KEY][TableModel.REGION_KEY] = f"{region}-{suffix}"
```
where renaming table.uns[TableModel.ATTRS_KEY][TableModel.REGION_KEY] mistakenly converts a list of labels to a string and appends the suffix, instead of renaming each label by appending suffix. My fix is shown here:
```
regions, region_key, _ = get_table_keys(table)  # changed region to regions
table.obs[region_key] = (table.obs[region_key].astype("str") + f"-{suffix}").astype("category")
table.uns[TableModel.ATTRS_KEY][TableModel.REGION_KEY] = [f"{region}-{suffix}" for region in regions]
```

The second issue is in line 68:
```
TableModel.REGION_KEY: merged_table.obs[TableModel.REGION_KEY].unique().tolist(),
```
where TableModel.Region_KEY does not account for unique region keys as described in #349, but can be resolved simply by providing region_key manually


The third issue is in line 196:
```
attrs = attrs_merge([sdata.attrs for sdata in sdatas])
```
where I couldn't find a definition for .attrs for spatialdata. I commented it out here.

Keeping in mind, my aim was to concatenate different CosMx datasets, which is different from the Merscope problem described here.@benemead: @LucaMarconato and @AmosFong1 I'm encountering a variant of this issue running on latest `spatialdata` (pip installing from main branch). 

I've got 2 spatialdatas from 2 xenium slides which I loaded using latest `spatialdata-io`. I also aligned an IF image and used `sopa` to aggregate into a second table `if_table`. The structure including keys is identical for the 2 spatialdatas. 

At this point it's unclear to me if there's a bug, or this kind of concatenation is not possible (multiple tables and images/shapes/labels each getting a unique coordinate system). 

Any insight would be greatly appreciated!!@LucaMarconato: Thanks @AmosFong1. The first problem you described is still open, I will have a look into it. The second (https://github.com/scverse/spatialdata/issues/349) is now addressed. Finally the `.attrs` was available in `main` but not released. Now it is released so you should be able to uncommend the line that you commented.@LucaMarconato: @benemead could you please check if the issue is still open after the fix to https://github.com/scverse/spatialdata/issues/349 that I introduced: https://github.com/scverse/spatialdata/pull/871?

If the problem persist, it would be great if you could reproduce it using the a variation of the test that I added here: https://github.com/scverse/spatialdata/blob/22d5b0ac9fd16471284bb9405a95d11675f847a9/tests/core/operations/test_spatialdata_operations.py#L333 or using the `blobs` dataset (`from spatialdata import blobs`). Thank you!@LucaMarconato: Thanks @AmosFong1, I added a fix and a test in the linked PR.
@benemead I will close this issue. If what you report is still open please open a new issue, so we keep the discussion focused. 

Solution patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
index 1d7a79b..788d020 100644
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -45,7 +45,7 @@ jobs:
               run: |
                   echo "::set-output name=dir::$(pip cache dir)"
             - name: Restore pip cache
-              uses: actions/cache@v2
+              uses: actions/cache@v4
               with:
                   path: ${{ steps.pip-cache-dir.outputs.dir }}
                   key: pip-${{ runner.os }}-${{ env.pythonLocation }}-${{ hashFiles('**/pyproject.toml') }}
diff --git a/src/spatialdata/_core/concatenate.py b/src/spatialdata/_core/concatenate.py
index 58e278f..da1865d 100644
--- a/src/spatialdata/_core/concatenate.py
+++ b/src/spatialdata/_core/concatenate.py
@@ -236,7 +236,13 @@ def _fix_ensure_unique_element_names(
             # fix the region_key column
             region, region_key, _ = get_table_keys(table)
             table.obs[region_key] = (table.obs[region_key].astype("str") + f"-{suffix}").astype("category")
-            table.uns[TableModel.ATTRS_KEY][TableModel.REGION_KEY] = f"{region}-{suffix}"
+            new_region: str | list[str]
+            if isinstance(region, str):
+                new_region = f"{region}-{suffix}"
+            else:
+                assert isinstance(region, list)
+                new_region = [f"{r}-{suffix}" for r in region]
+            table.uns[TableModel.ATTRS_KEY][TableModel.REGION_KEY] = new_region
 
             # fix the obs names
             if rename_obs_names:


Test patch:
diff --git a/tests/core/operations/test_spatialdata_operations.py b/tests/core/operations/test_spatialdata_operations.py
index 9b669fc..dd76502 100644
--- a/tests/core/operations/test_spatialdata_operations.py
+++ b/tests/core/operations/test_spatialdata_operations.py
@@ -5,6 +5,7 @@ import math
 import numpy as np
 import pytest
 from anndata import AnnData
+from geopandas import GeoDataFrame
 
 from spatialdata._core.concatenate import _concatenate_tables, concatenate
 from spatialdata._core.data_extent import are_extents_equal, get_extent
@@ -425,6 +426,37 @@ def test_concatenate_sdatas_from_iterable(concatenate_tables: bool, obs_names_ma
     assert sdata0["table"].obs_names[0] == "1"
 
 
+def test_concatenate_two_tables_each_annotating_two_elements() -> None:
+    # let's define 4 polygon elements. Two of them are annotated by the first table and the other two by the second
+    # table. The two tables have the same region key and instance key.
+    def _get_table_and_poly(i: int) -> tuple[AnnData, GeoDataFrame]:
+        poly = _get_shapes()["poly"]
+        n = len(poly)
+        region = f"poly{i}"
+        table = TableModel.parse(
+            AnnData(obs={"region": [region] * n, "instance_id": list(range(n))}),
+            region=region,
+            region_key="region",
+            instance_key="instance_id",
+        )
+        return table, poly
+
+    table0_a, poly0_a = _get_table_and_poly(0)
+    table1_a, poly1_a = _get_table_and_poly(1)
+    table0_b, poly0_b = _get_table_and_poly(0)
+    table1_b, poly1_b = _get_table_and_poly(1)
+
+    table01_a = _concatenate_tables([table0_a, table1_a])
+    table01_b = _concatenate_tables([table0_b, table1_b])
+
+    sdata01_a = SpatialData.init_from_elements({"poly0": poly0_a, "poly1": poly1_a, "table": table01_a})
+    sdata01_b = SpatialData.init_from_elements({"poly0": poly0_b, "poly1": poly1_b, "table": table01_b})
+    sdata = concatenate({"a": sdata01_a, "b": sdata01_b}, concatenate_tables=True)
+    region, _, _ = get_table_keys(sdata["table"])
+    assert region == ["poly0-a", "poly1-a", "poly0-b", "poly1-b"]
+    assert len(sdata["table"]) == 4 * len(poly0_a)
+
+
 def test_concatenate_sdatas_single_item() -> None:
     sdata = blobs()