Repository: ecmwf/earthkit-data (python)

Issue:
RecursionError: maximum recursion depth exceeded
### What happened?

I have some grib2 files for 3 different variables and I am trying to group them into a single file. However when trying to save the results to this new file I am hitting a  `RecursionError`

### What are the steps to reproduce the bug?

See code snippet below:
```
from earthkit.data import from_source
import sys
import datetime
ds = from_source('empty')
for path in sys.argv[1:]:
    print(path)
    new = from_source('file', path)

    # Define start and end dates for the loop
    start_date = datetime.datetime(2021, 1, 1, 0, 5)
    end_date = datetime.datetime(2021, 1, 10, 10, 50)
    delta = datetime.timedelta(minutes=15)

    # Loop from start_date to end_date
    current_date = start_date
    while current_date <= end_date:
        ds += new.sel(valid_datetime=current_date.isoformat())
        current_date += delta

# Save the final dataset
ds.save('combined.grib')
```

Where sys.argv[1:] can be a path pointing to the grib file you want to modify. 

error logs below:
![Image](https://github.com/user-attachments/assets/c1dc12e5-2495-40ef-aef3-5e4df49b436f)

### Version

0.12.0

### Platform (OS and architecture)

Linux

### Relevant log output

```shell

```

### Accompanying data

_No response_

### Organisation

ECMWF

Pull request:
Fix max recursion depth exceeded when concatenate
Fixes #588

Patch:
diff --git a/docs/release_notes/version_0.12_updates.rst b/docs/release_notes/version_0.12_updates.rst
index 0b5bdf4..b7227ab 100644
--- a/docs/release_notes/version_0.12_updates.rst
+++ b/docs/release_notes/version_0.12_updates.rst
@@ -1,6 +1,16 @@
 Version 0.12 Updates
 /////////////////////////
 
+
+Version 0.12.2
+===============
+
+Fixes
+++++++++
+
+- Fixed issue when failed to build a fieldlist with concatenation because maximum recursion depth exceeded (:pr:`599`)
+
+
 Version 0.12.1
 ===============
 
diff --git a/src/earthkit/data/core/index.py b/src/earthkit/data/core/index.py
index 1d06524..c13cf50 100644
--- a/src/earthkit/data/core/index.py
+++ b/src/earthkit/data/core/index.py
@@ -633,11 +633,18 @@ class MaskIndex(Index):
 
 class MultiIndex(Index):
     def __init__(self, indexes, *args, **kwargs):
-        self._indexes = list(indexes)
+        self._indexes = list(self._flatten(indexes))
         super().__init__(*args, **kwargs)
         # self.indexes = list(i for i in indexes if len(i))
         # TODO: propagate  index._init_args, index._init_order_by, index._init_kwargs, for each i in indexes?
 
+    def _flatten(self, indexes):
+        for i in indexes:
+            if isinstance(i, MultiIndex):
+                yield from self._flatten(i._indexes)
+            else:
+                yield i
+
     def sel(self, *args, **kwargs):
         if not args and not kwargs:
             return self


Test patch:
diff --git a/tests/grib/test_grib_concat.py b/tests/grib/test_grib_concat.py
index 5c92f47..1e94e1d 100644
--- a/tests/grib/test_grib_concat.py
+++ b/tests/grib/test_grib_concat.py
@@ -150,6 +150,17 @@ def test_grib_from_empty_3():
     _check_save_to_disk(ds3, 8, md)
 
 
+# See github issue #588
+def test_grib_concat_large():
+    ds_e = from_source("empty")
+    ds1 = from_source("file", earthkit_examples_file("test.grib"))
+
+    for _ in range(2000):
+        ds_e += ds1.sel(param="msl")
+
+    assert len(ds_e) == 2000
+
+
 if __name__ == "__main__":
     from earthkit.data.testing import main