Repository: DKISTDC/dkist

Issue:
`TiledDataset` needs more helpers for working over all tiles
Maybe we need some kind of slicing helper and also the `TiledDataset` also probably needs it's own `FileManager`

Discussion:
SolarDrew: Well I'm slightly confused and concerned by the 10% speed improvement to `test_generate_celestial()` because I didn't think I'd changed anything that would affect it...
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/DKISTDC/dkist/pull/437?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 96.71%. Comparing base [(`fb3ca0d`)](https://app.codecov.io/gh/DKISTDC/dkist/commit/fb3ca0d00dd64352ed5088bd48fe850fb0888073?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC) to head [(`795c4b6`)](https://app.codecov.io/gh/DKISTDC/dkist/commit/795c4b6f789934171e77c9ed1ba36c0cc715ccb0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #437      +/-   ##
==========================================
+ Coverage   96.69%   96.71%   +0.02%     
==========================================
  Files          68       68              
  Lines        4321     4352      +31     
==========================================
+ Hits         4178     4209      +31     
  Misses        143      143              
```

| [Flag](https://app.codecov.io/gh/DKISTDC/dkist/pull/437/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC) | Coverage Δ | |
|---|---|---|
| [](https://app.codecov.io/gh/DKISTDC/dkist/pull/437/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC) | `42.89% <21.87%> (-0.15%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/DKISTDC/dkist/pull/437?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DKISTDC).


Patch:
diff --git a/changelog/437.feature.rst b/changelog/437.feature.rst
new file mode 100644
index 0000000..550c332
--- /dev/null
+++ b/changelog/437.feature.rst
@@ -0,0 +1,1 @@
+Add a FileManager property to TiledDataset for tracking files more easily.
diff --git a/dkist/dataset/tiled_dataset.py b/dkist/dataset/tiled_dataset.py
index f63cfc4..cd9474e 100644
--- a/dkist/dataset/tiled_dataset.py
+++ b/dkist/dataset/tiled_dataset.py
@@ -13,6 +13,9 @@ import numpy as np
 
 from astropy.table import vstack
 
+from dkist.io.file_manager import FileManager, StripedExternalArray
+from dkist.io.loaders import AstropyFITSLoader
+
 from .dataset import Dataset
 from .utils import dataset_info_str
 
@@ -190,3 +193,39 @@ class TiledDataset(Collection):
 
     def __str__(self):
         return dataset_info_str(self)
+
+    @property
+    def files(self):
+        """
+        A `~.FileManager` helper for interacting with the files backing the data in this ``Dataset``.
+        """
+        return self._file_manager
+
+    @property
+    def _file_manager(self):
+        fileuris = [[tile.files.filenames for tile in row] for row in self]
+        dtype = self[0, 0].files.fileuri_array.dtype
+        shape = self[0, 0].files.shape
+        basepath = self[0, 0].files.basepath
+        chunksize = self[0, 0]._data.chunksize
+
+        for tile in self.flat:
+            try:
+                assert dtype == tile.files.fileuri_array.dtype
+                assert shape == tile.files.shape
+                assert basepath == tile.files.basepath
+                assert chunksize == tile._data.chunksize
+            except AssertionError as err:
+                raise AssertionError("Attributes of TiledDataset.FileManager must be the same across all tiles.") from err
+
+        return FileManager(
+            StripedExternalArray(
+                fileuris=fileuris,
+                target=1,
+                dtype=dtype,
+                shape=shape,
+                loader=AstropyFITSLoader,
+                basepath=basepath,
+                chunksize=chunksize
+            )
+        )


Test patch:
diff --git a/dkist/dataset/tests/test_dataset.py b/dkist/dataset/tests/test_dataset.py
index 966a62b..0dc0653 100644
--- a/dkist/dataset/tests/test_dataset.py
+++ b/dkist/dataset/tests/test_dataset.py
@@ -130,7 +130,6 @@ def test_file_manager():
         dataset.files = 10
 
     assert len(dataset.files.filenames) == 11
-    assert len(dataset.files.filenames) == 11
 
     assert isinstance(dataset[5]._file_manager, FileManager)
     assert len(dataset[..., 5].files.filenames) == 11
diff --git a/dkist/dataset/tests/test_tiled_dataset.py b/dkist/dataset/tests/test_tiled_dataset.py
index 0db139b..e72b811 100644
--- a/dkist/dataset/tests/test_tiled_dataset.py
+++ b/dkist/dataset/tests/test_tiled_dataset.py
@@ -96,3 +96,23 @@ def test_repr(simple_tiled_dataset):
 @pytest.mark.accept_cli_tiled_dataset
 def test_tiles_shape(simple_tiled_dataset):
     assert simple_tiled_dataset.tiles_shape == [[tile.data.shape for tile in row] for row in simple_tiled_dataset]
+
+
+def test_file_manager(large_tiled_dataset):
+    ds = large_tiled_dataset
+    with pytest.raises(AttributeError):
+        ds.files = 10
+
+    assert len(ds.files.filenames) == 27
+    assert ds.files.shape == (1, 4096, 4096)
+    assert ds.files.output_shape == (3, 3, 3, 4096, 4096)
+
+    # Have some slicing tests here
+    assert len(ds.slice_tiles[0].files.filenames) == 9
+    assert len(ds[:2, :2].files.filenames) == 12
+
+    # TODO Also test that the other checks raise errors
+    # This at least demonstrates that the structure works
+    ds[1, 1].files.fileuri_array.dtype = np.dtype("<i")
+    with pytest.raises(AssertionError, match="must be the same across all tiles"):
+        ds.files