Repository: napari/napari

Issue:
new zarr release / numcodecs
## 🐛 Bug

Looks like zarr's new release requires numcodecs > 0.6.4, but we pinned to exclude it see discussion #666. I think we need to resolve this ASAP and then make the 0.2.10 release (which also includes the #866 bug fix). Thoughts @tlambert03 @jni? Has the 0.6.4 numcodecs install problem been resolved? You can see our failing tests in #867. 


Discussion:
that's funny... I was literally about to hit "submit" on a similar issue.  
[here is the commit](https://github.com/zarr-developers/zarr-python/pull/497) where they bump the dependency, and they do it " to use `ensure_text` text handling functionality from it".  I don't know zarr deeply enough to know how frequently that is likely to happen...

numcodecs 0.6.4 is still not working, (this actually was just [checked in the dependabot fork today](https://github.com/tlambert-forks/napari-dependabot/pull/4) 😊)...  so not sure what to do about that
actually... looks like it was just the windows builds that [broke with numcodecs](https://github.com/tlambert-forks/napari-dependabot/pull/4/checks?check_run_id=384243596)... so maybe something has changed in the meantime?
ok - also if I remember https://github.com/zarr-developers/numcodecs/issues/210 correctly the issue is just with pip install on our CI, so people in the wild should still be installing ok. Maybe then this can wait till tomorrow
pip install works fine on my machine. imho the fix is to stop depending on zarr. We can use `dask.array.from_zarr` instead of `zarr.open`, and it should work whenever the file has standard compression, if I understand correctly.
Ah. I just saw why we started using zarr directly: we can tell whether there is a dataset instead of an array. Guh. =)
hmm - relevant part of the codebase is here https://github.com/napari/napari/blob/e2b90e96189cc18f4b5e5594b19f302ee305d380/napari/utils/io.py#L118

I agree that we should figure out a way to drop our dependency on both zarr and numcodecs, and just leverage dask, possibly with try catches to do everything, or say that additional functionality belongs in a plugin. We can still have zarr as one of our test dependencies (if that works with our CI) or just use dask functions for all our tests too. I'll think more about this tomorrow too
so i'm still really not sure how to deal with this one. On the one hand I like the idea of not explicitly depending on zarr and numcodecs and just leveraging fileIO options here in dask, but then there is no good way to deal with the pyramid case where we want a list of dask arrays out. We could try raising an issue on dask to see if they'd ever be interested in adding the functionality we have here to their [from_zarr](https://github.com/dask/dask/blob/56943ed99618153b8a973a963b980054c5874a7d/dask/array/core.py#L2778) method, but it might be strange for them to return a list of dask arrays, or a dask array where each subarray along the first axis doesn't have the same shape.

I will also note that when I do pyramid loading from my laptop it worked fine, but when I did it from s3 it couldn't find the components properly. I need to look into that more, but it makes me think that we might not have a perfect solution even now.



Resolving patch:
diff --git a/napari/utils/io.py b/napari/utils/io.py
--- a/napari/utils/io.py
+++ b/napari/utils/io.py
@@ -9,7 +9,6 @@
 
 from dask import delayed
 from dask import array as da
-import zarr
 
 
 def magic_imread(filenames, *, use_dask=None, stack=True):
@@ -99,13 +98,13 @@ def magic_imread(filenames, *, use_dask=None, stack=True):
     return image
 
 
-def read_zarr_dataset(filename):
+def read_zarr_dataset(path):
     """Read a zarr dataset, including an array or a group of arrays.
 
     Parameters
     --------
-    filename : str
-        Path to file ending in '.zarr'. File can contain either an array
+    path : str
+        Path to directory ending in '.zarr'. Path can contain either an array
         or a group of arrays in the case of pyramid data.
     Returns
     -------
@@ -114,13 +113,17 @@ def read_zarr_dataset(filename):
     shape : tuple
         Shape of array or first array in list
     """
-    zr = zarr.open(filename, mode='r')
-    if isinstance(zr, zarr.core.Array):
+    if os.path.exists(os.path.join(path, '.zarray')):
         # load zarr array
-        image = da.from_zarr(filename)
+        image = da.from_zarr(path)
         shape = image.shape
-    else:
+    elif os.path.exists(os.path.join(path, '.zgroup')):
         # else load zarr all arrays inside file, useful for pyramid data
-        image = [da.from_zarr(filename, component=c) for c, a in zr.arrays()]
+        image = []
+        for subpath in sorted(os.listdir(path)):
+            if not subpath.startswith('.'):
+                image.append(read_zarr_dataset(os.path.join(path, subpath))[0])
         shape = image[0].shape
+    else:
+        raise ValueError(f"Not a zarr dataset or group: {path}")
     return image, shape


Test patch:
diff --git a/requirements/test.txt b/requirements/test.txt
--- a/requirements/test.txt
+++ b/requirements/test.txt
@@ -4,3 +4,4 @@ pytest-qt
 pytest-ordering
 pytest-timeout
 xarray
+zarr