Repository: dcs4cop/xcube

Issue:
Resolve/document dependency discrepancies
conda-forge releases (e.g. [here](https://github.com/conda-forge/xcube-feedstock/pull/48) ) trigger a dependency analysis which always finds a significant number of discrepancies between the dependencies listed in `meta.yaml`. Automated dependency anaylsis isn't 100% reliable, but we should resolve as many of these as possible and document the rest so we don't waste time looking into them again. for the 1.4.1 release, the list is as follows:

### Packages found by source code inspection but not in the meta.yaml:
- cftime
- numcodecs
- botocore

### Packages found in the meta.yaml but not found by source code inspection:
- werkzeug
- scipy
- adlfs
- python-blosc
- fontconfig
- lz4
- requests-oauthlib

| Package | Comments | Action |
| --- | --- | --- |
| `cftime` | imported in select.py etc. | add to environment.yml and meta.yaml |
| `numcodecs` | imported in zarrstore/generic.py | add to environment.yml and meta.yaml |
| `botocore` | imported in dsio.py | add to environment.yml and meta.yaml |
| `werkzeug` | only imported in tests | remove from meta.yaml |
| `scipy` | not imported | remove from environment.yml and meta.yaml |
| `adlfs` | not imported, but has comment ‘for azure blob filesystem’ | clarify |
| `python-blosc` | not imported, but has comment ‘required by Coiled’ | clarify -- see below |
| `fontconfig` | not imported | remove from environment.yml and meta.yaml |
| `lz4` | not imported, but has comment ‘required by Coiled’ | clarify -- see below |
| `requests-oauthlib` | not imported | remove from environment.yml and meta.yaml |

Notes:

- We'll also need to update `DEFAULT_DEPENDENCY_NAMES` in `versions.py`.
- From the above list, it appears that we specify explicit dependencies on two transitive (soft?) dependencies of the `coiled` package, but the `coiled` package is only a soft dependency (i.e. we try to import locally at runtime if/when it's actually needed and raise an error if it's not available). If these are hard dependencies of coiled, we can probably omit them entirely: if coiled is importable, they must be present. If they're soft dependencies of coiled which we require for our use case, we should handle them the same way we handle the `coiled` package itself -- i.e. check at runtime if/when required.


Discussion:

Patch:
diff --git a/CHANGES.md b/CHANGES.md
index 39684cd1..9225d17e 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,5 +1,8 @@
 ## Changes in 1.4.2 (in development)
 
+* Update dependencies to better match imports; remove the defaults channel;
+  turn adlfs into a soft dependency. (#945)
+
 ## Changes in 1.4.1
 
 ### Enhancements
diff --git a/docs/source/installation.md b/docs/source/installation.md
index 3ad8bf54..5ee8494d 100644
--- a/docs/source/installation.md
+++ b/docs/source/installation.md
@@ -144,3 +144,16 @@ docker ps
 This will start a service in the background which can be accessed 
 through port 8001, as the startup of a service is configured as default
 behaviour.
+
+## Installing soft dependencies
+
+In addition to xcube’s ‘hard’ dependencies, which the standard installation
+methods install automatically when required, there are ‘soft’ dependencies.
+These are packages which are not required to install or use xcube, but enable
+additional functionality if they are present. Users who need this functionality
+should install the requisite soft dependencies explicitly. xcube’s current soft
+dependencies are listed below.
+
+- `adlfs`: required by the abfs data store, which is used for access to
+  Azure Blob storage. Trying to create an abfs data store without `adlfs`
+  installed will raise an exception advising that you install it.
diff --git a/environment.yml b/environment.yml
index c40c46f6..6d4d0bbd 100644
--- a/environment.yml
+++ b/environment.yml
@@ -1,13 +1,13 @@
 name: xcube
 channels:
   - conda-forge
-  - defaults
 dependencies:
   # Python
   - python >=3.9
   # Required
-  - adlfs >=2023.1  # for azure blob filesystem
   - affine >=2.2
+  - botocore >=1.34.51
+  - cftime >=1.6.3
   - click >=8.0
   - cmocean >=2.0
   - dask >=2021.6
@@ -15,7 +15,6 @@ dependencies:
   - deprecated >=1.2
   - distributed >=2021.6
   - fiona >=1.8
-  - fontconfig
   - fsspec >=2021.6
   - gdal >=3.0
   - geopandas >=0.8
@@ -24,6 +23,7 @@ dependencies:
   - mashumaro
   - matplotlib-base >=3.0
   - netcdf4 >=1.5
+  - numcodecs >=0.12.1
   - numba >=0.52
   - numpy >=1.16
   - openssl
@@ -34,25 +34,19 @@ dependencies:
   - pyyaml >=5.4
   - rasterio >=1.2
   - requests >=2.25
-  - requests-oauthlib >=1.3
   - rfc3339-validator >=0.1  # for python-jsonschema date-time format validation
   - rioxarray >=0.11
   - s3fs >=2021.6
-  - scipy >=1.6.0
   - setuptools >=41.0
   - shapely >=1.6
   - tornado >=6.0
   - urllib3 >=1.26
-  - werkzeug
   - xarray >=2022.6
   - zarr >=2.11
-  # Required by Coiled
-  # These are very likely transitive deps anyway
-  - lz4
-  - python-blosc
   # Testing
   - flake8 >=3.7
   - moto >=4
   - pytest >=4.4
   - pytest-cov >=2.6
   - requests-mock >=1.8
+  - werkzeug
diff --git a/xcube/core/store/fs/accessor.py b/xcube/core/store/fs/accessor.py
index a5fdbad1..811a8934 100644
--- a/xcube/core/store/fs/accessor.py
+++ b/xcube/core/store/fs/accessor.py
@@ -132,9 +132,9 @@ class FsAccessor:
                 root,
                 params
             )
-        except (ValueError, ImportError):
-            raise DataStoreError(f"Cannot instantiate"
-                                 f" filesystem {protocol!r}")
+        except (ValueError, ImportError) as error:
+            raise DataStoreError(
+                f"Cannot instantiate filesystem {protocol!r}: {error}")
 
     @classmethod
     def add_storage_options_to_params_schema(


Test patch:
diff --git a/test/core/store/test_store.py b/test/core/store/test_store.py
new file mode 100644
index 00000000..44e03629
--- /dev/null
+++ b/test/core/store/test_store.py
@@ -0,0 +1,12 @@
+from fsspec.registry import register_implementation
+from xcube.core.store import new_data_store, DataStoreError
+import pytest
+
+
+def test_fsspec_instantiation_error():
+    error_string = "deliberate instantiation error for testing"
+    register_implementation(
+        "abfs", "nonexistentmodule.NonexistentClass", True, error_string
+    )
+    with pytest.raises(DataStoreError, match=error_string):
+        new_data_store("abfs").list_data_ids()