Repository: h5netcdf/h5netcdf (language: python)

Issue:
complex type failing in 32-bit systems
#### What happened:

The complex tests in h5netcdf 1.4.0 are failing on 32-bit systems.
Test logs at https://ci.debian.net/data/autopkgtest/unstable/armel/p/python-h5netcdf/54210152/log.gz
(also, [armhf](https://ci.debian.net/packages/p/python-h5netcdf/unstable/armhf/54210153/), [i386](https://ci.debian.net/packages/p/python-h5netcdf/unstable/i386/54210154/))

```
 79s ______________________ test_complex_type_creation_errors _______________________
 79s 
 79s tmp_local_netcdf = '/tmp/pytest-of-debci/pytest-0/test_complex_type_creation_err0/testfile.nc'
 79s 
 79s     @pytest.mark.skipif(
 79s         version.parse(netCDF4.__version__) < version.parse("1.7.0"),
 79s         reason="does not work before netCDF4 v1.7.0",
 79s     )
 79s     def test_complex_type_creation_errors(tmp_local_netcdf):
 79s         complex_array = np.array([0 + 0j, 1 + 0j, 0 + 1j, 1 + 1j, 0.25 + 0.75j])
 79s     
 79s         with legacyapi.Dataset(tmp_local_netcdf, "w") as ds:
 79s             ds.createDimension("x", size=len(complex_array))
 79s             with pytest.raises(TypeError, match="data type 'c4' not understood"):
 79s                 ds.createVariable("data", "c4", ("x",))
 79s     
 79s         with legacyapi.Dataset(tmp_local_netcdf, "w") as ds:
 79s             ds.createDimension("x", size=len(complex_array))
 79s             with pytest.raises(
 79s                 TypeError,
 79s                 match="Currently only 'complex64' and 'complex128' dtypes are allowed.",
 79s             ):
 79s >               ds.createVariable("data", "c32", ("x",))
 79s 
 79s tests/test_h5netcdf.py:2713: 
 79s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 79s /usr/lib/python3/dist-packages/h5netcdf/legacyapi.py:245: in createVariable
 79s     return super().create_variable(
 79s /usr/lib/python3/dist-packages/h5netcdf/core.py:1234: in create_variable
 79s     return group._create_child_variable(
 79s /usr/lib/python3/dist-packages/h5netcdf/core.py:1040: in _create_child_variable
 79s     dtype = _check_dtype(self, dtype)
 79s /usr/lib/python3/dist-packages/h5netcdf/core.py:702: in _check_dtype
 79s     group._root._check_valid_netcdf_dtype(dtype)
 79s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 79s 
 79s self = <Closed h5netcdf.legacyapi.Dataset>, dtype = 'c32'
 79s 
 79s     def _check_valid_netcdf_dtype(self, dtype):
 79s >       dtype = np.dtype(dtype)
 79s E       TypeError: data type 'c32' not understood
 79s 
 79s /usr/lib/python3/dist-packages/h5netcdf/core.py:1604: TypeError
 79s 
 79s During handling of the above exception, another exception occurred:
 79s 
 79s tmp_local_netcdf = '/tmp/pytest-of-debci/pytest-0/test_complex_type_creation_err0/testfile.nc'
 79s 
 79s     @pytest.mark.skipif(
 79s         version.parse(netCDF4.__version__) < version.parse("1.7.0"),
 79s         reason="does not work before netCDF4 v1.7.0",
 79s     )
 79s     def test_complex_type_creation_errors(tmp_local_netcdf):
 79s         complex_array = np.array([0 + 0j, 1 + 0j, 0 + 1j, 1 + 1j, 0.25 + 0.75j])
 79s     
 79s         with legacyapi.Dataset(tmp_local_netcdf, "w") as ds:
 79s             ds.createDimension("x", size=len(complex_array))
 79s             with pytest.raises(TypeError, match="data type 'c4' not understood"):
 79s                 ds.createVariable("data", "c4", ("x",))
 79s     
 79s         with legacyapi.Dataset(tmp_local_netcdf, "w") as ds:
 79s             ds.createDimension("x", size=len(complex_array))
 79s >           with pytest.raises(
 79s                 TypeError,
 79s                 match="Currently only 'complex64' and 'complex128' dtypes are allowed.",
 79s             ):
 79s E           AssertionError: Regex pattern did not match.
 79s E            Regex: "Currently only 'complex64' and 'complex128' dtypes are allowed."
 79s E            Input: "data type 'c32' not understood"
 79s 
 79s tests/test_h5netcdf.py:2709: AssertionError
 79s =========================== short test summary info ============================
 79s FAILED tests/test_h5netcdf.py::test_complex_type_creation_errors - AssertionE...
 79s ============ 1 failed, 99 passed, 51 skipped, 1 deselected in 6.28s ============

```

I suspect it's probably easy to fix. Add "c32" alongside "c4" or something.

If needed, 32-bit can be tested in-python with `platform.architecture()[0]`


#### MCVE Code Sample

```
pytest-3 -v  -k test_complex_type_creation_errors h5netcdf/tests/test_h5netcdf.py
```

#### Anything else we need to know?:
Testing on linux (debian unstable)

#### Version

<details><summary>Output of <tt>print(h5py.version.info, f"\nh5netcdf {h5netcdf.__version__}")</tt></summary>
[equivalent versions on amd64 not 32-bit]

```
$ python3 -c 'import h5py; import h5netcdf; print(h5py.version.info, f"\nh5netcdf {h5netcdf.__version__}")'
Summary of the h5py configuration
---------------------------------

h5py    3.11.0
HDF5    1.10.10
Python  3.12.7 (main, Nov  8 2024, 17:55:36) [GCC 14.2.0]
sys.platform    linux
sys.maxsize     9223372036854775807
numpy   1.26.4
cython (built with) 3.0.11
numpy (built against) 1.26.4
HDF5 (built against) 1.10.10
 
h5netcdf 1.4.0
```
</details>



Discussion / hints:

Fix patch:
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 64ac364..eba4382 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -77,7 +77,7 @@ jobs:
       fail-fast: false
       matrix:
         python-version: ["3.11", "3.12"]
-        h5py-version: ["3.11"]
+        h5py-version: ["3.11", "3.12"]
         numpy-version: ["2"]
     steps:
     - uses: actions/checkout@v4
@@ -86,7 +86,7 @@ jobs:
       with:
         environment-name: h5netcdf-tests
         cache-environment: true
-        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-h5py${{matrix.h5py_version}}-numpy${{matrix.numpy_version}}-${{env.TODAY}}"
+        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-h5py${{matrix.h5py-version}}-numpy${{matrix.numpy-version}}-${{env.TODAY}}"
         create-args: >-
           python=${{ matrix.python-version }}
           pip
@@ -102,6 +102,45 @@ jobs:
       run: |
         pytest -v --durations=0 h5netcdf/tests/
 
+  build_0b:
+    name: h5py3 hdf5 1.10 tests
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        shell: bash -l {0}
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11"]
+        h5py-version: ["3.11"]
+        hdf5-version: ["1.10.*"]
+        numpy-version: ["1"]
+    steps:
+    - uses: actions/checkout@v4
+    - name: Install micromamba
+      uses: mamba-org/setup-micromamba@v2
+      with:
+        environment-name: h5netcdf-tests
+        cache-environment: true
+        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-h5py${{matrix.h5py-version}}-hdf5${{matrix.hdf5-version}}-numpy${{matrix.numpy-version}}-${{env.TODAY}}"
+        create-args: >-
+          python=${{ matrix.python-version }}
+          pip
+          pytest
+          wheel
+          hdf5=${{ matrix.hdf5-version }}
+          libnetcdf
+          numpy=${{ matrix.numpy-version }}
+    - name: Install h5py/netCDF4
+      run: |
+        python -m pip install h5py==${{ matrix.h5py-version }} netCDF4==1.5.*
+    - name: Install h5netcdf
+      run: |
+        python -m pip install . --no-deps --ignore-installed --no-cache-dir -vvv
+    - name: Test with pytest
+      run: |
+        pytest -v --durations=0 h5netcdf/tests/
+
   build_1:
     name: h5py3/h5pyd tests
     runs-on: ubuntu-latest
@@ -121,7 +160,7 @@ jobs:
       with:
         environment-name: h5netcdf-tests
         cache-environment: true
-        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-numpy${{matrix.numpy_version}}-h5pyd-${{env.TODAY}}"
+        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-numpy${{matrix.numpy-version}}-h5pyd-${{env.TODAY}}"
         create-args: >-
           python=${{ matrix.python-version }}
           pip
@@ -158,7 +197,7 @@ jobs:
       with:
         environment-name: h5netcdf-tests
         cache-environment: true
-        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-numpy${{matrix.numpy_version}}-netcdf4_nightly-${{env.TODAY}}"
+        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-numpy${{matrix.numpy-version}}-netcdf4_nightly-${{env.TODAY}}"
         create-args: >-
           python=${{ matrix.python-version }}
           pip
@@ -199,7 +238,7 @@ jobs:
       with:
         environment-name: h5netcdf-tests
         cache-environment: true
-        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-numpy${{matrix.numpy_version}}-xarray_nightly-${{env.TODAY}}"
+        cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-numpy${{matrix.numpy-version}}-xarray_nightly-${{env.TODAY}}"
         create-args: >-
           python=${{ matrix.python-version }}
           dask
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 623ed64..b19b736 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -1,6 +1,12 @@
 Change Log
 ----------
 
+Development Version:
+
+- Add CI run for hdf5 1.10.6, fix complex tests, fix enum/user type tests ({pull}`244`).
+  By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_
+
+
 Version 1.4.0 (October 7th, 2024):
 
 - Add UserType class, add EnumType ({pull}`229`).


Test patch:
diff --git a/h5netcdf/tests/test_h5netcdf.py b/h5netcdf/tests/test_h5netcdf.py
index 502f05b..6d2709d 100644
--- a/h5netcdf/tests/test_h5netcdf.py
+++ b/h5netcdf/tests/test_h5netcdf.py
@@ -2242,7 +2242,9 @@ def test_user_type_errors_new_api(tmp_local_or_remote_netcdf):
             if tmp_local_or_remote_netcdf.startswith(remote_h5):
                 testcontext = pytest.raises(RuntimeError, match="Conflict")
             else:
-                testcontext = pytest.raises(KeyError, match="name already exists")
+                testcontext = pytest.raises(
+                    (KeyError, TypeError), match="name already exists"
+                )
             with testcontext:
                 ds.create_enumtype(np.uint8, "enum_t", enum_dict2)
 
@@ -2290,7 +2292,9 @@ def test_user_type_errors_legacyapi(tmp_local_or_remote_netcdf):
             if tmp_local_or_remote_netcdf.startswith(remote_h5):
                 testcontext = pytest.raises(RuntimeError, match="Conflict")
             else:
-                testcontext = pytest.raises(KeyError, match="name already exists")
+                testcontext = pytest.raises(
+                    (KeyError, TypeError), match="name already exists"
+                )
             with testcontext:
                 ds.createEnumType(np.uint8, "enum_t", enum_dict1)
 
@@ -2704,6 +2708,8 @@ def test_complex_type_creation_errors(tmp_local_netcdf):
         with pytest.raises(TypeError, match="data type 'c4' not understood"):
             ds.createVariable("data", "c4", ("x",))
 
+    if "complex256" not in np.sctypeDict:
+        pytest.skip("numpy 'complex256' dtype not available")
     with legacyapi.Dataset(tmp_local_netcdf, "w") as ds:
         ds.createDimension("x", size=len(complex_array))
         with pytest.raises(