Repository: spack/spack

Issue:
Testing issue: py-eccodes fails to find ecCodes library
### Steps to reproduce the failure(s) or link(s) to test output(s)

```console
$ spack test run py-eccodes
...
```


### Error message

...
RuntimeError: Cannot find the ecCodes library

1 error found in test log:
     248        from gribapi.errors import GribInternalError
     249      File "/usr/WS1/dahlgren/releases/spack/opt/spack/linux-rhel7-broa
            dwell/gcc-8.3.1/py-eccodes-1.5.0-bo5qyzancy64dexo72xtnzsmvmivquuf/l
            ib/python3.7/site-packages/gribapi/errors.py", line 16, in <module>
     250        from .bindings import ENC, ffi, lib
     251      File "/usr/WS1/dahlgren/releases/spack/opt/spack/linux-rhel7-broa
            dwell/gcc-8.3.1/py-eccodes-1.5.0-bo5qyzancy64dexo72xtnzsmvmivquuf/l
            ib/python3.7/site-packages/gribapi/bindings.py", line 35, in <modul
            e>
     252        raise RuntimeError("Cannot find the ecCodes library")
     253    RuntimeError: Cannot find the ecCodes library
  >> 254    FAILED: Command exited with status 1:
     255        '/usr/tce/bin/python3.7' '-m' 'eccodes' 'selfcheck'
     256    Traceback (most recent call last):
     257      File "/usr/tce/packages/python/python-3.7.2/lib/python3.7/runpy.p
            y", line 183, in _run_module_as_main
     258        mod_name, mod_spec, code = _get_module_details(mod_name, _Error
            )
     259      File "/usr/tce/packages/python/python-3.7.2/lib/python3.7/runpy.p
            y", line 142, in _get_module_details
     260        return _get_module_details(pkg_main_name, error)
...

### Information on your system or the test runner

* **Spack:** 0.20.0.dev0 (f5418ac34466c25583df26c3966488852cbac6e8)
* **Python:** 3.7.2
* **Platform:** linux-rhel7-broadwell
* **Concretizer:** clingo

### Additional information

Test log:

```
==> Testing package py-eccodes-1.5.0-bo5qyza
==> [2023-01-24-12:37:46.718801] Detected the following modules: ['eccodes', 'eccodes.highlevel', 'gribapi']
==> [2023-01-24-12:37:46.720874] checking import of eccodes
==> [2023-01-24-12:37:46.721235] '/usr/tce/bin/python3.7' '-c' 'import eccodes'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/WS1/dahlgren/releases/spack/opt/spack/linux-rhel7-broadwell/gcc-8.3.1/py-eccodes-1.5.0-bo5qyzancy64dexo72xtnzsmvmivquuf/lib/python3.7/site-packages/eccodes/__init__.py", line 13, in <module>
    from .eccodes import *  # noqa
...
  File "/usr/WS1/dahlgren/releases/spack/var/spack/repos/builtin/packages/py-eccodes/package.py", line 47, in test
    work_dir="spack-test",
==> [2023-01-24-12:37:51.570153] Completed testing
```

### General information

- [X] I have reported the version of Spack/Python/Platform/Runner
- [X] I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
- [X] I have uploaded any available logs
- [X] I have searched the issues of this repo and believe this is not a duplicate


Discussion:
Which of `setup(_dependent)?_(build|run)_environment` does `spack test` use? We set all of them to locate the libraries (Python doesn't support RPATH). Can you share the environment variables (specifically `ECCODES_HOME` and `ECMWFLIBS_ECCODES`) during testing?
> Which of `setup(_dependent)?_(build|run)_environment` does `spack test` use? We set all of them to locate the libraries (Python doesn't support RPATH). 

`setup_run_environment`.  Wondering if we should be using `setup_build_environment` when `test_requires_compiler` but IIRC it wouldn't matter for this package.

> Can you share the environment variables (specifically `ECCODES_HOME` and `ECMWFLIBS_ECCODES`) during testing?

Sure.

```
ECCODES_HOME = $SPACK_ROOT/opt/spack/linux-rhel7-broadwell/gcc-8.3.1/eccodes-2.25.0-lausxfryeujmknncpejpshymdzks3ew6
ECMWFLIBS_ECCODES = $SPACK_ROOT/opt/spack/linux-rhel7-broadwell/gcc-8.3.1/eccodes-2.25.0-lausxfryeujmknncpejpshymdzks3ew6/lib64/libeccodes.so
```
Are those paths correct? Does the library exist at that location?
> Are those paths correct? Does the library exist at that location?

The HOME is correct.  But it only contains a `lib/python3.7/site-packages/` subdirectory with the contents below.  (The hashes are different because I had uninstalled so re-installed the package.)

```
$ ls opt/spack/linux-rhel7-broadwell/gcc-8.3.1/py-eccodes-1.5.0-qln4twxzwo4q7fjdt5uxzfctha4cg7yq/lib/python3.7/site-packages/*

opt/spack/linux-rhel7-broadwell/gcc-8.3.1/py-eccodes-1.5.0-iuhkxyn4xycun4mrak37njzckctmr5oz/lib/python3.7/site-packages/eccodes:
__init__.py  __main__.py  __pycache__  eccodes.py  highlevel

opt/spack/linux-rhel7-broadwell/gcc-8.3.1/py-eccodes-1.5.0-iuhkxyn4xycun4mrak37njzckctmr5oz/lib/python3.7/site-packages/eccodes-1.5.0.dist-info:
INSTALLER  METADATA  REQUESTED	direct_url.json  zip-safe
LICENSE    RECORD    WHEEL	top_level.txt

opt/spack/linux-rhel7-broadwell/gcc-8.3.1/py-eccodes-1.5.0-iuhkxyn4xycun4mrak37njzckctmr5oz/lib/python3.7/site-packages/gribapi:
__init__.py  bindings.py  errors.py   gribapi.py
__pycache__  eccodes.h	  grib_api.hi
```

Does `lib/libeccodes.so` exist? I'm not sure where it would be getting `lib64` from. Also pinging @skosukhin who maintains the eccodes package.
> Does `lib/libeccodes.so` exist? I'm not sure where it would be getting `lib64` from.

Yes.  It's from the `eccodes` installation, whose contents I forgot to include above: 

```
ls opt/spack/linux-rhel7-broadwell/gcc-8.3.1/eccodes-2.25.0-lausxfryeujmknncpejpshymdzks3ew6/*
opt/spack/linux-rhel7-broadwell/gcc-8.3.1/eccodes-2.25.0-lausxfryeujmknncpejpshymdzks3ew6/include:
eccodes.h	  eccodes_ecbuild_config.h  eccodes_windef.h
eccodes_config.h  eccodes_version.h	    grib_api.h

opt/spack/linux-rhel7-broadwell/gcc-8.3.1/eccodes-2.25.0-lausxfryeujmknncpejpshymdzks3ew6/lib64:
cmake  libeccodes.a  libeccodes.so  pkgconfig

opt/spack/linux-rhel7-broadwell/gcc-8.3.1/eccodes-2.25.0-lausxfryeujmknncpejpshymdzks3ew6/share:
eccodes
```


Oh okay, so both paths are correct then. Not sure why it can't find the library since we're telling it exactly where it lives...
The problem is that `libeccodes.so` is installed to `<prefix>/lib` on some platforms and to `<prefix>/lib64` on some others (due to [`GNUInstallDirs`](https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html), if I am not mistaken) but `py-findlibs` considers only the `libs` subdirectory of `ECCODES_HOME` (see [here](https://github.com/ecmwf/findlibs/blob/c21f47400d3cb513b92d6b1c684582643282ee92/findlibs/__init__.py#L34)). A solution/workaround here could be to partially revert #32792, which I would like to avoid because `LD_LIBRARY_PATH/DYLD_LIBRARY_PATH` pollute the environment (and, therefore, are often filtered out from the environment modules) and are used by `py-findlibs` not the way they are supposed to be. A more preferable approach is to patch `py-findlibs` to make it consider `lib64` as well, which, actually, already happens in [some cases](https://github.com/ecmwf/findlibs/blob/c21f47400d3cb513b92d6b1c684582643282ee92/findlibs/__init__.py#L48).

**UPD.:** see also #32728.
Hi, I have an issue using eccodes, which may be related with this bug.
I am using Windows and installed eccodes/grib with Mamba (conda-forge) in a dedicated environment named airpython. I code in python using Spyder.

When importing eccodes (or cfgrib) I receive the following error:
  File ~\mambaforge\envs\airpython\lib\site-packages\gribapi\bindings.py:35
    raise RuntimeError("Cannot find the ecCodes library")

When running "python -m eccodes selfcheck" everything looks good:
Found: ecCodes v2.28.0.
Library: ~\mambaforge\envs\airpython\Library\bin\eccodes.dll
Definitions: /MEMFS/definitions
Samples: /MEMFS/samples
Your system is ready.
(I checked manually and could find eccodes.dll in place)

bindings.py:33-35 
library_path = findlibs.find("eccodes")
if library_path is None:
    raise RuntimeError("Cannot find the ecCodes library")

"import findlibs" works fine in my environment, but findlibs.find("eccodes") returns nothing.

This leads to findlibs:
https://github.com/ecmwf/findlibs/blob/c21f47400d3cb513b92d6b1c684582643282ee92/findlibs/__init__.py

Searching os.environ, I have no ECCODES_DIR, ECCODES_HOME, LD_LIBRARY_PATH or DYLD_LIBRARY_PATH variables.
However, I have 'CONDA_ENV_PATH': '~/mambaforge/envs/airpython' and 'CONDA_ENV_PYTHON': '~\\mambaforge\\envs\\airpython\\python.exe'
If these had been used, we would have been close to giving an answer to findlibs. I do not know if this is the right way to go.

I will just solve my problem by adding a ECCODES_HOME variable to my environment, but maybe you can implement a more permanent change.

I am new here, feel free to tell me that I should have started a new topic.
GD
@sydomine this repo is for the Spack package manager, not for Mamba. Even if we fixed this issue in Spack, it wouldn't help you on Mamba.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/py-findlibs/package.py b/var/spack/repos/builtin/packages/py-findlibs/package.py
--- a/var/spack/repos/builtin/packages/py-findlibs/package.py
+++ b/var/spack/repos/builtin/packages/py-findlibs/package.py
@@ -15,3 +15,10 @@ class PyFindlibs(PythonPackage):
     version("0.0.2", sha256="6c7e038496f9a97783ab2cd5736bb68522d5bebd8b0eb17c976b6a4ae4032c8d")
 
     depends_on("py-setuptools", type="build")
+
+    # See https://github.com/ecmwf/findlibs/pull/5
+    patch(
+        "https://github.com/ecmwf/findlibs/commit/66a99a99d5d0a3215ed44aeb0af57991f3f7b8af.patch?full_index=1",
+        sha256="6701e0e38d377b6459c0eedca76030f7400ac0bcc9c00aaba7ddf4fee6143159",
+        when="@0.0.2",
+    )


Test patch: