Repository: easybuilders/easybuild-easyblocks

Issue:
meson/ninja scipy install not producing egg / dist info 
And that means it does not appear in the result of `pip list` and causes anything requiring `scipy` to fail the sanity check.


Discussion:
We should have noticed this, so perhaps we should implement support for running `pip list | grep ext_name` as a part of the sanity check while we're at it?
```python
check_pip_list = True
```

@jfgrimm Any idea if we overlooked something, or whether `scipy` does something extra (that we don't) so `pip` is aware of it being installed?
furthermore:
> It looks like there is a bug in SciPy-bundle/2023.02-gfbf-2022b: It does not install the include files for scipy and numpy, so other packages that compiles code building on top cannot install.  For example, I tried to make an easyconfig for scikit-learn, but that cannot build due to missing scipy header files.
I was wrong when I reported that, the include files are where they should be (not where I expected them), and my trouble are solely due to pip claiming that scipy is not installed

It looks like we just need to copy the `PKG-INFO` that's included in the `scipy` source tarball (cfr. changes made in https://github.com/msys2/MINGW-packages/pull/12405):

```console
$ module load SciPy-bundle/2023.02-gfbf-2022b
$ pip list | grep scipy

[notice] A new release of pip available: 22.3.1 -> 23.0.1
[notice] To update, run: pip install --upgrade pip
```

```console
$ cp /tmp/$USER/scipy-1.10.1/PKG-INFO $EBROOTSCIPYMINBUNDLE/lib/python3.10/site-packages/scipy-1.9.3.egg-info
$ pip list | grep scipy
scipy                             1.10.1

[notice] A new release of pip available: 22.3.1 -> 23.0.1
[notice] To update, run: pip install --upgrade pip
```
> We should have noticed this, so perhaps we should implement support for running `pip list | grep ext_name` as a part of the sanity check while we're at it?
> 
> ```python
> check_pip_list = True
> ```

Wouldn't it be wise to make it part of the standard test applied to any pip-installed package, instead of a specific flag that one needs to remember to put in the config file?  Then a variable could be used to disable it if necessary.
@schiotz For now, I'll just add a `pip list | grep scipy` check for `scipy` specifically (that's enabled by default).

We can consider extending this to all Python packages, but we need to check more carefully first that this won't cause trouble (failing sanity checks).

Resolving patch:
diff --git a/easybuild/easyblocks/s/scipy.py b/easybuild/easyblocks/s/scipy.py
--- a/easybuild/easyblocks/s/scipy.py
+++ b/easybuild/easyblocks/s/scipy.py
@@ -42,8 +42,8 @@
 from easybuild.easyblocks.generic.mesonninja import MesonNinja
 from easybuild.easyblocks.generic.pythonpackage import PythonPackage, det_pylibdir
 from easybuild.framework.easyconfig import CUSTOM
-from easybuild.tools.build_log import EasyBuildError
-from easybuild.tools.filetools import change_dir, copy_dir
+from easybuild.tools.build_log import EasyBuildError, print_warning
+from easybuild.tools.filetools import change_dir, copy_dir, copy_file
 
 
 class EB_scipy(FortranPythonPackage, PythonPackage, MesonNinja):
@@ -198,6 +198,18 @@ def install_step(self):
         """Custom install step for scipy: use ninja for scipy >= 1.9.0"""
         if self.use_meson:
             MesonNinja.install_step(self)
+
+            # copy PKG-INFO file included in scipy source tarball to scipy-<version>.egg-info in installation,
+            # so pip is aware of the scipy installation (required for 'pip list', 'pip check', etc.);
+            # see also https://github.com/easybuilders/easybuild-easyblocks/issues/2901
+            pkg_info = os.path.join(self.cfg['start_dir'], 'PKG-INFO')
+            target_egg_info = os.path.join(self.installdir, self.pylibdir, 'scipy-%s.egg-info' % self.version)
+            if os.path.isfile(pkg_info):
+                copy_file(pkg_info, target_egg_info)
+            else:
+                cwd = os.getcwd()
+                print_warning("%s not found in %s, so can't use it to create %s!", pkg_info, cwd, target_egg_info,
+                              log=self.log)
         else:
             FortranPythonPackage.install_step(self)
 
@@ -211,4 +223,9 @@ def sanity_check_step(self, *args, **kwargs):
             'dirs': [det_pylibdir()],
         }
 
-        return PythonPackage.sanity_check_step(self, custom_paths=custom_paths)
+        # make sure that scipy is included in output of 'pip list',
+        # so that 'pip check' passes if scipy is a required dependency for another Python package;
+        # use case-insensitive match, since name is sometimes reported as 'SciPy'
+        custom_commands = [r"pip list | grep -iE '^scipy\s+%s\s*$'" % self.version.replace('.', r'\.')]
+
+        return PythonPackage.sanity_check_step(self, custom_paths=custom_paths, custom_commands=custom_commands)


Test patch: