Repository: spack/spack

Issue:
Installation issue: py-scipy does not build correctly
### Steps to reproduce the issue

Building py-scipy as a dependent package in py-scikit-learn fails to properly build the FFT code internally included.  In this case py-scikit-learn and py-scipy were built as part of an environment.  Note that falling back to a py-scipy@1.3.3 avoids this issue (according to the release notes FFT was added in 1.4.0). @adamjstewart @rgommers

```console
==> Concretized py-scikit-learn%gcc
 -   bkt6lvj  py-scikit-learn@1.1.1%gcc@8.3.1+openmp arch=linux-rhel7-power9le
[+]  ynfcyea      ^py-cython@0.29.30%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  6ggss2n          ^py-pip@21.3.1%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  vj5oeps              ^python@3.9.10%gcc@8.3.1+bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,f2fd060 arch=linux-rhel7-power9le
[+]  4nmedyo                  ^bzip2@1.0.6%gcc@8.3.1~debug~pic+shared arch=linux-rhel7-power9le
[+]  k6rz4t6                  ^expat@2.4.8%gcc@8.3.1+libbsd arch=linux-rhel7-power9le
[+]  xtzjy2w                      ^gnuconfig@2021-08-14%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  x633ywv                      ^libbsd@0.11.5%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  fi63v6x                          ^libmd@1.0.4%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  xs4fkq4                  ^gdbm@1.23%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  32om6zc                      ^readline@8.1%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  p4uz2jr                          ^ncurses@5.9.20130511%gcc@8.3.1~symlinks+termlib abi=5 patches=f84b270 arch=linux-rhel7-power9le
[+]  ll5js2v                  ^gettext@0.21%gcc@8.3.1+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-rhel7-power9le
[+]  jansxvo                      ^libiconv@1.16%gcc@8.3.1 libs=shared,static arch=linux-rhel7-power9le
[+]  el5rrpn                      ^libxml2@2.9.13%gcc@8.3.1~python arch=linux-rhel7-power9le
[+]  hdyv36t                          ^pkg-config@0.27.1%gcc@8.3.1+internal_glib patches=49ffcd6 arch=linux-rhel7-power9le
[+]  7pg67zm                          ^xz@5.2.5%gcc@8.3.1~pic libs=shared,static arch=linux-rhel7-power9le
[+]  i2ruf5x                          ^zlib@1.2.12%gcc@8.3.1+optimize+pic+shared patches=0d38234 arch=linux-rhel7-power9le
[+]  voomhfq                      ^tar@1.26%gcc@8.3.1 patches=08921fc,5c314db,d428578 zip=pigz arch=linux-rhel7-power9le
[+]  gr76p7e                  ^libffi@3.4.2%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  kohqqzt                  ^openssl@1.0.2k-fips%gcc@8.3.1~docs~shared certs=mozilla arch=linux-rhel7-power9le
[+]  md3x2ao                  ^pkgconf@1.8.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  xcoahod                  ^sqlite@3.7.17%gcc@8.3.1+column_metadata+dynamic_extensions~fts~functions+rtree arch=linux-rhel7-power9le
[+]  psjcxyv                  ^util-linux-uuid@2.37.4%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  46lho3x          ^py-setuptools@59.4.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  nam32gv          ^py-wheel@0.37.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  ecm3vfg      ^py-joblib@1.1.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  ht2z6lc      ^py-numpy@1.22.4%gcc@8.3.1+blas+lapack patches=873745d arch=linux-rhel7-power9le
[+]  75aylgu          ^openblas@0.3.12%gcc@8.3.1~bignuma~consistent_fpcsr~ilp64+locking+pic+shared patches=865703b symbol_suffix=none threads=openmp arch=linux-rhel7-power9le
[+]  fnxha5a              ^perl@5.16.3%gcc@8.3.1+cpanm+shared+threads patches=0eac10e,3bbd7d6 arch=linux-rhel7-power9le
[+]  umfswia          ^py-pytest@6.2.5%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  qk4pfqw              ^py-attrs@21.4.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  gmibnvd              ^py-iniconfig@1.1.1%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  e74h5pb              ^py-packaging@21.3%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  cdm7v2d                  ^py-pyparsing@3.0.6%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  nmxbugx              ^py-pluggy@1.0.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  x5prmlr                  ^py-setuptools-scm@6.3.2%gcc@8.3.1+toml arch=linux-rhel7-power9le
[+]  jlduczs                      ^py-tomli@1.2.2%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  bic2zkm              ^py-py@1.11.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  xtiha4m              ^py-toml@0.10.2%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  zrdzvhq      ^py-scipy@1.4.0%gcc@8.3.1 patches=5433f60 arch=linux-rhel7-power9le
[+]  clh4mhd          ^py-pybind11@2.3.0%gcc@8.3.1~ipo build_type=RelWithDebInfo arch=linux-rhel7-power9le
[+]  xs5u5o6              ^cmake@3.23.2%gcc@8.3.1~doc+ncurses+ownlibs~qt build_type=Release arch=linux-rhel7-power9le
[+]  q2fesn7              ^ninja@1.10.2%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  d3cwfdn                  ^python@3.9.10%gcc@8.3.1+bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,f2fd060 arch=linux-rhel7-power9le
[+]  4nmedyo                      ^bzip2@1.0.6%gcc@8.3.1~debug~pic+shared arch=linux-rhel7-power9le
[+]  xs4fkq4                      ^gdbm@1.23%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  32om6zc                          ^readline@8.1%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  p4uz2jr                              ^ncurses@5.9.20130511%gcc@8.3.1~symlinks+termlib abi=5 patches=f84b270 arch=linux-rhel7-power9le
[+]  ll5js2v                      ^gettext@0.21%gcc@8.3.1+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-rhel7-power9le
[+]  voomhfq                          ^tar@1.26%gcc@8.3.1 patches=08921fc,5c314db,d428578 zip=pigz arch=linux-rhel7-power9le
[+]  kohqqzt                      ^openssl@1.0.2k-fips%gcc@8.3.1~docs~shared certs=mozilla arch=linux-rhel7-power9le
[+]  xcoahod                      ^sqlite@3.7.17%gcc@8.3.1+column_metadata+dynamic_extensions~fts~functions+rtree arch=linux-rhel7-power9le
[+]  zfh2u4w      ^py-threadpoolctl@3.0.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  5o4loc6          ^py-flit@3.6.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  qhsok5f              ^py-docutils@0.18.1%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  rci7wuo              ^py-flit-core@3.6.0%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  ly5fkwp              ^py-requests@2.26.0%gcc@8.3.1~socks arch=linux-rhel7-power9le
[+]  njwa77h                  ^py-certifi@2021.10.8%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  35ffmio                  ^py-charset-normalizer@2.0.12%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  lcnf3my                  ^py-idna@3.3%gcc@8.3.1 arch=linux-rhel7-power9le
[+]  25pxfll                  ^py-urllib3@1.26.6%gcc@8.3.1~brotli~secure~socks arch=linux-rhel7-power9le
[+]  kesuztb              ^py-tomli-w@1.0.0%gcc@8.3.1 arch=linux-rhel7-power9le

$ spack env activate -p lbann-atom-bve-power9le
$ [lbann-atom-bve-power9le] (5) vanessen@lassen28 [ATOM] %> python3 -m scipy                                                                                                                                                                                                                                                                                                                                                                
Traceback (most recent call last):
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/._view/7gzz3vsftrjbrrjiwvjngjopxgjtz6jn/lib/python3.9/runpy.py", line 188, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/._view/7gzz3vsftrjbrrjiwvjngjopxgjtz6jn/lib/python3.9/runpy.py", line 147, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/._view/7gzz3vsftrjbrrjiwvjngjopxgjtz6jn/lib/python3.9/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/__init__.py", line 156, in <module>
    from . import fft
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/__init__.py", line 81, in <module>
    from ._helper import next_fast_len
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_helper.py", line 4, in <module>
    from . import _pocketfft
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_pocketfft/__init__.py", line 3, in <module>
    from .basic import *
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_pocketfft/basic.py", line 8, in <module>
    from . import pypocketfft as pfft
ImportError: /usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-powerpc64le-linux-gnu.so: undefined symbol: pthread_atfork
[lbann-atom-bve-power9le] (5) vanessen@lassen28 [ATOM] %> User defined signal 2
```


### Error message

<details><summary>Error message</summary><pre>
[lbann-atom-bve-power9le] (5) vanessen@lassen28 [ATOM] %> which python3
/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/bin/python3
[lbann-atom-bve-power9le] (5) vanessen@lassen28 [ATOM] %> python3 -m scipy                                                                                                                                                                                                                                                                                                                                                                
Traceback (most recent call last):
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/._view/7gzz3vsftrjbrrjiwvjngjopxgjtz6jn/lib/python3.9/runpy.py", line 188, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/._view/7gzz3vsftrjbrrjiwvjngjopxgjtz6jn/lib/python3.9/runpy.py", line 147, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/._view/7gzz3vsftrjbrrjiwvjngjopxgjtz6jn/lib/python3.9/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/__init__.py", line 156, in <module>
    from . import fft
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/__init__.py", line 81, in <module>
    from ._helper import next_fast_len
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_helper.py", line 4, in <module>
    from . import _pocketfft
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_pocketfft/__init__.py", line 3, in <module>
    from .basic import *
  File "/usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_pocketfft/basic.py", line 8, in <module>
    from . import pypocketfft as pfft
ImportError: /usr/WS1/vanessen/spack_bvanessen.git/var/spack/environments/lbann-atom-bve-power9le/.spack-env/view/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-powerpc64le-linux-gnu.so: undefined symbol: pthread_atfork
</pre></details>


### Information on your system

```
* **Spack:** 0.19.0.dev0 (e153625aa752eedcc8e7f3d2de17a21ce1911fac)
* **Python:** 3.7.2
* **Platform:** linux-rhel7-power9le
* **Concretizer:** clingo
```

### Additional information

_No response_

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
- [X] I have uploaded the build log and environment files
- [X] I have searched the issues of this repo and believe this is not a duplicate


Discussion:
Does the same problem occur with scipy 1.8? Is there a reason you need scipy 1.4?

@rgommers are there any known issues with scipy FFT on power9 that have been fixed in newer releases?
This was fixed in scipy `1.4.2` and `1.5.0`: https://github.com/scipy/scipy/pull/11324
I don't think this is worth carrying patches in Spack or other such systems for by the way. As a rule, even if you really need 1.4 and not a newer release, you should always use the last bugfix release, and not the `.0` release - that's why bugfix releases exist.
If these aren’t viable versions on Power, can we mark them as invalid for Power architectures?

Brian C. Van Essen
***@***.***
(w) 925-422-9300
(c) 925-290-5470


From: Ralf Gommers ***@***.***>
Reply-To: spack/spack ***@***.***>
Date: Wednesday, August 10, 2022 at 11:26 PM
To: spack/spack ***@***.***>
Cc: Brian Van Essen ***@***.***>, Author ***@***.***>
Subject: Re: [spack/spack] Installation issue: py-scipy does not build correctly (Issue #32043)


I don't think this is worth carrying patches in Spack or other such systems for by the way. As a rule, even if you really need 1.4 and not a newer release, you should always use the last bugfix release, and not the .0 release - that's why bugfix releases exist.

—
Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https:/github.com/spack/spack/issues/32043*issuecomment-1211595015__;Iw!!G2kpM7uM-TzIFchu!hzJKSAJlYa-XassMrd6adymTbgAURy_rP5kdG0w6vpOEfPJIiulxAwm2ZOFeWWqp2w$>, or unsubscribe<https://urldefense.us/v3/__https:/github.com/notifications/unsubscribe-auth/ABPME6ZPXW2NFUTRTGUU5RDVYSMHTANCNFSM56GGMQBQ__;!!G2kpM7uM-TzIFchu!hzJKSAJlYa-XassMrd6adymTbgAURy_rP5kdG0w6vpOEfPJIiulxAwm2ZOEbdoISRQ$>.
You are receiving this because you authored the thread.Message ID: ***@***.***>


Resolving patch:
diff --git a/var/spack/repos/builtin/packages/py-scipy/package.py b/var/spack/repos/builtin/packages/py-scipy/package.py
--- a/var/spack/repos/builtin/packages/py-scipy/package.py
+++ b/var/spack/repos/builtin/packages/py-scipy/package.py
@@ -95,6 +95,9 @@ class PyScipy(PythonPackage):
     # https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
     depends_on("lapack@3.4.1:", when="@1.2:")
 
+    # https://github.com/scipy/scipy/pull/11324
+    conflicts("@1.4.0:1.4.1", when="target=ppc64le:")
+
     # https://github.com/scipy/scipy/issues/12860
     patch(
         "https://git.sagemath.org/sage.git/plain/build/pkgs/scipy/patches/extern_decls.patch?id=711fe05025795e44b84233e065d240859ccae5bd",


Test patch: