Repository: scipy/scipy
Pull request: BLD: Fix meson build on Cygwin.
Mostly fixing dependency contents and order.
Some inclusion of visibility macros before stdlib headers.

<!-- 
Thanks for contributing a pull request! Please ensure that
your PR satisfies the checklist before submitting:
http://scipy.github.io/devdocs/dev/contributor/development_workflow.html#checklist-before-submitting-a-pr

Also, please name and describe your PR as you would write a
commit message:
http://scipy.github.io/devdocs/dev/contributor/development_workflow.html#writing-the-commit-message

Note that we are a team of volunteers; we appreciate your
patience during the review process.

Again, thanks for contributing!
-->

#### Reference issue
<!--Example: Closes gh-WXYZ.-->
Closes #17925

#### What does this implement/fix?
<!--Please explain your changes.-->
Allows a meson build on Cygwin

#### Additional information
<!--Any additional information you think is important.-->
Tests available at DWesl/scipy#4

Problem statement:
BUG: Import error with Cygwin (tested versions 1.9.0, 1.9.3, 1.10.0, 1.11.0)
### Describe your issue.

Build works fine except that I need to add -D_DEFAULT_SOURCE to the C and C++ build arguments in the meson.build file because otherwise math symbols such as M_PI can't be found on my platform.

But then when trying to import scipy.interpolate.UnivariateSpline, I get the error message below.

Not sure why it's looking for a pyx file when the build is already over. 

### Reproducing Code Example

```python
from scipy.interpolate import UnivariateSpline
```


### Error message

```shell
Traceback (most recent call last):
  File "/cygdrive/c/Users/Andi/Desktop/sharedBox/plotBody.py", line 19, in <module>
    from scipy.interpolate import UnivariateSpline
  File "/usr/local/lib/python3.9/site-packages/scipy/interpolate/__init__.py", line 167, in <module>
    from ._interpolate import *
  File "/usr/local/lib/python3.9/site-packages/scipy/interpolate/_interpolate.py", line 8, in <module>
    import scipy.special as spec
  File "/usr/local/lib/python3.9/site-packages/scipy/special/__init__.py", line 671, in <module>
    from . import _orthogonal
  File "/usr/local/lib/python3.9/site-packages/scipy/special/_orthogonal.py", line 81, in <module>
    from scipy import linalg
  File "/usr/local/lib/python3.9/site-packages/scipy/__init__.py", line 200, in __getattr__
    return _importlib.import_module(f'scipy.{name}')
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.9/site-packages/scipy/linalg/__init__.py", line 209, in <module>
    from ._matfuncs import *
  File "/usr/local/lib/python3.9/site-packages/scipy/linalg/_matfuncs.py", line 20, in <module>
    from ._matfuncs_expm import pick_pade_structure, pade_UV_calc
  File "scipy/linalg/_matfuncs_expm.pyx", line 1, in init scipy.linalg._matfuncs_expm
ImportError: No such process
```


### SciPy/NumPy/Python version information

blas_mkl_info:   NOT AVAILABLE blis_info:   NOT AVAILABLE openblas_info:   NOT AVAILABLE atlas_3_10_blas_threads_info:   NOT AVAILABLE atlas_3_10_blas_info:   NOT AVAILABLE atlas_blas_threads_info:   NOT AVAILABLE atlas_blas_info:   NOT AVAILABLE accelerate_info:   NOT AVAILABLE blas_info:     libraries = ['blas', 'blas']     library_dirs = ['/usr/lib']     include_dirs = ['/usr/include']     language = c     define_macros = [('HAVE_CBLAS', None)] blas_opt_info:     define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]     libraries = ['blas', 'blas']     library_dirs = ['/usr/lib']     include_dirs = ['/usr/include']     language = c lapack_mkl_info:   NOT AVAILABLE openblas_lapack_info:   NOT AVAILABLE openblas_clapack_info:   NOT AVAILABLE flame_info:   NOT AVAILABLE atlas_3_10_threads_info:   NOT AVAILABLE atlas_3_10_info:   NOT AVAILABLE atlas_threads_info:   NOT AVAILABLE atlas_info:   NOT AVAILABLE lapack_info:     libraries = ['lapack', 'lapack']     library_dirs = ['/usr/lib']     language = f77 lapack_opt_info:     libraries = ['lapack', 'lapack', 'blas', 'blas']     library_dirs = ['/usr/lib']     language = c     define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]     include_dirs = ['/usr/include']

Hints from issue discussion:
> Not sure why it's looking for a `.pyx` file when the build is already over

For ease of debugging, since you will usually edit the `.pyx` file to fix problems rather than the `.dll` file.  I've run into a few causes for `ImportError` on compiled extension modules: what happens when you run `cygcheck` on the file corresponding to `scipy.linalg._matfuncs_expm`?

EDIT: In DWesl/scipy#1 (1.8 era), I used `_XOPEN_SOURCE=600` rather than `_DEFAULT_SOURCE`; we'll see if that works now when DWesl/scipy#4 finishes compiling in a couple of hours.
Output of cygcheck is 

```
cygcheck ./_matfuncs_expm.cpython-39-x86_64-cygwin.dll
C:\cygwin64\usr\local\lib\python3.9\site-packages\scipy\linalg\_matfuncs_expm.cpython-39-x86_64-cygwin.dll
  C:\cygwin64\bin\cygwin1.dll
    C:\Windows\system32\KERNEL32.dll
      C:\Windows\system32\ntdll.dll
      C:\Windows\system32\KERNELBASE.dll
  C:\cygwin64\bin\libpython3.9.dll
    C:\cygwin64\bin\cygintl-8.dll
      C:\cygwin64\bin\cygiconv-2.dll
    C:\cygwin64\bin\cyggcc_s-seh-1.dll
```

I just tried compiling it with _XOPEN_SOURCE=600. Same ImportError.
Well there goes that idea.  

The error message reminds me most of attempts to open the Tk DLL using Linux functions on Cygwin ([Pillow's ImageTk did this](https://github.com/friedrichromstedt/bughunting-02#description-of-the-bug-and-of-the-expected-behaviour)), but there doesn't seem to be anything of the sort in the source file, especially at top level where it would produce an `ImportError`.

Do other Cython packages on your system import cleanly?  What about extensions compiled using meson?
Just compiled a (pretty much hello-world-ish) example with meson and Cython and it worked without giving the ImportError.

I made the observation that when I comment out _matfuncs_expm in _matfuncs.py, the next import error is:

```
File "scipy/linalg/_decomp_update.pyx", line 1, in init scipy.linalg._decomp_update
ImportError: No such process
```

Seems that both these pyx files are compiled from pyx.in files.. This is another preprocessing step, right?, producing pyx files from the pyx.in files before cythonizing the pyx files? Could the problem stem from this preprocessing step somehow?
My most recent build is telling me that it should be `_XOPEN_SOURCE=700` because that's what's in `pyconfig.h`, so the underlying problem might be that some files include standard headers before `pyconfig.h` sets the visibility macros that say which parts of the standard headers we want to see.  Which headers are you seeing problems in?

Almost certainly separate from the `ImportError`; I still have no idea what's causing that.  If you have the build tree, you should have corresponding `.pyx` files around somewhere you can look through.  My best guess is still "trying to get functions out of a DLL the wrong way", but I don't see anything obvious in either file.
Out of curiosity, how did you get a complete meson build on Cygwin?  When I try, [it doesn't bother linking `_flapack` against blas, so the link fails with undefined symbol references](https://github.com/DWesl/scipy/actions/runs/4093587262/jobs/7059128133#step:9:470). 
I created a file openblas.pc in /usr/pkgconfig with

```
libdir=/usr/lib
includedir=/usr/include

Name: OPENBLAS
Description: openblas
Version: 3.10.1
URL: http://www.netlib.org/lapack/
Libs: -L${libdir} -lblas -llapack
```

Didn't get any linker errors so it seems all the symbols are present in either libblas or liblapack. I have the default Cygwin liblapack-devel and libopenblas (as you probably know, there is no libopenblas-devel, but it seems like all the neccessary headers were around and found somehow).
Just created a simple file with only
```
from _matfuncs_expm import pick_pade_structure, pade_UV_calc
```
in the same directory as the _matfuncs_expm.cpython-39-x86_64-cygwin.dll.

The error I get is
```
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/scipy/linalg/test.py", line 1, in <module>
    from _matfuncs_expm import pick_pade_structure, pade_UV_calc
  File "scipy/linalg/_matfuncs_expm.pyx", line 1, in init scipy.linalg._matfuncs_expm
  File "/usr/local/lib/python3.9/site-packages/scipy/linalg/__init__.py", line 209, in <module>
    from ._matfuncs import *
  File "/usr/local/lib/python3.9/site-packages/scipy/linalg/_matfuncs.py", line 20, in <module>
    from ._matfuncs_expm import pick_pade_structure, pade_UV_calc
ImportError: cannot import name 'pick_pade_structure' from partially initialized module '_matfuncs_expm' (most likely due to a circular import) (/usr/local/lib/python3.9/site-packages/scipy/linalg/_matfuncs_expm.cpython-39-x86_64-cygwin.dll)
```
Also just fired up gdb to step through this import process (as far as debug symbols were available) starting from `PyInit__matfuncs_expm`

The result is here
[https://github.com/akandi/Scipy_1.10.0_BuildOnCygwin/blob/master/gdb_log.txt](https://github.com/akandi/Scipy_1.10.0_BuildOnCygwin/blob/master/gdb_log.txt)
> I created a file openblas.pc in /usr/pkgconfig with
> 
> ```
> libdir=/usr/lib
> includedir=/usr/include
> 
> Name: OPENBLAS
> Description: openblas
> Version: 3.10.1
> URL: http://www.netlib.org/lapack/
> Libs: -L${libdir} -lblas -llapack
> ```

I'd prefer `-llapack -lblas`, but your order might grab Lapack functions from OpenBLAS rather than the reference Lapack.

> Didn't get any linker errors so it seems all the symbols are present in either libblas or liblapack. I have the default Cygwin liblapack-devel and libopenblas (as you probably know, there is no libopenblas-devel, but it seems like all the neccessary headers were around and found somehow).

`liblapack-devel` provides headers for the standard Lapack and Blas routines, but some configury will check for OpenBLAS-specific functions (usually related to setting threading, I think).  I used [meson options to tell the build to use reference lapack and blas](https://scipy.github.io/devdocs/dev/contributor/meson_advanced.html), but that seems to be broken.

Commit message: BLD: Fix meson build on Cygwin.

Mostly fixing dependency contents and order.
Some inclusion of visibility macros before stdlib headers.

Patch to review:
diff --git a/dev.py b/dev.py
index 1594931996e0..26f46274ec4c 100644
--- a/dev.py
+++ b/dev.py
@@ -436,6 +436,13 @@ def setup_build(cls, dirs, args):
                 raise RuntimeError("Can't build into non-empty directory "
                                    f"'{build_dir.absolute()}'")
 
+        if sys.platform == "cygwin":
+            # Cygwin only has netlib lapack, but can link against
+            # OpenBLAS rather than netlib blas at runtime.  There is
+            # no libopenblas-devel to enable linking against
+            # openblas-specific functions or OpenBLAS Lapack
+            cmd.extend(["-Dlapack=lapack", "-Dblas=blas"])
+
         build_options_file = (
             build_dir / "meson-info" / "intro-buildoptions.json")
         if build_options_file.exists():
@@ -547,6 +554,11 @@ def install_project(cls, dirs, args):
         with open(dirs.installed / ".gitignore", "w") as f:
             f.write("*")
 
+        if sys.platform == "cygwin":
+            rebase_cmd = ["/usr/bin/rebase", "--database", "--oblivious"]
+            rebase_cmd.extend(Path(dirs.installed).glob("**/*.dll"))
+            subprocess.check_call(rebase_cmd)
+
         print("Installation OK")
         return
 
diff --git a/scipy/integrate/meson.build b/scipy/integrate/meson.build
index 97c85b99dcf4..1b38a9ed9f46 100644
--- a/scipy/integrate/meson.build
+++ b/scipy/integrate/meson.build
@@ -136,7 +136,7 @@ py3.extension_module('_vode',
   c_args: [numpy_nodepr_api, Wno_unused_variable],
   include_directories: [inc_np, inc_f2py],
   link_args: version_link_args,
-  dependencies: [lapack, fortranobject_dep],
+  dependencies: [lapack, blas, fortranobject_dep],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/integrate'
diff --git a/scipy/linalg/meson.build b/scipy/linalg/meson.build
index 69087e16b785..0b9e28820641 100644
--- a/scipy/linalg/meson.build
+++ b/scipy/linalg/meson.build
@@ -66,7 +66,7 @@ py3.extension_module('_fblas',
   c_args: numpy_nodepr_api,
   include_directories: [inc_np, inc_f2py],
   link_args: version_link_args,
-  dependencies: [blas, lapack, fortranobject_dep],
+  dependencies: [lapack, blas, fortranobject_dep],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/linalg'
@@ -87,7 +87,7 @@ py3.extension_module('_flapack',
   c_args: [numpy_nodepr_api, Wno_empty_body],
   include_directories: [inc_np, inc_f2py],
   link_args: version_link_args,
-  dependencies: [lapack, fortranobject_dep],
+  dependencies: [lapack, blas, fortranobject_dep],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/linalg'
@@ -215,7 +215,7 @@ cython_blas = py3.extension_module('cython_blas',
   link_with: fwrappers,
   include_directories: inc_np,
   link_args: version_link_args,
-  dependencies: [blas, lapack],
+  dependencies: [lapack, blas],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/linalg'
@@ -231,7 +231,7 @@ cython_lapack = py3.extension_module('cython_lapack',
   link_with: fwrappers,
   include_directories: inc_np,
   link_args: version_link_args,
-  dependencies: [blas, lapack],
+  dependencies: [lapack, blas],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/linalg'
diff --git a/scipy/optimize/_trlib/meson.build b/scipy/optimize/_trlib/meson.build
index 66fa0a39205c..c2cabb6c9796 100644
--- a/scipy/optimize/_trlib/meson.build
+++ b/scipy/optimize/_trlib/meson.build
@@ -8,7 +8,7 @@ _trlib = py3.extension_module('_trlib',
     'trlib_tri_factor.c'
   ],
   c_args: cython_c_args,
-  dependencies: [lapack],
+  dependencies: [lapack, blas],
   include_directories: [
     inc_np,
     '../../_lib',
diff --git a/scipy/optimize/meson.build b/scipy/optimize/meson.build
index 37a1f4cd850e..9b030aa44c32 100644
--- a/scipy/optimize/meson.build
+++ b/scipy/optimize/meson.build
@@ -109,7 +109,7 @@ _lbfgsb = py3.extension_module('_lbfgsb',
   fortran_args: fortran_ignore_warnings,
   include_directories: [inc_np, inc_f2py],
   link_args: version_link_args,
-  dependencies: [lapack, fortranobject_dep],
+  dependencies: [lapack, blas, fortranobject_dep],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/optimize'
diff --git a/scipy/sparse/linalg/_dsolve/meson.build b/scipy/sparse/linalg/_dsolve/meson.build
index bf16643fe94c..4634cae247f8 100644
--- a/scipy/sparse/linalg/_dsolve/meson.build
+++ b/scipy/sparse/linalg/_dsolve/meson.build
@@ -199,7 +199,7 @@ _superlu = py3.extension_module('_superlu',
   link_with: [superlu_lib],
   include_directories: [incdir_numpy, 'SuperLU/SRC'],
   link_args: version_link_args,
-  dependencies: [blas, lapack],
+  dependencies: [lapack, blas],
   install: true,
   subdir: 'scipy/sparse/linalg/_dsolve'
 )
diff --git a/scipy/sparse/linalg/_eigen/arpack/meson.build b/scipy/sparse/linalg/_eigen/arpack/meson.build
index bc77d411aabb..5b469a37914f 100644
--- a/scipy/sparse/linalg/_eigen/arpack/meson.build
+++ b/scipy/sparse/linalg/_eigen/arpack/meson.build
@@ -103,7 +103,7 @@ _arpack = py3.extension_module('_arpack',
   link_with: [arpack_lib],
   include_directories: [incdir_numpy, incdir_f2py],
   link_args: version_link_args,
-  dependencies: [lapack, fortranobject_dep],
+  dependencies: [lapack, blas, fortranobject_dep],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/sparse/linalg/_eigen/arpack'
diff --git a/scipy/sparse/linalg/_isolve/meson.build b/scipy/sparse/linalg/_isolve/meson.build
index 3009201862af..9af9fa146e6e 100644
--- a/scipy/sparse/linalg/_isolve/meson.build
+++ b/scipy/sparse/linalg/_isolve/meson.build
@@ -74,7 +74,7 @@ _iterative = py3.extension_module('_iterative',
   fortran_args: fortran_ignore_warnings,
   include_directories: [inc_np, inc_f2py],
   link_args: version_link_args,
-  dependencies: [lapack, fortranobject_dep],
+  dependencies: [lapack, blas, fortranobject_dep],
   install: true,
   link_language: 'fortran',
   subdir: 'scipy/sparse/linalg/_isolve'
diff --git a/scipy/sparse/linalg/_propack/meson.build b/scipy/sparse/linalg/_propack/meson.build
index 745d010ceb0a..898ecfb6b9be 100644
--- a/scipy/sparse/linalg/_propack/meson.build
+++ b/scipy/sparse/linalg/_propack/meson.build
@@ -109,7 +109,7 @@ foreach ele: elements
     link_with: propack_lib,
     c_args: ['-U_OPENMP', _cpp_Wno_cpp],
     fortran_args: _fflag_Wno_maybe_uninitialized,
-    dependencies: [lapack, fortranobject_dep],
+    dependencies: [lapack, blas, fortranobject_dep],
     include_directories: [inc_np, inc_f2py],
     link_args: version_link_args,
     install: true,
diff --git a/scipy/special/_faddeeva.cxx b/scipy/special/_faddeeva.cxx
index 6b619fb70bc5..7378b2770078 100644
--- a/scipy/special/_faddeeva.cxx
+++ b/scipy/special/_faddeeva.cxx
@@ -1,3 +1,4 @@
+#include <pyconfig.h>
 #include <complex>
 #include <cmath>
 
diff --git a/scipy/special/_faddeeva.h b/scipy/special/_faddeeva.h
index d4ec3a755e2c..4def793ec61d 100644
--- a/scipy/special/_faddeeva.h
+++ b/scipy/special/_faddeeva.h
@@ -9,6 +9,7 @@
 #define EXTERN_C_END
 #endif
 
+#include <pyconfig.h>
 #include <complex>
 #include <Python.h>
 
diff --git a/scipy/special/wright.hh b/scipy/special/wright.hh
index ef168cba00ec..749f5f67af31 100644
--- a/scipy/special/wright.hh
+++ b/scipy/special/wright.hh
@@ -1,6 +1,7 @@
 #ifndef WRIGHT_HH
 #define WRIGHT_HH
 
+#include <pyconfig.h>
 #include <complex>
 
 namespace wright {


Code review comments:
diff_hunk: @@ -1,6 +1,7 @@
 #ifndef WRIGHT_HH
 #define WRIGHT_HH
 
+#include <pyconfig.h>
path: scipy/special/wright.hh
text: @user1:
Instead of this, can you move `#include <Python.h>` from line 75 to up here?

@author:
Given that the file is sixteen lines total, I don't think I can do that.

@user1:
Ah sorry, my comment was confusing. What I meant was to make a change in `wright.cc`, moving `#include <Python.h>` above `#include "wright.hh`. And then this change is unnecessary.

@author:
What's the reasoning for preferring to keep `pyconfig.h` out of the header?  Ease of comparison with external source?

Also changed `_wright.h` the same way.

@user1:
It's that `pyconfig.h` isn't actually used in these files. It's a side effect of its contents (`_XOPEN_SOURCE` in this case) that you're after. Which is a sign that something else is wrong. Which is pretty clear cut here. `Python.h` must always be imported before any standard library C/C++ headers, not doing so is a bug. And once the bug is fixed, there's no need for `pyconfig.h`.

@user1:
Unfortunately there's no static checker to enforce the header ordering - if there was one, I'd gladly use it. This bites us too often.

diff_hunk: @@ -9,6 +9,7 @@
 #define EXTERN_C_END
 #endif
 
+#include <pyconfig.h>
path: scipy/special/_faddeeva.h
text: @user1:
Rather than adding this, can you switch the two includes below so `Python.h` comes first?

diff_hunk: @@ -1,3 +1,4 @@
+#include <pyconfig.h>
path: scipy/special/_faddeeva.cxx
text: @user1:
Can you instead move `include "_faddeeva.h"` to here?