Repository: scipy/scipy

Issue:
cython_blas does not use const in signatures
Usually, BLAS interfaces for C use function signatures with const qualifiers - examples:
```c
void F77_daxpy( FINT, const double *, const double *, FINT, double *, FINT);
```
```c
F77_NAME(daxpy)(const int *n, const double *da,
		const double *dx, const int *incx,
		double *dy, const int *incy);
```

However, the Cython `pxd` file for `cython_blas` generates the function signatures **without** `const` qualifier:
```python
cdef void daxpy(int *n, d *da, d *dx, int *incx, d *dy, int *incy) nogil
```

Which makes it harder to wrap into existing programs if they are using `const`.


Discussion:
That seems like a good idea to add. Thanks for the suggestion @david-cortes. 
Cython did not support the ```const``` qualifier when these wrappers were written. That is the reason ```const``` was omitted. This was not done by accident or sloppiness. Cython does support ```const``` now, however, so we can use it.
It is not difficult to use the wrappers in the presence of ```const``` data though. We just have to write a tiny inline wrapper function that e.g. uses a C++ ```const_cast``` operator to cast away the constness. In Cython ```const_cast``` can be made available using the statement
```from libcpp.cast import const_cast```
When we do this the overhead in the generated machine code is exactly zero, as the C++ compiler will optimize away the wrapper.
Basically we can work around const like this, which is not difficult, just a bit tedious:

```Cython
from scipy.linalg.cython_blas cimport daxpy
from libcpp.cast cimport const_cast

cdef inline void const_daxpy(const int *n, const double *da,
		const double *dx, const int *incx,
		double *dy, const int *incy) nogil:
    cdef: 
        double *tda, *tdx
        int *tn, *tincx, *tincy
    tda = const_cast[double*](da)
    tdx = const_cast[double*](dx)
    tn = const_cast[int*](n)
    tincx = const_cast[int*](incx)
    tincy = const_cast[int*](incy)
    daxpy(tn, tda, tdx, tincx, dy, tincy)
```







And if we instead use Cython to generate C, it is

```cython
from scipy.linalg.cython_blas cimport daxpy

cdef inline void const_daxpy(const int *n, const double *da,
		const double *dx, const int *incx,
		double *dy, const int *incy) nogil:
    cdef: 
        double *tda, *tdx
        int *tn, *tincx, *tincy
    tda = <double*>(da)
    tdx = <double*>(dx)
    tn = <int*>(n)
    tincx = <int*>(incx)
    tincy = <int*>(incy)
    daxpy(tn, tda, tdx, tincx, dy, tincy)
```    
I assume the module is calling these functions from a dynamically-linked BLAS library, which already has them with `const`.

Shouldn't it be enough with just adding `const` in the `extern` declarations and in the script that generates the exported .pxd?
Is a PR welcome that adds the `const` qualifier? Or is it more complicated with backwards compatibility issues?
It should be ok. The compiler will cast ```T *``` to ```const T *``` but not the other way around. It is the absence of ```const``` that cause the problems (i.e. requiring an explicit ```const_cast``` of ```const T *``` to ```T *``` when calling BLAS). Adding ```const``` qualifier to the prototypes can be done with impunity. It will only help to solve problems. It should not cause any backwards compatibility issues. In my opinion such a PR would be welcome.
https://www.damnyouautocorrect.com
I know we have a pandemic and all, but impunity is not immunity…
Looking a bit deeper, adding `const` qualifier seems tricky. The Cython interface is generated automatically by `_cython_signature_generator.py` which inspects at the BLAS fortran files. The problem then arises because there are 2 different routines:
- routines that do not modify arguments like `ddot(const int, n, const double *x, const int incx, const double *y, const int incy)`, i.e. all arguments can be declared const. See [MKL C reference cblas_?dot](https://www.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/blas-and-sparse-blas-routines/blas-routines/blas-level-1-routines-and-functions/cblas-dot.html)
- routines that modify an argument like `daxpy(const int n, const double a, const double *x, const int incx, double *y, const int incy)`,i.e. argument `y` is modified inplace. See [MKL C reference cblas_?axpy](https://www.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/blas-and-sparse-blas-routines/blas-routines/blas-level-1-routines-and-functions/cblas-axpy.html)

Does someone has a good idea how to do that distinction automatically rather than listing them manually (which would obfuscate `_cython_signature_generator.py`)? It seems save to assume that if the return type is not void than all arguments can be declared const.
The reference implementation only seems to list intent in the docs (which is not surprising since `intent(in, out) ` qualifiers are fortran 90 only. So I guess it's easiest to list it manually. 
Yeah, it is not a trivial problem. The constness is declared in the CBLAS and LAPACKE headers, but ```cython_blas``` and ```cython_lapack``` expose the ABI of the Fortran 77 routines rather than the official C interface. If you parse down the CBLAS and LAPACE headers, you should be able to interfere the constness in the corresponding Fortran routines from the C signatures.

Arguably we might doing this the wrong way and should perhaps just leave ```cython_blas``` and ```cython_lapack``` as they are. After all, there are no concept of const arrays in Fortran 77. Instead the subroutines in BLAS and LAPACK have comments that tell if input will be altered or not. I think the more correct solution would be to create two new modules, ```cython_cblas``` and ```cython_lapacke```, that expose the official C interface of BLAS and LAPACK. The constness would follow naturally as it is already declared manually in the official C headers.
> I think the more correct solution would be to create two new modules, cython_cblas and cython_lapacke, that expose the official C interface of BLAS and LAPACK. The constness would follow naturally as it is already declared manually in the official C headers.

This would also make things easier for C contiguous arrays which we are jumping multiple hoops for and losing no-nnegligible performance. However LAPACKE and CBLAS is a tiny bit less stable and I'm not sure if OpenBLAS is up-to-spec
BTW the non-cblas C headers (which are F77 wrappers) from MKL, OpenBLAS, BLIS, and R’s blas do come with `const`: https://github.com/wch/r-source/blob/79298c499218846d14500255efd622b5021c10ec/src/include/R_ext/BLAS.h

But then I guess the `const`ness would either have to be specified manually, or determined from some automated parser plus a reference header from a specific implementation like OpenBLAS.
In Pyodide's build process, we get a few compile errors related to `const` inconsistency in signatures so we have add a simple patch. I wonder if this would help.
> I think the more correct solution would be to create two new modules, cython_cblas and cython_lapacke, that expose the official C interface of BLAS and LAPACK. The constness would follow naturally as it is already declared manually in the official C headers.

IIUC, a PR would "just" need to copy over those 2 header files from blas and lapack? But from which version? How do deal with `lapack_manual_wrappers`, `blas_exclusions` and `lapack_exclusions` in the file `_cython_signature_generator.py`?
Ηey everyone! I plan to tackle this issue in the next days and I’d like a quick sanity check on a grob plan.

First of all, here's my understanding of the current pipeline to generate the Cython`pxd`:
- `scipy/linalg/_cython_signature_generator.py` generates `scipy/linalg/cython_blas_signatures.txt`
- The signatures in the txt file are then used to generate `cython_blas.pxd` and company from `scipy/linalg/_generate_pyx.py`
- Question: Is my understanding correct that the fortran symbols, used in `scipy/linalg/_cython_signature_generator.py`, are from LAPACK 3.4.0?

A grob plan of what I think is a sensible way going forward is:
- Use CBLAS headers to parse the function signatures and generate `scipy/linalg/cython_blas_signatures.txt`
- Make any necessary transformations in `scipy/linalg/_generate_pyx.py` in order to be able to read the new signatures (not sure if needed)
- Question: If this makes sense, should we be using the [LAPACK 3.7.1 headers](https://github.com/Reference-LAPACK/lapack/blob/v3.7.1/CBLAS/include/cblas.h) (according to [the toolchain roadmap](https://docs.scipy.org/doc/scipy/dev/toolchain.html#other-libraries)). What if there are differences with the current list of exported symbols in `scipy.linalg.blas`? Should the list remain the same?
> Question: Is my understanding correct that the fortran symbols, used in `scipy/linalg/_cython_signature_generator.py`, are from LAPACK 3.4.0?

Yes, that looks correct - with the minor note that the routines that were removed in LAPACK 3.6.0 are not included in `cython_lapack_signatures.txt`, that should have been taken care of by commit 0528f14.


> Use CBLAS headers to parse the function signatures and generate `scipy/linalg/cython_blas_signatures.txt`

I'm not sure about that. The `cython_signature_generator.py` script uses the Fortran headers and the Netlib BLAS/LAPACK sources. That script should already do the right thing, mostly. Maybe enhancing that script by fishing const arguments from CBLAS/LAPACKE headers would be easier than starting from scratch? Really not sure though, and the script is <200 LoC so maybe it's not too bad to change it completely.

> Question: If this makes sense, should we be using the [LAPACK 3.7.1 headers](https://github.com/Reference-LAPACK/lapack/blob/v3.7.1/CBLAS/include/cblas.h) (according to [the toolchain roadmap](https://docs.scipy.org/doc/scipy/dev/toolchain.html#other-libraries)). What if there are differences with the current list of exported symbols in `scipy.linalg.blas`? Should the list remain the same?

Using LAPACK 3.7.1 sounds good. It may drag in some new routines, and that'd be a good thing. If there are other differences that show up, then not sure - it depends on what those would look like. 


Resolving patch:
diff --git a/scipy/linalg/_cython_signature_generator.py b/scipy/linalg/_cython_signature_generator.py
--- a/scipy/linalg/_cython_signature_generator.py
+++ b/scipy/linalg/_cython_signature_generator.py
@@ -9,14 +9,19 @@
 To generate the LAPACK wrapper signatures call:
 python _cython_signature_generator.py lapack <lapack_src_directory> <out_file>
 
-This script expects to be run on the source directory for
-the oldest supported version of LAPACK (currently 3.4.0).
+This script expects to be run on the root directory of the LAPACK or BLAS source code, that
+contains the Fortran implementation.
+
+It was last run with netlib's LAPACK 3.7.1 (https://netlib.org/lapack/#_lapack_version_3_7_1).
 """
 
-import glob
-import os
+import pathlib
+import re
 from numpy.f2py import crackfortran
 
+PARAM = re.compile(r"\*> \\param\[(?P<const>(in|out|in,out))\]\s+(?P<name>\w+)\s*")
+
+
 sig_types = {'integer': 'int',
              'complex': 'c',
              'double precision': 'd',
@@ -27,21 +32,50 @@
              'logical': 'bint'}
 
 
-def get_type(info, arg):
-    argtype = sig_types[info['vars'][arg]['typespec']]
-    if argtype == 'c' and info['vars'][arg].get('kindselector') is not None:
+def get_type(info, name, const_args=None):
+    argtype = sig_types[info['vars'][name]['typespec']]
+    if argtype == 'c' and info['vars'][name].get('kindselector') is not None:
         argtype = 'z'
+    if const_args is not None and name.lower() in const_args:
+        argtype = f"const {argtype}"
     return argtype
 
 
-def make_signature(filename):
-    info = crackfortran.crackfortran(filename)[0]
-    name = info['name']
-    if info['block'] == 'subroutine':
-        return_type = 'void'
-    else:
-        return_type = get_type(info, name)
-    arglist = [' *'.join([get_type(info, arg), arg]) for arg in info['args']]
+def get_const_args(file):
+    const_args = set()
+    with open(file) as f:
+        for line in f.readlines():
+            sline = line.strip()
+            if sline.startswith("SUBROUTINE"):
+                break
+            mo = PARAM.match(line)
+            if mo is None:
+                continue
+            if 'out' not in mo.group('const'):
+                const_args.add(mo.group('name').lower())
+    return const_args
+
+
+def make_signature(file):
+    info = iter(crackfortran.crackfortran(str(file)))
+    const_args = get_const_args(file)
+
+    while True:
+        try:
+            subinfo = next(info)
+        except StopIteration:
+            return None
+
+        name = subinfo['name']
+        if subinfo['block'] == 'subroutine':
+            return_type = 'void'
+            break
+        elif subinfo['block'] == 'function':
+            return_type = get_type(subinfo, name)
+            break
+
+    arglist = [' *'.join([get_type(subinfo, arg, const_args), arg])
+               for arg in subinfo['args']]
     args = ', '.join(arglist)
     # Eliminate strange variable naming that replaces rank with rank_bn.
     args = args.replace('rank_bn', 'rank')
@@ -53,19 +87,19 @@ def get_sig_name(line):
 
 
 def sigs_from_dir(directory, outfile, manual_wrappers=None, exclusions=None):
-    if directory[-1] in ['/', '\\']:
-        directory = directory[:-1]
-    files = sorted(glob.glob(directory + '/*.f*'))
+    files = sorted(directory.glob("*.f*"))
     if exclusions is None:
         exclusions = []
     if manual_wrappers is not None:
         exclusions += [get_sig_name(l) for l in manual_wrappers.split('\n')]
     signatures = []
-    for filename in files:
-        name = os.path.splitext(os.path.basename(filename))[0]
+    for file in files:
+        name = file.stem
         if name in exclusions:
             continue
-        signatures.append(make_signature(filename))
+        signature = make_signature(file)
+        if signature is not None:
+            signatures.append(signature)
     if manual_wrappers is not None:
         signatures += [l + '\n' for l in manual_wrappers.split('\n')]
     signatures.sort(key=get_sig_name)
@@ -172,12 +206,17 @@ def sigs_from_dir(directory, outfile, manual_wrappers=None, exclusions=None):
               'clarscl2', 'dlarscl2', 'slarscl2', 'zlarscl2',
               'clascl2', 'dlascl2', 'slascl2', 'zlascl2',
               'cla_wwaddw', 'dla_wwaddw', 'sla_wwaddw', 'zla_wwaddw',
+              'dbdsvdx', 'dtrevc3', 'sbdsvdx', 'strevc3', # These 4 fail, because of a bug in f2py
+                                                          # see https://github.com/numpy/numpy/issues/23533
+              'ssyconvf', # This fails because the symbol does not get exported correctly on macOS,
+                          # see gh-18247
               ]
 
 
 if __name__ == '__main__':
     from sys import argv
-    libname, src_dir, outfile = argv[1:]
+    libname, src_dir_name, outfile = argv[1:]
+    src_dir = pathlib.Path(src_dir_name)
     if libname.lower() == 'blas':
         sigs_from_dir(src_dir, outfile, exclusions=blas_exclusions)
     elif libname.lower() == 'lapack':
diff --git a/scipy/linalg/_generate_pyx.py b/scipy/linalg/_generate_pyx.py
--- a/scipy/linalg/_generate_pyx.py
+++ b/scipy/linalg/_generate_pyx.py
@@ -15,20 +15,34 @@
 BASE_DIR = os.path.abspath(os.path.dirname(__file__))
 
 fortran_types = {'int': 'integer',
+                 'const int': 'integer',
                  'c': 'complex',
+                 'const c': 'complex',
                  'd': 'double precision',
+                 'const d': 'double precision',
                  's': 'real',
+                 'const s': 'real',
                  'z': 'complex*16',
+                 'const z': 'complex*16',
                  'char': 'character',
-                 'bint': 'logical'}
+                 'const char': 'character',
+                 'bint': 'logical',
+                 'const bint': 'logical'}
 
 c_types = {'int': 'int',
+           'const int': 'const int',
            'c': 'npy_complex64',
+           'const c': 'const npy_complex64',
            'd': 'double',
+           'const d': 'const double',
            's': 'float',
+           'const s': 'const float',
            'z': 'npy_complex128',
+           'const z': 'const npy_complex128',
            'char': 'char',
+           'const char': 'const char',
            'bint': 'int',
+           'const bint': 'const int',
            'cselect1': '_cselect1',
            'cselect2': '_cselect2',
            'dselect2': '_dselect2',
@@ -201,14 +215,14 @@ def make_blas_pyx_preamble(all_sigs):
 cdef extern from "_lapack_subroutines.h":
     # Function pointer type declarations for
     # gees and gges families of functions.
-    ctypedef bint _cselect1(npy_complex64*)
-    ctypedef bint _cselect2(npy_complex64*, npy_complex64*)
-    ctypedef bint _dselect2(d*, d*)
-    ctypedef bint _dselect3(d*, d*, d*)
-    ctypedef bint _sselect2(s*, s*)
-    ctypedef bint _sselect3(s*, s*, s*)
-    ctypedef bint _zselect1(npy_complex128*)
-    ctypedef bint _zselect2(npy_complex128*, npy_complex128*)
+    ctypedef bint _cselect1(const npy_complex64*)
+    ctypedef bint _cselect2(const npy_complex64*, const npy_complex64*)
+    ctypedef bint _dselect2(const d*, const d*)
+    ctypedef bint _dselect3(const d*, const d*, const d*)
+    ctypedef bint _sselect2(const s*, const s*)
+    ctypedef bint _sselect3(const s*, const s*, const s*)
+    ctypedef bint _zselect1(const npy_complex128*)
+    ctypedef bint _zselect2(const npy_complex128*, const npy_complex128*)
 
 '''
 
@@ -511,14 +525,14 @@ def generate_blas_pxd(all_sigs):
 
 # Function pointer type declarations for
 # gees and gges families of functions.
-ctypedef bint cselect1(c*)
-ctypedef bint cselect2(c*, c*)
-ctypedef bint dselect2(d*, d*)
-ctypedef bint dselect3(d*, d*, d*)
-ctypedef bint sselect2(s*, s*)
-ctypedef bint sselect3(s*, s*, s*)
-ctypedef bint zselect1(z*)
-ctypedef bint zselect2(z*, z*)
+ctypedef bint cselect1(const c*)
+ctypedef bint cselect2(const c*, const c*)
+ctypedef bint dselect2(const d*, const d*)
+ctypedef bint dselect3(const d*, const d*, const d*)
+ctypedef bint sselect2(const s*, const s*)
+ctypedef bint sselect3(const s*, const s*, const s*)
+ctypedef bint zselect1(const z*)
+ctypedef bint zselect2(const z*, const z*)
 
 """
 
@@ -621,14 +635,14 @@ def c_sub_decl(name, return_type, args):
 """
 
 lapack_decls = """
-typedef int (*_cselect1)(npy_complex64*);
-typedef int (*_cselect2)(npy_complex64*, npy_complex64*);
-typedef int (*_dselect2)(double*, double*);
-typedef int (*_dselect3)(double*, double*, double*);
-typedef int (*_sselect2)(float*, float*);
-typedef int (*_sselect3)(float*, float*, float*);
-typedef int (*_zselect1)(npy_complex128*);
-typedef int (*_zselect2)(npy_complex128*, npy_complex128*);
+typedef int (*_cselect1)(const npy_complex64*);
+typedef int (*_cselect2)(const npy_complex64*, const npy_complex64*);
+typedef int (*_dselect2)(const double*, const double*);
+typedef int (*_dselect3)(const double*, const double*, const double*);
+typedef int (*_sselect2)(const float*, const float*);
+typedef int (*_sselect3)(const float*, const float*, const float*);
+typedef int (*_zselect1)(const npy_complex128*);
+typedef int (*_zselect2)(const npy_complex128*, const npy_complex128*);
 """
 
 cpp_guard = """


Test patch: