Repository: scipy/scipy

Issue:
ENH: Add wrapper for LAPACK functions stgsyl and dtgsyl
### Is your feature request related to a problem? Please describe.

I need the LAPACK functions ```stgsyl``` and ```dtgsyl``` for a dynamic system analyzing package.

### Describe the solution you'd like.

A low-level wrapper in the sub-package [scipy.linalg.lapack](https://docs.scipy.org/doc/scipy/reference/linalg.lapack.html#module-scipy.linalg.lapack) would be sufficient.

### Describe alternatives you've considered.

Currently I use the [python-julia interface](https://pypi.org/project/julia/) and the [sylvsys function](https://docs.juliahub.com/MatrixEquations/1uOBF/1.2.0/sylvester.html) in the MatrixEquations.jl package.
However, a pure Python solution would be advantageous.

### Additional context (e.g. screenshots, GIFs)

_No response_


Discussion:
Hi. I want to give this one a go. But since it's my first time going through the scipy source, I could use some help. I'm trying to base changes on https://github.com/scipy/scipy/commit/2aac3f31fe4a382ddf637af55d963ae9527acf57.  But I have some questions about  the *.pyf.src syntax and will likely need some help with testing too. Let me know if anyone can help.
@Kai-Striega maybe you would be able to help here?
I can also help. The initial steps are writing a `<prefix2>`  wrapper since only s, and d flavors are relevant here. It can go into `flapack_other.pyf.src`. We will also need a wrapper for `<prefix2>tgsyl_lwork` to query the `lwork` parameter.

 In the protoargument we `<ctype2>` and in the body of the wrapper wherever needed this is reflected as `<ftype2>` to generate wrapper both for s and d in one wrapper. These are the ones that need templating. The definitions of these are found in https://github.com/scipy/scipy/blob/main/scipy/linalg/flapack.pyf.src#L10-L25

![image](https://github.com/scipy/scipy/assets/1303842/7f82cc19-42db-4b1a-9801-ccb2274b06cb)

A simpler example can be found here 

https://github.com/scipy/scipy/blob/main/scipy/linalg/flapack_sym_herm.pyf.src#L239-L259

@mainak33 I've got two unmerged PRs for you to look at: https://github.com/scipy/scipy/pull/11890, https://github.com/scipy/scipy/pull/11985 these implement `?gbsvx` and `?pbsvx` respectively, including tests. These have not been merged because we didn't support the required LAPACK version. This could also be a problem for `?tgsyl` (I don't know in what version it was introduced, off the top of my head). According to the [Toolchain roadmap](https://docs.scipy.org/doc/scipy/dev/toolchain.html#other-libraries) we currently use LAPACK 3.7.1. Does `?tgsyl` fit within that requirement?

@ilayn LAPACK version 3.7.1 was released 25/06/17 - this seems a bit dated to me, do you think it would be worth bumping the version a bit?
> LAPACK version 3.7.1 was released 25/06/17 - this seems a bit dated to me, do you think it would be worth bumping the version a bit?

Historically, it depends on what Mac/Linux distros offer, by default, but because OpenBLAS has become more prominent, I think that situation got a bit better. I am following if we are missing very crucial ones on the new LAPACK versions but not much has changed substantially

@ilayn Thanks for the information. I was unaware of the lapack version dependency and had started with the 3.11 doc, You can see the ongoing work here: https://github.com/scipy/scipy/compare/main...mainak33:scipy:mainak33/ENH18404 . Lapack v3.11 also had c/ztgsyl in addition to s/dtgsyl. So, I was trying to wrap those as well as using <ctype> instead of <ctype2>. However the IWORK dimension in the real vs complex types is M+N+6 vs M+N+2. Does that mean I have to write separate templates for real/complex versions or is there a way to check the template type within the function? Also, I thought into flapack_sym_herm.pyf.src because in the doc *sytrf function is in the same sections and that wrapper is in this file. I am also not clear about the usage of the _lwork wrappers routines? What is their purpose?  
> @mainak33 I've got two unmerged PRs for you to look at: #11890, #11985 these implement `?gbsvx` and `?pbsvx` respectively, including tests. These have not been merged because we didn't support the required LAPACK version. This could also be a problem for `?tgsyl` (I don't know in what version it was introduced, off the top of my head). According to the [Toolchain roadmap](https://docs.scipy.org/doc/scipy/dev/toolchain.html#other-libraries) we currently use LAPACK 3.7.1. Does `?tgsyl` fit within that requirement?
> 
> @ilayn LAPACK version 3.7.1 was released 25/06/17 - this seems a bit dated to me, do you think it would be worth bumping the version a bit?

Thank you for the info. I'll take a look and maybe find some answers to my questions above. I cannot find the doc for Lapack v3.7.1 easily to see if the *tgsyl functions are there. If the functions are not there, is it worth pursuing a PR so that it can be merged in the future?
> Does that mean I have to write separate templates for real/complex versions

yes that's why we have `<prefix2>` and `<prefix2c>` distinctions. It's just painful to do it but unfortunately necessary. Also we hide all the interim variables instead of outputting them such as work and iwork and bunch of other things that it wants to spit out. 

> Also, I thought into flapack_sym_herm.pyf.src 

Lapack classification is a complete chaos. So doxygen ordering is not to be trusted. These arrays are not symmetric so let's bunch them into `...other.pyf.src`. 

> I am also not clear about the usage of the _lwork wrappers routines?

Because LAPACK is based on F77 there is this tendency to send in the variables and get a result in the same variables in place. So if you want to get the machine specific value of `lwork` you first invoke the function with `lwork` set to -1. That overrides the first element of `work` to the correct value you take that value and invoke the function again with `lwork` set to that value. Ideally it doesn't reference the remaining variables so essentially they can even be NULLs. It's just ... history. But there are also lower bounds to this value that allows the routines to do blocked calls and those are the numbers you are seeing as minimum numbers. So if you don't want to do lwork query then you use at least that much of amount. 

But because we are using f2py which automatically generates those interim arrays, just to ask for lwork number it is very wasteful to create and destroy those big arrays. Instead we create tiny array just for LAPACK to fill in and it writes into that array. We read the number and ask again for the actual routine to work with all variables initialized. 

You can read off from just above your work for the differences between sytrf wrapper and the sytrf_lwork  wrapper. 


> If the functions are not there,

I just checked, they are quite old functions so no problem in that regard. In fact they are from `?xxsyl` family.
The good ones are here https://github.com/Reference-LAPACK/lapack/pull/651 but they are brand new and we can't jump on to that yet.

Resolving patch:
diff --git a/scipy/linalg/lapack.py b/scipy/linalg/lapack.py
--- a/scipy/linalg/lapack.py
+++ b/scipy/linalg/lapack.py
@@ -695,6 +695,9 @@
    ctgsen_lwork
    ztgsen_lwork
 
+   stgsyl
+   dtgsyl
+
    stpttf
    dtpttf
    ctpttf


Test patch:
diff --git a/scipy/linalg/tests/test_lapack.py b/scipy/linalg/tests/test_lapack.py
--- a/scipy/linalg/tests/test_lapack.py
+++ b/scipy/linalg/tests/test_lapack.py
@@ -18,7 +18,7 @@
 from numpy.random import rand, randint, seed
 
 from scipy.linalg import (_flapack as flapack, lapack, inv, svd, cholesky,
-                          solve, ldl, norm, block_diag, qr, eigh)
+                          solve, ldl, norm, block_diag, qr, eigh, qz)
 
 from scipy.linalg.lapack import _compute_lwork
 from scipy.stats import ortho_group, unitary_group
@@ -3298,3 +3298,106 @@ def test_gges_tgsen(dtype):
 
     assert_allclose(s[0, 0] / t[0, 0], d2, rtol=0, atol=atol)
     assert_allclose(s[1, 1] / t[1, 1], d1, rtol=0, atol=atol)
+
+
+@pytest.mark.parametrize(
+    "a, b, c, d, e, f, rans, lans",
+    [(np.array([[4.0,   1.0,  1.0,  2.0],
+                [0.0,   3.0,  4.0,  1.0],
+                [0.0,   1.0,  3.0,  1.0],
+                [0.0,   0.0,  0.0,  6.0]]),
+      np.array([[1.0,   1.0,  1.0,  1.0],
+                [0.0,   3.0,  4.0,  1.0],
+                [0.0,   1.0,  3.0,  1.0],
+                [0.0,   0.0,  0.0,  4.0]]),
+      np.array([[-4.0,  7.0,  1.0, 12.0],
+                [-9.0,  2.0, -2.0, -2.0],
+                [-4.0,  2.0, -2.0,  8.0],
+                [-7.0,  7.0, -6.0, 19.0]]),
+      np.array([[2.0,   1.0,  1.0,  3.0],
+                [0.0,   1.0,  2.0,  1.0],
+                [0.0,   0.0,  1.0,  1.0],
+                [0.0,   0.0,  0.0,  2.0]]),
+      np.array([[1.0,   1.0,  1.0,  2.0],
+                [0.0,   1.0,  4.0,  1.0],
+                [0.0,   0.0,  1.0,  1.0],
+                [0.0,   0.0,  0.0,  1.0]]),
+      np.array([[-7.0,  5.0,  0.0,  7.0],
+                [-5.0,  1.0, -8.0,  0.0],
+                [-1.0,  2.0, -3.0,  5.0],
+                [-3.0,  2.0,  0.0,  5.0]]),
+      np.array([[1.0,   1.0,  1.0,  1.0],
+                [-1.0,  2.0, -1.0, -1.0],
+                [-1.0,  1.0,  3.0,  1.0],
+                [-1.0,  1.0, -1.0,  4.0]]),
+      np.array([[4.0,  -1.0,  1.0, -1.0],
+                [1.0,   3.0, -1.0,  1.0],
+                [-1.0,  1.0,  2.0, -1.0],
+                [1.0,  -1.0,  1.0,  1.0]]))])
+@pytest.mark.parametrize('dtype', REAL_DTYPES)
+def test_tgsyl_NAG(a, b, c, d, e, f, rans, lans, dtype):
+    atol = 1e-4
+
+    tgsyl = get_lapack_funcs(('tgsyl'), dtype=dtype)
+    rout, lout, scale, dif, info = tgsyl(a, b, c, d, e, f)
+
+    assert_equal(info, 0)
+    assert_allclose(scale, 1.0, rtol=0, atol=np.finfo(dtype).eps*100,
+                    err_msg="SCALE must be 1.0")
+    assert_allclose(dif, 0.0, rtol=0, atol=np.finfo(dtype).eps*100,
+                    err_msg="DIF must be nearly 0")
+    assert_allclose(rout, rans, atol=atol,
+                    err_msg="Solution for R is incorrect")
+    assert_allclose(lout, lans, atol=atol,
+                    err_msg="Solution for L is incorrect")
+
+
+@pytest.mark.parametrize('dtype', REAL_DTYPES)
+@pytest.mark.parametrize('trans', ('N', 'T'))
+@pytest.mark.parametrize('ijob', [0, 1, 2, 3, 4])
+def test_tgsyl(dtype, trans, ijob):
+
+    atol = 1e-3 if dtype == np.float32 else 1e-10
+    rng = np.random.default_rng(1685779866898198)
+    m, n = 10, 15
+
+    a, d, *_ = qz(rng.uniform(-10, 10, [m, m]).astype(dtype),
+                  rng.uniform(-10, 10, [m, m]).astype(dtype),
+                  output='real')
+
+    b, e, *_ = qz(rng.uniform(-10, 10, [n, n]).astype(dtype),
+                  rng.uniform(-10, 10, [n, n]).astype(dtype),
+                  output='real')
+
+    c = rng.uniform(-2, 2, [m, n]).astype(dtype)
+    f = rng.uniform(-2, 2, [m, n]).astype(dtype)
+
+    tgsyl = get_lapack_funcs(('tgsyl'), dtype=dtype)
+    rout, lout, scale, dif, info = tgsyl(a, b, c, d, e, f,
+                                         trans=trans, ijob=ijob)
+
+    assert info == 0, "INFO is non-zero"
+    assert scale >= 0.0, "SCALE must be non-negative"
+    if ijob == 0:
+        assert_allclose(dif, 0.0, rtol=0, atol=np.finfo(dtype).eps*100,
+                        err_msg="DIF must be 0 for ijob =0")
+    else:
+        assert dif >= 0.0, "DIF must be non-negative"
+
+    # Only DIF is calculated for ijob = 3/4
+    if ijob <= 2:
+        if trans == 'N':
+            lhs1 = a @ rout - lout @ b
+            rhs1 = scale*c
+            lhs2 = d @ rout - lout @ e
+            rhs2 = scale*f
+        elif trans == 'T':
+            lhs1 = np.transpose(a) @ rout + np.transpose(d) @ lout
+            rhs1 = scale*c
+            lhs2 = rout @ np.transpose(b) + lout @ np.transpose(e)
+            rhs2 = -1.0*scale*f
+
+        assert_allclose(lhs1, rhs1, atol=atol, rtol=0.,
+                        err_msg='lhs1 and rhs1 do not match')
+        assert_allclose(lhs2, rhs2, atol=atol, rtol=0.,
+                        err_msg='lhs2 and rhs2 do not match')