Repository: scipy/scipy

Issue:
ENH: Add wrapper for ?gejsv
See [NumFOCUS Small Development Grant Proposal 2019 Round 3 proposal](https://www.dropbox.com/s/i24h3j2osol56uf/Small%20Development%20Grant%20-%20LAPACK%20%20%28Condensed%29.pdf?dl=0) for background.

Add wrapper for [?gejsv](http://www.netlib.org/lapack/explore-html/d1/d7e/group__double_g_esing_ga8767bfcf983f8dc6ef2842029ab25599.html).

Suggested signature:
```
! sva, u, v, work, iwork, info  = gejsv(a, lwork=max(2*M+N, 4*N+N*N,2*N+N*N+6), joba="C", jobu="U", jobv="V", jobr="R", jobt="N", jobp="N", overwrite_A=0)
```

Test idea:
- Generate random `A`.
- Perform SVD with `?gejsv`
- Compare `A` against `u @ SIGMA @ v.T` where `SIGMA` is `np.diag(work[1]/work[0]*sva[:N])` (I think - see second thought below)
- Compare `u.T @ u` against identity, e.g. `np.testing.assert_allclose(u.T @ u, np.eye(u.shape[1]))`, with appropriate tolerances
- Compare `v @ v.T` against identity. 
- Check `info==0`.
- Check `iwork[0]==min(m,n)`
- Check `iwork[1]==min(m,n)`


Thoughts:

1. Computing the optimal LWORK looks quite complicated. In order to do it properly, it looks like we would need to create six other `lwork` routines - `DGEQP3`, `DGEQRF`, `DPOCON`, `DGELQF`, `DGEQRF`, `DORMLQ` - in order to write an `?dgejsv_lwork`. That is outside the scope of what I intended for this project, and I think that can be left to future work. For now, using the default value shown above is sufficient for any set of options.
2. The documentation about the scaling of the singular values in `SVA` is confusing. In the description of `SVA`: "The singular values of A are (WORK(1)/WORK(2)) * SVA(1:N)". In the description of `WORK`: "SCALE = WORK(2) / WORK(1) is the scaling factor such that SCALE*SVA(1:N) are the computed singular values of A." These seem contradictory.
3. We should test with M < N, M = N, and M > N.
4. We might test with non-full-row/column-rank matrices. In that case, the routine should succeed, but I think `iwork[0]` and `iwork[1]` will now be the rank of the matrix. One way to generate such a matrix is to compute the SVD of a random matrix, zero `k` of the singular values, and multiply the factors back together.
5. There are 6x4x4x2x2x2 = 768 combinations of `job?` options. I don't think we need to test every combination. **I suggest we just vary one thing at a time from the defaults.**
6. For `joba`, I don't think we need to verify that the different options are doing what they say they will; we should just check that the basic test above passes with every option.
7. For `jobu`, I think we can test options `U` and `F` with the basic test above. I'll describe tests with options `W` and `N` below.
8. For `jobv`, we can test option `V` with the basic test above. I don't want to dig into what option `J` means; maybe just test that the code runs without error. I'll describe tests with options `W` and `N` below.
9. For `jobr`, test with both options `N` and `R` normally.
10. For `jobt`, test with option `N`. I'll describe a test with option `T` below.
11. For `jobp`, test with options `P` and `N` normally.
12. I'd suggest putting an invalid argument for each `job?` option and confirming that the returned `info` indicates which argument is invalid.

Test with `jobu='N'` and `jobv ='N'`: since only the singular values are generated, we can't reproduce the original matrix from the factors or test `u` or `v` for orthogonality. Just check that the singular values are correct.

Test with `jobt='T'`, `jobu='W'`, `jobv='V'`, and `m==n`: do we need a special test for this? Originally I thought we might need to do something special, now I think we might be able to throw these options in with the regular tests above.

We could get fancy and design special tests with numerically non-full-(column/row) rank matrices to confirm that this routine is doing the things it claims, but I don't think that's necessary.

Maybe make sure there is reasonable behavior when:
- `A` is not 2D?
- `A` is 1 x 1
- `A` is empty
- `lwork` is zero

Also: implement all examples from the [NAG manual](https://www.nag.com/numeric/fl/nagdoc_latest/html/f08/f08khf.html) as additional tests.


Discussion:
@swallan

Resolving patch:
diff --git a/scipy/linalg/lapack.py b/scipy/linalg/lapack.py
--- a/scipy/linalg/lapack.py
+++ b/scipy/linalg/lapack.py
@@ -113,6 +113,9 @@
    cgehrd_lwork
    zgehrd_lwork
 
+   sgejsv
+   dgejsv
+
    sgels
    dgels
    cgels


Test patch:
diff --git a/scipy/linalg/tests/test_lapack.py b/scipy/linalg/tests/test_lapack.py
--- a/scipy/linalg/tests/test_lapack.py
+++ b/scipy/linalg/tests/test_lapack.py
@@ -1743,24 +1743,24 @@ def test_syequb():
         assert_equal(np.log2(s).astype(int), desired_log2s)
 
 
-def test_heequb():
+@pytest.mark.parametrize("dtype", COMPLEX_DTYPES)
+def test_heequb(dtype):
     desired_log2s = np.array([[-2, -7, -2, -4, -2, -3, -2, -2, -1, -2],
                               [1, -10, 0, -6, -1, -4, -1, -2, -1, -2]])
-    for ind, dtype in enumerate(COMPLEX_DTYPES):
-        heequb = get_lapack_funcs('heequb', dtype=dtype)
-
-        d = np.array([dtype(1j) * 2**x for x in range(-5, 5)], dtype=dtype)
-        A = np.diag(d)
-        subdiags = np.array([dtype(1j) * 2**(9-x) for x in range(9)],
-                            dtype=dtype)
-        A[range(1, 10), range(0, 9)] = subdiags
-        s, scond, amax, info = heequb(A, lower=1)
+    heequb = get_lapack_funcs('heequb', dtype=dtype)
 
-        # See gh-10741
-        pre3_7_lapack_result = np.log2(s).astype(int) == desired_log2s[0, :]
-        post3_7_lapack_result = np.log2(s).astype(int) == desired_log2s[1, :]
+    d = np.array([dtype(1j) * 2**x for x in range(-5, 5)], dtype=dtype)
+    A = np.diag(d)
+    subdiags = np.array([dtype(1j) * 2**(9-x) for x in range(9)],
+                        dtype=dtype)
+    A[range(1, 10), range(0, 9)] = subdiags
+    s, scond, amax, info = heequb(A, lower=1)
 
-        assert pre3_7_lapack_result.all() or post3_7_lapack_result.all()
+    assert_equal(info, 0)
+    try:
+        assert_allclose(np.log2(s), desired_log2s[0, :])
+    except AssertionError as e:
+        assert_allclose(np.log2(s), desired_log2s[1, :])
 
 
 def test_getc2_gesc2():
@@ -1794,6 +1794,190 @@ def test_getc2_gesc2():
                                       x/scale, decimal=4)
 
 
+@pytest.mark.parametrize('size', [(6, 5), (5, 5)])
+@pytest.mark.parametrize('dtype', REAL_DTYPES)
+@pytest.mark.parametrize('joba', range(6))  # 'C', 'E', 'F', 'G', 'A', 'R'
+@pytest.mark.parametrize('jobu', range(4))  # 'U', 'F', 'W', 'N'
+@pytest.mark.parametrize('jobv', range(4))  # 'V', 'J', 'W', 'N'
+@pytest.mark.parametrize('jobr', [0, 1])
+@pytest.mark.parametrize('jobt', [0, 1])
+@pytest.mark.parametrize('jobp', [0, 1])
+def test_gejsv_general(size, dtype, joba, jobu, jobv, jobr, jobt, jobp):
+    """Test the lapack routine ?gejsv.
+
+    This function tests that a singular value decomposition can be performed
+    on the random M-by-N matrix A. The test performs the SVD using ?gejsv
+    then performs the following checks:
+
+    * ?gejsv exist successfully (info == 0)
+    * The returned singular values are correct
+    * `A` can be reconstructed from `u`, `SIGMA`, `v`
+    * Ensure that u.T @ u is the identity matrix
+    * Ensure that v.T @ v is the identity matrix
+    * The reported matrix rank
+    * The reported number of singular values
+    * If denormalized floats are required
+
+    Notes
+    -----
+    joba specifies several choices effecting the calculation's accuracy
+    Although all arguments are tested, the tests only check that the correct
+    solution is returned - NOT that the prescribed actions are performed
+    internally.
+    """
+    seed(42)
+
+    # Define some constants for later use:
+    m, n = size
+    atol = 100 * np.finfo(dtype).eps
+    A = generate_random_dtype_array(size, dtype)
+    gejsv = get_lapack_funcs('gejsv', dtype=dtype)
+
+    # Set up checks for invalid job? combinations
+    # if an invalid combination occurs we set the appropriate
+    # exit status.
+    lsvec = jobu < 2  # Calculate left singular vectors
+    rsvec = jobv < 2  # Calculate right singular vectors
+    l2tran = (jobt == 1) and (m == n)
+    is_complex = np.iscomplexobj(A)
+
+    invalid_real_jobv = (jobv == 1) and (not lsvec) and (not is_complex)
+    invalid_cplx_jobu = (jobu == 2) and not (rsvec and l2tran) and is_complex
+    invalid_cplx_jobv = (jobv == 2) and not (lsvec and l2tran) and is_complex
+
+    # Set the exit status to the expected value.
+    # Here we only check for invalid combinations, not individual
+    # parameters.
+    if invalid_cplx_jobu:
+        exit_status = -2
+    elif invalid_real_jobv or invalid_cplx_jobv:
+        exit_status = -3
+    else:
+        exit_status = 0
+
+    if (jobu > 1) and (jobv == 1):
+        assert_raises(Exception, gejsv, A, joba, jobu, jobv, jobr, jobt, jobp)
+    else:
+        sva, u, v, work, iwork, info = gejsv(A,
+                                             joba=joba,
+                                             jobu=jobu,
+                                             jobv=jobv,
+                                             jobr=jobr,
+                                             jobt=jobt,
+                                             jobp=jobp)
+
+        # Check that ?gejsv exited successfully/as expected
+        assert_equal(info, exit_status)
+
+        # If exit_status is non-zero the combination of jobs is invalid.
+        # We test this above but no calculations are performed.
+        if not exit_status:
+
+            # Check the returned singular values
+            sigma = (work[0] / work[1]) * sva[:n]
+            assert_allclose(sigma, svd(A, compute_uv=False), atol=atol)
+
+            if jobu == 1:
+                # If JOBU = 'F', then u contains the M-by-M matrix of
+                # the left singular vectors, including an ONB of the orthogonal
+                # complement of the Range(A)
+                # However, to recalculate A we are concerned about the
+                # first n singular values and so can ignore the latter.
+                # TODO: Add a test for ONB?
+                u = u[:, :n]
+
+            if lsvec and rsvec:
+                assert_allclose(u @ np.diag(sigma) @ v.conj().T, A, atol=atol)
+            if lsvec:
+                assert_allclose(u.conj().T @ u, np.identity(n), atol=atol)
+            if rsvec:
+                assert_allclose(v.conj().T @ v, np.identity(n), atol=atol)
+
+            assert_equal(iwork[0], np.linalg.matrix_rank(A))
+            assert_equal(iwork[1], np.count_nonzero(sigma))
+            # iwork[2] is non-zero if requested accuracy is not warranted for
+            # the data. This should never occur for these tests.
+            assert_equal(iwork[2], 0)
+
+
+@pytest.mark.parametrize('dtype', REAL_DTYPES)
+def test_gejsv_edge_arguments(dtype):
+    """Test edge arguments return expected status"""
+    gejsv = get_lapack_funcs('gejsv', dtype=dtype)
+
+    # scalar A
+    sva, u, v, work, iwork, info = gejsv(1.)
+    assert_equal(info, 0)
+    assert_equal(u.shape, (1, 1))
+    assert_equal(v.shape, (1, 1))
+    assert_equal(sva, np.array([1.], dtype=dtype))
+
+    # 1d A
+    A = np.ones((1,), dtype=dtype)
+    sva, u, v, work, iwork, info = gejsv(A)
+    assert_equal(info, 0)
+    assert_equal(u.shape, (1, 1))
+    assert_equal(v.shape, (1, 1))
+    assert_equal(sva, np.array([1.], dtype=dtype))
+
+    # 2d empty A
+    A = np.ones((1, 0), dtype=dtype)
+    sva, u, v, work, iwork, info = gejsv(A)
+    assert_equal(info, 0)
+    assert_equal(u.shape, (1, 0))
+    assert_equal(v.shape, (1, 0))
+    assert_equal(sva, np.array([], dtype=dtype))
+
+
+@pytest.mark.parametrize(('kwargs'),
+                         ({'joba': 9},
+                          {'jobu': 9},
+                          {'jobv': 9},
+                          {'jobr': 9},
+                          {'jobt': 9},
+                          {'jobp': 9})
+                         )
+def test_gejsv_invalid_job_arguments(kwargs):
+    """Test invalid job arguments raise an Exception"""
+    A = np.ones((2, 2), dtype=float)
+    gejsv = get_lapack_funcs('gejsv', dtype=float)
+    assert_raises(Exception, gejsv, A, **kwargs)
+
+
+@pytest.mark.parametrize("A,sva_expect,u_expect,v_expect",
+                         [(np.array([[2.27, -1.54, 1.15, -1.94],
+                                     [0.28, -1.67, 0.94, -0.78],
+                                     [-0.48, -3.09, 0.99, -0.21],
+                                     [1.07, 1.22, 0.79, 0.63],
+                                     [-2.35, 2.93, -1.45, 2.30],
+                                     [0.62, -7.39, 1.03, -2.57]]),
+                           np.array([9.9966, 3.6831, 1.3569, 0.5000]),
+                           np.array([[0.2774, -0.6003, -0.1277, 0.1323],
+                                     [0.2020, -0.0301, 0.2805, 0.7034],
+                                     [0.2918, 0.3348, 0.6453, 0.1906],
+                                     [-0.0938, -0.3699, 0.6781, -0.5399],
+                                     [-0.4213, 0.5266, 0.0413, -0.0575],
+                                     [0.7816, 0.3353, -0.1645, -0.3957]]),
+                           np.array([[0.1921, -0.8030, 0.0041, -0.5642],
+                                     [-0.8794, -0.3926, -0.0752, 0.2587],
+                                     [0.2140, -0.2980, 0.7827, 0.5027],
+                                     [-0.3795, 0.3351, 0.6178, -0.6017]]))])
+def test_gejsv_NAG(A, sva_expect, u_expect, v_expect):
+    """
+    This test implements the example found in the NAG manual, f08khf.
+    An example was not found for the complex case.
+    """
+    # NAG manual provides accuracy up to 4 decimals
+    atol = 1e-4
+    gejsv = get_lapack_funcs('gejsv', dtype=A.dtype)
+
+    sva, u, v, work, iwork, info = gejsv(A)
+
+    assert_allclose(sva_expect, sva, atol=atol)
+    assert_allclose(u_expect, u, atol=atol)
+    assert_allclose(v_expect, v, atol=atol)
+
+
 @pytest.mark.parametrize("dtype", DTYPES)
 def test_gttrf_gttrs(dtype):
     # The test uses ?gttrf and ?gttrs to solve a random system for each dtype,