Repository: scipy/scipy

Issue:
Add wrapper for ?geqrfp
See [NumFOCUS Small Development Grant Proposal 2019 Round 3 proposal](https://www.dropbox.com/s/i24h3j2osol56uf/Small%20Development%20Grant%20-%20LAPACK%20%20%28Condensed%29.pdf?dl=0) for background.

Add wrapper for [?geqrfp](http://www.netlib.org/lapack/explore-html/dd/d9a/group__double_g_ecomputational_ga5dc72a0f55ff4c08f1a3596f56e04aaa.html).

Existing [wrapper](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.lapack.dgeqrf.html) for [?geqrf](http://www.netlib.org/lapack/explore-html/df/dc5/group__variants_g_ecomputational_ga3766ea903391b5cf9008132f7440ec7b.html) provides some guidance.

Suggested signature:
```
! rq_a,tau,work,info = geqrfp(a,lwork=3*n,overwrite_a=0)
```
We will also want to create a routine to compute the optimal `lwork` value:
```
! work,info = geqrfp_lwork(n)
```

I'm a bit confused about how `overwrite_a` works - is that optional argument added automatically by F2PY? It is an optional argument for `?geqrf` but there's no "code" for it in the wrapper.
_Update: See @ilayn's response below._

If the user provides `lwork=-1`, it seems that `?geqrfp` is supposed to determine the optimal `lwork` rather than actually solving the problem. I suppose the idea is that if you don't know what to choose for `lwork`, you call  `?geqrfp` once with `lwork=-1` just to get the optimal value of `lwork`, then call it again with that value of `lwork` to actually solve the problem. Is that right? Should the wrapper go through that process if no `lwork` is provided by the user? The wrapper for `?geqrf` doesn't use this approach, it seems; it just defaults to 3*n. Where does 3*n come from? In the [NAG manual](https://www.nag.com/numeric/fl/nagdoc_latest/html/f08/f08aef.html), it looks like 64*n is used.
_Update: See @ilayn's response below._

As for tests, the [NAG manual](https://www.nag.com/numeric/fl/nagdoc_latest/html/f08/f08aef.html) provides an example for `?geqrf`, so these could be adapted for `?geqrfp`. Alternatively, we could follow the approach used by [`scipy.linalg.qr`](https://github.com/scipy/scipy/blob/master/scipy/linalg/decomp_qr.py) to convert the output into matrices `Q` and `R` and write tests based on those, e.g.

- Create random matrix `A`
- Decompose using wrapper
- Convert to `Q` and `R` using approach in `scipy.linalg.qr`
- Verify that `R` is upper triangular and has all non-negative diagonal 
- Verify that `Q` is orthogonal
- Verify that `A` = `Q @ R`

etc.
Probably should do this for an example where regular QR gives negative entries on diagonal of `R`. We should also check for empty matrix and both tall and wide matrices as input.


Discussion:
> I'm a bit confused about how overwrite_a works - is that optional argument added automatically by F2PY? It is an optional argument for ?geqrf but there's no "code" for it in the wrapper.

When variable declaration has the option `copy` in the `intent` it is generated automatically by f2py

> If the user provides lwork=-1, it seems that ?geqrfp is supposed to determine the optimal lwork rather than actually solving the problem. I suppose the idea is that if you don't know what to choose for lwork, you call ?geqrfp once with lwork=-1 just to get the optimal value of lwork, then call it again with that value of lwork to actually solve the problem. Is that right? Should the wrapper go through that process if no lwork is provided by the user?

Calling the same routine twice (one with lwork=-1) is a decades old trick. The main reason is that a variable, `NB`, changes from hardware to hardware for optimal size computation. 

The user could call with `-1`, but what we do is to wrap another routine `<lapack_name>_lwork(...)` (see examples in the wrapper files) so that you call that and receive the lwork number. Then you call the actual routine with that lwork value. 

> The wrapper for ?geqrf doesn't use this approach, it seems; it just defaults to 3n. Where does 3n come from? In the NAG manual, it looks like 64*n is used.

It is probably the absolute minimum value allowed. 


Thanks @ilayn that's really helpful. I've noticed that for the other `*_lwork` routines most of the arguments are hidden, but some of the arguments are still (optionally) allowed. Can these affect the optimal `lwork` value? How do the authors of the wrappers know which to hide and which to keep as optional arguments?

Also, it looks like there is no `?geqrf_lwork` even though `?geqrf` is already wrapped. Should we create a `?geqrf_lwork`  routine while we're at it?
That would be indeed nice. Often you can look at which arguments are passed to the lwork computation in the routine (ilaenv) and make them visible. For modern personal computers most of the times it shouldn't make any difference but better safe than sorry. 
Finally, I'm curious - I played with `scipy.linalg.dgeqrf` with argument `lwork=-1` and it gave me a value of `lwork` in `work[0]`. Then I called the routine with that value of `lwork`, and the routine exited (successfully) with a different value in `work[0]`. Supposedly, `On exit, if INFO = 0, WORK(1) returns the optimal LWORK.`, so why did I get different values depending on whether I'm doing a workspace query or not?
`WORK` array is the name LAPACK uses for "I use this array for scratch and temp values".  In complex ones you also get `iwork`, `rwork` etc. And when you do `LWORK=-1` it also uses the first entry of that array that to report the optimal value. That array size is optimal if you only give it big enough such that it uses Level 3 blocked algorithms comfortably and and small enough that you don't waste memory space. 

That is the reason we wrap WORK as a scalar when we wrap `<LAPACK_routine>_lwork` so that it doesn't allocate 1000 array for just a scalar reporting. 

TL;DR the value you see is often the intermediate value used for computing something except when `LWORK=-1`. You shouldn't see it because it should be hidden from user as it has no prescribed information unless explicitly stated by LAPACK devs.
I suppose I would have expected the documentation to say something like:
 `On exit, if INFO = 0 and LWORK = -1, WORK(1) returns the optimal LWORK.`
But OK! Thanks for the tip about making `work` as a scalar in the `*_lwork` wrapper.

Resolving patch:
diff --git a/scipy/linalg/lapack.py b/scipy/linalg/lapack.py
--- a/scipy/linalg/lapack.py
+++ b/scipy/linalg/lapack.py
@@ -169,6 +169,16 @@
    cgeqrf_lwork
    zgeqrf_lwork
 
+   sgeqrfp
+   dgeqrfp
+   cgeqrfp
+   zgeqrfp
+
+   sgeqrfp_lwork
+   dgeqrfp_lwork
+   cgeqrfp_lwork
+   zgeqrfp_lwork
+
    sgerqf
    dgerqf
    cgerqf


Test patch:
diff --git a/scipy/linalg/tests/test_lapack.py b/scipy/linalg/tests/test_lapack.py
old mode 100644
new mode 100755
--- a/scipy/linalg/tests/test_lapack.py
+++ b/scipy/linalg/tests/test_lapack.py
@@ -24,6 +24,7 @@
 from scipy.linalg import _flapack as flapack, lapack
 from scipy.linalg import inv, svd, cholesky, solve, ldl, norm
 from scipy.linalg.lapack import _compute_lwork
+from scipy.linalg import qr
 
 try:
     from scipy.linalg import _clapack as clapack
@@ -1656,6 +1657,96 @@ def test_getc2_gesc2():
                                       x/scale, decimal=4)
 
 
+@pytest.mark.parametrize('dtype', DTYPES)
+@pytest.mark.parametrize('shape', [(3, 7), (7, 3), (2**18, 2**18)])
+def test_geqrfp_lwork(dtype, shape):
+    geqrfp_lwork = get_lapack_funcs(('geqrfp_lwork'), dtype=dtype)
+    m, n = shape
+    lwork, info = geqrfp_lwork(m=m, n=n)
+    assert_equal(info, 0)
+
+
+def generate_random_dtype_array(shape, dtype):
+    # generates a random matrix of desired data type of shape
+    if dtype in COMPLEX_DTYPES:
+        return (np.random.rand(*shape)
+                + np.random.rand(*shape)*1.0j).astype(dtype)
+    return np.random.rand(*shape).astype(dtype)
+
+
+@pytest.mark.parametrize('dtype', DTYPES)
+@pytest.mark.parametrize('matrix_size', [(3, 4), (7, 6), (6, 6)])
+def test_geqrfp(dtype, matrix_size):
+    # Tests for all dytpes, tall, wide, and square matrices.
+    # Using the routine with random matrix A, Q and R are obtained and then
+    # tested such that R is upper triangular and non-negative on the diagonal,
+    # and Q is an orthagonal matrix. Verifies that A=Q@R. It also
+    # tests against a matrix that for which the  linalg.qr method returns
+    # negative diagonals, and for error messaging.
+
+    # set test tolerance appropriate for dtype
+    np.random.seed(42)
+    rtol = 250*np.finfo(dtype).eps
+    atol = 100*np.finfo(dtype).eps
+    # get appropriate ?geqrfp for dtype
+    geqrfp = get_lapack_funcs(('geqrfp'), dtype=dtype)
+    gqr = get_lapack_funcs(("orgqr"), dtype=dtype)
+
+    m, n = matrix_size
+
+    # create random matrix of dimentions m x n
+    A = generate_random_dtype_array((m, n), dtype=dtype)
+    # create qr matrix using geqrfp
+    qr_A, tau, info = geqrfp(A)
+
+    # obtain r from the upper triangular area
+    r = np.triu(qr_A)
+
+    # obtain q from the orgqr lapack routine
+    # based on linalg.qr's extraction strategy of q with orgqr
+
+    if m > n:
+        # this adds an extra column to the end of qr_A
+        # let qqr be an empty m x m matrix
+        qqr = np.zeros((m, m), dtype=dtype)
+        # set first n columns of qqr to qr_A
+        qqr[:, :n] = qr_A
+        # determine q from this qqr
+        # note that m is a sufficient for lwork based on LAPACK documentation
+        q = gqr(qqr, tau=tau, lwork=m)[0]
+    else:
+        q = gqr(qr_A[:, :m], tau=tau, lwork=m)[0]
+
+    # test that q and r still make A
+    assert_allclose(q@r, A, rtol=rtol)
+    # ensure that q is orthogonal (that q @ transposed q is the identity)
+    assert_allclose(np.eye(q.shape[0]), q@(q.conj().T), rtol=rtol,
+                    atol=atol)
+    # ensure r is upper tri by comparing original r to r as upper triangular
+    assert_allclose(r, np.triu(r), rtol=rtol)
+    # make sure diagonals of r are positive for this random solution
+    assert_(np.all(np.diag(r) > np.zeros(len(np.diag(r)))))
+    # ensure that info is zero for this success
+    assert_(info == 0)
+
+    # test that this routine gives r diagonals that are positive for a
+    # matrix that returns negatives in the diagonal with scipy.linalg.rq
+    A_negative = generate_random_dtype_array((n, m), dtype=dtype) * -1
+    r_rq_neg, q_rq_neg = qr(A_negative)
+    rq_A_neg, tau_neg, info_neg = geqrfp(A_negative)
+    # assert that any of the entries on the diagonal from linalg.qr
+    #   are negative and that all of geqrfp are positive.
+    assert_(np.any(np.diag(r_rq_neg) < 0) and
+            np.all(np.diag(r) > 0))
+
+
+def test_geqrfp_errors_with_empty_array():
+    # check that empty array raises good error message
+    A_empty = np.array([])
+    geqrfp = get_lapack_funcs('geqrfp', dtype=A_empty.dtype)
+    assert_raises(Exception, geqrfp, A_empty)
+
+
 @pytest.mark.parametrize("driver", ['ev', 'evd', 'evr', 'evx'])
 @pytest.mark.parametrize("pfx", ['sy', 'he'])
 def test_standard_eigh_lworks(pfx, driver):