Repository: scipy/scipy

Issue:
PERF: scipy.linalg.pinv is very slow compared to numpy.linalg.pinv
The functions should do the same, but are implemented differently. NumPy uses SVD and SciPy uses a QR based solver from LAPACK. The effect is obvious for anyone who cares to look. 

Is there any reason we should not just alias the NumPy version instead?

```
C = np.random.rand(1000,20)
F = np.asfortranarray(C.copy())

%timeit np.linalg.pinv(C)
610 µs ± 17.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit np.linalg.pinv(F)
583 µs ± 2.88 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit scipy.linalg.pinv(C)
34.8 ms ± 1.34 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit scipy.linalg.pinv(F)
33.6 ms ± 329 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
```


Discussion:
Perhaps compare to scipy.linalg.pinv2, which uses SVD under the hood as well?
Yes, closing in on NumPy's performance. The difference might be due to f2py.

Is there any reason we have ```scipy.linalg.pinv``` when it does the same as ```scipy.linalg.pinv2```, only with 60x longer run-time? The functions do the same, except the former uses a less stabile algorithm that also happens to be slow as hell? It seems to be a dubious justification.

```
%timeit scipy.linalg.pinv2(C)
663 µs ± 23.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit scipy.linalg.pinv2(F)
645 µs ± 22.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
```
It's not less stable but unfortunately quite wasteful. If you flip the shape of the arrays they will have comparable performance results

```python
%timeit np.linalg.pinv(C)
5.1 ms ± 59 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

%timeit sp.linalg.pinv(C)
5.18 ms ± 19.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
```

The issue we have in the SciPy pinv is, not taking the shorter route and thus forming a huge linear problem when we don't need it since pseudoinverse commutes with transposition.

Resolving patch:
diff --git a/scipy/linalg/basic.py b/scipy/linalg/basic.py
--- a/scipy/linalg/basic.py
+++ b/scipy/linalg/basic.py
@@ -673,8 +673,8 @@ def solve_toeplitz(c_or_cr, b, check_finite=True):
         c = _asarray_validated(c_or_cr, check_finite=check_finite).ravel()
         r = c.conjugate()
 
-    # Form a 1-D array of values to be used in the matrix, containing a reversed
-    # copy of r[1:], followed by c.
+    # Form a 1-D array of values to be used in the matrix, containing a
+    # reversed copy of r[1:], followed by c.
     vals = np.concatenate((r[-1:0:-1], c))
     if b is None:
         raise ValueError('illegal value, `b` is a required argument')
@@ -1302,7 +1302,9 @@ def pinv(a, cond=None, rcond=None, return_rank=False, check_finite=True):
 
     """
     a = _asarray_validated(a, check_finite=check_finite)
-    b = np.identity(a.shape[0], dtype=a.dtype)
+    # If a is sufficiently tall it is cheaper to compute using the transpose
+    trans = a.shape[0] / a.shape[1] >= 1.1
+    b = np.eye(a.shape[1] if trans else a.shape[0], dtype=a.dtype)
 
     if rcond is not None:
         cond = rcond
@@ -1310,12 +1312,13 @@ def pinv(a, cond=None, rcond=None, return_rank=False, check_finite=True):
     if cond is None:
         cond = max(a.shape) * np.spacing(a.real.dtype.type(1))
 
-    x, resids, rank, s = lstsq(a, b, cond=cond, check_finite=False)
+    x, resids, rank, s = lstsq(a.T if trans else a, b,
+                               cond=cond, check_finite=False)
 
     if return_rank:
-        return x, rank
+        return (x.T if trans else x), rank
     else:
-        return x
+        return x.T if trans else x
 
 
 def pinv2(a, cond=None, rcond=None, return_rank=False, check_finite=True):


Test patch:
diff --git a/scipy/linalg/tests/test_basic.py b/scipy/linalg/tests/test_basic.py
--- a/scipy/linalg/tests/test_basic.py
+++ b/scipy/linalg/tests/test_basic.py
@@ -1260,6 +1260,13 @@ def test_native_list_argument(self):
         a_pinv2 = pinv2(a)
         assert_array_almost_equal(a_pinv, a_pinv2)
 
+    def test_tall_transposed(self):
+        a = random([10, 2])
+        a_pinv = pinv(a)
+        # The result will be transposed internally hence will be a C-layout
+        # instead of the typical LAPACK output with Fortran-layout
+        assert a_pinv.flags['C_CONTIGUOUS']
+
 
 class TestPinvSymmetric(object):