Repository: scipy/scipy

Issue:
pinv and pinv2 behave differently
I am working with bidimensional arrays on Numpy. One of my arrays, H, is random, and I want to compute its pseudoinverse. If I use scipy.linalg.pinv2 everything runs smoothly. However, if I use scipy.linalg.pinv, sometimes problems arise.

The problem is that, if the j-th column of H is all 1s, pinv(H) has huge coefficients at the j-th row. This is in turn a problem because, in such cases, np.dot(pinv(H), Y) contains some nan values (Y is an array of small integers).

Now, I am not into linear algebra and numeric computation enough to understand if this is a bug or some precision related property of the two functions. If this is not a bug, please close this issue and I apologize.

I saved the arrays with np.savetxt(fn, a, '%.2e', ';'): please, see https://dl.dropboxusercontent.com/u/48242012/example.tar.gz to find them.

In the provided file, you can see in pinv(H).csv that rows 14, 33, 55, 56 and 99 have huge values, while in pinv2(H) the same rows have more decent values.

(I posted the same question on StackOverflow yesterday, but it did not receive much attention: http://stackoverflow.com/questions/27775986/numpy-scipy-pinv-and-pinv2-behave-differently . I am reposting here just to signal this behaviour and to suggest that it may be documented - I  can survive with pinv2.)

I appreciate your efforts.



Discussion:
Hi, what version of scipy are you using?  With the file you provided (which gives only barely similar array since it has been clipped to 2 decimal points) I find results that match to ~1e-8 between `pinv` and `pinv2` using scipy 0.14.  

Hi,

sorry I did not provide this information: I am using Python 3.4.1, Numpy
1.9.1 and Scipy 0.14.0.

Apologies for the files as well: I generated some new (different) files
with %f and I dumped them so there is no loss of precision;
you can load them with

import pickle
with open(path, 'rb') as in_f:
     return pickle.load(in_f)

Now the problematic rows in pH.csv are 2, 4 and 35.
(Names are different in this archive: pH is pinv(H), pH2 pinv2(H), beta is np.dot(pinv(H), Y) and beta2 is np.dot(pinv2(H), Y) )

I uploaded them in
https://dl.dropboxusercontent.com/u/48242012/example2.tar.gz .

On 8 January 2015 at 09:35, Eric Moore notifications@github.com wrote:

> Hi, what version of scipy are you using? With the file you provided (which
> gives only barely similar array since it has been clipped to 2 decimal
> points) I find results that match to ~1e-8 between pinv and pinv2 using
> scipy 0.14.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/scipy/scipy/issues/4372#issuecomment-69105056.

## 

Marco

Sorry for the delay in getting back to this.  I think the difference you are seeing is that these two functions differ in their `rcond` parameter.  `pinv` uses machine precision (i.e. ~1e-16 for doubles) and `pinv2` use 1e6 times machine precision.  If you try your calculation again and specify a value for `rcond` you'll find nearly equivalent result from the two functions.

It would be nice if there was just one of these functions, but that ship may have sailed.  We could at least add these defaults to the documentation.

Since there seem to be three methods scipy.linalg.pinv, scipy.linalg.pinv2, numpy.linalg.pinv
the documentation should also state if some are duplicates.
Or if there are cases where one of them is more suitable.

http://stackoverflow.com/a/13265930


Resolving patch:
diff --git a/scipy/linalg/basic.py b/scipy/linalg/basic.py
--- a/scipy/linalg/basic.py
+++ b/scipy/linalg/basic.py
@@ -1259,6 +1259,8 @@ def pinv(a, cond=None, rcond=None, return_rank=False, check_finite=True):
         Cutoff for 'small' singular values in the least-squares solver.
         Singular values smaller than ``rcond * largest_singular_value``
         are considered zero.
+        If None, it is set to ``np.finfo(a.dtype).eps``.
+        If `a` is an array of integers, it is set to ``np.finfo('float64').eps``.
     return_rank : bool, optional
         if True, return the effective rank of the matrix
     check_finite : bool, optional
@@ -1318,9 +1320,11 @@ def pinv2(a, cond=None, rcond=None, return_rank=False, check_finite=True):
         Cutoff for 'small' singular values.
         Singular values smaller than ``rcond*largest_singular_value``
         are considered zero.
-        If None or -1, suitable machine precision is used.
+        If None and the dtype of `a` is ``np.float32``, it is set to
+        ``np.finfo('float32').eps * 1e3``.
+        Otherwise, it is set to ``np.finfo('float64').eps * 1e6``.
     return_rank : bool, optional
-        if True, return the effective rank of the matrix
+        If True, return the effective rank of the matrix.
     check_finite : bool, optional
         Whether to check that the input matrix contains only finite numbers.
         Disabling may give a performance gain, but may result in problems
@@ -1331,7 +1335,7 @@ def pinv2(a, cond=None, rcond=None, return_rank=False, check_finite=True):
     B : (N, M) ndarray
         The pseudo-inverse of matrix `a`.
     rank : int
-        The effective rank of the matrix.  Returned if return_rank == True
+        The effective rank of the matrix.  Returned if `return_rank` is True.
 
     Raises
     ------
@@ -1385,16 +1389,17 @@ def pinvh(a, cond=None, rcond=None, lower=True, return_rank=False,
     a : (N, N) array_like
         Real symmetric or complex hermetian matrix to be pseudo-inverted
     cond, rcond : float or None
-        Cutoff for 'small' eigenvalues.
-        Singular values smaller than rcond * largest_eigenvalue are considered
-        zero.
-
-        If None or -1, suitable machine precision is used.
+        Cutoff for 'small' singular values.
+        Singular values smaller than ``rcond*largest_singular_value``
+        are considered zero.
+        If None and the dtype of `a` is ``np.float32``, it is set to
+        ``np.finfo('float32').eps * 1e3``.
+        Otherwise, it is set to ``np.finfo('float64').eps * 1e6``.
     lower : bool, optional
         Whether the pertinent array data is taken from the lower or upper
-        triangle of a. (Default: lower)
+        triangle of `a`. (Default: lower)
     return_rank : bool, optional
-        if True, return the effective rank of the matrix
+        If True, return the effective rank of the matrix.
     check_finite : bool, optional
         Whether to check that the input matrix contains only finite numbers.
         Disabling may give a performance gain, but may result in problems
@@ -1405,7 +1410,7 @@ def pinvh(a, cond=None, rcond=None, lower=True, return_rank=False,
     B : (N, N) ndarray
         The pseudo-inverse of matrix `a`.
     rank : int
-        The effective rank of the matrix.  Returned if return_rank == True
+        The effective rank of the matrix.  Returned if `return_rank` is True.
 
     Raises
     ------


Test patch: