Repository: scipy/scipy

Issue:
`scipy.linalg.interpolative.interp_decomp` modifies input matrix 
`scipy.linalg.interpolative.interp_decomp` modifies the input matrix when `randomized=False` and input matrix is transposed.

### Reproducing code example:
```
import numpy as np
from scipy.linalg.interpolative import interp_decomp

A = np.array([[-1, -1, -1, 0, 0, 0],
              [0, 0, 0, 1, 1, 1],
              [1, 0, 0, 1, 0, 0],
              [0, 1, 0, 0, 1, 0],
              [0, 0, 1, 0, 0, 1]], dtype=float)

idx, proj = interp_decomp(A.T, rank, rand=False)

print(A)
```
prints
```
[[-1.00000000e+00  1.00000000e+00 -1.00000000e+00 -1.00000000e+00
  -0.00000000e+00 -0.00000000e+00]
 [ 0.00000000e+00  1.73205081e+00 -0.00000000e+00 -5.77350269e-01
  -5.77350269e-01 -5.77350269e-01]
 [-5.77350269e-01  5.77350269e-01  1.15470054e+00 -1.92450090e-01
   3.84900179e-01  3.84900179e-01]
 [-5.77350269e-01  5.77350269e-01 -5.77350269e-01  1.00000000e+00
   8.22780956e-01 -1.58285666e-02]
 [-1.00000000e+00  1.00000000e+00 -1.00000000e+00 -1.00000000e+00
   1.11022302e-16  9.71445147e-17]]
```
which is not the original matrix.

This doesn't occur when `randomized=True` or when the input matrix is `A` instead of `A.T`.

A workaround, of course, is to copy the matrix before passing it in.

### Scipy/Numpy/Python version information:
```
1.3.0.dev0+1b00745 1.15.0 sys.version_info(major=3, minor=6, micro=7, releaselevel='final', serial=0)
```


Discussion:
TLDR: `A` is changed in-place in the FORTRAN subroutine [iddr_id](https://github.com/scipy/scipy/blob/12fe2866a9d489d2e4eabd279d1abb930a0c96b6/scipy/linalg/src/id_dist/src/idd_id.f#L142-L260).  

Looking at the output from the pdb, `A` changes in ``_interpolative_backend.py``:

    > /mnt/c/dev/scipy/scipy/linalg/_interpolative_backend.py(232)iddr_id()
    -> idx, rnorms = _id.iddr_id(A, k)
    (Pdb) display A
    display A: array([[-1.,  0.,  1.,  0.,  0.],
           [-1.,  0.,  0.,  1.,  0.],
           [-1.,  0.,  0.,  0.,  1.],
           [ 0.,  1.,  1.,  0.,  0.],
           [ 0.,  1.,  0.,  1.,  0.],
           [ 0.,  1.,  0.,  0.,  1.]])
    (Pdb) s
    > /mnt/c/dev/scipy/scipy/linalg/_interpolative_backend.py(233)iddr_id()
    -> n = A.shape[1]
    (Pdb) display A
    display A: array([[ 0.        ,  0.        , -0.33333333, -0.33333333, -0.33333333],
           [-0.33333333,  0.        , -0.57735027,  0.78867513, -0.21132487],
           [-0.33333333,  0.        , -0.57735027, -0.21132487,  0.78867513],
           [-0.33333333,  1.        ,  1.        ,  0.        ,  0.        ],
           [-0.        ,  1.        ,  0.        ,  1.        ,  0.        ],
           [-0.        ,  1.        ,  0.        ,  0.        ,  1.        ]])

Which is pretty much a wrapper function of the FORTRAN function (docstring omitted):

https://github.com/scipy/scipy/blob/12fe2866a9d489d2e4eabd279d1abb930a0c96b6/scipy/linalg/_interpolative_backend.py#L231-L235

Line 232 calls the FORTRAN subroutine which stores the value of `proj` in `A`. Then reads the value of `proj` into it's own variable (line 234).

I think the simplest way to *not* modify `A` would be to change line 231 to ``B = np.asfortranarray(A)`` and use `B` throughout the remaining calculation. This may (?) require an extra copy, I'm not sure if this induce a significant performance cost.

@mdhaber The FORTRAN code is well explained but I don't know FORTRAN well enough to apply any changes (I don't really think we should anyway). Could you suggest someone to take a look at that? 
Would that fix the problem? The issue seems to be that `iddr_id` is changing the array in-place. If you change the name of the variable to `B`, it will still refer to the array in memory that we're trying to preserve. 

Remember that the issue only occurs when we're working on the transpose of a matrix. `np.asfortranarray` is probably making a copy when the array is in row-major order, and that's why we don't see the issue when we're working on a regular matrix (not a transpose). But if it's already in Fortran's column-major order (as with a transpose), it doesn't create a copy. [This](https://en.m.wikipedia.org/wiki/Row-_and_column-major_order#Transposition) might help explain.

I think the fix should go in the Python code where you say, but I think you just need to create a copy (when `np.asfortranarray` isn't already doing it). That is, use `ndarray.flags` to check whether the area is already `F_CONTIGUOUS`. If so, explicitly make a copy, and refer to that copy with the same name, `A`.
This check is often done via `_datacopied`. [Here are many checks ](https://github.com/scipy/scipy/blob/master/scipy/linalg/basic.py#L139) that linalg.solve does. You can start from there and make it more sophisticated if needed. 

The main machinery is 

https://github.com/scipy/scipy/blob/master/scipy/linalg/misc.py#L177-L187

Ok, naively, I thought `np.asfortranarray` always makes a copy. I'll add a test case and see if that fixes it.
I've added some extra checks s.t. ``A = np.asfortranarray(A)`` becomes:

    if A.flags['F_CONTIGUOUS']:
        A = A.copy()
    else:
        A = np.asfortranarray(A)

and no longer changes the user's `A`.

@ilayn @mdhaber Other fortran functions also change the underlying  data  (e.g. ``iddr_svd``, ``iddp_svd``,  ``idd_permuter``). And I suspect a fortran array is not being copied before any of them.
@kai-striega that's unfortunate, but I think they'll need to be patched individually. We can't have `asfortranarray` making unnecessary copies. Even though some people have assumed it makes copies, I imagine that others have used it correctly. We shouldn't slow down the correct code to fix the incorrect code.
Maybe it would help to add a note about this in the `asfortranarray`  documentation. Even an optional argument `copy=False` that, when true, forces a copy even if the array is already Fortran-style.
@Kai-Striega BTW this might be a good first PR for someone. It's not bothering me right now, so could we leave the issue open, at least through the summer? If we are able to update the documentation (#9951) this would be a good PR for someone to learn on.
Yes, good idea. I've also been drafting an issue for numpy to expose the
'copy' argument. Even if it's not used here, it will be useful to have in
general.

On Thu., 28 Mar. 2019, 11:21 Matt Haberland, <notifications@github.com>
wrote:

> @Kai-Striega <https://github.com/Kai-Striega> BTW this might be a good
> first PR for someone. It's not bothering me right now, so could we leave
> it? If we are able to update the documentation (#9951
> <https://github.com/scipy/scipy/issues/9951>) this would be a good PR for
> someone to learn on.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/scipy/scipy/issues/9793#issuecomment-477431791>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AcrktcwBtHBqO8GJVrijXtDN6ZUdqt1Oks5vbDVFgaJpZM4awUTZ>
> .
>

@mdhaber rather than update the documentation we can fix this by changing `np.asfortranarray(...) ` to `np.array(a, dtype, copy=True, order='F', ndmin=1)`. But if these are changed it should happen for *all* related fortran functions where the values need to be copied.

Resolving patch:
diff --git a/scipy/linalg/_interpolative_backend.py b/scipy/linalg/_interpolative_backend.py
--- a/scipy/linalg/_interpolative_backend.py
+++ b/scipy/linalg/_interpolative_backend.py
@@ -37,6 +37,18 @@
 _RETCODE_ERROR = RuntimeError("nonzero return code")
 
 
+def _asfortranarray_copy(A):
+    """
+    Same as np.asfortranarray, but ensure a copy
+    """
+    A = np.asarray(A)
+    if A.flags.f_contiguous:
+        A = A.copy(order="F")
+    else:
+        A = np.asfortranarray(A)
+    return A
+
+
 #------------------------------------------------------------------------------
 # id_rand.f
 #------------------------------------------------------------------------------
@@ -203,7 +215,7 @@ def iddp_id(eps, A):
         Interpolation coefficients.
     :rtype: :class:`numpy.ndarray`
     """
-    A = np.asfortranarray(A)
+    A = _asfortranarray_copy(A)
     k, idx, rnorms = _id.iddp_id(eps, A)
     n = A.shape[1]
     proj = A.T.ravel()[:k*(n-k)].reshape((k, n-k), order='F')
@@ -228,7 +240,7 @@ def iddr_id(A, k):
         Interpolation coefficients.
     :rtype: :class:`numpy.ndarray`
     """
-    A = np.asfortranarray(A)
+    A = _asfortranarray_copy(A)
     idx, rnorms = _id.iddr_id(A, k)
     n = A.shape[1]
     proj = A.T.ravel()[:k*(n-k)].reshape((k, n-k), order='F')
@@ -997,7 +1009,7 @@ def idzp_id(eps, A):
         Interpolation coefficients.
     :rtype: :class:`numpy.ndarray`
     """
-    A = np.asfortranarray(A)
+    A = _asfortranarray_copy(A)
     k, idx, rnorms = _id.idzp_id(eps, A)
     n = A.shape[1]
     proj = A.T.ravel()[:k*(n-k)].reshape((k, n-k), order='F')
@@ -1022,7 +1034,7 @@ def idzr_id(A, k):
         Interpolation coefficients.
     :rtype: :class:`numpy.ndarray`
     """
-    A = np.asfortranarray(A)
+    A = _asfortranarray_copy(A)
     idx, rnorms = _id.idzr_id(A, k)
     n = A.shape[1]
     proj = A.T.ravel()[:k*(n-k)].reshape((k, n-k), order='F')


Test patch:
diff --git a/scipy/linalg/tests/test_interpolative.py b/scipy/linalg/tests/test_interpolative.py
--- a/scipy/linalg/tests/test_interpolative.py
+++ b/scipy/linalg/tests/test_interpolative.py
@@ -30,7 +30,9 @@
 import numpy as np
 from scipy.linalg import hilbert, svdvals, norm
 from scipy.sparse.linalg import aslinearoperator
+from scipy.linalg.interpolative import interp_decomp
 import time
+import itertools
 
 from numpy.testing import assert_, assert_allclose
 from pytest import raises as assert_raises
@@ -276,3 +278,18 @@ def test_full_rank(self):
         B = pymatrixid.reconstruct_skel_matrix(A, k, idx)
         assert_allclose(A, B.dot(P))
 
+    def test_bug_9793(self):
+        dtypes = [np.float_, np.complex_]
+        rands = [True, False]
+        epss = [1, 0.1]
+
+        for dtype, eps, rand in itertools.product(dtypes, epss, rands):
+            A = np.array([[-1, -1, -1, 0, 0, 0],
+                          [0, 0, 0, 1, 1, 1],
+                          [1, 0, 0, 1, 0, 0],
+                          [0, 1, 0, 0, 1, 0],
+                          [0, 0, 1, 0, 0, 1]],
+                         dtype=dtype, order="C")
+            B = A.copy()
+            interp_decomp(A.T, eps, rand=rand)
+            assert_(np.array_equal(A, B))