Repository: numpy/numpy

Issue:
np.linalg.svd with hermitian=True does not properly sort singular values
### Reproducing code example:

<!-- A short code example that reproduces the problem/missing feature. It should be
self-contained, i.e., possible to run as-is via 'python myproblem.py' -->

`svd` with `hermitian=True` (introduced by @eric-wieser in https://github.com/numpy/numpy/pull/12693) has a bug with the order of singular values:
```python
import numpy as np

a = np.array([[0, 1, 0], [1, 0, 1], [0, 1, 0]])
u, s, v = np.linalg.svd(a, hermitian=True)
print(s)
```
Outputs: `[1.41421356e+00 6.07153217e-18 1.41421356e+00]`
<!-- Remove these sections for a feature request -->

Per the contract of `svd`, the singular values should be sorted in descending order, but they aren't here. That's because we simply reverse the order of eigenvalues than sorting, which only suffices if all eigenvalues happen to be positive.

### Numpy/Python version information:

<!-- Output from 'import sys, numpy; print(numpy.__version__, sys.version)' -->
1.17.5 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]



Discussion:
I assume you would like to see this fixed in a 1.17.6?
Would make sense probably. I just +labeled it with high, because it seems to me like this can cause silent result corruption with a harmless looking change.
Darn, that was silly of me, ordering `x` doesn't order `x**2`. I suppose we need an `argsort` in there

Resolving patch:
diff --git a/numpy/linalg/linalg.py b/numpy/linalg/linalg.py
--- a/numpy/linalg/linalg.py
+++ b/numpy/linalg/linalg.py
@@ -24,7 +24,7 @@
     add, multiply, sqrt, fastCopyAndTranspose, sum, isfinite,
     finfo, errstate, geterrobj, moveaxis, amin, amax, product, abs,
     atleast_2d, intp, asanyarray, object_, matmul,
-    swapaxes, divide, count_nonzero, isnan, sign
+    swapaxes, divide, count_nonzero, isnan, sign, argsort, sort
 )
 from numpy.core.multiarray import normalize_axis_index
 from numpy.core.overrides import set_module
@@ -1608,24 +1608,29 @@ def svd(a, full_matrices=True, compute_uv=True, hermitian=False):
     True
 
     """
+    import numpy as _nx
     a, wrap = _makearray(a)
 
     if hermitian:
-        # note: lapack returns eigenvalues in reverse order to our contract.
-        # reversing is cheap by design in numpy, so we do so to be consistent
+        # note: lapack svd returns eigenvalues with s ** 2 sorted descending,
+        # but eig returns s sorted ascending, so we re-order the eigenvalues
+        # and related arrays to have the correct order
         if compute_uv:
             s, u = eigh(a)
-            s = s[..., ::-1]
-            u = u[..., ::-1]
-            # singular values are unsigned, move the sign into v
-            vt = transpose(u * sign(s)[..., None, :]).conjugate()
+            sgn = sign(s)
             s = abs(s)
+            sidx = argsort(s)[..., ::-1]
+            sgn = _nx.take_along_axis(sgn, sidx, axis=-1)
+            s = _nx.take_along_axis(s, sidx, axis=-1)
+            u = _nx.take_along_axis(u, sidx[..., None, :], axis=-1)
+            # singular values are unsigned, move the sign into v
+            vt = transpose(u * sgn[..., None, :]).conjugate()
             return wrap(u), s, wrap(vt)
         else:
             s = eigvalsh(a)
             s = s[..., ::-1]
             s = abs(s)
-            return s
+            return sort(s)[..., ::-1]
 
     _assert_stacked_2d(a)
     t, result_t = _commonType(a)


Test patch:
diff --git a/numpy/linalg/tests/test_linalg.py b/numpy/linalg/tests/test_linalg.py
--- a/numpy/linalg/tests/test_linalg.py
+++ b/numpy/linalg/tests/test_linalg.py
@@ -680,6 +680,14 @@ def do(self, a, b, tags):
         assert_allclose(a, dot_generalized(np.asarray(u) * np.asarray(s)[..., None, :],
                                            np.asarray(vt)),
                         rtol=get_rtol(u.dtype))
+        def hermitian(mat):
+            axes = list(range(mat.ndim))
+            axes[-1], axes[-2] = axes[-2], axes[-1]
+            return np.conj(np.transpose(mat, axes=axes))
+        
+        assert_almost_equal(np.matmul(u, hermitian(u)), np.broadcast_to(np.eye(u.shape[-1]), u.shape))
+        assert_almost_equal(np.matmul(vt, hermitian(vt)), np.broadcast_to(np.eye(vt.shape[-1]), vt.shape))
+        assert_equal(np.sort(s)[..., ::-1], s)
         assert_(consistent_subclass(u, a))
         assert_(consistent_subclass(vt, a))