Repository: scipy/scipy

Issue:
lsq_linear hangs/infinite loop with 'trf' method
<!-- 
Thank you for taking the time to report a SciPy issue.
Please describe the issue in detail, and for bug reports
fill in the fields below. You can delete the sections that 
don't apply to your issue. You can view the final output
by clicking the preview button above.
-->

I have found several cases where scipy.optimize.lsq_linear with non-negative bounds (i.e. (0, numpy.Inf)) hangs, seemingly stuck in an infinite loop in some C code (LAPACK?) that can't be terminated via ctrl+c. It ran for at least two days the first time I noticed it. The non-default 'bvls' method and scipy.optimize.nnls() both work on the same data, one example of which I have attached:
[x.txt](https://github.com/scipy/scipy/files/3010094/x.txt)
[y.txt](https://github.com/scipy/scipy/files/3010095/y.txt)

### Reproducing code example:
<!-- 
If you place your code between the triple backticks below, 
it will be marked as a code block automatically 
-->


```
import numpy as np; import scipy.optimize as spopt
x = np.loadtxt('x.txt')
y = np.loadtxt('y.txt')
print(spopt.nnls(x,y))
print(spopt.lsq_linear(x, y, bounds=(0, np.Inf), method='bvls'))
print(spopt.lsq_linear(x, y, bounds=(0, np.Inf), method='trf', verbose=2))
```

### Output:
<!-- If any, paste the *full* error message inside a code block
as above (starting from line Traceback)
-->

```
In [1]: import numpy as np; import scipy.optimize as spopt
   ...: x = np.loadtxt('x.txt')
   ...: y = np.loadtxt('y.txt')
   ...: print(spopt.nnls(x,y))
   ...: print(spopt.lsq_linear(x, y, bounds=(0, np.Inf), method='bvls'))
   ...: print(spopt.lsq_linear(x, y, bounds=(0, np.Inf), method='trf', verbose=2))
   ...: 
(array([ 2.09932938,  0.        ,  0.        , 14.74758632]), 1.1295995521670104)
 active_mask: array([ 0., -1., -1.,  0.])
        cost: 0.6379975741279486
         fun: array([-0.003566  , -0.00431135, -0.00317054, ...,  0.00151165,
        0.00256816,  0.00488628])
     message: 'The first-order optimality measure is less than `tol`.'
         nit: 3
  optimality: 4.209012793594848e-15
      status: 1
     success: True
           x: array([ 2.09932938,  0.        ,  0.        , 14.74758632])
   Iteration        Cost      Cost reduction    Step norm     Optimality   
       0         5.9926e+01                                    6.86e+01    
       1         9.5818e+00      5.03e+01       2.39e+00       1.62e+01    
       2         1.5210e+00      8.06e+00       1.07e+00       3.26e+00    
       3         8.3612e-01      6.85e-01       3.31e-01       3.94e-01    
       4         7.9232e-01      4.38e-02       6.33e-01       7.22e-02    
       5         6.9727e-01      9.51e-02       9.96e+00       4.75e-02    
       6         6.9645e-01      8.16e-04       1.43e-02       7.09e-02    
/software/lsstsw/stack_20181012/python/miniconda3-4.5.4/envs/lsst-scipipe/lib/python3.6/site-packages/scipy/optimize/_lsq/common.py:321: RuntimeWarning: invalid value encountered in add
  y = a * t**2 + b * t + c
/software/lsstsw/stack_20181012/python/miniconda3-4.5.4/envs/lsst-scipipe/lib/python3.6/site-packages/scipy/optimize/_lsq/common.py:362: RuntimeWarning: invalid value encountered in double_scalars
  return 0.5 * q + l
       7             inf            nan            inf            inf      
```

### Scipy/Numpy/Python version information:
<!-- You can simply run the following and paste the result in a code block
-->
```
1.1.0 1.14.5 sys.version_info(major=3, minor=6, micro=6, releaselevel='final', serial=0)
```
I have also reproduced this on a different machine with 1.0.0 and 1.2.1.


Discussion:
I can reproduce this as well. `x` has magnitudes in the range (1e-192, 0.65) (with most values <1e-50) while `y` in range (1.4e-7, 1.64). So this is pretty pathological. Still, infinite loops should never happen.
@taranu thanks for finding this bug!

I quickly checked your example and found that the (final) reason it failed was returning a position of `nan` when calling `np.argmin`. After I changed this call to `np.nanargmin` it converged to the same solution. However, I want to investigate whether such case is handled correctly at all. Getting `nan`s is not a good thing generally.

Resolving patch:
diff --git a/scipy/optimize/_lsq/common.py b/scipy/optimize/_lsq/common.py
--- a/scipy/optimize/_lsq/common.py
+++ b/scipy/optimize/_lsq/common.py
@@ -319,7 +319,7 @@ def minimize_quadratic_1d(a, b, lb, ub, c=0):
         if lb < extremum < ub:
             t.append(extremum)
     t = np.asarray(t)
-    y = a * t**2 + b * t + c
+    y = t * (a * t + b) + c
     min_index = np.argmin(y)
     return t[min_index], y[min_index]
 


Test patch:
diff --git a/scipy/optimize/tests/test_lsq_common.py b/scipy/optimize/tests/test_lsq_common.py
--- a/scipy/optimize/tests/test_lsq_common.py
+++ b/scipy/optimize/tests/test_lsq_common.py
@@ -156,20 +156,45 @@ def test_minimize_quadratic_1d(self):
 
         t, y = minimize_quadratic_1d(a, b, 1, 2)
         assert_equal(t, 1)
-        assert_equal(y, a * t**2 + b * t)
+        assert_allclose(y, a * t**2 + b * t, rtol=1e-15)
 
         t, y = minimize_quadratic_1d(a, b, -2, -1)
         assert_equal(t, -1)
-        assert_equal(y, a * t**2 + b * t)
+        assert_allclose(y, a * t**2 + b * t, rtol=1e-15)
 
         t, y = minimize_quadratic_1d(a, b, -1, 1)
         assert_equal(t, 0.1)
-        assert_equal(y, a * t**2 + b * t)
+        assert_allclose(y, a * t**2 + b * t, rtol=1e-15)
 
         c = 10
         t, y = minimize_quadratic_1d(a, b, -1, 1, c=c)
         assert_equal(t, 0.1)
-        assert_equal(y, a * t**2 + b * t + c)
+        assert_allclose(y, a * t**2 + b * t + c, rtol=1e-15)
+
+        t, y = minimize_quadratic_1d(a, b, -np.inf, np.inf, c=c)
+        assert_equal(t, 0.1)
+        assert_allclose(y, a * t ** 2 + b * t + c, rtol=1e-15)
+
+        t, y = minimize_quadratic_1d(a, b, 0, np.inf, c=c)
+        assert_equal(t, 0.1)
+        assert_allclose(y, a * t ** 2 + b * t + c, rtol=1e-15)
+
+        t, y = minimize_quadratic_1d(a, b, -np.inf, 0, c=c)
+        assert_equal(t, 0)
+        assert_allclose(y, a * t ** 2 + b * t + c, rtol=1e-15)
+
+        a = -1
+        b = 0.2
+        t, y = minimize_quadratic_1d(a, b, -np.inf, np.inf)
+        assert_equal(y, -np.inf)
+
+        t, y = minimize_quadratic_1d(a, b, 0, np.inf)
+        assert_equal(t, np.inf)
+        assert_equal(y, -np.inf)
+
+        t, y = minimize_quadratic_1d(a, b, -np.inf, 0)
+        assert_equal(t, -np.inf)
+        assert_equal(y, -np.inf)
 
     def test_evaluate_quadratic(self):
         s = np.array([1.0, -1.0])