Repository: scipy/scipy

Issue:
fmin_slsqp constraint violation with 0 exit code
In the thread of the same name on SciPy-Users last month

   http://thread.gmane.org/gmane.comp.python.scientific.user/34744

a reproducible example was requested; here's one, based on p. 142 of D. M. Greig's _Optimisation_ (1980, Longman Mathematical Texts).

``` python
def f(x):
    return sum((x - np.array([1., 2., 3., 4.]))**2)

def constraint(x):
    return np.concatenate([np.dot(np.array([[-1, -1, -1, -1],
                                            [-3, -3, -2, -1]]),
                                  x) + np.array([5, 10]),
                           x])

out = fmin_slsqp(f, np.array([0.5, 1., 1.5, 2.]), f_ieqcons=constraint,
                 full_output=True)
print out
print constraint(out[0])
```

Running this with scipy.version.version=0.13.0 (installed via pip) on x86_64 Linux Mint Debian Edition, I get 

```
(array([-0.17391304,  0.82608696,  2.2173913 ,  3.60869565]), 3.5217391304347778, 14, 0, 'Optimization terminated successfully.')
[ -1.47826087e+00  -7.10542736e-15  -1.73913042e-01   8.26086956e-01
   2.21739130e+00   3.60869565e+00]
```

which is wrong (the first three, or at least the first and third, constraints are negative).  The exact solution (Greig) is np.array([0., 2, 5, 8])/3.

Running on another x86_64 LMDE machine with scipy 0.12.0 (installed from APT python-scipy), I do get the correct answer.



Discussion:
Thank you. The issue is ccecd81bc44. The bounds are enforced by additional constraint functions inserted by the Fortran code, and too large numbers apparently break the constrained minimization in some cases.

Pauli Virtanen a écrit :

> Thank you. The issue is ccecd81
> https://github.com/scipy/scipy/commit/ccecd81bc44. The bounds are
> enforced by additional constraint functions inserted by the Fortran
> code, and too large numbers apparently break the constrained
> minimization in some cases.

Just `git revert` that change?

That, and document the default values for the bounds and how `inf` is interpreted. Also, a test case... OTOH, bounds of order `1e100` still seem to work fine, but on the other hand the 1e12 is probably good enough in practice.

Pauli Virtanen a écrit :

> That, and document the default values for the bounds and how |inf| is
> interpreted. OTOH, bounds of order |1e100| still seem to work fine, but
> on the other hand the 1e12 is probably good enough in practice.

Ok, I'll work on it.


Resolving patch:
diff --git a/scipy/optimize/_minimize.py b/scipy/optimize/_minimize.py
--- a/scipy/optimize/_minimize.py
+++ b/scipy/optimize/_minimize.py
@@ -201,7 +201,9 @@ def minimize(fun, x0, args=(), method='BFGS', jac=None, hess=None,
     Method *SLSQP* uses Sequential Least SQuares Programming to minimize a
     function of several variables with any combination of bounds, equality
     and inequality constraints. The method wraps the SLSQP Optimization
-    subroutine originally implemented by Dieter Kraft [12]_.
+    subroutine originally implemented by Dieter Kraft [12]_. Note that the
+    wrapper handles infinite values in bounds by converting them into large
+    floating values.
 
     References
     ----------
diff --git a/scipy/optimize/slsqp.py b/scipy/optimize/slsqp.py
--- a/scipy/optimize/slsqp.py
+++ b/scipy/optimize/slsqp.py
@@ -99,6 +99,7 @@ def fmin_slsqp(func, x0, eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None,
     bounds : list
         A list of tuples specifying the lower and upper bound
         for each independent variable [(xl0, xu0),(xl1, xu1),...]
+        Infinite values will be interpreted as large floating values.
     fprime : callable `f(x,*args)`
         A function that evaluates the partial derivatives of func.
     fprime_eqcons : callable `f(x,*args)`
@@ -317,7 +318,7 @@ def cjac(x, *args):
 
     # Decompose bounds into xl and xu
     if bounds is None or len(bounds) == 0:
-        xl, xu = array([-finfo(float).max]*n), array([finfo(float).max]*n)
+        xl, xu = array([-1.0E12]*n), array([1.0E12]*n)
     else:
         bnds = array(bounds, float)
         if bnds.shape[0] != n:
@@ -332,8 +333,8 @@ def cjac(x, *args):
 
         # filter -inf, inf and NaN values
         infbnd = ~isfinite(bnds)
-        xl[infbnd[:, 0]] = -finfo(float).max
-        xu[infbnd[:, 1]] = finfo(float).max
+        xl[infbnd[:, 0]] = -1.0E12
+        xu[infbnd[:, 1]] = 1.0E12
 
     # Initialize the iteration counter and the mode value
     mode = array(0,int)


Test patch:
diff --git a/scipy/optimize/tests/test_optimize.py b/scipy/optimize/tests/test_optimize.py
--- a/scipy/optimize/tests/test_optimize.py
+++ b/scipy/optimize/tests/test_optimize.py
@@ -504,6 +504,18 @@ def check(method):
                        'cobyla', 'slsqp']:
             yield check, method
 
+    def test_slsqp_respect_bounds(self):
+        # github issue 3108
+        def f(x):
+            return sum((x - np.array([1., 2., 3., 4.]))**2)
+        def cons(x):
+            a = np.array([[-1, -1, -1, -1], [-3, -3, -2, -1]])
+            return np.concatenate([np.dot(a, x) + np.array([5, 10]), x])
+        x0 = np.array([0.5, 1., 1.5, 2.])
+        res = optimize.minimize(f, x0, method='slsqp',
+                                constraints={'type': 'ineq', 'fun': cons})
+        assert_allclose(res.x, np.array([0., 2, 5, 8])/3, atol=1e-12)
+
 
 class TestLBFGSBBounds(TestCase):
     """ Tests for L-BFGS-B with bounds """