Repository: scipy/scipy

Issue:
SLSQP inconsistent handling of invalid bounds
```
>>> from scipy.optimize import minimize
>>> minimize(lambda x: x[0]+x[1], [0,0], bounds=[(7, 1), (1, 2)], method='slsqp')
[...]
 message: 'Inequality constraints incompatible'
[...]

>>> minimize(lambda x: x[0]+x[1], [0,0], bounds=[(1, 2), (7, 1)], method='slsqp')
Traceback (most recent call last):
[...]
  File "/usr/lib/python3.5/site-packages/scipy/optimize/slsqp.py", line 344, in _minimize_slsqp
    ', '.join(str(b) for b in bnderr))
ValueError: SLSQP Error: lb > ub in bounds 1.
```

In the above examples, you can see that one of the bounds is invalid, with the lower bound greater than the upper bound.  However, a `ValueError` is raised solely depending on the order in which bounds are specified.

This appears to be caused by [this check in slsqp.py](https://github.com/scipy/scipy/blob/master/scipy/optimize/slsqp.py#L341-L344).  It appears that the `where` function is used incorrectly, as it ends up checking if any indices are non-zero.  Instead, line 341 should probably drop the `where` and the `[0]`; that would make the `ValueError` appear if any bounds are invalid, including the first one.

Or maybe those four lines should be removed altogether.  It looks like the Fortran solver has no trouble handling invalid bounds.  To me, it makes more sense to simply have the failure and the message given in the `OptimizeResult` object.


Discussion:
Thanks for the report @randyheydon 

> Or maybe those four lines should be removed altogether. It looks like the Fortran solver has no trouble handling invalid bounds. To me, it makes more sense to simply have the failure and the message given in the OptimizeResult object.

That's not a great idea; invalid input should give an exception not an apparently valid result with a ``message`` attribute that says "no I'm actually invalid".

Your analysis of the issue seems correct. Would you be interested to submit a PR to fix the issue?
> That's not a great idea; invalid input should give an exception not an apparently valid result with a message attribute that says "no I'm actually invalid".

I trimmed it out of the output in my message above, but don't forget that there is also a `success` attribute that will be `False` when the bounds are invalid.  There are many reasons that an optimization can be unsuccessful, so users of this function will need to check the `success` attribute no matter what.  As such, I don't think it would be wrong to remove this `ValueError`.

However, I have been finding that the Fortran code does not always properly respect its constraints, so maybe it's better if we raise a `ValueError` and don't let invalid constraints reach it in the first place.  On the other hand, raising `ValueError` for all invalid bounds is a somewhat backward-incompatible change; any univariate optimization problems would never have encountered the error (and the error is undocumented), so some end-user code may be unprepared for this correction.

In conclusion, I don't think there's a right answer for which fix we should apply.  But if you want to make that decision, I can try to find some time to make a PR.
> On the other hand, raising ValueError for all invalid bounds is a somewhat backward-incompatible change;

It seems more like a bugfix than a backwards compat break.

> any univariate optimization problems would never have encountered the error (and the error is undocumented), so some end-user code may be unprepared for this correction.

Well, also for univariate problems the only time users will get this exception is if they mis-specified the bounds in the first place right? If so, they may now be getting incorrect results, so this doesn't look like an obstacle to making the change to me.

> In conclusion, I don't think there's a right answer for which fix we should apply. But if you want to make that decision, I can try to find some time to make a PR.

I'd say go for an exception for invalid bounds. Unless someone disagrees in the next day or so, let's go with that. A PR would be great!
Both behaviors (return value signaling error, raise exception) can be
argued for. ValueError for invalid bounds is likely the more obvious
behavior, as it does not make sense to do optimization if the feasible
set is manifestly empty.

Thanks for the feedback.  I'll try to have a PR ready within a week.

Resolving patch:
diff --git a/scipy/optimize/slsqp.py b/scipy/optimize/slsqp.py
--- a/scipy/optimize/slsqp.py
+++ b/scipy/optimize/slsqp.py
@@ -338,7 +338,7 @@ def cjac(x, *args):
             raise IndexError('SLSQP Error: the length of bounds is not '
                              'compatible with that of x0.')
 
-        bnderr = where(bnds[:, 0] > bnds[:, 1])[0]
+        bnderr = bnds[:, 0] > bnds[:, 1]
         if bnderr.any():
             raise ValueError('SLSQP Error: lb > ub in bounds %s.' %
                              ', '.join(str(b) for b in bnderr))


Test patch:
diff --git a/scipy/optimize/tests/test_slsqp.py b/scipy/optimize/tests/test_slsqp.py
--- a/scipy/optimize/tests/test_slsqp.py
+++ b/scipy/optimize/tests/test_slsqp.py
@@ -374,6 +374,20 @@ def func(x):
         sol = minimize(func, [0, 0, 0], method='SLSQP')
         assert_(sol.jac.shape == (3,))
 
+    def test_invalid_bounds(self):
+        # Raise correct error when lower bound is greater than upper bound.
+        # See Github issue 6875.
+        bounds_list = [
+            ((1, 2), (2, 1)),
+            ((2, 1), (1, 2)),
+            ((2, 1), (2, 1)),
+            ((np.inf, 0), (np.inf, 0)),
+            ((1, -np.inf), (0, 1)),
+        ]
+        for bounds in bounds_list:
+            with assert_raises(ValueError):
+                minimize(self.fun, [-1.0, 1.0], bounds=bounds, method='SLSQP')
+
 
 if __name__ == "__main__":
     run_module_suite()