Repository: scipy/scipy

Issue:
BUG: unclear error for invalid bracketing 
### Describe your issue.

 I have enclosed a minimum reproducible example with data  and I am trying to use two methods in the scipy optimize package - https://docs.scipy.org/doc/scipy/reference/optimize.html

One is minimize_scalar and the other is Brent's method. But both the outputs have large differences. Can anybody explain why ?

Also I get a runtime error while running Brent- 

test.py:29: RuntimeWarning: invalid value encountered in power
  ls = np.sum((x1[1:] - x1[:-1]*np.sign(a)*a**dt)**2) 

Additionally the Brent one has another one if I specify a triple interval
ala -  dum = brent(func,brack=(-1,0,1),tol=3.0e-8)
ValueError  : Not a bracketing interval. 

https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brent.html But docs say that I can specify a triplet in the bracketing interval. 


### Reproducing Code Example

```python
`import numpy as np
from scipy.optimize import brent,minimize_scalar
import math

def main():

    data = np.loadtxt("x.dat",skiprows=0)
    t = data[:,0] 
    x = data[:,1]
    dum = minls(t,x)
    
def minls(t,x):
    a = 1/math.e
    dt = np.diff(t)
    def func(a):
        try:
            ls = np.sum((x[1:] - x[:-1]*a**dt)**2)
        except Warning:
            print(traceback.format_exc())
        return ls

    dum = brent(func,brack=(-1,0,1),tol=3.0e-8)
    dum1 = minimize_scalar(func, bounds=[0, 1], method='bounded').x
    print(dum,dum1)
    return dum

main()
`
```


### Error message

```shell
I encounter an error and a warning wi this

Test.py:29: RuntimeWarning: invalid value encountered in power 


ValueError  : Not a bracketing interval.
```


### SciPy/NumPy/Python version information

Scipy 1.7.1 /Numpy 1.19.1/Python3.8 


Discussion:
Hi @winash12, thank you for reporting.

The warning is just because `a` is getting some values such as 0 during the optimization which then is troubling for the power.

For the bracketing the message (and doc) is not clear and could be improved. It's not only the values of `x` which should be monotone but also `f(x)` should be convex. See in the code

```python
if not ((xa < xb) and (xb < xc)):
    raise ValueError("Not a bracketing interval.")
fa = func(*((xa,) + args))
fb = func(*((xb,) + args))
fc = func(*((xc,) + args))
if not ((fb < fa) and (fb < fc)):
    raise ValueError("Not a bracketing interval.")
```

The message should be different to indicate what's wrong.
The title of your issue is misleading, I don't know what you mean by large difference between the algorithms. I do see an issue with our error messaging and you could rename your issue accordingly.
@tupui. I agree there are several issues.

If I remove the second number in the bracket the code works fine but the outputs between minimum_scalar and brent are very different.

`dum = brent(func,brack=(-1,1),tol=3.0e-8)` - This code works with no issues. 
If I keep it this way 
`dum = brent(func,brack=(-1,0,1),tol=3.0e-8)`
`
test.py:17: RuntimeWarning: invalid value encountered in power
  ls = np.sum((x[1:] - x[:-1]*a**dt)**2)
Traceback (most recent call last):
  File "test.py", line 27, in <module>
    main()
  File "test.py", line 10, in main
    dum = minls(t,x)
  File "test.py", line 22, in minls
    dum = brent(func,brack=(-1,0,1),tol=3.0e-8)
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.7.3-py3.8-linux-x86_64.egg/scipy/optimize/optimize.py", line 2364, in brent
    res = _minimize_scalar_brent(func, brack, args, **options)
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.7.3-py3.8-linux-x86_64.egg/scipy/optimize/optimize.py", line 2396, in _minimize_scalar_brent
    brent.optimize()
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.7.3-py3.8-linux-x86_64.egg/scipy/optimize/optimize.py", line 2180, in optimize
    xa, xb, xc, fa, fb, fc, funcalls = self.get_bracket_info()
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.7.3-py3.8-linux-x86_64.egg/scipy/optimize/optimize.py", line 2168, in get_bracket_info
    raise ValueError("Not a bracketing interval.")
ValueError: Not a bracketing interval.
`

With Brent I get -1.9999 and minimize_scalar I get -0.471 
Yes, this is just because `brack` is not equivalent to `bounds` at all. In `minimize_scalar` you are forcing to stay within [0, 1] while with `brent` it's not bounded.
See the documentation of `bracket` https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.bracket.html
> See the documentation of `bracket` https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.bracket.html

Ok thanks. If I were to rephrase your point you are saying the output of Brent and Minimize_scalar will be different because Brent is not  bounded and the other one is  bounded. Am I correct ?

So then that leaves me with only 1 error. The value error issue. How do iI fix  that ? 
> So then that leaves me with only 1 error. The value error issue. How do iI fix that ?

Note that this is not a user forum. As I said above this is related to what's said in the documentation of `bracket`

```
Given a function and distinct initial points, search in the downhill direction (as defined by the initial points) and return new points xa, xb, xc that bracket the minimum of the function f(xa) > f(xb) < f(xc). It doesn’t always mean that obtained solution will satisfy xa<=x<=xb.
```

To this issue, I renamed it as there is something we should do regarding the error message about being able to bracket or not.




> > So then that leaves me with only 1 error. The value error issue. How do iI fix that ?
> 
> Note that this is not a user forum. As I said above this is related to what's said in the documentation of `bracket`
> 
> ```
> Given a function and distinct initial points, search in the downhill direction (as defined by the initial points) and return new points xa, xb, xc that bracket the minimum of the function f(xa) > f(xb) < f(xc). It doesn’t always mean that obtained solution will satisfy xa<=x<=xb.
> ```
> 
> To this issue, I renamed it as there is something we should do regarding the error message about being able to bracket or not.

I understand. I asked this on the scipy mailing list and got no response and I had to ask here. 
The user mailing list is not very followed. StackOverflow is more appropriate nowadays.
We can go ahead and close this issue then. 
No, as I said there are improvements to do on the error message.
Ok great ! Glad I got to do something useful. 
Yes, thank you 😃 And this is a fairly simple change if you want to try a PR.
I would like to try a PR, what should the error message be?
Feel free to make propositions. We have 2 checks, the 2 messages should be different. One saying something like the values should respect xa<xb<xc and the other saying f(xa) > f(xb) < f(xc). This will require 2 tests to check (if they don't already exist) to verify that we raise the correct warning given the correct wrong inputs.
Thank you so much, @tupui, I am excited about my first pull request in an open-source project. I have created different error messages for the two test cases.

```python
if not ((xa < xb) and (xb < xc)):
    raise ValueError("Your bracketing values (xa,xb,xc) does not fulfill this requirement,(xa < xb) and (xb < xc), https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brent.html")
fa = func(*((xa,) + args))
fb = func(*((xb,) + args))
fc = func(*((xc,) + args))
if not ((fb < fa) and (fb < fc)):
    raise ValueError("For you bracketing values, (xa,xb,xc), this requirement is not fulfilled,(f(xb) < f(xa)) and (f(xb) < f(xc)), Check docs, https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brent.html")
```

I want to write tests for this now:

```python
class TestOptimizeScalar:
    def setup_method(self):
        self.solution = 1.5

   def fun(self, x, a=1.5):
         """Objective function"""
        return (x - a)**2 - 0.8

    def test_brent(self):
        x = optimize.brent(self.fun)
        assert_allclose(x, self.solution, atol=1e-6)

        x = optimize.brent(self.fun, brack=(-3, -2))
        assert_allclose(x, self.solution, atol=1e-6)

        x = optimize.brent(self.fun, full_output=True)
        assert_allclose(x[0], self.solution, atol=1e-6)

        x = optimize.brent(self.fun, brack=(-15, -1, 15))
        assert_allclose(x, self.solution, atol=1e-6)

```
I would like to know if I can create another function, say func1, and use the function in winash's code to test for the changes and then do an assert.raises in the test case.
@Chinwendu20 No need for the link to the doc. The following is fine
```
Bracketing values (xa, xb, xc) do not fulfill this requirement: (xa < xb) and (xb < xc)
Bracketing values (xa, xb, xc) do not fulfill this requirement: (f(xb) < f(xa)) and (f(xb) < f(xc))
```

You could create another function or use a lambda if you can write something concise. What matters in the test is that we check the message (`with pytest.raises(ValueError, match="...")`)

@tupui thanks. Please how did you format the code sample in the comment? I was trying to do that.
You can click on the three dots on the right corner of your comments to edit. You will see how to format code.
> You can click on the three dots on the right corner of your comments to edit. You will see how to format code.

Got it, thank you. I appreciate your time and patience with me. Expect my PR soon.
@Chinwendu20 are you still working on this?
Yes I am. I am trying to set up my development environment with conda but I was getting conflicting package dependency error. I followed the steps on contribution.md.  
@Chinwendu20 you should try to follow the relevant guide from your setup https://scipy.github.io/devdocs/dev/contributor/contributor_toc.html#development-environment

If you have specific issues, let me know.
Thanks, I am doing that now. I would let you know if I encounter errors. 
Look at the error message:

![image](https://user-images.githubusercontent.com/59079323/172166366-9784ecb0-a5c1-4cbb-abde-88395c6f8bc4.png)

@Chinwendu20 Please try with Python 3.9. I remember an issue with some dependencies.

@melissawm this was your problem right?
Thanks. I am doing that now,
@tupui I believe so - let me check. Thanks for pinging!
@tupui I did this conda env create -f environment.yml python=3.9 and I am still getting same error
It seems I have not been able to get past this error. I have tried following the solutions online. I would have really liked to try a PR
@Chinwendu20 I cannot reproduce the error you are having. I am on macOS and I start from outside any environment. Using either conda or mamba, creating the env is working as expected without any errors. All the following is getting through

```
conda env create -f environment.yml python=3.9
conda env create -f environment.yml
mamba env create -f environment.yml python=3.9
mamba env create -f environment.yml
```

On what platform and version are you?

Another alternative is to use `gitpod`. See these instructions to work in an online dev environment: https://scipy.github.io/devdocs/dev/contributor/quickstart_gitpod.html#quickstart-gitpod
Okay I would try using gitpod. I am using conda on anaconda prompt on a Windows 10 pro system

Resolving patch:
diff --git a/scipy/optimize/_optimize.py b/scipy/optimize/_optimize.py
--- a/scipy/optimize/_optimize.py
+++ b/scipy/optimize/_optimize.py
@@ -2248,12 +2248,19 @@ def get_bracket_info(self):
             if (xa > xc):  # swap so xa < xc can be assumed
                 xc, xa = xa, xc
             if not ((xa < xb) and (xb < xc)):
-                raise ValueError("Not a bracketing interval.")
+                raise ValueError(
+                    "Bracketing values (xa, xb, xc) do not"
+                    " fulfill this requirement: (xa < xb) and (xb < xc)"
+                )
             fa = func(*((xa,) + args))
             fb = func(*((xb,) + args))
             fc = func(*((xc,) + args))
             if not ((fb < fa) and (fb < fc)):
-                raise ValueError("Not a bracketing interval.")
+                raise ValueError(
+                    "Bracketing values (xa, xb, xc) do not fulfill"
+                    " this requirement: (f(xb) < f(xa)) and (f(xb) < f(xc))"
+                )
+
             funcalls = 3
         else:
             raise ValueError("Bracketing interval must be "
@@ -2613,12 +2620,18 @@ def _minimize_scalar_golden(func, brack=None, args=(),
         if (xa > xc):  # swap so xa < xc can be assumed
             xc, xa = xa, xc
         if not ((xa < xb) and (xb < xc)):
-            raise ValueError("Not a bracketing interval.")
+            raise ValueError(
+                "Bracketing values (xa, xb, xc) do not"
+                " fulfill this requirement: (xa < xb) and (xb < xc)"
+            )
         fa = func(*((xa,) + args))
         fb = func(*((xb,) + args))
         fc = func(*((xc,) + args))
         if not ((fb < fa) and (fb < fc)):
-            raise ValueError("Not a bracketing interval.")
+            raise ValueError(
+                "Bracketing values (xa, xb, xc) do not fulfill"
+                " this requirement: (f(xb) < f(xa)) and (f(xb) < f(xc))"
+            )
         funcalls = 3
     else:
         raise ValueError("Bracketing interval must be length 2 or 3 sequence.")


Test patch:
diff --git a/scipy/optimize/tests/test_optimize.py b/scipy/optimize/tests/test_optimize.py
--- a/scipy/optimize/tests/test_optimize.py
+++ b/scipy/optimize/tests/test_optimize.py
@@ -1540,6 +1540,14 @@ def test_brent(self):
         x = optimize.brent(self.fun, brack=(-15, -1, 15))
         assert_allclose(x, self.solution, atol=1e-6)
 
+        message = r"\(f\(xb\) < f\(xa\)\) and \(f\(xb\) < f\(xc\)\)"
+        with pytest.raises(ValueError, match=message):
+            optimize.brent(self.fun, brack=(-1, 0, 1))
+
+        message = r"\(xa < xb\) and \(xb < xc\)"
+        with pytest.raises(ValueError, match=message):
+            optimize.brent(self.fun, brack=(0, -1, 1))
+
     def test_golden(self):
         x = optimize.golden(self.fun)
         assert_allclose(x, self.solution, atol=1e-6)
@@ -1563,6 +1571,14 @@ def test_golden(self):
             nfev0, nfev = x0[2], x[2]
             assert_equal(nfev - nfev0, maxiter)
 
+        message = r"\(f\(xb\) < f\(xa\)\) and \(f\(xb\) < f\(xc\)\)"
+        with pytest.raises(ValueError, match=message):
+            optimize.golden(self.fun, brack=(-1, 0, 1))
+
+        message = r"\(xa < xb\) and \(xb < xc\)"
+        with pytest.raises(ValueError, match=message):
+            optimize.golden(self.fun, brack=(0, -1, 1))
+
     def test_fminbound(self):
         x = optimize.fminbound(self.fun, 0, 1)
         assert_allclose(x, 1, atol=1e-4)