Repository: scipy/scipy

Issue:
`shgo` error since scipy 1.8.0.dev0+1529.803e52d 
<!-- 
Thank you for taking the time to file a bug report. 
Please fill in the fields below, deleting the sections that 
don't apply to your issue. You can view the final output
by clicking the preview button above.
Note: This is a comment, and won't appear in the output.
-->

`shgo` raises an error with the following example since scipy 1.8.0.dev0+1529.803e52d.

This error was noticed in the hyperspy test suite (https://github.com/hyperspy/hyperspy-extensions-list/actions/runs/1111267440) and previous scipy development version (1.8.0.dev0+1503.8647ee9) didn't raise an error. Scipy development version are installed from https://pypi.anaconda.org/scipy-wheels-nightly/simple.

#### Reproducing code example:
<!-- 
If you place your code between the triple backticks below, 
it will be rendered as a code block. 
-->

```python
import sys

import numpy as np
import scipy
from scipy.optimize import shgo

print(scipy.__version__, np.__version__, sys.version_info)


def func(x_values, param):
    a, b = param
    return a*x_values + b


def errfunc(param, y):
    return func(x_values, param) - y


def errfunc_sq(param, y, weights=None):
    if weights is None:
        weights = 1.0
    return ((weights * errfunc(param, y)) ** 2).sum()

x_values = np.arange(100)
y_values = func(x_values, (2, 10)) + np.random.random(x_values.size)

results = shgo(errfunc_sq,
               bounds=((-100, 100), (-100, 100)),
               args=(y_values, None)
               )

print(results)
```

#### Error message:
<!-- If any, paste the *full* error message inside a code block
as above (starting from line Traceback)
-->

```python
1.8.0.dev0+1529.803e52d 1.21.1 sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0)
Traceback (most recent call last):
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 630, in __getitem__
    return self.cache[x]
KeyError: (0, 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "untitled0.py", line 34, in <module>
    results = shgo(errfunc_sq,
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 420, in shgo
    shc.construct_complex()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 733, in construct_complex
    self.iterate()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 876, in iterate
    self.iterate_complex()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 895, in iterate_hypercube
    self.HC = Complex(self.dim, self.func, self.args,
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 25, in __init__
    self.n_cube(dim, symmetry=symmetry)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 76, in n_cube
    self.C0.add_vertex(self.V[origintuple])
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 634, in __getitem__
    xval = Vertex(x, bounds=self.bounds,
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 557, in __init__
    self.f = func(x_a, *func_args)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/optimize.py", line 464, in function_wrapper
    fx = function(np.copy(x), *(wrapper_args + args))
TypeError: errfunc_sq() takes from 2 to 3 positional arguments but 5 were given
```

#### Scipy/Numpy/Python version information:
<!-- You can simply run the following and paste the result in a code block
```
import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
```
-->
1.8.0.dev0+1529.803e52d 1.21.1 sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0)


Discussion:
Thanks for the report. @Stefan-Endres is working on a PR that is going to refactor quite some things. Stefan, does your PR address this?
Thank you for the report @ericpre. @tupui this is a new bug, but on the old caching please keep the issue open I will fix this in the new caching library.
It seems that the regression has been introduced in https://github.com/scipy/scipy/pull/14274.
This bug got me too. FWIW, I reproduced it with the following code:

```python
import scipy.optimize as so

def obj_fun(x, a, b):
    return (x[0]/a)**2 + (x[1]/b)**2

def main():
    a = 1.0
    b = 1.0

    bounds = [(-1.0, 1.0),
              (-1.0, 1.0)]

    result = so.shgo(obj_fun,
                     bounds,
                     args = (a,b))

    if result.success:
        print(f"x = {result.x} and obj_fun(x) = {result.x}")
    else:
        print(f"""
 Message from minimizer: {result.message}
 Status code from minimizer: {result.status}
 Optimization "solution" at failure: {result.x}
        """)

if __name__ == "__main__":
    main()
```
It looks like the problem is in how `so.shgo` handles the `args` parameter. It appears to double the arguments fed to the objective function. This problem does not occur in SciPy 1.5.0.
I think I've found a solution. On line 900 of `scipy/optimize/_shgo.py`, change
```python
self.HC = Complex(self.dim, self.func, self.args,
```
to
```python
self.HC = Complex(self.dim, self.func, (),
```
What's happening is that since `self.func` already contains the contents of `self.args`, when `Complex` is constructed, it effectively includes `self.args` *twice*, so when it calls the objective function, it effectively calls it as `func(x, *(args+args))`.

Unfortunately, an alternative like setting `self.func` to `_wrap_scalar_function(func, ())` doesn't work, since there are other places in the code where it's assumed that `self.func` wraps the contents of `self.args`. `self.args` is also used in other places in the code, so setting it to `()` probably won't work either. If the sampling method is changed from the default `simplicial`, SHGO still appears to work properly, regardless of my change. The change I recommend appears to be the most minimal change needed to get SHGO to work with externally supplied arguments again.
There are large development in progress changing the whole architecture of `shgo`. We will keep these in mind.
Understood, but bear in mind that this bug is not only almost a year old, but breaks some very basic documented functionality. A minimal fix now will provide an indicator to those making large changes of what should or shouldn't be done with function arguments, while solving a problem facing end users of SHGO now.
@jjramsey you are welcomed to make such a fix.
The large developments are happening here #14013
Looks like the pull request for those large developments has been stalled since Jul 26, 2021, apparently due to some automatic tests failing for some unclear reason. Any way to rerun the tests to see if the failure was a fluke (or perhaps some fault in the Azure pipelines)?
@jjramsey the PR is not staled due to failing tests but it's conditioned to maintainers time to work on SciPy. Most of us are volunteers and don't have much time to work on SciPy.

Resolving patch:
diff --git a/scipy/optimize/_shgo.py b/scipy/optimize/_shgo.py
--- a/scipy/optimize/_shgo.py
+++ b/scipy/optimize/_shgo.py
@@ -464,6 +464,7 @@ def __init__(self, func, bounds, args=(), constraints=None, n=None,
                               " Valid methods: {}").format(', '.join(methods)))
 
         # Initiate class
+        self._raw_func = func  # some methods pass args in (e.g. Complex)
         _, self.func = _wrap_scalar_function(func, args)
         self.bounds = bounds
         self.args = args
@@ -897,8 +898,11 @@ def iterate_hypercube(self):
         """
         # Iterate the complex
         if self.n_sampled == 0:
-            # Initial triangulation of the hyper-rectangle
-            self.HC = Complex(self.dim, self.func, self.args,
+            # Initial triangulation of the hyper-rectangle. Note that
+            # we use `self.raw_func` as `self.func` is a *wrapped* function
+            # that already takes the original function arguments into
+            # account.
+            self.HC = Complex(self.dim, self._raw_func, self.args,
                               self.symmetry, self.bounds, self.g_cons,
                               self.g_args)
         else:


Test patch:
diff --git a/scipy/optimize/tests/test__shgo.py b/scipy/optimize/tests/test__shgo.py
--- a/scipy/optimize/tests/test__shgo.py
+++ b/scipy/optimize/tests/test__shgo.py
@@ -1,5 +1,6 @@
 import logging
 import numpy
+from numpy.testing import assert_allclose
 import pytest
 from pytest import raises as assert_raises, warns
 from scipy.optimize import shgo, Bounds
@@ -469,6 +470,13 @@ def callback_func(x):
             shgo(test.f, test.bounds, n=1, sampling_method='simplicial',
                  callback=callback_func, options={'disp': True})
 
+    def test_args_gh14589(self):
+        # Using `args` used to cause `shgo` to fail; see #14589, #15986, #16506
+        res = shgo(func=lambda x, y, z: x*z + y, bounds=[(0, 3)], args=(1, 2))
+        ref = shgo(func=lambda x: 2*x + 1, bounds=[(0, 3)])
+        assert_allclose(res.fun, ref.fun)
+        assert_allclose(res.x, ref.x)
+
     @pytest.mark.slow
     def test_4_1_known_f_min(self):
         """Test known function minima stopping criteria"""