Repository: scipy/scipy

Issue:
Custom sampling methods in shgo do not work
My issue is about: custom sampling in shgo

### Reproducing code example:

Trying to use my own sampling method. But to illustrate the problem, will use random. Note, from the docs:

> ```
> sampling_method : str or function, optional
>     Current built in sampling method options are ``sobol`` and
>     ``simplicial``. The default ``simplicial`` uses less memory and provides
>     the theoretical guarantee of convergence to the global minimum in finite
>     time. The ``sobol`` method is faster in terms of sampling point
>     generation at the cost of higher memory resources and the loss of
>     guaranteed convergence. It is more appropriate for most "easier"
>     problems where the convergence is relatively fast.
>     User defined sampling functions must accept two arguments of ``n``
>     sampling points of dimension ``dim`` per call and output an array of
>     sampling points with shape `n x dim`. 
> ```

My code:
```
import numpy as np
import scipy.optimize

# Toy problem
fun = lambda x: (x[1]-0.2)**2 + x[0]
bounds = np.array([[0,+1],
                   [0,+1]])

sample = lambda n,d:np.random.uniform(size=(n,d))

res = scipy.optimize.shgo(fun,bounds,sampling_method=sample)
print(res)
```

### Error message:

Following error with personal information removed and replaces with `[...]`

```
  File "err_demo.py", line 11, in <module>
    res = scipy.optimize.shgo(fun,bounds,sampling_method=sample)
  File "[...]/site-packages/scipy/optimize/_shgo.py", line 420, in shgo
    options=options, sampling_method=sampling_method)
  File "[...]/site-packages/scipy/optimize/_shgo.py", line 459, in __init__
    " Valid methods: {}").format(', '.join(methods)))
ValueError: Unknown sampling_method specified. Valid methods: sobol, simplicial
  ...
```

### Scipy/Numpy/Python version information:
```
>>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
1.2.1 1.16.2 sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
```




Discussion:
Yes, the input check in the SHGO class is incorrect; it should accept a user-defined function as it appears to do in line 621 (```elif (sampling_method == 'sobol') or (type(sampling_method) is not str):```). 

I suppose that the check for ```sampling_method``` not being a string should be included as a check on line 457. Probably though using ```isinstance``` for the type comparison. Can @Stefan-Endres, the author of the code, perhaps take a look at this? Otherwise, I can probably make a PR at some later point.
Ah yes that is a clear bug.

> Can @Stefan-Endres, the author of the code, perhaps take a look at this? Otherwise, I can probably make a PR at some later point.

Thanks for pinging @Stefan-Endres. He'll want to fix this in his standalone implementation as well.

This seems very straightforward to fix, one liner change + regression test. A PR would be very welcome @reneeotten if there's no response within a couple of days.
> This seems very straightforward to fix, one liner change + regression test. A PR would be very welcome @reneeotten if there's no response within a couple of days.

@rgommers okay, I will submit a PR this weekend as there has been no response. I did open a PR though to fix another issue with ```shgo``` a few days (see #10114), which also added a test. Can that be reviewed/merged first so that there will be no conflicts, of do you prefer me to add commits to that PR to fix this issue as well? 
> Can that be reviewed/merged first so that there will be no conflicts

done:)
okay, I changed the two lines below (only the first one should be strictly necessary, but changing to ```isinstance``` in the latter would be better as well):
- lines 457:
```
-        if sampling_method not in methods:
+        if isinstance(sampling_method, str) and sampling_method not in methods:
```

- line 621:
```
-        elif (sampling_method == 'sobol') or (type(sampling_method) is not str):
+        elif sampling_method == 'sobol' or not isinstance(sampling_method, str):
```

that  bring us past the ```ValueError```, but still gets stuck a bit later using the example (which I think should work).
```
Traceback (most recent call last):
  File "shgo_custom_sampling_method.py", line 11, in <module>
    res = scipy.optimize.shgo(fun, bounds, sampling_method=sample)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/scipy/optimize/_shgo.py", line 423, in shgo
    shc.construct_complex()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/scipy/optimize/_shgo.py", line 726, in construct_complex
    self.iterate()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/scipy/optimize/_shgo.py", line 869, in iterate
    self.iterate_complex()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/scipy/optimize/_shgo.py", line 906, in iterate_delaunay
    self.sampled_surface(infty_cons_sampl=self.infty_cons_sampl)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/scipy/optimize/_shgo.py", line 1233, in sampled_surface
    self.sorted_samples()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/scipy/optimize/_shgo.py", line 1390, in sorted_samples
    self.Ind_sorted = np.argsort(self.C, axis=0)
AttributeError: 'SHGO' object has no attribute 'C'
```

I did look at the code a bit more, but trying to debug this isn't probably something I can do very quickly...
Thank you for pinging me (and sorry for the late reply, I did not have internet access last week).

I will take a look at this tonight. Hopefully it shouldn't take too long to fix.



Resolving patch:
diff --git a/scipy/optimize/_shgo.py b/scipy/optimize/_shgo.py
--- a/scipy/optimize/_shgo.py
+++ b/scipy/optimize/_shgo.py
@@ -454,7 +454,7 @@ def __init__(self, func, bounds, args=(), constraints=None, n=None,
 
         # Input checks
         methods = ['sobol', 'simplicial']
-        if sampling_method not in methods:
+        if isinstance(sampling_method, str) and sampling_method not in methods:
             raise ValueError(("Unknown sampling_method specified."
                               " Valid methods: {}").format(', '.join(methods)))
 
@@ -492,7 +492,7 @@ def __init__(self, func, bounds, args=(), constraints=None, n=None,
                 constraints = (constraints,)
 
             for cons in constraints:
-                if cons['type'] is 'ineq':
+                if cons['type'] == 'ineq':
                     self.g_cons.append(cons['fun'])
                     try:
                         self.g_args.append(cons['args'])
@@ -618,7 +618,7 @@ def _restrict_to_keys(dictionary, goodkeys):
             self.minimizers = self.simplex_minimizers
             self.sampling_method = sampling_method
 
-        elif (sampling_method == 'sobol') or (type(sampling_method) is not str):
+        elif sampling_method == 'sobol' or not isinstance(sampling_method, str):
             self.iterate_complex = self.iterate_delaunay
             self.minimizers = self.delaunay_complex_minimisers
             # Sampling method used
@@ -633,7 +633,9 @@ def _restrict_to_keys(dictionary, goodkeys):
             else:
                 # A user defined sampling method:
                 # self.sampling_points = sampling_method
-                self.sampling = sampling_method
+                self.sampling = self.sampling_custom
+                self.sampling_function = sampling_method  # F(n, d)
+                self.sampling_method = 'custom'
 
         # Local controls
         self.stop_l_iter = False  # Local minimisation iterations
@@ -740,7 +742,7 @@ def find_minima(self):
         and sort the results into a global return object.
         """
         self.minimizers()
-        if len(self.X_min) is not 0:
+        if len(self.X_min) != 0:
             # Minimise the pool of minimisers with local minimisation methods
             # Note that if Options['local_iter'] is an `int` instead of default
             # value False then only that number of candidates will be minimised
@@ -1372,6 +1374,21 @@ def sampling_sobol(self, n, dim):
                             + self.bounds[i][0])
         return self.C
 
+    def sampling_custom(self, n, dim):
+        """
+        Generates uniform sampling points in a hypercube and scales the points
+        to the bound limits.
+        """
+        # Generate sampling points.
+        # Generate uniform sample points in [0, 1]^m \subset R^m
+        self.C = self.sampling_function(n, dim)
+        # Distribute over bounds
+        for i in range(len(self.bounds)):
+            self.C[:, i] = (self.C[:, i] *
+                            (self.bounds[i][1] - self.bounds[i][0])
+                            + self.bounds[i][0])
+        return self.C
+
     def sampling_subspace(self):
         """Find subspace of feasible points from g_func definition"""
         # Subspace of feasible points.


Test patch:
diff --git a/scipy/optimize/tests/test__shgo.py b/scipy/optimize/tests/test__shgo.py
--- a/scipy/optimize/tests/test__shgo.py
+++ b/scipy/optimize/tests/test__shgo.py
@@ -652,6 +652,13 @@ def test_16_disp_bounds_minimizer(self):
         run_test(test1_2, sampling_method='simplicial',
                  options=options, minimizer_kwargs=minimizer_kwargs)
 
+    def test_17_custom_sampling(self):
+        """Test the functionality to add custom sampling methods to shgo"""
+        def sample(n, d):
+            return numpy.random.uniform(size=(n,d))
+        
+        run_test(test1_1, n=30, sampling_method=sample)
+
 # Failure test functions
 class TestShgoFailures(object):
     def test_1_maxiter(self):