def fmin_cobyla(func, x0, cons, args=(), consargs=None, rhobeg=<NUM_LIT:1.0>,<EOL>rhoend=<NUM_LIT>, iprint=<NUM_LIT:1>, maxfun=<NUM_LIT:1000>, disp=None, catol=<NUM_LIT>):

err = "<STR_LIT>""<STR_LIT>"<EOL>try:<EOL><INDENT>len(cons)<EOL><DEDENT>except TypeError:<EOL><INDENT>if callable(cons):<EOL><INDENT>cons = [cons]<EOL><DEDENT>else:<EOL><INDENT>raise TypeError(err)<EOL><DEDENT><DEDENT>else:<EOL><INDENT>for thisfunc in cons:<EOL><INDENT>if not callable(thisfunc):<EOL><INDENT>raise TypeError(err)<EOL><DEDENT><DEDENT><DEDENT>if consargs is None:<EOL><INDENT>consargs = args<EOL><DEDENT>con = tuple({'<STR_LIT:type>': '<STR_LIT>', '<STR_LIT>': c, '<STR_LIT:args>': consargs} for c in cons)<EOL>if disp is not None:<EOL><INDENT>iprint = disp<EOL><DEDENT>opts = {'<STR_LIT>': rhobeg,<EOL>'<STR_LIT>': rhoend,<EOL>'<STR_LIT>': iprint,<EOL>'<STR_LIT>': iprint != <NUM_LIT:0>,<EOL>'<STR_LIT>': maxfun,<EOL>'<STR_LIT>': catol}<EOL>sol = _minimize_cobyla(func, x0, args, constraints=con,<EOL>**opts)<EOL>if iprint > <NUM_LIT:0> and not sol['<STR_LIT:success>']:<EOL><INDENT>print("<STR_LIT>" % (sol.message,))<EOL><DEDENT>return sol['<STR_LIT:x>']<EOL>

Minimize a function using the Constrained Optimization BY Linear
Approximation (COBYLA) method. This method wraps a FORTRAN
implentation of the algorithm.

Parameters
----------
func : callable
    Function to minimize. In the form func(x, \\*args).
x0 : ndarray
    Initial guess.
cons : sequence
    Constraint functions; must all be ``>=0`` (a single function
    if only 1 constraint). Each function takes the parameters `x`
    as its first argument.
args : tuple
    Extra arguments to pass to function.
consargs : tuple
    Extra arguments to pass to constraint functions (default of None means
    use same extra arguments as those passed to func).
    Use ``()`` for no extra arguments.
rhobeg :
    Reasonable initial changes to the variables.
rhoend :
    Final accuracy in the optimization (not precisely guaranteed). This
    is a lower bound on the size of the trust region.
iprint : {0, 1, 2, 3}
    Controls the frequency of output; 0 implies no output.  Deprecated.
disp : {0, 1, 2, 3}
    Over-rides the iprint interface.  Preferred.
maxfun : int
    Maximum number of function evaluations.
catol : float
    Absolute tolerance for constraint violations.

Returns
-------
x : ndarray
    The argument that minimises `f`.

See also
--------
minimize: Interface to minimization algorithms for multivariate
    functions. See the 'COBYLA' `method` in particular.

Notes
-----
This algorithm is based on linear approximations to the objective
function and each constraint. We briefly describe the algorithm.

Suppose the function is being minimized over k variables. At the
jth iteration the algorithm has k+1 points v_1, ..., v_(k+1),
an approximate solution x_j, and a radius RHO_j.
(i.e. linear plus a constant) approximations to the objective
function and constraint functions such that their function values
agree with the linear approximation on the k+1 points v_1,.., v_(k+1).
This gives a linear program to solve (where the linear approximations
of the constraint functions are constrained to be non-negative).

However the linear approximations are likely only good
approximations near the current simplex, so the linear program is
given the further requirement that the solution, which
will become x_(j+1), must be within RHO_j from x_j. RHO_j only
decreases, never increases. The initial RHO_j is rhobeg and the
final RHO_j is rhoend. In this way COBYLA's iterations behave
like a trust region algorithm.

Additionally, the linear program may be inconsistent, or the
approximation may give poor improvement. For details about
how these issues are resolved, as well as how the points v_i are
updated, refer to the source code or the references below.

References
----------
Powell M.J.D. (1994), "A direct search optimization method that models
the objective and constraint functions by linear interpolation.", in
Advances in Optimization and Numerical Analysis, eds. S. Gomez and
J-P Hennart, Kluwer Academic (Dordrecht), pp. 51-67

Powell M.J.D. (1998), "Direct search algorithms for optimization
calculations", Acta Numerica 7, 287-336

Powell M.J.D. (2007), "A view of algorithms for optimization without
derivatives", Cambridge University Technical Report DAMTP 2007/NA03

Examples
--------
Minimize the objective function f(x,y) = x*y subject
to the constraints x**2 + y**2 < 1 and y > 0::

    >>> def objective(x):
    ...     return x[0]*x[1]
    ...
    >>> def constr1(x):
    ...     return 1 - (x[0]**2 + x[1]**2)
    ...
    >>> def constr2(x):
    ...     return x[1]
    ...
    >>> fmin_cobyla(objective, [0.0, 0.1], [constr1, constr2], rhoend=1e-7)

       Normal return from subroutine COBYLA

       NFVALS =   64   F =-5.000000E-01    MAXCV = 1.998401E-14
       X =-7.071069E-01   7.071067E-01
    array([-0.70710685,  0.70710671])

The exact solution is (-sqrt(2)/2, sqrt(2)/2).


def _minimize_cobyla(fun, x0, args=(), constraints=(),<EOL>rhobeg=<NUM_LIT:1.0>, tol=<NUM_LIT>, iprint=<NUM_LIT:1>, maxiter=<NUM_LIT:1000>,<EOL>disp=False, catol=<NUM_LIT>, **unknown_options):

_check_unknown_options(unknown_options)<EOL>maxfun = maxiter<EOL>rhoend = tol<EOL>if not disp:<EOL><INDENT>iprint = <NUM_LIT:0><EOL><DEDENT>if isinstance(constraints, dict):<EOL><INDENT>constraints = (constraints, )<EOL><DEDENT>for ic, con in enumerate(constraints):<EOL><INDENT>try:<EOL><INDENT>ctype = con['<STR_LIT:type>'].lower()<EOL><DEDENT>except KeyError:<EOL><INDENT>raise KeyError('<STR_LIT>' % ic)<EOL><DEDENT>except TypeError:<EOL><INDENT>raise TypeError('<STR_LIT>'<EOL>'<STR_LIT>')<EOL><DEDENT>except AttributeError:<EOL><INDENT>raise TypeError("<STR_LIT>")<EOL><DEDENT>else:<EOL><INDENT>if ctype != '<STR_LIT>':<EOL><INDENT>raise ValueError("<STR_LIT>"<EOL>"<STR_LIT>" % con['<STR_LIT:type>'])<EOL><DEDENT><DEDENT>if '<STR_LIT>' not in con:<EOL><INDENT>raise KeyError('<STR_LIT>' % ic)<EOL><DEDENT>if '<STR_LIT:args>' not in con:<EOL><INDENT>con['<STR_LIT:args>'] = ()<EOL><DEDENT><DEDENT>m = len(constraints)<EOL>def calcfc(x, con):<EOL><INDENT>f = fun(x, *args)<EOL>for k, c in enumerate(constraints):<EOL><INDENT>con[k] = c['<STR_LIT>'](x, *c['<STR_LIT:args>'])<EOL><DEDENT>return f<EOL><DEDENT>info = np.zeros(<NUM_LIT:4>, np.float64)<EOL>xopt, info = _cobyla.minimize(calcfc, m=m, x=np.copy(x0), rhobeg=rhobeg,<EOL>rhoend=rhoend, iprint=iprint, maxfun=maxfun,<EOL>dinfo=info)<EOL>if info[<NUM_LIT:3>] > catol:<EOL><INDENT>info[<NUM_LIT:0>] = <NUM_LIT:4><EOL><DEDENT>return OptimizeResult(x=xopt,<EOL>status=int(info[<NUM_LIT:0>]),<EOL>success=info[<NUM_LIT:0>] == <NUM_LIT:1>,<EOL>message={<NUM_LIT:1>: '<STR_LIT>',<EOL><NUM_LIT:2>: '<STR_LIT>'<EOL>'<STR_LIT>',<EOL><NUM_LIT:3>: '<STR_LIT>'<EOL>'<STR_LIT>',<EOL><NUM_LIT:4>: '<STR_LIT>'<EOL>'<STR_LIT>'<EOL>'<STR_LIT>'<EOL>}.get(info[<NUM_LIT:0>], '<STR_LIT>'),<EOL>nfev=int(info[<NUM_LIT:1>]),<EOL>fun=info[<NUM_LIT:2>],<EOL>maxcv=info[<NUM_LIT:3>])<EOL>

Minimize a scalar function of one or more variables using the
Constrained Optimization BY Linear Approximation (COBYLA) algorithm.

Options for the COBYLA algorithm are:
    rhobeg : float
        Reasonable initial changes to the variables.
    tol : float
        Final accuracy in the optimization (not precisely guaranteed).
        This is a lower bound on the size of the trust region.
    disp : bool
        Set to True to print convergence messages. If False,
        `verbosity` is ignored as set to 0.
    maxiter : int
        Maximum number of function evaluations.
    catol : float
        Tolerance (absolute) for constraint violations

This function is called by the `minimize` function with
`method=COBYLA`. It is not supposed to be called directly.


result.time = t<EOL>result.name = name<EOL>if not hasattr(result, "<STR_LIT>"):<EOL><INDENT>result.njev = <NUM_LIT:0><EOL><DEDENT>if not hasattr(result, "<STR_LIT>"):<EOL><INDENT>result.nhev = <NUM_LIT:0><EOL><DEDENT>self.results.append(result)<EOL>


results = self.average_results()<EOL>results = sorted(results, key=lambda x: (x.nfail, x.mean_time))<EOL>print("<STR_LIT>")<EOL>print("<STR_LIT>")<EOL>print("<STR_LIT>" % (self.function_name))<EOL>print("<STR_LIT>" % (results[<NUM_LIT:0>].ndim, str(self.minimizer_kwargs)))<EOL>print("<STR_LIT>" % (results[<NUM_LIT:0>].ntrials))<EOL>print("<STR_LIT>")<EOL>print("<STR_LIT>")<EOL>for res in results:<EOL><INDENT>print("<STR_LIT>" %<EOL>(res.name, res.nfail, res.mean_nfev, res.mean_njev, res.mean_nhev, res.mean_time))<EOL><DEDENT>


grouped_results = defaultdict(list)<EOL>for res in self.results:<EOL><INDENT>grouped_results[res.name].append(res)<EOL><DEDENT>averaged_results = dict()<EOL>for name, result_list in grouped_results.items():<EOL><INDENT>newres = scipy.optimize.OptimizeResult()<EOL>newres.name = name<EOL>newres.mean_nfev = np.mean([r.nfev for r in result_list])<EOL>newres.mean_njev = np.mean([r.njev for r in result_list])<EOL>newres.mean_nhev = np.mean([r.nhev for r in result_list])<EOL>newres.mean_time = np.mean([r.time for r in result_list])<EOL>newres.ntrials = len(result_list)<EOL>newres.nfail = len([r for r in result_list if not r.success])<EOL>try:<EOL><INDENT>newres.ndim = len(result_list[<NUM_LIT:0>].x)<EOL><DEDENT>except TypeError:<EOL><INDENT>newres.ndim = <NUM_LIT:1><EOL><DEDENT>averaged_results[name] = newres<EOL><DEDENT>return averaged_results.values()<EOL>

group the results by minimizer and average over the runs


def bench_run(self, x0, **minimizer_kwargs):

kwargs = self.minimizer_kwargs<EOL>fonly_methods = ["<STR_LIT>", '<STR_LIT>']<EOL>for method in fonly_methods:<EOL><INDENT>t0 = time.time()<EOL>res = scipy.optimize.minimize(self.fun, x0, method=method,<EOL>**kwargs)<EOL>t1 = time.time()<EOL>self.add_result(res, t1-t0, method)<EOL><DEDENT>gradient_methods = ['<STR_LIT>', '<STR_LIT>', '<STR_LIT>', '<STR_LIT>', '<STR_LIT>']<EOL>if self.der is not None:<EOL><INDENT>for method in gradient_methods:<EOL><INDENT>t0 = time.time()<EOL>res = scipy.optimize.minimize(self.fun, x0, method=method,<EOL>jac=self.der, **kwargs)<EOL>t1 = time.time()<EOL>self.add_result(res, t1-t0, method)<EOL><DEDENT><DEDENT>hessian_methods = ["<STR_LIT>", '<STR_LIT>', '<STR_LIT>']<EOL>if self.hess is not None:<EOL><INDENT>for method in hessian_methods:<EOL><INDENT>t0 = time.time()<EOL>res = scipy.optimize.minimize(self.fun, x0, method=method,<EOL>jac=self.der, hess=self.hess,<EOL>**kwargs)<EOL>t1 = time.time()<EOL>self.add_result(res, t1-t0, method)<EOL><DEDENT><DEDENT>

do an optimization test starting at x0 for all the optimizers


results = self.average_results()<EOL>results = sorted(results, key=lambda x: (x.nfail, x.mean_time))<EOL>print("<STR_LIT>")<EOL>print("<STR_LIT>")<EOL>print(("<STR_LIT>" % (self.function_name)))<EOL>print(("<STR_LIT>" %<EOL>(results[<NUM_LIT:0>].ndim, str(self.minimizer_kwargs))))<EOL>print(("<STR_LIT>" %<EOL>(results[<NUM_LIT:0>].ntrials)))<EOL>print("<STR_LIT>")<EOL>print("<STR_LIT>")<EOL>for res in results:<EOL><INDENT>print(("<STR_LIT>" %<EOL>(res.name, res.nfail, res.mean_nfev, res.mean_njev,<EOL>res.mean_nhev, res.mean_time)))<EOL><DEDENT>


grouped_results = defaultdict(list)<EOL>for res in self.results:<EOL><INDENT>grouped_results[res.name].append(res)<EOL><DEDENT>averaged_results = dict()<EOL>for name, result_list in grouped_results.items():<EOL><INDENT>newres = OptimizeResult()<EOL>newres.name = name<EOL>newres.mean_nfev = np.mean([r.nfev for r in result_list])<EOL>newres.mean_njev = np.mean([r.njev for r in result_list])<EOL>newres.mean_nhev = np.mean([r.nhev for r in result_list])<EOL>newres.mean_time = np.mean([r.time for r in result_list])<EOL>newres.ntrials = len(result_list)<EOL>newres.nfail = len([r for r in result_list if not r.success])<EOL>try:<EOL><INDENT>newres.ndim = len(result_list[<NUM_LIT:0>].x)<EOL><DEDENT>except TypeError:<EOL><INDENT>newres.ndim = <NUM_LIT:1><EOL><DEDENT>averaged_results[name] = newres<EOL><DEDENT>return averaged_results.values()<EOL>


kwargs = self.minimizer_kwargs<EOL>if hasattr(self.fun, "<STR_LIT>"):<EOL><INDENT>kwargs["<STR_LIT:T>"] = self.function.temperature<EOL><DEDENT>if hasattr(self.fun, "<STR_LIT>"):<EOL><INDENT>kwargs["<STR_LIT>"] = self.function.stepsize<EOL><DEDENT>minimizer_kwargs = {"<STR_LIT>": "<STR_LIT>"}<EOL>x0 = self.get_random_configuration()<EOL>if hasattr(self.function, '<STR_LIT>'):<EOL><INDENT>minimizer_kwargs['<STR_LIT>'] = True<EOL>t0 = time.time()<EOL>res = basinhopping(<EOL>self.energy_gradient, x0, accept_test=self.accept_test,<EOL>callback=self.stop_criterion, niter=<NUM_LIT:1000>,<EOL>minimizer_kwargs=minimizer_kwargs,<EOL>**kwargs)<EOL>t1 = time.time()<EOL>res.success = True<EOL>if not self.found_target(res):<EOL><INDENT>res.success = False<EOL><DEDENT>self.add_result(res, t1 - t0, '<STR_LIT>')<EOL><DEDENT>x0 = self.get_random_configuration()<EOL>minimizer_kwargs['<STR_LIT>'] = False<EOL>t0 = time.time()<EOL>res = basinhopping(<EOL>self.fun, x0, accept_test=self.accept_test,<EOL>callback=self.stop_criterion, niter=<NUM_LIT:1000>,<EOL>minimizer_kwargs=minimizer_kwargs,<EOL>**kwargs)<EOL>t1 = time.time()<EOL>res.success = True<EOL>if not self.found_target(res):<EOL><INDENT>res.success = False<EOL><DEDENT>self.add_result(res, t1 - t0, '<STR_LIT>')<EOL>t0 = time.time()<EOL>res = differential_evolution(self.fun,<EOL>self.bounds,<EOL>popsize=<NUM_LIT:20>,<EOL>polish=True)<EOL>t1 = time.time()<EOL>if not self.found_target(res):<EOL><INDENT>res.success = False<EOL><DEDENT>self.add_result(res, t1 - t0, '<STR_LIT>')<EOL>


if x < <NUM_LIT:0>:<EOL><INDENT>return -<NUM_LIT:1.><EOL><DEDENT>elif x > <NUM_LIT:0>:<EOL><INDENT>return <NUM_LIT:1.><EOL><DEDENT>else:<EOL><INDENT>return <NUM_LIT:0.><EOL><DEDENT>


def line_search_wolfe1(f, fprime, xk, pk, gfk=None,<EOL>old_fval=None, old_old_fval=None,<EOL>args=(), c1=<NUM_LIT>, c2=<NUM_LIT>, amax=<NUM_LIT:50>, amin=<NUM_LIT>,<EOL>xtol=<NUM_LIT>):

if gfk is None:<EOL><INDENT>gfk = fprime(xk)<EOL><DEDENT>if isinstance(fprime, tuple):<EOL><INDENT>eps = fprime[<NUM_LIT:1>]<EOL>fprime = fprime[<NUM_LIT:0>]<EOL>newargs = (f, eps) + args<EOL>gradient = False<EOL><DEDENT>else:<EOL><INDENT>newargs = args<EOL>gradient = True<EOL><DEDENT>gval = [gfk]<EOL>gc = [<NUM_LIT:0>]<EOL>fc = [<NUM_LIT:0>]<EOL>def phi(s):<EOL><INDENT>fc[<NUM_LIT:0>] += <NUM_LIT:1><EOL>return f(xk + s*pk, *args)<EOL><DEDENT>def derphi(s):<EOL><INDENT>gval[<NUM_LIT:0>] = fprime(xk + s*pk, *newargs)<EOL>if gradient:<EOL><INDENT>gc[<NUM_LIT:0>] += <NUM_LIT:1><EOL><DEDENT>else:<EOL><INDENT>fc[<NUM_LIT:0>] += len(xk) + <NUM_LIT:1><EOL><DEDENT>return np.dot(gval[<NUM_LIT:0>], pk)<EOL><DEDENT>derphi0 = np.dot(gfk, pk)<EOL>stp, fval, old_fval = scalar_search_wolfe1(<EOL>phi, derphi, old_fval, old_old_fval, derphi0,<EOL>c1=c1, c2=c2, amax=amax, amin=amin, xtol=xtol)<EOL>return stp, fc[<NUM_LIT:0>], gc[<NUM_LIT:0>], fval, old_fval, gval[<NUM_LIT:0>]<EOL>

As `scalar_search_wolfe1` but do a line search to direction `pk`

Parameters
----------
f : callable
    Function `f(x)`
fprime : callable
    Gradient of `f`
xk : array_like
    Current point
pk : array_like
    Search direction

gfk : array_like, optional
    Gradient of `f` at point `xk`
old_fval : float, optional
    Value of `f` at point `xk`
old_old_fval : float, optional
    Value of `f` at point preceding `xk`

The rest of the parameters are the same as for `scalar_search_wolfe1`.

Returns
-------
stp, f_count, g_count, fval, old_fval
    As in `line_search_wolfe1`
gval : array
    Gradient of `f` at the final point


def scalar_search_wolfe1(phi, derphi, phi0=None, old_phi0=None, derphi0=None,<EOL>c1=<NUM_LIT>, c2=<NUM_LIT>,<EOL>amax=<NUM_LIT:50>, amin=<NUM_LIT>, xtol=<NUM_LIT>):

if phi0 is None:<EOL><INDENT>phi0 = phi(<NUM_LIT:0.>)<EOL><DEDENT>if derphi0 is None:<EOL><INDENT>derphi0 = derphi(<NUM_LIT:0.>)<EOL><DEDENT>if old_phi0 is not None and derphi0 != <NUM_LIT:0>:<EOL><INDENT>alpha1 = min(<NUM_LIT:1.0>, <NUM_LIT>*<NUM_LIT:2>*(phi0 - old_phi0)/derphi0)<EOL>if alpha1 < <NUM_LIT:0>:<EOL><INDENT>alpha1 = <NUM_LIT:1.0><EOL><DEDENT><DEDENT>else:<EOL><INDENT>alpha1 = <NUM_LIT:1.0><EOL><DEDENT>phi1 = phi0<EOL>derphi1 = derphi0<EOL>isave = np.zeros((<NUM_LIT:2>,), np.intc)<EOL>dsave = np.zeros((<NUM_LIT>,), float)<EOL>task = b'<STR_LIT>'<EOL>maxiter = <NUM_LIT:30><EOL>for i in xrange(maxiter):<EOL><INDENT>stp, phi1, derphi1, task = minpack2.dcsrch(alpha1, phi1, derphi1,<EOL>c1, c2, xtol, task,<EOL>amin, amax, isave, dsave)<EOL>if task[:<NUM_LIT:2>] == b'<STR_LIT>':<EOL><INDENT>alpha1 = stp<EOL>phi1 = phi(stp)<EOL>derphi1 = derphi(stp)<EOL><DEDENT>else:<EOL><INDENT>break<EOL><DEDENT><DEDENT>else:<EOL><INDENT>stp = None<EOL><DEDENT>if task[:<NUM_LIT:5>] == b'<STR_LIT>' or task[:<NUM_LIT:4>] == b'<STR_LIT>':<EOL><INDENT>stp = None  <EOL><DEDENT>return stp, phi1, phi0<EOL>

Scalar function search for alpha that satisfies strong Wolfe conditions

alpha > 0 is assumed to be a descent direction.

Parameters
----------
phi : callable phi(alpha)
    Function at point `alpha`
derphi : callable dphi(alpha)
    Derivative `d phi(alpha)/ds`. Returns a scalar.

phi0 : float, optional
    Value of `f` at 0
old_phi0 : float, optional
    Value of `f` at the previous point
derphi0 : float, optional
    Value `derphi` at 0
amax : float, optional
    Maximum step size
c1, c2 : float, optional
    Wolfe parameters

Returns
-------
alpha : float
    Step size, or None if no suitable step was found
phi : float
    Value of `phi` at the new point `alpha`
phi0 : float
    Value of `phi` at `alpha=0`

Notes
-----
Uses routine DCSRCH from MINPACK.


def line_search_wolfe2(f, myfprime, xk, pk, gfk=None, old_fval=None,<EOL>old_old_fval=None, args=(), c1=<NUM_LIT>, c2=<NUM_LIT>, amax=<NUM_LIT:50>):

fc = [<NUM_LIT:0>]<EOL>gc = [<NUM_LIT:0>]<EOL>gval = [None]<EOL>def phi(alpha):<EOL><INDENT>fc[<NUM_LIT:0>] += <NUM_LIT:1><EOL>return f(xk + alpha * pk, *args)<EOL><DEDENT>if isinstance(myfprime, tuple):<EOL><INDENT>def derphi(alpha):<EOL><INDENT>fc[<NUM_LIT:0>] += len(xk) + <NUM_LIT:1><EOL>eps = myfprime[<NUM_LIT:1>]<EOL>fprime = myfprime[<NUM_LIT:0>]<EOL>newargs = (f, eps) + args<EOL>gval[<NUM_LIT:0>] = fprime(xk + alpha * pk, *newargs)  <EOL>return np.dot(gval[<NUM_LIT:0>], pk)<EOL><DEDENT><DEDENT>else:<EOL><INDENT>fprime = myfprime<EOL>def derphi(alpha):<EOL><INDENT>gc[<NUM_LIT:0>] += <NUM_LIT:1><EOL>gval[<NUM_LIT:0>] = fprime(xk + alpha * pk, *args)  <EOL>return np.dot(gval[<NUM_LIT:0>], pk)<EOL><DEDENT><DEDENT>if gfk is None:<EOL><INDENT>gfk = fprime(xk, *args)<EOL><DEDENT>derphi0 = np.dot(gfk, pk)<EOL>alpha_star, phi_star, old_fval, derphi_star =scalar_search_wolfe2(phi, derphi, old_fval, old_old_fval,<EOL>derphi0, c1, c2, amax)<EOL>if derphi_star is not None:<EOL><INDENT>derphi_star = gval[<NUM_LIT:0>]<EOL><DEDENT>return alpha_star, fc[<NUM_LIT:0>], gc[<NUM_LIT:0>], phi_star, old_fval, derphi_star<EOL>

Find alpha that satisfies strong Wolfe conditions.

    Parameters
    ----------
    f : callable f(x,*args)
        Objective function.
    myfprime : callable f'(x,*args)
        Objective function gradient.
    xk : ndarray
        Starting point.
    pk : ndarray
        Search direction.
    gfk : ndarray, optional
        Gradient value for x=xk (xk being the current parameter
        estimate). Will be recomputed if omitted.
    old_fval : float, optional
        Function value for x=xk. Will be recomputed if omitted.
    old_old_fval : float, optional
        Function value for the point preceding x=xk
    args : tuple, optional
        Additional arguments passed to objective function.
    c1 : float, optional
        Parameter for Armijo condition rule.
    c2 : float, optional
        Parameter for curvature condition rule.

    Returns
    -------
    alpha : float
        Alpha for which ``x_new = x0 + alpha * pk``.
    fc : int
        Number of function evaluations made.
    gc : int
        Number of gradient evaluations made.
    new_fval : float
        New function value ``f(x_new)=f(x0+alpha*pk)``.
    old_fval : float
        Old function value ``f(x0)``.
    new_slope : float
        The local slope along the search direction at the
        new value ``<myfprime(x_new), pk>``.

    Notes
    -----
    Uses the line search algorithm to enforce strong Wolfe
    conditions.  See Wright and Nocedal, 'Numerical Optimization',
    1999, pg. 59-60.

    For the zoom phase it uses an algorithm by [...].


def scalar_search_wolfe2(phi, derphi=None, phi0=None,<EOL>old_phi0=None, derphi0=None,<EOL>c1=<NUM_LIT>, c2=<NUM_LIT>, amax=<NUM_LIT:50>):

if phi0 is None:<EOL><INDENT>phi0 = phi(<NUM_LIT:0.>)<EOL><DEDENT>if derphi0 is None and derphi is not None:<EOL><INDENT>derphi0 = derphi(<NUM_LIT:0.>)<EOL><DEDENT>alpha0 = <NUM_LIT:0><EOL>if old_phi0 is not None and derphi0 != <NUM_LIT:0>:<EOL><INDENT>alpha1 = min(<NUM_LIT:1.0>, <NUM_LIT>*<NUM_LIT:2>*(phi0 - old_phi0)/derphi0)<EOL><DEDENT>else:<EOL><INDENT>alpha1 = <NUM_LIT:1.0><EOL><DEDENT>if alpha1 < <NUM_LIT:0>:<EOL><INDENT>alpha1 = <NUM_LIT:1.0><EOL><DEDENT>if alpha1 == <NUM_LIT:0>:<EOL><INDENT>alpha_star = None<EOL>phi_star = phi0<EOL>phi0 = old_phi0<EOL>derphi_star = None<EOL><DEDENT>phi_a1 = phi(alpha1)<EOL>phi_a0 = phi0<EOL>derphi_a0 = derphi0<EOL>i = <NUM_LIT:1><EOL>maxiter = <NUM_LIT:10><EOL>for i in xrange(maxiter):<EOL><INDENT>if alpha1 == <NUM_LIT:0>:<EOL><INDENT>break<EOL><DEDENT>if (phi_a1 > phi0 + c1 * alpha1 * derphi0) or((phi_a1 >= phi_a0) and (i > <NUM_LIT:1>)):<EOL><INDENT>alpha_star, phi_star, derphi_star =_zoom(alpha0, alpha1, phi_a0,<EOL>phi_a1, derphi_a0, phi, derphi,<EOL>phi0, derphi0, c1, c2)<EOL>break<EOL><DEDENT>derphi_a1 = derphi(alpha1)<EOL>if (abs(derphi_a1) <= -c2*derphi0):<EOL><INDENT>alpha_star = alpha1<EOL>phi_star = phi_a1<EOL>derphi_star = derphi_a1<EOL>break<EOL><DEDENT>if (derphi_a1 >= <NUM_LIT:0>):<EOL><INDENT>alpha_star, phi_star, derphi_star =_zoom(alpha1, alpha0, phi_a1,<EOL>phi_a0, derphi_a1, phi, derphi,<EOL>phi0, derphi0, c1, c2)<EOL>break<EOL><DEDENT>alpha2 = <NUM_LIT:2> * alpha1   <EOL>i = i + <NUM_LIT:1><EOL>alpha0 = alpha1<EOL>alpha1 = alpha2<EOL>phi_a0 = phi_a1<EOL>phi_a1 = phi(alpha1)<EOL>derphi_a0 = derphi_a1<EOL><DEDENT>else:<EOL><INDENT>alpha_star = alpha1<EOL>phi_star = phi_a1<EOL>derphi_star = None<EOL><DEDENT>return alpha_star, phi_star, phi0, derphi_star<EOL>

Find alpha that satisfies strong Wolfe conditions.

    alpha > 0 is assumed to be a descent direction.

    Parameters
    ----------
    phi : callable f(x,*args)
        Objective scalar function.

    derphi : callable f'(x,*args), optional
        Objective function derivative (can be None)
    phi0 : float, optional
        Value of phi at s=0
    old_phi0 : float, optional
        Value of phi at previous point
    derphi0 : float, optional
        Value of derphi at s=0
    args : tuple
        Additional arguments passed to objective function.
    c1 : float
        Parameter for Armijo condition rule.
    c2 : float
        Parameter for curvature condition rule.

    Returns
    -------
    alpha_star : float
        Best alpha
    phi_star
        phi at alpha_star
    phi0
        phi at 0
    derphi_star
        derphi at alpha_star

    Notes
    -----
    Uses the line search algorithm to enforce strong Wolfe
    conditions.  See Wright and Nocedal, 'Numerical Optimization',
    1999, pg. 59-60.

    For the zoom phase it uses an algorithm by [...].


def _cubicmin(a, fa, fpa, b, fb, c, fc):

<EOL>with np.errstate(divide='<STR_LIT>', over='<STR_LIT>', invalid='<STR_LIT>'):<EOL><INDENT>try:<EOL><INDENT>C = fpa<EOL>db = b - a<EOL>dc = c - a<EOL>denom = (db * dc) ** <NUM_LIT:2> * (db - dc)<EOL>d1 = np.empty((<NUM_LIT:2>, <NUM_LIT:2>))<EOL>d1[<NUM_LIT:0>, <NUM_LIT:0>] = dc ** <NUM_LIT:2><EOL>d1[<NUM_LIT:0>, <NUM_LIT:1>] = -db ** <NUM_LIT:2><EOL>d1[<NUM_LIT:1>, <NUM_LIT:0>] = -dc ** <NUM_LIT:3><EOL>d1[<NUM_LIT:1>, <NUM_LIT:1>] = db ** <NUM_LIT:3><EOL>[A, B] = np.dot(d1, np.asarray([fb - fa - C * db,<EOL>fc - fa - C * dc]).flatten())<EOL>A /= denom<EOL>B /= denom<EOL>radical = B * B - <NUM_LIT:3> * A * C<EOL>xmin = a + (-B + np.sqrt(radical)) / (<NUM_LIT:3> * A)<EOL><DEDENT>except ArithmeticError:<EOL><INDENT>return None<EOL><DEDENT><DEDENT>if not np.isfinite(xmin):<EOL><INDENT>return None<EOL><DEDENT>return xmin<EOL>

Finds the minimizer for a cubic polynomial that goes through the
points (a,fa), (b,fb), and (c,fc) with derivative at a of fpa.

If no minimizer can be found return None


<EOL>with np.errstate(divide='<STR_LIT>', over='<STR_LIT>', invalid='<STR_LIT>'):<EOL><INDENT>try:<EOL><INDENT>D = fa<EOL>C = fpa<EOL>db = b - a * <NUM_LIT:1.0><EOL>B = (fb - D - C * db) / (db * db)<EOL>xmin = a - C / (<NUM_LIT> * B)<EOL><DEDENT>except ArithmeticError:<EOL><INDENT>return None<EOL><DEDENT><DEDENT>if not np.isfinite(xmin):<EOL><INDENT>return None<EOL><DEDENT>return xmin<EOL>

Finds the minimizer for a quadratic polynomial that goes through
the points (a,fa), (b,fb) with derivative at a of fpa,


def _zoom(a_lo, a_hi, phi_lo, phi_hi, derphi_lo,<EOL>phi, derphi, phi0, derphi0, c1, c2):

maxiter = <NUM_LIT:10><EOL>i = <NUM_LIT:0><EOL>delta1 = <NUM_LIT>  <EOL>delta2 = <NUM_LIT:0.1>  <EOL>phi_rec = phi0<EOL>a_rec = <NUM_LIT:0><EOL>while True:<EOL><INDENT>dalpha = a_hi - a_lo<EOL>if dalpha < <NUM_LIT:0>:<EOL><INDENT>a, b = a_hi, a_lo<EOL><DEDENT>else:<EOL><INDENT>a, b = a_lo, a_hi<EOL><DEDENT>if (i > <NUM_LIT:0>):<EOL><INDENT>cchk = delta1 * dalpha<EOL>a_j = _cubicmin(a_lo, phi_lo, derphi_lo, a_hi, phi_hi,<EOL>a_rec, phi_rec)<EOL><DEDENT>if (i == <NUM_LIT:0>) or (a_j is None) or (a_j > b - cchk) or (a_j < a + cchk):<EOL><INDENT>qchk = delta2 * dalpha<EOL>a_j = _quadmin(a_lo, phi_lo, derphi_lo, a_hi, phi_hi)<EOL>if (a_j is None) or (a_j > b-qchk) or (a_j < a+qchk):<EOL><INDENT>a_j = a_lo + <NUM_LIT:0.5>*dalpha<EOL><DEDENT><DEDENT>phi_aj = phi(a_j)<EOL>if (phi_aj > phi0 + c1*a_j*derphi0) or (phi_aj >= phi_lo):<EOL><INDENT>phi_rec = phi_hi<EOL>a_rec = a_hi<EOL>a_hi = a_j<EOL>phi_hi = phi_aj<EOL><DEDENT>else:<EOL><INDENT>derphi_aj = derphi(a_j)<EOL>if abs(derphi_aj) <= -c2*derphi0:<EOL><INDENT>a_star = a_j<EOL>val_star = phi_aj<EOL>valprime_star = derphi_aj<EOL>break<EOL><DEDENT>if derphi_aj*(a_hi - a_lo) >= <NUM_LIT:0>:<EOL><INDENT>phi_rec = phi_hi<EOL>a_rec = a_hi<EOL>a_hi = a_lo<EOL>phi_hi = phi_lo<EOL><DEDENT>else:<EOL><INDENT>phi_rec = phi_lo<EOL>a_rec = a_lo<EOL><DEDENT>a_lo = a_j<EOL>phi_lo = phi_aj<EOL>derphi_lo = derphi_aj<EOL><DEDENT>i += <NUM_LIT:1><EOL>if (i > maxiter):<EOL><INDENT>a_star = None<EOL>val_star = None<EOL>valprime_star = None<EOL>break<EOL><DEDENT><DEDENT>return a_star, val_star, valprime_star<EOL>

Part of the optimization algorithm in `scalar_search_wolfe2`.


def line_search_armijo(f, xk, pk, gfk, old_fval, args=(), c1=<NUM_LIT>, alpha0=<NUM_LIT:1>):

xk = np.atleast_1d(xk)<EOL>fc = [<NUM_LIT:0>]<EOL>def phi(alpha1):<EOL><INDENT>fc[<NUM_LIT:0>] += <NUM_LIT:1><EOL>return f(xk + alpha1*pk, *args)<EOL><DEDENT>if old_fval is None:<EOL><INDENT>phi0 = phi(<NUM_LIT:0.>)<EOL><DEDENT>else:<EOL><INDENT>phi0 = old_fval  <EOL><DEDENT>derphi0 = np.dot(gfk, pk)<EOL>alpha, phi1 = scalar_search_armijo(phi, phi0, derphi0, c1=c1,<EOL>alpha0=alpha0)<EOL>return alpha, fc[<NUM_LIT:0>], phi1<EOL>

Minimize over alpha, the function ``f(xk+alpha pk)``.

    Parameters
    ----------
    f : callable
        Function to be minimized.
    xk : array_like
        Current point.
    pk : array_like
        Search direction.
    gfk : array_like
        Gradient of `f` at point `xk`.
    old_fval : float
        Value of `f` at point `xk`.
    args : tuple, optional
        Optional arguments.
    c1 : float, optional
        Value to control stopping criterion.
    alpha0 : scalar, optional
        Value of `alpha` at start of the optimization.

    Returns
    -------
    alpha
    f_count
    f_val_at_alpha

    Notes
    -----
    Uses the interpolation algorithm (Armijo backtracking) as suggested by
    Wright and Nocedal in 'Numerical Optimization', 1999, pg. 56-57


def line_search_BFGS(f, xk, pk, gfk, old_fval, args=(), c1=<NUM_LIT>, alpha0=<NUM_LIT:1>):

r = line_search_armijo(f, xk, pk, gfk, old_fval, args=args, c1=c1,<EOL>alpha0=alpha0)<EOL>return r[<NUM_LIT:0>], r[<NUM_LIT:1>], <NUM_LIT:0>, r[<NUM_LIT:2>]<EOL>

Compatibility wrapper for `line_search_armijo`


def scalar_search_armijo(phi, phi0, derphi0, c1=<NUM_LIT>, alpha0=<NUM_LIT:1>, amin=<NUM_LIT:0>):

phi_a0 = phi(alpha0)<EOL>if phi_a0 <= phi0 + c1*alpha0*derphi0:<EOL><INDENT>return alpha0, phi_a0<EOL><DEDENT>alpha1 = -(derphi0) * alpha0**<NUM_LIT:2> / <NUM_LIT> / (phi_a0 - phi0 - derphi0 * alpha0)<EOL>phi_a1 = phi(alpha1)<EOL>if (phi_a1 <= phi0 + c1*alpha1*derphi0):<EOL><INDENT>return alpha1, phi_a1<EOL><DEDENT>while alpha1 > amin:       <EOL><INDENT>factor = alpha0**<NUM_LIT:2> * alpha1**<NUM_LIT:2> * (alpha1-alpha0)<EOL>a = alpha0**<NUM_LIT:2> * (phi_a1 - phi0 - derphi0*alpha1) -alpha1**<NUM_LIT:2> * (phi_a0 - phi0 - derphi0*alpha0)<EOL>a = a / factor<EOL>b = -alpha0**<NUM_LIT:3> * (phi_a1 - phi0 - derphi0*alpha1) +alpha1**<NUM_LIT:3> * (phi_a0 - phi0 - derphi0*alpha0)<EOL>b = b / factor<EOL>alpha2 = (-b + np.sqrt(abs(b**<NUM_LIT:2> - <NUM_LIT:3> * a * derphi0))) / (<NUM_LIT>*a)<EOL>phi_a2 = phi(alpha2)<EOL>if (phi_a2 <= phi0 + c1*alpha2*derphi0):<EOL><INDENT>return alpha2, phi_a2<EOL><DEDENT>if (alpha1 - alpha2) > alpha1 / <NUM_LIT> or (<NUM_LIT:1> - alpha2/alpha1) < <NUM_LIT>:<EOL><INDENT>alpha2 = alpha1 / <NUM_LIT><EOL><DEDENT>alpha0 = alpha1<EOL>alpha1 = alpha2<EOL>phi_a0 = phi_a1<EOL>phi_a1 = phi_a2<EOL><DEDENT>return None, phi_a1<EOL>

Minimize over alpha, the function ``phi(alpha)``.

    Uses the interpolation algorithm (Armijo backtracking) as suggested by
    Wright and Nocedal in 'Numerical Optimization', 1999, pg. 56-57

    alpha > 0 is assumed to be a descent direction.

    Returns
    -------
    alpha
    phi1


@deprecate(message='<STR_LIT>')<EOL>def anneal(func, x0, args=(), schedule='<STR_LIT>', full_output=<NUM_LIT:0>,<EOL>T0=None, Tf=<NUM_LIT>, maxeval=None, maxaccept=None, maxiter=<NUM_LIT>,<EOL>boltzmann=<NUM_LIT:1.0>, learn_rate=<NUM_LIT:0.5>, feps=<NUM_LIT>, quench=<NUM_LIT:1.0>, m=<NUM_LIT:1.0>, n=<NUM_LIT:1.0>,<EOL>lower=-<NUM_LIT:100>, upper=<NUM_LIT:100>, dwell=<NUM_LIT:50>, disp=True):

opts = {'<STR_LIT>': schedule,<EOL>'<STR_LIT>': T0,<EOL>'<STR_LIT>': Tf,<EOL>'<STR_LIT>': maxeval,<EOL>'<STR_LIT>': maxaccept,<EOL>'<STR_LIT>': maxiter,<EOL>'<STR_LIT>': boltzmann,<EOL>'<STR_LIT>': learn_rate,<EOL>'<STR_LIT>': feps,<EOL>'<STR_LIT>': quench,<EOL>'<STR_LIT:m>': m,<EOL>'<STR_LIT:n>': n,<EOL>'<STR_LIT>': lower,<EOL>'<STR_LIT>': upper,<EOL>'<STR_LIT>': dwell,<EOL>'<STR_LIT>': disp}<EOL>res = _minimize_anneal(func, x0, args, **opts)<EOL>if full_output:<EOL><INDENT>return res['<STR_LIT:x>'], res['<STR_LIT>'], res['<STR_LIT:T>'], res['<STR_LIT>'], res['<STR_LIT>'],res['<STR_LIT>'], res['<STR_LIT:status>']<EOL><DEDENT>else:<EOL><INDENT>return res['<STR_LIT:x>'], res['<STR_LIT:status>']<EOL><DEDENT>

Minimize a function using simulated annealing.

Uses simulated annealing, a random algorithm that uses no derivative
information from the function being optimized. Other names for this
family of approaches include: "Monte Carlo", "Metropolis",
"Metropolis-Hastings", `etc`. They all involve (a) evaluating the
objective function on a random set of points, (b) keeping those that
pass their randomized evaluation critera, (c) cooling (`i.e.`,
tightening) the evaluation critera, and (d) repeating until their
termination critera are met.  In practice they have been used mainly in
discrete rather than in continuous optimization.

Available annealing schedules are 'fast', 'cauchy' and 'boltzmann'.

Parameters
----------
func : callable
    The objective function to be minimized.  Must be in the form
    `f(x, *args)`, where `x` is the argument in the form of a 1-D array
    and `args` is a  tuple of any additional fixed parameters needed to
    completely specify the function.
x0: 1-D array
    An initial guess at the optimizing argument of `func`.
args : tuple, optional
    Any additional fixed parameters needed to completely
    specify the objective function.
schedule : str, optional
    The annealing schedule to use.  Must be one of 'fast', 'cauchy' or
    'boltzmann'.  See `Notes`.
full_output : bool, optional
    If `full_output`, then return all values listed in the Returns
    section. Otherwise, return just the `xmin` and `status` values.
T0 : float, optional
    The initial "temperature".  If None, then estimate it as 1.2 times
    the largest cost-function deviation over random points in the
    box-shaped region specified by the `lower, upper` input parameters.
Tf : float, optional
    Final goal temperature.  Cease iterations if the temperature
    falls below `Tf`.
maxeval : int, optional
    Cease iterations if the number of function evaluations exceeds
    `maxeval`.
maxaccept : int, optional
    Cease iterations if the number of points accepted exceeds `maxaccept`.
    See `Notes` for the probabilistic acceptance criteria used.
maxiter : int, optional
    Cease iterations if the number of cooling iterations exceeds `maxiter`.
learn_rate : float, optional
    Scale constant for tuning the probabilistc acceptance criteria.
boltzmann : float, optional
    Boltzmann constant in the probabilistic acceptance criteria
    (increase for less stringent criteria at each temperature).
feps : float, optional
    Cease iterations if the relative errors in the function value over the
    last four coolings is below `feps`.
quench, m, n : floats, optional
    Parameters to alter the `fast` simulated annealing schedule.
    See `Notes`.
lower, upper : floats or 1-D arrays, optional
    Lower and upper bounds on the argument `x`.  If floats are provided,
    they apply to all components of `x`.
dwell : int, optional
    The number of times to execute the inner loop at each value of the
    temperature.  See `Notes`.
disp : bool, optional
    Print a descriptive convergence message if True.

Returns
-------
xmin : ndarray
    The point where the lowest function value was found.
Jmin : float
    The objective function value at `xmin`.
T : float
    The temperature at termination of the iterations.
feval : int
    Number of function evaluations used.
iters : int
    Number of cooling iterations used.
accept : int
    Number of tests accepted.
status : int
    A code indicating the reason for termination:

    - 0 : Points no longer changing.
    - 1 : Cooled to final temperature.
    - 2 : Maximum function evaluations reached.
    - 3 : Maximum cooling iterations reached.
    - 4 : Maximum accepted query locations reached.
    - 5 : Final point not the minimum amongst encountered points.

See Also
--------
basinhopping : another (more performant) global optimizer
brute : brute-force global optimizer

Notes
-----
Simulated annealing is a random algorithm which uses no derivative
information from the function being optimized. In practice it has
been more useful in discrete optimization than continuous
optimization, as there are usually better algorithms for continuous
optimization problems.

Some experimentation by trying the different temperature
schedules and altering their parameters is likely required to
obtain good performance.

The randomness in the algorithm comes from random sampling in numpy.
To obtain the same results you can call `numpy.random.seed` with the
same seed immediately before calling `anneal`.

We give a brief description of how the three temperature schedules
generate new points and vary their temperature.  Temperatures are
only updated with iterations in the outer loop.  The inner loop is
over loop over ``xrange(dwell)``, and new points are generated for
every iteration in the inner loop.  Whether the proposed new points
are accepted is probabilistic.

For readability, let ``d`` denote the dimension of the inputs to func.
Also, let ``x_old`` denote the previous state, and ``k`` denote the
iteration number of the outer loop.  All other variables not
defined below are input variables to `anneal` itself.

In the 'fast' schedule the updates are::

    u ~ Uniform(0, 1, size = d)
    y = sgn(u - 0.5) * T * ((1 + 1/T)**abs(2*u - 1) - 1.0)

    xc = y * (upper - lower)
    x_new = x_old + xc

    c = n * exp(-n * quench)
    T_new = T0 * exp(-c * k**quench)

In the 'cauchy' schedule the updates are::

    u ~ Uniform(-pi/2, pi/2, size=d)
    xc = learn_rate * T * tan(u)
    x_new = x_old + xc

    T_new = T0 / (1 + k)

In the 'boltzmann' schedule the updates are::

    std = minimum(sqrt(T) * ones(d), (upper - lower) / (3*learn_rate))
    y ~ Normal(0, std, size = d)
    x_new = x_old + learn_rate * y

    T_new = T0 / log(1 + k)

References
----------
[1] P. J. M. van Laarhoven and E. H. L. Aarts, "Simulated Annealing: Theory
    and Applications", Kluwer Academic Publishers, 1987.

[2] W.H. Press et al., "Numerical Recipies: The Art of Scientific Computing",
    Cambridge U. Press, 1987.

Examples
--------
*Example 1.* We illustrate the use of `anneal` to seek the global minimum
of a function of two variables that is equal to the sum of a positive-
definite quadratic and two deep "Gaussian-shaped" craters.  Specifically,
define the objective function `f` as the sum of three other functions,
``f = f1 + f2 + f3``.  We suppose each of these has a signature
``(z, *params)``, where ``z = (x, y)``, ``params``, and the functions are
as defined below.

>>> params = (2, 3, 7, 8, 9, 10, 44, -1, 2, 26, 1, -2, 0.5)
>>> def f1(z, *params):
...     x, y = z
...     a, b, c, d, e, f, g, h, i, j, k, l, scale = params
...     return (a * x**2 + b * x * y + c * y**2 + d*x + e*y + f)

>>> def f2(z, *params):
...     x, y = z
...     a, b, c, d, e, f, g, h, i, j, k, l, scale = params
...     return (-g*np.exp(-((x-h)**2 + (y-i)**2) / scale))

>>> def f3(z, *params):
...     x, y = z
...     a, b, c, d, e, f, g, h, i, j, k, l, scale = params
...     return (-j*np.exp(-((x-k)**2 + (y-l)**2) / scale))

>>> def f(z, *params):
...     x, y = z
...     a, b, c, d, e, f, g, h, i, j, k, l, scale = params
...     return f1(z, *params) + f2(z, *params) + f3(z, *params)

>>> x0 = np.array([2., 2.])     # Initial guess.
>>> from scipy import optimize
>>> np.random.seed(555)   # Seeded to allow replication.
>>> res = optimize.anneal(f, x0, args=params, schedule='boltzmann',
                          full_output=True, maxiter=500, lower=-10,
                          upper=10, dwell=250, disp=True)
Warning: Maximum number of iterations exceeded.
>>> res[0]  # obtained minimum
array([-1.03914194,  1.81330654])
>>> res[1]  # function value at minimum
-3.3817...

So this run settled on the point [-1.039, 1.813] with a minimum function
value of about -3.382.  The final temperature was about 212. The run used
125301 function evaluations, 501 iterations (including the initial guess as
a iteration), and accepted 61162 points. The status flag of 3 also
indicates that `maxiter` was reached.

This problem's true global minimum lies near the point [-1.057, 1.808]
and has a value of about -3.409.  So these `anneal` results are pretty
good and could be used as the starting guess in a local optimizer to
seek a more exact local minimum.

*Example 2.* To minimize the same objective function using
the `minimize` approach, we need to (a) convert the options to an
"options dictionary" using the keys prescribed for this method,
(b) call the `minimize` function with the name of the method (which
in this case is 'Anneal'), and (c) take account of the fact that
the returned value will be a `OptimizeResult` object (`i.e.`, a dictionary,
as defined in `optimize.py`).

All of the allowable options for 'Anneal' when using the `minimize`
approach are listed in the ``myopts`` dictionary given below, although
in practice only the non-default values would be needed.  Some of their
names differ from those used in the `anneal` approach.  We can proceed
as follows:

>>> myopts = {
        'schedule'     : 'boltzmann',   # Non-default value.
        'maxfev'       : None,  # Default, formerly `maxeval`.
        'maxiter'      : 500,   # Non-default value.
        'maxaccept'    : None,  # Default value.
        'ftol'         : 1e-6,  # Default, formerly `feps`.
        'T0'           : None,  # Default value.
        'Tf'           : 1e-12, # Default value.
        'boltzmann'    : 1.0,   # Default value.
        'learn_rate'   : 0.5,   # Default value.
        'quench'       : 1.0,   # Default value.
        'm'            : 1.0,   # Default value.
        'n'            : 1.0,   # Default value.
        'lower'        : -10,   # Non-default value.
        'upper'        : +10,   # Non-default value.
        'dwell'        : 250,   # Non-default value.
        'disp'         : True   # Default value.
        }
>>> from scipy import optimize
>>> np.random.seed(777)  # Seeded to allow replication.
>>> res2 = optimize.minimize(f, x0, args=params, method='Anneal',
                             options=myopts)
Warning: Maximum number of iterations exceeded.
>>> res2
  status: 3
 success: False
  accept: 61742
    nfev: 125301
       T: 214.20624873839623
     fun: -3.4084065576676053
       x: array([-1.05757366,  1.8071427 ])
 message: 'Maximum cooling iterations reached'
 nit: 501


def _minimize_anneal(func, x0, args=(),<EOL>schedule='<STR_LIT>', T0=None, Tf=<NUM_LIT>, maxfev=None,<EOL>maxaccept=None, maxiter=<NUM_LIT>, boltzmann=<NUM_LIT:1.0>,<EOL>learn_rate=<NUM_LIT:0.5>, ftol=<NUM_LIT>, quench=<NUM_LIT:1.0>, m=<NUM_LIT:1.0>, n=<NUM_LIT:1.0>,<EOL>lower=-<NUM_LIT:100>, upper=<NUM_LIT:100>, dwell=<NUM_LIT:50>, disp=False,<EOL>**unknown_options):

_check_unknown_options(unknown_options)<EOL>maxeval = maxfev<EOL>feps = ftol<EOL>x0 = asarray(x0)<EOL>lower = asarray(lower)<EOL>upper = asarray(upper)<EOL>schedule = eval(schedule+'<STR_LIT>')<EOL>schedule.init(dims=shape(x0), func=func, args=args, boltzmann=boltzmann,<EOL>T0=T0, learn_rate=learn_rate, lower=lower, upper=upper,<EOL>m=m, n=n, quench=quench, dwell=dwell)<EOL>current_state, last_state, best_state = _state(), _state(), _state()<EOL>if T0 is None:<EOL><INDENT>x0 = schedule.getstart_temp(best_state)<EOL><DEDENT>else:<EOL><INDENT>best_state.x = None<EOL>best_state.cost = numpy.Inf<EOL><DEDENT>last_state.x = asarray(x0).copy()<EOL>fval = func(x0, *args)<EOL>schedule.feval += <NUM_LIT:1><EOL>last_state.cost = fval<EOL>if last_state.cost < best_state.cost:<EOL><INDENT>best_state.cost = fval<EOL>best_state.x = asarray(x0).copy()<EOL><DEDENT>schedule.T = schedule.T0<EOL>fqueue = [<NUM_LIT:100>, <NUM_LIT>, <NUM_LIT>, <NUM_LIT>]<EOL>iters = <NUM_LIT:0><EOL>while <NUM_LIT:1>:<EOL><INDENT>for n in xrange(dwell):<EOL><INDENT>current_state.x = schedule.update_guess(last_state.x)<EOL>current_state.cost = func(current_state.x, *args)<EOL>schedule.feval += <NUM_LIT:1><EOL>dE = current_state.cost - last_state.cost<EOL>if schedule.accept_test(dE):<EOL><INDENT>last_state.x = current_state.x.copy()<EOL>last_state.cost = current_state.cost<EOL>if last_state.cost < best_state.cost:<EOL><INDENT>best_state.x = last_state.x.copy()<EOL>best_state.cost = last_state.cost<EOL><DEDENT><DEDENT><DEDENT>schedule.update_temp()<EOL>iters += <NUM_LIT:1><EOL>fqueue.append(squeeze(last_state.cost))<EOL>fqueue.pop(<NUM_LIT:0>)<EOL>af = asarray(fqueue)*<NUM_LIT:1.0><EOL>if all(abs((af-af[<NUM_LIT:0>])/af[<NUM_LIT:0>]) < feps):<EOL><INDENT>retval = <NUM_LIT:0><EOL>if abs(af[-<NUM_LIT:1>]-best_state.cost) > feps*<NUM_LIT:10>:<EOL><INDENT>retval = <NUM_LIT:5><EOL>if disp:<EOL><INDENT>print("<STR_LIT>"<EOL>% (squeeze(last_state.cost),<EOL>str(squeeze(last_state.x)))<EOL>+ "<STR_LIT>")<EOL><DEDENT><DEDENT>break<EOL><DEDENT>if (Tf is not None) and (schedule.T < Tf):<EOL><INDENT>retval = <NUM_LIT:1><EOL>break<EOL><DEDENT>if (maxeval is not None) and (schedule.feval > maxeval):<EOL><INDENT>retval = <NUM_LIT:2><EOL>break<EOL><DEDENT>if (iters > maxiter):<EOL><INDENT>if disp:<EOL><INDENT>print("<STR_LIT>")<EOL><DEDENT>retval = <NUM_LIT:3><EOL>break<EOL><DEDENT>if (maxaccept is not None) and (schedule.accepted > maxaccept):<EOL><INDENT>retval = <NUM_LIT:4><EOL>break<EOL><DEDENT><DEDENT>result = OptimizeResult(x=best_state.x, fun=best_state.cost,<EOL>T=schedule.T, nfev=schedule.feval, nit=iters,<EOL>accept=schedule.accepted, status=retval,<EOL>success=(retval <= <NUM_LIT:1>),<EOL>message={<NUM_LIT:0>: '<STR_LIT>',<EOL><NUM_LIT:1>: '<STR_LIT>',<EOL><NUM_LIT:2>: '<STR_LIT>',<EOL><NUM_LIT:3>: '<STR_LIT>',<EOL><NUM_LIT:4>: '<STR_LIT>',<EOL><NUM_LIT:5>: '<STR_LIT>'<EOL>'<STR_LIT>'}[retval])<EOL>return result<EOL>

Minimization of scalar function of one or more variables using the
simulated annealing algorithm.

Options for the simulated annealing algorithm are:
    disp : bool
        Set to True to print convergence messages.
    schedule : str
        Annealing schedule to use. One of: 'fast', 'cauchy' or
        'boltzmann'.
    T0 : float
        Initial Temperature (estimated as 1.2 times the largest
        cost-function deviation over random points in the range).
    Tf : float
        Final goal temperature.
    maxfev : int
        Maximum number of function evaluations to make.
    maxaccept : int
        Maximum changes to accept.
    maxiter : int
        Maximum number of iterations to perform.
    boltzmann : float
        Boltzmann constant in acceptance test (increase for less
        stringent test at each temperature).
    learn_rate : float
        Scale constant for adjusting guesses.
    ftol : float
        Relative error in ``fun(x)`` acceptable for convergence.
    quench, m, n : float
        Parameters to alter fast_sa schedule.
    lower, upper : float or ndarray
        Lower and upper bounds on `x`.
    dwell : int
        The number of times to search the space at each temperature.

This function is called by the `minimize` function with
`method=anneal`. It is not supposed to be called directly.


assert(self.dims is not None)<EOL>lrange = self.lower<EOL>urange = self.upper<EOL>fmax = _double_min<EOL>fmin = _double_max<EOL>for _ in range(self.Ninit):<EOL><INDENT>x0 = random.uniform(size=self.dims)*(urange-lrange) + lrange<EOL>fval = self.func(x0, *self.args)<EOL>self.feval += <NUM_LIT:1><EOL>if fval > fmax:<EOL><INDENT>fmax = fval<EOL><DEDENT>if fval < fmin:<EOL><INDENT>fmin = fval<EOL>best_state.cost = fval<EOL>best_state.x = array(x0)<EOL><DEDENT><DEDENT>self.T0 = (fmax-fmin)*<NUM_LIT><EOL>return best_state.x<EOL>

Find a matching starting temperature and starting parameters vector
        i.e. find x0 such that func(x0) = T0.

        Parameters
        ----------
        best_state : _state
            A _state object to store the function value and x0 found.

        Returns
        -------
        x0 : array
            The starting parameters vector.


tableau = kwargs["<STR_LIT>"]<EOL>nit = kwargs["<STR_LIT>"]<EOL>pivrow, pivcol = kwargs["<STR_LIT>"]<EOL>phase = kwargs["<STR_LIT>"]<EOL>basis = kwargs["<STR_LIT>"]<EOL>complete = kwargs["<STR_LIT>"]<EOL>saved_printoptions = np.get_printoptions()<EOL>np.set_printoptions(linewidth=<NUM_LIT>,<EOL>formatter={'<STR_LIT:float>':lambda x: "<STR_LIT>".format(x)})<EOL>if complete:<EOL><INDENT>print("<STR_LIT>".format(phase))<EOL>print("<STR_LIT>")<EOL><DEDENT>elif nit == <NUM_LIT:0>:<EOL><INDENT>print("<STR_LIT>".format(phase))<EOL><DEDENT>else:<EOL><INDENT>print("<STR_LIT>".format(nit, phase))<EOL>print("<STR_LIT>")<EOL><DEDENT>if nit >= <NUM_LIT:0>:<EOL><INDENT>print("<STR_LIT>" + str(tableau) + "<STR_LIT:\n>")<EOL>if not complete:<EOL><INDENT>print("<STR_LIT>".format(pivrow, pivcol))<EOL><DEDENT>print("<STR_LIT>", basis)<EOL>print()<EOL>print("<STR_LIT>")<EOL>print("<STR_LIT>", xk)<EOL>print()<EOL>print("<STR_LIT>")<EOL>print("<STR_LIT>", -tableau[-<NUM_LIT:1>, -<NUM_LIT:1>])<EOL>print()<EOL><DEDENT>np.set_printoptions(**saved_printoptions)<EOL>

A sample callback function demonstrating the linprog callback interface.
This callback produces detailed output to sys.stdout before each iteration
and after the final iteration of the simplex algorithm.

Parameters
----------
xk : array_like
    The current solution vector.
**kwargs : dict
    A dictionary containing the following parameters:

    tableau : array_like
        The current tableau of the simplex algorithm.
        Its structure is defined in _solve_simplex.
    phase : int
        The current Phase of the simplex algorithm (1 or 2)
    nit : int
        The current iteration number.
    pivot : tuple(int, int)
        The index of the tableau selected as the next pivot,
        or nan if no pivot exists
    basis : array(int)
        A list of the current basic variables.
        Each element contains the name of a basic variable and its value.
    complete : bool
        True if the simplex algorithm has completed
        (and this is the final call to callback), otherwise False.


nit = kwargs["<STR_LIT>"]<EOL>if nit == <NUM_LIT:0>:<EOL><INDENT>print("<STR_LIT>")<EOL><DEDENT>print("<STR_LIT>".format(nit), end="<STR_LIT>")<EOL>print(xk)<EOL>

A sample callback function demonstrating the linprog callback interface.
This callback produces brief output to sys.stdout before each iteration
and after the final iteration of the simplex algorithm.

Parameters
----------
xk : array_like
    The current solution vector.
**kwargs : dict
    A dictionary containing the following parameters:

    tableau : array_like
        The current tableau of the simplex algorithm.
        Its structure is defined in _solve_simplex.
    vars : tuple(str, ...)
        Column headers for each column in tableau.
        "x[i]" for actual variables, "s[i]" for slack surplus variables,
        "a[i]" for artificial variables, and "RHS" for the constraint
        RHS vector.
    phase : int
        The current Phase of the simplex algorithm (1 or 2)
    nit : int
        The current iteration number.
    pivot : tuple(int, int)
        The index of the tableau selected as the next pivot,
        or nan if no pivot exists
    basics : list[tuple(int, float)]
        A list of the current basic variables.
        Each element contains the index of a basic variable and
        its value.
    complete : bool
        True if the simplex algorithm has completed
        (and this is the final call to callback), otherwise False.


def _pivot_col(T, tol=<NUM_LIT>, bland=False):

ma = np.ma.masked_where(T[-<NUM_LIT:1>, :-<NUM_LIT:1>] >= -tol, T[-<NUM_LIT:1>, :-<NUM_LIT:1>], copy=False)<EOL>if ma.count() == <NUM_LIT:0>:<EOL><INDENT>return False, np.nan<EOL><DEDENT>if bland:<EOL><INDENT>return True, np.where(ma.mask == False)[<NUM_LIT:0>][<NUM_LIT:0>]<EOL><DEDENT>return True, np.ma.where(ma == ma.min())[<NUM_LIT:0>][<NUM_LIT:0>]<EOL>

Given a linear programming simplex tableau, determine the column
of the variable to enter the basis.

Parameters
----------
T : 2D ndarray
    The simplex tableau.
tol : float
    Elements in the objective row larger than -tol will not be considered
    for pivoting.  Nominally this value is zero, but numerical issues
    cause a tolerance about zero to be necessary.
bland : bool
    If True, use Bland's rule for selection of the column (select the
    first column with a negative coefficient in the objective row,
    regardless of magnitude).

Returns
-------
status: bool
    True if a suitable pivot column was found, otherwise False.
    A return of False indicates that the linear programming simplex
    algorithm is complete.
col: int
    The index of the column of the pivot element.
    If status is False, col will be returned as nan.


def _pivot_row(T, pivcol, phase, tol=<NUM_LIT>):

if phase == <NUM_LIT:1>:<EOL><INDENT>k = <NUM_LIT:2><EOL><DEDENT>else:<EOL><INDENT>k = <NUM_LIT:1><EOL><DEDENT>ma = np.ma.masked_where(T[:-k, pivcol] <= tol, T[:-k, pivcol], copy=False)<EOL>if ma.count() == <NUM_LIT:0>:<EOL><INDENT>return False, np.nan<EOL><DEDENT>mb = np.ma.masked_where(T[:-k, pivcol] <= tol, T[:-k, -<NUM_LIT:1>], copy=False)<EOL>q = mb / ma<EOL>return True, np.ma.where(q == q.min())[<NUM_LIT:0>][<NUM_LIT:0>]<EOL>

Given a linear programming simplex tableau, determine the row for the
pivot operation.

Parameters
----------
T : 2D ndarray
    The simplex tableau.
pivcol : int
    The index of the pivot column.
phase : int
    The phase of the simplex algorithm (1 or 2).
tol : float
    Elements in the pivot column smaller than tol will not be considered
    for pivoting.  Nominally this value is zero, but numerical issues
    cause a tolerance about zero to be necessary.

Returns
-------
status: bool
    True if a suitable pivot row was found, otherwise False.  A return
    of False indicates that the linear programming problem is unbounded.
row: int
    The index of the row of the pivot element.  If status is False, row
    will be returned as nan.


def _solve_simplex(T, n, basis, maxiter=<NUM_LIT:1000>, phase=<NUM_LIT:2>, callback=None,<EOL>tol=<NUM_LIT>, nit0=<NUM_LIT:0>, bland=False):

nit = nit0<EOL>complete = False<EOL>solution = np.zeros(T.shape[<NUM_LIT:1>]-<NUM_LIT:1>, dtype=np.float64)<EOL>if phase == <NUM_LIT:1>:<EOL><INDENT>m = T.shape[<NUM_LIT:0>]-<NUM_LIT:2><EOL><DEDENT>elif phase == <NUM_LIT:2>:<EOL><INDENT>m = T.shape[<NUM_LIT:0>]-<NUM_LIT:1><EOL><DEDENT>else:<EOL><INDENT>raise ValueError("<STR_LIT>")<EOL><DEDENT>while not complete:<EOL><INDENT>pivcol_found, pivcol = _pivot_col(T, tol, bland)<EOL>if not pivcol_found:<EOL><INDENT>pivcol = np.nan<EOL>pivrow = np.nan<EOL>status = <NUM_LIT:0><EOL>complete = True<EOL><DEDENT>else:<EOL><INDENT>pivrow_found, pivrow = _pivot_row(T, pivcol, phase, tol)<EOL>if not pivrow_found:<EOL><INDENT>status = <NUM_LIT:3><EOL>complete = True<EOL><DEDENT><DEDENT>if callback is not None:<EOL><INDENT>solution[:] = <NUM_LIT:0><EOL>solution[basis[:m]] = T[:m, -<NUM_LIT:1>]<EOL>callback(solution[:n], **{"<STR_LIT>": T,<EOL>"<STR_LIT>":phase,<EOL>"<STR_LIT>":nit,<EOL>"<STR_LIT>":(pivrow, pivcol),<EOL>"<STR_LIT>":basis,<EOL>"<STR_LIT>": complete and phase == <NUM_LIT:2>})<EOL><DEDENT>if not complete:<EOL><INDENT>if nit >= maxiter:<EOL><INDENT>status = <NUM_LIT:1><EOL>complete = True<EOL><DEDENT>else:<EOL><INDENT>basis[pivrow] = pivcol<EOL>pivval = T[pivrow][pivcol]<EOL>T[pivrow, :] = T[pivrow, :] / pivval<EOL>for irow in range(T.shape[<NUM_LIT:0>]):<EOL><INDENT>if irow != pivrow:<EOL><INDENT>T[irow, :] = T[irow, :] - T[pivrow, :]*T[irow, pivcol]<EOL><DEDENT><DEDENT>nit += <NUM_LIT:1><EOL><DEDENT><DEDENT><DEDENT>return nit, status<EOL>

Solve a linear programming problem in "standard maximization form" using
the Simplex Method.

Minimize :math:`f = c^T x`

subject to

.. math::

    Ax = b
    x_i >= 0
    b_j >= 0

Parameters
----------
T : array_like
    A 2-D array representing the simplex T corresponding to the
    maximization problem.  It should have the form:

    [[A[0, 0], A[0, 1], ..., A[0, n_total], b[0]],
     [A[1, 0], A[1, 1], ..., A[1, n_total], b[1]],
     .
     .
     .
     [A[m, 0], A[m, 1], ..., A[m, n_total], b[m]],
     [c[0],   c[1], ...,   c[n_total],    0]]

    for a Phase 2 problem, or the form:

    [[A[0, 0], A[0, 1], ..., A[0, n_total], b[0]],
     [A[1, 0], A[1, 1], ..., A[1, n_total], b[1]],
     .
     .
     .
     [A[m, 0], A[m, 1], ..., A[m, n_total], b[m]],
     [c[0],   c[1], ...,   c[n_total],   0],
     [c'[0],  c'[1], ...,  c'[n_total],  0]]

     for a Phase 1 problem (a Problem in which a basic feasible solution is
     sought prior to maximizing the actual objective.  T is modified in
     place by _solve_simplex.
n : int
    The number of true variables in the problem.
basis : array
    An array of the indices of the basic variables, such that basis[i]
    contains the column corresponding to the basic variable for row i.
    Basis is modified in place by _solve_simplex
maxiter : int
    The maximum number of iterations to perform before aborting the
    optimization.
phase : int
    The phase of the optimization being executed.  In phase 1 a basic
    feasible solution is sought and the T has an additional row representing
    an alternate objective function.
callback : callable
    If a callback function is provided, it will be called within each
    iteration of the simplex algorithm. The callback must have the
    signature `callback(xk, **kwargs)` where xk is the current solution
    vector and kwargs is a dictionary containing the following::
    "T" : The current Simplex algorithm T
    "nit" : The current iteration.
    "pivot" : The pivot (row, column) used for the next iteration.
    "phase" : Whether the algorithm is in Phase 1 or Phase 2.
    "basis" : The indices of the columns of the basic variables.
tol : float
    The tolerance which determines when a solution is "close enough" to
    zero in Phase 1 to be considered a basic feasible solution or close
    enough to positive to to serve as an optimal solution.
nit0 : int
    The initial iteration number used to keep an accurate iteration total
    in a two-phase problem.
bland : bool
    If True, choose pivots using Bland's rule [3].  In problems which
    fail to converge due to cycling, using Bland's rule can provide
    convergence at the expense of a less optimal path about the simplex.

Returns
-------
res : OptimizeResult
    The optimization result represented as a ``OptimizeResult`` object.
    Important attributes are: ``x`` the solution array, ``success`` a
    Boolean flag indicating if the optimizer exited successfully and
    ``message`` which describes the cause of the termination. Possible
    values for the ``status`` attribute are:
     0 : Optimization terminated successfully
     1 : Iteration limit reached
     2 : Problem appears to be infeasible
     3 : Problem appears to be unbounded

    See `OptimizeResult` for a description of other attributes.


def _linprog_simplex(c, A_ub=None, b_ub=None, A_eq=None, b_eq=None,<EOL>bounds=None, maxiter=<NUM_LIT:1000>, disp=False, callback=None,<EOL>tol=<NUM_LIT>, bland=False, **unknown_options):

_check_unknown_options(unknown_options)<EOL>status = <NUM_LIT:0><EOL>messages = {<NUM_LIT:0>: "<STR_LIT>",<EOL><NUM_LIT:1>: "<STR_LIT>",<EOL><NUM_LIT:2>: "<STR_LIT>"<EOL>"<STR_LIT>",<EOL><NUM_LIT:3>: "<STR_LIT>",<EOL><NUM_LIT:4>: "<STR_LIT>"}<EOL>have_floor_variable = False<EOL>cc = np.asarray(c)<EOL>f0 = <NUM_LIT:0><EOL>n = len(c)<EOL>Aeq = np.asarray(A_eq) if A_eq is not None else np.empty([<NUM_LIT:0>, len(cc)])<EOL>Aub = np.asarray(A_ub) if A_ub is not None else np.empty([<NUM_LIT:0>, len(cc)])<EOL>beq = np.ravel(np.asarray(b_eq)) if b_eq is not None else np.empty([<NUM_LIT:0>])<EOL>bub = np.ravel(np.asarray(b_ub)) if b_ub is not None else np.empty([<NUM_LIT:0>])<EOL>L = np.zeros(n, dtype=np.float64)<EOL>U = np.ones(n, dtype=np.float64)*np.inf<EOL>if bounds is None or len(bounds) == <NUM_LIT:0>:<EOL><INDENT>pass<EOL><DEDENT>elif len(bounds) == <NUM_LIT:2> and not hasattr(bounds[<NUM_LIT:0>], '<STR_LIT>'):<EOL><INDENT>L = np.asarray(n*[bounds[<NUM_LIT:0>]], dtype=np.float64)<EOL>U = np.asarray(n*[bounds[<NUM_LIT:1>]], dtype=np.float64)<EOL><DEDENT>else:<EOL><INDENT>if len(bounds) != n:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT>else:<EOL><INDENT>try:<EOL><INDENT>for i in range(n):<EOL><INDENT>if len(bounds[i]) != <NUM_LIT:2>:<EOL><INDENT>raise IndexError()<EOL><DEDENT>L[i] = bounds[i][<NUM_LIT:0>] if bounds[i][<NUM_LIT:0>] is not None else -np.inf<EOL>U[i] = bounds[i][<NUM_LIT:1>] if bounds[i][<NUM_LIT:1>] is not None else np.inf<EOL><DEDENT><DEDENT>except IndexError:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT><DEDENT><DEDENT>if np.any(L == -np.inf):<EOL><INDENT>n = n + <NUM_LIT:1><EOL>L = np.concatenate([np.array([<NUM_LIT:0>]), L])<EOL>U = np.concatenate([np.array([np.inf]), U])<EOL>cc = np.concatenate([np.array([<NUM_LIT:0>]), cc])<EOL>Aeq = np.hstack([np.zeros([Aeq.shape[<NUM_LIT:0>], <NUM_LIT:1>]), Aeq])<EOL>Aub = np.hstack([np.zeros([Aub.shape[<NUM_LIT:0>], <NUM_LIT:1>]), Aub])<EOL>have_floor_variable = True<EOL><DEDENT>for i in range(n):<EOL><INDENT>if(L[i] > U[i]):<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>" % (i, i))<EOL><DEDENT>if np.isinf(L[i]) and L[i] > <NUM_LIT:0>:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT>if np.isinf(U[i]) and U[i] < <NUM_LIT:0>:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT>if np.isfinite(L[i]) and L[i] > <NUM_LIT:0>:<EOL><INDENT>Aub = np.vstack([Aub, np.zeros(n)])<EOL>Aub[-<NUM_LIT:1>, i] = -<NUM_LIT:1><EOL>bub = np.concatenate([bub, np.array([-L[i]])])<EOL>L[i] = <NUM_LIT:0><EOL><DEDENT>if np.isfinite(U[i]):<EOL><INDENT>Aub = np.vstack([Aub, np.zeros(n)])<EOL>Aub[-<NUM_LIT:1>, i] = <NUM_LIT:1><EOL>bub = np.concatenate([bub, np.array([U[i]])])<EOL>U[i] = np.inf<EOL><DEDENT><DEDENT>for i in range(<NUM_LIT:0>, n):<EOL><INDENT>if L[i] < <NUM_LIT:0>:<EOL><INDENT>if np.isfinite(L[i]) and L[i] < <NUM_LIT:0>:<EOL><INDENT>beq[:] = beq[:] - Aeq[:, i] * L[i]<EOL>bub[:] = bub[:] - Aub[:, i] * L[i]<EOL>f0 = f0 - cc[i] * L[i]<EOL><DEDENT>else:<EOL><INDENT>Aeq[:, <NUM_LIT:0>] = Aeq[:, <NUM_LIT:0>] - Aeq[:, i]<EOL>Aub[:, <NUM_LIT:0>] = Aub[:, <NUM_LIT:0>] - Aub[:, i]<EOL>cc[<NUM_LIT:0>] = cc[<NUM_LIT:0>] - cc[i]<EOL><DEDENT><DEDENT>if np.isinf(U[i]):<EOL><INDENT>if U[i] < <NUM_LIT:0>:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT><DEDENT><DEDENT>mub = len(bub)<EOL>meq = len(beq)<EOL>m = mub+meq<EOL>n_slack = mub<EOL>n_artificial = meq + _count_nonzero(bub < <NUM_LIT:0>)<EOL>try:<EOL><INDENT>Aub_rows, Aub_cols = Aub.shape<EOL><DEDENT>except ValueError:<EOL><INDENT>raise ValueError("<STR_LIT>")<EOL><DEDENT>try:<EOL><INDENT>Aeq_rows, Aeq_cols = Aeq.shape<EOL><DEDENT>except ValueError:<EOL><INDENT>raise ValueError("<STR_LIT>")<EOL><DEDENT>if Aeq_rows != meq:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT>if Aub_rows != mub:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT>if Aeq_cols > <NUM_LIT:0> and Aeq_cols != n:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT>if Aub_cols > <NUM_LIT:0> and Aub_cols != n:<EOL><INDENT>status = -<NUM_LIT:1><EOL>message = ("<STR_LIT>"<EOL>"<STR_LIT>")<EOL><DEDENT>if status != <NUM_LIT:0>:<EOL><INDENT>raise ValueError(message)<EOL><DEDENT>T = np.zeros([m+<NUM_LIT:2>, n+n_slack+n_artificial+<NUM_LIT:1>])<EOL>T[-<NUM_LIT:2>, :n] = cc<EOL>T[-<NUM_LIT:2>, -<NUM_LIT:1>] = f0<EOL>b = T[:-<NUM_LIT:2>, -<NUM_LIT:1>]<EOL>if meq > <NUM_LIT:0>:<EOL><INDENT>T[:meq, :n] = Aeq<EOL>b[:meq] = beq<EOL><DEDENT>if mub > <NUM_LIT:0>:<EOL><INDENT>T[meq:meq+mub, :n] = Aub<EOL>b[meq:meq+mub] = bub<EOL>np.fill_diagonal(T[meq:m, n:n+n_slack], <NUM_LIT:1>)<EOL><DEDENT>slcount = <NUM_LIT:0><EOL>avcount = <NUM_LIT:0><EOL>basis = np.zeros(m, dtype=int)<EOL>r_artificial = np.zeros(n_artificial, dtype=int)<EOL>for i in range(m):<EOL><INDENT>if i < meq or b[i] < <NUM_LIT:0>:<EOL><INDENT>basis[i] = n+n_slack+avcount<EOL>r_artificial[avcount] = i<EOL>avcount += <NUM_LIT:1><EOL>if b[i] < <NUM_LIT:0>:<EOL><INDENT>b[i] *= -<NUM_LIT:1><EOL>T[i, :-<NUM_LIT:1>] *= -<NUM_LIT:1><EOL><DEDENT>T[i, basis[i]] = <NUM_LIT:1><EOL>T[-<NUM_LIT:1>, basis[i]] = <NUM_LIT:1><EOL><DEDENT>else:<EOL><INDENT>basis[i] = n+slcount<EOL>slcount += <NUM_LIT:1><EOL><DEDENT><DEDENT>for r in r_artificial:<EOL><INDENT>T[-<NUM_LIT:1>, :] = T[-<NUM_LIT:1>, :] - T[r, :]<EOL><DEDENT>nit1, status = _solve_simplex(T, n, basis, phase=<NUM_LIT:1>, callback=callback,<EOL>maxiter=maxiter, tol=tol, bland=bland)<EOL>if abs(T[-<NUM_LIT:1>, -<NUM_LIT:1>]) < tol:<EOL><INDENT>T = T[:-<NUM_LIT:1>, :]<EOL>T = np.delete(T, np.s_[n+n_slack:n+n_slack+n_artificial], <NUM_LIT:1>)<EOL><DEDENT>else:<EOL><INDENT>status = <NUM_LIT:2><EOL><DEDENT>if status != <NUM_LIT:0>:<EOL><INDENT>message = messages[status]<EOL>if disp:<EOL><INDENT>print(message)<EOL><DEDENT>return OptimizeResult(x=np.nan, fun=-T[-<NUM_LIT:1>, -<NUM_LIT:1>], nit=nit1, status=status,<EOL>message=message, success=False)<EOL><DEDENT>nit2, status = _solve_simplex(T, n, basis, maxiter=maxiter-nit1, phase=<NUM_LIT:2>,<EOL>callback=callback, tol=tol, nit0=nit1,<EOL>bland=bland)<EOL>solution = np.zeros(n+n_slack+n_artificial)<EOL>solution[basis[:m]] = T[:m, -<NUM_LIT:1>]<EOL>x = solution[:n]<EOL>slack = solution[n:n+n_slack]<EOL>masked_L = np.ma.array(L, mask=np.isinf(L), fill_value=<NUM_LIT:0.0>).filled()<EOL>x = x + masked_L<EOL>if have_floor_variable:<EOL><INDENT>for i in range(<NUM_LIT:1>, n):<EOL><INDENT>if np.isinf(L[i]):<EOL><INDENT>x[i] -= x[<NUM_LIT:0>]<EOL><DEDENT><DEDENT>x = x[<NUM_LIT:1>:]<EOL><DEDENT>obj = -T[-<NUM_LIT:1>, -<NUM_LIT:1>]<EOL>if status in (<NUM_LIT:0>, <NUM_LIT:1>):<EOL><INDENT>if disp:<EOL><INDENT>print(messages[status])<EOL>print("<STR_LIT>".format(obj))<EOL>print("<STR_LIT>".format(nit2))<EOL><DEDENT><DEDENT>else:<EOL><INDENT>if disp:<EOL><INDENT>print(messages[status])<EOL>print("<STR_LIT>".format(nit2))<EOL><DEDENT><DEDENT>return OptimizeResult(x=x, fun=obj, nit=int(nit2), status=status, slack=slack,<EOL>message=messages[status], success=(status == <NUM_LIT:0>))<EOL>

Solve the following linear programming problem via a two-phase
simplex algorithm.

maximize:     c^T * x

subject to:   A_ub * x <= b_ub
              A_eq * x == b_eq

Parameters
----------
c : array_like
    Coefficients of the linear objective function to be maximized.
A_ub :
    2-D array which, when matrix-multiplied by x, gives the values of the
    upper-bound inequality constraints at x.
b_ub : array_like
    1-D array of values representing the upper-bound of each inequality
    constraint (row) in A_ub.
A_eq : array_like
    2-D array which, when matrix-multiplied by x, gives the values of the
    equality constraints at x.
b_eq : array_like
    1-D array of values representing the RHS of each equality constraint
    (row) in A_eq.
bounds : array_like
    The bounds for each independent variable in the solution, which can take
    one of three forms::
    None : The default bounds, all variables are non-negative.
    (lb, ub) : If a 2-element sequence is provided, the same
              lower bound (lb) and upper bound (ub) will be applied
              to all variables.
    [(lb_0, ub_0), (lb_1, ub_1), ...] : If an n x 2 sequence is provided,
              each variable x_i will be bounded by lb[i] and ub[i].
    Infinite bounds are specified using -np.inf (negative)
    or np.inf (positive).
maxiter : int
   The maximum number of iterations to perform.
disp : bool
    If True, print exit status message to sys.stdout
callback : callable
    If a callback function is provide, it will be called within each
    iteration of the simplex algorithm. The callback must have the
    signature `callback(xk, **kwargs)` where xk is the current solution
    vector and kwargs is a dictionary containing the following::
    "tableau" : The current Simplex algorithm tableau
    "nit" : The current iteration.
    "pivot" : The pivot (row, column) used for the next iteration.
    "phase" : Whether the algorithm is in Phase 1 or Phase 2.
    "bv" : A structured array containing a string representation of each
           basic variable and its current value.
tol : float
    The tolerance which determines when a solution is "close enough" to zero
    in Phase 1 to be considered a basic feasible solution or close enough
    to positive to to serve as an optimal solution.
bland : bool
    If True, use Bland's anti-cycling rule [3] to choose pivots to
    prevent cycling.  If False, choose pivots which should lead to a
    converged solution more quickly.  The latter method is subject to
    cycling (non-convergence) in rare instances.

Returns
-------
A scipy.optimize.OptimizeResult consisting of the following fields::
    x : ndarray
        The independent variable vector which optimizes the linear
        programming problem.
    slack : ndarray
        The values of the slack variables.  Each slack variable corresponds
        to an inequality constraint.  If the slack is zero, then the
        corresponding constraint is active.
    success : bool
        Returns True if the algorithm succeeded in finding an optimal
        solution.
    status : int
        An integer representing the exit status of the optimization::
         0 : Optimization terminated successfully
         1 : Iteration limit reached
         2 : Problem appears to be infeasible
         3 : Problem appears to be unbounded
    nit : int
        The number of iterations performed.
    message : str
        A string descriptor of the exit status of the optimization.

Examples
--------
Consider the following problem:

Minimize: f = -1*x[0] + 4*x[1]

Subject to: -3*x[0] + 1*x[1] <= 6
             1*x[0] + 2*x[1] <= 4
                        x[1] >= -3

where:  -inf <= x[0] <= inf

This problem deviates from the standard linear programming problem.  In
standard form, linear programming problems assume the variables x are
non-negative.  Since the variables don't have standard bounds where
0 <= x <= inf, the bounds of the variables must be explicitly set.

There are two upper-bound constraints, which can be expressed as

dot(A_ub, x) <= b_ub

The input for this problem is as follows:

>>> c = [-1, 4]
>>> A = [[-3, 1], [1, 2]]
>>> b = [6, 4]
>>> x0_bnds = (None, None)
>>> x1_bnds = (-3, None)
>>> res = linprog(c, A, b, bounds=(x0_bnds, x1_bnds), options={"disp":True})
>>> print(res)
Optimization terminated successfully.
     Current function value: 11.428571
     Iterations: 2
status: 0
success: True
fun: 11.428571428571429
x: array([-1.14285714,  2.57142857])
slack: array([], dtype=np.float64)
message: 'Optimization terminated successfully.'
nit: 2

References
----------
.. [1] Dantzig, George B., Linear programming and extensions. Rand
       Corporation Research Study Princeton Univ. Press, Princeton, NJ, 1963
.. [2] Hillier, S.H. and Lieberman, G.J. (1995), "Introduction to
       Mathematical Programming", McGraw-Hill, Chapter 4.
.. [3] Bland, Robert G. New finite pivoting rules for the simplex method.
       Mathematics of Operations Research (2), 1977: pp. 103-107.


def linprog(c, A_ub=None, b_ub=None, A_eq=None, b_eq=None,<EOL>bounds=None, method='<STR_LIT>', callback=None,<EOL>options=None):

meth = method.lower()<EOL>if options is None:<EOL><INDENT>options = {}<EOL><DEDENT>if meth == '<STR_LIT>':<EOL><INDENT>return _linprog_simplex(c, A_ub=A_ub, b_ub=b_ub, A_eq=A_eq, b_eq=b_eq,<EOL>bounds=bounds, callback=callback, **options)<EOL><DEDENT>else:<EOL><INDENT>raise ValueError('<STR_LIT>' % method)<EOL><DEDENT>

Minimize a linear objective function subject to linear
equality and inequality constraints.

Linear Programming is intended to solve the following problem form:

Minimize:     c^T * x

Subject to:   A_ub * x <= b_ub
              A_eq * x == b_eq

Parameters
----------
c : array_like
    Coefficients of the linear objective function to be minimized.
A_ub :
    2-D array which, when matrix-multiplied by x, gives the values of the
    upper-bound inequality constraints at x.
b_ub : array_like
    1-D array of values representing the upper-bound of each inequality
    constraint (row) in A_ub.
A_eq : array_like
    2-D array which, when matrix-multiplied by x, gives the values of the
    equality constraints at x.
b_eq : array_like
    1-D array of values representing the RHS of each equality constraint
    (row) in A_eq.
bounds : sequence, optional
    ``(min, max)`` pairs for each element in ``x``, defining
    the bounds on that parameter. Use None for one of ``min`` or
    ``max`` when there is no bound in that direction. By default
    bounds are ``(0, None)`` (non-negative)
    If a sequence containing a single tuple is provided, then ``min`` and
    ``max`` will be applied to all variables in the problem.
method : str, optional
    Type of solver.  At this time only 'simplex' is supported.
callback : callable, optional
    If a callback function is provide, it will be called within each
    iteration of the simplex algorithm. The callback must have the signature
    `callback(xk, **kwargs)` where xk is the current solution vector
    and kwargs is a dictionary containing the following::

        "tableau" : The current Simplex algorithm tableau
        "nit" : The current iteration.
        "pivot" : The pivot (row, column) used for the next iteration.
        "phase" : Whether the algorithm is in Phase 1 or Phase 2.
        "basis" : The indices of the columns of the basic variables.

options : dict, optional
    A dictionary of solver options. All methods accept the following
    generic options:

        maxiter : int
            Maximum number of iterations to perform.
        disp : bool
            Set to True to print convergence messages.

    For method-specific options, see `show_options('linprog')`.

Returns
-------
A `scipy.optimize.OptimizeResult` consisting of the following fields:

    x : ndarray
        The independent variable vector which optimizes the linear
        programming problem.
    slack : ndarray
        The values of the slack variables.  Each slack variable corresponds
        to an inequality constraint.  If the slack is zero, then the
        corresponding constraint is active.
    success : bool
        Returns True if the algorithm succeeded in finding an optimal
        solution.
    status : int
        An integer representing the exit status of the optimization::

             0 : Optimization terminated successfully
             1 : Iteration limit reached
             2 : Problem appears to be infeasible
             3 : Problem appears to be unbounded

    nit : int
        The number of iterations performed.
    message : str
        A string descriptor of the exit status of the optimization.

See Also
--------
show_options : Additional options accepted by the solvers

Notes
-----
This section describes the available solvers that can be selected by the
'method' parameter. The default method is *Simplex*.

Method *Simplex* uses the Simplex algorithm (as it relates to Linear
Programming, NOT the Nelder-Mead Simplex) [1]_, [2]_. This algorithm
should be reasonably reliable and fast.

.. versionadded:: 0.15.0

References
----------
.. [1] Dantzig, George B., Linear programming and extensions. Rand
       Corporation Research Study Princeton Univ. Press, Princeton, NJ, 1963
.. [2] Hillier, S.H. and Lieberman, G.J. (1995), "Introduction to
       Mathematical Programming", McGraw-Hill, Chapter 4.
.. [3] Bland, Robert G. New finite pivoting rules for the simplex method.
       Mathematics of Operations Research (2), 1977: pp. 103-107.

Examples
--------
Consider the following problem:

Minimize: f = -1*x[0] + 4*x[1]

Subject to: -3*x[0] + 1*x[1] <= 6
             1*x[0] + 2*x[1] <= 4
                        x[1] >= -3

where:  -inf <= x[0] <= inf

This problem deviates from the standard linear programming problem.
In standard form, linear programming problems assume the variables x are
non-negative.  Since the variables don't have standard bounds where
0 <= x <= inf, the bounds of the variables must be explicitly set.

There are two upper-bound constraints, which can be expressed as

dot(A_ub, x) <= b_ub

The input for this problem is as follows:

>>> c = [-1, 4]
>>> A = [[-3, 1], [1, 2]]
>>> b = [6, 4]
>>> x0_bounds = (None, None)
>>> x1_bounds = (-3, None)
>>> res = linprog(c, A_ub=A, b_ub=b, bounds=(x0_bounds, x1_bounds),
...               options={"disp": True})
>>> print(res)
Optimization terminated successfully.
     Current function value: -11.428571
     Iterations: 2
status: 0
success: True
fun: -11.428571428571429
x: array([-1.14285714,  2.57142857])
message: 'Optimization terminated successfully.'
nit: 2

Note the actual objective value is 11.428571.  In this case we minimized
the negative of the objective function.


def newton(func, x0, fprime=None, args=(), tol=<NUM_LIT>, maxiter=<NUM_LIT:50>,<EOL>fprime2=None):

if tol <= <NUM_LIT:0>:<EOL><INDENT>raise ValueError("<STR_LIT>" % tol)<EOL><DEDENT>if fprime is not None:<EOL><INDENT>p0 = <NUM_LIT:1.0> * x0<EOL>fder2 = <NUM_LIT:0><EOL>for iter in range(maxiter):<EOL><INDENT>myargs = (p0,) + args<EOL>fder = fprime(*myargs)<EOL>if fder == <NUM_LIT:0>:<EOL><INDENT>msg = "<STR_LIT>"<EOL>warnings.warn(msg, RuntimeWarning)<EOL>return p0<EOL><DEDENT>fval = func(*myargs)<EOL>if fprime2 is not None:<EOL><INDENT>fder2 = fprime2(*myargs)<EOL><DEDENT>if fder2 == <NUM_LIT:0>:<EOL><INDENT>p = p0 - fval / fder<EOL><DEDENT>else:<EOL><INDENT>discr = fder ** <NUM_LIT:2> - <NUM_LIT:2> * fval * fder2<EOL>if discr < <NUM_LIT:0>:<EOL><INDENT>p = p0 - fder / fder2<EOL><DEDENT>else:<EOL><INDENT>p = p0 - <NUM_LIT:2>*fval / (fder + sign(fder) * sqrt(discr))<EOL><DEDENT><DEDENT>if abs(p - p0) < tol:<EOL><INDENT>return p<EOL><DEDENT>p0 = p<EOL><DEDENT><DEDENT>else:<EOL><INDENT>p0 = x0<EOL>if x0 >= <NUM_LIT:0>:<EOL><INDENT>p1 = x0*(<NUM_LIT:1> + <NUM_LIT>) + <NUM_LIT><EOL><DEDENT>else:<EOL><INDENT>p1 = x0*(<NUM_LIT:1> + <NUM_LIT>) - <NUM_LIT><EOL><DEDENT>q0 = func(*((p0,) + args))<EOL>q1 = func(*((p1,) + args))<EOL>for iter in range(maxiter):<EOL><INDENT>if q1 == q0:<EOL><INDENT>if p1 != p0:<EOL><INDENT>msg = "<STR_LIT>" % (p1 - p0)<EOL>warnings.warn(msg, RuntimeWarning)<EOL><DEDENT>return (p1 + p0)/<NUM_LIT><EOL><DEDENT>else:<EOL><INDENT>p = p1 - q1*(p1 - p0)/(q1 - q0)<EOL><DEDENT>if abs(p - p1) < tol:<EOL><INDENT>return p<EOL><DEDENT>p0 = p1<EOL>q0 = q1<EOL>p1 = p<EOL>q1 = func(*((p1,) + args))<EOL><DEDENT><DEDENT>msg = "<STR_LIT>" % (maxiter, p)<EOL>raise RuntimeError(msg)<EOL>

Find a zero using the Newton-Raphson or secant method.

Find a zero of the function `func` given a nearby starting point `x0`.
The Newton-Raphson method is used if the derivative `fprime` of `func`
is provided, otherwise the secant method is used.  If the second order
derivate `fprime2` of `func` is provided, parabolic Halley's method
is used.

Parameters
----------
func : function
    The function whose zero is wanted. It must be a function of a
    single variable of the form f(x,a,b,c...), where a,b,c... are extra
    arguments that can be passed in the `args` parameter.
x0 : float
    An initial estimate of the zero that should be somewhere near the
    actual zero.
fprime : function, optional
    The derivative of the function when available and convenient. If it
    is None (default), then the secant method is used.
args : tuple, optional
    Extra arguments to be used in the function call.
tol : float, optional
    The allowable error of the zero value.
maxiter : int, optional
    Maximum number of iterations.
fprime2 : function, optional
    The second order derivative of the function when available and
    convenient. If it is None (default), then the normal Newton-Raphson
    or the secant method is used. If it is given, parabolic Halley's
    method is used.

Returns
-------
zero : float
    Estimated location where function is zero.

See Also
--------
brentq, brenth, ridder, bisect
fsolve : find zeroes in n dimensions.

Notes
-----
The convergence rate of the Newton-Raphson method is quadratic,
the Halley method is cubic, and the secant method is
sub-quadratic.  This means that if the function is well behaved
the actual error in the estimated zero is approximately the square
(cube for Halley) of the requested tolerance up to roundoff
error. However, the stopping criterion used here is the step size
and there is no guarantee that a zero has been found. Consequently
the result should be verified. Safer algorithms are brentq,
brenth, ridder, and bisect, but they all require that the root
first be bracketed in an interval where the function changes
sign. The brentq algorithm is recommended for general use in one
dimensional problems when such an interval has been found.


def bisect(f, a, b, args=(),<EOL>xtol=_xtol, rtol=_rtol, maxiter=_iter,<EOL>full_output=False, disp=True):

if not isinstance(args, tuple):<EOL><INDENT>args = (args,)<EOL><DEDENT>if xtol <= <NUM_LIT:0>:<EOL><INDENT>raise ValueError("<STR_LIT>" % xtol)<EOL><DEDENT>if rtol < _rtol:<EOL><INDENT>raise ValueError("<STR_LIT>" % (rtol, _rtol))<EOL><DEDENT>r = _zeros._bisect(f,a,b,xtol,rtol,maxiter,args,full_output,disp)<EOL>return results_c(full_output, r)<EOL>

Find root of a function within an interval.

Basic bisection routine to find a zero of the function `f` between the
arguments `a` and `b`. `f(a)` and `f(b)` cannot have the same signs.
Slow but sure.

Parameters
----------
f : function
    Python function returning a number.  `f` must be continuous, and
    f(a) and f(b) must have opposite signs.
a : number
    One end of the bracketing interval [a,b].
b : number
    The other end of the bracketing interval [a,b].
xtol : number, optional
    The routine converges when a root is known to lie within `xtol` of the
    value return. Should be >= 0.  The routine modifies this to take into
    account the relative precision of doubles.
rtol : number, optional
    The routine converges when a root is known to lie within `rtol` times
    the value returned of the value returned. Should be >= 0. Defaults to
    ``np.finfo(float).eps * 2``.
maxiter : number, optional
    if convergence is not achieved in `maxiter` iterations, an error is
    raised.  Must be >= 0.
args : tuple, optional
    containing extra arguments for the function `f`.
    `f` is called by ``apply(f, (x)+args)``.
full_output : bool, optional
    If `full_output` is False, the root is returned.  If `full_output` is
    True, the return value is ``(x, r)``, where x is the root, and r is
    a `RootResults` object.
disp : bool, optional
    If True, raise RuntimeError if the algorithm didn't converge.

Returns
-------
x0 : float
    Zero of `f` between `a` and `b`.
r : RootResults (present if ``full_output = True``)
    Object containing information about the convergence.  In particular,
    ``r.converged`` is True if the routine converged.

See Also
--------
brentq, brenth, bisect, newton
fixed_point : scalar fixed-point finder
fsolve : n-dimensional root-finding


def ridder(f, a, b, args=(),<EOL>xtol=_xtol, rtol=_rtol, maxiter=_iter,<EOL>full_output=False, disp=True):

if not isinstance(args, tuple):<EOL><INDENT>args = (args,)<EOL><DEDENT>if xtol <= <NUM_LIT:0>:<EOL><INDENT>raise ValueError("<STR_LIT>" % xtol)<EOL><DEDENT>if rtol < _rtol:<EOL><INDENT>raise ValueError("<STR_LIT>" % (rtol, _rtol))<EOL><DEDENT>r = _zeros._ridder(f,a,b,xtol,rtol,maxiter,args,full_output,disp)<EOL>return results_c(full_output, r)<EOL>

Find a root of a function in an interval.

Parameters
----------
f : function
    Python function returning a number.  f must be continuous, and f(a) and
    f(b) must have opposite signs.
a : number
    One end of the bracketing interval [a,b].
b : number
    The other end of the bracketing interval [a,b].
xtol : number, optional
    The routine converges when a root is known to lie within xtol of the
    value return. Should be >= 0.  The routine modifies this to take into
    account the relative precision of doubles.
rtol : number, optional
    The routine converges when a root is known to lie within `rtol` times
    the value returned of the value returned. Should be >= 0. Defaults to
    ``np.finfo(float).eps * 2``.
maxiter : number, optional
    if convergence is not achieved in maxiter iterations, an error is
    raised.  Must be >= 0.
args : tuple, optional
    containing extra arguments for the function `f`.
    `f` is called by ``apply(f, (x)+args)``.
full_output : bool, optional
    If `full_output` is False, the root is returned.  If `full_output` is
    True, the return value is ``(x, r)``, where `x` is the root, and `r` is
    a RootResults object.
disp : bool, optional
    If True, raise RuntimeError if the algorithm didn't converge.

Returns
-------
x0 : float
    Zero of `f` between `a` and `b`.
r : RootResults (present if ``full_output = True``)
    Object containing information about the convergence.
    In particular, ``r.converged`` is True if the routine converged.

See Also
--------
brentq, brenth, bisect, newton : one-dimensional root-finding
fixed_point : scalar fixed-point finder

Notes
-----
Uses [Ridders1979]_ method to find a zero of the function `f` between the
arguments `a` and `b`. Ridders' method is faster than bisection, but not
generally as fast as the Brent rountines. [Ridders1979]_ provides the
classic description and source of the algorithm. A description can also be
found in any recent edition of Numerical Recipes.

The routine used here diverges slightly from standard presentations in
order to be a bit more careful of tolerance.

References
----------
.. [Ridders1979]
   Ridders, C. F. J. "A New Algorithm for Computing a
   Single Root of a Real Continuous Function."
   IEEE Trans. Circuits Systems 26, 979-980, 1979.


def brentq(f, a, b, args=(),<EOL>xtol=_xtol, rtol=_rtol, maxiter=_iter,<EOL>full_output=False, disp=True):

if not isinstance(args, tuple):<EOL><INDENT>args = (args,)<EOL><DEDENT>if xtol <= <NUM_LIT:0>:<EOL><INDENT>raise ValueError("<STR_LIT>" % xtol)<EOL><DEDENT>if rtol < _rtol:<EOL><INDENT>raise ValueError("<STR_LIT>" % (rtol, _rtol))<EOL><DEDENT>r = _zeros._brentq(f,a,b,xtol,rtol,maxiter,args,full_output,disp)<EOL>return results_c(full_output, r)<EOL>

Find a root of a function in given interval.

Return float, a zero of `f` between `a` and `b`.  `f` must be a continuous
function, and [a,b] must be a sign changing interval.

Description:
Uses the classic Brent (1973) method to find a zero of the function `f` on
the sign changing interval [a , b].  Generally considered the best of the
rootfinding routines here.  It is a safe version of the secant method that
uses inverse quadratic extrapolation.  Brent's method combines root
bracketing, interval bisection, and inverse quadratic interpolation.  It is
sometimes known as the van Wijngaarden-Dekker-Brent method.  Brent (1973)
claims convergence is guaranteed for functions computable within [a,b].

[Brent1973]_ provides the classic description of the algorithm.  Another
description can be found in a recent edition of Numerical Recipes, including
[PressEtal1992]_.  Another description is at
http://mathworld.wolfram.com/BrentsMethod.html.  It should be easy to
understand the algorithm just by reading our code.  Our code diverges a bit
from standard presentations: we choose a different formula for the
extrapolation step.

Parameters
----------
f : function
    Python function returning a number.  f must be continuous, and f(a) and
    f(b) must have opposite signs.
a : number
    One end of the bracketing interval [a,b].
b : number
    The other end of the bracketing interval [a,b].
xtol : number, optional
    The routine converges when a root is known to lie within xtol of the
    value return. Should be >= 0.  The routine modifies this to take into
    account the relative precision of doubles.
rtol : number, optional
    The routine converges when a root is known to lie within `rtol` times
    the value returned of the value returned. Should be >= 0. Defaults to
    ``np.finfo(float).eps * 2``.
maxiter : number, optional
    if convergence is not achieved in maxiter iterations, an error is
    raised.  Must be >= 0.
args : tuple, optional
    containing extra arguments for the function `f`.
    `f` is called by ``apply(f, (x)+args)``.
full_output : bool, optional
    If `full_output` is False, the root is returned.  If `full_output` is
    True, the return value is ``(x, r)``, where `x` is the root, and `r` is
    a RootResults object.
disp : bool, optional
    If True, raise RuntimeError if the algorithm didn't converge.

Returns
-------
x0 : float
    Zero of `f` between `a` and `b`.
r : RootResults (present if ``full_output = True``)
    Object containing information about the convergence.  In particular,
    ``r.converged`` is True if the routine converged.

See Also
--------
multivariate local optimizers
  `fmin`, `fmin_powell`, `fmin_cg`, `fmin_bfgs`, `fmin_ncg`
nonlinear least squares minimizer
  `leastsq`
constrained multivariate optimizers
  `fmin_l_bfgs_b`, `fmin_tnc`, `fmin_cobyla`
global optimizers
  `anneal`, `basinhopping`, `brute`, `differential_evolution`
local scalar minimizers
  `fminbound`, `brent`, `golden`, `bracket`
n-dimensional root-finding
  `fsolve`
one-dimensional root-finding
  `brentq`, `brenth`, `ridder`, `bisect`, `newton`
scalar fixed-point finder
  `fixed_point`

Notes
-----
`f` must be continuous.  f(a) and f(b) must have opposite signs.

References
----------
.. [Brent1973]
   Brent, R. P.,
   *Algorithms for Minimization Without Derivatives*.
   Englewood Cliffs, NJ: Prentice-Hall, 1973. Ch. 3-4.

.. [PressEtal1992]
   Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T.
   *Numerical Recipes in FORTRAN: The Art of Scientific Computing*, 2nd ed.
   Cambridge, England: Cambridge University Press, pp. 352-355, 1992.
   Section 9.3:  "Van Wijngaarden-Dekker-Brent Method."


def brenth(f, a, b, args=(),<EOL>xtol=_xtol, rtol=_rtol, maxiter=_iter,<EOL>full_output=False, disp=True):

if not isinstance(args, tuple):<EOL><INDENT>args = (args,)<EOL><DEDENT>if xtol <= <NUM_LIT:0>:<EOL><INDENT>raise ValueError("<STR_LIT>" % xtol)<EOL><DEDENT>if rtol < _rtol:<EOL><INDENT>raise ValueError("<STR_LIT>" % (rtol, _rtol))<EOL><DEDENT>r = _zeros._brenth(f,a, b, xtol, rtol, maxiter, args, full_output, disp)<EOL>return results_c(full_output, r)<EOL>

Find root of f in [a,b].

    A variation on the classic Brent routine to find a zero of the function f
    between the arguments a and b that uses hyperbolic extrapolation instead of
    inverse quadratic extrapolation. There was a paper back in the 1980's ...
    f(a) and f(b) cannot have the same signs. Generally on a par with the
    brent routine, but not as heavily tested.  It is a safe version of the
    secant method that uses hyperbolic extrapolation. The version here is by
    Chuck Harris.

    Parameters
    ----------
    f : function
        Python function returning a number.  f must be continuous, and f(a) and
        f(b) must have opposite signs.
    a : number
        One end of the bracketing interval [a,b].
    b : number
        The other end of the bracketing interval [a,b].
    xtol : number, optional
        The routine converges when a root is known to lie within xtol of the
        value return. Should be >= 0.  The routine modifies this to take into
        account the relative precision of doubles.
    rtol : number, optional
        The routine converges when a root is known to lie within `rtol` times
        the value returned of the value returned. Should be >= 0. Defaults to
        ``np.finfo(float).eps * 2``.
    maxiter : number, optional
        if convergence is not achieved in maxiter iterations, an error is
        raised.  Must be >= 0.
    args : tuple, optional
        containing extra arguments for the function `f`.
        `f` is called by ``apply(f, (x)+args)``.
    full_output : bool, optional
        If `full_output` is False, the root is returned.  If `full_output` is
        True, the return value is ``(x, r)``, where `x` is the root, and `r` is
        a RootResults object.
    disp : bool, optional
        If True, raise RuntimeError if the algorithm didn't converge.

    Returns
    -------
    x0 : float
        Zero of `f` between `a` and `b`.
    r : RootResults (present if ``full_output = True``)
        Object containing information about the convergence.  In particular,
        ``r.converged`` is True if the routine converged.

    See Also
    --------
    fmin, fmin_powell, fmin_cg,
           fmin_bfgs, fmin_ncg : multivariate local optimizers

    leastsq : nonlinear least squares minimizer

    fmin_l_bfgs_b, fmin_tnc, fmin_cobyla : constrained multivariate optimizers

    anneal, brute : global optimizers

    fminbound, brent, golden, bracket : local scalar minimizers

    fsolve : n-dimensional root-finding

    brentq, brenth, ridder, bisect, newton : one-dimensional root-finding

    fixed_point : scalar fixed-point finder


A, b = map(asarray_chkfinite, (A, b))<EOL>if len(A.shape) != <NUM_LIT:2>:<EOL><INDENT>raise ValueError("<STR_LIT>")<EOL><DEDENT>if len(b.shape) != <NUM_LIT:1>:<EOL><INDENT>raise ValueError("<STR_LIT>")<EOL><DEDENT>m, n = A.shape<EOL>if m != b.shape[<NUM_LIT:0>]:<EOL><INDENT>raise ValueError("<STR_LIT>")<EOL><DEDENT>w = zeros((n,), dtype=double)<EOL>zz = zeros((m,), dtype=double)<EOL>index = zeros((n,), dtype=int)<EOL>x, rnorm, mode = _nnls.nnls(A, m, n, b, w, zz, index)<EOL>if mode != <NUM_LIT:1>:<EOL><INDENT>raise RuntimeError("<STR_LIT>")<EOL><DEDENT>return x, rnorm<EOL>

Solve ``argmin_x || Ax - b ||_2`` for ``x>=0``. This is a wrapper
for a FORTAN non-negative least squares solver.

Parameters
----------
A : ndarray
    Matrix ``A`` as shown above.
b : ndarray
    Right-hand side vector.

Returns
-------
x : ndarray
    Solution vector.
rnorm : float
    The residual, ``|| Ax-b ||_2``.

Notes
-----
The FORTRAN code was published in the book below. The algorithm
is an active set method. It solves the KKT (Karush-Kuhn-Tucker)
conditions for the non-negative least squares problem.

References
----------
Lawson C., Hanson R.J., (1987) Solving Least Squares Problems, SIAM