scipy/scipy — pull request #3088 by js850 (2013-11-20 15:30:25+00:00)
Title: OPT: Add a benchmark framework for the optimizers

This pull request adds a framework for benchmarking the optimizers, along with 8 or so benchmarks on different functions. I used the numpy.testing.Tester().bench() framework so it already works alongside the other scipy benchmarks.

Currently all the test functions are smooth functions like Rosenbrock, Booth, Beale, and the Lennard-Jones potential. It would be great to have some other types of functions, especially ones where the gradient is not available and constrained problems. Towards this end I wrote it to be really easy to add new functions with just a few lines of code, e.g.

b = _BenchOptimizers("Rosenbrock function",
fun=rosen, der=rosen_der, hess=rosen_hess)
for i in xrange(10):
b.bench_run(np.random.uniform(-3,3,3))
b.print_results()

An example output is

=========================================================
Optimizer benchmark: Rosenbrock function
dimensions: 3, extra kwargs: {'tol': 0.0001}
averaged over 10 starting configurations
Optimizer nfail nfev njev nhev time
---------------------------------------------------------
L-BFGS-B | 0 | 41 | 0 | 0 | 0.00386183
SLSQP | 0 | 54 | 37 | 0 | 0.00547173
BFGS | 0 | 58 | 58 | 0 | 0.00888143
trust-ncg | 0 | 45 | 41 | 40 | 0.0109362
Newton-CG | 0 | 66 | 118 | 52 | 0.0186812
Powell | 0 | 981 | 0 | 0 | 0.0508468
CG | 1 | 103 | 102 | 0 | 0.0107903
dogleg | 2 | 19 | 17 | 16 | 0.00508533
TNC | 5 | 86 | 0 | 0 | 0.00598359
COBYLA | 8 | 831 | 0 | 0 | 0.0220224

Is this something scipy would be interested in?