gonum/optimize — pull request #8 by vladimir-ch (2014-11-14 00:35:36+00:00)
Title: Fix L-BFGS

This PR fixes a bug in the computation of the descent direction and modifies the step size on the first iteration.

The bug was in updating the L-BFGS history which was basically one iteration off. Consequently, the generated descent directions were not correct and too many iterations and function/gradient evaluations were necessary for convergence. Fixing this bug reduces the stats to some 38 iterations and 60 function evaluations for the Rosenbrock function from (-1.2,1).

Also, the step size on the first iteration is modified so that the first attempted step is in fact the *normalized* steepest descent step. This is consistent with other optimization codes, e.g., with the original Nocedal's L-BFGS code. Furhermore, it improves the performance on the above test problem to some 34 iterations and 45 function evaluations which are very reasonable and expected numbers.