For instance, PROC NLIN of SAS provides an option for executing a grid search to choose a starting value. Searching a very fine grid would match the performance of the random procedure. But fine grid search does not readily scale up for multiple parameters. In both cases, the BFGS uses the analytical first derivative. The use of a Newton algorithm does not significantly alter the results.
Newton methods are sometimes inferior.
The following three equations define the normal mixture densities we use. The second and third of these densities are particularly difficult to optimize. Graphs of the densities appear in Figures 1 through 4 . The path would be the same if the operator set, population size, starting values and pseudorandom number generator seeds were all the same. Each of the densities has several local maxima.