This is repeated until the gradient reaches zero within some tolerance. Simple and only requires storage of the gradient. One of the few methods guaranteed to lower the energy. The steepest descent will never actually reach the minimum, it crawls toward it with ever decreasing speed. Conjugate Gradients Related to the method of steepest descents but also uses the gradient information at previous points to correct the gradient direction of the current step.
First step is done with steepest descents.
Each subsequent step is performed along a line, which is a mixture of the current negative gradient and the previous search direction. . Less oscillations that the method of steepest descents. Requires the storage of previous gradients as well as the current gradient. Newton-Raphson Methods Uses 2 nd derivative information to improve the convergence a stationary point. Expand the true function to second order around the current point x o .