Robust descent in differentiable optimization using automatic finite differences
Descent algorithms use comparisons of a merit function, to ensure the global convergence of generated iterates to at least a stationary point, even to solutions satisfying the second order necessary optimality conditions. Unfortunately, the comparison of the merit function values may fail to be precise enough for estimates reaching the square root of the machine precision. We present a clever way to compare merit function values, ensuring that full machine precision may be reached.
unconstrained optimization numerical methods automatic differentiation high-precision computing line search procedures trust region methods