We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.
REF showed that even without adding noise, in the limit gradient descent converges to (second order) local minimum with random initialization.
Gradient Descent Converges to Minimizers
ArXiv
ArXiv
Mathematics Computer Science