Abstract-We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization due to their projection-free property and their ability to exploit structured constraints. However, our understanding of these algorithms in the nonconvex setting is fairly limited. In this paper, we propose nonconvex stochastic Frank-Wolfe methods and analyze their convergence properties. Furthermore, for objective functions that decompose into a finite-sum, we leverage ideas from variance reduction for convex optimization to obtain new variance reduced nonconvex Frank-Wolfe methods that have provably faster convergence than the classical Frank-Wolfe method.
REF further studied the convergence rate of non-convex stochastic Frank-Wolfe algorithm in the finite-sun optimization setting.
Stochastic Frank-Wolfe methods for nonconvex optimization
2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
Computer Science Mathematics