We propose and analyze zeroth-order algorithms for optimization over Riemannian manifolds, where we observe only potentially noisy evaluations of the objective function. Our approach is based on estimating the Riemannian gradient from the objective function evaluations . We consider three settings for the objective function: (i) deterministic and smooth , (ii) stochastic and smooth , and (iii) composition of smooth and non-smooth parts. For each of the setting , we characterize the oracle complexity of our algorithm to obtain appropriately defined notions of \epsilon-stationary points . Notably, our complexities are independent of the ambient dimension of the Euclidean space in which the manifold is embedded in, and only depend on the intrinsic dimension of the manifold . As a proof of concept, we demonstrate the applicability of our method to the problem of black-box attacks to deep neural networks, by providing simulation and real-world image data based experimental results.
Stochastic zeroth-order optimization concerns problems where only noisy function evaluations are available. Such problems arises frequently in many important applications. In this paper, we consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in an Euclidean space, an important but less studied area, and propose four algorithms for solving this class of problems under different settings. Our algorithms are based on estimating the Riemannian gradient and Hessian from noisy objective function evaluations , based on a Riemannian version of the Gaussian smoothing technique. In particular, we consider the following settings for the objective function: (i) stochastic and gradient-Lipschitz (in both nonconvex and geodesic convex settings) , (ii) sum of gradient-Lipschitz and non-smooth functions , and (iii) Hessian-Lipschitz. For these settings , we characterize the oracle complexity of our algorithms to obtain appropriately defined notions of \epsilon-stationary point or \epsilon-approximate local minimizer . Notably, our complexities are independent of the dimension of the ambient Euclidean space and depend only on the intrinsic dimension of the manifold under consideration. We demonstrate the applicability of our algorithms by simulation results.
R
We propose and analyze
Stochastic
R
algorithms for optimization over Riemannian manifolds, where we observe only potentially noisy evaluations of the objective function. Our approach is
optimization concerns problems where only noisy function evaluations are available. Such problems arises frequently in many important applications. In this paper, we consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in an Euclidean space, an important but less studied area, and propose four algorithms for solving this class of problems under different settings. Our algorithms are
R
from the
and Hessian from noisy
R
. We consider three
, based on a Riemannian version of the Gaussian smoothing technique. In particular, we consider the following
R
deterministic and smooth
stochastic and gradient-Lipschitz (in both nonconvex and geodesic convex settings)
R
stochastic and smooth
sum of gradient-Lipschitz and non-smooth functions
R
composition of smooth and non-smooth parts. For each of the setting
Hessian-Lipschitz. For these settings
R
algorithm
algorithms
R
points
point or \epsilon-approximate local minimizer
D
ambient
R
Euclidean space in which the manifold is embedded in, and only depend
ambient Euclidean space and depend only
R
. As a proof of concept, we
under consideration. We
R
method to the problem of black-box attacks to deep neural networks, by providing simulation and real-world image data based experimental
algorithms by simulation