Stochastic zeroth-order optimization concerns problems where only noisy function evaluations are available. Such problems arises frequently in many important applications. In this paper, we consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in an Euclidean space, an important but less studied area, and propose four algorithms for solving this class of problems under different settings. Our algorithms are based on estimating the Riemannian gradient and Hessian from noisy objective function evaluations, based on a Riemannian version of the Gaussian smoothing technique. In particular, we consider the following settings for the objective function: (i) stochastic and gradient-Lipschitz (in both nonconvex and geodesic convex settings), (ii) sum of gradient-Lipschitz and non-smooth functions, and (iii) Hessian-Lipschitz. For these settings, we characterize the oracle complexity of our algorithms to obtain appropriately defined notions of \epsilon-stationary point or \epsilon-approximate local minimizer. Notably, our complexities are independent of the dimension of the ambient Euclidean space and depend only on the intrinsic dimension of the manifold under consideration. We demonstrate the applicability of our algorithms by simulation results  .
We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve Riemannian optimization problem with only noisy objective function evaluations. Towards this, our main contribution is to propose estimators of the Riemannian gradient and Hessian from noisy objective function evaluations, based on a Riemannian version of the Gaussian smoothing technique. The proposed estimators overcome the difficulty of the non-linearity of the manifold constraint and the issues that arise in using Euclidean Gaussian smoothing techniques when the function is defined only over the manifold. We use the proposed estimators to solve Riemannian optimization problems in the following settings for the objective function: (i) stochastic and gradient-Lipschitz (in both nonconvex and geodesic convex settings), (ii) sum of gradient-Lipschitz and non-smooth functions, and (iii) Hessian-Lipschitz. For these settings, we analyze the oracle complexity of our algorithms to obtain appropriately defined notions of \epsilon-stationary point or \epsilon-approximate local minimizer. Notably, our complexities are independent of the dimension of the ambient Euclidean space and depend only on the intrinsic dimension of the manifold under consideration. We demonstrate the applicability of our algorithms by simulation results and real-world applications on black-box stiffness control for robotics and black-box attacks to neural networks .
R
Stochastic zeroth-order optimization concerns problems where only noisy function evaluations are available. Such problems arises frequently in many important applications. In this paper, we
We
D
an
R
an important but less studied area, and propose four algorithms for solving this class of problems under different settings. Our algorithms are based on estimating
where the task is to solve Riemannian optimization problem with only noisy objective function evaluations. Towards this, our main contribution is to propose estimators of
R
In particular, we consider the
The proposed estimators overcome the difficulty of the non-linearity of the manifold constraint and the issues that arise in using Euclidean Gaussian smoothing techniques when the function is defined only over the manifold. We use the proposed estimators to solve Riemannian optimization problems in the
R
characterize
analyze
A
and real-world applications on black-box stiffness control for robotics and black-box attacks to neural networks