Title: Derivative-Free Optimization via Monotonic Stochastic Search
Primary area: optimization
Keywords: Derivative-Free Optimization, Zeroth-order optimization

Abstract: We consider the problem of minimizing a differentiable function $f:\mathbb{R}^d \to \mathbb{R}$ using only function evaluations, without access to gradients. This zeroth-order (derivative-free) optimization setting arises frequently in machine learning applications where computing gradients is impractical or impossible. We propose two novel stochastic derivative-free algorithms: the Monotonic Stochastic Search (MSS) algorithm and its gradient-approximation variant (MSSGA). Our first method, MSS, extends deterministic direct search (DDS) to the stochastic setting by sampling a single random perturbation at each iteration and accepting a new point only if it decreases the function value. We show that MSS achieves, in expectation, a convergence rate of $\mathcal{O}(\sqrt{d}/\sqrt{T})$ in the smooth non-convex case. This improves upon the $\mathcal{O}(d/\sqrt{T})$ complexity of DDS methods and matches the best known bound among derivative-free methods. For non-smooth objectives, we further prove that when the objective is continuously differentiable, the MSS iterates admit an accumulation point that is stationary. The second algorithm, MSSGA, builds on directional derivative approximation techniques while enforcing monotonic descent. We establish theoretical guarantees for MSSGA in non-convex, convex, and strongly convex settings, under the smoothness assumption, with complexity bounds matching the best known results for zeroth-order methods without momentum. Finally, we strengthen existing results by proving the best known almost-sure convergence rate of $o(1/\sqrt{T})$ in the smooth non-convex case for stochastic derivative free algorithms.