Title: On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
Venue: NeurIPS 2025
Accepted: yes
Keywords: zeroth-order optimization

stochastic optimization

stochastic gradient descent
Abstract: Zeroth-order optimization (ZOO) is an important framework for stochastic optimization when gradients are unavailable or expensive to compute. A potential limitation of existing ZOO methods is the bias inherent in most gradient estimators unless the perturbation stepsize vanishes. In this paper, we overcome this biasedness issue by proposing a novel family of *unbiased* gradient estimators based solely on function evaluations. By reformulating directional derivatives as a telescoping series and sampling from carefully designed distributions, we construct estimators that eliminate bias while maintaining favorable variance. We analyze their theoretical properties, derive optimal scaling distributions and perturbation stepsizes of four specific constructions, and prove that SGD using the proposed estimators achieves optimal complexity for smooth non-convex objectives. Experiments on synthetic tasks and language model fine-tuning confirm the superior accuracy and convergence of our approach compared to standard methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a framework for constructing unbiased estimators of the true directional derivative of a smooth function using only zeroth-order queries. The authors reformulate the directional derivative as an infinite series and, leveraging the absolute convergence of this series, derive an expectation-based formulation. Within this framework, they propose and analyze specific estimators, providing theoretical guarantees on the expected stationarity gap in the context of non-convex stochastic optimization.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The paper introduces a general framework for analyzing a class of zeroth-order estimators that estimate directional derivatives in an unbiased manner. This provides a unifying perspective on various estimation strategies.\n2. The observation that an unbiased single-point estimator of the true gradient can have infinite variance—contrasting with the bounded variance of zeroth-order estimators for smoothed functions—is both surprising and insightful.\n\n**Weaknesses:**\n\n1. The contributions and practical implications of the work are not clearly articulated. The motivation for introducing this framework, as well as its advantages over existing methods, remains vague. Furthermore, the writing lacks a clear sense of direction regarding potential applications or future developments based on the proposed theory.\n2. The proposed technique relies on the construction of a countably infinite set $p_n$ and corresponding smoothing parameters $\\mu_n$ for $n = 1, \\cdots, \\infty$. It is not immediately clear how this formulation translates into a practical implementation. Further clarification on how this can be approximated or truncated in practice would be valuable. If truncation is necessary, then using standard smoothed gradient estimators—which are better understood and widely adopted—might be more practical and theoretically grounded.\n3. The experimental setup is limited in scope. To better demonstrate the practical utility of the proposed method, it would be beneficial to include experiments on larger-scale models or more diverse benchmark tasks that highlight tangible improvements over existing approaches.\n\nQUESTIONS:\n1. The result that unbiased estimators can attain the same level of variance in the smooth setting as standard single- or two-point estimators is certainly interesting. However, it remains unclear how this observation contributes either theoretically or practically. Specifically, does exact unbiasedness offer any concrete advantages over the existing analyses of zeroth-order optimization that rely on the gradient of a smoothed function?\n\n2. The construction of these estimators involves selecting a countably infinite set of distributions $p_n$ and corresponding smoothing parameters $\\mu_n$. It is unclear how this construction guarantees exact unbiasedness in practice. Could this be interpreted instead as an approximation or a limiting form of unbiasedness? If so, why would this approach be preferable to, or more suitable than, using standard smoothed gradient estimators in algorithmic applications?\n\nLIMITATIONS:\nYes. The authors have adequately addressed the limitations of this work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-13"}

comments: SUMMARY:
The central contribution of this paper is the introduction of a novel family of provably unbiased zeroth-order gradient estimators, which the authors term Pk-estimators (where k = 1, 2, 3, 4 denotes the number of function queries required). A detailed variance analysis reveals that while the P1 estimator may exhibit infinite variance, the P2, P3, and P4 estimators are shown to possess finite variance under certain conditions.

STRENGTHS AND WEAKNESSES:
- The paper's objective is clearly articulated, and the manuscript is well-written, making the core concepts and contributions accessible.

- The novelty of the proposed technique is a point of concern, as it appears to share similarities with methods presented in Chen (2023). Furthermore, a more robust justification for the pursuit of an unbiased estimator is needed. Given that the resulting optimization algorithm demonstrates a convergence rate comparable to methods using biased estimators, what is the fundamental advantage of this approach? Why should one prefer this unbiased estimator over a simpler, biased alternative?

QUESTIONS:
Clarification is requested regarding the plots in Figure 2. The x-axis of the right panel is labeled "function calls," while the left panel shows the "number of updates." For the one-sided and two-sided two-point estimators, one would expect the total number of function calls to be twice the number of gradient updates. However, the plots for these estimators conclude at approximately 80,000 function calls respectively, rather than the expected 40,000 corresponding to 20,000 updates. Could you please explain this apparent discrepancy? If I am wrong, please correct me.

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 4
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-22

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes new  unbiased estimates of the directional derivative, based on a telescopic series representation of the directional derivative. The paper is very well written, clear, and the proofs are easy to follow. My only real concern is how this work fits within Neurips. I understand there is still plenty of legacy code (often written in Fortran) for simulations that requires zero order training. But here the connection to Neurips + ML is through the proposed application of model fine-tuning, and to save of memory when using autodiff. But here I have a question/issue. See [Application to LLM fine-tuning] below. Based on the response of the authors on this question, and the other reviewers, I am open to reconsidering my score.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is very clear, the quality of the writing, and proofs is high. The unbiased zero order estimators are mostly new, excluding a 2020 paper by Chen.\n\nQUESTIONS:\n1. The reference lower bound in [Duchi et al. 2015] is for zero-order optimization, and not first-order. The fact that you have an unbiased estimator, means you can leverage the proof structure of SGD, such as the one in [Khaled and Richtarik]. Because of this, your statements in the main text and abstract such as \"... prove that SGD using the proposed estimators achieves optimal complexity for smooth non-convex objectives\" can be misleading. A reader could interpret that you have matched the optimal complexity of \"SGD for smooth non-convex objectives\" which is \n\n\n$$\\frac{ \\sqrt{2} L (f(x^0) - \\inf f) + (\\inf f - \\mathbb{E}[\\inf f\\_{\\xi}])}{\\sqrt{T}} $$\n\nsee Theorem 5.12 in \nGarrigos et. al, Handbook of Convergence Theorems for (Stochastic) Gradient Methods, arXiv, 2024\n\nI think you should re-word this contribution, to make it clear you are talking about the optimal zero-order complexity.\n\n2. How do your methods compare to other zero order methods used for fine-tuning, such as Mezo in [Malladi et al 2023]? Or is Mezo equivalent here to what you call Two-side point estimator? On this note, I see a small mistake in your Two-side point estimator in Table 1. You are off by a factor of 2, since it should be\n$$ \\frac{f(x+\\mu v_i)- f(x+\\mu v_i)}{2\\mu} v_i$$\nThis same mistake is also repeated on page 14 in three locations. \nA factor of two would make no difference if you tuned the learning rate. But since you have a fixed $\\eta = 10^{-4}$, this factor could make a difference.\n\n\n\n[Application to LLM fine-tuning]. To justify this application, you have cited some prior work, and stated \"storing the full computational graph ... can be prohibitively large for modern language models\". I checked your references, and only one [Malladi et al 2023] has been peer reviewed. There, their claims is the using zero-order uses a fraction of the memory of backpropagtion. This I agree with, since backpropagation requires storing all weights, and adjoints of your computational graph. But here is where I see an issue. These zero order methods should instead be compared to the cost/memory of computing the directional derivative $\\nabla_v f(x)$ when using forward propagation, which can be done with a very small memory overhead. To justify the case of using zero-order for fine-tuning, you would need to compare to using forward prop for computing $\\nabla_v f(x)$, and compare both the time, memory usage, and the resulting performance. Otherwise I don't think it is a compelling argument to apply these methods to fine-tuning.\n\n### Forward-Mode AD: Flow and Memory Usage\n\nIn forward-mode automatic differentiation, to compute the directional derivative $\\nabla f(x)^T d $, we propagate both the primal values $ x_\\ell $ and tangent vectors $\\dot{x}_\\ell $ through each layer of the function composition $f = f_L \\circ \\cdots \\circ f_1 $:\n\n$x_{\\ell} = f_{\\ell}(x_{\\ell-1}),$\n\n$\\dot{x}\\_\\ell = J\\_\\ell(x\\_{\\ell-1}) \\dot{x}\\_{\\ell-1}$\n\nwhere $J\\_{\\ell}$ is the Jacobian of the $\\ell$th layer.\n**Memory usage** is minimal: only the current pair $(x_{\\ell-1}, \\dot{x}_{\\ell-1})$ must be stored to compute the next. Both values and tangents can be discarded once the next layer is evaluated, making forward-mode memory-efficient for directional derivatives.\n\n\n**Minor points**\n\n1. Line 625, if $a_n = \\mu_{n+1}-\\mu_n$ then $\\sum_{n=1}^{\\infty} a_n = -\\mu_1$ instead of $\\mu_1.$ The proof here is nonetheless correct\n\nLIMITATIONS:\nNo explicit mention of limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

comments: SUMMARY:
Authors propose new way to build unbiased estimate of gradient via gradient-free setup. The idea is interesting and according to authors claim propose for free a way to eliminate bias (by conserving the variance estimate).

STRENGTHS AND WEAKNESSES:
So the paper is well written and propose a useful way to reduce bias. But I'm not sure that this way leads to the same variance at the end as it will be for biased case, see e.g. Tsybakov A. et al. http://www.jmlr.org/papers/volume25/23-0733/23-0733.pdf or Gasnikov A. V. et al. Stochastic online optimization. Single-point and multi-point non-linear multi-armed bandits. Convex and strongly-convex case // Automation and remote control. – 2017. – V. 78. – P. 224-234. Roughly speaking, for me it seems that $d^3\mu^2$ terms could be improved as $d^2\mu^2$. May be it's worth to include more accurate discussion about this.

QUESTIONS:
Why do authors use such a finite difference? It seems that symmetric one will be better
Shamir O. An optimal algorithm for bandit and zero-order convex optimization with two-point feedback //Journal of Machine Learning Research. – 2017. – V. 18. – №. 52. – P. 1-11.

LIMITATIONS:
Main limitation-- if $f$ is available with noise like in Gasnikov A. et al. The power of first-order smooth optimization for black-box non-smooth problems //International Conference on Machine Learning. – PMLR, 2022. – V. 7241-7265. In this case the construction seems to be quite sensitive and may work worse than classical one approach.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors considered the problem of zeroth-order optimization (ZOO) in a stochastic optimization setup. They argued that most existing ZOO methods have a biased estimate of the gradient. Therefore, they proposed a family of unbiased estimators of the gradient based on merely function evaluations. In particular, they constructed four unbiased estimators that depend on 1 to 4 function evaluations in estimating the gradient. The main idea in devising these estimators is to write $\\nabla_v f(x)$ in a direction of vector $v$ as a telescoping series of gradient estimators with diminishing perturbation stepsize $\\mu_n$. This formulation provides an unbiased estimator given in (6) (which is called $\\text{P}_4$ estimator). Moreover, they suggested three other estimators with a reduced number of function evaluations. Under some regularity assumption on the objective function, they showed that the proposed family of the estimator is unbiased. Moreover, they gave upper bounds on the variance of gradient estimators in terms of perturbation step sizes $\\{\\mu_n\\}$ and sampling distribution $\\{p_n\\}$. In particular, for the estimator with one evaluation function, the variance might be unbounded, but for the rest of the estimators, by tuning $\\mu_n$ and $p_n$, the variances match the optimal order of classical two-point estimators. Moreover, for ease of implementation, they proposed a method that, given a sampling distribution, derived the perturbation stepsize, and they gave a formulation for the perturbation step size for two distributions of Geometric and Zipf distributions. The experimental results showed improvement over some previous work on synthetic functions and training of a language model.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The authors proposed a family of unbiased estimators of the gradient in the ZOO setting. It seems that there is only one unbiased estimator in the literature by (Chen 2020), which was a special case of $\\text{P}_4$ estimator.\n\n- The authors provided theoretical guarantees on the unbiasedness of the proposed family of estimators and also gave upper bounds on the variance for the three estimators mentioned in Section 2.3.\n\n- They derived optimal sampling distributions for given perturbation step sizes that minimize the upper bound on the variance (Theorem 3.2). They also provided an efficient implementation of the estimator by fixing first the sampling distribution and then adjusting the perturbation step sizes accordingly. \n\n### Weaknesses:\n- The motivation behind the unbiased estimator is not well explained. In fact, it is not clear what the advantages of having an unbiased estimator in the ZOO setup are.\n\n- The assumption on the objective function being a second-order continuously differentiable function is not well motivated, and it is not clear in which theorems this assumption is needed. Apparently, it is needed in all theorems and propositions, but it is not explicitly mentioned in the statements. I think this assumption is also restrictive, as neural networks with ReLU activation may not satisfy this assumption.\n\n- In the experiments, it seems that the authors did not compare with the closest related work of (Chen 2020).\n\nQUESTIONS:\n- What is the motivation behind having an unbiased estimator of the gradient? For instance, in the literature of variance reduction methods in stochastic optimization, most proposed methods are biased estimators while they reduce the variance. Overall, they can achieve the best known sample complexity while the estimator is biased.\n\n- Compared to biased estimators in the ZOO setup, what are the advantages of the current work in terms of sample complexity? It is good to clarify any advantages in the paper.\n\n- The explanation in lines 89-97 is not so clear to me. In particular, would you please explain more about ``the outcomes of a random variable have no naturally given order (which makes it different from a series), which requires a random variable’s expectation 95 to be well-defined regardless of any such ordering''. It seems that the assumption in Proposition 2.1 is needed because of this argument, which limits the applicability of the current work.\n\n- Would you please compare the current method with (Chen 2020) in Figures 1 and 2? Moreover, it is also good to add optimization algorithms such as SGD and Adam in these figures as a reference.\n\n- Could you elaborate on the impact of sampling distribution on the performance of the estimator or the optimization algorithm in general? In particular, in what cases, which one of the two considered sampling distributions should be used?\n\nLIMITATIONS:\nThe assumptions are mentioned in the paper, but it is good to explain what the impacts of these assumptions are and for what applications the proposed estimators can be used. It seems that there was no discussion on the potential negative societal impact of the work. It is good to add any societal impacts or otherwise, mention that there is no immediate impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}