Title: Gaussian Process Upper Confidence Bound Achieves Nearly-Optimal Regret in Noise-Free Gaussian Process Bandits
Venue: NeurIPS 2025
Accepted: yes
Keywords: Gaussian process bandits

regret analysis

near optimality
Abstract: We study the noise-free Gaussian Process (GP) bandit problem, in which a learner seeks to minimize regret through noise-free observations of a black-box objective function that lies in a known reproducing kernel Hilbert space (RKHS).
The Gaussian Process Upper Confidence Bound (GP-UCB) algorithm is a well-known approach for GP bandits, where query points are adaptively selected based on the GP-based upper confidence bound score.
While several existing works have reported the practical success of GP-UCB, its theoretical performance remains suboptimal. However, GP-UCB often empirically outperforms other nearly-optimal noise-free algorithms that use non-adaptive sampling schemes.
This paper resolves the gap between theoretical and empirical performance by establishing a nearly-optimal regret upper bound for noise-free GP-UCB. Specifically, our analysis provides the first constant cumulative regret bounds in the noise-free setting for both the squared exponential kernel and the Matérn kernel with some degree of smoothness.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper improves upon the regret analysis of [Iwazaki and Takeno, 2025a] for the Gaussian Process Upper Confidence Bound (GP-UCB) algorithm in the noise free GP bandits. In particular, the authors consider squared exponential and Matérn kernels, where they manage to improve the existing upper bounds on regret.\n\nSTRENGTHS AND WEAKNESSES:\nI find the problem addressed in the paper to be important, and the proposed method interesting, particularly because it enables the translation of noisy-regime guarantees to the noise-free setting. The paper is clearly written and generally easy to understand. Moreover, the technique has the potential to be generalized to a broader class of kernels.\n\nHowever, I find the contribution, in its current form, insufficient for publication. While the refined analysis is valuable, it does not appear to be technically challenging and relies heavily on existing works. The scope of applications seems limited, and the difference from the prior work [Iwazaki and Takeno, 2025a] appears to be incremental.\n\nIn addition, I have concerns regarding the originality of the writing. Specifically, several parts of the manuscript—including the formulations of main theorems: Theorem 1 and Theorem 2, Table 1 and Table 2 (including their captions), Assumption 1, problem setting, and the subsections “Gaussian Process Model” and “Maximum Information Gain”—appear almost identical to the corresponding sections in [Iwazaki and Takeno, 2025a], without proper acknowledgment. This raises potential plagiarism concerns that should be carefully addressed.\n\nQUESTIONS:\nI suggest extending the applications to other kernels, such as the Neural Tangent Kernel (NTK), which would broaden the impact and relevance of the proposed technique.\n\nI find the presentation of the paper to be visually very similar to [Iwazaki and Takeno, 2025a]. I recommend revising the manuscript to better emphasize the originality of the contribution and to avoid potential plagiarism.\n\nAdditionally, in the contributions section, it is stated that GP-UCB achieves constant regret for $d > \\nu$; however, according to Table 1 and as also indicated by Figure 1, it should be $d < \\nu$. I assume this is a typo.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n**Setting**  \nThis paper considers the noise-free Gaussian Process Bandit setting, where in each round, an arm $x_t$ is selected and the corresponding reward is $f(x_t)$. The function $f(\\cdot)$ is unknown, and only function values can be observed. Since the observation is exactly the function value, this constitutes a noise-free scenario. Two performance measures are considered: one is the standard cumulative regret  and the other is the simple regret of the final round. The paper analyzes two types of kernel functions: the squared exponential (SE) kernel and the Matérn kernel.\n\n**Approach**  \nThe paper does not propose a new algorithm, but instead provides a refined theoretical analysis for the classical GP-UCB method. In the case of the squared exponential kernel, it improves the previously known regret bound from $\\mathcal{O}(\\ln T)$ to $\\mathcal{O}(1)$. For the Matérn kernel, the analysis also yields improved regret bounds. The theoretical improvement mainly comes from a tighter control of the posterior standard deviations, specifically the term $\\sum_{t=1}^T \\sigma(x_t| X_{t-1})$.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**:\n\n- The theoretical analysis is very solid, with clearly stated improvements and contributions.\n- A key strength of the paper is its effective exploitation of structural properties in the noise-free setting. Previous works that considered the noise-free case typically focused on refining the estimation error $\\beta$, which, in the absence of noise, becomes a constant instead of a time-dependent term, thereby improving the regret order from a factor of $\\log T$. This paper goes further by observing that the cumulative posterior standard deviation $\\sum_{t=1}^T \\sigma(x_t, X_{t-1})$ can also be more tightly controlled in the noise-free setting. This observation is very reasonable: in the noise-free case, once an arm $x$ is queried, its uncertainty $\\sigma(x)$ drops immediately to zero. Therefore, each arm only needs to be explored once to determine its value, significantly reducing the amount of exploration required.\n\n**Weakness**:\n\nI did not find any obvious weaknesses. If the theoretical results hold as claimed, I believe this is a highly meaningful and valuable contribution.\n\nQUESTIONS:\nI'm confused about why GP-UCB can achieve exponentially decaying simple regret, since the UCB strategy is fundamentally designed for balancing exploration and exploitation, rather than for pure exploration. In the context of pure exploration or Best Arm Identification (BAI), wouldn't existing BAI strategies be more suitable or more effective than UCB? Are there BAI algorithms that can achieve even better convergence rates? Or are UCB-based methods and pure exploration algorithms essentially equivalent in this noise-free setting?\n\nLIMITATIONS:\nThe main limitation lies in the fact that the analysis is restricted to only two kernel functions, and there is no algorithmic innovation throughout the paper. The contribution is primarily a refined theoretical analysis of an existing algorithm rather than the proposal of a new method. But I believe the contribution of this work is sufficient.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper resolves a significant gap between the empirical performance and theoretical understanding of the Gaussian Process Upper Confidence Bound (GP-UCB) algorithm for noise-free bandit optimization. Although GP-UCB is a popular and practically effective adaptive algorithm, its existing regret guarantees were suboptimal. This work provides a new, refined analysis that proves GP-UCB achieves nearly-optimal regret, aligning its theoretical guarantees with its strong empirical results. The core technical contribution is a new, algorithm-independent upper bound on the sum of posterior standard deviations, derived by connecting analysis techniques from the noisy GP setting (specifically, the Maximum Information Gain) to the noise-free case.\n\nThe new analysis establishes significantly tighter regret bounds that match known lower bounds up to polylogarithmic factors. For cumulative regret, the paper shows that GP-UCB achieves constant $O(1)$ regret for the squared exponential (SE) kernel, and different near-optimal rate for the Matérn kernel depending on the relationship of the input dimension $d$ and the smoothness parameter $v$ (summarized in Table 1). For simple regret, the analysis provides rates of $O(\\sqrt{T} exp(-\\frac{1}{2} CT^{\\frac{1}{d+1}})$ for the SE kernel, and $\\tilde{O}(T^{-\\frac{v}{d}})$ for the Matérn kernel (summarized in Table 2). Based on the summary in the literature review, these two results advance the state-of-the-art regret bounds and are near-optimal compared to the best-known lower bounds.\n​ \nBecause the main technical lemmas are algorithm-independent, this work not only solidifies the theoretical foundation of GP-UCB but also provides a powerful new tool that could be used to improve the analysis of many other GP-based bandit algorithms.\n\nI have checked the proofs in the main paper and the proofs are sound to me. The new analysis on the posterior standard deviation is also neat and new to me. I think the paper is a solid contribution to the analysis of GP-UCB regret with a simple yet new idea.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Very clear presentation and neat idea.\n- Solid improvement of the regret bound in both the simple and cumulative settings.\n- Very clear summary of the literature\n\nNeutral feedback:\n- As the authors mention, the idea can also be applied to other kernels. It would be great if the results of other kernels can also be shown or briefly summarized in the paper.\n\nQUESTIONS:\nNothing specific in my mind. The paper is very clear and well-structured. It would be great if the authors can also briefly share the results of applying the same idea to RBF and linear kernels, and if this idea advances the best-known regret bounds for these kernels.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}