Title: Generalized Kernelized Bandits: Self-Normalized Bernstein-Like Dimension-Free Inequality and Regret Bounds
Venue: NeurIPS 2025
Accepted: no
Keywords: Kernelized Bandits

Generalized Linear Bandits

Concentration Inequality

Bernstein
Abstract: We study the regret minimization problem in the novel setting of *generalized kernelized bandits* (GKBs), where we optimize an unknown function $f^*$ belonging to a *reproducing kernel Hilbert space* (RKHS) having access to samples generated by an *exponential family* (EF) noise model whose mean is a non-linear function $\mu(f^*)$. This model extends both *kernelized bandits* (KBs) and *generalized linear bandits* (GLBs). We propose an optimistic algorithm, GKB-UCB, and we explain why existing self-normalized concentration inequalities do not allow to provide tight regret guarantees. For this reason, we devise a novel self-normalized Bernstein-like dimension-free inequality resorting to Freedman's inequality and a stitching argument, which represents a contribution of independent interest. Based on it, we conduct a regret analysis of GKB-UCB, deriving a regret bound of order $\widetilde{O}( \gamma_T \sqrt{T/\kappa_*})$, being $T$ the learning horizon, $\gamma_T$ the maximal information gain, and $\kappa_*$ a term characterizing the magnitude the reward nonlinearity. Our result matches, up to multiplicative constants and logarithmic terms, the state-of-the-art bounds for both KBs and GLBs and provides a *unified view* of both settings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes and tackles RKHS version of generalized linear bandit (GLB), which unifies both the finite-dimensional GLB and kernelized linear bandits. The main technical novelty is the derivation of a new self-normalized, Bernstein-type concentration bound by combining the \"stitching\" argument of Howard et al. (2021) with Freedman's inequality. Based on this, the authors propose GKB-UCB that attains a regret bound of $\\tilde{O}(\\gamma_T \\sqrt{T / \\kappa_*})$, where $\\gamma_T$ is the maximal information gain and $\\kappa_*$ is the inverse of slope at the optimal arm, reminiscent and strict generalization of the best known regret for finite-dimensional GLB.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- Well-written\n- Technically sound, and the proposed approach to building new concentration seems distinct from prior works.\n\n\n**Weaknesses**\n- No experimental results. Maybe take some inspirations from [1]. I feel that this will strengthen the paper, especially as it highlights efficient implementation of GKB-UCB. I would be interested to see the empirical regret comparison between the non-efficient ver and efficient ver, and other applicable baselines.\n\n\n[1] https://proceedings.mlr.press/v70/chowdhury17a\n\nQUESTIONS:\n1. The regret bounds, when instantiated to finite-dimensional scenarios, seem to have extra logarithmic factors. What is the reason for this? Is this an inherent limitation of the algorithm, or simply an artifact of the current proof?\n2. The authors state that for the analysis purpose, the worst-case radius $B_t(\\delta, \\mathcal{H}) = \\max_{f \\in \\mathcal{H}} B_t(\\delta, f)$ is used. Why is this necessary? I don't recall seeing similar arguments when proving the regret bounds for finite-dimensional GLBs (please correct me if I'm wrong here)\n3. The data-driven Freedman's inequality, although itself seems novel, seems to resemble prior such bounds: [1, Theorem 1], [2, Lemma 3] with simpler proofs. Can the authors elaborate on why the current form of the data-driven Freedman is necessarily tight for this paper's purpose?\n\n\n[1] https://proceedings.mlr.press/v15/beygelzimer11a.html\n\n[2] https://proceedings.mlr.press/v238/lee24d.html\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the regret minimization problem is studied for Generalized Kernelized Bandits (GKBs) to optimize an unknown function $f^*$ that belongs to a Reproducing Kernel Hilbert Space (RKHS), having access to the noise being generated from an exponential family noise source.  The proposed model is an extension to Kernelized Bandits (KBs) and Generalized Linear Bandits (GLBs). An Upper Confidence Bound (UCB) based algorithm is proposed. As tight regret guarantees can not be obtained using existing self-normalized concentration inequalities, the authors have proposed a novel Bernstein-like dimension-free inequality and claimed that it is a contribution of independent interest. An upper bound on the regret is obtained as a function of learning horizon, maximal information gain, and magnitude of reward nonlinearity. This bound matches the state-of-the-art bounds for KBs and GLBs up to multiplicative constants and logarithmic terms.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well-written and technically solid. However, more intuitions need to be provided to understand the impact of the result better. Moreover, at some places, it is difficult to follow due to deferring some of the discussions towards the end and insufficient details regarding a few important concepts. No simulation results are presented.\n\nQUESTIONS:\n1.\tAssumption 3.3 assumes the existence of a bounded noise, which may not hold for practical scenarios, e.g., Gaussian noise. The discussion following Assumption 3.3 needs to be elaborated further for more insight. \n2.\tLemma 5.1 reveals that there is a dependence on the maximum slope of the inverse link function and no dependence on the minimum slope. Please discuss why the former is a better thing to happen.\n3.\tIn deriving Theorem 6.2, it is stated that the dependence on $d$ is removed by replacing it with the norm of the feature map, which is bounded under Assumption 3.1? Why can’t we directly assume that $d$ is finite? Would that be a stronger assumption than Assumption 3.1?\n4.\tThe authors are advised to add a small discussion on the heteroscedastic characteristics of the noise for better readability. \n5.\tFor algorithmic and analysis purposes, two different versions of $B_t(\\delta;f)$ are used. Moreover, another worst-case version is also introduced. This seems a little confusing. Please clarify what purpose do these different versions serve.\n6.\tThe proposed algorithm can be compared using simulations with suitably adapted versions of the existing algorithms for KBs and GLBs.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors proposed a new nonparametric form subsuming both generalized linear models and function approximation in reproducing kernel Hilbert spaces. Under this framework of general function approximation, they adapted weighted ridge regression to this bandit setting so as to achieve a $\\sqrt{T}$ high-probability regret bound using optimism in the face of uncertainty. The authors also came up with an efficient approximation of their algorithm.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- Most mathematical derivations are clear and easy to follow.\n- Theorem 6.2 and its proof align with the intuition of the community of variance-aware online learning.\n- The effort in deriving the efficient implementation in Appendix A sets a good role model for the theory community.\n\n### Weaknesses on Credit Assignment and Over-claiming\n\n- At the message and technical level, the idea of \"kernel + link function\" already appears in the bandit literature, e.g., [1]. In particular, [1] achieved nearly the same regret bound as that in this paper using similar (and even simpler) technical components, in which their \"effective dimension\" is essentially the same as the author's \"maximal information gain\". [1] appeared on arXiv about a week before the submission deadline and the authors did not discuss it in the paper.\n- Table 1 is made to show that this paper is the only one that can tackle heteroskedastic noise besides the work [2]. However, what the authors actually considered is the \"variance-revealing\" case, in which the potentially heteroskedastic variance of the picked arm is given to the learner. When $\\mu$ and $f^*$ are unknown, such an implicit assumption is not entirely reasonable. In contrast, certain papers on linear and generalized linear bandits back to two years again, e.g., [3] and [4], are already able to handle heteroskedastic noise without such type of variance-revealing assumption.\n- Both quantities appear in Assumption 3.1 and Assumption 3.2 should not be treated as \"constants\". In particular, the norm bouund $B$ and the kernel bound $K$ are important problem-dependent quantities. Therefore, \"state-of-the-art\" statement in the Abstract of this paper is an **over-claiming** because the dependency on $B$ and $K$ is not optimal when specialized to generalized linear bandits, given the true SoTA [5], which the authors also mentioned in the conclusion.\n- The authors called the standard peeling technique \"stitching\" and claimed its application to deriving \"empirical\" Freedman to be \"of independent interest\". However, this type of peeling in improving Freedman has been widely used in the variance-aware online learning community. To be concrete, Theorem 9 of [6] and Appendix A of [7] are two examples of such peeling techniques.\n\n### Weaknesses on Rigorousness and Correctness\n\n- When deriving equation (68), which is a key part in proving Theorem 6.2, the authors invoked *their version of elliptical potential lemma*, Lemma C.6. In the last step of deriving Lemma C.6, the authors claimed that is follows from the proof in [8]. However, if we truely follow the derivation of Lemma 12 of [8], two issues arise:\n  1. Even in the finite-dimensional case, the term $\\lambda^{-1}$ on the right-hand side of equation (196) does not really align with $(\\det(\\lambda I))^{-1}$ if the dimension is not one.\n  2. To make matters worse, in the infinite-dimensional case, there is in general no well-defined determinant for an operator. Technically speaking, the only known and widely used notion of determinant for infinite-dimensional real Hilbert space is the Fredholm determinant, which is only applicable for operators of the for $I + A$ where $\\sqrt{A^\\top A}$ has a finite trace. See, e.g., https://en.wikipedia.org/wiki/Fredholm_determinant for details. Therefore, even though equation (196) might intuitively make sense for infinite-dimensional Hilbert spaces, it is not mathematically clear what the authors actually did to achieve the $\\leq$ in equation (196). I suggest the authors to first try to write down **the concrete definition** of determinant of some necessarily involved operators in deriving this inequality (196), instead of citing [8] in a sloppy way.\n\n#### References\n\n[1] Bae, S., & Lee, D. (2025). Neural Logistic Bandits. arXiv preprint arXiv:2505.02069. (https://arxiv.org/pdf/2505.02069)\n\n[2] Faury, L., Abeille, M., Calauzènes, C., & Fercoq, O. (2020, November). Improved optimistic algorithms for logistic bandits. In International Conference on Machine Learning (pp. 3052-3060). PMLR.\n\n[3] Zhao, H., He, J., Zhou, D., Zhang, T., & Gu, Q. (2023, July). Variance-dependent regret bounds for linear bandits and reinforcement learning: Adaptivity and computational efficiency. In The Thirty Sixth Annual Conference on Learning Theory (pp. 4977-5020). PMLR.\n\n[4] Di, Q., Jin, T., Wu, Y., Zhao, H., Farnoud, F., & Gu, Q. (2023). Variance-aware regret bounds for stochastic contextual dueling bandits. arXiv preprint arXiv:2310.00968.\n\n[5] Lee, J., Yun, S. Y., & Jun, K. S. (2024). A unified confidence sequence for generalized linear models, with applications to bandits. arXiv preprint arXiv:2407.13977.\n\n[6] Zimmert, J., & Lattimore, T. (2022, June). Return of the bias: Almost minimax optimal high probability bounds for adversarial linear bandits. In Conference on Learning Theory (pp. 3285-3312). PMLR.\n\n[7] [Li, G., Cai, C., Chen, Y., Wei, Y., & Chi, Y. (2021). Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysis. arXiv preprint arXiv:2102.06548.](https://arxiv.org/pdf/2102.06548)\n\n[8] Abeille, M., Faury, L., & Calauzènes, C. (2021, March). Instance-wise minimax-optimal algorithms for logistic bandits. In International Conference on Artificial Intelligence and Statistics (pp. 3691-3699). PMLR.\n\nQUESTIONS:\n$\\gamma_T$ in this paper is essentially the same notion as the \"effective dimension\" in the neural bandits literature like [1] and corresponds to the dimension $d$ in the linear bandits literature, as stated in [2], right?\n\n### References\n\n[1] Zhou, D., Li, L., & Gu, Q. (2020, November). Neural contextual bandits with ucb-based exploration. In International Conference on Machine Learning (pp. 11492-11502). PMLR.\n\n[2] Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. (2009). Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995.\n\n> By the way, the missing citation of [2] is really not as expected, because this GP-UCB paper proposes the $\\gamma_T$ for the first time in the bandit community.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}