Title: Exploration via Feature Perturbation in Contextual Bandits
Venue: NeurIPS 2025
Accepted: yes
Keywords: Generalized Linear Bandits

Contextual Bandits

Thompson Sampling

Feature Perturbation
Abstract: We propose *feature perturbation*, a simple yet effective exploration strategy for contextual bandits that injects randomness directly into feature inputs, instead of randomizing unknown parameters or adding noise to rewards. Remarkably, this algorithm achieves $\widetilde{\mathcal{O}}(d\sqrt{T})$ worst-case regret bound for generalized linear contextual bandits, while avoiding the $\widetilde{\mathcal{O}}(d^{3/2}\sqrt{T})$ regret typical of existing randomized bandit algorithms. Because our algorithm eschews parameter sampling, it is both computationally efficient and naturally extends to non-parametric or neural network models. 
We verify these advantages through empirical evaluations, demonstrating that feature perturbation not only  surpasses existing methods but also unifies strong practical performance with the near-optimal regret guarantees.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies a contextual bandit algorithm adapted to Gaussian Linear Models (i.e. the density of the reward given context $x$ and parameter of interest $\\theta^\\star$ has a density exp$(r x^T \\theta^\\star - g(x^T \\theta*^\\star) + h(r))$ and a mean being the derivative $\\mu$ of the link function $g$). The novelty of the proposed method is to add randomness in the features instead of adding it to the reward. The contribution of the paper is to achieve the same regret bound as optimism-based algorithms ($O(d\\sqrt{T}), d$ being the dimension of the context space) while relying on a randomized-like approach and similar practical performance (better in practice). Hence, the paper claims to bridge the gap between theoretical and practical performances of these two kinds of approaches, \"By shifting the focus of exploration from parameter space to feature space\", line 33.\n\nSTRENGTHS AND WEAKNESSES:\nI would summarize the strengths of the paper as follows:\n- the paper is clear and goes straight to the point, which is pleasant.\n- many experimental setups are provided\n\nHowever, I believe that the following point represent weaknesses of the paper:\n- the idea of randomizing the features is nice but it does not outperform the state-of-the-art guarantees for contextual bandits and does not  induce major technical challenges, see below.\n- the theoretical work in the proofs is mostly a combination of pieces of proofs that already exist, typically, almost all the elements in appendix \"G - Proof supplement and Lemmas\" come from existing results. Once they are given, combining them in the proof of Theorem 1 does not appear extremely challenging.\n- all together, the contributions of the paper seem quite incremental to me.\n\nQUESTIONS:\nThe authors suggest that it can happen to work with overparametrized bandits, is it actually the case in practice?\n\nIt might be a stupid question but since $\\mu$ is Lipschitz, is there a form of equivalence (or maybe inequalities) between randomly perturbing $x_{ti}$ and randomly perturbing the reward $\\mu(x_{ti}\\hat{\\theta}_t)$?\n\nLIMITATIONS:\nIt is more a suggestion but I have the feeling that section 4.2 is very important to understand the method (and interesting in itself): I think that it could be even more detailed.\n\nPart 6 of the main text is interesting (and almost represents the most important part of the paper to me) but I believe that it could be made clearer (for instance, elements from B.2. could be added there).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes feature perturbation, a novel and simple technique that injects noise in the feature space. Contrary to known randomised bandit algorithms, this new algorithm achieves the worst case regret bound for generalised linear bandits. The paper also tries to provide the intuition behind the superior regret of the proposed algorithm compared to TS-like methods. The proposed approach is straightforward to apply, even for deep neural networks, and shows strong empirical performance compared to existing methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The paper is well written and accessible.\n- The proposed method closes a known gap between OFU based methods and TS-like method for GLMs.\n- The method is simple and even applies to overparametrised models.\n- The method shows good empirical performance.\n\n\n**Weaknesses:**\n- In the linear setting, feature perturbation is formally equivalent to TS with a well chosen noise. The paper tries to give intuition why FP has a lower regret than TS in this case but this needs to be clarified more. See questions please.\n\nQUESTIONS:\nIn the linear case, the algorithms of TS and FP are equivalent with an appropriate choice of noise. \n\nIn the discussion, the authors state that \" To achieve this (TS regret), a union bound over d coordinates is typically applied, introducing an additional $\\sqrt{d}$ factor into the regret analysis.\" \n\n- What do the authors mean here? Is $\\sqrt{d}$ an artefact of the proof?\n- In the linear case, what are the main algorithmic differences between TS and FP? If the algorithm is the same, why the regret is different?\n\nThese points should be clarified more in the paper, and the emphasis should be on **why we are getting an improved regret**. \n\nIn addition, a relatively recent paper [1] discusses conditions where TS can achieve a regret with optimal dependence on $d$. A discussion of these results and comparison with the obtained guarantees for FP should be included.\n\nClarifying these points will increase my confidence.\n\n[1] When and why randomised exploration works (in linear bandits). Marc Abeille, David Janz, Ciara Pike-Burke\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n* This paper describes a new bandit learning algorithm by injecting noise directly into the context vectors. Specifically, at each round, the algorithm adds a random gaussian vector sample to every arm's feature vector.\n* The paper provides a tight regret bound ($\\widetilde{O} (d \\sqrt{T}) $), matching the best-known rates without relying on arm counts or parameter-space sampling.\n\nSTRENGTHS AND WEAKNESSES:\n---\nQuality (3): \n* Strength: The proposed method is easy to implement, and new in the specific line of contextual bandit literature. The proposed regret analysis is tight and does not dependent on the number of ars.\n* Weakness: Overall, computing and inverting the weighted Gram matrix may be very costly, which potentially hinders the scalability.\n\n---\n\nClarity (3):\n* Strengths: The paper is well-written and easy to read. The figures are clearly plotted.\n* Weaknesses: The authors may consider improving their notations for better readability. For example, using \\mathbf for random vectors, \\bm for vectors. In addition, adding toy calculation in $d = 2$ may be helpful to illustrate the analysis of the confidence ellipsoid.\n\n---\n\nSignificance (3):\n* Strengths: It matches the best known bound for GLMs and directly extends to last-layer features in deep nets. This is a non-trivial improvement.\n* Weaknesses: It is unclear how the principles/insights in this paper can be applied to other domains or data type (e.g., images, language). It would be nice to add more discussions related to that. Also, it may be helpful to add more discussions on how the proposed method and analysis can shed light on other domains.\n\n---\n\nOriginality (3): \n* Strengths: As the authors mentioned, adding noise to features is relatively new in the domain of contextual bandits.\n* Weaknesses: Feature perturbation is commonly used in other areas like computer vision. Please consider adding more discussion on such related works.\n\nQUESTIONS:\n* Can the proposed method and analysis be applied to satisficing bandits and other domains in ML?\n* What happens if the link function is not self-concordant (e.g., heavy-tailed rewards)?\n\nLIMITATIONS:\nThe paper briefly discusses its limitations in Appendix D, admits that the proposed analysis may not work for more general cases like non-GLM. It would be nice to discuss some future works beyond the current package.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n1. Authors propose a new randomized algorithm for generalized linear contextual bandits (GLB) based on feature perturbation.\n2. Prior randomized algorithms for linear bandits (LB) and GLB perturb model parameters instead and incur regret that scales as $d^{3/2}$ where $d$ is dimensionality of the feature space. On the other hand, the proposed algorithm's regret scales linearly with $d$.\n3. By using appropriately scaled gram matrices, the authors are also able to remove unwanted dependency on a non-linearity parameter, similar to other recent deterministic GLB algorithms. However, they claim this has not been achieved by randomized algorithms in the literature.\n4. They provide theoretical analysis of their regret guarantee (outlining the assumptions made) and also demonstrate strong empirical performance in multiple settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths of the paper:\n\n1. Introduces a new randomized algo paradigm for contextual bandit problems. Since it is based on feature perturbation it is advantageous to Thompson sampling (TS) in many scenarios where computing posterior distributions could be challenging.\n\n2. Provides optimal regret guarantees that match the state-of-the-art deterministic algos and is a $\\sqrt{d}$ factor better than TS algos.\n\n3. The writing is decent. Intuition in 4.2, proof sketch in 5 and comparison to TS (for LB) in Section 6 are all helpful. Though I think 4.2 and 6 can be made more concise and crisper. Another feedback here would be to make the contributions section more compact and focused mainly on the results. E.g., bullet 3 and 4 can be merged to focus on empirical results.\n\n4. A reasonable amount of empirical validation is provided covering many known baselines. However, some more recent OFU based algorithms focused on GLB seem to be missing (See below).\n\nWeaknesses:\n\n1. While the algorithm is a new randomized algorithm for GLB, essentially (in its arm selection policy) it multiplies the exploration bonus of OFU algorithms (such as LinUCB [1], Logistic-UCB-1, Logistic-UCB-2[13]) with a random scalar (also mentioned in the paper in line 256). This in my opinion makes the algorithm and its analysis pretty much like the OFU counterparts. Therefore, it can be argued while the algorithm is randomized, it's not much different from OFU algorithms. This reduces the novelty aspect of the paper (a little) in my opinion.\n\n2. The main idea is essentially captured in the Linear Bandit problem. To me there does not seem to be any specific new contributions for GLB. Both the confidence bound (wrt scaled gram matrices) and the regret calculation thereof is pretty much what prior works ([13], [3], [A]) have already done to achieve (1/$\\kappa$ independence). Hence, even though this is the first randomized algorithm with regret that is independent of the non-linearity, the paper does not make novel contributions towards that. The authors can please correct me about this, if they think I am mistaken.\n\n3. In the empirical comparisons, I think the authors should also compare to state-of-the-art logistic bandit algorithms such as ada-OFU-EcoLog [A] and subsequent improvements to it ([B], [C] etc). Empirically these seem to be doing really well in the logistic case compared to the OFU and TS algorithms that are among the baselines in this paper. Also, in the experiments section, the authors should explain the instances a little bit.\n\n4. The paper does not discuss the limitations of feature perturbation. For eg., in large action spaces, this would amount to scanning the entire action space before selecting the action to play. The TS algorithm on the other hand can be more efficient by using Linear Optimization oracles? There is literature on deterministic algorithms which explores large action spaces using such oracles [D].\n\n5. In Assumption 1, authors assume both the features and parameters are inside the unit ball. While this is fine for LB, for GLB one would typically assume the parameter space to be contained in an S-ball and the regret should not scale badly with $S$ as $exp(S)$ can be proportional to the non-linearity $1/\\kappa$. I'm assuming that such dependence does not come in the proof. It would be great if the authors can confirm.\n\n6. The self-concordance assumption in Assumption 2 might not be explicitly needed. See [B].\n\n[A] Jointly Efficient and Optimal Algorithms for Logistic Bandits, L Faury, M Abeille, KS Jun, C Calauzènes, AISTATS 2022.\n\n[B] Generalized Linear Bandits with Limited Adaptivity, A Sawarni, N Das, S Barman, G Sinha, NeurIPS 2024.\n\n[C] Online (Multinomial) Logistic Bandit: Improved Regret and Constant Computation Cost, Yu-Jie Zhang, Masashi Sugiyama, NeurIPS 2023.\n\n[D] Efficient batched algorithm for contextual linear bandits with large action space via soft elimination, O Hanna, L Yang, C Fragouli, NeurIPS 2023.\n\nQUESTIONS:\nAll my questions are in the Strengths and Weaknesses part.\n\nLIMITATIONS:\nPlease see strengths and weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of regret minimization for contextual bandits via feature perturbation. Contrarily to usual sampling method that perturb the parameter space, they show that by directly injecting randomness in the feature space, they can recover a $O(d\\sqrt{T})$ bound for generalized linear models, matching the performances of UCB-like algorithms and improving on existing sampling-based algorithms. They provide empirical evidence of their theoretical results as well as empirical evidence that their method can be applied beyond the setting of generalized linear models.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is very well written and clear. The setting, assumptions and results are well stated and clearly demonstrated both in the proofs and the empirical demonstration. The setting considered is of great importance in practice. Moreover some care is given to explaining how their proposed methods overcomes the limitation of Thompson Sampling to get the extra $\\sqrt{d}$ factor.\nWhile their method matches the theoretical guarantees of UCB-based methods for linear bandits, they show that their algorithm adapts to the curvature of the space leading to tighter guarantees for Generalized-Linear-Bandits.\nOn the empirical side, one of the main appeal is that their method is directly applicable to very general function classes in a principled and scalable way.\n\nThe main limitation of the theoretical results is that they are limited to the Generalized-Linear-Settings. However, this is a very common setting for this kind of theoretical guarantees and their experimental guarantees extend beyond this setting.\nIt would be extremely interesting to see if the methods and algorithm they propose can be extended beyond the settings of bandits (For  Decision-Making with Side Observations for instance).\n\nI think that this paper is a valuable contribution for our community.\n\nQUESTIONS:\n- Do you think there is a way to extend this type of method beyond the setting of bandits(For instance for Reinforcement learning, one could perturb the feature of the state action pair to make a decision)\n- Do you think this type of method is limited to greedy action selection or could be suitable to more complex sampling methods such as Information Directed Sampling ?\n- Do you think these type of methods are limited to these generalized linear settings or could be adapted without too much effort to more general parametrized settings (For instance finite action and lipschitzness of the reward function) ?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}