Title: Sparse Optimistic Information Directed Sampling
Venue: NeurIPS 2025
Accepted: yes
Keywords: regret minimization

sparse linear models

Information Directed Sampling

Bayesian methods
Abstract: Many high-dimensional online decision-making problems can be modeled as stochastic sparse linear bandits. Most existing algorithms are designed to achieve optimal worst-case regret in either the data-rich regime, where polynomial dependence on the ambient dimension is unavoidable, or the data-poor regime, where dimension-independence is possible at the cost of worse dependence on the number of rounds. In contrast, the Bayesian approach of Information Directed Sampling (IDS) achieves the best of both worlds: a Bayesian regret bound that has the optimal rate in both regimes simultaneously. In this work, we explore the use of Sparse Optimistic Information Directed Sampling (SOIDS) to achieve the best of both worlds in the worst-case setting, without Bayesian assumptions. Through a novel analysis that enables the use of a time-dependent learning rate, we show that OIDS can be tuned without prior knowledge to optimally balance information and regret. Our results extend the theoretical guarantees of IDS, providing the first algorithm that simultaneously achieves optimal worst-case regret in both the data-rich and data-poor regimes. We empirically demonstrate the good performance of SOIDS.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a new method (SO-IDS) that introduces sparsity to the IDS framework. It is an adaptation of the OIDS algorithm in Neu, Papini, and Schwartz [2024]. The key idea is to reduce the per-round computational burden by restricting sampling to a small, adaptively chosen subset of arms, rather than computing over the full action set. The authors propose two sparse variants: Uniform SO-IDS and Greedy SO-IDS. They prove that SO-IDS preserves the core theoretical performance guarantees of IDS in terms of regret bounds while significantly improving computational efficiency. The approach is validated empirically on synthetic and real-world bandit problems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: (1) Very solid technical paper. The authors provide solid theoretical regret guarantees and proof to justify the soundness of the sparse sampling design. (I do need to mention a caveat that I did not read the full proof, and I am not very familiar with the prior work by Neu, Papini, and Schwartz in terms of the technical details.) (2) A new perspective to improve IDS efficiency through sparsity. (3) The algorithm shows significantly better computational efficiency without sacrificing performance -- this highly relevant for large-scale online learning applications, where sparsity can allow more efficient design. \n\nWeaknesses: (1) The proof part can be further streamlined to improve readability. (2) The greedy SO-IDS algorithm is empirically strong, but a better (more sound) motivation could be provided. (3) It is unclear how the framework generalizes to structured or infinite action spaces (e.g., linear or contextual bandits).\n\nQUESTIONS:\n1) Can the sparse sampling strategy extend to linear or contextual bandit settings, where the action space is large or continuous? If so, how would sparsity be enforced?\n\n2) Can the authors better explain how the framework developed in this paper depends on & differs from the prior work by Neu, Papini, and Schwartz 2024? \n\n3) How sensitive is the performance to the size of the subset of arms considered each round? Could the authors suggest a principled or adaptive way to select this size?\n\nLIMITATIONS:\nThis paper is theoretical and there is no specific discussion on limitations or potential societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper applies the OIDS algorithm by Neu et al. (2024) to the sparse linear bandit problem and establishes a cumulative regret bound of $\\tilde{\\mathcal{O}}(\\min\\lbrace \\sqrt{sdT}, sT^{2/3} \\rbrace)$.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The algorithm achieves both $\\tilde{\\mathcal{O}}(\\sqrt{T})$ and $\\tilde{\\mathcal{O}}(T^{2/3})$ regret bounds, making it the first algorithm to do so in the sparse linear bandit setting.\n\n2. The paper proposes adaptive learning rate schedules that do not require knowledge of the time horizon, which is not common in optimization-based sequential decision-making algorithms.\n\nWeaknesses\n\n1. In Appendix E.3, it appears that the bound for the 3-surrogate information ratio only holds under the sparse optimal action assumption. While the authors argue that the regret lower bound is invariant under this assumption, it is unclear why the upper bound on the information ratio should also remain the same. Without this assumption, it seems the $s$ factor must be replaced with a $d$ factor, resulting in a significantly worse regret bound of $\\tilde{\\mathcal{O}}(dT^{2/3})$ than the stated $\\tilde{\\mathcal{O}}(sT^{2/3})$ bound. Additionally, the $\\tilde{\\mathcal{O}}(sT^{2/3})$ bound is still looser than the previously known $\\tilde{\\mathcal{O}}((sT)^{2/3})$ bound [Hao et al. 2020] by a factor of $s^{1/3}$.\n\n2. The paper’s independent contribution is not sufficiently explained. The summary of contributions at the end of the introduction gives an impression that the main contribution is setting an appropriate prior and computing the information ratio for the sparse linear bandit problem, and the rest follows directly from Neu et al. (2024). The method of choosing history-dependent learning rates in Theorem 3, which is another emphasized contribution, is the same with the analysis of AdaGrad in Orabona (2019) and is somewhat hard to say it is novel - although I admit it was never explicitly mentioned in the sequential decision-making context.\n\nQUESTIONS:\n1. In Appendix C.1, why does the prior assign additional probability to $k = 1, \\ldots, s-1$-sparse vectors? If the sparsity level $s$ is known, wouldn't a uniform distribution over $\\Theta$ be more natural?\n\n2. In the proof outline of Theorem 1 in the main text, the equation after line 245 contains a term $\\mathbb{E} \\left[ \\frac{\\eta}{\\lambda_T}( L_T^{(1)}(\\theta_0) - L_T^{(1)}(\\theta_T)) \\right]$. While this term is not bounded in the outline, it seems that it should be included in the statement of Lemma 4. Could the authors clarify this point?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the sparse linear bandit problem. The minimax regret in this setting depends on whether the problem instance falls within the data-rich regime or the data-poor regime. In the former, the time horizon $T$ is significantly larger than the dimension $d$ and the optimal regret is of order $\\sqrt{s d T}$ with $s$ being the sparsity level, while the latter concerns high-dimensional problems, where the optimal regret has a worse $T^{2/3}$ rate but can be dimension-independent in some cases. While achieving near-optimal regret in both regimes by one adaptive algorithm has only been shown in the Bayesian setting (via the information directed sampling approach), this paper presents the first algorithm that achieves this feat in the non-Bayesian (frequentist) regime. The adopted approach extends the optimistic information-directed sampling algorithm of Neu et al. (2024) to the sparse linear bandit setting. It features a modified choice of posterior distribution and employs a time-dependent (or history-dependent) learning rate.\n\nSTRENGTHS AND WEAKNESSES:\nThe primary contribution here is achieving non-Bayesian analogues of the regime-adaptive Bayesian bounds of Hao et al. (2021). In this manner, the paper addresses a more practically relevant setup and shows that regime adaptivity is also possible there. Additionally, the adopted time-dependent and history-dependent learning rates lend the algorithm more flexibility compared to similar approaches.\n\nOn the other hand, I feel that little has been highlighted in terms of the shortcomings of the results. For example, the data-poor bound scales with $s$ instead of $s^{2/3}$ like in the upper bound of Hao et al. (2020) or $s^{1/3}$ in the lower bound presented in the same paper. This is not a huge drawback as the focus here is on dimension-independence (indeed, this is also the case in Hao et al. 2021); however, this should still be mentioned to clarify what is meant by 'near-optimality'. Likewise, it is important to emphasize that $C_\\text{min}$ (featured in the data-poor bounds) can be dimension-dependent, depending on the geometry of the action set. Concerning the instance-dependent guarantee of Theorem 3, I find its significance a bit questionable; even if it improves upon the worst-case bound, which (interpretable/relevant) property of the instance can be exploited by this result? Overall, even if novelty is present in some parts of the algorithm and the analyses, the obtained results arguably constitute a modest leap from the current state-of-the-art.\n\nMinor correction: in line 94, I believe it should be $C_\\text{min}^{-1/3}$.\n\nQUESTIONS:\n* Aside from the time-dependent learning rate, was the new adjustment in the definition of the optimistic posterior (compared to Neu et al. 2024) crucial for obtaining regime-adaptive bounds in the present setup? You mentioned that it simplifies the analysis, could you expand more on that? I also wonder if the 'optimistic' label is still accurate here as the posterior seems to lend importance to parameters under which the learner would have incurred large regret up to the present time.\n* How strict is the assumption that the sparsity level is known? Can it be dropped at the cost of a slightly worse bound as in Hao et al. (2020)?\n\nLIMITATIONS:\nAside from the brief discussion on computational challenges in the last two sections, limitations of this work are scarcely discussed as I mentioned in the strengths/weaknesses section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Sparse Optimistic Information Directed Sampling (SOIDS), an extension of the Optimistic Information Directed Sampling (OIDS) algorithm for sparse linear bandits. The key contribution is achieving optimal worst-case regret bounds in both data-rich $O\\bigl(\\sqrt{sdT}\\bigr)$  and data-poor $O\\bigl(sT^{2/3}\\bigr)$ regimes simultaneously, without Bayesian assumptions. The authors adapt the optimistic posterior framework to allow time-dependent learning rates and provide theoretical guarantees showing SOIDS is the first frequentist algorithm to achieve best-of-both-worlds performance in sparse linear bandits. The paper includes experimental validation demonstrating competitive performance against specialized algorithms in both regimes.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n- **Solid Theoretical contribution**: First frequentist algorithm achieving optimal regret in both data-rich and data-poor regimes for sparse linear bandits. Provides both worst-case bounds and instance-dependent guarantees.\n- **Novel technical approach**: Extends optimistic posterior framework to handle time-dependent learning rates, enabling adaptive algorithms.\n- **Clear presentation**: Paper is well-written with good intuition for the main ideas.\n\n### Weaknesses:\n- **Limited experimental validation**: Only basic synthetic experiments with small dimensions and specific parameter structures.\n- **Restrictive assumptions**: The assumption of known sparsity level s is restrictive in practice. The authors should discuss potential adaptations for unknown sparsity.\n\nQUESTIONS:\n1. In Theorem 2, the learning rate λt = min(1/2, max(λt^(2), λt^(3))) switches between two regimes. Could you provide more intuition about why this specific switching mechanism is necessary? How sensitive is the algorithm's performance to the exact switching point?\n2. Theorem 3 provides an instance-dependent bound for the data-rich regime. Is it possible to obtain a unified instance-dependent bound that smoothly interpolates between both regimes, as mentioned in Section 4.3?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}