Title: Geometry Meets Incentives: Sample-Efficient Incentivized Exploration with Linear Contexts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multi-armed bandits

Bayesian Incentive Compatible

Exploration
Abstract: In the incentivized exploration model, a principal aims to explore and learn over time by interacting with a sequence of self-interested agents. It has been recently understood that the main challenge in designing incentive-compatible algorithms for this problem is to gather a moderate amount of initial data, after which one can obtain near-optimal regret via posterior sampling. With high-dimensional contexts, however, this \emph{initial exploration} phase requires exponential sample complexity in some cases, which prevents efficient learning unless initial data can be acquired exogenously. We show that these barriers to exploration disappear under mild geometric conditions on the set of available actions, in which case incentive-compatibility does not preclude regret-optimality. Namely, we consider the linear bandit model with actions in the Euclidean unit ball, and give an incentive-compatible exploration algorithm with sample complexity that scales polynomially with the dimension and other parameters.


Peer reviews:
comments: SUMMARY:
The paper considers how geometric conditions can help the initial exploration phase. They use a Bayesian framework and some not-that-intuitive algorithms to prove a polynomial complexity. It avoids exponential dependency.

STRENGTHS AND WEAKNESSES:
The problem is interesting and important to learning theory. In my opinion, the results are non-trivial and inspiring. However, the paper lacks enough comparison of existing literature; for example, more discussion on "Incentivizing Exploration with Linear Contexts and Combinatorial Actions" is needed. I'm looking forward to a new but small section about the difference (new results under what conditions) between the paper and others.

QUESTIONS:
Can you provide more motivation and any potential applications of the problem?

Do you have any results of corresponding lower bounds? Lower bounds can make the paper sound and more complete.

The results heavily rely on the agents being myopic. However, in practice, some agents may consider future revenue and misreport now. In other words, they may sacrifice current revenue to mislead the planner. Could you discuss the importance of this assumption and any potential difficulty extending the existing results?

The authors don't provide any experiments. Even a simulation comparing their algorithms and existing algorithms will improve the paper significantly.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper \"Geometry Meets Incentives: Sample-Efficient Incentivized Exploration with Linear Contexts\" proposes to explore the problem of a planner that provides recommendations to selfish and rational agents that aim only to maximize their own expected reward based on its posterior. In order to minimize cumulative regret, the planner needs to incentivize exploration in its recommendations. However, if these recommandations are not in line with agent's own interest, they won't be followed and the process does not collect the expected exploration knowledge. The goal in the paper is to build a BIC planner as a linear bandit, which ensures that recommandations allow to cover the entire action space (in the sense of the \\lambda-spectral exploration following [Sellke et al., 2023]. The proposal strongly build on  [Sellke et al., 2023], that proposes to divide the process in two steps of pure exploration followed by a Thompson Sampling procedure. The core idea is that after enough free exploration steps, where the agent has no information at all to decline the recommandation, the parameter posterior is sufficiently concentrated to make sampled recommendations always the most credible from the agent's perspective. Authors show that under some assumptions over the actions space and rewards distribution, the length of the exploration phase can be significantly reduced. More specifically, they propose an algorithm that performs exploration coverage without greatly fewer initial steps of initial data collection.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths : \n- Significant theoretical contribution (I did not check all the proofs though)\n\nWeaknesses :\n\n- The paper is only theoretical. It would have been very useful to get some experimental results of the proposed algorithm, both for use cases comprehension and also to capture the benefits of the proposal in various settings, including some of them not fully respecting working assumtpions\n- Lack of pedagogy: the setting is not clearly / formally defined. For readers that do not know the incentivized exploration problem, it is very hard to capture what are the objectives of this kind of problem. I had to read [Sellke, 2023] to understand the problem, what is given to agents, what is the goal, etc. In comparison [Sellke, 2023] is far more clear in presenting the problem. \n- Beyond the lack of formal presentation of the problem, the paper also uses many notations that are never defined : e.g.,  the symbol used for positive semidefinite partial order on symmetric matrices (I had to find it in Sellke, 2023), the symbol used to define product tensors at order 2, $\\Omega$, etc\n- While I find the problem theoretically interesting, I am not fully convinced by its real interest for pratical applications, as users can interact multiple times, have their own interests, their own history, the knowledge and understanding of the planner's algorithm, etc. Practical uses should be more discussed in the paper.  \n- The central assumption of the algorithm is that actions live in a d-dimentionnal unit ball. Isn't it limitative for some applications ? And more importantly, in that setting what would justify a prior that would not be uniform over every vector from that ball ? This can be very limitative for the paper if no reasonnable pratical justification for this  can be found since taking a fully non informative prior is a trivial setting, for which TS is BIC without any initial exploration phase. Or am I wrong on this point ?\n\nQUESTIONS:\nsee strengths and weaknesses\n\nLIMITATIONS:\n.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the incentivized exploration problem in linear bandits, where a principal needs to gather information by recommending actions to a stream of myopic self-interested agents while remaining Bayesian Incentive Compatible (BIC).\nEarlier work (Sellke ’23) showed a hard instance in which the amount of data needed before one can safely switch to Thompson sampling could be exponentially many rounds. The authors show that this exponential barrier is not intrinsic under mild conditions. More specifically, when the action set is a unit ball, there exists an algorithm whose sample complexity is only polynomial in the dimension.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- Given the hard result by (Sellke ’23), authors show that when the action set is a unit ball, there exists an algorithm which can break the exponential barrier under mild conditions. Authors present two illustrative examples that satisfy those conditions, including r-regular distributions and a type of log-concave distributions with a certain smoothness. To me, this finding is interesting since it opens some new problems, e.g., can we find a broader class of action sets?\n\n- The algorithm and subroutines are easy to follow. I appreciate the explanations and intuitions behind the algorithm design.\n\n- The paper is generally well-written and well-organized. Prior works are clearly contrasted.\n\nWeaknesses:\n\n- The main results rely on the unit ball assumption. Extending to general smooth convex bodies is hinted at but left open.\n\n- Though the sample complexity is polynomial in parameter d, it suffers large poly(d) factors. It could be helpful to discuss the tightness of dependence on d.\n\n- There is no discussion or analysis on the computational cost and memory cost.\n\nOther minors: \n\n- It could be better to split section 1.1 into two parts: one is problem setting and the other one shows your main results. Mixing them together seems to be a mess to me.\n\n- Adding a link of references and equations is helpful for readers. There is a ref issue in line 218 and 226, it should be Figure 1 instead of Figure 2.3.\n\nQUESTIONS:\n- Can authors discuss the tightness of dependence on d?\n\n- What's the computational cost and memory cost of the proposed algorithms?\n\n- If the reward mean is not as a linear function but as a general function, can we get similar results under the same conditions? If not, what's could be the main challenges?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

comments: SUMMARY:
This work studies the problem of incentivized exploration in linear bandits. Prior work of Sellke 2023 has shown that Thompson Sampling is BIC with an exponentially (in dimensions) number of initial samples. In this work, the authors show that it is possible to improve on this initial sample complexity when the action set is contained in a $d$-dimensional unit ball. Specifically, the authors provide an initial sampling algorithm with polynomial sample complexity. This is a significant improvement over the current literature on incentivized exploration in bandits.

STRENGTHS AND WEAKNESSES:
Strengths: 
- The improvement over prior work is a significant contribution. 
- The paper is generally well-written. 
- The problem of incentivized exploration in linear bandits is not new, but the initial sampling algorithm is novel. 
- The proof sketches are intuitive and easy to follow.

Weaknesses:
- The paper did not include any experiments. 
- The paper focuses solely on the initial sampling exploration, and needs to be combined with Sellke 2023's result to have an end-to-end BIC algorithm. Thus, the contribution may seem a bit incremental.
- The paper did not explain the constants $c_v, c_d, \epsilon_d$ in assumption 2. Having more elaborate justification of what each constant means and how they should be chosen in practice would help with understanding these assumptions.

QUESTIONS:
Please answer the questions listed above in the weaknesses. Furthermore, do the authors have any insight on whether this initial sample complexity is tight or can it still be improved in the future?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-06-29