Title: Local Anti-Concentration Class: Logarithmic Regret for Greedy Linear Contextual Bandit
Venue: NeurIPS 2024
Accepted: yes
Keywords: contextual bandit

greedy algorithm

exploration-exploitation

sequential decision-making

bandit algorithms
Abstract: We study the performance guarantees of exploration-free greedy algorithms for the linear contextual bandit problem. 
We introduce a novel condition, named the \textit{Local Anti-Concentration} (LAC) condition, which enables a greedy bandit algorithm to achieve provable efficiency. 
We show that the LAC condition is satisfied by a broad class of distributions, including Gaussian, exponential, uniform, Cauchy, and Student's~$t$ distributions, along with other exponential family distributions and their truncated variants. 
This significantly expands the class of distributions under which greedy algorithms can perform efficiently. 
Under our proposed LAC condition, we prove that the cumulative expected regret of the greedy algorithm for the linear contextual bandit is bounded by $\mathcal{O}(\operatorname{poly} \log T)$. 
Our results establish the widest range of distributions known to date that allow a sublinear regret bound for greedy algorithms, further achieving a sharp poly-logarithmic regret.

Local Anti-Concentration Class: Logarithmic Regret for Greedy Linear Contextual Bandit

Abstract

We study the performance guarantees of exploration-free greedy algorithms for the linear contextual bandit problem. We introduce a novel condition, named the Local Anti-Concentration (LAC) condition, which enables a greedy bandit algorithm to achieve provable efficiency. We show that the LAC condition is satisfied by a broad class of distributions, including Gaussian, exponential, uniform, Cauchy, and Student's t distributions, along with other exponential family distributions and their truncated variants. This significantly expands the class of distributions under which greedy algorithms can perform efficiently. Under our proposed LAC condition, we prove that the cumulative expected regret of the greedy algorithm for the linear contextual bandit is bounded by O (poly log T ) . Our results establish the widest range of distributions known to date that allow a sublinear regret bound for greedy algorithms, further achieving a sharp poly-logarithmic regret.

1 Introduction

In the contextual multi-armed bandit problem [2, 6, 24, 25], an agent uses revealed contextual information in each round to decide which arm to pull, receiving a reward corresponding to the pulled arm. The stochastic version of this problem observes rewards as random samples, with their expectation tied to the arm's contextual information. The agent's goal is to design a sequential arm-pulling strategy to maximize cumulative rewards, necessitating a balance between exploration and exploitation. Linear contextual bandits, where expected reward is modeled as a linear function of contextual information, serve as the fundamental framework for contextual bandits [1, 10, 26]. Various exploration strategies, including upper confidence bound (UCB) [1, 11], Thompson sampling (TS) [3, 4], and ϵ -greedy [19] are widely used and studied in the theoretical analysis for linear contextual bandits. However, exploration can often be challenging in practice, possibly leading to over-exploration and performance deterioration. Some domains may find exploration infeasible or even unethical, and it may appear unfair in applications such as healthcare and clinical domains. Furthermore, exploration strategies tend to add complexity for algorithm designers and decisionmaking systems.

A greedy policy, i.e., pure exploitation without exploration, selects arms greedily based on current problem parameter estimates. While a greedy policy's effectiveness cannot be guaranteed in general since it may fail to find optimality in the worst case, the possibility of its favorable performances in certain scenarios has been of interest both practically and theoretically. Therefore, understanding when a greedy policy can perform effectively, i.e., when exploration is not needed, is a fundamental research question. Recently, a simple greedy policy has been proved to achieve near-optimal regret bounds for linear contextual bandit problems under some stochastic conditions of contexts [8, 20, 30, 33, 34]. Such efficient learning is possible if the greedy policy can benefit from suitable diversity in the contexts (or the features of arms) - so that even with exploration-free action selection, parameter estimation is

Min-hwan Oh

effectively possible. However, distributions known in the existing literature to allow efficient greedy algorithms are mostly limited only to Gaussian and uniform distributions [8, 20, 30, 33]. Hence, the following research questions arise.

Is it possible for a wider range of distributions to allow efficient learning for greedy algorithms? If so, how can we characterize such distributions?

We answer the above questions affirmatively by proposing a general distributional condition that allows for a broad range of distributions to achieve provable efficiency of greedy linear bandit algorithms. In this work, we present a new Local Anti-Concentration (LAC) condition for distributions that encompasses a wider range of context distributions compared to the previous findings. We demonstrate that the class of distributions that satisfy LAC, which we denote as LAC class , include Gaussian, exponential, Cauchy, Student's t , and uniform distributions, as well as other exponential family distributions and their truncated variants. This study provides the first evidence that greedy algorithms can perform efficiently beyond Gaussian and uniform distributions. Our findings significantly expand the class of admissible distributions that are suitable for greedy algorithms for linear contextual bandits.

Our proposed LAC condition not only broadens the class of permissible distributions for greedy bandit algorithms but also facilitates a sharper regret guarantee, achieving a poly-logarithmic O (poly log T ) regret for greedy algorithms. Our regret analysis constitutes a distinct improvement over the previously known results for greedy linear contextual bandit algorithms. The existing results are primarily categorized into two folds: (i) Gaussian-distributed contexts could only yield O ( √ T ) regret for greedy algorithms for single-parameter linear contextual bandits [20, 30, 33]; (ii) Context diversity (e.g., Assumption 3 in [8]) alone was previously regarded as not sufficient to derive a poly-logarithmic regret but additionally assuming a margin condition (e.g., Assumption 2 in [8]) can achieve O (poly log T ) regret. 1 In either case, there are limited prior results about context diversity beyond Gaussian and uniform distributions. As for the margin condition, to the best of our knowledge, no prior work in greedy contextual bandits has rigorously derived the scaling of the margin constant, rather than simply treating it as a universal constant. To this end, we establish that Gaussian and uniform distributions as well as all of the common distributions that satisfy the LAC condition (see Table 1) induce O (poly log T ) regret without having to additionally assume a margin condition.

The key difference between the analysis of greedy algorithms and that of exploration-based algorithms, such as UCB and TS, for linear contextual bandits lies in the estimation bounds. While UCB [1] and TS [4] analyses involve bounding the weighted estimation error of the parameter using selfnormalized martingales, the analysis of greedy algorithms relies on the ℓ 2 estimation bound in all directions. Ensuring this estimation consistency is more challenging, especially when actions are chosen adaptively, resulting in non-i.i.d. data.

In this work, we prove that for a broad class of context distributions, √ t -consistency of the estimator can be guaranteed, enabling poly-logarithmic regret for greedy algorithms. Our newly proposed class of context distributions represents the largest known class from which √ t -consistency of the estimator can be derived, even with adaptively chosen (non-i.i.d.) contexts. To establish this consistency, we derive two key technical results. First, we show that the minimum eigenvalue of the Gram matrix increases sufficiently under the LAC condition. Additionally, we demonstrate that under this condition, the suboptimality gap can be bounded probabilistically-a result derived from our analysis rather than assumed explicitly.

1.1 Contributions

The main contributions of our paper are summarized as follows:

We propose a novel condition, called Local Anti-Concentration (LAC) condition, for a greedy linear contextual bandit algorithm to achieve provable efficiency. The newly proposed

LAC condition is satisfied by a wide rage of common distributions, including Gaussian, exponential, Cauchy, Student's t , and uniform distributions, and many common distributions, as well as their truncated variants. This significantly expands the class of admissible distributions that are suitable for greedy algorithms and is, to our best knowledge, by far the largest class of distributions that induces efficient learning for greedy algorithms.

Under our proposed LAC condition, we prove that the cumulative expected regret for the greedy algorithm is bounded by O (poly log T ) (Theorem 1), the sharpest known bound for greedy algorithms in linear contextual bandits with a single parameter.

By leveraging the proposed condition, we can guarantee both (i) the growth of the minimum eigenvalue of the Gram matrix and (ii) a probabilistically bounded suboptimality gap. These two steps are key technical components for analyzing greedy bandit algorithms and were explicitly assumed in existing literature [8] to achieve poly-logarithmic regret. Notably, we do not assume these steps; instead, we prove that distributions satisfying the LAC condition inherently induce these two technical results (Theorems 2 and 3), which may be of independent interest.

Various context distributions have been empirically shown to allow favorable performances of greey algorithms (see Appendix M). However, the distributions previously known in the literature that enable efficient greedy algorithms were primarily limited to Gaussian and uniform distributions. Our theoretical results offer a significant step toward bridging this gap between theory and practice, providing insights into why greedy algorithms can be effective under a wide range of distributions.

1.2 Related Work

Linear contextual bandits and generalized linear bandits have been widely studied [1, 2, 4, 6, 10, 11, 18, 23, 27, 32]. Upper confidence bound (UCB) algorithms for the linear contextual bandit have been proposed and analyzed for their regret performance [1, 6, 10, 11, 32]. Thompson sampling [35] algorithms for linear contextual bandits have also been widely studied, with results demonstrating

their effectiveness both theoretically and empirically [3, 4, 9]. While UCB [1] and Thompson sampling [3, 4] analyses rely on bounding the weighted estimation error of the parameter using self-normalized martingales, the analysis of greedy bandit algorithms depends on the ℓ 2 estimation bound in all directions. Ensuring this estimation consistency is more challenging, especially in adaptive action settings where data are not i.i.d.

Recent studies [8, 20, 30, 33] have shown that a greedy algorithm can achieve near-optimal regret performance for linear contextual bandit problems under stochastic contexts by providing sufficient conditions under which the greedy algorithm can be efficient. These conditions typically focus on the diversity of the context distribution, ensuring that the greedy policy benefits from sufficient context diversity for effective parameter estimation even without exploratory actions.

However, the existing literature has mainly limited itself to Gaussian [8, 20, 30, 33] and uniform [8] distributions, leaving open questions about broader applicability. Specifically, it is unclear if other distributions could also support efficient greedy algorithms and what fundamental characteristics these distributions should have to enable consistent parameter estimation without exploration. Our work addresses this gap by identifying broader conditions under which diverse distributions can effectively support greedy algorithms in linear contextual bandits.

2 Preliminaries

2.1 Notations

We use ∥ x ∥ p to denote the ℓ p -norm of vector x ∈ R d . For a positive definite matrix A ∈ R d × d , we define ∥ x ∥ A = √ x ⊤ Ax . We use λ min ( A ) to denote the minimum eigenvalue of the positive definite matrix A . We denote D d R := [ -R,R ] d and B d R := { x ∈ R d : ∥ x ∥ 2 ≤ R } . If d is clear, we just write B d R := B R and D d R := D R . We define [ n ] for a set [ n ] := { 1 , 2 , . . . , n } . We write S d -1 for a d -dimensional unit sphere. We set ∥ X ∥ ψ 1 = sup p ≥ 1 { p -1 E 1 /p | X | p } , ∥ X ∥ ψ 2 = sup p ≥ 1 { p -1 2 E 1 /p | X | p } for a random variable X . If X is a d -dimensional random vector, then we write ∥ X ∥ ψ 2 = sup ∥ u ∥ 2 =1 ∥⟨ u, X ⟩∥ ψ 2 , ∥ X ∥ ψ 1 = sup ∥ u ∥ 2 =1 ∥⟨ u, X ⟩∥ ψ 1 . We use the notation O () or ≲ to hide constants, and ˜ O () to hide constants and logarithmic terms. We use the notation a ≍ b when a ≲ b and b ≲ a . We use c, c 1 , c 2 . . . for absolute constant, which may differ from line by line .

2.2 Linear Contextual Bandits with Stochastic Contexts

We consider the linear contextual bandit problem with K arms ( K ≥ 2 ), where in each round t = 1 , 2 , . . . , T , the set of context vectors X ( t ) = { X i ( t ) ∈ R d , i ∈ [ K ] } is drawn from some unknown distribution P X ( t ) . Each arm's feature X i ( t ) ∈ X ( t ) for i ∈ [ K ] need not be independent of each other and can possibly be correlated. The agent then pulls an arm a ( t ) ∈ [ K ] . Each context vector X i ( t ) for i ∈ [ K ] is associated with stochastic reward Y i ( t ) ∈ R with mean X i ( t ) ⊤ θ ⋆ where θ ⋆ ∈ R d is a fixed, unknown parameter. For simplicty, we assume ∥ θ ⋆ ∥ 2 ≤ 1 . After pulling arm a ( t ) , the agent receives a stochastic reward Y a ( t ) ( t ) as a bandit feedback: Y a ( t ) ( t ) = X a ( t ) ( t ) ⊤ θ ⋆ + η a ( t ) ( t ) , where η a ( t ) ( t ) ∈ R is a zero mean noise. We assume that there is an increasing sequence of sigma fields {H t } such that each η a t ( t ) is H t -measurable with E [ η a t ( t ) |H t -1 ] = 0 . In our problem, H t is the sigma field generated by random variables of the arms chosen { a (1) , ..., a ( t ) } , their context vectors { X a (1) (1) , ..., X a ( t ) ( t ) } , and the corresponding rewards { Y a (1) (1) , ..., Y a ( t ) ( t ) } . Also, η a ( t ) ( t ) is assumed to be conditionally σ -sub-Gaussian, i.e., for all λ ∈ R , E [ e λη a ( t ) ( t ) |H t -1 ] ≤ exp ( λ 2 σ 2 / 2 ) for σ ≥ 0 . Observing context vector X ( t ) , let a ∗ ( t ) denote the optimal arm in round t , that is, a ∗ ( t ) = arg max i ∈ [ K ] X i ( t ) ⊤ θ ⋆ . Then the instantaneous expected regret ( reg( t ) ) and cumulative expected regret ( Reg ( T ) ) are defined respectively as

which are respectively the instantaneous and cumulative differences between the optimal expected reward and the expected reward of the pulled arms. The expectation is taken with respect to the stochasticity of history, containing randomness of contexts. The goal of the agent is to minimize the cumulative expected regret.

2.3 LinGreedy : Exploration-Free Algorithm for Linear Contextual Bandits

In this work, we focus on identifying sufficient conditions that enable exploration-free greedy algorithms to efficiently learn the optimal policy. Specifically, we analyze a greedy algorithm for linear contextual bandits, which we refer to as LinGreedy (Algorithm 1). The LinGreedy algorithm selects arms greedily based on the OLS estimator, without any exploratory actions.

Description of Algorithm 1. The algorithm performs a greedy action in each round based on estimated rewards. In the initial rounds, when the Gram matrix Σ( t ) is not yet invertible, parameter estimation is deferred, and the algorithm selects actions based on an initial parameter θ 0 . Once the Gram matrix becomes invertible-which can be shown with high probability after sufficient time, the algorithm computes an OLS estimator and performs a greedy action based on the estimated parameter in each subsequent round. This algorithm is exploration-free. In the following sections, we present a novel and more general condition that enables efficient learning for greedy algorithms.

3 Local Anti-Concentration Class

In this section, we introduce a new sufficient condition for efficient greedy contextual bandits. This condition is general and encompasses a wide range of common distributions, including Gaussian, exponential, uniform, Cauchy, and Student's t distributions, as well as their truncated variants. To the best of our knowledge, this is the most extensive class of distributions considered in the greedy contextual bandit literature [8, 20, 28, 30, 31, 33], which has primarily focused on Gaussian, uniform distributions, and their truncated variants.

Our proposed condition centers on the rate of the log density of stochastic contexts, a concept we term Local Anti-Concentration (LAC). We now formally introduce the novel LAC class.

Definition 1 (Local Anti-Concentration (LAC)) A density function f X of a random variable X ∈ R n is said to satisfy the Local Anti-Concentration (LAC) condition with a non-decreasing polynomial L if for all x ∈ R n . We refer to L as the LAC function of X . We denote the class of distributions that satisfy this LAC condition as the Local Anti-Concentration class.

3.1 Intuition of LAC Condition

The LAC condition implies that a density is not overly concentrated at any given point, leading to a gradual decay in density across all directions-hence the term local anti-concentration. A geometric interpretation of the LAC condition and a rigorous definition of this decay rate are provided in Appendix D. Section 3.2 demonstrates that the LAC condition applies to a broad range of common distributions. To the best of our knowledge, very few distributions have been previously shown to support efficient performance guarantees for greedy algorithms. However, we prove that the LAC condition holds for a wide range of distributions, including a variety of exponential families. Note that L can be a constant when contexts have bounded support (see Appendix C). In the following sections, we further explore the characteristics of the LAC condition.

3.2 Generality of LAC Condition

We show that the LAC condition is applicable to various distributions, significantly expanding the class of admissible distributions for greedy linear contextual bandits. The LAC condition is satisfied when the exponential component in the exponential family has a polynomial scale. Included in the LAC class are distributions such as Gaussian, exponential, uniform, Student's t , and Cauchy distributions, along with their truncated variants.

The following proposition demonstrates that the LAC function does not directly depend on the dimension of X . We further discuss in Appendix C that the LAC condition is more closely related to the correlation structure of X rather than its dimensionality. Therefore, we suggest that the LAC condition provides a suitable framework for comparing regret when both the number of arms and the dimension are large.

Proposition 1 Suppose the random variable X = ( X 1 , X 2 ) , where X 1 ∈ R n 1 and X 2 ∈ R n 2 , consists of two independent components. If X 1 and X 2 satisfy the LAC condition with functions L 1 ( · ) and L 2 ( · ) , respectively, then X satisfies the LAC condition with L ( x ) = max( L 1 ( x ) , L 2 ( x )) .

This holds because, when we take the logarithm of the density, the independent coordinates decompose as the sum of each density. Upon taking the gradient and evaluating the ℓ ∞ norm, the expression decomposes perfectly. Using this proposition, the LAC condition remains robust across dimensions if the coordinates are independent, making it dimension-free in such cases. Furthermore, this condition is very accessible because it can be readily computed for a given density function. For many well-known exponential families, the exponential component of the density often scales polynomially.

Examples of Distributions with LAC Condition

We present a few examples of known distributions satisfying the LAC condition. We provide rigorous proofs for the examples in Appendix C.

Gaussian distribution: For a Gaussian random variable X = ( x 1 , . . . , x n ) ∼ N ( µ, Σ) , if Σ is diagonal, it satisfies the LAC condition with L ( x ) = 4 λ min (Σ) ( ∥ x ∥ ∞ + ∥ µ ∥ ∞ ) . For the general (non-diagonal) case of Σ , see Appendix C.

Exponential distribution: The exponential distribution's density f X ( x ) = 1 λ exp( -λx ) satisfies the LAC condition with a constant function L ( x ) = λ .

Student's t -distribution: The 1 -dimensional Student's t -distribution has density f X ( x ) = Γ( ν +1 2 ) √ νπ Γ ( ν 2 ) · ( 1 + x 2 ν ) -( ν +1) / 2 and satisfies the LAC condition with L ( x ) = c ν for some ν dependent constant c ν > 0 .

Uniform distribution: The uniform distribution has constant density and satisfies the LAC condition with a constant function L ( x ) = 1 .

Laplace distribution: The Laplace distribution has density f ( x ) = 1 2 b exp ( -| x -µ | b ) and satisfies the LAC condition with L ( x ) = c for some constant c > 0 .

If each coordinate's density independently adheres to one of the aforementioned distributions, according to Proposition 1, they all share the same LAC function irrespective of the dimension.

Consider the density f ( x ) with f ( x ) ∝ exp( -V ( x )) for some differentiable function V ( x ) . If ∇ V ( x ) has polynomial growth, i.e., is bounded by a polynomial, then the density f ( x ) meets the LAC condition. This holds because f ( x ) = C exp( -V ( x )) , ∇ log f ( x ) = ∇ log exp( -V ( x )) = -∇ V ( x ) . If ∇ V ( x ) exhibits polynomial-scale growth, then the supremum norm confirms that the LAC condition is satisfied.

This observation makes the LAC condition easily verifiable for exponential family distributions with density forms f X ( x | θ ) = h ( x ) exp[ η ( θ ) · T ( x ) -A ( θ )] , where the exponential part T ( x ) has polynomial growth. In many exponential family cases, T ( · ) indeed exhibits polynomial growth. Proofs and further details can be found in Appendix C.

4 Statistical Challenges of Greedy Linear Contextual Bandits

In this section, we outline the key statistical challenges in analyzing greedy algorithms for linear contextual bandits: (i) ensuring the diversity of the adapted Gram matrix (Section 4.1) and (ii)

bounding the suboptimality gap to achieve logarithmic regret (Section 4.2). For ease of exposition, we use the vectorized context expression X ( t ) = ( X ⊤ 1 ( t ) , . . . X ⊤ K ( t )) ∈ R dK of X ( t ) , which combines context vectors X i ( t ) for i ∈ [ K ] . We define X ij ( t ) as the j -th coordinate of the context X i ( t ) .

4.1 Diversity of Adapted Gram Matrix

The first key challenge lies in ensuring sufficient ℓ 2 -concentration of the estimator. This requires sufficient eigenvalue growth of the Gram matrix, constructed from the policy-selected contexts. For the OLS estimator used in Algorithm 1, if the minimum eigenvalue of the adapted Gram matrix λ min (Σ( t )) increases linearly with the number of rounds t , we can obtain the high probability ℓ 2 error bound ∥ ˆ θ t -θ ⋆ ∥ 2 with a convergence rate of O (1 / √ t ) using martingale concentration [13, 29]. In fact, the growth of λ min (Σ( t )) is a necessary condition to obtain √ t -consistency of the estimator.

However, estimating the covariance of the selected contexts X a ( t ) ( t ) is relatively challenging, as its distribution differs significantly from the overall distribution (before selection) of X ( t ) . Some studies have investigated the statistical properties of selected contexts [20, 28]; however, the known results are limited to specific distributions, such as arm-independent Gaussian and uniform distributions. Therefore, it remains an open question whether the growth of λ min (Σ( t )) can be ensured for a broader class of distributions and what characteristics such distributions would need to satisfy.

4.2 Bounding Suboptimality Gap: Road to Logarithmic Regret

The next challenge that we face particularly in order to achieve logarithmic regret is to bound the suboptimality gap. We first denote the suboptimality gap as the difference between the optimal expected reward and the second highest expected reward:

̸

which is determined by the true parameter θ ⋆ . We aim to bound this suboptimality gap probabilistically, as described precisely in Challenge 2 of Section 4.3.

When this challenge is resolved, along with the growth of the minimum eigenvalue of the adapted Gram matrix discussed in Section 4.1, we can achieve logarithmic expected regret using analysis techniques for linear contextual bandit with stochastic contexts [7, 19]. A high-level description of the role of the margin constant in the regret bound is provided in Appendix D.1, with a rigorous analysis in Appendix J.

4.3 Formal Statements of Two Key Challenges

As mentioned above, we encounter two primary challenges: ensuring the diversity of the chosen contexts (i.e., the growth of the minimum eigenvalue of the Gram matrix of the selected contexts) and bounding the suboptimality gap. Importantly, we do not assume these conditions to hold a priori; rather, we will demonstrate that they are satisfied in the stochastic context under the LAC condition. In this section, we formally define these challenges to be addressed.

Before delving into the formal statements for each of the two challenges, we first define the concept of the diversity constant, which depends on the minimum eigenvalue of the adapted Gram matrix.

Definition 2 (Diversity Constant) For a linear contextual bandit with contexts X ( t ) and history H t -1 , the diversity constant λ ⋆ ( t ) is defined as the value satisfying

for all t > 0 , where a ( t ) denotes the arm selected by the algorithm in round t .

Then, the first challenge is to ensure a positive diversity constant λ ⋆ ( t ) > 0 , which involves sufficient eigenvalue growth of the adapted Gram matrix. We explore this further in Appendix F.1.

Challenge 1 (Positive Diversity Constant) Our goal is to ensure λ ⋆ ( t ) > 0 .

Achieving a positive diversity constant is challenging, as it requires analyzing the behavior of a context selected by the greedy policy in a specific direction rather than relying on the overall context distribution. In Section 5.3.1, we demonstrate that the minimum eigenvalue of the Gram matrix grows sufficiently, thereby ensuring a positive diversity constant.

We now formally state our second challenge of bounding the suboptimality gap.

Challenge 2 (Probabilistic Suboptimality Gap) We aim to bound the constant C ∆ ( t ) , which holds under the given history H t -1 and for any ε > 0 ,

We also refer to this constant C ∆ ( t ) as the margin constant.

Note that Eq. (2) is a relaxed version of the margin condition presented in [5, 7, 8, 19]. The aforementioned literature explicitly assumes this condition to hold. However, we instead show that the suboptimality gap can be bounded without directly assuming it (Section 5.3.2). Rigorously, C ∆ ( t ) depends on T , as it is a function of T . However, we emphasize that our algorithm does not require prior knowledge of T ; this dependency is needed only for the analysis.

5 Regret Analysis

We present the main results of our paper. We prove that the regret of the greedy algorithm (Algorithm 1) for linear contextual bandits can be bounded at a logarithmic scale in the time horizon T , provided that the context distribution satisfies the LAC condition with a polynomial function L .

Assumption 1 (Independently distributed contexts) The context sets X (1) , . . . , X ( T ) are independently distributed across time.

Discussion of Assumption 1. To the best of our knowledge, all analyses of greedy linear contextual bandits assume the independence and identical distribution (i.i.d.) of context sets [8, 20, 28, 33]. In Assumption 1, we only require context sets to be independent; they may be non-identically distributed. Additionally, much of the literature on linear contextual bandits that investigates √ t -consistency of estimators also assumes independence of context sets [21, 22]. Note that under Assumption 1, context vectors within the same round are permitted to be dependent.

5.1 Considerations for Context Boundedness

We first provide detailed considerations on context boundedness. In the linear contextual bandit setting, ℓ 2 boundedness is commonly assumed. However, for light-tailed distributions (such as Gaussian or exponential), the ℓ 2 norm is unbounded. In such cases, a general approach in the statistical literature is to assume bounded ψ 1 or ψ 2 norms [14, 17, 37, 38]. Therefore, we divide our analysis into cases of bounded and unbounded contexts.

Bounded Contexts vs. Unbounded Contexts. In linear contextual bandit studies, boundedness of the ℓ 2 norm of contexts is commonly assumed. In this paper, for bounded contexts, we consider both truncated contexts (e.g., truncated Gaussian, truncated Cauchy distributions) and naturally bounded contexts (e.g., uniform distribution). For unbounded contexts, we assume a bounded ψ 1 norm, which is a standard assumption for handling light-tailed distributions (e.g., as in [14, 17], which assume bounded ψ 2 norms).

Assumption 2 (Boundedness) For unbounded contexts, we assume ∥ X i ( t ) ∥ ψ 1 ≤ x max . For bounded contexts, we assume ∥ X i ( t ) ∥ 2 ≤ x max for all i ∈ [ K ] , t ∈ [ T ] .

Discussion of Assumption 2 The bounded context assumption is widely used in the literature [1, 7, 8, 21, 22, 28]. For unbounded contexts, our assumption of ψ 1 boundedness is notably weaker than the sub-Gaussianity (or ψ 2 ) assumption commonly used in statistical regression literature to handle random design covariates [14, 39]. If ℓ 2 boundedness holds, it automatically implies boundedness of the ψ 1 norm. However, as the analysis differs slightly depending on whether the support is restricted to a bounded ball or is unbounded, we address these cases separately.

5.2 Regret Bound of LinGreedy for LAC Distribution

We first introduce our main result, the regret bound of the greedy algorithm under the LAC condition.

Theorem 1 (Regret bound of LinGreedy ) Suppose X ( t ) satisfies the LAC condition with the polynomial function L and also satisfies Assumptions 1 and 2 for all t . Then, the cumulative expected regret of LinGreedy (Algorithm 1) is bounded by

where O concerns only the dependency on T . Considering the dependency on d and K , for unbounded contexts, we have

For bounded contexts, refer to Appendix H for explicit results, as we consider several cases.

Discussion of Theorem 1. Theorem 1 states that if the contexts are drawn from a distribution in the LAC class, the regret scales as O (poly log T ) . While our primary objective is not solely to achieve the sharpest regret bounds, attaining poly-logarithmic regret is highly favorable. Our main goal is to demonstrate that a large class of context distributions satisfies the LAC condition. When they do, a simple greedy algorithm can suffice or even outperform exploration-based algorithms (see numerical experiments in Section 6). The worst-case dependence on d and K for bounded contexts is detailed in Appendix H, where dependencies remain at most polynomial in d and K . As these dependencies vary across distributions, refer to Appendix H for precise information. Proofs for unbounded contexts are provided in Appendix G, with a proof sketch in Appendix D. Proofs for bounded contexts are included in Appendix I.

Theorem 1 is the first result to expand the class of admissible distributions for greedy bandit algorithms beyond Gaussian and uniform distributions. Our result demonstrates, for the first time, that distributions in the LAC class inherently exhibit margin behavior, achieving sharp poly-logarithmic regret without requiring an additional margin assumption. This finding is of independent interest beyond the analysis of greedy bandit algorithms.

5.3 Proof Sketch of Theorem 1

We first present our key results for addressing each of Challenges 1 and 2 stated in Section 4.3.

5.3.1 Ensuring the Positive Diversity Constant

In this section, we present our key result for estimating lower bounds on the diversity constant for densities that satisfy the LAC condition, therefore addressing Challenge 1. Theorem 2 is the analysis under the case of unbounded contexts, where contexts have full support. A similar result is presented in Appendix H for contexts with bounded or truncated support.

Theorem 2 (Diversity constant for unbounded contexts) If unbounded contexts X ( t ) has the LAC condition with L ( x ) := A 1 + A 2 x α and satisfies Assumption 2 for all t ,

holds for R 1 := c 2 x max (log d +log K +2) . Here, c 1 , c 2 are absolute constants.

Discussion of Theorem 2. Theorem 2 implies that λ ⋆ ( t ) ≥ Ω( 1 d ) , hence ensuring the growth of the minimum eigenvalue of the adapted Gram matrix. In Appendix D.1, we discuss how 1 λ ⋆ ( t ) factors into the regret bounds. Note that α is generally small in many distributions. For example, for Gaussian distributions, α = 1 , and for exponential distributions, α = 0 .

5.3.2 Bounding Suboptimality Gap

Next, we present our result addressing Challenge 2 by computing the suboptimality gap constant C ∆ ( t ) , which satisfies the inequality in Eq.(2) for every ε > 0 . By combining this condition with the estimates for λ ⋆ ( t ) , we can obtain an O (poly(log T )) regret bound in terms of T by applying the analysis techniques of linear contextual bandits with stochastic contexts [7, 8, 19] to our setting.

Theorem 3 (Suboptimality gap for unbounded contexts) In the same setup as Theorem 2,

holds for R 2 = c 4 x max (1 + log K +log d + 1 2 log T ) + 1 and absolute constants c 3 , c 4 > 0 .

Discussion of Theorem 3. Note that the suboptimality gap constant C ∆ ( t ) is multiplied linearly in regret bounds [5, 7, 19]. For bounded contexts, we present a similar result in the Appendix H.

5.3.3 Proof Intuitions

We provide a high-level proof overview of Theorem 1 in Appendix D. Note that once Challenges 1 and 2 are satisfied, achieving logarithmic regret becomes straightforward, as detailed in Appendix J.

The remaining task is to address these two challenges, with a particular focus on bounding the constants λ ⋆ ( t ) and C ∆ ( t ) . A key implication of the LAC condition is that the density decays slowly at every point. Another useful property is that the LAC condition is preserved under conditioning, meaning that X ( t ) | { X ( t ) ∈ A } also satisfies the LAC condition with the same function for any set A ⊂ R K × d . This can be verified using the fact that log f X ( t ) |{ X ( t ) ∈ A } ( x ) = log f X ( t ) ( x ) -log P [ A ] , where P [ A ] is a constant (see Appendix C.3 for details).

The main challenge of analyzing the statistical concentration in greedy linear contextual bandits lies in the fact that the distribution of selected contexts X a ( t ) ( t ) differs significantly from the distribution of the overall (pre-selected) contexts X ( t ) . The preservation of LAC under conditioning ensures that LAC still holds when conditioning on the event of selecting arm i , enabling our analysis.

The full proofs for unbounded contexts are provided in Appendix G. For results on bounded contexts, see Appendix H (and their proofs in Appendix I).

5.4 √ t -Consistency of Estimator

In addition to achieving logarithmic regret, an independently valuable result is obtained: the ℓ 2 -consistency of the estimator ˆ θ t . This is a property that even typical sublinear-regret algorithms, such as UCB and TS, do not generally guarantee. Under the same setup as Theorem 1, we achieve ∥ ˆ θ t -θ ⋆ ∥ 2 ≤ ˜ O ( d √ t ) with high probability (see Corollaries 6 and 7). This additional result may also facilitate analysis of sample complexity, such as PAC bounds.

6 Experiments

To validate our theoretical findings numerically, we conducted experiments using various context distributions: Gaussian, Laplace, uniform, and truncated Cauchy distributions. We compared the performance of LinGreedy with the LinUCB and LinTS algorithms. The results showed that LinGreedy exhibited significantly superior regret performance compared to the other explorationbased algorithms, achieving a logarithmic scale of regret. Detailed experimental results are provided in Appendix M.

0

200

400

600

800

1000

Round (

t

)

0

50

100

150

200

250

300

350

400

Cumulative Regret

d

=20,

K

=20, Uniform

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

100

200

300

400

Cumulative Regret

d

=20,

K

=100, Laplace

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

50

100

150

200

250

300

Cumulative Regret

d

=20,

K

=20, Truncated Cauchy

LinUCB

LinTS

LinGreedy

References

Abbasi-Yadkori, Y., Pál, D., and Szepesvári, C. (2011). Improved algorithms for linear stochastic bandits. Advances in neural information processing systems , 24.

Abe, N. and Long, P. M. (1999). Associative reinforcement learning using linear probabilistic concepts. In International Conference on Machine Learning , pages 3-11.

Abeille, M. and Lazaric, A. (2017). Linear thompson sampling revisited. In Artificial Intelligence and Statistics , pages 176-184. PMLR.

Agrawal, S. and Goyal, N. (2013). Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning , pages 127-135. PMLR.

Ariu, K., Abe, K., and Proutière, A. (2022). Thresholded lasso bandit. In International Conference on Machine Learning , pages 878-928. PMLR.

Auer, P. (2002). Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research , 3(Nov):397-422.

Bastani, H. and Bayati, M. (2020). Online decision making with high-dimensional covariates. Operations Research , 68(1):276-294.

Bastani, H., Bayati, M., and Khosravi, K. (2021). Mostly exploration-free algorithms for contextual bandits. Management Science , 67(3):1329-1349.

Chapelle, O. and Li, L. (2011). An empirical evaluation of thompson sampling. In Advances in neural information processing systems , pages 2249-2257.

Chu, W., Li, L., Reyzin, L., and Schapire, R. E. (2011). Contextual bandits with linear payoff functions. Journal of Machine Learning Research .

Dani, V., Hayes, T. P., and Kakade, S. M. (2008). Stochastic linear optimization under bandit feedback. In Proceedings of the 21st Annual Conference on Learning Theory , page 355-366.

De Haan, L., Ferreira, A., and Ferreira, A. (2006). Extreme value theory: an introduction , volume 21. Springer.

de la Pena, V. H., Klass, M. J., and Leung Lai, T. (2004). Self-normalized processes: exponential inequalities, moment bounds and iterated logarithm laws.

DUAN, Y. and WANG, K. (2023). Adaptive and robust multi-task learning. The Annals of Statistics , 51(5):2015-2039.

Durrett, R. (2019). Probability: theory and examples , volume 49. Cambridge university press.

Evans, L. C. (2022). Partial differential equations , volume 19. American Mathematical Society.

Fan, J., Wang, D., Wang, K., and Zhu, Z. (2019). Distributed estimation of principal eigenspaces. Annals of statistics , 47(6):3009.

Filippi, S., Cappe, O., Garivier, A., and Szepesvári, C. (2010). Parametric bandits: The generalized linear case. In Advances in Neural Information Processing Systems , pages 586-594.

Goldenshluger, A. and Zeevi, A. (2013). A linear response bandit problem. Stochastic Systems , 3(1):230-261.

Kannan, S., Morgenstern, J. H., Roth, A., Waggoner, B., and Wu, Z. S. (2018). A smoothed analysis of the greedy algorithm for the linear contextual bandit problem. Advances in neural information processing systems , 31.

Kim, G.-S. and Paik, M. C. (2019). Doubly-robust lasso bandit. In Advances in Neural Information Processing Systems , pages 5869-5879.

Kim, W., Kim, G.-s., and Paik, M. C. (2021). Doubly robust thompson sampling for linear payoffs. In Advances in neural information processing systems .

Kveton, B., Zaheer, M., Szepesvari, C., Li, L., Ghavamzadeh, M., and Boutilier, C. (2020). Randomized exploration in generalized linear bandits. In International Conference on Artificial Intelligence and Statistics , pages 2066-2076.

Langford, J. and Zhang, T. (2007). The epoch-greedy algorithm for contextual multi-armed bandits. Advances in neural information processing systems , 20(1):96-1.

Lattimore, T. and Szepesvári, C. (2019). Bandit Algorithms . Cambridge University Press (preprint).

Li, L., Chu, W., Langford, J., and Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web , pages 661-670. ACM.

Li, L., Lu, Y., and Zhou, D. (2017). Provably optimal algorithms for generalized linear contextual bandits. In International Conference on Machine Learning , pages 2071-2080.

Oh, M.-H., Iyengar, G., and Zeevi, A. (2021). Sparsity-agnostic lasso bandit. In Meila, M. and Zhang, T., editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 8271-8280. PMLR.

Peña, V. H., Lai, T. L., and Shao, Q.-M. (2009). Self-normalized processes: Limit theory and Statistical Applications . Springer.

Raghavan, M., Slivkins, A., Vaughan, J. W., and Wu, Z. S. (2023). Greedy algorithm almost dominates in smoothed contextual bandits. SIAM Journal on Computing , 52(2):487-524.

Raghavan, M., Slivkins, A., Wortman, J. V., and Wu, Z. S. (2018). The externalities of exploration and how data diversity helps exploitation. In Conference on Learning Theory , pages 1724-1738. PMLR.

Rusmevichientong, P. and Tsitsiklis, J. N. (2010). Linearly parameterized bandits. Mathematics of Operations Research , 35(2):395-411.

Sivakumar, V., Wu, S., and Banerjee, A. (2020). Structured linear contextual bandits: A sharp and geometric smoothed analysis. In International Conference on Machine Learning , pages 9026-9035. PMLR.

Sivakumar, V., Zuo, S., and Banerjee, A. (2022). Smoothed adversarial linear contextual bandits with knapsacks. In International Conference on Machine Learning , pages 20253-20277. PMLR.

Thompson, W. R. (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika , 25(3/4):285-294.

Tropp, J. A. (2011). User-friendly tail bounds for matrix martingales. Technical report, CALIFORNIA INST OF TECH PASADENA.

Vershynin, R. (2018). High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press.

Wainwright, M. J. (2019). High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge University Press.

Wang, K. (2023). Pseudo-labeling for kernel ridge regression under covariate shift. arXiv preprint arXiv:2302.10160 .

Contents

Appendix

A Additional Notations

We define additional notation. For v ∈ R d , define

For S ⊂ R n , we define

For the intervals I ⊂ R and v ∈ R d , define

For instance, [ -1 , 1] v := { x | -1 ≤ x ⊤ v ≤ 1 } . We define π V ( · ) as the projection onto the subspace V . Also, in the appendix, we write B R = B d R = { x ∈ R d | ∥ x ∥ 2 ≤ R } with slight abuse of notation. For a random variable X , we define supp( X ) as the support of X . Recall that we define the expected regret in round t as reg( t ) and the unexpected regret as reg ′ ( t ) . Also, for Ω = ⋃ i ∈ I E i with disjoint events E i , we use the following notation

For an event A , we define P [ X ; A ] and E [ X ; A ] as P [ X ∩ A ] and E [ X ∩ A ] , respectively, following Durrett [15]'s notation.

B ∥ θ ⋆ ∥ 2 Dependency of Suboptimality Gap

Throughout the proof, we first bound the margin constant C ∆ by assuming ∥ θ ⋆ ∥ 2 = 1 . If we obtain C ∆ for ∥ θ ⋆ ∥ 2 = 1 , then in the general case where ∥ θ ⋆ ∥ 2 = κ , observe that

̸

holds. Therefore, from now on, we assume ∥ θ ⋆ ∥ 2 = 1 and adjust for the general case later. Additionally, we highlight that we only need to bound C ∆ for ε < 1 2 , since for ε > 1 2 , it holds that C ∆ = 2 .

C More Details of LAC: Distributions, Conditioning, & Truncation

In this section, we provide the omitted details of LAC and its properties.

C.1 LAC of Various Distributions

We first provide more details on the LAC of various distributions, as presented in Section 3.2.

LAC of Gaussian contexts. For density f ( x ) = C exp ( -( x -µ ) ⊤ V ( x -µ ) ) , x ∈ R n , where V = 1 2 Σ -1 ,

First, when V is diagonal, after taking the log and gradient, we get

Since V = 1 2 Σ -1 , we obtain the desired results. For general V , we can see that

where V i is the i -th row of V .

̸

LAC of bounded support contexts. If the support of the contexts is bounded within a compact set, ∇ log f is bounded provided it is continuous. Therefore, in this case, LAC can be a constant function. Additionally, if ∥ x ∥ 2 ≤ R , we have

hold. Since L is defined in the sense of the supremum norm, to investigate the LAC function, the maximum value max i ∥ ∂ i V ( x ) ∥ ∞ is important. We claim that it is a dimension-free property, and the correlation structure is more important than the dimension n itself. For instance, if x = ( x 1 , . . . , x n ) is coordinate-wise independent, we have V ( x ) = V 1 ( x 1 ) + V 2 ( x 2 ) + · · · + V n ( x n ) . Hence,

and we can see it is a dimension-free value.

LAC with shifted mean distribution. Let the mean-zero contexts X have density f ( x ) with LAC function L ( · ) . Then, the shifted contexts with mean µ have density g ( x ) = f ( x + µ ) , and we observe that

Hence, the density g ( x ) has LAC with function L ′ ( x ) = L ( x + ∥ µ ∥ ∞ ) and when ∥ µ ∥ ∞ = O (1) , it has the same rate. Thus, LAC does not require contexts to be mean-zero and can be defined for distributions with a general mean.

C.2 Proof of Proposition 1

If X = ( X ⊤ 1 , X ⊤ 2 ) and X 1 , X 2 are independent, the density f X of X can be decomposed as

by the monotonicity of L ( · ) .

LAC and context correlations. We claim that the LAC function L ( · ) of a random vector x = ( x 1 , . . . , x n ) is related to the correlation structure of x i , x j rather than the dimension n itself. Consider f ( x ) ∝ exp( -V ( x )) . Then, and

Then,

holds, and taking the supremum norm, we get

where the third inequality holds by the monotonicity of L ( · ) . By the definition of the LAC function, we finally obtain the desired result: f has LAC with the function L ( x ) = max( L 1 ( x ) , L 2 ( x )) for x ∈ R .

C.3 LAC and Conditioning

First, we observe that the LAC condition is preserved under conditioning.

Let the density of X = ( X ⊤ 1 , . . . , X ⊤ K ) ∈ R dK be f ( · ) .

For the event E := { X ∈ A } where A ⊂ R dK , the conditional density of X given E is formulated as f | E ( x ) = f ( x ) P [ E ] ∈ R d for x ∈ R dK .

Then, the following holds:

This means X | E also satisfies the LAC condition with the same function, hence LAC is robust under conditioning.

̸

Especially, if the event E has the form E = { X i ∈ D and X j = x j for j = i } for some D ⊂ R d , we define the density of X i | E as

Then, holds. We have

We summarize the above observations in the following lemma, noting that L ( · ) is a non-decreasing function.

which indicates that the conditional density also satisfies the LAC property with L ( · ) .

̸

Lemma 1 (LAC of conditional contexts) For a random vector X = ( X ⊤ 1 , . . . , X ⊤ K ) ∈ R dK , let its density satisfy the LAC condition with function L ( · ) . Then, for any event E := { X ∈ A } where A ⊂ R dK and ∥ A ∥ ∞ ≤ R , the conditional random vector X | E satisfies the LAC condition with the constant function L ( R ) . Specifically, if E has the form E = { X i ∈ D and X j = x j for j = i } for some D ⊂ R d with ∥ D ∥ ∞ ≤ R , then the conditional random vector X i | E satisfies the LAC condition with L ( R ) .

C.4 LAC and Truncation

We introduce the property that can compute the LAC of truncated contexts. Truncating the contexts X i ( t ) to a d -dimensional ball B R ⊂ R d for every i ∈ [ K ] still satisfies the LAC condition with a constant function, L ( R ) .

Lemma 2 (LAC with truncation) Suppose the density of contexts X ∈ R dK satisfies the LAC condition with the function L ( · ) . Consider the case we truncate X into the region ( B R ) K and define truncated contexts as X . Then, X satisfies the LAC condition with constant function L ( R ) .

Proof The density of truncated contexts is calculated as

Then, when taking the log and gradient, we get

Since ∥ x ∥ ≤ R for x ∈ ( B R ) K , and L is defined by a non-decreasing function, we get

C.5 LAC and Decay Rate of Density

Next, we rigorously define the decay rate of a density.

Definition 3 (Decay rate) For a density f ( x ) , x ∈ R d , and for a set A ⊂ R d , we say it has decay rate M > 0 if

for all x 1 , x 2 ∈ A .

We next present a lemma that states a density with a bounded LAC function L has a bounded decay rate in every direction. This property is especially useful throughout the paper.

Lemma 3 (Decay rate and LAC) Suppose a density f is defined on a domain D ⊂ R d and satisfies the LAC condition with a constant function L . Then, the decay rate of f in D is bounded by √ d L .

Proof By using the Cauchy-Schwarz inequality, we can bound the directional derivative for any v ∈ S d -1 as

Next, by applying the Gronwall inequality (Lemma 22), we obtain

for any . Since and are arbitrary, we achieve the desired result.

x, x + hv ∈ D h v

For a univariate function, we define a one-sided decay rate, which is a weaker condition than the standard decay rate.

Definition 4 (One-sided decay rate) For a univariate function g , we say it has a one-sided decay rate M on set A if for all y, y ′ ∈ A with y < y ′ ,

Using these observations, we can see that the LAC condition provides an upper bound on the decay rate of the density. For example, in the one-dimensional case, a density with f ( x ) ∝ exp( -Mx ) has a decay rate M . We can then bound the lower bound of the variance as 1 M 2 and the maximum density is bounded by M . Since Challenge 1 is related to the variance lower bound and Challenge 2 is related to the maximum density (of the suboptimality gap), we aim to investigate the decay rate of the contexts' density.

We first present our key lemma, which provides a relationship between the one-sided decay rate and the variance lower bound. In the following parts, we present the rigorous relationships between LAC, (one-sided) decay rate, variance lower bound, and maximum density.

Lemma 4 (One-sided decay rate and variance lower bound) Consider the density g ( · ) of a random variable Y ∈ R with support I = [ a, b ] where b > 1 M for some M > 0 . Suppose g satisfies

for all y, y ′ ∈ I ∩ [ -1 M , 1 M ] with y < y ′ . Then, we have

for some absolute constant c > 0 .

Proof Using Lemma 5, we find that the density g is bounded by 3 M in the interval [ -1 2 M , 1 2 M ] ∩ I . By applying Lemma 6, we obtain the desired result.

Next, we present another key lemma that provides a relationship between the one-sided decay rate and the maximum density.

Lemma 5 (One-sided decay rate leads maximum density) Suppose a real-valued random variable Y has density g and for y 0 ∈ R , [ y 0 , y 0 + 1 2 M ] ⊂ supp( Y ) for some M > 0 . If g satisfies

for any 0 ≤ h < 1 2 M , then g ( y 0 ) ≤ 3 M .

Proof Since the integral of the density is 1, we have

Lemma 6 (Maximum density leads to variance lower bound) Let Y be a univariate random variable with support A = supp( Y ) . For an interval I = [ -1 3 M , 1 3 M ] , if the density of Y in I ∩ A is bounded by M , then we have

Therefore,

If I ∩ A = ∅ , above inequality still holds.

Proof First, since the density is bounded by M ,

Therefore, with a probability greater than 1 3 , Y ∈ A ∩ {| Y | > 1 3 M } . Thus,

If I ∩ A = ∅ , it means that | Y | ≥ 1 3 M almost surely, hence we get the wanted result.

D High-Level Proof of Theorem 1 (Unbounded Contexts)

In this section, we briefly outline the high-level proof of Theorem 1 for unbounded contexts. This overview provides a summary and high-level sketch of the proofs, with full details presented in Appendices F and G.

We begin by explaining how overcoming the two primary challenges (discussed in Section 4) leads to a logarithmic regret bound in the proof of Theorem 1. Next, we provide a sketch of the proofs for the two key theorems: Theorem 2 for lower-bounding the diversity constant and Theorem 3 for upper-bounding the suboptimality gap.

For bounded contexts, the result statements are given in Appendix H, and the full proofs are in Appendix I. Since the proof for bounded contexts follows a similar approach to that of unbounded contexts, we start with a proof sketch for the unbounded case.

D.1 Regret Analysis: Overcoming Two Challenges

In this section, we briefly present our proof sketch for the regret bounds. We describe how we can achieve logarithmic regret by addressing the two challenges: Challenges 1 and 2.

Addressing Challenge 1, we can obtain the O ( 1 √ t ) rate ℓ 2 bound of the parameter ˆ θ t and θ ⋆ by using the combination of self-normalized concentration [1] and properties that Σ t ⪰ 1 4 λ ⋆ t holds with high probability (by using Corollary 9). The details are in Appendix J. Hence, by tackling Challenge 1, ∥ ˆ θ t -θ ⋆ ∥ 2 ≤ c √ d log T √ λ ⋆ t holds with high probability. The following part describes how we can achieve logarithmic regret by addressing Challenge 2. First, simply assume the contexts are bounded, such as ∥ X i ( t ) ∥ 2 ≤ x max . Later, in our main proof, we also modify it to the relaxed condition ∥ X i ( t ) ∥ ψ 1 ≤ x max .

Challenge 1: √ t -rate ℓ 2 concentration. Ensuring the first Challenge 1 can lead to the ℓ 2 statistical resolution of the estimator. Hence, we can get

holds with high probability. Details are in Appendix J.1.

However, this resolution is insufficient for logarithmic regret; it can only achieve an O ( √ T ) regret bound.

Challenge 2: Towards logarithmic regret. Furthermore, addressing Challenge 2 (margin condition), we can obtain a logarithmic expected regret upper bound. When the greedy policy selects a ( t ) , it means that

and by the definition of the optimal arm,

holds. Ensuring Challenge 1, we get

Next, we define the event E as the event where reg ′ ( t ) > 0 . Under the event E , the suboptimality gap of X ( t ) satisfies

̸

and by overcoming Challenge 2, we get

By combining these results, we can bound the expected regret as

This observation enables us to achieve a logarithmic regret bound. Using this argument, our only remaining goal is to bound the two constants in Challenges 1 and 2. We summarize our above observations in the following lemma.

Lemma 7 Assume that ∥ X i ( t ) ∥ 2 ≤ R for some R > 0 for all i ∈ [ K ] . Also assume that the estimator ∥ ˆ θ t -1 -θ ⋆ ∥ ≤ A 1 √ t -1 holds for some constant A > 0 . Then the (unexpected) regret in round t is bounded by reg ′ ( t ) ≤ 2 AR 1 √ t -1 . Furthermore, under the margin condition (Challenge 2), we have E X ( t ) [reg ′ ( t )] ≤ 6 R 2 A 2 C ∆ 1 t -1 .

Using the above observation, we can achieve a logarithmic regret bound when ∥ X i ( t ) ∥ 2 is bounded. For the case where ∥ X i ( t ) ∥ ψ 1 is bounded, we use the peeling technique. Given the history H t -1 , ˆ θ t -1 -θ ⋆ is a fixed vector, and using the results from Appendix J.2, we can bound | X i ( t ) ⊤ v | , v ∈ { ˆ θ t -1 -θ ⋆ , θ ⋆ } with high probability. Hence, we can apply similar arguments, and details are presented in Appendix J.

D.2 Tackling Two Challenges and Fixed History Arguments

Now, we summarize our proof strategy to prove diversity (Challenge 1) and suboptimality gap (Challenge 2). First, we consider the problem with a fixed history setup, which involves analysis under the given history H t -1 . We set history-conditioned contexts as X := X ( t ) | H t -1 and X = ( X ⊤ 1 , . . . , X ⊤ K ) , where X i ∈ R d . We describe more about this history fixing in the next appendix F. Under the given event H t -1 , ˆ θ t -1 is a deterministic value and no longer random. Thus, the policy a ( t ) , conditioned on H t -1 , is a greedy policy with respect to ˆ θ t -1 , making it deterministic as well. To address any ˆ θ t -1 , we propose an analysis that applies to any θ ∈ R d . In Appendix F.1, we argue that it suffices to bound the variance of selected contexts of the greedy policy with respect to any fixed θ . Since we fix θ , which corresponds to the value of ˆ θ t -1 under the given history H t -1 , we define the policy-selected arm a = arg max X ⊤ i θ . To address Challenge 1, our first goal is to find the lower bound of E [ X a X ⊤ a ] . Next, recall that we defined ∆( X ) := X ⊤ a ⋆ θ ⋆ -max i = a ⋆ X ⊤ i θ ⋆ and we aim to bound this suboptimality gap. Next, we introduce our two important goals to achieve the two challenges.

̸

Goal 1. For any θ, v ∈ S d -1 and a greedy policy with respect to θ , we aim to find the lower bound of

Goal 2. For a fixed θ ⋆ , our aim is to find the upper bound of C ∆ that satisfies

It is straightforward that when we achieve Goal 1, then Challenge 1 can be achieved easily since Goal 1 prepares for all greedy policies with θ .

D.3 Starting with Truncation

Before we tackle the two above goals, we start by truncating our contexts X to high-probability regions. Since X i has a bounded ψ 1 norm as x max , roughly speaking, there exists a set D ⊂ R d with ∥ D ∥ ∞ = ˜ O (1) and

We can view X as a mixture of X | { X ∈ D K } and X | { X ∈ ( D K ) c } and with high probability, X is sampled from the distribution X | { X ∈ D K } . In this section (Appendix D), since we are providing a proof sketch, we simply regard X as sampled from the distribution with bounded support D K where ∥ D ∥ ∞ = ˜ O (1) . In Appendix C.3, we observed that X | { X ∈ D K } also has LAC, and since ∥ D ∥ ∞ = ˜ O (1) , it has a constant LAC with L ( ∥ D ∥ ∞ ) = L ( ˜ O (1)) = ˜ O (1) since L is polynomial.

Therefore, for this section only, we assume that X has bounded support D K and it has a bounded constant LAC function L = ˜ O (1) . For a rigorous proof and justification, we provide them throughout Appendix F and Appendix G.

D.4 Event Decompositions

Next, we start to bound the constants in the two challenges. Before that, we present event decompositions for our analysis.

̸

Definition 5 (Event decomposition 1) We define the event { a = i } as Ω i and Ω i ( { x j } j = i ) := { a = i } ∩ { X j = x j for all j = i } .

Pick any v ∈ S d -1 . We can decompose the variance term as

̸

For notations E [ E [ · | · ]] , please refer to Appendix A. From now on, we aim to bound

̸

for any i, { x j } j = i . Hence, we investigate the property of the conditional density

̸

and especially we are interested in the density of projected conditional contexts

̸

We define new event decompositions for bounding the suboptimality gap. Under the fixed history, set a ⋆ := arg max i ∈ [ K ] X ⊤ i θ ⋆ .

̸

̸

and

These are projected contexts with some direction v and θ ⋆ . In the remaining part, we aim to bound the one-sided decay rate of these projected random variables' densities to apply Lemma 4 and Lemma 5.

̸

Definition 6 (Event decomposition 2) We define the event { a ⋆ = i } as Ω ⋆ i and Ω ⋆ i ( { x j } j = i ) := { a ⋆ = i } ∩ { X j = x j for all j = i } .

̸

̸

̸

Next, we aim to bound C ∆ , which is another key constant in Challenge 2. Define the optimal arm a ⋆ = arg max X ⊤ i θ ⋆ and the suboptimal arm a ′ = arg max i = a ⋆ X ⊤ i θ ⋆ . By definition, for any ε > 0 , we have to calculate

̸

Then, we aim to bound

̸

̸

and hence we investigate the properties of the conditional density

̸

̸

It is sufficient to bound the maximum density of X ⊤ i θ ⋆ | Ω ⋆ i ( { x j } j = i ) since if it is bounded by U , we can see that

̸

holds, and we can set C ∆ = U .

D.5 Conditional Contexts are Still in LAC Class

̸

In the previous Appendix C.3 and Lemma 1, we saw that LAC is preserved under conditioning. Our conditioned contexts of interest, X i | Ω i ( { x j } ) and X i | Ω ⋆ i ( { x j } ) , involve conditioning on events such as X i | { X j = x j for all j = i, X ⊤ i θ ≥ max j = i x ⊤ j θ } , which meet the conditions of Lemma 1. Then, we can apply Lemma 1 and conclude that these two conditional densities

̸

̸

also have LAC with a bounded constant function L .

D.6 Remaining Goal: Bounding Decay Rate of Projected Contexts

Lemma 4 tells us that a bounded one-sided decay rate of a density guarantees a lower bound on the variance. Lemma 5 tells us that a bounded one-sided decay rate of a density guarantees a bound on the maximum density.

̸

̸

In summary, we need to find the lower bound of equation (6) and the maximum density of (7). We present Lemma 4 and 5, which tells us that if we can bound the one-sided decay rate, then we can bound the variance of (6) and the maximum density of (7). By the property that LAC is preserved under conditioning, we can bound the decay rate of X i | Ω i ( { x j } j = i ) and X i | Ω ⋆ i ( { x j } j = i ) by √ d L , by combining Lemma 3 and the LAC property of conditional contexts. However, our interests are projected contexts, defined as

̸

̸

̸

̸

̸

̸

̸

̸

Support of conditional densities. We first observe the support of the conditional density X i | Ω i ( { x j } j = i ) . By the definition of Ω i , the arm i should be optimal under the greedy policy θ in this event. Therefore, its support is restricted to { x ∈ D | x ⊤ θ ≥ max j = i x ⊤ j θ } .

̸

̸

Next, we consider the density of projected contexts, X ⊤ i v | Ω i ( { x j } j = i ) . Since the support of X i | Ω i ( { x j } j = i ) is of the form { x ∈ D | x ⊤ θ ≥ max j = i x ⊤ j θ } , our projected density is an integrated form:

̸

̸

̸

̸

̸

Geometrically, this represents the total density at the intersection of the hyperplane { x | x ⊤ v = y } and the set { x ∈ D | x ⊤ θ ≥ max j = i x ⊤ j θ } . We refer to these as section densities, as they represent the total density of sections sliced by the hyperplane { x | x ⊤ v = y } . Further discussion is provided in Appendix E.

̸

Conclusion. Later in Appendix F and Appendix G, since X i | Ω i ( { x j } j = i ) has a bounded decay rate √ d L = ˜ O ( √ d ) , we prove that the projected densities also have a bounded one-sided decay rate of ˜ O ( √ d ) . Lastly, by applying Lemma 4, we can prove the quantity

̸

for some c = ˜ O (1) . Also, using Lemma 5, we can prove that the density of (7) has a bounded density √ d L and we can prove that C ∆ = ˜ O ( √ d ) .

To summarize, we proceed with the following steps:

Event decomposition by conditioning.

̸

̸

We aim to bound the decay rate of the projected densities X ⊤ i v | Ω i ( { x j } j = i ) and X ⊤ i θ ⋆ | Ω ⋆ i ( { x j } j = i ) .

We know X i | Ω i ( { x j } j = i ) and X i | Ω ⋆ i ( { x j } j = i ) have a bounded decay rate √ d L = ˜ O ( √ d ) .

̸

̸

If we are able to bound the decay rate of the above projected densities, we can bound the two key constants in the two challenges using Lemma 4 and Lemma 5.

For step 3, the remaining important task is to bound the decay rate of the projected densities, which we address in Appendix E, Appendix F, and Appendix G.

E Sections, Section Densities and Decay Rate

̸

In this section, we define sections and section densities and investigate their properties. Previously in Appendix D, we discussed decomposition by conditioning, and our interest became the properties of conditioned contexts, the form of X i | Ω i ( { x j } j = i ) and X i | Ω ⋆ i ( { x j } j = i ) . Our first goal is to bound the variance of projected contexts, such as X ⊤ i v | Ω i ( { x j } j = i ) for all v ∈ R d . To do that, we investigate the projected density of X ⊤ i v = y | Ω i ( { x j } j = i ) . To apply Lemma 4 and Lemma 5, we only need to bound the one-sided decay rate of that projected density. We saw that its density is the total density of intersections with { x | x ⊤ θ ≥ max j = i x ⊤ j θ } and hyperplane { x | x ⊤ v = y } . We can view it as sections sliced by hyperplanes, and we provide some theory related to sections and section densities.

E.1 Motivation

In the proof strategy (Appendix D), we aim to bound the one-sided decay rate and maximum density of projected conditional density, X ⊤ i v | Ω i ( { x j } j = i ) and X ⊤ i θ ⋆ | Ω ⋆ i ( { x j } j = i ) . Recall that we define one-sided decay rate of univariate density g (Definition 4) at Appendix D as a constant M satisfying

̸

̸

̸

̸

̸

̸

for all y < y ′ .

Then, how to bound the (one-sided) decay rate of section density? We present a technique to bound the (one-sided) decay rate of section densities defined in equal and expanding sections.

E.2 Sections and Section Densities

Our remaining goal is to bound the one-sided decay rate of section density. To build some general theory, we first define sections and section densities explicitly.

Definition 7 (Sections) For the set A ⊂ R d and v ∈ S d -1 , we define sections as for y ∈ R .

This means that a section is the intersection of region A and hyperplane { x ∈ R d | x ⊤ v = y } .

Definition 8 (Section density) We define the section density of the region A . Let f be the density defined in the region A ⊂ R d . Then we define g ( y ) as the section density of Sec( A,v, y ) as

Remark 1 If the density of X ∈ R d is f , the section density corresponds to the density of X ⊤ v for v ∈ S d -1 .

Next, we define the areas with special section structures: equal sections and expanding sections . First, we define equal sections, the area whose section with direction v is equal.

Definition 9 (Equal sections) We define the set A to have equal sections with v ∈ S d -1 when Sec( A,v, y ) is congruent in shape for y > 0 .

Next, we define expanding sections, where the sections of A with direction v expand when y increases.

Definition 10 (Expanding sections) We define the sections of A with direction v as expanding sections when

for every h > 0 . Technically, equal sections are included in expanding sections.

Sec(

A,v, y

+

h

)

Sec(

A,v, y

)

A

v

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

<latexi sh

1_b

64="

0kgB

y

dnQ3V+2

P/rmu

MZ

>A

G

X

cj

TL

FHU

I

WE

o

R

v

J

q

O

7

f

K

p

S

8

N

9

Y

z

w

C

D

5

<latexi sh

1_b

64="JYNRW

qwCdmKGn

Uj

Q

>

S3

c

T

M

EH

Lg

+

BF

I

r

u

X

2

yO

7

P9

f

o

k

Z

5v

z

/

p

0

<latexi sh

1_b

64="P

A

g2Q5fZEm

C+Y

H

qO/

DU

>

GTX

cjVRN

M

8

F

I

0

K3

J

ky

7

v

B

n

S

z

W

d

9

p

w

u

r

E.3 Decay Rate of Section Density

Recall that we define the one-sided decay rate of univariate density g (Definition 4) at Appendix D as a constant M satisfying

for all y < y ′ .

Then, how to bound the (one-sided) decay rate of section density? We present a technique to bound the (one-sided) decay rate of section densities defined in equal and expanding sections.

Lemma 8 (One-sided decay rate: equal and expanding sections) For A ⊂ R d , v ∈ R d , and y ∈ R , a density f has support A and has a bounded decay rate M . Assume that the sections of A with direction v are expanding sections. We define the section density of Sec( A,v, y ) as g ( y ) , then we have for any y 1 < y 2 :

Proof Since the section is expanding with the direction v , we can observe for any h > 0 ,

The second inequality holds since the section is expanding and using Lemma 3. Since equal sections are also expanding sections, we end the proof.

Lemma 9 (Maximum density: equal and expanding sections) The density f ( · ) has support A ⊂ R d and has a bounded decay rate M . Furthermore, sections with direction v are expanding sections and define the section density of Sec( A,v, y ) as g ( y ) . If the support of g ( · ) contains an interval [ a, b ] , then g ( y ) ≤ 3 M for y ∈ [ a, b -1 2 M ] .

Proof This can be obtained directly by applying the result of the previous Lemma 8 and Lemma 5.

Sec(

A

1

, v, y

)

A

1

v

θ

-

v

<latexi sh

1_b

64="CE

Lf

/+AU

mdWXS

32QrM

>

GNn cjVT

FH

g

J

Y

R

o

p

q

Z

y

z

7

P

BD

k

w

O9

vK

8

0

u

I

5

<latexi sh

1_b

64="g/CHMj

9I

2oAn dZK

JT

5Oc

>

G

3

V

L

F

U

8

N

YRE

kqQ

m

y

7

S

+

Dv

0

W

r

z

f

p

X

B

w

u

P

<latexi sh

1_b

64="kVTjU

KHngX

Wcd

P

QCrL

yI

>A

G

RN

M

E

+

B

q

uJ

p

2

zO

F

0

3

/

v8

7

Z

w

m

9

D

YS

o

5

f

<latexi sh

1_b

64="XYV5ZI9

/J

2AkcWP

T

U

7nQ

>

GM

j

L

FH

8

B

C

m

o

p

q

Sr

E

f

u

03z

g

O

K

+

dv

N

y

D

w

R

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

F Two Key Propositions in Fixed Histories (Unbounded Contexts)

This section aims to provide key propositions to prove Theorem 2 and Theorem 3. Our two challenges aim to bound

We define X ( t ) = ( X 1 ( t ) ⊤ , . . . , X K ( t ) ⊤ ) ∈ R dK . Under the given event H t -1 , ˆ θ t -1 is a deterministic value and is no longer random.

F.1 Details for Diversity Constant (Challenge 1)

In the definition of the diversity constant, we recall that the expectation is taken with respect to X ( t ) and the history H t -1 . In round t , for any fixed history H t -1 , we perform the greedy policy with the estimator ˆ θ t -1 . Hence, when the entire set of contexts X ( t ) = ( X 1 ( t ) ⊤ , . . . , X K ( t ) ⊤ ) is revealed, a ( t ) is determined immediately, given the history H t -1 . Then the exact statement is:

'In round t , for any given history H t -1 ,

holds with some λ ⋆ > 0 .'

In the proof, we proved a stronger statement:

'For any greedy policy with any θ ∈ R d and selected arm a ( t ) with that policy,

and holds with some λ ⋆ > 0 .'

Weprove a stronger quantity (second argument). For any given parameter θ ∈ R d , define a θ ( X ( t )) := argmax i ∈ [ K ] X i ( t ) ⊤ θ and define

Since ˆ θ t -1 can be an arbitrary value, we prepare for all greedy policies w.r.t. θ ∈ S d -1 . Then the newly defined λ ⋆ ( t ) satisfies equation (1), and we discuss this in Appendix F.

Thus, the policy a ( t ) , conditioned on H t -1 , is a greedy policy with respect to ˆ θ t -1 , making it deterministic as well. To address any ˆ θ t -1 , we propose an analysis that applies to any θ ∈ R d , aiming to bound the variance of the selected contexts. In Appendix F.1, we argue that it suffices to bound this variance for any fixed θ .

Note that a θ ( X ( t )) is defined as arg max i ∈ [ K ] X i ( t ) ⊤ θ . This quantity is equal to

and we prove this lower bound for any θ and v . In summary, our aim is to prove that for any history H t -1 and θ, v , we can bound

To bound the margin constant, our aim is to prepare for all θ ⋆ , since the true model parameter θ ⋆ can also be arbitrary. For simplicity, when contexts are clear, we write a θ ( X ) = a (when we fix θ ).

F.2 The Setup of Fixed History Analysis

We aim to find a class of densities F such that if X ( t ) ∈ F , then equation (9) is lower bounded for any θ, v . Similarly, we aim to find a class of densities G , such that if X ( t ) ∈ G , then equation (8) can be upper bounded for any θ ⋆ ∈ R d .

We first prove that for a fixed θ, v , for densities with LAC and supports satisfying certain geometric conditions, we can bound equation (9).

We also prove that for a fixed θ ⋆ , for densities with LAC and supports satisfying certain geometric conditions, we can bound equation (8).

Next, we prove that the densities with LAC and bounded ψ 1 norm, for any given θ, v , can be truncated (with high probability) to densities contained in class 1.

Lastly, we prove that for densities with LAC and bounded ψ 1 norm, for any given θ ⋆ , we can truncate it (with high probability) to densities contained in class 2.

In this section, we provide the results of 1 and 2. In the next Appendix G, we proceed to 3 and 4.

Now, we build the theory with the assumptions that θ, v and H t -1 are fixed. We assume that random vectors Z = ( Z ⊤ 1 . . . Z ⊤ K ) , where Z i ∈ R d , arise from some distribution with LAC L ( · ) . We fix an arbitrary θ ∈ R d and define the random index variable

We also fix v ∈ S d -1 and first aim to bound E [ v ⊤ Z a Z ⊤ a v ] . Our goal is to find the lower bound of E [ v ⊤ Z a Z ⊤ a v ] for any v ∈ S d -1 ; from now on, we fix an arbitrary v ∈ S d -1 . We then aim to bound

for fixed θ and v . Proposition 2 below shows that when the support of Z satisfies certain geometric conditions, E [ v ⊤ Z a Z ⊤ a v ] can be effectively bounded.

We next define C ∆ as the margin constant of Z with the parameter θ ⋆ , which satisfies

where ∆( Z ) is a suboptimality gap, defined similarly in Section 4. Proposition 3 below shows that when the support of Z satisfies certain geometric conditions, the margin constant of Z can be effectively bounded.

Same as previous definitions, we define the event

̸

̸

̸

F.3 Key Proposition for Diversity Constant

Recall that we define [ -1 , 1 ] v := { x | -1 ≤ x ⊤ v ≤ 1 } for any v ∈ R d .

Proposition 2 (Key proposition for diversity constant) The random vector Z = ( Z ⊤ 1 . . . Z ⊤ K ) , Z i ∈ R d , satisfies the following conditions:

Z has LAC with L ( · ) and ∥ supp( Z ) ∥ ∞ ≤ R .

For all i ∈ [ K ] , supp( Z i ) is identical for some subset A ⊂ R d . That is, supp( Z ) = A K for some A ⊂ R d .

̸

and

Simiarly, we define and

The support A ∩ [ -1 , 1] v has equal sections with direction v .

Then there exists an absolute constant c > 0 such that

Remark 2 For unbounded contexts X ( t ) in the original bandit problem, we truncate to some region C 1 with positive probability and make the truncated contexts satisfy the condition of the above proposition.

F.3.1 Proof of Proposition 2

Since supp( Z ) ≤ R , it has bounded decay rate √ d L ( R ) by Lemma 3. By the established event decomposition, we see

̸

̸

and by applying the tower property, we get

̸

Hence, we only need to bound

̸

̸

for every Ω i ( { z j } j = i ) . Recall that we defined [ -1 , 1 ] v := { x | x ⊤ v ∈ [ -1 , 1] } . Then we decompose it as

̸

̸

̸

̸

The last inequality holds because E [ v ⊤ Z i Z ⊤ i v ∣ ∣ Ω i ( { z j } j = i ) ∩ { Z i ∈ ([ -1 , 1] v ) c } ] ≥ 1 and E [ v ⊤ Z i Z ⊤ i v ∣ ∣ Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } ] ≤ 1 .

̸

̸

From now on, we focus on the conditional density of Z i | Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } and its projected density v ⊤ Z i | Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } . For simplicity, we set the conditional density of Z i | Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } as f 1 ( · ) and the projected density of v ⊤ Z i | Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } as g 1 ( · ) . Using Lemma 1, the density f 1 ( · ) has LAC with constant function L ( R ) , and it has bounded decay rate √ d L ( R ) by Lemma 3.

̸

̸

̸

[1] Investigating the support of f 1 and g 1 . First, we examine the support of f 1 . Since arm i is optimal with estimator θ , all z in the support of f 1 must satisfy z ⊤ θ ≥ max j = i z ⊤ j θ . Also, because -1 ≤ z ⊤ v ≤ 1 holds, this support is the intersection of these two areas. Define

̸

̸

Recall that A := supp( Z i ) is designed to have equal sections with direction v . Hence A ∩ [ -1 , 1] v has equal sections with v . Therefore, the support of Z ⊤ i v | Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } is an interval by our design of A .

̸

Bounding one-side decay rate of section density in [ -1 , 1] . We aim to apply Lemma 4 to bound the variance E [ v ⊤ Z i Z ⊤ i v | Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } ] . It tells us that it is enough to bound the one-side decay rate of the section density g 1 ( · ) in the interval [ -1 / 2 , 1 / 2] .

Claim 1 One of Sec( A 1 , v, y ) or Sec( A 1 , -v, y ) consists of expanding sections when y increases for y ∈ [ -1 , 1] .

̸

̸

̸

̸

̸

̸

Proof of Claim 1 Choose one of v, -v such that ⟨· , θ ⟩ ≥ 0 . We want to use the result of Lemma 10. Since [ -1 , 1] v has equal sections with direction v , by rotating the axis, we can satisfy the condition of Lemma 10. Then Lemma 10 implies that at least one direction among v or -v yields expanding sections. See Figure 5 for intuition.

Sec(

A

1

, v, y

)

A

1

v

θ

-

v

̸

Claim 2 The one-side decay rate of P [ v ⊤ Z i = y | Ω i ( { z j } j = i ) ∩ [ -1 , 1] v ] is bounded by √ d L ( R ) for y ∈ [ -1 , 1] .

Claim 1 tells us that we can apply Lemma 8, and thus bound the one-side decay rate of the section density g 1 ( · ) . Without loss of generality, assume Sec( A 1 , v, y ) is expanding when y increases. Then the support of Z ⊤ i v | Ω i ( { z j } j = i ) ∩ { Z i ∈ [ -1 , 1] v } is an interval of the form [ c, 1] with some -1 < c < 1 .

̸

Proof of Claim 2 To bound the one-side decay rate, we apply Lemma 8. By Claim 1, without loss of generality, sections with direction v , i.e., Sec( A,v, y ) , form an expanding section. By Lemma 8, we obtain the desired result.

[3] Bounding the desired variance. We now see that the support of g 1 ( · ) has the form [ c, 1] for some c < 1 . Hence, by Lemma 4, we conclude

̸

for some absolute constant c > 0 .

Below, we prove Lemma 10, which is used in the proof.

Lemma 10 Consider the (cylindrical-shaped) set S = U × I for U ⊂ R d -1 and an interval I ⊂ R . For θ ∈ R d with θ ⊤ e n ≥ 0 and any b ∈ R , define the set S ′ := S ∩ { x | x ⊤ θ ≥ b } . Then S ′ yields expanding sections with direction e n .

Proof To prove that they are expanding sections, we need to show that for any h > 0 , x 0 + he n ∈ Sec( S ′ , e n , y + h ) for any x 0 ∈ Sec( S ′ , e n , y ) . By the definition of sections, clearly x 0 + he n ∈ { x ∈ R d | x ⊤ e n = y + h } . Next, we must show x 0 + he n ∈ { x | x ⊤ θ ≥ b } . Since e ⊤ n θ ≥ 0 ,

holds.

<latexi sh

1_b

64="CE

Lf

/+AU

mdWXS

32QrM

>

GNn cjVT

FH

g

J

Y

R

o

p

q

Z

y

z

7

P

BD

k

w

O9

vK

8

0

u

I

5

<latexi sh

1_b

64="g/CHMj

9I

2oAn dZK

JT

5Oc

>

G

3

V

L

F

U

8

N

YRE

kqQ

m

y

7

S

+

Dv

0

W

r

z

f

p

X

B

w

u

P

<latexi sh

1_b

64="kVTjU

KHngX

Wcd

P

QCrL

yI

>A

G

RN

M

E

+

B

q

uJ

p

2

zO

F

0

3

/

v8

7

Z

w

m

9

D

YS

o

5

f

<latexi sh

1_b

64="XYV5ZI9

/J

2AkcWP

T

U

7nQ

>

GM

j

L

FH

8

B

C

m

o

p

q

Sr

E

f

u

03z

g

O

K

+

dv

N

y

D

w

R

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

<latexi sh

1_b

64="RySjdC5DkrowgBQ

q

GL

0Y

>A

M3

c

VT

FHU

J

E

m

p

ZX

W

OP

v2

I

+

8

K

9

zN

/

n

fu

7

<latexi sh

1_b

64="OYuXwyv

KB9

TrRG

ED

U

z

>A

Mn cjV

L

FHW

QCm

g

Z

pqo

S

2

J

8

I

f

d

P

N

75

3

/

0

k

+

<latexi sh

1_b

Pd5D

fY8A9yGZE7IoFU0

>

MX

cjVTL

H

g

J

R

mS

Cp

k

K

/

N

v

Q

w

B

3

zr+

u

2

O

n

<latexi sh

1_b

64="MuOA

2

Q

z/I

Eo

0F

D

7k

>

G

X

cjVTL

g

J

Y

R

m

Cp

q

Z

B

W

y

5

S

n

d

8

f

N

K

r

3

v

9

P

w

+

S

<latexi sh

1_b

64="CE

Lf

/+AU

mdWXS

32QrM

>

GNn cjVT

FH

g

J

Y

R

o

p

q

Z

y

z

7

P

BD

k

w

O9

vK

8

0

u

I

5

<latexi sh

1_b

64="ZgO

MnUdp

3oGB

9

w/

2

K

S

>A

X

cjVTL

FH

J

Y

E

R

m

C

q

v

7

I

f

+

k0

y

u

8

W

r

D

N

Q

z

P5

S

′

θ

I

U

e

n

F.4 Key Proposition for Suboptimality Gap

Next, we provide a fixed-history analysis to bound the margin constant of Challenge 2.

Before we start, we define the cylindrical set used to characterize the support of densities.

Definition 11 (Cylindrical region) We define A ⊂ R d as a cylindrical region with direction v ∈ S d -1 and length 2 H if

for some subset B ⊂ { x | x ⊤ v = 0 } . We denote this set by cyl( B,v,H ) .

v

A

2

H

B

Next, we show that if the support of each context Z i is a cylindrical set, we can bound the margin constant. We state our key result on bounding the margin constant in the fixed-history setup.

Proposition 3 (Key proposition: suboptimality gap) Assume that the random vector Z = ( Z ⊤ 1 . . . Z ⊤ K ) , Z i ∈ R d , satisfies LAC with function L ( · ) and ∥ supp( Z ) ∥ ∞ ≤ R . Additionally, suppose it satisfies the following conditions:

<latexi sh

1_b

64="WCqN

yGcU/VS7v3

JpDI

PFR0

>A

Mn

j

TL

H

g

Y

E

m

o

Z

K

z

d

rO

+

f

8

Q

X9

u

2

B

k

w

5

<latexi sh

1_b

64="Ojr5zyNG

Mkm0PRKvZu2p

IY

>A

X

c

VTL

FHU

g

J

E

oC

q

B

W

S7

n

d

8

f

3

D9

w

Q

+

/

<latexi sh

1_b

64="vuBQ

L

3

K

2

k

IMHU

yp

>A

G

X

F

g

J

Y

E

R

m

oC

r

W

O5

S7

n

d

8

f

N

z

D9

P

w

Z

+

/

q

0

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

Then

For all i ∈ [ K ] , supp( Z i ) ⊂ R d is identical for some set A ⊂ R d . That is, supp( Z ) = A K .

For some H > 0 , Z i 's support supp( Z i ) = A = cyl( B,θ ⋆ , H +1) for all i ∈ [ K ] , and P [ Z ∈ (cyl( B,θ ⋆ , H )) K ] ≥ 1 -δ .

θ

∗

2

H

1

1

F.4.1 Proof of Proposition 3

Define Z ⊤ i θ ⋆ = U i for all i ∈ [ K ] . With a slight abuse of notation, define the optimal arm a ⋆ = arg max i Z ⊤ i θ ⋆ and the suboptimal arm a † = arg max i = a ⋆ Z ⊤ i θ ⋆ .

̸

Decomposition by conditioning. We can bound the margin probability by conditioning as

̸

̸

̸

̸

̸

̸

̸

̸

We set the density of Z i | { max j = i z ⊤ j θ ⋆ ≤ H } ∩ Ω ⋆ i ( { z j } j = i ) and Z ⊤ i θ ⋆ | { max j = i z ⊤ j θ ⋆ ≤ H } ∩ Ω ⋆ i ( { z j } j = i ) as f 2 and g 2 .

̸

Our goal is to bound the maximum density of g 2 , which is a density of

̸

̸

̸

<latexi sh

1_b

64="DUYzA

frNGgpQd mR

W/+

HM

>

X

cjVTL

F

J

E

oC

q

Z

0

S

K

k

9

v

y

n

u

5

P

O

3

B8

7

w

I

2

̸

<latexi sh

1_b

64="WCqN

yGcU/VS7v3

JpDI

PFR0

>A

Mn

j

TL

H

g

Y

E

m

o

Z

K

z

d

rO

+

f

8

Q

X9

u

2

B

k

w

5

<latexi sh

1_b

64="gWDd

YAnw zqQL

vJ

Fc+5rSy

>

GOH

M

B

Cm

E

p

o

ZX

U

uP

2

I

8

f

R

K

3

7

/

N

0

9

k

<latexi sh

1_b

64="DUYzA

frNGgpQd mR

W/+

HM

>

X

cjVTL

F

J

E

oC

q

Z

0

S

K

k

9

v

y

n

u

5

P

O

3

B8

7

w

I

2

̸

[2] Support of f 2 ( · ) , g 2 ( · ) and their geometry. We examine the support of the conditional density f 2 , which is

̸

̸

̸

Under the event Ω ⋆ i ( { z j } j = i ) , we have z ⊤ θ ⋆ ≥ max j = i z ⊤ j θ ⋆ by the definition of Ω ⋆ i ( { z j } j = i ) . Since max j = i z ⊤ j θ ⋆ = b ≤ H , the support of f 2 becomes

̸

Recall A := supp( Z i ) and it has equal sections with direction θ ⋆ . Using Lemma 1, f 2 ( · ) has LAC with L ( R ) , and it has bounded decay rate √ d L ( R ) by Lemma 3.

̸

[3] Bounding one-side decay rate. Next, we aim to bound the one-side decay rate of Z ⊤ i θ ⋆ = y | { max j = i z ⊤ j θ ⋆ ≤ H } ∩ Ω ⋆ i ( { z j } ) . Define the (conditional) density of

̸

as g 2 ( y ) . Its support is restricted to the region { y | b ≤ y ≤ H +1 } .

Claim 3 The one-side decay rate of

̸

is bounded by √ d L ( R ) in the interval [ b, b + 1 2 ] .

Proof of Claim First, we observe that the support of

is an interval [ b, H +1] . This follows from the definition of Ω ⋆ i ( { z j } ) and our design of A . Moreover, this support has equal sections with direction θ ⋆ . Since b ≤ H , Lemma 8 applies, yielding the desired result.

̸

3) Bounding maximum density of g 2 by applying Corollary 5. From Lemma 5, the maximum density is bounded by 3 √ d L ( R ) in the interval [ b, b + 1 2 ] . Hence,

̸

̸

̸

2

G Proofs of Results for Unbounded Contexts

In this section, we prove our main theorems stated in Section 4.3. We aim to apply the key propositions, Proposition 2 and 3. To apply them, we first truncate our contexts to an appropriate region to fulfill the conditions of the propositions. First, we present ways to construct a truncation set, which is used to prove Theorems 2 and 3. After that, we prove the two Theorems 2 and 3 by applying Proposition 2 and 3.

G.1 Constructing Truncation Sets

This section presents operations to construct truncation sets that will be used in the proof of our main results. In the proof, we first truncate our contexts X ∈ R dK to a truncation set C = ∏ K i =1 D , where D ⊂ R d , and work with the truncated contexts. If the supremum norm of the set is bounded, by combining it with LAC, we can bound the decay rate of the truncated contexts.

First, we define the directional completion of v . For a set A and a vector v , we define the process of filling A in the direction of v . This process expands A so that its cross-section remains the same when cut by a hyperplane orthogonal to v . This procedure ensures that all sections of A along the direction v are equal. Recall that π S ( A ) denotes the projection of A onto the subspace S . If S is the subspace spanned by a vector v , we write π v ( A ) , which is a subset of a straight line.

̸

̸

Definition 12 (Set completion) For A ⊂ R d and v ∈ S d -1 , we define

After this completion, the section sliced by the normal hyperplane of v is the same.

A

v

C

[

A,v

]

Lemma 11 Suppose a set A ∈ R d and a unit vector v satisfy π v ( A ) is an interval with length ℓ . Then,

Proof For all p ∈ C [ A,v ] , there exists q ∈ A and some h with | h | < ℓ such that p = q + hv . Hence,

Next, we define partial completion, which makes the section with direction v in the region of C ( A,v ) ∩ [ -1 , 1] v equal.

Definition 13 (Partial completion) For a set A ⊂ R d and a unit vector v ∈ R d , define

Recall that π v ( · ) is a projection onto the direction v .

A

v

v

P

(

A,v

)

<latexi sh

1_b

64="cyN

8

0

oOK

+jH

RX

I5w

p

>A

GMn

VTL

F

U

g

J

Y

E

m

C

2kr

B

W

u

f

z

S/

Z

7

q

d

3

D

9

v

Q

P

<latexi sh

1_b

64="23

/u

XcdnkFGvVZ

S

C5Dm

I

>A

M

j

TL

HWBQ

w

Y

yq

U

r

K

P

z

J

p

9

EN

+

g8

R0

f

O

7

o

<latexi sh

1_b

64="HnLrOK3cCzSDW

Go/

u

5

0

>A

Q

jZTN

M

E

fd

q

U

R

Ik

m

V

v

y

2

F

+

P

9J

7

g

p

B

w

8

Y

X

<latexi sh

1_b

64="vuBQ

L

3

K

2

k

IMHU

yp

>A

G

X

F

g

J

Y

E

R

m

oC

r

W

O5

S7

n

d

8

f

N

z

D9

P

w

Z

+

/

q

0

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

<latexi sh

1_b

TU

E

C

Y

/

c

>

GQ

N

M

d

IF

K

wo3u

S

mq

R

0

y

v

k

r2

HP

J

5

O

7

W

8

p

B

g

z

X

L

<latexi sh

1_b

64="23

/u

XcdnkFGvVZ

S

C5Dm

I

>A

M

j

TL

HWBQ

w

Y

yq

U

r

K

P

z

J

p

9

EN

+

g8

R0

f

O

7

o

P ( A,v ) has the same section with the direction v within the region [ -1 , 1] v . Equivalently,

̸

for every u 1 = u 2 ∈ [ -1 , 1] . The partial completion operator has a bounded supremum norm, as shown in the following lemma.

Lemma 12 (Sup norm bound of P ( A,v ) ) For any A ⊂ R d and v ∈ S d -1 ,

holds.

Proof For any y ∈ P ( A,v ) , there is y ′ ∈ A such that y = y ′ + tv for some | t | ≤ 2 . Hence,

for any t and θ . Fix an arbitrary history H t -1 and any θ ∈ S d . For simplicity, define ( X ⊤ 1 , . . . , X ⊤ K ) as the conditioned random variable of ( X 1 ( t ) ⊤ , . . . , X K ( t ) ⊤ ) |H t -1 . Under the history H t -1 , we aim to apply Proposition 2 for any θ and v . We also set a = arg max X ⊤ i θ . For any fixed v ∈ R d with ∥ v ∥ = 1 , our goal is to calculate the lower bound of

We view it as a fixed history random variable , and we aim to use arguments developed in Section F.

To use Proposition 2, we first truncate our contexts ( X ⊤ 1 , . . . , X ⊤ K ) into some region C v 1 , then apply Proposition 2 to the truncated contexts. To satisfy the geometric conditions of Proposition 2, we construct the truncation as follows.

G.2.1 Constructing Truncation Sets

We define the truncation set as follows:

Define R 1 = c 0 x max (2 + log dK ) .

Define D := [ -R 1 , R 1 ] d .

Define D v := P [ D,v ] .

Set the truncation set C v 1 := ( D v ) K .

Then ∥ C v 1 ∥ ∞ ≤ R 1 +2 holds (by Lemma 12).

G.2 Proof of Theorem 2

Our goal is to prove

<latexi sh

1_b

64="z

HR

p5G

y

ZVvjND

2C

o

>A

X

c

TL

F

U

g

J

Y

E

mS

q

0

W

K

9

7

3

w

f

n

Ok

+

u

/

P

B

Q

I

r

8

<latexi sh

1_b

64="CE

Lf

/+AU

mdWXS

32QrM

>

GNn cjVT

FH

g

J

Y

R

o

p

q

Z

y

z

7

P

BD

k

w

O9

vK

8

0

u

I

5

<latexi sh

1_b

64="

XkrS

pKN

Y

0TM

/

2n oU

>A

G

cjV

L

FHWBQCmv

w

Iyq

+

uP

8

Jz

d

E

5

g

fO

3

DR

7

Z

9

<latexi sh

1_b

64="Kk3Or+y jz

d

7

pX

9JP

A

>

GM

c

VTL

FHU

82

I

N

YREo qQ

m

B

S

n

f

5

CW

v

0

/

wZ

g

D

u

<latexi sh

1_b

64="2PNKgnMZ

D

Q

pc kv

uGw+W0

>A

X

jVTL

FHU

J

Y

E

R

m

oC

q

S

/

9

8

5

f

B

3

zr

d

O

7

I

y

<latexi sh

1_b

64="ygGISCp

8

Av

E

c

d0D

oB

>

MX

jVTL

FHU

J

Y

R

m

q

Z

W

K

/k

9

N

Q

5

fw

z

n

+

u

P

O

3

7

r

2

O

v

D

v

D

θ

2

Claim 4 For any X satisfying Assumption 2, P [ X ∈ C v 1 ] ≥ 1 2 .

Proof First, we show P [ X ∈ D K ] ≥ 1 2 . Since C v 1 ⊃ D K , it suffices to prove the latter. Using (14), we get

Define the truncated contexts W = X | { X ∈ C v 1 } := ( W ⊤ 1 , . . . , W ⊤ K ) .

Claim 5 The truncated contexts W satisfy

Proof Recall a = arg max i ∈ [ K ] X ⊤ i θ . Note that

̸

̸

Since I { X = W } ≤ 1 , and P [ X = W ] ≥ 1 2 , we get

Hence, we only need to bound the diversity of the truncated contexts W = ( W ⊤ 1 , . . . , W ⊤ K ) .

G.2.2 Properties of Truncated Contexts W

We need to check two conditions: (i) The sections of supp( W i ) in the direction v within [ -1 , 1] v are the same for all i . (ii) W has a bounded decay rate √ d L ( R 1 +2) .

Since X satisfies LAC with functions L and ∥ C 1 ∥ ∞ ≤ R 1 +2 , it follows from Lemma 3 that X has a bounded decay rate of √ d L ( R 1 +2) .

Claim 6 The truncated contexts W have LAC with L ( · ) , and ∥ supp( W ) ∥ ∞ ≤ R 1 +2 .

Proof By Appendix C.3, conditioning on the event { X ∈ C v 1 } preserves LAC with the same function L . Also, by construction of C v 1 and Lemma 11, ∥ supp( W ) ∥ ∞ ≤ R 1 +2 .

Claim 7 For every i ∈ [ K ] , the support supp( W i ) is identical for all i , and supp( W i ) ∩ [ -1 , 1] v has equal sections in the v -direction.

Proof Since we set D v as the partial completion of D with direction v , the sections in [ -1 , 1] v become equal by construction.

G.2.3 Applying Proposition 2

We have verified that our truncated contexts W satisfy the conditions of Proposition 2. Hence, by Proposition 2, for some absolute constant c ′ > 0 ,

Combining with Claim 5, we get

for some absolute constant c > 0 and it ends proof.

G.3 Proof of Theorem 3

We now use the result of Proposition 3. First consider the case ∥ θ ⋆ ∥ 2 = 1 . For the general case, we can adjust via the argument in Appendix B.

Truncate our contexts X = ( X ⊤ 1 . . . X ⊤ K ) to some region C 2 = D K , where D ⊂ R d , with a slight abuse of notation. Define the truncated contexts as W = ( W ⊤ 1 , . . . , W ⊤ K ) . Note that the support of W i is D . To satisfy the conditions of Proposition 3, we want the sections of supp( W i ) in the direction θ ⋆ to be the same.

G.3.1 Constructing Truncation Sets

We construct the truncation set C 2 as follows. First, set R 3 = c 0 x max (1 + log dK + 1 2 log T ) and R 2 = R 3 +1 .

Set D 1 := ([ -R 2 , R 2 ]) d ⊂ R d .

Set D 3 := D 1 ∩ D 2 .

Set D 2 := { x ∈ R d | -R 2 ≤ x ⊤ θ ⋆ ≤ R 2 } ⊂ R d .

Set D := C [ D 3 , θ ⋆ ] .

The final truncation set is C 2 = D K .

Claim 8 The sup norm of the truncation set satisfies ∥ D ∥ ∞ ≤ 3 R 2 . Also, W has bounded LAC with L (3 R 2 ) .

Proof By Lemma 1, the LAC of W is the same as the original, L ( · ) . By Lemma 11, ∥ D ∥ ∞ ≤ 3 R 2 since π θ ⋆ ( D 2 ) is an interval with length 2 R 2 . Since L ( · ) is increasing and ∥ D ∥ ∞ ≤ 3 R 2 , W has bounded LAC with L (3 R 2 ) .

Claim 9 The region C 2 has equal sections in the direction θ ⋆ and P [ X ∈ C 2 ] ≥ 1 -1 √ T .

Proof By Assumption 2, P [ | X ij | ≤ R 2 ] ≥ 1 -1 2 dK √ T holds for all i, j . Hence, P [ X ∈ D K 1 ] ≥ 1 -1 2 √ T . By the same assumption, P [ X ∈ D K 2 ] ≥ 1 -1 2 √ T . Thus, P [ X ∈ D K 3 ] ≥ 1 -1 √ T . Since D ⊃ D 3 , P [ X ∈ C 2 ] ≥ 1 -1 √ T .

G.3.2 Truncation with a High-Probability Region

We truncate X into the high-probability region C 2 and define the truncated contexts W = ( W ⊤ 1 , . . . , W ⊤ K ) . We then work with W .

The following calculation shows that it suffices to bound the suboptimality gap of W :

Therefore, if we find the constant C ′ ∆ satisfying we get C ∆ ≤ C ′ ∆ .

Lemma 13 The truncated contexts W = X | { X ∈ C 2 } meet the condition of Proposition 3 with H = R 2 +1 and δ = 1 √ T .

Proof Let D = cyl( B,θ ⋆ , R 2 ) for some set B ⊂ P θ ⋆ , and define D ′ = cyl( B,θ ⋆ , R 2 -1) . By a similar argument to Claim 9, P [ X ∈ D ′ ] ≥ 1 -1 √ T . Hence,

Thus, the condition of Proposition 3 is satisfied with δ = √ T

G.3.3 Applying Proposition 3

By Proposition 3, we can ensure that (12) holds for

Hence,

G.4 Proof of Theorem 1: Unbounded Contexts Case

Combining Proposition 9 with our results from Theorems 2 and 3, we get

By substituting our bounds on C ∆ and λ ⋆ , we have

H Results for Bounded Contexts

Now, we present our main result for bounded contexts.

H.1 Two Cases of Bounded Contexts

In the linear contextual bandit setting, ℓ 2 boundedness is widely used. However, for light-tailed distributions (such as Gaussian or exponential), the ℓ 2 norm is unbounded. Therefore, we divide our analysis into two cases: unbounded and bounded contexts. While our previous focus was on unbounded contexts, here we summarize and present results for bounded contexts.

We classify the analysis into two cases: the first is for truncated contexts, and the second is for naturally bounded contexts, which include cases where the uniform distribution on the ball B R or a distribution with bounded density on the ball.

Truncated contexts. Truncated contexts refer to cases where the context distribution is truncated from the original distribution. Examples include truncated Gaussian and truncated exponential distributions.

Naturally bounded contexts. Naturally bounded contexts include uniform distributions on a ball or distributions with bounded density defined on the ball.

H.2 Regret Bounds for Bounded Contexts

Next, we present our result for the regret bound. The first case is when we receive truncated contexts, generated by truncating unbounded contexts to ( B R ) K , where B R is a d -dimensional ball of radius R . For the ℓ 2 -bounded case, R may depend on the dimension d when we choose a large R , so we do not hide the R term in our main regret bound. Corollary 1 and 2 gives the regret bound for truncated contexts. Corollary 3 gives the regret bound for naturally bounded contexts.

Corollary 1 (Regret bound: truncated contexts) Let X ( t ) be unbounded contexts with the same condition as Theorem 1. For R ′ > 0 with P [ X ( t ) ∈ ( B R ′ ) K ] = p > 0 , we receive truncated contexts X ( t ) | ( B R ′ + r ) K for some r > 0 each round t ≥ 1 . In this case, for R = R ′ + r , the regret of Algorithm 1 is bounded by

when r ≍ R ′ .

Corollary 2 (Regret bound: high-prob truncated contexts) Let X ( t ) be unbounded contexts with the same condition as Theorem 1. We receive truncated contexts X ( t ) | ( B L ) K for L ≳ √ dx max (1 + log d +log K ) each round t . In this case, the regret of Algorithm 1 is bounded by

Discussion. For Corollary 1, when we receive truncated light-tailed distributions (e.g., Gaussian, exponential, Laplace), this theorem states that they enjoy a logarithmic regret bound. It has a poly(log T ) regret bound for T and depends on R , the truncation radius. Corollary 2 states that when we choose a sufficiently large truncation radius, the bound becomes radius-free, matching the unbounded case (Theorem 1). Proofs for these corollaries are presented in Appendix I.

Next, we introduce the regret bound results for naturally bounded contexts. Here, new parameters c ⋆ and p ⋆ are introduced, along with Condition 1. This condition will be discussed in Appendix H.3, where we clarify that both uniform distributions and bounded-density distributions satisfy it. We will provide detailed explanations and state the range of c ⋆ , p ⋆ for several distributions, including the uniform distribution.

Corollary 3 (Regret bound: naturally bounded contexts) Let naturally bounded contexts X ( t ) ∈ ( B R ) K satisfy the LAC condition with function L ( · ) and Condition 1 holds with concentration

parameters c ⋆ , p ⋆ each round t . Under Assumption 1 and 2, the regret of Algorithm 1 is bounded by

Discussion. Corollary 3 applies to the uniform distribution and any distribution with bounded density on the ball. It also covers the truncation of heavy-tailed distributions. For Corollary 3, the polynomial dependency on K is due to boundedness, and it matches the asymptotic result of the uniform distribution studied in [12] when d = 1 . Combining with Lemma 15, for a uniform distribution or a bounded density distribution in the ball, we have Reg ( T ) ≤ ˜ O ( K d +5 d +1 d 2 . 5 ) . For Condition 1 and parameters c ⋆ and p ⋆ , we deeply discuss it in the following part. Proofs for these corollaries are presented in Appendix I.

Comparison with known results. For truncated contexts, to the best of our knowledge, there are no known results for greedy bandits. For naturally bounded contexts, Oh et al. [28] studied the case of a uniform distribution on the sphere, where each context X i for arm i is independent across arms, and they only considered the case K ≤ d . Their work assumes the minimum eigenvalue of the context covariance matrix is constant; however, for a uniform distribution, it scales as ≍ 1 d . Taking this into account, their regret bound is ˜ O ( d 3 √ T ) when K ≤ d . When K ≤ d , our result has the bound ˜ O ( d 2 . 5+ d +5 d +1 ) for the uniform distribution. Moreover, for the multi-parameter shared context setup, Bastani and Bayati [7], Bastani et al. [8] studied the uniform context case, and their worst-case regret bound is ˜ O ( K 4 d 4 ) , considering that the minimum eigenvalue of the context covariance scales as 1 d . In conclusion, our result is the sharpest among known previous results.

H.3 Concentration Parameters for Bounded Contexts

We introduce a new condition that defines two concentration parameters measuring sufficient concentration for bounded contexts. We will show that truncated contexts satisfy this condition, as do naturally bounded contexts with bounded density. This condition does not need to be assumed for truncated contexts, as it is automatically satisfied.

Condition 1 (Concentration parameters for bounded contexts) For the random vectors (contexts) X = ( X ⊤ 1 . . . X ⊤ K ) ∈ R dK where X i ∈ B R , there exist 0 < c ⋆ < 1 and 0 < p ⋆ < 1 such that for every η with ∥ η ∥ 2 = 1 ,

We call p ⋆ , c ⋆ the concentration parameters.

Discussion. Two parameters, c ⋆ and p ⋆ , must exist if X is random. Below, we discuss 1 -c ⋆ and p ⋆ ≍ 1 for truncated contexts truncated into the positive-probability region. Additionally, we show that a uniform distribution within the ball, as well as any distribution within the ball with bounded density, satisfies this condition, and we explicitly calculate the range of these two parameters. For our regret bound, we have a dependence on p ⋆ (1 -c ⋆ ) 2 .

Example: truncated contexts. The next lemma states that truncated contexts satisfy Condition 1 with 1 -c ⋆ , p ⋆ ≍ 1 . If contexts are generated by truncating the original distribution into a positiveprobability region, then the parameters in Condition 1, c ⋆ , p ⋆ , are well-defined and do not harm the regret bound when we choose a sufficiently large truncation set.

Lemma 14 (Concentration parameters for truncated contexts) Suppose the unbounded contexts X ∈ R dK satisfy for some R ′ > 0 . Then, if we truncate each X i to B R ′ + r for any r > 0 , i ∈ [ K ] , the truncated contexts X = ( X ⊤ 1 , . . . , X ⊤ K ) satisfy Condition 1 with p ⋆ = p, c ⋆ = R ′ R ′ + r .

Proof Recall that we truncate X to ( B R ′ + r ) K . If P [ X ∈ ( B R ′ ) K ] = p > 0 and we truncate to B R ′ + r ,

Therefore, we can set p ⋆ = p and c ⋆ = R ′ R ′ + r .

Example: Bounded Density Distributions The lemma below tells us that a uniform distribution or a bounded density distribution on B R satisfies Condition 1 and provides the range of the two parameters. Without loss of generality, we prove it for a distribution defined on the unit ball B 1 .

Lemma 15 Consider a random vector X i ∈ B 1 , i ∈ [ K ] , each with density upper-bounded by c u ω d . Here, ω d is the volume of the d -dimensional unit ball B 1 . (Recall that the density of the uniform distribution in B 1 is 1 ω d .)

When X 1 = · · · = X K (strongly correlated), Condition 1 holds with

When X 1 , . . . , X K are independent (not correlated), we have

Proof We use the result of Claim 10 below.

Correlated Case. If X 1 = X 2 = · · · = X K , then the 1 2 -quantile c ⋆ of X ⊤ i η satisfies

Since ( ω d ω d -1 ) 2 d +1 ≍ 1 , it follows that

Independent Case. If X ⊤ 1 , . . . , X ⊤ K are independent, then for a certain c ⋆ with

we have P [ X ⊤ i η ≤ c ⋆ R ] ≤ 1 -1 K for all i ∈ [ K ] . By independence, p ⋆ = (1 -1 K ) K ≤ 1 2 .

Claim 10 Consider a random vector X ∈ R d defined in B 1 with density upper-bounded by c u ω d . For any η ∈ S d -1 , let the (1 -p ) -quantile of X ⊤ η be α . Then

Proof We show that α with 1 -α 2 = ( p c u ω d ω d -1 ) 2 d +1 satisfies

Since the density of X ⊤ η = r satisfies

H.4 Results for Two Challenges: Bounded Contexts

Now, we present results related to two challenges for bounded contexts. Proofs are provided in Appendix I.

Proposition 4 (Diversity constant: naturally bounded contexts) Suppose X ( t ) is a random vector supported in ( B R ) K , its density satisfies the LAC condition with constant function L ( R ) , and Condition 1 with p ⋆ , c ⋆ holds. Then

for some absolute constant c > 0 .

Proposition 5 (Diversity constant: high-prob truncated contexts) Suppose X ( t ) ∈ R d × K satisfies LAC with L ( · ) and Assumption 2. Then we truncate X ( t ) to ( B L ) K for some L ≥ c √ dx max ( 1 + log ( dKx max λ ⋆ )) and define these truncated contexts as X ( t ) = ( X 1 ( t ) , . . . , X K ( t )) . Then X ( t ) has a diversity constant (Challenge 1) with 1 2 λ ⋆ ( t ) , where λ ⋆ ( t ) is the diversity constant of X ( t ) .

Proposition 6 (Suboptimality gap: truncated contexts) Let X ( t ) be a truncated random variable of X ( t ) into ( B R ) K with P [ X ∈ ( B R ) K ] ≥ 1 -δ for some δ > 0 . Let C ∆ ( t ) be the margin constant of the contexts before truncation. Then the margin constant of truncated contexts X ( t ) , denoted by C ∆ ( t ) , satisfies

Proposition 7 (Suboptimality gap: naturally bounded contexts) For naturally bounded contexts X ( t ) with LAC function L ( · ) , the margin constant is bounded by for some absolute constant c > 0 .

Discussion of Proposition 7. It has a linear factor in K , which can worsen the regret bound compared to LinUCB and LinTS when the number of arms K is large. Nevertheless, this factor arises in the logarithmic regret bound, which is still advantageous compared to algorithms that achieve O ( √ T ) regret when T ≫ K . Also, if every X i ( t ) follows a uniform distribution independently, then extreme value theory [12] implies there should be dependence on K . Hence, our result matches the lower bound for the uniform distribution, which indicates it cannot be improved.

we have

I Proofs of Results for Bounded Contexts

We provide proofs for Appendix H. For simplicity, we assume R ≥ 1 . Using the observation of Appendix C.1, any density defined in B R with LAC L ( · ) has a bounded decay rate √ d L ( R ) by combining Lemma 3.

I.1 Fixed History Arguments

̸

We fix again the time step t and the history H t -1 and derive the fixed history results for the diversity and suboptimality gap. We use the same fixed history arguments in Appendix F. We set the historyconditioned contexts as X := X ( t ) | H t -1 , and X = ( X ⊤ 1 , . . . , X ⊤ K ) , X i ∈ R d . To address any greedy policy with respect to ˆ θ t -1 , we propose an analysis that applies to any greedy policy with an arbitrary θ ∈ R d . In Appendix F.1, we already argued that it suffices to bound this variance for any fixed θ . Since we fix θ , which is the corresponding value of ˆ θ t -1 under the given history H t -1 , we define the policy-selected arm a = arg max X ⊤ i θ . Following the previous definitions, define the event Ω i := { a = i } and Ω i ( { x j } j = i ) := { a = i } ∩ { X j = x j for all j = i } . Similarly, define Ω ⋆ i := { a ⋆ = i } and Ω ⋆ i ( { x j } j = i ) := { a ⋆ = i } ∩ { X j = x j for all j = i } .

̸

̸

̸

̸

Event decomposition and conditional density. We first define b = max i = a X ⊤ i θ . Similar to the unbounded contexts, we decompose our diversity as

̸

Therefore, our next interest is the projected contexts, defined as

̸

Let us define this projected context's density as g ( · ) . We first investigate the support of the conditional random vector,

̸

and define its density as g ( · ) . The projected context's density g is the integration of f within the section { x ∈ B R | x ⊤ θ ≥ b } ∩ { x ∈ B R | x ⊤ v = y } . Hence, the density of X ⊤ i v ∣ ∣ ∣ Ω i ( { x j } ) ∩{ b ≤ c ⋆ R } is a section density of { x ∈ B R | x ⊤ θ ≥ b } with direction v .

I.2 Sections of the Ball

Next, we aim to bound the one-side decay rate of the section density in the ball. Unlike the previous sections, the sections corresponding to Ω i ( { x j } j = i ) no longer make expanding sections due to the boundary of B R . To deal with sections for bounded contexts, we define several sections of the ball.

̸

Definition 14 (Sliced ball) We define the sliced ball S R ( v, y ) as

Also define the double sliced ball

̸

S

R

(

θ, b

)

S

R

(

θ, b, v, y

)

θ

v

For bounded contexts, we need to bound the one-side decay rate of the section density, where the sections are sliced balls. Recall that we set b = max j = a x ⊤ j θ . We aim to obtain the lower bound of

̸

̸

̸

for any Ω i ( { x j } j = i ) , b . Observe that the support of v ⊤ X i = y ∣ ∣ Ω i ( { x j } j = i ) ∩ { b ≤ c ⋆ R } is the section of a double sliced ball, S R ( θ, b, v, y ) . Here, we aim to bound the one-side decay rate of these section densities to apply Lemma 4.

I.3 Linear Section Maps

To deal with varying sections, we define linear section maps , and using linear section maps, we can bound the one-side decay rate of the section densities. We first define a projection map, a projection to the center of similarity.

Definition 15 (Projection map) We define the projection map between A ⊂ R d and P ∈ R d as a map Φ : A → P , which is an affine point projection with the center of similarity at P . Furthermore, we call P the projection point.

A

P

<latexi sh

1_b

64="rSLI8B

ov

OjcknHJ

Zp

R

C

Q

>A

GTX

V

M

F

U

5

d3

E

mq uK

W

2

f

+

D

0

w

7

N

Y

P

g

y

9z

/

<latexi sh

1_b

64="CE

Lf

/+AU

mdWXS

32QrM

>

GNn cjVT

FH

g

J

Y

R

o

p

q

Z

y

z

7

P

BD

k

w

O9

vK

8

0

u

I

5

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

<latexi sh

1_b

64="P

MgC

kv/

YLAN

8

F

+E

03

>

GSX

H

U

p5

B

o

ZW

K

Dq

J

R2

d

O

r

z

I

w

m

9

f

u

n

Q

y

7

<latexi sh

1_b

64="

kP

nOm0pQ

MY

2

Zq

IF

w

H

>A

G

X

cjVTL

U

g

J

E

R

oC

K

r

v

B

W

y

5

S7

d

8

f

N

u

z

3

D9

+

/

<latexi sh

1_b

64="y cd

UPZ

9Q+E

82W

Y

qm

M

>A

G

X

jVTL

FH

g

J

R

Cp

k

0S

K

/

N

vo

5

fw

B

3

zr

u

O

7

n

I

D

Definition 16 (Linear section maps) We define linear section maps between sections. For two sections of A , Sec( A,v, y ) and Sec( A,v, y + h ) , we define the linear section map Φ h y as

which satisfies Φ h y (Sec( A,v, y )) ⊂ Sec( A,v, y + h ) and Φ h y is a part of some projection map with center P .

P

Sec(

A,v, y

)

Sec(

A,v, y

+

h

)

v

Φ

h

y

H

I.4 One-side Decay Rate of Linear Section Maps

Our subgoal is to bound the one-side decay rate of the section density, and we use linear section maps to achieve this. Previously, we dealt with expanding or equal sections; hence section maps directly gave the lower bound of the density. However, for general linear section maps, they are no longer expanding sections due to boundaries. Assume the density f is defined in B R and it has a decay rate M . If f has LAC with function L ( · ) , we already studied in Lemma 3 that it has a decay rate with M = √ d L ( R ) . We can set L ( R ) ≥ 10 .

Slope of linear section maps. We first bound the maximum length between Φ h y ( x ) and x . It is related to the slope of the section maps, and we define s > 0 such that

for any x ∈ Sec( A,v, y ) . We call s the slope of the linear section map Φ h y ( · ) .

Volume element. We define | det( ∇ Φ h y ( x )) | := u y ( h ) . This value is the same for any x ∈ Sec( A,v, y ) , since we use linear section maps. Using that, we can calculate the lower bound of g ( y + h ) as follows:

<latexi sh

1_b

64="72v0O59ZLRATJ

q

K

cEI

>

GS3

jV

N

M

HY

+W

BF

U

r

z

f

C

D

/

P

o

n

d

y

m

k

8

w

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

<latexi sh

1_b

64="LV

IEHnJ

z

MY

ZygjSv

>A

GO

c

T

F

U

R

m

oCp

q

X

r

N

8ku

03

2

/

d

wQW7

5

f

+

9

P

B

D

K

<latexi sh

1_b

64="2rMB

/

y

kd

7

q

vJ

R+Gj

>A

TX

c

V

N

EHULo

8

C

F

I

Y

0

QO

K3

m

g

nP

D

S

z

W

9

f

5

Z

p

w

u

<latexi sh

1_b

64="y3

5F

MT8

Q

r

X0KCo

+

k

>A

G

cjV

L

g

J

Y

E

R

m

p

q2

d

SZW

/

9

v

fw

BP

z

N

u

O

n

I

D

7

<latexi sh

1_b

64="ENyO9HGAqF

CT

U

nw5+Wrk

>

MX

cjV

L

g

J

Y

R

mSo

p

Z

0

K

/

8

v

Q

f

BP

3

z

d

u

2

7

I

D

The third inequality uses the Gronwall inequality, Lemma 22, applying that the decay rate is bounded by M and the length between x and Φ h y ( x ) is bounded by s h . This formula tells us that to bound the one-side decay rate of the section density, there are two quantities: slope s and volume element u y ( h ) .

A way to bound u y ( h ) . Let us define H as the distance between Sec( A,v, y ) and P . The volume element can be calculated as

since H -h H is the ratio of similarity. Then,

When h ≤ H 2 , we have log ( H -h H ) ≥ 1 -2 h H , so

This implies that when we can find the linear section map Φ h y , we can bound the one-side decay rate of the section density by Ms + 2 d H .

I.5 Linear Section Maps between Sliced Balls, S R ( v, y )

P

S

R

(

θ, y

)

S

R

(

θ, y

+

h

)

H

Lemma 16 (One-side decay rate of S R ( v, y ) ) The projected density g ( y ) of sections S R ( v, y ) satisfies

Proof Let y ′ = y + h ∈ [ -1 √ d , 1 √ d ] . Set P as the similarity center of S R ( v, y ) and S R ( v, y + h ) . We aim to apply the arguments of Appendix I.4. Using this P , we can form section maps Φ h y . First, the slope of Φ h y is bounded by s ≤ R √ R 2 -( y + h ) 2 ≤ 2 for | y | ≤ 1 √ d . The volume element of Φ h y

<latexi sh

1_b

64="fO+7KXz/

J

Sk9ZP

AV

FM

>

G

cj

TL

H

U

p5

B

go

Dq

R2

0C

8

d

r

E

W

I

w

m3

N

v

u

Q

y

Y

n

<latexi sh

1_b

64="rPGIf

M

JT

C0vU

Yo

HLKc

>A

Sn jV

F

p5

B

k

Zmq

Ru

W

2

E

w9

/

gNd

O

z

+

7

3X

y

Q

D

8

<latexi sh

1_b

64="JkPAMd7

fD

qIR

T

W

CY

>

G

cjV

L

FH

BQ

mv

n

w

y

U

2

r

0

SZ

Ko9z

3

5g

p

N

E

+

8

/

O

<latexi sh

1_b

64="ENyO9HGAqF

CT

U

nw5+Wrk

>

MX

cjV

L

g

J

Y

R

mSo

p

Z

0

K

/

8

v

Q

f

BP

3

z

d

u

2

7

I

D

can be interpreted as the ratio of similarity. Let H be the distance of P and S R ( v, y ) . Note that H ≥ R 2 -( y + h ) 2 y + h and hence H ≥ 2 h holds. By applying the result in Appendix I.4, its one-side decay rate in [ -1 √ d , 1 √ d ] is bounded by

I.6 Linear Section Maps for Double Sliced Balls, S R ( θ, b, v, y )

Goal. We investigate the one-sided decay rate of section densities with sections S R ( θ, b, v, y ) for fixed θ, v, b . For density f ( x ) defined in B R with LAC function L ( · ) , we define the section density of S R ( θ, b, v, y ) as g ( y ) . We set M = 4 √ d L ( R ) + 16 √ d R 2 (1 -c ⋆ ) . We prove that the one-side decay rate of g at y ∈ [ -1 M , 1 M ] is bounded by M . Also, we define a small angle τ 0 := sin -1 ( 1 M R ) .

I.6.1 Case v ⊥ θ .

For this case, the sections S R ( θ, b, v, y ) are no longer expanding sections when ⟨ θ, v ⟩ is close to π 2 . In that case, we can construct a linear section map, and the following figure illustrates the procedure. In each section S R ( θ, b, v, y ) and S R ( θ, b, v, y + h ) , assume the highest coordinate with respect to direction θ is Q and Q h . Assume the directed half-line starting from connecting Q to Q h and the hyperplane { x | x ⊤ θ = b } meets at some point P . Then we can set the intersection point P as the center of the projection map and we can construct the linear section map using P . Please see Figure 15 for the intuition.

θ

v

S

R

(

θ, b, v, y

)

P

H

H

′

S

R

(

θ, b, v, y

+

h

)

b

Q

Q

h

Lemma 17 (One-side decay rate of double sliced ball) Suppose b ≤ αR for some α < 1 . If v ⊥ θ , suppose we construct a linear section map by the sliced balls between S R ( θ, b, v, y ) and S R ( θ, b, v, y + h ) for b ≤ αR in the way described above. By setting M = 2 √ d L ( R ) + 8 √ d R 2 (1 -α ) , the one-side decay rate of the section density g ( · ) in the interval -1 M < y < y + h < 1 M is bounded by

<latexi sh

1_b

64="

ukJQp qc+9

ro7U

X

wV

/K

Y

>A

GM

j

TL

FH

g

E

R

m

C

Z

0

S

W

N

v

f

y

z

n

5

D

P

O

3

B8

I

2

d

<latexi sh

1_b

64="M3EAGo mKQkuyz

YHO

7Zr

>

Tn cjV

L

F

U

W8

gBC

Iq

N

R

p

S

0

X

9

J

w

/

v

P

5

f

D

+

2

d

<latexi sh

1_b

64="CE

Lf

/+AU

mdWXS

32QrM

>

GNn cjVT

FH

g

J

Y

R

o

p

q

Z

y

z

7

P

BD

k

w

O9

vK

8

0

u

I

5

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

<latexi sh

1_b

64="BCE

OGm

7yPVpJL

N2nH

k

I

>A

MX

c

T

F

U

g

Y

R

o

q

0

S

W

K

/

9

8

v

Q

5

fw

z

+

u

D

3

r

d

<latexi sh

1_b

64="nk

HUqd5

V9A

v

L

u8C

m

/F

E

>

GM3

cj

T

g

I

W

oy

Q

r

J

w

2

N

0

SP

B

Z

O

R

p

zf

+

Y

7

D

K

X

<latexi sh

1_b

64="

MXdkD

nj wrC

K

pu7

Z0c

>A

GT

V

L

FHU

82

IOW

ES

q

3

N

y

R

P

v

f

B

g

9

/

5

J

oY

m

Q

+

z

<latexi sh

1_b

64="y3

5F

MT8

Q

r

X0KCo

+

k

>A

G

cjV

L

g

J

Y

E

R

m

p

q2

d

SZW

/

9

v

fw

BP

z

N

u

O

n

I

D

7

<latexi sh

1_b

64="

I9+YFpUWBS

P

GDLyno

T

qzQ

>A

M

cjV

H

Cmv

w

g

EZ

K

N

f

0

2

/

O

87

3d

R

r

J

5k

<latexi sh

1_b

64="fQ

W2MLwS

E9

Np

U

Ju

>A

G

X

F

g

Y

R

m

oC

q

Z

k

0

K

z

7

3

y

5

v

d

n

O

/

D

P

B

I

8

Proof We aim to estimate s and H to apply the arguments in Appendix I.4. To bound s , we can easily see that s ≤ R √ R 2 -( y + h ) 2 ≤ 2 , since it is related to the slope of the tangent line at y + h .

To bound H , with some elementary calculations, we have

H ′ is defined in Figure 15. Then we see

for any y, y + h ∈ [ -1 M , 1 M ] . Then,

The remaining part is proved by the argument from Appendix I.4.

I.6.2 Case π 2 -τ 0 ≤ ∠ ( v, θ ) ≤ π 2

In this case, we define θ ′ := θ -v ( v ⊤ θ ) ∥ θ -v ( v ⊤ θ ) ∥ 2 and we aim to define section maps between S R ( θ, b, v, y ) using v and θ ′ . For new sections, S R ( θ ′ , b, v, y ) , we can make section maps as described in Lemma 17. We can see that these section maps also become section maps between S R ( θ, b, v, y ) . By using Lemma 17 for α with 1 -α = 1 -c ⋆ 2 and θ ′ , v , we have

holding for M = 2 √ d L ( R ) + 16 R 2 (1 -c ⋆ ) and any -1 M < y < y + h < 1 M .

I.6.3 Case 0 ≤ ∠ ( v, θ ) ≤ π 2 -τ 0

In this case, the natural section map between S R ( v, y ) and S R ( v, y + h ) for any -1 M < y < y + h < 1 M can be an expanding section map for y, y + h ∈ [ -1 M , 1 M ] . We already proved that the one-side decay rate of the section density is bounded by

<latexi sh

1_b

64="M3EAGo mKQkuyz

YHO

7Zr

>

Tn cjV

L

F

U

W8

gBC

Iq

N

R

p

S

0

X

9

J

w

/

v

P

5

f

D

+

2

d

<latexi sh

1_b

64="

MXdkD

nj wrC

K

pu7

Z0c

>A

GT

V

L

FHU

82

IOW

ES

q

3

N

y

R

P

v

f

B

g

9

/

5

J

oY

m

Q

+

z

<latexi sh

1_b

64="CE

Lf

/+AU

mdWXS

32QrM

>

GNn cjVT

FH

g

J

Y

R

o

p

q

Z

y

z

7

P

BD

k

w

O9

vK

8

0

u

I

5

<latexi sh

1_b

64="XzJ8PTcN

O

d

yo7

q

D

k

Q

>A

GS3

jV

MwF

Y

H+

rgC

Z

m

p

0

2

U

n

Wv

K

f

EB

5u

L

9

/

R

I

̸

<latexi sh

1_b

64="S3MY0AHnz

m

FgJwK

f

j

o

>

G

X

c

VTL

U

E

R

Cp

q

Z

W

/k

9

N

8

v

Q

5

+

u

D

P

O

y

B

7

r

d

2

I

<latexi sh

1_b

2PvJzH

RCnuK

LFS

qMk

>A

GQ3

cjZTN

fd

Ur

I

0

VB

9

o

X8

W/

y

7

w

g

5

O

D

Y

m

+

̸

Further observe the density of

It is a section density of sections S R ( θ, b, v, y ) .

Case 1: π 2 -τ 0 ≤ ∠ ( θ, v ) ≤ π 2 . By applying results from Appendix I.6 and Lemma 4, we have

Case 2: 0 ≤ ∠ ( θ, v ) ≤ π 2 -τ 0 . By applying results from Appendix I.6 and Lemma 4, we have

Case 3: ∠ ( θ, v ) ≥ π 2 . When we replace v with -v , it falls into Case 1 or 2.

<latexi sh

1_b

64="cNg0

A7WwTvq2EO

B

p8GI

>

RX

M

fdQ

CUr5

JV

9

o

K

H

S

/

y

F

L

z+

u

3

P

n

D

Ym

k

θ

v

{

X

/latticetop

θ

b

}

S

R

(

θ, b, v, y

+

h

)

S

R

(

θ, b, v, y

)

S

R

(

v, y

)

S

R

(

v, y

+

h

)

I.7 Proof of Proposition 4

We first fix θ and v with the same argument as in the proof of Theorem 2. For fixed history contexts X = ( X ⊤ 1 , . . . , X ⊤ K ) and fixed θ and v , define b as the second largest value of ( X ⊤ 1 θ, . . . , X ⊤ K θ ) . By Condition 1, we first bound

̸

Then, we aim to bound

̸

I.8 Proof of Proposition 5

Proof Using a similar argument, we use the fixed history arguments. Write X = ( X 1 , . . . , X K ) = X ( t ) | H t -1 and we first fix θ and consider the greedy policy a with estimator θ . We define λ ⋆ as a diversity constant of X . Also, we define L n := c 0 √ dx max ( 1 + log dK + n log γ ) and set the event B n := { X i ∈ B (0 , L n ) for all i ∈ [ K ] } . We determine γ > 0 later. By Lemma 19, we have P [ B n ] ≥ 1 -1 γ n . We apply the peeling technique to bound the truncated contexts:

By setting α = c 0 √ dx max , β = 1 + log dK , observe that

If γ ≥ 3 α 2 β 2 1 λ ⋆ , we have

Using Theorem 2, we get and hence

Finally, we get

I.9 Proof of Proposition 6

Define ∆( X ) as the suboptimality gap of X and denote its density as f X ( x ) , x ∈ ( R d ) K . By direct expectation, we get which holds.

I.10 Proof of Proposition 7

By conditioning on H t -1 , we define fixed history contexts X = ( X ⊤ 1 , . . . , X ⊤ K ) , similar to previous proofs in Appendix G. We decompose the probability of the suboptimality gap as

̸

̸

̸

the last equality holds by the definition of Ω ⋆ i . Next, we aim to bound

̸

̸

̸

Similarly, we only need to bound the maximum density of

̸

and it is enough to bound its one-side decay rate by Lemma 5. Since the conditional density of

̸

̸

̸

say f 3 ( · ) , has the same LAC with constant function L ( R ) , it thus has a bounded decay rate √ d L ( R ) by Lemma 3. Also, the density P [ X ⊤ i θ ⋆ = y ∣ ∣ { X j = x j } j = i ] is a section density of S R ( θ ⋆ , y ) for y ∈ [ -R,R ] . Then we observe that in at least one of the directions of θ ⋆ or -θ ⋆ , the sections S R ( θ ⋆ , y ) are expanding sections in B R . By applying Lemma 8, we can bound the one-side decay rate of the section density X ⊤ i θ ⋆ ∣ ∣ { X j ( t ) = x j } j = i by √ d L ( R ) , and finally we can bound the maximum density by 3 √ d L ( R ) using Lemma 5. Therefore, by the decomposition above, we finally get

I.11 Proof of Corollary 1

We have λ ⋆ ( t ) ≥ c p d 1 ( ( R )+ 1 R (1 -c⋆ ) ) 2 with 1 -c ⋆ = R ′ r ≍ 1 by using Proposition 4 and Lemma 14.

Hence, we get λ ⋆ ( t ) ≥ c d L ( R ) 2 := λ ⋆ . Also, using Proposition 6, we have the margin constant of truncated contexts ¯ C bounded by 1 C , where C is defined in Theorem 3. We can see that

L p ∆ 1 -p ∆ ∆

holds.

I.12 Proof of Corollary 2

If L ≥ √ dc 0 x max (3 + log dK ) , we can use the same proof as in Theorem 2. For the diversity constant of truncated contexts, it is lower bounded by 1 2 λ ⋆ ( t ) , where λ ⋆ ( t ) is defined in Theorem 2. Also, using Proposition 6, we have the margin constant of truncated contexts ¯ C ∆ bounded by 1 1 -p C ∆ , where C ∆ is defined in Theorem 3. Combining these observations, we can finally apply Proposition 9 since truncated contexts also have bounded ψ 1 -norm by x max , and we get the desired result.

̸

I.13 Proof of Corollary 3

We can prove it directly by combining the results of Proposition 4 and Proposition 7. This can be obtained directly by combining those results with Proposition 8.

J Analysis After Challenges 1 and 2 are Satisfied

We now present the results and proofs to obtain an exact regret bound after the two challenges are addressed. Recall that we define the (unexpected) regret as reg ′ ( t ) := X a ⋆ ( t ) ( t ) ⊤ θ ⋆ -X a ( t ) ( t ) ⊤ θ ⋆ and the expected regret as reg( t ) = E H t -1 , X ( t ) [reg ′ ( t )] . We can achieve logarithmic regret bounds if the contexts meet Challenges 1 and 2.

We first present our regret analysis for bounded contexts.

Proposition 8 (Regret analysis: bounded contexts) For bounded contexts where ∥ X i ( t ) ∥ 2 ≤ R , suppose that E [ X a ( t ) X a ( t ) | H t -1 ] ⪰ λ ⋆ I and C ∆( X ( t )) ≤ C ∆ for all t ∈ [ T ] and H t -1 . Under Assumption 1, the expected regret of Algorithm 1 is bounded by

Next, we present the regret bound result for unbounded contexts, under the satisfaction of the two challenges.

Proposition 9 (Regret analysis: unbounded contexts) For unbounded contexts where ∥ X i ( t ) ∥ ψ 1 ≤ x max , suppose that E [ X a ( t ) X a ( t ) | H t -1 ] ⪰ λ ⋆ I and C ∆( X ( t )) ≤ C ∆ for all t ∈ [ T ] and H t -1 . Under Assumption 1, the expected regret of Algorithm 1 is bounded by

J.1 Proof of Proposition 8

Firstly, set T 0 := 1 c 1 λ ⋆ R (3 log T +log d ) for the constant c 1 defined in Corollary 9. Before starting the proof, we define the good events that satisfy the sufficient concentration of the estimator ˆ θ t .

Definition 17 From now on, in this section, we define the event E t by

This is the event that the Gram matrix has sufficiently large minimum eigenvalue. The event E t holds with high probability according to Corollary 9.

Corollary 4 For t ≥ T 0 , the following holds:

Proof Using Corollary 9, with probability 1 -1 2 T 2 , E t holds.

Definition 18 (Self-normalized bound of OLS estimator) Next, we define the event F t as

which satisfies the self-normalized concentration of the estimator.

Lemma 18 (Concentration of OLS Estimator) For any t > 4 λ ⋆ ,

holds.

Proof Under E t , we have Σ( t ) ⪰ 1 4 λ ⋆ tI d . Then the concentration of the OLS estimator satisfies

To bound I , we use Lemma 24 from Abbasi-Yadkori et al. [1] and obtain the desired result.

Next, we introduce an important lemma used to obtain the concentration of the minimum eigenvalue of the Gram matrix. We define G t := ⋂ t τ = T 0 E τ ∩ F τ .

Corollary 5 (Good events) For the event G t defined above, for any T 0 ≤ t ≤ T ,

holds.

Then, under the event G t , we have the following corollary by the definition of event E t and F t :

Corollary 6 ( ℓ 2 concentration: bounded contexts) For any t ≥ T 0 , under the event G t , we have for some absolute constant c > 0 .

Proof of the Proposition. We are now ready to prove Proposition 8. First, for time t ≥ T 0 , the history H t -1 is contained in G t -1 with probability 1 -1 T . The expectation of reg ′ ( t ) is calculated with respect to the randomness of the whole history H t -1 and the distribution of contexts X ( t ) . We can observe the following for γ ( d ) := 4 σ √ d log( T + T 2 R 2 ) :

By summing these inequalities until

T , we obtain the desired result.

J.2 Concentration of Sub-exponential Contexts

We now provide regret analysis for unbounded contexts. To do that, we first provide some known facts for sub-exponential vectors. Under Assumption 2, X i ( t ) has ψ 1 norm with x max . Using a result from Wainwright [38], for any v ∈ S d -1 , there exists an absolute constant c 0 > 0 such that

holds for all u > 0 . If we set v = e i = (0 , 0 , . . . , 1 , . . . , 0) , then we get the concentration for each coordinate,

First, we investigate the concentration of the ℓ 2 norm of the contexts.

Lemma 19 (High probability ℓ 2 bound of the contexts) Suppose the contexts X ( t ) satisfy Assumption 2, then

holds with probability at least 1 -δ .

Proof For any v ∈ S d -1 , we know

by the result from Appendix J.2. Then, we get

J.3 Proof of Proposition 9

We now prove Proposition 9. It consists of three steps. First, we investigate the concentration of the Gram matrix for unbounded contexts. Next, we define several high-probability good events. Finally, we bound the regret using the peeling technique.

J.3.1 Gram Matrix Concentration

For bounded contexts, we can apply Lemma 23 and Corollary 9 to ensure the linear growth of the Gram matrix. However, for unbounded contexts, we cannot apply Lemma 23 directly since it requires ℓ 2 boundedness. We apply the same technique used in Kannan et al. [20], who also deal with Gaussian contexts, which are not bounded. They use a truncation technique to guarantee the growth of the Gram matrix: interpret it as the mixture of truncated contexts and large ℓ 2 norm contexts. We apply similar arguments here. In this section, we set our truncation radius L = c √ dx max ( 1+3log ( dKT λ ⋆ )) and define T 1 := 2 L (2 log T +log d ) .

Lemma 20 For any t ≥ T 1 , the following holds with probability 1 -2 T 2 :

Proof We prove it using a similar argument to Kannan et al. [20] to bound the Gram matrix. We view X i ( t ) given the history H t -1 as a mixture of X i ( t ) | B L and X i ( t ) | B c L . Consider an invisible

coin toss c s for every time s ∈ [ t ] , and if c s = 1 , the contexts are drawn in X ( t ) | ( B L ) K ; if c s = 0 , they are drawn from X ( t ) | ( B c L ) K . By our choice of high-probability region radius L , for any c s , P [ c s = 1] ≥ 1 -1 T 2 . Define Σ( t ) as the Gram matrix of the contexts sampled from the truncated distribution for all 1 ≤ s ≤ t . For truncated contexts, Proposition 5 tells us that it has diversity constant 1 2 λ ⋆ under our choice of L . Then, by our independence assumption and Lemma 5,

Hence, if we choose t ≥ T 1 , we get the desired result.

Recall that L = c √ dx max ( 1 + 3 log ( dKT 1 λ ⋆ )) .

J.3.2 Good Events

We present 4 concentrations and define good events that satisfy the concentrations.

Concentration 1. For any t ≥ 1 , with probability 1 -1 T 2 , ∥ X i ( t ) ∥ 2 ≤ L holds for all i ∈ [ K ] . Then, under that event, and

Concentration 2. For any 1 ≤ t ≤ T , with probability 1 -1 T 2 , using Lemma 24, we get

Concentration 3. In the previous section, we showed that for any t ≥ T 1 , holds with probability 1 -2 T 2 .

Combining these three concentrations, we get the following result: with probability 1 -4 T 2 for any t ≥ T 1 ,

The second inequality holds since, under Concentration 3, Σ( t ) ⪰ 1 2 (Σ( t ) + I d ) . We finally obtain the following ℓ 2 -concentration result:

Corollary 7 ( ℓ 2 concentration: unbounded contexts) For any t ≥ T 1 , we have

for some absolute constant c with probability 1 -4 T 2 .

We define event G t as the event in which Corollary 7 holds and set G t = ⋃ τ = T 1 G τ . Finally, we present a key analysis to bound the regret. Recall that we set T 1 := 1 L log d (1 +2 log T ) .

c 1 λ ⋆

Corollary 8 (Good events) For the event G t defined above, for any T 1 ≤ t ≤ T ,

holds.

J.3.3 Bounding Regret by the Peeling Technique

Lemma 21 For t ≥ T 1 , under the good event G t -1 , and if ∥ X i ( t ) ∥ ψ 1 ≤ x max , then

holds for some absolute constant c > 0 .

Proof Under any history contained in G t -1 , by using (14), with probability 1 -1 T 2 we have

for some absolute constant c > 0 . We define this event as K t . Then, the (unexpected) regret is bounded by

We now bound the expected regret:

In the last inequality, we use the definition of Challenge 2 and Lemma 7.

Peeling technique for tail events. Next, we bound E [reg ′ ( t ) | G t -1 ∩ K c t ] using the peeling technique. Let L n = c 0 √ dx max (1 + log dK + n log γ ) for n ≥ 1 , where we will determine γ > 0 later. Then, by results from Appendix J.2,

t

We define the event V n as { max i ∈ [ K ] | X i ( t ) ⊤ θ ⋆ | ∈ [ L n , L n +1 ] } . Then P [ V n ] ≤ 1 γ n and E [reg ′ ( t ); V n ] ≤ 2 L n +1 . Set α = c 0 √ dx max , β = 1 + log dK . The regret can be decomposed as

when γ ≥ ( αβ ) 3 + T 2 .

Now set γ = ( αβ ) 3 + T 2 . Then we have

Main proof of Proposition 9 We apply the result of Lemma 21. For t ≥ T 1 , we get

Next, we examine E [reg ′ ( t ); G c t -1 ] . We also bound this using the peeling technique. We define the same L n and V n as in the previous Lemma 21:

for some absolute constant c > 0 . Thus,

By summing up, we get the desired result. For t ≤ T 1 , using a peeling technique shows E [reg( t )] ≤ cL 1 . Hence

Then we can get wanted result easily.

K Discussion on Discrete-Supported Contexts

In this paper, we considered only context distributions with differentiable densities.

Single-Parameter Linear Contextual Bandits. To the best of our knowledge, no study has addressed the greedy algorithm for linear contextual bandits with discrete-supported stochastic contexts.

Multiple-Parameter Linear Contextual Bandits. For multi-parameter linear contextual bandits, Bastani et al. [8] proved that the Gibbs distribution, which has discrete support, satisfies the margin condition and their diversity condition, achieving logarithmic regret for the greedy algorithm. However, their proof applies only to the two-arm case.

We assert that, for the K -armed multi-parameter linear contextual bandit with K ≥ 3 , Gibbs distribution can fail under the greedy policy. For example, the two-dimensional Gibbs distribution has support points (1 , 1) , (1 , -1) , ( -1 , 1) , ( -1 , -1) . However, if three parameters are given as β ⋆ 1 = (1 , 1) , β ⋆ 2 = (1 , -1) , β ⋆ 3 = ( -1 , 1) , the diversity assumption of Bastani et al. [8] is violated.

We further claim that for multiple-parameter linear contextual bandits, the effectiveness of discretesupported contextual bandits with an arbitrary number of arms K has not yet been thoroughly studied.

L Dicussions, Limitations and Further Ideas

Since our LAC class primarily includes differentiable densities, examining the performance of the greedy bandit with discrete valued contexts would be a valuable future direction. For discrete valued contexts, the only existing result by Bastani et al. [8] establishes performance for a Gibbs distribution in a 2 -arm (shared-context) bandit, but this generally fails when K ≥ 3 2 and also it differs somewhat from our setup. In our setup, the linear contextual bandit, no results are currently known for discrete contexts. Given that Bastani et al. [8] studied a shared contexts setup, our work represents the largest class of distributions for which the greedy bandit shows efficient performance in the linear contextual bandit problem.

To derive the concentration of minimum eigenvalue, the boundedness of contexts (random variables) is required. However, for heavy tail contexts, the upper bound of the contexts' norm can be large, so it leads to poor concentration. Dealing with non-truncated heavy tail contexts can be another interesting problem.

M Numerical Experiments

We conducted numerical experiments to evaluate the performance of the greedy algorithm and compare it with existing bandit algorithms, LinUCB from Abbasi-Yadkori et al. [1] and LinTS from Agrawal and Goyal [4]. We conducted experiments for three cases with varying parameters: d = 20 , K = 20 , T ≤ 1000 , d = 100 , K = 20 , T ≤ 1000 , and d = 20 , K = 100 , T ≤ 1000 , and five different distributions of contexts: Uniform in a ball, truncated Student's t, Laplace, Gaussian, and exponential. The experiments were repeated 10 times for each case, and the deviation was also displayed on the graph.

̸

We note that the results were obtained as √ d times larger than the actual size because of the absence of dimensional correction. For the uniform distribution, we used Unif( B (0 , √ d )) . The Laplace contexts were generated by independently sampling each component of a Laplace distribution with parameters µ = 0 , b = 1 . The Gaussian context was created so that each element of the feature vectors was drawn from a multivariate Gaussian distribution with a covariance matrix V with V i,i = 1 and V i,j = 0 . 7 for any i = j . The truncated Cauchy contexts were generated by independently sampling each component from a truncated Cauchy with loc 0 , scale 1 and truncation range [ -5 , 5] .

In most cases, the greedy algorithm produced the best results. Our theory predicted a polynomial scale dependency on dimension for the regret of the greedy algorithm, and the experimental results

confirmed good performance even with an increased dimension. This discrepancy is due to the fact that we considered the worst case. The experimental results for the three cases are listed. 3

0

200

400

600

800

1000

Round (

t

)

0

50

100

150

200

Cumulative Regret

d

=20,

K

=20, Guassian

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

50

100

150

200

250

300

350

400

Cumulative Regret

d

=20,

K

=20, Uniform

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

50

100

150

200

250

300

350

Cumulative Regret

d

=20,

K

=20, Laplace

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

50

100

150

200

250

300

Cumulative Regret

d

=20,

K

=20, Truncated Cauchy

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

50

100

150

200

250

Cumulative Regret

d

=20,

K

=100, Guassian

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

100

200

300

400

500

Cumulative Regret

d

=20,

K

=100, Uniform

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

100

200

300

400

Cumulative Regret

d

=20,

K

=100, Laplace

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

100

200

300

400

Cumulative Regret

d

=20,

K

=100, Truncated Cauchy

LinUCB

LinTS

LinGreedy

Cumulative Regret

0

200

400

600

800

1000

Round (

t

)

0

100

200

300

400

500

Cumulative Regret

d

=100,

K

=20, Guassian

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

200

400

600

800

Cumulative Regret

d

=100,

K

=20, Uniform

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

200

400

600

800

1000

1200

d

=100,

K

=20, Laplace

LinUCB

LinTS

LinGreedy

0

200

400

600

800

1000

Round (

t

)

0

200

400

600

800

1000

Cumulative Regret

d

=100,

K

=20, Truncated Cauchy

LinUCB

LinTS

LinGreedy

N Technical Lemmas

Lemma 22 (Gronwall Inequality) For g ( y ) ∈ R satisfies g ′ g ( y ) ≥ -M in [ y, y + h ] , then

Proof See classic PDE books like Evans [16].

N.1 Concentration Inequalities

Lemma 23 (Matrix Chernoff : Adapted Sequence from [36]) Consider a finite adapted sequence { X k } with filtration {F t } t ≥ 0 of positive-semi definite matrices with dimension d , and suppose that

Define the finite series

For all µ ≥ 0

Corollary 9 (Eigenvalue Growth of Adaptive Gram Matrix) If ∥ X i ∥ 2 ≤ x max and λ min ( E [ X i X ⊤ i |H t -1 ]) ≥ λ 0 , then with probability 1 -d exp( -c 1 λ 0 t x max )

holds for some absolute constant c 1 .

Proof Put δ = 3 4 , µ = λ 0 t, R = x max at the Lemma 23.

Lemma 24 (Theorem 1 from [1]) Let { F t } ∞ t =0 be a filtration. Let { η t } ∞ t =1 be a real-valued stochastic process such that η t is F t -measurable and η t is conditionally R -sub-Gaussian for some R ≥ 0 i.e.

Let { X t } ∞ t =1 be an R d -valued stochastic process such that X t is F t -1 -measurable. Assume that V is a d × d positive definite matrix. For any t ≥ 0 , define

Then, for any δ > 0 , with probability at least 1 -δ , for all t ≥ 0 ,

Lemma 25 (Lemma 10 from [1]) Suppose X 1 , X 2 , . . . , X t ∈ R d and for any 1 ≤ s ≤ t , ∥ X s ∥ 2 ≤ L . Let V t = λI + ∑ t s =1 X s X ⊤ s for some λ > 0 . Then,

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Yes, we think our abstract and introduction well reflect the whole paper's contributions.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We wrote it in Appendix L.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Yes, we state whole assumptions and provide rigorous proofs.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We fully provide informations about experiments in Appendix M.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide code in supplementary material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Yes, we provide whole details in Appendix M.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Yes, we provide whole details in Appendix M.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Yes, we provide whole details in Appendix M.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes, we checked.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Yes, we include them in Introduction and Appendix L.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Not relevant.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: Not relevant.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Not relevant.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Not relevant.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Not relevant.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyzes a Greedy bandit algorithm, in the context of a linear bandit problem where (1) a regression parameter is unknown and fixed for the experiment and (2) the $K$ arms from which the decision-maker can choose are sampled at the beginning of each step $t$, from a fixed context distribution. Recent works have shown that in similar problems exploration is not necessary, and greedy algorithms are sufficient to achieve satisfying theoretical guarantees. This observation is interesting because (1) some standard methods like UCB can be costly in linear bandits, and (2) in some settings exploration might be controversial, since it might imply not acting optimally with the knowledge at hand with consequences on living subjects. \n\nThe authors first propose a generic condition on the context distributions, called the LAC condition, under which they analyze Greedy. They then describe the two main statistical challenges needed for the analysis, before presenting poly-logarithmic regret guarantees under bounded context distributions. Finally, they showcase experiments that validate their approach, advocating for the use of Greedy to tackle the linear bandit problem that they consider.\n\nSTRENGTHS:\nFirst, for the reasons mentioned in previous paragraph, I believe that Greedy algorithms work in some learning settings and understanding their properties is an interesting research question. It is true that, with the development of powerful tools like optimism or Thompson Sampling, we tend to jump on these methods to tackle every bandit problem without necessarily questioning whether their exploration mechanism is necessary or not. Hence, this line of work is in my opinion interesting to recall that simple methods should be tried first before moving to those more elaborate tools.\n\nThen, the authors put a lot of efforts in describing and providing intuitions on the two statistical challenges posed by their problem, namely the positivity of the eigenvalues of the design matrix (i.e. the implicit exploration of greedy), and the fact that randomly sampling contexts guarantee relatively large sub-optimality gaps at each time step. These two challenges are the key ingredients to derive the poly-logarithmic regret guarantees for the greedy algorithm proposed by the authors. Furthermore, the LAC condition is well-described and rather intuitive.\n\nWEAKNESSES:\nEdit: after reading the rebuttal and discussing with other reviewers I decided to increase my score \n______________________________\n\nIn my opinion, the first weakness is the setting itself. Other works in the literature seem to tackle two settings: the one presented in the paper, and another one which actually seems to be the main setting of the other works, in which the arms are fixed but the regression parameter is sampled at each round. This second setting looks more natural to me: the arms are fixed but their performance vary according to external environmental factors, eventually making each of them optimal in some context. I can see the applications of this setting, but I cannot see the applications of the setting presented in the paper.\n\nThen I believe that, although the main paper is not very technical, the presentation makes it difficult to grasp the main technical insights of the analysis. Many arguments are discussed but not properly sketched, and it is hard to precisely correlate the two « challenges » with the actual regret analysis. I decided to check the proof of the regret bound in appendix, and then things got even worse: the appendix is so badly organized that it requires at least 4 screens to check any single results, due to no clear proof scheme and various technical results spread everywhere. Overall, after considerable efforts I did not succeed in understanding precisely how each result is used in the regret analysis. I would suggest the authors to re-organize the paper by properly writing the regret analysis, and along the analysis pointing out how the main technical results are used, that would themselves be properly proved in dedicated sections. In appendix their should not be proof sketches but rigorous proofs, and discussions about intuitions should be clearly separated from the proofs. Furthermore, the unbounded context case seems to be tackled in an extremely complicated way. I do not get the point of this, it does not even seem important in the main paper, maybe tackling the bounded case only would be sufficient and remove some confusion. Second, I do not get why the authors do not eventually use the proof for bounded context with $x_{max}=O(polylog(T))$, which is eventually true with high probability with light-tailed contexts.\n\nOverall, it seems to me that the paper requires significant re-writing to (1) make the proofs rigorous and easy to check (or even just that an expert non-author reader might be able to check them), (2) clarify the contributions by focusing on precise interesting cases,  (3) more precisely linking the « challenges » with the regret analysis. Furthermore, the setting appears as artificial in the current version of the paper.\n\nQUESTIONS:\n* The LAC condition implies a mode of the context distribution in zero, and some flatness around $0$. I am wondering how instrumental is this property to guarantee the implicit exploration of Greedy and the not-too-small sub-optimality gap. For instance, with a Gaussian in mind, do the guarantees generalize if we assume the existence of a mode and shift the condition to be on the norm of $x-a$ for some vector $a$? The adaptation does not seem direct to me because sampling around $0$ facilitate that diverse directions are sampled, making the gap emerge.\n\n* Could you be provide some justification of the settings, i.e. some applications where it would make sense to assume that $K$ arms are drawn i.i.d from a context distribution at each time step, with independence between steps? \n\n* What is special about unbounded context that make it relevant to keep the distinction in the paper? \n\n* The dependency in $t$ in challenge 2 looks fishy to me: I don’t see why the probability bound should depend on $T$ (or 1): the probability only depends on the context distribution and K, so the bound should only depends on $\\epsilon$. Furthermore, by independence the same bound should hold for all $t$. Am I missing something? In addition, assuming the result is true, how is the $1/\\sqrt{T}$ shaved off in the analysis? Because I would assume that the regret analysis would consist in multiplying the error by $T$ under the case the event does not hold.\n\n* Additional related question: what $\\epsilon$ is used in the analysis?\n\nLIMITATIONS:\nNot applicable", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to expand the range of distributions that can be used efficiently in exploration-free greedy linear contextualized bandits. For this purpose, a new condition called Local-Anti Concentration is introduced. It is claimed that different distributions from the exponential family satisfy this property and they do not require the margin property to achieve $O(poly logT)$ regret.\n\nSTRENGTHS:\nFor the Gaussian, the authors show that LAC condition holds. Hence, they show an improvement in the regret without assuming any margin condition.\n\nWEAKNESSES:\n1. 89, 264: It’s unclear what the entries of this Gram matrix represent.\n\n2. 24-25: It’s not explained why healthcare and clinical domains might find exploration infeasible or unethical.\n\n3. 57: The specifications of margin condition are not explained.\n\n4. Even though there are experiments in the paper, in the guideline the answer to the question 5 is marked as N/A. It's unclear whether the code belongs to the authors or another researcher and hence, the answer. If the code belongs to the authors, the answer should have been YES or NO. \n\n5. Lines 63 to 67 are repeated word for word in 106 to 109, except for the addition of the word \"bandit\". Please delete one of these sections.\n\n6. 120: It is not clear on what D and B are supposed to be. The authors can add the word \"sets\" to be more clear.\n\n7. 526-540: Appendix C.1 is not written extensively; the proof is provided only for the Gaussian distribution.  This is the main contribution of the paper as stated in 91-92 and used in different places such as 311. Without the proof, it’s unclear how well the rest of the statements for the distributions hold.\n\n8. 490: There is a typo as these algorithms are stated as linTS and linUCB instead of LinTS and LinUCB.\n\n9. 114: There is another typo: \"unecessary\" should be \"unnecessary\".\n\n10. 352: Experiment results are not discussed extensively, making it hard to understand how the experiments support the claims.\n\nQUESTIONS:\n- 352: LinTS and LinUCB algorithms are show in the Figures in the Experiments Section. However, it is unclear what these algorithms are. Were they designed by the authors or are they the work of other researchers? In 1444, they are stated as existing bandit algorithms. If so, please provide the appropriate references.\n\n- 246: What do the authors mean by if two challenges are “satisfied”? Do you mean “overcome”?\n\nLIMITATIONS:\n- Technical contribution: The most significant proof, which is the proof that the given distributions satisfy the LAC condition, was left out.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel condition for context distribution, called *Local Anti-Concentration (LAC)*. Under LAC, the authors prove the regret of greedy algorithms for stochastic contextual linear bandits is $\\mathcal{O}(\\mathrm{poly} \\log T)$, without additional margin assumption. The efficacy of the greedy approach for various distributions is validated numerically as well.\n\nSTRENGTHS:\n- Clearly, very well-written\n- Clear-cut contributions that significantly improve greedy algorithms for contextual linear bandits, including a provably larger class of context distributions that can achieve $\\mathcal{O}(\\mathrm{poly} \\log T)$ regret without additional margin condition and other technical contributions.\n- Numerical verification\n\nWEAKNESSES:\n- The authors should provide (possibly not-to-far-fetched) distributions in which the LAC fails.\n- The initial parameter $\\theta_0$ is mentioned briefly but not much discussed. What is the dependency of the regret on $\\theta_0$? How was $\\theta_0$ chosen for the experiments? In practice, should one choose $\\theta_0$ randomly, or is it okay to fix it? I feel this plays an important role, as the initial \"exploration\" (til the diversity becomes positive) heavily depends on $\\theta_0$.\n\n\n**(Minor) Typography suggestions**\n- Some of the sentences in Section 1.2 overlap with those in the paragraph above Section 1.1. I think it would be more appropriate if Section 1.2 is absorbed into the beginning of Section 1, and Section 1.1 becomes a paragraph (\\paragraph{..})\n- The pseudocode in Algorithm 1 seems wrong...? the while loop should be an If-Else.\n- In pg. 6, \"Consider the desnity\" -> \"Consider the density\"\n- If accepted, the authors should include the full regret bound from Appendix B.3 in the main text.\n\nQUESTIONS:\n- Is LAC necessary for the polylog(T) regret of the greedy algorithm, or at least close to it? In other words, if the distribution is not LAC, does greedy always fail? I would be curious to see the numerical performance of greedy algorithms for distributions that are not LAC.\n- Does this work for generalized linear bandits as well? How about generally structured bandits (although this seems quite unlikely) and kernelized linear bandits (this, I have no idea)?\n- There was an interesting paper [1] in which, for linear bandits with a rich (continuous) action set, sublinear regret *implies* a lower bound on the minimum eigenvalue of the design matrix. Of course, in this case, the margin doesn't make sense, and there is a $\\Omega(d\\sqrt{T})$ regret lower bound, but as this paper and [1] are conceptually similar (to my eyes), would the greedy algorithm achieve $O(d\\sqrt{T})$ regret? Even if not, it would also be nice to include some discussions regarding this in the paper.\n- (minor) \n\n\n\n[1] https://proceedings.mlr.press/v206/banerjee23b.html\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of linear contextual bandits with randomly generated contexts from a distribution \n$f$. The goal is to determine under which conditions on $f$ a greedy algorithm (outlined in Algorithm 1) achieves reasonable regret.\n\nBy introducing the notion of Local Anti-Concentration (LAC) in Definition 1, the authors demonstrate that if $f$ satisfies the LAC condition, then the greedy algorithm achieves poly-logarithmic regret. The LAC condition encompasses a wide range of distributions, such as Gaussian, uniform, Laplace, and Student’s t-distribution, making the results of the paper general and applicable across several frameworks.\n\nSTRENGTHS:\nThe paper is well-written and addresses a very interesting problem. It makes a significant contribution by covering a large family of distributions for the contexts and demonstrating that, for this wide range, a greedy algorithm can achieve poly-logarithmic regret. The main results, Theorems 3 and 4, are noteworthy and can be of independent interest.\n\nWEAKNESSES:\nThe only weakness I see is the lack of discussion about the optimality of the achieved regret concerning the parameters of interest, i.e., \n$d$ (dimension of the context), \n$K$ (number of arms), and $\\alpha$ (the parameter associated to the LAC condition). While the authors addressed this by stating, \"Our focus here is not solely on attaining the sharpest regret bounds, although achieving poly-logarithmic regret is highly favorable,\" it would have been beneficial to discuss this matter in more detail.\n\nQUESTIONS:\n1. I would like to know the authors' thoughts on the necessity of the LAC condition. While the paper's focus is on showing that the LAC condition is sufficient, I am curious if there are significant distributions that do not satisfy the LAC condition but for which the greedy approach still achieves poly-logarithmic regret.\n\n2. In line 269, the authors mention, \"Eq.(3) is a relaxed version of the margin condition\" and \"The aforementioned existing literature explicitly assumes the condition to hold.\" Does this mean that the margin condition imposed in previous works can be dropped and relaxed to provide a bound on Eq.(3)?\n\n3. I would appreciate it if the authors addressed my question in the weaknesses section.\n\nLIMITATIONS:\nThe authors adequately address the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}