Original: Instance-optimal PAC Algorithms for Contextual Bandits
Revised: Instance-optimal PAC Algorithms for Contextual Bandits

Original: Abstract
Revised: Abstract

Original: In the stochastic contextual bandit setting, regret-minimizing algorithms have been extensively researched, but their instance-minimizing best-arm identiﬁcation counterparts remain seldom studied.
Revised: In the stochastic contextual bandit setting, regret-minimizing algorithms have been extensively researched, but their instance-minimizing best-arm identiﬁcation counterparts remain seldom studied.

Original: In this work, we focus on the stochastic bandit problem in the ( ✏ ,  ) - PAC setting: given a policy class ⇧ the goal of the learner is to return a policy ⇡ 2 ⇧ whose expected reward is within ✏ of the optimal policy with probability greater than 1   .
Revised: In this work, we focus on the stochastic bandit problem in the ( ✏ ,  ) - PAC setting: given a policy class ⇧ the goal of the learner is to return a policy ⇡ 2 ⇧ whose expected reward is within ✏ of the optimal policy with probability greater than 1   .

Original: We characterize the ﬁrst instancedependent PAC sample complexity of contextual bandits through a quantity ⇢ ⇧ , and provide matching upper and lower bounds in terms of ⇢ ⇧ for the agnostic and linear contextual best-arm identiﬁcation settings.
Revised: We characterize the ﬁrst instancedependent PAC sample complexity of contextual bandits through a quantity ⇢ ⇧ , and provide matching upper and lower bounds in terms of ⇢ ⇧ for the agnostic and linear contextual best-arm identiﬁcation settings.

Original: We show that no algorithm can be simultaneously minimax-optimal for regret minimization and instance-dependent
Revised: We show that no algorithm can be simultaneously minimax-optimal for regret minimization and instance-dependent

Original: PAC for best-arm identiﬁcation.
Revised: PAC for best-arm identiﬁcation.

Original: Our main result is a new instance-optimal and computationally efﬁcient algorithm that relies on a polynomial number of calls to an argmax oracle.
Revised: Our main result is a new instance-optimal and computationally efﬁcient algorithm that relies on a polynomial number of calls to an argmax oracle.

Original:  Introduction
Revised:  Introduction

Original: We consider the stochastic contextual bandit problem in the PAC setting.
Revised: We consider the stochastic contextual bandit problem in the PAC setting.

Original: Fix a distribution ⌫ over a potentially countable 1 set of contexts C .
Revised: Fix a distribution ⌫ over a potentially countable 1 set of contexts C .

Original: The action space is A , and for computational tractability, we assume |A| is ﬁnite.
Revised: The action space is A , and for computational tractability, we assume |A| is ﬁnite.

Original: We have a set of policies ⇧ of interest where each policy ⇡ 2 ⇧ is a map from contexts to an action space ⇡ :
Revised: We have a set of policies ⇧ of interest where each policy ⇡ 2 ⇧ is a map from contexts to an action space ⇡ :

Original: C !
Revised: C !

Original: A .
Revised: A .

Original: The reward function is r : C ⇥ A !
Revised: The reward function is r : C ⇥ A !

Original: R .
Revised: R .

Original: At each time t = 1 , 2 , . . .
Revised: At each time t = 1 , 2 , . . .

Original: a context c t ⇠ ⌫ arrives, the learner chooses an action a t 2 A , and receives reward r t :=
Revised: a context c t ⇠ ⌫ arrives, the learner chooses an action a t 2 A , and receives reward r t :=

Original: r t ( c t , a t ) 2 R with E [ r t | c t , a t ] = r ( c t , a t ) 2 R .
Revised: r t ( c t , a t ) 2 R with E [ r t | c t , a t ] = r ( c t , a t ) 2 R .

Original: The value of a policy V ( ⇡ ) is the expected reward from playing action ⇡ ( c ) in context c : V ( ⇡ ) =
Revised: The value of a policy V ( ⇡ ) is the expected reward from playing action ⇡ ( c ) in context c : V ( ⇡ ) =

Original: E c ⇠ ⌫ [ r ( c, ⇡ ( c ))] .
Revised: E c ⇠ ⌫ [ r ( c, ⇡ ( c ))] .

Original: Given a collection of policies ⇧ , the objective is to identify the optimal policy ⇡ ⇤ := arg max ⇡ 2 ⇧ V ( ⇡ ) , with high probability.
Revised: Given a collection of policies ⇧ , the objective is to identify the optimal policy ⇡ ⇤ := arg max ⇡ 2 ⇧ V ( ⇡ ) , with high probability.

Original: Formally, for any ✏ > 0 and  2 (0 , 1) , we seek to characterize the sample complexity of identifying a policy ⇡ 2 ⇧ such that V ( ⇡ )  V ( ⇡ ⇤ )
Revised: Formally, for any ✏ > 0 and  2 (0 , 1) , we seek to characterize the sample complexity of identifying a policy ⇡ 2 ⇧ such that V ( ⇡ )  V ( ⇡ ⇤ )

Original:  ✏ , with probability at least 1   .
Revised:  ✏ , with probability at least 1   .

Original: That is, we wish to minimize the total amount of interactions with the environment to learn an ✏ -optimal policy.
Revised: That is, we wish to minimize the total amount of interactions with the environment to learn an ✏ -optimal policy.

Original: We study both the agnostic setting, where ⇧ is an arbitrary set of policies with no assumed relationship with the reward function r ( c, a ) ; and the realizable setting, where the policy class and the reward function follow a linear structure, known as the linear contextual bandit problem.
Revised: We study both the agnostic setting, where ⇧ is an arbitrary set of policies with no assumed relationship with the reward function r ( c, a ) ; and the realizable setting, where the policy class and the reward function follow a linear structure, known as the linear contextual bandit problem.

Original: In both cases, we are interested in instance-dependent sample complexity bounds.
Revised: In both cases, we are interested in instance-dependent sample complexity bounds.

Original: That is, the upper and lower bounds we seek do not simply depend on coarse quantities like | ⇧ | , |A| , and 1 / ✏ 2 , but more ﬁne- grained relationships between the context distribution ⌫ , geometry of policies ⇧ , and the reward function r : C ⇥ A !
Revised: That is, the upper and lower bounds we seek do not simply depend on coarse quantities like | ⇧ | , |A| , and 1 / ✏ 2 , but more ﬁne- grained relationships between the context distribution ⌫ , geometry of policies ⇧ , and the reward function r : C ⇥ A !

Original: R .
Revised: R .

Original: Our motivation is that instance-dependent bounds describe the difﬁculty of a particular problem instance, allowing optimal algorithms to adapt to the true difﬁculty of the problem, whether easy or hard.
Revised: Our motivation is that instance-dependent bounds describe the difﬁculty of a particular problem instance, allowing optimal algorithms to adapt to the true difﬁculty of the problem, whether easy or hard.

Original: We seek algorithms that take advantage of “easy” instances instead of optimizing for the worst-case [23].
Revised: We seek algorithms that take advantage of “easy” instances instead of optimizing for the worst-case [23].

Original: 1.1 Related work
Revised: 1.1 Related work

Original: Minimax regret bounds for general policy classes The vast majority of research in contextual bandits focuses on regret minimization.
Revised: Minimax regret bounds for general policy classes The vast majority of research in contextual bandits focuses on regret minimization.

Original: That is, for a time horizon T , the goal of the player is to minimize E
Revised: That is, for a time horizon T , the goal of the player is to minimize E

Original: [Equation]
Revised: [Equation]

Original: i .
Revised: i .

Original: The landmark algorithm EXP4 for non-stochastic multi-armed bandits [5] achieves a regret bound of p |A| T log( | ⇧ | ) .
Revised: The landmark algorithm EXP4 for non-stochastic multi-armed bandits [5] achieves a regret bound of p |A| T log( | ⇧ | ) .

Original: Unfortunately, the running time of EXP4 is linear in | ⇧ | which is prohibitive for many problems of interest.
Revised: Unfortunately, the running time of EXP4 is linear in | ⇧ | which is prohibitive for many problems of interest.

Original: The algorithms proposed in [3, 11] achieve the same regret bound with a computational complexity that is only polynomial in T and log( | ⇧ | ) .
Revised: The algorithms proposed in [3, 11] achieve the same regret bound with a computational complexity that is only polynomial in T and log( | ⇧ | ) .

Original: Both approaches can be used to obtain an ✏ -optimal policy with probability at least 1   using a sample complexity no more than |A| log( | ⇧ | /  ) ✏.
Revised: Both approaches can be used to obtain an ✏ -optimal policy with probability at least 1   using a sample complexity no more than |A| log( | ⇧ | /  ) ✏.

Original: None of these works made any assumption on the connection between the reward function r and the policy class ⇧ (i.e. the agnostic setting).
Revised: None of these works made any assumption on the connection between the reward function r and the policy class ⇧ (i.e. the agnostic setting).

Original: Instance-dependent regret bounds for general policy classes
Revised: Instance-dependent regret bounds for general policy classes

Original: The epoch-greedy algorithm of [26] achieved the ﬁrst instance-dependent bounds on regret with a coarse guarantee depending only on the minimum policy gap  pol :
Revised: The epoch-greedy algorithm of [26] achieved the ﬁrst instance-dependent bounds on regret with a coarse guarantee depending only on the minimum policy gap  pol :

Original: = V ( ⇡ ⇤ )
Revised: = V ( ⇡ ⇤ )

Original:  max ⇡ 6 = ⇡ ⇤ V ( ⇡ ) .
Revised:  max ⇡ 6 = ⇡ ⇤ V ( ⇡ ) .

Original: In the pursuit of more ﬁne-grained regret bounds achievable by computationally efﬁcient algorithms, many authors resort to the realizability assumption [14–16, 34].
Revised: In the pursuit of more ﬁne-grained regret bounds achievable by computationally efﬁcient algorithms, many authors resort to the realizability assumption [14–16, 34].

Original: The learner knows a hypothesis class H where each f 2 H is a map f : C ⇥ A !
Revised: The learner knows a hypothesis class H where each f 2 H is a map f : C ⇥ A !

Original: R , and there exists an f ⇤ 2 H such that r ( c, a ) =
Revised: R , and there exists an f ⇤ 2 H such that r ( c, a ) =

Original: f ⇤ ( c, a ) for all ( c, a ) 2 C ⇥ A .
Revised: f ⇤ ( c, a ) for all ( c, a ) 2 C ⇥ A .

Original: Under this assumption, [16] proves lower and upper bounds on the instance-dependent regret.
Revised: Under this assumption, [16] proves lower and upper bounds on the instance-dependent regret.

Original: Their bounds are in term of the uniform gap  uniform :=
Revised: Their bounds are in term of the uniform gap  uniform :=

Original: min c
Revised: min c

Original: [Equation]
Revised: [Equation]

Original: In general, for any policy class, they establish matching minimax lower and upper regret bounds of the form min {
Revised: In general, for any policy class, they establish matching minimax lower and upper regret bounds of the form min {

Original: [Equation]
Revised: [Equation]

Original: C
Revised: C

Original: [Equation]
Revised: [Equation]

Original: H } , where C pol H is the policy disagreement coefﬁcient , a parameter depending on the geometry of H and the context distribution ⌫ .
Revised: H } , where C pol H is the policy disagreement coefﬁcient , a parameter depending on the geometry of H and the context distribution ⌫ .

Original: That is, these bounds hold with respect to a worst-case family of instances parameterized by  uniform and C pol H .
Revised: That is, these bounds hold with respect to a worst-case family of instances parameterized by  uniform and C pol H .

Original: Using the standard online-to-batch conversion, this translates to a sample complexity (i.e. the time required to ﬁnd an ✏ -good policy with constant probability) of roughly |A| log( |H| ) ✏  uniform C pol H .
Revised: Using the standard online-to-batch conversion, this translates to a sample complexity (i.e. the time required to ﬁnd an ✏ -good policy with constant probability) of roughly |A| log( |H| ) ✏  uniform C pol H .

Original: We show in Corollary 2.
Revised: We show in Corollary 2.

Original: that this sample complexity is at least as large as our bounds.
Revised: that this sample complexity is at least as large as our bounds.

Original: Further, unlike our bounds below, this sample complexity is unbounded as ✏ goes to 0 .
Revised: Further, unlike our bounds below, this sample complexity is unbounded as ✏ goes to 0 .

Original: Recent work reﬁnes these kinds of regret bounds further, and provides minimax regret bounds in terms of the decision-estimation coefﬁcient [17].
Revised: Recent work reﬁnes these kinds of regret bounds further, and provides minimax regret bounds in terms of the decision-estimation coefﬁcient [17].

Original: Regret bounds for linear contextual bandits A special case of the realizable case assumes a linear structure for H .
Revised: Regret bounds for linear contextual bandits A special case of the realizable case assumes a linear structure for H .

Original: Assume there exists a known feature map  : C ⇥ A !
Revised: Assume there exists a known feature map  : C ⇥ A !

Original: R d and an unknown ✓ ⇤ 2 R d such that the true reward function is given as r ( c, a ) =
Revised: R d and an unknown ✓ ⇤ 2 R d such that the true reward function is given as r ( c, a ) =

Original: h  ( c, a ) , ✓ ⇤ i .
Revised: h  ( c, a ) , ✓ ⇤ i .

Original: For this setting, popular optimism-based algorithms like LinUCB [27] and Thompson sampling [31, 33] achieve a regret bound of min { d p T, d 2  uniform } [1].
Revised: For this setting, popular optimism-based algorithms like LinUCB [27] and Thompson sampling [31, 33] achieve a regret bound of min { d p T, d 2  uniform } [1].

Original: Appealing to the online-to-batch conversion, this translates to a PAC guarantee of d 2 ✏  uniform .
Revised: Appealing to the online-to-batch conversion, this translates to a PAC guarantee of d 2 ✏  uniform .

Original: More precise instance-dependent upper bounds on regret match instance-dependent lower bounds asymptotically as T ! 1 [19, 36].
Revised: More precise instance-dependent upper bounds on regret match instance-dependent lower bounds asymptotically as T ! 1 [19, 36].

Original: These works are most similar to our setting and have qualitatively similar style algorithms.
Revised: These works are most similar to our setting and have qualitatively similar style algorithms.

Original: However, both approaches rely on asymptotics with large problem-dependent terms that may dominate the bounds in ﬁnite time.
Revised: However, both approaches rely on asymptotics with large problem-dependent terms that may dominate the bounds in ﬁnite time.

Original: Our work is focused on upper bounds that nearly match lower bounds for all ﬁnite times.
Revised: Our work is focused on upper bounds that nearly match lower bounds for all ﬁnite times.

Original: Recently, instance-dependent sample complexity results for reinforcement learning in the tabular and linear function approximation settings have appeared [4, 39, 40].
Revised: Recently, instance-dependent sample complexity results for reinforcement learning in the tabular and linear function approximation settings have appeared [4, 39, 40].

Original: As contextual bandits is a special case of ﬁnite-horizon reinforcement learning with a horizon length of 1 , their results immediately can be applied here.
Revised: As contextual bandits is a special case of ﬁnite-horizon reinforcement learning with a horizon length of 1 , their results immediately can be applied here.

Original: However, the cost of this generality is that these algorithms have very large lower order terms (i.e., problem-dependent factors that multiply a 1 / ✏ term) making them far from optimal in our setting.
Revised: However, the cost of this generality is that these algorithms have very large lower order terms (i.e., problem-dependent factors that multiply a 1 / ✏ term) making them far from optimal in our setting.

Original: Moreover, the leading order term of [39] cannot be related to our lower bounds.
Revised: Moreover, the leading order term of [39] cannot be related to our lower bounds.

Original: PAC sample complexity for contextual bandits As we will describe, all contextual bandits with an arbitrary policy class can be reduced to PAC learning for linear bandits.
Revised: PAC sample complexity for contextual bandits As we will describe, all contextual bandits with an arbitrary policy class can be reduced to PAC learning for linear bandits.

Original: Once we made this reduction, our sample complexity analysis draws inspiration from the nearly instance-optimal algorithm for linear best-arm identiﬁcation [13].
Revised: Once we made this reduction, our sample complexity analysis draws inspiration from the nearly instance-optimal algorithm for linear best-arm identiﬁcation [13].

Original: The work in [10] provides a simple regret bound assuming a kernel structure on the reward function, while their bound is minimax and they assume a lower bound on eigenvalues of the covariance matrix of the context distribution.
Revised: The work in [10] provides a simple regret bound assuming a kernel structure on the reward function, while their bound is minimax and they assume a lower bound on eigenvalues of the covariance matrix of the context distribution.

Original: PAC sample complexity of linear contextual bandits was also studied in [41], who shows a minimax guarantee sample complexity that scales with d 2 ✏ 2 log(1 /  ) .
Revised: PAC sample complexity of linear contextual bandits was also studied in [41], who shows a minimax guarantee sample complexity that scales with d 2 ✏ 2 log(1 /  ) .

Original: Similar to our work, [3] deﬁne their action sampling distribution as a convex combination over policies.
Revised: Similar to our work, [3] deﬁne their action sampling distribution as a convex combination over policies.

Original: Our sampling distribution, as well as the optimal sampling distribution, cannot be represented this way and is actually derived from the dual of the optimal experimental design objective.
Revised: Our sampling distribution, as well as the optimal sampling distribution, cannot be represented this way and is actually derived from the dual of the optimal experimental design objective.

Original: Contributions.
Revised: Contributions.

Original: In this work, our contributions include:
Revised: In this work, our contributions include:

Original: In the agnostic setting, we introduce a quantity ⇢ ⇧ that characterizes the instance-dependent sample complexity of PAC learning for contextual bandits (see Equation 1).
Revised: In the agnostic setting, we introduce a quantity ⇢ ⇧ that characterizes the instance-dependent sample complexity of PAC learning for contextual bandits (see Equation 1).

Original: We show
Revised: We show

Original: that
Revised: that

Original: ⇢ ⇧ appears
Revised: ⇢ ⇧ appears

Original: in
Revised: in

Original: information
Revised: information

Original: theoretic lower bound
Revised: theoretic lower bound

Original: on the sample
Revised: on the sample

Original: complexity
Revised: complexity

Original: of
Revised: of

Original: any PAC algorithm as ✏ !
Revised: any PAC algorithm as ✏ !

Original: in Theorem 2.2.
Revised: in Theorem 2.2.

Original: To
Revised: To

Original: ground this,
Revised: ground this,

Original: we
Revised: we

Original: describe
Revised: describe

Original: it carefully in
Revised: it carefully in

Original: the setting of the trivial policy class (
Revised: the setting of the trivial policy class (

Original: Section
Revised: Section

Original: and linear policy
Revised: and linear policy

Original: classes (
Revised: classes (

Original: Section
Revised: Section

Original: We construct an instance on which any regret minimax-optimal algorithm necessarily has a sample complexity that scales quadratically with the optimal sample complexity (Theorem 2.6).
Revised: We construct an instance on which any regret minimax-optimal algorithm necessarily has a sample complexity that scales quadratically with the optimal sample complexity (Theorem 2.6).

Original: This shows that no algorithm can be both regret minimax-optimal and instance-optimal PAC.
Revised: This shows that no algorithm can be both regret minimax-optimal and instance-optimal PAC.

Original: Finally, we propose Algorithm 3 whose sample complexity nearly matches the lower bound based on ⇢ ⇧
Revised: Finally, we propose Algorithm 3 whose sample complexity nearly matches the lower bound based on ⇢ ⇧

Original: By appealing to
Revised: By appealing to

Original: an
Revised: an

Original: argmax
Revised: argmax

Original: oracle this
Revised: oracle this

Original: algorithm has a runtime polynomial in ⇢ ⇧ , 1 / ✏ ,
Revised: algorithm has a runtime polynomial in ⇢ ⇧ , 1 / ✏ ,

Original: log(1 /  ) , |A| , and log( | ⇧ | ) , assuming a unit cost of invoking the oracle.
Revised: log(1 /  ) , |A| , and log( | ⇧ | ) , assuming a unit cost of invoking the oracle.

Original:  Problem statement and main results
Revised:  Problem statement and main results

Original: More formally, deﬁne F t =
Revised: More formally, deﬁne F t =

Original:  ( c 1 , a 1 , r 1 , . . .
Revised:  ( c 1 , a 1 , r 1 , . . .

Original: , c t , a t , r t ) as the natural  -algebra ﬁltration capturing all observed random variables up to time t .
Revised: , c t , a t , r t ) as the natural  -algebra ﬁltration capturing all observed random variables up to time t .

Original: For simplicity, we assume Gaussian noise in some of our analysis.
Revised: For simplicity, we assume Gaussian noise in some of our analysis.

Original: At each time t an algorithm deﬁnes a sampling rule F t 7!
Revised: At each time t an algorithm deﬁnes a sampling rule F t 7!

Original: A which deﬁnes a t +1 , an {F t } t  1 -adapted stopping time ⌧ 2 N , and a selection rule F t 7!
Revised: A which deﬁnes a t +1 , an {F t } t  1 -adapted stopping time ⌧ 2 N , and a selection rule F t 7!

Original: ⇧ that is only called once at the stopping time t = ⌧ .
Revised: ⇧ that is only called once at the stopping time t = ⌧ .

Original: Deﬁnition 2.1.
Revised: Deﬁnition 2.1.

Original: Fix ✏  0 and  2 (0 , 1) .
Revised: Fix ✏  0 and  2 (0 , 1) .

Original: We say an algorithm is ( ✏ ,  ) -PAC for contextual bandits with policy class ⇧ , if for every instance, at the stopping time ⌧ 2 N with ⌧ < 1 almost surely, the algorithm outputs
Revised: We say an algorithm is ( ✏ ,  ) -PAC for contextual bandits with policy class ⇧ , if for every instance, at the stopping time ⌧ 2 N with ⌧ < 1 almost surely, the algorithm outputs

Original: [Equation]
Revised: [Equation]

Original: The sample complexity of an ( ✏ ,  ) -PAC algorithm for contextual bandits is the time at which the algorithm stops and outputs b ⇡ .
Revised: The sample complexity of an ( ✏ ,  ) -PAC algorithm for contextual bandits is the time at which the algorithm stops and outputs b ⇡ .

Original: As we will discuss, the following quantity governs the sample complexity :
Revised: As we will discuss, the following quantity governs the sample complexity :

Original: [Equation]
Revised: [Equation]

Original: Here, for any countable set X we have that 4 X
Revised: Here, for any countable set X we have that 4 X

Original: = { p 2 R |X| :
Revised: = { p 2 R |X| :

Original: P x
Revised: P x

Original: [Equation]
Revised: [Equation]

Original: X p x = 1 , p x  0 8 x 2 X} so that p c for every c 2 C deﬁnes a probability distribution over actions A .
Revised: X p x = 1 , p x  0 8 x 2 X} so that p c for every c 2 C deﬁnes a probability distribution over actions A .

Original: In addition we use the notation a
Revised: In addition we use the notation a

Original: [Equation]
Revised: [Equation]

Original: We begin with a necessary condition on the sample complexity for the particular case of exact policy identiﬁcation ( ✏ = 0 ).
Revised: We begin with a necessary condition on the sample complexity for the particular case of exact policy identiﬁcation ( ✏ = 0 ).

Original: Theorem 2.2 (Lower bound) .
Revised: Theorem 2.2 (Lower bound) .

Original: Fix ✏ = 

Revised: Fix ✏ =

Original: and  2 (0 , 1) .
Revised: and  2 (0 , 1) .

Original: Moreover, ﬁx a contextual bandit instance µ = ( ⌫ , r ) and a collection of policies ⇧ .
Revised: Moreover, ﬁx a contextual bandit instance µ = ( ⌫ , r ) and a collection of policies ⇧ .

Original: Then any (0 ,  ) -PAC algorithm for contextual bandits satisﬁes E µ [ ⌧ ]  ⇢ ⇧ , 0 log(1 / 2 .
Revised: Then any (0 ,  ) -PAC algorithm for contextual bandits satisﬁes E µ [ ⌧ ]  ⇢ ⇧ , 0 log(1 / 2 .

Original: 4  ) .
Revised: 4  ) .

Original: The proof of the lower bound follows from standard information theoretic arguments [24].
Revised: The proof of the lower bound follows from standard information theoretic arguments [24].

Original: The lower bound implicitly applies to learners that know the distribution ⌫ precisely.
Revised: The lower bound implicitly applies to learners that know the distribution ⌫ precisely.

Original: In practice, such knowledge would never be available however the learner may have a large dataset of ofﬂine data.
Revised: In practice, such knowledge would never be available however the learner may have a large dataset of ofﬂine data.

Original: Assumption 1.
Revised: Assumption 1.

Original: Prior to starting the game, the learning algorithm is given a large dataset of contexts D = { c t } Tt =1 , where each c t is drawn IID from ⌫ for all t 2 [ T ] , and T =
Revised: Prior to starting the game, the learning algorithm is given a large dataset of contexts D = { c t } Tt =1 , where each c t is drawn IID from ⌫ for all t 2 [ T ] , and T =

Original: O ( poly (1 / ✏ , |A| , log(1 /  ) , log( | ⇧ | ))) .
Revised: O ( poly (1 / ✏ , |A| , log(1 /  ) , log( | ⇧ | ))) .

Original: The above only assumes access to samples from the context distribution, not rewards or the value function.
Revised: The above only assumes access to samples from the context distribution, not rewards or the value function.

Original: Importantly, since C could be uncountable, we do not assume D covers the support of ⌫ .
Revised: Importantly, since C could be uncountable, we do not assume D covers the support of ⌫ .

Original: Assumption 1 is satisﬁed, for example, in an e-commerce setting where the context is the demographic information about visitors to the site for which massive troves of historical data may be available.
Revised: Assumption 1 is satisﬁed, for example, in an e-commerce setting where the context is the demographic information about visitors to the site for which massive troves of historical data may be available.

Original: Other works in PAC learning have made similar assumptions [20].
Revised: Other works in PAC learning have made similar assumptions [20].

Original: We would like our algorithm to be computationally efﬁcient in the sense that it makes a polynomial number of calls to what we refer to as argmax oracle.
Revised: We would like our algorithm to be computationally efﬁcient in the sense that it makes a polynomial number of calls to what we refer to as argmax oracle.

Original: Such an assumption is common in the contextual bandits literature [3, 11, 25].
Revised: Such an assumption is common in the contextual bandits literature [3, 11, 25].

Original: Deﬁnition 2.3 (Argmax oracle ( AMO )) .
Revised: Deﬁnition 2.3 (Argmax oracle ( AMO )) .

Original: The oracle AMO ( ⇧ , { ( c t , s t ) } nt =1 ) is an algorithm that given contexts and cost vectors ( c 1 , s 1 ) , · · · , ( c n , s n ) 2 C ⇥ R |A| , returns arg max
Revised: The oracle AMO ( ⇧ , { ( c t , s t ) } nt =1 ) is an algorithm that given contexts and cost vectors ( c 1 , s 1 ) , · · · , ( c n , s n ) 2 C ⇥ R |A| , returns arg max

Original: [Equation]
Revised: [Equation]

Original: The constrained argmax oracle C-AMO , given an upper bound l on the loss, returns arg max ⇡ 2 ⇧
Revised: The constrained argmax oracle C-AMO , given an upper bound l on the loss, returns arg max ⇡ 2 ⇧

Original: P nt
Revised: P nt

Original: [Equation]
Revised: [Equation]

Original: ) subject to
Revised: ) subject to

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: In general we can implement AMO by calling to cost-sensitive classiﬁcation [6, 11] and C-AMO through a Lagrangian relaxation and a cost-sensitive classiﬁcation oracle [2, 8].
Revised: In general we can implement AMO by calling to cost-sensitive classiﬁcation [6, 11] and C-AMO through a Lagrangian relaxation and a cost-sensitive classiﬁcation oracle [2, 8].

Original: Our algorithm uses an argmax oracle as a subroutine at most polynomially in ✏  1 , log(1 /  ) , |A| and log( | ⇧ | ) .
Revised: Our algorithm uses an argmax oracle as a subroutine at most polynomially in ✏  1 , log(1 /  ) , |A| and log( | ⇧ | ) .

Original: In this sense, it is computationally efﬁcient.
Revised: In this sense, it is computationally efﬁcient.

Original: The following sufﬁciency result holds for general ✏  0 .
Revised: The following sufﬁciency result holds for general ✏  0 .

Original: Theorem 2.4 (Upper bound) .
Revised: Theorem 2.4 (Upper bound) .

Original: Fix ✏  0 and  2 (0 , 1) .
Revised: Fix ✏  0 and  2 (0 , 1) .

Original: Under Assumption 1, there exists a computationally efﬁcient ( ✏ ,  ) -PAC algorithm for contextual bandits that satisﬁes ⌧  ⇢ ⇧ , ✏ log( | ⇧ | log 2 (1 / ✏ ) /  ) log(1 /  ✏ ) , where  ✏ = max { ✏ , min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )
Revised: Under Assumption 1, there exists a computationally efﬁcient ( ✏ ,  ) -PAC algorithm for contextual bandits that satisﬁes ⌧  ⇢ ⇧ , ✏ log( | ⇧ | log 2 (1 / ✏ ) /  ) log(1 /  ✏ ) , where  ✏ = max { ✏ , min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )

Original:  V ( ⇡ ) } .
Revised:  V ( ⇡ ) } .

Original: Furthermore, this sample complexity never exceeds |A| (log( | ⇧ | )+log(1 /  ))log(1 / ✏ ) ✏.
Revised: Furthermore, this sample complexity never exceeds |A| (log( | ⇧ | )+log(1 /  ))log(1 / ✏ ) ✏.

Original: The second part of the theorem follows from the ﬁrst, since ⇢ ⇧ , ✏  2 |A| / ✏ 2 by taking p c,a = 1 / |A| for all ( c, a ) 2 C ⇥ A .
Revised: The second part of the theorem follows from the ﬁrst, since ⇢ ⇧ , ✏  2 |A| / ✏ 2 by taking p c,a = 1 / |A| for all ( c, a ) 2 C ⇥ A .

Original: 2.1 Inefﬁciency of low-regret algorithms
Revised: 2.1 Inefﬁciency of low-regret algorithms

Original: Computationally efﬁcient algorithms are known to exist, such as ILOVETOCONBANDITS [3], which achieve a minimax-optimal cumulative regret of p
Revised: Computationally efﬁcient algorithms are known to exist, such as ILOVETOCONBANDITS [3], which achieve a minimax-optimal cumulative regret of p

Original: T |A| log( | ⇧ | /  ) .
Revised: T |A| log( | ⇧ | /  ) .

Original: Inspecting the proof in [3], one can extract a sample complexity of ✏  2 |A| log( | ⇧ | /  ) from such results (which is also minimax optimal for PAC).
Revised: Inspecting the proof in [3], one can extract a sample complexity of ✏  2 |A| log( | ⇧ | /  ) from such results (which is also minimax optimal for PAC).

Original: The previous section showed that the sample complexity of our algorithm,
Revised: The previous section showed that the sample complexity of our algorithm,

Original: Theorem 2.4, nearly matches the instance-dependent lower bound of Theorem 2.2.
Revised: Theorem 2.4, nearly matches the instance-dependent lower bound of Theorem 2.2.

Original: In other words, our algorithm achieves a nearly optimal instance-dependent PAC sample complexity.
Revised: In other words, our algorithm achieves a nearly optimal instance-dependent PAC sample complexity.

Original: However, it is natural to wonder if perhaps with a tighter analysis, the minimax regret optimal algorithm in [3] also obtains the instance-optimal PAC sample complexity.
Revised: However, it is natural to wonder if perhaps with a tighter analysis, the minimax regret optimal algorithm in [3] also obtains the instance-optimal PAC sample complexity.

Original: In this section, we show that this is not the case.
Revised: In this section, we show that this is not the case.

Original: Indeed, we show that any algorithm that is minimax regret optimal must have a sample complexity that is at least quadratic in the optimal PAC sample complexity of some instance.
Revised: Indeed, we show that any algorithm that is minimax regret optimal must have a sample complexity that is at least quadratic in the optimal PAC sample complexity of some instance.

Original: Deﬁnition 2.5 (Hard instance) .
Revised: Deﬁnition 2.5 (Hard instance) .

Original: Fix m 2 N ,  2 (0 , 1] and let C =
Revised: Fix m 2 N ,  2 (0 , 1] and let C =

Original: [ m ] with uniform distribution,
Revised: [ m ] with uniform distribution,

Original: A = { 0 , 1 } .
Revised: A = { 0 , 1 } .

Original: For i = 1 , . . .
Revised: For i = 1 , . . .

Original: , m , let ⇡ i ( j )
Revised: , m , let ⇡ i ( j )

Original: = 1 { i = j } and deﬁne r ( i, j ) =
Revised: = 1 { i = j } and deﬁne r ( i, j ) =

Original:  1 { j = ⇡ 1 ( i ) } .
Revised:  1 { j = ⇡ 1 ( i ) } .

Original: Then V ( ⇡ 1 ) =  and V ( ⇡ i )
Revised: Then V ( ⇡ 1 ) =  and V ( ⇡ i )

Original: =
Revised: =

Original:  (1  2 /m ) for all i 2 C \ { 1 } .
Revised:  (1  2 /m ) for all i 2 C \ { 1 } .

Original: Note that for the hard instance, m = | ⇧ | .
Revised: Note that for the hard instance, m = | ⇧ | .

Original: If observations are corrupted by N (0 , 1) additive noise, then a straightforward calculation shows that ⇢ ⇧ , 0 = 4 /m (2  /m ) 

Revised: If observations are corrupted by N (0 , 1) additive noise, then a straightforward calculation shows that ⇢ ⇧ , 0 = 4 /m (2  /m )

Original: = m   2 for the hard instance.
Revised: = m   2 for the hard instance.

Original: Theorem 2.6.
Revised: Theorem 2.6.

Original: Fix  2 (0 , 1) and  2 (0 , 1] .
Revised: Fix  2 (0 , 1) and  2 (0 , 1] .

Original: We say an algorithm is an ↵ -minimax regret algorithm if for some ↵ > 0 and all T 2 N :
Revised: We say an algorithm is an ↵ -minimax regret algorithm if for some ↵ > 0 and all T 2 N :

Original: [Equation]
Revised: [Equation]

Original: p ↵
Revised: p ↵

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: [Equation]
Revised: [Equation]

Original: T where the maximum is taken over all contextual bandit instances µ 0 = ( ⌫ 0 , r 0 ) and T c,a ( T ) =
Revised: T where the maximum is taken over all contextual bandit instances µ 0 = ( ⌫ 0 , r 0 ) and T c,a ( T ) =

Original: P
Revised: P

Original: Tt =1 1 { c t = c, a t = a } .
Revised: Tt =1 1 { c t = c, a t = a } .

Original: For any ↵ -minimax regret algorithm, it is (0 ,  ) -PAC if at a stopping time ⌧ it outputs the optimal policy ⇡
Revised: For any ↵ -minimax regret algorithm, it is (0 ,  ) -PAC if at a stopping time ⌧ it outputs the optimal policy ⇡

Original: ⇤ w. p. at least 1   .
Revised: ⇤ w. p. at least 1   .

Original: Any ↵ -minimax regret algorithm that is (0 ,  ) -PAC satisﬁes E µ [ ⌧ ]  m 2   2 log 2 (1 / 2 .
Revised: Any ↵ -minimax regret algorithm that is (0 ,  ) -PAC satisﬁes E µ [ ⌧ ]  m 2   2 log 2 (1 / 2 .

Original: 4  ) / 4 ↵ for the instance µ = ( ⌫ , r ) deﬁned in 2.5.
Revised: 4  ) / 4 ↵ for the instance µ = ( ⌫ , r ) deﬁned in 2.5.

Original: We point out that the minimax regret optimal rate takes ↵
Revised: We point out that the minimax regret optimal rate takes ↵

Original: = log( m ) =
Revised: = log( m ) =

Original: log( | ⇧ | ) .
Revised: log( | ⇧ | ) .

Original: Thus, taking  =and  = 0 . 1 , the minimax regret optimal algorithm has a PAC sample complexity of m 2 / log( m ) ; whereas the PAC sample complexity of our algorithm, Theorem 2.4, is just m log( m ) .
Revised: Thus, taking  =and  = 0 . 1 , the minimax regret optimal algorithm has a PAC sample complexity of m 2 / log( m ) ; whereas the PAC sample complexity of our algorithm, Theorem 2.4, is just m log( m ) .

Original: That is, algorithms with optimal minimax regret have a sample complexity that is at least nearly the optimal
Revised: That is, algorithms with optimal minimax regret have a sample complexity that is at least nearly the optimal

Original: PAC sample complexity squared .
Revised: PAC sample complexity squared .

Original: This demonstrates that no algorithm can simultaneously be minimax regret optimal and obtain the optimal PAC sample complexity.
Revised: This demonstrates that no algorithm can simultaneously be minimax regret optimal and obtain the optimal PAC sample complexity.

Original: 2.2 Trivial policy class
Revised: 2.2 Trivial policy class

Original: As a warm-up to discussing linear policy classes, let us consider the simplest policy class.
Revised: As a warm-up to discussing linear policy classes, let us consider the simplest policy class.

Original: Deﬁnition 2.7 (Trivial policy class) .
Revised: Deﬁnition 2.7 (Trivial policy class) .

Original: Assume |C| < 1 and let ⇧ = { ⇡ ( c )
Revised: Assume |C| < 1 and let ⇧ = { ⇡ ( c )

Original: = a : ( c, a ) 2 C ⇥ A} so that | ⇧ | = |A| |C| .
Revised: = a : ( c, a ) 2 C ⇥ A} so that | ⇧ | = |A| |C| .

Original: The trivial policy class has the ﬂexibility to predict any action a 2 A individually for each c 2 C .
Revised: The trivial policy class has the ﬂexibility to predict any action a 2 A individually for each c 2 C .

Original: This allows us to show that ⇢ ⇧ , 0  max c 2 ⌫ c
Revised: This allows us to show that ⇢ ⇧ , 0  max c 2 ⌫ c

Original: P a 0   2 c,a 0 (see Appendix A.3).
Revised: P a 0   2 c,a 0 (see Appendix A.3).

Original: An immediate corollary of Theorem 2.4 is obtained by simply noting that | ⇧ | = |A| |C| .
Revised: An immediate corollary of Theorem 2.4 is obtained by simply noting that | ⇧ | = |A| |C| .

Original: Corollary 2.8 (Trivial class, upper) .
Revised: Corollary 2.8 (Trivial class, upper) .

Original: Fix ✏ > 0 and  2 (0 , 1) .
Revised: Fix ✏ > 0 and  2 (0 , 1) .

Original: Let ⇧ be the trivial policy class applied to some ﬁxed C , A spaces.
Revised: Let ⇧ be the trivial policy class applied to some ﬁxed C , A spaces.

Original: Then under Assumption 1 there exists a computationally efﬁcient ( ✏ ,  ) -PAC algorithm for contextual bandits that satisﬁes ⌧  min { A ✏  2 , max c 1 ⌫ c
Revised: Then under Assumption 1 there exists a computationally efﬁcient ( ✏ ,  ) -PAC algorithm for contextual bandits that satisﬁes ⌧  min { A ✏  2 , max c 1 ⌫ c

Original: P a 0   2 c,a 0 } ( |C| log( |A| )
Revised: P a 0   2 c,a 0 } ( |C| log( |A| )

Original: + log(1 /  ))
Revised: + log(1 /  ))

Original: log(1 /  ✏ ) , where  ✏ = max { ✏ , min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )
Revised: log(1 /  ✏ ) , where  ✏ = max { ✏ , min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )

Original:  V ( ⇡ ) } .
Revised:  V ( ⇡ ) } .

Original: Furthermore, this sample complexity never exceeds |A| ( |C| log( |A| )
Revised: Furthermore, this sample complexity never exceeds |A| ( |C| log( |A| )

Original: +log(1 /  ))
Revised: +log(1 /  ))

Original: ✏log(1 / ✏ ) .
Revised: ✏log(1 / ✏ ) .

Original: Ignoring log factors, the minimax sample complexity of the trivial class is just ✏  2 |A| ( |C| +log(1 /  )) .
Revised: Ignoring log factors, the minimax sample complexity of the trivial class is just ✏  2 |A| ( |C| +log(1 /  )) .

Original: This is actually a somewhat surprising result, because it says lim  !
Revised: This is actually a somewhat surprising result, because it says lim  !

Original: E [ ⌧ ] log(1 /  ) !
Revised: E [ ⌧ ] log(1 /  ) !

Original: ✏  2 |A| which is independent of |C| .
Revised: ✏  2 |A| which is independent of |C| .

Original: To see why this result is somewhat remarkable, if we played a best-arm identiﬁcation algorithm for each of the |C| contexts, then this would lead to a sample complexity of ✏  2 |C| · |A| log(1 /  ) .
Revised: To see why this result is somewhat remarkable, if we played a best-arm identiﬁcation algorithm for each of the |C| contexts, then this would lead to a sample complexity of ✏  2 |C| · |A| log(1 /  ) .

Original: It is somewhat of a surprise that such a natural strategy is not optimal.
Revised: It is somewhat of a surprise that such a natural strategy is not optimal.

Original: For intuition for why we can avoid the multiplicative |C| , note that to identify an ✏ -good policy among just two policies ( ⇡ , ⇡ ⇤ ) using uniform exploration requires just ✏  2 |A| log(1 /  ) samples.
Revised: For intuition for why we can avoid the multiplicative |C| , note that to identify an ✏ -good policy among just two policies ( ⇡ , ⇡ ⇤ ) using uniform exploration requires just ✏  2 |A| log(1 /  ) samples.

Original: When we have more than two policies, a union bound achieves the claimed result.
Revised: When we have more than two policies, a union bound achieves the claimed result.

Original: The minimax sample complexity of Corollary 2.8 (i.e., the second statement) is nearly tight:
Revised: The minimax sample complexity of Corollary 2.8 (i.e., the second statement) is nearly tight:

Original: Theorem 2.9 (Trivial class, lower) .
Revised: Theorem 2.9 (Trivial class, lower) .

Original: Fix ✏ > 0 and  2 (0 , 1 / 6) .
Revised: Fix ✏ > 0 and  2 (0 , 1 / 6) .

Original: Let ⇧ be the trivial policy class applied to some ﬁxed C , A spaces.
Revised: Let ⇧ be the trivial policy class applied to some ﬁxed C , A spaces.

Original: Moreover, ﬁx a contextual bandit instance µ = ( ⌫ , r ) and a collection of policies ⇧ .
Revised: Moreover, ﬁx a contextual bandit instance µ = ( ⌫ , r ) and a collection of policies ⇧ .

Original: Then any (0 ,  ) -PAC algorithm for contextual bandits satisﬁes
Revised: Then any (0 ,  ) -PAC algorithm for contextual bandits satisﬁes

Original: E µ [ ⌧ ]  max c 1 ⌫ c
Revised: E µ [ ⌧ ]  max c 1 ⌫ c

Original: P a   2 c,a log(1 / 2 . 4  ) .
Revised: P a   2 c,a log(1 / 2 . 4  ) .

Original: Furthermore, sup µ E µ [ ⌧ ]  ✏  2 |A| ( |C| + log(1 /  )) .
Revised: Furthermore, sup µ E µ [ ⌧ ]  ✏  2 |A| ( |C| + log(1 /  )) .

Original: 2.3 Linear policy class
Revised: 2.3 Linear policy class

Original: A particularly compelling model-class of policies is the set of linear policies.
Revised: A particularly compelling model-class of policies is the set of linear policies.

Original: Deﬁnition 2.10 (Linear policy class) .
Revised: Deﬁnition 2.10 (Linear policy class) .

Original: Fix a feature map  : C ⇥ A !
Revised: Fix a feature map  : C ⇥ A !

Original: R d and assume it is known to the learner.
Revised: R d and assume it is known to the learner.

Original: Let ⇧ = { ⇡ ( c ) =
Revised: Let ⇧ = { ⇡ ( c ) =

Original: arg max a 2 A h  ( c, a ) , ✓ i , 8 ✓ 2 R d } .
Revised: arg max a 2 A h  ( c, a ) , ✓ i , 8 ✓ 2 R d } .

Original: We can consider two settings: the agnostic setting and the realizable setting.
Revised: We can consider two settings: the agnostic setting and the realizable setting.

Original: In the agnostic setting, there is no assumed relationship between the true reward function r ( c, a ) and  : C ⇥ A !
Revised: In the agnostic setting, there is no assumed relationship between the true reward function r ( c, a ) and  : C ⇥ A !

Original: R d .
Revised: R d .

Original: In this case, Theorem 2.4 applies directly by taking a cover of ⇧ .
Revised: In this case, Theorem 2.4 applies directly by taking a cover of ⇧ .

Original: Corollary 2.11 (Agnostic, upper bound) .
Revised: Corollary 2.11 (Agnostic, upper bound) .

Original: Fix ✏  0 and  2 (0 , 1) .
Revised: Fix ✏  0 and  2 (0 , 1) .

Original: Let ⇧ be the linear policy class in R d .
Revised: Let ⇧ be the linear policy class in R d .

Original: Under Assumption 1 there exists a computationally efﬁcient ( ✏ ,  ) -PAC algorithm for contextual bandits that satisﬁes ⌧  ⇢ ⇧ , ✏ · ( d log(1 / ✏ ) + log(1 /  )) log(1 /  ✏ ) where  ✏ = max { ✏ , min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )
Revised: Under Assumption 1 there exists a computationally efﬁcient ( ✏ ,  ) -PAC algorithm for contextual bandits that satisﬁes ⌧  ⇢ ⇧ , ✏ · ( d log(1 / ✏ ) + log(1 /  )) log(1 /  ✏ ) where  ✏ = max { ✏ , min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )

Original:  V ( ⇡ ) } .
Revised:  V ( ⇡ ) } .

Original: Furthermore, this sample complexity never exceeds
Revised: Furthermore, this sample complexity never exceeds

Original: [Equation]
Revised: [Equation]

Original: Comparing to the lower bound of Theorem 2.2, the instance dependent upper bound of Corollary 2.
Revised: Comparing to the lower bound of Theorem 2.2, the instance dependent upper bound of Corollary 2.

Original: matches up to a factor of the dimension and negligible log factors.
Revised: matches up to a factor of the dimension and negligible log factors.

Original: In contrast to the “model-free” feel of the agnostic case, we can also consider a “model-based” type setting, i.e. the realizable setting.
Revised: In contrast to the “model-free” feel of the agnostic case, we can also consider a “model-based” type setting, i.e. the realizable setting.

Original: Deﬁnition 2.12 (Realizable) .
Revised: Deﬁnition 2.12 (Realizable) .

Original: We say the linear policy class is realizable if there exists a ✓ ⇤ 2 R d such that r ( c, a ) =
Revised: We say the linear policy class is realizable if there exists a ✓ ⇤ 2 R d such that r ( c, a ) =

Original: h  ( c, a ) , ✓ ⇤ i for all c 2 C and a 2 A .
Revised: h  ( c, a ) , ✓ ⇤ i for all c 2 C and a 2 A .

Original: Thus, for any ⇡ 2 ⇧ we have V ( ⇡ ) =
Revised: Thus, for any ⇡ 2 ⇧ we have V ( ⇡ ) =

Original: E c ⇠ ⌫ [ r ( c, ⇡ ( c ))]
Revised: E c ⇠ ⌫ [ r ( c, ⇡ ( c ))]

Original: =
Revised: =

Original: E c ⇠ ⌫ [ h  ( c, ⇡ ( c )) , ✓ ⇤ i ]
Revised: E c ⇠ ⌫ [ h  ( c, ⇡ ( c )) , ✓ ⇤ i ]

Original: = h  ⇡ , ✓ ⇤ i with  ⇡ :=
Revised: = h  ⇡ , ✓ ⇤ i with  ⇡ :=

Original: E c ⇠ ⌫ [  ( c, ⇡ ( c ))] .
Revised: E c ⇠ ⌫ [  ( c, ⇡ ( c ))] .

Original: Finally, at the start of the game the learner knows this model.
Revised: Finally, at the start of the game the learner knows this model.

Original: The setting in Deﬁnition 2.12 is commonly referred to as the linear contextual bandit problem [1].
Revised: The setting in Deﬁnition 2.12 is commonly referred to as the linear contextual bandit problem [1].

Original: Clearly, we have that ⇡ ⇤ ( c ) =
Revised: Clearly, we have that ⇡ ⇤ ( c ) =

Original: arg max a 2 A h  ( c, a ) ,
Revised: arg max a 2 A h  ( c, a ) ,

Original: ✓ ⇤ i .
Revised: ✓ ⇤ i .

Original: We begin by deﬁning a quantity fundamental to our sample complexity results:
Revised: We begin by deﬁning a quantity fundamental to our sample complexity results:

Original: [Equation]
Revised: [Equation]

Original: Theorem 2.13 (Realizable, lower bound) .
Revised: Theorem 2.13 (Realizable, lower bound) .

Original: Fix ✏ = 

Revised: Fix ✏ =

Original: and  2 (0 , 1) .
Revised: and  2 (0 , 1) .

Original: Let ⇧ be the linear policy class in R d and assume it is realizable (see Deﬁnitions 2.10 and 2.12).
Revised: Let ⇧ be the linear policy class in R d and assume it is realizable (see Deﬁnitions 2.10 and 2.12).

Original: Any (0 ,  ) -PAC algorithm in this setting satisﬁes E [ ⌧ ]  ⇢ lin , 0 · log(1 / 2 .
Revised: Any (0 ,  ) -PAC algorithm in this setting satisﬁes E [ ⌧ ]  ⇢ lin , 0 · log(1 / 2 .

Original: 4  ) .
Revised: 4  ) .

Original: We now state our nearly matching upper bound.
Revised: We now state our nearly matching upper bound.

Original: However, in this case we note that the algorithm is not computationally efﬁcient.
Revised: However, in this case we note that the algorithm is not computationally efﬁcient.

Original: Theorem 2.14 (Realizable, upper bound) .
Revised: Theorem 2.14 (Realizable, upper bound) .

Original: Fix ✏  0 and  2 (0 , 1) .
Revised: Fix ✏  0 and  2 (0 , 1) .

Original: Let ⇧ be the linear policy class in R d and assume it is realizable (see Deﬁnitions 2.10 and 2.12).
Revised: Let ⇧ be the linear policy class in R d and assume it is realizable (see Deﬁnitions 2.10 and 2.12).

Original: Under Assumption 1 there exists an ( ✏ ,  ) -PAC algorithm (see Algorithm 1) for this setting satisfying
Revised: Under Assumption 1 there exists an ( ✏ ,  ) -PAC algorithm (see Algorithm 1) for this setting satisfying

Original: [Equation]
Revised: [Equation]

Original: where  ✏ = max { ✏ , min ⇡
Revised: where  ✏ = max { ✏ , min ⇡

Original: [Equation]
Revised: [Equation]

Original: Furthermore, this sample complexity never exceeds
Revised: Furthermore, this sample complexity never exceeds

Original: [Equation]
Revised: [Equation]

Original: log
Revised: log

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: We remark that the algorithm that achieves this upper bound is very different than popular optimismbased algorithms for linear contextual bandits e.g., UCB or Thompson sampling [1].
Revised: We remark that the algorithm that achieves this upper bound is very different than popular optimismbased algorithms for linear contextual bandits e.g., UCB or Thompson sampling [1].

Original: Indeed, our algorithm computes an experimental design and is related to instance-dependent linear bandit algorithms developed for best-arm identiﬁcation [9, 12, 13, 35] and regret minimization [19, 36].
Revised: Indeed, our algorithm computes an experimental design and is related to instance-dependent linear bandit algorithms developed for best-arm identiﬁcation [9, 12, 13, 35] and regret minimization [19, 36].

Original: To our knowledge, Theorem 2.14 provides the ﬁrst instance-dependent sample complexity for the PAC setting of linear contextual bandits.
Revised: To our knowledge, Theorem 2.14 provides the ﬁrst instance-dependent sample complexity for the PAC setting of linear contextual bandits.

Original: The most relevant work to Theorem 2.14 is the work of [41] which demonstrated a minimax sample complexity of d 2 / ✏ 2 log(1 /  ) .
Revised: The most relevant work to Theorem 2.14 is the work of [41] which demonstrated a minimax sample complexity of d 2 / ✏ 2 log(1 /  ) .

Original: Also, we remark that the lower and upper bounds in this section require an additive Gaussian noise.
Revised: Also, we remark that the lower and upper bounds in this section require an additive Gaussian noise.

Original: Remark 2.15 (Agnostic vs. Realizable) .
Revised: Remark 2.15 (Agnostic vs. Realizable) .

Original: Contrasting the above results, we note that the sample complexity of the agnostic case is always bounded by |A| d/ ✏ 2 .
Revised: Contrasting the above results, we note that the sample complexity of the agnostic case is always bounded by |A| d/ ✏ 2 .

Original: whereas it never exceeds d 2 / ✏ 2 for the realizable case.
Revised: whereas it never exceeds d 2 / ✏ 2 for the realizable case.

Original: This matches the intuition that when the number of actions is much larger than the dimension, assuming realizability can signiﬁcantly reduce the sample complexity.
Revised: This matches the intuition that when the number of actions is much larger than the dimension, assuming realizability can signiﬁcantly reduce the sample complexity.

Original: 2.4 Comparison to the Disagreement Coefﬁcient
Revised: 2.4 Comparison to the Disagreement Coefﬁcient

Original: The work of [16] provides regret bounds in terms of instance-dependent quantities inspired by the disagreement coefﬁcient , a notion of complexity common in the active learning literature [18].
Revised: The work of [16] provides regret bounds in terms of instance-dependent quantities inspired by the disagreement coefﬁcient , a notion of complexity common in the active learning literature [18].

Original: The following corollary relates our sample complexity to these notions of disagreement coefﬁcients.
Revised: The following corollary relates our sample complexity to these notions of disagreement coefﬁcients.

Original: Deﬁne the policy disagreement coefﬁcient as
Revised: Deﬁne the policy disagreement coefﬁcient as

Original: [Equation]
Revised: [Equation]

Original: ✏ where ⇧ ✏ := { ⇡ 2 ⇧ : P ⌫ ( ⇡ ( c ) 6 = ⇡ ⇤ ( c ))  ✏ } and the cost-sensitive disagreement coefﬁcient as
Revised: ✏ where ⇧ ✏ := { ⇡ 2 ⇧ : P ⌫ ( ⇡ ( c ) 6 = ⇡ ⇤ ( c ))  ✏ } and the cost-sensitive disagreement coefﬁcient as

Original: [Equation]
Revised: [Equation]

Original: ✏ .
Revised: ✏ .

Original: The
Revised: The

Original: AdaCB algorithm of [16] achieves a regret of roughly
Revised: AdaCB algorithm of [16] achieves a regret of roughly

Original: R T =
Revised: R T =

Original: O ⇣ min 
Revised: O ⇣ min

Original: [Equation]
Revised: [Equation]

Original: Observe that at time T , given the outputs ⇡ 1 , ⇡ 2 , · · · , ⇡ T from AdaCB algorithm, one could return a (randomized) policy ˜ ⇡ which on observing a context, samples from the empirical distribution over the outputs.
Revised: Observe that at time T , given the outputs ⇡ 1 , ⇡ 2 , · · · , ⇡ T from AdaCB algorithm, one could return a (randomized) policy ˜ ⇡ which on observing a context, samples from the empirical distribution over the outputs.

Original: By Markov’s inequality we have ˜ ⇡ , V ( ⇡ ⇤ )
Revised: By Markov’s inequality we have ˜ ⇡ , V ( ⇡ ⇤ )

Original:  V (˜ ⇡ )
Revised:  V (˜ ⇡ )

Original:  O ( ✏ ) with constant probability for ✏ = R T T .
Revised:  O ( ✏ ) with constant probability for ✏ = R T T .

Original: Therefore, an upper bound on the regret translates to a PAC sample complexity of |A| log( | ⇧ | ) ✏  uniform C pol ⇧ ( ✏ /  uniform ) or |A| log( | ⇧ | ) ✏
Revised: Therefore, an upper bound on the regret translates to a PAC sample complexity of |A| log( | ⇧ | ) ✏  uniform C pol ⇧ ( ✏ /  uniform ) or |A| log( | ⇧ | ) ✏

Original: C csc ⇧ ( ✏ ) .
Revised: C csc ⇧ ( ✏ ) .

Original: Finally, Corollary 2.16 shows that this sample complexity bound is at least as large as our upper bound, see Appendix A.5 for the proof.
Revised: Finally, Corollary 2.16 shows that this sample complexity bound is at least as large as our upper bound, see Appendix A.5 for the proof.

Original: Corollary 2.16.
Revised: Corollary 2.16.

Original: Recall that  uniform := min c 2 C min a 2 A r ( c, ⇡ ⇤ ( c ))
Revised: Recall that  uniform := min c 2 C min a 2 A r ( c, ⇡ ⇤ ( c ))

Original:  r ( c, a ) .
Revised:  r ( c, a ) .

Original: For any ✏ 0 > 0 we have that
Revised: For any ✏ 0 > 0 we have that

Original: [Equation]
Revised: [Equation]

Original: Moreover, for all ✏ 0  0 we have that ⇢ ⇧ , ✏ 0 < 1 whenever  pol :
Revised: Moreover, for all ✏ 0  0 we have that ⇢ ⇧ , ✏ 0 < 1 whenever  pol :

Original: =
Revised: =

Original: V ( ⇡ ⇤ )
Revised: V ( ⇡ ⇤ )

Original:  max ⇡ 6 = ⇡ ⇤ V ( ⇡ ) > 0 .
Revised:  max ⇡ 6 = ⇡ ⇤ V ( ⇡ ) > 0 .

Original:  Optimal Algorithms for Contextual Bandits 3.1 Reduction to linear realizability and a simple elimination scheme
Revised:  Optimal Algorithms for Contextual Bandits 3.1 Reduction to linear realizability and a simple elimination scheme

Original: The astute reader may have noticed that if we ignore computation, Theorem 2.4 is actually an immediate corollary of Theorem 2.14 by taking  ( c, a )
Revised: The astute reader may have noticed that if we ignore computation, Theorem 2.4 is actually an immediate corollary of Theorem 2.14 by taking  ( c, a )

Original: = vec ( e c e > a ) 2 R |C|·|A| where e i is a one-hot encoded vector so that r ( c, a ) =
Revised: = vec ( e c e > a ) 2 R |C|·|A| where e i is a one-hot encoded vector so that r ( c, a ) =

Original: h  ( c, a ) , ✓ ⇤ i with ✓ ⇤ 2 R |C|·|A| .
Revised: h  ( c, a ) , ✓ ⇤ i with ✓ ⇤ 2 R |C|·|A| .

Original: This observation is key to our sample complexity results.
Revised: This observation is key to our sample complexity results.

Original: Recalling  ⇡ :=
Revised: Recalling  ⇡ :=

Original: E c ⇠ ⌫ [  ( c, ⇡ ( c ))] ( from Deﬁnition 2.12), we have that V ( ⇡ ) =
Revised: E c ⇠ ⌫ [  ( c, ⇡ ( c ))] ( from Deﬁnition 2.12), we have that V ( ⇡ ) =

Original: E c ⇠ ⌫ [ r ( c, ⇡ ( c ))]
Revised: E c ⇠ ⌫ [ r ( c, ⇡ ( c ))]

Original: =
Revised: =

Original: E c ⇠ ⌫ [ h  ( c, ⇡ ( c )) , ✓ ⇤ i ]
Revised: E c ⇠ ⌫ [ h  ( c, ⇡ ( c )) , ✓ ⇤ i ]

Original: =
Revised: =

Original: h  ⇡ , ✓ ⇤ i .
Revised: h  ⇡ , ✓ ⇤ i .

Original: We stress that C can be uncountable, and thus we would never actually instantiate any of the vectors  ( c, a ) .
Revised: We stress that C can be uncountable, and thus we would never actually instantiate any of the vectors  ( c, a ) .

Original: For notational convenience, deﬁne the feasible set of (context, action) probability distributions as ⌦ =
Revised: For notational convenience, deﬁne the feasible set of (context, action) probability distributions as ⌦ =

Original: [Equation]
Revised: [Equation]

Original: a
Revised: a

Original: [Equation]
Revised: [Equation]

Original: o .
Revised: o .

Original: Note that for each context, p c := { w c,a / ⌫ c } a 2 A 2  A deﬁnes a probability distribution over actions.
Revised: Note that for each context, p c := { w c,a / ⌫ c } a 2 A 2  A deﬁnes a probability distribution over actions.

Original: Also deﬁne A ( w ) :=
Revised: Also deﬁne A ( w ) :=

Original: P c,a w c,a  ( c, a )  ( c, a ) > for any w 2 ⌦ .
Revised: P c,a w c,a  ( c, a )  ( c, a ) > for any w 2 ⌦ .

Original: Under this notation, recalling the right hand side from Theorems 2.13 and 2.14 we have
Revised: Under this notation, recalling the right hand side from Theorems 2.13 and 2.14 we have

Original: [Equation]
Revised: [Equation]

Original: To show that the sample complexity of Theorem 2.4 is a corollary of Theorem 2.14, it sufﬁces to show that equation (1) and the above display are equal.
Revised: To show that the sample complexity of Theorem 2.4 is a corollary of Theorem 2.14, it sufﬁces to show that equation (1) and the above display are equal.

Original: To see this, observe
Revised: To see this, observe

Original: [Equation]
Revised: [Equation]

Original: Due to this equivalence, the lower bound of Theorem 2.2 is also a corollary of Theorem 2.13.
Revised: Due to this equivalence, the lower bound of Theorem 2.2 is also a corollary of Theorem 2.13.

Original: The lower bound of Theorem 2.13 follows almost immediately from the lower bound argument in [13].
Revised: The lower bound of Theorem 2.13 follows almost immediately from the lower bound argument in [13].

Original: The conclusion of this section is that from a sample complexity analysis alone, all that is left is to prove Theorem 2.14.
Revised: The conclusion of this section is that from a sample complexity analysis alone, all that is left is to prove Theorem 2.14.

Original: In the next section we propose an algorithm that achieves this sample complexity but assumes precise knowledge of the context distribution ⌫ (this is relaxed in following sections).
Revised: In the next section we propose an algorithm that achieves this sample complexity but assumes precise knowledge of the context distribution ⌫ (this is relaxed in following sections).

Original: While the algorithm is highly impractical for a number of reasons, its analysis provides a great deal of intuition and motivation for our ﬁnal algorithm.
Revised: While the algorithm is highly impractical for a number of reasons, its analysis provides a great deal of intuition and motivation for our ﬁnal algorithm.

Original: 3.2 A simple, impractical, elimination-style algorithm
Revised: 3.2 A simple, impractical, elimination-style algorithm

Original: Algorithm 1 provides an initial elimination based method for the PAC-contextual bandit problem.
Revised: Algorithm 1 provides an initial elimination based method for the PAC-contextual bandit problem.

Original: The algorithm runs in stages.
Revised: The algorithm runs in stages.

Original: Before the start of each stage ` 2 N , the algorithm deﬁnes a distribution p ( ` ) c
Revised: Before the start of each stage ` 2 N , the algorithm deﬁnes a distribution p ( ` ) c

Original: 2 4 A for each c 2 C .
Revised: 2 4 A for each c 2 C .

Original: At each successive time t 2 [ n ` ] , it plays random action a t ⇠ p ( ` )
Revised: At each successive time t 2 [ n ` ] , it plays random action a t ⇠ p ( ` )

Original: c t in response to context c t ⇠ ⌫ , and receives random reward r t with E [ r t | c t , a t ] =
Revised: c t in response to context c t ⇠ ⌫ , and receives random reward r t with E [ r t | c t , a t ] =

Original: h  ( c t , a t ) , ✓ ⇤ i .
Revised: h  ( c t , a t ) , ✓ ⇤ i .

Original: Observe that
Revised: Observe that

Original: [Equation]
Revised: [Equation]

Original: using the identity w ( ` ) c,a := ⌫ c p ( ` ) c,a .
Revised: using the identity w ( ` ) c,a := ⌫ c p ( ` ) c,a .

Original: Thus, if we set O t = A ( w ( ` ) )  1  ( c t , a t ) r t then E
Revised: Thus, if we set O t = A ( w ( ` ) )  1  ( c t , a t ) r t then E

Original: [ O t ] = ✓ ⇤ .
Revised: [ O t ] = ✓ ⇤ .

Original: A straightforward calculation also shows that Cov ( O t ) =
Revised: A straightforward calculation also shows that Cov ( O t ) =

Original: A ( w ( ` ) )  1 if r t is perturbed with additive unit variance noise.
Revised: A ( w ( ` ) )  1 if r t is perturbed with additive unit variance noise.

Original: Thus, an unbiased estimator of  ( ⇡ , ⇡ ⇤ ) :
Revised: Thus, an unbiased estimator of  ( ⇡ , ⇡ ⇤ ) :

Original: = V ( ⇡ ⇤ )  V ( ⇡ ) =
Revised: = V ( ⇡ ⇤ )  V ( ⇡ ) =

Original: h  ⇡ ⇤   ⇡ , ✓ ⇤ i is simply h  ⇡ ⇤   ⇡ , 1 n `
Revised: h  ⇡ ⇤   ⇡ , ✓ ⇤ i is simply h  ⇡ ⇤   ⇡ , 1 n `

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: O t i which has variance 1 n ` k  ⇡ ⇤   ⇡ k 2 A ( w ( ` ) )  1 .
Revised: O t i which has variance 1 n ` k  ⇡ ⇤   ⇡ k 2 A ( w ( ` ) )  1 .

Original: Intuitively, h  ⇡ ⇤   ⇡ , 1 n
Revised: Intuitively, h  ⇡ ⇤   ⇡ , 1 n

Original: [Equation]
Revised: [Equation]

Original: 1 so we can safely conclude that a policy ⇡ is sub-optimal (i.e., ⇡ 6 = ⇡ ⇤ )
Revised: 1 so we can safely conclude that a policy ⇡ is sub-optimal (i.e., ⇡ 6 = ⇡ ⇤ )

Original: if there exists any policy ⇡ 0 such that h  ⇡
Revised: if there exists any policy ⇡ 0 such that h  ⇡

Original: [Equation]
Revised: [Equation]

Original:   ⇡ , 1 n
Revised:   ⇡ , 1 n

Original: [Equation]
Revised: [Equation]

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: This is the intuition behind Contextual RAGE (Algorithm 1), which inherits its name from the best-arm identiﬁcation algorithm of [13] that inspired its strategy.
Revised: This is the intuition behind Contextual RAGE (Algorithm 1), which inherits its name from the best-arm identiﬁcation algorithm of [13] that inspired its strategy.

Original: However, while h  ⇡ ⇤   ⇡ , 1 n `
Revised: However, while h  ⇡ ⇤   ⇡ , 1 n `

Original: P t O t i is unbiased and has controlled variance, it is potentially heavy-tailed because w ( ` ) c,a can be arbitrarily small.
Revised: P t O t i is unbiased and has controlled variance, it is potentially heavy-tailed because w ( ` ) c,a can be arbitrarily small.

Original: Instead of trying to control w ( ` ) c,a and appealing to
Revised: Instead of trying to control w ( ` ) c,a and appealing to

Original: Bernstein’s inequality, in line 7 we use the robust mean estimator of Catoni [28].
Revised: Bernstein’s inequality, in line 7 we use the robust mean estimator of Catoni [28].

Original: We can then show:
Revised: We can then show:

Original: Lemma 3.1.
Revised: Lemma 3.1.

Original: ⇡
Revised: ⇡

Original: ⇤ 2 ⇧ ` and max ⇡ 2 ⇧ ` h  ⇡ ⇤   ⇡ , ✓ ⇤ i  4 ✏ ` for all ` > 1 w.p. at least 1   .
Revised: ⇤ 2 ⇧ ` and max ⇡ 2 ⇧ ` h  ⇡ ⇤   ⇡ , ✓ ⇤ i  4 ✏ ` for all ` > 1 w.p. at least 1   .

Original: The lemma states that if ⇧ ` is the active set of policies still under consideration, the optimal policy ⇡ ⇤ is never discarded from ⇧ ` , and moreover, the quality of all policies remaining in ⇧ ` is getting better and better.
Revised: The lemma states that if ⇧ ` is the active set of policies still under consideration, the optimal policy ⇡ ⇤ is never discarded from ⇧ ` , and moreover, the quality of all policies remaining in ⇧ ` is getting better and better.

Original: We are now ready to state the main sample complexity result, with proof in Appendix B.
Revised: We are now ready to state the main sample complexity result, with proof in Appendix B.

Original: Theorem 3.2.
Revised: Theorem 3.2.

Original: Fix any policy class ⇧ = { ⇡ : C !
Revised: Fix any policy class ⇧ = { ⇡ : C !

Original: A} ⇡ , distribution over contexts ⌫ ,  2 (0 , 1) , ✏  0 , and feature map  : C ⇥ A !
Revised: A} ⇡ , distribution over contexts ⌫ ,  2 (0 , 1) , ✏  0 , and feature map  : C ⇥ A !

Original: R d such that r ( c, a ) =
Revised: R d such that r ( c, a ) =

Original: h  ( c, a ) , ✓ ⇤ i
Revised: h  ( c, a ) , ✓ ⇤ i

Original: (w.l.o.g. one can always take  ( c, a ) =
Revised: (w.l.o.g. one can always take  ( c, a ) =

Original: vec ( e c e > a ) ).
Revised: vec ( e c e > a ) ).

Original: With probability at least 1   , if  ⇡ = E c ⇠ ⌫ [  ( c, ⇡ ( c ))] and ⇡ ⇤ = arg max ⇡ h  ⇡ , ✓ ⇤ i then Contextual-RAGE returns a policy b ⇡ 2 ⇧ such that V ( b ⇡ )  V ( ⇡ ⇤ )
Revised: With probability at least 1   , if  ⇡ = E c ⇠ ⌫ [  ( c, ⇡ ( c ))] and ⇡ ⇤ = arg max ⇡ h  ⇡ , ✓ ⇤ i then Contextual-RAGE returns a policy b ⇡ 2 ⇧ such that V ( b ⇡ )  V ( ⇡ ⇤ )

Original:  ✏ after taking at most
Revised:  ✏ after taking at most

Original: [Equation]
Revised: [Equation]

Original: samples, where c is an absolute constant and  = min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )
Revised: samples, where c is an absolute constant and  = min ⇡ 2 ⇧ \ ⇡ ⇤ V ( ⇡ ⇤ )

Original:  V ( ⇡ ) .
Revised:  V ( ⇡ ) .

Original: 3.3 Towards a more efﬁcient algorithm
Revised: 3.3 Towards a more efﬁcient algorithm

Original: One major issue with Algorithm 1 is that it explicitly maintains a set of policies ⇧ ` from round to round.
Revised: One major issue with Algorithm 1 is that it explicitly maintains a set of policies ⇧ ` from round to round.

Original: Since ⇧ could be exponential in |A| , this is a non-starter for any implementation.
Revised: Since ⇧ could be exponential in |A| , this is a non-starter for any implementation.

Original: As a motivation for our approach, we consider a non-elimination algorithm, Algorithm 2, as an intermediate step.
Revised: As a motivation for our approach, we consider a non-elimination algorithm, Algorithm 2, as an intermediate step.

Original: It does not maintain ⇧ ` and instead just solves the optimization problem (2) over ⇧ .
Revised: It does not maintain ⇧ ` and instead just solves the optimization problem (2) over ⇧ .

Original: The design computed in (2) is chosen to ensure that for all ⇡ 2 ⇧ , | b 
Revised: The design computed in (2) is chosen to ensure that for all ⇡ 2 ⇧ , | b

Original: [Equation]
Revised: [Equation]

Original: b ⇡
Revised: b ⇡

Original: [Equation]
Revised: [Equation]

Original: 
Revised:

Original: [Equation]
Revised: [Equation]

Original: )   ( ⇡
Revised: )   ( ⇡

Original: [Equation]
Revised: [Equation]

Original: ⇡ ⇤ ) |  2 ✏ `  1 + 14  ( ⇡ , ⇡ ⇤ ) with high probability (Lemma C.3).
Revised: ⇡ ⇤ ) |  2 ✏ `  1 + 14  ( ⇡ , ⇡ ⇤ ) with high probability (Lemma C.3).

Original: Equivalently, we estimate gaps up to a constant factor for policies with  ( ⇡ , ⇡ ⇤ ) >
Revised: Equivalently, we estimate gaps up to a constant factor for policies with  ( ⇡ , ⇡ ⇤ ) >

Original: ✏ ` , while our gap estimates are bounded by ✏ ` for those policies satisfying  ( ⇡ , ⇡ ⇤ )  ✏ ` .
Revised: ✏ ` , while our gap estimates are bounded by ✏ ` for those policies satisfying  ( ⇡ , ⇡ ⇤ )  ✏ ` .

Original: This ensures that our choice of b ⇡ ` is good enough, i.e. satisﬁes
Revised: This ensures that our choice of b ⇡ ` is good enough, i.e. satisﬁes

Original: V ( ⇡ ⇤ )
Revised: V ( ⇡ ⇤ )

Original:  V ( b ⇡ ` )  ✏ ` with high probability.
Revised:  V ( b ⇡ ` )  ✏ ` with high probability.

Original: The full proof is in Appendix C.
Revised: The full proof is in Appendix C.

Original: Unfortunately, Algorithm 2 introduces additional problems.
Revised: Unfortunately, Algorithm 2 introduces additional problems.

Original: It is not clear whether solving (2) is computationally efﬁcient.
Revised: It is not clear whether solving (2) is computationally efﬁcient.

Original: Also, we need to ﬁnd an estimator b  l that is computationally efﬁcient even if the policy space ⇧ is inﬁnite.
Revised: Also, we need to ﬁnd an estimator b  l that is computationally efﬁcient even if the policy space ⇧ is inﬁnite.

Original: In addition, it requires precise knowledge of ⌫ to even deﬁne the domain of distributions ⌦ optimized over, and store the solution w 2 C ⇥ A explicitly.
Revised: In addition, it requires precise knowledge of ⌫ to even deﬁne the domain of distributions ⌦ optimized over, and store the solution w 2 C ⇥ A explicitly.

Original: But in general, such precise knowledge will not be available and is only estimable using past data (Assumption 1).
Revised: But in general, such precise knowledge will not be available and is only estimable using past data (Assumption 1).

Original: 3.4 An instance-optimal and computationally efﬁcient algorithm.
Revised: 3.4 An instance-optimal and computationally efﬁcient algorithm.

Original: In this section we provide Algorithm 3, which witnesses the guarantees of Theorem 2.4 for the general agnostic contextual bandit problem.
Revised: In this section we provide Algorithm 3, which witnesses the guarantees of Theorem 2.14 for the general agnostic contextual bandit problem.

Original: We now address the caveats of the previous approaches.
Revised: We now address the caveats of the previous approaches.

Original: Access to Ofﬂine Data.
Revised: Access to Ofﬂine Data.

Original: By Assumption 1, we have access to a large amount of sampled ofﬂine contexts D , where each c t 2 D is drawn IID from ⌫ .
Revised: By Assumption 1, we have access to a large amount of sampled ofﬂine contexts D , where each c t 2 D is drawn IID from ⌫ .

Original: Having access to D allows us to approximate
Revised: Having access to D allows us to approximate

Original: E c ⇠ ⌫ [ · ] with expectations over the empirical distribution E c ⇠ ⌫ D [ · ] , where ⌫ D is the uniform distribu- tion over historical data
Revised: E c ⇠ ⌫ [ · ] with expectations over the empirical distribution E c ⇠ ⌫ D [ · ] , where ⌫ D is the uniform distribu- tion over historical data

Original: D .
Revised: D .

Original: The number of ofﬂine contexts we need only scales logarithmically over the size of the policy set ⇧ , more speciﬁcally, poly( |A| , ✏  1 , log( | ⇧ | ) , log(1 /  )) .
Revised: The number of ofﬂine contexts we need only scales logarithmically over the size of the policy set ⇧ , more speciﬁcally, poly( |A| , ✏  1 , log( | ⇧ | ) , log(1 /  )) .

Original: We quantify the precise number of samples needed in Appendix D.2.
Revised: We quantify the precise number of samples needed in Appendix D.2.

Original: Input
Revised: Input

Original: [Equation]
Revised: [Equation]

Original: 1: Initialize
Revised: 1: Initialize

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: 2: for ` = 1 , 2 , · · · , d log 2 (1 / ✏ ) e do 3: ✏
Revised: 2: for ` = 1 , 2 , · · · , d log 2 (1 / ✏ ) e do 3: ✏

Original: [Equation]
Revised: [Equation]

Original: 
Revised:

Original: [Equation]
Revised: [Equation]

Original: ⇧
Revised: ⇧

Original: [Equation]
Revised: [Equation]

Original: Let n ` be the minimum value s.t.:
Revised: Let n ` be the minimum value s.t.:

Original: min
Revised: min

Original: [Equation]
Revised: [Equation]

Original: max
Revised: max

Original: [Equation]
Revised: [Equation]

Original: with solution w (
Revised: with solution w (

Original: [Equation]
Revised: [Equation]

Original: For each t 2 [ n ` ] , get c t ⇠ ⌫ , pull a t ⇠ p ( ` )
Revised: For each t 2 [ n ` ] , get c t ⇠ ⌫ , pull a t ⇠ p ( ` )

Original: c t , observe reward r t 6:
Revised: c t , observe reward r t 6:

Original: Compute O t =
Revised: Compute O t =

Original: A ( w ( ` ) )
Revised: A ( w ( ` ) )

Original:  1  ( c t , a t ) r t .
Revised:  1  ( c t , a t ) r t .

Original: For ⇡ , ⇡ 0 2 ⇧ ` b
Revised: For ⇡ , ⇡ 0 2 ⇧ ` b

Original: [Equation]
Revised: [Equation]

Original: Update ⇧
Revised: Update ⇧

Original: [Equation]
Revised: [Equation]

Original: b 
Revised: b

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: , ⇡
Revised: , ⇡

Original: [Equation]
Revised: [Equation]

Original: > ✏
Revised: > ✏

Original: [Equation]
Revised: [Equation]

Original: 9: end for
Revised: 9: end for

Original: Output: ⇧ ` +
Revised: Output: ⇧ ` +

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: : Initialize:
Revised: : Initialize:

Original: [Table]
Revised: [Table]

Original: arbitrarily
Revised: arbitrarily

Original: [Table]
Revised: [Table]

Original: : for
Revised: : for

Original: [Table]
Revised: [Table]

Original: =
Revised: =

Original: [Table] [Table]
Revised: [Table] [Table]

Original: , · · · , d log 2 (
Revised: , · · · , d log 2 (

Original: [Table]
Revised: [Table]

Original: / ✏ ) e do
Revised: / ✏ ) e do

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Equation]
Revised: [Equation]

Original:  / (
Revised:  / (

Original: [Table]
Revised: [Table]

Original: 2 | ⇧ | ) 4:
Revised: 2 | ⇧ | ) 4:

Original: Let n ` be the minimum value s.t.:
Revised: Let n ` be the minimum value s.t.:

Original: min
Revised: min

Original: [Equation]
Revised: [Equation]

Original: max
Revised: max

Original: [Equation]
Revised: [Equation]

Original: r
Revised: r

Original: [Equation]
Revised: [Equation]

Original:  ✏
Revised:  ✏

Original: [Equation]
Revised: [Equation]

Original: (2) with solution w ( ` ) 5:
Revised: (2) with solution w ( ` ) 5:

Original: For each t 2 [ n ` ] , get c t ⇠ ⌫ , pull a t ⇠ p ( ` )
Revised: For each t 2 [ n ` ] , get c t ⇠ ⌫ , pull a t ⇠ p ( ` )

Original: c t , observe reward r t 6:
Revised: c t , observe reward r t 6:

Original: Compute O t =
Revised: Compute O t =

Original: A ( w ( ` ) )
Revised: A ( w ( ` ) )

Original:  1  ( c t , a t ) r t .
Revised:  1  ( c t , a t ) r t .

Original: For each ⇡ 2 ⇧ , let b  ` ( ⇡ , b ⇡ `  1 )
Revised: For each ⇡ 2 ⇧ , let b  ` ( ⇡ , b ⇡ `  1 )

Original: =
Revised: =

Original: Cat ( { h  ⇡   b ⇡ `  1 , O i i } n ` i =1 ) .
Revised: Cat ( { h  ⇡   b ⇡ `  1 , O i i } n ` i =1 ) .

Original: Set b ⇡ ` := arg min ⇡ 2 ⇧ b  ` ( ⇡ , b ⇡ `  1 ) (3) 9: end for
Revised: Set b ⇡ ` := arg min ⇡ 2 ⇧ b  ` ( ⇡ , b ⇡ `  1 ) (3) 9: end for

Original: Output: b ⇡ l
Revised: Output: b ⇡ l

Original: Computing the design efﬁciently.
Revised: Computing the design efﬁciently.

Original: As described, the context space C may be inﬁnite so maintaining a distribution !
Revised: As described, the context space C may be inﬁnite so maintaining a distribution !

Original: 2 ⌦ ⇢  C ⇥ A is not possible.
Revised: 2 ⌦ ⇢  C ⇥ A is not possible.

Original: To overcome this issue, we consider the dual problem of equation (2).
Revised: To overcome this issue, we consider the dual problem of equation (2).

Original: We can remove the square root by noticing that 2 p xy =
Revised: We can remove the square root by noticing that 2 p xy =

Original: min  > 0  x + y  , and introducing an additional minimization over the variable  ⇡ , ⇡ 2 ⇧ .
Revised: min  > 0  x + y  , and introducing an additional minimization over the variable  ⇡ , ⇡ 2 ⇧ .

Original: Then, the dual problem becomes
Revised: Then, the dual problem becomes

Original: [Equation]
Revised: [Equation]

Original: Exchanging the order of the minimums on !
Revised: Exchanging the order of the minimums on !

Original: and  , somewhat surprisingly we have the close-form expression (Lemma E.6)
Revised: and  , somewhat surprisingly we have the close-form expression (Lemma E.6)

Original: [Equation]
Revised: [Equation]

Original: where for ⇡ 0 2 ⇧ , t ( c ) a ( ⇡ 0 ) 2 { 0 , 1 } | ⇧ | with [ t ( c ) a ( ⇡ 0 )]
Revised: where for ⇡ 0 2 ⇧ , t ( c ) a ( ⇡ 0 ) 2 { 0 , 1 } | ⇧ | with [ t ( c ) a ( ⇡ 0 )]

Original: ⇡ := 1 { ⇡ ( c ) = a, ⇡ 0 ( c ) 6 = a } + 1 { ⇡ ( c ) 6 = a, ⇡ 0 ( c ) = a } and [    ] ⇡ :=
Revised: ⇡ := 1 { ⇡ ( c ) = a, ⇡ 0 ( c ) 6 = a } + 1 { ⇡ ( c ) 6 = a, ⇡ 0 ( c ) = a } and [    ] ⇡ :=

Original:  ⇡  ⇡ .
Revised:  ⇡  ⇡ .

Original: Interestingly, this value is achieved at a sampling distribution ! , which is a non-linear function of  rather than a convex combination over policies (as in [3]).
Revised: Interestingly, this value is achieved at a sampling distribution ! , which is a non-linear function of  rather than a convex combination over policies (as in [3]).

Original: Because we have an expectation over contexts, this expectation can be replaced by an empirical estimate using historical data, thus avoiding any issues with an inﬁnite context space.
Revised: Because we have an expectation over contexts, this expectation can be replaced by an empirical estimate using historical data, thus avoiding any issues with an inﬁnite context space.

Original: The ﬁnal algorithm utilizing these observations found is in Algorithm 3.
Revised: The ﬁnal algorithm utilizing these observations found is in Algorithm 3.

Original: The main challenge is ﬁnding a solution to the design (5).
Revised: The main challenge is ﬁnding a solution to the design (5).

Original: First, we can reduce it to a saddle point problem over (  ,  ) by considering only a dyadic sequence of n 2 { 2 k : k 2 N } .
Revised: First, we can reduce it to a saddle point problem over (  ,  ) by considering only a dyadic sequence of n 2 { 2 k : k 2 N } .

Original: We use an alternating ascent/descent method, with the caveat that  lives in a simplex, and  in a box.
Revised: We use an alternating ascent/descent method, with the caveat that  lives in a simplex, and  in a box.

Original: Both spaces are deﬁned over a potentially inﬁnite set of policies ⇧ (in the worst case exponential in |C| ).
Revised: Both spaces are deﬁned over a potentially inﬁnite set of policies ⇧ (in the worst case exponential in |C| ).

Original: To handle this, we use the Frank-Wolfe (FW) method on  .
Revised: To handle this, we use the Frank-Wolfe (FW) method on  .

Original: Referring to the iterates of FW as  t , FW guarantees that the size of the support of  t in each iterate grows by at most 1 .
Revised: Referring to the iterates of FW as  t , FW guarantees that the size of the support of  t in each iterate grows by at most 1 .

Original: Thus, if initialized as a 1 -sparse vector, we only need to maintain a sparse  t in each iteration.
Revised: Thus, if initialized as a 1 -sparse vector, we only need to maintain a sparse  t in each iteration.

Original: Each iterate of Frank-Wolfe involves computing
Revised: Each iterate of FW computes

Original: [Equation]
Revised: [Equation]

Original: To do so, we show that we can appeal to a constrained argmax oracle ( AMO ) to run the Frank-Wolfe algorithm, a similar approach to [3].
Revised: To do so, we show that we can appeal to a constrained argmax oracle ( AMO ) to run the Frank-Wolfe algorithm, a similar approach to [3].

Original: To optimize over  we use a gradient descent procedure.
Revised: To optimize over  we use a gradient descent procedure.

Original: Wewill show that in each iterate, the support of  is contained in that of  , and we can quantify the number of steps of gradient descent needed to ﬁnd an ✏ -good solution.
Revised: We show that in each iterate, the support of  is contained in that of  , and we can quantify the number of steps of gradient descent needed to ﬁnd an ✏ -good solution.

Original: Though h l (  ,  , n ) might not be convex in  , we nevertheless are able to argue that it has a unique minima and that gradient descent converges to this minima.
Revised: Though h l (  ,  , n ) might not be convex in  , we nevertheless are able to argue that it has a unique minima and that gradient descent converges to this minima.

Original: We introduce our subroutine and further discuss the above claims in Appendix D.
Revised: We introduce our subroutine and further discuss the above claims in Appendix D.

Original: Regularized Estimator.
Revised: Regularized Estimator.

Original: While Algorithms 1 and 2 use a robust mean estimator as in equation (3), this estimator is impractical with a very large number of policies ⇧ .
Revised: While Algorithms 1 and 2 use a robust mean estimator as in equation (3), this estimator is impractical with a very large number of policies ⇧ .

Original: Instead, we use a regularized IPS estimator that can be computed using historical data and an argmax oracle.
Revised: Instead, we use a regularized IPS estimator that can be computed using historical data and an argmax oracle.

Original: [Table] [Equation] [Table] [Equation]
Revised: Algorithm

Original: [Table]
Revised: Input: policies

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised: 1: initiate b

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised:

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised:

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised:  max appropriately 2: for l = 1 , 2 , · · · do 3: ✏

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised: =

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised: Deﬁne

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table] Deﬁne
Revised: i

Original: [Equation]
Revised: [Equation]

Original: Let  l ,  l , n l = FW-GD ( ⇧ , |A| , b ⇡ l  1 , ✏ l ) .
Revised: Let  l ,  l , n l = FW-GD ( ⇧ , |A| , b ⇡ l  1 , ✏ l ) .

Original: These are the solutions to 
Revised: These are the solutions to n

Original: [Equation]
Revised: [Equation]

Original: For i 2 [ n ` ] get c i ⇠ ⌫ , pull a i ⇠ p ( ` ) c i where p ( ` ) c s ,a s / q
Revised: := min

Original: [Equation]
Revised: [Equation]

Original: ) , observe rewards r
Revised: N :

Original: [Equation]
Revised: [Equation]

Revised: For i 2 [ n ` ] get c i ⇠ ⌫ , pull a i ⇠ p ( ` ) c i where p ( ` ) c s ,a s / q (  l   l ) >

Revised: t ( c s ) a s ( b ⇡ l  1 ) , observe rewards r s 7:

Original: For each ⇡ 2 ⇧ , deﬁne the IPS estimator 
Revised: For each ⇡ 2 ⇧ , deﬁne the IPS estimator b

Original: [Equation]
Revised: [Equation]

Original:  8: set
Revised: b ⇡ l  1 ) = n l X s = r s p ( ` ) c s ,a s + [  l ] ⇡ ( 1 { b ⇡ l  1 ( c s ) = a s }  1 { ⇡ ( c s ) = a s } ) 8: set

Original: [Equation]{+b ⇡ l = arg min ⇡ 2 ⇧ b   l l ( ⇡ , b ⇡ l  1 ) + E c ⇠ ⌫ D ✓ [  l ] ⇡ p ( ` ) c,a + [  l ] ⇡ p ( ` ) c,a 0 ◆ 1 { b ⇡ l  1 ( c ) 
+}
Revised: {+b ⇡ l = arg min ⇡ 2 ⇧ b   l l ( ⇡ , b ⇡ l  1 ) + E c ⇠ ⌫ D ✓ [  l ] ⇡ p ( ` ) c,a + [  l ] ⇡ p ( ` ) c,a 0 ◆ 1 { b ⇡ l  1 ( c ) 
+}

Original:  9: end for
Revised: = ⇡ ( c ) }  + log(1 /  l ) [  l ] ⇡ n l (6) 9: end for

Original: Output: b ⇡ l
Revised: Output: b ⇡ l

Original: Algorithm 3 puts all the pieces together and Theorem 3.3 shows our main result.
Revised: Algorithm 3 puts all the pieces together and Theorem 3.3 shows our main result.

Original: Note that for exposition purposes, we have omitted some additional regularization terms in the optimization problems that have no effect on the sample complexity, but ensure ﬁnite-time convergence.
Revised: Note that for exposition purposes, we have omitted some additional regularization terms in the optimization problems that have no effect on the sample complexity, but ensure ﬁnite-time convergence.

Original: Appendix E shows the full algorithm and the proof.
Revised: Appendix E shows the full algorithm and the proof.

Original: In what follows, poly 1 ( |A| , ✏  1 , log(1 /  )) · log( | ⇧ | ) and poly 2 ( |A| , ✏  1 , log(1 /  ) , log( | ⇧ | )) are polynomials in their arguments.
Revised: In what follows, poly 1 ( |A| , ✏  1 , log(1 /  )) · log( | ⇧ | ) and poly 2 ( |A| , ✏  1 , log(1 /  ) , log( | ⇧ | )) are polynomials in their arguments.

Original: Theorem 3.3.
Revised: Theorem 3.3.

Original: Fix set of policies ⇧ , context distribution ⌫ and reward function r ( c, a ) 2 [0 , 1] .
Revised: Fix set of policies ⇧ , context distribution ⌫ and reward function r ( c, a ) 2 [0 , 1] .

Original: With probability at least 1   , provided a history D whose size exceeds poly 1 ( |A| , ✏  1 , log(1 /  )) · log( | ⇧ | ) , Algorithm 3 returns a policy b ⇡ satisfying V ( ⇡ ⇤ )
Revised: With probability at least 1   , provided a history D whose size exceeds poly 1 ( |A| , ✏  1 , log(1 /  )) · log( | ⇧ | ) , Algorithm 3 returns a policy b ⇡ satisfying V ( ⇡ ⇤ )

Original:  V ( b ⇡ )  ✏ in a number of samples not exceeding
Revised:  V ( b ⇡ )  ✏ in a number of samples not exceeding

Original: O ( ⇢ ⇤ , ✏ log( | ⇧ | log 2 (1 /  ✏ ) /  ) log 2 (1 /  ✏ ) where  ✏ := max { ✏ , min ⇡ 2 ⇧ V ( ⇡ ⇤ )
Revised: O ( ⇢ ⇤ , ✏ log( | ⇧ | log 2 (1 /  ✏ ) /  ) log 2 (1 /  ✏ ) where  ✏ := max { ✏ , min ⇡ 2 ⇧ V ( ⇡ ⇤ )

Original:  V ( ⇡ ) } .
Revised:  V ( ⇡ ) } .

Original: In addition,
Revised: In addition,

Original: Algorithm is computationally efﬁcient and requires at most poly 2 ( |A| , ✏  1 , log(1 /  ) , log( | ⇧ | )) calls to a constrained argmax oracle.
Revised: Algorithm is computationally efﬁcient and requires at most poly 2 ( |A| , ✏  1 , log(1 /  ) , log( | ⇧ | )) calls to a constrained argmax oracle.

Original: Conclusion.
Revised: Conclusion.

Original: This work provides the ﬁrst instance-dependent lower bounds for the ( ✏ ,  ) -PAC contextual bandit problem.
Revised: This work provides the ﬁrst instance-dependent lower bounds for the ( ✏ ,  ) -PAC contextual bandit problem.

Original: One limitation of this work is that our analysis of Algorithm 3 does not immediately extend to the realizable linear setting.
Revised: One limitation of this work is that our analysis of Algorithm 3 does not immediately extend to the realizable linear setting.

Original: That is, a computationally efﬁcient algorithm that achieves the same bound is not known to exist.
Revised: That is, a computationally efﬁcient algorithm that achieves the same bound is not known to exist.

Original: In the general agnostic settings discussed in this work, we proposed a computationally efﬁcient algorithm.
Revised: In the general agnostic settings discussed in this work, we proposed a computationally efﬁcient algorithm.

Original: A second limitation is the assumption that we have access to a large pool of ofﬂine data.
Revised: A second limitation is the assumption that we have access to a large pool of ofﬂine data.

Original: Because it seems necessary to plan with some information about the context distribution, it is not clear how one would completely remove such an assumption and achieve the same sample complexity bounds.
Revised: Because it seems necessary to plan with some information about the context distribution, it is not clear how one would completely remove such an assumption and achieve the same sample complexity bounds.

Original: As with any recommender system, there is the potential for unintended consequences from optimizing just a single metric.
Revised: As with any recommender system, there is the potential for unintended consequences from optimizing just a single metric.

Original: Moreover, other potential pitfalls can arise, such as negative feedback loops, if our assumptions fail to hold in real-world environments.
Revised: Moreover, other potential pitfalls can arise, such as negative feedback loops, if our assumptions fail to hold in real-world environments.

Original: Such consequences can be mitigated by tracking a diverse set of metrics.
Revised: Such consequences can be mitigated by tracking a diverse set of metrics.

Revised: Acknowledgement and Disclosure of Funding This work was supported, in part, by NSF award 1907907.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on how to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section.
Revised: [Yes] See Section.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] Please see our conclusion.
Revised: [Yes] Please see our conclusion.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] Please see our conclusion.
Revised: [Yes] Please see our conclusion.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] 3.
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [N/A]
Revised: [N/A]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether
Revised: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]