Original: The price of unfairness in linear bandits with biased feedback
Revised: The price of unfairness in linear bandits with biased feedback

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: In this paper, we study the problem of fair sequential decision making with biasedlinear bandit feedback.
Revised: In this paper, we study the problem of fair sequential decision making with biasedlinear bandit feedback.

Original: At each round, a player selects an action described by acovariate and by a sensitive attribute.
Revised: At each round, a player selects an action described by acovariate and by a sensitive attribute.

Original: The perceived reward is a linear combinationof the covariates of the chosen action, but the player only observes a biasedevaluation of this reward, depending on the sensitive attribute.
Revised: The perceived reward is a linear combinationof the covariates of the chosen action, but the player only observes a biasedevaluation of this reward, depending on the sensitive attribute.

Original: To characterize thedifﬁculty of this problem, we design a phased elimination algorithm that correctsthe unfair evaluations, and establish upper bounds on its regret.
Revised: To characterize thedifﬁculty of this problem, we design a phased elimination algorithm that correctsthe unfair evaluations, and establish upper bounds on its regret.

Original: We show that theworst-case regret is smaller than O (  1 / 3 ⇤ log( T ) 1 / 3 T 2 / 3 ) , where  ⇤ is an explicitgeometrical constant characterizing the difﬁculty of bias estimation.
Revised: We show that theworst-case regret is smaller than O (  1 / 3 ⇤ log( T ) 1 / 3 T 2 / 3 ) , where  ⇤ is an explicitgeometrical constant characterizing the difﬁculty of bias estimation.

Original: We provelower bounds on the worst-case regret for some sets of actions showing that thisrate is tight up to a possible sub-logarithmic factor.
Revised: We provelower bounds on the worst-case regret for some sets of actions showing that thisrate is tight up to a possible sub-logarithmic factor.

Original: We also derive gap-dependentupper bounds on the regret, and matching lower bounds for some problem instance.
Revised: We also derive gap-dependentupper bounds on the regret, and matching lower bounds for some problem instance.

Original: Interestingly, these results reveal a transition between a regime where the problemis as difﬁcult as its unbiased counterpart, and a regime where it can be much harder.
Revised: Interestingly, these results reveal a transition between a regime where the problemis as difﬁcult as its unbiased counterpart, and a regime where it can be much harder.

Original:  Introduction
Revised:  Introduction

Original: Artiﬁcial intelligence is increasingly used in a wide range of decision making scenarii with higher andhigher stakes, with application in online advertisement [28], credit [3], health care [10], education[25] and job interviews [32], in the hope of improving accuracy and efﬁciency.
Revised: Artiﬁcial intelligence is increasingly used in a wide range of decision making scenarii with higher andhigher stakes, with application in online advertisement [27], credit [3], health care [10], education[24] and job interviews [30], in the hope of improving accuracy and efﬁciency.

Original: Recent works haveshown that the decisions made by algorithms can be dangerously biased against certain categories ofpeople and have endeavored to mitigate this behavior [20, 13, 6, 24].
Revised: Recent works haveshown that the decisions made by algorithms can be dangerously biased against certain categories ofpeople and have endeavored to mitigate this behavior [19, 12, 6, 23].

Original: Studies have underlined thatthe main cause of algorithmic unfairness is the presence of bias in the training set [24] which led tothe development of methods aiming to guarantee the fairness of the algorithms.
Revised: Studies have underlined thatthe main cause of algorithmic unfairness is the presence of bias in the training set [23] which led tothe development of methods aiming to guarantee the fairness of the algorithms.

Original: This paper, in lineswith these works, addresses the problem of online decision making under biased feedback.
Revised: This paper, in lineswith these works, addresses the problem of online decision making under biased feedback.

Original: Linear bandits have become a very popular tool in online decision making problems, when sideinformation on the actions is available in the form of covariates.
Revised: Linear bandits have become a very popular tool in online decision making problems, when sideinformation on the actions is available in the form of covariates.

Original: In the present paper, we consider avariant of this problem, where the agent only has access to an unfair assessment of the action taken,that is systematically biased against a group of actions.
Revised: In the present paper, we consider avariant of this problem, where the agent only has access to an unfair assessment of the action taken,that is systematically biased against a group of actions.

Original: For example, examiners may be prejudicedagainst people from a minority group, and give them lower grades; similarly, algorithms trained onbiased data may produce unfair assessments of the credit risk of individuals belonging to a minoritygroup.
Revised: For example, examiners may be prejudicedagainst people from a minority group, and give them lower grades; similarly, algorithms trained onbiased data may produce unfair assessments of the credit risk of individuals belonging to a minoritygroup.

Original: Note that not correcting biased evaluation can have adverse effects for all parties: on the onehand, actions disadvantaged by the evaluation mechanism will be unfairly discriminated against;on the other hand, the agent may spend his budget on an unfairly advantaged action that is actuallysub-optimal.
Revised: Note that not correcting biased evaluation can have adverse effects for all parties: on the onehand, actions disadvantaged by the evaluation mechanism will be unfairly discriminated against;on the other hand, the agent may spend his budget on an unfairly advantaged action that is actuallysub-optimal.

Original: The problem of sequential decision making under biased feedback can be formalized asfollows
Revised: The problem of sequential decision making under biased feedback can be formalized asfollows

Original: Biased linear bandit problem A player is presented with a set of k distinct actions characterizedby covariates x 2 X ⇢ R d , and by known sensitive attributes z x 2 {  1 , 1 } indicating the group of 36 the action.
Revised: Biased linear bandit problem A player is presented with a set of k distinct actions characterizedby covariates x 2 X ⇢ R d , and by known sensitive attributes z x 2 {  1 , 1 } indicating the group of 36 the action.

Original: At each round t  T , the player chooses the action x t and receives an unobserved rewardx > t  ⇤ , where  ⇤ 2 R d is the regression parameter specifying the true value of the action.
Revised: At each round t  T , the player chooses the action x t and receives an unobserved rewardx > t  ⇤ , where  ⇤ 2 R d is the regression parameter specifying the true value of the action.

Original: The regretof the player is given by
Revised: The regretof the player is given by

Original: [Equation]
Revised: [Equation]

Original: i , where
Revised: i , where

Original: [Equation]
Revised: [Equation]

Original: By contrast to the classical linear bandit, the player does not observe a noisy version of the unbiased reward x > t  ⇤ .
Revised: By contrast to the classical linear bandit, the player does not observe a noisy version of the unbiased reward x > t  ⇤ .

Original: Instead, she observes an unfair evaluation y t of the value of the action x > t  ⇤ , given by the following biased linear model:
Revised: Instead, she observes an unfair evaluation y t of the value of the action x > t  ⇤ , given by the following biased linear model:

Original: [Equation]
Revised: [Equation]

Original: where ⇠ t i.i.d ⇠ N (0 , 1) is a noise term.
Revised: where ⇠ t i.i.d ⇠ N (0 , 1) is a noise term.

Original: The evaluation are systematically biased against a certaingroup: this unequal treatment of the groups is captured by the bias parameter !
Revised: The evaluation are systematically biased against a certaingroup: this unequal treatment of the groups is captured by the bias parameter !

Original: ⇤ 2 R .
Revised: ⇤ 2 R .

Original: Preliminary discussion The biased linear bandit is a variant of the linear bandit.
Revised: Preliminary discussion The biased linear bandit is a variant of the linear bandit.

Original: By contrast,in the classical linear bandit model, the agent observes a noisy version of the reward.
Revised: By contrast,in the classical linear bandit model, the agent observes a noisy version of the reward.

Original: Obviously,applying directly an algorithm designed for linear bandit to biased linear bandits without correctingthe evaluations would lead to a linear regret if the evaluation mechanism is prejudiced against thegroup of the best action in terms of reward, and if the best action in terms of feedback belongsto the advantaged group.
Revised: Obviously,applying directly an algorithm designed for linear bandit to biased linear bandits without correctingthe evaluations would lead to a linear regret if the evaluation mechanism is prejudiced against thegroup of the best action in terms of reward, and if the best action in terms of feedback belongsto the advantaged group.

Original: To avoid this pitfall, one must estimate the bias in order to correct theevaluations.
Revised: To avoid this pitfall, one must estimate the bias in order to correct theevaluations.

Original: This implies a change in the exploration-exploitation trade-off, as exploration becomesmore expensive.
Revised: This implies a change in the exploration-exploitation trade-off, as exploration becomesmore expensive.

Original: Indeed, in classical bandit problems, one can compare the rewards of two actionsby repeatedly sampling them - or, to put it differently, one can ﬁnd the best action by sampling onlythose actions that seem optimal.
Revised: Indeed, in classical bandit problems, one can compare the rewards of two actionsby repeatedly sampling them - or, to put it differently, one can ﬁnd the best action by sampling onlythose actions that seem optimal.

Original: This does not hold in the biased linear bandit: if, at some point,the set of potentially optimal actions contains representatives from both groups, and does not span
Revised: This does not hold in the biased linear bandit: if, at some point,the set of potentially optimal actions contains representatives from both groups, and does not span

Original: R d , one is forced to sample sub-optimal actions to estimate the bias and improve the estimation ofthe unbiased rewards.
Revised: R d , one is forced to sample sub-optimal actions to estimate the bias and improve the estimation ofthe unbiased rewards.

Original: For this reason, classical algorithm for linear bandit that only sample actionsconsidered as potentially optimal, such as OFUL [1] or Phase Elimination [22], can suffer linearregret.
Revised: For this reason, classical algorithm for linear bandit that only sample actionsconsidered as potentially optimal, such as OFUL [1] or Phase Elimination [21], can suffer linearregret.

Original: This underlines the necessity to ensure sufﬁcient estimation of the bias parameter, even whenit implies sampling sub-optimal actions.
Revised: This underlines the necessity to ensure sufﬁcient estimation of the bias parameter, even whenit implies sampling sub-optimal actions.

Original: 1.1 Related work
Revised: 1.1 Related work

Original: Fairness in bandit problems has mostly been studied from the perspective of fair budget allocationbetween actions.
Revised: Fairness in bandit problems has mostly been studied from the perspective of fair budget allocationbetween actions.

Original: This problem is motivated by the fact that classical bandit algorithms select suboptimal actions only a vanishing fraction of the time, which may be undesirable in many situations.
Revised: This problem is motivated by the fact that classical bandit algorithms select suboptimal actions only a vanishing fraction of the time, which may be undesirable in many situations.

Original: To mitigate this problem and guarantee diversity in the actions selected, some papers [4, 26, 8, 14, 39]have proposed new algorithms ensuring fairness of the selection frequency of each action.
Revised: To mitigate this problem and guarantee diversity in the actions selected, some papers [4, 25, 8, 13, 34]have proposed new algorithms ensuring fairness of the selection frequency of each action.

Original: Theframework studied in this paper is different: we consider here that the mechanism for observing therewards is unfair, and we aim at correcting it in order to maximize a (fair) true cumulative reward.
Revised: Theframework studied in this paper is different: we consider here that the mechanism for observing therewards is unfair, and we aim at correcting it in order to maximize a (fair) true cumulative reward.

Original: The biased linear model has been studied in the batch setting in [7], where the authors investigatethe optimal trade-off between minimax risk and Demographic Parity.
Revised: The biased linear model has been studied in the batch setting in [7], where the authors investigatethe optimal trade-off between minimax risk and Demographic Parity.

Original: Detection of systematic bias,interpreted as a treatment effect, has been investigated in a batch setting in [16].
Revised: Detection of systematic bias,interpreted as a treatment effect, has been investigated in a batch setting in [15].

Original: In [2], the authorsconsider a similar model, with unobserved sensitive attribute z and known bias parameter !
Revised: In [2], the authorsconsider a similar model, with unobserved sensitive attribute z and known bias parameter !

Original: ⇤ , underadditional assumption that the sensitive attribute z is independent from the covariate x .
Revised: ⇤ , underadditional assumption that the sensitive attribute z is independent from the covariate x .

Original: By contrast,we show that bias estimation is one of the main difﬁculties of the biased bandit problem.
Revised: By contrast,we show that bias estimation is one of the main difﬁculties of the biased bandit problem.

Original: The linear bandit with biased feedback can be viewed as a stochastic partial monitoring game.
Revised: The linear bandit with biased feedback can be viewed as a stochastic partial monitoring game.

Original: Withthe terminology of partial monitoring, the biased problem considered in the present paper is globallyobservable but not locally observable: in this case, the optimal worst-case regret rate typicallyincreases as ˜ O
Revised: Withthe terminology of partial monitoring, the biased problem considered in the present paper is globallyobservable but not locally observable: in this case, the optimal worst-case regret rate typicallyincreases as ˜ O

Original: ( T 2 / 3 ) .
Revised: ( T 2 / 3 ) .

Original: This regret rate is for example achieved in the related problem of partial linearmonitoring with linear feedback and linear reward using an Information Directed Sampling algorithm[18].
Revised: This regret rate is for example achieved in the related problem of partial linearmonitoring with linear feedback and linear reward using an Information Directed Sampling algorithm[17].

Original: However, the dependence of the regret on the geometry of the action set and on the dimension dremains in most cases an open question [23, 5, 18].
Revised: However, the dependence of the regret on the geometry of the action set and on the dimension dremains in most cases an open question [22, 5, 17].

Original: In this paper, we characterize the geometry of thebiased linear bandit problem, and we investigate dependence of the regret on the gaps.
Revised: In this paper, we characterize the geometry of thebiased linear bandit problem, and we investigate dependence of the regret on the gaps.

Original: 1.2 Contribution and outline
Revised: 1.2 Contribution and outline

Original: In this paper, we introduce the linear bandit problem with biased feedback.
Revised: In this paper, we introduce the linear bandit problem with biased feedback.

Original: We design a new algorithmbased on optimal design for this problem.
Revised: We design a new algorithmbased on optimal design for this problem.

Original: We derive an upper bound on the worst case regret ofthis algorithm of order  1 / 3 ⇤ log( T ) 1 / 3 T 2 / 3 for large T , where  ⇤ is an explicit constant dependingon the geometry of the action set.
Revised: We derive an upper bound on the worst case regret ofthis algorithm of order  1 / 3 ⇤ log( T ) 1 / 3 T 2 / 3 for large T , where  ⇤ is an explicit constant dependingon the geometry of the action set.

Original: We provide matching lower bounds on some problem instances,showing that the constant  ⇤ characterizes the difﬁculty of the action set.
Revised: We provide matching lower bounds on some problem instances,showing that the constant  ⇤ characterizes the difﬁculty of the action set.

Original: Note that this regret ishigher than the classical rates of order ˜ O ( dT 1 / 2 ) obtained for d -dimensional linear bandits: thisincrease corresponds to the price to pay for debiasing the unfair evaluations.
Revised: Note that this regret ishigher than the classical rates of order ˜ O ( dT 1 / 2 ) obtained for d -dimensional linear bandits: thisincrease corresponds to the price to pay for debiasing the unfair evaluations.

Original: We also characterize the gap-depend regret, showing that it is of order ( d /  min
Revised: We also characterize the gap-depend regret, showing that it is of order ( d /  min

Original: _  (  ) /  2 6 = ) log( T ) ,where  min is the minimum gap,  6 = is the gap between the best actions of the two groups, and  (  )corresponds to the minimum regret to pay for estimating the bias with a given variance.
Revised: _  (  ) /  2 6 = ) log( T ) ,where  min is the minimum gap,  6 = is the gap between the best actions of the two groups, and  (  )corresponds to the minimum regret to pay for estimating the bias with a given variance.

Original: This boundunderlines the relative difﬁculties of the d -dimensional linear bandit and of the bias estimation.
Revised: This boundunderlines the relative difﬁculties of the d -dimensional linear bandit and of the bias estimation.

Original: When 91 d /  min   (  ) /  2 6 = , i.e. when one group contains all near-optimal actions, the difﬁculty is dominatedby that of the corresponding linear bandit problem.
Revised: When 91 d /  min   (  ) /  2 6 = , i.e. when one group contains all near-optimal actions, the difﬁculty is dominatedby that of the corresponding linear bandit problem.

Original: When both groups contain near-optimal actions,and d /  min   (  ) /  2 6 = , the regret corresponds to the price of debiasing the rewards.
Revised: When both groups contain near-optimal actions,and d /  min   (  ) /  2 6 = , the regret corresponds to the price of debiasing the rewards.

Original: The rest of the paper is organized as follows.
Revised: The rest of the paper is organized as follows.

Original: In Section 2, we present the F AIR P HASED E LIMINATION algorithm:
Revised: In Section 2, we present the F AIR P HASED E LIMINATION algorithm:

Original: we ﬁrst discuss parameter estimation in Section 2.1, before presenting a sketchof the algorithm in Section 2.2 (a detailed version of this algorithm is provided in Appendix B).
Revised: we ﬁrst discuss parameter estimation in Section 2.1, before presenting a sketchof the algorithm in Section 2.2 (a detailed version of this algorithm is provided in Appendix B).

Original: Then, in Section 3, we establish an upper bound on its worst-case regret.
Revised: Then, in Section 3, we establish an upper bound on its worst-case regret.

Original: In Section 4, we derive agap-dependent upper bound on the regret of our algorithm.
Revised: In Section 4, we derive agap-dependent upper bound on the regret of our algorithm.

Original: In Section 5, we establish lower boundson some action sets for both the worst-case and the gap-dependent regret, showing that these rates aresharp respectively up to a sub-logarithmic factor and an absolute multiplicative constant.
Revised: In Section 5, we establish lower boundson some action sets for both the worst-case and the gap-dependent regret, showing that these rates aresharp respectively up to a sub-logarithmic factor and an absolute multiplicative constant.

Original: Additionaldiscussions on the geometry of bias estimation are postponed to Appendix A.
Revised: Additionaldiscussions on the geometry of bias estimation are postponed to Appendix A.

Original: 1.3 Notations and additional assumptions
Revised: 1.3 Notations and additional assumptions

Original: We assume that all covariates x 2 X are distinct, which implies that the group z x of action x is welldeﬁned.
Revised: We assume that all covariates x 2 X are distinct, which implies that the group z x of action x is welldeﬁned.

Original: We also assume that no group is empty, that the set { ⇣ xz x ⌘ : x 2 X} spans R d +1 (whichguarantees identiﬁability of the parameters), and that the rewards are bounded: max x 2 X | x >  ⇤ |  1 .
Revised: We also assume that no group is empty, that the set { ⇣ xz x ⌘ : x 2 X} spans R d +1 (whichguarantees identiﬁability of the parameters), and that the rewards are bounded: max x 2 X | x >  ⇤ |  1 .

Original: When necessary, we underline the dependence of the regret on the parameter ✓ by denoting it R ✓ T .
Revised: When necessary, we underline the dependence of the regret on the parameter ✓ by denoting it R ✓ T .

Original: We denote by a x = ⇣ xz x ⌘ the vector describing an action and its group, by ✓ ⇤ =
Revised: We denote by a x = ⇣ xz x ⌘ the vector describing an action and its group, by ✓ ⇤ =

Original: ⇣ 
Revised: ⇣

Original: [Equation]
Revised: [Equation]

Original: ⌘ 2 R d +1 theunknown parameter, and by A = { a x : x 2 X} the set of actions and of corresponding sensitiveattributes.
Revised: ⌘ 2 R d +1 theunknown parameter, and by A = { a x : x 2 X} the set of actions and of corresponding sensitiveattributes.

Original: We denote by 
Revised: We denote by

Original: =
Revised: =

Original: (  x ) x 2 X the vector of gaps
Revised: (  x ) x 2 X the vector of gaps

Original:  x = max x 0 2 X ( x 0  x ) >
Revised:  x = max x 0 2 X ( x 0  x ) >

Original:  ⇤ , andby C ( X ) =
Revised:  ⇤ , andby C ( X ) =

Original:   2 R d : 8 x 2 X , | x >  |  1 the set of admissible parameters.
Revised:   2 R d : 8 x 2 X , | x >  |  1 the set of admissible parameters.

Original: Note that for allx 2 C ( X ) ,  x  2 .
Revised: Note that for allx 2 C ( X ) ,  x  2 .

Original: For i  d +1 , let e i be the i -th vector of the canonical basis of R d +1 , and for anymatrix M , let M + be a generalized inverse of M .
Revised: For i  d +1 , let e i be the i -th vector of the canonical basis of R d +1 , and for anymatrix M , let M + be a generalized inverse of M .

Original: We denote by P X the set of probability measures on
Revised: We denote by P X the set of probability measures on

Original: X , and M X = { µ : X 7!
Revised: X , and M X = { µ : X 7!

Original: R + } .
Revised: R + } .

Original: For any µ 2 P X or µ 2 M X , we denote V ( µ ) =
Revised: For any µ 2 P X or µ 2 M X , we denote V ( µ ) =

Original: P x 2 X µ ( x ) a x a > xthe covariance matrix corresponding to this allocation.
Revised: P x 2 X µ ( x ) a x a > xthe covariance matrix corresponding to this allocation.

Original: Moreover, for u 2 R d +1 (resp.
Revised: Moreover, for u 2 R d +1 (resp.

Original: U 2 R d +1 ),we denote by P X u (resp.
Revised: U 2 R d +1 ),we denote by P X u (resp.

Original: M X u ) the measures µ in P X (resp.
Revised: M X u ) the measures µ in P X (resp.

Original: in M X ) such that u 2 Range( V ( µ )) .
Revised: in M X ) such that u 2 Range( V ( µ )) .

Original: For U ⇢ R d +1 , we denote by P XU (resp.
Revised: For U ⇢ R d +1 , we denote by P XU (resp.

Original: M XU )
Revised: M XU )

Original: the measures µ such that µ 2 P X u (resp.
Revised: the measures µ such that µ 2 P X u (resp.

Original: M X u ) for all 117 u 2 U .
Revised: M X u ) for all 117 u 2 U .

Original:  Fair Phased Elimination algorithm
Revised:  Fair Phased Elimination algorithm

Original: The Fair Phased Elimination algorithm belongs to the category of sequential elimination algorithms.
Revised: The Fair Phased Elimination algorithm belongs to the category of sequential elimination algorithms.

Original: Classical sequential elimination algorithms typically proceed by phases, indexed by l = 1 , 2 , . . . .
Revised: Classical sequential elimination algorithms typically proceed by phases, indexed by l = 1 , 2 , . . . .

Original: Atphase l , these algorithms consider a set of potentially optimal actions X l .
Revised: Atphase l , these algorithms consider a set of potentially optimal actions X l .

Original: The rewards of all actionsx 2 X l are then estimated with a given precision O ( ✏ l ) , typically chosen as ✏ l = 2 2  l , by samplingactions in X l .
Revised: The rewards of all actionsx 2 X l are then estimated with a given precision O ( ✏ l ) , typically chosen as ✏ l = 2 2  l , by samplingactions in X l .

Original: Actions sub-optimal by a gap larger than the precision level are then removed from theset X l +1 of potentially optimal actions for the phase l + 1 .
Revised: Actions sub-optimal by a gap larger than the precision level are then removed from theset X l +1 of potentially optimal actions for the phase l + 1 .

Original: As underlined previously, sequential elimination algorithms may suffer linear regret in the biasedlinear bandit problem if actions allowing to estimate the bias are discarded by the algorithm beforethe best group is identiﬁed.
Revised: As underlined previously, sequential elimination algorithms may suffer linear regret in the biasedlinear bandit problem if actions allowing to estimate the bias are discarded by the algorithm beforethe best group is identiﬁed.

Original: To mitigate this problem, we ﬁrst estimate the biased evaluations of the potentially optimal actions, using ordinary least squares estimation.
Revised: To mitigate this problem, we ﬁrst estimate the biased evaluations of the potentially optimal actions, using ordinary least squares estimation.

Original: We then debias the estimationsusing an estimator for the bias relying on independent observations, which may be obtained bysampling sub-optimal actions.
Revised: We then debias the estimationsusing an estimator for the bias relying on independent observations, which may be obtained bysampling sub-optimal actions.

Original: Before presenting the algorithm, let us discuss the estimation of theevaluations and of the bias parameter.
Revised: Before presenting the algorithm, let us discuss the estimation of theevaluations and of the bias parameter.

Original: 2.1 Optimal design for parameter estimation in the biased linear bandit
Revised: 2.1 Optimal design for parameter estimation in the biased linear bandit

Original: G-optimal design for biased evaluation estimation As in the Phased Elimination algorithm
Revised: G-optimal design for biased evaluation estimation As in the Phased Elimination algorithm

Original: [22],we rely on G-optimal design to estimate the biased evaluations a > x ✓ ⇤ with small error uniformly overa set of actions X l .
Revised: [21],we rely on G-optimal design to estimate the biased evaluations a > x ✓ ⇤ with small error uniformly overa set of actions X l .

Original: More precisely, for a given set of potentially optimal actions X l , we compute the
Revised: More precisely, for a given set of potentially optimal actions X l , we compute the

Original: G-optimal design solution to the problem
Revised: G-optimal design solution to the problem

Original: [Equation]
Revised: [Equation]

Original: This can be done using polynomial-time algorithms, relying for example on interior points method[37], or on mixed integer second-order cone programming [34].
Revised: This can be done using polynomial-time algorithms, relying for example on interior points method[32], or on mixed integer second-order cone programming [31].

Original: The celebrated General Equivalencetheorem of Kiefer [17] and Pukelsheim [30] states that the value of Equation (2) is bounded by d + 1 .
Revised: The celebrated General Equivalencetheorem of Kiefer [16] and Pukelsheim [29] states that the value of Equation (2) is bounded by d + 1 .

Original: Let ⇡ ⇤ denote any design solution to the G-optimal design problem (2), and let b ✓ denote the ordinaryleast square estimator obtained by sampling each action x 2 X l exactly d n ⇡ ⇤ ( x ) e times for a given 142 n > 0 .
Revised: Let ⇡ ⇤ denote any design solution to the G-optimal design problem (2), and let b ✓ denote the ordinaryleast square estimator obtained by sampling each action x 2 X l exactly d n ⇡ ⇤ ( x ) e times for a given 142 n > 0 .

Original: Then, for all x 2 X l , the General Equivalence theorem implies that the variance of theestimate a > x b ✓ is smaller than ( d +1) / n .
Revised: Then, for all x 2 X l , the General Equivalence theorem implies that the variance of theestimate a > x b ✓ is smaller than ( d +1) / n .

Original: Moreover, the G-optimal design ⇡ ⇤ can be chosen so that it issupported by at most ( d +1)
Revised: Moreover, the G-optimal design ⇡ ⇤ can be chosen so that it issupported by at most ( d +1)

Original: ( d +2) / 2 points, so the total number of samples is at most n + ( d +1)
Revised: ( d +2) / 2 points, so the total number of samples is at most n + ( d +1)

Original: ( d +2) / 2 . -optimal design for bias evaluation In this paragraph, we introduce the  -optimal design, whichis discussed in greater depth in Appendix A.6.
Revised: ( d +2) / 2 . -optimal design for bias evaluation In this paragraph, we introduce the  -optimal design,which is discussed in greater depth in Appendix A.5.

Original: To estimate the bias parameter !
Revised: To estimate the bias parameter !

Original: ⇤ , we use the estimatorb !
Revised: ⇤ , we use theestimator

Original: =
Revised: [Equation]

Original: e > d + b ✓ , where b ✓ is the ordinary least square estimator for the full parameter ✓ ⇤ .
Revised: b ✓ , where b ✓ is the ordinary least square estimator for the full parameter ✓ ⇤ .

Original: Now, if wesample each action x 2 X exactly µ ( x ) time, the variance of b ! is equal to e > d +1 V ( µ ) + e d +1 .
Revised: Now,if we sample each action x 2 X exactly µ ( x ) time, the variance of b ! is equal to e > d +1 V ( µ ) + e d +1 .

Original: Giventhe vector of gaps  , the design µ minimizing the regret of this exploration phase, while ensuringthat the variance of b ! is smaller than 1 , is solution of the problemminimize
Revised: Given the vector of gaps  , the design µ minimizing the regret of this exploration phase, whileensuring that the variance of b ! is smaller than 1 , is solution of the problemminimize

Original: [Equation]
Revised: [Equation]

Original: such that
Revised: such that

Original: [Equation]
Revised: [Equation]

Original: Let us denote µ  a minimizer of (3), and  (  ) =
Revised: Let us denote µ  a minimizer of (3), and  (  ) =

Revised: P

Original: [Equation]
Revised: [Equation]

Original:  X µ  ( x )  x . Lemma 9 in Appendix Aexplains how to compute the design µ  in polynomial time by adapting tools from c -optimal design.
Revised: 2 X µ  ( x )  x . Lemma 9 in Appendix Aexplains how to compute the design µ  in polynomial time by adapting tools from c -optimal design.

Original: This lemma also shows that the support of µ  can be chosen to be of cardinality at most d + 1 .
Revised: This lemma also shows that the support of µ  can be chosen to be of cardinality at most d + 1 .

Original: Then,choosing each action exactly d nµ  ( x ) e times for a given n > 0 allows us to estimate the bias withvariance lower than n  1 and a regret no larger than n  (  ) + 2( d + 1) .
Revised: Then,choosing each action exactly d nµ  ( x ) e times for a given n > 0 allows us to estimate the bias withvariance lower than n  1 and a regret no larger than n  (  ) + 2( d + 1) .

Original: Obviously, we do not knowthe gap vector  beforehand, so we must estimate it as we go.
Revised: Obviously, we do not knowthe gap vector  beforehand, so we must estimate it as we go.

Original: 2.2 Outline of the Fair Phased Elimination algorithm
Revised: 2.2 Outline of the Fair Phased Elimination algorithm

Original: The Fair Phased Elimination algorithm, sketched in Algorithm 3, relies on the following key ideas.
Revised: The Fair Phased Elimination algorithm, sketched in Algorithm 3, relies on the following key ideas.

Original: First, note that within a group, the order of the true rewards and of the biased evaluations are thesame.
Revised: First, note that within a group, the order of the true rewards and of the biased evaluations are thesame.

Original: Hence, within a group, we can use classical algorithms for linear bandits to choose theactions and estimate the biased evaluations with a controlled within-group regret: this is done using
Revised: Hence, within a group, we can use classical algorithms for linear bandits to choose theactions and estimate the biased evaluations with a controlled within-group regret: this is done using

Original: G -exploration and elimination .
Revised: G -exploration and elimination .

Original: Second, to compare actions belonging to different groups, weindependently estimate the bias parameter !
Revised: Second, to compare actions belonging to different groups, weindependently estimate the bias parameter !

Original: ⇤ , using  -exploration and elimination .
Revised: ⇤ , using  -exploration and elimination .

Original: Finally, weunderline that bias estimation may require to sample very sub-optimal actions.
Revised: Finally, weunderline that bias estimation may require to sample very sub-optimal actions.

Original: Therefore, it can beoverly costly to estimate the bias up to the precision level required to identify the best group.
Revised: Therefore, it can beoverly costly to estimate the bias up to the precision level required to identify the best group.

Original: Toprevent this, we use a stopping criterion .
Revised: Toprevent this, we use a stopping criteria .

Original: G-exploration and elimination At each phase l = 1 , 2 , ... , we keep two sets of potentially optimalactions belonging to the groups +1 and  1 , denoted respectively X (+1) l and X (  1) l .
Revised: G-exploration and elimination At each phase l = 1 , 2 , ... , we keep two sets of potentially optimalactions belonging to the groups +1 and  1 , denoted respectively X (+1) l and X (  1) l .

Original: If we have notidentiﬁed the group containing the best action, we run a G-E XP -E LIM routine 1 on each set X ( z ) lfor z = 1 and z =  1 .
Revised: If we have notidentiﬁed the group containing the best action, we run a G-E XP -E LIM routine 1 on each set X ( z ) lfor z = 1 and z =  1 .

Original: This routine samples actions according to a rounded G-optimal design on
Revised: This routine samples actions according to a rounded G-optimal design on

Original: X ( z ) l , with a total number of observations chosen so that the biased evaluations of all actions in X ( z ) l are known with an error at most ✏ l .
Revised: X ( z ) l , with a total number of observations chosen so that the biased evaluations of all actions in X ( z ) l are known with an error at most ✏ l .

Original: The set X ( z ) l +1 is obtained by removing from X ( z ) l actions whoseestimated evaluations are sub-optimal by a gap larger than 3 ✏ l , compared to the empirical best actionin the group.
Revised: The set X ( z ) l +1 is obtained by removing from X ( z ) l actions whoseestimated evaluations are sub-optimal by a gap larger than 3 ✏ l , compared to the empirical best actionin the group.

Original: This allows to ensure that only actions sub-optimal by a gap O ( ✏ l ) remain in X ( z ) l +1 , andto estimate the gap vector  with a precision sufﬁcient for  -optimal estimation.
Revised: This allows to ensure that only actions sub-optimal by a gap O ( ✏ l ) remain in X ( z ) l +1 , andto estimate the gap vector  with a precision sufﬁcient for  -optimal estimation.

Original: If the group containing the best action has been identiﬁed, we discard the other group, and run a
Revised: If the group containing the best action has been identiﬁed, we discard the other group, and run a

Original: G-E XP -E LIM routine 1 on the set of potentially optimal actions in this group.
Revised: G-E XP -E LIM routine 1 on the set of potentially optimal actions in this group.

Original: [Equation]
Revised: [Equation]

Original:  -exploration and elimination If the group of the best action has not been found before phasel , we run the  -E XP -E LIM routine 2.
Revised:  -exploration and elimination If the group of the best action has not been found before phasel , we run the  -E XP -E LIM routine 2.

Original: More precisely, relying on a previous estimate b  l of the gapvector  , we compute the b  l -optimal design b µ .
Revised: More precisely, relying on a previous estimate b  l of the gapvector  , we compute the b  l -optimal design b µ .

Original: We then estimate the bias using actions sampledaccording to a rounded version of this design, with a total number of observations chosen so that theerror of bias estimation is smaller than ✏ l , and use it to debias the reward estimation.
Revised: We then estimate the bias using actions sampledaccording to a rounded version of this design, with a total number of observations chosen so that theerror of bias estimation is smaller than ✏ l , and use it to debias the reward estimation.

Original: If the debiasedevaluation of the best action of each group are separated by a gap larger than 4 ✏ l , we consider thatthe best group is the one containing the empirical best action in terms of biased evaluation, and wediscard the other group.
Revised: If the debiasedevaluation of the best action of each group are separated by a gap larger than 4 ✏ l , we consider thatthe best group is the one containing the empirical best action in terms of biased evaluation, and wediscard the other group.

Original: If we cannot ﬁnd the best group, we rely on estimates of the bias and of the biased evaluationsobtained during the previous round to update the estimate of the gap vector b  l +1 .
Revised: If we cannot ﬁnd the best group, we rely on estimates of the bias and of the biased evaluationsobtained during the previous round to update the estimate of the gap vector b  l +1 .

Original: b  -optimal bias
Revised: -optimal bias

Original: Stopping criterion
Revised: Stopping criteria

Original: As underlined previously, the  -E XP -E LIM routine samples actions that canbe very sub-optimal.
Revised: As underlined previously, the  -E XP -E LIM routine samples actions that can bevery sub-optimal.

Original: As a consequence, when the gap between the best two actions of each group issmall, ﬁnding the best group can be overly costly in terms of regret.
Revised: As a consequence, when the gap between the best two actions of each group issmall, ﬁnding the best group can be overly costly in terms of regret.

Original: To prevent this, if the best grouphas not been found at stage l fulﬁlling ✏ l    ( b  l )log( T ) / T  1 / 3 , the bias estimation is stopped and theempirical best action in X (1) l
Revised: To prevent this, if the best grouphas not been found at stage l fulﬁlling ✏ l    ( b  l )log( T ) / T  1 / 3 , the bias estimation is stopped and theempirical best action in X (1) l

Original: +1 [ X (  1) l +1 is sampled for the remaining time (see Algorithm 3)
Revised: +1 [ X (  1) l +1 is sampled for the remaining time (see Algorithm 3)

Original:  Upper bound on the worst-case regret of F AIR P HASED E LIMINATION
Revised:  Upper bound on the worst-case regret of F AIR P HASED E LIMINATION

Original: The regret of the F AIR P HASED E LIMINATION depends on the difﬁculty of estimating the bias parameter, captured by  (  ) .
Revised: The regret of the F AIR P HASED E LIMINATION depends on the difﬁculty of estimating the bias parameter, captured by  (  ) .

Original: Lemma 7 in Appendix A.6 shows that for all parameter  ⇤ 2 X ,  (  ) is upper bounded by 2  ⇤ , where  ⇤ is the minimal variance of the bias estimator given by
Revised: Lemma 7 in Appendix A.5 shows that for all parameter  ⇤ 2 X ,  (  ) is upper bounded by 2  ⇤ , where  ⇤ is the minimal variance of the bias estimator given by

Original: [Equation]
Revised: [Equation]

Original: The following theorem provides a bound on the worst case regret depending on  ⇤ .
Revised: The following theorem provides a bound on the worst case regret depending on  ⇤ .

Original: Proofs arepostponed to Appendix C.2. 1: input:  T X k =
Revised: Proofs arepostponed to Appendix C.1.

Original: X

Original: , ✏ l = 2 2  l for 2: initialize ){ x : z x } , b l 4: while do do 6: ⇣b
Revised: b

Original: [Equation]
Revised: [Equation]

Original: 7: if Z

Original: = {  1 , + then 8: if then estimation 9: Sample best 10: else 11: ⇣ Z l +
Revised: +

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Theorem 1.
Revised: Theorem 1.

Original: For the choice  = T  1 , there exists two numerical constants C, C 0 > 0 such that thefollowing bound on the regret of the F AIR P HASED E LIMINATION algorithm 4 holds[Equation] for
Revised: For the choice  = T  1 , there exists two numerical constants C, C 0 > 0 such that thefollowing bound on the regret of the F AIR P HASED E LIMINATION algorithm 4 holds

Original: [Equation]
Revised: [Equation]

Original: In Section 5.1, we show that the upper bound obtained in Theorem 1 is sharp in some settings, up tothe sub-logarithmic factor log( T ) 1 / 3 .
Revised: In Section 5.1, we show that the upper bound obtained in Theorem 1 is sharp in some settings, up tothe sub-logarithmic factor log( T ) 1 / 3 .

Original: Theorem 1 shows that the worst-case regret of the Fair Phased Elimination algorithm asymptotically grows as C  1 / 3 ⇤ T 2 / 3 log ( T ) 1 / 3 .
Revised: Theorem 1 shows that the worst-case regret of the Fair Phased Elimination algorithm asymptotically grows as C  1 / 3 ⇤ T 2 / 3 log ( T ) 1 / 3 .

Original: This worst-case regret rate is higher than the typical rate
Revised: This worst-case regret rate is higher than the typical rate

Original: Cd log( T ) T 1 / 2 obtained under unbiased feedback on the rewards (see, e.g., [1]).
Revised: Cd log( T ) T 1 / 2 obtained under unbiased feedback on the rewards (see, e.g., [1]).

Original: This increase inthe regret corresponds to the cost of learning from unfair evaluations.
Revised: This increase inthe regret corresponds to the cost of learning from unfair evaluations.

Original: It is due to the fact that thealgorithm may need to sample actions that are sub-optimal in order to estimate the bias parameter.
Revised: It is due to the fact that thealgorithm may need to sample actions that are sub-optimal in order to estimate the bias parameter.

Original: Note that this rate e
Revised: Note that this rate e

Original: O ( T 2 / 3 ) is typical for globally observable bandit problems with partial linearmonitoring, and can be obtained by applying results established in [18] for in the partial linearmonitoring setting to the biased linear bandit problem.
Revised: O ( T 2 / 3 ) is typical for globally observable bandit problems with partial linearmonitoring, and can be obtained by applying results established in [17] for in the partial linearmonitoring setting to the biased linear bandit problem.

Original: By contrast to previous results, Theorem 1 characterizes precisely the dependence of the worst-caseregret on the geometry of the action set.
Revised: By contrast to previous results, Theorem 1 characterizes precisely the dependence of the worst-caseregret on the geometry of the action set.

Original: The relevant constant  ⇤ is the minimal variance forestimating the bias, which appears when considering the related c -optimal design problem.
Revised: The relevant constant  ⇤ is the minimal variance forestimating the bias, which appears when considering the related c -optimal design problem.

Original: While theconnection between G-optimal design and the linear bandit problem has already been exploited, itis to the best of our knowledge the ﬁrst time that c -optimal design is related to a partial monitoringproblem.
Revised: While theconnection between G-optimal design and the linear bandit problem has already been exploited, itis to the best of our knowledge the ﬁrst time that c -optimal design is related to a partial monitoringproblem.

Original: The constant  ⇤ corresponds to the minimum number of samples required for estimating the bias witha variance equal to 1 (up to rounding issues).
Revised: The constant  ⇤ corresponds to the minimum number of samples required for estimating the bias witha variance equal to 1 (up to rounding issues).

Original: Intuitively, if the actions are very correlated with theirsensitive attributes, more samples will be needed to estimate the bias with the same precision.
Revised: Intuitively, if the actions are very correlated with theirsensitive attributes, more samples will be needed to estimate the bias with the same precision.

Original: Thissituation corresponds to cases where  ⇤ is large, and leads to a higher regret.
Revised: Thissituation corresponds to cases where  ⇤ is large, and leads to a higher regret.

Original: Lemma 1, illustrated in
Revised: Lemma 1, illustrated in

Original: Figure 2, relates  ⇤ to the margin between the two groups of actions.
Revised: Figure 1, relates  ⇤ to the margin between the two groups of actions.

Original: Lemma 1.
Revised: Lemma 1.

Original:  ⇤ is the largest constant   0  such that, there exists an hyperplane Hcontaining 220 zero and separating the  two groups, and such that, the margin to
Revised:  ⇤ is the largest constant   0 , such that, there exists an hyperplane H separating the 220 two groups, and such that the margin to

Original: H is at least p   1 / p  +1 times themaximum distance of all points to the hyperplane (see Figure 2).
Revised: H is at least p   1 / p  +1 times the maximum distance of allpoints to the hyperplane (see Figure 1).

Original: When no such hyperplane exists,then  ⇤ = 1 .
Revised: When no such hyperplane exists, then  ⇤ = 1 .

Original: Interestingly, Lemma 1 underlines that under reasonable assumptions, the constant  ⇤ may not dependon the ambient dimension d , and it can even be equal to 1 .
Revised: Interestingly, Lemma 1 underlines that under reasonable assumptions, the constant  ⇤ may not dependon the ambient dimension d , and it can even be equal to 1 .

Original: By contrast, the previous bounds obtainedfor an Information Directed Sampling algorithm are of order ↵ 1 / 3 d 1 / 2 T 2 / 3 log( kT ) 1 / 2 , where ↵ is ameasure of the complexity of the action set called the worst-case alignment constant.
Revised: By contrast, the previous bounds obtainedfor an Information Directed Sampling algorithm are of order ↵ 1 / 3 d 1 / 2 T 2 / 3 log( kT ) 1 / 2 , where ↵ is ameasure of the complexity of the action set called the worst-case alignment constant.

Original: Lemma 6 in
Revised: Lemma 6 in

Original: [Figure]

Original: Appendix A shows that ↵ is equivalent to the minimal variance of the bias estimator  ⇤ .
Revised: Appendix A shows that ↵ is equivalent to the minimal variance of the bias estimator  ⇤ .

Original: Hence, ourbound improves over previous results by a factor d 1 / 2 log( T ) 1 / 6 (log( kT ) / log( T )) 1 / 2 . 
Revised: Hence, ourbound improves over previous results by a factor d 1 / 2 log( T ) 1 / 6 (log( kT ) / log( T )) 1 / 2 . [Figure]

Original: The gaps are not involved in the deﬁnition of the minimal variance of bias estimation  ⇤ .
Revised: The gaps are not involved in the deﬁnition of the minimal variance of bias estimation  ⇤ .

Original: The reader may have expected to get, instead of  ⇤ , the minimax regret for estimating the bias
Revised: The reader may have expected to get, instead of  ⇤ , the minimax regret for estimating the bias

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: , such that µ 2 M X e d +1 and e > d +1 V ( µ ) + e d +1  1 .
Revised: such that µ [Figure] µ [Figure]

Revised: +

Original:  Next lemma shows that  ⇤ and e  are in equivalent up to a factor 2.
Revised: [Figure]  [Figure] Next lemma shows that  ⇤ and e  are in equivalent up to a factor 2.

Original: We refer the interested readerto Appendix A, where further discussions on the geometry of bias estimation are postponed, due tospace constraints.
Revised: We refer the interested readerto Appendix A, where further discussions on the geometry of bias estimation are postponed due tospace constraints.

Original: Lemma 2.
Revised: Lemma 2.

Original: [Equation]
Revised: [Equation]

Original:  Upper bound on the gap-depend regret of F AIR P HASED E LIMINATION
Revised:  Upper bound on the gap-depend regret of F AIR P HASED E LIMINATION

Original: In this section, we provide an upper bound on the worst-case regret that depends on the gap betweenthe two best actions, and on the gap between the best actions of the two groups.
Revised: In this section, we provide an upper bound on the worst-case regret that depends on the gap betweenthe two best actions, and on the gap between the best actions of the two groups.

Original: Compared toinstance-dependent bounds, established in the linear bandit problem in [21, 19], gap-dependentbounds characterize the dependence of the regret on a small number of parameters.
Revised: Compared toinstance-dependent bounds, established in the linear bandit problem in [20, 18], gap-dependentbounds characterize the dependence of the regret on a small number of parameters.

Original: They are typicallyless sharp than instance-dependent bounds, but allow to better highlight the inﬂuence of the parameterson the difﬁculty of the problem.
Revised: They are typicallyless sharp than instance-dependent bounds, but allow to better highlight the inﬂuence of the parameterson the difﬁculty of the problem.

Original: The bound established in the following theorem relates the difﬁcultyof the biased linear bandit to that of bias estimation, and to that of the corresponding d -dimensionallinear bandit.
Revised: The bound established in the following theorem relates the difﬁcultyof the biased linear bandit to that of bias estimation, and to that of the corresponding d -dimensionallinear bandit.

Original: Proofs are postponed to Appendix C.2.
Revised: Proofs are postponed to Appendix C.1.

Original: Theorem 2.
Revised: Theorem 2.

Original: Assume that x ⇤ 2 argmax x 2 X x >  ⇤ is unique.
Revised: Assume that x ⇤ 2 argmax x 2 X x >  ⇤ is unique.

Original: Then, there exists two numericalconstants C, C 0 > 0 such that, for the choice  = T  1 , the following bound on the regret of the F AIR
Revised: Then, there exists two numericalconstants C, C 0 > 0 such that, for the choice  = T  1 , the following bound on the regret of the F AIR

Original: P HASED E LIMINATION algorithm 4 holds
Revised: P HASED E LIMINATION algorithm 4 holds

Original: [Equation]
Revised: [Equation]

Original: where  min =
Revised: where  min =

Original: min
Revised: min

Original: x 2 X\ x
Revised: x 2 X\ x

Original: ⇤  x ,  6 = =
Revised: ⇤  x ,  6 = =

Original: min x 2 X : z x =  z x
Revised: min x 2 X : z x =  z x

Original: ⇤  x , and " T = (  ⇤ log( T ) / T ) 1 / 3 .
Revised: ⇤  x , and " T = (  ⇤ log( T ) / T ) 1 / 3 .

Original: The term d /  min _  (  _  6 = _ " T ) /  2 6 = highlights the two sources of difﬁculty of the problem.
Revised: The term d /  min _  (  _  6 = _ " T ) /  2 6 = highlights the two sources of difﬁculty of the problem.

Original: On the onehand, the term d /  min is unavoidable: even if the algorithm knew beforehand the group containing thebest action, it would still need to play a game of d -dimensional linear bandits in this group, and suffer,in the worst-case, the corresponding gap-dependent regret [1].
Revised: On the onehand, the term d /  min is unavoidable: even if the algorithm knew beforehand the group containing thebest action, it would still need to play a game of d -dimensional linear bandits in this group, and suffer,in the worst-case, the corresponding gap-dependent regret [1].

Original: Note that lower bounds on gap-dependregret of classical linear bandits follow from considering a setting with one near-optimal action withgap  min in each of the d dimensions.
Revised: Note that lower bounds on gap-dependregret of classical linear bandits follow from considering a setting with one near-optimal action withgap  min in each of the d dimensions.

Original: Then, any algorithm needs to explore each dimension upto   2min log( T ) times in order to ﬁnd the best action, but can do so by choosing the near-optimal actions, thus having a regret   1min log( T ) in each direction.
Revised: Then, any algorithm needs to explore each dimension upto   2min log( T ) times in order to ﬁnd the best action, but can do so by choosing the near-optimalactions, thus having a regret   1min log( T ) in each direction.

Original: By contrast, the term  (  _  6 = _ " T ) /  2 6 = ischaracteristic of the biased linear bandit problem: it is due to the fact that the algorithm may needto sample very sub-optimal actions in order to ﬁnd the group containing the best action.
Revised: By contrast, the term  (  _  6 = _ " T ) /  2 6 = ischaracteristic of the biased linear bandit problem: it is due to the fact that the algorithm may need to sample very sub-optimal actions in order to ﬁnd the group containing the best action.

Original: Indeed, toidentify this group, one must estimate the bias with a precision  6 = , i.e. sample sub-optimal actionswith average regret  (  ) approximately   2 6 = log( T ) times.
Revised: Indeed, toidentify this group, one must estimate the bias with a precision  6 = , i.e. sample sub-optimal actionswith average regret  (  ) approximately   2 6 = log( T ) times.

Original: When d /  min   (  _  6 = _ " T ) /  2 6 = , the regret corresponds to the regret of this bias estimation phase.
Revised: When d /  min   (  _  6 = _ " T ) /  2 6 = , the regret corresponds to the regret of this bias estimation phase.

Original: Inother words, when both groups contain near-optimal actions, the difﬁculty of the problem is dominatedby the price to pay for debiasing the unfair evaluations.
Revised: Inother words, when both groups contain near-optimal actions, the difﬁculty of the problem is dominatedby the price to pay for debiasing the unfair evaluations.

Original: Interestingly, when d /  min >  (  _  6 = _ " T ) /  2 6 = ,the difﬁculty of the linear bandit with systematic bias is dominated by that of the classical d -linearbandit.
Revised: Interestingly, when d /  min >  (  _  6 = _ " T ) /  2 6 = ,the difﬁculty of the linear bandit with systematic bias is dominated by that of the classical d -linearbandit.

Original: In this case, the algorithm is able to ﬁnd the group containing the best action, and the problemreduces to a linear bandit in dimension d .
Revised: In this case, the algorithm is able to ﬁnd the group containing the best action, and the problemreduces to a linear bandit in dimension d .

Original: Thus, the linear bandit with systematic bias is a non trivialexample of a globally observable game that can be locally observable around the best action.
Revised: Thus, the linear bandit with systematic bias is a non trivialexample of a globally observable game that can be locally observable around the best action.

Original: Finally, we underline that the magnitude of the bias does not appear in the regret: intuitively, nomatter its magnitude, the algorithm always need to estimate it up to the same precision (of order  6 = )in order to ﬁnd the best group and to be optimal in terms of gap-depend regret.
Revised: Finally, we underline that the magnitude of the bias does not appear in the regret: intuitively, nomatter its magnitude, the algorithm always need to estimate it up to the same precision (of order  6 = )in order to ﬁnd the best group and to be optimal in terms of gap-depend regret.

Original: This indicates that ouralgorithm is robust against important discriminations in the evaluation mechanism.
Revised: This indicates that ouralgorithm is robust against important discriminations in the evaluation mechanism.

Original:  Lower bounds on the regret
Revised:  Lower bounds on the regret

Original: In this section, we derive lower bounds on the worst-case regret and the gap-dependent regret thatrespectively match the upper bounds established in Theorems 1 and 2 up to sub-logarithmic factorsor numerical constants.
Revised: In this section, we derive lower bounds on the worst-case regret and the gap-dependent regret thatrespectively match the upper bounds established in Theorems 1 and 2 up to sub-logarithmic factorsor numerical constants.

Original: 5.1 Lower bound on the worst-case regret
Revised: 5.1 Lower bound on the worst-case regret

Original: Theorems 1 and 2 underline the dependence of the regret on the geometry of the action set.
Revised: Theorems 1 and 2 underline the dependence of the regret on the geometry of the action set.

Original: Beforestating our result, we begin by introducing the notion of  ⇤ -correlated action set.
Revised: Beforestating our result, we begin by introducing the notion of  ⇤ -correlated action set.

Original: Deﬁnition 1 (  ⇤ -correlated action set) .
Revised: Deﬁnition 1 (  ⇤ -correlated action set) .

Original: For  ⇤  1 , a set of actions A is  ⇤ -correlated
Revised: For  ⇤  1 , a set of actions A is  ⇤ -correlated

Original: if A 2 A  ⇤ ,d , 277 where
Revised: if A 2 A  ⇤ ,d , 276 where

Original: [Equation]

Original: ⇤

Original: [Equation]
Revised: [Equation]

Original: is the set of actions sets such that the minimal variance of the bias estimator is larger than  ⇤ .
Revised: is the set of actions sets such that the minimal variance of the bias estimator is larger than  ⇤ .

Original: In the following theorem, we establish a lower bound on the regret valid for all  ⇤  1 by designing 280  ⇤ -correlated sets of actions A 2 A  ⇤ ,d , and obtaining lower bounds on the regret of any algorithm 281 on these sets of actions.
Revised: In the following theorem, we establish a lower bound on the regret valid for all  ⇤  1 by designing 279  ⇤ -correlated sets of actions A 2 A  ⇤ ,d , and obtaining lower bounds on the regret of any algorithm 280 on these sets of actions.

Original: Theorem 3.
Revised: Theorem 3.

Original: Let  ⇤  1 , d  2 and T  4 3  ⇤ .
Revised: Let  ⇤  1 , d  2 and T  4 3  ⇤ .

Original: There exists an action set A 2 A  ⇤ ,d such that forany algorithm, there exists a bandit problem with parameter ✓ T 2 R d +1 such that the regret of thisalgorithm on the problem characterized by ✓ T satisﬁes R ✓ T T   1 / 3 ⇤ T 2 / 3 / 8 e .
Revised: There exists an action set A 2 A  ⇤ ,d such that forany algorithm, there exists a bandit problem with parameter ✓ T 2 R d +1 such that the regret of thisalgorithm on the problem characterized by ✓ T satisﬁes R ✓ T T   1 / 3 ⇤ T 2 / 3 / 8 e .

Original: Previous lower bounds on the regret of linear bandits with partial monitoring, established in [18], statethat the regret must be at least c A T 2 / 3 for some parameter ✓ T 2 R d +1 , where c A > 0 is a constantdepending (not explicitly) on A .
Revised: Previous lower bounds on the regret of linear bandits with partial monitoring, established in [17], statethat the regret must be at least c A T 2 / 3 for some parameter ✓ T 2 R d +1 , where c A > 0 is a constantdepending (not explicitly) on A .

Original: By contrast, Theorem 3 provides an explicit characterization of thedependence of the regret rate on the geometry of the problem, which matches the upper bound of
Revised: By contrast, Theorem 3 provides an explicit characterization of thedependence of the regret rate on the geometry of the problem, which matches the upper bound of

Original: Theorem 1 up to a sub-logarithmic factor.
Revised: Theorem 1 up to a sub-logarithmic factor.

Original: Note that the assumption d  2 is necessary here: if d = 1 ,there are at most two potentially optimal actions (namely, max { x : x 2 X} and min { x : x 2 X} ).
Revised: Note that the assumption d  2 is necessary here: if d = 1 ,there are at most two potentially optimal actions (namely, max { x : x 2 X} and min { x : x 2 X} ).

Original: Then, the problem becomes locally observable, and regret of order e
Revised: Then, the problem becomes locally observable, and regret of order e

Original: O ( T 1 / 2 ) can be achieved [18].
Revised: O ( T 1 / 2 ) can be achieved [17].

Original: 5.2 Lower bound on the gap-dependent regret
Revised: 5.2 Lower bound on the gap-dependent regret

Original: We now present a lower bound on the gap-dependent regret.
Revised: We now present a lower bound on the gap-dependent regret.

Original: More precisely, for given values of min and  

Revised: More precisely, for given values of min and

Original: = , we establish a lower bound on the worst case regret among parameters ✓ verifying min  min x 2 X\ x
Revised: = , we establish a lower bound on the worst case regret among parameters ✓ verifying min  min x 2 X\ x

Original: ⇤  x , and  6 =  min x 2 X : z x =  z x ⇤  x .
Revised: ⇤  x , and  6 =  min x 2 X : z x =  z x ⇤  x .

Original: Before stating formally the result, let us deﬁne the corresponding parameter set.
Revised: Before stating formally the result, let usdeﬁne the corresponding parameter set.

Original: For an action set A 2 A  ⇤ ,d , and for (  min ,  6 = ) 2 (0 , 1) 2297 such that  min   6 = , we denote 
Revised: For an action set A 2 A  ⇤ ,d , and for (  min ,  6 = ) 2 (0 , 1) 2 such that  min   6 = , we denote [Equation] that

Original: [Equation]
Revised: [Equation]

Original: the set of parameters with minimum gap  min , and minimum between-group-gap  6 = .
Revised: the set of parameters with minimum gap  min , and minimum between-group-gap  6 = .

Original: The upper bounds established in Theorem 2 underline the dependence of the gap-dependent regreton the minimal regret  (  ) for estimating the bias.
Revised: The upper bounds established in Theorem 2 underline the dependence of the gap-dependent regreton the minimal regret  (  ) for estimating the bias.

Original: Before stating our results, we deﬁne a classof problems ⇥
Revised: Before stating our results, we deﬁne a classof problems ⇥

Original: A  min ,  6 = ,  such that  (  )   .
Revised: A  min ,  6 = ,  such that  (  )   .

Original: For a parameter  2 C ( X ) , let us denote  (  ) x
Revised: For a parameter  2 C ( X ) , let us denote  (  ) x

Original: =max x 0 2 X ( x 0  x ) >  , and  (  ) =
Revised: =max x 0 2 X ( x 0  x ) >  , and  (  ) =

Original: (  (  ) x ) x 2 X .
Revised: (  (  ) x ) x 2 X .

Original: Moreover, for a given set A , let us denote
Revised: Moreover, for a given set A , let us denote

Original: [Equation]
Revised: [Equation]

Original: Theorem 4.
Revised: Theorem 4.

Original: For all   2 and all d  4 , there exists a set of actions A 2 R d +1 such that for all 304 (  min ,  6 = ) 2 (0 , 1 / 8 ) 2 with  min   6 = ,lim inf T ! sup [Equation]
Revised: For all   2 and all d  4 , there exists a set of actions A 2 R d +1 such that for all 303 (  min ,  6 = ) 2 (0 , 1 / 8 ) 2 with  min   6 = ,

Original: [Equation]
Revised: [Equation]

Original: Theorem 4 shows that for some action sets A , the gap-depend regret of the F AIR P
Revised: Theorem 4 shows that for some action sets A , the gap-depend regret of the F AIR P

Original: HASED E LIMINATION algorithm is asymptotically optimal up to a numerical constant.
Revised: HASED E LIMINATION algorithm is asymptotically optimal up to a numerical constant.

Original: Note that the assumption d  4 is necessary in our proof to design an action set A such that Equation (4) holds forall  min ,  6 = 2 (0 , 1 / 8 ) .
Revised: Note that the assumption d  4 is necessary in our proof to design an action set A such that Equation (4) holds forall  min ,  6 = 2 (0 , 1 / 8 ) .

Original: On the other hand, as discussed in Appendix C.6, for d  2 , for all min ,  6 = 2 (0 , 1 / 8) , we can show that there exists action sets A and ✓ 2 ⇥ A  min ,  6 = such thatthe lower bound in Equation (4) still holds, by considering separately the cases d /  min >  /  2 [-
-] = and 311 d /  min   /  2 6 = .
Revised: On the other hand, as discussed in Appendix C.4, for d  2 , for all min ,  6 = 2 (0 , 1 / 8) , we can show that there exists action sets A and ✓ 2 ⇥ A  min ,  6 = such thatthe lower bound in Equation (4) still holds, by considering separately the cases d /  min >  /  2 [-
-]6 = and 310 d /  min   /  2 6 = .

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we addressed the problem of online decision making under biased bandit feedback.
Revised: In this paper, we addressed the problem of online decision making under biased bandit feedback.

Original: We designed a new algorithm based on  - and G-optimal design, and obtained worst-case andgap-dependent upper bounds on its regret.
Revised: We designed a new algorithm based on  - and G-optimal design, and obtained worst-case andgap-dependent upper bounds on its regret.

Original: We obtained lower bounds on the regret for some probleminstances showing that these rates are tight up to sub-logarithmic factors in some settings.
Revised: We obtained lower bounds on the regret for some probleminstances showing that these rates are tight up to sub-logarithmic factors in some settings.

Original: These rateshighlight two behaviors: on the one hand, the worst case rate O (  1 / 3 ⇤ log( T ) 1 / 3 T 2 / 3 ) highlights thecost induced by the biased feedback, and the need to select sub-optimal actions in order to debias it.
Revised: These rateshighlight two behaviors: on the one hand, the worst case rate O (  1 / 3 ⇤ log( T ) 1 / 3 T 2 / 3 ) highlights thecost induced by the biased feedback, and the need to select sub-optimal actions in order to debias it.

Original: On the other hand, the gap-dependent bound shows that for some instance, the problem can be locallyobservable around the best action: then, the difﬁculty of the problem is dominated by the difﬁculty ofthe corresponding linear bandit problem, and is no more difﬁcult than this problem.
Revised: On the other hand, the gap-dependent bound shows that for some instance, the problem can be locallyobservable around the best action: then, the difﬁculty of the problem is dominated by the difﬁculty ofthe corresponding linear bandit problem, and is no more difﬁcult than this problem.

Original: When this isnot the case, the regret scales as  (  )   2 6 = log( T ) , where  6 = is the gap between the best actions ofthe two groups, and  (  ) is the minimum regret for estimating the bias with a given precision.
Revised: When this isnot the case, the regret scales as  (  )   2 6 = log( T ) , where  6 = is the gap between the best actions ofthe two groups, and  (  ) is the minimum regret for estimating the bias with a given precision.

Original: Thiswork paves the way for studying other bandit models with unfair feedback, considering for examplecontinuous, multi-dimensional sensitive attributes.
Revised: Thiswork paves the way for studying other bandit models with unfair feedback, considering for examplecontinuous, multi-dimensional sensitive attributes.

Original: Broader impact
Revised: Broader impact

Original: In this work, we propose a model for sequential decision making under biased feedback.
Revised: In this work, we propose a model for sequential decision making under biased feedback.

Original: Our goalis primarily to provide a good strategy for sequential learning in an unfair environment, and tocharacterize the difﬁculty of this problem by bounding the regret.
Revised: Our goalis primarily to provide a good strategy for sequential learning in an unfair environment, and tocharacterize the difﬁculty of this problem by bounding the regret.

Original: On the one hand, our results revealthat maximizing the fair rewards instead of unfair evaluations may be more difﬁcult in terms of regret,which may discourage practitioners from correcting unfair feedbacks.
Revised: On the one hand, our results revealthat maximizing the fair rewards instead of unfair evaluations may be more difﬁcult in terms of regret,which may discourage practitioners from correcting unfair feedbacks.

Original: On the other hand, we believethat as fairness is an important long-term key objective, rather than discouraging the practitioner, itwill inform them to better plan the adaptation of their methods toward this aim.
Revised: On the other hand, we believethat as fairness is an important long-term key objective, rather than discouraging the practitioner, itwill inform them to better plan the adaptation of their method toward this aim.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes](b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes](c)
Revised: [Yes]

Original:  Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]2.
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes](b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes]3.
Revised: [Yes]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [N/A]
Revised: [N/A](b)

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you include the total amount of compute and the type of resources used (e
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?
Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Original: [N/A]
Revised: [N/A]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]