Original: Adaptive Data Debiasing through Bounded Exploration
Revised: Adaptive Data Debiasing through Bounded Exploration

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Biases in existing datasets used to train algorithmic decision rules can raise ethicaland economic concerns due to the resulting disparate treatment of different groups.
Revised: Biases in existing datasets used to train algorithmic decision rules can raise ethical and economic concerns due to the resulting disparate treatment of different groups.

Original: We propose an algorithm for sequentially debiasing such datasets through adaptiveand bounded exploration in a classiﬁcation problem with costly and censoredfeedback.
Revised: We propose an algorithm for sequentially debiasing such datasets through adaptive and bounded exploration in a classiﬁcation problem with costly and censored feedback.

Original: Exploration in this context means that at times, and to a judiciouslychosen extent, the decision maker deviates from its (current) loss-minimizing rule,and instead accepts some individuals that would otherwise be rejected, so as toreduce statistical data biases.
Revised: Exploration in this context means that at times, and to a judiciouslychosen extent, the decision maker deviates from its (current) loss-minimizing rule, and instead accepts some individuals that would otherwise be rejected, so as to reduce statistical data biases.

Original: Our proposed algorithm includes parameters that canbe used to balance between the ultimate goal of removing data biases – which willin turn lead to more accurate and fair decisions, and the exploration risks incurredto achieve this goal.
Revised: Our proposed algorithm includes parameters that can be used to balance between the ultimate goal of removing data biases – which will in turn lead to more accurate and fair decisions, and the exploration risks incurred to achieve this goal.

Original: We analytically show that such exploration can help debiasdata in certain distributions.
Revised: We analytically show that such exploration can help debias data in certain distributions.

Original: We further investigate how fairness criteria can workin conjunction with our data debiasing algorithm.
Revised: We further investigate how fairness criteria can work in conjunction with our data debiasing algorithm.

Original: We illustrate the performance ofour algorithm using experiments on synthetic and real-world datasets.
Revised: We illustrate the performance of our algorithm using experiments on synthetic and real-world datasets.

Original:  Introduction
Revised:  Introduction

Original: Data-driven algorithmic decision making is being adopted widely to aid humans’ decisions, inapplications ranging from loan approvals to determining recidivism in courts.
Revised: Data-driven algorithmic decision making is being adopted widely to aid humans’ decisions, in applications ranging from loan approvals to determining recidivism in courts.

Original: Despite their ability toprocess vast amounts of data and make accurate predictions, these algorithms can also exhibit andamplify existing social biases (e g., [8, 17, 24]).
Revised: Despite their ability to process vast amounts of data and make accurate predictions, these algorithms can also exhibit and amplify existing social biases (e g., [11, 23, 33]).

Original: There are at least two possible sources of unfairnessin algorithmic decision rules: (statistical) biases in the data itself and (social) biases arising fromthe algorithm’s decisions [21].
Revised: There are at least two possible sources of unfairness in algorithmic decision rules: (data) biases in the training datasets and (prediction) biases arising from the algorithm’s decisions [29].

Original: The latter problem has been receiving increasing attention and isoften addressed by imposing fairness constraints on the algorithm.
Revised: The latter problem has been receiving increasing attention and is often addressed by imposing fairness constraints on the algorithm.

Original: In contrast, in this paper, we areprimarily focused on the former problem of statistical biases in the training data itself.
Revised: In contrast, in this paper, we are primarily focused on the former problem of statistical biases in the training data itself.

Original: The datasets used for training machine learning algorithms might not accurately represent the agentsthey make decisions on, due to, e.g., historical biases in decision making and feature selection, orchanges in the populations’ characteristics or participation rates since the data was initially collected treatment  or disadvantaged groups
Revised: The datasets used for training machine learning algorithms might not accurately represent the agents they make decisions on, due to, e.g., historical biases in decision making and feature selection, or changes in the populations’ characteristics or participation rates since the data was initially collected. Such data biases in turn can result in disparate treatment of underrepresented or disadvantaged groups; i.e., data bias can cause prediction/model bias, as also veriﬁed by recent work [18, 40, 43, 25].

Original: Motivated  algorithm  accurate (and fair) decisions also  unbiased estimates of the characteristics of agents interacting with it
Revised: Motivated by this, we focus on data biases, and propose an algorithm which, while attempting to make accurate (and fair) decisions, also aims to collect data in a way that helps it recover unbiased estimates of the characteristics of agents interacting with it.

Original: In particular we study a classiﬁcation problems with censored and costly feedback .
Revised: In particular, we study a classiﬁcation problems with censored and costly feedback .

Original: Censored feedback means that the decision maker only observes the true qualiﬁcation state of those individuals  admits (e g., a bank will only observe whether an individual defaults on or repays a loan if the loan is extended in the ﬁrst place; an employer only assesses the performance of applicants it hires).
Revised: Censored feedback means that the decision maker only observes the true qualiﬁcation state of those individuals it admits (e.g., a bank will only observe whether an individual defaults on or repays a loan if the loan is extended in the ﬁrst place; an employer only assesses the performance of applicants it hires).

Original: In such settings, any mismatch between the available training data and the true population may grow over time  to adaptive sampling bias once a decision rule is adopted  on the current training data the algorithm’s decisions  impact new data collected in the future that only agents  the requirements set by the current decision rule will be admitted going forward.
Revised: In such settings, any mismatch between the available training data and the true population may grow over time due to adaptive sampling bias: once a decision rule is adopted based on the current training 36th Conference on Neural Information Processing Systems (NeurIPS 2022). data, the algorithm’s decisions will impact new data collected in the future, in that only agents passing the requirements set by the current decision rule will be admitted going forward.

Original: In response, thedecision maker may attempt to collect more data from the population; however, such data collection iscostly (e.g., in the previous examples, may require extending loans to/hiring unqualiﬁed individuals).
Revised: In response, the decision maker may attempt to collect more data from the population; however, such data collection is costly (e.g., in the previous examples, may require extending loans to/hiring unqualiﬁed individuals).

Original: Given these challenges, we present an active debiasing algorithm with bounded exploration : ouralgorithm admits some agents that would otherwise be rejected (i.e., it explores), yet adaptively andjudiciously limits the extent and frequency of this exploration.
Revised: Given these challenges, we present an active debiasing algorithm with bounded exploration : our algorithm admits some agents that would otherwise be rejected (i.e., it explores), yet adaptively and judiciously limits the extent and frequency of this exploration.

Original: Formally, consider a population of agents with features x , true qualiﬁcation/labels y , and groupmemberships g based on their demographic features.
Revised: Formally, consider a population of agents with features x , true qualiﬁcation/labels y , and group memberships g based on their demographic features.

Original: To design a (fair) algorithm that can minimizeclassiﬁcation loss, the decision maker (implicitly) relies on estimates ˆ f yg,t ( x ) of the feature-label distribution of agents from group g , obtained from the current training dataset H t
Revised: To design a (fair) algorithm that can minimize classiﬁcation loss, the decision maker (implicitly) relies on estimates ˆ f yg,t ( x ) of the feature-label distribution of agents from group g , obtained from the current training dataset H t

Original: = { ( x n , y n , g n ) } N t n =1 .
Revised: = { ( x n , y n , g n ) } N t n =1 .

Original: However, the resulting assumed distribution ˆ f yg,t ( x ) may be different from the true underlying distribution f yg ( x ) ; this is the statistical data bias issue we focus on herein.
Revised: However, the resulting assumed distribution ˆ f yg,t ( x ) may be different from the true underlying distribution f yg ( x ) ; this is the statistical data bias issue we focus on herein.

Original: Speciﬁcally, we considerdistribution shifts between the estimates and the true distributions (Assumption 1).
Revised: Speciﬁcally, we consider distribution shifts between the estimates and the true distributions (Assumption 1).

Original: Our algorithm.
Revised: Our algorithm.

Original: We propose an active debiasing algorithm (Algorithm 1), which actively adjustsits decisions with the goal of ensuring unbiased estimates of the underlying distributions f yg ( x ) overtime.
Revised: We propose an active debiasing algorithm (Algorithm 1), which actively adjusts its decisions with the goal of ensuring unbiased estimates of the underlying distributions f yg ( x ) over time.

Original: In particular, at each time t , the algorithm selects a (fairness-constrained) decision rule thatwould minimize classiﬁcation error based on its current, possibly biased estimates ˆ f y g,t ( x ) ; adoptingthis decision rule corresponds to exploitation of the current information by the algorithm.
Revised: In particular, at each time t , the algorithm selects a (fairness-constrained) decision rule that would minimize classiﬁcation error based on its current, possibly biased estimates ˆ f y g,t ( x ) ; adopting this decision rule corresponds to exploitation of the current information by the algorithm.

Original: At the sametime, to circumvent the censored feedback nature of the problem, our algorithm also deviates fromthe prescriptions of this loss-minimizing classiﬁer to a judiciously chosen extent (the extent is chosenadaptively, based on the current estimates); this will constitute exploration .
Revised: At the same time, to circumvent the censored feedback nature of the problem, our algorithm also deviates from the prescriptions of this loss-minimizing classiﬁer to a judiciously chosen extent (the extent is chosen adaptively, based on the current estimates); this will constitute exploration .

Original: Our algorithm includestwo parameters to limit the costs of this exploration: one modulates the frequency of exploration (anexploration probability ✏ t which can be adjusted using current bias estimates), and another limitsthe depth of exploration (by setting a threshold LB t on how far from the classiﬁer one is willing togo when exploring).
Revised: Our algorithm includes two parameters to limit the costs of this exploration: one modulates the frequency of exploration (an exploration probability ✏ t which can be adjusted using current bias estimates), and another limits the depth of exploration (by setting a threshold LB t on how far from the classiﬁer one is willing to go when exploring).

Original: We show that these choices can strike a balance between the ultimate goal ofremoving statistical biases in the training data – which will in turn lead to more accurate and fairdecisions, and the cost of exploration incurred to achieve this goal.
Revised: We show that these choices can strike a balance between the ultimate goal of removing statistical biases in the training data – which will in turn lead to more accurate and fair decisions, and the cost of exploration incurred to achieve this goal.

Original: Summary of ﬁndings and contributions.
Revised: Summary of ﬁndings and contributions.

Original: Our main ﬁndings and contributions are as follows:
Revised: Our main ﬁndings and contributions are as follows:

Original: Comparison with baselines.
Revised: Comparison with baselines.

Original: We contrast our proposed algorithm against two baselines: an exploitation-only baseline (one that does not include any form of exploration), and a pure exploration baseline (which may randomly 
Revised: We contrast our proposed algorithm against two baselines: an exploitation-only baseline (one that does not include any form of exploration), and a pure exploration baseline (which may randomly accept some of the agents rejected by the classiﬁer, but does not bound exploration).

Original: accept some

Original: of

Original: the agents rejected by the classiﬁer,

Original: but does not bound exploration).

Original: We show (Theorem 1) that exploitation-only always leads tooverestimates of the underlying distributions
Revised: We show (Theorem 1) that exploitation-only always leads to overestimates of the underlying distributions.

Original: Further, while pure exploration can debias thedistribution estimates in the long-run (Theorem 2), it does so at the expense of accepting any agent,no matter how far from the classiﬁer’s threshold, leading to more costly exploration (Section 5).
Revised: Further, while pure exploration can debias the distribution estimates in the long-run (Theorem 2), it does so at the expense of accepting any agent, no matter how far from the classiﬁer’s threshold, leading to more costly exploration (Section 5).

Original: Analytical support for our proposed algorithm.
Revised: Analytical support for our proposed algorithm.

Original: We show (Theorem 3) that our proposed activedebiasing algorithm with bounded exploration can correct biases in unimodal distribution estimates.Our numerical experiments in Section 5 show that our algorithm can achieve the same results on certain asymmetric distributions (
Revised: We show (Theorem 3) that our proposed active debiasing algorithm with bounded exploration can correct biases in unimodal distribution estimates.

Original: including

Original: those that appear in the real-world Adult and FICOdatasets).

Original: We also provide an error bound for our algorithm (Theorem 4)
Revised: We also provide an error bound for our algorithm (Theorem 4).

Original: Interplay with fairness criteria.
Revised: Interplay with fairness criteria.

Original: We analyze the impact of fairness constraints on our algorithm’sperformance and show (Proposition 1) that existing fairness criteria may speed up debiasing of the data in one group, while slowing it down for another.
Revised: We analyze the impact of fairness constraints on our algorithm’s performance, and show (Proposition 1) that existing fairness criteria may speed up debiasing of the data in one group, while slowing it down for another.

Original: Numerical experiments.
Revised: Numerical experiments.

Original: We provide numerical support for the performance of our algorithm usingexperiments on synthetic and real-world (Adult and FICO) datasets.
Revised: We provide numerical support for the performance of our algorithm using experiments on synthetic and real-world (Adult and FICO) datasets.

Original: Related work.
Revised: Related work.

Original: Our paper is closely related to the works of [3, 16, 11, 4, 14], which study the impactof data biases on (fair) algorithmic decision making.
Revised: Our paper is closely related to the works of [4, 20, 14, 6, 17], which study the impact of data biases on (fair) algorithmic decision making.

Original: Among these works, Bechavod et al.
Revised: Among these works, Bechavod et al.

Original: [3] and
Revised: [4] and

Original: Kilbertus et al.
Revised: Kilbertus et al.

Original: [16] study fairness-constrained learning in the presence of censored feedback.
Revised: [20] study fairness-constrained learning in the presence of censored feedback.

Original: Whilethese works also use exploration, the form and purpose of exploration is different: the algorithm in [3]starts with a pure exploration phase, and subsequently explores with the goal of ensuring the fairnessconstraint is not violated; the stochastic (or exploring) policies in [16] conduct (pure) explorationto address the censored feedback issue.
Revised: While these works also use exploration, the form and purpose of exploration is different: the algorithm in [4] starts with a pure exploration phase, and subsequently explores with the goal of ensuring the fairness constraint is not violated; the stochastic (or exploring) policies in [20] conduct (pure) exploration to address the censored feedback issue.

Original: In contrast, we start with a biased dataset, and conductbounded exploration to debias data; fairness constraints may or may not be enforced separately and are orthogonal to our debiasing process.
Revised: In contrast, we start with a biased dataset, and conduct bounded exploration to debias data; fairness constraints may or may not be enforced separately and are orthogonal to our debiasing process.

Original: Also, as shown in Section 5, such pure exploration processescan incur higher exploration costs than our proposed bounded exploration algorithm.
Revised: Also, as shown in Section 5, such pure exploration processes can incur higher exploration costs than our proposed bounded exploration algorithm.

Original: Our work is also closely related to [7, 23, 22, 27], which study adaptive sampling biases inducedby a decision rule, particularly when feedback is censored.
Revised: Our work is also closely related to [10, 31, 30, 41], which study adaptive sampling biases induced by a decision rule, particularly when feedback is censored.

Original: Among these, Neel and Roth [22] alsoconsider an adaptive data gathering procedure, and show that no debiasing will be necessary if thedata is collected through a differentially private method.
Revised: Among these, Neel and Roth [30] also consider an adaptive data gathering procedure, and show that no debiasing will be necessary if the data is collected through a differentially private method.

Original: We similarly propose an adaptive debiasingalgorithm, but unlike [22], account for the costs of exploration in our data collection procedure.
Revised: We similarly propose an adaptive debiasing algorithm, but unlike [30], account for the costs of exploration in our data collection procedure.

Original: Therecent work of Wei [27] studies data collection in the presence of censored feedback, and similarto our work, accounts for the cost of exploration in data collection, by formulating the problem asa partially observable Markov decision processes.
Revised: The recent work of Wei [41] studies data collection in the presence of censored feedback, and similar to our work, accounts for the cost of exploration in data collection, by formulating the problem as a partially observable Markov decision processes.

Original: Using dynamic programming methods, the datacollection policy is shown to be a threshold policy that becomes more stringent (in our terminology,reduces exploration) as learning progresses.
Revised: Using dynamic programming methods, the data collection policy is shown to be a threshold policy that becomes more stringent (in our terminology, reduces exploration) as learning progresses.

Original: Our works are similar in that we both propose usingadaptive and cost-sensitive exploration, but we differ in the problem setup and our analysis of theimpact of fairness constraints.
Revised: Our works are similar in that we both propose using adaptive and cost-sensitive exploration, but we differ in the problem setup and our analysis of the impact of fairness constraints.

Original: More importantly, in contrast to both [22, 27], our starting point isa biased dataset (which may be biased for reasons other than adaptive sampling in its collection,including historical biases); we then show how, while attempting to debias this dataset by collectingnew data, any additional adaptive sampling bias during data collection can be prevented.
Revised: More importantly, in contrast to both [30, 41], our starting point is a biased dataset (which may be biased for reasons other than adaptive sampling in its collection, including historical biases); we then show how, while attempting to debias this dataset by collecting new data, any additional adaptive sampling bias during data collection can be prevented.

Original: Our workis also broadly related to Bandit learning and its focus on exploration-exploitation trade-offs.
Revised: Our work also falls within the ﬁelds of selective labeling bias, fair learning, and active learning. From the selective labeling bias perspective, Lakkaraju et al. [22] propose a contraction technique to compare the performance of the predictive model and human judge while they are forced to have the same acceptance rate. De-Arteaga et al.

Revised: [9] propose a data augmentation scheme by adding more samples that would be more likely rejected (we refer to this as exploration) to correct the sample selection bias.

Revised: From the fair learning perspective, Kallus and Zhou [18] propose a re-weighting technique (similarly, re-weighing ideas are explored in [1, 6, 17]) to solve the residual unfairness issue while accounting for the censoring/adaptive sampling bias.

Revised: From the active learning perspective,

Revised: Noriega-Campero et al.

Revised: [32] adaptively acquire additional information according to the needs of different groups or individuals given information budgets, to achieve fair classiﬁcation.

Revised: Similar to the approaches of these papers, we also compensate for adaptive sampling bias through exploration; the main difference, aside from the application, is in our analytical guarantees as well as our focus on the interplay of debiasing with fairness constraints.

Revised: More broadly, our work has similarities to Bandit learning and its focus on exploration-exploitation trade-offs.

Original: A key difference of our work with existing bandit algorithms ( ✏ -greedy, UCB, EXP3, etc.) is ourfocus on bounded exploration.
Revised: A key difference of our work with existing bandit algorithms ( ✏ -greedy, UCB, EXP3, etc.) is our focus on bounded exploration.

Original: We provide additional discussion on this, and review other relatedworks [25, 2, 15, 18, 19, 28], in Appendix B.
Revised: We provide additional discussion on this, and review other related works [35, 3, 19, 26, 27, 42, 22, 9, 18, 32, 1] in more detail, in Appendix B.

Original:  Model and Preliminaries
Revised:  Model and Preliminaries

Original: The environment.
Revised: The environment.

Original: We consider a ﬁrm or decision maker, who selects an algorithm to make decisionson a population of agents .
Revised: We consider a ﬁrm or decision maker, who selects an algorithm to make decisions on a population of agents .

Original: The ﬁrm observes agents arriving over times t = 1 , 2 , . .
Revised: The ﬁrm observes agents arriving over times t = 1 , 2 , . .

Original: . , makes adecision for agents arriving at time t based on the current algorithm, and can subsequently adjust itsalgorithm for times t + 1 onward based on the observed outcomes.
Revised: . , makes a decision for agents arriving at time t based on the current algorithm, and can subsequently adjust its algorithm for times t + 1 onward based on the observed outcomes.

Original: Each agent has an observable feature or score x 2 X ✓ R .
Revised: Each agent has an observable feature or score x 2 X ✓ R .

Original: These represent the agent characteristicsthat are leveraged by the ﬁrm in its decision; examples include credit scores or exam scores.
Revised: These represent the agent characteristics that are leveraged by the ﬁrm in its decision; examples include credit scores or exam scores.

Original: Eachagent is either qualiﬁed or unqualiﬁed to receive a favorable decision; this is captured by the agent’strue label or qualiﬁcation state y 2 { 0 , 1 } , with y = 1 and y = 

Revised: Each agent is either qualiﬁed or unqualiﬁed to receive a favorable decision; this is captured by the agent’s true label or qualiﬁcation state y 2 { 0 , 1 } , with y = 1 and y =

Original: denoting qualiﬁed and unqualiﬁedagents, respectively.
Revised: denoting qualiﬁed and unqualiﬁed agents, respectively.

Original: In addition, each agent in the population belongs to a different group based onits demographic or protected attributes (e.g., race, gender); the agent’s group membership is denotedg 2 { a, b } .
Revised: In addition, each agent in the population belongs to a different group based on its demographic or protected attributes (e.g., race, gender); the agent’s group membership is denoted g 2 { a, b } .

Original: We consider threshold-based, group-speciﬁc, binary classiﬁers h ✓ g,t ( x ) :
Revised: We consider threshold-based, group-speciﬁc, binary classiﬁers h ✓ g,t ( x ) :

Original: X ! { 0 , 1 } as122 (part of) the algorithm adopted by the ﬁrm, where ✓ g,t denotes the classiﬁer’s decision threshold.
Revised: X ! { 0 , 1 } as (part of) the algorithm adopted by the ﬁrm, where ✓ g,t denotes the classiﬁer’s decision threshold.

Original: Anagent from group g with feature x arriving at time t is admitted if x  ✓ g,t .
Revised: An agent from group g with feature x arriving at time t is admitted if x  ✓ g,t .

Original: Quantifying bias.
Revised: Quantifying bias.

Original: Let f yg ( x ) denote the true underlying probability density function for the featuredistribution of agents from group g with qualiﬁcation state y .
Revised: Let f yg ( x ) denote the true underlying probability density function for the feature distribution of agents from group g with qualiﬁcation state y .

Original: The algorithm has an estimate ofthese unknown distributions, at each time t , based on the data collected so far (or an initial trainingset).
Revised: The algorithm has an estimate of these unknown distributions, at each time t , based on the data collected so far (or an initial training set).

Original: Denote the algorithm’s estimate at t by ˆ f yg,t ( x ) .
Revised: Denote the algorithm’s estimate at t by ˆ f yg,t ( x ) .

Original: In general, there can be  mismatch between theestimates ˆ f yg,t ( x ) and the true f yg ( x ) ; this is what we refer to as bias.
Revised: In general, there can be a mismatch between the estimates ˆ f yg,t ( x ) and the true f yg ( x ) ; this is what we refer to as bias.

Original: We assume the following.
Revised: We assume the following.

Original: Assumption 1.
Revised: Assumption 1.

Original: The ﬁrm updates its estimates ˆ f yg,t ( x ) by updating a single parameter ˆ !
Revised: The ﬁrm updates its estimates ˆ f yg,t ( x ) by updating a single parameter ˆ !

Original: yg,t .
Revised: yg,t .

Original: This type of assumption is common in the multi-armed bandit learning literature  (there, the algorithmaims to learn the mean arm rewards).
Revised: This type of assumption is common in the multi-armed bandit learning literature [38, 39, 34, 24, 36] (there, the algorithm aims to learn the mean arm rewards).

Original: In our setting, it holds when the assumed underlying distributionis single-parameter, or when only one of the parameters of a multi-parameter distribution is unknown.
Revised: In our setting, it holds when the assumed underlying distribution is single-parameter, or when only one of the parameters of a multi-parameter distribution is unknown.

Original: Alternatively, it can be interpreted as identifying and correcting distribution shifts by updating areference point in the distribution (e.g., adjusting the mean).
Revised: Alternatively, it can be interpreted as identifying and correcting distribution shifts by updating a reference point in the distribution (e.g., adjusting the mean).

Original: 2 More speciﬁcally, we will let ˆ !
Revised: 2 More speciﬁcally, we will let ˆ !

Original: yg,t bethe ↵ -th percentile of ˆ f yg,t ( x ) .
Revised: yg,t be the ↵ -th percentile of ˆ f yg,t ( x ) .

Original: We discuss potential limitations of this assumption in Appendix A,and present an extension to a case with two unknown parameters in Appendix H.
Revised: We discuss potential limitations of Assumption 1 in Appendix A, and present an extension to a case with two unknown parameters in Appendix I.

Original: Under Assumption 1, the bias can be captured by the mistmatch between the estimated and trueparameters ˆ !
Revised: Under Assumption 1, the bias can be captured by the mismatch between the estimated and true parameters ˆ !

Original: yg,t and !
Revised: yg,t and !

Original: yg .
Revised: yg .

Original: In particular, we set the mean absolute error E [ | ˆ !
Revised: In particular, we set the mean absolute error E [ | ˆ !

Original: yg,t  !
Revised: yg,t  !

Original: yg | ] as the measure forquantifying bias, where the randomness is due to that in ˆ !
Revised: yg | ] as the measure for quantifying bias, where the randomness is due to that in ˆ !

Original: yg,t , the estimate of the unknown parameterbased on data collected up to time t .
Revised: yg,t , the estimate of the unknown parameter based on data collected up to time t .

Original: Algorithm choice without debiasing.
Revised: Algorithm choice without debiasing.

Original: Let ↵ yg be the fraction of group g agents with label y .
Revised: Let ↵ yg be the fraction of group g agents with label y .

Original: Aloss-minimizing fair algorithm selects its thresholds ✓ g,t at time t as follows:
Revised: A loss-minimizing fair algorithm selects its thresholds ✓ g,t at time t as follows:

Original: [Equation]
Revised: [Equation]

Original: Here, the objective is the misclassiﬁcation error, and C ( ✓ a , ✓ b ) = 0 is the fairness constraint imposedby the ﬁrm, if any.
Revised: Here, the objective is the misclassiﬁcation error, and C ( ✓ a , ✓ b ) = 0 is the fairness constraint imposed by the ﬁrm, if any.

Original: For instance, C ( ✓ a,t , ✓ b,t ) =
Revised: For instance, C ( ✓ a,t , ✓ b,t ) =

Original: ✓ a,t  ✓ b,t for same decision rule , or C ( ✓ a,t , ✓ b,t ) =
Revised: ✓ a,t  ✓ b,t for same decision rule , or C ( ✓ a,t , ✓ b,t ) =

Original: R 1 ✓ a,t ˆ f 1 a,t ( x )d x 
Revised: R 1 ✓ a,t ˆ f 1 a,t ( x )d x

Original: R 1 ✓ b,t ˆ f 1 b,t ( x )d x for equal opportunity .
Revised: R 1 ✓ b,t ˆ f 1 b,t ( x )d x for equality of opportunity .

Original: Note that both the objective function and thefairness constraint are affected by any inaccuracies in the current estimates ˆ f yg,t .
Revised: Note that both the objective function and the fairness constraint are affected by any inaccuracies in the current estimates ˆ f yg,t .

Original: As such, a biasedtraining dataset can lead to both loss of accuracy and loss in desired fairness.
Revised: As such, a biased training dataset can lead to both loss of accuracy and loss in desired fairness.

Original:  An Active Debiasing Algorithm with Bounded Exploration
Revised:  An Active Debiasing Algorithm with Bounded Exploration

Original: In this section, we present the active debiasing algorithm which uses both exploitation (thedecision rules of (1)) and exploration (some deviations) to remove any biases from the estimates ˆ f yg,t .
Revised: In this section, we present the active debiasing algorithm which uses both exploitation (the decision rules of (1)) and exploration (some deviations) to remove any biases from the estimates ˆ f yg,t .

Original: Although the deviations may lead to admission of some unqualiﬁed agents, they can be beneﬁcial tothe ﬁrm in the long-run: by reducing biases in ˆ f yg,t , both classiﬁcation loss estimates and fairnessconstraint evaluations can be improved.
Revised: Although the deviations may lead to admission of some unqualiﬁed agents, they can be beneﬁcial to the ﬁrm in the long-run: by reducing biases in ˆ f yg,t , both classiﬁcation loss estimates and fairness constraint evaluations can be improved.

Original: In this section, we drop the subscripts g from the notation;when there are multiple groups, our algorithm can be applied to each group’s estimates separately.
Revised: In this section, we drop the subscripts g from the notation; when there are multiple groups, our algorithm can be applied to each group’s estimates separately.

Original: As noted in Section 1, our algorithm is one of bounded exploration: it includes a lower bound LB t ,which captures the extent to which the decision maker is willing to deviate from the current classiﬁer✓ t , based on its current estimate ˆ f 0 t of the unqualiﬁed agents’ underlying distribution.
Revised: As noted in Section 1, our algorithm is one of bounded exploration: it includes a lower bound LB t , which captures the extent to which the decision maker is willing to deviate from the current classiﬁer ✓ t , based on its current estimate ˆ f 0 t of the unqualiﬁed agents’ underlying distribution.

Original: Formally,
Revised: Formally,

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: At time t , the ﬁrm selects a lower bound LB t such that
Revised: At time t , the ﬁrm selects a lower bound LB t such that

Original: [Equation]
Revised: [Equation]

Original: where ✓ t is the (current) loss-minimizing threshold determined from (1) , ˆ F 0 t , ( ˆ F 0 t )  1 are the cdf andinverse cdf of the estimated distribution ˆ f 0 t , respectively, and ˆ ! 

Revised: where ✓ t is the (current) loss-minimizing threshold determined from (1) , ˆ F 0 t , ( ˆ F 0 t )  1 are the cdf and inverse cdf of the estimated distribution ˆ f 0 t , respectively, and ˆ !

Original: t is (wlog) the ↵ -th percentile of ˆ f 0 t .
Revised: t is (wlog) the ↵ -th percentile of ˆ f 0 t .

Revised: In more detail, we choose LB t such that ˆ F 0 t (ˆ ! 0 t )  ˆ F 0 t ( LB t )

Original: Intuitively, this bound is selected such that once/if{+ˆ ! 
 t is+} the available training data is debiased, the estimatedparameter ˆ ! 

Revised: = ˆ F 0 t ( ✓ t )  ˆ F 0 t (ˆ ! 0 t ) ; that is, such that {+ˆ ! 
 t is+} the median in the interval ( LB t , ✓ t ) based on the current estimate of the distribution ˆ F 0 t at the beginning of time t . Then, once a new batch of data is collected, we update ˆ !

Original: t {+to ˆ ! 
 t +1 , the realized median of the distribution between ( LB t , ✓ t ) based on the data observed during [ t, t + 1) . Once the underlying distribution is correctly estimated, (in expectation) we+} will ﬂuctuate around the true parameter ! 0 .
Revised: t {+to ˆ ! 
 t +1 , the realized median of the distribution between ( LB t , ✓ t ) based on the data observed during [ t, t + 1) . Once the underlying distribution is correctly estimated, (in expectation) we+} will observe the same number of samples between ( LB t , ! 0 t ) and between ( ! 0 t , ✓ t ) , and hence ! t will no longer change.

Original: We also note that by selecting a high↵ -th percentile in the above deﬁnition, LB t can be increased so as to limit the depth of exploration.
Revised: We also note that by selecting a high ↵ -th percentile in the above deﬁnition, LB t can be increased so as to limit the depth of exploration.

Original: As shown in Theorem 3, and in our numerical experiments, these thresholding choices will enabledebiasing while controlling its costs.
Revised: As shown in Theorem 3, and in our numerical experiments, these thresholding choice will enable debiasing of the distribution estimates while controlling its costs.

Original: Our active debiasing algorithm is summarized below.
Revised: Our active debiasing algorithm is summarized below.

Original: A pseudo-code is given in Appendix C.
Revised: A pseudo-code is given in Appendix C.

Original: Algorithm 1 (The active debiasing algorithm) .
Revised: Algorithm 1 (The active debiasing algorithm) .

Original: Denote the loss-minimizing decision threshold determined from (1) by ✓ t , and let LB t be given by Deﬁnition 1.
Revised: Denote the loss-minimizing decision threshold determined from (1) by ✓ t , and let LB t be given by Deﬁnition 1.

Original: Let { ✏ t } be a sequence of exploration probabilities.
Revised: Let { ✏ t } be a sequence of exploration probabilities.

Original: At each time t , and for agents ( x † , y † ) arriving at t :
Revised: At each time t , and for agents ( x † , y † ) arriving at t :

Original: Step 
Revised: Step I

Original: I: Admit agents and collect data.
Revised: : Admit agents and collect data

Original: Admit all agents with x †  ✓ t .
Revised: Admit all agents with x †  ✓ t .

Original: Additionally, if LB t  x † < ✓ t, admit the agent with probability ✏ t .
Revised: Additionally, if LB t  x † < ✓ t admit the agent with probability ✏ t .

Original: Step II: Update the distribution estimates based on new data collected in Step I. • Qualiﬁed agents’ distribution update: Identify new data with LB t  x † and y † = 1 .
Revised: Step II: Update the distribution estimates based on new data collected in Step I. • Qualiﬁed agents’ distribution update: Identify new data with LB t  x † and y † = 1 .

Original: Use all such x † with LB t  x † < ✓ t , and such x † with ✓ t  x † with probability ✏ t , to update ˆ ! 1 t .
Revised: Use all such x † with LB t  x † < ✓ t , and such x † with ✓ t  x † with probability ✏ t , to update ˆ ! 1 t .

Original: • Unqualiﬁed agents’ distribution update: Identify new data with LB t  x † and y † = 0 .
Revised: • Unqualiﬁed agents’ distribution update: Identify new data with LB t  x † and y † = 0 .

Original: Use all such x † with LB t  x † < ✓ t , and such x † with ✓ t  x † with probability ✏ t , to update ˆ ! 0 t .
Revised: Use all such x † with LB t  x † < ✓ t , and such x † with ✓ t  x † with probability ✏ t , to update ˆ ! 0 t .

Original: In more detail, our algorithm repeatedly performs the following two steps:
Revised: In more detail, our algorithm repeatedly performs the following two steps:

Original: Step I: Data collection.
Revised: Step I: Data collection.

Original: At the beginning of a time period t , a loss-minimizing classiﬁer withthreshold ✓ t (according to (1)) and the exploration lower bound LB t (Deﬁnition 1) are selected basedon the data collected so far.
Revised: At the beginning of a time period t , a loss-minimizing classiﬁer with threshold ✓ t (according to (1)) and the exploration lower bound LB t (Deﬁnition 1) are selected based on the data collected so far.

Original: Then, given ✓ t , the new data collected during period t will consist ofarriving agents with features x  ✓ t .
Revised: Then, given ✓ t , the new data collected during period t will consist of arriving agents with features x  ✓ t .

Original: Additionally, to address the censored feedback issues, withprobability ✏ t , the algorithm will also accept agents with LB t  x < ✓ t .
Revised: Additionally, to address the censored feedback issues, with probability ✏ t , the algorithm will also accept agents with LB t  x < ✓ t .

Original: Note that this step balancesbetween exploration and exploitation through its choice of both reference points (which limit thedepth of exploration) and exploration probabilities ✏ t (which limit the frequency of exploration).
Revised: Note that this step balances between exploration and exploitation through its choice of both LB t (which limit the depth of exploration) and exploration probabilities ✏ t (which limits the frequency of exploration).

Original: Step II: Updating estimates.
Revised: Step II: Updating estimates.

Original: At the end of period t , the data collected in Step I will be used to updateˆ f 0 t and ˆ f 1 t .
Revised: At the end of period t , the data collected in Step I will be used to update ˆ f 0 t and ˆ f 1 t .

Original: Under Assumption 1, thedistribution estimates ˆ f yt are updated by updating parameterˆ !
Revised: Under Assumption 1, the estimates ˆ f yt are updated by updating the parameter ˆ !

Original: yt .
Revised: yt .

Original: We assume, without loss of generality, that the ﬁrm sets ˆ !
Revised: We assume, without loss of generality, that the ﬁrm sets ˆ !

Original: yt to the ↵ -th percentile of ˆ f yt .
Revised: yt to the ↵ -th percentile of ˆ f yt .

Original: This↵ -th percentile is the reference point that will be adjusted over time as new data is collected.
Revised: This ↵ -th percentile is the reference point that will be adjusted over time as new data is collected.

Original: As anexample, when the reference point ˆ ! 1 t is set to the median (the 50-th percentile), the parameter can beadjusted so that half the label 1 data collected in Step I will lie on each side of the reference point.
Revised: As an example, when the reference point ˆ ! 1 t is set to the median (the 50-th percentile), the parameter can be adjusted so that half the label 1 data collected in Step I will lie on each side of the reference point.

Original:  Theoretical Analysis
Revised:  Theoretical Analysis

Original: We begin by analyzing two baselines: exploitation-only (which only accepts agents with x  ✓ t ,and uses no exploration or thresholding) and pure exploration (which accepts arriving agentsat time t who have x < ✓ t with probability ✏ t , without setting any lower bound).
Revised: We begin by analyzing two baselines: exploitation-only (which only accepts agents with x  ✓ t , and uses no exploration or thresholding) and pure exploration (which accepts arriving agents at time t who have x < ✓ t with probability ✏ t , without setting any lower bound).

Original: The motivationfor the choice of these two baselines is as follows: the exploitation-only baseline tracks theperformance of a decision maker who is unaware of underlying data biases, and makes no attemptat ﬁxing them.
Revised: The motivation for the choice of these two baselines is as follows: the exploitation-only baseline tracks the performance of a decision maker who is unaware of underlying data biases, and makes no attempt at ﬁxing them.

Original: The pure exploration baseline, on the other hand, is motivated by the Banditlearning literature, and is also akin to debiasing algorithms proposed in recent work (see Section 1,
Revised: The pure exploration baseline, on the other hand, is motivated by the Bandit learning literature, and is also akin to debiasing algorithms proposed in recent work (see Section 1,

Original: Related Work).
Revised: Related Work).

Original: We, in contrast, propose and show the beneﬁts of bounded exploration through ouractive debiasing algorithm.
Revised: We, in contrast, propose and show the beneﬁts of bounded exploration through our active debiasing algorithm.

Original: 4.1 The exploitation-only baseline
Revised: 4.1 The exploitation-only baseline

Original: Our ﬁrst baseline algorithm only updates its estimates of the underlying distributions based on agentswith x  ✓ t who pass the (current) loss-minimizing classiﬁer (1).
Revised: Our ﬁrst baseline algorithm only updates its estimates of the underlying distributions based on agents with x  ✓ t who pass the (current) loss-minimizing classiﬁer (1).

Original: The following result shows thatthis approach consistently suffers from adaptive sampling bias, ultimately resulting in overestimationof the underlying distributions.
Revised: The following result shows that this approach consistently suffers from adaptive sampling bias, ultimately resulting in overestimation of the underlying distributions.

Original: Theorem 1.
Revised: Theorem 1.

Original: An exploitation-only algorithm overestimates !
Revised: An exploitation-only algorithm overestimates !

Original: y , i.e., lim t !
Revised: y , i.e., lim t !

Original: 1 E [ˆ ! yt ] > !
Revised: 1 E [ˆ ! yt ] > !

Original: y , 8 y .
Revised: y , 8 y .

Original: Proof.

Original: We detail the proof for label 0 estimates ˆ ! 0 t , and discuss two cases while assuming (wlog)that the unknown !

Original: y being estimated is the distribution

Original: ’s mean.

Original: [-First, if ˆ ! 
-]
Revised: [-First, if ˆ ! 
-]

Original: [-t is overestimated, i.e.ˆ ! 
-]
Revised: [-t is overestimated, i.e.ˆ ! 
-]

Original: t > ! 0 .

Original: Note that we have ✓ t  ˆ ! 0 t .

Original: [-Then, as only agents with x †  ✓ t are admitted, ˆ ! 
-]
Revised: [-Then, as only agents with x †  ✓ t are admitted, ˆ ! 
-]

Original: t may onlybe updated to stay the same or increase.

Original: [-Therefore, ˆ ! 
-]
Revised: [-Therefore, ˆ ! 
-]

Original: t will remain overestimated.

Original: [-Next consider the case that ˆ ! 
-]
Revised: [-Next consider the case that ˆ ! 
-]

Original: [-t is underestimated, ˆ ! 
-]
Revised: [-t is underestimated, ˆ ! 
-]

Original: t < ! 0 .

Original: From t on, consider the T  t nextsteps.

Original: [-First, since each observation is independently drawn, we know that at time t 
-]
Revised: [-First, since each observation is independently drawn, we know that at time t 
-]

Original: =

Original: t, ..., t + T ,x t 0  E

Original: [ X | X  ✓ t 0 ] forms a martingale; this is because of the independence of x t 0 and ✓ t 0 when 205 conditioned on the historical information, as well as the fact that E [ x t 0 ] =

Original: E [ X | X  ✓ t 0 ] .

Original: By deﬁnition of ! 0 , we also know that

Original: P

Original: Tt 0 =

Original: t E [ X | X  ✓ t 0 ] > T · ! 0 .

Original: Denote the gap by  :=

Original: P Tt 0 = t E [ X | X  ✓ t 0 ] T  ! 0 .
Revised: A detailed proof is given in Appendix D.

Original: Therefore using the Azuma-Hoeffding inequality we have

Original: [Equation]

Original: for any  < 0 .

Original: Letting

Original: =   · ( T  t + 1) , the above can be re-written as

Original: [Equation]

Original: 1 This proves that with high probability the mean of the new samples is higher than ! 0 .

Original: [-Therefore, at some time T that is signiﬁcantly higher than t , the new estimate ˆ ! 
-]
Revised: [-Therefore, at some time T that is signiﬁcantly higher than t , the new estimate ˆ ! 
-]

Original: T will be similar to1 T  t +

Original: P

Original: Tt 0 =

Original: t x t 0 , which is higher than the true ! 0 .

Original: [-From our arguments for the overestimated case,from this point on, ˆ ! 
-]
Revised: [-From our arguments for the overestimated case,from this point on, ˆ ! 
-]

Original: t will stay overestimated.

Original: The proof for ˆ ! 1 t is similar.

Original: 4.2 The pure exploration baseline
Revised: 4.2 The pure exploration baseline

Original: In this second baseline, at each time t , the algorithm may accept any agent with x < ✓ t withprobability ✏ t .
Revised: In this second baseline, at each time t , the algorithm may accept any agent with x < ✓ t with probability ✏ t .

Original: The following result establishes that using the data collected this way, the distributionscan be debiased in the long-run, if the data collected above the classiﬁer is also sampled withprobability ✏ t when updating the distributions.
Revised: The following result establishes that using the data collected this way, the distributions can be debiased in the long-run, if the data collected above the classiﬁer is also sampled with probability ✏ t when updating the distributions.

Original: Theorem 2.
Revised: Theorem 2.

Original: Using the pure exploration algorithm, ˆ ! yt !!
Revised: Using the pure exploration algorithm, ˆ ! yt !

Original: y as t ! 1 , 8 y .

Original: The proof follows from assuming (wlog) that the unknown parameter !
Revised: The proof follows from assuming (wlog) that the unknown parameter !

Original: y being estimated is thedistribution ’s mean (can be generalized to arbitrary statistics under Assumption 1).
Revised: y being estimated is the distribution’s mean (can be generalized to arbitrary statistics under Assumption 1).

Original: Then, as weare collecting i.i.d.
Revised: Then, as we are collecting i.i.d.

Original: samples from across the distribution, ˆ !
Revised: samples from across the distribution, ˆ !

Original: yt can be set to the sample mean, andthe conclusion follows from the strong law of large numbers.
Revised: yt can be set to the sample mean, and the conclusion follows from the strong law of large numbers.

Original: Note also that if all the data abovethe classiﬁer was considered when making the updates, following similar arguments to those inthe proof of Theorem 1, the algorithm would obtain overestimates of the distributions.
Revised: Note also that if all the data above the classiﬁer was considered when making the updates, following similar arguments to those in the proof of Theorem 1, the algorithm would obtain overestimates of the distributions.

Original: Lastly, wecould equivalently balance data by resampling the exploration data (rather than downsampling theexploitation data), to debias data through this procedure.
Revised: Lastly, we could equivalently balance data by resampling the exploration data (rather than downsampling the exploitation data), to debias data through this procedure.

Original: 4.3 The active debiasing algorithm
Revised: 4.3 The active debiasing algorithm

Original: While pure exploration can successfully debias data in the long-run, it does so at the expense ofaccepting agents with any x < ✓ t .
Revised: While pure exploration can successfully debias data in the long-run, it does so at the expense of accepting agents with any x < ✓ t .

Original: Below, we provide analytical support that our proposed explorationand thresholding procedure in the active debiasing algorithm can still debias data in certaindistributions, while limiting the depth of exploration to LB t < x < ✓ t .
Revised: Below, we provide analytical support that our proposed exploration and thresholding procedure in the active debiasing algorithm can still debias data in certain distributions, while limiting the depth of exploration to LB t < x < ✓ t .

Original: Theorem 3.
Revised: Theorem 3.

Original: Let f y and ˆ f yt denote the true feature distribution and their estimates at the beginningof time t , with respective ↵ -th percentiles !
Revised: Let f y and ˆ f yt denote the true feature distribution and their estimates at the beginning of time t , with respective ↵ -th percentiles !

Original: y and ˆ ! yt .
Revised: y and ˆ ! yt .

Original: Assume these are unimodal distributions,✏ t > 0 , 8 t , and ˆ ! 

Revised: Assume these are unimodal distributions, ✏ t > 0 , 8 t , and ˆ !

Original: t  ✓ t  ˆ ! 1 t , 8 t .
Revised: t  ✓ t  ˆ ! 1 t , 8 t .

Original: Then, using the active debiasing algorithm,
Revised: Then, using the active debiasing algorithm,

Original: (a) If ˆ ! yt is underestimated (resp. overestimated), then E 
Revised: (a) If ˆ ! y t is underestimated (resp. overestimated), then E [ˆ !

Original: [ˆ ! yt +1 ]
Revised: y t +1 ]

Original:  ˆ !
Revised:  ˆ !

Original: yt , (resp.
Revised: y t , (resp.

Original: E [ˆ ! yt +1 ]  ˆ ! yt ) 8 t, 8 y .(b)
Revised: E [ˆ ! y t +1 ]  ˆ ! y t ) 8 t, 8 y .

Original:  The sequence { ˆ ! yt } converges, with ˆ ! yt ! !
Revised: (b) The sequence { ˆ ! yt } converges, with ˆ ! yt ! !

Original: y as t ! 1 , 8 y .
Revised: y as t ! 1 , 8 y .

Original: We provide a proof sketch for debiasing ˆ f 0 t which highlights the main technical challenges addressedin our analysis.
Revised: We provide a proof sketch for debiasing ˆ f 0 t which highlights the main technical challenges addressed in our analysis.

Original: The detailed proof is given in Appendix D.
Revised: The detailed proof is given in Appendix E.

Original: Proof sketch:
Revised: Proof sketch:

Original: Our proof involves the analysis of statistical estimates ˆ ! 

Revised: Our proof involves the analysis of statistical estimates ˆ !

Original: t based on data collectedfrom truncated distributions.
Revised: t based on data collected from truncated distributions.

Original: In particular, by bounding exploration, our algorithm will only collectdata with features x  LB t , and can use only this truncated data to build estimates of the unknownparameter of the distributions.
Revised: In particular, by bounding exploration, our algorithm will only collect data with features x  LB t , and can use only this truncated data to build estimates of the unknown parameter of the distributions.

Original: Part (a) establishes that the sequence of { ˆ ! yt } produced by our active debiasing algorithm“moves” in the right direction over time, and ultimately converges.
Revised: Part (a) establishes that the sequence of { ˆ ! yt } produced by our active debiasing algorithm “moves” in the right direction over time, and ultimately converges.

Original: The main challenge in this analysisis that as the exploration and update intervals [ LB t , 1 ) are themselves adaptive, there is no guaranteeon the number of samples in each interval, and therefore we need to analyze the estimates in ﬁnitesample regimes.
Revised: The main challenge in this analysis is that as the exploration and update intervals [ LB t , 1 ) are themselves adaptive, there is no guarantee on the number of samples in each interval, and therefore we need to analyze the estimates in ﬁnite sample regimes.

Original: To proceed with the analysis, we assume the feature distribution estimates followunimodel distributions (such as Gaussian, Beta, and the family of alpha -stable distributions) with !
Revised: To proceed with the analysis, we assume the feature distribution estimates follow unimodel distributions (such as Gaussian, Beta, and the family of alpha -stable distributions) with !

Original: 0as reference points.
Revised: as reference points.

Original: We then consider the expected parameter update following the arrival of a batchof agents;
Revised: We then consider the expected parameter update following the arrival of a batch of agents;

Original: Denote the current left portion in ( LB t , ˆ ! 0 t ) as p 1 :
Revised: Denote the current left portion in ( LB t , ˆ ! 0 t ) as p 1 :

Original: =
Revised: =

Original: ˆ F 0 (ˆ ! 0 t )  ˆ F 0 ( LB t ) ˆ F 0 ( ✓ t )  ˆ F 0 ( LB t ) .
Revised: ˆ F 0 (ˆ ! 0 t )  ˆ F 0 ( LB t ) ˆ F 0 ( ✓ t )  ˆ F 0 ( LB t ) .

Original: Based on Deﬁnition1, we can also obtain the current portion in (ˆ ! 

Revised: Based on Deﬁnition 1, we can also obtain the current portion in (ˆ !

Original: t , ✓ t ) denoted as p 2 :
Revised: t , ✓ t ) denoted as p 2 :

Original: = ˆ F 0 ( ✓ t )  ˆ F 0 (ˆ ! 

Revised: = ˆ F 0 ( ✓ t )  ˆ F 0 (ˆ !

Original: t ) ˆ F 0 ( ✓ t )  ˆ F 0 ( LB t ) =
Revised: t ) ˆ F 0 ( ✓ t )  ˆ F 0 ( LB t ) =

Original: p 1 .
Revised: p 1 .

Original: Thenew expected estimates E
Revised: The new expected estimates E

Original: [ˆ ! 

Revised: [ˆ !

Original: t +1 ] is the sample median in ( LB t , ✓ t ) , where samples come from thetrue distribution.
Revised: t +1 ] is the sample median in ( LB t , ✓ t ) , where samples come from the true distribution.

Original: We establish that this expected update will be higher/lower than ! 

Revised: We establish that this expected update will be higher/lower than !

Original: t if the currentestimate is an under/over estimate of the true parameter.
Revised: t if the current estimate is an under/over estimate of the true parameter.

Original: Then, in Part (b) we ﬁrst show that the sequence of over- and under-estimation errors in { ˆ ! yt } relativeto the true parameter !
Revised: Then, in Part (b) we ﬁrst show that the sequence of over- and under-estimation errors in { ˆ ! y t } relative to the true parameter !

Original: y are supermartingales.
Revised: y are supermartingales.

Original: By the Doobs Convergence theorem and using resultsfrom part (a) , these will converge to zero mean random variables with variance going to zero as thenumber of samples increases.
Revised: By the Doobs Convergence theorem and using results from part (a) , these will converge to zero mean random variables with variance going to zero as the number of samples increases.

Original: This establishes that { ˆ ! y t } converges.
Revised: This establishes that { ˆ ! yt } converges.

Original: It remains to show that thisconvergence point is the true parameter of the distribution.
Revised: It remains to show that this convergence point is the true parameter of the distribution.

Original: To do so, as detailed in the proof, we notethat the density function of the sample median estimated on label 0 data collected in [ LB t , ✓ t ] is
Revised: To do so, as detailed in the proof, we note that the density function of the sample median estimated on label 0 data collected in [ LB t , ✓ t ] is

Original: [Equation]
Revised: [Equation]

Original: which is a beta distribution pushed forward by H ( ⌫ ) :
Revised: which is a beta distribution pushed forward by H ( ⌫ ) :

Original: = F 0 ( ⌫ )  F 0 ( LB t ) F 0 ( ✓ t )  F 0 ( LB t ) ; this is the CDF of thetruncated F 0 distribution in [ LB t , ✓ t ] .
Revised: = F 0 ( ⌫ )  F 0 ( LB t ) F 0 ( ✓ t )  F 0 ( LB t ) ; this is the CDF of the truncated F 0 distribution in [ LB t , ✓ t ] .

Original: We then establish that the convergence point will be the truemedian of the underlying distribution.
Revised: We then establish that the convergence point will be the true median of the underlying distribution.

Original: 4.4 Error bound analysis
Revised: 4.4 Error bound analysis

Original: Our error bound analysis compares the errors (measured as the number of wrong decisions made)of our adaptive debiasing algorithm with the errors  made by an oracle which knows the trueunderlying distributions.
Revised: Our error bound analysis compares the errors (measured as the number of wrong decisions made) of our adaptive debiasing algorithm against the errors that would be made by an oracle which knows the true underlying distributions.

Original: We measure the performance using 0-1 loss, ` ( ˆ y i , y i ) =
Revised: We measure the performance using 0-1 loss, ` ( ˆ y i , y i ) =

Original: [ ˆ y i 6 =
Revised: [ ˆ y i 6 =

Original: y i ] , whereˆ y i and y i denote the predicted and true label of agent i , respectively.
Revised: y i ] , where ˆ y i and y i denote the predicted and true label of agent i , respectively.

Original: We  split the total T samples thatare observed throughout the horizon into four groups, corresponding to four different distributionsf yg .There will be a total of m rounds for updating the estimates.
Revised: We consider the error accumulated when updating the estimates using a total of m batches of data. We split the total T samples that have arrived during [ t, t + 1) into four groups, corresponding to four different distributions f yg .

Original: We use{ b 0 a , b 1 a , b 0 b , b 1 b }| t todenote the number of samplescollected from each label-group pair at round t 2 { 1 , . . .
Revised: Speciﬁcally, we use b yg,t to denote the number of samples from each label-group pair at round t 2 { 0 , . . .

Original: , m } .
Revised: , m } .

Original: Weupdate the unknown distribution estimates once all batches meet a size requirement s , i.e, such thatmin( b 0 a , b 1 a , b 0 b , b 1 b  )  s .
Revised: We update the unknown distribution estimates once all batches meet a size requirement s , i.e, once min( b yg,t )  s, 8 y, 8 g .

Original: The regret is given by:
Revised: The error of our algorithm is given by:

Original: [Equation]
Revised: [Equation]

Original: i
Revised: i

Original: The following theorem provides an upper bound on the error incurred by active debiasing .
Revised: The following theorem provides an upper bound on the error incurred by active debiasing .

Original: Theorem 4.
Revised: Theorem 4.

Original: Let ˆ f yg,t ( x ) be the estimated feature distribution at round t 2 { 1 , . . .
Revised: Let ˆ f yg,t ( x ) be the estimated feature-label distributions at round t 2 { 0 , . . .

Original: , m } for the featuredistribution of agents with qualiﬁcation states y , and ✓ g,t be a v -approximate solution.
Revised: , m } .

Original: We considerthe threshold-based, group-speciﬁc, binary classiﬁer h ✓ g,t , and denote the Rademacher complexity ofthe classiﬁer family H with n training samples by R n ( H ) .
Revised: We consider the threshold-based, group-speciﬁc, binary classiﬁer h ✓ g,t , and denote the Rademacher complexity of the classiﬁer family H with n training samples by R n ( H ) .

Original:  At round t ,among the two groups, let279 N t be the larger net exploration errors made, n 0 t be thesmaller sample size collected, R n 00 t ( H  ) be280 the larger Rademacher complexity.
Revised: Let ✓ g,t be a v -approximately optimal classiﬁer based on data collected up to time t . At round t , let N g,t be the number of exploration errors incurred by our algorithm, n g,t be the sample size at time t from group g , d H  H ( ˜ D g,t , D g ) be the distance between true unbiased data distribution D g and the current biased estimate ˜ D g,t , and c ( ˜ D g,t , D g ) be the minimum error on an algorithm trained on unbiased and biased data.

Original: Then, with probability at least 4  with  > 0 , the activedebiasing algorithm has an error bound (Error):
Revised: Then, with probability at least 4  with  > 0 , the active debiasing algorithm’s error is bounded by:

Original: [Equation]
Revised: [Equation]

Revised: |

Original: A detailed proof is given in Appendix E. Since our algorithm can update the unknown estimatesonce all batches meet a size requirement, for simplicity, we focus on the analysis for each updateindividually.
Revised: [Equation] [Table] i More details on the deﬁnitions of the distance measure d H  H , and the error term c ( · ) , and the exploration error term N g,t , along with a a detailed proof, are given in Appendix F. From the expression above, we can see that the error incurred by our algorithm consists of four types of error:

Original: Then, the total error can be bounded above by the error incurred in the ﬁrst update timesthe number of updates.
Revised: errors due to approximation of the optimal (fair) classiﬁer at each round, empirical estimation errors, exploration errors, and errors due to our biased training data (viewed as source-target distribution mismatches); the latter two are speciﬁc to our active debiasing algorithm. In particular, as we collect more samples, n g,t will increase. Hence, the empirical estimation errors decrease over time.

Original: This is because the error decreases as our algorithm updates its estimates, sothat the ﬁrst update comes with the largest error.
Revised: Moreover, as the mismatch between ˜ D g,t and D g decreases using our algorithm (by Theorem 3), the error due to target domain and source domain mismatches also decrease. In the meantime, our exploration probability ✏ t also becomes smaller over time, decreasing N g,t .

Original: 4.5 Active debiasing and fairness criteria
Revised: 4.5 Active debiasing and fairness criteria

Original: We next consider our proposed active debiasing algorithm when used in conjunction with demographicfairness constraints (e.g., equality of opportunity, same decision rule, and statistical parity [21]).
Revised: We next consider our proposed active debiasing algorithm when used in conjunction with demographic fairness constraints (e.g., equality of opportunity, same decision rule, and statistical parity [29]).

Original: Imposing such fairness rules will lead to changes to the selected classiﬁers compared to the fairnessunconstrained case.
Revised: Imposing such fairness rules will lead to changes to the selected classiﬁers compared to the fairness-unconstrained case.

Original: Let ✓ Fg,t and ✓ Ug,t denote the fairness constrained and unconstrained decision rulesobtained from (1) at time t for group g , respectively.
Revised: Let ✓ Fg,t and ✓ Ug,t denote the fairness constrained and unconstrained decision rules obtained from (1) at time t for group g , respectively.

Original: We say group g is being over-selected (resp.under-selected) following the introduction of fairness constraints if ✓ Fg,t < ✓ Ug,t (resp.
Revised: We say group g is being overselected (resp. under-selected) following the introduction of fairness constraints if ✓ Fg,t < ✓ Ug,t (resp.

Original: ✓ Fg,t > ✓ Ug,t ).
Revised: ✓ Fg,t > ✓ Ug,t ).

Original: Below, we show how such over/under-selections can differently affect the debiasing of estimates ondifferent agents.
Revised: Below, we show how such over/under-selections can differently affect the debiasing of estimates on different agents.

Revised: In particular, let the speed of debiasing be the rate at which E [ | ˆ !

Revised: yt

Revised:  !

Revised: y | ] decreases with respect to t ; then, for a given t , an algorithm for which this error is larger has a slower speed of debiasing.

Revised: The following proposition identiﬁes the impacts of different fairness constraints on the speed of debiasing attained by our active debiasing algorithm.

Original: The proof is given in Appendix F.
Revised: The proof is given in Appendix G.

Original: Proposition 1.
Revised: Proposition 1.

Original: Let f yg and ˆ f yg,t be the true and estimated feature distributions, with respective ↵ -thpercentiles !
Revised: Let f yg and ˆ f yg,t be the true and estimated feature distributions, with respective ↵ -th percentiles !

Original: y and ˆ ! yt .
Revised: y and ˆ ! yt .

Original: Assume these are unimodal distributions, and active debiasing is applied.
Revised: Assume these are unimodal distributions, and active debiasing is applied.

Original: If group g is over-selected (resp.
Revised: If group g is over-selected (resp.

Original: under-selected) under a fairness constraint, i.e., ✓ Fg,t < ✓ Ug,t (resp.
Revised: under-selected) under a fairness constraint, i.e., ✓ Fg,t < ✓ Ug,t (resp.

Original: ✓ Fg,t > ✓ Ug,t ), the speed of debiasing on the estimates ˆ f yg,t will decrease (resp. increase).
Revised: ✓ Fg,t > ✓ Ug,t ), the speed of debiasing on the estimates ˆ f yg,t will decrease (resp. increase).

Original: Proposition 1 highlights the following implications of using both fairness rules and our activedebiasing efforts.
Revised: Proposition 1 highlights the following implications of using both fairness rules and our active debiasing efforts.

Original: Some fairness constraints (such as equality of opportunity) can lead to an increasein opportunities for (here, over-selection of) agents from disadvantaged groups, while others (such as same decision rule) can lead to under-selection from that group.
Revised: Some fairness constraints (such as equality of opportunity) can lead to an increase in opportunities for (here, over-selection of) agents from disadvantaged groups, while others (such as same decision rule) can lead to under-selection from that group.

Original: Proposition 1 shows that activedebiasing may in turn become faster or slower at debiasing estimates on this group.
Revised: Proposition 1 shows that active debiasing may in turn become faster or slower at debiasing estimates on this group.

Original: Intuitively, over-selection provides increased opportunities to agents from a group (compared to anunconstrained classiﬁer).
Revised: Intuitively, over-selection provides increased opportunities to agents from a group (compared to an unconstrained classiﬁer).

Original: In fact, the reduction of the decision threshold to ✓ Fg,t can itself be interpretedas introducing exploration (which is separate from that introduced by our debiasing algorithm).
Revised: In fact, the reduction of the decision threshold to ✓ Fg,t can itself be interpreted as introducing exploration (which is separate from that introduced by our debiasing algorithm).

Revised: When a group is over-selected under a fairness constraint, the fairness-constrained threshold ✓ Fg,t will be lower than the unconstrained threshold ✓ Ug,t .

Original: Suchincrease in exploration induced by the fairness constraint will increase theexploration lower bound LB t of our algorithm (see Deﬁnition 1); that is, our algorithm will become more conservative atexploration, and ultimately slowerat debiasing the respective estimates.
Revised: Therefore, the exploration range will be narrower, which means by adding a fairness constraint, the algorithm needs to wait and collect more samples (takes a longer time) before it manages to collect sufﬁcient data to accurately update the unknown distribution parameter, and hence, it has a slower debiasing speed.

Original: More broadly, these ﬁndingscontribute to our understanding of how fairness constraints can have long-term implications beyondthe commonly studied fairness-accuracy tradeoff when we consider their impacts on data collectionand debiasing efforts.
Revised: More broadly, these ﬁndings contribute to our understanding of how fairness constraints can have long-term implications beyond the commonly studied fairness-accuracy tradeoff when we consider their impacts on data collection and debiasing efforts.

Original:  Numerical Experiments
Revised:  Numerical Experiments

Original: In this section, we illustrate the performance of our algorithm through numerical experiments on both Gaussian and Beta distributed synthetic datasets, and on two real-world datasets: the Adult dataset [9]and the FICO credit score dataset [26] pre-processed by [13].
Revised: In this section, we illustrate the performance of our algorithm through numerical experiments on both Gaussian and Beta distributed synthetic datasets, and on two real-world datasets: the Adult dataset [12] and the FICO credit score dataset [37] pre-processed by [16].

Original: Additional details (ground-truthinformation) on the experiments, and larger versions of all ﬁgures, are available in Appendix G.
Revised: Additional details (groundtruth information) on the experiments, and larger versions of all ﬁgures, are available in Appendix H.

Revised: Our code is available at: https://github.com/Yifankevin/adaptive_data_debiasing .

Original: Throughout, we either choose a ﬁxed schedule for reducing the exploration frequencies { ✏ t } , orreduce these adaptively as a function of the estimated error.
Revised: Throughout, we either choose a ﬁxed schedule for reducing the exploration frequencies { ✏ t } , or reduce these adaptively as a function of the estimated error.

Original: For the latter, the algorithm can select arange (e.g., above the classiﬁer for label 0/1) and adjust the exploration frequency proportional to thediscrepancy between the number of observed errors in this interval relative to the number expectedgiven the distribution estimates.
Revised: For the latter, the algorithm can select a range (e.g., above the classiﬁer for label 0/1) and adjust the exploration frequency proportional to the discrepancy between the number of observed errors in this interval relative to the number expected given the distribution estimates.

Original: Comparison with the exploitation-only and pure exploration baselines: Our ﬁrst experiments in Fig.
Revised: Comparison with the exploitation-only and pure exploration baselines: Our ﬁrst experiments in Fig.

Original: 1, compare our algorithm against two baselines.
Revised: 1, compare our algorithm against two baselines.

Original: The underlying distributions are
Revised: The underlying distributions are

Original: Gaussian and no fairness constraint is imposed.
Revised: Gaussian and no fairness constraint is imposed.

Original: Our algorithm sets ↵ 1 = 50 and ↵ 0 = 60 percentiles,and exploration frequencies ✏ t are selected adaptively by both our algorithm and pure 
Revised: Our algorithm sets ↵ 1 = 50 and ↵ 0 = 60 percentiles, and exploration frequencies ✏ t are selected adaptively by both our algorithm and pure exploration.

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure] [Figure] [Figure] Speed of debiasing: Figs. 1(a) and 1(b) show that consistent with Theorem 1, exploitation-onlyoverestimates the distributions due to adaptive sampling biases.
Revised: Speed of debiasing: Figs. 1(a) and 1(b) show that consistent with Theorem 1, exploitation-only overestimates the distributions due to adaptive sampling biases.

Original: Further, consistent with Theorem 2, pure exploration successfully debiases data.
Revised: Further, consistent with Theorem 2, pure exploration successfully debiases data.

Original: We also observe that as expected, pureexploration debiases faster than active debiasing .
Revised: We also observe that as expected, pure exploration debiases faster than active debiasing .

Original: The difference is more pronounced inthe label 0 distributions compared to label 1, where pure exploration collects more “diverse”observations than our algorithm.
Revised: The difference is more pronounced in the label 0 distributions compared to label 1, where pure exploration collects more “diverse” observations than our algorithm.

Original: For this same reason, the gap between pure exploration and ouralgorithm is larger when f 0 is overestimated.
Revised: For this same reason, the gap between pure exploration and our algorithm is larger when f 0 is overestimated.

Original: This is because pure exploration observes sampleswith lower features x than active debiasing , and so can use these to reduce its estimate faster.
Revised: This is because pure exploration observes samples with lower features x than active debiasing , and so can use these to reduce its estimate faster.

Original: Regret: Figs.
Revised: Regret: Figs.

Original: 1(c) and 1(d) compare the regret and weighted regret of the algorithms.
Revised: 1(c) and 1(d) compare the regret and weighted regret of the algorithms.

Original: Regret ismeasured as the difference between the number of FN+FP decisions of an algorithm vs the oracle lossminimizing algorithm derived on unbiased data.
Revised: Regret is measured as the difference between the number of FN+FP decisions of an algorithm vs the oracle loss-minimizing algorithm derived on unbiased data.

Original: Regret is deﬁned as in Section 4.4; weighted regretis deﬁned similarly, but also adds a weight to each FN or FP decision, with the weight exponentialin the distance of the admitted agent from the classiﬁer.
Revised: Formally, regret is deﬁned as in Section 4.4; weighted regret is deﬁned similarly, but also adds a weight to each FN or FP decision, with the weight exponential in the distance of the admitted agent from the classiﬁer.

Original: We observe that exploitation-only ’sregret is super-linear, as not only it fails to debias, but has increasing error due to biases fromoverestimating.
Revised: We observe that exploitation-only ’s regret is super-linear, as not only it fails to debias, but has increasing error due to biases from overestimating.

Original: On the other hand, while algorithms that explore “deeper” have lower regret (e.g.pure exploration < active debiasing with ↵ 0 = 50 < active debiasing with ↵ 0 = 60 in Figs.
Revised: On the other hand, while algorithms that explore “deeper” have lower regret (e.g. pure exploration < active debiasing with ↵ 0 = 50 < active debiasing with ↵ 0 = 60 in Figs.

Original: 1(c)), they have higher weighted regret (the order is reversed in Fig. 1(d)).
Revised: 1(c)), they have higher weighted regret (the order is reversed in Fig. 1(d)).

Original: In other words,exploring to admit agents with low x leads to some errors, but ultimately helps reduce future mistakes,leading to sub-linear regret.
Revised: In other words, exploring to admit agents with low x leads to some errors, but ultimately helps reduce future mistakes, leading to sub-linear regret.

Original: However, if the risk/safety of these wrong decisions is taken into account,the ﬁrm maybe better off adopting slower, but less risky exploration thresholds (e.g. ↵ 0 = 70 ).
Revised: However, if the risk/safety of these wrong decisions is taken into account, the ﬁrm may be better off adopting slower, but less risky exploration thresholds (e.g. ↵ 0 = 70 ).

Original: Performance of active debiasing on
Revised: Performance of active debiasing on

Original: Beta distributions: Fig.
Revised: Beta distributions: Fig.

Original: 2 shows that ouralgorithm can debias data for which theunderlying feature-label distributions follow Beta distributions.
Revised: 2 shows that our algorithm can debias data for which the underlying feature-label distributions follow Beta distributions.

Original: We have assumeda mistmach between the parameter ↵ ofthe true and estimated distributions, and selected these so that the estimated and truedistributions have different relative skewness.
Revised: We have assumed a mistmach between the parameter ↵ of the true and estimated distributions, and selected these so that the estimated and true distributions have different relative skewness.

Original: This veriﬁes that Theorem 3 can holdbeyond symmetric distributions.
Revised: This veriﬁes that Theorem 3 holds for asymmetric distributions.

Original: [Figure]
Revised: [Figure]

Original: Interplay of debiasing and fairness constraints: Fig.
Revised: Interplay of debiasing and fairness constraints: Fig.

Original: 3 compares the performanceof active debiasing when there aretwo groups of agents with underlying Gaussian distributions, and the algorithm is chosen subject to three different fairness settings: no fairness, equality of opportunity(EO), and the same decision rule (SD).
Revised: 3 compares the performance of active debiasing when there are two groups of agents with underlying Gaussian distributions, and the algorithm is chosen subject to three different fairness settings: no fairness, equality of opportunity (EO), and the same decision rule (SD).

Original: The ﬁndings are consistent with Proposition 1.
Revised: The ﬁndings are consistent with Proposition 1.

Original: For instance,
Revised: For instance,

Original: SD will over-select the majority group (i.e., ✓ SDa,t < ✓ Ua,t ) so that, as shown in the left panel in Fig.
Revised: SD will over-select the majority group (i.e., ✓ SDa,t < ✓ Ua,t ) so that, as shown in the left panel in Fig.

Original: 3,the speed of debiasing on the estimates ˆ f ya,t will decrease.
Revised: 3, the speed of debiasing on the estimates ˆ f ya,t will decrease.

Original: In contrast, an opposite effect will happenin the minority group b which is under-selected (i.e., ✓ SDb,t > ✓ Ub,t ).
Revised: In contrast, an opposite effect will happen in the minority group b which is under-selected (i.e., ✓ SDb,t > ✓ Ub,t ).

Original: The effects of EO can be similarlyexplained by noting that it under-selects the majority group and over-selects the minority group.
Revised: The effects of EO can be similarly explained by noting that it under-selects the majority group and over-selects the minority group.

Original: Active debiasing on the Adult dataset: Fig. 4 illustrates the performance of our algorithm onthe Adult dataset.
Revised: Active debiasing on the Adult dataset: Fig. 4 illustrates the performance of our algorithm on the Adult dataset.

Original: Data is grouped based on race (White G a and non-White G b ), with labels y=
Revised: Data is grouped based on race (White G a and non-White G b ), with labels y

Original: 1for income > $50 k/year .
Revised: =for income > $50 k/year .

Original: A one-dimensional feature x 2 R is constructed by conducting logisticregression on four quantitative and qualitative features (education number, sex, age, workclass), basedon the initial training data.
Revised: A one-dimensional feature x 2 R is constructed by conducting logistic regression on four quantitative and qualitative features (education number, sex, age, workclass), based on the initial training data.

Original: 3 Using an input analyzer, we found Beta distributions as the best ﬁt to theunderlying distributions.
Revised: 3 Using an input analyzer, we found Beta distributions as the best ﬁt to the underlying distributions.

Original: We use 2.5% of the data to obtain a biased estimate of the parameter ↵ .
Revised: We use 2.5% of the data to obtain a biased estimate of the parameter ↵ .

Original: Theremaining data arrives sequentially.
Revised: The remaining data arrives sequentially.

Original: We use ↵ 1 = 50 and ↵ 0 = 60 and a ﬁxed decreasing { ✏ t } , withthe equality of opportunity fairness constraint imposed throughout.
Revised: We use ↵ 1 = 50 and ↵ 0 = 60 and a ﬁxed decreasing { ✏ t } , with the equality of opportunity fairness constraint imposed throughout.

Original: [Figure]
Revised: [Figure]

Original: We observe that our proposed algorithm can debias estimates across groups and for both labels, butthat this happens in the long-run and given access to sufﬁcient samples.
Revised: We observe that our proposed algorithm can debias estimates across groups and for both labels, but that this happens in the long-run and given access to sufﬁcient samples.

Original: In particular, we note thatfor label 1 agents from G b , as there are only 1080 samples in the dataset, although the bias initiallydecreases, the ﬁnal estimate still differs from the true value.
Revised: In particular, we note that for label 1 agents from G b , as there are only 1080 samples in the dataset, although the bias initially decreases, the ﬁnal estimate still differs from the true value.

Original: Fig.
Revised: Fig.

Original: 4(c) veriﬁes that this estimate wouldhave been debiased in the long-run, had additional samples from the underlying population becomeavailable (i.e., as more such agents arrive).
Revised: 4(c) veriﬁes that this estimate would have been debiased in the long-run, had additional samples from the underlying population become available (i.e., as more such agents arrive).

Original: Active debiasing on the FICO dataset: Fig. 4 also illustrates the performance of our algorithmon the FICO dataset [26, 13], and shows that it is successful in  both groups and on both labels.
Revised: Active debiasing on the FICO dataset: Fig. 4 also illustrates the performance of our algorithm on the FICO dataset [37, 16], and shows that it is successful in debiasing distribution estimates on both groups and on both labels.

Original:  Checklist
Revised:  Conclusion, Limitations, and Future Work

Original: For all authors...

Original: (a)
Revised: We proposed an active debiasing algorithm which recovers unbiased estimates of the underlying data distribution of agents interacting with it over time. We also analyzed the interplay of our proposed statistical/data debiasing effort with existing social/model debiasing efforts, shedding light on the potential alignments and conﬂicts between these two goals in fair algorithmic decision making.

Original: Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope
Revised: We further illustrated the performance of our proposed algorithm, and its interplay with fairness constraints, through numerical experiments on both synthetic and real-world datasets.

Original: [Yes
Revised: The single-unknown parameter assumption. Our work focuses on learning of a single unknown parameter (Assumption 1). Despite the commonality of this assumption in the multi-armed bandit learning literature, it also entails parametric knowledge of the underlying distribution with the other parameters such as variance or spread being known. We extend our algorithm to a Gaussian distribution with two unknown parameters in Appendix I. Extensions beyond this, especially those not requiring parametric assumptions on the underlying distributions, remain a main direction of future work.

Original: (b) Have you read
Revised: On one-dimensional features and threshold classiﬁers. Our analytical results have been focused on one-dimensional feature data and threshold classiﬁers.

Original:  the ethics review guidelines and ensured that your paper conforms tothem
Revised: These assumptions may not be too restrictive in some cases: the optimality of threshold classiﬁers has been established in the literature by, e.g., [8,

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: Thm 3.2] and [36], as long as a multi-dimensional feature can be mapped to a properly deﬁned scalar.

Original: [Yes] Fairness
Revised: Moreover, the recent advances in deep learning have helped enable this possibility: one can take the last layer outputs from a deep neural network and use it as the single dimensional representation. That said, any reduction of multi-dimensional features to a single-dimensional score may lead to some loss of information.

Original: implications of our  algorithm are discussed in Section 4.5. A limitation of our algorithmfor groups with smaller representation are discussed at the end of Section 5.
Revised: In particular, our experiments have considered the use of our active debiasing algorithm on the Adult dataset with multi-dimensional features by ﬁrst performing a dimension reduction to a single-dimensional score; we ﬁnd that this reduction can lead to a ( ⇠ 5% ) loss in performance (see Appendix A for details).

Original: See alsoAppendix A for more discussion.
Revised: One potential solution to this is to adopt a mapping from high-dimensional features to scores that is revised repeatedly as the algorithm collects more data.

Original: (d) Did you describe the limitations of your work? [Yes] See the remark in Section 3 and the paragraph before the proof of Theorem 3 in Section 4, and Appendix A.2. If you are including theoretical results...
Revised: Alternatively, one may envision a debiasing algorithm which targets its exploration towards collecting data on features that are believed to be highly biased; these remain as potential extensions of our algorithm.

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Section 4.

Original: (b) Did you include complete proofs of all theoretical results? [Yes] See Section 4 for the main proofs, and additional proofs in the supplementary material, Appendix D, E,and F. If you ran experiments...
Revised: Potential social impacts.

Original: (a) Did you include the code, data, and instructions needed to reproduce the main ex-

Original: perimental results (either in the supplemental material or as a URL)?

Original: [Yes] Code is

Original: included in the supplementary material.

Original: Dataset used are the Adult dataset, publicly

Original: available from the UCI machine learning repository [9], and the FICO credit scoredataset [26] pre-processed by [13]

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,

Original: how they

Original: were chosen)? [Yes] See Section 5 for details. (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
Revised: More broadly, while our debiasing algorithm imposes fairness constraints on its exploitation decisions (see problem (1)), it does not consider fairness constraints in its exploration decisions.

Original: [Yes] See Figure 2, 4 Adult Dataset, 12(a), 12(b) for

Original: details

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?

Original: [N/A] All experiments were run on a

Original: local computer.

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets.

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Section 5.

Original: (b) Did you mention the license of the assets?

Original: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]

Original: [Table] Our algorithm’s code is available in the supplementary material. No new data wascurated or generated. [Table]
Revised: That means that our proposed algorithm could be disproportionate in the way it increases opportunities for qualiﬁed or unqualiﬁed agents in different groups during exploration.

Original: (d) Did you discuss whether and how consent was obtained from people whose data

Original: [Table]

Original: you’re using/curating? [N/A] We use the publicly available Adult dataset from the UCI [Table] machine learning repository [9], for which data was extracted from a 1994 Censusdatabase; and the FICO credit score dataset [26] pre-processed by [13]
Revised: Imposing fairness rules on exploration decisions, as well as identifying algorithms that can improve the speed of debiasing of estimates on underrepresented populations, can be explored to address these potential social impacts, and remain as interesting directions of future work.

Original: [Table] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable [Table] information or offensive content?
Revised: Additional discussions on limitations, extensions, and social impacts, are given in Appendix A.

Original: [N/A] We use the publicly available Adult datasetfrom the UCI machine learning repository [9], for which data was extracted from a

Original: [Table] 1994 Census database; and the FICO credit score dataset [26] pre-processed by [13]. [Table] They do not contain PII or offensive content.
Revised: Acknowledgments and Disclosure of Funding

Original: [Table]

Original: If you used crowdsourcing or conducted research with human subjects...

Original: [Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable?
Revised: We sincerely thank the three reviewers and the area chair for their comments and feedback which helped improve our paper.

Original: [N/A]

Original: [Table]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: [Table] Board (IRB) approvals, if applicable? [N/A] [Table]
Revised: We are also grateful for support from the National Science Foundation (NSF) program on Fairness in AI in collaboration with Amazon under Award No. IIS-2040800, the NSF under grant IIS-2143895, and Cisco Research.

Original: (c) Did you include the estimated hourly wage paid to participants and  the total amount [Table] spent on participant compensation? [N/A]
Revised: Any opinion, ﬁndings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reﬂect the views of the NSF, Amazon, or Cisco.

Original: [Table]