Original: Active Approximately Metric-Fair Learning
Revised: Active Approximately Metric-Fair Learning

Original: Abstract
Revised: Abstract

Original: Existing studies on individual fairness focus on the passive setting and typically require O ( 1 ε 2 ) labeled instances to achieve an ε bias budget.
Revised: Existing studies on individual fairness focus on the passive setting and typically require O ( 1 ε 2 ) labeled instances to achieve an ε bias budget.

Original: In this paper, we build on the elegant Approximately MetricFair (AMF) learning framework and propose an active AMF learner that can provably achieve the same budget with only O (log 1 ε ) labeled instances.
Revised: In this paper, we build on the elegant Approximately MetricFair (AMF) learning framework and propose an active AMF learner that can provably achieve the same budget with only O (log 1 ε ) labeled instances.

Original: To our knowledge, this is a first and substantial improvement of the existing sample complexity for achieving individual fairness.
Revised: To our knowledge, this is a first and substantial improvement of the existing sample complexity for achieving individual fairness.

Original: Through experiments on three data sets, we show the proposed active AMF learner improves fairness on linear and non-linear models more efficiently than its passive counterpart as well as state-of-the-art active learners, while maintaining a comparable accuracy.
Revised: Through experiments on three data sets, we show the proposed active AMF learner improves fairness on linear and non-linear models more efficiently than its passive counterpart as well as state-of-the-art active learners, while maintaining a comparable accuracy.

Original: To facilitate algorithm design and analysis, we also design a provably equivalent form of the approximate metric fairness based on uniform continuity instead of the existing almost Lipschitz continuity.
Revised: To facilitate algorithm design and analysis, we also design a provably equivalent form of the approximate metric fairness based on uniform continuity instead of the existing almost Lipschitz continuity.

Original:  INTRODUCTION
Revised:  INTRODUCTION

Original: In modern machine learning applications, an important societal concern is fairness of the machine-learned models.
Revised: In modern machine learning applications, an important societal concern is fairness of the machine-learned models.

Original: One may think that model-made decisions have no discrimination against minority people, but many case studies show otherwise e.g., Feller et al.
Revised: One may think that model-made decisions have no discrimination against minority people, but many case studies show otherwise e.g., Feller et al.

Original: [2016], Chan and Wang [2018].
Revised: [2016], Chan and Wang [2018].

Original: As a result, how to attain fairness in machine-learned models becomes an important research problem and the literature has exploded in recent years.
Revised: As a result, how to attain fairness in machine-learned models becomes an important research problem and the literature has exploded in recent years.

Original: See e.g., some latest surveys
Revised: See e.g., some latest surveys

Original: Chouldechova and Roth [2018], Mehrabi et al.
Revised: Chouldechova and Roth [2018], Mehrabi et al.

Original: [2021], Pessach and Shmueli [2022] and references therein.
Revised: [2021], Pessach and Shmueli [2022] and references therein.

Original: Model fairness has been studied at both group level and individual level.
Revised: Model fairness has been studied at both group level and individual level.

Original: Roughly speaking, group fairness requires model outputs to have small disparity across different groups of people, while individual fairness requires model outputs to be similar on similar individuals.
Revised: Roughly speaking, group fairness requires model outputs to have small disparity across different groups of people, while individual fairness requires model outputs to be similar on similar individuals.

Original: This paper focuses on individual fairness.
Revised: This paper focuses on individual fairness.

Original: Individual fairness was initially formalized as the Lipschitz condition of a prediction model Dwork et al.
Revised: Individual fairness was initially formalized as the Lipschitz condition of a prediction model Dwork et al.

Original: [2012], and later relaxed to a probabilistic and almost Lipschitz condition called approximate metric-fairness Yona and Rothblum [2018].
Revised: [2012], and later relaxed to a probabilistic and almost Lipschitz condition called approximate metric-fairness Yona and Rothblum [2018].

Original: There are many studies on different aspects of individual fairness such as how to design the fairness metric
Revised: There are many studies on different aspects of individual fairness such as how to design the fairness metric

Original: Ilvento [2020], Mukherjee et al.
Revised: Ilvento [2020], Mukherjee et al.

Original: [2020], how to achieve fairness with limited resources Kim et al.
Revised: [2020], how to achieve fairness with limited resources Kim et al.

Original: [2018], Bechavod et al.
Revised: [2018], Bechavod et al.

Original: [2020], and how to combine individual fairness with group fairness Zemel et al.
Revised: [2020], and how to combine individual fairness with group fairness Zemel et al.

Original: [2013], Sharifi-Malvajerdi et al.
Revised: [2013], Sharifi-Malvajerdi et al.

Original: The sample complexity for achieving individual fairness in passive learning is studied in Balashankar and Lees [2019], Shabat et al.
Revised: The sample complexity for achieving individual fairness in passive learning is studied in Balashankar and Lees [2019], Shabat et al.

Original: This paper studies a new aspect of individual fairness.
Revised: This paper studies a new aspect of individual fairness.

Original: We ask can one obtain a more efficient sample complexity for achieving individual fairness through active learning?
Revised: We ask can one obtain a more efficient sample complexity for achieving individual fairness through active learning?

Original: To our knowledge, all prior studies focus on the passive setting and maintain an O ( 1 ε 2 ) sample complexity for bounding (properly defined) individual bias by ε .
Revised: To our knowledge, all prior studies focus on the passive setting and maintain an O ( 1 ε 2 ) sample complexity for bounding (properly defined) individual bias by ε .

Original: See Yona and Rothblum [2018], Balashankar and Lees [2019], Shabat et al.
Revised: See Yona and Rothblum [2018], Balashankar and Lees [2019], Shabat et al.

Original: [2020] for example.
Revised: [2020] for example.

Original: In this paper, we show it can be improved to O (log 1 ε ) through active learning.
Revised: In this paper, we show it can be improved to O (log 1 ε ) through active learning.

Original: To facilitate algorithm design and analysis, we first present a new form of approximate metric-fairness (AMF) based on uniform continuity and prove its equivalence to the original form using an interesting connection between uniform continuity and almost Lipschitz continuity Vanderbei [1991].
Revised: To facilitate algorithm design and analysis, we first present a new form of approximate metric-fairness (AMF) based on uniform continuity and prove its equivalence to the original form using an interesting connection between uniform continuity and almost Lipschitz continuity Vanderbei [1991].

Original: Based on this, we present a passive AMF learner and prove the generalization ability of its achieve fairness.
Revised: Based on this, we present a passive AMF learner and prove the generalization ability of its achieve fairness.

Original: Building on the above, we design an active AMF learner which labels instances that are fairly close to their neighbors but receive fairly different predictions.
Revised: Building on the above, we design an active AMF learner which labels instances that are fairly close to their neighbors but receive fairly different predictions.

Original: Under proper conditions, we prove this learner only takes O (log 1 ε ) labeling to bound the bias of its returned model by ε with high probability.
Revised: Under proper conditions, we prove this learner only takes O (log 1 ε ) labeling to bound the bias of its returned model by ε with high probability.

Original: Our analysis assumes boundness of a specially designed counter approximate metric-fairness coefficient, and we exemplify the calculation of it.
Revised: Our analysis assumes boundness of a specially designed counter approximate metric-fairness coefficient, and we exemplify the calculation of it.

Original: At the end, we experiment the proposed active AMF learner on three real-world data sets.
Revised: At the end, we experiment the proposed active AMF learner on three real-world data sets.

Original: We observe it improves individual fairness of both linear and non-linear models more efficiently than its passive counterpart as well as state-of-theart active learners while maintaining a comparable accuracy, achieving a more efficient fairness-accuracy trade-off.
Revised: We observe it improves individual fairness of both linear and non-linear models more efficiently than its passive counterpart as well as state-of-theart active learners while maintaining a comparable accuracy, achieving a more efficient fairness-accuracy trade-off.

Original: The rest of this paper is organized as follows.
Revised: The rest of this paper is organized as follows.

Original: We review related topics in Section 2 and present the proposed AMF form and passive learner in Section 3.
Revised: We review related topics in Section 2 and present the proposed AMF form and passive learner in Section 3.

Original: In section 4, we present the proposed active AMF learner and prove its sample complexity.
Revised: In section 4, we present the proposed active AMF learner and prove its sample complexity.

Original: We discuss several implementation issues in Section 5, present experimental results in Section 6, and conclude the study in Section 7.
Revised: We discuss several implementation issues in Section 5, present experimental results in Section 6, and conclude the study in Section 7.

Original: Proofs of certain theoretical results are elaborated in the supplementary material.
Revised: Proofs of certain theoretical results are elaborated in the supplementary material.

Original:  BACKGROUND 2.1 FAIRNESS IN MACHINE LEARNING
Revised:  BACKGROUND 2.1 FAIRNESS IN MACHINE LEARNING

Original: Today, machine-learned models are widely used in sensitive domains like healthcare and hiring, and it is imperative for them to give fair assessment on human candidates.
Revised: Today, machine-learned models are widely used in sensitive domains like healthcare and hiring, and it is imperative for them to give fair assessment on human candidates.

Original: Take hiring as an example, when a model is used to score the qualification of job candidates, for fairness it should give similar scores to similar candidates disregarding their race or gender (i.e. no racial or gender discrimination).
Revised: Take hiring as an example, when a model is used to score the qualification of job candidates, for fairness it should give similar scores to similar candidates disregarding their race or gender (i.e. no racial or gender discrimination).

Original: In reality, however, many model assessments are considered unfair Feller et al.
Revised: In reality, however, many model assessments are considered unfair Feller et al.

Original: [2016], Chan and Wang [2018].
Revised: [2016], Chan and Wang [2018].

Original: This has motivated intensive research on how to attain fairness in machine-learned models e.g. Grgic-Hlaca et al.
Revised: This has motivated intensive research on how to attain fairness in machine-learned models e.g. Grgic-Hlaca et al.

Original: Alabi et al.
Revised: Alabi et al.

Original: Grgi ´ c-Hla ˇ ca et al.
Revised: Grgi ´ c-Hla ˇ ca et al.

Original: [2018], Rothblum and
Revised: [2018], Rothblum and

Original: Yona [2018], Mozannar et al.
Revised: Yona [2018], Mozannar et al.

Original: [2020], to name a few.
Revised: [2020], to name a few.

Original: Model fairness has been studied at both group and individual levels Dwork et al.
Revised: Model fairness has been studied at both group and individual levels Dwork et al.

Original: In this paper, we focus on individual fairness which, roughly speaking, requires model output to be similar on similar individuals.
Revised: In this paper, we focus on individual fairness which, roughly speaking, requires model output to be similar on similar individuals.

Original: Individual fairness is first formalized as the Lipschitz condition of the model Dwork et al.
Revised: Individual fairness is first formalized as the Lipschitz condition of the model Dwork et al.

Original: [2012], and later relaxed to a probabilistic and almost Lipschitz condition called Approximate Metric-Fairness (AMF)
Revised: [2012], and later relaxed to a probabilistic and almost Lipschitz condition called Approximate Metric-Fairness (AMF)

Original: Yona and Rothblum [2018]
Revised: Yona and Rothblum [2018]

Original: Many later studies are built on AMF Balashankar and Lees [2019], Bechavod et al.
Revised: Many later studies are built on AMF Balashankar and Lees [2019], Bechavod et al.

Original: [2020], Kim et al.
Revised: [2020], Kim et al.

Original: The active fair learner proposed in this paper is also built on AMF, but we present a new and provably equivalent form based on uniform continuity instead of almost Lipschitz.
Revised: The active fair learner proposed in this paper is also built on AMF, but we present a new and provably equivalent form based on uniform continuity instead of almost Lipschitz.

Original: One research direction in individual fairness is to attain a proper metric for evaluating individual similarity Ilvento [2020], Mukherjee et al.
Revised: One research direction in individual fairness is to attain a proper metric for evaluating individual similarity Ilvento [2020], Mukherjee et al.

Original: In this paper, we assume a metric is given and focus on how to achieve individual fairness efficiently through active learning.
Revised: In this paper, we assume a metric is given and focus on how to achieve individual fairness efficiently through active learning.

Original: Finally, to our knowledge, existing studies on individual fairness focus on the passive setting, where training data are randomly labeled.
Revised: Finally, to our knowledge, existing studies on individual fairness focus on the passive setting, where training data are randomly labeled.

Original: Their typical sample complexity for achieving individual fairness is O
Revised: Their typical sample complexity for achieving individual fairness is O

Original: ( 1 ε 2 )
Revised: ( 1 ε 2 )

Original: Yona and Rothblum [2018], Balashankar and Lees [2019], Shabat et al.
Revised: Yona and Rothblum [2018], Balashankar and Lees [2019], Shabat et al.

Original: In this paper, we focus on the active setting, where training data are strategically labeled.
Revised: In this paper, we focus on the active setting, where training data are strategically labeled.

Original: We show the proposed active
Revised: We show the proposed active

Original: AMF learner admits an O (log 1 ε ) sample complexity, which substantially improves the state-of-the-art result.
Revised: AMF learner admits an O (log 1 ε ) sample complexity, which substantially improves the state-of-the-art result.

Original: 2.2 ACTIVE LEARNING
Revised: 2.2 ACTIVE LEARNING

Original: Active learning has been extensively studied in the literature
Revised: Active learning has been extensively studied in the literature

Original: Settles [2009], Aggarwal et al.
Revised: Settles [2009], Aggarwal et al.

Original: [2014], Hanneke et al.
Revised: [2014], Hanneke et al.

Original: Given a supervised learner, active learning assumes labels of the training data are expensive to query and aims to minimize the query cost by strategically labeling a few data for efficiently improving model accuracy.
Revised: Given a supervised learner, active learning assumes labels of the training data are expensive to query and aims to minimize the query cost by strategically labeling a few data for efficiently improving model accuracy.

Original: For example, the uncertainty-based strategy labels data with uncertain model predictions, and the query-by-committee strategy labels data receiving disagreed predictions from a committee of models.
Revised: For example, the uncertainty-based strategy labels data with uncertain model predictions, and the query-by-committee strategy labels data receiving disagreed predictions from a committee of models.

Original: Active labeling strategies have been successfully applied in many domains and shown to improve model accuracy more efficiently than random labeling Thompson et al.
Revised: Active labeling strategies have been successfully applied in many domains and shown to improve model accuracy more efficiently than random labeling Thompson et al.

Original: Warmuth et al.
Revised: Warmuth et al.

Original: [2003], Liu [2004], Hoi et al.
Revised: [2003], Liu [2004], Hoi et al.

Original: [2006], Abe et al.
Revised: [2006], Abe et al.

Original: [2006], Zhao and Hoi [2013].
Revised: [2006], Zhao and Hoi [2013].

Original: On the theory side, active labeling can allows one to learn a model with ε error by labeling only O (log 1 ε ) instances, and this is more efficient than random labeling which requires O ( 1 ε ) instances to achieve the same error guarantee
Revised: On the theory side, active labeling can allows one to learn a model with ε error by labeling only O (log 1 ε ) instances, and this is more efficient than random labeling which requires O ( 1 ε ) instances to achieve the same error guarantee

Original: Dasgupta [2005], Hanneke [2007], Balcan et al.
Revised: Dasgupta [2005], Hanneke [2007], Balcan et al.

Original: We notice that most active labeling strategies are designed for classification model but very few are for regression model Burbidge et al.
Revised: We notice that most active labeling strategies are designed for classification model but very few are for regression model Burbidge et al.

Original: [2007], Sugiyama and Nakajima [2009], Cai et al.
Revised: [2007], Sugiyama and Nakajima [2009], Cai et al.

Original: [2013], Yu and Kim [2010].
Revised: [2013], Yu and Kim [2010].

Original: To our knowledge, the state-of-the-art strategy for regression model is greedy sampling Wu et al.
Revised: To our knowledge, the state-of-the-art strategy for regression model is greedy sampling Wu et al.

Original: [2019], which labels data that are most different from the already labeled training data in both feature space and label space.
Revised: [2019], which labels data that are most different from the already labeled training data in both feature space and label space.

Original: Our study is related to active learning but differs in that they focus on improving accuracy for traditional learners, while we focus on improving individual fairness for AMF learners.
Revised: Our study is related to active learning but differs in that they focus on improving accuracy for traditional learners, while we focus on improving individual fairness for AMF learners.

Original: Despite the difference, our work is inspired by disagreement-based active learning Hanneke [2007].
Revised: Despite the difference, our work is inspired by disagreement-based active learning Hanneke [2007].

Original: 2.3 FAIRNESS IN ACTIVE LEARNING
Revised: 2.3 FAIRNESS IN ACTIVE LEARNING

Original: The intersection of fairness and active learning is a fairly new research direction, and existing studies can be roughly grouped into active labeling Anahideh et al.
Revised: The intersection of fairness and active learning is a fairly new research direction, and existing studies can be roughly grouped into active labeling Anahideh et al.

Original: [2020], Sharaf and Daumé III [2020] and adaptive sampling Abernethy et al.
Revised: [2020], Sharaf and Daumé III [2020] and adaptive sampling Abernethy et al.

Original: [2020], Shekhar et al.
Revised: [2020], Shekhar et al.

Original: This paper considers the active labeling setting, but differs from the existing study as they focus on improving group fairness for standard learner while we focus on improving individual fairness for AMF learner.
Revised: This paper considers the active labeling setting, but differs from the existing study as they focus on improving group fairness for standard learner while we focus on improving individual fairness for AMF learner.

Original: Besides, we are the first work that shows active learning can improve the sample complexity for individual fairness to O (log 1 ε ) , which is not presented in prior studies.
Revised: Besides, we are the first work that shows active learning can improve the sample complexity for individual fairness to O (log 1 ε ) , which is not presented in prior studies.

Original:  APPROXIMATE METRIC-FAIRNESS
Revised:  APPROXIMATE METRIC-FAIRNESS

Original: In this section, we present a new form of approximate metricfairness (AMF) and prove its equivalence to the original form.
Revised: In this section, we present a new form of approximate metricfairness (AMF) and prove its equivalence to the original form.

Original: Then we present the corresponding passive AMF learner and prove its generalization guarantee.
Revised: Then we present the corresponding passive AMF learner and prove its generalization guarantee.

Original: Our study is focused on the regression problem.
Revised: Our study is focused on the regression problem.

Original: Let X be an instance space equipped with a metric d and distribution D .
Revised: Let X be an instance space equipped with a metric d and distribution D .

Original: Let H be a class of models defined on X .
Revised: Let H be a class of models defined on X .

Original: The original form of AMF Yona and Rothblum [2018] is defined based on almost Lipschitz continuity, as follows.
Revised: The original form of AMF Yona and Rothblum [2018] is defined based on almost Lipschitz continuity, as follows.

Original: Definition 3.1.
Revised: Definition 3.1.

Original: A model h ∈ H is said to be ( ε, β ) approximately metric-fair with respect to d and D if
Revised: A model h ∈ H is said to be ( ε, β ) approximately metric-fair with respect to d and D if

Original: [Equation]
Revised: [Equation]

Original: To facilitate algorithm design and analysis, we propose the following new form of AMF based on uniform continuity.
Revised: To facilitate algorithm design and analysis, we propose the following new form of AMF based on uniform continuity.

Original: Definition 3.2.
Revised: Definition 3.2.

Original: A model h ∈ H is said to be ( α, β, ε ) approximately metric-fair with respect to d and D if
Revised: A model h ∈ H is said to be ( α, β, ε ) approximately metric-fair with respect to d and D if

Original: [Equation]
Revised: [Equation]

Original: Intuitively, the new form models individual fairness by stating that if two individuals x and x ′ are similar (in a sense that d ( x, x ′ ) ≤ α ), then their predictions should be similar (in a sense that | h ( x ) − h ( x ′ ) | > β ) with high probability.
Revised: Intuitively, the new form models individual fairness by stating that if two individuals x and x ′ are similar (in a sense that d ( x, x ′ ) ≤ α ), then their predictions should be similar (in a sense that | h ( x ) − h ( x ′ ) | > β ) with high probability.

Original: Our following theorem suggests the two forms are equivalent, and is inspired by an interesting discovery that uniform continuity is almost Lipschitz Vanderbei [1991].
Revised: Our following theorem suggests the two forms are equivalent, and is inspired by an interesting discovery that uniform continuity is almost Lipschitz Vanderbei [1991].

Original: Theorem 3.3.
Revised: Theorem 3.3.

Original: Fix any α, β > 0 .
Revised: Fix any α, β > 0 .

Original: Any model with a convex domain is ( ε, β ) approximately metric-fair with respect to d and D
Revised: Any model with a convex domain is ( ε, β ) approximately metric-fair with respect to d and D

Original: if it is ( α, β, ε ) approximately metric-fair with respect to metric d ′ =
Revised: if it is ( α, β, ε ) approximately metric-fair with respect to metric d ′ =

Original: α 2 β · d and D , and only if it is ( α, 3 β, ε ) approximately metric-fair with respect to d ′ and D .
Revised: α 2 β · d and D , and only if it is ( α, 3 β, ε ) approximately metric-fair with respect to d ′ and D .

Original: Proof.
Revised: Proof.

Original: Let h be a model with a convex domain.
Revised: Let h be a model with a convex domain.

Original: Define two sets Ψ 1 ( β ) =
Revised: Define two sets Ψ 1 ( β ) =

Original: ( x, x ′ ) | | h ( x ) − h ( x ′ )
Revised: ( x, x ′ ) | | h ( x ) − h ( x ′ )

Original: | ≤ d ( x, x ′ )+ β } and Ψ 2 ( α, β ) = { ( x, x ′ )
Revised: | ≤ d ( x, x ′ )+ β } and Ψ 2 ( α, β ) = { ( x, x ′ )

Original: | d ′ ( x, x ′ ) ≤ α ⇒
Revised: | d ′ ( x, x ′ ) ≤ α ⇒

Original: | h ( x ) − h ( x ′ )
Revised: | h ( x ) − h ( x ′ )

Original: | ≤ β } , where ‘ ⇒ ’ means ‘imply’.
Revised: | ≤ β } , where ‘ ⇒ ’ means ‘imply’.

Original: We first prove
Revised: We first prove

Original: [Equation]
Revised: [Equation]

Original: The left relation holds because, for any β , if there exists an α such that d ′ ( x, x ′ ) ≤ α implies | h ( x ) − h ( x ′ ) | ≤ β , then [Vanderbei, 1991, Theorem 1] implies that | h ( x ) − h ( x ′ )
Revised: The left relation holds because, for any β , if there exists an α such that d ′ ( x, x ′ ) ≤ α implies | h ( x ) − h ( x ′ ) | ≤ β , then [Vanderbei, 1991, Theorem 1] implies that | h ( x ) − h ( x ′ )

Original: | ≤ d ( x, x ′ ) + β , where d ( x, x ′ )
Revised: | ≤ d ( x, x ′ ) + β , where d ( x, x ′ )

Original: = 2 βα d ′ ( x, x ′ ) .
Revised: = 2 βα d ′ ( x, x ′ ) .

Original: The right relation follows from the fact that, if | h ( x ) − h ( x ′ ) |
Revised: The right relation follows from the fact that, if | h ( x ) − h ( x ′ ) |

Original: ≤ d ( x, x ′ )+ β = 2 βα d ′ ( x, x ′ )+ β , then d ′ ( x, x ′ ) ≤ α implies | h ( x ) − h ( x ′ )
Revised: ≤ d ( x, x ′ )+ β = 2 βα d ′ ( x, x ′ )+ β , then d ′ ( x, x ′ ) ≤ α implies | h ( x ) − h ( x ′ )

Original: | ≤ 2 βα · α + β = 3 β .
Revised: | ≤ 2 βα · α + β = 3 β .

Original: Then, by contrapositive, (3) implies
Revised: Then, by contrapositive, (3) implies

Original: [Equation]
Revised: [Equation]

Original: where ˜Ψ denotes the complement of Ψ .
Revised: where ˜Ψ denotes the complement of Ψ .

Original: This further implies
Revised: This further implies

Original: Pr { ˜Ψ 2 ( α, β ) } ≥ Pr { ˜Ψ
Revised: Pr { ˜Ψ 2 ( α, β ) } ≥ Pr { ˜Ψ

Original: [Equation]
Revised: [Equation]

Original: ( β ) } ≥ Pr { ˜Ψ 2 ( α, 3 β ) } , and the theorem follows by the two definitions of fairness.
Revised: ( β ) } ≥ Pr { ˜Ψ 2 ( α, 3 β ) } , and the theorem follows by the two definitions of fairness.

Original: Theorem 3.3 shows one form of AMF converges to the other as β decreases, which establishes an equivalence between them.
Revised: Theorem 3.3 shows one form of AMF converges to the other as β decreases, which establishes an equivalence between them.

Original: It also suggests we can achieve one form of AMF through the other.
Revised: It also suggests we can achieve one form of AMF through the other.

Original: In the rest of this paper, we will design and analyze AMF learners based on Definition 3.2.
Revised: In the rest of this paper, we will design and analyze AMF learners based on Definition 3.2.

Original: For conciseness, we will omit the subscripts in Pr x,x ′ ∼ D whenever they are clear from the context.
Revised: For conciseness, we will omit the subscripts in Pr x,x ′ ∼ D whenever they are clear from the context.

Original: 3.1 AMF LEARNING WITH PERFORMANCE GENERALIZATION GUARANTEE
Revised: 3.1 AMF LEARNING WITH PERFORMANCE GENERALIZATION GUARANTEE

Original: In this section, we present a passive AMF learner based on
Revised: In this section, we present a passive AMF learner based on

Original: Definition 3.2 and prove its generalization guarantee.
Revised: Definition 3.2 and prove its generalization guarantee.

Original: To facilitate discussion, define the fairness measure
Revised: To facilitate discussion, define the fairness measure

Original: [Equation]
Revised: [Equation]

Original: Then h is said to be ( α, β
Revised: Then h is said to be ( α, β

Original: [Equation]
Revised: [Equation]

Original: -AMF
Revised: -AMF

Original: [Equation]
Revised: [Equation]

Original: ∆ α,β ( h ) ≤ ε .
Revised: ∆ α,β ( h ) ≤ ε .

Original: Let S be a sample of X × X with cardinality m .
Revised: Let S be a sample of X × X with cardinality m .

Original: An estimate of the probability ∆ α,β ( h ) on sample S is
Revised: An estimate of the probability ∆ α,β ( h ) on sample S is

Original: [Equation]
Revised: [Equation]

Original: where I is an indicator function.
Revised: where I is an indicator function.

Original: It is natural for AMF learning to find a model h with small ∆ α,β ( h ; S ) and hope this could generalize to a small ∆ α,β ( h ) .
Revised: It is natural for AMF learning to find a model h with small ∆ α,β ( h ; S ) and hope this could generalize to a small ∆ α,β ( h ) .

Original: In this paper, we focus on a realizable case where
Revised: In this paper, we focus on a realizable case where

Original: H contains perfect AMF models that satisfy ∆ α,β ( h )
Revised: H contains perfect AMF models that satisfy ∆ α,β ( h )

Original: = 0 .
Revised: = 0 .

Original: Based on this, we define the passive AMF learner as follows.
Revised: Based on this, we define the passive AMF learner as follows.

Original: Definition 3.4.
Revised: Definition 3.4.

Original: Given a hypothesis class H , a loss function ℓ and a labeled training set L = { ( x 1 , y n ) , . . .
Revised: Given a hypothesis class H , a loss function ℓ and a labeled training set L = { ( x 1 , y n ) , . . .

Original: , ( x n , y n ) } where x i is the i th instance and y i is its label, an AMF learner returns a model h ∈ H by solving min h
Revised: , ( x n , y n ) } where x i is the i th instance and y i is its label, an AMF learner returns a model h ∈ H by solving min h

Original: [Equation]
Revised: [Equation]

Original: β ( h ; S )
Revised: β ( h ; S )

Original: = 0 , (7) where S = { ( x i , x j ) } i,j =1 ,...,n .
Revised: = 0 , (7) where S = { ( x i , x j ) } i,j =1 ,...,n .

Original: We can show the above AMF learner has a similar generalization guarantee as in Yona and Rothblum [2018] based on the following lemma.
Revised: We can show the above AMF learner has a similar generalization guarantee as in Yona and Rothblum [2018] based on the following lemma.

Original: Let R m ( · ) denote the Rademacher complexity of some hypothesis class for sample size m .
Revised: Let R m ( · ) denote the Rademacher complexity of some hypothesis class for sample size m .

Original: Lemma 3.5.
Revised: Lemma 3.5.

Original: Fix any t, β > 0 .
Revised: Fix any t, β > 0 .

Original: Let F : X × X
Revised: Let F : X × X

Original: → R be a hypothesis class induced from H such that ∀ f ∈ F , f ( x, x ′ )
Revised: → R be a hypothesis class induced from H such that ∀ f ∈ F , f ( x, x ′ )

Original: =
Revised: =

Original: τ tβ ( | h ( x ) − h ( x ′ ) | ) where τ tβ ( z ) is a piecewise model outputting 1 if z > β + 1 t , outputting 0 if z ≤ β and t ( z − β ) otherwise.
Revised: τ tβ ( | h ( x ) − h ( x ′ ) | ) where τ tβ ( z ) is a piecewise model outputting 1 if z > β + 1 t , outputting 0 if z ≤ β and t ( z − β ) otherwise.

Original: Then R m ( F ) ≤ 8 t · R m ( H ) .
Revised: Then R m ( F ) ≤ 8 t · R m ( H ) .

Original: Proof Sketch.
Revised: Proof Sketch.

Original: Repeatedly apply the Rademacher complexity property of composite function with Lipschitz condition e.g. [Bartlett and Mendelson, 2002, Theorem 12] on τ tβ and abs .
Revised: Repeatedly apply the Rademacher complexity property of composite function with Lipschitz condition e.g. [Bartlett and Mendelson, 2002, Theorem 12] on τ tβ and abs .

Original: See the supplementary material for details.
Revised: See the supplementary material for details.

Original: Based on the above, we can prove the proposed AMF learner has generalization guarantee based on an assumption that instances are sampled i.i.d..
Revised: Based on the above, we can prove the proposed AMF learner has generalization guarantee based on an assumption that instances are sampled i.i.d..

Original: The results is as follows.
Revised: The results is as follows.

Original: Theorem 3.6.
Revised: Theorem 3.6.

Original: Fix any α, β, t > 0 .
Revised: Fix any α, β, t > 0 .

Original: Suppose R m ( H ) ∈
Revised: Suppose R m ( H ) ∈

Original: O (1 / √ m ) .
Revised: O (1 / √ m ) .

Original: Any model h ∈ H returned by the AMF learner satisfies ∆ α,β
Revised: Any model h ∈ H returned by the AMF learner satisfies ∆ α,β

Original: +1 /t
Revised: +1 /t

Original: ( h ) ≤ ε with probability at least 1 − δ if m ≥ 

Revised: ( h ) ≤ ε with probability at least 1 − δ if m ≥

Original: ε 2 (cid:16) 16 tc + (cid:113) 12 log 1 δ (cid:17) , where m is the number of ( x, x ′ ) ∈ S satisfying d ( x, x ′ )
Revised: ε 2 (cid:16) 16 tc + (cid:113) 12 log 1 δ (cid:17) , where m is the number of ( x, x ′ ) ∈ S satisfying d ( x, x ′ )

Original: ≤
Revised: ≤

Original: α and c is a constant inherited from O (1 / √ m ) .
Revised: α and c is a constant inherited from O (1 / √ m ) .

Original: Proof Sketch.
Revised: Proof Sketch.

Original: The main challenge in our analysis is an extra d ( x, x ′ ) ≤ α term that cannot be directly removed using the Rademacher complexity property as in Lemma 3.5.
Revised: The main challenge in our analysis is an extra d ( x, x ′ ) ≤ α term that cannot be directly removed using the Rademacher complexity property as in Lemma 3.5.

Original: To tackle this, we introduce V = { ( x, x ′ ) ∈ S ; d ( x, x ′ ) ≤ α } .
Revised: To tackle this, we introduce V = { ( x, x ′ ) ∈ S ; d ( x, x ′ ) ≤ α } .

Original: We will first transform the analysis of joint event | h ( a ) − h ( b ) | > β and d ( a, b ) ≤ to an analysis of single event | h ( a ) − h ( b ) | > β by narrowing the domain to V .
Revised: We will first transform the analysis of joint event | h ( a ) − h ( b ) | > β and d ( a, b ) ≤ to an analysis of single event | h ( a ) − h ( b ) | > β by narrowing the domain to V .

Original: Then, we derive a generalization bound for the single event by first relaxing its indicator function to the piecewise function defined in Lemma 3.5, then applying the standard generalization argument with R m ( F ) e.g., Mohri et al.
Revised: Then, we derive a generalization bound for the single event by first relaxing its indicator function to the piecewise function defined in Lemma 3.5, then applying the standard generalization argument with R m ( F ) e.g., Mohri et al.

Original: [2018], and finally connecting R m ( F ) to R m ( H ) using Lemma 3.5.
Revised: [2018], and finally connecting R m ( F ) to R m ( H ) using Lemma 3.5.

Original: At the end, we transform the result for the single event back to a result for the joint event which completes the proof.
Revised: At the end, we transform the result for the single event back to a result for the joint event which completes the proof.

Original: See the supplementary material for details.
Revised: See the supplementary material for details.

Original: Theorem 3.6 implies one can achieve ( α, β, ε ) AMF with
Revised: Theorem 3.6 implies one can achieve ( α, β, ε ) AMF with

Original: O ( 1 ε 2 ) randomly labeled instances, which is consistent with the sample complexity in Yona and Rothblum [2018].
Revised: O ( 1 ε 2 ) randomly labeled instances, which is consistent with the sample complexity in Yona and Rothblum [2018].

Original: Constant c depends on the hypothesis class e.g., if H is the set of linear models with proper constraints, we can set c to the maximum norm of the instance Shalev-Shwartz and BenDavid [2014]; if H is the set of kernel machines with proper constraints, we can set c to the product of kernel function bound and gram matrix trace Mohri et al.
Revised: Constant c depends on the hypothesis class e.g., if H is the set of linear models with proper constraints, we can set c to the maximum norm of the instance Shalev-Shwartz and BenDavid [2014]; if H is the set of kernel machines with proper constraints, we can set c to the product of kernel function bound and gram matrix trace Mohri et al.

Original: In the theorem, variable t is the slope of a Lipschitz function introduced to approximate the indicator function.
Revised: In the theorem, variable t is the slope of a Lipschitz function introduced to approximate the indicator function.

Original: Its impact do
Revised: Its impact do

Original: Learn the
Revised: Learn the

Original: AMF in
Revised: AMF in

Original: Pick satisfying ∃
Revised: Pick satisfying ∃

Original: [Equation]
Revised: [Equation]

Original: Label the selected instances.
Revised: Label the selected instances.

Original: Then add them to sample L , and remove them from sample U .
Revised: Then add them to sample L , and remove them from sample U .

Original: 5: end while
Revised: 5: end while

Original: Output on the error bound is interesting twofold.
Revised: Output on the error bound is interesting twofold.

Original: A smaller t leads to a weaker fairness guarantee, in a sense that ∆ α,β +1 /t ′
Revised: A smaller t leads to a weaker fairness guarantee, in a sense that ∆ α,β +1 /t ′

Original: ≤ ε implies ∆ α,β +1 /t ≤ ε whenever t ′ ≤ t .
Revised: ≤ ε implies ∆ α,β +1 /t ≤ ε whenever t ′ ≤ t .

Original: But it also leads to higher sample efficiency, in a sense that a smaller t implies smaller m suffices for the generalization guarantee.
Revised: But it also leads to higher sample efficiency, in a sense that a smaller t implies smaller m suffices for the generalization guarantee.

Original:  ACTIVE AMF LEARNING
Revised:  ACTIVE AMF LEARNING

Original: In this section, we propose an active AMF learner based on
Revised: In this section, we propose an active AMF learner based on

Original: Definition 3.4 and derive its sample complexity.
Revised: Definition 3.4 and derive its sample complexity.

Original: Our key idea is to label instances that are fairly close to their neighbors but receive fairly different predictions from some hypothesis.
Revised: Our key idea is to label instances that are fairly close to their neighbors but receive fairly different predictions from some hypothesis.

Original: We characterize such instances using a set
Revised: We characterize such instances using a set

Original: [Equation]
Revised: [Equation]

Original: Next, we design a counter AMF coefficient, which will be used to derive the complexity.
Revised: Next, we design a counter AMF coefficient, which will be used to derive the complexity.

Original: Definition 4.1.
Revised: Definition 4.1.

Original: The counter ( α, β ) AMF coefficient with respect to a hypothesis class H is
Revised: The counter ( α, β ) AMF coefficient with respect to a hypothesis class H is

Original: [Equation]
Revised: [Equation]

Original: where B α,β ( r ) = { h ∈ H ; ∆ α,β ( h ) ≤ r } is the set of hypotheses that are ( α, β, r ) AMF.
Revised: where B α,β ( r ) = { h ∈ H ; ∆ α,β ( h ) ≤ r } is the set of hypotheses that are ( α, β, r ) AMF.

Original: Intuitively, the coefficient measures the largest volume of instance pairs that do not contribute to the fairness achievable in a hypothesis class.
Revised: Intuitively, the coefficient measures the largest volume of instance pairs that do not contribute to the fairness achievable in a hypothesis class.

Original: We could expect it to be smaller if hypotheses are more fair.
Revised: We could expect it to be smaller if hypotheses are more fair.

Original: For conciseness, we will omit the subscripts in ξ α,β whenever they are clear from the context.
Revised: For conciseness, we will omit the subscripts in ξ α,β whenever they are clear from the context.

Original: The proposed active AMF learner is shown in Algorithm 1.
Revised: The proposed active AMF learner is shown in Algorithm 1.

Original: In each round, it trains model h on the labeled set using the
Revised: In each round, it trains model h on the labeled set using the

Original: AMF learner, and then labels instances that are close to the training data but receive different predictions from h .
Revised: AMF learner, and then labels instances that are close to the training data but receive different predictions from h .

Original: It is clear that all labeled instances fall in C α,β ( H ) .
Revised: It is clear that all labeled instances fall in C α,β ( H ) .

Original: The fairness coefficients α, β are assumed preset by the problem, and we can stop labeling when a desired AMF degree is achieved.
Revised: The fairness coefficients α, β are assumed preset by the problem, and we can stop labeling when a desired AMF degree is achieved.

Original: Our following theorem shows that, under proper conditions,
Revised: Our following theorem shows that, under proper conditions,

Original: Algorithm 1 can return a model satisfying ( α, β, ε ) AMF through O (log 1 ε ) labeling with high probability.
Revised: Algorithm 1 can return a model satisfying ( α, β, ε ) AMF through O (log 1 ε ) labeling with high probability.

Original: Theorem 4.2.
Revised: Theorem 4.2.

Original: Fix any α, β > 0 .
Revised: Fix any α, β > 0 .

Original: If the counter ( α, β ) AMF coefficient w.r.t.
Revised: If the counter ( α, β ) AMF coefficient w.r.t.

Original: H is bounded, then with probability at least 1 − δ , any h ∈ H returned by Algorithm 1 satisfies ∆ α,β ( h ) ≤ ε after O (log 1 ε ) labeling.
Revised: H is bounded, then with probability at least 1 − δ , any h ∈ H returned by Algorithm 1 satisfies ∆ α,β ( h ) ≤ ε after O (log 1 ε ) labeling.

Original: Proof Sketch.
Revised: Proof Sketch.

Original: Let V q = { h ∈ H ; ∆ α,β ( h ; S q ) = 0 } be the set of ‘perfect’ AMF models at the end of q rounds of labeling.
Revised: Let V q = { h ∈ H ; ∆ α,β ( h ; S q ) = 0 } be the set of ‘perfect’ AMF models at the end of q rounds of labeling.

Original: The goal of our analysis is to show that, if we label k =
Revised: The goal of our analysis is to show that, if we label k =

Original: [Equation]
Revised: [Equation]

Original: (cid:16)
Revised: (cid:16)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: instances in each round, then by the generalization bound in Theorem 3.6, there is
Revised: instances in each round, then by the generalization bound in Theorem 3.6, there is

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: with high probability.
Revised: with high probability.

Original: This implies Q = log 2 1 ε rounds of labeling, which means Qk ∈ O (log 1 ε ) total labeling, suffices for Pr {C α,β ( V q +1 ) } ≤ ε .
Revised: This implies Q = log 2 1 ε rounds of labeling, which means Qk ∈ O (log 1 ε ) total labeling, suffices for Pr {C α,β ( V q +1 ) } ≤ ε .

Original: Since ∆ α,β ( h ) ≤ Pr {C α,β ( V q ) } for any h ∈ V q by definition, the theorem is proved.
Revised: Since ∆ α,β ( h ) ≤ Pr {C α,β ( V q ) } for any h ∈ V q by definition, the theorem is proved.

Original: Let & be logic ‘AND’ and define event
Revised: Let & be logic ‘AND’ and define event

Original: I
Revised: I

Original: [Equation]
Revised: [Equation]

Original: A key to prove (11) is to split the domain of ∆ α,β ( h ) =
Revised: A key to prove (11) is to split the domain of ∆ α,β ( h ) =

Original: Pr { I βα ( x, x ′ ; h ) } for any h ∈ V q +1 into ( x, x ′ ) ∈
Revised: Pr { I βα ( x, x ′ ; h ) } for any h ∈ V q +1 into ( x, x ′ ) ∈

Original: C α,β ( V q ) and ( x, x ′ ) /
Revised: C α,β ( V q ) and ( x, x ′ ) /

Original: ∈ C α,β ( V q ) .
Revised: ∈ C α,β ( V q ) .

Original: Probability on the second subdomain is zero, and probability on the first subdomain can be bounded using Theorem 3.6 conditioned on the fact that all labeled instances fall in C α,β ( V q ) .
Revised: Probability on the second subdomain is zero, and probability on the first subdomain can be bounded using Theorem 3.6 conditioned on the fact that all labeled instances fall in C α,β ( V q ) .

Original: That bound is smaller than 12 ξ by our choice of k and the definition of ξ , therefore implying V q +1 ⊆ B (cid:16)
Revised: That bound is smaller than 12 ξ by our choice of k and the definition of ξ , therefore implying V q +1 ⊆ B (cid:16)

Original: Pr {C α,β ( V q ) } 2 ξ (cid:17) and thus
Revised: Pr {C α,β ( V q ) } 2 ξ (cid:17) and thus

Original: Pr {C α,β ( V q +1 ) } ≤ Pr (cid:110) C α,β (cid:16) B α,β (cid:16) Pr {C α,β ( V q ) } 2 ξ (cid:17)(cid:17)(cid:111) ≤ ξ · Pr {C α,β ( V q ) } 2 ξ =
Revised: Pr {C α,β ( V q +1 ) } ≤ Pr (cid:110) C α,β (cid:16) B α,β (cid:16) Pr {C α,β ( V q ) } 2 ξ (cid:17)(cid:17)(cid:111) ≤ ξ · Pr {C α,β ( V q ) } 2 ξ =

Original: Pr {C α,β ( V q ) }, where the second inequality is by definition.
Revised: Pr {C α,β ( V q ) }, where the second inequality is by definition.

Original: This proves (11) and thus the theorem.
Revised: This proves (11) and thus the theorem.

Original: The proof of Theorem 4.2 also illuminates the key for Algorithm 1 to reduce labeled instances is in Step 3, where we label u if ( u, u ′ )
Revised: The proof of Theorem 4.2 also illuminates the key for Algorithm 1 to reduce labeled instances is in Step 3, where we label u if ( u, u ′ )

Original: ∈
Revised: ∈

Original: C α,β ( V q ) because only such pair can be used to further rule out hypotheses in V q and shrink
Revised: C α,β ( V q ) because only such pair can be used to further rule out hypotheses in V q and shrink

Original: C α,β ( V q ) , which guarantees the shrinkage of ∆ α,β ( h ) .
Revised: C α,β ( V q ) , which guarantees the shrinkage of ∆ α,β ( h ) .

Original: We should mention an implicit assumption of the derived sample complexity is that, the unlabeled set contains at least one instance satisfying (8) per epoch until convergence.
Revised: We should mention an implicit assumption of the derived sample complexity is that, the unlabeled set contains at least one instance satisfying (8) per epoch until convergence.

Original: This is similar to the analysis of disagreement-based active learning Hanneke et al.
Revised: This is similar to the analysis of disagreement-based active learning Hanneke et al.

Original: [2014], which assumes at least one unlabeled instance is disagreed by the committee models per epoch.
Revised: [2014], which assumes at least one unlabeled instance is disagreed by the committee models per epoch.

Original: From a practical perspective, when no valid instance is found, we could train another model or randomly label one instance and proceed to the next epoch.
Revised: From a practical perspective, when no valid instance is found, we could train another model or randomly label one instance and proceed to the next epoch.

Original: We should also mention the time complexity for Algorithmto find an instance satisfying (8).
Revised: We should also mention the time complexity for Algorithmto find an instance satisfying (8).

Original: In a centralized computing environment, the complexity is O ( | U || L | ) , where | U | is the size of unlabeled set and | L | is the size of labeled set.
Revised: In a centralized computing environment, the complexity is O ( | U || L | ) , where | U | is the size of unlabeled set and | L | is the size of labeled set.

Original: Typically | L | ≪ | U | .
Revised: Typically | L | ≪ | U | .

Original: This is higher than the complexity of uncertainty-based strategy which is typically O ( | U | ) , but more comparable to the complexity of query-by-committee which is typically O ( | U | t ) for t committee models.
Revised: This is higher than the complexity of uncertainty-based strategy which is typically O ( | U | ) , but more comparable to the complexity of query-by-committee which is typically O ( | U | t ) for t committee models.

Original: In a distributed computing environment, the complexity can be reduced to O ( | U | ) if the evaluations of an instance u ∈ U with all u ′ ∈ L can be parallelized.
Revised: In a distributed computing environment, the complexity can be reduced to O ( | U | ) if the evaluations of an instance u ∈ U with all u ′ ∈ L can be parallelized.

Original: Nonetheless, how to make selection more efficient remains an open challenge.
Revised: Nonetheless, how to make selection more efficient remains an open challenge.

Original: 4.1 THE COUNTER AMF COEFFICIENT
Revised: 4.1 THE COUNTER AMF COEFFICIENT

Original: An important factor in our analysis is the counter AMF coefficient.
Revised: An important factor in our analysis is the counter AMF coefficient.

Original: We give an example on how to calcualte it.
Revised: We give an example on how to calcualte it.

Original: Example 4.3.
Revised: Example 4.3.

Original: Fix α, β, B > 0 .
Revised: Fix α, β, B > 0 .

Original: Let
Revised: Let

Original: h w ( x ) = w · x be a 1-dimensional linear hypothesis defined on [ − B, B ] , and define H
Revised: h w ( x ) = w · x be a 1-dimensional linear hypothesis defined on [ − B, B ] , and define H

Original: = { h w ; w ≥ 0 } .
Revised: = { h w ; w ≥ 0 } .

Original: Assume instances are uniformly distributed on [ − B, B ] .
Revised: Assume instances are uniformly distributed on [ − B, B ] .

Original: If B > α , then
Revised: If B > α , then

Original: [Equation]
Revised: [Equation]

Original: and the counter ( α, β ) AMF coefficient w.r.t.
Revised: and the counter ( α, β ) AMF coefficient w.r.t.

Original: to H is 1.
Revised: to H is 1.

Original: Proof.
Revised: Proof.

Original: The roadmap of this proof is as follows.
Revised: The roadmap of this proof is as follows.

Original: We will first derive ∆ α,β ( h w ) through case study and show it is nondecreasing with respect to w .
Revised: We will first derive ∆ α,β ( h w ) through case study and show it is nondecreasing with respect to w .

Original: Based on this, we will then argue the probability in (10) is equivalent to
Revised: Based on this, we will then argue the probability in (10) is equivalent to

Original: [Equation]
Revised: [Equation]

Original: 14) where h ∗ is the model satisfying ∆ α,β ( h
Revised: 14) where h ∗ is the model satisfying ∆ α,β ( h

Original: [Equation]
Revised: [Equation]

Original: = r , hence
Revised: = r , hence

Original: [Equation]
Revised: [Equation]

Original: Now we show the detailed proof.
Revised: Now we show the detailed proof.

Original: For conciseness, we will write h for h w but with the mind that each h is associated with a w .
Revised: For conciseness, we will write h for h w but with the mind that each h is associated with a w .

Original: Also, recall the event notation I βα ( x, x ′ ; h ) :
Revised: Also, recall the event notation I βα ( x, x ′ ; h ) :

Original: = d ( x, x ′ ) ≤ α & | h ( x ) − h ( x ′ ) | > β .
Revised: = d ( x, x ′ ) ≤ α & | h ( x ) − h ( x ′ ) | > β .

Original: Step
Revised: Step

Original: ∆
Revised: ∆

Original: any
Revised: any

Original: Fix any h .
Revised: Fix any h .

Original: Consider two cases.
Revised: Consider two cases.

Original: (i)
Revised: (i)

Original: If αw < β , simple geometric analysis shows that event I βα ( x, x ′ ; h ) is always false so Pr x,x ′ { I βα ( x, x ′ ; h ) } = 0 .
Revised: If αw < β , simple geometric analysis shows that event I βα ( x, x ′ ; h ) is always false so Pr x,x ′ { I βα ( x, x ′ ; h ) } = 0 .

Original: (ii)
Revised: (ii)

Original: If αw ≥ β (which implies w ̸ = 0 ), then α ≥ β/w .
Revised: If αw ≥ β (which implies w ̸ = 0 ), then α ≥ β/w .

Original: In this
Revised: In this

Original: case
Revised: case

Original: , we can
Revised: , we can

Original: properly
Revised: properly

Original: partition the
Revised: partition the

Original: domain and have
Revised: domain and have

Original: [Equation]
Revised: [Equation]

Original: x,x
Revised: x,x

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: x, x
Revised: x, x

Original: [Equation]
Revised: [Equation]

Original: where Pr x { I βα ( x, x ′ ; h ) } is the probability defined for x with a fixed x ′ .
Revised: where Pr x { I βα ( x, x ′ ; h ) } is the probability defined for x with a fixed x ′ .

Original: In (15), the first equality is by definition, and the second equality is by the observation that
Revised: In (15), the first equality is by definition, and the second equality is by the observation that

Original: Pr x { I βα ( x, x ′ ; h ) } is symmetric on [ − B, B ] (which will become more clear in later analysis).
Revised: Pr x { I βα ( x, x ′ ; h ) } is symmetric on [ − B, B ] (which will become more clear in later analysis).

Original: Note that p ( x ′ )
Revised: Note that p ( x ′ )

Original: = 12 B .
Revised: = 12 B .

Original: Now we study each integral separately.
Revised: Now we study each integral separately.

Original: (ii.a)
Revised: (ii.a)

Original: If x ′ ∈ [0 , B − α ] , we can show
Revised: If x ′ ∈ [0 , B − α ] , we can show

Original: [Equation]
Revised: [Equation]

Original: To verify this, let us first fix x ′ =
Revised: To verify this, let us first fix x ′ =

Original: and identify the set of x in [ − α, α ] that makes event I βα ( x, 0; h ) true.
Revised: and identify the set of x in [ − α, α ] that makes event I βα ( x, 0; h ) true.

Original: This case is illustrated in Figure 1.
Revised: This case is illustrated in Figure 1.

Original: We see all targeted x fall in [ β/w, α ] and (by symmetry) in [ − α, − β/w ] .
Revised: We see all targeted x fall in [ β/w, α ] and (by symmetry) in [ − α, − β/w ] .

Original: This implies
Revised: This implies

Original: Pr x { I βα ( x, 0; h ) } = 2 · ( α − β/w ) 2 · α = 1 − βαw .
Revised: Pr x { I βα ( x, 0; h ) } = 2 · ( α − β/w ) 2 · α = 1 − βαw .

Original: Since h is linear, the above result applies to all x ′ ∈ [0 , B − α ] , which implies (16) and thus the first integral equals to (1 − βwα )(1 − αB ) .
Revised: Since h is linear, the above result applies to all x ′ ∈ [0 , B − α ] , which implies (16) and thus the first integral equals to (1 − βwα )(1 − αB ) .

Original: Note it is non-negative since αw ≥ β and B > α .
Revised: Note it is non-negative since αw ≥ β and B > α .

Original: (ii.b)
Revised: (ii.b)

Original: If x ′ ∈ ( B − α, B − βw ] , we can show
Revised: If x ′ ∈ ( B − α, B − βw ] , we can show

Original: [Equation]
Revised: [Equation]

Original: We can verify this in a similar way as in (ii.a), with additional shift of the origin to x ′ and constraint x ≤ B .
Revised: We can verify this in a similar way as in (ii.a), with additional shift of the origin to x ′ and constraint x ≤ B .

Original: Then, geometric analysis suggests all targeted x fall in [ x ′ + βw , B ] (shorter than interval [ β/w, α ] in Figure 1) and thus Pr x { I βα ( x, x ′ ; h ) }
Revised: Then, geometric analysis suggests all targeted x fall in [ x ′ + βw , B ] (shorter than interval [ β/w, α ] in Figure 1) and thus Pr x { I βα ( x, x ′ ; h ) }

Original: = 2( B − x ′ − β/w ) 2( B − x ′ ) .
Revised: = 2( B − x ′ − β/w ) 2( B − x ′ ) .

Original: This implies the second integral is 1 B ( α − βw (1 − ln βwα )) .
Revised: This implies the second integral is 1 B ( α − βw (1 − ln βwα )) .

Original: Note it is non-negative as (17) is non-negative by the domain of x ′ .
Revised: Note it is non-negative as (17) is non-negative by the domain of x ′ .

Original: (ii.c)
Revised: (ii.c)

Original: If x ′ ∈ ( B − βw , B ] , it is easy to see no ( x ′ , x ) makes event I βα ( x, x ′ ; h ) true so
Revised: If x ′ ∈ ( B − βw , B ] , it is easy to see no ( x ′ , x ) makes event I βα ( x, x ′ ; h ) true so

Original: Pr x { I βα ( x, x ′ ; h ) } = 0 .
Revised: Pr x { I βα ( x, x ′ ; h ) } = 0 .

Original: Then the third integral is zero.
Revised: Then the third integral is zero.

Original: Plugging the integrals of (ii.a), (ii.b) and (ii.c) back to (15), and combining results of cases (i) and (ii) gives (13).
Revised: Plugging the integrals of (ii.a), (ii.b) and (ii.c) back to (15), and combining results of cases (i) and (ii) gives (13).

Original: Step 2: Show ∆
Revised: Step 2: Show ∆

Original: [Equation]
Revised: [Equation]

Original: h ) is non-decreasing w.r.t.
Revised: h ) is non-decreasing w.r.t.

Original: w .
Revised: w .

Original: All we need to show is ∆ α,β ( h ) is non-negative and nondecreasing when w ≥ βα .
Revised: All we need to show is ∆ α,β ( h ) is non-negative and nondecreasing when w ≥ βα .

Original: The first property is guaranteed since all integrals in (15) are non-negative.
Revised: The first property is guaranteed since all integrals in (15) are non-negative.

Original: To see the second property, take derivative ∂
Revised: To see the second property, take derivative ∂

Original: ∆ α,β
Revised: ∆ α,β

Original: [Equation]
Revised: [Equation]

Original: Since w ≥ βα and B > α , we can easily show the derivative is bigger than zero and hence ∆ α,β ( h ) is non-decreasing.
Revised: Since w ≥ βα and B > α , we can easily show the derivative is bigger than zero and hence ∆ α,β ( h ) is non-decreasing.

Original: Step 3: Equivalent Probability .
Revised: Step 3: Equivalent Probability .

Original: Let h ∗ =
Revised: Let h ∗ =

Original: w ∗ x be the model satisfying ∆ α,β ( h ∗ ) =
Revised: w ∗ x be the model satisfying ∆ α,β ( h ∗ ) =

Original: r .
Revised: r .

Original: It is not hard to show it exists for every r ∈ [0 , 1) based on (13).
Revised: It is not hard to show it exists for every r ∈ [0 , 1) based on (13).

Original: Then, results of Step 1 and Step 2 suggest B α,β ( r ) is the set of linear models satisfying w ≤ w ∗ , which implies
Revised: Then, results of Step 1 and Step 2 suggest B α,β ( r ) is the set of linear models satisfying w ≤ w ∗ , which implies

Original: [Equation]
Revised: [Equation]

Original: To verify this, we first show every ( x, x ′ ) ∈
Revised: To verify this, we first show every ( x, x ′ ) ∈

Original: C α,β ( B α,β ( r )) makes event I βα ( x, x ′ ; h ∗ ) true.
Revised: C α,β ( B α,β ( r )) makes event I βα ( x, x ′ ; h ∗ ) true.

Original: This is true because, for any x, x ′ with d ( x, x ′ ) ≤ α , if there exists an w ≤ w ∗ such that | wx − wx ′ | > β , then | w ∗ x − w ∗ x ′ | ≥ | wx − wx ′ | > β .
Revised: This is true because, for any x, x ′ with d ( x, x ′ ) ≤ α , if there exists an w ≤ w ∗ such that | wx − wx ′ | > β , then | w ∗ x − w ∗ x ′ | ≥ | wx − wx ′ | > β .

Original: We then show every ( x, x ′ ) that makes event I βα ( x, x ′ ; h ∗ ) true is also in C α,β ( B α,β ( r )) .
Revised: We then show every ( x, x ′ ) that makes event I βα ( x, x ′ ; h ∗ ) true is also in C α,β ( B α,β ( r )) .

Original: This is true since h ∗ exists.
Revised: This is true since h ∗ exists.

Original: The equivalence implies ξ = sup r> 

Revised: The equivalence implies ξ = sup r>

Original: Pr { I βα ( x,x ′ ; h ∗ ) } r = sup r> 0 rr = 1 .
Revised: Pr { I βα ( x,x ′ ; h ∗ ) } r = sup r> 0 rr = 1 .

Original: The proof is completed.
Revised: The proof is completed.

Original:  IMPLEMENTATION
Revised:  IMPLEMENTATION

Original: In this section, we discuss three implementation issues.
Revised: In this section, we discuss three implementation issues.

Original: The first issue is related to the AMF Learner in Definition 3.4.
Revised: The first issue is related to the AMF Learner in Definition 3.4.

Original: Directly solving (7) is not easy since ∆ α,β ( h ) is nonconvex.
Revised: Directly solving (7) is not easy since ∆ α,β ( h ) is nonconvex.

Original: We propose to approximate the solution by solving min
Revised: We propose to approximate the solution by solving min

Original: [Equation]
Revised: [Equation]

Original: instead, where λ is a regularization coefficient and ˜∆ α,β ( h
Revised: instead, where λ is a regularization coefficient and ˜∆ α,β ( h

Original: [Equation]
Revised: [Equation]

Original: with M being an n -by- n matrix whose entries are defined as M ij =
Revised: with M being an n -by- n matrix whose entries are defined as M ij =

Original: I { d ( x i , d j ) ≤ α } .
Revised: I { d ( x i , d j ) ≤ α } .

Original: Such approximation can be justified by the following relation, which implies that minimizing ˜∆ α,β ( h ; S ) also minimizes ∆ α,β ( h ; S ) .
Revised: Such approximation can be justified by the following relation, which implies that minimizing ˜∆ α,β ( h ; S ) also minimizes ∆ α,β ( h ; S ) .

Original: Lemma 5.1.
Revised: Lemma 5.1.

Original: Fix any α, β > 0 .
Revised: Fix any α, β > 0 .

Original: We have ∆ α,β ( h ; S ) ≤ ˜∆ α,β ( h ; S ) for any h ∈ S and sample S .
Revised: We have ∆ α,β ( h ; S ) ≤ ˜∆ α,β ( h ; S ) for any h ∈ S and sample S .

Original: In practice, the approximate learner (19) may not always return a model with zero bias on training data.
Revised: In practice, the approximate learner (19) may not always return a model with zero bias on training data.

Original: In this case, the proposed algorithm remains applicable and sample-efficient on fairness.
Revised: In this case, the proposed algorithm remains applicable and sample-efficient on fairness.

Original: There are two possible theoretical explanations on the maintained efficiency.
Revised: There are two possible theoretical explanations on the maintained efficiency.

Original: First, if the bias is sufficiently small e.g., ∆ α,β ( h ; S ) ∈ O ( ε ) , then the passive bound in
Revised: First, if the bias is sufficiently small e.g., ∆ α,β ( h ; S ) ∈ O ( ε ) , then the passive bound in

Original: Theorem 3.6 can be extended to ∆ α,
Revised: Theorem 3.6 can be extended to ∆ α,

Original: β ( h ) ∈
Revised: β ( h ) ∈

Original: O ( ε ) .
Revised: O ( ε ) .

Original: Plugging this back to Theorem 4.2, we can obtain a similar complexity with an additional constant factor.
Revised: Plugging this back to Theorem 4.2, we can obtain a similar complexity with an additional constant factor.

Original: Second, we may borrow ideas from agnostic active learning e.g., Dasgupta et al.
Revised: Second, we may borrow ideas from agnostic active learning e.g., Dasgupta et al.

Original: [2007], Balcan et al.
Revised: [2007], Balcan et al.

Original: [2009] and develop a new complexity for the non-realizable case (i.e., when h has zero bias).
Revised: [2009] and develop a new complexity for the non-realizable case (i.e., when h has zero bias).

Original: These possible extensions are left for future study.
Revised: These possible extensions are left for future study.

Original: The second implementation issue is related to the base model.
Revised: The second implementation issue is related to the base model.

Original: We propose to implement a linear model and a kernel regression model approximated by Random Fourier Feature
Revised: We propose to implement a linear model and a kernel regression model approximated by Random Fourier Feature

Original: Rahimi and Recht [2007] – we call it ‘rff model’.
Revised: Rahimi and Recht [2007] – we call it ‘rff model’.

Original: For the linear model, if instances x 1 , . . .
Revised: For the linear model, if instances x 1 , . . .

Original: , x n ∈ R p , we can show ˜∆ α,β ( h ; S ) =
Revised: , x n ∈ R p , we can show ˜∆ α,β ( h ; S ) =

Original: [Equation]
Revised: [Equation]

Original: · h
Revised: · h

Original: [Equation]
Revised: [Equation]

Original: [ x ]( D
Revised: [ x ]( D

Original: [Equation]
Revised: [Equation]

Original: M )
Revised: M )

Original: [ x ] T h , where [ x ] is an n -by- p matrix with the i th row being x T i .
Revised: [ x ] T h , where [ x ] is an n -by- p matrix with the i th row being x T i .

Original: Further, if squared loss is used, then solution to (19) is
Revised: Further, if squared loss is used, then solution to (19) is

Original: [Equation]
Revised: [Equation]

Original: where [ y ] ∈ R n is a vector with the i th entry being y i and
Revised: where [ y ] ∈ R n is a vector with the i th entry being y i and

Original: D is an n -by- n diagonal matrix with D ii = (cid:80) nj =1 M ij .
Revised: D is an n -by- n diagonal matrix with D ii = (cid:80) nj =1 M ij .

Original: For the rff model, we first calculate random features Rahimi and Recht [2007] and then train a linear model based on them using the AMF learner.
Revised: For the rff model, we first calculate random features Rahimi and Recht [2007] and then train a linear model based on them using the AMF learner.

Original: Note random features are only used to approximate the prediction model, and we still measure d ( x, x ′ ) using the original features.
Revised: Note random features are only used to approximate the prediction model, and we still measure d ( x, x ′ ) using the original features.

Original: The last issue is related to active learning.
Revised: The last issue is related to active learning.

Original: Given a labeled training set L and an unlabeled set U , the proposed active
Revised: Given a labeled training set L and an unlabeled set U , the proposed active

Original: AMF learner labels a candidate instance u if there exists
Revised: AMF learner labels a candidate instance u if there exists

Original: u ′ ∈ L satisfying d ( u, u ′ ) ≤
Revised: u ′ ∈ L satisfying d ( u, u ′ ) ≤

Original: α
Revised: α

Original: and | h ( u ) − h ( u ′ )
Revised: and | h ( u ) − h ( u ′ )

Original: | > β .
Revised: | > β .

Original: In principle, we can also pair u with instances in U , as long as the labeled instances fall in C α,β ( V q ) .
Revised: In principle, we can also pair u with instances in U , as long as the labeled instances fall in C α,β ( V q ) .

Original: In practice, pairing u with instances in L is often more efficient (since the label set is often way smaller than the unlabeled set), and leads to slightly better performance as we observe in experiments.
Revised: In practice, pairing u with instances in L is often more efficient (since the label set is often way smaller than the unlabeled set), and leads to slightly better performance as we observe in experiments.

Original:  EXPERIMENT
Revised:  EXPERIMENT

Original: We experiment on three real-world data sets.
Revised: We experiment on three real-world data sets.

Original: The Insurance data setdat [a] has individual medical costs billed by health insurance company, and the task is to predict the cost based other attributes.
Revised: The Insurance data setdat [a] has individual medical costs billed by health insurance company, and the task is to predict the cost based other attributes.

Original: The Life data setdat [b] has the life expectancy in different countries, and the task is to predict the expectancy.
Revised: The Life data setdat [b] has the life expectancy in different countries, and the task is to predict the expectancy.

Original: We also use a data set collected from public resources.
Revised: We also use a data set collected from public resources.

Original: It contains the COVID death rates of 3142 counties in United States and the task is to predict the rate based on other attributes including population density, obesity rate, smoking rate, diabetes rate, elderly population and vaccine rate.
Revised: It contains the COVID death rates of 3142 counties in United States and the task is to predict the rate based on other attributes including population density, obesity rate, smoking rate, diabetes rate, elderly population and vaccine rate.

Original: To learn more data sets used to evaluate algorithmic fairness, we refer interested readers to Le Quy et al.
Revised: To learn more data sets used to evaluate algorithmic fairness, we refer interested readers to Le Quy et al.

Original: We encode categorical features by dummy variables, address missing data using mean imputation and standardize all features.
Revised: We encode categorical features by dummy variables, address missing data using mean imputation and standardize all features.

Original: For higher numerical stability, we re-scale the labels: on the Insurance data set, we divide the medical cost which varies from 4k to 40k by 10k; on the Life data set, we divide the life expectancy which varies from 40 toby 100; on the COVID data set, we multiply the death rate which varies from 0 to 0.01 by 100.
Revised: For higher numerical stability, we re-scale the labels: on the Insurance data set, we divide the medical cost which varies from 4k to 40k by 10k; on the Life data set, we divide the life expectancy which varies from 40 toby 100; on the COVID data set, we multiply the death rate which varies from 0 to 0.01 by 100.

Original: We randomly split each data set into an initial training set (assumed labeled), an unlabeled set (for query) and a testing set.
Revised: We randomly split each data set into an initial training set (assumed labeled), an unlabeled set (for query) and a testing set.

Original: Size of the initial training set is chosen as follows: for the linear base model, it is the feature number on the Insurance and Life data sets, and twice that number on the COVID data set; for the rff base model, it is half of the random feature number.
Revised: Size of the initial training set is chosen as follows: for the linear base model, it is the feature number on the Insurance and Life data sets, and twice that number on the COVID data set; for the rff base model, it is half of the random feature number.

Original: Size of the testing data is 25% of the total data size.
Revised: Size of the testing data is 25% of the total data size.

Original: The remaining data are treated as unlabeled
Revised: The remaining data are treated as unlabeled

Original: On a data set, we run an active learner for 20 random trials and report the average model performance on the testing sets.
Revised: On a data set, we run an active learner for 20 random trials and report the average model performance on the testing sets.

Original: Model bias is measured by ∆ α,β ( h ; S n ) defined in (6), with ( α, β ) set to (2, 0.1) on Insurance, (10, 0.2) on Life and (1.5, 0.001) on COVID.
Revised: Model bias is measured by ∆ α,β ( h ; S n ) defined in (6), with ( α, β ) set to (2, 0.1) on Insurance, (10, 0.2) on Life and (1.5, 0.001) on COVID.

Original: We also experimented with other fairness coefficients and observed similar comparative performance.
Revised: We also experimented with other fairness coefficients and observed similar comparative performance.

Original: Model error is measured by the root-mean-squared-error.
Revised: Model error is measured by the root-mean-squared-error.

Original: We evaluate the proposed active labeling strategy on the linear base model and rff base model respectively, and compare its performance with the following three strategies.
Revised: We evaluate the proposed active labeling strategy on the linear base model and rff base model respectively, and compare its performance with the following three strategies.

Original: – Random : It randomly selects instances to label.
Revised: – Random : It randomly selects instances to label.

Original: – Query-by-Committee (QBC) :
Revised: – Query-by-Committee (QBC) :

Original: It labels instances which receive the largest prediction variance from a committee of models.
Revised: It labels instances which receive the largest prediction variance from a committee of models.

Original: Following Burbidge et al.
Revised: Following Burbidge et al.

Original: [2007], we construct a committee of five models and train each one using a bootstrap sample of the training data, with sample size equals to the training set size divided by the committee size.
Revised: [2007], we construct a committee of five models and train each one using a bootstrap sample of the training data, with sample size equals to the training set size divided by the committee size.

Original: – Uncertainty : It labels instances which are most different from the training data in both feature space and label space Wu et al.
Revised: – Uncertainty : It labels instances which are most different from the training data in both feature space and label space Wu et al.

Original: To our knowledge, this is a state-of-the-art active labeling method for regression model.
Revised: To our knowledge, this is a state-of-the-art active labeling method for regression model.

Original: – Cluster : It is a clustering based baseline method that relies on the distance between instances.
Revised: – Cluster : It is a clustering based baseline method that relies on the distance between instances.

Original: It first identifies the top m uncertain instances in the candidate pool using the above method, then runs k-means clustering to identify their k centers, and finally labels the identified instances.
Revised: It first identifies the top m uncertain instances in the candidate pool using the above method, then runs k-means clustering to identify their k centers, and finally labels the identified instances.

Original: For the metric-fair learner, we pick its regularization coefficient λ that strikes a good balance between fairness and accuracy.
Revised: For the metric-fair learner, we pick its regularization coefficient λ that strikes a good balance between fairness and accuracy.

Original: Example results of the linear base model on Insurance are shown in Figure 3 (m - n).

Original: We see model bias on both training data and testing data decrease as λ increases, which suggests the metric-fair learner can effectively reduce bias and the reduction can generalize (which supports Theorem 3.6.)

Original: We also see model error first decreases and then increases, exhibiting an overfitting

Original: [Figure]

Original: this case, we will set λ = 1 as it strikes a good balance between fairness and accuracy.

Original: Using this strategy, for the linear base model, λ is set to 1 on Insurance and Life and 0. on COVID; for the rff base model, λ is set to 1 on Insurance, 5 on Life and 0.5 on COVID.
Revised: For the linear base model, λ is set to 1 on Insurance and Life and 0.1 on COVID; for the rff base model, λ is set to 1 on Insurance, 5 on Life and 0.5 on COVID.

Original: For the rff base model, we generate the random features that approximate Gaussian kernel Rahimi and Recht [2007].
Revised: For the rff base model, we generate the random features that approximate Gaussian kernel Rahimi and Recht [2007].

Original: The random feature number is set to 100 on Insurance,on Life and 200 on COVID.
Revised: The random feature number is set to 100 on Insurance,on Life and 200 on COVID.

Original: The gamma coefficient is set  to 1e-4 on Insurance, 1e-9 on Life and 1e-2 on COVID.
Revised: The gamma coefficient is set [Figure] to 1e-4 on Insurance, 1e-9 on Life and 1e-2 on COVID.

Original: In practice, we observe these configurations lead to good and stable performance of active metric-fair learning.
Revised: In practice, we observe these configurations lead to good and stable performance of active metric-fair learning.

Original: For the clustering based baseline method, we set m = 10 and k =as they give consistently good performance (except k is set to 10 for linear model on the Life dataset).
Revised: For the clustering based baseline method, we set m = 10 and k =as they give consistently good performance (except k is set to 10 for linear model on the Life dataset).

Original: 6.1 RESULTS AND DISCUSSIONS
Revised: 6.1 RESULTS AND DISCUSSIONS

Original: Results of the experimented strategies on both base models across three data sets are shown in Figure 3.
Revised: Results of the experimented strategies on both base models across three data sets are shown in Figure 3.

Original: In Figure 3 (a-f), we see the proposed active AMF learner reduces model bias more efficiently than other learners, which empirically verifies its efficient sample complexity.
Revised: In Figure 3 (a-f), we see the proposed active AMF learner reduces model bias more efficiently than other learners, which empirically verifies its efficient sample complexity.

Original: We notice it achieves almost zero bias in all cases, supporting our assumption on the realizable case.
Revised: We notice it achieves almost zero bias in all cases, supporting our assumption on the realizable case.

Original: (And note this is not achieved at the cost of significantly deteriorating accuracy, as explained in the next paragraph.)
Revised: (And note this is not achieved at the cost of significantly deteriorating accuracy, as explained in the next paragraph.)

Original: There seems no consistent pattern on the efficiency of other learners.
Revised: There seems no consistent pattern on the efficiency of other learners.

Original: We notice QBC and uncertainty are often less efficient than random, implying the importance of (efficiently) achieving individual fairness by design, as presented in this study.
Revised: We notice QBC and uncertainty are often less efficient than random, implying the importance of (efficiently) achieving individual fairness by design, as presented in this study.

Original: In Figure 3 (g-l), it is not surprising to see that uncertainty
Revised: In Figure 3 (g-l), it is not surprising to see that uncertainty

Original: based labeling reduces error faster than other strategies.
Revised: based labeling reduces error faster than other strategies.

Original: Comparatively, the proposed active AMF learner manages to achieve a comparable reduction rate, suggesting its has an efficient fairness-accuracy trade-off.
Revised: Comparatively, the proposed active AMF learner manages to achieve a comparable reduction rate, suggesting its has an efficient fairness-accuracy trade-off.

Original: We also perform sensitivity analysis on the proposed strategy and present results in Figure 2.
Revised: We also perform sensitivity analysis on the proposed strategy and present results in Figure 2.

Original: Figures 2 (a-b) show the performance versus regularization coefficient λ .
Revised: Figures 2 (a-b) show the performance versus regularization coefficient λ .

Original: We both training and testing δ decrease as λ increases.
Revised: We both training and testing δ decrease as λ increases.

Original: This suggests the metric-fair learner can effectively reduce bias and the reduction is generalizable, which supports Theorem 3.6.
Revised: This suggests the metric-fair learner can effectively reduce bias and the reduction is generalizable, which supports Theorem 3.6.

Original: We also see model error first decreases and then increases, exhibiting an overfitting phenomenon.
Revised: We also see model error first decreases and then increases, exhibiting an overfitting phenomenon.

Original: Figures 2 (c-d) show the performance versus different choices of ( α, β ) when selecting instances in Step 3 of
Revised: Figures 2 (c-d) show the performance versus different choices of ( α, β ) when selecting instances in Step 3 of

Original: Algorithm 1.
Revised: Algorithm 1.

Original: (But all δ ’s are evaluated based on the same ( α, β ) for fair comparison.)
Revised: (But all δ ’s are evaluated based on the same ( α, β ) for fair comparison.)

Original: We see using smaller α to select instances leads to faster convergence of δ but more slowly convergence of RMSE.
Revised: We see using smaller α to select instances leads to faster convergence of δ but more slowly convergence of RMSE.

Original: There seems no clear pattern on the impact of β .
Revised: There seems no clear pattern on the impact of β .

Original: Overall, we see one can balance fairness and accuracy of the proposed strategy through adjusting α .
Revised: Overall, we see one can balance fairness and accuracy of the proposed strategy through adjusting α .

Original:  CONCLUSION
Revised:  CONCLUSION

Original: In this paper, we propose the first active approximate metricfair (AMF) learner and prove it can achieve an ε bias budget by labeling only O (log 1 ε ) instances.
Revised: In this paper, we propose the first active approximate metricfair (AMF) learner and prove it can achieve an ε bias budget by labeling only O (log 1 ε ) instances.

Original: To our knowledge, this result is a first and substantial improvement over the existing O ( 1 ε 2 ) sample complexity for achieving individual fairness by the passive learners.
Revised: To our knowledge, this result is a first and substantial improvement over the existing O ( 1 ε 2 ) sample complexity for achieving individual fairness by the passive learners.

Original: Through extensive experiments across three public data sets, we show the proposed active AMF learner improves fairness of two regression models more efficiently than its passive counterpart as well as state-of-the-art active learners, while being able to maintain comparable accuracy.
Revised: Through extensive experiments across three public data sets, we show the proposed active AMF learner improves fairness of two regression models more efficiently than its passive counterpart as well as state-of-the-art active learners, while being able to maintain comparable accuracy.

Original: Another contribution of this study is to present a provably equivalent form of AMF based on uniform continuity instead of the existing almost Lipschitz.
Revised: Another contribution of this study is to present a provably equivalent form of AMF based on uniform continuity instead of the existing almost Lipschitz.

Original: Acknowledgements
Revised: Acknowledgements

Original: We thank all anonymous reviewers for their insightful feedback for enhancing the quality of the paper.
Revised: We thank all anonymous reviewers for their insightful feedback for enhancing the quality of the paper.

Original: We also thank
Revised: We also thank

Original: Skyler Sprecker for sharing the COVID data set.
Revised: Skyler Sprecker for sharing the COVID data set.

Original: This paper is based upon work supported by the US National Science
Revised: This paper is based upon work supported by the US National Science

Original: Foundation under Grant No. 2101936.
Revised: Foundation under Grant No. 2101936.

Original: [Figure]
Revised: [Figure]