Original: T HE C OST OF P RIVACY IN F AIR M ACHINE L EARNING
Revised: T HE C OST OF P RIVACY IN F AIR M ACHINE L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: A common task in fair machine learning is training ML models that preserve certain summary statistics across subpopulations defined by sensitive attributes.
Revised: A common task in fair machine learning is training ML models that preserve certain summary statistics across subpopulations defined by sensitive attributes.

Original: However, access to such sensitive attributes in training data is restricted and the learner must rely on noisy proxies for the sensitive attributes.
Revised: However, access to such sensitive attributes in training data is restricted and the learner must rely on noisy proxies for the sensitive attributes.

Original: In this paper, we study the effect of a privacy mechanism that obfuscates the sensitive attributes from the learner on the fairness of the resulting classifier.
Revised: In this paper, we study the effect of a privacy mechanism that obfuscates the sensitive attributes from the learner on the fairness of the resulting classifier.

Original: We show that the cost of privacy in fair ML is a decline in the generalizability of fairness constraints.
Revised: We show that the cost of privacy in fair ML is a decline in the generalizability of fairness constraints.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The fairness of machine learning systems is gaining increasing attention in recent years.
Revised: The fairness of machine learning systems is gaining increasing attention in recent years.

Original: Among the numerous fairness objectives is ensuring that a machine learning model does not discriminate against subpopulations that are typically identified by sensitive attributes ( e.g. , race, gender).
Revised: Among the numerous fairness objectives is ensuring that a machine learning model does not discriminate against subpopulations that are typically identified by sensitive attributes ( e.g. , race, gender).

Original: When training a fair model and evaluating model bias, it is necessary to possess sensitive attributes; however, access to and use of such sensitive data is frequently prohibited by laws and regulations.
Revised: When training a fair model and evaluating model bias, it is necessary to possess sensitive attributes; however, access to and use of such sensitive data is frequently prohibited by laws and regulations.

Original: Credit card companies, for instance, are not permitted to inquire about a person’s race when they apply for credit, yet they must demonstrate that their decisions are not discriminatory (Chen et al., 2019).
Revised: Credit card companies, for instance, are not permitted to inquire about a person’s race when they apply for credit, yet they must demonstrate that their decisions are not discriminatory (Chen et al., 2019).

Original: Ideally, sensitive personal information should not be disclosed during the training of ML models.
Revised: Ideally, sensitive personal information should not be disclosed during the training of ML models.

Original: However, it is impossible to ensure exact notions of fairness (such as demographic parity or equality of opportunity) without any knowledge of the sensitive data.
Revised: However, it is impossible to ensure exact notions of fairness (such as demographic parity or equality of opportunity) without any knowledge of the sensitive data.

Original: Fortunately, differential privacy (Dwork et al., 2006) is a promising workaround, which can offer a graceful compromise between privacy and utility.
Revised: Fortunately, differential privacy (Dwork et al., 2006) is a promising workaround, which can offer a graceful compromise between privacy and utility.

Original: Mozannar et al.
Revised: Mozannar et al.

Original: (2020) propose to release sensitive attributes in a locally differentially private way: adding noise to the sensitive data so that adversaries cannot infer any information with high confidence about a single record.
Revised: (2020) propose to release sensitive attributes in a locally differentially private way: adding noise to the sensitive data so that adversaries cannot infer any information with high confidence about a single record.

Original: The advantage of the privacy mechanism proposed by Mozannar et al.
Revised: The advantage of the privacy mechanism proposed by Mozannar et al.

Original: (2020) is an invariance property: exact notions of fairness with regard to true sensitive attributes and noisy sensitive attributes are equivalent.
Revised: (2020) is an invariance property: exact notions of fairness with regard to true sensitive attributes and noisy sensitive attributes are equivalent.

Original: An implication of the invariance property is that the optimal model of fairness can be learned at the population level.
Revised: An implication of the invariance property is that the optimal model of fairness can be learned at the population level.

Original: Nonetheless, it remains unclear what the precise statistical impact of privacy on fairness is.
Revised: Nonetheless, it remains unclear what the precise statistical impact of privacy on fairness is.

Original: In this work, we study the statistical cost of privacy on fairness in the task of learning fair ML models with differentially private sensitive attributes.
Revised: In this work, we study the statistical cost of privacy on fairness in the task of learning fair ML models with differentially private sensitive attributes.

Original: The main benefits of the developed theory are
Revised: The main benefits of the developed theory are

Original: statistically principled: We propose a statistically principled metric to characterize the cost of privacy on fairness.
Revised: statistically principled: We propose a statistically principled metric to characterize the cost of privacy on fairness.

Original: A restricted notion of statistical efficiency precisely quantifies the privacy cost asymptotically.
Revised: A restricted notion of statistical efficiency precisely quantifies the privacy cost asymptotically.

Original: 2. interpretable:
Revised: 2. interpretable:

Original: Privacy leads to a decline in the statistical efficiency.
Revised: Privacy leads to a decline in the statistical efficiency.

Original: Such efficiency loss is interpretable: it explicitly depends on the privacy budget, the subpopulation imbalance level, and few other problem-specific parameters.
Revised: Such efficiency loss is interpretable: it explicitly depends on the privacy budget, the subpopulation imbalance level, and few other problem-specific parameters.

Original: The rest of this paper is organized as follows.
Revised: The rest of this paper is organized as follows.

Original: In Section 2, we formalize the problem setup, which consists of the constrained stochastic optimization problem for fair machine learning, the local differential privacy mechanism for releasing sensitive attributes, the learning procedure of fair model using private sensitive attributes, and the definition of asymptotic relative efficiency in terms of fairness violations.
Revised: In Section 2, we formalize the problem setup, which consists of the constrained stochastic optimization problem for fair machine learning, the local differential privacy mechanism for releasing sensitive attributes, the learning procedure of fair model using private sensitive attributes, and the definition of asymptotic relative efficiency in terms of fairness violations.

Original: In Section 3, we develop theory for the privacy cost under a single exact fairness constraint and then generalize this theory to some extent.
Revised: In Section 3, we develop theory for the privacy cost under a single exact fairness constraint and then generalize this theory to some extent.

Original: By simulating a risk-parity linear regression problem in Section 4, we validate our theory and illustrate the utility of our tools.
Revised: By simulating a risk-parity linear regression problem in Section 4, we validate our theory and illustrate the utility of our tools.

Original: Finally, we summarize our work in Section 5 and point out an interesting avenue of future work.
Revised: Finally, we summarize our work in Section 5 and point out an interesting avenue of future work.

Original: 1. R ELATED WORK
Revised: 1. R ELATED WORK

Original: The interaction between fairness and privacy has been investigated from three perspectives: learning approximately fair models without sensitive attributes (Hashimoto et al., 2018; Lahoti et al., 2020), learning approximately fair models with wildly noisy sensitive attributes (Kallus et al., 2019;
Revised: The interaction between fairness and privacy has been investigated from three perspectives: learning approximately fair models without sensitive attributes (Hashimoto et al., 2018; Lahoti et al., 2020), learning approximately fair models with wildly noisy sensitive attributes (Kallus et al., 2019;

Original: Awasthi et al., 2020; Wang et al., 2020), and learning exactly fair models with structured noisy sensitive attributes (Lamy et al., 2020; Mozannar et al., 2020).
Revised: Awasthi et al., 2020; Wang et al., 2020), and learning exactly fair models with structured noisy sensitive attributes (Lamy et al., 2020; Mozannar et al., 2020).

Original: This paper focuses on the third aspect.
Revised: This paper focuses on the third aspect.

Original: The works that are most pertinent to ours are Lamy et al.
Revised: The works that are most pertinent to ours are Lamy et al.

Original: and Mozannar et al.
Revised: and Mozannar et al.

Original: Lamy et al.
Revised: Lamy et al.

Original: (2020) assume that the sensitive attributes are subject to noise from the mutually contaminated learning model.
Revised: (2020) assume that the sensitive attributes are subject to noise from the mutually contaminated learning model.

Original: Under such a structured noise mechanism, the noise rates can be consistently estimated, and when enforcing fairness with regard to noisy groups, scaling the fairness tolerance parameter more tightly is all that is required.
Revised: Under such a structured noise mechanism, the noise rates can be consistently estimated, and when enforcing fairness with regard to noisy groups, scaling the fairness tolerance parameter more tightly is all that is required.

Original: Mozannar et al.
Revised: Mozannar et al.

Original: (2020) suggest a differentially private model to release the sensitive attributes, which is a special type of the mutually contaminated learning model.
Revised: (2020) suggest a differentially private model to release the sensitive attributes, which is a special type of the mutually contaminated learning model.

Original: Under such a designed noise mechanism, Mozannar et al.
Revised: Under such a designed noise mechanism, Mozannar et al.

Original: (2020) show that if the classifier is independent of the sensitive attributes, then exact fairness with regard to noisy sensitive attributes is equivalent to that with regard to true sensitive attributes.
Revised: (2020) show that if the classifier is independent of the sensitive attributes, then exact fairness with regard to noisy sensitive attributes is equivalent to that with regard to true sensitive attributes.

Original: The idea of the equivalence can be found in Lamy et al.
Revised: The idea of the equivalence can be found in Lamy et al.

Original: (2020) while Mozannar et al.
Revised: (2020) while Mozannar et al.

Original: (2020) put it into a formal statement.
Revised: (2020) put it into a formal statement.

Original: We basically study the statistical cost of privacy on the generalizability of fairness when using Lamy et al.
Revised: We basically study the statistical cost of privacy on the generalizability of fairness when using Lamy et al.

Original: (2020)’s method under Mozannar et al.
Revised: (2020)’s method under Mozannar et al.

Original: (2020)’s privacy mechanism.
Revised: (2020)’s privacy mechanism.

Original:  P ROBLEM SETUP 2. F AIR MACHINE LEARNING AS CONSTRAINED STOCHASTIC OPTIMIZATION
Revised:  P ROBLEM SETUP 2. F AIR MACHINE LEARNING AS CONSTRAINED STOCHASTIC OPTIMIZATION

Original: In-processing fair machine learning is typically a supervised learning problem with fairness constraints (Zafar et al., 2017; Agarwal et al., 2018).
Revised: In-processing fair machine learning is typically a supervised learning problem with fairness constraints (Zafar et al., 2017; Agarwal et al., 2018).

Original: Such a problem can most often be formulated as a constrained stochastic optimization problem: (empirical) risk minimization subject to (empirical) fairness constraints.
Revised: Such a problem can most often be formulated as a constrained stochastic optimization problem: (empirical) risk minimization subject to (empirical) fairness constraints.

Original: Consider a fair binary classification problem.
Revised: Consider a fair binary classification problem.

Original: Let X ⊂ R d be the input space,  { 0 , 1 } be the set of possible labels, and A be the set of possible values of the protected/sensitive attribute.
Revised: Let X ⊂ R d be the input space, Y = { 0 , 1 } be the set of possible labels, and A be the set of possible values of the protected/sensitive attribute.

Original: In this setup, training and test examples are tuples of the form ( X, A, Y ) ∈
Revised: In this setup, training and test examples are tuples of the form ( X, A, Y ) ∈

Original: X ×
Revised: X ×

Original: A × Y , and a classifier is a map f : X → { 0 , 1 } .
Revised: A × Y , and a classifier is a map f : X → { 0 , 1 } .

Original: Two popular definitions of algorithmic fairness for binary classification are demographic parity (Dwork et al., 2011) and equality of opportunity (Hardt et al., 2016).
Revised: Two popular definitions of algorithmic fairness for binary classification are demographic parity (Dwork et al., 2011) and equality of opportunity (Hardt et al., 2016).

Original: Definition 2.1 (Demographic parity) .
Revised: Definition 2.1 (Demographic parity) .

Original: Let (cid:98) Y ≜
Revised: Let (cid:98) Y =

Original: f ( X ) be the output of the classifier.
Revised: f ( X ) be the output of the classifier.

Original: Demographic parity entails P { (cid:98) Y = 1 | A = a } = P { (cid:98) Y = 1 | A = a ′ } for all a, a ′ ∈ A .
Revised: Demographic parity entails P { (cid:98) Y = 1 | A = a } = P { (cid:98) Y = 1 | A = a ′ } for all a, a ′ ∈ A .

Original: Demographic parity, also known as statistical parity, means that the prediction (cid:98) Y =
Revised: Demographic parity, also known as statistical parity, means that the prediction (cid:98) Y =

Original: h ( X ) is statistically independent of the protected attribute A .
Revised: f ( X ) is statistically independent of the protected attribute A .

Original: Definition 2.2 (Equality of opportunity) .
Revised: Definition 2.2 (Equality of opportunity) .

Original: Let Y = 1 be the advantaged label that is associated with a positive outcome and (cid:98) Y ≜
Revised: Let Y = 1 be the advantaged label that is associated with a positive outcome and (cid:98) Y =

Original: f ( X ) be the output of the classifier.
Revised: f ( X ) be the output of the classifier.

Original: Equality of opportunity entails
Revised: Equality of opportunity entails

Original: P { (cid:98) Y = 1 | A = a, Y = 1 } =
Revised: P { (cid:98) Y = 1 | A = a, Y = 1 } =

Original: P { (cid:98) Y = 1 | A = a ′ , Y = 1 } for all a, a ′ ∈ A .
Revised: P { (cid:98) Y = 1 | A = a ′ , Y = 1 } for all a, a ′ ∈ A .

Original: Equality of opportunity, also known as true positive rate parity, means that the prediction (cid:98) Y =
Revised: Equality of opportunity, also known as true positive rate parity, means that the prediction (cid:98) Y =

Original: h ( X ) conditioned on the advantaged label Y = 1 is statistically independent of the protected attribute A .
Revised: f ( X ) conditioned on the advantaged label Y = 1 is statistically independent of the protected attribute A .

Original: Given a parametric model space H
Revised: Given a parametric model space H

Original: = { f θ ( · ) : θ ∈ Θ } and loss function ℓ : Θ × X × Y
Revised: = { f θ ( · ) : θ ∈ Θ } and loss function ℓ : Θ × X × Y

Original: → R + (where Θ ⊂ R d is a finite-dimensional parameter space), an in-processing fair ML routine is to minimize the (empirical) risk E (cid:2) ℓ ( θ ; X, Y ) (cid:3) while satisfying some fairness constraints.
Revised: → R + (where Θ ⊂ R d is a finite-dimensional parameter space), an in-processing fair ML routine is to minimize the (empirical) risk E (cid:2) ℓ ( θ ; X, Y ) (cid:3) while satisfying some fairness constraints.

Original: To keep things simple, we assume there are only two demographic groups; i.e. |A| = 2 .
Revised: To keep things simple, we assume there are only two demographic groups; i.e. |A| = 2 .

Original: Without loss of generality, we refer to one group as advantaged ( A = 1 ) and the other as disadvantaged ( A = 0 ).
Revised: Without loss of generality, we refer to one group as advantaged ( A = 1 ) and the other as disadvantaged ( A = 0 ).

Original: Consider fair learning with demographic parity as an example.
Revised: Consider fair learning with demographic parity as an example.

Original: At the population level, the goal is to solve the problem:
Revised: At the population level, the goal is to solve the problem:

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: where the expectation is with respect to the distribution of tuple ( X, A, Y ) .
Revised: where the expectation is with respect to the distribution of tuple ( X, A, Y ) .

Original: The true underlying distribution is unknown, so we cannot solve (2.1) directly.
Revised: The true underlying distribution is unknown, so we cannot solve (2.1) directly.

Original: Instead, we observe IID training samples { ( X i , A
Revised: Instead, we observe IID training samples { ( X i , A

Original: i , Y i ) } ni =1 from the true distribution and solve the empirical version of (2.1):
Revised: i , Y i ) } ni =1 from the true distribution and solve the empirical version of (2.1):

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: where 0 < α n = o ( 1 √ n ) is a slackness term shrinking to zero at a rate faster than 1 √ n .
Revised: where 0 < α n = o ( 1 √ n ) is a slackness term shrinking to zero at a rate faster than 1 √ n .

Original: Through the rest of the work, we always let α n be a positive number of order o ( 1 √ n ) .
Revised: Through the rest of the work, we always let α n be a positive number of order o ( 1 √ n ) .

Original: 2. L OCAL DIFFERENTIAL PRIVACY MECHANISM FOR RELEASING SENSITIVE ATTRIBUTES
Revised: 2. L OCAL DIFFERENTIAL PRIVACY MECHANISM FOR RELEASING SENSITIVE ATTRIBUTES

Original: Consider the randomized response mechanism (Warner, 1965; Kairouz et al., 2014) for releasing privatized sensitive attribute:
Revised: Consider the randomized response mechanism (Warner, 1965; Kairouz et al., 2014) for releasing privatized sensitive attribute:

Original: [Equation]
Revised: [Equation]

Original: ) for all a, z ∈ A , where ε > 0 controls the privacy level.
Revised: ) for all a, z ∈ A , where ε > 0 controls the privacy level.

Original: The privatized sensitive attribute Z of the true sensitive attribute A is defined as Z = Q ( · | A ) .
Revised: The privatized sensitive attribute Z of the true sensitive attribute A is defined as Z = Q ( · | A ) .

Original: In addition, the sampling mechanism Q requires Z ⊥⊥ X, Y
Revised: In addition, the sampling mechanism Q requires Z ⊥⊥ X, Y

Original: | A .
Revised: | A .

Original: Then the private dataset { ( X i , Z i , Y i ) } ni =1 is generated from the original dataset { ( X i , A
Revised: Then the private dataset { ( X i , Z i , Y i ) } ni =1 is generated from the original dataset { ( X i , A

Original: i , Y i ) } ni =1 via the transition kernel Q .
Revised: i , Y i ) } ni =1 via the transition kernel Q .

Original: The randomized response mechanism (2.3) is a locally ε -differentially private mechanism (Duchi et al., 2013), that is
Revised: The randomized response mechanism (2.3) is a locally ε -differentially private mechanism (Duchi et al., 2013), that is

Original: [Equation]
Revised: [Equation]

Original: Here a smaller parameter ε indicates a stronger privacy guarantee.
Revised: Here a smaller parameter ε indicates a stronger privacy guarantee.

Original: Moreover, the mechanism Q is considered optimal for distribution estimation under local differential privacy constraints (Kairouz et al., 2014; 2016).
Revised: Moreover, the mechanism Q is considered optimal for distribution estimation under local differential privacy constraints (Kairouz et al., 2014; 2016).

Original: From this point forward (with the exception of the general theory presented in Section 3.1), we assume that there are only two demographic groups, i.e. |A|
Revised: From this point forward (with the exception of the general theory presented in Section 3.1), we assume that there are only two demographic groups, i.e. |A|

Original: = 2 .
Revised: = 2 .

Original: The mechanism (2.3) becomes
Revised: The mechanism (2.3) becomes

Original: [Equation]
Revised: [Equation]

Original: for a ∈ { 0 , 1 } , where γ ∈ [0 , 0 . 5) .
Revised: for a ∈ { 0 , 1 } , where γ ∈ [0 , 0 . 5) .

Original: The parameter γ = 0 (or equivalently ε = ∞ ) signifies complete lack of privacy, whereas γ → 0 .
Revised: The parameter γ = 0 (or equivalently ε = ∞ ) signifies complete lack of privacy, whereas γ → 0 .

Original: 5 (or equivalently ε → 0 ) corresponds to perfect privacy.
Revised: 5 (or equivalently ε → 0 ) corresponds to perfect privacy.

Original: 2. F AIR MACHINE LEARNING WITH PRIVATE SENSITIVE ATTRIBUTES
Revised: 2. F AIR MACHINE LEARNING WITH PRIVATE SENSITIVE ATTRIBUTES

Original: The privatized sensitive attribute Z can be served as a noisy proxy for the true sensitive attribute A .
Revised: The privatized sensitive attribute Z can be served as a noisy proxy for the true sensitive attribute A .

Original: One may wish to learn a fair classifier by directly enforcing fairness notion on Z i ’s, the proxies for
Revised: One may wish to learn a fair classifier by directly enforcing fairness notion on Z i ’s, the proxies for

Original: A i ’s.
Revised: A i ’s.

Original: This approach is feasible and justifiable (at the population level) due to the invariance of exact fairness under local differential privacy.
Revised: This approach is feasible and justifiable (at the population level) due to the invariance of exact fairness under local differential privacy.

Original: Proposition 2.3 (Proposition 1 in Mozannar et al.
Revised: Proposition 2.3 (Proposition 1 in Mozannar et al.

Original: Consider any exact fairness notion among demographic parity and equality of opportunity.
Revised: Consider any exact fairness notion among demographic parity and equality of opportunity.

Original: Let (cid:98) Y =
Revised: Let (cid:98) Y =

Original: f
Revised: f

Original: ( X ) be a binary classifier.
Revised: ( X ) be a binary classifier.

Original: Then (cid:98) Y is fair with respect to A if and only if (cid:98) Y is fair with respect to Z .
Revised: Then (cid:98) Y is fair with respect to A if and only if (cid:98) Y is fair with respect to Z .

Original: Proposition 2.3 requires (cid:98) Y is only a function of X .
Revised: Proposition 2.3 requires (cid:98) Y is only a function of X .

Original: Mozannar et al.
Revised: Mozannar et al.

Original: (2020) shows by construction
Revised: (2020) shows by construction

Original: the
Revised: the

Original: existence of a classifier (cid:98) Y = (cid:101) f
Revised: existence of a classifier (cid:98) Y = (cid:101) f

Original: ( X, Z ) which is fair with respect to Z but unfair to A .
Revised: ( X, Z ) which is fair with respect to Z but unfair to A .

Original: Now we consider fair ML with private sensitive attributes by (empirical) risk minimization subject to fairness constraints with respect to Z .
Revised: Now we consider fair ML with private sensitive attributes by (empirical) risk minimization subject to fairness constraints with respect to Z .

Original: Take fair learning with demographic parity as an example.
Revised: Take fair learning with demographic parity as an example.

Original: At the population level, the goal is to solve the problem
Revised: At the population level, the goal is to solve the problem

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: where the expectation is with respect to the distribution of tuple ( X, Z, Y ) .
Revised: where the expectation is with respect to the distribution of tuple ( X, Z, Y ) .

Original: Here Z is the proxy sensitive attribute but the true sensitive attribute A is unobservable.
Revised: Here Z is the proxy sensitive attribute but the true sensitive attribute A is unobservable.

Original: The true underlying distribution is unknown, so we cannot solve (2.5) directly.
Revised: The true underlying distribution is unknown, so we cannot solve (2.5) directly.

Original: Instead, we observe IID (private) training samples { ( X i , Z i , Y i ) } ni =1 from the true distribution and solve the empirical version of (2.5): (cid:101)θ n ∈   arg
Revised: Instead, we observe IID (private) training samples { ( X i , Z i , Y i ) } ni =1 from the true distribution and solve the empirical version of (2.5): (cid:101)

Original: [Equation]
Revised: [Equation]

Original: Y
Revised:   arg

Original: [Equation]
Revised: [Equation]

Original: ) subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: A direct corollary of Proposition 2.3 is that (2.1) and (2.5) have exactly the same solution θ ⋆ (assuming uniqueness of the solution).
Revised: A direct corollary of Proposition 2.3 is that (2.1) and (2.5) have exactly the same solution θ ⋆ (assuming uniqueness of the solution).

Original: One can also show that under regularity conditions both (cid:98) θ n and (cid:101) θ n , the solution to (2.2) and to (2.6), are √ n -consistent for θ ⋆ .
Revised: One can also show that under regularity conditions both (cid:98) θ n and (cid:101) θ n , the solution to (2.2) and to (2.6), are √ n -consistent for θ ⋆ .

Original: We wish to compare the estimating quality of (cid:98) θ n and (cid:101) θ n , and quantify the quality difference in terms of the privacy level parameter γ (or ε ) and few other problem-specific parameters.
Revised: We wish to compare the estimating quality of (cid:98) θ n and (cid:101) θ n , and quantify the quality difference in terms of the privacy level parameter γ (or ε ) and few other problem-specific parameters.

Original: 2. A SYMPTOTIC RELATIVE EFFICIENCY
Revised: 2. A SYMPTOTIC RELATIVE EFFICIENCY

Original: In statistics, consistency and efficiency are popular notions to evaluate the performance of estimators.
Revised: In statistics, consistency and efficiency are popular notions to evaluate the performance of estimators.

Original: Definition 2.4 (Consistency) .
Revised: Definition 2.4 (Consistency) .

Original: An estimator (cid:98) θ n is consistent for θ ⋆ if (cid:98) θ n p → θ ⋆ as n → ∞ .
Revised: An estimator (cid:98) θ n is consistent for θ ⋆ if (cid:98) θ n p → θ ⋆ as n → ∞ .

Original: Suppose that we have two consistent estimators (cid:98) θ n and (cid:101) θ n .
Revised: Suppose that we have two consistent estimators (cid:98) θ n and (cid:101) θ n .

Original: Both of them are reasonable, but which one should be preferred?
Revised: Both of them are reasonable, but which one should be preferred?

Original: To answer this question, we can employ the notion of efficiency, i.e. measuring how spread out about (cid:98) θ n (or (cid:101) θ n ) is the sampling distribution of the estimator.
Revised: To answer this question, we can employ the notion of efficiency, i.e. measuring how spread out about (cid:98) θ n (or (cid:101) θ n ) is the sampling distribution of the estimator.

Original: In light of this, we now adapt the concept of statistical efficiency to fair machine learning.
Revised: In light of this, we now adapt the concept of statistical efficiency to fair machine learning.

Original: In fair ML, the most important metric to evaluate the performance of a classifier is fairness violation.
Revised: In fair ML, the most important metric to evaluate the performance of a classifier is fairness violation.

Original: Let c : Θ → R be the constraint function.
Revised: Let c : Θ → R be the constraint function.

Original: For example, demographic parity constraint corresponds to c ( θ ) =
Revised: For example, demographic parity constraint corresponds to c ( θ ) =

Original: E (cid:2) 1 { f θ ( X ) = 1 }| A = 1 (cid:3) − E (cid:2) 1 { f θ ( X ) = 1 }|
Revised: E (cid:2) 1 { f θ ( X ) = 1 }| A = 1 (cid:3) − E (cid:2) 1 { f θ ( X ) = 1 }|

Original: A = 0 (cid:3) .
Revised: A = 0 (cid:3) .

Original: Since the exact fairness notion entails a classifier f θ is fair if c ( θ ) = 0 , we define the (signed) fairness violation of θ as c ( θ ) itself.
Revised: Since the exact fairness notion entails a classifier f θ is fair if c ( θ ) = 0 , we define the (signed) fairness violation of θ as c ( θ ) itself.

Original: Definition 2.5 (Efficiency in terms of constraint violations) .
Revised: Definition 2.5 (Efficiency in terms of constraint violations) .

Original: Suppose that we have two consistent estimators (cid:98) θ n and (cid:101) θ n satisfying 
Revised: Suppose that we have two consistent estimators (cid:98) θ n and (cid:101) θ n satisfying √

Original: [Equation]
Revised: [Equation]

Revised: −

Revised: [Equation]

Revised: ( θ

Revised: [Equation]

Revised: [Equation]

Revised: N

Revised: [Equation]

Revised: [Equation]

Revised: and √ n

Revised: [Equation]

Original: as n → ∞ .
Revised: as n → ∞ .

Original: We say that the estimator (cid:98) θ n is more efficient (in terms of constraint violations) than (cid:101) θ n if σ 2 ≤ (cid:101) σ 2 .
Revised: We say that the estimator (cid:98) θ n is more efficient (in terms of constraint violations) than (cid:101) θ n if σ 2 ≤ (cid:101) σ 2 .

Original: The asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n is
Revised: The asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n is

Original: [Equation]
Revised: [Equation]

Original: In other words, the estimator (cid:98) θ n is more efficient than (cid:101) θ n if ARE( (cid:101) θ n , (cid:98) θ n ) ≤ 1 .
Revised: In other words, the estimator (cid:98) θ n is more efficient than (cid:101) θ n if ARE( (cid:101) θ n , (cid:98) θ n ) ≤ 1 .

Original: The first-order Taylor approximation to  c (  θ  ) around θ ⋆ is given by
Revised: Another way to examine the efficiency loss is to look at the asymptotic joint distribution of c ( (cid:98) θ n ) and c ( (cid:101) θ n ) . Let ρ be the asymptotic correlation between c ( (cid:98) θ n ) and c ( (cid:101) θ n ) . The fairness violations of the two estimators can be compared using the ratio of c ( (cid:98) θ n ) to c ( (cid:101) θ n ) , which converges in distribution to a Cauchy random variable U :

Original: [Equation]
Revised: [Equation]

Original: where ∇ c (  θ ⋆ ) is the the gradient of c ( · ) evaluated at θ ⋆ . From (2.7) we see that the constraint violations only depend on the part of the estimation error (cid:98) θ − θ ⋆ in ran ( ∇ c ( θ ⋆ )) , the range space of the linear transform x (cid:55)→ ∇ c ( θ ⋆ )
Revised: Constraint violation inflates if we observe a value of the ratio | c ( (cid:98) θ n ) /c ( (cid:101) θ n ) | less than one. Assume (cid:101) θ n is more efficient than (cid:98) θ n , i.e. σ 2 < (cid:101) σ 2 .

Original: ⊤ x : R d ⊃ Θ → R .
Revised: Since | ρ | ≤ 1 , the median and mode of U , α , satisfies | α | < 1 , which indicates a high likelihood of constraint violation inflation.

Original: Therefore, the efficiency in terms of constraint violations only concerns how fast an estimator converges to θ ⋆ in ran ( ∇ c ( θ ⋆ )) .
Revised: Precisely, the asymptotic probability of constraint violation inflation is [Equation]

Original: The asymptotic relative efficiency (ARE)of (cid:101) θ n to (cid:98) θ n is the key quantity of our interest because it quantifies the constraint violation inflation : | c ( (cid:101) θ n ) |
Revised: In the rest of the paper, the asymptotic relative efficiency (ARE) is the key quantity of interest, which compares the asymptotic variances of two estimators by ARE =

Original: ≈ ARE( (cid:101) θ n , (cid:98) θ n ) − 1 / 2 × | c ( (cid:98) θ n ) | for large n .
Revised: lim n →∞ Var[ c ( (cid:98) θ n )] / Var[ c ( (cid:101) θ n )] .

Original:  P RIVACY COST IN FAIR MACHINE LEARNING
Revised:  P RIVACY COST IN FAIR MACHINE LEARNING

Original: In this section, we wish to study ARE( (cid:101) θ n , (cid:98) θ n ) , the asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n given by solving (2.6) and (2.2).
Revised: In this section, we wish to study ARE( (cid:101) θ n , (cid:98) θ n ) , the asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n given by solving (2.6) and (2.2).

Original: To this end, we extend the notion of demographic parity and equality of opportunity to a more general form: we say that θ is fair (with respect to A ) if
Revised: To this end, we extend the notion of demographic parity and equality of opportunity to a more general form: we say that θ is fair (with respect to A ) if

Original: [Equation]
Revised: [Equation]

Original: The fairness notion (3.1) is known as linear-fractional fairness constraint (Celis et al., 2021).
Revised: The fairness notion (3.1) is known as linear-fractional fairness constraint (Celis et al., 2021).

Original: Note that demographic parity is a special case of (3.1) if we take g ( θ ; X, Y )
Revised: Note that demographic parity is a special case of (3.1) if we take g ( θ ; X, Y )

Original: =
Revised: =

Original: 1 { f θ ( X ) = 1 } and h ≡ 1 .
Revised: 1 { f θ ( X ) = 1 } and h ≡ 1 .

Original: Besides, (3.1) becomes equality of opportunity if we take g ( θ ; X, Y )
Revised: Besides, (3.1) becomes equality of opportunity if we take g ( θ ; X, Y )

Original: =
Revised: =

Original: 1 { f θ ( X ) = 1 , Y = 1 } and h ( X, Y )
Revised: 1 { f θ ( X ) = 1 , Y = 1 } and h ( X, Y )

Original: = 1 { Y = 1 } .
Revised: = 1 { Y = 1 } .

Original: When h ≡ 1 , (3.1) degenerates to linear fairness (see Appendix A).
Revised: When h ≡ 1 , (3.1) degenerates to linear fairness (see Appendix A).

Original: Let the marginal distribution of A and conditional distribution of ( X, Y ) given A be
Revised: Let the marginal distribution of A and conditional distribution of ( X, Y ) given A be

Original: [Equation]
Revised: [Equation]

Original: Then the distribution of ( X, A, Y ) is uniquely identified by (3.2).
Revised: Then the distribution of ( X, A, Y ) is uniquely identified by (3.2).

Original: Moreover, ( X, Y ) ∼ π 0 Q 0 + π 1 Qis a mixture of Q 0 and Q 1 weighted by π 0 and π 1 .
Revised: Moreover, ( X, Y ) ∼ π 0 Q 0 + π 1 Qis a mixture of Q 0 and Q 1 weighted by π 0 and π 1 .

Original: Denote the marginal distribution of Z and conditional distribution of ( X, Y ) given Z by [Equation]
Revised: Denote the marginal distribution of Z and conditional distribution of ( X, Y ) given Z by P ( Z = k ) =

Original: Q [Equation]
Revised: (cid:101) π k , ( X, Y )

Original: Under the private mechanism Q in (2.4), we have [Equation] The marginal distribution and conditional distribution in (3.3) uniquely identify the joint distribution of ( X, Z, Y ) .
Revised: | Z = k ∼ (cid:101) Q k for k ∈ { 0 , 1 } .

Original: Enforcing fairness notion (3.1) with respect to Z is
Revised: Enforcing fairness notion (3.1) with respect to Z is

Original: [Equation]
Revised: [Equation]

Original: By some algebra [Equation] we find that the proxy constraint function (cid:101) c
Revised: By some algebra, we find that the proxy constraint function (cid:101) c

Original: ( θ ) is equal to the true constraint function c ( θ ) up to a scaling factor: (cid:101) c
Revised: ( θ ) is equal to the true constraint function c ( θ ) up to a scaling factor: (cid:101) c

Original: ( θ ) =
Revised: ( θ ) =

Original: ψ frac ( γ, π 0 , π 1 , m 0 , m 1 ) × c ( θ ) , where [Equation] as well m 0 ≜ E Q
Revised: ψ frac ( γ, π 0 , π 1 , m 0 , m 1 ) × c ( θ ) , where ψ frac ( γ, π 0 , π 1 , m 0 , m

Original: [Equation]
Revised: [Equation]

Original: h
Revised: ) ≜

Original: [Equation]
Revised: [Equation]

Original: X
Revised: as well m 0 ≜ E Q 0 (cid:2)

Original: [Equation] Y [Equation] and m 1 ≜ E Q 
Revised: h ( X, Y ) (cid:3) and m 1 ≜ E Q 1 (cid:2) h

Original: [Equation] Y [Equation]
Revised: ( X, Y ) (cid:3) .

Original: This also implies c ( θ ) = 

Revised: This also implies c ( θ ) =

Original: if and only if (cid:101) c ( θ ) = 0 , offering an alternative proof for Proposition 2.3 and extending Proposition 2.3 to linearfractional fairness notions (3.1).
Revised: if and only if (cid:101) c ( θ ) = 0 , offering an alternative proof for Proposition 2.3 and extending Proposition 2.3 to linearfractional fairness notions (3.1).

Original: Now we are ready to show the privacy cost in linear-fractional fairness (3.1)-aware learning.
Revised: Now we are ready to show the privacy cost in linear-fractional fairness (3.1)-aware learning.

Original: First, let the true parameter θ ⋆ , i.e. the solution to the population problem, be
Revised: First, let the true parameter θ ⋆ , i.e. the solution to the population problem, be

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .
Revised: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .

Original: Then, let the estimator (cid:98) θ n be the solution to the empirical problem given the true sensitive attribute, (cid:98) 
Revised: Then, let the estimator (cid:98) θ n be the solution to the empirical problem given the true sensitive attribute, (cid:98) θ n ∈

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: 
Revised: Finally, let (cid:101) θ n be the solution to the empirical problem given the proxy sensitive attribute, (cid:101) θ n ∈

Original: [Equation]
Revised: [Equation]

Original: Finally, let (cid:101) θ n be the solution to the empirical problem given the proxy sensitive attribute, (cid:101) θ n ∈   arg min θ ∈ Θ 1 n (cid:80) ni =1 ℓ ( θ ; X i , Y i ) subject to(cid:12)(cid:12)(cid:12) (cid:80) ni
Revised: subject to

Original: =1 g ( θ ; X i ,Y i ) 1 { Z i =1 } (cid:80) ni
Revised: [Equation]

Original: =1 h ( X i ,Y i ) 1 { Z i =1 } − (cid:80) ni =1 g ( θ ; X i ,Y i ) 1 { Z i =0 } (cid:80) ni
Revised:  

Original: =1 h ( X i ,Y i ) 1 { Z i =0 } (cid:12)(cid:12)(cid:12) ≤ α n   .
Revised: [Equation]

Original: We made the following technical assumptions on the population problem (3.4).
Revised: We made the following technical assumptions on the population problem (3.3).

Original: smoothness and concentration: ℓ and g are twice continuously differentiable with respect to θ , and ℓ ( θ ⋆ ; X, Y ) , ∇ ℓ ( θ ⋆ ; X, Y ) , g ( θ ⋆ ; X, Y ) , ∇ g ( θ ⋆ ; X, Y ) , h ( X, Y ) are sub-Gaussian.
Revised: smoothness and concentration: ℓ and g are twice continuously differentiable with respect to θ , and ℓ ( θ ⋆ ; X, Y ) , ∇ ℓ ( θ ⋆ ; X, Y ) , g ( θ ⋆ ; X, Y ) , ∇ g ( θ ⋆ ; X, Y ) , h ( X, Y ) are sub-Gaussian.

Original: uniqueness: the stochastic optimization problem with a single expected value constraint (3.4) has unique optimal primal-dual pair ( θ ⋆ , λ ⋆ ) , and θ ⋆ belongs to the interior of the compact set Θ .
Revised: uniqueness: the stochastic optimization problem with a single expected value constraint (3.3) has unique optimal primal-dual pair ( θ ⋆ , λ ⋆ ) , and θ ⋆ belongs to the interior of the compact set Θ .

Original: positive definiteness: The Hessian of the Lagrangian evaluated at ( θ ⋆ , λ ⋆ ) is positive definite.
Revised: positive definiteness: The Hessian of the Lagrangian evaluated at ( θ ⋆ , λ ⋆ ) is positive definite.

Original: The preceding assumptions are not the most general, but they are easy to interpret.
Revised: The preceding assumptions are not the most general, but they are easy to interpret.

Original: The smoothness conditions on ℓ and g with respect to θ , the concentration conditions of ℓ ( θ ⋆ ) , g ( θ ⋆ ) and h , and the uniqueness condition facilitate the use of standard tools from asymptotic statistics to study the large sample properties of the constraint value.
Revised: The smoothness conditions on ℓ and g with respect to θ , the concentration conditions of ℓ ( θ ⋆ ) , g ( θ ⋆ ) and h , and the uniqueness condition facilitate the use of standard tools from asymptotic statistics to study the large sample properties of the constraint value.

Original: The positive definiteness condition postulates the
Revised: The positive definiteness condition postulates the

Original: Lagrangian of the equality constrained optimization problem is locally strongly convex at ( θ ⋆ , λ ⋆ ) .
Revised: Lagrangian of the equality constrained optimization problem is locally strongly convex at ( θ ⋆ , λ ⋆ ) .

Original: The main technical result characterizes the efficiency of (cid:98) θ n and (cid:101) θ n (see proof in Appendix C).
Revised: The main technical result characterizes the efficiency of (cid:98) θ n and (cid:101) θ n (see proof in Appendix C).

Original: Theorem 3.1 (Privacy cost in linear-fractional fairness (3.1)-aware learning) .
Revised: Theorem 3.1 (Privacy cost in linear-fractional fairness (3.1)-aware learning) .

Original: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then √
Revised: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then √

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: The asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n is
Revised: The asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n is

Original: ARE( (cid:101) θ n , (cid:98) θ n ) =
Revised: ARE( (cid:101) θ n , (cid:98) θ n ) =

Original: φ (cid:18)
Revised: φ (cid:18)

Original: [Equation]
Revised: [Equation]

Original: 5) where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Recall that demographic parity corresponds to h ≡ 1 and equality of opportunity corresponds to h ( X, Y )
Revised: Recall that demographic parity corresponds to h ≡ 1 and equality of opportunity corresponds to h ( X, Y )

Original: = 1 { Y = 1 } .
Revised: = 1 { Y = 1 } .

Original: In order to interpret (3.5), we therefore take h ( X, Y )
Revised: In order to interpret (3.4), we therefore take h ( X, Y )

Original: =
Revised: =

Original: 1 {E ( X, Y ) } , where E ( X, Y ) is an event of X and Y .
Revised: 1 {E ( X, Y ) } , where E ( X, Y ) is an event of X and Y .

Original: Then the ARE (3.5) becomes
Revised: Then the ARE (3.4) becomes

Original: ARE( (cid:101) θ n , (cid:98) θ n ) =
Revised: ARE( (cid:101) θ n , (cid:98) θ n ) =

Original: φ (cid:18) γ, P ( E ( X, Y ) ,
Revised: φ (cid:18) γ, P ( E ( X, Y ) ,

Original: A = 0)
Revised: A = 0)

Original: P ( E ( X, Y ) , A = 1) ,
Revised: P ( E ( X, Y ) , A = 1) ,

Original: Var[ g ( θ ⋆ ; X, Y ) |E
Revised: Var[ g ( θ ⋆ ; X, Y ) |E

Original: ( X, Y ) ,
Revised: ( X, Y ) ,

Original: A = 0]
Revised: A = 0]

Original: Var[ g ( θ ⋆ ; X, Y ) |E
Revised: Var[ g ( θ ⋆ ; X, Y ) |E

Original: ( X, Y ) ,
Revised: ( X, Y ) ,

Original: A = 1] (cid:19) .
Revised: A = 1] (cid:19) .

Original: Note that the ARE is jointly determined by the level of privacy, a ratio of marginal probabilities of the minority and majority groups, and a ratio of their conditional variances.
Revised: Note that the ARE is jointly determined by the level of privacy, a ratio of marginal probabilities of the minority and majority groups, and a ratio of their conditional variances.

Original: Theorem 3.1 demonstrates that the cost of privacy is the efficiency loss in terms of fairness violations.
Revised: Theorem 3.1 demonstrates that the cost of privacy is the efficiency loss in terms of fairness violations.

Original: For fixed ratios
Revised: For fixed ratios

Original: [Equation]
Revised: [Equation]

Original: and r 2 ≜
Revised: and r 2 ≜

Original: [Equation]
Revised: [Equation]

Original: Var[ g ( θ ⋆ ; X, Y )
Revised: Var[ g ( θ ⋆ ; X, Y )

Original: [Equation]
Revised: [Equation]

Original: E [Equation] X
Revised: E ( X

Original: [Equation]
Revised: [Equation]

Original: Y
Revised: Y

Original: [Equation]
Revised: [Equation]

Original: A = 1] > 0 , function φ ( γ, r 1 , r 2 ) is decreasing in γ .
Revised: A = 1] > 0 , function φ ( γ, r 1 , r 2 ) is decreasing in γ .

Original: In the absence of privacy, φ (0 , r 1 , r 2 ) = 1 means no efficiency loss.
Revised: In the absence of privacy, φ (0 , r 1 , r 2 ) = 1 means no efficiency loss.

Original: Under perfect privacy, φ (0 . 5 , r 1 , r 2 ) = 0 indicates total loss of efficiency.
Revised: Under perfect privacy, φ (0 . 5 , r 1 , r 2 ) = 0 indicates total loss of efficiency.

Original: Moreover, (cid:98) θ n is always more efficient than (cid:101) θ n because ARE( (cid:101) θ n , (cid:98) θ n ) ≤ 1 .
Revised: Moreover, (cid:98) θ n is always more efficient than (cid:101) θ n because ARE( (cid:101) θ n , (cid:98) θ n ) ≤ 1 .

Original: [Figure]
Revised: [Figure]

Original: r
Revised: r

Original: [Figure]
Revised: [Figure]

Original: =
Revised: =

Original: [Figure]
Revised: [Figure]

Original: r
Revised: r

Original: [Figure]
Revised: [Figure]

Original: r
Revised: r

Original: [Figure]
Revised: [Figure]

Original: =
Revised: =

Original: [Figure]
Revised: [Figure]

Original: Figure 1 demonstrates the asymptotic relative efficiency (ARE) curve of privacy level γ for varying ratios r 1 and r 2 .
Revised: Figure 1 demonstrates the asymptotic relative efficiency (ARE) curve of privacy level γ for varying ratios r 1 and r 2 .

Original: The ARE is always upper bounded by (1 − 2 γ ) 2 , which is achieved only if r 1 = 1 .
Revised: The ARE is always upper bounded by (1 − 2 γ ) 2 , which is achieved only if r 1 = 1 .

Original: Therefore for any fixed γ and r 2 , the ARE achieves its maximum only if the dataset is balanced in the sense
Revised: Therefore for any fixed γ and r 2 , the ARE achieves its maximum only if the dataset is balanced in the sense

Original: that P ( E ( X, Y ) , A = 0) =
Revised: that P ( E ( X, Y ) , A = 0) =

Original: P ( E ( X, Y ) , A = 1) .
Revised: P ( E ( X, Y ) , A = 1) .

Original: Moreover, for any fixed γ and r 2 , the ARE is strictly increasing in r 1 (assuming r 1 ≤ 1 ).
Revised: Moreover, for any fixed γ and r 2 , the ARE is strictly increasing in r 1 (assuming r 1 ≤ 1 ).

Original: This implies the effect of subpopulation size imbalance: demographic group imbalance degrades the efficiency loss in privately fair learning.
Revised: This implies the effect of subpopulation size imbalance: demographic group imbalance degrades the efficiency loss in privately fair learning.

Original: In the literature, the effect of group size imbalance on the difficulty of learning fair classifier from contaminated data (note that private sensitive attribute is a particular type of data contamination) was also reported in Konstantinov & Lampert (2022) and the references therein.
Revised: In the literature, the effect of group size imbalance on the difficulty of learning fair classifier from contaminated data (note that private sensitive attribute is a particular type of data contamination) was also reported in Konstantinov & Lampert (2022) and the references therein.

Original: Lastly, the ARE is strictly increasing in the problem-specific parameter r 2 , given fixed γ and r 1 < 1 .
Revised: Lastly, the ARE is strictly increasing in the problem-specific parameter r 2 , given fixed γ and r 1 < 1 .

Original: 3. G ENERAL THEORY
Revised: 3. G ENERAL THEORY

Original: In this subsection, we discuss some extensions to the established theory.
Revised: In this subsection, we discuss some extensions to the established theory.

Original: Multiple demographic groups.
Revised: Multiple demographic groups.

Original: It is natural to extend our theory of two demographic groups to general number of groups.
Revised: It is natural to extend our theory of two demographic groups to general number of groups.

Original: Suppose we have K + 1( K ≥ 2) groups indexed by 0 , 1 , . . .
Revised: Suppose we have K + 1( K ≥ 2) groups indexed by 0 , 1 , . . .

Original: , K .
Revised: , K .

Original: The notion of linear-fractional fairness (3.1) can be adapted to more than two groups: we say θ is fair if
Revised: The notion of linear-fractional fairness (3.1) can be adapted to more than two groups: we say θ is fair if

Original: [Equation]
Revised: [Equation]

Original: where group 0 is referred to as a reference group.
Revised: where group 0 is referred to as a reference group.

Original: Let the marginal distribution of A and conditional distribution of ( X, Y ) given A be
Revised: Let the marginal distribution of A and conditional distribution of ( X, Y ) given A be

Original: [Equation]
Revised: [Equation]

Original: Then the distribution of ( X, A, Y ) is uniquely identified by (3.7).
Revised: Then the distribution of ( X, A, Y ) is uniquely identified by (3.6).

Original: Moreover, the distribution of ( X, Y ) ∼ (cid:80) Kk =

Revised: Moreover, the distribution of ( X, Y ) ∼ (cid:80) Kk =

Original: π k Q k d = Q ⋆ is a mixture of Q k ’s weighted by π k ’s.
Revised: π k Q k d = Q ⋆ is a mixture of Q k ’s weighted by π k ’s.

Original: Let the private mechanism Q be
Revised: Let the private mechanism Q be

Original: [Equation]
Revised: [Equation]

Original: where γ ∈ (cid:104) 0 , 1 K +1 (cid:17) .
Revised: where γ ∈ (cid:104) 0 , 1 K +1 (cid:17) .

Original: The mechanism Q perturbs the membership of a group to a different group that is evenly picked at random from the other groups.
Revised: The mechanism Q perturbs the membership of a group to a different group that is evenly picked at random from the other groups.

Original: The parameter γ = 0 (or equivalently ε = ∞ ) signifies complete lack of privacy, whereas γ → 1 K +1 (or equivalently ε → 0 ) means perfect privacy.
Revised: The parameter γ = 0 (or equivalently ε = ∞ ) signifies complete lack of privacy, whereas γ → 1 K +1 (or equivalently ε → 0 ) means perfect privacy.

Original: The joint distribution of ( X, Z, Y ) is uniquely identified by the marginal distribution and conditional distribution as follows:
Revised: The joint distribution of ( X, Z, Y ) is uniquely identified by the marginal distribution and conditional distribution as follows:

Original: [Equation]
Revised: [Equation]

Original: Let the true parameter θ ⋆ , i.e. the solution to the population problem, be θ ⋆ ∈
Revised: Let the true parameter θ ⋆ , i.e. the solution to the population problem, be θ ⋆ ∈

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original: [Equation]
Revised: [Equation]

Original: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .
Revised: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .

Original: Then, let the estimator (cid:98) θ n be the solution to the empirical problem given the true sensitive attribute, (cid:98)
Revised: Then, let the estimator (cid:98) θ n be the solution to the empirical problem given the true sensitive attribute, (cid:98)

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: Finally, let (cid:101) θ n be the solution to the empirical problem given the proxy sensitive attribute,
Revised: Finally, let (cid:101) θ n be the solution to the empirical problem given the proxy sensitive attribute,

Original: [Equation]
Revised: [Equation]

Original: n ∈
Revised: n ∈

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: The true fairness constraint function c ( θ ) : R d → R K is defined as c ( θ ) ≜ ( c 1 ( θ ) , . .
Revised: The true fairness constraint function c ( θ ) : R d → R K is defined as c ( θ ) ≜ ( c 1 ( θ ) , . .

Original: c K ( θ ))
Revised: c K ( θ ))

Original: ⊤ with c k ( θ ) =
Revised: ⊤ with c k ( θ ) =

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: g
Revised: g

Original: [Equation]
Revised: [Equation]

Original: X
Revised: X

Original: [Equation]
Revised: [Equation]

Original: Under the same assumptions as the two-group problem, we have the main technical result as follows (see Appendix D for a complete treatment to the general-number-of-groups problem).
Revised: Under the same assumptions as the two-group problem, we have the main technical result as follows (see Appendix D for a complete treatment to the general-number-of-groups problem).

Original: Theorem 3.2 (Privacy cost in linear-fractional fairness (3.6)-aware learning) .
Revised: Theorem 3.2 (Privacy cost in linear-fractional fairness (3.5)-aware learning) .

Original: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then
Revised: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then

Original: [Equation]
Revised: [Equation]

Original: } d → 
Revised: } d → N

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: } for k, l ∈ [ K ]
Revised: } for k, l ∈ [ K ]

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Missing sensitive attributes.
Revised: Missing sensitive attributes.

Original: Some users may choose not to disclose their demographic identities during data collection due to privacy concerns.
Revised: Some users may choose not to disclose their demographic identities during data collection due to privacy concerns.

Original: We investigate how the absence of sensitive attributes impacts the generalizability of fairness constraints.
Revised: We investigate how the absence of sensitive attributes impacts the generalizability of fairness constraints.

Original: Consider the following missing data mechanism for sensitive attributes :
Revised: Consider the following missing data mechanism for sensitive attributes :

Original: [Equation]
Revised: [Equation]

Original: where R = 1 corresponds to response ( i.e. , A is observed) and otherwise R = 0 corresponds to non-response ( i.e. , A is missing).
Revised: where R = 1 corresponds to response ( i.e. , A is observed) and otherwise R = 0 corresponds to non-response ( i.e. , A is missing).

Original: The missingness mechanism (3.9) is a particular type of missing at random (MAR) at the population level and missing completely at random (MCAR) within each subpopulation.
Revised: The missingness mechanism (3.8) is a particular type of missing at random (MAR) at the population level and missing completely at random (MCAR) within each subpopulation.

Original: One common approach for analyzing data with missing values is to just use the completely observed samples (i.e., samples with all features observed) and discard the samples with some missing features.
Revised: One common approach for analyzing data with missing values is to just use the completely observed samples (i.e., samples with all features observed) and discard the samples with some missing features.

Original: We employ this strategy by solving the following empirical problem:
Revised: We employ this strategy by solving the following empirical problem:

Original: [Equation]
Revised: [Equation]

Original: to
Revised: to

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original: [Equation]
Revised: [Equation]

Original: of which the empirical risk function is computed with all samples while the fairness constraint function is calculated with samples that include the sensitive attribute.
Revised: of which the empirical risk function is computed with all samples while the fairness constraint function is calculated with samples that include the sensitive attribute.

Original: With the same assumptions as the two-group problem and further assuming that the response probability is non-vanishing, i.e. , ω a > 0 for a ∈ { 0 , 1 } , we have the asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n as follows (see
Revised: With the same assumptions as the two-group problem and further assuming that the response probability is non-vanishing, i.e. , ω a > 0 for a ∈ { 0 , 1 } , we have the asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n as follows (see

Original: Appendix E for a complete treatment to the missing sensitive attributes problem):
Revised: Appendix E for a complete treatment to the missing sensitive attributes problem):

Original: [Equation]
Revised: [Equation]

Original: This indicates that any probability of missing data degrades the asymptotic efficiency of the estimator inversely proportionally.
Revised: This indicates that any probability of missing data degrades the asymptotic efficiency of the estimator inversely proportionally.

Original:  S IMULATIONS
Revised:  S IMULATIONS

Original: We simulate the asymptotic relative efficiency (ARE) for the risk-parity linear regression problem: min β
Revised: We simulate the asymptotic relative efficiency (ARE) for the risk-parity linear regression problem: min β

Original: [Equation]
Revised: [Equation]

Original: Θ
Revised: Θ

Original: E (cid:2) (
Revised: E (cid:2) (

Original: [Equation]
Revised: [Equation]

Original: subject to E [( Y − β ⊤ X ) 2 |
Revised: subject to E [( Y − β ⊤ X ) 2 |

Original: A = 1] − E [( Y − β ⊤ X ) 2 |
Revised: A = 1] − E [( Y − β ⊤ X ) 2 |

Original: A = 0] =
Revised: A = 0] =

Original: (4.1) where we generate n ∈ { 300 , 3000 } samples by the following data generating process:
Revised: (4.1) where we generate n ∈ { 300 , 3000 } samples by the following data generating process:

Original: A ∼ Bernoulli(1 − π 0 ) , X | A = a ∼ N ( µ a , Σ a ) and Y | X, A
Revised: A ∼ Bernoulli(1 − π 0 ) , X | A = a ∼ N ( µ a , Σ a ) and Y | X, A

Original: = a ∼ N ( β ⊤ a X, σ 2 a ) for a ∈ { 0 , 1 } .
Revised: = a ∼ N ( β ⊤ a X, σ 2 a ) for a ∈ { 0 , 1 } .

Original: We pick µ 0 =
Revised: We pick µ 0 =

Original: (1 , 2) ⊤ , µ 1 = (2 , 1) ⊤ , Σ 

Revised: (1 , 2) ⊤ , µ 1 = (2 , 1) ⊤ , Σ

Original: = Σ 1 =
Revised: = Σ 1 =

Original: I 2 , σ 20 = σ 21 = 1 and investigate two scenarios: imbalanced subgroups with π 0 = 0 .
Revised: I 2 , σ 20 = σ 21 = 1 and investigate two scenarios: imbalanced subgroups with π 0 = 0 .

Original: 3 and balanced subgroups with π 0 = 0 .
Revised: 3 and balanced subgroups with π 0 = 0 .

Original: The goal of the optimization problem (4.1) is to minimize the population risk (in least square) while satisfying the parity of subpopulation risks (in least square) of group
Revised: The goal of the optimization problem (4.1) is to minimize the population risk (in least square) while satisfying the parity of subpopulation risks (in least square) of group

Original: A = 0 and group A = 1 .
Revised: A = 0 and group A = 1 .

Original: In Figure 2, we plot relative efficiency curves for π 0 = 0 .
Revised: In Figure 2, we plot relative efficiency curves for π 0 = 0 .

Original: 3 and π 0 = 0 .
Revised: 3 and π 0 = 0 .

Original: 5 , all of which are averaged over 500 replicates.
Revised: 5 , all of which are averaged over 500 replicates.

Original: For large sample size n , the relative efficiency curves are close to the theoretical line of asymptotic relative efficiency curve, validating our theory in the large sample regime.
Revised: For large sample size n , the relative efficiency curves are close to the theoretical line of asymptotic relative efficiency curve, validating our theory in the large sample regime.

Original: As a by-product, our theory can visualize the fairness-privacy trade-off without retraining models with varying privacy budgets.
Revised: As a by-product, our theory can visualize the fairness-privacy trade-off without retraining models with varying privacy budgets.

Original: [Figure]
Revised: [Figure]

Original:  S UMMARY AND DISCUSSION
Revised:  S UMMARY AND DISCUSSION

Original: In this work, we study the statistical impact of privacy on fairness under the task of learning fair machine learning models with private sensitive attributes.
Revised: In this work, we study the statistical impact of privacy on fairness under the task of learning fair machine learning models with private sensitive attributes.

Original: We define a restricted notion of asymptotic statistical efficiency in order to examine such impact.
Revised: We define a restricted notion of asymptotic statistical efficiency in order to examine such impact.

Original: Quantitatively, the cost of privacy on fairness generalizability is represented by a relative decline in statistical efficiency.
Revised: Quantitatively, the cost of privacy on fairness generalizability is represented by a relative decline in statistical efficiency.

Original: The relative efficiency loss is interpretable: it explicitly depends on the privacy budget, subpopulation imbalance level, and a number of other problem-specific quantities.
Revised: The relative efficiency loss is interpretable: it explicitly depends on the privacy budget, subpopulation imbalance level, and a number of other problem-specific quantities.

Original: We validate and demonstrate the utility of our theory by a synthetic task of risk-parity linear regression with private group membership.
Revised: We validate and demonstrate the utility of our theory by a synthetic task of risk-parity linear regression with private group membership.

Original: For the sake of clarity, we consider h ≡ 1 .
Revised: For the sake of clarity, we consider h ≡ 1 .

Original: Denote the loss vectors with regard to the true sensitive attribute A and the noisy sensitive attribute Z , and the Markov transition matrix induced by the privacy mechanism Q (2.4) by
Revised: Denote the loss vectors with regard to the true sensitive attribute A and the noisy sensitive attribute Z , and the Markov transition matrix induced by the privacy mechanism Q (2.4) by

Original: [Equation]
Revised: [Equation]

Original: Further, let b =
Revised: Further, let b =

Original: (1 , − 1) ⊤ .
Revised: (1 , − 1) ⊤ .

Original: Noiseless, noisy, and debiased constraints are equivalent to each other at the population level in the way that b ⊤ L A ( θ ) = 

Revised: Noiseless, noisy, and debiased constraints are equivalent to each other at the population level in the way that b ⊤ L A ( θ ) =

Original: ⇐⇒ b ⊤ L Z ( θ ) = 

Revised: ⇐⇒ b ⊤ L Z ( θ ) =

Original: ⇐⇒ b ⊤ M − 1 L Z ( θ ) = 0 .
Revised: ⇐⇒ b ⊤ M − 1 L Z ( θ ) = 0 .

Original: Consider their empirical counterparts, we note that b ⊤ (cid:98) L Z,n ( θ ) = 

Revised: Consider their empirical counterparts, we note that b ⊤ (cid:98) L Z,n ( θ ) =

Original: ⇐⇒ b ⊤ M − 1 (cid:98) L Z,n ( θ ) = 0 .
Revised: ⇐⇒ b ⊤ M − 1 (cid:98) L Z,n ( θ ) = 0 .

Original: Combined with our theory, this empirical level equivalence of two constraints implies that using the inverse of the empirical transition matrix to match the noisy constraint to the noiseless constraint cannot improve the efficiency of the in-processing training procedure.
Revised: Combined with our theory, this empirical level equivalence of two constraints implies that using the inverse of the empirical transition matrix to match the noisy constraint to the noiseless constraint cannot improve the efficiency of the in-processing training procedure.

Original: Developing a principled inprocessing method to increase the statistical efficiency is an intriguing direction for future research.
Revised: Developing a principled inprocessing method to increase the statistical efficiency is an intriguing direction for future research.

Original: A L INEAR FAIRNESS CONSTRAINT
Revised: A L INEAR FAIRNESS CONSTRAINT

Original: We extend the notion of demographic parity to a more general form: we say that θ is fair (with respect to A ) if
Revised: We extend the notion of demographic parity to a more general form: we say that θ is fair (with respect to A ) if

Original: [Equation]
Revised: [Equation]

Original: The fairness notion (A.1) is known as linear fairness constraint Celis et al.
Revised: The fairness notion (A.1) is known as linear fairness constraint Celis et al.

Original: Note that demographic parity is a special case of (A.1) if we take g ( θ ; X, Y )
Revised: Note that demographic parity is a special case of (A.1) if we take g ( θ ; X, Y )

Original: =
Revised: =

Original: 1 { f θ ( X ) = 1 } .
Revised: 1 { f θ ( X ) = 1 } .

Original: On the one hand, enforcing fairness notion (A.1) with respect to A is
Revised: On the one hand, enforcing fairness notion (A.1) with respect to A is

Original: [Equation]
Revised: [Equation]

Original: or equivalently
Revised: or equivalently

Original: [Equation]
Revised: [Equation]

Original: On the other hand, enforcing fairness notion (A.1) with respect to Z is
Revised: On the other hand, enforcing fairness notion (A.1) with respect to Z is

Original: [Equation]
Revised: [Equation]

Original: or equivalently
Revised: or equivalently

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: the true fairness constraint function is
Revised: the true fairness constraint function is

Original: [Equation]
Revised: [Equation]

Original: while the proxy fairness constraint function is
Revised: while the proxy fairness constraint function is

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: By (A.2)
Revised: By (A.2)

Original: [Equation]
Revised: [Equation]

Original: the proxy constraint function (cid:101) c
Revised: the proxy constraint function (cid:101) c

Original: ( θ ) is equal to the true c ( θ ) up to a scaling factor
Revised: ( θ ) is equal to the true c ( θ ) up to a scaling factor

Original: [Equation]
Revised: [Equation]

Original: This also implies c ( θ ) = 

Revised: This also implies c ( θ ) =

Original: if and only if (cid:101) c ( θ ) = 0 , providing an alternative proof for Proposition 2.3.
Revised: if and only if (cid:101) c ( θ ) = 0 , providing an alternative proof for Proposition 2.3.

Original: Now we are ready to show the privacy cost in linear fairness (A.1)-aware learning.
Revised: Now we are ready to show the privacy cost in linear fairness (A.1)-aware learning.

Original: First, let the true parameter θ ⋆ , i.e. the solution to the population problem, be
Revised: First, let the true parameter θ ⋆ , i.e. the solution to the population problem, be

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .
Revised: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .

Original: Then, let the estimator (cid:98) θ n be the solution to the empirical problem given the true sensitive attribute,
Revised: Then, let the estimator (cid:98) θ n be the solution to the empirical problem given the true sensitive attribute,

Original: [Equation]
Revised: [Equation]

Original: to
Revised: to

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally

Original: [Equation]
Revised: [Equation]

Original: let the estimator (cid:101) θ n be the solution to the empirical problem given the proxy sensitive attribute,
Revised: let the estimator (cid:101) θ n be the solution to the empirical problem given the proxy sensitive attribute,

Original: [Equation]
Revised: [Equation]

Original: We made the following technical assumptions on the problem (A.4).
Revised: We made the following technical assumptions on the problem (A.4).

Original: smoothness and concentration: ℓ and g are twice continuously differentiable with respect to θ , and ℓ ( θ ⋆ ; X, Y ) , ∇ ℓ ( θ ⋆ ; X, Y ) , g ( θ ⋆ ; X, Y ) , ∇ g ( θ ⋆ ; X, Y ) are sub-Gaussian random variables.
Revised: smoothness and concentration: ℓ and g are twice continuously differentiable with respect to θ , and ℓ ( θ ⋆ ; X, Y ) , ∇ ℓ ( θ ⋆ ; X, Y ) , g ( θ ⋆ ; X, Y ) , ∇ g ( θ ⋆ ; X, Y ) are sub-Gaussian random variables.

Original: uniqueness: the stochastic optimization problem with
Revised: uniqueness: the stochastic optimization problem with

Original: a single expected value constraint (A.4) has a unique optimal primal-dual pair ( θ ⋆ , λ ⋆ ) , and θ ⋆ belongs to the interior of the compact set Θ .
Revised: a single expected value constraint (A.4) has a unique optimal primal-dual pair ( θ ⋆ , λ ⋆ ) , and θ ⋆ belongs to the interior of the compact set Θ .

Original: positive definiteness: The Hessian of the Lagrangian evaluated at ( θ ⋆ , λ ⋆ ) is positive definite.
Revised: positive definiteness: The Hessian of the Lagrangian evaluated at ( θ ⋆ , λ ⋆ ) is positive definite.

Original: We have the main technical result as follows.
Revised: We have the main technical result as follows.

Original: Theorem A.1 (Privacy cost in linear fairness (A.1)-aware learning) .
Revised: Theorem A.1 (Privacy cost in linear fairness (A.1)-aware learning) .

Original: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then √ n { c
Revised: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then √ n { c

Original: [Equation]
Revised: [Equation]

Original: n )
Revised: n )

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: The asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n is
Revised: The asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n is

Original: ARE
Revised: ARE

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Proof of Theorem A.1.
Revised: Proof of Theorem A.1.

Original: Note that Theorem 3.1 implies Theorem A.1 by letting h ( X, Y ) ≡ 1 .
Revised: Note that Theorem 3.1 implies Theorem A.1 by letting h ( X, Y ) ≡ 1 .

Original: Therefore, it is sufficient to prove Theorem 3.1, whose proof can be found in Appendix C.
Revised: Therefore, it is sufficient to prove Theorem 3.1, whose proof can be found in Appendix C.

Original: □
Revised: □

Original: Theorem A.1 demonstrates that the cost of privacy is the efficiency loss in terms of fairness violations.
Revised: Theorem A.1 demonstrates that the cost of privacy is the efficiency loss in terms of fairness violations.

Original: For fixed ratios r 1 ≜ π 0 /π 1 > 0 and r 2 ≜ Var[ g ( θ ⋆ ; X, Y )
Revised: For fixed ratios r 1 ≜ π 0 /π 1 > 0 and r 2 ≜ Var[ g ( θ ⋆ ; X, Y )

Original: |
Revised: |

Original: A = 0] / Var[ g ( θ ⋆ ; X, Y )
Revised: A = 0] / Var[ g ( θ ⋆ ; X, Y )

Original: |
Revised: |

Original: A = 1] > 0 , φ lin ( γ, r 1 , r 2 ) is a decreasing function in γ .
Revised: A = 1] > 0 , φ lin ( γ, r 1 , r 2 ) is a decreasing function in γ .

Original: In the absence of privacy, φ lin (0 , r 1 , r 2 ) =means no efficiency loss.
Revised: In the absence of privacy, φ lin (0 , r 1 , r 2 ) =means no efficiency loss.

Original: Under perfect privacy, φ lin (0 . 5 , r 1 , r 2 ) = 0 indicates total loss of efficiency.
Revised: Under perfect privacy, φ lin (0 . 5 , r 1 , r 2 ) = 0 indicates total loss of efficiency.

Original: Moreover, (cid:98) θ n is always more efficient than (cid:101) θ n because ARE( (cid:101) θ n , (cid:98) θ n ) ≤ 1 .
Revised: Moreover, (cid:98) θ n is always more efficient than (cid:101) θ n because ARE( (cid:101) θ n , (cid:98) θ n ) ≤ 1 .

Original: Figure 3 demonstrates the asymptotic relative efficiency (ARE) curve of privacy level γ for varying ratios r 1 and r 2 .
Revised: Figure 3 demonstrates the asymptotic relative efficiency (ARE) curve of privacy level γ for varying ratios r 1 and r 2 .

Original: The ARE is always upper bounded by (1 − 2 γ ) 2 , which is achieved only if π 0 = π 1 = 0 .
Revised: The ARE is always upper bounded by (1 − 2 γ ) 2 , which is achieved only if π 0 = π 1 = 0 .

Original: Recall that π 0 = P
Revised: Recall that π 0 = P

Original: ( A = 0) and π 1 = P ( A = 1) .
Revised: ( A = 0) and π 1 = P ( A = 1) .

Original: Therefore for any fixed γ and r 2 , the
Revised: Therefore for any fixed γ and r 2 , the

Original: ARE achieves its maximum only if the dataset is balanced in the sensitive attribute A .
Revised: ARE achieves its maximum only if the dataset is balanced in the sensitive attribute A .

Original: Moreover, for any fixed γ and r 2 , the ARE is strictly increasing in π 0 (assuming π 0 < 0 . 5 ).
Revised: Moreover, for any fixed γ and r 2 , the ARE is strictly increasing in π 0 (assuming π 0 < 0 . 5 ).

Original: This implies the effect of subgroup size imbalance: demographic group imbalance degrades the efficiency loss in privately fair learning.
Revised: This implies the effect of subgroup size imbalance: demographic group imbalance degrades the efficiency loss in privately fair learning.

Original: Lastly, the ARE is strictly increasing in the problem-specific parameter r 2 , given fixed γ and r 1 < 1 .
Revised: Lastly, the ARE is strictly increasing in the problem-specific parameter r 2 , given fixed γ and r 1 < 1 .

Original: [Figure]
Revised: [Figure]

Original: a s y m p t
Revised: a s y m p t

Original: o t i c r e l a t i v e e ff
Revised: o t i c r e l a t i v e e ff

Original: i c
Revised: i c

Original: i e n c y (
Revised: i e n c y (

Original: A
Revised: A

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: B L INEAR - FRACTIONAL FAIRNESS CONSTRAINT
Revised: B L INEAR - FRACTIONAL FAIRNESS CONSTRAINT

Original: We provide further discussion to supplement Section 3.
Revised: We provide further discussion to supplement Section 3.

Original: Recall the marginal distributions and conditional distributions in (3.2) and (3.3).
Revised: Recall the marginal distributions and conditional distributions in (3.2) and (cid:40) [Equation]

Revised: Under the private mechanism Q in (2.4), we have

Revised: [Equation]

Revised: The marginal distribution and conditional distribution in (B.1) uniquely identify the joint distribution of ( X, Z, Y )

Revised: [Equation]

Original: On the one hand, enforcing fairness notion (3.1) with respect to A is
Revised: On the one hand [Equation] enforcing fairness notion (3.1) with respect to A is

Original: [Equation]
Revised: [Equation]

Original: or equivalently
Revised: or equivalently

Original: [Equation]
Revised: [Equation]

Original: On the other hand, enforcing fairness notion (3.1) with respect to Z is
Revised: On the other hand [Equation] enforcing fairness notion (3.1) with respect to Z is

Original: [Equation]
Revised: [Equation]

Original: or equivalently 
Revised: or equivalently (cid:101)

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  

Original: [Equation]
Revised: [Equation]

Revised: g ( θ ; X, Y ) (cid:3) + (1 − γ ) π 1 E Q 1 (cid:2) g

Revised: ( θ ; X, Y ) (cid:3) γπ 0 E Q 0 (cid:2) h

Revised: ( X, Y ) (cid:3)

Revised: + (1 − γ )

Revised: π 1 E Q 1 (cid:2) h ( X, Y ) (cid:3) − (1 − γ )

Revised: π 0 E Q 0 (cid:2) g

Revised: ( θ ; X, Y ) (cid:3) + γπ 1 E Q 1 (cid:2) g

Revised: ( θ ; X, Y ) (cid:3) (1 − γ )

Revised: π 0 E Q 0 (cid:2) h

Revised: ( X, Y ) (cid:3) + γπ 1 E Q 1 (cid:2) h

Revised: ( X, Y ) (cid:3)   = 0 .

Original: By some algebra, we find that the proxy constraint function (cid:101) c
Revised: By some algebra, we find that the proxy constraint function (cid:101) c

Original: ( θ ) is equal to the true constraint function c ( θ ) up to a scaling factor: (cid:101) c
Revised: ( θ ) is equal to the true constraint function c ( θ ) up to a scaling factor: (cid:101) c

Original: ( θ ) =
Revised: ( θ ) =

Original: ψ frac ( γ, π 0 , π 1 , m 0 , m 1 ) × c ( θ ) , where 
Revised: ψ frac ( γ, π 0 , π 1 , m 0 , m 1 ) × c ( θ ) , where ψ frac ( γ, π 0 , π 1 , m 0 , m 1 ) ≜

Original: [Equation]
Revised: [Equation]

Original: as well m 0 ≜ E Q 0 (cid:2)
Revised: as well m 0 ≜ E Q 0 (cid:2)

Original: h ( X, Y ) (cid:3) and m 1 ≜ E Q 1 (cid:2) h
Revised: h ( X, Y ) (cid:3) and m 1 ≜ E Q 1 (cid:2) h

Original: ( X, Y ) (cid:3) .
Revised: ( X, Y ) (cid:3) .

Original: By comparing the scaling factor (B.1) with the functional form of (A.3), we can rewrite ψ frac ( · ) by [Equation] frac ( γ [Equation]
Revised: By comparing the scaling factor (B.2) with the functional form of (A.3), we can rewrite ψ frac ( · ) by ψ frac ( γ, π 0 , π 1 , m 0 , m 1 )

Original: π

Original: [Equation]

Original: π

Original: [Equation]

Original: [Equation]

Original: [Equation] lin ( γ [Equation] π 0 m 0 , π 1 m 1 ) .
Revised: = ψ lin ( γ, π 0 m 0 , π 1 m 1 ) .

Original: Therefore, we can interpret the scaling factor ψ frac ( · ) by treating π 0 m 0 and π 1 m 1 as a whole, allowing us to understand the privacy cost from a different perspective.
Revised: Therefore, we can interpret the scaling factor ψ frac ( · ) by treating π 0 m 0 and π 1 m 1 as a whole, allowing us to understand the privacy cost from a different perspective.

Original: Note that for equality of opportunity, we have π a m a = P
Revised: Note that for equality of opportunity, we have π a m a = P

Original: ( A = a ) P ( Y = 1 |
Revised: ( A = a ) P ( Y = 1 |

Original: A = a ) =
Revised: A = a ) =

Original: P ( Y = 1 , A = a ) for a ∈ { 0 , 1 } .
Revised: P ( Y = 1 , A = a ) for a ∈ { 0 , 1 } .

Original: For equality of opportunity, Mozannar et al.
Revised: For equality of opportunity, Mozannar et al.

Original: (2020) show a sample complexity bound for the fairness violation of the estimator (cid:101) θ n :
Revised: (2020) show a sample complexity bound for the fairness violation of the estimator (cid:101) θ n :

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − δ , where p = min { P ( Y = 1 , A = 0) , P ( Y = 1 , A = 1) } , R · ( · ) is the Rademacher complexity, and C i ’s (1 ≤ i ≤ 4) are some universal constants.
Revised: with probability at least 1 − δ , where p = min { P ( Y = 1 , A = 0) , P ( Y = 1 , A = 1) } , R · ( · ) is the Rademacher complexity [Figure] and C i ’s (1 ≤ i ≤ 4) are some universal constants.

Original: Not precisely, the upper bound (B.2) reflects the effect of privacy level via γ and the effect of dataset imbalance through p .
Revised: Not precisely, the upper bound (B.3) reflects the effect of privacy level via γ and the effect of dataset imbalance through p .

Original: Comparing to this, our theory states that
Revised: Comparing to this, our theory states that

Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: which is depicted by Figure 4.
Revised: which is depicted by Figure 4.

Original: [Figure]
Revised: [Figure]

Original: a s y m p t

Original: o t i c r e l a t i v e e ff

Original: i c

Original: i e n c y (

Original: [-A R E ) 
-]
Revised: [-A R E ) 
-]

Original: m 0 : 1 m 1 =1: 0 m 0 : 1 m 1 =1: 0 m 0 : 1 m 1 =1:

Original: C P ROOF OF T HEOREM 3. 
Revised: C P ROOF OF T HEOREM 3.

Original: First, we prove the case when α n = 0 for all n .
Revised: First, we prove the case when α n = 0 for all n .

Original: For this case both the population problem and the empirical problem are subject to equality constraints.
Revised: For this case both the population problem and the empirical problem are subject to equality constraints.

Original: Consider a stochastic optimization problem with linear-fractional constraint ( P0 ) :
Revised: Consider a stochastic optimization problem with linear-fractional constraint [Equation] P

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .
Revised: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .

Original: The corresponding empirical problem given the true sensitive attribute is [Equation] P n ) :
Revised: The corresponding empirical problem given the true sensitive attribute is ( P n ) :

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  

Original: [Equation]
Revised: [Equation]

Original: The corresponding empirical problem given the proxy sensitive attribute is 
Revised: The corresponding empirical problem given the proxy sensitive attribute is (

Original: [Equation]
Revised: [Equation]

Original: P n ) :
Revised: P n ) :

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original:   
Revised: Z i = 0 } (cid:80) ni =1 h ( X i , Y i ) 1 { Z i = 0 } =  . We denote F ( θ

Original: [Equation]
Revised: [Equation]

Original: We denote
Revised: = E (cid:2) [Equation] θ

Original: [Equation]
Revised: [Equation]

Original: and
Revised: X, Y [Equation] [Equation] F n

Original: [Equation]
Revised: [Equation]

Revised: n n

Original:  Note that (cid:98) F n ( · ) and (cid:98) G n ( · ) ’s are random functions serving as approximations to F ( · ) and G ( · ) ’s.
Revised: [Equation] and (cid:98) [Equation] Note that (cid:98) F n ( · ) and (cid:98) G n ( · ) ’s are random functions serving as approximations to F ( · ) and G ( · ) ’s.

Original: Consider the Lagrangian functions
Revised: Consider the Lagrangian functions

Revised: L ( θ, λ ) =

Revised: F ( θ ) + λG ( θ ) and (cid:98) L n

Original: [Equation]
Revised: [Equation]

Original: of the programs ( P 0 ) and ( P n ) respectively. Lemma C
Revised: , λ ) =

Original: [Equation]
Revised: [Equation]

Original: 1 (A version of Theorem 6.
Revised: F n

Original: [Equation]
Revised: [Equation]

Revised: θ ) + λ (cid:98) G n ( θ ) of the programs ( P 0 ) and ( P n ) respectively.

Original: 2 in Rubinstein & Shapiro (1993)) .
Revised: Lemma C.1 (A version of Theorem 6.6.2 in Rubinstein & Shapiro (1993)) .

Original: Suppose that:
Revised: Suppose that:

Original: (i)
Revised: (i)

Original: The functions F ( θ ) and G ( θ ) are twice continuously differentiable.
Revised: The functions F ( θ ) and G ( θ ) are twice continuously differentiable.

Original: (ii)
Revised: (ii)

Original: The true program ( P 0 ) has a unique optimal solution θ ⋆ and a unique Lagrange multiplier λ
Revised: The true program ( P 0 ) has a unique optimal solution θ ⋆ and a unique Lagrange multiplier λ

Original: ⋆ with θ ⋆ being an interior point of Θ .
Revised: ⋆ with θ ⋆ being an interior point of Θ .

Original: (iii)
Revised: (iii)

Original: The Hessian matrix ∇ 2 L ( θ ⋆ , λ ⋆ ) is positive definite.
Revised: The Hessian matrix ∇ 2 L ( θ ⋆ , λ ⋆ ) is positive definite.

Original: (iv) The random functions (cid:98) G n ( θ ) , k ∈
Revised: (iv) The random functions (cid:98) G n ( θ ) , k ∈

Original: [ K ] , are Lipschitz continuous in a neighborhood of θ ⋆ and differentiable at θ ⋆ with probability 1.
Revised: [ K ] , are Lipschitz continuous in a neighborhood of θ ⋆ and differentiable at θ ⋆ with probability 1.

Original: (v)
Revised: (v)

Original: [Equation]
Revised: [Equation]

Original: and there is a neighborhood U of θ ⋆ such that
Revised: and there is a neighborhood U of θ ⋆ such that

Original: [Equation]
Revised: [Equation]

Original: Here we define random mappings ∆ 1 n ( θ ) = ∇ (cid:98) F n ( θ )
Revised: Here we define random mappings ∆ 1 n ( θ ) = ∇ (cid:98) F n ( θ )

Original: − ∇ F ( θ ) , ∆ 2 n ( θ ) =
Revised: − ∇ F ( θ ) , ∆ 2 n ( θ ) =

Original: (cid:98) G n ( θ ) − G ( θ ) , and ∆ 3 n ( θ ) =
Revised: (cid:98) G n ( θ ) − G ( θ ) , and ∆ 3 n ( θ ) =

Original: ∇ (cid:98) G n ( θ ) − ∇ G ( θ ) .
Revised: ∇ (cid:98) G n ( θ ) − ∇ G ( θ ) .

Original: (vi) Random vectors √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , (cid:98) G n ( θ ⋆ )) converge in distribution to Y =
Revised: (vi) Random vectors √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , (cid:98) G n ( θ ⋆ )) converge in distribution to Y =

Original: ( Y 1 , Y 2 ) as n → ∞ , where Y 1 is a random vector and Y 2 is a random variable.
Revised: ( Y 1 , Y 2 ) as n → ∞ , where Y 1 is a random vector and Y 2 is a random variable.

Original: Let (cid:98) θ n be an optimal solution of ( P n ) converging in probability as n → ∞ to θ ⋆ .
Revised: Let (cid:98) θ n be an optimal solution of ( P n ) converging in probability as n → ∞ to θ ⋆ .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where ¯ x = ¯ x ( Y ) is the optimal solution to the quadratic programming problem minimize xx
Revised: where ¯ x = ¯ x ( Y ) is the optimal solution to the quadratic programming problem minimize x

Original: [Equation]
Revised: [Equation]

Original: +
Revised: subject to ∇ G

Original: [Equation]
Revised: [Equation]

Original: x subject to ∇ G ( θ ⋆ )

Original: ⊤ x + Y 2 =.

Original: Recall the standing assumptions, (i) , (iv) , (v) are guaranteed by the smoothness and concentration assumption, (ii) is postulated by the uniqueness assumption, and (iii) is made by our assumption.
Revised: Recall the standing assumptions, (i) , (iv) , (v) are guaranteed by the smoothness and concentration assumption, (ii) is postulated by the uniqueness assumption, and (iii) is made by our assumption.

Original: Now we derive the limiting distribution of random vectors √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , (cid:98) G n ( θ ⋆ )) required in (vi) .
Revised: Now we derive the limiting distribution of random vectors √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , (cid:98) G n ( θ ⋆ )) required in (vi) .

Original: For a ∈ { 0 , 1 } , we have
Revised: For a ∈ { 0 , 1 } , we have

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Similarly, for a ∈ { 0 , 1 } , we have
Revised: Similarly, for a ∈ { 0 , 1 } , we have

Original: E (cid:2) h ( X, Y ) 1 { A = a } (cid:3) =
Revised: E (cid:2) h ( X, Y ) 1 { A = a } (cid:3) =

Original: π a E Q a [ h ] and
Revised: π a E Q a [ h ] and

Revised: Var[ h ( X, Y )

Original: [Equation]
Revised: [Equation]

Original: Moreover, we have

Original: Cov
Revised: a [

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Revised: Moreover, we have

Original: [Equation]{+Cov( g ( θ ⋆ ; X, Y ) 1 { A = 1 } , g ( θ ⋆ ; X, Y ) 1 { A = 0 } ) = E (cid:2) g 2 ( θ ⋆ ; X, Y ) 1 { A = 0 } 1 { A = 1 } (cid:3) − E (cid:2) g ( θ ⋆ ; X, Y ) 1 { A = 0 } (cid:3) × E (cid:2) g ( θ ⋆ ; X, Y ) 1 { A = 1 } (cid:3) = − π 0 π 1 E Q 
+}
Revised: {+Cov( g ( θ ⋆ ; X, Y ) 1 { A = 1 } , g ( θ ⋆ ; X, Y ) 1 { A = 0 } ) = E (cid:2) g 2 ( θ ⋆ ; X, Y ) 1 { A = 0 } 1 { A = 1 } (cid:3) − E (cid:2) g ( θ ⋆ ; X, Y ) 1 { A = 0 } (cid:3) × E (cid:2) g ( θ ⋆ ; X, Y ) 1 { A = 1 } (cid:3) = − π 0 π 1 E Q 
+}

Original: [Equation] ] and similarly we can derive
Revised: [ g ] E Q 1 [ g ] and similarly we can derive

Original: [Equation]{+Cov( h ( X, Y ) 1 { A = 1 } , h ( X, Y ) 1 { A = 0 } ) = − π 0 π 1 E Q 
 [ h ] E Q 1 [ h ] , Cov( g ( θ ⋆ ; X, Y ) 1 { A = a } , h ( X, Y ) 1 { A = a } ) =+}
Revised: {+Cov( h ( X, Y ) 1 { A = 1 } , h ( X, Y ) 1 { A = 0 } ) = − π 0 π 1 E Q 
 [ h ] E Q 1 [ h ] , Cov( g ( θ ⋆ ; X, Y ) 1 { A = a } , h ( X, Y ) 1 { A = a } ) =+}

Original:  and 
Revised: π a Cov Q a [ g, h ] + π 0 π 1 E Q a [ g ] E Q a [ h ] and Cov( g ( θ ⋆ ; X, Y ) 1 { A = a } , h ( X, Y ) 1 { A = 1 − a } )

Original: [Equation] for a ∈ { 0 , 1 } .
Revised: = − π 0 π 1 E Q a [ g ] E Q 1 − a [ h ] for a ∈ { 0 , 1 } .

Original: Let η 

Revised: Let η

Original: =
Revised: =

Original: E (cid:2) ∇ ℓ ( θ ⋆ ; X, Y ) (cid:3) , η 2 = π 1 E Q 1 (cid:2) ∇ g ( θ ⋆ ; X, Y ) (cid:3) and η 3 = π 0 E Q 0 (cid:2) ∇ g ( θ ⋆ ; X, Y ) (cid:3) .
Revised: E (cid:2) ∇ ℓ ( θ ⋆ ; X, Y ) (cid:3) , η 2 = π 1 E Q 1 (cid:2) ∇ g ( θ ⋆ ; X, Y ) (cid:3) and η 3 = π 0 E Q 0 (cid:2) ∇ g ( θ ⋆ ; X, Y ) (cid:3) .

Original: By central limit theorem, 
Revised: By central limit theorem, √ n     [Equation] ( θ ⋆ ; X i , Y i ) n − [Equation]

Revised: (cid:80) ni =

Original: [Equation]
Revised: [Equation]

Revised: g ( θ

Revised: [Equation]

Revised: X i , Y i )

Revised: [Equation]

Original:  n 
Revised: A [Equation] } n − [Equation]

Revised: (cid:80) ni =

Revised: [Equation]

Revised: ∇ g ( θ ⋆

Revised: [Equation]

Revised: X

Revised: [Equation]

Revised: Y i

Original: [Equation]
Revised: [Equation]

Revised: A

Revised: [Equation]

Revised: 0 } n −

Revised: [Equation]

Revised: (cid:80) ni =

Revised: [Equation]

Revised: g ( θ ⋆

Revised: [Equation]

Revised: X i

Revised: [Equation]

Revised: Y i

Revised: [Equation]

Revised: A i

Revised: [Equation]

Revised: 1 } n −

Revised: [Equation]

Revised: (cid:80) ni

Original: 
Revised: [Equation] 1 g ( θ ⋆ [Equation] X i , Y i )

Revised: [Equation]

Revised: A i

Revised: [Equation]

Revised: 0 } n −

Revised: [Equation]

Revised: (cid:80) ni =

Revised: [Equation]

Revised: h ( X i , Y i )

Revised: [Equation]

Revised: A

Revised: [Equation]

Revised: 1 } n −

Revised: [Equation]

Revised: (cid:80) ni

Revised: [Equation]

Revised: h ( X i , Y

Revised: [Equation]

Revised: { A

Original: [Equation]
Revised: [Equation]

Original: {+0 }   −   η 1 ηη 3 π 1 E Q 1 [ g ] π 0 E Q 
 [ g ] π 1 E Q 1 [ h ] π 0 E Q 
+}
Revised: {+0 }   −   η 1 ηη 3 π 1 E Q 1 [ g ] π 0 E Q 
 [ g ] π 1 E Q 1 [ h ] π 0 E Q 
+}

Original: [Equation]
Revised: [ h ] 

Original:    
Revised:   

Revised: d −→ N (cid:18) 0 , (cid:20) ΩΩ 12 ΩΩ(cid:21)(cid:19) , (C.1) where Ω 11 ∈ R 3 d × 3 d , Ω 21 ∈ R 4 × 3 d , Ω 12 = Ω ⊤ 21 , Ω 22 is given by   π

Original: [Equation]
Revised: [Equation]

Original: Ω
Revised: [ g ] +

Original: [Equation]
Revised: [Equation]

Original: (cid:21)(cid:19) , (C.1) where Ω 11 ∈ R 3 d × 3 d , Ω 21 ∈ R 4 × 3 d , Ω 12 = Ω ⊤ 21 , Ω 22 is given by  
Revised: π

Original: [Equation]
Revised: [Equation]

Original:  
Revised: 1 g )

Original: [Equation]
Revised: [Equation]

Original: Let function w : R d × R d × R d × R × R × R × R → R d +1 be
Revised: Let function w : R d × R d × R [Equation] × [Equation]

Revised: 1 be

Original: [Equation]
Revised: [Equation]

Original: The gradient of function w evaluated at ( v 1 , v 2 , v 3 ) =
Revised: The gradient of function w evaluated at ( v 1 , v 2 , v 3 ) =

Original: ( η 1 , η 2 , η 3 ) and ( s 1 , s 2 , s 3 , s 4 )
Revised: ( η 1 , η 2 , η 3 ) and ( s 1 , s 2 , s 3 , s 4 )

Original: =
Revised: =

Original: ( π 1 E Q 1 [ g ] , π[-0 E Q 
-]
Revised: ( π 1 E Q 1 [ g [Equation] , π[-0 E Q 
-]

Original: [-[ g ] , π 1 E Q 1 [ h ] , π 0 E Q 
-]
Revised: [-[ g ] , π 1 E Q 1 [ h ] , π 0 E Q 
-][Equation] [Equation] [Equation] [Equation]

Original: [ h ]) is given by
Revised: ]) is given by

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Applying delta method to (C [Equation] 1) with w ( · )
Revised: Applying delta method to (C.1) with w ( · )

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original:  where
Revised: , where

Original: [Equation]
Revised: [Equation]

Original: Note that KKT condition implies
Revised: Note that KKT condition implies

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Combining (C.2) and (C.3), we have
Revised: Combining (C.2) and (C.3), we have

Original: [Equation]
Revised: [Equation]

Original: Therefore, we conclude that the limiting distribution of √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , G n ( θ ⋆ )) is( Y 1 , Y 2 ) ∼ N (cid:18) 0 , (cid:20) ΣΣΣσ
Revised: Therefore [Equation] we conclude that the limiting distribution of √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , G n ( θ ⋆ )) is

Original: [Equation]
Revised: [Equation]

Original: (cid:21)(cid:19) .

Original: By Lemma (C.1), we have
Revised: By Lemma (C.1)

Original: [Equation]
Revised: [Equation]

Original: where ¯ x is given by the linear system (cid:20)
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: ) ∇ G ( θ ⋆ ) ⊤ (cid:21) (cid:124) (cid:123)(cid:122) (cid:125) ≜ B (cid:20) ¯ 
Revised: where ¯ x is given by the linear system

Original: [Equation]
Revised: [Equation]

Original:  ¯ λ (cid:21) =− (cid:20) Y
Revised: (cid:20) ¯ x ¯ λ (cid:21) =

Original:  Y[Equation]
Revised: − (cid:20) Y

Original: (cid:21) ∼ N (cid:18) 0 , (cid:20) ΣΣΣσ(cid:21)(cid:19) , or (cid:20) ¯ x ¯ λ (cid:21) ∼ N (cid:18) 0 , B − 1 (cid:20) ΣΣΣσ(cid:21)
Revised: Y(cid:21) ∼ N (cid:18) 0 , (cid:20) ΣΣΣσ(cid:21)(cid:19) , or (cid:20) ¯ x ¯ λ (cid:21) ∼ N (cid:18) 0 , B − 1 (cid:20) ΣΣΣσ(cid:21)

Original: B − 1 (cid:19) , (C.5) which implies √ n ( (cid:98) θ n − θ ⋆ ) d −→ ¯ x ∼ N ( 0 , ¯Σ) for some ¯ µ and ¯Σ determined by (C.5).
Revised: B − 1 (cid:19) , (C.5) which implies √ n ( (cid:98) θ n − θ ⋆ ) d −→ ¯ x ∼ N ( 0 , ¯ Σ) for some ¯ µ and ¯ Σ determined by (C.5).

Original: By delta method, we have
Revised: By delta method, we have

Original: [Equation]
Revised: [Equation]

Original: Now we calculate ∇ G ( θ ⋆ )
Revised: Now we calculate ∇ G ( θ ⋆ )

Original: ⊤ ¯ Σ ∇ G ( θ ⋆ ) .
Revised: ⊤ ¯Σ ∇ G ( θ ⋆ ) .

Original: For notation simplicity, we denote ∇ 2 L = ∇ 2 L ( θ ⋆ , λ ⋆ ) , ∇ G = ∇ G ( θ ⋆ ) and H = ( ∇ 2 L ) − 1 ∇ G [ ∇ G ⊤ ( ∇ 2 L ) − 1 ∇ G ] − 1 .
Revised: For notation simplicity, we denote ∇ 2 L = ∇ 2 L ( θ ⋆ , λ ⋆ ) , ∇ G = ∇ G ( θ ⋆ ) and H = ( ∇ 2 L ) − 1 ∇ G [ ∇ G ⊤ ( ∇ 2 L ) − 1 ∇ G ] − 1 .

Original: By block matrix inversion, we have
Revised: By block matrix inversion, we have

Original: [Equation]
Revised: [Equation]

Original: Note that ∇ G ⊤ H = 1 and ∇ G ⊤ (cid:8) ( ∇ 2 L ) − 1 − H ∇ G ⊤ ( ∇ 2 L ) − 1 (cid:9) = 0 .
Revised: Note that ∇ G ⊤ H = 1 and ∇ G ⊤ (cid:8) ( ∇ 2 L ) − 1 − H ∇ G ⊤ ( ∇ 2 L ) − 1 (cid:9) = 0 .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: = σ 2 .
Revised: = σ [Equation]

Original: Therefore, we conclude that 
Revised: Therefore, we conclude that √

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: By a similar argument
Revised: By a similar argument

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: which implies
Revised: which implies

Original: [Equation]
Revised: [Equation]

Original: Now
Revised: Now

Original: [Equation]
Revised: [Equation]

Original: we prove the case when α n = o ( 1 √ n ) .
Revised: we prove the case when α n = o ( 1 √ n ) .

Original: For this case note that the equality constraint for the population problem can be rewritten as two inequality constraints:
Revised: For this case note that the equality constraint for the population problem can be rewritten as two inequality constraints:

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: to

Original: [Equation]
Revised: [Equation]

Original: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .
Revised: where the expectation is with respect to the underlying distribution of tuple ( X, A, Y ) .

Original: The corresponding empirical problem given the true sensitive attribute is [Equation] P n ) :
Revised: The corresponding empirical problem given the true sensitive attribute is ( P n ) :

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original:    .
Revised:    .

Original: The corresponding empirical problem given the proxy sensitive attribute is
Revised: The corresponding empirical problem given the proxy sensitive attribute is

Original: [Equation]
Revised: [Equation]

Original: n ) :
Revised: n ) :

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  

Original: [Equation]
Revised: [Equation]

Original:   .
Revised:   .

Original: We denote
Revised: We denote

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Consider the Lagrangian functions
Revised: Consider the Lagrangian functions

Original: L ( θ, λ ) =
Revised: L ( θ, λ ) =

Original: F ( θ ) + λ 1 G 1 ( θ ) + λ 2 G 2 ( θ ) and
Revised: F ( θ ) + λ 1 G 1 ( θ ) + λ 2 G 2 ( θ ) and

Original: [Equation]
Revised: [Equation]

Original: of the programs ( P 0 ) and ( P n ) respectively.
Revised: of the programs ( P 0 ) and ( P n ) respectively.

Original: Lemma C.2 (A version of Theorem 6.6.2 in Rubinstein & Shapiro (1993)) .
Revised: Lemma C.2 (A version of Theorem 6.6.2 in Rubinstein & Shapiro (1993)) .

Original: Suppose that:
Revised: Suppose that:

Original: (i)
Revised: (i)

Original: The functions F ( θ ) , G 1 ( θ ) and G 2 ( θ ) are twice continuously differentiable.
Revised: The functions F ( θ ) , G 1 ( θ ) and G 2 ( θ ) are twice continuously differentiable.

Original: (ii)
Revised: (ii)

Original: The true program ( P 0 ) has a unique optimal solution θ ⋆ and a unique Lagrange multiplier λ
Revised: The true program ( P 0 ) has a unique optimal solution θ ⋆ and a unique Lagrange multiplier λ

Original: ⋆ with θ ⋆ being an interior point of Θ .
Revised: ⋆ with θ ⋆ being an interior point of Θ .

Original: (iii)
Revised: (iii)

Original: The Hessian matrix ∇ 2 L ( θ ⋆ , λ ⋆ ) is positive definite.
Revised: The Hessian matrix ∇ 2 L ( θ ⋆ , λ ⋆ ) is positive definite.

Original: (iv) The random functions (cid:98) G 1 n ( θ ) and (cid:98) G 2 n ( θ ) , k ∈
Revised: (iv) The random functions (cid:98) G 1 n ( θ ) and (cid:98) G 2 n ( θ ) , k ∈

Original: [ K ] , are Lipschitz continuous in a neighborhood of θ ⋆ and differentiable at θ ⋆ with probability 1.
Revised: [ K ] , are Lipschitz continuous in a neighborhood of θ ⋆ and differentiable at θ ⋆ with probability 1.

Original: (v)
Revised: (v)

Original: [Equation]
Revised: [Equation]

Original: and there is a neighborhood U of θ ⋆ such that
Revised: and there is a neighborhood U of θ ⋆ such that

Original: [Equation]
Revised: [Equation]

Original: Here we define random mappings ∆ 1 n ( θ ) = ∇ (cid:98) F
Revised: Here we define random mappings ∆ 1 n ( θ ) = ∇ (cid:98) F

Original: [Equation]
Revised: [Equation]

Original: θ )
Revised: θ )

Original: [Equation]
Revised: [Equation]

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: θ ) , ∆ 2 n ( θ ) = (cid:98) G n ( θ ) − G ( θ ) , and ∆ 3 n ( θ ) = ∇ (cid:98) G n ( θ )
Revised: θ ) , ∆ 2 n ( θ ) = (cid:98) G n ( θ ) − G ( θ ) , and ∆ 3 n ( θ ) = ∇ (cid:98) G n ( θ )

Original: − ∇ G ( θ ) .
Revised: − ∇ G ( θ ) .

Original: (vi) Random vectors √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , (cid:98) G 1 n ( θ ⋆ )) , (cid:98) G 2 n ( θ ⋆ )) converge in distribution to Y
Revised: (vi) Random vectors √ n ( ∇ (cid:98) L n ( θ ⋆ , λ ⋆ ) , (cid:98) G 1 n ( θ ⋆ )) , (cid:98) G 2 n ( θ ⋆ )) converge in distribution to Y

Original: =
Revised: =

Original: ( Y 1 , Y 2 , Y
Revised: ( Y 1 , Y 2 , Y

Original: as
Revised: as

Original: n →
Revised: n →

Original: ∞ , where
Revised: ∞ , where

Original: Y 1 is
Revised: Y 1 is

Original: a
Revised: a

Original: random vector and Y 2 and Y 3 are random variables
Revised: random vector and Y 2 and Y 3 are random variables

Original: Let (cid:98) θ n be an optimal solution of ( P n ) converging in probability as n → ∞ to θ ⋆ .
Revised: Let (cid:98) θ n be an optimal solution of ( P n ) converging in probability as n → ∞ to θ ⋆ .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where ¯ x = ¯ x ( Y ) is the optimal solution to the quadratic programming problem minimize
Revised: where ¯ x = ¯ x ( Y ) is the optimal solution to the quadratic programming problem minimize

Original: [Equation]
Revised: [Equation]

Original: Note that
Revised: Note that

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Therefore the last quadratic programming problem with two inequality constraints reduces to the quadratic programming problem with single equality constraint when α n ≡ 0 .
Revised: Therefore the last quadratic programming problem with two inequality constraints reduces to the quadratic programming problem with single equality constraint when α n ≡ 0 .

Original: The limiting distributional results thus persist as we proved for the α n ≡ 0 case.
Revised: The limiting distributional results thus persist as we proved for the α n ≡ 0 case.

Original: Lastly
Revised: Lastly

Original: [Equation]
Revised: [Equation]

Original: we calculate the asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n .
Revised: we calculate the asymptotic relative efficiency (ARE) of (cid:101) θ n to (cid:98) θ n .

Original: Recall that
Revised: Recall that

Original: [Equation]
Revised: [Equation]

Original: and ψ frac =
Revised: and ψ frac =

Original: [Equation]
Revised: [Equation]

Original: γ
Revised: γ

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Therefore, we have
Revised: Therefore, we have

Original: ARE
Revised: ARE

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: ≜
Revised: ≜

Original: [Equation]
Revised: [Equation]

Original: Hence we complete the proof of Theorem 3.1.
Revised: Hence we complete the proof of Theorem 3.1.

Original: □
Revised: □

Original: D M ULTIPLE DEMOGRAPHIC GROUPS
Revised: D M ULTIPLE DEMOGRAPHIC GROUPS

Original: We provide further discussion to supplement Section 3.1.
Revised: We provide further discussion to supplement Section 3.1.

Original: Note that the fairness notion (3.6) uses group 0 as a reference group.
Revised: Note that the fairness notion (3.5) uses group 0 as a reference group.

Original: One can also define a fairness notion by
Revised: One can also define a fairness notion by

Original: [Equation]
Revised: [Equation]

Original: which is symmetric in group indices.
Revised: which is symmetric in group indices.

Original: Due to the equivalence of (D.1) and (3.6), we opt to use (3.6) for a comparison with two-group theory.
Revised: Due to the equivalence of (D.1) and (3.5), we opt to use (3.5) for a comparison with two-group theory.

Original: Theorem 3.2 is a direct extension of Theorem 3.1 and follows the same proof procedure as of Theorem 3.1.
Revised: Theorem 3.2 is a direct extension of Theorem 3.1 and follows the same proof procedure as of Theorem 3.1.

Original: Moreover, let h ≡ 1 , the linear-fractional fairness (3.6) degenerates into linear fairness:
Revised: Moreover, let h ≡ 1 , the linear-fractional fairness (3.5) degenerates into linear fairness:

Original: [Equation]
Revised: [Equation]

Original: By Theorem 3.2, we immediately have the following corollary.
Revised: By Theorem 3.2, we immediately have the following corollary.

Original: Theorem D.1 (Privacy cost in linear fairness (D.2)-aware learning) .
Revised: Theorem D.1 (Privacy cost in linear fairness (D.2)-aware learning) .

Original: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then
Revised: Under the standing assumptions, let estimators (cid:98) θ n and (cid:101) θ n be consistent for θ ⋆ , then

Original: [Equation]
Revised: [Equation]

Original: n ) − (cid:8)(cid:8)(cid:8)
Revised: n ) − (cid:8)(cid:8)(cid:8)

Original: [Equation]
Revised: [Equation]

Original: θ ⋆ ) } d → N
Revised: θ ⋆ ) } d → N

Original: [Equation]
Revised: [Equation]

Original: 0 , Σ) and √ n { c ( (cid:101) θ n ) − (cid:8)(cid:8)(cid:8) c
Revised: 0 , Σ) and √ n { c ( (cid:101) θ n ) − (cid:8)(cid:8)(cid:8) c

Original: ( θ ⋆ ) } d → N ( 0 , Ψ − 1 lin (cid:101) ΣΨ −⊤ lin ) , where
Revised: ( θ ⋆ ) } d → N ( 0 , Ψ − 1 lin (cid:101) ΣΨ −⊤ lin ) , where

Original: [Equation]
Revised: [Equation]

Original: for k, l ∈ [ K ] .
Revised: for k, l ∈ [ K ] .

Original: E M ISSING SENSITIVE ATTRIBUTES
Revised: E M ISSING SENSITIVE ATTRIBUTES

Original: Under the missingness mechanism (3.9), the probability of observing a complete sample from group a is
Revised: Under the missingness mechanism (3.8), the probability of observing a complete sample from group a is

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original:  =
Revised: ) =

Original: [Equation]
Revised: [Equation]

Original: π a for a ∈ { 0 , 1 } .
Revised: π a for a ∈ { 0 , 1 } .

Original: By the intermediate conclusion of Theorem 3.1, we have
Revised: By the intermediate conclusion of Theorem 3.1, we have

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Comparing the two asymptotic variances
Revised: Comparing the two asymptotic variances

Original: [Equation]
Revised: [Equation]

Original: we conclude that
Revised: we conclude that

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]