Original: When Fairness Meets Privacy: Fair Classification with Semi-Private Sensitive Attributes
Revised: When Fairness Meets Privacy: Fair Classification with Semi-Private Sensitive Attributes

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Machine learning models have demonstrated promising performances in manyareas.
Revised: Machine learning models have demonstrated promising performances in many areas.

Original: However, the concerns that they can be biased against specific groupshinder their adoption in high-stake applications.
Revised: However, the concerns that they can be biased against specific groups hinder their adoption in high-stake applications.

Original: Thus, it is essential to ensurefairness in machine learning models.
Revised: Thus, it is essential to ensure fairness in machine learning models.

Original: Most of the previous efforts require access tosensitive attributes for mitigating bias.
Revised: Most of the previous efforts require access to sensitive attributes for mitigating bias.

Original: Nevertheless, it is often infeasible to obtaina large scale of data with sensitive attributes due to people’s increasing awarenessof privacy and the legal compliance.
Revised: Nevertheless, it is often infeasible to obtain a large scale of data with sensitive attributes due to people’s increasing awareness of privacy and the legal compliance.

Original: Therefore, an important research questionis how to make fair predictions under privacy .
Revised: Therefore, an important research question is how to make fair predictions under privacy .

Original: In this paper, we study a novelproblem of fair classification in a semi-private setting, where most of the sensitiveattributes are private and only a small amount of clean ones are available.
Revised: In this paper, we study a novel problem of fair classification in a semi-private setting, where most of the sensitive attributes are private and only a small amount of clean ones are available.

Original: To thisend, we propose a novel framework F AIR SP that can first learn to correct the noisysensitive attributes under the privacy guarantee by exploiting the limited cleanones.
Revised: To this end, we propose a novel framework F AIR SP that can first learn to correct the noisy sensitive attributes under the privacy guarantee by exploiting the limited clean ones.

Original: Then, it jointly models the corrected and clean data in an adversarial way fordebiasing and prediction.
Revised: Then, it jointly models the corrected and clean data in an adversarial way for debiasing and prediction.

Original: Theoretical analysis shows that the proposed model canensure fairness when most sensitive attributes are private.
Revised: Theoretical analysis shows that the proposed model can ensure fairness when most sensitive attributes are private.

Original: Extensive experimentalresults in real-world datasets demonstrate the effectiveness of the proposed modelfor making fair predictions under privacy and maintaining high accuracy.
Revised: Extensive experimental results in real-world datasets demonstrate the effectiveness of the proposed model for making fair predictions under privacy and maintaining high accuracy.

Original:  Introduction
Revised:  Introduction

Original: Machine learning has shown promising performances in various high-stake applications such asdisease identification [40], crime prediction [39] and loan application filtering [21].
Revised: Machine learning has shown promising performances in various high-stake applications such as disease identification [40], crime prediction [39] and loan application filtering [21].

Original: However, in these applications, an emerging concern is that the prediction derived from machine learning modelscan often be biased and unfair to specific (and often marginalized) groups.
Revised: However, in these applications, an emerging concern is that the prediction derived from machine learning models can often be biased and unfair to specific (and often marginalized) groups.

Original: For example, racial biasin medical analysis can lead to disparate treatments [40].
Revised: For example, racial bias in medical analysis can lead to disparate treatments [40].

Original: In addition, a recent Forbes report showsthat machine learning bias has caused 80% of the black mortgage applicants to be denied 1 .
Revised: In addition, a recent Forbes report shows that machine learning bias has caused 80% of the black mortgage applicants to be denied 1 .

Original: Suchdiscrimination can have detrimental societal effects that weaken the public trust among individuals,groups and the society.
Revised: Such discrimination can have detrimental societal effects that weaken the public trust among individuals, groups and the society.

Original: Therefore, it is critical to ensure fairness in machine learning for social good.
Revised: Therefore, it is critical to ensure fairness in machine learning for social good.

Original: The recent advancements of fair machine learning–aiming to develop effective algorithms to achievefairness and maintain good prediction performance–have attracted increasing attention [36, 27].
Revised: The recent advancements of fair machine learning–aiming to develop effective algorithms to achieve fairness and maintain good prediction performance–have attracted increasing attention [36, 27].

Original: Themajority of existing fair machine learning models require direct access to sensitive attributes (e.g.,race, gender, age) to preprocess the training data, regularize the model training or post-process theprediction results to derive fair predictions [33, 17, 27].
Revised: The majority of existing fair machine learning models require direct access to sensitive attributes (e.g., race, gender, age) to preprocess the training data, regularize the model training or post-process the prediction results to derive fair predictions [33, 17, 27].

Original: However, sensitive attributes are often hard tocollect or properly protected due to people’s increasing awareness of privacy and the legal compliancesuch as Electronic Communications Privacy Act (ECPA) 2 and General Data Protection Regulation(GDPR) 3 that restrict the direct access to sensitive attributes.
Revised: However, sensitive attributes are often hard to collect or properly protected due to people’s increasing awareness of privacy and the legal compliance such as Electronic Communications Privacy Act (ECPA) 2 and General Data Protection Regulation (GDPR) 3 that restrict the direct access to sensitive attributes.

Original: Thus, analysts may only get access to
Revised: Thus, analysts may only get access to

Original: [Figure]
Revised: [Figure]

Original: mostly private sensitive attributes processed by privacy mechanisms like Local Differential Privacy(LDP) [12], which is widely deployed by data analysts and can provide a strong privacy guarantee onthe data by injecting noise [34, 19].
Revised: mostly private sensitive attributes processed by privacy mechanisms like Local Differential Privacy (LDP) [12], which is widely deployed by data analysts and can provide a strong privacy guarantee on the data by injecting noise [34, 19].

Original: In practice, it is often possible to collect a small amount of sensitive attributes due to people’s differentprivacy preferences (e.g., male users may be less sensitive about their age than female users)
Revised: In practice, it is often possible to collect a small amount of sensitive attributes due to people’s different privacy preferences (e.g., male users may be less sensitive about their age than female users)

Original: [24],organizations’ desire to build effective models [7] or other certain incentives [38, 30].
Revised: [24], organizations’ desire to build effective models [7] or other certain incentives [38, 30].

Original: Therefore, asshown in Figure 1, we propose to study a novel problem of fair classification in a semi-private setting ,where most of the sensitive attributes (e.g., gender) are protected under certain privacy mechanismsand only very limited clean ones are available.
Revised: Therefore, as shown in Figure 1, we propose to study a novel problem of fair classification in a semi-private setting , where most of the sensitive attributes (e.g., gender) are protected under certain privacy mechanisms and only very limited clean ones are available.

Original: However, it is nontrivial to build fair machine learning models with mostly private sensitive attributes.
Revised: However, it is nontrivial to build fair machine learning models with mostly private sensitive attributes.

Original: First, private sensitive attributes are noisy, and directly applying conventional debiasing techniqueson them can lead to sub-optimal performances.
Revised: First, private sensitive attributes are noisy, and directly applying conventional debiasing techniques on them can lead to sub-optimal performances.

Original: Some initial efforts have verified that noise-protectedsensitive attributes may hurt the performance of conventional debiasing models [32, 42].
Revised: Some initial efforts have verified that noise-protected sensitive attributes may hurt the performance of conventional debiasing models [32, 42].

Original: Second,it is unknown whether or not it is helpful to incorporate the limited clean sensitive attributes.
Revised: Second, it is unknown whether or not it is helpful to incorporate the limited clean sensitive attributes.

Original: Suchlimited instances with clean sensitive attributes are inadequate for training a fair classification modeldirectly because the model can easily overfit the small amount of data [14].
Revised: Such limited instances with clean sensitive attributes are inadequate for training a fair classification model directly because the model can easily overfit the small amount of data [14].

Original: It is under explorationto study the impact of combining the very limited data with clean sensitive attributes and the mostprivate data.
Revised: It is under exploration to study the impact of combining the very limited data with clean sensitive attributes and the most private data.

Original: Third, the conventional model design cannot effectively leverage both the most privatesensitive attributes and limited clean ones.
Revised: Third, the conventional model design cannot effectively leverage both the most private sensitive attributes and limited clean ones.

Original: Conventional debiasing models under the semi-privatesetting would treat each instance the same way.
Revised: Conventional debiasing models under the semi-private setting would treat each instance the same way.

Original: Since the clean sensitive attributes are limited, it isimportant to explore how to effectively exploit them.
Revised: Since the clean sensitive attributes are limited, it is important to explore how to effectively exploit them.

Original: Therefore, to address these challenges, we first conduct a preliminary study on the impact of privacy onfairness performance of debiasing and non-debiasing models.
Revised: Therefore, to address these challenges, we first conduct a preliminary study on the impact of privacy on fairness performance of debiasing and non-debiasing models.

Original: We found that the fairness performanceof debiasing models is enhanced with a smaller noise rate on the sensitive attributes, which isdetermined by a constant privacy budget .
Revised: We found that the fairness performance of debiasing models is enhanced with a smaller noise rate on the sensitive attributes, which is determined by a constant privacy budget .

Original: This finding validates that the semi-private setting isbeneficial for mitigating fairness bias.
Revised: This finding validates that the semi-private setting is beneficial for mitigating fairness bias.

Original: Based on the pilot study, we propose a novel framework
Revised: Based on the pilot study, we propose a novel framework

Original: F AIR SP for fair classification with semi-private sensitive attributes.
Revised: F AIR SP for fair classification with semi-private sensitive attributes.

Original: It first learns to correct the noisysensitive attributes under privacy guarantee by exploiting the limited clean ones.
Revised: It first learns to correct the noisy sensitive attributes under privacy guarantee by exploiting the limited clean ones.

Original: Specifically, ourmodel learns a correction matrix for the noisy sensitive attributes by leveraging both the noisy onesand very limited clean ones.
Revised: Specifically, our model learns a correction matrix for the noisy sensitive attributes by leveraging both the noisy ones and very limited clean ones.

Original: Then, it jointly models the corrected and clean data in an adversarialway for debiasing and prediction.
Revised: Then, it jointly models the corrected and clean data in an adversarial way for debiasing and prediction.

Original: In summary, our main contributions are as follows:
Revised: In summary, our main contributions are as follows:

Original: • We empirically study the impact of privacy on fairness performance of debiasing and non-debiasingmodels.
Revised: • We empirically study the impact of privacy on fairness performance of debiasing and non-debiasing models.

Original: • We study a novel and practical problem of fair classification with semi-private sensitive attributes.•
Revised: • We study a novel and practical problem of fair classification with semi-private sensitive attributes.

Original:  We provide a new end-to-end framework F AIR SP which simultaneously derives corrected sensitiveattributes from private ones and learns a fair classifier with adversarial learning on both clean andcorrected data.• We conduct a theoretic analysis demonstrating that fairness can be achieved with mostly privatesensitive attributes.•
Revised: • We provide a new end-to-end framework F AIR SP which simultaneously derives corrected sensitive attributes from private ones and learns a fair classifier with adversarial learning on both clean and corrected data. • We conduct a theoretic analysis demonstrating that fairness can be achieved with mostly private sensitive attributes.

Original:  We perform extensive experiments on real-world datasets to validate the effectiveness of theproposed learning to correct method for fair classification in the semi-private setting.
Revised: • We perform extensive experiments on real-world datasets to validate the effectiveness of the proposed learning to correct method for fair classification in the semi-private setting.

Original:  Assessing the Intersection of Privacy and Fairness
Revised:  Assessing the Intersection of Privacy and Fairness

Original: In this section, we first introduce the definition of local differential privacy.
Revised: In this section, we first introduce the definition of local differential privacy.

Original: Then we conduct apreliminary study to assess the impact of privacy on fairness performances.
Revised: Then we conduct a preliminary study to assess the impact of privacy on fairness performances.

Original: 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 PrivacyBudget    ∆ E O ( % ) withdebiasingwithoutdebiasing (a) ADULT 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 PrivacyBudget  ∆ D P ( % ) withdebiasingwithoutdebiasing (b) ADULT 0.5 1 1.5 2 2.5 3 3.5 4 4.5 

Revised: 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 PrivacyBudget    ∆ E O ( % ) withdebiasingwithoutdebiasing (a) ADULT 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 PrivacyBudget  ∆ D P ( % ) withdebiasingwithoutdebiasing (b) ADULT 0.5 1 1.5 2 2.5 3 3.5 4 4.5

Original: PrivacyBudget    ∆ E O ( % ) withdebiasingwithoutdebiasing (c)
Revised: PrivacyBudget    ∆ E O ( % ) withdebiasingwithoutdebiasing (c)

Original: COMPAS 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 PrivacyBudget    ∆ D P ( % ) withdebiasingwithoutdebiasing (
Revised: COMPAS 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 PrivacyBudget    ∆ D P ( % ) withdebiasingwithoutdebiasing (

Original: [Figure]
Revised: [Figure]

Original: ) COMPAS
Revised: ) COMPAS

Original: 2.1 Local Differential Privacy Guarantee
Revised: 2.1 Local Differential Privacy Guarantee

Original: [Figure]

Original: The Local differential privacy (LDP) mechanism provides guarantees by directly injecting noises intodata before aggregation.
Revised: The Local differential privacy (LDP) mechanism provides guarantees by directly injecting noises into data before aggregation.

Original: The formal Local differential privacy definition is as follows:
Revised: The formal Local differential privacy definition is as follows:

Original: Definition 1.
Revised: Definition 1.

Original: Given ϵ > 0, a randomized mechanism M satisfies ϵ -local differential privacy, if for allpossible pairs of users’ private data s i and s j , the following equation holds:∀ [Equation] ) : P [Equation]
Revised: Given ϵ > 0, a randomized mechanism M satisfies ϵ -local differential privacy, if for all possible pairs of users’ private data s i and s j , the following equation holds:

Original: M

Original: [Equation]
Revised: [Equation]

Original: s [Equation] [Equation] ) P ( M ( s j )= a ) ≤ e ϵ where Range( M ) denotes every possible output of M .
Revised: ϵ where Range( M ) denotes every possible output of M .

Original: The parameter ϵ denotes the privacy budget to balance the utility and privacy guarantee of the model.
Revised: The parameter ϵ denotes the privacy budget to balance the utility and privacy guarantee of the model.

Original: A smaller ϵ represents stronger privacy guarantee and weaker utility.
Revised: A smaller ϵ represents stronger privacy guarantee and weaker utility.

Original: In order to ensure the privacy ofsensitive attributes, we obtain the following lemma:
Revised: In order to ensure the privacy of sensitive attributes, we obtain the following lemma:

Original: Lemma 1.
Revised: Lemma 1.

Original: To achieve ϵ -local differential privacy on the binary sensitive attribute, we can randomlyflip the sensitive attributes with a probability of p = 1 exp ( ϵ )+1 .
Revised: To achieve ϵ -local differential privacy on the binary sensitive attribute, we can randomly flip the sensitive attributes with a probability of p = [Equation] [Equation] +

Revised: [Equation]

Original: The detailed proof of the lemma refers to the lemma 3 in [32].
Revised: The detailed proof of the lemma refers to the lemma 3 in [32].

Original: Based on the lemma, we can obtaindifferentially private sensitive attributes by flipping at a certain probability.
Revised: Based on the lemma, we can obtain differentially private sensitive attributes by flipping at a certain probability.

Original: If the flipping probabilitysatisfies
Revised: If the flipping probability satisfies the condition that p =

Original: the condition that p = 1 exp ( ϵ )+1 , then the sensitive attributes have the ϵ -local differential privacy guarantee.
Revised: [Equation] ( ϵ )+1 , then the sensitive attributes have the ϵ -local differential privacy guarantee

Original: 2.2 The Impact of Privacy on Fairness
Revised: 2.2 The Impact of Privacy on Fairness

Original: In this subsection, we investigate the impact of privacy on fairness.
Revised: In this subsection, we investigate the impact of privacy on fairness.

Original: Here we adopt the notion of
Revised: Here we adopt the notion of

Original: Local Differential Privacy (LDP) for the privacy guarantee.
Revised: Local Differential Privacy (LDP) for the privacy guarantee.

Original: The original LDP mechanism focuses onthe trade-off between privacy and utility.
Revised: The original LDP mechanism focuses on the trade-off between privacy and utility.

Original: As for the impact of local differential privacy on fairness,we conduct two groups of preliminary experiments.
Revised: As for the impact of local differential privacy on fairness, we conduct two groups of preliminary experiments.

Original: For the first group of experiments, we study theimpact of LDP on models like vanilla multi-layer
Revised: For the first group of experiments, we study the impact of LDP on models like vanilla multi-layer

Original: per-
Revised: per-

Original: ceptron
Revised: ceptron

Original: (MLP) network without debiasing.
Revised: (MLP) network without debiasing.

Original: For the second group of experiments, we study the impact of LDP on debiasing models such asadversarial debiasing.
Revised: For the second group of experiments, we study the impact of LDP on debiasing models such as adversarial debiasing.

Original: We conduct each group of experiments on two datasets ADULT and COMPAS,which are two typical fairness datasets.
Revised: We conduct each group of experiments on two datasets ADULT and COMPAS, which are two typical fairness datasets.

Original: We set six privacy budgets for each group of experiments as0.5, 1, 1.5, 2, 2.5, 3.
Revised: We set six privacy budgets for each group of experiments as 0.5, 1, 1.5, 2, 2.5, 3.

Original: We follow Lemma 1 to implement the LDP mechanism.
Revised: We follow Lemma 1 to implement the LDP mechanism.

Original: From Figure 2, we havethe following observations:
Revised: From Figure 2, we have the following observations:

Original: • For vanilla models without debiasing, stronger privacy guarantee improves the fairnessperformance.
Revised: • For vanilla models without debiasing, stronger privacy guarantee improves the fairness performance.

Original: We can observe that with lower privacy budget, the vanilla models have betterfairness performance on ∆ EO and ∆ DP .
Revised: We can observe that with lower privacy budget, the vanilla models have better fairness performance on ∆ EO and ∆ DP .

Original: With stronger privacy guarantee, the privacy budgetdecreases and the flipping rate increases, which means there is more noise injected into the sensitiveattributes of the dataset.
Revised: With stronger privacy guarantee, the privacy budget decreases and the flipping rate increases, which means there is more noise injected into the sensitive attributes of the dataset.

Original: With more noise injected, the vanilla models cannot learn the explicit biascontained in the sensitive attributes.•
Revised: With more noise injected, the vanilla models cannot learn the explicit bias contained in the sensitive attributes.

Original:  For debiasing models, stronger privacy guarantee leads to worse fairness performance.
Revised: • For debiasing models, stronger privacy guarantee leads to worse fairness performance.

Original: FromFigure 2, we can see that with lower privacy budget, the debiasing models have a worse fairnessperformance on ∆ EO and ∆ DP on the two datasets.
Revised: From Figure 2, we can see that with lower privacy budget, the debiasing models have a worse fairness performance on ∆ EO and ∆ DP on the two datasets.

Original: This is because the debiasing models need toexplicitly leverage sensitive attributes for mitigating bias.
Revised: This is because the debiasing models need to explicitly leverage sensitive attributes for mitigating bias.

Original: With stronger privacy guarantee, there isa lower privacy budget and more noisy sensitive attributes, which causes that the debiasing modelsineffective in mitigating the implicit bias contained in non-sensitive attributes.
Revised: With stronger privacy guarantee, there is a lower privacy budget and more noisy sensitive attributes, which causes that the debiasing models ineffective in mitigating the implicit bias contained in non-sensitive attributes.

Original: Based on the preliminary experiments , if we want to improve the fairness performance of debiasingmodels under privacy, one method is to reduce the noise in sensitive attributes , which illustratesthe benefit of thesemi-private setting.
Revised: Based on the preliminary experiments, if we want to improve the fairness performance of debiasing models under privacy, one method is to reduce the noise in sensitive attributes , which illustrates the

Revised: benefit

Revised: of the semi-private setting.

Original:  Fair Classification with Semi-Private Sensitive Attributes
Revised:  Fair Classification with Semi-Private Sensitive Attributes

Original: In this section, we mainly introduce our method.
Revised: In this section, we mainly introduce our method.

Original: More details on the setting and motivation ofsemi-private sensitive attributes are in the Appendix A.
Revised: More details on the setting and motivation of semi-private sensitive attributes are in the Appendix A.

Original:  Proposed Model - F AIR SP
Revised:  Proposed Model - F AIR SP

Original: Having defined the problem setting for fair classification in the presence of a small set of sampleswith the clean sensitive attribute and a large set of private ones, we now propose our approach toleverage them jointly to learn an end-to-end model.
Revised: Having defined the problem setting for fair classification in the presence of a small set of samples with the clean sensitive attribute and a large set of private ones, we now propose our approach to leverage them jointly to learn an end-to-end model.

Original: Next, we present the details of the proposedframework for fair classification with semi-private sensitive attributes.
Revised: Next, we present the details of the proposed framework for fair classification with semi-private sensitive attributes.

Original: 4.1 Semi-Private Adversarial Debiasing
Revised: 4.1 Semi-Private Adversarial Debiasing

Original: In our semi-private scenario, we have two distinct types of sensitive attributes: clean and private.
Revised: In our semi-private scenario, we have two distinct types of sensitive attributes: clean and private.

Original: Ourobjective is to build a framework that leverages the information from both clean and private samplesand learn an underlying common representation that can induce the fair classification.
Revised: Our objective is to build a framework that leverages the information from both clean and private samples and learn an underlying common representation that can induce the fair classification.

Original: Recently,adversarial debiasing has been proven to be effective in alleviating the bias of representations.
Revised: Recently, adversarial debiasing has been proven to be effective in alleviating the bias of representations.

Original: Inadversarial debiasing, an adversary is used to predict sensitive attributes from the representationsof the classifier; the classifier is trained to learn representations that make the adversary unable topredict the sensitive attributes while keeping high accuracy in the classification.
Revised: In adversarial debiasing, an adversary is used to predict sensitive attributes from the representations of the classifier; the classifier is trained to learn representations that make the adversary unable to predict the sensitive attributes while keeping high accuracy in the classification.

Original: Directly applyingadversarial debiasing is not feasible in our scenario since only limited clean sensitive attributes areavailable and most of the sensitive attributes are private (noisy).
Revised: Directly applying adversarial debiasing is not feasible in our scenario since only limited clean sensitive attributes are available and most of the sensitive attributes are private (noisy).

Original: To this end, we propose to utilize ashared encoder layer to learn the embedding vector, which is fed into separate layers for debiasing.
Revised: To this end, we propose to utilize a shared encoder layer to learn the embedding vector, which is fed into separate layers for debiasing.

Original: Specifically, we first propose a label predictor to minimize the prediction error of the labels with thefollowing objective function: min θ h ,θ Y L Y =
Revised: Specifically, we first propose a label predictor to minimize the prediction error of the labels with the following objective function: min θ h ,θ Y L Y =

Original: E ( X,Y ) ∈D ℓ ( Y, f θ Y ( h ( X ))) , where f θ Y is to predictthe labels, h ( · ) is an embedding layer to encode the features into the latent representation space,and ℓ is a cross entropy loss.
Revised: E ( X,Y ) ∈D ℓ ( Y, f θ Y ( h ( X ))) , where f θ Y is to predict the labels, h ( · ) is an embedding layer to encode the features into the latent representation space, and ℓ is a cross entropy loss.

Original: In addition, to learn fair representations and make fair predictions, weincorporate two adversaries f c and f p to predict the clean and private sensitive attributes, respectively.h ( · ) tries to learn the representation that can fool the adversaries.
Revised: In addition, to learn fair representations and make fair predictions, we incorporate two adversaries f c and f p to predict the clean and private sensitive attributes, respectively. h ( · ) tries to learn the representation that can fool the adversaries.

Original: f θ c and f θ p are jointly optimizedwith the following objective function: min θ h max θ c ,θ p L a =
Revised: f θ c and f θ p are jointly optimized with the following objective function: min θ h max θ c ,θ p L a =

Original: L c + α L p , where α is a hyper-parameterthat controls the relative importance of the loss functions computed over the data of clean and privatesensitive attributes, and L c and L p are defined as follows:[Equation]
Revised: L c + α L p , where α is a hyper-parameter that controls the relative importance of the loss functions computed over the data of clean and private sensitive attributes, and L c and L p are defined as follows:

Original: [Equation]
Revised: [Equation]

Original: where θ c and θ p are the parameters for the adversaries predicting the clean and private sensitive attributes.
Revised: where θ c and θ p are the parameters for the adversaries predicting the clean and private sensitive attributes.

Original: Finally, the overall objective function of adversarial debiasing for fair classification is a minmax function, where β controls the importance of the sensitive attribute predictors:
Revised: Finally, the overall objective function of adversarial debiasing for fair classification is a minmax function, where β controls the importance of the sensitive attribute predictors:

Original: [Equation]
Revised: [Equation]

Original: 4.2 Private Sensitive Attribute Correction
Revised: 4.2 Private Sensitive Attribute Correction

Original: Since private sensitive attributes are noisy, directly applying adversarial debiasing on them may leadto sub-optimal results.
Revised: Since private sensitive attributes are noisy, directly applying adversarial debiasing on them may lead to sub-optimal results.

Original: An intuitive approach is to consider learning to correct these noisy sensitiveattributes before feeding them into the above model (i.e., L adv ).
Revised: An intuitive approach is to consider learning to correct these noisy sensitive attributes before feeding them into the above model (i.e., L adv ).

Original: Inspired by the idea of learninga Corruption Matrix under the scenario with severe label noise and trusted data [23], we proposeto leverage the limited clean sensitive attributes to learn a Correction Matrix to estimate the truesensitive attributes from the private ones without assuming the underlying noise distribution amongthe private sensitive attributes.
Revised: Inspired by the idea of learning a Corruption Matrix under the scenario with severe label noise and trusted data [23], we propose to leverage the limited clean sensitive attributes to learn a Correction Matrix to estimate the true sensitive attributes from the private ones without assuming the underlying noise distribution among the private sensitive attributes.

Original: Specifically, given the dataset D c = {X , A c , Y} with instances containing clean sensitive attributesof l categories, and D p = {X , A p , Y} with instances’ sensitive attributes being private, we aim toestimate a sensitive attribute correction matrix
Revised: Specifically, given the dataset D c = {X , A c , Y} with instances containing clean sensitive attributes of l categories, and D p = {X , A p , Y} with instances’ sensitive attributes being private, we aim to estimate a sensitive attribute correction matrix

Original: C ∈ R l × l to model the sensitive attribute correctionprocess.
Revised: C ∈ R l × l to model the sensitive attribute correction process.

Original: We first train a sensitive attribute predictor g on the private data D p as follows:
Revised: We first train a sensitive attribute predictor g on the private data D p as follows:

Original: g ( X )
Revised: g ( X )

Original: =
Revised: =

Original: ˆ p ( A p | X ) .
Revised: ˆ p ( A p | X ) .

Original: Let X m be the subset of X with sensitive attribute A c = m .
Revised: Let X m be the subset of X with sensitive attribute A c = m .

Original: Based on the Lemma 1, the
Revised: Based on the Lemma 1, the

Original: [Figure]
Revised: [Figure]

Original: flipping probability in LDP is a constant determined by ϵ .
Revised: flipping probability in LDP is a constant determined by ϵ .

Original: Thus, the noise distribution is independentfrom the sensitive attribute value.
Revised: Thus, the noise distribution is independent from the sensitive attribute value.

Original: Then we can infer that A p is conditional independent from A cgiven X , i.e., p ( A p | A c , X ) =
Revised: Then we can infer that A p is conditional independent from A c given X , i.e., p ( A p | A c , X ) =

Original: p ( A p | X ) .
Revised: p ( A p | X ) .

Original: Thus, we can estimate the correction matrix C , where eachelement C mr (the transition probability from A c = m to A p = r ) is calculated as follows:
Revised: Thus, we can estimate the correction matrix C , where each element C mr (the transition probability from A c = m to A p = r ) is calculated as follows:

Original: [Equation]
Revised: [Equation]

Original: With the estimated C , then we can train a new prediction model g ′
Revised: With the estimated C , then we can train a new prediction model g ′

Original: ( X ) =
Revised: ( X ) =

Original: ˜ p
Revised: ˜ p

Original: ( A c | X ) solving thefollowing optimization problem:
Revised: ( A c | X ) solving the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: (4) where ℓ is a differentiable loss function to measure the prediction error, such as cross-entropy loss.
Revised: (4) where ℓ is a differentiable loss function to measure the prediction error, such as cross-entropy loss.

Original: 4. F AIR SP: Integrating Corrected Sensitive Attributes into Adversarial Debiasing
Revised: 4. F AIR SP: Integrating Corrected Sensitive Attributes into Adversarial Debiasing

Original: We aim to leverage the corrected sensitive attributes and integrate them to the adversarial debiasingalgorithm for fair classification.
Revised: We aim to leverage the corrected sensitive attributes and integrate them to the adversarial debiasing algorithm for fair classification.

Original: Specifically, using the sensitive attribute correction network in Eqn. 4,we learn a sensitive attribute correction function g ′ ( X ) that rectifies the private sensitive attributesfor each instance X ∈ D p .
Revised: Specifically, using the sensitive attribute correction network in Eqn. 4, we learn a sensitive attribute correction function g ′ ( X ) that rectifies the private sensitive attributes for each instance X ∈ D p .

Original: We now obtain a set of instances with corrected sensitive attributes
Revised: We now obtain a set of instances with corrected sensitive attributes

Original: D ′ p
Revised: D ′ p

Original: = { X, A ′ p , Y } , ∀ X ∈ D p and A ′ p =
Revised: = { X, A ′ p , Y } , ∀ X ∈ D p and A ′ p =

Original: g ′ ( X ) .
Revised: g ′ ( X ) .

Original: Note that the sensitive attribute correction networkreduces the noise but the rectified sensitive attributes could still be erroneous.
Revised: Note that the sensitive attribute correction network reduces the noise but the rectified sensitive attributes could still be erroneous.

Original: Essentially, we firstfeed the instances with private sensitive attributes D p into the sensitive attributes correction networkto obtain the rectified instances D ′ p .
Revised: Essentially, we first feed the instances with private sensitive attributes D p into the sensitive attributes correction network to obtain the rectified instances D ′ p .

Original: These rectified instances are used as the input for the adversarialdebiasing of private sensitive attributes (i.e., Eqn. 2).
Revised: These rectified instances are used as the input for the adversarial debiasing of private sensitive attributes (i.e., Eqn. 2).

Original: Formally, the objective function of the adversaryf θ p is as defined as follows:
Revised: Formally, the objective function of the adversary f θ p is as defined as follows:

Original: [Equation]
Revised: [Equation]

Original: The overall objective function of our final model F AIR SP is:
Revised: The overall objective function of our final model F AIR SP is:

Original: [Equation]
Revised: [Equation]

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct experiments to evaluate the performance of our method.
Revised: In this section, we conduct experiments to evaluate the performance of our method.

Original: More details onthe experiment analysis are in the Appendix C.
Revised: More details on the experiment analysis are in the Appendix C.

Original: [Table]
Revised: [Table]

Original: We compare F AIR SP with various baselines on three benchmark datasets for fair classification.
Revised: We compare F AIR SP with various baselines on three benchmark datasets for fair classification.

Original: Foreach experiment, we select five random seeds to partition the original dataset into a training set anda test set.
Revised: For each experiment, we select five random seeds to partition the original dataset into a training set and a test set.

Original: We randomly select 80% of the training set as the private training set and the others asthe non-private part (i.e., clean ratio =20%).
Revised: We randomly select 80% of the training set as the private training set and the others as the non-private part (i.e., clean ratio =20%).

Original: For the private training set, we set the privacy budget ϵas 0.5, which means the flipping probability on sensitive attributes is around 38% according to the
Revised: For the private training set, we set the privacy budget ϵ as 0.5, which means the flipping probability on sensitive attributes is around 38% according to the

Original: Lemma 1.
Revised: Lemma 1.

Original: The average performance and standard deviation over five times are reported in Table 1.
Revised: The average performance and standard deviation over five times are reported in Table 1.

Original: We can make following observations:
Revised: We can make following observations:

Original: • In general, we observe that F AIR SP can better improve fairness performance of ∆ DP and ∆ EOwithout causing significant drop in prediction performance on three datasets, compared to otherbaselines.
Revised: • In general, we observe that F AIR SP can better improve fairness performance of ∆ DP and ∆ EO without causing significant drop in prediction performance on three datasets, compared to other baselines.

Original: For example, compared with recently proposed state-of-the-art debiasing model RNF,F AIR SP has achieved 42.5% improvement in terms of ∆ EO and 6.0% on ∆ DP on ADULT.
Revised: For example, compared with recently proposed state-of-the-art debiasing model RNF, F AIR SP has achieved 42.5% improvement in terms of ∆ EO and 6.0% on ∆ DP on ADULT.

Original: • We observe that conventional debiasing methods that directly leverage sensitive attributes aregenerally ineffective under the semi-private scenario.
Revised: • We observe that conventional debiasing methods that directly leverage sensitive attributes are generally ineffective under the semi-private scenario.

Original: For example, comparing RNF with RemoveSmodel, we can observe that their fairness performances are similar w.r.t.
Revised: For example, comparing RNF with RemoveS model, we can observe that their fairness performances are similar w.r.t.

Original: ∆ DP and ∆ EO on threedatasets.•
Revised: ∆ DP and ∆ EO on three datasets.

Original:  The proposed private sensitive attribute correction can help better utilize the limited clean sensitiveattributes for improving fairness.
Revised: • The proposed private sensitive attribute correction can help better utilize the limited clean sensitive attributes for improving fairness.

Original: For example, F AIR SP has 32.3% and 3.7% improvement over“Clean+Private” on ADULT by ∆ DP and ∆ EO respectively.
Revised: For example, F AIR SP has 32.3% and 3.7% improvement over “Clean+Private” on ADULT by ∆ DP and ∆ EO respectively.

Original: The reason is that “Clean+Private”directly utilizes the private sensitive attributes; while the corrected sensitive attributes contain lesssevere noise and can better guide the debiasing process.•
Revised: The reason is that “Clean+Private” directly utilizes the private sensitive attributes; while the corrected sensitive attributes contain less severe noise and can better guide the debiasing process.

Original:  We can see that it is important to leverage sensitive attributes to ensure fairness and maintain prediction performance in the semi-private scenario.
Revised: • We can see that it is important to leverage sensitive attributes to ensure fairness and maintain prediction performance in the semi-private scenario.

Original: For example, F AIR SP has significant improvementw.r.t.
Revised: For example, F AIR SP has significant improvement w.r.t.

Original: Accuracy, F1, ∆ DP and ∆ EO than FairRF consistently, which does not leverage the sensitiveattributes directly.• Exploiting the limited clean sensitive attributes and private ones jointly is important for debiasingin the semi-private setting.
Revised: Accuracy, F1, ∆ DP and ∆ EO than FairRF consistently, which does not leverage the sensitive attributes directly. Exploiting the limited clean sensitive attributes and private ones jointly is important for debiasing in the semi-private setting.

Original: We can generally observe that F AIR SP > Clean+Private > Clean ≈Private ≈ RemoveS > Vanilla for debiasing performances.
Revised: We can generally observe that F AIR SP > Clean+Private > Clean ≈ Private ≈ RemoveS > Vanilla for debiasing performances.

Original: First, F AIR SP and “Clean+Private”perform better than the other three baselines shows that leveraging both clean and private sensitiveattributes is necessary.
Revised: First, F AIR SP and “Clean+Private” perform better than the other three baselines shows that leveraging both clean and private sensitive attributes is necessary.

Original: Second, the observation that “Clean”, “Private” and RemoveS performsimilar indicates that only relying on the clean data or noisy data is less effective for debiasing.
Revised: Second, the observation that “Clean”, “Private” and RemoveS perform similar indicates that only relying on the clean data or noisy data is less effective for debiasing.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: In this paper, we study a novel problem of fair classification with semi-private sensitive attributes.
Revised: In this paper we study a novel problem of fair classification with semi-private sensitive attributes.

Original: We develop an end-to-end adversarial debiasing model F AIR SP to jointly learn from a small amount of instances with clean sensitive attributes and a large amount of instances with private ones.
Revised: We develop an end-to-end adversarial debiasing model F AIR SP to jointly learn from a small amount of instances with clean sensitive attributes and a large amount of instances with private ones.

Original: We provide a theoretic analysis to demonstrate that we can learn a fair model prediction under mild assumptionson privacy.
Revised: We provide a theoretic analysis to demonstrate that we can learn a fair model prediction under mild assumptions on privacy.

Original: Extensive experimental results on real-world datasets demonstrate the effectiveness of the proposed framework to achieve better fair classification performance compared to existing approaches.
Revised: Extensive experimental results on real-world datasets demonstrate the effectiveness of the proposed framework to achieve better fair classification performance compared to existing approaches.

Original: For future work, first, we can study a more general semi-private fairness setting in a variety of data types such as text and graphs and analyze the fairness and privacy guarantee on these data.
Revised: For future work, first, we can study a more general semi-private fairness setting in a variety of data types such as text and graphs, and analyze the fairness and privacy guarantee on these data.

Original: Second, we will investigate the fairness under privacy in distributed machine learning algorithms such as federated learning and split learning
Revised: Second, we will investigate the fairness under privacy in distributed machine learning algorithms such as federated learning and split learning.

Original: A More Details on MethodA.1 Semi-Private Sensitive Attributes
Revised: A More Details on Method A.1 Semi-Private Sensitive Attributes

Original: Training fair classification models with only private sensitive attributes is challenging since mostconventional debiasing methods rely on the sensitive information for mitigating fairness bias.
Revised: Training fair classification models with only private sensitive attributes is challenging since most conventional debiasing methods rely on the sensitive information for mitigating fairness bias.

Original: Thepreliminary experiments have demonstrated that privacy noise on sensitive attributes can severely hurtthe fairness performance.
Revised: The preliminary experiments have demonstrated that privacy noise on sensitive attributes can severely hurt the fairness performance.

Original: Therefore, we study a more practical problem where most of the sensitiveattributes are private and only very limited ones are clean because it is often possible to obtain a smallamount of clean sensitive attributes as discussed earlier [24, 7, 38, 30].
Revised: Therefore, we study a more practical problem where most of the sensitive attributes are private and only very limited ones are clean because it is often possible to obtain a small amount of clean sensitive attributes as discussed earlier [24, 7, 38, 30].

Original: To better reflect the reality inour experimental setup, we keep at most 20% of all the samples to be non-private (see Section C.1.1for more details).
Revised: To better reflect the reality in our experimental setup, we keep at most 20% of all the samples to be non-private (see Section C.1. for more details).

Original: Such non-private samples provide the clean sensitive attributes and will not overlapwith those private ones.
Revised: Such non-private samples provide the clean sensitive attributes and will not overlap with those private ones.

Original: A.2 Problem Statement
Revised: A.2 Problem Statement

Original: We first introduce the notations of this paper and then give the formal problem definition.
Revised: We first introduce the notations of this paper and then give the formal problem definition.

Original: Let
Revised: Let

Original: D =
Revised: D =

Original: {X , A , Y} denote the data, where X , A , and Y represent the set of data samples, sensitiveattributes, and corresponding labels.
Revised: {X , A , Y} denote the data, where X , A , and Y represent the set of data samples, sensitive attributes, and corresponding labels.

Original: For the sensitive attributes A , it consists of a small numberof clean ones A c , and a large number of private ones A p , i.e., A =
Revised: For the sensitive attributes A , it consists of a small number of clean ones A c , and a large number of private ones A p , i.e., A =

Original: A c ∪ A p .
Revised: A c ∪ A p .

Original: Usually, the sizeof clean sensitive attributes is much smaller than the size of private sensitive attributes due to theincreasing awareness of privacy of users and service providers.
Revised: Usually, the size of clean sensitive attributes is much smaller than the size of private sensitive attributes due to the increasing awareness of privacy of users and service providers.

Original: The private sensitive attributes A pcan be collected from user-obfuscated input or a third-party privacy-preserving algorithm.
Revised: The private sensitive attributes A p can be collected from user-obfuscated input or a third-party privacy-preserving algorithm.

Original: In thispaper we adopt LDP mechanism [12].
Revised: In this paper we adopt LDP mechanism [12].

Original: Following the existing works of fair classification [5, 35], we evaluate the performance of fairnessusing metrics such as Equal Opportunity and Demographic Parity.
Revised: Following the existing works of fair classification [5, 35], we evaluate the performance of fairness using metrics such as Equal Opportunity and Demographic Parity.

Original: Without loss of generality, weconsider the binary classification.
Revised: Without loss of generality, we consider the binary classification.

Original: Equal Opportunity requires that the probability of positive instanceswith arbitrary sensitive attributes A being assigned to a positive outcome are equal: E ( ˆ Y |
Revised: Equal Opportunity requires that the probability of positive instances with arbitrary sensitive attributes A being assigned to a positive outcome are equal: E ( ˆ Y |

Original: A =a, Y = 1) =
Revised: A = a, Y = 1) =

Original: E ( ˆ Y |
Revised: E ( ˆ Y |

Original: A = b, Y = 1) , where ˆ Y is predicted label.
Revised: A = b, Y = 1) , where ˆ Y is predicted label.

Original: Demographic Parity requires thebehavior of prediction model to be fair on different demographic groups.
Revised: Demographic Parity requires the behavior of prediction model to be fair on different demographic groups.

Original: Concretely, it requires thatthe positive rate across sensitive attributes are equal: E ( ˆ Y |
Revised: Concretely, it requires that the positive rate across sensitive attributes are equal: E ( ˆ Y |

Original: A = a ) =
Revised: A = a ) =

Original: E ( ˆ Y | A = b ) , ∀ a, b .
Revised: E ( ˆ Y | A = b ) , ∀ a, b .

Original: Theproblem of fair classification with semi-private sensitive attributes is formally defined as follows:
Revised: The problem of fair classification with semi-private sensitive attributes is formally defined as follows:

Original: Problem Statement:
Revised: Problem Statement:

Original: Given the training data D with a limited number of clean sensitive attributes and a large amount of private sensitive attributes, learn an effective classifier that generalizes well to unseen instances, while satisfying the fairness criteria such as demographic parity.
Revised: Given the training data D with a limited number of clean sensitive attributes and a large amount of private sensitive attributes, learn an effective classifier that generalizes well to unseen instances, while satisfying the fairness criteria such as demographic parity.

Original: A.3 The Optimization of F AIR SP
Revised: A.3 The Optimization of F AIR SP

Original: Require:
Revised: Require:

Original: Clean data D c = { X, A c , Y } , private data D p = { X, A p , Y } , α , β
Revised: Clean data D c = { X, A c , Y } , private data D p = { X, A p , Y } , α , β

Original: Ensure: Learn a fair classifier f θ Y 1: //
Revised: Ensure: Learn a fair classifier f θ Y 1: //

Original: Correcting private sensitive attributes 
Revised: Correcting private sensitive attributes 2:

Revised: Learn

Revised: classifier

Revised: 3: Initialize correction

Revised: : Estimate

Revised: correction

Original:  We adopt the mini-batch gradient descent with Adadelta [45] optimizer to learn the parameters.
Revised: : Initialize Generate sensitive We adopt the mini-batch gradient descent with Adadelta [45] optimizer to learn the parameters.

Original: Asshown in Algorithm 1, we first train the sensitive attribute correction network to obtain the data with corrected sensitive attributes D ′ p .
Revised: As shown in Algorithm 1, we first train the sensitive attribute correction network to obtain the data with corrected sensitive attributes D ′ p .

Original: To this end, we train a predictor g on the data with private sensitiveattributes D p and estimate the sensitive attribute correction matrix C as in Eqn. 3.
Revised: To this end, we train a predictor g on the data with private sensitive attributes D p and estimate the sensitive attribute correction matrix C as in Eqn. 3.

Original: Thereafter, wetrain a new predictor g ′ with the correction matrix C on the private data, and obtain the data withcorrected sensitive attributes D ′ p .
Revised: Thereafter, we train a new predictor g ′ with the correction matrix C on the private data, and obtain the data with corrected sensitive attributes D ′ p .

Original: Finally, we train the adversarial debiasing model for classificationin both D ′ P and D c .
Revised: Finally, we train the adversarial debiasing model for classification in both D ′ P and D c .

Original: B A Theoretic Analysis on Fairness Guarantee
Revised: B A Theoretic Analysis on Fairness Guarantee

Original: In this section, we perform a theoretic analysis of the fairness guarantee under the proposed F AIR SP.
Revised: In this section, we perform a theoretic analysis of the fairness guarantee under the proposed F AIR SP.

Original: The model essentially integrates two major modules: (1) the private sensitive attribute correction
Revised: The model essentially integrates two major modules: (1) the private sensitive attribute correction

Original: ( L c in Eqn. 4); and (2) the semi-private adversarial debiasing with the corrected sensitive attributes( L c + α L ′ p in Eqn. 6).
Revised: ( L c in Eqn. 4); and (2) the semi-private adversarial debiasing with the corrected sensitive attributes ( L c + α L ′ p in Eqn. 6).

Original: First, to understand the theoretic guarantee for the semi-private adversarial debiasing, we need toconsider the induced noise in the large amount of private sensitive attributes as they are non-negligible.
Revised: First, to understand the theoretic guarantee for the semi-private adversarial debiasing, we need to consider the induced noise in the large amount of private sensitive attributes as they are non-negligible.

Original: We can obtain the condition of the global optimum of the adversaries by Theorem 1.
Revised: We can obtain the condition of the global optimum of the adversaries by Theorem 1.

Original: Theorem 1.
Revised: Theorem 1.

Original: The global optimum of the adversaries in L ( f θ c and f θ p ) can be achieved if and onlyifp ( X | A c = 1) =
Revised: The global optimum of the adversaries in L ( f θ c and f θ p ) can be achieved if and only

Original:  p ( X | A c = 0) and p ( X | A ′ p = 1) = p ( X | A ′ p = 0) .
Revised: if p ( X | A c = 1) = p ( X | A c = 0) and p ( X | A ′ p = 1) = p ( X | A ′ p = 0) .

Original: Proof.
Revised: Proof.

Original: This is because according to the Proposition 1.
Revised: This is because according to the Proposition 1.

Original: in [20], the adversaries can reach to the optimalf ∗ θ c
Revised: in [20], the adversaries can reach to the optimal f ∗ θ c

Original: = p ( X | A c =1) p ( X
Revised: = p ( X | A c =1) p ( X

Original: | A c =1)+ p ( X |
Revised: | A c =1)+ p ( X |

Original: A c =0) and f ∗ θ p = p ( X | A ′ p =1) p ( X |
Revised: A c =0) and f ∗ θ p = p ( X | A ′ p =1) p ( X |

Original: A c =1)+ p ( X | A ′ p =0) .
Revised: A c =1)+ p ( X | A ′ p =0) .

Original: Then the optimal of the min-maxgame can be written as follows:
Revised: Then the optimal of the min-max game can be written as follows:

Original: [Equation]
Revised: [Equation]

Original: Since α is a non-negative hyper-parameter and the Jensen–Shannon divergence (JSD) betweentwo distributions is always non-negative and zero only when they are equal, so if and only ifp ( X | A ′ p = 1) =
Revised: Since α is a non-negative hyper-parameter and the Jensen–Shannon divergence (JSD) between two distributions is always non-negative and zero only when they are equal, so if and only if p ( X | A ′ p = 1) =

Original: p ( X | A ′ p = 0) and p ( X |
Revised: p ( X | A ′ p = 0) and p ( X |

Original: A c = 1) =
Revised: A c = 1) =

Original: p ( X |
Revised: p ( X |

Original: A c = 0) , the above equation will reach tothe minimum.
Revised: A c = 0) , the above equation will reach to the minimum.

Original: Next, we theoretically show in Theorem 2 that under mild conditions, we can satisfy fairness metricssuch as demographic parity if reaching the global optimum.
Revised: Next, we theoretically show in Theorem 2 that under mild conditions, we can satisfy fairness metrics such as demographic parity if reaching the global optimum.

Original: Theorem 2.
Revised: Theorem 2.

Original: Let ˆ Y denotes the predicted labels, if
Revised: Let ˆ Y denotes the predicted labels, if

Original: (1) For all X ∈ D ′ p , the corrected sensitive attributes A ′ p and X are independent conditioned on their ground-truth sensitive attributes ˜
Revised: (1) For all X ∈ D ′ p , the corrected sensitive attributes A ′ p and X are independent conditioned on their ground-truth sensitive attributes ˜

Original: A p , i.e., p ( A ′ p , X | ˜
Revised: A p , i.e., p ( A ′ p , X | ˜

Original: A p ) =
Revised: A p ) =

Original: p ( A ′ p | ˜
Revised: p ( A ′ p | ˜

Original: A p ) p ( X | ˜
Revised: A p ) p ( X | ˜

Original: A p ) ;  The corrected sensitive attributes are not random, i.e., p ( A ′ p = 1 | ˜
Revised: A p ) ; (2) The corrected sensitive attributes are not random, i.e., p ( A ′ p = 1 | ˜

Original: A p = 1) ̸ = p ( A ′ p = 0 | ˜
Revised: A p = 1) ̸ = p ( A ′ p = 0 | ˜

Original: A p = 0))
Revised: A p = 0))

Original: If L reaches the global optimum, the label prediction f θ Y will achieve demographic parity, i.e.,p ( ˆ Y | ˜
Revised: If L reaches the global optimum, the label prediction f θ Y will achieve demographic parity, i.e., p ( ˆ Y | ˜

Original: A p = 0)
Revised: A p = 0)

Original: =
Revised: =

Original: p ( ˆ Y | ˜
Revised: p ( ˆ Y | ˜

Original: A p = 1) and p ( ˆ Y | A c = 0)
Revised: A p = 1) and p ( ˆ Y | A c = 0)

Original: = p ( ˆ Y | A c = 1)
Revised: = p ( ˆ Y | A c = 1)

Original: Proof.
Revised: Proof.

Original: We first explain the two assumptions: (1) since we use the objective function in L c to derivethe corrected sensitive attributes A ′ p , and learn the latent presentation through h ( · ) , between whichthey do not share any parameters; it generally holds that A ′ p is independent with the representationof X , i.e., p ( A ′ p = 1 | ˜
Revised: We first explain the two assumptions: (1) since we use the objective function in L c to derive the corrected sensitive attributes A ′ p , and learn the latent presentation through h ( · ) , between which they do not share any parameters; it generally holds that A ′ p is independent with the representation of X , i.e., p ( A ′ p = 1 | ˜

Original: A p = 1) ̸
Revised: A p = 1) ̸

Original: = p ( A ′ p = 0 | ˜
Revised: = p ( A ′ p = 0 | ˜

Original: A p = 0)) ; (2) Since we are using adversarial learning to learn an effective estimator f θ p for sensitive attributes, it is reasonable to assume that it does notproduce random prediction results.
Revised: A p = 0)) ; (2) Since we are using adversarial learning to learn an effective estimator f θ p for sensitive attributes, it is reasonable to assume that it does not produce random prediction results.

Original: We then prove Theorem 2 as follows: since p ( A ′ p = 1 | ˜
Revised: We then prove Theorem 2 as follows: since p ( A ′ p = 1 | ˜

Original: A p =1) ̸
Revised: A p = 1) ̸

Original: = p ( A ′ p = 0 | ˜
Revised: = p ( A ′ p = 0 | ˜

Original: A p = 0)) , we have p ( X | A ′ p , ˜
Revised: A p = 0)) , we have p ( X | A ′ p , ˜

Original: A p ) =
Revised: A p ) =

Original: p ( X | ˜
Revised: p ( X | ˜

Original: A p ) .
Revised: A p ) .

Original: When the algorithm converges,we have p ( X | A ′ p = 1) =
Revised: When the algorithm converges, we have p ( X | A ′ p = 1) =

Original: p ( X | A ′ p = 0) , which is equivalent with (cid:80) ˜
Revised: p ( X | A ′ p = 0) , which is equivalent with (cid:80) ˜

Original: A p p ( X, ˜
Revised: A p p ( X, ˜

Original: A p | A ′ p = 1) =(cid:80) ˜
Revised: A p | A ′ p = 1) = (cid:80) ˜

Original: A p p ( X, ˜
Revised: A p p ( X, ˜

Original: A p | A ′ p = 0) .
Revised: A p | A ′ p = 0) .

Original: Therefore, 
Revised: Therefore, (cid:88)

Original: [Equation]
Revised: [Equation]

Original: Based on the above equation, we can get,
Revised: Based on the above equation, we can get,

Original: [Equation]
Revised: [Equation]

Original: Since we already proof that p ( X | A c = 1) =
Revised: Since we already proof that p ( X | A c = 1) =

Original: p ( X |
Revised: p ( X |

Original: A c = 0) , and ˆ Y =
Revised: A c = 0) , and ˆ Y =

Original: f θ Y ( X ) , we can getp ( ˆ Y | ˜
Revised: f θ Y ( X ) , we can get p ( ˆ Y | ˜

Original: A p = 1) =
Revised: A p = 1) =

Original: p ( ˆ Y | ˜
Revised: p ( ˆ Y | ˜

Original: A p = 0) and p ( ˆ Y | A c = 1) =
Revised: A p = 0) and p ( ˆ Y | A c = 1) =

Original: p ( ˆ Y | A c = 0) , which is the demographicparity.
Revised: p ( ˆ Y | A c = 0) , which is the demographic parity.

Original: C More on Experiment Analysis
Revised: C More on Experiment Analysis

Original: In this section, we conduct experiments to evaluate the performance of our method.
Revised: In this section, we conduct experiments to evaluate the performance of our method.

Original: In theseexperiments, we try to answer the following research questions:
Revised: In these experiments, we try to answer the following research questions:

Original: • RQ1 : Can F AIR SP obtain fair predictions with mostly private sensitive attributes?• RQ2 : How does the private sensitive attribute correction affect the performance of prediction andfairness?• RQ3 : What is the impact of the amount of data with clean sensitive attributes?• RQ4 : How does the degree of privacy budget impact the fair classification performance?
Revised: • RQ1 : Can F AIR SP obtain fair predictions with mostly private sensitive attributes? • RQ2 : How does the private sensitive attribute correction affect the performance of prediction and fairness? • RQ3 : What is the impact of the amount of data with clean sensitive attributes? • RQ4 : How does the degree of privacy budget impact the fair classification performance?

Original: Note that RQ1 has been answered in Section 5.
Revised: Note that RQ1 has been answered in Section 5.

Original: C.1 Experimental Settings 
Revised: C.1 Experimental Settings C.1.1 Datasets

Original: C.1.1 Datasets

Original: We conduct experiments on three typical datasets for fair classification: COMPAS [26], ADULT [4]and MEPS [11].
Revised: We conduct experiments on three typical datasets for fair classification: COMPAS [26], ADULT [4] and MEPS [11].

Original: • COMPAS 4 : This dataset describes the task of predicting the recidivism of individuals in the UnitedStates.• ADULT 5 : This dataset contains personal yearly records.
Revised: • COMPAS 4 : This dataset describes the task of predicting the recidivism of individuals in the United States. • ADULT 5 : This dataset contains personal yearly records.

Original: The task is to predict whether an individual’s income exceeds 50k.• MEPS 6 : This dataset contains records of The Medical Expenditure Panel Survey (MEPS), which isa set of large-scale medical surveys across the United States.
Revised: The task is to predict whether an individual’s income exceeds 50k. • MEPS 6 : This dataset contains records of The Medical Expenditure Panel Survey (MEPS), which is a set of large-scale medical surveys across the United States.

Original: The task is to predict whether or not aperson would have a “high” utilization.
Revised: The task is to predict whether or not a person would have a “high” utilization.

Original: We report results on the test set and all experiments are repeated for 5 times and the average resultand standard deviation are reported in Table 1.
Revised: We report results on the test set and all experiments are repeated for 5 times and the average result and standard deviation are reported in Table 1.

Original: For each run, the dataset is randomly divided intotrain set and test set.
Revised: For each run, the dataset is randomly divided into train set and test set.

Original: The set of random seeds for five runs is {5, 7, 11, 19, 29}.
Revised: The set of random seeds for five runs is {5, 7, 11, 19, 29}.

Original: We compare differentmethods and techniques at different values of the clean ratio defined as:clean ratio =
Revised: We compare different methods and techniques at different values of the clean ratio defined as:

Original: #samples w/ clean SA #samples w/ clean SA + #samples w/ private SA (10) https://github.com/propublica/compas-analysishttps://archive.ics.uci.edu/ml/machine-learning-databases/adult/https://meps.ahrq.gov/mepsweb/
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: C.1.2 Baselines
Revised: C.1.2 Baselines

Original: We compare the proposed F AIR SP with four types of baselines.
Revised: We compare the proposed F AIR SP with four types of baselines.

Original: The first type of baseline is avanilla multi-layer perceptron (MLP) network (1,2).
Revised: The first type of baseline is a vanilla multi-layer perceptron (MLP) network (1,2).

Original: The second type is a state-of-the-art debiasingmodel that explicitly utilizes the sensitive information (3).
Revised: The second type is a state-of-the-art debiasing model that explicitly utilizes the sensitive information (3).

Original: The third type is a debiasing modelthat do not need the sensitive information (4).
Revised: The third type is a debiasing model that do not need the sensitive information (4).

Original: For the fourth type of baselines, we implement aadversarial debiasing model with three kinds of training strategies including “Clean”, “Private” and“Clean+Private” (5,6,7), which are utilized to illustrate the effectiveness of our proposed privatesensitive attribute correction method: (1) Vanilla:
Revised: For the fourth type of baselines, we implement a adversarial debiasing model with three kinds of training strategies including “Clean”, “Private” and “Clean+Private” (5,6,7), which are utilized to illustrate the effectiveness of our proposed private sensitive attribute correction method: (1) Vanilla:

Original: A classifier like MLP without any debiasingmethod.
Revised: A classifier like MLP without any debiasing method.

Original: ; (2) RemoveS: Directly removing the sensitive attributes in the input; (3) RNF [15]: Arecently proposed state-of-the-art debiasing method utilizing mixup for debiasing; (4) FairRF [47]:
Revised: ; (2) RemoveS: Directly removing the sensitive attributes in the input; (3) RNF [15]: A recently proposed state-of-the-art debiasing method utilizing mixup for debiasing; (4) FairRF [47]:

Original: A debiasing method that does not need any sensitive attributes, which can also be applied in ourproposed semi-private scenario; (5) Clean: An adversarial debiasing model on only the instanceswith clean sensitive attributes; (6) Private: An adversarial debiasing model on only the instances withprivate sensitive attributes; (7) C+P: We simply merge both the sets (essentially treating the privatesensitive attributes as the clean ones) and use them together for training an adversarial debiasingmodel.
Revised: A debiasing method that does not need any sensitive attributes, which can also be applied in our proposed semi-private scenario; (5) Clean: An adversarial debiasing model on only the instances with clean sensitive attributes; (6) Private: An adversarial debiasing model on only the instances with private sensitive attributes; (7) C+P: We simply merge both the sets (essentially treating the private sensitive attributes as the clean ones) and use them together for training an adversarial debiasing model.

Original: C.1.3 Evaluation Metrics
Revised: C.1.3 Evaluation Metrics

Original: Following existing work on fair classification, we measure the classification performance with
Revised: Following existing work on fair classification, we measure the classification performance with

Original: Accuracy (Acc.) and F1, and the fairness performance based on DemographicParity and Equal
Revised: Accuracy (Acc.) and F1, and the fairness performance based on Demographic

Original:  Opportunity [35].
Revised: Parity and Equal Opportunity [35].

Original: • Demographic Parity: it requires each demographic group has the same chance for a positiveoutcome: E ( ˆ Y |
Revised: • Demographic Parity: it requires each demographic group has the same chance for a positive outcome: E ( ˆ Y |

Original: A = 1) =
Revised: A = 1) =

Original: E ( ˆ Y |
Revised: E ( ˆ Y |

Original: A = 0) .
Revised: A = 0) .

Original: We report the difference of each group’s demographicparity:
Revised: We report the difference of each group’s demographic parity:

Original: [Equation]
Revised: [Equation]

Original: • Equal Opportunity: it requires the true positive rate of different groups is equal: E ( ˆ Y |
Revised: • Equal Opportunity: it requires the true positive rate of different groups is equal: E ( ˆ Y |

Original: A = 1 , Y =1)
Revised: A = 1 , Y = 1)

Original: =
Revised: =

Original: E ( ˆ Y |
Revised: E ( ˆ Y |

Original: A = 0 , Y = 1) .
Revised: A = 0 , Y = 1) .

Original: We report the difference of each sensitive group’s equal opportunity:
Revised: We report the difference of each sensitive group’s equal opportunity:

Original: [Equation]
Revised: [Equation]

Original: Note that demographic parity and equal opportunity measure the fairness performance in differentways.
Revised: Note that demographic parity and equal opportunity measure the fairness performance in different ways.

Original: The fairness performance is better with smaller values of ∆ EO and ∆ DP .
Revised: The fairness performance is better with smaller values of ∆ EO and ∆ DP .

Original: C.2 Assessing the Impact of Private Sensitive Attribute Correction
Revised: C.2 Assessing the Impact of Private Sensitive Attribute Correction

Original: Now we investigate the impact of private sensitive attribute correction, to answer RQ2 .
Revised: Now we investigate the impact of private sensitive attribute correction, to answer RQ2 .

Original: We keep thesetting of the training set and test set division the same as the main results in Table 1, and show theresults on ADULT dataset as we have similar observations on COMPAS.
Revised: We keep the setting of the training set and test set division the same as the main results in Table 1, and show the results on ADULT dataset as we have similar observations on COMPAS.

Original: We set the privacy budgetϵ as 0.5 and 1 for the training set to derive private sensitive attributes, and the corresponding noisyratio on sensitive attributes are 38% and 27% accordingly.
Revised: We set the privacy budget ϵ as 0.5 and 1 for the training set to derive private sensitive attributes, and the corresponding noisy ratio on sensitive attributes are 38% and 27% accordingly.

Original: The average performance and standarddeviation for five rounds are reported in Table ?? .
Revised: The average performance and standard deviation for five rounds are reported in Table 3.

Original: We can make following observations:
Revised: We can make following observations:

Original: • The proposed correction strategy on private sensitive attributes is effective in improving debiasingperformance consistently regardless of the privacy budget.
Revised: • The proposed correction strategy on private sensitive attributes is effective in improving debiasing performance consistently regardless of the privacy budget.

Original: For example, when the privacy budget ϵis 0.5, F AIR SP has achieved a 23.3% and 6.0% improvement in terms of ∆ EO and ∆ DP , comparingto F AIR SP without private attributes correction.
Revised: For example, when the privacy budget ϵ is 0.5, F AIR SP has achieved a 23.3% and 6.0% improvement in terms of ∆ EO and ∆ DP , comparing to F AIR SP without private attributes correction.

Original: Privacy Budget
Revised: Privacy Budget

Original: Metric
Revised: Metric

Revised: F AIR

Revised: [Table]

Revised: w/

Original: [Table]
Revised: [Table]

Original:  The proposed private sensitive attribute correction method does not cause a significant drop inprediction performance.
Revised: • The proposed private sensitive attribute correction method does not cause a significant drop in prediction performance.

Original: For example, when privacy budget ϵ is 1, F AIR SP has achieved comparableperformance with F AIR SP without correction in terms of both Accuracy and F1 metrics.
Revised: For example, when privacy budget ϵ is 1, F AIR SP has achieved comparable performance with F AIR SP without correction in terms of both Accuracy and F1 metrics.

Original: C.3 Impact of Private Data Ratio
Revised: C.3 Impact of Private Data Ratio

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: FairSPClean

Original: [Figure] [Figure] [Figure] In this subsection, we investigate the impact of different private data ratios, to answer RQ3 .
Revised: In this subsection, we investigate the impact of different private data ratios, to answer RQ3 .

Original: We showthe experimental results on ADULT dataset and we observe similar trends on COMPAS dataset.
Revised: We show the experimental results on ADULT dataset and we observe similar trends on COMPAS dataset.

Original: Weconduct four groups of experiments with different clean data ratios (defined as in Eqn. 10) as in therange of [0.02%, 0.2%, 2%, 20%].
Revised: We conduct four groups of experiments with different clean data ratios (defined as in Eqn. 10) as in the range of [0.02%, 0.2%, 2%, 20%].

Original: Each group of experiments have the same privacy budget ϵ as0.5.
Revised: Each group of experiments have the same privacy budget ϵ as 0.5.

Original: For each clean data ratio, we compare our proposed model F AIR SP with “Clean+Private” anddemonstrate the prediction metric Accuracy, F1 as well as fairness metric ∆ DP and ∆ EO .
Revised: For each clean data ratio, we compare our proposed model F AIR SP with “Clean+Private” and demonstrate the prediction metric Accuracy, F1 as well as fairness metric ∆ DP and ∆ EO .

Original: For eachexperiment, we run five times and report the average result in Figure 4.
Revised: For each experiment, we run five times and report the average result in Figure 4.

Original: From the figure, we can makefollowing observations:
Revised: From the figure, we can make following observations:

Original: • In general, we can observe from Figure 4 (c) (d) that our proposed F AIR SP has a consistentimprovement in fairness performance by a large margin compared with the baseline “Clean+Private”consistently at different clean data ratios.•
Revised: • In general, we can observe from Figure 4 (c) (d) that our proposed F AIR SP has a consistent improvement in fairness performance by a large margin compared with the baseline “Clean+Private” consistently at different clean data ratios.

Original:  It should be noted that our model F AIR SP performs stable on fairness when the clean data ratios aresmaller.
Revised: • It should be noted that our model F AIR SP performs stable on fairness when the clean data ratios are smaller.

Original: The performance of the baseline “Clean+Private” on fairness slightly drops with smallerclean data ratio, which is expected because smaller clean data ratio means the dataset contains lessuseful information about sensitive attributes.
Revised: The performance of the baseline “Clean+Private” on fairness slightly drops with smaller clean data ratio, which is expected because smaller clean data ratio means the dataset contains less useful information about sensitive attributes.

Original: However, our proposed model F AIR SP has stablefairness performance on both ∆ DP and ∆ EO with a smaller clean data ratio.• Even with extreme small clean data ratio such as 0.02%, F AIR SP maintains relatively stablefairness performances.
Revised: However, our proposed model F AIR SP has stable fairness performance on both ∆ DP and ∆ EO with a smaller clean data ratio. • Even with extreme small clean data ratio such as 0.02%, F AIR SP maintains relatively stable fairness performances.

Original: This indicates that the sensitive attributes correction is still effective withvery limited clean data and under strong privacy requirements.
Revised: This indicates that the sensitive attributes correction is still effective with very limited clean data and under strong privacy requirements.

Original: [Table] • From Figure 4 (a) (b), we observe that the proposed F AIR SP demonstrates comparable predictionperformance with the baseline “Clean+Private” regardless of different clean data ratios.
Revised: • From Figure 4 (a) (b), we observe that the proposed F AIR SP demonstrates comparable prediction performance with the baseline “Clean+Private” regardless of different clean data ratios.

Original: Theprediction performance of both models drops slightly with smaller clean data ratio.
Revised: The prediction performance of both models drops slightly with smaller clean data ratio.

Original: This is becausewith smaller clean data ratio, there is a larger portion of data with private sensitive attributes.
Revised: This is because with smaller clean data ratio, there is a larger portion of data with private sensitive attributes.

Original: C.4 Impact of Privacy Budget
Revised: C.4 Impact of Privacy Budget

Original: To answer RQ4 , in this subsection, we investigate the impact of different privacy budgets on thefair classification performances.
Revised: To answer RQ4 , in this subsection, we investigate the impact of different privacy budgets on the fair classification performances.

Original: We report the experimental results on ADULT dataset and similartrends are observed on COMPAS.
Revised: We report the experimental results on ADULT dataset and similar trends are observed on COMPAS.

Original: We conduct six groups of experiments with different privacybudget ϵ as {0.5, 1.0, 1.5, 2.0, 2.5, 3.0}, which means the flipping probability on sensitive attributesis around {38%, 27%, 18%, 12%, 7%, 4%} accordingly.
Revised: We conduct six groups of experiments with different privacy budget ϵ as {0.5, 1.0, 1.5, 2.0, 2.5, 3.0}, which means the flipping probability on sensitive attributes is around {38%, 27%, 18%, 12%, 7%, 4%} accordingly.

Original: The clean data ratio for each group ofexperiment is 20%.
Revised: The clean data ratio for each group of experiment is 20%.

Original: For each Privacy Budget ϵ , we compare our proposed model F AIR SP and baseline“Clean+Private” with performance metrics Accuracy, F1 and fairness metrics ∆ DP and ∆ EO .
Revised: For each Privacy Budget ϵ , we compare our proposed model F AIR SP and baseline “Clean+Private” with performance metrics Accuracy, F1 and fairness metrics ∆ DP and ∆ EO .

Original: From
Revised: From

Original: Figure 5, we have the following observations:
Revised: Figure 5, we have the following observations:

Original: • In general, we can observe from Figure 5 (c) (d) that our proposed model F AIR SP performsconsistently better compared to “Clean+Private”.
Revised: • In general, we can observe from Figure 5 (c) (d) that our proposed model F AIR SP performs consistently better compared to “Clean+Private”.

Original: With a larger privacy budget ϵ  the fairness performances of both models are improved.
Revised: With a larger privacy budget ϵ , the fairness performances of both models are improved.

Original: This is because the dataset has fewer noisy sensitiveattributes under larger ϵ .
Revised: This is because the dataset has fewer noisy sensitive attributes under larger ϵ .

Original: • With a larger privacy budget ϵ , we can observe that the gap between F AIR SP and “Clean+Private”is narrower.
Revised: • With a larger privacy budget ϵ , we can observe that the gap between F AIR SP and “Clean+Private” is narrower.

Original: This may be due to the estimation accuracy of the private sensitive attributes in F AIR SP becomes less effective when the flipping probability on sensitive attributes is small enough, which also indicates that F AIR SP is more effective when the privacy guarantee is strong.
Revised: This may be due to the estimation accuracy of the private sensitive attributes in F AIR SP becomes less effective when the flipping probability on sensitive attributes is small enough, which also indicates that F AIR SP is more effective when the privacy guarantee is strong.

Original: • From Figure 5 (a) and (b) we see that F AIR SP has comparable classification performances on Accuracy and F1 with the baseline “Clean+Private” regardless of privacy budget ϵ .
Revised: • From Figure 5 (a) and (b), we see that F AIR SP has comparable classification performances on Accuracy and F1 with the baseline “Clean+Private” regardless of privacy budget ϵ .

Original: With a larger privacy budget ϵ , the performances of the both models increase because a larger privacy budget ϵindicates fewer noisy sensitive attributes in the training data.
Revised: With a larger privacy budget ϵ , the performances of the both models increase because a larger privacy budget ϵ indicates fewer noisy sensitive attributes in the training data.

Original: C.5 Parameter Sensitivity Analysis
Revised: C.5 Parameter Sensitivity Analysis

Original: We now explore the parameter sensitivity of the two important hyperparameters of our model: αcontrols the impact of the adversarial private sensitive attribute predictor, while β controls theinfluence of the adversary to the debiasing.
Revised: We now explore the parameter sensitivity of the two important hyperparameters of our model: α controls the impact of the adversarial private sensitive attribute predictor, while β controls the influence of the adversary to the debiasing.

Original: We vary α in [0.6, 0.7, 0.8, 0.9, 1.0] and β from [0.6,0.7, 0.8, 0.9, 1.0].
Revised: We vary α in [0.6, 0.7, 0.8, 0.9, 1.0] and β from [0.6, 0.7, 0.8, 0.9, 1.0].

Original: The results are shown in Fig.
Revised: The results are shown in Fig.

Original: From the figure, we can observe that: (1) Theperformances of Accuracy and F1 are relatively consistent in the range, and the performance trendof ∆ DP and ∆ EO are also similar.
Revised: From the figure, we can observe that: (1) The performances of Accuracy and F1 are relatively consistent in the range, and the performance trend of ∆ DP and ∆ EO are also similar.

Original: (2) When β is larger, the prediction performance drops andfairness performance is improved, which illustrates the trade-off between fairness and predictionperformances.
Revised: (2) When β is larger, the prediction performance drops and fairness performance is improved, which illustrates the trade-off between fairness and prediction performances.

Original: (3) Based on the experiments, we can achieve optimal fairness performance andcomparable accuracy performance when selecting both α and β as 1.0.
Revised: (3) Based on the experiments, we can achieve optimal fairness performance and comparable accuracy performance when selecting both α and β as 1.0.

Original: D Related Work
Revised: D Related Work

Original: In this section, we briefly describe the related work on (1) Fairness in machine learning; and (2)
Revised: In this section, we briefly describe the related work on (1) Fairness in machine learning; and (2)

Original: Differential privacy in machine learning.
Revised: Differential privacy in machine learning.

Original: Fairness in Machine Learning
Revised: Fairness in Machine Learning

Original: Recent research on fairness in machine learning has drawn significantattention to develop effective algorithms to achieve fairness and maintain good prediction performance.
Revised: Recent research on fairness in machine learning has drawn significant attention to develop effective algorithms to achieve fairness and maintain good prediction performance.

Original: Existing methods generally focus on individual fairness [28, 10, 8] or group fairness [22, 46].
Revised: Existing methods generally focus on individual fairness [28, 10, 8] or group fairness [22, 46].

Original: Otherniche notions of fairness include subgroup fairness [29] and Max-Min fairness[31].
Revised: Other niche notions of fairness include subgroup fairness [29] and Max-Min fairness[31].

Original: The majority ofexisting debiasing techniques have been applied at different stages of a machine learning model [36]including pre-processing [27], in-processing [2, 6] and post-processing approaches [17].
Revised: The majority of existing debiasing techniques have been applied at different stages of a machine learning model [36] including pre-processing [27], in-processing [2, 6] and post-processing approaches [17].

Original: Suchmachine learning methods generally require the access to sensitive attributes, which is often infeasiblein practice.
Revised: Such machine learning methods generally require the access to sensitive attributes, which is often infeasible in practice.

Original: Very few recent work study fairness with limited or private sensitive attributes available.
Revised: Very few recent work study fairness with limited or private sensitive attributes available.

Original: For example, Zhao et al.
Revised: For example, Zhao et al.

Original: explore and utilize the related features as proxies of the sensitive attributesto improve fairness performances when sensitive attributes are unknown [47].
Revised: explore and utilize the related features as proxies of the sensitive attributes to improve fairness performances when sensitive attributes are unknown [47].

Original: Dai et al.
Revised: Dai et al.

Original: propose toachieve fairness on graph neural networks when the sensitive attributes are limited and private [13].
Revised: propose to achieve fairness on graph neural networks when the sensitive attributes are limited and private [13].

Original: However, these methods may design for a specific type of data (e.g., graphs) or not considering thesensitive attributes being private.
Revised: However, these methods may design for a specific type of data (e.g., graphs) or not considering the sensitive attributes being private.

Original: A CC ( % ) 84.0 85.
Revised: A CC ( % ) 84.0 85.

Original: (a) Accuracy 1.0 0.9 0.
Revised: (a) Accuracy 1.0 0.9 0.

Original: F 1 ( % ) 63.064.065.066.
Revised: F 1 ( % ) 63.064.065.066.

Original: (b) F 1.0 0.9 0.
Revised: (b) F 1.0 0.9 0.

Original: D P ( % ) 7.0 8.
Revised: D P ( % ) 7.0 8.

Original: (c) ∆ DP 1.0 0.9 0.
Revised: (c) ∆ DP 1.0 0.9 0.

Original: E O ( % ) 1.02.03.04.05.06.
Revised: E O ( % ) 1.02.03.04.05.06.

Original: (d) ∆ EO
Revised: (d) ∆ EO

Original: Differential Privacy in Machine Learning.
Revised: Differential Privacy in Machine Learning.

Original: Differential privacy (DP) [16, 18] is a widely adoptedapproach to provide strong privacy guarantee regardless of the adversaries’ prior knowledge [34],which can protect user privacy in various machine learning tasks including supervised learning andunsupervised learning [25, 19, 1, 41].
Revised: Differential privacy (DP) [16, 18] is a widely adopted approach to provide strong privacy guarantee regardless of the adversaries’ prior knowledge [34], which can protect user privacy in various machine learning tasks including supervised learning and unsupervised learning [25, 19, 1, 41].

Original: Recently, local differential privacy (LDP) has been extensivelystudied in the distributed setting such that private data can be locally perturbed without a trustedaggregator [12, 43].
Revised: Recently, local differential privacy (LDP) has been extensively studied in the distributed setting such that private data can be locally perturbed without a trusted aggregator [12, 43].

Original: Due to the inherent connection of privacy and fairness (e.g., protecting ordebiasing on user sensitive attributes), several recent work look into trade-offs and mutual risksbetween privacy and fairness [3, 9].
Revised: Due to the inherent connection of privacy and fairness (e.g., protecting or debiasing on user sensitive attributes), several recent work look into trade-offs and mutual risks between privacy and fairness [3, 9].

Original: Other approaches aim to ensure both DP and fairness whilepreserving good utility [44], or learn fair models with only private data [37, 32, 42].
Revised: Other approaches aim to ensure both DP and fairness while preserving good utility [44], or learn fair models with only private data [37, 32, 42].

Original: However, theseinitial efforts assume that only noisy sensitive attributes are available and there is a specific type ofnoise model between the true sensitive attributes and noisy ones.
Revised: However, these initial efforts assume that only noisy sensitive attributes are available and there is a specific type of noise model between the true sensitive attributes and noisy ones.