Original: Fairness without Demographics through Knowledge Distillation
Revised: Fairness without Demographics through Knowledge Distillation

Original: Abstract
Revised: Abstract

Original: Most of existing work on fairness assumes available demographic information inthe training set.
Revised: Most of existing work on fairness assumes available demographic information in the training set.

Original: In practice, due to legal or privacy concerns, when demographicinformation is not available in the training set, it is crucial to find alternativeobjectives to ensure fairness.
Revised: In practice, due to legal or privacy concerns, when demographic information is not available in the training set, it is crucial to find alternative objectives to ensure fairness.

Original: Existing work on fairness without demographicsfollows Rawlsian Max-Min fairness objective.
Revised: Existing work on fairness without demographics follows Rawlsian Max-Min fairness objectives.

Original: However, such constraints could betoo strict to achieve expected improvement in group fairness, and could lead to agreat decrease in accuracy.
Revised: However, such constraints could be too strict to improve group fairness, and could lead to a great decrease in accuracy.

Original: In light of these limitations, in this paper, we propose tosolve the problem from a new perspective, i.e., through knowledge distillation.
Revised: In light of these limitations, in this paper, we propose to solve the problem from a new perspective, i.e., through knowledge distillation.

Original: Ourmethod uses soft label from an overfitted teacher model as an alternative, and weshow from preliminary experiments that soft labelling is beneficial for improvingfairness.
Revised: Our method uses soft label from an overfitted teacher model as an alternative, and we show from preliminary experiments that soft labelling is beneficial for improving fairness.

Original: We analyze theoretically the fairness of our method, and we show thatour method can be treated as an error-based reweighing.
Revised: We analyze theoretically the fairness of our method, and we show that our method can be treated as an error-based reweighing.

Original: Experimental resultson three datasets show that our method outperforms state-of-the-art alternatives,with notable improvements in group fairness and with relatively small decrease inaccuracy.
Revised: Experimental results on three datasets show that our method outperforms state-of-the-art alternatives, with notable improvements in group fairness and with relatively small decrease in accuracy.

Original:  Introduction
Revised:  Introduction

Original: Machine learning has been widely adopted in real-world scenarios that has great social influence andfairness in automatic decision-making systems has become an arising concern.
Revised: Machine learning has been widely adopted in real-world scenarios that has great social influence, and fairness in automatic decision-making systems has become an arising concern.

Original: It has been shown thatwithout proper intervention, models can be biased against certain demographic groups (Hardt et al. ,2016; Chouldechova, 2017).
Revised: It has been shown that without proper intervention, models can be biased against certain demographic groups (Hardt et al., 2016; Chouldechova, 2017; Ntoutsi et al., 2020).

Original: Due to distributional disparities across different demographic groups,simply removing sensitive attributes during training does not improve fairness.
Revised: Due to distributional disparities across different demographic groups, simply removing sensitive attributes during training does not improve fairness.

Original: Therefore, researchon fair classification has received extensive attention and different methods have been proposed toimprove fairness (Roh et al., 2020; Choi et al., 2020).
Revised: Therefore, research on fair classification has received extensive attention and different methods have been proposed to improve fairness (Roh et al., 2020a; Choi et al., 2020).

Original: These methods usually focus on specifiedfairness notions and most of these works assume access to predefined sensitive information duringtraining.
Revised: These methods usually focus on specified fairness notions, and most of these works assume access to predefined sensitive information during training.

Original: In practice, however, due to legal or regulatory concerns, it is often infeasible to collect sensitiveinformation, which greatly limits the usage of conventional methods on fairness.
Revised: In practice, however, due to legal or regulatory concerns, it is often infeasible to collect sensitive information, which greatly limits the usage of conventional methods on fairness.

Original: Besides, in manyreal-world applications, we expect the decision-making systems to be fair w.r.t.
Revised: Besides, in many real-world applications, we expect the decision-making systems to be fair w.r.t.

Original: multiple sensitiveattributes.
Revised: multiple sensitive attributes.

Original: For example, Consumer Financial Protection Bureau (CFPB) requests creditors to achievefairness, without requesting or collecting information about an applicant’s race, color, religion,national origin, or sex.
Revised: For example, Consumer Financial Protection Bureau (CFPB) requests creditors to achieve fairness, without requesting or collecting information about an applicant’s race, color, religion, national origin, or sex.

Original: As most works on fairness assumes only one predefined sensitive attribute, dueto difference in distributional disparities across different sensitive attributes, improving fairness underone sensitive attribute does not always imply fairness under another sensitive attribute.
Revised: As most works on fairness assumes only one predefined sensitive attribute, due to difference in distributional disparities across different sensitive attributes, improving fairness under one sensitive attribute does not always imply fairness under another sensitive attribute.

Original: Therefore, itis crucial to study the problem of fairness without demographics.
Revised: Therefore, it is crucial to study the problem of fairness without demographics.

Original: Current methods on fairness without demographics can be divided into two categories: fairnesswith proxy sensitive attribute and Max-Min fairness.
Revised: Current methods on fairness without demographics can be divided into two categories: fairness with proxy sensitive attribute and Max-Min fairness.

Original: Methods on fairness with proxy sensitive attribute (Yan et al., 2020; Grari et al., 2021) utilizes surrogate group information, including clusteringinformation and group information through estimation as alternative, and enforces fairness undersurrogate group partition.
Revised: Methods on fairness with proxy sensitive attribute (Yan et al., 2020; Grari et al., 2021) utilizes surrogate group information, including clustering information and group information through estimation as alternative, and enforces fairness under surrogate group partition.

Original: The problem with proxy attributes is that it requires a strong assumptionthat the clustering or surrogate group information is correlated with the sensitive attribute(s) ofconcern, and improving group fairness w.r.t.
Revised: The problem with proxy attributes is that it requires a strong assumption that the clustering or surrogate group information is correlated with the sensitive attribute(s) of concern, and improving group fairness w.r.t.

Original: several sensitive arrtibutes could be infeasible under thecondition of large distributional discrepancy over different sensitive attributes of interest.
Revised: several sensitive arrtibutes could be infeasible under the condition of large distributional discrepancy over different sensitive attributes of interest.

Original: Methods on
Revised: Methods on

Original: Max-Min fairness (Hashimoto et al., 2018;
Revised: Max-Min fairness (Hashimoto et al., 2018;

Original: Lahoti et al., 2020) focuses on improving the worst utilityamongst all sensitive groups.
Revised: Lahoti et al., 2020) focuses on improving the worst utility amongst all sensitive groups.

Original: Ideally, this formulation is in accord with group fairness; however, inpractice, such formulation could be too strict to obtain improvement in group fairness, as improvingworst-group utility does not necessarily imply decrease in inter-group disparities, and since Max-Minfairness tends to require similar utility over all groups, this could lead to a significant decrease inaccuracy.
Revised: Ideally, this formulation is in accord with group fairness; however, in practice, such formulation could be too strict to obtain improvement in group fairness, as improving worst-group utility does not necessarily imply decrease in inter-group disparities, and since Max-Min fairness tends to require similar utility over all groups, this could lead to a significant decrease in accuracy.

Original: Instead, we seek to find alternative ways to quantify disparities without observing sensitiveinformation during training.
Revised: Instead, we seek to find alternative ways to quantify disparities without observing sensitive information during training.

Original: We draw inspiration from debiased learning (Choi et al., 2019;
Revised: We draw inspiration from debiased learning (Choi et al., 2019;

Original: Nam et al., 2020; Liu et al., 2021),where the goal is to unlearn spurious correlations present in training set.
Revised: Nam et al., 2020; Liu et al., 2021), where the goal is to unlearn spurious correlations present in training set.

Original: One common way ofdebiased learning is through reweighing, where samples of worst groups specified by spuriousfeatures are up-weighed so as to mitigate biased prediction.
Revised: One common way of debiased learning is through reweighing, where samples of worst groups specified by spurious features are up-weighed so as to mitigate biased prediction.

Original: For classification tasks, one simpleway to assign weight is to replace the discrete one-hot encoding with continuous labelling that areobtained based on expected properties.
Revised: For classification tasks, one simple way to assign weight is to replace the discrete one-hot encoding with continuous labelling that are obtained based on expected properties.

Original: However, for most classification tasks, continuous labellingis unrealistic in real-world scenarios.
Revised: However, for most classification tasks, continuous labelling is unrealistic in real-world scenarios.

Original: In light of this, we consider obtaining the soft labels throughknowledge distillation, where training labels for student model are replaced with normalized logits ofteacher model, and we expect knowledge from the overfitted teacher model to help fit hard samplesin training set.
Revised: In light of this, we consider obtaining the soft labels through knowledge distillation, where training labels for student model are replaced with normalized logits of teacher model, and we expect knowledge from the overfitted teacher model to help fit hard samples in training set.

Original: In this paper, we propose a knowledge distillation framework for improving fairness without accessingsensitive information.
Revised: In this paper, we propose a knowledge distillation framework for improving fairness without accessing sensitive information.

Original: Compared with previous works on fairness without demographics, our methoddoes not impose surrogate constraints on fairness.
Revised: Compared with previous works on fairness without demographics, our method does not impose surrogate constraints on fairness.

Original: Instead, we try to find a soft labelling for samplesthrough knowledge distillation to help the student model better fit.
Revised: Instead, we try to find a soft labelling for samples through knowledge distillation to help the student model better fit.

Original: During training, we first train ateacher network to be overfitted to training data, and we use the normalized logits of teacher model assoft label to train a student network.
Revised: During training, we first train a teacher network to be overfitted to training data, and we use the normalized logits of teacher model as soft label to train a student network.

Original: In this way, we seek to correct bias in training set via reweighingtraining samples.
Revised: In this way, we seek to correct bias in training set via reweighing training samples.

Original: We summarize our contribution as follows:
Revised: We summarize our contribution as follows:

Original: Fairness without Demographics : We study the effectiveness of knowledge distillation in improving fairness without observing sensitive information.
Revised: Fairness without Demographics : We study the effectiveness of knowledge distillation in improving fairness without observing sensitive information.

Original: We discuss two different mapping functions oflogits by teacher model, and we discuss theoretically the impact of label smoothing and knowledgedistillation on fairness.2.
Revised: We discuss two different mapping functions of logits by teacher model, and we discuss theoretically the impact of label smoothing and knowledge distillation on fairness.

Original: Our method effectively utilizes knowledge from complex teacher model and achieves comparableclassification results than unconstrained baseline student model.3.
Revised: Our method effectively utilizes knowledge from complex teacher model and achieves comparable classification results than unconstrained baseline student model.

Original: Empirical Observation : We validate the effectiveness of label smoothing on new Adult dataset,and we show from experiments that our method achieves remarkable improvement in fairness onthree benchmark datasets under different sensitive attributes
Revised: Empirical Observation : We validate the effectiveness of label smoothing on new Adult dataset, and we show from experiments that our method achieves remarkable improvement in fairness on three benchmark datasets under different sensitive attributes.

Original:  Related Work2.1 Group Fairness in Classification
Revised:  Related Work 2.1 Group Fairness in Classification

Original: Group fairness is a class of definitions to quantify fairness, which measures the disparity of predictedoutcome among the subgroups with certain sensitive attributes.
Revised: Group fairness is a class of definitions to quantify fairness, which measures the disparity of predicted outcome among the subgroups with certain sensitive attributes.

Original: Several works were proposedto mitigate the disparity and ensure the independence of the performance measures between thesubgroups to achieve group fairness.
Revised: Several works were proposed to mitigate the disparity and ensure the independence of the performance measures between the subgroups to achieve group fairness.

Original: Equal opportunity (Hardt et al., 2016) states that true positiverates are the same for the subgroups.
Revised: Equal opportunity (Hardt et al., 2016) states that true positive rates are the same for the subgroups.

Original: Similarly, predictive equality (Chouldechova, 2017) requiresthe equality of false positive rates.
Revised: Similarly, predictive equality (Chouldechova, 2017) requires the equality of false positive rates.

Original: Applying calibration conditions (Kleinberg et al., 2016; Pleisset al., 2017) were proposed so that the probability estimate matches the actual ratio of positivedistribution within the group.
Revised: Applying calibration conditions (Kleinberg et al., 2016; Pleiss et al., 2017) were proposed so that the probability estimate matches the actual ratio of positive distribution within the group.

Original: Diverse approaches are presented to meet particular group fairnessnotions.
Revised: Diverse approaches are presented to meet particular group fairness notions.

Original: Pre-processing method (Chen et al., 2018) suggests improving the skewed sample sizeproblem.
Revised: Preprocessing method (Chen et al., 2018; Jang et al., 2021) suggests improving the skewed sample size problem.

Original: Adversarial learning (Madras et al., 2018) is proposed to learn a fair representation that isindependent of sensitive attributes.
Revised: Adversarial learning (Madras et al., 2018) is proposed to learn a fair representation that is independent of sensitive attributes.

Revised: In-processing methods (Iosifidis and Ntoutsi, 2019; Roh et al., 2020b; Chai and Wang, 2022) aim at regulating the training process with fairness constraints by reweighing training samples or enforcing fairness regularization.

Original: Post-processing the output of the biased model (Kim et al., 2020; Jang et al., 2021) with multiple fairness objectives are introduced as they are more efficient thantraining a model from scratch.
Revised: Post-processing the output of the biased model (Kim et al., 2020; Jang et al., 2022) with multiple fairness objectives are introduced as they are more efficient than training a model from scratch.

Original: However, to ensure fairness, the aforementioned approaches requiresensitive information such as race, gender, etc.
Revised: However, to ensure fairness, the aforementioned approaches require sensitive information such as race, gender, etc.

Original: This restricts their versatility because the sensitiveinformation is not always available owing to regulations and privacy concerns.
Revised: This restricts their versatility because the sensitive information is not always available owing to regulations and privacy concerns.

Revised: Work including (Celis et al., 2021a,b; Mehrotra and Celis, 2021) studies the problem of fair classification under noisy or adversarially perturbed sensitive attributes, which can be considered as a relaxed version of fairness without demographics.

Original: 2.2 Fairness without Demographics
Revised: 2.2 Fairness without Demographics

Original: Due to the practicality and regulatory limitations, methods to achieve fairness without demographicshave emerged.
Revised: Due to the practicality and regulatory limitations, methods to achieve fairness without demographics have emerged.

Original: The methods in this category adhere to Rawlsian Max-Min fairness (Rawls, 2009):minimize the risk of the group with the highest risk.
Revised: The methods in this category adhere to Rawlsian Max-Min fairness (Rawls, 2009): minimize the risk of the group with the highest risk.

Original: Distributionally Robust Optimization (DRO)(Hashimoto et al., 2018) utilizes χ 2 -divergence to find the worst-case distribution that is close tothe empirical distribution and minimizes the risk of such distribution.
Revised: Distributionally Robust Optimization (DRO) (Hashimoto et al., 2018) utilizes χ 2 -divergence to find the worst-case distribution that is close to the empirical distribution and minimizes the risk of such distribution.

Original: ARL (Lahoti et al., 2020)conduct weighted empirical risk minimization with adversary network to maximize the loss with theperspective of computationally-identifiability.
Revised: ARL (Lahoti et al., 2020) conduct weighted empirical risk minimization with adversary network to maximize the loss with the perspective of computationally-identifiability.

Original: Other methods  on fairness without demographics seekto find surrogate group information based on input features.
Revised: Other methods (Zhao et al., 2022) on fairness without demographics seek to find surrogate group information based on input features.

Original: Fair class balancing (Yan et al., 2020)proposes to quantify disparities w.r.t.
Revised: Fair class balancing (Yan et al., 2020) proposes to quantify disparities w.r.t.

Original: sensitive attribute using clustering information and upsampleminor class in each cluster to achieve balance.
Revised: sensitive attribute using clustering information and upsample minor class in each cluster to achieve balance.

Original: 2.3 Debiased learning
Revised: 2.3 Debiased learning

Original: Debiased learning focuses on learning representations that are unbiased w.r.t.
Revised: Debiased learning focuses on learning representations that are unbiased w.r.t.

Original: spurious features.
Revised: spurious features.

Original: Some works on debiased learning focus on learning unbiased representation through adversarialtraining.
Revised: Some works on debiased learning focus on learning unbiased representation through adversarial training.

Original: Choi et al.
Revised: Choi et al.

Original: (2019) propose to augment the standard cross-entropy loss for action classificationwith an adversarial loss for scene types and a human mask confusion loss for background.
Revised: (2019) propose to augment the standard cross-entropy loss for action classification with an adversarial loss for scene types and a human mask confusion loss for background.

Original: Kimet al.
Revised: Kim et al.

Original: (2019) proposes to unlearn the bias information by minimizing mutual information betweenlatent representation and bias prediction with one adversarial network.
Revised: (2019) proposes to unlearn the bias information by minimizing mutual information between latent representation and bias prediction with one adversarial network.

Original: Another approach employs abiased model to explore the samples with high loss.
Revised: Another approach employs a biased model to explore the samples with high loss.

Original: Nam et al.
Revised: Nam et al.

Original: (2020) proposed to apply GCE loss(Zhang and Sabuncu, 2018) to train a bias amplifying classifier and parallelly train a fair classifier byreferencing the biased classifier.
Revised: (2020) proposed to apply GCE loss (Zhang and Sabuncu, 2018) to train a bias amplifying classifier and parallelly train a fair classifier by referencing the biased classifier.

Original: Liu et al.
Revised: Liu et al.

Original: (2021) suggest a simple two-stage training process.
Revised: (2021) suggest a simple two-stage training process.

Original: Aftertraining a classifier with ERM in the first stage, during the second stage, the model emphasizes themisclassified samples in the first stage.
Revised: After training a classifier with ERM in the first stage, during the second stage, the model emphasizes the misclassified samples in the first stage.

Original: However, the objective to minimize the maximum loss couldmake the training unstable and deteriorate the overall performance.
Revised: However, the objective to minimize the maximum loss could make the training unstable and deteriorate the overall performance.

Original: 2.4 Label Augmentation for Classification
Revised: 2.4 Label Augmentation for Classification

Original: Unlike the previous works, we explore a new approach to ensure fairness without demographics,which is to utilize knowledge distillation (Hinton et al., 2015).
Revised: Unlike the previous works, we explore a new approach to ensure fairness without demographics, which is to utilize knowledge distillation (Hinton et al., 2015).

Original: While cross entropy (Rumelhart et al. ,1986) has been a dominant loss function for classification tasks, some works question its shortcomings,such as being sensitive to noisy label (Zhang and Sabuncu, 2018), poor margin (Nar et al., 2018),and vulnerable to adversarial samples (Nar et al., 2019).
Revised: While cross entropy (Rumelhart et al., 1986) has been a dominant loss function for classification tasks, some works question its shortcomings, such as being sensitive to noisy label (Zhang and Sabuncu, 2018), poor margin (Nar et al., 2018), and vulnerable to adversarial samples (Nar et al., 2019).

Original: To overcome this problem, perturbing thelabel distribution empirically verified the improvement.
Revised: To overcome this problem, perturbing the label distribution empirically verified the improvement.

Original: Label smoothing (Müller et al., 2019) andlabel augmentation (Zhang et al., 2017) effectively deal with noisy label and adversarial samples.
Revised: Label smoothing (Müller et al., 2019) and label augmentation (Zhang et al., 2017) effectively deal with noisy label and adversarial samples.

Original: Knowledge distillation (Hinton et al., 2015) was proposed for model compression to effectivelytransfer knowledge from a deep teacher model to a smaller student model.
Revised: Knowledge distillation (Hinton et al., 2015) was proposed for model compression to effectively transfer knowledge from a deep teacher model to a smaller student model.

Original: Surprisingly, by convexcombination of classical cross entropy loss and supervision from the teacher model, the student modelcould obtain a competitive or even a superior performance at inference.
Revised: Surprisingly, by convex combination of classical cross entropy loss and supervision from the teacher model, the student model could obtain a competitive or even a superior performance at inference.

Original: Even though a family ofknowledge distillation methods is proposed, its impact on fairness is not actively studied yet.
Revised: Even though a family of knowledge distillation methods is proposed, its impact on fairness is not actively studied yet.

Original: In thiswork, we study the effectiveness of knowledge distillation on fairness in the absence of sensitiveinformation, and we claim that it noticeably improves fairness by distilling intrinsic informationwhile volatilizing data bias.
Revised: In this work, we study the effectiveness of knowledge distillation on fairness in the absence of sensitive information, and we claim that it noticeably improves fairness by distilling intrinsic information while volatilizing data bias.

Original:  Method 
Revised:  Method 3.1 Background

Original: 3.1 Background

Original: Consider labelled samples ( x, y, a ) ∼ X × Y × A , with y being the ground truth labels and abeing the sensitive attributes, the goal of fair classification is to learn a classifier that satisfies certainfairness criteria w.r.t.
Revised: Consider labelled samples ( x, y, a ) ∼ X × Y × A , with y being the ground truth labels and a being the sensitive attributes, the goal of fair classification is to learn a classifier that satisfies certain fairness criteria w.r.t.

Original: a .
Revised: a .

Original: These criteria can generally be formulated as independence between sensitive information and expected statistical features.
Revised: These criteria can generally be formulated as independence between sensitive information and expected statistical features.

Original: For example, disparate impact (Feldman et al., 2015)requires the predicted label ˆ y to be independent of sensitive attributes: ˆ y ⊥⊥ a , and equalized odds issatisfied when the predictions and sensitive attributes are independent conditional on ground truthlabel: ˆ y ⊥⊥ a | y .
Revised: For example, disparate impact (Feldman et al., 2015) requires the predicted label ˆ y to be independent of sensitive attributes: ˆ y ⊥⊥ a , and equalized odds is satisfied when the predictions and sensitive attributes are independent conditional on ground truth label: ˆ y ⊥⊥ a | y .

Original: 3.2 Preliminary Results
Revised: 3.2 Preliminary Results

Original: Conventional methods on fairness fails when no sensitive information is available in the training set.
Revised: Conventional methods on fairness fails when no sensitive information is available in the training set.

Original: Instead, we seek to find surrogate training objective that is beneficial for improving fairness.
Revised: Instead, we seek to find surrogate training objective that is beneficial for improving fairness.

Original: We startwith preliminary results on new Adult dataset.
Revised: We start with preliminary results on new Adult dataset.

Original: The new Adult dataset (Ding et al., 2021) is a reconstructed version of Adult dataset (Dua and Graff,2017).
Revised: The new Adult dataset (Ding et al., 2021) is a reconstructed version of Adult dataset (Dua and Graff, 2017).

Original: Instead of containing binary labels indicating whether or not an tindividual’s income exceeds50 K , the new Adult dataset provides specific incomes for each individual, and the goal is to predictwhether an individual’s income exceeds certain predefined threshold.
Revised: Instead of containing binary labels indicating whether or not an tindividual’s income exceeds 50 K , the new Adult dataset provides specific incomes for each individual, and the goal is to predict whether an individual’s income exceeds certain predefined threshold.

Original: Research on new Adult datasetshows that by adjusting the income threshold for binary encoding we are able to improve fairness,without introducing extra regularization terms during training (Ding et al., 2021).
Revised: Research on new Adult dataset shows that by adjusting the income threshold for binary encoding we are able to improve fairness, without introducing extra regularization terms during training (Ding et al., 2021).

Original: Intuitively, aproperly adjusted threshold for binary labelling helps reduce hard samples in dataset, which could bebeneficial for the unprivileged subgroup.
Revised: Intuitively, a properly adjusted threshold for binary labelling helps reduce hard samples in dataset, which could be beneficial for the unprivileged subgroup.

Original: Inspired by this, we consider reweighing samples throughlabel smoothing.
Revised: Inspired by this, we consider reweighing samples through label smoothing.

Original: Although labelling bias have been widely studied in fairness literature (Jiang and
Revised: Although labelling bias have been widely studied in fairness literature (Jiang and

Original: Nachum, 2020; Krasanakis et al., 2018), the effect of label smoothing without accessing sensitiveinformation remains unclear.
Revised: Nachum, 2020; Krasanakis et al., 2018), the effect of label smoothing without accessing sensitive information remains unclear.

Original: Does label smoothing also help improve fairness during training?
Revised: Does label smoothing also help improve fairness during training?

Original: We validate this hypothesis on new Adult dataset.
Revised: We validate this hypothesis on new Adult dataset.

Original: Specifically, we repeat experiments three times andreport average results, and in each repetition, we randomly split data into 50% training data and 50%testing data.
Revised: Specifically, we repeat experiments three times and report average results, and in each repetition, we randomly split data into 50% training data and 50% testing data.

Original: Following the traditional setting on the Adult data, we set the binary cutoff threshold ofincome as ξ = 50 K .
Revised: Following the traditional setting on the Adult data, we set the binary cutoff threshold of income as ξ = 50 K .

Original: Let s i be the income of i -th individual, on testing data, we have y i =
Revised: Let s i be the income of i -th individual, on testing data, we have y i =

Original: ( s i > ξ ) ,and we consider four different types of labelling during training:
Revised: ( s i > ξ ) , and we consider four different types of labelling during training:

Original: • Binary : The training labels are determined by y i =
Revised: • Binary : The training labels are determined by y i =

Original: ( s i > ξ ) .
Revised: ( s i > ξ ) .

Original: [Table]

Original: • Random :
Revised: • Random :

Original: The training labels are determined as follows: for s i > ξ , we sample y i from auniform distribution on (0 . 5 , 1] ; for s i ≤ ξ , we sample y i from a uniform distribution on[0 , 0 . 5] .[Table]
Revised: The training labels are determined as follows: for s i > ξ , we sample y i from a uniform distribution on (0 . 5 , 1] ; for s i ≤ ξ , we sample y i from a uniform distribution on [0 , 0 . 5] .

Original: • Softmax : The training labels are determined by y i
Revised: • Softmax : The training labels are determined by y i

Original: =
Revised: =

Original: (1 + e si − ξ ξ ) − 1 .•
Revised: (1 + e si − ξ ξ ) − 1 .

Original:  Linear :
Revised: • Linear :

Original: The training labels are determined by a piecewise linear function:
Revised: The training labels are determined by a piecewise linear function:

Original: y i =(
Revised: y i =

Original:  s i − min i s i ) 2( ξ − min
Revised: ( s i − min i s i ) 2( ξ − min

Original: i s i ) 

Revised: i s i )

Original: ( s i ≤ ξ ) + (cid:16) ( s i − ξ ) 2(max
Revised: ( s i ≤ ξ ) + (cid:16) [Table] [Equation] [Table]

Revised: [Table]

Original: i s i − ξ)
Revised: i s [Table] ξ

Original:  +0 .
Revised: [Table] +

Original: [-5 (cid:17) 
-]
Revised: [-5 (cid:17) 
-][Table] [Table]

Original: ( s i > ξ ) .
Revised: i > [Table] [Table]

Original: Results are shown in Tab. 1.
Revised: Results are shown in Tab. 1.

Original: Compared with binary labelling, both linear and softmax labelling improves fairness under different sensitive attributes, while random labelling achieves little improvementin fairness.
Revised: Compared with binary labelling linear and softmax labelling improves fairness under different sensitive attributes, while random labelling achieves little improvement in fairness.

Original: This shows the effectiveness of proper label smoothing
Revised: This shows the effectiveness of proper label smoothing.

Original: [Table]
Revised: Points of different shapes refer to samples of different labels, different colors refer to different sensitive groups and larger pattern indicates higher weight.

Original: [Table] 3.3 Knowledge Distillation
Revised: By focusing on hard samples that are correctly classified (marginal samples that are correctly classified), our method learns a better decision boundary which improves the performance on hard samples, and disadvantaged group(s) are more likely to benefit from this.

Original: For most classification tasks with categorical encoding, continuous labels are not available.
Revised: For most classification tasks with categorical encoding, continuous labels are not available.

Original: In light ofthis, we instead consider knowledge distillation for obtaining soft labels.
Revised: In light of this, we instead consider knowledge distillation for obtaining soft labels.

Original: We considering the binaryclassification setting, although our method may be readily generalized to multi-class classification.
Revised: We considering the binary classification setting, although our method may be readily generalized to multi-class classification.

Original: Knowledge distillation is a form of model compression through the process of transferring theknowledge from a large pre-trained teacher model or set of models to a single smaller student model that can be practically deployed under real-world scenarios, and is performed commonly on neuralnetworks associated with complex architectures.
Revised: Knowledge distillation is a form of model compression through the process of transferring the knowledge from a large pre-trained teacher model or set of models to a single smaller student model that can be practically deployed under real-world scenarios, and is performed commonly on neural networks associated with complex architectures.

Original: We consider response-based knowledge, whichfocuses on the final output layer of the teacher model, and the student model learns to mimic thepredictions of the teacher model.
Revised: We consider response-based knowledge, which focuses on the final output layer of the teacher model, and the student model learns to mimic the predictions of the teacher model.

Original: Generally, given a training set { ( x i , y i ) , 1 ≤ i ≤ N } ∼ X × Y , let g : X → Y , x (cid:55)→ ˆ y t be theteacher model, and f : X → Y , x (cid:55)→ ˆ y s be the student model, the training objective for studentmodel can be formulated as
Revised: Generally, given a training set { ( x i , y i ) , 1 ≤ i ≤ N } ∼ X × Y , let g : X → Y , x (cid:55)→ ˆ y t be the teacher model, and f : X → Y , x (cid:55)→ ˆ y s be the student model, the training objective for student model can be formulated as

Original: [Equation]
Revised: [Equation]

Original:  where ˆ y t i = g
Revised: (1) where ˆ y t i = g

Original: ( x i ) is the soft label from teacher model, L cls is the classification loss, and α ∈ (0 , 1)is the trade-off hyper-parameter.
Revised: ( x i ) is the soft label from teacher model, L cls is the classification loss, and α ∈ (0 , 1) is the trade-off hyper-parameter.

Original: For a classification task with c classes, the teacher model can bedecomposed into two functions, g =
Revised: For a classification task with c classes, the teacher model can be decomposed into two functions, g =

Original: ϕ ◦ h , where h : X → R c , x (cid:55)→ z is the nonlinear function thatpredicts the logit, and ϕ : R c
Revised: ϕ ◦ h , where h : X → R c , x (cid:55)→ z is the nonlinear function that predicts the logit, and ϕ : R c

Original: → [0 , 1] c , z (cid:55)→ ˆ y t is the mapping function for soft labelling.
Revised: → [0 , 1] c , z (cid:55)→ ˆ y t is the mapping function for soft labelling.

Original: We consider two different mapping functions ϕ :
Revised: We consider two different mapping functions ϕ :

Original: Softmax function .
Revised: Softmax function .

Original: Given the output vector z i
Revised: Given the output vector z i

Original: = h ( x i ) by the teacher model
Revised: = h ( x i ) by the teacher model

Original: , we apply softmaxactivation ϕ softmax to convert the logits into probability vector ˆ y t i , and the predicted probability forj -th class is computed by comparing the logit z ij with the other logits:
Revised: , we apply softmax activation ϕ softmax to convert the logits into probability vector ˆ y t i , and the predicted probability for j -th class is computed by comparing the logit z ij with the other logits:

Original: [Equation]
Revised: [Equation]

Original: where T is the temperature that controls the probability distribution over classes.
Revised: where T is the temperature that controls the probability distribution over classes.

Original: Linear function .
Revised: Linear function .

Original: Given the logits, z ij , computed for each class by teacher model, let z ij ′ =
Revised: Given the logits, z ij , computed for each class by teacher model, let z ij ′ =

Original: max l z ilbe the largest logit prediction for x i and S j = { i | z ij = z ij ′ } , we first use linear function to convertthe largest logit z ij ′ of sample x i into a probability r ij ′ :
Revised: max l z il be the largest logit prediction for x i and S j = { i | z ij = z ij ′ } , we first use linear function to convert the largest logit z ij ′ of sample x i into a probability r ij ′ :

Original: [Equation]
Revised: [Equation]

Original: where the decision threshold for logits that corresponds to predicted probability 0 .
Revised: where the decision threshold for logits that corresponds to predicted probability 0 .

Original: 5 is chosen asthe smallest logits values of samples whose raw logits predictions are the highest for the j -th class.
Revised: 5 is chosen as the smallest logits values of samples whose raw logits predictions are the highest for the j -th class.

Original: Probability for the other logits can then be linearly distributed as follows:
Revised: Probability for the other logits can then be linearly distributed as follows:

Original: [Equation]
Revised: [Equation]

Original: 3.4 Theoretical Analysis
Revised: 3.4 Theoretical Analysis

Original: In this subsection, we provide formal analysis on the impact of label smoothing and knowledge distillation on fairness.
Revised: In this subsection, we provide formal analysis on the impact of label smoothing and knowledge distillation on fairness.

Original: We first discuss the impact of label smoothing and its connection to reweighing.
Revised: We first discuss the impact of label smoothing and its connection to reweighing.

Original: For ease of exposition, here we consider binary classification where y ∈ { 0 , 1 } .
Revised: For ease of exposition, here we consider binary classification where y ∈ { 0 , 1 } .

Original: Denote as y ′ =
Revised: Denote as y ′ =

Original: α ˆ y t + (1 − α )
Revised: α ˆ y t + (1 − α )

Original: y the soft label, the cross entropy loss under soft label can be formulated as
Revised: y the soft label, the cross entropy loss under soft label can be formulated as

Original: [Equation]
Revised: [Equation]

Original: And the cross entropy loss under binary label can be formulated as
Revised: And the cross entropy loss under binary label can be formulated as

Original: [Equation]
Revised: [Equation]

Original: Suppose the classifier f makes same predictions on x under both binary and soft labelling at currentiteration, we have
Revised: Suppose the classifier f makes same predictions on x under both binary and soft labelling at current iteration, we have

Original: [Equation]
Revised: [Equation]

Original: where z i is the logit of student network for i -th class.
Revised: where z i is the logit of student network for i -th class.

Original: Since α ≥ 0 , for L soft − L hard < 0 , we havey = 0 , z 1 > z 0 or y = 1 , z 1 < z 0 , which shows that label smoothing down-weighs samples that areincorrectly predicted.
Revised: Since α ≥ 0 , for L soft − L hard < 0 , we have y = 0 , z 1 > z 0 or y = 1 , z 1 < z 0 , which shows that label smoothing down-weighs samples that are incorrectly predicted.

Original: For samples that are correctly predicted, larger discrepancy between y andˆ y t indicates larger ∆ , which shows that the classifier pays more attention to hard samples that arecorrectly predicted, and larger α indicates larger discrepancy between L soft and L hard .
Revised: For samples that are correctly predicted, larger discrepancy between y and ˆ y t indicates larger ∆ , which shows that the classifier pays more attention to hard samples that are correctly predicted, and larger α indicates larger discrepancy between L soft and L hard .

Original: Similarly, we can compute the induced weight by label smoothing as follows:
Revised: Similarly, we can compute the induced weight by label smoothing as follows:

Original: [Equation]
Revised: [Equation]

Original: Here we only consider positive samples for ease of discussion.
Revised: Here we only consider positive samples for ease of discussion.

Original: Let τ ( x ) = log

Original: [Equation]

Original: [Equation] x [Equation] log ( f ( x )) , we have
Revised: Let τ ( x ) = log (1 − f ( x )) log ( f ( x )) , we have

Original: [Equation]
Revised: [Equation]

Original: Since for f ( x ) > 0 . 5 , we have τ ( x ) > 1 , and w decreases as ˆ y t increases.
Revised: Since for f ( x ) > 0 . 5 , we have τ ( x ) > 1 , and w decreases as ˆ y t increases.

Original: In this way, samples withhigh soft label but low predicted probability, i.e. positive hard samples that are correctly classified areassigned with higher weight.
Revised: In this way, samples with high soft label but low predicted probability, i.e. positive hard samples that are correctly classified are assigned with higher weight.

Original: For samples that are wrongly classified, we have τ ( x ) < 1 , and sampleswith lower soft label and lower predicted probability, i.e. hard samples that are wrongly classified areassigned with the lowest weight.
Revised: For samples that are wrongly classified, we have τ ( x ) < 1 , and samples with lower soft label and lower predicted probability, i.e. hard samples that are wrongly classified are assigned with the lowest weight.

Original: Besides, since ˆ y t ≤ y , as α increases, y ′ decreases, and samples thatare correctly predicted are assigned with higher weights, while samples that are incorrectly predictedare assigned with lower weights.
Revised: Besides, since ˆ y t ≤ y , as α increases, y ′ decreases, and samples that are correctly predicted are assigned with higher weights, while samples that are incorrectly predicted are assigned with lower weights.

Original: Similar results also holds for negative samples.
Revised: Similar results also holds for negative samples.

Original: We further derive theoretical analysis to show that our trained with label smoothing and knowledgedistillation guarantees fairness in terms of equal odds.
Revised: We further derive theoretical analysis to show that our trained with label smoothing and knowledge distillation guarantees fairness in terms of equal odds.

Original: Theorem 3.1.
Revised: Theorem 3.1.

Original: Consider a classifier f : X → [0 , 1] for binary classification.
Revised: Consider a classifier f : X → [0 , 1] for binary classification.

Original: Denote the classificationloss as 
Revised: Denote the classification loss as L soft = − y ′ log( f ( x )) − (1 − y ′ )

Original: [Equation] 1 − f ( x )) with soft label y ′ =
Revised: log(1 − f ( x )) with soft label y ′ =

Original: α ˆ y t + (1 − α )
Revised: α ˆ y t + (1 − α )

Original: y , whereˆ y t ∈ [0 , 1] is the predicted label from teacher model, y ∈ { 0 , 1 } is the binary label, and α is thebalance parameter.
Revised: y , where ˆ y t ∈ [0 , 1] is the predicted label from teacher model, y ∈ { 0 , 1 } is the binary label, and α is the balance parameter.

Original: The equal odds fairness metric w.r.t.
Revised: The equal odds fairness metrics w.r.t.

Original: classifier f is upper bounded by L soft .
Revised: classifier f is upper bounded by L soft .

Original:  Experiments 
Revised:  Experiments 4.1 Experimental Setup

Original: 4.1 Experimental Setup

Original: We validate our model on three benchmark dataset:
Revised: We validate our model on three benchmark dataset:

Original: • New Adult :
Revised: • New Adult :

Original: The Adult reconstruction dataset (Ding et al., 2021) contains 49,531 sampleswith 14 attributes.
Revised: The Adult reconstruction dataset (Ding et al., 2021) contains 49,531 samples with 14 attributes.

Original: The goal is to predict whether an individual’s income exceeds certainthreshold.
Revised: The goal is to predict whether an individual’s income exceeds certain threshold.

Original: During experiments, we set the binary cutoff threshold as 50 K , and we chooserace and gender as sensitive attribute.• COMPAS : The COMPAS dataset (Larson et al., 2016) contains 7,215 samples with 11attributes.
Revised: During experiments, we set the binary cutoff threshold as 50 K , and we choose race and gender as sensitive attribute. • COMPAS : The COMPAS dataset (Larson et al., 2016) contains 7,215 samples withattributes.

Original: Following previous works on fairness (Zafar et al., 2017), we only select blackand white defendants in COMPAS dataset, and the modified dataset contains 6,150 samples.
Revised: Following previous works on fairness (Zafar et al., 2017), we only select black and white defendants in COMPAS dataset, and the modified dataset contains 6,150 samples.

Original: The goal is to predict whether a defendant reoffends within two years, and we choose sexand race as sensitive 
Revised: The goal is to predict whether a defendant reoffends within two years, and we choose sex and race as sensitive attributes.

Original: attributes.• CelebA : The CelebA dataset (Liu et al., 2015) contains 202,599 face images, each ofresolution 178 × 218 , with 40 binary attributes.
Revised: • CelebA : The CelebA dataset (Liu et al., 2015) contains 202,599 face images, each of resolution 178 × 218 , with 40 binary attributes.

Original: We consider two different binary classifcationtasks on CelebA dataset: predicting attractiveness with gender as sensitive attribute andpredicting gender with age as sensitive attribute.
Revised: We consider two different binary classifcation tasks on CelebA dataset: predicting attractiveness with gender as sensitive attribute, and predicting gender with age as sensitive attribute.

Original: We implement our method in PyTorch 1.10.1 with one NVIDIA RTX-3090 GPU.
Revised: We implement our method in PyTorch 1.10.1 with one NVIDIA RTX-3090 GPU.

Original: We use accuracyas evaluation metric, and we apply disparate impact (Kamiran and Calders, 2012) and equalizedodds (Hardt et al., 2016) as fairness metrics.
Revised: We use accuracy as evaluation metric, and we apply disparate impact (Kamiran and Calders, 2012) and equalized odds (Hardt et al., 2016) as fairness metrics.

Original: We build the teacher model using ResNet-152 (Heet al., 2016) and student model using ResNet-18 (He et al., 2016).
Revised: We build the teacher model using ResNet-152 (He et al., 2016) and student model using ResNet-18 (He et al., 2016).

Original: We compare our method with thefollowing related methods:
Revised: We compare our method with the following related methods:

Original: • Teacher model :
Revised: • Teacher model :

Original: Teacher network trained on binary label.
Revised: Teacher network trained on binary label.

Original: The teacher model is expected to overfit training data.
Revised: The teacher model is expected to overfit training data.

Original: • Student model (binary) :
Revised: • Student model (binary) :

Original: Student network trained on binary label.
Revised: Student network trained on binary label.

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: • Student model (softmax) :
Revised: • Student model (softmax) :

Original: Student network trained on soft label obtained by the output ofteacher model with softmax activation as in (2).
Revised: Student network trained on soft label obtained by the output of teacher model with softmax activation as in (2).

Original: • Student model (linear) :
Revised: • Student model (linear) :

Original: Student network trained on soft label obtained by the output ofteacher model with linear normalization as in (3).• Distributionally robust optimization (DRO) :
Revised: Student network trained on soft label obtained by the output of teacher model with linear normalization as in (3). • Distributionally robust optimization (DRO) :

Original: Student network with distributionally robust optimization (Hashimoto et al., 2018).
Revised: Student network with distributionally robust optimization (Hashimoto et al., 2018).

Original: The network is trained on binary label.
Revised: The network is trained on binary label.

Original: • Adversarially reweighted learning (ARL) :
Revised: • Adversarially reweighted learning (ARL) :

Original: Student network with adversarially reweightedlearning (Lahoti et al., 2020).
Revised: Student network with adversarially reweighted learning (Lahoti et al., 2020).

Original: The network is trained on binary label.
Revised: The network is trained on binary label.

Original: The adversarialnetwork is chosen as one linear layer as suggested by the authors (Lahoti et al., 2020).
Revised: The adversarial network is chosen as one linear layer as suggested by the authors (Lahoti et al., 2020).

Revised: • FairRF : Student network with FairRF (Zhao et al., 2022).

Revised: The network is trained on binary label.

Revised: We include results of FairRF only on new Adult and COMPAS dataset, as it is hard to regulate sensitive-related features for images.

Original: We repeat experiments on each dataset five times and report the average results.
Revised: We repeat experiments on each dataset five times and report the average results.

Original: To avoid largediscrepancies in testing data, before each repetition, we randomly spilt data into 50 % training data,10 % validation data and 40 % test data.
Revised: To avoid large discrepancies in testing data, before each repetition, we randomly spilt data into 50 % training data, 10 % validation data and 40 % test data.

Original: All the methods evaluated are trained and tested on the samedata partitions each time.
Revised: All the methods evaluated are trained and tested on the same data partitions each time.

Original: For student model trained on softmax label, the temperature is tuned tofind the best validation accuracy.
Revised: For student model trained on softmax label, the temperature is tuned to find the best validation accuracy.

Original: The hyperparameters of comparing methods are tuned with binarysearch to find global minimum, as suggested in the original paper (Hashimoto et al., 2018).[Table] [Table]
Revised: The hyperparameters of comparing methods are tuned with binary search to find global minimum, as suggested in the original paper (Hashimoto et al., 2018).

Original: 4.2 Experimental Results
Revised: 4.2 Experimental Results

Original: Experimental results are shown in Table 2-7.
Revised: Experimental results are shown in Table 2-7.

Original: On all three datasets, both DRO and ARL show a drop inaccuracy, especially for DRO which strictly follows the formulation of Max-Min fairness.
Revised: On all three datasets, both DRO and ARL show a drop in accuracy, especially for DRO which strictly follows the formulation of Max-Min fairness.

Original: However,both methods do not show notable improvement in fairness.
Revised: However, both methods do not show notable improvement in fairness.

Original: This is in line with our hypothesisthat Max-Min fairness could be too strict to achieve expected improvement in group fairness.
Revised: This is in line with our hypothesis that Max-Min fairness could be too strict to achieve expected improvement in group fairness.

Original: Incomparison, our method does not induce as much decrease in accuracy, but achieves significantimprovement in terms of fairness on all three datasets, especially for equalized odds. 
Revised: In comparison, our method does not induce as much decrease in accuracy, but achieves significant improvement in terms of fairness on all three datasets, especially for equalized odds. This shows the [Table] [Table]

Original: This shows theeffectiveness of knowledge from teacher model and error-based reweighing for soft labelling.
Revised: [Table] [Table] [Table] effectiveness of knowledge from teacher model and error-based reweighing for soft labelling.

Original: Weobserve that the pre-trained overfitted teacher model achieves higher accuracy on all three datasets,which is in line with the expectation of knowledge distillation.
Revised: We observe that the pre-trained overfitted teacher model achieves higher accuracy on all three datasets, which is in line with the expectation of knowledge distillation.

Revised: However, the overfitted teacher model does not improve fairness

Revised: [Table]

Original: However, the overfitted teacher modeldoes not improve fairness, and the equalized odds of teacher model for COMPAS and new Adultdataset are much worse than baseline student model, and we conclude that improvement in fairnesswith knowledge distillation is due to soft labelling, instead of complexity of teacher model. 
Revised: and the equalized odds of teacher model for COMPAS and new Adult dataset are much worse than baseline student model [Table] and we conclude that improvement in fairness with knowledge distillation is due to soft labelling, instead of complexity of teacher model. [Table]

Original: 4.3 Parameter Analysis
Revised: 4.3 Parameter Analysis

Revised: [Figure]

Original: We move on to discuss the influence of the trade-off parameter α in (1) on accuracy and fairness.
Revised: We move on to discuss the influence of the trade-off parameter α in (1) on accuracy and fairness.

Original: It is straightforward to see that as α varies, the corresponding loss term can be treated as linearinterpolation between binary cross-entropy loss and soft cross-entropy loss, and increasing α increasethe role of soft label in knowledge distillation, which we observe to help improve fairness as in Fig.
Revised: It is straightforward to see that as α varies, the corresponding loss term can be treated as linear interpolation between binary cross-entropy loss and soft cross-entropy loss, and increasing α increase the role of soft label in knowledge distillation, which we observe to help improve fairness as in Fig.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, as α increases, there is a small decrease in accuracy, and on all three datasets, asα increases, under both softmax and linear labelling, although with fluctuation under certain variationof α on COMPAS and new Adult datasets, overall the equalized odds regarding different sensitiveattributes show downtrend, which validates the effectiveness of soft labelling.
Revised: 3, as α increases, there is a small decrease in accuracy, and on all three datasets, as α increases, under both softmax and linear labelling, although with fluctuation under certain variation of α on COMPAS and new Adult datasets, overall the equalized odds regarding different sensitive attributes show downtrend, which validates the effectiveness of soft labelling.

Original:  Conclusion
Revised:  Conclusion

Original: Fairness without demographics has been an important yet less studied task.
Revised: Fairness without demographics has been an important yet less studied task.

Original: In this paper, we proposeto solve this problem through knowledge distillation.
Revised: In this paper, we propose to solve this problem through knowledge distillation.

Original: We consider a pre-trained complex teachermodel that is overfitted to training data, and we use the normalized logits as soft label to train asimple student model.
Revised: We consider a pre-trained complex teacher model that is overfitted to training data, and we use the normalized logits as soft label to train a simple student model.

Original: We consider two different normalization functions, the softmax functionand linear function.
Revised: We consider two different normalization functions, the softmax function and linear function.

Original: We theoretically discuss the connection between soft labelling and reweighing,and we prove theoretically that with knowledge distillation, the training objective of student modelupper-bounds several fairness metrics.
Revised: We theoretically discuss the connection between soft labelling and reweighing, and we prove theoretically that with knowledge distillation, the training objective of student model upper-bounds several fairness metrics.

Original: We validate from preliminary experiments on new Adultdataset the effectiveness of soft labelling in improving fairness, and we show from experiments thatknowledge distillation achieves remarkable improvement in fairness with relatively small sacrifice inaccuracy. [Figure]
Revised: We validate from preliminary experiments on new Adult dataset the effectiveness of soft labelling in improving fairness, and we show from experiments that knowledge distillation achieves remarkable improvement in fairness with relatively small sacrifice in accuracy.

Original: [Figure]
Revised: [Figure]

Original: There are two main directions of interest for future work.
Revised: There are two main directions of interest for future work.

Original: First is the alternatives of mappingfunction: can we define other mapping functions for the logits such that the student model achievebetter fairness?
Revised: First is the alternatives of mapping function: can we define other mapping functions for the logits such that the student model achieve better fairness?

Original: Understanding the influence of mapping functions is crucial in extending our work.
Revised: Understanding the influence of mapping functions is crucial in extending our work.

Original: Another topic is robustness of our method to distributional shift, especially the robustness of fairness,as robustness to distributional shift has been an important but challenging problem in the contextfairness. Checklist For all authors...
Revised: Another topic is robustness of our method to distributional shift, especially the robustness of fairness, as robustness to distributional shift has been an important but challenging problem in the context fairness.

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?

Original: [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
Revised: Other topics include few-shot tweaking with partially available sensitive attributes.

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)? [Yes]
Revised: Acknowledgements

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators? [Yes] (b) Did you mention the license of the assets? [Yes]
Revised: This work was partially supported by the EMBRIO Institute, contract #2120200, a National Science

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?

Original: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content? N/A]5. If you used crowdsourcing or conducted research with human subjects...( a)
Revised: Foundation (NSF) Biology Integration Institute, Purdue’s Elmore ECE Emerging Frontiers Center and NSF IIS #1955890, IIS #

Original: Did

Original: you include the full text of instructions given to participants and screenshots

Original: , if

Original: applicable?

Original: [N/A

Original: b) Did you describe any potential participant

Original: risks

Original: , with

Original: links

Original: to

Original: Institutional Review

Original: Board

Original: (IRB) approvals, if applicable?

Original: [N/A]

Original: (c)

Original: Did

Original: you include

Original: the estimated hourly

Original: wage paid

Original: to

Original: participants

Original: and the total

Original: amountspent on participant

Original: compensation?

Original: [N/A