Adversarial Latent Feature Augmentation for Fairness
Authors: Hoin Jung
Junyi Chai
Xiaoqian Wang
Venue: ICLR 2025 Poster (2025)
Keywords: Fairness
Data Augmentation
Adversarial Attack
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Achieving fairness in machine learning remains a critical challenge, especially due to the opaque effects of data augmentation on input spaces within nonlinear neural networks. Nevertheless, current approaches that emphasize augmenting latent features, rather than input spaces, offer limited insights into their ability to detect and mitigate bias. In response, we introduce the concept of the "unfair region" in the latent space, a subspace that highlights areas where misclassification rates for certain demographic groups are disproportionately high, leading to unfair prediction results. To address this, we propose Adversarial Latent Feature Augmentation (ALFA), a method that leverages adversarial fairness attacks to perturb latent space features, which are then used as data augmentation for fine-tuning. ALFA intentionally shifts latent features into unfair regions, and the last layer of the network is fine-tuned with these perturbed features, leading to a corrected decision boundary that enhances fairness in classification in a cost-effective manner. We present a theoretical framework demonstrating that our adversarial fairness objective reliably generates biased feature perturbations, and that fine-tuning on samples from these unfair regions ensures fairness improvements. Extensive experiments across diverse datasets, modalities, and backbone networks validate that training with these adversarial features significantly enhances fairness while maintaining predictive accuracy in classification tasks.

Published as a conference paper at ICLR 2025
ADVERSARIAL LATENT FEATURE AUGMENTATION FOR FAIRNESS
Hoin Jung, Junyi Chai & Xiaooian Wang" Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907, USA
(jung414, chai28, o joywang! @purduu. edu
ABSTRACT
Achieving fairness in machine learning remains a critical challenge, especially due to the opaque effects of data augmentation input spaces within nonlinear neural networks. Nevertheless, current approaches that emphasize augmenting latent features, rather than input spaces, offer limited insights into their ability to detect and mitigate bias. In response, we introduce the concept of the "unfair region" in the latent space, subspace that highlights areas where misclassification rates for certain demographic groups are disproportionately high, leading to unfair prediction results. To address this, we propose Adversarial Latent Feature Aug- mentation (ALFA), method that leverages adversarial fairness attacks to perturb latent space features, which are then used as data augmentation for fine-tuning. ALFA intentionally shifts latent features into unfair regions, and the last layer of the network is fine-tuned with these perturbed features, leading to a corrected decision boundary that enhances faimess in classification in cost-effective manner. We present a theoretical framework demonstrating that our adversarial faimesss objec- tive reliably generates biased feature perturbations, and that fine-tuning samples from these unfair regions ensures fairness improvements. Extensive experiments across diverse datasets, modalities, and backbone networks validate that training with these adversarial features significantly enhances fairness while maintaining predictive accuracy in classification tasks. The code is available on GitHub
INTRODUCTION
The issue of fairness in machine learning is a well-recognized and multifaceted challenge. Addressing fairness often involves manipulating or augmenting data to address inequalities between demographic groups in the input space, as studied in (Jang et al.) [021] Rajabi & Garibay 2022). However, the transparency and efficacy of data augmentation in the input space to foster fairness are not always clear due to the challenge of determining how transformations affect the nonlinear decision boundary. This complexity has led to exploring augmentation strategies in the latent space, allowing for more nuanced analysis of augmentation's 's impact.
The linearity of the last layer in neural networks' latent space, such as in Multilayer Perceptron (MLP) and Convolutional Neural Networks (CNNs), facilitates the examination of fairness issues at the decision boundary. For example, (Buolamwini & Gebru] 2018)) highlights how demographic imbalances, like race, can lead to disproportionately higher misclassification rates, such as higher false positives for individuals with darker skin tones in facial analysis software (Klare et al. 2015). This example illustrates how biased data result in unfair outcomes. In the latent space, the linear classifier enables a clearer examination of how such imbalances in data distribution impact the decision boundary. Beyond group-level analysis, exploring the latent space helps to identify misclassified segments influenced by the linear classifier.
In short, exploring fairness within the latent space of neural networks is crucial for understanding and mitigating biases. However, existing latent augmentation methods often overlook the fundamental
"Coreeponniigg author.

---

Published as a conference paper at ICLR 2025
Decision Boundary
Label
Scnsitive Attribute 0
Modified Decision Boundary
Label
Sensitive Attribute 1
RecoVer Unfair Region
Unfair Region
(a) Definition of Unfair Region
(b) Recovering Unfair Regions by ALFA
Figure 1: A synthetic data example illustrating faimess issues and the identification of unfair regions within the latent features. Let the demographic group { 1} privileged be predicted Y 1. The misclassification rates of subgroup {A 1, Y = 0} and {A = 0, Y 1} are disproportionnaely high, indicated as unfair region the left figure. ALFA generates adversarial perturbations in the latent space against the fairness constraint, pushing the features towards biased direction that the perturbed features overlap with the unfair region. Fine-tuning the last layer on these perturbed features adjusts the decision boundary, correcting the unfair region and resulting in fairer predictions.
question of where and how fairness issues originate within the latent distribution. For example, Fair-Mixup (Mroueh et al.) (021) operates under the assumption that a manifold exists in the latent space between two demographic groups and advocates for data generation via interpolation on this manifold. However, this assumption may be overly stringent, and Fair-Mixup does not specifically address where fairness issues arise in the latent space.
Similarly, FAAP (Wang et al.) 2022) attempts to obfuscate sensitive attributes in the latent repre- sentation by projecting features towards the sensitive decision boundary. The challenge, however, arises when perturbed features align along this boundary, potentially distorting the original feature distribution. If these features remain confined to a linear alignment, the resulting decision boundary can vary significantly, reducing the model's 's generalization capacity.
To address these challenges, we propose the concept of the unfair region to analyze the root causes of fairness issues in the latent space. This region is characterized by disproportionate misclassification rates between privileged and underprivileged groups. Figure (a) illustrates this concept, highlighting areas where biased predictions most prevalent. We further demonstrate the extent of bias within the unfair region using synthetic data, as shown in Figure Details of the synthetic data are introduced in Appendix C
However, examining the nigh-dimensional latent distribution is not straightforward. To automate the detection and correction of unfair regions in the latent space, we propose a novel approach called Adversarial Latent Feature Augmentation (ALFA). This method employs a counter-intuitive use of adversarial attacks and data augmentation. Specifically, we introduce fairness attack by perturbing latent features based a fairness constraint (Zafrr et al. 2017). For instance, the perturbation pushes privileged groups toward favorable outcomes while directing underprivileged groups toward unfavorable outcomes, regardless of their true labels. This manipulation strengthens the correlation between sensitive attributes and decision outcomes. To maintain the semantic integrity of perturbed features, ALFA minimizes the Sinkhom distance (Genevay al. 2018) between the original and perturbed features.
Consequently, fine-tuning the classifier with these perturbed features helps correct the decision boundary, directly addressing the unfair regions and achieving more balanced misclassification rates across demographic groups. Figure (b) illustrates this concept using synthetic data, showcasing the
2

---

Published as a conference paper at ICLR 2025
gel anl
get pnl
Unfair Misclassification Rate
yo0 and
gml ped
Misclas soo Rabe
y=1.3=0
ye0, anl
y=0,
Unfair Regions
Fair Misclassification Rate
(a) Synthetic Data
a-1 1-1
Biased Prediction
gat
gao
Ovarall Nisclasiiiiation Rate PCd
Debiased Prediction an13
330
420
abs
km
(b) Perturbation on Synthetic Data
(c) Misclassification Rate VS. 8
Figure 2: (a) In the synthetic dataset, regions exhibiting disproportionately high misclassification rates are identified, indicating potential unfairness in predictions. (b) A fairness attack introduces perturbations, denoted by 5 towards these unfair regions, attempting to balance the misclassification rate across groups by the corrected decision boundary (red line in (b)). (c) An appropriate d can equalize misclassification rates, achieving debiased predictions with lower A FFdd (sum of FPR gap and TPR gap), without significant compromising accuracy, as shown in the gray line in (c). The improvement in AEOd is theoretically guaranteed as described in Section 3.3
impact of adversarial perturbations on the decision boundary. Furthermore, we provide theoretical proof that training on perturbed datasets improves fairness Section 3.3
Our method is validated through extensive experiments on various datasets, including tabular datasets such as Adult, COMPAS, German, and Drug; images from CelebA; and text from Wikipedia, demonstrating its versatility. These experiments confirm that our method preserves accuracy while significantly enhancing group fairness across diverse datasets and backbone networks.
We summarize our contributions as follows:
Introduced a novel latent space data augmentation method aimed at identifying and rectifying areas of unfairness in classification models.
2. Provided a theoretical foundation that elucidates the counter-intuitive impact of adversarial perturbations on improving fairness, supported by visual illustrations of the corrected decision boundary.
3. Demonstrated that our method consistently achieves group faimess without compromising accuracy through experiments on tabular, image, and text datasets.
2 RELATED WORK
2.1 FAIRNESS IN MACHINE LEARNING
Diverse approaches have been proposed to secure fairness the classification tasks. Chai & Wang (2022) and Li & Liu (2022) proposed data reweighing; allocating weights for all samples according to their importance. Chai & Wang (022) balanced the gap between demographic groups weighing error- prone samples in adaptive way. Li & Liu (2022) adopted influence function (Koh & Liang 2017) to evaluate individual sample's 's importance in affecting prediction. As an in in-processing approach, Zafar et al. (2017) and WW et al. (2019) developed fairness constraint adopting covariance between sensitive attribute and classifier, and extending the constraint having convexity, respectively. Jang al. (2024) integrated both data and model perspectives improve faimess.
3

---

Published as a conference paper at ICLR 2025
Modified
Baseline
Sensitive Dec. Bound. Projection IPAAP) [c] Interpolation (FaisMixupp
(d) Ours (ALPA)
Beciivar Unfair Region
Unfair Region
Toward Sensitive Decision Boundary
Interpolation Between pemographic Groups
9 97750 0.1995 EOM DL2222
NCC/9/9666, A. 0.1265 LOR 0.1190
ACC: M.71A 2.1.A, & EOU 02132
ALC: BLNN DM OPIS, ESAT 6.10MS
Figure 3: Comparison of each data manipulation in latent space for synthetic unfair data, (a) a naive classifier, (b) FAAP, (c) Fair-Mixup, and (d) ALFA. The solid black line represents the original decision boundary obtained via Logistic Regression, while the red dashed line shows the updated decision boundary after feature manipulation. The manipulated features, indicated in cyan and red, correspond to perturbed features in FAAP and ALFA, and interpolated features in Fair-Mixup. The blue dashed line in FAAP indicates the sensitive decision boundary. The generated features in FAAP and Fair-Mixus Mixup do not directly consider the region where fairness issue happens, and might not sufficiently mitigate bias.
Some approaches use data augmentation improve fairness. Jang et al. (202T) and Rajahi & Garibay (2022) generated new fair data using VAE and GAN, respectively. Hsu et al. (2022) and /Zhao ct al. (020)) adopted adversarial samples as data augmentation to improve accuracy and robustness, respectively. Similarly, Li et al. (202)) generated antidote data analogous to the original data but containing the opposite sensitive attribute to enhance individual fairness.
Manipulating features in the latent space becomes popular. Mroueh et al. (2021) proposed to generate
new data in the latent space by interpolation between latent features from different sensitive groups to optimize fairness constraints. Wang et al. (2022) suggested adversarial perturbation on the latent features towards the sensitive hyperplane which predicts the demographic group. [Sun et al. (2023) disentangle the latent feature into the sensitive feature and non-sensitive feature and obfuscate the sensitive feature only. Mao et al. (2023) fine-tune the pre-trained classifier by training the last layer with the balanced latent features under the designated fairness constraint. We demonstrate in Figure how our latent feature manipulation differs from that of Mrouch el al. (2021) and Sun et al. (023
In contrast, there exist attempts to attack fairesss. Koh al. (2018) suggested attacking anomaly detectors by blending perturbed data with the natural data and by optimizing influence- based gradient ascent. Mchrabi al. (202T) extended the idea of (Koh et al., (018)) combining the fairness constraint suggested by /Zffar et al. (2017). Similarly, (Solans et al. (2020) developed a gradient- based poisoning attack on algorithmic fairness. Chhabra et al. (022) proposed a fairness attack and defense framework in terms of unsupervised learning and fair clustering.
3 PROPOSED METHOD
Motivation. We use Equalized Odds (EOd) as the criterion for group faimesss. Demographic Parity (DP) requires independence between the predicted outcome and the sensitive attribute A E {0, 1}. such that P( = 0) = =P(ÝA = ie.. Ý 1L. A. However, DP's usefulness is limited when there correlation between Y and A, where Y LL A. EOd overcomes this limitation by conditioning on the true label Y. It requires that P(Ý]A= 1, Y y) = P(Ý)A = 0, Y = y) for all y E {0. 1}. In other words, EOd ensures that the misclassification rates between the two demographic groups are equal for each true label. In general, when a classifier is biased, misclassification occurs in specific regions of the latent space. For instance, if individuals in the privileged group more likely to be predicted positive, i.e., P( group {A = 1} and the false negative rate for group {A = 0} will be disproportionately higher. Specifically, P(ř = 1|A- = 1,Y = 0) 2 P(ý = 1|A = 0, Y = 0) and P(Ý = 0J.4 = 0,Y = 1) > P(Ý 0/44- = 1,Y = 1). To quantify this disparity, we define the evaluation metric as
1) P( 0), the false positive rate for
AFOd = Ey P( 11/=1, 1,Y =)) P( 1 = 0, =
4

---

Published as a conference paper at ICLR 2025
Unfair Region. As shown in Figure I and 2 the wnfair regions represent areas in the latent space where certain demographic groups are disproportionately misclassified by the biased classifier.
Definition 3.1. Consider a linear classifier g(2) sign(w"= 2 + b), where = f(x) represents the latent space generated by an encoder f. and w and are the classifier's 's weights and bias. The classifier g(2) is considered biased if its decision boundary w + b results in sigmificantly different outcomes (e.g., higher misclassification rates) for the groups defined by the sensitive attribute A E {0, }}. We define the unfair region Ranfair as the subspace of the latent space where the classifier's decision boundary results in disproportionately high differences in error rates between these groups:
Runfair {= E Rd P(r(=) = y A/A= = -(g(s) = y/ = 0) > r) Vy E (0, 13,
(I)
where 7 is threshold indicating significant bias.
The proposed method automatically identifies the unfair regions Runtair and generates perturbed samples that directly cover the area by over- under-representing demographic groups for each label, leveraging a fairness attack. This region highlights where the classifier exhibits significant discrepancies outcomes across demographic groups. Notably, this identification does not rely predefined threshold r but instead uses the attack to pinpoint areas where bias is most pronounced. Consequently, training the last layer of the network on these perturbed latent features corrects the decision boundary, reducing the misclassification rates for biased subgroups.
3.1 FAIRNESS ATTACK
In this section, we adopt an objective function suggested in Zafar et al., 2017) for fairness attack, Lruir Zafar al. (2017) suggested measurement for disparate impact using covariance between the sensitive attribute a and the signed distance d& from x to the decision boundary, i.e. Cov(a, de) z 0 means fair where the signed distance de obtained by the logit (inverse sigmoid) function from the predicted probability U. i.e. dg = (g). Contrary to (Zafar et al., 2017), we maximize the covariance between the sensitive feature and the signed distance between the perturbed feature and the decision boundary of the pre-trained classifier. Therefore, the fairness constraint Lfair is defined
Lair = [Cov(a, R (0)) = (Cou(a, + 6))l
(2)
where the overall model consists of an encoder f and linear classifier such that 9 == ((f(x)) = g(z). the input, x e RVx is the latent feature, 8 E R.N is the perturbation, N is the number of samples, and d is the dimension of latent feature. Let E + 8 and d: = g( g(2,), then Eq. 2 becomes
N₂ Σ
1 N a) (d)=
Liair = (Coola, (()))(((2)))))))))))))))))))(( = [[(a= E(o(())
where Np the number of target samples and d is the mean of all d, In the fairness attack, we adopt upsampling strategy selecting the same size of samples from each subgroup as an attacking target such that N, = 4 max (noo. no1. n1: n11). where nay denotes the number of samples for each subset such that Nay Sagl Say = {ila; a, y/ y), {0, 1} and y E {0, 1}.
In fact, any type of fairness constraint can be applied for Liair during the faimess attack. We demonstrate convex fairness constraint (Wu et al., 2219)) as an alternative the covariance fairness constraint, also showing significant improvements in fairness as demonstrated in Appendix L
positive covariance between two variables indicates that they tend to increase or decrease together, while a negative covariance means an inverse relationship. A faimess attack aims to maximize the covariance to make the sensitive attribute significantly affect the decision of the given classifier.
Instead of [Cov(a, g(2)) in Ltirr we follow the sign of covariance (Cov(a, y)) of the clean dataset to determine Lrair for fairness attack to effectively exacerbate the fairness for the given classifier,
Coc(a, g())) Ltaic - -Coo(a, ((nn((ag())) g(
if Cov(a, y) 2 0
3
if Cou(a, y) < 0.
In this way, we observe in Table 3 in Appendix that the consequent sign of Cov(a, ②) also follows the sign of covariance in clean dataset.
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Adversarial Latent Feature Augmentation
Require: Clean dataset (Xer Yo), hyperparameter a, the number of epochs T,, pretrained encoder
and classifier g.
Ensure: Fair classifier go
Obtain (Xp. Xp) by balanced upsampling for (X.,) Y.).
Obtain latent feature set ,Z, Y,) where Z = f(Xp).
Compute mean absolute distance between latent features and the decision boundary.
Fairness attack to obtain 8 arg max 18]|2Se (fiair a2( + 8)). V/2 E Zp
for = -,...,, do
Fine-tune the classifier g with the adversarial latent feature 2 5*
0 +- 8 = arg ming |x.1+1z,| Ex,ex, Lee (g([(x)))) me 0) Ez,ez, Leelg(Zi + 63), us. 0)).
end for
We prove that Ltair the lower bound AEOd through Proposition B.1 and Theorem B.2 in Appendix B Consequently, we can conduct fairness attack by maximizing Crair causing the perturbed latent features to result unfair predictions with a high AFOd on the given pre-trained classifier.
3.2 SINKHORN DISTANCE
The goal of an adversarial fairness attack is to lead a pre-trained classifier to predict biased results on perturbed samples while maintaining the distribution of given data to keep semantically meaningful. In order to effectively attack the classifier, we adopt the Wasserstein Distance (Arjovsky et al.) 2017) to minimize the statistical distance between and Σ, i.e. D(z, E). Wasserstein distance is powerful tool for measuring the statistical distance between two probability distributions and is sensitive to small perturbations. One drawback of Wasserstein distance is its burden on computational cost. However, a faster and more accurate algorithm is developed to approximate the Wasserstein distance using Sinkhorn iteration, namely Sinkhorn distance (Genevay et al., 2018). Sinkhorn Distance is an approximate entropy regularized Wasserstein distance using the Sinkhorn algorithm measuring the distance between two probability distributions terms of optimal transport problem. detailed explanation and cost effectiveness of Sinkhorn distance is in Appendix D.I
3.3 ADVERSARIAL LATENT FEATURE AUGMENTATION
We propose a novel data augmentation technique in the latent space, Adversarial Latent Fea- ture Augmentation (ALFA) to mitigate the bias in the binary classification. We pre-train the encoder and classifier by empirical risk minimization with binary cross entropy loss Lee. ming N Σ Loe (x;;), w:), where is the input data and E (0, is the class label. The trained classifier potentially biased the particular sensitive attribute due the imbalance in the dataset. As shown in Figure I unfair regions are identified for each label caused by a given classifier which we aim to cover by introducing perturbed latent features having corresponding labels with the over/underrstimated demographic group.
The adversarial latent features are generated by the fairness attack while maintaining their distribution by the Sinkhorn distance denoted as D(z, + 8). During the attacking step, parameters of both encoder ʃ and linear classifier g are frozen. The direction and magnitude perturbation are determined by the fairesss attack introduced in Section 3.1 and 3.2
(Cuir - -alz, +5)
max |
where is hyperparatmer, and is the mean absolute distance between latent features and pre- trained decision boundary. The Sinkhorn distance term is obtained by batch-wise computation. Finally, the classifier is fine-tuned using both the original and adversarial latent features, with the encoder ff remaining frozen, and only the parameters of the linear classifier g being updated. The objective function for the fine-tuning
min
Σ E.
(cc(o(((2))))u.)) + Σ Looglez + 83),,,,0))
(5)
X
ZpI
2,EZ,

---

Published as a conference paper at ICLR 2025
Baseline influnce-reweiiht
Covariance loss
Fair-Mixup
FOR
FairCDA
FairDRo
FAAP
ADULT
COMPAS
GERMAN
DRUG
144
LOGISTIC
0.00
141
800
6:2
5.5 0.8
514
00 125 1P 115 KOS
05
A5Dd
AEDd
ADULT
COMPAS
GERMAN
DRUG
036
MUP
ACC-
0.2
072
0.8
330
4.0
320
312
511
3.1
0.2
2. 03
0.08 0220 0.00 0319
0.50 0100 CE0 2213
1850
06
Figure 4: Faireess- Accuracy trade-off for Logistic Regression (top) and MLP (bottom) on tabular datasets. The x-axis shows AEOd, where smaller values (to the right) indicate better fairness. Thus, the upper-right region reflects better performance. ALFA consistently outperforms other methods, achieving lower AFOd with minimal impact accuracy.
where X is the original dataset and Zp is the upsampled feature dataset to be attacked, respectively. Theorem 3.2. Retraining the classifier using Eg. results in fairer classifier by reducing A F.Od:
( I ΔBOα()).
(6)
where 0 and Op denote the classifier's 's parameter when trained original dataset and combined datuset, respectively. The detailed proof of Theorem 3.2 is provided in Appendix
In the neural networks, the encoder and the last layer are easily defined. However, in the Logistic Regression, there's no encoder is defined. As a special case, in the Logistic Regression, the linear classifier pre-trained in the same manner produce adversarial samples and trained again with data augmentation, while the perturbation conducted on the input space. The detailed algorithm is introduced in Algorithm
EXPERIMENTAL DETAIL
4.1 DATASET
In this paper, we use four different tabular datasets Adult (Dua et al. 2017). COMPAS (Jeff Larson &
Angwin 2016), German (Dua et al. 2017). and Drug (Dua al. 2017). Also CelebA (Liu al. 2018) and Wikipedia Toxicity (Thain et al. 2017) datasets are used for verify the performance of the proposed method in image and text classification, respectively. All datasets are split into 60:20:20 for train, validation, and test subset, respectively. The detailed description of datasets is in Appendix
4.2 EXPERIMENTAL SETUP
To verify our approach, we apply our method to two base classifiers for tabular datasets, Logistic Regression and MLP with ReLU activation function and two hidden layers of 128 dimensions. For the CelebA dataset, we adopt ResNet-50 (He et al.) 2016). VIT (Dosovitskiy. 2020). and Swin Transformer (Liu et al. 2021) as baselines. For the Wiki dataset, use LSTM (Hochreiter & Schmidhuber 1997). BERT (Devlin 2018), and DistilIBERT (Sanh) 2019) as baselines. During the pre-training, choose the best parameter when the validation accuracy the highest. In the
7

---

Published as a conference paper at ICLR 2025
Baseline
FDR
FAAP
Fair-Mixup
ALFA
Attractive
Wavy Hair
Smile
ResNet
0.6
0.55
1:3
0641500400.227500 05300 0200
VIT
0.04
Acc.
0.00
3.000 0.423
3.40
0.0
0.40
0.00
0:000
AEOd
AEDd
AEOd
Swin Transformer ACC.
0.84
Acc.
0.00
3.48 AFOd
5.42
6.04
0:024
AFOd
AzOd
Figure 5: Fairness-Accurayy trad--off for the CclebA dataset. Each row subfigures represents different backbone network: ResNet, VIT, and Swin Transformer, while each column corresponds to a different target attribute for classification. Similar to the results on tabular datasets, ALFA consistently outperforms other methods in fairness without compromising accuracy.
attacking step, parameters of both the encoder and classifier are fixed, while only the last layer is newly initialized for fine-tuning with the augmented latent features. The different learning rates are used in each step, Adam optimizer with learning rate 1e-3 in pre-training and fine-tuning, Adam optimizer with learning rate 0.1 adversarial attack. For each experiment, we take the result when the validation accruacy is the highest. For fair comparison, we train each case 10 times and report the mean and the standard deviation for tabular datasets and text dataset.
To evaluate the fairness improvement of our method, we compare its performance against other approaches using data augmentation, fairness constraints, data reweighing, or latent space manipula- tion methods, such as Covariance Loss (Zafar et al. (017), Fair-Mixup (Mrouch et al.] 202 l), FDR
(Sun et al. (023). Influence- Reweeehnn (Li Liu 2022). FairDRO (Jung et al. 2023), , FAAP (Wang al. 2022 and TabFairGAN (Rajabi & Garibay (022), as shown in Figure For a fair comparison, we follow each method's implementations, and adjust
(Ma et al. 2023).
hyperparameters detailed in Appendix G For the CelebA and Wiki datasets, adopt Fair-Mixup, FAAP, and FDR as comparisons which can operate in the latent space.
Furthermore, we acknowledge the issue of fairness in various tasks, including multi-class, multi-label, and handling multi-sensitive attributes scenarios. These tasks can be considered variants of binary classification, making ALFA applicable to them. We present the extension of ALFA and provide experimental results in Appendix Moreover, we discuss the applicability of our framework as input perturbation on neural networks Appendix
4.3
RESULT ANALYSIS
4.3.1
ACCURACY-PAIRNESS TRADE-OFF
Figure 45 and Gillustrate the trade-off between AEOd and accuracy. Since each comparison method, including ALFA, involves multiple hyperparameters, conduct extensive experiments for each,
8

---

Published as a conference paper at ICLR 2025
Baseline
Fair-Mixup
FAAP
FDR
ALFA
LSTM
BERT
DistilIBERI
1.00
1.003
1.000
1975
6,775
0.815
1.50
6,753
0.350
1.60
6.323
0.373
002
0.55
0.25
0.04 0.32
0.37
0.38
AEOd
AEOd
Figure 6: Fairness-Accuracy trade-off for Wikipedia dataset. Each column of subfigure represents different backbone networks, LSTM, BERT and DistilIBERT. Similar to the tabular datasets, ALFA consistently outperfroms other methods in fairness without compromising accuracy.
displaying the results as line plots. For ALFA, the only hyperparameter used to generate the plot is the weight of the Sinkhorn distance in the fairness attack, denoted by
Across datasets with different modalities, including tabular, image, and text, and with various backbone networks, ALFA consistently outperforms other methods. While ALFA may not always achieve the top performance. it consistently ranks either first or second across all comparison methods, with no other approach demonstrates such high level of overall performance. Notably, in the COMPAS, German, Drug, CelebA, and Wikipedia datasets, across all backbone networks, ALFA achieves significant improvement in EOd with minimal impact on accuracy.
In addition to the trade-off plots, detailed experimental results, including the standard deviation of quantitative outcomes, are presented in Appendix HH further highlighting ALFA's consistent fairness improvements. Moreover, Appendix M presents an in-depth analysis of the comparison methods shown in Figure focusing on the differences in approach between FAAP, Fair-Mixup, and ALFA.
4.3.2 ABLATION STUDY
We visualize the impact of the hyperparameter a, which controls the weight of the Sinkhorn distance in the faimess attack. Intuitively, larger helps preserve the original distribution of the perturbed features, maintaining the accuracy of the fine-tuned classifier, as shown Figure 7 (a). On the other hand, small zero may alter the semantic meaning of the perturbed features, potentially impacting accuracy, as also shown in Figure 7 (a). While the relationship is not perfectly linear, smaller values of a generally improve fairness, introducing trade-off between fairness and accuracy. This suggests that the Sinkhorn distance helps balance the two objectives, as shown in both subfigures of Figure 7
CelebA Attractive Classficiation with Swin Transformer 4ce
CelebA Smile C assiiciaaion with ResNet-50
9:95
1.00
Accuracy
Accuracy
6.94
3.05
Accuracy
3.5
Accuracy
.0250
ΛEOd
AEOd
-0200
0
1e:3
1e.2
Te:1
1e+1
16:3
1e.2
1e:1
1e+1
Figure 7: Ablation study varying from 0 to 10 for the CelebA dataset across different tasks and backbone networks. While not entirely consistent, smaller a generally improves fairness but may negatively impact accuracy compared to larger

---

Published as a conference paper at ICLR 2025
Roundarry
Decision Boundary COMIAS Accurscy: C. 0888, EDd: 0.4601
pre-trained 0,8527. E0d:0.3188
trained D.S. Acc. 0525: EDd: 0,3420
With fine tuned D.B. Act: 0 68117, EDd 0.1349
With fine tuned D.B. Azc: 0,6814, EDd: 0.2218
Yot
Corrected Dectzon Bour dary ta 1.01
Corrected Dectson Boundary la- 0.51 are traited 0.6333, 50d:
Corrected Decisoo Boundary (a
With A 0.5943 tne tured funee
pre D.S. Acc.: 1.1682
D.E. 0.6805, EDd:
0.1094
With tuned 0.6760, EDd:
Figure 8: t-SNE plots for the COMPAS dataset. The black line represents the pre-trained decision boundary, while the red line represents the newly trained decision boundary on the combined dataset, where equal weighting is applied to the original and each perturbed dataset.
4.3.3 VISUALIZATION OF DECISION BOUNDARY CORRECTION WITH REAL-WORLD DATASET
For deeper analysis, we provide t-SNE plots for the COMPAS dataset, including the original dataset and the perturbed datasets (with reveals that under the pre-trained decision boundary, the perturbed samples exhibit extremely high AEOd, indicating the success of our fairness attack. However, fine-tuning the concatenated dataset results in a corrected decision boundary (represented by the red line) that maintains accuracy while achieving significant improvements in fairness.
0 and 1, respectively). The visualization in Figure B
Moreover, the effect of a aligns with our intuition. A higher a retains the original distribution more closely, resulting in less corrected decision boundary and less pronounced fairness attack. Nevertheless, both cases (a = 0 and a 1) demonstrate significant improvements fairness after fine-tuning.
CONCLUSION
In this research, we address the critical issue of fairness in machine learning models, specifically focusing on biases caused by demographic data imbalances. We propose novel method, Adversarial Latent Feature Augmentation (ALFA), to effectively identify and mitigate unfairness in classification models, promoting more equitable decision-make making. ALFA generates biased perturbed features using faimess attack based fairness constraint. Fine-tuning the classifier these biased samples reduces discrepancies in misclassification rates across different demographic groups. We provide theoretical proof of our claims, and our method is validated through extensive experiments range of datasets, modalities, and backbone networks. ALFA consistently achieves group fairness without compromising accuracy, demonstrating its effectiveness in promoting unbiased machine learning models. As future work, we aim to explore integrating individual fairness metrics and extending ALFA to larger, complex datasets assess its scalability and broader applicability.
wide
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was partially supported by the EMBRIO Institute, contract #2120200, a National Science Foundation (NSF) Biology Integration Institute, Purdue's Elmore ECE Emerging Frontiers Center, and NSF ISS #1955890, IIS #2146091, IIS #2345235.
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214-223. PMLR, 2017.
Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanic Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang. AI Faimesss 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias, October 2018. URL https
7/arxiv. wog.orgg/aao..... abs /aop///8800 18100. 1943
Francesco Bodria, Riccardo Guidotti, Fosca Giannotti, and Dino Pedreschi. Interpretable latent space to enable counterfactual explanations. International Conference on Discovery Science, pp. 525- Springer, 2022.
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pp. 77-91. PMLR, 2018.
Junyi Chai and Xiaoqian Wang. Fairness with adaptive weights. In International Conference Machine Learning, pp. 2853-2866. PMLR, 2022.
Anshuman Chhabra, Peizhao Li, Prasant Mohapatra, and Hongfu Liu. Robust fair clustering: A novel faireess attack and defense framework. arXiv preprint arXiv:2210.01953, 2022.
Christophe Denis, Romuald Elie, Mohamed Hebiri, and François Hu. Fairness guarantee multi-class classification. arXiv preprint 1.XXi:2109.13644 2021.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint
2018.
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1tt::2000.119200 19999, 2020.
Dheeru Dua, Casey Graff, et al. Uci machine learning repository. 2017.
James R Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. An intersectional definition of fairness. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp. 1918-1921.
IEEE, 2020.
Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning generative models with sinkhorn di- vergences. In International Conference on Artificial Intelligence and Statistics, pp. 1608-1617. PMLR, 2018.
Yury Gorishniy. Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems, 14:189322-8894 2021.
Riccardo Guidotti, Anna Monreale, Stan Matwin, and Dino Pedreschi. Black box explanation by learning image exemplars in the latent feature space. In Machine Learning and Knowledge Discov- ery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16-20, 2019, Proceedings, Part I, pp. 189-205. Springer, 2020.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
11

---

Published as a conference paper at ICLR 2025
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
1735-1780, 1997.
Chia-Yo Yi Hsu, Pin-Y Yu Chen, Songtao Lu, Sijia Liu, and Chia-Mu Yu. Adversarial examples can be effective data augmentation for unsupervised machine learning. In AAAI Conference Artificial Intelligence, 2022.
Taeuk Jang, Feng Zheng, and Xiaoqian Wang. Constructing a fair classifier with generated fair data. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 7908-7916, 2021.
Taeuk Jang, Xiaoqian Wang, and Heng Huang. Adversarial fairness network. In Proceedings the AAAI Conference on Artificial Intelligence, volume 38, pp. 22159-22166 2024.
Lauren Kirchner Jeff Larson, Surya Mattu and Julia Angwin, How we analyzed the compas recidivism algorithm. 2016.
Sangwon Jung, Taccon Park, Sanghyuk Chun, and Taesup Moon. Re-weighting based group fairness regularization via classwise robust optimization. arXiv preprint arXiv:2303.004443 2023.
Brendan F Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, and Anil K Jain. Pushing the frontiers of unconstrained face detection and recogmition: larpa janus benchmark a. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1931-1939, 2015.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1895-1889. PMLR, 2017.
Pang Wei Koh, Jacob Steinhardt, and Percy Liang. Stronger data poisoning attacks break data 2018.
sanitization defenses. arXiv preprint
Peizhao Li and Hongfu Liu. Achieving faireess no utility cost via data reweighing with influence. In International Conference on Machine Learning, pp. 12917-12930. PMLR, 2022.
Peizhao Li, Ethan Xia, and Hongfu Liu. Learning antidote data to individual unfairness. In International Conference Machine Learning, pp. 20168-20181. PMLR, 2023.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision, pp.
2021.
Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(2018 (10018):11, 2018.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference Machine Learning, pp. 2384-3393, PMLR, 2018.
Yuzhen Mao, Zhun Deng, Huaxiu Yao, Ting Ye, Kenji Kawaguchi, and James Zou. Last-layer fairness fine-tuning is simple and effective for neural networks. arXiv preprint arXf::2304.03935, 2023.
Ninarch Mchrabi, Muhammad Naveed, Fred Morstatter, and Aram Galstyan. Exacerbating algorith- mie bias through fairness attacks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 8930-8938, 2021.
Youssef Mrouch et al. Fair mixup: Fairness via interpolation. In International Conference on Learning Representations, 2021.
Amirarsalan Rajabi and Ozlem Ozmen Garibay. Tabfairgan: Fair tabular data generation with generative adversarial networks. Machine Learning and Knowledge Extraction, 4(2):488-501 2022.
12

---

Published as a conference paper at ICLR 2025
V Sanh. Distilbert, distilled version of bert: Smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01100 108, 2019.
David Solans, Battista Biggio, and Carlos Castillo. Poisoning attacks on algorithmic fairness. In Joint European Conference on Machine Learning and Knowledge Discovery in Dutabases, pp. 162-177. Springer, 2020.
Rui Sun, Fengwei Zhou, Zhenhua Dong, Chuanlong Xie, Lanqing Hong, Jiawei Li, Rui Zhang, Zhen Li, and Zhenguo Li. Fair-cda: Continuous and directional augmentation for group faimess. arXiv preprint arXiv:2304.0022 2023.
Nithum Thain, Lucas Dixon, and Ellery Wulezyn. Wikipedia Talk Labels: Toxicity. 2 2017. doi:
10.6084/m9 http://99..gghaaa.4455....
Zhibo Wang. Xiaowei Dong, Henry Xue, Zhifei Zhang. Weifeng Chiu, Tao Wei, and Kui Ren. Fairness-aware adversarial perturbation towards bias mitigation for deployed deep models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10379-10388, 2022.
Yongkai Wu, Lu Zhang, and Xintao Wu. On convexity and bounds of fairness-aware classification. In The World Wide Web Conference, pp. 3366-3362, 2019.
Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust to be fair: Towards fairness in adversarial training. In International conference on machine learning, pp. 11492-11500 15501. PMLR, 2021.
Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair ml models with sensitive subspace robustness. arXiv preprint orXiv:1907.00022 2019.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pp. 962-970.
PMLR, 2017.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint 17000:17 2017.
Long Zhao, Ting Liu, Xi Peng, and Dimitris Metaxas. Maximum-entropy adversarial data augmen- tation for improved generalization and robustness. Advances Neural Information Processing Systems, 33:14435-1444 2020.
13

---

Published as a conference paper at ICLR 2025
A PROOF FOR THEOREM 3.2
Consider a linear classifier g(z) = sign(w"= b), where = = f()) represents the latent space generated by an encoder and and b are the classifier's weights and bias. The classifier g(z) is considered biased if its decision boundary w ++b O results in significantly different outcomes (e.g. higher misclassification rates) for the groups defined by the sensitive attribute A E {0. We define the unfair region Ramjair as the subspace of the latent space where the classifier's 's decision boundary results in disproportionately high differences error rates between these groups:
Runfair = {= Rd P(((z)) - y A 1)) P(g(z) y 0]] E0,
(7)
where T a threshold indicating significant bias.
Retraining the classifier using Eq. results in a fairer classifier by reducing AEOd:
( & AEOd(H).
(8)
where 0 and 0g denote the classifier's parameter when trained on original dataset and combined dataset, respectively.
Proof. When the classifier exhibits bias, the decision boundary poorly separates classes within Runfair- To address this, we retrain the classifier by focusing on the unfair region with the generated samples by faimess attack, minimizing a new loss function:
L(w, b)
Loolyis ((zi))+
E Loa Lea(((s9((z)))
N
jERune
where D represents the original dataset, Loe is the cross-entropy loss, and N' is the size of the combined dataset. Eq. the gradient of L is more heavily influenced by the samples in Runfair than it would be if the model were trained solely on D. This induces an upweighting effect in the unfair region during retraining, leading an adjustment of the decision boundary. which can be specifically approximated by:
AW V
where y is the learning rate. The adjustment Au, driven by the samples in Runtair reduces the misclassification rates in this region, particularly benefiting the disadvantaged group.
Let's define RW the unfair region for such that
Rv=1 = f= E gd P(g(2) = 1|A = 1)1 - P(g(z) 1/A [|= >T}
As the decision boundary adjusts correct misclassifications within Rantiir the impact is most significant for the group with higher error rates. This leads to a reduction in the False Positive Rate (FPR) within Ra especially if a sensitive group initially exhibits disproportionately high FPR when predictions in RW result in g(2) 1. Consequently, this reduces the FPR gap:
P(g(z) 1|A 1)1 P(g(z) 1|A 0)}.
Similarly, derive the reduction in the TPR gap. When considering the unfair region R in relation to the False Negative Rate (FNR), a sensitive group may exhibit a disproportionately high FNR, resulting in g(2) 0:
P f E Rd :P(((=))))))))))))))))))))))): = 0 1) P(g(2) = 0 r}
high FNR gap is problematic because it implies higher gap in True Positive Rate (TPR), given that TPR, = 1 - FNR., (PRR, - TPRi = (1 F.) - (1 FNN = |FN R1 - FN F. Therefore, the unfair region R becomes a critical target area to address for fairness. Similar to RV=1 for FPR gap, retraining that focuses Ry=D also leads reduction in the TPR gap.
Therefore, after retraining on samples in py=1 and Ry=0 the following inequalities hold:
PRo FPR Iretrain S PPRo FPR1 origin
14

---

Published as a conference paper at ICLR 2025
and
TPRp - -PPR1 (TPR TPR1 origin.
Consequently, retraining on samples within the unfair regions identified by the faireess attack leads reduction in both the FPR and TPR disparities across demographic groups.
As the Equalized Odds gap A is defined as the sum of the TPR and FPR gaps:
AoOd = [TPR] - TPRol + [FPR] FPRol.
retraining on samples from the unfair regions ensures fairer classifier by minimizing these gaps:
(FOd((,)) S AEOA()).
(10)
where 0 and Op denote the classifier's parameter when trained on original dataset and combined dataset, respectively. This strategy not only addresses the existing bias but also actively improves the fairness the model a measurable and theoretically grounded manner.
B PROPERTIES
B. PROPORTIONALITY OF CEAIR
Proposition B.1. Lioir is proportional to the mean signed distance gap (Adáp) between two sensitive attribute groups, and the sum the mean signed distance gap (Adeod between the sensitive groups for each ground truth label y E {0,11 10,1}.
Cjai 4 Addp
and Adead,1 + Adeedd.
8
where Addp
L Eies, di 1 Eieso d3
Adeod,
Mi1 EiESh di 1 nos
Ejeso ds}.
ni
no
Adcod.0
R10 EiEsio d₃ 1 Ejesoo dz}
non
Proof. Let d, = g()), d is the mean of all de and
Adap
74
iES,
jESo
1
Adeod,1
dj
1111
R01
iE81
JESo1
Adeod.0
d:
dj
1/10
T100
SES10
JES00
where S, is subset containing each sensitive attributes S. {ila: = a). E 10. and Tay means the number samples each sensitive subset for given y, Sag {ila; = a, y: = y). {0, 1]. In experiments, select samples with the same size such that R10 = M11, and N
{0, 1} and N =
where no = + noi, nn nno + 711 and
N n100 + noi + 1i00 m11-
The objective function Lrir = |Cov(a, ((2)) can be rewritten as
Lii
1
Σ
a, a) (/dd--0 -0-) a)
N
i=1
1
Σ
(1- a)(d: -d) +
Σ (0-a) d)
N
iES
jESo
ng Σ (d; d) n1 Σ (dj a)
2
SESp
15

---

Published as a conference paper at ICLR 2025
N
1gg
Σ
dj
dj
non d + nond
iESt
jE8o
1
d
N
71
n4
iE
jESo
1
Addp.
(11)
Similarly, we can conditionize Ltaie in terms of y,
1
(ddd+ -ayl
[a=
N
i=1
1
Σ (1 a) d)+ (0-aa E (1-aaee dd) (0-
N
rESm
SES01
iES10
JES00
(no1 + noo)
(n11 + 120)
(dl -d)
(dg - d)
N
iESm
JESo
(no1 + nuu))
(n11 + muv) (dg - d))
Σ
(di (dd Σ
+
N
N
IES
JE
M11(no1 + n1o))
1
Roi (1 +
n11)) 1
Σ dj
N
711
N
no1
VESt1
JESu1
(no1 + +o0)
1
Too0 (111 + n10)
1
+
d:
dz
N2
1110
N2
1400
RES10
S
=
111
N2
711
d2
R01
d
+
7110
d2
7100
Σ
d,
1E.111
jES81
iES10
JE800
Adeod, + Adeoad
(12)
B.2 BOUNDEDNESS OF LEAIR
Theorem B.2. If the cardinalities of subgroups Say {ila; a, = y}. a (0. (0.
are equal, Cjarr is the lower bound of ADP and AEOd when approximate the logit (inverse sigmoid) function as a piecewise linear function with m segments s.t = fx( = ag + by for k E (1,2,-.. .m).. >> 1, m E N. and amaz max(a1,".. ,ag). Then,
1 (amax ADP+
Cjair S
1/4
(13)
Cjair
S 8
(ome + Co + Ci).
(14)
C= (k)
(1) Tb, and C.
k- (n) ngg bb, constants where (k) is the
(k) (k)
number of samples in k-th segment for E {0, 1}. We set B < ú < 1-8 when we compute the signed
distance to avoid dg In( in(1n)) + (co or -c0) so that 0max 1-20 1,git(,3 + 1-23 (ogit(()) thcoretically. In this work, we set 3 1e-7 and
m
10 for all experiments.
Proof. As we fix the sign of Cair following the sign of Coo(a, y), the sign of Adap and ADP are particularly defined
Ad
1
Σ d:
Σ d
11
170
VES,
JeSo
16

---

Published as a conference paper at ICLR 2025
1
dj
dj
T11
T1
iES1
JESo
ADP
1
Σ
1/1
1
Σ
710
16
jeSO
1
1
Σ
91
Σ
90
11
TA
jESo
when we assume that Cov(a, y) is positive. In the negative case, the sign of Adap and ADP will be changed simultaneously.
we assume the logit function as a piecewise linear function with m segments s.t m 1,m E N. and recall that ddp no n1 and N noo0 Ro1 n10 A11- Let each linear function is = (ū) = a&ÿ + bk. 1,2, ... m. Then the Addp and ADP becomes
Addp
Σ
Σ dj
Th
7O
iES:
JESO
(an0i + bbb)
Σ
(ak0s + bz))
.21
no
iES(N)
jESH
di Σ ún
Tldp
kml
(E)(
JES!
m
(K)
[n1
Tuo
Tldp
Omax
Σ
0p)
+
Tldp
k= (E
4max
+C
Tldp
#ES1
JESo
Gmax
ADP
(15)
and where na means the number of samples in k-th segment for a E (0,1) and C
= 1 Map Σm (n1
(k) bb, is constant. Therefore, maximizing Liai maximizes ADP since
1 Adas - 7 (amax ADP+C).
Lisir
(16)
Similarly, the same proof can be applied to the relationship between Adeod, Adeod, and AEOd as explained Eq. [5]
such that
1
AEOd
+
7111
R01
72:0
T100
vESn
JESo1
/ES10
je
1
Σ
úg)
+
ú))
Read
iES1:
jE8o:
iES10
JESu0
1
Adcod,1
d:
d
Mi1
7001
VESu
fES01
1
d
Σ
d
amax
Σ
81
Σ
+
nead
nead
JESo
iES11
JE
Adeod,0
d:
d
7110
vES
17

---

Published as a conference paper at ICLR 2025
Σ
d
Σ
dj
4max
Σ
Σ
Co
Need
Read
iES10
jESo
iES10
S00
Therefore,
Ljair
Adeod, + Adeod.
Omax
y/
Σ
+
Σ
Σ
0U)
Co
ci]
8 Reod
iESu1
JESo1
IE.110
JE.S00
8 4max
AEOd + Co + Ci|
(17)
where C₂ Em k== (n)
no )b. a E {0,1} are constants.
B.2.1 USAGE OF PIECEWISE LINEAR APPROXIMATION
We empirically verify that the naive logit function feasible as well and effectively attacks the fairness in terms of ADP and AEOd. However, the upper bound of Caarr with the naive logit function is not fully supported mathematically, while the piecewise linear logit function can be proved as Appendix B.2 Moreover, there's no significant difference in the fairness performances between the naive logit function and its piecewise linear approximation. We choose the piecewise linear function ensure the upper bound of Liir-
B.2.2 INSIGHTS
We randomly choose equal number of samples for each subset for effective fairness attack, i.e. N₂ Yoo = Ro1 n110 7h1 satisfy the condition Theorem B.2
Consequently. since Caii is the lower bound of ADP and AEOd, we can attack fairness by maximizing Liir as the perturbed latent features produce unfair prediction with high ADP and AEOd on given pre-trained classifier.
C SYNTHETIC DATASET
We provide the details of the synthetic data, illustrating the concept of the unfair region and how the decision boundary is corrected. We simplify the binary classification task with 2D Gaussian mixture model, as assumed in (Xu et al. 2021),, consisting two classes E (0, 1} and two sensitive attributes A {0, 1} (indicating unprivileged and privileged groups).
groupl N(
.02)
if: 1,a
Cathon IN
02)
if: 0,a
(18)
group3 N
(Ko)²)
if: 1,0=0
group4: N(
(Ko)²)
y 0
where μl rp, r < 1 and K > 1, where the number of samples in each group is N1 N₂ N3 .4. We arbitrarily set K = 3, r 0.7. 1, Ni = N₂. 100, and N3- = N₂ = 400. From the synthetic data, observe a decision boundary like Figure 2 (a) in the paper. Due to dataset
imbalance, the subgroup a = 1,y = 0 is overestimated as label 1, and the subgroup a = 0, =
underestimated label 0. The disparity in misclassification rates is depicted in Figure (c). We define these disparities as 'unfair regions' where the misclassification rate is disproportionately high.
18

---

Published as a conference paper at ICLR 2025
D SINKHORN DISTANCE
D.1 DEFINITION
Optimal transport with lowest cost defined as Lo min Ei, CypPu where C cost matrix
(2- Wasserstein Distance), and P is the coupling matrix. Genevay al. (2018) suggested regularized optimal transport scheme which includes entropy term to secure stability and smoothness of P. Lc min Eij Cit Pij C, (P) where H(P) Eij P.y log P.y. Le can be solved by Sinkhom iteration, s.t. P.j- = diag(u, Kiddiag( uz), and updated alternately,
u (k+1) = a
Kv(k)
(k+1)
K (1=+1)
where P1 a, P and Gibbs kernel K;j s0/0, Therefore, the distance between clean data a and perturbed data o can be rewritten as follows,
D अ) = Sinkhorn Ristance(xx.2
min (, P(z i.j
D.2 COMPUTATIONAL. COST OF SINKHORN DISTANCE
The Sinkhorn distance involves solving regularized optimal transport problem using the Sinkhorn- Knopp algorithm, which benefits from linear convergence rates and efficient GPU execution. This makes the computation the Sinkhorn distance tractable even for large-scale problems.
This is further supported by the non-significant empirical time cost observed in ALFA. We report the attack times for each dataset. Each result represents the mean and standard deviation across 10 runs. Notably, the attack time is proportional to the dataset size, independent of the data type (e.g.. images), as the attack occurs in the latent space rather than the input space. Note that this attack step conducted only once before fine-tuning, enhancing the effectiveness of framework.
Table 1: Time cost for one-time attacking step
Dataset
Number of Samples
Feature Dimension
Attacking Times (s)
Adult COMPAS
271,320
512
15.9613=0.3242
43,280
512
2.6102+:0. 1050
German
6,000
512
0.2692++0.111
Drug
11,280
512
0.6949
CelebA
162,748
768
115.5459-22.313
Wikipedia
342,528
384
377.0901=75.1673
19

---

Published as a conference paper at ICLR 2025
E COVARIANCE BETWEEN THE LABEL AND THE SENSITIVE ATTRIBUTE
Table The estimated value of Cov(a, y) and Cov(a, i). We set the sign of Lfair the same as covariance.
Cov(a,
Cov(a, 9)
Cair
Adult
0.0439
0.0441
(ov(a,g())
COMPAS
-0.0198
-0.0194
-Covla, g(2))
German
0.0210
0.0188
Cov(a, g()))
Drug
0.0434
0.0401
Cov(a, g(2))
F HYPERPARAMETERS
Table 3: Hyperparameters for the experiments for ALEA
Hyperparameter
Search-range
[0.0, 0.001,000001,11 1.0, 10]
Total Epoch T Attacking Iteration Batch Size
50
10
128
G EXPERIMENTAL SETTINGS FOR FAIR COMPARISON
Fair-Mixup. Fair-Mixup is an in-processing data augmentation using interpolation on manifold between two sensitive groups. Smooth regularizers for linear interpolation on DP and EOd are as follows
P = -f.1f(Va (Vaf(to(o)) + ()-t))(z))), - (( d (2o (d (z1) (z)
Σ [1 (11 21)dP (No) )PP (x1) dt.
REOd
VE (E(1)
where g x + a feature encoder. The final objective function of Fair Mixup is
Lmirup Lacc + X&misuu (f).
For fair comparison, we vary the ratio of regularizer adjusting À E (.31, 0.5, 0.73 for tabular datasets and X 20 for CelebA dataset as suggested in the released implementation.
TabFairGAN. It aims to produce high-quality tabular data containing the same joint distribution as the original dataset using Wasserstein GAN. The training algorithm in (Rajabi & Garibay 2022) consists of two phases, training for accuracy (phase 1) and training for both accuracy and fairness (phase 2). In both phases, the loss function for critics C' adopts gradient penalty (Gulrajani et al. 2017).
Ve =B=a,, [C(#) - (C(x), + A.Ee (()= - 1)21
The loss function for the generator differs from each phase.
Ve Eg-P, [C(+)
(phase 1)
VG Ex. ".a a- a- C(e y. a)] AB lūjà 0] Ba.j.a Po P 1))
(phase 2)
where Xy is hyperparameter. We excute TabFairGAN with various X, E (0.1, 0.3, 0.5, 0.7) for fair comparison since the implementation uses X, 0.5.
20

---

Published as a conference paper at ICLR 2025
FAAP FAAP aims to generate perturbation using GANs, while the generator makes perturbation and the discriminator predicts the perturbed features' sensitive attributes. formula,
CD ())()((() - CJuir -
C - (oooo((((()))))
where G a generator, D is discriminator, is the perturbed samples, H is the entropy, is label predictor (classifier), and is an encoder. The final formulation becomes
arg max min Lce (D(g(E)), + aH((Dgg()) 3C2
As the architectures for the generator and discriminator not provided, we set generator as an MLP model with two hidden layers with 128 nodes, having a ReLU activation function. For the discriminator, we adopt the same network with the label predictor each tabular dataset and image dataset. For the fair comparison, we grid search and & by {0.1, 1.0, 10) and {0.1, 1.0, 10} since the value not given the original paper.
where - -
FDR FDR is simple fine-tuning method, including balanced sampling in the latent features, and use fairness constraint as a objective function. In detail, the Equalized-od wawdliized-ddd-baaa-a-aa fairness constraint is
fpr Σ, P.P.11 v1aa: E.pi(1 y1) a:)
Σ,
11 - -a;)
for E. Pi)yaa:
E1 P: (1 a;) (1(1-a) a,)
Σ, as
where p; denotes the predicted probability. The final objective function for the fine-tuning is
min (o) a(fpr + fnr)
As suggested the original paper, we search a {0.5, 5,
Fair-CDA Fair-CDA aims to disentangle latent features into 'sensitive feature" and 'non-sensitive feature', and obfuscate the sensitive features to obtain fairer classifier. Fair-CDA consists of three extractor, h, fyy. and ha
= h(xx),,, (2.), = (z;)))
hy should extract features only related to the label predictions, while ha is related to the sensitive attribute only. The regularization becomes
3(C C + C
and
C ( V))
C. Loo (zf²) ai)
C (V C. V2, C9)
IV 2,
where By and Ia are two classifier for y and respectively. In stage 1 for the first 450 epochs, the objective function
Σ C. C + C.).
where L. = Lee(o((>), =f1): yi). For stage 2, Fair-CDA conducts semantic augmentation make the sensitive features along the direction to increase the attribute loss
2 Lee (za a.)
En = 2 + 0
Loe Og (2 ). a.)l
21

---

Published as a conference paper at ICLR 2025
Based on the obtained 2ạ and the solution of the task model in stage 1; 9. obtain two loss functions for stage 2 for 50 epochs,
?, Lee (o[[=) (. 2(1), vi)
Ć foe (g((=)) =1 [(=], 2;1))
Then, the final objective function for stage becomes
+4 + (1- 3(C + + CF).
72 =1
Fair-CDA requires five hyperparameters, perturbation size os randomly drawn by (O,, A) where E {0, 1, 10, 100, 1000). = 0.9 as written in the paper, and B is the initial loss value. As the learning rate for stages and 2 are not given, we grid search 7/, 72 {0.0001, 0.001, 0.01} as well as A.
LAFTR LAFTR Madras et al. 2018) includes a classifier model named adversary, aiming to predict the sensitive attribute, while an encoder wants to deceive the adversary. also an adversarial training, but is different from ours as LAFTR does not involve perturbation, data augmentation, or fairness attack. Because of the architecture of LAFTR, only applicable to MLP baseline.
H DETAILS IN EXPERIMENTAL RESULTS
In this appendix, we quantitatively demonstrate the superiority of ALFA across datasets and backhone networks. We also include results for a ResNet-like architecture applied to tabular datasets (Gorishny et al., 202I) to show versability of ALFA. The results compare accuracy. ADP, and AEOd with other fairness approaches. For the tabular and text datasets, the mean and standard deviation from 10 experiments are reported. In each table, Blue indicates the best result for each dataset, and Cyan represents the second and third best results. The results for each method obtained by varying the hyperparameters, and for each method, we report the result corresponding to the point closest the upper right in Figures 5 and 6. The findings show that ALFA consistently ranks as either the best or second-best in terms of EOd across all comparison methods, without compromising accuracy- distinction not achieved by any other approach. This highlights ALFA's superiority over the alternatives.
H.1 EXPERIMENTAL RESULTS OF TABULAR DATASETS
Table 4: Experimental Results for Adult dataset with Logistic Regression
Logistic Regression
Adult
Accuracy
ADP
AEOd
Baseline
0.8470=1.0007
0.182940.0020
0.1982:0.0077
Influence-
0.8359+000000
0.1815
0.119010.0000
Covariance-Looss
0.8427+40.0008
0.1174+0.0042
0.1001+0.0065
Lonve-Cooncave-Loss
0.8296+t0.0056
0.1194+000016
0.0687+0.0235
Fair-DRO
0.8012++0.0033
0.2123++0.029
0.1261
Fair-Mixup
0.8376+10.0222
0.00115
0.0063+0.0103
FAAP
0.8461+0.0012
0.1822+0.0116
0.193710.0235
FDR
0.8465=10.0077
0.1797+0.0041
0.1685+0.0190
TabFairGAN
0.8327+000007
0.1488+00.003
0.1112+0.0117
Fair-CDA
00.0005
0.183940.0022
0.2013+0.0054
Ours (ALFA)
0.8064+0.0004
0.1555+0.0013
0.0616+0.0022
22

---

Published as a conference paper at ICLR 2025
Table 5: Experimental Results for Adult dataset with MLP
MLP
Adult
Accuracy
ADP
AEOd
Baseline
0.8525++00010
0.182440.0114
0.1761+0.0411
LAFTR
0.8470=+0.0020
0.1497+0.0191
0.1117#0.0443
lnfluet
0,8470=10.0020
0.1497+0.0111
0.111710.044
Covariance-Loss
0.8018::0.015
0..069
0.111410.0454
Convex-Concvve-Loss
0.8455=10.0024
0.1219+0.0272
0.105010.0366
Fair-DRO
0.0011
0.1795
0.104410.0077
Fair-Mixup
0.8516
0.1515
0.0912#0.0399
FAAP
0.0024
0.1695
0.1416+0.0432
FDR
0.8253+000001
0.1830+0.0002
0.1677+0.0010
TabFairGAN
0.8516=10.0022
0.1752
0.167410.0392
Fair-CDA
0.0015=00.0012
0.1786+0.0055
0.1711
Ours (ALFA)
0.8244+0.0150
0.1012+00.083
0.0660+0.0434
Table 6: Experimental Results for Adult dataset with ResNet-like network
ResNet-like ADP
Adult
Accuracy
AEOd
Baseline
0.8565+10.0022
0.1800+400005
0.102510.0341
Covariance-Loss
0.8557:10.015
0.183410.0127
0.1916:0.0483
Convex-Conee Concave--oss
0.8305=#0.0029
0.1064
0.1155
Fair-DRO
0.6627+000048
0.0154
0.2991
Fair-Mixup
0.1881
0.1888+ 0.0399
FAAP
0.8558 0.002
0.17241000131
0.1631
FDR
0.8472+0.0001
0.1930+0.0002
0.1765
TabFairGAN
0.8544+0.00007
0.1699
0.1317#0.0316
Fair-CDA
0.8529:10.0012
0.1021
0.1714+0.0100
Ours (ALFA)
0.8262+0.0014
0.1956+000006
0.1304
Table 7: Experimental Results for COMPAS dataset with Logistic Regression
Logistic Regression ADP
COMPAS
Accuracy
AOOd
Baseline
0.6578+0.0034
0.2732+0.0129
0.5319+0.0245
Seltuence-Reweighhe
0.6791+10.000
0.287410.0000
0.5374
Covariance-Looss
0.6065+000036
0.0243++0.010
0.0671+
0.6462-+000050
0.019410.0118
0.0825+0.0192
Fair-DRO
0.6427-0.0155
0.098940.0576
0.210610.1308
Fair-Mixup
0.6352+0.0109
0.0536+400335
0.0911+0.0576
FAAP
0.6577+000033
0.2670+00.022
0.5184+0.0464
FDR
0.6677-10.0039
0.0377+000464
0.1001
TabFairGAN
0..000-+..0100
0.1006
0.3005
Fair-CDA
0.0084-0.00199
0.1629+00.099
0.
Ours (ALFA)
0.669440.0036
0.019340.0156
0.087610.0354
23

---

Published as a conference paper at ICLR 2025
Table 8: Experimental Results for COMPAS dataset with MLP
MLP
COMPAS
Accuracy
ADP
AEOd
Baseline
0.6711+0.0044
0.2059+0.0277
0.3699
LAFTR
0.6397=#0.0284
0.1164+0.0183
0.2089+0.0252
lnfleet
0.6424-10.0000
0.1513+0.0000
0.2810+0.0000
Covariance-Loss
0.6729:10.0118
0.0820:10.0555
0.1121
Convex-Concave-Loss
0.6631=000004
0.0641+0.0261
0.102610.0566
Fair-DRO
0.6782+1000066
0.0457+000222
0.0690=0.0136
Fair-Mixup
0.6661
0.0634+0.0331
0.0978=0.0537
FAAP
0.6741=0.0066
0,3495+0.1144 0.1976++0.011
FDR
0.6550+000010
0.0771+0.0027
0.1104+0.0055
TabFairGAN
0.6767=10.0019
0.1616+1000339
0.2726=000727
Fair-CDA
0.5885=00.0155
0.1088+0.0291
0.2131
Ours (ALFA)
0.6002++00021
0.0204+0.0151
0.0410#0.0188
Table 9: Experimental Results for COMPAS dataset with ResNet-like network
ResNet-like ADP
COMPAS
Accuracy
AEOd
Baseline
0.6753:10.0337
0.2055+10.0307
0.368010.0700
Covariance-Loss
0.6793+10.003
0.0957+0.0342
0.1349=0.0599
Convex Niwex-Concave-Loss
0.6671
0.0856+10.0707
0.1795#0.1099
Fair-DRO
0.6099
0.0376++0.062
0.0546 0.0236
Fair-Mixup
0.6729
0.3752 0.2065+0.0355
FAAP
0.6718+10.003
0,2020:10.0254
0.3626=10.056
FDR
0.6725+0.0006
0.1288
0.198410.0034
TabFairGAN
0.6769+1000222
0..051
0.3046+0.0872
Fair-CDA
0.6001=000006
0.1850+0.0139
0.324410.0306
Ours (ALFA)
0.6756=10.0322
0.0124+0.0137
0.0659#0.0316
Table 10: Experimental Results for German dataset with Logistic Regression
Logistic Regression ADP
German
Accuracy
AEO
Baseline
0.7220+1000311
0.1186+0.0642
0.3382 12668
Influence- Reweigh
0.6050+L0.0000
0.0665+10.000
0.2475
Covariance-LLoss
0.7410+400018
0.0758+0.0453
0.1247
Cowvex-Concave-Loss
0.7625-10.044
0.0590110.373
0.0979+0.0058
Fair-DRO
0.6005+0.0023
0.0627+0.0401
0.141910.0610
Fair-Mixup
0.0085
0.0778+00.014
0.1694+0.0533
FAAP
0.7295+0.0250
0.1128+00.0100
0.3083+0.1518
FDR
0.7640-L0.0044
0.0398
0.2382+1.0523
TabFairGAN
0.7460
0.067740.0341
0.2762
Fair-CDA
0.7115-L0.0211
0.0662+00.032
0.2191
Ours (ALFA)
0.7940=10.008
0.047010.0199
0.046910.0276
24

---

Published as a conference paper at ICLR 2025
Table 11: Experimental Results for German dataset with MLP
MLP
German
Accuracy
ADP
AEOd
Baseline
0.7800++0.0150
0.0454+0.0282
0.2096+000924
LAFTR
0.7308=+000270
0.0419+000410
0.1677#0.1433
lnfleet
0,8000=10.0000
0.0297+000000
0.134710.0000
Covariance-Loss
0.7000
0.0088
0.1175
Convex-Concvve-Loss
0.7570=1..0100
0.0318+0.0217
0.0793+0.0459
Fair-DRO
0.7005
0.0622
0.1190#0.0705
Fair-Mixup
0.0055
0.1378 0.004350.0235
FAAP
0.7785++0.014
0.0191+00.022
0.1788 +000638
FDR
0.7650+0.0000
0.0000+000000
0.2036
TabFairGAN
0.7015=10.0123
0.0021
0.1799
Fair-CDA
0.7325=100288
0.0408+0.0253
0.1651
Ours (ALFA)
0.7570=10.024
0.0053+00.064
0.0813+0.0110
Table 12: Experimental Results for German dataset with ResNet-llie network
ResNet-like ADP
German
Accuracy
AEOd
Baseline
0.7595-10.0224
0.0524:10.0668
0.2193:10.122
Covariance-Loss
0.7605+00.019
0.0360+0.0263
0.108610.0479
Convex-Conee Concave--oss
0.7500=1.0020
0.0493+0.0271
0.1076#0.0601
Fair-DRO
0.7475++0.021
0.0629+0.0248
0.1015+0.0432
Fair-Mixup
0.7280+0.0188
0.0693+000366
0.1907+0.0719
FAAP
0.7030
0.0459:10.038
0.
FDR
0.7190+0.0000
0.122411.0007
0.3487
TabFairGAN
0..035
0.0407+0.0247
0.2278+0.0527
Fair-CDA
0.7095-0.0019
0.0457+0.0288
0.1858#0.0958
Ours (ALFA)
0.7325+0.0186
0.0309+0.0147
0.0665+0.0228
Table 13: Experimental Results for Drug dataset with Logistic Regression
Logistic Regression ADP
Drug
Accuracy
A AOOd
Baseline
0.6626+00.0135
0.2938+0.0761
0.506410.1616
Selteence-Reweigeee
0.64446:10.0000
0.1245
0.1094
Covariance-Loos
0.6491++00008
0.0736+000036
0.2060+0.0549
Cowvex--oncave-Loss
0.6225-10.0338
0.0081
0.1429
Fair-DRO
0.6403=10.0091
0,0710+:0.04A
0.178910.0515
Fair-Mixup
0.6533+10.0077
0.0979+40.0483
0.1787+0.0793
FAAP
0.6029-00.0117
0.3220+000086
0.5576+000998
FDR
0.6599-10.0011
0.3008+000012
0.539710.0268
TabFairGAN
0..013
0.2796+40.027
1.666840.111
Fair-CDA
0.6015-0.0021
0.3085
0.501710.0156
Ours (ALFA)
0.655440.0067
0.090940.0261
0.1170::0.025
25

---

Published as a conference paper at ICLR 2025
Table 14: Experimental Results for Drug dataset with MLP
MLP ADP
Drug
Accuracy
AEOd
Baseline
0.66741000006
0.2760#0.0415
0..718-000838
LAFTR
0.6195=0.0352
0.1848
0.3235#0.1715
Influence-Rene Reweight
0.6525=10.000
0.1610+0.0000
0.2362+100000
Covariance-LLoss
0.6488+000009
0.0095+000331
0.1410+0.0354
Convex-
Manee--Concave-Loss
0.6467=000004
0.0529++0.094
0.1040#0.0251
Fair-DRO
0.6528:10.0112
0.0841
0.119810.0216
Fair-Mixup
0.6499:1000126
0.07231000318
0.11681:0.039
FAAP
0.6732=t0.009
0.2792++0.0410
0.4707+0.0860
FDR
0.6366=1000000
0.1296+000000
0.1956#0.0000
TabFairGAN
0.6828=10.0122
0.2132++0.025
0.3258#0.0421
Fair-CDA
0.0069
0.1250++0.095
0.2608 1..1844
Ours (ALFA)
0.6050+
0.0511 +0.0366
0.0640+0.0528
Table 15: Experimental Results for Drug dataset with ResNet-like network
ResNet-like ADP
Drug
Accuracy
AEOd
Baseline
0.654110.0150
0.255640.00116
0.436610.0038
Covariance-Loss
0.64674000089
0.081740.0474
0.123310.0847
Convex-Comee Consave-Loss
0.6491=1.0114
0.0733+00.084
0.1258
Fair-DRO
0.6281
0.082940.0228
0.146010.0267
Fair-Mixup
0.6363-10.0063
0.0006
0.1785
FAAP
0.6637:10.022
0.2186+10.037
0.3577+0.0792
FDR
0.6313+0.0000
0.2146++0.0000
0.3625
TabFairGAN
0.6759-10.0100
0.2143+000400
0.3358+00.044
Fair-CDA
0.6655+000006
0.231140.0261
0.3749
Ours (ALFA)
0.6557+0.0132
0.0763-40.0419
0.0066+000053
26

---

Published as a conference paper at ICLR 2025
H.2 EXPERIMENTAL RESULTS OF IMAGE DATASET
Table 16:
Experimental results for CelebA dataset with various backbones. The best results are marked in Blue, and the second best results marked by Cyan.
CelebA Attractive
ResNet-50 ADP
VIT
Twin-Transformer ADP
Accuracy
ABOd
Accuracy
ADP
AEO
Accuracy
AEOd
Baseline
0.8196
0.4374
0.4457
0.8208
0.4675
0.4934
0.8301
0.4692
0.4817
FairMixup
0.8032
0.4035
0,4188
0.8192
0.4536
0.4822
0.8261
0.4526
0.4804
FAAP
0.8098
0,4146
0,4218
0,8210
0.4633
0,4865
0.8287
0.4616
0.4852
FDR
0.8142
0.4269
0.4382
0.8071
0.3983
0.3867
0.8249
0.4452
0.4666
ALFA (Ours)
0.8092
0.4137
0.4209
0.8051
0.3956
0.3777
0.8005
0.3796
0.3698
CelebA Wavy Hair
ResNet-50
VIT
Swin Transformer ADP
Accuracy
ADP
AFOd
Accuracy
ADP
AEOd
Accuracy
A F.Od
Baseline
0.8339
0.3141
0,4112
0.8532
0,391
0.5579
0.8483
0.3391
0.4877
FairMixup
0.8052
0.2486
0.3432
0,8502
0.3521
0.4683
0.8581
0.3712
0.4935
FAAP
0.8352
0.3170
0.4126
0.8528
0.3931
0.5623
0.8478
0.3376
0.4864
FDR
0.8432
0.3353
0.3981
0.8490
0.3367
0.2969
0.8607
0.3779
0.4638
ALFA (Ours)
0.8391
0.3457
0,3419
0.8449
0,3425
0.2809
0,8600
0.3750
0.4022
CelebA
ResNet-50
VIT
Smile
Accuracy
ADP
AEOd
Accuracy
ADP
AEOd
Accuracy
ADP
AEOd
Baseline
0.9275
0.1566
0.0403
0.9239
0.1642
0.0587
0.9340
0.1629
0.0491
FairMixup
0.9281
0.1566
0.0384
0,9219
0.1592
0.0482
0.9338
0.1609
0.0453
FAAP
0,9273
0.1563
0.0400
0.9243
0.1647
0.0602
0.9326
0.1593
0.0427
FDR
0,9276
0.1558
0.0395
0.9234
0.1446
0.0361
0,9286
0.1399
0.0380
ALFA (Ours)
0.9284
0.1509
0.0271
0.9219
0.1552
0.0402
0,9269
0.1234
0.0344
H.3 EXPERIMENTAL RESULTS TEXT DATASET
Table 17: Experimental Results for Wiki dataset with LSTM
LSTM ADP
Wiki
Accuracy
AEO
Base
0,9384=100006
0.176440.0046
0,07791000089
Fair-Mixup FAAP
0.8737=0.0200
0.183140.0618
0.0701
0.9395+0.000
0.1801 +0.0016
0.0807+000063
FDR
0.9256+0.0014
0.2110+0.0022
0.0715++00000
Ours (ALFA)
0.936010.0000
0.197410.0011
0.054810.0009
Table 18: Experimental Results for Wiki dataset with BERT
BERT ADP
Wiki
Accuracy
AEOd
Base
0.938410.0003
0.204240.0041
0.0742+10.0068
Fair-Mixup
0.9106+000041
0.1858
0.0537+00.029
FAAP
0.9512+00.018
0.2128
0.0039+0.0145
FDR
0.9142+0.0029
0.242740.0089
0.0680+0.0166
Ours (ALFA)
0.9045
0.221410.0019
1..0035
27

---

Published as a conference paper at ICLR 2025
Table 19: Experimental Results for Wiki dataset with DistillBERT
DistilBBET ADP
Wiki
Accuracy
AEOd
Base
0.9062:10.0003
0.2103+1.0040
0.0856:1000088
Fair-Mixup FAAP
0.9332:100006
0.1561
0.0088
0.9047=000046
0.2240+0.0240
0.0977+000222
FDR Ours (ALFA)
0.0013
0.192210.0111
0.0592+0.0163
0.9242=000003
0.2078+0.0005
0.0397+000005
I DATASET DETAILS
We follow the existing data pre-processing, (Mroueh et al. 2021 for the Adult and CelebA dataset, and (Mehrabi et al. 202T for other datasets.
Table 20: Features used from the Adult, COMPAS, German Credit, and Drug Consumption datasets.
Adult
age occupation capital-gain COMPAS
workclass relationship capital-loss
education-num race hours-per-week
marital-status
sex
sex juv_ dov_misd_oee miunt
age_ces
race priors_count count
juv foun_ charge_degree degree
juv saw_othee_coome _cuutt
German
Checking Account Credit amount Gender
Duration Savings account
Credit history Employment Residence Housing
Purpose Installment rate
Debtors/guarantors Installment plans
Property
Age Job
Existing credits
Liability
Telephone
Foreigner credits
Drug
Age Ethnicity
Gender Nscore Cscore
Education Escore Impulsive
Country
Oscore SS
Ascore
UCI Adult Dataset. Adult dataset Duaa et al. 2017) contains 48,842 individuals' information about income obtained from the 1994 US Census database. The target label is binarized to determine whether the income exceeds $50K/yr. Similar to (Mroeh et al. [021] and (Yurochkin et al. 2019). samples including missing values are dropped so the number of available samples is 45,222. The sex feature is used as sensitive attribute.
COMPAS Dataset. COMPAS dataset (Jeff Larson & Angwin 2016) contains 7,214 samples about criminal defendants and risk of recidivism with 8 attributes. It aims classify whether a person commits crime in the two years after they were scored. The sex feature used as a sensitive attribute.
German Credit Dataset. German dataset (Dua et al. 2017) contains the credit profiles for 1,000 individuals with 20 attributes such accounts, income, properties, and gender. The prediction goal to classify whether a person has good or bad credit risks. The gender feature is used as a sensitive attribute.
Drug Consumption Dataset. Drug Consumption dataset (Dua et al. 2017) contains records from 1,885 respondents about drug consumption. Each data point has 12 attributes including the level of
28

---

Published as a conference paper at ICLR 2025
education, age, gender, and on. The original task is multi culti-classification for 7 classes of whether and when respondents experienced drugs, but our prediction goal is abridged whether they consumed cocaine not. The gender feature is used sensitive attribute.
CelebA Dataset. CelebA dataset (Liu et al. 2018) contains more than 200,000 celebrity face images, each coupled with 40 numan-annotater binary characteristics such as gender. From these characteristics, we specifically choose attractive, smile and wavy hair, utilizing them to establish three binary classification assignments, with gender regarded as the sensitive attribute following (Zhang et al. 2017). We select these particular attributes as, in every task, a sensitive group is present which has higher number of positive samples compared to the other.
Wikipedia Talk Toxicity Dataset. Moreover, we further explore the adaptability of the proposed method the Natural Language Processing (NLP) dataset. We utilize Wikipedia Talk Toxicity Prediction Thain et al. 2017) which is comprehensive collection aimed at identifying toxic content within discussion comments posted on Wikipedia's 's talk pages, produced by the Conversation Al project. In this context, toxicity is defined content that may be perceived "rude, disrespectful, or unreasonable." consists of over 100,000 comments from the English Wikipedia, each meticulously annotated by crowd workers, as delineated in their associated research paper. A challenge presented by this dataset is the underrepresentation of comments addressing sensitive subjects such as sexuality, religion, gender identity, and race. In this paper, the existence of sexuality terms such 'gay'. 'lesbian", "bisexual', "homosexual", 'straight', and 'heterosexual" is used as the sensitive attribute, for existing, and 0 for absence.
J ADDITIONAL EXPERIMENTS
J.1 MULTI-LABEL CLASSIFICATION SCENARIO
We clarify that ALFA can be applied to the multi-label classification with binary-protected features it can be seen in multiple binary classification scenarios having individual decision boundaries. In this case, the fairness loss is newly defined as covariance between sensitive attribute and the mean of the signed distances, Lfair Cov(a, $ Et= 96(2) + 6z)) where T is the number of targeted prediction.
Luckily, one of our datasets, the Drug Consumption dataset (Dua et al. 2017) has multiple labels. To further investigate the feasibility of our framework for the multi-label classification, we conduct additional experiments the Drug Consumption dataset choosing four prediction goals, Cocaine, Benzodiazepine, Ketamine, and Magic Mushrooms while only Cocaine is considered as a prediction goal in the manuscript. The experimental result shows that ALFA effectively mitigates biases in the multi-label classification.
Table 21: Experimental results for multi-labed classification
Accuracy
Cocaine
Benzos
Ketamine
Mushrooms
Logistic Regression
0.7057 0.0099
0.6689 + 0.0113
0.6989 0.0267
0.7223
#0.0044
Logistic Regression + ALFA
0.6816 0.0114
0.6643 1 0.0122
0.7505
0.0023
0.7307
10.0082
MLP
0.6802 1 0.0144
0.6527 1 0.0138
0.7551 1 0.0094
0.7053 10.01
MLP ALFA
0.6701 +: 0.0057
0.6138 1: 0.0036
0.7343 1 0.0031
0.6587 10.0057
ADP
Cocaine
Benzos
Ketamine
Mushrooms
Logistic Regression
0.2691 + 0.0232
0.3597 + 0.0298
0.2478 + 0.1140
0.4151=0.0372
Logistic Regression + ALFA
0.0986 0.0289
0.2666 + 0.0424
0.0248 0.0070
0.3993 0.0425
MLP
0.2183 0.0222
0.3179 1 0.0278
0.0903 + 0.1320
0.0206
MLP ALFA
0.0760 + 0.0114
0.1808 + 0.0137
0.0368 + 0.0103
4.2384-000099
AEOd
Cocaine
Benzos
Ketaminc
Mushrooms
Logistic Regression
0,4411 t 0.0483
0.6448 + 0.0635
0.5184 + 0.2320
0.7096 #0,0732
Logistic Regression + ALFFA
0.1234 + 0.0471
0,4498 + 0.0858
0.0689 + 0.0158
0.6621+0.0911
MLP
0.3505 0.0449
0.5601 + 0.0597
0.2492 0.0385
0.691240.0441
MLP ALFA
0.0963 0.0249
0.2971 + 0.0193
0.1215 + 0.0153
0.3628
29

---

Published as a conference paper at ICLR 2025
J.2 MULTIPLE SENSITIVE ATTRIBUTE SCENARIO
In the binary classification with multi-protected features, the Differential Fairness (DF) is measured by binarization of each multi-protected features. For example, Foulds et al. 2020) defined DF
Ply 1 Ply 1
DF max max IES jES((i)
log
where i, S, and S denotes the set of multiple sensitive attributes. Therefore, in the multi- culti-proteeted feature case, we can define 'unfair region' by finding particular sensitive attribute provoking the maximum mistreatment and reducing the misclassification rate of the unfair region as well as the binary sensitive attribute case.
For the multiple sensitive attribute setting, we adopt COMPAS dataset and MEPS dataset. MEPS (Bellamy et al. 2018) data consists of 34,655 instances with 41 features(e.g. demographic information, health services records, costs, etc.) Among all the features, only 42 features are used. The sum of total medicare visiting is used as binary target label. When the total number of visiting is greater or equal to 10, a patient labeled as I, otherwise O. And 'race' is used as multiple sensitive attributes, for White, 1 for Black, and for others. The experimental result shows that ALFA is also applicable to the multiple sensitive attributes scenario.
Table 22: Experimental results for multiple sensitive attributes fairness
COMPAS
Accuracy
DF
MLP MLP ALFA MEPS
0.6875=#0.0048 0.6895+10.0023
1.7500=0.5794
1.3960==00092
Accuracy
DF
MLP MLP + ALFA
0..0031..0017
0.2000=0.0700
0.6860-1..024
0.0225
J.3 MULTI-CLASS CLASSIFICATION SCENARIO
For the multi-class classification, the decision boundaries not linear, so framework might not be directly applicable. However, multi-class classification can indeed be conceptualized as multiple binary classifications in a certain strategy called One-Vs-All. In this approach, for a problem with N classes, we can create N different binary classifiers. Each classifier is trained to distinguish between one of the classes and all other classes combined.
As each classifier can be seen as a binary classification task, we can utilize ALFA for the multi-class classification scenario by detecting unfair regions and covering the region by fairness attack. The evaluation metric for multi-class fairness takes maximum Demographic Parity across the classes
(Denis et 2021 In details,
ADPmuhi = max p( = k|a = 1) P(ý = k/aa 0)
kE[K]
where Ý is the predicted class, and & E [K] denotes each class & in the multi-class classification.
Among existing datasets for faimess research, Drug dataset can be used for multi-class classification. In fact, the original labels of the Drug dataset are multi-class settings, from 'CLO' to 'CL6' indicating the frequency of drug abuse. We have binarized them as 'never used' and 'ever used' regardless of the frequency in the main paper. However, for the multi-class classification setting, we adopt the original multi-class setting and report the mean accuracy and ADPaain with ML..
30

---

Published as a conference paper at ICLR 2025
Table 23: Experimental results for multi-class classification
Drug Multi-class
Accuracy
ADPmut
MLP MLP + ALFA
0.5196=10.0322 0,4960+10.0119
0.10322
0.1733=0.0287
K INTERPRETABILITY OF THE AUGMENTED FEATURE AND INPUT PERTURBATION
In this work, we can consider interpretability from two aspects: interpretability on decision boundary (latent space), and interpretability on original feature (input space). While we have focused on the first aspect, we argue that the proposed method can cover the second aspect as well.
At first, we are focusing on the interpretability of decision boundaries, which a common approach to understand the classifier's behavior (Guidotti et al.] 2020 Bodria et al. (002)). By manipulating features the latent space by the fairness attack, we can interpret decision boundary by discover- ing an unfair region and adjusting the decision boundary. In this case, it is true that it can't 't analyze how the changes input features affect the decision boundary. On the other hand, the interpretability of the input feature might make it possible to analyze how the fairness attack perturbs input data. However, it may lose the interpretability decision boundary, such as discovering unfair regions and understanding the last layer's 's behavior.
Fortunately, our framework is applicable to the input space by deploying the faireess attack and perturbation in the input space. In this case, the entire model will be fine-tuned, while offering input- level interpretability. We conducted additional experiments with MI.P to show the validity of our framework the input space in Table 24 Consequently, our method can offer either interpretabiliiyy on latent space or input space. In both cases, we can maintain the accuracy level while mitigating the fairness issue. We opt to freeze the pretrained encoder and deploy perturbations in the latent space, as this approach generally leads to greater improvements in fairness compared to perturbation in input space various datasets.
Table 24: Experimental results for input and latent perturbation with MLP.
Adult
Accuracy
ADP
AOO
MLP
0,8525 + 0.0010
0.1824 + 0.0114
0.1768 + 0.0411
MLP Latent Perturb.
0.8380 0.0045
0.1642 + 0.0261
0.0971 + 0.0098
MLP + Input Perturb.
0.8473 = 0.0016
0.1588 = 0.0135
0.1016 + 0.0394
COMPAS
Accuracy
ADP
AEO
MLP MIP + Latent Perturb.
0.6711 + 0.0049
0.2059 + 0.0277
0.3699 + 0.0597
0.6701 + 0.0020
0.0207 + 0.0142
0.0793 + 0.0418
MLP Input Perturb.
0.6629 + 0.0051
0.0610 + 0.0389
0.1086 + 0.0649
German
Accuracy
ADP
ΑEОd
MLP
0.7800 0.0150
0.0454 + 0.0282
0.2096 + 0.0924
MLP + Latent Perturb.
0.7570 + 0.0024
0,0053 + 0.0064
0.0813 + 0.0110
MI.P Input Perturb.
0.7465 0.0067
0.0188 + 0.0106
0.1700 0.0400
Drug
Accuracy
ADP
AEOd
MLP
0.6674 + 0.0096
0.2760 0.0415
0.4718 0.0838
MIP + Latent Perturb.
0.6382 + 0.0061
0.0820 + 0.0259
0.1068
0.0476
MLP + Input Perturb.
0.6188 + 0.0146
0.0571 + 0.0365
0.1893 = 0.0809
31

---

Published as a conference paper at ICLR 2025
L ANOTHER FAIRNESS CONSTRAINT
In this part, we show that ALFA can adopt any types of fairesss constraint during the fairenss attack.
As alternative of (Zafar et al. 2017), we present (Wu et al. (019) below.
Let's say f(X) is a logit of binary classifier given data X and define indicator functions 1(-) where
denotes each condition for the indicator function.
The empirical DP Gap is
ADP()
[[a Σ1(1()) > 0) [[(a =0]
Σ ((((X) > 0).
a=1
a=0
and can be rewritten in the expected form as
ADP(S) = E L(a =
1(a = 1(a=0)
1((()) > 0)
1 (((() < O))]
where P1 p(a 1).
Moreover, the relaxed form replacing the indicator function to real-valued function is written as
1 (a ((x
1(a 0) f(x))
ADP(F) EE
P
1 - P1
In (Wu et al. 2019). ). [(X) is replaced again to construct a convex form using two different surrogate functions use ADP as fairness constraint,
ADPk(f) = E (a =)
((((())) - -(1 (1- [(a = 0) x((f(())))
P1
1 1 P1
(aa 1)
1(a 0)
ADPE()) |
8(f(X))
8(- 8(-f(x)))
P
1 - P1
where 6 is convex surrogate function A:(2
max(= + 1. and 5 is concave surrogate function 2 If (AD(()) 2 we directly use ADPk(f) as
5(=) = min(2, 1) as proposed in (Wu al.
fairness constraint, otherwise use ADPO)).
ADDP(() if 2 0
fair
ADP5(J) if <.
Also, it can be extended to use AEOD directly faimess constraint, by conditioning ADP for each y E {0,1}
AEOD
Σ
1ff(z) 1(a 0, 1)
Σ 0=
1(((x) > 0)]
[| 1 1)
a=1,==
Σ
1({c)>
Σ x)]
(a 1,y=
[ 0, 0)]
1
a=0.y=0
and can be rewritten in the expected form as
AEOD(S) = E[ L(a = 1, 1)
1((X) > 0) - -01 1(a = 0. 1 1((()) < 0))
P1,1
P1,1
+
(aa
1. 0)
L(f(X) 0)
1(a = 0,y = 0)
1(f(X) < <0)]
P1.0
1
P1.0
since 1 = E
(a==0.y=)
= E (a=0.y
E Ma=0
(x(()) < 0) + 1a0=0-11) 1()
P1.1
P1.1
T-PL.1
0)] and 1 E
(a
0.y
0)
EE
[(a (1a-0.y
0.y
1a=00.0
1(f(X) <0) + 1-5-P1,0
10000.000
1(f(X)
Po.o
P1.0
P1.0
32

---

Published as a conference paper at ICLR 2025
(). = ply = 1) and ply = 0) = - where P1,1 P(a 1, 1) and PI,0 P(a 1,
0). AEOD can be expressed as a convex form,
1aa = 1,y x(((X)) 1(a = 10 0, K< x(-s(x)))]
AFODR(J) E
P1,1
P1,1
1(a 1,y = x(f(X)) (1
1(a = (aa=0,y= 0)
K² x(-f(x)))]
+E
P1.0
- -1-10
AEODE(J) E 1(a = 11 1) 8(s(X)) 1(a = 0, = 1)
8( 6(-s(x)))
P1,1
P1,1
1 1aa 1,y 8(f(x)) (1 1(a y 8(
8(-f(x)))
+B
P1.0
1- 7 P1,0 P1,0
where
(EODK(f) IE 20
Lfarr
AEODE(F) iE <.
Therefore, different from the covariance Zatar et al. 2017) between prediction and sensitive attribute, the convex fairness constraint takes into account the empirical outputs considering all potential dependencies, not focusing a particular attribute.
We report the experimental results in the table below by comparing the baseline, the covariance- -ase fairness attack (suggested in the paper), and the convex fairness attack. The experiment shows that method can adopt any type of fairness constraint during the attacking step, both showing improvement in fairness.
While our framework has wide adaptability in the choice of fairness constraint during the fairness attack, the reason we chose covariance instead of convex fairness constraint is it doesn't depend on the empirical outputs and offers clear proof illustrated in Proposition B.1 and Theorem B.2
M ANALYSIS FOR THE COMPARISONS
We analyze how such approaches. FAAP, Fair-Mixup. and ALFA improve fairness on a synthetic dataset as shown in Figure 3 In FAAP, the author generates adversarial perturbation using GAN model towards the sensitive hyperplane to make the sensitive attributes not recognizable, while trying maintain the accuracy. In the simplified form the objective function becomes,
min (Lno (So, 8,9) Lee (Jo; + a)).
However, in FAAP. the perturbations are not necessarily towards the sensitive hyperplane as shown in Figure (b), especially in the tabular dataset. There could potentially be two reasons for the observed discrepancies: the variations in the population sizes of each demographic group and the possible unsuitability of GAN-based perturbation for tabular datasets. Moreover, although the perturbed samples are correctly projected to the sensitive hyperplane, doesn't 't necessarily lead the fairer classifier. In Fair- Mtir. the author uses interpolation strategy to generate data in the manifold. However, the manifold assumptions could be too strict. Moreover, although the interpolated data may compensate for the imbalance in the dataset, it doesn't take into account the unfair regions, where the misclassification rates are disproportionately high, as shown in Figure (c). On the other hand, as discussed in Section 3 and Figure ALFA directly discovers and covers the unfair regions correct the classifier to become fairer.
33

---

Published as a conference paper at ICLR 2025
Table 25: Experimental results with different fairness attack objective function.
Adult
Accuracy
ADP
AEOd
Logistic
0.8470 0.0007
0.1829 + 0.0020
0.1982 + 0.0077
Logistic + ALFA (Covariance)
0,8464 + 0.0004
0.1555 0.0013
Logistic + ALFA (Convex)
0.8227 0.0026
0.0852 0.007
0.1547 = 0.0133
ML.
0.8525 + 0.0010
0.1824 0.0114
0.1768 + 0.0411
MI.P ALFA (Covariance)
0.8380 0.0045
0.1642 + 0.0261
0.0971 + 0.0098
MLP + ALFA (Convex)
0,8324 + 0.0031
0.1400 + 0.0166
0.0904 = 0.0184
COMPAS
Accuracy
ADP
AEOd
Logistic
0.6578 = 0.0034
0.2732 + 0.0129
0.5319 = 0.0245
Logistic + ALFA (Covariance)
0.6682 0.0040
0.0210 0.0167
0.0931 0.0323
Logistic ALFA (Convex)
0.6740 + 0.0034
0.0470 0.0180
0.1444 + 0.0379
MLP
0.6711 0.0049
0.2059 0.0277
0.3699 + 0.0597
MLP ALBA (Covariance)
0.6701 + 0.0020
0.0207 + 0.0142
0.0793 + 0.0418
MLP + ALFA (Convex)
0.6624 + 0.0010
0.0130 + 0.0075
0.0738 + 0.0150
German
Accuracy
ADP
AEOd
Logistic
0.7220 + 0.0131
0.1186 + 0.0642
0.3382 + 0.1268
Logistic + ALFA (Covariance)
0.7660 0.0189
0.0397 + 0.0261
0.1596 + 0.0354
Logistic + ALFA (Convex)
0.7410 + 0.0130
0.0240 0.0179
0.1030 + 0.0360
MLP
0.7800 0.0150
0.0454 0.0282
0.2096 + 0.0924
MLP ALFA (Covariance)
0.7570 + 0.0024
0.0053
0.0064
0.0813 + 0.0110
MLP + ALFA (Convex)
0.7575 0.0087
0.0181 + 0.0120
0.1960 + 0,0079
Drug
Accuracy
ADP
AEOd
Logistic
0.6626 + 0.0135
0.2938 + 0.0761
0.5064 0.1616
Logistic + ALFA (Covariance)
0.6554 + 0.0067
0.0909 = 0.0261
0.1170 + 0.0255
Logistic + ALFA (Convex)
0.6509 + 0.0072
0.0596 = 0.0198
0.1284 0.0286
MLP
0.6674 + 0.0096
0.2760 0.0415
0.4718
0.0838
MLP + ALFA (Covariance)
0,6382 + 0,0104
0.0820 = 0.0259
0.1068 + 0.0476
MLP ALFA (Convex)
0.6329 = 0.0173
0.1002 = 0.0826
0.1955 0.0956
N BALANCING ACCURACY AND FAIRNESS WITH A NEW HYPERPARAMETER
In the paper, we intentionally designed the framework with only one hyperparameter, to maintain simplicity. Under our cost-effective setup, where fine-tuning is applied only to the last layer in the latent space, we rely on grid search to find the optimal to control accuracy and faimess. However, we consider adding more control factors to balance accuracy and faimess to enhance the framework's flexibility.
To address this, we introduce an additional hyperparameter, X, to control accuracy in ALFA by modifying 5
als follows:
min
(1-
Σ (c(()((())) 2r, + Σ Cool0(z) I.EX
X,
+ [zp
2jEZ,
In this setting, lower À reduces the contribution of perturbed samples, resulting in higher accuracy. In the original configuration, 0.5 serves as the default value. À 0.0 corresponds the baseline without any fairness constraints data augmentation.
In the COMPAS dataset, accuracy remains stable across varying X, aligning with baseline results. In contrast, for the German and Drug datasets, accuracy decreases with increasing X, as expected, since perturbed samples contribute more to the training objective. On the other hand, small À is sufficient to improve fairesss, with the improvement remaining consistent across X. However, controlling fairness purely through À challenging. As demonstrated in Figures and 6 varying a provides an alternative way to influence fairness.
34

---

Published as a conference paper at ICLR 2025
COMPAS
Drug
040
0.5
Accuracy
1425
104
Accuracy
0.2
10.00
AFGO
Accuracy
AEDO
2070
Accuracy N000
040
Accuracy AFOP
Accuracy AFOD
230
Figure 9: The impact of À on accuracy and fairness performance AEOd for the COMPAS, German, and Drug datasets.
O IMPACT OF REPRESENTATION SIZE
We conduct an additional ablation study to analyze the impact of representation dimensionality on fairness performance. In practice, the representation size is typically pre-defined foundational models for computer vision and NLP tasks. However, for tabular datasets, it is feasible to train custom encoders with varying output dimensions. Specifically, for the Adult and COMPAS dataset, we vary the dimension size d across [32, 64, 128, 256, 512, 1024, 2048] both for the encoder's output E Rxx and perturbation
IR
The results in Figure TO indicate that while accuracy remains consistent, larger dimensions result in improved fairness. We analyze this that larger dimensions allow for greater perturbation capacity, enabling richer representations that more effectively attack the fairness constraint. Furthermore, richer representations provide the re- re-trained classifier with more detailed information, enhancing the overall fairness performance.
Adult Accuracy
Adult AEOd
0.90
0.275
Base
ALFA
0,250
Accuracy
0.80
AEOd
0.225
6.75
0.200
6.70
Base
0.65
ALFA
0.125
0.60
0.100
Hidden Dimension Size
Hidden Dimension Size
COMPAS Accuracy
COMPAS aEOd
3.675
0.650
Accuracy
0.625
AFOd
0.-
0.550
Base
0.1
Base
3.525
ALFA
ALFA
0.50
2
2f
2²
2
232
0.0
24
2
Hidden Dimension Size
Hidden Dimension Size
Figure 10: The impact of representation size on accuracy and fairness performance FEOd for the Adult and COMPAS datasets.
35

---

Published as a conference paper at ICLR 2025
P COMPUTATIONAL RESOURCE
Table 26: Compute Resources Used for Experiments
Component
Details
CPU GPU
AMD Ryzen Threadripper 3960X 24-Core Processor NVIDIA GeForce RTX 3090
36


Reviews:
[{"review_id": "FiGpXZFs7y", "reviewer": "Reviewer_aVVS", "summary": "This paper solves unfairness issues in classification tasks, which is caused by data imbalance. They proposed adversarial latent feature augmentation method which identifies the possible unfair region in latent space and fine-tune the model to mitigate such unfair decisions. The experiments demonstrated that ALFA achieves better group fairness without compromising accuracy.", "questions": "Please refer to the weaknesses.\n\nMy further questions are: (1) Adversarial learning idea is also used for learning sensitive invariant representation. I think the authors should discuss the connection with this branch of works. (2) Regarding the misclassification, should fairness metric like group utility difference, i.e., Rawlsian fairness, be used for the paper?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper presents an interesting perspective that biased model can be fine-tuned in an adversarial learning fashion. The key of the proposed method is to identify the unfair features in latent space, which is based on Zafar 2017's work with covariance formulation. Eq. 4 uses L_fair as an reference which help calculate the desired perturbation.\n2. The concept of unfair region looks like an interesting idea for promoting fairness. This concept is an extension for adversarial learning in the context of fairness learning.\n3. Algorithm 1 shows the overall procedure for fairness tuning, which is convincing and also easy to follow.", "weaknesses": "1. I realised the unfair region concept is defined by misclassification rate but the used fairness metric, i.e., EOd, is only for positive predictions. This misalignment has raised a big concern for the proposed method.\n2. From my understanding, given any biased training set, the unfair region should be not only produced by the observed data and pre-trained model ,but also captured by the perturbation, i.e., augmented data. Is this the correct way to understand Fig.1?\n3. It is widely believed that adversarial perturbation improves the robustness with the sacrifice of hurting the model performance on benign data. In the context of fairness learning, the performance drops for data samples which fall in the fair region will be affected. I understand during the tuning phase, i.e., Eq. 5, the benign data is also retrained to mitigate this issue. But any other strategies can be used?\n4. The theoretical framework is highlighted in abstract while it is not clearly presented in the main body of the paper. Am I missing it?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4wBHDxKjlE", "reviewer": "Reviewer_tbL6", "summary": "The authors propose a method to perturb latent space features, to augment the learning by highlighting areas where the performance within certain subgroups is sub-optimal. They support the methodology with theoretical analysis and thorough experiments, demonstrating the effectiveness of the method in reducing the bias, without compromising on the accuracy. The paper is extremely well written, presenting a novel approach, with strong motivation, solid backing for the methodology both empirically and theoretically and detailed discussions and code.", "questions": "- [Q1] With regards to [S3], is there a comparison available for usage of Wasserstein over Sinkhorn, in terms of possible gain in performance at the cost of some computational overhead? Could it be feasible for smaller datasets like German/COMPAS? An additional analysis in Appendix Section D would prove insightful to support this choice.\n- [Q2] Is there analysis for how the covariance of the label and sensitive attribute affect the effectiveness of the method? I like the table in Section E of the Appendix, but I would be curious to know under what regimes is the method particularly effective? Also what if the covariates shift from train to test? Is the Table E for only the train data? Would be nice to see some more discussion along these lines as I think it could shed some light on the practical scenarios in which the proposed method would be most effective.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- [S1] Well motivated paper, particularly from the point of view of understanding how to use the latent space better to reduce biases in neural networks. Lot of the literature as the authors mention is quite superficial with respect to justifying the methodology with intuition or analysis for why it works for any random latent space.\n- [S2] Objective $\\mathscr{L}_{fair}$ seems well defined and linked to Equalized Odds in the appendix. Proof overview looks fine.\n- [S3] The use of Sinkhorn over Wasserstein is a clever experimental choice, which sidesteps a lot of computational issues associated with such algorithms.\n- [S4] Inclusion of clean and well written code is a big plus towards implementation of the proposed method.\n- [S5] The plotting of Pareto curves as opposed to simply singleton points in Figure 4 and Figure 5 is a pleasant sight to understand the patterns across different fairness-accuracy tradeoffs.", "weaknesses": "- [W1] Visualization of the latent space of real world datasets (say via PCA), for smaller ones at least perhaps akin to the figures for synthetic datasets, would help understand the unfair regions a bit better.\n- [W2] I am slightly concerned about the practicality of the proposed approach in high dimensional latent spaces, do the authors have thoughts or any experiments about how the method scales with (keeping everything else constant) a) Input Feature Dimensions b) Latent Space Dimension", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yxDgI9k8Vv", "reviewer": "Reviewer_ZWM9", "summary": "This paper introduces ALFA (Adversarial Latent Feature Augmentation), an approach to enforce Equalized Odds fairness in binary classification through adversarial data augmentation. The method generates adversarial perturbations in the latent space by maximizing the covariance between sensitive attributes and logits of the predicted probability. The approach operates in two phases: first creating biased latent features through fairness attacks, then fine-tuning on both original and perturbed features using a dual cross-entropy loss to balance fairness and accuracy. Comprehensive experiments across multiple datasets (Adult, COMPAS, German, Drug, Wikipedia and CelebA) demonstrate ALFA's effectiveness in improving Equalized Odds while maintaining competitive predictive performance.", "questions": "The authors' responses have addressed most of my concerns. Based on the current revision, I have updated my rating to 6.\n\n##################################\n\n\nUpsampling Strategy and Distribution Mismatch:\n\n1) How is the upsampling implemented - are instances repeated multiple times to achieve Np = 4 · max(n00, n01, n10, n11)?\n2) My main concern is regarding that the fairness improvement by reducing ∆EOd (i.e., ∆EOd(θp) ≤ ∆EOd(θ)) is demonstrated on this upsampled distribution rather than the original dataset. For a test set (20% of the original data) that maintains its original unbalanced distribution, how does your method guarantee fairness on the test data, particularly with highly unbalanced combinations of A and Y?  Being fair on the upsampled training data does not necessarily imply fairness on the original distribution present in your test set.\n\nTraining Procedure:\n\n3) Why does the method use a single pass of the two-step process, unlike traditional adversarial learning approaches where the adversary and model are iteratively updated multiple times? \n\nTechnical Details of δ Implementation:\n\n4) How exactly is δ maximized? Is it implemented as a neural network trained via gradient descent?\n5) Does the maximization of δ take $X_p$ as input?\n6) What mathematical constraints ensure proximity in latent space? Are there Lipschitz constraints on δ maximization?\n\nLoss Function Balance:\n\n7) Equation 5 uses equal weighting between the balanced and unbalanced distributions, which shifts half the focus to the upsampled distribution. Without a hyperparameter to control this split, how can the model maintain proper task accuracy? Even when fairness enforcement is not needed (i.e., when α approaches infinity), the attention remains partially shifted to the balanced distribution $D(X_p,Y_p,A_p)$, which could differs from the test distribution.\n\n8) Could this equal weighting explain why your method shows lower accuracy than SOTA in the unfair regions of Figure 4? Additionally, the blue curve is largely absent in unfair regions (high α values) except for Adult UCI dataset. Did other datasets also show poor performance in these regions with higher values of α?\"", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper presents a novel integration of adversarial perturbations and data augmentation in latent space to address fairness constraints.\n- The proposed method achieves competitive results", "weaknesses": "1) **Problem Formulation and Distribution Mismatch:**\nThe fairness enforcement is performed on artificially balanced demographic data (n00 = n01 = n10 = n11), while the test distribution could be imbalanced. This fundamental distribution mismatch is not adequately addressed.\nThe method achieves Demographic Parity proportional to Equalized Odds on this really specific case, which contradicts the known incompatibility between these metrics and loses the essential dependence between Y and A.\n\n2) **Incomplete Baseline Comparisons:**\nThe experimental comparisons raise concerns about completeness and fair representation of SOTA methods:\nOn standard datasets (German/Adult and Wikipedia - Figure 6), the method achieves EO violations no better than 0.1, while existing approaches [1] show that Zafar et al. (2013) using covariance loss, typically achieve EO violations close to 0.03 under the same test conditions on adult uci(20% test split).\nThis performance gap suggests the experimental evaluation may not fully represent the capabilities of existing state-of-the-art methods\n\n3) **Technical Gaps:**\nThe methodology lacks sufficient technical explanations and critical implementation details are missing, particularly regarding the generation of δ\n\n[1] Andrew Lowy, Sina Baharlouei, Rakesh Pavan, Meisam Razaviyayn, Ahmad Beirami:\nA Stochastic Optimization Framework for Fair Risk Minimization. Trans. Mach. Learn. Res. 2022 (2022)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7OAERWFELc", "reviewer": "Reviewer_wUPu", "summary": "The paper proposes a novel bias mitigation approach that leverages adversarial attacks on model fairness to augment samples in the latent space region where the model makes the most mistakes. The last layer of the model is then retrained with augmented latent features to correct the decision boundary for better, fairer predictions. The proposed method is evaluated on several benchmarks, showing the improvement over the baseline methods considered. Theoretical insights also back the proposed method.", "questions": "Please see the above Weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, well-organized, and relatively easy to follow. \n2. The counterintuitive idea of using fairness attacks on a pretrained model to mitigate bias is interesting and novel. \n3. The experiments are thorough and spanned across different types of datasets and model classes.\nThe proposed method is flexible and can be adapted to different types of fair constraints. \n4. The results demonstrate the effectiveness of the adversarial data augmentation method in mitigating bias, and the theoretical results are intuitive.", "weaknesses": "1. While Figure 1 provides an intuitive comparison with related work, the related work section can be improved to amplify the paper's position. In particular, using adversarial learning to identify regions where the model makes the most mistakes is not novel and has been studied before by `Lahoti, Preethi, et al.` [1].  I recommend the authors discuss this work and consider it as a baseline for comparison.\n2. The proof of theorem 3.2 seems to rely on the assumption that samples from high error regions mostly influence the gradient updates. This seems to depend on the success rate of the fairness attack, and the authors did not demonstrate that the attack can fully construct `R_unfair`.  I suggest to clarify the assumptions used in theorem 3.2.\n3. It is suggested the author could perform additional experiments to show the success rate of the fairness attack on real datasets, i.e., the unfair region is actually the region targeted by the fairness attack. This will support the assumption that the attack can effectively identify `R_unfair` and oversample data points from that region. For example, predicting the accuracy of a sensitive attribute classifier in these regions and comparing it with the ground truth `R_unfair`. These experiments showing that the fairness attack effectively augments data points in the ground truth `R_unfair` would support the inherent assumption made in theorem 3.2.  \n4. Some parts of the paper can be improved for clarity; for example, in Figure 4-6, some baseline methods have fewer tradeoff points than others; the number of parameters controlling the tradeoff between $\\Delta_{EOD}$ and Acc. should be provided. In Fig. 2(a), providing the decision boundary for different perturbation levels $\\delta$ might provide better insight into the magnitude of the fairness attack over the corrected boundary.\n5. Throughout the paper (e.g., Line 137), it’s stated that the method improves group fairness _without compromising accuracy_; as the results show the tradeoff between fairness and accuracy, the statement should be updated to reflect this fact. For example, the authors formulate the statement as \"_improves group fairness while maintaining accuracy_ (as much as possible)\". This would highlight the fact that there is a potential tradeoff.\n\n[1] Lahoti, Preethi, et al. \"Fairness without demographics through adversarially reweighted learning.\" Advances in neural information processing systems 33 (2020): 728-740.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)