Debiasing Attention Mechanism in Transformer without Demographics
Authors: Shenyu Lu
Yipei Wang
Xiaoqian Wang
Venue: ICLR 2024 poster (2024)
Keywords: Fairness
Transformer
Attention
Without demographics
Primary area: societal considerations including fairness, safety, privacy
TL;DR: Improve fairness in transformers without demographics

Abstract: Although transformers demonstrate impressive capabilities in a variety of tasks, the fairness issue remains a significant concern when deploying these models. Existing works to address fairness issues in transformers require sensitive labels (such as age, gender, etc.), which can raise privacy concerns or violate legal regulations. An alternative way is through fairness without demographics. However, existing works that improve Rawlsian Max-Min fairness may impose overly restrictive constraints. Other methods that use auxiliary networks could be parameter inefficient. In this paper, we present a new approach to debiasing transformers by leveraging their inherent structure.  By reconsidering the roles of important components (queries, keys, and values) in the attention mechanism, we introduce a simple yet effective debiasing strategy from two perspectives: 1) Grounded in theoretical analysis, we normalize and apply absolute value operations to queries and keys to minimize the bias in attention weight allocation; 2) We reduce the bias within values through local alignment via contrastive learning. Throughout the entire process, our approach does not require any sensitive labels. Furthermore, to enhance memory efficiency in the training phase, we propose a strategy that debias only the last encoder to improve fairness in pre-trained models. We conduct experiments in computer vision and natural language processing tasks and show that our method is comparable and even outperforms the state-of-the-art method with substantially lower energy consumption.

Published as a conference paper at ICLR 2024
DEBIASING ATTENTION MECHANISM IN TRANS- FORMER WITHOUT DEMOGRAPHICS
Shenyu Lu, Yipei Wang & Xiaoqian Wang"
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47906, USA {1u876, wang4865, joywang @purdue. edu
ABSTRACT
Although transformers demonstrate impressive capabilities in a variety of tasks, the fairness issue remains significant concern when deploying these models. Existing works to address fairness issues in transformers require sensitive labels (such as age, gender, etc.), which can raise privacy concerns or violate legal reg- ulations. An alternative way is through fairness without demographies. However, existing works that improve Rawlsian Max-Min fairness may impose overly re- strictive constraints. Other methods that use auxiliary networks could be parame- ter inefficient. In this paper, we present a new approach to debiasing transformers by leveraging their inherent structure. By reconsidering the roles of important components (queries, keys, and values) in the attention mechanism, we introduce simple yet effective debiasing strategy from two perspectives: 1) Grounded in theoretical analysis, we normalize and apply absolute value operations to queries and keys to minimize the bias in attention weight allocation; 2) We reduce the bias within values through local alignment via contrastive learning. Throughout the entire process, our approach does not require any sensitive labels. Furthermore, to enhance memory efficiency in the training phase, we propose a strategy that de- biases only the last encoder to improve fairness in pre-trained models. We conduct experiments in computer vision and natural language processing tasks and show that our method is comparable and even outperforms the state-of-the- method with substantially lower energy consumption.
INTRODUCTION
Transformer- based models demonstrate immense power and achieve remarkable success in both computer vision and natural language processing fields. The transformer architecture is based on the concept of self-attention, which allows the model to weigh the importance of different tokens or patches of the input sequence when making predictions (Vaswani al. 2017).
Fairness is a crucial factor to consider in machine learning models. For example. Gong et al. (202I) highlight the potential risk of applying biased face recognition systems in law enforcement. Al- though transformer models exhibit outstanding performance, (Sudhakar et al. (2023), Qiang et al. (2023), and Baldini al. (2021) observe that transformers make biased predictions vision and NLP domains. Addressing faimess issues in transformers is significant but challenging task. Our work focuses on mitigating biases within transformer architectures.
Existing works attempt to address fairness issues in transformers in two ways: targeted alignment for debiasing transformers (TADeT) and debiasing self-attention (DSAA)/Sudhakee Sudhakar et al. (2023) pro- posed to debias transformer by aligning "Query" in different sensitive groups within the same task. DSA QQing et al., (202)) generates adversarial examples by attacking spurious features. and aligns attention weight between training samples and adversarial examples. However, these methods ne- cessitate sensitive attributes (such as gender and race) improve fair transformers. Collecting such information is not only costly but may also raise privacy concerns.
Several studies
(Hashimoto et al., 2018 Lahoti et al. 2020 Liu et al.) 2021 Creager et al., Chai et al. 2022) propose to resolve general fairness issues without sensitive information.
2021
"Corresoonding author.

---

Published as a conference paper at ICLR 2024
propose to optimize the worst group achieve equal utility in different sen- sitive groups. However, this method is likely to be affected by outliers, leading to a significant drop in performance. Lahoti et al. (2020) share the same idea to solve fairness issues but use reweight- ing strategy. However, their method incorporates adversarial training, which introduces instability. Moreover, improving the worst-performing group imposes strong constraint the objective func- tion. [Lu et al. (2021) suggest a method where a network is trained twice, emphasizing samples misclassified by the initial model. However, this double-training approach lacks efficiency and runs counter to green deep learning initiatives (Xu et al., 2021). Creager et al. (2021) propose a method perform environment inference, followed
Hashimoto al. (2018)
that is also a two-stage method, where the first step by leveraging a robust optimization method to train the network. The limitation here is that their environment inference relies on the Bernoulli distribution, making it applicable only to binary sensi- tive labels. A recent study (Chai et al.) (022) tackles fairness without demographies via knowledge distillation. However, this approach involves teacher network that is significantly larger than the student network for making predictions, thus being time-consuming and parameter-inefficienn
Additionally, large-scale pre-trained models also face fairess issues, Many existing methods require tuning all the parameters to achieve balance between faimess and accuracy, but updating large number parameters can be prohibitively expensive (Petersen et al. 2021).
In summary, existing works of fair transformers require sensitive attributes. Existing methods for fairness without demographics either need a large auxiliary network or lay a strong constraint. For addressing fairness concerns in pre-trained models, tuning all parameters lacks efficiency and ne- cessitates high-demand computational resources.
To resolve the above challenges, we propose a novel approach tailoring the fairness considera- tions to the transformer encoder. Instead of debiasing the entire representation of the transformer encoder which places significant constraints on the representation, we propose to debias each com- ponent (Query, Key, and Value) in the attention formulation. This approach, derived from theoretical foundations, presents simple yet effective strategy address the fairness concern in transformers without the necessity of auxiliary networks or iterative training.
We debias from two perspectives within the attention mechanism.
Attention allocation: Drawing inspiration from theoretical analyses, find that normal- izing and applying the absolute value to q and reduces diserepancies in attention weight between different sensitive groups
Local alignment on value: We mitigate bias in v through local alignment. This is ac- complished with supervised contrastive learning approach, which encourages the core segments from different sensitive groups be similar.
We conduct extensive experiments on real-world datasets, encompassing various classification tasks in computer vision and natural language processing (NLP) fields. Furthermore, we provide GPU memory-efficient solution to address fairness in pre-trained models. We append a fairness-aware encoder on these models and only train that encoder. We demonstrate its efficacy in enhancing fairness within pre-trained models.
We summarize our contributions follows:
To the best of our knowledge, we are the first to enhance faimess in transformers without demographics.
Our algorithm achieves effective results in both computer vision and NLP domains.
Our method could plug in pre-trained model to improve faimess without the need for re-training, resulting improved memory efficiency.
2
RELATED WORK
2.1 FAIRNESS INTERVENTION
Existing fairness intervention strategies can be categorized as pre-processing, in-processing, and post-processing (Wang et al. 2024). Pre-processing methods mitigate biases by optimizing the data, including sampling (Qraitem et al.; 2023 Kamiran & Calders 2012 Chakraborty et al. 2020; Yao & Liu 2023), transformation ( YYoo al. 2022|calleee Calmon et al. 2017), and augmentation (Zietlow al.
2022 Jang et al.] 2021 Wang et al. 2020 Cheong et al. (022). Nonetheless, the methods discussed
incur significant costs due to the necessity for human annotation of sensitive attributes (Chen al.
2

---

Published as a conference paper at ICLR 2024
2024] Post-processing methods adjust outputs directly to meet fairness objectives. with various ap-
proaches such as flip prediction (Hardt et al. 2016). score transformation (Alghamdi et al.) 2022
Wei et al. 202T). threshold adjustment (Jang et al.] 2022 Xian et al.. 2023); In propoeessing methods
are integrated into the model design and training phase, fostering the development of fair models
(Wan al. 2023). These
be classified into fair regularizer (Agarwal et al. 2018 Kamishima
iet al. 2011 Zemel et al. |003 Jiang et al. 2000 Zafar et al. 2019 et al. 2021). adversarial
learning (Zhang et al. 2018: Wadsworth et al., 2018), disentanglement (Sarhan et al. 2020 Lo-
catello et al. [019]] Creager et 2019) an so on. In this work, we adopt an in in-processing strategy because it offers more flexibility (Song et al., 2024) and allows us to directly address fairness issues during the training process. Compared to existing in-processing methods, our approach leverages the unique structure of the transformer and incorporates a tailored design to specifically address fairness concerns within these architectures.
2.2 FAIRNESS WITHOUT DEMOGRAPHICS
Distributionally robust optimization (DRO) (Hashimoto al. 2018) achieves Rawlsian Max-Min
fairness by optimizing the risk of the worst-case samples. Adversarially reweighted learning (ARL) (Lahoti et al. 2020) aims to optimize the performance of computationally identifiable samples by reweighting them, thus improving the worst-case performance. Knowledge distillation (KD) (Chai ct al. 2022) leverages the soft labels generated by a teacher network and combines them with hard labels to debias student model. Just Train Twice (JTT) (Liu et al., (021) follows reweighting paradigm, emphasizing error samples in the training set by giving them elevated weights. The training phase is split into two stages: an Empirical Risk Minimization (ERM) stage, succeeded by a reweighted ERM stage. Environment Inference for Invariant Learning (EIIL) (Creager et al. 202I) is also a two-stage method. The initial step employs trained network to infer group labels, and the second step utilizes a robust optimization strategy to re-train the network. Learning from Failure (LfF) (Nam et al. (2020) involves training two neural networks concurrently: one with amplified bias and another focusing on samples that contradict this bias. Through this process, the second network is conditioned to correct the errors made by the first.
2.3 FAIRNESS- TRANSFORMERS
Recently, Sudhakar et al. (2023) and Qiang et al. (2023) aim to solve fairness issues in vision trans- formers. Sudhakar et al. (2023) proposed to use adversarial training technique to hide the sensi- tive information in the classification token. Plus, they apply L2 Loss to penalize query vectors with large discrepancies among different sensitive groups with the same target label. However, debias- ing on query may not be sufficient for resolving fairness issues in transformers, and the introduced adversarial training may incur instability issues. Qiang al. (2023) propose two-stage method to mitigate bias in vision transformer ( (ViT). They first identify sensitive related patches and use adver- sarial attacks to perturb those patches. Next, they train a ViT using both the original training set and the attacked dataset to perform attention weight alignment. However, this work hypothesizes that the sensitive attribute must be identifiable and has no overlap with target related patch. This laid a too strong constraint and may have negative effect on the model performance. Moreover, both existing methods require sensitive attributes during the training phase.
2.4 CONTRASTIVE LOSS
The contrastive loss has been proven to be more robust Xue et al. 2022) and has a better generaliza- tion performance than cross-entropy loss (Khosla al., 2020). Supervised contrastive loss (Supcon)
Khoola et al. 2020) utilizes label information to pull together normalized embeddings of the same class and push apart normalized embeddings from different classes. Differing from self-supervised representation learning (Chen et al. 22200] Tian et al.) 2020), SupCon selects positive samples from the same class within a mani- leading to many positive and negative samples for each anchor, which enhances performance.
3 METHOD
Problem setup We denote a dataset D (X,Y, E X x y x A, with X as features, Y being target labels, and A being sensitive attributes. Our goal is to learn a classifier fo X H V, where Y a prediction labels that satisfies fairness criteria. For example, Demographic parity (DP) requires Y L A, Equal opportunity (EOp) requires Y A|Y = 1, and Equalized odds (EOd) requires Y I A|Y = y, y E {0, We focus on the setting where we do not have access to sensitive attributes in the training set.
Background and notation to its attention mechanism. In transformer, input image (or sequence in NLP) is divided into
The informative capability of the transformer model can be attributed
3

---

Published as a conference paper at ICLR 2024
patches (tokens in NLP). These patches tokens are first processed through embedding layer. Subsequently, they undergo a transformation in the encoder through three linear modules, which map them to query,
and value (Vaswani et al. 2017 Dosovitskiy et al. 2020). In this work, we denote Q E N as query, K E Ig as key, and V E Ik as value, where N is the number of tokens. Additionally, we denote q E Rd as a query vector corresponding to a patch, extracted from Q. Similarly, we denote k E IRd as a key vector, v E 1d as value vector. We use the subscript "cls" as special symbol to denote patch or token used for downstream task. This notation aligns with the conventions established in (Devlin et al. 2018).
q
Motivation A recent study (Sudhakar et al., 2023) reveals disparities in average query across dif- ferent sensitive groups when performing the same task, highlighting discriminatory behavior in the attention mechanism. In addition. Sudhakar et al. (2023) Qiang et al. (2023) underscore the impor- tance of the attention mechanism in mitigating bias issues. They empirically validate that addressing fairness concerns within the attention mechanism is effective and yields good performance compared to other in- pr-occesssigg methods. Based on these findings, our work focuses on debiasing within the attention mechanism. To this end, we revisit the attention mechanism (Vaswani et al. 2017):
QK Vdx
Attention Q K, V) softmax
V
The attention mechanism equation can be viewed as two-stage operation: Attention allo- cation performed by the inner product of Q and K, and 2) Weighted sum over V. We attribute the discrimination in a transformer-be based model to 1) a misallocation of attention weight and 2) biased representation of V. Based on the observations, we present two-stage approach that does not require sensitive attributes to mitigate discrimination in a transformer encoder.
3.1 FAIRNESS-AWAR ATTENTION WEIGIIT RELOCATION
Drawing inspiration from the principle of minimizing the statistical distance between representations of various groups to enhance fairness (Balunović et al., 2021), introduce method that aligns the expected attention weights across diverse demographic groups without accessing sensitive attributes.
k To be specific, we denote w vłz as attention weight of two tokens (patches). Given the fairness consideration, our optimization problem can be written as:
min |8| s.t. 8 B[wla 0] Eiwla= =
which to minimize the disparity in attention weights w between different sensitive groups. Di- rectly solving equation 2 can be challenging due to the inaccessibility of the sensitive attribute. We present Theorem 1, which shows that a series operation can effectively reduce the attention alloca- tion disparity among different sensitive groups
Assumption 1. Denote P(qia = i) = (( E Eg.). P(k|a= = = ((p,s Ex,)), i E f0. For qj E and k E k, Oj and k; are independen m j E (1,...,da d. The base rate for P(a 0) = λo and P(a 1) = >1. The distribution of query q and the key k of a transformer can be written as:
a~aoN((Mn, Eg,)) + (
(3)
<~>aN(HHO Σx,0) + AIN Ex.1)
(4)
For a transformer, the expected difference in attention weight is
|6| = [ q k
q
1 a
d2
aa = 0]
-
EE
1]
(Elg,ky la a= 0] -Eagk, = 1))l
dx
vdk
dd
(5)
de (P.0..ME.0.
8 in vanilla transformer is unbounded. To resolve this issue, we reference Theorem 1.
Theorem 1. For a vector x [x1]..za] E IRd, define n(x) = (01-B((1) 3i 3a-B/ma) and m(x)
- /Warlan]
/ /Wrlmaa
E [m_] ... [xal]- Given query q E dd and key k E TRdde, denote the debiased query and key as
q = m o n(q) E Rd kde -d==ne m o n(k) E dRde. Under Assumption 1, the disparity in debiased attention weight is upper bounded by
'The independence of zj and k, follows the same assumption presented in Vaswani et al. 2017
4

---

Published as a conference paper at ICLR 2024
16d1
=
B3
de T
la =
0]
E
q T
kd 1a
S 11
v du (V
2 To
+
A1 A
+(V
2 TÀ
Ao
2
vd,
d2
For the attention mechanism in transformer encoder, as shown in equation 5 the disparity of the attention weights across different demographic groups has the potential to diverge toward infinity. Theorem I indicates that upon normalization of vectors in Q and K, and taking their absolute values, the disparity in attention allocation could be bounded at a constant, wherein the constant incorporates the base rate of the training set and the embedding dimension. Note that when the sensitive attribution is balanced (e.g. The numbers of male and female samples are the same in CelebA), the disparity is minimal. We leave the details of the proof for Theorem in the appendix.
In practice we compute attention weight based on qde and kde, where q = m o n²(q) E R&,, de m n(k) Rd stated in Theorem 1. During the inference period, we the mean and standard deviation estimated from the training set to normalize q and k for the test samples. To estimate these statistics, we utilize the running mean (e) and running standard deviation (s) as suggested by (Ioffe & Szegedy 2015). The running mean is initialized with zero tensors, and the running standard deviation is initialized with one tensors. We update the running mean and running standard deviation using momentum- -base approach:
e (L - p)e + pe s (1) p)s
ps noom
where p is the momentum. we set p = 0.1 by following existing protocols (Ba et al. 2016 loffe & SSegedyy 2015). c and sorm are sample mean and sample standard deviation in batch.
3.2 LOCAL ALIGNMENT ON VALUE
In transformer structures, the value of token (a patch) can be considered as local representation. Tokens (patches) with high attention weights encapsulate core objects, ensuring target-specific infor- mation encoding. For considerations of fairness, is crucial that the encoding within these tokens (patches) remains consistent. Conversely, tokens (patches) with low attention weights, typically representative of background elements, enhance the richness of the final representation.
In representation learning, Chen et al. (2020) demonstrates that using nonlinear projection head im- proves the representation quality. We follow this finding and define a small nonlinear head g ++ to map value v to a latent representation z. To mitigate bias in the representation while preserving the discriminative power, we aim to ensure similar representations within the same target label, while dissimilar representations from different target labels, regardless of the sensitive attribute. A natural choice for achieving this objective through supervised contrastive loss (Khosla et al. 2020):
Σ log exp(z; 2p/T) PEP(1)
Calignment = Σ
ΣKER() exp(z: 2x/T)"
where: i E I (1....,M. M) is the index of anchor, M is the number of samples in batch B(i) = [1fi). P(i) = {p E B(i) : y, = yi} is the set of indices of all positive samples in mini-batch. is temperature hyperparameter, we set Contrastive Loss in accordance with our objective of achieving consistent representations. Though the recently proposed Fair Supervised Contrastive Loss (FSCL) integrates fairness by specifying negative samples within an identical sensitive group (Park et al. 2022). FSCL might compromise the efficacy of contrastive learning and necessitates the inclusion of both target and sensitive labels.
0.07. We selected the Supervised
We argue that directly employing supervised contrastive learning on the representation of the entire image or sentence does not necessarily contribute to fairness. Park et al. (2022) demonstrate that with the use of Supervised Contrastive Learning (SupCon) to pre-train a ResNet model, issues of unfairness still persist. We attribute this phenomenon to the correlation between the target and sensi- tive labels. Since the network contrasts the entire representation, it can still leverage sensitive-relatee information. From this observation, we propose an attention weight guided local value alignment technique to effectively tackle faimess concerns in the representation. The learning framework con- tains the following steps:
We define w; qit k" as the attention weight of the i-th token (patch), W va
[w..... w/ where is the number of patches (in vision) or the number of tokens (in NLP).
We select v.- with top attention weights: s* arg max, Eiey Wi, s.l. w; E W, |s*| =
5

---

Published as a conference paper at ICLR 2024
We concatenate all V.. and apply nonlinear projection head g(-) to map from the value vector to representations: 2 g(v=)), ). We then apply supervised contrastive loss equation to perform the local alignment of the value vector.
Why does local alignment help fairness: Our method debiases on the attention mechanism using two steps. The first step of debiasing the attention weight (in section 3.1) is crucial for the second step (in section 3.2 when selecting patches with high attention weights. Utilizing these debiased weights from the first step, select the top t patches based target label relevance. Incorporating supervised contrastive learning encourages patches within a class to share similar representations, focusing encoding information related to the target label, disregarding sensitive information. This approach reduces the sensitive information encoded in the representations, thereby enhancing fairness. It's important note that without debiasing the attention weights in the first step, the technique of local alignment on value in the second step does not contribute to fairness. This is evidenced by our ablation study in Appendix FF where using local alignment alone barely impacts fairness. In contrast, combining debiased attention with local alignment achieves optimal results in terms of fairness.
3.3 LEARNING FRAMEWORK
We present our learning framework in Figure 1 Our learning framework contains two steps: First, we normalize each q and k and take their absolute values in the last encoder layer to compute atten- tion weights. This approach ensures that attention allocation less affected by sensitive attributes. Next, we select value v associated with top attention weights to perform local value alignment. Throughout this process, we do not leverage sensitive attributes. We summarized our method in an algorithm, detailed in Appendix G
dostibation
Value veciere with top
Vasilla
Dehinacd
9(-)
k
Nommaliattion Alwertnte
Nit
Tomga--
relited
rellard
Angonaa
Bacodes pro-traized traosfomme)
Linese projection panchess
Figure 1: Our learning framework is illustrated using the vision transformer applied to the CelebA dataset, specifically for y blond hair, a male. An input image is processed through L --1 transformer encoder layers. In the L-th layer, for each image patch, we normalize and take the absolute value of the query vectors and key vectors debias the attention weights. Subsequently, we select the value vectors corresponding to the top debiased attention weights, concatenate these I vectors, and map them new representation z using the function g(-). We apply a supervised contrastive loss to this representation z for contrastive learning.
3.4
GPU MEMORY EFFICIENT COMPUTATION
Last encoder training Even if the pre-trained model is powerful, fairness issues still exist. However, mitigating bias in large pre- trained models is challenging. Fine-tuning all the parameters is not only time-consuming but also heavily relies
nigh-performance computational devices.
The bias is spread in all the layers of encoders. However, as demonstrated by (Kirichenko et al,, 2022). the objective of empirical risk minimization (ERM) training is sufficient for learning the features on which a debiasing procedure can be deployed at the very last layer of the network.
Based on this observation, append our proposed encoder layer to the top of the large pre-trained model. The pre-trained model is capable of capturing the useful features within an image (Caron al. 2021 sentence (Clark et al. 2019). We employ the pre-trained model as feature extractor

---

Published as a conference paper at ICLR 2024
and only train fairess- aware one-layer encoder. This method does not require a GPU with large memory capacity, because parameters in pre-trained models are not updated. We show it has maintained utility and improved fairness in section 4.3
4 EXPERIMENTS
We evaluate our method across various settings. We first explore our method in CV by conducting experiments on two real-world datasets. We next extend the application of our method the NLP field. Additionally, we test the capability of our method to address fairness issues in pre-trained models with limited resources. Finally, we explore the trade-off between faimess and accuracy. To reproduce our experiment, we have made the code available at http:: / wig.tb..b...... com u Debiasing -aasingg-Attenttn. -n- sm- in-iii--rraas frraaforrserr.er thout tho wew.ttootttt--mggaphhhhh
4.1 VISION EXPERIMENTS
Experimental setup
We test all methods on two real-world datasets:
CelebA (Luu et al. 2015
and UTK (hhng & Qi 2017). For CelebA dataset, we follow the fairness study's s protocol and take Male (Sagawa et al., 2019). UTK dataset is widely used facial dataset in
y Blond Hair, fairness research. In this study, we follow the task outlined in (Hong & Yang 2021). predict y = gender, the sensitive attributes are <(White, not White).
The task is to
We compare our method with ERM and other state-of-the-ar methods without demographics: DRO (Hashimoto al. (0088), ARL (Lahoti al. 2020). KD (Chai et al. 2022). JTT (Liu al. 2021). and LIF (Nam et al. 2020). Please refer to the Appendix for detailed implementation information. In our implementation (Our)) the backbone is the same the ERM model. We normalize and apply the absolute value to Q and K in the last encoder layer. We select V with the 2 highest attention weights to perform the local aligmment on value procedure. For vision tasks, all methods are trained from scratch. We evaluate each method w.r.t accuracy and fairness. For the fairness metric, we use demographic parity (DP), Equal Opportunity (EOp), and equalized odd (EOd), we leave details on the computation of fair metrics in the appendix.
We train all methods on a single NVIDIA RTX-3090 GPU with 24576 MiB memory. Each method is independently trained three times, and we report the mean and standard deviation of the results. We follow a protocol for efficiency assessment in machine learning and report the overall energy consumption, which can be computed by (Wang al. 2023):
E E= 1 u(O)odl (Wh),
(8)
where ul() E [0, (in percent) is the GPU utilization level, H represents the duration of a training process lasting H hours on GPU. po is the power requirement for the GPU. For an NVIDIA RTX- 3090 GPU, Po = 350 W. In practice, we sample u(t) per minute.
Vision experimental results Results from experiments on the UTK and CelebA datasets can be found in Tables [ and 2 We mark the best results in dark blue, and second-best in light blue. In the UTK dataset, our method achieves enhanced fairness performance with a slight decrease in accuracy. In the CelebA dataset, we observe that LfF attains optimal fairness. However, this comes at the expense of a significant reduction in accuracy, where the base rate for y = is 84.67%, and the accuracy of LIF is merely 87.97%. JTT demonstrates balanced performance between fairness and utility. However, in a smaller-scale dataset, the performance of JTT approaches that of ERM training. We hypothesize that this is due to JTT's tendency to overfit the up- up-weighted training set. Our proposed method not only yields outcomes that are comparable to those of JTT, but also effectively mitigates bias existing in smaller datasets. Remarkably, our method requires significantly lower energy consumption compared to JTT. We provide a visualization of attention weight in the last encoder layer. please refer the appendix.
ERM
DRO
ARL
KD
DP
11.66
0.87
13.04 0.77 0.31
9.89 1.02
11.29
: 1.42
11.20
1.07
9.72
0.50
E0p
10.19 = 1.38
10.47
1.01
11.9
+.30
4.69
0.59
11.55
0.14
0.
E0d
9.50
+ 0.92
9.66
4 0.71
11.72
0.37
7.97
0.93
9.31
1.31+1.35
9.39
0.86
7.76
1
0.41
ACC
f
84.26 0.43 83.37
0.66 80.27
+ 1.53
81.33
83.08 = 0.48
80.58
1.03
83.11
11.18 1.38
26.54
12.05
34.77
$5.94
62.20
27.48
Table I: Classification results in terms of accuracy (ACC), fairness (DP, EOp, EOd) and energy
consumption (E(Wh) on UTK dataset: sensitive attribute race, label y gender.
7

---

Published as a conference paper at ICLR 2024
FRM
DRO
ART.
KD
311
Ours
DP
16.9240.55 + 0.55
16.88
+ 0.33
12.74 = 3 46
17.41
0 0.59
18.04
= 0.32
19.95
4
3.50
16.31
199
E0p
12.82
10.21
4.38
41.93
0.47
36.17
1.88
31.79-
9.91
38.01
1.32
E0d 4
22.90
1: 0.10
22.57
1 0.58
21.32
1: 2.65
22.54
1 0.40
20.26
1.72
19.26
3.55
20.43
1 0.68
ACC
94.26
0.06 94.06
0.02 92.55
4: 1.34
94.37
0.28
92.75
0.02
87.97
4.08 94.06
0.43
B(Wh)
144.86
141.75
251.71
267.58
936.24
611.10
172.18
Table 2: Classification results on CelebA dataset: a a= male, y blond hair.
4.2 NLP EXPERIMENTS
Expermental setup We evaluate our method within the domain of natural language processing (NLP) utilizing both the HateXplain (Mathew et al., 202T and MultiNLI (Williams al. 2017) datasets. For the HateXplain dataset, the target label, denoted by y. is equal to for normal sentences and 1 for those that are considered hateful offensive. In line with (Baldini et al.. 2021 for choosing sensitive attributes, we assess whether a given corpus contains information that pertains to the attribute of gender (a). For the MultiNLI dataset, we follow (Liu et al.) 202I). the task is to predict the entailment of the two sentences, y=(entailed, neutral, contradictory, ). The sensitive labels are (no negation, negation).
Pre-trained models have significantly advanced the field of NLP. To ensure an equitable comparison, we leverage pre-trained model and augment with an additional encoder. Specifically, we select "BERT Large" (Devlin et al. (018)) and "BERT Base" (Devlin et al.) (018) to represent large-scale (340M parameters) and medium-scale (100M parameters) models, respectively. For NLP tasks, we exclude ARL, KD, and LfF from the comparison because these methods necessitate an auxiliary network, making it challenging to align with the specifications of pre-trained model. We apply AdamW to optimize the parameters. The learning rates of 10 are used for both "BERT Large" "and "BERT Base". All methods share the same batch size and optimizer configuration. In the baseline model, the add-on layer is the same as one of the encoder layers of the backbone model.
NLP experimental results Table 3 presents results for the HateXp dataset. It can be observed that our model not only achieves optimal fairness but also retains utility without compromising performance. Remarkably, when the backbone is the BERT base model, our method outperforms other methods in fairness and utility. Given the relatively small size of the HateXplain dataset, JTT tends to overfit, leading to suboptimal performance. This observation aligns with the results observed in the CV task.
Table 4 presents the results for the MultiNL dataset. Our method outperforms both the ERM and DRO approaches. However, it falls short in comparison to JTT for fairness. This can be attributed to the base rates of the two groups in MultiNI.I, which are An 92.9% and ' = 7.1%. According to Theorem 1, the upper bound on the disparity in the expected attention allocation is proportional Consequently, highly skewed base rate results in an elevated upper bound.
to No A
ERM (BL)
DRO
JTT
Ous
ERM (BB)
DRO
JTT
Our
DP4
12.38 = 2.78
10.45
0.35
3.32
4
3.00
11.65
0.91
8,85
+ 0,81
8.31 + 1.87
6.12
11.30 + 1.97 4.74
EOp
3.56
6.40
+ 1.10
13.12
3.59
5.31 + 1.38
8.55
1.27
7,80
+ 2.54
9.98
3.28
BOd 1
5.37
+ 1.75
5.53
+ 0.63
7.55
+ 0.95
3.51
+ 1.08
6.62
0.11
5.65
+ 1.72
5.11
1.53
3.40
0.97
ACC
79.69
+ 0.15
79.21
0.26
76.25
79.61
0.32
77.74
77.56
77.21
77.89 0.78
B(Wh)
L
124.32
144.14
265.48
142.51
22.58
36.87
39.75
34,00
Table 3: Fine-tuning on HateXplain dataset. (BL denotes "BERT Large", BB denotes ''EERT base')
IBM
DRO
Ours
ERM
DRO
JTT
DP
47.76 = 0.07
49 1
1.34
43.88
+ 0.21
44.63
= 2.43
47.35 = 0.97
48.72 0.71
44.27
1 0.23
47.
1 0.22
EOp 4
14.71
1.40
12.81
0.86
4.62
0.01
10.30
= 0.16
18.03 = 2.23
17.47
2.82
8.98
+ 1.14
13.86
0.43
10.81
= 0.02
9.95
1 0.95
5.53
0.47
8.59
1 0.34
13.52
1.30
14.11
1 1.05
8.35
1 0.64
12.67
1 0.25
ACC1
84.33
0.02
83.33
82.70
+ 0.08
83.95
= 0.02
81.48 = 0.53
81.63
1:63+000 0.12
76.75
0.600
81.36
+ 0.01
B(Wh)
1827.56
1791.07
5417.70
1934.80
507.64
631.87
1642.90
687.42
Table 4: Fine-tuning on MultiNLI dataset. (BL denotes "BERT Large", BB denotes "'EERT base")
4.3 EXPERIMENT ON THE EFFICIENCY OF GPU MEMORY TRAINING
We evaluate the effectiveness of our method for mitigating bias in pre-trained models. We employ the CelebA and HateXplain datasets, with tasks as delineated in sections 4.1 and 4.2
For the CelebA dataset, we append faimess-aware encoder DINO pre- traitri ViT- 16 model (Caron et al., 202T). For the HateXplain dataset, we incorporate a fairness fairness-wware aware encoder into pre- trained models, specifically "BERT' Large" and "BERT Base". Additionally, for fair comparison,
8

---

Published as a conference paper at ICLR 2024
append standard VIT/ BERT encoder to the ERM model. We evaluate two approaches: 1) fine-tuning all network parameters, denoted as "-FT", and 2) training only the last encoder, denoted as "-LE". We evaluate all methods accuracy, faimess metrics, and GPU memory consumption. The results are shown in 6
We observed that the ERM model maintains comparable utility even when the parameters in the backbone model are frozen. This indicates that pre-trained backbones can effectively extract mean- ingful features. Additionally, only training the last encoder layer demands considerably less GPU memory during training. Furthermore, training only the last encoder layer resulted in enhanced fair- ness outcomes. This improvement is due the fewer parameters updated, potentially mitigating overfitting issues. Notably, Ours-LE approach achieves superior faimess results while preserving comparable utility.
ERM-FT
Ours-FT
ERM-LE
Ours-LE
DP
19.11
18.07
0.71
19.12
0.71
17.10
0.86
EOp
45.18 + 2.77
33.94
+ 0.76
38.84
= 1.79
36.33
+ 1.96
EOd
24.27
1.38
18.73 + 0.34
20.83
= 1.05
19.71
0.86
ACC
95.59
0.09
94.05
0.22
94.98
+ 0.18
91.14
0.17
GPU (MIB))
11792
11926
2925
3210
Table 5: Comparing Fine- Fine-Tuning vs. Training the Last Encoder on the CelebA Dataset.
BERT Large
BERT Base
ERM-FT
Ours-FT
ERM-LE
Ours-LE
ERM-FT
Ours-FT
ERM-LE
Ours-EB
DPL
12.38 + 2.78
11.05
0.91 10.11
2.19
12.74
1.09
8.88 + 0.84
11.30 = 1.97
10.17
2.23 10.29
1.18
EOp 1
5.43 + 3.56
5.31
+ 1.38
8.27
1.68
2.61 + 0.97
8.55
+ 1.27
4.71
3.
6.62
1.80
1.97 = 1.14
E0d 1
5.37
1.75
3.51
4: 1.08
6.67
3.35
2.17 1 1.05
6.62
4 0.44
3.40
1 0.97
6.07
+: 1.62
3.24 : 0.78
ACC
79.69
1 0.15
79.61
0.32
78.74 1 0.74
76.84
0.45
77.74
0.43
77.89 0.78
79.13 0.26
76.12 1 0.05
GPU(MBB)
16411
16493
7022
7253
7830
7899
5303
5393
Table 6: Comparing Fine-Tuning vs. Training the Last Encoder on the HateXplain Dataset.
4.4 FAIRNESS-ACCURACY TRADE-OFF
We explore the effect of hyperparameters. We tune the following hyperparameters: Ours: We change 6, where t is the number of value vectors to perform the align- ment. We choose t E {2, 4, 6); KD: We change À in the loss function that governs how much soft-label is be- ing used. We choose E [0.3, 0.7, 1]: DRO: We change y in the loss function that controls the range of the worst- case groups. We choose E {0.1, 0.3, 0.5}: ARL: We change / the number of encoders stacked for the adversar- ial network. We choose E {2, 4, 6): JTT: We change the up-weight coefficient Aup E {20, 50, 100). Inspired by (Kim et al.] (020)), we depict Pareto frontier to ana- lyze the trade-off between fairness and accuracy. We use CelebA dataset with =male and y =Blond hair. All other hyperparameters remain unchanged with Table 2 All methods share the same seed. The results can be found in Fig
0.31
0.92
Pareto
0.9
Ours
ERM
DEOO
ARL
0.30
0.28
026
0.24
0.2
020
0.8
ais
8od
Figure Effects of hyperparameters on the fairness-accurac tradeoff.
Observations indicate that JTT outperforms in fairness performance. However, its sensitivity to hy- perparameters suggests reduced robustness. Both our method and KD show comparable utility, but our approach outperforms in terms of faimess.. DRO demonstrates less sensitivity to hyperparam- eter variations. Our method is close the theoretical tradeoff line between faimess and accuracy, achieving an optimal balance between fairness and accuracy.
SUMMARY
In this paper, we introduce a novel method to tackle fairness issues in transformers without demo- graphics. To mitigate bias in attention weight allocation, we normalize q and k, then take their absolute values to compute the attention weight. We address bias in value vectors through local alignment. We conduct rigorous theoretical analysis to validate our method. We address fairness issues in pre-trained models and propose method of using an additional fairness-aware encoder to mitigate these concerns. To validate our approach, we conduct experiments on various datasets in computer vision and NLP domains. Our results demonstrate the efficacy of our method in ad- dressing fairness issues. Furthermore, we verify that our approach is useful for improving fairness in mre-trained models even with limited GPU memory resources.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
This work was partially supported by the EMBRIO Institute, contract #2120200, a National Science Foundation (NSF) Biology Integration Institute, and NSF IIS #1955890, IIS #2146091.
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reduc- tions approach to fair classification. In International conference on machine learning, pp. 60-69. PMLR, 2018.
Wael Alghamdi, Hsiang Hsu, Haewon Jeong, Hao Wang, Peter Michalak, Shahab Asoodeh, and Flavio Calmon. Beyond adult and compas: Fair multi-class prediction via information projection. Advances in Neural Information Processing Systems, 35:38747-38760, 2022.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization, arXiv preprint arXiv: 160v:06677.06550. 2016.
Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Mikhail Yurochkin, and Moninder Singh. Your fairness may vary: Pretrained language model faimess in toxic text classification. arXiv preprint arXiv:2108.0125 2021.
Mislav Balunović, Anian Ruoss, and Martin Vechev. Fair normalizing flows. arXiv preprini arXiv:2/06.05937, 2021.
Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre- pre-processing for discrimination prevention. Advances in neural informa- fion processing systems, 30, 2017.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-965000 2021.
Junyi Chai, Taeuk Jang, and Xiaoqian Wang. Fairness without demographics through knowledge distillation. Advances in Neural Information Processing Systems, 15:19122-19144 2022.
Joymallya Chakraborty, Suvodeep Majumder, Zhe Yu, and Tim Menzies. Fairway: way to build fair software. In Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium the foundations of software engineering, pp. 654 664,665,55 2020.
Ruizhe Chen, Jianfei Yang, Huimin Xiong, Jianhong Bai, Tianxiang Hu, Jin Hao, Yang Feng. Joey Tianyi Zhou, Jian Wu, and Zuozhu Liu. Fast model debias with machine unlearning, Ad- vances in Neural Information Processing Systems, 36, 2024.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pp. 1597-1607. PMLR, 2020.
Jiace Cheong, Sinan Kalkan, and Hatice Gunes. Counterfactual fairness for facial expression recog- nition. In European Conference on Computer Vision, pp. 245-261. Sprrinrr,,,,, 2022.
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. What does bert look at? an analysis of bert's attention. arXiv preprint 3xXiv:1906.04330 2019.
Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. In International conference on machine learning, pp. 1436-1445 PMLR, 2019.
Elliot Creager, Jöm-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. International Conference on Machine Learning, pp. 2189-2200. PMLR, 2021.
Jacob Devlin, Ming-Wer Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: /8/0. 04805, 2018.
10

---

Published as a conference paper at ICLR 2024
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Sixue Gong, Xiaoming Liu, and Anil K Jain. Mitigating face recognition bias via group adaptive classifier. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- fion, pp. 1414-3424 2021.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference Machine Learning, pp. 1929-1938. PMLR, 2018.
Youngkyu Hong and Eunho Yang. Unbiased classification through bias-contrastive and bias- balanced learning. Advances in Neural Information Processing Systems, 34:26449-26461, 2021.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456. pmlr, 2015.
Taeuk Jang, Feng Zheng, and Xiaoqian Wang. Constructing fair classifier with generated fair data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 7908-7916, 2021.
Taeuk Jang, Pengyi Shi, and Xiaoqian Wang. Group-aware threshold adaptation for fair classifica- tion. In Proceedings the 2022.
the AAAI Conference on Artificial Intelligence, volume 36, pp. 1988-6995,
Ray Jiang. Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty artificial intelligence, pp. 862-872. PMLR, 2020.
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrim- ination. Knowledge and information systems, 33(1):1-33, 2012.
Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. Fairness- aware learning through regular- ization approach. In 2011 IEEE 11th international conference on data mining workshops, pp- 643-650. IEEE, 2011.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 13:18661-18673 2020.
Joon Sik Kim, Jiahao Chen, and Ameet Talwalkar. Fact: A diagnostic for group fairness trade-offs. In International Conference on Machine Learning, pp. 5264-5274. PMLR, 2020.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.029337 2022.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Chi. Fairness without demographics through adversarially reweighted learning. Advances in neural information processing systems, 33:728-777 2002887-0.000 2020.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) December 2015.
11

---

Published as a conference paper at ICLR 2024
Francesco Locatello, Gabriele Abbati, Thomas Rainforth, Stefan Bauer, Bernhard Schölkopf, and Olivier Bachem. On the fairness of disentangled representations. Advances in neural information processing systems, 32, 2019.
Andrew Lowy, Rakesh Pavan, Sina Baharlouci, Meisam Razaviyayn, and Ahmad Beirami. Fermi: Fair empirical risk minimization via exponential rényi mutual information. 2021.
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 14867-14875, 2021.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De-hiasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33:20673-20663 2020.
Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang. Dohyung Kim, and Hyeran Byun. Fair contrastive learning for facial attribute classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10389-10398, 2022.
Felix Petersen, Debarghya Mukherjee, Yuckai Sun, and Mikhail Yurochkin. Post-processing for individual fairness. Advances in Neural Information Processing Systems, 34:25944-25955, 2021.
Yao Qiang, Chengyin Li, Prashant Khanduri, and Dongxiao Zhu. Fairness-aware vision transformer via debiased self-attention. arXiv preprint arXiv:2301. 13803, 2023.
Maan Qraitem, Kate Saenko, and Bryan A Plummer. Bias mimicking: A simple sampling approach for bias mitigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2031 1-20320, 2023.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang, Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. arXiv preprint 19XX::991100773111 2019.
Mhd Hasan Sarhan, Nassir Navab, Abouzar Eslami, and Shadi Albarqouni. Fairness by learning orthogonal disentangled representations. In Computer Vision-ECCV 2020: 16th European Con- ference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIX 16, pp. 746-761. Springer, 2020.
Xiaobin Song, Zeyuan Liu, and Benben Jiang. Adaptive boosting with fairness-aware reweighting technique for fair classification. arXiv preprint arXiv:2401.03097, 2024.
Sruthi Sudhakar, Viraj Prabhu, Arvindkumar Krishnakumar, and Judy Hoffman. Mitigating bias in visual transformers via targeted alignment. arXiv preprint arXiv:2302.04338, 2023.
Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In Computer ECCcnnE0 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pp. 776-794. Springer, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention you need. Advances in neural informa- tion processing systems, 30, 2017.
Christina Wadsworth, Francesca Vera, and Chris Piech. Achieving faimness through adversarial learning: application recidivism prediction. arXiv preprint arXiv:1807.00119, 2018.
Mingyang Wan, Daochen Zha, Ninghao Liu, and Na Zou. In-processing modeling techniques for machine learning faimess: A survey. ACM Transactions Knowledge Discovery from Data, 17 (3):1-27, 2023.
Hao Wang, Luxi He, Rui Gao, and Flavio Calmon. Aleatoric and epistemic discrimination: Funda- mental limits of fairness interventions. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Published as a conference paper at ICLR 2024
Sid Wang, John Nguyen, Ke Li, and Carole-Jean Wu. Read: Recurrent adaptation of large trans- formers. arXiv preprint arXiv:2305. 15348, 2023.
Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
8919-8928, 2020.
Dennis Wei, Karthikeyan Natesan Ramamurthy, and Flavio P Calmon. Optimized score transfor- mation for consistent fair classification. Journal of Machine Learning Research, 22(258):1-78, 2021.
Adina Williams, Nikita Nangia, and Samuel R Bowman. broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426, 2017.
Ruicheng Xian, Lang Yin, and Han Zhao. Fair and optimal classification via post-processing. In
International Conference on Machine Learning, pp. 37977-38012. PMLR, 2023.
Jingjing Xu, Wangchunshu Zhou, Zhiyi Fu, Hao Zhou, and Lei Li. A survey on green deep learning. arXiv preprint (rXXi:22/1.05//00 2021.
Yihao Xue, Kyle Whitecross, and Baharan Mirzasoleiman. Investigating why contrastive learning benefits robustness against label noise. In International Conference Machine Learning, pp- 14851-24871. PMLR, 2022.
Ruichen Yao, Ziteng Cui, Xiaoxiao Li, and Lin Gu. Improving fairness in image classification via sketching. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022, 2022.
Yuanshun Yao and Yang Liu. Understanding unfairness via training concept influence. arXiv preprint arXiv:2306. 17828, 2023.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P Gummadi. Fair- constraints: A flexible approach for fair classification. Journal of Machine Learning Re- search, 20(75):1-42, 2019.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning, pp. 325-333. PMLR, 2013.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adver- sarial learning. In Proceedings of the 2018 AAAI/ACM Conference Al, Ethics, and Society, pp. 335-340, 2018
Song Yang Zhang, Zhifei and Hairong Qi. Age progression/regression by conditional adversarial autoencoder. In IEEE Conference Computer Vision and Pattern Recognition (CVPR). IEBE, 2017.
Dominik Zietlow, Michael Lohaus, Guha Balakrishnan, Matthäus Kleindessner, Francesco Lo- catello, Bernhard Schölkopf, and Chris Russell, Leveling down in computer vision: Pareto ineff- ciencies in fair deep classifiers. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 10410-10421, 2022.
13

---

Published as a conference paper at ICLR 2024
A EMPIRICAL VALIDATION ASSUMPTION
We use ERM train VIT the CelebA dataset. We save the model that achieves the highest validation accuracy. From the last encoder, we extract qCLS and K where "CLS" refers to a special token and K 1o represents the key vector of the 10-th token. We randomly select one dimension from ICLS and K in to generate two conditional distribution plots based on a.
q
4p-o
qja=1
kp=1
0.8
Dersity
Derselt
0.4
0.4
0.0
-1.5
-1.0
-0.5
0.0
0.5
3.0
0.0
second of
The elementin in second dimension the 10th patch.
Figure 3: Histogram of a random selected dimension of qets and k
B PROOF FOR THEROREM
We first consider a 1-D case and next generalize the derivation to dy dimensions.
For the 1-D derivation,
focus only on g, then
g ~ NoN on) + A1N
(9)
where Ao + \ 1. Then the expectation of is
AnHoo + A1 / Elq]
(10)
and the variance can be written as
0² = Var(ql =B]g²] (B(a))?,
(11)
=>glR [q²] X F. N (NoHo + A1
(12)
=>a(ug + og) + A1 + +of) (AoHo + A1m1)²
(13)
=(Aoo) -v1o)) + do (nO m1)²
(14)
=(Nuo) >1oo) + d= WLOG, let A p1 11
(15)
After normalization with morm are still Gaussian, and the conditioned expectations can be written as
Eig
the conditioned variable qmorm a = and g norm
/arlg
P0,norm = norm 0j 14 1
A
(16)
+ (Aoo
+ dodi
M1,norm == Ely norm 1| P1 ı
An
(17)
V(Noo + x11of) + Aod1
and the corresponding variances are
Tinnar
i= 0,
(18)
14

---

Published as a conference paper at ICLR 2024
After taking the absolute values, that is qth |g the conditioned expectations can be written
as
E Elgue a = = S tp(t)dt + S/² tp(t)dt, p(t) = norm R, 0,1
(19)
2 cxp(
Minnorm )+ + Hi,norm Pi 20 n.oom
(20)
v 20
π
(21)
Recall that Pimorm 14-1 and Jsnorm = 04/0, we have
Blg
2
exp(
+ -
1 erf(
(22)
2 0 )2
12
2
Mi 14 (Hi + (Mi -perr((
Mi 1
V20, )
exp(
(23)
π
V201
2
X1-14
X1-14
exp( (-(
)2) + (Arfi
(24)
V2a;
/20z
which is because μl - = p1 - \1pl Aopa >o uo)
A and 1r - 1 po A1
AuHoo = A2 M1) = A And (-x)erf(-22 -I) = zerf(x).
Note that A= 0, we have exp(- ( < 1 and erf( )1= < 1. As result, the expecta-
/20,
V20,
tion can be bounded by
Eig la
il
<
+ 0
(25)
oiV
3
A1-1
(26)
VAoog +
+ (fof) \o
VDooog + {fof)
+ Andi
0./2
2
A
lim
lim A+0
(27)
+00
V(Nuog + \20 + Aodi
Vovoo + )noo)) + 2
2
11-- d
(28)
TÀ
Similar results hold for k, too. Now the original statement can be re-written as
Pgn Meg de delkki.dd .ee
(29)
Elga de la = E| 11: la = 0] Elq de a = E| de =
1]
(30)
<Elgde la- = 0) OjElkeele = 0] + Elgde la = de 1]
(31)
2
A1
2
2
Ao 2
+
+
V
+ V A1
(32)
TÀu
Au
15

---

Published as a conference paper at ICLR 2024
Now consider the high- digh-dimensional scenario, we have
de
k
de
de
de
8dde|
E
a
0]
E
1]
(33)
dx
dx
dx
d2
E0
de ked
Ila
0]
gg
de kde) )1 a
11
(34)
v dz
j=1
dg
dy
E (3
de Rde la =0-
Elaj
de dede
Kig la 11
(35)
d2
j=1
du
[3] fe high a =0 - -B(ug la 11}
(36)
ddk
d2
E8 [gj
de de a 0] - E la 11
(37)
v dd
d2
EB l0 a = 0JBTk40 aa = 0]
Eloga |a = 1| 1a 11
(38)
vd 5=1
Svde (V
2
21)²
+ (V
2
2
do 27
+ V Ao
+
(39)
Td1
A1
C IMPLEMENTATION DETAILS
We summarize the implementation details as follows:
We take the transformer with 8 stack encoders as the baseline model. Each encoder has 8 attention heads. We implement the ERM model using the Huggingface library with- out pre-trained weights. We resize the image from CelebA and UTK datasets to 64 x 64, and divide it into 16 patches. For CelebA and UTK, we take the Adam as the optimizer with learning rate of 10-1, and no scheduler is applied for the fair comparison. For NLP tasks, we take the AdamW as the optimizer with a learning rate of 10 5, We share all methods with the same configuration as the ERM model.
ERM:
Distributionally robust optimization (DRO): We adapt the backbone to the same as the ERM model. We tune the hyper-parameter 1) at the validation set to achieve the highest accuracy. For CelebA and UTK experiments, we set 1) 0.15 and v 0.10 respectively. For HateXplain and MultiILI, we set 1) = 0.25.
Adversarially reweighted learning (ARL): We adapt the learner network to the same as the ERM model. For the adversarial network, we apply a 6-layer stack encoder in the trans- former. This is a smaller configuration compared to the learner network, as recommended by the authors.
Fairness without demographics through knowledge distillation (KD): We adapt the student model that is the same as the ERM model. For the teacher network, we follow the sugges- tion of the authors that use larger network, hence we adopt the vision transformer with 12 stacked encoder layers. The student network is trained by the output of the teacher model with softmax activation.
Just train twice (JTT): We adapt the backbone network is the same as the ERM model. For CelebA we follow the suggestion in the paper choose the number of epochs of training the identification model T 1. During the second training, choose the upsampling factor Aup = 50. For UTK, we set T = 10, Aup = 50. For HateXplain, we set T = 20, Aup = 50. For Multi we set T 2, Aup 20.
Learning from failure (LfF): We take the networks for a biased model and a debiased model are the same as the ERM model. We set the amplification coefficient in generalized cross entropy loss q 0.7 suggested in the original paper.
16

---

Published as a conference paper at ICLR 2024
For vision tasks, method employs backbone identical to the ERM model, which comprises an 8-stacked encoder vision transformer. the final encoder, we normalize and apply the absolute value to q and k for each head. From this encoder layer, select the v vectors associated with the two highest attention weights and execute a local alignment. For NLP tasks, extend pre-trained model with additional encoder layer, applying our method specifically to this added layer. Similarly, in the NLP tasks, we choose the v vectors with the two highest attention weights for alignment.
Ours:
D EVALUATION METRICS
The group fairness metrics are measurements of the performance of different sensitive groups. We focus on three specific metries: Demographic Parity, Equal Opportunity, and Equalized Odds. De- mographic Parity (DP): DP focuses on the equality of the outcomes across different demographic groups, regardless of their abilities. It ensures that each group receives positive outcomes at the same rate. Equal Opportunity (EOp): EOp ensures that samples who should receive positive outcome have an equal chance being correctly identified, regardless their group. Equalized Odds (EOd): EOd requires both that samples that should receive positive outcome have equal chance of being correctly identified (like EOp), and also that samples that should receive a negative outcome have an equal chance of being correctly identified, across all sensitive groups. The computations for the fairness metrics are as follows:
DP PPP- - PPPl: EOp |TRR TPRs,
EOd 1 - + FPRR FPRg)), i
(40)
where PP,TPR, and FPR are the positive prediction rate, the true positive rate, and the false positive rate.
E ATTENTION WEIGHT VISUALIZATION
ERM (=),
Original
(b) a-1
Original
Ortginal
(d)
Figure Visualization of attention weight. y -blond hair, male in CelebA.
We present a visualization of the average attention weight in the last encoder layer for both the ERM and our proposed models in Fig We opted for models that demonstrated the highest validation accuracy. The ERM model achieved 94.08%. We observe inconsistencies in attention allocation using the ERM training objective func- tion. Despite its high accuracy, the ERM model predominantly focuses facial features, failing to distribute attention adequately. In contrast, our model provides a more uniform attention allocation, effectively reducing the focus on facial features other irrelevant features.
accuracy of 94.27%, our model achieved accuracy of
17

---

Published as a conference paper at ICLR 2024
F ABLATION STUDY
We perform an ablation experiment from three perspectives to understand their importance. w/o local alignment: We train our model without incorporating the local value alignment technique. The training process is solely guided by the cross-entropy loss. w/o debias attention: While training our model, we exclude the normalization and exclude the absolute values in vectors q and when calculating the attention weight. w/o absolute value: We train our model using normalized vectors and k to compute the attention weight, but we exclude applying the absolute value on these vectors. We use CelebA dataset with male and = Blond hair. All the methods share the same seed with the Baseline model.
Method
DPL
EOp 4
EOd 4 ACCT
Baseline
16.99
43.04
23.05
94.20
w/o local alignment
18.70
39.59
21.72
93.94
w/o debias attention
16.59
43.36
22.92
94.63
w/o absolute value
18.24
37.44
20.59
94.39
Ours
15.81
36.49
19.47
93.96
Table 7: Ablation study on CelebA dataset.
Table 7 demonstrates the efficacy of our design modules. Notably, without the debias attention, the outcomes are close those of the Baseline model. This highlights the significance of debias attention as an essential component for debiasing attention. Concurrently, the local value alignment technique further improves fairness, with a large impact on EOd. When the network integrates both techniques, it achieves optimal faimess with a slight drop in accuracy.
G ALGORITHM
We summarize our training algorithm in Algo
H LIMITATION
In the realm of both computer vision and natural language processing, the transformer architecture stands pivotal framework. Our study introduces method specifically designed for transformers, avoiding the use of sensitive information. We recognize that our approach's 's primary limitation lies in its application to binary-sensitive attributes, simplification compared the multifaceted nature of real-world scenarios. Expanding our methodology to accommodate arbitrary sensitive attributes represents a significant and promising avenue for future research.
18

---

Published as a conference paper at ICLR 2024
Algorithm Debias Attention mechanism
Input: Input tokens X R N Xdduss label y. Weight matrices Wo E Rudds xd, WK E Rd de. Wv E IR daank xdy for Query, Key, Value. Output: The loss of local alignment on value
Project X to Query, Key, and Value matrices: Q XW K XWK, V XWv 2: For each q E Q. k E K:
q = m = kde = m n(k)
m and n are defined in Theorem 1. 3: Compute attention weights wy:
de 9als
Kde
W/-
vdg
4: Let W = {W1, ...... wi / where is the number of tokens. Select value vectors corresponding to the top attention weights:
arg max
w. s.t. w, E W,W, |||s =
EEB
5: Extract and concatenate the top f value vectors:
v + concat(VV-)
6: Mapping to the contrastive space:
g(v)
7:
L Lalignment
Σ P(t)
log
exp(z; /2 2p/T)
ELEB( exp z; 24 24/T)
(EP(()
where i E I= (1, n), B(i) = -M.), P(i) {p E B(i) :3p = y1), T = 0.07.
return Caligmment
19


Reviews:
[{"review_id": "IzF7Uz8dHP", "reviewer": "Reviewer_JY1q", "summary": "**Rebuttal Update** After reading the author's rebuttal, I improved my score from a 5 to a 6. However, I would like to see the writing and clarity improve for a final paper if accepted. Specifically, the introduction and use of notation needs cleaning, as well as the motivations in the introduction. I would also like a bit more explanation of the contrastive learning method and its motivations in the context of the whole work.\n\nIn this paper, the authors propose a new method for debiasing the attention mechanism to achieve fairness without prior subgroup definitions. Their method consists of two components. First, they normalize the token embeddings in their Query and Key matrices and absolute value them to bound the attention weight difference across sensitive attributes. Second, they use a contrastive loss to encourage the embeddings of samples from the same class to be similar to each other, encouraging equal representation across sensitive attributes while maintaining performance. Then, the authors provide extensive empirical evaluation of their fairness method across two vision and two language tasks.", "questions": "What is the practical/fairness motivation behind the fairness optimization problem, and the motivation behind debiasing attention as a whole? \n\nAlso, it seems as though instead of using subgroup attributes in the contrastive loss, you use classes. However, what happens if each class is dominated by a single sensitive group attribute? For example, if class 1 is all male and class 2 is all female, then requiring all class 1 (male) representations to be similar with each other and different than class 2 (female) will actually cause more disparity by pushing the representations away from each other. Don't we have to assume sensitive groups are balanced within classes?\n\nFinally, just as a curiosity, won't normalizing reduce the expressivity of the model? I would love to see the distribution of attention weights before and after normalizing, as well as before and after absolute valuing (as well as an ablation study of the two steps).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "This paper builds on a wide literature of debiasing methods, and does a great job of synthesizing methods from fairness aware transformers with methods in fairness without demographics. Furthermore, the method is straightforward and the first part can be applied easily to any attention mechanism, whereas the second requires a small amount of fine tuning of a nonlinear head. The entire system can be applied out of the box to existing models with little or no training. Another thing I really liked about the paper is the green analysis of the power consumption of the method. I think this is a great step forward for researchers in the field.", "weaknesses": "One weakness of the paper is the motivation. While debiasing models / fairness is generally a strong motivation, the authors do not explain in much detail why it is important to debias attention mechanisms. Instead, most of the introduction feels like a \"related works\" section, where the main motivation of the paper is the failings of previous methods. I would like to see a better written introduction that explains why the failings of previous methods are bad or costly, and why a new method is needed. \n\nOther parts of the paper are not very clear as well. What motivates the fairness optimization problem (2)? Why do we care about attention weight values when in fairness we traditionally care about outcomes? While having the same activations results in the same outputs of course, it seems to me to be extreme to limit the expressivity of the model across sensitive attributes.\n\nFurthermore, the notation in the paper is pretty messy and unclear. In section 3.1, in the first paragraph the vectors $\\textbf{q}, \\textbf{k}$ are not introduced as slices of the $Q$ and $K$ matrices. Also the relationship between the dataset $\\mathcal{D}$ and how it is inputted into the attention mechanism/transformer model is not mentioned at all (it just jumps straight from dataset notation to attention notation with no connection between the two). In Section 3.2 near the end, what is $q_{cls}$?\n\nI would like to see a better figure explaining the pipeline or mathematical equation of the entire model. $g$ is only mentioned before eqn. (7), but not ever shown visually, this makes it very unclear as to how to implement the second modification and is a bit misleading as we require an additional layer to train to align the model. \n\nFinally, I would like to see an ablation study of the two mechanisms to compare which one impacts fairness more.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hAjEF39pBN", "reviewer": "Reviewer_sKv2", "summary": "This work focuses on the attention mechanism to debias the transformers without assuming access to the sensitive attribute information. For this, two steps have been performed. First, they propose a weight relocation mechanism by normalizing (subtracting mean and dividing by standard deviation) and taking the absolute value of query ank key vectors in the attention module. \nA theoretical insight is also provided to show that this bounds the discrepancy for various sensitive attributes. \nThen, a nonlinear mapping is applied on tokens with higher attention values to map them to a latent representation $v \\rightarrow z$. Then, supervised contrastive learning is applied to the latent representation to make sure that the embedding for the samples from the same class has core similarity.", "questions": "please refer to weaknesses", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper is generally written well and easy to follow, even though some parts are missing (will discuss later).\n\nThe idea of focusing on the attention module and debiasing the transformer without assuming access to the sensitive attributes is really interesting. \n\nAs this approach is not computationally intensive (based on the experimental results provided in section 4) and is not making significant changes to the architecture, it can be easily added during training to most of the transformer-based structures and improve fairness.", "weaknesses": "This paper has some weaknesses and I believe addressing these weaknesses can improve the quality of the paper:\n\n1. Intuitively I understand that as the attention mechanism shows the importance of different patches during training/ inference, it can have a large impact on introducing bias. However, authors need to justify in a more systematic way, why using only the attention mechanism is powerful enough to debias a transformer structure. As an example, an analysis in a controlled setup (when we have access to sensitive attributes) can be provided to show that the attention module is the one that mostly affects the bias, or similar experiments to justify this.\n\n2. Similarly, authors need to justify why the optimization objective in Eqn. (2) (minimizing the disparity in attention weights) can be a good approximation to the fairness metrics? I believe there should be an approximation error as instead of considering the output, we are just considering the attention weights. A detailed analysis is required to justify this alternative definition.\n\n3. The statistics of the $q$ and $k$ are estimated during training and then used as an estimation during inference. Authors should provide more details on the accuracy of this choice, as the distribution shift during training and inference might affect both fairness and classifier performance.\n\n4. In section 3.4., more details are required regarding debiasing the pre-trained network by inserting the encoder layer. This part was not clear to me.\n\n5. The experimental results are not convincing enough as a very limited number of combinations for label $y$ and sensitive attributes $A$ are used. \n- Authors should provide various combinations to show the generalizability of the results. \n- In addition, for different datasets, different combinations are used which may give a bad impression of cherry picking.\n- On average, the proposed method is not better than previous approaches and the main benefit is less compute. I wonder wether this approach can be combined with previous methods to give better performance in terms of reducing bias and preventing the degradation in the classifier accuracy?\n\nsmall typo in Figure 1: dotted line $\\rightarrow$ solid line\n\nI am willing to increase my score if the authors provide proper responses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wlhZtX3ANV", "reviewer": "Reviewer_mDe1", "summary": "This paper proposes a new method to address fairness issues in vision transformers and natural language processing transformers without requiring access to sensitive demographic attributes during training. The key contributions are:\n\n- They identify two sources of bias in transformers: misallocation of attention weights and bias in value vector representations. \n\n- To address attention weight bias, they normalize and take the absolute value of query and key vectors before computing attention. This is motivated by theoretical analysis showing it reduces disparity in attention weights between groups.\n\n- For value vector bias, they use a supervised contrastive loss on the core value vectors to encourage consistency between groups. \n\n- The method is evaluated on vision and NLP tasks, showing improved fairness metrics compared to prior work without demographics. It also enables efficiently debiasing pretrained models by only retraining the last encoder layer.\n\n- Overall, the method provides a simple and effective way to improve transformer fairness without needing sensitive attributes, auxiliary networks, or restrictive constraints. The ablation studies demonstrate tradeoffs between fairness and accuracy.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper tackles an important problem - improving fairness in transformers without needing sensitive attributes. This is challenging but highly relevant given privacy regulations. \n\n- The approach of debiasing attention weights and value vectors specifically is novel. Prior work either operates on the full representations or relies on adversarial training, which can be unstable. Deconstructing the transformer in this way is creative.\n\n- The method is simple, leveraging existing operations like normalization and contrastive loss. Avoiding complex auxiliary networks is a plus for efficiency.\n\n- Results on vision and NLP datasets demonstrate improved fairness over prior art like DRO and knowledge distillation. The method also enables efficient debiasing of pretrained models.\n\n- Theoretical analysis provides justification for the attention weight debiasing, and empirically shows the approach achieves near optimal fairness-accuracy tradeoffs.", "weaknesses": "- For NLP, the scheme of picking top value vectors may be less effective for long sequences. Dynamic selection based on attention may work better.\n\n- The last layer retraining is convenient but provides no guarantees. Analyzing how bias propagates through the full network could further improve this.\n\n- The contrastive loss operates locally on values. Extending the alignment more globally could potentially improve fairness further without sacrificing accuracy.\n\n- No rigorous ablation study is provided to analyze the individual effects of the attention and value debiasing components. Their relative contributions are unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DfIZgxXri6", "reviewer": "Reviewer_DJw2", "summary": "The paper aims at the fairness issue when deploying models. This paper presents an approach to debiasing transformers using their inherent structure. The authors propose some methods to handle the queries, keys and values to reduce the bias. Also, the memory efficiency in the training phase is enhanced.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The overall writing is clear. \n2. The problem of fairness issue when deploying models is important.", "weaknesses": "The biggest problem is the experiment settings and results.\n\n1. In the experiment tables, which method belongs to \"Fairness without Demographics\", which method requires Demographics?\n2. The proposed method cannot beat SOTA methods. For example, in Table 2, the proposed method is worse than LfF on EOp dataset. In Table 4, the proposed method is worse than JTT method on EOp and EOd dataset.\n3. Following problem 2, the authors may claim that they have much less energy consumption. However, the comparison is not straightforward. The authors need to show one of the two results to claim this point: 1) same energy consumption and higher accuracy; 2) same accuracy and less energy consumption. If the authors can show these results compared to LfF on EOp dataset, and JTT method on EOp and EOd dataset. I will raise my score.\n4. Energy consumption is not a stable indicator when comparing the models. The hardware may have big influence. I would recommend to use FLOPs to compare these methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)