FAIRER: Fairness as Decision Rationale Alignment
Authors: Tianlin Li
Qing Guo
Venue: Submitted to ICLR 2023 (2023)
Keywords: Fairness
deep neural network
decision rationale alignment
neuron parity score

Abstract: Deep neural networks (DNNs) have achieved remarkable accuracy, but they often suffer from fairness issues, as deep models typically show distinct accuracy differences among some specific subgroups (e.g., males and females). Existing research addresses this critical issue by employing fairness-aware loss functions to constrain the last-layer outputs and directly regularize DNNs. Although the fairness of DNNs is improved, it is unclear how the trained network makes a fair prediction, which limits future fairness improvements. In this paper, we investigate fairness from the perspective of decision rationale and define neuron parity scores to characterize the fair decision process of networks by analyzing neuron behaviors in various subgroups. Extensive empirical studies show that the unfair issue could arise from the unaligned decision rationales of subgroups. Existing fairness regularization terms fail to achieve decision rationale alignment because they only constrain last-layer outputs while ignoring intermediate neuron alignment. To address the issue, we formulate the fairness as a new task, i.e., decision rationale alignment that requires DNNs’ neurons to have consistent responses on subgroups at both intermediate processes and the final prediction. To make this idea practical during optimization, we relax the naive objective function and propose gradient-guided parity alignment, which encourages gradient-weighted consistency of neurons across subgroups. Extensive experiments on a variety of datasets show that our method can improve fairness while maintaining high accuracy and outperforming other baselines by a large margin. We have released our codes at https://anonymous.4open.science/r/fairer_submission-F176/.

Under review as a conference paper at ICLR 2023
FAIRER: FAIRNESS AS DECISION RATIONALE ALIGNMENT
Anonymous authors Paper under double-blind review
ABSTRACT
Deep neural networks (DNNs) have achieved remarkable accuracy, but they of- ten suffer from fairness issues, as deep models typically show distinct accuracy differences among some specific subgroups (e.g., males and females). Existing research addresses this critical issue by employing fairness-awar loss functions to constrain the last-layer outputs and directly regularize DNNs. Although the fairness of DNNs is improved, it is unclear how the trained network makes fair prediction, which limits future fairness improvements. In this paper, we investi- gate fairness from the perspective of decision rationale and define neuron parity scores to characterize the fair decision process of networks by analyzing neuron behaviors in various subgroups. Extensive empirical studies show that the unfair issue could arise from the unaligned decision rationales of subgroups. Existing fairness regularization terms fail to achieve decision rationale alignment because they only constrain last-layer outputs while ignoring intermediate neuron align- ment. To address the issue, we formulate the fairnes as a new task, i.e., decision rationale alignment that requires DNNs' neurons to have consistent responses on subgroups at both intermediate processes and the final prediction. To make this idea practical during optimization, we relax the naive objective function and propose gradient- uuddenttguided consistency of neurons across subgroups. Extensive experiments datasets show that our method can improve fairness while maintaining high accu- racy and outperforming other baselines by a large margin. We have released our codes at https://monymoous.ppe...... wccnppppeescccccccaaaaiiererrr http://frrrrrruumissiioooooo submission-Fll F176//
guided parity aligument, which encourages pradient-weightee
variety of
INTRODUCTION
In the current society, there a desperate desire for social fairness among individuals. However, as deep learning is increasingly adopted for many applications that have brought convenience to our daily lives (He et al., 2016; Devlin et al., 2019; Deng et al., 2013), DNNs still suffer from the fairness problem and often exhibit undesirable discrimination behaviors (News, 2021; 2020). For example, for an intelligent task (e.g., salary prediction), a trained DNN casily presents distinct accuracy values in different subgroups (e.g., male and female). The discriminatory behaviors contradict with people's growing demand for fairness, which would cause severe social consequences. To alleviate such fairness problems, a line of mitigation strategies has been constantly proposed (Zemel al., 2013; Sarhan et al., 2020; Wang et al., 2019).
direct regularization method to improve fairness is relax fairness metrics as constraints in the training process (Madras et al., 2018). This regularization method is designed to reduce the disparities between different subgroups in the training and testing data (See Fig. 1 (a) vs. (b)). Although this method easily improves the fairness of DNN models, it is still unclear how the trained network makes fair decision. For example, we do not know how the fairness regularization terms actually affect the final network parameters and let them make a fair prediction. Without such an understanding, we would not know the effective direction for further fairness enhancement. Existing work does not address this question and the majority of them concentrate the last-layer outputs (i.e., predictions) while ignoring the internal process. In this work, we propose to study the fairness from the perspective of decision rationale and analyze existing faimess-regulariiee methods through a decision- decision-rationale aware analysis method. The term 'decision rationale' is known as the reason for making decision and could be represented as the behaviors of neurons in DNN (Khakzar et al.,

---

Under review as a conference paper at ICLR 2023
Figure 1:
Schemattc
Inalegt one supale A&A
diagrams existing
solutions and the proposed
de le bunddyy
seaple
(a) and
Busble reloral
(b) represent results the
11
111
standard trained network
the regularized faimess network. (c) show the results the decision rationale- aligned
A-1
APg
dg Regaberie
Alignal network.
2021). Specifically, for each intermediate neuron (i.e., a parameter of the DNN), we can calculate the loss change on subgroup before and after removing the neuron. As a result, can characterize the decision rationale of a network on the subgroup by collecting the loss changes of all neurons. For example, the solid green and yellow lines in Fig. represent the neurons leading high loss changes at each layer and characterize the decision rationales of the two subgroups. Then, we define the neuron parity score the decision rationale shifting across different subgroups. which actually reveals the influences of intermediate neurons (i.e., parameters) to the decision rationale changes. With the new analysis tool, we find that the network fairness is directly related to the consistency of the decision rationales different subgroups and existing fairness regularization terms could partially achieve this goal (Compare the solid lines in Fig. (b)) since they only add constraints to regularization terms to minimize parity scores
the final outputs. Intuitively, we could define of all neurons and encourage them to have similar behaviors across subgroups. We name this new task the decision rationale alignment that requires DNNs to have consistent decision rationales as well as final predictions on different subgroups. Although straightforward, the task is challenging for reasons: First, the decision rationale and parity score defined based a dataset and it is impractical to calculate them at each iteration during the training process. Second, different neurons have different effects on fairness and such differences should be carefully considered.
To address the above two challenges, we propose the gradient-guider parity alignment method by relaxing the calculation of decision rationale from the tataset-based strategy the sample-based one. As a result, the corresponding regularization term is compatible with the epoch-based training process. Moreover, we use the first-order Taylor expansion to approximate the parity score between decision rationales and the effects of different neurons to the fairness are weighted via their gradient magnitudes automatically. Overall, the proposed method can achieve much higher fairness than tftc-offthhe-rr methods. In summary, the work makes the following contributions:
To understand how a network makes a fair decision, we define neuron parity score to characterize the decision rationales of the network on different subgroups. We reveal that the fairness of a network is directly related to the consistency of its decision rationales on different subgroups and existing regularization terms cannot achieve this goal.
2. To train fairer network, we formulate the decision rationale alignment task and propose the gradient-guiie guided parity alignment method solve by addressing the complex optimization challenges.
3. Extensive experiments on three public datasets, Le.,, Adult, CelebA, and Credit, demonstrate that our method can enhance the faimess of DNNs effectively and outperform others largely.
2 RELATED WORK
Fairness in deep learning. There different methods to evaluate fairness in deep learning, among which individual fairness (Zhang et al., 2020; 2021; George John et al., 2020), group fairness (Louppe al., 2016; Moyer et al., 2018; Gupta et al., 2021; Garg et al., 2020), and counterfactual fairness (Kus- ner et al., 2017) are the mainstream. We focus on group fairness which is derived by calculating and comparing the predictions for each group. There is line of work dedicated to alleviating unjustified
2

---

Under review as a conference paper at ICLR 2023
bias. For example, Wang et al. (2020) compare mitigation methods including oversampling, adversar- ial training, and other nomain-independee methods. Some work proposes to disentangle unbiased representations to ensure fair DNNs. On the contrary, Du et al. (2021) directly repair the classifier head even though the middle representations are still biased. To improve fairness, it is also popular to constrain the training process by imposing regularization. Woodworth al. (2017) regularize the covariance between predictions and sensitive attributes. Madras et al. (2018) relax the fairness metrics for optimization. Although such methods are easy be implemented and integrated into the training process, these constraints suffer from overfitting (Cotter et al., 2019). The model with large number of parameters could memorize the training data, which causes the fairness constraints fit well only in the training process. Chuang & Mroueh (2021) ensure better generalization via data augmentation (e.g., mix-up) to reduce the trade-off between fairness and aceuracy.
However, these methods barely pay attention to the rationale behind the fair decision results. In this paper, we further analyze the decision rationales behind the fair decision results in the training process and reveal that ensuring the fair decision rationale could further improve fairness.
Understanding DNNs decision rationale. There are some interpretable methods enabling DNNs models to present their behaviors in understandable ways to humans (Zhang & Zhu, 2018; Fong & Vedaldi, 2017; Koh & Liang, 2017). Specifically, there is a line of work that decompose DNNs to depict the decision rationale. Routing paths composed of the critical nodes (e.g. neurons with the most contribution to the final classification on each layer) can be extracted in a learnable way to reflect the network's semantic information flow regarding a group of data (Khakzar et al., 2021). Conquering the instability existing in the learnable method, Qiu et al. (2019) propose an activation based proppppaagttion algorithm decompose the entire DNN model into multiple components composed of structural neurons. Meanwhile, Xie et al. (2022) base the model function analysis on the neuron contribution calculation and reveal that the neuron contribution patterns of OOD samples and ad adversarial samples are different from that of normal samples, resulting in wrong classification results. Zheng et al. (2022) analyze neurons sensitive to individual discrimination and generate testing cases according to sensitive neuron behaviors. However, these post-hoc methods decompose the DNNs and extract the neuron contributions pattern via static analysis or in a learnable way. These analysis methods result in huge time overhead, making their integration into the training process difficult, which restricts these methods to be applied in optimizing the training process.
In our paper, follow the spirit of decomposing DNNs to understand the model decision rationale. Different from these previous methods, our method successfully simplifies the estimation process of neuron contribution and can be easily integrated into the training process to optimize the model.
3 PRELIMINARIES: GROUP FAIRNESS VIA REGULARIZATION LOSSES
3.1 PROBLEM FORMULATION
In general, given a dataset 'D containing data samples (i.e., x E X) and corresponding labels (i.e., y E Y), we can train DNN to predict the labels of input samples, i.e., 9 F(x) with ÿ E y being the prediction results. In the real word, the samples might be divided into subgroups according to some sensitive attributes a E A such as gender and race. Without loss of generality, we consider the binary classification and binary attribute setup, i.e., y E (0, and a E {0. 1). For example, = 0 and a = 1 could represent male and female, respectively. A fair DNN (i.e., F(-)) is desired to obtain similar accuracy in the two subgroups.
3.2 FAIRNESS REGULARIZATION
Existing fairness works (Feldman et al., 2015; Hardt et al., 2016; Madras et al., 2018; Chuang & Mroueh, 2021) focus on designing fairness regularization terms and adding them to the loss function, which encourages the targeted DNN to predict similar results across subgroups. Specifically, Feldman et al. (2015) develop the demographic parity (DP) regularization term to encourage the predicted label to be independent of the sensitive attribute (i.e., a), that is, P(ola = 0) P(ala 1) which means that the probability distribution of ÿ condition on a = 0 should be the same as the condition on 1. Hardt et al. (2016) further propose the equalized odds (EO) regularization to consider the ground truth label y and make the prediction and sensitive attribute conditionally independent w.r.t. y. ie., P(ğla- 0, y) P(ğla- 1, v). Although straightforward, it is difficult optimize the above
3

---

Under review as a conference paper at ICLR 2023
regularization terms and Madras et al. (2018) propose relaxed counterparts:
(F) [B (F(x)) Ex- (F(x))I:
where Po = P(x|a = 0) and P1 P(xla = 1) are the distributions of x condition on a = 0 and = respectively, and the function E(-) is to calculate the expectation under the distributions.
(FO = Σ
Ex (F(x)) - Px (F()))
vE(0,1)
where Po = P(x|a 0,y = 1) denotes the distribution of x condition on the = 0 and 1, and we have similar notations for Po. Pl. P if we set the DNN for binary classification task and have the label y E 0, 1. We can add Eq. (1) and Eq. (2) to the classification loss (e.g., cross-entropy loss) to regularize the fairness of the targeted DNN, respectively, and obtain the whole loss function
= P(xx))) Let (F(x),)) + ALtii(F))
where P denotes the joint distribution of x and Lets is the classification loss, and the term Lairr could be ADP(F) or AFO(F) defined Eq. (1) Eq. (2). We can minimize the above loss function and get fairness-regularizee DNNs. Although effective, the above method presents some generalization limitations. To alleviate this issue, Chuang Mroueh (2021) embed the data augmentation strategy into the fairness regularization method and propose FairMixup with novel DP- and EO-dependent regularization terms. Please refer to Chuang Mroueh (2021) for more details. Our method also applicable to other fairness metrics that quantify the expected difference between groups. More analysis is put in supplementary materials.
Overall, get several fairness regularization methods via different regularization terms. Specifically, we denote the methods without augmentation as FairReg (ADP, noAug) and PairReg(AFOO noAug) based regularization functions (i.e., Eq. (1) and Eq. (2)). We denote the methods equipped with data augmentation as FairReg Aug) and FairReg( AEO, Aug), respectively.
3.3 LIMITATIONS
Although the above methods are able enhance the fairness of DNNs, they still present some lim- itations. We conduct an experiment on the Adult dataset (Dua & Graff, 2017a) with neural network with 3-layer MLPs. Specifically, we train the net- work with two fairness regularization methods (i.e., FairReg (AP,, noAug) and FairReg (DP, Aug) ) and five different À {0.2, 0.3, 0.4, 0.5, 0.6}. that is, for each method, we get five trained networks. Then,
5.78
477
3.77
47
3.74
878
0.75
375
AP
2.24
874
DRAlign(APPI Our
Par rolage
470
Par Ragiton Aug!
SaiRes
50
nataileg Sampe
calculate the accuracy scores and fair- ness scores of all networks on the testing dataset. We employ average precision for the accuracy score and -DP for the faireess score since a smaller DP means better fairness. For each method, we can draw a plot w.r.t. different X. Besides, we also train a network without the faimesss regularization term and denote as w.o. FairReg. Based on FairReg, we can conduct oversampling on the training samples to balance the samples across different subgroups (Wang et al., 2020) and denote it as w.o. ShairReg-Oversannin w.o. W.o.FairReg presents obvious fairness issue and the oversampling solution has limited capability to fix it. @ When we use the regularization methods and gradually increase the weight in Eq. (3) from 0.2 to 0.6, FairReg( ADP, noAug) is able to generate fairer networks with higher faimess scores (i.e., higher -DP) than the one from W.o. FairRe. However, the corresponding accuracy decreases by large margin, that is, existing methods could hardly generate enough fair networks under similar accuracy. 2 The data augmentation-bant based method (i.e., FairReg( ADP, Aug)) can alleviate such an issue to some extent and achieves higher fairness than FairReg( ADP, noAug) under similar accuracy.
watorflp
Figure 2: Accuracy and faimess comparison of five different methods on the Adult dataset. The byperparameter À increases from 0.2 to 0.6 along the -DP axis as becomes larger.
As shown in Figure 2, we see that: The standard trained network via
Such methods only provide faimess metric results but neglect the decision decision-making process. Although intuitively, the consistent decision process of different groups could improve the fairness performance,
We have similar observations the AFO-hased methods and remove them for clear explanation.
4

---

Under review as a conference paper at ICLR 2023
to explore the concrete relationship between the decision-makine process and faimess, provide an analysis method by extending the decision rationale- aware explainable methods in Sec. 4. Specifically, instead of using the final fairness metrics, define the neuron parity score for each parameter of network that measures whether the parameter is fair, that is, whether has consistent responses to different subgroups.
DECISION RATIONALE- FAIRNESS ANALYSIS
In recent years, decision rationale- aware explainable methods are developed and help understand how a trained network makes a decision (Khakzar et al., 2021; Wang et al., 2018). In these works, the decision rationale is represented by measuring the importance of intermediate neurons. Inspired by this idea, to understand a fair decision, we study the decision process of networks by analyzing their neuron behaviors under different subgroups and define the decision rationales for different subgroups. Then, we define the parity score for network that actually measures whether the decision rationales on different subgroups are consistent. Besides, we can use the parity score to compare the networks trained with different regularization terms.
4.1 NEURON PARITY SCORE
Inspired by recent work on understanding the importance of the neuron for the classification loss (Molchanov et al., 2019), define the neuron parity score based on the independent as- sumption across neurons (i.e., parameters). When we have a trained network F(-) with its parameters W {wo..... wk J. we can calculate classification losses on samples from two distributions Po = P((x v)le = 0) and P1 = P((x, y)(a = 1), which correspond to the training subsets of and
1) and get the losses J [F, Po) and [F, Pi), respectively. Meanwhile, we can modify F(.) by removing a specific parameter wi and denote the new counterpart as Fu,=o. and we can also obtain losses via [F...= Po) and J (F.,.=0, Pi). Then, for each subgroup (i.e., Po or Pi), we calculate the loss change before and after removing the parameter wk by
two subgroups (i.e.,
C wk. P,) [T J(F P.P.. Wi E 10, k E K0,
(4)
where the function J (F, P.) is to calculate the classification loss (i.e., Cots in Eq. (3)) of examples in P. with Vi E {0, 1} based on the network F. With subgroup Pt and K K-neuron network F, we can get of - c of
,,,ck a= that is regarded as a representation the decision rationale on the subgroup P. (Khakzar et al., 2021).
Then, we define the parity score of the parameter we as the difference between of an= = C(F, Wk, Po) and ck c-= C(F, Wk: Pi), i.e.
d |C(F, Wk: Po) C(F, WA, P.)
Intuitively, if the network Fis fair to kind sensitive attribute, each parameter should have consistent responses to different subgroups, and the changes before and after removing the parameter should be the same. As a result, smaller d, means that the parameter W) is less sensitive to the attribute changes. For the entire network with K neurons, we get K parity scores and cf [co [co= c-=
ck
and can represent the network with dp [do, di...., dx] and aggregate all scores for network levell parity score, i.e., de Ex=o dg = |cp=0 of on the two subsets are consistent (i.e., properly aligned).
- /1, which measures whether the decision rationales
4.2 RELATIONSHIP BETWEEN PARITY SCORE AND FAIRNESS
With the neuron parity score, we conduct an empirical study based on the Adult dataset and neural network with 3-layer MLPs. Specifically, we train six networks with the regularization terms defined in Sec. 3.3, e.g., the ADP-based regularization terms with six different weights (i.e.. FairReg ADP, noAug) with À E {0.0, 0.2, 0.3, 0.4, 0.5, 0.6}). Note that, FairReg noAug) with = 0.0 represents the standard trained network without fair regularization terms (i.e., w.r.Fair Then, for each method, we can train neural network and calculate the parity score, i.e., d; E de |c ca to measure the decision rationale shifting across subgroups and the fairness score defined by -DP. As reported in Table 1, we see that: the parity score of the network gradually decreases as the -DP becomes higher, which demonstrates that the fairness
5

---

Under review as a conference paper at ICLR 2023
Table Parity scores, faireess scores, the first-order Taylor approximation parity scores networks trained FairReg( (AP,, noAug) with different À in Eq. (3), For each network, we train 10 runs with different seeds and the average results are reported.
FutRopl (DDP madug)
A-00 0.0
A- 0.2
4 03
A-044 0.4
A- 0.5
A 0.6
Party scom (np)
0.604 + 0.555
0.301 1 0-338
0.101 1: 01051
0.070 + 0015
| 0.00 4 0.003
D1000 in 0,000
Frimess DP)
-0.100 a: 0,012
-0.001
0.00
| -0.068 a 0.006
-0.022 a 0.000
-0.020 #: 0.00
-0.000 dE 0.003
Approx. Yein coster" 6"
1)
0.070 &: GAME -1.90 0,000
-1.500000
-1,00 0.18 -1,000 0.156
-1,800 0.25
of a network is highly related to the decision rationale shifting across subgroups. @ adding the fairness regularization term on the last-layer outputs (ie., > 0) can decrease the decision rationale shifting to some extent. However, such an indirect way could hardly achieve the optimized results and a more effective way is to actively align the decision rationale explicitly. Note that we can observe similar results on other regularization methods and focus FairReg( (DP,, noAug) due to the limited space. We conclude that the existing fairness regularization-ben the consistency between decision rationales of the network on different subgroups to some extent and get smaller neuron parity scores than the standard trained network. This inspires our method in Sec.5 that conducts alignment of the decision rationales of different subgroups explicitly.
methods can encourage
5 DECISION RATIONALE ALIGNMENT
5.1 FORMULATION AND CHALLENGES
According to Eq. (5), we can achieve fairer network by aligning the decision rationales of subgroups and a straightforward way is to set the parity score d:- = = dg as an extra loss function and minimize it directly, that is, add new loss to Eq. (3) and have,
K
L
(Cas (F(x), u)) + ACrair (F) + 3
Σ de.
k=0
where de is the parity score of the kth neuron and calculated by Eq. (5). Such loss should calculate parity scores for all neurons and all samples in dataset, leading high cost and is not practical.
5.2 GRADIENT-GUIDED PARITY ALIGNMENT
To address the challenges, we relax Eq. (4) to the sample-based counterpart a=i C(F, WK, P3) = (xxx)) (Let (F(x), y) E (x.)) (fux(FF)
e {0, 1), K].
We use the first-order Taylor expansion to approximate o- similar to Molchanov et al. (2019) and get
& شسمع Č(F, wk: P.) = (9) wx) vi E {0, 13, k e K].
(8)
where g denotes the gradient of the ktth neuron (i.e., wx) w.r.t. the loss function on the examples sampled from the distribution of the ith subgroup (i.e., P.). Intuitively, the above definition means that we should pay more attention to the neurons with higher gradients and make them have similar responses to different subgroups. However, neurons (i.e., parameters) of different layers may have
different score ranges. To avoid this influence, we further normalize ég- by lēp=q Vi E (0, 1), k E KG, where Kc, contains the indexes of the neurons in the ith layer, and parity scores of neurons in the 1th layer (i.e., {2g=1[A lk E K.)) form a vector of= vec(
( (2g=*(k [k E K. Then, we can get new c lk E Kc)) by normalizing each element. Then, we can ب
vector for the Ith layer द. vec( {
update Eq. (6) by minimizing the distance between 27-0 and c vl E L), i.e.,
L
L= E(x.n)- v)) + M (F) - B
cos cos (ar= (.-=1)
(9)
1=0
where L denotes the number of layers in the network, and the function cos(.) is the cosine sim- ilarity function. The last two terms are used to align the final predictions and the responses of the intermediate neurons across subgroups, respectively. To validate the approximation (i.e., cos( d)=l)) can reflect the decision rationale alignment degree like the parity score Ek d4. We conduct an empirical study on FairReg( (ADP, noAug) as done in Sec. 4.2 and caleu- late the value of - El= cos(c) 1) for all trained networks. From Table 1, we see that the approximation has consistent variation trend with the parity score under different X.

---

Under review as a conference paper at ICLR 2023
Algorithm 1: couudient-guided Parity Alignment
Data: Network F with parameters = [Wo, ..., wx) epoch index set E, training data D. batch size B. network layers L, neurons in the /th layer Ki.hyper-parameter parameters À and B, learning rate 1/.
// Training process for DP
for E & do
// Sampling samples from different subgroups in D (Xo, Yo) <- Sample(D, 0, B); (X,, Y1) f- Sample(D, 1, B,
// Calculating loss and updating the model
(a))) Yo), Leu (F(X.). Y1); Las ADP(F, X.); for I E L do for E
(F Yoll,
9
Buk
(gk -=
wx
Ck
t (gh 1- wx)+
100-0 2
c- 1200 11 ...., Kg
Las (F())), Yo) (F(X1) Y1) + Aa 88 Et cos
10
w f w 17 Ww, Yu W.
5.3 IMPLEMENTATION DETAILS
We detail the whole training process in Algorithm 1. In particular, given training dataset DD we first sample two groups of samples (ie, (Xo, Yo) and X1, Y2) from the two subgroups the dataset, respectively (See line 2). Then, we calculate the cross-entropy loss for both samples (See line 3) and calculate the fairness regularization loss (i.e., Lfair = ADP Xo, X1). After that, we can calculate the gradient of each parameter (i.e., neuron wy& w.r.t. the classification loss (See line 6) and calculate the decision rationale for each neuron and layer (See line 7 and 8). Finally, we calculate the cosine similarity between oy=0 and d:=1 and use the whole loss update the parameters. We defer the algorithm depiction for the EO metric to the Supplementary Materials.
6 EXPERIMENTS
6.1 EXPERIMENTAL SETUP
Datasets. our experiments, we use two tabular benchmarks (Adult and Credit) and one image dataset (CelebA) that are all for binary classification tasks: Adult (Dua & Graff, 2017a). The original aim of the dataset Adult is to determine whether a person makes salaries over 50K a year. We consider gender the sensitive attribute, and the vanilla training will lead the model predict females to earn less salaries. @ CelebA (Liu et al., 2015). The CelebFaces Attributes dataset is to predict the attributes of face. We split into two subgroups according to the attribute gender. Here we consider two attributes classification tasks. For the task to predict whether the hair in an image is wavy or not, the standard training will show discrimination towards the male group; when predicting whether the face is attractive, the standard training will result in model prone to predict males as less attractive. O Credit (Dua & Graff, 2017b). This dataset is to give an assessment of credit based on personal and financial records. In our paper, we take the attribute gender as the sensitive attribute.
Models. For tabular benchmarks, we use the MLP (multilayer perception) (Bishop, 1996) as the classification model, which is commonly adopted in classifying tabular data. For the CelebA dataset, we use AlexNet (Krizhevsky et al., 2012) and ResNet-18 (He et al., 2016), both of which are popular in classifying image data (Alom et al., 2018). We mainly show the experimental results of predicting wavy hair using AlexNet. More results and training details are in supplementary materials.
Metrics. For fairness evaluation, we take two group fairness metrics DP and EO as we introduced in the section 3.2 and define -DP and -EO as fairness scores since smaller DP and EO mean better fairness. We use the average precision (AP) for classification accuracy evaluation. A desired fairness method should achieve smaller DP or EO but higher AP (i.e., the top left corner in Fig. 3).
Baselines. Following the common setups in Chuang & Mroueh (2021), compare method with several baselines: 1 Standard training based on empirical risk minimization (ERM) princi- ple (i.e., FaiRee. DNNs trained only with the cross entropy loss. @ Oversample (i.e.,
7

---

Under review as a conference paper at ICLR 2023
C2V
2.56
Farii aEArela!
CRARGHAENT OWn
Sagest And
110
sofefay Doston
Freccitg
DP
010
AP.
AP
BRAGgwORR Our
Exhedlii soke)
6-24 may Oee
-10
-50
Adult
CalebA
(6) Credit
Figure 3: Comparing different methods on AP (-DPLFO), According the common setups, evaluate P-haee and AEO-hased methods via DP and EO, respectively. We train networks with the compared methods for 10 times and the averaging results are reported.
(Wang et al., 2020). This method samples from the subgroup with rare examples more often, making balanced sampling in each epoch. 2 PairReg(ADP or AEO, noAug) (Madras et al., 2018). This method is to directly regularize the fairness metrics, i.e., ADP or AEO. ! FairReg(ADD (DDP or AEO, Aug) (i.e., FairMixup) (Chuang & Mroueh, 2021). This method regularizes the models on paths of interpolated samples between subgroups to achieve fairesss. @ Adversarial (Zhang al., 2018). This method minimizes the adversary's ability to predict sensitive attributes. G Post-processing (Bellamy et al., 2018). This method modifies the predictions of an accurate model with fairness objective.
W.i.Rei
Oversample) No.Faairee--Overamplle
6.2 FAIRNESS IMPROVEMENT PERFORMANCE
As shown in Fig. 3, we have following observations: With the Adult and CelebA datasets, our method (i.e., DRAlign) achieves higher fairness (i.e., higher -DP or -EO scores) than all baseline methods when they have similar accuracy. In particular, on the Adult dataset, DRAlign has relative 41.6% DP improvement over the second best method (i.e., FairReg( ADP, Aug)) when both get around 0.770 AP. Overall, our method can enhance the faimess significantly with much less accuracy sacrifice. @ Data augmentation method does not always improve DNN's fairness. For example, on the dataset Adult, FairReg(* *, noAug) presents slightly higher faimess score (i.e., higher -DP or -FO) than FairReg(*, +, Aug). A potential reason is that the augmented data becomes less realistic due to the rich information in the image modality, which leads to less effective learning. @ Although oversampling could improve faimess to some extent, it is less effective than the fairess regularization- based methods (i.e., FairReg(*, *, noAug)). For example, on the CelebA dataset, w.o. w.o.FairReg Feo.Faiir---vesssie Oversample only obtains -0.069 -EO score with the 0.812 AP score, while FairReg(+,, noAug) achieves the 0.054 -EO score with 0.817 AP score. The networks trained by FairReg(*, (+, noAug) are not only fairer but also of higher accuracy. On the tabular dataset, wai.Faarree-Ovee WairFaiRRe by small margin. 4 On the Credit dataset, CairReg(ADD Aug) achieves better results than DRAlign under the DP metric although our method still outperforms the regularization- based one. A potential reason is that the data size of the Credit is small (i.e., 500 training samples) and the data augmentation can present obvious advantages by enriching the training data significantly. The data augmentation and our decision rationale alignment are two independent ways to enhance fairness. Intuitively, we can combine the two solutions straightforwardly. We do further experiments and find that our DR Align could still improve FairReg( ADP, Aug). More details are put in supplementary materials.
outperforms the
6.3 DISCUSSION AND ANALYSIS
Connection with nor-parammeteiiationn To better understand the cause of the decision rationale misalignment, we further investigate the connection between decision rationale misalignment and model over paraaaterreziinnoonn We conduct an empirical study on the Adult dataset using 3-layer MLP networks based on FairReg( ADP, noAug). Specifically, we explore 4 MLP architectures, where the hidden sizes are set as 10, 20, 50, and 200, respectively. The corresponding parameter sizes of the networks are 1331, 2861, 8651, and 64601. For each architecture, we draw plot w.r.t. different À for
8

---

Under review as a conference paper at ICLR 2023
2.0
Figure 4: (a) corre lation between and decision rationale sim- ilarity score. (b) layer- wise analysis for corre- lation between À and decision rationale simi- larity score.
0.6
2.5
TantagioD" Augi 0c221
0.4
RerReguAEP Mockag (:20)
1.0
DarkeçiADo reaugless)
Kgl RegiREE (1951
PartestAD ratael 1022 eporl
PaSTeSt redugl 12220 bee
Partectops
0.0
0.0
A-0
A-
1-02 A-0.3 A-04 (b)
A-5.5
A-
A-0
A--02
A-0.3 0- 4- 0-0.5
[b]
Valar
FairReg (APP, noAug) to show the decision rationale similarity score (i.e.. Σ cos cos(cr-1, c in Sec. 5.2). We denote the four trained models as FairReg( ADP,noouu (c10), FairReg(noAug) (c20). FairReg(ADP,ooAuu
(c50), and FairReg( (ADRReg(DDP.o.... nDo..o (c200), respectively, according to their hidden sizes. More results under AO metric are pul in Supplementary Materials. With Fig. 4 (a), we have the following observations: The decision rationale similarity consistently ascends when À increases. When À becomes 0.5, decision rationale similarities of FairReg( ADP,noAug (c10), FairReg( Auug) (c20)and FairReg( ADP Aug) (c50) reach the same maximum score (i.e., 3.0 for any 3-layer MLP network). We conclude that larger À (stricter fairness constraint) results in a higher decision rationale similarity. @ The misalignment of decision rationale is more likely to occur in the wver-parameterized networks. For the largest network FairReg ParregAAA..,..Auuu. .noAAu) (c200), even when the À is set as 0.6 for strict fairness constraint, the decision rationale similarity score only reaches 1.8 which is much smaller than the values on other variants and infers that the decision rationale is still far from being aligned.
Furthermore, we also report the results of augmentation--asee method, i.e., (A). We find that data augmentation can generally mitigate the misalignment but still fails to completely align the decision rationale (See the plot of PairReg(ADP (nogu) (c200)). Our method DRAlign is able to achieve the maximum similarity on all X settings even on the architecture with hidden size 200. This enlightens us that common methods such as data augmentation that aims to address weer-parameterization can not completely solve the misalignment, while our gradient-guide parity alignment method can directly improve the alignment.
Layer-wise decision rationale alignment analysis. We further conduct a layer-wise analysis to understand which layer owns better decision rationale alignment. We calculate the decision rationale similarity for the 1st and 2nd layer (i.e., cos(c)=0) ) and cos(? c From Fig. 4 (b), we see that: for both layers, the layer-wise similarity score ascends when À increases. This is consistent with the observation that stricter fairness constraint results in higher decision rationale similarity. As we compare the 1st and 2nd layers, we can observe that the similarity score of the first layer is generally higher. Moreover, we can see that for smaller models (i.e., models with hidden size 50), the similarity gap between the first layer and the second layer is relatively trivial. However, for models with hidden size 200, the similarity score of the second layer is rather low (i.e., the score is 0.113 even when the X is 0.6). We conclude that the misalignment of the deeper layer is severer.
CONCLUSIONS AND FUTURE WORK
In this work, we have studied the fairness issue of deep models from the perspective of decision rationale and defined the neuron parity score characterize the decision rationale shifting across subgroups. We observed that such a decision rationale-aware characterization has high correlation to the fairness of deep models, which means that a fairer network should have aligned decision rationales across subgroups. To this end, we formulated fairness as the decision rationale alignment (DRAlign) and proposed the gradient-guided parity alignment to implement the new task. The results on three public datasets demonstrate the effectiveness and advantages of our methods and show that DRAlign is able to achieve much higher fairness with less accuracy sacrifice than all existing methods. Although promising, our method also presents some drawbacks: (1) it requires the computation of second-order derivatives; and (2) the gradient-guidee parity alignment method limited to the layer-wise DNN architecture. In the future, are interested in solving these limitations.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Md Zahangir Alom, Tarek M Taha, Christopher Yakopcic, Stefan Westberg, Paheding Sidike, Mst Shamima Nasrin, Brian Van Esesn, Abdul Awwal, and Vijayan K Asari. The his- tory began from alexnet: A comprehensive survey on deep learning approaches. arXiv preprint arXiv: 2018.
Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang. Al Faimess 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias, October 2018. URL https: //arxiv. owg/ abs 1810. 01943.
Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Inc., USA, 1996. ISBN 0198538499.
Ching-Yao Chuang and Youssef Mroueh. Fair mixup: Fairness via interpolation. In International Conference on Learning Representations, 2021. URL https: http:////peeeeeeeee.eee.. net.:/iwwwweeeeeeeeeeeeeeeeeee htwp://ee/eeoeeeeeeeeeeeeeeeeee net/forum? forum? id-DN15s5BXeBn
Andrew Cotter, Maya Gupta, Heinrich Jiang, Nathan Srebro, Karthik Sridharan, Serena Wang, Blake Woodworth, and Seungil You. Training well-generalizing classifiers for fairness metries and other data-dependent constraints. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 1397-1405, PMLR, 09-15 Jun 2019. URL https: pptce//d//occedggs/.//o. mlr. molr.pres//ve v77 cotterl 9b. html.
Li Deng, Geoffrey Hinton, and Brian Kingsbury. New types of deep neural network learning for speech recognition and related applications: an overview. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8599-8600 2013. doi: 10.1 10.0SSS//AS 6639344.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
Mengnan Du, Subhabrata Mukherjee, Guanchu Wang, Ruixiang Tang, Ahmed Hassan Awadallah, and Xia Hu. Fairness via representation neutralization. In NeurlPS, 2021.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017a. URL /ttphtt:///rcch.ve ics. eooi/ed
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017b. URL CRl.http///achhvee ics. bici.dd//ml.
Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD Knowledge Discovery and Data Mining, KDD '15, pp. 259-268, International Conference Association for Computing Machinery. ISBN 978 450336642.. doi: URL https:://// /oi. http://dooooro/..1111/1111 11451111222 / 3258 2783311.
New York, NY, USA, 2015. 10.1145/2783258.27883
R. C. Fong and A. Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3449-3457, Los Alamitos, CA, USA, oct 2017. IEEE Computer Society, doi: 10. 10.1109//CCV22011..33.. URL https: //doi.
11t./1111.. org/ wttp///..109//.v.....o http:/ccc..211....11
Garg, J. Villasenor, and V. Foggo. Fairness metrics: A comparative analysis. In 2020 IEEE International Conference Big Data (Big Data), pp. 3662-3666. Los Alamitos, CA. USA, dec 2020, IEEE Computer Society. doi: 10. 10110099BiiDDeaa00200000..99 //doi. eeecomputersoc ety org/ 110.11109/110 http:////aaaaaaa.a0a220022
URL https:
wttps/cccittttttttttooooeoeoe ott.:.rrrr..........aaaa.a...... 2020 9378025.
10

---

Under review as a conference paper at ICLR 2023
Philips George John, Deepak Vijaykeerthy, and Diptikalyan Saha. Verifying individual fairness in machine learning models. In Jonas Peters and David Sontag (eds.), Proceedings of the 36th Conference Uncertainty in Artificial Intelligence (UAI), volume 124 of Proceedings Machine Learning Research, pp- 749-758. PMLR, 03-06 Aug 2020. URL https: wtt.hhttss://p/oeaee prooeedings. george- mlr. w.r..peas/vv222 v/lss/v1224 wttp:///////ooeeoaeeooree john20a a html.
Umang Gupta, Aaron Ferber, Bistra Dilkina, and Greg Ver Steeg. Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation. 2021.
Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity supervised learning. In NIPS, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. doi: D..110/CVPR.2016.9000
Ashkan Khakzar, Soroosh Baselizadeh, Saurabh Khanduja, Christian Rupprecht, Seong Tae Kim, and Nassir Navab. Neural response interpretation through the lens of critical pathways. In 202/ IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13523-13533, 2021. doi: 10.1109/CVPR4443312222.1
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, ICML' 17, pp. 1885-1894. JMLR.org, 2017.
Alex Krizhevsky. Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. berger (eds.). Advances in Neural Information Processing Systems, volume 25. Curran Asso- ciates, Inc., 2012. URL https: httpss://poocceeiiiiiii ngs neurips pdf.
F. Pereira, C.J. Burges, L. Bottou, and K.Q. Wein-
c399862d3b9ddse
Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairesss. In Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL https: / http://ooceeeenggseece .errisennuspppp wwtp:/nwww...ppppeeeeeeeeee..eee pttppppee//./////11ee. a166:007e2c1122777112242444244 (ewe::2222436115---222222222232- p.f.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
G. Louppe, M. Kagan, and K. Cranmer. Learning to Pivot with Adversarial Networks. ArXiv e-prints, November 2016.
David Madras, Elliot Creager, Toniann Pitassi, and Richard S. Zemel. Learning adversarially fair and transferable representations. ICML, 2018.
Pavlo Molchanov, Arun Mallya, Stephen Tyree, Turi Frosio, and Jan Kautz. Importance estimation for neural network pruning. In 2019 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 1225-11264, 2019. doi: 10.1109/CVPR.2019.01155
Daniel Moyer, Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, and Aram Galstyan. Invariant representations without adversarial training. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 9102-9111, Red Hook, NY, USA, 2018. Curran Associates Inc.
BBC News. Ibm abandons 'biased' facial recognition tech, 2020. URL https: /ttp::///wwwwwee http:/wwwbbbbb... uk/news http///ss/teccnollo----o-ao wtt.llgyyyy99989..
BBC News. Ai at work: Staff 'hired and fired by algorithm', 2021. URL httph::t//wwtt news /ttccholloo--551552222222
com/news/
Yuxian Qiu, Jingwen Leng, Cong Guo, Quan Chen, Chao Li, Minyi Guo, and Yuhao Zhu. Adversarial defense through network profiling based path extraction. pp. 4772-47777 4781, 06 2019. doi: 10.1109/ NVPP.2019.00499
11

---

Under review as a conference paper at ICLR 2023
Mhd Hasan Sarhan, Nassir Navab, Abouzar Eslami, and Shadi Albargouni. Fairness by learning orthogonal disentangled representations. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision ECCV 2020, pp. 746-761, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58522--6
Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias deep image representations. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
Yulong Wang, Hang Su, Bo Zhang, and Xiaolin Hu. Interpret neural networks by identifying critical data routing paths. In 2018 IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 8906-89144 2018. doi: 10. 10.1
Z. Wang, K. Qinami, I. Karakozis, K. Genova, P. Nair, K. Hata, and O. Russakovsky. Towards faimess in visual recognition: Effective strategies for bias mitigation. In 2020 JEEE/CVF Conference on Computer Vision and Patlenn Recognition (CVPR), pp. 8916-8925, Los Alamitos, CA, USA, jun 2020. IEEE Computer Society. doi: 10.1 10.110990222420002200000. 00894. URL https:/ //ooi. 2020. 00894.
leeecomput wwwpppttrssciitt..ooooooo org. /10 1109/ CVRRC wwt..1111991/2110000000
Blake Woodworth, Suriya Gunasekar, Mesrob I. Ohannessian, and Nathan Srebro. Learning non- discriminatory predictors. In Satyen Kale and Ohad Shamir (eds.). Proceedings of the 2017 Conference on Learning Theory, volume 65 of Proceedings of Machine Learning Research, pp. 1920-1953, PMLR, 07-10 Jul 2017. URL https: httpss:/pprcceee::e http///pooeeee//ee//ee/ee///..se mll. www.mrea.....pt.mm..m...m.. www.pprssseeee... /v65/ woodworth 7 ht.ml.
Xiaofei Xie, Tianlin Li, Jian Wang, Lei Ma, Qing Guo, Felix Juefei-Xu, and Yang Liu. Npc: Neuron path coverage via characterizing decision logic of deep neural networks. ACM Trans. Softw. Eng. Methodol.. 31(3), apr 2022. ISSN 1049-331X. doi: 10. 10.1145/349048, URL https: //doi. oroii.org/11 10. 1145/14944443
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair repre- sentations. In Sanjoy Dasgupta and David McAllester (eds.), Proceedings of the 30th In- ternational Conference on Machine Learning, volume 28 of Proceedings of Machine Learn- ing Research, pp. 325-333, Atlanta, Georgia, USA, 17-19 Jun 2013. PMLR. URL https: //proceeding www.inn.s.m.............mm.m.... www.mml.pp.... www.mmrrrsrrrsm..mmmmmmmmmmmm.ee / v22//22//////// zeeee/eeeelee 13. html.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adver- sarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES '18, pp. 335-340, New York. NY, USA, 2018. Association for Computing Machinery. ISBN 9781 450360128.. doi: 10. 10.11455327872213327888 3278721 3278779.
Lingfeng Zhang, Yueling Zhang, and Min Zhang. Efficient white-box faimess testing through gradient search. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2021, pp. 103-114, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450384599. doi: 10. 00.1155336033193444422 URL https: 1141/1145350 / 60319.031.. ....119..444444 3464820.
Peixin Zhang, Jingyi Wang, Jun Sun, Guoliang Dong, Xinyu Wang, Xingen Wang, Jin Song Dong, and Ting Dai. White-box fairness testing through adversarial sampling. In Proceedings of the
ACM/IEEE 42nd International Conference on Software Engineering, ICSE '20, pp. 940-960, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450371 10.11455337781 1.138033.33333311 URL https: //w.. www.ooogo/.......ooo 1111110..111/331 30044//33777111 781117771111111 018777111113....1111333113333311 3380331.
doi:
Quanshi Zhang and Song Zhu. Visual interpretability for deep learning: a survey. Frontiers of Information Technology and Electronic Engineering, 19, 02 2018. doi: 1..166//FFTEE.1111
Haibin Zheng, Zhiqing Chen, Tianyu Du, Xuhong Zhang, Yao Cheng, Shouling Ti, Jingyi Wang, Yue Yu, and Jinyin Chen. Neuronfair: Interpretable white-box fairness testing through biased neuron identification. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE), pp. 1519-1533 1531, 2022. doi: 10. 00.114//550000335100111
12


Reviews:
[{"review_id": "sjc31JLCAIy", "reviewer": "Reviewer_X7LC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a new mechanism, called *gradient-guided parity alignment* to ensure fairness in neural models. This is done by studying the fairness problem from the perspective of decision rationale, which has been one of the mainstream areas of focus in interpretability literature. The authors implement their proposed mechanism along with the standard fairness regularizer which is the mainstream strategy in the in-processing group fairness literature. More concretely, first the loss change before and after removing a specific parameter $w_k$ of the neural model if computed in eq 4. This is then used to define the *neuron parity score* in eq 5. Intuitively, this characterizes that the change in the loss based on a given $w_k$ should be the same across the subgroups. The parity score is computed for all the parameters in the model. Furthermore, since directly computing this for all samples and neurons is expensive, the authors utilize a sampling strategy. Eq 7 is relaxed to obtain the gradient weighted estimation of the change in loss based on the first order Taylor expansion. Lastly, these parity scores are normalized layerwise and used in the final loss objective, with the underlying parity computed based on cosine similarity. Extensive experiments have been performed against popular fairness methods along with interesting analyses.\n", "summary_of_the_review": "Based on the questions, comments and concerns raised in the weakness section, I lean towards weak rejection of the work. I am willing to reconsider my score if the authors can answer the questions raised above.", "strength_and_weaknesses": "1. The proposed method can be easily integrated in the training procedure for an arbitrary network, making it highly generalizable.\n2. Experiments have been performed across multiple values of $\\lambda$ and clearly details the pareto frontier. \n3. The subsection on “Layer-wise Decision Rationale Alignment” provides interesting characterization against the depth of the model. \n4. An important citation to  (Zheng et al 2021; NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification) is missing. This work has some similarities to that of Zheng et al, in that both focus on proposing a function over the network neurons.\n5. The computation of the second order derivatives is a significant overhead. It would be interesting to see some results comparing the runtime of the proposed method against the baselines. \n6. While computing eq 7, do the authors turn off the dropout layer in the model? Not turning it off could have detrimental results on the performance.\n7. There is no study for the variation of results against $\\beta$, which is the hyperparameter of the proposed misalignment objective. \n8. The complete hyperparameter details such as the optimizer used, batch size, weight decay etc are missing. \n9. Using cosine similarity in eq 9 is a reasonable strategy. Is it chosen for simplicity? Do the authors have some intuition on how the cosine will compare against other metrics, a simple one being the metric induced by the L2-norm?\n10. The final objective in eq 9 utilizes a fairness regularizer as well. It will be more useful to observe the results solely based on the cross-entropy loss along with the proposed alignment regularizer to ensure that the claims hold.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "o3M5XwRS98-", "reviewer": "Reviewer_3JWw", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The motivation of this paper is to achieve fairness by modifying the learning of the neuron of deep networks (addressing decision rationale misalignment). The proposed algorithm is feasible for large weights in deep networks, and experiments results show that the algorithm makes the result more fair, combined with a conventional regularizer for fairness (measured by DP, EO, etc.) Analysis shows the data augmentation can help achieve fairness in some datasets.\n", "summary_of_the_review": "This paper considers an interesting approach with alignment, which can make the fairness algorithm plenty. Experiment results seem promising. The deficit is not thoroughly studied with various fairness metrics.", "strength_and_weaknesses": "Pros: \nThis paper proposed a new way to achieve fairness by adjusting the inner learning neurons. This approach is interesting and sheds a way to achieve fairness thoroughly. Experiments were conducted, combined with DP and EO. Intensive   \n\nCons:\nMore analytics analyses such as the relationship between alignment and various fairness metrics, are not thoroughly studied. In the experiments of Credit, FairReg (Aug) shows better performance in DP. It raises the issue of how augmentation can help the DRAlign algorithm. However, it was not addressed.   \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "KqxVH59YyEz", "reviewer": "Reviewer_HMGh", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new fairness regularization technique based on an interpretation method called neuron parity score. In a nutshell, the proposed method penalizes the gap among the _subgroup loss increments_ from zero-ing out a specific parameter, in addition to the model-level fairness loss. Empirically, the proposed method achieves a better fairness-performance frontier than FairMixup and LAFTR.", "summary_of_the_review": "Although the method seems to provide decent performance gain over the baselines, I think there is a significant room for the manuscript to improve in terms of (1) the diversity of the baseline methods, and (2) clarity of the motivation behind the algorithm.", "strength_and_weaknesses": "__Strength__\n\n- The proposed method is versatile, in the sense that it can be provided to several different notions of fairness.\n- The method shows an impressive performance over the baselines in most empirical setups considered in the paper.\n- Empirical validation has been performed in terms of the full tradeoff curve, instead of showing only on data point.\n\n__Weakness__\n\n- The paper argues that it will give an analysis of the existing fairness-via-regularization technique via decision rationale analysis, but I do not see it. I can only see the experimental results showing that the existing method performs worse. Am I missing anything?\n\n- The \"limitations\" pointed out in Section 3.3 is not very clear to me. The paper says that existing methods are limited, as they could \"hardly generate enough fair networks under similar accuracy.\" But the \"enough\"-ness is in the eye of the beholder; why is 0.058 DP enough, and 0.078 not enough? The logic sounds like, 'existing methods are limited, as it does not perform as well as the method that we are going to propose, which motivates us to develop a new method.' This does not really point out any fundamental, structural limitations of the existing methods.\n\n- The intuition behind using the neuron parity score for fairness is unclear. Authors seem to motivate the technique from the fact that \"how the fairness regularization term affect the final network parameter is not well understood,\" but I am not sure how the existing method provides a strictly better interpretation, or why such better parameter-wise interpretability should help improving the ultimate performance.\n\n- The range of baselines considered is quite limited. There is a wealth of methods for fair classification, including the ones based on adversarial debiasing, optimal transport, etc. Is there a reason why authors confined the baselines to only a few?\n\n- Eq (5) is quite atypical, in the sense that it is squaring the already-squared quantities. Although this is not really used in the main algorithm, I am curious about the reason why such nested squaring is a desirable thing to do. Could authors give a little more explanation?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-QMgMctFFxH", "reviewer": "Reviewer_rJaT", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposed a novel method of training a fair classifier. In this work, the authors are concerned about the parity of accuracy across protected groups and propose a new parity score that is computationally manageable at a level of individual neurons. This work illustrates the advantages of this method in several empirical settings. ", "summary_of_the_review": "While this work has potential and is interesting, there are areas that could be improved. Specifically, \n(1) what kind of decision problems is this work concerned about and why the choice of fairness metrics are desired? (2) what is the statistical significance of the results (confidence intervals)? (3) what are the computational costs and how it is compared with the prior methods? how the accuracy changed with $\\lambda$ and the parameter size.  ", "strength_and_weaknesses": "Strength: \n* This work is easy to follow and clear.\n* AI Fairness is an important, emerging topic. \n* The insight from network size and fairness is valuable. \n* Imposing fairness regularization at the neuron level that is computationally scalable is interesting.\n\nWeakness: \n* Prediction is confused with decision-making (see below).\n* This work does not provide an argument why parity in the accuracy of prediction, specifically in their experiments, is desired at a cost of lowering the accuracy (see below).  \n* The error bars are not provided, so it is hard to evaluate the statistical significance of the results (e.g., Figure 3). The results seem marginal and statistically insignificant. \n* This work does not discuss the limitations of this work. \n* Computational costs are not discussed nor how it is compared with the other methods.  \n\n\n**More details.**\n\n1. Fairness, prediction, and decision-making. \n\nThis work is concerned with the parity of accuracy across protected groups in a prediction task. As an example, they ask how well a model can predict the income level (binary prediction >50k or below) of male and female participants. There is no decision-making downstream component here though. So I do not understand why parity is the accuracy of prediction is considered a fairness criterion. Potentially there is more intrinsic stochasticity in the income level of the female labor force participants and less stochasticity in male participants. This inherent feature reduces the accuracy of a classifier for female participants. By imposing an accuracy parity constraint we essentially degrade the quality of our prediction for male participants while keeping the quality of prediction for the female participant the same, hence degrading total accuracy. Why is this a desired feature? If there is no decision-making component why the authors are after a degraded classifier?\n\nIn the section *Connection with human society*, prediction task that is the focus of this work is confused with the decision-making task. While a classifier can be used as an ingredient in a decision-making task the connection to decision-making and what type of problem is solved here is vague. Suppose I want to predict which galaxy is star-forming and which galaxy is non-star-forming and I have a prediction disparity between spiral galaxies and elliptical galaxies. Is this fair or unfair? But now I can ask how to allocate telescope resources for a set of science applications; then we can actually discuss the fairness in the allocation of resources even though we are talking about galaxies. Now, let's go back to the income prediction example. If a classifier has a poor performance for the female participant, is that classifier unfair? a classifier cannot be intrinsically fair or unfair without specifying the downstream decision-making task, especially if we achieved the optimal Bayes classifier. \n\n2. Lack of confidence intervals. \n\nThe results are marginal and improve the previous methods only by a small margin with a significant additional computational cost (even though the proposed algorithm is scalable). This work should provide error bars for the results so the reader can understand whether the results are significant and whether the additional computational cost is worth the improvement. Without confidence intervals, it is impossible to discuss the significance of the results.  \n\n3. Connection with over-parameterization.\n\nThis section actually provides an interesting insight. However, the comparison is incomplete. Still, we do not know how the accuracy degrades across different models. Does the accuracy also converge across over parameterized models or does the accuracy differs? Hence, a designer does not know whether they should shoot for overparameterization and larger regularization or lower parameterization and smaller regularization.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject