FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders
Authors: Pengyu Cheng
Weituo Hao
Siyang Yuan
Shijing Si
Lawrence Carin
Venue: ICLR 2021 Poster (2021)
Keywords: Fairness
Contrastive Learning
Mutual Information
Pretrained Text Encoders

Abstract: Pretrained text encoders, such as BERT, have been applied increasingly in various natural language processing (NLP) tasks, and have recently demonstrated significant performance gains. However, recent studies have demonstrated the existence of social bias in these pretrained NLP models. Although prior works have made progress on word-level debiasing, improved sentence-level fairness of pretrained encoders still lacks exploration. In this paper, we proposed the first neural debiasing method for a pretrained sentence encoder, which transforms the pretrained encoder outputs into debiased representations via a fair filter (FairFil) network. To learn the FairFil, we introduce a contrastive learning framework that not only minimizes the correlation between filtered embeddings and bias words but also preserves rich semantic information of the original sentences. On real-world datasets, our FairFil effectively reduces the bias degree of pretrained text encoders, while continuously showing desirable performance on downstream tasks. Moreover, our post hoc method does not require any retraining of the text encoders, further enlarging FairFil's application space.

Published as a conference paper at ICLR 2021
FAIRFIL: CONTRASTIVE NEURAL DEBIASING METHOD FOR PRETRAINED TEXT ENCODERS
Pengyu Cheng", Weituo Hao", "aa",, Yuan, Shijing Si, Lawrence Carin
Department of Electrical and Computer Engineering, Duke University (pengyu. cheng, we tut. hao, yang. yuan, shiji shijing.h. si, 1carrinedduke.ed)
ABSTRACT
Pretrained text encoders, such BERT, have been applied increasingly in various natural language processing (NLP) tasks, and have recently demonstrated signifi- cant performance gains. However, recent studies have demonstrated the existence social bias in these pretrained NLP models. Although prior works have made progress word-level debiasing, improved sentence-level fairness of pretrained encoders still lacks exploration. In this paper, we proposed the first neural debi- asing method for a pretrained sentence encoder, which transforms the pretrained encoder outputs into debiased representations via fair filter (FairFil) network. To learn the FairFil, we introduce a contrastive learning framework that only min- imizes the correlation between filtered embeddings and bias words but also pre- serves rich semantic information of the original sentences. On real-world datasets, our FairFil effectively reduces the bias degree of pretrained text encoders, while continuously showing desirable performance on downstream tasks. Moreover, our post hoc method does not require any retraining of the text encoders, further larging FairFil's 's application space.
INTRODUCTION
Text encoders, which map raw-text data into low-dimensional embeddings, have become one of the fundamental tools for extensive tasks in natural language processing (Kiros et al. 2015 Lin jet al. 2017 Shen et al., 2019 Cheng et al. 2020b). With the development deep learning. large- scale neura sentence encoders pretrained on massive text corpora, such as Infersent (Conneau et al.) 2017). ELMo (Peters et al. 2018), BERT (Devlin et al.) 2019). and GPT (Radford et al.] 2018). have become the mainstream to extract the sentence-level text representations, and have shown desirable performance on many NLP downstream tasks (MacAvaney et al., 2019 Sun et al. 2019. Zhang et al. 2019). Although these pretrained models have been studied comprchensively from many perspectives, such as performance (Joshi et al., 2020), efficiency (Sanh et al., 2019), and robustness (Liu et al. 2019), the fairness of pretrained text encoders has not received significant research attention.
The fairness issue is also broadly recognized social bias, which denotes the unbalanced model behaviors with respect to some socially sensitive topics, such as gender, race, and religion (Liang al. 2020). For data-driven NLP models, social bias is intrinsic problem mainly caused by the unbalanced data of text corpora (Bolukbasi et al. 2016). To quantitatively measure the bias degree of models, prior work proposed several statistical tests (Caliskan et al. 2017 Chaloner & Maldon- ado, 2019 2 Brunet et al. 2019). mostly focusing on word-level embedding models. To evaluate the sentence-level bias in the embedding space, |May al.) (2019) extended the Word Embedding As- sociation Test (WEAT) (Caliskan et al.) 2017) into a Sentence Encoder Association Test (SEAT). Based on the SEAT test, May et al. (2019) claimed the existence of social bias in the pretrained sentence encoders.
Although related works have discussed the measurement social bias in sentence embeddings, debiasing pretrained sentence encoders remains challenge. Previous word embedding debiasing methods (Bolukbasi et al. 2016 Kaneko Bollegala Manzini Manzini al. 2019) have limited as- sistance to sentence-level debiasing, because even if the social bias is eliminated at the word level,
"Equal contribution.

---

Published as a conference paper at ICLR 2021
the sentence-level bias can still be caused by the unbalanced combination of words in the training text. Besides, retraining a state-of-the art sentence encoder for debiasing requires a massive amount of computational resources, especially for large-scale deep models like BERT (Devlin et al. 2019) and GPT (Radford et al. 2018). To the best of our knowledge, Liang et al. (200)) proposed the only sentence-level debiasing method (Sent-Debias) for pretrained text encoders, dings are revised by subtracting the latent biased direction vectors learned by Principal Component Analysis (PCA) (Wold al. (987). However, Sent-Debias makes a strong assumption on the linear- ity of the bias in the sentence embedding space. Further, the calculation of bias directions depends highly on the embeddings extracted from the training data and the number of principal components, preventing the method from adequate generalization.
which the embed-
In this paper, we proposed the first neural debiasing method for pretrained sentence encoders. For given pretrained encoder, our method learns fair filter (FairFil) network, whose inputs are the original embeddings of the encoder, and outputs are the debiased embeddings. Inspired by the multi-view contrastive learning (Chen et al. 2020), for each training sentence, we first generate an augmentation that has the same semantic meaning but in a different potential bias direction. We con- trastively train our FairFil by maximizing the mutual information between the debiased embeddings of the original sentences and corresponding augmentations. To further eliminate bias from sensitive words in sentences, we introduce a debiasing regularizer, which minimizes the mutual information between debiased embeddings and the sensitive words' embeddings. In the experiments, our Fair- Fil outperforms Sent-Debias (Liang et al. 2020) in terms of the faimess and the representativeness of debiased embeddings, indicating our FairFil not only effectively reduces the social bias in the sentence embeddings, but also successfully preserves the rich semantic meaning of input text.
2 PRELIMINARIES
Mutual Information (MI) is a measure of the "amount of information" between two variables (Kull- back 1997). The mathematical definition of MI is
I(:; y) Ep((.)) log p( y)
p())p(())
where p(x, y) is the joint distribution of two variables (z, y), and p(x), p(y) are respectively the marginal distributions of x, y. Recently, mutual information has achieved considerable success when applied a learning criterion in diverse deep learning tasks, such conditional genera- tion (Chen et al.] 2016)) domain adaptation (Ghollami et al.) 2020), representation learning (Chen) al. 20220), and fairness (Song al., 2019). However, the calculation of exact MI in ! is well-recognized as a challenge, because the expectation w.r. p(æ, y) is always intractable, espe- cially when only samples from p(æ, y) are provided. To this end, several upper and lower bounds have been introduced to estimate the MI with samples. For MI maximization tasks (Hjelm et al.) 2018 [Chn et al., (020), Oord et al. (2018) derived a powerful MI estimator, InfoNCE, based on noise contrastive estimation (NCE) (Gutmann & Hyvärinen 2010). Given a batch of sample pairs ((rr,, y: the InfoNCE estimator is defined with a learnable score function f(x, y):
N
exp(f(2,, 9:))
INCE =
Σ
log
N
N E exp(f(x,,)))
N
For MI minimization tasks (Alemi al., 2017 Song et al. 2019), Cheng et al. (2020a) introduced contrastive log-ratio upper bound (CLUB) based on variational approximation qo (n (z) of condi- tional distribution p(y))):
N Σ
N
ICLUB = N
log go (y: (v:lzz))
Σ 1=1
log qo (2, lz.)
(3)
N
In the following, we use the above two MI estimators to induce the sentence encoder, eliminating the biased information and preserving the semantic information from the original raw text.
3 METHOD
Suppose E(-) is a pretrained sentence encoder, which encode sentence into low-dimensional embedding z = E(x). Each sentence æ = (w" w² w....., w is a sequence words. The embed- ding space of z has been recognized to have social bias in series of studies (May et al. 2019 Kurita
of
2

---

Published as a conference paper at ICLR 2021
Table Examples of generating an augmentation sentence under the sensitive topic "gender".
Bias direction
Sensitive Attribute words
Text content
Original
male
he, his
(He) is good at playing {his} basketball.
Augmentation
female
shc, her
[She) is good at playing (her) baskethall.
et al., 2019 Liang et al. 2020). To eliminate the social bias in the embedding space, we aim to learn a fair filter network f(-) on top of the sentence encoder E(-), such that the output embedding of our fair filter d = [(z can be debiased. To train the fair filter, we design a multi-view contrastive learning framework, which consists of three steps. First, for each input sentence æ, we generate an augmented sentence x' that has the same semantic meaning
but in different potential bias di- rection. Then, we maximize the mutual information between the original embedding & = f(æ) and the augmented embedding z' = f(x') with the InfoNCE (Oord al. 2018) contrastive loss. Fur- ther, we design a debiasing regularizer to minimize the mutual information between d and sensitive attribute words in æ. In the following, we discuss these three steps in detail.
3.1 DATA AUGMENTATIONS WITH SENSITIVE ATTRIBUTES
We first describe the sentence data augmentation process for our FairFil contrastive learning. De- note a social sensitive topic as T = {D1, D where D; (k 1,..., is one of the potential bias directions under the topic. For example, if T represents the sensitive topic "gen- der". then T consists two potential bias directions {D1, D2] = {f ("male", "female") "). Similarly, if T is set the major "religions" " of the world, then T could contain ( Ds, Da} { ("Christianity, "Islam", "Judaism", "Buddhism" ") as four components.
For given social sensitive topic T = (DI,.. Dx); ). if word is related to of the potential bias direction Dr. (denote as w E Dz). we call w a sensitive attribute word of Dr (also called bias attribute word in Liang et al. 2020)). For sensitive attribute word w E DD, suppose we always find another sensitive attribute word u E Dj, such that w and u has the equivalent semantic meaning but in a different bias direction. Then call u as replaceable word of w in direction Dj. and denote as u = (w)). For the topic "gender" = {"male", "female") }, the word w "boy"" is in the potential bias direction D1= "male": ; a replaceable word of "boy" in "female" " direction is r2(w)= "girl" E 'D2.
With the above definitions, for each sentence æ. we generate an augmented sentence x' such that x' has the same semantic meaning as æ but in different potential bias direction. More specifically, for a sentence x = (w-,, w² .u..................... that each w (p E P) is sensitive attribute words in direction DR. We further make a reasonable assumption that the embedding bias of direction D; is only caused by the sensitive words {w> JPEP in To sample an augmentation to œ, first select another potential bias direction Dj, and then replace all sensitive attribute words by their replaceable words in the direction Dj. That is, x' = [u², ",....,,") ". where o' = w if I & P, and of = ry(" if I E P. In Table we provide an example for sentence augmentation under the " "gender" topic.
...... w²), we first find the sensitive word positions as an index set P, such
3.2 CONTRASTIVE LEARNING FRAMEWORK
After obtaining the sentence pair (ær, x²) with the augmentation strategy from Section 3.1 we con- struet contrastive learning framework to learn our debiasing fair filter f()). As shown in the Figure D (a), our framework consists of the following two steps:
(I) We encode sentences (a, into embeddings (z, z') with the pretrained encoder E(-). Since and æ' have the same meaning but different potential bias directions, the embeddings (z, will have different bias directions, which are caused by the sensitive attributed words in x and x'.
(2) We then feed the sentence embeddings (z,z') through our fair filter f(-) to obtain the debiased embedding outputs (d, d'). Ideally, d and d' should represent the same semantic meaning with- out social bias. Inspired by SimCLR (Chen et al. 2020). encourage the overlapped semantic information between d and d' by maximizing their mutual information [(d; d').
However, the calculation of I(d; d') is practically difficult because only embedding samples of d and are available. Therefore, we use the InfoNCE mutual information estimator (Oord et al.) 2018) to minimize the lower bound of [(d; d') instead. Based learnable score function g(
3

---

Published as a conference paper at ICLR 2021
w
ICLUB
d
d'
E(-)
2
f())
d
I(d,d'
INC
E(-)
Z
f())
d'
(a)
(b)
(a) Contrastive learning framework of FairFil: Sentence æ and its augmentation x' are encoded into embeddings d and d'. respectively. w is the embedding of sensitive attribute word selected from æ. INCE maximizes the mutual information between d and d'; IClUB eliminates the bias information of w" from d. (b) Illustration of information in d and d': The blue and red circles represent the information between d and d'. The shadow area represents the bias information of both embeddings.
Figure 1:
d and d', respectively. The intersection the mutual information
the contrastive InfoNCE estimator is calculated within a batch of samples {(d, )
1 N
N
explo(di d')) log
INCE
(4)
N j= explo(d,, d,))
By maximize INCE, we encourage the difference between the positive pair score g(d, d') and the negative pair score g(d;, d'); that d: can share more semantic information with d' than other embeddings d'zzi-
3.3 DEBIASING REGULARIZER
Practically, the contrastive learning framework Section 3.2 can already show encouraging debi- asing performance (as shown in the Experiments). However, the embedding d can contain extra biased information from z, that only maximizing [d;; d') fails to eliminate. To encourage no extra bias in d, we introduce a debiasing regularizer which minimizes the mutual information between embedding d and the potential bias from embedding z. As discussed in Section [3.1 in our frame- work the potential bias of is assumed to come from the sensitive attribute words in æ. Therefore, we should reduce the bias word information from the debiased representation d. Let w" be the embedding of sensitive attribute word wP in sentence ac. The word embedding up always be obtained from the pretrained text encoders (Bordia & Bowman 2019). We then minimize the mutual information I(wP: d)), using the CLUB mutual information upper bound (Cheng al., (020a) to estimate Z(w"; d) with embedding samples. Given a batch of embedding pairs {(d,, w ) we can calculate the debiasing regularizer as:
N
ICLUB N
log go (wo
1 N
log go (wef) [d.)]
(5)
where go is variational approximation to ground-truth conditional distribution pl (wnld). We parame- terize go with another neural network. proved in Cheng et al. (2020a), the better qo (w d) approx- imates p(w w (d), the more accurate ICLUB serves as the mutual information upper bound. Therefore, besides the loss in ( we also maximize the log-likelitn liglliill of qe (w ()) with samples {(d, )
Based on the above sections, the overall learning scheme of our fair filter (FairFil) is described in Algorithm [ Also, we provide an intuitive explanation to the two loss terms in our framework. In Figure l[b,, the blue and red circles represent d and d', respectively, in the embedding space. The intersection I(d; d') is the common semantic information extracted from sentences æ and æ', while the shadow parts to coincident circles. By maximizing INCE term, we enlarge the overlapped area of d and d': by minimizing ICLUB. we shrink the biased shadow parts.
the extra bias. Note that the perfect debiased embeddings lead
4

---

Published as a conference paper at ICLR 2021
Algorithm Updating the FairFil with sample batch
Begin with the pretrained text encoder E'-), and batch of sentences (æ; }
Find the sensitive attribute words {w²) and corresponding embeddings (w").
Generate augmentation z; from 2;, by replacing {w²) with (r"(w")
Encode (z,, x²) into embeddings d, f (x(,)) )). ). d' fEE()))
Calculate INCE with {(d, d and score function g. if' adding debiasing regularizer then
Update the variational approximation go(w)d) by maximizing log-likelihood with {(d, w()}
Calculate ICLUB with qo (w d) and {(ds, w,))
Learning loss L -INCE + BICLUB.
else
Learning loss == =- -INCE- end
Update FairFil f and score function by gradient descent with respect L.
RELATED WORK
4.1 BIAS NATURAL LANGUAGE PROCESSING
Social bias has recently been recognized as important issue in natural language processing (NLP) systems. The studies on bias in NLP are mainly delineated into two categories: bias in the embed- ding spaces, and bias in downstream tasks (Blodgett al.) 2020). For bias in downstream tasks, the analyses cover comprehensive topics, including machine translation (Stanovsky et al., 2019). lan- guage modeling (Bordia & Bowman 2019), sentiment analysis (Kiritchenko & Mohammad 2018) and toxicity detection Dixon et al., 2018). The social bias in embedding spaces has been stud- ied from two important perspectives: bias measurements and and debiasing methods. To measure the bias in an embedding space. Caliskan et al. (2017) proposed a Word Embedding Association Test (WEAT), which compares the similarity between two sets of target words and two sets of at- tribute words. May et (2019) further extended the WEAT to a Sentence Encoder Association Test (SEAT), which replaces the word embeddings by sentence embeddings encoded from pre-defined biased sentence templates. For debiasing methods, most of the prior works focus on word-level rep- resentations (Bolukbasi et al. 2016] [Brddia & Bowman 2019). The only sentence-level debiasing method is proposed by [Liang et al. (2020), which learns bias directions by PCA and subtracts them in the embedding space.
4.2 CONTRASTIVE LEARNING
Contrastive learning a broad class of training strategies that learns meaningful representations by making positive and negative embedding pairs more distinguishable. Usually, contrastive learning requires a pairwise embedding critic as a similarity/distance of data pairs. Then the learning ob- jective is constructed by maximizing the margin between the critic values of positive data pairs and negative data pairs. Previously contrastive learning has shown encouraging performance in many tasks, including metric learning (Weinberger et al. 2006 Davis et al.) 2007). word representation learning (Mikolov et al., 2013). graph learning ( (Tang et al. [015 Grover & Leskovec) 2016), etc. Recently, contrastive learning has been applied the unsupervised visual representation learning task, and significantly reduced the performance gap between supervised and unsupervised learn- ing (He et al. 2020 Chen al. 2020 Qian et al. 2020). Among these unsupervised methods, Chen et al. (2020) proposed a simple mieti-view contrastive learning framework (SimCLR). For each im- age data, SimCLR generates two augmented images, and then the mutual information of the two augmentation embeddings is maximized within a batch of training data.
EXPERIMENTS
We first describe the experimental setup in detail, including the pretrained encoders, the training of FairFil, and the downstream tasks. The results of our FairFil are reported and analyzed, along with evaluate our neural debiasing method from two perspectives: (1) fairness: we compare the bias degree of the original and debiased sentence embed- dings for debiasing performance; and (2) representativeness: we apply the debiased embeddings into downstream tasks, and compare the performance with original embeddings.
the previous Sent-Debias method. In general,
5

---

Published as a conference paper at ICLR 2021
5.1 BIAS EVALUATION METRIC
To evaluate the bias in sentence embeddings, we use the Sentence Encoder Association Test (SEAT) (May et al.] 2019), which is an extension of the Word Embedding Association Test (WEAT) (Caliskan et al. (017). The WEAT test measures the bias in word embeddings by com- paring the distances of two sets of target words to two sets of attribute words. More specifically, denote and y as two sets of target word embeddings (e.g., X' includes "male" words such as "boy"" and "man": y contains "female" words like "girl" and "woman"). The attribute sets A and B are selected from some social concepts that should be "equal" to and ) (e.g., career or personality words). Then the bias degree w.zf attributes B) of each word embedding is defined as:
s (t. At B) a cos(t, (t, a.) meanbes cos(t. b),
6
where cos(s.) ) is the cosine similarity. Based on (6. the normalized WEAT effect size is:
mealleexss A. B) mean
dWEAT
stdexxuys(tAA,
The SEAT test extends WEAT by replacing the word embeddings with sentence embeddings. Both target words and attribute words are converted into sentences with several semantically bleached sentence templates (e.g., "This is <word>"). Then the SEAT statistic is similarly calculated with based on the embeddings of converted sentences. The closer the effect size is zero, the more fair the embeddings are. Therefore, we report the absolute effect size as the bias measure.
5.2 PRETRAINED ENCODERS
We test our neural debiasing method on BERT (Devlin et al. 2019). Since the pretrained BERT requires the additional fine-tuning process for downstream tasks, we report the performance of our FairFil under two scenarios: (1) pretrained BERT: we directly learn our FairFil network based on pretrained BERT without any additional fine-tuning: and (2) BERT post tasks: we fix the pa- rameters of the FairFil network learned on pretrained BERT, and then fine-tune the BERT+FairFil together on task-specific data. Note that when fine-tuning, our FairFil will no longer update, which satisfies a fair comparison to Sent- Debtas (Liang et al. 2020).
For the downstream tasks of BERT, we follow the setup from Sent-Debias (Liang al., 2020) and conduct experiments on the following three downstream tasks: (1) SST-2: A sentiment classifi- cation task on the Stanford Sentiment Treebank (SST-2) dataset (Socher et al.] 2013), on which sentence embeddings are used to predict the corresponding sentiment labels; (2) CoLA: Another sentiment classification task on the Corpus of Linguistic Acceptability (CoLA) grammatical accept- ability judgment (Warstadt et al. 2 and (3) QNLI: A binary question answering task on the Question Natural Language Inference (QNLI) dataset (Wang et al. 2018).
5.3 TRAINING OF FAIRFIL
We parameterize the fair filter network with one-layer fully-connected neural networks with the ReLU activation function. The score function g in the InfoNCE estimator is set to two-layer fully-connected network with one-dimensional output. The variational approximation qo in CLUB estimator is parameterized by multi-variate Gaussian distribution go d) = N(u(d), a² (d)), where uu(-) and (-)) are also two-layer fully-connected neural nets. The batch size is set to 128. The learning rate is 1 10-3 We train the fair filter for 10 epochs.
For an appropriate comparison, we follow the setup of Sent-Debias (Liang et al. 2020 and select the same training data for the training of FairFil. The training corpora consist 183.060 sentences from the following five datasets: WikiText-2 (Merity et al. 201y), Stanford Sentiment Treebank (Socher jt al. 2013), Reddit (V'
al. 2017). MELD (Poria et al. 2019) and POM (Park et al. 2014). Following Liang et al. (2020), we mainly select "gender" as the sensitive topic T, and use the same pre-defined word sets of sensitive attribute words and their replaceable words as Sent-Debias did. The word embeddings for training the debiasing regularizer is selected from the token embedding of the pretrained BERT.
5.4 DEBIASING RESULTS
In Tables 2 and 3 we report the evaluation results of debiased embeddings on both the absolute SEAT effect size and the downstream classification accuracy. For the SEAT test, we follow the

---

Published as a conference paper at ICLR 2021
Table 2: Performance of debiased embeddings on Pretrained BERT and BERT post SST-2.
Pretrained
BERT
BERT post SST-2
Origin
Sent-D
FairF
FairF
Origin
Sent-D
FairF-
FairF
Names, Carcer/Family
0,477
0.096
0.218
0.182
0,036
0.109
0.237
0.218
Terms, Career/Family
0.108
0.437
0.086
0.076
0.010
0.057
0.376
0.377
Terms, Math/Arts
0.253
0.194
0.133
0.124
0.219
0.221
0.301
0.263
Names, Math/Arts
0.254
0.194
0.101
0.082
1.153
0.755
0.084
0.099
Terms, Science/Arts
0.399
0.075
0.218
0.204
0.103
0.081
0.133
0.127
Names, Science/Art,
0.636
0,540
0.320
0.235
0.222
0.047
0,017
0.005
Avg. Abs. Effect Size
0.354
0.256
0.179
0.150
0.291
0.212
0.191
0.182
Classification Acc.
92.7
89.1
91.7
91.6
Table 3: Performance of debiased embeddings on BERT post CoLA and BERT post QNLI
BERT post CoLA
BERT post QNLI
Origin
Sent-D
FairF
FairF
Origin
Sent-D
FairF
FairF
Names, Carcer/Family
0.009
0.149
0.273
0.034
0.261
0.054
0.196
0.103
Terms, Career/Family
0.199
0.186
0.156
0.119
0.155
0.004
0.050
0.206
Terms, Math/Arts
0.268
0.311
0.008
0.092
0.584
0.083
0.306
0.323
Names, Math/Arts
0.150
0.308
0.060
0.101
0.581
0.629
0.168
0.288
Terms, Science/Arts
0.425
0.163
0.245
0.249
0.087
0.716
0.500
0.245
Names, Science/Art,
0.032
0.192
0.102
0.127
0.521
0.443
0.378
0.167
Avg. Abs. Bffect Size
0.181
0.217
0.141
0.120
0.365
0.321
0.266
0.222
Classification Acc.
57.6
55.4
56.5
56.5
91.3
90.6
91.0
90.8
setup in Liang al. 2) and test the sentence templates of Terms/Names under different domains
designed by CCliskann et al. (2017). The column name Origin refers to the original BERT results, and Sent-D is short for Sent-Debias (Liang et al. 2020). FairFil and FairFil (as FairF and FairF in the tables) are our method without/with the debiasing regularizer in Section 3.3 The best results of effect size (the lower the better) and classification accuracy (the higher the better) are bold among Sent-D, FairFil-, and FairFil. Since the pretrained BERT does not correspond to any downstream task, the classification accuracy is not reported for it.
From the SEAT test results, our contrastive learn- ing framework effectively reduces the gender bias for both pretrained BERT and fine-tuned BERT under most test scenarios. Comparing with Sent-Debias, our FairFil reaches lower the individual
Table 4: Comparison of average debiasing per- formance on pretrained BERT
Method
Bias Degree
BERT origin (Devlin al. 2019
0.354
bias degree on the majority SEAT tests. Considering the average of abso- lute effect size, our FairFil is distinguished by significant margin to Sent-Debias. Moreover, our FairFil achieves higher downstream classi- fication accuracy than Sent-Debias. which indi- cates learning neural filter networks can preserve more semantic meaning than subtracting bias di- rections learned from PCA.
FastText (Bojanowski et al. 2017
0.565
BERT word (Bolukbast et al 2016
0.861
BERT simple (May al.
0.298
Sent-Debias (Liang et al.
2020
0.256
FairFil (Ours)
0.179
FairFil (Ours)
0.150
For the ablation study, we also report the results of FairFil without the debiasing regularizer, in FairF-. Only with the contrastive learning framework, FairF- already reduces the bias effectively and even achieves better effect size than the FairF on some of the SEAT tests. With the debiasing regularizer, FairF has better average SEAT effect sizes but slightly loses in terms of the downstream performance. However, the overall performance of FairF and FairF- shows trade-off between fairness and representativeness of the filter network.
We also compare the debiasing performance on a broader class baselines, including wevel debiasing methods, and report the average absolute SEAT effect size the pretrained BERT en- coder. Both FairF- and FairF achieve a lower bias degree than other baselines. The word-level debiasing methods (FastText (Bojanowski al. 2017) and BERT word (Bolukbasi al. 2016))

---

Published as a conference paper at ICLR 2021
average absolute effect sizee
SST-2
CoLA
QNJI
Ours Sent-D
0.2-
0.40
0.49
0.35
0.35
032-
0.30
0.32
0.23
0.25
0.4
0.29
0.20
percentage of WikIText22
Figure 2: Influence of the training data proportion to debias degree of BERT.
technology art
family
science
6
family
he
shg
dance
she he
math
science math
career
literature technology
art
literature
career
dance
Figure 3: T-SNE plots of sentence embedding mean of each words contextualized in templates. The left-hand side is from the original pretrained BERT; the right-hand side is from our FairFil.
have the worst debiasing performance, which validates our observation that the word-level debias- ing methods cannot reduce sentence-level social bias in NLP models.
5.5 ANALYSIS
To test the influence of data proportion on the model's s debiasing performance, we select WikiText-2 with 13,750 sentences as the training corpora following the setup in Liang et al. (2020). Then we randomly divide the training data into equal-sized partitions. We evaluate the bias degree of the sentence debiasing methods on different combinations of the partitions, specifically with training data proportions (20%, 40%, 60%, 80%, 100%). Under each data proportion, we repeat the training 5 times to obtain the mean and variance of the absolute SEAT effect size. In Figure 2 we plot the bias degree of BERT post tasks with different training data proportions. In general, both Sent-Debias and FairFil achieve better performance and smaller variance when the proportion of training data is larger. Under a 20% training proportion, our FairFil can better remove bias in text encoder, which shows FairFil has better data efficiency with the contrastive learning framework.
To further study output debiased sentence embedding, we visualize the relative distances of at- tributes and targets of SEAT before/after debiasing process. We choose the target words "he" and "she." Attributes are selected from different social domains. We first contextualize the selected words into sentence templates described in Section 5.1 We then average the original/debiased em- beddings of these sentence template and plot the t-SNE (Maaten & Hinton 2008) in Figure From the t-S,E, the debiased encoder provides more balanced distances from gender targets "he/she" to the attribute concepts.
6 CONCLUSIONS
This paper has developed a novel debiasing method for large-scale pretrained text encoder neural networks. We proposed fair filter (FairFil) network, which takes the original sentence embeddings as input and outputs the debiased sentence embeddings. To train the fair filter, we constructed multi-view contrast learning framework, which maximizes the mutual information between each sentence and its augmentation. The augmented sentence is generated by replacing sensitive words in the original sentence with words in similar semantic but different bias directions. Further, we designed debiasing regularizer that minimizes the mutual information between the debiased embeddings and the corresponding sensitive words in sentences. Experimental results demonstrate the proposed FairFil not only reduces the bias in sentence embedding space, but also maintains the semantic meaning of the embeddings. This post hoc method does not require access to the training corpora, or any retraining process of the pretrained text encoder, which enhances its applicability.
8

---

Published as a conference paper at ICLR 2021
ACKNOWLEDGEMENTS
This research was supported in part by the DOE, NSF and ONR.
REFERENCES
Alexander A Alemi, Ian Fischer, Joshua Dillon, and Kevin Murphy. Deep variational information bottleneck. In International Conference Leurning Representations, 2017.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. Language (technology) is power: A critical survey of" bias" in nlp. arXiv preprint arXiv:2005. 14050, 2020.
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information, Transactions of the Association for Computational Linguistics, 5:135-146,
2017.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman homemaker? debiasing word embeddings. In Advances in neural information processing systems, pp. 4349-4357, 2016.
Shikha Bordia and Samuel R. Bowman. Identifying and reducing gender bias word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pp. 7-15, Minneapolis, Minnesota, 2019.
Brunet, Colleen standing the origins of bias in word embeddings. In International Conference on Machine Learn- ing. pp. 803-811, 2019.
Ashton Anderson, and Richard Zemel. Under-
Aylin Caliskan, Joanna Bryson, and Arvind Narayanan, Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334) 2017.
Kaytlin Chaloner and Alfredo Maldonado. Measuring gender bias in word embeddings across do- mains and discovering new gender bias word categories. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pp. 25 12- 2019.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th international conference on Machine learning, 2020.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in neural information processing systems, pp- 2172-2180, 2016.
Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: A contrastive log-ratio upper bound of mutual information. In Proceedings of the 37th international conference on Machine learning, 2020a.
Pengyu Cheng, Martin Renqiang Min, Dinghan Shen, Christopher Malon, Yizhe Zhang, Yitong Li, and Lawrence Carin. Improving disentangled text representation learning with information- theoretic guidance. In Proceedings of the 58th Annual Meeting of the Association for Computa- tional Linguistics, pp. 7530-7541, 2020b.
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. Supervised learning of universal sentence representations from natural language inference data. In EMNLP, 2017.
Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S Dhillon. Information-theoree metric learning. Proceedings of the 24th international conference on Machine learning. pp- 209-216, 2007.

---

Published as a conference paper at ICLR 2021
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4171-4186, 2019.
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and miti- gating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on Al. Ethics, and Society, pp. 67-73, 2018.
Behnam Gholami, Pritish Sahu, Ognjen Rudovic, Konstantinos Bousmalis, and Vladimir Pavlovic. Unsupervised multi-target domain adaptation: An information theoretic approach. IEEE Trans- actions on Image Processing. 29:3993 4002, 2020.
Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 855-864, 2016.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. Proceedings of the Thirteenth International Conference Artificial Intelligence and Statistics, pp. 297-304, 2010.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729- 9738,, 2020.
Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchilde Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In International Conference on Learning Representations, 2018.
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel Weld, Luke Zettlemoyer, and Omer Levy. Span- bert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64-77, 2020.
Masahiro Kaneko and Danushka Bollegala. Gender ponder-preeervig debiasing for pre-trained word em- beddings. arXiv preprint arXiv: 1906000200 2019.
Svetlana Kiritchenko and Saif Mohammad. Examining gender and race bias in two hundred senti- ment analysis systems. Proceedings of the Seventh Joint Conference on Lexical and Computa- tional Semantics, pp. 23-53,220 2018.
Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In Advances in neural information processing systems, pp. 3294 3302, 2015.
Solomon Kullback. Information theory and statistics. Courier Corporation, 1997.
Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. Measuring bias in contextualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing. pp. 166-172. 2019.
Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philippe Morency. Towards debiasing sentence representations. In Proceedings the 58th Anmual Meeting of the Association for Computational Linguistics, pp. 5502-5515, 2020.
Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 17Xiv:1703.03110,, 2017.
A structured self-attentive sentence embedding. arXiv preprint
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv::1077 1692, 2019.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 2579 2008
10

---

Published as a conference paper at ICLR 2021
Sean MacAvaney, Andrew Yates, Arman Cohan, and Nazli Goharian. Cedr: Contextualized embed- dings for document ranking. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1101-1104, 2019.
Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W Black. Black to criminal as cau- casian is to police: Detecting and removing multiclass bias in word embeddings. arXiv preprint wrXiv:1904.04000 2019.
Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. On mea- suring social biases in sentence encoders. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume (Long and Short Papers), pp. 622-628, 2019.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In ICLR, 201y.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word represen- tations in vector space. arXiv preprint arXiv: 2013.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.0374 2018
Sunghyun Park, Han Suk Shim, Moitreya Chatterjee, Kenji Sagae, and Louis-Philippe Moreney. Computational analysis of persuasiveness in social multimedia: A novel dataset and multimodal prediction approach. In Proceedings of the 16th International Conference on Multimodal Inter- action, pp. 50-57, 2014.
Matthew Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of NAACL-HLT, pp. 2227-2237, 2018.
Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. MELD: A multimodal multi-party dataset for emotion recognition in conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.
527-536, 2019.
Rui Qian, Tianjian Meng, Boqing Gong, and Yin Cui. Spatiotemporal contrastive video representation learning. arXiv preprint arXiv:2008.0 03800, 2020.
Yang, Huisheng Wang, Serge Belongie,
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language under- standing by generative pre-training, 2018.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
Dinghan Shen, Pengyu Cheng, Dhanasekar Sundararaman, Xinyuan Zhang, Qian Yang. Meng Tang. Asli Celikyilmaz, and Lawrence Carin. Learning compressed sentence representations for on- device text processing. In Proceedings of the 57th Annual Meeting of the Association for Compu- tational Linguistics, pp. 107-116, 2019.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference empirical methods in natural language pro- cessing, pp. 1631-1642, 2013.
Jiaming Song. Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. Learning controllable fair representations. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2164-2173, 2019.
Gabriel Stanovsky, Noah A Smith, and Luke Zettlemoyer. Evaluating gender bias in machine trans- lation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguis- fies, pp. 1679-1684, 2019.
11

---

Published as a conference paper at ICLR 2021
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequen- tial recommendation with bidirectional encoder representations from transformer. In Proceed- ings of the 28th ACM International Conference Information and Knowledge Management, pp- 1441-1450, 2019.
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: information network embedding. Proceedings of the 24th international conference on world wide web, pp. 1067-1077, 2015.
Large-scale
Michael Voolske, Martin Potthast, Shahbaz Syed, and Benno Stein. TL:DR: Mining Reddit to learn automatic summarization. In Proceedings of the Workshop on New Frontiers in Summarization, pp. 11-559-55 53, 2017.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 8 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353-355, 2018.
Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641, 2019.
Kilian Q Weinberger, John Blitzer, and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. In Advances in neural information processing systems, pp. 1473- 1480, 2006.
Svante Wold, Kim Esbensen, and Paul Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37-52, 1987.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluat- ing text generation with bert. arXiv preprint arXiv: (00.:::99440075555 2019.
12


Reviews:
[{"review_id": "tEknSDd6Qv", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method for debiasing pretrained sentence representations.\nThe existing representations are mapped to a new space and the mapping is trained to disregard specific bias words. Alternative versions of an input sentence are constructed by replacing specific bias words with other bias words. The representations mapping is then optimised to still produce a similar mapping in both cases. In addition, the representations are pushed away from the individual bias word representations.\nEvaluation is performed on SST-2, CoLA and QNLI, showing that the method is able to produce more similar representations for sentences containing bias words, while sacrificing a small amount of performance over regular BERT.\n\nThe method of optimizing different versions of the sentence to be similar is clear and intuitive. It also shows good empirical performance.\n\nHowever, the idea of maximizing the log-likelihood for the debiasing regularizer seems less motivated. Results also indicate that adding this component negatively affects classification accuracy.\n\nThe idea of measuring bias only through cosine similarity is questionable. Presumably a system that returns an identical representation for every sentence would score very highly on this evaluation? There should be some human evaluation involved or actually measuring cases where the bias word affects classification output on a downstream task.\n\nIt is unclear where the lexicon of bias words used in this work came from, is it available to everyone, how large it is and how much work would be required to construct it for other types or biases or other languages.\n\nIt is unclear whether the method only affects those word embeddings that are present in the required bias lexicon anyway. If so, then some simpler approaches could possible be just as effective, e.g. replacing all the gendered words with one gender equivalents or averaging over different versions of the same sentence containing different genders.", "extra_scores": {}, "extra_text": {}}, {"review_id": "pOIfSPcnUY", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper builds on recent working attempts to debais sentence encoders by considering modified sentences. Sentences are selected that, for example, contain gender cueing words, and then swap those words with a predetermined 'opposite' (i.e. man<->woman). The core novelty in the work is to train a lightweight modification the encoding of the sentence and its swap to (a) reduce the distance between the two embeddings, using a contrastive learning objective and (b) reduce the mutual information between cueing words and the new embeddings. Evaluated on a WHEAT style task, modified for sentences, the method performs significantly better than existing recent work. \n\nOverall I am extremely excited about the work: it is conceptually simple, the two pieces that are proposed are both evaluated, both seem to help, and it is in practice a lightweight modification to a BERT style encoder.\n\nPositives:\n+ The method is conceptually simple and the modification of the existing embedding is cheap to compute.\n+ Works better than existing recent work, while maintaining equal or more of the original model's downstream accuracy\n+ Lightweight modification of existing embedding system that does not require retraining of large transformer models.\n+ Conceptually simple method\n+ Overall extremely clean and self contained presentation\n\nNegatives:\n+ (nitpicking) The T-SNE experiment is unclear. I am unsure how it was constructed and why the graph on the right is better than the one on the left. \n+ the evaluation measure (not a complaint specific to this work), is hard to interpret. Significance testing on differences between baseline and this approach would help (in particular, Table 4)\n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "ol-6lYPgjkc", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studied a debiasing method to remove social bias in pretrained NLP models. The authors proposed to train a neural network which takes the sentence representations of a pretrained NLP model as input and outputs the unbiased representations. The neural network is trained by maximizing the mutual information between a sentence and its “counterpart sentence”, which is automatically generated by replacing sensitive words by other values (e.g. replacing “he” with “she”). Moreover, the network can be further trained by minimizing the mutual information between the sentence representation and its sensitive word representation. The experiments show that the proposed method can effectively reduce bias while achieving better downstream task performance of the pretrained model.\n\nStrengths:\n- The studied research problem is important and would have a strong social impact. \n- The proposed methodology is well-motivated and generally reasonable.\n- The experimental results show strong performance\n\nWeakness:\n- The data augmentation process seems to require pairs of replaceable words to be available.\n\nThe research problem of mitigating social bias in NLP models is important to our community and can have strong social impact. Despite the popularity of pretrained NLP models, there are limited studies on fairness of BERT-like models. Therefore, I think this work is relatively novel on this specific topic. \n\nThe high-level intuitions of the proposed methodology is reasonable. I find both the applied losses quite intuitive. There are some minor limitations in the proposed method. For example, in the data augmentation process, it seems that replaceable words in different directions $r_j(\\cdot)$ need to be available. This is more strict than having a list of words for each direction in each topic: “boy” can be replaceable with “girl”, but not necessarily “she” or “her”. For some common topics, of course, it may not be difficult to find such resources. However, for less common social topics or topics with many values (e.g., countries, regions), it could be hard to construct such a lexicon. I think it would be better if the authors experiment their methods with less restrictive constraints on what are replaceable (e.g., allowing “boy” to be replaced by “her”) and see if the performances would be severely hurt. Alternatively, the authors can discuss how to automatically construct $r_j(\\cdot)$. \n\nI have a question about the instantiation of Eq (5). Since the authors already assume that they have replaceable words $r_j(\\cdot)$ available for any socially sensitive words, would it be better to minimize $q_{\\theta}(w_i^p|d_i)$ with $q_{\\theta}(r_j(w_i^p)|d_i)$ for all directions $j$’s? Currently it seems like the negatives are basically other words in the same batch, which are not necessarily in the same direction. \n\nThe experiments conducted show pretty strong performance, where the proposed methods achieve better debiasing performances and better classification performances. Some additional experiments could provide more insights. For example, how about using more than one layer of fully-connected neural networks for the fair filter? \n\nGenerally, I recommend acceptance of this paper. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "XtqbsDdHe2_", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a novel technique for debiasing pretrained contextual embedding models. Their approach trains a 2 layer fully-connected neural network which takes as input the output from the pretrained model and outputs a new, \"debiased\" representation. This model is trained by minimizing the InfoNCE between the representation produced of original sentence and the representation of that same sentence with some tokens replaced with differently-biased tokens (e.g. \"his\" -> \"hers\"). This paper also introduces a regularizer which minimizes the CLUB between the generated representation and a word embedding for a biased token. \n\nGenerally this paper is clearly written, addressing an interesting problem, presents some relevant supporting experiments, and seems original (I'm less familiar with other debiasing work, so I leave a better originality estimate up to the other reviewers).\n\nThe regularizer is described as a neural network, but that seems unnecessary. If it's parameterized as a neural network, and the weights are updated, then I think this is just part of the model and not really a regularizer. In addition, the motivation doesn't seem great -- it's not clear that we should be directly comparing (the mutual information of) a single word embedding and a contextual word embedding (which is built from the full sentence). All that said, empirically it seems to work well at debiasing the word embeddings, so it's a valuable contribution.\n\nI think the full list of debiasing dimensions isn't included -- maybe I missed it somewhere? That should definitely be included in the paper (not just as a citation), and if there isn't space it should be added to the appendix.\n\nThe three fine-tuning datasets could be improved, CoLA especially is known to have really high variance even just fine-tuning BERT multiple times with different random seeds. Since there are two other fine-tuning datasets there is sufficient evidence that the approach works well. As this is a fairly general approach, clearly written up, and seems to work well, I recommend it for acceptance. The experiments are a little light, and the regularization approach is a little unorthodox, and I would increase my score if there were further experiments (on other fine-tuning tasks and measuring other types of bias) and the regularization was better motivated.\n\nEdit: after reading the author response, my score remains unchanged. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a fair filter network to mitigating bias in sentence encoders by constructive learning. The approach reduces the bias in the embedding while preserves the semantic information of the original sentences. 

Overall, all the reviewers agree that the paper is interesting and the experiment is convincing. Especially the proposed approach is conceptually simple and effective. 

One suggestion is that the model only considers fairness metric based on the similarity between sentence embedding; however, it would be better to investigate how the "debiased embedding" helps to reduce the bias in more advanced downstream NLP applications such as coreference resolution, in which researchers demonstrate that the bias in underlying representation causing bias in the downstream model predictions. 
Decision: Accept (Poster)