Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper builds on recent working attempts to debais sentence encoders by considering modified sentences. Sentences are selected that, for example, contain gender cueing words, and then swap those words with a predetermined 'opposite' (i.e. man<->woman). The core novelty in the work is to train a lightweight modification the encoding of the sentence and its swap to (a) reduce the distance between the two embeddings, using a contrastive learning objective and (b) reduce the mutual information between cueing words and the new embeddings. Evaluated on a WHEAT style task, modified for sentences, the method performs significantly better than existing recent work. 

Overall I am extremely excited about the work: it is conceptually simple, the two pieces that are proposed are both evaluated, both seem to help, and it is in practice a lightweight modification to a BERT style encoder.

Positives:
+ The method is conceptually simple and the modification of the existing embedding is cheap to compute.
+ Works better than existing recent work, while maintaining equal or more of the original model's downstream accuracy
+ Lightweight modification of existing embedding system that does not require retraining of large transformer models.
+ Conceptually simple method
+ Overall extremely clean and self contained presentation

Negatives:
+ (nitpicking) The T-SNE experiment is unclear. I am unsure how it was constructed and why the graph on the right is better than the one on the left. 
+ the evaluation measure (not a complaint specific to this work), is hard to interpret. Significance testing on differences between baseline and this approach would help (in particular, Table 4)
 
review_title: Clean idea that works well with a good presentation 

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper introduces a novel technique for debiasing pretrained contextual embedding models. Their approach trains a 2 layer fully-connected neural network which takes as input the output from the pretrained model and outputs a new, "debiased" representation. This model is trained by minimizing the InfoNCE between the representation produced of original sentence and the representation of that same sentence with some tokens replaced with differently-biased tokens (e.g. "his" -> "hers"). This paper also introduces a regularizer which minimizes the CLUB between the generated representation and a word embedding for a biased token. 

Generally this paper is clearly written, addressing an interesting problem, presents some relevant supporting experiments, and seems original (I'm less familiar with other debiasing work, so I leave a better originality estimate up to the other reviewers).

The regularizer is described as a neural network, but that seems unnecessary. If it's parameterized as a neural network, and the weights are updated, then I think this is just part of the model and not really a regularizer. In addition, the motivation doesn't seem great -- it's not clear that we should be directly comparing (the mutual information of) a single word embedding and a contextual word embedding (which is built from the full sentence). All that said, empirically it seems to work well at debiasing the word embeddings, so it's a valuable contribution.

I think the full list of debiasing dimensions isn't included -- maybe I missed it somewhere? That should definitely be included in the paper (not just as a citation), and if there isn't space it should be added to the appendix.

The three fine-tuning datasets could be improved, CoLA especially is known to have really high variance even just fine-tuning BERT multiple times with different random seeds. Since there are two other fine-tuning datasets there is sufficient evidence that the approach works well. As this is a fairly general approach, clearly written up, and seems to work well, I recommend it for acceptance. The experiments are a little light, and the regularization approach is a little unorthodox, and I would increase my score if there were further experiments (on other fine-tuning tasks and measuring other types of bias) and the regularization was better motivated.

Edit: after reading the author response, my score remains unchanged. 
review_title: Clearly written, relatively simple, fairly effective

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes a method for debiasing pretrained sentence representations.
The existing representations are mapped to a new space and the mapping is trained to disregard specific bias words. Alternative versions of an input sentence are constructed by replacing specific bias words with other bias words. The representations mapping is then optimised to still produce a similar mapping in both cases. In addition, the representations are pushed away from the individual bias word representations.
Evaluation is performed on SST-2, CoLA and QNLI, showing that the method is able to produce more similar representations for sentences containing bias words, while sacrificing a small amount of performance over regular BERT.

The method of optimizing different versions of the sentence to be similar is clear and intuitive. It also shows good empirical performance.

However, the idea of maximizing the log-likelihood for the debiasing regularizer seems less motivated. Results also indicate that adding this component negatively affects classification accuracy.

The idea of measuring bias only through cosine similarity is questionable. Presumably a system that returns an identical representation for every sentence would score very highly on this evaluation? There should be some human evaluation involved or actually measuring cases where the bias word affects classification output on a downstream task.

It is unclear where the lexicon of bias words used in this work came from, is it available to everyone, how large it is and how much work would be required to construct it for other types or biases or other languages.

It is unclear whether the method only affects those word embeddings that are present in the required bias lexicon anyway. If so, then some simpler approaches could possible be just as effective, e.g. replacing all the gendered words with one gender equivalents or averaging over different versions of the same sentence containing different genders.
review_title: Could be a useful method; needs a stronger evaluation

final_score: rating: 7: Good paper, accept
review_content: This paper studied a debiasing method to remove social bias in pretrained NLP models. The authors proposed to train a neural network which takes the sentence representations of a pretrained NLP model as input and outputs the unbiased representations. The neural network is trained by maximizing the mutual information between a sentence and its “counterpart sentence”, which is automatically generated by replacing sensitive words by other values (e.g. replacing “he” with “she”). Moreover, the network can be further trained by minimizing the mutual information between the sentence representation and its sensitive word representation. The experiments show that the proposed method can effectively reduce bias while achieving better downstream task performance of the pretrained model.

Strengths:
- The studied research problem is important and would have a strong social impact. 
- The proposed methodology is well-motivated and generally reasonable.
- The experimental results show strong performance

Weakness:
- The data augmentation process seems to require pairs of replaceable words to be available.

The research problem of mitigating social bias in NLP models is important to our community and can have strong social impact. Despite the popularity of pretrained NLP models, there are limited studies on fairness of BERT-like models. Therefore, I think this work is relatively novel on this specific topic. 

The high-level intuitions of the proposed methodology is reasonable. I find both the applied losses quite intuitive. There are some minor limitations in the proposed method. For example, in the data augmentation process, it seems that replaceable words in different directions $r_j(\cdot)$ need to be available. This is more strict than having a list of words for each direction in each topic: “boy” can be replaceable with “girl”, but not necessarily “she” or “her”. For some common topics, of course, it may not be difficult to find such resources. However, for less common social topics or topics with many values (e.g., countries, regions), it could be hard to construct such a lexicon. I think it would be better if the authors experiment their methods with less restrictive constraints on what are replaceable (e.g., allowing “boy” to be replaced by “her”) and see if the performances would be severely hurt. Alternatively, the authors can discuss how to automatically construct $r_j(\cdot)$. 

I have a question about the instantiation of Eq (5). Since the authors already assume that they have replaceable words $r_j(\cdot)$ available for any socially sensitive words, would it be better to minimize $q_{\theta}(w_i^p|d_i)$ with $q_{\theta}(r_j(w_i^p)|d_i)$ for all directions $j$’s? Currently it seems like the negatives are basically other words in the same batch, which are not necessarily in the same direction. 

The experiments conducted show pretty strong performance, where the proposed methods achieve better debiasing performances and better classification performances. Some additional experiments could provide more insights. For example, how about using more than one layer of fully-connected neural networks for the fair filter? 

Generally, I recommend acceptance of this paper. 

review_title: Reasonable method for an important research topic

Meta-review: The paper presents a fair filter network to mitigating bias in sentence encoders by constructive learning. The approach reduces the bias in the embedding while preserves the semantic information of the original sentences. 

Overall, all the reviewers agree that the paper is interesting and the experiment is convincing. Especially the proposed approach is conceptually simple and effective. 

One suggestion is that the model only considers fairness metric based on the similarity between sentence embedding; however, it would be better to investigate how the "debiased embedding" helps to reduce the bias in more advanced downstream NLP applications such as coreference resolution, in which researchers demonstrate that the bias in underlying representation causing bias in the downstream model predictions. 
Decision: Accept (Poster)