Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper tackles the problem of indirect bias. The authors define how indirect bias in a corpus can be quantified, then introduce a method to reweight adjectives to correct their frequency in association with gendered nouns. These methods are then tested on a variety of embeddings, showing marginal improvements in some specific cases.
strength_and_weaknesses: * This paper tackles a relatively novel area, the systematic study and mitigation of implicit bias in word embeddings, which is an important area due to the wide use of such embeddings in a variety of applications, and the potential harm associated with the presence of implicit biases.
* The method introduced is novel and a useful step towards more robust mitigation of indirect biases in word embeddings and language models.
* The mitigation method introduced in this paper is quite basic and it is unclear if it would be possible to extend it to other domains with less well defined characterizing words (in this study the authors use "he","him","his"..../"she","her","hers",..., but I don't think such simple definition would be possible for most marginalized demographics.
* From the experiments section, it is clear that further study is needed to successfully mitigate indirect biases across models.
clarity,_quality,_novelty_and_reproducibility: * The paper is clear and well written, and introduces a simple but novel method for mitigating indirect biases through a replacement of sensitive pairs.
* There is no public code and few examples of replacements are given, but for the most part the paper is reproducible.
summary_of_the_review: This is an interesting study of indirect stereotypes in word embeddings, an important issue for language models. This paper provides an interesting first step towards gaining an understanding on building fairer and less biases models. While the mitigation procedure is relatively constrained in its application and provides only modest improvements, it is a useful step in the right direction.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Authors discuss the evaluation of indirect bias/stereotypes in word embeddings and propose a methodology to mitigate them. They do this by modifying the relationships between words before the embeddings are learned. They compare their results with CDS method (Counterfactual Data Substitution) and find that their proposed method is better than CDS in some cases. The improvement, however, is not uniform and there are cases (like the analogy task) where the accuracy is reduced by around 20%. They also calculate p values and effect sizes for undirect bias to evaluate if this approach also removes biases that are not being attempted to remove (e.g Flower/Insect–Pleasant/Unpleasant)
strength_and_weaknesses: Strength:
The paper targets an important problem of mitigating indirect bias in word embeddings. They propose a novel approach to modify relationships between co-occurring words before the embeddings are learned.

Weakness:

I found the details of the methodology slightly lacking. The experiment details have been briefly explained in the appendix which can be included in the main paper. Additionally, the math has been explained but the visualization of the approach through examples would have benefited the reader. The improvements are not uniform so details on which cases are positively/negatively impacted by the approach would have helped understand the impact of the approach.
clarity,_quality,_novelty_and_reproducibility: The approach for mitigating bias is novel but I am skeptical about its impact. The problem also focuses only on gender-occupation bias which is justified since they wanted to evaluate the impact of the approach but notes on whether or not it can extended for biases other than gender would be a good contribution. The method seems generic but more details on the extension will be helpful. I also felt that the authors can be a bit more descriptive while explaining the methodology through examples.
summary_of_the_review: The paper is well motivated but lacks in-depth analysis to actually evaluate the impact of the approach. It will benefit from a more detailed explaination of the approach and additional analysis/experiments to understand which cases are positively impacted and why.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper introduces introduces a technique to mitigate indirect bias. To use an example from the paper, after debiasing sentences [ "he is a handsome engineer", "she is a sensitive engineer" ] and removing the gendered pronoun, the indirect biases ("handsome engineer" and "sensitive engineer") still persist. Empirical motivation for addressing these indirect biases is provided by showing that implementing this co-occurrence probability value substitution produces a more pronounced debiasing effect.

The solution recommended is substituting the entry for a word-pair (a,b) in the word co-occurrence matrix with the formula from equation 2.

Overall we get mixed results at de-biasing with degradation in semantics - the abstract claims no or minimal degradation but the results section seems to show a significant (at least from what I can see) hit compared to using CDS.
strength_and_weaknesses: Strengths: The paper does introduce a good line of thought about indirect biases. Recent work analyzing WEAT has highlighted similar shortcomings and so this is a welcome idea. The substitution suggested is straightforward to implement.

These are the weaknesses I have identified:
- The empirical justification for this task is weak. For instance, WEAT is treated as a gold-standard and the effect score is treated as evidence. WEAT scores themselves are highly sensitive to the word-lists being used and recent results call into question using WEAT directly [1]. It could be that we only get semantic degradation but no additional improvements over CDS with this method. Ideally we like to see raw co-occurrence statistics or something decoupled from the metric itself.
- There seems to be a non-trivial semantic degradation compared to CDS (unless I am misunderstanding the results section).
- From the paper: "Therefore, the vocabulary must be limited so that all words in the corpus occur in a word–context pair satisfying X and in a pair satisfying Y." - This in particular seems like a strong imposition (I know the paper claims otherwise). The benefit of methods that try to identify and drop subspaces is that we can work with non-comprehensive word-lists. I am curious if this doesn't work until a fully fleshed out word-list for the embeddings is developed. I am happy to be corrected on this since it feels to me like a huge chunk of the vocabulary would need to be dropped if we are unable to acquire a large word-list (say for addressing a second form of bias beyond gender like caste or race).

[1]: https://aclanthology.org/P19-1166/

clarity,_quality,_novelty_and_reproducibility: - The idea is novel - the authors have identified a valid source of bias and provided a technique to mitigate it.
- The new co-occurrence probability value is straightforward to compute and implement so reproducibility is not an issue.
- No additional resources needed beyond existing word-lists (for at least the types of bias addressed in the work).
summary_of_the_review: - Overall, a good idea and a well-identified problem and a reasonable attempt at addressing it.
- The focus of the paper is an area of significant social import.
- There are some mild empirical concerns - mixed results, semantic degradation and a few issues with how the problem is motivated.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper looks at a novel aspect of biases (indirect stereotypes) in word embeddings and introduces a new approach to modify the biased relationships between words before embeddings are learned. All reviewers agreed that this is a novel area, and this work is a useful step toward more robust mitigation of indirect biases. Currently, the reviewers have concerns about the very constrained mitigation method, and whether it could be extended for biases other than gender.   There are also other mild empirical concerns around the mixed results. The authors’ responses helped to some extent. However, after reading the paper and the authors’ responses, as well as reviewers’ questions, the concerns seem to outweigh the novel aspect introduced in this work. Given it tackles a societally important problem, it would be great to have a robust and relatively general approach to quantify/mitigate indirect biases. This work could be strengthened with a stronger quantitative part to make a stronger submission in the future.
justification_for_why_not_higher_score: As pointed out in the meta-review, we have considered this paper from very different aspects, by taking into account the paper content, the authors' responses, as well as reviewers' questions. By weighing the strengths of this work and its weaknesses (mainly around the constrained technical mitigation method, mixed results, and its extension for biases other than gender), we do not recommend a higher score at this moment and would like to encourage the authors to incorporate reviewers' suggestions to further improve this work.
Decision: Reject