Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work addresses the problem of removing bias from word embeddings where the challenge is to retain information while removing bias associated with a concept. Specifically it looks addressing the shortcomings of a previous work, OSCaR, which looks at the linear subspace formed by two concepts (e.g. gender and occupation) and applies transformation to the subspace in an attempt to make the two concepts orthogonal. However, the resulting vectorial representation of the two concepts is not guaranteed to be orthogonal. The current work attempts to make the representations orthogonal by iteratively applying OSCaR with a centering step. Experimental results are provided to demonstrate the superiority of the proposed improvement over OSCaR and other debiasing techniques. The key claim is that unlike previous techniques based on projections, there is no loss of information  with the proposed method. 

The modification to OSCaR proposed in the current work consist primarily of finding a point other than origin for rotation step. This center point is chosen to be the midpoint of midpoints of concept pairs. However, if  the centering, projection, graded rotation, un-project, and un-center operations are applied only once, the resulting subspaces are not always fully orthogonal. So the work proposed to repeat these steps iteratively till orthogonality is achieved.


strength_and_weaknesses: Strengths:
      1. Simple approach which seems to do well in the evaluation.

Weaknesses:
       1. The proposed improvement is targeted for a specific debiasing technique and is not relevant for others.
       2. No mathematical proof for the convergence of iterative method is provided.
 

Questions:
        1. How robust are WEAT/SEAT scores wrt the size of the lists?
	2. Table 4 --> INLP is consistently better than ISR and iOSCAR. Why?
	3. Why is there significant difference in the absolute effect size for BERT and RoBERTa?
	4. How sensitive is the proposed method to the list of words (both number and specific words) used for training? Have you done an ablation study on the no of words used? 
	5. Why's the idea of vectorize sentences containing those words contained in a Wikipedia dump and average of sentences from Wikipedia containing the concept words correct representation of the concept? Might have concepts unrelated to the male vs. female gender and these might get lost.
	6. In Table 7, SWEAT score increased for ISR relative to Orig (1.8705 vs 1.8677) for Achieve/Anx Gen(M/F). Why?
        7. How's SWEAT score a measure of information preserved?
        8. Can you prove that the debiasing technique proposed in the submission doesn't introduce new bias inadvertently? 
clarity,_quality,_novelty_and_reproducibility: The paper is difficult to read and needs significant revision. There is not much novelty in the proposed solution. 
summary_of_the_review: The work is a minor improvement of a previous work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents an approach for "debiasing" word vectors (and sentence vectors). Because it is rotation-based, this method does not destroy information the way projection-based methods do. The main approach is to extend the OSCaR subspace correction by applying it iteratively. 
strength_and_weaknesses: STRENGTHS
- Unlike most other debiasing attempts, this work measures performance with a train/test split.
- The ISR model (this work) scores much better on the debiasing task than previous works do (95% reduction in bias) on the WEAT task.
- Iteratively running OSCaR is a good baseline to compare against, because it allows for disentangling where the "secret sauce" of ISR is or isn't coming from.

WEAKNESSES
- Nearly all of this work is dedicated to measuring if the bias is removed. Very little measures if the vectors are still useful.
- The only experiment to measure utility is Table 7's measurement of "Information Preserved." However, this doesn't measure something like downstream prediction performance or ability to reconstruct the original vectors, but instead measures a statistic of the original vectors and of the new vectors, and compares how much has changed. However, it seems like it could be possible to produce new vectors that are very different from the old ones but whose SWEAT scores happen to be similar. That would be reported as information-preserving.
- Instead of spending real state extending the approach (eg to sentence vectors, to three concepts, etc), it would be more useful to spend more time demonstrating the new vectors maintain their utility despite the orthogonalization of certain subspaces.

MINOR-ISH
- This paper uses a novel approach for defining subspaces based on the mean of a small set of coded poles (eg for male-female, words like him/he/king vs her/she/queen). This seems as sensible as any other approach, but the authors assert this method is more "reliable" than other approaches (eg normal direction of a linear classifier or the first principal component of the union of a pair) without defining what they mean by "reliable."
- On page 5, although an experiment is run 10 times, and its average is reported... the standard deviation is not also reported. If space is a concern, it could go in the appendix.
- Although this work claims that it can be extended to debias multiple subspaces which could "potentially address[] intersectional issues," this misunderstands the standard concept of intersectionality. Intersectionality is about when the sum is greater than its parts (a model that hires white women and black men but not black women might not be discriminating w.r.t. race  or w.r.t. gender but w.r.t race+gender). However, this method attempts to make each subspace orthogonal to each other, which is (arguably) useful in some settings but is explicitly counter to the concept of effects from intersectionality.

clarity,_quality,_novelty_and_reproducibility: Clarity: This work's clarity is good. It is helpful when things are emphasized for clarity with italics (eg "The underlying operation is based on a rotation, and our insight is <i>how to choose the central point that the data is rotated around.<i>"

Quality: The quality of the work is fine. There are some methodological concerns (eg evaluate utility) and quibbles (eg report std dev for experiments run many times), but there are no obvious errors in the experiments.

Novelty: The novelty of this work is somewhat low. It extends an existing algorithm and evaluates it on an existing task/dataset.

Reproducibility: The code for this work is not available.
summary_of_the_review: I have revised my assessment from marginal-reject to marginal-accept because of the additional experiments and clarifications from the authors.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an iterative recentering-rotation method to orthogonalize word embeddings to remove bias. Merits of the new method are demonstrated on debias embedding of words in standard dataset as well as large language models. Furthermore, it is shown that the information is preserved.
strength_and_weaknesses: - Overall, it is a well written paper. General audience who don't directly work on language embedding can understand this paper. The research topic is timely and the proposed method looks promising.
- Numerical experiments show significant improvement over existing methods.
- About information preservation of ISR: suppose concept A has been orthogonalized by ISR w.r.t. concept B, and concept C is orthogonalized w.r.t. concept B with ISR, would the information between A and C be preserved? One example may be that gender is orthogonalized w.r.t. un/pleasant, and race (e.g. white and black) is orthogonalized with un/pleasant, would the information between gender and race be preserved in terms of, e.g. WEAT metric? 
clarity,_quality,_novelty_and_reproducibility: The paper is well written. The method is novel.
summary_of_the_review: The contribution is timely and the merits are corroborated by strong empirical results. I have only a minor concern on information preservation, which could be important for some applications.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a mechanism of post-hoc modification of word embeddings such that problematic associations between concepts (i.e., biases) are disentanged while information about concepts is still retained. The proposed approach is a modification of previously proposed OSCaR that rectifies to desired directions (male-female vs occupations) so that they become orthogonal and rotates the rest of the components. Modifications made in the current work are: (1) choosing the central point of the rotation, (2) rerunning OSCaR iteratively to obtain better orthogonality, (3) extension to multiple concept debiasing. 
strength_and_weaknesses: ------ Strengths ------

The proposed modifications seem to improve both concept disentangling and preserving information about concepts.
Multiple subspace debiasing is an interesting direction of research.

------ Weaknesses ------

Choice of evaluation methods is questionable, presentation is highly misleading and overselling the approach.

Presentation: 

1) starting from the abstract, contribution list, etc, the paper states that it’s them who propose using disentangling of concepts rather than removing information from the embeddings. However, (1) this was first done by OSCaR they modify, (2) quality of preserving information is not noticeably better than that of OSCaR.

2) In the contribution list, they write “we can even perform a train-test split experiment (which is rarely performed in this domain)...” - again, the train-test split has already been done when evaluating the original OSCaR.

All in all, it gives the impression that the authors are trying to take credit for some of the OSCaR’s core ideas.

Evaluation: 

1) Preserving concept information has been first done and evaluated by OSCaR. However, the authors highlight preserving as a novel thing and propose their own evaluation set. Why not follow the original evaluation by OSCaR?

2) Evaluation is only intrinsic while other works (e.g., OSCaR) have also used extrinsic evaluation.

clarity,_quality,_novelty_and_reproducibility: Novelty: see comments in the weaknesses section.
summary_of_the_review: Overall, the paper leaves mixed feelings: while proposed modifications do seem to improve disentanglement quality, the paper seems to try taking credit for previous work.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a way to debias the embedding space of language representations. The reviewers had some concerns about the paper, but the authors properly addressed them in their comments. The reviewers are now, overall, quite happy to see this paper at ICLR.
justification_for_why_not_higher_score: n/a
justification_for_why_not_lower_score: Clearly there is an agreement the paper is reasonably well developed.
Decision: Accept: poster