Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes a method for debiasing internal model components, based on similarity-regularization. This is meant to encourage the internal representations to be either similar to representations of an _unbiased_ model, or dissimilar from representations of a _biased_ model. The authors apply this regularization to either the model representations or its gradients. The paper evaluates the approach on three tasks---natural language inference, fact checking, and paraphrase identification---and demonstrates that it improves performance on one out-of-distribution dataset for each task while incurring little degradation in in-distribution performance.
strength_and_weaknesses: - The paper is very confusing. Exposition lacks structure, claims are made without providing evidence to back them up, and there are some hand wavy arguments in this paper. Here are several examples of these issues:
  - "While improving out of distribution (OOD) performance, it was recently observed that the internal representations of the presumably debiased models are actually more, rather than less biased": Biased to what? Bias is a statistical term that's in reference to something specific. 
  - The paper relies on having a _biased_ model and an _unbiased_ model, and what those mean doesn't become clear until Section 3.2
  - It is not clear to me how you train a biased model for the unknown biases. You mention you use TinyBert but that's it.
  - You make a point in Section 3.2 that excluding samples on which a _biased_ model is correct and confident would result in an unbiased dataset. That makes the assumptions that (i) instances where your biased model is correct and confident are actually instances which contain confounding which is not always true (see [1], but even other than that, they could just be correct and confident for the right reason), and (ii) your biased model captured all instances that contain confounding. There's nothing stated as to why these assumptions might be valid.
  - "Choosing the threshold $c_t$ is performed manually by plotting the confidence of the bias model over the training set." Yet I can't tell how Figure 3 tells you that $c_t$ has to be 0.8 and not say 0.7 or 0.9 or something.
  - "Appendix A.4) shows that regularizing only $D^B$ results in better OOD performance, supporting our intuition." Did you use same hyperparameters when just regularizing $D^B$ versus when regularizing all examples or different? How were they chosen?
  - "We repeat some of the experiments using DeBERTa to verify our model is not specific to BERT." How did you select which experiments would be repeated and which not?
  - The results tables show means and error terms but its not clear: (i) are these standard error terms or standard deviation?; (ii) what is the source of randomness?; (iii) are differences in performance statistically significant?
- The paper claims that the proposed method leads to better OOD performance. Better performance on one OOD dataset does not imply better OOD performance. This is like testing the model on one example and claiming 100% accuracy if it correctly classifies that example. But the issue with that is that if the second example is incorrectly classified, it's accuracy would suddenly be 50%. Perhaps this model did better on one OOD dataset, to show that it will likely have better OOD performance overall, you need to evaluate on a battery of OOD datasets. A sound claim would be that it leads to better performance on the specific datasets (which is good).
- In related work, a whole line of work on debiasing methods is missing. Some relevant papers below:
  - Victor Veitch, Alexander D'Amour, Steve Yadlowsky, and Jacob Eisenstein. "Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests." arXiv preprint arXiv:2106.00545 (2021).
  - Zhao Wang and Aron Culotta. "Robustness to spurious correlations in text classification via automatically generated counterfactuals." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 16, pp. 14024-14031. 2021.
  - Divyansh Kaushik, Amrith Setlur, Eduard H. Hovy, and Zachary Chase Lipton. "Explaining the Efficacy of Counterfactually Augmented Data." In International Conference on Learning Representations. 2021.
  - Zhao Wang and Aron Culotta. "Identifying Spurious Correlations for Robust Text Classification." In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3431-3440. 2020.
  - Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. "Learning The Difference That Makes A Difference With Counterfactually-Augmented Data." In International Conference on Learning Representations. 2020.

- Errata:
  - incorrect quotation usage in Line 5 of introduction.
  - Third to last line of introduction: "This is different from previous methods" ---> "This is different from some previous methods" (refer to the citations above that do indeed talk about this)
  - Section 3.1: "In the case of decreaseing" ---> "In the case of decreasing"
  - Section 4.1.1, second paragraph: "As OOD test sets, we use HANS" ---> "As OOD test set, we use HANS"
  - Section 4.1.3: Thorne et al. should be \citep and not \citet
  - Section 4.1.4, second last line: "bias.For" ---> "bias. For"
  - Section 5.2, second last line: "support out hypothesis" ---> "support our hypothesis"

[1] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. "Understanding deep learning (still) requires rethinking generalization." Communications of the ACM 64, no. 3 (2021): 107-115.
clarity,_quality,_novelty_and_reproducibility: The paper is confusing, the experimental setup is halfway there and additional evaluations are needed. The novelty appears limited and it is unclear what the results tables are reporting. The authors have shared implementation details in Appendix that should help with reproducibility so that is not much of a concern to me right now.
summary_of_the_review: The paper is very confusing and lacks structure. Statements are often made without providing evidence to back them up. The paper relies on several assumptions but there is nothing stated as to why these assumptions might be valid. It is not clear what the results table shows (mean and standard deviation? mean and standard error?) but it's not clear what the source of randomness is (mean over what, different samples of training data or different seeds of a model or both?) or if the differences in performance are statistically significant. The paper claims that the proposed method leads to better OOD performance, but better performance on one OOD dataset does not imply better OOD performance overall. In related work, a whole line of work on debiasing methods is missing.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a method to address the spurious correlations / bias problem in the context of NLU. Starting from the observation in Mendelson & Belinkov (2021) that most de-biasing methods consisting in re-weighting data result in actually more biased representations, they explicitly consider to regularize the representations (or the gradients) of the full model to be more similar to a de-biased "guidance" model (trained on small amounts of de-biased data found by either an oracle or by a biased model), in order to train de-biased representations. The method is tested on NLI, Fact checking and paragraph identification.

strength_and_weaknesses: Strengths:
- *The motivation of the paper is sound*: Recent approaches (such as Deep Feature Reweighting https://arxiv.org/pdf/2204.02937.pdf) show that de-bias models just modify the last output layer, thus re-weighting robust features. It is natural to ask how to make data representations intrinsically less biased.
- *The paper is well-written*: I found the paper easy to read and self-contained, most of the details are present either in the main text or the appendix.

Weaknesses:
- *Lack of enough empirical support*:
The main hypothesis of this paper is that the guidance model representations are less biased and thus can inform the de-biasing methods into learning representations invariant to the bias features. Right now, I don't feel that this hypothesis is substantiated enough by the experiments. I would first expect convincing evidence supporting the fact that the bias is less extractable from the guidance model. An analysis of bias extractability only comes in Section 6.1, Figure 3, but it is only applied to the full regularized model and not to the "guidance" model. I am equally perplexed by Figure 3: the model that *minimizes* the similarity w.r.t the guidance model seem to have representations that are *less* biased than the model that *maximizes* the similarity w.r.t. the guidance model, which seems against the thesis of the paper? I think that most of the paper should be focused on proving that the representations of the final de-biased model are indeed less biased on several datasets: right now this analysis is confined to one dataset (HANS). It would good to include in this study the simple method RWG where the loss is re-weighted by the size of each group.

- *Results are not strong enough / missing baselines*:
Overall, the results obtained by the proposed de-biased model are not significantly better than other baselines. This would not be such an issue if the paper was focused on training de-biased representations without caring so much for improved performance (although it would leave to the authors to justify why having less biased representations would ultimately be useful). For example, if we compare the results of the paper with https://arxiv.org/pdf/1911.03861.pdf, we see lower performance on HANS and PAWS (unknown bias mitigation setting). It would also be suitable to include results for "Correct-n-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations": this method also operates on the representation space, by aligning representations between groups, thus it has an inherent similarity with the proposed approach.

- *Manual hyper-parameter search*:
I am a bit uncertain what is the rationale by which the authors chose their hyper-parameters. In Section 3.2, the authors mention "choosing the threshold c_t is performed manually by plotting the confidence of the bias model over the training set". I am not sure what is the rationale for choosing the threshold c_t, and definitely think that an automatic method to do so should be described and analyzed.

clarity,_quality,_novelty_and_reproducibility: ## Questions and concern
1. An additional concern is that the guidance model is trained on less data and therefore its representations / gradients might not be of good quality, hindering generalization of the base model. This point might be discussed in the paper.
2. Some of the recent de-biasing baselines would be needed in Table 1 (even RWG would be fine)
3. The following references are missing:
- "Increasing Robustness to Spurious Correlations using Forgettable Examples", https://arxiv.org/pdf/1911.03861.pdf
- "Deep Feature Re-weighting", https://arxiv.org/pdf/2204.02937.pdf
- "Correct-n-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations", https://arxiv.org/pdf/2203.01517.pdf

summary_of_the_review: This paper focuses on an important problem: how to estimate less biased representations rather than just creating de-biased models by re-weighting features in a biased representation. In its current form, I don't feel confident enough to state that the paper advances on its goals. Moreover, performance is on par - or lower - than standard re-weighting approaches. More empirical evidence to support the validity of the method is needed.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes SimReg, a method that achieves the goal of debiasing by increasing(decreasing) the similarity between the final model and an unbiased(biased) model. They explore two different ways to impose the regularization: on model representation or on the gradients. 
Except for similarity-based regularization, another key component of this paper is the creation of an unbiased dataset, which is critical for regularization's success. 
By manipulating internal model representations, SimReg differs from previous methods that downweigh the importance of training samples during training. 
Upon extensive evaluation, SimReg is effective in improving models' OOD performance.

strength_and_weaknesses: Strength:

1. The problem is well-motivated and the paper is easy to follow. 
2. The method is simple yet effective.
3. The experiments are carried over several different datasets that cover different types of bias, and the results are convincing. 

Weakness:
1.  One limitation of SimReg is that exclude highly-confident examples using a biased model (e.g., tinyBERT) might also filter out some "easy" or "unbiased" examples, because neural models often get over-confident. 
2. Some related works/baselines are missing. SimReg share many similarity with knowledge distillation (eq 3 is basically a variant of the typical KD loss). So some short discussion about KD would improve the paper and KD loss might be be used as another model variant. Although "intrinsic debiasing"(manipulating representations) methods are novel in the area of spurious correlation mitigation, they have been widely used in debiasing hate speech. Despite the differences in terms of areas, they might also need to be included in the related works. 
clarity,_quality,_novelty_and_reproducibility: The paper is technically solid and clearly presented. 
summary_of_the_review: The submission proposes a novel method that is effective in mitigating spurious correlation.  But there are some limitations and missing related works that have to be addressed before acceptance.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a regularization method for mitigating artifacts. Their method (SimReg+) is simple: take a model that is known to be artifact-free, and train another model to be similar to that model. The idea is somewhat counter-intuitive, but works reasonably well on some datasets, and is also shown to have fewer internal biases. The authors also propose several variants of the main idea (e.g., decreasing the similarity between the model and some model that is known to be biased), but those do not work as well as the main idea. 
strength_and_weaknesses: 
Strengths:
- A simple approach for mitigating artifacts
- Results for SimReg+ are generally better than most baselines
- The second-order similarity measure seems interesting
- The internal bias representation results are interesting

Weaknesses: 
- The proposed approach is somewhat counter-intuitive: why would making a model imitate a weak model (i.e., one trained on less data) result in a stronger model? This is sort of like doing knowledge distillation where the teacher is the smaller model. I see that empirically the idea sort-of works, although sometimes it does much worse (e.g., HANS non-entailment, PAWS) and in some cases the differences are small (Table 2). I wonder how generalizable this approach is.
- Related to the previous point - for some reason the Guidance baseline is missing from the Deberta experiments (Table 12 doesn't show any baselines except the original model).   
- Given that SimReg+ works best among all proposed methods, it might make sense to focus on it, and present the rest as ablations

Other questions:
- It is not clear why the authors chose HANS- as the X axis in Figure 3. 
clarity,_quality,_novelty_and_reproducibility: The paper is generally clear. The approach is closely tied to existing methods such as distillation (see above), though I haven't seen it applied for this purpose. 

Typos and such: 
Section 3.3: the the 

summary_of_the_review: The paper presents a simple method that seems to overall work well. However, it is not entirely clear how generalizable it is, given that it is not very intuitive and the results are sometimes mixed.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper addresses the problem of bias (spurious correlation) in the context of natural language understanding. Given the counterintuitive observations made in (Mendelson & Belinkov, 2021) that pushing language models towards debiased regimes actually yields inner reprensentations that are more biased, the authors propose to explicitly regularize the representations of their models to be more similar to a "guidance" model known to be debiased. While the motivation of this work seems sound, the reviewers had various concerns about the execution and the lack of support for some of the claims:
* The authors claimed better OOD performance while evaluating on only one dataset. They should have instead evaluated on a battery of OOD datasets to make such a claim. While the authors later rectified their claim, this also weakens the paper as claiming improved OOD performance was the main empirical claim of the abstract.
* Some of the analyzes of the paper seem to be contradictory to its motivation: "the model that minimizes the similarity w.r.t the guidance model seem to have representations that are less biased than the model that maximizes the similarity w.r.t. the guidance model, which seems against the thesis of the paper?" (Review qJJe). This reviewer wasn't satisfied by the authors' "there is no guarantee" response, as it just sidestepped the reviewer's concern. In a private discussion among reviewers, one reviewer felt the authors often limited themselves to acknowledge identified weaknesses while providing analyses on less important points.
* The reviewers also felt results were not strong enough and lacked important baselines: The authors added more experiments, but this further highlighted that results were weak (e.g., 42.9% accuracy vs. 47.6% in prior work).

While the authors addresses some of the minor points raised the reviewers (e.g., on manual hyper-parameter search), most of the main concerns still remain, and I therefore recommend rejection. 
justification_for_why_not_higher_score: While the motivation of the paper is good, the paper in its current form doesn't really provide tangible contributions (claims are not well supported by experiments).
Decision: Reject