Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: *Summary*: This paper proposes a method for training model that are robust to spurious correlations, building upon prior work that uses product-of-experts and a model explicitly trained on a dataset bias (e.g., a hypothesis-only model). Instead of using a model explicitly trained to learn the dataset bias, the authors use a “weak learner” with limited capacity. Then, this model is used in the PoE setting as in past work. The advantage of this method is that a model developer doesn’t need to know that a bias exists, since the hope is that the weak learner will implicitly learn the bias.

*Strengths*: A thorough study of using a limited-capacity auxiliary model to train more robust models, which helps a final model ignore spurious correlations that are easy to learn.

*Weaknesses*: The work is a rather straightforward extension of prior work. Furthermore, the authors only evaluate on 2 textual tasks---I would have liked to see more experiments with spurious correlations in vision (e.g., VQA or the datasets used in https://openreview.net/forum?id=ryxGuJrFvS), and other experiments on text (e.g., the TriviaQA-CP dataset in the Clark paper). As is, it’s hard to glean how broadly applicable this method actually is. I would have also liked to see more of a comparison with methods that use known bias (e.g., Clark et al or He et al)---it seems like some of the comparisons in the table aren’t completely fair.

*Recommendation*: 6 . I think this paper is a potentially-useful extension of a prior method, but I’m still somewhat unconvinced that this method is applicable in settings where the bias is hard to detect, which is what we really care about (since, if the bias is easy to detect, we can use Clark et al and other methods).

Comments and Questions:

1. The comparisons to Clark et al aren’t fair comparisons for adversarial SQuAD, since the Clark et al paper uses a different base model for adversarial SQuAD (modifed BIDAF).

2. The weak learner is a rather blunt instrument. It picks up dataset biases, but it also likely picks up features that are actually useful---not all robust features have to be difficult to learn. Is it possible to better quantify what useful information is being learned (and subsequently thrown out) by the weak learner? This would make it easier to determine if using it is worthwhile.

3. While it’s true that the weak model empirically learns to re-learn the same dataset biases targeted in prior work (e.g., negation correlates with contradiction), it’s somewhat unclear to me how well this method would translate to a setting with unknown biases. The MNLI / SQuAD examples are a bit artificial since we already have knowledge of the bias---it’s possible that weak learners can pick up on spurious features that are “easy to learn”, which are the same ones that humans notice. I’d like to see whether this method applies well to tasks where it isn’t immediately obvious that the bias is easy to learn; perhaps a synthetic experiment would be useful here. Is it possible to modulate the learnability of the bias? The synthetic experiments in the paper suggest that for cases the bias is hard to learn, this method isn’t very effective, which makes sense---in how many of the cases in the literature is the bias hard to learn? This is another reason why I think more experiments would be useful.
review_title: Potentially useful extension of prior work

final_score: rating: 7: Good paper, accept
review_content: Summary:

This paper focuses on the known problem that current NLP models tend to solve tasks by exploiting superficial properties of the training data that do not generalize. For example, in the NLI task, models learn that negation words are indicative of the label "contradiction" and high word overlap is indicative of the label "entailment". There have been many recent solutions proposed for mitigating such behavior, but existing methods have tended to assume knowledge of the specific dataset biases a priori. In this paper, the authors propose a method based on product of experts that doesn't assume particular knowledge of specific dataset biases. The method works by first training a weak model and then training a "main" model using a loss that upweights examples on which the weak model performs poorly (namely, predicts the wrong answer with high confidence). The assumption is that weak models will exploit heuristics, and so this method will deincentivize the main model to use those same heuristics. The authors evaluate on a range of tasks, including a simulated bias setting, and NLI setting, and a QA setting, and offer a fair amount of analysis of their results. In particular, the analysis showing that the weak learners do in fact adopt the biases which have been documented elsewhere in the literature is interesting, and the discussion of "how weak does the weak learner need to be" is appreciated (a few questions on this below).

Strengths:
* Straightforward method for addressing an important known problem with neural NLP models
* Thorough analysis, not just a "method and results" paper

Weaknesses:
* Novelty might be somewhat limited, method is not wildly creative (but I don't necessarily think "wild creativity" is a prerequisite for scientific value). The authors do a good job of directly contending with the similar contemporaneous work in their paper

Additional Comments/Questions:

Just a few thoughts that came up while reading...
* The weakness-of-weak-learner analysis is interesting. I imagine this is not something that can be understood in absolute terms, i.e., I would not expect there to be some level of weakness that is sufficient for all biases and all datasets. E.g., surely the lexical overlap bias is "harder" to learn than a lexical bias like the presence of negation words, since recognizing lexical overlap presupposes recognizing lexical identity. Therefore, I'd imagine knowing how weak the weak learner needs to be requires some intuition about which biases you are trying to remove, which runs counter to the primary thrust of the paper, namely, removing bias without knowing what the bias is. Thoughts?
* Its interesting that even with this the performance on hans non-entailed is still only 56%, which is better but still not exactly good, and doesn't suggest the model has learned the "right" thing so much as its has learned not to use that particular wrong thing. For research questions such as this ("is the model using the heuristic?") I always find it unsatisfying to think about performance gains that are in between 0 and 100. E.g., when we talk about human learning, we usually see an abrupt shift when the learner "gets it", and our hope in removing the spurious features with methods like yours would be that we'd help the neural models similarly "get it" and reach 100% at least on examples that isolate the effect of this spurious feature. I don't expect you to have an answer for this, but just raising to hear your thoughts. 
review_title: Straightforward method for reducing model's reliance on spurious features

final_score: rating: 7: Good paper, accept
review_content: ## Reason for score

The research problem is critical. The solution is appropriate and novel. The claims are validated. The experiments are interesting.
However, the writing in section 3, 4 et 5 should be improved. If so, I would be willing to raise my score.

## My background

My research is focused on detecting and avoiding data biases (or spurious correlations) learned by deep neural networks. This is the exact scope of this paper. However, my area of expertise is computer vision and multimodal text-image, not natural language processing.

## Summary

Context:
The paper focuses on automatically detecting data biases learned by natural language processing models and overcoming them using a learning strategy.

Problem:
The authors identify and tackle issues of state-of-the-art methods:
- they are required to already know about a certain bias to be overcome.

Solution and novelty:
The proposed method consists in 1) training a weak model that aims at detecting biases 2) overcoming these biases by training a main model using a product of experts (Hinton, 2002) with the predictions of the fixed weak model.

Claim:
- A weak model can be used to discover data biases
- The proposed method produces a main model that generalize better to out-of-distribution examples

## What I liked the most

- meta-problem of automatically detecting and overcoming biases in neural networks is critical
- well contextualized
- relevant issues of state of the art have been identified
- intro and related work are easy to read and understand
- novel, simple and interesting method to tackle them
- interesting figures
- experiments are  interesting and well chosen

## What could be improved

1. Abstract, introduction and 2. Related work
- Your research problem and solution are general and can be applied to many fields. Is there a specific reason why you decided to focus on NLP only?
- You could improve the impact of your approach by citing papers that tackle the same problem with similar solutions from different fields. "Clark et al. 2019 Don’t take the easy way out: Ensemble-based methods for avoiding known dataset biases" that you already cite ran some experiments in multiple fields (NLP, VQA, etc.). "Cadene et al. Rubi: Reducing unimodal biases for visual question answering (NeurIPS2019)" in VQA could also be cited.

3. Proposed Method
- Next to Eq1: Why an element wise sum is equivalent to an element wise multiplication after softmax? It seems wrong to me.
- It could be useful to have a general definition of the PoE loss (instead of just an example of binary cross entropy in Eq2)
- See 4.3, you should define PoE+CE here.

4. Experiments
- Overall, I think it is important that you improve the writing for this section and reduce jargon. It is really difficult to understand for readers that are not familiar with the datasets on which you perform your study. Also it is really difficult to understand which dataset is "in-distribution" or "out-of-distribution".
- You don't define "development matched accuracy" before using it.
4.1
- You use too many footnotes that could be included in the text.
4.2
- You don't define "CE" (even in the caption of Figure2).
- In Table 2, you could reduce jargon by using Weak and Main instead of "W" and "M".
- In Table 2, you don't define "An." even in the caption.
4.3
- I don't understand why "PoE+CE" is better on "Hard" 
- I don't like that you propose to use "PoE+CE" as your method of choice "to counteract these effects" without defining it in section 3. To be clear, I still don't understand what is the learning method that you propose PoE or PoE+CE?

5. Analysis
5.2
- Title is on two lines instead of one
- I don't understand "When trained jointly with the larger MediumBERT weak learner......" How many parameters? Don't expect your reader to look at Figure 4 to obtain this information.

6. Conclusion
- Could you add a discussion about the limitations of your approach. In particular: How to choose the number of parameters of your weak learner? What to choose between PoE and PoE+CE? And most critically, if you don't assess the type of biases and the amount of biases included in the dataset, how to be sure that your method will have a beneficial impact? Then, if you need to assess the type of biases, using another method that specifically targets them could be more efficient.


review_title: Official Blind Review #4

final_score: rating: 2: Strong rejection
review_content: Paper summary:
The authors argue that they have proposed a method to train robust models to biases without having prior knowledge of the biases. They argue also to provide analysis on how weak learner capacity impacts the in-domain/out-of-domain performance.

Reasons to reject:
1) The authors argue they have shown the model with limited capacity capture biases. However, this has been shown already in [1] in 2019 and therefore is not a contribution of the authors.
2) The main method proposed in this paper, is exactly the same method proposed in [2]. Please note that [2] was already available in early July 2020, and on top of existing work, the paper does not provide other contributions. 
3) About the third argued contribution on showing how the performance of the debiasing method change based on the capacity of weak learners, in [1], the authors included the discussion between the choice of weak learners on their impact. Though the method in [1] is different, the discussion in that paper still would apply here as well.  Please refer to table 1-3 and Figure 1 in [1]. 

Given the points above, and since the main method in the paper is proposed in [2], the paper does not provide enough contributions to be suitable for the ICLR venue. 

[1] Robust Natural Language Inference Models with Example Forgetting, Yaghoobzadeh et al, https://arxiv.org/pdf/1911.03861.pdf, 2019 
[2] Towards Debiasing NLU Models from Unknown Biases, Utama et al, 13 July 2020, https://openreview.net/forum?id=UHpxm2K-jHE, EMNLP 2020 
review_title: review 

Meta-review: This paper considers the problem of learning models for NLP tasks that are less reliant on artifacts and other dataset-specific features that are unlikely to be reliable for new datasets. This is an important problem because these biases limit out-of-distribution generalization. Prior work has considered models that explicitly factor out known biases. This work proposes using an ensemble of weak learners to implicitly identify some of these biases and train a more robust model. The work shows that weak learners can capture some of the same biases that humans identify, and that the resulting trained model is significantly more robust on adversarially designed challenge tasks while sacrificing little accuracy on the test sets of the original data sets.

The paper's method is useful, straightforward, and intuitively appealing. The experiments are generally well conducted. Some of the reviewers raised questions about evaluating on tasks with unknown biases. The authors addressed these concerns in discussion and we encourage them to include this in the final version of the paper using the additional page.
Decision: Accept (Poster)