Venue: ICLR 2020 Conference

Reviews:

final_score: rating: null
review_content: It seems that reviewer 1 and I have very similar concerns. 

I would highly encourage the authors to address these while they can. I think it's crucial to answer these, and without a response, it's unlikely that I'll keep recommending an accept for the paper. 
review_title: Important concerns

final_score: rating: 6: Weak Accept
review_content: This manuscript discusses the problem of bias shortcut employed by many machine learning algorithms (due to dataset problems or underlying effects of any algorithmic bias within an application). The authors argue that models tend to underutilize their capacities to extract non-bias signals when bias shortcuts provide enough cues for recognition. This is an interesting and important aspect of machine learning models neglected by many recent developments. 

The only problem is that the paper seems to be a bit immature as the exemplar application is too naive for illustrating the idea. The authors’ idea is to assume that
‘there is a family of feature extractors, such that features learned by any of these extractors would correspond to pure bias. Then in order to learn unbiased features, the goal is to construct a feature extractor that is ''as different as this family" as possible’
in practice, their claim is texture and color are biases; one should learn shape instead of texture and color. So the family of feature extractors are the ones with small receptive field that can only capture texture and color. Therefore, what they eventually achieved is the unbiased feature extractor only learns the shape of object and avoids learning any texture and color. 

So, the problem is, in practice, it is very hard to define the family of biased feature extractors. It really depends on the dataset and the goal. Texture and color, in general, are still important cues for object recognition, removing this information is NOT equivalent to removing bias. Just as a suggestion, the background scene might be a better definition of bias. However, with the proposal in this paper, it would be unclear how to define the family of feature extractors for describing background. Therefore, the solution given for this important problem seems to be too ad-hoc and not generalizable. 

The second example (that does not have an experiments on) is action recognition; the family of biased feature extractors is 2D-frame-wise CNNs (object recognition). The authors claim that objects are biases for action recognition systems, but again a large part of action recognition is indeed object recognition. Many actions are defined based interaction of humans with objects (e.g., opening bottle or pouring water from bottle). Some objects may be instroducing bias in the task, but not all. Again, the proposed solution in this paper cannot disentangle this. 

The authors need to survey previous texture-shape disentanglement works and then compare with those methods.



review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The paper describes a methodology for reducing model dependance on bias by specifying a model family of biases (i.e. conv nets with only 1x1 convs to model color biases), and then forcing independence between feature representations of the bias model and the a full model (i.e. conv nets with 3x3 convs to also model edges). 

Overall the method is very interesting. A few very related recent works were missing: https://arxiv.org/pdf/1908.10763.pdf and https://arxiv.org/abs/1909.03683. These works provide different formulations for factoring out know "bias oriented" models (and focus more on NLP, although the second is applied to VQA). Although, I do appreciate that the bias models used in this work are slightly more general in terms of family than those studied before, they do encode significant intuition about the target bias to be removed and perhaps in this context, the methods cited above could also be compared. That being said, I don't feel the paper needs to provide this comparison given how recent those works are, but it would improve the quality of the paper. 

One aspect that worries me about this paper is that most of the biases studied are synthetic, so specification of the bias family is trivial, in contrast to the works I mentioned above (where the bias model is potentially somewhat misspecified and needed to be discovered by different researchers). But I do really like the experiments on Imagenet-a. 

Overall the paper presents an interesting contribution that would be useful for future study in reducing bias dependance in ml.


review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: ###  Summary 

The paper proposes a method for regularizing neural networks to mitigate certain known biases from the representations learned by CNNs. The authors look at the setting in which the distribution of biases in the train and test set remains the same, but the distribution of targets given biases changes.

To remove the bias, they propose hand-designing a set of models that rely exclusively on these biases to make predictions. They then train a new unbiased model by making sure that this new model uses sufficiently different information than the biased models to make predictions (By adding a regularization term)

### Decision and reasons

I vote for a weak accept. 

Strengths: 
1: The paper is well written with a clearly defined problem-setting. The proposed method is sound and interesting, and the empirical results, thorough. 

Weaknesses:
2: The solution just pushes the problem of 'learning a model that only uses the true signal to make predictions' to 'learning a set of models that only use the noise to make predictions.' It's not clear why the latter is easier than the former. 

3: The paper does not have any baselines that directly try to remove the bias (Instead of using the two-step process). As a result, it's hard to judge how meaningful the improvements are. 


### Supporting arguments for the reasons for the decision.

Strengths: 
1: The paper does a really good job of defining the problem-setting. Contrasting cross-bias with cross-domain and in-distribution makes the goal of the paper very clear. The notation used to formalize the problem setting in Section 2.1 is also clear and concise. Moreover, the experiments on the toy dataset help clarify the proposed solution whereas experiments on Biased MNIST and Imagenet show that it successfully mitigates the bias. Finally, the authors show the importance of each component of the proposed solution by factor analysis.

Weaknesses:
2: In the most general case, it is not obvious why it is easier to define and learn a set of models that only use noise to make predictions (which is the required first step for their proposed solution) as opposed to learning a model that only uses signal (which is the goal of the problem). These two problems seem equally hard. The paper builds on the premise that in some cases former is easier (i.e. in some cases, it is easier to learn a set of models that use only noise as opposed to learning a debiased model directly). The authors give two such examples (They only explore the first experimentally.) 
1. Learning a model that relies on local texture. They achieve this by limiting the receptive field of the features. 
2. Learning a model that relies on static images to make predictions about actions in videos. 
I feel that the two given examples are very narrow. It would be nice if the authors could identify a broader class of problems for which G is given or can easily be defined. Moreover, even in these two examples, I'm not convinced that the proposed biased models only use B for making predictions. For example, for some classification tasks, the local texture could be part of the signal and not just the bias. Similarly, static images from a video do contain important information for making the prediction. 

3: The authors only compare their method to a baseline that does nothing to debias the representations. Even though this is an important comparison (as it shows that the proposed method can debias the representations), it does not tell the reader how effective the proposed method is compared to other possible solutions. The results would be more meaningful if the authors could include at-least a simple baseline that tries to remove the bias in other ways (For example, they could use the style-transfer baseline used by Geirhos et al., 2019). 

I vote for accepting the paper as a poster. It introduces an interesting approach for debiasing representations. However, due to its narrow scope and missing baselines, I would not recommend the paper for an oral presentation. 


### Questions

1. What are some broader class of problems for which defining G is easier than directly regularizing for debiased representations?

2. How well do Bagnets alone perform on the benchmarks in Table 3? I would expect to see that Bagnets alone do worse than vanilla Resnets on Unbiased and IN-A. Is that so? 

3. How well do other methods do in these domains? (Such as methods that directly debias the training data against texture by applying style-transfer). 


### Update after Author's response

The author's response has clarified the motivation behind the proposed approach to an extent. They have also added a comparison with a method that directly promotes learning the shape as opposed to the texture ( by training on stylized Imagenet) 

I agree with R1 on all accounts (i.e. it is very hard to define the family of biased feature extractors, the proposed approach is ad-hoc, the authors need to compare to texture-shape disentanglement methods, etc), however at the same time, I can see that proposed approach can act as a useful heuristic for regularizing neural networks to pay attention to certain kind of information. 

An interesting use-case of the proposed method (which the authors indirectly mentioned in their response to my review) is in a multi-modal setting. It's not trivial to enforce deep learning systems to utilize all data modalities in a multi-modal setting. By defining G to be models trained on individual modalities, it would be possible to nudge our models to pay attention to the information in all modalities. 

Some important results in deep learning have been ad-hoc (For example skip connections in deep networks, ReLUs) and have nonetheless progressed the field. This work is not as widely applicable as skip connections or ReLUs, but it is, nonetheless, providing a heuristic for solving an important problem. 
review_title: Official Blind Review #3

Meta-review: This paper provides and analyzes an interesting approach to "de-biasing" a predictor from its training set.  The work is valuable, however unfortunately just below the borderline for this year.  I urge the authors to continue their investigations, for instance further addressing the reviewer comments below (some of which are marked as coming after the end of the feedback period).
Decision: Reject