Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The authors presented a comprehensive study on the role of background in image classification. They designed a new set of data and a lot of experiments to find answers to the following questions: (1) How much decrease in classification accuracy if the background signal is removed? (2) Can a model successfully classify an image solely based on its background? (3) Will an image be misclassified if the image's background is replaced by a different background? (4) With the advance of the model architecture, are the more advanced models like ResNet more robust to background effect? 

The paper is well written, and the figures and tables are clearly presented. The newly created dataset ImageNet-9, that contains background- and foreground-free images, are publicly available. 

Comments:
- I am not sure if I understand the purpose for presenting the ONLY-BG-T results in Figure 7. In my opinion, by comparing the BG-Required numbers in MIXED-RAND and ORIGINAL models, it is already clear enough to demonstrate that the background is a necessary component for many images to obtain correct classification. 
- As shown in the section of related work, similar topics have been studied before. One of the main contributions of this paper is the newly created dataset. It can be generally useful for ML research in robustness and out-of-distribution detection.  

review_title: A comprehensive study on the role of background in image classification

final_score: rating: 5: Marginally below acceptance threshold
review_content: 
The submission performs similar foreground-background analysis for object recognition as in [1], but with more modern networks in mind. As such, the main takeaways indicate that this phenomenon still exists - networks today continue to suffer from background bias as they did four years ago with AlexNet, although maybe to a lesser extent. This submission curates more careful evaluation setups by using segmentation of foreground objects, tiled backgrounds to create multiple datasets that serve to illustrate the trends in a more disambiguated way. 

Is there some way to quantify the overall diversity of adversarial backgrounds? For example, is it possible that owing to strong correlations of a few objects with easily-learned backgrounds, these few backgrounds always cause misclassification for other objects? Could there be a way to detect such backgrounds?

The Appendix says "The ORIGINAL-trained model performs similarly on NO-FG and ONLY-BG-B, indicating that it does not use object shape effectively”  but there seems to be a 10% gap in Table 5, indicating that the shape mask is fairly useful. The IN-9L numbers seem 21% up instead of 13%. Am I misreading this table?

Re. "Indeed, the ONLY-BG trend observed in Figure 8 suggests that…”, could an additional possibility be that around 20% of  classes are fully correlated with their backgrounds? I.e. how can we know how much of the findings are to do with model "failure", and not dataset quirks?

In summary,

(+) While it is not particularly surprising to learn that backgrounds can be misleading even with the correct foreground or that there exists vulnerability to adversarially picked backgrounds, given the evidence we have of background biases already, it can be useful to have a quantification of the “BG-gap” for a range of modern models.

(-) The takeaways are mostly already recognised from the many works that have pointed out reliance of object recognition models on backgrounds. The experiments provide a quantified view of how much modern networks trained on the particular datasets rely on backgrounds, but it is unclear how widely applicable this information is, given that this only analyses a specific dataset. The curated datasets might be useful for benchmarking progress; however, if one were to set up the goal of providing such a testset, then perhaps it might be more appropriate to curate an entire testset of adversarial backgrounds alone (rather than mixed-rand) across a range of modern networks and for all of ImageNet, which, along with the usual test set would provide a background-robustness sanity check (with the caveat from the authors that backgrounds may actually be informative when the foreground is confusing).


[1] Object recognition with and without objects, Zhu et al. 2017

Post Rebuttal:
I appreciate the authors' responses. The "novelty" over Zhu et al. was never under question in my review, I was mostly confused about how to weigh the significance of the findings, how useful it is to know some numbers for the version of the dataset created by the authors (which is not really the original Imagenet classification task), and if the submission actually does "pinpoint" what the problems are, how and when they manifest, to what extent the dataset is responsible vs. the training choices. Having read the other reviews, responses, looked at the updates, I'm still unsure --  if there were something in this paper that was new or surprising and not more or less already known from existing works (perhaps not precise numbers, but then the paper is essentially using a synthetic, modified Imagenet anyway), I'd be more enthusiastic about pushing up the rating. But as of now, I'm retaining my initial rating.
review_title: Interesting empirical study quantifying Imagenet-trained models' reliance on background on Imagenet

final_score: rating: 6: Marginally above acceptance threshold
review_content: ##########################################################################

Summary:

The paper studies the effect of background noise on image classification tasks for neural networks. The paper suggests the following based on empirical results from ImageNet classifiers.

1. It is possible to achieve reasonable accuracy by just using the background information.
2. Image classification models suffer from a decrease in accuracy if inference images have a different background.
3. Image classification models have higher accuracy tends to depend on the background image less.

The paper also introduces a toolkit for evaluating the ImageNet classifiers' dependence on background images. 

##########################################################################

Pros:
1. The paper provides a detailed quantitative analysis of the effects of using different backgrounds (both training and testing). It constructs various synthetic test dataset that analyzes different scenarios.
2. The result sections (sections 3 and 4) are well structured and carefully study the impact of using different backgrounds. 
3. The proposed toolkit can be used to evaluate the model's reliance on the background.

##########################################################################

Cons:
1. The key concern about the paper is the lack of novelty. While the synthetic dataset was constructed to study the effects of background in detail, the findings from the paper are not new. 
2. The experiments do not study/relate how data augmentation techniques affect background reliance. The paper also mentions OOD techniques such as distributionally robust optimization (Sagawa et al, 2020), but does not study how these techniques affect background reliance. Moreover, I cannot find a discussion on which factors in training might force the model to less rely on image backgrounds (or robust to foreground images).
3. The dataset mainly used, IN-9, is also a small dataset that contains less than 50,000 train images. Moreover, the paper only considers the ImageNet type dataset. Some results may not hold for the other datasets. The authors do not address this in the manuscript.
4. There are some minor concerns about the experimental set-up used in the paper that I describe in the section below.
5. Writings can be significantly improved. 

##########################################################################

Questions:
1. On page 5, it says, "MIXED-RAND models perform poorly on datasets with only backgrounds and no foregrounds." What is the insight from this experiment? Does this imply that the model might be learning shape features as it is doing better than random?
2. For images processed with GrabCut, wouldn't the model use shape-related features along with the background? If it is learning the shape features, can the positive correlation in section 4 mean that a stronger model might be learning more shape features?
3. What is each point in figure 8 represent? Is it using different architectures?
4. Appendix B.1, doesn't the change in the number of classes also result in a change in total dataset size?
5. What is the main insight of this paper when training neural networks?
6. What are the main contributions of the toolkit? How accurate is the segmentation?

Minor notes:
* On page 1, "standard models misclassify 88% of images ..." --> does this refer to 88% of test images?
* On page 1, I don't understand this sentence: "tend to simultaneously exploit background correlations more." Is there a stronger correlation between backgrounds for more accurate models? 
* For figure 2, what is the test accuracy for a model trained on the original dataset (original/original)?
* In table 2, "on select test sets ..." --> selected.
* In figure 3, I don't understand what it means by "note that the gap decreases much more on the right side of the graph."
* It isn't easy to interpret table 3. It will be easier to understand if it contains some illustrative examples.

##########################################################################

I raised my score based on the author's response.
review_title: Official Blind Review #1

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: I think this a very good contribution to ICLR given the topic and the quality of the submission (originality, contribution to the stare of the art, experimental evidence, etc) although the study might need to be supported in a more theoretical framework to make it worth of an oral presentation (I would recommend a poster or short presentation)

 Some of the strong points of the submission are summarized as follows:

1.	Studies in the interpretability of the results of deep learning models is a very important aspect, as well as the robustness of the obtained models in a variety of circumstances and under adversarial attacks. 
2.	A sufficient introduction and motivations sections, but I would suggest introducing the state of the art at the beginning of the paper as it would help to get a better grasp of how the works builds upon previous work.
3.	The state of the art (despite the previous comment) contextualizes the subject matter in a succinct but comprehensive manner. Although there are certain aspects that could be improved, such as including a table outlining in a clearer manner the contributions of the authors in this context.
4.	The experimental design is good, showing a careful analysis to validate the proposal and several ablation studies to assess the validity of the authors' hypotheses
5.	The foundations for the method are presented in great detail in a formalized manner and provides sufficient elements (i.e. examples) to assess the validity of the proposed approach.


However, there are certain things that in my opinion could be improved:

1.	The authors make a very interesting contribution that leverages knowledge from several research areas and thus, sometimes the contributions with regards to the state of the art are difficult to follow. I would suggest making a table summarizing the main features of some previous works so the readers can better grasp the limitations of those previous works and understand better the improvements in each of the areas outlined in this research
2.	The organization of the paper is confusing, some effort should be given at creating a clearer layout that makes the paper easier to read and follow the flow of ideas.
3.	Future work could be further elaborated and discussion in specific domains (medical imaging, for instance) could be further discussed.
4.	The abstract mentions that the proposed work can be used as a blueprint for assessing the out of distribution performance of deep learning models, but this aspect is not sufficiently explored in my opinion.


review_title: A very interesting work, but  possibly not original enough for an oral presentation. The work can attract a possible larger audience and get better feedback if presented as poster

Meta-review: The paper investigates the tendency of image recognition models to depend on image backgrounds, and propose a suite of datasets to study this phenomenon.

All the reviewers agree that the paper investigates an important problem, is well-written and contains several interesting insights that should be of interest to the community. I recommend acceptance.

Decision: Accept (Poster)