Noise or Signal: The Role of Image Backgrounds in Object Recognition

1 INTRODUCTION . Object recognition models are typically trained to minimize loss on a given dataset , and evaluated by the accuracy they attain on the corresponding test set . In this paradigm , model performance can be improved by incorporating any generalizing correlation between images and their labels into decision-making . However , the actual model reliability and robustness depend on the specific set of correlations that is used , and on how those correlations are combined . Indeed , outside of the training distribution , model predictions can deviate wildly from human expectations either due to relying on correlations that humans do not perceive ( Jetley et al. , 2018 ; Ilyas et al. , 2019 ; Jacobsen et al. , 2019 ) , or due to overusing correlations , such as texture ( Geirhos et al. , 2019 ; Baker et al. , 2018 ) and color ( Yip & Sinha , 2002 ) , that humans do use ( but to a lesser degree ) . Characterizing the correlations that models depend on thus has important implications for understanding model behavior , in general . Image backgrounds are a natural source of correlation between images and their labels in object recognition . Indeed , prior work has shown that models may use backgrounds in classification ( Zhang et al. , 2007 ; Ribeiro et al. , 2016 ; Zhu et al. , 2017 ; Rosenfeld et al. , 2018 ; Zech et al. , 2018 ; Barbu et al. , 2019 ; Shetty et al. , 2019 ; Sagawa et al. , 2020 ; Geirhos et al. , 2020 ) , and suggests that even human vision makes use of image context for scene and object recognition ( Torralba , 2003 ) . In this work , we aim to obtain a deeper and more holistic understanding of how current state-of-the-art image classifiers utilize image backgrounds . To this end , in contrast to most of the prior work ( which tends to study relatively small and often newly-curated image datasets1 ) , our focus is on ImageNet ( Russakovsky et al. , 2015 ) —one of the largest and most widely used datasets , with state-of-the-art training methods , architectures , and pre-trained models tuned to work well for it . Zhu et al . ( 2017 ) analyze ImageNet classification ( focusing on the older , AlexNet model ) to find that AlexNet achieves small but non-trivial test accuracy on a dataset consisting of only backgrounds ( where foreground objects are replaced by black rectangles ) . While sufficient for establishing that backgrounds can be used for classification , we aim to go beyond those initial explorations to get a more fine-grained understanding of the relative importance of backgrounds and foregrounds , for newer , state-of-the-art models , and to provide a versatile toolkit for others to use . Specifically , we investigate the extent to which models rely on backgrounds , the implications of this reliance , and how models ’ use of backgrounds has evolved over time . Concretely : • We create a suite of datasets that help disentangle ( and control for different aspects of ) the impact of foreground and background signals on classification . The code and datasets 1We discuss these works in greater detail in Section 5 , Related Works . are publicly available for others to use in this repository : https : //github.com/ MadryLab/backgrounds_challenge . • Using the aforementioned toolkit , we characterize models ’ reliance on image backgrounds . We find that image backgrounds alone suffice for fairly successful classification and that changing background signals decreases average-case performance . In fact , we further show that by choosing backgrounds in an adversarial manner , we can make standard models misclassify 88 % of images as the background class . • We demonstrate that standard models not only use but require backgrounds for correctly classifying large portions of test sets ( 35 % on our benchmark ) . • We study the impact of backgrounds on classification for a variety of classifiers , and find that models with higher ImageNet test accuracy tend to simultaneously have higher accuracy on image backgrounds alone and have greater robustness to changes in image background . 2 METHODOLOGY . To properly gauge image backgrounds ’ role in image classification , we construct a synthetic dataset for disentangling background from foreground signal : ImageNet-9 . Base dataset : ImageNet-9 . We organize a subset of ImageNet into a new dataset with nine coarse-grained classes and call it ImageNet-9 ( IN-9 ) 2 . To create it , we group together ImageNet classes sharing an ancestor in the WordNet ( Miller , 1995 ) hierarchy . We use coarse-grained classes because there are not enough images with annotated bounding boxes ( which we need to disentangle backgrounds and foregrounds ) to use the standard labels . The resulting IN-9 dataset is class-balanced and has 45405 training images and 4050 testing images . While we can ( and do ) apply our methods on the full ImageNet dataset as well , we choose to focus on this coarse-grained version of ImageNet because of its higher-fidelity images . We describe the dataset creation process in detail and discuss the advantages of focusing on IN-9 in Appendix A . 2These classes are dog , bird , vehicle , reptile , carnivore , insect , instrument , primate , and fish . Variations of ImageNet-9 From this base set of images , which we call the ORIGINAL version of IN-9 , we create seven other synthetic variations designed to understand the impact of backgrounds . We use both rectangular bounding boxes and the foreground segmentation algorithm GrabCut ( Rother et al. , 2004 ) , as implemented in OpenCV , to disentangle backgrounds and foregrounds . We visualize these variations in Figure 1 , and provide a detailed reference in Table 1 . These subdatasets of IN-9 differ only in how they process the foregrounds and backgrounds of each constituent image . Larger dataset : IN-9L We finally create a dataset called IN-9L that consists of all the images in ImageNet corresponding to the classes in ORIGINAL ( rather than just the images that have associated bounding boxes ) . This dataset has about 180k training images in total . We leverage this larger dataset to train better generalizing models , and prefer to analyze models trained on IN-9L whenever possible . 3 QUANTIFYING RELIANCE ON BACKGROUND SIGNALS . With ImageNet-9 in hand , we now assess the role of image backgrounds in classification . Backgrounds suffice for classification . Prior work has found that models are able to make accurate predictions based on backgrounds alone ; we begin by directly quantifying this ability . Looking at the ONLY-BG-T , ONLY-BG-B , and NO-FG datasets , we find ( cf . Figure 2 ) that models trained on these background-only training sets generalize reasonably well to both their corresponding test sets and to unmodified images from the ORIGINAL test set ( around 40-50 % for every model , far above the baseline of 11 % representing random classification ) . Our results confirm that image backgrounds contain signal that models can accurately classify standard images with . Models exploit background signal for classification . We discover that models can misclassify due to background signal , especially when the background class does not match that of the foreground . As a demonstration , we study model accuracies on the MIXED-RAND dataset , where image backgrounds are randomized and thus provide no information about the correct label . By comparing test accuracies on MIXED-RAND and MIXED-SAME 3 , where images have class-consistent backgrounds , we can measure classifiers ’ dependence on the correct background . We denote the resulting accuracy gap between MIXED-SAME and MIXED-RAND as the BG-GAP ; this difference represents the drop in model accuracy due to changing the class signal from the background . In Table 2 , we observe a BG-GAP of 13-22 % and 4-11 % for models trained on IN-9L and ImageNet , respectively , suggesting that backgrounds often mislead state-of-the-art models even when the correct foreground is present . More Training Data can reduce the BG-GAP . Our results indicate that ImageNet-trained models are less dependent on backgrounds than their IN-9L-trained counterparts—they have a smaller ( but still significant ) BG-GAP , and perform worse when predicting solely based on backgrounds ( i.e. , on the ONLY-BG-T dataset ) . We explore two ways that ImageNet differs from IN-9L to understand this phenomena—ImageNet has ( a ) more datapoints than IN-9L , and ( b ) a more fine-grained class structure . Figure 3 shows that more training data reduces the BG-GAP , particularly when the training dataset size approaches the size of ImageNet . This indicates that training on much more data ( and thus , more backgrounds ) can reduce ( but not eliminate ) the effect of backgrounds on model predictions . An ablation study of ImageNet ’ s more fine-grained class structure does not find strong evidence supporting its helpfulness ( cf . Appendix B ) . Models are vulnerable to adversarial backgrounds . To understand how worst-case backgrounds impact models ’ performance , we evaluate model robustness to adversarially chosen backgrounds . We find that 88 % of foregrounds are susceptible to such backgrounds ; that is , for these foregrounds , there is a background that causes the classifier to classify the resulting foreground-background combination as the background class . For a finer grained look , we also evaluate image backgrounds based on their attack success rate ( ASR ) , i.e. , how frequently they cause models to predict the ( background ) class in the presence of a conflicting foreground class . As an example , Figure 4 shows the five backgrounds with the highest ASR for the insect class—these backgrounds ( extracted from insect images in ORIGINAL ) fool a IN-9L-trained ResNet-50 model into predicting insect on up to 52 % of 3MIXED-SAME controls for artifacts from image processing presented in MIXED-RAND . For further discussion , see Appendix D. non-insect foregrounds . We plot a histogram of ASR over all insect backgrounds in Figure 24 of the Appendix—it has a long tail . Similar results are observed for other classes as well ( cf . Appendix G ) . Training on MIXED-RAND reduces background dependence . Next , we explore how to reduce models ’ dependence on background . To this end , we train models on MIXED-RAND , a synthetic dataset where background signals are decorrelated from class labels . As we would expect , MIXEDRAND-trained models extract less signal from backgrounds : evaluation results show that MIXEDRAND models perform poorly ( 15 % accuracy—barely higher than random ) on datasets with only backgrounds and no foregrounds , ( ONLY-BG-T or ONLY-BG-B ) . Indeed , such models are also more accurate on datasets where backgrounds do not match foregrounds . In Figure 5 , we observe that a MIXED-RAND-trained model has 17.3 % higher accuracy than its ORIGINAL-trained counterpart on MIXED-RAND , and 22.3 % higher accuracy on MIXED-NEXT , a dataset where background signals class-consistently mismatch foregrounds . ( Recall that MIXEDNEXT images have foregrounds from class y mixed with backgrounds from class y + 1 , labeled as class y . ) The MIXED-RAND-trained model also has little variation ( at most 3.8 % ) in accuracy across all five test sets that contain the correct foreground . Qualitatively , the MIXED-RAND-trained model also appears to place more relative importance on foreground pixels than the ORIGINAL-trained model ; the saliency maps of the two models in Figure 6 show that the MIXED-RAND-trained model ’ s saliency maps highlight more foreground pixels than those of ORIGINAL-trained models . A fine grained look at dependence on backgrounds . We now analyze models ’ reliance on backgrounds at an image-by-image level and ask : for which images does introducing backgrounds help or hurt classifiers ’ performance ? To this end , for each image in ORIGINAL , we decompose how models use foreground and background signals by examining classifiers ’ predictions on the corresponding image in MIXED-RAND and ONLY-BG-T . Here , we use the MIXED-RAND and ONLY-BG-T predictions as a proxy for which class the foreground and background signals ( alone ) point towards , respectively . We categorize each image based on how its background and foreground signals impact classification ; we list the categories in Table 3 and show the counts for each category as a histogram per classifier in Figure 7 . Our results show that while few backgrounds induce misclassification ( see Appendix H for examples ) , a large fraction of images require backgrounds for correct classification—approximately 35 % on the ORIGINAL trained classifiers , as calculated by combining the “ BG Required ” and “ BG+FG Required ” categories . Further insights derived from IN-9 are discussed in the Appendix D. We focus on key findings in this section , but also include more comprehensive results and examples of other questions that can be explored by using the toolkit of IN-9 in the Appendix .

Summary: The submission performs similar foreground-background analysis for object recognition as in [1], but with more modern networks in mind. As such, the main takeaways indicate that this phenomenon still exists - networks today continue to suffer from background bias as they did four years ago with AlexNet, although maybe to a lesser extent. This submission curates more careful evaluation setups by using segmentation of foreground objects, tiled backgrounds to create multiple datasets that serve to illustrate the trends in a more disambiguated way.