Finding and Fixing Spurious Patterns with Explanations
Authors: Gregory Plumb
Marco Tulio Ribeiro
Ameet Talwalkar
Venue: Accepted by TMLR ()

Abstract: Image classifiers often use spurious patterns, such as “relying on the presence of a person to detect a tennis racket,” which do not generalize. In this work, we present an end-to-end pipeline for identifying and mitigating spurious patterns for such models, under the assumption that we have access to pixel-wise object-annotations. We start by identifying patterns such as “the model’s prediction for tennis racket changes 63% of the time if we hide the people.” Then, if a pattern is spurious, we mitigate it via a novel form of data augmentation. We demonstrate that our method identifies a diverse set of spurious patterns and that it mitigates them by producing a model that is both more accurate on a distribution where the spurious pattern is not helpful and more robust to distribution shift.

Published in Transactions on Machine Learning Research (08/2022)
Finding and Fixing Spurious Patterns with Explanations
Gregory Plumb CMU
gdplumb Bandrew www.ewwwwwmm...mmmmmmommmm. eduu
Marco Tulio Ribeiro
Microsoft Research
Ameet Talwalkar CMU
talvalkar@ocmu.ck edu
Reviewed OpenReview:
https: httpe://openrevieel/ openreview.
net/ forum? id= Jdgg..P
Abstract
Image classifiers often use spurions patterns, such as "relying on the presence of a person to detect a tennis racket," which do not generalize. In this work, we present an end-to- end pipeline for identifying and mitigating spurious patterns for such models, under the assumption that we have access pixel-wise object-annotations. We start by identifying patterns such "the model's prediction for tennis racket changes 63% of the time if hide the people." Then, if pattern is spurious, augmentation. We demonstrate that our method identifies a diverse set of spurious patterns and that it mitigates them by producing a model that is both more accurate on distribution where the spurious pattern not helpful and more robust to distribution shift.
mitigate it via novel form of data
1 Introduction
With the growing adoption of machine learning mod- els, there is growing concern about Spurious Pat- terns (SPs) when models rely on patterns that do not align with domain knowledge and do not gener-
Identification
Remove people:
B
Labeling
63% predictions change
Spurious
Mitigation
alize (Ross al.] 2017 Shelly et al.] 2019 Rieger le al.) 20200 Teney et al.) 2020 Singh et al.) 2020).
For example, model trained to detect tennis rackets
P(racket)-994
the COCO dataset [Lin et al. 2014 learns to rely on the presence of person, which leads to systemic errors: it is significantly less accurate at detecting tennis rackets for images without people than with people (41.2% vs 86.6%) and only ever has false pos- itives on images with people. Relying on this SP works well on COCO, where the majority of images with tennis rackets also have people, but would not be as effective for other distributions. Further, rely- ing on SPs may also lead to serious concerns when they relate to protected attributes such as race or gender [Buolamwini Gebru 2018).
Through data
augmentation
P(racket)=18
Figure For the tennis racket example, SPIRE identifies this pattern by observing that, when remove the people from images with both tennis racket and person, the model's prediction changes 63% of the time. Since we do not remove the tennis racket itself, we label this pattern spurious. Then, SPIRE carefully adds/removes /removes tennis rackets/ packets/people people from different images to create an augmented training set where tennis rackets and people independent while minimizing any new correlations between the label and artifacts the counterfactual images (e.g., grey boxes).
We focus on SPs where an image classifier relying spurious object (e.g., using people to detect

---

Published in Transactions on Machine Learning Research (08/2022)
tennis rackets) and we propose Spurious Pattern Identification and REpair (SPIRE as an end-to-end solution for these SPs. As illustrated in Figure 1 SPIRE: identifies which patterns the model is using by measuring how often it makes different predictions on the original and counterfactual versions of an image. Since it reduces a pattern to a single value that has a clear interpretation, it is casy for a user to, when needed, label that pattern as spurious or valid. Then, it miligales SPs by retraining the model using a novel form of data augmentation that aims to shift the training distribution towards the balanced distribution, a distribution where the SP is no longer helpful, while minimizing any new correlations between the label and artifacts in the counterfactual images. Each of these steps is based on the assumption that we have access to pixel-wise object-annotations to use to create counterfactual images by adding or removing objects.
To verify that the baseline model relies on SP and quantify the impact of mitigation methods, we measure gaps in accuracy between images with and without the spurious object (e.g., there is a 45.4% accuracy drop between images of tennis rackets with and without people). Intuitively, the more model relies on a SP, the larger these gaps will be and the less robust the model is to distribution shift. Consequently, an effoctive miligation method will decrease these gap metries and improve performance on the balanced distribution. Empirically, we show SPIRE's effectiveness with three sets of experiments:
Benchmark Experiments. We induce SPs with varying strengths by sub- sampling COCO in order to observe how mitigation methods work in a controlled setting. Overall, we find that SPIRE is more effective than prior methods. Interestingly, we also find that most prior methods are ineffective at mitigating negatice SPs (e.g., when the model learns that the presence of "cat" means that there is no "tie"). Full Experiment. We show that SPIRE is useful "in the wild" on the full COCO dataset. For identification, it finds a diverse set of SPs and is the first method to identify negative SPs, and, for mitígation, it is more effective than prior methods. Additionally, we show that it improves zero-shot generalization (iee, evaluation without re-training) to two challenging datasets: UnRel (Peyre et al. 2017] and SpatialSense Yang et al., 2019). These results are notable because most methods produce no improvements in terms of robustness to natural distribution shifts (Taori et al. 2020).
Generalization Esperiments. We illustrate how SPIRE generalizes beyond the setting from our prior experiments, where we considered the obsect-clasiificttion task and assumed that the dataset has pixel-wise object-annotations to use to create counterfactual images. Specifically, we explore three examples that consider different task and/or or do not make this assumption.
2 Related Work
We discuss prior work as it pertains to identifying and mitigating SPs for image classification models.
Identification. The most common approach is to use explainable machine learning (Simonyan et al. 2013
Ribeiro et al., 2016 Selvaraju et al.] 2017 Ross et al. 2017 Singh et al. 2018 Dhurandhar et al., 2018 Goyal
et al. 2019 Koh et al. 2020 Joo & Kärkkäinen 2020 Rieger et al. 2020). For image datasets, these moth rely on local explanations, resulting in a slow process that requires the user to look at the explanation for each image, infer what that explanation is telling them, and then aggregate those inferences to assoss whether
or not they represent a consistent pattern (Figure 2). In addition to this procedural difficulty, there is uncertainty about the usefulness of some of these methods for model debugging (Adebayo et al. 2020 2022
In contrast, SPIRE avoids these challenges by measuring the aggregated effect that a specific counterfactual has on the model's predictions (e.g., the model's prediction changes 63% of the time when we remove the people from images with both a tennis racket and a person). While prior work has defined similar measurements, (hhetty et al. (2019) and (Singh et al. (2020) fail to identify negative SPs and Xiao et al. 22021 fail to identify specific SPs (e.g., they find that the model is relying on "something" rather than "people").
Mitigation. While prior work has explored data augmentation for mitigation (Hendricks ct al. 2018 Shetty et al. 2019 Teney et al. 2020 Chen et al., 2020a Agarwal et al. 2020), it has done so with augmentation (e.g..|Shetty
strategies that are agnostic to the training distribution (e.g., Shhety et al. (2019) simply remove either the tennis rackets or the people, as applicable, uniformly at random for each image). In contrast, SPIRE aims to
use counterfactual images to create a training distribution where the label is independent of the spurious
1 "Code is available at htttps///iithii www.ttttuu.ccomttt wwtpw//wwwwwc///////./c
2

---

Published in Transactions on Machine Learning Research (08/2022)
J.L.MVOI
J.L. III
J.L.VVOT
PEEO)
PEEO
P(Racket) 99%
P(Racket) = 1%
P(Racket) 99%
Figure Based on the saliency map (Smonnyan et al. 2013 (Left), one might mistakenly infer that the model is not relying on the person. However, the model fails to detect the racket after the penson is removed (Center) and incorrectly detects racket after it is removed (Right).
object, while minimizing any new correlations between the label and artifacts in the counterfactual images. We hypothesize that this is why we find that SPIRE is more effective than these methods.
Another line of prior work adds regularization to the model training process (Ross et al. 2017 Hendricks
ct al.
2018 Wang et al.
2019 Rieger et al. 2020: Teney et al. 2020 Liang et al.) 2020 Singh et al. 2020
Some of these methods specify which parts of the image should not be relevant to the model's prediction
[Ross et al. 2017 Rieger et al. 2020). Other methods encourage the model's predictions to be consistent
across counterfactual versions of the image (Hendricks et al., 2018 Teney al. 2020 Liang al. 2020). All
of these methods could be used in conjunction with SPIRE.
Finally, there are two additional lines of work that make different assumptions than SPIRE. Making weaker assumptions, there are methods based on sub-sampling, re-weighting, or grouping the training set (Chawla et al. 2002 Sagawa* et al.) 2020 Creager et al., 2020). These methods have been found to be less effective than methods that use data augmentation or regularization (Rieger et al.] 2020 Neto 2020 Singh et al., 2020 Goel et al.] 2021). Making stronger assumptions, there are methods which assume access to several distinct training distributions (Wen et al. 2020: Chen et al. 2020b).
Consequently, the methods designed for image classification that use data augmentation regularization represent SPIRE's most direct competition. As result, we compare against "Right for the Right Reasons" (RRR) [Ross et al. 2017). "Quantifying and Controlling the Effects of Context." (QCEC) (Shetty et al. 2019). "Contextual Decomposition Explanation Penalization" (CDEP) (Rieger et al. 2020). "Gradient Supervision"
(GS) Teney et al. 2020). and the "Feature Splitting" (FS) method from (Singh et al. 2020
3 Spurious Pattern Identification and REpair
In this section, we explain SPIRE's approach for addressing SPs. We use the classt-clasiiiicttin task as a rumning example, where Main is the object being detected and Spurious is the other object in SP. The same methodology applies to any binary classification problem with binary "spurious feature;" see Appendix B. for discussion on how to do this.
We view a dataset as a probability distribution over set of image splils, which we call Both, Just Main, Just Spurious, and Neither, depending on which of Main and/or Spurious they contain (e.g.. Just Main is the set of images with tennis rackets, without people, and either with or without any other object). Figure 3 (Left) shows these splits for the tennis racket example. Critically, we can take an image from one split and create counterfactual version of it in a different split by either adding removing either Main Spurious (e.g., removing the people from image in Both moves it to Just Main). To do this, we assume access to pixel-wisc object-amottaions; note that this is common assumption (i.c., RRR, QCEC, CDEP, and GS also make it). See Appendix B.2 for more details on the counterfactual images.
Preliminaries.
To summarize, SPIRE makes two general assumptions:
That we can determine which of these splits an image belongs to.
That we can create a counterfactual version of an image from one split that belongs to different split.
3

---

Published in Transactions on Machine Learning Research (08/2022)
Original
Balanced
Image Splits
Count & Examples
Accuracy
Both f
Both S
3,361 (2.8%)
86.6%
Image
Just Main
Spurious Nelther Ø
f
e
(correct/incorecct
Just Main
33 (3.0%)
41.2%
P(Main-T)
2.8%
2.8%
Maintain PMMain
Just Spurious
60,754 (51%)
99.5%
P(Spurious77 Main=7)
99.0%
50
Ensure SP
Neither Ø
54,139 (47%)
100.0%
PiSpurioue-T T| Main=F)
52.9%
50.0%
is not helpful
Figure 3: Left. The training image splits and the original training distribution for the tennis racket example. Because of the strong positive correlation between Main and Spurious, it is helpful for the model rely this SP. Right. The balanced distribution for the temnis racket example. This SP is no longer helpful because Main and Spurious are now independent and there the same number of images in Both and Just Main.
Identification. SPIRE measures how much the model relies on Spurious to detect Main by measuring the probability that, when we remove Spurious from an image from Both, the model's prediction changes (e.g., the model's prediction for tennis racket changes 63% of the time when we remove the people from an image with both a tennis racket and a person). Intuitively, higher probabilities indicate stronger patterns.
To identify the full set of patterns that the model is using, SPIRE measures this probability for all (Main, Spurious) pairs, where Main and Spurious are different, and then sorts this list to find the pairs that represent the strongest patterns. Recall that not all patterns patterns as spurious or valid as needed before moving to the mitigation step.
necessarily spurious and that the user may label
Mitigation. It is often, but not always, the case that there is a strong correlation between Main and Spurious in the original training distribution, which incentivizes the model to rely on this SP. As a result, we want to define a distribution, which we call the balanced distribution, where relying on this SP is neither inherently helpful nor harmful. This is a distribution, exemplified in Figure (Right), that:
Preserves P(Main). This value strongly influences the model's relative accuracy on {Both, Just Main} versus (Just Spurious, Neither) but does not incentivize the SP. As a result, we preserve it in order to maximize the similarity between the original and balanced distributions.
Sels P(Spurious Muin) = P(Spurious nol Main) = 0.5. This makes Main and Spurious independent, which removes the statistical benefit of relying on the SP, and assigns equal importance to images with and without Spurious. However, this does not go so far as to invert the original correlation, which would directly punish reliance on the SP.
As shown in Figure (Right), SPIRE's mitigation strategy uses counterfactual images to manipulate the training distribution. The specifics are described in Section 3.1 but they implement two goals:
Primary: Shift the training distribution lowards the balanced distribution. While the original distribution often incentivizes the model to rely on the SP, the balanced distribution docs not. However, adding too many counterfactual images may compromise the model's 's accuracy on natural images. As a result, we want shift the training distribution towards, but not necessarily all the way to, the balanced distribution. Secondary: Minimize the potential for new SPs. While shifting towards the balanced distribution, we may inadvertently introduce new potential SPs between Main and artifacts in the counterfactual images. For example, augmenting the dataset with the same counterfactuals that SPIRE uses for identification (i.e., images from Both where Spurious has been covered with grey box) introduces the potential for a new SP because P(Main| "grey box") = 1.0. Because the augmentation will be less effective the model learns to rely on new SPs, we minimize their potential by trying to set P(Main Artifact) = 0.5.
3.1 Specific Mitigation Strategies
While SPIRE's augmentation strategy follows the aforementioned goals, its specific details depend on the problem setting, which we characterize using two factors:
4

---

Published in Transactions on Machine Learning Research (08/2022)
Table 1: Setting 1. For p= 0.9 and p= 0.1, we show the original size of each split for a dataset of size 200 as well as the size of each split after SPIRE's QCEC's augmentation. Note that SPIRE produces the balanced distribution, while QCEC does not even make Main and Spurious independent.
0.9
0.1
Split
Original
SPIRE
QCEC
Original
SPIRE
QCEC
Both
90
90
90
10
90
10
Just Main
10
90
55
900
90
95
Just Sparions
90
95
Neither
90
110
10
90
190
Can the counterfacluals change on image's 's label? For tasks such as object-classifiationn, counterfactuals can change an image's label by removing adding Main. However, for tasks such scene identification, we may not have counterfactuals that can change an image's label. For example, we cannot turn a runway into a street or a street into a runway by manipulating a few objects. This fundamentally shapes how counterfactuals can change the training distribution.
Is the datasel class bolanced? While working with class balanced datasets drastically simplifies the problem and analysis, it is not an assumption that usually holds in practice.
These two factors define the three problem settings that consider, which correspond to the experiments in Sections 5.1 5.2 and 5.3 respectively. For each setting, summarize what makes it interesting, define SPIRE's specific augmentation strategy for it, and then discuss how that strategy meets SPIRE's goals.
Setting 1: Counterfactuals can change an imagc's label and the dataset is class-balanced. Here, P(Main)= P(Spurious) = 0.5 and we can define the training distribution by specifying p = P(Main Spurious). p > 0.5, SPIRE moves images from (Both, Neither] to (Just Main, Just Spurious) with probability 2--1 for each of those four combinations. If p < 0.5, SPIRE moves images from {Just Main, Just Spurions} to (Both, Neither} with probability p-1
2p
p-0.5
Table D shows how SPIRE changes the training distributions for p = 0.9 and p 0.1. For p = 0.9, it succeeds at both of its goals. For p 0.1, it produces the balanced distribution, but does add the potential for new SPs because P[Main| | Removed an object) 0 and P(Main Added an object) to the most closcly related method, QCEC (Shetty et al.) 2019), which removes either Main or Spurions uniformly at random, as applicable, from each image. For both values of p, QCEC does not make Main and Spurious independent and adds the potential for new SPs. This example highlights the fact that, while prior work has used counterfactuals for data augmentation, SPIRE uses them a fundamentally different way by considering the training distribution.
1. We contrast SPIRE
Setting 2: Counterfactuals can change an image's label, but the dataset has class imbalance. Class imbalance makes two parts of the definition of the balanced distribution problematic for augmentation:
Preserves P(Main). When P(Main) is small, this means that we generate many more counterfactual images without Main than with it, which can introduce new potential SPs.
Sets P(Spurious not Main) = 0.5. When P(Spurious) is also small, this constraint requires that most of the counterfactual images we generate belong to Just Spurious, which can lead to the counterfactual data outnumbering the original data by factor of 100 more for this split.
Consequently, we relax these constraints. PSPpuuuss Main) > P(Spurious), SPIRE creates an equal number of images to add to Just Main/Spurious by removing the appropriate object from an image from Both. Specif- Juust Main
ically, this number is the smallest positive solution for å to: [Bthh++JJsst Spurious|+5
Both
Just Main + [Netherr
Otherwise, SPIRE creates an equal number of images to add to Both/Just Spurious by adding Spurious
to Just Main/Neither Neither. Specifically, this number solves: [Both+ Sparious|+28 [Just Main|+ (Ncther]] SPIRE achieves its primary goal by making P(Main Spurious) = P(Main not Spurious) (i.c., Main and Spurious
[Both|+5
(Just Main]
are now independent) and it achieves its secondary goal by adding an equal number of counterfactual images with and without Main (i.e., P(Main Artifact) 0.5).
5

---

Published in Transactions on Machine Learning Research (08/2022)
Sctting 3: Counterfactuals cannot change an image's label. This constraint prevents the previous strategies from being applied. Therefore, SPIRE removes Spurious from every image with Spurious and adds Spurious to every image without Spurious, While this process does achieve SPIRE's primary goal, it does not achiove SPIRE's secondary goal (i.c., the original correlation between the label and Spurious is the same as the correlation between the label and grey boxes from removing Spurious).
4 Evaluation
Because relying on the SP is usually helpful on the original distribution, we cannot measure the effectiveness of a mitigation method using that distribution. Instead, we measure the model's performance on the balanced distribution, using metrics such as accuracy and average precision. Intuitively, using the balanced distribution provides a fairer comparison because the SP is neither helpful nor harmful on it. However, like any performance metric that is aggregated over a distribution, these metrics hide potentially useful details.
We address this limitation by measuring the model's accuracy on each of the image splits. These per split accuracies yield a more detailed analysis, allow us to estimate the model's performance on any distribution (e.g., the balancod distribution) by re-weighting the model's accuracy on them, and allow us to calculate two "gap metrics". The Recall Gap is the difference in accuracy between Both and Just Main; the Hallucination Gap is the difference in accuracy between Neither and Just Spurious. Intuitively, a smaller recall gap means that the model is more robust to distribution shifts that move weight between Both and Just Main. The same is true for the hallucination gap and shifts between Neither and Just Spurious. As concrete example of these metrics, consider the tennis racket example (Figure 3 the model is much more likely to detect a tennis racket when a person is present) and a hallucination gap of 0.5% (i.c., the model is more likely to hallucinate a tennis racket when a person is present; see Appendix C.2)
Left), where we observe that the recall gap is 45.4% (i.e.,
Note that these per split accuracies are measured using only natural (i.e., not counterfactual) images, order to prevent the model from "cheating" by learning to use artifacts in the counterfactual images. As a result, the gap metries and the performance the balanced distribution also only use natural images.
Class Balanced vs Imbalanced Evaluation. When there is class balance, we use the standard prediction threshold of 0.5 to measure a model's performance using accuracy on the balanced distribution (i.e., balanced accuracy) and its gap metries. When there is class imbalance, Average Precision (AP), which is the area under the precision vs recall curve, is the standard performance metric. Analogous to AP, we can calculate the Average Rocall Gap by finding the area under the "absolute value of the recall gap" vs recall curve; the Average Hallucination Gap is defined similarly. As result, we measure a model's performance using AP Gaps. Appendix D
the balanced distribution (i.e., balanced AP) and the Average Recall/Hallucinattee provides more details for these metrics by showing how they are caleulated using the tennis racket example.
5 Experiments
We divide our experiments into three groups:
In Section 5.1 we induce SPs with varying strengths by sub-sampling COCO in order to understand how mitigation methods work in a controlled setting. We show that SPIRE is more effective at mitigating these SPs than prior methods. We also use these results to identify the best prior method, which we use for comparison for the remaining experiments.
In Scction 5.2, we find and fix SPs "in the wild" using all of COCO; this means finding multiple naturally occurring SPs and fixing them simultaneously. We show that SPIRE identifies wider range of SPs than prior methods and that it is more effective at mitigating them. Additionally, we show that it improves zero-shot generalization to two challenging datasets (UnRel and SpatialSense).
In Section 5.3 we demonstrate that SPIRE is effective for problems other than COCO. To do so, we consider lasks hth than object-classffeation ion and/or explore techniques for constructing counterfactuals for datasets without pixel-wise object- annieattnnntttions..
For the bascline models (i.e., the normally trained models that contain the SPs that we are going to identify and mitigate), we fine-tune a pre-trained version of ResNet18 (He et al. 2016) (see Appendix E.. We compare
6

---

Published in Transactions on Machine Learning Research (08/2022)
0.5
0.775
0.5
Balanced Accuracy
Hallucination Gap
0.4
0.750
0.4
Recall Gap
Baseline
0.3
0.725
0.3
SPIRE
FS
0.700
0.2
0.2
RRR
0.675
QCEC
0.1
CDEP
0.650
0.0
GS
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
p P(Main Spurious)
p = P(Main Spurious)
p P(Main Spurious)
Figure 4: A comparison of the baseline model to various mitigation methods. The results shown are averaged across both the pairs accepted for our benchmark and across eight trials. Left Balanced Accuracy. For p S 0.2 and 2 0.8, SPIRE produces the most accurate models. None of the methods have much of an impact for 0.4 or Recall/Han, Hallnciiaiiion Gaps. SPIRE generally shrinks the absolute value of both of the gap metrics by more than prior methods.
p= 0.6, likely because those create weak SPs. Center/Right
SPIRE to RRR [ooss et al. 2017). 7 QCEC (Shetty al. 2019) CDEP (Rieger el al. 2020), GS (Teney et al.) 2020), and FS (Singh et al 2020). We use the evaluation described in Section 4 and, for any split that is too small to produce a reliable accuracy estimate, we acquire additional images (using Google Images) such that each split has at least 30 images to use for evaluation.
Measuring variance across trials. Because we are applying these mitigation methods to the baseline model, the results of some metric (e.g., Balanced AP) for the mitigated model and the baseline model are not independent. As result, we do not directly report the variance of this metric across trials and, instead, report the variance its difference between the mitigated and baseline models. Subsequently, the "[o=]" entries in our tables will denote the standard deviation of particular metric's ' difference across trials.
5.1 Benchmark Experiments
We construct set of benchmark lasks from COCO consisting of different SPs with varying strengths, by manipulating the model's training distribution, in order to better understand how mitigation methods work in a controlled setting. Appondix has additional details.
Creating the benchmark. We start by finding each pair of objects that has at least 100 images in each split of the testing set (13 pairs). For each of those pairs, we create series of controlled training sets of size 2000 by sampling images from the full training set such that P(Main) = P(Spurious) = 0.5 and p P P(Main Spurious) ranges between 0.025 and 0.975. Each controlled training set represents binary task, where the goal is to predict the presence of Main.
While varying allows us to control the strength of the correlation between Main and Spurious (i.e., P near 0 indicates a strong negative correlation while p near 1 indicates a strong positive correlation), it does not guarantee that the model actually relies on the intended SP. Indeed, when we measure the models' balanced accuracy as p varies, we observe that out of the 13 pairs show little to no loss in balanced accuracy as p approaches 1. Consequently, subsequent evaluation considers the other 8 pairs. For these pairs, the model's reliance the SP increases p approaches 0 or 1 evidenced by the increasing loss of balanced accuracy,
Results. Figure 4 (Left) presents the balanced accuracy results. We find that SPIRE consistently improves balanced accuracy and that it does so by more than prior methods. Interestingly, while most prior methods are beneficial for strong positive SPs (p 2 0.9), only FS also (mildly) beneficial for negative SPs (p < 0.5).
Figure (Center/ Rigght) presents the gap metric results. We find that SPIRE the most effective method shrinking these metrics, which indicates that it produces model that is more robust to distribution shift.
7

---

Published in Transactions on Machine Learning Research (08/2022)
Table 3: Mitigation results for the Full Experiment. Bal- anced AP is averaged across the SPs identified by SPIRE. Similarly, the gap metrics are reported as the "mean (meddaan) change from the baseline model, aggregated across those SPs.
A few examples of the SPs identified by SPIRE for the Full Experiment. For each pair, we report several basic dataset statistics including bios, P(Sparimus Main) P(Sparions) which captures how far P(Spurions)
Table 2:
Main and Spurious are away from being independent
Original MAP
Balanoed AP
%4 Aug. Recall Gup
RA Avg. Hallmisssion Gap
as well as the sign of their correlation.
Main tie toothbrush
Spurious
P(M)
P(S)
P(S
M)
bias
Baseline SPIRE
64.1
46.2
63.7 [0-0.1
47.3 [r-0.5]
-14.2 (-14.5)
-28.1 (-27.3)
cat
0.03
0.04
0.01
-0.66
FS
62.5 (0m100)
44.7 (0=1.))
9.7 (-5.9)
25.7 (-6.9)
person
0.01
0.54
0.54
-0.01
bird
sheep
0.03
0.01
0.01
0.00
frisbee
person
0.02
0.54
0.83
0.54
Table 4: The MAP results a zero-shot evaluation on
tie
person
0.03
0.54
0.76
the classes that are in the InRel/SpatialSense datasets
tennis racket
person
0.03
0.54
0.99
0.83
that SPIRE also identified as being Main in a SP.
dog
sheep
0.04
0.01
0.03
1.05
frisbee
dog
0.02
0.04
0.24
5.44
UnRel
SpatinlSease
fork dining table
0.03
0.10
0.76
6.56
Baseline
SPIRE
41.3 1.3)
20.7 [.50
FS
39.6 (21)
18.6 (==33
Interestingly, QCEC and GS, which are the two prior methods that include data augmentation, are the only prior methods that substantially shrink the gap metrics (at the cost of balanced accuracy for p << 0.9).
Overall, this experiment shows that SPIRE is an effective mitigation method and that evaluation framework enables us lo easily undersland how methods affect the behavior of model. We use FS as the baseline for comparison for the remaining experiments because, of the prior methods, it had the best average balanced accuracy across p's range.
5.2 Full Experiment
We evaluate SPIRE "in the wild" by identifying and mitigating SPs learned by multi-label binary object- classification model trained on the full COCO dataset. Appendices D and have additional details.
Identification. Out of all possible (Main, Spurious) pairs, we consider those which have at least 25 training images in Both (<x 2700). From these, SPIRE identifies 29 where the model's prediction changes at least 40% of the time when we remove Spurious. Table 2 shows few of the identified SPs; overall, they are quite diverse: the spurious object ranges from common (c.g., person) to rarc (c.g., shoep); the SPs range from objects that are commonly co-located (e.g., tie-person) to usually separate (e.g., dog-sheep); and a few Main objects (e.g., tie and frisbee) have more than one associated SP. Notably, SPIRE identifies negative SPs (e.g., tie-cat) while prior work (Shetty et al. 2019 Singh et al., 2020 Teney et al. 2020) only found positive SPs. Appendix includes discussion of how verified the veracity of these identified SPs.
Mitigation. Unlike the Benchmark Experiments, this experiment requires mitigating many SPs simultane- ously. Because we found that SPIRE was more effective if we only re-train the final layer of the model in the Benchmark Experiments (see Appendix E 1, we do this by re-training the slice of the model's final layer that corresponds Main's class on an augmented dataset that combines SPIRE's augmentation for each SP associated with Main. All results shown (Tables and 4 are averaged across eight trials.
We conclude that SPIRE significantly reduces the model's reliance on these SPs based on two main observations. First, it increases balanced AP by 1.1% and shrinks the average recall/hallueee haw.liitaalliiniionnn gaps by factor of 14.2/28.11. relative to the bascline model, on COCO. As expected, this does slightly decrcase Mcan Average Precision (MAP) by 0.4% on the original (biased) distribution. Second, it increases MAP on the UnRel Peyre et al. 2017) and SpatialSense (Yang et al. 2019) datasets. Because this evaluation was done in a zero-shot manner and these datascts are designed have objects in unusual contexts, this further evidence that SPIRE improves distributional robustness. In contrast, FS decreases the model's performance, has inconsistent effects on the gap metrics, and has mixed results on the zero-shot evaluation.
8

---

Published in Transactions on Machine Learning Research (08/2022)
SPIRE and Distributional Robustness. Noting that robustness to specific distribution shifts is one of the consequences of mitigating SPs, we can contextualize the impact of SPIRE by considering an extensive meta-analysis of methods that aim to provide general robustness (Taori et al. 2020). This analysis finds that the only methods that consistently work are those that re-train the bascline model on several orders of magnitude more data. It also describes two necessary conditions for a method to work. Notably, SPIRE satisfics both of those conditions: first, it improves performance on the shifted distributions (i.c., the balancod distributions, UnRel, and SpatialSense) and, second, this improvement cannot be explained by increased performance on the original distribution. Consequently, SPIRE 's results are significant because they show improved robustness without using orders of magnitude more training data. We hypothesize that SPIRE is successful because it targets specific SPs rather than using a less targeted approach.
5.3 Generalization Experiments
We illustrate how SPIRE generalizes beyond the setting from COCO, where we considered the object- classification task and assumed that the dataset has pixcl-wisc object-annotations
Table 5: Results for the Scene Identification Experiment (averaged across sixteen trials).
Original AP
Balanced AP
%A Avg. Recall Gap
%A Avg.
Specifically, we ex- plore three examples that consider different task (Generalization 1) do not assume this (General- isation 2).
Hallucination Gap
Baseline SPIRE FS
95.0
48.9
92.8 [o=1.6]
83.2
(==7
-82.1 -11.5
-75.5 3.7
93.7 1=
47.8
8.68
Scene Identification Experiment (Generaliza- tion 1). In this experiment, we construct a scene identification task using the image captions from COCO and show that SPIRE can identify and mit- igate a naturally occurring SP. To do this, we de- fine two classes: one where the word "runway" (the part of an airport where airplanes land) appears in the caption (1,134 training images) and another where "street" appears (12,543 training images); im- ages with both or without either are discarded. For identification we observe that removing all of the airplanes from image of runway changes the model's prediction 50.7% of the time.
Table 6: Results for the No Object Annotation Experiment for the tennis racket example (averaged across eight trials).
Original AP
Balanced AP
%A Avg. Rocall Cup
X& Avg.
Hullucisation Gap
Baseline SPIRE SPIRE-R FS
93.9
79.9
92.9 [r=00.]
80.5
[-=118]
-31.3
-27.3
94.0 0o=
81.0
1==17
-9.1 -44.9
92.9 1=0
80.7
-10.4
-22.3
Table 7: Results for the ISIC Experiment (averaged across eight trials).
Original AP
Balanced AP
M& Aug. Recal Gap
Aug Hallacinattion Gup
Baseline SPIRF-RM
78.3
71.0
78.8
(n-4.9
76.4
(n-2.4)
-20.5 -31.3
-30.0
PS
70.7
6.8
68.0
(.88
-61.0
Table reliance on this SP because it substantially increases balanced AP and it reduces the average recall and hallucination gaps by factors of 82.1% and 75.5%. In contrast, FS is not effective at mitigating this SP.
shows shows the results. SPIRE reduces the model's
No Object Annotation Experiment (Generalization 2). In this experiment, we mitigate the SP from the tennis racket example without assuming that we have pixel-wise object-annotations; instead, we make the weaker assumption that we have binary labels for the presence of Spurious. To do this, we train linear (in the model's representation space) classifier to predict whether or not an image contains a person (similar to Kim et al. (2018)). Then, we project across this linear classifier to essentially add or remove a person from an image's representation (so we call this method SPIRE-R) (see Appendix H.I
Table G shows the results. First, note that SPIRE provides a small increase in Balanced AP while providing the largest. average docrease in the gap metrics. Sccond, note that SPIRE-R preferable to FS because it produces a larger reduction in the hallucination gap while being otherwise comparable.
ISIC Experiment (Generalizations 1 & 2). In this experiment, we imitate the setup from (Regger et al, 2020) for the ISIC datasct (Codella et al.) 2019). Specifically: the task is to predict whether an image of a skin lesion is malignant or benign; the model learns to use a SP where it relies on a "brightly colored sticker that is spuriously correlated with the label; and the dataset does not have annotations for those stickers to use create counterfactual images. For this experiment, we illustrate another approach for working with datasets that do not have pixel-wise object-annotations: using enternal models (so we call this method SPIRF-FM) to produce potentially noisy annotations with less effort than actually annotating the entire
9

---

Published in Transactions on Machine Learning Research (08/2022)
dataset. The external model could be off-the-shelf model (e.g., model that locates text in image) or simple pipeline such as the super-pixel clustering one we use (see Appendix H..).
Table 7 shows the results. We can see that SPIRE-EM is effective at mitigating this SP because it generally improves performance while also shrinking the gap metrics P In contrast, FS does not seem to be beneficial because substantially reduces performance on both the original and balanced distributions (which outweighs shrinking the gap metrics).
6 Conclusion
In this work, we introduced SPIRE as an end-to-end solution for addressing Spurious Patterns for image classifiers that are relying on spurious objects to make predictions. SPIRE identifies potential SPs by measuring how often the model's prediction changes when we remove the Spurious object from an image with positive label and mitigates SPs by shifting the training distribution towards the balanced distribution while minimizing any correlations between the label and artifacts in the counterfactual images. We demonstrated that SPIRE is able identify and, at least partially, mitigate a diverse set of SPs by improving the model's performance on the balancod distribution and by making it more robust to specific distribution shifts. We found that these improvements lead to improved zero-shot generalization to challenging datasets. Then, we showed that SPIRE can be applied to tasks other than object-classification and illustrated two potential ways to apply SPIRE to datasets without pixel-wise object-annotations to use to create counterfactuals (creating counterfactual representations and using external models). More generally, in terms of identifying SPs, our findings provide additional evidence for the claim that a correlation is neither sufficient nor necessary for a model to learn a SP.
References
Julius Adebayo, Michael Muelly, Ilaria Liccardi, and Been Kim. Debugging tests for model explanations. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Newral Information Processing Systems, volume 33, pp. 700-712. Curran Associates, Inc., 2020. URL hntps://proceedngs
Julius Adebayo, Michael Muelly, Harold Abelson, and Been Kim. Post hoc explanations may be inefTcctive for detecting unknown spurious correlation. In International Conference on Learning Representations, 2022.
URL https://opearrevier..n http:///ww....eeeoe/oooooooddtoo
Vedika Agarwal, Rakshith Shetty, and Mario Fritz. Towards causal vqa: Revealing and reducing spurious correlations by invariant and covariant semantic editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 9690 9690-9698, 2020.
Joy Buolamwini and Timnit Gebru, Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pp. 77-91, 2018.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-333 357, 2002.
Long Chen, Xin Yan, Jun Xiao, Hanwang Zhang, Shiliang Pu, and Yucting Zhuang. Counterfactual samples synthesizing for robust visual question answering. Tn Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10800 10809, 2020a.
Yining Chen, Colin Wei, Ananya Kumar, and Tengyu Ma. loff-training avoids using spurious features under domain shift. In H. Larochelle, M. Ranzato, R. Hadsoll, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 21061-21077 Curran Associates, Inc., 2020b. URL https : .dd
NNote that, because the datasct does not contain enough malignant images with stickers to produce reliable accuracy estimate, we had to use counterfactual images for this evaluation. This may influence the results for balanced AP and the recall gap metric. However, SPIRE-EM still an improvement over the baseline because of the original AP and hallucination gap metric results.
10

---

Published in Transactions on Machine Learning Research (08/2022)
Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint NEf::1902.03666 2019.
Elliot Creager, Jörn-Heurik Jacobsen, and Richard Zemel. Exchanging lessons between algorithmic fairness and domain generalization. arXiv preprint arXtt:2010.07249 2020.
Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. Advances in neural information processing systems, 31:592-603, 2018.
Karan Goel, Albert Gu, Yixuan Li, and Christopher Re. Model patching: Closing the subgroup performance gap with data augmentation. In International Conference on Learning Representations, 2021. URL https://openreview
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lec. Counterfactual visual explanations. In International Conference on Machine Learning, pp. 2376 2384, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEFF Conference on Computer Vision and Pattern Recognation (CVPR), pp- 770-778, 2016. doi: 10.1109/CVPP 2V1...00.....0
Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Overcoming bias in captioning models. In Proceedings of the European Conference Computer Vision (ECCV), pp. 787, 2018.
Jungseock Joo and Kimmo Kärkkäinen. Gender slopes: Counterfactual fairness for computer vision models by attribute manipulation. In Proceedings of the 2nd International Workshop on Fairness, Accountability, Transporency and Ethics in Mullimedia, pp. 5, 2020.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, al. Inter- pretability beyond feature attribution: Quantitative testing with concept activation vectors (teav). Tn International conference on machine learning, pp. 2668 2677. PMLR, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint NNiu:1412.6988 2014.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pp. 5338 5348. PMLR, 2020.
Zujie Liang, Weitao Jiang, Haifeng Hu, and Jiaying Zhu. Learning to contrast the counterfactual samples for robust visual question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (FMNLP), pp.
2020.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Europeon conference on computer vision, pp. Springer, 2014.
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of out-of-distribution generalization. In International Conference on Learning Representations, 2021.
Kamyar Nazeri, Eric Ng, Tony Josoph, Faisal Qureshi, and Mchran Ebrahimi. Edgeconnect: Generative image inpainting with adversarial edge learning. ar preprint arXiv:1901.00212, 2019.
Elias Chaibub Neto. Counterfactual confounding adjustment for feature representations learned by deep models: with an application to image classification tasks. arXiv preprint (tl::2004.094660 2020.
11

---

Published in Transactions on Machine Learning Research (08/2022)
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca. Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performanee deep learning library. In H. Wallach, H. Larochelle, A. Beygekzimer F. d' Alché-BBuc, E. Fox, and R. Garnett (eds.), Advances in Neural Informa- tion Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019. URI http::/ppperrrss.tt////// ptw.ppparrr...rupppsspppp neurips ott.::/r........iiii.aaareeeeeaa
lecepp-eeaannnngg wiwpnng--bbbrrpype ppdf
Julia Peyre, Ivan Laptev, Cordelia Schmid, and Josef Sivic. seakly-supervised learning of visual relations. In ICCV, 2017.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. ' 'why should i trust you?" explaining the prodictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135 1144, 2016.
Laura Rieger, Chandan Singh, William Murdoch, and Bin Yu. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. In International Conference on Machine Learning, pp. 1116-8126 PMLR, 2020.
Andrew Slavin Ross, Michael C Hughes, and Finale Doshi-Velez. Right for the right reasons: training differentiable models by constraining their explanations. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 2662-2670, 2017.
Shiori Sagawa* Pang Wei Koh*, Tatsunori B. Hashimoto, and Perey Liang. Distributionally robust neural networks. In International Conference on Learning Represenlations, 2020. URL httpp::/openrevier:
Axel Sauer and Andreas Geiger. Counterfactual generative networks. In International Conference on Learning Representations, 2021. URL
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 618 626, 2017.
Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in noural networks. In NewrIPS, 2020.
Rakshith Shetty, Bernt Schiele, and Mario Fritz. Not using the car to see the andewalk-quantifyy and controlling the effects of context in classification and segmentation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pottern Recognition, pp. 8218 8826 2019.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6033: 2013.
Chandan Singh, W James Murdoch, and Bin Yu. Hierarchical interpretations for neural network predictions. In International Conference on Learning Representations, 2018.
Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, and Deepti Ghadiyaram. Don't judge object by its context: Learning to overcome contextual bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11070-11078, 2020.
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt.
Measuring robustness to natural distribution shifts in image classification. arXiv preprint ar arXiv:2007.0064, 2020.
Damien Teney, Ehsan Abbasnedjad, and Anton van den Hengel. Learning what makes difference from counterfactual examples and gradient supervision. arXiv preprint ar arX:::204400033, 2020.
Angelina Wang, Arvind Narayanan, and Olga Russakovsky. Revise: A tool for measuring and mitigating bias in visual datasets. In European Conference on Computer Vision, pp. 733 751. Springer, 2020.
12

---

Published in Transactions on Machine Learning Research (08/2022)
Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5310 5319, 2019.
Jun Wen, Changjian Shui, Kun Kuang, Junsong Yuan, Zenan Huang, Zhefeng Gong, and Nenggan Zheng. Interventional domain adaptation. arXic preprint arXa::001.009797, 2020.
Kai Yuanqing Xiao, Logan Engstrom, Andrew llyas, and Aleksander Madry. Noise or signal: The role of image backgrounds in object recognition. In International Conference on Learning Representations, 2021.
URL http:////ppppareeeeeeeeee/o///aa
Kaiyu Yang, Olga Russakovsky, and Jia Deng. Spatialsense: An adversarially crowdsourced benchmark for spatial relation recognition. In Proceedings of the LEEE/CVF International Conference on Computer Vision, pp. 2051-2060 2019.
Yi Zhang and Jitao Sang. Towards for visual debiasing. In Proceedings of the 28th ACM International Conference on Multimedia, pp. 4346 4354, 2020.
paradox: Adversarial example-based data augmentation
13

---

Published in Transactions on Machine Learning Research (08/2022)
A
Discussion
In this section, we claborate on SPIRE's strongths, its weaknesses, and suggested directions for future work. While there are many ways to improve SPIRE, we have demonstrated that it is clear step forwards for the problem of addressing SPs.
Generating Counterfactual Images. SPIRE relics on the ability to produce counterfactual images. As a result, finding ways to produce similar counterfactuals with fewer assumptions (e.g., being able to add/remove objects without relying on having an annotated dataset) to produce different types of counterfactuals (e.g,, changing attributes such as "color") are both directions for future work. The former would improve the general applicability of SPIRE while the later would increase the scope of the types of SPs SPIRE could address.
Identification. SPIRE's strategy for identification can be summarized as "measure the probability that the model's prediction changes when we lake an image from Group X and apply Counterfactual Transformation Y. Intuitively, this strategy is effective because the original and counterfactual versions of an image differ only in terms of the effect of the counterfactual transformation while, if we were to compare natural images in one group to another group, there are probably going to be additional differences. Because SPIRE uses X = Both, it may not be as effective as possible for identifying negative SPs because this split is likely to be very small for negatively correlated objects. As a result, future work could increase the scope of the types of SPs SPIRE could identify by considering different definitions of X (e.g., X is the set of images that have objects I,..., m and do not have objects m + 1,...,,;; nn X is the set of images where objects and appear near to/far from each other) or Y (e.g., Y removes objects and 2; Y changes the location of object 1).
Interestingly, we find that a strong correlation is neither sufficient (Figure 7 shows that the model ignore a strong correlation) nor necessary (Table 2 shows that some SPs are between objects that are almost uncorrelated) for a model to learn to use a SP, which is consistent with prior findings (Shah et al. 2020) Nagarajan et al. 2021). These result demonstrates SPIRE's advantage over identification methods that only consider the training distribution (e.g.. Wang et al. 2020).
Mitigation. To begin with, it worth noting that mitigating a SP may not always be worthwhile (c.g., when one is certain that the distribution will not shift).
At a high level, SPIRE's strategy for mitigation works by removing the statistical incentive for the model to rely on the SP, while trying not to add new SPs; this strategy may be less effective for SPs that do not arisc from correlations in the training distribution. Previous augmentation-basee mitigation methods might be less effective because they are intuitive rather than statistical (e.g., it makes intuitive sense that removing people should lessen the model's reliance on people to detect tennis rackets, but this intuition does not carry over to the dataset statistics). Previous regularization- based mitigation methods might be less effective because they may interfere with the learning process (e-g., cause the model to become stuck in a local minimum) may have effects that are too local to matter (e.g., changing the model's gradient at a point may not change its predictions very far away from that point). In particular, the Feature Splitting (FS) method from (Singh et al. 2020) assumes that one half of the features learned by the model are relevant for detecting objects "in context " and that the other half are relevant for objects "out of context:" while plausible for a single SP, this assumption becomes more tenuous as the number of SPs being mitigated increases.
they
While SPIRE's mitigation strategy is defined by two high-level goals, it is not always successful at realizing those goals (c.g., for p < 0.5 in Scction 5.1 SPIRE introduces the potential for new SPs) and the way those goals are realized depends on the problem setting, As result, future work could improve the general applicability of SPIRE by finding a unified strategy that does not depend on the problem setting, generalizing that strategy to work for more general SPs, and extending that strategy to problems other than image classification. Additionally, future work could develop a theoretical framowork to help understand the effects of augmentation- based mitigation strategies.
14

---

Published in Transactions on Machine Learning Research (08/2022)
B Method Details
B.1 When can SPIRE be applied to problem?
In this section, we walk through the assumptions needed to apply SPIRE to a model that has been trained to perform some task using some data.
SPIRE is model-agnostic and, a result, can be applied any type of model.
SPIRE assumes that task is binary-llasiffcation and, consequently, that each image has binary label. SPIRE assumes that each image has a different binary label for the "spurious feature" that we are interested in studying. For example, this could be variable indicating whether or not the image contains some high
frequency signal.
SPIRE assumes we can manipulate the images in such a way that we can create a counterfactual version of an image in one split that belongs to a different split. This entails being able to change an image's label and/or the value of the "spurious feature."
Collectively, these assumptions allow us to define the image splits (Table 8) and produce the counterfactual images that SPIRE uses to identify and mitigate spurious patterns.
B.2 Generating Counterfactual Images
Similar to prior work, SPIRE generates counterfactual images by adding objects to removing objects from the original image (hhetty et al.] 2019: Teney et al. 2020 Xiao et al. 2021 Chen et al. 2020a Liang et al., 2020: Agarwal et al. 2020). In this work, use the pixel-wise object-annotations that are part of various datasets such as COCO to generate the counterfactual images. Figure shows examples. Orthogonally, there
is prior work that generates fundamentally different types of counterfactual images (Neto 2020 Zhang
Sang
2020
Goel et al.] 2021 Sauer Geiger 2021
Removing Object. We consider two different ways to define which region of the image replace (pixel-wise nounding-bax) and two different ways in-fill that region (using constant grey color in-painting with the model from (Nazeri et al. 2019)). When say that we "remove" an object, we mean that we found its bounding-box region and in-filled it with grey. When we say that we "in-paint" an object, we mean that we found its pixel-wise region and in-painted it. In order to minimize label noise, we make sure we do not include Main the region that is going to be removed when we are removing Spurious.
going to
Adding an Object. To add an object to an image, find the pixel-wise region for that object in a different image and then replace that region's counterpart in the original image with it. In order to minimize label noise, make sure that do not cover Main when add Spurious.
Table 8: A more general definition of the image splits defined in Section 3
Binary Image Label
Binary "Spurious Feature"
Image Split
1
Both
1
Just Main
0
1
Just Spurious
0
0
Neither
15

---

Published in Transactions on Machine Learning Research (08/2022)
J.P.Morgan
POLO
Figure 5: Example counterfactual images for the tennis racket example. (Left) An example of moving image from Just Spurious to Neither by Removing Spurious. (Center) An example of moving an image from Both to Just Spurious by In-Painting Main. (Right) An example of moving an image from Neither to Just. Main by Adding Main.
B.3 What does it mean to introduce new potential SPs?
We try to minimize the potential for new SPs by ensuring that P(Main | Artifact) = 0.5, where the Artifact could be "Grey Box" from removing objects from an image or objects with "Unusual Placement" from adding objects an image. However, it not clear whether 0.5 or (Main) the "correct" choice for this value. One one hand, using (Main Artifact) 0.5 maximizes the loss that the model will receive if it relies Artifact. On the other hand, setting P(Main Artifact) P(Main) means that Main is independent of Artifact and that there is no statistical incentive for the model to rely Artifact. Because will not. be evaluating the model (in terms of accuracy) on images with Artifact, we chose 0.5 because it actively discourages using Artifact rather than simply not encouraging it.
B.4 Setting 1: Working through SPIRE's augmentation strategy
In this setting, (Both, Neither) each have size 0.5p while {Just Main, Just Spurious} have size 0.5(1 p).
For > 0.5, SPIRE removes (Main, Spurious} from Both with probability 3p-1 and, as a result, P(Main
2p
Grey Box) = 0.5. Similarly, SPIRE adds {Main, Spurious} to Neither with the same probability and, as result, P(Main Unusual Placement) = 0.5. As a result, {Just Main, Just Spurious) each receive 0.25(2p 1) images from each {Both, Neither) and have an augmented size of 0.5p. So SPIRE produces the balanced distribution without creating the potential for new SPs.
For p < 0.5, SPIRE adds Main to Just Spurious and adds Spurious to Just Main with probability 2-0.5 p-1 and, as a result, P(Main Umusual Placement) = Similarly, SPIRE removes Spurious from Just Spurious and removes Main from Just Main with the same probability and, as a result, P(Main | Grey Box) = As a result, (Both, Neither) each receive 0.5(0.5- -0pp p) images from each of (Just Main, Just Spurious} and have an augmented size of 0.5(1 p). So SPIRE produces the balanced distribution while creating the potential for new SPs.
16

---

Published in Transactions on Machine Learning Research (08/2022)
C Evaluation Details
C.1 Why not set P(Spurious Distribution?
Main) = P(Spurious not Main) = P(Spurious) for the Balanced
Using P(Spurious) instead of 0.5 may be an intuitive choice because it would mean that the main statistical difference between the original and balanced distributions is that Main and Spurious are now independent. However, doing so can have dramatic and unexpected effects on which splits are more important for cvaluation. To see this, consider Main "fork" and Spurious = "dining table". For the original distribution, we have P(Spurious Main) 0.76 which means have, roughly, 3:1 ratio of images in Both to Just Main. For the balanced distribution, using À - P(Spurious) = 0.1 would change this ratio to 1:9. Not only would this choice change which split is more important for evaluation (from Both to Just Main) but it would also would increase the degree to which that split is more important (from a factor of 3 to a factor of 9). Without. domain knowledge telling us that such a dramatic shift is warranted, using 0.5 is the more conservative option because assigning equal importance to images with and without Spurious never flips which splits are more important for evaluation.
C.2 Why do small, in absolute terms, Hallucination gaps matter?
To understand this, consider the per split accuracies for the tennis racket example (Figure 3 Left) where we obscrve that the Hallucination gap "only" 0.5% and may be tempted to conclude that is not significant. However, when we look at where the model's errors come from on the original distribution, we find that roughly 40% of them come from Just Spurious, despite the model's 99.5% accuracy on this split. This means that the model's performance is sensitive to both small changes to its accuracy on Just Spurious and Neither and distribution shifts that move weight between Just Spurious and Neither.
As a result, small, in absolute terms, changes to the Hallucination gap can have large impacts on the model's robustness to distribution shift. In general, we adjust for this by measuring changes in the gap metrics relative to their original value (e.g., if the new model had a hallucination gap of 0.25% we would say that it "reduced the hallucination gap by a factor of 50%").
C.3 Why can the Gap Metrics change much more than performance on the Balanced Distribution?
In general, mitigation methods shrink the gap metries by sacrificing accuracy on the splits where relying on the SP helpful in order to gain accuracy on the other splits; whether or not this trade-off improves performance the balanced distribution depends result, the size of the gap metries and performance on the balanced distribution are not necessarily closely connected. As an extreme example of this, consider a hypothetical mitigation method that works by reducing the model's accuracy on the higher performing splits to match its accuracy on the lower performing splits: this will improve the gap metrics by setting them to zero, but it will harm performance on the balanced distribution.
how much accuracy is sacrificed and gained. As
C.4 Counterfactual Evaluation
While the evaluations described in Section f are all based on the natural images, we also run a counterfactual cvaluation. Unlike SPIRE's identification step, where we only measure the probability that "removing Spurious from an image from Both" changes the model's prediction, this evaluation measures the probability that the model's prediction changes when move an image from one split to another for each pairs of splits that differs by one object. This acts as additional sanity check that a mitigation strategy has reduced the model's reliance on a SP, but we consider it to be less important than the model's performance on the balanced distribution and the gap metrics because its results depend on the specific definition of the counterfactuals used (c.g., it is casy to do well on this ovaluation for a specific type of counterfactual by training the model on that same type of counterfactual).
17

---

Published in Transactions on Machine Learning Research (08/2022)
D Tennis Racket Example: Metrics and Mitigation Results.
Here, we walk through the evaluation described in Section for class imbalanced problems using the tennis racket example. Figure G shows the results. The numbers in the legends are "mean (standard deviation)" across 8 trials for the metric measured in that plot.
Top Left: Average Precision. This panel shows the model's Precision vs Recall curve, for the balancod distribution, which we use calculate Average Precision by finding its Area Under the Curve (AUC). SPIRE improves Average Precision on the balanced distribution for this SP by 0.6%.
Top Middle: Average Recoll Gap. This panel shows the model's recall gap (the absolute value of the difference of the model's accuracy on Both and Just Main) vs its Recall on the balanced distribution. We calculate this metric by finding the AUC. SPIRE decreases this metric by 31.4% which means that it produces a model that is more robust to distribution shifts that move probability between Both and Just Main.
Top Right: Average Hallucination Gap. This panel shows the model's hallucination gap (the absolute value of the difference of the model's accuracy Just Spurious and Neither) vs its Recall the balanced distribution. We calculate this metric by finding the AUC. SPIRE decreases this metric by 25.0% which means that it produces a model that is more robust to distribution shifts that move probability between Just Spurious and Neither.
Center Row: Accuracy on Both and Just Main. These panels plot the model's accuracy on Both/Just Main vs Recall on the balanced distribution. The value shown the AUC of this curve. Because the baseline model uses the presence of a person help detect tennis racket, we expect a model that does not rely on this SP to lose accuracy Both and gain it Just Main. SPIRE does this.
Boltom Row: Accuracy on Just Spurious and Neither. These panels plot the log of the model's accuracy on Just Sourious/Neither vs its Recall on the balanced distribution. The value shown is the AUC of this curve (before taking the log). Because the baseline model uses the presence of a person to help detect tennis rackets, we expect a model that does not rely on this SP to lose accuracy on Neither and gain it on Just Spurious. Because SPIRE improved AP, we do not see this because it's accuracy on these splits higher for most levels of recall.
18

---

Published in Transactions on Machine Learning Research (08/2022)
Hallucination Gap
Baseline (6 (000)
0.04
SPIRE 0.006 (0.001)
0.8
Recall Gap
Precision
0.3
0.6
0.2
0.02
0.1
0.01
Baseline 0.799 (0.016)
Baseline 0.264 (0.042)
(0.015)
SPIRE (0. (0.034)
0.0
0.0
0.00
0.00
0.25
0.50
0.75
1000
0.00
0.25
0.50
0.75
100
0.00
0.25
0.50 Recall
0.75
1.00
Recall
Recall
1.0
1.0
Baseline 0.368 (0 (01021)
SPIRE 0 409 (0. (0.017)
Just Main
Both
0.6
0.6
0.
Baseline
0.2
SPIRE 0.591 (0.017)
0.0
0.0
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
Recall
Recall
log( Just Spurious
-0.2
log( Neither )
-0.2
-0.4
-0.4
-0.6
0.8
-1.0
-1.
Baseline 0.889 (0.014)
Baseline 0.897 (0 (0.014)
SPIRE 0 0.902 (0.017)
-12-
SPIRE 0 908 (0.017)
0.25
0.75
1.00
0.75
Recall
Recall
The results of our evaluation for the tennis racket example. The numbers in the legends are "mean (standard deviation)" across 8 trials. SPIRE improved Average Precision on the balanced distribution by 0.6%, decreased the average recall gap by 31.4%, and decreased the average hallucination gap by 25.0%. Further, it had the expected effect of decreasing accuracy on Both and increasing it on Just Main. As a result, conclude that it reduced the model's reliance this SP.
Figure 6:
19

---

Published in Transactions on Machine Learning Research (08/2022)
E
Model Training Details
Many of our experiments are based on the COCO dataset (Lin et al. 2014). Because the test set for this dataset is not publicly available, we used its validation set as our test set and divided its training set into 90-10 training and validation splits.
All of our exporiments started with the pretrained ResNet18 (He et al., 2016) that available from PyTrrch Paszke et al. 2019). For each task, the classification layer was replaced with one of the appropriate dimension and then trained via transfer learning (i.e., only the classification layer had its weights updated). The resulting model was then fine-tuned (i.e,, all of its weights were updated) to produce what we call the Baseline Model throughout this work.
Optimization. We minimized the binary cross entropy loss using Adam Kingma & Ba 2014) with a batch size of 64. For transfer- learning, we used a learning rate of 0.001 and, for fine-tuning, we used a learning rate of 0.0001; we explored other options during early experiments, but found there was no benefit. to doing so. If the training loss failed to decrease sufficiently after some number of epochs, we lowered the learning rate.
Model selection. During the training process, we selected the best model weights using their performance the validation set. For the Benchmark Experiments, we measured performance using Accuracy and, for the Full Experiment, No Object Annotation Experiment, Scene Identification Experiment, and ISIC Experiment, we used F1. If the validation performance failed to increase sufficiently after some number of epochs, we stopped training.
Benchmark Experiments: Hyper-parameter selection. For this experiment, we tuned the hyper- parameters using balancod accuracy the bottle-person pair with p 0.95. For all methods, we considered both transfer-learingg and fine- tne-tuning, as applicable. For SPIRE, we considered both removing objects by covering them with a grey box and by in-painting them; we found that transfer-learning while covering objects with a grey box was the most effective (see Table 9h. RRR, CDEP, and GS all have regularization weights that can be tuned. FS has a minimum weight for images of objects "out of context" that can be tuned. For these methods, we considered values that are powers of 10 ranging from 0.1 to 10,000; no method chose one of the extreme values.
Full Experiment: Hyper-parameter selection. For this experiment, we tuned the dyper-parameee using the mean, across SPs, Average Precision on the balanced distribution for a model trained on 50% of the training dataset and then evaluated on the remaining 50% of the training dataset; we used such large chunk of the datasel for evaluation in order to be able to estimate the per split accuracies, which are required to calculate Average Precision on the balanced distribution.
For SPIRE, we use transfer-learning while covering objects with a grey box because this is what we found worked best in the Benchmark Experiments. However, we tune the weight of the augmentation by scaling 5 from Setting 2 in Section 3.1 by factor of (0.1,0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0); intuitively, this is to prevent from adding too many counterfactual images. Note that the weight for each SP is tuned independently and each weight is tuned by training a linear classifier.
For FS, the configuration chosen by this procodure yielded poor results and, consequently, we used the default value of for our results (Singh et al. 2020).
Selection for SPIRE on the Benchmark Experiments. We see that SPIRE is consistently more effective when it retrains the model using transfer-learning and when it removes objects by covering them with grey boxes. Results shown are averaged across eight trials.
Table 9: The results of the
Removal Strategy
Parameters Adjusted
Mean (Standard Deviation)
Grey Box
Transfer-Leaningg
70.4 (I.7)
Tine-Tuning
69.4 (1.3)
In-Painting
Transfer- Transfer-Learning
70.2 (1.3)
Fine- The-Tuning
68.2 (1.4)
20

---

Published in Transactions on Machine Learning Research (08/2022)
F Additional Results: Benchmark Experiments Section 5.1
Creating the benchmark. While varying allows us to control the strength of the correlation between Main and Spurious (i.e., near O indicates a strong negative correlation while near 1 indicates a strong positive correlation), it does not guarantee that the model actually relies on the intended SP. Indeed, when we plot the models' balanced accuracy as p varies (Figure 7 we observe that out of the 13 pairs show little to no loss in balanced accuracy as p approaches 1 (dashed lines). Consequently, subsequent evaluation considers the other 8 pains (solid lines). For these pairs, the model's reliance on the SP increases as p approaches 0 or evidenced by the increasing loss of balanced accuracy and confirmed via counterfactuals (Figure g
Counterfactual Evaluation. For models that are trained on dataset augmented with specific type of counterfactual images, the results of this evaluation for that type of counterfactual are often skewed and, consequently, we exclude those results. Specifically, this means that: SPIRE is only evaluated on In-Painting counterfactuals, QCEC is not evaluated on In-Painting counterfactuals, and GS is not evaluated on counterfactuals that In-Paint Main.
Figure g shows the results (averaged across the chosen object pairs and eight trials per pair). The first thing to note is that all of the counterfactual evaluations show that the Baseline model is relying the intended SP because their results get worse as P(Main Spurious) approaches 0 or 1 (i.e., there is a strong negative or positive correlation between Main and Spurious in the training dataset). Observe that SPIRE improves all of evaluations based on In- In-Painting with the exception of "Just Spurious and In-Paint Spurious" for 0.05 < < 0.5. In contrast, the other mitigation methods have clear and consistent failures (e-g-: RRR, CDEP, GS, and FS all make the evaluation worse for "Both and Remove Main", QCEC makes the evaluation worse for "Neither and Add Main").
Per split analysis. By looking at the models' accuracy on each split (Figure 10 averaged across the chosen object pairs and eight trials per pair), we see that SPIRE exhibits all of the expected signs of a method that is reducing a model's reliance on a SP:
It sacrifices accuracy on splits where relying on the SP is helpful (e-g., Both for p> 0.5 and Just. Main for p < 0.5) in order to gain accuracy on the splits where the SP not helpful (c.g., Just Main for p > 0.5 and Both for p < 0.5).
It substantially flattens the per split accuracy curves for images with Spurious and, to lesser extent, flattens them for images without Spurious. This indicates that it produces model that is less sensitive to the original training distribution.
Comparing Augmentation Strategies. In this experiment, we want to explore the specific effect of SPIRE's augmentation strategy. We do this by comparing SPIRE to modified versions of the two augmentation- methods that we consider:
QCEC-Aug, which augments the dataset with images where either Main Spurious has been removed (uniformly at random, applicable). Compared to QCEC, it differs in that it removes objects by covering them with a grey box (as SPIRE does) instead of in- in-painting them.
GS-Aug, which augments the dataset with images where Main has been removed (if possible). Compared to GS, it differs in that it it removes objects by covering them with a grey box (as SPIRE does) instead of in-painting them and in that does not include GS's regularization term.
In Figure shows the results:
SPIRE is the most effective method for increasing Balanced Accuracy and decreasing the Recall Gap.
While QCEC-Aug and GS-Aug are somewhat better at reducing the Hallucination Gap, this usually is not
worth the loss in Balanced Accuracy.
Note that the results for the baseline model and for SPIRE are slightly different than they are in Figure because these results are based on re-running this experiment from scratch.
21

---

Published in Transactions on Machine Learning Research (08/2022)
Normalized cemalizzd Balanced Accuracy
bood: penerl
Ibaok persent
(cr, persos
(bowl, person]
Ichar. persen
Normaliized Both and Remove Spurious
(car, person)
-0.05
(diing
(cup, person)
(dining table, person)
(bottik,
0.2
(bowl, cup)
arang letwe
(bass dintng mole)
(chair. cup)
Ichar: aring tabi))
0.1
-0.15
tap. drring lebw
(bowl. dining table!
0.0
0.2
0.4
0.6
0.8
1.0
(chair. diring table)
p
P(Main Spurious)
0.0
(cup, dining
0.0
0.2
0.4
0.6
0.8
1.0
Figure For each pair of objects, we plot the models' balanced accuracy as we vary p for the training set. The y-axis is normalized that we can easily compare the curvature of the plots. We either accept (solid line) reject (dashed line) pairs based on whether or not we see a significant drop in balanced accuracy both as approaches and as it approaches The rejected pairs show insufficient drop as p approached 1.
p P(Main Spurious)
Figure 8: For each pair of objects, we plot the metric SPIRE uses to identify SPs as we vary for the training set. The y-axis is normalized so that we can easily compare the curvature the plots. We see that SPIRE generally agroes with the docision accept (solid line) or (dashed line) pairs from the benchmark because the accepted pairs generally show more curvature.
22

---

Published in Transactions on Machine Learning Research (08/2022)
0.4
Both and Remove Spurious
Both and In-Paint Spuriaus
0.30
Just Main and Add Spurious
0.25
0.2
0.3
0.20
0.2
0.15
0.1
0.10
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
P(Main Spurious)
P(Main Spurious)
p P(Main Spurrouu)
Jlust Spurious and Remove Spurious
0.4
Juutt Spurious and In-Paint Spurious
Neither and Add Spurious
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0.1
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
p
P(Main Spurious)
p
P(Main Spurious)
p
P(Main (Spurious)
0.20
0.15
0.125
Just Spurious and Add Main
Both and Remove Main
Both and In-Paint Main
0.15
0.100
0.10
0.10
0.075
0.05
0.050
0.05
0.0
1.0
0.0
1.0
0.5
= P(Main Spurious)
p = P(Main Spurious)
p P(Main Spurroous
0.25
Baseline
Just Main and Remove Main
0.20
Just Main and In-Paint Main
0.150
Neither and Add Main
SPIRE
0.125
FS
0.15
0.06
RRR
0.100
QCEC
0.10
CDEP
0.075
0.04
GS
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
P(Main Spurious)
P(Main Spurious)
p P(Main Spurious)
Figure The columns correspond to Removing, In-Painting, and Adding an object. The first two rows do that to Spurious and, as a result, a lower value is better. The last two rows do that to Main and, as a result, higher value is better. Methods that train augmented dataset that contains certain type of counterfactual are excluded from ils evaluation because their results are usually skewed.
23

---

Published in Transactions on Machine Learning Research (08/2022)
1.0
1.0
Accuracy: just Main
Accuracy: Both
0.8
0.8
0.6
0.4
0.6
0.2
0.4
0.0
0.5
1.0
0.0
0.5
1.0
P(Main Spurious)
P(Main Spurrouu)
1.0
Accuracy: Just Spurious
Accuracy: Neither
0.8
0.8
Baseline
SPIRE
FS
0.6
0.6
RRR
QCEC
0.4
0.4
CDEP
G5
0.0
0.5
1.0
0.0
0.5
1.0
P(Main Spurious)
P(Main | Spurious)
Figure 10: The models' accuracies each split.
0.5
0.5
Balanced Accuracy
0.4
0.75
Hallucination Gap
0.4
Recall Gap
0.3
0.2
0.2
Baseline
SPIRE
0.65
0.1
0.1
QCEC-Aug
GS-Aug
0.0
0.0
0.5
1.0
0.5
1.0
0.0
0.5
p P(Main Spurious)
P(Main Spurious)
p P(Main Spurrouus
Figure 11: A summary of the comparison of various augmentation strategies for the Benchmark Experiments.
24

---

Published in Transactions on Machine Learning Research (08/2022)
G Additional Results: Full Experiment Section 5.2
Validating the Identified SPs. In Figure 12 we verify that the model is generally robust to masking objects. In Figure 13 we verify that the model has large recall and hallucination gaps for the identified SPs. Both of these results indicate, in different ways, that the model is indeed relying on the SPs that SPIRE identifies.
Number (Main. Spurious) pairs
350
300
0.175
Hallucination Gap
0.150
Recall Gap
0.125
0.100
150
0.2
0.075
0.050
100
0.025
-0.2
0.000
0.0
0.8 prediction changes when we remove Spurious from an image in Both
0.2
0.4
0.6
bias
bias
Probability
SP, expect positive gap metries, if it has positive bias, and negative gap metrics, if it has negative bias. In general, this what we find. (Left) A comparison the Recall Gap the bias of the dataset for the SP. (Right) A comparison of the Hallucination Gap to the bias.
Figure 13:
When model is relying
Figure 12: A histogram of the metric that SPIRE uses to identify SPs. In general, the model's predictions are quite robust to masking objects: changing a mean of 10.0% and a median of 7.7% of the time. As a result, it is unlikely we can explain the fact that the model's prediction changes more than 40% of the time for the identified SPs using the fact that these images are out-of-distributte (they contain grey boxes).
SPIRE's effect on each SP. Figures 14 15 and 16 show SPIRE's effect on the Balanced Average Precision, the Average Recall Gap, and Average Hallucination Gap respectively. SPIRE improved Balanced Average Precision by an average of 1.1% with positive change for 21 of the SPs. SPIRE decreased the Average Decall/Hallucinationn Gaps by average factor of 14.2%/28. 1% for 24/29 of the SPs. Overall, these results indicate that SPIRE consistently reduces the model's reliance the identified SPs.
% a Hallucination Gap
Average Precision
A Recall Gap
0.25
0.0
00se Pooon Roo Roppo
0.05
00o0
0.00
0.2
0.00
-0.4
-0.50
-0.05
-066
10
10
20
10
20
Figure 14: The Average Precision the balanced distribution for SPIRE compared to the baseline for each SP.
Figure 15: The percent change of the Average Recall Gap for SPIRE compared to the baseline model for each SP.
Figure 16: The percent change of the Average Hallucination Gap for SPIRE compared to the base- line model for each SP.
25

---

Published in Transactions on Machine Learning Research (08/2022)
H Additional Results:
Generalization
Experiments
Section 5.3
H.1 SPIRE-R Projection PseudoCode
Algorithm details the process that we use for "adding" or "removing" Spurious from a model's representation. This process is guaranleed to change the initial linear model's prediction for whether or not the represent ation contains Spurious, which suites our goal of changing the most "obvious" signal for Spurious in the representation. Future work could explore more ambitious goals (c.g., removing all of the information about Spurious from the model's representation).
Algorithm Our algorithm for "adding" or "removing" Spurious from model's representation.
Require: {ri]==1
The model's penultimate layer's representation of each image A binary label for whether not each image contains Spurious D A confidence threshold, we used 0.0001 > A step size, we used 0.1 Train a linear model to predict an image contains Spurious
Require: {y6)==1
Require: c
Require: s
w. - LogisticRegression((t]] (v:))
for i = ...,, n do
r ri
b Initialize the counterfactual represcntation If the image contains Spurious, remove it
if y1 then
3/ +-0 0
while
(wr do
of
5 7
> Maximally decrease the linear model's confidence that Spurious is present
end while
else
b If the image doesn't contain Spurious, add it
y 1-1 while
< do
of 1te (wr) 187
+ + w
Maximally increase the linear model's confidence that Spurious is present
end while
end if
end for return (x/)
H.2 ISIC Experiment- Pipeline for creating counterfactuals
Our pipeline, which is based on clustering image segments (i.e., super-pixels), is constructed as follows:
We use an image segmentation algorithm to extract segments from an image and represent each segment using its mean RGB value.
We run hierarchical clustering those RGB values to produce nine clusters. Then, we manually inspect several randomly sampled images from each cluster and label those clusters based whether or not they represent stickers.
Finally, we use a K-NearestNeighhor classifier to predict which of those nine clusters an image segment belongs to.
Overall, this pipeline produces a per-image map of which pixels belong a sticker and identifies stickers with 86.7% recall and 99.0% precision. We use this map produce counterfactual images.
Note that this pipeline significantly reduced the cost of producing these pixcl-wise annotations because it only required labeling small number of image segments as "sticker or not." However, the annolations are noisier than manually collected annotations would be.
26


Reviews:
[{"review_id": "Bgu7BXZKhf", "reviewer": "Reviewer_tKuP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, I thought this was a solid contribution to the study of spurious correlations in object detection. The reliance on explicit object annotations for the majority of the paper is a limitation, as is the inability of the method to distinguish what is truly a \"spurious\" correlation, but these both seem like reasonable limitations for the problem being studied, with the latter being more of a philosophical limitation than a technical one.\n\n### Strengths:\n\n- The categorization of images into Main, Spurious, Both, and Neither subgroups is very simple but still powerful in that it lets the paper formally define many of the concepts it seeks to detect and mitigate.\n- The secondary objective of the mitigation step (de-correlating the augmentation itself from the label) is clever and missing from many of the spurious correlation mitigation papers I have seen.\n- More generally, I appreciate the shift in perspective from identifying spurious correlations on a given image to identifying them across the entire dataset.\n\n### Weaknesses: \n\n- Requiring object-level annotations is a rather major limitation---while it does not limit the significance of the work in my opinion, it should be more prominently featured (reading the introduction one would guess that the paper automatically discovers the tennis racket-human association, when in practice one can \"detect\" this correlation by just counting co-occurence between people and tennis rackets using the bounding box annotations). In particular, the \"identification\" part of SPIRE seems to be more like \"verification\" of biases that are rather easy to detect by just counting co-occurence frequencies. \n- Presenting the performance in terms of gaps can be misleading, since one way to make the gap zero is to degrade performance on both subgroups. At least in the Appendix there should be a plot showing the accuracy on each subgroup as P(Main|Spurious) is varied. In my opinion, this plot should replace Figure 4 (b) and (c) in the paper as these make it difficult to tell what the tradeoffs SPIRE is making are.\n- The graphs are all missing error bars---given the (seemingly) highly random nature of SPIRE (since it is primarily implemented using data augmentation), it seems worth having error bars around, e.g., Figure 4(a).\n- The paper was a bit confusing to follow---the writing was clear but the organization could be significantly improved. My suggestion would be to work through the tennis racket example fully in the introduction or shortly afterwards: show example images from each subgroup, show the counterfactual images and results from the identification step of SPIRE, then show what data augmentation would look like in this setting, then show subgroup accuracies before and after applying both SPIRE and one baseline method.\n\n### Questions and Suggestions:\n\n- Does the \"only spurious\" subgroup contain images from classes other than the class of interest? E.g., in the tennis racket example, would a person cooking be an example of an image in the \"only spurious\" subgroup? \n- On page 6, the paper says \"As a result, the gap metrics and the performance on the balanced distribution also only use natural images.\" This makes sense for the gap metrics, but my understanding was that the balanced distribution is constructed via counterfactual augmentation. If so, how does the evaluation account for the fact that SPIRE has seen counterfactually augmented images while the baseline methods have not? If not, might there be biases introduced by, e.g., differences between the distribution of the different subgroups?\n- Are there cases where counting co-occurence with a label does *not* find a spurious correlation, but SPIRE does? (e.g., where the presence of spurious object A is not correlated with label X in terms of count) but a counterfactual analysis reveals a dependence? Similarly, are there instances where there *is* a co-occurence bias but a counterfactual analysis suggests that the model does not actually use this correlation? If so, this seems worth mentioning in the main text of the paper.\n- The paper briefly mentions that the baseline methods make the same assumptions, but this is a bit vague. Does this just mean they also assume access to annotated bounding boxes?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a framework for identifying and repairing spurious correlations given object-level bounding box annotations (towards the end of the paper it also explores identifying the correlations without explicit annotations). The main idea is---given a \"main\" object that is causally related to the label and a \"spurious\" object that is non-causally related to the label---to view the dataset as a distribution over images containing the main object, the spurious object, both objects, or neither. This allows the authors to formally define a spurious correlation, and use image-level counterfactuals to find them. Finally, the paper repairs these correlations by counterfactually augmenting the data, while also being careful to remove any potential correlation between the augmentation artifacts and the label.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- Show example counterfactual images generated by SPIRE (beyond just Figure 1)\n- Clarification on the composition of the subgroups\n- Reporting subgroup accuracy instead of gaps [Critical]\n- Error bars for Figure 4 and other results [Critical]\n- Precise definition of what the \"balanced dataset\" used for evaluation is [Critical]\n- Better structuring and/or worked example to improve understanding\n- More precise comparison to prior works---many of the works cited as comparable are not actually about identifying and mitigating spurious annotated objects (for example, Xiao et al studies only the effect of backgrounds on classification). Perhaps a table that describes what each baseline method studies, what assumptions it makes, and what outcomes it measures would be useful.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AmXUb6zB1w", "reviewer": "Reviewer_aW1B", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n-  **Important Problem**: The task of identifying which spurious signal a model depends on, and then removing this reliance is an important one. This paper takes a stab at this problem for a concrete case/setting. \n\n- **Simple and Straightforward Solution**: The augmentation strategy proposed is simple and feasible under controlled settings where the inputs can be partitioned into 'main', 'spurious' etc as required by this method. The idea of balancing the spurious and main signals in the new dataset is also particularly straightforward, and is shown to be effective for removing dependence on spurious signals.\n\n- **Comprehensive Evaluation & Comparison with other methods**: Spurious signal detection is still a nascent problem. Several methods that claim to address this problem don't explicitly state their underlying assumptions, so it can be difficult to assess their effectiveness. This paper compares SPIRE to several proposed methods, so it is one of the first papers I have seen to perform a comprehensive evaluation, which is helpful for the field.\n\n## Weaknesses\nWhile this paper tackles an important problem, there are several weaknesses that'll highlight here. Some of these weakness are structural and probably unlikely to be easily fixed without overhauling the method, so I don't penalize the paper too much for those. In the requested changes section, I'll note the specific ones that should be addressed. \n\n- **Clarity/Difficulty in Writing**: It is a bit difficult to digest portions of this paper, particularly section 3.1, which discuss the approach to creating the balanced datasets. In addition, SPIRE as a method/approach is not really properly spelled out. Here is what I mean: let's say I have new model trained on a different dataset than COCO or the ones discussed here, it is unclear what the specific assumptions that the training data, model, and task should meet for SPIRE to be useful. \n\n- **Spurious Pattern Definition**: The definition of spurious signals in the opening paragraph is too vague. One could argue that any signal in the training set will satisfy this definition. I think the definition that the paper actually goes with is something like: a predictive signal that is correlated to the label for a particular distribution, but whose correlation 'goes away' once this distribution changes. \n\n- **Data set partition question. Where do the 'main', 'just spurious', 'neither' etc partitions come from?**: It is unclear to me how one determines the sets main, spurious etc that is required for the SPIRE method to work. In the COCO dataset, I guess this is just assumed to be the possible object segmentations? In my opinion, defining these partitions or something akin to the partition is crucial for this approach to work. The No Object Annotation Experiment is interesting as a counter to my claim, however, I don't understand the specifics of this experiment enough. What does the statement: \"we project across this linear classifier to essentially add or remove a person from an image's representation\" mean? If I know the concept of a person already,  then you already have an implicit partitioning, so the process of removing the person from the representations is really the same as creating the just main distribution as you do in the COCO dataset. To me, this experiment doesn't quite demonstrate that the explicit pixel splits are not needed. I am fine with stating that SPIRE requires explicit pixel annotations to work, which can be a stated limitation of the method. In addition, why is tennis racket main and the person spurious? If the task were person detection, wouldn't these two categories be flipped? Further, does this mean that the data splits are dependent on the target signal/label being considered? \n\n- **An example to help clear up my confusion**: Let's say we trained a COCO object detection model that happens to rely on some high frequency signals in the image for a particular object. How would the proposed partitions capture this signal? Note that this signal is not one that you have explicit pixel annotation for. \n\n- **Spurious Signals from Memorized Examples**:  The proposed approached will catch spurious signals that are due to statistical skews (in the language of Nagarajan et al. (cited in the paper). However, recent work has suggested that it is possible for a model to learn a spurious signal from just *3* examples (see: https://arxiv.org/abs/2202.05189). This is likely out of scope for this work, but are the partitions required by this paper effective when a spurious dependence is induced by only a few examples. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents SPIRE an approach for finding and fixing a model's dependence on spurious training artifacts. The approach is based on the assumption that several versions of training/validation dataset can be created such that objects in a scene/image can be partitioned into a 'main' object, 'spurious' object, 'both', and neither. If a model is relying on a spurious artifact, then its prediction changes when the spurious signal is not present in the image (neither, and main distributions).  Accuracy/performance gaps across the different data splits is used to detect whether a model is relying on a spurious signal. \n\nIn the second part of the paper, the work proposes to fix a model's dependence on spurious signals using counterfactual image generation. Specifically, the hypothesis here is that a model's dependence on spurious signals comes statistical skews in the data, i.e., all images with tennis rackets also include a person, so a person is often a spurious signal for tennis racket. To counter this, the proposed approach is via data augmentation, where a 'balanced' dataset is created such that the distribution of spurious and main signals are equalized; hence, the statistical skew is removed. Consequently, a model fined-tuned on this dataset has no use for the spurious signal. The paper considers two settings depending on whether there is class imbalance and it is possible to generate image counterfactuals where an object can be removed. \n\nSPIRE is compared to several approaches that aim to reduce a model's dependence on spurious artifacts with favorable performance.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "I list the requested changes below. All of these changes are based on the weaknesses as discussed in the previous section. Overall, I don't see any of the following requested changes as disqualifying, but would help improve readability and impact of the work. \n\n- **Writing/Clarity**: As noted in the previous section, Section 3.1 could use another pass to help improve clarity. I'd also implore the authors to explicitly clarify and state SPIRE assumptions and requirements upfront.\n\n- **Dataset partition**: In the previous section, I list a bunch of questions relating to the dataset partition requirement. Some of these are partly addressed in the paper, but I believe that the requirement for a partition is the key to getting SPIRE to work. It'll be really helpful if these issues were further clarified in the paper. \n\n- **The No Object Annotation Experiment**: This section needs further clarification. It is unclear what, \"we project the linear classifier to essentially add or remove a person from the image\" means. How was this done? How was this verified, i.e., is it still possible to predict presence of a person from the representations? \n\nThe questions above are the key ones that need to be further clarified in the draft.", "broader_impact_concerns": "I don't see any ethical concern with this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DVzhTIbTBn", "reviewer": "Reviewer_3jVt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\nSPIRE is a simple method and the paper makes simple claims and verifies them using experiments. The paper is of interest to a subset of the TMLR readership.\n\nIn particular, SPs are biases in the model which can be harmful depending on the application domain. They also limit out-of-distribution performance for instance when the spurious class is not present in the new domain or if two objects coexist in the new distribution but did not co-exist in the training set. SPIRE demonstrates that it mitigates SPs better than prior work.\n\nFigure 4 is especially informative where the training dataset bias is varied and mitigation strategies are compared. SPIRE does not loose out on model performance while reducing bias.\n\n### Weaknesses\n\nOutside of issues in the presentation, which are discussed in the subsequent section, the main weakness is that most experiments use object level annotations which are often not available. SPIRE-R uses image level annotations but it is evaluated only on one SP. SPIRE-EM requires \"manually inspect several randomly sampled images from each cluster\" which is basically doing few-shot active learning to circumvent the lack of labels. So this is not \"working without annotations\".\n\nDuring identification of SPs, how do you control for classification changes because of the input image being OOD due to the gray box? If the gray box did not cover either spurious or main, would the model output remain unchanged?\n\nAnother important weakness is that the experiments in Table 3 only fine-tune the last layer. What happens if we fine-tune end-to-end? Does it get worse because low level features adjust to gray boxes? Are we missing out by assuming that SP related issues could be fixed by changing the last layer only?\n\nLastly one ablation is missing: What if we gray out a random box in the image instead of the box around the correct object?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Image classification models often learn spurious correlations because of biases in the training dataset. For example, a model tends to correlate tennis rackets with people and therefore classifies tennis rackets only when people are present and not otherwise. Such behavior is referred to as spurious pattern (SP).\n\nThis paper presents a method, SPIRE, for identifying and mitigating SPs in image classification models. SPIRE, in its default state, requires bounding box annotations for all objects of the concerned classes in the training set. The paper proposes a couple of methods for working around this requirement, towards the end.\n\nIdentification of SPs: For each pair of class labels, SPIRE computes how often the removal of label-2 changes the prediction of label-1. This percentage can be thresholded to decide whether the pair of class labels presents an SP or not.\n\nMitigation: SPIRE manipulates the data by replacing objects with gray boxes or adding them into new images to create a balanced training set. SPIRE also takes care to not introduce an SP w.r.t. grey boxes. Either the model, or just the last layer, are then retrained on this balanced training set to mitigate the SP.\n\nEvaluation: The authors report classification AP on the original dataset, AP on the balanced dataset (but in most cases not evaluating on manipulated images), and recall / hallucination gaps. Higher the AP on the balanced dataset, the better is the effect of mitigation; similarly lower the recall / hallucination gap, better is the mitigation.\n\nMore experiments: The authors demonstrate that after treating a model using SPIRE, it does better zero-shot generalization to UnRel and Spatial Sense datasets. They also show that SPIRE can be adapted to either SPIRE-R or SPIRE-EM. The former works without pixel level object annotations but uses image level annotations. The latter learns a few-shot nearest neighbor classifier on a handful of manually annotated images to get around the lack of annotations in the dataset to try to make SPIRE even more generally applicable when no formal image level annotations are available for the SP in question.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "### Experiments\n\n- For a handful of SPs, could Table 3 experiments be run with end-to-end fine-tuning? If I understood correctly, the problem with fine-tuning end-to-end in general is that one cannot mitigate multiple SPs simultaneously when doing so. But reporting results for a single SP at a time in this setting will be helpful. [Critical]\n\n- Analysis: Demonstrate that identification reports no SPs when a similarly sized box not covering any object is grayed out. [Critical]\n\n- Ablation: What if we gray out a random box in the image instead of the box around the correct object? This baseline should ideally demonstrate no mitigation or lower mitigation than SPIRE. [Critical]\n\n### Presentation\n- Please state explicitly the assumptions made for the type of annotations used by SPIRE for mitigation and identification. [Critical]\n- In the submission, \"Object detection\" is often used to refer to the task of classifying whether an object is present in the scene or not. In computer vision, however, object detection refers to joint classification and localization of a tight fitting bounding box around the object. Please replace \"object detection\" with \"object classification\" or similar. [Critical]\n- Figure 4 cannot be parsed without colour. Please use -. -* and other line styles to make this easier for colour blind people and for grayscale printing. [Moderate]\n- Please provide equations for the projection in \"generalization 2\" experiments. [Critical]\n- Please provide equations for average recall gap and hallucination gap (or pseudo code). [Moderate]\n- Please rewise the claim \"no dataset annotations\" as manually inspecting several randomly sampled images from each cluster is data annotation. [Critical]\n- Lots of repetitions were used in the experiments to report average performance but no error bars were reported. Please report error bars in atleast a few cases. [Critical]\n\n### Typos\nPage 6 bottom \"Indeed, when -we- measure ...\"\n\n", "broader_impact_concerns": "I do not see any broader impact concerns. Rather this work helps address the broader impact concerns of image classification models.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an approach for identifying spurious patterns in a training set when pixel-wise object-annotations are available, and a data-augmentation method for removing such patterns. The reviewers generally liked the paper, but had concerns about the clarity of the presentation. Reviewers also thought that the requirement of having pixel-wise object-annotations available was a major weakness. 

The authors have responded to these concerns and improved the presentation as well as making other improvements requested by the reviewers. The reviewers seemed to be satisfied by these changes and recommended acceptance. 

While the authors have clarified in the main text that their approach requires the availability of pixel-wise object-annotations, I believe that it is important to also mention this in the abstract. I recommend acceptance conditioned on the minor revision of making this clear in the abstract.  
Decision: Accept (poster)