Original: S IMPLE S POT and Evaluating Systemic Errors using Synthetic Image Datasets Anonymous Authors
Revised: Evaluating Systemic Error Detection Methods using Synthetic Images

Original: Abstract
Revised: Abstract

Original: We introduce SynthSpot , a framework for generating synthetic datasets to use for evaluating methods for discovering blindspots ( i.e., systemic errors) in image classifiers, and SimpleSpot , a method for discovering such blindspots.
Revised: We introduce SpotCheck , a framework for generating synthetic datasets to use for evaluating methods for discovering blindspots ( i.e., systemic errors) in image classifiers.

Original: We use SynthSpot to run controlled studies of how various factors influence  blindspot discovery method performance.
Revised: We use SpotCheck to run controlled studies of how various factors influence the performance of blindspot discovery methods.

Original: Our experimental results reveal severalimportant shortcomings of existing methods, such astheir relatively poor performance in settings with multiplemodel blindspots andtheir sensitivity to hyper-parameters.
Revised: Our experiments reveal several shortcomings of existing methods, such as relatively poor performance in settings with multiple blindspots and sensitivity to hyperparameters.

Original: Further, we find that SimpleSpot is competitive with existing methods, which has promising implications for developing an interactive tool based on it.
Revised: Further, we find that a method based on dimensionality reduction, PlaneSpot , is competitive with existing methods, which has promising implications for the development of interactive tools.

Original: 1. Introduction
Revised: 1. Introduction

Original: A growing body of research has established that machine learning models with high test performance often make systemic errors (Buolamwini and Gebru, 2018; 
Revised: A growing body of research has found that machine learning models with high test performance often make systemic errors (Buolamwini and Gebru, 2018; Chung et al., 2019;

Original: Chung et al., 2019; Oakden-Rayner et al., 2020;Ribeiro et al., 2020;
Revised: Oakden-Rayner et al., 2020;

Original:  Singla et al., 2021; Ribeiro and Lundberg,2022; Eyuboglu et al., 2022), which occur when the model performs significantly worse on a coherent ( i e., semantically meaningful) subset of the data.
Revised: Ribeiro et al., 2020; Singla et al., 2021; Ribeiro and Lundberg, 2022), which occur when the model performs significantly worse on a coherent ( i e., semantically meaningful) subset of the data.

Original: For example, past works (Winkler et al., 2019; Mahmood et al., 2021) have demonstrated that models trained to diagnose skin cancer from dermoscopic images rely on spurious artifacts (such as the presence of a surgical skin marker that some dermatologists use to mark lesions) to make predictions.
Revised: For example, past works (Winkler et al., 2019; Mahmood et al., 2021) have demonstrated that models trained to diagnose skin cancer from dermoscopic images rely on spurious artifacts (such as the presence of a surgical skin marker that some dermatologists use to mark lesions) to make predictions.

Original: As a result, these models have different performance on images with or without those spurious artifacts.
Revised: As a result, these models have different performance on images with or without those spurious artifacts.

Original: More broadly, being able to discover systemic errors is critical for many different reasons, including detecting algorithmic bias (Buolamwini and Gebru, 2018; Cabrera et al., 2019) or sensitivity to distribution shift (Sagawa et al., 2020; Singh et al., 2020).
Revised: More broadly, discovering systemic errors is critical in a range of applications, such as detecting algorithmic bias (Buolamwini and Gebru, 2018) or sensitivity to distribution shifts (Sagawa et al., 2020; Singh et al., 2020).

Original:  While there is a rich history of work that studies how to find systemic errors in settings where there is useful metadata for defining coherent subsetsof the data (Buolamwini and Gebru, 2018;Chung et al., 2019; Cabrera et al., 2019;
Revised: edu While there is a rich body of work that studies how to find systemic errors in settings where there is useful metadata to define coherent subsets (Buolamwini and Gebru, 2018;

Original:  Singh et al., 2020), finding systemic errors is much harder in settings without such metadata.
Revised: Chung et al., 2019; Cabrera et al., 2019; Singh et al., 2020), finding systemic errors is much harder in settings without such metadata.

Original: For example,in many image classification settings we  do nota-priori know  which images may or may not have some artifact (such as a surgical skin marker) present.
Revised: For example, we often do not know a-priori which images may or may not have some artifact (such as a surgical skin marker).

Original: To address this challenge, several 
Revised: To address this challenge, several Blindspot Discovery Methods (BDMs) 1 , such as Sohoni et al.

Original: Blindspot Discovery Methods (BDMs) 1 , such as (Sohoni et al., 2020; Singla et al., 2021; d’Eon et al., 2021; Eyuboglu et al., 2022), have been proposed to discover blindspots( i.e., systemic errors) in settings without useful metadata.
Revised: ; Singla et al. (2021); d’Eon et al. Eyuboglu et al. (2022), have been proposed to discover blindspots in settings without useful metadata.

Original: While the goals and technical approaches of these BDMs are well-defined, there is no standardized approach for evaluating them.
Revised: While the motivation and technical approaches of these BDMs are well defined, there is no standardized approach for evaluating them.

Original: Evaluating the hypothesized blindspots returned by BDMs is fundamentally challenging because it is unclear how to measure their coherence and because we do not have the complete set of true blindspots to compare them against.
Revised: Evaluating the hypothesized blindspots returned by BDMs is fundamentally challenging because it is unclear how to measure their coherence and because we do not have the complete set of true blindspots to compare them against.

Original: One approach to address these challenges is to compare the hypothesized blindspots to a subset of the true blindspots that have either been artificially induced or identified in prior work (Sohoni et al., 2020; Eyuboglu et al., 2022).
Revised: One approach to address these challenges is to compare the hypothesized blindspots to a subset of the true blindspots that have either been artificially induced or identified by existing work (Sohoni et al., 2020; Eyuboglu et al., 2022).

Original: While this is a promising direction, it makes it difficult to measure a BDM’s recall or false positive rate or to identify various factors that influence a BDM’s performance.
Revised: While this is a promising direction, it makes it difficult to measure a BDM’s recall or false positive rate or to identify various factors that influence a BDM’s performance.

Original: To address these challenges, we propose SynthSpot which builds onthe ideas from (Kim et al., 2022) by generating synthetic datasets with varying degrees of complexity and then training models on those datasets to have specific blindspots of varying types.
Revised: To address these challenges, we propose a synthetic evaluation framework for BDMs called SpotCheck . SpotCheck builds on ideas from Kim et al. by generating synthetic datasets with varying degrees of complexity and then training models on those datasets to have various types of blindspots.

Original: This allows us to measure a
Revised: This allows us to measure a

Original: BDM’s recall and false positive rate, because we know the complete set of true blindspots for each model, and to measure how various factors, such as the number  of blindspots in the model, influence a BDM’s performance.
Revised: BDM’s recall and false positive rate, since we know the full set of true blindspots for each model, and to measure how various factors, such as the number or types of blindspots in the model, influence a BDM’s performance.

Original: We use SynthSpot to conduct a evaluation of 3 recent
Revised: We use SpotCheck to conduct an evaluation of 3 recent

Original: BDMs and compare them to a new approach that we introduce, called SimpleSpot
Revised: BDMs and compare them to a new baseline method called PlaneSpot .

Original: Our evaluation reveals several insights about BDM performance: performance degrades quickly as the number of blindspots in a model increases, performance depends on the type of features that define a blindspot, and that performance is very sensitive to the
Revised: Our evaluation reveals several insights about BDM performance: performance degrades quickly as the number of blindspots in a model increases, performance label:label:label:label:label:label:

Original: BDM’s hyper-parameters (which are challenging to tune).
Revised: [Figure]

Original:  Interestingly, we observe that SimpleSpot consistently outperforms existing BDMs and is comparatively robust to hyper-parameter choice.
Revised: depends on the type of features that define a blindspot, and performance is sensitive to the BDM’s hyperparameters. Interestingly, we observe that PlaneSpot consistently outperforms existing BDMs.

Original: Because SimpleSpot utilizes a 2-dimensional representation, our suggests that it could bebe useful in an interactive setting.
Revised: Because PlaneSpot utilizes a 2D representation, our findings suggest that it could be useful in an interactive setting.

Original: We plan to explore this direction using real datasets in future work.
Revised: We plan to explore this direction using real datasets in future work.

Original: 2. Synthetic Evaluation Framework
Revised: 2. Synthetic Evaluation for BDMs

Original: SynthSpot builds on ideas from (Kim et al., 2022) to generate synthetic datasets of varying complexity and to train models with specific blindspots on those datasets. 
Revised: SpotCheck builds on ideas from Kim et al. (2022) to generate synthetic datasets of varying complexity and to train models with specific blindspots on those datasets. Below, we summarize its key steps; see Appendix A for details.

Original: Dataset Definition.
Revised: Dataset Definition.

Original: Each dataset can be described using semantic features that define the possible types of imagesthat it contains.
Revised: Each dataset can be defined using semantic features that describe the possible types of images it contains.

Original: Datasets that have a larger number of features have a larger variety of images andtherefore are  more complex.
Revised: Datasets that have a larger number of features have a larger variety of images and are therefore more complex.

Original: For example, avery simple dataset may only contain images withblue squares  while a more complicated dataset may also contain images withorange rectangles, striped circles, small text, and dotted backgrounds.
Revised: For example, a simple dataset may only contain images with squares and blue or orange circles (see Figure 1), while a more complicated dataset may also contain images with striped rectangles, small text, or grey backgrounds.

Original: Blindspot Definition.
Revised: Blindspot Definition.

Original: Each blindspot is defined using a subset of the semantic features that define its associated dataset.
Revised: Each blindspot is defined using a subset of the semantic features that define its associated dataset (see Figure 1).

Original: Similar to how a dataset with more features is more complex, a blindspotthat is defined using more  features is more specific.
Revised: Similarly to how a dataset with more features is more complex, a blindspot defined using more semantic features is more specific.

Original: For example, the “blue squares without circles” blindspot includes all such images, regardless of whether or not they have other objects or what the background looks like, and is defined with three features (the presence of a square, the color of the square, and the absence of a circle).

Original: Training a Model with Specific Blindspots.
Revised: Training a Model with Specific Blindspots.

Original: For each dataset and set of blindspots, we train a [-ResNet-
-]
Revised: For each dataset and blindspot specification, we train a [-ResNet-
-]ResNetmodel

Original: (He et al., 2016) to predict whetheror not a square is present in the image.
Revised: (He et al., 2016) to predict whether a square is present.

Original: To do this, we generate data where the label for each image in the training and validation sets has the correctlabel if and only if it does not belong to any of the blindspots (the test set images are always correctly labeled).
Revised: To induce blindspots, we generate data where the label for each image in the training and validation sets is correct if and only if it does not belong to any of the blindspots (see Figure 1). The test set images are always correctly labeled.

Original: Generating Diverse Experimental Configurations.
Revised: Generating Diverse Experimental Configurations.

Original: Since our goal is to study how various factors influence BDM performance, we generate a diverse set of experimental configurations  ( i.e., dataset, blindspots, and model triplets).
Revised: Since our goal is to study how various factors influence BDM performance, we generate a diverse set of experimental configurations , ( i.e., dataset, blindspots, and model triplets).

Original: To do this, we randomize the features that define a dataset (both the number of them and what theyactually are) as well as the blindspots (the number of them, the number of features that define them, and what those featuresactually are).Note that, while generating the blindspots for a dataset, we make sure that they are uniquely identifiable.
Revised: To do this, we randomize the features that define a dataset (both the number of them and what they are) as well as the blindspots (the number of them, the number of features that define them and what those features are).

Original: 3. Evaluation Metrics
Revised: 3. Evaluation Metrics

Original: A BDM returns an ordered list of hypothesized blindspots, ˆ Ψ : [ˆΨ k ]
Revised: Each BDM returns an ordered list of hypothesized blindspots, ˆ Ψ : [ˆΨ k ]

Original: Kk =1 , sorted by decreasing importance.
Revised: Kk =1 , sorted by decreasing importance.

Original: In our experiments, we have access to the model’s true blindspots, Ψ : { Ψ m } Mm =1 .
Revised: Further, because we are using SpotCheck , we have the complete set of the model’s true blindspots, Ψ : { Ψ m } Mm =1 .

Original: Our goal is to measure how well  ˆ Ψ captures Ψ . Note that the  ˆΨ k and Ψ m are sets of images.
Revised: Our goal is to measure how well the hypothesized ˆ Ψ capture the true Ψ , where each ˆΨ k and Ψ m are sets of images.

Revised: We start by measuring how well a BDM finds each individual true blindspot (Blindspot Recall) and build on that to measure how well a BDM finds the complete set of true blindspots (Discovery Rate and False Discovery Rate).

Original: Blindspot Precision.
Revised: Blindspot Precision.

Original: We start by checking if ˆΨ k is a subset of Ψ m .
Revised: We start by checking if ˆΨ k is a subset of Ψ m .

Original: If it is, we know that the model under-performs on ˆΨ k and that ˆΨ k is coherent.
Revised: If it is, we know that the model underperforms on ˆΨ k and that ˆΨ k is coherent.

Original: We measure this using the precision of ˆΨ k with respect to Ψ m :
Revised: We measure this using the precision of ˆΨ k with respect to Ψ m :

Original: [Equation]
Revised: [Equation]

Original: Then, we say that ˆΨ k belongs to Ψ m if BP (ˆΨ k , Ψ m ) >
Revised: We say that ˆΨ k belongs to Ψ m if, for some threshold λ p :

Original: λ p .
Revised: [Equation]

Original: However, ˆ Ψ k can belong to Ψ m without capturing the same information as Ψ m .
Revised: However, ˆΨ k can belong to Ψ m without capturing the same information as Ψ m .

Original: For example, ˆΨ k could be “striped blue squares” while Ψ m could be “striped squares, that are blue or orange”.
Revised: For example, ˆΨ k could be “squares and blue circles” while Ψ m could be “squares and blue or orange circles”.

Original: Because this excessive specificity could result in the user arriving at insufficiently general conclusions, we need to incorporate some notion of recall into the evaluation.
Revised: Because this excessive specificity could result in the user arriving at insufficiently general conclusions, we need to incorporate some notion of recall into the evaluation.

Original: Blindspot Recall.
Revised: Blindspot Recall.

Original: One approach to measure recall is to calculate the proportion of Ψ m that ˆΨ k covers individually :
Revised: One approach to measure recall is to calculate the proportion of Ψ m that ˆΨ k covers individually :

Original: [Equation]
Revised: [Equation]

Original: We relax this definition by allowing Ψ m to be covered by the union of multiple of the ˆΨ k that belong to it:
Revised: We relax this definition by allowing Ψ m to be covered by the union of multiple of the ˆΨ k that belong to it:

Original: [Equation]
Revised: [Equation]

Original: Then, we say that ˆ Ψ covers Ψ m if BR ( ˆ Ψ , Ψ m ) >
Revised: We say that ˆ Ψ covers Ψ m if, for some threshold

Original: λ r .
Revised: λ r :

Original:  We use thisrelaxation because “striped blue squares” and“striped orange squares” both belong to and collectively cover “striped squares, that are blue or orange”.
Revised: [Equation] We do this because “squares and blue circles” and “squares and orange circles” belong to and jointly cover “squares and blue or orange circles.”

Original: So a user could use them to arrive at the correct conclusion.
Revised: So, if a BDM returns both, a user could combine them to arrive at the correct conclusion.

Original: Discovery Rate (DR).
Revised: Discovery Rate (DR).

Original: We define the Discovery Rate of ˆ Ψ and Ψ as the fraction of the Ψ m that are covered by ˆ Ψ .
Revised: We define the discovery rate of ˆ Ψ and Ψ as the fraction of the Ψ m that are covered by ˆ Ψ :

Original:  False Positive Rate (FPR).
Revised: [Equation] False Discovery Rate (FDR).

Original: If the DR is non-zero, we definethe false positive rate of ˆ Ψ and Ψ as the fraction of the ˆΨ k that do not belong to any of the Ψ . If the DR is zero, we exclude the FPR from our analysis. Method DR
Revised: When the DR is non-zero, we define false discovery rate of ˆ Ψ and Ψ as the fraction of the ˆΨ k that do not belong to any of the Ψ :

Original: [Table]
Revised: [Equation]

Original: Barlow [Table] Spotlight [Table]
Revised: Note that, without the complete set of true blindspots (as in

Original: Domino

Original: [Table] SimpleSpot
Revised: SpotCheck ) it is impossible to calculate FDR.

Original: [Table]
Revised: [Table]

Original: 4. SimpleSpot
Revised: 4. PlaneSpot

Original: SimpleSpot starts from the representation defined by the model’s penultimate layer and uses scvis (Ding et al., 2018) to learn a 2-d embedding of that representation.
Revised: PlaneSpot starts from the representation defined by the model’s penultimate layer and uses scvis (Ding et al., 2018) to learn a 2D embedding of that representation.

Original: This 2-d embedding is then normalized so each dimension has range [0, 1] and a 3rd dimension,consisting of w ∗ ModelConfidence(x, y), is appended.
Revised: This 2D embedding is then normalized, so each dimension has range [0, 1], and a 3rd dimension, w ∗ ModelConfidence(x, y), is appended.

Original: This 3-d input isthen passed to a Gaussian Mixture Model clustering algorithm where the number of clusters is chosen using the Bayesian Information Criterion.
Revised: This 3D input is passed to a Gaussian Mixture Model clustering algorithm where the number of clusters is chosen using the Bayesian Information Criterion.

Original: The clusters are then sorted by the product of their error rate and the number of errors in them.
Revised: The clusters are then sorted by the product of their error rate and the number of errors in them.

Original: w is a hyper-parameter that controls the relative importance of the 2-d embedding of an image and the model’s confidencein its prediction for that image for the clustering.
Revised: w is a hyperparameter that controls the relative importance of the 2D embedding of an image and the model’s confidence for that image.

Original: 5. Experiments
Revised: 5. Experiments

Original: We use  100 experimental configurations generated by SynthSpot to study BDM performance.
Revised: We use SpotCheck to generate 100 experimental configurations whose datasets have 6-8 semantic features and whose models have 1-3 blindspots with 5-7 features.

Revised: We evaluate the following BDMs: Spotlight (d’Eon et al., 2021), Barlow (Singla et al., 2021), and Domino (Eyuboglu et al., 2022).

Original: For each BDM, we use a held-out set of 20 configurations to select hyperparameters.
Revised: For each BDM, we use a held-out set of 20 configurations to select hyperparameters.

Original: We use λ p = λ r = 0 . 
Revised: We use λ p = λ r = 0 . 8 .

Original: 8 for our metrics.

Original: Overall Results.
Revised: Overall Results.

Original: Table 1 shows the DR and FPR results averaged across all 100 experimental configurations.
Revised: Table 1 shows the DR and FDR results averaged across all 100 experimental configurations.

Original: We observe that, in comparison to other methods, SimpleSpot has the highest DR, on average finding 85% of the true blindspots per experimental configuration.
Revised: We observe that, in comparison to other methods, PlaneSpot has the highest DR, on average finding 85% of the true blindspots per experimental configuration.

Original: SimpleSpot and Barlow have a lower FPR than Spotlight and Domino. 
Revised: PlaneSpot and Barlow have a lower FDR than Spotlight and Domino. Identifying factors that influence BDM performance.

Revised: We study two types of factors: holistic factors , which measure properties of the dataset ( e.g., how complex is it?)

Revised: or of the model ( e.g., how many blindspots does it have?), and specific factors , which measure properties of a blindspot ( e.g., is it defined with this feature?).

Revised: For holistic factors, we average DR and FDR across the experimental configurations.

Revised: For specific factors, we find the “fraction of true blindspots covered” averaged across each individual blindspot from the experimental configurations (see Equation 5).

Original: The number of blindspots matters.
Revised: The number of blindspots matters.

Original: In Figure 1, we plot the average DR for experimental configurations with 1 , 2 , and 3 true blindspots.
Revised: In Figure 2, we plot the average DR for experimental configurations with 1 , 2 , and 3 true blindspots.

Original: Average DR decreases for all methods as the number oftrue model blindspots increases, albeit at different rates.
Revised: Average DR decreases for all methods as the number of blindspots increases.

Original: Further, FPR increases as this happens as well (not shown).
Revised: Figure 3, shows that FDR increases as this happens as well.

Original: This insight particularly significant because past evaluations have almost exclusively focused on settings that consider only onetrue blindspot.
Revised: The conclusion that methods perform worse in settings with multiple blindspots is particularly significant because past evaluations have primarily focused on settings with one blindspot.

Original: The specificity of blindspots matters.
Revised: The specificity of blindspots matters.

Original: In Figure 2, we plot the fraction of true blindspots covered for blindspots defined using 5 , 6 , and 7 features.
Revised: In Figure 4, we plot the fraction of true blindspots covered for blindspots defined using 5 , 6 , and 7 features.

Original: With the exception of
Revised: With the exception of

Original: Spotlight, all of these methods are less able to find more specific/smaller blindspots.
Revised: Spotlight, all of these methods are less capable of finding more specific/less frequently occurring blindspots.

Original: The features that define a blindspot matter.
Revised: The features that define a blindspot matter.

Original: In Figure 3, we plot the fraction of true blindspots covered for blindspots thateither are or are not defined using the “relative position” feature; this feature is an indicator for whether the squarein the image islocated above its horizontal center-line.
Revised: In Figure 5, we plot the fraction of true blindspots covered for blindspots that are or are not defined using the “relative position” feature; this feature is an indicator for whether the square is above the image’s horizontal center line.

Original: With the exception of SimpleSpot , all of these these are less able to find blindspots defined using this feature.
Revised: With the exception of PlaneSpot , all methods are less likely to find blindspots defined using this feature.

Original: This suggests that the types of features used to define a blindspot ( e.g., the presence of spurious objects, color or texture information, backgroundinformation, or location information) may influence BDM performance.
Revised: This shows that the types of features used to define a blindspot ( e.g., the presence of spurious objects, color or texture information, background information) can influence BDM performance.

Original: There should be more discussion on hyper-parameter tuning.
Revised: There should be more discussion on hyperparameter tuning.

Original: In Figure 4, we observe that two hyper-parameter settings that perform nearly identically on averagecan exhibit very different performance in different settings.
Revised: In Figure 6, we observe that two hyperparameter settings that perform nearly identically on average exhibit significantly different performance at identifying blindspots Despite the fact that these hyperparameter choices perform similarly on average, they behave differently based on the number of features used to define the blindspot.

Original:  This suggests that their may not be a single best hyper-parameter choice to discover all of the blindspots in a single model, which could contain multiple blindspots of different specificity or size.
Revised: defined using differing numbers of features. This suggests that there may not be a single best hyperparameter choice to discover all of the blindspots in a single model, which could contain multiple blindspots of different specificity or frequency.

Original: In conjunction with the general sensitivity that these methods have to their hyper-parameters, this suggests that there should be more discussion on hyper-parameter tuning (especially since this is much easier in our controlled setting than in real applications).
Revised: In conjunction with the general sensitivity that these methods have to their hyperparameters, this suggests that there should be more discussion on hyperparameter tuning (especially since this is much easier in our controlled setting than in real applications).

Original: 6. Related Work
Revised: 6. Related Work

Original: Finding Blindspots.
Revised: Finding Blindspots.

Original: Many methods have been proposed to help users discover blindspots across a wide set of applications.
Revised: Numerous methods have been proposed to help users discover blindspots across a wide set of applications.

Original: We focus on methods that make the least restrictive assumptions andthat work for image classification models.
Revised: We focus on methods that make the least restrictive assumptions and work for image classification models.

Original: Specifically, we evaluate the methods from Singlaet al.
Revised: Specifically, we evaluate the methods from Singla

Original:  (2021); d’Eon et al.
Revised: [Table] et al. (2021); d’Eon et al.

Original: ; Eyuboglu et al.
Revised: ; Eyuboglu et al.

Original: (2022) because they do not assume any of the following:
Revised: (2022) because they do not assume any of the following:

Original: • Access to metadata (Kim et al., 2018;
Revised: • Access to metadata (Kim et al., 2018;

Original: Buolamwini and Gebru, 2018; Chung et al., 2019; Singh et al., 2020).
Revised: Buolamwini and Gebru, 2018; Chung et al., 2019; Singh et al., 2020; Plumb et al., 2021).

Original: • Access to tools that manipulatethe data (Shetty et al., 2019; Singla et al., 2020; Xiao et al., 2021; Leclerc et al., 2021; Bharadhwaj et al., 2021) • Any specific model structure (Alvarez-Melis and Jaakkola, 2018; Koh et al., 2020).
Revised: • Access to tools that manipulate data (Shetty et al., 2019; Singla et al., 2020; Xiao et al., 2021; Leclerc et al., 2021; Bharadhwaj et al., 2021) • Any specific model structure (Alvarez-Melis and Jaakkola, 2018; Koh et al., 2020).

Original: • Any specific model training process (Higgins et al., 2017; Tsipras et al., 2019;
Revised: • Any specific model training process (Higgins et al., 2017; Tsipras et al., 2019;

Original: Wong et al., 2021) • A human in the loop, either through an interactive interface (Cabrera et al., 2019; Balayn et al., 2022; Ribeiro and Lundberg, 2022) or inspecting explanations (Yeh et al., 2020; Adebayo et al., 2021). 
Revised: Wong et al., 2021) • A human in the loop, either through an interactive interface (Cabrera et al., 2019; Balayn et al., 2022; Ribeiro and Lundberg, 2022) or inspecting explanations (Yeh et al., 2020; Adebayo et al., 2021). Table 2 summarizes the design choices of these methods: • They use a model to extract a representation of an image. Typically, this is the same model whose blindspots we are trying

Revised: to discover, but it can be a different model.

Revised: • They apply some form of dimensionality reduction to that image representation.

Original: The methods we evaluate share a common approach to discovering blindspots: (1) they use a, potentially separate, representation model to get an embedding for each point, and (2) learn ablindspot membership model  to predict if a point belongs to aparticular blindspot given its embedding.
Revised: • They learn a model from a specified hypothesis class to predict if an image belongs to a blindspot from the image’s (potentially reduced) representation. Note that this leaves out important details on how that model is learned. We note that PlaneSpot ’s main change is learning a 2D representation using scvis (Ding et al., 2018). Beyond this, it uses the most common choices for the ‘Image Representation’ and ‘Hypothesis class’ while also using standard techniques for learning a model from that hypothesis class.

Original: Quantitative Evaluations of BDMs.
Revised: Quantitative Evaluations of BDMs.

Original: One evaluation approach focuses on measuring properties of the hypothesized blindspots, such as their error rate or size (Singla et al., 2021; d’Eon et al., 2021).
Revised: One evaluation approach focuses on measuring properties of the hypothesized blindspots, such as their error rate or size (Singla et al., 2021; d’Eon et al., 2021).

Original: However, while these properties are important, they do not capture whether  the hypothesized blindspots are coherent.
Revised: However, while these properties are important, they do not capture whether or not the hypothesized blindspots are coherent.

Original: Another approach is to compare the hypothesized blindspots to a subset of the true blindspots that haveeither been artificially induced or identified in prior work (Sohoni et al., 2020; Eyuboglu et al., 2022).
Revised: Another approach is to compare the hypothesized blindspots to a subset of the true blindspots that have been artificially induced or identified in prior work (Sohoni et al., 2020; Eyuboglu et al., 2022).

Original: This approach is similar to
Revised: This approach is similar to

Original: SynthSpot , with a few key differences:
Revised: SpotCheck , with a few key differences:

Original: • Past works use unrealistic definitions of what it means to discover a true blindspot that eitherconsider only  precision (Eyuboglu et al., 2022) or consider both precision and recall but use thresholds for precision that are too lenient (Sohoni et al., 2020).
Revised: • Past work uses unrealistic definitions of what it means to discover a true blindspot that either only considers precision (Eyuboglu et al., 2022) or considers both precision and recall but uses thresholds for precision that are too lenient (Sohoni et al., 2020).

Original: Further, neither allow hypothesized blindspots to be combined.
Revised: Further, neither allow hypothesized blindspots to be combined.

Original: • Without access to the complete set of true blindspots, they cannot measure  DR or FPR.
Revised: • Without access to the complete set of true blindspots, they cannot measure method DR or FDR.

Original: • They do not isolate factors that influence a BDM’s performance ( e.g., number of blindspots, the specificity of those blindspots , or the features that define the blindspots).
Revised: • They do not isolate factors that influence a BDM’s performance ( e.g., number of blindspots, the specificity of those blindspots, or the features that define the blindspots).

Original: 7. Conclusion
Revised: 7. Conclusion

Original: We proposed SynthSpot , a synthetic evaluation framework for BDMs, and ran controlled studies of how various factors influence BDM performance.
Revised: We propose SpotCheck , a synthetic evaluation framework for BDMs, and ran controlled studies of how various factors influence BDM performance.

Original: While running a fully synthetic evaluation has several limitations ( e.g., it remains a question for follow-up work to see if our observed trends generalize to settings with real image data), our evaluation yields fundamental insights about when and why different 
Revised: This evaluation yields fundamental insights about when and why different BDMs are less effective and is an important step towards formalizing a more rigorous and complete set of desiderata for

Original: BDMs are less effective.
Revised: BDMs.

Original: More broadly, our work is an important step towards formalizing a more rigorous and complete set of desiderata for BDMs.
Revised: However, it remains a question for follow-up work to see if our observed trends generalize to settings with real images; in general, we believe that poor performance on synthetic data implies poor performance on real data, but not necessarily the other way around.

Original: Our experimental results have many interesting implications for future work.
Revised: Our experimental results have many interesting implications for future work.

Original: First, it is important to evaluate BDMs in settings with models that have multiple blindspots, as this setting is more realistic andmore challenging.
Revised: First, it is important to evaluate BDMs in settings with models that have multiple blindspots, as this setting is more realistic and challenging.

Original: Second, finding a way to tune BDM hyper-parameters in realistic settings is an open challenge with significant practical importance.
Revised: Second, finding a way to tune BDM hyperparameters in realistic settings is an open challenge with significant practical importance.

Original: Finally, we are intrigued by the result that SimpleSpot , which learns a GMM on a 2-d embedding, performs competitively with prior methods, which use significantly higher dimensional embeddings. 
Revised: Finally, we are intrigued by the result that PlaneSpot , which learns a GMM on a 2D embedding, performs competitively with prior methods, which use significantly higherdimensional embeddings. In future work, we hope to evaluate PlaneSpot on real image data and to explore methods to visualize its 2D embedding as the basis for an interactive blindspot discovery tool. A. Generating Experimental Configurations, Extended

Original: In future work, we hope to evaluate SimpleSpot on real image data to see if this result generalizes.
Revised: In this section we detail how we use SpotCheck to generate random experimental configurations. • In Section A.1, we define the different types of semantic features that can appear in each image. • In Section A.2, we define a synthetic image dataset, how we generate random datasets, and how we sample images from a dataset. • In Section A.3, we define a blindspot for a synthetic image dataset, how we generate a random blindspot, and how we generate an unambiguous set of blindspots.

Original: If it does, this has promising implications for the development of an interactive blindspot discovery tool by replacing the GMM with a human in the loop.
Revised: Table 3 defines all of the semantic features that SpotCheck uses to generate synthetic images. We call these semantic features Attributes and group them into Layers based on what part of an image they describe. Each Attribute has two possible

Revised: Values : a Default and Alternative Value.

Revised: Each synthetic image has an associated list of (Layer, Attribute, Value) triplets that describes the image.

Revised: Figure 7 shows this triplet list for two synthetic images.

Revised: We sometimes refer to the Square/Rectangle/Circle/Text Layers as Object Layers because they all describe a specific object that can be present in an image.

Revised: The location of each object within an image is chosen randomly, subject to the constraint that each object doesn’t overlap with any other object.

Revised: [Table]

Revised: A.2. Defining a Dataset using these Semantic Features

Revised: At a high level, SpotCheck defines a Dataset by deciding whether or not each Attribute of each Layer is Rollable ( i.e., the

Revised: Attribute can take either its Default or Alternative Value, uniformly at random) or not Rollable ( i.e., the Attribute only takes its Default Value).

Revised: We measure a Dataset’s complexity using the number of Rollable Attributes it has.

Revised: Figure 7 describes the

Revised: Rollable and Not Rollable Attributes for an example Dataset.

Revised: Generating a Random Dataset.

Revised: We start by picking which Layers will be part of the Dataset:

Revised: • Images need a background, so all Datasets have the Background Layer.

Revised: • The task is to predict whether there is a square in the image, so all Datasets have the Square Layer.

Revised: • We add 1-3 (chosen uniformly at random) of the other Object Layers (chosen uniformly at random without replacement from the set { Rectangle, Circle, Text } ) to the Dataset.

Revised: Once the Layers are chosen, we make 6-8 (chosen uniformly at random) of the Attributes Rollable:

Revised: • Each Object Layer has its Presence Attribute made Rollable.

Revised: Then, the remaining Rollable Attributes are chosen by iteratively: – Selecting a Layer uniformly at random from those that have at least one Not Rollable Attribute.

Revised: – Selecting an Attribute from that Layer uniformly at random from those that are Not Rollable.

Revised: Sampling an Image from a Dataset.

Revised: Once a Dataset’s Rollable Attributes have been defined, generating a random image is straightforward:

Revised: • For each Attribute from each Layer in the Dataset, we pick a random Value if the Attribute is Rollable.

Revised: Attributes that are Not Rollable will take their Default Value.

Revised: – If the Layer is an Object Layer: * If the Presence Attribute is True, the location of the object is chosen randomly (subject to the non-overlapping constraint).

Revised: * If the Presence Attribute is False, the object will not be rendered (regardless of the Values chosen for the other Attributes of this Layer).

Revised: • We then use the resulting (Layer, Attribute, Value) triplet list and the list of object locations to render a 224x224 RGB image.

Revised: Finally, we calculate any MetaAttributes (explained next) and

Revised: append these (Layer, MetaAttribute, Value) triplets to the image’s definition list.

Revised: Calculating MetaAttributes.

Revised: While each Attribute corresponds to a semantic feature, there are a potentially infinite number of MetaAttributes that one could calculate as semantically meaningful functions of an image.

Revised: We list the MetaAttributes that we calculate in our experiments in Table 4.

Revised: Because this space is infinitely large and grows with the number of Attributes, we exclude MetaAttributes from our measure of Dataset complexity.

Revised: [Figure]

Revised: Definition: Rollable Attributes

Revised: Background: {Color: False, Texture: True }

Revised: Square: {Presence: True , Size: False,

Revised: Color: True , Texture: True ,

Revised: Number: False}

Revised: Text: {Presence: True , Size: False,

Revised: Color: True , Texture: False}

Revised: Example Image # (Background, Texture, Solid), (Square, Presence, True), (Square, Color, Blue), (Square, Texture, Vertical Stripes), (Text, Presence, True), (Text, Color, Orange}

Revised: Example Image # (Background, Texture, Salt & Pepper), (Square, Presence, False), (Square, Color, Blue), (Square, Texture, Solid), (Text, Presence, True), (Text, Color, Blue)

Revised: [Table]

Revised: A.3. Defining the Blindspots for a Dataset

Revised: SpotCheck defines a Blindspot using a list of (Layer, (Meta)Attribute, Value) triplets.

Revised: We measure a Blindspot’s specificity using the length of its definition list.

Revised: An image belongs to a blindspot if and only if the Blindspot’s definition list is a subset of the image’s definition list.

Revised: Figure 8 shows two example Blindspots.

Revised: Generating a Random Blindspot.

Revised: SpotCheck generates a random Blindspot consisting of 5-7 (chosen uniformly at random)

Revised: (Layer, (Meta)Attribute, Value) triplets for a Dataset by iteratively:

Revised: • Selecting a Layer (uniformly at random from those that have at least one Rollable Attribute 4 that is not already in this Blindspot)

Revised: Selecting a Rollable Attribute from that Layer: – Object Layers: If the Layer’s Presence Attribute is not in this Blindspot, select its Presence Attribute.

Revised: Otherwise, select an Attribute uniformly at random from those that are not already in this Blindspot and set the Layer’s Presence Attribute Value to True for this Blindspot.

Revised: – Background Layers: Select an Attribute uniformly at random from those that are not already in this Blindspot.

Revised: • Selecting a Value for that Attribute (uniformly at random)

Revised: Notice that, if an Object Layer is selected more than once, then we ensure that the Object’s Presence Attribute has a Value of True in the Blindspot definition.

Revised: We enforce this Feasibility Constraint to ensure that every triplet in the Blindspot’s definition list correctly describes the images belonging to the Blindspot ( e.g., [(Circle, Presence, False), (Circle, Color,

Revised: Blue)] is infeasible because an image with a blue circle must have a circle in it).

Revised: Generating an Unambiguous Set of Blindspots.

Revised: For each Dataset, we generate 1-3 (chosen uniformly at random)

Revised: Blindspots using the process described above.

Revised: However, when generating multiple blindspots, they can be ambiguous which causes problems when using them to evaluate BDMs.

Revised: Definition.

Revised: A set of Blindspots, S 1 , is ambiguous if there exists a different set of Blindspots, S 2 , such that both:

Revised: The union of images belonging to S 1 is equivalent to the union of images belonging to S 2 .

Revised: As a result, S 1 and S 2 would both correctly describe the model’s blindspots.

Revised: An evaluation that uses Discovery Rate (Equation 6) would penalize a BDM if it returns S 2 instead of S 1 .

Revised: More precisely, DR ( S 2 , S 1 ) <

Revised: 1 for λ p = λ r = 1 .

Revised: Example.

Revised: Suppose that we have a very simple Dataset with two Rollable Attributes, X and Y which are uniformly distributed and independent, and consider two different sets of Blindspots for this Dataset:

Revised: • S 1 = { B 1 , B 2 } where B 1 =

Revised: [( X = 1)] and B 2 =

Revised: [( X = 0) , ( Y = 1)] • S 2 = { B ′ 1 , B ′ 2 } where B ′ 1 =

Original: {+[( X = 1) , ( Y = 0)] and B ′ 
+}
Revised: {+[( X = 1) , ( Y = 0)] and B ′ 
+}

Revised: =

Revised: [( Y = 1)]

Revised: Then, S 1 is ambiguous because:

Revised: • S 1 and S 2 induce the same behavior in the model: they both mislabel an image if X = 1 ∨ Y = 1 .

Revised: • A BDM would be penalized for returning S 2 :

Revised: [Equation]

Revised: In fact, for this example, there are only two sets of two unambiguous Blindspots, ( { [( X = 0) , ( Y = 0)] , [( X = 1) , ( Y = 1)] } and { [( X = 0) , ( Y = 1)] , [( X = 1) , ( Y = 0)] } ), and there exists no unambiguous set of three Blindspots.

Revised: Preventing Ambiguity.

Revised: In general, ambiguity occurs whenever the union of two blindspots forms a contiguous region in the discrete space defined by the Rollable Attributes.

Revised: Consequently, we prevent ambiguity by ensuring that any pair of blindspots has at least two of the same Rollable Attributes with different Values in their definition lists.

Revised: We call this the

Revised: Ambiguity Constraint .

Revised: Implications of the Ambiguity and Feasibility Constraints.

Revised: In our experiments, our goal is to generate experimental configurations with a diverse set of Datasets and associated Blindspots.

Revised: However, the Ambiguity Constraint (AC) and

Revised: Feasibility Constraint (FC) limit the number of valid Blindspots for any specific Dataset.

Revised: To see this, notice that the AC places more constraints on each successive Blindspot added to an experimental configuration.

Revised: This has two implications.

Revised: First, that generating an experimental configuration with more Blindspots requires a Dataset with more Rollable Attributes (more complexity) and Blindspots with more triplets (more specificity).

Revised: Further, because we cannot set the Attribute Values of a Blindspot’s triplets independently of each other [FC], we need more complexity and specificty than a simple analysis based only on the AC suggests.

Revised: Second, that each successive Blindspot is more closely related to the previous ones which means that larger sets of Blindspots are “less diverse” or “less random” in some sense.

Revised: With these trade-offs in mind, we generated experimental configurations with:

Revised: • Background, Square, and 1-3 other Object Layers • A total of 6-8 Rollable Attributes • 1-3 Blindspots • 5-7 triplets per Blindspot

Revised: because an experimental configuration with any combination of these values is able to satisfy the AC and the FC while still having a diverse set of Blindspots.

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: is defined by a list of (Layer

Revised: ) triplets as shown above.

Revised: We also display belonging to all of the images inside of the blue border belong to Blindspot #1, while only the subset of images inside of the orange border belong to Blindspot #2.

Revised: In this example, Blindspot #2 is more specific (defined using 6 semantic features) than Blindspot #1 (defined using 5 semantic features).