Towards a More Rigorous Science of Blindspot Discovery in Image Models
Authors: Gregory Plumb
Nari Johnson
Angel Cabrera
Ameet Talwalkar
Venue: Submitted to ICLR 2023 (2023)

Abstract: A growing body of work studies Blindspot Discovery Methods (BDMs): methods for finding semantically meaningful subsets of the data where an image classifier performs significantly worse, without making strong assumptions. Motivated by observed gaps in prior work, we introduce a new framework for evaluating BDMs, SpotCheck, that uses synthetic image datasets to train models with known blindspots and a new BDM, PlaneSpot, that uses a 2D image representation. We use SpotCheck to run controlled experiments that identify factors that influence BDM performance (e.g., the number of blindspot in a model) and show that PlaneSpot outperforms existing BDMs. Importantly, we validate these findings using real data. Overall, we hope that the methodology and analyses presented in this work will serve as a guide for future work on blindspot discovery.

Under review as a conference paper at ICLR 2023
TOWARDS A MORE RIGOROUS SCIENCE OF BLINDSPOT DISCOVERY IN IMAGE MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
A growing body of work studies Blindspot Discovery Methods (BDMs): methods for finding semantically meaningful subsets of the data where an image classifier performs significantly worse, without making strong assumptions. Motivated by observed gaps in prior work, we introduce new framework for evaluating BDMs, SpotCheck, that uses synthetic image datasets to train models with known blindspots and new BDM, P PlaneSpot, that uses 2D image representation. We use Spot Check to run controlled experiments that identify factors that influence BDM performance (e.g., the number of blindspot in model) and show that P PlaneSpot outperforms existing BDMs. Importantly, we validate these findings using real data. Overall, we hope that the methodology and analyses presented this work will serve as guide for future work on blindspot discovery.
INTRODUCTION
A growing body of work has found that models with high test performance can still make systemic errors, which occur when the model performs significantly worse on a semantically meaningful subset of the data (Buolamwini & Gebru, 2018; Chung et al., 2019; Dakden-Raynet et al., 2020; Singla et al., 2021; Ribeiro & Lundberg, 2022). For example, past works have demonstrated that models trained to diagnose skin cancer from dermoscopic images sometimes rely spurious artifacts (e.g.. surgical skin markers that some dermatologists use to mark lesions); consequently, they have different performance images with without those spurious artifacts (Winkler et al., 2019; Mahmood et al., 2021). More broadly, finding systemic errors can help us detect algorithmic bias (Buolamwini & Gebru, 2018) or sensitivity to distribution shifts (Sagawa al., 2020; Singh et al., 2020).
In this work, we focus on what we call the blindspot discovery problem, which is the problem of finding an image classification model's systemic errors' without making many of the assumptions considered in related works (e.g.. we do not assume access to metadata to define semantically meaningful subsets of the data, tools to produce counterfactual images, a specific model structure or training process, or human in the loop). We call methods for addressing this problem Blindspot Discovery Methods (BDMs) (e.g., Kim et al., 2019; Sohoni et al., 2020; Singla al., 2021; d'Fon et al., 2021; Eyuboglu et al., 2022).
We note that blindspot discovery is an emerging research area and that there has been more emphasis on developing BDMs than on formalizing the problem itself. Consequently, we propose a problem formalization, summarize different approaches for evaluating BDMs, and summarize several high- level design choices made by BDMs. When we do this, observe the following two gaps. First, existing evaluations are based on an incomplete knowledge of the model's 's blindspots, which limits the types of measurements and claims they can make. Second, dimensionality reduction is relatively underexplored aspect of BDM design.
Motivated by these gaps in prior work, we propose evaluation framework, SpotCheck, and new BDM, P1 PlaneSppoc Spot. SpotCheck is synthetic evaluation framework for BDMs that gives us complete knowledge of the model's blindspots and allows us identify factors that influence BDM
In past work, "systemic errors" have also been called "failure modes" or "hidden stratification." We introduce "blindspot" to mean the same thing and use it make it clear when are specifically discussing blindspot discovery.

---

Under review as a conference paper at ICLR 2023
performance. Additionally, we refine the metrics used by past evaluations. PlaneSpot is simple BDM that finds blindspots using a 2D image representation.
We use Spot Check to rum controlled experiments that identify factors that influence BDM perfor- mance (e.g,, the number of blindspot in a model) and show that PlaneSpot outperforms existing BDMs. We additional semi-controlled experiments using the COCO dataset (Lin et al., 2014 4) and find that these trends discovered using SpotCheck generalize to real image data. Overall, we hope that the methodology and analyses presented in this work will help facilitate more rigorous science of blindspot discovery.
BACKGROUND
In this section, formalize the problem of blindspot discovery for image classification. We then discuss general approaches for evaluating the Blindspot Discovery Methods (BDMs) designed to address this problem as well as high-level design choices made by BDMs.
Problem Definition. The broad goal of finding systematic errors has been studied across range of problem statements and method assumptions. Some common assumptions are:
Access to metadata help define coherent subsets of the data (e.g., Kim et al., 2018; Buolamwini & Gebru, 2018; Singh et al., 2020).
The ability to produce counterfactual images (e.g., Shetty et al., 2019; Singla et al., 2020; Xiao
et al., 2021; Leclerc et al., 2021; Bharadhwaj et al., 2021; Plumib et al., 2022).
A specific structure for the model we are analyzing (e.g., Alvarez-Melis & Jaakkola, 2018 8: Koh et al., 2020) or for training process used to learn the model's parameters (e.g.. Wong et al., 2021). A human-in--he loop, either through an interactive interface (e.g.. Cabrera et al., 2019; Balayn et al., 2022) or by inspecting explanations (e.g., Ych et al., 2020; Adebayo et al., 2022).
While appropriate at times, these assumptions all restrict the applicability of their respective methods. For example, consider assuming access to metadata to help define coherent subsets of the data. To start, this metadata is much less common in applied settings than it is for common ML benchmarks. Further, the efficacy of methods that rely on this metadata is limited by the quantity and relevance of this metadata; in general, efficiently collecting large quantities of relevant metadata is challenging because it requires anticipating the model's systemic errors.
Consequently, we define the problem of blindspot discovery as the problem of finding an image classification model's systemic errors without making any of these assumptions. More formally, suppose that we have an image classifier, f. and a dataset of labeled images, D = [ Then, blindspot is a coherent (i.e., semantically meaningful) set of images, V C D, where f performs significantly worse (i.e., p(f. V) < p(f, D v for some performance metric, p, such as recall). We denote the set of f's true blindspots as V ( }M Next, we define the problem of blindspor discovery as the problem of finding V using only f and D. Then, BDM method that takes as input f and D and outputs an ordered (by some definition of importance) list of hypothesized blindspots, i | [ Note that the V n and V are sets of images.
Approaches to BDM evaluation. We observe that existing approaches to quantitatively evaluate BDMs fall in two categories. The first category of evaluations simply measure the error rate or size of vx (Singla et al., 2021; d' Eon et al., 2021). However, these evaluations have two problems. First, none of the properties they measure capture whether V/ is coherent (e.g., random sample of misclassified images has high error but may not match a single semantically meaningful description). Second, f's performance on Uh may not be representative of f's performance on similar images because BDMs are optimized to return high error images (e.g., suppose that f has a 90% accuracy on images of "squares and blue circles"; then, by returning the 10% of such images that are misclassified, a BDM could mislead us into believing that f has a 0% accuracy on this type of image).
The second category of evaluations compares to a subset of V that have either been artificially induced or previously found (Sohoni et al., 2020; Eyuboglu et al., 2022). While these evaluations address the issues with those from the first category, they require knowledge of V, which is usually incomplete (i.e,, usually only know subset of L) This incompleteness makes it difficult to identify factors that influence BDM performance or to measure a BDM's s recall or false positive rate. It is fundamentally impossible fix this incompleteness using real data because cannot enumerate
2

---

Under review as a conference paper at ICLR 2023
Table 1: A high level overview the high-level design choices made by different BDMs.
Method
1. Image Representation
2. Dimensionality Reduction
3. Hypothesis Class
Multiaccuracy (Kim al., 2019)
VAE representation
iinear model
GEORGE (Sohoni et al., 2020)
Model representation
UMAP (d = 1,
Gaussian kernels
Spotlight Eon al., 2021)
Model representation
Gaussian kernels
Barlow (Singla et al., 2021)
Adversarially Haariallly-obusst Model representation
Decision Tree
Domino (Fyuhoglu et al., 2022)
CLIP representation
PCA = 128)
Gaussian Kerness
PlaneSpot
Model representation
Gaussian Keraels
all of the possible coherent subsets of D to check if they are blindspots. To address these limitations, introduce Spot which gives complete knowledge of Y by using synthetic data.
High-level design choices of BDMs. In Table 1, we summarize three of the high-level design choices made by existing BDMs. First, each BDM uses a model extract an image representation. Many BDMs use a representation from f. but some use pre-trained external models or other models trained on D. Second, some of the BDMs apply some form of dimensionality reduction to that image representation. Third, each BDM learns model from a specified hypothesis class to predict if an image belongs to a blindspot from that image's (potentially reduced) representation.
Interestingly, while there has been significant effort focused on the choice of BDM's image representation and hypothesis class (along with its associated learning algorithm), we note that dimensionality reduction has received much less attention. This is surprising because these BDMs are all solving clustering or learning problems, which are generally easier in lower dimensions. Motivated by this gap in prior work, we introduce simple BDM, P Pla SaneSpot, that uses 2D representation.
3 EVALUATING BDMs USING KNOWLEDGE OF THE TRUE BLINDSPOTS
In Section 3.1, we introduce SpotCheck which is an evaluation framework for BDMs that gives us complete knowledge of the model's true blindspots by using synthetic images and allows us to identify factors that influence BDM performance. In Section 3.2, we define the metrics that we use to measure the performance of BDMs given knowledge of the model's true blindspots.
3.1 SPOTCHECK: A SYNTHETIC EVALUATION FRAMEWORK FOR BDMS
SpotCheck builds ideas from Kim et al. (2022) by generating synthetic datasets of varying complexity and training models to have specific blindspots on those datasets. We summarize its key steps below; see Appendix A for details.
Dataset Definition. Each dataset is defined using semantic features that describe the possible types of images it contains. Datasets that have a larger number of features have larger variety of images and are therefore more complex. For example, a simple dataset may only contain images with squares and blue orange circles (see Figure 1), while a more complicated dataset may also contain images with striped rectangles, small text, or grey backgrounds.
Blindspot Definition. Each blindspot is defined using subset of the semantic features that define its associated dataset (see Figure 1). Similarly to how a dataset with more features is more complex, blindspot defined using more semantic features is more specific.
Training a Model to have Specific Blindspots. For each dataset and blindspot specification, we train ResNet- 18 model (He et al., 2016) predict whether a square is present. To induce blindspots. we generate data where the label for each image in the training and validation sets is correct if and only if it does not belong to any of the blindspots (see Figure 1). The test set images are always correctly labeled. Then, because know the full set of semantic features that define the dataset, we can verify that the model learned to have exactly the set of blindspots that we intended it to. As a result, SpotCheck gives complete knowledge of the model's true blindspots.
Generating Diverse Experimental Configurations (ECs). Since our goal is to study how various factors influence BDM performance, we generate diverse set of experimental configurations, (i.e., dataset, blindspots, and model triplets). To do this, we randomize several factors: the features that define a dataset (both the number of them and what they are) well the blindspots (the number
3

---

Under review as a conference paper at ICLR 2023
label: 0
label:
label: 0
label:
label:
label: 0
blindspot: images with squares and circles
Figure 1: By controlling the data generation process. SpotChect gives complete knowledge of a model's true blindspots. Here, we show images sampled from a dataset created with SpotCheck. Dataset Complexity. This dataset is defined by 3 semantic features that vary across images: the presence of a square, the presence of a circle, and the color of the circle. We do not count the "color of the square" because is always blue. Blindspot Specificity. This blindspot defined by semantic features: the presence of a square and the presence of circle. As a result, it contains any image with both square and a cirele, regardless of the circle's 's color. Data Labels. In general, the label indicates if a square is present. However, training images belonging to this blindspot are mislabeled.
of them, the number of features that define them, and what those features are). Importantly, we sample these factors independently of one another across this of ECs so that estimate their individual influence on BDM performance.
3.2 EVALUATION METRICS BASED ON KNOWLEDGE OF THE TRUE BLINDSPOTS
These metrics measure how well the hypothesized blindspots returned by BDM, V capture model's true blindspots, Recall that the fx and em are sets of images. To do this, we start by measuring how well BDM finds each individual true blindspot (Blindspot Recall) and build on that to measure how well BDM finds the complete ser of blindspots (Discovery Rate and False Discovery Rate); see Appendix B for discussion of how these metrics refine prior definitions.
Blindspot Precision. If Uh is subset of V mm we know that the model underperforms on V and that V is coherent. We measure this using the precision of V with respect to Vn:
1/ m
B ...)
Then, we say that V belongs to un if, for some threshold Ap:
BP vm) Ap
(2)
However, V can belong to V without capturing the same information as Fn For example, V could be "squares and blue circles" while V m could be "squares and blue or orange circles". Because this excessive specificity could result the user arriving conclusions that are too narrow, we need to incorporate some notion of recall into the evaluation.
Blindspot Recall. One way to incorporate recall is using the proportion of V., that th covers:
| O mm
BR (V,, V Mm)
We relax this definition by allowing V. to be covered by the union of the V that belong to it:
V
B. T
BR + V V
4

---

Under review as a conference paper at ICLR 2023
Then, say that v covers um if, for some threshold A,:
BR( (R Vm) 2 A
(5)
We do this because "squares and blue circles" and "squares and orange circles" belong to and jointly cover "squares and blue or orange circles." So, if a BDM returns both, user could combine them to arrive at the correct conclusion.
Discovery Rate (DR). We define the DR of + and v as the fraction of the LD that & covers:
[(BR V Wm) 2 Ar) DR( V M Σ
(6)
False Discovery Rate (FDR). When the DR is non-zero, we define FDR of & and v the fraction of the Ue that do not belong to any of the u!?²
FDR T L
Σ K
1(max BP vm) < Xp)
922
Note that it is impossible calculate the FDR without the complete set of true blindspots. While SpotCheck gives us this knowledge, it is generally not available.
PLANESPOT: A SIMPLE BDM BASED ON DIMENSIONALITY REDUCTION
In this section, define PlaneSpot. As shown in Table 1, PlaneSpot. the most common choices for the image representation (i.e. f's own representation) and the hypothesis class (i.e. gaussian kernels). P PlaneSpot also uses standard techniques to learn a model from that hypothesis class. As a result, the most interesting aspect of P aneSpot's 's design is that it finds blindspots using a 2D representation. We start by defining some additional notation and then explain P PlaneSpot choice for each of the high-level design choices made by a BDM.
Notation. Suppose that we want to find f's blindspots for class, c. and let Dr be the of images from D that belong to c. Further, suppose that we have divided f into two parts: g X - Rdd which extracts f's representation of image (i.e., its penultimate layer activations), and h Rd [o, 1). which gives f's predicted confidence for
Image Representation. We use g to extract I's representation for D=, G (D9)DD) E Raxd and to extract s's predicted confidences for DE, H = 1(G)) [0, 1|ax1, Note that, because all of the images in D belong to c, entries of H closer to denote higher confidence in the true class.
Dimensionality Reduction. We use G to train sevis (Ding et al., 2018), which combines the objective functions of (ŠNE and an autoencoder, in order to learn a 2D representation of f's representation, R 2 Then, we use get the 2D representation of D S s(G) g=x2 Finally, we normalize the columns of S to be in [o. 1]. 5 E [0, 1|
be aware of both the representation of D S, and f's predicted confidences for D H, so we combine them into single representation, R [S: w H] Rox3 where is hyperparameter controlling the relative weight of these components. Then, pass R to a Gaussian Mixture Model clustering algorithm, where the number of clusters is chosen using the Bayesian Information Criteria. Finally, we return those clusters in order of decreasing importance, which we define using the product of their error rate and the number of errors in them.
Hypothesis Class. We want P PlaneSpot
5 EXPERIMENTS
In Section 5.1, we use SpotCheck run series of controlled experiments using synthetic data, In Section 5.2, we demonstrate how setup semi-controlled experiments to validate that the findings from Shec generalize to settings with real data.
Whilee calculating DR, we may only need the top-u u items of V As a result, we only calculate the FDR over those top-u items. This prevents the FDR from being overly pessimistic when we intentionally pick K too large in our experiments. However, it is unclear what value of to use for ECss that have DR of zero, we exclude these EC's from our FDR analysis.
5

---

Under review as a conference paper at ICLR 2023
Freetion of Utedsppee Covered
Table 2: Average BDM DR and FDR with their standard errors across 100 SpotCheck ECs.
0.8
Spotiggtt Spetlight
0.6
Method
DR
FDR
0.
Barlow
0.43 (0.04)
0.03 (0.01)
6
Spotlight
0.79 (0.03)
(0.01)
Domino
0.64 (0.04)
0.07 (0.01)
Figure The fraction of blindspots covered (with shaded 95% CIs) for blindspots defined using 5, 6, and 7 features using 2 different Spot- light hyperparameter choices.
P Planee PlaneSpot
0.88 (0.03)
0.02 (0.01)
1.0
Trectoon Blindspots Covered
1.0
Rate
0.8
0.8
0.15
PaneSpot
Discovery
06
0.6
0.20
Spotlight
Domino
Bindspots
Bindspots
Blindspot
Figure 3: (Left/Center)) Average BDM DR/FDR (with shaded 95% confidence intervals) for ECs that have 1, 2, and blindspots. (Right) The fraction of blindspots covered, across the individual blindspots from the ECs, for blindspots defined using 5, 6, and features.
Fraction of Blindspots Covered
1.0
1.0
0.8
0.8
0.8
PlaneSpot
Barlow
0.6
0.6
Spotlight
Domino
0.4
0.4
0.4
0.2
0.2
0.2
False
Taise
True
False
True
Blindspot
position"
Blindspot uses "texture"
Blindspot uses "presence of circle"
Figure 4: The fraction of blindspots covered (with shaded 95% confidence intervals) that are or are not defined with (Left) the "relative position" feature (whether the square appears above the centerline of the image), (Center) "texture" feature (whether any object image has vertical stripes), or (Right) the presence of circle (whether there is a circle in the image).
5.1 SYNTHETIC DATA EXPERIMENTS
We use SpotCheck to generate 100 ECs whose datasets have 6-8 semantic features and whose models have 1-3 blindspots defined with 5-7 features. We evaluate three recent BDMs: Spotlight (d'Eon et al., 2021), Barlow (Singla et al., 2021). Domino (Fyuboglu et al., 2022), and PlaneSpot. give all BDMs the positive examples (i.e., images with squares) from the test set and limit them to returning 10 hypothesized blindspots. We use held-out set of 20 ECs tune each BDM's hyperparameters (see Appendix C). We use Ao = A,- 0.8 for our metrics.
Overall Results. Table 2 shows the DR and FDR results averaged across all 100 ECs. We observe that PlaneSpot has the highest DR and that P PlaneSpot and Barlow have a lower FDR than Spotlight and Domino. In Appendix D, we take deeper look at why these BDMs failing and conclude that a significant portion of all of their failures can be explained by their tendency to merge multiple blindspots into single hypothesized blindspot.

---

Under review as a conference paper at ICLR 2023
Identifying factors that influence BDM performance. We study two types of factors: per- configuration factors, which measure properties of the dataset (e.g. how complex is it?) or of the model (e.g., how many blindspots does it have?), and per-blindspot factors, which measure properties of a blindspot (e.g.. is it defined with this feature?). For per-configuration factors, we average DR and FDR across the ECs. For per-blindspot factors, we report the fraction of blindspots covered averaged across each individual blindspot from the ECs (see Equation 5).
The number of blindspots matters. In Figure 3 (Left), we plot the average DR for ECs with 1, and 3 blindspots. Average DR decreases for all methods as the number of blindspots increases. Figure 3 (Center) shows that FDR increases as the number of blindspots increases. Together these observations show that BDMs perform worse in settings with multiple blindspots, which is particularly significant because past evaluations have primarily focused on settings with one blindspot.
The specificity of blindspots matters. In Figure 3 (Right), we plot the fraction of blindspots covered for blindspots defined using 5, 6, and 7 features. With the exception of Spotlight, all of these methods are less capable of finding more specific/less frequently occurring blindspots.
The features that define a blindspot matter. In Figure 4, we plot the fraction of blindspots covered for blindspots that either are or are not defined using various features. In general, we observe that the performance of these BDMs is influenced by the types of features used to define blindspot (e.g., the presence of spurious objects, color texture information, background information). For example, all methods are less likely to find blindspots defined using the "relative position" feature. Interestingly, BDMs that use the model's representation for their image representation (i.e., P PlaneSpot and Spotlight) are less sensitive to the features that define a blindspot than those that use an external model's 's representation (i.e., Barlow and Domino).
Hyperparameters matter. We make two observations about BDM hyperparameters that, jointly, suggest that it is critical that future BDMs provide ways to tune their hyperparameters. First, in Appendix C, we observe that many BDMs have significant performance differences when we vary their hyperparameters. However, hyperparameter tuning harder in real settings than in these experiments because there is no information about the true blindspots to use. Second, in Figure 2, we observe that two hyperparameter settings that perform nearly identically on average exhibit significantly different performance at finding blindspots defined using differing numbers of features. This suggests that there may not be a single best hyperparameter choice to find all of the blindspots in single model, which could contain multiple blindspots of different specificity or size.
5.2 REAL DATA EXPERIMENTS
We demonstrate how to design semi-controlled experiments using the COCO dataset (Lin et al., 2014) to test whether or not some of the findings observed using Check generalize to settings with real data. Despite the fact COCO is a fairly large dataset with extensive annotations, it does not have enough metadata to test all of the findings from Spot Check (e.g., we cannot induce blindspots that depend on the texture of an object). Therefore, we study the following questions using real data:
Are BDMs still less effective for models with more true blindspots?
Are BDMs still less effective at finding more specific blindspots?
Will the 2D image representation used by P PlaneSpot still be effective?
Notice that means we we are interested in two factors: the number of blindspots in a model and of the specificity of those blindspots. Then, in order to try to estimate their influence on BDM performance, we use the same strategy as Spot Check and generate a set of ECs where we randomize these factors independently of one another. We provide details of how we generate these ECs in Appendix E and show an example EC in Figure 5.
However, one key difference from our synthetic experiments is that when we use real data, we cannot guarantee that the model will learn the blindspots that we try to induce in it. Then, if the model fails to learn the intended blindspots in non-random way, the factors that we want to investigate may be correlated and, consequently, their effects may be confounded, which makes it difficult to estimate their individual effects. Table 3 shows that this confounding occurs between the effects of the number and the specificity of the blindspots in the COCO ECs. To eliminate this confounding, we generate a new set of ECs for each factor where we hold the other confounding factor constant.
7

---

Under review as a conference paper at ICLR 2023
blindspot 1: elephants with people
label:
label:
label:
label:
label:
label:
blindspot 2: zebras
Figure 5: Images sampled from an example EC created using COCO. this example, the task is predict detect whether an animal present. The EC has two blindspots: "elephants with people," which is more-specific because it is defined using two objects, and "zebras," which is less-specific because it defined using only one object.
Table 3: The effect of number of blindspots is confounded with the effect of the specificity of those blindspots: as we try to induce more blindspots in the model, the model is less likely to learn more specific blindspots. Overall, the model learned the intended blindspots in 65 of the 90 ECs in the general pool.
Table 4: For the 65 ECs where the model Iearned the intended blindspots, we report BDM DR with its standard error. We only have knowledge of the blindspots that we induced, so the DR is calculated relative to those blindspots and we cannot calculate FDR.
Method
DR
Fraction of successfully induced Blindspots
Barlow
0.09 (0.03)
Num Blindspots
Spotlight
0.14 (0.04)
that are more specific
Domino
0.38 (0.05)
0.54
PlaneSpot
0.48 (0.05)
2
0.47
3
0.40
0.6
0.6
Discovery Rate
PlaneSpot Spotlight Domino
Fraction Blindspots
PlaneSpot
0.4
0.4
Barlow
Spotlight
Domino
0.2
0.2
2 Blindspots
Less
More
Blindspot Specificity
Figure 6: When we condition on only having less specific blindspots, we see that BDM perfor- mance decreases we increase the number of blindspots. These results are based on 82 ECs where the model learned the intended blindspots.
Figure 7: When we condition on only having single blindspot, we see that BDMs are less effec- tive at finding more specific blindspots. These results based on 70 ECs where the model learned the intended blindspots.
We call the set of ECs, where all of the factors are chosen independently. the general pool because covers wider range of possible ECs, which is better for measuring overall BDM performance. We call this new set of ECs the conditioned pool because it is generated by conditioning the distribution used generate the general pool specific confounding factors. Because blindspot discovery is harder with real data, we use more lenient thresholds for our metrics and set A, X = 0.5.
Results. Overall, all of the findings that observed using SpotCheck generalized to these real data experiments. In Figure 6, using the conditioned pool, we observe that BDM performance generally decreases as the number of blindspots increases. Figure 7, using the conditioned pool, we observe that these BDMs are less able to find blindspots that are more specific. In Table 4,
8

---

Under review as a conference paper at ICLR 2023
using the general pool, observe that PlaneSpot is still effective real data. These results may generalize to a wider range of more realistic
provide evidence that results from settings. Additionally, in Appendix F, we include qualitative examples of the 2D representation used by PlaneSpot for models trained on benchmark datasets.
Interpreting these results. While these results are promising. there general challenges associated with running semi-controlled experiments on real data that influence the interpretation of these results (or the results of any similar experiments). At a high-level, these challenges all add noise (or bias) to the results.
First, models trained real data probably have naturally occurring blindspots (i.e., blindspots that we did not intend the model to learn) that may influence the results. One example of how they do this that we do not know how many hypothesized blindspots ask BDM return because do not know how many true blindspots (including both the naturally occurring ones and the ones we induced) a model has.
Second, the chosen BDM hyperparameters are probably sub-optimal because we cannot tune them for a specific EC. Because the set of ECs from the synthetic data experiments is relatively homogeneous, it is reasonable to use a single set of hyperparameters for all of those ECC.. However, this is not the case for our COCO experiments.
Third, the results may be biased in potentially unknown ways towards certain BDMs whenever some of the models in the set of ECs fail to learn the intended blindspots. For example, our general pool of ECs contains more less-specific blindspots than more-specific blindspots, which will bias the results towards BDMs that are more effective at finding less-specifi blindspots. Further, there are probably other biases that we cannot identify because we lack the metadata needed to measure them.
Fourth, there can be false positives in verifying that model actually learned an intended blindspot. This is because there could be sub-blindspot (that we may not have the metadata define) within that intended blindspot that is actually causing the model to perform significantly worse on the intended blindspot.
CONCLUSION
In this work, we introduced SpotCheck, a synthetic evaluation framework for BDMs, and P PlaneSpot, a simple BDM that uses a 2D image representation. Using SpotCheck, we identified factors that influence BDM performance and two technical challenges for future work on developing new BDMs to address (i.e., providing a practical way to tune BDM hyperparameters and preventing BDMs from merging multiple true blindspots into a single hypothesized blindspot). We found that P PlaneSpot outperforms existing BDMs on both synthetic and real data.
Beyond the insights made in the experiments that we ran, we believe that our demonstrated evaluation workflow has laid further groundwork for more rigorous science of blindspot discovery. Researchers interested in benchmarking their own BDM or answering scientific questions about blindspot dis- covery first use SpotCheck (or harder variations of it) run controlled experiments using synthetic data. Next, researchers can also set up semi-controlled experiments using real data. While both harder and noisier (in part, because of the challenges that we identified), these experiments are also more realistic. A natural open direction for future work is how to evaluate BDMs in more realistic settings where we have no knowledge of the model's true blindspots.
We note that this workflow is particularly well suited to identifying factors that influence BDM performance and that understanding these factors can benefit both researchers and practitioners using BDMs. For researchers, understanding these factors is essential to running fair and informative experiments (e.g., only evaluating BDMs in settings that only have one blindspot, which we have identified as being easier, may misrepresent how BDMs perform in more realistic settings). For practitioners, understanding these factors may help them use any prior knowledge they may have to select an effective BDM for their problem. Two potential factors that future work could consider are: First, how does the methodology used to induce blindspots (i.e., training with mislabeled images) influence BDM performance? Second, for models trained real data, how different are the naturally occurring blindspots from those that we have the metadata to artificially induce and how do those differences influence BDM performance?
9

---

Under review as a conference paper at ICLR 2023
REFERENCES
Julius Adehayo, Michael Muelly, Harold Abelson, and Been Kim. Post hoc explanations may be ineffective for detecting unknown spurious correlation. In International Conference on Learning Representations, 2022. URL https: wwt.oopeeevviiiie rewie..eeeww..... net/ forum? 1==i=dd ott.eeeee...o....o.ooooooooCCoo.
David Alvarez-Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. pp. 7786-7795, 2018.
Agathe Balayn, Natasa Rikalo, Christoph Lofi, Jie Yang, and Alessandro Bozzon. How can ex- plainability methods be used to support bug identification in computer vision models? In CHI Conference on Human Factors in Computing Systems, pp. 1-16, 2022.
Homanga Bharadhwaj, De-An Huang, Chaowei Xiao, Anima Anandkumar, and Animesh Garg. Auditing ai models for verified deployment under semantic specifications. arXiv preprint arXiv:2109, 12456, 2021.
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pp. 77-91. PMLR, 2018
Ángel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, and Duen Horng Chau. Fairvis: Visual analytics for discovering intersectional bias in machine learning. In 2019 IEEE Conference Visual Analytics Science and Technology (VAST), pp. 46-56. IEEE, 2019.
Yeounoh Chung, Tim Kraska, Neoklis Polyzotis, Ki Hyun Tae, and Steven Euijong Whang. Slice finder: Automated data slicing for model validation. In 2019 IEEE 35th International Conference on Data Engineering (ICDE), pp. 1550-1553. IEEE, 2019.
Greg d' Eon, Jason d' Lon, James R Wright, and Kevin Leyton-Browe The spotlight: general method for discovering systematic errors in deep learning models. arXiv preprint arXiv:2107.00758, 2021.
Jiarui Ding, Anne Condon, and Sohrab P Shah. Interpretable dimensionality reduction of single cell transcriptome data with deep generative models. Nature communications, 91)::1113, 2018 2018.
Sabri Eyuboglu, Maya Varma, Khaled Kamal Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Re. Domino: Discovering systematic errors with cooss-modal embeddings. In International Conference on Learning Representations, 2022. URL https: openreview net / forum I 1 jXN.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 770-778, 2016.
Fred Hohman, Hackyu Park, Caleb Robinson, and Duen Horng Polo Chau. Summit: Scaling deep learning interpretability by visualizing activation and attribution summarizations. IEEE transactions on visualization and computer graphics, 26(1): 26(1):1096-1) 1166, 2019.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference machine learning, pp. 2668-666822 2676. PMLR, 2018.
Joon Sik Kim, Gregory Plumb, and Ameet Talwalkar. Sanity simulations for saliency methods. ICML, 2022.
Michael P Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on Al, Ethics, and Society, pp. 247-254, 2019.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pp. $338-5345 PMLR, 2020.
10

---

Under review as a conference paper at ICLR 2023
Guillaume Leclere, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang, et al. 3db: A framework for debugging computer vision models. arXiv preprint arXiv:2106.038800 2021.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer viston, pp. 740-755. Springer, 2014.
Usman Mahmood, Robik Shrestha, David DB Bates, Lorenzo Mannelli, Giuseppe Corrias, Yusuf Emre Erdi, and Christopher Kanan. Detecting spurious correlations with sanity tests for artificial intelligence guided radiology systems. Frontiers in digital health, pp. 85, 2021.
Luke Oakden- Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, and Christopher Ré. Hidden stratification causes clinically meaningful failures machine learning for medical imaging. In Proceedings of the ACM conference health, inference, and learning, pp. 151-159, 2020.
Gregory Plumb, Marco Tulio Ribeiro, and Ameet Talwalkar. Finding and fixing spurious pat- terns with explanations. Transactions on Machine Learning Research, 2022. URL https: //opereeview net forumrrrmiooo dwt.mm/iwdwhhuuhmSS
Marco Tulio Ribeiro and Scott Lundberg. Adaptive testing and debugging of nlp models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2022.
I: Long Papers), pp.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https:
// //openreviww.... net. forum? perrrmmid=ryyyuudd d=rryGuJ==VV..
Rakshith Shetty, Bernt Schiele, and Mario Fritz. Not using the car to see the sidewalk - quantifying and controlling the effects of context classification and segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, and Deepti Ghadiyaram. Don't judge an object by its context: Learning to overcome contextual bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pullern Recognition, pp. 11070-11077 2020.
Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, and Eric Horvitz. Understanding failures of deep networks via robust feature extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 12853-12862, 2021.
Sumedha Singla, Brian Pollack, Junxiang Chen, and Kayhan Batmanghelich. Explanation by progressive exaggeration. In International Conference on Learning Representations, 2020. URL https: /ttt:s:// net forum? dddi Hidd-HHxxxxFFFF.
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse- coarse-grained classification problems. Advances in Neural Information Processing Systems, 33:19339-1993 19352, 2020.
Julia K Winkler, Christine Fink, Ferdinand Toberer, Alexander Enk, Teresa Deinlein, Rainer Hofmann- Wellenhof, Luc Thomas, Aimilios Lallas, Andreas Blum, Wilhelm Stolz, et al. Association between surgical skin markings in dermoscopic images and diagnostic performance of deep learning convolutional neural network for melanoma recognition. JAMA dermatology, 155(10): 11:5-1111:1111,, 2019.
Eric Wong, Shibani Santurkar, and Aleksander Madry. Leveraging sparse linear layers for debuggable deep networks. In Marina Meila and Tong Zhang (eds.). Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 11205-11216. PMLR, 18-24 Jul 2021. URL ht.ps: /ttp///oocceeccc/ooo hgs.ml. w...mr.pprre. w.w.pprrs////.. ppppsss/v39) wong2lb. html.
Kai Yuanqing Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise signal: The role of image backgrounds in object recognition. In International Conference on Learning Representations, 2021. URL https: opwnrevveeeeeeee/////p//e net www.eetttttiieeeeeeeeeee 3D-xY7wT.q.
11

---

Under review as a conference paper at ICLR 2023
Chih- Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-awatt concept- explanations in deep neural networks. Advances in Neural Information Processing Systems, 33:20554-205666 2020.
12


Reviews:
[{"review_id": "yDyR704Fykx", "reviewer": "Reviewer_w8rw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors first propose a formalization of the Blindspot Discovery problem, which they claim was not previously done in a clear manner, and then proceed to propose a synthetic image benchmark for evaluating blindspot discovery methods, that they call SpotCheck. Furthermore they propose a novel blindspot discovery method, called PlaneSpot which outperforms existing methods. Finally, the authors evaluate existing methods as well as their proposed method on real-data sources and find that the results of their proposed synthetic benchmark SpotCheck, seem to correlate with the results of the benchmarks sourced from real datasets. They therefore conclude that their benchmark can be a good complementary and thorough way of evaluating BDMs. ", "summary_of_the_review": "While I am expecting the authors to provide me with more information wrt to the weaknesses I outlined above, I also believe that the work represents a good step forward for BDMs, both in terms of better formalization and benchmarks as well as models. ", "strength_and_weaknesses": "Strengths: \n\nClear and concise writing.\nThe work attempts to provide a more clear formalization of Blindspot Discovery problem and they do so well. \nThe authors motivate the need for a more ‘controllable’ benchmark and provide one in a satisfactory way. \nGood empirical evaluation of the new proposed model.\n\nWeaknesses:\n\nWhile the proposed benchmark is a good step forward, I do feel that the paper suffers from some ‘overclaiming’, in the sense that, even their proposed benchmark may have ‘naturally’ occurring blindspots. For example, the model itself may have blindspots when the data presents more than two objects, or perhaps when it presents two objects that are on the right hand side of the square, or on the top left corner of the square. These are blindspots that do not exist within the authors’ predefined schema for what label 0 and 1 belong to, but the model may none-the-less develop sensitivities to them. Frankly I think, trying to create a ‘perfect’ synthetic dataset for blindspot discovery can be a good way to create a blindspot for a research project. In the sense that perhaps a better way to frame the work would be an attempt to create a synthetic dataset with more specific and controllable attributes. However, and this leads to the second weakness.\nIt’s not clear what the difference between existing benchmarks on real world data, that have previously discovered or defined blindspots, and the proposed synthetic dataset are, since even the synthetic dataset can’t be said to not have ‘naturally’ occurring blindspots that the authors can’t predict. A better way to quantify or justify this is necessary I believe. \nIt’s not clear what the exact usefulness of the new benchmark is. Is it cheaper to evaluate on? Or is it the fact that it has more ‘controllable’ blindspots? If so, the above concerns must be addressed. Also, given the correlation between the real world datasets and the results from your synthetic dataset, wouldn’t you say that perhaps one could omit using SpotCheck and simply use the real world datasets?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "qNmdV5kGD2", "reviewer": "Reviewer_pnqW", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses the issue of Blindspot Discovery Methods (BDM), which is the task of finding semantically meaningful subsets of data that significantly degrade the performance of image classifiers. This paper proposes SpotCheck as an evaluation framework using synthetic data and PlaneSpot as a new method. Experiments are also conducted on real data sets.", "summary_of_the_review": "Overall, the reviewer leans toward rejecting this paper. A response from the authors regarding the above weaknesses could improve the score.", "strength_and_weaknesses": "Strengths\n1. The definition of BDM and SpotCheck is clear.\n1. A simple BDM, PlaneSpot, is proposed using dimensionality reduction and Gaussian Mixture Model.\n1. Experimental results on both synthesized and real datasets.\n\nWeaknesses\n1. Motivation to use scvis (Ding et al., 2018) as a dimensionality reduction method is unclear. There are many methods, such as PCA, LLE, t-SNE, and UMAP. The reviewer would like to know why scvis is selected.\n1. It is also unclear how Ψ is generated using PlaneSpot. It seems clusters found by Gaussian Mixture Model are used for Ψ, but its particular definition is not described.\n1. While the reviewer can agree that PlainSpot has the best accuracy on synthetic data, Domino sometimes outperforms PlainSpot on real data. This result is not merely a slight concern to say that PlainSpot is the best method among the other methods but also suggests that the evaluation of BDM by synthetic data proposed as SpotCheck may not correlate very strongly with the result using real data. In fact, there is also a discrepancy about the superiority of the methods, including Barlow and Spotlight, between the artificial data and the actual data.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "e9QoeGw5QN", "reviewer": "Reviewer_X8MU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper puts forth a methodology to understand the underlying factors (in the model and/or data) that affect blindspot detection methods. They rely on synthetic data wherein the number and nature of blindspots can be controlled. They also propose a new blindspot detection approach, PlaneSpot.", "summary_of_the_review": "Overall, the analysis in the paper and its connections with blindspots in practical models on large-scale datasets is not yet sufficiently compelling. Therefore, I recommend rejection.", "strength_and_weaknesses": "Blindspot detection is an important problem and understanding how to quantify the effectiveness of different approaches and identify the factors that influence them in real datasets could be valuable. \n\nHowever, there seems to be a huge gap between the simplistic settings the authors study in this paper and real-world datasets. That would still be ok if the authors showed how their analysis could help improve blindspot detection in more realistic settings (eg, on ImageNet). As of now, the paper just reads as an analysis of a simple setting that may or may not be reflective of practice. As the authors themselves mention in the paper, it is not clear whether model/BDM trends on artificially induced blindspots are predictive of trends on naturally occurring ones.\n\nMore specific comments/questions:\n\n- In general, in the results in Figures 3, 4 and 6, one thing that is not clear is how sensitive the model is actually to these blindspots. In particular, an important quantity to visualize is model performance on blindspot datasets as the number of blindspots/specificity, etc. is varied. After all, just because the dataset has a blindspot doesn’t mean the model learns it. It is entirely possible that as the number of blindspots increase, the model tend to not learn all of them. This is a key confounder in the results that must be ablated.\n- What exactly are the features in the synthetic dataset?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "ICoR5D11p7", "reviewer": "Reviewer_E6Ce", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": " In this work the authors give a framework for evaluating blindspot detection methods (BDMs). The authors define BDMs as methods taking labeled examples and a classifier, and outputting \"semantically meaningful\" groups of points. The authors introduce a new BDM, and a toy evaluation setting in which they perform image classification under a known distribution of images and a known set of \"blind spots.\" The introduced method, PlaneSpot, outperforms the other compared-with methods in both this toy setting and in a more realistic setting.", "summary_of_the_review": "The paper studies an important problem, but does not have experimentally convincing results.", "strength_and_weaknesses": "The paper tackles an import problem: providing a taxonomy and evaluation framework for these \"blind spot detection\" methods. However, the work suffers from a number of issues around the completeness of the real world experiments and clarity. \n\nReal world experiments: All of the experiments are synthetic. Even on the real world experiments, the authors only evaluated on synthetic blindspots that are not representative of conditions in the real world. While this is an interesting experiment, the authors must go beyond these synthetic experiments to identify naturally arising blindspots in models --- otherwise it is not clear whether the presented method is actually useful. The approaches to doing so are relatively straightforward: the authors could\n* evaluate if identified blindspots match known blindspots of examples (for example, on ImageNet humans can lead to fish), or\n* investigate identified blindspots (test whether or not these blindspots actually matter with counterfactuals or another.\n\nClarity:\n* Missing definition of semantic meaningfulness: what makes a blindspot semantically meaningful? This is a critical question that defines the conceptual objects investigated in this paper. What is to stop one from outputting all the incorrect examples as a subpopulation? After all, incorrect examples are semantically meaningful as a unit in that they are all incorrect.\n* Notation: The paper uses a large amount of notation. There are many symbols, acronyms, and named methods that clutter the messages that the paper tries to get across. For example, Table 2.\n* Confidence intervals: Not defined; are they bootstrapped?\n* Figure 7: What does the x axis mean here? This specificity should be a number value.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject