PRIME: Prioritizing Interpretability in Failure Mode Extraction
Authors: Keivan Rezaei
Mehrdad Saberi
Mazda Moayeri
Soheil Feizi
Venue: ICLR 2024 poster (2024)
Keywords: interpretability
failure modes
bias
Primary area: visualization or interpretation of learned representations
TL;DR: We propose a new method to detect and explain failure mode of trained models in human-understandable terms.

Abstract: In this work, we study the challenge of providing human-understandable descriptions for failure modes in trained image classification models.
Existing works address this problem by first identifying clusters (or directions) of incorrectly classified samples in a latent space and then aiming to provide human-understandable text descriptions for them.
We observe that in some cases, describing text does not match well
with identified failure modes, partially owing to the fact that shared interpretable attributes of failure modes may not be captured using clustering in the feature space.
To improve on these shortcomings, we propose a novel approach that prioritizes interpretability in this problem: we start by obtaining human-understandable concepts (tags) of images in the dataset and
then analyze the model's behavior based on the presence or absence of combinations of these tags.
Our method also ensures that the tags describing a failure mode form a minimal set,
avoiding redundant and noisy descriptions.
Through several experiments on different datasets, we show that our method successfully identifies failure modes and generates high-quality text descriptions associated with them.
These results highlight the importance of prioritizing interpretability in understanding model failures.

Published as a conference paper at ICLR 2024
PRIME: PRIORITIZING INTERPRETABILITY IN FAIL- URE MODE EXTRACTION
Keivan Rezaci** Mehrdad Saberi Mazda Moayeri', Soheil Feizi" Depaarmment of Computer Science, University of Maryland fkrezaei, msaberi, masberr,,moayeri. sfeizi @umd.
ABSTRACT
In this work, we study the challenge of providing tions for failure modes in trained image classification models. Existing works address this problem by first identifying clusters (or directions) of incorrectly clas- sified samples in a latent space and then aiming to provide duman-understandable text descriptions for them. We observe that in some cases, describing text does not match well with identified failure modes, partially owing to the fact that shared interpretable attributes of failure modes may not be captured using clus- tering in the feature space. To improve on these shortcomings, we propose a novel approach that prioritizes interpretability in this problem: taining numan-understandahle concepts (tags) of images in the dataset and then analyze the model's behavior based on the presence or absence of combinations of these tags. Our method also ensures that the tags describing a failure mode form a minimal set, avoiding redundant and noisy descriptions. Through several experiments on different datasets, we show that our method successfully iden- tifies failure modes and generates high-quality text descriptions associated with them. These results highlight the importance of prioritizing interpretability in un- derstanding model failures.
descrip-
start by ob-
INTRODUCTION
plethora of reasons (spurious correlations, imbalanced data, corrupted inputs, etc.) may lead a model to underperform are challenging to identify due to the black-box nature of deep models, and further, they are often obfuscated by common metrics like overall accuracy, leading false these failures can have significant real-world consequences, such as perpetuating algorithmic bias (Buolamwini & Gebru 2018) or unexpected catastrophic failure under distribution shift. Thus, the discovery and description of failure modes is crucial in building reliable AI, as we cannot fix problem without first diagnosing it.
specific subpopulation; we term this failure mode. Failure modes
of security. However,
Detection of failure modes or biases within trained models has been studied in the literature. Prior work (Tsipras et al., [2020] Vasudevan et al. 2022) requires humans in the loop to get sense of biases or subpopulations on which a model underperforms. Some other methods (Sohoni et al. 2020b Nam et al. 2020; Kim al., 2019 Liu al., (02T) do the process of capturing and intervening in hard inputs without providing human- unddrsstnnndd descriptions for challenging subpopulations.
Providing uuman-understandable and interpretable descriptions for failure modes not only enables humans to easily understand hard subpopulations, but enables the use of text-to-image methods
(Ramesh et al. 2022. Rombach et al. 2022 Saharia et al., [0022] Kattakinda et al.) 2022] to generate relevant images corresponding to failure modes improve model's Ii's accuracy over them.
Recent work (Eyuboglu et al. 2022 Jain et al. 2022 Kim et al. 2023 E'Enn et al. 2021 takes an important step in improving failure mode diagnosis by additionally finding natural language de- scriptions of detected failure modes, namely via leveraging modern llaguanla models. These methodologies leverage the shared vision- languaee latent space, discerning intricate clusters di- rections within this space, and subsequently attributing human-comprehensible descriptions to them.
"Equal contribution.

---

Published as a conference paper at ICLR 2024
fox (81.96% 96%)
fox white z00 (35.29%)
fax
grass
stand
field + dry (378
Figure 1: Visualization of two detected failure modes of class "fox" on a model trained on Living Overall accuracy for images of class "fox" is 81.96%. However, we identify two coherent subsets of images with significant accuracy drops: foxes standing in dry grass fields (47.83) (47.83% accuracy) and foxes in a zoo where a white object (fox or other objects) is detected 295. accuracy), See Appendix A.4 for more examples.
However, questions have been raised regarding the quality of the generated descriptions, i.c., there need to ascertain whether the captions produced genuinely correspond to the images within the identified subpopulation. Additionally, it is essential to determine whether these images convey shared semantic attributes that can be effectively articulated through textual descriptions.
In this work, we investigate whether not the latent representation space is a good proxy for seman- tic space. In fact, we consider two attributed datasets: CelebA et al. 2015) and CUB-2001 (20(@0 et al. (01T)) and observe that samples sharing many semantic attributes may indeed lie far away in latent space, while nearby instances may not share any semantics (see Section 5.2). Hence, exist- ing methods may suffer from relying on representation space clusters and directions found in this space may contain images with different semantic attributes leading to less coherent descriptions.
Inspired by this observation and the significance of faithful descriptions, we propose PRIME. In this method, we suggest to reverse the prevailing paradigm in failure mode diagnosis. That is, we put interpretability first. our method, we start by obtaining human- uumman-underttadaalle con- cepts (tags) of images using pre-trained tagging model and examine model's behavior condi- tioning on the presence or absence of combination of those tags. In particular, we consider different groups of tags and check whether (1) there is significant drop in model's accuracy over images that represent all tags the group and (2) that group is minimal, i.c. images hav- ing only some of those tags are easier images for the model. When group of tags satisfies both of these conditions, we identify as a failure mode which can be effectively described by these tags. Figure 2 shows the overview of our approach and compares it with existing methods.
As example, by running PRIME trained model Li, we realize that images where a black ape hang- ing from a tree branch identify hard subpopulation such that model's accuracy drops from 86. 23% to 41.88%. Crucially, presence of all 3 of these tags is necessary, i.e., when we con- sider images that have or 2 of these 3 tags, the accuracy of model higher. Figure 3 illustrates these failure modes. We further study the effect of number of tags in Section 5.1
Existing Methods
(T Fomaciig Fallre Moda
(T)) Incerpeting Fallare Stedo
Guegage gyace
PRIME
Interpestable
Modec
To further validate our method, we examine data unseen during the computation of our failure mode descriptions. We observe that the images that match failure mode lead the model to similarly struggle. That is, demonstrate generalizability of our failure modes, crucially, directly from the succinct text descriptions. While reflecting the quality of our descriptions, this allows for bringing in generative models. We validate this claim by generating hard images using some of the failure mode's 's descriptions and compare the accuracy of model on them with some other generated images that correspond easier subpopulations.
Figure PRIME illustration.
2

---

Published as a conference paper at ICLR 2024
(86.23%)
ape hang (70.09%)
age black (73.21%)
ape
branch (69.23%)
upe + hang branch black (41 88%)
All Tugs
Figure 3: Although appearance tags "hang", "black", and "branch" individually lowers model's accuracy, when all of them appear in the images, model's accuracy drops from 86.23% to 41.88%.
Finally, we show that PRIME produces better descriptions for detected failure modes in terms of similarity, coherency, and specificity of descriptions, compared to prior work that does not prior- itize interpretability. Evaluating description quality is challenging and typically requires human assessment, which can be impractical for extensive studies. To mitigate that, inspired by CLIPScore (Hessel al. 2021), present suite of three automated metrics that harness vision-language models to evaluate the quality. These metrics quantify both the intra-group image-description simi- larity and coherency, while also assessing the specificity of descriptions to ensure they are confined to the designated image groups. We mainly observe that due to putting interpretability first and considering different combinations of tags (concepts), we observe improvements in the quality of generated descriptions. We discuss PR PRIM''s limitations in Appendix A.1
Summary of Contribution.
1. We propose PRIME extract and explain fallure modes of model in human- understandable terms by prioritizing interpretability.
2. Using suite of three automated metrics to evaluate the quality of generated descriptions, we observe improvements in our method compared to strong baselines such as Eyuboglu ct al. (2022) and et al. (2022) on various datasets.
3. We advocate for the concept of putting interpretability first by providing empirical evidence derived from latent space analysis, suggesting that distance in latent space may at times be a misleading measure of semantic similarity for explaining model failure modes.
2 LITERATURE REVIEW
Failure mode discovery. The exploration of biases or challenging subpopulations within datasets, where a model's performance significantly declines, has been the subject of research in the field. Some recent methods for detecting such biases rely on human intervention, which can be time- consuming and impractical for routine usage. For instance, recent works (Tsipras et al. 2020) Va- sudevan et al. 2022) depend on manual data exploration to identify failure modes in widely used datasets like ImageNet. Another line of work uses crowdsourcing (Nushi et al. 2018 Idrissi et al.) 2022 Plumb et al.) 2021) or simulators (Leclere et al.) 2022) to label visual features, but these methods are expensive and not universally applicable. Some researchers utilize feature visualization (Engstrom et al.] 2019 Olah et al. 2017) or saliency maps (Selvaraju et al.) 2017,, AAdeaay et al.) 2018) to gain insights into the model's failure, but these techniques provide information specific to
3

---

Published as a conference paper at ICLR 2024
individual samples and lack aggregated knowledge across the entire dataset. Some other approaches
(Sohoni et al. 2020a Nam et al. 2020 Liu et al. 2021 Hashimoto et al. 2018) automatically
identity failure modes of model but do not provide human- lumman-udderttaddallee descriptions for them.
Recent efforts have been made to identify difficult subpopulations and assign suman-understandabbl
descriptions to them (Eyuboglu et al. 2022 Jain al. 2022 Kim al. 2023). DOMINO (Eyuboglu
model to cluster difficult these clusters. (Jain et al. (2022) identifies a failure direction in the latent space and assigns description to images aligned with that direction. Hoffmann et al. (202I) shows that there semantic gap between similarity in latent space and similarity in imput space, which can corrupt the output of methods that rely on assigning descriptions to latent embeddings. In (Kim al., 2023), concepts are identified whose presence in images leads to a substantial decrease in the model's ss accuracy. Recent studies (Johnson et al. [023] Gao et al. 2023) highlight the challenge of producing high-quality descriptions in the context of failure mode detection.
eet al. 2022 uses the latent representation of images in a vision-lang
images and then assigns human-unde undraan-nnnrtt descriptions
Vision-Language and Tagging models. Vision-language models have achieved remarkable success through pre-training large-scale image-text pairs (Radford et al.) 202T). These models can be utilized to incorporate vision-language space and evaluate captions generated to describe images. Recently Moayeri et al. (2023) Li ct al. (2023) bridge the modality gap and enable
shelf vision encoders to access shared vision-language space. Furthermore, in our method, we utilize
models capable of generating tags for input images (Huang al. 2023 Zhang al. 2023)
3 EXTRACTING FAILURE MODES BY CONDITIONING ON HUMAN-UNDERSTTANDABLE TAGS
Undesirable patterns or spurious correlations within the training dataset can lead to performance discrepancies in the learned models. For instance, in the Waterbirds dataset (Sagawa et al.) 2019). images of landbirds are predominantly captured in terrestrial environments such as forests or grass- lands. Consequently. model can heavily rely on the background and make a prediction based on that. Conversely, the model may also rely on cues such as the presence of the ocean, sea, or boats to identify the input as waterbirds. This can result in performance drops for images where water- bird is photographed on land or landbird is photographed at sea. Detecting failure modes involves identifying groups of inputs where the model's performance significantly declines. While locating failure inputs is straightforward. categorizing them into distinct groups characterized by human- understandable concepts is challenging task. To explain failure modes, we propose PRIME. Our method consists of two steps: (I) obtaining relevant tags for the images, and (II) identifying failure modes based extracted tags.
3.1 OBTAINING RELEVANT TAGS
We start our method by collecting concepts (tags) over the images in the dataset. For example, for a photo of a fox sampled from ImageNet (Deng et al. 2009), we may collect tags "'rangee,, "grass", "trees", "walking", "zoo", and others. To generate these tags for each image in our dataset, we employ the state-of-the-art Recognize Anything Model (RAM) (Zhang et al. [023] Huang et al., 2023), which is a model trained on image-caption pairs to generate tags for the input images. RAM makes a substantial step for large models in computer vision, demonstrating the zero-shot ability to recognize any common category with high accuracy.
Let D be the set of all images. We obtain tags over all images of D. Then, we analyze the effect class-wise manner. In fact, the effect of tags and patterns on the model's prediction depends on the main object in the images, e.g., presence of water in the background improves performance on images labeled as waterbird while degrading performance on landbird images. For each class in the dataset, we take the union of tags generated by the model over images of class c. Subsequently, we eliminate tags that occur less frequently than a predetermined threshold. This threshold varies depending the dataset size, specifically set at 50, 100, and 200 in our experimental scenarios. In fact, we remove rare (irrelevant) tags and obtain a set of tags T, for each class e in the dataset, e.g., T. ("red", "orange", "snow", "grass" :..].
of tags on prediction
4

---

Published as a conference paper at ICLR 2024
3.2 DETECTING FAILURE MODES
After obtaining tags, we mainly focus on tags whose presence in the image leads to performance drop in the model. Indeed, for each class e, we pick a subset S. C To of tags and evaluate the model's performance on the images of class e including all tags in Se. We denote this set of images by Is.- Ise is a coherent image set in the sense that those images share at least the tags in S..
For Is.. to be failure mode, we require that the model's 's accuracy over images of Is, significantly drops, i.e., denoting the model's accuracy over images of Is, by As, and the model's overall ac- Parameter plays pivotal role in
curacy over the images of class by Ac. then As; I A. determining the severity of the failure modes we aim to detect. Importantly, we want the tags in S. to be minimal, i... none of them should be redundant. In order to ensure that, expect that the removal of any of tags in S. determines a relatively easier subpopulation. In essence, presence of all tags in S. is deemed essential to obtain that hard subpopulation.
More precisely, Let n to be the cardinality of Se, i.e., n = - We require all tags E S. to be necessary. i.e., if we remove a tag t from S., then the resulting group of images should become an easier subpopulation. More formally, for all & E S., As, 2 As. + bn where b2, ba, b4, are some hyperparameters that determine the degree of necessity of appearance of all tags in a group. We generally pick by = 10%, by = 5% and by = 2.5% in our experiments. These values help us fine- tune the sensitivity to tag necessity and identify meaningful failure modes. Furthermore, we require a minimum of s samples in Is. for reliability and generalization. This ensures a sufficient number of instances where the model's performance drops, allowing us to confidently identify failure modes. Figure I shows some of the obtained failure modes.
How to obtain failure modes. We generally use Exhaustive Search to obtain failure modes. In exhaustive search, we systematically evaluate various combinations of tags to identify failure modes, employing a brute-force approach that covers all possible combinations of tags up to I ones. More precisely, consider all subsets Se C Te such that [Sel ( and evaluate the model's performance on Ts.. As mentioned above, we detect S. as a failure mode if (1) Ts. 2 8, (2) model's accuracy over Ts. is at most A, a, and (3) S. is minimal, i.e., for all t Se, \ 2 As, + bbjs.j- It is worth noting that the final output of the method is all sets 1s. that satisfy those conditions and description for this group consist of class name (c) and all tags in S..
We note that the aforementioned method runs with a complexity of O (IT.)'ID). However, / is generally small, i.e., for a failure mode to be generalizable, we mainly consider cases where / < 4. Furthermore. in our experiments over different datasets [Tel R 100, thus, the exhaustive search is relatively efficient. For instance, running exhaustive search (1 4, s 30, a 30%) on Liv- ing17 dataset having 17 classes with 88400 images results in obtaining 132 failure modes within a time frame of under minutes. We refer to Appendix A.6 for more efficient algorithms and Ap- pendix A.15 for more detailed explanation of PRIME's hyperparameters.
Experiments and Comparison to Existing Work. We run experiments on models trained on Liv- i7 NonLiving22 Entity 13 (Santurkar et al. 2020). Waterbirds (Sagawa et al. 2019). and CelebA [Liu et al., 2015) (for age classification). We refer to Appendix A.. for model training details and the different hyperparameters we used for failure mode detection. We refer to Appendix AA.3 for the full results of our method on different datasets. We engage two of the most recent failure mode detection approaches DOMINO(Eyuboge) et al. [022] and Distilling Failure Directions al.,
2022) as strong baselines and compare our approach with them.
EVALUATION
Let D be the dataset on which we detect failure modes of a trained model. The result of a uamaan--nddrtanddalle failure mode extractor this dataset consists of sets of images, denoted as 11, .2., Im- along with corresponding descriptions, labeled as Ti, T2, ..., Tmm Each set I, com- prises images that share similar attributes, leading to a noticeable drop in model accuracy. Number of detected failure modes, m, is influenced by various hyperparameters, e.g., in our method, mini- mum accuracy drop (a), values for b2, by. .... and the minimum group size (s) are these parameters.
One of the main goals of detecting failure modes in uumaannnddestaadaalle terms is to generate high-quality captions for hard subpopulations. We note that these methods should also be evaluated
5

---

Published as a conference paper at ICLR 2024
Eaxy
gray water
Acc: 98%
close-up blue
Acc: 98%
red lay stare
Aoc: 44%
photo of gray bear in water
close-up photo of blue parrot
photo of red fox laying and staring the camera
Hard
Acc 8%
Ace: 38%
Acc: 2%
cub climbing
elimbing
white parrot grass
photo fox behind fence
Figure 4: Accuracy of model over 50 generated images corresponding one the success modes and failure modes for classes "bear", "paroot" and "fox" from Living Accuracy gap shows that our method can identify hard and easy subpopulations. Images show that extracted tags are capable of describing detailed images.
in terms of coverage, i.e., what portion of failure inputs are covered along with the performance gap in detected failure modes. All these methods extract hard subpopulations which the model's accuracy significantly drops, and coverage depends on the dataset and the hyperparameters of method, thus, mainly focus generalizability of our approach and quality of descriptions.
4.1 GENERALIZATION ON UNSEEN DATA
In order to evaluate generalizability of the resulted descriptions. we take dataset D' including un- seen images and recover relevant images in that to each of captions Ti, T2. ....Tm thus, obtaining 1, 12 .... T Indeed, I includes images in DD' that are relevant to Tj. 3- If captions can describe hard subpopulations, then we expect hard subpopulations in I I2 .... I',. Additionally, since D and D' share the same distribution, we anticipate the accuracy drop I to closely resemble that lz.
In our method, for a detected failure mode Is., we obtain I's by collecting images of D' that have all tags in S.- For example, if appearance of tags "black", "snowing", and "forest" is detected as a failure mode for class "bear". we evaluate model's performance on images of "bear" D' that include those three tags, expecting a significant accuracy drop for model seen Figure 9 PRIME shows a good level of generalizability. We refer to Appendix A.5) for generalization other datasets with respect to different hyperparameters (s and a.). While all our detected failure modes generalize well, we observe stronger generalization when using more stringent hyperparameter values (high and a), though it comes at the cost of detecting fewer modes.
those images.
In contrast, existing methods (Eyuboglu et al. 2022 Jain et al., 2022) do not provide a direct way to assess generalization from text descriptions alone. See Appendix A.9 for more details.
4.2 GENERALIZATION ON GENERATED DATA
In this section we validate PRIME on synthetic images. To utilize image generation models, we employ language models to create deseriptive captions for objects and tags associated with failure modes in images. We note that use of language models is just for validation on synthetic images, not part of PRIME framework. We discuss about limitations of using language models in Appendix A.I These captions serve as prompts for text-to-image generative models, enabling the creation of artificial images that correspond to the identified failure modes. To achieve this, we adopt the methodology outlined in Vendrow et al. (2023), which leverages denoising diffusion model (Ho et al.) 2020) Rombach et al.) (022). We fine-tune the generative model on the Living17 dataset generate images that match the distribution of the data that the classifiers is trained

---

Published as a conference paper at ICLR 2024
For each class in Living17 dataset, we apply our approach to identify two failure modes (hard sub- populations) and two success modes (easy subpopulations). We then employ ChatGPT to generate descriptive captions for these groups. Subsequently, we generate 50 images for each caption and assess the model's 's accuracy on these newly generated images. We refer to Appendix A.10 for more details on this experiment and average discrepancy in accuracy between the success modes and fail- ure modes which further validates PRIME. Figure 4 provides both accuracy metries and sample images for three hard and three easy subpopulations.
4.3 QUALITY OF DESCRIPTIONS
Within this section, our aim is to evaluate the quality of the descriptions for the identified failure modes. In contrast to Section 4.2, where language models were employed to create sentence de- scriptions using the tags associated with each failure mode, here we combine tags and class labels in a bfg-of-wordds manner. For instance, when constructing the description for failure mode in the "ape" class with the tags "black" + "branch," " we formulate it as "a photo of ape black branch". We discuss more about it in Appendix A.
In order evaluate the quality of descriptions, we propose suite of three complementary auto- mated metrics that utilize vision-language models (such as CLIP) as a proxy to obtain image-text similarity (Hessel al. 2021). Let t be the failure mode's description, fix denote the normal- ized embedding of text prompt & and fvision (x) denote the normalized embedding of an image x. The similarity of image x to this failure mode's description t the dot product of image and text representation in shared vision-language space. More precisely, sim(x, t) = (fvision (z), Fiea (t)).
For high-quality failure mode I, and its description T,. we wish Ty to be similar to images in Ij. thus, we consider the average similarity of images in I and Tj. we further expect high level of coherency among all images in Tj. i.e., these images should all share multiple semantic attributes described by text, thus, we wish the standard deviation of similarity scores between images in I, and T3 to be low. Lastly, we expect generated captions to be specific, capturing the essence of the failure mode, without including distracting irrelevant information. That is, caption Tý should only describe images in I and not images outside of that. As a result, we consider the AUROC between the similarity score of images inside the failure mode (I,) and some randomly sampled images outside of that. We note that in existing methods as well as our method, all images in a failure mode have the same label, so we sample from images outside of the group but with the same label.
In F5 we show (1) the average similarity score, i.e., for all 1 and x E Ij. we take the mean of sim(2, J. (2) the standard deviation of similarity score, i.c., the standard deviation of sim (z, Tÿ) for all Ij and x E Lj. and (3) the AUROC between the similarity scores of images inside failure modes to their corresponding description and some randomly sampled images outside of the failure mode to that. As shown in Figure 5] PRIME improves over DOMINO (Eyuboglu et al. 2022) in terms of all AUROC, average similarity, and standard deviation on different datasets. noting that this improvement comes even though DOMINO chooses a text caption for the failure mode to maximize the similarity score in latent space. We use hyperparameters for DOMINO to obtain fairly the same number of failure modes detected by PRIME. Results in Figure S show that PRIME is better than DOMINO in the descriptions it provides for detected failure modes. In Appendix A.8 we provide more details on these experiments. Due to the limitations of Jain et al. (2022) for automatically generating captions, we cannot conduct extensive experiments on various datasets. More details and results on that can be found in Appendix A.11
worth
ON COMPLEXITY OF FAILURE MODE EXPLANATIONS
We note that the main advantage of our method is its more faithful interpretation of failure modes. This comes due to (1) putting interpretability first, i.e., start by assigning interpretable tags to images and then recognize hard subpopulations and (2) considering combination of several tags which leads to higher number of attributes (tags) in the description of the group.
CChtGPP 3.5, August 3 version
7

---

Published as a conference paper at ICLR 2024
1.0
0.40
0.050
Method
Domino
0.35
0.045
0.
0.30
0.040
2.00
0.035
6.03
0:79
0.74
Moain
1002
0.05
0.015
0.00
0.010
Nont Nongko
Emtoy13
Liingt?
Waterbindd
Colebd
NonLiving2
Entiyy
Liiigg
Waterrindee
Colodd
Nont 2oing
Entiyy13
Livigg?
Waterbonde
Colebd
Figure 5: The mean and standard deviation of similarity scores between images in failure modes and their respective descriptions, along with the AUROC measuring the similarity score between descriptions and images inside and outside of failure modes, demonstrate that our method outper- forms DOMINO in descriptions it generates for detected failure modes across various datasets.
5.1 Do WE NEED TO CONSIDER COMBINATION OF TAGS?
We shed light on the number of tags the failure modes detected by our approach. We note that unlike Bias2Text (Kim et al. 2023) that finds biased concepts on which model's behavior changes, we observe that sometimes appearance of several tags (concepts) all together leads to a severe failure mode. As example, refer Table [ where observe that appearance of all 3 tags together leads to a significant drop while single tags and pairs of them show relatively better performance.
Accuracy on unseen images (D') for class "ape"" when given tags appear in the in- puts (see Table 5 for visualization).
Table 1:
Table 2: Average accuracy drop unseen images (D') on failure modes with 3 tags and images have least 2 of those tags least one of them.
# of Tags
Tags
Accuracy
3
hang: branch; black;
11.18%
hang; branch;
56.33%
Dataset
3 Tags
2 Tags
1 Tag
2
hang: black;
56.25%
Entity 13
34.75%
25.29%
14.86%
branch; black;
54.67%
Living
26.82%
17.13%
8.18%
hang;
70.09%
Waterbirds
23.35%
14.43%
7.19%
1
branch; 69.23%
CelebA
23.25%
16.84%
9.02%
black;
73.21%
In PRIME, we emphasize the necessity of tags. Specifically, for any detected failure mode, the re- moval of any tag would result in easier subpopulation. Consequently, failure modes with more tags not only provide more detailed description of their images but also characterize more chal- lenging subpopulations. Table 2 presents the average accuracy drop on unseen images for groups identified by three tags, compared to the average accuracy drop groups identified by subsets of two tags even single tag from those failure modes. These results clearly demonstrate that involving more tags leads to the detection of more challenging subpopulations.
8

---

Published as a conference paper at ICLR 2024
5.2 CLUSTERING- METHODS MAY STRUGGLE IN GENERATING COHERENT OUTPUT
We empirically analyze the reverse direction of detecting suman-understandable failure modes. We note that in recent work where the goal is to obtain interpretable failure modes, those groups are found by clustering images in the la- tent space. Then, when a group of images or a direction in the latent space is found, these methods leverage the shared space of vision- language to find the text that best describes the images inside the group.
Table 3: Statistics of the distance between two
points in CelebA conditioned on number of shared tags. Distances are reported using CLIP ViT-B/16 representation space. The last column shows the probability that the distance between two sampled images with at least d common tags be more than that of two randomly sampled images.
# of shared tags 2
mean
standard deviation
Probability
We argue that these approaches, based on distance-based clusters in the representation space, may produce less detailed descriptions. This is because the representation space doesn't always align perfectly with the semantic space. Even points close to each other in the feature space may differ in certain attributes, and conversely, points sharing human- uaaaan--ndesttaabbleee attributes may not be proximate in the feature space. Hence, these approaches cannot generate high-quality descriptions as their detected clusters in the represen- tation space may contain images with other semantic attributes.
d= 0
9.49
0.98
0.50
9.47
1.00
0.49
9.23
1.00
0.42
d d= 5
8.89
1.21
0.34
8.32
1.80
0.25
To empirically test this idea, we use two attribute-rich datasets: CelebA (Liu et al. 2015) and CUB- 200 (Wah et al. 2011). CelebA features 40 human-understandee a dataset of birds, includes 312 tags per image, all referring to semantic attributes. We use CLIP VIT-B/16 (Radford et al.) 2021) and examine its representation space in terms of datasets' tags. Table 3 the statistics of the distance between the points conditioned on the number of shared tags. As seen in the Table 3 although the average of distance between points with more common tags slightly decreases, the standard deviation of distance between points is high. In fact, points with many common tags can still be far away from each other. Last column in Table 3 shows the probability that the distance between two points with at least d shared tags be larger than the distance of two randomly sampled points. Even when least 5 tags are shared between two points, with the probability of 0.34, the distance can be larger than two random points. Thus, if we plant a methods cannot
tags per image, while CUB-200.
failure mode on a group of images sharing subset of tags, these clustering- find a group consisting of only those images; they will inevitably include other irrelevant images, leading to an incoherent failure mode set and, consequently, a low-quality description. This can be observed in Appendix A.7 where we include DOMINO's 's output.
We also run another experiment to foster our hypothesis that distance-based clustering methods cannot fully capture semantic similarities. We randomly pick an image x and find N closest images to x in the feature space. Let C' be the set of these images. We inspect this set in terms of the number of tags that commonly appear in its images as recent methods (Eyuboglu et al. 2022 d d'Eon et al. 2021 Jain et al., 2022), take the average embedding of images in C' and then assign a text to describe images of C. Table shows the average number of tags that appear in at least aN images of set C' (we sample many different points z). If representation space is a good proxy for semantic space, then we expect a large number of shared tags in close proximity to point æ. At the same time, for the point z, we find the maximum number of tags that appear in x and at least N other images. This is the number of shared tags in close proximity of point x but in semantic space. As shown in Table 6 average number of shared tags in semantic space is significantly larger than the average number of shared tags in representation space.
6 CONCLUSIONS
In this study, drawing from the observation that current techniques in humaat-comprehensibbee failure mode detection sometimes produce incoherent descriptions, along with empirical findings related to the latent space of vision-language models, we introduced PRIME. novel approach that prioritizes interpretability in failure mode detection. Our results demonstrate that it generates descriptions that more similar, coherent, and specific compared existing methods for the detected failure modes.
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This project was supported in part by a grant from an NSF CAREER AWARD 1942230, ONR YIP award N00014-22-1-221 1-2271, ARO's Early Career Program Award 310902-00001, Meta grant 23010098, HR0011 12090132 (DARPA/RED), HR0011190 19S0026 (DARPA/GARD) Army Grant No. W91 I 20076, NIST 60NANB20DI34, the NSF award CCF2212458, an Amazon Research Award and an award from Capital One.
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commer- cial gender classification. In Conference on fafaiesss,, accountability and transparency, pp. 77-91. PMLR, 2018.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. Proceedings of the International Conference on Computer Vision (ICCV), 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Greg d' Bon, Jason d' d'Eon, James R. Wright, and Kevin Seyton-Brown The spotlight: A general method for discovering systematic errors deep learning models, 2021.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Alek- sander Madry. Adversarial robustness as a prior for learned representations. arXiv preprint arXiv:1906.009900 2019,
Sabri Eyuboglu, Maya Varma, Khaled Saab, lean-Bence Delbrouck, Christopher Lee-Messe Jared Dunnmon, James Zou, and Christopher Ré. Domino: Discovering systematic errors with cross- modal embeddings. 2022.
Irena Gao, Gabriel Ilharco, Scott Lundberg, and Marco Tulio Ribeiro. Adaptive testing of computer vision models, 2023.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, pp. 1929-19938. PMLR, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition, 2015.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. ArXiv, abs/2 1od/2104408718. 2021. URL
wtt.:oaaw....ccooosi.i..... :3:329961
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Adrian Hoffmann, Claudio Fanconi, Rahul Rade, and Jonas Moritz Kohler. This looks like that... does it? shortcomings of latent space prototype interpretability deep net-
works. ArXiv, abs/2105.02968. 2021. URL https::/ pattss:/aapi.. www.sttesscchhhoooeoooo .or/ org/ Corpus 2D: Corpus1D:234094683
Xinyu Huang, Youcai Zhang, Jinyu Ma. Weiwei Tian, Rui Feng, Yuejie Zhang, Yaqian Li, Yandong Guo, and Lei Zhang. Tag2text: Guiding languae model via image tagging. arXiv preprint arXiv:2303.nm 05657, 2023.
10

---

Published as a conference paper at ICLR 2024
Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov, Caner Hazirbas, Nico- las Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, and Mark Ibrahim. Imagenet- Understanding model mistakes with factor of variation annotations. arXiv preprint arXiv:221 1.01866, 2022.
Saachi Jain, Hannah Lawrence, Ankur Moitra, and Aleksander Madry. Distilling model failures as directions latent space, 2022.
Nari Johnson, Ángel Alexander Cabrera, Gregory Plumb, and Ameet Talwalkar. Where does my model underperform? human evaluation of slice discovery algorithms, 2023.
Priyatham Kattakinda, Alexander Levine, and Soheil Feizi. Invariant learning via diffusion dreamed distribution shifts. arXiv preprint arXiv:2211.10370 2022.
Michael P Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 247-254, 2019.
Younghyun Kim, Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jaeho Lee, and Jinwoo Shin. Bias- to-text: Debiasing unknown visual biases through language interpretation, 2023.
Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang. et al. 3db: A framework for debugging computer vision models. Advances in Neural Information Processing Systems, 35:8498 -111,, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and Soheil Feizi. Text-to-concept (and back) via cross-model alignment. arXiv preprint arXiv:2305.063886 2023.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33:20673 226864 2020.
Besmira Nushi, Ece Kamar, and Eric Horvitz. Towards accountable ai: Hybrid human-machaae machine analyses for characterizing system failure. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 6, pp. 126-135, 2018.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11)e77 2017.
Gregory Plumb, Marco Tulio Ribeiro, and Ameet Talwalkar, Finding and fixing spurious patterns with explanations. arXiv preprint arXiv:2106.02112 2021.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint 06652222..66.,5, 1(2):3, 2022.
11

---

Published as a conference paper at ICLR 2024
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10695, 2022.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. CoRR, abs/1911.08733 2019. URL http::ttpt//xiaa
08731
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Nearal Informa- tion Processing Systems, 01166479364944 2022.
Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. Breeds: Benchmarks for subpopulation shift. In ArXiv preprint arXiv: 04859, 2020.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No sub- class left behind: Fine-grained robustness in gaaseegrained classification problems. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 19339-19352. Curran Associates, Inc., 2020a. URL https::/prooeedddss...eeee neurips ccppppr files/paper/20200 ELl1//8888a333332002222365533333
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Advances in Neural Information Processing Systems, 33: 13:39999-1935, 2020b.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. From imagenet image classification: Contextualizing progress on benchmarks. In International Con- ference on Machine Learning, pp. 0025-9635. PMLR, 2020.
Vijay Vasudevan, Benjamin Caine, Raphael Gontijo Lopes, Sara Fridovich- Keil, and Rebecca Roelofs. When does dough become bagel? analyzing the remaining mistakes on imagenet. Advances in Neural Information Processing Systems, 35:6720-677 6734, 2022.
Joshua Vendrow, Saachi Jain, Logan Engstrom, and Aleksander Madry. Dataset interfaces: Diagnos- ing model failures using controllable counterfactual generation. arXiv preprint arXiv:2302.07865 2023.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Caltech-ucsd birds-200-2011 (cub- 200-2011). Technical Report
California Institute of Technology. 2011.
Youcai Zhang. Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, et al. Recognize anything: strong image tagging model. arXiv preprint 2rXiv:2306.03514 2023.
Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D Manning, and Curtis P Langlotz. Con- trastive learning of medical visual representations from paired images and text. In Machine Learn ing for Healthcare Conference, pp. 2-25. PMLR, 2022.
12

---

Published as a conference paper at ICLR 2024
A
APPENDIX
A.1 LIMITATIONS
Utilizing pre-trained tagging models We note that in this work, we used RAM (Zang et al., 2023) extract tags from images and then used those tags to identify hard subpopulations. Ac- curacy of RAM which is the state-of-the-ard tagging model well studied in (Zhang et al. 2023). Authors observe high-quality performance of their method on different common datasets over dif- ferent tasks. We also conducted a small-scale human validation study to measure the accuracy of RAM Living17 dataset (see Appendix A. for more details) and observe strong performance. However, we acknowledge that PRIME, like other existing methods, utilizes an auxiliary model to bridge the gap between vision and language modalities. Thus, its performance is affected and lim- ited by the auxiliary method. Notably, our use of the auxiliary model (tagging models) is closely aligned to the task they are optimized for. That is, these models are trained to efficiently detect various objects and concepts in images, assigning tags accordingly. As tagging models advance, PRIME's effectiveness expected to enhanced.
PRIME for specific domains We note that PRIME relies on tagging model to extract informative tags from image, making it less effective on specific domains for which the tagging model is not optimally suited. However, we believe it is highly likely that tagging models like RAM that are finetuned on specific domains will arise very soon, similar to how CIRT (CLIP for medical data) (Zhang et al. 2022) came about soon after CLIP. Leveraging finetuned tagging models, PRIME remains effective for failure mode extraction specific domains such as medical images.
Tag interpretation Acknowledging potential challenges in interpreting certain tags within PRIME PRIME, particularly adjectives, imperative. For instance, the tag "white" may not consistently align with the primary object in the image, as its detection could be influenced by background el- ements or other objects present. Despite this, the utility of PRIME persists, serving as a valuable tool to highlight instances where model failures occur the convergence of specific tags, not exclusively tied to the main object in the image. We note that, due to this limitation, we do not use language models to generate descriptions from tags (except for validating PRIME with image generation) because language models may associate tags with class labels, which is not necessarily correct. Therefore, in evaluating the quality of descriptions, we used bag-of-word manner obtain PRIME's results.
if
A.2 TRAINING MODELS
We train a model on each dataset we are considering.
For ImageNet, we use the standard pretrained ResNet50 model (He et al. 2013).
For Living17, Entity13, and NonLiving26, we utilize a DINO self-supervised model CCron et al. 2)21) with ResNet50 backbone and fine-tune it over Living17. used SGD with following hyperparameters to finetune the model for 5 epochs.
- - 0.001
momeentm 0.9
For CelebA (age classification), we used pretrained ResNet18 model (He et al. 2015) which is finetuned for 5 epochs using SGD the following hyperparameters.
0.001
momentum = 0.9
weight decay 5c
We note that classifier is trained in way that it is biased toward images of young women and old men.
For Waterbirds, we fine-tuned a pretrained ResNet18 model (He et al. 2015) for 20 epoch using SGD the following hyperparameters.
Ir= 0.001
13

---

Published as a conference paper at ICLR 2024
momentum = 0.9
weight decay 5e
A.3 DETAILED RESULTS OF DETECTED FAILURE MODES
Our method on 1iving17:
Results: 36 failure modes with tag. 68 failure modes with 2 tags, 24 failure modes with tags, and 4 failure modes with 4 tags.
Hyperparameters: 30, a 30, b2 10%, by 5%, and by 2.5%.
class "wolf" (Accuracy: 83.69%):
hide (54.86%); floor + hide (38.71%); floor + hide (38.71%): den (49.06%); den + hide (22.22 222:2%)):
lay + red (41.86%): - den + lay (29. (79); - hide lay (41551%): - night (44,44%); -- floor cub (48.48%); - floor + den + cub (22.22%); grass stare + hide + red (66.67%) log + red (62.86%); grass + tree brown (63.64%)
class "cat" (Accuracy: 89.58%): enclosure (52.63%):
(50.00%); habitat (34.09%%;; - grassy (63.64) 663.64%); - tiger walk (28.57%); - tiger + grass (58.54%); bengal tiger walk (28.57%% bengal tiger + grass (60.00%); floor tree (57.14%); - log (56.76%); white + grass (63.64%); tiger + tree (37.50%);
hide stand (82.5%));
Our method Entity13:
Results: 45 failure modes with tag. 45 failure modes with 2 tags, and 18 failure modes with 3 tags. Hyperparameters: 100, a 30, b2 10%, by 5%, and b₃ 2.5%.
class "wheeled vehicle" (Accuracy: 88.05%)
shopping cart (53.03%); - floor + cart (54.17%): sit shopping cart (43.90%); cage (44.20%); -- basket (50;45%); - man + pole (65.79%);
class "produce, green goods, green groceries, garden truck" (Accuracy: 92.91%): floor food (74.77%);
class "accessory, accoutrement, accouterment" (Accuracy: 63.98%): swimwear + pose (18.18%); stand pose black (31.15%); brunette (19.23%); swimwear brunette (4.20%); - person + graduation (33.92%);
Our method CelebA (Young Old classification):
Results: 45 failure modes with tag, 27 failure modes with 2 tags, and 11 failure modes with 3 tags. Hyperparameters: 100, 30, b2 10%, ba 5%, and b4 2.5%.
class "young" (Accuracy: 80%):
beard (32.34%); man laugh (41.21 %); smile tie stand (58.27%);
goggles (41.04%): - man + sunglasses (36.11%); - man + white + stand (69.32%); man sing (33.78%); man + microphone (40.00%); business suit smile + stand (51.11%); - black goggles (42.65%);
Our method on CelebA (Young vs Old classification):
Results: 45 failure modes with tag, 27 failure modes with 2 tags, and 11 failure modes with 3 tags. Hyperparameters: 100, a 30, b2 = 10%, by = 5%, and by = 2.5%.
class "young" (Accuracy: 80%):
beard 632.34%); - man + laugh (41.21 (41.21%); - smile tie + stand (68.27%); - man goggles (41.04%); man sunglasses (36.11%): man white + stand (69.32%); man + sing (33.78%); - man + microphone (40.00%); - business suit + smile + stand (51.11%); black goggles (42.65%);
Our method Waterbirds:
Results: 4 failure modes with tag, failure modes with 2 tags, and failure modes with 3 tags. Hyperparameters: 100, 30, by 10%, by 5%. and by = 2.5%.
14

---

Published as a conference paper at ICLR 2024
bear (0.
fence: cage: (69. 49%)
snake (92.31 (11)
woman: (56 25%)
bear (9%%
black; climb; tree branch: (51 (5%5)
snake (92 31%)
catch; stand: (12 50%)
parrot (93 (5%)
lush; grass, grassy, field; (40.74%)
(81. (6.666)))
zoo; (35.29%)
parrot (93.85%)
pigeon: (51.28%)
Fox (81.96%) grass: stand: field; dry, (47.83%)
Figure 6: Some visualization of detected failure modes Living17.
class "landbird" (Accuracy: 87.41%):
black sea (62.50%); crow water (64.5%%): water person (69.86%); - black + water beak (71.43%); water man (71.93%); -- stand + sea blue (65.71%); sea sit + boat (75.68%); black + ledge sea (58.82%);
class "waterbird" (Accuracy: 33.00%): wood (10.47%); stem (6.67%); stand + tree + pole (16.67%);
A.4 VISUALIZATION ON SOME OF THE DETECTED FAILURE MODES
We refer Figures and for more visualization of failure modes detected in approach Living17, ImageNet, and Waterbirds.
A.5 RUNNING THE METHOD USING DIFFERENT VALUES OF (s,a)
In this section, we inspect the effect of different hyperparameters (s, a) on the result of our method. By increasing a, we aim detect harder subpopulations, thus, the number of detected failure modes will decrease. By increasing s, we detect group of images associated with a set of tags as failure mode, if there are significant number of images within that group. This brings more generalization over detected failure modes while a fewer number of them will be detected by the method. Figure IO shows the generalization plot over different datasets with respect different hyperparameters.
In Table 4 we also report correlation coefficien between train drop and test drop of failure modes over different datasets and values and
A.6 GREEDY SEARCH
We note that in experiments, exhaustive search efficient enough that do not need to consider any other approaches. We used some heuristic approaches to improve the efficiency of exhaustive search such eliminating combination of tags that few images represent them, etc.
15

---

Published as a conference paper at ICLR 2024
smooth newt (73.85%)
forest floor; (57.41%)
smooth (73 85%)
black; (47 837%
great white shark (91.23%)
reef: (63.89%)
great white shark (91 23%)
diver; underwater: aquarium; (70.00%
tailed frog (78.625 (88.2%)
green; swamp: (57.50%)
tailed frog (78.62%) leaf; hide; plant: (57.69%)
Figure 7: Some visualization detected failure modes on ImageNet.
landbird (87.41%) 41%)
black: water: (69.09%)
landbird (87.41%) - water: person; (69 (6%
Figure 8: Some visualization of detected failure modes on Waterbirds.
16

---

Published as a conference paper at ICLR 2024
Living17 (s 30,a 30%)
CelebA (s 50, 30%)
[PL.aS: (23/25)
60
50
20%
(82.00% (26/29))
Teat Drop (%)
Test (%)
60
30
1,00i 4.58
10
[mn m9% (10 1lrt (116/1372)
40 GB
70
35
40
50
65
Train Drop (%)
Train Drop (%)
Figure 9: Evaluating detected failure modes on unseen data. (Left)) we extract failure modes on Living17 dataset using 30 and a 30%. 132 failure groups (over 17 classes) are detected and it is observed that around 86.01% % of detected failure modes exhibit at least 25% drop in accuracy over unseen data that shows significant degree of generalization. (Right): same results for CelebA dataset where the parameters for failure mode detection is 50 and 30%. Around 79.31% of failure modes show the drop of least 20%. The trend = x is seen in these plots.
Dataset
corrcoef
25
25
0.70
Living
50
30
0.73
30
30
0.72
100
30
0.73
Entity 13
200
30
0.73
200
25
0.85
50
30
0.60
CelebA
100
30
0.83
100
25
0.81
Table 4: Correlation coefficien between train drop and test drop.
17

---

Published as a conference paper at ICLR 2024
Trals Drea (h)
Thah Drea 1M
Emilyd3 3010
Dmily33
Exiiy11
cam jouns jacn
2.25 Docoff 2010
Trans Ovea 0%)
Ever
Tean Draa (%)
Coldb
3/51
purt Jears pecid
230 Joual 2011
2.2 (ll
Trans Drea 0%)
Oveo
Trans Drac (%)
Figure 10: Evaluating detected failure modes on unseen (test) dataset. We extract failure modes on different datasets using different values of (s, a).
However, we developed another greedy search algorithm where at each stage, we pick top tags that condition on them, significantly dropping model accuracy. By running this approach, the space of different choices shrinks and the algorithm becomes faster while missing some the failure modes.
A.7 DOMINO'S s OUTPUT DESCRIPTIONS
To compare the results of DOMINO with PRIME, picked DOMINO's 's hyperparameters in way that generates relatively the same number of failure modes. Some of the outputs on Living 17 dataset follows:
class "salamander":
photo of the bullet wound.
photo of lizard.
photo of trout fishing.
a photo of a frog.
a photo a hippo.
a photo of the ventral fin.
class "fox":
a photo of a gorillas.
a photo of the titanic sinking.
a photo of the tract.
a photo of a coyote.
photo of oil shale.
photo of the desert.
photo of the antarctic.
18

---

Published as a conference paper at ICLR 2024
class 'cat': photo of the gardener.
class 'cat': photo of two dogs.
class 'cat': photo the zoological garden.
'fox': photo the arctic.
class fox': photo of the desert.
class 'fox': photo of the titanic sinking.
class fox': photo a gorillas.
class 'salamander': photo hippo.
class 'sllamandrr: photo of frog.
class 'salamander': photo of trout fishing
Figure 11: Some of the detected failure modes along with descriptions using DOMINO.
photo of the arctie.
class "cat":
photo of the zoological garden.
photo of stray dogs.
photo of two dogs.
photo of the gardener.
photo of the tehsil leader.
Figure 11 shows some visualization of these failure modes. Lack of coherency among images in some the groups as well as low-quality descriptions can be seen in detected failure modes.
A.8 QUALITY DESCRIPTION
We note that failure modes detected by PRIME and DOMINO might be different. However, our metries only evaluate how well failure modes are described with their corresponding captions and do not consider what images are assigned as failure modes. This enables us to compare different methods with each other. Notably, hyperparameters of different methods play a role the number of failure modes that method detects. In order to ensure fair comparison, hyperparameters for both of methods to yield a similar number detected failure modes.
carefully set the
Furthermore, it is worth noting that the similarity score is normalized, comparing this different failure modes and different methods is possible. This is why we aggregate similarity score, standard deviation, auroc over all failure modes.
A.9 DOMINO'S GENERALIZATION
To compare our results with DOMINO (Eyuboglu et al. 2022). we note that this method also outputs some groups well descriptions for them. For failure mode I, with T₃ its description,
19

---

Published as a conference paper at ICLR 2024
Entity13 (DOMINO)
Living17 DOMINO
0.00s + 6.71
60
Ka3 + K.1
Tex Drop (%)
30
Test Drop (%)
10
0
-20
-10
Train Drop (%)
Train Drap (%)
CelebA (DOMINO)
Waterbirds (DOMINO)
40
- 0.54s 37
0.33
25
Test Drop (%)
20
Text Drop
-00
Train Drop (%)
Train Drop (%)
Figure 12: Evaluating detected failure modes by DOMINO on unseen (test) dataset.
use the same vision-language model that DOMINO uses to collect highly similar images to caption T3 in D' and obtain I/. We then evaluate model's accuracy images of Ií and expect hard subpopulation, it should be as hard as Ij. Figure [2 shows the generalization of this method. We see lower degree of generalization captions cannot fully describe as hard subpopulations as subpopulations detected on D.
this approach than our method. We observe that generated
We note that the other method (Jain al. 2002)) only detects a single failure mode for each class in the input and reports around 10 images for that, thus, comparing this method with DOMINO and ours is bit unfair as those methods detect multiple failure modes with significantly more coverage.
A.10 IMAGE GENERATION
In this section, we elaborate more on the way we generate hard and easy images. To detect easy subpopulations, randomly pick subset of tags way that the model's accuracy images representing those tags is 100%. For the failure modes, we randomly pick two of the detected failure modes in way that those two groups do share any common images. worth noting that for classes "butterfly" and "dog" we don't report any results as model's accuracy for these classes is almost 100%. Figure 13 shows the accuracy gap for different classes.
Here we provide some of the failure/success modes we took and corresponding descriptions used for generative models.
class: "bear"
gray + water - "a photo of gray bear in water";
river + "a photo of bear in the river"
cub + climbing + tree + "a photo of a bear cub climbing a tree";
black + cub branch - "a photo of black bear cub on tree branch";
class: "ape"
20

---

Published as a conference paper at ICLR 2024
Average Gap: 0.41
Go
0.6
Accordto
0.4
0.2
mond
beer
grossee
Spe
bossle
salomander
crib
painot
Rlord
damostic cat
tumil
Cosses
Figure 13: The difference in classifier accuracy between images generated from success mode and failure mode captions on Living 17.
0.60
0.030
Jain et
0.040
0.007
012
0..8
0.25
0.003
AUROC
Mex
0.20
0.000
0002
6.99
0.7
0.15
0.025
6001
0.5
0.05
0.015
0.5
0.00
0.010
Uwing1?
Celibut
Uiingi?
ColdA
Liinsg7
Colobul
Figure 14: Comparison of the quality of description our method and Jain et (022)) AUROC, Mean, and Standard over different datasets are reported.
black + branch - "a photo of black ape on a tree branch";
sky + "a photo of ape in the sky";
gorilla trunk + sitting + "a photo of a gorilla ape sitting near a tree trunk";
mother -> "a photo of mother ape":
A.11 JAIN ET AL. QUALITY OF DESCRIPTION
We note that Jain et al. 2002 detects 10 hard and 10 easy images for each class in the dataset and assigns a description to them. However, the way this method generates captions needs humans the loop as they use a bunch of dataset-oriented words (tokens) that explain different variants of semantic attributes the dataset. Hence, their method is not scalable to run on many different datasets so only considered that Living17 and CelebA. When datasets become larger and more complex, there will exist several failure modes, and Jain et al. (2022) that only extracts a single direction cannot cover many of the failure inputs. Figure [howswws the results different datasets. We note that on CelebA, they use very detailed and manually collected set of tokens to generate outputs. However, in that dataset, their performance slightly better than ours.
21

---

Published as a conference paper at ICLR 2024
Table Accuracy on unseen (test) images for class ape when given tags appear in the inputs. More detailed descriptions not only bring more accurate explanation of images but also can lead to harder example where model's performance drops more severly. Some randomly sampled images within each group is visualized.
Num of lags
Tags
Sampled Images
Test Accuracy
3
hang; branch; black;
41.18%
bang: branch;
56.33%
2
hang; black;
56.25%
branch; black;
54.67%
hang:
70.09%
branch:
69.23%
black;
73.21%
Table 6: Average number of tags appear in at least aN images among N closest images to ran- domly sampled image in the representation space well average number of shared tags in mantic space (CelebA Dataset).
Representation Space
Semantic Space
0.6
0.7
a
0.8
N 50
4.17
3.56
2.91
7.55
100
3.94
3.34
2.71
7.25
A.12 NUMBER OF TAGS IN THE DESCRIPTIONS
In Table we bring a more detailed version Table that includes some sampled images of detected failure modes.
22

---

Published as a conference paper at ICLR 2024
Table Statistics the distance between two points in CUB-200 dataset conditioned on number of shared tags. Distances are reported using CLIP ViT-B/16 representation space.
of shared tags
mean
standard deviation
Probability
d
7.37
0.92
0.50
0.94
7.06
1.08
0.42
6.29
1,8
0.30
Table 8: Average number of tags appear in at least aN images among N closest images domly sampled image the representation space as well as average number of shared tags in se- mantic space (CUB-200 Dataset).
Representation Space
Semantic Space
a 0.7
a = 0.8
N 50
2.53
1.46
12.34 10.91
N = 100
3.00
1.99
A.13 CUB-200 AND CELEBA
In this section, we show the results reported in Section 5.2 the results of shared tags in the proximity of images and Table 7 includes the statistics of distance between two images in the latent space.
5.2 over CUB-200 dataset. T
A.14 RAM EVALUATION ON LIVING17
we run a small-scale human validation study on Living17 (one of the main datasets we used in our paper) evaluate the accuracy of RAM. We first obtain all tags of class "butterfly" in Living17 and filter out low- frefereyy tags as discussed in 3.1 55 tags are remained, i.e..
T butterly [black, purple, red, flower, leaf, sit, land, gravel, stone, mud, wildflower, sky, grass,...}
We take 100 random images from the class "butterfly" in Livingl7 and evaluate precision/recall of tagging model on those images over tags of Touttrry.. Average Precision of RAM over those 100 images is 86.85% and average recall 81.85%. This shows that RAM is accurate in detection of tags in butterlyy It is worth noting that bumently includes a wide range of different objects and attributes, covering wide range of concepts in those images.
A.15 PRIME HYPERPARAMETER
In this section, we elaborate more on each of the hyperparameters in our work. We note that all of the PRIME's hyperparameters have intuitive definitions, enabling a user to calibrate PRIME toward their specific preferences.
Parameter a controls trade-off between the difficulty and the quantity of detected failure modes. For example, selecting a high value of results in failure modes that are more difficult but fewer in number. Figure 10 shows this trade-off.
Parameter s determines the minimum number of required images inside a group to be detected failure mode. Small groups may not be reliable, thus, filter them out. Larger value for results in more reliable and generalizable failure modes. The choice of also depends the number of samples within the dataset. For larger datasets, can assume that different subpopulations are sufficiently represented in the dataset, thus, larger value for can be used. We refer Figure 10 for observing the effect of
determines the maximum number of tags we consider for combination. In datasets we considered, combination of more than 5 tags would not result in groups with at least images, thus, we set < 4 in our experiments. The choice depends on the dataset and tagging model.
23

---

Published as a conference paper at ICLR 2024
b, refers to the degree of necessity for tags inside the failure modes. The current choice of b,8 is only a sample choice, requiring the appearance of each tag to have a significant impact the difficulty of detected failure modes. One can adjust these hyperparameters based on their preference.
24


Reviews:
[{"review_id": "u8rPvyGc9z", "reviewer": "Reviewer_WEiw", "summary": "This paper proposes a method to find human-understandable descriptions of situations under which a given image classification model fails. The authors argue that prior works obtain these descriptions using clustering in latent spaces which results in poor identification of shared attributes. In their method, they first obtain _tags_ for images using a SOTA model, and then find combinations of tags for which the model performs poorly to understand failure modes. They claim that this _interpretability-first_ approach results in a more robust failure mode extraction method.", "questions": "See weaknesses.\n\nOverall, I feel that although the presented approach may yield good failure mode descriptions for some general purpose models, the study is not sound enough. Moreover, the paper lacks a section on limitations (in my view, there are many). All this fails to excite me to support of the paper for acceptance.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "1. Experiments done on multiple datasets to support the approach.\n2. Visual examples of the method working look promising. The proposed method maybe a good practical method to generate descriptions of failure modes in a general purpose entity recognition models.", "weaknesses": "1. The method relies heavily on an auxiliary model (like Recognize Anything Model). Although this makes an important tool for describing failure modes, everything that the proposed method can do is bottlenecked by this auxiliary model's capability. For example, the authors present this method as a tool to describe \"failure modes in trained image classification models\". However, what happens when the image classification model is trained on a domain-specific dataset like chest X-Rays? How can we expect the auxiliary general purpose model to provide tags for this dataset? This is a practical paper but we do not see ablation studies with other general purpose auxiliary models (apart from RAM), or other experiments with domain-specific image classification models.\n2. The tag generation process requires a threshold hyper-parameter. Similarly, there a lot of hyper-parameters in the proposed method's setup like $b_i$, $a$, $s$, $l$, etc. Some of them look more sensitive than others. However, no insights are given on how they are selected apart from saying \"we generally\" use these values. \n3. In 4.1 paragraph 1, the authors say that $D$ and $D'$ are from the same distribution, and hence the accuracy drop should translate. This is not what generalization means. Generalization applies to the scenario when unseen data is from a shifted distribution. As the authors clearly mention that the unseen dataset is drawn from the same distribution (and not explicitly curated in some way like selective sampling), this dismisses all claims of generalization. The generalization on generated data is an interesting experiment but the fact that authors fine-tuned the generated model on the training data simply nullifies all claims of generalization. \n4. The authors say, \"We utilize language models to create descriptive captions for objects and tags associated with failure modes in images\". It is unclear how this is done. And how is the quality of the generated captions measured?\n5. Regarding the quality of descriptions metrics: Firstly, I have no idea what the AUROC metric means at all. Moreover, these \"three\" metrics are again based on another auxiliary model. It suffers from all the same issues as mentioned in point 1. The authors don't talk about this potential pitfall. Additionally, this evaluation goes through multiple models to come up with embeddings to calculate a distance metric. First, the tags are generated using RAM. After the failure mode tags are identified, they are passed through a language model to generate captions. Then, the caption is passed through another foundation model to generate text embeddings while the original image is passed through the same foundation model to generate image embeddings. I don't see how we could make any definitive claim about these metrics (which don't seem to be very different from the corresponding values of DOMINO).\n6. Regarding the similarity between representation space and semantic space: This analysis is confusing and potentially flawed. The representation space may carry nuanced information which is not captured by the tags. For example, a common tag which says \"water\" may have very different representations depending on whether it is ocean blue water or river green water. Therefore, it can not be expected that closeness in representation space should closely align with closeness in semantic space. The semantic space is formed after (potentially several) layers of abstraction on the representation space which is supposed to store all fine-grained details. The fact that table 5 still shows a significant number of overlapping tags in nearby images is something that I would expect to begin with. The images that share maximum number of tags might have very different subtle differences that make them distant in the representation space.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bqhRUp0GDc", "reviewer": "Reviewer_dBy1", "summary": "The authors introduce PRIME, a new method to identify (text descriptions of) coherent groups where a trained image classifier underperforms. PRIME works by (1) \"tagging\" each image using a pretrained model, and (2) performing an exhaustive search over the sets defined by all possible combinations of tags, for the sets which are \"minimal\", larger than some threshold, and have accuracy lower than some threshold. Like past methods, PRIME is \"unsupervised\" in that it does not need a HIL or a comprehensive existing set of tags.\n\n\nThe authors evaluate PRIME in Section 4 by reporting the \"generalization\" of the generated text descriptions (when used to retrieve matching unseen images from a test set or new generated images), and propose three new \"metrics\" that compare the CLIP embeddings of the text description to the images in each group.  They also run additional experiments in Section 5 that further support their claims by demonstrating the \"value\" of defining more specific subgroups using combinations of tags, and also exploring properties of pretrained embeddings such as CLIP.", "questions": "See Weaknesses above.\n\nIn addition my listed weaknesses (which are more important to address), I had a few remaining clarifying questions/suggestions:\n* Make clear in the introduction that your method doesn't rely on human-labeled tags, but rather uses a pre-trained model\n* nit (p2): \"images who match group of tags identified as a failure mode\" => \"the images that match a failure mode\"...\n* nit (Section 4): rather than saying \"difficulty (of the detected failure modes)\", explicitly use terms like \"accuracy\" or \"performance gap\", as \"difficulty\" is vague\n* Section 5.2: I want to see more nuance in this discussion about how it seems like your method will always incentivize choosing a higher-error subclass (rather than a larger superclass), when in reality it may be of interest to return the superclass.  I don't think that \"smaller\"/\"more specific\" is always better in practice.  I think what's interesting about the exhaustive search here is that you can present the performance over all of the possible subgroups to the model developer/user, who can then choose which ones they would like to prioritize for fixing or further evaluation moving forward. \n* Two helpful references to consider adding to your related work section that illustrate the difficulty of generating high-quality text descriptions for error discovery are (1) Appendix C of AdaVision [1], and (2) the user study in Johnson et al. [2], \n\n[1] https://arxiv.org/abs/2212.02774\n[2] https://arxiv.org/abs/2306.08167", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors contextualize related work on error discovery well in the Introduction, and clearly motivate gaps in the existing literature (i.e., that generating text descriptions of groups where an image classifier underperforms is difficult). This problem is timely and significant.\n2. The authors' instructive figures (e.g., Figures 1, 3, and 5) clearly illustrate the benefits of their proposed approach.\n3. The authors' proposed PRIME method is straightforward, and explained in a way that is clear and reproducible as a reader. I specifically like how the authors motivated why they are interested in searching for a minimal set – to my knowledge, this criteria, while intuitive, has never been discussed in the context of error discovery.\n4. I appreciated the author's exploration of potential limitations with clustering-based approaches by running experiments using CLIP in Section 5.2.  Given the popularity of such embeddings in the field of error discovery, I thought this analysis was particularly timely and significant.", "weaknesses": "My primary critique of this work is that I believe the authors should directly address potential limitations of their proposed method in their paper.  In the present draft, many important limitations are excluded completely.  I am willing to adjust my score if my below concerns are addressed.\n\n* **Weakness #1: Understanding limitations of relying on a pre-trained tagging model (Step 1)**. PRIME relies on a pre-trained tagging model (in this case, RAM) to provide a set of tags for each image. However, relying on a pre-trained model may have several limitations: \n  * As a simple example, when we consider the \"fox+white+zoo\" group in Figure 1, it is completely possible that RAM may fail to tag a large number of images in the dataset that are in a \"zoo\" (false negatives), or include images that are not taken in a zoo in the group (false positives), etc.  \n  * The tags output by RAM also are likely not comprehensive – there are some objects that perhaps RAM is worse at recognizing due to their absence in the pretraining data.  \n  * I may be missing something but the present draft is also unclear on how many tags are sampled for each image in this first part, and how the tags are sampled (do you only look at the 100 \"most likely\" tags for each image?).  \n  * All of the authors' evaluation experiments in Section 5 appear to assume that the tags generated by RAM identify groups of images that \"match\" them; however, without further evidence I believe that this assumption is unfounded. I would be interested in whether the authors can perform any kind of small-scale evaluations where they report the precision/recall of the RAM tagger – perhaps you can even compare its performance at tagging some classes that you already have metadata labels available for.\n* **Weakness #2: Clarify how the \"tags\" should be interpreted (moving from tags to descriptive captions)**.  \n  * Each group is defined as the intersection of a set of tags that are all present in that group's images. But, some ambiguity remains in how to interpret the tags.  While the example groups that the authors selected in Figures 1 and 2 have clear and natural interpretations, in general I think the ambiguity of modifiers, attributes, and propositions (relationships between multiple objects) may be difficult to interpret.  Even for the group \"fox + white\" (which the authors interpret as foxes with white fur), it's unclear to me if this is a correct interpretation because the tagging model may include any photo that has \"white\" anywhere at all in it (ex, an orange fox in the snow).  I believe that leaving this point unaddressed may mislead users of PRIME to draw the wrong conclusions.\n\n* **Weakness #3:  Clarify experimental methodology.**  I would like to see more detail in the main text about some parts of the experimental methodology to ensure that the examples provided in the paper are indeed representative of PRIME's performance (instead of being cherry-picked to make PRIME look good).  \n  * Specifically, how exactly did you \"use ChatGPT\" to move from sets of tags, to semantically meaningful captions, in Section 4.2 (and did you do any qualitative verification that these generated captions indeed seemed to \"match\" their corresponding groups of images)? \n  * How did you select the images that you're visualizing for each of the groups in Figures 1, 3, and 5 – were they randomly selected, or chosen (related to my Weakness #1)?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uxZ7I9qfMN", "reviewer": "Reviewer_j2ri", "summary": "This paper addresses the problem of understanding misclassifications in image classifiers. It proposes an approach called PRIME, that uses a set of tags that correspond to human understandable concepts, and finds what combinations of tags align (using a vision-language model) with misclassified subgroups of images. This contrasts with prior work that first clusters misclassified images and then attempts to label them with human understandable concepts, but suffers from the problem that similarity in feature space might not necessarily imply similar features in the image, leading to noisy concept descriptions. A comprehensive experimental evaluation is performed to demonstrate the effectiveness of the proposed approach.", "questions": "Discussion on the points raised in Weaknesses would be helpful.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The paper addresses the important problem of understanding failure modes in deep networks. Given the black-box nature of such models, this is crucial building trust and deploying such models safely.\n1. It recognizes and addresses a drawback in prior clustering based approaches, that similarity in representation space need not imply similarity in semantic space. This corroborates findings reported in other contexts in prior work (e.g. [1]). It proposes a simple alternative, i.e. tagging all images and finding subgroups that perform poorly, and shows that it is much more effective.\n1. By labelling tags first, PRIME finds exactly what combinations of tags cause the model to fail, instead of just finding what is common in failure cases. The latter may not answer whether every tag in the set is necessary, and so the set may not be minimal. This represents more faithful explanations for the failure modes.\n1. The evaluation performed is comprehensive and across a variety of dimensions, including generalization to generated images, unseen images, and for the quality of the tags. The evaluation also shows the effectiveness of PRIME as compared to prior work.\n1. The paper is well-organized and easy to follow.\n\n\n[1] Hoffmann et al. This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks. ICML-W 2021.", "weaknesses": "1. The method seems highly reliant on being able to find all relevant tags first. Failure modes caused by concepts not in the tag set $T_c$ would remain undetected. Given that low frequency tags are filtered out, this could miss potentially important but less frequently occurring failure modes, such as spurious correlations. Broadly, there seems to be a tradeoff involved in choosing the size of the tag set -- larger the set, more the failure modes that can be caught, but also more computationally expensive the method is.\n1. The effectiveness of the method also depends on ensuring that a tag for a concept is actually matched exactly to images containing the concept (i.e., to the accuracy of the Recognize Anything Model). An evaluation of whether this actually happens would have been helpful, especially with datasets that have attribute labels that could potentially be matched to generated tags.\n1. The results in Figure 4 and other similar plots could be more rigorously shown by reporting the correlation between the accuracy drop in the training and test sets for each subgroup.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zl9Z4xVRpc", "reviewer": "Reviewer_eU94", "summary": "The paper proposes a novel approach for identifying and describing failure modes in image classification models. The authors prioritize interpretability by analyzing the model's behavior based on the presence or absence of human-understandable concepts (tags) associated with images in the dataset. The method aims to generate high-quality text descriptions of failure modes while avoiding redundancy and noise. The paper presents experimental results on different datasets to demonstrate the effectiveness of the proposed approach.", "questions": "Please check the weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novel approach: The paper introduces a new method that prioritizes interpretability in failure mode extraction. By analyzing the presence or absence of human-understandable tags, the proposed approach aims to provide more accurate and meaningful descriptions of failure modes. \n2. Experimental validation: The authors conduct experiments on different datasets to evaluate the effectiveness of their method. The results demonstrate that the proposed approach successfully identifies failure modes and generates high-quality text descriptions associated with them. \n3. Importance of interpretability: The paper highlights the significance of prioritizing interpretability in understanding model failures. By providing human-understandable descriptions, the proposed approach enables easier identification and diagnosis of failure modes, leading to improved model reliability.", "weaknesses": "1. The authors mention comparisons with other studies (e.g., Eyuboglu et al., 2022; Jain et al., 2022) in the text but do not elaborate on the strengths and weaknesses of these studies and how this study compares favorably to them. It is suggested that the authors provide a more detailed explanation in this regard.\n2. This paper mentions the impact of using different hyperparameter values on the results when discussing generalizability, but does not explicitly state the specific range and selection methods for these hyperparameter values. It is suggested that the authors provide detailed explanations in the paper regarding this so that readers can better understand and apply these methods.\n3. Lack of comparison with state-of-the-art: The paper does not compare the proposed method with the current state-of-the-art methods for failure mode detection and description, making it difficult to assess its advancement. It is recommended that the authors perform a quantitative comparison with the latest methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)