SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Formally, given a slice  $\Phi$ , we first compute the influence embeddings of the training data,  $[\mu(z_1'), \ldots, \mu(z_{N'})]$ . Then, we compute the sum of the influence embeddings of the slice,  $v := \sum_{i \in \Phi} \mu(z_i)$ . Finally, due to the properties of influence embeddings, the top- $k$  slice opponents of the given slice,  $O$ , are the  $k$  training examples  $z'$  for which  $v^{\top} \mu(z')$  is the most negative, i.e.  $O = \operatorname{argmin}_{O' \subseteq [N'], |O'| = k} v^{\top} (\sum_{i \in O'} \mu(z_{i'}))$ . Inspecting slice opponents can provide insight into the key features responsible for low performance in a slice.
Here, we perform a quantitative evaluation of InfEmbed on the dcbench benchmark, and also perform several case studies showing InfEmbed's usefulness. Please see Appendix Section H on a case study applying InfEmbed to a small dataset where additional metadata can help explain slices. For scalability, to compute influence embeddings, at times we will only consider gradients in some layers of the model when calculating the gradient  $\nabla_{\theta}L(z;\theta)$  and inverse-Hessian factors  $M,\lambda^{1 / 2}$  in Equation 6. Furthermore, following Schioppa et al. [2022], we do not compute those factors using the entire training data, i.e. we pass in a subset of the training data to FactorHessian. For all experiments, we use Arnoldi dimention  $P = 500$  , and influence embedding dimension  $D = 100$  unless noted otherwise. In the experiments that use InfEmbed-Rule, we used branching factor  $\mathrm{B} = 3$  . The rationale is that B should not be too large, to avoid unnecessarily dividing large slices with sufficient low accuracy into smaller slices. In practice,  $\mathrm{B} = 2$  and  $\mathrm{B} = 3$  did not give qualitatively different results.
4.1 InfEmbed on dcbench
Overview & Experimental Procedure: dcbench [Eyuboglu et al., 2022a] provides 1235 pre-trained models that are derived from real-world data, where the training data is manipulated in 3 ways to lead to test-time errors. This gives 3 tasks - discovering slices whose errors are due to the following manipulations: 1) "rare" (examples in a given class are down-sampled), 2) "correlation" (a spurious correlation is introduced), and 3) "noisy label" (examples in a given class have noisy labels). Since the error causing slice is known a priori, dcbench can serve as a way to assess a new SDM. The
underlying datasets for those tasks include 3 input types: natural images, medical images, and medical time-series. Eyuboglu et al. [2022a] also introduced Domino, the SDM that is currently the best-performing SDM on dcbench, which uses multi-modal CLIP embeddings as input to an error-aware mixture model. Following Eyuboglu et al. [2022a], we compare InfEmbed to Domino using the precision-at-k measure, which measures the proportion of the top k ( $k = 10$ ) elements in the discovered slice that are in the ground truth slice. We evaluate in the setting where errors are due to a trained model.
Results: Table 1 compares InfEmbed to Domino across the 3 tasks and 3 input types. InfEmbed always beats Domino except on the "noisy" task for the EEG Medical Time series input type. We can also compute coherence scores—the k-means objective within each cluster—for each setting in the domino benchmark,
We also compute the coherence scores, the K-means objective that we cluster, across all slices for influence embedding clustering as well as for domino. We find a similar correspondence to the previous results. In every setting where we outperform Domino (1219 out of 1235 trained
settings), our coherence scores are better, in some cases, by almost 50 percent. Regarding label homogeneity: In high performing clusters, we find that influence embedding clustering has lower label homogeneity than domino. However, for error clusters we find the reverse. The goal of our scheme is to find clusters where the model is 'wrong for the same reason'. We conjecture that influence embeddings are most effective for these settings.
SDM Approach	Rare	Correlation	Noisy Label
Natural Images
Domino	0.4	0.45	0.6
Inf-Embed	0.65	0.55	0.67
Medical Images
Domino	0.39	0.6	0.58
Inf-Embed	0.57	0.62	0.73
Medical Time Series
Domino	0.6	0.55	0.9
Inf-Embed	0.64	0.65	0.81
4.2 InfEmbed on the SpotCheck Benchmark
Overview & Experimental Procedure.
The SpotCheck benchmark Plumb et al. [2022] is based on a synthetic task consisting of 3 semantic features that can be easily controlled to determine the number of 'blindspots' in a dataset. A blindspot is a feature responsible for a model's mistake. Similar to dcbench, a model trained on data generated from SpotCheck is induced to make mistakes on an input that has a set of blindspots. The task is to predict the presence of a square in the image. Attributes of
Table 1: InfEmbed generally outperforms Domino across 3 input types and 3 tasks on the dcbench benchmark in terms of precision (higher is better).
DR	FDR
SDM Approach
Barlow	0.43(0.04)	0.03(0.01)
Spotlight	0.79(0.03)	0.09 (0.55)
Domino	0.64 (0.04)	0.07(0.01)
PlaneSpot	0.85 (0.03)	0.07 (0.01)
Inf-Embed	0.91 (0.09)	0.15 (0.62)
Table 2: Spotcheck Benchmark results.
the input image such as the background, object color, and co-occurrence can be varied to induce mistakes in models obtained from the manipulated data generation process. Plumb et al. [2022] find that current SDM approaches struggle in the presence of several blindspots. Consequently, they propose PlaneSpot, an SDM that uses the representation of the model's last layer, projected to 2 dimensions, along with the model's 'confidence' as part of a mixture model to partition the dataset. They show that PlaneSpot outperforms current approaches as measured by discovery and false discovery rates. We replicate the SpotCheck benchmark, for a subset of features, and compare InfEmbed to PlaneSpot. We refer to the Appendix Section D.1 for additional details.
Results. We report the performance of the influence embedding approach compared to PlaneSpot on the Spotcheck benchmark in Table 2. We find that Inf-Embed approaches improves upon PlaneSpot's performance on the benchmark, which indicates that the proposed approach is an effective SDM.
Alternative Clustering Approaches. To check that the proposed scheme is robust to the choice of clustering algorithm, we replicate the spotcheck experiments with 2 additional clustering algorithms: 1) DBSCAN, and 2) Spectral Clustering. Similar to Table 2, we find that the discovery rate for these algorithms are: 0.92, and 0.89 respectively. In both both cases, the performance of the InfEmbed
Figure 2: Overview of Influence embedding Clustering for a Resnet-18 on ImageNet. We show two slices from applying the influence embedding approach to the ImageNet data. We apply the RuleFind algorithm and identify slices with accuracy under 40 percent and at least 25 samples. Left: On the left, we show the distribution of labels and model predictions in that slice (Labels are in red, and Prediction are blue). Middle: Representative examples from the slice. Right: We show four opponent examples for the identified slice.
procedure does not degrade, which indicates that the influence embedding representations are robust to the clustering algorithm used.
4.3 Imagenet & High Dimensional Settings
Overview & Experimental Procedure: Here, we identify problematic slices in a "natural" dataset - the test split of Imagenet [Deng et al., 2009], containing 5000 examples. This image-classification problem is a 1000-class problem, and the analyzed model is a pre-trained Resnet-18 model achieving  $69\%$  test accuracy. To compute influence embeddings, we consider gradients in the "fc" and "layer4" layers, which contain  $8.9\mathrm{M}$  parameters. We use InfEmbed-Rule to find slices with at most  $40\%$  accuracy, and at least 25 examples. To diagnose the root-cause of errors for a slice, we examine the slice's opponents, following Section 3.6.
Results: We find 25 slices, comprising a total of 954 examples, whose overall accuracy is  $34.9\%$ . Figure 2 displays 2 of these slices (See Section E for additional examples). For each slice, we show the distribution of predicted and true labels, the 4 examples nearest the slice center in influence embedding space, and the 4 strongest opponents of the slice. First, we see that the slices have the label homogeneity property as explained in Section 3.5—the predicted and true labels typically come from a small number of classes. In the first slice, the true label is mostly "sidewinner", and often predicted to be "horned viper". The slice's strongest opponents are horned vipers, hard examples of another class, which the model considers to be similar to sidewinders. In the second slice, we observe two dominant labels. Here, vine snakes and analog clocks are often mis-predicted. A priori, one would not have guess the predictions, for these seemingly, unrelated classes would be wrong for the same reasons. However, looking at the opponents, we see all of them have a clock hand, which turns out to be similar to a snake. Also, because the opponents are of a variety of classes, the mis-predictions are also of a variety of classes. Overall, we demonstrate here that our procedure can identify problematic slices with low performance and then trace the cause of the errors to a subset of the training data.
4.4 AGNews & Mis-labelled Data
Overview & Experimental Procedure: We identify problematic slices in another "natural" dataset - the test split of AGNews [Zhang et al., 2015], comprising 7600 examples. This text-classification problem is a 4-class problem, and the pre-trained model is a BERT base model fine-tuned on the training dataset  $^3$ , achieving  $93\%$  test accuracy, making 475 errors. We use ImfEmbed with  $K = 25$  and examine slices with at most  $10\%$  accuracy, and at least 10 examples.
Results: We find 9 such slices, comprising a total of 452 examples, whose overall accuracy is only  $3\%$ . Table 3 shows 2 slices, both of which have label homogeneity. In the first slice, the model achieves  $0\%$  accuracy, systematically mis-predicting "business" examples to be "world", perhaps predicting any text with a country name in it to be "world". In the second slice, most labels are
<sup>3</sup>https://huggingface.co/fabriceyhc/bert-base-uncased-ag_news
Slice with 45 examples, 0% accuracy. Predictions: 100% “world”, labels: 100% “business”
1. “British grocer Tesco sees group sales rise 12.0-percent. Britain's biggest supermarket chain …”
2. “Nigerian Senate approves $1.5 bn claim on Shell LAGOS - Nigeria's Senate has passed …”
3. “Cocoa farmers issue strike threat. Unions are threatening a general strike in the Ivory Coast …”
Slice with 139 examples, 3% accuracy. Predictions: 93% “business”, labels: 94% “sci/tech”
1. “Google Unveils Desktop Search, Takes on Microsoft. Google Inc. rolled out a version of its ...”
2. “PalmOne to play with Windows Mobile? Rumors of Theo’s using a Microsoft operating system ...”
3. “Intel Posts Higher Profit, Sales. Computer-chip maker Intel Corp. said yesterday that earnings ...”
Table 3: Representative examples for 2 slices obtained using InfEmbed on AGnews. In the 1st slice, the model mis-predicts business text containing country names to be "world". The 2nd slice appears to be mis-labelled, showing InfEmbed can detect mis-labeled data.
"science/technology", and most predictions are "business". However looking at the examples, they may in fact be mis-labeled, as they are about businesses and their technology. This shows that InfEmbed can also identify mis-labeled data.
4.5 Detecting Spurious Signals in Boneage Classification
Overview & Experimental Procedure: We now artificially inject a signal that spuriously correlates with the label in training data, and see if InfEmbed-rule can detect the resulting test errors as well as the spurious correlation. Here, the classification problem is bone-age classification  $^{4}$  - to predict which of 5 age groups a person is in given their x-ray. Following the experimental protocol of Zhou et al. [2022] and Adebayo et al. [2021], we consider 3 possible spurious signals: tag, stripe, or blur (see Appendix for additional details). For a given signal, we manipulate the bone-age training dataset by injecting the spurious signal into all training examples from the "mid-pub" class (short for mid-puberty). Importantly, we leave the test dataset untouched, so that we expect a model trained on the manipulated training data to err on "mid-pub" test examples.
This setup mimics a realistic scenario where x-ray training data comes from a hospital where one class is over-represented, and also contains a spurious signal (i.e. x-rays from the hospital might have the hospital's tag). We then train a resnet50 on the manipulated training data, and apply InfEmbed-Rule on the untouched test data to discovered slices with at least 25 examples and at most  $40\%$  accuracy (same as for the Imagenet analysis). We repeat this manipulate-train-slice discovery procedure for 3 runs: once for each possible signal.
Results: For all 3 runs / signals, InfEmbed-Rule returned 2 slices satisfying the rule. From the distribution of labels and predictions, we observe that for each signal, the labels in the most problematic slice are of the "mid-pub" class, showing that InfEmbed-Rule is able to detect test errors due to the injected spurious training signal. Furthermore, for each run, around  $80\%$  of the slice's opponents are training examples with the spurious training signal, i.e. examples with the "mid-pub" label. Together, this shows InfEmbed-Rule can not only detect slices that the model errs on because of a spurious training signal, but also identify training examples with the spurious training signal by using slice opponents.
Error analysis, more generally, has been studied extensively even within the deep learning literature; however, here we focus our discussions on the recently formalized slice discovery problem—as termed by Eyuboglu et al. [2022b].
Error Analysis, Slice Discovery, Related Problems: To the best of our knowledge, InfEmbed is the first SDM based on influence functions. More broadly, it represents the first use of influence functions for clustering and global explainability; they have been used for local explainability, i.e. returning the influence explanation for a single test example, but not for clustering, as we do. Pruthi et al. [2020] also define influence embeddings, but use a different definition and also use it for a different purpose - accelerating the retrieval of an example's most influential training examples. Crucially, their embeddings were high-dimensional (exceeding the number of model parameters) and thus ill-suited
4https://www.kaggle.com/datasets/kmader/rsna-bone-age
for clustering, and lacked any dimension reduction procedure preserving the dot-product property, aside from random projections.
Previous SDMs use a variety of representations as input into a partitioning procedure. Some rely on external feature extractors, using PCA applied to pre-trained CLIP embeddings [Eyuboglu et al., 2022b] along with an "error-aware" mixture model, or features from a robustly pre-trained image model [Singla et al., 2021] along with a decision tree. This external reliance limits the generalizability of these SDMs—what if one wanted to perform slice discovery on a domain very different from the one CLIP was trained on, or wanted to consider a new modality, like audio or graphs? Other SDM's use the pre-trained model's last-layer representations [d'Eon et al., 2022], or transform them via SCVIS [Plumb et al., 2022] or UMAP [Sohoni et al., 2020] before applying K-Means clustering. While they are performant, it is less clear why this is. In contrast, InfEmbed does not rely on external feature extractors and is theoretically framed by our influence function-based definition of coherency.
Other methods solve problems related to, but not the same as slice discovery. Rajani et al. [2022], Wiles et al. [2022], Hua et al. [2022] leverages the aforementioned SDM's to create interactive systems for discovering slices, focusing on improving the user interface and how to explain slices. In contrast, we focus on solving the core slice discovery problem. All these methods use last-layer representations along with K-Means or an "error-aware" mixture model [Hua et al., 2022]. Jain et al. [2022] globally ranks examples to identify examples in a single failure mode, but being unable to output multiple slices, is not a SDM. Numerous works also rank examples using scores capturing various properties: the degree to which a sample is out-of-distribution [Liu et al., 2020], reliability [Schulam and Saria, 2019], or difficulty Simsek et al. [2022]. Complementary to SDMs are methods for identifying low-performance subgroups when tabular metadata is available [Lakkaraju et al., 2017].
Influence Functions: Influence functions [Koh and Liang, 2017] have been used to calculate influences, which can be used for ranking training examples for local explainability [Barshan et al., 2020] for both supervised and unsupervised models [Kong and Chaudhuri, 2021], identifying mis-labeled data interactively [Teso et al., 2021], data augmentation [Lee et al., 2020], active learning [Liu et al., 2021], quantifying reliability [Schulam and Saria, 2019], and identifying mis-labeled data [Pruthi et al., 2020]. However, influence functions have not been used for slice discovery or global explainability.
We present a method, InfEmbed, that addresses the slice discovery problem. Our proposed solution departs from previous work in that it both identifies slices—group of data points—on which a model under-performs, and that satisfy a certain coherence property. We formalize coherence—predictions being wrong for the same reasons within a slice—as a key property that all slice discovery methods should satisfy. In particular, we leverage influence functions [Koh and Liang, 2017] to define coherent slices as ones whose examples are influenced similarly by the training data, and then develop a procedure to return coherent slices based on clustering a representation we call influence embeddings. We demonstrate on several benchmarks that InfEmbed out-performs current approaches on a variety of data modalities. In addition, we show through case studies that InfEmbed is able to recover known errors in a variety of settings, and identifies coherent slices on realistic models.
Limitations. One major limitation of the proposed approach is that the validation set needs to include the kind of error that one is seeking to detect. In addition, the proposed procedure suggests to inspect the slice opponents as the root cause of the wrong predictions in a slice. However, it is still up to the user of the proposed Algorithm to infer which feature(s) of the slice opponents is responsible for causing the errors. Despite these limitations, error discovery by clustering influence embeddings can be seen as additional tool in the ML model debugging toolbox.
Fulton Wang, Sarah Tan, Diego Garcia-Olano and Narine Kokhlikyan were employed by Meta, while Julius Adebayo was employed by Genentech over the course of work on this project. No external funding was received or used by the authors over the course of the project.