SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Fulton Wang* Meta
Julius Adebayo*  
Prescient Design / Genentech
Sarah Tan Cornell University
Diego Garcia-Olano Meta
Narine Kokhlikyan Meta
We present a method for identifying groups of test examples—slices—on which a model under-performs, a task now known as slice discovery. We formalize coherence—a requirement that erroneous predictions, within a slice, should be wrong for the same reason—as a key property that any slice discovery method should satisfy. We then use influence functions to derive a new slice discovery method, InfEmbed, which satisfies coherence by returning slices whose examples are influenced similarly by the training data. InfEmbed is simple, and consists of applying K-Means clustering to a novel representation we deem influence embeddings. We show InfEmbed outperforms current state-of-the-art methods on 2 benchmarks, and is effective for model debugging across several case studies.[2]
Error analysis is a longstanding challenge in machine learning [Pope, 1976, Amershi et al., 2015, Sculley et al., 2015, Chung et al., 2019, Chakarov et al., 2016, Cadamuro et al., 2016, Kearns et al., 2018, Kim et al., 2019, Zinkevich Martin, 2020]. Recently, Eyuboglu et al. [2022b] formalized a type of error analysis termed the slice discovery problem. In the slice discovery problem, given a multi-class classification model and test data, the goal is to partition the test data into a set of slices—groups of test examples—by model performance. An effective slice discovery method (SDM) should satisfy two desiderata:
1. Error surfacing: Identify under-performing slices, i.e. slices with low accuracy, if they exist.
2. Coherence: Return slices that are coherent in the following sense: Erroneous predictions in a given slice should have the same root cause, i.e. be "wrong for the same reason".
To illustrate, suppose we have a pre-trained model that detects hip fractures from an x-ray, trained on data from one hospital, that is now deployed in a new hospital. The model might err frequently on two groups: 1) examples containing a spurious signal like the frequency signature of the scanner from the first hospital [Badgeley et al., 2019], and 2) examples from patients not represented in the new hospital. An effective SDM should uncover these two groups. The slice discovery problem has taken on renewed importance given that models trained via empirical risk minimization exhibit degraded performance on certain data groups—slices—due to distribution shift [Koh et al., 2021], noisy labels [Northcutt et al., 2021], and imbalanced data [Idrissi et al., 2022].
In developing a SDM, there are two key challenges. First, formalizing the coherence property, i.e., defining the root cause of an error, identifying whether two errors have similar root causes, and ultimately, defining whether a slice is coherent. Second, identifying the representation to use:
*The two authors contributed equally. Correspondence to fultonwang@meta.com
$^{2}$ Code to replicate our findings is available at: https://github.com/adebayoj/infembed
Figure 1: Schematic of the InfEmbed slice discovery method. In slice discovery, the goal is to partition the test points into groups—slices—by model performance. First, for each test point, we compute its influence embeddings—a low dimensional representation of the vector of influence scores of the training set. In the second stage, we cluster this influence embedding set to group points by training set influence.
returning to the x-ray example, there is typically no additional metadata that indicates whether an x-ray contains the spuriously correlated scanner pattern. The input to a SDM will typically be the x-ray or a representation thereof, without additional metadata that indicates which features are spurious. Thus, identifying a representation that can help distinguish under-performing groups from high performing ones while maintaining coherence is a key challenge.
Influence Explanations: To formalize the coherence property, we appeal to influence functions [Koh and Liang, 2017], which tractably estimate the "effect" of a training example on a test example—how much the loss of the test example would change if the model were re-trained without that training example. We define the influence explanation of a given test example to be the vector of the 'influence' of each training example on the test example. Since a test example's influence explanation is the "cause" of its prediction from a training dataset perspective, we say a SDM achieves coherence if the slices it returns have similar influence explanations.
Influence Embeddings: To create coherent slices, one might naively cluster test examples' influence explanations. However, each influence explanation is high dimensional—equal to the size of the training dataset. Instead, we transform influence explanations into influence embeddings. We define influence embeddings to be vectors with the dot-product property: that the influence of a training example on a test example is the dot-product of their respective embeddings.
Clustering Influence Embeddings: Given the coherence-inducing properties of influence embeddings, we propose InfEmbed, an SDM that applies K-means clustering to the influence embeddings of all test examples. Despite its simplicity, InfEmbed satisfies the two aforementioned desiderata. First, it is error-surfacing, as mis-predicted examples that are "wrong for the same reasons", i.e. have similar influence explanations, tend to be grouped into the same slice. Second, InfEmbed satisfies several desirable properties. It produces coherent slices, and can accommodate user-defined choices regarding the subset of a model's representations to use, ranging from all to only the last layer [d'Eon et al., 2022, Sohoni et al., 2020]. In addition, the slices produced by InfEmbed tend to have label homogeneity, i.e. examples are homogeneous in terms of the true and predicted label. Unlike other SDMs, these properties are implicit consequences the rigorous influence function-based derivation.
Contributions: We summarize our contributions as follows:
1. New slice discovery method: We propose InfEmbed, a SDM to identify under-performing groups of data that applies K-Means to a new representation: influence embeddings.
2. Coherence & Theoretical Justification. We formalize the coherence property, which requires that erroneous predictions within a slice should be "wrong for the same reason". We then define influence embeddings, and prove that a procedure which clusters them achieves coherence: it returns slices whose examples are influenced similarly by the training data.
3. Empirical Performance. We show that InfEmbed outperforms previous state-of-the-art SDMs on 2 benchmarks—DCBench [Eyuboglu et al., 2022a], and SpotCheck [Plumb et al., 2022]. InfEmbed is scalable, able to recover known errors in variety of settings, and identifies coherent slices on realistic models.
At a high level, the goal of the slice discovery problem is to partition a test dataset into slices-groups of data points—such that some slices are under-performing, i.e. have low accuracy, and slices are coherent, i.e. erroneous predictions in a given slice are "wrong for the same reason". We first formally define the slice discovery problem, and then overview influence functions, which we use to define what it means for a slice to be coherent - one of our key contributions, and which will be detailed in Section 3.
Slice Discovery Problem: Given a trained multi-class classification model  $f(\cdot; \theta)$  over  $C$  classes with model parameters  $\theta$  and an example  $x$  from some input space  $X$ ,  $f(x; \theta) \in \mathbb{R}^C$  is the prediction for the example and  $f(x; \theta)_c$  is the pre-softmax prediction for class  $c$ . Given a test dataset  $\mathbf{Z} := [z_1, \ldots, z_N]$  with  $N$  examples where  $z_i := (x_i, y_i)$  and  $x_i \in \mathcal{X}$ ,  $y_i \in \mathcal{Y} := [0, 1]^C$ , the goal of the slice discovery problem is to partition the test dataset into  $K$  slices:  $\{\Phi_k\}_{k=1}^K$ , where  $\Phi_k \subseteq [N]$ ,  $\Phi_k \cap \Phi_{k'} = \emptyset$  for  $k \neq k'$ ,  $\cup_{k=1}^K \Phi_k = [N]$ . As we will make precise shortly, each slice  $\Phi_k$  should be coherent, and some slices should be under-performing slices. The model is assumed to be trained on a training dataset with  $N'$  examples,  $\mathbf{Z}' := [z'_1, \ldots, z'_N']$  where  $z'_i := (x'_i, y'_i)$  with  $x'_i \in \mathcal{X}$ ,  $y'_i \in \mathcal{Y}$ , so that  $\theta = \operatorname*{argmin}_{\theta'} \frac{1}{N'} \sum_{i=1}^{N'} L(z'_i; \theta')$ . We assume the loss function  $L$  is cross-entropy loss. Thus, given test point  $z = (x, y)$  with  $x \in \mathcal{X}$ ,  $y = [y_1, \ldots, y_C] \in \mathcal{Y}$ ,  $L(z; \theta) = -\sum_c y_c \log p_c$ , with  $p = [p_1, \ldots, p_C]$  and  $p_c := \frac{\exp(f(x; \theta)_c)}{\sum_{c'} \exp(f(x; \theta)_{c'})}$ , i.e.  $p = \text{softmax}(f(x; \theta))$ .
Influence Functions: Influence functions estimate the effect of a given training example,  $z'$ , on a test example,  $z$ , for a pre-trained model. Specifically, the influence function approximates the change in loss for a given test example  $z$  when a given training example  $z'$  is removed from the training data and the model is retrained. Koh and Liang [2017] derive the aforementioned influence to be  $I(z', z) \coloneqq \nabla_{\theta} L(z'; \theta)^{\top} H_{\theta}^{-1} \nabla_{\theta} L(z; \theta)$ , where  $H_{\theta}$  is the loss Hessian for the pre-trained model:  $H_{\theta} \coloneqq 1 / n \sum_{i=1}^{n} \nabla_{\theta}^2 L(z; \theta)$ , evaluated at the pre-trained model's final parameter checkpoint. The loss Hessian is typically estimated with a random mini-batch of data.
The main challenge in computing influence is that it is impractical to explicitly form  $H_{\theta}$  unless the model is small, or if one only considers parameters in a few layers. Schioppa et al. [2022] address this problem by forming a low-rank approximation of  $H_{\theta}^{-1}$  via a procedure that does not explicitly form  $H_{\theta}$ . In brief, they run the Arnoldi iteration [Trefethen and Bau III, 1997] for  $P$  iterations to get a  $P$ -dimensional Krylov subspace for  $H_{\theta}$ , which requires only  $P$  Hessian-vector products. Then, they find a rank-  $D$  approximation of the restriction of  $H_{\theta}$  to the Krylov subspace, a small  $P \times P$  matrix, via eigendecomposition. Their algorithm, FactorHessian (see Appendix B for details), which we use in our formulation, returns factors of a low-rank approximation of  $H_{\theta}^{-1}$
$$
M, \lambda = \operatorname {F a c t o r H e s s i a n} \left(\mathbf {Z} ^ {\prime}, \Theta , P, D\right) \text {w h e r e} \tag {1}
$$
$$
\hat {H} _ {\theta} ^ {- 1} := M \lambda^ {- 1} M ^ {\top} \approx H _ {\theta} ^ {- 1}, \tag {2}
$$
$M \in \mathbb{R}^{\left|\theta\right| \times D}, \lambda \in \mathbb{R}^{D \times D}$  is a diagonal matrix,  $|\theta|$  is the parameter count,  $\hat{H}_{\theta}^{-1}$  approximates  $H_{\theta}^{-1}$ ,  $D$  is the rank of the approximation,  $P$  is the Arnoldi dimension, i.e. number of Arnoldi iterations, and here and everywhere, configuration  $\Theta := (L, f, \theta)$  denotes the loss function, model and parameters.  $\hat{H}_{\theta}^{-1}$  is then used to define the practical influence of training example  $z'$  on test example  $z$ :
$$
\hat {I} \left(z ^ {\prime}, z\right) := \nabla_ {\theta} L \left(z ^ {\prime}; \theta\right) ^ {\top} \hat {H} _ {\theta} ^ {- 1} \nabla_ {\theta} L (z; \theta). \tag {3}
$$
3 Error Discovery By Clustering Influence Embeddings
We propose a slice discovery method (SDM) whose key contribution is to require returned slices have coherence, and unlike past works, rigorously define coherence, which we do using influence functions. In this section, we derive our SDM as follows: 1) We formally define that a slice has coherence if its examples have similar influence explanations—which we define using influence functions, and leverage them to formalize a clustering problem. 2) Because influence explanations are high-dimensional, we derive influence embeddings, a low-dimensional similarity-preserving approximation. 3) Using influence embeddings, we give a simple and efficient procedure to solve the clustering problem to give our SDM, InfEmbed. 4) We then describe an extension, InfEmbed-Rule, which instead of returning a partition with a user-specified number of slices, returns the largest slices
satisfying user-specified rules. 5) We then show that the proposed formulation implicitly encourages a property, label homogeneity, that previous methods needed to explicitly encourage, in a manner that requires specifying a hard-to-tune hyperparameter. 6) Lastly, explain slices using training samples.
3.1 Influence Explanations and Problem Formulation
We seek to partition the test data into slices such that predictions for examples in the same slice have similar root causes as quantified by influence functions. Therefore, we propose to represent the root cause of the prediction for a test example,  $z$ , with its influence explanation  $E(z)$ :
$$
E (z) := \left\{\hat {I} \left(z ^ {\prime} _ {j}, z\right) \right\} _ {j = 1} ^ {N ^ {\prime}}, \tag {4}
$$
the vector containing the influence of every training example on the test example. Thus, test examples with similar influence explanations are "influenced similarly by the training data". We then want each slice be coherent in the following sense: the Euclidean distance between influence explanations of examples in the same slice should be low.
Algorithm 2 Our SDM, InfEmbed, applies K-Means to influence embeddings of test examples.
1: procedure GETEMBEDDINGS(Z, Z', Θ, P, D)  
2: Outputs: D-dimensional embeddings for test data Z  
3: M, λ ← FactorHessian(Z', Θ, P, D) ▷ see Equation 2  
4: μi ← λ-1/2 M'∇θL(zi; θ) for i = 1,..., N ▷ see Equation 6  
5: μ ← [μ1,..., μN]  
6: Return: μ  
7: end procedure  
8: procedure INFEMBED(K, Z, Z', Θ, P, D)  
9: Inputs: number of slices K, training dataset Z', test dataset Z, configuration Θ, Arnoldi dimension P, influence embedding dimension D  
10: Outputs: partition of test dataset into slices, Φ  
11: μ ← GetEmbeddings(Z, Z', Θ, P, D)  
12: [r1,..., rN] ← K-Means(μ, K) ▷ compute cluster assignments for all N examples  
13: Return: {{{i ∈ [N] : ri = k} for k ∈ [K]}} ▷ convert cluster assignments to a partition  
14: end procedure
Therefore, we solve the slice discovery problem described in Section 2 via the following formulation—return a partition of the test dataset into slices,  $\{\Phi_k\}_{k=1}^K$ , that minimizes the total Euclidean distance between influence explanations of examples in the same slice. Formally, we seek:
$$
\operatorname {a r g m i n} _ {\left\{\Phi_ {k} \right\} _ {k = 1} ^ {K}} \sum_ {k} \sum_ {i, i ^ {\prime} \in \Phi_ {k}} \left\| E \left(z _ {i}\right) - E \left(z _ {i ^ {\prime}}\right) \right\| ^ {2}. \tag {5}
$$
3.2 Influence Embeddings to approximate Influence Explanations
To form coherent slices, a naive approach would be to form the influence explanation of each test example, and apply K-Means clustering with Euclidean distance to them. However, influence explanations are high-dimensional, with dimensionality equal to the size of the training data,  $N'$ . Instead, we will use influence embeddings: vectors such that the practical influence of a training example on a test example is the dot-product of their respective influence embeddings. Looking at Equations 2 and 3, we see the influence embedding  $\mu(z)$  of an example  $z$  must be defined as
$$
\mu (z) := \lambda^ {- 1 / 2} M ^ {\intercal} \nabla_ {\theta} L (z; \theta) \tag {6}
$$
and  $M, \lambda$  are as defined in Section 2, because for any training example  $z'$  and test example  $z$ ,  $\hat{I}(z', z) = \mu(z')^{\top} \mu(z)$ . Procedure GetEmbeddings of Algorithm 1 finds  $D$ -dimensional influence embeddings for the test dataset  $\mathbf{Z}$ , which has runtime linear in the Arnoldi dimension  $P$ . The rank  $D$  of the factors from FactorHessian determines the dimension of the influence embeddings.
Influence embeddings satisfy a critical property—that if two examples have similar influence embeddings, they also tend to have similar influence explanations. This is formalized by the following lemma, whose proof follows from the Cauchy-Schwartz inequality, and that influence is the dot-product of influence embeddings (see Appendix Section B.1)
Lemma 1: There is a constant  $C > 0$  such that for any test examples  $z_{i},z_{j},||E(z_{i}) - E(z_{j})||^{2}\leq C||\mu (z_{i}) - \mu (z_{j}))||^{2}$
3.3 InfEmbed: Discovering Problematic Slices by Clustering Influence Embeddings
We will solve the formulation of Equation 5 via a simple procedure which applies K-Means with Euclidean distance to the influence embeddings of the test dataset,  $\pmb{\mu} \coloneqq [\mu(z_1), \dots, \mu(z_N)]$ . This procedure is justified as follows: Using Lemma 1, given any partition  $\{\Phi_k\}$ , we know
$$
\sum_ {k} \sum_ {i, i ^ {\prime} \in \Phi_ {k}} | | E (z _ {i}) - E (z _ {i ^ {\prime}}) | | ^ {2} \leq C \sum_ {k} \sum_ {i, i ^ {\prime} \in \Phi_ {k}} | | \mu (z _ {i}) - \mu (z _ {i ^ {\prime}}) | | ^ {2}. \tag {7}
$$
However, the quantity in the right of Equation 7 is the surrogate objective minimized by this procedure—it is the K-Means objective applied to influence embeddings, scaled by  $C > 0$  which does not matter. The quantity in the left of Equation 7 is the actual objective which the formulation of Equation 5 minimizes. Therefore, the procedure is minimizing a surrogate objective which upper bounds the actual objective we care about. Put another way, by applying K-Means to influence embeddings, examples within the same slice will not only have similar influence embeddings, but also similar influence explanations as desired, by Lemma 1. Algorithm 1 describes InfEmbed. In addition, we find that normalizing the centroid centers in each iteration of K-Means is helpful, perhaps due to lessening the effect of outlier influence embeddings. Overall, instead of K-means, other clustering algorithms can be used as part of formulation.
Computational Complexity: We now take a step back to examine the computational complexity of the proposed procedure. The main computational bottleneck is the implicit Hessian estimation required to compute influence embeddings. Schioppa et al. [2022]'s approach, which we rely on, has complexity  $\mathcal{O}(P)$  where  $P$  is the Arnoldi dimension. This is because each of the  $P$  steps in the Arnoldi iteration requires computing a Hessian-vector product. Consequently, the complexity of computing influence embeddings is exactly the same as that of the influence function (IF) estimation. For the K-means portion, the complexity is  $\mathcal{O}$  (number of samples  $\times$  number of k-means iterations  $\times$  number of clusters). In practice, the clustering step is near instantaneous, so the method is dominated by implicit Hessian estimation step.
3.4 InfEmbed-Rule: Finding Slices Satisfying a Rule
The key hyper-parameter of the InfEmbed method is  $K$ , the number of slices to return. In practice, it may not be intuitive for a user to choose  $K$ . Instead, the user may want to know if there exist any coherent slices that are problematic, where 'problematic' is pre-specified by a rule, e.g., a slice with greater than  $T$  samples and that has accuracy less than  $A$ . Without specifying the rule, a practitioner has to iterate through all the returned slices to figure out which one is problematic. Therefore we propose a procedure that recursively clusters influence embeddings until slices satisfying a pre-specified rule are found, or until the slices are too small.
The proposed approach, InfEmbed-Rule, is analogous to building a tree to identify slices satisfying the rule, where the splits are determined by K-Means clustering of influence embeddings. In addition to letting the user specify more intuitive hyperparameters, this procedure also has the advantage that if a large slice with sufficiently low accuracy is found, it will not be clustered further. Appendix Algorithm B.2 outlines this InfEmbed-Rule method. Its inputs are the same as InfEmbed, except instead of  $K$ , one specifies accuracy threshold  $A$ , size threshold  $S$ , and branching factor  $B$ , which sets how many clusters the K-means call in each step of the recursion should return. It outputs a set of slices each with accuracy less than  $A$  and size greater than  $S$ .
3.5 Properties of Discovered Slices
We now show that the slices discovered by InfEmbed possess properties similar to past SDM's. To derive these properties, we consider what factors result in two examples  $z = (x,y)$ ,  $z' = (x',y')$  having influence embeddings with low Euclidean distance, so that K-Means would place them in the same slice. To simplify analysis, we note that two examples would be placed in the same slice if their gradients have high dot-product, because influence embeddings are linearly transformed gradients (see Equation 6) and Euclidean distance is equal to their negative dot-product plus a constant. For further simplicity, we consider the case when the model  $f$  is a linear model, i.e. when only gradients in the last fully-connected layer are considered when computing influence embeddings. Then, a straightforward computation shows that two examples  $z, z'$  will be placed in the same slice if
$$
\nabla_ {\theta} L (z; \theta) ^ {\intercal} \nabla_ {\theta} L \left(z ^ {\prime}; \theta\right) = (y - p) ^ {\intercal} \left(y ^ {\prime} - p ^ {\prime}\right) x ^ {\intercal} x ^ {\prime}, \tag {8}
$$
is high, where  $L$  is cross-entropy loss,  $\theta$  are the parameters, and for example  $z, p \coloneqq [p_1, \dots, p_C] = \mathrm{softmax}(f(x))$  is the predicted probabilities for each class,  $y$  is the one-hot encoded label,  $x$  is its last-layer representation, and  $p', y', x'$  are defined analogously for example  $z'$ .
Looking at the  $x^{\top}x^{\prime}$  term of Equation 8, we see that two examples will be placed in the same slice if, all else being equal, their last-layer representations are similar as measured by the dot-product. Past SDM's [d'Eon et al., 2022, Sohoni et al., 2020] also use heuristic similarity measures involving last-layer representations to decide if two examples should be in the same slice. On the other hand, our influence function-based derivation shows that their dot-product is the "canonical" similarity measure to use. Note that since influence embeddings consider gradients in layers beyond the last, they are able to consider information beyond the last-layer representations, generalizing past SDM's.
Looking at the  $(y - p)^{\mathsf{T}}(y' - p')$  term of Equation 8, we see that two examples will be placed in the same slice if, all else being equal, their margins are similar, as measured by dot-product. The margin of an example, eg.  $(y - p)$  or  $(y' - p')$ , is the vector containing the difference between the true label and predicted probability for each class. Thus, examples with similar labels and predictions will tend to be in the same slice, i.e. slices tend to have label homogeneity. Past SDM's [Eyuboglu et al., 2022b] also produce slices that tend to have label homogeneity, but do so through an "error-aware" mixture model that requires choosing a hard-to-tune tradeoff parameter. On the other hand, our influence function-based derivation shows that the dot-product of margins is the "canonical" way to encourage label homogeneity, and does not require tuning a trade-off constant. Note that label homogeneity is not the only factor in consideration when forming slices.
3.6 Slice Explanation via Slice Opponents
Given an under-performing slice, we are interested in identifying the root-cause of the erroneous predictions in it. We therefore compute the top- $k$  slice opponents and consider them as the root-cause. These are the  $k$  training examples whose influence on the slice are most harmful, i.e. whose influence on the total loss over all examples in the slice is the most negative.