SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jang-Hyun Kim
Seoul National University janghyun@mllab.snu.ac.kr
Sangdoo Yun
NAVER AI Lab  
sanggoo.yun@navercorp.com
Hyun Oh Song*
Seoul National University hyunoh@mllab.snu.ac.kr
Diagnosing and cleaning data is a crucial step for building robust machine learning systems. However, identifying problems within large-scale datasets with real-world distributions is challenging due to the presence of complex issues such as label errors, under-representation, and outliers. In this paper, we propose a unified approach for identifying the problematic data by utilizing a largely ignored source of information: a relational structure of data in the feature-embedded space. To this end, we present scalable and effective algorithms for detecting label errors and outlier data based on the relational graph structure of data. We further introduce a visualization tool that provides contextual information of a data point in the feature-embedded space, serving as an effective tool for interactively diagnosing data. We evaluate the label error and outlier/out-of-distribution (OOD) detection performances of our approach on the large-scale image, speech, and language domain tasks, including ImageNet, ESC-50, and SST2. Our approach achieves state-of-the-art detection performance on all tasks considered and demonstrates its effectiveness in debugging large-scale real-world datasets across various domains. We release codes at https://github.com/snu-mllab/Neural-Relation-Graph.
Identifying problems within datasets is crucial for improving the robustness of machine learning systems and analyzing the model failures [45]. For instance, identifying mislabeled or uninformative data helps construct concise and effective training datasets [33], while identifying whether test data is OOD or corrupted allows for more accurate model evaluation and analysis [54].
In recent years, efforts have been made to identify problematic data by utilizing unary scores on individual data from trained models, such as estimating data influence [22], monitoring prediction variability throughout training [52], and calculating prediction error margins [34]. However, identifying such data can be challenging, particularly when dealing with large-scale datasets from real-world distributions. In real-world settings, datasets may have complex problems, including label errors, under-representation, and outliers, each of which can lead to the model error and prediction sensitivity [23]. For example, Figure 1 shows that a neural network exhibits low negative prediction margins and high loss values for both a sample with label error and outlier data. This observation indicates that previous unary scoring methods may have limitations in discerning whether the problem lies with the label or the data itself.
In this work, we propose a unified framework for identifying label errors and outliers by leveraging the feature-embedded structure of a dataset that provides richer information than individual data alone [47, 36]. We measure the relationship among data in the feature embedding space while comparing the assigned labels independently. By comparing input data and labels separately, we are able to
Sorrel (clean) / Margin: 0.87, Loss: 0.1
Figure 1: ImageNet samples with their labels and the corresponding relation maps by an MAE-Large model [13]. We report the prediction margin score  $(\in [-1,1])$  and the loss value next to the label. The relation map draws a scatter plot of the mean and variance of relation values of a data pair throughout the training process. Here the color represents the relation value at the last converged checkpoint. We present the detailed procedure for generating the relation maps in Section 3.5.
Coil (label error) / Margin: -0.89, Loss: 4.1
Envelope (outlier) / Margin: -0.84, Loss: 3.6
Figure 2: The conceptual illustration of the conventional approaches (left) and our proposed approach (right). In the relation graph, positive edges signify complementary relations, negative edges denote conflicting relations, and dashed lines indicate negligible relations between data.
isolate the factors contributing to model errors, resulting in improved identification of label errors and outlier data, respectively. Based on the relational information, we construct a novel graph structure on the dataset and identify whether the data itself or the label is problematic (Figure 2). To this end, we develop scalable graph algorithms that accurately identify label errors and outlier data points.
In Section 3.5, we further introduce a visualization tool named data relation map that captures the relational structure of a data point. Through the map, we can understand the underlying relational structure and interactively diagnose data. In Figure 1, we observe different patterns in the relation maps of the second and third samples, despite their similar margin and loss scores. This highlights that the relational structure provides complementary information not captured by the unary scoring methods.
Our approach only requires the model's feature embedding and prediction score on data, making it more scalable compared to methods that require calculating the network gradient on each data point or retraining models multiple times to estimate data influence [39, 17]. Furthermore, our method is domain- and model-agnostic, and thus is applicable to various tasks. We evaluate our approach on label error and outlier/OOD detection tasks with large-scale image, speech, and language datasets: ImageNet [43], ESC-50 [37], and SST2 [55]. Our experiments show state-of-the-art performance on all tasks, demonstrating its effectiveness for debugging and cleaning datasets over various domains.
Label error detection Label errors in datasets can negatively impact model generalization and destabilize evaluation systems [16, 34]. Prior works address this issue through label error detection using bagging and bootstrapping [46, 41], or employing neural networks [18, 27, 9, 19]. To mitigate overfitting on label errors, Pleiss et al. [38] propose tracking the training process to measure the area under the margin curve. Recent studies demonstrate that simple scoring methods with large pre-trained models, such as prediction margins or loss values, achieve comparable results to previous complex approaches [33, 5]. Meanwhile, Wu et al. [57] propose a unified approach for learning with open-world noisy data. However, the method involves a complicated optimization process during training, which is not suitable for large-scale settings. Another line of approach to identifying label errors involves measuring the influence of a training data point on its own loss [22, 39]. However, these approaches require calculating computationally expensive network gradients on each data point, and their performance is known to be sensitive to outliers and training schemes [2, 3]. In this work, we present a scalable approach that leverages the data relational structure of trained models without additional training procedures, facilitating practical analysis of label issues.
Outlier/OOD detection Detecting outlier data is crucial for building robust machine learning systems in real-world environments [23]. A recent survey paper defines the problem of finding outliers in training set as outlier detection and finding outliers in the inference process as OOD detection [59]. The conventional approach for detecting outliers involves measuring k-nearest distance using efficient sampling methods [48]. More recently, attempts have been made to detect outlier data using scores obtained from trained neural networks, such as Maximum Softmax Probability [14], Energy score [29], and Max Logit score [15]. Other approaches suggest adding perturbations on the inputs or rectifying the activation values to identify the outlier data [28, 49]. Lee et al. [26] propose fitting a Gaussian probabilistic model to estimate the data distribution. Recently, Sun et al. [50] propose a non-parametric approach measuring the  $k$ -nearest feature distance. In our work, we explore the use of the relational structure on the feature-embedded space for identifying outlier data. Our approach is applicable to a wide range of domains without requiring additional training while outperforming existing scoring methods on large-scale outlier/OOD detection benchmarks.
In this section, we describe our method for identifying label errors and outliers using a model trained on the noisy training dataset. We exploit the feature-embedded structure of the learned neural networks, which are known to effectively capture the underlying semantics of the data [40]. We define data relation to construct a data relation graph on the feature space, and introduce our novel graph algorithms for identifying label errors and outlier data. In Section 3.5, we introduce the data relation map as an effective visualization tool for diagnosing and contextualizing data.
3.1 Data relation
We describe our approach in the context of a classification task, while also noting that the ideas are generalizable to other types of tasks as well. We assume the presence of a trained neural network on a noisy training dataset with label errors and outliers,  $\mathcal{T} = \{(x_i,y_i) \mid i = 1,\dots,n\}$ . By utilizing data features extracted from the network, we measure the semantic similarity between data points with a bounded kernel  $k:\mathcal{X}\times \mathcal{X}\to [0,M]$ , where a higher kernel value indicates greater similarity between data points. Our framework can accommodate various bounded kernels such as RBF kernel or cosine similarity [60]. We provide detailed information on the kernel function used in our main experiments in Section 3.4.
By incorporating the assigned label information with the similarity kernel  $k$ , we define the relation function  $r: \mathcal{X} \times \mathcal{Y} \times \mathcal{X} \times \mathcal{Y} \to [-M, M]$ :
$$
r \left(\left(x _ {i}, y _ {i}\right), \left(x _ {j}, y _ {j}\right)\right) = 1 \left(y _ {i} = y _ {j}\right) \cdot k \left(x _ {i}, x _ {j}\right), \tag {1}
$$
where  $1(y_{i} = y_{j})\in \{-1,1\}$  is a signed indicator value. The relation function reflects the degree to which data samples are complementary or conflicting with each other. In Figure 3, the center image with a label error has negative relations to the left samples that belong to the same ground-truth class. In contrast, the two left samples with correct labels have a positive relation. We also note that samples with dissimilar semantics exhibit near-zero relations.
Our relation function  $r$  relies solely on the parallelizable forward computation of neural networks, ensuring scalability in large-scale settings.
Figure 3: Relation values of samples from ImageNet with MAE-Large [13]. We denote the assigned label above each sample. Here, the center image has a label error.
3.2 Label error detection
We consider a fully-connected undirected graph  $\mathcal{G} = (\mathcal{V},\mathcal{E},\mathcal{W})$ , where the set of nodes  $\mathcal{V}$  corresponds to  $\mathcal{T}$  and the weights  $\mathcal{W}$  on edges  $\mathcal{E}$  are the negative relation values defined in Equation (1). For notation clarity, we denote a data point by an index, i.e.,  $\mathcal{T} = \{1,\dots ,n\}$ . Then, for nodes  $i$  and  $j$ , the edge weight is  $w(i,j) = -r(i,j) = -r((x_i,y_i),(x_j,y_j))$ . We set  $w(i,i)$  to 0, which does not
Figure 4: Illustration of our scoring algorithms for identifying label noise (left) and outliers (right).
correspond to any edges in the graph. Consistent with previous works [33], we aim to measure the label noisiness score for each data, where a higher score indicates a higher likelihood of label error. We denote the label noisiness scores for  $\mathcal{T}$  as  $s\in \mathbb{R}^n$ , where  $s[i]$  is the score for data  $i$ .
As depicted in Figure 3, data with label errors exhibit negative relations with other samples, implying that the data have similar features in the embedding space yet have dissimilarly assigned labels. This suggests that the edge weights  $w(i,\cdot)$  quantify the extent to which the label assigned to node  $i$  conflicts with the labels of other nodes. However, simply aggregating all edge weights of a node can yield suboptimal results, as negative relations can also contribute to the score for clean data, as shown in Figure 3. In Appendix D.2, we provide a more detailed experimental analysis of this issue.
To rectify this issue, we develop an algorithm that considers the global structure of the graph instead of simply summing the edge weights of individual nodes. Specifically, we identify subsets of data likely to have correct/incorrect labels and calculate the label noisiness score based on the subsets. We partition the nodes in  $\mathcal{T}$  into two groups, where  $\mathcal{N} \subset \mathcal{T}$  denotes the estimated noisy subset and  $\mathcal{T} \setminus \mathcal{N}$  denotes the clean subset. To optimize  $\mathcal{N}$ , we aim to maximize the sum of the edges between the two groups, indicating that the label information of the two groups is the most conflicting. To ensure that  $\mathcal{N}$  contains data with incorrect labels, which constitute a relatively small proportion of  $\mathcal{T}$ , we impose regularization to the cardinality of  $\mathcal{N}$  with  $\lambda \geq 0$  and formulate the following max-cut problem:
$$
\mathcal {N} ^ {*} = \underset {\mathcal {N} \subset \mathcal {T}} {\operatorname {a r g m a x}} \operatorname {c u t} (\mathcal {N}, \mathcal {T} \backslash \mathcal {N}) \left(:= \sum_ {i \in \mathcal {N}} \sum_ {j \in \mathcal {T} \backslash \mathcal {N}} w (i, j)\right) - \lambda | \mathcal {N} |. \tag {2}
$$
The max-cut problem is NP-complete [11]. To solve this problem, we adopt the Kerningham-Lin algorithm, which finds a local optimum by iteratively updating the solution [20]. However, the original algorithm that swaps data one by one at each optimization iteration is not suitable for large-scale settings. To this end, we propose an efficient set-level algorithm in Algorithm 1 that alternatively updates the noisy set  $\mathcal{N}$  and label noisiness score vector  $s$ .
Specifically, given the current estimation of  $\mathcal{N}$ , the cut value excluding edges of node  $i\in \mathcal{T}$  is  $\mathrm{cut}(\mathcal{N}\backslash \{i\},\mathcal{T}\backslash \mathcal{N}\backslash \{i\})$ . Algorithm 1 measures the label noisiness score of node  $i$  by comparing the objective cut values when including  $i$  in  $\mathcal{N}$  and when including  $i$  in  $\mathcal{T}\backslash \mathcal{N}$ :
$$
s [ i ] = \operatorname {c u t} (\mathcal {N} \cup \{i \}, \mathcal {T} \backslash \mathcal {N} \backslash \{i \}) - \operatorname {c u t} (\mathcal {N} \backslash \{i \}, \mathcal {T} \backslash \mathcal {N} \cup \{i \}) = \sum_ {j \in \mathcal {T} \backslash \mathcal {N}} w (i, j) - \sum_ {j \in \mathcal {N}} w (i, j).
$$
Here we use the assumption  $w(i,i) = 0$ . In practice, the cardinality of  $\mathcal{N}$  is small, so we can efficiently update the score vector  $s$  by caching the initial score vector  $\bar{s}$  as in Algorithm 1. After calculating the score vector  $s$ , we update the noisy set  $\mathcal{N}$  by selecting nodes with score values above the value  $\lambda$ . Figure 4 illustrates the optimization process. Here larger values of  $\lambda$  result in smaller  $\mathcal{N}$  consisting of data samples that are more likely to have label noise. We provide the sensitivity analysis of  $\lambda$  in Appendix C.1, with Table 7.
Algorithm 1 satisfies the convergence property in Proposition 1. In Appendix A, we provide proof and present an empirical convergence analysis on large-scale datasets. We also conduct a runtime
Algorithm 1 Label noise identification
Input: Relation function  $r (= -w)$
Notation: The number of data  $n$
$\bar{s} [i] = \sum_{j = 1}^{n}w(i,j)$  # caching initial score
$s = \bar{s}$
$\mathcal{N} = \{i\mid s[i] > \lambda ,i\in [1,\dots ,n]\}$
$s[i]\gets \bar{s} [i] - 2\sum_{j\in \mathcal{N}}w(i,j)$
Output:  $s, \tilde{\mathcal{N}}$
analysis of our algorithm in Appendix A.3, demonstrating that the computation overhead of Algorithm 1 is negligible in large-scale settings.
Proposition 1. Algorithm 1 with a single node update at each iteration converges to local optimum.
Complexity analysis The time complexity of Algorithm 1 is  $O(n^{2})$ , proportional to the number of edges in a graph. It is noteworthy that our method maintains the best performance when used with graphs consisting of a small number of nodes, as shown in Figure 5. This implies that we can partition large datasets and run the algorithm repeatedly for each partition to enhance efficiency while maintaining performance. In this case, the complexity becomes  $O(n / k \cdot k^2) = O(nk)$ , with  $k$  representing the size of each partition and  $n / k$  being the number of partitions. Also, computations on these partitions are embarrassingly parallelizable, meaning that the complexity becomes  $O(k^2)$  for  $k \ll n$  in distributed computing environments.
3.3 Outlier/OOD detection
In the previous section, we presented a method for detecting label errors based on data relations with similar feature embeddings but different label information. By employing the identical feature embedding structure, we identify outlier data by measuring the extent to which similar data are absent in the feature embedding space. To quantify the extent of a data point being an outlier, we aggregate the similarity kernel values of a data point in Equation (3), thereby processing the entire relational information of the data point. Our approach leverages global information about the data distribution, resulting in a more robust performance across a range of experimental settings compared to existing methods that rely on local information such as  $k$ -nearest distance [50]. Specifically, for a subset  $S \subseteq T$  and data  $x$ , we measure the outlier score as
$$
\operatorname {o u t l i e r} (x) = \frac {1}{\sum_ {i \in \mathcal {S}} k \left(x , x _ {i}\right)}.
$$
Higher values in the outlier score indicate that the data are more distributionally outliers. We propose to use a uniform random sampling for  $S$ , adjusting the computational cost and memory requirements for the outlier score calculation to suit the inference environment. In Section 4.2, we verify our method maintains the best OOD detection performance even when using only  $0.4\%$  of the data in ImageNet.
3.4 Proposed similarity kernel
For  $x_{i} \in \mathcal{X}$ , we extract the feature representation  $\mathbf{f}_{i}$  and the prediction probability vector  $\mathbf{p}_{i}$  from the trained model. We propose a class of bounded kernel  $k: \mathcal{X} \times \mathcal{X} \to [0, M]$  with the following form:
$$
k \left(x _ {i}, x _ {j}\right) = \left| s \left(\mathbf {f} _ {i}, \mathbf {f} _ {j}\right) \cdot c \left(\mathbf {p} _ {i}, \mathbf {p} _ {j}\right) \right| ^ {t}. \tag {3}
$$
A positive scalar value  $t$  controls the sharpness of the kernel value distribution. A larger value of  $t$  makes a small kernel value smaller, which is effective in handling small noisy kernel values. A scalar value  $s(\mathbf{f}_i,\mathbf{f}_j)\in \mathbb{R}^+$  denotes a similarity measurement between features. In our main experiments, we adopt the truncated cosine-similarity that has been widely used in representation learning [44, 42]. We use the hinge function at zero, resulting in the following positive feature-similarity function:
$$
s \left(\mathbf {f} _ {i}, \mathbf {f} _ {j}\right) = \max  \left(0, \cos \left(\mathbf {f} _ {i}, \mathbf {f} _ {j}\right)\right).
$$
It is worth noting the utility of our framework is not limited to a specific kernel design. In Section 4.3, we verify our approach maintains the best performance with  $s(\mathbf{f}_i,\mathbf{f}_j)$  defined as the RBF kernel [60].
While the feature similarity captures the meaningful semantic relationship between data points, we observe that considering the prediction scores  $\mathbf{p}_i$  can further improve the identification of problematic data. To incorporate prediction scores into our approach, we introduce a scalar term  $c(\mathbf{p}_i,\mathbf{p}_j)$  that measures the compatibility between the predictions on data points. Any positive and bounded compatibility function is suitable for the kernel class defined in Equation (3). In our main experiments, we use the predicted probability of belonging to the same class as the compatibility term  $c(\mathbf{p}_i,\mathbf{p}_j)$ . Specifically, given the predicted label random variables  $\hat{y}_i$  and  $\hat{y}_j$ , the proposed compatibility term is
$$
c \left(\mathbf {p} _ {i}, \mathbf {p} _ {j}\right) = P \left(\hat {y} _ {i} = \hat {y} _ {j}\right) = \mathbf {p} _ {i} ^ {\intercal} \mathbf {p} _ {j}. \tag {4}
$$
From a different perspective, we interpret this term as a measure of confidence for feature similarity. In Section 4.3, we verify the effectiveness of the compatibility term through an ablation study.
Interpretation To better understand our relation function with kernel defined in Equation (3), we draw a connection to the influence function [39], which estimates the influence between data points by computing the inner product of the network gradient on the loss function  $\ell$  of each data point as  $\nabla_w\ell (x_i)^\top \nabla_w\ell (x_j)$ , where  $w$  denotes the network weights. Following the convention, we consider an influence function on the feed-forward layer, where  $\ell (x_{i}) = h(\mathbf{f}_{i}^{\prime}) = h(w^{\top}\mathbf{f}_{i})$ . By the chain rule, we can decompose the network gradient as  $\nabla_w\ell (x_i) = \nabla_{\mathbf{f}'}h(\mathbf{f}_i')\mathbf{f}_i^\top$ , and represent the influence as  $\nabla_{\mathbf{f}'}h(\mathbf{f}_i')^\top\nabla_{\mathbf{f}'}h(\mathbf{f}_j')\cdot \mathbf{f}_i^\top\mathbf{f}_j$ . Our relation function differs from the influence function in that it does not rely on feature gradients  $\nabla_{\mathbf{f}'}h(\mathbf{f}_i')$  to evaluate the relationship between data points. Instead, our relation function compares model predictions and assigned labels independently using the terms  $c(\mathbf{p}_i,\mathbf{p}_j)$  and  $1(y_{i},y_{j})$ . Our formulation does not require computationally expensive back-propagation and more robustly identifies conflicting data information than influence functions which are known to be sensitive to outliers [2]. We provide a more detailed theoretical analysis in Appendix A.4.
3.5 Data relation map
In this section, we present a visualization method based on our data relation function to contextualize data and comprehend its relational structure. One of the effective approaches for visualizing a dataset is dataset cartography [51], which projects the dataset onto a 2D plot. This approach draws a scatter plot of the mean and standard deviation of the model's prediction probabilities for each data sample during training. Inspired by the dataset cartography, we propose a data relation map, which visualizes the relationship between data along the training process. To this end, we uniformly store checkpoints during training. We denote a set of these checkpoints as  $\mathcal{K}$ , where  $r_k$  refers to the relation function for checkpoint  $k \in \mathcal{K}$ . For each data sample  $i \in \mathcal{T}$ , we draw a scatter plot of the mean and standard deviation of relation values  $\{r_k(i,j) \mid k \in \mathcal{K}\}$  for  $j \in \mathcal{T} \setminus \{i\}$ .