Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Authors: Shuangpeng Han
Mengmi Zhang
Venue: ICML 2025 poster (2025)
Keywords: error prediction
Primary area: deep_learning->robustness
TL;DR: We present a novel approach where one AI model is utilized to predict the errors made by another AI model.

Abstract: AI models make mistakes when recognizing images—whether in-domain, out-of-domain, or adversarial. Predicting these errors is critical for improving system reliability, reducing costly mistakes, and enabling proactive corrections in real-world applications such as healthcare, finance, and autonomous systems. However, understanding what mistakes AI models make, why they occur, and how to predict them remains an open challenge. Here, we conduct comprehensive empirical evaluations using a "mentor" model—a deep neural network designed to predict another "mentee" model’s errors. Our findings show that the mentor excels at learning from a mentee's mistakes on adversarial images with small perturbations and generalizes effectively to predict in-domain and out-of-domain errors of the mentee. Additionally, transformer-based mentor models excel at predicting errors across various mentee architectures. Subsequently, we draw insights from these observations and develop an "oracle" mentor model, dubbed SuperMentor, that can outperform baseline mentors in predicting errors across different error types from the ImageNet-1K dataset. Our framework paves the way for future research on anticipating and correcting AI model behaviors, ultimately increasing trust in AI systems. Our data and code are available at [here](https://github.com/ZhangLab-DeepNeuroCogLab/UnveilAIBlindSpot).

Unveiling Al's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Shuangpeng Han
Mengmi Zhang 12
Abstract
(OOD), or adversarial examples. In-domain errors occur when models misclassify familiar data within the training domain, while OOD errors arise when faced with unseen or out-of-domain data. Adversarial errors are particularly concerning, as they result from carefully crafted perturbations designed mislead the model.
Al models make mistakes when recognizing images-whether in-domain, out-of-ooman, or adversarial. Predicting these errors is critical for improving system reliability, reducing costly mistakes, and enabling proactive corrections in real-world applications such healthcare, finance, and autonomous systems. However, understanding what mistakes AI models make, why they occur, and how to predict them remains an open challenge. Here, we conduct comprehensive empirical evaluations using "mentor" model-a a deep neural network designed to predict another "mentee" model's errors. Our findings show that the mentor excels at learning from mentee's mistakes on adversarial images with small perturbations and generalizes effectively to predict in-domain and out-of-domain errors of the mentee. Additionally, transformer-bas based mentor models excel at predicting errors across various mentee architectures. Subsequently,
Accurately predicting these errors is critical to enhancing the robustness and reliability of AI, especially in high-stakes real-world applications such as healthcare (Habehh & Gohel, 2021), finance (Mashrur et al., 2020), and autonomous driving (Huang et al., 2022). Proactively identifying potential errors enables more efficient corrections, reducing costly mistakes and safeguarding against catastrophic failures. By predicting when models are likely to err, can implement strategies that either mitigate or entirely avoid the risks associated with those errors, ultimately leading to more trustworthy AI deployments.
Understanding the specific types of errors AI systems make, the reasons why they make these errors, and most importantly, how to predict these errors remains unresolved challenge. literature on error monitoring systems for Al models encompasses various approaches, including uncertainty estimation (Nado et al., 2021; Lakshminarayanan et al., 2017), anomaly detection (Bogdoll al., 2022), outlier detection (Boukerche
Existing
draw insights from these observations and develop an "oracle" mentor model, dubbed SuperMentor, that can outperform baseline mentors in predicting errors across different error types from the ImageNet-Il IK dataset. Our framework paves the way for future research on anticipating and correcting Al model behaviors, ultimately increasing trust in Al systems. Our data and code are available at here.
al., 2020), and out-of-domain detection (Yang al., 2024). While these methods crucial for assessing model reliability, they mainly focus determining whether a given data point falls outside the scope of the model's training. Thus, these approaches misalign with our primary objective of predicting whether Al models will make mistakes, as models can err on familiar data while behaving correctly on out-of-scope samples.
1. Introduction
Al models are prone to making errors in image recognition tasks, whether dealing with in-domain, out-of-domain
Subsequent research in out-of-domain detection has demonstrated that model's accuracy is often correlated with how far the data deviates from in-domain samples (Hendrycks & Dietterich, 2019; Shankar et al., 2021; Li et al., 2017). These methods typically rely on predefined metries, such as model parameter distances (Yu et al., 2022), model disagreements (Jiang et al.. 2022: Madani et al., 2004) and confidence scores (Guillory et al., 2021), which limits their ability to generalize predictions
'Deep NeuroCogaition Lab, College of Computing and Data Science, Nanyang Technological University, Singapore *Agency for Science, Technology and Research (A (A"AT))), Singapore. Correspondence to: Mengmi Zhang zhengmi.zhang edu.sg
Proceedings of the nnnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Ground truth label
Class label
Correctness
Ground truth
for mentee; dog
prediction
of mentee
label for mentor
In-domain images
Cut
Wrong
Mentee
Mentor
Out-of-domain
Dog
Correct
images
Wrong
Inference
Adversarial
Gibbon
Would the mentee's predictions be correct not?
images
Figure 1. Al models make mistakes and an "oracle" mentor model predicts when they will happen. A "mentee" neural network (black) was trained for multi-class image recognition, but it can still misclassify in-domain, out-of-domain, and adversarial images. For instance, it might mislahel an in-domain dog image a cat. The mentor model (blue), inputting the same images the mentee, predicts whether the mentee will make a mistake. For example, if the mentee incorrectly labels an adversarial dog image, the mentor's ground truth label is "wrong": conversely, if the mentee correctly labels out-of-domain dog image, the mentor's label is "correct". The mentee's s parameters are frozen ssnowflake), while the mentor's ' are trainable (fire). During inference (orange), the mentor predicts whether the mentee will make an error on test images that have never been seen by both the mentee and the mentor.
across various data types, including errors arising from in-domain data or adversarial attacks (Szegedy et al., 2014). In parallel, earlier efforts in trustworthiness prediction mostly depend on shallow neural networks (Corbière
on each of three distinct error types specified by the mentees Poattess-I--omeeee In-Dommin (ID) Errors, Out-of-Dommil (OOD) Errors, and Adversarial Attack (AA) Errors-affes the performance of error predictions over three increasingly complex image datasets CIFAR-I0 (Krizhevsky et al.. 2009), CIFAR- (Krizhevsky et al., 2009) and ImageNet-1K K (Deng et al., 2009). Our results reveal that training mentors with adversarial attack errors from the mentee has the most significant impact on improving the mentor's error prediction accuracy.
al., 2019; Qiu & Miikkulainen, 2022; Jiang et al., 2018) or cersinely-desiineed loss functions (Luo et al., 2021). However, they do not investigate how different error types influence trustworthiness prediction.
Another line of research improves the robustness of the AI models with adversarial training approaches (Ilyas et al., 2019; Gowal et al., 2020; Balunović & Vechev, 2020); however, these approaches primarily focus improving the model's overall performance rather than predicting when errors may occur in the models. Moreover, unlike selective prediction (Geifman & El- Yaniv, 2017), rejection learning (Cortes et al., 2016), and learning to defer (Madras et al., 2018), which jointly train the selection/rejeccionn function with the mentee, our method trains the mentor and mentee independently. This separation is especially beneficial when mentee training is time- and resource-intensive or when its training data is inaccessible.
2. We investigate how various mentor model architectures affect error prediction performance. Our experiments demonstrate that transformer- mentor models outperform other architectures in predicting errors.
3. We explore how varying levels of distortion in OOD and adversarial images affect the accuracy of error predictions. The findings indicate that training mentors on images with small perturbations can improve error prediction accuracy. In addition, we show that mentor trained to learn error patterns from one mentee can successfully generalize its error predictions to another mentee.
Different from all these works, we delve into the underlying principles of errors generated by Al models in the task of image classification with another AI model. Specifically, we designate the AI model that predicts errors as the mentor and the AI model being evaluated for performance the mentee. The mentor strives to predict whether the mentee makes a mistake for any given data. See Fig. 1 for the illustration of the problem setup. Training the mentor on the error patterns made by the mentee can potentially reveal the strengths and weaknesses of the mentee's 's learned representations across various visual contexts.
4. Based on our findings from points to 3, we present the SuperMentor model, which predicts errors across diverse mentee architectures and error types. Experimental results show that SuperMentor outperforms baseline mentors, demonstrating its superior error- preei-tredeetive capabilities.
2. Related work
Error monitoring systems for AI models. With the growing deployment of AI models across diverse fields, ensuring their reliability and understanding their limitations has become increasingly crucial. This has led to numerous research in safe AI such as uncertainty estimation (Nado al., 2021; Lakshminarayanan al.,
Our main contributions are highlighted below:
1. We conduct an in-depth analysis of how training mentors
2

---

Unveiling AI's Blind Spots: An Oracle for In- In-Domain, Out-of-Domain, and Adversarial Errors
Mentee
logits Zg
in error monitoring systems, though it differs in several key aspects. As highlighted by (Guérin et al., 2023), error prediction is distinct from OOD detection (Liu et al., 2020a; Sun et al., 2021; Lee et al., 2018; Sun et al., 2022) in their objectives. While OOD detection aims to detect whether the given data comes from the same domain as the training set, the aim of error prediction is to learn whether the mentee will make a mistake on the given data. In other words, out-of-domain data may not necessarily cause the model to err, and model errors can also occur on in-domain data.
Correct or
OR,
Neural Network
Predictions
Wrong
Ld
MLP
Backbone
logits 2g
Correct or Wrong
CR
MLP
Predictions
Mentor
Figure 2. Overview of mentor model. Given a fixed mentee model (snowflake) the mentor model takes an input image and uses a pre-trate tre-trained backbone on ImageNet-Ik IK (Deng al., 2009) to extract features. The feature maps are then processed in two streams via multi-layer perceptrons (MLP)s. The output logits zn from one stream are compared with the mentee's ' output logits ze using distillation loss La. The other stream performs a binary prediction of whether the mentee makes a mistake or not. The prediction supervised by a logistic regression loss L.. The parameters of MLPs in the two streams are not shared.
Recent studies (Hendrycks & Dietterich, 2019; Shankar et al., 2021; Li et al., 2017) have shown that a model's accuracy on a given dataset is often correlated with how far the data deviates from in-domain samples. However, these studies typically rely on pre-defined metries, such as model parameter distances (Yu et al., 2022), model disagreements (Jiang et al., 2022; Madani et al., 2004), confidence scores (Guillory et al., 2021), domain-invariant representations (Chuang et al., 2020), and domain augmentation (Deng et al., 202 limiting their ability to generalize error prediction beyond in-domain data. In contrast, our mentor is capable of predicting both OOD and in-domain errors for a mentee. Additionally, our mentor is an AI model trained end-to-end without relying on manually defined criteria.
2017), anomaly detection (Bogdoll et al., 2022), outlier detection (Boukerche et al., 2020) and out-of-domain detection (Yang et al., 2024). Unlike these areas, which mainly aim to predict whether the input data falls outside the training domain, our focus is on monitoring and predicting errors in AI models by determining whether the model's '' output is correct, inrespective of whether the data comes from the training domain. To detect whether the input data is out of scope, the prior approaches mainly rely on softmax outputs (Granese et al., 2021; Hendrycks & Gimpel, 2017; DANG et al.), activations from network layers (Wang et al., 2020; Cheng et al., 2019; Ferreira et al., 2023). shallow neural networks (Corbière et al., 2019; Qiu & Miikkulainen, 2022; Jiang et al., 2018), and carefully-designed loss functions (Luo et al., 2021) in applications such as object detection (Kang et al., 2018) and trajectory prediction (Shao et al., 2023; 2024). However, these methods often rely on manually defined metrics to estimate the likelihood of a mentee making mistakes, or they fail to examine how different error types affect error prediction. In contrast, our strategy employs a separate deep neural network to automatically learn and approximate the mentee's decision boundaries for specific error types, providing an end-to-end trainable framework for error prediction. Moreover, our research direction differs from selective prediction (Geifman & El-ivn 2017), rejection learning (Cortes al., 2016), and learning to defer (Madras et al., 2018) by avoiding joint training with the mentee and requiring no knowledge of the mentee's ' training data. Besides, although rejecting a mentee's 's unreliable predictions and predicting the correctness of a mentee's 's outputs are often positively correlated, they remain distinct tasks.
Adversarial attack and defense. In addition to OOD error, (Szegedy et al., 2014) discovered that deep neural networks can be fooled using input perturbations of extremely low magnitude. Building upon this finding, a substantial number of adversarial attacks have been proposed, including white-box attacks (Goodfellow et al., 2015; Mędry et al., 2017; Carlini & Wagner, 2017; Schwinn et al., 2023; Gao et al., 2020), black-box attacks (Uesato et al., 2018; Rahmati et al., 2020; Brendel et al., 2021; Chen et al., 2020), and backdoor attacks (Liu et al., 2020b; Xie et al., 2019; Kolouri et al., 2020). To defend against these adversarial attacks, various defence mechanisms (Qin et al., 2020; Deng et al., 2021b; Liu et al., 2019) have been developed to withstand detect adversarial inputs. Furthermore, although the primary objective of adversarial attacks is to deceive Al models, there are instances where adversarial perturbations are exploited to enhance the model performance - a technique known as adversarial training (Ilyas et al., 2019; Gowal et al., 2020; Balunović Vechev, 2020). Unlike adversarial training, which involves using adversarial samples train the mentee, our approach focuses teaching mentors to learn the mentee's error patterns revealed by these adversarial attack samples.
3. Experimental setups
Out-of-domain detection. Our research on predicting mentee errors is closely related to out-of-dommin detection
We denote the mentor and mentee networks as f-(() and fE(() respectively. We also define X' as the domain- domain-specific
3

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
containing all the test images for a mentee, and y as their ground-truth object class labels. Therefore, a mentor is expected to make perfect predictions about the correctness of the mentee's s responses (1 for "correct" and 0 for "wrong") given any image æ from X::
mentors and 3 x 10-3 for ViT mentors. All mentors load the weights of the feature extractor pre-trained on the ImageNet-II dataset for 1000-way image classification tasks (Deng et al., 2009) and further fine-tune on the eπror prediction task. During training, images are resized and center-cropped to 224 x 224 pixels. All mentors are trained for 30 epochs with a batch size of 32 on the CIFAR-10 and CIFAR-100 and 384 on the ImageNet- K. The value of q in the loss function is chosen for mentors to approach optimal performance. Specifically, q is set to 1.0 for the mentors on CIFAR-10, 2.0 for those on CIFAR-100 100, 3.0 for ResNer50 mentors on ImageNet-IK IK (Deng et al., 2009), and 0.1 for ViT mentors on ImageNet-1K.
{
1 if fz(x)=
=y, y.
VI E XG fR(x)
(1)
otherwise.
where E y/ is the ground object class label of the corresponding image x.
3.1. Mentors
Model architecture: We propose mentor models, as illustrated in Fig. 2. Given an input image, the backhone of
3.2. Mentees and their datasets
a mentor model extracts features from the input image. We adopt either of the two backbones for the feature extractors of mentors: 2D Convolutional Neural Network (2D-CNN) ResNet50 (He et al., 2016) and transformer-basee ViT (Dosovitskiy et al., 2020). The extracted feature maps are further processed in two streams implemented as multi-layer perceptrons (MLP)s. The parameters of the MLPs in the two streams are not shared.
We employ two architectures as the mentees' backbones: ResNet50 (He et al., 2016), which is a 2D-Convolutional neural network (2D-CNN), and ViT (Dosovitskly et al., 2020), which is transformer based on self-attention mechanisms.
To train and test our mentees, we include three image datasets of varying sizes and follow their standard data splits: CIFAR-10 (C10, (Krizhevsky et al., 2009)) with 10 object classes, CIFAR-100 with 100 object classes (C100, (Krizhevsky et al., 2009)) and ImageNet-1K with 1000 object classes (IN, (Deng et al., 2009)). The multi-class recognition accuracy on the standard test sets of C1O, C100 and IN datasets are 96.98%, 84.54%, 76.13% for the ResNet50 mentee and 97.45% 86.51% 81.07% for the ViT mentee respectively. The parameters of the mentees are frozen in all the experiments conducted on mentors.
The first stream generates logits zn by predicting the probability distribution of a mentee over all the object classes when the mentee classifies the given image. The mentee network is kept fixed while training the mentor. Let us define the mentee's s output logit as Zg. We introduce the distillation loss proposed by (Hinton et al., 2015): La= = T2. T2 K ( ()lor( ))) to align žR with Zg, where (T(--I-) represents Kullback-Leibler divergence. a(.) is the softmax function. T = 1.0 is the temperature, which controls the smoothness of the soft probability distribution.
3.3. Datasets for training and testing mentors
In the second stream, the mentor is prompted to predict whether the mentee will make a mistake on the given image or not. We denote the predicted binary label as CR. where indicates that the mentee does not make a mistake and vice versa for 0. This prediction is supervised by I, - Icg log(zz)) (1 - cg) log(1 =p)]. where cg is the ground truth correctness label of a mentee and Zp represents the mentor's 's predicted probability that the mentee's 's prediction is correct. The overall loss is L = aL, + (1 - L.d where a = (%) ) controls the dynamic weighting between La and L, over training epochs. Here, n denotes the current training epoch, N is the total number of training epochs, and q governs the rate at which a evolves throughout training.
The mentor's objective is to predict whether the mentee will misclassify a given image, regardless of its source. The mentor is trained on correctly and wrongly classified images by a mentee. Next, we introduce how these images are curated and collected. A mentee may encounter various types of errors when dealing with real-world data, To explore which error types most effectively reveal the mentee's 's learning patterns, we categorize errors into three types: (1) errors from in-domain images, (2) errors from out-of-domain images, and (3) errors from adversarial images generated by adversarial attacks. Next, we introduce these three error types in detail.
In- Ho-Domain (ID) Errors occur on data that come from the same domain as the mentee's 's training dataset. Specifically, errors on images from the standard validation set of IN or the test sets of C10 and C100 are considered ID errors. Along with the correctly classified images from these standard test sets, we create three datasets for mentor: IN-ID, C10-ID, and C100-ID, following the naming convention of
Training and implementation details: All mentors are trained on NVIDIA RTX A6000 GPUs, utilizing the AdamW optimizer (Loshchilov & Hutter, 2019) with a cosine annealing scheduler (Loshchilov & Hutter, 2022).
The initial learning rate is set to x 10-1 for ResNet50
4

---

Unveiling AI's Blind Spots: An Oracle for In- In-Domain, Out-of-Domain, and Adversarial Errors
80
80
000
000
OOD
75
75
73
70
70
70
Arcamnry
83
60
60
60
53
53
33
50
50
50
(a)
(b)
(c)
Figure 3. Mentors trained on adversarial images of a mentee outperform mentors trained on OOD and ID images of the same mentee. Average accuracy of mentor trained one type of error of mentee for (a) C10, (b) C100 and (c) IN datasets is presented. Three types of errors made by mentee are categorized based on in-domain (ID, blue), out-of-domai (OOD, orange), and images generated by adversarial attacks (AA, green). each subplot, the labels on the x-axis are interpreted as mmentee]-(mentor where "V" and "R" represent ViT and ResNet50 architectures for mentee or mentor respectively. Error bars indicate the standard deviation. The dotted black line indicates the chance level. See Sec. 3.3 and See. 3.4 for error types and the evaluation metric. The four sets of bars in each subfigure correspond to the confusion matrices shown in subfigures (a), (b), (c), and (d) of Appendix, Fig. S4- S6.
(Dataset]-[Eroor Type].
et al., 2020) crafts patch-wise noise instead of pixel-wise noise. We set e = 1.0 in the CW attack, and perturbation bound = 255 for other attacks by default. See their papers for these hyper-parameter definitions. Intuitively, the attacks are stronger with higher hyper-paramette values; hence, the mentees make more mistakes.
Out-of-domain (OOD) Errors refer errors that arise when the mentee encounters data outside the training domain. To obtain OOD samples of a dataset, adopt four types of image corruptions from (Hendrycks & Dietterich, 2019): speckle noise (SpN) (noise category), Gaussian blur (GaB) (blur category). spatter (Spat) (weather category), and saturate (Sat) (digital category). The noise levels can vary and we select level for image corruptions as specified in (Hendrycks & Dietterich, 2019) by default. As noise levels increase, distortions OOD images become more pronounced, causing the mentee to make more errors.
Note that adversarial attacks are not always successful, and mentees can still correctly classify some adversarial images. We collect both the correctly and incorrectly classified adversarial images by a mentee based on C10 images, curating four datasets for the mentor: C10-AA-PGD, CI0-AA-PIFGSM.
C10-AA-CW, C10-AA-Jitter, Similarly, we can also curate four datasets each for a mentor based on C100 and IN images of a mentee.
Following the naming conventions of [Dataset]-[Error Type]-[Error Source], we collect correctly and wrongly classified OOD samples based on C10 images of a mentee and curate four datasets for a mentor: C10-OOD-SpN, C10-OOD-GaB. C10-OOD-Spat and C10-OOD-Sat. Without the loss of generality, we can also curate four datasets each for a mentor based on C100 and IN images of a mentee.
Training and test splits for mentors. Given the three types of errors described above, we partition the data for mentors into training and testing sets using 70/30 split. For example, 70% of the original C10 evaluation images, along with their corresponding error- aroo--mddiied versions based on mentee performance, are used for training, while the remaining 30% are reserved for testing. This ensures that mentor models are trained and evaluated on samples derived from the same set of original images, without any overlap in the original image content, but with distinct domain shifts, The same strategy is consistently applied to the C100 and IN datasets. To mitigate the effects of long-tailed class distributions, each training batch includes an equal number of correctly and incorrectly classified samples.
Adversarial Attack (AA) Errors. Errors from adversarial images are specifically generated by adversarial attack methods to mislead or confuse the mentee. Given our assumption that the mentor has full access to the student model's parameters, we focus exclusively on white-box adversarial attacks as they typically produce more subtle yet effective perturbations compared to their black-box counterparts. To generate adversarial images, we employ four untargeted adversarial attack methods: PGD (Mądry et al., 2017) creates adversarial examples by repeatedly taking steps along the loss gradient; CW (Carlini & Wagner, 2017) attempts to minimize the L2 norm of the perturbation while ensuring misclassification. Jitter (Schwinn et al., 2023) adds Gaussian noise to the output logits to encourage a diverse set of target classes for the attack. PIFGSM (Gao
3.4. Baselines and evaluation metric
Baselines. We include seven baseline methods for error prediction of a mentee. 1) Self Error Rate (SER) predicts the correctness of the mentee's outputs by randomly assigning "correct" or "wrong" based on the mentee's in-domain accuracy; 2) Maximum Class
5

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Probability (MCP) (Hendrycks & Gimpel, 2017) classify the mentee's predictions as correct if its MCP exceeds predefined threshold. 3) Class Probability Entropy (CPE) evaluates mentee's prediction as correct if its entropy of the probability distribution over all classes is below a predefined threshold. 4) Distance To Centroid (DTC) considers a prediction correct if its feature embedding within a predefined distance from the class centroid. 5) ConfidNet (Corbière et al., 2019) predicts failure by learning True Class Probability (TCP) on the training samples. 6) TrustScore (Jiang et al., 2018) employs Trust Score to quantify the agreement between the classifier and a modified nearest-neighbor classifier.7) Steep Slope Loss (SSL) (Luo et al., 2021) develops a neural network for predicting trustworthiness by utilizing a steep slope loss function. See Appendix, Sec. A for more details.
deeper insights into the mentee's decision process compared to out-of-domain (OOD) and in-domain (ID) errors. To validate this, we first conducted an ANOVA test on the three groups of results (ID, OOD, and AA) for each of the three datasets (C10, C100, and IN). In all cases, the p-values are below 0.05, indicating a statistically significant difference in error prediction performance among mentors trained on different error types. Next, we performed two-tailed t-tests to compare AA-trained mentors against IID-trained and OOD-trained mentors across four [mentee][[mentor] configurations: R-R, V-R, R-V, and V-V. All pairwise p-values are below 0.05, confirming that AA-trained mentors significantly outperform those trained on ID or OOD error types. In addition, is worth noting that, in some cases, mentors trained on OOD errors slightly outperformed those trained on ID errors, although both still performed worse than those trained on AA errors. Interestingly, our experiments reveal that training mentors exclusively on AA errors performs marginally worse than joint training on all three error types (see Appendix, Sec. C). This result underscores a significant overlap of mentee error patterns among ID, OOD, and AA data.
Evaluation metric. To assess the performance of mentors, we report their error prediction accuracy on the test set corresponding to each specified error source. For instance, a mentor trained on the C10-ID training set is evaluated on the C10-OOD-SpN test set. The error prediction accuracy is calculated by averaging the mentor's accuracies on the samples that the mentee correctly classified and those that the mentee incorrectly classified. However, since mentee can make mistakes across various real-world scenarios, a mentor must accurately predict errors across all error types. Therefore, we compute the average accuracy, named as Accuracy (%) of a mentor across all test sets, including one ID error, four OOD errors, and four AA errors. See Appendix, Sec. B for an example calculation of the average accuracy of a mentor.
Loss landscape analysis. A loss landscape of mentee reflects how mentee's loss function behaves across different parameter configurations. Mentors' performance offers insights into the structure of a mentee's loss landscape. Consistent with (Ilyas et al., 2019), the high accuracy of mentors trained on AA errors suggests that adversarial images lie closer to the mentee's decision boundary, enabling more accurate prediction of the mentee's mistakes and a deeper understanding of the loss landscape. Similarly, OOD data aids mentors in learning boundaries by shifting ID samples closer to the boundary. However, it does not explore the boundary as thoroughly as adversarial images. ID data, with fewer samples near the boundary, provides more limited exploration compared to adversarial images. To support our claims, we include quantitative loss landscape analyses in Appendix, Sec. D. The results show that mentors trained on AA errors exhibit a wider loss landscape compared to those trained on ID or OOD errors.
4. Results
4.1. Training on specific errors of mentees impacts the performance of mentors
A mentee's mistakes can reveal their learning biases, behaviors, or traits. Here, we investigate which types of errors offer the most insight into understanding a mentee's decision boundaries during image recognition tasks. We train mentors with identical architectures datasets containing specific error types made by the mentee across C10 (Fig. 3(a)), C100 (Fig. 3(b)), and IN (Fig. 3(c)). For instance, if a mentor trained on C10-0OD achieves higher accuracy error prediction compared to one trained on C10-ID, this suggests that in-domain errors provide less diagnostic information about the mentee's 's decision-making process than out-of-domain errors. Both mentors and mentees may have the same or different backbones, such as ResNet50 (R) or ViT (V).
4.2. Mentor architectures matter in error predictions
To computationally model the decision boundary of mentee using mentor, the mentor requires more complex architectures with a larger number of parameters than the mentee. Indeed, from Fig. 3, over all the datasets, we observed that utilizing ViT (V) as the mentor backbone consistently achieves higher accuracy across all error types of ViT-based and ResNet- mentees compared to the mentor based on ResNet50 (R). One example of this performance disparity is observed in the context of the adversarial attack error type on C10. The mentors
As shown in Fig. 3. over C10. C100, and IN images, the high average accuracies for mentors trained on adversarial attack (AA) errors indicates that these AA errors offer
leveraging a ViT backbone achieves 73.1 % accuracy in
6

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
CIFAR-10
CIFAR-100
ImageNet-1K
CIFAR-10
CIFAR-100
ImageNel-IK
80
PIFGSM
80
SpN
80
OOD
AA
Accuracy on Mentee (%)
75
75
75
Accuracy (%)
70
70
70
65
65
65
60
60
60
55
55
50
50
55
45
L1
L3 14 45
12
L3
L4
50
55
60
65
70 75
80
Image Distortion Levels
Accuracy on Mentee A (%)
Figure mentor's accuracy is heavily influenced by the levels of image distortions introduced by out-of-domain perturbations and adversarial attacks. ViT mentor's 's accuracy is function of varying image distortion levels from PIFGSM (Gao al., 2020) and Speckle Noise (SpN) (Hendrycks Dietterich, 2019) to the C1O, C100 and IN images of ResNet50-baset mentee. The black dashed line indicates the chance level.
Figure Mentors can generalize their error predictions across different mentee architectures. Mentors trained on mentee A's predictions (x-axis) are evaluated against the predictions from mentee B (y-axis). Each marker is a generalization experiment a mentor trained on different error types (marker shapes) different image datasets (colours) of mentec. The black dash line indicates the diagonal.
predicting the correctness of ResNet50 mentee's 's outputs, which is significantly higher than the 61.9% accuracy of the ResNet-based mentor (compare R-R versus R-V).
smaller perturbations yield more benefits for mentor error prediction. This phenomenon can be attributed to the fact that adversarial images with minimal perturbations maintain closer proximity to the decision boundary of a mentee.
Loss landscape analysis. The performance difference between mentors' architectures is due to VIT's superior ability to identify features from error patterns. Its self-attention mechanism captures complex relationships among data samples, providing deeper understanding of the mentee's loss landscape, particularly in modelling irregular, rugged landscapes with sharp peaks and valleys. To validate this point, we include loss landscape analyses in Appendix, Sec. D. Results show that ViT mentors possess a notably broader loss landscape than ResNet50 mentors.
Building on the findings above, we investigate whether the mentor's performance is influenced by how far OOD images are from the ID data. Specifically, whether the degree of deviation from the training domain impacts the mentor in a similar way to our observations on adversarial images. To explore this, we analyze images corrupted with Speckle Noise (SpN) and adjust the standard deviation a of SpN to 0.01, 0.06, 0.15, and 0.6, representing four distinct levels of distortion. The outcomes are depicted in Fig. We observe that the mentor's accuracy improves as the distortion introduced by SpN decreases. For example, the mentor achieves accuracy of 66.2% on level of C C10-OOD-SpN. while the accuracy drops significantly to 47.9% on level 4 of C100-OOD-SpN, This suggests that OOD error types with smaller perturbations enhance the mentor's performance. However, unlike adversarial attacks, caution is necessary because the mentor's accuracy can plateau with extremely small distortion levels, as shown by the minimal difference in accuracy between levels and 2 of SpN in Fig. 4.
aim to determine
4.3. Training on images with smaller perturbations helps error predictions
Although adversarial images have been demonstrated to aid in error prediction (Sec. 4.1), remains unclear whether adversarial images with varying degrees of image distortion exhibit the same effect. A straightforward method to regulate the level of image distortion caused by adversarial attacks is to set the perturbation bound e. We employ
four corruption levels by setting = 255 255' 255 and 255 We use the adversarial attack PIFGSM as an example since the error patterns from PIFGSM are most effective for the mentor's prediction (see Appendix, Fig. S4- S6). As shown in Fig. 4, the mentor's 's accuracy significantly decreases as the distortion level increases. In particular, for the C10-AA-PIFGSM, the accuracy at level 1 is 74.3%, which is notably higher than 49.7% at level Our findings suggest that adversarial attacks employing
4.4. Mentors generalize across mentees
In Sec. 4.1, mentors have demonstrated their ability to learn the emorr patterns of mentees. This observation raises an important question: can the error patterns learned from one mentee (mentee A) be generalized to another mentee (mentee B) when the two mentees employ different model

---

Unveiling AI's Blind Spots: An Oracle for In- In-Domain, Out-of-Domain, and Adversarial Errors
Accuracy
Lg
L
Accuracy
brambling
brambling brambiing
brambling snowhied
baseball baseball
basebll
basoball
1.0
SER
50.110.0
X
X
59.3= 0.7
brambblig
1D
Spat
PIPGSM
ID
baseball
Spat
sombrere
PIPGSM
MCP (7 = 0.7)
65.1=0.0
C10
X
64.3= 0.3
Meator Montee Input
0.8
CPE (a: 0.1)
63.040.0
ours
64.9= 1.1
X
56.1# 0.6
0.6
DTC (d= = 10)
51.8=1000
C100
X
62.8+ 0.1
ConfidNet (Corbière ct al., 2019)
59.3
63.62 0.7
0.4
TrustScore (Jiang et al., 2018)
61.6:10.0
X
61.8# 0.4
SSL (Luo et al., 2021)
66.710..3
IN
X
62.52 0.1
0.2
SuperMentor (ours)
70.4:10.1
ours
63.1# 0.2
0.0
(b)
(c)
Figure 6. Mentor analysis comprising: (a) performance comparison with baselines, (b) ablation of loss components, and (c) CigenCAM (Gildenblat& & contributors, 2021; Muhammad & Yeasin, 2020) visualizations. (a) compares the average accuracy of haselines and our SuperMentor various error sources and severity levels from the IN dataset. The ResNet50 is used as the mentee, with mentors predicting its correctness. Results are reported as average accuracy with variance over 3 runs; best results are in bold. Full baseline results across hyperparameter settings are Appendix, Tab. S1. Comparison with VIT mentee is provided in Appendix, Tab. S2. (b) presents an ablation study where La is the distillation loss (see Sec. 3.1) and La is the alignment loss between mentor and mentee predictions. The ResNet50-based mentors are evaluated on VIT mentees, with each cell showing mean accuracy and standard deviation over 3 runs. Grey cells indicate our mentors' performance. (c) shows EigenCAM visualizations of the SuperMentor and VIT mentee on sample images from brambling and baseball classes across IN-ID, IN-OOD-Spal and IN-AA-PIFGSS datasets. For each sample, rows display the input image (Input), mentee's 's attention map (Mentee), and mentor's attention map (Mentor), using the first LayerNorm the final transformer block as the visualization layer. Column headers indicate Ground Truth Label and Mentee's s. Prediction.
architectures? To explore this, we evaluate all 324 mentors on the alternate mentee. Specifically, mentors trained on the errors of the ResNe150 mentee are tested on the predictions of the ViT mentee, and vice versa. The outcomes of these evaluations are illustrated Fig. 5. Surprisingly, most points lie near the dashed diagonal line, implying that the mentors' performance does not significantly deteriorate when evaluated on the predictions of different mentee architectures. This finding indicates that ResNet50 and ViT mentees tend to produce similar error patterns when trained on the same dataset.
comparing it with the baselines introduced in Sec. 3.4. To conduct comprehensive evaluations, we extend image perturbations from different error sources to multiple severity levels. Specifically, we use corruption levels 1, 3, and 5 for SpN, GaB, Spat, and Sat error sources as specified in (Hendrycks & Dietterich, 2019); set c to 255 255 and 255 for PIFGSM, Jitter, and PGD error sources; and vary the learning rate as 0.01, 0.1, and 1.0 for the CW error source.
As shown in Fig. 6(a), our SuperMentor can achieve higher average accuracies across all severity levels and error types than baselines. Notably, as indicated in Appendix, Tab. S1, in the AA scenarios, SuperMentor has significantly higher accuracy than the baselines. For example, SuperMentor achieves an error prediction accuracy of 73.1% % on IN-AA-PGD of the ResNet50 mentee, whereas the best baseline only reaches 64.7%. Baseline methods like SER, MCP, CPE, and DTC rely on fixed thresholds or predetermined values of manually defined criteria such as confidence or entropy, making them less adaptable to AA scenarios. In contrast, SuperMentor leverages deep neural networks capture the complexity of error prediction. Additionally, methods like ConfidNet. TrustScore, and SSL are trained on ID error types. As shown in Fig. 6(a), these baselines are inferior to SuperMentor, which is trained exclusively on AA data. This suggests that the source of error data used to train the mentor plays a more critical role than the choice of loss functions or the sophisticated training strategies in these baselines. Moreover, we provide the visualization of SuperMentor's embeddings based the correctness of a mentee's prediction across three error types (Appendix, Sec. G). Two distinct clusters are observed between samples wrongly or correctly classified by the
In addition, we evaluate the mentors' generalization ability on natural adversarial samples introduced in (Hendrycks al., 2021b), where each image is misclassified by multiple mentees simultancously. See details in Appendix, See. F. Results show that, beyond natural adversarial examples, the mentors also exhibit robust generalization on non-natural adversarial examples.
4.5. Analysis on our Super-Mentor reveals key insights
By drawing insights from observations in the subsections above, we propose an "oracle" mentor model, dubbed SuperMentor. We introduce the technical novelties of our SuperMentor below. First, as demonstrated in Sec. 4.1 and Sec. 4.3, mentors trained on adversarial images with small perturbations of mentee outperform those trained on OOD and ID images; thus, our SuperMentor adopts the training data from the PIFGSM error source of mentees with 255 Second, since ViT has been proven to be a more effective architecture for mentors than ResNet50 (Sec. 4.2), SuperMentor adopts ViT as the backbone architecture.
We demonstrate the effectiveness of SuperMentor by
8

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
mentee. This suggests that our SuperMentor is capable of accurately predicting the errors of mentee regardless of the error sources.
mentee's predictions across three distinct error types: in-domain errors, out-of-domain errors, and adversarial attack errors. Our results show that the mentor excels at learning from a mentee's ' enrors on adversarial images with minimal perturbations and, surprisingly, generalizes well to both in-domain and out-of-domain predictions of the same mentee. Additionally, we highlight the effectiveness of transformer-basee mentor architectures compared to 2D-CNN-based ones, demonstrating their superior generalization capabilities across mentees with diverse backbones. Lastly, we introduce the SuperMentor, which outperforms all existing mentor baselines.
Next, we examine the effect of the distillation loss Ld (Fig. 2) on the mentors' performance. The results are presented in Fig. 6(b). It is clear that excluding La results in a decrease in mentors' accuracy across all datasets. For example, in the C10 dataset, the average accuracy of mentors decreases from 64.9% to 59.3%. This suggests that La encourages mentors to learn the fine-grained decision boundaries among different object classes of a mentee. Alternatively, instead of utilizing the mentee's 's logits, mentors can incorporate a cross- eross-entropy loss to align the mentor's ' predicted object class labels with those of the mentee, denoted La From Fig. 6(b), we observe that replacing La with La leads to a drop in accuracy. This is due to the fact that the mentee's logits contain more information than its class labels.
Despite the promising results, our framework may raise concerns regarding the vulnerabilities of AI mentors. Indeed, vulnerability is a well-known challenge for all deep neural networks (DNNs), yet this has not hindered their transformative applications in the real world. For example, DNNs are used to detect diabetic retinopathy (Jumper et al., 2021; Alyoubi et al., 2020), outperforming traditional diagnostic methods. Similarly, our work uses one Al to diagnose the other AI, which is a valid and impactful use case. To demonstrate its real-world utility, we extended our experiments a medical image classification task for colorectal cancer diagnosis (Appendix, Sec. H). Our mentor achieves high error prediction accuracy compared to all competitive baselines, showcasing its practical uses in high-stake applications. More importantly, beyond the OOD domains evaluated in Sec. 4.5, results in Appendix, See. I show that our SuperMentor remains robust to mentee error patterns across an extra set of five unseen OOD domains. Moreover, the vulnerabilities of the mentor AI do not overlap with those of the mentees since mentors operate independently of the mentee's 's architecture or training details, reducing the risk of shared weaknesses (see Sec.4.5).
Finally, we provide EigenCAM (Gildenblat & contributors, 2021; Muhammad & Yeasin, 2020) visualization in Fig, 6(c) to illustrate the following two points: 1) The vulnerabilities of the mentee do not overlap with those of the mentors. As illustrated in Fig. 6(c), the mentor does not merely replicate the mentee's ss learning patterns, as their activation maps for identical input images can differ significantly. For example, in the brambling images shown in Fig. 6(C), the activation maps of the mentor and mentee diverge greatly when the input image is ID or subjected to AA using PIFGSM method. In the IN-AA-PIFGSM image. the mentee demonstrates vulnerability under adversarial attacks, whereas our SuperMentor does not exhibit such vulnerabilities on this AA image. consistently focusing on the brambling object. This indicates the vulnerabilities of mentors and mentees are distinct due to their non-overlapping objectives. 2) Our Al mentor can serve as a valuable diagnostic tool for Al mentees. As supported by the baseball images in Fig. 6(c), although the mentee correctly classifies the ID image, its activation map does not concentrate on the baseball but instead highlights background cues (spurious features). When the input image is subjected to IN-AA-PIFGSM attacks, the mentee fails to make accurate predictions. In contrast, our SuperMentor effectively focuses on the stitches of the baseball regardless of whether the image is corrupted or adversarially attacked. This suggests that the mentor can identify vulnerabilities in the mentee, even though they both share the VIT architecture.
Our work paves the way for several promising research directions in safe and trustworthy AL. First, while our current research focuses on image classification, there is potential to extend this approach to other vision and language tasks, such as object detection and machine translation. Second, future research could explore mutual learning between mentors and mentees, where mentors not only learn from the mentee's 's error patterns but also provide valuable feedback to help refine the mentee. Third, we can establish more rigorous evaluation criteria for mentors, broadening their predictive capabilities. For example, beyond predicting whether mentee is likely to make errors, mentors could also forecast the specific types of errors a mentee may encounter. Fourth, drawing parallels with AI mentors for Al mentees, we can explore the possibility of using AI mentors to investigate recognition errors in humans and primates. Such studies could provide insights into error pattern alignment between biological and artificial intelligent systems.
5. Discussion and Conclusion
In our work, we tackle the challenge of predicting errors of AI models through extensive empirical evaluations using an end-to-end trainable "mentor" model. This mentor is designed to assess the correctness of
9

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Acknowledgements
Cheng, C.-H., Nührenberg, G., and Yasuoka, H. Runtime monitoring neuron activation patterns. In 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 300-303. IEEE, 2019.
This research is supported by the National Research Foundation, Singapore under its NRFF award NRF-NREFI5-2023 and Mengmi Zhang's Startup Grant from Nanyang Technological University.
Chuang, C.-Y., Torralba, A., and Jegelka, S. Estimating generalization under distribution shifts via domain-invarian representations. In Proceedings of the 37th International Conference on Machine Learning, pp. 1984-1994, 2020.
Impact Statement
As Al integrates into our daily lives and critical applications like healthcare, finance, and autonomous driving, ensuring its safety and reliability is imperative. Our research highlights the error patterns and characteristics of Al models for object recognition, paving the way for building trustworthy AI. By enabling one Al model to anticipate and correct another's behavior, our framework helps ensure Al systems operate in a more predictable and reliable manner. Overall, our work lays the foundation for developing systems capable of anticipating the errors of others, offering practical value in high-stakes real-world applications.
Corbière, C., Thome, N., Bar-Hen, A., Cord, M., and Pérez, P. Addressing failure prediction by learning model confidence. Advances in Neural Information Processing Systems, 32, 2019.
Cortes, C., DeSalvo, G., and Mohri, M. Learning with rejection. In Algorithmic Learning Theory: 27th International Conference, ALT 2016, Bari, Italy, October 19-21, 2016, Proceedings 27, pp. 67-82. Springer, 2016.
DANG, K. T., Delmas, K., Guiochet, J., and Guerin, J. Can we defend against the unknown? an empirical study about threshold selection for neural network monitoring. The 40th Conference on Uncertainty in Artificial Intelligence.
References
Alyoubi, W. L., Shalash, W. M., and Abulkhair, M. F. Diabetic retinopathy detection through deep learning techniques: A review. Informatics in Medicine Unlocked, 20:100377, 2020.
Deng. J., Dong, W., Socher, R., Li, L.-J,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Ieee, 2009.
Balunovié, M. and Vechev, M. Adversarial training and provable defenses: Bridging the gap. In 8th International Conference Learning Representations (ICLR 2020Xvir International Conference on Learning Representations, 2020.
Deng, W., Gould, S., and Zheng, L. What does rotation prediction tell us about classifier accuracy under varying testing environments? In International Conference on Machine Learning, pp. 2579 2589. PMLR, 2021a.
Bogdoll, D., Nitsche, M., and Zöllner, J. M. Anomaly detection in autonomous driving: A survey. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4488-4499, 2022.
Deng, Z., Yang, X., Xu, S., Su, H., and Zhu, J. Libre: practical bayesian approach adversarial detection. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 972 982, 2021b.
Boukerche, A., Zheng, L., and Alfandi, O. Outlier detection: Methods, models, and classification. ACM Computing Surveys (CSUR), 53(3):1-37, 2020.
Dosovitskiy, A., Beyer, L.., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M.. Heigold, G., Gelly, S., et al. An image is worth 16x 16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
Brendel, w., Rauber, J., Bethge, M., and Adversarial, D.-B. basessoobbbudddddveerraaatttte Rellableatacksgginsthhkkk
machine learning models. Advances in Reliably Evaluating and Improving Adversarial Robustness, pp. 77, 2021.
Ferreira, R. S., Guerin, J.. Guiochet, J., and Waeselynek, H. Sena: Similarity- based error-checking of neural activations. In ECAI 2023, pp. 724-731. IOS Press, 2023.
Carlini, N. and Wagner, D. Towards evaluating the robustness neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. Icce, 2017.
Gao, L., Zhang, Q., Song, J., Liu, X., and Shen, H. T. Patch-wise attack for fooling deep neural network. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part XXVIII 16, pp. 307-322, Springer, 2020.
Chen, W. Zhang, Z., Hu, X., and Wu, B. Boosting decision-based black-box adversarial attacks with random sign flip. In European Conference on Computer Vision pp. 276-293. Springer, 2020.
10

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Geifman, Y. and El-Yaniv, R. Selective classification for deep neural networks. Advances in neural information processing systems, 30, 2017.
al. The many faces of robustness: critical analysis of out-of-distributinn generalization. In Proceedings the IEEE/CVF international conference computer vision, pp. 8340-84-00 8349, 202
Gildenblat, and contributors. Pytorch library for cam methods. https: pectps:/ggtthtn /githab.. com/ hannjaccbgi// pytorch-grade grad--radd--a cam, 2021.
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 15262-15277) 2021b.
Goodfellow, J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Hinton, G.. Vinyals, O., and Dean, J. Distilling the knowledge a neural network. arXiv preprint 000300231,2222 2015.
Huang, Y., Du, J., Yang. Z., Zhou, Z., Zhang, L., and methods for autonomous driving. IEEE Transactions Intelligent Vehicles, 7(3):652-6744 2022.
Gowal, S., Qin, C., Huang, P.-S., Cemgil, T., Dvijotham, K., Mann, T., and Kohli, P. Achieving robustness in the wild via adversarial mixing with disentangled representations. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1211-1220 2020.
Chen, H. A survey on
Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., and Madry, A. Adversarial examples are not bugs, they are features. Advances neural information processing systems, 32, 2019.
Granese, F., Romanelli, M., Gorla, D., Palamidessi, C., and Piantanida, P. Doctor: A simple method for detecting misclassification errors. Advances in Neural Information Processing Systems, 34:5669-5681, 2021.
Jiang, H., Kim, B., Guan, M., and Gupta, M. To trust or not to trust a classifier. Advances in neural information processing systems, 31, 2018.
Guérin, J. Delmas, K., Ferreira, R., and Guiochet, J. Out-of-distribution detection is not all you need. In Proceedings of the AAAI conference artificial intelligence, volume 37, 14829-14837, 2023.
Jiang, Y., Nagarajan, V., Back, C., and Kolter, Z. Assessing generalization of SGD via disagreement. In The Tenth International Conference on Learning Representations, ICLR 2022, Viriual Event, April 25-29, 2022. OpenReview.net, 2022. URL https: / //openreview http://ffruummiiwvoccceee..
Guillory, D., Shankar, V., Ebrahimi, S., Darrell, T., and Schmidt, L. Predicting with confidence unseen distributions. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 1134-1144, 2021.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Zidek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. nature, 596(7773):583-589 2021.
Habehh, H. and Gohel, S. Machine learning in healthcare. Current genomics, 22(4):291, 2021.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Kang, D., Raghavan, D., Bailis, P., and Zaharia, M. Model assertions for debugging machine learning. In NeurIPS MLSys Workshop, volume 2018.
Kather, J. N., Krisam, J., Charoentong, P., Luedde, T. Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al. Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS medicine, 16(1):
Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common coruptiinns and perturbations. Proceedings of the International Conference Learning Representations, 2019.
Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and networks. In International Conference on Learning Representations, 2017.
e e1002730, 2019.
examples in neural
Kolouri, S., Saha, A., Pirsiavash, H., and Hoffmann, H. Universal litmus patterns: Revealing backdoor attacks in cnns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 301-310, 2020.
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M.,

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Madras, D., Pitassi, T., and Zemel, R. Predict responsibly: improving faimess and accuracy by learning to defer. Advances in neural information processing systems, 31, 2018.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Madry, A., Makelov, A., Schmidt, I.., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. stat, 1050(9), 2017.
Lee, K., Lee, K., Lee. H., and Shin, J. A simple unified framework for detecting ouiit--diitiiuttinn samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Mashrur, A., Luo, W., Zaidi, N. A., and Robles-Kelly, A.
Machine learning for financial risk management: a survey. leee Access, 8:203203-203223, 2020.
Li, D., Yang, Y., Song, Y.-Z., and Hospedales, T. M. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pp. 5542-5550, 2017.
Muhammad, M. B. and Yeasin, M. Eigen-cam: Class activation map using principal components. In 2020 international joint conference on neural networks (IJCNN), pp. 1-7, IEEE, 2020.
Li, H., Xu, Z., Taylor, G., Studer, C., and Goldstein, T. Visualizing the loss landscape of neural nets. Advances in neural information processing systems, 31, 2018.
Nado, Z.. Band, N., Collier, M., Djolonga, J., Dusenberry, M. W., Farquhar, S., Feng, Q., Filos, A., Havasi, M., Jenatton, R., et al. Uncertainty baselines: Benchmarks for uncertainty & robustness in deep learning. 2021.
Liu, J., Zhang, W., Zhang, Y., Hou, D.. Liu, Y., Zha, H., and Yu, N. Detection based defense against adversarial examples from the steganalysis point of view. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4825-4834, 2019.
Qin, Y., Frosst, N., Sabour, S., Raffel, C., Cottrell, G. w. and Hinton, G. E. Detecting and diagnosing adversarial images with class-conditional capsule reconstructions. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https:/ wwtp///pppnpeeeeeeeeepeeeeoeeeep net nwt//orunt id aad-Sky44m 44Kvr.
Liu, W., Wang, X., Owens, J., and Li, Y. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464-21145, 2020a.
Liu, Y., Ma, X., Bailey, J., and Lu, F. Reflection backdoor: A natural backdoor attack on deep neural networks. In Computer Vision-ECCC 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pp. 182-199. Springer, 2020b.
Qiu, X. and Mikkkulanenn, R. Detecting misclassification errors in neural networks with gaussian process model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8017-8027, 2022.
Loshehilov, I. and Hutter, F. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL pettps://openreeeee wtwpiiewweeeeeeeeeeeeeeeeeeeeeee forum? id=Bkg6Ricqy7.
Rahmati, A., Moosavi-Dezfooll S.-M., Frossard, P., and Dai, H. Geoda: a geometric framework for black-bon adversarial attacks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8446-8455, 2020.
Schwinn, L., Raab, R., Nguyen, A., Zanca, D., and Eskofier, B. Exploring misclassifications of robust neural networks to enhance adversarial attacks. Applied Intelligence, 53 (17):19843- 2023.
Loshchilov, I. and Hutter, F. Sgdr: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2022.
Luo, Y., Wong, Y., Kankanhalli, M. S., and Zhao, Q. Learning to predict trustworthiness with steep slope loss. Advances in Neural Information Processing Systems, 34: 21533-21544 2021.
Shankar, V., Dave, A.,, Roclofs, R., Ramanan, D., Recht, B., and Schmidt, L. Do image classifiers generalize across time? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9661-9669, 2021.
Madani, O., Pennock, D., and Flake, G. Co-validation: Using model disagreement on unlabeled data to validate classification algorithms. Advances in neural information processing systems, 17, 2004.
Shao, W., Li, J., and Wang, H. Self-aware trajectory prediction for safe autonomous driving. In 2023 JEEE Intelligent Vehicles Symposium (IV), pp. 1-8. IEFE, 2023.
12

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Shao, W., Li, B., Yu, W., Xu, J., and Wang, H. When is it likely fail? performance monitor for black-box trajectory prediction model. IEEE Transactions Automation Science and Engineering, 2024.
Sun, Y., Guo, C., and Li, React: Out-of-distribution detection with rectified activations. Advances in Neural Information Processing Systems, 34:144-157, 2021.
Sun, Y., Ming. Y., Zhu, X., and Li, Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning, pp. 20827-20840. PMLR. 2022.
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. In 2nd International Conference on Learning Representationss ICLR 2014, 2014.
Uesato, J., O' O'donoghue, B., Kohli, P., and Oord, A. Adversarial risk and the dangers of evaluating against weak attacks. In International conference on machine learning, pp. 5025-5034. PMLR, 2018.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008
Wang. H., Ge, S., Lipton, Z., and Xing, E. P. Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems, 32, 2019.
Wang, H., Xu, J., Xu, C., Ma, X., and Lu, J. Dissector: Input validation for deep learning applications by crossing-layer dissection. In Proceedings of the ACM/EEE 42nd International Conference on Software Engineering, pp. 727-738, 2020.
Xiao, K. Y., Engstrom, L., Ilyas, A., and Madry, A. Noise or signal: The role image backgrounds in object recognition. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https:// www.:/teeee/tttttte wwt.:ooommmmmmdd----odd -1
Xie, C., Huang, K., Chen, P.Y., and Li, B. Dba: Distributed backdoor attacks against federated learning. In International conference learning representations, 2019.
Yang. J., Zhou, K., Li,
Li, Y., and Liu, Z. Generalized nut-of-distribution detection: A survey. International Journal of Computer Vision, pp. 1-28, 2024.
Yu, Y., Yang, Z., Wei, A., Ma, Y., and Steinhardt, Predicting out-of-distribution error with the projection norm. International Conference on Machine Learning, pp. 25721-25746. PMLR, 2022.
13

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
A. Details of baselines
For performance comparison, we adopt seven baselines, as introduced in See. 3.4. Their details are outlined below.
Self Error Rate (SER): We predict the correctness of the mentee's outputs by referencing its accuracy on in-domain samples. For instance, if the mentee's 's in-domain accuracy is 70%, we use random binary generator that produces 1 ("correct") with 70% probability and 0 ("wrong") with 30% probability, applying this to all testing scenarios.
Maximum Class Probability (MCP) (Hendrycks & Gimpel, 2017): As mentioned in Sec. 3.1, the mentee's output logit is denoted as 2g. The maximum class probability of the mentee's ' output is defined as MCP()E) = max (==)) where (-) is the softmax function. The MCP indicates how confident the mentee is in its most probable prediction. We set a threshold y to distinguish between the mentee's confident and unconfident predictions. Specifically, if MCP()E)) > 7, the mentee's s prediction will be considered a correct prediction due to its high confidence; otherwise, it will be regarded as an incorrect prediction. We employ three predefined thresholds for yy: 0.5, 0.7, and 0.9.
Class Probability Entropy (CPE): The class probability entropy of the mentee's output is defined as CPE(zE) = II (=E)), where (-) denotes the softmax function and (-) represents the entropy measure quantifying the uncertainty in the probability distribution. A high entropy value signifies a high level of uncertainty in the mentee's predictions. The entropy reaches its maximum value, Max PEEz)) when the mentee's ' class probabilities in (EE)) are equal. We define the uncertainty threshold as a (azzC where a E [0, 1]. If CPE(2e) < a MarCCEEEE)) the mentee's s prediction is considered correct, indicating sufficient certainty in its prediction. Otherwise, the prediction is regarded as incorrect. We set three predefined values for 0.01, 0.1 and 0.3.
Distance to centroid (DTC): The embedding generated by the mentee before the final binary classification layer represents the mentee's feature interpretation of each sample. To determine the feature centroid for each class, we first average the features of all testing samples within that class based on the mentee's predictions. Next, we calculate the 1.2 distance between each sample's s feature and its corresponding class centroid, denoted as dg. We establish a distance threshold d; if dg < d, the mentee's 's prediction is considered correct since the sample is close the class centroid. Otherwise, the prediction is deemed incorrect. We have set three predefined values for d: 10, 20, and 30.
NonfidNet (Corbière et al., 2019): A shallow failure prediction neural network that learns True Class Probability (TCP) from the training set instead of relying on Maximum Class Probability (MCP).
TrustScore (Jiang et al., 2018): It introduces a new score called trust score by measuring the agreement between the classifier and a modified nearest-neighbor classifier on the testing samples for error prediction.
Steep Slope Loss (SSL) (Luo et al., 2021): Training an Al model for trustworthiness prediction by leveraging a darefful steep slope loss function.
B. Example of calculating a mentor's average accuracy using the proposed evaluation metric
As an example of computing the accuracy of a mentor evaluated using the metric presented in Sec. 3.4, we consider a mentor trained on CIO-AA-PIFGSM and evaluate it on all the following datasets with their accuracies of 10% on C10-ID, 20% on C10-OOD-SpN, 30% on C10-00D-Gal GaB, 40% on CI0-OOD-Spaa 50% on C10-OOD-Sat 60% on C10-AA-Jitter, 70% on CIO-AA-PGD, 80% on CI0-AA-CW, and 90% on CIO-AA-PIFGSS testing samples, the average accuracy of this mentor is calculated as (10%+ 20%+ + 30% 40% 50% + 60% + 70% 80% + 90%) / = 50%. For simplicity, we refer to this average accuracy across all nine error sources as Accuracy. A mentor randomly guessing whether mentee's 's image classification is correct or incorrect for a given image would achieve an accuracy of 50%.
C. Joint training
As mentioned in Sec. 4.1, we add an experiment by training a Vision Transformer (ViT)-based mentor model on the correctness of a mixture of ID, OOD, and AA data for the ResNet-based mentee model from the C10 dataset. The OOD data was corrupted with speckle noise (SpN), and the AA data was generated using PIFGSM. Compared to training the mentor solely on PIFGSM data which achieved an average accuracy of 74.3%, including ID and SpN data improved the average accuracy slightly to 77.5%. However, this joint training setup involves 21,000 samples-tiin three times more than the 7,000 used for PIFGSM-only training. The marginal improvement observed in joint training may be attributed to the larger training sample size. This suggests that the quantity of samples is not the key factor in enhancing a mentor's error prediction
14

---

Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
ID
OOD
AA
V(ID)
V(GaB)
V(PGD)
70
R(ID)
R(GaB)
R(GGD)
70.0
68
67.5
Accuracy (%)
66
Accuracy (%)
62.5
64
60.0
57.5
62
55.0
60
52.5
58
50.0
-0.4
-0.2
0.0 0.2
0.4
-0.4
-0.2
0.0 0.2
0.4
Weight Perlurbalion
Weight Perturbation
Figure $1. Loss landscape analysis of mentors trained three error types: in-domain (ID, blue), out-of-domain (OOD, orange), and adversarial attack images (AA, green) from the C100 dataset. The average accuracies of mentors evaluated across various error sources under different weight perturbations are reported. The ResNet50 model serves the mentee, and the ViT model adopted as the mentor architecture.
Figure S2. Loss landscape analysis of mentors with different backbones. Mentors can he built with two different backbones: ViT (V, solid line) and ResNet50 (R, dotted line). We evaluate mentors trained on three error sources: C100-ID (blue), C100-00D ALd-ODD-aa GaB (green), and C100-AA-POO (red). The legend follows the format [mentor backbone] ([training error source]). The average accuracies of mentors evaluated across various error sources under different weight perturbations are reported. The ResNet50 model serves as the mentee.
performance. Training mentors on samples that accurately capture the error patterns of mentees is crucial.
In addition, introduce another mentor that is jointly trained on all error sources (ID, OOD, and AA, a total of error sources) from the C10 dataset, achieving average evaluation accuracy of 78.4%. In this scenario, all testing samples for the mentor are in distribution because the mentor has seen samples from every error source during training. Despite this, the improvement compared to a mentor trained solely on the PIFGSM error source (78.4% vs. 74.3%) is marginal. Consequently, this finding further indicates: (1) Training mentors exclusively on the AA error source offers a more data-efficient strategy for real-world applications. (2) Remarkably, a mentor trained solely on AA generalizes well to unseen error sources, achieving performance comparable the upper bound obtained by jointly training on all error sources.
D. Loss landscape analysis
We conducted two quantitative loss landscape analyses using the method from (Li et al., 2018) to support the claims in Sec. 4.1 and Sec. 4.2. In these analyses, we apply small perturbations to the mentor's weights and observe the resulting changes in the mentors' average accuracy.
As shown Appendix, Fig. S1, we examine mentors trained on three different error types: ID, OOD, and AA from the C100 dataset. The results indicate that mentors trained on AA errors exhibit wider loss landscape than those trained on ID OOD errors. This suggests that mentors trained with adversarial images capture the generic features for predicting the mentee's decision-making This finding strongly supports our loss landscape analysis discussed in Sec. 4.1.
The second loss landscape analysis in Appendix, Fig. S2 examines the mentor's s eгтоr prediction performance using two different backbones: ResNet50 and VIT. The figure shows that mentors with ViT architectures have a considerably wider loss landscape compared to those with ResNet50. This further reinforces our claims in Sec. 4.2 that transformer based mentor models outperform their ResNet-based counterparts in error prediction.
15

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
E. More comparisons between SuperMentor and baselines
As noted in the caption Fig. 6(a), baseline performances across multiple hyperparameter configurations are indicated in Appendix, Tab. S1. In addition, when ViT serves the mentee, the SuperMentor's performance is shown in Appendix, Tab. S2. It can be seen that our SuperMentor can still achieve the best performance when the mentee has backbone of ViT. This emphasizes the importance of the error source used during training.
Table SI. Performance comparison of baselines (multiple hyperparameter configurations) and our Super:Mentor on various error sources from the IN dataset. The ResNet50 model serves the mentee, with its correctness of predictions evaluated by mentors. Best results are in bold.
ID
SpN
GaB
Spat
Sat
PGD
CW
Jitter
PIFGSM
Average
SER
50.3
49.9
50.1
50.2
50.2
50.1
50.1
50.1
50.3
50.1
MCP (7 = 0.5)
69.4
73.0
71.7
71.2
72.6
51.7
67.5
52.5
53.6
64.4
MCP (7 = 0.7)
77.5
71.6
70.7
73.9
76.5
50.3
70.0
50.4
53.6
65.1
MCP (7 0.9)
78.7
66.2
65.4
70.4
73.7
45.7
67.1
47.8
51.9
61.7
CPE (a 0.01)
69.3
57.0
56.9
60.3
62.7
43.4
58.5
47.0
48.9
54.9
CPE (a = 0.1)
78.1
67.9
66.9
72.4
75.2
46.9
68.6
49.8
51.7
63.0
CPE (a = 0.3)
63.8
71.9
69.6
69.2
68.9
51.1
65.1
52.4
52.3
62.6
DTC (d = 10)
52.6
52.4
51.9
52.6
52.4
52.3
51.1
50.5
51.3
51.8
DTC (d = 20)
51.6
50.2
50.1
50.7
50.9
51.0
50.0
50.1
52.8
50.8
DTC (d 30)
50.0
50.0
50.0
50.0
50.0
50.0
50.0
50.0
50.0
50.0
ConfidNet (Corbière al., 2019)
67.8
59.0
61.1
61.8
64.1
51.8
61.9
57.5
54.3
59.3
TrustScore (Jiang al., 2018)
71.2
58.6
60.1
63.4
65.7
58.2
63.3
59.4
60.8
61.6
SSL (Luo al., 2021)
68.6
66.2
69.2
69.6
68.4
64.7
63.4
63.2
67.9
66.7
SuperMentor (ours)
73.3
69.2
65.6
71.1
69.9
73.1
67.4
70.3
75.4
70.4
Table S2. Performance comparison baselines (multiple hyperparameter configurations) and our Super:Mentor on various error sources from the IN dataset. The VIT model serves as the mentee, with its correctness of predictions evaluated hy mentors. Best results are bold.
SpN
GaB
Spat
Sat
PGD
CW
Jitter
PIFGSM
Average
SER
49.8
50.2
49.9
50.1
50.0
49.7
50.1
50.1
49.9
50.0
MCP (7 0.5)
70.0
75.0
73.1
73.5
72.2
51.6
58.9
45.2
53.5
63.2
MCP (7 0.7)
78.3
75.5
74.4
75.8
77.7
49.3
57.7
38.8
50.6
63.1
MCP (? = 0.9)
58.3
52.3
56.1
54.2
56.6
46.2
50.8
38.7
46.6
50.5
CPE (ar = 0.01)
50.0
50.0
50.0
50.0
50.0
49.9
50.0
49.7
50.0
50.0
CPE (a = 0.1)
51.4
50.4
51.1
51.0
51.4
47.2
49.9
43.7
47.7
49.1
CPE (a = 0.3)
72.0
72.6
72.9
72.1
73.5
47.4
56.7
36.2
47.6
60.4
DTC (d 10)
68.8
66.2
63.9
64.2
67.5
56.8
55.2
39.4
58.3
59.3
DTC (d 20)
50.1
50.0
50.0
50.1
50.1
50.1
50.0
50.0
50.1
50.1
DTC (d 30)
50.0
50.0
50.0
50.0
50.0
50.0
50.0
50.0
50.0
50.0
ConfidNet (Corbière al., 2019)
73.4
69.0
67.0
68.6
71.1
58.8
59.8
46.8
60.7
63.2
TrustScore (Jiang et al., 2018)
74.4
67.5
64.8
67.6
70.8
57.9
59.7
48.1
60.5
62.6
SSL (Luo et al., 2021)
66.7
67.2
68.0
66.1
66.9
58.1
56.7
43.4
59,2
60.9
SuperMentor (ours)
70.9
65.5
61.7
67.5
68.8
81.6
64.1
68.5
81.9
70.0
F. Relationship between mentor generalization and natural adversarial samples
As diseussed in Sec. 4.4, aim to examine the relationship between the generalization ability of mentors and natural adversarial samples. Following the definition in (Hendrycks et al., 2021b), natural adversarial samples are those that consistently lead to incorrect predictions across a wide range of Al models. In line with this definition, for each error source, we define N, as the set of samples misclassified by both ResNet50 and ViT mentees, serving as an analogue to the natural adversarial samples (Hendrycks et al., 2021b). The samples misclassified only by the ResNet50 mentee (the ViT mentee can classify them correctly) are labeled set N₂, and those misclassified only by the VIT mentee (the ResNet50 mentee can
16

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Table S3. Role natural adversarial samples in SuperMentor's generalization across mentee architectures in the dataset. Each cell is formatted as: [number of samples correctly predicted by the mentor] [total number of samples in the set] ([mentor prediction accuracy for this set]..
Error Source
N
N2
N3
ID
1750/2225 (78.7%)
1055/1352 (78.0%)
429/561 (76.5%)
SpN
2568/2973 (86.4%)
2310/3075 (75.1%)
462/579 (79.8%)
GaB
2710/3078 (88.0%)
1368/1657 (82.6%)
642/757 (84.8%)
Spat
2131/2587 (82.4%)
1300/1627 (79.9%)
464/584 (79.5%)
Sat
2433/2880 (84.5%)
1585/2012 (78.8%)
502/618 (81.2%)
PGD
3785/4689 (80.7%)
377/709 (53.2%)
2360/3286 (71.8%)
CW
2528/3281 (77.0%)
977/1952 (61.4%)
1325/1886 (70.3%)
Jitter
2527/3501 (72.2%)
701/1032 (67.9%
1714/2685 (63.8%)
classify them correctly) are labeled set N3. The SuperMentor is trained on performance from the ResNet50 mentee and then generalizes its error prediction ability to the VIT mentee.
The result Appendix, Tab. S3 indicates that the natural adversarial samples in N1 play key role in explaining SuperMentor's strong generalization across different architectures. However, SuperMentor can also perform well on the non-natural adversarial samples N2 and N3 For example, it achieves 76.5% accuracy the set N3 for ID error types. This demonstrates that SuperMentor's robust generalization ability is not limited solely to natural adversarial samples.
G. Visualization
To gain an intuitive understanding SuperMentor's binary classification performance error prediction, as mentioned in Sec. 4.5, we present the visualization of SuperMentor's embeddings on three types of error sources of mentee in Appendix, Fig. S3. It is evident that SuperMentor can effectively segregate samples correctly classified by the mentee from those that misclassified, forming two distinct clusters.
(a)
(b)
(c)
Figure S3. 3D visualization of the embeddings extracted from our SuperMentor Model for the classification of: a) C10-ID samples, b) C10-OOD-GaB samples and e) C10-AA-Jitter samples. use t-SNE (Van der Maaten Hinton, 2008) perform clusterings on the representations of our SuperMentor model for classifications of different error sources on the C-10 datasct. Red points indicate samples that mentee fails to classify correctly, whereas blue points represent samples points and 50 blue points are randomly selected from the test sets and presented here. The visualized features are the embeddings in the second stream of the SuperMentor. Specifically, they are extracted before the final binary classification layer on whether the mentec makes mistake.
mentee successfully classifies.
H. Super.Mentor in real-world practice
To reflect the real-world contribution of our work, as mentioned in Sec. 5, we expanded our experiments to the NCTCRCHE100K (Kather et al., 2019) dataset which is used for medical image classification on colorectal cancer. This
17

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
dataset comprises around 100K images across tissue classes. Predicting Al models' errors in medical image classification has significant real-world implications, such reducing misdiagnoses and increasing the reliability of Al-assisted medical tools. Appendix, Tab. S4 demonstrates that our SuperMentor accurately predicts the correctness of mentee outputs with an average accuracy of 81.8%, significantly outperforming other baseline methods. This highlights that our proposed framework can offer greater value and reliability for Al error prediction compared to existing approaches in medical image classification.
Given the increasing integration of Al models into our daily lives, ensuring their accuracy is paramount, especially in high-stakes fields such medicine and finance. This experiment with medical image datasets underscores the critical role that SuperMentor plays in enhancing reliability and trustworthiness in these vital areas, showcasing its potential to make meaningful impact where precision and dependability are essential.
Table S4, Performance comparison of baselines (multiple hyperparameter configurations) and our Super on various error sources from the NCTCRCHE100K (Kather al., 2019) dataset. The ResNet50 model serves as the mentec, with its correctness of predictions evaluated by mentors. Best results are in bold.
ID
SpN
GaB
Spat
Sat
PGD
CW
Jitter
PIFGSM
Average
SER
50.6
50.1
50.1
51.1
50.4
50.0
51.4
50.2
49.8
50.4
MCP (7 = 0.5)
53.0
59.1
53.0
53.2
50.6
53.8
53.3
53.2
54.8
53.8
MCP (7 0.7)
62.8
70.5
64.6
64.7
50.2
65.2
64.0
63.2
65.2
63.4
MCP (7 0.9)
72.5
78.5
72.1
75.0
48.8
77.0
73.8
76.0
75.2
72.1
CPE (a = 0.01)
76.2
75.2
80.5
77.7
44.7
76.0
75.1
76.6
75.1
73.0
CPE (a = 0.1)
75.5
79.5
76.4
76.2
48.2
77.8
76.2
77.6
76.9
73.8
CPE (a = 0.3)
64.9
74.7
66.9
66.4
51.6
69.2
65.8
68.4
67.0
66.1
DTC (d = 10)
78.9
66.2
77.8
80.8
67.1
79.2
78.0
78.3
79.1
76.1
DTC (d= 20)
70.6
47.8
69.9
62.9
52.6
60.4
65.1
63.7
60.0
61.4
DTC (d 30)
51.8
49.5
52.8
50.3
50.0
50.8
51.6
51.8
50.8
51.0
ConfidNet (Corbière al., 2019)
85.1
61.3
86.2
74.0
58.4
79.3
82.9
81.8
79.4
76.5
TrustScore (Jiang et al., 2018)
73.3
60.0
73.4
62.2
62.4
71.0
72.8
71.4
71.5
68.6
SSL (Luo et al., 2021)
85.3
58.2
87.8
75.5
55.4
77.6
82.8
79.7
78.0
75.6
SuperMentor (ours)
88.5
64.1
89.2
84.1
56.9
88.2
88.7
88.3
88.3
81.8
I. SupeerMneto performance on additional OOD domains on ImageNet dataset
As mentioned in Sec. 5, we included experiments to evaluate the SuperMentor more diversified OOD domains on ImageNet datasets, including MaaeeeNNee--MMXE--RRAR al., 2021), ImageNed-MIXED-NEET (IN9-MN) (Xiao et al., 2021), ImageNet-R (IN-R) (Hendrycks al., 2021a) and ImageNet-Sketch (IN-S) (Wang et al., 2019). Specifically, the MIXED-RAND, MIXED-SAME, and MIXED-NEXT datasets are derived from 9 classes in ImageNet and contain varying amounts of background and foreground signals. These datasets aim to demonstrate that models often classify objects based on background cues (often spurious features), rather than the objects themselves. Specifically, MIXED-SAME, MIXED-RAND, and MIXED-NEXT represent images with random backgrounds from the same class, random backgrounds from a random class, and random backgrounds from the next class, respectively. The ImageNet-R dataset comprises images featuring artistic representations of objects, such as cartoons, coammunity-generatee art, and graffiti renditions. The ImageNet- dataset consists of sketch- like images that match the ImageNet validation set in both categories and scale.
(IN9-MR) (Xiao et al., 2021), 1), MaageNer--MIXEDD-SS
(IN9-MS) (Xiao
To achieve more thorough evaluation of our SuperMentor (see Sec. 4.5) in OOD domains, we assess its performance on these additional OOD datasets. It is worth noting that the SuperMentor learns from the mentee's 's errors on adversarial ImageNet images generated by the PIFGSM attack only. In other words, the SuperMentor was NOT trained on any of these additional ImageNet-relatee OOD datasets. Yet, the SuperMentor achieves average accuracies of 70.9%, 71.0%. 69.6%, 62.2%, and 62.6% on the IN9-MR, IN9-MS, IN9-MN, IN-R, and IN-S datasets respectively, surpassing the 50% chance level. Despite that our Supermentor model is simple ViT, it still quite remarkable achieve above-chance error prediction performance. We hope work inspires researchers explore more sophisticated AI mentors, incorporating advanced architectures or loss functions, to further enhance the ability predict the correctness of another AI model's outputs.
18

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
J. Detailed performance of mentors across various error sources
As mentioned in the caption Fig. 3, the detailed results of mentors across various error sources for the C1O, C100, IN datasets are shown Appendix, Fig. S4, Fig. S5 and Fig. S6 respectively.
39.2
51.0
59.0
57.2
58.9
56.8
57.1
56.2
56.1
57-2
54.8
19.9
54.9
53.8
58.7
54.1
53.2
53.5
53.9
54.1
+0.5
+3.2
21.1
20.4
21.1
20.6
21.2
=0.3
#1.4
#0.3
20.3
20.9
$1.0
21.0
20.4
20.9
20.3
11.0
21.1
+0.3
5P
57.7
71.5
58.8
57.4
62.8
58.5
57.1
59.1
59.2
60.2
SpN
57.3
63.4
57.1
54.4
60.5
56.3
55.3
55.8
57.5
57.6
#2.3
20.7
21.6
20.7
21.8
21.6
20.7
20.8
21.5
#1.0
20.3
114
20.6
20.4
24.0
11.0
20.8
21.1
$1.9
$1.1
Gal
58.3
58.3
53.5
56.3
Gaf
22.3
$1.0
Spat
38.9
58.5
60.9
57.8
39.2
39.3
59.4
Spat
56.1
54.4
55.7
$7.9
59.5
57.0
56.9
56.8
57.7
56.9
11.8
24.3
+1.4
+1.0
+0.5
20.8
21.5
20.5
#1.8
+1.0
#1.44
#4.9
=1.1
+1.8
#1.1
#2.3
=1.2
#2.1
=2.8
#1.8
Sat
59.5
60.0
59.1
56.8
63.2
58.6
57.1
59.3
59.2
59.2
60.2
56.3
21.8
#2.7
21.8
#2.3
20.6
#2.4
22.3
#2.0
#1.1
#1.5
PGD
60.3
59.6
60.3
61.2
PGD
67.2
63.3
66.9
64.6
66.6
65.6
64.8
63.2
66.8
63.7
23.1
20.9
#2..5
20.6
#2.1
21.6
#1.0
20.2
20.5
20.3
20.3
#9.6
85.0
39.1
60.0
61.0
58.9
59.0
63.00
62.3
64.3
62.3
64.22
61.7
121.4
21.8
20.3
20.5
20.7
20.6
20.0
20.7
122.0
20.4
11.
$1.7
20.4
20.8
10.8
1144
=0.2
$0.8
Jitor
56.4
55.3
56.5
59.3
58.3
59.7
59.6
60.8
58.8
58.3
Itttor
12.7
14.2
22.0
12.1
14.0
21.0
20.7
10.3
13.5
11.8
#1.1
20.8
75.0
PIFGSM
66.7
PIFGSM
64.7
64.0
64.3
62.3
66.7
65.6
64.6
64.0
67.0
64.9
#1.6
#2.3
#1.6 #1.3
#1.9 #0.3
#1.0 =0.4
=0.8
#1.1
Accuracy (%)
1
Sph
GaB
PGD
Iitter
PIFGSM
Sph
Gall
Spat
Sat
PGD
CW
litter
PIFGSM
65.0
10
67.5
63.0
67.6
67.9
69.4
65.3
61.0
63.4
62.1
65.7
62.1
62.9
61.4
64.0
67.1
60.9
59.1
60.3
61.2
62.1
=1.6
11.2
22.4
20.9
1.2
11.1
20.8
+1.1
+3.3
22.6
22.2
11.2
11.
#1.5
#2.1
SpN
SpN
70.2
64.5
214
21.1
20.5
20.7
-5..0
Can
66.3
61.4
65.5
65.4
66.1
62.5
62.8
62.4
60.1
63.6
Gaf
63.3
63.6
62.9
62.9
64.9
59.3
58.0
50.4
59.6
61.5
#2.9
#3.1
#2.8
#2.9
#2.0
#2.4
21.4
#2.4
#2.0
#2.3
#3.9
23.33
#2.9
#4.4
#4.2
23.6
#3.6
24.0
=3.7
#3.8
Spat
Spat
66.6
12.7
SAt
70.1
63.0
69.7
69.8
73.4
66.7
63.9
67.4
63.4
67.7
71.9
73.1
70.8
71.6
76.6
67.0
64.8
67.7
67.8
70.1
22.22
21.6
21.88
#1.1
20.3
20.7
20.8
21.1
20.5
20.4
200.3
#1.6
20.2
20.9
20.3
#1.1
20.4
20.6
20.9
20.3
PGD
27.6
71.0
77.2
76.1
76.1
75.7
70.4
75.5
70.3
74.4
PGD
81.3
77.7
80.8
77.8
82.1
77.1
74.1
77.2
77.3
78.4
20066
=0.2
+1.4
20.9
+1.1
20.9
20.9
+1.5
+0.9
+0.8
:22.3
=1.0
#2.3
+1.0
20.6
+1.5
11.2
+1.0
+1.1
CW
Jltor
73.0
69.0
73.7
74.
72.9
72.8
60.0
72.2
68.4
71.7
litter
80.00
77.7
79.6
79.3
79.7
75.0
74.4
76.9
76.1
77.7
#2.2
20.3
1.88
116
+1.8
#1.3
#1.4
+1.7
#0.9
+1.3
=0.7
1.2
112
20.9
20.8
20.5
#1.1
20.6
20.6
#0.3
PIFGSM
PIFGSM
77.5
75.5
76.6
76.2
74.3
79.0
78.1
75.3
78.6
20.7
20.7
10
SpN
GaB
Sal
PGD
Jntter
PIFGSM
SpN
Spal
Sal
PGD
CW
Jiter
PIrGBM
Figure S4. Confusion matrices showing the average performance of mentor models across various error sources for the C10 datasct, presented in the format prentee]-[mentort
ResNet50- Nestee5550 b) ViT-ResNet50, e) ResNet50-ViT, and d) VIT-VIT. The confusion matrices' row labels indicate the training error source for the mentor, while the column labels denote the testing error sources for the mentor. Results cach cell denote the average accuracy with the standard deviation runs. The pink-highlighted column displays the row-wise mean and standard deviation over 3 runs.
19

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
10
62.1
63.2
62.1
61.9
60.7
60.8
59.8
61.7
59.8
61.3
60.7
59.0
60.1
61.0
59.4
58.4
57.4
58.2
58.9
59.2
#1.0
+0.5
+1.0
+1.3
+0.5
+1.1
+0.9
#0.9
=0.8
=0.7
+1.2
+0.7
#1.1
+0.9
+0.4
+0.3
+0.9
+0.3
=0.1
=0.6
SpN
Spi
+1.0
+0.2
+1.3
+0.4
+1.1
+0.8
+0.9
+0.4
=0.6
+0.4 +0.4
+1.1
+0.8
+0.2
#1.2
+0.9
+0.6
GaB
62.1
63.5
62.6
62.5
60.4
61.5
59.6
61.8
60.1
61.6
Gan
58.9
59.5
59.4
58.8
58.1
57.5
56.3
57.0
57.3
58.1
#0.4
#0.5
#0.5
20.9
11.0
#0.4
#0.5
#1.4 +0.7
=0.2
#0.8
#0.4
#0.3
#1.1
+0.4
20.5
20.8
20.8
=0.8
#0.5
Spat
Spat
#2.1
+1.9
#0.3
#1.8
#1.5
#0.4
#1.6
#0.9
#0.7
#0.7
#1.0
62-8
60.9
62.6
62.0
67.3
60.4
58.9
61.1
59.5
61.7
61.4
56.4
60.8
61.3
65.4
59.5
58.2
59.5
60.1
60.3
20.2
1.9
20.3
20.0
20.5
20.5
#1.1
20.8
=0.3
20.6
.20.9
11.
#1.0
#1.3
20.4
20.2
20.4
#1.0
20.0
20.6
PGD
PGD
20.7
20.4
85.0
60.7
61.9
60.7
60.8
37.7
61.1
61.3
61.5
61.6
60-8
CW
59.9
58.5
60.4
59.0
58.6
60.3
59.7
59.5
60.2
59.6
20.6
+1.0
20.5
20.2
21.0
00.0
20.5
20.8
20.1
20.3
.2..
11.1
20.4
22.4
11.9
20.7
20.4
20.0
20.6
20.5
Jittor
61.4
64.5
62.4
litter
61.8
61.6
62.1
61.4
75.0
PIRGSM
PIFGSM
64.5
63.9
63.7
63.8
62.7
65.2
63.9
65.4
66.2
64.4
63.5
63.8
64.3
63.0
62.6
63.9
62.7
63.1
65.4
63.6
-0.9
=0.3
#0.4
+1.
=1.
=0.2
=0.8
20.2
=0.5
=0.2
+0.5
#1.
=0.7
=2.2
#1.0
=0.3
=0.8
=0.9
=0.6
=0.7
Accuracy (%)
10
SpN
GaB
Spar
Sat
PGD
CW
litter
PIFGSM
10
SpN
GaB
Spat
Sat
PGD
CW
Iitter
PIFGSM
65.0
10
67.9
65.6
67.3
65.7
64.6
65.5
63.7
66.4
63.6
65.6
10
66.4
64.6
66.2
63.2
64.4
62.9
62.2
62.0
62.9
64.1
+1.0
20.2
20.8
20.3
20.7
20.9
20.9
20.8
20.5
20.6
20.5
11.
20.6
20.4
11.5
20.6
20.1
20.3
20.7
20.4
SpN
Spi
-55.0
Gaf
68.3
65.3
68.5
66.
65.0
65.7
63.7
66655
63.3
65.8
GaB
67.33
65.6
67.1
66.
65.0
63.4
63.1
62.4
63.8
64.9
+.8
1.2
+1.5
1.3
22.5
0.99
11.3
1.0
1.6
=1.1
20.3
20.5
20.2
20.3
20.7
20.3
20.1
20.2
20.7
20.3
Spat
68.6
67.0
Spat
67.1
67.8
20.44
00.7
45.0
Sat.
68.9
67.4
68.3
66.9
71.0
67.99
64.2
67.8
65.5
67.5
648.0
67.1
67.5
67.2
71.3
64.9
63.0
64.5
64.7
66.5
2066
20.7
20.5
20.5
=0.6
20.1
=0.0
20.1
20.5
=0.2
20.1
1.6
=0.3
20.1
20.7
20.3
=0.1
20.7
+0.2
=0.3
PGD
71.1
68.4
PGD
71.3
68.3
CW
69.5
66.9
69.1
67.1
65.8
68.6
67.2
69.4
67.1
67.9
69.6
67.8
69
67.0
67.5
66.5
66.5
67.7
67.9
20.6
=0.6
20.5
20.1
=2.0
20.8
#0.7
20.8
20.6
=0.7
2:0.8
20.7
20.3
=1.0
=0.2
=0.6
=0.9
#0.5
#1.0
=0.6
Hitor
70.9
67.6
70.5
68.3
67.1
69.8
66.6
70.4
67.6
68.8
litter
69.3
68.5
68.8
69.2
66.8 67.1
65.6
66.8
67.1
67.7
PIFGSM
PIFGSM
69.2
65.6
69.0
67.2
65.3
68.9
66.5
69.1
68.9
67.5
71.1
69.6
70.7 70.1
68.8 68.8
67.6
68.5
69.7
69.4
#0.4
#0.8
#0.4
#0.2
#0.4
#0.6
#0.3
#0.5
#0.3
#0.3
#1.1
+0.0
#1.0
#0.9
#0.6
#0.9
#0.7
#0.8
#0.1
#0.5
1D
SpN
Gab
Spat
Sat
PGD
CW
lttter
PIFOSM
1D
SpN
GaB
Spat
Sat
PGD
CW
litter
PIFGSM
(d)
Figure Confusion matrices showing the average performance mentor models across various error sources for the C100 dataset, presented in the format [mentee]][mentonn
ResNet50-RRss Nes5e b) VeT-Res e) ResNet50-ViT, and d) ViT-VIT. The confusion matrices' row labels indicate the training error source for the mentor, while the column labels denote the testing emoo sources for the mentor. Results in each cell denote the average accuracy with the standard deviation over runs. The pink highhiigh column displays the row-wise mean and standard deviation over 3 runs.
20

---

Unveiling AT's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
10
65.5
61.9
64.0
61.9
62.6
64.0
62.8
64.1
63.9
63.7
10
62.7
58.8
60.8
61.4
59.8
60.7
60.7
60.6
61.8
60.8
20.3
20.4
20.2
20.2
20.2
20.2
20.2
20.2
20.4
20.2
20.3
20.2
20.3
20.1
20.2
20.3
20.3
20.5
=0.4
20.2
SpN
SpN
58.7
=0.7
Gad
61.8
52.1
67.0
61.0
59.5
59.1
58.3
39.6
59.4
59.7
GaB
139.6
52.4
64.2
58.7
58.2
57.4
57.2
56.8
58.2
58.1
=0.1
+1.0
=0.0
=0.1
=0.6
20.1
20.3
20.6
=0.3
=0.5
=0.1
=0.8
=0.2
=0.4
=0.5
=0.2
#0.3
20.0
=0.0
=0.2
Spat
66.0
63.9
66.9
63.9
64.1
62.9
64.4 64.2
64.6
Spat
63.6
60.6
63.3
61.4
62.4
61.7
61.7
63.7
62.4
Saf
63.0
58.6
60.1
62.3
66.9
60.9
60.2
61.5
60.6
61.6
Sat
61.6
56.3
60.2
60.6
63.1
61.6
60.2
60.1
62.7
60.7
#0.4
#0.4
#0.3
#0.3
#0.3
#0.5
#0.4
#0.4
#0.3
#0.3
=0.00
#0.2
#0.4
#0.1
#0.4
=0.2
#0.3
#0.2
=0.2
#0.2
PGD
65.9
62.6
65.1
63.3
66.0
61.3
65.3
66.3
61.8
PCD
62.7
57.3
61.3
62.2
60.8
67.7
63.9
63.5
68.5
63.1
#0.2
#0.3
85.0
65.4
61.8
62.0
61.7
62.0
64.6
64.4
61.6
65.2
63.8
63.6
57.8
61.1
62.6
60.9
66.1
64.3
63.6
67.6
63.1
#0.1
#0.9
#0.2
20.4
20.1
20.2
20.1
20.1
#0.2
#0.2
#0.4
#1.0
#0.9
#0.5
20.5
20.0
20.2
#0.3
=0.3
#0.4
Jtter
65.9
61.9
63.3
61.9
62.8
64.9
64.1
65.2
65.0
61.2
litter
61.7
58.8
61.7
63.3
61.7
66.4
63.9
61.3
67.6
63.6
20.2
20.3
20.3
20.1
20.2
20.4
20.1
20.2
+0.1
20.3
=0.5
20.3
20.1
20.2
20.3
=0.1
20.1
75.0
PIFGSM
PIRGSM
62.7
63.0
65.6
67.5
64.9
64.3
63.8
68.9
63.1
20.3
20.5
20.3
20.2
20.5
20.3
20.1
20.2 20.3 20.2
#0.1
#0.6
#0.3
20.3
=0.3
20.3
20.2
20.1
=0.2
20.2
Accuracy (%)
10
SpN
Gall
Spatt
Sal
PGD
CW
litter
PIFGSM
1D
SpN
Gall
Spat
Sat
PGD
CW
Jitter
PIFGSM
a
1D
73.0
71.7
72.4
73.2
71.7
71.7
69.3
71.0
71.6
71,7
1D
71.8
71.5
72.6
71.8
71.0
69.0
67.7
64.5
70.3
70.0
20.6
20.5
20.2
20.4
20.3
20.7
20.6
20.3
20.7
20.5
20.2
20.3
20.2
20.3
20.3
20.5
20.3
20.4
20.4
20.2
SpN
71.9
73.5
72.2
72.8
71.8
71.0
68.5
70.2
70.8
71.4
SpN
71.2
72.7
72.9
72.0
71.1
69.2
67.4
64.2
70.3
70.1
20.0
10.4
20.4
10.2
20.1
20.2
10.1
20.5
20.2
20.2
20.4
-55.0
Galt
72.6
72.5
73.5
73.4
72.0
72.4
69.5
71.0
72.6
72.2
Gal
71.8
71.7
72.5
71.6
69.0
67.9
64.2
70.5
70.4
10.4
20.2
20.4
20.3
20.2
20.3
20.3
20.4
20.1
20.3
20.3
20.3
20.3
20.3
20.3
20.1
20.2
20.4
00.2
20.2
Spat
72.5
72.0
72.9
73..4
72.3
71.7
68.9
70.7
71.5
71.8
Spat
71.3
71.5
72.7
72.3
70.7
68.6
67.2
63.9
69.66
69.7
20.5
20.1
20.3
20.1
20.7
20.6
20.5
20.8
20.1
20.8 20.4
20.7
116
1.3
11.
20.8
Sat.
72.2
71.7
71.9
73.2
71.5
69.0
70.4
71.5
Sat
72.4
69.5
67.7
64.4
70.6
70.1
=0.2
=0.1
=0.1
20.3
=0.2
-0.5
20.3
20.3
20.7
=0.2
=0.2
=0.3
20.0
20.2
=0.1
=0.2
20.1
20.0
=0.2
=0.1
PGD
73.2
72.4
72.2
73.8
71.9
73.2
70.4
71.8
73.8
72.5
PGD
71.0 70.5
70.2
71.0
69.0
75.7
71.2
69.7
77.0
71.7
:20.2
20.1
20.6
20.1
20.0
=0.2
20.3
20.3
20.1
20.1
=0.11
=0.1
=0.3
20.1
20.2
20.1
20.0
20.2
=0.1
=0.0
70.0
72.1
70.6
73.1
71.0
Cw
73.0
67.9
74.7
:20.6
=0.7
=1.0
=0.7
20.6
=0.3
=0.4
=0.4
20.1
=0.5
:0066
20.6 20.5
20.6
=1.1
=0.9
+1.2
20.6
00.7
20.4
Iltter
733.5
71.9
71.8
73.4
72.0
73.3
70.9
72.4
73.8
72.6
litter
71.2 70.4
69.6
71.0
69.1
73.9
70.7
69.5
75.5
71.2
+00.2
#0.1
+0.3
#0.3
+0.1
=0.3
+0.2
=0.2
+0.5
+0.1
+0.33 #0.4 #0.3 +0.3 =0.6
=0.4
+0.3
+0.5
=0.3
#0.3
PIEGSM
PIRGSM
70.9
69.6
78.6
#0.1
=0.2
#0.4 #0.2 #0.4 =0.2
#0.2
20.2
+0.2
#0.1
#0.7
#0.2 #0.7 #0.7 #0.3 =0.3
#0.3
#0.3
00.5
#0.3
1D
SpM
GaB
Spat
Sat
PGD
CW
litter
PIFOSM
1D
5pN
GaB
Spat
Sat
PCD
CW
litter
PIFGSM
(
(d)
Figure S6. Confusion matrices showing the average performance mentor models across various error sources for the ImageNet-IK -K dataset, presented in the format [mentre))-meento):: and d) VIT-VIT. The confusion matrices' row labels indicate the training error source for the mentor, while the column labels denote the testing
ResNet55-ResNetfe b) VeT-Res,, ResNet50- VIT,
for the Results in each cell denote the average accuracy with the standard deviation over runs. The pink hokhhhiglighhed column displays the row-wise mean and standard deviation over 3 runs.
21


Reviews:
[{"review_id": "Rjt7cXDwr1", "reviewer": "Reviewer_LEYc", "summary": "The authors propose training a \"mentor\" model to learn to predict the errors of a \"mentee\" model. The authors evaluate multiple choices for training the mentor model, such as training with in-distribution, out-of-distribution, and adversarial examples to predict errors. The authors combine these results to propose a \"SuperMentor\" which can outperform the baseline mentors in predicting errors.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[LEYc.1-Claims And Evidence]** We appreciate the reviewer’s thoughtful question and would like to clarify the following two points:\n\nFirst, our evaluation settings are fair for all mentors. The dataset split presented in **Appendix, Tab. S1** applies only to the error source used for training the mentor, not to all error sources. For instance, if Mentor A is trained on adversarial images generated by PIFGSM error source, only the PIFGSM-generated samples are split into training and evaluation sets according to **Appendix, Tab. S1**. All samples from other ID, OOD, and AA error sources are fully used for evaluation. Thus, comparing its performance with that of other mentors on these error sources is fair. It is true that Mentor A may perform better on the PIFGSM error source since the mentor was trained on it. However, even when excluding its performance on the PIFGSM error source, its overall average performance ranking compared to other mentors remains largely unchanged. This observation applies to mentors trained on other error sources as well. Confusion matrices in **Appendix, Fig. S3, S4, and S5** show that the evaluation results on the trained error sources (accuracy in the white background of confusion matrices) do not dominate the average performance (accuracy in the pink background of confusion matrices). \n\nSecond, we appreciate the reviewer's question regarding a potential confounding variable in mentor training—specifically, whether AA, OOD, or ID examples inherently provide more predictive information, or if some error examples are simply easier or harder to learn. To address this, we adopted an alternative dataset splitting strategy for mentor training, distinct from the one in **Appendix, Tab. S1**. For example, we start with the original images from CIFAR-10 and separately apply two perturbation methods, PIFGSM and SpN, to produce two sets of error sources. Next, use 60% of the PIFGSM-generated images to train a mentor model, and reserve the remaining 40% for testing. Similarly, use 60% of the SpN-generated images (corresponding to the same original CIFAR-10 images as the PIFGSM training set) to train another mentor model, and reserve the remaining 40% (corresponding to the same original CIFAR-10 images as the PIFGSM testing set) for testing. \n\nThis approach is applied consistently across mentors trained on other error sources, ensuring that all mentors are trained and evaluated on images derived from the same original CIFAR-10 images but different domain shifts. Consequently, the confounding factor related to the inherent ease or difficulty of learning certain samples is eliminated, leaving only the question of whether AA, OOD, or ID samples inherently provide more predictive information.\n\n**Figures R1, R2, and R3 in [link](https://drive.google.com/file/d/1JtoYDxTmAcNfgN_6SVtoBGO1r7u49T6Q/view?usp=sharing)** present the performance of mentors on various error sources for the CIFAR-10, CIFAR-100, and ImageNet-1K datasets, respectively, under this new data split strategy. The results reveal that training mentors with adversarial attack errors from the mentee leads to a greater improvement in error prediction accuracy compared to training with ID or OOD errors. Consistent with our original finding reported in the paper, this finding confirms that adversarial images carry more predictive information than either ID or OOD images.\n\nWe will add these discussions and new experimental results in the final version.\n\n**[LEYc.2-Claims And Evidence]** Thank you for the suggestions! To support our claims in **Fig. 3**, we computed p-values to compare the error prediction accuracy of mentors trained with different error types. Our primary claim is that mentors trained on the adversarial attack (AA) error type achieve higher accuracy than those trained on in-domain (ID) or out-of-domain (OOD) error types. To validate this, we calculated p-values using a two-tailed t-test and compared the performance of AA-trained mentors to ID-trained mentors and AA-trained mentors to OOD-trained mentors across four different [mentee]-[mentor] settings (ResNet50-ResNet50, ViT-ResNet50, ResNet50-ViT, ViT-ViT) on three datasets: CIFAR-10, CIFAR-100, and ImageNet-1K. All pairwise p-values are below 0.05, indicating that these performance differences are statistically significant. Therefore, our claim is well-supported. We will include these p-value analyses in the final version.\n\nWe also agree with the reviewer's suggestion regarding **Tab 1**. The updated version, now including variance over 3 runs, is shown in **Tab. R1 in [link](https://drive.google.com/file/d/1JtoYDxTmAcNfgN_6SVtoBGO1r7u49T6Q/view?usp=sharing)**. This confirms that the performance differences between our proposed method and the baselines are not simply attributable to noise. We will include this updated table in the final version.", "claims_and_evidence": "Strengths:\n- The paper trains and evaluates their mentor models on a number of settings, across different architectures and datasets, and compares to many relevant baselines.\n- The discussion regarding what training data (between in-distribution, out-of-distribution, and adversarial examples, plus some investigation of level of distortion of the images) is very interesting\n- Investigating the role of how different architectures affect the the accuracy was also interesting.\n- The paper tackles an important problem of predicting when neural networks make errors.\n\n\nWeaknesses:\n- The main technical weakness is the evaluation. In particular, the authors choose to select a data split for each model and each dataset where half of the examples were classified correctly and half of the examples were classified incorrectly. This means that each model is evaluated on different datasets. Plus, after generating the dataset, the remaining examples were used for training, and so the training differ from model to model and setting to setting. This makes comparisons across settings very hard to evaluate. For example, there is a confounding variable where it is unclear if adversarial vs OOD vs ID examples carry more information about predictions themselves, or if the examples for which the models make errors are easier/harder to learn. I think to address this the authors should consider how to control for this setting—or, if a reasonable method to control for this doesn't exist—at least ablating the choice of exactly which examples were chosen for training and evaluation. While this is hopefully easily addressable, I believe that this issue can potentially confound the takeaways from the paper, and so it should be addressed before publication. If this were addressed at all my score would increase.\n- A more minor issue: In, e.g., Figure 3, the variance appears quite large. It would be likely important to run some sort of a statistical test to ensure that the differences are in fact statistically significant before making any claims. Further, Table 1 should include some measure of variance to understand if the difference in performance between the proposed method and baselines (which is itself relatively small) is a result of noise.", "methods_and_evaluation_criteria": "The methods and evaluation criteria make sense, although please refer to Claims and Evidence for additional comments on the validity of the methods.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Please refer to Claims and Evidence for comments.", "relation_to_broader_scientific_literature": "The contributions are connected to relevant literature.", "essential_references_not_discussed": "No essential references are not discussed, as far as I can tell.", "other_strengths_and_weaknesses": "See above.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rpmRQtBAPl", "reviewer": "Reviewer_B54K", "summary": "This paper proposes to use mentor models to predict the errors of mentee models. The mentor model can learn from the mistakes of the mentees on adversarial images and generalize to predict the in-domain and out-of-domain errors of the mentees. In addition, the mentor trained on a mentee generalizes well in predicting the errors of other mentees.\n\n## update after rebuttal\nThe rebuttal solved my concerns, and I will keep my positive rating after the rebuttal if the authors could incorporate the clarification into their draft.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[B54K.1-Other Strengths And Weaknesses]** Thank you for your insightful comments! We address the points raised as follows:\n\nFirst, training a mentor to distinguish specific error types is a promising future research direction, as we have mentioned in **Sec. 5** of our paper. This task is more challenging than predicting whether a mentee will make an error. It requires a mentor to first achieve high accuracy in error prediction before extending its capabilities to distinguish the exact types of errors a mentee may encounter. Therefore, our work serves as a solid foundation for this promising research direction.\n\nSecond, we agree that our method requires access to the mentee's model parameters to generate adversarial images. Training the mentor without using these parameters (i.e., treating the mentee as a black box) presents an interesting and challenging future research direction. However, we respectfully disagree that our mentor needs access to the mentee's training data. Our mentor is instead trained solely on the mentee's behaviors on its evaluation datasets. This is a significant advantage for real-world applications, especially when training the mentee is resource-intensive (e.g., high memory usage) or when the training data of a mentee is inaccessible.\n\nThird, as described in **Appendix, Sec. I**, we conducted experiments evaluating our SuperMentor on additional OOD domains of the ImageNet dataset, including errors arising from spurious correlations. Specifically, we tested our SuperMentor on datasets such as ImageNet9-MIXED-RAND [a], ImageNet9-MIXED-SAME [a], and ImageNet9-MIXED-NEXT [a], which include images with random backgrounds from a random class, random backgrounds from the same class, and random backgrounds from the subsequent class, respectively. These datasets are designed to reveal how models can misclassify objects by focusing on spurious background cues rather than the objects themselves. For instance, an AI model might label an object as a bird based solely on a tree in the background. The experimental results demonstrate that our SuperMentor can still achieve above-chance error prediction performance on these datasets.\n\nWe will include these discussions in the final version of our paper.\n\n[a] Xiao, et al. \"Noise or Signal: The Role of Image Backgrounds in Object Recognition.\" ICLR 2021.\n\n**[B54K.2-Questions For Authors]** Thank you for the reviewer's insightful question! As mentioned in **Sec. 5** of our paper, the reviewer’s point is indeed a promising future research direction. We would like to emphasize that our work serves as an important foundation in this research direction, as mentors with higher error prediction accuracy may enhance the training of mentee models by correcting their mistakes on the fly.\n\n**[B54K.3-Questions For Authors]** We would like to clarify that the number of samples differs between Joint training and AA training. According to **Appendix, Tab. S1**, Joint training uses 25092 samples, whereas AA training uses only 6774 samples. The marginal improvement observed in Joint training may be attributed to the larger training sample size. This suggests that the quantity of samples is not the key factor in enhancing a mentor’s error prediction performance. Training mentors on samples that accurately capture the error patterns of mentees is crucial. For instance, the adversarial images generated using the PIFGSM method in our paper effectively illustrate this point. With only 6774 training samples from PIGFSM—one-fourth the size of the Joint training dataset—our SuperMentor on AA training achieves performance comparable to Joint training. These findings further underscore the significance of our study by providing critical insights into optimal training practices for mentor models.", "claims_and_evidence": "The claims are supported by experimental evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation make sense for the problem setting.", "theoretical_claims": "There is no theoretical claim in this paper.", "experimental_designs_or_analyses": "The experiment design makes sense in this problem setting, and the authors compared the performance of their proposed method with a series of baselines.", "relation_to_broader_scientific_literature": "Previous relevant work on error monitoring relies on the manually defined metrics to estimate the likelihood of errors, while this work aims to train a mentor model to predict various errors that might happen in mentee model predictions.", "essential_references_not_discussed": "I didn't identify any such references.", "other_strengths_and_weaknesses": "Strengths:\n\nThis paper proposes to predict model errors in a unified way, including errors on the in-domain, out-of-domain, and adversarial images. The mentor model is also generalizable to other mentee models and error types. This paper is well-written and easy to follow.\n\nWeaknesses:\n\nThis work focuses on the accuracy of predicting various errors for mentee models but does not distinguish the specific error type. In addition,  applying mentor models requires training and aligning with the mentee models, which relies on the accessibility of mentee models and training data and, therefore, might limit its applicability. In addition, the evaluation of some common shifts between training and test data might be worth exploring, such as errors from spurious correlation, which can be categorized as ID errors and have great potential in applications.", "other_comments_or_suggestions": "NaN", "questions_for_authors": "Except for the weaknesses I mentioned above, I only have minor questions:\n\n1. Given that mentor models can recognize ID errors, will the mentor model facilitate the training of mentee models?\n\n2. The paper mentioned that the joint training with ID, OOD, and AA data marginally outperforms the mentor trained only with AA. Was the number of samples the same for joint training or AA training? The improvement might be from the increased number of samples", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PvOwA47A2S", "reviewer": "Reviewer_YWy5", "summary": "The authors trained a secondary model (a mentor network) to predict whether a deep learning model would make a mistake or not. A mentor network consists of a backbone and two MLPs. The first MLP is trained to replicate latent variables of the mentee network whose answers are analyzed, and the second MLP is trained to predict the correctness of the mentee network. More specifically, the authors analyzed mentors’ accuracy on in-domain, out-of-domain and adversarial inputs.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[YWy5.1-Claims And Evidence]**  First, our paper serves as a proof-of-concept, showing that training mentors with adversarial attack (AA) errors from the mentee has a greater impact on improving error prediction accuracy than training with in-domain (ID) or out-of-domain (OOD) errors. In **Tab. 1**, we use the adversarial images produced in the final iteration of the PIFGSM method to train SuperMentor. \n\nAs requested by the reviewer, to probe the decision boundary of the mentees, we used adversarial images from all iterations of the PIFGSM method (with 3 iterations in total) to train our SuperMentor and named this new SuperMentor as \"En-SuperMentor\". By doing so, we can further diversify the samples and better capture the mentee's decision boundary. As shown in **Tab. R2 in [link](https://drive.google.com/file/d/1JtoYDxTmAcNfgN_6SVtoBGO1r7u49T6Q/view?usp=sharing)**, En-SuperMentor boosts average accuracy from 77.0% to 84.1%, which is significantly better than the other baselines and original SuperMentor. \n\nNext, as requested, we also added two quantitative loss landscape analyses using the method in [b]. In these analyses, we apply small perturbations to the mentor's weights and monitor the resulting changes in SuperMentor’s average accuracy. \n\nIn the first loss landscape analysis (**Fig. R4 in [link](https://drive.google.com/file/d/1JtoYDxTmAcNfgN_6SVtoBGO1r7u49T6Q/view?usp=sharing)**), we examine mentors trained on three different error types: ID, OOD, and AA from the ImageNet-1K dataset. The results indicate that mentors trained on AA errors exhibit a wider loss landscape than those trained on ID or OOD errors. This suggests that training mentors with adversarial images makes them more robust to weight perturbations, thereby enhancing their ability to capture the generic features for predicting the mentee's decision-making. This finding strongly supports our loss landscape analysis discussed in **Sec. 4.1**.\n\nThe second loss landscape analysis (**Fig. R5 in [link](https://drive.google.com/file/d/1JtoYDxTmAcNfgN_6SVtoBGO1r7u49T6Q/view?usp=sharing)**) examines the mentor’s error prediction performance using two different backbones: ResNet50 and ViT. The figure shows that mentors with ViT architectures have a considerably wider loss landscape compared to those with ResNet50. This further reinforces our claims in **Sec. 4.2** that transformer-based mentor models outperform their ResNet-based counterparts in error prediction.\n\nWe will add these results and discussions in the final version.\n\n[b] Li et al. \"Visualizing the loss landscape of neural nets.\" NeruIPS 2018.\n\n**[YWy5.2-Questions For Authors]** The MLP branch associated with distillation loss $L_d$ enables the mentor to mimic the mentee's predictions, and the MLP branch associated with loss $L_r$ ​is responsible for predicting whether the mentee will make mistakes on the input images. Without the first MLP for $L_d$​, the mentor cannot effectively capture the learning patterns of the mentee within its feature extraction backbone. We conducted an ablation study (see **Appendix, Tab. S4**) by removing the first MLP (i.e., eliminating the distillation loss $L_d$). The experimental results indicate that excluding the first MLP leads to a significant decrease in the mentor’s performance across all datasets.\n\n**[YWy5.3-Questions For Authors]** The concept of out-of-domain data remains somewhat ambiguous (Guérin, et al., 2023) in computer vision. In our paper, we define out-of-domain data as any data that falls outside the training domain (see **Sec. 3.3**), following the settings outlined in (Hendrycks & Gimpel, 2016; Luo et al., 2021; Yu et al., 2022). Although corrupted images have the same semantic meaning as clean images, the corruption methods introduce distributional shifts relative to the training domain. Whether these noisy images are classified as ID or OOD is irrelevant; what truly matters is that our SuperMentor can predict the errors of mentees when noise is added to the original images. \n\n**[YWy5.4-Questions For Authors]** According to the suggested paper by the reviewer, the natural adversarial samples refer to the samples that always lead to wrong predictions regardless of which AI model we use. Following the same definition, we defined sets $N_1, N_2, N_3$ in the caption of **Tab. R3 in [link](https://drive.google.com/file/d/1JtoYDxTmAcNfgN_6SVtoBGO1r7u49T6Q/view?usp=sharing)**. The result in **Tab. R3** indicates that the natural adversarial samples in $N_1$ play a key role in explaining SuperMentor’s strong generalization across different architectures. However, SuperMentor can also perform well on the non-natural adversarial samples ($N_2$ and $N_3$​). For example, it achieves 77.7% accuracy on the set $N_3$ for ID error types. This demonstrates that SuperMentor’s robust generalization ability is not limited solely to natural adversarial samples.\n\nWe will cite the suggested paper and add these new results in the final version.", "claims_and_evidence": "The topic is of high importance because probing deep learning models’ decision-making process will help us build reliable models, but it remains unclear what we can learn from the authors’ reported results. First, the reported accuracy of mentor networks is not much higher than the baseline measures. Second, a mentor network is another black box, meaning we still do not fully understand their decision-making process. The authors speculated that mentor network errors can reveal the loss landscape of mentee networks (see loss landscape analyses), but they do not provide any quantitative results. If they can provide some evidence supporting that mentors can explain decision-boundary (or even loss landscape) to mentees, their study may have a profound impact.", "methods_and_evaluation_criteria": "The authors compare their own algorithm with multiple baselines using multiple datasets, which is a strong point.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "See my comments above.", "relation_to_broader_scientific_literature": "The topic is highly important. But I do not think using another black-box model (i.e., mentor) can help us to better understand deep learning models (i.e., mentee) decision-making. The results presented are not sufficient.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Q1: It is not clear why a mentor network would need two MLPs, each of which is independently trained. What happens if the first MLP, trained on the embedding of the mentee network, is removed?  \nQ2: The authors created out-of-domain examples by adding noises. As the corrupted inputs have the same semantic meanings, I do not think they are out-of-domain inputs. Can the authors further explain why the corrupted inputs are used as out-of-domain inputs?\n\nQ3: The authors found that mentors can generalize across mentees. This result may be related to natural adversarial examples that many ImageNet models fail to correctly classify. Can the authors provide any insights into the implications of natural adversarial examples for the mentor-based analysis?\n\nReference for natural adversarial examples: Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., & Song, D.X. (2019). Natural Adversarial Examples. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15257-15266.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper conducts comprehensive empirical evaluations that train a “mentor” model to predict the behaviors of a “mentee” model. The topic is considered important as probing deep learning models’ decision-making processes will help us build reliable models.
Decision: Accept (poster)