Prior Distribution and Model Confidence

## Abstract

## 1 Introduction

In recent years, we have seen unprecedented development of deep learning models and an extension of the problems that can be effectively solved by them. They have dramatically advanced AI capabilities, but their training paradigm imposes fundamental constraints. Leading companies, such as OpenAI, have utilized nearly all accessible data to build these models, which require immense computational resources and time [1]. However, this approach presents major limitations, such as the difficulty in optimizing model architectures and the exclusivity of achieving such results to a handful of research institutions and corporations. Consequently, most scientists and engineers are limited to using pre-trained models without the ability to modify or improve them [2].
Shortly after deployment, numerous instances of erroneous outputs, termed ’hallucinations,’ were identified in these models. These are cases where the response sounds convincing and well-structured but conveys false or unverified information [3]. A significant challenge with these errors is the difficulty in detecting their root causes through automated means, even though users can often recognize them with ease. This poses a serious risk in industrial environments, where such inaccuracies can trigger unforeseen system behaviors or failures in AI-driven technologies.

Although understanding the factors behind such behavior in AI systems is challenging due to their inherent complexity, models of that type (such as Huawei’s PanGu-$\Sigma$) often contain over 1 trillion parameters [4]. Mathematical structures of this magnitude are extremely difficult to analyze from both theoretical and technical perspectives [5].

The field of computer vision continues to be highly suitable for theoretical exploration. Unlike NLP, where constructing representative datasets for industrial-level models is difficult due to the enormous data requirements, computer vision faces fewer such challenges. As one of the earliest domains enhanced by deep learning, it now benefits from a rich collection of well-curated datasets ideal for research purposes [6].

Consequently, using computer vision models to investigate the nature of ”hallucinations” appears more reasonable. As is well-known, training data is a primary factor influencing model performance, as it fundamentally determines the model’s scope and applicability [7]. Therefore, understanding the distribution of training data is crucial for assessing a model’s predictability [8, 9].

This paper presents a foundational approach for designing algorithms to detect hallucinations. The core idea is to leverage the latent space representation of training data to assess the model’s predictive capabilities on specific instances. While the current study focuses on deep learning models in computer vision, the proposed methodology is general enough to be extended to other machine learning and deep learning domains, including natural language processing.

## 2 Related Work

There are several approaches developed to mitigate “hallucinations” without recreating the model.

A practical first step is to adapt the model’s parameters for a given task using tuning strategies like fine-tuning or alignment [10, 11, 12, 13] . However, these techniques often require significant amounts of extra training data and may still lack dependable reliability.

Another solution is to mitigate hallucinations by using RAG (Retrieval-Augmented Generation) techniques, which integrate information retrieval with generation. This allows the model to reference real-world information, making the output more accurate and reliable [14]. Another option is to apply statistical and information-theoretic methods to detect hallucinations, aiming to flag outputs that deviate from expected or reliable patterns [15, 16]. While these methods focus on analyzing the output of deep learning networks, they often overlook the distribution of the training data. Yet, the structure and spread of training data across complex feature spaces can critically impact a deep learning model’s performance [17, 18]. Accurately measuring and modeling input data distribution is a crucial strategy for enhancing model performance in machine learning. Numerous methods have been developed to address this challenge effectively [17, 19].

First, training data is formally described and cataloged using tools like Data Cards, and visualized through techniques such as t-SNE or UMAP applied to CLIP or DINO embeddings. Mislabeled instances can be identified and corrected with Confident Learning [20], while Dataset Cartography [21] enables categorization of data points into ”easy,” ”hard,” or ”ambiguous” classes. Second, distribution shift and coverage are measured. To detect samples outside the training distribution, OOD (out-of-distribution) methods like ODIN [22], Mahalanobis Distance [23], Energy-based OOD [24], and Open OOD [25] are used. As an opposite approach, we can start from the defined distribution of the training data and try to fit the data to it. The Core Set method [26] selects diverse representatives to cover the input space. To select uncertain and diverse examples, the BADGE method [27], based on gradient embeddings, can be used. If you have a set of specific features, the Submodular Selection method [28] ensures their coverage. In the case of class imbalance, the Focal Loss method [29] reweights samples from imbalanced classes. Methods like LAION [30] and DataComp [31] demonstrate that targeted filtering toward the desired distribution can significantly outperform simply increasing dataset size in terms of accuracy.

In addition, several methods focus on evaluating training data by analyzing the contribution of its individual components. Influence Functions [32] and TracIn [33] help quantify the impact of specific training samples on model predictions, making it possible to identify non-representative data points that may degrade performance. Other evaluation approaches, such as VQA v2 [34], aim to reduce shortcut learning by rebalancing answer priors, thereby encouraging models to rely on visual information. Similarly, ObjectNet is designed to assess generalization capabilities under varying viewpoints and backgrounds.

## 3 Methods

This manuscript presents a method for understanding the influence of the prior data distribution on the confidence of prediction, independent of the predictive signal of the model.

To investigate the effect of data distribution on model confidence, we selected several models trained on the same dataset. Specifically, all models were trained using the ImageNet-1K dataset [35], ensuring a consistent distribution of training data between experiments. In order to understand the difference between the performance of our algorithm, we intentionally took several models from different classes with different numbers of parameters. The chosen models are: ResNet type models(Resnet-101 and Resnet-50 [36]), Visual Transformers [37] (Deit-T, Deit-S and Deit-B) and a relatively small model ShuffleNet-V2 [38]. The brief description of the models is shown in Table [1](https://arxiv.org/html/2509.05485v1#S3.T1).

**Table 1: Overview of the models used in our experiments. All models were trained on the ImageNet-1K dataset to ensure consistent data distribution.**
| Model | Training Dataset | Architecture Type | Reference | # Parameters |
| --- | --- | --- | --- | --- |
| ResNet-101 | ImageNet-1K | Convolutional (CNN) | [36] | 44.5M |
| ResNet-50 | ImageNet-1K | Convolutional (CNN) | [36] | 25.6M |
| ShuffleNet-V2 | ImageNet-1K | Lightweight CNN | [38] | 2.3M |
| DeiT-Tiny | ImageNet-1K | Vision Transformer (ViT) | [37] | 5.7M |
| DeiT-Small | ImageNet-1K | Vision Transformer (ViT) | [37] | 22.1M |
| DeiT-Base | ImageNet-1K | Vision Transformer (ViT) | [37] | 86.4M |

To understand the data distribution on which the models were trained, we analyzed the structure of the ImageNet-1K training set by computing image embeddings using several pretrained models. Specifically, we selected a diverse set of architectures to generate embeddings for each image in the dataset. These embedding-based representations allowed us to examine the statistical properties of the data in different feature spaces. The analysis provides insight into how various models perceive and encode the visual information present in the training distribution. It is worth mentioning that we took only the ”train” part of the set since exactly on this set the models in Table [1](https://arxiv.org/html/2509.05485v1#S3.T1) were trained. For the embedding analysis, we chose the following models: The Dino-V1 model [39], Dino-V2 models [40] with different embedding sizes (384, 768, 1024) and the MobileNet-V2 model [41]. Our choice of embedding models was motivated by two key considerations: the model size and the nature of the data used for pre-training. To assess the sensitivity of our algorithm to embedding dimensionality, we evaluated its performance on closely related models with varying embedding sizes. In addition, we were interested in understanding the effects of the data on which the embedding models were trained. We include a brief description of the embedding models in Table  [2](https://arxiv.org/html/2509.05485v1#S3.T2).

**Table 2: Description of the models and their embedding sizes.**
| Model | Training Data | Embedding Size | Reference |
| --- | --- | --- | --- |
| DINO-V1 ViT-S/16 | ImageNet-1K (self-supervised) | 384 | [4] |
| DINO-V1 ViT-B/16 | ImageNet-1K (self-supervised) | 768 | [4] |
| DINO-V1 ViT-B/8 | ImageNet-1K (self-supervised) | 768 | [4] |
| DINO-V2 ViT-S/14 | 142M curated dataset (no labels) | 384 | [5] |
| DINO-V2 ViT-B/14 | 142M curated dataset (no labels) | 768 | [5] |
| MobileNet-V2 | ImageNet-1K (supervised) | 1000 | [6] |

In order to investigate the effect of the prior data distribution on the prediction capabilities of the classification models, we have chosen two distinct datasets:
ObjectNet[42] and imageNetV2[43]. We divided the imageNetV2 into train and internal test sets with a split (75% / 25%) and left the ObjectNet dataset as the external test set. We used only ObjectNet classes that coincide with the classes in the original ImageNet1K dataset (which gave us $10\,121$ images). We used the train dataset to find the optimal parameter: the number of neighbors $N$. We divided the test sets (external and internal) into three subsets grouped by the labels. So, images with the same label can only be present in one subset. The goal of this division is to report the variation of the metrics.

For the calculation of embeddings, we used cosine similarity [44], which is widely regarded as an effective metric to identify neighbors in high-dimensional spaces. For storing and querying the embeddings, we used ChromaDB [45] with ClickHouse [46] as the backend server to enable efficient retrieval. All calculations were performed on a MacBook with an Apple M1 chip.

Figure: Figure 1: The scheme of the experiment. With (*) the training (adjusting) steps are indicated. With (**) supplementary combination predictions steps are shown
Refer to caption: figures/prior.png

## 4 Experiments

Confidence estimation was conducted on both the training and test sets. In the training set, we adjusted the parameter $N$
to optimize performance. The procedure for adjusting confidence in the training set involves the following steps. An illustration of both the training and test pipelines is provided in Figure [1](https://arxiv.org/html/2509.05485v1#S3.F1).

Figure: (a) The Confidence Curve
Refer to caption: figures/accuracy_vs_coverage_multiple_N.png

**Table 3: $N$ and best Normalized Confidence Gain for each classification model across embedding models.**
| Classification Model | DINO-V1 ViT-B/16 | DINO-V1 ViT-B/8 | DINO-V1 ViT-S/16 | DINO-V2 ViT-B/14 | DINO-V2 ViT-S/14 | MobileNet-V2 |
| --- | --- | --- | --- | --- | --- | --- |
| deit_base_patch16_224 | 1 / 0.398 | 1 / 0.441 | 1 / 0.368 | 24 / 0.480 | 4 / 0.419 | 1 / 0.194 |
| deit_small_patch16_224 | 1 / 0.408 | 1 / 0.449 | 1 / 0.379 | 7 / 0.472 | 2 / 0.427 | 1 / 0.199 |
| deit_tiny_patch16_224 | 1 / 0.402 | 1 / 0.433 | 1 / 0.379 | 23 / 0.442 | 4 / 0.406 | 1 / 0.218 |
| resnet101 | 1 / 0.394 | 1 / 0.434 | 1 / 0.364 | 60 / 0.477 | 4 / 0.417 | 1 / 0.195 |
| resnet50 | 1 / 0.407 | 1 / 0.448 | 1 / 0.380 | 24 / 0.487 | 2 / 0.426 | 1 / 0.209 |
| shufflenet_v2_x1_0 | 1 / 0.418 | 1 / 0.436 | 1 / 0.401 | 7 / 0.417 | 4 / 0.395 | 1 / 0.244 |

- 1.
For each image in the original training dataset ImageNet-1K (let us call it Base Image Set), we calculate the embeddings for embedding models presented in Table [2](https://arxiv.org/html/2509.05485v1#S3.T2) using cosine similarity. We store these embeddings in the Base Embedding Database. Although the Base Image Set serves as a training set for the models we investigated, we do not fine-tune its parameters. Therefore, the word ”training” could be misleading.
- 2.
For the training dataset (a subset of ImageNet-V2), we compute the embeddings for each image and identify $1000$ closest neighbors from a base embedding database. This allows us to incorporate information about the local distribution of embeddings. We choose this particular number to ensure that we fully describe the neighbors of a particular image. We refer to this collection as the Train Embedding Set. The term ”train” emphasizes that this dataset is used to tune the parameter $N$, which is then kept fixed during the evaluation on the test sets.
- 3.
For each image in the train image set, we calculate and store the predictions of the classification model.
- 4.
For each image in the train set, we calculate the following statistics. We use the parameters $L_{threshold}$ and $N$ as the threshold value for the distance of the neighbors and the necessary number of neighbors within the proximity $L_{threshold}$. Should the quantity of neighboring elements for the embedding of the image exceed $N$ within the $L_{threshold}$ vicinity, we accept the model’s prediction as valid. Conversely, if this threshold is not met, we conclude that the model lacks sufficient confidence to make a prediction. We calculate the accuracy for all images where the confidence is enough. Let us call the percentage of approved images the Coverage. The Confidence Curve would be the dependence of the accuracy upon the coverage. The examples of Confident Curves for ResNet-101 & DINO-V2 ViT-B/14 for three different $N$ are presented in Figure [2](https://arxiv.org/html/2509.05485v1#S4.F2)(a). In Figure LABEL:fig:two_model_groups(b) Maximum Confidence Gain and Confidence Gain are shown for the same embedding / model pair, and $N=1$. For more details on how the values are calculated, please see Appendix 1.
- 5.
We find the values of $N$ that maximize the Confidence Gain (the ratio between the Maximum Confidence Gain and the Confidence Gain). We use the value of neighbors for the test sets.
- 6.
We repeated steps 3,4,5 for each of the classification models.

**Table 4: The Normalized Confidence Gain for different pairs of embedding/classification models and for the combination of embedding models. Benchmark accuracy is shown for reference. All values are mean ± std across three label subsets for internal and external test sets.**
| Set | Classification Model | Benchmark Acc. | DINO-V1 ViT-B/16 | DINO-V1 ViT-B/8 | DINO-V1 ViT-S/16 | DINO-V2 ViT-B/14 | DINO-V2 ViT-S/14 | MobileNet-V2 | Combination |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Internal Test | deit_base_patch16_224 | 0.696 $\pm$ 0.014 | 0.390 $\pm$ 0.052 | 0.414 $\pm$ 0.048 | 0.357 $\pm$ 0.035 | 0.449 $\pm$ 0.077 | 0.404 $\pm$ 0.046 | 0.194 $\pm$ 0.046 | 0.361 $\pm$ 0.058 |
| deit_small_patch16_224 | 0.666 $\pm$ 0.019 | 0.376 $\pm$ 0.068 | 0.403 $\pm$ 0.071 | 0.343 $\pm$ 0.062 | 0.431 $\pm$ 0.076 | 0.376 $\pm$ 0.058 | 0.189 $\pm$ 0.058 | 0.320 $\pm$ 0.060 |  |
| deit_tiny_patch16_224 | 0.582 $\pm$ 0.037 | 0.376 $\pm$ 0.076 | 0.388 $\pm$ 0.079 | 0.353 $\pm$ 0.078 | 0.408 $\pm$ 0.070 | 0.380 $\pm$ 0.067 | 0.202 $\pm$ 0.069 | 0.366 $\pm$ 0.067 |  |
| resnet101 | 0.687 $\pm$ 0.025 | 0.345 $\pm$ 0.074 | 0.369 $\pm$ 0.066 | 0.308 $\pm$ 0.072 | 0.410 $\pm$ 0.106 | 0.366 $\pm$ 0.058 | 0.169 $\pm$ 0.065 | 0.316 $\pm$ 0.071 |  |
| resnet50 | 0.676 $\pm$ 0.014 | 0.377 $\pm$ 0.050 | 0.408 $\pm$ 0.049 | 0.347 $\pm$ 0.050 | 0.446 $\pm$ 0.075 | 0.398 $\pm$ 0.044 | 0.189 $\pm$ 0.068 | 0.344 $\pm$ 0.061 |  |
| shufflenet_v2_x1_0 | 0.538 $\pm$ 0.022 | 0.363 $\pm$ 0.056 | 0.374 $\pm$ 0.063 | 0.345 $\pm$ 0.056 | 0.379 $\pm$ 0.053 | 0.357 $\pm$ 0.046 | 0.220 $\pm$ 0.050 | 0.351 $\pm$ 0.046 |  |

In Figure [3](https://arxiv.org/html/2509.05485v1#S4.F3), the results of the adjustment (or training) are shown. On the x-axis, the number of neighbors $N$ is presented, and on the y-axis, the calculated Normalized Confidence Gain is depicted. We can see that the highest Normalized Confidence Gain value is achieved for all models in besides the ShuffleNet-V2 by the DINO-V2 ViT-B / 14 embedding model, which is so far the strongest model among embedding models in terms of the number of parameters. Interestingly, the second version of the DINO models outperforms the first version. In table [3](https://arxiv.org/html/2509.05485v1#S4.T3), the best parameter $N$ and the corresponding Normalized Confidence Gain are presented.

Figure: (a) DeiT-Base
Refer to caption: figures/deit_base_patch16_224_normalized_adjusted_integral.png

For the test pipeline, we use the number of parameters $N$, which maximizes the Normalized Confidence Gain for the specific pair of the classification model and the embedding model. We use this value to calculate the metric for the Internal Test and External Test sets. The metrics and their standard deviations (for 3 subsets of each of the External and Internal Tests) are reported in Table [4](https://arxiv.org/html/2509.05485v1#S4.T4). We can observe that the highest value of Normalized Confidence
Gain is reached almost for all cases by Dino-v2 ViT-B14. As mentioned earlier, this is quite logical and can be explained by the model’s higher capacity and the higher resolution of the input images.

The next logical step would be to combine several embedding models (or use an approach similar to the ensemble of models, as in [47]), which has been done using greedy heuristics. We fix the same coverage for each embedding model. We take the model with the highest Normalized Confidence Gain and make predictions. We accept a portion of the predictions and leave the rest to other models. It should be clear that the total coverage for all embedding models is slightly higher than the parameter $coverage$ for individual models, but is not fixed. The results of the experiments (accuracy vs. coverage) are presented in Figure [4](https://arxiv.org/html/2509.05485v1#S4.F4). The Normalized Confidence Gain for the combination of embedding models is shown in Table [4](https://arxiv.org/html/2509.05485v1#S4.T4) in the column ”Combination”. The observed Confidence Gain for the combination of embedding models is lower than that of the best-performing individual model. In particular, at very low coverage values for the internal dataset, the accuracy of all models converges to similar levels. This suggests that the underlying data distribution plays a more decisive role than the intrinsic performance of the models. A plausible explanation is that, within the test datasets, certain images are located in close proximity (in the embedding space) to images from the training set. Consequently, under such conditions, the specific choice of the classification model becomes less critical. For the external dataset, the same conclusions cannot be drawn. This is probably related to the fact that the images in the external dataset (ObjectNet) exhibit a greater distributional shift relative to the original training set (ImageNet-1K) than those in the internal test set.

Figure: (a) ResNet50 — Internal
Refer to caption: figures/resnet50_accuracy_vs_coverage_INTERNAL.png

## 5 Discussion

From the experiment chapter, we can infer that a more sophisticated embedding model is more likely to have a better Confidence Curve (or higher Normalized Confidence Gain). With ”sophisticated” we mean a larger number of dimensions, larger size of the entering image, and a richer dataset on which the model was trained. In particular, the dimensions of the images and the capacity of the embedding model significantly determine the performance.

We showed that combining embeddings from multiple models can be a promising strategy for integrating diverse representation spaces. However, the Normalized Confidence Gain obtained from such combinations was lower than that of the best performing individual embedding model. Unlike classical ensemble methods , where aggregating weaker models often leads to improved performance, we did not observe a performance gain in this setting. We identify two possible reasons for this. First, our combination strategy was a simple greedy approach and more sophisticated methods could yield better results. For example, a meta-classifier could be trained to select the most suitable embedding model for confidence estimation on a per-sample basis. Second, most of the models in our study were trained on only two distinct datasets. We expect that training on more diverse datasets could enrich the combined confidence estimates. However, achieving such diversity is challenging, as off-the-shelf models are typically trained on large, overlapping datasets, limiting the potential for complementary knowledge.

We can see that after training, most of the models achieved the highest Normalized Confidence Gain with $N=1$. Only the best-performing embedding models achieved the maximum value at significantly larger values of $N$ ($N>1$). We believe that the more complicated the model, the larger the optimal value of $N$.

Our experiments were aimed solely at the image classification model. However, we suggest that a similar approach can work for other areas of Machine Learning, particularly for Natural Language Processing. But there is a caveat: we do not know how to partition the data efficiently. Images are distinct entities, and it is logical to compute the embeddings for each image. How exactly to divide the text corpora is unclear. There are two main questions in this investigation. What size of window to use and what jump step to use? Perhaps we need to have embedding collections of different window sizes, which could enrich the understanding of the neighborhood locality. Regarding the size of the jump, it would be interesting to calculate the embeddings with a step of one token. However, it would be computationally very heavy. In fact, English Wikipedia contains approximately 13.5 billion token instances across all articles, according to the TokTrack dataset [48] (as of 2017). The training corpora used for large language models (LLMs) are estimated to be two to three orders of magnitude larger, although the exact size is unknown. For example, whereas GPT-3 was trained on approximately 499 billion tokens [1]. In comparison, ImageNet-1K (used in this paper) contains approximately 1 million images and, correspondingly, 1 million embedding vectors. Moreover, it remains unclear how much efficiency is lost in neighborhood locality calculations when the step size exceeds a single token.

The current framework does not directly improve the performance of the classification model (or potentially other models). However, it enables the identification of samples that are out-of-distribution with respect to the original training set, thus making the model more confident in certain predictions.

## 6 Conclusion

In this paper, we showed that knowledge of the training dataset and its embeddings can be beneficial for determining a model’s confidence in its predictions on newly seen data. The presented framework provides an alternative way to estimate prediction confidence. In particular, we significantly improved the accuracy of classification models without additional retraining—only by adjusting the number of neighbors—while skipping a minimal number of images that lie far from the training distribution. We demonstrated that using a single embedding model to filter out low-confidence samples can improve overall accuracy. Finally, we propose that the framework is not limited to object recognition and computer vision tasks, but could also be applied to other sensitive areas, such as Natural Language Processing, where hallucinations are common. Code is available at [GitHub repository](https://github.com/maksimkazanskii/prior_hallucinations) ([https://github.com/maksimkazanskii/prior_hallucinations](https://github.com/maksimkazanskii/prior_hallucinations)).

## References

- [1]
Tom Brown et al.
Language models are few-shot learners.
In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901, 2020.
- [2]
Rishi Bommasani et al.
On the opportunities and risks of foundation models.
arXiv preprint arXiv:2108.07258, 2021.
- [3]
L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu.
A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.
arXiv preprint arXiv:2311.05232, 2023.
- [4]
X. Ren, P. Zhou, X. Meng, X. Huang, Y. Wang, W. Wang, P. Li, X. Zhang, A. Podolskiy, G. Arshinov, A. Bout, I. Piontkovskaya, J. Wei, X. Jiang, T. Su, Q. Liu, and J. Yao.
Pangu-$\sigma$: Towards trillion-parameter language model with sparse heterogeneous computing.
arXiv preprint arXiv:2303.10845, 2023.
- [5]
N. Hudson, J. G. Pauloski, M. Baughman, A. Kamatar, M. Sakarvadia, L. Ward, R. Chard, A. Bauer, M. Levental, W. Wang, W. Engler, O. P. Skelly, B. Blaiszik, R. Stevens, K. Chard, and I. Foster.
Trillion parameter ai serving infrastructure for scientific discovery: A survey and vision.
arXiv preprint arXiv:2402.03480, 2024.
- [6]
Z. Wan.
A survey of dataset refinement for problems in computer vision.
ACM Computing Surveys, 2024.
- [7]
X. Chen, Z. Ma, X. Zhang, S. Xu, S. Qian, J. Yang, D. F. Fouhey, and J. Chai.
Multi-object hallucination in vision-language models.
arXiv preprint arXiv:2407.06192, 2024.
- [8]
N. G. Győri et al.
Training data distribution significantly impacts the accuracy and precision of parameter estimates.
Quantitative Imaging in Medicine and Surgery, 2022.
- [9]
S. Dabiri, V. Lioutas, B. Zwartsenberg, Y. Liu, M. Niedoba, X. Liang, D. Green, J. Sefas, J. W. Lavington, F. Wood, and A. Scibior.
Realistically distributing object placements in synthetic training data improves the performance of vision-based object detection models.
arXiv preprint arXiv:2305.14621, 2023.
- [10]
S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, and G. Wang.
Instruction tuning for large language models: A survey.
arXiv preprint arXiv:2308.10792, 2023.
- [11]
Y. Wu, J. Li, Z. Guo, and L. Li.
Learning like humans: Resource-efficient federated fine-tuning through cognitive developmental stages.
arXiv preprint arXiv:2508.00041, 2025.
- [12]
V. Lialin, V. Deshpande, and A. Rumshisky.
Scaling down to scale up: A guide to parameter-efficient fine-tuning.
arXiv preprint arXiv:2303.15647, 2023.
- [13]
N. J. Prottasha, A. Mahmud, M. S. I. Sobuj, P. Bhat, M. Kowsher, N. Yousefi, and Ö. Ö. Garibay.
Parameter-efficient fine-tuning of large language models using semantic knowledge tuning.
Scientific Reports, 14:30667, 2024.
- [14]
P. Béchard and O. M. Ayala.
Reducing hallucination in structured outputs via retrieval-augmented generation.
arXiv preprint arXiv:2404.08189, 2024.
- [15]
S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal.
Principled detection of hallucinations in large language models using entropy-based uncertainty estimators.
Nature, 2024.
- [16]
Jiawei Li.
Principled detection of hallucinations in llms via multiple-testing framework.
arXiv preprint arXiv:2508.18473, 2025.
- [17]
Zayd Hammoudeh and Daniel Lowd.
Training data influence analysis and estimation: a survey.
Machine Learning, 113:2351–2403, 2024.
- [18]
J. Lin et al.
Measuring the effect of training data on deep learning: A query-wise perspective.
In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 13253–13271. PMLR, 2022.
- [19]
Lakpa Tamang, Mohamed Reda Bouadjenek, Richard Dazeley, and Sunil Aryal.
Handling out-of-distribution data: A survey.
arXiv preprint arXiv:2507.21160, 2025.
- [20]
Curtis G. Northcutt, Lu Jiang, and Isaac L. Chuang.
Confident learning: Estimating uncertainty in dataset labels.
In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pages 16921–16940, 2021.
- [21]
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi.
Dataset cartography: Mapping and diagnosing datasets with training dynamics.
In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9275–9293. Association for Computational Linguistics, 2020.
- [22]
Shiyu Liang, Yixuan Li, and R. Srikant.
Enhancing the reliability of out-of-distribution image detection in neural networks.
In International Conference on Learning Representations (ICLR), 2018.
- [23]
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin.
A simple unified framework for detecting out-of-distribution samples and adversarial attacks.
In Advances in Neural Information Processing Systems (NeurIPS), volume 31, 2018.
- [24]
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li.
Energy-based out-of-distribution detection.
In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 21464–21475, 2020.
- [25]
Jingkang Yang, Yiyou Li, Juntao Li, Chixiang Xu, Yuxuan Wang, Chenyang Zhang, Bing Xu, and Ziwei Lin.
Openood: Benchmarking generalized out-of-distribution detection.
In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 29983–29997, 2022.
- [26]
Ozan Sener and Silvio Savarese.
Active learning for convolutional neural networks: A core-set approach.
In International Conference on Learning Representations (ICLR), 2018.
- [27]
Jordan Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal.
Deep batch active learning by diverse, uncertain gradient lower bounds.
In International Conference on Learning Representations (ICLR), 2020.
- [28]
Kai Wei, Rishabh Iyer, and Jeff Bilmes.
Submodularity in data subset selection and active learning.
In Proceedings of the 32nd International Conference on Machine Learning (ICML), volume 37 of Proceedings of Machine Learning Research. PMLR, 2015.
- [29]
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.
Focal loss for dense object detection.
In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980–2988. IEEE, 2017.
- [30]
Christoph Schuhmann, Romain Vencu, Romain Beaumont, Cade Gordon, Ross Wightman, Mehdi Cherti, Thea Coombes, Ashish Katta, Charlie Mullis, Mitchell Wortsman, Patrick Schramowski, Shubhendu Kundurthy, Katherine Crowson, Ludwig Schmidt, Robin Kaczmarczyk, and Jenia Jitsev.
LAION-400M: Open dataset of CLIP-filtered 400 million image-text pairs.
arXiv preprint arXiv:2111.02114, 2021.
- [31]
Syed Yousuf Gadre, Gabriel Ilharco, Mitchell Wortsman, Ludwig Schmidt, and Hannaneh Hajishirzi.
Datacomp: In search of the next generation of multimodal datasets.
In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 13271–13295, 2023.
- [32]
Pang Wei Koh and Percy Liang.
Understanding black-box predictions via influence functions.
In Proceedings of the 34th International Conference on Machine Learning (ICML), volume 70 of Proceedings of Machine Learning Research, pages 1885–1894. PMLR, 2017.
- [33]
Gaurav Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan.
Estimating training data influence by tracing gradient descent.
In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 20013–20024, 2020.
- [34]
Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh.
Making the V in VQA matter: Elevating the role of image understanding in visual question answering.
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6904–6913. IEEE, 2017.
- [35]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.
Imagenet: A large-scale hierarchical image database.
In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE, 2009.
- [36]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Deep residual learning for image recognition.
In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 770–778, 2016.
- [37]
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou.
Training data-efficient image transformers & distillation through attention.
In Proceedings of the 38th International Conference on Machine Learning (ICML), pages 10347–10357. PMLR, 2021.
- [38]
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun.
Shufflenet v2: Practical guidelines for efficient cnn architecture design.
In Proceedings of the European Conference on Computer Vision (ECCV), pages 116–131, 2018.
- [39]
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.
Emerging properties in self-supervised vision transformers.
arXiv preprint arXiv:2104.14294, 2021.
- [40]
Maxime Oquab, Thomas Darcet, Theo Moutakanni, Maciej Szafraniec, Vladislav Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Wojciech Galuba, et al.
Dinov2: Learning robust visual features without supervision.
arXiv preprint arXiv:2304.07193, 2023.
- [41]
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.
Mobilenetv2: Inverted residuals and linear bottlenecks.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4510–4520, 2018.
- [42]
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Joshua Tenenbaum, and Boris Katz.
Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models.
In Advances in Neural Information Processing Systems (NeurIPS), pages 9448–9458, 2019.
- [43]
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar.
Do imagenet classifiers generalize to imagenet?
In Proceedings of the 36th International Conference on Machine Learning (ICML), pages 5389–5400. PMLR, 2019.
- [44]
Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze.
Introduction to Information Retrieval.
Cambridge University Press, 2008.
- [45]
Chroma Team.
Chroma: The ai-native open-source embedding database.
[https://github.com/chroma-core/chroma](https://github.com/chroma-core/chroma), 2023.
Accessed: 2025-07-27.
- [46]
Inc. ClickHouse.
Clickhouse: Open-source column-oriented database management system.
[https://clickhouse.com](https://clickhouse.com), 2016.
Accessed: 2025-07-27.
- [47]
Thomas G. Dietterich.
Ensemble methods in machine learning.
In Multiple Classifier Systems: First International Workshop, MCS 2000, Cagliari, Italy, June 21–23, 2000, Proceedings, pages 1–15. Springer, 2000.
- [48]
Fabian Flöck, Kenan Erdogan, and Maribel Acosta.
Toktrack: A complete token provenance and change tracking dataset for the english wikipedia.
arXiv preprint arXiv:1703.08244, 2017.
Token count: approximately 13.545 billion.
- [49]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.
Scikit-learn: Machine learning in Python.
Journal of Machine Learning Research, 12:2825–2830, 2011.

## Appendix A Appendix

In this section, we explain in detail how the Confidence Gain is computed, alongside its key properties. Figure 1(a) illustrates the confidence curve for the [ResNet-101 & DINO-V2 ViT-B/14] model–embedding pair; however, the method applies to all model–embedding combinations.

We can observe that the distribution of points along the curve is not uniform: for large coverage values, there are almost no points, whereas for small coverage values, the points are densely packed. This is because a small coverage corresponds to more confident predictions, which occur more frequently. To handle the lack of points near full coverage (that is, near $coverage=1$), we linearly extrapolated the confidence curve using the 20 points with the highest value of $coverage$. Furthermore, we noticed that for small values (below values of 0.1), the confidence curve tends to behave chaotically. Therefore, to ensure robustness, we computed the Confidence Gain by integrating over the interval $coverage\in[0.1,1]$. Empirically, we observed that for $coverage>0.1$, the accuracy behaves monotonically with the coverage and is less sensitive to noise.

We define the Confidence Gain as the area between the confidence curve (including extrapolation when necessary), the vertical line $\text{coverage}=0.1$, and the horizontal line $\text{accuracy}=acc_{b}$. The $acc_{b}$ is the accuracy of the classification model when using 100% of the data, i.e., without any coverage-based filtering. This value is intrinsic to the classification model and the dataset and does not depend on the embedding model. As $\text{coverage}\to 1$, the accuracy along the confidence curve naturally approaches $acc_{b}$. We calculate the area using the Scikit-Learn library [49].

We define the Maximum Confidence Gain as the area bounded by the four lines:
$x=0.1$,
$x=1.0$,
$y=1.0$,
$y=acc_{b}$.

The value of the Confidence Gain is always less than the Maximum Confidence Gain. In other words, the Maximum Confidence Gain is a theoretical upper bound for the Confidence Gain. Therefore, it is logical to introduce the Normalized Confidence Gain as a ratio between these two values. The Normalized Confidence Gain would be determined in the range $[0,1]$, where the value $1$ corresponds to the perfect Confidence Gain, and the value 0 corresponds to no gain at all.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. How does this method compare to established OOD detection or uncertainty estimation methods (e.g., ODIN, Energy-based, Mahalanobis, Deep Ensembles)?

2. What is the conceptual or empirical advantage of the “Normalized Confidence Gain” metric over standard metrics such as AUROC or calibration error?

3. Why do results on ObjectNet show minimal improvement? Does this indicate the approach is only effective for in-distribution filtering?

4. What is the computational complexity of nearest-neighbor search at ImageNet scale? Is the method practical for large models or datasets?

5. If the approach is intended to generalize to NLP, can the authors provide a concrete prototype or preliminary experiment?
soundness: 2
strengths: 1. The paper tackles an important question — how the training data distribution affects model confidence and reliability.

2. The experimental setup is transparent and uses public datasets and pretrained models.

3. Results across CNNs (ResNet, ShuffleNet) and Vision Transformers (DeiT) provide a broad view of the observed phenomenon.
summary: This paper proposes a framework for estimating model prediction confidence based on the distribution of training data in the embedding space. By computing embeddings of training samples (using pre-trained models such as DINO or MobileNet), the authors filter out low-confidence predictions according to their distance to the training distribution. The method does not require model retraining and reportedly improves classification accuracy on ImageNet-V2 and ObjectNet.
weaknesses: 1. The method is essentially a nearest-neighbor–based confidence filtering strategy in embedding space. Similar concepts have been extensively explored in prior work on OOD detection and confidence estimation (e.g., Mahalanobis distance, energy-based OOD detection, influence functions, etc.). The contribution is incremental and not theoretically or algorithmically new.

2. The proposed “Normalized Confidence Gain” metric is a simple normalized area under an accuracy–coverage curve, but the paper does not provide any theoretical justification or comparison to standard uncertainty metrics such as AUROC, ECE, or NLL.

3. (a) The improvements reported are small and inconsistent, especially on the external test set (ObjectNet).
    (b) There is no comparison to strong baselines such as ODIN, Energy-based OOD, OpenOOD, or ensemble calibration methods.
    (c) The “embedding combination” experiment fails to improve results, which undermines one of the paper’s main claims.

4. The paper claims to address hallucination detection, but the actual experiments are standard image classification filtering tasks. The connection to hallucination phenomena in generative models is speculative and unsupported.

5. The paper does not explain why some embedding models yield better confidence estimation or analyze the effect of key parameters such as neighborhood size N. No computational cost or scalability discussion is included.

6. The paper is overly long relative to its conceptual simplicity, with repetitive discussion and speculative remarks (e.g., about NLP extension) without experimental grounding.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. Could you report results on ImageNet-V2 (adjusting/test) and ObjectNet against ODIN, Mahalanobis, and energy-based scoring, using common selective metrics (e.g., AURC/EAURC) in addition to your NCG? This would contextualise the gains beyond intra-framework comparisons.

2. You explicitly tune N on the adjusting split but do not detail a procedure or sensitivity study for L_threshold. Can you 
(a) specify how L_threshold is set,
(b) provide a 2D sensitivity plot over (N, L_threshold),
(c) state whether the same L_threshold generalizes across backbones/datasets?

3. Retrieval uses a fixed candidate pool (e.g., top-1000) prior to the N/L_threshold test. Please ablate the pool size (100/500/1000/2000) to show its effect on the coverage–accuracy curve and NCG. 

4. Beyond NCG and the baseline full-coverage accuracy (acc_b), can you provide accuracy at a few operational coverage levels (e.g., 30%, 50%, 70%) for the main models? This would make the trade-off immediately interpretable for deployment.

5. Results show DINO-V2 ViT-B/14 often achieves the highest NCG. Could you quantify how much of this comes from embedding capacity vs. pretraining data differences (e.g., V1 vs. V2)? A controlled comparison (matched dimension/resolution when possible) would clarify the driver of gains.
soundness: 2
strengths: 1. Model-agnostic confidence: Uses only proximity to the training distribution in embedding space—no reliance on the classifier’s own scores.

2. Systematic evaluations: Experiments cover multiple classifiers (ResNet-50/101, DeiT-Tiny/Small/Base, ShuffleNet-V2) and multiple embeddings (DINO-V1, DINO-V2 ViT-S/14, ViT-B/14, MobileNet-V2).

3. Consistent trend: Across most backbones, DINO-V2 ViT-B/14 gives the highest NCG; simple greedy combos of embeddings don’t beat the best single embedding.

4. Simple tuning: Only N is selected via NCG on the adjusting split and then reused.
summary: The paper studies selective prediction for image classification without modifying the trained classifiers. It builds a base embedding database from ImageNet-1K training images using pretrained embedding models (e.g., DINO-V1/V2 and MobileNet-V2). For a test image, the system retrieves nearest neighbors from the database and accepts the prediction of the classifier only if at least N neighbors meet the similarity threshold L_threshold; otherwise, it abstains. Adjusting acceptance criteria produces an Accuracy-Coverage (Confidence) curve. The paper introduces Confidence Gain and Normalized Confidence Gain. N is chosen on an ImageNet-V2 ''adjusting'' split to maximise NCG and then fixed for internal (ImageNet-V2 test split) and external (ObjectNet) evaluations. Overall, the method improves effective accuracy without retraining by leveraging proximity to the training distribution to guide abstention.
weaknesses: 1. Lack of external baselines. No comparisons with standard selective/OOD methods (ODIN, Mahalanobis, energy). Results are interpreted mainly via the paper’s own NCG and curves.

2. Threshold under-specification. The decision uses L_threshold, but only N is tuned on the ImageNet-V2 adjusting split; there’s no procedure or sensitivity for L_threshold.

3. Neighbour pool & scalability untested. Retrieval uses a fixed candidate pool (e.g., 1000) with no ablation on pool size vs. coverage–accuracy, and quantification of latency, index growth, or incremental updates; ANN backends (FAISS/ScaNN/HNSW) aren’t evaluated. This limits deployability claims.

4. No accuracy-at-coverage reporting. Tables provide NCG and baseline full-coverage accuracy (acc_b), but not accuracy at fixed coverage levels (e.g., 30%, 50%), which would aid operational decisions.

5. Scattered definitions. Formal details for NCG (integration over 0.1, 1, linear extrapolation near 1) are split across sections; consolidating formulas would help reproducibility.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: The results omit standard confidence and calibration metrics, such as Expected Calibration Error (ECE) [4] Brier Score, or Negative Log-Likelihood.  Without such comparisons, it remains unclear whether the approach truly enhances confidence estimation or calibration quality.
soundness: 3
strengths: The paper starts from the hypothesis that OOD detection in independent embedding spaces correlates well with a trained classifier’s confidence; its focus is the empirical evaluation of this hypothesis.

By using external, high-quality embedding models (e.g., DINO-v2), the method provides a robust confidence measure that is independent of the classifier’s internal biases and overconfidence.
summary: The paper leverages recent progress in high-quality embeddings, learned independently of the target training set (via self/unsupervised learning or transfer learning), in order to assess a trained model’s confidence without retraining. The main idea is to estimate data density in the embedding space: for a new image, if too few similar training images lie nearby, the prediction is flagged as low-confidence and skipped. This simple filter is shown to raise overall test accuracy by avoiding likely mistakes. The authors evaluate several vision models and embedding extractors, finding that stronger embeddings work best; combining multiple embeddings can help but doesn’t always beat the best single one. Overall, the approach is claimed to be model-agnostic and also useful for spotting likely out-of-distribution cases.
weaknesses: Main weakness: the underlying hypothesis is already well established in the OOD literature, so the novelty is limited.

There is a body of literature on classification with a reject option [5], which is essential when using confidence to gate predictions. Appropriate metrics exist for evaluating such procedures, but they are neither discussed nor employed in this paper.

Additional points:

1. Missing Complexity Analysis: The paper does not include any analysis of the method’s time or space complexity, leaving its computational scalability unclear.

2. No Comparison to Fine-Tuning: The framework is not quantitatively compared against a minimally fine-tuned model. This omits the essential trade-off between the method's computational overhead and the performance gain achieved by a common alternative.

3. No Comparison to Established OOD Detection or Confidence Calibration Methods. The method effectively operates as a density-based OOD detector, yet it is not compared to widely accepted OOD baselines such as: (i) ODIN [1] which enhances the Maximum Softmax Probability baseline via temperature scaling and small input perturbations. (ii) Mahalanobis Distance-based OOD Detection [2] which models class-conditional feature distributions and measures Mahalanobis distance for OOD detection. (iii) Energy-based OOD Detection [3] which uses the energy score of the network’s logit outputs to distinguish in- and out-of-distribution samples.

4. Ambiguous Parameter Definitions: The critical distance threshold L-threshold, a central component in determining whether a sample lies within or outside the prior distribution, is not explicitly defined or accompanied by a quantitative rule for selection. The absence of a formula, heuristic, or sensitivity analysis limits reproducibility and interpretability.

5. Unclear Algorithmic Presentation: The main algorithm is described in a verbose and fragmented manner across multiple sections, without a concise pseudocode or structured workflow. This presentation makes it difficult to follow or reimplement the procedure. Moreover, the workflow diagram (Figure 1) adds visual complexity rather than clarity, as it does not clearly map to a step-by-step algorithmic sequence.

References
1. Liang, S., Li, Y., & Srikant, R. (2018). Enhancing the Reliability of Out-of-Distribution Image Detection in Neural Networks (ODIN). ICLR 2018.
2. Lee, K., Lee, H., Lee, K., & Shin, J. (2018). A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. NeurIPS 2018.
3. Liu, W., Wang, X., Owens, J., & Liang, Y. (2020). Energy-based Out-of-Distribution Detection. NeurIPS 2020.
4. Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. ICML 2017.
5. Geifman, Y., and El-Yaniv R. (2019). Selectivenet: A deep neural network with an integrated reject option. ICML 2019.

---

confidence: 4
contribution: 1
initial_rating: 0
presentation: 2
questions: The authors really need to do more comparisons to alternative techniques, especially since this domain has been so widely studied.
They also need to do something to demonstrate that the computer vision to NLP gap can be bridged by this technique
soundness: 1
strengths: - Important problem in real-world AI 
- Use of both supervised and self-supervised embedding models
summary: ### Problem: 
- Hallucinations in AI are widespread problem
- Studying hallucinations from a data-centric perspective seems promising, especially by leveraging latent space representations of training data. This does not require modifying or retraining models
- The term ‘hallucination’ is typically used in the context of generative language models. However, constructing datasets big enough for NLP is challenging. P

## Solution:
- The authors propose to use computer vision models as a test-bed to understand the causes of hallucinations 
- The paper introduces a model-agnostic framework that estimates confidence using the distance of new samples from the training data in a learned embedding space. 
- Their framework has the following steps:
    - Embedding distance: The authors compute embeddings of the training dataset using self-supervised and supervised vision models (DINO-V2 and MobileNet-V2). For each test image, they measure its distance to its nearest neighbors in this embedding space.
    - Confidence filtering: Predictions are accepted only if the test image lies close to a certain number of examples from training data. A threshold is introduced.
    - Multiple embedding models: The authors also combine multiple embedding models to capture different aspects of the training distribution.
weaknesses: - The paper discusses LLM hallucinations, but then studies computer vision. This is a big leap. It’s not clear that this idea could be as easily applied to LLMs. 
    - One issue is because of information leakage. How do I know that my test set wasn’t actually used in the LLM’s training data? This whole idea breaks down
- Severe lack of novelty. Embedding-based distance has been studied many times over. See below
    - Weak experiments: no comparison to existing methods
    - The paper mentions several related works (ODIN, Mahalabis OOD, Information theoretic OOD). Why do they not compare?
    - Other baselines: 
        - Jiang et al. 2018. To Trust Or Not To Trust A Classifier
        - Papernot et al. 2018. Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
        - In NLP: Khandelwal et al. ICLR’2020
- Hyperparameters $L_threshold$ and $N$. No hyperparameter study. Difficult to choose these in practice.