Title: Prior Distribution and Model Confidence
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Prior distribution, Model confidence, Confidence estimation, Out-of-distribution detection, Embedding-based confidence

Abstract: This paper investigates the impact of training data distribution on the performance of image classification models. By
analyzing the embeddings of the training set, we propose a framework to understand the confidence of model predictions
on unseen data without the need for retraining. Our approach filters out low-confidence predictions based on their distance
from the training distribution in the embedding space, significantly improving classification accuracy. We demonstrate this
on the example of several classification models, showing consistent performance gains across architectures. Furthermore,
we show that using multiple embedding models to represent the training data enables a more robust estimation of confidence, as different embeddings capture complementary aspects of the data. Combining these embeddings allows for better detection and exclusion of out-of-distribution samples, resulting in further accuracy improvements. The proposed method is model-agnostic and generalizable, with potential applications beyond computer vision, including domains such as Natural Language Processing where prediction reliability is critical.

TL;DR: We show that leveraging prior distribution in embedding space improves model confidence estimation, enhancing accuracy–coverage tradeoffs without retraining.