# Improving Re-Identification by Estimating and Utilizing Diverse Uncertainty Types for Embeddings
## 1 Introduction  
The development of self-driving cars [1, 2] and robots for collaborative assembly [3, 4] represents a significant advancement in autonomous systems. However, these technologies rely on intelligent components to perceive their environment, especially for perception of humans and interaction with them. In the current state of the art, the majority of perception systems are based on neural networks, which, like many machine learning methods, compute outputs for any input, regardless of whether they can make safe decisions or not. Therefore, estimating the uncertainty in the decision-making process is crucial for the development of a reliable autonomous system that can assist humans without causing harm. As a result, uncertainty quantification or estimation for neural networks has emerged as a growing research trend in recent years. Starting from basic work on uncertainty quantification [5, 6], work on uncertainty estimation for neural networks has been mainly focused on classification and regression tasks [7, 8, 9, 10]. This includes perception tasks such as object detection [11, 12, 13, 14], semantic and medical image segmentation [15, 16, 17, 18], as well as action recognition [19], in which actions and objects are classified, and localization is achieved through bounding box regression. The determined uncertainty in these cases relates to the final classification or regression layer. However, there is currently a trend to use neural networks as encoders to describe images or language as feature vectors, which is known as learning a feature embedding. The estimation of uncertainties for embedding vectors is much less explored [8]. To quantify uncertainties for feature vectors, most state-of-the-art approaches use probabilistic embeddings [20; 21], which are able to estimate data uncertainty. Only few approaches [22; 23; 24; 25] estimate model uncertainty for feature vectors by using a Bayesian module on top of the deterministic backbone. These are the two most relevant types of uncertainty in literature [26; 27]. However, for autonomous agents it is also important to recognize when the data are out-of-distribution, which means that the current input is very dissimilar from data seen during training. Then, it is likely that the agent cannot make safe decisions even if model uncertainty and data uncertainty are low. If such situations are relevant, according to literature, uncertainty can be categorized into three types of uncertainty [7]: (1) model uncertainty (also known as epistemic uncertainty), (2) data uncertainty (also known as aleatoric uncertainty), and (3) distributional uncertainty. Neither probabilistic embeddings nor Bayesian modules are able to quantify distributional uncertainty. Indeed, to the best of our knowledge, the estimation of distributional uncertainty for feature vectors has not been explored at all. We show that it is beneficial to estimate all three types of uncertainty and how each of the uncertainty types has its individual benefits for predicting different types of error sources for the feature vector. Notably, this information can be used to improve the embedding during training and inference. In particular, we demonstrate how a component-wise _model uncertainty_ estimate can be leveraged to enhance the feature vector during inference. Furthermore, we show that during training estimated _data uncertainty_ can be employed to scale the embedding vector elements in accordance with the degree of uncertainty, as proposed by [PERSON] et al. [25]. Additionally, we illustrate how estimated _distributional uncertainty_ can be utilized to determine the extent to which an input is out-of-distribution. While our approach can be used in general to estimate and utilize the three types of uncertainty, for the purposes of benchmarking, we decided in favor of person re-identification as the application. Person re-identification represents an exemplary human perception task that is relevant for autonomous agents that require the ability to track individuals or interact with humans.