# Improving Re-Identification by Estimating and Utilizing Diverse Uncertainty Types for Embeddings
## 4 Experiments
### Experimental Validation of the Distributional Uncertainty Estimate
#### 4.4.2 Analysis of Uncertainty Score Distributions  
As mentioned above, the OOD sets (D1)-(D4) presented in the previous section represent classes of progressively greater dissimilarity to the training data. This means that the OOD degree, i.e., the amount to which an image deviates from the training distribution, increases from (D1) to (D4). If our distributional uncertainty estimate \(\Sigma^{(V)}\) is indeed a good estimate of distributional uncertainty, it should measure the OOD degree. Thus we expect to observe a gradual increase of the distributional uncertainty score from the distractor sets (D1)-(D4). Since the sets are not perfectly separable, an overlap between the distributions is to be expected. However, the distributions should be somewhat separable. At the very least, the means of the uncertainty score distributions for the sets (D1) to (D4) should appear in this order. Furthermore, we expect the distributional uncertainty estimate to outperform the other uncertainty estimates regarding the ability to separate the distributions. Experimental Setup  
As was laid out in detail in Section 3.3, the data uncertainty estimate \(\Sigma^{(D)}\) is the diagonal of the covariance matrix of the probabilistic embedding, while the model and distributional uncertainty estimates \(\Sigma^{(M)}\) and \(\Sigma^{(V)}\) are the variance induced by the Bayesian module in the raw mean vector \(\widehat{\mu}\) and data uncertainty estimate \(\Sigma^{(D)}\). All three uncertainty estimates are vectors, so it is not immediately clear how to calculate a scalar uncertainty score from them. As described in Section 3.3.6, we have decided in favor of entropy for this purpose. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Set** & & **\# of Images** \\ \hline Training Data & (T) & 12,936 \\ Query & (Q) & 3368 \\ Full Human & (D1) & 278 \\ Human Fragments & (D2) & 1729 \\ Objects & (D3) & 567 \\ Background/Blur & (D4) & 224 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Number of images in the training (T) and query (Q) set, as well as in the annotated subsets (D1)–(D4) of the distractors in Market-1501. After training our model on Market-1501, we generate the three uncertainty estimates for all relevant images regarding this experiment and compute the scalar uncertainty scores according to Equation (17) for each of them. Then, we employ kernel density estimation (KDE) to model the probability density functions corresponding to the observed values for each of the considered sets, i.e., the query set (Q) as the in-distribution set and the OOD sets (D1)-(D4). The resulting KDE plots are shown in Figure 8a-c.  
Model Uncertainty Score Distribution  
In the mathematical model (Equation (2)), the model uncertainty term \(p(\theta\,|\,\mathcal{D})\) is both separate and independent of the distributional uncertainty term \(p(\xi\,|\,x,\theta)\). Therefore, we would expect the model uncertainty score distributions for the different sets to be indifferent to the OOD degree, i.e., we should see no ordered separation of the sets. Figure 8: Distribution of uncertainty scores over all images in certain subsets of Market-1501 for the three uncertainty types (**a**) model uncertainty, (**b**) data uncertainty, and (**c**) distributional uncertainty. The five subsets (Q), (D1)–(D4) are described in Section 4.4.1 and examples are provided in Figure 7. (**d**) Data uncertainty in case of low distributional uncertainty. (**e**) Distributional uncertainty in case of low data uncertainty. (**f**) Data uncertainty in case of high distributional uncertainty. On top of each of the probability density plots, the means and standard deviations of each distribution are plotted in relation to the query set (Q). Figure 8a shows the mean of the model uncertainty scores for all sets on top of the distributions. Table 7a lists how far away the mean of the uncertainty scores of the distractor sets (D1)-(D4) are from the in-distribution set (Q) in terms of the standard deviation of the in-distribution set.