# Uncertainty Flow Facilitates Zero-Shot Multi-Label Learning in Affective Facial Analysis
## 3 Uncertainty Flow Framework
### IV. Prediction Related Uncertainty Indexes  
With the flowed parameter posterior in the target task, i.e., \((\theta_{t}|x)\), it is feasible to form the predictive posterior distribution for each upcoming novel observation, i.e., \(x_{new}\), where,  
\[p(x_{new}|x)=\int_{\theta^{*}}p(x_{new}|\theta^{*})p(\theta|x)d\theta \tag{16}\]  
Armed with this predictive posterior distribution, it allows the production of prediction related uncertainty indexes. As lengthy discussion in previous literature [20; 32], one overwhelming claim insists that the probabilistic outputs can be produced by the soft-max function in (It is often placed in the final layer of neural networks to allow the real-valued prediction to be 'pushed' in presenting the pseudo-probabilisitic output.) permitting the averaging over the repetitive forward propagations of new observation in either Bayesian neural network or non-Bayesian neural networks. This type of probabilistic output merely tells the most probable output given the input, not how uncertain is the prediction. For the comparative purpose, we refer this type of uncertainty index as **soft-max uncertainty**. The quantification of this **soft-max uncertainty** has been previously approximated via averaged \(T\) times of forward model(input) propagation [33], expressed in Equations (17) and (18):  
\[Soft-Max =\mathbb{E}(y*)\approx\frac{1}{T}\sum_{t=1}^{T}(y*(x_{new}))|_{p( \theta|x_{new})} \tag{17}\] \[=Class-Type|_{Non-Bayesian} \tag{18}\]  
From Bayesian learning perspective, the above-mentioned **softmax uncertainty** reflects the belief of applying predictive mean in indexing the prediction uncertainty. Numerically, this type ofuncertainty index captures the mere classification type in multiple-object discrimination, is equivalent with the class type probability in non-Bayesian neural networks. However, as the predictive mean does not capture the full picture of parameter posterior distribution, we draw our attention towards the predictive variance instead.  
We argue that the yielded predictive variance reflects the degree of uncertainty that is associated with each prediction. As a result, based on the approximated weight posterior, i.e., \(p(\theta|x)\), a better prediction related uncertainty index is expressed below in Equation (19):  
\[Pure=Var[y*]\approx Var\big{[}\sum_{t=1}^{T}\int_{\Theta}p(x_{new})p(\theta|x)d\theta \big{]} \tag{19}\]  
We denote this measure of prediction uncertainty index as **pure uncertainty**.  
One step further, rather than the dichotomous uncertainty indexes, e.g., **pure uncertainty** and **soft-max uncertainty**, these two indexes can be fused together, which allows the uncertainty index to reflect both class-type probabilistic prediction and the model uncertainty associated with each prediction. In consistent with the previous naming tradition, this type of uncertainty index is marked as **uncertainty plus**. The production of this **uncertainty plus** shows that each class type probabilistic prediction should be proportionately adjusted according to its associated prediction uncertainty, expressed in Equation (20):  
\[Uncert+\approx\frac{\frac{1}{T}\sum_{t=1}^{T}(y*(x_{new}))|_{p(\theta|x_{new})} }{Var\big{[}\sum_{t=1}^{T}\int_{\Theta}p(x_{new})p(\theta|x)d\theta\big{]}} \tag{20}\]  
For illustrative purposes, how each of three above mentioned uncertainty indexes, e.g., **soft-max uncertainty**, **pure uncertainty**, and **uncertainty plus**, influences on a simple binary classifier, is demonstrated in Figure 4.