# Bayesian Feature Fusion Using Factor Graph in Reduced Normal Form
## 2 Materials and Methods
### Feature Fusion Model
In every single inference phase, when all observed variables are instantiated, the backward messages \(\mathbf{b}_{X_{i}}=\boldsymbol{\delta}_{k_{i}}^{i}\) are injected into the network, where \(k_{i}\) is the index position of the instantiated value \(\overline{x}_{i}:=\bar{\xi}_{k_{i}}^{i}\) for the variable \(X_{i}\). In functional notation the backward message would be \(b_{X_{i}}(x_{i})=\delta(x_{i}-\overline{x}_{i})|_{\overline{x}_{i}\in\mathcal{ X}_{i}}\), with \(i=1,\ldots,N\). The class label is not observed and its forward message, \(\mathbf{f}_{\mathcal{C}}\), is set to a uniform distribution over class alphabet \(\mathcal{C}\). After message propagation, the product of the backward and the forward messages at the class label \((b_{\mathcal{C}}(c)f_{\mathcal{C}}(c))\) is proportional to the posterior probability of the class given all the other instantiated observed variables, i.e. : \(b_{\mathcal{C}}(c)f_{\mathcal{C}}(c)=p_{X_{1},\ldots,X_{N},\mathcal{C}}( \overline{x}_{1},\ldots,\overline{x}_{N},c)\propto p_{\mathcal{C}|X_{1}, \ldots,X_{N}}(c|\overline{x}_{1},\ldots,\overline{x}_{N})\). Suppose that all observed variables except one (e.g., \(X_{1}\)) are instantiated and that the class variable is unknown. Once we injected the messages in the network properly, after the message propagation we obtain:  
\[ \begin{array}{rcl}f_{X_{1}}(x_{1})&=&\sum_{c}p_{X_{1}|\mathcal{C}}(x_{1}|c)p _{X_{2}|\mathcal{C}}(\overline{x}_{2}|c)\ldots p_{X_{N}|\mathcal{C}}( \overline{x}_{N}|c)\pi_{\mathcal{C}}(c)\\ &\propto&p_{X_{1}|X_{2},\ldots,X_{N}}(x_{1}|\overline{x}_{2},\ldots,\overline{ x}_{N})\end{array}  \tag{2}\]  
in other words, the forward distribution of the non-instantiated variable is proportional to its posterior probability given all the other instantiated observed variables. If also the class label is instantiated, the forward distribution of the non-instantiated variable (e.g., \(X_{1}\)), is proportional to its posterior probability given the class variable, i.e. : \(f_{X_{1}}(x_{1})\propto p_{X_{1}|\mathcal{C}}(x_{1}|\overline{c})\). This is coherent with the Naive Bayes model where each observed variable is conditionally independent from other variables given the class label. The forward messages that we can collect at the observed variables represent the most probable configuration given the evidence and the learned model. Injected messages consistent with the forward values are considered plausible, while when this accordance is low an error, or a strange behavior, may have occurred. We can also try to condition the behavior of the system based on the reliability (estimated or assumed) of each detector. If we have low confidence on a particular observed variable, \(X_{e}\), we can try to reduce its contribution raising the message \(\mathbf{b}_{\mathcal{C}^{(i)}}\) to an exponent \(0<\
u<1\) and normalizing the resulting message. The effect of this operation is to make \(\left(\mathbf{b}_{\mathcal{C}^{(i)}}\right)^{\
u}\) more and more uniform with \(\
u\to 0\), a sort of smoothing for the message. A uniform message does not make any contribution in the element-by-element product performed in the replicator block and successive normalization of the resulting message. All the other messages \(\mathbf{b}_{\mathcal{C}^{(i)}}\), with \(i\in\{1,\ldots,N\}\setminus\{e\}\), can remain raised to \(1\) (no effect), or can be slightly augmented (raising to an exponent \(\
u>1\)) to weight more their contribution since the distribution thickens around the most probable value, a sort of sharpening for the message.