# Bayesian Feature Fusion Using Factor Graph in Reduced Normal Form
## 2 Materials and Methods
### Feature Fusion Model  
The proposed feature fusion architecture is based on the Naive Bayes model where \(N\) observed categorical variables \(\{X_{1},X_{2},...,X_{N}\}\) are connected to a single class variable \(C\). Each observed variable represents the output of a sensor, detector or, more in general, an information source that need to be fused together with the other ones. It assumes values in a discrete alphabet: \(\mathcal{X}_{i}=\{\xi_{1}^{i},\xi_{2}^{i},...,\xi_{L_{i}}^{i}\}\); where the dimension \(L_{i}\) is the number of valuesthat each variable can assume, if discrete, or the number of levels we use to quantize it (and which is therefore generally different for each variable). For the continuous variables several quantization schemes may be used, but here we propose a simple approach: the values assumed by each \(X_{i}\) are clipped in \([m_{X_{i}},M_{X_{i}}]\), where \(m_{X_{i}}\) and \(M_{X_{i}}\) are, respectively, the minimum and maximum permitted value for variable \(X_{i}\). The range is then divided uniformly using \(L_{i}\) levels, so that the generic continuous value \(v\) is associated with level \(l\) if \((l-1)<v\leq l\). It should be noted that in an Internet of Things (IoT) context, the number of levels used to quantize the sensor measure may be an important design parameter. In fact, generally, the trade-off between accuracy and available hardware resources need to be evaluated, for every specific application, also in terms of overall system energy consumption [15]. Finally, in the training phase, each variable \(X_{i}\) is represented through a discrete distribution obtained using a smooth one-hot encoding. More specifically, for representing the \(k\)-th value of \(X_{i}\) (\(\xi_{k}^{i}\)), instead of use a sharp distribution \(\mathbf{\delta}_{k}^{i}\) (an \(L_{i}\) size vector representing the Kronecker delta, i.e., with all zero and only a one at the \(k\)-th position), we use a smoother distribution:  
\[\mathbf{\delta}_{k}^{i}=\overbrace{\left[\frac{\epsilon}{(L_{i}-1)},\ldots,1- \epsilon,\ldots,\frac{\epsilon}{(L_{i}-1)}\right]}^{L_{i}}\]  
where \(\epsilon\) is a small positive number. Since the values assumed by the detections are always positive, we set the minimum value \(m_{X_{i}}=\frac{M_{X_{i}}}{L_{i}}\) in order to \"use\" all \(L_{i}\) levels. Differently, with \(m_{X_{i}}=0\), the first quantization level will be underused since there are no negative values. Furthermore, we assume that all observed variables are connected to one class variable \(C\) that assumes values in the discrete alphabet \(\mathcal{C}=\{\gamma_{1},\gamma_{2},...,\gamma_{L_{c}}\}\). The relationship between each observed and class variable, is formalized by a Conditional Probability Table (CPT): \(P(X_{i}|C)=[Pr\{X_{i}=x_{i}|C=\epsilon\}]_{c\in\mathcal{C}}^{x_{i}\in X_{i}}\). This model is the classical Naive Bayes, shown in Figure 2 together with its FGRn representation, which represents the joint probability distribution:  
\[p_{X_{1},X_{2},...,X_{N},C}(x_{1},x_{2},\ldots,x_{N},c)=\pi_{C}(c)\prod_{i=1} ^{N}p_{X_{i}|C}(x_{i}|c) \tag{1}\]  
where \(\pi_{C}\) is the prior on \(C\). Please note that in the FGRn formulation the CPTs in Figure 2b are represented as Single Input-Single Output (SISO) blocks, making the model more flexible [13; 27; 28] with respect to other factor graph representation [29]. Learning each CPT is performed locally through backward and forward messages using the optimized Maximum Likelihood algorithm as  
Figure 2: The Naive Bayes model as a Bayesian graph (**a**) and as a Factor Graph in Reduced Normal Form (**b**). described in [30]. The usage of FGrn provides us with a formal probabilistic framework for learning and allows easy handling of classification, error correction, missing values, etc.