# On the Optimum Linear Soft Fusion of Classifiers
## 5 Real Data Experiments
corresponding to the Nearest Mean, which is the minimum \(MSE\) among all the 5 classifiers, as deduced from Table 2. This probably means facing an additional problem to achieve a good fit of the NEOC model to the actual model, because in the NEOC model, the bias is assumed to be the same for all classifiers, and then both \(\vartheta_{D}\) and \(MSE_{D}\) correspond to the same classifier. In any case, notice that \(I_{/NEOC}\) is still far from 5, the maximum achievable value if all the classifiers are unbiased. This is evidence of the significant limitation imposed by bias in the fusion of classifiers. Another limitation to achieving maximum improvement in the NEOC model was the unbalance factor \(U=\frac{D-C}{D-1}\). In our case, \(U\) has a value between 0.5 and 0.6 for all values of \(D\), which is between the perfectly balanced case (\(U=0\)) and the totally unbalanced case (\(U=1\)). Finally, we show in Figure 7 the variation in estimated accuracy with the number of classifiers for the correlated (blue solid) and the uncorrelated (red solid) cases. Similar to \(I_{/r}\), the accuracy slowly increases with \(D\) in the correlated case. And, similar to \(I_{/r}\), the increase is more significant in the uncorrelated case. We also show in Figure 7 (blue dot and red dot) the theoretical accuracies predicted by Equation (30), assuming \(P_{0}=P_{1}=0.5\), namely:  
\[ \begin{split}& A_{ft}=1-P_{eft}=1-0.5\cdot\left(P_{eDI}\sqrt{ \
icefrac{{\gamma_{f1}}}{{\gamma_{i1}}}}+P_{eDI1}\sqrt{\
icefrac{{\gamma_{f1 }}}{{\gamma_{i1}}}}\right)\\ & A_{ft}=1-P_{efut}=1-0.5\cdot\left(P_{eDI1}\sqrt{\
icefrac{{ \gamma_{f1}}}{{\gamma_{i1}}}}+P_{eDI1}\sqrt{\
icefrac{{\gamma_{f1}}}{{\gamma_ {i1}}}}\right)\end{split}  \tag{32}\]  
where \(P_{eDI}\) and \(P_{eDI1}\) are the respective actual probabilities of errors type I and type II of the best classifiers in the correlated case. And similarly, with respect to \(P_{eDI1}\) and \(P_{eDI1}\). Figure 7: Variation in the accuracy with the number, \(D\), of classifiers. \(A_{f}\), estimated accuracy with correlated scores. \(A_{ft}\), accuracy predicted by Equation (32). \(A_{f\mu}\), accuracy with uncorrelated scores. \(A_{ft}\), accuracy predicted by Equation (32). \begin{table}
\begin{tabular}{l l l l l l l} \hline  & \(\hat{b}_{d/0}\) & \(\hat{b}_{d/1}\) & \(\hat{\upsilon}_{d/0}\) & \(\hat{\upsilon}_{d/1}\) & \(M\hat{S}E_{d}\) & \(\hat{A}cc_{d}\) \\ \hline Nearest-mean & 0.1034 & 0.2656 & 0.0533 & 0.1146 & 0.1246 & 82.93\% \\ Linear & 0.1180 & 0.3905 & 0.0223 & 0.0638 & 0.1263 & 82.80\% \\ Bayes & 0.0680 & 0.2859 & 0.0476 & 0.1672 & 0.1506 & 82.65\% \\ Naive Bayes & 0.0649 & 0.2952 & 0.0458 & 0.1788 & 0.1580 & 81.72\% \\ Logistic & 0.0398 & 0.3525 & 0.0370 & 0.2236 & 0.1932 & 80.34\% \\  & \(\hat{b}_{f/0}\) & \(\hat{b}_{f/1}\) & \(\hat{\upsilon}_{f/0}\) & \(\hat{\upsilon}_{f/1}\) & \(M\hat{S}E_{f}\) & \(\hat{A}cc_{f}\) \\ FUSION & 0.0845 & 0.3081 & 0.0110 & 0.0449 & **0.0790** & **89.60\%** \\ \hline \end{tabular}
\end{table} 
Table 3: Estimates of the score statistics and accuracy of the classifiers. Uncorrelated case. We can see the good fitting between the theoretical prediction and the actually computed accuracies. This suggests that the exponential model assumed to deduce (30) is appropriate for the fused scores and for the scores of the best classifier. This can be corroborated if we look at Figure 8, which shows histograms normalized to represent the non-parametric estimates of pdfs. We show in Figure 8a the histograms corresponding to the pdf estimates of the best classifiers \(\hat{p}(s_{D}/c_{0})\) (left) and \(\hat{p}(s_{D}/c_{1})\) (right). Furthermore, we show in Figure 8b the histograms corresponding to the pdf estimates of the fused scores \(\hat{p}\Big{(}s_{f}/c_{0}\Big{)}\) (left) and \(\hat{p}\Big{(}s_{f}/c_{1}\Big{)}\) (right). The exponential approximations are superimposed. Every exponential approximation was made from the ML estimates of the corresponding parameter, i.e., \(\hat{\lambda}=\frac{1}{\hat{E}[s]}\), where \(\hat{E}[s]\) is the sample mean of the scores.