# On the Optimum Linear Soft Fusion of Classifiers
## 4 From Mean Square Error to Probability of Error  
Given a score, \(s\), we have to decide between \(c_{0}\) and \(c_{1}\). Interpreting \(s\) as the posterior probability of \(c_{1}\), we implement the MAP test:  
\[\begin{split}\begin{subarray}{c}\overset{c_{1}}{>}& 1\\ \overset{s}{<}& 2\end{subarray}.\end{split} \tag{23}\]  
Then, the performance of the classifier can be evaluated by the probability of error, \(P_{\epsilon}\). Obviously, the lower the \(MSE\), the lower \(P_{\epsilon}\) should be. However, this is not demonstrable in a general case, as both the \(MSE\) and \(P_{\epsilon}\) are quite different integrals of the class-conditional probability density functions (pdfs), namely:  
\[\begin{split}& MSE=E\Big{[}(u-s)^{2}\Big{]}=P_{0}\cdot E\Big{[} (u-s)^{2}/c_{0}\Big{]}+P_{1}\cdot E\Big{[}(u-s)^{2}/c_{1}\Big{]}=\\ &=P_{0}\int_{0}^{1}(0-s)^{2}p(s/c_{0})ds+P_{1}\int_{0}^{1}(1-s)^{ 2}p(s/c_{1})ds\\ & P_{\epsilon}=P_{0}\cdot\int_{\dfrac{1}{2}}^{1}p(s/c_{0})ds+P_{1 }\cdot\int_{0}^{\dfrac{2}{2}}p(s/c_{1})ds\end{split}. \tag{24}\]  
Figure 3: (**a**) Improvement factor as a function of unbalance; bias is assumed to be 0; (**b**) improvement factor as a function of bias; unbalance is assumed to be 0.5.  
For this reason, we now assume a simple but representative model of the conditional pdfs. Let us consider that \(p(s/c_{1})\) and \(p(s/c_{0})\) are exponential pdfs in the interval \((0,1)\), i.e.,  
\[p(s/c_{0})=\lambda_{0}e^{-\lambda_{0}s}u(s)\quad p(s/c_{1})=\lambda_{1}e^{- \lambda_{1}(1-s)}u(1-s). \tag{25}\]  
where \(u(s)\) is the unit step function. In (25), we assumed that \(\lambda_{0}\) and \(\lambda_{1}\) are large enough so that \(\lambda_{0}e^{-\lambda_{0}s}\simeq 0\,s>1\quad\lambda_{1}e^{-\lambda_{1}(1-s)} \simeq 0\,s<0\), and so the exponentials are appropriate pdfs for scores. We show in Figure 4 two exponential class-conditional pdfs for \(\lambda_{0}=\lambda_{1}=4\).  
We can express \(P_{\varepsilon}\) is (24) as the combination of the probability of false-positives, \(P_{\varepsilon I}\) (type I error), and the probability of false negatives, \(P_{\varepsilon II}\) (type II error), and considering (23) and the exponential models (25), we have:  
But, in general, we need to define a separate improvement factor for every type of error:  
\(I_{f1}=\frac{MSE_{DI}}{MSE_{f1}}\), \(I_{fII}=\frac{MSE_{DII}}{MSE_{fII}}\), so that  
\[P_{ef}=P_{0}\cdot P_{tDI}\sqrt{I_{f1}}+P_{1}\cdot P_{tDI}\sqrt{I_{fII}}. \tag{30}\]  
Let us show some illustrative examples for the above. We consider, for simplicity, the case \(\lambda_{0}=\lambda_{1}\). Let us define the theoretical accuracy \(A=1-P_{t}\). Using (29), we represent in Figure 5a the post-fusion accuracy, \(A_{f}\), as a function of the accuracy, \(A_{D}\), of the best classifier for different values of \(I_{f}\). The significant increase in accuracy with the increase in \(I_{f}\) is notable. Thus, for example, a value of \(A_{D}=0.70\) increases to \(A_{f}=0.87\) for \(I_{f}=3\). From the previous analysis, we know that 3 uncorrelated and unbiased classifiers would be enough to provide such an improvement factor. Furthermore, we represent in Figure 5b the accuracy, \(A_{f}\), as a function of \(I_{f}\) for different values of \(A_{D}\). Once again, the significant accuracy improvement that can be achieved with improvement factors that are not excessively high is notable. This is especially true for weak classifiers that provide relatively low accuracy.