## 4 Experimental Methodology
### Performance Evaluation of WvEnSL against Ensemble Self-Labeled Algorithms
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{2}{c}{**Ratio = 10\%**} & \multicolumn{2}{c}{**Ratio = 20\%**} & \multicolumn{2}{c}{**Ratio = 30\%**} & \multicolumn{2}{c}{**Ratio = 40\%**} \\ \cline{2-9}  & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) \\ \hline CST-Voting (SMO) & 83.08\% & 75.32\% & 83.26\% & 75.64\% & 83.39\% & 75.80\% & 83.39\% & 75.80\% \\ WvEnSL\({}_{1}\) (SMO) & **83.39\%** & **75.80\%** & **83.48\%** & **75.96\%** & **83.76\%** & **76.44\%** & **83.85\%** & **76.60\%** \\ \hline CST-Voting (C4.5) & 85.52\% & 79.97\% & 85.85\% & 80.45\% & 86.68\% & 81.73\% & 86.58\% & 81.57\% \\ WvEnSL\({}_{1}\) (C4.5) & **85.65\%** & **80.13\%** & **86.08\%** & **80.77\%** & **86.78\%** & **81.89\%** & **86.92\%** & **82.05\%** \\ \hline CST-Voting (\(k\)NN) & 82.91\% & 75.48\% & 83.09\% & 75.80\% & 83.15\% & 75.96\% & 83.73\% & 76.76\% \\ WvEnSL\({}_{1}\) (\(k\)NN) & **83.63\%** & **76.60\%** & **83.73\%** & **76.76\%** & **83.95\%** & **77.08\%** & **84.23\%** & **77.56\%** \\ \hline DTCo & 86.79\% & 81.41\% & 87.21\% & 82.05\% & 87.21\% & 82.05\% & 87.74\% & 82.85\% \\ WvEnSL\({}_{2}\) & **87.12\%** & **81.89\%** & **87.44\%** & **82.37\%** & **87.54\%** & **82.53\%** & **87.97\%** & **83.17\%** \\ \hline EnSL & 87.19\% & 82.05\% & 86.92\% & 81.57\% & 87.34\% & 82.21\% & 87.61\% & 82.69\% \\ WvEnSL\({}_{3}\) & **87.51\%** & **82.53\%** & **87.70\%** & **82.69\%** & **88.23\%** & **83.49\%** & **88.17\%** & **83.49\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance evaluation of WvEnSL against ensemble self-labeled algorithms for Pneumonia dataset. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{2}{c}{**Ratio = 10\%**} & \multicolumn{2}{c}{**Ratio = 20\%**} & \multicolumn{2}{c}{**Ratio = 30\%**} & \multicolumn{2}{c}{**Ratio = 40\%**} \\ \cline{2-9}  & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) & \(\mathbf{F_{1}}\) & \(\mathbf{Acc}\) \\ \hline CST-Voting (SMO) & 69.27\% & 69.43\% & 68.65\% & 68.37\% & 69.50\% & 69.61\% & 70.42\% & 70.32\% \\ WvEnSL\({}_{1}\) (SMO) & **69.73\%** & **69.79\%** & **69.73\%** & **69.79\%** & **70.32\%** & **70.32\%** & **71.00\%** & **70.85\%** \\ \hline CST-Voting (C4.5) & 66.67\% & 67.31\% & 68.19\% & 68.02\% & 67.51\% & 68.20\% & 69.52\% & 69.79\% \\ WvEnSL\({}_{1}\) (C4.5) & **67.86\%** & **68.20\%** & **69.26\%** & **69.26\%** & **69.63\%** & **69.79\%** & **69.98\%** & **70.14\%** \\ \hline CST-Voting (\(k\)NN) & 65.71\% & 66.08\% & 66.43\% & 66.96\% & 68.21\% & 68.55\% & 68.93\% & 69.26\% \\ WvEnSL\({}_{1}\) (\(k\)NN) & **65.83\%** & **66.25\%** & **67.14\%** & **67. Let \(r_{i}^{j}\) be the rank of the \(j\)-th of \(k\) learning algorithms on the \(i\)-th of \(M\) problems. Under the null-hypothesis \(H_{0}\), which states that all the algorithms are equivalent, the Friedman aligned ranks test statistic is defined by:  
\[F_{AR}=\frac{(k-1)\left[\sum_{j=1}^{k}\hat{R}_{j}^{2}-(kM^{2}/4)(kM+1)^{2} \right]}{\frac{kM(kM+1)(2 kM+1)}{6}-\frac{1}{k}\sum_{i=1}^{M}\hat{R}_{i}^{2}}\]  
where \(\hat{R}_{i}\) is equal to the rank total of the \(i\)-th dataset and \(\hat{R}_{j}\) is the rank total of the \(j\)-th algorithm. The test statistic \(F_{AR}\) is compared with the \(\chi^{2}\) distribution with \((k-1)\) degrees of freedom. It is worth noticing that, FAR test does not require the commensurability of the measures across different datasets, since it is non-parametric, neither assumes the normality of the sample means, and thus, it is robust to outliers. Additionally, in order to identify which algorithms report significant differences, the Finner test [38] with a significance level \(\alpha=0.05\), is applied as a post-hoc procedure. More analytically, the Finner procedure adjusts the value of \(\alpha\) in a step-down manner.