## 5 Results and Industrial Interpretation
### Classification: State and Mode Prediction  
Table 3 shows the 13 classification methods tested in the experiment. Each method is compared with the other 12 methods for the 2 classification problems State and Mode (i.e., \(12\times 2=24\) comparisons). One method for each one of these 24 comparisons can receive a significantly better result than the other method. So, in this case, we say that the method \"wins\". The method can also get a significantly worse result than the other method, and then we say that it \"loses\". Finally, the comparison may return no significant differences. In Table 3, columns **V** and **D** respectively represent the number of these significant wins and losses in the two classification problems for the method in that row versus all other methods. Then, the difference \"A\" is computed by subtracting D from V. This difference is taken as an indicator of what the best method is. In Table 3, the methods are ordered by \(\Delta\). We can see from the table that the two best positioned methods are tied with \(\Delta\) equal to 16. They win 16 times from the 24 matches, and they never lose. The two rightmost columns of Table 3 contain the average F-Macro values for each method and predicted variable. These average values are computed from the 100 experiments from the \(10\times 10\)CV. When these values are followed by \"*\", they are significantly worse than the one achieved by the first \(\Delta\) ranked method (i.e., Rotation Forest of Random Forest in the table, but we also tested that \"*\"'s keep unchanged, if the best method was considered Random Forest). The best F-Macros for each predicted variable are highlighted in bold (i.e., LogitBoost for State and Bagging of Random Balance of C4.5 for Mode). We also tested that \"*\"'s in the State column keep unchanged, if the methods were compared to LogitBoost, as well as in the Mode column, if the methods were compared to Bagging of Random Balance. The following conclusions may be identified from Table 3:  
1. Very good results may be found with just one single C4.5 decision tree. It could, on the one hand, mean that the input variables adequately describe the output variables and, on the other hand, that the number of instances is also sufficient. 2. Both problems are not suitable for a linear classification in view of the results of the linear SVM. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \(\Delta\) & **V** & **D** & **Method** & **State** & **Mode** \\ \hline
16 & 16 & 0 & Rotation Forest of Random Forest & 0.99342 & 0.92407 \\
16 & 16 & 0 & Random Forest & 0.99283 & 0.92444 \\
15 & 16 & 1 & LogitBoost of REPTree & **0.99370** & 0.92056 \\
14 & 15 & 1 & AdaBoost.M1 of C4.5 & 0.99297 & 0.92297 \\
10 & 15 & 5 & Bagging of Random Balance of C4.5 & 0.98857 * & **0.92785** \\
8 & 13 & 5 & Rotation Forest of C4.5 & 0.99298 & 0.91275 * \\
2 & 11 & 9 & Bagging of C4.5 & 0.99115 * & 0.91359 * \\ \(-3\) & 9 & 12 & C 4.5 & 0.99056 * & 0.90944 * \\ \(-6\) & 8 & 14 & kNN & 0.98119 * & 0.90899 * \\ \(-13\) & 5 & 18 & Radial Basis Function SVM & 0.82836 * & 0.77250 * \\ \(-15\) & 4 & 19 & Multilayer Perceptron & 0.79272 * & 0.66775 * \\ \(-21\) & 1 & 22 & Naive Bayes & 0.64369 * & 0.44891 * \\ \(-23\) & 0 & 23 & Linear SVM & 0.64819 * & 0.39074 * \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Ranking of F-Macro in the classification problems. V is the number of times the method has a better statistically significant F-Macro when compared with the other 12 methods through the 2 predicted variables. D is the number of times the method has a worse statistically significant F-Macro when compared with the other 12 methods through the 2 predicted variables. The ordering criteria \(\Delta\), is equal to V - D. The columns State and Mode have the average F-Macro figures for the method using \(10\times 10\)CV. (*) represents this method has an average F-Macro that is significantly worse than the one for the best-ranked method. The best F-Macros for State and Mode are highlighted in bold.