## 5 Results and Industrial Interpretation
### Classification: State and Mode Prediction
The best F-Micros for State and Mode are highlighted in bold. this method in Table 5 indicates that the MCC obtained is significantly better than the first method in the ranking. These last 2 conclusions can be explained by the fact that the Mode variable has a much clearer majority class (i.e., \(Mode=1\) has 74% of the instances); while the State variable is distributed between states 1 and 2 in a quite balanced way for 90% of the instances. The computation of confusion matrices for some of the methods can provide deeper insight into their accuracy. In a confusion matrix, the element in row \(i\) and column \(j\) represents the number of times the class in row \(i\) is predicted by the method as the class in column \(j\). In an ideal classifier that is correct 100% of the time, the elements of the diagonal will achieve 100% values (i.e., the elements of class \(i\) will always be correctly predicted as belonging to class \(i\)). Also, the rest of the cells within that ideal classifier that lie outside the diagonal will have a value of 0% (i.e., the elements of class \(i\) would never be predicted as belonging to a different class \(j\)). A \(5\times 2\) CV was performed (i.e., 5 repetitions of 2 folds cross validation), to produce reliable confusion-matrix values. Hence, the data set for a \(5\times 2\) CV was randomly divided into 2 parts at 50%. One part was used to train the classifier and the other, to validate the model by counting which class it rightly or wrongly predicted for each test instance. Then, the partitions were swapped (i.e., The partition that was previously used for training was then used for validation and vice-versa). The 50% random split differed for each of the 5 repetitions. Once that procedure had finished, \(5\times 2=10\) values for each cell of the matrix were obtained. The figures on the following page represent the average values of those 10 results. The data are shown as percentages instead of absolute values, so that minority classes are not underrepresented. Figure 2 shows the confusion matrices for the Mode classification problem. The two upper matrices represent the best methods in that problem (i.e., Bagging of Random Balance, because it has the highest values for F-Macro and MCC, and Rotation Forest of C4.5, because it has the highest value for F-Micro). The two lower matrices represent the two best non-ensemble methods (i.e., a C4.5 decision tree and k-NN). All the classifiers show diagonal values above 90%, except for _Prog Line_ class, which is confused half the time with _Automatic_ class. This confusion can be explained, as the first time that a new machining program is executed, the machine operator will usually run it line by \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \(\mathbf{\Delta}\) & \(\mathbf{V}\) & \(\mathbf{D}\) & **Method** & **State** & **Mode** \\ \hline
17 & 18 & 1 & Rotation Forest of C4.5 & **0.99596** & 0.85325 \\
12 & 17 & 5 & Bagging of Random Balance of C4.5 & 0.99406 * & **0.86118**+ \\
12 & 14 & 2 & Bagging of C4.5 & 0.99504 * & 0.85069 \\
10 & 14 & 4 & C 4.5 & 0.99475 * & 0.85193 \\
10 & 13 & 3 & LogitBoost of REPTree & 0.99591 & 0.84690 * \\
9 & 13 & 4 & AdaBoostM1 of C4.5 & 0.99579 & 0.84381 * \\
8 & 12 & 4 & Rotation Forest of Random Forest & 0.99569 & 0.84476 * \\
2 & 10 & 8 & Random Forest & 0.99483 * & 0.84530 * \\ \(-8\) & 8 & 16 & kNN & 0.98624 * & 0.83288 * \\ \(-12\) & 6 & 18 & Multilayer Perceptron & 0.87607 * & 0.76677 * \\ \(-16\) & 4 & 20 & Radial Basis Function SVM & 0.81216 * & 0.68454 * \\ \(-21\) & 2 & 22 & Naive Bayes & 0.64746 * & 0.29481 * \\ \(-23\) & 0 & 23 & Linear SVM & 0.62066 * & 0.26261 * \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Ranking of MCC in the classification problems. V is the number of times the method has a better statistically significant MCC when compared with the other 12 methods through the 2 predicted variables. D is the number of times the method has a worse statistically significant MCC when compared with the other 12 methods through the 2 predicted variables.