## 4 The Experiments
### Models Used for Evaluation  
One of the problems in machine learning applications is selecting an appropriate predictive model. Among the available methods, three main families can be determined: models derived from decision trees, distance-based methods, and neural networks. The first group of algorithms, i.e., based on decision trees, usually refer to complex models or ensembles of models that address the limitations of individual trees. For single \begin{table}
\begin{tabular}{c l l} \hline \hline  & **Class Name** & **Number of Records** \\ \hline
1 & No-fail & 68,345 \\
2 & Failure 1 & 47,207 \\
3 & Failure 2 & 14,333 \\
4 & Failure 3 & 16,472 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Preprocessed data records. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \#\(Class\)0 & \#\(Class\)1 & \#\(Samples\) & \#\(Class\)0 & \#\(Class\)1 \\  & & & & \#\(Samples\) & \#\(Samples\) \\ \hline Failure 1 & 53,345 & 47,207 & 100,552 & 0.5305 & 0.469  
decision trees, the problem lies in the shape of the decision boundary, which, for continuous data, takes the form of a stepwise curve. In this case, the application of complex models, including ensuring their diversity, results in multiple trees working together to increasingly better predict the true decision boundary. As mentioned earlier, it is important to ensure the diversity of individual trees. To solve this problem, many different algorithms have been developed, among which the most popular is Random Forest [5], where tree diversity is ensured by creating individual trees on independent subsets of features, and the final decision of the model is generated through a democratic vote of the component models. Another popular solution is the Gradient-Boosted Trees algorithm [6]. In this case, unlike the Random Forest algorithm, subsequent trees are added sequentially, focusing on the areas where previous trees caused the greatest model error. This is achieved by adapting the probability distribution used for sampling training vectors with replacement so that each subsequent tree is generated on a subset of training data. In this case, the final decision of the model is the result of a weighted vote of the individual component models. In the experiments carried out, another tree-based algorithm was also used, the rotation forest algorithm [7]. It operates similarly to Random Forest, with the difference that the algorithm generates random subspaces for which the tree is generated; however, before training the tree, the PCA algorithm is used to rotate the coordinate system. This algorithm is particularly useful when correlated variables exist, which simplifies the construction of a predictive model. All the tree-based algorithms have an additional significant advantage, namely, they provide the possibility of partial interpretation of the model results using the MDI (Mean Decrease in Impurity) coefficients. These coefficients are used to assess the importance of attributes, thus they can be interpreted as a measure of evaluating the quality of individual variables. Another important advantage of tree-based algorithms is the speed of training and prediction, as trees ensure low computational complexity and ease of parallelization. A separate group of models consists of distance-based algorithms. A typical example of this group is the k-Nearest Neighbors (kNN) algorithm [8]. It belongs to the group of lazy learning methods because the entire inference process occurs during prediction, where the k nearest neighbors for a given vector are identified from among the stored training vectors, and a vote is taken among them to select the appropriate class. Besides the voting method, the factor determining the quality of kNN is also the selection of reference vectors stored in the model's memory. These methods are also called instance selection methods. This enables the interpretation of the model, including the use of so-called prototype-based rules [25] and case-based reasoning methods [26]. The last method evaluated is the neural network [9; 27].