## 2 Materials and Methods
### Machine Learning Algorithms
#### 2.2.2 Adaptive and Gradient Boosting Classifiers  
Boosting is a general method to increase the effectiveness of any learning algorithm. However, it is often used with decision trees. The decision tree is a machine learning algorithm that divides the dataset into smaller subsets according to their features. The splitting process continues until all the remaining data belong to only one class. The boosting method belongs to ensemble learning, where subsequent classifiers of the same algorithm (e.g., decision tree) are trained, and an ensemble classifier is created based on them. [PERSON] and [PERSON] [69] discussed whether a weak learning algorithm, whose efficiency is slightly better than random guessing, could create an effective strong classifier. The boosting method was introduced by [PERSON] and [PERSON] [70]. There are three main types of boosting algorithms: adaptive boosting (AdaBoost), gradient boosting (GB), and extreme gradient boosting (XGBoost). The first two were selected for use in this article.  
The AdaBoost algorithm was proposed by [PERSON] and [PERSON] [71]. This algorithm initially sets the same weights for all samples in the training dataset. Next, a new classifier is built based on the training dataset, considering the sample weights. This new classifier is added to the ensemble classifier, with a weight corresponding to its accuracy. Finally, the sample weights are updated; correctly classified samples receive a lower weight, and incorrectly classified samples receive a higher weight. In the AdaBoost algorithm, each subsequent classifier tries to classify samples that the previous ones misclassified. The final prediction in the AdaBoost algorithm is based on a majority vote of the weak classifiers' predictions weighted by their accuracy. The entire process is repeated until the residual error, or the difference between the actual and predicted values, falls below the established threshold. The AdaBoost algorithm used in this article was based on an elementary form of decision tree known as stumps (i.e., the maximum depth of these trees equaled 1). The maximum number of decision trees at which boosting was terminatedequaled 50. Stagewise additive modeling using a multiclass exponential loss function for real (SAMME.R) was applied during model training. This algorithm uses the class probabilities during training.  
The gradient boosting algorithm was developed by [PERSON] [72], and is similar to the AdaBoost algorithm in that it is also based on a sequential training technique. All classifiers influence the final decision regarding the classification of individual samples (e.g., decision trees) used to build the algorithm model. However, this algorithm does not increase the weights of incorrectly classified samples. The contribution of the weak classifier to the ensemble classifier is based on the gradient descent algorithm, an iterative optimization algorithm that minimizes the loss function. The subsequent classifiers are maximally correlated with the negative gradient of the loss function associated with the whole ensemble classifier. The GB algorithm used in this article was based on decision trees, and the number of boosting stages was 100. The log loss function was optimized. The maximum depth, which limits the number of nodes in each tree, equaled 3. The learning rate, which determines the share of each tree in the ensemble classifier, equaled 0.1. Such a small learning rate value requires more trees but has better generalization ability. This regularization method is referred to as shrinkage. The Friedman mean square error (Friedman mse) criterion was applied to measure the quality of a split in decision trees.