# Prediction of Flotation Deinking Performance: A Comparative Analysis of Machine Learning Techniques
## 3 Results  
SVR was implemented using LIBSVM library [46] with the linear and RBF kernel functions. The optimal hyperparameters of the SVR model (C in case of the linear kernel; \(\gamma\) and C in case of the RBF kernel) are determined using the grid search. An example of hyperparameter tuning for the RBF kernel for estimation of cellulose fiber recovery in the sink product (E\({}_{\text{m}}\)) is shown in Figure 1, in case the surfactant is oleic acid, and in Figure 1, in case the surfactant is oleic acid + CaCl\({}_{2}\). Hyperparameter C determines the tradeoff between the model complexity and the amount of error that can be tolerated. In general, a lower C tolerates a larger error at the cost of model accuracy, whereas a larger C increases the model complexity but enables better prediction. Note that in both subfigures, the model performance is extremely sensitive to the value of hyperparameter \(\gamma\), which must be chosen very carefully. When \(\gamma\) is too large, the area of influence of the support vectors is too narrow, so the overfitting appears. On the other hand, when \(\gamma\) is too small, the hyperplane becomes too flat, again leading to poor performance on the test dataset. Ensembles of regression trees were realized using random forests and boosting. In case of random forests, the hyperparameters to be optimized were the number of trees and the minimum leaf size. A grid search was used for the optimization. An example of hyperparameter tuning for the estimation of cellulose fiber recovery in the sink product  
Figure 1: Optimization of the hyperparameters \(\gamma\) and C in SVR with the RBF kernel for estimating cellulose fiber recovery in the sink product (E\({}_{\text{m}}\)) when the surfactant is (**a**) oleic acid and (**b**) oleic acid + CaCl\({}_{2}\). (E\({}_{\text{m}}\)) is shown in Figure 2a in case the surfactant is oleic acid, and in Figure 2b, in case the surfactant is oleic acid + CaCl\({}_{2}\). In general, more trees usually lead to better estimates; however, note that after 100 trees in Figure 2a and 50 trees in Figure 2b, the MSE mostly stabilizes and there is no point to further increase the number of trees, since this would increase the model complexity. The minimum leaf size determines the smallest number of observations a node is allowed to have. If a child node should be created by splitting with fewer observations than the minimum leaf size, the node is not split. Note that the most accurate models are obtained for the smallest minimum leaf sizes (equal to two in our experiments). However, this leads to deeper trees; so, better performance comes at the cost of increased complexity. In the case of boosting, the number of trees and the learning rate are optimized using a grid search. An example of hyperparameter tuning for the estimation of cellulose fiber recovery in the sink product (E\({}_{\text{m}}\)) is shown in Figure 3a, in case the surfactant is oleic acid, and in Figure 3b, in case the surfactant is oleic acid + CaCl\({}_{2}\). As in the case of random forests, the MSE mostly stabilizes for more than 50 trees and there is no point to further increase the number of trees, since this would increase the model complexity. The learning rate is a number between 0 and 1, which multiplies the step magnitude in each gradient step and defines how quickly the prediction error is corrected in the subsequent tree of the model. In other words, shrinkage appears in each gradient step. A learning rate equal to 1 means there is no shrinkage. Small learning rates cause sample predictions to slowly converge towards the observed values and can improve the model's generalization ability. However, smaller learning rates require larger trees and might become computationally expensive. The learning rate equal to 0.25 is the most optimal in our experiments. GPR is realized with exponential, squared exponential, Matern 3/2, Matern 5/2 and rational quadratic covariance functions.