## 3 Results
### ML Models and Developed Ensembles Results  
The research methodology is organized as follows:  
1. We divide the available sample into training and testing. For training, we will select 70% of the original sample. 2. We train nine models for selected features, for selected and tentative features, and the whole dataset, respectively: 1. Decision forest (RTree using CART algorithm, tuning parameters is used too), 2. Logistic regression (LG, without L1 or L2 regularization), 3. Built-in Adaptive Boosting (AdaBoost), 4. Perceptron (MLP, hyperparameters after Bayesian Optimization {'activation':'sigmoid', 'batch_size': 209, 'epochs': 87, 'layers1': 15, 'layers2': 3, 'learning_rate': 0.07160)), 5. K Nearest neighbor (We will estimate three hyperparamters, namely n-neighbors, weights and metric. Next, the Exhaustive Grid Search technique for hyperparameter optimization is used. Since we have provided the class validation score as 3 (cv = 3), Grid Search will evaluate the model \(6\times 2\times 3\times 3=108\) times with different hyperparameters. The best values are the following: n_neighbors = 5, weights = 'uniform', algorithm = 'brute', metric ='minkowski'), 6. Random Forest (RF, 500 trees and the maximum depth is equal to 8), 7. Support vector Machine with Polynomial (SVMPoly, Grid Search is used too, hyperparameters C = 10, gamma=0.0001), 8. Support vector Machine with Radial (SVMRadial, Grid Search is used too, hyperparameters C = 10, gamma=0.0001), 9. Support vector Machine with Linear (SVMLin, Grid Search is used too, hyperparameters C = 10, gamma=0.0001). 3. We built four ensembles: 1. Boosted random forest, 2. Boosted logistic regression, 3. Bagged decision tree based on CARD algorithm, 4. Stacking. Figure 8: Evaluation of graft function depending on erythrocyturia. We carry out resampling of the obtained models, perform a statistical analysis of two generated model quality metrics (Accuracy and [PERSON]'s Kappa index) on the training set, rank the models, and estimate the confidence intervals of the criteria used. 5. The analysis is organized for the whole dataset and selected features, respectively. That is why the research methodology is used separately for the whole dataset and dataset based on selected features. 6. We check is the model overfitted using k-fold. At the first stage, the separated machine learning models are analyzed. The worst accuracy is for MLP trained on the whole dataset and selected features (Table 3). The best weak classifier is ADABOOST, with an accuracy equal to 0.8533677. That is why MLP classifier is excluded. The difference in accuracy for the whole dataset and selected features for each model is given in Table 3 in the last column. The accuracy of LG is low for the entire dataset but 0.13848 higher for the selected features. At the next stage, two Boosting modes based on regression tree and logistic regression are built. Bosted LG shows higher accuracy than all weak classifiers built on the whole dataset. However, the result of SVMRadial built on selected features is better than for boosted ensembles. Bagged CART without tuned parameters is trained at the next stage. However, the accuracy of this ensemble is worse than for boosted LG both for all features and for selected features. The next step is the combination of multiple predictors using stacking. KNN, SVMLin, SVMPoly, SVMRadial, trree, logistic regression (LG) are used for ensemble development. The random forest (RF) algorithm combines the predictions. 100 trees are built for RF. Tuning parameters are used too. Ensemble results are given below for the whole dataset. Accuracy and Kappa index are used for results evaluation. Random Forest results:  
* 207 samples,
* 6 predictors,
* 2 classes: 'first_class','second_class'
* No pre-processing
* Resampling: Cross-Validated (10 fold, repeated 3 times)
* Summary of sample sizes: 6, 7, 6, 6, 5,...
* Resampling results across tuning parameters: \begin{tabular}{c c c} mtry & Accuracy & Kappa \\
2 & 0.8990257 & 0.4704559 \\
4 & 0.8114255 & 0.4695021 \\ \end{tabular}