# Explainable Machine Learning Model to Accurately Predict Protein-Binding Peptides
## 6 Results and Discussion
### Selecting a Base Classifier to Build Our Model  
To build our model, we use different classifiers for our extracted features. We studied several classifiers and, among them, chose RF, SVM, and gradient-boosting decision trees (XGBoost). As discussed in the Introduction Section, these three classifiers have been shown to be effective methods in predicting protein-binding peptides [26; 37; 38; 39; 40; 41; 42]. Introduced by [PERSON], Random Forest is considered one of the best ensemble classifiers [48]. Based on the Bagging approach, RF promotes randomness to obtain better results. For RF, the employed data are first randomly divided into several groups. Then, for each group, a base decision tree is used for training using a random subset of a total number of features. Therefore, both data and features are randomly used to train each decision tree. After that, the results of all the base learners are combined to present the final classification. On the other hand, XGBoost is based on a boosting method [49]. In this model, a base classifier is trained iteratively on a subset of the data, and based on its performance, the weights for the classified samples are adjusted for the next iteration. In the next iteration, the weights of those samples that are incorrectly classified are increased to make it more costly to misclassify them. This process is being repeated several times. In the end, all those predictors in different iterations are combined to provide the final predictor. For the XGBoost, the employed base learner is gradient descent as its base learner. We use these three classifiers with different parameters for our three extracted feature groups. We use SVM with linear and Radial Basis function kernels. We also use RF with 50, 100, and 200 decision trees. In this way, we can choose the parameters that perform the best. We used grid search as well as several values manually (those that performed well for similar studies) to tune our classifiers and optimize them. After careful comparison, we used the parameters that are general, and also their results were aligned with the outcome of the grid search to avoid overfitting. The results achieved using these classifiers for Composition, Occurrence, Bigram, and Physicochemical-based feature groups for the independent test set are presented in Table 1, Table 2, Table 3, and Table 4, respectively. As shown in Tables 1-4, using SVM, we achieve the best results. After SVM, XGBoost achieves better results than RF. Among different numbers of base learners, using 100, RF achieves its best results. Still, it performs poorer than SVM and XGBoost. As shown in Table 1, using SVM with a linear kernel, we achieved 92.1% accuracy, 86.0% sensitivity, 93.6% specificity, 0.765 MCC, 0.937 AUC, and 0.813 F1 score, which outperforms other combinations. It highlights the importance of the monogram feature group. Also, better results using linear kernel demonstrate that linear transformation of the input data to a \begin{table}
\begin{tabular}{c c c c c c c} \hline
**Model** & **Acc (\%)** & **Sn (\%)** & **Sp (\%)** & **MCC** & **AUC** & **F1-Score** \\ \hline SVM-RBF & 90.6 & 86.0 & 91.8 & 0.730 & **0.945** & 0.785 \\ \hline
**SVM-Linear** & **92.1** & **86.0** & **93.6** & **0.765** & **0.937** & **0.813** \\ \hline RF-50 & 89.4 & 84.0 & 90.8 & 0.698 & 0.933 & 0.760 \\ \hline RF-100 & 90.8 & 88.0 & 91.5 & 0.740 & 0.941 & 0.793 \\ \hline RF-200 & 90.5 & **86.5** & 91.5 & 0.729 & 0.944 & 0.784 \\ \hline XGBoost & 90.1 & 85.5 & 91.3 & 0.718 & 0.942 & 0.775 \\ \hline \end{tabular}
\end{table} 
Table 1: Performance of our employed classifiers using composition feature group on the independent dataset.