## 3 Results  
### Training Models Using Physico-Chemical Features  
In our experiments, firstly we have used the above-mentioned ten physico-chemical features and different machine learning methods: (i) to learn whether the peptides in each of our datasets have antimicrobial activity or not and (ii) to classify them accordingly. To this end, we have applied methods such as AdaBoost, Decision Tree, LogitBoost, RF, and SVM. As shown in Tables 2 and 3, for both Gram-negative and Gram-positive datasets, RF classifier resulted in the best performance metrics. While the AUC rate reached up to 90% for Gram-positive data, this rate was 92% for Gram-negative data. Not only for AUC rate, but also for other measures such as accuracy, recall, specificity, precision, and F1 measure, RF yielded the best performance metrics. Figure 3 displays the comparative evaluation of different models using AUC values for (a) Gram-negative dataset and (b) Gram-positive dataset. As it can be seen in Figure 3a and in Table 2, while 92% AUC value is obtained for Gram-negative dataset, 90% AUC value is obtained for Gram-positive dataset (shown in Figure 3b and in Table 3) using RF classifier. While the AUC values of other classifiers range between 0.77-0.87 for Gram-positive dataset (shown in Figure 3b and in Table 3), it ranges between 0.78-0.89 for Gram-negative dataset. To analyze the pairwise correlations of the features, Pearson correlation values between all pairs of features have been calculated using Python Seaborn Library. These relations were illustrated in Supplementary Figures S1 and S2 using a heatmap. The only statistically significant pairwise correlation worth mentioning was observed between the \"Isoelectric Point\" and \"Net Charge\" features. Between any other pairs of features, no significant correlation is observed. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Accuracy** & **Recall** & **Specificity** & **Precision** & **Area Under** & **F1** & **Balanced Acc.** \\ \hline AdaBoost & 0.85 \(\pm\) 0.06 & 0.92 \(\pm\) 0.06 & 0.72 \(\pm\) 0.20 & 0.87 \(\pm\) 0.07 & 0.88 \(\pm\) 0.06 & 0.89 \(\pm\) 0.04 & 0.82 \(\pm\) 0.13 \\ Decision Tree & 0.79 \(\pm\) 0.06 & 0.87 \(\pm\) 0.07 & 0.66 \(\pm\) 0.24 & 0.84 \(\pm\) 0.07 & 0.78 \(\pm\) 0.07 & 0.85 \(\pm\) 0.04 & 0.76 \(\pm\) 0.15 \\ LogitBoost & 0.86 \(\pm\) 0.05 & 0.92 \(\pm\) 0.06 & 0.74 \(\pm\) 0.16 & 0.88 \(\pm\) 0.06 & 0.89 \(\pm\) 0.06 & 0.90 \(\pm\) 0.03 & 0.83 \(\pm\) 0.11 \\ RF & **0.89 \(\pm\) 0.05** & **0.93 \(\pm\) 0.04** & **0.79 \(\pm\) 0.16** & **0.90 \(\pm\) 0.06** & **0.92 \(\pm\) 0.05** & **0.91 \(\pm\) 0.03** & **0.86 \(\pm\) 0.10** \\ SVM & 0.80 \(\pm\) 0.05 & 0.93 \(\pm\) 0.06 & 0.56 \(\pm\) 0.21 & 0.81 \(\pm\) 0.07 & 0.82 \(\pm\) 0.06 & 0.86 \(\pm\) 0.03 & 0.74 \(\pm\) 0.13 \\ SVM + kNN & 0.80 \(\pm\) 0.07 & 0.93 \(\pm\) 0.05 & 0.56 \(\pm\) 0.25 & 0.81 \(\pm\) 0.08 & 0.82 \(\pm\) 0.08 & 0.86 \(\pm\) 0.04 & 0.74 \(\pm\) 0.15 \\ LogitBoost + kNN & 0.80 \(\pm\) 0.05 & 0.93 \(\pm\) 0.06 & 0.56 \(\pm\) 0.21 & 0.81 \(\pm\) 0.07 & 0.82 \(\pm\) 0.06 & 0.86 \(\pm\) 0.03 & 0.74 \(\pm\) 0.13 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of different models according to different performance metrics for Gram-negative dataset, using physico-chemical features. Figure 3: Comparison of the performances of different models in terms of their AUC values with standard deviation values for (**a**) Gram-negative, and (**b**) Gram-positive dataset, using physico-chemical features. Feature selection procedure tries to reduce the computational costs by removing redundant or irrelevant variables from input data. This technique contributes to better understanding the generated model and allows one to improve the model via focusing on the important features. In order to perform this task, one needs to score or rank the features in terms of how useful they are at predicting the output. There are different approaches for feature ranking that are based on statistics measurements or wrapper approaches that are based on machine learning [86].