# A Machine Learning Approach for the Prediction of Thermostable \(\beta\)-Glucosidases
## 4 Results and Discussion  
I collected 1717 \(\beta\)-glucosidase 3D structures from the AlphaFold Protein Structure Database. Approximately 50% of them were supposed to be from thermophilic organisms. Then, I calculated their structural signatures using the aCSM algorithm. Structural signatures are mathematical methods that extract representative features from three-dimensional structures of macromolecules. The aCSM-ALL signature returns a numerical vector of 3600 values for each protein using the parameters 10/0.1 A. These numerical vectors were used as input for the various machine learning models built. I aimed to evaluate whether there is a structural pattern in the enzymes extracted from thermophilic organisms that are not present in the \(\beta\)-glucosidase enzymes of non-thermophilic organisms. Finally, I wanted to verify whether there are enzymes from non-thermophilic organisms with patterns similar to those of thermophilic ones. A k-fold cross-validation (k = 10) was performed using 2/3 of the entire dataset as the training set. Among the evaluated models, logistic regression obtained the highest accuracy (77.1%), followed by gradient boosting (76.2%), neural network (74.8%), CatBoost (74.1%), random forest (73.0%), K-nearest neighbors (KNN, 72.3%), and support vector machine (SVM, 55.8%). Table 1 summarizes the cross-validation results. Then, I performed the test validation using approximately 1/3 of the database. For the test, the CatBoost model obtained the highest accuracy of 81.6%, followed by the gradient boosting (scikit-learn) model (80.4%), the neural network model (79.6%), the random forest model (78.2%), the logistic regression model (76.2%), the KNN model (72.6%), and the SVM model (55.6%). Table 2 summarizes the test results (details can be found in Supplementary Table S1). The logistic regression model achieved the best accuracy in training but ranked fifth in testing. However, its accuracy in cross-validation was less than 1% higher than in testing (77.1% in training and 76.2% in testing). This suggests that its fall in the ranking of the best models in testing is due more to an improvement in the results of the other models than to any deficiency of its own. During training, the top six models achieved a difference of less than 5% in accuracy. In testing, this difference was 9%. Except for the KNN, logistic regression, and SVM models, all other models were superior in testing than in training. The ROC curve plot (Figure 3) confirms that the top six models achieved similar results in both training and testing. Only the SVM model achieved low accuracies in both training and testing. Briefly, CatBoost was the model that generalized best, outperforming the others in accuracy on the test set. On the other hand, SVM presented generalization challenges and limitations in learning, being less suitable for the structural signature data. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Model** & **Accuracy** & **F1-Score** & **Precision** & **Recall** & **Specificity** \\ \hline Logistic Regression & 0.771 & 0.782 & 0.773 & 0.791 & 0.749 \\ \hline Gradient Boosting & 0.762 & 0.774 & 0.764 & 0.784 & 0.738 \\ \hline Neural Network & 0.748 & 0.764 & 0.742 & 0.788 & 0.705 \\ \hline CatBoost & 0.741 & 0.754 & 0.743 & 0.766 & 0.714 \\ \hline Random Forest & 0.730 & 0.729 & 0.763 & 0.698 & 0.765 \\ \hline kNN & 0.723 & 0.738 & 0.725 & 0.752 & 0.692 \\ \hline SVM & 0.558 & 0.610 & 0.563 & 0.664 & 0.444 \\ \hline \end{tabular}
\end{table} 
Table 1: Results of cross-validation (k = 10)—stratified.