# A Machine Learning Approach for the Prediction of Thermostable \(\beta\)-Glucosidases
## 4 Results and Discussion
### Thermostability Patterns in Enzymes Obtained from Non-Thermostable Organisms
#### 4.1.1 Case Study 1--Comparison to Another Tool  
Then, I performed a comparison with another thermostability prediction software: TemStaPro [42]. TemStaPro is a deep learning method that predicts, based on the sequence, whether the 3D structure will be stable at certain temperatures. For this case study, I used the predictions made for the temperature of 55 \({}^{\circ}\)C, which is the optimal temperature at which saccharification occurs [18].  
Thus, the 583 sequences from the test database were submitted for thermostability prediction by TemStaPro. Table 3 compares the results for TemStaPro predictions at 55 \({}^{\circ}\)C (TemStaPro-t55) with the prediction results using the aCSM-ALL signature and the CatBoost model obtained in this work.  
TemStaPro achieved a lower accuracy (66.2%) than the model presented in this study (81.6%). However, this does not directly imply that the competing model was worse. When we evaluate other metrics, such as precision and specificity, we can see that TemStaPro was superior (84% vs. 80.5% in precision and 93.4% vs. 81.4% in specificity). TemStaPro obtained a significantly higher number of false negatives (i.e., entries labeled as positive but classified as negative). While the model built with CatBoost based on the aCSM-ALL signatures obtained 51 false negatives, TemStaPro obtained 177. There are several explanations for this, such as an error in the competing method (although TemStaPro-t55 obtained 93.4% accuracy in a test with the SAPPHIRE dataset [42]) or even inconsistency in the definition of the labels. The labels originally assigned are based only on the notation of the organism that encodes the protein. There is no robust experimental validation indicating that the protein is actually stable at 55 \({}^{\circ}\)C. Therefore, the TemStaPro result may indicate results closer to the real one. However, it should also be considered that TemStaPro predictions are made only based on the sequences. Incomplete sequences or sequences with annotation errors can induce errors in the models. More experiments are necessary to evaluate these results.  
\begin{table}
\begin{tabular}{c c c} \hline  & **This Study** & **TemStaPro-t55** \\ \hline Accuracy & 0.816 & 0.662 \\ \hline F1 & 0.812 & 0.516 \\ \hline Precision & 0.805 & 0.840 \\ \hline Recall & 0.819 & 0.372 \\ \hline Specificity & 0.814 & 0.934 \\ \hline TP & 231 & 105 \\ \hline TN & 245 & 281 \\ \hline FP & 56 & 20 \\ \hline FN & 51 & 177 \\ \hline \end{tabular}
\end{table}
Table 3: Results for case study 1 (\(n\) = 583; Neg = 282; Pos = 301). Comparison to TemStaPro (5 \({}^{\circ}\)C).  
Furthermore, the model proposed in this work obtained 56 false positives, while TemStaPro-t55 obtained only 20 false positives. False positives are a group of interest for this work since they represent inputs labeled as negative that were classified as positive. Assuming that the model can generalize the prediction of thermostable \(\beta\)-glucosidase structures correctly, false positives would indicate possible structures obtained in non-thermostable organisms with thermostable characteristics. In total, eight \(\beta\)-glucosidase structures from non-thermophilic organisms were predicted as possibly thermostable by both software. They are A0A0G0 PX59, A0A0S8 EAX1, A0A0U5J538, A0A1C5 QUE8, A0A367G1L2, A0A498 KCE1, A0A7C3M843, and M1 CZQ6. Further investigations into these structures are necessary. More details about TemStaPro results can be found in the Supplementary Material (Table S4).