## 3 Results
### Comparison with Previous Models  
Previously reported methods for the prediction of thermophilic proteins used physicochemical feature extraction methods. Although deep learning had already been used to identify thermophilic proteins by previous works, such as DeepTP [12], these models did not avoid using physicochemical features such as AAC, DC, and CTD. In a striking departure from this approach, our work is the first demonstration of the utility of deep representation learning algorithms for identifying thermophilic enzymes. We compared the performance of our BertThermo model with the previously proposed prediction algorithms. Since most previous models reported outcomes based cross validation experiments, while others showed the outcome of independent tests, we summarized these comparisons in two tables. Table 3 illustrates the findings of cross validation tests, while Table 4 illustrates the outcomes of independent tests using the same test set as us.  
Our model showed the highest accuracy in cross validation tests, reaching an Sn and Sp of 0.9697. These values are significantly better than those obtained with previous models. In terms independent tests, the best accuracy achieved by our model was 0.9751, with other metrics being also better than those produced by the previous model, iThermo. Since our BertThermo algorithm was tested on the same dataset as iThermo [10], these results represent direct comparisons and clearly demonstrate the improved performance  
\begin{table}
\begin{tabular}{l l l l l l l l l l l l l l l} \hline \multirow{2}{*}{**Top**} & \multirow{2}{*}{**SOME**} & \multicolumn{6}{c}{**5-Fold Cross-Validation**} & \multicolumn{6}{c}{**Independent Test**} & \multicolumn{6}{c}{**Average**} \\ \cline{3-14}  & & **ACC** & **MCC** & **Sn** & **Sp** & **AUC** & **ACC** & **MCC** & **Sn** & **Sp** & **AUC** & **ACC** & **MCC** & **Sn** & **Sp** & **AUC** \\ \hline
90 & Yes & 0.9697 & 0.9394 & 0.9697 & 0.9697 & 0.9957 & 0.9751 & 0.9504 & 0.9853 & 0.9654 & 0.9935 & 0.9724 & 0.9449 & 0.9775 & 0.9675 & 0.9946 \\
1024 & Yes & 0.9697 & 0.9397 & 0.9653 & 0.9740 & 0.9947 & 0.9662 & 0.9327 & 0.9780 & 0.9550 & 0.9935 & 0.9679 & 0.9362 & 0.9717 & 0.9645 & 0.9941 \\
90 & No & 0.9675 & 0.9350 & 0.9662 & 0.9688 & 0.9954 & 0.9751 & 0.9504 & 0.9853 & 0.9654 & 0.9935 & 0.9713 & 0.9427 & 0.9757 & 0.9671 & 0.9945 \\
1024 & No & 0.9675 & 0.9351 & 0.9661 & 0.9688 & 0.9955 & 0.9751 & 0.9504 & 0.9853 & 0.9654 & 0.9936 & 0.9713 & 0.9428 & 0.9757 & 0.9671 & 0.9946 \\ \hline \end{tabular}
\end{table}
Table 2: Before or after feature selection and with or without the SMOTE, performance of the model on the independent test and cross-validation.  
of the new approach. From another perspective, compared with the complex classifiers and higher computational complexity chosen by other groups, such as SVM [2; 5; 6; 7; 8] and MLP [10], our model uses LR as a classifier, resulting in reduced computational complexity. This also means that BertThermo is easier to train and is more stable. In more general terms, the impressive performance of BertThermo shows the promise of NLP-based algorithms in predicting thermophilic proteins and suggests the utility of NLP-based approaches in studying other features of protein function and behavior in the future.