## 4 Conclusions  
The work presented here compares the performance of models using three different implementations of BERT in identifying thermophilic proteins. We found that the use of BERT-bfd is the most suitable approach for this task, significantly outperforming three other deep representation learning models, UniRep, SSA, and W2V. We refined the algorithm further, showing that out of seven tested classifiers, LR results in the highest accuracy in both cross-validation and independent tests. In addition, the use of feature engineering methods, such as the SMOTE and the LGBM, could increase the performance of the model even further. In final testing, the optimized model achieved an accuracy of 96.97% in 5-fold cross-validation and 97.51% in independent tests. Comparing the performance of the BertThermo algorithm reported here with previous models proves that BertThermo is currently the best available model for identifying thermophilic proteins. This model provides the scientific community with a promising new tool for the detection of enzymes that are stable at higher temperatures, and may provide valuable information for use in biotechnology. The following supporting information can be downloaded at: [[https://www.mdpi.com/article/10.3390/app13052858/s1](https://www.mdpi.com/article/10.3390/app13052858/s1)]([https://www.mdpi.com/article/10.3390/app13052858/s1](https://www.mdpi.com/article/10.3390/app13052858/s1)), Table S1. The evaluation metrics score of models with various machine learning method. [PERSON] performed the data analyses and wrote the manuscript; [PERSON], [PERSON] and [PERSON] performed the data collection; [PERSON] helped write the source code; [PERSON] contributed significantly to constructive discussions; [PERSON] supervised the work. All authors have read and agreed to the published version of the manuscript. \begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{\({}^{\mathbf{b}}\)**Balanced Dataset**} & \multicolumn{4}{c}{\({}^{\mathbf{b}}\)**Imbalanced Dataset**} & \multicolumn{4}{c}{\({}^{\mathbf{b}}\)**Homology Dataset**} \\ \cline{2-13}  & **ACC** & **MCC** & **Sn** & **Sp** & **AUC** & **AP** & **ACC** & **MCC** & **Sn** & **Sp** & **AUC** & **AP** & **ACC** & **MCC** & **Sn** & **Sp** & **AUC** & **AP** \\ \hline \({}^{\mathrm{a}}\) TMPPred [62] & 0.708 & 0.418 & 0.659 & 0.758 / & / & 0.725 & 0.129 & 0.733 & 0.724 / & / & 0.663 & 0.327 & 0.690 & 0.636 / & / & \\ \({}^{\mathrm{a}}\) SCMTPP [63] & 0.761 & 0.545 & 0.621 & 0.902 & 0.846 & 0.857 & 0.882 & 0.237 & 0.733 & 0.884 / & & 0.248 & 0.720 & 0.440 & 0.730 & 0.710 & 0.808 & 0.792 \\ \({}^{\mathrm{a}}\)ThermoFisher [10] & 0.791 & 0.583 & 0.749 & 0.832 & 0.868 & 0.867 & 0.841 & 0.196 & 0.800 & 0.814 / & / & 0.297 & 0.740 & 0.482 & 0.780 & 0.700 & 0.834 & 0.836 \\ \({}^{\mathrm{a}}\) SAPPHIRE [11] & 0.821 & 0.657 & 0.711 & 0.890 & 0.904 & 0.916 & 0.930 & 0.316 & 0.733 & **0.933** / & / & 0.527 & 0.785 & 0.570 & 0.790 & 0.790 & 0.871 & 0.862 \\ \({}^{\mathrm{a}}\) DeepTP [12] & 0.873 & 0.746 & 0.854 & 0.891 & 0.944 & 0.946 & 0.886 & 0.277 & 0.833 & 0.887 & / & 0.536 & 0.830 & 0.671 & 0.920 & 0.740 & 0.909 & 0.906 \\ BertThermo & **0.906** & **0.813** & **0.919** & 0.894 & **0.968** & **0.966** & 0.903 & **0.326** & **0.900** & 0.903 & 0.974 & **0.711** & **0.988** & **0.793** & **0.930** & **0.857** & **0.957** & **0.965** \\ \hline \hline \end{tabular}  
* Data from reference DeepTP. \({}^{\mathrm{b}}\) The best value is in bold.  
\end{table} 
Table 6: Comparison with state-of-the-art methods with various independent datasets.