## 3 Results
### Comparing with Supervised Learning Methods  
On the testing sets of Ngly and Kgly datasets, the PTG-PLM model is compared with four state-of-the-art supervised learning methods: SVM, LR, RF, and XGB based on the embedding extracted for the six PLMs. Table 7 illustrates the performance of PTG-PLM compared with supervised learning methods based on embedding extracted from the six PLMs on the Ngly independent dataset. The comparison is measured by three performance metrics: accuracy, AUC, and MCC. Generally, the PTG-PLM outperforms the other supervised learning methods in terms of accuracy, AUC, and MCC. Moreover, Table 7 shows that the highest performance of PTG-PLM on the independent dataset is based on the embedding of the ESM-1b transformer with a 0.965 accuracy, 0.978 AUC, and 0.902 MCC. On the other hand, the highest performance between the four supervised learning methods is with the XGB model and based on the embedding of ProtBert-BFD transformer with a 0.952 accuracy, 0.970 AUC, and 0.871 MCC. The comparison of performance between the best PTG-PLM model based on ESM-1b and the best-supervised learning model (XGB) based on ProtBert-BFD is shown in Figure 5. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**PLM** & **Accuracy** & **Recall** & **Precision** & **F1** & **AUC** & **MCC** \\ \hline ProtBert-BFD & 0.599 \(\pm\) 0.01 & 0.612 \(\pm\) 0.04 & 0.594 \(\pm\) 0.017 & 0.603 \(\pm\) 0.02 & 0.599 \(\pm\) 0.01 & 0.198 \(\pm\) 0.014 \\ ProtBert & 0.597 \(\pm\) 0.014 & **0.64 \(\pm\) 0.041** & 0.588 \(\pm\) 0.017 & **0.613 \(\pm\) 0.017** & 0.59 \(\pm\) 0.015 & 0.196 \(\pm\) 0.031 \\ ProtAlbert & 0.593 \(\pm\) 0.012 & 0.584 \(\pm\) 0.05 & 0.614 \(\pm\) 0.026 & 0.599 \(\pm\) 0.05 & 0.591 \(\pm\) 0.012 & 0.186 \(\pm\) 0.024 \\ ProtXLNet & 0.568 \(\pm\) 0.01 & 0.576 \(\pm\) 0.05 & 0.57 \(\pm\) 0.021 & 0.573 \(\pm\) 0.06 & 0.568 \(\pm\) 0.008 & 0.144 \(\pm\) 0.02 \\ ESM-1b & **0.599 \(\pm\) 0.009** & 0.563 \(\pm\) 0.035 & **0.604 \(\pm\) 0.009** & 0.583 \(\pm\) 0.018 & **0.599 \(\pm\) 0.008** & **0.199 \(\pm\) 0.017** \\ TAPE & 0.595 \(\pm\) 0.009 & 0.572 \(\pm\) 0.043 & 0.598 \(\pm\) 0.021 & 0.594 \(\pm\) 0.03 & 0.595 \(\pm\) 0.007 & 0.192 \(\pm\) 0.014 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Cross-validation performance of CNN model based on the six PLMs on the Kgly dataset. The results of the five-fold cross-validation on the training dataset are represented by the average \(\pm\) standard deviation for all performance measures. Figure 4: Cross-validation performance results of PTG-PLM on the Kgly training dataset based on the six PLMs. Based on the Kgly independent set, the PTG-PLM is also compared with the four supervised learning based on the embedding features extracted using the six PLMs. Table 8 shows the comparative performance in terms of accuracy, AUC, and MCC. As shown in the table, the PTG-PLM outperforms the four supervised learning methods across all the PLMs. Furthermore, PTG-PLM with the ProtBert-BFD PLM has the best performance on the Kgly independent dataset with a 0.641 accuracy, 0.641 AUC, and 0.282 MCC. When comparing the four supervised learning methods, SVM with the ProtBert-BFD PLM has the highest performance on the Kgly independent dataset with a 0.631 accuracy, 0.631 AUC, and 0.261 MCC. The comparison of performance between the two best models, PTG-PLM with the ProtBert-BFD and SVM with the ProtBert-BFD, on the Kgly dataset is shown in Figure 6.