## 4 Discussion  
In this work, we built a new model, PTG-PLM, for glycosylation/glycation site prediction based on features extracted from pre-trained protein language models (PLMs) and CNN deep learning. Generally, although CNN is ideal for image data while RNN is ideal for sequential data, there are successful studies that use CNN applied to sequence data and got great performance like [23; 41; 42]. In addition, CNN is easier and faster in training when compared with RNN [41]. Thus, we used CNN deep learning method to build our PTG-PLM model. Regarding feature extraction, According to the advances in protein language models (PLMs) that are based on state-of-the-art transformers, six pre-trained PLMs were used for extracting embedding features from protein-peptide sequences. These PLMs include ProtBERT-BFD, ProtBERT, ProtALBERT, ProtXLNet, ESM-1b, and TAPE. Experimental results indicate that PTG-PLM based on these PLMs embedding outperforms the existing tools for Ngly/Kgly site prediction except the PTG-PLM based on ProtXLNet embedding for Ngly site prediction that gave lower performance than some existing tools (Tables 9 and 10). Generally, we can see that PTG-PLM models based on ESM-1b and ProtBERT-BFD have better performance than the models based on the other four PLMs and they were considered for comparison with the existing tools for Ngly and Kgly site prediction, respectively. We think this advantage comes from the fact that ESM-1b and ProtBERT-BFD are trained on large datasets and have larger layers (33 layers and 30 layers, respectively) when compared with the other four PLMs. Overall, we found that Ngly's performance results with all models are far superior to Kgly's performance results. That is because the Ngly sites typically occur in the pattern or motif N-X-S/T where X is any amino acid other than Proline (P), whereas Kgly sites lack any pattern or motifs. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Metric** & & & & & & \\ \hline
**Tool** & **Accuracy** & **Recall** & **Precision** & **F1** & **AUC** & **MCC** \\ \hline NetGlycate [19] & 0.54 & 0.54 & 0.53 & 0.54 & 0.54 & 0.07 \\ Gly-PseAAC [21] & 0.17 & 0.16 & 0.16 & 0.16 & 0.17 & 0 \\ BPB\_GlySite [1] & 0.51 & 0.51 & 0.51 & 0.51 & 0.5 & 0.16 \\ BERT-Kgly [23] & 0.61 & 0.49 & 0.64 & 0.56 & **0.69** & 0.23 \\ PTG-PLM & **0.64** & **0.67** & **0.64** & **0.65** & 0.64 & **0.28** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparison of the performance of PTG-PLM against the existing tool for Kgly site prediction. Figure 8: Chart for the performance of PTG-PLM and the existing tools for Kgly site prediction. Furthermore, four supervised learning methods (SVM, LR, RF, and XBG) were trained on the Ngly and Kgly datasets based on the same embedding extracted from the six PLMs. These machine learning methods were compared with PTG-PLM on the independent datasets for a fair evaluation of PTG-PLM. The performance was compared based on each PLM individually and the results showed that PTG-PLM has the best performance on the independent set (Tables 7 and 8). Although the extracted features from the embedding of PLMs are large (Table 2), some supervised learning methods achieved great performance as XBG based on ProtBert-BFD on the Ngly dataset and SVM based on ProtBert-BFD on the Kgly dataset. This indicates that features extracted using the embedding from PLMs transformer-based have a better performance than the features extracted by using the physicochemical, evolutionary, MSA, or structural-based properties for the peptide sequences. Likewise, the combination of PLMs embedding and CNN deep learning produces a model which outperformed the existing tools and the state-of-the-art machine learning methods on both Ngly and Kgly datasets. Thus, PTG-PLM can be appropriate to predict the other types of PTMs. We can do that by configuring some important parameters of PTG-PLM like:  
1. [noitemsep,topsep=0 pt]
2. Fasta file that contains the protein sequences. 3.