# LPBERT: A Protein-Protein Interaction Prediction Method Based on a Pre-Trained Language Model
## 4 Discussion  
In this work, we proposed a novel deep learning method, LPBERT, to address the challenges of PPI prediction in proteomics. It is a computational method that relies solely on protein sequences for prediction. As the primary structure of proteins, protein sequences consist of various amino acids arranged in a specific order. These sequences inherently contain rich and critical information about proteins. By utilizing this information, researchers achieved remarkable results in multiple protein-related fields that drove the development of biomedical technology. The DeepMind team developed AlphaFold2 [21], a method based solely on amino acid sequences that achieved the accurate prediction of protein three-dimensional structures at the atomic precision level. Its prediction accuracy led the field in the 14 th Critical Assessment of Protein Structure Prediction (CASP14) [44] competition, where it significantly outperformed other methods. This result confirms that deep and interpretable information is indeed embedded within protein sequences. Earlier research employed various methods to extract features from protein sequences efficiently. For instance, Auto Covariance (AC) [42] and the Position-Specific Scoring Matrix (PSSM) [45] were used to extract information about the physicochemical properties and evolutionary characteristics of proteins. DeepFE-PPI [40] focused on the semantic relationships between amino acids using Word2 Vec, while HNSPPI [41] concentrated on extracting structural features from protein sequences using Seq2 Vec. While these methods successfully extracted features from protein sequences, they typically focus on only one aspect of the sequence, which leads to a limited understanding of the overall information. In practice, researchers overcame this limitation by combining multiple feature extraction techniques to complement each other and obtain a more comprehensive view of protein sequence information. This strategy is effective, but its technical implementation tends to be more complex. Researchers are required to invest time and effort into selecting suitable feature extraction methods, seeking the optimal combination among various methods, and avoiding redundant extraction of the same feature by different methods. These factors directly affect the final predictive performance of deep learning-based methods. Therefore, exploring a method that can comprehensively capture protein sequence information while simplifying the feature extraction process has become a new research objective. Protein \begin{table}
\begin{tabular}{c c c c c c} \hline
**Dataset** & **Positive** & **Negative** & **Accuracy** & **Precision** & **F1-Score** \\  & **Samples** & **Samples** & & & \\ \hline BioGRID _S. cerevisiae_ & 13,462 & 13,462 & 93.05 \(\pm\) 0.21 & 98.96 \(\pm\) 0.06 & 92.61 \(\pm\) 0.25 \\ _S. cerevisiae_ core & 5271 & 5266 & 51.81 \(\pm\) 0.55 & 51.10 \(\pm\) 0.32 & 63.93 \(\pm\) 0.63 \\ Guo & 5594 & 5594 & 51.82 \(\pm\) 0.07 & 51.12 \(\pm\) 0.04 & 63.28 \(\pm\) 0.13 \\ Multi-species & 32,959 & 32,959 & 47.79 \(\pm\) 0.36 & 48.74 \(\pm\) 0.21 & 62.05 \(\pm\) 0.34 \\ Human–virus & 8929 & 8929 & 86.48 \(\pm\) 1.07 & 93.84 \(\pm\) 1.16 & 85.22 \(\pm\) 1.53 \\ \hline \end{tabular}  
* This table reports the mean and standard deviation of the three test results.  
\end{table} 
Table 11: The cross-species generalization ability validation of LPBERT (%). language models, pre-trained on large-scale datasets and specifically designed for protein-related tasks, provide a viable solution. LPBERT is an effort based on this understanding. It utilized the protein language model to directly obtain the initial embedding representations of protein sequence pairs, which laid a solid foundation for subsequent computations within the framework. This feature extraction process is significantly simpler than other methods, and the extracted protein information is relatively rich and comprehensive.