# LPBERT: A Protein-Protein Interaction Prediction Method Based on a Pre-Trained Language Model
## 4 Discussion
The test results of LPBERT on multiple datasets in this study validated this conclusion and demonstrated the superior performance and good generalizability of LPBERT in PPI prediction tasks. Although LPBERT is at a disadvantage in terms of performance compared with xCAPT5, it surpassed xCAPT5 in terms of computational resource savings. In addition to the RAM resources mentioned above, we also compared the two methods in terms of storage resource consumption. When processing the 8347 protein sequences of the Pan dataset, the resulting file of xCAPT5 was approximately 17 GB, whereas LPBERT required only approximately 550 MB. Obviously, when facing large datasets, the run threshold of xCAPT5 is significantly greater than LPBERT. LPBERT achieved several times more computational resource savings than xCAPT5, with a performance evaluation metric gap of only 1% to 2%. An analysis of this phenomenon found that the main difference lay in the methods used to process protein embedding representations extracted from pre-trained protein language models. xCAPT5 chooses to fully preserve the extracted embedding representations, whereas LPBERT employs global max pooling operations to reduce the dimensionality of local embedding representations. Although this operation may result in the loss of some feature information compared with the savings in computing resources, it is clearly a better choice to perform pooling operations. This also significantly reduces the number of computational parameters and the consumption of computational resources. In this study on PPI prediction methods, LPBERT demonstrated excellent predictive abilities and showed outstanding performance on multiple public datasets. However, it still has certain limitations. We observed that the predictive performance of LPBERT was relatively insufficient when handling small datasets. Taking the _S. cerevisiae_ core dataset in Table 7 as an example, which contains only 10,537 data samples, the performance of LPBERT on this dataset showed a noticeable gap compared with other datasets. This gap indicates that the predictive ability of LPBERT relies to some extent on the size of the dataset. Furthermore, the comparison of LPBERT's performance with xCAPT5, SDNN-PPI, and DeepCF-PPI, indicates that there is still room for improvement in the performance of LPBERT. Moreover, when LPBERT is applied to cross-species prediction scenarios, its predictive ability appears somewhat inadequate. This reveals that its ability to infer unknown data from known data needs improvement, providing a valuable reference direction for future researchers. Although using only protein sequences to mine the deep information contained in them to achieve a higher-performance PPI prediction is a very valuable research direction, modeling only from the perspective of sequences is not enough. Multimodal information fusion is a feasible improvement direction. In order to fully express the characteristic information of proteins and achieve higher-performance PPI task prediction, in the future, we can consider further integrating the protein structure, the function, and other information based on protein sequences to construct a multimodal pre-trained protein language model. In this way, the generalization ability and feature expression ability of the pre-trained protein language model can be further improved. At the same time, minimizing the demand for computing resources of the protein language model will also help promote its application in downstream protein tasks and provide more powerful tools for biomedical research.