# LPBERT: A Protein-Protein Interaction Prediction Method Based on a Pre-Trained Language Model
## 3 Results
### Ablation Experiment  
To assess LPBERT's component effectiveness, we performed ablation experiments on the BioGRID \(H\). _sapiens_ dataset by systematically removing and replacing specific modules. Table 8 presents the detailed experimental results. The model's performance metrics declined to varying degrees when either the GCT or LCR module was removed, which demonstrated the essential role of both modules in LPBERT's architecture. In addition, we replaced the GCT with a Transformer and the LCR with CNN blocks in our experiments. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Method** & **Sequence Encoding** & **Accuracy** & **MCC** \\ \hline \multirow{3}{*}{_S. cerevisiae_ (DeepFE, 5-fold)} & DeepFE-PPI & Word2 Vec & 94.78 \(\pm\) 0.61 & 89.62 \(\pm\) 1.23 \\  & SDNN-PPI & AAC + CT + AC & **95.48 \(\pm\) 0.37** & **91.02 \(\pm\) 0.74** \\  & LPBERT & ProteinBERT & 94.83 \(\pm\) 0.16 & 89.69 \(\pm\) 0.30 \\  & DeepCF-PPI & Word2 Vec + AAC + PseAAC + APAAC & **95.6 \(\pm\) 0.57** & **91.4 \(\pm\) 1.13** \\  & & + QSO + DPC & & \\  & LPBERT & ProteinBERT & 95.04 \(\pm\) 0.48 & 90.1 \(\pm\) 0.96 \\ \multirow{3}{*}{Human (10-fold)} & HNSPPI & Seq2 Vec & 94.92 \(\pm\) 0.19 & NA \\  & LPBERT & ProteinBERT & **98.49 \(\pm\) 0.36** & 96.99 \(\pm\) 0.71 \\ \hline \hline \end{tabular}  
* The data of the comparison methods are from the original paper. NA indicates not applicable. The best values are in bold.  
\end{table} 
Table 6: Comparison of LPBERT and different sequence-encoding PPI methods (%). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset** & **Method** & **Accuracy** & **Precision** & **Recall** & **Specificity** & **F1-Score** & **MCC** \\ \hline \multirow{3}{*}{Bio \(H\). _sapiens_} & MPB-PPI & 98.18 \(\pm\) 0.05 & 97.32 \(\pm\) 0.19 & 99.10 \(\pm\) 0.22 & 97.27 \(\pm\) 0.21 & 98.20 \(\pm\) 0.05 & 96.39 \(\pm\) 0.09 \\  & LPBERT & **99.31 \(\pm\) 0.08** & **99.30 \(\pm\) 0.13** & **99.32 \(\pm\) 0.08** & **99.30 \(\pm\) 0.13** & **99.31 \(\pm\) 0.08** & **98.62 \(\pm\) 0.16** \\  & MPB-PPI & 92.85 \(\pm\) 0.31 & 93.95 \(\pm\) 1.25 & 91.51 \(\pm\) 1.43 & 94.17 \(\pm\) 1.31 & 92.69 \(\pm\) 0.39 & 85.77 \(\pm\) 0.63 \\  & LPBERT & **94.83 \(\pm\) 0.16** & **96.06 \(\pm\) 0.54** & **93.50 \(\pm\) 0.64** & **96.17 \(\pm\) 0.45** & **94.76 \(\pm\) 0.20** & **89.69 \(\pm\) 0.30** \\ \multirow{3}{*}{Multi-species} & MPB-PPI & 98.33 & 99.30 & 97.36 & 99.31 & 98.32 & 96.69 \\  & LPBERT & **98.87 \(\pm\) 0.12** & **99.56 \(\pm\) 0.13** & **98.19 \(\pm\) 0.12** & **99.56 \(\pm\) 0.14** & **98.87 \(\pm\) 0.11** & **97.6 \(\pm\) 0.24** \\ \multirow{3}{*}{Pan} & xCAPT5 & **99.77 \(\pm\) 0.02** & **99.75 \(\pm\) 0.03** & **99.75 \(\pm\) 0.02** & **99.80 \(\pm\) 0.02** & **99.62 \(\pm\) 0.06** & **99.55 \(\pm\) 0.03** \\  & LPBERT & 98.73 \(\pm\) 0.09 & 98.65 \(\pm\) 0.32 & 98.51 \(\pm\) 0.14 & 98.92 \(\pm\) 0.25 & 98.58 \(\pm\) 0.11 & 97.44 \(\pm\) 0.18 \\ \hline \hline \end{tabular}  
* Bio: BioGRID. This table reports the mean and variance of the 5-fold cross-validation results. The best values are in bold.  
\end{table} 
Table 7: Performance comparison of LPBERT at various sequence lengths (%). This allowed us to compare the performance differences between the original modules (GCT and LCR) and the conventional architectures. The experimental results of 'rp GCT with Trans' and 'rp LCR with CNNs' in Table 8 show the advantages of the GCT and LCR. To evaluate the contribution of individual components within the GCT and LCR to LP-BERT's performance, we conducted experiments by separately removing the Transformer encoder from the GCT and the BiLSTM from the LCR. While removing either component increased the recall, all other metrics showed degradation. Removing both components simultaneously decreased all performance metrics, which demonstrated the essential roles of both the Transformer encoder and BiLSTM.