# LPBERT: A Protein-Protein Interaction Prediction Method Based on a Pre-Trained Language Model
## 3 Results
### Ablation Experiment
Further experiments that replaced BiLSTM with an alternative RNN variant also resulted in performance degradation, which confirmed that the Transformer encoder-BiLSTM combination provided optimal performance for LPBERT. We also examined performance changes when replacing ProteinBERT with other current pre-trained protein language models. After an in-depth analysis of the experimental results from applying different protein language models to the LPBERT model, we observed that LPBERT showed the best performance when using the ProtT5 model. In contrast, ProteinBERT's performance ranking was relatively low, where it performed only slightly better than the TAPE model. However, considering that ProteinBERT's parameter size is significantly smaller than other language models (except TAPE), its accuracy only lagged behind by about 1%. This result is basically consistent with the conclusion reached by [PERSON] et al. in other benchmark tests: ProteinBERT's performance was comparable with or exceeded that of existing protein language models with larger parameter sizes. The experimental data are presented in Table 9. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Experiment** & **Accuracy** & **Precision** & **Recall** & **Specificity** & **F1-Score** & **MCC** \\ \hline rm GCT & 97.47 \(\pm\) 0.01 & 97.48 \(\pm\) 0.15 & 97.47 \(\pm\) 0.15 & 97.48 \(\pm\) 0.16 & 97.47 \(\pm\) 0.01 & 94.95 \(\pm\) 0.01 \\ rm LCR & 98.73 \(\pm\) 0.10 & 99.15 \(\pm\) 0.06 & 98.32 \(\pm\) 0.23 & 99.15 \(\pm\) 0.06 & 98.73 \(\pm\) 0.10 & 97.47 \(\pm\) 0.19 \\ rp GCT with Trans & 98.73 \(\pm\) 0.01 & 98.91 \(\pm\) 0.16 & 98.55 \(\pm\) 0.17 & 98.91 \(\pm\) 0.16 & 98.73 \(\pm\) 0.01 & 97.46 \(\pm\) 0.02 \\ rp LCR with CNNs & 98.8 \(\pm\) 0.07 & 99.01 \(\pm\) 0.20 & 98.57 \(\pm\) 0.16 & 99.02 \(\pm\) 0.20 & 98.8 \(\pm\) 0.07 & 97.59 \(\pm\) 0.15 \\ GCT (rm Trans) & 98.89 \(\pm\) 0.02 & 99.09 \(\pm\) 0.07 & 98.69 \(\pm\) 0.04 & 99.09 \(\pm\) 0.07 & 98.89 \(\pm\) 0.02 & 97.78 \(\pm\) 0.03 \\ LCR (rm BiL) & 98.88 \(\pm\) 0.10 & 99.05 \(\pm\) 0.21 & **98.71 \(\pm\) 0.02** & 99.05 \(\pm\) 0.21 & 98.88 \(\pm\) 0.10 & 97.75 \(\pm\) 0.20 \\   LRCR (GRU) & 98.85 \(\pm\) 0.16 & 99.14 \(\pm\) 0.20 & 98.56 \(\pm\) 0.22 & 99.15 \(\pm\) 0.19 & 98.85 \(\pm\) 0.17 & 97.71 \(\pm\) 0.33 \\   LRCR (GRU) & 98.74 \(\pm\) 0.09 & 98.80 \(\pm\) 0.16 & 98.68 \(\pm\) 0.07 & 98.80 \(\pm\) 0.16 & 98.74 \(\pm\) 0.09 & 97.48 \(\pm\) 0.17 \\   LRCR (BiGRU) & 98.64 \(\pm\) 0.10 & 99.09 \(\pm\) 0.28 & 98.19 \(\pm\) 0.24 & 99.10 \(\pm\) 0.28 & 98.64 \(\pm\) 0.10 & 97.29 \(\pm\) 0.21 \\   LRCR (LSTM) & 98.63 \(\pm\) 0.12 & 98.76 \(\pm\) 0.37 & 98.49 \(\pm\) 0.17 & 98.77 \(\pm\) 0.38 & 98.63 \(\pm\) 0.12 & 97.25 \(\pm\) 0.24 \\
**LPBERT (Ours)** & **98.93 \(\pm\) 0.13** & **99.23 \(\pm\) 0.19** & 98.62 \(\pm\) 0.13 & **99.23 \(\pm\) 0.19** & **98.93 \(\pm\) 0.13** & **97.85 \(\pm\) 0.26** \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Results of the LPBERT ablation experiment (%). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Method** & **Encoding Parameters** & **Accuracy** & **MCC** \\ \hline \multirow{5}{*}{BioGRID _H. sapiens_ (5-fold)} & LPBERT (with TAPE) & \(\sim\)38 M & 97.66 \(\pm\) 0.23 & 95.32 \(\pm\) 0.47 \\  & LPBERT (with ProtBert) & \(\sim\)420 M & 99.75 \(\pm\) 0.06 & 99.49 \(\pm\) 0.11 \\  & LPBERT (with ESM-2) & \(\sim\)650 M & 99.80 \(\pm\) 0.02 & 99.61 \(\pm\) 0.05 \\  & LPBERT (with ProtT5) & \(\sim\)3 B & **99.82 \(\pm\) 0.02** & **99.65 \(\pm\) 0.04** \\  & LPBERT (Ours) & \(\sim\)16 M & 98.93 \(\pm\) 0.13 & 97.85 \(\pm\) 0.26 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Comparison of different sequence encoding methods (%).