ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction
Authors: Mingyu Jin
Haochen Xue
Zhenting Wang
Boming Kang
Ruosong Ye
Kaixiong Zhou
Mengnan Du
Yongfeng Zhang
Venue: COLM (2024)
Keywords: Protein-Protein-Interaction(PPI) , Large Language Model(LLM), Chain of Thought
TL;DR: Our study introduces ProLLM, a novel Large Language Model fine-tuned with ProCoT prompts reflecting singal passways, showcasing a significant leap in predicting protein-protein interactions with enhanced accuracy and generalizability.

Abstract: The prediction of protein-protein interactions (PPIs) is crucial for understanding biological functions and diseases. Previous machine learning approaches to PPI prediction mainly focus on direct physical interactions, ignoring the broader context of nonphysical connections through intermediate proteins, thus limiting their effectiveness. The emergence of Large Language Models (LLMs) provides a new opportunity for addressing this complex biological challenge. By transforming structured data into natural language prompts, we can map the relationships between proteins into texts. This approach allows LLMs to identify indirect connections between proteins, tracing the path from upstream to downstream. Therefore, we propose a novel framework ProLLM that employs an LLM tailored for PPI for the first time. Specifically, we propose Protein Chain of Thought (ProCoT), which replicates the biological mechanism of signaling pathways as natural language prompts. ProCoT considers a signaling pathway as a protein reasoning process, which starts from upstream proteins and passes through several intermediate proteins to transmit biological signals to downstream proteins. Thus, we can use ProCoT to predict the interaction between upstream proteins and downstream proteins. The training of ProLLM employs the ProCoT format, which enhances the model's understanding of complex biological problems. In addition to ProCoT, this paper also contributes to the exploration of embedding replacement of protein sites in natural language prompts, and instruction fine-tuning in protein knowledge datasets. We demonstrate the efficacy of ProLLM through rigorous validation against benchmark datasets, showing significant improvement over existing methods in terms of prediction accuracy and generalizability. Our results highlight the potential of LLMs to transform the field of PPI, serving as a robust potential tool for various categories of biological and medical research. The code is available at: https://anonymous.4open.science/r/ProLLM-AB04.

Published as a conference paper at COLM 2024
ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction
Mingyu Jiin Rutgers University
Haochen Xue University of Liverpool
Zhenting Wang Rutgers University
Boming Kang Peking University
Ruosong Ye Rutgers University
Kaixiong Zhou Massachusetts Institute of Technology
Mengnan Du New Jersey Institute of Technology
Yongfeng Zhang* Rutgers University
Abstract
The prediction of protein-protein interactions (PPIs) is crucial for under- standing biological functions and diseases. Previous machine learning approaches to PPI prediction mainly focus on direct physical interactions, ignoring the broader context of nonphysical connections through interme- diate proteins, thus limiting their effectiveness. The emergence of Large Language Models (LLMs) provides new opportunity for addressing this complex biological challenge. By transforming structured data into natural language prompts, we map the relationships between proteins into texts. This approach allows LLMs to identify indirect connections between proteins, tracing the path from upstream to downstream. Therefore, we pro- pose a novel framework ProLLM that employs an LLM tailored for PPI for the first time. Specifically, we propose Protein Chain of Thought (ProCoT), which replicates the biological mechanism of signaling pathways as natural language prompts. ProCo'T considers signaling pathway as protein reasoning process, which starts from upstream proteins and passes through several intermediate proteins transmit biological signals to downstream proteins. Thus, we can use ProCoT to predict the interaction between up- stream proteins and downstream proteins. The training of ProLLM employs the ProCoT format, which enhances the model's understanding of complex biological problems. In addition ProCoT, this paper also contributes to the exploration of embedding replacement of protein sites in natural lan- guage prompts, and instruction fine-tuning in protein knowledge datasets. We demonstrate the efficacy of ProLLM through rigorous validation against benchmark datasets, showing significant improvement over existing meth- ods in terms of prediction accuracy and generalizability. Our results high- light the potential of LLMs to transform the field of PPI, serving as robust potential tool for various categories of biological and medical research. The code is available at: https: //g: thub. cwm com/Mingy.J666/ (6tpg//o666PPPrLL..
1 Introduction
Protein-protein interactions (PPIs) play an essential role in various biological processes of all living organisms, which are crucial for biomedical, genetic, and pharmaceutical research. Thus, numerous experimental methods have been proposed for PPI detection, such as yeast two-hybrid (Ito et al., 2001) and quantitative proteomics methods (Rotilio et al., 2012).
'Equal Contribution.
Author Emails: mingyu, mingyy.jin@rutgers.cm edu, hicaca945@gmaill.com zhenting wanggrutererrrreeee.uu kangbm@@hs.pku.edu.cr guosong-ye@rutgen
kr344mit.eduu mengnan.duunjjt.edu.
yongteng/hang@ertgers...

---

Published as a conference paper at COLM 2024
Input question
Answer
Impact
Interactions type prediction
Interactions type prediction
activate
bind
Answer: The neletionship between
inhibit
What relationship
and
is ac activation.
between
and
ProLLM
PPI
ProLLM (Based Flan-T5-large)
Biological Research
Prediction
Figure 1: Illustration of the ProLLM Framework. We fine-tuning ProLLM under Human, SHS27K, SHS148K, and STRING datasets, enabling it to solve various PPI related tasks with the structure information purely described by natural language.
However, wet-lab methods for PPI prediction are often time-consuming and labor-intensive, highlighting the need for more precise and efficient computational tools.
In recent years, computational biology has developed rapidly. Methods such as the Convo- lutional Neural Network (CNN) and Graph Neural Network (GNN) have become powerful tools for studying protein interaction. CNN-based approaches like TAG-PPI (Song et al., 2022) typically use pre-trained embedding models to convert protein sequences into numeri- cal vector representations, and then employ one dimensional convolutional neural networks to extract features from the vectors for subsequent PPI tasks.
Although CNN methods have shown some effectiveness in PPI prediction, they still have limitations due to their fixed receptive fields and the lack of well-defined spatial relation- ships in protein sequences, which limit the accuracy and interpretability of the predictions. GNN-based methods such as GNN-PPI (Lv et al., 2021) treat proteins as nodes and their relationships as edges, constructing network composed of proteins, which better cap- tures protein relationships and interactions, and outperforms CNNs in predicting protein interactions. However, while GNNs can effectively extract network structural information, they neglected the non-physical connections between two proteins without direct physical interactions, resulting in the worse performance in learning protein chains than transformer- based models (Zhou et al., 2020). Furthermore, GNNs cannot fully capture the relationships and dynamic changes in the signal passing process of living organisms, restricting their performance for PPI prediction (Zhou et al., 2023).
Following the GNN and CNN methods, Large Language Models have also been applied to this PPI area, such as ProBert (Elnaggar et al., 2021a) and ProteinLM (Xiao et al., 2021). As long as these models can obtain a protein representation, we can use the direct cosine similarity of the representation methods still cannot capture the chain relationships between proteins, such as the signaling pathways. Besides, previous literature only used LLMs using LLM as a link predictor has shown that it can better capture relational information between nodes in knowledge graph tasks and its performance surpasses traditional GNN baselines (Ye et al., 2024; Zhuo et al., 2024; Shu et al., 2024). Therefore, it is promising to introduce LLM for protein-protein interaction (PPI) tasks, since the most important biological signal for PPI tasks is the chain relationships of proteins, i.e., the signaling pathways.
train an MLP to perform PPI prediction. However, these
feature extractor. Recently,
To bridge the gap, we propose ProLLM, with its key ideas illustrated in Figure 1, and the difference between the existing method and our ProLLM shown in Figure 2. Existing methods only focus on the single protein-protein interaction, overlooking the application of protein links to predict PPI in signaling pathways. Instead, we employ a large language model to learn the law of signal pathways and adapt the LLM to directly predict the type of interaction between proteins.
The signaling pathway addresses the traditional method's ignorance of global, non-physical connections between proteins. Signaling pathways typically start with an upstream protein that sends biological signal through several intermediates to downstream protein, hence
2

---

Published as a conference paper at COLM 2024
Traditional Method
Physical
Node Embeddding1
Interaction
Upstream
Downstream
Node Embeddding2
Protein
Protein
PPI Prediction
ProLLM
Signaling Pathway- Connection
ProCoT
Large Language Model
Activate
Bind
Protein_A
Protein_B
Protein_X
PPI Prediction
Figure/ 2: The difference between the existing method and our method in PPI prediction. Existing method focus on the property of upstream protein and downstream protein, our method focus on signaling pathway-like connection.
requiring consideration of the cumulative effect of multiple protein interactions. This series of interactions form sequential chains. Therefore, we propose Protein Chain of Thought (ProCoT) to overcome the limitation in understanding signaling pathways and protein functions. ProCoT is a data format that simulates the signal transduction process using thought-chain approach, thereby enabling the prediction of protein interactions in signaling pathway problems. CoT can express the thinking process step by step to form a reasoning chain (Jin et al., 2024b), while ProCoT extends this principle further into the protein-related domain to simulate protein signaling pathways, giving LLMs a deeper insight into protein.
Additionally, our approach addresses the issue of poor protein comprehension in LLMs by replacing the standard language model embedding with embedding infused with protein information. When we process the protein name in the prompt, we replace its original embedding by Prot'rans (Elnaggar et al., 2021b), because its embedding contains protein's structual information. We also perform the instruction fine-tuning on the protein knowledge dataset to infuse protein domain knowledge into LLMs. Following these steps, the LLM acquires robust ability to reason about the direct relationships between proteins, as demonstrated in Figure 1. It can provide answers to questions about protein relationships, which play a significant role in biological research.
Our contributions are summarized as follows:
To the best of our knowledge, we are the first to explore the PPI prediction problem as natural language processing problem. The proposed Protein Chain of Thought (ProCo'T) is a novel method for understanding complex multi-step protein interactions, i.e., those present in signaling pathways.
We propose embedding replacement and instruction fine-tuning our model to enhance its ability to understand and reason about proteins. This also provides our model with rich background knowledge of protein sequences and protein interactions before training.
Experiments on four widely used PPI datasets (i.e., Human, SHS27K, SHS148K, and STRING) demonstrate that ProLLM outperforms graph-based methods such as GNN-PPT and SemiGNN-PPL It also has better performance than L.MM-based methods like Instruct- GLM. The micro-F1 scores on these datasets are 91.05, 78.09, 87 .6, 89.21, respectively.
2 Related Work
Protein-protein Iteractions Protein-protein interactions (PPIs) are indispensable for all living organisms, and so many efforts have been made for PPI detection up to now. Yeast two-hybrid (Y2H) assays (Brückner et al., 2009), synthetic lethality (O'Neil et al., 2017), quantitative proteomics (Wilm, 2009), and mass spectrometry (Mann et al., 2001), are widely used for identifying PPIs. To be specific, Y2H assays explore binary PPIs in living cells, offer-
3

---

Published as a conference paper at COLM 2024
ing insight into protein functions and regulations, although labor-intensive and limited in genomic coverage. Synthetic lethality identifies essential gene pair interactions by revealing compensatory relationships when both are inactivated. Meanwhile, quantitative proteomics, especially through mass spectrometry, illuminates the dynamic nature of the interactome under various conditions. Although informative, these methods require significant labor, time, and resources.
Traditional Machine Learning Models for PPI Prediction In the realm of machine learn- ing, sequence- approaches include Shen's SVM method (Shen et al., 2007), which uses a 3-mer count vector from protein sequences as features and groups 20 amino acids into seven classes to handle synonymous mutations and reduce feature space dimensionality. SVM-based methods (Guo et al., 2008) and the ensemble model PCA-EELM (Principal Component Analysis-Ensembll Extreme Learning Machine) (You al., 2013) utilize various types of protein sequence information for PPI prediction. In the domain of deep learning, DeepPPI (Du et al., 2017) extracts a multitude of features from protein sequences and em- ploys dual deep neural network structure for feature fusion and prediction. Sun et al. (Sun et al., 2017) introduced a PPI predictor based on a stacked autoencoder, emphasizing the importance of sample balance. DPPI (Hashemifar et al., 2018) and TAGPPI (Song et al., 2022) further extend the application of convolutional neural networks and integrate text convolution networks with graph representation learning to enhance the accuracy of PPI pre- dictions. GNNs have significantly advanced PPI predictions, improving our understanding of biological mechanisms. GNN-PPI (Lv et al., 2021) enhances inter- nnte--novel-protiin prediction accuracy by utilizing protein relationships and a new evaluation approach. PT-GNN (Long et al., 2022) integrates diverse data for link prediction, learning node features from sequence and structure. DeepRank-GNN (Réau et al., 2023) offers a modular, Python-packe packaged framework for GNN-based interaction pattern predictions. HIGH-PPI (Gao al., 2023) introduces a hierarchical graph learning model for effective PPI prediction and molecular detail extrapolation. Geometric GNNs excel in modeling spatial intricacies, enhancing biomolecule prediction accuracy. Geo-PPI (Liu et al., 2021) utilizes self-supervised learning for geometric protein structure representations, excelling in detailing protein interactions. mmCSM-PPI (Rodrigues et al., 2021) captures multifaceted features for mutation impact pre- dictions on protein interactions. MAPE-PPI (Wu et al., 2024) defines the microenvironment of amino acid residues in proteins and encodes it into discrete codes, which can capture the connections between different microenvironments, enhancing the prediction accuracy of PPI.
Large Language Model for PPI prediction Recent advances in large language models, such as BERT (Devlin et al., 2018), GPT (Peng et al., 2023), LLaMA (Touvron et al., 2023), and T5 (Raffel et al., 2020), have significantly advanced the field of Natural Language Processing (NLP) to new heights. These models, having been trained on extensive textual corpora, exhibit exceptional capabilities across diverse range of NLP applications (Shengyuan et al., 2024; Jin et al., 2024a; Fan et al., 2024; Hua et al., 2024). Inspired by LLMs, Protein Large Language Models(PLMs) pre-trained on large-scale protein sequences have emerged, such as ESM (Hsu et al., 2022), ProtTrans (Elnaggar et al., 2021b) and ProteinBert (Elnaggar et al., 2021b). PLMs provide a better representation of protein sequences by converting the protein sequences into the form of high-dimenssional numerical vectors, known as embedding. With the protein sequences captured by the PLMs, the performances on diverse downstream tasks, such as structure prediction (Lin et al., 2023), subcellular localization prediction (Thumuluri et al., 2022), single peptide prediction (Teufel et al., 2022) and N- linked glycosylation sites prediction (Hou et al., 2023), have been transformed. It can be expected that PLMs will assist in PPI prediction tasks. ProtLLM (Zhuo et al., 2024) utilizes dynamic protein mounting mechanism, protein-as-word language modeling approach, and the InterPT dataset for pre-training, enabling it to handle complex inputs and achieve superior performance on various proteins-related tasks. However, ProtLLM is used for general protein tasks, and it is not used for PPI tasks exactly. The methodology of training these LLMs to convert text inputs to desired text outputs positions them as particularly ad advantageous for tasks such generative link prediction (Ye et al., 2024; Shu et al., 2024). In such tasks, the model is tasked with inferring and generating the relationship between two entities based on provided textual cues. Moreover, the extensive pre-training phase
4

---

Published as a conference paper at COLM 2024
Sector 1: Protein data natural language in ProCoT format
Sector 2-Replace with Prot Troft Embedding
Renerare protzin sasbedding
keplane vesture
Froteln date
Question:
Flan- T5- 15-large
acthoet
+Protein_A> receptor protein,
activate <Protccn_B- <Protein_ agraiin preteis, <Protein_C>
<Protein_Co
protein.
catolyas
Prof
step by step.
-Promenn relerorr with dPrethin -Pratiin
?
<Protein
<Protuin Bi hor rulotion_ 4 with (Prensing C Prratien
Answer:
Sy
The relationship octriction.
Sector 3: Instruction fine-tuning on ProLLM
Sector 4: Train on ProCoT format dataset
sscuance Based on the protem
informalton
Dacnent
Quastion: To low the inctruction, coower rustisn <The gresein esqueeces, the sons of protinn is slrreter, sroich function
Translate dataset
Pepur queeries
musiel
output
fanc anniiinnann and brolggo
qaai prefe'n
dretell As octs
The nelatiio
protien seguence:
protcins a Ammotis.
ors scraing protein. drotein C oete efrecter gesher, satalysa
chroten -
activation.
Arsweri The unsver Output..
osctore-Ce loon cctiitty App and boloo cal procensa of pronena)
lete thikk it stop by step betwver Af dhroten_Drt
Real aroter
Outputt Upan hupectics of the previded that
The reist in
Negative Leg Likelihadd Loss
call proterrs
Figure 3: The process of ProLLM. Sector 1: Transfer the original protein data into ProCoT
format of natural language that indicates the signaling pathways between proteins; Sector 2: Replace protein information embeddings with natural language embeddings to enhance the model's understanding of proteins; Sector 3: Inject knowledge about protein function; Sector 4: Fine-tuning on the ProCoT format dataset in Sector 1.
enables LLMs to exhibit a remarkable capacity for generalization. This capacity allows them to effectively tackle and respond to tasks or prompts that were not explicitly covered during their initial training (Wei et al., 2022). In addition to the outlined capabilities, the inherent flexibility and generalization potential of LLMs suggests that their applicability extends well beyond the conventional boundaries of NLP tasks (Yang et al., 2023). Specifically, their proficiency in generalizing from expansive pre-training sessions paves the way for their application in fields like bioinformatics and complex network analysis.
3 Proposed ProLLM Framework
In this section, we will introduce the implementation details of ProLLM, framework designed to transform protein interaction data into ProCoT format natural language de- scriptions to simulate protein signaling pathways. By translating the structure relationships of proteins into natural language, we effectively transform protein interaction problems into natural language processing (NLP) tasks. To enhance the understanding of proteins by ProLLM, we directly integrate the protein vectors generated from ProtTrans (Elnaggar et al., 2021b) into our model to replace the original word embedding in the protein name's place. This approach allows our model to understand and utilize the biological attributes of proteins when predicting protein interactions. The ProLLM process is shown in Figure 3.
3.1 ProCoT
3.1.1 Protein Data to Natural Language in CoT Format
Transformation of the original protein data into ProCoT natural language critical step, this kind of natural language type that represents the relationships of proteins is shown as Definition 1.
Definition 1 (Protein Interaction) Given the set P representing all proteins under consideration in the study, we define the interaction between any two proteins Pa, Pm E P with the specific type interaction R, interaction set R. This relationship is encapsulated as tuple (pu, Pm, R₂).
5

---

Published as a conference paper at COLM 2024
Protein chain of thought (ProCo T)
catalysis
activate
bind
react
activate
catalysis
ProCoT T in fine-tuning
Question
Enter model
Model's output
What is the relationship between <Protein and <Protein_6>?
The relation activation.
Compare groundtruth
Figure The fine-tuning process of ProCoT. Within the first dashed box, solid lines between proteins represent
the signaling pathway, and the dashed lines connecting the head and tail proteins indicate the masked interaction. Our model will predict the type of masked interaction.
Definition (Protein Signaling Pathway) In cellular signal transduction, receptor protein Re is the beginning of the signal pathway, and it can interact with downstream signaling proteins Sp. Sp responsible for continuing to fransmit signals effector protein Ep inside the cell. After the Ep response to the signal, it will trigger specific biological effects and activate secondary messengers M. M may the end of this signal level, or it may open a second signal hierarchy. If in the second signal level, the M will replace the 'Rc in the initial signal hierarchy the starting point. The signal will continue to propagate from Rcc through new signaling proteins, and effector proteins, and then generate secondary messengers. After this stage, the propagation of the signal repeats the structure of the second signal level until the signal is interrupted.
We aim to improve the model's understanding of biological signaling pathways, enhancing its ability to learn and reason interactions within complex signal networks. We create prompts in the ProCoT (Protein Chain of Thought) format to incrementally decompose the signal transduction process like Definition 2, simulating real pathways of signal propagation. Specifically, we clarify the structure knowledge within the signaling pathway, such as signaling proteins and effector proteins. We then design rules for signal transmission across different levels to simulate the iterative process of signal transduction in proteins. Finally, we also design signal interruptions simulate the continuity of protein transmission in real life. We use hard code to convert protein interaction information as Definition 1 in dataset into ProCoT format natural language to imitate protein signaling pathways as Definition Figure 7 in the appendix is an example of ProCoT. The answer to this is "The relationship is activation".
3.1.2 Training on ProCot Format Dataset
In biology, co-expression and co-localization refer to the phenomenon in which proteins that are often expressed or located together in the cell tend to participate in the same or interconnected biological processes (Zhang al., 2019). Thus, biologists frequently use known protein information to infer unknown protein interactions. Based on the principles of co-expression and co-localization After constructing our ProCoT training data using DFS, we will obtain circular protein interaction as Figure 4. We mask the relationship between the initial and final proteins in the signaling pathway and let our model predict this relationship.
biology, we have formulated our training strategy.
3.1.3 Why ProCot Works: Biological Intuition behind ProCot
In this section, we will explore ProCoT through the lenses of biology and AL. (1) Simulating Biological Signaling Pathways: The biological signaling pathway is a series of ordered, rule-based interaction processes. During the process, the output of each step serves as the input for the next step, forming highly organized information transmission chain. This pattern is consistent with the methods of handling serialized information in natural language processing (NLP). Additionally, Flan-T5's self-attention blocks can capture the indirect relationships between proteins that are distant and do not have direct interactions.

---

Published as a conference paper at COLM 2024
It is crucial for understanding biological signal transduction processes that involve multiple steps and complex intermediate links. (2) Signal Interruption Mechanism: This mech- anism aims to mimic protein adaptability since the signaling between proteins needs to be constantly interrupted to ensure the cell's accurate response to external changes. This mechanism designed satisfy the complex feedback mechanisms and regulatory networks within the cell. Overall, our ProCoT design follows biological principles.
3.2 Enhancing Protein Sequence and Function Comprehension
In our methodology, we enhance the model's understanding of protein sequence by replac- ing the 15 embedding with the Pro'Trans embedding vectors and also perform instruction fine-tuning enable the model to learn protein functions.
3.2.1 Embedding replacement
We use a new embedding mechanism facilitated by the ProTrans (Elnaggar et al., 2021a) model to perform embedding replacement. ProTrans is a large-scale protein model capa- ble of transforming protein sequences into embedding vectors with the biophysical and structural features of proteins.
(Embedding Replacement) Given the protein id Pit, we can query its protein sequence Pang by the professional bioinformatics tool Ensembl BioMart. Pseg will be the input of ProTrans and ProTrans will output a 1024-dimensional vector Pemb-
Definition 3
We add all protein IDs in the protein dataset and their corresponding embeddings the T5 vocabulary to solve the OOV (out of vocabulary) problem: The T5 vocabulary does not contain words for protein ID numbers. Because Prot'rrans is trained based on T5-large, the process of adding to the vocabulary does not involve any change in dimensions, and there is no information loss. Additionally, this approach can also avoid the issue where protein ID is tokenized into multiple tokens, causing the model to fail to understand it.
The replaced embedding vectors from ProtTrans can add a lot of prior knowledge about the intrinsic patterns and biophysical features of proteins to our model, better applying it to subsequent protein interaction prediction tasks.
3.2.2 Instruction Finetuning
Mol-Instructions (Mol) (Fang et al., 2023) dataset is applied to conduct instruction fine- tuning on our model. Mol is comprehensive instruction dataset designed for the field of biomolecules that aims to address the limitations of LLMs in biomolecular research. We use Mol to teach our model with knowledge of protein functions.
Definition 4 (Description of Mol) The content of the Mol dataset can be defined as follows: instruc- tional text related to L.IMs queries, an input amino acid sequence S that includes essential protein information, and metadata M that sheds light on vital details like the protein's ss subcellular location, its primary function, and its participation in biological processes, followed O that serves the response to I.
followed by corresponding output
To facilitate instruction-based fine-tuning, we convert these objects into prompt-answer pairs (P, A). This is shown as Figure 8 in the appendix. We convert the entire Mol dataset into a prompt-answe tuning of our (Mdel(ProLLM). This part can enhance the model's predictive performance on pretein-related tasks.
format and using these prompt-answer pairs for instruction fine-
Experiments
In this section, we present the experimental results to answer the following research ques- tions (RQs).
7

---

Published as a conference paper at COLM 2024
RQ1 How does the performance of the proposed ProLLM framework compare to other baselines in terms of PPI prediction accuracy and generalizability?
RQ2 How do different LLM backbones (e.g., Flan-T5-base and LLaMA-7b) affect the performance of the proposed ProLLM framework?
RQ3 What are the contributions of each component the PPI prediction performance of the ProLLM framework?
4.1 Experimental Settings
Datasets: We undertake comprehensive evaluations on trio of public Protein-Protein Interaction (PPI) datasets: Human (Song et al., 2022), STRING (Szklarczyk et al., 2019), SHS27k, and SHS148k (Chen et al., 2019). In these four datasets, we employ DFS dataset partitioning techniques. By prioritizing depth, DFS can effectively capture the step-by-step signal transmission and the hierarchical signal level of protein signaling pathways. Each dataset is split into training, validation, and testing sets, maintaining proportion of 70%, 10,, and 20% respectively.
Baselines: Earlier studies on predicting PPI are not pre-trained, they do not have prior knowledge about the proteins. Hence, we choose SVM (Romero-Molina et al., 2019), DPPI (Hashemifar et al., 2018), DNN-PPI (Li et al., 2018), PIPR (Chen et al., 2019), GNN- PPI (Lv et al., 2021) and SemiGNN-PPI (Zhao et al., 2023) as the baseline without pre- training. Furthermore, we explore how protein pre-training can influence the PPI, we include ProBERT (Elnaggar et al., 2021a), SM-1b (Rives et al., 2021), GearNet-Edge (Zhang et al., 2022), and KeAP (Zhou et al., 2022) as pre-trained baselines. MAPE-PPI (Wu al., 2024) and InstrucGLM (Ye et al., 2024) have two versions: one with pre-training and one without pre-training.
evalu- ation metric because the PPI dataset exhibits class imbalance, making the F1 score a very relevant reference. Note that micro-Fl is widely used in the protein-pant -pretenn interaction task (Tran & Kavuluru, 2018). Each dataset follows a 70% training, 10% validation, and 20% testing data split. Subsequently, we will choose different random seeds for training and testing, conducting a total of 10 tests. The mean micro-Fl score across these trials will serve as the definitive measure of model performance, with the accompanying standard deviation reflecting variability in different experimental runs.
Evaluation Metrics: We selected the micro-Fl score (Harbecke et al., 2022)
4.2 Comparative Experiment (RQ1)
We compare the performance of ProLLM with other baselines (w/ and w/o additional pre-training data) on four datasets in Table 1. Note that here we use Flan-T5-large (Raffel et al., 2020) as the backbone model. Based on the results, we can make three important observations: (1) Our method outperforms other baselines without pre-training. Although InstructGLM also LLM based, it lags behind ProLLM;; (2) As for the models pre-trained on protein dataset, they cannot achieve the performance of ProLLM without prior knowledge; (3) ProLLM outperforms GearNet-Edge, KeAP, and MAPE-PPI, although they utilize significantly larger dataset comprised of structural and knowledge graph data for pre- training than the Mol dataset.
4.3 Influence of Different Backbones (RQ2)
We choose Flan-T5-base, Flan-T5-large, Flan- (Raffel et al., 2020) and LLaMA-7b (Tou- vron et al., 2023) models as the backbone for ProLLM. We report the micro-F1 performance comparison as in Table We should replace the embedding in Flan-T5 with ProtTraas and ProtTrans is pre-trained model based on Flan-T5-Large. Therefore, the embedding generated by ProtTrans will match Flan-T5-large better. Additionally, despite having more model parameters, LLaMA-v1-7b exhibits worse lower micro-Fl scores in four datasets compared to lighter models: Flan-T5-base Flan- Ta-laee and Flan-T5-XL. Furthermore, the greater standard deviation of I.LMMAv1-77b highlights its instability.
8

---

Published as a conference paper at COLM 2024
Method
Pre-training Dataset
Human
SHS27k
SHS148k
STRING
SVM
61 61.28:1.25
53.07=5.16
58.59:0.077
64.59:00005
DPPI
54.19=0.78
46.12=3.02
52.0811118
6688210:22
DNN-PPI
61.72=1.30
54.34=1.30
58,42+2.05
64 44.4400.95
PIPR
62.72=0.50
57.80=3.24
63.98:10.76
07.45:0..30
GNN-PPI
76.61=1.38
66.52-5.26
1.54
84,28
SemiGNN-PPI
80.79-1.44
69.25-3.91
77.62-41..08
04.85+0.65
MAPE-PPI
83.14=1.47
32.04-3.44
80.45++:112
06.5
InstrucIGLM
81.35=2.04
70.01=3.75
75,35:::998
84.1511.85
ProlLMM(FlanT55tarrg)
87.12
ProBERT
BFD
79.38=0.76
68.85=3.18
74.76:11.21
83.82:00.49 0.49
EMS-Ib
UniRef50
81.48=1002
70.69=3440
79 19.64+11.98
06.21+0..66
KeAP
ProteinKG25
82.31=0,71
72 2.99
80.20=:1.26
86.58:00.11
GearNettEdge
AlphaFoldDD
8288=1102
72.06-3.56
79.8411..5
85,9611.01
MAPE-PPi
CATH4.2
83.64 1.22
73.21 2.97
81.78 1-24
67.23+-0.35
InstructGLM
dataset
85.71-2.01
75.64-3.49
13.41+1.788
15.25++1.72
ProLLM(Flan-TS-argge
Mol dataset
91.05=1.63
78.09=3.24
87.66:21.68
89.21=:145
Table The micro-F1 of different methods (w/o and w/ additional pre-training data) on different datasets, where bold and underline denote the best and second best metrics, respectively. Higher micro-F1 denotes better performance.
Method
Pre-training Dataset
Human
SHS27k
SHS148k
STRING
M- T5-base
82662-2.01
71.67-4.38
78.19-1 1.82
80.63+ 1.97
ProLLM-Flan-TT-lan MoLLLM-Flan-TT-XL
87.32-1.93
7..13-3.76
85.13-1.86
87.12-1.68
84.32-2.65
73.28=3.96
81.73=2.21
82994=193
ProLLM-LLaMA-x1-70
70.33-6.52
77.08-3.59
79.41:287
HALLLM-Plan-TT-baee
Mol dataset
87.92=2.07
74.28=4.09
85.07=1.87
87.11=1888
ProLLM--lan-II--arge
Mol dataset
91.05=1.33
78.09=3.24
87.66=1.93
89.21=1145
ProLLM-FFan-TT-XL.
Mol dataset
89 16=2411
25.96-3.38
86.13=1.9 97
87
ProLLM-LLaMA-<1-7
Mol dataset
87.08=4.72
74.87
84.61=3.33
86.71=2.61
Table 2: The micro-F1 score of ProLLM on different backbones. Higher micro-F1 denotes bet- ter performance. Bold and underline denote the best and second-best metrics, respectively.
4.4 Ablation Study (RQ3)
In our ablation study (Table 3), we evaluated the impact of different configurations on PPI prediction. The configurations are as follows: ProLLM w/o ProCoT: This setup shuffles ProCoT data, disrupting signaling pathways to mimic the model's performance without pathway understanding. It limits the model's ability to learn from signaling sequences. ProLLM relies on memorizing fixed relations instead of reasoning through intermediate protein relationships; ProLLM w/o Embedding Replacement: It will compare the model with replaced embeddings to the model without expended vocabulary, which evaluates the effect of protein-specific embedding features; ProLLM w/o Instruction Fine-tuning: This setup examines the model's capability to predict protein-protein interactions without the application of instruction fine-tuning on Mol dataset; ProLLM w/o Embedding and Instruction Fine-tuning: This configuration tests the model's performance without utilizing both protein-specific embedding features and instruction fine-tuning on Mol dataset.
In our ablation study, we have identified that ProCoT has the most significant impact on the performance of ProLLM. Our experiments revealed that introducing ProCoT led to substantial improvements in the performance of the model. Additionally, we explored other techniques such as embedding replacement and instruction fine-tuning on the Mol dataset. While these approaches did show some positive effects the model's performance, their impact was found to be comparatively smaller when compared the influence ProCoT.
5 Conclusions and Future Work
In this paper, we propose ProLLM, novel framework that leverages LLMs for protein- protein interaction prediction by representing protein data in natural language formats. Our key contributions include: 1) ProCoT (Protein Chain of Thought) to convert multi-step protein signaling pathways to natural language prompts, and the design of ProCoT can reflect the actual protein signaling passing within biological organism. Additionally,

---

Published as a conference paper at COLM 2024
ProCoT
Embedding Replacement
Instruction Fine-tuning
Dataset
Human
SHS227
SHS148K
STRING
13.871125
70.83:11.54
79.12:12.64
62.68-1.336
87.32+1.93
74.26+3.53
85.13++1.66
87.12+
18.53+1445
73.594:2.29
85.944:3.39
67642411
78.32+22.5
61.98:42.06
74.10=1.27
77,85+1:.3 1.44
91.03:11.63
78.09:13.24
67.6411:.93
89.28.1:145
Table 3: Ablation study. The metric here is micro-F1. Where bold denote the best metrics. Higher micro-F1 denotes better performance.
the format of ProCoT is sequential, which is type of information that LLMs are good at processing, 2) Integration of protein-specific embeddings from Prot'raas, and 3) Instruction fine-tuning on protein knowledge datasets. Through extensive experiments on four PPI datasets, ProLLM significantly outperformed existing graph-based and language model methods in prediction accuracy and generalizability. By unifying language models with structured biological data, our work opens up new possibilities for driving discoveries in computational biology, drug discovery, and broader scientific domains.
5.1
Acknowledgement
We thank Wenyue Hua, Kai Mei and Taowen Wang for their valuable discussions and suggestions during the project.
References
Anna Brückner, Cécile Polge, Nicolas Lentze, Daniel Auerbach, and Uwe Schlattner. Yeast two-hybrid, a powerful tool for systems biology. International journal of molecular sciences, 10(6))2763-2777 -7888, 2009.
Muhao Chen, Chelsea J-T Ju, Guangyu Zhou, Xuelu Chen, Tianran Zhang, Kai-Wei Chang, Carlo Zaniolo, and Wei Wang. Multifaceted cootein-protein interaction prediction based siamese residual renn. Bioinformatics, 35(14):1305-13 2019.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04800, 2018
Xiuquan Du, Shiwei Sun, Changlin Hu, Yu Yao, Yuanting Yan, and Yanping Zhang, Deepppi: boosting prediction of protein-protenn interactions with deep neural networks. Journal of chemical information and modeling, 57(6) 2017.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: Toward understanding the language of life through self-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 44(10):7112-1127, 2021a.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Wang Yu, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, Deb- sindhu Bhowmik, and Burkhard Rost. Prottrans: Towards cracking the language of lifes code through self-supervised deep learning and high performance comput- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021b. doi: 1T.1109/TAAMM.200133055555
Lizhou Fan, Wenyue Hua, Xiang Li, Kaijie Zhu, Mingyu Jin, Lingyao Li, Haoyang Ling, Jinkui Chi, Jindong Wang, Xin Ma, et al. Nphardeval4v: A dynamic reasoning benchmark of multimodal large language models. arXiv preprint ar Xiv::2403..0177 2024.
Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: large-scale biomolecular instruction dataset for large language models. arXiv preprint arXio:2306.08018 2023.
10

---

Published as a conference paper at COLM 2024
Ziqi Gao, Chenran Jiang, Jiawen Zhang, Xiaosen Jiang, Lanqing Li, Peilin Zhao, Huanming Yang, Yong Huang, and Jia Li. Hierarchical graph learning for protein-protene interaction. Nature Communications, 14(1):1093, 2023.
Yanzhi Guo, Lezheng Yu, Zhining Wen, and Menglong Li. Using support vector machine combined with auto covariance to predict protein-protest interactions from protein sequences. Nucleic acids research, 36(9):3025-3030, 2008.
David Harbecke, Yuxuan Chen, Leonhard Hennig, and Christoph Alt. Why only micro-f1? class weighting of measures for relation classification. arXiv preprint ar Xfx::2205.094600 2022.
Somaye Hashemifar, Behnam Neyshabur, Aly A Khan, and Jinbo Xu. Predicting protein- protein interactions through sequence-based deep learning. Bioinformatics, 34((7):8802- i810, 2018.
Xiaoyang Hou, Yu Wang, Dongbo Bu, Yaojun Wang, and Shiwei Sun. Emngly: predicting linked glycosylation sites using the language models for feature extraction. Bioinformatics, 39(11):btad650, 2023.
Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. ICML, 2022. doi: 2022/04/10/2022
URL httpp://wwwt biorxiv. wwgg/onneen//errly/
$4.10.4877773
Wenyue Hua, Kaijie Zhu, Lingyao Li, Lizhou Fan, Shuhang Lin, Mingyu Jin, Haochen Xue, Zelong Li, JinDong Wang, and Yongfeng Zhang. Disentangling logic: The role of context in large language model reasoning capabilities. arXiv preprint 7XXo:2406.02781 2024.
Takashi Ito, Tomoko Chiba, Ritsuko Ozawa, Mikio Yoshida, Masahira Hattori, and Yoshiyuki Sakaki. A comprehensive two-hybrid analysis to explore the yeast protein interactome. Proceedings of the National Academy of Sciences, 98(8):4569-455 2001.
Mingyu Jin, Qinkai Yu, Chong Zhang, Dong Shu, Suiyuan Zhu, Mengnan Du, Yongfeng Zhang, and Yanda Meng. Health-llm: Personalized retrieval-augmented disease predic- tion model. Xiv preprint ar 400::2200.00746, 2024a.
Mingyu Jin, Qinkai Yu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, Meng- nan Du, et al. The impact of reasoning step length on large language models. arXiv preprint wxXx::2401049922 2024b.
Hang Li, Xiu-Jun Gong, Hua Yu, and Chang Zhou. Deep neural network based predictions of protein interactions using primary sequences. Molecules, 23(8):1923, 2018.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniy Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with language model. Science, 379(6637):1123--111 2023.
Xianggen Liu, Yunan Luo, Pengyong Li, Sen Song, and Jian Peng. Deep geometric repre- sentations for modeling effects of mutations on protein-protein binding affinity. PLoS computational biology, 17(8):e1009284, 2021.
Yahui Long, Min Wu, Yong Liu, Yuan Fang, Chee Keong Kwoh, Jinmiao Chen, Jiawei Luo, and Xiaoli Li. Pre-training graph neural networks for link prediction in biomedical networks. Bioinformatics, 38(8):2254 2266)222,,, 2022.
Guofeng Lv, Zhiqiang Hu, Yanguang Bi, and Shaoting Zhang. Learning unknown from correlations: graph neural network for inter-novel-protein interaction prediction. Xiv preprint (rXi::2005006700 2021.
Matthias Mann, Ronald C Hendrickson, and Akhilesh Pandey. Analysis of proteins and proteomes by mass spectrometry. Annual review of biochemistry, 70(1)4437-473, 2001.
11

---

Published as a conference paper at COLM 2024
Nigel J O'Neil, Melanie L Bailey, and Philip Hieter. Synthetic lethality and cancer. Nature Reviews Genefics, 18(10):613-6)) (2006613363,,2000 2017.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arviv preprint arXtv:2304.03277 2023.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485-5551, 2020.
Manon Réau, Nicolas Renaud, Li C Xue, and Alexandre MJ] Bonvin. Deeprank-gnn: graph neural network framework to learn patterns in protein-protein interfaces. Bioinformatics, 39(1):btac759, 2023.
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15):2016239111 2021.
Carlos HM Rodrigues, Douglas EV Pires, and David B Ascher. mmcsm-ppar predicting the effects of multiple point mutations protein-protent interactions. Nucleic Acids Research, 49(W1):W417-W424, 2021.
Sandra Romero-Molina, Yasser B Ruiz-Blanco, Mirja Harms, Jan Münch, and Elsa Sanchez- Garcia. Ppi-detect: A support vector machine model for sequence-based prediction of protein-protein interactions. Journal of computational chemistry, 40(11))1233-1244 2019.
Domenico Rotilio, Anna Della Corte, Marco D'Imperio, Walter Coletta, Simone Marcone, Cristian Silvestri, Lucia Giordano, Michela Di Michele, and Maria Benedetta Donati. Proteomics: bases for protein complexity understanding. Thrombosis research, 129(3): 257-262, 2012.
Juwen Shen, Jian Zhang, Xiaomin Luo, Weiliang Zhu, Kunqian Yu, Kaixian Chen, Yixue Li, and Hualiang Jiang. Predicting protein -protein interactions based only on sequences information. Proceedings of the National Academy of Sciences, 104(11):433--4441 2007.
Chen Shengyuan, Yunfeng Cai, Huang Fang, Xiao Huang, and Mingming Sun. Differentiable neuro-symbolic reasoning on large-scale knowledge graphs. Advances in Neural Information Processing Systems, 36, 2024.
Dong Shu, Tianle Chen, Mingyu Jin, Yiting Zhang, Mengnan Du, and Yongfeng Zhang Knowledge graph large language model (kg-Ilm) for link prediction, 2024.
Bosheng Song, Xiaoyan Luo, Xiaoli Luo, Yuansheng Liu, Zhangming Niu, and Xiangxi- ang Zeng. Learning spatial structures of proteins improves protein-protei interaction prediction. Briefings in bioinformatics, 23(2):bbab558, 2022.
Tanlin Sun, Bo Zhou, Luhua Lai, and Jianfeng Pei. Sequence-based prediction of protein protein interaction using deep-learning algorithm. BMC bioinformatics, 18:1-8, 2017.
Damian Szklarczyk, Annika L Gable, David Lyon, Alexander Junge, Stefan Wyder, Jaime John
Huerta-Cepas, Milan Simonovic, Nadezhda T Doncheva, John H Morris, Peer Bork, al. String v11: protein-proter association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets. Nucleic acids research, 47(D1): D607-D613, 2019.
Felix Teufel, José Juan Almagro Armenteros, Alexander Rosenberg Johansen, Mag- nús Halldór Gislason, Silas Irby Pihl, Konstantinos D Tsirigos, Ole Winther, Søren Brunak, Gunnar von Heijne, and Henrik Nielsen. Signalp 6.0 predicts all five types of signal peptides using protein language models. Nature biotechnology, 40(7):1023-1022 2022.
12

---

Published as a conference paper at COLM 2024
Vineet Thumuluri, José Juan Almagro Armenteros, Alexander Rosenberg Johansen, Henrik Nielsen, and Ole Winther. Deeploc 2.0: multi-label subcellular localization prediction using protein language models. Nucleic Acids Research, 50(W1):W22-W224, 2022.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint ar arXi::2302.13971 2023.
Tung Tran and Ramakanth Kavuluru. An end-to-end deep learning architecture for ex- tracting protein-prec prriein interactions affected by genetic mutations. Database, 2018:1-13, 2018.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arviv preprint arXiv:2206.0768)) 2022.
Matthias Wilm. Quantitative proteomics in biological research. Proteomics, 9(20) 2009.
Lirong Wu, Yijun Tian, Yufei Huang, Siyuan Li, Haitao Lin, Nitesh V Chawla, and Stan Z Li. Mape-ppi: Towards effective and efficient protein-protein interaction prediction via microenvironmentawwarr protein embedding. In The Tweeffth International Conference on Learning Represenlations, 2024. URL https: http///ppeeeeeeeeeeeeeeeeeeeeeee
Yijia Xiao, Jiezhong Qiu, Ziang Li, Chang- Yu Hsieh, and Jie Tang. Modeling protein using large-scale pretrain language model, 2021.
Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Shaochen Zhong, Bing Yin, and Xia Hu. Harnessing the power of Ilms in practice: A survey on chatgpt and beyond. ACM Transactions on Knowledge Discovery from Data, 2023.
Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. Language is all graph needs. EACL, 2024.
Zhu-Hong You, Ying-Ke Lei, Lin Zhu, Junfeng Xia, and Bing Wang. Prediction of protein- protein interactions from amino acid sequences with ensemble extreme learning machines and principal component analysis. In BMC bioinformatics, volume 14, pp. 1-11. Springer, 2013.
Jinxiong Zhang, Cheng Zhong, Yiran Huang, Hai Xiang Lin, and Mian Wang. A method for identifying protein complexes with the features of joint co-localization and joint co-expression in static ppi networks. Computers in Biology and Medicine, 111:103333, 2019.
Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. arXiv preprint ar arXt:2203300612 2022.
Ziyuan Zhao, Peisheng Qian, Xulei Yang, Zeng Zeng, Cuntai Guan, Wai Leong Tam, and Xiaoli Li. Semignn-ppi: Self-ensembling multi-graph neural network for efficient and generalizable protein-protein interaction prediction. ar Xiv preprint ar arXiv:2305.088
2023.
Hong-Yu Yu Zhou, Yunxiang Fu, Zhicheng Zhang, Bian Cheng, and Yizhou Yu. Protein representation learning via knowledge enhanced primary structure reasoning. In The Eleventh International Conference on Learning Representations, 2022.
Hong-Yu Zhou, Yunxiang Fu, Zhicheng Zhang, Cheng Bian, and Yizhou Yu. Protein representation learning via knowledge enhanced primary structure modeling, 2023,
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. Al open, 1:57-81, 2020.
Le Zhuo, Zewen Chi, Minghao Xu, Heyan Huang, Heqi Zheng, Conghui He, Xian-Ling Mao, and Wentao Zhang. Protlim: An interleaved crotein-language Ilm with protein-as-word pre-training. arXiv preprint ar 2r00:22403,,,,20 2024.
13

---

Published as a conference paper at COLM 2024
6 Appendix
6.1 Dataset Partition Algorithm
In the field of graph learning, DES (Depth-First Search), BFS (Breadth-First) Search), and random sampling are three common graph traversal or sampling strategies. Figure 5 shows the difference in between DFS and BFS.
Depth-First Search (DES): DFS starts from a starting node and explores the graph's depth until it cannot go further, then backtracks to the nearest unvisited node from the starting node. This approach makes DFS inclined to explore the deep structure the graph. Breadth- First Search (BFS): BFS starts from starting node and visits its neighboring nodes one by one, then proceeds to visit the neighboring nodes' neighboring nodes, and so on. This approach prioritizes exploring the breadth of the graph. Random Sampling: Random sampling is method of randomly selecting nodes for traversal or sampling, It can employ uniform random selection or select nodes based on certain probabilities.
The choice of strategy depends on the specific problem requirements and DFS is for exploring entire connected components. DFS can be used to find paths in a graph, especially when finding all paths from one node to another. DFS selects the next node for in-depth exploration at each step until the target node is found or cannot continue deeper. This is very similar to the protein signaling pathway in biology. From one protein to the target protein through different proteins, to simulate the signaling pathway. Additionally, after the DFS, we can obtain cyclic structure of connected proteins, where one side of the cycle represents the signaling pathway between all proteins from the head protein to the tail protein, and the other side represents the direct interaction between the head and tail proteins. This is the data format we need in training our model. To simulate signaling pathways for training, we propose ProCot, and only DFS for dataset partition.
$5
56
S3
BF5
DFS
Figure 5: Demo of BFS and DFS dataset partition method.
6.2 The disscusion about the embedding replacement
As shown in Figure Figure 6, the model with expended vocabulary treats the entire protein ID whole when processing protein IDs. In contrast, the tokenizer of the model with original vocabulary will split the protein IDs. The split protein IDs will affect the model's performance in subsequent PPT tasks.
14

---

Published as a conference paper at COLM 2024
Expended Vocabulary
Original Vocabulary
ENSP00000255381
ENSP00000255381
Tokenizer
Tokenizer
ON
2BC
Figure 6: Expended vocabulary vs original vocabulary
6.3
The Datasets
6.3.1
Human dataset
The Human dataset contains 4577 unique proteins and 75875 interactions between proteins. The distribution of the Human dataset is shown on Table 4.
Type
Count
Percentage
Binding
17977
23.70%
Activation
15470
20.41%
Catalysis
11115
14.65%
Inhibition
10611
13.99%
Expression
9052
11.93%
Post-translational
6255
8.25%
Modification and reaction
5354
7.07%
Total Count
75875
Table 4: Distribution of Human dataset.
6.3.2 SHS27K, SHS148K and STRING
The STRING dataset is large collection that contains 4,775,154 protein-protein interaction (PPI) records relevant to human biology, covering 15,335 distinct proteins and 572,568 unique interaction events. Two subsets of the STRING database are SHS27k and SHS148k. These subsets are curated by applying specific filters, such as selecting only proteins that are more than 50 amino acids long and exhibit less than 40% sequence similarity each other, to ensure diversity and relevance. The SHS27k subset is smaller, with 16,912 PPI entries involving 1,690 proteins and a total of 63,408 interactions. The SHS148k subset is more extensive, containing 99,782 PPI entries, 5,189 proteins, and high interaction count of 369,041. The distribution of the datasets is shown Table 5
6.4 Implementation Details and Hyperparameters
ProLLM is trained A40-48G. During training, the number of training epochs is 10, the learning rate is 3e-4, the per-device train batch size is 2, the per-device evaluation batch size 2, the warmup steps are 400, and the weight decay is 0.01.
6.5 Detail Embedding Replacement
To enhance the understanding of protein sequences, adopt method that integrates protein sequence vectorization with vocabulary expansion. First, we query the correspond- ing protein sequence Sp., based the protein's unique identifier Pia using the Ensemble
15

---

Published as a conference paper at COLM 2024
SHS27K
SHS148K
STRING
Type
Count
Percentage
Count
Percentage
Count
Percentage
Reaction
18,162
28.65%
102,964
27.91%
1,669,750
34.98%
Activation
7,400
11.67%
42,516
11.52%
232,240
4.86%
Catalysis
11,796
18.60%
67,168
18.20%
998,266
20.91%
Binding
16,056
25.33%
93,632
25.37%
1,610,314
33.73%
Ptmod
2,872
4.53%
20,153
5.46%
88,424
1.85%
Inhibition
5,550
8.75%
34,712
9.41%
147,676
3.09%
Expression
1,572
2.48%
7,896
2.14%
28,484
0.60%
Table 5: Distribution of SHS27K, SHS148K, STRING dataset.
Input template:
This is the protein rational: The relationship between ((ENSP000800191003) and ((ENSP00000275216)) is catalysis, the relationship between
(ENSPG0000275216)) and {<ENSP00000267396) is activation, the relationship between } and ((ENSP00000318994)) is binding, the relationship between (1ENSP008003189444) and (<ESSP00800252266)} reaction. Let's think step by step, what relationship between ((ENSP660000191003) and
((ENSP8800275216))
Target template:
The relationship is activation.
Figure 7: One example of ProCoT prompt.
BioMart tool. Subsequently, the retrieved protein sequence Spa is fed into the ProtTrans model, which outputs 1 x 1024-dimensional vector Vp encapsulating key information of the sequence. This vector is then used as the embedding vector for the new vocabulary item Pi added to the Tokenizer's 's vocabulary. Through this approach, whenever the model encounters the identifier Pat, it utilizes the embedding vector V, generated by ProtTrans for processing, enabling the model to gain a deeper understanding of protein sequences.
6.6 Prompt in ProLLM
The prompt in ProLLM has two type: ProCot prompt and Instruction finetuning prompt. Figure 7 is example prompt of ProCot. Figure 8 is the prompt of instruction fine-tuning.
16

---

Published as a conference paper at COLM 2024
Raw data:
Instruction Analyze this protein sequence and, based on conserved domains or motifs, deduce its possible cellular function(s).
Sequence S: MALKKAWARPEMEASPLCC..
Metadata M: tRNA
-N-
activity...
Output 0: Upon evaluating the structure of the protein with sequence, it can be predicted that its biological function is primarily associated with tRNA (guanine-N7. metyyy--emtttttafferrsss activity.
Input template:
Follow the instruction to answer the question. The protein sequence is S. M is the information about subcellular localization, primary function, and biological process.
Target template:
The answer is
Figure 8: Example of instruction fine-tuning prompt.
17


Reviews:
[{"review_id": "xK4UVea8PC", "reviewer": "Reviewer_rKT9", "summary": "The authors introduce a novel framework, ProLLM that uses LLMs to investigate PPIs. For this, they devise ProCoT that transforms the gene signaling pathways and protein functions into NL prompts. They discuss how to train and fine tune ProCoT. Experiments and ablation studies are provided that demonstrate the robustness of ProLLM. \n\nThe paper is well written, and previous literature and gaps in the field are thoroughly discussed.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question 1: How do prompts ensure canonical pathway directions among proteins are captured despite diverse feedback/feedforward loops and unknown environmental contexts?**\n\n**Answer:**\n\nProtein-protein interactions are primarily determined by the intrinsic properties of the proteins themselves, such as protein structure, and are largely independent of external environmental factors [1]. The records on protein-protein interaction data included in our databases are supported by wet experimental data and are highly credible. This ensures the high quality of our training data. It is important to note that the primary factors influencing protein interactions include the proteins' expression level and subcellular localization [3]. For instance, if two proteins are expressed concurrently in the same cell and are proximal within the cellular space, they are more likely to interact. Conversely, distant proteins are less likely to interact [4]. This phenomenon allows our ProCoT to accurately reflect classic biological pathways well recognized within the scientific community and have significant implications in disease treatment and drug development [5].\n\n[1] Pawson, T. Protein modules and signaling networks. *Nature* 1995.\n\n[2] Szklarczyk D. The STRING database in 2023: protein-protein association networks and functional enrichment analyses for any sequenced genome of interest. *Nucleic Acids Res.* 2023\n\n[3] Villanueva, E., Smith, T., Pizzinga, M. et al. System-wide analysis of RNA and protein subcellular localization dynamics. *Nat Methods* 2024.\n\n[4] Bickmore WA. Addressing protein localization within the nucleus. *EMBO J.* 2002.\n\n[5] Pham M. Discovery of disease- and drug-specific pathways through community structures of a literature network. *Bioinformatics* 2020.\n\n\n**Question 2: Section 3.1.2. Expand DFS in the main text (and then refer to it in the appendix)**\n\n**Answer:**\n\nThank you for your suggestion. We will expand the explanation of depth first search (DFS) in Section 3.1.2 by including a brief description, and then refer readers to Section 6.1 in the Appendix for more detailed information. This revision will be incorporated in the updated version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed method seems to be effective; the work shows the importance of using an LLM for PPI for the first time.", "reasons_to_reject": "None", "questions_to_authors": "* Related to Definition 2 and thereafter: How can prompts handle the diverse and various nature of feedback/feedforward loops amongst the genes/proteins? For example, many of these connections are triggered by environmental cues. While it is impossible to know all/what the environmental contexts are, how are the authors making sure that, at least, the canonical pathway directions are captured sans context?\n* Section 3.1.2. Expand DFS in the main text (and then refer to it in the appendix)", "extra_scores": {}}, {"review_id": "d1JJylGaQ3", "reviewer": "Reviewer_F5aN", "summary": "This manuscript introduces an approach for predicting protein-protein interactions, especially those due to indirect interactions, such as those found in a pathway. The approach uses an LLM that has been upgraded with protein vectors, fine-tuned for protein tasks, and - most importantly - set up using chain-of-thought to reason through the process step by step.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Why are indirect protein-protein interactions crucial and challenging to understand?**\n\nIn living organisms, signals are transmitted through pathways where upstream proteins send biological signals to downstream proteins via intermediary signal molecule proteins [1]. These indirect protein-protein interactions are crucial for essential life processes, making their understanding vital.\n\nSignal pathways involve multiple intermediary proteins, resulting in complexity and heterogeneity. To address this, we propose leveraging large language models' reasoning capabilities, specifically the Chain of Thought method [2], to solve signal pathway problems with ProCoT.\n\n[1] Anti-cancer effects of Polyphyllin I: An update in 5 years. *Chemico-Biological Interactions*.\n\n[2] Chain-of-thought prompting elicits reasoning in large language models. *NeurIPS*.\n\n**Q2+Q4: Re-wording some technical terms**\n\nWe will describe our approach as natural language inference, which is more specific. Additionally, we will correct 'learn the law of signal pathways' to 'learn the mechanism of signal pathways.'\n\n**Q3: Section 3.1.2 references the principles of \"co-expression\" and \"co-localization\" but these are not defined**\n\nCo-expression means that proteins expressed together are likely to interact; co-localization means that proteins in the same subcellular area have a higher chance of interaction [3]. We will add the descriptions to the paper.\n\n[3] Deciphering cell-cell interactions and communication from gene expression. *Nat Rev Genet*.\n\n**Q5: How does ProTran align well with ProLLM despite different training processes?**\n\nIt is noteworthy that the ProTrans embeddings and other embeddings used in our project align strongly. ProTrans, trained using the T5 model, naturally aligns with our ProLLM which is also based on T5. Both models share similar embedding spaces due to their common architecture and training paradigms. Additionally, fine-tuning ProLLM ensures that ProTrans embeddings and natural language tokens align with each other.\n\nIf we replace our backbone with a different model, such as LLaMA, we must fine-tune the ProTrans embeddings or use alternative embeddings compatible with the new model's embedding space. As shown in Section 4.3 Table 2, where we experiment with different backbones (Flan-T5 and LLaMA), ProLLM based on Flan-T5-large performs better due to the shared embedding space with ProTrans. Thanks for the suggestion and we will re-emphasize this in the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Protein-protein interaction prediction is a critical task for understanding biological systems, with applications across all of biology and medicine. This manuscript is well-written, with a discussion of related work that appears to be comprehensive, and with a very comprehensive evaluation and results that clearly support the methods proposed. Moreover, as stated by the manuscript, this may be the first work to explore protein-protein interaction as a natural language inference problem.", "reasons_to_reject": "The only concerns I have are quite minor: \n\n1. It would be helpful to explain a bit better why indirect protein-protein interactions are so important and difficult.\n\n2. The manuscript refers to the approach as converting the PPI prediction into a natural language processing problem. While quite possibly true, natural language processing is a very large field. I believe that the task here would be more specifically termed natural language inference or, possibly, natural language understanding. \n\n3. Section 3.1.2 references the principles of \"co-expression\" and \"co-localization\" but these are not defined.\n\nAnd a couple of edits:\n1. I think \"learn the law of signal pathways\" (page 2) would be more accurate as \"learn the mechanism of signal pathways.\"\n\n2. In section 4.2 \"Procot\" should probably by \"ProCoT\"", "questions_to_authors": "Given the strong performance, the ProTrans embeddings clearly work well with the other embeddings used. However different training processes typically do not produce embeddings that align in this way. How was this accomplished?", "extra_scores": {}}, {"review_id": "AjwmfLeOGm", "reviewer": "Reviewer_bZnW", "summary": "- The paper presents a sophisticated approach that leverages large language models (LLMs) to predict protein-protein interactions (PPIs) by representing the complex signaling pathways in a natural language format. The use of the Protein Chain of Thought (ProCoT) to simulate signaling pathways as a reasoning chain is a unique method that effectively blends computational biology with advanced NLP techniques.\n- The paper is well-structured, methodologically sound, and systematically presented. The inclusion of detailed figures, such as the ProLLM framework and examples of ProCoT, enhances the clarity and understandability of the proposed method.\n- The concept of using LLMs, specifically the adapted Flan-T5 model infused with protein-specific embeddings, for modeling PPIs through natural language is novel and innovative. The introduction of ProCoT as a means to contextualize protein interactions within signaling pathways demonstrates original thinking.\n- The significance of this work is high, as it proposes a new avenue for PPI prediction which is crucial for understanding biological functions and the mechanism of diseases. The results show a significant improvement over existing methods, which could influence future research in computational biology and drug development.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question 1: Can you elaborate on how ProLLM handles the variability and potential noise in real-world biological datasets?**\n\n**Answer:**\n\nProLLM predicts protein-protein interactions (PPI) through protein signaling pathways, which are commonly present in all proteins and are mostly decided by the fundamental structure of the protein. As shown in Table 1, ProLLM maintains high performance even when encountering proteins not present in the training set. This indicates that our model has learned the patterns of signaling pathways between proteins, demonstrating its wide applicability.\n\n---\n\n**Question 2: How does ProLLM ensure that the model does not overfit to the training data, given the complexity of the model and the specificity of the datasets used?**\n\n**Answer:**\n\nProLLM learns the signaling pathways between proteins rather than specific patterns in the training data, which helps our ProLLM avoid overfitting. Table 1 demonstrates that our model has learned the signaling pathway patterns because it performs efficiently on unseen datasets, further evidencing that our model does not suffer from overfitting. Additionally, we used a validation set to monitor the training process of the model. The average loss on the validation sets for the Human, SHS27K, SHS148K, and STRING datasets was 0.0026, 0.0024, 0.0023, and 0.0021, respectively. This indicates that the model does not exhibit signs of overfitting.\n\n\n\n|         | SHS27K       | SHS148K      | STRING       |\n|---------|--------------|--------------|--------------|\n| micro-F1 | 78.54±4.21   | 81.27±2.04   | 82.59±1.95   |\n\nTable 1: The Performance of ProLLM in different unseen datasets; the ProLLM is only trained on Human proteins.\n\n\n\n\n---\n\n**Question 3: Given the innovative use of language models in predicting PPIs, what are the limitations of this approach in terms of computational resources and scalability?**\n\n**Answer:**\n\nOur inference and training can be accomplished on standard A40 GPUs, indicating that our model does not require high-end hardware. We will include these details about computational resources in the manuscript to help readers better understand our method's computational resource requirements and scalability.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Innovative Conceptual Framework: The paper introduces an innovative method of simulating protein signaling pathways using natural language processing models. By transforming protein interactions into a chain of reasoning with the ProCoT format, the paper leverages the power of LLMs, such as Flan-T5, in a novel biological context. This approach is pioneering as it merges two distinct fields: computational biology and language modeling. This methodology could lead to more intuitive and explainable models for protein interaction, which are crucial for understanding complex biological processes and drug discovery.\n- Impressive Performance: The model demonstrates superior performance metrics, particularly in terms of prediction accuracy and generalizability across multiple datasets, as indicated by the micro-F1 scores provided. The performance improvements are robust, with ProLLM outperforming existing graph-based and LLM-based methods significantly. The high performance indicates that ProLLM can be a more effective tool in the predictive analysis of PPIs, which could transform how researchers approach the prediction of biological interactions.\n- Potential for Broad Impact: The model's ability to accurately predict interactions can directly contribute to better understanding of diseases and the development of therapeutic strategies, especially in complex diseases like cancer where protein interactions play a crucial role. Beyond immediate research applications, the methodology could be adapted for other types of molecular or genetic interactions, broadening its impact.\n- Well-Documented and Reproducible Research: The authors provide access to source code and detailed documentation, which facilitates replication and further investigation by other researchers. This openness not only enhances the scientific integrity of the research but also encourages further innovations and improvements by the broader scientific community.", "reasons_to_reject": "- Scope of Evaluation and Applicability: While the paper presents strong performance on benchmark datasets, these datasets often have well-defined and controlled conditions that may not adequately represent the messy, heterogeneous data encountered in real-world biological research. This raises concerns about the model's robustness and effectiveness outside of academic test conditions. The focus on specific datasets might have tailored the model excessively to those conditions. There is insufficient evidence on how well the model adapts to datasets with different characteristics or to PPI predictions in less-studied organisms, which limits the understanding of its broader applicability.\n- Practical Implications for Broader Adoption: The computational demands for training and running such large-scale models, especially when fine-tuning with extensive protein datasets, might be prohibitive for many research institutions, particularly those with limited IT infrastructure. The dynamic nature of both biological research (with continually updating data and new protein interactions being discovered) and machine learning models (requiring regular updates to maintain performance) poses significant challenges in maintaining the relevance and accuracy of the model over time.\n- Potential Overfitting and Model Bias: The use of a highly complex model architecture, such as a large language model tailored with specific protein data, increases the risk of overfitting. This concern is exacerbated if the model has been overly optimized for high performance on the benchmark datasets without sufficient regularization or cross-validation strategies. The adaptation of language models to biological data involves assumptions and simplifications that may introduce biases, particularly in how protein interactions are represented and predicted. These biases could affect the predictive accuracy when the model is applied to new or different types of protein data, leading to erroneous conclusions or missed interactions.", "questions_to_authors": "- Can you elaborate on how ProLLM handles the variability and potential noise in real-world biological datasets?\n- How does ProLLM ensure that the model does not overfit to the training data, given the complexity of the model and the specificity of the datasets used?\n- Given the innovative use of language models in predicting PPIs, what are the limitations of this approach in terms of computational resources and scalability?", "extra_scores": {}}, {"review_id": "kDTXdvnlT0", "reviewer": "Reviewer_YsHM", "summary": "The paper introduces ProLLM, a new framework employing a LLM specifically tailored for predicting Protein-Protein Interactions (PPIs), crucial in understanding biological functions and diseases. It leverages a unique approach called Protein Chain-of-Thought (ProCoT), which simulates signaling pathways in natural language prompts to predict interactions, focusing on both physical and non-physical connections between proteins. ProLLM outperforms traditional graph-based methods and existing LLM approaches by incorporating protein-specific embeddings and instruction fine-tuning.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question 1: My primary concern lies with the practice in this paper behind flattening highly structured protein data into sequential language. And the paper needs descriptions of how this embedding of ProTrans align with the natural language embedding of Flan-T5-large.**\n\n**Answer:**\n\nFirstly, besides the sequential language we used for PPI prediction, the 'ProTrans' [1] embedding we incorporate includes protein structural information.\n\nSince ProTrans is trained on protein sequences based on T5, its embeddings share similar embedding spaces with Flan-T5-large due to their common architecture and training paradigms. Therefore, ProTrans naturally aligns well with our ProLLM, which is based on the Flan-T5 model. Additionally, during the fine-tuning process of ProLLM, the embeddings of ProTrans will be updated and aligned with the embeddings of natural language to predict protein interactions. Table 2 in section 4.3 demonstrates that ProLLM based on Flan-t5-large has better alignment performance for ProTrans compared to other backbones.\n\nWe will re-emphasize the above descriptions and the results in Table 2 in the manuscript so as to clarify how the protein encoder integrates with the backbone LLM during fine-tuning and alignment.\n\n[1] Elnaggar. ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning. IEEE TPAMI. (2021).\n\n---\n\n**Question 2: The authors could further explore how the proposed ProCoT functions and examine potential failure scenarios. Implementing well-designed experiments to validate the mechanisms described in section 3.1.3 would significantly enhance the paper's conclusions and robustness.**\n\nAs discussed in section 3.1.3, ProCoT functions by representing signaling pathways as chain-of-thought reasoning rules. This is because in biological systems, signals between proteins are transmitted through signaling pathways, facilitating protein-protein interactions. ProCoT can embed these signaling pathways into a chain input for LLM, enabling it to repeatedly utilize these pathways to predict PPIs. We further conducted experiments to validate this. In Table 3 of Section 4.4, we disrupt the sequence of protein signals in ProCoT, which causes a significant drop in prediction accuracy. This is consistent with the working mechanism of ProCoT. Thank you for your feedback. We will re-emphasize the results of Section 4.4 Table 3 in Section 3.1.3 so as to allow readers to gain a deeper understanding of ProCoT.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper introduces an innovative approach to solving PPI task, which improves prediction accuracy and meanwhile enhances the interpretability. \n\n2. The authors provide a comprehensive survey in the Related Work section, detailing previous studies and advancements.\n\n3. The overall presentation of the paper is of high quality. The text is well-crafted and fluid. The figures are clear and effective.", "reasons_to_reject": "1. My primary concern lies with the practice in this paper behind flattening highly structured protein data into sequential language. The authors mention that \"we directly integrate the protein vectors generated from ProtTrans into our model to replace the original word embedding in the protein name’s place\", which describes their method of encoding protein data for the LLM. However, the paper lacks essential details and descriptions of how this protein encoder aligns with the backbone LLM. The specifics of how alignment training is conducted are also not clear.\n\n\n2. The experimental analysis appears to be the weakest aspect of the paper. The analysis conducted is neither thorough nor solid. The authors do not delve deeply into exploring how their proposed ProCoT actually functions, nor do they examine scenarios where it might fail. There is a need for well-designed experiments to validate the mechanisms described in section 3.1.3, which would significantly strengthen the conclusions and its robustness.", "questions_to_authors": "See above", "extra_scores": {}}]

Author rebuttal:

Meta-review: Submission 51, titled "ProLLM: Leveraging Large Language Models for Predicting Protein-Protein Interactions," introduces an innovative framework that utilizes large language models (LLMs) to investigate protein-protein interactions (PPIs). The authors present ProCoT, a novel method that transforms gene signaling pathways and protein functions into natural language prompts, allowing for fine-tuning and training to predict PPIs. The submission has been reviewed by four reviewers, all of whom recognize the novelty and potential impact of the proposed method.

Based on the reviewers' feedback and the authors' thorough rebuttals, Submission 51 presents a highly innovative approach with the potential to significantly advance the field of PPI prediction. While there are some concerns regarding the generalizability and detailed experimental validation, the overall contribution, novelty, and potential impact of the work are substantial. The authors have shown a commitment to addressing the concerns raised, which further strengthens the case for acceptance. The authors should ensure that the final manuscript includes the clarifications and additional experimental results promised in their rebuttals, particularly addressing the alignment of embeddings and the robustness of the model on diverse datasets.
Decision: Accept