ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction

Authors: mingyu  jin
haochen  xue
zhenting  wang
boming  kang
ruosong  ye
kaixiong  zhou
mengnan  du
yongfeng  zhang
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-11 05:25:18

Abstract: The prediction of protein-protein interactions (PPIs) is crucial for understanding biological functions and diseases. Previous machine learning approaches to PPI prediction mainly focus on direct physical interactions, ignoring the broader context of nonphysical connections through intermediate proteins, thus limiting their effectiveness. The emergence of Large Language Models (LLMs) provides a new opportunity for addressing this complex biological challenge. By transforming structured data into natural language prompts, we can map the relationships between proteins into texts. This approach allows LLMs to identify indirect connections between proteins, tracing the path from upstream to downstream. Therefore, we propose a novel framework ProLLM that employs an LLM tailored for PPI for the first time. Specifically, we propose Protein Chain of Thought (ProCoT), which replicates the biological mechanism of signaling pathways as natural language prompts. ProCoT considers a signaling pathway as a protein reasoning process, which starts from upstream proteins and passes through several intermediate proteins to transmit biological signals to downstream proteins. Thus, we can use ProCoT to predict the interaction between upstream proteins and downstream proteins. The training of ProLLM employs the ProCoT format, which enhances the model's understanding of complex biological problems. In addition to ProCoT, this paper also contributes to the exploration of embedding replacement of protein sites in natural language prompts, and instruction fine-tuning in protein knowledge datasets. We demonstrate the efficacy of ProLLM through rigorous validation against benchmark datasets, showing significant improvement over existing methods in terms of prediction accuracy and generalizability. Our results highlight the potential of LLMs to transform the field of PPI, serving as a robust potential tool for various categories of biological and medical research. The code is available at: https://anonymous.4open.science/r/ProLLM-AB04.

Main paper: 
 introduction: 
 Protein-protein interactions (PPIs) play an essential role in various biological processes of all living organisms, which are crucial for biomedical, genetic, and pharmaceutical research. Thus, numerous experimental methods have been proposed for PPI detection, such as yeast two-hybrid (Ito et al., 2001) and quantitative proteomics methods (Rotilio et al., 2012) .Figure 1 : Illustration of the ProLLM Framework. We fine-tuning ProLLM under Human, SHS27K, SHS148K, and STRING datasets, enabling it to solve various PPI related tasks with the structure information purely described by natural language.However, wet-lab methods for PPI prediction are often time-consuming and labor-intensive, highlighting the need for more precise and efficient computational tools.In recent years, computational biology has developed rapidly. Methods such as the Convolutional Neural Network (CNN) and Graph Neural Network (GNN) have become powerful tools for studying protein interaction. CNN-based approaches like TAG-PPI (Song et al., 2022) typically use pre-trained embedding models to convert protein sequences into numerical vector representations, and then employ one dimensional convolutional neural networks to extract features from the vectors for subsequent PPI tasks.Although CNN methods have shown some effectiveness in PPI prediction, they still have limitations due to their fixed receptive fields and the lack of well-defined spatial relationships in protein sequences, which limit the accuracy and interpretability of the predictions. GNN-based methods such as GNN-PPI (Lv et al., 2021) treat proteins as nodes and their relationships as edges, constructing a network composed of proteins, which better captures protein relationships and interactions, and outperforms CNNs in predicting protein interactions. However, while GNNs can effectively extract network structural information, they neglected the non-physical connections between two proteins without direct physical interactions, resulting in the worse performance in learning protein chains than transformerbased models (Zhou et al., 2020) . Furthermore, GNNs cannot fully capture the relationships and dynamic changes in the signal passing process of living organisms, restricting their performance for PPI prediction (Zhou et al., 2023) .Following the GNN and CNN methods, Large Language Models have also been applied to this PPI area, such as ProBert (Elnaggar et al., 2021a) and ProteinLM (Xiao et al., 2021) . As long as these models can obtain a protein representation, we can use the direct cosine similarity of the representation or train an MLP to perform PPI prediction. However, these methods still cannot capture the chain relationships between proteins, such as the signaling pathways. Besides, previous literature only used LLMs as a feature extractor. Recently, using LLM as a link predictor has shown that it can better capture relational information between nodes in knowledge graph tasks and its performance surpasses traditional GNN baselines (Ye et al., 2024; Zhuo et al., 2024; Shu et al., 2024) . Therefore, it is promising to introduce LLM for protein-protein interaction (PPI) tasks, since the most important biological signal for PPI tasks is the chain relationships of proteins, i.e., the signaling pathways.To bridge the gap, we propose ProLLM, with its key ideas illustrated in Figure 1 , and the difference between the existing method and our ProLLM shown in Figure 2 . Existing methods only focus on the single protein-protein interaction, overlooking the application of protein links to predict PPI in signaling pathways. Instead, we employ a large language model to learn the law of signal pathways and adapt the LLM to directly predict the type of interaction between proteins.The signaling pathway addresses the traditional method's ignorance of global, non-physical connections between proteins. Signaling pathways typically start with an upstream protein that sends a biological signal through several intermediates to a downstream protein, hence Large Language ModelFigure 2 : The difference between the existing method and our method in PPI prediction.Existing method focus on the property of upstream protein and downstream protein, our method focus on signaling pathway-like connection.requiring consideration of the cumulative effect of multiple protein interactions. This series of interactions form sequential chains. Therefore, we propose Protein Chain of Thought (ProCoT) to overcome the limitation in understanding signaling pathways and protein functions. ProCoT is a data format that simulates the signal transduction process using a thought-chain approach, thereby enabling the prediction of protein interactions in signaling pathway problems. CoT can express the thinking process step by step to form a reasoning chain (Jin et al., 2024b) , while ProCoT extends this principle further into the protein-related domain to simulate protein signaling pathways, giving LLMs a deeper insight into protein.Additionally, our approach addresses the issue of poor protein comprehension in LLMs by replacing the standard language model embedding with embedding infused with protein information. When we process the protein name in the prompt, we replace its original embedding by ProtTrans (Elnaggar et al., 2021b) , because its embedding contains protein's structual information. We also perform the instruction fine-tuning on the protein knowledge dataset to infuse protein domain knowledge into LLMs. Following these steps, the LLM acquires a robust ability to reason about the direct relationships between proteins, as demonstrated in Figure 1 . It can provide answers to questions about protein relationships, which play a significant role in biological research.Our contributions are summarized as follows:• To the best of our knowledge, we are the first to explore the PPI prediction problem as a natural language processing problem. The proposed Protein Chain of Thought (ProCoT) is a novel method for understanding complex multi-step protein interactions, i.e., those present in signaling pathways.• We propose embedding replacement and instruction fine-tuning on our model to enhance its ability to understand and reason about proteins. This also provides our model with rich background knowledge of protein sequences and protein interactions before training.  
related work: 
 Protein-protein Iteractions Protein-protein interactions (PPIs) are indispensable for all living organisms, and so many efforts have been made for PPI detection up to now. Yeast two-hybrid (Y2H) assays (Brückner et al., 2009) , synthetic lethality (O'Neil et al., 2017) , quantitative proteomics (Wilm, 2009) , and mass spectrometry (Mann et al., 2001) , are widely used for identifying PPIs. To be specific, Y2H assays explore binary PPIs in living cells, offer-ing insight into protein functions and regulations, although labor-intensive and limited in genomic coverage. Synthetic lethality identifies essential gene pair interactions by revealing compensatory relationships when both are inactivated. Meanwhile, quantitative proteomics, especially through mass spectrometry, illuminates the dynamic nature of the interactome under various conditions. Although informative, these methods require significant labor, time, and resources.Traditional Machine Learning Models for PPI Prediction In the realm of machine learning, sequence-based approaches include Shen's SVM method (Shen et al., 2007) , which uses a 3-mer count vector from protein sequences as features and groups 20 amino acids into seven classes to handle synonymous mutations and reduce feature space dimensionality. SVM-based methods (Guo et al., 2008) and the ensemble model PCA-EELM (Principal Component Analysis-Ensemble Extreme Learning Machine) (You et al., 2013) utilize various types of protein sequence information for PPI prediction. In the domain of deep learning, DeepPPI (Du et al., 2017) extracts a multitude of features from protein sequences and employs a dual deep neural network structure for feature fusion and prediction. Sun et al. (Sun et al., 2017) introduced a PPI predictor based on a stacked autoencoder, emphasizing the importance of sample balance. DPPI (Hashemifar et al., 2018) and TAGPPI (Song et al., 2022) further extend the application of convolutional neural networks and integrate text convolution networks with graph representation learning to enhance the accuracy of PPI predictions. GNNs have significantly advanced PPI predictions, improving our understanding of biological mechanisms. GNN-PPI (Lv et al., 2021) enhances inter-novel-protein prediction accuracy by utilizing protein relationships and a new evaluation approach. PT-GNN (Long et al., 2022) integrates diverse data for link prediction, learning node features from sequence and structure. DeepRank-GNN (Réau et al., 2023) offers a modular, Python-packaged framework for GNN-based interaction pattern predictions. HIGH-PPI (Gao et al., 2023) introduces a hierarchical graph learning model for effective PPI prediction and molecular detail extrapolation. Geometric GNNs excel in modeling spatial intricacies, enhancing biomolecule prediction accuracy. Geo-PPI (Liu et al., 2021) utilizes self-supervised learning for geometric protein structure representations, excelling in detailing protein interactions. mmCSM-PPI (Rodrigues et al., 2021) captures multifaceted features for mutation impact predictions on protein interactions. MAPE-PPI (Wu et al., 2024) defines the microenvironment of amino acid residues in proteins and encodes it into discrete codes, which can capture the connections between different microenvironments, enhancing the prediction accuracy of PPI.Large Language Model for PPI prediction Recent advances in large language models, such as BERT (Devlin et al., 2018) , GPT (Peng et al., 2023) , LLaMA (Touvron et al., 2023) , and T5 (Raffel et al., 2020) , have significantly advanced the field of Natural Language Processing (NLP) to new heights. These models, having been trained on extensive textual corpora, exhibit exceptional capabilities across a diverse range of NLP applications (Shengyuan et al., 2024; Jin et al., 2024a; Fan et al., 2024; Hua et al., 2024) . Inspired by LLMs, Protein Large Language Models(PLMs) pre-trained on large-scale protein sequences have emerged, such as ESM (Hsu et al., 2022) , ProtTrans (Elnaggar et al., 2021b) and ProteinBert (Elnaggar et al., 2021b) . PLMs provide a better representation of protein sequences by converting the protein sequences into the form of high-dimensional numerical vectors, known as embedding. With the protein sequences captured by the PLMs, the performances on diverse downstream tasks, such as structure prediction (Lin et al., 2023) , subcellular localization prediction (Thumuluri et al., 2022) , single peptide prediction (Teufel et al., 2022) and Nlinked glycosylation sites prediction (Hou et al., 2023) , have been transformed. It can be expected that PLMs will assist in PPI prediction tasks. ProtLLM (Zhuo et al., 2024) utilizes a dynamic protein mounting mechanism, a protein-as-word language modeling approach, and the InterPT dataset for pre-training, enabling it to handle complex inputs and achieve superior performance on various proteins-related tasks. However, ProtLLM is used for general protein tasks, and it is not used for PPI tasks exactly. The methodology of training these LLMs to convert text inputs to desired text outputs positions them as particularly advantageous for tasks such as generative link prediction (Ye et al., 2024; Shu et al., 2024) .In such tasks, the model is tasked with inferring and generating the relationship between two entities based on provided textual cues. Moreover, the extensive pre-training phase The relation is activation.Real answer (label)The relation is inhibition.Likelihood Loss enables LLMs to exhibit a remarkable capacity for generalization. This capacity allows them to effectively tackle and respond to tasks or prompts that were not explicitly covered during their initial training (Wei et al., 2022) . In addition to the outlined capabilities, the inherent flexibility and generalization potential of LLMs suggests that their applicability extends well beyond the conventional boundaries of NLP tasks (Yang et al., 2023) . Specifically, their proficiency in generalizing from expansive pre-training sessions paves the way for their application in fields like bioinformatics and complex network analysis. 
proposed prollm framework: 
 In this section, we will introduce the implementation details of ProLLM, a framework designed to transform protein interaction data into ProCoT format natural language descriptions to simulate protein signaling pathways. By translating the structure relationships of proteins into natural language, we effectively transform protein interaction problems into natural language processing (NLP) tasks. To enhance the understanding of proteins by ProLLM, we directly integrate the protein vectors generated from ProtTrans (Elnaggar et al., 2021b) into our model to replace the original word embedding in the protein name's place. This approach allows our model to understand and utilize the biological attributes of proteins when predicting protein interactions. The ProLLM process is shown in Figure 3 .Transformation of the original protein data into a ProCoT natural language is a critical step, this kind of natural language type that represents the relationships of proteins is shown as Definition 1.Definition 1 (Protein Interaction) Given the set P representing all proteins under consideration in the study, we define the interaction between any two proteins p n , p m ∈ P with the specific type interaction R i interaction set R. This relationship is encapsulated as a tuple (p n , p m , R i ). M may be the end of this signal level, or it may open a second signal hierarchy. If in the second signal level, the M will replace the Rc in the initial signal hierarchy as the starting point. The signal will continue to propagate from Rc through new signaling proteins, and effector proteins, and then generate new secondary messengers. After this stage, the propagation of the signal repeats the structure of the second signal level until the signal is interrupted.We aim to improve the model's understanding of biological signaling pathways, enhancing its ability to learn and reason interactions within complex signal networks. We create prompts in the ProCoT (Protein Chain of Thought) format to incrementally decompose the signal transduction process like Definition 2, simulating real pathways of signal propagation. Specifically, we clarify the structure knowledge within the signaling pathway, such as signaling proteins and effector proteins. We then design rules for signal transmission across different levels to simulate the iterative process of signal transduction in proteins. Finally, we also design signal interruptions to simulate the continuity of protein transmission in real life. We use hard code to convert protein interaction information as Definition 1 in dataset into ProCoT format natural language to imitate protein signaling pathways as Definition 2. Figure 7 in the appendix is an example of ProCoT. The answer to this is "The relationship is activation".In biology, co-expression and co-localization refer to the phenomenon in which proteins that are often expressed or located together in the cell tend to participate in the same or interconnected biological processes (Zhang et al., 2019) . Thus, biologists frequently use known protein information to infer unknown protein interactions. Based on the principles of co-expression and co-localization in biology, we have formulated our training strategy.After constructing our ProCoT training data using DFS, we will obtain a circular protein interaction as in Figure 4 . We mask the relationship between the initial and final proteins in the signaling pathway and let our model predict this relationship.In this section, we will explore ProCoT through the lenses of biology and AI. (1) Simulating Biological Signaling Pathways: The biological signaling pathway is a series of ordered, rule-based interaction processes. During the process, the output of each step serves as the input for the next step, forming a highly organized information transmission chain. This pattern is consistent with the methods of handling serialized information in natural language processing (NLP). Additionally, Flan-T5's self-attention blocks can capture the indirect relationships between proteins that are distant and do not have direct interactions.It is crucial for understanding biological signal transduction processes that involve multiple steps and complex intermediate links.(2) Signal Interruption Mechanism: This mechanism aims to mimic protein adaptability since the signaling between proteins needs to be constantly interrupted to ensure the cell's accurate response to external changes. This mechanism we designed can satisfy the complex feedback mechanisms and regulatory networks within the cell. Overall, our ProCoT design follows biological principles.In our methodology, we enhance the model's understanding of protein sequence by replacing the T5 embedding with the ProTrans embedding vectors and also perform instruction fine-tuning to enable the model to learn protein functions.We use a new embedding mechanism facilitated by the ProTrans (Elnaggar et al., 2021a) model to perform embedding replacement. ProTrans is a large-scale protein model capable of transforming protein sequences into embedding vectors with the biophysical and structural features of proteins.Definition 3 (Embedding Replacement) Given the protein id P id , we can query its protein sequence P seq by the professional bioinformatics tool Ensembl BioMart. P seq will be the input of ProTrans and ProTrans will output a 1 × 1024-dimensional vector P emb .We add all protein IDs in the protein dataset and their corresponding embeddings to the T5 vocabulary to solve the OOV (out of vocabulary) problem: The T5 vocabulary does not contain words for protein ID numbers. Because ProtTrans is trained based on T5-large, the process of adding to the vocabulary does not involve any change in dimensions, and there is no information loss. Additionally, this approach can also avoid the issue where a protein ID is tokenized into multiple tokens, causing the model to fail to understand it.The replaced embedding vectors from ProtTrans can add a lot of prior knowledge about the intrinsic patterns and biophysical features of proteins to our model, better applying it to subsequent protein interaction prediction tasks.Mol-Instructions (Mol) (Fang et al., 2023) dataset is applied to conduct instruction finetuning on our model. Mol is a comprehensive instruction dataset designed for the field of biomolecules that aims to address the limitations of LLMs in biomolecular research. We use Mol to teach our model with knowledge of protein functions.Definition 4 (Description of Mol) The content of the Mol dataset can be defined as follows: instructional text I related to LLMs queries, an input amino acid sequence S that includes essential protein information, and metadata M that sheds light on vital details like the protein's subcellular location, its primary function, and its participation in biological processes, followed by a corresponding output O that serves as the response to I.To facilitate instruction-based fine-tuning, we convert these objects into prompt-answer pairs (P, A). This is shown as Figure 8 in the appendix. We convert the entire Mol dataset into a prompt-answer format and using these prompt-answer pairs for instruction finetuning of our model(ProLLM). This part can enhance the model's predictive performance on protein-related tasks. 
experiments: 
 In this section, we present the experimental results to answer the following research questions (RQs).• RQ1 -How does the performance of the proposed ProLLM framework compare to other baselines in terms of PPI prediction accuracy and generalizability? • RQ2 -How do different LLM backbones (e.g., Flan-T5-base and LLaMA-7b) affect the performance of the proposed ProLLM framework? • RQ3 -What are the contributions of each component to the PPI prediction performance of the ProLLM framework?Datasets: We undertake comprehensive evaluations on a trio of public Protein-Protein Interaction (PPI) datasets: Human (Song et al., 2022) , STRING (Szklarczyk et al., 2019) , SHS27k, and SHS148k (Chen et al., 2019) . In these four datasets, we employ DFS dataset partitioning techniques. By prioritizing depth, DFS can effectively capture the step-by-step signal transmission and the hierarchical signal level of protein signaling pathways. Each dataset is split into training, validation, and testing sets, maintaining a proportion of 70%, 10%, and 20% respectively.Baselines: Earlier studies on predicting PPI are not pre-trained, they do not have prior knowledge about the proteins. Hence, we choose SVM (Romero-Molina et al., 2019) , DPPI (Hashemifar et al., 2018) , DNN-PPI (Li et al., 2018) , PIPR (Chen et al., 2019) , GNN-PPI (Lv et al., 2021) and SemiGNN-PPI (Zhao et al., 2023) as the baseline without pretraining. Furthermore, we explore how protein pre-training can influence the PPI, we include ProBERT (Elnaggar et al., 2021a) , SM-1b (Rives et al., 2021) , GearNet-Edge (Zhang et al., 2022) , and KeAP (Zhou et al., 2022) as pre-trained baselines. MAPE-PPI (Wu et al., 2024) and InstrucGLM (Ye et al., 2024) have two versions: one with pre-training and one without pre-training.Evaluation Metrics: We selected the micro-F1 score (Harbecke et al., 2022) as our evaluation metric because the PPI dataset exhibits class imbalance, making the F1 score a very relevant reference. Note that micro-F1 is widely used in the protein-protein interaction task (Tran & Kavuluru, 2018) . Each dataset follows a 70% training, 10% validation, and 20% testing data split. Subsequently, we will choose different random seeds for training and testing, conducting a total of 10 tests. The mean micro-F1 score across these trials will serve as the definitive measure of model performance, with the accompanying standard deviation reflecting variability in different experimental runs.We compare the performance of ProLLM with other baselines (w/ and w/o additional pre-training data) on four datasets in Table 1 . Note that here we use Flan-T5-large (Raffel et al., 2020) as the backbone model. Based on the results, we can make three important observations: (1) Our method outperforms other baselines without pre-training. Although InstructGLM is also LLM based, it lags behind ProLLM; (2) As for the models pre-trained on protein dataset, they cannot achieve the performance of ProLLM without prior knowledge;(3) ProLLM outperforms GearNet-Edge, KeAP, and MAPE-PPI, although they utilize a significantly larger dataset comprised of structural and knowledge graph data for pretraining than the Mol dataset.We choose Flan-T5-base, Flan-T5-large, Flan-T5-XL (Raffel et al., 2020) and LLaMA-7b (Touvron et al., 2023 ) models as the backbone for ProLLM. We report the micro-F1 performance comparison as in Table 2 . We should replace the embedding in Flan-T5 with ProtTrans and ProtTrans is a pre-trained model based on Flan-T5-Large. Therefore, the embedding generated by ProtTrans will match Flan-T5-large better. Additionally, despite having more model parameters, LLaMA-v1-7b exhibits worse lower micro-F1 scores in four datasets compared to lighter models: Flan-T5-base, Flan-T5-large, and Flan-T5-XL. Furthermore, the greater standard deviation of LLaMA-v1-7b highlights its instability. Table 3 : Ablation study. The metric here is micro-F1. Where bold denote the best metrics. Higher micro-F1 denotes better performance.the format of ProCoT is sequential, which is a type of information that LLMs are good at processing. 2) Integration of protein-specific embeddings from ProtTrans, and 3) Instruction fine-tuning on protein knowledge datasets. Through extensive experiments on four PPI datasets, ProLLM significantly outperformed existing graph-based and language model methods in prediction accuracy and generalizability. By unifying language models with structured biological data, our work opens up new possibilities for driving discoveries in computational biology, drug discovery, and broader scientific domains. 
acknowledgement: 
 We thank Wenyue Hua, Kai Mei and Taowen Wang for their valuable discussions and suggestions during the project. 
dataset partition algorithm: 
 In the field of graph learning, DFS (Depth-First Search), BFS (Breadth-First Search), and random sampling are three common graph traversal or sampling strategies. Figure 5 shows the difference in between DFS and BFS.Depth-First Search (DFS): DFS starts from a starting node and explores the graph's depth until it cannot go further, then backtracks to the nearest unvisited node from the starting node. This approach makes DFS inclined to explore the deep structure of the graph. Breadth-First Search (BFS): BFS starts from a starting node and visits its neighboring nodes one by one, then proceeds to visit the neighboring nodes' nodes, and so on. This approach prioritizes exploring the breadth of the graph. Random Sampling: Random sampling is a method of randomly selecting nodes for traversal or sampling. It can employ uniform random selection or select nodes based on certain probabilities.The choice of strategy depends on the specific problem requirements and DFS is for exploring entire connected components. DFS can be used to find paths in a graph, especially when finding all paths from one node to another. DFS selects the next node for in-depth exploration at each step until the target node is found or cannot continue deeper. This is very similar to the protein signaling pathway in biology. From one protein to the target protein through different proteins, to simulate the signaling pathway. Additionally, after the DFS, we can obtain a cyclic structure of connected proteins, where one side of the cycle represents the signaling pathway between all proteins from the head protein to the tail protein, and the other side represents the direct interaction between the head and tail proteins. This is the data format we need in training our model. To simulate signaling pathways for training, we propose ProCot, and we only use DFS for dataset partition. As shown in Figure Figure 6 , the model with expended vocabulary treats the entire protein ID as a whole when processing protein IDs. In contrast, the tokenizer of the model with original vocabulary will split the protein IDs. The split protein IDs will affect the model's performance in subsequent PPI tasks. The STRING dataset is a large collection that contains 4,775,154 protein-protein interaction (PPI) records relevant to human biology, covering 15,335 distinct proteins and 572,568 unique interaction events. Two subsets of the STRING database are SHS27k and SHS148k. These subsets are curated by applying specific filters, such as selecting only proteins that are more than 50 amino acids long and exhibit less than 40% sequence similarity to each other, to ensure diversity and relevance. The SHS27k subset is smaller, with 16,912 PPI entries involving 1,690 proteins and a total of 63,408 interactions. The SHS148k subset is more extensive, containing 99,782 PPI entries, 5,189 proteins, and a high interaction count of 369,041. The distribution of the datasets is shown in Table 5 6.4 Implementation Details and HyperparametersProLLM is trained on A40-48G. During training, the number of training epochs is 10, the learning rate is 3e-4, the per-device train batch size is 2, the per-device evaluation batch size is 2, the warmup steps are 400, and the weight decay is 0.01.To enhance the understanding of protein sequences, we adopt a method that integrates protein sequence vectorization with vocabulary expansion. First, we query the corresponding protein sequence S p id based on the protein's unique identifier P id using the Ensemble Target template:The relationship is activation. BioMart tool. Subsequently, the retrieved protein sequence S p id is fed into the ProtTrans model, which outputs a 1 × 1024-dimensional vector V p encapsulating key information of the sequence. This vector is then used as the embedding vector for the new vocabulary item P id added to the Tokenizer's vocabulary. Through this approach, whenever the model encounters the identifier P id , it utilizes the embedding vector V p generated by ProtTrans for processing, enabling the model to gain a deeper understanding of protein sequences.The prompt in ProLLM has two type: ProCot prompt and Instruction finetuning prompt. Figure 7 is an example prompt of ProCot. Figure 8 is the prompt of instruction fine-tuning. 
 
 Appendix:  


Decision: Submission 51, titled "ProLLM: Leveraging Large Language Models for Predicting Protein-Protein Interactions," introduces an innovative framework that utilizes large language models (LLMs) to investigate protein-protein interactions (PPIs). The authors present ProCoT, a novel method that transforms gene signaling pathways and protein functions into natural language prompts, allowing for fine-tuning and training to predict PPIs. The submission has been reviewed by four reviewers, all of whom recognize the novelty and potential impact of the proposed method.

Based on the reviewers' feedback and the authors' thorough rebuttals, Submission 51 presents a highly innovative approach with the potential to significantly advance the field of PPI prediction. While there are some concerns regarding the generalizability and detailed experimental validation, the overall contribution, novelty, and potential impact of the work are substantial. The authors have shown a commitment to addressing the concerns raised, which further strengthens the case for acceptance. The authors should ensure that the final manuscript includes the clarifications and additional experimental results promised in their rebuttals, particularly addressing the alignment of embeddings and the robustness of the model on diverse datasets.

Reviews: review: paper_summary: paper_summary: The authors introduce a novel framework, ProLLM that uses LLMs to investigate PPIs. For this, they devise ProCoT that transforms the gene signaling pathways and protein functions into NL prompts. They discuss how to train and fine tune ProCoT. Experiments and ablation studies are provided that demonstrate the robustness of ProLLM. 

The paper is well written, and previous literature and gaps in the field are thoroughly discussed.
strength_weakness: strength_weakness: None
questions: questions: * Related to Definition 2 and thereafter: How can prompts handle the diverse and various nature of feedback/feedforward loops amongst the genes/proteins? For example, many of these connections are triggered by environmental cues. While it is impossible to know all/what the environmental contexts are, how are the authors making sure that, at least, the canonical pathway directions are captured sans context?
* Section 3.1.2. Expand DFS in the main text (and then refer to it in the appendix)
paper_summary: paper_summary: This manuscript introduces an approach for predicting protein-protein interactions, especially those due to indirect interactions, such as those found in a pathway. The approach uses an LLM that has been upgraded with protein vectors, fine-tuned for protein tasks, and - most importantly - set up using chain-of-thought to reason through the process step by step.
strength_weakness: strength_weakness: The only concerns I have are quite minor: 

1. It would be helpful to explain a bit better why indirect protein-protein interactions are so important and difficult.

2. The manuscript refers to the approach as converting the PPI prediction into a natural language processing problem. While quite possibly true, natural language processing is a very large field. I believe that the task here would be more specifically termed natural language inference or, possibly, natural language understanding. 

3. Section 3.1.2 references the principles of "co-expression" and "co-localization" but these are not defined.

And a couple of edits:
1. I think "learn the law of signal pathways" (page 2) would be more accurate as "learn the mechanism of signal pathways."

2. In section 4.2 "Procot" should probably by "ProCoT"
questions: questions: Given the strong performance, the ProTrans embeddings clearly work well with the other embeddings used. However different training processes typically do not produce embeddings that align in this way. How was this accomplished?
paper_summary: paper_summary: - The paper presents a sophisticated approach that leverages large language models (LLMs) to predict protein-protein interactions (PPIs) by representing the complex signaling pathways in a natural language format. The use of the Protein Chain of Thought (ProCoT) to simulate signaling pathways as a reasoning chain is a unique method that effectively blends computational biology with advanced NLP techniques.
- The paper is well-structured, methodologically sound, and systematically presented. The inclusion of detailed figures, such as the ProLLM framework and examples of ProCoT, enhances the clarity and understandability of the proposed method.
- The concept of using LLMs, specifically the adapted Flan-T5 model infused with protein-specific embeddings, for modeling PPIs through natural language is novel and innovative. The introduction of ProCoT as a means to contextualize protein interactions within signaling pathways demonstrates original thinking.
- The significance of this work is high, as it proposes a new avenue for PPI prediction which is crucial for understanding biological functions and the mechanism of diseases. The results show a significant improvement over existing methods, which could influence future research in computational biology and drug development.
strength_weakness: strength_weakness: - Scope of Evaluation and Applicability: While the paper presents strong performance on benchmark datasets, these datasets often have well-defined and controlled conditions that may not adequately represent the messy, heterogeneous data encountered in real-world biological research. This raises concerns about the model's robustness and effectiveness outside of academic test conditions. The focus on specific datasets might have tailored the model excessively to those conditions. There is insufficient evidence on how well the model adapts to datasets with different characteristics or to PPI predictions in less-studied organisms, which limits the understanding of its broader applicability.
- Practical Implications for Broader Adoption: The computational demands for training and running such large-scale models, especially when fine-tuning with extensive protein datasets, might be prohibitive for many research institutions, particularly those with limited IT infrastructure. The dynamic nature of both biological research (with continually updating data and new protein interactions being discovered) and machine learning models (requiring regular updates to maintain performance) poses significant challenges in maintaining the relevance and accuracy of the model over time.
- Potential Overfitting and Model Bias: The use of a highly complex model architecture, such as a large language model tailored with specific protein data, increases the risk of overfitting. This concern is exacerbated if the model has been overly optimized for high performance on the benchmark datasets without sufficient regularization or cross-validation strategies. The adaptation of language models to biological data involves assumptions and simplifications that may introduce biases, particularly in how protein interactions are represented and predicted. These biases could affect the predictive accuracy when the model is applied to new or different types of protein data, leading to erroneous conclusions or missed interactions.
questions: questions: - Can you elaborate on how ProLLM handles the variability and potential noise in real-world biological datasets?
- How does ProLLM ensure that the model does not overfit to the training data, given the complexity of the model and the specificity of the datasets used?
- Given the innovative use of language models in predicting PPIs, what are the limitations of this approach in terms of computational resources and scalability?
paper_summary: paper_summary: The paper introduces ProLLM, a new framework employing a LLM specifically tailored for predicting Protein-Protein Interactions (PPIs), crucial in understanding biological functions and diseases. It leverages a unique approach called Protein Chain-of-Thought (ProCoT), which simulates signaling pathways in natural language prompts to predict interactions, focusing on both physical and non-physical connections between proteins. ProLLM outperforms traditional graph-based methods and existing LLM approaches by incorporating protein-specific embeddings and instruction fine-tuning.
strength_weakness: strength_weakness: 1. My primary concern lies with the practice in this paper behind flattening highly structured protein data into sequential language. The authors mention that "we directly integrate the protein vectors generated from ProtTrans into our model to replace the original word embedding in the protein name’s place", which describes their method of encoding protein data for the LLM. However, the paper lacks essential details and descriptions of how this protein encoder aligns with the backbone LLM. The specifics of how alignment training is conducted are also not clear.


2. The experimental analysis appears to be the weakest aspect of the paper. The analysis conducted is neither thorough nor solid. The authors do not delve deeply into exploring how their proposed ProCoT actually functions, nor do they examine scenarios where it might fail. There is a need for well-designed experiments to validate the mechanisms described in section 3.1.3, which would significantly strengthen the conclusions and its robustness.
questions: questions: See above

Comments: title: A Friendly Reminder
A Friendly Reminder
comment: Dear Reviewer bZnW,

Thanks again for your valuable comments and precious time. As the author-reviewer discussion period draws to a close, we genuinely hope you could have a look at the new results and clarifications and kindly let us know if they have addressed your concerns. We would appreciate the opportunity to engage further if needed.

Best,

Authors of Paper 51
Dear Reviewer YsHM,

Thanks again for your valuable comments and precious time. As the author-reviewer discussion period draws to a close, we genuinely hope you could have a look at the new results and clarifications and kindly let us know if they have addressed your concerns. We would appreciate the opportunity to engage further if needed.

Best,

Authors of Paper 51
Thank you very much for your support and feedback. We will add the new clarification in the revised version.
Thank you for the response. These all make sense and I assume there will be (very minor) updates to the manuscript. My (positive) recommendation remains unchanged.