Protein Multimer Structure Prediction via Prompt Learning
Authors: Ziqi Gao
Xiangguo Sun
Zijing Liu
Yu Li
Hong Cheng
Jia Li
Venue: ICLR 2024 poster (2024)
Keywords: docking path prediction
protein complex structure
prompt learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: A pre-training and prompt tuning approach for predicting complex 3D structures in a step-wise assembly fashion.

Abstract: Understanding the 3D structures of protein multimers is crucial, as they play a vital role in regulating various cellular processes. It has been empirically confirmed that the multimer structure prediction (MSP) can be well handled in a step-wise assembly fashion using provided dimer structures and predicted protein-protein interactions (PPIs). However, due to the biological gap in the formation of dimers and larger multimers, directly applying PPI prediction techniques can often cause a poor generalization to the MSP task. To address this challenge, we aim to extend the PPI knowledge to multimers of different scales (i.e., chain numbers). Specifically, we propose PromptMSP, a pre-training and Prompt tuning framework for Multimer Structure Prediction. First, we tailor the source and target tasks for effective PPI knowledge learning and efficient inference, respectively. We design PPI-inspired prompt learning to narrow the gaps of two task formats and generalize the PPI knowledge to multimers of different scales. We provide a meta-learning strategy to learn a reliable initialization of the prompt model, enabling our prompting framework to effectively adapt to limited data for large-scale multimers. Empirically, we achieve both significant accuracy (RMSD and TM-Score) and efficiency improvements compared to advanced MSP models.

Published as a conference paper at ICLR 2024
PROTEIN MULTIMER STRUCTURE PREDICTION VIA PROMPT LEARNING
Ziqi Gao 2, Xiangguo Sun Zijing Liu", Yu Li Hong , Jía Li Hong Kong University of Science and Technology (Guangzhou)
2 Hong Kong University of Science and Technology
The Chinese University of Hong Kong IDDE Research, International Digital Economy Academy
ABSTRACT
Understanding the 3D structures of protein multimers is crucial, they play a vital role in regulating various cellular processes. has been empirically con- firmed that the multimer structure prediction (MSP) can be well handled in a step-wise assembly fashion using provided dimer structures and predicted protein- protein interactions (PPIs). However, due to the biological gap in the forma- tion dimers and larger multimers, directly applying PPI prediction techniques often cause a poor generalization to the MSP task. To address this chal- lenge, we aim to extend the PPI knowledge to multimers of different scales (i.e., chain numbers). Specifically, we propose PROMPTMSP, pre-training and Prompt tuning framework for Multimer Structure Prediction. First, we tailor the source and target tasks for effective PPI knowledge learning and efficient inference, respectively. We design PPI-inspired prompt learning to narrow the gaps of two task formats and generalize the PPI knowledge to multimers of dif- ferent scales. We provide a meta-learning strategy to learn a reliable initial- ization of the prompt model, enabling our prompting framework to effectively adapt to limited data for large-scale multimers. Empirically, we achieve both sigmificant accuracy (RMSD and TM-Score) and efficiency improvements com- pared to advanced MSP models. The code, data and checkpoints are released at https: / wig.ghthhhuhhooggg com http:////gggg2////////ttooo rMMp
INTRODUCTION
Recent advances in deep learning have driven the development of AlphaFold 2 (AF2) (Jumper al., 2021), a groundbreaking method for predicting protein 3D structures. With minor modifications, AF2 can be extended AlphaFold-Multine (AFM) (Evans et al., 2021) to predict the 3D structure of multimers (i.e., proteins that consist of multiple chains), which is fundamental in understanding molecular functions and cellular signaling of many biological processes. AFM has been verified to accurately predict the structures of multimers with small scales (i.e., chain numbers). However, its performance rapidly declines as the scale increases.
For multimer structure prediction (MSP), another re- search line (Esquivel-
al., 2012; Aderinwale
et al., 2022; Inbar et al., 2005; Bryant et al., 2022) fol- lows the idea of step-wise assembly (Figure IA), where the assembly action indicates the protein-protein inter- action (PPI). sequentially expands the assembly size by adding a chain with the highest docking probability. The advantage of this step-wise assembly is that it can effectively handle multimers with large scales by enjoy- ing the breakthrough in dimer structure prediction methods (Ganea et al., 2021; Wang et al., 2023; Ketata et al., 2023; Ghani et al., 2021; Luo al., 2023; Chu al., 2023; Evans et al., 2021).
Docking Probobility
Docking Probobiltty
B
Durdasping
Interfice
-991-277
Condition
Figure 1: (A). Step-wise assembly for MSP (B). Motivation for extending I-PPI to C-PPI.
Work done during an internship at IDEA Research. Corresponndece Jia Li (ji (jial ee@usst.hs .k).

---

Published as a conference paper at ICLR 2024
As the most advanced assembly-based method, MoLPC (Bryant et al., 2022) applies independent PPI (I-PPI, i.e., both proteins are independent without the consideration of other proteins) to score the quality of a given assembly. Despite great potential, it does not consider important conditions in the assembly such as the influence of third-party proteins to PPI pairs. For example, in Figure 1B, if chain has already docked chain y. the interface on that will contact with is partially occupied. Under this condition, the docking probability of (I, z) may decrease to lower than that of (y, 2). We name this observation as condition PPI, or C-PPI. In short, neglecting the influence of C-PPI may easily lead to poor generalization. In this work, we focus on assembly- based MSP by leamnngg C-PPI knowledge than I-PPI.
Learning effective C-PPI knowledge for MSP presents two main challenges. Firstly, we observe significant gaps in the C-PPI knowledge contained in multimers with var- ied scales (chain numbers), which suggests that the bi-
8000
80000
Count
ological formation process of multimers may vary de- pending on their scales. Secondly, as shown in Figure 2, experimental structure data for large-scale multimers is extremely limited, making it even more difficult for the model generalize them.
4000
2000
2
Number
Recently, the rapidly evolving prompt learning (Liu et al., 2023; Sun et al., 2023a) techniques have shown promise to enhance the generalization of models to novel tasks and datasets. Inspired by this, a natural question arises: can we prompt the model to predict C-PPIs for multimers with arbitrary scales?
Figure 2: Distribution in chain numbers of multimers from the PDB database.
To address this, our core idea is design learnable prompts transform arbitrary-scall multimers into fixed-scale ones. Concretely, we first define the target task for training (tuning) the prompt model, which is conditional link prediction. Then, we additionally design the pre-training (source) task that learns to identify arbitrary assembled multimer's correctness. In the target task, we trans- form the two query chains to virtual assembled multimer, which is input into the pre-trained model for the correctness score. We treat such score as the linking probability of the query chains. There- fore, arbitrary-see prediction in the target task reformulated as the fixed-scale one in the source task.
Empirically, we investigate three settings: (1) assembly with ground-truth dimer structures to evaluate the accuracy of predicted docking path; (2) assembly with pre-computed dimers from AFM (Evans et al., 2021); and (3) assembly with pre-computed dimers from ESMFold (Lin et al., 2023). We show improved accuracy (in RMSD and TM-Score) and leading computational efficiency over recent state-of-the-an MSP baselines methods under these 3 settings. Overall, experiments demonstrate that our method has exceptional capacity and broad applicability.
2 RELATED WORK
Multimer Structure Prediction. Proteins typically work in cells in the form of multimers. How- ever, determining the structures of multimers with biophysical experiments such X-ray crystallog- raphy (Maveyraud & Mourey, 2020; llri & Savino, 2008) and cryogenic electron microscopy (Costa al., 2017; Ho al., 2020) can be extremely difficult and expensive. Recently, the deep learn- ing (DL)-based AlphaFold-2 (Jumper et al., 2021) model can milestone-accurattly predict protein structures from residue sequences. Moreover, recent studies have explored its potential for predict- ing multimer structures. However, they mostly require time-consuming multiple sequence align- ment (MSA) operations and the performance significantly decreases for multimers with great chain numbers. Another research line assumes that the multimer structures can be predicted by adding its Sisquivel-Rodrigunn et al., 2012) and RL-MLZerD (Aderinwale et al., 2022) respectively apply the genetic optimization and reinforcement learning strategies to se- lect proper dimer structures for assembly. However, even when targeting only small-scale (3-, 4- and 5-chain) multimers, they still have low efficiency and are difficult scale up for large multimers. By assuming that the dimer structures are already provided, MoLPC (Bryant et al., 2022) further simplifies this research line with the goal to predict just the correct docking path. With the help of
chains one by one. Multi-LZerD
2

---

Published as a conference paper at ICLR 2024
additional pIDDT and dimer structure information, MoLPC has shown for the first time to predict the structures of large multimers with up to 30 chains.
Prompt Learning for Pre-trained Models. In the field of natural language processing (NLP), the prevailing prompt learning approach (Brown et al., 2020; Min et al., 2021) has shown gratifying success in transferring prior knowledge across various tasks. Narrowing the gap between the source and target tasks is important for the generalization of pre-trained models on novel tasks or data, which has not been fundamentally addressed with the pre-training fine--tnntitteeee paradigm (Zhou et al., 2022). To achieve this, researchers have turned their attention to prompts. Specifically, a language prompt refers to piece of text attached to the original input that helps guide pre-trained model to produce desired outputs (Gao et al., 2020). Prompts can be either discrete or continuous (Sun et al., 2023b; Li et al., 2023). The discrete prompt (Gao al., 2020; Schick & Schütze, 2020; Shin al., 2020) usually refers to task descriptions from a pre-defined vocabulary, which can limit the flexibility of the prompt design due to the limited vocabulary space. In contrast, learnable prompts (Li & Liang, 2021: Zhang et al., 2021; Sun et al., 2023a) can be generated in continuous space. Inspired by the success of prompt learning, we associate
ppreeinn interaction (PPI) knowledge (Kovács et al., 2019; Gao et al., 2023a), which is commonly present in multimers across various scales, to the pre- training phase. By fine-tuning only the prompt model, we can effectively adapt the PPI knowledge to the target task.
3 PRELIMINARIES
3.1 PROBLEM SETUP
Assembly Graph. We given set of chains (monomers), which is used to form protein multimer. We represent a multimer with an assembly graph G = (V, In G, for the i-th chain, obtain its chain-level embedding e; by the emhedding function proposed in Chen et al. (2019). Each node E V can thus represent one chain with node attribute c, The assembly graph is an undirected, connected and acyclic (UCA) graph, with each edge representing an assembly action.
Assembly Process. For clarity, we apply an example in Figure to illustrate the assembly process, which is achieved with the prepared dimer structures and the pre- dicted assembly graph. Let us consider a multimer with 3 chains, whose undocked 3D structures are denoted as X1, X2, Xs. We consider an assembly graph with the edge set [122, va). (V3, un)). and the dimer structures [(X|2, x, (22), (x13 x.3). (X33 x. First, we se- lect the dimer of chains 2 and 3 as the start point, i.e., X = X and X = X Next, to dock the chain onto chain 3, compute the SE(3) coordinate transfor- mation T that aligns X onto X₁. Lastly, we apply to X,, resulting in the update coordinate X of chain 1.
Guide
Start
Assemble
Assambly
Compute SE(3)
Graph
Apply T
x
x
Prepared Dimer Structures
Assembly process with the pre- dicted assembly graph and prepared dimers.
Figure 3:
Definition (Assembly Correctness) For an N-chain multimer with a 3D structure X. its chains are represented by the nodes of an assembly graph G. The assembly correctness F(G,X) equivalent the TM-Score (Zhang & Skolnick, 2004) between the assembled multimer and the ground-truth.
With the above definitions, paper aims to predict assembly graphs that maximize the TM-Scores, taking as inputs the residue sequences chains and pre-calculated dimer structures.
3.2 SOURCE AND TARGET TASKS
In this paper, we adopt pre-training (source task) and prompt fine-tuning (target task) framework to address the MSP problem. We consider two points for task designs: 1) With given multimers for pre-training, the model benefits from common intrinsic task subspace between the source and target task. 2) The target task should be designed to effectively learn the condition PPI (C-PPI) knowledge and efficiently complete the MSP inference.
3

---

Published as a conference paper at ICLR 2024
Source lank clen
Source depres
Figure 4: Analysis multimers with varied chain numbers. We select some samples for evaluation and visualize heatmaps that show the similarity of the sample embeddings obtained from different pre-trained models. Each value on the axis suggests that the model is trained on data with the specific chain number degree value. For example in the heatmap titled 'Source task: chain', the darkness at the [5.7] block represents the similarity between the embeddings extracted from two models that are trained under the source task with 5- and 7-chain multimers, respectively.
60
23
66
Thrget tade chain
Thiper tneks dogree.
44
Definition 2 (Source Data son) Each data instance in 'D son involves assembly graph Ssou and a continuous label Ysow ie., (9zour Vsou) E DD Deow- For an N --chain multimer, Ison is randomly generated its N -node UCA assembly graph, and Ysou is the assembly correctness.
Definition 3 (Target Data Dear) Each data instance in Dtar involves an assembly graph Gtar (Vior, Eter), an indicated node Ud E Vear, an isolated node Uy & Star and the continuous label Ytar, L.e,, (Gtar, Vd, Vas Mtar) E Dror- For an -cchain multimer, Gear defined as its (N 1)-chain assembly graph. Ytar is calculated by year F(((Vtar U.u.. M. {Etar U Uatuu) X).
Source Task. We design the source task as the graph-level regression task. Based on source data defined in Def. 2. the model is fed with UCA assembly graph and is expected output the con- tinuous correctness score between O and 1. Note that theoretically, we can generate N--2 different UCA graphs and their corresponding labels with an N-chain multimer. This greatly broadens the available training data, enhancing the effectiveness of pre-training in learning MSP knowledge.
Target Task. We design the target task as the link prediction task (i.e., predicting the C-PPI prob- ability). Based on target data defined Def. the target task aims to predict the presence of the link between nodes ta and Ua. which represent the docked and undocked chains, respectively.
We provide the detailed process to generate sou and Dtar in Appendix A. Overall, the source and target tasks learn assembly knowledge globally and locally, respectively. Unfortunately, multimers of varied scales exhibit distribution shifts in the target task, preventing the direct use of it for MSP. Next, we will empirically verify the existence of these shifts and their influence on MSP.
3.3 GAPS BETWEEN MULTIMERS WITH VARIED SCALES
We have presented significant imbalance in the data of multimers (see Figure 2). Here, we analyze the gap in MSP knowledge among multimers with various scales (i.c., chain numbers), which can further consolidate our motivation of utilizing prompt learning for knowledge transfer. We also offer explanations for the reasons behind the gaps based on empirical observations.
We begin by analyzing the factor of chain number. We randomly select multimers for evaluation and divide the remaining ones into various training sets based on the chain number, which are then trained with independent models. We obtain the chain representations of the evaluating samples in each model. Lastly, we apply Centered Kernel Alignment (CKA) (Raghu et al., 2021), function for evaluating representation similarity, to quantify the gaps knowledge learned by any two models. We show the heatmaps of CKA in the Figure 4 and have two observations. (a) Low similarities
are shown between data with small and large scales. (b) Generally, there positive correlation between the C-PPI knowledge gap and the difference in scales. In short, C-PPI knowledge greatly depends on the multimer scale.
To further explain these gaps, re-divide the training sets based on the degree (i.e., neighbor number of node) of assembly graphs and perform additional experiments. Specifically, we define the degree value as the highest node degree within each graph Goon in the source task, and as the
4

---

Published as a conference paper at ICLR 2024
(A) Pre-training
Graph-level
(B) Prompt Tuning
H1
Ha
π Prompt
0
Φ
Correctness
Model
Docked chain
Undocked chain
Chain embedding
Prompt embeddings
Hg
Probability
Encoder
(C) Inference
for
for Step
for Step 8
Step
Task
Prompt
Head
Tuning for x Result
Docked
7x2 14
Pick the beat
Correctness
Undocked
probabiiities
Figure 5: The overview of PromptMSP (A). Firstly, pre-train the GIN encoder and the task head under the graph-level regression task. After pre-training, given an arbitrary graph, 0* and o jointly output the correctness. (B). During prompt tuning, the prompt model takes embeddings of pair of docked and undocked (query) chains as input and learns to produce prompt embeddings which form the entire 4-node path. 8* and o² then jointly predict the correctness, which is equivalent to the linking probability. We use fø to denote the well trained pipeline that outputs the linking probability of query chains with target data instance chains, we sequentially perform 8 steps for inference. In each step, we use the well trained pipeline calculate the probabilities for all possible chain pairs and select the most possible pair to assemble.
input. (C). If the target multimer has
degree of node va be docked on in the target task. As shown in Figure CKA heatmaps indicate that training samples with different degrees exhibit the gap in knowledge, which becomes even more significant than that between data with varying chain numbers. that the gap in chain numbers between data may be primarily due to the difference in degree of assembly graphs. Therefore, we intuitively associate the degree with the biological phenomenon of competitive docking that may occur in MSP, evidenced by previous studies (Chang & Perez, 2022: Yan et al., 2020; Chang Perez, 2023). In other words, multimers with more chains are more likely to yield assembly graphs with high degrees, and consequently, more instances of competitive docking. We expect that prompt learning can help bridge this knowledge gap.
observed, we conclude
4 PROPOSED APPROACH
Overview of Our Approach. Our approach is depicted in Figure 5, which follows a pre-training and prompt tuning paradigm. Firstly, using abundant data of small-scale (i.c., 3 < N < 5) mul- timers, we pre-train the graph neural network (GNN) model on the source graph regression task. We then design the learnable prompt model, which reformulate the conditional link predic- tion (target) task to the graph-level regression (source) task. In the process of task reformulation, an arbitrary-scale multimer in the target task is converted source task. For inference, an N-chain multimer will go through - steps to be fully assembled. In each step, our model predicts the probabilities of all possible conditional links and selects the highest one to add new chain. Besides, to further enhance the generalization ability, we provide meta-learning strategy in Appendix C.5.
fixed-scale (i.e., N 4) multimer in the
4.1 PRE-TRAINING ON THE SOURCE TASK
We apply graph neural network (GNN) architecture (Xu et al., 2018; Veličković et al., 2017; Tang et al., 2023) for graph-level regression (Cheng et al., 2023; Gao et al., 2023b). Our model first computes the node embeddings of an input UCA assembly graph using Graph Isomorphism Net- work (GIN, Xu et al. (2018)) to achieve its state-of-thh-aar performance. Kindly note that we can also apply other GNN variants (Kipf & Welling, 2016; Veličković al., 2017; Tang al., 2022; Liu et al., 2024: Li et al., 2019) for pre-training. Following Def. 2, we construct source data D..o by using the oligomer (i.e., small-scale multimer) data only. The pre-training model approximates the assembly correctness with data instances of Doou:
ssu GNN 0, 0) Re F(Gsons X) Ysous
5

---

Published as a conference paper at ICLR 2024
where GNN represents combination of GIN with parameters 9 for obtaining node embeddings, ReadOut function after the last GIN layer, and a task head with parameters e to yield the prediction Ysou As defined in Def. 1, FF represents the assembly correctness function for computing TM-Score between the assembled structure and the ground-truth (GT) structure X.
We train the GNN by minimizing the discrepancy between predicted and GT correctness values:
0* = arg ming,e ; E LGan ou JED Cyre F(Gsou: X), Uson = GNN sou; 6, +)),
(2)
where Cpre is the mean absolute error (MAE) loss function. After the pre-training phase, we obtain the pre-trained GIN encoder and the task head parameterized by 0* and d", respectively.
4.2 ENSURING CONSISTENCY BETWEEN SOURCE AND TARGET TASKS
Reformulating the Target Link Prediction Task. The inference of an N-chain multimer under the source task setting requires all of its N assembly graphs and their corresponding correct- ness from the pre-trained model. Therefore, when dealing with large-scale multimers, such inference manner requires effective UCA graph traversal algorithms and significant computational resources. The target task proposed in Section 3.2 can address this issue, which aims to predict the link pres- ence (probability) between a pair of docked and undocked chains. As shown Figure 5C, we can inference the structure of an N-chain multimer with just N - steps. At each step, we identify the most promising pair of (docked and undocked) chains.
The success of traditional pre-training and fine-tuning paradigm is due to that source and target tasks having a common task subspace, which allows for unobstructed knowledge transfer (Sun et al., 2023a). However, in this paper, source and target tasks are naturally different, namely graph- level and edge-level tasks, respectively. Here, we follow three principles to reformulate the target task: (1) To achieve consistency with the source task, the target task needs to be reformulated graph-level problem. (2) Due to the distribution shifts in multimers with varied chain numbers (Fig- ure 4), a multimer of arbitrary scale in the target conditional link prediction task should be refor- mulated into a fixed-scale one in the source task. (3) The pre-trained GNN model is expected to effectively handle multimers of such "fixed-scale" in the source task. The upcoming introduction of prompt design will indicate that the fixed-scale value is 4. Therefore, to ensure (3), we limit the data used for pre-trate pre-training to only multimer of N < 5.
Prompt Design. Following Def. 3, we create the target data Deor for prompt tuning. For clar- ity, we denote each data instance as tuple form (Grar, vd: Uus )tar) E Dtar, where Star denotes the current assembled multimer (i.e., condition), Ua is a query chain within Star and vn is another query chain representing the undocked chain. We compute the last layer embeddings H of all nodes in Star and the isolated v. with the pre-trained GIN encoder. To enable communications be- tween target nodes Ud and Vas the prompt model parameterized by л contains multiple cross atten- tion layers (Vaswani al., 2017: Wang et al., 2023) that map Hu, Hy IRd to vectors Hg, Hy E Rdd which represent the initial features of nodes Ug and Vg. Finally, the pre-trained model outputs the assembly correctness of the 4-node prompt graph Spro- The whole target task pipeline of our method is represented by the equations on the right side.
H [(Pharl+1)xd = 0" ("Vaarr Un; Etar); (3)
Hg = =on (Isoftmax (H] Ha) H ])
(4)
Hy on (ssoftmax (1I Ha) HJ]
(5)
Epro (Vpro {was Uu: Vaey Uy];
(6)
Epro {eder Eagy Eynt)s
Drar 6" (0" (Gpro))
(7)
Specifically. 6* is the pre-trained GIN encoder, o² is the pre-trained task head and d denotes the dimension of features. The prompt model π, which outputs vector Rd, includes non-trainable cross attention layers and the parametric function (Multi-Layer Perceptron, MI.P a. Moreover, we use fo ,a- to represent the entire pipeline (Figure 5B) which takes input (Gtor, Cd v.) and outputs ytar A more detailed model architecture is shown in Appendix A.2.

---

Published as a conference paper at ICLR 2024
Prompt Design Intuition. First of all, the link between two query chains equivalent to the protein-protein interac- tion (PPI) in biology. We introduce the = 3 path (Kováes et al., 2019; Yuen & Jansson, 2023), a widely validated biolog- ical rule for modeling PPI probabilities. Figure 6 describes the é = 3 rule, which is based on the fact that docking-basee PPI generally requires proteins to have complementary surface rep- resentations for contact. It claims that the PPI probability of any two chains is not reflected by the number of their common neighbors (a.k.a., triadic closure principle (Lou et al., 2013; Sin- tos & Tsaparas, 2014)), but rather by the presence of a path of length 3. In short, there exists 4-node path with query chains at the ends, they are highly likely to have PPI (link).
High
2
3
probability
PPI ?
PPI
3.
1.
2.
Figure 6: The = 3 PPI rule.
Regardless of the node number of Stor. we treat the pre-trained model output based on this 4- node Gpro as the linking probability between Ug and V.. Unlike most of exsiting prompt learning techniques, the proposed task reformulation manner naturally interpretable. Let us assume that these two query chains are highly likely to have link, it will be reasonable to find two virtual chains that help form valid & = 3 path. This also suggests that the assembly of Gpro tends to be correct, i.e., o² 5000 (Gpra)) - 1. Therefore, intuitively, the correctness of Gyro fed the pre-trained model implies the linking probability between Uy and td-
4.3 INFERENCE PROCESS WITH THE PROMPTING RESULT
f=""00-00"
With prompt tuning, we obtain the whole framework pipeline J .r.. For inference on a multimer with N chains, we perform N - 1 assembly steps, in each of which we apply the pipeline fx- 16-10-.0 & to predict the linking probabilities of all pairs chains and select the most likely pair for assembly.
5 EXPERIMENTS
Datasets. We collect all publicly available multimers (3 < N < 30) from the Protein Data Bank (PDB) database (Berman et al., 2000) on 2023-02-20. Referring the data preprocessing method in MoLPC (Bryant et al., 2022), we obtain a total of 9,254 multimers. To further clarity, we the abbreviation PDB-M to refer to the dataset applied in this paper. Overall, the preprocessing method ensures that PDB-M contains high resolution, non-redundani multimers and is free from data leakage (i.e., sequences with similarity greater than 40% between training and
sets). Due to the commonly low efficiency of all baselines, we define a data split for 3 < N < 30 with small test set size enable comparison. Specifically, we select 10 for each scale of 3 S N < 10, and for each scale of 11 < N <. 30. Morcover, for comprehensive evalaution, we re-split the PDB- M dataset based on the release date of the PDB files to evaluate our method. Detailed information about the data preprocessing methods and the data statistic of split is in Appendix B.
Baselines and Experimental Setup. We compare our PROMPTMSP method with recent deep learning (DL) models and traditional software methods. For DL-based state-of-the-art methods, RL-MLZZer (Aderinwale et al., 2022) and AlphaFold-Multing (AFM) (Evans et al., 2021) are included. For software methods, Multi-LZerd et al., 2022) are included.
et al., 2012) and MoLPC (Bryant
Since assembly-based methods require given dimers, we first use the ground-truth (GT) dimer struc- tures (represented as GT Dimer) to evaluate the assembled multimer structures. For the pair of chains with contact, GT Dimer includes their native dimer structure drawn from the GT multimer. For those without contact, we use EquiDock (Ganea et al., 2021) for outputting dimer structures due its fast inference speed. Moreover, since GT dimers are not always available, for pratical reasons, we consider to prepare dimers with AFM (Evans et al., 2021) (AFM Dimer) and ESM- Fold (Lin et al., 2023) (ESMFold Dimex). For baselines not requiring given dimers, we use these three kinds of dimers to reassemble based on the docking path mined in their predicted multimer, which is referred to as the i version of the baselines. Our experiments consist of 3 settings: 1) Since not handle multimers with chain numbers N 10. We follow GT Dimer, AFM
most baselines D Dimer and ESMF old Dimer to evaluate all baselines on the small-scale multimers (3 < N < 10) in the test set. 2) We evaluate MoL.PC and our method by using these three types of dimers the

---

Published as a conference paper at ICLR 2024
Table 1: Multimer structure prediction results. Methods are evaluated on the test set of <
N S 10 by using three types of pre-computed dimers. The test set includes 80 multimer samples in total (10 samples for each scale). For each dimer type and metric, the best method bold and the second best is underlined. i represents the reassembly version of baselines. R(Avg): average RMSD; R(Med): median RMSD; T(Avg): average TM-Score; T(Med): median TM-Score.
GT Dimer
AFM Dimer
ESMFold Dimer
Methods
((Avg)/R(Med) T(Avg)/T(Med) ( RKAgg//MMed T(Avg)/T(Med) R(Avg)/R(Med) T(Avg)T(Mee)
Multi- LZer
31.50/ 33,94
0.28 0.25
31.50/33.94
0.28 0.25
31.50/ 33.94
0.28 / 0.25
Multi-LZerD
18.90 19.30
0.54 0.38
29.68 27.96
0.30/ 0.33
33.00/31.07
0.25 0.29
RI-MI.ZerD
31.04 27.44
0.29 / 0.32
31.04 /
0.29 0.32
31.04 / 27.44
0.29 / 0.32
RL-MLZerDi
17.77 / 17. .69
0,51 0.55
28.57 / 26.20
0.30 0.35
27.76 32.91
0.32 0,25
AFM
20.99 24.76
0.47 0.42
20.99 / 24.76
0,47 0,42
20,99 24.76
0.47 0.42
AFM!
16.79 16.02
0.59 0.59
18.98 / 19.05
0.50 / 0.48
26.76 / 29.95
0.33 0,30
Mol PC
18.53 18.08
0.52 / 0.55
23.00 / 23.92
0.43 / 0.42
30.17 / 29.45
0.31 / 0..1
PromptMSP
13.57 11.74
0.67 / 0.71
17.36 / 17.09
0.55 / 0.56
22.55 / 24.85
0.45 0.37
(A)10 1.0
(B)
(C)
0.8
TM-Scare
0.6
0.6
TM TMM-Score
0.8
0.4
0.2
0.2
0.
R: 0.96
R: 0.63
0.0
0.0
MPLPC-AFM
McLPC-GT
Average C-PPI
Average I-PPI
Figure 7: (A). TM-Score distribution of MoLPC and our method tested multimers of 10 S N & 30. The mean and median values are marked with white and black 'x', respectively. (B). The relationship of learned C-PPI knowledge and the actual TM-Score. (C). The relationship of I-PPI learned by MoLPC and the actual TM-Score. We show the Pearson's 's correlation R for both.
entire test set (3 N < 3) We additionally split the PDB-M dataset based on the release date of multimers to evaluate the generalization ability of our method. We run all methods on A100 SXM4 40GB GPUs and consider exceeding the memory limit the resource of 10 GPU hours as failures, which are padded by the upper bound performance of all baselines.
Evaluation Metrics. To evaluate the performance of multimer structure prediction, we calculate the root-mean-square deviation (RMSD) and the TM-score both at the residue level. We report the mean and median values of both metrics.
Multimer Structure Prediction Results. Model performance on multimers of two kinds of scales (3 < N < 10, 11 N < 30) are summarized in Table I and Figure 7A, respectively. For small-scale multimers, our model achieves -tate-of-the-art on all metrics. In addition, we find that most MSP methods can benefit from the reassembly of GT AFM dimer structures. No- tably, our model can significantly outperform MoLPC, even though it does not require additional PIDDT information or coarse information for protein interactions. For larger-scale multimers, our model also outperforms MoLPC, and outputs completely accurate prediction results for certain sam- ples (i.e.. TM-Score 1.0 under GT Dimer). As for the failed inference samples of MoLPC, we relax the model's penalty term to successfully obtain the predictions instead of simply considering its TM-Score as O. Despite this, our model can still achieve significant improvements under Dimer, AFM Dimer and ESMFold Dimer. The experimental results under the data split based on the release dates Appendix C.I.
8

---

Published as a conference paper at ICLR 2024
Table 2 shows the inference efficiency of all baselines. As assembly-base methods require given dimer struc- tures, report the separate rumning time for predicting the docking path and preparing dimers, as well as the total time consumption. Kindly note that during infer- ence, our method predicts the docking path without the need for pre-computed dimers. Therefore, to predict the structure of an N-chain multimer, our method (always) requires N
Table 2: Efficiency comparison (average MSP inference time).
Time(inin)
10
s Dince
Path
Dinge
Total
Path
Total
Mulni I IZerD
187.51
187.50
RedD
173.88
173.88
APM
155.72
McLPC
11.64
165.73
177.37
[1.64
354,23
365.87
Ours-Gr
0.01
0.01
0.04
0.04
pre-computeds dimers. We note that re- gardless of the dimer type used, our method is signifi- cantly faster than the other baselines. Our method also achieves higher efficiency in predicting the docking path compared to MoL.C. We provide more docking path inference results of our method (in Figure 9 in Appendix). We can find that as the scale increases, the inference time for single assembly process (the orange curve) of our method does not increase, which suggests that the applicability of our model is not limited by the scale.
Ours-APH
0.01
80.79
80.80
0.04
187.44
187.48
Ours-CSMFe14
0.01
0.35
0.36
0.01
1,09
1,10
Ablation Study. We perform ablation study in Ta- ble 3 to explore the significance of the prompt model and the C-PPI modelling strategy. If we remove the prompt model and apply the link prediction task both for pre-training and fine-tuning, the performance will greatly decrease by about 21.6% on large-scale multi- mers. This implies the contribution of prompt in unify- ing the C-PPI knowledge in multimers of different scales. Similarly, the significance of applying C-PPI modelling can be further illustrated through its relationship with the MSP problem. Fig- ure 7(BC) indicates that I-PPI will bring negative transfer to the MSP task, ultimately hurting the performance.
Table 3: Ablation study with GT dimers.
Prompt C-PPI 3 N S 10
11 S N 30
0.55(-17.9%)
0.299-21.6%
0.54(-19.4%)
0.33(-10.8%)
0.67
0.37
In Figure 8, we show the generalization ability of our method. The term 'w'o prompt' refers to the direct use of GNNs for conditional link prediction for MSP. We find that when introducing training mul- timers with the scale (c.g., N > I1)
0.50
WiS proeree
PrompshiS7
TM. Score
0.55
0.50
differs significantly from the testing mul- timers (i.e., = 7), the performance of the 'w/o prompt' method notably de- clines. Conversely, for PROMPTMSP, adding arbitray-scale multimers the training set will improves the model's gen- eralization ability. This indicates that our model can effectively capture shared knowledge between varied-scale multimers, while blocking the knowledge gaps caused by distribution shifts.
0.10
1 6.8
4- 3
3-22 3-30
Scalo(s) multimers for training
Figure 8: Results tested on N=7. We train our model and its 'w/o prompt' version on multimers of varied scale ranges.
CONCLUSION
Fast and effective methods for predicting multimer structures are essential tools to facilitate protein engineering and drug discovery. We follow the setting of sequentially assembling the target multimer according to the predicted assembly actions for multimer structure prediction (MSP). To achieve this, our main goal is to learn conditional PPI (C-PPI) knowledge that can adapt to multimers of varied scales (i.e., chain numbers). The proposed pre-training and prompt tuning framework can successfully narrow down the gaps between different scales of multimer data. To further enhance the adaptation of our method when facing data insufficiency, we introduce a meta-learning framework to learn a reliable prompt model initialization, which can be rapidly fine-tuned on scarce multimer data. Empirical experiments show that our model can always outperform the state-of of of-the-rrt art MSP methods in terms of both accuracy and efficiency.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
This work was supported by NSFC Grant No. 62206067, HKUST-HKUST(GZ) 20 for 20 Cross- campus Collaborative Research Scheme C019 and Baingghou-HKUSTTCCE Joint Funding Scheme 2023A03J0673, in part by grants from the Research Grant Council of the Hong Kong Special Ad- ministrative Region, China (No. CUHK 14217622).
REFERENCES
Tunde Aderinwale, Charles Christoffer, and Daisuke Kihara. Rl-mizerd: Multimeric protein docking using reinforcement learning. Frontiers in Molecular Biosciences, 9:969394, 2022.
Helen M Berman, John Westbrook, Zukang Feng, Gary Gilliland, Talapady N Bhat, Helge Weissig, Ilya N Shindyalov, and Philip Bourne. The protein data bank. Nucleic acids research, 28(1):
235-242, 2000.
Tom Brown, Benjamin Mann, Nick Ryder. Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Patrick Bryant, Gabriele Pozzati, Wensi Zhu, Aditi Shenoy, Petras Kundrotas, and Arne Elofsson. Predicting the structure of large protein complexes using alphafold and monte carlo tree search. Nature communications, 13(1):6028, 2022.
Liwei Chang and Alberto Perez. Alphafold encodes the principles to identify high affinity peptide binders. BioRxiv, pp. 2022-03, 2022.
Liwei Chang and Alberto Perez. Ranking peptide binders by affinity with alphafold. Angewandte Chemie, (35(7))e20221336,, 2023.
Muhao Chen, Chelsea J-T Ju, Guangyu Zhou, Xuelu Chen, Tianran Zhang, Kai- Wei Chang, Carlo Zaniolo, and Wei Wang. Multifaceted protein-protein interaction prediction based siamese residual renn. Bioinformatics, 35 35(14):305-13144 2019.
Jiashun Cheng, Man Li, Jia Li, and Fugee Tsung. Wiener graph deconvolutional network improves graph self-supervised learning. In AAAI, pp. 7131-7139, 2023.
Lee-Shin Chu, Jeffrey A Ruffolo, Ameya Harmalkar, and Jeffrey J Gray. Flexible protein-protein docking with a multi-track iterative transformer. bioRxiv, 2023.
Tiago RD Costa, Athanasios Ignatiou, and Elena V Orlova. Structural analysis of protein complexes by cryo electron microscopy. Bacterial Protein Secretion Systems: Methods and Protocols, pp- 377-413 2017.
Juan Esquivel-Rodrigue Yifeng David Yang, and Daisuke Kihara. Multi-lzerd: multiple protein docking for asymmetric complexes. Proteins: Structure, Function, and Bioinformatics, 80(7): 1818-1833, 2012.
Richard Evans, Michael O'Neill, Alexander Pritzel, Natasha Antropova, Andrew Senior, Tim Green, Augustin Zidek, Russ Bates, Sam Blackwell, Jason Yim, et al. Protein complex prediction with multimer. biorxiv, pp. 2021-10, 2021.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135. PMLR, 2017.
Octavian- Eugenn Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi Jaakkola, and Andreas Krause. Independent se e3uiqvarar models for end-to-end rigid protein docking. arXiv preprint arXiv:2 / 1.07766. 2021.
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012. 15723, 2020.
10

---

Published as a conference paper at ICLR 2024
Ziqi Gao, Chenran Jiang, Jiawen Zhang, Xiaosen Jiang, Lanqing Li, Peilin Zhao, Huanming Yang, Yong Huang. and Jia Li. Hierarchical graph learning for protein protein-protein interaction. Nature Communications, 14(1):1093, 2023a.
Ziqi Gao, Yifan Niu, Jiashun Cheng, Jianheng Tang, Lanqing Li. Tingyang Xu, Peilin Zhao, Fugee Tsung, and Jia Li. Handling missing data via max-entropy regularized graph autoencoder. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 7651-7659, 2023b.
Usman Ghani, Israel Desta, Akhil Jindal, Omeir Khan, George Jones, Nasser Hashemi, Sergey Kotelnikov, Dzmitry Padhorny, Sandor Vajda, and Dima Kozakov. Improved docking of protein models by combination of alphafold2 and cluspro. Biorxiv, pp. 2021-09, 2021.
Chi-Min Ho, Xiaorun Li, Mason Lai, Thomas C Terwilliger, Josh R Beck, James Wohlschlegel, Daniel E Goldberg, Anthony WP Fitzpatrick, and Z Hong Zhou. Bottom-up structural proteomics: cryoem of protein complexes enriched from the cellular milieu. Nature methods, 17(1):79-85, 2020.
Andrea Ilaii and Carmelinda Savino. Protein structure determination by x-ray crystallography. Bioinformatics: Data, Sequence Analysis and Evolution, pp. 63-87, 2008
Yuval Inbar, Hadar Benyamini, Ruth Nussinov, and Haim Wolfson. Prediction multimolecular assemblies by multiple docking. Journal of molecular biology, 349(2):435 2005.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-58) 2021.
Mohamed Amine Ketata, Cedrik Laue, Ruslan Mammadov, Hannes Stärk, Menghua Wu, Gabriele Corso, Céline Marquet, Regina Barzilay, and Tommi Jaakkola. Diffdock-pp- Rigid protein- protein docking with diffusion models. arXiv preprint arXiv:2304.03889 2023.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. arXiv preprint arXiv: 1609000700 2016.
István A Kováes, Katja Luck, Kerstin Spirohn, Yang Wang, Carl Pollis, Sadie Schlabach, Wenting Bian, Khe-Kyum Kim, Nishka Kishore, Tong Hao, et al. Network-based prediction of protein interactions. Nature communications, 10(1):1240, 2019.
Jia Li, Yu Rong, Hong Cheng, Helen Meng, Wenbing Huang, and Junzhou Huang. Semi-supervised graph classification: A hierarchical graph perspective. In The World Wide Web Conference, pp- 972-982, 2019.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint wwXiv:2101.00110 2021.
Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. A survey of graph meets large language model: Progress and future directions. arXiv preprint 1rX.1:231..12399, 2023.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 37966637):1123-11 2023.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey prompting methods in natural language pro- cessing. ACM Computing Surveys, 55(9):1-35, 2023.
Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao Fugee Tsung. Jia Li, and Yu Rong. Improving generalization in equivariant graph neural networks with physical induc- tive biases. In The Twelfth International Conference on Learning Representations, 2024. URL pettps://ppereeie net 20

---

Published as a conference paper at ICLR 2024
Tiancheng Lou, Jie Tang, John Hopcroft, Zhanpeng Fang, and Xiaowen Ding. Learning to predict reciprocity and triadic closure in social networks. ACM Transactions on Knowledge Discovery from Data (TKDD), 7(2):1 2013.
Yujie Luo. Shaochuan Li, Yiwu Sun, Rujjia Wang, Tingting Tang, Beiqi Hongdu, Xingyi Cheng, Chuan Shi, Hui Li, and Le Song. xtrimodock: Rigid protein docking via cross-modal representa- tion learning and spectral algorithm. bioRxiv, pp. 2023-02, 2023.
Laurent Maveyraud and Lionel Mourey. Protein x-ray crystallography and drug discovery. Molecules, 25(5):1030, 2020.
Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. Recent advances in natural language processing via large pre-trained language models: survey. ACM Computing Surveys, 2021.
Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy.
Do vision transformers see like convolutional neural networks? Advances in Neural Information Processing Systems, 34:121 1221116-22288 2021.
Timo Schick and Hinrich Schütze. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.0760 2020.
Taylor Shin, Yasaman Razeghi, Robert Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010. 1 15980, 2020.
Stavros Sintos and Panayiotis Tsaparas. Using strong triadic closure to characterize ties in social networks. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1466-1475, 2014.
Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference knowledge discovery & data mining (KDD 23), pp. 2120-2131, 2023a.
Xiangguo Sun, Jiawen Zhang, Xixi Wu, Hong Cheng, Yun Xiong, and Jia Li. Graph prompt learn- ing: A comprehensive survey and beyond. arXiv preprint 1.Xiv:231111653: 2023b.
Jianheng Tang, Jiajin Li, Ziqi Gao, and Jia Li. Rethinking graph neural networks for anomaly detection. In International Conference on Machine Learning, pp. (0076-21089, PMLR, 2022.
Jianheng Tang, Fengrui Hua, Ziqi Gao, Peilin Zhao, and Jia Li. Gadbench: Revisiting and bench- marking supervised graph anomaly detection. In Thirty-seventh Conference on Neural Informa- tion Processing Systems, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention you need. Advances in neural informa- fion processing systems, 30, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10900 2017.
Yiqun Wang, Yuning Shen, Shi Chen, Lihao Wang, Fei Ye, and Hao Zhou. Learning harmonic molecular representations riemannian manifold. arXiv preprint arXiv:2303.15520, 2023.
Keyulu Xu, Weihua Hu. Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826. 2018.
Yumeng Yan, Huanyu Tao, Jiahua He, and Sheng- You Huang. The hdock server for integrated protein-protein docking. Nature protocols, 15(5):1829-1852 2020.
Ho Yin Yuen and Jesper Jansson. Normalized b3-based link prediction in protein protein-protein interaction networks. BMC bioinformatics, 24(1):59, 2023.
12

---

Published as a conference paper at ICLR 2024
Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. Differentiable prompt makes pre-trained language models better sho-shot learners. arXiv preprint arXiv:2108.131100 2021.
Yang Zhang and Jeffrey Skolnick. Scoring function for automated assessment of protein structure template quality. Proteins: Structure, Function, and Bioinformatics, 57(4))702-710, 2004.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 168 16-16825, 2022.
13

---

Published as a conference paper at ICLR 2024
A IMPLEMENTATIONS
A.1 DATA PREPARATIONS
Preparing Assembly Graphs. We start from the residue sequences of the given N chains to form a multimer. We denote the j-th residue of the i-th chain as sj. The embedding function pro- posed in Chen et al. (2019) produces initial embedding for each residue, denoted vector E(s)). Specifically, the embedding vector is concatenation of two sub-embeddings, which measure the residue co-oceurrence similarity and chemical properties, respectively. We average all residue em- bedding vectors of each chain to obtain the chain-level embedding vectors, i.c., C = J Σ, Eis)), V1 < i < N, where n, denotes the residue number of the i-th chain. As for a specific multimer, we create the assembly graph Geon (Veou: Eoou) whose node attribute represents the pre-computed chain-level embeddings {chhsigN. Subsequently, according to Algorithm we randomly gener- ated the edge for the multiemrs. In short, we randomly generate several UCA graphs based on the number of nodes (chains).
Alg Algorithm Formation of Eoon-
Initialization; P1 while NV + 0 do
(iNN" pr, 1, -
Select dt from N N N {qp}
y + U
Evou f- Esou {p))
1: + h: ++1
Select Pş. from y
end while
Preparing the Source Task Labels. We denote the 3D unbound (undocked) structures of N chains form multimer as {X.hssiN. In advance, we prepare the set of dimer structures {(Xa,, hsosssss For an input assembly graph with the edge index set ((e))) (2) JhigNN-, we follow the Algorithm to obtain the corresponding label.
Algorithm Calculation of Ysou-
Initialization: X' (l) +- X.(1)
for (e(1), (2) in Esou do
Calculate transformation: T Kabsch( X X Apply T: X 0(2) (1 end for Output: (A-SSooe((x (X.MssKs)
x e(1)
6(2)
Preparing Target Data. For an N-chain multiemr, the data for the target task consists of correctly assembled graphs with less than N nodes and one the remaining nodes. For convenience, we randomly generated multiple assembly graphs with less than N nodes and kept those labeled as 1.0. For each graph, we randomly add one of the remaining nodes and calculated the new label for assembly correctness, which is the final label for the target task. Algorithm show the process to create target dataset using one multimer with N chains. For each element (G) in the output set B., nodes the ends of the last added edge in G are Us and u.. respectively. Each element in Y, represents the label Star- We use the output of Algorithm 3 to prepare each data instance (Grer. Ud: Vn, )tar) D Prar-
14

---

Published as a conference paper at ICLR 2024
Algorithm 3 Preparation for Diar.
Initialization:
Diar f- 0, S. f {1, 3, ..., ... ] (undocked chain set), Sa + (docked chain set),
Start + Sample 1) (starting chain), B3 + 0 (set of best assembly graphs), Y, t- (set of target labels)
B, < B, [Vstart }
Sa - S4 U [Start]
for 1
do for G in B, do Update S,, and Sa for in S2 do for vu S. do
Calculate the TM-Score y with Algorithm 2
if y > 0.99 then
G 6 GU (Uny Eud
B, + B, U{G
Y, Y, (g)
end end for end for end for end for for in B, do
Update S, and Sa for vu in So do for in S. do
Calculate the TM-Score y with Algorithm 2
G + GU {Vus End)
B, + B, [f}
Y, - Y {y)
end for end for end for
Output: B, and Y,
A.2 MODEL ARCHITECTURE
GNN Model for Pre-Training. We apply source dataset son to pre-train the graph regression model. We denote H: the embedding of node after the k-th GIN layer. Therefore, we have the following output with each layer in the GIN encoder.
(L) (a+ (1+c(tt) H.! (1
H (k)
Σ
(()
(8)
mEN(()
where (1)) represents the learnable parameter of the k-th layer.
Finally, we have the GIN encoder output with 'sum' graph-level readout for the last layer:
H READOUT ( (1H(2) sis
where MLP denotes the Multilayer Perceptron and L means the total layer number.
Prompt Model. For a data instance (Gtar, Vds Var ytar) E Dtar in the target dataset, we consider Va isolated node in graph Ghor (Viar Vus Etor). The pre-trained GIN encoder computes the node embedding matrix H for G We obtain the prompt embeddings with a cross attention module:
H. = ox (Isoftmax (H) Hv) H !
(10)
15

---

Published as a conference paper at ICLR 2024
Hg = ox ([softmax (H, Ha) H
(11)
where (x) is parametric function (3-layer MLP).
A.3 HYPERPARAMETERS
The choice of hyperparameters of our model shown in Table 4.
Table 4: Hyperparameter choices of PROMPTMSP.
Hyperparameters
Values
Embedding function dimension (input)
13
GIN layer number K
Dimension
Eq.
1024, 1024
Dimension in Eq.
256,
Dropout rate
0.2
Number of attention head
Source/target hatch-size
512, 512
Source/targe learning rates
0.01, 0.001
Task head layer number
Task head dimension
256,
Optimizer
Adam
B DATASET
The overall statistic of our dataset PDB-M is shown in Table 5. Overall, we have obtained 9,254 non-redundant multimers after processing and filtering.
Download all of the multimer structures as their first assembly version
Remove multimers whose resolution NMR structures less than 3.0
Remove the chains whose residue number is less than 50
If more than 30% of the chains have already been removed from the multiemr, the entire multiemr will be removed.
Remove all nucleic acids
Cluster all individual chains on 40% identity with CD-HIT (https://gitt com/ we i zhongl i //chiit)
Remove the multimer if all of its chains have overlap with any other multimer (remove the subcomponents of larger multimers)
Randomly select multimers to form the test set and the remaining multimers for training and validation.
Kindly note that due to the generally lower efficiency of the baseline, the size of the test set we divided was relatively small. Moreover, we show the experimental results with a data split according to release date in the next section.
Table 5: Statistics of PDB-M
Table 6: Dataset split based
N
Train
Valid
Test on the released date
Before (train)
After
3
1325
265
10
Date
(test)
4
942
188
10
5
981
196
10
1-1
459
8786
6-10
3647
730
50
2004-1-1
1056
8198
11-15
267
53
25
2008-1-1
2091
7163
16-20
198
40
25
2012-1-1
3665
5589
21-25
135
27
25
2016-1-1
4780
4474
26-30
66
14
25
2020-1-1
7002
2252
2024-1-1
9454
Total
7561
1513
180
16

---

Published as a conference paper at ICLR 2024
C ADDITIONAL EXPERIMENTAL RESULTS
C.I DATA SPLIT WITH RELEASE DATE.
We show the results with 6 thresholds of release dates. Using them, have 6 types of data split based on the entire PDB-M. The data split statistic is shown in Table 6. As the datasets all contain large-scale multimers, we show the comparison only between our method and MoLPC in Table 7.
Table 7: Model performance under the data split based released dates (3 N S 30). Threshold represents the boundary separating the training (before) and test (after) sets.
Date
2000-1-1
2004-1-1
2008-1-1
2012-1-1
2016-1-1
2020-1-1
Avg.
Threshold
Metric
TM-Score (mean) / TM-Score (median)
Ours(GT)
0.27 /..24
0.42/
0.42 0.42
0.47 / 0.50
0.52 0.49
0.57 / 0.54
0.45 / 0.42
Ours(ESMFold)
0.31 /0.28
0.33 0.29
0.34/ 0.36
0.36 / 0.36
0.38 0.38
0.37 /0.41
0.35 / 0.35
C.2 RUNNING TIME OF PROMPTMSP
We provide docking path inference results of our method in Figure 9. We can find that the scale increases, the inference time for a single assembly process (the orange curve) of our method does not increase, which suggests that the applicability of model is limited by the scale.
Time
Time'Seale
11
23
29
Scale
Figure 9: Inference running time of our method tested various scales of multimers.
C.3 THE ROLE OF OUR META-LEARNING FRAMEWORK
We test the performance of our method in extreme data scarcity scenarios. In Table 8, data ratio means the proportion randomly retained multimer samples with chain numbers greater than 10. For example. 10% suggests that we only use 10% of the large-scale multimer data in PDB-M for training. can be seen that the performance of our model decreases with the degree of data scarcity. However, even with only 10% of the training data retained, our method can still slightly outperform MoL.PC This imples that our method can effectively generalize knowledge from data with fewer chains, without strong reliance the amount of large-scale multimer data.
Data ratio
80%
60%
40%
20%
10%
Metric
TM--corr(mmaan) (MM-Score(media)
MoI.PC
0.47 0.45
PromptMSP
0.57 0.60
0.55 / 0.53
0.58 / 0.55
0.53 / 0.53
0.49 / 0.47
Table 8: Performance with less training samples.
17

---

Published as a conference paper at ICLR 2024
C.4 VISUALIZATION
In Figure 10, demonstrate that PromptMSP successfully assemble unknown multimers, where no chain has a similarity higher than 40% with any chain the training set.
PDB: 6XBL(N=5)
TM-Score:1.0
TM-Score:0.58
PDB: 5X0G(N=15) TM-Score:0.88
TM-Scorr:0.44
Ground-truth
Ours
AFM
Ground-truth
MoLPC
Figure 10: Visualization of multimers with chain numbers of 5 and 15. They are both successfully predicted by PROMPTMSP. For 5XOG, our model correctly predicted 12 out of 14 assembly actions.
C.5 PROMPT TUNING WITH META-LEARNING
Inspired by the ability of meta-learning learn an initialization on sufficient data and achieve fast adaptation scarce data, follow the framework of MAML (Finn et al., 2017) to enhance the prompt tuning process. Specifically, we use small-scale multimers (sufficient data) to obtain reliable initialization, which is then effectively adapted to large-scale multimers (scarce data).
Following Def. 3, we construct datasets ((ema) and D(far) using data of small-scale (NV < 7) and large-scale multi- mers (N 2 8), respectively. Let fnjo-_a- be the pipeline with prompt model (rr) fixed GIN model (8*) and fixed task head (o*). In our proposed meta-learning framework, we perform prompt initialization and prompt adaptation using D (mma) and ((lrr) tar resulting two pipeline versions, fa10-,0- and fx 10 - respectively.
(smae), Sufficient Car
Support
task
Query set
Task pool
tar
Sample
Eq. 3
2
n
(P)
Prompt Initialization to obtain sxjø-.e.. The objective of prompt initialization is to learn a proper initialization of
Eq.
4
pipeline parameters such that f=jo- . can effectively learn the
π
Dilar)
Scarce π
common knowledge of (smm) and performs well on ((sma)) Before training, first create pool of tasks, each of which randomly sampled from the data points of ((sma)
Figure 11: Prompting MAML.
During each training epoch, do three things in order. 1 We draw batch of B tasks (TI,.., TB). Each task T, contains support set Di,, and a query set D4.-
We perform gradient computation and update for separately on the support sets of B tasks.
π ( = a CDz (fr(0-,0))
(12)
where (1) is π after gradient update for task T..
After obtaining B updated prompt models π (+), 1,2, 2,.,, B. the update of π for this epoch is:
B
π = V Co, (Jm1)o0
(13)
After multiple epochs in loop
@ and 3 in order), obtain the prompt model initialization म.
18

---

Published as a conference paper at ICLR 2024
We apply all data points from D D(ler) to update with
Prompt adaptation to obtain the prompt initialization ă:
1x 10-.0 .
π
C pfter) (faoor..))
(14)
With Eq. 14, obtain the prompt adaptation result π²
Inference under the MAML strategy. With prompt tuning enhanced by the meta-learning tech- nique, obtain ãे and n¹ based small- and large-scale (chain number) multimers, respectively. For inference on multimer of < N < 7, we perform N - assembly steps, in each of which apply the pipeline fajø-_a . predict the linking probabilities of all pairs of chains and select the most likely pair for assembly. For inference on multimer of N 2 8 (shown in Figure 5C), we first apply
assemble a part of the 7 chains of the multimer, and then use fa (0- o for the subsequent N 7 assembly steps.
19


Reviews:
[{"review_id": "IDjhIDh9CG", "reviewer": "Reviewer_PqHe", "summary": "This paper introduces a new algorithm to predict multimer structure with multiple chains via a pre-training and prompt tuning framework. The overall idea is novel and interesting. Different from MoLPC, where proteins docking are independent without the consideration of other protein, this method considers the influence of third-party proteins when performing docking. This paper compared several baselines on N chains datasets (N>=3). The experimental results show improvement on AlphaFold-Multimer and MoLPC. Although this paper introduces some new idea, many details are unclear. Also, the baselines are so weak and the experimental setting is not realistic. \nI vote to reject this paper.", "questions": "- when comparing with AlphaFold-Multimer, do you input monomer's ground-truth structure as the template?\n\n- how does your method perform when using predicted monomer structure? is the method robust?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Solving multimer structure prediction via pre-training and prompt tuning is interesting.\n\n- It's reasonable to consider conditional docking for multiple protein.", "weaknesses": "- defintion 1 is problematic. Because in real-world setting for docking, monomer's ground-truth structures could not be provided. So that the correctness could never be 1 in real-world setting.\n\n- the baselines are so weak. when taking ground-truth structure as input, HDock[2] and xTrimoDock[3] are strong baselines. \n\n- it could be interesting if you can compare different baselines over different the number of chains. The performance could be reduced when increasing the number of chains. \n\n- missing some related references: [1], [3], [4] [5]\n\n[1] Ghani, Usman, et.al. Improved docking of protein models by a combination of alphafold2 and cluspro.\n\n[2] Yumeng Yan, et.al. The HDOCK server for integrated protein–protein docking.\n\n[3] Yujie Luo, et.al. xTrimoDock: Rigid Protein Docking via Cross-Modal Representation Learning and Spectral Algorithm.\n\n[4] Mohamed Amine Ketata et.al. DIFFDOCK-PP: RIGID PROTEIN-PROTEIN DOCKING WITH DIFFUSION MODELS. use torsional diffusions to solve rigid protein docking, and the source code is released. \n\n[5] Lee-Shin Chu et.al. Flexible Protein-Protein Docking with a Multi-Track Iterative Transformer.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AKQldrUZUQ", "reviewer": "Reviewer_6yks", "summary": "This paper proposes a sequential protein complex assembly method called PromptMSP. In each assembly step, PromptMSP predicts where a protein should be assembled to the current complex. During training, PromptMSP learns a continuous score for a given protein assembly graph and during testing, it uses the learned score model to find the most likely assembly graph. To avoid training and testing distributional mismatch, PromptMSP employs prompt learning to reduce the gap of input formats. PromptMSP is compared with existing multimer prediction baselines and outperforms AlphaFold-multimer baseline.", "questions": "* At test time, what prompt do you provide to the model? It seems that the prompt is basically the assembly graph that model predicted. I don't see why prompt engineering is useful during training.\n* It would be helpful to report model performance for each number of chain (from 3 to 30).", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "* The proposed method outperforms AlphaFold-Multimer (AFM), which is impressive.\n* The evaluation setting is comprehensive. It includes both ground truth dimer setting and predicted dimer setting, which ensures a fair comparison with AFM.\n* Ablation studies show that each proposed component is effective.\n* Incorporation of L=3 PPI rule into the inference procedure is an interesting contribution.", "weaknesses": "* The method description is very confusing. Figure 5 is very crowded and rather uninformative.\n* It is very hard to understand what meta-learning part (section 4.3) is actually doing. A visual step-by-step illustration of prompt fine-tuning can be helpful.\n* The introduction of prompt fine-tuning seems an overkill. A simpler approach should work equally well. For example, we can adopt a standard autoregressive link prediction algorithm to this problem. In each step, you predict the link between a pair of proteins and train the model to predict the right link given different prefix graphs.\n* Analysis in section 3.3 is unclear. How did you compute Centered Kernel Alignment between two models?\n* It's unclear how a new protein is docked to the current assembly in each step. Did you use EquiDock? If so, how do you ensure that EquiDock is not trained on any of your test set instances?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sydWKlDPJr", "reviewer": "Reviewer_UpSS", "summary": "The paper treats the problem of multimer assembly: given a set of sequences, and the structure of all possible dimers (e.g. from AF2), we wish to assemble the multimer by iteratively selecting the next chain and aligning dimer structures—represented with an assembly graph. The paper proposes a multi-stage solution to this problem. (1) A GNN is pre-trained to predict the multimer TMScore from an assembly graph (2) The “next link” prediction problem is  framed as a TMScore prediction over a fictitious assembly graph, i.e., akin to “prompting” the pretrained GNN. This fictitious assembly graph is created by a “prompting model” and its design is inspired by network-based PPI prediction in bioinformatics. (3) The prompting model–which is specific for each multimer size–is obtained via meta-learning, where the meta-training tasks are small multimer sizes, and the meta-tuning tasks are large multimer sizes.", "questions": "* Can the authors confirm that there is only one pretrained model, despite the discussion in section 3.3?\n* Where are the node embeddings H is used in prompt model? Are only $H_u, H_d$ used?\n* Is there precedent for learning a prompt _model_ that generates a different prompt for each input, as opposed to simply learning a _prompt_?\n* Are the encoder parameters $\\theta$ and task head parameters $\\phi$ ever separated? If not, then denoting them separately only makes the paper more confusing.\n* How is runtime calculated? I assume the dimer structures are completed \"lazily.\" What explains the large gap in runtime relative to MCTS? It would be nice to report the total number of dimer structures \"required\" by MCTS vs the proposed method.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "* The paper proposes a novel solution to the difficult problem of multimer structure prediction. Multiple strategies are employed to make this extremely data-scarce problem tractable for deep learning. These strategies are impressive in their sophistication and the bar for originality / novelty has clearly been surpassed.\n* The experimental results are good in terms of both performance and runtime relative to the best existing methods.\n* The paper is a nice illustration of the concept of learning on top of foundation models such as AF2, a paradigm which arose in NLP and is becoming increasingly useful in biological ML.", "weaknesses": "* The paper integrates multiple technical ideas with a complex problem domain, but unfortunately the presentation is very confusing.\n   * The paper relies on many ideas that are less familiar to the average reader in protein ML. There should be an extensive background section explaining meta-learning, prompt learning, L=3 PPI prediction, etc.\n    * For a procedure with this many moving parts, it is absolutely essential to provide an explicit inference algorithm somewhere.\n    * The paper is made even more confusing by certain particular choices of emphasis which serves only to distract the reader on a first pass.\n        * It is not clear why it is important to emphasize C-PPI vs I-PPI. Perhaps the authors are trying to draw a distinction with MCTS, but this is really not necessary or within scope. Fully appreciating the difference would require a detailed explanation of the MCTS method, which the paper has no time (or need) to fully explain.\n       * The extended discussion in Section 3.3 seems disconnected from the context of the paper and serves only to make it more confusing.\n       * The authors repeatedly distinguish between oligomers and multimers based on size, which is very unconventional and should be fixed.\n    * L=3 PPI prediction is not obvious and is very confusing when referred to in-passing the first few times it is brought up. \n\n* The pipeline seems unnecessarily complicated and poorly justified. All else held equal, solutions to hard problems should be as simple as possible, and complexity (even if novel) should at least be sensible and easy to justify once understood. Here, it is really not clear why the problem requires such a complex formulation. The so-called source task is a nice way of framing the multimer assembly problem to make it much more data-rich. But then, the most natural solution would seem to be to run the TMScore predictor on all possible next-link additions to the current assembly graph. It seems quite convoluted to instead obtain a prompting model to convert each possible next-link prediction to a fictitious 4-graph when a real (N+1)-graph would also seem to work.\n\nJustification for score. Although I like the wealth of ideas presented in the paper, the presentation is too unclear and the complexity insufficiently justified to recommend acceptance in its current form.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cNPmw7OuPM", "reviewer": "Reviewer_y71p", "summary": "The authors introduce an interesting task of multimer structure prediction in the form of assembly graph where each node is a monomer and each edge represents an assembly action. They propose to first pretrain a model to predict the TM-score between the structure obtained from the given assembly graph and the ground truth, then finetune with prompt and meta-learning to perform link prediction to construct the assembly graph step by step. The prompt is crafted with $l=3$ path to form a 4-node graph so that the link prediction can be implemented as graph-level prediction on small graphs which is well aligned with the pretraining phase.", "questions": "1. Can you show the correlation between the number of chains and the node degrees to directly validate the claim \"multimers with more chains are more likely to yield assembly graphs with high degrees\" in section 3.3?\n2. How is the ablation of the C-PPI modelling strategy implemented?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well written and clear. I really enjoy reading the paper.\n2. The paper introduces an interesting task (i.e. multimer structure prediction) to the community with clear formalization (i.e. prediction of the assembly graph given pairwise dimers).\n3. The experiments are solid, testing the performance on multimers ranging from 3 chains to 30 chains. The authors also compare the results when given ground-truth dimers or alphafold-predicted dimers as inputs. The results are promising, exhibiting obvious improvement over baselines.", "weaknesses": "1. The $l=3$ graph prompt is proposed to tackle the distribution shift of chain numbers. However, I notice that in section 4.2 the initial embeddings are obtained from the last layer of the pretrained GIN encoder with the full assembly graph as input. This step may already suffer from the distribution shift and produces out-of-distribution embeddings.\n2. The ablation of the pretraining phase is missing. An experiment without pretraining should be conducted to demonstrate the necessity of the proposed pretraining strategy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)