Title: Injecting Multimodal Information into Rigid Protein Docking via Bi-level Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: complex structure prediction

rigid docking

protein docking

antibody-antigen docking
Abstract: The structure of protein-protein complexes is critical for understanding binding dynamics, biological mechanisms, and intervention strategies. Rigid protein docking, a fundamental problem in this field, aims to predict the 3D structure of complexes from their unbound states without conformational changes. In this scenario, we have access to two types of valuable information: sequence-modal information, such as coevolutionary data obtained from multiple sequence alignments, and structure-modal information, including the 3D conformations of rigid structures. However, existing docking methods typically utilize single-modal information, resulting in suboptimal predictions. In this paper, we propose xTrimoBiDock (or BiDock for short), a novel rigid docking model that effectively integrates sequence- and structure-modal information through bi-level optimization. Specifically, a cross-modal transformer combines multimodal information to predict an inter-protein distance map. To achieve rigid docking, the roto-translation transformation is optimized to align the docked pose with the predicted distance map. In order to tackle this bi-level optimization problem, we unroll the gradient descent of the inner loop and further derive a better initialization for roto-translation transformation based on spectral estimation. Compared to baselines, BiDock achieves a promising result of a maximum 234% relative improvement in challenging antibody-antigen docking problem.

Injecting Multimodal Information into Rigid Protein Docking via Bi-level Optimization

Abstract

The structure of protein-protein complexes is critical for understanding binding dynamics, biological mechanisms, and intervention strategies. Rigid protein docking, a fundamental problem in this field, aims to predict the 3D structure of complexes from their unbound states without conformational changes. In this scenario, we have access to two types of valuable information: sequence-modal information, such as coevolutionary data obtained from multiple sequence alignments, and structure-modal information, including the 3D conformations of rigid structures. However, existing docking methods typically utilize single-modal information, resulting in suboptimal predictions. In this paper, we propose xTrimoBiDock α (or BiDock for short) 4 , a novel rigid docking model that effectively integrates sequenceand structure-modal information through bi-level optimization. Specifically, a crossmodal transformer combines multimodal information to predict an inter-protein distance map. To achieve rigid docking, the roto-translation transformation is optimized to align the docked pose with the predicted distance map. In order to tackle this bi-level optimization problem, we unroll the gradient descent of the inner loop and further derive a better initialization for roto-translation transformation based on spectral estimation. Compared to baselines, BiDock achieves a promising result of a maximum 234% relative improvement in challenging antibody-antigen docking problem.

1 Introduction

Protein-protein interactions (PPIs) are essential to the basic functioning of cells and larger biological systems. Due to their importance, elucidating such interactions up to atomic detail is necessary for understanding multicomponent complexes like ribosomes and discovering protein-based drugs, e.g., antibodies and peptides. However, the experimental golden standard for determining the structure of protein complexes, such as X-ray crystallography and cryo-EM, is extremely time-consuming.

Computational protein docking [49, 7, 57, 45] provides an alternative route to predict the 3D structures of complexes from unbound states. Here, we focus on the fundamental problem of rigid protein docking [25] where no deformations occur within any protein during the docking process.

This assumption is reasonable in many biological environments and stabilizes the prediction of natural structures. Therefore, all we need is an appropriate SE(3) transformation shown in Figure 1, i.e., roto-translation transformation, that places the ligand protein at the correct orientation and location concerning the receptor protein. In this context, two types of information are available. The first is sequence-modal information, such as the coevolutionary signals captured in multiple sequence alignments (MSAs). The second is structure-modal information, like 3D coordinates of rigid bodies and bond angles. Both types of information are indispensable for performing rigid protein docking.

protein complex

T = (R ∈ SO(3),t ∈ R3)

SE(3) transformation

rigid docking

receptor protein

Classical docking software [13, 19, 46, 29, 58] generally follow a three-step framework for predicting complex structures. Firstly, a large number of candidate structures are randomly sampled to explore the conformational space of the complex. Secondly, a scoring function is employed to evaluate and rank the sampled structures based on their compatibility with the binding interface. Finally, the top-ranked structures are refined using an energy model to improve their accuracy and eliminate steric clashes. Due to the evaluation of lots of candidates, these methods tend to be computationally expensive, particularly for high-throughput workflows.

Recently, deep learning has shown significant computational speed-up in this field. EquiDock [25] has emerged as a pioneering method for applying deep learning to rigid protein docking. However, it solely relies on structure-modal information, neglecting the valuable sequence-modal information in databases. This drawback hampers its ability to capture evolutionary constraints and exploit the intricate sequence-structure relationships. Building upon the success of AlphaFold2 [28], AlphaFoldMultimer [23] has been hailed as a breakthrough in directly folding complex structures from amino acid sequences, which considers the sequence-modal information but fails to effectively utilize the given rigid structures, leading to unnatural complexes in some cases.

To overcome the aforementioned limitations, we propose xTrimoBiDock (or BiDock for short), a novel rigid docking model that seamlessly integrates sequence- and structure-modal information through bi-level optimization. (i) To effectively utilize multimodal information, we introduce a cross-modal transformer that injects multimodal information into an inter-protein distance map. (ii) To satisfy rigid docking, we optimize a roto-translation transformation to minimize the difference between the docked pose of unbound proteins and the predicted inter-protein distance map. This framework naturally lends itself to a nested bi-level optimization paradigm, where the outer loop is the learning of the cross-modal transformer and the inner loop is dedicated to solving the roto-translation transformation. Inspired by gradient-based bi-level optimization [24, 32, 11], we unroll the gradient descent of the inner loop for approximation. Due to the ruggedness of the optimization landscape, a substantial number of iterations are required for convergence. Thus, we further derive a better initialization for the roto-translation transformation using spectral estimation. Extensive experiments conducted on diverse datasets and evaluation protocols validate the effectiveness of BiDock.

In summary, our contributions are three-fold:

We effectively leverage sequence- and structure-modal information for rigid protein docking. By naturally integrating the fusion of multimodalities and the docking of rigid bodies through bi-level optimization, we set up a new avenue for solving rigid protein docking.

We solve the above bi-level optimization with unrolled gradient and spectral initialization. By unrolling the gradient descent of the inner loop and deriving a spectral estimation for initialization, BiDock enhances the convergence and controls the computational cost.

Comprehensive experiments on three representative datasets demonstrate the effectiveness of the proposed model. Compared to state-of-the-art baselines, BiDock achieves the maximum 234% relative improvement in challenging but practical antibody-antigen docking.

2 Related Work

Molecular Docking Molecular docking aims to characterize the binding poses between small molecule compounds and protein targets [42, 43, 63], playing an essential role in the discovery of effective and safe treatments for various diseases [48]. Traditional computational methods have been greatly enhanced by deep learning techniques [53], which offer increased expressive power in identifying, processing, and extrapolating complex patterns in molecular data [39, 31, 21, 35, 4, 38, 16]. It is important to note that these methods have primarily focused on molecular ligands and often assume the availability of known binding pockets, limiting their direct applicability to protein-protein docking. Protein-protein docking is more challenging due to larger, flexible proteins and the need to predict unknown binding interfaces.

Protein Docking Computational docking software [46, 47, 44, 57, 58, 45, 15] predict complex structures based on a framework of candidate sampling, ranking [37, 6, 22], and refinement [50]. These methods can be financially restrictive and time-consuming, as they often involve scoring and ranking thousands of candidate structures. Recently, deep learning has made significant contributions to structural biology [30, 17, 18, 36]. Notably, AlphaFold2 [28] and RoseTTAFold [2] have been employed to improve protein structure prediction from various angles [26, 40], such as integrating physics-based docking methods [29] or extending multiple sequence alignments [10]. Additionally, methods like AlphaFold-Multimer [23] and HSRN [27] have been developed to simultaneously fold and dock two proteins. Despite their remarkable achievements, these methods violate the rigidity of rigid docking and do not consider unbounded structures. EquiDock [25] is tailored for effective rigid docking but does not fully leverage the evolutionary information encoded in protein sequences, resulting in limited performance improvement compared to traditional docking software.

Bi-level Optimization Bi-level optimization has gained attention in the deep learning community for its ability to handle nested problem structures. It finds applications in various domains, including hyperparameter optimization [14] and metaknowledge extraction [24]. Traditional bi-level optimization methods rely on game theory [51] or mathematical programming [9], which may not scale well to large datasets or have strict mathematical requirements. Alternatively, gradient descent methods offer a promising solution, and they can be divided into explicit gradient update [60], explicit proxy update [1, 8], implicit function [41], and closed-form methods [59]. The first three are approximation methods suitable for general functions, while the last one is an accurate method specifically designed for certain functions. Recent surveys [33, 11] provide a more comprehensive review of these methods. In addition, the merging AI4Science directions, such as topology design [61] and protein representation learning [12], also present nested problem structures that can be compatible with bi-level optimization. By incorporating bi-level optimization into rigid protein docking, we anticipate that our work will have a significant impact at the intersection of these research areas.

3 BiDock Methodology

Multimodal Input The available information on the sequence modality mainly includes information inside the primary sequence itself and co-evolutionary information from MSAs. Following the existing work [28, 23], we extract type features F typ ∈ R N res × 21 and primary pair features F pp ∈ R N res × N res × 73 from the primary sequence, where N res is the number of residues. In terms of MSAs, we leverage cluster MSA features F msa ∈ R N cls × N res × 49 , where N cls is the number of cluster centers. For the structure modality, we extract angle features F ang ∈ R N res × 57 and pair features F p ∈ R N res × N res × 88 from the rigid protein structures. The angle features provide the orientation and position of each amino acid, while the pair features contain the distance information between amino acids. We present a brief introduction for features and give further details in Appendix B.

Architecture The illustration of the framework is presented in Figure 2. Given sequence-modal features { F typ , F msa , F pp } and structure-modal features { F ang , F p } , the cross-modal transformer with parameters ϕ transforms available features and integrates them to predict an inter-protein distance map ˆ D . To maintain the rigid-body assumption, we define the objective function of learning rototranslation transformation ( R, t ) based on the distance map ˆ D and coordinates of rigid proteins

outer loop

inner loop

Fpp

C(R,t,Φ)=min

1

m

n

(IIX - RY -t- Di(Φ)

R,tmn

i=1 j=1

Fmsa

(R,t)

X

Y

sequencemodality

cross-modal

transformer

FangFp

(parameters Φ)

distance map D

rigid proteins

spectral initialization (Ro,to)

predicted pose

Rt+1 = OPT(Rt,VRCt(Rt,tt,Φ))

structuremodality

tt+1= OPT(t,VtLt(Rt,tt,Φ))

hypergradient

{ X ∈ R 3 × m , Y ∈ R 3 × n } as follows

where X i is the i -th column of X and Y j is the j -th column of Y . Considering that optimizing roto-translation transformation ( R, t ) and parameters of cross-modal transformer ϕ constitutes a nested structure, we reformulate the rigid protein docking as a bi-level optimization problem

where L out refers to the outer loss for the cross-modal transformer and its specific details will be introduced in the following subsections. To solve this bi-level optimization, we unroll the gradient descent of the inner loop to approximate ( R ∗ ( ϕ ) , t ∗ ( ϕ )) by

where OPT represents the optimization algorithm, such as the stochastic gradient descent (SGD). However, due to the complexity of the problem and the rugged optimization landscape, the gradient descent in the inner loop often requires a large number of iterations to converge and faces challenges in finding the global minima. Therefore, we further derive spectral initialization to provide a more favorable starting point for better convergence.

3.1 Cross-modal Transformer

Based on the multimodal features, the cross-modal transformer predicts an inter-protein distance map, as depicted in Figure 3. We first use multilayer perceptrons (MLPs) to capture the nonlinear relationship inside features and project them into the same space

where { P pp , P p ∈ R N res × N res × c z } are transformed pair features, { M typ , M ang ∈ R N res × c m } and M msa ∈ R N cls × N res × c m are transformed intra-sequence features. Then we add pair features as the input pair feature P of Evoformer [28]

Similarly, we integrate intra-sequence features as another input

sequencemodality

W

nonlinear

+

concat

residue featureFtyp

evolution representation

M

nonlinear

Mmsa

linear&softmax

MSA feature Fmsa

ppp

nonlinear

evoformer

pairrepresentation

P

primary pair feature Fpp

distance map D

structuremodality

pair featureFp

nonlinear

pp

linear

Mang

Ldist

angle feature Fang

nonlinear

where ∥ is the concatenation. Please note that the broadcast operation is used to make the dimensions consistent. Specifically, M typ is broadcasted along the newly added first dimension during addition. Similarly, broadcasting is applied to the newly added first dimension of M ang for concatenation. Through Evoformer, pair representation ˆ P and evolution representation ˆ M are obtained

Finally, we utilize pair representation ˆ P to predict the inter-protein distance map. Here we define two loss functions to supervise the learning of distance map and evolution representation. Concerning the distance map, we can directly calculate the ground truth from rigid proteins. However, this ground truth is naturally noisy because of the experimental resolution. In light of successful practice in protein structure prediction [28, 23], we use a discretized distance map to replace the exact one. Specifically, distances are discretized into 64 bins ranging from 2 to 22 Å. The prediction of discretized distances is converted into a classification problem by

where W is the learnable parameter and σ is the activation function. Then the cross-entropy loss averaged over all residue pairs is

where ¯ G k ij represents the k th-element of one-hot encoding of discretized actual distance. It is worth noting that the predicted distance map ˆ D can be obtained by using the mean of each bin.

Inspired by the masked language model [20], we leverage the evolution representation to reconstruct masked MSA values. We consider 23 classes, including 20 common amino acid types, an unknown type, a gap token, and a mask token, and introduce the mask policy in Appendix B. Thus, a masked MSA loss can be defined as follows

where N mask denotes the number of masked tokens and A is the ground truth.

Similar to Equation (1), the SE(3) transformation optimized in the inner loop will exhibit differences from the ground truth complex, generating hypergradients through the cross-modal transformer

where D is the ground truth distance between amino acids. Overall, the outer loss for the cross-model transformer is

where hyperparameters λ 1 , λ 2 and λ 3 balance the importance of different loss terms.

3.2 Unrolled Algorithm for Hypergradient

To solve the bi-level optimization formulated as Equation (2), hyper gradient d ϕ L out is required in the outer level and can be unrolled via the chain rule

One method to calculate it is approximating ( R ∗ ( ϕ ) , t ∗ ( ϕ )) via optimizer

where T denotes the number of iterations. We explicitly calculate the gradients

where rotation matrix R and quaternion q are converted to each other through the Bar-Itzhack algorithm [5], and ⊙ means Hadamard product. After T rounds of iterations, ( R ∗ ( ϕ ) , t ∗ ( ϕ )) can be approximated as ( R T , t T ) . We can compute the hypergradient by substituting

Due to the vast search space and the rugged landscape of the loss, the gradient descent algorithm with random initialization often requires a large number of iterations to converge and struggles to find the global minima. In the next subsection, we will derive a spectral initialization to enhance convergence.

3.3 Spectral Initialization

Recall that we intend to derive a good initialization ( R 0 , t 0 ) for the gradient descent of Equation (1). To simplify it, we denote ˆ Y = RY -t1 ⊤ n , where 1 n is the all-ones vector. Then substitute ˆ Y into

According to these variable combinations, we define four variables and two centering matrices

where I represents the identity matrix and J refers to the all-ones matrix. If the condition ∀ i, j, ∥ X i -ˆ Y j ∥ = ˆ D ij holds, the following equation exists

By performing singular value decomposition (SVD) for H m X ⊤ and Y H n respectively

the rotation matrix can be solved as

Given the rotation matrix R , the translation vector t can be gotten using

By replacing random initialization with the above spectral initialization, the gradient descent will approach the global minima faster and better. We will empirically verify this conclusion through subsequent experiments.

4 Experiments

Datasets We leverage Docking Benchmark 5.5 (DB5.5) [52], a gold standard dataset tailored for rigid docking. Following the experimental setting of EquiDock [25], the dataset is randomly partitioned into a test split of size 24. For a comprehensive comparison, we curate two datasets of antibodies (VH-VL) and antibody-antigen complexes (AB-AG) from Protein Data Bank (PDB) [3] and expect them to become new benchmarks. Specifically, the training set consists of 4,890 complexes containing at least one antibody chain, while the test set comprises 68 antibody-antigen complexes released after October 2022. The docking results of variable heavy-light chains and antigen-antibody can be separately evaluated. We direct the readers of interest to Appendix A for extraction criteria and detailed identifier lists. The statistics of the dataset are summarized in Table 1.

Evaluation Protocol We compare BiDock with two categories of representative methods, including three docking software ZDOCK [13], ClusPro [29], and HDOCK [58], and two deep learning models EquiDock [25], and AlphaFold-Multimer (Multimer for short [23]). To measure the quality of

(a)

(b)

1.0

0.8

0.8

0.7

0.6

0.6

E

0.5

0.4

0.4

M

0.3

0.2

0.2

0.1

Er.

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

HDOCK

Multimer

BiDock

BiDock

predictions, we report universally accepted metrics Root Mean Square Deviation (RMSD), TM-score (Template Modeling score), and DockQ [6]. Please note that in the context of antibody-antigen docking, the original DockQ metric evaluates the docking performance by treating the entire antibody as a single entity to the antigen. Furthermore, we can assess the docking results separately for variable heavy/light chain (VH/VL) to the antigen and denote the maximum value as maxDockQ. Refer to Appendix B for details of experiments, including implementation and hyperparameters.

4.1 Main Results and Analysis

Docking Results Table 2 demonstrates that BiDock generally produces acceptable predictions on all datasets. Notably, BiDock achieves a significant improvement in performance on the more challenging antibody-antigen docking, with a 234% relative gain over the runner-up on the DockQ metric. These results highlight the effectiveness of our bi-level optimization, which effectively leverages multimodal information. We also observe that some established docking software, such as HDOCK, still provides reliable predictions. In contrast, deep learning methods may have some leeway in performance. For instance, the mean and deviation of RMSD evaluated from EquiDock are relatively large, indicating that some inappropriate SE(3) transformations are learned.

To be more intuitive, we select representative baselines, HDOCK and Multimer, along with BiDock to analyze their performance differences on antibody-antigen docking. The distribution of DockQ for each method is displayed using a violin plot, and a direct comparison between Multimer and BiDock is also drawn in Figure 4. It reveals that the distribution of DockQ for BiDock is more concentrated around high values, indicating a higher percentage of successful docking predictions. Overall, these results suggest that our proposed BiDock is a promising approach for rigid protein docking.

Visualization To visually demonstrate the superiority of our proposed BiDock, we chose the spike glycoprotein 7 F 6 Z as an example antigen. The spike glycoprotein is crucial for the entry of coronaviruses into host cells, making it a target of great interest for therapeutic intervention and vaccine development. In Figure 5, we align the ground truth structure of this protein with predictions from competitive baselines. Upon inspection, it is evident that predictions from baselines exhibit noticeable deviations from the ground truth structures. Such inaccuracies can significantly impact our understanding of the binding mechanisms and hinder the design of effective interventions. In contrast,

ground truth v.s.HDOCK

groundtruthv.s.Multimer

ground truth v.s. BiDock

DockQ:0.088

DockQ:0.037

DockQ:0.912

Table 5: Impacts of different optimizers on antibody-antigen docking. (number in parentheses: gradient descent steps; unit: thousand; bold: best)

RMSD

SGD(1)

SGD(2)

SGD(10)

Adam(1)

Adam(2)

9.939

±

8.525

±

9.940

±

9.914

±

9.861

±

9.780

9.707

±

8.557

8.556

8.548

8.773

8.759

±

0.333

0.341

±

0.335

±

0.337

±

0.337

±

0.341

±

0.342

0.344

0.345

0.344

0.350

0.351

BiDock successfully captures the correct docking interface and accurately predicts the binding pose, highlighting its potential for providing biological insights and aiding drug design.

Computational Efficiency Table 3 shows the total inference time for antibody-antigen complexes. Traditional docking software, involving candidate sampling, ranking, and refinement steps, incurs substantial computational costs. Fortunately, deep learning methods provide a significant speedup, which is particularly important in efficient screening. Although EquiDock is the fastest, its performance falls short of traditional software due to limitations in leveraging coevolution information and simple networks. On the other hand, Multimer and BiDock exhibit comparable inference times. Considering the performance improvement of our model, this trade-off is acceptable.

4.2 Ablation Studies

Effects of Spectral Initialization Recalling our utilization of spectral estimation to derive a numerical solution for initializing the gradient descent in the inner loop, we conduct ablation experiments to demonstrate the effectiveness of this spectral initialization. Table 4 presents the results on antibody-antigen docking, where the variant without spectral initialization is denoted as "w/o SI" and the number of gradient descent steps is listed in parentheses. It is seen that the "w/o SI" variants exhibit slower convergence and lower accuracy, even when we further increase the number of gradient descent steps. These findings further support the benefits of our proposed spectral initialization in accelerating the optimization process and achieving superior results.

Effects of Bi-level Optimization To justify the effectiveness of bi-level optimization, we use a two-stage strategy: training outer and inner loops separately. Specifically, we use cross-entropy and masked MSA losses to train cross-modal transformer. Based on the resulting distance map, we compute roto-translation transformation with spectral initialization. The results on antibody-antigen docking are presented in Table 4, where "w/o Bi" denotes the variant without bi-level optimization (results reported from the original paper [34]). Results support our contributions of employing bi-level optimization for end-to-end optimization, which customizes the parameter learning of the cross-modal transformer for rigid docking.

Effects of Masked MSA Loss In addition to the essential loss for the distance map, we introduce a masked MSA loss in the outer loop to supervise the learning of evolution representations. We also conduct an ablation study to investigate its significance, as shown in Table 4. The "w/o MM" refers to models without masked MSA loss. According to the results, it can be concluded that the masked MSA loss enables the cross-modal transformer to more effectively leverage the rich evolutionary information and seamlessly integrate it with the structure-modal information.

Effects of Optimizer As shown in Equation (15), we have the flexibility to choose different optimizers for the inner loop. To explore the impact of different optimizers on convergence speed and performance, we compare the effects of SGD and Adam optimizers. We vary the number of

gradient descent steps and evaluate the final performance, as displayed in Table 5. With Adam, BiDock achieves faster convergence and attains higher accuracy, indicating that a better optimizer can navigate the landscape and find better minima during the optimization of the inner loop.

5 Conclusion

In this study, we introduce BiDock, a novel model for rigid protein docking that tackles the challenge of accurately predicting the 3D structure of protein complexes from unbound states. By formulating this problem as a bi-level optimization, BiDock combines the advantages of integrating multimodal information by a cross-modal transformer in the outer loop and maintaining the rigidity of learning roto-translation transformation in the inner loop. Additionally, we derive a spectral initialization to expedite convergence. The maximum 234% relative performance improvement validates the effectiveness of BiDock in rigid protein docking.

Limitations and Broader Impact Despite the encouraging results, BiDock does not explicitly incorporate geometric constraints between residues when learning the distance map and does not account for potential atom clashes. Our future work will address these limitations and extend our framework to general proteins. Protein docking deepens our understanding of biological mechanisms and aids in the design of targeted interventions. This research may inspire the AI4Science community to pay more attention to the practical challenges in docking and promote further advancements in this field with significant real-world implications.

Acknowledgments and Disclosure of Funding

Acknowledgments We would like to extend our heartfelt appreciation to the anonymous reviewers who dedicated their valuable time and expertise to meticulously review this paper. Their constructive feedback significantly contributed to the refinement and enhancement of the quality of our work. Our gratitude also extends to our team members, whose insightful discussions and collaborative brainstorming sessions were instrumental in shaping this research.

We wish to express our sincere thanks to the Data Science team at BioMap, especially to Tingting Tang, Nachuan Shan, Shiro Hong, Di Wang, Dacheng Li, Chenrui Xu, Ke Wang, and Cheng Chen, for their exceptional support in data collection, processing, and analysis. Their efforts with complex biological datasets were indispensable to the success of our research. Furthermore, we also express our sincere thanks to the AI Infracture team at BioMap, especially to Ming Yang, Chuang Yu, Zedong Zheng, Zezhi Wang, Qian Wang, and Xiaoming Zhang, for their outstanding technical support and seamless collaboration. Their proactive approach and effective troubleshooting played a vital role in ensuring the smooth progress of our research.

Finally, thanks Aaron for proofreading the manuscript.

Author Contributions The author's contributions to this paper encompass a range of vital aspects. Ruijia Wang made substantial contributions to content, writing, and experiment design. Yiwu Sun took on a central role in implementing the bi-level optimization algorithm and conducting experiments. Yujie Luo and Shaochuan Li were crucial in developing the precursor algorithm, xTrimoDock [34], and also aiding in data preparation.

Le Song and Hui Li lead the antibody-antigen structure prediction project in BioMap. They made invaluable contributions by providing algorithmic insights and guidance to the project, influencing the overall conceptualization and direction of the research. Xingyi Cheng, Chuan Shi and Cheng Yang provided valuable guidance in organizing the writing of the paper.

This work was funded by BioMap. The intellectual property of BiDock is 100% owned by BioMap.

References

Juhan Bae and Roger B Grosse. Delta-stn: Efficient bilevel optimization for neural networks using structured response jacobians. In NeurIPS , pages 21725-21737, 2020.

Minkyung Baek, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov, Gyu Rie Lee, Jue Wang, Qian Cong, Lisa N Kinch, R Dustin Schaeffer, et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science , 373(6557):871-876, 2021.

Protein Data Bank. Protein data bank. Nature New Biol , 233:223, 1971.

Jingxiao Bao, Xiao He, and John ZH Zhang. Deepbsp-a machine learning method for accurate prediction of protein-ligand docking structures. Journal of Chemical Information and Modeling , 61(5):2231-2240, 2021.

Itzhack Y Bar-Itzhack. New method for extracting the quaternion from a rotation matrix. Journal of guidance, control, and dynamics , 23(6):1085-1087, 2000.

Sankar Basu and Björn Wallner. Dockq: a quality measure for protein-protein docking models. PloS one , 11(8):e0161879, 2016.

Jacek Biesiada, Aleksey Porollo, Prakash Velayutham, Michal Kouril, and Jaroslaw Meller. Survey of public domain software for docking simulations and virtual screening. Human genomics , 5(5):1-9, 2011.

Ondrej Bohdal, Yongxin Yang, and Timothy Hospedales. Evograd: Efficient gradient-based meta-learning and hyperparameter optimization. In NeurIPS , pages 22234-22246, 2021.

Jerome Bracken and James T McGill. Mathematical programs with optimization problems in the constraints. Operations research , 21(1):37-44, 1973.

Patrick Bryant, Gabriele Pozzati, and Arne Elofsson. Improved prediction of protein-protein interactions using alphafold2. Nature communications , 13(1):1-11, 2022.

Can Chen, Xi Chen, Chen Ma, Zixuan Liu, and Xue Liu. Gradient-based bi-level optimization for deep learning: A survey. arXiv preprint arXiv:2207.11719 , 2022.

Can Chen, Jingbo Zhou, Fan Wang, Xue Liu, and Dejing Dou. Structure-aware protein selfsupervised learning. Bioinformatics , 39(4):btad189, 2023.

Rong Chen, Li Li, and Zhiping Weng. Zdock: an initial-stage protein-docking algorithm. Proteins: Structure, Function, and Bioinformatics , 52(1):80-87, 2003.

Yihong Chen, Bei Chen, Xiangnan He, Chen Gao, Yong Li, Jian-Guang Lou, and Yue Wang. λ opt: Learn to regularize recommender models in finer levels. In KDD , pages 978-986, 2019.

Charles Christoffer, Siyang Chen, Vijay Bharadwaj, Tunde Aderinwale, Vidhur Kumar, Matin Hormati, and Daisuke Kihara. Lzerd webserver for pairwise and multiple protein-protein docking. Nucleic Acids Research , 49(W1):W359-W365, 2021.

Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi S Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking. In NeurIPS , 2022.

Bowen Dai and Chris Bailey-Kellogg. Protein interaction interface region prediction by geometric deep learning. Bioinformatics , 37(17):2580-2588, 2021.

Ana Davila, Zichang Xu, Songling Li, John Rozewicki, Jan Wilamowski, Sergei Kotelnikov, Dima Kozakov, Shunsuke Teraguchi, and Daron M Standley. Abadapt: an adaptive approach to predicting antibody-antigen complex structures from sequence. Bioinformatics Advances , 2(1):vbac015, 2022.

Sjoerd J De Vries, Marc Van Dijk, and Alexandre MJJ Bonvin. The haddock web server for data-driven biomolecular docking. Nature protocols , 5(5):883-897, 2010.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 , 2018.

Jerome Eberhardt, Diogo Santos-Martins, Andreas F Tillack, and Stefano Forli. Autodock vina 1.2. 0: New docking methods, expanded force field, and python bindings. Journal of Chemical Information and Modeling , 61(8):3891-3898, 2021.

Stephan Eismann, Raphael JL Townshend, Nathaniel Thomas, Milind Jagota, Bowen Jing, and Ron O Dror. Hierarchical, rotation-equivariant neural networks to select structural models of protein complexes. Proteins: Structure, Function, and Bioinformatics , 89(5):493-501, 2021.

Richard Evans, Michael O'Neill, Alexander Pritzel, Natasha Antropova, Andrew W Senior, Timothy Green, Augustin Žídek, Russell Bates, Sam Blackwell, Jason Yim, et al. Protein complex prediction with alphafold-multimer. BioRxiv , 2021.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML , pages 1126-1135. PMLR, 2017.

Octavian-Eugen Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi S. Jaakkola, and Andreas Krause. Independent se(3)-equivariant models for end-to-end rigid protein docking. In ICLR , 2022.

Ian R Humphreys, Jimin Pei, Minkyung Baek, Aditya Krishnakumar, Ivan Anishchenko, Sergey Ovchinnikov, Jing Zhang, Travis J Ness, Sudeep Banjade, Saket R Bagde, et al. Computed structures of core eukaryotic protein complexes. Science , 374(6573):eabm4805, 2021.

Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Antibody-antigen docking and design via hierarchical structure refinement. In ICML , pages 10217-10227. PMLR, 2022.

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature , 596(7873):583-589, 2021.

Dima Kozakov, David R Hall, Bing Xia, Kathryn A Porter, Dzmitry Padhorny, Christine Yueh, Dmitri Beglov, and Sandor Vajda. The cluspro web server for protein-protein docking. Nature protocols , 12(2):255-278, 2017.

Elodie Laine, Stephan Eismann, Arne Elofsson, and Sergei Grudinin. Protein sequence-tostructure learning: Is this the end (-to-end revolution)? Proteins: Structure, Function, and Bioinformatics , 89(12):1770-1786, 2021.

Ingoo Lee, Jongsoo Keum, and Hojung Nam. Deepconv-dti: Prediction of drug-target interactions via deep learning with convolution on protein sequences. PLoS computational biology , 15(6):e1007129, 2019.

Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. In ICLR , 2018.

Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(12):10045-10067, 2021.

Yujie Luo, Shaochuan Li, Yiwu Sun, Ruijia Wang, Tingting Tang, Beiqi Hongdu, Xingyi Cheng, Chuan Shi, Hui Li, and Le Song. xtrimodock: Rigid protein docking via cross-modal representation learning and spectral algorithm. bioRxiv , pages 2023-02, 2023.

Andrew T McNutt, Paul Francoeur, Rishal Aggarwal, Tomohide Masuda, Rocco Meli, Matthew Ragoza, Jocelyn Sunseri, and David Ryan Koes. Gnina 1.0: molecular docking with deep learning. Journal of cheminformatics , 13(1):1-20, 2021.

Matthew McPartlon and Jinbo Xu. Deep learning for flexible and site-specific protein docking and design. bioRxiv , pages 2023-04, 2023.

Iain H Moal, Mieczyslaw Torchala, Paul A Bates, and Juan Fernández-Recio. The scoring of poses in protein-protein docking: current capabilities and future directions. BMCbioinformatics , 14(1):1-15, 2013.

Thin Nguyen, Hang Le, Thomas P Quinn, Tri Nguyen, Thuc Duy Le, and Svetha Venkatesh. Graphdta: Predicting drug-target binding affinity with graph neural networks. Bioinformatics , 37(8):1140-1147, 2021.

Hakime Öztürk, Arzucan Özgür, and Elif Ozkirimli. Deepdta: deep drug-target binding affinity prediction. Bioinformatics , 34(17):i821-i829, 2018.

Jimin Pei, Jing Zhang, and Qian Cong. Human mitochondrial protein complexes revealed by large-scale coevolution analysis and deep learning-based structure modeling. Bioinformatics , 38(18):4301-4311, 2022.

Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. In NeurIPS , 2019.

Anna Rutkowska, Douglas W Thomson, Johanna Vappiani, Thilo Werner, Katrin M Mueller, Lars Dittus, Jana Krause, Marcel Muelbaier, Giovanna Bergamini, and Marcus Bantscheff. A modular probe strategy for drug localization, target identification and target occupancy measurement on single cell level. ACS chemical biology , 11(9):2541-2550, 2016.

Rita Santos, Oleg Ursu, Anna Gaulton, A Patrícia Bento, Ramesh S Donadi, Cristian G Bologa, Anneli Karlsson, Bissan Al-Lazikani, Anne Hersey, Tudor I Oprea, et al. A comprehensive map of molecular drug targets. Nature reviews Drug discovery , 16(1):19-34, 2017.

Christina EM Schindler, Isaure Chauvot de Beauchêne, Sjoerd J de Vries, and Martin Zacharias. Protein-protein and peptide-protein docking and refinement using attract in capri. Proteins: Structure, Function, and Bioinformatics , 85(3):391-398, 2017.

Sharon Sunny and PB Jayaraj. Fpdock: Protein-protein docking using flower pollination algorithm. Computational Biology and Chemistry , 93:107518, 2021.

Mieczyslaw Torchala, Iain H Moal, Raphael AG Chaleil, Juan Fernandez-Recio, and Paul A Bates. Swarmdock: a server for flexible protein-protein docking. Bioinformatics , 29(6):807809, 2013.

Ilya A Vakser. Protein-protein docking: From interaction to interactome. Biophysical journal , 107(8):1785-1793, 2014.

Thirumalaisamy P Velavan and Christian G Meyer. The covid-19 epidemic. Tropical medicine &international health , 25(3):278, 2020.

Vishwesh Venkatraman, Yifeng D Yang, Lee Sael, and Daisuke Kihara. Protein-protein docking using region-based 3d zernike descriptors. BMC bioinformatics , 10(1):1-21, 2009.

Jacob Verburgt and Daisuke Kihara. Benchmarking of structure refinement methods for protein complex models. Proteins: Structure, Function, and Bioinformatics , 90(1):83-95, 2022.

Heinrich Von Stackelberg. Market structure and equilibrium . Springer Science & Business Media, 2010.

Thom Vreven, Iain H Moal, Anna Vangone, Brian G Pierce, Panagiotis L Kastritis, Mieczyslaw Torchala, Raphael Chaleil, Brian Jiménez-García, Paul A Bates, Juan Fernandez-Recio, et al. Updates to the integrated protein-protein interaction benchmarks: docking benchmark version 5 and affinity benchmark version 2. Journal of molecular biology , 427(19):3031-3041, 2015.

Izhar Wallach, Michael Dzamba, and Abraham Heifets. Atomnet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery. arXiv preprint arXiv:1510.02855 , 2015.

Yining Wang, Xumeng Gong, Shaochuan Li, Bing Yang, Yiwu Sun, Yujie Luo, Hui Li, and Le Song. Fast de novo antibody structure prediction with atomic accuracy. Cancer Research , 83(7_Supplement):4296-4296, 2023.

Yining Wang, Xumeng Gong, Shaochuan Li, Bing Yang, YiWu Sun, Chuan Shi, Hui Li, Yangang Wang, Cheng Yang, and Le Song. xtrimoabfold: Improving antibody structure prediction without multiple sequence alignments. arXiv preprint arXiv:2212.00735 , 2022.

Yining Wang, Xumeng Gong, Shaochuan Li, Bing Yang, Yiwu Sun, Chuan Shi, Yangang Wang, Cheng Yang, Hui Li, and Le Song. xtrimoabfold: De novo antibody structure prediction without msa. ArXiv, abs/2212.00735 , 2022.

Gaoqi Weng, Ercheng Wang, Zhe Wang, Hui Liu, Feng Zhu, Dan Li, and Tingjun Hou. Hawkdock: a web server to predict and analyze the protein-protein complex based on computational docking and mm/gbsa. Nucleic acids research , 47(W1):W322-W330, 2019.

Yumeng Yan, Huanyu Tao, Jiahua He, and Sheng-You Huang. The hdock server for integrated protein-protein docking. Nature protocols , 15(5):1829-1852, 2020.

Junjie Yang, Kaiyi Ji, and Yingbin Liang. Provably faster algorithms for bilevel optimization. In NeurIPS , pages 13670-13682, 2021.

Chia-Hung Yuan and Shan-Hung Wu. Neural tangent generalization attacks. In ICML , pages 12230-12240. PMLR, 2021.

Jonas Zehnder, Yue Li, Stelian Coros, and Bernhard Thomaszewski. Ntopo: Mesh-free topology optimization using implicit neural representations. In NeurIPS , pages 10368-10381, 2021.

Tian-ming Zhou, Sheng Wang, and Jinbo Xu. Deep learning reveals many more inter-protein residue-residue contacts than direct coupling analysis. BioRxiv , page 240754, 2018.

Marinka Zitnik, Francis Nguyen, Bo Wang, Jure Leskovec, Anna Goldenberg, and Michael M Hoffman. Machine learning for integrating data in biology and medicine: Principles, practice, and opportunities. Information Fusion , 50:71-91, 2019.

A Details of Dataset

Background of Antibodies Antibodies are vital components of the immune system and are classified into various classes, including IgG, IgM, IgA, IgD, and IgE. Among them, IgG antibodies are the most abundant in the bloodstream and play a primary role in immune responses against pathogens.

As depicted in Figure 6, IgG antibodies exhibit a Y-shaped structure composed of two identical light chains and two identical heavy chains, where heavy chains provide structural stability. Each antibody chain is further divided into distinct regions. (1) The variable regions, referred to as the variable heavy (VH) and variable light (VL) regions, are lo-

Antigen-binding site

VH

Fabregion

CH

3

Hinge

Disulfide bond

CH

C=constantdomain

V=variabledomain

Fcregion

H=heavy chain

cated at the tips of the Y arms. These regions contribute to the specificity of antibodies in recognizing and binding to antigens. The VH and VL regions collaborate to form the fragment antigen-binding (Fab) region. (2) At the base of the Y structure, the constant regions, also known as the fragment crystallizable (Fc) region, are important in the effector functions of antibodies. The Fc region interacts with immune cells and triggers immune responses, such as the activation of complement proteins for pathogen destruction and the promotion of phagocytosis.

Given this background knowledge of antibodies, it becomes clear that antibody-antigen docking is fundamental in immune responses, therapeutic applications, vaccine development, and drug discovery. Therefore, our study places a particular emphasis on antibody-antigen docking, contributing to this field by curating a high-quality benchmark. This dataset will serve as a valuable resource for evaluating computational models in predicting antibody-antigen interactions, ultimately facilitating the development of novel therapeutics and immunological interventions.

Antibody-antigen Benchmark The training set comprises 4,890 complexes of antibody-antigen pairs, each consisting of proteins with a minimum of 30 residues. These complexes encompass three chains, including the light and heavy chains of the antibody, along with one antigen chain. All complexes were released before January 2022. Similarly, the test set consists of 68 antibody-antigen complexes with three chains, released after October 2022. Thus, we ensure that neither baselines nor our proposed model was trained using the test set and avoid data leakage.

In practical applications, obtaining the ground truth structures of antibody-antigen complexes poses significant challenges. Researchers often turn to existing folding models to predict them. To simulate real-world scenarios, we employ a specialized antibody model called xTrimoABFold [55, 56, 54] to predict the conformations of antibodies and AlphaFold2 [28] for antigens. Given these predicted structures as rigid structures, we construct training and test datasets essential for further analysis and investigation. Here are the PDB identifiers and their corresponding chain identifiers for the test set.

8dls:A,H,L; 8dlr:A,H,L; 8dfi:A,H,L; 8dfh:A,H,L; 8dcc:E,L,H; 8dad:H,L,B; 7zr8:A,H,L; 7zf8:H,L,E;

7xxl:C,A,B; 7xh8:A,B,C; 7x26:H,I,K; 7wsl:L,D,H; 7wsi:A,H,L; 7ws6:C,H,I; 7ws2:D,A,E;

7wrz:H,L,R; 7wrv:C,U,V; 7wro:H,L,R; 7wrl:A,B,R; 7wrj:R,A,B; 7wog:C,A,B; 7wlc:E,H,L;

7wef:C,E,L; 7wee:E,H,L; 7wed:E,H,L; 7wcr:A,a,b; 7wbz:A,H,L; 7urq:A,H,L; 7uaq:A,H,L;

7tty:A,H,L; 7ttx:A,H,L; 7ttm:A,H,L; 7tpj:B,H,L; 7tp4:Z,H,L; 7tp3:Z,H,L; 7tlz:A,B,J;

7the:A,B,C; 7tc9:B,H,L; 7t8w:L,H,D; 7t7b:A,H,L; 7t01:A,H,L; 7swp:A,H,L; 7su1:H,L,C;

7str:L,H,C; 7sem:B,F,C; 7sd5:A,H,L; 7sbu:A,H,L; 7sbg:H,L,C; 7sbd:H,L,C; 7sa6:A,H,L;

7s5p:A,H,L; 7rxp:L,H,A; 7rxi:L,H,A; 7rbu:B,H,L; 7qtk:D,B,C; 7n0a:C,A,B; 7lo8:Z,H,L;

7lo7:Z,H,L; 7kql:H,L,T; 7fjc:H,L,E; 7f7e:C,L,E; 7f6z:R,H,L; 7f6y:R,H,L; 7eng:L,H,B;

7ek0:R,H,L; 7ejz:R,H,L; 7ejy:R,H,L; 7e9p:B,L,H

B Details of Implementation

Baselines ZDOCK 5 , ClusPro 6 , and HDOCK 7 are user-friendly local packages suitable for automated experiments or web servers for manual submissions. We select the top1 predicted structure from each of these methods for subsequent evaluation. For EquiDock 8 and Multimer 9 , we utilize their pretrained models available on GitHub for the inference. It is worth emphasizing that all methods except Multimer are designed for docking two chains. Therefore, during the evaluation, we employ a sequential docking strategy. This entails initially docking the light chain and heavy chain together, followed by treating them as a unified entity for docking with the antigen. And we calculate evaluation metrics using the tools USalign 10 and DockQ 11 .

MSA Extraction We utilize the heuristic approach described in [23] to pair sequences from perchain multiple sequence alignments (MSAs). Initially, the per-chain MSA sequences are grouped based on species, with the species labels obtained from UniProt's idmapping 12 . Within each specific species group, the sequences are paired together. We match the chain MSAs by minimizing the base-pair distance between the chains for prokaryotic species. While in terms of eukaryotic species, we order them based on sequence identity to the target sequence [62]. To reduce computational and memory costs, we employ the MSA clustering approach from AlphaFold2 [28]. We randomly select N cls = 252 sequences as the MSA cluster centers, with the primary protein sequence always set as the first cluster center. The remaining sequences are assigned to their closest cluster based on the Hamming distance.

Sequence-modal Input The sequence modality incorporates information derived from the primary sequence itself and co-evolutionary information obtained from MSAs. Following prior research [28, 23], we extract two types of features: type features F typ ∈ R N res × 21 and primary pair features F pp ∈ R N res × N res × 73 from the primary sequence, where N res represents the number of residues. Regarding MSAs, we utilize cluster MSA features F msa ∈ R N cls × N res × 49 , where N cls denotes the number of cluster centers. Specifically,

The type feature F typ ∈ R N res × 21 comprises one-hot representations of the amino acid types, encompassing the 20 known amino acids and one additional category for unknown types.

The primary pair feature F pp ∈ R N res × N res × 73 contains positional information within or across chains, including three components. (1) The relative positional feature of size [ N res , N res , 66] represents the relative residue indices, which are clipped between [ -32 , 32] . The 66 -th index is used to indicate cross-chain pairs. (2) The entity indicator of size [ N res , N res , 1] identifies whether residues i and j originate from the same chain. (3) The relative index feature of size [ N res , N res , 6] introduces the relative sym_id 13 indices clipped between [ -2 , 2] . The 6 -th index is assigned to pairs where the two residues have different sym_id s.

The cluster MSA feature F msa ∈ R N cls × N res × 49 consists of five components. (1) The one-hot representation of the amino acid types with size [ N cls , N res , 23], including 20 amino acids, one unknown type, one gap or missing residue, and one mask token as introduced in Section 3.1. (2) The amino acid distribution of size [ N cls , N res , 23] represents the distribution of amino acid types within each MSA cluster. (3) The deletion indicator of size [ N cls , N res , 1] indicates whether there is a deletion to the left of each residue. (4) The deletion value of size [ N cls , N res , 1] is calculated using the formula 2 π arctan c 3 , where c refers to the number of deletions to the left of each position.

(5) The mean deletion value of size [ N cls , N res , 1] is computed as 2 π arctan ¯ c 3 , where ¯ c represents the average number of deletions to all residues on the left of each position.

Structure-modal Input For the structure modality, we extract angle features F ang ∈ R N res × 57 and pair features F p ∈ R N res × N res × 88 from the rigid protein structures. These features capture important structure-modal information and are used as input for our docking model. Specifically,

The angle feature F ang ∈ R N res × 57 comprises three components. (1) The one-hot representation of the amino acid types with a size of [ N res , 22] , including 20 amino acids, one unknown type, and one gap or missing residue. (2) The angle representations of size [ N res , 28] use sine and cosine to encode three backbone torsion angles, four side-chain torsion angles, and alternative torsion angles with 180 ◦ rotation symmetry for each local frame of residue. (3) The angle indicator with size [ N res , 7] indicates the presence or absence of torsion angles.

The pair feature F p ∈ R N res × N res × 88 comprises five components. (1) The distogram feature of size [ N res , N res , 39] represents the discretized distances between C β atoms. In the case of glycine, which lacks C β atoms, C α is used instead. The distances are discretized into 38 bins of equal width ranging from 3.25 to 50.75Å, with an additional bin accounting for larger distances. (2) The residue type feature of size [ N res , N res , 44] is derived from expanding one-hot representations of residue types with dimensions of [ N res , 1 , 22] and [ N res , 22 , 1] . (3) The backbone feature of size [ N res , N res , 3] is obtained by constructing the unit vector of the local frame through the Gram-Schmidt process based on the original N-C α -C coordinates. (4) The residue indicator with size [ N res , N res , 1] is expanded from the indicator of residue existence. (5) The pair indicator of size [ N res , N res , 1] indicates whether the pair is masked.

MSA Mask Policy Reflecting on Section 3.1, we design a masked MSA loss to supervise the learning of evolution representations and the integration of cross-modal information. Specifically, we randomly mask each position in an MSA cluster center with a 15% probability. Each masked token is replaced according to the following policies:

70% probability of substitution with a special token ⋆

10% probability of substitution with a randomly selected amino acid from a uniform distribution

10% probability of substitution with an amino acid sampled from the MSA profile that corresponds to the position

10% probability of no substitution

Hyperparameter Settings We initialize specific parameters of the cross-modal transformer with the checkpoint of Multimer and implement bi-level optimization using TorchOpt 14 library. The crop size is set to 412, and the batch size is set to 1. The coefficients in Equation (13) are λ 1 = 0 . 2 , λ 2 = 2 . 0 , and λ 3 = 10 . 0 . For optimization, we employ the Adam optimizer with a learning rate of 10 -4 and integrate learning rate warmup, gradually increasing the learning rate from 0 to 10 -4 within the first 100 steps. The exponential moving average (EMA) strategy applies a decay rate of β = 0 . 999 and undergoes updates every 200 steps. The environment where we run all experiments is:

Operating system: Linux version 5.13.0-30-generic

CPU information: AMD EPYC 7742 64-Core Processor

GPU information: NVIDIA A100-SXM4-80GB

C Additional Results

Effects of Noisy Structures Classical software rely on score functions derived from statistics in the protein data bank. This dependency renders them susceptible to noise. When using folding algorithms to predict unbounded proteins, the performance of these software can degrade significantly. To validate this intuition, we conduct a docking performance analysis on the DB5.5 dataset using ground truth and predicted structures from folding models as unbounded structures, respectively. As shown in Table 6, these results illustrate that although HDOCK performs exceptionally well with

ground truth, minor noise in predicted structures leads to a substantial decline in its performance. On the contrary, BiDock consistently generates acceptable predictions regardless of the input type, showcasing its robustness to noise. In real-world applications, reliance on the availability of ground truth structures is impractical. The ability of BiDock to maintain high prediction quality when confronted with noisy structures makes it an invaluable tool.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new framework for rigid protein docking called BiDock. BiDock is based on an Evoformer-based model that predicts a distance matrix and a gradient-based optimization of the optimal roto-translation based on the predicted distance matrix. In order to be able to pass gradients through the optimization steps at training time the authors derive a bi-level optimization procedure and a spectral initialization.\n\nThe model is trained specifically for rigid docking of antibody chains and evaluated on a range of complexes divided in: general proteins, antibody heavy and light chains and antibody-antigens. Across all three categories, the method shows improved performances.\n\nSTRENGTHS:\nThe paper proposes an interesting new approach to deep-learning-based rigid docking using gradient based pose optimization and propagation through the optimization. Moreover, the spectral initialization provides an interesting and meaningful improvement to the performance.\n\nWEAKNESSES:\nThe paper has good methodological and empirical contributions, however, to make it a strong accept I believe a few components in the manuscript should be improved and/or clarified.\n\nIntroduction and novelty claims:\n\n1. Personally, I do not understand the strong emphasis that the authors put on “being the first to effectively leverage sequence and structure modal information in rigid protein docking”. This is I believe the least significant part of the paper as: (1) AlphaFold-Multimer already uses sequence and structural template based informations and the architecture proposed is very similar to that of the EvoFormer blocks, (2) adding sequence based information to EquiDock would be a very straightforward addition, (3) other existing deep-learning based docking methods not referenced by the authors such as [1] already uses sequence and structure informations for the predictions. I believe that the claims of novelty in these regards should be adjusted taking into consideration (1-3). \n\nUnclear parts of the method:\n\n2. The loss used to train the model is not clear to me. Equation (12) says that L^out(R, t, \\phi)  = \\lambda_1 L_dist + \\lambda_2 L_msa, however neither L_dist nor L_msa seem to depend on R nor t. Is this correct? If so I don’t understand the point of the bi-level optimization, as the gradient of L^out with respect to \\phi would not depend on R* or t*.\n3. line 167 “\\hat{D} can be obtained by using the center of each bin”, what is meant specifically by center of each bin? Center of the bin with highest likelihood, or median of the predicted binned distribution, or something else? \n\nThere are several missing details (both in paper and in appendix) about the evaluation set-up that should be clarified:\n\n4. How was AlphaFold-Multimer run (and with what set-up in terms of MSA etc)? The appendix links to the OpenFold repo but this does not yet provide Multimer’s replicas.\n5. For the other methods, what structures were used as input? I know DB5.5 provides also apo (unbound) structures that I assume the authors used for the input, however for the two benchmarks they propose how are the input structures generated?\n6. I assume the different methods were run on different hardware (GPUs vs CPUs), this should be reported in Table 3 or standardized. \n\n[1] Ketata, Mohamed Amine, et al. \"DiffDock-PP: Rigid Protein-Protein Docking with Diffusion Models.\" *arXiv preprint* (2023).\n\nQUESTIONS:\nSee weaknesses section and the following:\n\n7. Especially with the flexible regions in antibodies it appears important to consider a more flexible form of binding, could the proposed framework be extended in that direction?\n\nLIMITATIONS:\nSee weaknesses section", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the rigid protein-protein docking problem. The authors fuse the sequence features and structural features of proteins and unify these features into single features and pair features as done in AlphaFold2. These features are then fed into an evoformer-like cross-modal transformer to produce updated representations. The updated representations are used to predict inter-chain distance map and masked MSA tokens. The final rigid-docking results (i.e., a global translation and a global rotation) can be obtained by solving an optimization problem that minimizes the discrepancy between the distance map of the predicted complex and that of the ground-truth.\n\nThe authors claim that they solve the above optimization problem better by providing a better initial rotation and translation with the help of spectral initialization.\n\nSTRENGTHS:\n1. The proposed cross-modal transformer is clear and easy to follow. \n2. The spectral initialization of the optimization problem offers good insights for those who are working on the structure prediction problems.\n\nWEAKNESSES:\n1. Some important related works are not discussed and compared. For example, (1) Diffdock-PP (https://arxiv.org/pdf/2304.03889.pdf) use torsional diffusions to solve rigid protein docking, and the source code is released. (2) McPartlon & Xu proposed DockGPT for\nflexible and site-specific protein docking and design. Although they target the flexible docking, they also use evoformer to predict the inter-chain distance map. (3) Very recently, Chu, Lee-Shin, et al. propose geodock (https://www.biorxiv.org/content/10.1101/2023.06.29.547134v1),  a multi-track iterative transformer network for fast and flexible\n88 protein-protein docking. (4) Finally, Luo et al. proposed xTrimoDock (https://www.biorxiv.org/content/10.1101/2023.02.06.527251v1.full.pdf), which is quite related to this work, as they also predict the inter-chain distance map and compute a global rotation and translation that best-fit the predicted distance map.\nAs a submission of the NeurIPS conference, I highly recommend the authors to discuss the connection of this work with the works mentioned above.\n\n2. Figure 1. of the current manuscript is borrowed from xTrimoDock (https://www.biorxiv.org/content/10.1101/2023.02.06.527251v1.full.pdf), this could potentially be problematic.\n\n3. The experimental results are not that clear and convincing. (1) First, Diffdock-PP is not included as a baseline. (2) Second, previous works use ligand-RMSD, complex-RMSD, or interface-RMSD to evaluate the performance of the model. How RMSD is calculated in this work is not revealed.\n\n4. As shown in Eq. 12, 9, 11, the definition of the L^{out} loss does not involve $R^{\\star}$ and $t^{\\star}$. Therefore, the first term in Eq. 13 is equal to zero, and it seems that the outer loss can be directly optimized by minimizing the distance prediction loss and MSA loss. I'm not sure how the story of bilevel-optimization holds in this situation. Can the authors further clarify this point?\n\n5. The authors claim that they use bilevel optimization to solve the problem. However, the effectiveness of the bilevel optimization is not justified.  The authors are encouraged to report the results of the variant that does not rely on bi-level optimization, i.e., directly training a distance-map predictor, and recovery the complex structures from predicted distance map.\n\nQUESTIONS:\nPlease see the Weaknesses.\n\nLIMITATIONS:\nPlease see the Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces BiDock, a novel approach that integrates sequence- and structure-modal information to improve the accuracy of rigid protein docking predictions. The proposed method uses multimodal information through bi-level optimization, enabling joint optimization of the docking score and the weights of the multimodal features. A gradient descent method is employed to predict the rotation and translation instead of relying on the Kabsch algorithm. Additionally, a spectral initialization is proposed for the gradient descent.\nThe authors demonstrate the effectiveness of BiDock by comparing it to existing docking methods on various datasets, including two newly curated datasets from PDB database. Remarkably, BiDock outperforms other methods, particularly excelling in the challenging antibody-antigen docking task. \n\nThe paper's contributions include the development of a novel approach for integrating multimodal information in protein docking, the introduction of two curated datasets, a thorough evaluation of the proposed method with baselines across multiple datasets.\n\nSTRENGTHS:\n- **Integration of multimodal protein representation**: The paper introduces a novel approach that combines both information representations and structural information, resulting in a comprehensive protein representation. This integration enhances the accuracy of rigid protein docking predictions.\n- **Bi-level optimization formulation of the protein docking problem**: The authors formulate the docking problem as a bi-level optimization task, which allows for joint optimization of the rigid docking poses and the weights of the multimodal features. By employing a gradient descent method for predicting rotation and translation, the paper offers an alternative to the traditional Kabsch matching algorithm.\n- **Spectral initialization for gradient descent**: The paper proposes a spectral initialization technique for the gradient descent method used in the bi-level optimization. \n- **Introduction of two new curated datasets**: The authors curate two new datasets from the Protein Data Bank (PDB) database, specifically focusing on antibodies (VH-VL) and antibody-antigen complexes (AB-AG). These datasets provide valuable resources for evaluating protein docking methods, particularly in the context of antibody-antigen interactions.\n- **Strong experimental results**: The paper demonstrates the superiority of the BiDock model over existing docking methods through comprehensive experiments on multiple datasets. The proposed method particularly excels in the challenging antibody-antigen docking task, showcasing its efficacy in accurately predicting protein docking conformations.\n- **Clear presentation**: The authors provide clear explanations of the proposed approach, experimental setup, and evaluation metrics, enhancing the clarity and accessibility of the research findings.\n\nWEAKNESSES:\n- **Unclear motivation**: The paper does not clearly explain why sequential/coevolution representations are useful specifically for rigid docking problems, leaving the reader questioning the relevance and benefits of these representations in this context. Ablation study between implementation with and without sequential information is appreciated.\n- **Choice of bi-level optimization**: The paper formulates the problem as a bi-level optimization task, but it lacks a clear explanation as to why this approach is chosen over direct end-to-end optimization using the distance map as input. The division of the optimization into an outer loop (distance map prediction) and inner loop (pose optimization) raises questions about the necessity and effectiveness of this bi-level optimization scheme.\n- **Curated dataset filtering process**: The paper lacks detailed information about the filtering process used to curate the datasets, leaving uncertainty about how the quality of complex structures in the datasets is ensured. Without transparent and rigorous filtering criteria, the reliability of the curated datasets may be compromised.\n- **Computational complexity**: BiDock exhibits higher inference time compared to existing methods like EquiDock and AlphaFold-Multimer. The computational complexity associated with the bi-level optimization process might contribute to this increased inference time, which can impact practical usability and scalability.\n- **Limited training and testing data**: The paper acknowledges that the available training and testing data are limited, which raises concerns about the generalizability and robustness of the proposed method. Insufficient data may lead to overfitting and an incomplete evaluation of BiDock's performance.\n- **Incremental novelty**: The paper lacks significant innovation in terms of network structure for BiDock. Instead, it builds upon existing techniques such as cross-modal transformers and roto-translation transformations, which may limit its originality and novelty compared to other state-of-the-art approaches in the field.\n\nQUESTIONS:\nPlease see the Weaknesses part.\n\nLIMITATIONS:\nYes, the authors discuss about limitation of the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes BiDock, a novel rigid protein docking model that integrates sequence and structure information through bi-level optimization. It achieves promising results, outperforming baselines by up to 234% in challenging antibody-antigen docking.\n\nSTRENGTHS:\nAs claimed by the authors in the paper, this work represents the first attempt to effectively leverage sequence- and structure-modal information for rigid protein docking.\n\nWEAKNESSES:\nThe authors seem to overlook the discussion and comparison of some protein docking methods based on pre-trained models. For example, [1] utilizes the protein language model ESM2 [2] for Protein Docking task, and there are other pre-trained models focusing on protein structures [3] as well as embeddings that combine both sequence and structure information [4]. These methods are worth discussing and applying in the experimental section for a comprehensive evaluation and comparison. In general, applying pre-trained methods directly to downstream protein-related tasks often leads to significant improvements in performance.\n\nThe paper lacks sufficient details, such as the specific approach for concatenating MSA. It is unclear whether the MSA is directly concatenated or arranged based on homology, similar to AF2-multimer. Providing these implementation details, and making the code available, would enhance the reproducibility and transparency of the research.\n\n​​For antibody-antigen tasks, reporting the RMSDs of the CDRs are crucial. The high variability of CDRs makes predicting this region particularly challenging. Especially, results on CDR3 region is a critical metric for evaluating the model's performance, and its absence in the author's discussion is noteworthy. \n\n\n[1] Chu, Lee-Shin, et al. \"Flexible Protein-Protein Docking with a Multi-Track Iterative Transformer.\" bioRxiv (2023): 2023-06.\n\n[2] Lin, Zeming, et al. \"Evolutionary-scale prediction of atomic-level protein structure with a language model.\" Science 379.6637 (2023): 1123-1130.\n\n[3] Guo, Yuzhi, et al. \"Self-supervised pre-training for protein embeddings using tertiary structures.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 6. 2022.\n\n[4] Chen, Can, et al. \"Structure-aware protein self-supervised learning.\" Bioinformatics 39.4 (2023): btad189.\n\nQUESTIONS:\nDoes the proposed multi-model model have the capability to handle tasks when there is missing structure? Are there any potential improvements or modifications that could address this issue?\n\nDo MSAs of the antibody part really work for this task?\n\nLIMITATIONS:\nThe paper's limitations include the absence of explicit geometric constraints between residues in learning the distance map and neglecting potential atom clashes. The authors plan to address these issues and extend the framework to encompass general proteins in future work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-28"}