Injecting Multimodal Information into Rigid Protein Docking via Bi-level Optimization
Authors: Ruijia Wang
YiWu Sun
Yujie Luo
Shaochuan Li
Cheng Yang
Xingyi Cheng
Hui Li
Chuan Shi
Le Song
Venue: NeurIPS 2023 poster (2023)
Keywords: complex structure prediction
rigid docking
protein docking
antibody-antigen docking

Abstract: The structure of protein-protein complexes is critical for understanding binding dynamics, biological mechanisms, and intervention strategies. Rigid protein docking, a fundamental problem in this field, aims to predict the 3D structure of complexes from their unbound states without conformational changes. In this scenario, we have access to two types of valuable information: sequence-modal information, such as coevolutionary data obtained from multiple sequence alignments, and structure-modal information, including the 3D conformations of rigid structures. However, existing docking methods typically utilize single-modal information, resulting in suboptimal predictions. In this paper, we propose xTrimoBiDock (or BiDock for short), a novel rigid docking model that effectively integrates sequence- and structure-modal information through bi-level optimization. Specifically, a cross-modal transformer combines multimodal information to predict an inter-protein distance map. To achieve rigid docking, the roto-translation transformation is optimized to align the docked pose with the predicted distance map. In order to tackle this bi-level optimization problem, we unroll the gradient descent of the inner loop and further derive a better initialization for roto-translation transformation based on spectral estimation. Compared to baselines, BiDock achieves a promising result of a maximum 234% relative improvement in challenging antibody-antigen docking problem.

Injecting Multimodal Information into Rigid Protein Docking via Bi-level Optimization
Ruijia W2ngl2-t Yiwu Sun't
Yujie Luo
Shaochuan Li Cheng Yang²
Xingyi Cheng'
Hui Li
Chuan Shi Le Song't
BioMap Research
2 Beijing University Posts and Télecommunications (vangruijia, yangcheng, shichuan]&bupt edu. en, (yiuu, luoyujie, shaochuan, xingyi, lihui, @bigoap@.imma..... com
Abstract
The structure of protein-protein complexes is critical for understanding binding dy- namics, biological mechanisms, and intervention strategies. Rigid protein docking, fundamental problem in this field, aims predict the 3D structure of complexes from their unbound states without conformational changes. this scenario, we have access two types of valuable information: sequence-modal information, such as coevolutionary data obtained from multiple sequence alignments, and structure-modal information, including the 3D conformations of rigid structures. However, existing docking methods typically utilize single-modal information, resulting in suboptimal predictions. In this paper, propose x XTrimoBiDock (or BiDock for short [ novel rigid docking model that effectively integrates sequence- and structure-modal information through bi-level optimization. Specifically, cross- modal transformet combines multimodal information to predict an inter- protein distance map. To achieve rigid docking, the roto-translation transformation is optimized to align the docked pose with the predicted distance map. In order to tackle this bi-level optimization problem, we unroll the gradient descent of the inner loop and further derive a better initialization for roto-translation transformation based on spectral estimation. Compared baselines, BiDock achieves promising result of maximum 234% relative improvement in challenging antibody-antigen docking problem.
Introduction
Protein-protein interactions (PPIs) essential the basic functioning of cells and larger biological systems. Due to their importance, elucidating such interactions up to atomic detail is necessary for understanding multicomponent complexes like ribosomes and discovering protein-based drugs, e.g., antibodies and peptides. However, the experimental golden standard for determining the structure of protein complexes, such X-ray crystallography and cryo-EM, extremely time-consuming.
45 provides an alternative route to predict the 3D struc- tures of complexes from unbound states. Here, focus the fundamental problem of rigid protein docking (25] where no deformations occur within any protein during the docking process.
Computational protein docking 44
Work done during an internship at BioMap
i CCottributed equally this rescarch. Each author's contribution is provided in Section
'Corresponding authors
'oDr member BioMap's large-scale engine "xTrimo" series. "a" denotes the academic version, distinguish from the commercial product Trock
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

This assumption is reasonable in many bio- logical envirouments and stabilizes the pre- diction of natural structures. Therefore, all we need an appropriate SE(3) transforma- tion shown in [ i.e., roto-translation transformation, that places the ligand pro- tein at the correct orientation and location concerning the receptor protein. this context, two types of information avail- able. The first is sequence-mode infor- mation, such as the coevolutionary signals captured in multiple sequence alignments (MSAs). The second is structure-modal information, like 3D coordinates of rigid bodies and bond angles. Both types of in- formation are indispensable for performing rigid protein docking.
ligand protein
protein complex
T (R 90(3), R*) SEE(3 transformation
rigid docking
reveplor protein
Figure 1: Surface views of rigid protein docking. Keep receptor protein fixed location, and roto- translation transformation T = (R, t) is predicted to place ligand protein the correct docked pose.
generally follow a three-step framework for predicting complex structures. Firstly, a large number of candidate structures are randomly sampled to explore the conformational space of the complex. Secondly, a scoring function employed to evaluate and rank the sampled structures based on their compatibility with the binding interface. Finally, the top-ranked structures are refined using an energy model to improve their accuracy and eliminate steric clashes. Due to the evaluation of lots of candidates, these methods tend to be computationally expensive, particularly for high-throughput workflows.
Classical docking software 13
46
29 58
Recently, deep learning has shown significant computational speed-up in this field. EquiDock [25] has emerged as a pioneering method for applying deep learning to rigid protein docking. However, solely relies structure-modal information, neglecting the valuable sequence-modal information in databases. This drawback hampers its ability to capture evolutionary constraints and exploit the intricate sequence-stree struuturerr relationships. Building upon the success of AlphaFold2 (28). AlphaFold- Multimer [23] has been hailed as breakthrough in directly folding complex structures from amino acid sequences, which considers the sequence- modal information but fails to effectively utilize the given rigid structures, leading to unnatural complexes in some cases.
To overcome the aforementioned limitations, propose xTrimoBiDock (or BiDock for short), novel rigid docking model that seamlessly integrates sequence- and structure-modal information through bi-level optimization. (i) To effectively utilize multimodal information, introduce cross-modal transformer that injects multimodal information into an inter-protein distance map. (ii) To satisfy rigid docking, optimize roto- translation transformation to minimize the difference between the docked pose of unbound proteins and the predicted inter-protein distance map. This framework naturally lends itself to a nested bi-level optimization paradigm, where the outer loop is the learning of the cross-modal transformer and the inner loop is dedicated to solving the roto-translation transformation. Inspired by gradient-based based bi-level optimization (24 32 11, we unroll the gradient descent of the inner loop for approximation. Due to the ruggedness of the optimization landscape, substantial number of iterations are required for convergence. Thus, we further derive better initialization for the roto-translation transformation using spectral estimation. Extensive experiments conducted on diverse datasets and evaluation protocols validate the effectiveness of BiDock.
In summary, our contributions are three-fold:
We effectively leverage sequence- and structure- modal information for rigid protein docking. By naturally integrating the fusion of multimodalities and the docking of rigid bodies through bi-level optimization, we set up a new avenue for solving rigid protein docking.
We solve the above bi-level optimization with unrolled gradient and spectral initialization. By unrolling the gradient descent of the inner loop and deriving spectral estimation for initialization, BiDock enhances the convergence and controls the computational cost.
Comprehensive experiments three representative datasets demonstrate the effectiveness of the proposed model. Compared to tate-of-the-aa baselines, BiDock achieves the maximum 234% relative improvement in challenging but practical antibody-antigen docking.

---

2 Related Work
Molecular Docking Molecular docking aims characterize the binding poses between small molecule compounds and protein targets [42
43
631, playing an essential role in the discovery of
effective and safe treatments for various diseases 48 Traditional computational methods have
been greatly enhanced by deep learning techniques [53
which offer increased expressive power in
identifying, processing, and extrapolating complex patterns in molecular data [39 31 21 35 4 38 [6]. It is important to note that these methods have primarily focused on molecular ligands and often assume the availability of known binding pockets, limiting their direct applicability protein-protein docking, Protein-protein docking is more challenging due to larger, flexible proteins and the need to predict unknown binding interfaces.
Protein Docking Computational docking software 46
57 58
45
15
predict complex
structures based on a framework of candidate sampling. ranking [37 22 and refinement (50). These methods can be financially restrictive and time-consuming, as they often involve scoring and ranking thousands of candidate structures. Recently, deep learning has made significant contributions
to structural biology [30 17
18 36). Notably, AlphaFold2 [28] and RoseTd [2] have been
employed to improve protein structure prediction from various angles [26 40 such as integrating
thysics-based docking methods 229 or extending multiple sequence alignments [0] Additionally,
methods like AlphaFold-Multingg
23 and HSRN (27) have been developed to simultaneously fold
and dock two proteins. Despite their remarkable achievements, these methods violate the rigidity of rigid docking and do not consider unbounded structures, EquiDock [25] is tailored for effective rigid docking but does not fully leverage the evolutionary information encoded in protein sequences, resulting in limited performance improvement compared to traditional docking software.
Bi-level Optimization Bi-level optimization has gained attention in the deep learning community for its ability to handle nested problem structures. It finds applications in various domains, including hyperparameter optimization 14] and metaknowledge extraction (24]. Traditional bi-level optimiza- tion methods rely on game theory [T]] or mathematical programming ()) which may not scale well to large datasets or have strict mathematical requirements. Alternatively. gradient descent methods offer a promising solution, and they can be divided into explicit gradient update [60]. explicit proxy update [], implicit function [41], and closed form methods [59]. The first three are approxima- tion methods suitable for general functions, while the last one is an accurate method specifically designed for certain functions. Recent surveys [33] [1] provide a more comprehensive review of these methods. addition, the merging A14Science directions, such as topology design [6T] and protein representation learning [12), also present nested problem structures that can be compatible with bi-level optimization. By incorporating bi-level optimization into rigid protein docking, we anticipate that our work will have significant impact at the intersection of these research areas.
BiDock Methodology
Multimodal Input The available information on the sequence modality mainly includes information inside the primary sequence itself and co-evolutionary information from MSAs. Following the existing work 28]. we extract type features FLup E R. x21 and primary pair features EPP e R.Nea.XN.e. x33 from the primary sequence, where Nrgs is the number of residues. In terms of MSAs, we leverage cluster MSA features Fmsen E R N x49 where Nets is the number of cluster centers. For the structure modality, we extract angle features pang E N.N.. x57 and pair features FP E IR Nrea X.rea x88 from the rigid protein structures. The angle features provide the orientation and position of each amino acid, while the pair features contain the distance information between amino acids. We present brief introduction for features and give further details in Appendix B
Architecture The illustration of the framework is presented in Figure 2 Given sequence-modal features {Etup, Emssa [Etu, }oP} and structure-mode features {Fang, F²). J. the cross-modal transformer with parameters & transforms available features and integrates them to predict inter-protell distance map D. To maintain the rigid-body assumption, we define the objective function of learning roto- translation transformation (R, t) based on the distance map D and coordinates of rigid proteins
3

---

outer loop
inner loop
&,R,E, c,
min
(IXx 18--R ROs 11 Dul0)))))
(R. t)
soquence modality pung Py
cross-modal transformer (parumeters +)
distance map
rigid proteins
spectral initialization (Ro, tu)
predicted pose
Re+1 OPT(R, VRC, d.) f,+1 OPT(s, V,C,(R,, 6))
structure modality
hypergradient
The overall framework of BiDock. Taking the information from sequence and structure modality input, the cross-modal transformer fuses them and predicts inter-proteis distance map. To achieve rigid docking, the tro-rranslation transformation is learned by minimizing the difference between the docked pose of rigid structures and the predicted distance map. We unroll the gradient descent and further derive a spectral initialization to address the formed bi-level optimization.
Figure 2:
{X E 33xm Y E R3xn] as follows
|X, RY, t|| Du)
C(R,t = min R.t mi i=1
where X; is the i-th column of X and Yj is the j-th column of Y. Considering that optimizing roto-translation transformation (R, and parameters of cross-modal transformer á constitutes nested structure, we reformulate the rigid protein docking as a bi-level optimization problem
Φ" argmin Cont (R* (Ø), t" (Ø), σ)
R (Ø), (0)
argmin C(R, t, d),
R.t
where Loet refers to the outer loss for the cross-modal transformer and its specific details will be introduced in the following subsections. To solve this bi-level optimization, unroll the gradient descent of the inner loop to approximate (R* (ø), to)) by
R OPT(R,, RRCi (Re, Lr Φ)),
t-1 = OP,(,,, V.C, (Re, tz, +)),
where OPT represents the optimization algorithm, such as the stochastic gradient descent (SGD). However, due to the complexity of the problem and the rugged optimization landscape, the gradient descent in the inner loop often requires large number of iterations to converge and faces challenges in finding the global minima. Therefore, we further derive spectral initialization to provide a more favorable starting point for better convergence.
3.1 Cross-modal Transformer
Based on the multimodal features, the cross-modal transformer predicts an inter-protein distance map, as depicted in Figure We first use multilayer perceptrons (MLPs) capture the nonlinear relationship inside features and project them into the same space
pPp MLP(FEP) PP MLP(F
(4)
'M VD MLP((MPP) maa MMP(( Mang MLP(()an0).
where (PPP, PP X. are transformed pair features, (M Mang E Ne. xcm) } and Mmsa E X are transformed intra-sequence features. Then we add pair features as the input pair feature P of Evoformer
P pPp + PP.
(5)
Similarly, we integrate intra-sequence features as another input
M [M + msay ms || ang]
(6)

---

sqquemce modality
Milac
nonlincar
concat
evabioe represerration A/
nonlincar
M
XSA Sostare prus
pep
nonlinear
evoformer
par ripooraiaiionn P
primur par
dieance 3
modality
nonlinear
linear
Cavet
2aglo Buture
nonlinear
Figure 3:
Details on the architecture of cross-modal transformer that enables the interaction of different modalities and outputs updated representations. In particular, the pair representation is used to predict the inter-protein distance map.
where is the concatenation. Please note that the broadcast operation is used to make the dimensions consistent. Specifically, M is broadcasted along the newly added first dimension during addition. Similarly, broadcasting is applied to the newly added first dimension of Mang for concatenation. Through Evoformer, pair representation Í and evolution representation M are obtained
P. Mr EVOFORMER(P? M).
Finally, we utilize pair representation to predict the inter-protein distance map. Here we define two loss functions to supervise the learning of distance map and evolution representation. Concerning the distance map, we can directly calculate the ground truth from rigid proteins. However, this ground truth is naturally noisy because of the experimental resolution. In light of successful practice protein structure prediction [28] 23), we use a discretized distance map to replace the exact one. Specifically, distances are discretized into 64 bins ranging from to 22 A. The prediction of discretized distances converted into classification problem by
Dij = a (W:3 + P3a)),
(8)
where W is the learnable parameter and a is the activation function. Then the cross- cnosspyntropy loss averaged over all residue pairs is
Cdist
G log D
mn
where City represents the kthhelement of one-hot encoding of discretized actual distance. It is worth noting that the predicted distance map f) can be obtained by using the mean of each bin.
Inspired by the masked language model [20], leverage the evolution representation reconstruct masked MSA values. We consider 23 classes, including 20 common amino acid types, an unknown type, gap token, and mask token, and introduce the mask policy in Appendix Thus, a masked MSA loss can be defined as follows
M Softmax W Ad),
(10)
Net
Lmaa
Σ A log log(logg (M,))
(11)
N Nmask
JE
A=1
where Nmask denotes the number of masked tokens and A is the ground truth.
Similar to Equation ( the SE(3) transformation optimized in the inner loop will exhibit differences from the ground truth complex, generating hypergradients through the cross-modal transformer
m
cm (R' (+))
(I -XX,-R(()YY, (O)Y,, --())|1-D)) 2
(12)
5

---

where D is the ground truth distance between amino acids. Overall, the outer loss for the cross-model transformer is
Loue X1 Edist + Amh Az
(13)
where hyperparameters \1, X and A3 balance the importance of different loss terms.
3.2 Unrolled Algorithm for Hypergradient
To solve the bi-level optimization formulated as Equation the outer level and can be unrolled via the chain rule
2.. hyper gradient Loue is required in
Ooo
(t).
L outt
(14)
0(R,t))
dΦ
One method calculate is approximating
(o)) via optimizer (Re, (2) = OPT 1,-,,, #), t= 1, .
(15)
where T denotes the number iterations. We explicitly calculate the gradients
Dij X; RíY, tr,
2 Dij - Disl Dis @Y,,
0R. 0q.
Vqhi (Ro, tr, Ø) x= = =
||DiEll
(16)
i.j
2 Dis Dssl Dis.
VeL. (Re, t2 Φ) -= Σ 4
|1
where rotation matrix R and quaternion are converted to each other through the Bar-Itzhack algorithm (5) and means Hadamard product. After T rounds of iterations, (R (Ø), (()) can be approximated as (Rr,lr). We can compute the hypergradient by substituting
02 a (()) (o)) - a(Rr. tr)
(17)
0Φ
Due to the vast search space and the rugged landscape of the loss, the gradient descent algorithm with random initialization often requires a large number of iterations converge and struggles to find the global minima. In the next subsection, we will derive a spectral initialization to enhance convergence.
3.3 Spectral Initialization
Recall that we intend to derive a good initialization (Ry, to) for the gradient descent of Equation ( To simplify it, we denote
RY T where 1, is the all-ones vector. Then substitute into
[|x, RY, D.)) X, X, 2XT7, + Y, D3 2D,; XX Y, ||-
(18)
According to these variable combinations, we define four variables and two centering matrices
X
X
YTh
Y.Y.
X
X
YTh
B
c
X
X X
9T8
YT
mXn
XT
XTV
DJ
DJ
(19)
E E=XTÝ=
x, Yı
x Yn
F.
D
D
XTV
TTV
D2
D
mxn
mXn
Hm T
Jm
H
T
Jus
m
n
where represents the identity matrix and J refers to the all-ones matrix. If the condition Vi.j, | Y5ll D., holds, the following equation exists
HmFFH Hm. (B- - 2E + C)H, -2Hm -2H RY H
(20)

---

Table Statistics datasets.
Datasets
# Pairs of Proteins
Residues per Protein
# Atoms per Protein
Training Set
4890
565.9 (1:264.9)
4334.4 (1:2028.3)
DB5.5 (Test)
24
428.4 (+132.0)
3308.0 (+ (+1000.5)
VH-VL (Test)
68
230.1 (15.4)
1749.9 (L53.77
AB-AG (Test)
68
433.0 (+72.0)
3346.7 (+:568.8)
By performing singular value decomposition (SVD) for H and Y H, respectively
Hm X Ex
YH, Uy
(21)
the rotation matrix can be solved as
R 2
(22)
Given the rotation matrix R, the translation vector can be gotten using
HmF H [B 2E + HmB - 2H RY - 22mxxtt,,
(23)
2X NtY),.
(24)
By replacing random initialization with the above spectral initialization, the gradient descent will approach the global minima faster and better. We will empirically verify this conclusion through subsequent experiments.
Experiments
Datasets We leverage Docking Benchmark 5.5 (DB5.5) [52), gold standard dataset tailored for rigid docking. Following the experimental setting of EquiDock [25], the dataset is randomly partitioned into test split of size 24. For comprehensive comparison, antibodies (VH-VL) and antibody-antigge complexes (AB-AG) from Protein Data Bank (PDB) B and expect them to become benchmarks. Specifically, the training set consists of 4,890 complexes containing at least one antibody chain, while the test set comprises 68 antibody-antigen complexes released after October 2022. The docking results of variable heavy-light chains and antigen-antibodd can be separately evaluated. We direct the readers of interest to Appendix A for extraction criteria and detailed identifier lists. The statistics the dataset are summarized Table [
curate two datasets of
Evaluation Protocol We compare BiDock with two categories of representative methods, including three docking software ZDOCK [13,, ClusPro (2), and HDOCK (58). and two deep learning models
EquiDock (25). and AlphaFold-Multimer (Multimer for short 23 To measure the quality of
Table 2: Quantitative results on protein docking. (bold: best; underline: runner-up)
Datasets
Metrics
ZDOCK
ClusPro
HDOCK
EquiDock
Multimer
BiDock
RMSD +
12.491 +6224
14. 135
11.328:a80r3
14.982+5.30
7.797=74t
7,280+s.117
DB5.5
TM-score t
0.689 14
0.702+o.1s
0.742+0.167
0.714
0.821
0.847+0158
DockQ f
1..84
10.11
0,314.1..990
1.,00
0.469 10.396
0.564.10366
RMSD +.
10.982+3864
5.89945.6s
2.032+42388
28.29342871
1.325
1.242+t0mm
VH-VL
TM-score t
0.596100775
0.792.1.156
0,926110.100
0.559.10.07
0.962.10000
0.966.10021
DockQ f
0.108.10.18
0.40410.27
0,705
0.032.10.016
0.765
10.094
0.773
RMSD 4
18.892117757
15.67016896
15.779
18. 4468-2..66
13.650-15.68
9.707.18.79
AB-AG
TM-score t
0.504
1.59610.143
0.612:1..337
0.502.10.066
0.640
0.773.10.167
DockQ 1
0.035 4ouss1
108
0.090++0.157
+..43
0.108
0.342-+0.38
maxDockQ
0.042
0. 136
0.1 0.111+4..0
0.043++0.08
0.125
0.414-+0.30
7

---

1.4
Table 3: Total inference time of differ- ent methods antibody-antigen dock- ing. (unit: hour)
6.7
65
03
Methods
Inference time
03
02
0.3
04 05
66
07
08
(1)
(1)
ZDOCK
72.61
Figure 4: An intuitive comparison of DockQ metric on antibody-antigen docking. The box inside (a) violin plot represents 25-75 percentiles, and the median is shown by white dot. Seatters (b) scatter plot appear under the dashed diagonal line, indicating that BiDock outperforms Multimer on these complexes.
ClusPro
87.27
HDOCK
20.40
EquiDock
0.60
Multimer
1.07
BiDock
1.47
predictions, we report universally accepted metrics Root Mean Square Deviation (RMSD), T'-ssorre (Template Modeling score), and DockQ [6]. Please note that in the context of antibody-antige docking, the original DockQ metric evaluates the docking performance by treating the entire antibody as single entity to the antigen. Furthermore, we can assess the docking results separately for variable heavy/light chain (VH/VL) to the antigen and denote the maximum value maxDockQ. Refer to Appendix B for details of experiments, including implementation and hyperparameters.
4.1 Main Results and Analysis
Docking Results Table demonstrates that BiDock generally produces acceptable predictions on all datasets. Notably, BiDock achieves a significant improvement in performance on the more challenging caubbody-antigen docking, with 234% relative gain over the runner-up on the DockQ metric. These results highlight the effectiveness of our bi-level optimization, which effectively leverages multimodal information. We also observe that some established docking software, such as HDOCK, still provides reliable predictions. In contrast, deep learning methods may have some leeway in performance. For instance, the mean and deviation RMSD evaluated from EquiDack are relatively large, indicating that some inappropriate SE(3) transformations are learned.
To be intuitive, we select representative baselines, HDOCK and Multimer, along with BiDock to analyze their performance differences on antibody-antigen docking. The distribution of DockQ for each method displayed using violin plot, and a direct comparison between Multimer and BiDock is also drawn in Figure 4 It reveals that the distribution of DockQ for BiDock is more concentrated around high values, indicating higher percentage of successful docking predictions. Overall, these results suggest that our proposed BiDock is promising approach for rigid protein docking.
Visualization To visually demonstrate the superiority of our proposed BiDock, we chose the spike glycoprotein 7F6Z coronaviruses into host cells, making it a target of great interest for therapeutic intervention and vaccine development. In Figure 5 align the ground truth structure of this protein with predictions from competitive baselines. Upon inspection, is evident that predictions from baselines exhibit noticeable deviations from the ground truth structures. Such inaccuracies significantly impact understanding of the binding mechanisms and hinder the design of effective interventions. In contrast,
example antigen. The spike glycoprotein is crucial for the entry of
ground fruth vs. HDOCK DeciQ: 0.088
ground truth vss. Multimer DockQ: 0,037
ground truth vAs BiDock DockQ 0.912
Figure 5: Structure comparison between predictions and the ground truth of protein complex 7 7F6Z. The ground truth structures are represented in light gray, while predictions are colored cyan.

---

Table 4: Ablation studies on spectral initialization, bi- level optimization, and masked MSA loss. (number in parentheses: gradient descent steps; unit: thou- sand; bold: best)
Table 5: Impacts of different optimizers on antibody-antigen docking. (number in parentheses: gradient descent steps; unit: thousand: bold: best)
RMSD
DockQ
RMSD
DockQ
w/o SI(1)
14. =.0003
0.187
SGD(1)
9.939+8.525
0.333
w/o SI(2)
11.191
0.2901103228
SGD(2)
0.335 9.940118:57
w/o SI(4)
9.806 8.646
0.335
SGD(5)
0.337:10.345 9.914
w/o Bi(
T0.090
0.220
SGD(10)
9.861 +8.548
0.337++0.44
w/o MM(2)
9.821 8.6888
0.330
Adam(1)
9.780+t8773
0.341 =0.35
BiDock(22) 9.707 8.759 0.342++0.351
Adam(2) 9.707
0.34210.351
BiDock successfully captures the correct docking interface and accurately predicts the binding pose, highlighting its potential for providing biological insights and aiding drug design.
Computational Efficiency Table shows the total inference time for antibody-antigee complexes. Traditional docking software, involving candidate sampling. ranking, and refinement steps, incurs substantial computational costs. Fortunately, deep learning methods provide a significant speed- up. which is particularly important in efficient screening. Although EquiDock the fastest, its performance falls short of traditional software due to limitations in leveraging coevolution information and simple networks. On the other hand, Multimer and BiDock exhibit comparable inference times. Considering the performance improvement of our model, this trade-off is acceptable.
4.2
Ablation Studies
Effects of Spectral Initialization Recalling our utilization of spectral estimation to derive numerical solution for initializing the gradient descent in the inner loop, we conduct ablation experiments to demonstrate the effectiveness of this spectral initialization. fable&Apresent the results on antibody-antigen docking, where the variant without spectral initialization is denoted as "w/o SI" and the number gradient descent steps listed in parentheses. It is seen that the "w/o SI" variants exhibit slower convergence and lower accuracy, even when we further increase the number of gradient descent steps. These findings further support the benefits of our proposed spectral initialization in accelerating the optimization process and achieving superior results.
Effects Bi-level Optimization To justify the effectiveness bi-level optimization, we use cross-entropy
two-stage strategy: training outer and inner loops separately. Specifically, and masked MSA losses to train cross-modal transformer. Based on the resulting distance map, we compute roto-translation transformation with spectral initialization. The results antibody-antigen docking are presented in Table where "wfo Bi" denotes the variant without bi-level optimization (results reported from the original paper (34). Results support our contributions of employing bi-level optimization for end-to-end optimization, which customizes the parameter learning of the cross-modal transformer for rigid docking.
Effects of Masked MSA Loss In addition to the essential loss for the distance map, we introduce a masked MSA loss in the outer loop supervise the learning of evolution representations. We also conduct an ablation study to investigate its significance, as shown in Table The "w/o MM" refers models without masked MSA loss. According the results, it can be concluded that the masked MSA loss enables the cross-modal transformer to more effectively leverage the rich evolutionary information and seamlessly integrate with the structure-modal information.
we have the flexibility to choose different optimizers for the inner loop. To explore the impact of different optimizers on convergence speed and performance, compare the effects of SGD and Adam optimizers. We vary the number of
Effects of Optimizer As shown in Equation 15

---

gradient descent steps and evaluate the final performance, as displayed in Table 5 With Adam, BiDock achieves faster convergence and attains higher accuracy, indicating that a better optimizer can navigate the landscape and find better minima during the optimization of the inner loop.
5 Conclusion
In this study, we introduce BiDock, a novel model for rigid protein docking that tackles the challenge of accurately predicting the 3D structure of protein complexes from unbound states. By formulating this problem as bi-level optimization, BiDock combines the advantages of integrating multimodal information by cross-modal transformer in the outer loop and maintaining the rigidity of learning roto-translation transformation in the inner loop. Additionally, we derive a spectral initialization to expedite convergence. The maximum 234% relative performance improvement validates the effectiveness of BiDock in rigid protein docking.
Limitations and Broader Impact Despite the encouraging results, BilDock does not explicitly incorporate geometric constraints between residues when learning the distance map and does not account for potential atom clashes. Our future work will address these limitations and extend framework to general proteins. Protein docking deepens our understanding of biological mechanisms and aids in the design of targeted interventions. This research may inspire the A14Science community to pay more attention to the practical challenges in docking and promote further advancements in this field with significant world implications.
10

---

Acknowledgments and Disclosure of Funding
Acknowledgments We would like to extend our heartfelt appreciation to the anonymous reviewers who dedicated their valuable time and expertise to meticulously review this paper. Their constructive feedback significantly contributed the refinement and enhancement of the quality of our work. Our gratitude also extends to our team members, whose insightful discussions and collaborative brainstorming sessions were instrumental in shaping this research.
We wish to express our sincere thanks to the Data Science team at BioMap, especially to Tingting Tang, Nachuan Shan, Shiro Hong. Di Wang. Dacheng Li, Chenrui Xu, Ke Wang, and Cheng Chen, for their exceptional support in data collection, processing, and analysis. Their efforts with complex biological datasets were indispensable to the success of our research. Furthermore, we also express our sincere thanks to the Al Infracture team at BioMap, especially to Ming Yang, Chuang Yu, Zedong Zheng, Zezhi Wang. Qian Wang. and Xiaoming Zhang, for their outstanding technical support and seamless collaboration. Their proactive approach and effective troubleshooting played a vital role in ensuring the smooth progress of our research.
Finally, thanks Aaron for proofreading the manuscript.
Author Contributions The author's contributions to this paper encompass range of vital aspects. Ruijia Wang made substantial contributions to content, writing, and experiment design. Yiwu Sun took on central role in implementing the bi-level optimization algorithm and conducting experiments. Yujie Luo and Shaochuan Li were crucial in developing the precursor algorithm, x XTrimoDock (34] and also aiding in data preparation.
Le Song and Hui Li lead the antibody-antigen structure prediction project in BioMap. They made invaluable contributions by providing algorithmic insights and guidance to the project, influeneing the overall conceptualization and direction of the research. Xingyi Cheng, Chuan Shi and Cheng Yang provided valuable guidance in organizing the writing of the paper.
This work was funded by BioMap. The intellectual property of BiDock is 100% owned by BioMap.
References
[1] Juhan Bae and Roger B Grosse. Delta-stn: Efficient bilevel optimization for neural networks using structured response jacobians. In NeurlPS, pages 21725-21737, 2020.
[2] Minkyung Back, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov, Gyu Rie Lee, Jue Wang, Qian Cong, Lisa N Kinch, R Dustin Schaeffer, et al. Accurate prediction of protein structures and interactions using three-track neural network. Science, 373(6557).871-877 2021.
[3] Protein Data Bank. Protein data bank. Nature New Biol, 233:223, 1971.
[4] Jingxiao Bao, Xiao He, and John ZH Zhang. Deepbsp-a machine learning method for accurate prediction of protein-ligand docking structures. Journal of Chemical Information and Modeling, ((65):221-22440 2021.
[5] Itzhack Y Bar-Itzhack. New method for extracting the quaternion from rotation matrix. Journal of guidance, control, and dynamics, 2366) 2000.
[6] Sankar Basu and Björn Wallner. Dockq: a quality measure for protein-protein docking models. PloS one, 1188)10061887, 2016.
[7] Jacek Biesiada, Aleksey Porollo, Prakash Velayutham, Michal Kouril, and Jaroslaw Meller. Survey of public domain software for docking simulations and virtual screening. Human genomics, 5(5):199, 2011.
[8] Ondrej Bohdal, Yongxin Yang, and Timothy Hospedales. Evograd: Efficient gradient-based meta-learning and hyperparameter optimization. In NeurIPS, pages 22234-22246, 2021.
[9] Jerome Bracken and James T McGill. Mathematical programs with optimization problems in the constraints. Operations research, 21(1):37-44, 1973.

---

[10] Patrick Bryant, Gabriele Pozzati, and Ame Elofsson. Improved prediction of protein-protein interactions using alphafold2. Nature communications, 13(1):1-11, 2022.
[11] Can Chen, Xi Chen, Chen Ma, Zixuan Liu, and Xue Liu. Gradient- based bi-level optimization for deep learning: survey, arXiv preprint arXiv:2207. 11719, 2022.
[12] Can Chen, Jingbo Zhou, Fan Wang, Xue Liu, and Dejing Dou. Structure-aware protein self- supervised learning. Bioinformatics, 39(4):btad18) 2023.
[13] Rong Chen, Li Li, and Zhiping Weng. Zdock: an initial-stage protein-docking algorithm. Proteins:
Structure, Function, and Bioinformatics, 52(1))80-87, 2003.
[14]
Yihong Chen, Bei Chen, Xiangnan He, Chen Gao, Yong Li, Jian-Guang Lou, and Yue Wang. Aopt: Learn to regularize recommender models finer levels. In KDD, pages 978-986, 2019.
[15] Charles Christoffer, Siyang Chen, Vijay Bharadwaj. Tunde Aderinwale, Vidhur Kumar, Matin Hormati, and Daisuke Kihara. Lzerd webserver for pairwise and multiple protein- docking, Nucleic Acids Research, 49(W1):W39-W336. 2021.
[16] Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi S Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking, In NeurIPS, 2022.
[17] Bowen Dai and Chris Bailey-Kelloge Protein interaction interface region prediction by geo- metric deep learning. Bioinformatics, 37(17):2580-25 25800, 2021.
[18] Ana Davila, Zichang Xu, Songling Li, John Rozewicki, Jan Wilamowski, Sergei Kotelnikov, Dima Kozakov, Shunsuke Teraguchi, and Daron M Standley. Abadapt: adaptive approach to predicting antibody-antigen complex structures from sequence. Bioinformatics Advances, 2(1 (1)abbac015, 2022.
[19] Sjoerd De Vries, Mare Van Dijk, and Alexandre MJJ Bonvin. The haddock web server for data-driven biomolecular docking. Nature protocols, 5(5):883-897, 2010.
[20] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04800 04005 2018.
[21] Jerome Eberhardt, Diogo Santos-Martins, Andreas Tillack, and Stefano Forli. Autodock vina 1.2. O: New docking methods, expanded force field, and python bindings. Journal of Chemical Information and Modeling, 61(8338891-388 -3898, 2021.
[22] Stephan Eismann, Raphael JL Townshend, Nathaniel Thomas, Milind Jagota, Bowen Jing, and Ron O Dror. Hierarchical, rotation-equivariant neural networks to select structural models of protein complexes. Proteins: Structure, Function, and Bioinformatics, (9(55:4933555 -001, 2021.
[23] Richard Evans, Michael O'Neill, Alexander Pritzel, Natasha Antropova, Andrew W Senior, Timothy Green, Augustin Zidek, Russell Bates, Sam Blackwell, Jason Yim, al. Protein complex prediction with alphafold-multimer. multimer. BioRxiv, 2021.
[24] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In ICML, pages 1126-1135. PMLR, 2017.
[25] Octavian Setavian-Eugen Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi S. Jaakkola, and Andreas Krause. Independent se(3)-equivariant models for end-to-end rigid protein docking. In ICLRR 2022.
[26] Ian R Humphreys, Jimin Pei, Minkyung Back, Aditya Krishnakumar, Ivan Anishchenko, Sergey Ovehinnikov, Jing Zhang, Travis Ness, Sudeep Banjade, Saket Bagde, et Computed structures of core eukaryotic protein complexes. Science, 374(6573)eabm488)) 2021.
[27] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Antibody-antigen docking and design via hierarchical structure refinement. In ICML, pages 10217-10227. PMLR, 2022.
12

---

[28] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger. Kathryn Tunyasuvunakool, Russ Bates, Augustin Zídek. Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-589, 2021.
[29]
Dima Kozakov, David R Hall, Bing Xia, Kathryn Porter, Dzmitry Padhorny, Christine Yueh, Dmitri Beglov, and Sandor Vajda. The cluspro web server for protein-protein protein docking, Nature protocols, 12(2):255-21) 2017.
[30] Elodie Laine, Stephan Eismann, Ame Elofsson, and Sergei Grudinin. Protein sequence-to- structure learning: Is this the end (-to-end revolution)? Proteins: Structure, Function, and Bioinformaticx, (12);: (1112):17700 1770-1786 2021.
[31] Ingoo Lee, Jongsoo Keum, and Hojung Nam. Deepconv-dti: Prediction of drug-target interac- tions via deep learning with convolution protein sequences. PLoS computational biology, (506)ee 2019.
[32] Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. In ICLR, 2018.
[33] Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):0045-10067, 2021.
[34] Yujie Luo, Shaochuan Li, Yiwu Sun, Ruijia Wang, Tingting Tang, Beiqi Hongdu, Xingyi Cheng, Chuan Shi, Hui Li, and Le Song. xtrimodock; Rigid protein docking via cross-modal representation learning and spectral algorithm. bioRxiv, pages 2023-02, 2023.
[35] Andrew T McNut, Paul Francoeur, Rishal Aggarwal, Tomohide Masuda, Rocco Meli, Matthew Ragoza, Jocelyn Sunseri, and David Ryan Koes. Gnina 1.0: molecular docking with deep learning. Journal of cheminformattis, 13(1):1-20, 2021.
[36] Matthew McPartlon and Jinbo Xu. Deep learning for flexible and site-specific protein docking and design. bioRxiv, pages 2023-04, 2023.
[37] lain H Moal, Mieczyslaw Torchala, Paul A Bates, and Juan Fernández-Recio. The scoring of poses in protein-proteii protein docking: current capabilities and future directions. BMC bioinformatics, 14(1):1-15, 2013.
[38] Thin Nguyen, Hang Le, Thomas P Quinn, Tri Nguyen, Thue Duy Le, and Svetha Venkatesh. Graphdta: Predicting drug-target binding affinity with graph neural networks. Bioinformatics, 37(8):1140-1147 2021.
[39] Hakime Öztürk, Arzucan Özgür, and Elif Ozkirimli. Deepdta: deep drug-target binding affinity prediction. Bioinformaticc, 34( (4((1))i821-)) -8229, 2018
[40] Jimin Pei, Jing Zhang, and Qian Cong. Human mitochondrial protein complexes revealed by large-scale coevolution analysis and deep samming-based structure modeling. Bioinformatics, 38(18):4301-4311 2022.
[41] Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. In NeurIPS, 2019.
[42] Anna Rutkowska, Douglas W Thomson, Johanna Vappiani, Thilo Werner, Katrin M Mueller, Lars Dittus, Jana Krause, Marcel Muelbaier, Giovanna Bergamini, and Marcus Bantscheff. modular probe strategy for drug localization, target identification and target occupancy measurement on single cell level. ACS chemical biology, 11(9):2541 11-22511,,5,,,,, 2016.
[43] Rita Santos, Oleg Ursu, Anna Gaulton, A Patrícia Bento, Ramesh S Donadi, Cristian G Bologa, Anneli Karlsson, Bissan Al-Lazikani, Anne Hersey, Tudor Oprea, et al. A comprehensive map of molecular drug targets. Nature reviews Drug discovery, 16(1):19-34, 2017.
[44] Christina EM Schindler, Isaure Chauvot de Beauchêne, Sjoerd J de Vries, and Martin Zacharias. Protein-protein and peptide-proted docking and refinement using attract in capri. Proteins: Structure, Function, and Bioinformatics, 85(3):391-398) 2017.
13

---

[45] Sharon Sunny and PB Jayaraj. Fpdock: Protein Protein-protein docking using flower pollination algorithm. Computational Biology and Chemistry, 93:107518, 2021.
[46] Mieczyslaw Torchala, lain H Moal, Raphael AG Chaleil, Juan Fernandez and Paul Bates. Swarmdock: a server for flexible
protein docking. Bioinformatics, 29(6):077-
809, 2013.
[47] Ilya Vakser. Protein-pon prottin docking: From interaction to interactome. Biophysical journal, 107(8):1785-1799 1793, 2014.
[48] Thirumalaisamy P Velavan and Christian G Meyer. The covid- 19 epidemic. Tropical medicine international health, 25(3):278, 2020.
[49] Vishwesh Venkatraman, Yifeng D Yang. Lee Sael, and Daisuke Kihara. Protein- Protein-protein docking using region-basee 3d zermike descriptors. BMC bioinformatics, 10(1):1-11, 2009.
[50] Jacob Verburgt and Daisuke Kihara. Benchmarking of structure refinement methods for protein complex models. Proteins: Structure, Function, and Bioinformatics, 90(1):8339)) 2022.
[51] Heinrich Von Stackelberg. Market structure and equilibrium. Springer Science Business Media, 2010.
[52] Thom Vreven, lain H Moal, Anna Vangone, Brian G Pierce, Panagiotis L Kastritis, Mieczyslaw Torchala. Raphael Chaleil, Brian Limnnzz-Garch Paul A Bates, Juan Fernandez-Recio et al. Updates to the integrated protein protinnppoennnnt interaction benchmarks: docking benchmark version
5 and affinity benchmark version 2. Journal of molecular biology, 427(11)33031-300 1-301,, 2015.
[53] Izhar Wallach, Michael Dzamba, and Abraham Heifets. neural network for bioactivity prediction in structure-basse drug discovery. arXiv preprint arXiv:1510.022 028555, 2015.
Atomnet: a deep convolutional
[54] Yining Wang, Xumeng Gong, Shaochuan Li, Bing Yang, Yiwu Sun, Yujie Luo, Hui Li, and Le Song. Fast de novo antibody structure prediction with atomic accuracy. Cancer Research, 3377SSuplemeen):429444226 2023.
[55] Yining Wang, Xumeng Gong, Shaochuan L.,, Bing Yang. YiWu Sun, Chuan Shi, Hui Li, Yangang Wang, Cheng Yang, and Le Song. xtrimoabfold: Improving antibody structure prediction without multiple sequence alignments. arXiv preprint arXiv:2212.00735, 2022.
[56) Yining Wang, Xumeng Gong, Shaochuan Li, Bing Yang. Yiwu Sun, Chuan Shi, Yangang Wang, Cheng Yang, Hui Li, and Le Song. xtrimoabfold: De novo antibody structure prediction without msa. ArXiv, abs/2212.00735, 2022.
[57] Gaoqi Weng, Ercheng Wang, Zhe Wang, Hui Liu, Feng Zhu, Dan Li, and Tingjun Hou. Hawk- dock: web server to predict and analyze the protein-protein complex based on computational docking and mm/gbsa. Nucleic acids research, 47211::332-W330 2019.
[58] Yumeng Yan, Huanyu Tao, Jiahua He, and Sheng- You Huang. The hdock server for integrated protein-protein docking. Nature protocols, 1885)11829-1852 2020.
[59] Junjie Yang, Kaiyi Ji, and Yingbin Liang. Provably faster algorithms for bilevel optimization. In NeurIPS, pages 13670-13682, 2021.
[60] Chia-Hung Yuan and Shan-Hung Wu. Neural tangent generalization attacks. ICML,, pages 12230-12240. PMLR, 2021.
[61] Jonas Zehnder, Yue Li. Stelian Coros, and Bernhard Thomaszewski. Ntopo: Mesh-free topology optimization using implicit neural representations. In NeurlPS, pages 10368-10381, 2021.
[62] Tian-ming Zhou, Sheng Wang, and Jinbo Xu. Deep learning reveals many more inter inter-protein residue-residue contacts than direct coupling analysis. BioRxiv, page 240754, 2018.
[63] Marinka Zitnik, Francis Nguyen, Bo Wang, Jure Leskovec, Anna Goldenberg, and Michael M Hoffman. Machine learning for integrating data in biology and medicine: Principles, practice, and opportunitics. Information Fusion, 50:71-91, 2019.
14

---

A Details of Dataset
Background of Antibodies Antibodies are vital components of the immune system and are classified into various classes, including IgG, IgM, IgA, IgD, and IgE. Among them, IgG antibodies are the most abundant in the bloodstream and play primary role in im- mune responses against pathogens.
Antigenbinding
Fab region
Hinge
As depicted in Figure 6 IgG antibodies ex-
Disufide bond
C
C- constant domain
hibit a Y-shaped structure composed of two identical light chains and two identical heavy chains, where heavy chains provide struc- tural stability. Each antibody chain is further divided into distinct regions. (1) The vari- able regions, referred to the variable heavy (VH) and variable light (VL) regions, are lo-
Foregion
V. variable domain
chain
light
Figure Structure of an IgG antibody. The heavy chain is colored orange, while the light chain is blue.
cated at the tips of the Y arms. These regions contribute to the specificity of antibodies in recognizing and binding to antigens. The VH and VL regions collaborate to form the fragment untigen-bindingg (Fab) region. (2) At the base of the Y structure, the constant regions, also known the fragment crystallizable (Fe) region, are important in the effector functions of antibodies. The Fe region interacts with immune cells and triggers immune responses, such as the activation of complement proteins for pathogen destruction and the promotion of phagocytosis.
Given this background knowledge of antibodies, it becomes clear that antibody-antigen docking is fundamental in immune responses, therapeutic applications, vaceine development, and drug discovery. Therefore, our study places particular emphasis on antibody-antigen docking, contributing to this field by curating high-quality benchmark. This dataset will serve as a valuable resource for evaluating computational models in predicting antibody-anttigen interactions, ultimately facilitating the development of novel therapeutics and immunological interventions.
Antibody-antigen Benchmark The training set comprises 4,890 complexes of antibody-antigenn pairs, each consisting of proteins with minimum of 30 residues. These complexes encompass three chains, including the light and heavy chains of the antibody, along with one antigen chain. All complexes were released before January 2022. Similarly, the test set consists of 68 antibody- complexes with three chains, released after October 2022. Thus, we ensure that neither baselines nor proposed model trained using the test set and avoid data leakage.
In practical applications, obtaining the ground truth structures of aatibody-antigee complexes poses significant challenges. Researchers often turn to existing folding models to predict them. To simulate real-world scenarios, we employ a specialized antibody model called xTrimoABoold [55] 56 (4]
to predict the conformations of antibodies and AlphaFold2 (28) for antigens. Given these predicted structures as rigid structures, we construct training and test datasets essential for further analysis and investigation. Here are the PDB identifiers and their corresponding chain identifiers for the test set.
8dls:A.H.L; 8dIr:A,H.L;; &dfi:A,H,LL 8dfh:A.H.L; Sdc,:E,L,H: Sdad:H,L,B; 72r8:A,H,L;; 7 H,LEE
7x26:H,1,K 72 D,,, H,,;:A,H,L; 7ws6:C.H,I; 7ws2:D.A.E; 7xxl:C,A,B; 7wro 7w
7wrl:A,B,R; 7wrj:R, W.w.::....BBBB.lc....... 7wIc:E,H,I;
Tee 7urq:A.H.I Aua
1rw.4444.....lll......l.l1111111 J.C.
7tx:
|02.444|..SSHHH..............010 7,,,:
hrt.A 710::,,H..
7107:2,H,L; 701.444411..1111117.//4.44411111 H. 7eng:L,,H.B;
H.kO:R.H.L; 7ejz:R, 7ey. 7100
15

---

B Details of Implementation
Baselines ZDOCK ClusPrdj and HDOCK" are user-friendly local packages suitable for auto- mated experiments web servers for manual submissions. We select the top-1 predicted structure from each of these methods for subsequent evaluation. For EquiDock and Multime we utilize their pretrained models available on GitHub for the inference. It is worth emphasizing that all methods except Multimer are designed for docking two chains. Therefore, during the evaluation, we employ a sequential docking strategy. This entails initially docking the light chain and heavy chain together, followed by treating them as unified entity for docking with the antigen. And we calculate evaluation metries using the tools USalign 10 and Dock
MSA Extraction We utilize the heuristic approach described in [23 pair sequences from per- chain multiple sequence alignments (MSAs). Initially. the per-chain MSA sequences are grouped based on species, with the species labels obtained from UniProt's idmapping species group, the sequences are paired together. We match the chain MSAs by minimizing the base-pair distance between the chains for prokaryotic species. While in terms of eukaryotic species, we order them based sequence identity to the target sequence [62]. To reduce computational and memory costs, we employ the MSA clustering approach from AlphaFold2 [28]. We randomly select Neta 252 sequences as the MSA cluster centers, with the primary protein sequence always set as the first cluster center. The remaining sequences are assigned to their closest cluster based on the Hamming distance.
Within each specific
Sequence-modal Input The sequence modality incorporates information derived from the primary sequence itself and co-evolutionary information obtained from MSAs. Following prior research [28] 233. extract two types of features: type features Ftyp R Noes x21 and primary pair features FPP E R Nrea x xNrexx7' from the primary sequence, where Nres represents the number of residues. Regarding MSAs, utilize cluster MSA features pmsaa Na. X.e.s x49 where Nets denotes the number of cluster centers. Specifically,
The type feature FLye E Ne X21 comprises one-hot representations of the amino acid types, encompassing the 20 known amino acids and one additional category for unknown types.
The primary pair feature TPP N N.e.s x73 contains positional information within or across chains, including three components. (1) The relative positional feature of size [Nree- Nres 66] represents the relative residue indices, which are clipped between [-32, 32]. The 66-th index used to indicate cross-chain pairs. (2) The entity indicator of size [Nres. Nres, 1] identifies whether
residues and j originate from the same chain. (3) The relative index feature of size [Nress Nres, 6)
introduces the relative sym id 13 indices clipped between [-2, 2]. The 6-th index is assigned to pairs where the two residues have different sym_ids.
The cluster MSA feature pmsa E Nets X consists of five components. (1) The one-hot representation of the amino acid types with size [Ncl.. Nre.. 23], including 20 amino acids, one unknown type, one gap or missing residue, and one mask token as introduced in Section 3.1 (2)
The amino acid distribution of size [Nel:: Nyes 23] represents the distribution of amino acid types within each MSA cluster. (3) The deletion indicator of size [Nets: Nres. 1] indicates whether there is a deletion to the left of each residue. (4) The deletion value of size [Note. Nres. 1] is calculated using the formula arctan 5. where c refers to the number of deletions to the left of each position.
htttps://zdee umassmed. edu
https://cllssee org hust. edu.
11
[dmapping
'TThe sym id is used to distinguish chains with the same sequence. For example, we consider a complex comprising five chains {A, B, B, C, C). where A, B, and C' represent three unique chains. The corresponding sym_ids for each chain would be {1, 1, 1,2), respectively.
16

---

(5) The mean deletion value of size (Nats, Nres 1] is computed as 2 arctan 3. where é represents the average number of deletions all residues on the left of each position.
Structure-modal Input For the structure modality, we extract angle features paag E R -.er and pair features FD E R. X.r.. x88 from the rigid protein structures. These features capture important Structure-modal information and are used as input for our docking model. Specifically,
The angle feature Fong R.Neexxtt comprises three components. (I) The one-hot representation of the amino acid types with size of [Nres: 22], including 20 amino acids, one unknown type, and one gap or missing residue. (2) The angle representations of size ,e 28] use sine and cosine to encode three backbone torsion angles, four side-chain torsion angles, and alternative torsion angles with 180 rotation symmetry for each local frame of residue. (3) The angle indicator with size [Nres, 7] indicates the presence or absence of torsion angles.
The pair feature FP E R.N XN... x88 comprises five components. (1) The distogram feature of size ree Nres: 39] represents the discretized distances between CB atoms, In the case of glycine, which lacks C3 atoms, Ca is used instead. The distances are discretized into 38 bins of equal width ranging from 3.25 to 50.75Á, with an additional bin accounting for larger distances. (2) The residue type feature of size [Nrear Nrea, 44] is derived from expanding one-hot representations of residue types with dimensions of [Nres, 1, 22] and [Nres, 22, IJ. (3) The backbone feature of size [Nress Nrea 3] is obtained by constructing the unit vector of the local frame through the Gram-Schman Schmidt process based on the original N-Ca-C coordinates. (4) The residue indicator with size Frees,Nree 1] is expanded from the indicator of residue existence. (5) The pair indicator of size [Nres, Nre.. 1] indicates whether the pair is masked.
MSA Mask Policy Reflecting on Section 3.1 we design masked MSA loss supervise the learning of evolution representations and the integration of cross-modal information. Specifically, we randomly mask each position an MSA cluster center with 15% probability. Each masked token replaced according to the following policies:
70% probability of substitution with special token *
10% probability of substitution with randomly selected amino acid from uniform distribution
10% probability of substitution with an amino acid sampled from the MSA profile that corresponds the position
10% probability of no substitution
Hyperparameter Settings We initialize specific parameters of the cross-modal transformer with the checkpoint of Multimer and implement bi-level optimization using TorchOp library. The crop size is set to 412, and the batch size is set to 1. The coefficients in Equation (13) are X = 0.2, A2 = 2.0, and >3- 10.0. For optimization, we employ the Adam optimizer with learning rate of 10-4 and integrate learning rate warmup, gradually increasing the learning rate from 0 to 10-1 within the first 100 steps. The exponential moving average (EMA) strategy applies a decay rate of 3 0.999 and undergoes updates every 200 steps. The environment where we run all experiments is:
Operating system: Linux version 5.13.0-30 -.3.0-30--eeeriic
CPU information: AMD EPYC 7742 64-Core Processor
GPU information: NVIDIA A100-SXM4-80GB
C
Additional Results
Effects of Noisy Structures Classical software rely on score functions derived from statistics in the protein data bank. This dependency renders them susceptible to noise. When using folding algorithms to predict unbounded proteins, the performance of these software can degrade significantly. To validate this intuition, we conduct a docking performance analysis on the DB5.5 dataset using ground truth and predicted structures from folding models as unbounded structures, respectively. As shown in Table 6 these results illustrate that although HDOCK performs exceptionally well with
"AApache-2 2.0 otcees/srrre////taoooooooorrrrrr
17

---

Table Impacts of noisy structures on the docking performance of classical software and BiDock.
(bold: best; underline: runner-up)
Ground Truth
Predicted Structure
RMSID +
TM-score
DockQ f
RMST>
TM-score f
DockQ 1
ZDOCK
11.830=5527
0.095=a010 0.
12.491
0.689
0.084421133
ClusPro
11.486=7.99
0.780+0153
0.204
14
0.
0.1 0.118
HDOCK
3.464.17294
0.935 1:0164
0.815 1-0.66
11.328 1.0073
0.742 10.167
0.314 10.390
BiDock
6.173 L8825
0.892 1.0156
0.648 L..432
7.280
0.847.10.1
0.564
ground truth, minor noise in predicted structures leads substantial decline in its performance. On the contrary, BiDock consistently generates acceptable predictions regardless of the imput type, showcasing robustness noise. real-world applications, reliance the availability of ground truth structures is impractical. The ability of BiDock to maintain high prediction quality when confronted with noisy structures makes an invaluable tool.
18


Reviews:
[{"review_id": "VB1ogyZV0E", "reviewer": "Reviewer_DgVe", "summary": "This paper proposes BiDock, a novel rigid protein docking model that integrates sequence and structure information through bi-level optimization. It achieves promising results, outperforming baselines by up to 234% in challenging antibody-antigen docking.", "questions": "Does the proposed multi-model model have the capability to handle tasks when there is missing structure? Are there any potential improvements or modifications that could address this issue?\n\nDo MSAs of the antibody part really work for this task?", "limitations": "The paper's limitations include the absence of explicit geometric constraints between residues in learning the distance map and neglecting potential atom clashes. The authors plan to address these issues and extend the framework to encompass general proteins in future work.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "As claimed by the authors in the paper, this work represents the first attempt to effectively leverage sequence- and structure-modal information for rigid protein docking.", "weaknesses": "The authors seem to overlook the discussion and comparison of some protein docking methods based on pre-trained models. For example, [1] utilizes the protein language model ESM2 [2] for Protein Docking task, and there are other pre-trained models focusing on protein structures [3] as well as embeddings that combine both sequence and structure information [4]. These methods are worth discussing and applying in the experimental section for a comprehensive evaluation and comparison. In general, applying pre-trained methods directly to downstream protein-related tasks often leads to significant improvements in performance.\n\nThe paper lacks sufficient details, such as the specific approach for concatenating MSA. It is unclear whether the MSA is directly concatenated or arranged based on homology, similar to AF2-multimer. Providing these implementation details, and making the code available, would enhance the reproducibility and transparency of the research.\n\n​​For antibody-antigen tasks, reporting the RMSDs of the CDRs are crucial. The high variability of CDRs makes predicting this region particularly challenging. Especially, results on CDR3 region is a critical metric for evaluating the model's performance, and its absence in the author's discussion is noteworthy. \n\n\n[1] Chu, Lee-Shin, et al. \"Flexible Protein-Protein Docking with a Multi-Track Iterative Transformer.\" bioRxiv (2023): 2023-06.\n\n[2] Lin, Zeming, et al. \"Evolutionary-scale prediction of atomic-level protein structure with a language model.\" Science 379.6637 (2023): 1123-1130.\n\n[3] Guo, Yuzhi, et al. \"Self-supervised pre-training for protein embeddings using tertiary structures.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 6. 2022.\n\n[4] Chen, Can, et al. \"Structure-aware protein self-supervised learning.\" Bioinformatics 39.4 (2023): btad189.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- We sincerely thank the Reviewer for spending time and providing valuable feedback. We appreciate all of your suggestions and we have addressed all your questions below by providing our responses.\n\n1. > The authors seem to overlook the discussion and comparison of some protein docking methods based on pre-trained models. For example, (1) some pre-trained models focus on protein structures [1] as well as embeddings that combine both sequence and structure information [2]. (2) GeoDock[3] utilizes the protein language model ESM2 [4] for protein docking task.\n\n**Answer:** Thanks for your comments. (1) General pre-trained models [1,2] are evaluated on simpler tasks such as classification or GraphQA. However, rigid docking requires a specialized decoder and standardized evaluation criteria is lacking. Following existing rigid docking models [7,8], we exclude general pre-trained models from our baselines. (2) GeoDock [3] is tailored for flexible docking. Firstly, flexible docking contradicts the rigid-body assumption [5,6], rendering it incomparable with rigid docking models. Secondly, GeoDock is released after the submission deadline, explaining its absence in our references.\n\nIn the revised version, we will cite and discuss these methods in the \"Related Work\" section to differentiate the focus.\n\n2. > The paper lacks sufficient details, such as the specific approach for concatenating MSA. It is unclear whether the MSA is directly concatenated or arranged based on homology, similar to AF2-multimer.\n\n**Answer:** Apologies for the confusion. (1) The extraction of cluster MSA features $F^{msa}\\in\\mathbb{R}^{N_{cls}\\times N_{res}\\times 49}$ is detailed in Appendix B. Roughly, we employ the heuristic approach from AlphaFold-Multimer to pair sequences, then utilize the MSA clustering method like AlphaFold2. (2) As mentioned in the paper,  Eq. (6) requires broadcasting operations. Specifically, ${M^{typ}\\in\\mathbb{R}^{N_{res}\\times c_m}}$ is broadcasted along the newly added first dimension during addition. Similarly, broadcasting is applied to the newly added first dimension of $M^{ang}$ for concatenation.\n\nThanks for your careful review. We will provide clear explanations in the revised version.\n\n3. > For antibody-antigen tasks, reporting the RMSDs of the CDRs are crucial. \n\n**Answer:** Thanks for your comments. Rigid docking [5,6] ensures that the conformation of the antibody is rigid, resulting in identical CDRs across all methods. Therefore, a comparison in this regard becomes unnecessary.\n\n4. > Does the proposed multi-model model have the capability to handle tasks when there is missing structure?\n\n**Answer:** Thanks for your comments. (1) Rigid docking [5,6] assumes access to 3D structures of unbound proteins. (2) When only amino acid sequences are available, advanced folding models like AlphaFold2 can predict the 3D structures of unbound proteins.\n\n5. > Do MSAs of the antibody part really work for this task?\n\n**Answer:** We perform an ablation study by retraining BiDock with all MSA features set to zero. The results for antibody-antigen docking are shown in the table below, labeled as \"w/o MSA\" for the variant without MSAs.\n\n**Table**\tAblation studies on sequential information from MSAs. (bold: best)\n||RMSD $\\downarrow$|TM-score $\\uparrow$|DockQ $\\uparrow$|maxDockQ $\\uparrow$|\n| ------------------- | ----- | -------- | ------- | ----- |\n| BiDock w/o MSA |20.658 $\\pm$ 5.141|0.581 $\\pm$ 0.094|0.0392 $\\pm$ 0.081|0.046 $\\pm$ 0.104|\n|BiDock|**9.707** $\\pm$ 8.759|**0.773** $\\pm$ 0.187|**0.342** $\\pm$ 0.351|**0.414** $\\pm$ 0.386|\n\nThe results indicate the essential role of sequential information from MSAs in enhancing docking performance. This underscores the significance of our framework, which combines multimodal fusion and rigid body docking via bi-level optimization.\n\nIn the revised version, we will incorporate these ablation studies to further support our contributions. We value your feedback.\n\n[1] Guo, Yuzhi, et al. \"Self-supervised pre-training for protein embeddings using tertiary structures.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 6. 2022.\n\n[2] Chen, Can, et al. \"Structure-aware protein self-supervised learning.\" Bioinformatics 39.4 (2023): btad189.\n\n[3] Chu, Lee-Shin, et al. \"Flexible Protein-Protein Docking with a Multi-Track Iterative Transformer.\" bioRxiv (2023): 2023-06.\n\n[4] Lin, Zeming, et al. \"Evolutionary-scale prediction of atomic-level protein structure with a language model.\" Science 379.6637 (2023): 1123-1130.\n\n[5] Vakser I A. Protein-protein docking: From interaction to interactome[J]. Biophysical journal, 2014, 107(8): 1785-1793.\n\n[6] Desta I T, Porter K A, Xia B, et al. Performance and its limits in rigid body protein-protein docking[J]. Structure, 2020, 28(9): 1071-1081. e3.\n\n[7] Yan Y, Tao H, He J, et al. The HDOCK server for integrated protein–protein docking[J]. Nature protocols, 2020, 15(5): 1829-1852.\n\n[8] Ganea O E, Huang X, Bunne C, et al. Independent SE (3)-Equivariant Models for End-to-End Rigid Protein Docking[C]//International Conference on Learning Representations. 2022."}, {"review_id": "CvuXIsxGpJ", "reviewer": "Reviewer_ZMi5", "summary": "This paper introduces BiDock, a novel approach that integrates sequence- and structure-modal information to improve the accuracy of rigid protein docking predictions. The proposed method uses multimodal information through bi-level optimization, enabling joint optimization of the docking score and the weights of the multimodal features. A gradient descent method is employed to predict the rotation and translation instead of relying on the Kabsch algorithm. Additionally, a spectral initialization is proposed for the gradient descent.\nThe authors demonstrate the effectiveness of BiDock by comparing it to existing docking methods on various datasets, including two newly curated datasets from PDB database. Remarkably, BiDock outperforms other methods, particularly excelling in the challenging antibody-antigen docking task. \n\nThe paper's contributions include the development of a novel approach for integrating multimodal information in protein docking, the introduction of two curated datasets, a thorough evaluation of the proposed method with baselines across multiple datasets.", "questions": "Please see the Weaknesses part.", "limitations": "Yes, the authors discuss about limitation of the proposed method.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- **Integration of multimodal protein representation**: The paper introduces a novel approach that combines both information representations and structural information, resulting in a comprehensive protein representation. This integration enhances the accuracy of rigid protein docking predictions.\n- **Bi-level optimization formulation of the protein docking problem**: The authors formulate the docking problem as a bi-level optimization task, which allows for joint optimization of the rigid docking poses and the weights of the multimodal features. By employing a gradient descent method for predicting rotation and translation, the paper offers an alternative to the traditional Kabsch matching algorithm.\n- **Spectral initialization for gradient descent**: The paper proposes a spectral initialization technique for the gradient descent method used in the bi-level optimization. \n- **Introduction of two new curated datasets**: The authors curate two new datasets from the Protein Data Bank (PDB) database, specifically focusing on antibodies (VH-VL) and antibody-antigen complexes (AB-AG). These datasets provide valuable resources for evaluating protein docking methods, particularly in the context of antibody-antigen interactions.\n- **Strong experimental results**: The paper demonstrates the superiority of the BiDock model over existing docking methods through comprehensive experiments on multiple datasets. The proposed method particularly excels in the challenging antibody-antigen docking task, showcasing its efficacy in accurately predicting protein docking conformations.\n- **Clear presentation**: The authors provide clear explanations of the proposed approach, experimental setup, and evaluation metrics, enhancing the clarity and accessibility of the research findings.", "weaknesses": "- **Unclear motivation**: The paper does not clearly explain why sequential/coevolution representations are useful specifically for rigid docking problems, leaving the reader questioning the relevance and benefits of these representations in this context. Ablation study between implementation with and without sequential information is appreciated.\n- **Choice of bi-level optimization**: The paper formulates the problem as a bi-level optimization task, but it lacks a clear explanation as to why this approach is chosen over direct end-to-end optimization using the distance map as input. The division of the optimization into an outer loop (distance map prediction) and inner loop (pose optimization) raises questions about the necessity and effectiveness of this bi-level optimization scheme.\n- **Curated dataset filtering process**: The paper lacks detailed information about the filtering process used to curate the datasets, leaving uncertainty about how the quality of complex structures in the datasets is ensured. Without transparent and rigorous filtering criteria, the reliability of the curated datasets may be compromised.\n- **Computational complexity**: BiDock exhibits higher inference time compared to existing methods like EquiDock and AlphaFold-Multimer. The computational complexity associated with the bi-level optimization process might contribute to this increased inference time, which can impact practical usability and scalability.\n- **Limited training and testing data**: The paper acknowledges that the available training and testing data are limited, which raises concerns about the generalizability and robustness of the proposed method. Insufficient data may lead to overfitting and an incomplete evaluation of BiDock's performance.\n- **Incremental novelty**: The paper lacks significant innovation in terms of network structure for BiDock. Instead, it builds upon existing techniques such as cross-modal transformers and roto-translation transformations, which may limit its originality and novelty compared to other state-of-the-art approaches in the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- We highly appreciate constructive comments from the Reviewer on our work.\n\n1. > (1) The paper does not clearly explain why sequential/coevolution representations are useful specifically for rigid docking problems. (2) Ablation study between implementation with and without sequential information is appreciated.\n\n**Answer:** Apologies for any confusion. (1) MSAs capture the evolutionary conservation of interacting residues, providing insights into residue interactions and spatial proximity. This aids in identifying crucial residues at the protein-protein interaction interface and enhances docking accuracy. (2) We perform an ablation study by retraining BiDock with all MSA features set to zero. The results for antibody-antigen docking are shown in the table below, labeled as \"w/o MSA\" for the variant without MSAs.\n\n**Table** Ablation studies on sequential information from MSAs. (bold: best)\n||RMSD $\\downarrow$|TM-score $\\uparrow$|DockQ $\\uparrow$|maxDockQ $\\uparrow$|\n| ------------------- | ----- | -------- | ------- | ----- |\n| BiDock w/o MSA |20.658 $\\pm$ 5.141|0.581 $\\pm$ 0.094|0.0392 $\\pm$ 0.081|0.046 $\\pm$ 0.104|\n|BiDock|**9.707** $\\pm$ 8.759|**0.773** $\\pm$ 0.187|**0.342** $\\pm$ 0.351|**0.414** $\\pm$ 0.386|\n\nThe results indicate the essential role of sequential information from MSAs in enhancing docking performance. This underscores the significance of our framework, which combines multimodal fusion and rigid body docking via bi-level optimization.\n\nIn the revised version, we will incorporate these explanations and ablation studies to further bolster our contributions. Your valuable feedback is greatly appreciated.\n\n2. > The paper formulates the problem as a bi-level optimization task, but it lacks a clear explanation as to why this approach is chosen over direct end-to-end optimization using the distance map as input.\n\n**Answer:** Thanks for your comments. The \"direct end-to-end optimization\" is equivalent to performing only one gradient descent step in the inner loop, approximating $(R^*, t^*)$ with $(R_1, t_1)$ in Eq. (14). Table 5 demonstrates that performance decreases when using gradient descent in the inner loop for 1000 steps compared to 2000 steps, let alone performing only one step. The reduction in gradient descent steps leads to larger approximation errors. Given this, we naturally opt for bi-level optimization over \"direct end-to-end optimization.\"\n\nWe value this question and will further emphasize the conclusions shown by the ablation studies in the revised version.\n\n3. > The paper lacks detailed information about the filtering process used to curate the datasets, leaving uncertainty about how the quality of complex structures in the datasets is ensured.\n\n**Answer:** Thanks for your comments. The dataset extraction principles and PDB identifiers are detailed in Appendix A. Roughly speaking, the training set comprises 4,890 complexes of antibody-antigen pairs with proteins containing a minimum of 30 residues. These complexes involve three chains, including the light and heavy chains of the antibody and one antigen chain, released before January 2022. Similarly, the test set includes 68 antibody-antigen complexes with three chains, released after October 2022. This ensures no data leakage for baselines or our proposed model.\n\nAdditionally, our dataset has been selected as a benchmark for the antibody-antigen complex structure prediction task on the \"Life Science Leaderboard\" (https://www.biomap.com/sota/), facilitating public access and fair comparisons.\n\n4. > BiDock exhibits higher inference time compared to existing methods like EquiDock and AlphaFold-Multimer.\n\n**Answer:** Thanks for your comments. Although EquiDock is fast, its performance falls short of traditional software due to its limitations in leveraging coevolution information and simple networks. On the other hand, AlphaFold-Multimer and our proposed BiDock exhibit comparable inference times. Considering the performance improvement of our model, this trade-off is acceptable.\n\nThanks for your thorough review. We will place greater emphasis on this point in the revised version.\n\n5. > The paper acknowledges that the available training and testing data are limited, which raises concerns about the generalizability and robustness of the proposed method.\n\n**Answer:** Thanks for your comments. (1) The three test sets are held-out, and both our proposed BiDock and the baselines do not select checkpoints based on the evaluations of test sets. (2) Similar to the evaluation of AlphaFold2 and AlphaFold-Multimer, we save only one checkpoint after convergence and evaluate its performance on all test sets. Notably, the three test sets have distinct characteristics: DB5.5 includes general proteins; VH-VL focuses on light and heavy chains of antibodies; AB-AG concerns antibody-antigen docking. The performance improvement across these diverse test sets reflects the generalization and robustness of BiDock to some extent.\n\nWe will include these clarifications in the revised version as suggested. Thanks for your feedback.\n\n6. > The paper lacks significant innovation in terms of network structure for BiDock. Instead, it builds upon existing techniques such as cross-modal transformers and roto-translation transformations.\n\n**Answer:** Thanks for your comments. (1) Roto-translation transformation is a standard operation for 3D spatial rotation and translation, and Evoformer is a well-established module like Resnet. (2) Our contributions do not center on network structure design, emphasizing these unique aspects:\n- We design a framework to naturally integrate the fusion of multimodalities and the docking of rigid bodies through bi-level optimization establishing a new avenue for rigid protein docking.\n- We solve the above bi-level optimization with unrolled gradient and the derived spectral initialization.\n- The promising results on three representative datasets demonstrate the effectiveness of the proposed model."}, {"review_id": "bb3L4n0AWx", "reviewer": "Reviewer_XfrH", "summary": "This paper studies the rigid protein-protein docking problem. The authors fuse the sequence features and structural features of proteins and unify these features into single features and pair features as done in AlphaFold2. These features are then fed into an evoformer-like cross-modal transformer to produce updated representations. The updated representations are used to predict inter-chain distance map and masked MSA tokens. The final rigid-docking results (i.e., a global translation and a global rotation) can be obtained by solving an optimization problem that minimizes the discrepancy between the distance map of the predicted complex and that of the ground-truth.\n\nThe authors claim that they solve the above optimization problem better by providing a better initial rotation and translation with the help of spectral initialization.", "questions": "Please see the Weaknesses.", "limitations": "Please see the Weaknesses.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed cross-modal transformer is clear and easy to follow. \n2. The spectral initialization of the optimization problem offers good insights for those who are working on the structure prediction problems.", "weaknesses": "1. Some important related works are not discussed and compared. For example, (1) Diffdock-PP (https://arxiv.org/pdf/2304.03889.pdf) use torsional diffusions to solve rigid protein docking, and the source code is released. (2) McPartlon & Xu proposed DockGPT for\nflexible and site-specific protein docking and design. Although they target the flexible docking, they also use evoformer to predict the inter-chain distance map. (3) Very recently, Chu, Lee-Shin, et al. propose geodock (https://www.biorxiv.org/content/10.1101/2023.06.29.547134v1),  a multi-track iterative transformer network for fast and flexible\n88 protein-protein docking. (4) Finally, Luo et al. proposed xTrimoDock (https://www.biorxiv.org/content/10.1101/2023.02.06.527251v1.full.pdf), which is quite related to this work, as they also predict the inter-chain distance map and compute a global rotation and translation that best-fit the predicted distance map.\nAs a submission of the NeurIPS conference, I highly recommend the authors to discuss the connection of this work with the works mentioned above.\n\n2. Figure 1. of the current manuscript is borrowed from xTrimoDock (https://www.biorxiv.org/content/10.1101/2023.02.06.527251v1.full.pdf), this could potentially be problematic.\n\n3. The experimental results are not that clear and convincing. (1) First, Diffdock-PP is not included as a baseline. (2) Second, previous works use ligand-RMSD, complex-RMSD, or interface-RMSD to evaluate the performance of the model. How RMSD is calculated in this work is not revealed.\n\n4. As shown in Eq. 12, 9, 11, the definition of the L^{out} loss does not involve $R^{\\star}$ and $t^{\\star}$. Therefore, the first term in Eq. 13 is equal to zero, and it seems that the outer loss can be directly optimized by minimizing the distance prediction loss and MSA loss. I'm not sure how the story of bilevel-optimization holds in this situation. Can the authors further clarify this point?\n\n5. The authors claim that they use bilevel optimization to solve the problem. However, the effectiveness of the bilevel optimization is not justified.  The authors are encouraged to report the results of the variant that does not rely on bi-level optimization, i.e., directly training a distance-map predictor, and recovery the complex structures from predicted distance map.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- We sincerely thank the Reviewer for your careful reading. We would like to address the concerns by providing responses as well as additional experimental results.\n\n1. > Some important related works are not discussed and compared. For example, Geodock, DockGPT, xTrimoDock and Diffdock-PP.\n\n**Answer:** Thanks for the suggestion. (1) GeoDock [1] and DockGPT [2] target flexible docking. Firstly, flexible docking contradicts the rigid-body assumption [3,4], rendering it incomparable with rigid docking models. Secondly, GeoDock is released after the submission deadline, explaining its absence in the references. (2) Regarding xTrimoDock, NeurIPS permits sharing of early versions on platforms like Biorxiv. (3) Before the submission deadline, issues (https://github.com/ketatam/DiffDock-PP/issues/10) with the released code of DiffDock-PP [5] exist. The authors have resolved them now, so we evaluate the performance of DiffDock-PP shown in the tables below.\n\n**Table** Quantitative comparisons between DiffDock-PP and the proposed BiDock on DB5.5 test set. (bold: best)\n||RMSD $\\downarrow$|TM-score $\\uparrow$|DockQ $\\uparrow$|\n| ------------------- | ----- | -------- | ------- |\n|DiffDock-PP|17.364 $\\pm$ 7.262|0.670 $\\pm$ 0.118 | 0.031 $\\pm$ 0.047|\n|BiDock|**7.280** $\\pm$ 8.117|**0.847** $\\pm$ 0.158|**0.564** $\\pm$ 0.369|\n\n**Table** Quantitative comparisons between DiffDock-PP and the proposed BiDock on VH-VL test set. (bold: best)\n||RMSD $\\downarrow$|TM-score $\\uparrow$|DockQ $\\uparrow$|\n| ------------------- | ----- | -------- | ------- |\n|DiffDock-PP|9.680 $\\pm$ 5.266|0.653 $\\pm$ 0.123|0.130 $\\pm$ 0.115|\n|BiDock|**1.242** $\\pm$ 0.602|**0.966** $\\pm$ 0.021|**0.773** $\\pm$ 0.187|\n\n**Table** Quantitative comparisons between DiffDock-PP and the proposed BiDock on AB-AG test set. (bold: best)\n||RMSD $\\downarrow$|TM-score $\\uparrow$|DockQ $\\uparrow$|maxDockQ $\\uparrow$|\n| ------------------- | ----- | -------- | ------- | ----- |\n|DiffDock-PP|20.562 $\\pm$ 4.237|0.578 $\\pm$ 0.072|0.197 $\\pm$ 0.036| 0.021 $\\pm$ 0.020|\n|BiDock|**9.707** $\\pm$ 8.759|**0.773** $\\pm$ 0.187|**0.342** $\\pm$ 0.351|**0.414** $\\pm$ 0.386|\n\nFrom the results, BiDock generally outperforms DiffDock-PP. One possible reason is that DiffDock-PP directly utilizes MSA embeddings obtained from a pre-trained protein language model. Therefore, the learning of MSA embeddings lacks guidance from rigid structures, and PLMs trained on single chains cannot handle the pairwise contact prediction essential for docking. In contrast, BiDock employs raw MSA features and effectively leverages the rich evolutionary information through a cross-modal transformer, seamlessly integrating it with the structure-modal information.\n\nIn the revised version, we will include these methods in the \"Related Work\" section, highlighting their differences and reporting additional results in the experiments.\n\n2. > The experimental results are not that clear. Previous works use ligand-RMSD, complex-RMSD, or interface-RMSD to evaluate the performance of the model. How RMSD is calculated in this work is not revealed.\n\n**Answer:** Apologies for the confusion. We used complex-RMSD in our experiments. We will clarify this point in the revised version.\n\n3. > As shown in Eq. (12), (9), (11), the definition of the $L^{out}$ loss does not involve $R^*$ and $t^*$. Therefore, the first term in Eq. (13) is equal to zero, and it seems that the outer loss can be directly optimized by minimizing the distance prediction loss and MSA loss.\n\n**Answer:** Apologies for the confusion. The outer and inner loops share the common goal of learning the optimal roto-translation transformation for rigid docking. Therefore, the inner loss $L^{in}$ in Eq. (1) is also the objective function of the outer loop. To clarify this, we will explicitly include the inner loss $L^{in}$ in Eq. (12) as:\n\n$L^{out}=\\lambda_1 L_{dist}+\\lambda_2 L_{msa}+\\lambda_3 L^{in}$\n\nThanks for your careful review, and we will fix this error in the revised version.\n\n4. > The effectiveness of the bi-level optimization is not justified.\n\n**Answer:** Thanks for the suggestion. We conduct an ablation study using two-stage optimization: training outer and inner loops separately. Specifically, we use cross-entropy and masked MSA losses (Eq. (9) and Eq. (11)) to train cross-modal transformer. With the resulting distance map, we compute roto-translation transformation using inner loss and spectral initialization. The results on antibody-antigen docking are presented in the table below, with \"w/o Bi\" denoting the variant without bi-level optimization.\n\n**Table**\tAblation studies on bi-level optimization. (bold: best)\n||RMSD $\\downarrow$|TM-score $\\uparrow$|DockQ $\\uparrow$|maxDockQ $\\uparrow$|\n| ------------------- | ----- | -------- | ------- | ----- |\n| BiDock w/o Bi |10.090 $\\pm$ 7.817|0.733 $\\pm$ 0.173|0.220 $\\pm$ 0.232|0.307 $\\pm$ 0.365|\n|BiDock|**9.707** $\\pm$ 8.759|**0.773** $\\pm$ 0.187|**0.342** $\\pm$ 0.351|**0.414** $\\pm$ 0.386|\n\nResults support our contributions of using bi-level optimization for end-to-end optimization, where the parameter learning of the cross-modal transformer is tailored for rigid docking.\n\nIn the revised version, we will incorporate these ablation studies. Your valuable feedback is highly appreciated.\n\n[1] Chu L S, Ruffolo J A, Harmalkar A, et al. Flexible Protein-Protein Docking with a Multi-Track Iterative Transformer[J]. bioRxiv, 2023: 2023.06. 29.547134.\n\n[2] McPartlon M, Xu J. Deep Learning for Flexible and Site-Specific Protein Docking and Design[J]. bioRxiv, 2023: 2023.04. 01.535079.\n\n[3] Vakser I A. Protein-protein docking: From interaction to interactome[J]. Biophysical journal, 2014.\n\n[4] Desta I T, Porter K A, Xia B, et al. Performance and its limits in rigid body protein-protein docking[J]. Structure, 2020.\n\n[5] Ketata M A, Laue C, Mammadov R, et al. DiffDock-PP: Rigid Protein-Protein Docking with Diffusion Models[C]//ICLR Machine Learning for Drug Discovery workshop. 2023."}, {"review_id": "waGh9mxKrE", "reviewer": "Reviewer_HJaR", "summary": "The paper proposes a new framework for rigid protein docking called BiDock. BiDock is based on an Evoformer-based model that predicts a distance matrix and a gradient-based optimization of the optimal roto-translation based on the predicted distance matrix. In order to be able to pass gradients through the optimization steps at training time the authors derive a bi-level optimization procedure and a spectral initialization.\n\nThe model is trained specifically for rigid docking of antibody chains and evaluated on a range of complexes divided in: general proteins, antibody heavy and light chains and antibody-antigens. Across all three categories, the method shows improved performances.", "questions": "See weaknesses section and the following:\n\n7. Especially with the flexible regions in antibodies it appears important to consider a more flexible form of binding, could the proposed framework be extended in that direction?", "limitations": "See weaknesses section", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper proposes an interesting new approach to deep-learning-based rigid docking using gradient based pose optimization and propagation through the optimization. Moreover, the spectral initialization provides an interesting and meaningful improvement to the performance. ", "weaknesses": "The paper has good methodological and empirical contributions, however, to make it a strong accept I believe a few components in the manuscript should be improved and/or clarified.\n\nIntroduction and novelty claims:\n\n1. Personally, I do not understand the strong emphasis that the authors put on “being the first to effectively leverage sequence and structure modal information in rigid protein docking”. This is I believe the least significant part of the paper as: (1) AlphaFold-Multimer already uses sequence and structural template based informations and the architecture proposed is very similar to that of the EvoFormer blocks, (2) adding sequence based information to EquiDock would be a very straightforward addition, (3) other existing deep-learning based docking methods not referenced by the authors such as [1] already uses sequence and structure informations for the predictions. I believe that the claims of novelty in these regards should be adjusted taking into consideration (1-3). \n\nUnclear parts of the method:\n\n2. The loss used to train the model is not clear to me. Equation (12) says that L^out(R, t, \\phi)  = \\lambda_1 L_dist + \\lambda_2 L_msa, however neither L_dist nor L_msa seem to depend on R nor t. Is this correct? If so I don’t understand the point of the bi-level optimization, as the gradient of L^out with respect to \\phi would not depend on R* or t*.\n3. line 167 “\\hat{D} can be obtained by using the center of each bin”, what is meant specifically by center of each bin? Center of the bin with highest likelihood, or median of the predicted binned distribution, or something else? \n\nThere are several missing details (both in paper and in appendix) about the evaluation set-up that should be clarified:\n\n4. How was AlphaFold-Multimer run (and with what set-up in terms of MSA etc)? The appendix links to the OpenFold repo but this does not yet provide Multimer’s replicas.\n5. For the other methods, what structures were used as input? I know DB5.5 provides also apo (unbound) structures that I assume the authors used for the input, however for the two benchmarks they propose how are the input structures generated?\n6. I assume the different methods were run on different hardware (GPUs vs CPUs), this should be reported in Table 3 or standardized. \n\n[1] Ketata, Mohamed Amine, et al. \"DiffDock-PP: Rigid Protein-Protein Docking with Diffusion Models.\" *arXiv preprint* (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- We sincerely thank the Reviewer for all the comments, and it is a great honor for us to inspire your interest. We have addressed all your questions below and hope they have clarified all confusion you had about our work.\n\n1. > I do not understand the strong emphasis that the authors put on “being the first to effectively leverage sequence and structure modal information in rigid protein docking”. There are some related works, such as AlphaFold-Multimer, EquiDock, and DiffDock-PP.\n\n**Answer:** Thanks for your insightful question. (1) AlphaFold-Multimer targets flexible docking, which makes it different from our scope. The main reason is that AlphaFold-Multimer lacks a specific design that allows the direct input of 3D rigid bodies to the structure module for rigid docking. (2) The equivariant GNNs in EquiDock require the feature of each amino acid to be a vector. However, MSA features are tensors with dimensions $F^{msa}\\in\\mathbb{R}^{N_{cls}\\times N_{res}\\times 49}$. Thus there is no straightforward extension. (3) Regarding DiffDock-PP, due to space constraints, please refer to the rebuttal provided for Reviewer XfrH's Q1. We apologize for any inconvenience this may have caused you.\n\nIn the revised version, we will include these methods in the \"Related Work\" section, highlighting their differences and reporting additional results in the experiments.\n\n2. > The loss used to train the model is not clear to me. Equation (12) says that L^out(R, t, \\phi) = \\lambda_1 L_dist + \\lambda_2 L_msa, however neither L_dist nor L_msa seem to depend on R nor t.\n\n**Answer:** Apologies for the confusion. The outer and inner loops share the common goal of learning the optimal roto-translation transformation for rigid docking. Therefore, the inner loss $L^{in}$ in Eq. (1) is also the objective function of the outer loop. To clarify this, we will explicitly include the inner loss $L^{in}$ in Eq. (12) as:\n\n$L^{out}=\\lambda_1 L_{dist}+\\lambda_2 L_{msa}+\\lambda_3 L^{in}$\n\nThanks for your careful review, and we will fix this error in the revised version.\n\n3. > Line 167 “\\hat{D} can be obtained by using the center of each bin”, what is meant specifically by center of each bin?\n\n**Answer:**  Apologies for the confusion. $\\hat{D}$ is obtained by using the mean of each bin. The term \"the center of each bin\" was not mathematically precise, and we will clarify this point in the revised version. Thanks for your careful review and valuable feedback.\n\n4. > How was AlphaFold-Multimer run (and with what set-up in terms of MSA etc)?\n\n**Answer:** Apologies for any confusion. We reproduce AlphaFold-Multimer based on AlphaFold2 released by OpenFold and evaluate the reproduced version by converting the official JAX checkpoint (https://github.com/deepmind/alphafold) into PyTorch. The setup of MSA features is the same as those in BiDock, as detailed in Appendix B. Additionally, we feed the rigid bodies of each chain as templates to AlphaFold-Multimer.\n\nThanks for your careful review. We will include additional information on the experimental setup of AlphaFold-Multimer in the revised version. Your feedback is much appreciated.\n\n5. > For the other methods, what structures were used as input?\n\n**Answer:** Thanks for your comments. (1) It's worth noting that baselines and our proposed BiDock use the same input. (2) For DB5.5, we utilize the unbound structures (ground truth) provided by the benchmark. Detailed information on the antibody-antigen benchmark is explained in Appendix A. Specifically, obtaining the ground truth structures of antibody-antigen complexes poses significant challenges in practical applications. Researchers often rely on existing folding models to predict them. To simulate real-world scenarios, we use a specialized antibody model called xTrimoABFold [2] to predict the conformations of antibodies and AlphaFold2 for antigens.\n\nFurthermore, our dataset has been selected as a benchmark for the antibody-antigen complex structure prediction task on the \"Life Science Leaderboard\" (https://www.biomap.com/sota/), facilitating public access and fair comparisons.\n\n6. > I assume the different methods were run on different hardware (GPUs vs CPUs), this should be reported in Table 3 or standardized.\n\n**Answer:** All experiments for the baselines and our proposed BiDock were conducted on an A100 cluster. The specific environment details are provided in Appendix B as follows:\n- Operating system: Linux version 5.13.0-30-generic\n- CPU information: AMD EPYC 7742 64-Core Processor\n- GPU information: NVIDIA A100-SXM4-80GB\n\nThanks for your suggestion, and we will clarify this point in the revised version to avoid any misunderstandings.\n\n7. > Especially with the flexible regions in antibodies it appears important to consider a more flexible form of binding, could the proposed framework be extended in that direction?\n\n**Answer:** Thanks for your insightful question. Rigid docking has practical applications that motivated our focus on this problem. Specifically, (1) rigid docking allows us to input partially known structures, facilitating the solution of challenging docking problems. (2) In certain scenarios, we only need to determine whether a protein exhibits a specific function, focusing on the docking pocket without requiring detailed interface information. In such cases, flexible docking is redundant. (3) Regarding the extension to flexible docking, a well-optimized distance map (cross-modal transformer) and the overall bi-level optimization serve as a good starting point. We can design an inner loop tailored for flexible docking, considering conformational changes, geometric constraints, atom clashes, etc.\n\n[1] Ketata M A, Laue C, Mammadov R, et al. DiffDock-PP: Rigid Protein-Protein Docking with Diffusion Models[C]//ICLR 2023-Machine Learning for Drug Discovery workshop. 2023.\n\n[2] Wang Y, Gong X, Li S, et al. xtrimoabfold: Improving antibody structure prediction without multiple sequence alignments[J]. 2022."}]

Author rebuttal:

Meta-review: This paper focuses on rigid protein docking and puts forth a new bi-level optimization approach that combines predicting a distance matrix and optimizing the predicted roto-translation accordingly. 

Though the reviewer scores were mixed, after careful analysis it was decided that the paper merits acceptance. 

Below, I summarize the main concerns of the reviewers and a justification of why they did not lead to a rejection of the paper:

* There were some doubts expressed about the merit of the bi-level optimization approach and specifically about how gradient flows between the two levels. It was determined after discussion that the proposed approach is valid. Reviewers expressed diverging opinions about the significance of the idea.
 
* A reviewer complained about the relatively small size of the training and testing data. This is a valid criticism but seems like a characteristic of the problem and not something specific to the work. 

* There was some discussion about the lack of novelty in the proposed network architecture for BiDock. However, it is the opinion of the AC that the architecture doesn't need to be different for an approach to be novel. Novelty may come from the problem formulation or the way the standard components are used - here the bilevel optimization approach. 

* The complexity of the method was deemed high. Though low complexity is desirable, it's not the main priority when considering a problem as hard as docking. After the problem is solved well with a slow approach the community can focus on efficiency. 

Some of the initial negative opinions of the reviewers can be attributed to an imperfect presentation of the main ideas. The authors can improve upon this based on the provided feedback for the camera-ready version.
Decision: Accept (poster)