PPDiff: Diffusing in Hybrid Sequence-Structure Space for Protein-Protein Complex Design
Authors: Zhenqiao Song
Tianxiao Li
Lei Li
Martin Renqiang Min
Venue: ICML 2025 poster (2025)
Keywords: protein complex design
protein-binding protein design
protein complex sequence and structure co-design
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: We propose PPDiff, a diffusion model building upon our developed SSINC for protein-protein complex sequence and structure co-design.

Abstract: Designing protein-binding proteins with high affinity is critical in biomedical research and biotechnology. Despite recent advancements targeting specific proteins, the ability to create high-affinity binders for arbitrary protein targets on demand, without extensive rounds of wet-lab testing, remains a significant challenge. Here, we introduce PPDiff, a diffusion model to jointly design the sequence and structure of binders for arbitrary protein targets in a non-autoregressive manner. PPDiff builds upon our developed Sequence Structure Interleaving Network with Causal attention layers (SSINC), which integrates interleaved self-attention layers to capture global amino acid correlations, $k$-nearest neighbor ($k$NN) equivariant graph convolutional layers to model local interactions in three-dimensional (3D) space, and causal attention layers to simplify the intricate interdependencies within the protein sequence. To assess PPDiff, we curate PPBench, a general protein-protein complex dataset comprising 706,360 complexes from the Protein Data Bank (PDB). The model is pretrained on PPBench and finetuned on two real-world applications: target-protein mini-binder complex design and antigen-antibody complex design. PPDiff consistently surpasses baseline methods, achieving success rates of 50.00\%, 23.16\%, and 16.89\% for the pretraining task and the two downstream applications, respectively.

PPDiff: Diffusing in Hybrid Sequence-Structure Space for Protein-Protein Complex Design
Zhenqiao Song" 1 Tianxiao Li
Lei Li
Martin Renqiang Min
Abstract
spanning therapeutic development (Nelson et al., 2010), di- agnostics (Brennan et al., 2010) and imaging reagents (Stern et al., 2013). Empirical selection methods, such as screening high-complexity random libraries antibodies (Chao et al., 2006) or alternative scaffolds (Hackel et al., 2008), have demonstrated the ability to generate binders for specific targets. While effective, these approaches require consider- able experimental effort, making them both time-consuming and resource-intensive. Traditional computational methods leveraging physicochemical properties offer more efficient alternative, enabling the design of binders by focusing on specific surface patches the protein target (Chevalier et al., 2017: Silva et al., 2019). However, these approaches are often limited by the lack of obvious surface pockets on many target proteins, thereby restricting their applicability to narrow set of binding interactions.
Designing motein-binning proteins with high affinity is critical in biomedical research and biotechnology. Despite recent advancements tar- geting specific proteins, the ability to create high- affinity binders for arbitrary protein targets on de- mand, without extensive rounds of wet-lab testing. remains a significant challenge. Here, we intro- duce PPDiff, diffusion model jointly design the sequence and structure of binders for arbi- trary protein targets in a non-autoregressive man- ner. PPDiff builds upon our developed Sequence Structure Interleaving Network with Causal atten- tion layers (SSINC), which integrates interleaved self-attention layers to capture global amino acid correlations, k-nearest neighbor (kNN) equivari- ant graph layers to model local interactions in three-dimensional (3D) space, and causal atten- tion layers to simplify the intricate interdepen- dencies within the protein sequence. To assess PPDiff, we curate PPBench, a general protein- protein complex dataset comprising 706,360 com- plexes from the Protein Data Bank (PDB). The model is pretrained on PPBench and finetuned on two real-world applications: target-protein mini- binder complex design and antigen-antibody com- plex design. PPDiff consistently surpasses base- line methods, achieving success rates of 50.00%. 23. 16%, and 16.89% for the pretraining task and the two downstream applications, respectively.
In parallel with advances in traditional computational binder design, deep learning methods have achieved remarkable accuracy predicting and designing protein-protein inter- actions (Evans et al., 2021; Bennett et al., 2023; Zambaldi et al., 2024). These approaches have made it possible to design binders for specific targets without relying high- throughput screening (Goudy et al., 2023). In certain cases, such as peptides (Vázquez Torres et al., 2024) and disor- dered targets (Wu et al., 2024), high binding affinity has been achieved without the need for extensive experimental optimization. Despite these successes, the overall success rate remains low due two primary failure modes: the designed sequence may not fold into the intended structure, and the designed structure may not bind to the target protein. Moreover, most existing approaches are highly specialized for particular interaction types, leaving many target proteins intractable.
1. Introduction
To address these challenges, we present PPDiff, diffu- sion model building upon our developed Sequence Structure Interleaving Network with Causal attention layers (SSINC), to co-design the sequence and structure of binders for ar- bitrary protein targets in non-autoregressive manner. In this framework, proteins are represented as residue point sets 3D space, with each residue linked to 3D Cartesian coordinate. The model employs diffusion process for both continuous residue coordinates and discrete residue types,
Designing proteins with high affinity and specificity to tar- get proteins important in biomedicine, with applications
"Work done during the internship at NEC Laboratories America. 'Language Technologies Institute, Carnegie Mellon University NNEC Laboratories America. Correspondence to: Zhenqiao Song <chenqiaosong @ emu.edu>..
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

PPDiff: Diffusing in Hybrid Sequence-Structure Space for Protcin-Proteee Complex Design
progressively adding noise train a joint generative model. SSINC integrates interleaved self-attention layers, capturing global amino acid correlations, with ANN equivariant graph convolutional layers to model local 3D interactions. Causal attention layers are also introduced to simplify the intricate dependencies within the protein sequence. By simultane- ously modeling sequence, structure, and their complex in- terdependencies, PPDiff mitigates primary failure modes in binder design while improving the diversity and novelty introduced by the diffusion process to enhance success rates.
accounting for their physicochemical constraints. Cao et al. (2022) propose general Rosetta-bases approach to design binder proteins using only the structure of the target. While these approaches provide systematic route for binder de- sign, the reliance on a limited number of hotspot residues restricts them to a narrow range of interaction types. Recent advances in deep learning have significantly improved the prediction and design of protein- protein interactions (Evans et al., 2021; Humphreys et al., 2021; Bryant et al., 2022; Pacesa et al., 2024). These advancements have enabled the design of binders for certain targets without the need for high-throughput screening (Watson et al., 2023; Gainza et al., 2023; Goudy et al., 2023). Bennett et al. (2023) explore the augmentation of energy-based protein binder design with deep learning, while Zambaldi et al. (2024) in- troduce AlphaProteo for designing binders targeting eight different proteins. Despite these developments, the overall success rate for binder design remains low, with many pro- tein targets still intractable (Yang et al., 2024; Berger et al., 2024).
Our contributions are listed as follows:
We propose PPDiff, a diffusion model building upon the SSINC network to co-design the sequence and structure of binder proteins for arbitrary protein targets. We create PPBench, a general protein-protein complex dataset comprising 706,360 complexes curated from PDB.
Our model pretrained on PPBench demonstrates a suc- cess rate of 50.00% on top-1 candidate, evaluated using a combination of metries: ipTM, pTM, PAE, and pLDDTT assessed by AlphaFoldd (Abramson et al., 2024). Notably, our model also achieves superior nov- city and diversity scores, highlighting its ability to design novel and diverse protein- binding proteins with high success rates. The t candidates are uploaded in the supplementary material.
Diffusion Models. Diffusion models (Sohl-Dickstein et al., 2015) have emerged as a prominent class of latent gener- ative models. Ho et al. (2020) introduce denoising diffu- sion probabilistic models (DDPM), establishing a connec- tion between diffusion models and denoising score- based models (Song & Ermon, 2019). Diffusion models have demonstrated remarkable success in generating high-quality images (Ho et al., 2020; Nichol & Dhariwal, 2021) and texts (Hoogeboom et al., 2021: Austin et al., 2021; Li et al., 2022). More recently, they have been applied to protein design. For instance, RFdiffusion (Watson et al., 2023) and FrameDiff (Yim et al., 2023) are continuous diffusion mod- els capable of generating novel protein structures. EvoD- iff (Alamdari et al., 2023) and DPLM (Wang et al., 2024) are discrete diffusiio models designed for protein sequence generation. However, the application of diffusion models for the joint design of protein sequences and structures binding to specific protein targets still remains under-explored.
We finetune the pretrained PPDiff on two important real-world applications: target-protein mini-binder complex design and antigen-antibody complex design. The finetuned models achieve success rates of 23.16% and 16.89% respectively, across these tasks, demon- strating its effectiveness in addressing significant chal- lenges in biomedicine.
2. Related Work
Methods for Protein-Protein Complex Design. Design- ing proteins with high affinity and specificity for protein targets has been extensively explored using a wide range of approaches. The most commonly employed methods involve immunizing an animal with a target to induce an- tibody (Gray et al., 2020) or screening high-complexity random libraries of protein scaffolds (Chao et al., 2006; Hackel et al., 2008). While these approaches have proven effective, they are eesource-intensiog and require significant experimental effort. Traditional computational methods for binder design have sought to model physicochemical proper- ties to accelerate the design process. Chevalier et al. (2017) introduce a massively parallel approach for designing, man- ufacturing, and screening mini-protein binders, with bio- physical property characterization. Fleishman et al. (2011) develop general computational framework for designing proteins that bind to specific surface patches of targets by
3. Proposed Method: PPDiff
3.1. Problem Definition
A protein consists of a chain of amino acids connected by peptide bonds, which folds into a proper 3D struc- ture. Let A be the set of 20 common amino acids. Given target protein T with its sequence of M amino acids ST = {sT ST. ...., T E AM and the 3D coordinates of the amino acids' alpha carbons xT = [=] _ ,..., "M1"" E RM x3 the goal is to generate a protein B that binds to the target. That is to predict the amino acid sequence R = ( 58 ..., N E A and their alpha carbon (C.)) coordinates " {a 2 ..,,..,.. " E RN3, M and N are the sequence lengths of the target and binder proteins.
2

---

PPDiff: Diffusing in Hybrid Sequence-Struclure Space for Protcin-Protcie Complex Design
(a) PPDIff
(b)
target-protein mini-binder
SSINC Network
PDB
complex design
Causal Attention Layer
x Leausal
Protein-Protein Complex
Finetuning
antigen-antibddy complex design
KNN Equivariant Graph Layer
XL1
Self-Attention Layer
x Laattn
Pretraining
Po(BL_1|BB
Generative Process
BT
B₁
B--1
B
Bb
PPDiff
Noise
( (T)
Diffusion Process
Figure 1. (a) Overall architecture of proposed PPDiff. (b) We first pretrain PPDiff PPBench, general protein-proter complex dataset curated from PDB. Then we can finetune the pretrained model on important real-world protein-proted complex design applications, such as target-protein mini-binder complex design and antigen-antibont complex design.
We formulate the protein-protein complex design problem as learning the probabilistic generative model pe (B(T) with parameters 0. To this end, we propose novel model PPDiff.
To reduce the failure modes of previous methods and en- hance the consistency between the designed binder sequence and structure, PPDiff is designed to jointly generate both the discrete residue sequence and the continuous backbone structure. This requires implementing diffusion and genera- tive processes that operate on both discrete residue types and continuous spatial coordinates. In the following subsections, we will provide a detailed introduction of the diffusion and generative processes in PPDiff, followed by a discussion of the learning objective and the informative guidance strategy used to generate high-quality candidates during inference.
3.2. Overall Model Architecture
PPDiff aims to simultancously generate the sequence and backbone structure of binder protein for a given protein target. PPDiff is denoising diffusion probabilistic model (DDPM) (Ho et al., 2020) with a joint sequence-structure generation network (Figure I(a)). The model comprises a forward diffusion process to add noises and a reverse generative process, both defined as Markov chains. The diffusion process gradually perturbs the ground-truth data Bo (T) into stationary distribution Br quoise with T increasingly noisy steps:
3.3. PPDiff Diffusion Process
Following Guan et al. (2023), we model the joint diffusion distribution of protein residue sequences 8 and backbone structures x. independently. Such formulation allows for efficient sampling of noisy data for both modalities, i.e. discrete residue types and continuous coordinates. For the discrete protein sequence s we adopt a categorical distribution Cat(s";:) p) as proposed by Hoogeboom et al. (2021), where p represents vector on the (IAA 1 19)-dimensional probability simplex. For the continuous backbone structure, we follow Ho et al. (2020) and use a Gaussian distribution N to model spatial coordinates. At each diffusion step I,, uniform noise is added to the residue types across all categories, and Gaussian noise is applied to the structural coordinates. This process follows a Markov chain with a predefined schedule $1, B2. ... Br as follows:
q(BuTT Bo, T) = 1B..q(B.18..1177
(1)
where B1, .... Br is a sequence of latent variables, each contains both amino acid sequence sf and alpha carbon coordinates z. Notice that it is a big challenge to model the latent variables because B, contains both discrete and continuous components. The generative process parameter- izes the transition kernel po(B,... (B.. BB, T) to recover the data distribution from the noise distribution:
(B T) = =p(Br p(BBr (B
(2)
To fit the model po (T) to the data distribution (B,
the denoising model typically optimized by the variational bound of the log- ligliiill (Ho al., 2020):
(BB,11,-1,, T) = q(5; |. [ T) {(x [| 1= T) q( B f T) = Cat(s 8 : (1 ) 9. (4)
Bo (T) (BB.r (...7|Bo.7. T)
Ex(8,177 po (Bol [T)] 2 (1og
q(x=" [<= = NN /I 1 3.1)
ET Po po (B (B, 7) + const
In practice, the schedules for different data modalities can be different.
[l Be, Bo, T) (B
Denoting a = 1 - 3g and ã. = II 1 we can calculate the noisy data distributions q(>,51 1155 'T) and g( T)
(3)
3

---

PPDiff: Diffusing in Hybrid Sequence-Structure Space for Protcin-Protee Complex Design
in closed-form:
structure at time step I, i.e. ag. To make the learning easier, Tccausal causal attention layers are added top of the SSINC network to capture the sequential dependencies:
a(>f1 Su: 7) Cat(s) a.sy + (1- k) q(>]
(5)
B xo 7) -Naals va.z (1 x
Based on Eq. 4 and Eq. 5, we can calculate the posterior dis-
out h
Causal-Attention (ht
H.
1,1:))
(9)
tributions q [ .6. T) and qaaf-1l<< ag. T) using
8 = (H
Bayes rule in closed-form:
where the i-th residue can only attend to its previous residues H/.1. We can then approximate the posterior distributions
8 18 8( ( st S0:
T)
Cat(s 0 (o .6))
IA|
(e | xx," T) and po |2, sf: T) using the pre-
0
@ [s 80
5
=
Σ
0%
dicted & and ag.
k=1
0- = jass;
B + (1 - ax) K] @ ( (1-aa-1) [)
3.5. Training Objective
q( - lm
B
1
T) = N ( 14, , (00,,.1))
We learn the model by maximizing the variational lower bound of the log-likelihood defined in Eq. 3, where La is calculated as:
18 x vor(1 01-1)
i4, (x
1 E.
ae
a2
3. 1- - 3:
1 ae.
C. = -LLLgg( o6 86 T) (Po 188 100 a. T)) B
(6)
8 Ti] (L(g - x T) Ipo (xr 100
3.4. The SSINC Network for PPDifT Generative Process
Σ 0feal (of oi)a log O (5t ,80 500 )+
gez (8l A6)
The generative process recovers the data distribution from the noise distribution. We parameterize the reverse genera- tive process using our developed SSINC network by 0:
ji (ar ax p a T)1 + const
?
ps PB 1 [8,, T) pa pa 81 Lt 8 T) po (= | (...TT
pa (s |s." L T) Cat(s 1: 0 ( (.)))
(10)
ps (w
| s T) (V ng (8 of L, (, II
where Oa 0 (5, ..)) is defined in Eq. 6, with ao calculated in Eq. 9. LE can be further symplified as:
The SSINC network is composed of L, interleaved blocks, each containing Lam self-attention layers (Vaswani, 2017) to capture global correlations among amino acids and a KNN equivariant graph convolutional layer (Satorras et al., 2021) to model local interactions among neighboring residues in 3D space, as illustrated in Figure (a). the 1-th interleaved block, the representations are calculated as follows:
at- 188 CT = - - 1² + const) X
20 a, )2
(11)
Following previous work (Guan et al., 2023), we set A 1. Furthermore, log Pa ( (B, ; T)
can be calculated as
log po (Bo)BB T) log pa (=0 s1 af. T)
H = Self-Attention T)
+ log (2 B 1 wi, $1. T)
1+0.5
(hh 1-0.5, h,,
1+0.5
|| 21,=ll2):77 T)
(12)
m,
Σ 8 log $b. - 1|=
wii
140.5
Softmax(m)tt2"
(40.), m
241 140.5
wiik
240.5
dt
141 E
miske weight = ((f(()))))
(8)
3.6. Informative Guidance for Inference
Σ KENG) weight- m 4+1
h
As suggested by previous studies (Graikos et al., 2022; Wu et al., 2022), an informative prior distribution can signifi- cantly enhance model performance. To leverage this insight, we introduce an informative guidance to select better start- ing points rather than random noise. For backbone structure guidance, we define the following energy function:
1+1 = Lt, + EnENCO (ahh wh.a) f'r (tr (+1)
where H:- = [hi,1- 2!.2.... ..., n',,"' denotes the residue rep- resentation matrix at the 1-th block and time step i, while Hº = |sf 5 .... 17 is the embedding matrix of binder protein sequence. Functions f- denote ford-forward layers and a denotes sigmoid function. (1) is the set of k-nearest neighbors of the i-th residue. Finally, a. is the predicted
Exnn (B)
(knn-dist(z.) (2.) - -kkmn)
(13)

---

PPDifT: Diffusing in Hybrid Sequence-Struclure Space for Protcin-Protcin Complex Design
where knn-dist(a;,) 2/+ JEN( |,, - -21l denotes the average distance from i-th residue its k nearest neigh- bors and Jkon is the empirical mean of knn-dist(a:) across all residues in the training dataset. In our setting, k is set to 4. During inference, we sample 10 noisy structures and choose the one with the lowest energy. For sequence guid- ance, we randomly sample secondary structure fragments from the training dataset, identified by using DSSP (Kab- sch& & Sander, 1983). This strategy ensures the initialized structures and sequences maintain geometric similarities to the training data, providing the model with more in- formed starting point and enhancing the quality of candidate generation.
the pretrained 650M ESM-2 model (Lin et al., 2022). PPDiff has a total of 692M parameters. The diffusion process is configured with 1,000 steps, employing cosine schedule for sequence diffusion and a sigmoid schedule for structure diffusion. Additional model training details are provided in Appendix C.2. Baseline Models. Given the absence of large-scale mod- els for protein-protein complex sequence and structure co-design, we evaluate the effectiveness of PPDiff by comparing it with several representative baselines: (1) SEnctProteinMPNN first employs a continuous diffusion model based on structured encoder (Ingraham et al., 2019) enhanced with spatial features to design complex back- bone structures, followed by ProteinMPNN (Dauparas et al., 2022) predict binder protein sequences based on the de- signed backbone structure. (2) InterleavingDiff is variant of PPDiff without the casual attention layers, of which the backbone model is the interleaving network (Song et al., 2024b). (3) SSINC Network has the same architecture as PPDiff, but it is trained without diffusion process. To en- sure fair comparison, all models are trained on the curated PPBench using their official implementations.
4. Experiments
In this section, we first describe our PPBench construction process in Section 4.1 and experimental setup in Section 4.2. Then we conduct extensive experiments on General Protein-Protein Complex Design task (Section 4.3) and two real-world applications, Target-Protein Mini-Binder Complex Design (Section 4.4) and Antigen-Antibody Complex Design (Section 4.5). The specific experimen- tal settings are introduced in each task section.
4.3. General Protein- Complex Design
4.1. PPBench Construction
This task aims to develop general protein-protein complex design foundational model that can be readily finetuned for diverse real-works
Our goal is to develop a general protein-protein complex design model that serves foundational model, adaptable for finetuning in diverse real-world pooeiin-protein interac- tion applications. To achieve this, we curate a large-scale scale protein-protein complex dataset by identifying chain-pair interfaces, following the data processing pipeline in Al- phaFold3 (AF3). Details of the data curation process are provided in Appendix C.I. This process results in 367,016 chain-pair interfaces. Treating each chain in a chain-pair interface as potential target protein, gather a total of 734,032 complexes. Protein sequences are then clustered based on 50% sequence identity, producing 22,847 clusters. To prepare the dataset for training and evaluation, we des- ignate 10 clusters each for validation and testing, with the remaining clusters reserved for training. To ensure efficient processing, training data are further filtered to exclude se- quences longer than 1,024 residues, while validation and testing data are restricted to sequences no longer than 512 residues, finally resulting in total of 706,360 complexes. Detailed data statistics are provided in the Appendix A.
protein-protein interaction applications (Figure 1 (b)). To this end, we pretrain proposed PPDiff on the curated PPBench.
Evaluation Metrics. We evaluate the designed protein- protein complexes based on structure stability, reliability, and functional interaction using AF3 metrics. Specifically, PLDDT assesses structural stability, pTM and PAE mea- sure structural reliability, and ip evaluates interface in- teractions, reflecting functionalityy Detailed explanations of these metrics are provided in Appendix C.3. As per prior studies (Binder et al., 2022; Bennett et al., 2023; Abram- son et al., 2024), a successful complex satisfies ipTM20.8, PTM>0.8, PAESIO and PLDDT280. For each target pro- tein in the test set Dees. we generate 100 binder candidates and rank them by their complex DLDDTT scores, selecting the top-k candidates C. The success rate (SR) for the top-k candidates is:
SR
Σ Σ U(J(T.B.)) TED 1=1
(14)
Dresel k
4.2. Experimental Setup
Implementation Details. Our architecture incorporates three interleaving blocks and one causal attention layer. Each interleaving block comprises 11 self-attention layers and one baseb equivariant graph convolutional layer. The parameters for self-attention layers are initialized using
where /(T, .B)) ensures if the thresholds for ipTM, PTM, PAE, and PLDDT are met, and II(true) = 1, 11(false) = O. Here, B, is the i-th candidate. We also evaluate diversity and novelty scores. Diversity quantifies sequence variability among the top-k candidates, while novelty measures the
5

---

PPDifT: Diffusing in Hybrid Sequence-Structtue Space for Photein-Protcin Complex Design
Methods
ipTM (1)
pTM (1)
PAE (4)
pL.DDT(1)
Success Rate (1)
Novelty (1)
Diversity (1)
Ground Truth
0.691
0.782
7.901
86,987
69.64%
SEnč +ProteinMPNN"
0.629
0.713
I03374
779.819
33.08%
39.76%
InterleavingDiff
0.674
0.757
9,934
82.411
45.61%
89,45%
top-1
SSINC Network
0.660
0.751
9.713
82.998
47.36%
37.46%
PPDiff
0.700
0.779
9.153
83.765
50.00%
89.46% %
SEnc +ProteinMPNN
0.617
0.682
11.789
78.021
29.82%
61.78%
58.72%
InterleavingDiff
0,648
0.732
11.010
79.944
37.85%
90.66 %
91.82% %
1op-5
SSINC Network
0.647
0.745
9.903
82.290
43.85%
37.49%
15.12%
PPDiff
0.665
0.747
10.231
81.659
45.71%
88.93%
90.76%
SEnc +ProteinMPNN
0.582
0.671
12.347
76.892
21.05%
62.98%
58.93%
InterleavingDiff
0.629
0.716
11.745
78,328
24.28%
90.64% %
91.89
top-10
SSINC Network
0.620
0.721
10.075
81.852
36.84%
37.45%
15.03%
PPDiff
0.633
0.729
10.895
80.322
37.68% %
89.10%
91.09%
Table Model performance on general protein- complex design task. (t): higher the better. (U): the lower the better. Calculating the diversity score for the top-1 candidate is unnecessary, as it consists of only a single candidate, PPDiff consistently achieves the highest success rates for top-1, top-5, and top-10 candidates out 100 candidates for each target protein.
deviation of candidates from the ground truth sequence:
set to create a zero-shot evaluation scenario. Detailed data statisties are provided in Appendix A.2.
diversity
Σ
AAR(a
* (Ie SJEC,t
Evaluation Metries. Following prior works (Bennett et al., 2023; Watson al., 2023; Song et al., 2024a), we evaluate the binding affinities between the designed binders and tar- get proteins using the AlphaFold2 (AF2) PAE inferracctn developed by Bennett et al. (2023). Their work demonstrates that AF2 interractnn effectively differentiates exper- imentally validated binders from non-binders, achieving success rates ranging from 1.5% and 7% for target pro- teins FGFR2, II.7Ra, TrkA, InsulinR, PDGER, and VirB8. Based on the calculated inEeittean score, deter- mine the success rate by generating five mini-binder can- didates for each target protein. The success rate is de- fined as the proportion of designed binders that achieve lower pAE. interaction score with the target protein than the ground truth positive binder. More evaluation details are provided in Appendix C.4.
(15)
novelly- - 11 AAR(#
where sf is the sequence of B;, 8 is the ground truth binder sequence, and AAR denotes the amino acid recovery rate. The final diversity and novelty scores are averaged across the whole test set.
The results for top- top-5, and top-10 can- didates are reported in Table 1. These results demonstrate that PPDiff excels in designing high-quality, diverse, and novel protein-binding proteins across a wide range of protein targets. For the top-1 candidate, PPDiff achieves the best performance across all metrics. While PPDiff per- forms slightly worse than the SSINC Network on PAE and PLDDT for top-5 and top-10 candidates, it still achieves the highest success rate and significantly performs better in diversity and novelty. Although the diversity and nov- elty scores for PPDiff's top-5 and top-10 candidates are marginally lower than those of InterleavingDiff, PPDiff produces designs of much higher quality, validating the effectiveness of the causal attention layer.
Main Results.
Main Results. We finetune PPDiff on the target-protein mini-binder complex design task and the success rates are summarized in Table 2. Our PPDiff achieves a superior success rate compared to all baselines, demonstrating its effectiveness in designing high-affinity binders. However, we observe that all models fail to design successful binders for EGFR, likely due to the lack of deep pockets or distinct features in its binding sites (Callaway, 2024).
4.4. Target-Protein
Complex Design
In this section, we aim to use PPDiff to design proteins that bind specific protein targets with high affinity.
4.5. Antigen-Antibode Complex Design
In this section, apply PPDiff to design high-quality anti- bodies binding to given antigens.
Datasets. We collect experimentally confirmed positive target-protein mini-binder complexes against ten targets with diverse structural properties from Bennett et al. (2023). For categories containing more than 50 complexes, an 8:1:1 random split is applied for training, validation, and test sets. For smaller categories, all complexes are included in the test
Following Jin et al. (2022), we curate antigen- complexes from the Structural Antibody
Datasets.
6

---

PPDifI: Diffusing in Hybrid Sequence-Structtre Space for Photcin-Protcee Complex Design
Seen Class
Zero-Shot
Models
Average
FGFR2
InsulinR
PDGFR
TGFb
VirB8
H3
I1.7Ra
EGFR
TrkA
Tie2
SEnc+ProteinMPNN
8.07%
6.08%
4.61%
15.56%
11.42%
4.73%
17.14%
0.0
5.00%
0.0
7.12%
IntorleavingDiff
10.00%
6.95%
11.53%
35.56%
8.57%
48.42%
34.28%
0.0
10.00%
10.00%
19.32%
SSINC Network 21.05%
0.0
3.07%
20.00%
2.85%
7.36%
22.85%
0.0
5.00%
0.0
10.96%
PPDiff
7.36%
10.43%
14.61 %
35.56%
11.42%
55.26%
60.00%
0.0
20.00%
30.00%
23.16%
Table 2. The success rate (% (%, 1) for all models on the parget-protein mini-hinder complex design task. "Average" refers to the overall success rate across the entire test set, rather than simple mean across target categories. Our proposed PPDiff demonstrates significant improvement in success rate, outperforming all previous methods by substantial margin.
Methods
ipTM (1) pTM (1) PAE (4) pLDDT(1) Success Rate (1) HI Novelty (1) H2 Novelty (t) H3 Novelty (1)
SEnc + -ProteinMPNN
0.448
0.578
14,418
77.786
6.24%
61.78%
59.53%
70.85%
InterleawingDiff
0.496
0.653
13.583
80.939
12.00%
57.54%
63.59%
75.58%
SSINC Network
0.539
0.654
12.064
82.706
16.74%
30.44%
37.70%
59.86%
PPDiff
0,541
0.668
12.999
82.827
16.89 %
57.79%
66.39%
76.17% %
Table Model performance on antigen-antibody complex design task. (f): the higher the better. (J): the lower the better. Novelty scores for CDR-HI, CDR-H2, H2, and CDR-HE are denoted as HI, H2, and H3 novelty, respectively, PPDiff demonstrates superior performance, achieving the highest average success rate across the designed CDRs.
Database (Dunbar et (2014), SAbDab), resulting in 4,261 valid complexes after removing cases lacking light chain or antigen. The dataset is split into training, validation, and test sets based on the clustering of CDRs. Taking CDR-H3 as example, we use MMseqs2 (Steinegger & Söding, 2017) to cluster CDR-H3 sequences at 40% sequence identity, calcu- lated using the BLOSUM62 substitution matrix (Henikoff Henikoff, 1992). Clusters are then randomly divided into training, validation, and test sets in an 8:1:1 ratio. The same procedure is applied create splits for CDR-HI and CDR- H2, resulting in 1,034, 1,418, and 2,254 clusters, respec- tively. Details of the training, validation and test set sizes for each CDR clustering are provided in the Appendix A.3.
5. Analysis: Diving Deep into PPDiff
5.1. How Does The Causal Attention Layer Function?
To assess the influence of causal attention layers, we train the model with varying layer size (0 to 4) on PPBench. For each setting, we generate one candidate and evaluate the resulting complexes using AF3 with the same seed. The results, visualized in Figure 2 (a), illustrate that introducing causal attention layers significantly enhances design quality, However, increasing the number of layers above one does not provide additional benefits. Thus, using a single causal attention layer achieves the optimal design quality.
Evaluation Metrics, To guarantee a comprehensive eval- uation of the designed antigen-antibody complex, we use the MM,,, PTM, PAE, PLDDT, success rate and novelty metrics introduced in the general protein-protein complex design evaluation (Section 4.3).
5.2. Effect of Diffusion Steps on Model Performance
To explore the impact of diffusion steps on PPDiff's per- formance, we train the model with different diffusion steps ranging from 500 to 2,000. For each trained model, single candidate is generated and evaluated using AF3 under the same seed. The results, presented in Figure 2 (b), demon- strate that PPDiff performs best when configured with 1,000 diffusion steps.
Main Results. We finetune the pretrained PPDiff on each of the antigen-antibodd complex dataset derived from the clustering process and present the average performance in Table 3. Detailed results for each CDR clustering are given in Appendix I.I. PPDiff achieves the highest antigen- antibody binding affinity (ipTM), success rate, and nov- elty scores. It demonstrates PPDiff's ability to design high- affinity and novel antibodies for given antigens, highlighting its potential significance for real-world applications.
5.3. Does Model Scale Help?
PPDiff is scalable. To evaluate the scalability of PPDiff, we compare the performance of our PPDiff (692M) with a version comprising 30 self-attention layers (144). For each trained model, a single candidate is generated and evaluated using AF3 under the same seed. Figure (e) shows the ipTM and pTM scores. The results reveal that the scaled-up PPDiff achieves significantly higher ipTM and pTTM scores, highlighting the scalability and enhanced performance of our approach as the model size increases.
7

---

PPDifT: Diffusing in Hybrid Sequence-Structtre Space for Complex Design
Cluaal Atsamioo Layer Seea In nolo
Diiussonn Stap inBuerce
20
PUDDT of SABS PRDE DJD Pistaaroo
65
Success Roys
Swiss Dott Pretrirr
Iitte gFT
p430T
2TM
0.00
032
F7
100
klayer
2 Zlaget
kybss
592
750
1980
122
1550
155 2018
trp-2
15p-
705 30
tep-1
202-2
5:0
(a) Causal Attention Layer Size
(b) Diffusion Step
(c) pLDDT of Pretraining
(d) Success Rate of Pretraining
0.3
ferformance arnng Different Model Scales
PLOOT Diffeeent "aal Size
Success Bare 190 Diftrent Ttral Candicate Siow
PrO-i-154M
Candidato 3ize-30
60-
Ta Candl dott Sizc-
0.7
632M
Candisate Stee-
90
Candidate 10arm100
lotal Caralidkus dluie Sieem100
85-
80-
0.5
0.5-
70-
0.4
0.4-
10
0.3
prTM
PTM
Isp-1
1op-5
1op-10
60
top
top 5
top
In0-1
Tep-5
top-20
(e) Model Scaling
(1) TM score
(g) PLDDT score
(h) Success Rate (%)
Figure Ablation study on: (a) causal attention layer size, (b) different diffusion steps, (c-d) additional Swiss-Prot data pretraining, (e) different model scales, (f-h) different total candidate sizes.
ITMII BLDOTHOO.E 12, IITM-CI,, ELDOT-B7.14 18,
BIMMIGI ELOOT-68.00,
IITMIOAL PLOOTHE8. UTTAIOE, 2LDOT STM-Q.77 BLOOT IITMO PLOOT (STMid #L.OT UTMMIOE, PLDOTHO2880 STMIIBT, CORMISTO pTMLO.B CORRE.75% PTM-OUS,, CORMOLSON
pTMLOO 8R, Novelyu77%
PTMUO ST, Novelyy50%
pTMAd Novelyus0%s
pTMLLdaS, Novelyu895
pTM...87, Novelys825
pTT6.0.80 Noweyuuer.
Figure 3. Designed protein complexes from: (a-c) general protein-proted complex design, (d-f) target-protein mini-binder complex design, and (g-i) antigen-antibody complex design. Target proteins are shown in blue, and the designed binder proteins in green, with light chains pink for antigen-antihody complexes. PPDiff is able design high-affinity protein- protein-binding proteins diverse target scaffolds.
5.4. Impact of Additional Swiss-Prot Data Pretraining
of top-k candidates (I:=1,5, and 10), particularly in terms of success rate. This demonstrates that PPDiff can produce higher-quality complexes when considering larger pool of candidates, which is potentially advantageous for wet-lab validation in real-world applications.
To assess whether pretraining on additional data improves design quality, we pretrain PPDiff on the Swiss-Pre dataset (Boeckmann et al., 2003) and its AlphaFold-derived structures (Varadi et al., 2022). Since Swiss-Prot proteins are monomers, pseudo complexes are generated by identi- fying residue pairs within 5 in the 3D structure and with sequence length longer than 60, designating the interacting residues as the last and first residues of two chains. The model is first pretrained on this pseudo complex dataset before continuing training on PPBench. Figure (c) and (d) compare the PLDDT and success rates of top-1, top-5, and top-10 candidates from this model with those of the original PPDiff. The results indicate that pretraining on addi- tional Swiss-Pro Prot data provides no significant benefits. Detailed results are included in the Appendix B.2.
5.6. In-Depth Exploration of Target Protein-Mini Binder Complex Design
To better evaluate the quality of our designed binders, we first compute the docking scores between the target pro- teins and the designed binders using HDOCK (Yan et al., 2020). The results, summarized Table 4, show that PPDiff achieves better docking performance to existing baseline methods, exhibiting similar phenomenon using AF2_pp AF2 interaction scores. This validates both the reliability of the AF2 DAE_interaction score as proxy for binder quality and the ability of PPDiff to generate binders with strong binding affinity to the target proteins.
5.5. Influence of Total Candidate Size
To evaluate the impact of total candidate size, we generate 30 binder proteins for each target protein in the test set of PPBench and analyze the top-1, top-5, and top-10 results, as shown in Figure (f-h). Detailed results provided in Appendix B.3. The results indicate that increasing the total candidate size to 100 significantly enhances the quality
We further compare PPDiff against strong baseline, RFD- iffusion combined with ProteinMPNN (RF+MPNN), with results presented in Table 5. Our model achieves higher average success rate across the 10 target proteins. More- over, the binders generated by PPDiff exhibit significantly greater diversity and novelty compared to those produced
8

---

PPDifI: Diffusing in Hybrid Sequence-Structtre Space for Photein-Protcin Complex Design
Seen Class
Zero-Shot
Models
Average
FGFR2
InsulinR
PDGER
TGFb
VirB8
H3
I1.7Ra
EGFR
TrkA
Tie2
SEnc+ProteinMPNN
-197.82
-192.56
-231.46
-203441
-235.67
-198.23
-19..85
-178.23
-224,91
-201.32
-205.64
InterleavingDiff
-230.40
-233.49
-234.60
-231.93
-222.56
-227.03
-229.93
-218.26
-234,12
-230.43
-230.25
SSINC Network
-207.76
-193.18
-226.77
-211.04
-220.34
-206.02
-207.24
-183.53
-217.91
-196.22
-208.48
PPDiff
-256.86
-260.95
-270.55
-251.35
-252.69
-244.23
-261.36
-244.75
-266.06
-265.19
-256.45
Table 4. The docking score (4) for all models on the target-protein mini-hinder complex design task. "Average" refers to the overall docking score across the entire test. set, rather than simple mean across target categories. Our proposed PPDiff demonstrates the hest binding affinity across all targets, outperforming all baseline methods.
Seen Class
Zero-Shor
Models
Average
Nov.
Div.
FGFR2
InsulinR PDGER TGFb
VirB8
H3
IL7Ra
EGFR
TrkA
Tie2
RF+MPNN
28.07% %
8.69%
15.38 % 22.22% 57.14%
7.89%
28.57%
25.00% %
100.00 %
0.0
21.46% 78. 10%
25.71%
PPDiff
7.36%
10.43%
14.61 % 35.56% 11.42% 55.26% 60.00% %
0.0
20.00%
30.00%
23.16%
91.39 % 91.79%
Table The success rate (%, 1) for PPDiff and strong baseline RFDDffusionn-ProeiiMPPM (called RF+MPNN here due to the limited space) the target-protein mini-binder complex design task. "Average" refers to the overall success rate across the entire test set, rather than simple mean across target categories. Our PPDiff achieves a higher average success rate, as well as significantly higher novelty (Nov.) and diversity (Div.) scores.
by the baseline. These findings highlight the effectiveness of PPDiff in designing functional, diverse, and novel protein binders.
research to further validate the model's practical utility.
Impact Statement
5.7. Designing Novel and Diverse Protein Complexes
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Figure 3 showcases nine designed protein- protein com- plexes across different design tasks: (a-c) general protein- protein complex design, (d-f) target-protein mini-binder complex design, and (g-i) antigen-antibody complex design. The target proteins in these examples exhibit diverse struc- tural scaffolds, and all complexes achieve outstanding met- rics, including ipTM scores approaching 0.8 or higher, pTM scores exceeding 0.8, PL.DDT values above 80, and novelty scores surpassing 50%. These results highlight PPDifI's ca- pability to design novel, high-affinity binder proteins for a wide range of protein targets. More designed complexes are provided in Appendix D.
References
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneherger, O., Willmore, L.,, Ballard, A. J., Bambrick, J., et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, pp. 1-3, 2024.
Alamdari, S., Thakkar, N., van den Berg, R., Tenenholtz, N.,
Strome, B., Moses, A., Lu, A. X., Fusi, N., Amini, A. P., and Yang, K. K. Protein generation with evolutionary diffusion: sequence is all you need. BioRxiv, pp. 2023-09, 2023.
6. Conclusion
In this paper, we present PPDiff, diffusion model building upon SSINC network to co-design binder protein sequences and structures for specified protein targets. SSINC incorpo- rates interleaved self-attention layers and k:Nd-based equiv- ariant graph layers, complemented by causal attention layers, enabling it to jointly model sequences, structures, and their complicated interdependencies. The model is pretrained on general pootein-protenn complex design task and fine- tuned for two critical real-world applications, achieving success rates of 50.00%, 23.16% and 16.89% respectively. A potential limitation of this work is the absence of wet-lab validation, which is an essential step planned for our future
Austin, J., Johnson, D. D., Ho, J., Tarlow, D., and Van Den Berg, R. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34:17981-17993 2021.
Bennett, N. R., Coventry, B., Goreshnik, I., Huang, B., Allen, A., Vafeados, D., Peng, Y. P., Dauparas, J., Back, M., Stewart, L., et al. Improving de novo protein binder design with deep learning. Nature Communications, 14 (1):2625, 2023.
Berger, S., Seeger, F,, Yu, T.-Y,, Aydin, M., Yang, H., Rosen-

---

PPDifT: Diffusing in Hybrid Sequence-Strucctue Space for Protcin-Protein Complex Design
blum, D., Guenin-Macé, L., Glassman, C.,, Arguinchona, L.,, Sniezek, C.,, al. Preclinical proof of principle for orally delivered th17 antagonist miniproteins. Cell, 187 (16):4305-431) 2024.
Evans, R., O'Neill, M., Pritzel, A., Antropova, N., Senior, A., Green, T., Zidek, A., Bates, R., Blackwell, S., Yim, J., et al. Protein complex prediction with alphafold alphafold-multmeer biorxiv, pp. 2021-10, 2021.
Binder, J. I.., Berendzen, J., Stevens, A. O., He, Y., Wang. J., Dokholyan, N. V., and Oprea, T. I. Alphafold illuminates half of the dark human proteins. Current Opinion in Structural Biology, 74: 102372, 2022.
Fleishman, S. J., Whitehead, T. A., Ekiert, C., Dreyfus, C., Com,, J. E.,, Strauch, E.-M., Wilson, A., and Baker, D. Computational design of proteins targeting the con- served stem region of influenza hemagglutinin. Science, 332(6031) 1::::11):8666:) 1:16682,, 2011.
Boeckmann, B., Bairoch, A., Apweiler, R., Blatter, M.-C., Estreicher, A., Gasteiger, E., Martin, M. J., Michoud, K., O'Donovan, C., Phan, I., et The swiss-prot pro- tein knowledgebase and its supplement trembl in 2003. Nucleic acids research, (111):6653370 2003.
Gainza, Wehrle, S., Van Hall-Beauvais, A., Marchand, A., Scheck. A., Harteveld, Z., Buckley, S., Ni, D., Tan, S., Sverrisson, F., et al. De novo design of protein in- teractions with learned surface fingerprints. Nature, 617 2023.
Brennan, D. J., O' O'connor, D. P., Rexhepaj. E., Ponten, F., and Gallagher, W. M. Antibody-basee proteomics: fast- tracking molecular diagnostics in oncology. Nature Re- views Cancer, 10(9):605-617, 2010.
Goudy, O. J., Nallathambi, A.. Kinjo, T., Randolph, N. Z., and Kuhlman, B. In silico evolution of autoinhibitory do- mains for pd-11 antagonist using deep learning models. Proceedings of the National Academy of Sciences, 120 (49):22307371222 120, 2023.
Bryant, P.. Pozzati, G., and Elofsson, A. Improved pre- dietion of protein-protein interactions using alphafold2. Nature communications, 13(1))1265, 2022.
Graikos, A., Malkin, N., Jojic, N., and Samaras, D. Diffu- sion models plug-and- and-play priors. Advances in Neural Information Processing Systems, 35: 35:14715-14728, 2022.
Callaway, E. Ai has dreamt up a blizzard of new proteins. do any of them actually work? Nature, 634(8034):532-533 2024.
Gray, A., Bradbury, A. R., Knappik, A., Plückthun, A., Borrebaeck, C. A., and Dübel, S. Animal-free alternatives and the antibody iceberg. Nature Biotechnology, 38(11): 1234-1239, 2020.
Cao, L., Coventry, B., Goreshnik, I., Huang, B., Sheffler, W., Park, J. S., Jude, K. M., Marković, Kadam, R. U., Verschueren, K. H., et al. Design of protein-binding proteins from the target structure alone. Nature, 605 (7910):551-560. 2022.
Guan, J., Qian, W. W., Peng, X., Su, Y., Peng, J., and Ma, J. 3d equivariant diffusion for target-aware molecule genera- tion and affinity prediction. In The Eleventh International Conference on Learning Representations, 2023.
Chao, G., Lau, W. L., Hackel, B. J., Sazinsky, S. L., Lippow, S. M., and Wittrup, K. D. Isolating and engineering human antibodies using yeast surface display. Nature protocols, 1(2):755-768, 2006.
Hackel, B.J. Kapila, A., and Wittrup, K. D. Picomolar affin- ity fibronectin domains engineered utilizing loop length diversity, recursive mutagenesis, and loop shuffling. Jour- nal of molecular biology, 38 381(5): 1238-1181111 12588, 2008
Chevalier, A., Silva, D.-A., Rocklin, G. J., Hicks, D. R., Vergara, R., Murapa, P., Bernard, S. M., Zhang, L., Lam, K.-H., Yao, G., et al. Massively parallel novo protein design for targeted therapeutics. Nature, 550(7674):74- 79, 2017.
Henikoff, and Henikoff, J. G. Amino acid substitution matrices from protein blocks. Proceedings of the National Academy of Sciences, 89(22): 10915-0999111 1992.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Dauparas, J., Anishchenko, I., Bennett, N., Bai, H., Ragotte, R. J. Milles, I. F., Wicky, B. I., Courbet, A., de Haas, R. J., Bethel, N., et al. Robust deep learning-based pro- tein sequence design using proteinmpnn. Science, 378 (6615):49-56, 2022.
Hoogeboom, E., Nielsen, D., Jaini, P., Forré, P., and Welling, M. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34: 1444141441165 2021.
Dunbar, J., Krawczyk, K., Leem, J., Baker, T., Fuchs, A., Georges, G., Shi, J., and Deane, C. M. Sabdab: the structural antibody database. Nucleic acids research, 42 (D1):D1140-D114) 146, 2014.
Humphreys, I. R., Pei, J., Baek, M., Krishnakumar, A., Anishchenko, I., Ovchinnikov, S., Zhang, J., Ness, T. J., Banjade, S., Bagde, S. R., et al. Computed structures of
10

---

PPDiff: Diffusing in Hybrid Sequence-Strucctre Space for Protcin-Protein Complex Design
eukaryotic protein complexes. Science, 374(6)73): eabm4805, 2021.
and selective mimics of il-2 and il-15. Nature, 565(7738):
2019.
Ingraham, J., Garg, V., Barzilay, R., and Jaakkola, T. Gener- ative models for graph-based protein design. Advances in neural information processing systems, 32, 2019.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. In International conference on machine learning, pp. 2256-2266 PMLR, 2015.
Jin, W. Wohlwend, J., Barzilay, R., and Jaakkola, T. S. Iterative refinement graph neural network for antibody sequence-structure co-design. In International Confer- ence on Learning Representations, 2022.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Kabsch, W. and Sander, C. Dietionary of protein secondary structure: pattern recognition of sydrogen-bonded and geometrical features. Biopolymers: Original Research on Biomolecules, 22(12):2577-2653 1983.
Song, Z., Huang, T., Li, L., and Jin, W. Surfpro: Functional protein design based on continuous surface. In Forty-first International Conference on Machine Learning, 2024a.
Song, Z., Zhao, Y., Shi, W., Jin, W., Yang. Y., and Li, Generative enzyme design guided by functionally impor- tant sites and small-molecul substrates. In Forty-first International Conference on Machine Learning, 2024b.
Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint ar arXiv: 1412.6980, 2014.
Li, X., Thickstun, J., Gulrajani, I., Liang, P. S., and Hashimoto, T. B. Diffusion- Im improves controllable text generation. Advances Neural Information Process- ing Systems, 35:4328 43488 2022.
Steinegger, M. and Söding, J. Mmseqs2 enables sensi- tive protein sequence searching for the analysis of mas- sive data sets. Nature biotechmology, 35611):1026-10228 2017.
Lin, Z. Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S., et al. Language models of pro- tein sequences at the scale of evolution enable accurate structure prediction. bioRxiv, 2022.
Stemn, L. A., Case, B. A., and Hackel, B. J. Alternative non-antibody protein scaffolds for molecular imaging of cancer. Current opinion in chemical engineering, 2(4): 425-432, 2013.
Lin, Z. Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. Evolutionary--eet prediction of atomic-level protein structure with a language model. Science, 379(6637)) 1123-1130, 2023.
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic acids research, 50(D)):D49--D44 2022.
Nelson, A. L., Dhimolea, E., and Reichert, J. M. Develop- ment trends for human monoclonal antibody therapeutics. Nature reviews drug discovery, 910))767-774, 2010.
Vaswani, A. Attention all you need. Advances in Neural Information Processing Systems, 2017.
Nichol, A. Q. and Dhariwal, P. Improved denoising diffu- sion probabilistic models. In International conference machine learning, pp. 8162-817 8171. PMLR, 2021.
Vázquez Torres, S., Leung. P. J., Venkatesh, P., Lutz, I. D., Hink, F., Huynh, H.-H., Becker, J., Yeh, A. H.-W., Juer- gens, D., Bennett, N. R., al. De novo design of high- affinity binders of bioactive helical peptides. Nature, 626 47998):435-44, 2024.
Pacesa, M., Nickel, L., Schellhaas, C., Schmidt, J., Pyatova, E., Kissling, L., Barendse, P., Choudhury, J., Kapoor, S.,
Alcaraz-Serna, A., et al. Bindcraft: one-shot design of functional protein binders. bioRxiv, pp. 2024-09, 2024.
Wang. X., Zheng, Z., Fei, Y., Xue, D., Huang, S., and Gu, Q. Diffusion language models are versatile protein learn- ers. In Forty-first International Conference on Machine Learning, 2024.
Satorras, G., Hoogeboom, E., and Welling, M. (n) equivariant graph neural networks. In International con- ference on machine learning, pp. 9323-9332. PMLR, 2021.
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., et al. De novo design of protein struc- ture and function with rfdiffusion. Nature, 620(7976): 1089-1100, 2023.
Silva, D.-A., Yu, S., Ulge, U. Y., Spangler, J. B., Jude, K. M., Labão-Almeida C., Ali, L. R., Quijano-Rubul Rubio, A., Ruterbusch, M., Leung, I., et al. De novo design of potent

---

PPDifI: Diffusing in Hybrid Sequence-Structure Space for Protcin-Proteiie Complex Design
Wu, K., Jiang, H., Hicks, D. R., Liu, C., Muratspahić, E.,
Ramelot, T. A., Liu, Y., McNally, K., Gaur, A., Coven- try, B., al. Sequence-specific targeting of intrinsically disordered protein regions. bioRxiv, pp. 2024-07, 2024.
Wu, L., Gong, C., Liu, X., Ye, M., and Liu, Q. Diffusion- based molecule generation with informative prior bridges. Advances in Neural Information Processing Systems, 35: 36533-36545, 2022.
Yan, Y. Tao, H., He, J., and Huang, S.-Y. The hdock server for integrated pooteinnprotein docking. Nature protocols, 15(5):122-11822 2020.
Yang. W., Hicks, D. R., Ghosh, A., Schwartze, T. A., Con- ventry, B., Goreshnik, I,, Allen, A., Halabiya, S. F., Kim, C. J., Hinck, C. S., et Design of high affinity binders to convex protein target sites. bioRxiv, 2024.
Yim, J., Trippe, B. L., De Bortoli, V., Mathieu, E., Doucet, A., Barzilay, R., and Jaakkola, Se (3) diffusion model with application to protein backbone generation. In In- ternational Conference Machine Learning, pp. 40001- 40039. PMLR, 2023.
Zambaldi, V., La, D., Chu, E., Patani, H., Danson, E., Kwan, T. O., Frerix, T., Schneider, G., Saxton, D., Thillaisundaram, A., et al. De novo design of high- affinity protein binders with alphaproteo. arXiv preprint arXiv:2409.08022, 2024.
Zhang, Y. and Skolnick, Scoring function for automated assessment of protein structure template quality. Proteins: Structure, Function, and Bioinformatics, 57(4)770 2004.
12

---

PPDiff: Diffusing in Hybrid Sequence-Structure Space for Photcin-Protciie Complex Design
Appendix
A. Data Statistics
A.I. Data Statistics for PPBench
Detailed data statistics for the curated popein-proteii complex dataset are reported in Table 6.
Cluster
Complex
Training
22,827
706,149
Validation
10
155
Test
10
56
Total
22,847
706,360
Table 6. Data statistics for PPBench.
Seen Class
Zero-Shot
Models
FGFR2
InsulinR
PDGFR
TGFb
VirB8
H3
IL7Ra
EGFR
TrKA
Tie2
Training
465
192
210
77
Validation
57
23
26
Test
57
23
26
9
38
7
2
Total
579
238
262
95
71
38
7
4
4
2
Table 7. Detailed data statistics for curated target-protein mini-binder complex dataset.
A.2. Data Statistics for Target-Protein Mini-Binder Complex Dataset
We collect experimentally confirmed positive target-protein mini-binder complexes against ten target proteins with diverse structural properties from Bennett et al. (2023). The detailed data statistics for the ten target proteins are reported in Table 7.
CDR-HI Clustering
CDR-H2 Clustering
CDR-H3 Clustering
Complexes 4,261
4,261
4261
Clusters
1,034
1,418
2,254
Training
3,590
3,576
3,427
Validation
403
399
451
Test
268
286
383
Table Detailed data statistics for antigen-antihody complex dataset.
A.3. Data Statistics for Antigen-Antibody Complex Dataset
The detailed data statistics for each CDR clustering reported in Table 8.
B. Supplementary Experimental Results
B.I. Antigen-Antibody Complex Design Performance for Each CDR Clustering
Model performance for each CDR clustering is provided Table 9, 10 and 11. Our proposed PPDiff achieves higher success rate than the strongest baseline SSINC Network on average and obtains much higher novelty scores.
B.2. Swiss-Prot Data Pretraining
To evaluate if pretraining our PPDiff on additional data will improve model performance, we first pretrain our model on curated pseudo Swiss-Prot complexes, and then continue training the model on our constructed PPBench. We compare this
13

---

PPDifI: Diffusing in Hybrid Sequence-Structtre Space for Protcin-Protee Complex Design
Methods
ipTM (1) pTM (1)
PAE ())
pL.DDT(1) Success Rate (1) H1 Novelty (1) H2 Novelty (1) H3 Novelty (1)
SEnc + +ProteinMPNN
0.364
0.566
15,021
76.063
4.85%
60.37%
60.95%
69.39%
InterleavingDiff
0.426
0.621
14,993
81.581
7.83%
62.38 %
68.21%
77.48%
SSINC Network
0.501
0.636
12.759
82.608
14.55
34.87%
38.50%
57.55%
PPDiff
0.484
0.638
14.058
82.509
11.11%
61.62%
69.69%
77.62 %
Table Model performance on -nngen-- complex design task CDR-HI clustering. (f): the higher better. (1): lower the better. Novelty scores for CDR-HI, CDR-H2, and CDR-H3 are reported H1, H2, and H3 novelty, respectively. PPDiff achieves the highest pTM and novelty scores across the designed CDR-H2 and CDR-HB.
Methods
ipTM (1) pTM (1)
PAE ()) PLDDT(1) Success Rate (f) HI Novelty (f) H2 Novelty (f) H3 Novelty (1)
SEnc + +ProtcinMPNN
0,525
0.629
13.339
79.260
7.34%
62.23 %
59.71%
74.66%
InterleavingDiff
0.539
0.681
12.546
81.155
17.48%
55.54%
59.03%
74.51
SSINC Network
0.547
0.658
11.929
82.829
20.27%
30.29%
39.94%
61.13%
PPDiff
0.561
0.682
12.535
83.162
22.03%
57.77%
65.31
75.95 %
Table Model performance on antigen-antibddy complex design task CDR-H2 clustering. (f): the higher the better. (}): lower the better. Novelty scores for CDR-HI, CDR-H2, and CDR-HI3 are reported as H11, H2, and H3 novelty, respectively. PPDiff achieves the highest scores on almost all metrics except CDR-HI novelty score.
Methods
ipTM (t)
pTM (t)
PAE ())
PLDDT((1) Success Rate (1) H1 Novelty (f) H2 Novelty (1) H3 Novelty (1)
SEnc + ProotiinMPNN
0.457
0.539
14.894
78.036
6.52%
62.75%
57.93%
68.49%
InterleavingDiff
0.523
0.659
13.210
80.082
10.70%
54.69%
63.53%
74.75%
SSINC Network
0.569
0.668
11.506
82.682
15.40%
26.
34.67%
60.91%
PPDiff
0.574
0.683
12.405
82.810
17.52%
53.98%
64.18%
74.94 %
Table Model performance on antigen-antibody complex design task for CDR-H3 clustering, (f): the higher the better. (1): lower the better. Novelty scores for CDR-HI, CDR-H2, and CDR-H3 reported as H1, H2, and H3 novelty, respectively. PPDiff achieves the highest scores on the antigen-antihody binding affinity (iPTM) and success rate.
Methods
ipTM (f)
pTM (t)
PAE (L)
PLDDT())
Success Rate (t)
Novelty (t)
Diversity (f)
Ground Truth
0.691
0.782
7.901
86.987
69.64%
PPDiff
0.700
0.779
9.153
83.765
50.00%
89.46%
top-1
With pretraining
0.666
0.770
9.647
82,952
53.57 %
88.89%
PPDiff
0.665
0.747
10.231
81.659
45.71
88.93%
90.76%
top-5
With pretraining
0.663
0.733
10.087
79.597
35.36%
89.90%
91.81
PPDiff
0.633
0.729
10.895
80.322
37.68%
89.10%
91.09%
top-10 10
With pretraining
0.626
0.710
11.255
78.068
27.68%
90.74%
92.17% %
Table Model performance on additional Swiss-Prot data pretraining. (f): the higher the better. (4): the lower better. Calculating the diversity score for the top-1 candidate is unnecessary, it consists of only single candidate.
model and our original PPDiff Table 12. shows pretraining on Swiss-Prot data does not provide significant benefit.
B.3. Smaller Candidate Pool Size
To study the effect of total candidate size, we provide the results for a candidate pool size in Table 13. shows reducing the candidate pool size leads worse model performance on almost all metrics.
14

---

PPDifT: Diffusing in Hybrid Sequence-Structtre Space for Photein-Protcin Complex Design
Methods
ipTM (1)
pTM (1)
PAE (1)
pLDDD((1)
Success Rate (1)
Novelty (1)
Diversity (1)
Ground Truth
0.691
0.782
7.901
86,987
69.64%
SEnč +ProteinMPNN
0.586
0.693
12.029
777.064
28.37%
63.28%
InterleavingDiff
0.631
0.722
11.199
79.651
33.92%
91.61% %
top-1
SSINC Network
0.634
0.736
10.256
81.461
37.50%
37.60%
PPDiff
0.644
0.739
10.289
81.926
39.28% %
88.76%
SEnc +ProteinMPNN
0.569
0.683
12.931
76.201
14.28%
65.37%
62.21%
InterleavingDiff
0.594
0.695
12.594
76,465
15.00%
91.69% %
92.09%
1op-5
SSINC Network
0.602
0.709
11.528
80,800
32.14%
37.26%
15.04%
PPDiff
0.606
0.713
11.459
79.176
30.36%
89.11%
91.45%
SEnc +ProteinMPNN
0.553
0.662
13.641
75.148
7.14%
67.89%
63.92%
interleavingDiff
0.581
0.677
13.512
74.090
7.50%
91.60 %
92.49
top-10
SSINC Network
0.586
0.691
11.856
78.920
21.42% %
37.43%
15.18%
PPDiff
0.592
0.695
12.394
77.014
18.21%
90.18%
91.68%
Table 13. Model performance with candidate pool size set to 30. (f): the higher the better. (4): the lower the better. Calculating the diversity score for the top-1 candidate is unnecessary, it consists of only single candidate.
C. Additional Experimental Details
C.1. PPBench Construction Details
We curate large-scale protein-protein complex dataset by identifying chain-pair interfaces, following the data processing pipeline introduced in AlphaFold3 (Abramson et al., 2024). The curation process begins by applying a series of quality filters to identify valid PDB entries:
Retain structures with reported resolution of 9Å or better.
Remove polymer chains containing unknown residues.
Exclude protein chains where consecutive C₂ atoms are separated by more than 10Å.
Filter out polymer chains with fewer than four resolved residues.
Eliminate clashing chains, defined as chains with more than 30% of their atoms located within 1.7A of an atom in another chain. When two chains clashed:
The chain with the higher percentage of clashing atoms is removed.
the clashing percentage equal, the chain with fewer total atoms is removed.
If both chains have the same number of atoms, the chain with the larger chain ID is removed.
This filtering process result in 126,569 valid PDB entries. From these entries, chain- pair interfaces are identified, defined as pairs of chains with minimum heavy atom separation of less than 5A. This step yields a total of 367,016 chain-pair interfaces.
C.2. Model Training Details
The embedding dimensionality is configured at 1,280. PPDiff' is trained for 1,000,000 steps on single NVIDIA RTX A6000 GPU using the Adam optimizer (Kingma, 2014). The batch size is set to 1,024 tokens, and the learning rate initialized at 5e-6. For structure diffusion, the starting and ending values of 3 are set to 1c-7 and 2e-3, respectively, with a variance schedule of 2. The cosine schedule offset for sequence diffusion configured at 0.01. The number of k-nearest neighbors fixed at 32. Additionally, learning rate warm-up is applied over the first 4,000 steps stabilize the training process.
C.3. Detailed Explanation of Metrics
We provide the detailed explanation of metrics follows:
15

---

PPDifI: Diffusing in Hybrid Sequence-Structure Space for Protcin-Protein Complex Design
ipTM is interfacial variant of the predicted TM-score, evaluating the interaction between different chains (Abramson et al., 2024). Values higher than 0.8 represent confident high-quality predictions.
pTM is a confidence score estimating the accuracy of global protein structure prediction. A pTM score above 0.5 means the overall predicted fold for the complex might be similar to the true structure (Zhang Skolnick, 2004).
PAE estimates the emor in the relative position and orientation between two tokens in the predicted structure. Higher values indicate higher predicted error and therefore lower confidence.
PLDDT aims predict modified LDDT score that only considers distances to polymers. is a per-atom confidence estimate on a 0-100 scale where higher value indicates higher structural stability.
C.4. Target-Protein Mini-Binder Complex Design Evaluation Details
To caleulate the pnt_erraaattion score, we first predict the structure of the designed binder sequence using ESMFold (Lin al., 2023). The predicted structure then superimposed the experimentally validated positive binder, and the AF2 PAE interaction score is calculated for the resulting target-protein mini-binder complex. Using these scores, we determine the success rate by generating five mini-binder candidates for each target protein. The success rate is defined as the proportion of designed binders that achieve lower pAE. interaacii score with the target protein than the ground truth positive binder:
4 Σ [(,((T,,,)))
SR
Σ
(16)
DDest lo
TED km
where k=5, S(T,((T,B))) B) = true if the i-th binder candidate B, for target protein T achieves lower pAEiintercciion score than the positive binder B. Otherwise, f(T, B,, B) = false.
D. A Additional Designed Cases
We provide more designed protein-protein complexes for general protein- protein complex design, target-protein mini-binder complex design and antigen-antibody complex design in Figure 4, 5 and 6, respectively, It shows our PPDiff is capable of designing novel and high-affinity protein-bing bodein-binding proteins for diverse protein targets across wide range of domains.
16

---

PPDifT: Diffusing in Hybrid Sequence-Structure Space for Photein-Protcin Complex Design
(c)
(d)
(a) ipTM=0.86, PTM=0.86, PAE=6.93, PLDDT=87.88
(b) ipTM=0085, PTM=-085, PAE=7.07, P1DD1=87.61, novelty=96.729% 72%
ipTM=0082, PTMM=088,
ipTM==088, PTM-0.86.
PAE=7.49, PLDDT=88.31 novelty=81.556
PAE=6.93, 588. novelty=93.44%
novelty=83.199
(c)
(e) ipTM=0.89, p8M=0.88, PAE=6.53, p.DDDT888.67 novclt=996.66%
(1) ipTM-0.76, pTM=0.81, PAE=8.25 25, =82.33
ipTM=0.87, pTM=0086. PAE=6.86, PLDDT=88. 10, novclty=94.37%
ipTM=0.86 pTM=0.85.
PAE=6.40, pLDDDT=89.75 novchy=92.211%
novelty=95.366 93%6%
Figure Designed complexes for general moteinn-roteenn complex design by PPDiff. All of them achieve ip approaching or higher than 0.8, pTM score above 0.8, PAE lower than 10 and PLDDT better than 80. These designed binder sequences also have novelty scores higher than 80%, validating that PPDiff is capable of designing novel and high-affinity protein-binding proteins diverse protein targets.
17

---

PPDifT: Diffusing in Hybrid Sequence-Structure Space for Photein-Protcinn Complex Design
(c)
(d)
(a) i pTM=0.87, PAE-5.24 24, PLDDT=85.87,
(b) ipTM=00778 pTM==088, PAF=4.37, p87D77=87.47, novelty=93.88 target protein=IL7Ra
ipTM=0072, pTM==0799
ipTM==0.77, pTM==074,
PAR=8.40, pI.DDT=80.70 70, novelty=91.95 target protein=IL7Ra
PAF=7.34, p8..DT=
novelty=92.06%% target
novelty=91.8 target
protein=InsulinR
protein=H3
Figure Designed complexes by our PPDiff for target-protein mini-binder complex design. them achieve an ipTM score higher than 0.7, pTM score above 0.7, PAE lower than 10 and pLDDT better than 80. These designed binder sequences also have novelty scores higher than 80%, validating that PPDiff is capable designing novel and high-affinity binder proteins across diverse protein targets.
(a) ipTM=0081, p7M=0.87,
(b) ipTM=0.83, pTM=0.89,
(c) ipTM-0.89, 9M.-9.91
ipTM=0.81 PTM=0.87,
PAB=6.30, PLDDT=84.22, CDR-HI 62v5l1
PAE-5.36, PLDDT=87.38,
PAE=4.01, PLDDT=91.11
PAE-6.09, pLDDT=84.18
CDR-HI novelty=50.00%.
CDR-HI novelty=62.50%
CDR-HI novelty=62.50%. CDR-H2 novelty=50.00%. CDR-H3 novelty=75.00%
CDR-H2 novelty=70.00%.
CDR-H2 novelty=50.00%
CDR-H2 novelty=50.00%%
CDR-H3 novelty=83.33%
CDR-H3 novelty=60.005%
CDR-H3 H3 novelt==3.33%%
Figure 6. Designed complexes by our PPDiff for antigen-antibody complex design. All of them achieve an ipTM score higher than 0.8.
PTM score above 0.8, PAE lower than 10 and PLDDT better than 80. These designed binder sequences also have CDR-HI, H1, H2 and H3 novelty scores higher than 50% validating that PPDiff is capable of designing novel and high-affinity antibodies for antigens.
18


Reviews:
[{"review_id": "ew4XdYPaA2", "reviewer": "Reviewer_oJL7", "summary": "The paper presents a diffusion based generative model to create binding molecules for given protein targets. To do so the paper proposes a joint model which combines both the coordinates and types of residue sites. The major novelty resides in the score function network which alternates between self-attention and graph convolution layers. They then experiment with their model on a subset of PDB which they call PPBench. This seems to be another contribution in terms of dataset. On this data, the model shows improved accuracy and novelty.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "I have not had the time to review this. I am placing some summary comments for aiding the review process, but do not expect any specific response for my concerns.", "methods_and_evaluation_criteria": "Diffusion model for protein binding challenges have been proposed before. Look at DiffBP: Generative Diffusion of 3D Molecules for Target Protein Binding and following works. None of these have been cited or compared as a baseline; especially as some of these are similar in flavor. For example diffbp while looking at molecules, almost straightforwardly can be used for protein protein binding as well.", "theoretical_claims": "I have not had the time to review this. I am placing some summary comments for aiding the review process, but do not expect any specific response for my concerns.", "experimental_designs_or_analyses": "I have not had the time to review this. I am placing some summary comments for aiding the review process, but do not expect any specific response for my concerns.", "relation_to_broader_scientific_literature": "I have not had the time to review this. I am placing some summary comments for aiding the review process, but do not expect any specific response for my concerns.", "essential_references_not_discussed": "I have not had the time to review this. I am placing some summary comments for aiding the review process, but do not expect any specific response for my concerns.", "other_strengths_and_weaknesses": "I have not had the time to review this. I am placing some summary comments for aiding the review process, but do not expect any specific response for my concerns.", "other_comments_or_suggestions": "I have not had the time to review this. I am placing some summary comments for aiding the review process, but do not expect any specific response for my concerns.", "questions_for_authors": "The authors mention that they have their proposed network SSINC also as a baseline but without diffusion loss. I am confused as to how it generates the output then. Is it trained like an auto-regressive language model? However in the introduction they say that their model is non-autoregressive.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qxglFBq98m", "reviewer": "Reviewer_zYHd", "summary": "The authors propsoe a new diffusion method to tackle the protein complex problem. They define a co-design diffusion method that generates both the structure and the sequence of a protein complex. Then, they introduce a new architecture based on causal attention mechanism as well as knn equivariant layers. They pretrain this new model on a protein complex dataset issued from PDB (and SwissProt) and finetuned the model on 2 downstream tasks (antibody-antigen generation and mini-binder complex) and show state-of-the-art results.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable comments and insightful suggestions. We have addressed all raised concerns and conducted additional experiments as recommended. Please find detailed responses to the specific points below:\n\n**W1:Essential References Not Discussed:I think [1] should be discussed. It is a diffusion method based on framed.** \n\nAns: We appreciate the reviewer for highlighting this relevant reference. Proteus is an innovative frame-based diffusion approach, integrating a graph-based triangle technique and a multi-track interaction network, which shows robust capabilities in designing protein backbone structures. We will include a citation and thorough discussion of this method ([1]) in the revised manuscript.\n\n[1] Proteus: pioneering protein structure generation for enhanced designability and efficiency. \n\n**Comment: The authors did not cite framediff but framflow**\n\nAns: Thank you for pointing out this oversight. We will appropriately cite the correct reference, Framediff ([2]), in our revised manuscript.\n\n[2] SE(3) diffusion model with application to protein backbone generation. Yim et al. ICML 2023.\n\n**Q1: Can you finetune your method for the binder task and evaluate it against RFDiffusion? This would make the paper much stronger in my opinion.**\n\nAns: Thank you for this valuable suggestion. We initially followed the binder design procedure outlined in the RFDiffusion paper to generate the backbone structures. Subsequently, ProteinMPNN was employed to design corresponding binder sequences based on these backbone structures. To ensure a fair comparison, we pretrained ProteinMPNN on our curated PPBench and then fine-tuned the pretrained model specifically for the downstream binder design task. As RFDiffusion does not provide a training script, we utilized their publicly available model weights directly. Evaluation of success rates was conducted consistent with the methodology described in our manuscript. The average success rates across each target protein category are presented below. **Our results indicate that our model outperforms RFDiffusion + ProteinMPNN for 5 out of 10 target proteins, and achieves a higher overall average success rate across all tested target proteins. Notably, our PPDiff achieves much higher novelty and diversity scores, demonstrating our model's superior capability in designing high-affinity, novel and diverse binders**. \n\n**Table: Success Rate, Novelty and Diversity on Target Protein-Mini Binder Design Task**\n\n|                         | Seen Class |          |        |        |        | Zero-Shot |        |      |         |        | Average Success Rate | Novelty | Diversity |\n|-------------------------|------------|----------|--------|--------|--------|-----------|--------|------|---------|--------|---------|---------|-----------|\n| Target Protein          | FGFR2      | InsulinR | PDGFR  | TGFb   | VirB8  | H3        | IL7Ra  | EGFR | TrkA    | Tie2   | Average Success Rate | Novelty | Diversity |\n| RFDiffusion+ProteinMPNN | **28.07%**     | 8.69%    | **15.38%** | 22.22% | **57.14%** | 7.89%     | 28.57% | **25.00%**  | **100.00%** | 0.0    | 21.46%  | 78.10%  | 25.71%    |\n| PPDiff                  | 7.36%      | **10.43%**   | 14.61% | **35.56%** | 11.42% | **55.26%**    | **60.00%** | 0.0  | 30.00%  | **30.00%** | **23.16%**  | **91.39%**  | **91.79%**    |", "claims_and_evidence": "They claim to define a new co-design method and this claim is true. They also claim that their method is competitive and this also seems true.", "methods_and_evaluation_criteria": "The method makes a lot of sense and I like the perspective to challenge existing architecture. To the best of my knowledge, the evaluation makes sense and follows the standard procedure.", "theoretical_claims": "Not applicable", "experimental_designs_or_analyses": "The experiments make sense and the analysis of their method is complete with a lot of ablation and sensitivity analysis (number of steps, different datasets, ...).", "relation_to_broader_scientific_literature": "This seems correct to me especially as most backbone generation method are trained for monomers and not protein complex generation.", "essential_references_not_discussed": "I think [1] should be discussed. It is a diffusion method based on framed\n\n[1] Proteus: pioneering protein structure generation for enhanced designability and efficiency", "other_strengths_and_weaknesses": "The paper is well written.", "other_comments_or_suggestions": "The authors did not cite framediff but framflow (page 2)", "questions_for_authors": "Can you finetune your method for the binder task and evaluate it against RFDiffusion? This would make the paper much stronger in my opinion.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1J7qdAYJO2", "reviewer": "Reviewer_RZ25", "summary": "This paper presents PPDiff, a novel diffusion-based model for protein-protein complex design. The model aims to generate protein binders with high affinity for arbitrary target proteins by simultaneously designing both the sequence and structure of the binder. PPDiff builds upon the Sequence Structure Interleaving Network with Causal attention layers (SSINC). The authors introduce PPBench, a dataset consisting of 706,360 protein-protein complexes curated from the Protein Data Bank (PDB). The model is pretrained on PPBench and further fine-tuned on two key applications: Target-protein mini-binder complex design and Antigen-antibody complex design.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the reviewer for the insightful and constructive feedback. We have conducted additional experiments as suggested. Detailed responses to specific questions are provided below:\n\n**Q1: To strengthen the in silico validation, could the authors provide docking scores for the designed binders, particularly for well-known targets?**\n\nAns: Thank you for this valuable suggestion. Because our PPDiff designs provide only alpha-carbon backbones, they are not directly applicable for docking. Therefore, we first used ESMFold to generate the structures of our designed binder sequences. We then performed docking simulations using HDOCK [1], pairing each predicted binder with its corresponding target protein. For each target, we produced five binder candidates and reported the average docking score for the top-1 candidate in Table 1. As noted in the original HDOCK paper, **a more negative docking score means a more possible binding model**; notably, this score should not be treated as the absolute measures of binding affinity because it has not been calibrated to the experimental data. The results show that **PPDiff consistently achieves more negative docking scores across all ten target proteins compared to baseline methods, indicating that binders designed by our model exhibit stronger potential affinities**. Additionally, Table 2 compares these scores with ground truth docking values, demonstrating that **PPDiff even outperforms experimentally confirmed binders (ground truth) in five categories** — a finding that highlights its potential for designing high-affinity protein binders.\n\n[1] The HDOCK server for integrated protein–protein docking. Yan et al. Nature Protocols. 2020.\n\n**Table1: Docking Scores Comparing with Baselines**\n\n|     | Seen Class |     |   |    |   | Zero-Shot |     |   |   |    | Average  |\n|------------|------|----|----|----|----|----|----|----|----|----|----------|\n|Target Protein|FGFR2|InsulinR|PDGFR|TGFb|VirB8|H3|IL7Ra|EGFR|TrkA|Tie2|Average|\n| SEnc +ProteinMPNN | -197.82 | -192.56  | -231.46 | -203.41 | -235.67 | -198.23   | -192.85 | -178.23 | -224.91 | -201.32 | -205.64  |\n| InterleavingDiff  | -230.40 | -233.49  | -234.60 | -231.93 | -222.56 | -227.03   | -229.93 | -218.26 | -234.12 | -230.43 | -230.25  |\n| SSINC Network     | -207.76 | -193.18  | -226.77 | -211.04 | -220.34 | -206.02   | -207.24 | -183.53 | -217.91 | -196.22 | -208.48  |\n| PPDiff  | **-256.86** | **-260.95**| **-270.55** | **-251.35** | **-252.69** | **-244.23**   | **-261.36** | **-244.75** | **-266.06** | **-265.19** | **-256.45**  |\n\n**Table2: Docking Scores Comparing with Ground Truth**\n\n|                | Seen Class |          |         |         |         | Zero-Shot |         |         |         |         | Average  |\n|----------------|------------|----------|---------|---------|---------|-----------|---------|---------|---------|---------|----------|\n| Target Protein | FGFR2      | InsulinR | PDGFR   | TGFb    | VirB8   | H3        | IL7Ra   | EGFR    | TrkA    | Tie2    | Average  |\n| Ground Truth   | -250.35 | **-339.78** | -218.2 | **-289.42** | **-282.15** | **-287.48** | -244.68 |**-316.38**| -227.56 | -261.83 | **-271.783** |\n| PPDiff         | **-256.86**    | -260.95  | **-270.55** | -251.35 | -252.69 | -244.23   | **-261.36** | -244.75 | **-266.06** | **-265.19** | -256.45  |", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes. The formulation part of this paper clearly showed how to jointly diffuse protein sequence and structure.", "experimental_designs_or_analyses": "Yes. This work mainly used top-k success rate to represent the effectiveness of the model.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "This paper is well-written and presents a strong contribution. The authors propose a co-design framework for generating binders conditioned on a target, demonstrating superior performance compared to existing models. Unlike previous works, such as this study (https://openreview.net/pdf?id=dq3g7Bl9of), which focus more on backbone and sequence optimization, this paper emphasizes binder design, potentially making it applicable to a broader range of scenarios. Additionally, the experiments are comprehensive, with a thorough analysis of the impact of different components of the model. Regarding weaknesses, a common challenge in protein design models is the lack of experimental validation through wet-lab experiments. To strengthen the in silico validation, could the authors provide docking scores for the designed binders, particularly for well-known targets? This would help assess the binding efficacy and further support the model’s effectiveness.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TEvcWiPRkS", "reviewer": "Reviewer_c1qo", "summary": "The paper focuses on the task of generative binder design, modeling protein-protein complexes. This is a critical and impactful task in protein design. The authors introduce *PPDiff*, which uses a protein sequence and backbone co-design strategy during generation leveraging a joint diffusion framework. An important component of the work is the new neural network architecture, which interleaves self-attention layers for sequence processing with equivariant graph layers to capture structure details. Furthermore, the work curates a new dataset, PPBench, consisting of around 700k protein complexes for training. The model is extensively validated on several binder design tasks and demonstrates strong success rates. The authors also extensively ablate and analyze their approach and the design choices.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's dedication in providing detailed feedback. We have clarified all the reviewer’s concerns and conducted additional experiments accordingly. All ablation studies were conducted by generating one candidate and we evaluated the resulting complexes using AF3 with the same seed. Responses to specific points are provided below:\n\n**Q1: What is the co-designability score?**\n\nAns: We evaluated three consistency metrics for the top-1candidate: (1) Seq RMSD: RMSD between the folded structure of the designed sequence and ground truth, (2) Struct RMSD: RMSD between the designed structure and ground truth, (3) Design scRMSD: RMSD between the folded structure of the generated sequence and the designed structure. Although Struct RMSD and Design scRMSD are higher than Seq RMSD, incorporating structural information significantly enhances binder sequence design. As we can see, PPDiff achieves a substantially lower Seq RMSD (1.13) than a sequence-only model (5.79, Finetuned ESM2 on PPBench), validating the importance of co-design in producing sequences that closely align with their intended structures.\n\n||Seq RMSD|Struct RMSD|Design scRMSD|\n|--|--|--|--|\n|Finetuned ESM2|5.79|--|--|\n|SEnc +ProteinMPNN|6.24|**6.03**|7.46|\n|InterleavingDiff| 2.89|6.45|7.51|\n|SSINC Network|1.23|6.89|7.94|\n|PPDiff |**1.13**|6.32|**6.87**|\n\n**Q2: It would be appropriate to run RFDiffusion with ProteinMPNN.**\n\nAns:  Please refer to the **response of reviewer zYHd Q1.**\n\n**Q3: Essential References Not Discussed**\n\nAns: We appreciate the reviewer's valuable suggestions. We will cite and discuss MultiFlow, D3PM, and BindCraft in the revised version. We kindly note that our discrete sequence diffusion part is based on [1] as introduced in Sec. 3.3 line 144-148 in our manuscript, which precedes and is referenced by D3PM.\n\n[1] Argmax flows and multinomial diffusion: Learning categorical distributions. Hoogeboom et al. NeurIPS 2021.\n\n**Q4: About dataset release**\n\nAns: We will release all datasets shortly.\n\n**Q5: Can the authors clarify how the target protein is fed to the model as conditioning?**\n\nAns: We concatenate the sequences of the target and binder proteins, and represent all residues in target and binder proteins jointly as a unified point set.\n\n**Q6: In Eq (7), $s_{t-1}$  should depend on $s_t$ and $x_t$, and similarly $x_{t-1}$.**\n\nAns: We appreciate the reviewer's suggestion and agree with the correction. We will update our manuscript in the revised version.\n\n**Q7:Why does the layer need to be causal?**\n\nAns: Without causal attention layers, sequences displayed repetitive residue types (e.g., \"EEEE\"), resembling multimodality issues observed in non-autoregressive MT [2]. We introduced causal attention layers for autoregressive dependency management, significantly improving performance, as shown in our ablation study below. \n\n[2] Non-Autoregressive Neural Machine Translation. ICLR 2018.\n\n||ipTM|pTM|PAE|pLDDT|\n|--|--|--|--|--|\n|PPDiff - Self Attention|0.562|0.642|15.153|70.185|\n|PPDiff - Casual Attention|**0.575**|**0.650**|**14.719**|**71.022**|\n\n**Q8: Ho et al. suggested $\\lambda_t=1$ for the noise prediction setting, not for $x_0$ prediction**\n\nAns: We apologize for any confusion. We set $\\lambda_t=1$ following previous work [3]. We will accordingly update our manuscript.\n\n[3] 3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction. ICLR 2023.\n\n**Q9: The authors propose a more informative prior and sequence guidance, which should be supported by ablation studies.**\n\nAns: Ablation studies, as presented below, clearly indicate that removing either the informative prior or sequence guidance decreases performance. \n\n| |ipTM|pTM|PAE|pLDDT|\n|--|--|--|--|--|\n|PPDiff|**0.575**|**0.650**|**14.719**|**71.022**|\n|- w/o informative prior|0.476|0.615|16.478|66.494|\n|- w/o sequence guidance|0.524|0.638|15.372|70.018|\n\n**Q10: Why exactly do the mini-binder generation experiments use the AF2 pAE_interaction score for binder evaluation**\n\nAns: Following established practices in previous studies [4], we utilized the AF2 pAE_interaction score to evaluate binder designs due to its proven effectiveness in distinguishing experimentally validated binders from non-binders. Previous research has shown that binder candidates selected using AF2 pAE_interaction scores yield experimental success rates ranging from 1.5% to 7% across various target proteins [4].\n\n[4] Improving de novo protein binder design with deep learning. Nature Communication. 2023. \n\n**Q11: How important is the initialization of self-attention layers from ESM2?**\n\nAns: We conducted an ablation study on removing ESM2 initialization below. Results indicate performance degradation upon removal, demonstrating the significance of leveraging ESM2's pretrained knowledge as an effective initialization for PPDiff. \n\n| |ipTM|pTM|PAE|pLDDT|\n|--|--|--|--|--|\n|PPDiff|**0.575**|**0.650**|**14.719**|**71.022**|\n|-w/o initialization|0.461|0.534|18.174|63.853|", "claims_and_evidence": "The paper's main claim is the development of a novel binder design generation model with high success rates. This claim is extensively validated in numerical experiments. The presented experimental results are indeed promising and evidence for PPDiff's strong binder generation performance. There are no concerns regarding claims and evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria do overall make sense and are appropriate for the problem at hand. I do have some concerns, though: \n\n1. PPDiff jointly generates sequences and structures. All quantitative evaluations, however, seem to be carried out on the structures based on the folded sequences with AF3 or AF2. This means the structure that is co-generated from PPDiff itself is not actually scored at all or used in any evaluations, if I understand the work correctly. An obvious question is, what is the co-designability score? Does the generated sequence actually fold into the structure that is output by PPDiff itself (e.g. what is the RMSD of the generated structure against the folded structure based on the sequence)? I believe this should be analyzed and evaluated for any model that jointly generates sequences and structures. This corresponds to the co-designability metric that is, for instance, used here, \"Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design\", https://arxiv.org/abs/2402.04997.\n\n2. As one baseline the authors choose a purely structure-based generator together with ProteinMPNN (\"SEnc+ProteinMPNN\"), and the authors point out that there are generally no baseline models to compare to. While it is true that there are no \"standard\" binder design benchmarks, I do believe that as an additional baseline, similar to the mentioned one, it would be appropriate to run RFDiffusion with ProteinMPNN, as it is known that RFDiffusion has been successfully used for binder design. Quite possibly, RFDiffusion+ProteinMPNN would perform better than the SEnc+ProteinMPNN baseline, and this would be the only true \"not-self-designed\" baseline.\n\nIt would be great if the authors could comment on these issues.", "theoretical_claims": "The paper does not have any advanced mathematical proofs or theoretical claims. The mathematical details in the method section seem mostly correct. There are some issues (discussed later), but these do not represent major flaws.", "experimental_designs_or_analyses": "Yes, I checked the soundness and validity of the experimental designs and analyses, but did not identify and major issues.", "relation_to_broader_scientific_literature": "Overall, the work is appropriately positioned with respect to the broader scientific literature. The paper's introduction is very informative in that regard and discusses prior approaches. A dedicated related work section extends the discussion.", "essential_references_not_discussed": "While overall the discussion of related work is appropriate, I believe some key citations are missing. The work relies on a joint sequence-structure co-generation framework. Maybe the first work to do such co-generation for protein design was *MultiFlow* (https://arxiv.org/abs/2402.04997, ICML 2024). This work is not cited or discussed. Moreover, the generation process of the categorical residue identities seems to exactly correspond to the process first described in the seminal *D3PM* paper, (https://arxiv.org/abs/2107.03006, NeurIPS 2021). Also this work is not cited or discussed. The authors only cite Guan et al., but I believe these two papers are even more important.\n\nFinally, a recent influential work for generative binder design is *BindCraft* (https://www.biorxiv.org/content/10.1101/2024.09.30.615802v1). This work also is not mentioned in the paper. I think this is okay, because BindCraft can be considered concurrent, and therefore this does not affect my paper rating. However, it would certainly make the paper stronger if also BindCraft was discussed and ideally evaluated as an additional baseline -- this can be considered an optional suggestion to the authors.", "other_strengths_and_weaknesses": "**Strengths:**\n- The paper is generally well written and (mostly) easy to follow.\n- I appreciate the paper's data engineering and the curation of the new PPBench dataset.\n- The extensive analyses and ablation experiments are insightful.\n- The paper demonstrates strong binder generation performance, according to the success metrics, an important task that has not been studied that broadly in the machine learning literature. This makes the work significant.\n- The paper provides an anonymous link to source code. I did not check this code, but I appreciate the sharing of the code.\n\n**Weaknesses:**\n- There are some concerns regarding the method's evaluation (see above).\n- There are some concerns regarding the discussion of related work (see above).\n- Some aspects in the paper are not well-explained and some details seem slightly incorrect (see \"Question For Authors\" below).\n- The paper makes some choices when designing the method that are not clear or well-motivated, see \"Question For Authors\" below.", "other_comments_or_suggestions": "I strongly encourage the authors to publicly release the PPBench dataset, as well as the separate curated datasets for the mini-binder and antibody/antigen design experiments.", "questions_for_authors": "1. In Section 3.4, the generation process of the binder protein is described in detail. However, it is not clear how exactly the target protein is fed to the model as conditioning, this is, how exactly the target sequence/structure enter the self-attention or equivariant graph layers. Also in figure 1, this is not clear. Can the authors clarify this? The conditioning implementation can be critical for strong performance, but this is not clear.\n\n2. I believe equation (7) is not correct. The way the generation process is written in the first line of equation (7) means that there is a strict independence between the sequence and structure generation. However, this is not true, but sequence and structure are generated jointly, dependent on each other with one network processing both. This is, $s_{t-1}$ depends both on $s_t$ and $x_t$, and similarly $x_{t-1}$ also depends on both. I think we should have $... = p_\\theta(s^B_{t-1} |s^B_t, x^B_t, T) p_\\theta(x^B_{t-1} |x^B_t, s^B_t, T)$. This applies to everywhere in the paper, where any of the $p_\\theta$ occur. I would like the authors to comment on that or clarify, if I am misunderstanding something.\n\n3. The authors add a causal attention layer to the model, which improves performance. However, why does this layer need to be *causal*? This is not well-motivated or discussed. Why not a regular attention layer instead? This causal layer imposes a direction in the sequence, but there is no natural direction in the protein sequence.\n\n4. \"As suggested by Ho et al. (2020), we set $\\lambda_t=1$, ...\". Ho et al. suggested this for the epsilon/noise prediction setting, not for $x_0$ prediction, so this sentence does not seem appropriate.\n\n5. In Eq. (13), the authors propose a more informative prior. That such a design choice in the method is relevant should be supported by an ablation experiment training with and without this informative prior, but such an experiment is missing.\n\n6. Also in Section 3.6, the authors write *\"For sequence guidance, we randomly sample secondary structure fragments\nfrom the training dataset, identified by using DSSP. ...\"* The authors should provide details what exactly they are doing here and also run an ablation over this design choice. This is a rather unusual choice, as most discrete diffusion models start their generation process sampling from a uniform categorical distribution (or all masked), but not with samples from the dataset. This may also bias the generation. I would like the authors to comment on this.\n\n7. Why exactly do the mini-binder generation experiments use the AF2 pAE_interaction score for evaluation and the other experiments the AF3-based ipTM, pTM, PAE and pLDDT scores? Wouldn't it be better to use all scores in all experiments?\n\n8. The authors initialize their self-attention layers from ESM2. How important is this? How would the model perform if those layers were initialized randomly? An ablation experiments over this would be quite relevant, too.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces a diffusion-based generative model for protein-protein complex design, which jointly generates sequences and structures using a co-design approach. The authors propose a novel neural architecture combining self-attention with equivariant graph layers and curate a large-scale dataset, PPBench, to support training and evaluation. 

Reviewers highlight several strengths: strong performance on binder design tasks, clear evidence of co-design benefits, and competitive results in docking simulations. The rebuttal provides additional validation, including success rates, co-designability metrics, and docking scores compared to RFDiffusion and experimental binders. These responses address the main technical and evaluation concerns.

The paper contributes a new method, dataset, and thorough analysis. While some aspects (e.g., conditioning mechanism, evaluation setup) could be clearer and more standard comparisons (e.g., with BindCraft) would strengthen the work, the current version meets the bar for acceptance.
Decision: Accept (poster)