An All-Atom Generative Model for Designing Protein Complexes
Authors: Ruizhe Chen
Dongyu Xue
Xiangxin Zhou
Zaixiang Zheng
xiangxiang Zeng
Quanquan Gu
Venue: ICML 2025 poster (2025)
Keywords: protein design
flow matching
antibody design
peptide design
multi-chain protein
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: This work presents a foundational model for all-atom protein complexes design.

Abstract: Proteins typically exist in complexes, interacting with other proteins or biomolecules to perform their specific biological roles. Research on single-chain protein modeling has been extensively and deeply explored, with advancements seen in models like the series of ESM and AlphaFold2. Despite these developments, the study and modeling of multi-chain proteins remain largely uncharted, though they are vital for understanding biological functions. Recognizing the importance of these interactions, we introduce APM (all-Atom Protein generative Model), a model specifically designed for modeling multi-chain proteins. By integrating atom-level information and leveraging data on multi-chain proteins, APM is capable of precisely modeling inter-chain interactions and designing protein complexes with binding capabilities from scratch. It also performs folding and inverse-folding tasks for multi-chain proteins. Moreover, APM demonstrates versatility in downstream applications: it achieves enhanced performance through supervised fine-tuning (SFT) while also supporting zero-shot sampling in certain tasks, achieving state-of-the-art results. We released our code at https://github.com/bytedance/apm.

An All-Atom Generative Model for Designing Protein Complexes
Ruizhe Chen '13
Dongyu Xue Xiangxiang Zeng'
Xiangxin Zhou 23 Quanquan Gu
Zaixiang Zheng? 3
Abstract
Proteins typically exist in complexes, interact- ing with other proteins or biomolecules per- form their specific biological roles. Research on single-chain protein modeling has been exten- sively and deeply explored. with advancements seen in models like the series of ESM and Al- phaFold2. Despite these developments, the study and modeling of multi-chain proteins remain largely uncharted, though they are vital for un- derstanding biological functions. Recognizing the importance of these interactions, we introduce APM (All-Atom Protein Generative Model), a model specifically designed for modeling multi- chain proteins. By integrating atom-level infor- mation and leveraging data on multi-chain pro- teins, APM is capable of precisely modeling inter- chain interactions and designing protein com- plexes with binding capabilities from scratch. It also performs folding and inverse-folding tasks for multi-chain proteins. Moreover, APM demon- strates versatility in downstream applications: it achieves enhanced performance through super- vised fine-tuning (SFT) while also supporting zero-shot sampling in certain tasks, achieving state-of-the-art results. We released our code https: /gthhtt...hgg.ggg/gggggggg..hhgt wwwp///t/waace/////oo///cctttcee apm.
Figure I. Interactions minor atom-level protein structure changes in the binding surface (middle colored part). Blue indi- cates the isolated structure, pink indicates the binding structure.
design models for specific functions. The former includes methods such as protein folding models (Jumper et al., 2021; Lin et al., 2023; Baek et al., 2021), 1, inverse-fodding mod- els (Dauparas et al., 2022: Hsu al., 2022: Zheng al., 2023), co-design models (Shi et al., 2023; Campbell et al., 2024), and protein language models (Lin et al., 2023). These works are not specifically designed for any particular pro- tein design task but aim to learn the general distribution of protein sequences and structures from extensive protein data. The latter approach focuses on the design of proteins with explicit biological activities, such as antibodies (Kong et al., 2023a), binding peptides (Li et al., 2024), and en- zymes (Song al., 2024).
General protein foundation models have demonstrated im- pressive performance across broad range of tasks. How- ever, these approaches focus solely modeling single- chain proteins. In contrast, when dealing with proteins involved in specific functions, the target proteins usually ap- pear in the form of complexes. Furthermore, in multi-chain protein modeling. inter-chain interactions that occur at the atom-level play crucial role (Figure 1). This necessitates incorporating models with atom-level information to enable precise learning of these interactions, which is fundamental for the effective modeling of multi-chain proteins.
Introduction
The application of Al technology in protein design has be- come a prominent research direction across biology, mate- rials science, and artificial intelligence (Notin et al., 2024). The existing works can be categorized into two distinct ap- proaches: general protein foundation models and protein
"qqual contribution Project Lead College Computer Sci- ence and Electronic Engineering, Hunan University "School of Artificial Intelligence, University of Chinese Academy of Sciences "ByteDance Seed (this work done during Ruizhe Chen and Xiangxin Zhou's 's internship at ByteDance Seed). Correspondence to: Quanquan Gu <quanquan gu @ bytedance.com com>>.
bridge this gap, we propose a novel method: APM (All- Atom Protein Generative Model). APM facilitates the gen- eration of multi-chain protein complexes with all-atom structures and can be applied to various tasks involving multi-chain protein complexes, including generation, fold- ing, inverse-folding. and specific functional protein designs. To develop such a generative model that can be used for designing bioactive complexes, we identify three core chal-
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

An All-Atom Generative Model for Designing Protein Complexes
lenges: multi-chain protein modeling, all-atom represen- tation, and sequence-struucture dependency.
dency. Second, there is 50% probability of performing a folding/inverse-foldin task, compelling the model to learn the dependencies from both directions.
Multi-Chain Protein Modeling. Some efforts have at- tempted to adapt single-chain models for multi-chain pro- tein tasks by using a poly-G pseudo sequence connect different chains, treating them as a single chain, includ- ing AlphaFold2 (Jumper et al., 2021), ESMFold (Lin et al., 2023), and Linker- Tuning (Zou et al., 2023). This enables compatibility with multi-chain data but constrains the struc- tural connectivity to head-to-tail linking, which is not repre- sentative of natural complex formations. In this work, we adopt native method for modeling multi-chain proteins through both data integration and modeling strategies. For data, we use a mixture of single and multi-chain data in the training of APM. We believe that intra- chain modeling will benefit from the extensive amount of single-chain data. For modeling, our efforts include improving the model design and introducing conditional generation tasks. Key changes in model design focus on encoding more information with- out altering the overall model structure, such as introducing inter-chain or intra-chain attention, thereby maintaining con- sistency across single and multi-chain proteins.
Finally, APM has demonstrated its capability in modeling multi-chain proteins and generating bioactive complexes. It achieved oatee-of-the-arr (SOTA) performance in antibody design and binding peptide design. Besides, APM also exhibited exceptional performance in conventional single- chain protein-related tasks.
We highlight our main contributions follows:
APM natively supports the modeling of multi-chain proteins without the need to use pseudo sequence to connect different chains;
APM generates proteins with all-atom structures ef- ficiently by utilizing an innovative integrated model structure;
Experiments related to general protein demonstrate that APM is capable of generating tightly binding protein complexes, as well as performing multi-chain protein folding and inverse folding tasks;
Experiments in specific functional protein design tasks show that APM outperforms the SOTA baselines in an- tibody and peptide design with higher binding affinity.
All-Atom Representation. In protein design with all-atom structures, the fundamental challenge lies in how to effec- tively represent atomic structures as different amino acid types have distinct atomic types, numbers, and basic struc- tures. When the protein sequence is not determined, the representation of its atom-level structure directly influences the modeling approach. Chu et al. (2024b) utilized an ensemble-based method to model the sidechain coordinates of all amino acid types simultaneously. Martinkus et al. (2024) represented sidechain structures by merging non- rotatable atoms into virtual atoms. Qu et al. (2024) fol- lowed the method used in Alphafoold3 (Abramson et al., 2024) to model all-atom coordinates directly. We choose to enhance residue-level information with the sidechain for all-atom protein representation that includes amino acid type, backbone structure, and the sidechain conformation parameterized by four torsion angles. This approach main- tains computational efficiency while supplying aevel information for modeling inter-chain interactions.
2 Related Work
Protein Foundation Models. The breakthrough achieve- ments in protein structure prediction, marked by AlphaFold series (AlphaFold1-3 (Senior et al., 2020; Jumper et al., 2021; Abramson et al., 2024) and RoseTTAFold (Baek et al., 2021; Krishna et al., 2024), have revolutionized the field of protein science. With these developments, protein language models (Rives al., 2019; Madani et al., 2020) have emerged as powerful tools. The series of ESM (Rives et al., 2019; Lin et al., 2023; Hayes et al., 2025), trained on large-scale protein sequence data, have demonstrated remarkable capabilities in protein understanding and gen- cration. Mcanwhile, certain methods considered protein design workflow in two stages: REdiffusion (Watson et al., 2023) tackles backbone structure generation using diffusion models, while ProteinMPNN (Dauparas et al., 2022) spe- cializes in sequence design through message-passing neural networks. FrameFlow (Yim et al., 2024; 2023b) and Fold- Flow (Bose et al., 2023) present developments applying SE(3) flow matching approaches to protein structure genera- tion. FoldFlow2 (Huguet et al., 2024) further demonstrates the integration of protein language models for structure generation. Besides, Chroma (Ingraham et al., 2023) intro- duces unified approach to protein design through a genera- tive model that can directly sample novel protein structures and sequences while being conditioned to target specific properties and functions The field has also seen approaches like Multiflow (Campbell et al., 2024), ProteinGenerator (Lisanza et al., 2024), and Protpardelle (Chu et al., 2024a),
Satuennc--ttuuture Dependency. The strong dependency between protein sequence and structure is the foundation for the success of folding and fnverse-folding models. However, in the joint generation of protein structure and sequence, this dependency is disrupted during the independent nois- ing process of each modality. This issue hampers effective learning of the dependency between sequence and structure. In APM, two strategies are implemented to enhance the dependency between the sequence and structure modalities. First, we decoupled the noising process for sequences and structures so that the noising level for each modality does not completely align, minimizing disruption of their depen-
2

---

An All-Atom Generative Model for Designing Protein Complexes
dicres Fow mascing
Refine Module
Module
Eom THELIII
Sidechain Module
APM
Random Initiliiztion
Denoising Repests
Protein with All-Atom Structure
Sequecne
Protein Language Model
Structure Module
Res Repr
Res
Seq& BB
Module
Chain ID
Interface
Resique Encoder
Repr
IPA
Transformer Encoder
Pair
SE(3) Transform
Module
Structure Module
Soq Pred Net
Logics'
Pain-Residue Encoder
Pair Repr
Repr
Structure
Structure'
Prozein Language Model
Structure Module
Ros Repr
Sidechain Module
Sequecne
Resique Encoder
Res Repe
Transformer Encoder
Res Repr
Structure Module
Structure
Tor Pred Nec
+
blocks
Pair Repr
Structure
Pain-Residue Encoder
Repr
Loigits'
Loights
Sequeerne'
Protein Languago Modal
Structure Module
ALoigis
Refine Module
Chain
Torsions
Residue Encoder
IPA
Transformer
Encoder
Res Repr
SE(3)
Module
Structure Module
Sog Mad
Pair Rep
Transform
Interface
Pair-Residue Encoder
Pair
Res Repr
Structure
Structure"
Figure Overview of APM. APM consists three modules: (1) A flow-matching hased Seq&BR Module for generating backhone structure and sequence simultancously; (2) Sidechain Module for generating the all-atom structure based on the previous module's generation; (3) A Re fine Module adjusts the sequence and structure with all atom information. The iterative denoising process enables the generation of multi-chain proteins with all-atom structure. The detailed architecture of each module presented below.
which enable generation of both sequence and structure. More recently, SaProt (Su et al., 2024a;b;c) and DPLM se- ries (Zheng et al., 2023; Wang et al., 2024a;b; Hsieh et al., 2025) have further advanced protein token modeling by incorporating structural information into the pre-training process, enabling better understanding of protein sequences. Functional Protein Design. Target-specific protein design has made remarkable advances recently. antibody design, approaches like HERN (Jin et al., 2022), DiffAb (Luo al., 2022), MEAN (Kong al., 2023a), and dyMEAN (Kong al., 2023b) have demonstrated the ability generate func- tional antibodies. Besides, Wu & Li (2024), Zhu et al. (2024), and Gao et al. (2023) introduced pre-trained protein language models as sequence priors to improve antibody de- sign. For peptide design, methods such as PPFlow (Lin et al., 2024), PepFlow (Li al., 2024), PepGLAD (Kong al., 2024) and CpSDE (Zhou et al., 2025) focus designing bioactive peptides.
3 APM
In this section, we present APM, an all-atom generative protein model for designing bioactive complexes with the all-atom structure. We first define how represent the all- atom structure in Section 3.1. Then introduce the model architecture of APM in Section 3.2, and in Section 3.3, we introduce the learning objective and training process. Finally, we introduce the sampling method in Section 3.4.
3.1 Representation for Protein All-Atom Structure In this study, we divide the goal of our approach into two parts: 1, foundational modeling of intra-chain sequences and structures (determining what constitutes a plausible protein sequence and structure); 2, modeling of inter-chain interactions (understanding how proteins interact with each other). Residue-level information is generally sufficient for intra-chain modeling of protein sequences and struc- tures. Methods such as AlphaFold2 (Jumper et al., 2021), ESMFold (Lin et al., 2023), and ProteinMPNN (Dauparas et al., 2022), which leverage residue-leva level information, have
3

---

An All-Atom Generative Model for Designing Protein Complexes
demonstrated high-quality protein structure modeling. How- ever, these models often require an additional relaxation step to resolve atomic clashes on the sidechains due to the lack of finer detail modeling. Therefore, we enhance residue-level information by incorporating sidechain conformations into the protein representation.
the generated proteins to make them more akin to natural proteins while resolving structural clashes.
When using APM for protein generation, the final result progressively generated from noise to data, with timestep from 0 to 1. Notably, the Sidechain Module and Refine Module are activated only after timestep 2 T (T = 0.8 here). We believe that the quality of proteins produced by the Sidechain Module with far from data time (1) is insufficient to support high-quality predictions by the Sidechain n Module and causes the meaningless in the refinement by the Ref finene Module.
While modeling the all-atom coordinates provides the most detailed view of interactions, it considerably raises complex- ity and limits the ability to model longer proteins, especially for multi-chain proteins. Sidechains of amino acids are not entirely free in their structures but have some conforma- tions. Each amino acid has up four rotatable bonds in its sidechain while maintaining largely consistent atomic structure between these bonds. Therefore, combining amino acid type with sidechain torsion angles offers a comprehen- sive representation of sidechain conformation.
The motivation for designing this integrated architecture, rather than using a single model to directly generate pro- teins with all-atom structures lies in the incompatibility of training the sidechain prediction model with the sequence and structure flow-matching model. Two key reasons pre- vent us from doing this: (1) sidechain prediction requires real sequences and structures to obtain accurate sidechain conformation labels, while the fow-matching model uses noised sequences and structures as input; (2) while it is possible to generate sidechain conformations using a flow- matching approach rather than packing model (one-step prediction). this would require providing a noised sidechain conformation, Xx, which still contains amino acid type infor- mation, leading to sequence information leakage (details in Appendix C). This is evidenced by the rapid convergence of amino acid type loss during training. During sampling, the absence of truly noised torsion angles, Xe= significantly degrades the model's performance in the inference phase. With the generation of sequence & backbone structures sep- arated from sidechain conformations, an additional module is necessary to allow all-atom information to influence the design of the backbone structure accordingly. For this pur- pose, we developed the third module of APM, the Re f ne Module. It receives outputs from both the Seq&BB Module and S Si didecain Module, making it all-atom aware. Based on this comprehensive information, the Refine Module further optimizes the sequence and backbone structure to ensure the overall structure more closely resembles natural proteins.
Ultimately, we adopted a representation that includes amino acid type, backbone structure, and sidechain torsion an- gles. This approach maintains computational efficiency while providing richer information for modeling inter-chain interactions.
The all-atom protein structure is represented as a collection of amino acid types, backbone frames, and sidechain torsion angles (Jumper et al., 2021; Lehninger et al., 2005). A multi-chain protein complex P com- posed of K chains and N = Σx kN residues in total. For the k-tth chain, the amino acid sequence is denoted ......
Notations.
as Ss = =Is skk,,.., s**], where Sk E A and A is the set of 20 standard amino acids. Meanwhile, the back- bone structure of this chain is characterized by rigid frames Tpxx where each Thk E SE(3) consists of rotation R E SO(3) and translation vec- tor x E RR, mapping rigid transformations from ideal peptide geometry (Engh & Huber, 2006). The sidechain torsion angles are denoted as X = IxE", x .x...... **)), k. E [0, 2xr corresponds to the torsions of rotat- where X
TE = T T
able bonds in the sidechain of i-th residue. For brevity, we slightly abuse the notation such that S = U₂ S T = Us Tv. X = Us Xk. where chain indices (i.e., k) are omitted hereafter unless needed.
3.2.2 SUB-MODULE ARCHITECTURES
3.2 An Integrated Architectural Design of APM
The core structure of the three submodules is essentially the same. We use stacked structure modules derived from AlphaFold2 as the trunk of APM. Each structure module is composed of IPA (Jumper et al., 2021) and a Transformer Encoder, which is employed to update residue informa- tion and pair-residue information. The differences among the submodules lie in the encoding of the input and the distinctions in the output, driven by the various model- ing tasks. Apart from this, the Seq&BB Module and the Refine Module maintain consistent model sizes, whereas the Si Sicchinn Module has fewer structure module blocks and a smaller hidden dimension. We believe that predict-
3.2.1 OVERALL ARCHITECTURE
To implement an All-Atom Protein Generative Model, we designed the APM consisting of three distinct modules: the Seq&BB Module, the Sidechain Module, and the Refine Module (Figure 2). The Seq&BB Module is a flw-matching-basse protein generative model that handles the co-generation of sequence and structure at the residue level. The Sidechain Module serves as an all-atom com- pletion model, predicting the sidechain conformations for proteins generated by the SB Module. The Re fine Module an All-Atom Protein Refinement model, refining
4

---

An All-Atom Generative Model for Designing Protein Complexes
Training Phase I
PDB- S8--SnueChhii
FDB MuriChain
Inverse-folding
ArDB-pl DDTats
Swiss-Poospp. DDDTst
Seq & BB Module
Folding
Sidechain Module
Uncond-generation
Cond-generation
noising -
Data
Training Phase II
k<t
Seq & BB Module
Sidechain Module
Refine
tast
Module
Figure 3. The two- two-phase of the training process of APM. In training phase the Seq&BB Module and Sidechain Module are trained separately. In training phase II, the three modules form the integral APM, and are trained an iterative paradigm. In any phase, the training data is mixture of PDB (Berman al., 2000) single/multi-chann proteins, Swiss-Prot proteins, and AFDB (Varadi et al., 2022) proteins.
ing sidechain conformations a relatively simple task, and utilizing a smaller model for enhancing efficiency.
or structure, or both, from a noisy state. As we decou- ple the noising processes of the two modalities, we de- note the noised sequence as Stg and the noised structure as Tzy, where ts; tr U(0, 1) represent intermediate time steps. We also denote the original sequence and structure as S and T₁. Then the learning objectives of Seq& BB Module are: Pseqess (S). T1 Stt. Ter: Ls, br) for uncon- ditional generation; Psequs ( [S] [Stg. T ts)) for inverse- folding: Pseqsas ( T tr) for folding.
3.2.3 INTEGRATION OF PROTEIN LANGUAGE MODEL
A robust understanding of protein sequences requires a large- scale model trained on tens of millions of sequence data (like the 3B-parameters ESM2 in ESMFold, which is trained on 65 million unique sequences, is responsible for sequence understanding), or alternatively, using MSA as the sequence representation (like AlphaFold2/3). APM is trained on pro- tein data with structural information, yet the available vol- ume of such data is not sufficient to support learning the intricacies of protein sequence understanding. To address this, we integrated protein language models (PLMs) into all modules to enhance protein sequence understanding.
For each of the three tasks, there is a conditional version for multi-chain data, in which part of the target modalities is set as the noiseless state. The flow-matching loss Laow www.mmnchingg is defined over sequence and backbone structure as:
Cnow-matching -discrete + LSF(3)
We utilized ESM2-650M, the widely adopted protein lan- guage model, to represent the input sequences. Drawing from ESMFold's approach, we used learnable weights to aggregate the representations from each layer of the protein language model, yielding the final amino acid encodings. It's important to note that ESM2 is only trained on single- chain data. Therefore, when encoding multi-chain proteins with ESM2, each chain is encoded individually.
It consists of two components. For the sequence, Laiscrete measures the cross-entropy between the predicted sequence distribution Pseysna (S₁ [Stg T,,) against the true sequence Sj. For the structure, it is the mean squared error between the vector fields calculated from the noisy structure Ter to the generated structure T, and to the true structure Tç, For the complete mathematical formulation, refer to Ap- pendix A.
3.3 Training of APM
The refinement by the Refine Module can be consid- ered as posterior correction, where the generated protein ($1, Tí) at each sampling step is corrected to (S1. Tı) be- fore being noised for the next step. We hope to ensure that the direction of correction each step is as consistent as possible. By achieving this, the corrections each step can accumulate, leading improved performance. This con- sistency necessitates that the predicted (S1, TT) at each (Is, (T) is aligned, which in turn requires the Seq& BB Module to maintain smoother generative trajectory. To fa- cilitate this, we incorporated a consistency loss, Econsistency- into the SBq&BB Module to minimize the variations be- tween predictions for adjacent (details in Appendix B.
In order to train APM with the integrated architecture, we designed a two-phase training approach (Figure 3). In phase I, Seq&RR RB Module and S decha in Module are trained separately. In phase II, the three modules are joint-trained in an iterative paradigm. All the details refer to Appendix B.
3.3.1 TRAINING OF $32$BB MODULE
BB Module is the foundation model in APM to gener- ate the sequence and backbone structure trained in a flow- matching manner with tasks of unconditional generation, conditional generation, folding, and inverse folding. The primary learning objective is reconstructing either sequence
5

---

An All-Atom Generative Model for Designing Protein Complexes
The final training loss of Seq& BB Module is defined as:
The ultimate training loss of APM is defined the expecta- tion over timesteps and each residue in the protein:
EeeqSBB moow-matching + 0.3 x Cconsistency
C (0. [CsegeBB + Lracking + CRefine]
The training loss is consistent in two phases, the only differ- ence is the learning rate.
3.4 Sampling Strategy
3.3.2 TRAINING OF STDECHATH NN MODULE
For structure sampling, we directly use the structure pre- dicted by Seq&BB Module or the Ref tnfine! Module cor- rected one as the model output if it is activated.
The learning objective of si didechaii in Module is predict the sidechain torsions, X. given protein with sequence and backbone structure. In the training phase I, the learning objective is packing, PSidechain (x(S1, T1), which is no dif- ferent from the normal packing model. While in phase II, the learning objective is switched to PSidechain (x|S,, TT), which means the reconstruction of the ground truth sidechain from the predicted ($1, TT). Besides, we also want S Sidechain Module to keep the ability of packing, there is 50% probability that packing will continue to be used as the learning objective in phase II.
For sequence sampling, the strategy is different. To fully leverage the Protein Language Model, we update all the residues at each inference step and only keep the residues located in the positions with top max max(log(prob)), For the top K positions (where is the number of amino acids be unmasked at the current t), we sample the amino acid types based the corresponding logits with carefully designed strategy composed of temperature annealing sampling and arg max. The remaining positions are set to [MASK] token (details refer to Appendix D.I). This decoding strategy also led us to abandon the Now-matching training approach for the S Sidechain Module, as the amino acid type at each position may change during the sampling process, making it inappropriate for the Sidechain Module to rely on the sequence from the previous step for prediction.
Training loss of Sidechain in Module is also different for each learning objective. For packing, the loss consists of supervised torsion angle loss and all-atom Frame Aligned Point Error (FAPE) loss (Jumper et al., 2021), is defined as:
Lracingg = Lxx+ + LEAPE
4 Experiments
For reconstruction, we only maintain torsion angle loss, fxx, as the input protein sequence and structure may not match the ground truth, resulting in the inappropriateness for calculating error on all frames (details in Appendix B.2).
4.1 Data Curation
Single-chain data is built from three sources: PDB (Berman et al., 2000), Swiss-Prot (Boeckmann et al., 2003), and AFDB (Varadi et al., 2022). For PDB samples, we followed the data processing flow in MultiFlow, resulting in 18684 samples. For Swiss-Prot samples, we selected the samples with a pLDDT (Jumper et al., 2021: Mariani et al., 2013) greater than 85, resulting in 140769 samples. For AFDB samples, take more rigorous filter, leaving samples with PLDDT greater than 95, which resulted in 28041 samples. Finally, we got 187494 single-chain samples. Multi-chain data is built from PDB Biological Assem- blies (Rose et al., 2016). To prevent potential information leakage in downstream tasks, we discarded samples that met any of the following conditions: (1) the sample's PDB ID is present in SAbDab (Dunbar et al., 2014); (2) the sample contains at least one chain with length less than 30, which is considered a peptide (Kong et al., 2024; Tsaban et al., 2022). The last condition led to the removal of substantial number of samples (12,163). In many cases, the peptides played crucial roles in stabilizing the complexes and only removing peptides is unreasonable. Consequently, we opted to exclude this subset of data entirely from training. We also removed samples with lengths exceeding 2048 or lacking cluster IDs. Finally, we got 11620 multi-chain samples. Cropping. During the training process, we performed random cropping (Evans et al., 2021) on multi-chain sam- ples with residues exceeding 384 to prevent out-of- -emooo.
3.3.3 TRAINING OF REF I NE MODULE
The Refine Module PRefine (S,, T, ts.ts, X) is tasked to predict the real protein based the generated one, ($1, T,), with the all-atom level information formed with the predicted sidechain torsions X. Thus, we define a correction loss on the corrected protein (S1. as the learning objective as:
Lcom - log (0 (1)) ||= x1 + || R l|
Besides, we also incorporate auxiliary objectives in the train- ing of Refine ne Module, including backbone FAPE loss, CBB-FAPE, and residue distogram prediction loss, Ldist (de- tails in Appendix B.3). Finally, the training loss of fine Module is defined as: LRefine Lear + 0.25 X LBB FAPE +
0.25x X Lais.
3.3.4 TRAINING IN PHASE II
In the second phase of the training, we did not train the three modules simultaneously as each module requires distinct range. Instead, we employed an iterative approach for train- ing these three modules. Given that the modules of Seq&BB and Si sidechain were already trained in phase I, they are only trained for steps in each iteration in phase II, whereas the Refine module requires steps of training. Each cycle comprises 12 steps, with the steps of 2-2-8 respectively.
6

---

An All-Atom Generative Model for Designing Protein Complexes
Table 1. Performance comparison of protein folding (blue high-
Table 3. Performance comparison in multi-chain protein folding (blue highlighted) and inverse- tasks (pink highlighted).
lighted) and inverse-folding tasks (pink highlighted), For each metric, we report the average/median performance.
Method
RMSD
TM 1
scTM AAR(%)
Method
RMSD 4
TM
sTM A
ppl 4
Boltz-1 w/MSA
5.40/1.95
0.87/0.97
ESMFold
2.84/1.99
0,93/0.97
Boltz-1 w/MMSA
17.86/18.43
0.44/0.45
ProteinMPNN
0,94/0977 46.58/46.76
11.44/11 1.48
ProteinMPNN
0.90/0.96 46.17/46.37
ESM3(1.4B)
4.71/2.27
0.83/0991
0.94/0.97
49.50/49.42 42
8.64/7.90
APM
12.6/13.67 0.64/0.22 0.85/0.95 61.26/59.48
MultiFlow*
15.64/16.08
0.53/0.49
0.94/0.96
37. 37.74/37.59
10.86/10.94
4.3 Multi-Chain Protein Related Tasks
APM
4.83/2.64
0.869091 0.94/0.97 50.44/50.41 8.74/8.10
4.3.1 FOLDING & DWVERSE-FOLDIN
Table 2. Performance comparison of different methods for various protein lengths. We evaluate the methods on three different length ranges (100, 200, 300) using SCTM and SCRMSD.
We also initially examined APM's capabilities in modeling multi-chain proteins through folding and inverse- swwrse-folding tasks. In these tasks, we used samples missing cluster IDs that were dropped during training as the test set, and we also removed samples exceeding a length of 512. The final test set comprised 273 proteins with number of chains of 2-6. Furthermore, in the two tasks, we only compared APM with two specialized models, Boltz-1 (Wohlwend et al., 2024) and ProtcinMPNN, as there are almost no other models that support multi-chain proteins. For the inverse-folding task, we employed Boltz-1 with MSA to refold the predicted sequences for calculating s STM. As depicted in Table 3, folding for multi-chain proteins represents an extreme chal- lenge. Even with the use of MSA, Boltz-1 1 exhibits a decline in prediction accuracy compared to single-chain proteins. Without MSA, achieving effective prediction becomes con- siderably more difficult. Although the performance of APM also degrades, it still surpasses that of Boltz-1 without MSA. Conversely, APM exhibits commendable performance in unwerse-folding for multi-chain proteins, with the SCTM nearly matching the folding performance of Boltz-1 when using reference sequences.
Length 100
Length 200
Length 300
Method
scTM
scRMSD
scTM
scRMSD
sc
scRMSD
NativePDBs
0.91
2.98
0.88
3,24
0.92
3.94
ESM3(1.4B)
0.72
13,80
0.63
21.18
0.59
25.5
MultiFoow
0.86
4.73
0.86
4.98
0.86
6.01
ProteinCieneratorr
0.91
3.75
0.88
6.24
0.81
9.26
ProtPardelle
0.56
12.90
0.64
13.67
0.69
14.91
APM
0.96
1.80
0.89
4.25
0.87
5.96
Cropping was centered around the randomly selected inter- chain residue pair at the binding interface, retaining the 384 amino acids nearest to the pair. PLM encodes the sequence of cropped samples before cropping.
4.2 Single-Chain Protein Related Tasks
While APM is specifically designed for modeling multi- chain proteins, it also possesses the capabilities of those foundation models designed for single-chain proteins, in- cluding folding and rnverse-folding We validated the fold- ing and fnverig-folding capabilities of APM on a PDB date split used by MultiFlow. We compared it with specialized models, including ESMFold and PorteinMPNN, as well as co-design models capable of performing multiple tasks, in- cluding ESM3 and mouttillo*"(withoor distillation). We utilize RMSD and TMscore (Zhang& & Skolnick, 2005) be- tween predicted and ground truth structures to evaluate fold- ing performance, and self-consistency (Trippe et al., 2022) TMscore (scTM), amino acid recovery (AAR) and perplex- ity to evaluate inverse-folding performance. The perplexity (pp1) is provided by ProGen2-base (Madani et al., 2020; Nijkamp et al., 2023). The results are shown in Table 1. APM can also perform unconditional protein generation. Besides ESM3 and MultiFlow* we compared two meth- ods capable of all-atom design, ProteinGenerator and Prot- Pardelle. For this task, we followed the evaluation methods in ProteinBench (Ye et al., 2024) and presented the average a SCRMSD and s SCTM for proteins with lengths of 100-300 in Table 2. APM achieved competitive performance compared to other co-design methods all three tasks.
4.3.2 MULTI-CHAIN PROTEIN GENERATION
The most significant difference between the APM and pre- vious methods is its ability to directly generate multi-chain protein complexes. These generated complexes do not re- quire the starting amino acids of each chain to be spatially close to the previous chain. Instead, they have indepen- dent spatial positions, yet each chain possesses precise and complementary binding interfaces with the others. How- ever, evaluating these generated complexes poses challenges. Calculating the self-consistency of the complexes does not accurately assess APM 's capabilities in complex generation as folding models cannot reliably predict structures. Evalu- ating each single-chain independently for self-consistency is also not entirely appropriate because single-chain pro- teins may undergo significant conformational changes upon binding, such as 1AKE to 4AKE, or some might only fold correctly in the presence of other proteins.
As APM has demonstrated its single-chain protein gener- ation capability in Section 4.2. We focus on the binding affinity between each chain in multi-chain proteins in this task. We use AG to represent the binding strength between
7

---

An All-Atom Generative Model for Designing Protein Complexes
Table 4. The inter-chain binding affinity between generated com- plexes. For each metric, we report the average/median value. APMss means using APM in a residue-level manner by only acti- vating the SequBB Module. We additionally use ProteinMPNN to redesign sequences for Chroma. (marked with *)
Longth
Model
AGasc
AGRAA
RMSDD
Chroma
133.64/46.51
$3.96/-86.66
1.33/1 .22
50-100
Chroma*
-27.53/-41.71
-78.41/-77.09
1 1.4/1.28
APM
-72.44/-71/91 -112.65/-116698
1.05/0.95
APM
-64.30/667.30
-114:94/-114.45
1,06/1.03
Chroma
89.47/22.97
102.33/48.34
1,46/1.39
Length Length 100
Length 100
Length 100
100-100
Chroma*
-31.09/311.51
162.15/-59.36
1,40/1330
-77.74; -97.80; 0.72
-89.74; -121.33; 1.28
-71.44; -109.18: 0.87
APM
91.611-94.54 -130.31-13457
1.04/.944
APMBB
36.74/169930
-117.53/-118.13
1.17/1.12
Figure 4. Showcases of the three length combinations. For each case, the gray structure represents APM's generated structure and the colored structure represents the backbone relaxed structure. Different chain is highlighted with different colors. We also report the two AG and the RMSD between the two structures.
Chroma
79.97/35.86
-59.32/-54.30
1,58/1.48
100-200
Chroma*
-32.14/-31.99
-62.79/-59.77
1,58/1,,39
APM
-44.02/39.42
-93.211-73.09
1.35/1.21
APMBB
-3,42/-33.71
-85.79/-69.12
1,58/1.42
two chains of multi-chain proteins with lengths of 50-100, 100-100, and 100-200 (the generation of more chain combi- nations is shown in the Appendix D.3). We report two types of AG: AGRSC, for sidechain-only relaxed complexes; and AGRAA, for all-atom relaxed complexes (both relaxation and AG calculation are performed by pyRosetta (Alford al., 2017; Chaudhury et al., 2010)). Additionally, we report the RMSD between the two structures. The average/median results are shown in Table 4. For comparison, we utilized Chroma (Ingraham et al., 2023) to sample unconditional complex with the same length combinations. Besides, we also performed the same task with only Seq&BB Module activated (APMgB), which means generated multi-chain proteins at residue-level. As shown in Table 4, compared with using all-atom information, APMgB achieved weaker binding strength and higher RMSD, which proves the im- portance of the all-atom information in the inter-chain interactions modeling.
about SFT refer to Appendices B.4, D.4 and D.5.
4.4.1 ANTIBODY CDR-H3 CO-DESIGN
Setup. The design of Complementarity Determining Re- gions(CDRs) is a crucial step in developing potent thera- peutic antibodies, especially CDR-H3. Following the data preprocessing pipeline introduced in (Ye et al., 2024; Zhou et al., 2024), we conduct training on the Structural Antibody Database (Dunbar et al., 2014) and perform evaluation on the RAbD benchmark (Adolf-Bryfogle et al., 2018).
We compare our model with four antigen-specifte anti- body design methods (dyMEAN (Kong et al., 2023b), Dif- fAb (Luo et al., 2022), AbDPO (Zhou et al., 2024) and its variant AbDPO++). Following previous works (Ye et al., 2024), we use multiple metrics to evaluate the qual- ity of designed CDRs: AAR and C. RMSD for generated sequence and backbone structure; Total Energy (E) and Bi Binding Energy (AG) for atomic rationality and functionality, which are provided by pyRosetta.
By default, APM generates all chains simultaneously, Un- der this manner, we observed that for complexes with chain lenghh combinations of 50-100 and 100-100, APM tends to generate multi-chain proteins in a "single-chain protein mode". leading to strong inter-chain interactions. Con- versely, in complexes with chain lengths of 100-200. APM generates two relatively independent single-chain proteins that are bound tightly, resulting in normal binding energies (Figure 4). APM also supports an alternative generation manner called chain-by-chain, raised from the conditional generation task in APM's training. The chain-by-chain ap- proach yields significantly different results, with each chain appearing fold independently before ultimately binding together. Related results can be found in Appendix D.3.
Results. As indicated in Table 6, APM performs signifi- cantly superior to other methods in all metrics. With re- spect to accuracy related metrics, including AAR and RMSD, APM's superior performance demonstrates its capability in generating antibodies that resemble natives. On rational and functional metrics, APM's generated antibodies exhibit the highest rationality and binding capability. Additionally, antibodies generated in zero-shot manner display excellent AG, proving APM's capacity in inter-chain interacted pro- tein generation, while the abnormal AAR/RMSD highlight the different binding patterns between general proteins and antibodies (refer to Appendix D.4 for details).
4.4 Downstream Tasks
4.4.2 PEPTIDE DESIGN
We further verify the capacity of APM on specific tasks including antibody design and peptide design, in both su- pervised fine-tune tne-tunnng (SFT) and zero-shot manner. Details
Setup. The design of functional and binding peptides plays a crucial role in pharmacological applications and targeted therapeutic development. To evaluate our APM's perfor-
8

---

An All-Atom Generative Model for Designing Protein Complexes
Table Comprehensive evaluation of peptide design methods across three key aspects: Functionality, Foldability, and Accuracy. The best
results are highlighted in bold. 5 out of 93 ground truth samples exhibit AG greater than O, are visualized in Appendix D.5.
Functionality
Foldability
Accuracy
Method
AG 4
% <
PLDDT1
ipTM t
Success
DockQ f
20.8 f
Truth
-24.34
94.62
88.31
0.94
100.00%
1.00
100.00
PPFlow
-8.56
16.72
55.72
0.57
13.01%
0.27
0.00
DiffPP
-12.40
38.17
55.10
0.57
16.55%
0.33
0.90
PepGLAD
-12.45
37.10
51.69
0.57
12.50%
0.35
0.00
RFDiffusion
-23.27
78.58
69.65
0.73
46.28%
0.28
0.00
APMSFT
-19.90
69.34
60.36
0.66
29.22%
0.40
11.29
APMzeroshot
-23.71
62.18
60.97
0,62
27.20%
0.24
0.12
accuracy, APM stands out as nearly the only method capable of generating peptides with a Docko score exceeding 0.8. Additionally, the peptides generated by APM a zero-shot manner perform similarly to antibodies, with high affinity but do not resemble natural ones. This is reflected in the comparable performance in functionality & foldability and the significant degradation in accuracy. We also present more results on longer binder design in Appendix F.
Table 6. Performance comparison of antibody design methods on RAbD henchmark. The best results are shown in bold.
Method
AAR f
RMSD
AG +
RAbD
100.00
0.00
-16.76
-15.33
dyMEAN
40.05
2.36
1239.29
612.75
DiffAb
35.04
2.53
495.69
489.42
ABDPO
31.29
2.79
270.12
116.06
ABDPO++
36.25
2.48
338.14
223.73
APMSFT
41.20
2.08
137.74
91.64
5 Discussions
APM shoe
28.35
5,81
284.24
81.12
In this paper, we introduce APM, a generative model for pro- tein complexes designing at all-atom level. APM is capable of generating tightly bound protein complexes, executing high-quality single-chain protein-related tasks, and achiev- ing remarkable performance in specific functional protein design tasks. Despite APM's potential in Al-based func- tional protein design tasks, several limitations remain to be addressed. These limitations are primarily in these as- pects: (1) the performance in folding tasks requires further improvement; (2) the functionality of Refine Module is relatively restricted; (3) the number of downstream tasks is limited. Future works are detailed in the Appendix G.
mance in receptor-targeted peptide design, we use the Pep- Bench (Kong et al., 2024) dataset for training and validation and use the LNR (Tsaban et al., 2022) dataset as the test set. We compare our model with: PepGLAD (Kong et al., 2024), PPFlow (Lin et al., 2024), and DimP (Lin et al., 2024). We also include RFDiffusion (Watson et al., 2023), which utilizes ProteinMPNN for sequence design. The peptide candidates are comprehensively evaluated across three key aspects: Functionality, Foldability, and Accuracy. For functionality, we evaluate the binding energy (AG) and the proportion of candidates with AG below zero, % <0. For foldability, we use Boltz- (WWMA)) to fold sequences of generated peptides, then evaluate the folded structure in two confidence metries: predicted Local Distance Difference Test (PLDDT) and interface predicted Template Modeling (ipTM) (Zhang& & Skolnick, 2004; Xu & Zhang, 2010) score. We also report a comprehensive metric, defined as the pro- portion of candidates with both PLDDT score > 70 and an 1pTM score 0.8, donated Success. For accuracy, we evaluate the DockQ (Basu & Wallner, 2016; Mirabello & Wallner, 2024) score and the proportion of candidates achieving a DockQ score of at least 0.8 (% 20.8), which is the threshold considered as high-quality.
Impact Statement
Our work on multi-chain protein generation can be used in developing potent therapeutic macromolecues such as antibodics and accelerating the research process of drug discovery. Our method may be adapted to other scenarios of computer-aided design, such small molecule design, material design, and chip design. It is also needed to ensure the responsible use of our method and refrain from using it for harmful purposes.
A Acknowledgements
Results. As indicated in Table 5, APM exhibits competitive performance across all three key aspects. For functionality, APM generates peptides with an average binding energy of -99.90 and achieves negative AG in 69.34% samples, sig- nificantly outperforming other methods. The performance in foldability metrics demonstrates APM's superiority in sequences generation, with the high pLDDT and ipTM. For
We thank anonymous reviewers for their insightful feedback. We would like to especially thank Dr. Hang Li for insightful discussions on the project and feedback on the manuscript that help shape this study. We thank Yi Zhou, Yuning Shen, Xinyou Wang, Yiming Ma, Fei Ye, and Lihao Wang for their valuable comments.

---

An All-Atom Generative Model for Designing Protein Complexes
References
Enabling multimodal flows with applications to protein co-design. In ICML., 2024.
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L.,, Ballard, A. J., Bambrick, J., et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 2024.
Chaudhury, S., Lyskov, S., and Gray, J. J. Pyrosetta: script- based interface for implementing molecular modeling algorithms using rosetta. Bioinformatics, 2010.
Adolf-Bryfogee J., Kalyuzhniy, O., Kubitz, M., Weitzner, B. D., Hu, X., Adachi, Y., Schief, W. R., and Dunbrack Jr, R. L. Rosettaantioodydesign (rabd): A general framework for computational antibody design. PLoS computational biology, 2018.
Chen, R. T. Q. and Lipman, Y. Flow matching general geometries. In ICLR, 2024.
Chu, A. E., Kim, J., Cheng, L.., EI Nesr, G., Xu, M., Shuai, R. W., and Huang, P.-S. An all-atom protein generative model. Proceedings of the National Academy of Sciences, 2024a.
Alamdari, S., Thakkar, N., van den Berg, R., Tenenholtz, N., Strome, B., Moses, A., Lu, A. X., Fusi, N., Amini, A. P., and Yang, K. K. Protein generation with evolutionary diffusion: sequence is all you need. BioRxiv, 2023.
Chu, A. E., Kim, J., Cheng, I.., El Nesr, G., Xu, M., Shuai, R. W., and Huang, P.-S. An all-atom protein generative model. Proceedings of the National Academy of Sciences, 2024b.
Alford, R. F., Leaver-Fay, A., Jeliazkov, J. R., O'Meara, J. DiMaio, F. P., Park, H., Shapovalov, M. V., Renfrew, P. D., Mulligan, V. K., Kappel, K., et al. The rosetta all- atom energy function for macromolecular modeling and design. Journal of chemical theory and computation, 2017.
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., and Ré, C. FlashAt- tention: Fast and memory-efficiet exact attention with IO-awareness. In Advances in Neural Information Pro- cessing Systems (NeurIPS), 2022.
Baek, M., DiMaio, F., Anishchenko, I. Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q.. Kinch, L. N., Schaeffer, R. D., et al. Accurate prediction of pro- tein structures and interactions using a three-track neural network. Science, 2021.
Dauparas, J., Anishchenko, I., Bennett, N., Bai, H., Ragotte, R. J., Milles, L. F., Wicky, B. I., Courbet, A., de Haas, R. J., Bethel, N., et al. Robust deep learning- -baend pro- tein sequence design using proteinmpon. Science, 2022.
Basu, S. and Wallner, B. Dockq: a quality measure for protein-protein docking models. PloS one, 2016.
Dunbar, J., Krawezyk, K., Leem, J., Baker, T., Fuchs, A., Georges, G., Shi, J., and Deane, C. M. Sabdab: the structural antibody database. Nucleic acids research, 2014.
Bennett, N. R., Watson, J. L., Ragotte, R. J., Borst, A. J., See, D. L.., Weidle, C., Biswas, R., Yu, Y., Shrock, E. L., Ault, R., et al. Atomically accurate de novo design of antibodies with rfdiffusion. bioRxiv, pp. 2024-0224333 03, 2025.
Eastman, P., Swails, J., Chodera, J. D., McGibbon, R. T.,
Zhao, Y., Beauchamp, K. A., Wang, L.-P., Simmonett, A. C., Harrigan, M. P., Stern, D., et al. Openmm 7: Rapid development of high performance algorithms for molecular dynamics. PLoS computational biology, 13(7): e e1005659, 2017.
Berman, M., Westbrook, J., Feng, Z., Gilliland, G., Bhat, T. N., Weissig, H., Shindyalov, N., and Bourne, E. The protein data bank. Nucleic acids research, 2000.
Boeckmann, B., Bairoch, A., Apweiler, R., Blatter, M.-C., Estreicher, A., Gasteiger, E., Martin, M. J., Michoud, K., O'Donovan, C., Phan, I., et al. The swiss-prot pro- tein knowledgebase and its supplement trembl in 2003. Nucleic acids research, 2003.
Engh, and Huber, R. Structure quality and target parame- ters. 2006.
Evans, R., O'Neill, M., Pritzel, A., Antropova, N., Senior, A., Green, T., Židek, A., Bates, R., Blackwell, S., Yim, J., et al. Protein complex prediction with alphafold mutttier.. biorxiv, 2021.
Bose, J., Akhound-Sadeghh T., Huguet, G., FATRAS, K., Rector-Brooks, J., Liu, C.-H., Nica, A. C., Korablyov, M., Bronstein, M. M., and Tong, A. Se (3)-stochastic flow matching for protein backbone generation. In The Twelfth International Conference Learning Representations, 2023.
Fred Zhangzhi Peng, P. C. and contributors. Faesm: An efficient pytorch implementation of evolutionary scale modeling (esm). pengzhangzhi/ faesm, 2024. Efficient PyTorch im- plementation of ESM with FlashAttention and Scalar Dot-Product Attention (SDPA).
//ppt:///ithub com/
Campbell, A., Yim, J., Barzilay, R., Rainforth. T., and Jaakkola, T. Generative flows on discrete state-spaces:
10

---

An All-Atom Generative Model for Designing Protein Complexes
Gao, K., Wu, L.,, Zhu, J., Peng, T., Xia, Y., He, L.., Xie, S., Qin, T., Liu, H., He, K., et al. Pre-training antibody language models for antigen-specific computational anti- body design. In KDD, 2023.
Kong, X., Huang, W., and Liu, Y. End-to-end full-atom antibody design. In ICML,, 2023b.
Kong, X., Jia, Y., Huang, W., and Liu, Y. Full-atom peptide design with geometric latent diffusion. In NeurlPS, 2024.
Guo, H.-B., Perminov, A., Bekele, S., Kedziora, G., Fara- jollahi, S., Varaljay, V., Hinkle, K., Molinero, V., Meister, K. Hung, C., et al. Alphafold2 models indicate that pro- tein sequence determines both structure and dynamics. Scientific reports, 12(1):10696, 2022.
Krishna, R., Wang, J., Ahern, W., Sturmfels, P., Venkatesh, P., Kalvet, I., Lee, G. R., Morey-Burrows, ishchenko, I., Humphreys, I. R., et al. Generalized biomolecular modeling and design with rosettafold all- atom. Science, 2024.
S., An-
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. Simulating 500 million years evolution with a language model. Science, 2025.
Lefranc, M.-P., Pommié, C., Ruiz, M., Giudicelli, V., Foulquier, E., Truong, L., Thouvenin-Conte,, V., and Lefranc, G. Imgt unique numbering for immunoglobulin and t cell receptor variable domains and ig superfamily v-like domains. Developmental & Comparative Immunol- ogy, 2003.
Henikoff, S. and Henikoff, J. G. Amino acid substitution matrices from protein blocks. Proceedings of the National Academy of Sciences, 19(22):10115-1091 1992.
Lehninger, A. L.,, Nelson, D. L.,, and Cox, M. M. Lehninger principles of biochemistry. Macmillan, 2005.
Hsieh, C.-Y., Wang, X., Zhang, D., Xue, D., Ye, F., Huang. S., Zheng, Z., and Gu, Q. Elucidating the design space of multimodal protein language models. arXiv preprint arXiv:2504. / 1454, 2025. URL https: //arxiv. wwg///g///....oooooo....e......e abs awtaabb//2550 11454.
Li, J., Cheng, C., Wu, Z., Guo, R., Luo, S., Ren, Z., Peng, J., and Ma, J. Full-atom peptide design based on multi- modal flow matching. ICML, 2024.
Lin, H., Zhang, O., Zhao, H., Jiang, D., Wu, L., Liu, Z., Huang, Y., and Li, S. Z. Ppflow: Target-aware peptide design with torsional flow matching. ICMI. 2024.
Hsu, C., Verkuil, R., Liu, J., Lin, Z., Hie, B., Sercu, T., Lerer, A., and Rives, A. Learning inverse folding from millions of predicted structures. (CML,2022
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmucli, Y., et al. Evolutionary-seat prediction of atomic-level protein structure with language model. Science, 2023.
Huguet, G., Vuckovic, J., Fatras, K., Thibodeau-Laufee E., Lemos, P., Islam, R., Liu, C.-H., Rector-Brooks, J., Schound--Saeghh T., Bronstein, M., et al. Sequence- augmented se (3)-flow matching for conditional protein backbone generation. Advances in neural information processing systems, 2024.
Lipman, Y., Chen, R. T., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In ICLR, 2023.
Ingraham, B., Baranov, M., Costello, Z., Barber, W., Wang, W., Ismail, A., Frappier, V., Lord, D. M., Ng- Thow-Hing, C., Van Vlack, E. R., Tie. S., Xue, V., Cowles, S. C., Leung, A., Rodrigues, J. a. V., Morales- Perez, C. L., Ayoub, A. M., Green, R., Puentes, K., Oplinger, F., Panwar, N. V., Obermeyer, F., Root, A. R., Beam, A. L., Poelwijk, J., and Grigoryan, G. Illuminat- ing protein space with programmable generative model. Nature, 2023. doi: 10.
Lisanza, S. L.,, Gershon, J. M., Tipps, W., Sims, J. N., Amollt, L., Hendel, J.. Simma, M. K., Liu, G., Yase, M., Wu, H. et al. Multistate and functional protein de- sign using rosettafold sequence space diffusion. Nature biotechnology, 2024.
Luo, S., Su, Y., Peng. X., Wang, S., Peng, J., and Ma, J. Antigen-specific antibody design and optimization with diffusion- based generative models for protein structures. NeurIPS, 2022.
Jin, W., Barzilay, D., and Jaakkola, T. Antibody-antigen docking and design via hierarchical structure refinement. In ICML, 2022.
Madani, A., McCann, B., Naik, N., Keskar, N. S., Anand, N., Eguchi. R. R., Huang, P.-S., and Socher, R. Progen: Language modeling for protein generation. arXiv preprint arXiv:2004.03499 2020.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnoy, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Židek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. nature, 2021.
Mariani, V., Biasini, M., Barbato, A., and Schwede, T. Iddt: a local superposition score for comparing protein structures and models using distance difference tests. Bioinformatics, 2013.
Kong, X., Huang, W., and Liu, Y. Conditional antibody de- sign as 3d equivariant graph translation. In /CLR, 2023a.

---

An All-Atom Generative Model for Designing Protein Complexes
Martinkus, K., Ludwiczak, J., Liang, W.-C., Lafrance- Vanasse, J., Hotzel, I., Rajpal, A., Wu, Y., Cho, K., Bon- neau, R., Gligorijevic, V., et al. Abdiffuser: full-atom generation of in-vitro functioning antibodies. NeurIPS, 2024.
Senior, A. W., Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L.,, Green, T., Qin, C., Zidek, A., Nelson, A. W., Bridg- land, A., et al. Improved protein structure prediction using potentials from deep learning. Nature, 577()2)
706-710, 2020.
McPartlon, M. and Xu, J. An end-to-end deep learning method for protein side-chain packing and inverse folding. Proceedings of the National Academy of Sciences, 120 (23):e2216438122) 2023.
Shi, C., Wang, C., Lu, J. Zhong, B., and Tang, J. Pro- tein sequence and structure co-design with equivariant translation. In ICLR, 2023.
Song, Y. and Dhariwal, P. Improved techniques for training consistency models. In ICLR, 2024.
Meng, E. C., Goddard, T. D., Pettersen, E. F., Couch, G. S., Pearson, Z. J.. Morris, J. H., and Ferrin, T. E. Uesf chimerax: Tools for structure building and analysis. Pro- tein Science, 2023.
Song, Z., Zhao, Y., Shi, W., Jin, W., Yang. Y., and Li, L. Generative enzyme design guided by functionally important sites and small-molecule substrates. arXiv preprint M.Xxx:2405.08200 2024.
Mirabello, C. and Wallner, B. Dockq v2: Improved au- tomatic quality measure for protein multimers, nucleic acids, and small molecules. Bioinformatics, 2024.
Steinegger, M. and Söding, J. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology. 2017.
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., and Madani, A. Progen2: exploring the boundaries of protein language models. Cell systems, (1111))9689999 0776889782220 2023.
Su, J., Han, C., Zhou, Y., Shan. J., Zhou, X., and Yuan, F. Saprot: Protein language modeling with structure-aware vocabulary. In ICLR, 2024a.
Notin, P., Rollins, N., Gal, Y., Sander, C., and Marks, D. Machine learning for functional protein design. Nature biotechnology, 2024.
Su, J., Li, Z., Han, C., Zhou, Y., He, Y., Shan, J., Zhou, X., Chang, X., Jiang, S., Ma, D., et al. Saprothub: Making protein modeling accessible to all biologists. bioRxiv, 2024b.
Qu, W., Guan, J., Ma, R., Zhai, K., Wu, W., and Wang, H. P (all-atom) is unlocking new path for protein design. bioRxiv, 2024.
Su, J., Zhou, X., Zhang, X., and Yuan, F. Protrek: Navi- gating the protein universe through tri-modal contrastive learning. bioRxiv, 2024c.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Ad- vances in Neural Information Processing Systems, 36: 53728-53744 2023.
Trippe, B. L., Yim, J., Tischer, D., Baker, D., Broderick, T., Barzilay, R., and Jaakkola, T. Diffusion probabilistic mod- eling of protein backhones in 3d for the problem. arXiv preprint arXiv:2206.0411 2022.
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., and Fergus, R. Bi- ological structure and function emerge from scaling unsu- pervised learning to 250 million protein sequences. PNAS, 2019. doi: 10.1101/622033 URL https: /www biorxiv. wwt.:oor/cccooooooee
Tsaban, T. Varga, J. K., Avraham, O., Ben-Aharon, Z., Khramushin, A., and Schueler-Furmant O. Harnessing protein folding neural networks for peptide-protein dock- ing. Nature communications, 2022.
1101/622803v4
Rose, W., Prlić, A., Altunkaya, A., Bi, C., Bradley, A. R., Christie, H., Costanzo, L. D., Duarte, M., Dutta, S., Feng, Z., et al. The resb protein data bank: integra- tive view of protein, gene and 3d structural information. Nucleic acids research, 2016.
Varadi, M., Anyango. S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic acids research, 2022.
Schnal, D., Bittrich, S., Deshpande, M., Svobodová, R., Berka, K., Bazgier, V., Velankar, S., Burley, S. K., Koča, J., and Rose, A. S. Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular struc- 05
Wallace, B., Dang, M., Rafailov, R., Zhou, L., Lou, A., Purushwalkam, S., Ermon, S., Xiong, C., Joty, S., and Naik, N. Diffusion model aligmment using direct pref- erence optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228-8238, 2024.
tures. Nucleic Acids Research, 49(W1):W431WWAH 2021. ISSN 0305- 1048. doi: 10. 10.10933na//kab3314, URL https://doi wrg/10....0o 1093 10t3//0r/gaabbbb///. /////////gg///.....//
12

---

An All-Atom Generative Model for Designing Protein Complexes
Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., and Gu, Q- Diffusion language models versatile protein learners. In ICML, 2024a.
Zhang, and Skolnick, J. Tm-align: a protein structure alignment algorithm based on the tm-score. Nucleic acids research, 2005.
Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., and Gu, Q. Dplm-2: A multimodal diffusion protein language model. arXiv preprint 14Xi::2410.13788. 2024b.
Zhang, Y., Zhang, Z., Zhong, B., Misra, S., and Tang, J. Diffpack:
torsional diffusion model for autoregressive protein side-chain packing. NewrIPS, 2024.
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L.,, Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, I.. F., et al. De novo design protein struc- ture and function with rfdiffusion. Nature, 620(7976): 1089-1100, 2023.
Zheng, Z., Deng, Y., Xue, D., Zhou, Y., Ye, F., and Gu, Q. Structure-informed language models are protein design- ers. In ICML, 2023.
Zhou, X., Xue, D., Chen, R., Zheng, Z., Wang, L., and Gu, Q. antigen-specing antibody design via direct energy- based preference optimization. In NeurlPS, 2024.
Wohlwend, J., Corso, G., Passaro, S., Reveiz, M., Leidal, K., Swiderski, W., Portnoi, T., Chimn, I., Silterra, J., Jaakkola, T., and Barzilay, R. Boltz-1: Democratizing biomolecular interaction modeling. bioRxiv, 2024.
Zhou, X., Li, M., Xiao, Y., Li, J., Xue, D., Zheng, Z. Ma, J., and Gu, Q. Designing cyclic peptides via harmonic sde with atom-bond modeling. In International Conference on Machine Learming, 2025.
Wu, F. and Li, S. Z. A hierarchical training paradigm for antibody structure-scquence co-design. NeurIPS, 2024.
Zhu, T., Ren, M., and Zhang, H. Antibody design using a score-based diffusion model guided by evolutionary, physical and geometric constraints. In ICMI., 2024.
Xu, J. and Zhang. Y. How significant is protein structure similarity with tm-score= 0.5? Bioinformatics, 2010.
Ye. F., Zheng, Z., Xue, D., Shen, Y., Wang, L., Ma, Y., Wang, Y., Wang, X., Zhou, X., and Gu, Q Proteinbench: A holistic evaluation of protein foundation models. arXiv preprint wwxx:2409.067444 2024.
Zou, S., Li, H., Mo, S., Cheng, X., Xing, E., and Song, L. Linker-tuning: Optimizing continuous prompts for heterodimeric protein prediction. arXiv preprint arXi:22312.0/186, 2023.
Yim, J. Campbell, A., Foong, A. Y., Gastegger, M., Jiménez-Luna, J., Lewis, S., Satorras, V. G., Veeling, B. S., Barzilay, R., Jaakkola, T., et Fast protein back- bone generation with se (3) flow matching. arXiv preprint arXiv:2310.05297, 2023a.
Yim, J. Campbell, A., Foong, A. Y., Gastegger, M., Jiménez-Luna, J., Lewis, S., Satorras, V. G., Veeling, B. S., Barzilay, R., Jaakkola, T., et al. Fast protein back- bone generation with se (3) flow matching. arXiv preprint arXiv:2310.05297, 2023b.
Yim, J., Campbell, A., Mathicu, E., Foong, A. Y. K., Gastegger, M., Jimenez-Luna, J., Lewis, S., Sator- ras, V. G., Veeling, B. S., Noe, F., Barzilay, R., and Jaakkola, Improved motif-scaffolding with SE(3) flow matching. Transactions on Machine Learning Re- search, 2024. ISSN 2835-888 8856. URL https:// openreview Seeareviee..nt//oommiiiccc. fan.:/oorrm2dddd1ana85555
Zambaldi, V., La, D., Chu, A. E., Patani, H., Danson, A. E., Kwan, T. O., Frerix, T., Schneider, R. G., Saxton, D., Thillaisundaram, A., et al. De novo design of high- affinity protein binders with alphaproteo. arXiv preprint arXiv:2409.08022 2024.
Zhang. Y. and Skolnick, J. Scoring function for automated assessment of protein structure template quality. Proteins: Structure, Function, and Bioinformatics, 2004.
13

---

An All-Atom Generative Model for Designing Protein Complexes
A Model
A.1
SE(3) Flow Matching
Flow Matching (FM) (Lipman et al., 2023) offers efficient framework for learning continuous normalizing flows by directly learning a time-dependent vector field that transforms samples from a prior distribution to a target data distribution, eliminating the need for expensive likelihood evaluations or ODE solving during training.
Consider prior distribution Po and target distribution P1- FM learns time-dependent vector field va R [0, 1] + R
that guides the transformation through continuous-time flow 1/a R + RR, governed by the ordinary differential equation (ODE):
d dt v1((x) vx)b
(x(()) =
(1)
where samples x Po are drawn from the prior distribution and transformed follow the target distribution pi at To enable tractable training, FM introduces an interpolant (x x1) that defines smooth path between pairs of points Xy ~ po and X₃ ~ P1- The conditional vector field u, is derived as the time derivative of this interpolant. The conditional flow matching objective then becomes:
(CFM( = 2o [llve(Xe, t) (xxxx/X0, x1]]
(2)
where x, x1). After training, new samples are generated by solving:
d
di xx - vo (xx:)) (), Xo ~ Po
(3)
When applying this framework SE(3), we need to consider both translations and rotations. For translations in R employ linear interpolation:
(x(= x1) - (1 ()xx ixi,
x6, E R
(4)
with the corresponding conditional vector field:
trans [x] Xb. x()
xi
x
x
x - t
(5)
For rotations in SO(3), following (Chen & Lipman, 2024; Campbell et al., 2024; Yim et al., 2023a), we utilize geodesic interpolation on the manifold. During training, we use linear schedule:
tt RR) R = exPR; (t log Rj (R))), R R E (SO(3)
(6)
where exp(-) and log(-) denote the exponential and logarithm maps on 50(3). During inference, we employ an exponential schedule k(()) e-e with 10:
(R) R2) = expr; ((1 - ot log (R)))
The conditional vector field in the tangent space TR, SO(3) takes different forms during training and inference. training, following the linear schedule, it is given by:
During
log (R2)
u, (R) R
(8)
1-t
while during inference, with the exponential schedule, it becomes:
(1 R2) = clogæ; (R))
(9)
For the choice of distributions, we consider the geometric properties of SO(3). We use the uniform distribution over SO(3) during training and sampling.
14

---

An All-Atom Generative Model for Designing Protein Complexes
For training, we define separate loss terms that jointly guide the learning of the vector field. The translation loss follows the Euclidean Flow Matching objective:
N l
trans (0) = Xoxxx
Frans (x,, - u, tumum (x/ xx)
[x/|>x0-
(10)
N
For rotations, following the Riemannian geometry of (3(3), we define the loss using the geodesic distance the Lie algebra:
N v RR) in (R. Rij Ri)lss)))))
(0) = (e Ro.
(11)
N
where Ri = ot ) represents the interpolated rotation at time t, and vo (Ri 1) E so(3) is the predicted velocity in the Lie algebra. The complete SE(3) Flow Matching objective combines both terms:
CSE(3) (f) = Luans (0) + (
(12)
During inference, we solve the ODE using the exponential schedule for rotations while maintaining the linear schedule for translations.
A.2 Discrete Flow Matching
While continuous Flow Matching effectively handles continuous data in 123 and SO(3), discrete data such as amino acid sequences require a different approach. We adopt Discrete Flow Matching (Campbell et al., 2024), and define a path from a masked token distribution to the data distribution. Let $ = [S] S a sequence from the data distribution, and M denote the mask token. The interpolant between S and the fully masked sequence is categorical distribution defined via the Kronecker delta. We define the i-th token Sj intermediate time as:
Prji (S) ( (S st) (1 ()o(S), M)
(13)
where 6(a, b) 1 if b and 0 otherwise. This interpolant linearly mixes the clean sequence and the mask state over time. The loss function the cross-entropy between the predicted and data distributions, written as:
[- log (S.
Ediscrete (0) =
(-140011 S -P (1S.:)
(14)
where S, (-IS)) samples corrupted sequence at time using the above conditional interpolant, (S,(S)) the neural network's 's predicted distribution given the corrupted sequence St.
A.3 Sub-module Architectures
The detailed structure is shown in Figure 2. In Sidechain Module, the protein language model is only activated in training phase II, and the protein language model encoding is weighted with a learnable zero-initialized parameter before merging into residue representation.
The resdiue level and pair-residue level information encoded in 384 and 192 dim in SeqaBB Module and Refine Module, while the dims are 256 and 128 in Sidechain Module. For model size, the overall APM contains 127M parameters, of which Seq&BB Module contains 52M, Sidechain Module contains 22M, and Refine Module contains 54M parameters.
Refine Module is initialized to apply zero change to the Seq&BB Module generated protein to ensure that Refine Module does not undergo negative optimization.
B Training Details
B.1 Loss for Seq&BB Module
Flow-matching Loss. The detailed flow-matching loss, Laow mutching. refers to Appendix A.1 and Appendix A.2. Consistency Loss. For arbitrary i for each modality (ls for sequence, 'r for backbone structure), we can get the corresponding noised data (Sts means noised sequence with tg, Ttr means noised structure with tr) and predict the final sample by the Seq&BB Module in APM, APMscq0BB:
i, f APMse Sty, Te,,
(15)
15

---

An All-Atom Generative Model for Designing Protein Complexes
We also get the noised data from the adjacent Sts+ar and +17+ and predict the final sample:
E. APMscqRBB (St Tey+ ts + At, + At)
(16)
Consistency loss is defined as the gap between the two predictions on the two modalities. Since the predictions from f closer are more accurate, we set the predictions from At teacher. For sequence, the gap the KL divergence between the two predicted logits:
consistency S (Illos(softmax(i))) (osssffmxxxxxx
(17)
For structure, the gap is the MSE between the two predicted structures:
Coonsistency MSE trans(( (T1), trans(T)) ) + MSE MattVec(rro(TT)) Matt)))aoo ddetccc)h)) detach()
(18)
Considering that the quality of the APM's predictions is not high when f approaches it is unreasonable to demand consistency this point. Thus, scale the consistency loss with respect to reducing the impact of consistency loss model training when / is small. Additionally, we also followed the construction method for consistency loss proposed by Song & Dhariwal (2024). Finally, the consistency loss used in Seq& Module is defined as:
Cconsistency = (V Cconsistency s² +++ c -cs) 16 (V Conssistency. T + o7 CT)
(19)
0.00054
dimms,
= 0.00054 * + /imy
(20)
B.2
Loss for Sidechain Module
We followed AlphaFold2 (Jumper et al., 2021) to build the loss for Sidechain Module, Cx and CFAPE. Cx is built ac- cording to Algorithm 27 in the supplementary information of AlphaFold2 and LEAPR is built according to Algorithm 28.
B.3 Loss for Refine Module
Correction Loss. The correction loss, Lears is similar Seq&BB Module's flow-matching loss.
Auxiliary Loss. The auxiliary loss consists backbone FAPE loss, LaB. and residue distogram prediction loss, Caist- FABE is the simplified version of LIAPE which only considers the backbone atoms.
Lais is the gap between the real residue-pair distance and the generated residue-pair distance. For protein with the length of N, the distance between any two residues i and is donated dij. The Laist is defined as:
N
Cdist
Σ j
||d's
20112
(21)
N(N 1)
B.4 Training
training phase I. In training phase I, the Seq&BB Module was trained on 64xH100 GPUs with 257,000 steps, with learning rate of le-4. The Sidechain Module trained 8x GPUs, accumulating total of 836,901 steps, also with learning rate of le-4.
training phase II. In training phase II, APM was trained on 64xH100 GPUs with 235,000 steps The learning rate for the le-5.
Seq&BB Module is
SFT. In the SFT phase, we used H10000 GPUs to fine-tune APM for antibody design and peptide design. The SFT phase lasted for 1200 epochs for every task. The learning rate for each module set to 5e-5 and the training cycle adjusted to 10-1-1.
Protein Language Model. We used a drop-in replacement for the ESM protein language model implementation named FAESM (Fred Zhangzhi Peng & contributors, 2024). We thank the authors for providing an efficient FlashAttention- based (Dao et al., 2022) implementation, which significantly accelerated the training speed.
16

---

An All-Atom Generative Model for Designing Protein Complexes
C Sidechain Torsion Angles Distribution
We calculated the sidechain torsion angles for each amino acid 22,281 single-chain proteins sourced from the PDB, with protein lengths ranging from 50 to 20,000. Subsequently, we analyzed the distribution of each sidechain torsion angle for each type of amino acid. As shown Figure 5, is evident that the number and distribution of sidechain torsion angles differ among various types of amino acids. There are exceptions, such as phenylalanine (F), tyrosine (Y), and tryttophan (W), which have similar sidechain torsion angle distributions due their structural similarity. In the BLOSUM62 matrix (Henikoff & Henikoff, 1992), the substitution scores between these three amino acids are positive, indicating that they can be substituted with each other to some extent. Therefore, the sidechain torsion angles retain substantial information about the amino acid types even being noised.
D Experimental Details
D.1 Sequence Sampling
The predicted sequence S₁ is sampled from the predicted logits L1 APMRetiee activated, the P1 comes from two modules, SequBB BE Module and Refine Module:
i APMse (Stg: Tip, Ls, IT): 0.8 x APMse (Seg, (t,,ts,ty + 0.2 x APMReine (S1, Tí. ts,ts,tr,, X), if 2 0.8
if < 0.8
(22)
Then, the $ is sampled following:
Si Caegorical$ootnax((4/71) arg ma))(Ln)
if < 0.85
(23)
ts 2 0.85
where the temperature T follows an exponential decay schedule:
T = Tmax x exp(-Xx
(24)
with hyperparameters Tmax 30 and decay rate À = 30. When noising the S₁ for the next step, we sort all the positions with their scores and only keep the amino acid with the top K scores, for a protein with the length N, K = int(ts x N). The score, Si, for any position defined as
s = log (0o(Stttaax + (1 - ts) x log(log( R + 10-0)+
R~ 10)
(25)
where log(log) R + 10 8) + 10 10-$) is a random term used to avoid decoding sequences in local optima. We denote the score of the K'th highest as Sk, then the sequence for the next sampling step defined as:
S1.
if > SK
Sig+Ar = -S E NJ,,
Sig+A
(26)
[MASK],
if SK
D.2 Statistical Validation on Folding
We conducted folding with APM and ESM3 using 20 random seeds. For RMSD, ESM3 shows marginally better mean (4. 44.708+0.099 vs 4.828:10.077) with statistical significance (p < 0.05). For TM-score, APM achieves better performance
10.05620.002 vs 0.828=0.002) with statistical significance (p < 0.05). The detailed results are shown below with the format of st))
Table 7. Folding performance comparison between ESM3 and APM
Method
RMSD +
TM 1
ESM3 (1.4B)
4.708 10.094
0.82810.0022
APM
4.828=1.077
0.856+10.002
D.3 Multi-Chain Protein
MULTI-CHAID PROTEIN GENERATION WITHOUT ALL-ATOM
During the phase II of APM training, the loss for the SBq&BB Module computed directly based on its own output, rather than relying on the output from the Re f i ne Module and then hack-prop to Seq&BR BB Module, As result, once the
17

---

An All-Atom Generative Model for Designing Protein Complexes
chil
chi2
chi3
chi4
R
Tuki
H
E
T
INN
C
G
A
M
F
Y
W
Figure 5. The distribution of four sidechain torsion angles in all amino acid types.
18

---

An All-Atom Generative Model for Designing Protein Complexes
Length Length 100
Langth 100
Langth
Langth Langth 300
Length 200
100
Langth 150
Longth 200
Length 200
Figure 6. APM generated proteins with chain lengths of 50-100, 100-100, 100-550, 100-200, 100-300, and 200-200.
entire training process is completed, the Seq&BB Module can be used independently, allowing for protein generation at the residue level. To verify the importance of all-atom information in multi-chain protein design, we conducted an ablation study by generating multi-chain proteins at the residue level using only the Seq&BB BB Module, and report the results in the main text (Table 4). Although the backbone structure tends to stabilize when the Sidecha in Module and Re f nine Module are activated during the last 20% of steps, the two different versions of the model still exhibit significant energy differences. When the design is carried out at the residue level, the inter-chain binding strength significantly decreases. Additionally, the difference AG and structure before and after all-atom relaxation becomes more pronounced, indicating that information the amino acid level alone is insufficient for modeling multi-chain interactions, as opposed the complete APM.
Furthermore, for proteins with lengths of 100-200, without using all-atom information, AG of the model-generated structures yields a mean value close to 0 and a median of -33. This suggests the presence of clashes at the binding interfaces, a situation not observed the complete APM. These findings underscore the importance of all-atom information modeling inter-chain protein interactions.
MORE CHAIN LENGTH COMBINATIONS
Length Combinations. In the main text, we have reported the results of multi-chain protein generation with chain length combinations 50- 100, 100- 100, and 100-200. Here, we present metrics for additional chain length combinations, including 50-200, 100-150, 100-300, and 200-200 in Table 8. Furthermore, we show more cases of each length combination in Figure 6.
Table AG and RMSD before and after all-atom relaxation for more chain length combinations. trend of affinity change consistent with that in the main text.
Length
AGRSC
AGRAA
RMSD
100-150
-33.34/53.04
102.74//90.41 -117.19/-110.37
1.28/1.10
50-200
-42.88/-3.771
1.19/1.10
200-200
-28.41/449.7
-90.10/-8..55
1.61/1.37
100-300
-23.43/-28.67
-73.03/-60.93
1.78/1.46
Chain Numbers. Theoretically, APM supports the generation of protein complexes composed of any number of chains.
However, we observed that APM's performance declines when the generated complex contains more than two chains. This evident in the increased presence of unstructured backbones or abnormally high ratios of a-helix/3-sheet structures (over
19

---

An All-Atom Generative Model for Designing Protein Complexes
Lengch 50
Longth 100
Length 100
200
Longth 50
Longsh 100
Length 200
Figure 7. APM generated proteins with more than 2 chains. The top row, generated protein complexes composed of 4 chains; the bottom row, generated protein complexes composed of chains. The length of each chain is highlighted with unique color.
90% of single secondary structure) in the generated complexes. Interestingly, these structures still exhibit no obvious clashes in the binding interface. We do apply detailed metric calculations since AG only computes the binding strength between two components. Instead, we present
cases for proteins composed of more chains in Figure 7.
attribute the degradation in performance of APM with more chains to two main reasons: 1) The majority of the complex samples consist of two chains, and complexes with more chains are significantly less; 2) With more chains, the overall length of the protein increases, which also results in reduced model generation quality.
CHAIN-BY-CHIIN GENERATION
By default, APM generates all the chains within a multi-chain protein simultaneously. However, we have also implemented an iterative generation manner, called "chain-by-chair" APM supports this manner because its training task of performing conditional generation, which means generating the remaining chains based on one or more chains a multi-chain protein. During the chain-by-chain" generation, after completing chain, we need to translate the generated parts. This necessary because APM requires initialization at the origin when generating structures meet the requirements of so (3) invariance. Therefore, need move the generated parts away from the origin make space for the next chain be generated (we also tried not translating the parts, but found that the subsequently generated chains tended to wrap uniformly around the preceding chains).
The location where the generated parts are moved to determines where the binding interfaces of the complex appear, and this process be manually specified. To avoid bias introduced by the manual specification of binding interfaces, randomly select an amino acid on the generated part to serve as the binding site (we calculated the distance of each amino acid the protein's 's center, then sorted them based on these distances, then we randomly selected from amino acids whose distances are ranked between the 33rd and 66th percentile). Subsequently, we translate the generated parts until the coordinate the chosen binding site amino acid is at the origin, and continue to translate a small distance with the same direction (default I APM then begins generating the next chain.
Table AG and the RMSD before and after all-atom relaxation for generated multi-chain proteins in by-hainn manner. Both of the two metrics show significant differences.
Length
AGsse
AGraa
RMSD
50-100
312.66/-14.05
-54.46/-54.55
1.64/1.44
100-100
235.52/-13.77
-31.79/-26.71
2.05/1.70
100-200
1073.51/-11.01
-40,29/-46.05
2,47/1.87
200
1012,53/-6.84
-10.96/-50.67
2,79/2.12
100-300
323.44/-10.94
-58.24/53.881
2.42/2.00
APM shows significant differences in generating multi-chain proteins the chain-by-chain manner compared to generating
20

---

An All-Atom Generative Model for Designing Protein Complexes
Length 50
Length 100
Length 50 Longth 200
Length 100
Length 100
Length 100
Longth 100
Longth 200
Longth 300
Figure 8. Samples generated by APM the "chain-by-chain" manner. The generation order is from short chain to long chain, the length of each chain highlighted with unique color.
all chains simultaneously, As shown in Table 9, there are three main differences observed multi-chain proteins generated in the chain-by- by-hain manner:
The binding strength between chains shows considerable decrease;
The structural differences before and after relaxation become more pronounced;
Without performing relaxation, structural clashes between chains are observed.
These variations may arise due to several reasons:
In the chain-by-chain manner, the chains more independent from another chain, which leads to reduced binding strength;
The binding interface also impacts binding strength, and a randomly chosen binding interface may not be optimal:
Once a chain is generated, its structure remains unchanged, which might result in some local structural clashes that the model cannot entirely resolve.
The samples generated in in chain-by-chain manner are shown in Figure 8.
D.4 Antibody Design
Data. Following (Ye al., 2024; Zhou et al., 2024), we the Structural Antibody Database (Dunbar et al., 2014) under IMGT (Lefrane et al., 2003) scheme the dataset. We collected matigen-antibont complexes with both heavy and light chains and protein antigens and discarded the duplicate data with the same CDR-L3 and CDR-H3 sequence. The remaining complexes are used to cluster via MMseqs2 (Steinegger & Söding, 2017) with 40% sequence similarity as the threshold based on the CDR-H3 sequence of each complex. We then select the clusters that do not contain complexes in RAbD benchmark (Adolf- Bryfflle et al., 2018) and split the complexes into training and validation sets with a ratio of 9:1 (1786 and 193 complexes respectively). The test set consists of 55 eligible complexes from the RAbD benchmark.
Methods. We follow the evaluation pipeline and results in Ye et al. (2024). We generate 64 candidates for each antigen. As our method directly generates all-atom structures, no additional sidechain packing tools are required.
Evaluation. Besides the traditional metrics like AAR and RMSD, we mainly focus the quality in terms of generated CDR-H3's rationality and functionality towards the specific antigens. We utilize pyRosetta to perform the sidechain-only relaxation and calculate energy terms. Baseline methods only design the backbone structure, while energy calculations require the all-atom structure. Therefore. sidechain packing is indispensable. To avoid performance bias introduced by packing methods and conduct fair comparison, we keep backbone structures fixed while applying relaxation sidechain
21

---

An All-Atom Generative Model for Designing Protein Complexes
only.
Additionally, we also followed the procedure in the actual experiments, where both sidechain and backbone undergo relaxation before energy calculation, even though the structure at this point differs from the one designed by the model. The results are shown in Table 10, APM still achieves the best performance.
Table 10. Performance comparison of antibody design methods on RAbD benchmark. (t/4) indicate whether higher or lower values are better. The best results are shown in bold.
Method
E. AG 4
dyMEAN DiffAb
72.76
36.43
14.56
2.29
APMsFT
-3.72
-4.08
APMzero-sba sbot
10.38
-1.77
Zero-shot. We observed that antibodies generated in a zero-shot manner exhibited stronger binding energy compared to those generated using the SFT model. However, there is significant decrease in performance in terms of accuracy/similarity to natural antibodies. This phenomenon is expected since APM is trained to generate proteins capable of binding to other chains, allowing it to produce binding-capable antibodies without SFT. However, because we excluded all antibody data from the training set, APM generates antibodies following the patterns of general proteins. As a result, the generated C3, in both sequence and structure, differs from the natural ones, highlighting the differences between antibodies and general proteins. Moreover, as we do not specify binding sites when designing proteins by using APM, the CDR-H3 designed without knowledge of antibody binding patterns might randomly bind antigens or antibody light chains. To further illustrate this phenomenon, we selected some typical samples for visualization, shown in the Figure 9.
D.5 Peptide Design
Data. The training and evaluation datasets are derived from PepBench (Kong et al., 2024) and LNR (Tsaban et al., 2022). Following Lin et al. (2024); Kong et al. (2024), we extract receptor pockets based on their spatial distances to the peptides. For data structure compatibility, we drop peptide-receptor samples that have non-standard amino acids during preprocessing. Methods. For baselines, we follow the official open-source code for training and sampling. We directly use the official checkpoint and scripts of PepGIAAD' to sample candidates. For PPFlow and DiffPP, we carefully use their official code2 for data preprocessing and follow their training instructions to obtain checkpoints at 200k steps. We also include RFDiffusion as a comparison method. Following the official guidelines', we generate peptide structures using 50 diffusion timesteps and employ ProteinMPNN for sequence design. Additionally, we define the 6 amino acids on the receptor that are closest the peptide as hot-spot residues, as we extract receptor pockets for other methods.
Evaluation. Following Kong et al. (2024), we generate 40 peptide candidates for each sample using all methods. Note that the length of each generated peptide is predefined match its corresponding ground-truth sequence. We evaluate the binding capabilities of generated peptide candidates from multiple perspectives.
Funetionality. We follow the approach used by Kong et al. (2024). Both relaxation and energy calculations are performed using pyRosetta. The evaluation proceeds by identifying the best candidate for each receptor, and reporting the median performance values across all receptors. Note that both the backbone and sidechain are applied relaxation. This procedure could achieve lower binding energies (AG < 0) rather than the hundreds in antibody designing. It should be noted that this procedure may fail to achieve perfect relaxation in specific cases. For instance, 5 out of 93 ground truth samples still retained slight clashes, resulting in AG slightly above Oilllssrratdd in Figure 10). Foldability. We only fold generated sequences with lengths greater than or equal to 10 residues (46 peptide-receptor pairs in total). As indicated in the documentation of Boltz-1 and AlphaFold3, the ipTM scores may not be reliable for sequences that are too short. To ensure reliability, we only consider reptite-reeeptor pairs where the ground -ruth samples achieve successful confidence scores in Boltz-1 (PLDDT> 70 and ipTM> 0.8.). filtering out cases where even the ground truth structures fail meet the confidence threshold. The two threshold values are adopted from the official output documentation' of AlphaFold3 and paper (Abramson et al., 2024). Considering the computational cost and MSA retrieval time, we select the top 16 sequences ranked by AG and perform folding 8 times for each peptide-receptor pair. For each sequence, we select the best folding result, then average these results across all 16
ww. DOon/THUNLP-MT/POOA / paatts:://ithhen wow/EDAP IONn/EDAFNNENN//pEELLo
com 'https://gtt http://gthaa..co.o///eeeeteeeeee www./Rossstaaaoooo///eeeee f ffusion
Phttps://gt thub. htt.:cccoo/oogo// http://oogee-deemmmin/aaaaaaaaea fol httpd////a//////////////o//////e
22

---

An All-Atom Generative Model for Designing Protein Complexes
sequences to obtain the final confidence score.
Accuracy. We measure the interface structural accuracy using the DockQ score, which provides quality measures quantify different aspects between generated and reference structures. Concretely, we utilize DockQv2' from their official codebase. Due to extreme structural conflicts observed in some samples generated by baseline methods, we only calculate DockQ score for samples with AG < 0.
Visualization
Generated Structure. We present the peptides designed by different methods in Figure 11. The showcased examples include peptides with diverse secondary structures (loops, helices, and sheets). As observed in the visualization, our method demonstrates the ability to understand and generate appropriate secondary structures while maintaining reasonable interactions with the receptor.
Folded Structure. As shown in Figure 12, we visualize the folded structures of sequences generated by different methods using Boltz-1. The structures are colored according to the pL.DDT confidence, where blue regions indicate high confidence. We borrow the color bar from AlphaFold server website". Receptors are shown in gray with 20% transparency for better visualization. The visualization demonstrates that our method generates sequences capable of folding into stable structures with high confidence scores, indicate the quality of the generated sequences.
E
Extended Related Works
Sidechain Prediction. Accurate prediction of sidechain conformation is crucial for protein design. Recent deep learning approaches have significantly advanced this field. DiflPack (Zhang et al., 2024) employs torsional diffusion model that learns the joint distribution of sidechain torsion angles by diffusing and denoising in torsional space. It autoregressively generates the four torsion angles. AttnPacker (McPartlon & Xu, 2023) directly incorporates backbone 3D geometry to simultancously compute all sidechain coordinates without relying on discrete rotamer libraries or conformation search.
Motif-Scaffolding, Motif-scaffolding, the design of proteins that incorporate specific functional motifs emerged powerful approach in functional protein design. Structure-based methods like RFDiffusion (Watson et al., 2023) and FrameFlow (Yim et al., 2023a) enable the generation of backbone scaffolds that can accommodate predefined motifs while maintaining overall structural stability. Sequence et al., 2024a), and ESM3 (Hayes et al., 2025) present capabilities by designing sequences that fold into structures compatible with functional motifs. These methods collectively provide a comprehensive toolkit for designing proteins with specific functional properties.
approaches including EvoDiff (Alamdari et al., 2023), DPLM (Wang
Protein Structure Refinement. Structure refinement is essential for optimizing protein designs to achieve native-like stabil- ity and function. Physics-based methods such as Rosetta relax (Alford et al., 2017) and OpenMM minimization (Eastman et al., 2017) remain widely used for local refinement of protein structures. These refinement methods play a crucial role in the protein design pipeline, helping to bridge the gap between computational designs and experimentally viable proteins.
F Binder Design
Settings. We further explore APM's zero-shot capabilities in binder design, focusing specifically on longer protein binders rather than short peptides. Following previous works (Watson et al., 2023; Zambaldi al., 2024), we selected several important targets: Interleukin-7 Receptor-a(L--RRA)
spike protein receptor-binding domain (SC2RBD), MDM2, Programmed 1 (PD1), Programmed Death-Ligand 1 (PD-L1), and CD3-epsilon (CD3E). We extracted the corresponding target chains and reference binders from PDB: 3di3, 6m0 j. 1 lyor, 4zqk, 4z1 8, and 1xiw, respectively. The evaluation settings remain consistent with peptide design experiments. However, due to uncertainty about which amino acids should be defined as hot-spot residues for these targets, we did not define hot-spot residues for RFDiffusion nor extract pockets for APM. Consequently, we do not report DockQ for this task. Additionally, due to computational resource constraints, folded only the top 8 sequences rather than 16 in previous experiments. The results of average metrics are presented in Table 11, with the 'Success' representing the proportion of samples (out of 8) that achieve both pLDDT > 80 and ipTM > 0.8. We present the generated binders across different methods and their corresponding folded structures with the highest PLDDT sequences in Figure 13.
Discussion. Overall, APM is comparable to RFDiffusion. Although these metrics have been proven by many studies to be predictive of wet lab experimental results (Zambaldi et al., 2024; Bennett et al., 2025), the actual effectiveness still requires validation through wet lab experiments. We observed that both APM and REdiffusion encounter cases where some samples exhibit lower PLDDT and ipTM scores. The PLDDT score can vary significantly along a protein chain. This means
'https://go thub. com / bjornwal ner/Bock phttps://al phwppp whaafflsseeeerrrr.
23

---

An All-Atom Generative Model for Designing Protein Complexes
Table 11. Performance comparison of binder design.
APMMPNN represents using ProteinMPNN redesign sequences.
Target
Method
AG
% <0 f
pLDDT
ipTM
Success
GroundTruth
-23.79
95.26
0.85
RFDiffusion
-50.49
82.50%
87.83
0.30
0%
3d13
APMMPNN
-80.09
92.50%
83.39
0.29
12.5%
APM
-80.10
95.00%
78.91
0.38
12.5%
GroundTruth
-20.11
81.55
0.15
RFDiffusion
-5..10
67.50%
70,90
0.45
0%
6m0j
APMMPNN
-88.99
65.00%
67.82
0.40
12.5%
APM
-9..47
67.50%
69.50
0.48
12.5%
GroundTruth
-25.24
90.42
0.93
RFDiffusion
-39.47
100%
78.49
0.81
25.0%
Iyer
APMMPNN
-33.27
90.00%
71.10
0.70
50%
APM
-37.94
90.00%
66.28
0.67
25.0%
GroundTruth
-39.36
94.03
0.87
RFDiffusion
-29.35
87.50%
75.79
0.39
0%
4zqk
APMMPNN
-43.33
77.50%
79.10
0.36
APM
-4.27
90.00%
80.18
0.39
0%
GroundTruth
-40.89
92.08
0.76
RFDiffusion
-18.69
57.50%
67.39
0.30
0%
4z18
APMMPNN
-63.37
55.00%
74.28
0.34
0%
APM
-54.24
55.00%
69.28
0.35
0%
GroundTruth
-71.69
92.64
0.95
RFDiffusion
-56.99
95.00%
77.22
0.76
62.5%
1xiw
APMMPNN
-46.96
82.50%
72.08
0.70
12.5%
APM
-43.25
85.00%
73.27
0.62
12.5%
the folding model can be very confident in the structure of some regions of the protein, but less confident in other regions. We hypothesize that the low PLDDT scores stem from the complexity of long binders. Specifically, certain regions may be naturally highly flexible or intrinsically disordered, leading the folding model to assign low pLDDT scores to these residues (as indicated in (Guo et al., 2022)). Regarding ipTM, we speculate that the lower scores may result from the larger binding interfaces typical of long binders, which often involve multiple contact points or complex features such as convex or polar epitopes, or hydrophobic regions (Zambaldi et al., 2024). These structural complexities and biological properties can contribute lower ipTM scores.
Future Directions. As suggested in (Zambaldi et al., 2024; Bennett et al., 2025), PLDDT and ipTM are predictive of binding success. We would like to discuss potential approaches to improve long binder design. APM was originally developed as a general-purposs model for complex modeling rather than task-specific one, which presents challenges in the context of long binder design. This can be reframed as a question of how adapt a general model into domain -amaain-ppeiillzeed one. Recent work (Bennett al., 2025), provides valuable practical directions. The authors successfully transformed REdiffusion into an antibody-specific model by fine-tuning it on antibody-anttieen complex structures, demonstrating that domain-speciff data can significantly enhance performance. Similarly, a feasible approach to enhance APM for long binder design would be to use a curated dataset of long binder-target complexes, potentially sourced from PDB or synthetic data. Besides, post-training techniques offer another strategy to optimize the model for generating high-confidence designs. As demonstrated (Zambaldi et al., 2024; Bennett et al., 2025), PLDDT and ipTM correlate with binding success. Building on this insight, we could implement preference optimization focused on these confidence metrics. Applying DPO-like (Rafailov al., 2023; Wallace et al., 2024) algorithms, can then train the model to favor high-confidence designs while avoiding low-confidence ones.
G Future Works
Model Scaling, We chose not to incorporate triangular attention APM, which considered key feature in the success of AlphaFold2/3,, because we aim to scale the model future to observe whether scaling laws exist in our model. Triangular attention significantly restricts our ability scale the model size.
Pair Information from PLM. In APM, PLM plays crucial role by providing the model with a robust understanding of protein sequences. However, we only utilized the representations of individual amino acids from the PLM. and not the levrl information (pair-level information refers the attention matrix in PLMs). Pair-level information has been proven
24

---

An All-Atom Generative Model for Designing Protein Complexes
provide significant benefits for protein structure learning. The reason did pair-level information is to accelerate the encoding process of sequences by PLM (especially in representing multi-chain data). We used PLM implemented with flash attention, which prevented us from obtaining complete pair-level information. In the future, we will attempt to resolve the encoding speed issue with PLM and use the original PLM implementation to gain the access to pair-level information. Refine Module. The Re inne Module is designed to refine the structure and sequence generated by the Seq& BB Module, which can be seen form of relaxation that allows modifications to the types of amino acids. Currently, our Refine Module primarily aims to make the generated proteins resemble real proteins more closely. In the future, we will attempt to incorporate more biological/hhyscal constraints (e.g., force fields) into the Refine Module to achieve better performance. Interchain Hotspot Residue Assignment. Hotspot residues, the amino acids playing crucial roles in interactions, also be considered as regions where interactions occur, serving important parameters in describing the binding proteins. However, the current version of APM does not support specifying hotspot residues during the generation of complexes or functional proteins. Instead, APM autonomously determines the regions where interactions occur. This design was made to avoid the impact of assigning different hotspot residues on the model's performance during general multi-chain protein generation. Consequently, this design leads to differences in the binding patterns of proteins generated directly using APM (in shot manner) compared to natural samples. In the future, we will address this issue to allow APM to support the assignment of hotspot residues, thereby enhancing APM's performance in zero-shot manner. Downstream Tasks. We will validate APM capability designing proteins with biological functions in more downstream tasks.
H Visualization
All protein visualizations in this paper were completed using ChimeraX (Meng et al., 2023) (Figure 1, Figure 2, Figure 3, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13) and Protein Viewer (Sehnal et al., 2021) (the remaining visualizations).
25

---

An All-Atom Generative Model for Designing Protein Complexes
Ground Truth
APM zero-shot
APM SFT
A
B
c
D
SUUTHN
Figure 9. The CDR-H3s generated in zero-shot manner display distinct patterns compared to natural ones. The antigen is represented pink, the antibody heavy chain purple, and the antibody light chain blue, with the CDR-H highlighted in red. differences between DDR-H3 generated in a zero-shot manner and those from natural antibodies can be categorized as follows: A. No obvious difference, generated CDR closely resembles that natural antibody. The generated CDR-H3 interacts with the antigen but binds a different position compared to the reference antibody. C. The generated CDR-H1 binds to the antibody light chain. D. While generated CDR-HB binds correctly with the antigen, its structure predominantly consists of beta-sheets, unlike the common looped structures in natural antibody CDRs. All the aforementioned differences disappear after undergoing SFT. The CDR-H3s generated by A APMsFT closely resemble that the natural ones.
26

---

An All-Atom Generative Model for Designing Protein Complexes
54.59
40.83
4.04
1.69
9.92
Figure 10. ground truth samples with AG exceeding The receptor is shown in white, with the peptide highlighted in red. Slight clashes are marked by red boxes. Additionally, AG for each sample are listed below them.
27

---

An All-Atom Generative Model for Designing Protein Complexes
GroundTruth
APM
APM(zero-shot
PepGLAD
PPFlow
Figure 11. Visualization of peptides generated by different methods. The blue regions represent the given receptors, while the pink regions show the generated peptides, with all-atom structures displayed the interface regions. From left to right: Ground truth structures. APM, APM zcro-shot, PepGLAD, and PPFlow. The PDB IDs for the six cases from top to bottom are: 1 ijrr, 2cnz, 3ayu, 4dcb, 5frs, and 6gg8.
28

---

An All-Atom Generative Model for Designing Protein Complexes
GroundTruth
APM
AP APM(zrro-shot)
PepGLAD
PPFlow
Very high (slDDT> 90)
Confident (90 pDDD 70)
Low (70 1 x
Very low (aIDOT
Figure 12. Folded structures of sequences generated by different methods using Boltz-1. The structures are colored according to the AlphaFoll-style PLDDT confidence scheme. From left right: Ground truth structures, APM, APM zero-shot, PepGLAD, and PPFlow. The PDB IDs for the six cases from top to bottom are: 1 ijrr, 2cn2, 3ayu, 4dcb, 5frs, and 6gg8.
29

---

An All-Atom Generative Model for Designing Protein Complexes
GroundTruth
APM
RFDiffusion
APM(folded)
RFDiffusion(fodedd
3di3
6m0j
lycr
4zqk
4z18
Ixiw
Very High (2DDT 90)
Confident (90 pIDT > 70)
Low (70 p
Very low (pIDDT 50
Figure 13. Visualization of binder design results six protein targets. From left right: (1) GroundTruth: native complex structures, (2) APM: structures generated by APM, (3) RFDiffusion: structures generated hy RFDiffusion, (4) APM(folded): highest pL.DTT APM sequences folded with Boltz, (5) RFDiffusionffolded; highest pDDDT sequences folded with Boltz. The blue regions represent the given targets, while the pink regions show the generated binders. For folded structures, targets are rendered transparent to highlight binders, with binders colored according AlphaFold's PLDDT scheme. Each row represents different PDB target: 3d13. 3, 6m0 j. lyer, 42gk. 4 221 8, and 1xiw.
30


Reviews:
[{"review_id": "W456f0hJ2U", "reviewer": "Reviewer_Ymkv", "summary": "This paper tackles the problem of designing multi-chain protein complexes at the atomic level. The authors propose APM (All-Atom Protein Generative Model), consisting of three modules:\n1. **Seq&BB Module**: A flow-matching based generative model that handles the co-generation of protein sequence and backbone structure.\n2. **Sidechain Module**: Predicts sidechain conformations (parameterized by torsion angles) to complete the all-atom structure.\n3. **Refine Module**: Adjusts the sequence and structure using all-atom information to increase naturalness and resolve structural clashes.\n\nThe model incorporates ESM2-650M (a protein language model) to enhance protein sequence understanding and uses a two-phase training approach with the first phase focused on Seq&BB Module, Sidechain Module, and the second phase focused on joint training of all three modules. The model is trained on a mixture of single and multi-chain data from PDB, Swiss-Prot, AlphaFoldDB, and PDB biological assemblies.\n\nThe authors perform extensive insilico evaluation of APM and demonstrate competitive performance compared to baseline models. For single-chain protein folding and inverse-folding, APM either outperforms or matches the performance of ESMFold, ESM3, MultiFlow, and ProteinMPNN. For multi-chain protein folding and inverse-folding, APM outperforms Boltz-1 (without MSA) and ProteinMPNN. Furthermore, the authors demonstrate APM's ability to generate tightly bound protein complexes and perform ablations to showcase the importance of sidechain conformation information. On downstream antibody design targeting specific antigens and recptor-targeted peptide design, both supervised fine-tuning and zero-shot sampling variants of APM outperform application-specific models.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your thorough review and helpful comments. We have carefully addressed your concerns below and welcome any additional feedback.\n\n**Q1: Claims And Evidence**\n\nA1: Thank you for highlighting the need for statistical validation. We conducted folding with APM and ESM3 using 20 seeds. For RMSD, ESM3 shows a marginally better mean (4.708±0.094 vs 4.828±0.077) with statistical significance (p < 0.05). For TM-score, APM achieves better performance (0.856±0.002 vs 0.828±0.002) with statistical significance (p < 0.05). We appreciate your reminder and will revise our statement to reflect that APM demonstrates \"competitive\" or \"comparable\" performance to ESM3.\n\n|Method|RMSD (mean±std)|TMscore (mean±std)|\n|-|-|-|\n|ESM3 (1.4B)|4.708±0.094|0.828±0.002|\n|APM|4.828±0.077|0.856±0.002|\n\n**Q2: Weaknesses on Perplexity**\n\nA2: Thank you for your valuable suggestion. To evaluate sequence quality, we compute perplexity using ProGen2-base across various inverse folding methods, including average, median, and std of perplexity. The results indicate that the sequences generated by APM achieve comparable perplexity to ground truth sequences.\n\n|Method|ppl_avg|ppl_median|ppl_std|\n|-|-|-|-|\n|GroundTruth|8.83|7.13|5.87|\n|APM|8.74|8.10|4.01|\n|Multiflow|10.86|10.94|2.66|\n|ESM3|8.64|7.90|4.23|\n|ProteinMPNN|11.44|11.48|3.25|\n\n**Q3: Experimental Designs Or Analyses and Weaknesses**\n\nA3.1: Thanks for your suggestion. We agree that a larger test set could enhance evaluation. Here, we focus on maximizing training data for design tasks, using only \"samples missing cluster IDs\" as the test set. This allowed us to retain more training data while ensuring reliable performance evaluation.\n\nA3.2: Thanks for this suggestion. Due to policy restrictions, we are unable to access AF3's weights, which is why we opted for Boltz. Regarding Chroma, we added the results on unconditional multimer generation. The results demonstrate that APM can generate reasonable multi-chain structures with high binding affinity, significantly outperforming Chroma.\n\n|Length|Model|dG_sc|dG_relax_bb+sc|RMSD|\n|-|-|-|-|-|\n|50-100|APM_all-atom|-72.44/-71.91|-112.65/-116.98|1.05/0.95|\n||APM_bb|-64.30/-67.30|-114.94/-114.45|1.06/1.03|\n||Chroma|113.64/46.51|-83.96/-86.66|1.33/1.22|\n|100-100|APM_all-atom|-91.61/-94.54|-130.31/-134.57|1.04/0.94|\n||APM_bb|-36.74/-69.30|-117.53/-118.13|1.17/1.12|\n||Chroma|89.47/22.97|-60.53/-52.64|1.45/1.35|\n|100-200|APM_all-atom|-44.02/-39.42|-93.21/-73.09|1.35/1.21|\n||APM_bb|-3.42/-33.71|-85.79/-69.12|1.58/1.42|\n||Chroma|79.97/35.86|-59.32/-54.30|1.58/1.48|\n\nA3.3: Thank you for your question. The results presented in Table 4 is indeed an ablation study. We provide two types of dG scores: one with sidechain-only relax and another with both backbone and sidechain relax. The lower dG scores from sidechain-only relax indicate high-quality backbone generation without structural conflicts. The lower RMSD scores between generated and relaxed backbone demonstrate that the initial structures are already near optimal conformations, requiring minimal adjustment to reach energy minima. Notably, APM with the Refine Module significantly outperforms APM using only the Backbone Module on these metrics, validating the effectiveness of the Refine Module in complex design.\n\nA3.4: Thank you for this suggestion. While we acknowledge the importance of wet lab experimental validation, we used pyRosetta's energy function as it is a widely accepted metric for evaluating binding affinity.\n\nA3.5: For the folding task, we have already reported the std in A1. For antibody design and peptide design tasks, the metrics are based on multiple sampling. By adjusting the temperature strategy for sequence decoding in the unconditional single-chain task, we achieve improved results and additionally provide the std for RMSD and TMscore to demonstrate the stability of results(see table below).\n\n|Method|Length 100||Length 200||Length 300||\n|-|-|-|-|-|-|-|\n||scTM|scRMSD|scTM|scRMSD|scTM|scRMSD|\n|APM (original)|0.92±0.11|3.65±5.37|0.88±0.12|5.06±6.57|0.87±0.14|7.33±6.76|\n|APM (updated)|0.96±0.06|1.63±2.12|0.90±0.11|3.43±3.14|0.90±0.11|4.90±4.49|\n\nA3.6: Thanks for your suggestion. APM currently supports 20 standard amino acids. This impacts our ability to evaluate certain enzyme-substrate complexes, as substrates can be diverse molecules beyond proteins. While enzymes are typically proteins, substrates can vary widely, including carbohydrates and lipids.\n\nWithin the current scope of the submission, we focus on protein-protein interaction tasks. We are working on extending our framework to support a broader range of biomolecules. \n\n**Q4: Essential References Not Discussed**\n\nA4: Thank you for your kind reminder. We have mentioned AF3 in the Related Work (line 77). We will revise manuscript to make it clearer. Regarding Chroma, we will include it in the revision.", "claims_and_evidence": "- **Claim**: APM achieved leading performance compared to other co-design methods in all three tasks related to single-chain proteins.\n  This claim is misleading for protein folding, as ESM3 has a slight edge for structure prediction as evidenced by Table 1. The differences are very small and it would be helpful to have interval estimates to assess significance.", "methods_and_evaluation_criteria": "**Strengths:**\n- Use of all-atom representation for protein structure is well motivated and addresses a critical problem in multi-chain protein modeling.\n- Flow-matching based generative modeling is well suited for the task at hand.\n- Chosen datasets for training and evaluation are appropriate, diverse, and standard making it easier to compare with existing works.\n- Relevant metrics like RMSD and TM-score for structure prediction evaluation as well as scTM, AAR for inverse-folding evaluation are used. Similarly, for application-specific tasks, using specialized metrics like DockQ, binding affinity, etc. aligns well with real world use cases.\n\n**Weaknesses:**\n- Sequence recovery aggregates binary decisions for correct sequence identity without taking into account the precise confidence of the model for the correct residue. Perplexity addresses this shortcoming, however, it is not included in the evaluation.", "theoretical_claims": "Did not check theoretical claims.", "experimental_designs_or_analyses": "**Strengths:**\n- Careful curation of training dataset to prevent potential information leakage for antibody and peptide design tasks.\n- Ablation study is performed to assess the impact of sidechain conformation information on (predicted) binding affinity.\n- Both average and median performance metrics are reported.\n- Downstream antibody and peptide design tasks which are relevant to real-world applications are evaluated.\n- Performant models from literature are used for comparison.\n  \n**Weaknesses:**\n- Inverse-folding and structure prediction evaluation for multi-chain complexes uses a test set of size 273, which makes it hard to assess generalizability of results.\n- Structure prediction and inverse-folding evaluation for multi-chain complexes is only compared with Boltz-1 and ProteinMPNN, with the authors arguing that there are almost no other models that support multi-chain proteins. However, AlphaFold3 (https://www.nature.com/articles/s41586-024-07487-w) and Chroma (https://www.nature.com/articles/s41586-023-06728-8) could be appropriate models to compare with for structure prediction and inverse-folding, respectively.\n- Ablation study is limited to a single evaluation with only one module ablated, making it difficult to assess the contribution of each module across different tasks.\n- Lack of experimental validation of generated complexes. For instance, it's unclear how well pyRosetta's energy function would align with measurements obtained from binding assays in the wet lab.\n- Only average/median performance metrics are reported as opposed to interval or standard deviation, which makes it hard to evaluate significance.\n- While antibody and peptide design applications are presented, other important complex types (e.g., enzyme-substrate complexes) aren't evaluated.", "relation_to_broader_scientific_literature": "Most of the prior work has focused on factorizing protein representations into amino acid sequence and backbone structure. Furthermore, training of protein foundation models has typically been done on single-chain protein data. The authors' work addresses this gap by incorporating information about sidechain conformation into the generative model as well as directly training on multi-chain protein data. The authors' work is also novel in that it uses flow matching based generative models whereas past works have used diffusion based generative models.", "essential_references_not_discussed": "The following references are missing:\n- Chroma (https://www.nature.com/articles/s41586-023-06728-8): Generative model for protein complexes that takes the sidechain conformation information into account in addition to backbone coordinates and sequence identity.", "other_strengths_and_weaknesses": "The paper is well written and comprehensive in describing the key details for reproducibility of both the model and the evaluation. Use of all-atom representation for protein structure is well motivated and addresses a critical problem in multi-chain protein modeling.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S5RAjTl5RV", "reviewer": "Reviewer_SdWK", "summary": "The paper introduces a new all atom protein backbone generation which is composed of a backbone structure model (that is equivalent to discret flow models from Campbell and al.), a side chain module and a refinement module. They are trained in two stages, first the backbone and side-chain modules separatly and then the three modules jointly. THe model is then evaluated on peptide, antibody-antigene complex, folding and inverse folding. The authors show competitive performance and the possibility to extend existing protein backbone models to inverse folding task.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We have addressed your concerns below and welcome any further feedback.\n\n**Q1: Relation To Broader Scientific Literature and Essential References Not Discussed.**\n\nA1: We appreciate the reviewer for pointing out these references. In our revision:\n1. We will include FoldFlow1 (Bose et al.) in the Related Work section to clarify the concurrent development of SE(3) flow matching approaches to FrameFlow.\n2. Although FoldFlow2 (Huguet et al.) showcases the integration of protein language models, the folding code for FoldFlow2 is currently unavailable. We will incorporate a discussion in the Related Work section: APM differs from FoldFlow2 in several aspects: our model is capable of generating all-atom structures and sequences, supports multi-chain generation, and is designed for complex design tasks.\n\n**Q2: Questions For Authors: Did you try a binder design task on the RFdiffusion benchmark? I would be very curious to know how it performs.**\n\nA2: We thank the reviewer for this valuable suggestion. As the Reviewer ZXig also mentioned the design of longer binder. Here, we design binders to the two targets showcased in RFDiffusion: SARS-CoV spike protein RBD (PDB id 6m0j) and IL-7RA (PDB id 3di3). For comparison, we first generate structures using RFDiffusion, then design sequences with ProteinMPNN. As APM currently does not support hot-spot residue, we only evaluate functionality and foldability. \n\nThe table demonstrates that APM in zero-shot mode significantly outperforms RFDiffusion in terms of dG, with a higher percentage of designs achieving dG < 0. Additionally, APM shows better foldability, as indicated by higher ipTM scores, and successfully designs binders for both targets. This indicates that APM is able to design  high-quality binders.\n\n|Method|Target|dG|%dG<0|pLDDT|ipTM|Success|\n|-|-|-|-|-|-|-|\n|GroundTruth|3di3|-23.79|-|95.26|.85|100%|\n||6m0j|-20.11|-|81.55|.15|0%|\n|APM_zero-shot|3di3|-80.10|95.00%|78.91|.38|12.5%|\n||6m0j|-96.47|67.50%|69.50|.48|12.5%|\n|RFDiffusion|3di3|-50.49|82.50%|87.83|.30|0%|\n||6m0j|-56.10|67.50%|70.90|.45|0%|", "claims_and_evidence": "The paper claims to define an all atom protein complex generation and this is true. They claim to be competitive on different downstream tasks, which is true.", "methods_and_evaluation_criteria": "The method makes sense and the evaluation follows the standard practice in the literature.", "theoretical_claims": "Not applicable", "experimental_designs_or_analyses": "The analysis is complete and makes sense.", "relation_to_broader_scientific_literature": "SE(3) flow matching for protein backbone was also introduced in [1] and is concurrent to FramFlow. I think it should be added to highlight the fact they are concurrent.\n\n[1] SE(3)-Stochastic Flow Matching for Protein Backbone Generation, Bose et al, ICLR 2024", "essential_references_not_discussed": "The integration of protein language model in protein backbone generation module was already done in [2] and therefore it should be discussed and evaluated against in folding tasks.\n\n[2] Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generation, Huguet et al. Neurips 2024.", "other_strengths_and_weaknesses": "The paper is well written.", "other_comments_or_suggestions": "NA", "questions_for_authors": "Did you try a binder design task on the RFdiffusion benchmark? I would be very curious to know how it performs.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wvgproqJPl", "reviewer": "Reviewer_fiXM", "summary": "we introduce APM (All-Atom Protein Generative Model), a model specifically designed for modeling multi-chain proteins. By integrating atom-level information and leveraging data on multi-chain proteins, APM is capable of precisely modeling inter-chain interactions and designing protein complexes with binding capabilities from scratch. It also performs folding and inversefolding tasks for multi-chain proteins. Moreover, APM demonstrates versatility in downstream applications: it can be supervised fine-tuned (SFT) for enhanced performance while also supporting zero-shot sampling in certain tasks, achieving state-of-the-art results.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your helpful feedback. We have responded to your concerns below and look forward to any additional comments.\n\n**Q1: Essential References Not Discussed**\n\nA1: Thank you for highlighting the need to include additional related works. We will enhance our Related Work section by incorporating references on sidechain prediction, such as DiffPack and AttnPacker. For motif-scaffolding, we will discuss both structure-based and sequence-based methods, including RFDiffusion, Frameflow, EvoDiff, DPLM, and ESM3. Regarding protein structure refinement, we hypothesize this refers to methods like Rosetta relax and OpenMM minimization, as well as other works like DeepAccNet (NC2021, https://www.nature.com/articles/s41467-021-21511-x). We welcome your feedback to ensure comprehensive coverage of these important areas in our Related Work section.\n\n**Q2: Other Weaknesses**\n\nA2.1: Thank you for your question. The results presented in Table 4 is indeed an ablation study. We provide two types of dG scores: one with sidechain-only relax and another with both backbone and sidechain relax. The lower dG scores from sidechain-only relax indicate high-quality backbone generation without structural conflicts. The small RMSD scores between generated and relaxed backbone demonstrate that the initial structures are already near optimal conformations, requiring minimal adjustment to reach energy minima. Notably, APM with the Refine Module significantly outperforms APM using only the Backbone Module on these metrics, validating the effectiveness of the Refine Module in complex design.\n\nA2.2 & A2.3: Thank you for your valuable feedback. We acknowledge that our current system combines widely accepted architectures to model protein sequences and structures. Exploring more efficient frameworks is indeed an important direction for the field, and we will discuss potential approaches to this in our response A4.2.\n\n**Q3: Other Comments Or Suggestions**\n\nA3.1: This is a great question! We believe that APM has the potential to generalize to other protein-related tasks in the future, such as protein docking or directly utilizing the pre-trained encoder for affinity prediction tasks. However, this remains an open question, and we leave this for future exploration.\n\nA3.2: Thank you for pointing this out! We acknowledge that APM currently does not achieve SOTA performance across all metrics in the task of unconditional single-chain generation. To address this, we have adjusted the temperature strategy for sequence sampling, resulting in significant improvements over previous methods for this task. Please refer to the table below for updated results.\n\n|Method|Length 100||Length 200||Length 300||\n|-|-|-|-|-|-|-|\n||scTM|scRMSD|scTM|scRMSD|scTM|scRMSD|\n|NativePDBs|0.91|2.98|0.88|3.24|0.92|3.94|\n|ESM3|0.72|13.80|0.63|21.18|0.59|25.5|\n|Multiflow (woSynthetic)|0.86|4.73|0.86|4.98|0.86|6.01|\n|ProteinGenerator|0.91|3.75|0.88|6.24|0.81|9.26|\n|ProtPardelle|0.56|12.90|0.64|13.67|0.69|14.91|\n|APM (original)|0.92|3.65|0.88|5.06|0.87|7.33|\n|APM (updated)|0.96|1.63|0.90|3.43|0.90|4.90|\n\nA3.3: Thank you for this question. APM's current implementation and experiments focus on CDR-H3 design in SFT and zero-shot manner, leveraging the conserved nature of antibody framework regions.\n\n**Q4: Questions For Authors**\n\nA4.1: Thank you for this question. To assess the effectiveness of the consistency loss, we conducted an experiment by removing the consistency loss and retraining the model to the same number of steps. When evaluated on unconditional single-chain generation tasks, the version with consistency loss showed improvements in both scTM and scRMSD compared to the ablated version, as detailed in the table below.\n\n|Method|Length 100||Length 200||Length 300||\n|-|-|-|-|-|-|-|\n||scTM|scRMSD|scTM|scRMSD|scTM|scRMSD|\n|APM (updated)|0.96|1.63|0.90|3.43|0.90|4.90|\n|APM (woConsistency)|0.93|2.66|0.89|3.72|0.88|5.18|\n\nA4.2: This is a great question, and we fully agree that it represents a crucial challenge in scaling of protein models. A recent work Proteina (https://openreview.net/forum?id=TVQLu34bdw) may serve as an important exploration of model and data scaling. Proteina employs a scalable, non-equivariant AF3-like diffusion transformer, focusing on alpha carbon coordinates without frames, which allows for scalability to many parameters and long protein generation. These considerations are crucial for advancing the field and addressing the challenges of scaling protein generative models. In the future, we plan to explore Proteina-like scalable framework to further enhance the scalability and efficiency of our protein generative models.\n\nA4.3: We achieve conditional generation by employing masking to ensure that the loss is not computed for the condition region. Additionally, we set the time step t=1 to ensure that the ground truth condition is provided.", "claims_and_evidence": "Claims:\n\n- APM natively supports the modeling of multi-chain proteins without the need to use pseudo sequence to connect different chains.\n- APM generates proteins with all-atom structures efficiently by utilizing an innovative integrated model structure.\n\nEvidence:\n\n- Experiments related to general protein demonstrate that APM is capable of generating tightly binding protein complexes, as well as performing multi-chain protein folding and inverse folding tasks.\n- Experiments in specific functional protein design tasks show that APM outperforms the SOTA baselines in antibody and peptide design with higher binding affinity.", "methods_and_evaluation_criteria": "For Multi-Chain Protein Modeling, APM uses a mixture of single and multi-chain data in the training of APM. \n\nFor all-atom representation, APM chooseS to enhance residue-level information with the sidechain for all-atom protein representation that includes amino acid type, backbone structure, and the sidechain conformation parameterized by four torsion angles.\n\nFor sequence-structure dependency, first, APM decoupled the noising process for sequences and structures so that the noising level for each modality does not completely align, minimizing disruption of their dependency. Second, there is a 50% probability of performing a folding/inverse-folding task, compelling the model to learn the dependencies from both directions. \n\nAPM has demonstrated its capability in modeling multi-chain proteins and generating bioactive complexes. It achieved state-of-the-art (SOTA) performance in antibody design and binding peptide design. It also show promise for conventional single-chain protein-related tasks.", "theoretical_claims": "No significant theoretical clains", "experimental_designs_or_analyses": "APM is tested on multiple tasks including folding, inverse folding, uncond. generation and functional protein design, both in complex generation or single chain.", "relation_to_broader_scientific_literature": "This is an application of flow matching methods on multi-chain protein generation. The modeling scheme and architeture comes from the famous AlphaFold2.", "essential_references_not_discussed": "It is appreciated if more papers on sidechain flexibility in complex (sidechain prediction or generation) are included. More dicussions on motif-scaffolding, an important functional protein design task are appreciated. Protein structure refinement should also be mentioned.", "other_strengths_and_weaknesses": "1. It is better to provide a comprehensive ablation study for this complex system\n2. The systems are a combination of Alphafold2 and ESM2 and very complicated. More novel and efficient models are appreciated.\n3. The generative framework is not novel. A specialized and efficient generative framework for this complicated task is more appreciated.", "other_comments_or_suggestions": "1. This paper aims to design a fundation model for multi-chain protein complex, according to its first part of Introduction (and the number of tasks it supported), but then it focuses on protein design task. Will APM generalizes well to other protein tasks in addition to protein design tasks? \n2. This paper shows great ambition in providing a fundation model for various protein design tasks, but currently it underperformed in several tasks.\n3. Does APM supports de novo Antibody design?", "questions_for_authors": "1. How effective is the consistency loss for the training of refinement module?\n2. The scaling of structure generative model you mentioned in future work is an open-question, how do you plan to do? I don’t think a stack of structure module in AF2 could achieve scaling because of the nature of 3D representations of protein, i.e, the similar redundency with pixels, where scaling has not succeeded yet. Open discussions are welcome!\n3. How to achieve cond-generation in Figure 3? Which control method do you employ?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ea6iuILvSM", "reviewer": "Reviewer_ZXig", "summary": "This paper proposed APM, a protein full-atom sequence and structure co-generation model. The model includes three parts: Seq & BB module, sidechain module and refine module. The learning is based on flow-matching and is achieved in two stages. The paper did a lot of experiments, both on single protein design and multi-chain protein generation.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and insightful comments that have helped us enhance the clarity and quality of our manuscript. We have carefully addressed your concerns below.\n\n**Q1: Claims And Evidence on Complex Design**\n\nA1: Thank you for your questions. We address your concerns as follows:\n- A1.1 & A1.2: We indeed condition on the target protein in tasks such as antibody design and peptide design. Additionally, the unconditional multimer generation in a \"chain-by-chain\" manner (in appendix), is also part of protein complex design.\n- A1.3: Thank you for your suggestion. We agree that explicitly defining notation for multi-chain proteins will enhance the clarity. In the revision, we will modify the notation system to include multi-chain notations as follows: For a protein $\\mathcal{P}$ consisting of $C$ chains, the $c$-th chain has length $N^{(c)}$, and each modality is represented as the combination of individual chains.\n- A1.4: RMSD is sensitive to local structural variations and can be affected by symmetry considerations during alignment, leading to high RMSD despite overall structural similarity and high TMscores. The image (https://anonymous.4open.science/r/Rebuttal-1784/multimer.pdf) illustrates this scenario, indicating that while RMSD is high, the backbone alignment is nearly perfect. We calculated RMSD for each chain in the multimer folding task, as shown in the table. The results indicate that APM achieves accurate predictions for each chain, outperforming Boltz_woMSA.\n\n|Method|TMscore|RMSD|RMSD_by_chains|\n|-|-|-|-|\n|Boltz|.87/.97|5.40/1.95|1.73/1.01|\n|Boltz_woMSA|.44/.45|17.86/18.43|10.14/10.91|\n|APM|.64/.62|12.60/13.67|5.19/3.12|\n\nThe aim of our work is to model protein complexes. To achieve this, we integrate single-chain data to learn general protein generation and multi-chain data for complex modeling. This approach makes sure that APM is able to handle both single chain and complex tasks. \n\n**Q2: Methods And Evaluation Criteria**\n\nThank you for your comments. We address your concerns as follows:\n- A2.1: Thank you for this suggestion. We will improve the clarity by reorganizing the figures in the main text.\n- A2.2: We would like to clarify the effectiveness of Sidechain Module. During the first stage of pretraining, we trained the Sidechain Module independently, allowing it to learn torsion angle information. In the second stage, we maintained a 50% probability of continuing to train the Sidechain Module, ensuring the network's parametrization captures sidechain information. The effectiveness of Refine Module is demonstrated in Table 4, which serves as an ablation study. The results show that APM with the Refine Module significantly outperforms the version using only the Backbone Module in terms of both complex generation quality and binding affinity.\n- A2.3: Thank you for your observation. We actually selected high-quality samples rather than dropping them, and we will correct this in the revision. We have checked and found only a small number of duplicate samples, which have been removed in training.\n- A2.4: We used AlphaFold2's crop function, which ensures spatial continuity, preventing situations where residues are far apart in space.\n\n**Q3: Experimental Designs Or Analyses**\n\nA3.1: Thanks for your suggestion! We use two longest targets from [2]: 3di3 (binder length 193) and 6m0j (binder length 194). For comparison, we use RFDiffusion to design binders by first generating structures and then designing sequences using ProteinMPNN.\n\nThe table demonstrates that APM outperforms RFDiffusion in terms of dG, with a higher percentage of dG < 0. Additionally, APM shows better foldability, as indicated by higher ipTM, and successfully designs binders for both targets.\n\n|Method|Target|dG|%dG<0|pLDDT|ipTM|Success|\n|-|-|-|-|-|-|-|\n|GroundTruth|3di3|-23.79|-|95.26|.85|100%|\n||6m0j|-20.11|-|81.55|.15|0%|\n|APM_zero-shot|3di3|-80.10|95.00%|78.91|.38|12.5%|\n||6m0j|-96.47|67.50%|69.50|.48|12.5%|\n|RFDiffusion|3di3|-50.49|82.50%|87.83|.30|0%|\n||6m0j|-56.10|67.50%|70.90|.45|0%|\n\nA3.2: We use 80 as the threshold. APM generates the most number of sequences with pLDDT scores above 80. We regret that we are unable to report on PAE at this time, as Boltz does not provide PAE metrics by default.\n\n|Method|Success|\n|-|-|\n|PPFlow|4.92%|\n|DiffPP|7.77%|\n|PepGLAD|5.30%|\n|APM_SFT|19.13%|\n|APM_zero-shot|20.83%|\n\nA3.3: We calculate dG without relax. The results demonstrate that APM maintains better performance compared to others, and successfully generating samples with negative dG.\n\n|Method|dG|%dG<0|\n|-|-|-|\n|GroundTruth|8.82|80.65%|\n|PPFlow|3785.05|0%|\n|DiffPP|1003.63|0%|\n|PepGLAD|663.29|0%|\n|APM_SFT|561.91|0%|\n|APM_zero-shot|428.08|0.05%|\n\nA3.4: Thank you for pointing this out. We will revise our statement accordingly. Due to policy restrictions, we are currently unable to use AF3, which is why we opted for Boltz.\n\n**Q4: Other issues:**\n\nA7: Thanks for your careful review. We will correct mentioned issues.", "claims_and_evidence": "No. The paper claims to design protein complex. However, it seems more like a general protein design model to me:\n\n1. In abstract, the paper  mentioned \"APM is capable of precisely modeling inter-chain interactions and designing protein complexes with binding capabilities from scratch.\" However for complex design, the model should condition on the target protein to design a high-affinity binder protein, right? Otherwise, you can control the designed interface type.\n\n2. In related work, it is not related to protein complex design at all. All the author mentioned are about protein design itself.\n\n3. In method, from the notation to model learning, the authors showed in a single protein manner. Only mentioned about complex design once at line 235-237. \n\n4. In experiments, the authors conducted experiments on both single-chain and multi-chain folding and inverse-folding tasks. While in single-chain task, the model performed well and achieved the best performance on most metrics. However in multi-chain task, the RMSD on folding task is too bad. It seems the model is more suitable for single-chain all-atom protein generation instead of complex design.\n\nTherefore, even though the author claimed \"APM is specifically designed for modeling multichain proteins,\" I feel this paper is more like a general protein design model and actually from the shown performance, it is more suitable for single-chain protein design.", "methods_and_evaluation_criteria": "There are some issues in the methods and Evaluation parts:\n\n1. One of the most biggest problem in methods part is after reading methods section, I really don't know what are the model architectures of the three modules: Seq & BB module, sidechain module and refine module. The only thing I know is they are built upon IPA and Transformer encoder. If I don't read appendix, I don't know the architectures. However, this part is the most vital information, which I think shouldn't be put in appendix.\n\n2. Another issue is the designing of sidechains. It seems the model only predicts the torsion angles based on the predicted $\\hat{S}, \\hat{T}$. it didn't add more information for reconstruct S and T. Why not just using a prediction model at the last step to predict the angles? This is actually this method did in inference. To me, this sidechain prediction here didn't add additional information to the follow-up module. I'm a little doubtful if the refine module is really useful. And there is also no ablation study on different modules, so I can not get a right answer for this point.\n\n3. In section 4.1 data curation line 294, why filter high-quality ones in swiss-prot adn train the model on low-quality ones? Is there any overlap between Swiss-prot samples and AFDB, cuz AFDB predicts the structures of all sequences from Uniprot, including swiss-prot.\n\n4. In section 4.1 cropping, in protein inference, residues are not contiguous, they have contact in 3D space but might be far away from each other in sequence. This way of cropping might cut off the residues in interface, which means the training data might not include complete protein interface in complexes.", "theoretical_claims": "This paper is an application paper and has no theoretical claims.", "experimental_designs_or_analyses": "There are also some problems regard the experiments:\n\n1. The two downstream tasks on complex design are both very short Peptide and only CDR-H3 in antibody (if I understand correctly). That means the model might only be suitable for short protein complex design, which is also aligned with the results on single-chain folding and multi-chain folding tasks. Can the author show some longer protein complex design, like binder design in [1, 2].\n\n[1] Improving de novo protein binder design with deep learning. 2023.\n\n[2] De novo design of high-affinity protein binders with AlphaProteo. 2024. \n\n2. The pLDDT in peptide design is too low, while peptide is short. And the author used threshold 70 to calculate success rate, while usually the threshold should be 80. Can the author show the success rate of using 80 as the threshold and also the performance on PAE ? \n\n3. In the multi-chain design task, all the performance is after pyRosseta relaxation.  Can the author do an ablation study on the model without relaxation by pyRosetta so that we can know the real performance of the designed complexes.\n\n4. In line 343 on the multi-chain folding task, the author mentioned \"there are almost no other models that support multi-chain proteins.\". Actually the author can use alphafold3.", "relation_to_broader_scientific_literature": "Yes", "essential_references_not_discussed": "[1] Improving de novo protein binder design with deep learning. 2023.\n\n[2] De novo design of high-affinity protein binders with AlphaProteo. 2024.", "other_strengths_and_weaknesses": "Please refer to the above sections to see the main weaknesses.\n\nOther minor issues:\n\nThere are many typos and grammar errors:\n\n1. In abstract, \"it can be supervised fine-tuned (SFT)\", supervised fine-tuned is awkward.\n\n2. Fig 1 illustration, pick -> pink.\n\n3. Last line of page4, \"All the details refer to Appendix B.\"  -> Refer to Appendix B for all the details.\n\n4. page 6 line 325, \"folding, inverse-folding, and inverse-folding\"", "other_comments_or_suggestions": "Please see above.", "questions_for_authors": "Please see above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel approach to model multi-chain proteins for designing protein complexes and performing folding and inverse-folding tasks. The methodology achieves strong empirical results on pertinent datasets.

The AC and reviewers all appreciate the contributions of this paper. The authors have done a great job at addressing the reviewers comments and the materials provided during rebuttal significantly strengthen the results. We strongly encourage the authors to incorporate them in the revised manuscript. In particular it would be important to include statistical significance results, ProGen perplexity results, the binder task results, and the updated results on unconditional single-chain generation.
Decision: Accept (poster)