P(all-atom) Is Unlocking New Path For Protein Design
Authors: Wei Qu
Jiawei Guan
Rui Ma
kezhai
Weikun.Wu
Haobo Wang
Venue: ICML 2025 spotlightposter (2025)
Keywords: Proteins
Generative models
Co-design
All-atom
Primary area: applications->health_medicine
TL;DR: A state-of- the-art all-atom protein generative model.

Abstract: We introduce Pallatom, an innovative protein generation model capable of producing protein structures with all-atom coordinates. Pallatom directly learns and models the joint distribution $P(\textit{structure}, \textit{seq})$ by focusing on $P(\textit{all-atom})$, effectively addressing the interdependence between sequence and structure in protein generation. To achieve this, we propose a novel network architecture specifically designed for all-atom protein generation. Our model employs a dual-track framework that tokenizes proteins into token-level and atomic-level representations, integrating them through a multi-layer decoding process with "traversing" representations and recycling mechanism. We also introduce the $\texttt{atom14}$ representation method, which unifies the description of unknown side-chain coordinates, ensuring high fidelity between the generated all-atom conformation and its physical structure. Experimental results demonstrate that Pallatom excels in key metrics of protein design, including designability, diversity, and novelty, showing significant improvements across the board. Our model not only enhances the accuracy of protein generation but also exhibits excellent sampling efficiency, paving the way for future applications in larger and more complex systems.

P(all-atom) Is Unlocking New Path For Protein Design
Wei Qu 12 Jiawei Guan? Rui Ma
Ke Zhai
Weikun Wu
Haobo Wang"
Abstract
tein structure prediction task, which involves determin- ing the three-dimensional structure of a protein given its amino acid sequence (Abramson et al., 2024; Jumper et al., 2021; Lin et al., 2023; Baek al., 2023). The latter, P(seq backbone), underpins the fixed-backbone design task, where the goal is to identify sequence that will fold into a given protein backbone structure (Dauparas et al., 2022; Hsu et al., 2022). In summary, these probability dis- tributions has successfully advanced the field of protein engineering.
We introduce Pallatom, an innovative protein generation model capable of producing protein structures with all-atom coordinates. Pallatom directly learns and models the joint distribution (ssructuure,sey by focusing on P(all-atom), ef- fectively addressing the interdependence between sequence and structure in protein generation. To achieve this, we propose a novel network archi- tecture specifically designed for all-atom protein generation. Our model employs a dual-track framework that tokenizes proteins into residue- level and atomic-level representations, integrating them through multi-layer decoding process with "traversing" representations and recycling mech- anism. We also introduce the at om14 4 represen- tation method, which unifies the description of unknown side-chain coordinates, ensuring high fidelity between the generated all-atom confor- mation and its physical structure. Experimental results demonstrate that Pallatom excels in key metrics of protein design, including designabil- ity, diversity, and novelty, showing significant im- provements across the board. Our model not only enhances the accuracy of protein generation but also exhibits excellent sampling efficiency, paving the way for future applications in larger and more complex systems.
With the advancement of deep learning in protein science, two distinct approaches for protein design have emerged. One approach is the protein hallucination (Anishchenko et al., 2021), which explores the landscape of a P(structure seq) model using Monte Carlo or gradient- based optimiza- tion techniques. This method yields valid protein structures, but requires an additional P(seq) model, such as protein lan- guage models (Rives al., 2021), to correct or redesign the sequence. Essentially, this approach can be viewed op- timization process of P(structure seq) P(seq). Another approach attempts to explore the P(backbone) distribution. a series of protein generation models based on SE(3) invari- ance or equivariance networks (Jing et al., 2020; Satorras et al., 2021) have recently emerged, these method rely on an additional P(seq backhone) process determine the pro- tein sequence. This optimization strategy can be regarded as P P(backbone) P(seq backbone).
This step-wise design process has limitation in approximat- ing the joint distribution through marginal distributions. The P(structure seq) P(seq) strategy faces challenges when sampling the high-dimensional sequence space, while the P P(backbone) P(seq backbone) strategy fails to account for explicit side-chain interactions and is bottlenecked by the capability of the fixed-backhone design model.
1. Introduction
The theoretical foundation of protein modeling has been built upon two key conditional probability distributions: P(structure seq) and P(seq backbone). The for- mer, P(structure seq), corresponds to the all-atom pro-
The ultimate goal of protein generation is to directly ob- tain a sequence along with its corresponding structure, i.e., to develop a model capable of describing the joint distribu- tion P Psstructure, seq) or P(baakbone, seq). Recently, some studies have started to adopt co-generation approaches, such as model based on co-diffusion (Campbell et al., 2024) or co-design (Ren et al., 2024). While these methods primar- ily rely on SE(3) networks, they still separately model the
"Equal contribution 'Institute of Science and Technology for Brain- Inspired Intelligence, Fudan University. Shanghai, China "LEVINTHAL Biotechnology Lt.Ltd, Hangzhou, China. Corre- spondence to: Weikun Wu <weikkn, wu bivo. Haobo Wang <haobowang @ livinthal..bo...
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Pallatom
backbone and sequence, without considering side-chain con- formations and leading an insufficient description of the structure. Protpardelle (Chu et al., 2024), an all-atom protein diffusion model, similarly adopts co- generation approaches with an explicit all-atom representation, taking a step fur- ther in the field. However, the experimental results indicate that the generated sequence fails to accurately encode the intended fold, necessitating an additional round of sequence redesign and side-chain refinement.
transformations of backbone and side-chain frames. Build- ing upon this foundation, AF3 introduces an innovative point cloud representation that directly encodes all-atom 3D coordinates in Cartesian space. However, these representa- tions are fundamentally inadequate for protein generation tasks due to a critical limitation: the complete absence of sequence information at the initialization stage prevents de- termination of the system's 's total atomic cardinality. To over- come these limitations, Protpardelle introduces the a at atmm73 representation, which implements quantum-inspired su- perposition framework. This approach stores coordinate information for all 20 possible side-chain configurations relative to aligned backbone atoms, enabling probabilistic representation of unknown amino acid types. The superpo- sition collapses definitive state upon sequence determi- nation, effectively resolving the initial uncertainty in atomic configuration.
In this study, we introduce novel approach for all-atom protein generation called Pallatom. Our extensive experi- ments show that by learning P(all-atom), high-quality all- atom proteins can be successfully generated, eliminating the need to learn marginal probabilities separately. To ad- dress the first critical challenge of representing side-chain coordinates for unknown amino acids in protein genera- tion, we introduce a atoml4 4, a novel all-atom representation framework. This approach employs virtual atoms across all amino acid types, effectively preventing sequence informa- tion leakage while maintaining structural integrity. Inspired by AlphaFold3 (AF3) (Abramson et al., 2024), we propose dual-track representation framework that encodes protein structures through concurrent residue-level and atom-level tokenization schemes. Central to this framework is the AtomDecoder unit, which seamlessly integrates and updates features across multiple representation spaces while simul- tancously executing coordinate prediction and diffcrentiable recycling process within a unified computational architec- ture. The fundamental insight underlying Pallatom is the recognition that protein all-atom coordinates intrinsically capsulate both structural and sequential information within their spatial configuration. Directly learning P(all-atom) opens a new path for co-generative modeling of structure and sequence.
2.2. Diffusion modeling on all-atom protein
The all-atom representation bypasses the complexities of SE(3) frames (Yim et al., 2023b) and Riemannian diffusion (De Bortoli et al., 2022). Gaussian-based diffusion models provide a robust theoretical framework, with EDM (Kar- ras et al., 2022) emerging as prominent approach. EDM has been successfully implemented in Alphafold3 and Prot- pardelle for all-atom coordinate diffusion, showcasing its efficacy in protein generation. We briefly outline EDM's mechanism below.
Assuming the data distribution of all-atom coordinates un- der any representation as pdata (x) with standard deviation data- the forward process involves adding Gaussian noise of varying scales to generate a series of noised distribu- tions (x) [pdata * ~N, (z) & p(x: o)) with particular configuration (() 1 and s(l) = When mmar ddeta Pamax (x) (0, af)) approximates pure
Our contributions are summarized as follows:
Gaussian noise. The probability flow ordinary differential equation (ODE) is given by:
We explore the atoml4 representation to achieve unified description of unknown amino acid side-chain coordinates in generative tasks.
dx (t(() Vx (oog((x;((()t))
(1)
We develop a network architecture for all-atom pro- tein generation tasks, which effectively represents both protein backhones and sidechains.
Here, Vx log 7) is the score function, which does not depend on the normalization constant of the underlying den- sity function pt (x). A neural network D [x a) is typically trained for each a using the following loss function to match the score function (Song et al., 2021):
We use our framework to develop Pallatom, state-of- the-art all-atom protein generative model.
+ +n; -y
(2)
2. Preliminaries
2. All-atom modeling and representation
Vx lx (Do(x; -x//0²
(3)
Recent advances in protein structure prediction have estab- lished diverse methodologies for all-atom representation. AlphaFold2 (AF2) (Jumper et al., 2021) pioneered a frame- based approach, modeling atomic coordinates through SE(3)
In practice, EDM introduced preconditioning technique that stabilizes training by adjusting noise-related scaling coefficients, ensuring that the model's inputs and outputs
2

---

Pallatom
remain within stable numerical range each time step. Consequently, Dg and the loss function can be derived as:
3. Method
3.1. atom14: the all-atom representation
The all-atom protein generation model faces many chal- lenges in constructing both backbone and side-chain atoms. A pivotal initial question arises: How to represent a system with variable number of atoms? At the initial sampling stage, both the backbone and sequence are unknown, how- ever, the atom number of a system depends on unique se- quence, once the sequence is determined, it also dictates the structure.
(x(x;o) = Cakiplo)) (ox) + Cout (a) Fo (o))))))))))))))) (noise(a))
Eo En ((or) (o)
(y + n), Croise (o))
(y Co) (y + =
(4)
where Oskiplo) = oo2 + o'datad, Cone (a) 'daata/ Vo + t Cin(a) = 1/ 1/ 2 + danta (n) a (n) and \(o) = (oo)=1/cowu) (/)) represent skip scaling, output scaling, input scaling, noise conditioning, and the Coefficient respectively.
=
To avoid potential conflicts arising from the simultane- ous design of sequence and structure, we introduce the atom14 representation, establishing unified framework through virtual atom integration that normalizes heavy atom counts across all amino acid types. This padding strat- egy that empirically positions additional virtual atoms to align with the Ca coordinates of each amino acid residue. Through this representation, a protein with N residues, de- noted as
2.3. Framework for structure-based protein generation
Currently, the field of protein structure design lacks com- prehensive methods for all-atom coordinate generation. The most relevant approaches are structure- backbone de- sign or sequence- bequence-backbone co-generation methods, which typically rely on a core building block to perform multi-view feature fusion and coordinate updates on protein representa- tions (e.g., 1D single-, 2D pair-, and 3D frame-ember eemeeddn By stacking multiple such blocks, these methods progres- sively refine protein structures. For instance, RFDiffusion (Watson al., 2023), built on Rose Follold (Baek al., 2021), integrates information from three tracks within each block, continuously updating and fusing representations through sophisticated network operations. Similarly, Multi- Flow (Campbell et al., 2024) and CarbonNovo (Ren et al., 2024) adopt Alpahfold2's Invariant Point Attention (IPA) layer as their building block, augmented with additional mechanisms to refine pairwise and sequence information. Through multi-view feature fusion and coordinate updates, these methods achieve precise backbone generation.
[ is transformed into a fixed-size point cloud x" E R N41433 For example, if residue x; is CYS, its coordinates N. C, Cg, C,O, E RRx3 are augmented with eight virtual atoms aligned at the Ca po- sition, yielding x E R14x3 We denote the system as F = {x) Fi E R 1.2...., L} with L = N x 14 and define the noise level as f.
The primary task of all-atom generation involves learning the score function vp log p(F') through a diffusion pro- cess that progressively perturbs the initial state Fo each noise level i. While the virtual atom framework inherently precludes explicit dlement-specific assignments, we posit that the all-atom coordinate distribution sufficiently encodes essential sidechain properties, including hydrophobicity, po- larity, hydrogen bonding, and salt bridge formation. There- fore, we introduce an auxiliary visualization module that predicts amino acid types. With discarding the redundant virtual atoms based on the predicted amino acid type as post-processing step, we can generate all-atom proteins with corresponding sequences from a 3D point cloud noise. Details are presented in Figure 3 and Appendix C.1
In contrast, ProtPardelle uses a simpler U-ViT (Bao et al., 2023) architecture to generate all-atom proteins but relies on a post-processing miniMPNN module to redesign sequences and guide side-chain placement, which limits its robustness. This highlights key limitation of simplified frameworks: their inability to effectively integrate diverse information sources. In contrast, the complex network frameworks offer significant advantages to incorporate additional auxiliary information. For example, RFDiffusion employs a self- conditioning mechanism (Chen et al., 2022) to generate preliminary structures as 2D templates for guiding sam- pling, while CarbonNovo leverages encoded single- and pair-features to perform sequence sampling through Markov Random Field (MRF). These strategies significantly im- prove the diversity and quality of generated proteins. Thus, exploring all-atom generation frameworks that integrate multiple encoding forms remains a critical research direc- tion.
3.2. MainTrunk: the denoising network
We refer to the protein generation task as the generation of all-atom coordinates. In the atoml4 4 representation, a protein with I. atoms can be expressed as Fd) = ( which Fọ E R represents an atom coordinate. For each noise level in the diffusion process, the network predicts the updated coordinates plenoised from the input ginput
The network comprises two main components: feature encoder and iterative decoding units. Figure 1A illustrates the main architecture. We implement a dual-track represen-
3

---

Pallatom
A
MainTrunk
Denclsed Coordinatss
Swi-Dondtion
AlomDacader Unil(1)
AlomDecader Unil(K)
Sequenne Loght
AtomFeelurre
Singlo
Dislogram Prodiction
Backform
NomConditoo
NomCondttoo
Traversing AlsrFaastune
B
AtomDecader
Update incommne"
Transtion
ooudgoinal
Attrricn fllw
Attansion
Trandiio
Dradaant Doadan
Resicvel
Racycing Ing
NommAtoniion Decoder
Lansool
Figure I. Pallatom model framework: (A, B) Architecture of the network and the AtomDecoder unit.
tation framework, employing local attention mechanisms for atomic-level feature extraction and global attention mecha- nisms for residue-level feature integration.
level, while atomic information is subsequently integrated back through a differentiable recycling process. These oper- ations are encapsulated within AtomDecoder unit (Figure IB), which is designed to continuously update the coordi- nates. In practical implementation, we identify funda- mental challenge in the decoding architecture: preserving residual connections hetween residue-level and atomic-level embeddings across successive decoder units may result in significant information redundancy, especially during the residue-level feature broadcasting process (where "feature broadcasting" refers to the process similar to AF3's ap- proach of replicating residue-level features and incorporat- ing them into corresponding atom-level features). A com- prehensive analysis of this phenomenon and our solution is provided in Appendix C.5. To address this challenge, the three "traversing" atomic embeddings serve as stable information carriers, facilitating efficient residue-level prop- agation while enabling coordinate refinement through the accumulated pppaltes in the k-th unit. (k)
For feature initialization and encoding (Algorithm 4), We first initialize "traversing" atomic embeddings (c) 9i skip and Pim by integrating three key components: (1) structural features from alanine's standard conformation establish stable backbone frame, (2) residue positional encoding, and (3) noisy coordinate vectors. For residue-level embed- ding, single embedding (s,) is initialized using diffusion timestep encoding and residue positional encoding, while pair features (2iz) incorporate relative positional encoding and self-condition template distograms from previous pre- dictions. Detailed features recorded in Appendix Table 5.
In the decoding phase, we implement an iterative refine- ment mechanism that facilitates bidirectional information flow: residuc-level embedding are propagated to atomic-

---

Pallatom
Algorithm Pallatom Inference
denoised (()
Cskip(() (t)
panppr + Cout
adaates
(R)
(5)
1: def SampleDiffusion (ff*). T 200, X 1.003,
" 2.25, 7oo 0.2, Emin 0.01, Lina 1.0)
Following coordinate denoising, the partially denoised posi-
2: 6 1/T
tions inlonossed are transformed into relative distance matrices
(k)
3: cr = GeiNoiseSchedulle(1 - uniform(0, 1) 66)
and subsequently recycled into the Zij pair embeddings.
4: Fę ~ cT -V (3)
Concurrently, the triangle update module selectively main- tains physically valid triangular constraints while eliminat- ing geometrically inconsistent features, enabling iterative improvement of structural quality throughout the decoding process. Details can be found Appendix Algorithm 2.
5: # Initialize the self-condition feature
6: cruplate.dsstoggee
0 (
for all t (...... do
8: i, t/T uniformm0. 1) -g
9: G GeiNoiseSchedull((,)
10: C--1 1 = GerNoiseSchhduule( - 8.)
3.3. SeqHead: sequence decoder
11:
ri +
The atoml 4 representation. while effective for structural denoising, inherently lacks elemental information in the gen- erated side chains, preventing direct extraction of protein sequence information from the atomic coordinates alone. To bridge the gap between structural generation and biological interpretation, we introduce a SeqHead module that con- verts the predicted atomic coordinates into corresponding amino acid sequences, enabling the transformation of geo- metric data into biologically meaningful protein structures.
12:
Yo if Eiin S t/T S tmaz else 0
13:
= cr +1
14:
ri moisy = ri + X - 2 1(
15:
# Update the self- celf-condition feature
Fi dennised pgaa logits MainTrunk(
16:
U F7 Ennossy t,)
17:
18:
stemplateddistogrraa distogram + Distogram [[]
denoised
19:
Calculate the score function
Specifically, we aggregate the atomic embedding (q:) corre- sponding to each residue and process them through linear transformation layer to predict probability distributions over the 20 canonical amino acid types (()) E IR N200 We take
20:
r
denoised
,4,000 _.0q.
MainTTunk(
21:
{
noiss .ittp) rl
22:
5 = (17
Fi Fedenoised )/C
23:
dt ---1-i
-
the predictions from the last unit as the final sequence logits,
24:
Fi r.moisy + n7 dt dt. 35
â(K)-
25: end for
26: return (ri). Joggis
3.4. Training Loss
Our training method mainly follows the application and im- provements of the EDM framework. The denoising all-atom positions score-matching losses are described according to (2). Given that the network architecture lacks inherent equivariance constraints and iteratively refines coordinates across multiple decoding stages, it is imperative that the coordinates transformations between these stages remain invariant under changes in orientation to ensure consistent geometric interpretations. Therefore, we employ an aligned MSE loss. We first perform rigid alignment of the ground truth TO on the denoised structure pilemoised using the Kabsch algorithm, yielding piligned, The MSE loss is then defined as: Latom
cal distance difference test (LDDT) loss from AF3 Lamooth www. (prhenoised, )r)). This can be found in the Ap- pendix algorithm 8. Additionally, we implement two distogram- loss function Ldistres and Edistaton to en- force global/local distance constraints at the residue/atomic level, ensuring proper maintenance of overall protein topol- ogy throughout the optimization process. We introduce an intermediate layer loss Lmedd that supervises the sequence and structure predictions from each AtomDecoder unit, sig- nificantly enhancing both model performance and inference stability through progressive refinement. Details of training losses can be found in Appendix D.
gdrenssal palines
3L
The total loss can be written as:
For sequence decoding, we use the standard cross-entropy loss function to evaluate the difference between the pre- dicted sequence å and the true sequence ag- The loss func- tion is defined Loeg CE(â, The primary loss function of the network is:
L = Lu + 01 Camooth Iddt + 022 Laist res
+ 03 Edist + a4 . Emed
3.5. Sampling
Lo = A(f) Latom + an Lseq
(6)
The sampling process is described in Algorithm 1. The initial atoms are sampled from a Gaussian distribution on R We only use the first-order Euler method as the ODE solver with T steps for discretization. Optionally, additional
To capture the fine- ane-grained characteristics of the all- atom structure, we introduce the simplified smooth lo-
5

---

Pallatom
noise can be injected during the sampling steps to introduce stochasticity into the ODE solving process. We focus only on the sequence distribution decoded by the network in the final sampling step, employing a low-temperature softmax strategy to derive an approximate discrete one-hot amino acid sequence as the final sequence.
designable sequences.
The last criterion is structure novelty, which evaluates the structural similarity between the generated backbones and natural proteins in the PDB, denoted as NOV-str. This is calculated by the T'-ssore of the generated designable backbones compared to the most similar proteins in the PDB. In our evaluation, we use the following two modes:
4. Experiments
4.1. Training setting
For methods that can predict both all-atom coordinates and sequences, DES-aa is used. Diversity and novelty are evaluated based on the all- atom designable proteins.
CO-DESIGN 1:
The training dataset of the model includes the PDB (Zardecki et al., 2022) and AlphaFold Database (AFDB) (Varadi et al.. 2021). We performed rigorous data cleaning on augmented data from AFDB to obtain high-quality re- sults. Details can be found in the Appendix B. We focus on small monomer proteins that can be easily synthesized using commercial oligo-pool method and the models are trained on crops of lengths up to 128. The model training utilized the Adam optimizer (Kingma & Ba, 2017) with learning rate of 1e-3, 31 = 0.9, B2 = 0.999, and a batch size of 32. Details are provided in the Appendix Table 6.
PMPNN 1: Other methods use DES-bb with Nseg = 1. Specifically, we calculate and display the two DES-bb (w/wo) under conditions with and without the pLDDT 80 constraint. Diversity and novelty are evaluated based on the proteins filtered by DES-bb (w).
4.3. Results
We sample Pallatom with 200 time steps using a noise scale
4.2. Metrics
% = 0.2, a step scale 1 = 2.25 and evaluate 250 proteins sampled for each length L = 60, 70, 80, 90, 100, 110, 120. Our primary comparisons are with state-of-the-art methods capable of generating all-atom proteins, such as Protpardelle and ProteinGenerator (Lisanza et al., 2023). For backbone generation, we compare with RFdiffusion. We also com- pared Multiflow, which is capable of generating both back- bones and sequences. All methods are evaluated using their open-source code and default parameters.
While some evaluation criteria used for protein backbone generation are not suited for the new task, we propose new metrics specifically designed for assessing all-atom protein generation.
The first criterion is structure designability. The self- consistency process assesses the designability of protein backbones (DES-bb). This involves using a exed-backboone design model (e.g.. ProteinMPNN (Dauparas et al., 2022)) to generate Naeq sequences for the backbone, which are then folded by structure prediction models like ESMfold (Lin et al., 2023). The backbone's designability is evaluated by the optimal TM-score or CMMRMSD between the folded and original backbones. However, this metric is not suit- able for all-atom proteins, which include side-chain atoms. Therefore, we similarly define the designability of all-atom protein generation, denoted as DES-aa. For the all-atom proteins, the sequence is used to predict the structure, and the sample is considered designable if the mean PLDDT of the predicted structure exceeds 80 and the all-atom RMSD (aaRMSD) is less than 2 A. This metric ensures atomic-level accuracy and provides strong confidence in the structural integrity and designability of the predicted protein.
As shown in Table 1 and Figure 2A, Pallatom demonstrates superior performance in the CO-DESIGN I benchmark for all-atom protein generation. Remarkably, despite lacking specific training on Exek-back design tasks, Pallatom achieves comparable results to ProteinMPNN by predict- ing sequences with a single linear layer derived from ag- gregated atomic embedding. Comprehensive analysis of sequence quality between our design and ProteinMPNN. including detailed performance metrics, is presented in the Appendix G.4. These results substantiate Pallatom's 's capac- ity for all-atom structure design, validating our hypothesis that modeling P(all-atom) effectively captures the funda- mental relationship between protein structure and sequence. Notably, Pallatom generates all-atom protein structures with substantially enhanced structural diversity, outperforming existing methods by achieving significantly higher diversity than both Multiflow and ProteinGenerator, while simulta- neously attaining the highest sequence diversity across all evaluated approaches.
The second criterion is structure diversity, denoted as DIV- str. This can be quantified by calculating the clusters num- ber of the designable structures using Foldseek ( (Van Kem- pen al., 2024). For all-atom proteins, we use similar diversity evaluation method for the generated sequences, denoted as DIV-seq. Specifically, we use MMseq2 (Steineg- ger & Söding, 2017) to calculate the clusters number of the
In the PMPNN benchmark, we observe that the DES- bb(wo) metric systematically overestimates designability relative to the pl.DDDT-constrained DES--b(y metric, as
6

---

Pallatom
Table Comparison of various methods. Protpardelic utilized ProteinMPNN auxiliary tool in all-atom proteins generation, resulting in identical results for CO DESIGN and PMPNN 1. In the case of Multiflow, which can only generate protein backbones and sequences without side chains, the reported DES metric based on CMRMSD rather than aaRMSD.
CO-DESIGN
PMPNN 1
Method
DES-aa (1)
DIV-str/seq (1)
NOV-str (1)
DES-bb (w/wo) (1)
DIV-str (f)
NOV-str (L)
Protpardelle*
30.00%
15 / 26
0.747
30.00% (80.23%)
15
0.747
ProteinGenerator
43. 14%
83 / 706
0.791
93.14% (96.34%)
151
0.785
Multiflow
62.74%
134 / 1042
0.753
84.69% (95.43%)
184
0.744
RFdiffusion
N/A
N/A
N/A
78.29% (84.40%)
165
0.805
Pallatom
85.03 %
291 1466
0.719
89.89% (94.46%)
318
0.716
stom RMSD: 0.423
RMSD:00844
RMSD: 0.8D:00.86 mean PLDDT: 08
mean PLDDT: 80.71 maxTM Mccccor::
mean PLDDT: 90.72
mean PLDDT: 84.42
maxTM iscore: 0.54
maxi MIXTTM--S M-scorr:: 0.633
maxTM-score: 0.53
Figure 2. Evaluation of proteins sampled from Pallatom. (A) Boxplot of aaRMSD for proteins sampled by various methods under the CO-DESIGN mode. Multiflow exhibits the C. RMSD. (B, C) The proportions secondary structures designable proteins across different lengths presented for CO-DESIGN and PMPNN I modes across various methods, with the total height of the y-axis representing the designability. (D) Examples of high-quality, novel all-atom proteins sampled by Pallatom.
demonstrated by substantial discrepancies in Protpardelle's designability evaluations, highlighting the importance of quality constraints in designability assessment. The back- bone generative models like ProteinGenerator, Multiflow and RFdiffusion demonstrate comparable performance. However, Protpardelle demonstrates suboptimal perfor- mance across all evaluation metrics, underscoring the crit- ical importance of both network architecture design and atomic representation strategies developing effective gen- erative models for protein design. Consistent with CO- DESIGN performance, Pallatom demonstrates robust des- ignability while simultaneously generating structurally di- and novel protein conformations, effectively balancing these traditionally competing objectives. A detailed analysis
in Figure 2BC shows Pallatom's 's stable performance across protein lengths and balanced secondary structure distribu- tions, in contrast to the preference of comparative methods for specific secondary structures like a-helical ones. The studies in Figure 2D highlight Pallatom's ability to design highly ordered proteins with hydrophobic cores and hydrophilic surfaces, consistent with high pLDDT structures predicted by ESMFold, underscoring its understanding of protein folding principles.
assess generalization capabilities, we extensively evalu- ated Pallatom's 's performance on out-of-distribution (OOD) protein lengths, conducting systematic sampling and anal- ysis across broad range of 150-400 residues. The results

---

Pallatom
demonstrate that Pallatom exhibits exceptional scalability, achieving the highest designability even over twice the maximum training length (L 128). We also analyze sec- ondary structure preferences in generated proteins across different methods, particularly for longer sequences, with comprehensive distribution statistics and detailed compar- isons provided in the Appendix G.I. We also performed statistical analysis of sampling duration, which demon- strates the efficiency of Pallatom. The results are presented in Appendix G.7.
superior designability performance. Table 3 presents the results, with the first row (atom14) serving the baseline. We first conducted an ablation study on the atoml 4 repre- sentation. For comparative analysis, we implemented the hybridi 4 representation and retrained the model. Similar to Protpardelle's methodology. the hybride aebridl 4 representa- tion maintains a superposition of 20 possible side-chain con- figurations, which collapse into a single hybridized residue state based predicted sequence probabilities during the generation process. However, as shown in the second row, the sequence- eqquence-guided all-atom approach not only failed to improve performance but actually resulted in reduced se- quence quality (lower DES-aa) and compromised backbone generation (decreased DES-bb), revealing fundamental lim- itations in this design strategy. We hypothesize that this phenomenon stems from the absence of well-defined map- ping between noisy coordinate space and sequence space during diffusion. The sequence predictions derived from noisy structural intermediates exhibit substantial deviations from native foldable sequences, and these discrepancies are amplified throughout the reverse diffusion process, ul- timately hindering the generation of biologically plausible sequences.
Table Pallatom sample metrics,
Noise Level 70
0.2
0.2
0.1
0.2
0.2
Step Scale 17
1.75
2.25
2.25
2.75
3.25
Nsteps
200
200
200
200
200
DES-aa (1)
57%
87%
89%
94 %
93%
DIV-str (1)
56
64
52
55
46
HHH(%)
28%
28%
45%
35%
38%
HEL(()
56%
54%
44%
47%
51%
EEE( ())
16%
18%
11%
18%
11%
4.4. Hyperparameter
We next assessed the impact of the differentiable recycling process (woRC, shown in the third row). Our evaluation indicated that while the recycling process had little effect on backbone quality (DES-bb bb remained almost unchanged), it significantly impacted sequence generation, leading to a substantial reduction in DES-aa. These results highlight the crucial role of the recycling process in the components and its differential influence on various aspects of protein design. The detailed setup of the ablation studies can be found in the Appendix E.
We analyzed the impact of the sampling parameters and Table presents the metrics for Pallatom when sampling 250 proteins with L 100. We observed that, under the same noise scale, increasing the step scale 1) leads to corre- sponding rise in designability, well as an increase in the proportion of all-helix structures within the secondary struc- ture. However, this improvement in designability comes at the cost of reduced structural diversity, indicating a trade-off between these two metrics. We further demonstrated the im- pact of varying step scales on the sampling of OOD-length proteins, with detailed results provided in the Appendix G.3. Furthermore, found that reducing the additional noise level to = 0.1 slightly enhances designability but also significantly decreases structural diversity, consistent with the findings of previous work (Yim et al., 2023b).
5. Discussion
We introduce Pallatom, a highly efficient end-to-end all- atom protein generation framework that simultaneously cap- tures the relationship between sequence and structure, en- abling state-of-the-arr performance. Our proposed all-atom protein at om14 4 representation eliminates the constraints of explicitly defining all amino acid types during generation, of- fering a more precise approach for representing coordinates in all-atom systems. We have redesigned fundamental unit capable of fusing, recycling and updating diverse fea- tures of all--atom proteins while predicting their coordinates. The new framework efficiently adapts to all-atom protein structure diffusion generation. The results demonstrate that models learning P(all-aton) exhibit strong performance and diversity in de novo protein generation, unlocking new pathways for protein design. Future work includes devel- oping more generalized model architecture and extending our framework to enable the design of complex systems.
Table Ablation Studies evaluating the key concept of Pallatom to designability.
Method
DES-aa
DES-bb(w)
DES-bb(wo)
atom14
87%
95%
95%
hybrid14
5%
68%
82%
woRC
21%
93%
95%
4.5. Ablation Studies
Building upon the hypothesis that all-atom coordinates in- trinsically encode essential protein information, we system- atically investigated the key factors underlying Pallatom's is
8

---

Pallatom
Dauparas, J., Anishchenko, I., Bennett, N., Bai, H., Ragotte, R. J.. Milles, L. F., Wicky, B. L M., Courbet, A., De Haas, R. J.. Bethel, N., Leung, P. J. Y., Huddy, T. F., Pellock, S., Tischer, D., Chan, F., Koepnick, B., Nguyen, H., Kang, A., Sankaran, B., Bera, A. K., King, N. P., and Baker, Impact Statement
Baker, D., and DiMaio, F. Efficient and accurate pre- diction of protein structure using RoseTTAFold2, May 2023.
The novel protein generation method proposed this pa- per, which is based learning the distribution of all-atom coordinates, aims to advance the paradigm of protein de- sign engineering. This work holds significant potential for broad societal impact and direct applications. For instance, it offers greater utility in tasks such as enzyme design, drug design involving small molecule binding pockets, and other applications requiring all-atom protein constraints.
Bao, F., Nie, S., Xue, K., Cao, Y., Li, C., Su, H., and Zhu, J. All are Worth Words: ViT Backbone for Diffusion Models, March 2023.
Barrio- Herio-Hernandez, I., Yeo, J., Jänes, J., Mirdita, M., Gilchrist, C. L. M., Wein, T., Varadi, M., Velankar, S., Beltrao, P., and Steinegger, M. Clustering predicted struc- tures the scale of the known protein universe. Nature, 622(7983))637-645 October 2023. ISSN 0028-0836, 1476-4687. doi:
Code Availibility
The Pallatom is available GitHub (https://gith))
Campbell, A., Yim, J., Barzilay, R., Rainforth, T., and Jaakkola, T. Generative Flows Discrete State-Spaces: Enabling Multimodal Flows with Applications to Pro- tein Co-Design. In Forty-first International Confer- ence on Machine Learning, 2024. URL https:// openreview net forum? ? i id=k@WSbvBBR.
References
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T.,
Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., Bodenstein, S. W., Evans, D. A., Hung, C.-
C., Neill, M., Reiman, D., Tunyasuvunakool, K., Wu, Z., Zemgulyte, A., Arvaniti, E., Beattie, C., Bertolli, O., Bridgland, A., Cherepanov, A., Congreve, M., Cowen- Rivers, A. I., Cowie, A., Figurnov, M., Fuchs, F. B., Gladman, H., Jain, R., Khan, Y. A., Low, C. M. R., Per- lin, K., Potapenko, A., Savy, P., Singh, S., Stecula, A., Thillaisundaram, A., Tong, C., Yakneen, S., Zhong, E. D., Zielinski, M., Zidek, A., Bapst, V., Kohli, P., Jaderberg, M., Hassabis, D., and Jumper, J. M. Accurate structure prediction of biomolecular interactions with AlphaFold Nature, 630(6000)))))) (00000000)9993330 June 2024. ISSN 0028-0836, 1476-4687. doi: 10.
Chen, T., Zhang, R., and Hinton, G. Analog bits: Gen- erating discrete data using diffusion models with self- conditioning, arXiv preprint arXiv:2088.04202. 2022.
Chu, A. E., Kim, J., Cheng, L., El Nest, G., Xu, M., Shuai, R. W., and Huang, P.-S. An all-atom protein generative model. Proceedings the National Academy of Sciences, 121(27) 15001211 July 2024. ISSN 0027-8424, 1091- 6490. doi: 10. 10.1073/pnas.231 1500121.
Anishchenko. I., Pellock, S. J., Chidyausiku, T. M., Ramelot, T. A., Ovchinnikov, S., Hao, J., Bafna, K., Nor,, C., Kang, A., Bera, A. K., DiMaio, F., Carter, I., Chow, C. M., Montelione, G. T., and Baker, D. De novo protein design by deep network hallucination. Nature, 600(7889): 547-552, December 2021. ISSN 0028-0836, 1476-4687. doi: 10. (088//41556-002-04411
D. Robust deep learning- based protein sequence design using ProteinMPNN, Science, 378(6615):49 October 2022. ISSN 0036-8075, 1095-9203. doi: 10. 126/science. add2187.
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., Millán, C., Park, H., Adams, C., Glassman, C. R., DeGiovanni, A., Pereira, J. H., Ro- drigues, A. V., Van Dijk, A. A., Ebrecht, A. C., Opperman, D. J.. Sagmeister, T., Buhlheller, C., Pavkov-Kelle T., Rathinaswamy, M. K., Dalwadi, U., Yip, C. K., Burke, J. E., Garcia, K. C., Grishin, N. V., Adams, P. D., Read, R. J., and Baker, D. Accurate prediction protein struc- tures and interactions using three-track neural network. Science, 373(6557):871-88))) 8075, 1095-9203, doi: 10. 1..116//ciencc.ab888555.
De Bortoli, V., Mathieu, E., Hutchinson, M., Thortoon, J., Teh, Y. W., and Doucet, A. Riemannian Score-Based Generative Modelling, November 2022.
Hsu, C., Verkuil, R., Liu, J., Lin, Z., Hie, B., Sercu, T., Lerer, A., and Rives, A. Learning inverse folding from millions of predicted structures, April 2022.
Huguet, G., Vuckovic, J., Fatras, K., Thibodeau-Laufed E., Lemos, P., Islam, R., Liu, C.-H., Rector-Brooks, J., Akhound-Sadegl T., Bronstein, M., Tong, A., and Bose, A. J. Sequence-Augmentt SE(3)-Flow Matching For Conditional Protein Backbone Generation, May 2024.
August 2021. ISSN 0036-
Baek, M., Anishchenko, Humphreys, I. R., Cong. Q.,

---

Pallatom
Ingraham, J. B., Baranov, M., Costello, Z.,, Barber, K. W., Wang, W., Ismail, A., Frappier, V., Lord, D. M., Ng- Thow-Hing, C., Van Vlack, E. R., Tie, S., Xue, V., Cowles, S. S., Leung, A., Rodrigues, J. V., Morales-Perez, C. L.,,, Ayoub, A. M., Green, R., Puentes, K., Oplinger, F,, Panwar, N. V., Obermeyer, F., Root, A. R., Beam, A. L.,, Poelwijk, F, J., and Grigoryan, G. Illuminating protein space with a programmable generative model. Na- fure, 1070 1078, November 2023. ISSN 0028- 0836, 1476-4687. doi: 10. 0010008641588--03-00777777
S., and Rives, A. Evolutionary-scale prediction of atomic- level protein structure with language model. Science, March 2023. ISSN 0036-8075,
379(6637): 11003777:113-11111 123-1130. 1095-9203. doi: 10. 126/sciennc..dee574..
Lisanza, S. L., Gershon, J. M., Tipps, S., Arnoldt L.,, Hen- del, S., Sims, J., Li, X., and Baker, D. Joint Generation of Protein Sequence and Structure with RoseTTAFold Se- quence Space Diffusion, May 2023.
Lu, A. X., Yan, W., Robinson, S. A., Kelow, S., Yang. K. K., Gligorijevic, V., Cho, K., Bonneau, R., Abbeel, P., and Frey, N. C. All-atom protein generation with latent diffu- sion. bioRxiv, 2025. doi: httpsiivvv.gcooooee////// URL https: /ww. wwtp::wwwww..................... biorxiv..ivvvvi... early/2025/02/11/// 12. 02. 626353.
Jing, B., Eismann, S., Suriana, P., Townshend, R. J. L., and Dror, R. Learning from protein structure with geomet- ric vector perceptrons. In International Conference on Learning Representations, 2020.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Zidek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Rorere-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals. O., Senior, A. W., Kavukcuoglu, K., Kohli, P., and Hassabis, D. Highly accurate protein structure prediction with AlphaFold. Nature, 596(777):583--88, August 2021. ISSN 0028-0836. 1476-4687. doi: 10.1038/ $41586-021-03819-2
Martinkus, K., Ludwiczak, J., Liang, W.-C., Lafrance- Vanasse, J., Hotzel, I., Rajpal, A., Wu, Y., Cho, K., Bon- neau, R., Gligorijevic, V., and Loukas, A. AbDiffuser: Full-atom generation of in-vito functioning antibodies. In Thirty-Seventh Conference on Neural Information Pro- cessing Systems, November 2023.
Ren, M., Zhu, T., and Zhang, H. CarbonNovo: Joint De- sign of Protein Structure and Sequence Using a Uni- fied Energy-based Model. In Forty-first International Conference on Machine Learning, 2024. URL https: / //openreview. net/
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the Design Space of Diffusion-Based Generative Models. In Advances in Neural Information Processing Systems, October 2022.
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., and Fergus, R. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 118 (15):e2016239) 18, April 2021. ISSN 0027-8424, 1091- 6490. doi: 10. 10.1073/pnas. 2016:39900011919111
Kingma, D. P. and Ba, Adam: A Method for Stochastic Optimization, January 2017.
Krishna, R., Wang, J., Ahern, W., Sturmfels, P., Venkatesh, P., Kalvet, I., Lee, G. R., Morey-Burrows, F. S., An- ishchenko, I., Humphreys, I. R., et al. Generalized biomolecular modeling and design with RoseTd All-Atom. Science, 384466993)eaad12558, 2024.
Satorras, V. G., Hoogeboom, E., and Welling, M. E(n) equiv- ariant graph neural networks. In International conference on machine learning, pp. 9323 9332.. PMLR, 2021.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-Based Generative Model- ing through Stochastic Differential Equations, February 2021.
Li, J., Cheng, C., Wu, Z., Guo, R., Luo, S., Ren, Z., Peng, J., and Ma, J. Full-atom peptide design based on multi- modal flow matching. In Proceedings of the 41st In- ternational Conference on Machine Learning, ICML'24. JMLR.org, 2024.
Steinegger, M. and Söding, J. MMseqs2 enables sensi- tive protein sequence searching for the analysis of mas- sive data sets. Nature biotechnology, 35(11): 35(11):0026-0008 2017.
Lin, Y., Lee, M., Zhang, Z., and AlQuraishi, M. Out of Many, One: Designing and Scaffolding Proteins at the Scale of the Structural Universe with Genie 2, 2024.
Touw, W. G., Baakman, C., Black, J., Te Beek, T. A., Krieger, E., Joosten, R. P., and Vriend, G. A series of PDB-related databanks for everyday needs. Nucleic acids research, 43(DD): D368, 2015.
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., Dos Santos Costa, A., Fazal-Zarandi, M., Sercu, T., Candido,
10

---

Pallatom
Van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L.,, Söding, J., and Steinegger, M. Fast and accurate protein structure search with Foldseek. Nature biotechnology, 42(2):243-2466 2024.
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natas- sia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., Zidck, A., Green, T., Tunyasuvunakool, K., Petersen, S., Jumper, J., Clancy, E., Green, R., Vora, A., Lutfi, M., Figurnov, M., Cowie, A., Hobbs, N., Kohli, P. Kleywegt, G., Bimey, E., Hassabis, D., and Velankar, S. AlphaFold Protein Structure Database: massively expanding the structural coverage of protein- sequence space with high-aceuracy models. Nucleic Acids Research, 50KDI):439-D444,
2021. ISSN 0305-1048. doi: 10. 10.10993/ar//kaa1061. URL ht ttsp:: ///doi.org/// http://...00//////gaaaaaaaa0
1093
Wang, C., Qu, Y., Peng. Z., Wang, Y., Zhu, H., Chen, D., and Cao, L. Proteus: Exploring Protein Structure Genera- tion for Enhanced Designability and Efficiency, February 2024.
Wang, G. and Dunbrack Jr, R. L. PISCES: protein se- quence culling server. Bioinformatics, 1911211559-1551, 2003.
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, E., Ahern, W., Borst, A. J., Ragotte,
R. J., Milles, L. F., Wicky,
M., Hanikel, N., Pellock,
S. J., Courbet, A., Sheffler, W. Wang. J., Venkatesh,
P. Sappington, I., Torres, S. V., Lauko, A., De Bortoli,
V., Mathieu, E., Ovchinnikov, S., Barzilay, R., Jaakkola, T. S., DiMaio, F., Baek, M., and Baker, D. De novo design of protein structure and function with RFdiffusion. Nature, (20(7976):1089-11
August 2023. ISSN 0028- 0836, 1476-4687. doi: 10.
Yim, Campbell, A., Foong, Y. K., Gastegger, M., Jiménez-Luna, J., Lewis, S., Satorras, V. G., Veeling, B. S., Barzilay, R., Jaakkola, T., and Noé, F. Fast protein backbone generation with SE(3) flow matching, October 2023a.
Yim, J., Trippe, B. L., De Bortoli, V., Mathieu, E., Doucet, A., Barzilay, R., Jaakkola, T. SE(3) diffusion model with application to protein backbone generation, May 2023b.
Zardecki, C., Dutta, S., Goodsell, D. S., Lowe, R., Voigt, M.. and Burley, S. K. PDB-101: Educational resources supporting molecular explorations through biology and medicine. Protein Science, (1()):129-144 140, 2022.
Zhang, Z., Lu, Z., Hao, Z., Zitnik, M., and Liu, Q. Full- Atom Protein Pocket Design via Iterative Refinement. In Uhirty-Seventhh Conference on Neural Information Pro- cessing Systems, November 2023.

---

Pallatom
A. Related work
Backbone Generation. Diffusion models based on SE())-eqiivarian network architectures and protein representations using rigid frames have achieved significant success in protein generation, evidenced by models like Chroma (Ingraham al., 2023), Genie2 (Lin et al., 2024), RFdiffusion (Watson al., 2023), FrameDiff (Yim et al., 2023b), FrameFlow (Yim al., 2023a), Proteus (Wang al., 2024), and FoldFlow2 (Huguet et al., 2024). These models now support multi-condition controlled generation and have been extensively validated through both in-silico and wet-lab experiments.
Codesign Models. Recent methods have explored co-design approach that simultancously designs both the backbone and sequence. Multiflow (Campbell et al., 2024) utilizes a diffusion process that jointly operates on discrete sequences and continuous SE(3) backbones, eliminating the need for sequence redesign with ProteinMPNN (Dauparas et al., 2022). CarbonNovo (Ren et al., 2024) employs a similar approach, using SB(3) diffusion on the protein backbone while simultane- ously designing a sequence at each step with the MRF decoder. The sequence information is then embeded using the protein language model ESM2-3B (Rives et al., 2021) to guide structural generation.
All-Atom Generation. Recent research teams have begun exploring fully all-atom generative representation systems. For instance, Protpardelle (Chu et al., 2024) employs a coordinates diffusion model to support the generation of all-atom protein structures. ProteinGenerator (Lisanza et al., 2023) applies Euclidean diffusion on one-hot encoded sequences, combined with structure prediction module to obtain all-atom structure. Similarly, PLAID (Lu al., 2025) achieves sequence design through latent space diffusion within ESMFold while decoding full-atom protein configurations. RFdiffusionAA (Krishna al., 2024), based on fine-tuning the RoseTTAFolds (Baek al., 2023), can produce backbone structures of proteins and small molecule complexes but lacks side-chain conformations for standard amino acids. We focus on the methods directly generate all-atom protein structures, the most relevant work is Protpardelle, which uses an end-to-end approach to generate all-atom structures.
Beyond protein modeling, recent works leverage atomic-level representations for localized design tasks. For instance, Abdiffuser (Martinkus al., 2023) introduces universal four-atom side-chain template for antibody CDR redesign, preserving dihedral freedom via pseudo-carbon atoms while integrating ideal amino acid templates for rotamer construction. FAIR (Zhang et al., 2023) adopts two-stage approach for protein pocket design: initial backbone/sequence generation followed by iterative refinement to ensure sequence-side-chaie consistency. PocketFlow (Zhang et al., 2023) extends this by simultaneously designing pocket sequences and all-atom structures through flow matching across backbone, side-chain torsion angles, and sequences. PepFlow (Li et al., 2024) similarly utilizes multi-modal flow matching (torsion angles + sequences) for peptide design. However, such decoupled representations risk grguence-structurg conflicts and steric clashes Pallatom addresses these limitations through its atom14 representation, which intrinsically fuses structural and sequential modalities to minimize explicit conflicts.
B. Training Datasets
B.1. PDB Data
We used PISCES (Wang& & Dunbrack Jr, 2003) to obtain the required PDB list. For training, we selected subset of PDB entries with a resolution of <3A and a 95% sequence identity threshold. We then performed standard filtering to remove any proteins with >50% loops and applied a series of folding quality filters (described below). This process resulted in 7,459 structures.
B.2. Augmented Data
Augmented data are widely used in protein modeling-related work. Consequently, we supplemented our dataset with the AlphaFold Database (AFDB) (Varadi et al., 2021). The AF2 predicted structure database is available under a CC-BY-4.0 license for both academic and commercial uses. The AFDB contains 214 million data points, while Hernande al., 2023) provides a redundancy-reduuce version (AFDB-cluster) through sequence-structure clustering. We performed additional curation on AFDB-clusted applying an average pLDDT threshold (280) and maximum sequence length restriction (128 residues) yielded 582,652 refined structures.
Furthermore, we implemented refined filtering strategies to acquire designable and high-quality data by evaluating two structural metrics- - Average Neighboring Density and Core Residue Ratio - for retaining proteins with superior packing quality. The curation workflow operates as follows:
12

---

Pallatom
Average Neighboring Density: Specifically, we used a 10 Å distance cutoff for neighbor identification. The neighbor count is averaged across the entire protein length. This serves packing quality filter. The threshold values empirically determined-we adopted 20.0 as the cutoff for Pallatom training. Note that this threshold may require further adjustment for protein datasets of varying lengths.
Core Residue Ratio: We calculated the solvent-acces accessible surface area (SASA) for each residue using DSSP (Touw et al., 2015), with residues having SASA S 0.2 classified as core residues. A final cutoff of 230% core residue content was applied to retain high-quality training data.
We found that these two filters effectively selected training data with satisfactory packing quality.
Furthermore, the number and distribution of secondary structures can define the compactness and diversity of folding. We used DSSP (Touw et al., 2015) to assign the protein secondary structures and removing structures with loop content >50%. We also apply filter on the total number of continuous segments of B-sheet and a-helix to maintain structural diversity. Specifically, we define continuous segments B-sheet and a-helix, where a continuous segment must contain at least 2-3 consecutive residues of the same secondary structure. The total number of continuous segments the entire protein is then calculated. For Pallatom training, we applied a cutoff of continuous segments as a filtering threshold during data processing. This step helps further remove overly simplistic secondary structures from the AFDB-cluste dataset.
For highly extended structures, we limited the radius of gyration (Ry) to less than 25.0. To avoid overly long continuous unstructured regions within the protein structures, we restricted the maximum length of each loop to 15. Finally, we used the FoldSeek easycluster algorithm to remove redundant structures, setting a TM score threshold of 0.8 and a coverage of 0.9, which removed approximately 30% of highly similar structures. After applying these stringent filters, only 27,697 protein structures remained. These structures typically exhibit good folding and high designability.
C. Algorithms
We present comprehensive description of Pallatom's 's all-atom representation at otoom14, architectural modules and algo- rithmic workflow, In the accompanying pseudocode, components highlighted in blue maintain substantial similarity to AlphaFold3's implementation and are not expanded to prevent redundancy while ensuring clarity.
C.1. atom14
To resolve the discrepancy in atom counts across amino acids within an all-atom diffusion model, we designed an atomistic representation called at om1 4. As shown in Figure 3, this representation differs from conventional amino acid encoding schemes. In the atoml4 framework, the side chains of all amino acids are uniformly represented using atom coordinates.
For atoms absent in the native conformation of given amino acid, their coordinates are overlapped with reference atom position, specifically the Ca atom in the Pallatom implementation. For example, glycine (Gly), which only contains four backbone atoms (C, Co,, N, and O), has the remaining 10 "excess" side-chain positions all assigned to the Ca coordinate. To avoid pre-specifying atom types, the atoml 4 representation retains elemental types only for backbone atoms. All side-chain atoms are uniformly predefined as masked types, regardless of their actual chemical identities. It is important to note that this unified representation introduces degeneracy; certain amino acids share similar side-chain configurations (e.g., Asn vs Asp, Glu vs Gln, Cys vs Ser, Thr Val). Consequently, the final model output must incorporate an additional mechanism to resolve ambiguities in amino acid type determination arising from this degeneracy.
C.2. MainTrunk
Algorithm details the denoising process of the Pallatom Ma in Tunk network.
C.3. TemplateEmbedddr
In the TemplateEmbeddder module, we retained only the necessary mask and distogram features, and additionally included the timestep feature for self-conditioning.
13

---

Pallatom
Algorithm 2 MainTrunk
def MainTrunk(f" .
rrtrpat, E, 1128 1pai>=128, Crea=256, catompair=10 16, data 166):
Initialize positions and conditions embedding
1: Fj s7scaled
inlput
Vo
srscaled E TR3 ri
ri
2: sinit Nonear (Preaidue idx))
sins Rere t Rore
3: t, "TimFFourieFEmbedddng Log(t/ dadaa)
4: simit t;
5: z RelativePPsstionEnncodnggg
Zij
init Rouin
6: Z,j zi5.. TemmlaaeeEmbedderr (f(*) Ziz inst ,t, Nblock -2 64, Cpair)
# Initialise single and alom embeddings
7: {s:}. {q] ( ) {c;*tp),
J. (xtt)) J. {qi}
AttomFettureEncoodr(({11...... Zigs scaaled
Creas Catompair, Catom)
8: s; += (Layrroooa(yyyoorommmm
# AtomDecoder Units
9: updatees 0
updates
E RR3
10: for all unit E {l...., Kunit] do
11:
S; NodeUpdate(ss, t;, Zij, C Cres)
1q updated,
}, {ri updute ), {ci}= AtomAntentiinDeeooder( skip skip skip
12:
Pim G
C₁2 S; Zij)
13:
Fi updaaess += rpupdate
14:
denoised 0 / (o
data + (2)= 17 rrimppat + ddata v/ data + i2 updates
15:
Freenter Fl (center-uide
denoissd
16: Zij PairUppate (Zij, center ,C Cpair)
17: end for
# SeqHead for amino acid decoding
18: a,
mean 1E(1.....N Nanma
ReLUU(LinearNoBias( updated
a, Re
tok ((==
19: pre logits LinearNoBias(aa)
preg lngits
C 20
return (17 117denossed, ). [1,0q.oogiis
logits
Algorithm 3 Template Embedder
def TemplateEmbeedder( Zij, t. Notock=2, c=44, d= 128):
Concat template features jjemplate.pseudo.ee
1: b pwtudo.bett..aa..... beta.mask
f! stemplate.pscude becud.b
f pramplaa..pwudo.bee.....
2: gime L fij /eempltte.peeudobbe betauo.
1~ 0,11)
3: a;j concatí plemplate distogram template pseudo beta, mask (lime)
.bi
# Embed the self-condition feature
4: Vij LinearNoBias (Layerorr + LinearNoBias(a.s)
Vij E R
5: Vij PairformerStack(V... Nbteck)
6: u.j <- LineaNooBiss(elllUllayyyoooonn
U E Rd
return {u:}}
14

---

Pallatom
LIW
LM
SUMUT
KMM
KOW
Figure 3. atoml 4 encoding schemes.
C.4. AtomFeatureEncode
The AtomFeatuueEnncoder is adapted from AF3, with modifications for all-atom generation. We exclusively utilize alanine as the reference conformer, eliminating the need for predefined amino acid sequence information during the initialization phase. Furthermore, we standardized residue representations using the atoml format, with virtual atoms to prevent information leakage. The initialization process for traversing atomic embeddings detailed as follows:
15

---

Pallatom
Algorithm 4 AtomfFeatureEnnoderr
def AtomFeatuurEEncoder(( suppt 2,j ri
input secaled
c=256, d=16, m= m=128):
Create the atom single conditioning
1: fref concat( fret pos fret element)
fref R
2: C; = [ineaaroBBass(tll((frrr-)))
C₁ Rm
3: { phoi_pos tile (Frr
f
pel
R
4: citip
# Embed offsets between atom reference positions
5: dum pel pos ptef pos
d E R
6: Um (f,
ppe.space..udyy
0lm R
7: Pim LinearNoBias(dImm. Ulm
Pim E Rd
8: Pim
+= LincarNoBias (1((1+) + [/dmll?)) Ulm
9: Pim LinearNoBias (1/)) Vim
10: Pin LinearNoBias(RaLL((c)) LinearNoBBas (REL
11: Pim Pim
Initialise the atom single representation the single conditioning
12: skspp C₁ LinearNooBass scaleed
q Rm
# Add atom positional and time conditioning
13: += LinearNBBia(Laayerorrm (s
wk idx(1))
14: Pim += LineanNoBia(LLayeeNorrfi
friit
tok teokidk())ookkdd (dx)))
15: Pim +
)))))))))
Cross attention transformer
16: qi skip = Atom Transformer(q, skip
CC,, Pim: Nibock = 3, Nhead = 4)
Aggregate per-atom representation to per-residue representation
17: S;
mean (cLUULLinearNBBissq"""""" ((P())
E R
LE IE(1...,NNsmm lok idx(!)=s
return {s:). (qi
skip 1 tor
hhiip, (pthin). {ci}
16

---

Pallatom
C.5. Deontteentioneecoder
We developed a novel information management strategy for decoder layers that simultaneously integrates updated single- and pair-embeddings into traversing atomic-level embeddings. This approach effectively prevents redundant structural information accumulation while maintaining essential feature propagation.
We present a mathematical derivation to analyze information redundancy arising from residual connections between residue- level and atomic-level embeddings, identifying key factors contributing to "traversing" strategies. For notational clarity, we define 8; as the residue-level embedding and q, as the atom-level embedding for the i-th unit, with broadcast(s;) denotes the operation that propagates residue-level information atomic-level. The network updates are represented by the functions f()) and g(-):
8; 1 f(8i-1) 9: = 4-1 + g (broadcast (8 (8i-1) + 96-1)
(8)
(9)
The iterative process can be organized as follows:
8 S1 +
f(s))
(10)
k=0
di Q1 +
g(broadcast(s)) + gb)
(11)
Q1 g(broadcast(ss) (m1
gl g(roodccatt(ss) dg)))
(12)
Through recursive expansion of the gis we observe that each term depends on its preceding de while incorporating broadcasted features from 8;-1 at each update step. This results in information redundancy, ultimately impeding efficient network learning and feature updating.
Algorithm 5 AtomAttentionnDecoder
def AtomAAteentionDecoderr skip skip skip Ct2 Z:y):
Pim
Add trunk embeddings
1: qi aareaaaoooBBssaaaaeeoonnlosaaaa + hhip
2: Pmm = LinearNoBias(LLayeroom Zonk (f)okk idx(()) + Pim
skip
3: pim +=
))
Cross attention transformer
4: qi Atom NommTrassoorerr(qu cı, Pimy Notock = 3, Nhead 4)
# Map to positions update
5: upddate
# Update trunk embedddngs to atom condition
6: LineattoBinsLLLayeNomm (S..aaa + c
skip
return updalee {-(
C.6. NodeUpdate
The NodeUpdate module manages residue-level embedding updates through time-conditioned AttentionPairBias mecha- nism, enabling adaptive scaling of network updates based on noise levels during the diffusion process. Additionally, we implemented dropout regularization during training to prevent overfitting and improve model generalization capabilities.
C.7. PairUpdate
The PairUpdate module exclusively employs a modified TriangleAttention algorithm, excluding TriangleMultiplicattonn while preserving robust generative capabilities. Our implementation utilizes decycling-derived pair representations as
17

---

Pallatom
Algorithm 6 NodeUpdate
def NodeUpdate(s;, t,, Zij, c=256):
# PatirniionPairBias with updated pair bias
1: 5; += DropoutRowwsseo.2 (Ateeootisooootttttittnnonnaaaaa (AttentiooaaiirBBiiss.... 2: S; +- (sa
Zij, Pij = O.ead = 8))
return }
attention biases, enhancing the model's ability to capture long-range interactions. To ensure pair feature space consistency, processed recycling structures using identical binning parameters the VemplateEmbeddee module, employing 39 discrete bins spanning from 3.25 to 50.75 angstroms for comprehensive distance representation.
Algorithm 7 PairUpdate
def PairUpdate(z..j. reenter, (=128):
Obtaining the pairwise distance matrix through RBF discretization center
1: d.j- r; center - -jj
di E R
2: b.j LinearNoBiassTranss (raansformm (BF(()))))
bij E R
TriangleAAtenntion with coordinates pair bias
3: 7,j += DropoutRowwise,
biz))
4: Zyj + DropoutColumnviso..o
Tiagroomiiiiiiiiigatttttttttttnn (TiagglcAttciioonEnddnnnnoo With bız))
5: Zij +
Transition Zzi))
return {z.z}
D. Loss Function
In this paper, in addition the primary loss function Co, which is composed of the denoising score matching loss Latom and the cross-entropy loss Lseg for sequence decoding, describe additional loss components that contribute model optimization.
D.1. Distogram Loss
The distogram loss function has become fundamental component in protein structure prediction pipelines, effectively guiding the estimation of inter-residue distance matrices from pairwise feature representations. In our implementation, symmetrize the global residue-level pair representations and project them into 64 distance bins with predicted probabilities pig. which are then supervised using one-hot encoded target distance bins viz-
1 L
C res
Vi log p
Similarly, supervise the local relative distance distribution, project the atomic-level pair representation from local atomic attention into 22 distance bins from 0 À to 10 À with dim. and construct the loss function using one-hot encoded target distance bins, with the local region defined by the attention window within the the 14L x 14L atomic-level map, ensuring precise local geometry constraints.
22
Edist.atom
1
Σ
Urum log Frm
NM
Eloaal
b=1
D.2. Intermediate Loss
To stabilize the training process, we implement intermediate supervision of sequence and structure predictions from each decoder unit. Additionally, we introduce a loss weight decay mechanism with 7 = 0.99 across decoder blocks, progressively
18

---

Pallatom
increasing the weight assigned to later layers. Therefore, the intermediate loss for the K units can be expressed as:
Fdenoised
Faligned
C med
= 1(1)
+ ao (E(å()).
3L
med
K-k
K
k=1
D.3. Smooth LDDT loss function
During the training, we adopt the smooth LDDT loss (Algorithm 8, simplified version from AF3) for all-atom protein design.
Algorithm 8 Smooth LDDT loss
def SmoothLLDDDLLossff r
Compute distances between all pairs of atoms
1: 5rim= f 1| - Fl
2: Gr < || FCT
Compute distance difference
3: 6tm = abs(ōr 5rim)
4: 6lm = 1 [iiimmodd(
dtm) sigmoid(1 - 5tm) ) sigmoid(2 - -mm) sigmoid(4 - dim)
# Set the radius threshold and compute mean
5: Ctm + 16xx < 15A)
5: Iddt = mean mean(Cim Ctmmlm)) h //mmam/ccmm
1m
ifm
return Iddt
E. Ablation Studies
We systematically investigate the fundamental factors contributing to Pallatom's superior co-design performance through complementary top-down and bottom-up analyses, aiming to uncover the underlying principles of its exceptional designabil- ity. The success of the at col4 4 all-atom representation demonstrates the viability of comprehensive atomic-level generative modeling. To further explore this paradigm, we develop hybrid14 representation that incorporates sequence information, enabling comparative analysis that highlights the distinct advantages of the atoml4 approach. Another key innovations in the AtomDecoder unit is the differentiable recyeling process, we further investigate through ablation studies.
E.1. hybrid14
To efficiently represent side-chain configurations across all 20 amino acids while avoiding the computational overhead of Protpardelle's atom73 representation, we propose the hybridl dll representation maintains a superposition of 20 possible side-chain configurations, which collapse into single hybridized residue state based on predicted sequence probabilities during the generation process.
Specifically, we implement the model's self-conditioning mechanism by matrix matrix-multiplying the predicted sequence proha- bilities 5 E iR Lx20 with standard amino acid reference conformer coordinates V 200014x3 generating hybrid reference R f. 14x3 that are subsequently integrated atomic embedding.
conformations Vinbbrd SV
Using identical loss functions and training configurations, we retrained the model and conducted ablation studies by generating 100 protein sequences of length L = 100, followed by comprehensive designability evaluation to assess the impact of atomic representation modifications.
E.2. woRC: Ablation Setup without the Recycling Process
To assess the contribution of the differentiable recycling process in the AtomDecoder unit, we conducted ablation experiments by substituting the recycling pair with a zero matrix and disabling gradient backpropagation. This modification prevents the residue-level pair embedding 2ij from receiving updates from partially denoised structures while maintaining the triangle update module for seff-conditioning during training.
19

---

Pallatom
E.3. smooth LDDT loss
We also conducted ablation studies the smooth LDDT loss function. Table 4 show the results. The training observations revealed that the LDDT loss maintained consistently low values throughout training, suggesting it may act implicit violation loss to mitigate side-chain atomic clashes. Ablation experiments confirm the LDDT loss's role in enhancing the quality of Pallatom- Pallatom-generated all-atom proteins, particularly by preserving critical sequence-structure self-consistency (DES-aa reduction).
Table 4. Ablation Studies on Smooth LDDT Loss.
Method
DES-aa
DES-bb(w)
DES-bb(wo)
Pallatom
87 %
95%
95%
wo-LDDT 84%
95%
95%
F. Experiment Details
Table 5 provides a detailed list of the input feature.
Table Input Feature Descriptions
Input Feature
Dimension
Description
ref pos
(14,3) 3)
Atom positions in the reference conformer are given in Á. The backbone atoms (N, C, C.. O, C3) are listed in the first five columns, while all side-chain atoms are moved to the Ca atom position. We encode the backbone atoms based on their elemental types (N, C, O]. while the side-chain atoms are encoded as single class using 'UN'' (unknown).
ref element
(14, 4)
ref.space_uid
(Natom.)
Numerical encoding of the residue index associated with this refer- ence conformer.
ref_center mask
(Natom.)
Masks indicating the center atom of the residue.
residue.index
(Nrest)
The pdb residue number for calculating relative positional embed- ding
residx_embedding template.distogram
(Nres, 32)
The absolute position embedding by sinusoidal positional encoding. Pairwise distogram of pseudo Css are discretized into 38 bins of equal width between of bin min=3.25A, bin max=50.75A, one more bin contains any larger distances.
(Nres, Nres, 39)
template. cb. mask
(Nres.)
Mask indicating if the C3 atom has coordinates for the template at this residue, where indicates existing residues and used for padding residues.
template_tine _time
(Nres, Nress
Normalized pairwise time-step feature, ranging from 0 to 1.
all_atom_poott iiiottoosiiions all atom mask ( hat
(Natom, 3)
The noisy position of atoms in the system.
(Natom.) (1,)
Mask indicating which atom slots are used in the the system.
The noise level value for adding noise.
20

---

Pallatom
Table 6 provides detailed list of the hyperparameters used for training.
Table 6. Pallatom training hyperparameters.
Parameter name
Value
Batch size
32
Learning rate
0.001, No warm-up.
Examples per epoch
35156
Crop size
128
Loss weights
Sequence loss weight ogg 0.25,
Smooth Iddt loss weight o1 1.0.
Residue-level distogram loss weight ag = 0.5, Atomic-level distogram loss weight 0g = 0.5, Intermediate loss weight 0x = 1.0
In the basic loss Lo. weight allocation was applied to residue types, with weight of 2.0 assigned to polar residues, and 1.0 to the others.
Diffusion timesteps (N (Nateps or 7)
200
Self-condition rate EDM
100%
Noise schedule lognormal. In(o) ((Pmean: 1.2, Patd 1.5, data 16,
mean: P2 std), P Pmeon
Stochastic sampler min 0.01, Imax = 1.0, noise level 7o = 0.2, noise scale À = 1.003, step scale 2.25
Transformer
single representation dimension 256, pair representation dimen- sion=128, number of heads = 8, number of decoder units 8
Training Steps
3 x 10
Training time
R 10 days
Device
x A6000
G. Additional Results
G.1. Out-of-distribution performance
Evaluation of Metrics We performed extensive evaluation of Pallatom's performance on out-of-distribution se- quence lengths. specifically testing its generalization capabilities proteins significantly longer than those encoun- tered during training. Specifically, we generated and analyzed 250 protein structures for each of six distinct lengths L = 150, 200, 250, 300, 350, 400 providing comprehensive evaluation across a broad spectrum of sequence lengths. Notably, while our training set employed a crop size of 128, all comparative methods were trained on datasets containing proteins with lengths up to 384, highlighting a significant difference in training data complexity. Figures illustrates the comparative performance across designability (DES), structural diversity (DIV-str), and structural novelty (NOV-str) metrics for various sequence lengths under both CO-DESIGN 1 and PMPNN evaluation modes, providing comprehensive insights into the model's length-dependent behavior.
Our analysis revealed that Pallatom consistently achieved superior designability metrics for sequences up to twice the maximum training length (L- = 150 - 250) across both evaluation modes, while simultancously demonstrating significantly greater structural diversity and novelty compared to Multiflow. Even when extending to more than three times the maximum training length at 350 and L 400, Pallatom, although less advantageous in backbone design, still maintained the ability generate all-atom proteins, feat unachievable by the other two comparison methods, Protpardelle and ProteinGenerator.
We further analyzed the secondary structure preferences of sampled structures from all methods. Specifically, we utilized DSSP to classify the secondary structure of each residue in the proteins.
Secondary Structure Analysis
We established a classification scheme based on helix-to-shedd ratios: proteins with o-helix residues exceeding five times 8- residues are categorized as HHH (all-helix), while those with 3 residues surpassing five times a- a-helix residues are classified as EEE (all-33), Proteins with balanced proportions of a-helices and f-sheets are classified as HEL, representing mixed o3
21

---

Pallatom
220
280
254
24
2.0
600
150
283
293
300
272
225
325
322
202
Figure Comparison of Evaluation Metries Sampled Proteins Longer Lengths. (A) (B) respectively show the designability, structural diversity, and structural novelty under the CO-DESIGN mode and the PMPNN mode.
structures, completing our comprehensive secondary structure classification framework.
Figure 5 presents the secondary structure distributions for all methods under both evaluation modes, along with the corresponding distributions for designable protein subsets. This result indicates the ProteinGenerator, multiflow, and Pallatom exhibit similar secondary structure preferences within the length range of 150-400, with roughly equal distribution between HEL and HHH structures. In contrast, REdiffusion and Protpardelle show stronger preference for HEL structures. Within the 150-400 length range, all models rarely succeed generating EEE structures.
A
Figure 5. Secondary Structure Distribution of Sampled Proteins at Longer Lengths. Figures (A), (B), and (C) show the secondary structure distribution of sampled proteins across methods, the designable proteins in CO-DESIGN mode, and designable proteins PMPNN mode, respectively.
All these experimental results demonstrate Pallatom's 's superior performance, highlighting its exceptional scalability and generalization capabilities. To further illustrate these advantages, we present additional case studies: Figure showcases novel designable protein structures generated by Pallatom, while Figure 12 demonstrates its ability to produce high-quality designs for length distributions beyond the training set scope.
22

---

Pallatom
G.2. Structural and Chemical Validity Assessment
As an all-atom protein generation model, we further analyzed the stereochemical properties of side chains in Pallatom- generated proteins. Through comprehensive statistical analyses of bond length distributions (Figure 6), bond angle distributions (Figure 7). X-angle distributions (Figure 8), and conformational clashes (Figure 9), we demonstrate that the side-chain structures generated by Pallatom rigorously adhere to protein physicochemical constraints.
Specifically, we conducted conformational plausibility evaluation on 750 generated samples (L100-L300). Conformational clashes and bond length/angle analyses were benchmarked against natural PDB datasets, while chi angle distributions were compared with training set structures (AFDB+PDB) for specific considerations: certain side chains exhibit 180 rotational symmetry where X and X + configurations are physically equivalent (differing only in atom naming) (Jumper al., 2021). Since Pallatom was trained on both AFDB and PDB data, we used training set comparisons for x-angles avoid potential distribution discrepancies between AFDB training data and natural PDB statisties.
Bond lengths and angles were evaluated at the residue level with all-atom assessment, including separate statistics for peptide bond parameters (the peptide bond length named "bond-peptide" and the peptide bond angle named "angle-INTER"). angle distributions were analyzed individually for x1-4 angles per residue. Conformational clashes were quantified using Clash Num (per 1000 contacts), defined as the count of non-bondee atom pairs within 5Å having distances < 1.5A ( (50% of summed van der Waals radii), categorized intra-/inter-residue clashes.
latom
bond ALA
ASN
CYS
GLU
5.0
2.5
2.5
2.5
0.0
0.0
0.0
0.0
1.0
1.5
2.0
1.0
1.5
2.0
1.0
1.5
2.0
1.0
1.5
2.0
1.0
1.5
2.5
1.0
1.5
2.0
1.5
3.5
2.0
bond-GUY
Lond-HIS
noni--LLE
bond-LEU
bond.LYS
bonc-MET
bond-PHE
Frequency
10
10
10
5
2.5
2.5
0.0
0.0
band- PRO
bond- SER
bond-THR
bons-TRP
bonG-TYR
bond-VAL
bond- peptide
10
10
5.0
2.5
1.0
1.5
2.0
1.0
155
2.0
1.0
1.5
2.0
1.0
1.5
2.0
1.0
1.5
2.0
1.0
1.5
2.0
1.5
1.5
2.0
bond length (A)
Figure 6. Comparison of bond length distributions between Pallatom-generaaeed proteins (blue) and natural protein counterparts (yellow) the PDB dataset.
G.3. Analysis of Sampling Hyperparameters
We analyzed the effect of the step scale 7 on sampling proteins of unseen longer lengths. Table 7, the left column for each length corresponds 17 2.5, while the right column corresponds to 17 = 3.0. We observed that a larger step size in the gradient update direction improves the designability of proteins, and as the number of designable samples increases, the structural diversity of generated proteins also increases, with only slight decrease in novelty. Additionally, consistent with the observations in the main text regarding the impact of sampling hyperparameters on secondary structure distribution, a larger step size is associated with a more pronounced preference for all-helix structures.
G.4. Sequence Quality of Designable Proteins
We compared the quality of the sequences generated by Pallatom with those produced by ProteinMPNN for the same protein structures designed by Pallatom. Figure 10 shows the PLDDT scores of the two sequences predicted by ESMfold. We found that the sequence confidence score of Pallatom is slightly lower than that of ProteinMPNN, with the maximum mean PLDDT difference not exceeding 2. We attribute this difference to both the training data and the task. Regarding training data, Pallatom was trained on monomer protein dataset, whereas ProteinMPNN was trained on a more diverse
23

---

Pallatom
REF
Pa latam
angle.ALA
angle:ARG
angle ASN
angle.ASP
angle-CY5
angle-GLN
angle-GW
120
130
180
Ang Ingle-GLLY
angle-HIS
angle-ILE
angleeLEE
angle-LYS
angle-MET
angle-PHE
Frequency
10
10
10-
5
00
120
150
18D
50
120
150
180
90
120
150
1.00
120
150
180
90
120
150
160
60
120
150
100
90
120
150
180
THR
angleTRP
angle TYR
10
10
0
150
180
150
bond angle (*)
Figure Comparison of bond angle distributions between Pallatom-generated proteins (bluc) and natural protein counterparts (yellow) in
the PDB dataset.
Table Pallatom sample metrics with step scale = 2.5 (left) and v = 3.0 (right).
Length
150
200
250
300
350
400
DES-aa
88%
93 %
79%
93 %
69%
81
35%
66 %
19%
41 %
4%
18 %
DIV-str
131
125
153
184
149
190
73
162
38
104
9
45
NOV-str
0.650
0.656
0.618
0.636
0.594
0.610
0.576
0.589
0.551
0.574
0.525
0.554
HHH((%
44%
53%
53%
64%
54%
70%
60%
77%
64%
85%
69%
92%
HEL(%) (%)
53%
44%
47%
36%
46%
30%
40%
23%
36%
15%
31 %
8%
EED(%)
2.8%
2.7%
0.0%
0.4%
0.0%
0.4%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
dataset that includes both monomer and multichain structures. Additionally, when preparing the training set. ProteinMPNN focused more on the sequence diversity under the same structure, while Pallatom needed to consider both sequence and structure diversity. In terms of the training task, the objectives of the two models are fundamentally different. ProteinMPNN concerned solely with sequence design given a real backbone, whereas Pallatom must balance the dual objectives of structure generation and sequence generation from pure noise.
G.5. Additional Comparative Methods and Results
In addition to the methodologies presented in the main text, we conducted extended benchmarking against three specialized approaches: CovbonNovo (Ren al., 2024) (for joint backbone-sequence co-design), FoldFlow2 (Huguet et al., 2024), and Proteus (Wang al., 2024) (both for backbone design). Systematic comparisons were performed under modes CO-DESIGN-1 and PMPNN-1 with results in Table 8, 10 and 11. Pallatom consistently achieved top-ranked performance across chain lengths from 100 to 300.
G.6. Pallatom-bb: Backbone-Only Configuration
Through systematic validation, established the viability of atom14 as comprehensive point cloud representation for all-atom protein coordinates, while concurrently investigating and implementing point cloud-bate based approaches for protein backbone description and design. To implement this, we simplified the atom14 representation to atom5 point cloud representation using four backbone atoms (N, C., C) plus Cg. We named this method Pallatom-bb. We trained Pallatom-bb using identical training data and employed PMPNN for backbone design evaluation. By analogy to the right side of Table and Figure 4, the experimental results for L=60 to 120 and L=150 to 400 are provided in Table 12, 13, 14 and 15.
24

---

Pallatom
Table Performance Comparison in CO-DESIGN
Mode.
Metric
Method
L=100
L=150
L=200
L=250
L=300
L=350
L=400
Pallatom
86.8%
92.9%
93.3%
81.0%
65.9%
41.3%
17.9%
DES-aa
CarbonNovo
75.6%
72.8%
70.0%
70.0%
59.6%
47.2%
38.8%
Pallatom
69
125
184
190
162
104
45
DIV-str
CarbonNovo
61
64
52
48
36
36
40
Pallatom 0.711
0.656
0.636
0.610
0.589
0.574
0.554
NOV-str
CarbonNovo
0.717
0.694
0.717
0.725
0.748
0.730
0.701
Table 9. Performance Comparison in PMPNN Mode for DES-bb(w).
Method
I==100
I.=150
I.=200
I.=250
L=300
L=350
L=400
Pallatom 90.4%
94.0%
90.8%
80.8%
64.4%
36.0%
17.6%
CarbonNovo
55.2%
46.8%
50.4%
45.6%
39.2%
26.4%
16.0%
FoldFlow2
71.6%
74.0%
70.0%
57.2%
51.2%
44.4%
34.8%
Proteus
74.8%
84.8%
78.0%
67.2%
62.0%
52.4%
57.2%
Table 10. Performance Comparison in PMPNN
Mode for DIV-str.
Method
L=100
L=150
L=200
L=250
L=300
L=350
L=400
Pallatom
79
131
171
190
157
90
44
CarbonNovo
35
37
29
36
24
22
17
FoldFlow2
30
32
32
39
25
26
18
Proteus
50
34
34
15
22
30
20
Table 11. Performance Comparison in PMPNN
Mode for NOV.
Method
L=100
L=150
L=200
L=250
L=300
L-350
L=400
Pallatom 0.709
0.659
0.631
0.607
0.588
0.577
0.557
CarbonNovo
0.730
0.717
0.733
0.725
0.753
0.739
0.703
FoldFlow2
0.740
0.694
0.679
0.656
0.641
0.636
0.631
Proteus 0.761
0.743
0.742
0.779
0.763
0.749
0.734
Pallatom-bb achieves high designability within training sequence lengths but exhibits reduced structural diversity. While showing limited extrapolation capability for longer sequences, marginally outperforms RFDiffusion overall metrics. Crucially, backbone-only point cloud approaches neglect critical side-chain/sequence interactions, whereas atom14's all- atom framework explicitly models structural-sequeet constraints, demonstrating superior holistic performance across design benchmarks.
25

---

Pallatom
Table 12, Comparative Evaluation Pallatom-bb Under PMPNN mode Across Chain Lengths 60-120.
Method
Des-bb(w/wo)
DIV-str
NOV-str
Protpardelle
30.00%(80023%)
15
0.747
ProteinGenerator
93. 93.14%(96.33%%
151
0.785
Multiflow
84.69%
184
0.744
RFdiffusion
78.29%((4440%))
165
0.805
Pallatom
89.89%(94446%))
318
0.716
Pallatom-bb
96.46 (97.4%%)))
164
0.760
Table 13. Comparative Evaluation of Pallatom-bb Under PMPNN
on Designability.
Method
L=150
L=200
L=250
L=300
L=350
L,=000
RFDiffusion
77.2%
58.0%
45.6%
32.4%
22.0%
20.8%
ProteinGenerator
91.2%
80.0%
64.8%
38.4%
14.8%
1.2%
ProtPardelle
29.2%
0
0
0
MultiFlow
88.8%
78.8%
79.6%
68.8%
66.4%
51.6%
Pallatom 94.0%
90.8%
80.8%
64.4%
36.0%
17.6%
Pallatom-bb 92.0%
80.4%
52.8%
43.6%
34.4%
36.8%
Table Comparative Evaluation of Pallatom-bb Under PMPNN on Structure Diversity.
Method
L=150
I.=200
L-250
L=300
L-350
L=400
RFDiffusion
62
62
52
44
32
27
ProteinGenerator
23
23
12
4
ProtPardelle
0
0
MultiFlow
63
66
73
64
63
57
Pallatom
131
171
190
157
90
44
Pallatom-bb
41
52
52
55
46
43
Table 15. Comparative Evaluation of Pallatom-bb Under PMPNN 1 on Structure Novelty.
Method
L=150
L=200
L-250
L=300
L=350
L-400
RFDiffusion
0.688
0.689
0.672
0.679
0.679
0.687
ProteinGenerator
0.752
0.721
0.733
0.734
0.756
0.737
ProtPardelle
0.790
MultiFlow
0.710
0.700
0.724
0.730
0.742
0.738
Pallatom
0.659
0.631
0.607
0.588
0.577
0.557
Pallatom-bb
0.720
0.694
0.667
0.655
0.662
0.647
26

---

Pallatom
Tulidom
AMO
APC
AMO
ANO
300
380
y
AD7
040
100
140
CLN
G.A d'
CLM
-30
-9
100
100
100
100
CLU
CLU
H5
120
4.0
LEL
1requeno
LYS g
19%
140
140
THe of
-30
-30
30
100
-006
28
59
130
55% y
0.23
4300
-30
100
TEP
TB4 y
0.2
140
Wi.
23
Dihedral (*)
Figure 8. Comparison of x-angle distributions between Dallatom-generate proteins (blue) and natural protein counterparts (yellow) in the PDB and AFDB datasct.
27

---

Pallatom
Intra Clashes
Inter Clashes
1.00
1.00
REF
REF
0.50
0.50
Frequency
0.03
0.03
0.02
0.02
0.01
0.01
0.00
0.00
5
Clashes (per contacts)
Clashes (per 1000
Figure 9. Comparison conformational clashes distributions between Pallatom-geet -aaneoon-eeeertedd proteins and natural protein counterparts in the PDB and AFDB dataset. The left panel displays intra-residue clash counts, while the right panel shows inter-residue clash counts.
100
Fallatom (PMPNI)
PLOOT
50
50
100
150
200
250
300
350
400
Length
Figure 10. Comparison of pLDDT between sequences designed by Pallatom and ProteinMPNN across different lengths. Sequences designed by Pallatom are labeled as "Pallatom (CO-DESIGN)," while sequenees designed by ProteinMPNN based on backbone are labeled as "Pallatom (PMPNN)."
28

---

Pallatom
G.7. Analysis of Sampling Time
We conducted comparative analysis sampling times for each method. Specifically, standardize the diffusion sampling steps to T 200 and sample 100 proteins for each length, calculating the and standard deviation. All methods were tested the same hardware: CPU: AMD EPYC 7402 @2.8GHz, GPU: NVIDIA GeForce RTX 4090 with 24GB VRAM.
Table 16 presents the results. Thanks to JAX's JIT compilation and our optimizations at the atom level of Attention, Pallatom achieved the second fastest sampling speeds for lengths ranging from 100 to 350, outperforming all methods except Protpardelle. At = 400, even with the atomic-level length reaching (14 x 400) x (14 x 400). Pallatom's 's performance remains comparable to the second-fastess method, Multiflow, and is times faster than RFdiffusion and 16 times faster than ProteinGenerator.
Table Sampling Time (in seconds). The shortest is highlighted bold, and second shortest indicated italics.
Length
100
150
200
250
300
350
400
Protpardelle
10.9+0.1
11.3+0.2
11.7+0.2
12.5+0.2
24.5+0.5
26.3+0.6
27.6+0.9
ProteinGenerator
110.1
389.5+2.3
38..
477.6*1.7
624.0#4.1
796.3+4.6
950,3+5.3
Multiflow
25.3+0.4
25.340.6
27.1+0.3
29.4+0.2
35.1+0.6
40.5+0.2
46.6+0.5
RFdiffusion
14.5
93,9+1.0
106.3+0.9
138.5+0.8
183.0#0.8
230.7#1.6
287.4+6.5
Pallatom
10.2#0.1
12.1+0.1
18.2+0.1
21.9#0.1
33.320.1
40.4+0.1
57.540.1
29

---

Pallatom
TM-&core: 0.619
1M-Sccore: 0.397 RMSD: 3.26
TM-S Saoraco:::p 0.523 RMSD: 2.86
TM- SM--Sore:00 0.63 RMSD: 2.14
TM-Score: 0.49 RMSD:
TM- TM-Score: 0.56 RMSD:
RMSD: 2.86
TM-Scare: 0.484 RMSD:
1M-Score: 0.602 RMSD: 2.63
TM- Score: 0.542 RMSD: 2.74
TM-Score: 0.66 RMSD: 2.47
TM-Seen MM-Score: 0.60 RMSD: 3.68
TM-Score: 0. 0.533
RMSD: 3.28
TM-Score: 0.655 RMSD:
TM-Score: 0.598 RMSD: 3.02
TM-Score: 0.139 RMSD: 3.8
TM-Score: 0.637 RMSD: 3.11
TM-Soore: 0.514 RMSD: 4.12
TM-Score: 0.435 AMSD: 4.86
TM-Score: RMSD:3.6
TM-Score: 0.591 RMSD: 3.61
TM-Score: RMSD: 3.38
TM-Score: 0. 0.605 RMSD:2.88
TM-Score: 0.517 RMSD: 3.54
TM-Score: 0.581 RMSD: 2.79
Figure 11. Additional novel designable proteins generated by Pallatom. The blue structures in the figure represent the designable protein sequences generated by Pallatom, which have been predicted using ESMfold and colored based on DLDDT scores. The white structures are the nearest neighbors from the Foldseck database (using the default cight databases on the Foldseck web server), with the distances between the two sets structures evaluated using TM-Score and RMSD.
30

---

Pallatom
L=200
L-250
L=300
L-350
L=400
Figure 12. Additional all-atom protein structures generated by Pallatom for longer sequence lengths. The white structures represent those generated by Pallatom, while the colored structures are predicted by ESMfold are colored according to PLDDT values, with bluer hues indicating higher PLDDT scores.
31


Reviews:
[{"review_id": "NfG8aNfGxD", "reviewer": "Reviewer_czVh", "summary": "The paper presents Pallatom, an end-to-end all-atom generative model that jointly learns protein sequences and their 3D coordinates. It uses an “atom14” representation to standardize side-chain atoms and employs a diffusion-based approach on Cartesian coordinates. A dual-track architecture updates residue-level and atomic-level embeddings through iterative decoding, incorporating a differentiable recycling mechanism that refines pairwise features as the structure is denoised. The model directly predicts side-chain positions and amino acid types during generation, showing improvements in designability, diversity, and novelty compared to existing methods like Protpardelle and Multiflow. Ablation studies confirm that the atom14 representation and recycling process substantially improve co-design quality.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Supp. figures/tables: [LINK](https://anonymous.4open.science/r/Pallatom-rebuttal-114C/README.md). Not repeated hereafter.\n\n**Q1:** The authors' self-defined 128-residue training crop size leads to inferior performance, and no validation evidence with larger crop sizes is provided. \n\nWe sincerely sorry that our current GPU resources and funding budget cannot support large-scale training. However, as Reviewer U5EF acknowledged, we have conducted evaluations at 3× the training sequence length while still achieving comprehensive superiority at 2× the training length, demonstrating Pallatom has robust sampling capability. More critically, Multiflow's training set maximum length is up to 384 residues. It should be noted that Multiflow's performance at its own training length distribution (L=200, 250) remains inferior to Pallatom's, even within its optimized range.\n\n**Q2:** The model architecture lacks innovation, primarily following AlphaFold3's core framework (with some algorithmic descriptions directly copied from AlphaFold3's appendix). \n\nWe emphasize that **Pallatom is a structure generation model**, fundamentally distinct from AlphaFold3's structure prediction paradigm. While reviewers noted high similarities in network frameworks (e.g., shared operators), they overlooked critical differences in computational workflows and objectives:  \n- **divergence**: Pallatom achieves *de novo* all-atom protein design with *unknown sequences*, whereas AF3 decodes pre-encoded sequence/MSA features to decode coordinates (e.g., replacing IPA with diffusion modules).  \n\nTo address all-atom representation for unknown sequences, we innovatively designed the `atom14` representation strategy – a foundational contribution that should not be overlooked.  For architectural innovations, the core module of Pallatom lies in the **AtomDecoder** unit (Figure 1B), which jointly updates residue- and atom-track single/pair representations. This differs fundamentally from AF3's diffusion module through:  \n1. **Dynamic pair representation**: Pallatom's pair features derive from *predicted intermediate structures* (with triangle updates), while AF3 uses static Pairformer-encoded pair features.  \n2. **Traversing embedding**: A novel mechanism to avoid accumulated broadcasting residue/atom features across layers via skip connections – absent in AF3.  \n\nIn Appendix C.Algorithms, we explicitly highlight AF3-derived base components in blue. All other architectures and workflows are original to Pallatom.  \n\n**Q3:** Comparisons to widely used structure-focused generative models like FoldFlow2, Proteus, CarbonNovo, and Genie2 are insufficient.  \n\nGenie2 (unpublished work) was excluded from comparison, while other methods were benchmarked. Pallatom performs best in the vast majority of cases. Tables are at **New comparison results** in LINK.\n\n**Q4:** Validation of designed protein structural plausibility (side-chain steric clash detection) and chemical bond validity. \n\nThrough comprehensive statistical analyses of bond length distributions, bond angle distributions, chi angle distributions, and conformational clashes, we demonstrate that the side-chain structures generated by Pallatom rigorously adhere to protein physicochemical constraints. Detailed visualizations and analyses are in **Sidechain analysis** in LINK.\n\n**Q5:** Consistency between sidechain design and predicted residue type.  \n\nThe DES-aa metric we employ intrinsically reflects the consistency between designed side chains and predicted sequences. To compute DES-aa, we compare Pallatom-generated structures (converted from atom14 to real atomic types) with their ESMFold-folded counterparts using all-heavy-atom RMSD. Consequently, mismatches (e.g., designed Arg vs. predicted Phe) induce aaRMSD deviations and reduce DES-aa values.\n\nTo verify consistency between sidechain design and predicted residue type, we sampled 1,000 non-training proteins (<256 residues) from AFDB and  conducted the following additional experiments:  \n1. Introduced minimal noise to native structures  \n2. Fed atom14 point clouds to Pallatom for single-step inference for sequence prediction\n3. Calculate overall average seq-recovery-rate (AAR) between predictions and ground truth and the accuracy (ACC) of amino acid recognition for four pairs of geometrically similar conformations (EQ, DN, TV, SC).\n\nResult: AAR = 93.3%, ACC=97.4%. This shows Pallatom achieves sequence-sidechain consistency.\n\n**Q6:** Concerns regarding insufficient empirical validation of training efficiency and provision of optimized code.  \n\nWe will revise the description to emphasize **sampling efficiency** . We have provided sampling efficiency comparisons showing superiority over most baseline methods (Table 7). We have released optimized code for local attention at **GPU Memory Optimization** in LINK, offering a crop-based optimization strategy (vs. basic masking), achieved **73% VRAM reduction**.", "claims_and_evidence": "1. The authors’ main claim in the abstract—that Pallatom “excels in key metrics of protein design, showing significant improvements across the board”—is not fully supported. Figure 3 indicates that Pallatom underperforms Multiflow for sequences exceeding 300 residues, so it does not consistently achieve better results in designability, diversity, and novelty. The authors suggest that the model’s training crop size of 128 residues may explain the poor performance on longer proteins, but this constraint was self-imposed. Without additional experiments or evidence showing improved outcomes when using larger crop sizes, the claim that Pallatom would rival or exceed Multiflow in long-sequence regimes remains unsubstantiated.\n\n2. The authors’ claim to have introduced a novel model architecture is equally problematic. The core framework seems largely adapted from AlphaFold3, modified mainly to handle settings without input sequences or MSAs. Several portions of the text may overstate these adaptations, and many of the algorithms included in the appendix appear to be taken verbatim from the AlphaFold3 supplementary. The manuscript should either reduce the duplicated content or provide a compelling justification for why extensive copying is required.\n\n3. While the authors claim that the proposed model exhibits excellent training efficiency in the Abstract, the only empirical metric provided is that training takes approximately ten days (see Table 5). There is no comparative analysis against other methods in terms of training duration or computational resources, making it difficult to evaluate the claimed efficiency.", "methods_and_evaluation_criteria": "More thorough assessments of sidechain geometry and validity are needed, especially since PallAtom diverges from other sequence–structure co-design methods (e.g., MultiFlow, CarbonNovo) in its treatment of sidechains. Specifically:\n1. Sidechain steric clashes: The authors should measure potential clashes among sidechains at different positions, as these can significantly impact a protein’s stability and folding kinetics.\n2. Amino-acid validity: The authors should confirm that the designed sidechains adhere to valid bond patterns matching one of the 20 canonical amino acids. Ensuring chemically feasible bond arrangements is crucial for real-world applicability.\n3. Consistency between sidechain design and predicted residue type: The authors should verify that the specified amino-acid identity aligns correctly with the geometry of the modeled sidechain, thereby avoiding mismatches (e.g., a backbone labeled “Arg” with sidechain atoms arranged as “Phe”).", "theoretical_claims": "I did not find any extensive proofs or purely theoretical derivations in the submission that would require rigorous verification of correctness.", "experimental_designs_or_analyses": "I evaluated the experimental design and noted some important gaps in the authors’ comparative analyses. \nSpecifically, the authors should provide comparisons to widely used structure-focused generative models like FoldFlow2, Proteus, CarbonNovo, and Genie2.", "relation_to_broader_scientific_literature": "The protein design method will benefit drug discovery by enabling the development of novel therapeutic proteins with tailored functions and improved properties.", "essential_references_not_discussed": "When introducing the “atom14” representation in the Introduction, the authors should cite the AlphaFold2 paper, as it originally introduced both the “atom14” concept and corresponding terminology. Providing this reference at the first mention of the “atom14” representation would offer appropriate historical context and credit.", "other_strengths_and_weaknesses": "Weaknesses:\n1. The authors have not clearly described the motivation for all-atom design in the Introduction.  Are there any specific applications that all-atom designs that have clear benefits over the step-wise design of sequence, structure, and sidechain packing? \n2. The technical novelty is very limited. The core model architecture seems largely adapted from AlphaFold3, modified mainly to handle settings without input sequences or MSAs.  Substantial portions of the algorithmic descriptions appear to be taken directly from AlphaFold2’s supplements.\n3. The paper does not compare its approach with several widely used protein-design methods, such as FoldFlow2, Genie2, CarbonNovo, and Proteus.", "other_comments_or_suggestions": "I have no other comments.", "questions_for_authors": "1. Why did the authors set such a small crop size of 128?  What motivated this design choice? There are numerous applications that require proteins longer than 128 amino acids—for example, beta-barrel or helical-bundle designs intended for ligand binding. Simply stating that the work focuses on monomer proteins that can be easily synthesized using oligo-pool methods overlooks a significant range of applications where larger or more complex proteins are necessary. \n\n2. As the authors claim training efficiency in the Abstract, did they employ any speed-up techniques to handle the local attention for the atom-level encoder/decoder? Directly applying a masking strategy to all flattened atoms would likely consume substantial memory and degrade computational efficiency. If you did implement optimizations, please consider sharing an anonymous link to your code so reviewers can examine these strategies. Otherwise, it might be that the crop size of 128 was chosen primarily to mitigate the memory and computational overhead inherent in an atom-level encoder/decoder—especially for longer proteins. Providing further details on how this trade-off is managed would strengthen the paper’s claims regarding training efficiency.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F3L0soL8W8", "reviewer": "Reviewer_U5EF", "summary": "This paper introduces a novel diffusion model for generating all-atom protein backbones and side-chains, enabling simultaneous sampling of protein structures and their corresponding amino acid sequences. The approach for all-atom protein generation relies on two key elements. First, an atom14 representation, which encodes each residue as a 14×3 vector specifying the coordinates of 14 heavy atoms. In ground truth data, missing atoms default to the Cα coordinates. Second, a state of the art denoiser architecture inspired by AlphaFold 3.\nThis paper demonstrates that a single-layer neural network operating on atomic embeddings, trained jointly with the denoiser by cross entropy, produces sequences that are highly consistent with the generated structures. Empirical results show significant improvements in protein structure–sequence co-generation, underscoring the effectiveness of this approach.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for their positive evaluation of our work. Regarding the suggestions and questions raised, we provide point-by-point responses below.\n\n**Q1:** Is it possible to use guidance with sequence-based information?\n\nBased on our ablation experiments, `hybrid14` exhibits extremely low designability. Therefore, we do not recommend introducing sequence guidance information when constructing side-chain atomic coordinates, particularly sequences estimated from the disordered structure at intermediate diffusion timesteps. According to the experimental results from Pallatom, we believe that diffusion on `atom14` is sufficient to learn effective sequence information.\n\n**Q2:** Could you include a discussion on potential limitations of the approach?\nWe acknowledge the following potential limitations in the current study:  \n1. While we adopt AlphaFold3's broadcast and aggregation operations to integrate atomic-level and residue-level features, the details of atomic information may be reduced during global attention due to averaging effects from feature aggregation.  \n2. Our local attention mechanism follows AF3's framework and focuses solely on sequence-level neighboring atoms. Incorporating spatially adjacent atomic information may further enhance the model.  \n3. The atom14 representation might inadequately capture polar side chains and complex interaction patterns in certain scenarios.  \n\n**Q3:** Can you contrast your all-atom sequence-structure cogeneration approach to that of PocketFlow [4] and PepFlow [5]? Does the atom14 representation provide an advantage over the residue representation used in these papers?\n\nWe recognize the practical significance of works like [4] and [5], which focus on full-atom design for specific applications such as pocket-ligand or protein-peptide interactions, where side-chain details are critical. However, our current work exclusively addresses *unconditional all-atom protein design* and does not yet support condition-guided pocket design or peptide-binding tasks involving small molecules. We plan to explore these application-specific extensions in future studies.  \n\n**Q4:** A figure to illustrate an amino acid structure, with backbone and side-chain atoms, alongside its atom14 representation, would improve clarity.\n\nTo enhance clarity, we added a comparative figure and supplementary descriptions detailing ATOM14's encoding of the 20 canonical amino acids in [ATOM14](https://anonymous.4open.science/r/Pallatom-rebuttal-114C/README.md).\n\n\n**Q5:** Some methodological details appear to be missing or insufficiently referenced. For instance, what's the algorithm to obtain raligned for the aligned MSE loss at line 257. Could this be \"Algorithm 28: Weighted Rigid Align\" from AF3?\n\nWe employ the standard **Kabsch** algorithm for rigid alignment, which minimizes the RMSD between structures through singular value decomposition (SVD). Unlike AlphaFold3, we do not implement weighting schemes during the alignment process.  \n\n**Q6:** Additionally, in some sections, the wording may give the impression that certain methods are new, even though they appear to have been documented previously. For instance, the mention of \"we introduce the smooth lddt loss\" at line 269 may actually refer to \"Algorithm 27: Smooth LDDT Loss\" from AF3.\n\nYes, we adopted a simplified version of the smooth LDDT loss from AlphaFold3, adapted exclusively for protein structures (excluding other biomolecular interactions).  \n\n**Q7:** Essential References Not Discussed.\n\nWe will make the following amendments to the related work.\n\n> ...ProteinGenerator employs Euclidean diffusion on one-hot encoded sequences combined with a structure prediction module to generate all-atom structures. Similarly, PLAID [2] achieves sequence design through latent space diffusion within ESM2 while decoding full-atom protein configurations. ...\n\n> Beyond protein modeling, recent works leverage atomic-level representations for localized design tasks. For instance, Abdiffuser [1] introduces a universal four-atom side-chain template for antibody CDR redesign, preserving dihedral freedom via pseudo-carbon atoms while integrating ideal amino acid templates for rotamer construction. FAIR [3] adopts a two-stage approach for protein pocket design: initial backbone/sequence generation followed by iterative refinement to ensure sequence-side-chain consistency. PocketFlow [4] extends this by simultaneously designing pocket sequences and all-atom structures through flow matching across backbone, side-chain torsion angles, and sequences. PepFlow [5] similarly utilizes multi-modal flow matching (torsion angles + sequences) for peptide design. However, such decoupled representations risk sequence-structure conflicts and steric clashes. Pallatom addresses these limitations through its atom14 representation, which intrinsically fuses structural and sequential modalities to minimize explicit conflicts.", "claims_and_evidence": "* The main claim of the paper is that \"that full-atom coordinates encodes essential protein information\". The comparison with protein backbone generation techniques as well as two-step structure-sequence prediction strategies using a tool like ProteinMPNN convincingly demonstrate that modelling all-atom 3D coordinates is an efficient strategy to generate protein structure alongside their amino acid sequences.\n\n\n* The comparison with Protpardelle supports the effectiveness of the proposed denoiser architecture. Although there are potential confounding factors (since Protpardelle differs in several other aspects) the observed performance improvements strongly suggest that the novel denoiser design is a key contributor.\n\n* The ablation study on the atom14 representation provides compelling evidence of its effectiveness.", "methods_and_evaluation_criteria": "**Evaluation Criteria**\n\n* The paper employs standard protein design metrics to assess designability and diversity, but it also adapts them to assess both all-atom structure and sequence generation. \n\n* Using two sets of metrics depending on whether ProteinMPNN is used for sequence design is a good idea. \n\n* This is complemented by an extensive benchmark against state-of-the-art methods and ablation studies (e.g., on the atom14 representation and recycling process).", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "* Given the sensitivity of diffusion models to hyperparameter choices, it is great that the authors reported how different sampler settings affect protein design metrics, as shown in Table 2.\n* While the model is trained on lengths up to 128 residues, an evaluation of the model’s generalisation capabilities on longer sequences is also included. \n* Appendices (B, E, F, G) include detailed experimental settings and further analyses that facilitate reproducibility.", "relation_to_broader_scientific_literature": "**Structure-sequence co-generation**\n\nThe paper is well contextualised within the existing literature on protein structure–sequence co-generation, referencing methods such as Protpardelle, Multiflow, and ProteinGenerator, which are also included in the benchmark. Each of these methods differs from the proposed approach in some respect. E.g. Multiflow does not operate at the level of all-atom resolution. The closest work is Protpardelle, which introduced an all-atom diffusion model for the purpose of simultaneously designing protein structures and sequences. However, this paper employs a substantially different denoiser parameterisation, an alternative way of representing protein atoms and a distinct training strategy for sequence prediction. This combination of ideas leads to significant performance improvements on standard protein design metrics.\n\n**Inspiration from AlphaFold 2 and 3**\n\nWhile AF2 and AF3 predict backbone and sidechain coordinates, and may employ a similar atom14 representation, they operate with a fixed amino acid sequence as input. In contrast, Pallatom must generate both the sequence and the corresponding 3D structure starting from an a priori undefined number of heavy atoms, which introduces additional challenges in how the atom14 representation is leveraged.", "essential_references_not_discussed": "The key contribution is a model for all-atom protein generation. It does not cite [1] and [2], respectively published at NeurIPS 2023 and NeurIPS 2024. Additionally, in the context of protein pocket design or peptide design, there are other approaches such as FAIR [3] and PocketFlow [4], and Pepflow [5] for all-atom sequence-structure cogeneration. These are not cited. \n\n\n\n[1] Martinkus, Karolis, et al. \"Abdiffuser: full-atom generation of in-vitro functioning antibodies.\" Advances in Neural Information Processing Systems 36 (2023): 40729-40759.\n\n[2] Lu, Amy X., et al. \"Controllable All-Atom Generation of Protein Sequence and Structure from Sequence-Only Inputs.\n\n[3] Zhang, Zaixi, et al. \"Full-atom protein pocket design via iterative refinement.\" Advances in Neural Information Processing Systems 36 (2023): 16816-16836.\n\n[4] ZHANG, ZAIXI, Marinka Zitnik, and Qi Liu. \"Generalized Protein Pocket Generation with Prior-Informed Flow Matching.\" The Thirty-eighth Annual Conference on Neural Information Processing Systems.\n\n[5] Li, Jiahan, et al. \"Full-Atom Peptide Design based on Multi-modal Flow Matching.\" Forty-first International Conference on Machine Learning.", "other_strengths_and_weaknesses": "**Stengths**\n\n* As the paper's title suggests, this work indeed represents a significant advance in protein design. It addresses the inherently complex challenge of joint structure–sequence generation with a conceptually simple and principled approach, while confining the intricate and ingenious parameterisation to the denoiser component of the diffusion model. This balance between overall simplicity and engineering sophistication is truly impressive. \n\n**Weaknesses** \n\nWhile the paper’s contributions appear significant, the presentation could benefit from clearer exposition:\n* A figure to illustrate an amino acid structure, with backbone and side-chain atoms, alongside its atom14 representation would improve clarity. \n* The parameterisation of the denoiser contains specialised jargon (e.g. \"traversing embeddings\" or \"feature broadcasting process\") that might be more accessible if explained in simpler terms, with an emphasis on the difference with AF3.\n* Some methodological details appear to be missing or insufficiently referenced. For instance, what's the algorithm to obtain $r^{\\text{aligned}}$ for the aligned MSE loss at line 257. Could this be \"Algorithm 28: Weighted Rigid Align\" from AF3?\n* Additionally, in some sections, the wording may give the impression that certain methods are new, even though they appear to have been documented previously. For instance, the mention of \"we introduce the smooth lddt loss\" at line 269 may actually refer to \"Algorithm 27: Smooth LDDT Loss\" from AF3.", "other_comments_or_suggestions": "* Is it possible to fix the indentation in Algorithm 1?", "questions_for_authors": "* Is it possible to use guidance with sequence-based information? \n* Could you include a discussion on potential limitations of the approach?\n* Can you contrast your all-atom sequence-structure cogeneration approach to that of [4] and [5]? Does the atom14 representation provide an advantage over the residue representation used in these papers?\n\n[4] ZHANG, ZAIXI, Marinka Zitnik, and Qi Liu. \"Generalized Protein Pocket Generation with Prior-Informed Flow Matching.\" The Thirty-eighth Annual Conference on Neural Information Processing Systems.\n\n[5] Li, Jiahan, et al. \"Full-Atom Peptide Design based on Multi-modal Flow Matching.\" Forty-first International Conference on Machine Learning.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PBbVKQwdc5", "reviewer": "Reviewer_jKdH", "summary": "The paper presents a novel diffusion-based approach for all-atom protein design. A key contribution is the atom14 representation, which unifies amino acid positions by padding them with virtual atoms. Another innovation is predicting amino acid types based on the atom14 representation instead of parallel generation. The proposed model is evaluated against state-of-the-art backbone-only and all-atom protein structure generators, demonstrating strong performance on the proposed metrics. The study includes an ablation analysis and discusses the advantages of all atom generation paradigm.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you to the reviewer for your affirmation of our work. We have summarized and replied to your questions as follows.\n\n**Q1:** The purpose of computing $f^{template-distogram}$ during inference in Algorithm 1. \n\nThis describes our self-conditioning mechanism: during sampling, the predicted structure from the previous timestep is converted into **2D distance bin encodings**. These **SE(3)-invariant features** are then initialized as residue-level pair representations via **Algorithm 3 (Template Embedder)**, which guides the diffusion process toward structurally plausible conformations.\n\nThis approach – transforming sampled structures into template-like pair representations – is similarly employed in RFDiffusion and Proteus to enhance sampling quality.  \n\n**Q2:** Evaluating backbone and side-chain generation separately. (Question 2 & Experimental Designs Or Analyses)\n\nYour suggestion to adapt the Pallatom framework for backbone-only design is insightful. To implement this, we simplified the `atom14` representation to `atom5` – a point cloud representation using four backbone atoms (N, Cα, C, O) plus Cβ. We named this method **Pallatom-bb**.\n\nHere are implementation details:\n- Retrained the model (Pallatom-bb) using identical training data.  \n- Evaluation: PMPNN 1 mode \n\nBy analogy to the right side of Table 1 and Figure 3, the experimental results for L=60 to 120 and L=150 to 400 are in [Pallatom-bb results](https://anonymous.4open.science/r/Pallatom-rebuttal-114C/README.md).\n\nPallatom-bb achieves high designability within training sequence lengths but exhibits reduced structural diversity. While showing limited extrapolation capability for longer sequences, it marginally outperforms RFDiffusion in overall metrics. Crucially, backbone-only point cloud approaches neglect critical side-chain/sequence interactions, whereas atom14's all-atom framework explicitly models structural-sequence constraints, demonstrating superior holistic performance across design benchmarks.  \n\n\n**Side-chain generation**\n\nWe believe there may be a conceptual distinction between *side-chain generation* and *all-atom design* that requires clarification. In canonical definitions:  \n- **Side-chain packing** assumes a *known sequence* and fixed backbone to optimize rotamer configurations  \n- **All-atom design** jointly designs *both sequence and structure* from noise \n\nPallatom's atom14 representation specifically addresses the latter scenario – modeling all-atom coordinates *without prior sequence knowledge*. Our framework achieves high-quality de novo protein design by explicitly resolving the ambiguity between amino acid types and their atomic geometries. While side-chain packing represents an important complementary task, it falls outside the scope of this work's objectives.  \n\n**Q3:** Have you considered two-step generation (backbone generation followed by side-chain packing) as a baseline in the experiments? If not, why?\n  \nOur understanding of this methodological distinction is as follows: side-chain packing inherently requires *prior sequence knowledge* (though sequences may originate from various sources). The field-standard approach employs inverse folding models like ProteinMPNN to derive sequences from designed backbones. This baseline methodology effectively decouples the process into two stages:  \n1. Backbone design (e.g., RFDiffusion)  \n2. Sequence design (e.g., ProteinMPNN)  \n\nUnder the PMPNN 1 protocol, Pallatom ranks second in designability (slightly below ProteinGenerator) while achieving superior diversity/novelty. For L=150-250, it delivers state-of-the-art performance, and maintains second-ranked performance for L=300-400, demonstrating robust extrapolation beyond training lengths.  \n\n**Q4:** Ablation study for LDDT loss.\n\nBelow are the comparative results from sampling 100 L=100 proteins after removing the smooth LDDT loss: \n\n| Method    | DES-aa | DES-bb(w) | DES-bb(wo) |\n|-----------|--------|-----------|------------|\n| Pallatom  | 87%    | 95%       | 95%        |\n| wo-LDDT   | 84%    | 95%       | 95%        |\n\n \nThe training observations revealed that the LDDT loss maintained consistently low values throughout training, suggesting it may act as an implicit violation loss to mitigate side-chain atomic clashes. Ablation experiments confirm the LDDT loss's role in enhancing the quality of Pallatom-generated full-atom proteins, particularly by preserving critical sequence-structure self-consistency (DES-aa reduction).\n\n**Q5:** Clarification for atom14 representation.\n\nWe have presented a comparison figure and detailed description showing the use of atom14 to represent 20 standard amino acids in the [ATOM14](https://anonymous.4open.science/r/Pallatom-rebuttal-114C/README.md).", "claims_and_evidence": "The paper supports its claim of simultaneous all-atom protein design with empirical results. However, it lacks crucial comparisons to a two-step strategy: backbone generation by FrameDiff[a], FrameFlow[b] followed by side-chain completion using DiffPack[c] or H-Packer[d]. Such comparisons would strengthen the claims.\n\na. SE(3) diffusion model with application to protein backbone generation, Yim et al., 2023\n\nb. Fast protein backbone generation with SE(3) flow matching, Yim et al., 2023\n\nc. DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packing, Zhang et al., 2023\n\nd. H-Packer: Holographic Rotationally Equivariant Convolutional Neural Network for Protein Side-Chain Packing, Visani et al., 2023", "methods_and_evaluation_criteria": "The methods align with the problem statement, but the evaluation is limited to only proposed metrics. The study does not separately analyze backbone and side-chain generation performance, which would provide more insights.", "theoretical_claims": "The mathematical description of the approach appears correct, with no major theoretical issues identified.", "experimental_designs_or_analyses": "The experimental design adequately demonstrates the model’s advantages, but the evaluation could be improved by comparing the model’s backbone and side-chain design capabilities separately. Based on the method description, it allows masking atoms, so it can be trained for backbone atom generation by ignoring side-chain atoms, and it can be trained for side-chain atom generation by setting amino acid types and backbone atoms while updating only side-chain atom positions. This would allow comparison against a wider range of baselines on additional tasks (backbone generation and side-chain packing) to evaluate model performance in more detail on task-dependent metrics. The metrics proposed in the paper do not allow decomposition of model performance for backbone and side-chain atoms.", "relation_to_broader_scientific_literature": "The paper contributes to protein design by introducing a model that simultaneously designs all atoms rather than focusing solely on backbone generation. While this direction is not entirely new, innovations such as the atom14 representation and amino acid prediction from atomic coordinates are novel.", "essential_references_not_discussed": "All essential references are discussed.", "other_strengths_and_weaknesses": "The paper is well-written but lacks justification for some architectural choices. For instance, it is unclear why $L_{atom}$ is insufficient, necessitating LDDT-based loss components. Further explanation would improve clarity.", "other_comments_or_suggestions": "1. Figures 2A-C should have larger fonts and clearer explanations.\n2. The process of setting virtual atom positions in the atom14 representation should be clarified in the main text.", "questions_for_authors": "1. What is the purpose of computing $f^{templatedistogram}$ during inference in Algorithm 1? While needed for training (LDDT subcomponent), its necessity during inference is unclear.\n2. Have you considered evaluating backbone and side-chain generation separately? Given the model’s atom masking capability, could it be trained for these tasks independently and compared with corresponding baselines?\n3. Have you considered two-step generation (backbone generation followed by side-chain packing) as a baseline in the experiments? If not, why?\n4. Why was the LDDT-based loss component included instead of relying solely on $L_{atom}$? A detailed explanation would be helpful.\n5. How are virtual atom positions assigned in the atom14 representation? Clarification would aid understanding.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oYSu9WXCtz", "reviewer": "Reviewer_iEVP", "summary": "The paper introduces Pallatom, a protein generation model that generates protein structures with all-atom coordinates. The model uses a dual-track framework with residue and atomic-level representations and introduces atom14 representation for modelling variable side-chain coordinates. Pallatom learns a diffusion model to denoise this representation reusing components from AlphaFold3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you to the reviewer for your affirmation of our work. We have replied to your questions as follows.\n\n**Q1:** Paragraph 3.3; why can't the amino acid be extracted from the atomic coordinates without using an AA classifier ? a proper all atom representation would yield the required information to extract directly the sequence based on which atom types are present.\n\nThe observed confusion likely arises from insufficient elaboration of our atom14 framework's design rationale. In this side-chain coordinate-based representation (`atom14`) that intentionally excludes elemental typing information, chemically distinct residues with similar conformations (e.g., GLU vs GLN differing in OE2/NE2 atom types) become geometrically indistinguishable – their coordinate differences are confined to minimal bond length variations. This necessitates two interdependent capabilities: (1) sequence decoding from atom point clouds, and (2) disambiguation of geometrically degenerate residues.  \n\nPallatom addresses this through a hierarchical attention mechanism: global attention captures holistic structural patterns while local attention learns fine-grained atomic interactions. This enables the model to develop rich atomic semantics that jointly encode spatial and sequential information. Crucially, our experiments reveal that a simple linear **SeqHead** layer suffices to accurately predict sequences from aggregated atomic representations, demonstrating that the learned high-dimensional embeddings inherently resolve the coordinate-to-sequence mapping without requiring complex auxiliary networks. This design philosophy ensures simultaneous optimization of structural plausibility and sequence-structure consistency during generation.  \n\n\nWe also have presented a comparison figure and detailed description showing the use of atom14 to represent 20 standard amino acids in the [ATOM14](https://anonymous.4open.science/r/Pallatom-rebuttal-114C/README.md).\n\n\n**Q2:** The practical novelty stems from the combination of various ideas from Protpardelle and AlphaFold3; as such these 2 work have introduced most of the ideas in this paper. \n\n\nThe most relevant work to ours in all-atom protein design is Protpardelle, but Pallatom exhibits fundamental differences in comparison. While Protpardelle's atom73 represents all-atom protein design by tracking 20 side-chain states per residue, its discontinuous gradient updates rely on intermediate sequence predictions. In contrast, Pallatom's atom14 achieves superior efficiency (14 vs. 73 atoms) and sampling stability through continuous coordinates gradient updates, eliminating sequence dependency. Experiments (Table 1, Figure 3) confirm Pallatom's enhanced designability, and ablation studies show that hybrid sequence-guided approaches degrade performance.\n\nRegarding differences with AlphaFold3 (AF3), these are fundamentally distinct tasks. AF3 performs conditional diffusion given sequences, utilizing a diffusion module to decode structures. Pallatom aims to design all-atom proteins with unknown sequences and introduces the atom14 representation for this purpose. While we do employ some basic components from AF3 in our model, the innovative integration of these components—particularly the fusion and update mechanisms between different features—required complete redesign. To our knowledge, Pallatom is the first protein design method in the field to utilize AF3's fundamental components, whereas existing approaches like FrameDiff primarily build upon AF2's Invariant Point Attention with minor modifications. The core of Pallatom lies in its AtomDecoder unit (Figure 1B), which integrates and updates residue- and atom-track single/pair features, featuring critical distinctions from AF3's diffusion module:\n\n1. **Dynamic pair representation**: Pallatom's pair features derive from *predicted intermediate structures* (with triangle updates), while AF3 uses nearly static Pairformer-encoded features.  \n2. **Traversing embedding**: A novel mechanism to avoid accumulated broadcasting residue/atom features across layers via skip connections – absent in AF3.  \n\n\n**Q3:** EDM is not defined anywhere; I assume it refers to Karras et al. ?\n\nThank you for identifying this oversight. We indeed employ the EDM framework [1] and will ensure proper citation of the reference in our work.  \n\n[1] Karras, Tero, et al. \"Elucidating the design space of diffusion-based generative models.\" *Advances in Neural Information Processing Systems* 35 (2022): 26565-26577.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "The claims in this paper are supported by some comparison against other representative protein generation models, including all-atom models. \nExperimental results showing superior performance on metrics like designability, diversity, and novelty. The ablation studies demonstrate the importance of some components like recycling mechanism and the atom-14 representation.", "theoretical_claims": "no theoretical claims", "experimental_designs_or_analyses": "yes", "relation_to_broader_scientific_literature": "This paper builds on top of previous work in protein structure prediction (AlphaFold) and generation. It advances the field by providing a new all-atom protein generation without separate sequence design step.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n* Novel atom14 representation that effectively handles variable side-chain atoms\n* performs competitievly with baselines across multiple metrics\n* efficient sampling time compared to other methods that performed similarly\noverall using some components from modern architectures for structure prediction (Alphafold3) in the setting of generation, as such the proposed model is interesting from an empirical point of view for its performance. moreover the generation of all-atom molecules is a challenging problem.\n\nWeakness\n* the practical novelty stems from the combination of various ideas from Protpardelle and AlphaFold3; as such these 2 work have introduced most of the ideas in this paper.\n* see my question below; it feels strange to me that the model is called an all-atom model if the amino acid cannot be properly inferred from the generated atom14 representation without resorting to a classifier.", "other_comments_or_suggestions": "EDM is not defined anywhere; I assume it refers to Karras et al. ?", "questions_for_authors": "Paragraph 3.3; why can't the amino acid be extracted from the atomic coordinates without using an AA classifier ? a proper all atom representation would yield the required information to extract directly the sequence based on which atom types are present.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a novel framework for training protein generative models from an all-atom representation. The proposed design is simple and it relies on the atom14 representation. A a dual-track framework is introduced that tokenizes proteins into token-level and atomic-level representations, integrating them through a multi-layer decoding process with "traversing" representations and recycling mechanism. Inspired by AlphaFold3, the model generates full-atom protein structures from noise, outperforming previous methods in designability, diversity, and novelty. 

The reviewers have recognized the novelty of proposed method, strong emphirical results and insightful ablations experiments. However, the writing in this paper requires further improvements, allowing the readers to truely appreciate the contributions of this paper.  As reviewers suggested please:
- Define termonology like "feature broadcasting" etc, and provide more insights into your architecture.
- Use visual examples to show atom14 representaton
- Include additional details on how data is processed such that your work can be reproduced by the community
- Give approperate credit to AlphaFold and other previous works

Although czVh made good critisms on the importance of long protein generation in practice which Pallatom currently lacks, the critism is not substantial enough for rejecting the paper in its current forms.

Given the merits of this submission, I'm happy to recommend acceptance at ICML.
Decision: Accept (spotlight poster)