Full-Atom Peptide Design with Geometric Latent Diffusion
Authors: Xiangzhe Kong
Yinjun Jia
Wenbing Huang
Yang Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Peptide Design
Geometric Latent Diffusion
Full-Atom
Affine Transformation
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Peptide design plays a pivotal role in therapeutics, allowing brand new possibility to leverage target binding sites that are previously undruggable. Most existing methods are either inefficient or only concerned with the target-agnostic design of 1D sequences. In this paper, we propose a generative model for full-atom Peptide design with Geometric LAtent Diffusion (PepGLAD) given the binding site. We first establish a benchmark consisting of both 1D sequences and 3D structures from Protein Data Bank (PDB) and literature for systematic evaluation. We then identify two major challenges of leveraging current diffusion-based models for peptide design: the full-atom geometry and the variable binding geometry. To tackle the first challenge, PepGLAD derives a variational autoencoder that first encodes full-atom residues of variable size into fixed-dimensional latent representations, and then decodes back to the residue space after conducting the diffusion process in the latent space. For the second issue, PepGLAD explores a receptor-specific affine transformation to convert the 3D coordinates into a shared standard space, enabling better generalization ability across different binding shapes. Experimental Results show that our method not only improves diversity and binding affinity significantly in the task of sequence-structure co-design, but also excels at recovering reference structures for binding conformation generation.

Full-Atom Peptide Design with Geometric Latent Diffusion
Xiangzhe Kong Yinjun Jia Wenbing Huang Yang Liu L2,5.
Dept. of Comp. Sci. & Tech., Tsinghua University 2 Institute for AIR, Tsinghua University Schooo of Life Sciences, Tsinghua University *Gaoling School of Artificial Intelligence, Renmin University of China Shanghai Artificial Intelligence Laboratory, Shanghai, China
Abstract
Peptide design plays a pivotal role in therapeutics, allowing brand new possibility leverage target binding sites that are previously undruggable. Most existing methods are either inefficient or only concerned with the target-agnostic design of ID sequences. In this paper, we propose a generative model for full-atom Peptide design with Geometric LAtent Diffusion (PepGLAD) given the binding site, We first establish a benchmark consisting of both ID sequences and 3D structures from Protein Data Bank (PDB) and literature for systematic evaluation. We then identify two major challenges of leveraging current diffusion-based models for peptide design: the full-atom geometry and the variable binding geometry. To tackle the first challenge, PepGL.AD derives a variational autoencoder that first encodes full- atom residues of variable size into -ieed--dmeiiinnll latent representations, and then decodes back to the residue space after conducting the diffusion process in the latent space. For the second issue, PepGLAD explores receptor-specific affine transformation to convert the 3D coordinates into shared standard space, enabling better generalization ability across different binding shapes. Experimental Results show that our method not only improves diversity and binding affinity significantly in the task of soquence-structure co-design, but also excels at recovering reference structures for binding conformation generation.
Introduction
Peptides are short chains of amino acids and acts as vital mediators of many protein-proted ppeeinn interac- tions in human cells. Designing functional peptides has attracted increasing attention in biological research and therapeutics, since the highly-flexibe conformation space of peptides allows brand new possibility target binding sites previously undruggable with antibodies small molecules [17 35
The key of peptide design to generate peptides that interact compactly with target proteins (see Figure D). since they mostly exhibit flexible conformations [20] unless bound to these receptors (60].
Conventional simulation or searching algorithms rely on frequent calculations of physical energy functions 66 29. which are inefficient and prone to poor local optimum. Recent advances illuminate the remarkable success of exploiting geometric deep generative models, particularly the equivariant diffusion models 45 722, for molecule design [21] (39), antibody design 29
44 (34 and protein
design 664 28), as well as latent diffusion models further enhancing the performance [7]] [8].
Inspired by these successes, natural idea is leveraging diffusion models for peptide design well, which, yet, is challenging in two aspects. From the dataset aspect, existing databases (PepBDB (65). Propedia (46)) merely collect data from Protein Data Bank (PDB) [5), neither performing filters according to practical relevance 44 and redundancy, nor providing an adequate split for evaluation.
"Correspondence Wenbing Huang <hwenbing @ 126.com>, Yang Liu <liuyang201 @tsinghua.edu.com
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Ni6. S.
Nipa E
Design
Poptide Recessor
MiRa
Interactions
NiR.
Peptide design requires generating peptides that form compact interactions with the binding site on the receptor. The intricacy of protein-peptide interactions demands efficient exploration the vast space for sequence-structure co-design. Right: Different binding sites (a, b, c, d) adopt disparate center offsets and geometric shapes, approximating variable 3D Gaussian distributions that deviate from (0, I). We propose to convert the geometry into a standard space approximating standard Gaussian, via affine transformation derived from the binding site ($3.3)
Figure Left:
Therefore, this paper first curates a benchmark from PDB S and the literature 59 and then systematically evaluates the generative models in terms of diversity, consistency, and binding affinity,
From the methodology aspect, nontrivial adopt latent diffusion models characterize the geometry of protein-peptide interactions. The first nontriviality stems from the full-atom geometry, which determines the comprehensive potein--eppide interactions in the atomic level, yet difficult to preserve. Throughout the generation process, the type of each amino acid always changes and thus requires us to generate different number of atoms, which is unfriendly to diffusion models that prefer fixed-size generation. Current latent diffusion models on molecules [71] or protein backbones [18] still tackle tasks with fixed number of atoms, thus leaving this challenge untouched. The second nontriviality lies in the variable binding geometry. Diffusion models are typically implemented directly in the data space, which might range), yet ill-suited for our case on 3D) coordinates where the value range is not fixed and even cursed with high variances due to the rich diversity in protein-peptide interactions. These variances define divergent target distributions of Gaussian with disparate expectation and covariance, which hinders the transferability of the diffusion process across different binding sites and thereby yields unsatisfactory generalization capability. Unfortunately, this point is seldom investigated previously.
suitable for regular data (e.g. images with fixed value
To address the above problems, we propose a powerful model for full-atom Peptide design with Geometric LAtent Diffusion (PepGLAD) with the following contributions:
We construct new benchmark from PDB and literature based on practical relevance and non-redundancy, then systematically evaluate available requence-structurr co-design models on the task of target-specific peptide design.
To capture the full-atom geometry, we first learn Variational AutoEncoder (VAE) obtain a fixed-size latent representation (including a 3D coordinate and a hidden feature) for each residue of the input peptide, and then conduct the diffusion process in this latent space, both of which are conditioned on the binding site to better model protein-peptide interactions. Notably, the proposed design enables our model to accommodate full-atom input and output.
Regarding the variable binding geometry, derive shared standard space from the binding sites by proposing a novel skill- transformation is computed by the center offset and covariance Cholesky decomposition of the binding site coordinates, serving as a mapping from the binding site distribution to standard Gaussian. With the affine transformation applied both the binding sites and the peptides, we are able to project the shape of all complexes into approximately standard Gaussian distribution, which facilitates generalization diverse binding sites.
afline transformation. Such affine
Favorably, all the aforementioned models and processes meet the desired symmetry, i.e., E(3)- equivariance, as proved by us. Experiments on sequence-structure co-design and complex conforma- tion generation demonstrate the superiority of PepGLAD over the existing generative models.

---

2 Related Work
Peptide Design Conventional methods directly sample residues [6] or building blocks from libraries containing small fragments proteins 226 57 g with guidance from delicate physical energy functions (2). These methods are time-consuming and easy to be trapped by local optimum. Recent advances with deep generative models mainly focuses on target-agnostic ID language models [48], antimicrobial peptides [13] 63], subtype of peptides with a-helix [68 69]. While geometric deep generative models are exhibiting notable potential in other domains of target-specific binder design (e.g. antibodies), their capability of target-specific peptide design remains unclear, which is the first problem we answer in this paper. Other contemporary work includes peptide design algorithms with
flow matching frameworks [38 40
Geometric Protein/Antibodd Design Protein design primarily aims to generate stable secondary or tertiary structures 127]. where diffusion models demonstrate inspiring performance [66] [88 3 722 In particular, RFDiffusion [64] first generates backbones via diffusion, and then designs the sequences through cycles of inverse folding and structure refining with empirical foree fields. Chroma [28] adopts a similar strategy, but further explores controllable generative process with custom energy functions. Antibody design, encompassing special family of proteins in the immune system to capture antigens, mainly focuses on inpainting
regions (CDRs) at the interface between the antigen and the framework 33 34 61], where the geometric diffusion models exhibit promising potential [44 455 in co-designing sequence and structure. Unlike antibodies which are constrained by framework regions, peptides exhibit more irregular binding pattern and greater flexibility, adapting to binding sites upon interaction [35]. Thus the target distributions are remarkably divergent on different binding sites, posing an urgent need for more robust generative modeling.
Geometric Latent Diffusion Models Diffusion models learn a denoising trajectory to generate
desired data distribution from prior distribution, commonly standard Gaussian [54] [5] 25 Recent
literature extends diffusion to 3D small molecules satisfying the Eq)uvauuiveiianee 770, which triggers subsequent advances in geometric design of macro molecules (e.g. antibody, protein) as
aforementioned. Further efforts are made to latent diffusion models [53] 71 18), which implement the generative process in the compressed latent space of pretrained auto- eeto-enccddes, improve the performance. Compared to the literature that either encodes ttom-wise representation in the latent space for small molecule generation 71. or compress fixed number of atoms into one latent node for protein backbone generation [18) we explore compression of the full-atom geometry by directly generating different residues with variable number of atoms in the latent space. Moreover, we propose a novel technique, namely the data-specific affine transformations, to enhance the generalization ability of diffusion models, which is barely explored before.
3 Our Method: PepGLAD
We first define the notations in the paper and formalize peptide design in $3.1 The overall workflow of our PepGLAD is presented in Figure which consists of three modules: (1) An autoencoder that defines the joint latent space for sequences and structures conditioned on the full-atom context of the binding site ($3.2); (2) An affine transformation derived from the binding site to project the 3D geometry into a standard space approximating standard Gaussian distribution ($3.3); (3) A latent diffusion model trained on the standard latent space ($3.4). Finally, summarize the training and
the sampling procedures in $3.
3.1 Definitions and Notations
We represent binding sites and peptides as geometric graphs g = {(x; X,)}, where each node i is residue with its amino acid type and the coordinates of all its atoms X, R In later sections, we use the simplified notations i E 9 denote that a node is in the geometric graph G. and [g] to denote the total number of nodes in G. We use G, and G3 to represent the geometric graph of the peptide and the binding site, respectively. In this work, the binding site incorporates residues on the target protein within 10Å distances to the peptide residues based on C3 atoms which alleviates leakage of the side-chain interactions. Note that the threshold (10Å) is chosen to be large to better reduce leakage of the peptide geometry.
3

---

(Lrg) X.I
{(=%, #])
{(, F(2)))
E.
Forward Diffusion
AER
R
Receptor
Peptidee
D
F-1
Denaising Diffusion Eg
Data Space
Latent Space
Standard Space
(A)
Figure 2: Overall architecture of PepGLAD. (A) Variational AutoEncoder ($3.2) compressing
the sequence and the structure {(x, x,) } of the peptide into the latent space {(z,. Z,)} with the encoder Eg. and decoding the sequence and full-atom geometry from the latent states with the decoder Dg. (B) Affine Transformation F' p: projecting the geometry approximately W I) via the receptor-specific affine transformation derived from the binding site, and recovering the data geometry with the inverse of F after the diffusion generative process. (C) Latent Diffusion ($3.4): jointly generating and ž; in the standard latent space.
Task Definition Given the binding site Go. we aim to obtain a generative model po conforming to the distribution of binding peptides (9,)))))
3.2 Variational AutoEncoder
The autoencoder
62 consists of an encoder Eg that encodes the peptide 9p the presence of the binding site Gú into a latent state Gz, and decoder 'De that reconstructs the peptide from the latent obtain Gy {</ X() ). To encourage & to learn contextual representations of residues, corrupt 25% of the residues in Gp with [MASK] type to obtain Gp as the input:
G = Ep. Gn), 9 = De( G,),
where G. {(2,, Zi)li E Gp} contains the latent states z; R (h in this paper) and ž, sampled from the encoded distribution (zz; μ4, o.) and (Z cū, o;) using the reparameterization trick 32.. We borrow the adaptive multi-channel equivariant encoder in dyMEAN 34 for both & and De to capture the full-atom geometry. In the decoder Dg. we factorize the joint distribution of sequences and structures as follows:
Pe G Gb) = Pεi (1/192, Go)pe, (1114 9114,,9.,990 9o).
where the sequence is first decoded and then the all-atom geometry, initialized with replications of 2, is reconstructed. The training objective the autoencoder consists of the reconstruction loss Crecon and the KL divergence LKL to constrain the latent space. The reconstruction loss includes cross entropy on the residue types, mean square error (MSE) on the full-atom structures, and an auxilary loss Loux bond lengths and angles [31
Lrccon ii) = H (p(ax) + MSE(X, X() + (i)
where denotes cross entropy. We include details of Laue in Appendix A The KL divergence con- strains 2, and E, with the prior \O0, , I) and ( I), respectively, where F. denotes the coordinate of the alpha carbon (C.) in node i:
(i) = A1 DKI. (N, (l,,, diag(o:))) + Xa (NI, (ig, diag(G:)),,
where DKL denotes the KL divergence, A1 and X2 reweight the contraints on the sequence and the structure, respectively. LxL prevents the scale of z; from exploding and constrains E, around C. to retain necessary geometric information. Such regularization also helps ensure consistent scales between the peptide latent coordinates and the pocket, mitigating potential issues arising from their different levels of abstraction. Then we have the overall training objective of the variational autoencoder as follows:
CAF Eneg, (Croon (1) + (xx(1))/1
(5)

---

We have explored E(3)-invarian) invariant latent space, which appears to have difficulties in reconstructing the full-atom structures since it lacks information of geometric interactions with the pocket atoms (Appendix
3.3 Receptor-Specific Affine Transformation
With the latent space given by the autoencoder, we further exploit standard space obtained from receptor-specife sppccifi affine transformations, which enhances the transferability of diffusions on disparate binding sites (see Figure I. Most peptides fold into complementary shape upon binding on the receptor [60] 4T] Thus, the target distribution is inherently characterized by the shape of the binding site. Given the wide disparity in binding geometries, directly implementing diffusion in the data space yields minimal transferability among different binding sites. To address this deficiency, we propose to implement the diffusion process on a shared standard space converted via an affine transformation derived from the binding site. Formally, denoting the C. coordinates of the residues in given binding site Go as R E 3 (66,,
derive their center ş E[R] R3 and covariance E Cov R, R) R3x3, so that these coordinates can be regarded as sampled from the distribution Np, Σ). We then calculate the Cholesky decomposition [9 of Σ.
Σ L B R3x3
(6)
where L is lower triangular matrix. L is unique 19 and invertible since the covariance matrix a real-valued symmetric positive-definite matrix Then we can define the affine transformation F R + R., which enables the projection of the geometry into the standard space approximating standard Gaussian F(R) _N(0, I). Further, can easily obtain the inverse of F as:
F = i i- ji),
(T) Li μZ.
With the above definitions, for each given binding site Gb, we transform the geometry via the derived F to obtain the standard space, where the diffusion model is implemented, and recover the original geometry with F-1 (see Figure 2 after generation. Notably, we have the following proposition to ensure that the equivariance is maintained under the proposed affine transformation with scalarization-base equivariant GNNs [23 [6]:
Proposition 3.1. Denote the invariant and equivariant outputs from a scalarization-basee E(3)- equivariant GNN as S(Ah,, T.3) and f([h;; H..), respectively. With the definition of in Eq. Ng E E(3), we have f((h,, F(æ:)3) = f((hi, Fg E:}}) and F- F(E:))) Ff Fglg- E.))),, where Fg is derived on the coordinates transformed by g. Namely, the E(3) equivvaaiee is preserved if we implement the GNN on the standard space and recover the original geometry from the outputs.
The proof is in Appendix B This is vital since it indicates the Markov kernel is E(3) B(3)-equivariant, and thus ensures the Invarince of the probability density in the diffusion process (70] Note that our variational autoencoder ($ (.22) and latent diffusion model ($ 3.44 are already designed to be equivariant component is to encourage better generalization of the diffusion processes. Indeed, it is nontrivial analyze whether such implementation will break the equivariance of our workflow. Luckily, equivariant networks [23). which used
without the proposed affine transformation here. The purpose of defining such
Proposition 3.I manages to prove that scalarization-bast in our autoencoder and diffusion model, are seamlessly compatible with such affine transformation, naturally preserving equivariance without any requirements of adaption.
3.4 Geometric Latent Diffusion Model
With the aforementioned preparations, the discrete residue types encoded continuous latent representations {2i}. and the full-atom geometry is also compressed and standardized into 3D) vectors (z; (O, I). Therefore,
are ready implement a diffusion model on the standard latent space to generate z; and E, The forward diffusion process gradually adds noise to the data from 0 to t T. resulting in the prior distribution (O, I). The reverse diffusion process
2The binding site has at least 3 nonoverlapping nodes, namely rank( R) 3, thus we can ignore the corner case of semi-posien semi-positive definite matrices.
5

---

generates data distribution by iteratively denosing the distribution from = T to = 0. We denote य | _ and 9 {(2{, } as the intermediate state for node and the entire peptide at time step I, respectively. For simplicity, we assume both G and the binding site Go are already standardized via the transformation Fb in Eq. 7 Then we have the forward process as:
q - = N( V1-B8. 1 1T))
B
g( N -(aj; Va u (1- a I),
(9)
where 8 is the noise scale increasing with the timestep from 0 to conforming to the cosine schedule [51] and a' -1:=i - ). Then the state at timestep t can be sampled as:
प. a'' u. + (1 - d')E,;
(10)
where E; .N(0, .). Following Ho et al. 225 the reverse process can be defined with the reparame- terization trick as:
po 9 Gb) (ū
pāo (9.. G.), I).
(9, 96) Va i
V1- ai co (6.t))))
(12)
where a'= =1 1- 9r, and Co is the denoising network also implemented with the equivariant adaptive multi-channel equivariant encoder in dyMEAN (34] to retain full-atom context of the binding site during generation and preserve the equivariance under affine transformations (Proposition [.1). Finally, we have the objective at time step / as MSE between the predicted noise and the added noise in Eq. 10 as well as the overall training objective LLOM as the expectation with respect to t:
LIDM = BpJUniform(1....T)
|& (0 Go (109/111111
(13)
3.5 Training and Sampling
The training of our PepGLAD can be divided into two phases where a variational au- toencoder first trained and then a diffusion model is trained on the standard latent space. We provide the overall training procedure in Algorithm [ (see Appendix D). Note that a smooth and informative latent space is necessary for the consecutive training of the diffusion model, thus we resort to unsupervised data from protein fragments apart from the limited protein- peptide complexes for training the autoencoder, which we describe in Appendix E
Training
Sampling in Ordered Subspace The sampling procedure includes generative diffusion process on the standard latent states, recovering the original geometry with the inverse of F in Eq. and decoding the sequence as well as the full-atom structure of the peptide (see Algorithm 2 in Appendix DD. A problem here is that the unordered nature of graphs is not compatible with the sequential nature of peptides, thus the generated residues may have arbitrary permutation on the sequence order. Inspired by the concept of glassifier-uuided sampling [15), we first assign an arbitrary permutation P on the sequence order to the nodes. Then we steer the sampling procedure towards the desired subspace conforming to P with the following empirical classifier p p[l| E }), which estimates the probability of the current coordinates belonging to the desired subspace:
r(1 ( }} = exp(- Σ EP
I))) R[
(14)
d (pa 30.). d pa + 30d E(d) = (HL 30d) d. d< pd 3ad,
(15)
0, otherwise,
where pa and oa are the mean and variance of the distances of adjacent residues in the latent space measured from the training set. Intuitively, this classifier gives higher confidence if the adjacent (defined by P) residues are within reasonable distances aligning with the statistics from the training set. Nevertheless, the effect of the guidance relatively minor, which is only technical trick to enhance the robustness. We provide more details in Appendix G
Experiments
4.1 Setup
Task We evaluate our PepGLAD and baselines on the following tasks: (1) sequence-structure co-design (42 aims to generate both the sequence and the structure of the peptide given the specific

---

binding site on the receptor (i.e. protein). (2) Binding Conformation Generation ($.3) requires to generate the binding state of the peptide given its sequence and the binding site of interest.
Dataset We first extract all dimers from the Protein Data Bank (PDB) [5] and select the complexes with receptor longer than 30 residues and a ligand between 4 25 residues [59]. Then remove the duplicated complexes with the criterion that both the receptor and the peptide has a sequence identity over 90% (56), after which 6105 non-redundant complexes are obtained. To achieve the cross-target generalization test, we utilize the large non-redundant dataset (LNR) from Tsaban et al. [59] as the test set, which contains 93 pooeiin-ppptid complexes with canonical amino acids curated by domain experts. We then cluster the data by receptor with sequence identity threshold of over 40%, and remove the complexes sharing the same clusters with those from the test set. Finally, the remaining data are randomly split based on clustering results into training and validation sets, yielding new bechmark calling PepBench. Further, we exploit 70k unsupervised data from protein fragments (ProtFrag) to facilitate training of the variational autoencoder. We also implement split on PepBDB 65 based on clustering results for evaluation. We show details and statistics of these datasets in Appendix
Baselines We first borrow three baselines from the antibody design domain. HSRN 300 autore- gressively decodes the sequence while keeps refining the structure hierarchically, from the C. to other atoms. dyMEAN [34] is equipped with an full-atom geometric encoder and exploits iterative non-autoregressive generation. DiffAb 44 jointly diffuses on the categorical residue type, the coordinate of C, as well as the orientation of each residue. Next, we explore two baselines from the general protein design. RFDiffusion [4 exploits a pipeline that first generates the backhone via diffusion and then alternates between inverse folding [14 and structure refining based on a physical energy function (2. AlphaFold 2 [31 is the well-known model for protein folding, which also shows certain abilities on peptide conformation prediction (59). We also include two traditional methods. (26] designs peptides by first docking an existing scaffold to the binding site, and then optimizing the peptide with cycles of mutations guided by energy functions. FlexPepDock 42] is designed for flexible peptide docking via optimization in the landscape of physical energy function 22 Implementation details are provided in Appendix[[
4.2 Sequence-Structure Co-Design
Metries A favorable generative model should produce diverse candidates while maintaining fidelity to the desired distribution. To comprehensively evaluate the models, we generate 40 candidates for each receptor and employ the following metries: (1) Diversity, Inspired by (72). we measure the diversity via unique clusters of sequences and structures. Specifically, we hierarchically cluster the structures based on pair-wise root mean square deviation (RMSD) of C.. The diversity of
structures Div is defined as the number of clusters versus the number of candidates. similar procedure can be applied to the sequences to obtain Div utilizing the similarity [66 derived from
alignment (24] Then the co-design diversity is v Div Div streer (2) Consistency. We measure how DiV well the models learn the ID&3D joint distribution by the sequence-structto
consistency, quantified via Cramér's ' V [12] association between the clustering labels (as in Diversity) of the sequences and the structures. High consistency indicates that candidates with similar sequences also have similar structures, implying that the generative model effectively captures the dependency between 1D and 3D. (3) AG. Aligned with the literature [34 441 we employ the binding energy (kcal/mol) provided by Rosetta (2). widely-used suite for biomolecular modeling with physical energy functions, to evaluate the binding affinity of the generated candidates. Lower AG indicates stronger binding between the peptide and the target. (4) Success. We report the proportion of successful designs (i.e.
AG < O, indicating no severe atomic clashes or twisted conformations) among all the candidates.
Table I: Evaluation on sequence-structure co-design. On each target, 40 candidates are generated for evaluation. Div. and Con. abbreviations for diversity and consistency, respectively.
PepBench
PepBDB
Model
Div.(t)
(o)
AG()
Success
Div.(t) Con
AG()
Success
Test Set
-35.25
95.70%
-35.96
95.79%
H
0.158
0.0
0
10.46%
0.111
0.0
0
10.86%
dyMEAN
0.150
0.0
-2.26
14.60%
0.150
0.0
-1.92
6.26%
DiffAb
0.427
0.670
-21.20
49.87%
0.269
0.463
-18.40
41 41.45%
PepGLAD (ours)
0.506
0.789
-21.94
55.97% %
0.692
0.923
-21.53
48.47% %

---

For all metries except AG, we first compute values for each receptor individually and then average the results across different receptors. For AG, we identify the best candidate on each receptor as the outputs and report the median value across different receptors. Details about the metrics are provided in Appendix H
including discussion on AAR (Appendix
[H.1) and consistency (Appendix H.2
Results Table TableDillustenn iillstratte that our PepGLAD generates significantly more diversified and consistent peptides with better binding energy and success rates compared to the baselines. When benchmarking HSRN, dyMEAN, and DiffAb, which perform well on antibody CDR design, we observe notable performance gap between non-diffusion baselines (i.e. HSRN, dyMEAN) and the diffusion- basedd baseline (i.e. DiffAb), suggesting the higher complexity in peptide design and the need for stronger modeling capabilities. Compared DiffAb, which operates on categorical residue types, C. coordinates and orientations, our PepGLAD (1) better captures the dependency between sequence and structure, as indicated by higher diversity and consistency, since diffusion is implemented on the latent space where the representation of sequence and structure are nicely correlated by the autoencoder; (2) more effectively captures the intricate pooein- interactions, demonstrated by better AG and success rates, since leverage the full-atom context of the binding site and enhances generalization capability by converting the geometry into standard space. We showcase two candidates designed by our PepGLAD with favorable binding energy given by Rosetta in Figure 3 Furthermore, the diversity within successful designs is 0.632, which higher than that of all designs (0.506), indicating the high structural flexibility of peptides upon successful binding.
PepGLAD
PepGLAD
Receptor
Peptide
Hydraphilic
Hydrophobic
Figure 3: Top: A generated candidate confined within the binding site (PDB=4cu4, AG=-34211.. Bottom: A generated candidate with complementary shape the binding site (PDB=3pkn, AG= 33.32). Both candidates form compact interactions at the interface.
We also evaluate our PepGLAD against two sophisticated pipeline systems Table 2 The traditional method (i.e. AnchorExtension) limited by low efficiency, thus we can only afford outputting 10 candidates for each receptor. For a relatively fair comparison with RFDiffusion. we refine the structure of the generated candidates using the empirical force field in RFDiffusion. However, the comparison may still disadvantage our PepGLAD, given that RFDiffusion is finetuned from a model pretrained on large-scale dataset [64]. Nevertheless, as demonstrated Table our model still exhibits marvelous superiority on diversity, consistency, and success rate, while achieving competitive binding energy AG, with obviously higher efficiency.
Table 2: Evaluation sequence-structure co-design with two well-established systems. Time cost is measured as the total time spent divided by the number of designed candiates.
Model
Div.(1)
Con.(t)
AG())
Success
Time
AnchorExtension
0.245
0.423
-26.80
84.30%
735s
RFDiffusion
0.259
0.696
-33.82
79.68%
61s
PepGLAD (ours)
0.506
0.789
-29.36
92.82%
'HSRN and dyMEAN generate homogeneous structures that are clustered together yet still sample very different sequences, leading association between squence and structure.

---

recover the 4.3 Binding Conformation Generation
Metries For each receptor, we generate 10 candidates and report the median value of the following metrics across different receptors measure how well the generated distribution
reference conformation: (1) RMSDs,, : Root mean square deviation on the coordinates of C, between a candidate and reference structure with the unit A. (2) RMSD RMSD on all atoms to measure the quality of the full-atom geometry. (3) DockQ 4 A comprehensive metric evaluating the full-atom similarity on the interface between a candidate and reference complex. It ranges from 1, with values above 0.23 and 0.49 considered as acceptable and medium quality, respectively.
Table 3:
Evaluation on binding conformation generation. On each target, 10 candidates are generated calculate the optimal recall of the reference conformation.
PepBench
PepBDB RMSDacm(U) DockQ(f)
Model
RMSDe, (L)
RMSD (4)
DockQ(f)
RMSDe, (D)
FlexPepDock
6.43
7.52
0.393
AlphaFold
8.49
9.20
0.355
dyMEAN
7.96
8.35
0.374
17.64
17.56
0.142
HSRN
6.02
7.59
0.508
9.28
9.72
0.394
DifiAb
4.23
7.60
0.586
13.96
13.12
0.236
LepGLAD (ours)
4.09
5.30
0.592
8.87
8.62
0.403
Results As shown in Table 3 PepGLAAD surpasses all the baselines in terms of both RMSD and DockQ by large margin, highlighting the superiority of incorporating the full-atom context and the binding-site shape into the latent diffusion process. Additionally, we present the distribution of the best RMSDs.. on different test receptors using box plots and showcase generated conformation highly resembling the reference in Figure The distribution reveals that our model achieves favorable performance on RMSDs, with lower variance on the test set compared to other baselines, exhibiting robust generalization ability across disparate binding sites.
FlexPepDock
Reference (PDO: 3vaw)
by PepGLAD
dyMEAN
HSRN
Alphafold
PepGLAD
RMSD 11.66
BMSD -M
RMSDICA)
Docka -0.846
Figure 4: The distribution of RMSDs, on the test set of PepBench and a visualized sample.
5 Analysis
We conduct the following ablations: the full-atom geometry (Full-Atom); the affine transformation (Affine); the unsupervised data from protein fragments (ProtFrag) and the mask policy (Mask) when training the autoencoder. Note that generative performance is assessed from various aspects, and improvement in one aspect at the disproportionate expense others might be meaningless. Thus, we additionally compute the average of all the metries to evaluate the comprehensive effect of each module, where AG is normalized by the statistics on the test set. Table demonstrates the following observations: (I) Discarding the full-atom context results in a significant degradation on all metrics, especially the success rate, implying the necessity of the full-atom context in capturing the intricate protein-peptice interactions; (2) Implementing the diffusion directly on the data space without the proposed affine transformation incurs a notably adverse impact on all metrics, indicating the remarkable enhancement on the generalization capability made by the affine transformation; (3) Training without the unsupervised data leads to a less informative latent space, exerting a negative effect on the binding energy and success rate; (4) Removal of the mask policy reduces the correlation between sequence and structure in the latent space, thus harms the consistency.

---

Table Ablations on different components. Avg. computes the average all metrics, where AG first normalized by the median value of the references test set.
Ablations
Div.(t)
Con.(1)
AG(L)
Success
Avg.
PepGLAD
0.506
0.789
-21.94
55.97%
0.619
w/o Full-Atom
0,441
0.751
-20.87
51.18%
0.574
w/o Affine
0,450
0.740
-19.08
52.39%
0.564
w/o ProtFrag
0.535
0.760
-20.16
52.15%
0.597
w/o Mask
0.422
0.741
-20.45
57.44%
0.579
6 Limitations
Despite the promising results, we acknowledge several limitations which might be addressed by future work. First, the binding affinity assessment relies on the Rosetta scoring function proxy for wetlab experiments. There may be discrepancies between the predicted and actual binding energies. The ultimate test of peptide utility its performance vivo, which is too costly for large-scale evaluation. Nevertheless, this is a problem confronting the entire community, and we hope future research might propose more reliable silico proxies bridge the gap. Second, while this paper addresses peptide design from the aspect of proteins, it might also be reasonable to think from the aspect of small molecules if the peptides are short enough. Under such cireumstances, it is also beneficial to further explore counterparts of methods for small molecule design [4] 52 39 we leave for future work.
which
Conclusion
In this paper, we first assemble dataset from Protein Data Bank (PDB) and literature to benchmark generative models on target-specific peptide design in terms of diversity, consistency, and binding energy. Subsequently, we propose PepGLAD. powerful diffusion-based model for full-atom peptide design. In particular, we explore diffusion on the latent space where the sequence and the full-atom structure are jointly encoded by a variational autoencoder. We further propose receptor-specific affine transformation technique to project variable geometries in the data space into a standard space, which enhances the transferability of diffusion processes on disparate binding sites. Our PepGLAD outperforms the existing models sequence-structure co-design and binding conformation generation, exhibiting high generalization across diverse binding sites. Our work represents pioneering effort in the exploration of deep generative models for simultaneous design of ID sequences and 3D structures of peptides, which could inspire future research this field.
Software and Data
The curated PepBench and ProtFrag are available at The codes for our PepGLAAD are open-sourced at
Impact Statements
This paper aims to advance the field of peptide design through the construction of a benchmark and the development of novel latent diffusion model, PepGL.AD, which addresses key limitations in current methods. Our work represents step forward in computational peptide design, with the potential to impact both scientific research and practical applications in various domains. For instance, more precise peptide design could lead to enhanced drugs in the pharmaceutical industry, and could facilitate the creation new biomaterials, sensors, and other innovative technologies in biology and materials science. We wish paper could inspire future research in this field.
Acknowledgments
This work is jointly supported by the National Key R&D Program of China NN..2022/D01605000 the National Natural Science Foundation of China (No. 61925601, No. 62376276, No. 62276152), and Beijing Nova Program (20230484278).
10

---

References
[1] J. Adolf-Bryfogle, O. Kalyuzhniy, M. Kubitz, B. D. Weitzner, X. Hu, Y. Adachi, W. R. Schief, and R. L.. Dunbrack Jr. Rosettaantioodydssinn (rabd): A general framework for computational antibody design. PLoS computational biology, 14(4):e10061 12, 2018.
[2] R. F. Alford, A. Leaver-Fay, J. R. Jeliazkov, M. J. O'Meara, F. P. DiMaio, H. Park, M. V. Shapovalov, P. D. Renfrew, K. Mulligan, K. Kappel, et al. The rosetta all-atom energy function for macromolecular modeling and design. Journal of chemical theory and computation, 13(6):3031-3044) 2017.
[3] N. Anand and T. Achim. Protein structure and sequence generation with equivariant denoising diffusion probabilistic models. arXiv preprint arXiv:2205. 2022.
[4] S. Basu and B. Wallner. Dockq: a quality measure for protein-protein docking models. PloS one, 11())ee061879, 2016.
[5] H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov, and P. E. Bourne. The protein data bank. Nucleic acids research, 28(1):235-242, 2000.
[6] G. Bhardwaj, V. K. Mulligan, C. D. Bahl, J. M. Gilmore, P. J. Harvey, O. Cheneval, G. W. Buchko, S. V. Pulavarti, Q. Kaas, A. Eletsky, et al. Accurate de novo design of hyperstable constrained peptides. Nature, 538(7625)3329-334 2016.
[7] J. Bose, T. Aahhonn--Sadggh K. FATRAS, G. Huguet, J. Rector-Brooks, C.-H. Liu, A. C. Nica, M. Korablyov, M. M. Bronstein, and A. Tong. Se (3)-stochastic flow matching for protein backbone generation. In The Twelfth International Conference on Learning Representations, 2023.
[8] P. Bryant and A. Elofsson. Evobind: in silico directed evolution of peptide binders with alphafold. bioRxiv, pages 2022-07, 2022.
[9] L. Cao, B. Coventry, I. Goreshnik, B. Huang. W. Sheffler, J. S. Park, K. M. Jude, I. Marković, R. U. Kadam, H. Verschueren, al. Design of protein-binding proteins from the target structure alone. Nature, 605(7910):551-566, 2022.
[10] C. Chothia and J. Janin. Principles of 1975.
recognition. Nature, 256(5520):705-708
[11] P. Cock, T. Antao, J. T. Chang, B. A. Chapman, C. Cox, A. Dalke, I. Friedberg. T. Hamelryck. F. Kauff, B. Wilczynski, et al. Biopython: freely available python tools for computational molecular biology and bioinformatics. Bioinformatics, 25(11):1422, 2009.
[12] H. Cramér. Mathematical methods of statistics, volume 26. Princeton university press, 1999.
[13] P. Das, T. Sercu, K. Wadhawan, I. Padhi, S. Gehrmann, F. Cipcigan, V. Chenthamarakshan. H. Strobelt, C. Dos Santos, P.-Y. Chen, al. Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations. Nature Biomedical Engineering, 5(6): 613-623, 2021.
[14] J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte, L. F. Milles, B. I. Wicky, A. Courbet, R. de Haas, N. Bethel, et al. Robust deep learning-based protein sequence design using proteinmpnn. Science, 378(6615):49-56, 2022.
[15] P. Dhariwal and A. Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8777 8794, 2021.
[16] A. Duval, S. V. Mathis, C. K. Joshi, V. Schmidt, S. Miret, F. D. Malliaros, T. Cohen, P. Lio, Y. Bengio, and M. Bronstein. A hitchhiker's guide to geometric gnns for 3d atomic systems. arXiv preprint arXiv:23 2023.
[17] K. Fosgerau and Hoffmann. Peptide therapeutics: current status and future directions. Drug discovery today, 20(1): 2015.

---

[18] C. Fu, K. Yan, L. Wang, W. Y. Au, M. C. McThrow, T. Komikado, K. Maruhashi, K. Uchino, X. Qian, and S. Ji. A latent diffusion model for protein structure generation. In Learning on Graphs Conference, pages 29-1. PMIR, 2024.
[19] G. H. Golub and C. F. Van Loan. Matrix computations. JHU press, 2013.
[20] C. Grathwohl and K. Wüthrich. The x-pro peptide bond as nmr probe for conformational studies of flexible linear peptides. Biopolymers: Original Research on Biomolecules, 15(10): 2025-2041, 1976.
[21] Guan, W. W. Qian, X. Peng, Y. Su, J. Peng, and J. Ma. 3d equivariant diffusion for target- aware molecule generation and affinity prediction. In The Eleventh International Conference on Learning Representations, 2022.
[22] K. Guruprasad, B. B. Reddy, and M. W. Pandit. Correlation between stability of a protein and its dipeptide composition: novel approach for predicting in vivo stability of a protein from its primary sequence. Protein Engineering, Design and Selection, 4(2):155-161, 1990.
[23] J. Han, Y. Rong, T. Xu, and W. Huang, Geometrically equivariant graph neural networks: A survey, arXiv preprint arXiv:2202.07250, 2022.
[24] Henikoff and J. G. Henikoff. Amino acid substitution matrices from protein blocks. Proceed- ings of the National Academy of Sciences, 89(22): 10915-10919, 1992.
[25] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
[26] P. Hosseinzadeh, P. R. Watson, T. W. Craven, X. Li, S. Rettie, F. Pardo-Avila, A. K. Bera, V. K. Mulligan, P. Lu, A. S. Ford, al. Anchor extension: structure-guided approach design eyclic peptides targeting enzyme active sites. Nature Communications, 12(1):3384, 2021.
[27] J. Ingraham, V. Garg, R. Barzilay, and T. Jaakkola. Generative models for graph-based protein design. Advances in neural information processing systems, 32, 2019.
[28] J. B. Ingraham, M. Baranov, Z. Costello, K. W. Barber, W. Wang, A. Ismail, V. Frappier, D. M. Lord, C. Ng-Thoww-ing.. E. R. Van Vlack, al. Illuminating protein space with programmable generative model. Nature, pages 1 1-9, 2023.
[29] W. Jin, J. Wohlwend, R. Barzilay, and T. Jaakkola. Iterative refinement graph neural network for antibody sequence-structure co-design. arXiv preprint arXiv:2110.04624 2021.
[30] W. Jin, R. Barzilay, and T. Jaakkola. Antibody-antigen docking and design via hierarchical equivariant refinement. arXiv preprint arXiv:2207.06116. 2022.
[31] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Židek, A. Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-588) 2021.
[32] D. P. Kingma and M. Welling, Auto-encoding variational bayes. arXiv preprint 131.:113, 2013.
[33] X. Kong, W. Huang. and Y. Liu. Conditional antibody design as 3d equivariant graph translation. arXiv preprint (XXiv:220800077. 2022.
[34] X. Kong, W. Huang, and Y. Liu. End-to-end full-atom antibody design. arXiv preprint arXiv:2302.00203, 2023.
[35] A. C.-L.. Lee, J. I.. Harris, K. K. Khanna, and J.-H. Hong. A comprehensive review on current advances in peptide drug development and design. International journal of malecular sciences, 20(10):2383, 2019.
[36] Y. Lei, S. Li, Z. Liu, F. Wan, T. Tian, S. Li, D. Zhao, and J. Zeng. A deep-learning framework for multi-level peptide-protein interaction prediction. Nature communications, 12(1 (1))55465, 2021.
12

---

[37] J. K. Leman, B. D. Weitzner, S. M. Lewis, J. Adolf-Bryfogle N. Alam, R. F. Alford, M. Apra- hamian, D. Baker, K. A. Barlow, P. Barth, et al. Macromolecular modeling and design rosetta: recent methods and frameworks. Nature methods, 17(7):665-688 680, 2020.
[38] J. Li, C. Cheng, Z. Wu, R. Guo, S. Luo, Z. Ren, J. Peng, and J. Ma. Full-atom peptide design based on multi-modal flow matching. In Forty-first International Conference on Machine Learning, 2024.
[39] H. Lin, Y. Huang, O. Zhang, Y. Liu, L. Wu, S. Li, Z. Chen, and S. Z. Li. Functional-grouw based difffusion for pocket-specific molecule generation and elaboration. Advances in Neural Information Processing Systems, 36, 2024.
[40] H. Lin, O. Zhang, H. Zhao, D. Jiang, L.. Wu, Z. Liu, Y. Huang, and S. 7. Li. Ppflow: Target- aware peptide design with torsional flow matching. In Forty-first International Conference on Machine Learning, 2024.
[41] N. London, D. Movshovitz-Attia, and O. Schueler-Furman. The structural basis of peptide- protein binding strategies. Structure, 18(2):188-19)) 2010.
[42] N. London, B. Raveh, E. Cohen, G. Fathi, and O. Schueler-Furmant Rosetta flexpepdock web server-higg resolution modeling of ceptide-proteni interactions. Nucleic acids research, 39 (suppl_2)) 222-:)9:299W)))) -253,, 2011.
[43] S. Luo, J. Guan, J. Ma, and J. Peng. A 3d generative model for structure-based drug design. Advances in Neural Information Processing Systems, 34:6229-6233 6239, 2021.
[44] Luo, Su, X. Peng, S. Wang, J. Peng, and J. Ma. Antigen-specific antibody design and optimization with diffusion-based generative models for protein structures. Advances in Neural Information Processing Systems, 0559774--767,. 2022.
[45] K. Martinkus, J. Ludwiezak, K. Cho, W.-C. Lian, J. Lafrance- Vanasse, I. Hotzel, A. Raj- pal, Y. Wu, R. Bonneau, V. Gligorijevic, et al. Abdiffuser: Full-atom generation of in-vitro functioning antibodies. arXiv preprint arXiv:2308. 05027, 2023.
[46] M. Martins, L. H. Santos, D. Mariano, C. Queiroz, L. L. Bastos, I. S. Gomes, H. Fischer, R. E. Rocha, S. A. Silveira, L. H. de Lima, et al. Propedia: a database for protein- peptide identification based on a hybrid clustering algorithm. BMC bioinformatics, 22: 1-20- 2021.
[47] S. Mitternacht. Freesasa: An open source e library for solvent accessible surface area calcula- tions. F1000Research, 5, 2016.
[48] A. T. Muller, J. A. Hiss, and G. Schneider. Recurrent neural network model for constructive peptide design. Journal of chemical information and modeling, 58(2):472-479, 2018.
[49] M. Muttenthaler, G. F. King, D. J. Adams, and P. F. Alewood. Trends in peptide drug discovery. Nature reviews Drug discovery, 20(4):309-325, 2021.
[50] S. B. Needleman and C. D. Wunsch. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of molecular hiology, 48(3):443 4433, 1970.
[51] A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In Interna- tional Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[52] X. Peng, S. Luo, J. Guan, Q Xie, J. Peng. and J. Ma. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. In International Conference on Machine Learning, pages 17644-1700 17655. PMLR, 2022.
[53] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[54] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International conference on machine learning, pages 2256-2265 PMLR, 2015.
13

---

[55] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances neural information processing systems, 32, 2019.
[56] M. Steinegger and Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology. 35(1 1:1662-1028, 2017.
[57] S. Swanson, V. Sivaraman, G. Grigoryan, and A. E. Keating. Tertiary motifs building blocks for the design of protein-binding peptides. Protein Science, (1()):4332,, 2022.
[58] B. L. Trippe, J. Yim, D. Tischer, T. Broderick, D. Baker, R. Barzilay, and T. Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. arXiv preprint arXiv: 041111106001,, 2022.
[59] T. Tsaban, J. K. Varga, O. Avraham, Z. Ben-Aharon, A. Khramushin, and O. Schhelee-Furmaat Harnessing protein folding neural networks for ceptide-proteine docking. Nature communica- lions, 13(1): 76,, 2022.
[60] P. Vanhee, A. M. van der Sloot, E. Verschueren, L. Serrano, F. Rousseau, and J. Schymkowitz Computational design of peptide ligands. Trends in biotechnology. 29(5):231-239 2011.
[61] Y. Verma, M. Heinonen, and V. Garg. Abode: Ab initio antibody design using conjoined odes. arXiv preprini arXiv:2306. 01005,0005000 2023.
[62] Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and Bottou. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(1)) 2010.
[63] D. Wang, Z. Wen, F. Ye, H. Zhou, and L.. Li. Accelerating antimicrobial peptide discovery with latent sequence-structure model. arXiv preprint arXiv:22/2.09455) 2022.
[64] J. L. Watson, D. Juergens, N. R. Bennett, B. L. Trippe, J. Yim, H. E. Eisenach, W. Ahern, A. J. Borst, J. Ragotte, L. F. Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature, 62097976): 100, 2023.
[65] Z. Wen, J. He, H. Tao, and S.-Y. Huang. Pepbdb: comprehensive structural database of biological peptide-protein interactions. Bioinformatics, 35(1) 2019.
[66] K. E. Wu, K. K. Yang, R. van den Berg, S. Alamdari, J. Y. Zou, A. X. Lu, and A. P. Amini. Protein structure generation via folding diffusion. Nature communications, 15(1):1059 2024.
[67] H. Xia, J. McMichael, M. Becker-Hapak, O. C. Onyeador, R. Buchli, E. McClain, P. Pence, S. Supabphol, M. M. Richters, A. Basu, et al. Computational prediction of mhe anchor locations guides neoantigen identification and prioritization. Science immunology, 8(82):eabg2200, 2023.
[68] X. Xie, P. A. Valiente, and M. Kim. Helixgan a deep-lcarning methodology for conditional de novo design of o-helix structures. Bioinformatics, (9(1 )bbaad033 2023.
[69] X. Xie, P. A. Valiente, J. Kim, and P. M. Kim. Helixdiff, score-based diffusion model for 2024.
generating all-atom a-helical structures. ACS Central Science,
[70] M. Xu, L. Yu, Y. Song, C. Shi, S. Ermon, and J. Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.0292. 2022.
[71] M. Xu, A. S. Powers, R. O. Dror, Ermon, and J. Leskovec. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning, pages 38592-38610. PMLR, 2023.
[72] J. Yim, B. L. Trippe, V. De Bortoli, E. Mathieu, A. Doucet, R. Barzilay, and T. Jaakkola. Se (3) diffusion model with application to protein backbone generation. arXiv preprint a:Xiv:2302.00227,, 2023.
[73] Y. Zhang, Z. Zhang, B. Zhong, S. Misra, and J. Tang. Diffspack: A torsional diffusion model for autoregressive protein side-chain packing, arXiv preprint arXiv:2306.0m 01794, 2023.
14

---

A Reconstruction of Full-Atom Geometry
A.1 Auxilary Loss for Training the AutoEncoder
To better recover the all-atom geometry, we employ an auxilary structural loss similar to the violation loss Jumper et [D including the supervision on C, coordinates, bond lengths, and side-chain dihedral angles. First, since the C, is critical in deciding the global geometry of the peptide, we exert additional loss on its coordinates to distinguish from other atoms:
CCa
MSE(T,, 7
(16)
where Fi and F; are the reconstructed and the ground truth coordinates of C, in node i. Next, implement L1 loss on the bond lengths:
Lhond = = 16- 11111//(()))
(17)
where ((i) includes all chemical bonds in node i, and D' denotes the reconstructed bond length. For simplicity, he bonds between residues are included into the bonds of the former residue. Finally, we supervise on the X1 X4 side-chain dihedral angles [73]:
Langle (1)
-xEEAOO IX XI/IA(i)),
Σ
(18)
where A(i) includes all side-chain dihedral angles in node i, x' and x denotes the reconstructed and the ground truth angles, respectively. The overall auxilary loss for node is then given by:
Lawr(t) = AcaCca(0) (1) + Arond + Aangie Cungte (1),
(19)
where we set Àca = 1.0, About = 1.0, Aungle = 0.5 in our experiments. We find that it is necessary to Aongle with a relatively small value to make the training process stable.
A.2 Idealization of Local Geometry
Preserving atom instances enhances the modeling of side chain interactions but introduces challenges due to potential twisting local geometry. While in sequence-structure co-design, samples undergo fast relax by physical force field to ensure valid local geometry, in binding conformation generation, we use an alignment technique to place the idealized side chains on the generated atom instances. Specifically, the idealized side chain can be represented as at most 4 dihedral angles (x-angles), treating fragments like phenyl group as rigid bodies. Suppose there are n, x-angles and o; atoms in node i, we can define a function to map from the x-angles with the backbone coordinates B, E R4x3 to atom instances M; T optimize X- xaaggl via gradient descent to minimize the MSE between the coordinates constructed from X-angles and those generated by the model:
- xR which is luckily differentiable (28). Thus can
X arg xi02 |M B.) X21
(20)
where X; E x denotes the coordinates of & atom instances in node i generated by the model. Now it should be easy to construct idealized side chain maintaining fidelity to the generated atom instances by M (x B.). In the experiments of conformation generation on PepBench, such idealization gives slightly better DockQ and RMSD stom than Rosetta side-chain packing algorithm but with higher efficiency.
B Proof of Proposition 3.1
Proposition 3.1. Denote the invariant and equivariant outputs from acalarization-baseel E(3)- equivariant GNN as f( R.3) and F(th,, F.)), respectively. With the definition of F in Eq. 7
Vg E E(3), we have F(E))) = f((hi, Fglg Z.} and g (f(b,,, F()))))
[fI(,((,,, Fglg dz)1i,, where Fg is derived on the coordinates transformed by g. Namely, the E -eqivvariee preserved if we implement the GNN on the standard space and recover the original geometry from the outputs.
15

---

For simplicity, given F derived from set of coordinates } following Eq. we use Fg to indicate the affine transformation derived from وg æ.). where g E E33. Additionally, we keep the terminology "standard space" for describing the space after the data-specific affine transformation F. We begin by proving key lemma, that is, the F(3)-invariance of distances between two nodes in the standard space converted by F:
Lemma C.I. Given two nodes and j in the geometric graph 9, denoting their coordinates as T. and J. their distance in the standard space is ((3)-iivariant. Namely, Vg E F(3), ((.)) - F( F z.) Fg Tz)1l-
Proof. Vg E E(3), 9 can be instantiated as an orthogonal matrix Q O(3) (including rotation and reflection), and a translation vector E' E TR Denoting all coordinates in the geometric graph as X E |331191, and the number of nodes in G as n, we can derive the B(3)-qquivariance of the expectation of the coordinates:
= 1 Σ 71
C. - 1 Σ QT, +8) 11
Elg-
X
g
(21)
1 n + +1= Q +E
(22)
g E[X]
(23)
With the -qu-vqquiveeiance of the expectation, it is easy derive the following equation the covariance matrix:
Covlg X,g- x)
11 (g- X Elg- X) X - XI)
(24)
π 1- [g- g- B[X X X -
g-
(25)
n [Q - E| Q - -(XII)
(26)
n Q( x E QQ
(27)
QCov T)QT.
(28)
Based on the Cholesky decomposition used in the derivation of F, we denote Cov X,, x) x) LT and Cov(g X,g x) L, with which we can immediately derive:
E,t = QLÉTQT
(29)
Considering Q-1 = QT, we further have the following equation:
9 Q'-
(30)
Given that F(Ø) = E- - (ā - R x) ), we are now ready to prove Lemma C.1 as follows:
[gyg z.) Fglg -,)1l
||ا (g ह. Elg x L (g ă, Blg x)II
(31)
|| | - g. 9-,, = Q(E. -1)1
(32)
V Q(. - -,)) (1, Q(,
(33)
V - T L -
(34)
V (0i- - -i--a) - - (6
(35)
= (
(36)
10--E(X8 E| = - F(
(37)
16

---

With Lemma C.1
we are able to give the proof of Proposition 3.1 as follows.
Proof. A first observation is that to prove Proposition 3.1 we only need to prove the equivariance in the 1-layer case, since the multi-layer case can be decomposed into the 1-layer case by inserting I= F F-1 between layers, where / is the identical mapping. Generally, each layer in scalarization- based E3))eqquivaiiant GNN has the following paradigm:
mij Φm (h,, h,,
[ - -B,11 exy):
(38)
x²= .. +
Σ JEN(()) (T, - 20)
(39)
h Φn Φn(h,, EjEN()) maz),
(40)
where ((i)) denotes the neighborhood of node i, and o2 outputs scalar. Therefore, for the invariant part, we have:
filth Fglg- (llo-$:1. t.)1) = (nri Σ JEN()) mij.Fg)
(41)
on (h;, JEN())
EVENC) pm (h, (, Fg ..) - (g 2,)11
(42)
dn Σ JEN(() bn F(z))Il,
(43)
= (bn DjEN() mij,F)
(44)
= Si(th, F(E.)))
(45)
For the equivariant features, recalling (+) = Lt + ] we have:
FØ Fi(h, Fgl. #.)1)
(46)
= (F z.)= + Σ SEN())
SEENTO) (Folg z.) - Fglg
(47)
(F 4.) + Σ SEN(()
[ Q(z
(48)
F (1, (Q + E EEg- xI) + Σ jEN(()
_ ã,, 02 (m...))
(49)
= Fj (I, (Qa, 1 g. E[X) + Σ jEN(() _ ā, (m.s.)))
(50)
FF ( Q Σ jEN()) g 1 ,,)0, (mis,))
(51)
= F- (I, Q(ū, E{X) + Σ SENN() E Q() 2 (m....))))
(52)
= F- ( Q(E. - E[X]+ + E jEN(1) (Ø)
- (
) (mi..)))
(53)
Q E Σ (EEN()) ( - F. 2o (mij.r) )) + Elgg x
(54)
Q + Σ SEN( (T. -2)00 (m.. QE + g E|}]
(55)
Q(ă, EjeNoo (#: (mij.F)) + i
(56)
= g (m + EjeNro (3.. - 3. 3,)0
(57)
By replacing g with identical element E(3), since F= = Fr, we can immediately derive:
F F((((h,, = (æ. + Σ JjENTO (σ. - &
(58)
g. F A((((,, = F Fi(he, Fg(0--990000 -x,1),
(59)
which concludes Proposition 3.1
17

---

D Algorithm for Training and Sampling
We present the pseudo codes for training in Algorithm 1 amd sampling in Algorithm
Algorithm Training Algorithm of PepGLAD
input geometric data of protein-peptide complexes S output encoder &g. decoder Dg denoising network 60
1: function Train Autninnnuon)co)e((())
2: Initialize Ed. Dg
3: while o have not converged do
4: Sample (gp, 9b) ~
5:
Gp + mmaak((p))
[Mask 25% Residues
6:
(H, üú; o.) +- Eo (G 9b)
{Encoding]
7:
[(e,, (0,
[Reparameterization]
8:
GR + (lu+ oi fh E O }
9:
S + Dg(9s,90)
Decoding]
10:
LAB = =A==Lieg, (Creom (i) + (k)(
11:
0, t optimizer( Cas: e)
end while
13: return Egg De
end function
15:
16:
function TrainLatenDiifuusionCC, De, S)
17:
Initialize Es
18:
while ! have not converged do
19:
Sample GG,, 96) ~ S
20:
[ o o. o.) + Eg(9p.
(Encoding)
21:
-o (pp. [7.}
22:
F t )
(Affine Transformation) )
23:
go, GO + F(G)), F(G))
(Standard Geometry) )
24:
[(= 2 (()1)))) - Go
25:
U(1,T), {(e,, {.) (0,,1)
26:
S - (vo 2 20
+ +11 (1-0") +1
27:
LiOM = E, [| cl
28:
0 - optimizer ((iimiiiee((iii)))) ()
end while
30: return Eg
end function
32:
Eg, Dg + TrainAutoEEnoder((()
34: Fix parameters ø and &
35: 6g Trainl Lateantiiffuuiionne Dg,
36: return Eg, De,
E
Data Preparation
We show details for constructing the datasets used in our paper here. Further statistics are presented in Table and distribution of peptide lengths in Figure
E.1 Unsupervised Data from Protein Fragments (ProtFrag)
We exploit unsupervised data from monomer proteins to enrich the training of the autoencoder. Specifically, we first extract all single chains from the Protein Data Bank (PDB) before December 8th, 2023, and remove the duplicated chains on a sequence identity threshold of over 90%. Then, for each chain, extract fragments satisfying the following criteria:
18

---

Algorithm 2 Sampling Algorithm of PepGLAD
input decoder Dg, denoising network cø, binding site G
output peptide Op
I: F affinc 9b)
(Affine Transformation) )
2: G. + F(G.)
(Standard Geometry]
3: {( x ) O,,
4: for Lo T.T-1 1..., do
do
5:
G x (= )
LLatent Denoising Loop)
6: u; + 2
7:
E,
E, E, (0,1)
8:
U t
Eo 90,
9b, t)]] + 8,e,
9:
2 1 2
u
10: end for
11: Go + {(+ )
Gg F (g),, (G))
(Data Geometry [Decoding] )
13: Gp 4 (g,, 9b)
return 9p
Length: The fragment should consist to 25 residues.
2. Balanced constitution: No single amino acid should constitute more than 25% of the fragment; Hydrophobic amino acids should comprise less than 45% of the fragment, with charged amino acids accounting for 25% to 45%.
3. Isolated Stability: Instability [22 should be below 40; Considering the surrounding amino acids interaction partners, the fragment should have buried surface area (BSA) above 400A*
10 with relative BSA above 20%.
We use FreeSASA to calculate the surface area of fragments. Let SAoand represent surface area of the isolated fragment when considering surrounding amino acids, and SAwnhound represent the surface area when not considering them. The buried surface area then calculated as BSA SAunbownd SAbownd, and the relative BSA is calculated BSA,e! BSA/SAamtoon In total, obtain 70,645 fragments meeting these criteria.
E.2 Construction of Our PepBench
Here illustrate the details for constructing the supervised dataset (PepBench) used in our paper. Similar to literature [65]. we also exploits available data from the Protein Data Bank (PDB) [5]. We first extract all dimers in PDB deposited before December 8th, 2023, and filter the complexes with a receptor longer than 30 residues and a ligand between 4 to 25 residues, which aligns with Tsaban al. [59] Peptides with lengths this range are more relevant to practical applications such drug discovery, as they exhibit favorable biochemical properties 44). Then we remove the duplicated complexes with the criterion that both the receptor and the peptide has a sequence identity over 90% [56]. after which 6105 non-redundant complexes are obtained. We use MMseqs2 for clustering based on sequence identity:
# create database from the sequences unseqs create seqe.fasta database # clustering with sequence identity abore 90%
unseqs cluster database detabase_cluetere results --min-seq-id 0.9 -o 0.95 --cov-node
To achieve the cross-target generalization test, we utilize the large non-redundant dataset (LNR) introduced by Tsaban et al. [59] as the test set, which is curated by domain experts. LNR originally includes 96 protein-peptine non-canonical amino acids. We then cluster the LNR along with the PDB data by receptor with sequence identity threshold of over 40%. Subsequently, we remove the complexes sharing the same clusters with those from the test set and those including non-canonical amino acids in the peptides. Finally, the remaining data are randomly split based on clustering results into training and validation sets. The characteristics of different splits presented in Table 5 In addition, the binding site
complexes. We obtain 93 complexes after excluding the ones with
19

---

contains residues on the receptor within 10A distances to the peptide, where the distance between two residues measured by the distance between their C coordinates.
E.3 Split of PepBDB
Similar to the split of PepBench, we use MMseqs2 for clustering and randomly split the data into training, validation, and test sets based on the clustering results. For the test set, we randomly select one protein-peptide complex in each cluster to avoid redundancy.
Table 5: Statistics of the constructed datasets.
Split
#entry
#cluster
source
PepBench (Training)
4,157
952
PDB [5]
PepBench (Validation)
114
50
PDB
PepBench (Test)
93
93
LNR [59]
PepBDB (Training)
8,434
1,617
PepBDB 65
PepBDB (Validation)
370
95
PepBDB 65
PcpBDB (Test)
190
190
PcpBDB
65
ProtFrag
70,645
monomers in PDB
800
8000
1000
800
Count
600
Count
400
4000
400
2000
10
15 length
25
5
10
155
20
10
30
40
50
length
length
(A] PepBench
(8) ProtFrag
[C] PepBDB
Figure 5: Peptide (or protein fragment) length distribution of three datasets.
I(3)-Invariant Latent Space
We found that the {(3)-equivariant latent vectors were significant, which can convey sufficient geometric information. If we use E(3)-invariant latent space without these geometric latent vectors, the reconstruction ability (RMSD, DockQ) of the VAE deteriorates significantly (Table (6). It is reasonable since E(3)-invariant latent space lacks geometrie interactions with the pocket atoms, leading to difficulties in reconstructing the full-atom structures on the binding site.
Table 6: Comparison of reconstruction ability of variational autoencoders with E(3)-equivariant latent space and E(3)-invariant latent space.
Latent Space
AART
RMSD +
DockQ1
E(3) eqqiivvaav
95.1%
0.79
0.898
E(3)-invarian)
93.4%
1.75
0.823
G
Guidance on Sequence Orders for Sampling
Peptides consist of linearly connected amino acids, which exert constraints on the 3D geometry. Specifically, residues adjacent in the sequence should also be close
structure, since they are connected by peptide bond. However, 3D graphs are unordered and do not incorporate such induct bias, which means the generated nodes might have arbitrary permutation on sequence orders. To tackle this problem, we take inspiration from classifier-uuided diffusion [15), which adds the gradient
20

---

of classifier to the denoising outputs to guide the generative diffusion process towards the subspace where the classifier gives high confidence. We utilize an empirical classifier ((, defined as follows:
p(11(=])) exp(- - E P()))-()== E(1
(60)
E(d) - -{
d (Mu + 30a), >> pd + 30 (14 30a) - d. < ptd - 30
(61)
0, otherwise,
where μl and da are the mean and variance of the distances of adjacent residues in the latent space measured from the training set. With p( p(1[{z]), we are able to assign an arbitrary permutation on sequence orders to the nodes, and steer the sampling procedure towards the desired subspace conforming to P, since this classifier gives higher confidence if the adjacent (defined by P) residues within reasonable distances aligning with the statistics from the training set. In particular, the coordinate denoising outputs are refined as follows:
E Eo Go, 1) AVI V (o1
(62)
where À adjusts the weight of the guidance. Besides the constraints on the distance between adjacent residues, we can also include guidance avoiding clashes between non-adjacent residues by defining the following energy term:
C(d) =
1 {f Pd -d d. d << pdd
(63)
0, otherwise,
Subsequently, we just need to revise the empirical classifier as:
)(1[{=])) = exp(- - Σ
E( 1)
Σ P(O
C(I
Σ
- -gll),
P()-
JEC
(64)
where Fj is the C. coordinate of node j in the binding site. We observe slight improvement upon including the clash energy term. Nevertheless, the guidance is only small technical trick with minor enhancement on the performance as shown in Table
Table 7: Results on binding conformation generation with and without guidance on sequence orders.
Guidance RMSDc, + RMSD J
DockQ1
w/
4.09
5,30
0.592
w/o
4.10
5.34
0.582
H Metrics for Evaluating Sequence-Structure Co-Design
H.1
Why not Amino Acid Recovery (AAR)
6.0
0.6
2.50
0.4
045
1.22
620
025
015
315
6:2
645
230
canddata poole
Figure 6: Best amino acid recovery (AAR) on samples constructed according to specified positive ratios. Each point contains the averaged result across 102 receptors.
21

---

we find it not informative enough for evaluating generative models on learning the distribution of binding peptides. due to the vast and highly diverse solution space. To elucidate this, we conducted an analysis of AAR using a sequence dataset from Xia al. [67 including 328 receptors and 600k peptides with binary binding labels, from which we filter out 102 receptors with well-explored solution space (i.e. with at least 500 binders). For each receptor, we randomly select one binder as the reference and sample N candidates according to a specified positive ratio r, resembling the scenario of evaluating generative models for peptide design. For example, setting r = 0.1 and N 100 involves sampling 10 binders and 90 non-binders to construct the candidates. Then compute the best AAR of these candidates with respect to the reference as the result. This process is repeated for 10 times per receptor, and the results are averaged across different receptors, which can be interpreted as the evaluation score of AAR on a generative model that can generates candidates with a positive ratio of We select N 10, 40, 100 and enumerate the choices of derive the relation plot in Figure 6 where the results of a random sequence generator is also included for comparison. It can be derived that the gap of AAR between the worst model (r = 0.0) and the best model (r 1.0) is insignificant. Furthermore, all models, including the best model (r = 1.0) which always produces positive samples, exhibit performance akin to the random sequence generator, with consistent trends regardless of the choice of N. We attribute this to the vast solution space of peptide design, where evaluating with dozens of candidates relative AAR is improbable since the model would need to fortuitously explore the subspace around the reference, which is arbitrary, on every test receptor; Conversely, low AAR does not necessarily denote poor generative model, as the model may be exploring distinct solutions from the single reference. Moreover, we calculate the Spearman correlation between the receptor-level best AAR and the positive ratio of the candidates, yielding 0.23, 0.22 and 0.24 for N = 10, 40, and 100, respectively, indicating very weak correlation. Based on this analysis, we assert that AAR is unsuitable for evaluating target-specific peptide design. The comparison of AAR and success rates on PepBench (Table 8 also indicates that models with similar AAR can have distinct success rates.
While the amino acid recovery (AAR) is widely used in antibody design 33 34 44
single reference is unreliable. In other words, achieving high
Table 8: Amino acid recovery and success rates of different models on PepBench.
Model
AAR
Success (AG 0)
HSRN
35.8%
10.46%
DiffAb
37.1%
49.87%
PepGLAD
36.7%
55.97%
H.2 "Unsupervised" Consistency
While it is common to evaluate consistency by comparing generated structures with AF2-predicted structures 77 such a "supervision-based" method suffers from severe limitations in the situation where AF2 33 fails to achieve acceptable performance. As highlighted not only in Tsaban et al. (59 but also demonstrated by our experiments, even state-of-the-ar models like AF2 struggles to consistently produce high-quality structures of protein- pppteide complexes. Our findings reveal that only 36% of test samples could be accurately predicted within 5Á RMSD (considered near-native conformation) by AF2, indicating even the ground truth can only achieve 36% success rates if we use such supervison-based consistency for evaluation, making such evaluation not reliable in the domain of peptide design.
In light of this limitation, we propose an "unspervised" evaluation framework to assess consistency, that is, the statistical association between the clustering results of sequences and structures. Theoreti- cally, this serves the necessary condition for true consistency. Namely, if model truly captures the consistency between sequence and structure, it will necessarily achieve a high score on the proposed metric. Conversely, if model fails to attain a high score on the proposed metric, it is not possible to capture true consistency.
In our experiments, we found that our proposed consistency metric effectively distinguishes non- consistent modeling methods, such as HSRN, which tend to produce disparate sequences while sharing identical structures.
22

---

H.3 Implememtation and Elaboration on Metrics
Indeed, evaluating generative models comprehensively crucial, requiring assessment from multiple perspectives. Generally, these evaluations can be categorized into two main aspects: diversity and fidelity the desired distribution. Regarding diversity, we take inspiration from Yim al. [72] and quantify it with the number of unique clusters relative to the number of candidates. This metric provides insight into the variety and richness of the generated samples. For fidelity, the primary focus should be the binding affinity, for which we adopt the physical energy from Rosetta 2 since it is widely used in various domains and exhibit robust generalization capability [ 37 64
Further, considering the dependency between sequence and structure, propose the consistency metric, which critical for distinguishing whether the generative model is capturing the ID&3D joint distribution and thereby truly facilitating "co-design". We have also discussed the reason for implementing the consistency metric with the "unsupervised" fashion in the above section. Below, outline the implementation of these metrics.
Diversity We hierarchically cluster the sequences and the structures with aligning score [36 and RMSD of C., respectively. In particular, implement the aligning score using Biopython ||| with
BLOSUM62 matrix [24] and Needleman- Wunsch algorithm [50]. The thresholds for clustering is similarity above 0.6 and RMSD below 4.0 for sequence and structure, respectively. We provide the python codes below for clearer presentation:
import aqrt from typing import List
from Bio. Aliga import subetituriion Pairvisehligner
import numpy as xp
align_ (sequence_A, sequence_B):
# load natrix sub_matrice cebeeittution_attiiiee load (BLOUUUUUUX
aligner Pairwisedligner() siigner.sbattiuution align
aub_natrice
alas aligner aligaaggggggsuncec sequence_B)
beat_aln alns
aligaed_A, aligned_B beat_aln
normalize
base_A allgger..correeeeeeee..e.ee (sequeeecc,.,__ sequence_At
base_8 = allgner.scoraasaquee www.a sequence_8)
sqrtebase_A base_B)
cimilarity awt.aor.ccoressauuee...o.. (squuussu,,,,,,,, sequence_B) / base
return similarity
def deg_dtvrrattyyeege [ist(, th: fleat=0.4) fleat:
..3 sequence distance (1 similarity)
diats 0
for sequence_A enueerate
dists. aists.append far J. aequence_B enumerate dists appendd align_score (erreuueneeesseccc........ (sequence_A, sequence_B))
dists arr.arryddiitss z (annaggesquurrrfoonnnttttt 'single")
cluster foluster(Z, 1-th, Eetterion='ditanne len(sees)
def iiverrdddieesstttttteyee
ndarray, th: float-4.0) >> float:
structs: NoL+3, alpha carben coordinates 30
th: threshold for clustering (dietance th)
ca_diats = sum ((structs(:: [:, Nonel structs [Nane, :]) ** 2, axism-1) rea up. agrt (op. (ap. (ca_dista, axis=-1))
= linkage (squareform (rasd), 'single')
fcluster (2, t-th, criterion= return len (en(op. unique (uqqueeee (cluster)) structs. structs.shape [0]
Denoting the diversity of the sequences and the structures as Div and Divarner respectively, we
calculate the co-design diversity as / DDiv Div street
23

---

Consistency The clustering process in the calculation of diversity will assign each sequence and each structure with a clustering label. The labels on the sequences and those on the structures can be regarded as two nominal variables. Since similar sequences should produce similar structures, these two variables should highly correlated if the model really learns the joint distribution. Naturally, we quantify the consistency via Cramér's ' V [12] association between these two variables, with 1.0 indicating perfect association, and 0.0 means no association.
We use Rosetta 2 to calculate the binding energy with the "ref2015" score function. Both the candidates and the references first endure the fast relax protocol in Rosetta to correct atomic clashes at the interface before the computation of binding energy. We use the implementation in pyRosetta (i.e. the python version of Rosetta), which is borrowed from Luo et al.
AG
import pyrosetta
from pyrosetta.
import protocole
from syrosetta.rozeet susasta.pratocols.red.
import FastRelax
from pyroaetta. rorettt..ot..t.................. wore.pack.... pack. inport TaskFactory
fron Pyroaetta. coraetta.crre.pcc.... select import greidue_selecte
inport operation
pycosetta.rozetta. from rosesttt..roeetta.... wo...eelcc...................... novenap import HoveHapPactory, aove_map_action
from rroeett core. scoriigg import ScoreType
from pyrosetta. protocols analyyei import Interfacelmallyzetooer:
analysis
pyrosetta. init(" ([
P-mute', 'all', "-use_ieput_sc',
'false', '.Ioad_P08_compeoonte !false', '-relax:defatte .-rellaziataallt.rqaatt. '2',
"-mo_fconfig',
sene_ttraieel_resddeee 'true', sii::ffle::ieeettstttre 1))
--use_
'binary'
class (elaxiegionnebjeect Fast Ralax
def _ninii__(eell (self, nxx_iter-1000):
super ()..int)... self. pyrosetta... sreate_scora_ffnttee ('ref2015)) !)
self fast.fast_reee: FastRelax
selt salf.fast_relax wtw..rolbx..tt.ccorraatccccc (self. scorefxn)
self tta. (max_iter)
def _.call_. (self, pdb_path, (ppride_chain)
pose prrosatttappcce.ttopppppppp
TaskFactory()
hagaaaaaaaaaaaaaaaaaaaaaaaaaaaaa if. htw.aaewww.eeeet..eeeeeeeeteeeep RestricctToRspcciigg()) Only allov residues to repack.
(opperatto hnt.aailssaaaaoomannnnnn
design any position.
gen_selector selecttons.Chattsssele www.hhheeeeoor (chain)
abr_selector selections.
()
abr_eelestor.edt eetttrrreet... set__focus_ellctoor (gen_ www_include..oouua subset_ abr_selector
pravont_ropacksng.rik o ProventRRopackiaglL1()
prevent__unset..rppccct operation @perataodnReai.ducuubbettt..
preventt.rppccilgg..l.
subset_solector.
flip.subaes-True,
self folfffasttrreax. oorignaal_poe.. clone()
anf XoveMapFactoryy
naf add_bb_action wtt..pppaattioo..ooo.il.l.. an gen_selector)
add_chd chi_action (nove_map_actiee action.. action..n_enatl. en_enable, subset_selectee
ant ant. www.teeeeetaooeeeee www.rrreeevaaaapp..oo saaaaaaa__roo.. tro__porrooppeeeeeeeeeee_ (pose)
sr.set_novve get_novenappce (nn)
fr. set bet_task_ff wtt.ttaattoraaaaaaaaaataattt factory (tf)
apply (pose)

---

retura pose
(rroattt..trroc...nrrr//pppppppp receptor,chaing
cigand_chaina): binding
energy
pase interface j.ooi[i(iaaaadccchhhh
(..oi(rreeptorrcchatt)) (aaaaaaallyaatoerrrtttraaaa
nover Interfacaknalyyarkovrrr nover wtt.eeacc..epparatte.eee.e
nover.apply (pase)
return poae. scores ("c _e__raeee
Success Since negative AG value typically indicates a potential for binding, we report the ratio of all generated candidates that satisfy this threshold. Moreover, candidates with AG < 0 usually are least physically valid (i.e. without obvious atomic clash), thus it meaningful to this success rate to evaluation the generative ability of the models.
I Experiment Details
LI Implementation of PepGLAD
We train PepGLAD on GPU with 24G memory with AdamW optimizer. For the autoencoder, we train for 60 epochs with dynamic batches, ensuring that the total number of edges (proportional the square of the number of nodes) remains below 60,000. The initial learning rate is 10-4 and decays by 0.8 if the loss on the validation set has decreased for consecutive epochs. Regarding the diffusion model, we train for 500 epochs with the same batching strategy as for the autoencoder. The learning rate is 10 and decay by 0.6 if the loss has not decreased for consecutive validations, where the validation is conducted every 10 epochs. In the experiment of binding conformation generation, we only use the supervised dataset, thus we extend the training epochs for autoencoder and the diffusion model to 500 and 1000, respectively. Consequently, the patience of learning rate decay is extended to 15 epochs for training the autoencoder. We keep other settings unchanged. The hyperparameters of PepGLAD used in our experiments are provided in Table 9
Table Hyperparameters of PepGLAD in sequence-structuse codesign and binding conformation generation.
Value
Name
Description
codesign
conformation
Variational AutoEncoder
embed size hidden size
128
128
Size of embeddings for residue types.
128
128
Size of hidden layers.
h
8
Size of the latent variable for residue types in the sequences.
layers
Number of layers.
A1
0.1
0.0
The weight of KL. divergence on the sequence.
0.5
0.5
The weight of KI. divergence on the structure.
Aca
1.0
1.0
The weight of C, loss in L...
Abond
1.0
The weight of bond loss Lowx-
Aangle
0.5
0.5
The weight of side-chain dihedral angle loss in Lawe-
Latent Diffusion Model
hidden size
128
128
Size of hidden layers the denoising network.
layers
3
3
Number of layers.
steps
100
100
Number of diffusion steps.
1.2 Implementation of the Baselines
we directly integrate their official implementation into the same training framework as PepGLAD, and adjust the batch size, learning rate, and training epochs to obtain the optimal performance, which we present in Table TO
For HSRN (0), dyMEAN 34], and DiffAb 44
We outline the implementation of other baselines below:
25

---

Table 10: Hyperparamenters for training the baselines.
Model
Batch Size
Learning Rate
Epoch
HSRN
1.0c-4
50
dyMEAN
1.0e-4
100
DiffAb
32
1.0c-4
50
RFDiffusion 66 We follow the official instruction to randomly select 20% residues on the binding site as "hotspots" and generate the backbone via diffusion followed by cycles of inverse folding with ProteinMPNN [1] and full-atom structural refining with the officially provided Rosetta protocol.
AnchorExtension 26 It is implemented with the Rosetta suite, thus we resort to the official release of the pipeline protocols for docking and optimizing. We use the default parameters and generate 10 candidates for each receptor due to its limitation of efficiency. We randomly pick one peptide in the training set with the same length as the reference peptide as the initial motif for docking.
FlexPepDock Rosetta.
42 We follow its official tutorial to implement this baseline in the C++ version of
AlphaFold2 31 We borrow the results from (59), which explores two strategy to use AlphaFold2 on peptide conformation generation, including modeling the receptor and the peptide separate chains or link them together with long loops. The results contain a total of 10 candidates for each receptor, with from the separate strategy and from the linked strategy.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: $Band $ illustrate the claims well from the methodology and the empirical aspects.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have an independent section ($6) 6 discuss limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
27

---

Justification:
See Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: addition to the methodology section ($ details in Appendix
3). we provide implementation
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
28

---

Answer: [Yes]
Justification:
We provide codes and data for our model and the experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com / swiniiiiicypyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: See
4.1 Appendix E and Appendix [
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The size of the dataset is large, and report results different datasets for more solid evaluation.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
29

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://aarrr...cccppoaaacttttcta
Answer: [Yes]
Justification:
We ensure the research to conform to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See $
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
30

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: They properly credited.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
31

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The README.md in the provided codes illustrate how construct the benchmark from publicly available resources.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve such topic.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
32


Reviews:
[{"review_id": "GWq9Iwb6WU", "reviewer": "Reviewer_8deM", "summary": "The authors proposed PepGLAD, a latent diffusion model for full-atom peptide design. This work mainly addressed two challenges for peptide design, (1) full-atom modeling, and (2) diverse binding geometry. Specifically, a variational auto-encoder was trained to learn latent representations of protein-peptide interactions. Then, a latent diffusion model was employed to capture the underlying distribution to sample binding site-conditioned peptide conformations. The authors curated a benchmark dataset to evaluate the model performance, and compared with existing models to showcase its effectiveness.", "questions": "- L#130. For each residue, the initial coordinates are initialized with the same $\\vec{\\boldsymbol{z}}_i$. How significant are these initial vectors? Can we use an SE(3)-invariant latent space and a regression model to generate atomic coordinates (similar to AlphaFold StructureModule, conditioned on the sampled residue type)?\n- Classifier guidance seems ad hoc and brings more complication to modeling. Would an architecture similar to that used in [ESM3](https://www.biorxiv.org/content/10.1101/2024.07.01.600583v1) or [PVQD](https://www.biorxiv.org/content/10.1101/2023.11.18.567666v1) help?\n- Table 1: out of 40 samples generated by PepGLAD, there are about 20 different conformations (Div.) and 20 valid peptides (Success). How should the readers interpret this? Peptides exhibit high structural flexibility upon binding?\n- Did I miss sequence recovery metrics in the model benchmark section (excluding Consistency, which only models the joint distribution)?", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Full-atom modeling is significant to better represent protein-protein/peptide/ligand interactions. This work goes beyond backbone-only modeling and realized full-atom level generation.\n- Employing latent diffusion with a pre-trained VAE is a reasonable choice, which has recently become prevalent for protein generative modeling.\n- Provided a new benchmark dataset with careful curation. Proposed novel evaluation metrics and discussed limitations of existing metrics (e.g., AAR).\n- Under the proposed evaluation metrics, PepGLAD outperforms existing models.", "weaknesses": "- Dataset is small (6105 non-redundant complexes). Data augmentation attempts (ProtFrag) do not seem to help (Table 4 ablation results). Although sequence-based clustering has been utilized to prevent data leakage, training/testing VAE and diffusion model on 6105/93 data points make the results less convincing, *especially considering the enormous design space for sequence-structure co-design tasks*.\n- Guidance with an additional classifier (Sec 3.5) makes the model more complicated (which seems a bit ad hoc to me).\n- From the results shown in Table1, it seems that the model is able to generate a variety of conformations (high Div.) which are reasonable ($\\Delta G < 0$). Given the extremely limited size of dataset, I am not sure if the model can properly capture the *conformation space* at all. In other words, I am a bit skeptical about the proposed evaluation metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable efforts and comments!\n\n> W1. Dataset is small. Data augmentation attempts do not seem to help. Although sequence-based clustering has been utilized to prevent data leakage, training/testing VAE and diffusion model on 6105/93 data points make the results less convincing.\n\nThanks for your comments! Indeed, to ensure our results are convincing, we follow the literature [a] and employ the 93 complexes for testing, which are carefully curated by [a] to represent a diverse and high-quality landscape of complexes. Our model trained on a non-redundant dataset of 6K complexes is able to deliver promising results on these testing complexes. Regarding data augmentation that is only applied to the autoencoder, it has a notable impact on energy scores, without which the success rate drops from 55.97% to 52.15% in Table 4. \n\nIn addition, we also conducted experiments on a public dataset PepBDB [b], which is of larger size (10K samples and 190 testing complexes). These results, presented in Table 1 and Table 3, reinforce the validity and robustness of our conclusions.\n\n[a] \"Harnessing protein folding neural networks for peptide–protein docking.\" Nature communications 2022.\n\n[b] \"PepBDB: a comprehensive structural database of biological peptide–protein interactions.\" Bioinformatics 35.1 (2019)\n\n> W2. Guidance with an additional classifier makes the model more complicated.\n\nThanks! The classifier guidance is used to sample from a sequential subspace within unordered point clouds.  We understand that adding guidance somehow makes our model more complicated, but this component does further enhance the performance. To show this, we have additionally compared the structure prediction model with and without guidance in the table below. As its effect is relatively minor, we did not emphasize it as a major contribution to the paper. We will highlight this point in the revision. \n|guidance|$\\text{RMSD}_{C\\alpha}\\downarrow$|$\\text{RMSD}_\\text{atom}\\downarrow$|$\\text{DockQ}\\uparrow$|\n|-|-|-|-|\n|w/|4.09|5.30|0.592|\n|w/o|4.10|5.34|0.582|\n> W3. It seems that the model is able to generate a variety of conformations (high Div.) which are reasonable. Given the extremely limited size of dataset, I am not sure if the model can properly capture the conformation space at all. In other words, I am a bit skeptical about the proposed evaluation metrics.\n\nWe understand your concern. It is true that using the Diversity metric alone is hard for correct evaluation. Hence, we also use rosetta energy to evaluate the fidelity to check whether the diversity is hacked by random conformations. As highlighted in Table 1, our model surpass the baselines both on diversity and fidelity (rosetta energy), therefore the improvement in diversity should be meaningful as the model is properly capturing the desired distribution. These findings are further confirmed on a larger dataset, PepBDB. For more discussion on the metrics, we kindly refer to appendix G.3.\n\n> Q1: L#130. For each residue, the initial coordinates are initialized with the same. How significant are these initial vectors? Can we use an SE(3)-invariant latent space and a regression model to generate atomic coordinates?\n\nThanks! The initial vectors from the latent equivariant space are significant, which can convey sufficient geometric informaion. As suggested, we further conduct an experiment by discarding the equivariant latent vectors and using an SE(3)-invariant latent space instead. The table below shows that the reconstruction ability (RMSD, DockQ) of the VAE  deteriorates significantly. It is reasonable since SE(3)-invariant latent space lacks geometric interactions with the pocket atoms, leading to difficulties in reconstructing the full-atom structures on the binding site.\n|Equivariant vectors|AAR$\\uparrow$|RMSD$\\downarrow$|DockQ$\\uparrow$|\n|-|-|-|-|\n|w/|95.1%|0.79|0.898|\n|w/o|93.4%|1.75|0.823|\n> Q2: Classifier guidance seems ad hoc and brings more complication to modeling. Would an architecture similar to that used in ESM3 or PVQD help?\n\nThanks for the question! Please see our response to W2 for the explanations of the classifier guidance. \nThe suggested architectures like ESM3 or PVQD are SE(3)-invariant, which are defect in integrating the geometric conditions of binding sites. They are primarily designed to generate entire proteins with a random coordinate system. However, our pocket-based generation requires the generated structures (i.e. the output) to be located in the same coordinate system as the pocket (i.e. the input), thereby necessitating SE(3)-equivariant modeling.\n\n> Q3: Table 1: out of 40 samples, there are about 20 different conformations (Div.) and 20 valid peptides (Success). How should the readers interpret this? Peptides exhibit high structural flexibility upon binding?\n\nThanks! To better interpret the results in Table 1, here we additionally calculate the diversity of generated peptides with successful binding (dG<0) in the table below. It reads that our generated peptides exhibit high structural flexibility upon successful binding. Thank you again for raising this valuable question and we will add this new result to the revised paper. \n|Div.|Div. ($\\Delta G < 0$)|\n|-|-|\n|0.506|0.632|\n> Q4: Did I miss sequence recovery metrics in the model benchmark section?\n\nSorry for the confusion. We did analyze the sequence recovery metric, namely, Amino Acid Recovery (AAR) in Appendix G.1 on a large dataset consisting of 600k peptide sequences against 328 receptors. Further, as shown in the table below, a high AAR might not be meaningful or reliable due to the extensive diversity in peptide binding. For example, HSRN achieves an AAR close to our method, but its success rate is much lower. Please find more details in our analyses in Appendix G.1, and we are willing to include the AAR results into the main body if requested. \n|Model|AAR|Success|\n|-|-|-|\n|HSRN|35.8%|10.46%|\n|DiffAb|37.1%|49.87%|\n|PepGLAD (ours)|36.7%|55.97%|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FegTAKengX", "reviewer": "Reviewer_KHMu", "summary": "This paper introduces a novel latent diffusion model, Peptide design with Geometric Latent Diffusion (PepGLAD), for the task of peptide design. The authors propose an affine transformation to project the raw Euclidean space into a standardized one, ensuring physical symmetry. Overall, I think it is a good paper, with solid experimental results and novelty in methods.", "questions": "1. What are the key differentiating factors of PepGLAD compared to GeoLDM [1]? While I recognize the shift from latent molecular generation to full-atom peptide generation, are there any specific model design elements (apart from the affine transformation) that set it apart from GeoLDM? Furthermore, does the model incorporate any strategies to reduce the costs associated with generating full-atom peptide scenarios?\n\n2. The most significant contribution made by the authors in this paper is the Receptor-Specific Affine Transformation. This transformation projects the geometry to approximately N(0,I), derived from the binding site. This approach appears to be the first of its kind applied to latent diffusion models for molecules and proteins. I am curious if there are any other works that have used this technique to ensure equivariance in their generation process, such as frame-averaging [6] [7].\n\n3. Compared to previously established databases [2] [3], the datasets utilized in your study appear relatively modest in size. Could you comment on the unique advantages of your proposed datasets over these larger ones? Furthermore, have you considered applying your model to existing datasets to reinforce the validity of your results?\n\n4. I suggest to add more citations on recently proposed related works on peptide design [3] [4] [5].\n\n[1] M Xu, et al. Geometric Latent Diffusion Models for 3D Molecule Generation\n\n[2] Z Wen, et al. Pepbdb: a comprehensive structural database of biological peptide-protein interactions.\n\n[3] L Lin, et al. PPFlow: Target-Aware Peptide Design with Torsional Flow Matching\n\n[4] Osama Abdin, et al. PepFlow: direct conformational sampling from peptide energy landscapes through hypernetwork-conditioned diffusion\n\n[5] Colin A Grambow, et al. RINGER: Conformer Ensemble Generation of Macrocyclic Peptides with Sequence-Conditioned Internal Coordinate Diffusion\n\n[6] W Jing, et al. DSMBind: SE(3) denoising score matching for unsupervised binding energy prediction and nanobody design\n\n[7] Omri Puny, et al. Frame Averaging for Invariant and Equivariant Network Design", "limitations": "See Appendix. J Limitations", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper proposes a latent diffusion model, Peptite design with Geometric LAtent Diffusion (PepGLAD) , applied to the peptide design task. This paper also proposes an an affine transformation to project the raw Euclidean space into a standardized one, so that the physics symmetry can be ensured. The main contributions of PepGLAD over GeoLDM seem to be its application to full-atom peptide generation and the introduction of a receptor-specific affine transformation. While GeoLDM¹¹ was designed for 3D molecule generation, PepGLAD extends this to the more complex scenario of full-atom peptide generation. The receptor-specific affine transformation is a significant contribution, as it projects the geometry to approximately N(0,I) derived from the binding site, which appears to be a novel technique in the field of latent diffusion models for molecules and proteins.", "weaknesses": "There are several doubts on method contributions, datasets, as shown in Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive comments!\n\n> Q1: What are the key differentiating factors of PepGLAD compared to GeoLDM [1]? While I recognize the shift from latent molecular generation to full-atom peptide generation, are there any specific model design elements (apart from the affine transformation) that set it apart from GeoLDM? Furthermore, does the model incorporate any strategies to reduce the costs associated with generating full-atom peptide scenarios?\n\nThanks for the question!\n\nThe motivations of our PepGLAD and GeoLDM are fundamentally different. We resort to the latent space where the full-atom geometry of each residue is compressed into a single node so that the diffusion can be implemented. Otherwise, when sampling residues of different type, the number of atoms varies, which is incompatible with the diffusion framework. On the contrary, GeoLDM maps each atom in the small molecule to one point in the latent space, without addressing the problem we are trying to tackle here.\n\nBy representing each residue's full-atom geometry as a single point in the latent space, PepGLAD inherently reduces the computational cost associated with generating full-atom scenarios. Specifically, the diffusion process only needs to handle a latent graph which is approximately 10 times smaller than the full-atom graph, resulting in improved efficiency and scalability.\n\n> Q2: The most significant contribution made by the authors in this paper is the Receptor-Specific Affine Transformation. This transformation projects the geometry to approximately N(0,I), derived from the binding site. This approach appears to be the first of its kind applied to latent diffusion models for molecules and proteins. I am curious if there are any other works that have used this technique to ensure equivariance in their generation process, such as frame-averaging [6] [7].\n\nYes, the proposed technique of receptor-specific affine transformation is novel and the first of its kind to the best of our knowledge. The technique differs from frame averaging from two perspectives. First, the motivations are totally different.  While frame averaging aims to design equivariant models, our technique serves as a normalization skill particularly for pocket-based geometric diffusion models. Second, while frame averaging uses PCA to obtain the principal components of the coordinates, we use cholesky decomposition of the covariance of the coordinates to identify unique invertible affine transformations for projections between the data distribution and standard Gaussian.\n\n> Q3: Compared to previously established databases [2] [3], the datasets utilized in your study appear relatively modest in size. Could you comment on the unique advantages of your proposed datasets over these larger ones? Furthermore, have you considered applying your model to existing datasets to reinforce the validity of your results?\n\nOur dataset is carefully curated to establish two advantages:\n\n1. Practical relevance: Consistent with previous research [a], we have focused on peptides ranging from 4 to 25 residues in length. This range is particularly relevant to practical applications such as drug discovery, as peptides within this length exhibit favorable biochemical properties [b]. In contrast, existing datasets like PepBDB [c] direct extract all complexes from PDB and lack filtering based on peptide lengths.\n\n2. Redundancy and split: We remove the redundant complexes and split the dataset according to the sequence identity of the receptor for testing generalization across different target. On the contrary, the existing dataset contains redundant entries and does not provides such splits.\n\nFurthermore, we did apply our model to an existing dataset, PepBDB, to reinforce the validity of our conclusions, the results on which are shown in Table 1 and Table 3.\n\n[a] Tsaban, Tomer, et al. \"Harnessing protein folding neural networks for peptide–protein docking.\" Nature communications 13.1 (2022): 176.\n\n[b] Muttenthaler, Markus, et al. \"Trends in peptide drug discovery.\" Nature reviews Drug discovery 20.4 (2021): 309-325.\n\n[c] Wen, Zeyu, et al. \"PepBDB: a comprehensive structural database of biological peptide–protein interactions.\" Bioinformatics 35.1 (2019): 175-177.\n\n> Q4: I suggest to add more citations on recently proposed related works on peptide design [3] [4] [5].\n\nThanks for the suggestion! We will definitely discuss and cite these recent works in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a1fRZ4o3Bm", "reviewer": "Reviewer_xBxR", "summary": "This paper explores the structure-based peptide design problem. A benchmark on this task and  a powerful diffusion-based model for full-atom peptide design, named PepGLAD, are proposed. PepGLAD explores the geometric latent diffusion, where the sequence and the full-atom structure are jointly encoded by a variational autoencoder. The proposed method outperforms the existing models on sequence-structure co-design and binding conformation generation.", "questions": "1. What’s the motivation for using Latent Diffusion instead of standard diffusion? Just because it’s better in image generation domain?\n2. How to incorporate geometric pocket condition (e.g. coordinates) into geometric latent diffusion process?\n3. Is the geometric latent are just the Ca atoms of each residues? Will using all four backbone atoms as latent improve the perforemance？\n4. Is the adaptive multi-channel equivariant encoder in dyMEAN a scalarization-based equivariant GNNs? It’s better to add some explainations in this paper.\n5. Does this Receptor-Specific Affine Transformation beat simple gaussian normalization, common trick in Structure-based Drug Design field?", "limitations": "The authors didn’t discuss about the limitations of this method. This paper failed to further explore the explicit interaction modeling in geometric latent space. I think it’s the key advantage compared to standard latent space (without the geometric latent).", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper tackles an under-explored problem and constructs a new benchmark which is beneficial for this field.\n2. The novel Geometric Latent Space can maintain the advantages of Latent Diffusion for efficiency and normalization and Geometric Diffusion for explicit interaction modeling.\n3. PepGLAD can achieve full-atom generation compared with backbone atom generation in previous methods, which is truly an improvement.\n4. PepGLAD utilizes a novel Receptor-Specific Affine Transformation to map the pockect-specific distribution to the standard distribution", "weaknesses": "1. This paper failed to further explore the explicit interaction modeling in geometric latent space. I think it’s the key advantage compared to standard latent space (without the geometric latent).\n2. The introduction to Receptor-Specific Affine Transformation can be more clear. I kindly suggest the authors can add one sentence or two summarizing the first paragraph in section 3.3 into the section 1 of Introduction. In such way readers may be more clear that this Affine Transformation is applied to the peptides and the pockets (i.e. binding site distribution in the introduction). Also the point 3 in line 62-65 has a mistake: according to the method section the affine transformation F is mapping binding site distribution (of peptides) into standard Gaussian (not the reverse)\n3.  The conditioning method of pockets is not explored. The denoiser takes the geometric latents of peptides and the coordinates of pockets as input, but the latents are coordinates are different levels of abstraction and this may cause some underlying issues.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful and constructive comments!\n\n> W1: This paper failed to further explore the explicit interaction modeling in geometric latent space.\n\nWe apologize for not making this point clear sufficiently.\n\nOur model inputs the pocket as a condition for diffusion. During denoising, we use an equivariant-GNN-based encoder to capture explicit geometric interactions between the peptide and the pocket. This is achieved by considering distances and inner products of relative positions between the full-atom pockets and the peptide latent points during geometric message passing.\n\n> W2: The introduction to Receptor-Specific Affine Transformation can be more clear. I kindly suggest the authors can add one sentence or two summarizing the first paragraph in section 3.3 into the section 1 of Introduction. In such way readers may be more clear that this Affine Transformation is applied to the peptides and the pockets (i.e. binding site distribution in the introduction). Also the point 3 in line 62-65 has a mistake: according to the method section the affine transformation F is mapping binding site distribution (of peptides) into standard Gaussian (not the reverse)\n\nThanks for the valuable suggestion!\n\nThe 3rd contribution in section 1 tries to summarize the receptor-specific affine transformation. To improve clarity, we will add the following sentence to the paragraph: \"While the complexes possess disparate distributions, we derive a receptor-specific affine transformation that is applied to both the binding sites and the peptides, projecting the shape of all complexes into approximately a standard Gaussian distribution.\"\n\nWe also appreciate you pointing out the mistake in lines 62-65. We will correct this in the revision.\n\n> W3: The conditioning method of pockets is not explored. The denoiser takes the geometric latents of peptides and the coordinates of pockets as input, but the latents are coordinates are different levels of abstraction and this may cause some underlying issues.\n\nThanks for the insightful observation! Since we have applied regularization on the equivariant latent coordinates by using KL divergence with respect to the $C_\\alpha$ coordinates in Eq.(4), which helps ensure consistent scales between the peptide latent coordinates and the pocket, mitigating potential issues arising from their different levels of abstraction.  We will highlight this point in the revision to address the reviewer's concern.\n\n> Q1: What’s the motivation for using Latent Diffusion instead of standard diffusion? Just because it’s better in image generation domain?\n\nThanks for the question! \n\nThe motivation for using Latent Diffusion in our work is to effectively address the challenges associated with full-atom generation, which is not well-suited to standard diffusion approaches due to two key problems:\n\n1. Fixed Data Lengths: In full-atom generation, changing the type of residue during the denoising process would alter the number of atoms per residue. This discrete change in data length is incompatible with standard diffusion framework.\n\n2. Generation with Paddings: If the number of atoms is fixed to the maximum number, as done in the baseline DiffAb, it is analogous to generating \"padding\" in NLP, which is suboptimal. Our results in Table 3 show that DiffAb’s performance in recovering full-atom geometry is significantly worse compared to PepGLAD.\n\nIn contrast, latent diffusion tackles these issues by using a full-atom encoder to map each residue to a single point in the latent space. This approach avoids changes in data length during diffusion, without need to generate paddings, therefore handles the full-atom generation more elegantly.\n\n> Q2: How to incorporate geometric pocket condition (e.g. coordinates) into geometric latent diffusion process?\n\nSorry for any confusion. We do incorporate the geometric pocket condition $\\mathcal{G}_b$ on both the autoencoder (please see Eq.(1)) and the diffusion model (please see Eq.(11-13)). The pocket is put into the same geometry graph with the peptides for geometric message passing.\n\n> Q3: Is the geometric latent are just the Ca atoms? \n\nWe apologize for the ambiguity in the presentation.\n\nThe geometric latents are not limited to just the Cα atoms, but regularized around the Cα atoms to prevent exploding coordinates. However, they still encode full-atom geometries, working jointly with invariant latent representations to reconstruct the full-atom geometry. \n\n> Q4: Is the encoder in dyMEAN a scalarization-based equivariant GNNs? \n\nYes, it is a scalarization-based equivariant GNN since the processing of coordinates in dyMEAN only includes invariant scalars via relative distances or inner products. We will add the explanations in the paper.\n\n> Q5: Does this Receptor-Specific Affine Transformation beat simple gaussian normalization?\n\nThanks! Yes, our experiments indicate that Receptor-Specific Affine Transformation outperforms simple Gaussian normalization.\nIn our preliminary experiments, we found that simple Gaussian normalization does not enhance generalization, as the high variability of peptide coordinate distributions and scales often leads to exploding coordinates on test complexes during generation. As Figure 4 shows, DiffAb, which employs simple Gaussian normalization, exhibits significant variations in RMSD across different test complexes, including many outliers with very large RMSD values. This suggests inferior generalization capability. In contrast, PepGLAD, which utilizes receptor-specific affine transformation, achieves stable RMSD values across various complexes, demonstrating superior generalization ability. Such discrepancy is unique to peptides, which often face more disparate geometry scales compared to antibodies and small molecules.\n\n> The authors didn’t discuss about the limitations of this method\n\nSorry, we put the discussion on limitations in Appendix J due to limited space. We will move the section to the main text in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wg7ygdX80Z", "reviewer": "Reviewer_XM8c", "summary": "The paper presents a new diffusion model for generating peptide binders given protein pockets. It also presents a new benchmark dataset, created by selecting examples from PDB and ensuring sequence dissimilarity between training and test data.", "questions": "Why do you need to generate all-atom designs? Do you think it could work just as well to generate C-alpha coordinates and discrete residue types? Mixed discrete/continuous diffusion models have shown some success for small-molecule drug design (DiffSBDD) and crystal design (MatterGen).\n\nLine 223: does ‘filter out’ mean ‘keep’ or ‘discard’? \n\nThe dataset seems small for deep learning and for the complexity of the task.   How could it be augmented?\n\nTable 3: what counts as a good RMSD, for practical purposes? Are all the methods given the same inputs, or are some doing blind docking?\n\nEquation (3): how is the MSE on full-atom structures defined when the residue type is wrong?", "limitations": "Like all peptide design in silico, this is hamstrung by lack of good in silico evaluation: dG values from Rosetta are not reliable estimates of real binding affinity. Ideally there would be some wet-lab evaluation of the generated designs.\n\nThe method requires binding site to be identified before peptide binders are designed.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "Thank you for constructing a benchmark dataset for this task with a proper train/test split. This is overlooked in some related work. Is the dataset available to download?\n\nNotation is clearly set out and there is a good survey of related work.\n\nThere are sensible ablations and comparisons to baselines, and the presented model performs well relative to these.", "weaknesses": "In the introduction and abstract, please state clearly that PepGLAD needs to be given the binding site (not the whole protein structure).\n\nLine 24 is hard to understand: ‘The key of peptide design …’.  It seems to say that strong binding requires the peptide to adopt a compact, inflexible shape, but I do not understand why.\n\nThe argument for the affine transformation is unclear.  I could not decipher line 48 ‘These variances define divergent target distributions of Gaussian…’.  Plenty of methods (AlphaFold, RFDiffusion, GeoDiff, …) are able to predict or design elongated protein and molecule shapes without any such transform, and here it should be easy because the binding site shape is given as a conditioning input. Surely a model can learn that the binder it generates should approximately match the shape of the binding site, even if the binding site is far from spherical?\n\nI think that proposition 3.1 could be stronger. Please check, but I think that if $x$ is rotated, then $L$ undergoes the same rotation, and hence $F_g(g.x)=F(x)$. Thus, $f(h, F(x))$ is invariant and $F^{-1}\\overrightarrow{f}(h, F(x))$ is equivariant even if $f$ and  $\\overrightarrow{f}$ are not.\n\nI am surprised that this transformation improves performance.  The score model in the transformed space now has to learn that bonds in different orientations should have different lengths, inferring from the inter-residue distances of the binding site how much each dimension is compressed or stretched.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful reviews!\n\n> W1: In the intro/abs, please state clearly that PepGLAD needs to be given the binding site.\n\nThanks for the suggestion! We will state it clearly in the revision.\n> W2: Line 24 is hard to understand: ‘The key of peptide design …’. \n\nSorry for the confusion. First, it means that to achieve strong binding, the peptide should form compact interactions with the pocket, since dense secondary bonding (e.g. hydrogen bond) contributes to the overal binding affinity. Second, the binding conformation of the peptide is largely affected by the pocket. Free peptides are usually flexible [a], whereas it may adopt certain modes upon binding to form denser interactions.\n\n[a] \"The X‐Pro peptide bond as an NMR probe for conformational studies of flexible linear peptides.\" Biopolymers: Original Research on Biomolecules 15.10 (1976).\n> W3: The argument for the affine transformation is unclear. I could not decipher line 48 ‘These variances define divergent target distributions of Gaussian…’. \n\nThanks for the comment!\n\nThe need for affine transformation arises from the unique challenges associated with peptides compared to general proteins and small molecules:\n\n1. **Irregular Conformation Scales**: Small molecules have minor scale variances, while protein spread is largely determined by residue number, regularized by the radius of gyration [b]. Conformations for peptides with the same length can vary widely, from linearly extended to compact ball-like structures, making normalization with a shape prior particularly crucial.\n\n2. **Dataset Size and Generalization**:  The smaller peptide dataset necessitates a well-defined geometric diffusion space for better generalization. Our affine-based normalization in PepGLAD achieves consistent and accurate RMSDs, as shown in Figure 4, compared to DiffAb's Gaussian normalization, which often results in outliers with high RMSDs.\n\n[b] \"Radius of gyration as an indicator of protein structure compactness.\" Molecular Biology 42 (2008).\n> W4: I think that proposition 3.1 could be stronger. Please check, I think that if $x$ is rotated, then $L$ undergoes the same rotation, and hence $F_g(g\\cdot x) = F(x)$.\n\nThanks for the comment, which raises a good theoretical question. Unfortunately, if $x$ is rotated by matrix $Q$, $L$ does not undergo the same rotation. Here is an example with 6 points: [ 0.687 -0.245  0.713], [ 1.164  1.221 -1.242], [-0.126 -0.797 -0.368], [-0.666 -0.501  1.189], [-0.315 -0.524  0.178], [-0.743  0.848 -0.47 ]\n\nCholesky Decomposition on the covariance matrix produces $L$, a lower triangular matrix:\n\n[ 0.766  0     0   ]\n\n[ 0.318  0.764  0   ]\n\n[-0.367 -0.496  0.623]\n\nRotation matrix $Q$:\n\n[-0.059 -0.357 -0.932]\n\n[0.519  0.786 -0.334]\n\n[0.852 -0.503  0.138]\n\nAfter rotation $Q$, Cholesky Decomposition produces $L_g$:\n\n[0.638  0     0   ]\n\n[0.639  0.904  0   ]\n\n[-0.087  0.033  0.632]\n\nHowever, $QL \\neq L_g$ as shown below! \n\n[0.183  0.19  -0.581]\n\n[0.771  0.767 -0.208]\n\n[0.441 -0.454  0.086]\n\n$QL$ is unnecessarily lower triangular, and thus not the solution of Cholesky decomposition. Specifically, multiple solutions exist for $Q\\Sigma Q^ = L_gL_g^T$(including $QL$), but only one unique solution exists (usually not $QL$) if restricted to lower triangular matrices. And we prefer a lower triangular matrix since its inverse matrix is easy to obtain. Hence, we can only derive $L_gL_g^T = QLL^TQ^T$, but not $QL = L_g$.\n> W5: I am surprised that this transformation improves performance. The score model in the transformed space now has to learn that bonds in different orientations should have different lengths, inferring from the inter-residue distances of the binding site how much each dimension is compressed or stretched.\n\nThanks for your insightful comments! The reasons are two-fold. First, The affine-based normalization enhances generalization, outweighing potential challenges from dimension twisting (please see response to W3). Second, The latent diffusion framework inherently addresses some challenges related to dimension twisting. During diffusion, we generate only latent point clouds with coarse-grained geometries. The detailed full-atom reconstruction is handled by the autoencoder, which operates in the original data space and is encouraged to capture the patterns of bond lengths and angles.\n> Q1: Why do you need to generate all-atom designs? Do you think it could work just as well to generate C-alpha coordinates and discrete residue types?\n\nThanks for the question! We think atom-level modeling provides better generalization as interactions are determined by secondary bonding between atoms. Ablations in Table 4 show significant degradation in diversity and success rate without full-atom context.\n> Q2: Line 223: does ‘filter out’ mean ‘keep’ or ‘discard’?\n\nSorry for the confusion. It means 'keep'. We will correct it in the revision.\n> Q3: How could the dataset be augmented?\n\nThanks for the question! We have explored data augmentation from monomer fragments, showing some enhancement. A synthetic dataset from models like AlphaFold 3 could be possible, but quality and balance issues need to be considered, which we leave for future work.\n> Q4: Table 3: what counts as a good RMSD, for practical purposes? Are all the methods given the same inputs, or are some doing blind docking?\n\nThanks for the question! Only Alphafold 2 is doing blind docking since it is a cofolding model. For practical purposes, RMSD < 5A ̊  indicates near-native conformations, and 2A ̊ typically indicates high-quality conformations [c].\n\n[c] \"Comprehensive evaluation of fourteen docking programs on protein–peptide complexes.\"Journal of chemical theory and computation 16.6 (2020).\n> Q5: Equation (3): how is the MSE on full-atom structures defined when the residue type is wrong?\n\nDuring training, we still compare the geometry to the ground truth. Both sequence loss and structure loss guide the model to adjust sequence and structure predictions simultaneously.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper focuses on the problem of designing peptides conditioned on a protein pocket. To that end, the authors first gather a dataset based on the Protein Data Bank, and then propose a diffusion-based model called PepGLAD, which combines several careful modelling and training choices appropriate for the domain. 

Reviewers generally considered PepGLAD to be a significant advancement on top of prior work, especially given its ability to do full-atom generation (instead of backbone only). The paper introduces several novel ideas, such as the affine transformation between the pocket position and a normal distribution, which most reviewers found interesting and appropriate for the downstream task. Apart from the model itself, this work also carefully curates a dataset with a reasonable train/test split; the reviewers noted the dataset is quite small, but they still considered it to be an important contribution to the field. While the initial version of the paper sparked many clarification questions, these were mostly addressed by the authors during the rebuttal period. In particular, one of the reviewers noted a subtle point around how the protein pocket is defined during data generation (which could potentially "leak" the shape of the ground-truth peptide); the authors did consider this nuance, and their current approach seems satisfactory. 

In summary, this paper curates a useful dataset for an important research problem, and builds an effective model on that dataset, combining a mix of novel and pre-existing ideas. I recommend the paper to be accepted, but I would invite the authors to take the clarification questions from the reviewers seriously, in order to make sure the manuscript is more accessible. In particular, for the final version of the paper I would encourage the authors to include a description of how the protein pocket is defined and what potential issues could arise from not doing this carefully.
Decision: Accept (poster)