Dynamics-Informed Protein Design with Structure Conditioning
Authors: Urszula Julia Komorowska
Simon V Mathis
Kieran Didi
Francisco Vargas
Pietro Lio
Mateja Jamnik
Venue: ICLR 2024 poster (2024)
Keywords: Diffusion Models
Generative Modeling
Protein Design
Normal Mode Analysis
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Current protein generative models are able to design novel backbones with desired shapes or functional motifs. However, despite the importance of a protein’s dynamical properties for its function, conditioning on dynamical properties remains elusive. We present a new approach to protein generative modeling by leveraging Normal Mode Analysis that enables us to capture dynamical properties too. We introduce a method for conditioning the diffusion probabilistic models on protein dynamics, specifically on the lowest non-trivial normal mode of oscillation. Our method, similar to the classifier guidance conditioning, formulates the sampling process as being driven by conditional and unconditional terms. However, unlike previous works, we approximate the conditional term with a simple analytical function rather than an external neural network, thus making the eigenvector calculations approachable. We present the corresponding SDE theory as a formal justification of our approach. We extend our framework to conditioning on structure and dynamics at the same time, enabling scaffolding of the dynamical motifs. We demonstrate the empirical effectiveness of our method by turning the open-source unconditional protein diffusion model Genie into the conditional model with no retraining. Generated proteins exhibit the desired dynamical and structural properties while still being biologically plausible. Our work represents a first step towards incorporating dynamical behaviour in protein design and may open the door to designing more flexible and functional proteins in the future.

Published as a conference paper at ICLR 2024
PROTEIN
DESIGN
WITH
STRUCTURE CONDITIONING
Urszula Julia Komorowska; Simon V Mathis; Kieran Didi, Francisco Vargas, Pietro Lio & Mateja Jamnik
Department of Computer Science and Technology
University of Cambridge
Cambridge, CB30FD, UK
{ujk21, svm34, ked48, fav25, p1219, mj201]&cam. ac. uk
ABSTRACT
Current protein generative models are able to design novel backhones with de- sired shapes or functional motifs. However, despite the importance of a protein's 's dynamical properties for its function, conditioning these dynamics remains elu- sive. We present new approach to include dynamical properties in protein gener- ative modeling by leveraging Normal Mode Analysis. We introduce a method for conditioning diffusion probabilistic models on protein dynamics, specifically on the lowest non- trivial normal mode of oscillation. Our method, similar to classifier guidance conditioning, formulates the sampling process as being driven by condi- tional and unconditional terms. However, unlike previous works, we approximate the conditional term with a simple analytical function rather than an external neu- ral network, thus making the eigenvector calculations approachable. We present the corresponding SDE theory as a formal justification of our approach. We ex- tend our framework to conditioning on structure and dynamics at the same time, enabling scaffolding of dynamical motifs. We demonstrate the empirical effec- tiveness our method by turning the open-source unconditional protein diffusion model Genie into
model with no retraining. Generated proteins exhibit the desired dynamical and structural propertics while still being biologically plausible. Our work represents first step towards incorpo- rating dynamical behaviour in protein design and may open the door to designing more flexible and functional proteins in the future.
INTRODUCTION
Generative Artificial Intelligence (AI) has rapidly accelerated protein design research. A common problem tackled with AI is the task of protein backbone design, which finding a new and realistic 3D structure tailored to the specific biological function. Recently, Al models based on the denoising diffusion framework (Ho et al., 2020; Song et al., 2021) have shown remarkable success in gener- ating realistic protein backbones, especially backbones with pre-defined, fixed substructures often referred to as motifs (Watson et al., 2022; Trippe et al., 2023). Since many functions have been linked to the presence of various functional motifs, enforcing the generation process to preserve such substructures is crucial in meaningful protein design. However, current modeling approaches do not incorporate important aspect of protein design structure alone is not enough determine the protein's 's functional properties. Information about protein flexibility, especially about its low- frequency collective motion, is crucial in determining protein functional properties (Bauer et al., 2019). In this work, we address this research gap and provide a framework for a diffusion model conditioned only on structural constraints but also on protein dynamics.
We analyse protein dynamics through the lens of Normal Mode Analysis (NMA) (Bahar et al., 2010). This is simple yet powerful method for obtaining eigenvectors of the motion of protein residues and their relative displacements in each mode. After performing NMA on real-life protein with known functionality, the obtained eigenvectors can be used as the dynamic targets when using a diffusion
"qqual contributions.

---

Published as a conference paper at ICLR 2024
model to sample novel backbone. We particularly interested in proteins which exhibit hinge- like motions, which are responsible for a number of protein functions and are strongly constrained in both structure and dynamics (Khade et al., 2020). Protein hinges usually involve two secondary structure elements rotating against each other about the common axis, similar to how a hinge at the door frame has closing and opening motions.
Our contributions are as follows:
We introduce a new methodology for conditioning protein generation dynamical prop- erties. Our approach is based on NMA which easy to compute and captures collective motions related protein function. Moreover, desired relative displacements, which we refer to as dynamics conditioning, can be accom- panied by structure conditioning. To substantiate this joint conditioning theoretically, present a formal interpretation in terms of stochastic differential equations.
demonstrate how conditioning the
We train our custom conditional diffusion model and generate dynamics bones. Thanks to the large number of real-life dynamics targets extracted from our data, provide detailed analysis of the effectiveness of the method. We measure the agreement of the displacements using custom loss function and manually inspect the agreement of target and sample displacement vectors for selected samples. Our method indeed allows us generate proteins with desired dynamics and easily transferable to other models.
dynamics-conditionee conditioned back-
We showcase the joint conditioning by applying it to trained Genie model (Lin & AlQuraishi, 2023). Through literature research, we select three proteins that exhibit hinge structures and motions, identify residues located in the hinge arms and use those as condi- tioning targets. Figure shows that we succeed in generating new and biologically plau- sible proteins with the targeted hinge dynamics, demonstrating that our framework can be transferred to other models in plug-and song--nddplaay -pla fashion.
Figure 1: Comparison of natural proteins (top) from which the hinge targets were extracted with conditional samples (bottom). Top row: from the left - lysozyme, adenylate kinase, haemoglobin. Bottom row: protein backbones synthesised with Genie that match the pre-selected hinge motif residues and have the desired dynamics, from the left with lysozyme, adenylate kinase, haemoglobin targets. Purple arrows are the displacements of selected residues in the normal mode, while green ones are the displacements in the same mode but in novel structure. Arrows have been scaled up for increased visual clarity. Note how the relative amplitudes and pair-wise angles of the green arrows match the constraints imposed by the target, and how the relative positions of the novel hinge residues in the original structure.
2

---

Published as a conference paper at ICLR 2024
2
BACKGROUND AND RELATED WORK
2.1 DIFFUSION PROBABILISTIC MODELING
The generative process in diffusion probabilistic models (Sohl-Dickstein et al., 2015) starts with sample from the standard normal distribution, æT ~ (N 1). The goal of this process is to transform JT into the sample To from the targeted data distribution (r(o), initially unknown and indirectly accessed by the trained model.
The key idea is formulate the model training as forward diffusion process in which the model predicts how much noise was added to the original sample. For a sample from the training set 2o, the forward process is defined as iteratively adding a small amount of Gaussian noise to the sample in T steps, which produces sequence of noisy samples Zorr such that the final sample 2T ~ (0, 1) to good approximation. In the Denoising Diffusion Probabilistic Modeling (DDPM) framework (Ho et al., 2020) the noise magnitude at each step is defined by variance schedule {3,,t E [0 T}} such that
(r( (z V1 -,
(1)
The above transition defines a Markov process in which the original data is transformed into a standard normal distribution. It is possible to write the density of x3 given zo in closed form
(0))) (xx, Vo (1 T))
F2 Varao + /1 GyCt,
where Or IT: 04 and a; - Br and é (0, 1). Transforming sample øт into the sample 20 is done in several updates that reverse the destructive noising, given by a reverse sampling scheme
VI 1 at co(
24-1
It
+ (1 oz)2,
va
where 2 ~ (0. 1). The neural network co (the denoiser) should be trained to predict noise added to zy. Ho et al. (2020) showed the following loss function is sufficient
Exg.t (1le= col V0rzo + vI o44,, t))1
(4)
Song et al. (2021) state that the DDPM is an example from the larger class of score-based models. They demonstrated that the discrete forward and reverse diffusion processes have their continuous time equivalents, that is, the forward Stochastic Differential Equation
dx 1 3(1)xal + v B(()dw.
(5)
and its reversal
dx - 8(1x 3(t) Imp((t) dt + VB(t)d
(6)
2
where the quantity V., (x:) is called the score and closely related to the noise in DDPM by the equivalence Va, (xt) -e//V-- ā. (derivation are in the Appendix F). Any model trained to predict the noise can be written in terms of the score. which is an essential prop- erty of our work. Whenever the nosse-based formulation for forward and reverse diffusion processes by simply substituting = a. Vz, In (e:e).
derive some expression with respect the score,
use
Related work on Diffusion Probabilistic Models for protein design. In the context of protein generative modelling, the real data samples zo are often represented by protein backbone coordinates (e.g., at the resolution of C2 atoms), optionally with amino-acid identity as scalar feature. Protein diffusion models operating on such representations were shown generate designable and novel samples to various degrees (Lin & AlQuraishi, 2023; Ingraham et al., 2022: Watson et al., 2022; Yim et al., 2023). Some of those were additionally designed to condition the sample on properties such as substructure, symmetry or structural motif; however, none of those works link the function to dynamics. Motif scaffolding has been done by, for example, providing the denoised motif residues positions in the conditional training (Watson al., 2022), by particle filtering methods (Trippe et al., 2023), or by empirically estimating the chances that the sample will have the query motif (Ingraham et al., 2022). Eigenfold (Jing al., 2023) attempts to incorporate the physical constraints for oscillations into the diffusion kernel, however, it did not improve the sample quality, and it was not tested whether it changes the dynamies of generated samples.
3

---

Published as a conference paper at ICLR 2024
2.2 NORMAL MODE ANALYSIS
Normal Mode Analysis (NMA) is a technique for describing collective motions of protein residues for a given energy function. It assumes that a protein in the energy minimum state in given force field, such that the protein residues will, to first approximation, undergo harmonic motions about their minima (Bahar et al., 2010). Amplitudes and frequencies of such oscillations are the solutions to the equations of motions for all residues. These equations of motions are compactly written in matrix form as M = - Kr, where z E R is a flattened vector of coordinates of N residues, M E I X3N a mass matrix and K E R x3N is the interaction constants matrix derived from the force field that describes the strength of interactions between residues. Despite the simplistic assumptions about the form of these force fields, NMA has been shown to successfully explain many dynamical phenomena amongst numerous proteins (Gibrat & Gö, 1990; Tama & Sanejouand, 2001; Bahar et al., 1997). Most functional properties of proteins that involve dynamics are related to the low-frequency motions, mathematically represented as the lowest non-trivial eigenvectors of the matrix equation.
3 METHODS
Consider the following problem: given a target matrix yn E R|(|x3 where rows correspond to displacement vectors of C residues, we aim generate new protein in which the displacement vectors of selected residues in their non-trivial lowest normal mode are close to those defined by yD. We use a coarse-grained protein representation, where each residue is represented with the Co carbon only, and aim to obtain new Ca chains that satisfy the dynamics constraint. To tackle this problem we employ score-based generative modelling (Song et al., 2021). We formulate the agreement of the displacement with a target as a condition in the reverse process and quantify the notion of "similar dynamics' with a custom loss function.
3.1 CONDITIONING DIFFUSION MODELS
The goal of conditional generative modeling is to sample from the posterior p(zoly) such that new samples To satisfy some chosen property y. We specify the following model (Song et al., 2023, Equation 4)
p(zoly) = p(ao) expl-1(4] (xoo)
and k(v) = p(xo) expl-1(y, e(zo)))daoo
p(0)) expl (1y,, v(xr)) (do)))do
where I④y, v(xo)) measures the loss for a measurement of Wi. k(y) is the normalisation con- stant, and v(z) maps to the relevant physical quantity represented by y. This specification, as shown in Song al. (2023), allows for guiding trained unconditional model along the path specified by the loss 1. Finding an appropriate p(y(<r) ) is where the novelty of our method lies. For the dynamics target y, if p(y) was a neural network, would need to approximate the eigenvectors of an arbi- trary symmetric matrix. To the best of our knowledge, finding matrix eigenvectors for any variable size symmetric matrix with a neural network is not considered solved problem yet (there exist neural network approaches to find eigenvectors, but those require retraining for every matrix (Gemp et al., 2021; Yi et al., 2004), and are not suitable for a large dataset of backbone structures). A method to reconstruct graph structure from set of learned eigenvectors via interactive Lapla- cian matrix refinement is presented in Martinkus et al. (2022). However, this approach has never been tested for reverse reconstruction. We escape the need to train a neural network and equate p(y) to simple analytical function.
One of the most common mathematical frameworks to obtain novel sample with any desired prop- erty y consists of estimating conditional scores. Different approximations for estimating said score have given rise variety of methods such as classifier guidance (Dhariwal & Nichol, 2021). clas- sifier free guidance (Ho & Salimans, 2022), and "reconstruction guidance' (Ho et al., 2022; Chung et al., 2022a). What all these approaches have common is that they decompose the conditional score
Va, Inpela)l)) = Va, In (np(pa + Va, In pe(x))
(8)
4

---

Published as a conference paper at ICLR 2024
where p(y( is a probability that the sample meets the condition at 0 given the state 2, some other time. Following Chung et al. (2022a), we re-express it with the integral
p(y(a))) p(yl (o) (x)
(9)
The integral is intractable and we cannot evaluate Po (x)) directly. But as in Chung et al. (2022a), overcome this via the approximation of the denoiser's transition density with delta function centred at the mean
(=(2 s óz (20)
(10)
Such approximations to the posteriors via point masses centred at their means rather than their modes (MAP) are known as Bayes point machines (Herbrich et al., 2001), and have been shown to outperform MAP. Under this approximation, the entire integral simplifies to
p(y( Rs p(vl E [2o/ (ra)).
(11)
Via Tweedie's formula (Chung et al., 2022a), the expected output of the model t
E [20| Tt (1 (
(12)
Var
Under our model specification, via Bayes rule
p(yl E [()) p(E (ma]) (BB(o)/
(13)
substituting back into the score we obtain
22, In p(E [z) [xpp= expl-1(y, e(E [xol2x))))))
p(v) = V., (y, vE (zo)=x))).
(14)
p( [x0o(()))))
Depending on the quantity y. different losses must be used in Equation 14. Note that even though the derivations are done in continuous time, the equivalence of the score and the noise still applies, and we can use the discretised sampling scheme as in Equation 3. Now, we explain our choices for dynamics and structure conditioning losses.
3.2 DYNAMICS LOSS
The next step to define the loss function Equation 14 that enforces the targeted dynamics while being invariant to the protein rotations and translations. Knowing the expected residues' positions at = 0 and the expected components of the normal mode of the conditioned residues given struc- ture Te at some time the invariance is preserved if one compares the relative pairwise angles between the displacement vectors and their relative magnitudes. Moreover, this makes the condi- tioning target independent of the protein length: eigenvectors of displacements of a subset of residues depend on the protein length. Therefore, we propose to use the following loss in Equation 14, which is simple combination of amplitude and angle terms between all pairwise residues. For the rest of this work, we refer to it as the NMA-loss.
normalised, hence the amplitudes
INMA (VDD v(x)) lagge (VD, v(x)) lamp v(x)),
(15)
langle
Σ AEE
cos(yD, vD.)) cos(v()))); u(T)g))
(16)
|!wD
[|x(2) [|x())]
lampl
Σ IEC
(17)
|!bal|
In this invariant loss, VDD and u(()); are displacement vectors of residue i E C in the target yo and in the displacements matrix v(x)) x3 derived from expected positions at = 0. The amplitude terms are normalised such that only their relative sizes matter, consistent with the fact that amplitude information from NMA can only make relative statements about the participation of given residue in a mode (Bahar et al., 2010). For the combined loss, in the process of minimisation of NMA-loss in the sampling steps, the aamp is scaled by 2, such that its contribution is similar in magnitude langte- We compute the NMA-loss using a differentiable implementation of the eigenvector calculations assuming the Hinsen force-field ((Hinsen & Kneller, 1999), more details in Appendix B.2).
5

---

Published as a conference paper at ICLR 2024
3.3 STRUCTURE LOSS AND JOINT CONDITIONING
The essential part of our work is building a connection between conditioning on dynamics and con- ditioning structure. Even though dynamics and structure are correlated, many structures will have similar low-frequency eigenvectors, and there is no guarantee that the particular protein packing will correspond to the biological function for which the dynamics were designed. Therefore, dynamics conditioning must be accompanied by structure conditioning. Structure conditioning enforces the generated protein backbone to have subset of residues Car positioned in pre-defined relative posi- tions. For example, structure conditioning might enforce the presence of a given functional motif M somewhere in the arbitrarily rotated protein. We denote the target positions as уM R Carl x3, and ICM E IR. Car |x3 is the prediction of conditioned residues' positions at / = 0 in the sampling pro- cess. In the language of score-based generative modeling, the conditional score for the joint target (yD. yM) will be now decomposed into three terms
vx, In ( yar) Va, p(yp/az)) + V Inp(yn(=x) vv, (xp)
(18)
Finally, the appropriate structure loss should be substituted to the Vz, Inp(y (x)) term. We define the structure loss be the misalignment between yM and IC specifically the L1 loss between all Cm residues' coordinates. In order not to violate equivariance, we use our custom differentiable im- plementation of the Kabsch algorithm (Kabsch, 1976; 1978) to find the best of the target residues yar and ICM at the reverse diffusion step and only then compute the misalignment. In the discus- sion of the results, we report the final moo--aean-square deviation (RMSD), which is related to but different from the structure loss (see Section 5.2).
4 MODELS AND THE EXPERIMENTAL SETUP
The aim of the experimental evaluation is two-fold. Firstly, we test whether the proposed con- ditioning method indeed results in better agreement between the target and the novel structure's dynamics. To do we use our custom denoiser model, perform conditional sampling using large number of dynamics targets and examine the conditioning effectiveness. Secondly, we utilise Ge- nie (Lin & AlQuraishi, 2023), the diffusion model able to produce high-quality samples and modify its sampling scheme with our joint conditioning. We therefore demonstrate the universality of our framework which leaves open path to transferring method to other large protein diffusion models. The modified Genie model produces samples conditioned on the hinge targets which we thoroughly evaluate for designability.
4.1 MODELS
GVP (Geometric Vector Perceptron (Jing et al., 2021b;a)) is the main building block of our equiv- ariant denoiser. We use Graph Neural Network with layers based on GVP (details in the Ap- pendix B. The denoiser was trained with the loss function given by Equation 4. We use the Hoogeboom schedule (Hoogeboom et al., 2022) with 250-step DDPM discretisation scheme. The model was trained for 1000 epochs with learning rate of lc-4.
Genie. Genie (Lin AlQuraishi, 2023) is a diffusion probabilistic model with the DDPM discreti- sation. It takes advantage of the protein geometry by extracting the Frenet-Serret frames of residues at each noise prediction step, which are then passed to the SE(3)-equivariant denoiser. Genie out- performed other models such ProtDiff (Trippe et al., 2023), FoldingDiff (Wu al., 2022) or FrameDiff (Yim et al., 2023), and remains comparable to RFDiffusion (Watson et al., 2022). For our experiments, we used the published weights of the model trained on the SCOPe dataset (Fox et al., 2014; Chandonia et al., 2021) able to work with proteins up to 256 residues long.
4.2 DATASET AND TARGETS
For our custom model training, we extract all short monomeric CATHv4.3 domains (Orengo et al., 1997) for structures with high resolution (< 3A), of lengths between 21-112 amino acids, clus- tered 95% sequence similarity to remove redundancy. The resulting dataset contained 10037 protein structures.
We extract random and strain dynamics targets from the proteins in the validation set. Random targets are the displacements in the randomly chosen sets of 10 consecutive residues; for the strain

---

Published as a conference paper at ICLR 2024
targets, perform strain-energy calculation (Hinsen & Kneller, 1999) (details in the Appendix B.2) and choose 10 consecutive residues with the largest summed energy.
Joint conditioning imposes constraints on both the protein normal mode and the specific residues' positions. Biologically relevant targets that require such constraints are the hinge parts of proteins. Three proteins were selected from the literature: lysozyme (PDB ID: 6lyz), adenylate kinase (PDB ID: 3adk), and haemoglobin (PDB ID: 2hhb). In each protein we analysed which residues participate in the hinge motion- those residues constitute the yM targets. For each protein we perform NMA calculation to obtain the displacements of the hinge residues - the yn targets (details in Appendix D).
4.3 EVALUATION METRICS
Population level. For the first set of experiments investigating dynamics conditioning, we focus on quick-to-compute statisties of the large sample set to understand the expected effects of con- ditioning on the sample quality. Apart from the NMA-loss, (I) the mean chain distance (Ca - Ca) that should be close to 3.8 A (2) the radius of gyration of the backbone, which is an indicator of whether the model produces samples with adequate compactness: (3) secondary structure statistics (SSE), that is, the proportion of a-helices, 3-sheets and disordered loops; (4) novelty in terms of the TM-score to the closest structure in the train set. TM-score measures the topological similarity of protein structures and has values in the range [0, 1J. TM-score > 0.5 suggests two structures are in the same fold (Xu & Zhang, 2010).
check the sample quality using:
Detailed statisties. In the case of joint conditioning, we sample novel protein backbones using Genie and check the designability of the new samples using the same in silico evaluation pipeline as in benchmarking unconditional Genie. For each backbone sample, we obtain 8 ProteinMPNN generated sequences and fold each sequence with ESMFold (Lin et al., 2022). We calculate the self-consistency TM-scores (scTM), that is, the TM-scores between the input structure and each of the FSSMFold predictions. scTM scores were also considered in other works (Trippe et al., 2023; Lin & AlQuraishi, 2023) as one of the standard metrics for sample quality evaluation. We report the proportion of conditional samples whose best scTM-score to one of the ESMFold designed struc- tures is > 0.5, in the same fashion as in Trippe et al. (2023) that tackles a similar motif conditioning problem.
4.4 SAMPLING DETAILS.
Dynamics conditioning with GVP. The sampling process consisted of 250 reverse diffusion steps (details in the Appendix B.3). We extracted 300 strain and 300 random targets from 300 randomly sampled proteins from the validation set. For each target, we took 3 conditional and unconditional samples, and for each group we selected the one with the lowest NMA-loss. Each sample had the same length as the protein from which the target was extracted.
Joint conditioning with Genie. The original Genie sampling loop with 1000 time steps in the generation was modified to include the conditional score (details in the Appendix B.3). The guidance scales were different for each target, and in the order of 2000-3000.
RESULTS AND DISCUSSION
5.1 STRAIN AND RANDOM DYNAMICS TARGETS
Here we present the results for the strain and random dynamics targets. At the start, we filter out the 'low quality' samples that evidently do not form biologically valid proteins (details in the Appendix C.We examine the conditioning has the desired effect of enforcing the target normal mode. Figure 2 shows that indeed, the NMA-loss is successfully minimised in the conditional samples as compared to the unconditional ones. Note that both the target normal mode and the mode of the newly sampled structure must obey some physical constraints imposed on all proteins and the degrees of freedom of all relative displacements are limited, therefore it is occasionally possible to obtain low loss for the unconditional sample. Encouraged by this finding, to the visual inspection of the samples. Figure 3 shows a pair of conditional and unconditional samples for one of the strain targets (additional sampled pairs are in Appendix G). There is better
proceed
7

---

Published as a conference paper at ICLR 2024
Uncond. Coad.
15
Uncond.
6-
Cond.
Density
Deusity
10
4
5
0
0.00
0.25
0.50
0.75
1.00
0.0
0.4
0.6
1.0
NMA-loss U(s o((r))
NMA-loes fag e(x)))
(a) Strain targets
(b) Random targets
Figure 2: Density histograms of the NMA-loss for the dynamics conditioning using random and strain targets. Conditioning shifts the distribution towards lower values, such that the distribution has an evident sharp peak.
alignment of the displacement vectors and target vectors for the conditional sample as compared to the unconditional one, which we also consistently observed for the rest of the sampled pairs. We conclude that our conditioning has the desired effect of enforcing the target dynamics. We therefore proceed to the quality check of the samples backbone structure. ensure that the sampled proteins are still biologically valid, we evaluate
must ensure the conditioning does not compromise
Figure 3: Comparison of two samples for the same strain target. Left : Conditional sample (NMA 1oss-0.114). Right: : Unconditional sample (NMA-loss 0.740). In the conditional sample, green vectors (new displacements) have much more similar relative amplitudes and pair-wist maie angles the purple vectors (target) when compared to the unconditional sample. In both left and right visualisa- tion, purples rotated match greens.
their geometry. In the end, we investigate the samples' novelty to check whether the diffusion model has not simply memorised the train set.
Figure 4 shows the SSE and Rg of the samples compared to the train CATH dataset. Uncondi- tional samples show a variety of SSE in proportions close to the CATH dataset. Interestingly, we found that conditioning increases the proportion of (3-hhett the expense of a-helices. R. dis- tributions of both unconditional and conditional samples have a visible overlap with the CATH R2 distribution, the second one is shifted larger values (but remains within the Rg values observed in CATH). Therefore, while the conditional samples do not violate physical constraints, the dy- namics conditioning introduces changes in protein packing. Whether this effect is significant for downstream applications when the conditioning is transferred into problem-specific models is left for future work. Respective Figures for the random targets can found Appendix A. Lastly, we calculate the novelty of the samples expressed in terms of TM-score to the closest structure in the train set. Both unconditional and conditional samples of both target types were highly novel, with TM-score lower than 0.5 in 90% of the samples.
5.2 HINGE TARGET
Finally, present results for the joint conditioning. The conditional samples were filtered using criteria of mean chain distances outside [3.75, 3.85] A interval and RMSD with respect to the motif smaller than 1 Á. These constraints left us with 43%, 60% and 23% of the conditional samples for
8

---

Published as a conference paper at ICLR 2024
0.3
CATH
Uncond.
50.2%
81.2%
50.0%
Density
0.2
Cond.
15.8%
0.1
17.1%
28.79%
21.7%
33.6%
0.0
Uncond.
Cond.
CATH
20
Radius of gyration Ry [A]
Helix
Sleet
(a) Rg
(b) SSE
Figure 4: Density histogram of Rg and SSE proportions for strain targets.
Uncond.
Uncond.
Coud.
Cond.
Count
0
0.2
0-4
0.6
0.8
2.5
10.0
12.5
15.0
NMA-loss Ugg v(x,))
Hingo RMSD
[A]
(a)
(b)
Figure 5: NMA-loss and RMSD for the lysozyme hinge target. Conditional samples achieve low values of NMA-loss and RMSD that none of the unconditional samples have.
lysozyme, adenylate kinase and haemoglobin, respectively, such that we ended up with 27 condi- tional samples. match that number, we sampled 27 unconditional ones. In the analysis of the remaining samples, we considered the distributions of NMA-loss (see Figure 5) and scT'-sscre. The distribution the NMA-loss confirms that our method can enforce the specific dynamics and conditions the structure at the same time. Analysis of the designability revealed that the dis- tribution of scTM-scores depends on the target we use. The proportions of conditional samples with scTM-sccree 0.5 were 0.48, 0.78, 0.41 for lysozyme, adenylate kinase and haemoglobin, spectively. Interestingly, when we sampled 27 structures just with the hinge dynamics conditioning, those values were 0.93, 1.0, and 0.89, respectively, and the decrease in designability purely to the difficulties in the structure conditioning (Appendix E). Additional experiments with conditionally trained Genie model and extra designability results can be found in Appendix H. We finish with the visual investigation of the generated hinge structures. Figure shows pairs of the targets and the new samples (more examples in the Appendix G). The new samples indeed possess the hinge structure, as well as the hinge-like low-frequency motion.
be attributed
CONCLUSIONS AND FURTHER WORK
For the first time, we condition the protein diffusion model on dynamics, thus paving the way to designing more functional proteins in the future. We also make the code publicly available', We generate novel proteins with a pre-defined lowest non-trivial normal mode of oscillation for a subset of residues. The large-scale statistics show that the conditioning effective and can be transferred to already trained unconditional models. The extended version of the conditioning that includes the structure conditioning implemented as part of the unconditional Genie model and we produce novel proteins that exhibit hinge structure and dynamics while remaining designable by the scTM
Coode available https: //t::////tttt www.cchww.......oooooceet com/ujk2111 /ndpdinnnorndddd...o...o

---

Published as a conference paper at ICLR 2024
criteria. Further work includes integrating the dynamies conditioning with other types of structure conditioning, and further evaluation with other types of motions.
REFERENCES
Ivet Bahar, Ali Rana Atilgan, and Burak Erman. Direct evaluation of thermal fluctuations in proteins using a single-parameter harmonic potential. Folding and Design, 173)117--81, 1997.
Ivet Bahar, Timothy R. Lezon, Ahmet Bakan, and Indira H. Shrivastava. Normal Mode Analysis of Biomolecular Structures: Functional Mechanisms of Membrane Proteins. Chemical Reviews, 1103))14433-447, 2010.
Jacob A. Bauer, Jelena Pavlović, and Vladena Bauerová-Hlinkore Normal mode analysis as routine part of structural investigation. Molecules, 24(18):3293, Sep 2019. ISSN 1420-3049. doi: 10.390/molccuts241833999 molecules24181 2832993..
URL http: dx. doi /..g/11..99.
Nathaniel Bennett, Brian Coventry, Inna Goreshnik, Buwei Huang, Aza Allen, Dionne Vafeados, Ying Po Peng, Justas Dauparas, Minkyung Baek, Lance Stewart, Frank DiMaio, Steven De Munck, Savvas N. Savvides, and David Baker. Improving de novo protein binder design with deep learning. bioRxiv, 2022. doi: 00.1100/2022.06.55... 15.495993. URL https: /twp///www////////////////tt/// biorxiv.
19....66..6.699..6 495993.
Bernard Brooks and Martin Karplus. Normal modes for specific motions of macromolecules: appli- cation to the hinge-bending mode of lysozyme. Proceedings of the National Academy of Sciences, 82(15):4995-4999, 1985.
Patrick Bryant. Structure prediction of alternative protein conformations. bioRxiv, 2023, doi: 10. 0001/2023.09.255559266 URL https: fittps://wwee http://wwwboooxxic org/ / 09/25/2023. 25. 559256.
John-Marc Chandonia, Lindsey Guan, Shiangyi Lin, Changhua Yu, Naomi K Fox, and Steven E Brenner. SCOPe: improvements to the structural classification of proteins extended database to facilitate variant interpretation and machine learning. Nucleic Acids Research, 50(D!):D553- D559, 12 2021. ISSN 0005-1048. doi: 10. 10.1093/nar/gkaa1003 URLhttps: http://ooo..o/.//////// 1093 nar hwwpaa/g///a04/
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. The Eleventh Interna- tional Conference on Learning Representations, 2022a.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683-25666. 2022b.
Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. Robust deep learning- based protein sequence design using proteinmpon. Science, $78(6615):499
2022.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neu- ral Information Processing Systems, 2021. URL htttpp:://operreview. net forum?id- AAWuCvzaVt.
Kieran Didi, Francisco Vargas, Simon Mathis, Vincent Dutordoir, Emile Mathieu, Urszula J Ko- morowska, and Pietro Lio. A framework for conditional diffusion modelling with applications in motif scaffolding for protein design. arXiv preprint arXiv:2312.09236 2023.
NK. Fox, Steven E. Brenner, and JM Chandonia. Scope: Structural classification of pro- teins extended, integrating scop and astral data and classification of new structures. Nucleic Acids Research, 42:D934--D309, 2014. doi: 10. 10900//a/ggkt112400
10

---

Published as a conference paper at ICLR 2024
Ian Gemp, Brian McWilliams, Claire Vernade, and Thore Graepel. Eigengame: (PCA) nash equilibrium. In International Conference on Learning Representations, 2021. URL https:
/ //openzeview net wwt.::/eoorr.aaaeeettt55e
Jean-François Gibrat and Nobuhiro Go. Normal mode analysis of human lysozyme: study of the relative motion of the two domains and characterization of the harmonic motion. Proteins: Struc- ture, Function, and Bioinformatics, 8(3):258-279 1990.
Jeanette Held and Sander van Smaalen. The active site of hen egg-white lysozyme: flexibility and chemical bonding. Acta Crystallgraahhca Section D: Biological Crystallography, 70(4):1136 1146, 2014.
Ralf Herbrich, Thore Graepel, and Colin Campbell. Bayes point machines. Journal of Machine Learning Research, 1(Aug):245-277 279 2001.
Konrad Hinsen and Gerald R. Kneller. A simplified force field for describing vibrational protein dynamics over the whole frequency range. The Journal of Chemical Physies, 111(24):10766 10769, 12 1999. ISSN 0021-9606. doi: 1111063/1480444 URL https: battps://dol: /doi. org/10. 1063/1 80441.
Jonathan Ho and Tim Salimans. arXiv:2207.12598, 2022.
Classifier-free
diffusion guidance.
arXiv preprini
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 6840-6851. Curran Associates, Inc., 2020. URL https://pptt pettp:://proceeddnggg neurips...... wtt.cccppppeeeeeeeeeeeeeee www.rps.cccpppprr.rr. http///prr.fflsepppeeeeeeeeeeeee les/paper/2020.. 9ca 4b-Paper.pdi
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models, 2022. URL hltps://arxiv. org/abs/2204.0344 03448, 2022.
Emiel Hoogeboom, Víctor Gareia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation in 3D. Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Confer- ence on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 8867-8887. PMLR, 17-23 Jul 2022. URL httpsttt///c.c....m httpss//pooceeddiege mlr press/v162/ hoogeboom22a.11 html.
John Ingraham, Max Baranov, 7ak Costello, Vincent Frappier, Ahmed Ismail, Shan Tie, Wujie Wang, Vincent Xue, Fritz Obermeyer, Andrew Beam, and Gevorg Grigoryan. Illuminating pro- tein space with programmable generative model. bioRxiv, 2022. doi: 10. 101110002222.1.011 518682. URL https::/www www.bbioxxii... wwwp.iiwwwgg//oooo...o..o 12. 01. 518682.
Bowen Jing, Stephan Eismann, Pratham N. Soni, and Ron O. Dror. Equivariant graph neural net- works for 3d macromolecular structure, 2021a.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. In International Confer- ence on Learning Representations, 2021b. URL httph:ttp/////c //pp//peeeeee...eooe net fest/foroue forum? id= 1YLJDVSx6J4.
Bowen Jing, Ezra Erives, Peter Pao-Huang, Gabriele Corso, Bonnie Berger, and Tommi Jaakkola. EigenFold: Generative Protein Structure Prediction with Diffusion Models. arXiv, 2023.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A Kohl, Andrew J Ballard, Andrew Cowie, Bernardino Romera- Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver. Oriol Vinyals, Andrew W Senior, Koray Kavukcuoglu, Push- meet Kohli, and Demis Hassabis. Highly accurate protein structure prediction with AlphaFold. Nature, 596((773))583--555 August 2021.
11

---

Published as a conference paper at ICLR 2024
W. Kabsch. A solution for the best rotation to relate two sets of vectors. Crystallographica Sec-
tion A, (2(())22222999))) 923, 1976. doi: http://oaarrr/011111///667777770 URL https:
//onlineeibraryyy wi
W. Kabsch. A discussion of the solution for the best rotation to relate two sets of vectors. Acta
Crystallographiica Secttion A, 34(5): Sep 1978. doi: 10.1107/06567739770006688 URL https: /tta:://doi.o http://.......01...0000777777770
Prashant M Khade, Amit Kumar, and Robert L Jernigan. Characterizing and predicting protein hinges for mechanistic insight. Journal of Molecular Biology, 432(2):508-522, Jan 2020. doi: jmb. 20199. 11.11.1... 018.
(..0016).mmb220991110000 URL https::///// //t:::///oo..../ wwt.o//g/oo.ooo 10111111111000
Philipp Kunzmann and Kay Hamacher. Biotite: unifying open source computational biology framework in python. BMC Bioinformatiis, 19(1))346, 2018. doi: 10.1186/s1 10.116641889---18--3335 2859-018-2367-2. URL https: doii http://t..1.///285--90000
Yeqing Lin and Mohammed AlQuraishi. Generating novel, designable, and diverse protein struc- tures by equivariantly diffusing oriented residue clouds, 2023.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, al. Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv, 2022.
Aaron Lou and Stefano Ermon. Reflected diffusion models. In International Conference on Machine Learning. PMLR, 2023.
Karolis Martinkus, Andreas Loukas, Nathanael Perraudin, and Roger Wattenhofer. Spectre Spec- tral conditioning helps to overcome the expressivity limits of one-shot graph generators. In Inter- national Conference on Machine Learning, 2022. URL https: http:/app...aaaiiciicceemm www.ntcsccooolll......o org/Corpuu ID: 24 7339990.
Christine A Orengo, Alex D Michie, Susan Jones, David T Jones, Mark B Swindells, and Janet M Thomton. Cath-a hierarchic classification of protein domain structures. Structure, 5(8): 1109, 1997.
David Perahia and Liliane Mouawad. Computation of fow-frequenc normal modes in macro- molecules: improvements the method of diagonalization in mixed basis and application to hemoglobin. Computers & chemistry, 19(3):241-246, 1995.
Herbert E. Robbins. An empirical bayes approach to statisties. 1956. URL https://aag
semant www.ntta..ccccc.....oo...m.a.... org/ hts//Crrpussp 261 61 411..
Jascha Sohl-Dickstein. Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using noncquilibrium thermodynamics. In Francis Bach and David Blei (eds.). Pro- ceedings of the 32nd International Conference Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 2256-2265, Lille, France, 07-09 Jul 2015. PMLR. URL https: http///pcceenngnneeee .lm.rrr...... press vv7 wtwss/ohclsssssiisliiaee html.
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 32483-32498, PMLR, 23-29 Jul 2023. URL https: ptw.ceeeoeedeeeopooooo ngs.... ml press v song23k..... h
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021. URL https: httpht/t///eeeeeeiiiii n
ForumPiG-PXTIG122RIS
Florence Tama and Yves-Hennt Sanejouand. Conformational change of proteins arising from normal mode calculations. Protein engineering, 14 1:1-6, 2001.
12

---

Published as a conference paper at ICLR 2024
Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif- scaffolding problem. In The Eleventh International Conference Learning Representattons, 2023. URL httpt:tt/////..... http//pppprreee..com
forum? (
David Vocadlo, Gideon Davies, Roger Laine, and Stephen G Withers. Catalysis by hen egg-white lysozyme proceeds via covalent intermediate, Nature, 412(6649):835- 838, 2001.
Donald Voet and Judith Voet. Biochemistry. John Wiley & Sons, 2010.
Joseph L. Watson, David Juergens, Nathaniel R. Bennett, Brian L. Trippe, Jason Yim, Helen E. Eisenach, Woody Ahern, Andrew J. Borst, Robert J. Ragotte, Lukas F. Milles, Basile M. Wicky, Nikita Hanikel, Samuel J. Pellock, Alexis Courbet, William Sheftler, Jue Wang, Preetham Venkatesh, Isaac Sappington, Susana Vázquez Torres, Anna Lauko, Valentin De Bortoli, Emile Mathieu, Regina Barzilay, Tommi S. Jaakkola, Frank DiMaio, Minkyung Back, and David Baker. Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models. bioRxiv, 2022. doi: 10. 10.1101/2022..22995551
URL https: //www. www.xiv..ivi.................... org/ wwwttot/eeetttttno / 2022 12 /22/22/222222222 5199842..
Kevin E. Wu, Kevin K. Yang, Rianne van den Berg. James Y. Zou, Alex X. Lu, and Ava Amini. Protein structure generation via folding diffusion, 2022.
Jinrui Xu and Yang Zhang. How significant is protein structure similarity with tm-score = 0.5? Bioinformatics, 26(7):889-95, Apr 2010. doi: 10. 1099/bioonfrrmmcc///0066.. URL https:
Zhang Yi, Yan Fu, and Hua Jin Tang. Neural networks based approach com- puting eigenvectors and eigenvalues of symmetric matrix. Computers & Mathematics
with Applications, 4709):155-1164, 1016/S0898- 122100)991110111111111 URL wttp:///wwwcicieeieeettettteteee htt.ediiieeetttteeeeeeeeeeeeeeee com/science/
2004. ISSN 0898-122 1221. doi: https://oi.org//
atticcle/p11/00981220000000
Jason Yim, Brian L Trippe. Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation. arXiv preprint 202v:23322.0227,20000 2023. 2023.
APPENDICES
A POPULATION STATISTICS FOR RANDOM TARGETS
0.3
Uncoodd.
50.6%
61.2%
Deneity
0.2
Coad.
15.8%
0.1
33.6%
18.1%
0.0
20
Rodius of geration R2 [A]
Helix
Sheet
Coil
(a) Rg
(b) SSE
Figure 6: Density histogram of R. and SSE proportions random targets.
13

---

Published as a conference paper at ICLR 2024
B IMPLEMENTATION DETAILS
B.I CUSTOM MODEL DETAILS
The custom model was based on the Geometric Vector Perceptron- Neural Network architec- ture Jing et al. (202lb;a), Each protein was represented as fully connected graph. The node sealar features were sinusoidal positional embeddings of the residues' order in the chain concatenated with a normalised time step feature. We perform a message- passsing on a fully connected graph of Co carbons. Edge features were distances between nodes in terms of 16 Gaussian radial basis functions layers and the
and the unit vectors pointing along the edge. The model used 5 GVP-Convolun output of the network (the noise) had the centre of mass subtracted ensure equivariance.
B.2 NMA CALCULATIONS DETAILS
Nowadays quick and ready-to-use implementations of NMA are available, such as the Biotite ex- tension Springcraft (Kunzmann & Hamacher, 2018), which we used and rewrote into PyTorch differentiable version.
Before any equations of motion can be written, one must specify a foree field that describes interac- tions between residues. We use a Hinsen force-field (Hinsen & Kneller, 1999) with a cutoff of 16 A. For the choice of the strain target we perform the strain-energy calculation as described in Hinsen & Kneller (1999)
(d, d -
F. M R
(19)
R.yl
where E; is the energy of residue i, Ray is a vector that is the equilibrium separation between the residues j. ((R,,)) is the interaction constant, and d,, dy are the displacements of residues i, in the mode to be analyzed (here, the lowest non-trivial normal mode).
B.3 SAMPLING DETAILS
B.3.1 GVP
The generation was run with 250 reverse time steps, but at the last two generation time steps the noise in the update step set to O, since we found that this results in chain distances remaining closer to 3.8 A. It is a common practice to upscale the conditional term Va, In p(y|z> by some guidance scale (Dhariwal Nichol, 2021). Guidance scales for strain targets were time-dependent and equal to 200a, for strain targets and 400a; for random targets. Conditioning was switched on in the middle of the generation process. Since each sample had the same length as the protein from which the random or strain target was extracted, the potential differences observed in SSE cannot be attributed to differences in the protein length distributions.
B.3.2 GENIE
To take samples with the Genie model we used additional parameter 7) to downscale the noise in the reverse process, as recommended in the Genie publication (Lin & AlQuraishi, 2023). We set " = 0.4 which was shown achieve the best trade-off between designability and diversity. We found achieving the balance between the conditional parts of the score for dynamics and for the structure be the most problematic aspect to optimise. With the means of trial and error fine-tuning of the guidance scales, we arrived at different values per each hinge target. The guidance scales for the dynamics term and structure term were 3000 and 2500 for 6lys; 3000 and 2000 for 3adk; 2500 and 2000 for 2hhb. These constants were scaled by the time-dependent factors: for dynamics and VI.5 155-a a, for structure. Since we fine-tuned the guidance scales and their time dependencies, we skipped the /1 ae factor when converting score to noise. The conditioning was switched on in the middle the generation. Within the Kabsch algorithm in the structure conditioning, we found the translation vector and rotation matrix get the best alignment of the target residues with the residues" positions at t = 0, applied those transformations to the target and calculated RMSD. The translation and rotation were recalculated each 5 time steps.
14

---

Published as a conference paper at ICLR 2024
C Low QUALITY SAMPLES
The 'low quality' samples are those where the mean chain distance is outside [3.75, 3.85] A interval (proteins with mean chain distance more extreme being rare in nature (Voet & Voet, 2010)). Occa- sionally during conditional sampling coordinate values increases by orders of magnitude along the sampling trajectory, or even explodes NaN values. This divergence effect has also been observed in many conditional diffusion models (Lou & Ermon, 2023); in our case, this tends to happen when the conditioning pushes sample's s coordinates outside of the realm of observed samples for which the denoiser was trained. Finding the right balance between the NMA-loss driven part of the score and the unconditional part of the score is an important part of the conditioning process. These di- verged samples are also filtered out during the evaluation process, In the end, about 20% of samples in each category were filtered out due to low-quality chain distances.
D HINGE TARGETS DESCRIPTION
To extract targets with prominent hinge motion performed literature survey. We identi- fied the lysozyme (Gibrat & Go, 1990), adenylate kinase (AdK) (Tama & Sanejouand, 2001), and haemoglobin (Perahia & Mouawad, 1995) as three prominent examples of proteins with hinge-type motions for which the lowest normal mode is also known to correlate strongly with functional mo- tion. To extract the hinge motion, perform an anisotropic elastic network formulation of normal mode analysis with an invariant force field on alpha carbon atoms, using a distance cut-off of 13 A. The lowest non-trivial normal mode is then computed from the Hessian, and the 16 residues with the largest displacement components are extracted as motif's to scaffold with the targeted motion. The target motifs are shown in the top column of Fig. 1. For lysozyme and adenylate kinase hinges, the newly sampled backhones had length max(hinge residue order) + 10, and for the haemoglobin residue .odder) is the order of non non-consecutive hinge
residue residue_order) 20, where (hinge_residue__ residues in the original backbone. Since haemoglobin is larger than the maximal backbone length that can fit to genie, the haemoglobin hinge was modified the backbone order for all hinge residues was shifted down by 190 residues, additionally the number of residues between the hinge arms was decreased by 250 residues.
max (hinge (hinngereeiduueeeeeeeeeeeeeeeeee
E DESIGNABILITY IN DYNAMICS CONDITIONING VS STRUCTURE CONDITIONING
Since experimental verification of protein designs is time-consuming and expensive, the research community has developed silico methods to assess design success computationally. Many of them fall under the framework of so-called self-consistency metrics (Trippe et al., 2023). meaning that the designed structure is evaluated by predicting sequence for it via inverse folding models like ProteinMPNN (Dauparas et al., 2022), predicting the resulting structure via structure prediction methods like AlphaFold2 (Jumper et al., 2021) or ESMFold (Lin et al., 2022) and comparing this predicted structure the designed one via structural similarity metrics.
The most common computational design criteria are the following:
scTM > 0.5: the TM-score between the designed structure and the self-consistency pre- dicted structure as described above. With the scTM-score ranging from to 1, higher numbers correspond to an increased likelihood of the input structure being designable. A threshold of 0.5 is often chosen and the percentage of samples above this threshold is re- ported.
scRMSD < 2 A: The scRMSD metric is similar to the scTM metric, however instead of the TM-score the RMSD between the designed and predicted structure is calculated. It is a much more stringent criterion than scTM since RMSD is local metric that is more sensitive to small structural differences.
PLDDT > 70 and pAE < 10: Since both scTM and scRMSD rely structure prediction method like AlphaFold2 to be reliable metrics, confidence metrics of these models like PLDDT and pλE are used as additional metrics to ensure the reliability of self-consistency metrics. Low scRMSD and high pLDDT have been linked to the experiment success of designing the backbone (Bennett al., 2022).
15

---

Published as a conference paper at ICLR 2024
scTm score alone is good indicator of whether two structures are in the same fold and for that reason, it has been used in previous works for assessing the general sample quality (Trippe et al., 2023: Yim et al., 2023). However, more recent works such Genie (Lin & AlQuraishi, 2023) apply more stringent criteria of pI.DDT > 70, pAE < 10, scTM >> 0.5. In our experiments with hinge targets, if those additional requirements were incorporated, the proportion of samples meeting those criteria in joint conditioning dropped to 0.04 for 6lys, 0.15 for 3adk, and 0.0 for 2hhb. When we incorporated the last most stringent criterion that scRMSD 2 A, those proportions dropped to 0.00, 0.04 and 0.0. Further investigation revealed that the lack of confidence in ESMFold predictions is due to the difficulty structure conditioning. When only the dynamies conditioning was used (with the same guidance scale as when being part of the joint conditioning) the proportions of designable structures without scRMSD criterion were 0.6 for 6lys, 0.82 for 3adk, 0.52 for 2hhb, and with scRMSD criterion 0.41, 0.63 and 0.37 respectively.
To put these values into perspective, we note that low designability scores are not uncommon for the models tackling motif scaffolding problem. The current state-of-the-art model, RFDiffusion, has designability O, or close to O, for some of the more difficult functional site targets ((Watson et al., 2022), Supplementary Methods Table 10). Since targets were extracted from a flexible part of the protein, consist of discontinuous motifs and have not been used as targets in the literature elsewhere, it is difficult to assess what designability scores might be considered 'good" for those targets. Moreover, we note that the confidence metric of AF2/ESMFold might not be well suited for the assessment of the quality of the flexible regions. As observed in Bryant (2023), pLDDT is "good' metric if a single protein conformation is considered, however, it becomes less informative as alternative conformations are included. The regions with lower PLDDT tend be flexible regions with conformational changes, which might explain why proteins with hinge structure tend to have lower PLDDT.
F SCORE-NOISE EQUIVALENCE
For completeness, we provide short derivation of the score-noise equivalence
Vx, log (x Vx, log. (are: Varso, (1 T)
(20)
Vx, x4 log N (re: v VorTo (1 - az) = = Vai (5) 20
(21)
2(1 σze)
(x, v /0,
(x, v /a,200)
(22)
2(1 āt))
(1 az.)
(2t v (400))
Et
(23)
(1 (i))
v1 - αx
G ADDITIONAL SAMPLES
Green arrows are the displacements of conditioned residues in the sampled protein, purple arrows are the targets rotated to fit the green arrows best.
16

---

Published as a conference paper at ICLR 2024
(a) Conditional sample (NMA-loss 0.088)
(b) Unconditional sample (NMA-loss 0.620)
Figure 7: Comparison of the conditional and unconditional sample for the same strain target.
(a) Conditional sample (NMA-loss 0.224)
(b) Unconditional sample (NMA-loss 0.625)
Figure 8: Comparison the conditional and unconditional sample for the same random target.
H JOINT STRUCTURAL MOTIF & DYNAMICS CONDITIONING WITH THE IMPROVED GENIE MODEL
When using the original, guidance-based formulation for motif conditioning used in the main text, we found that motif conditioning continued to be primary difficulty. This made harder to perform and analyse NMA conditioning jointly with motif conditioning. because the NMA condition only makes sense for a reasonably well formed motif.
We therefore sought to improve motif conditioning by using Genie model that re-trained with the explicit motif conditioning, as proposed Didi et al. (2023).
H.1 TARGET DEFINITION
We investigate the following question: Can we design a new backbone, such that the functional motif and its key dynamical behavior, represented by the lowest non-trivial normal mode components of the motif in the target structure, are preserved?
To test out biologically relevant scenario we choose to model dynamically relevant segment spanning the two active site residues in hen-egg white lysozyme as target motif. Lysozyme was (2019); Brooks & Karplus (1985))
chosen as case study since during function (Bauer undergoes a well-studied hinge motion, which is well captured by lowest non-trivial mode in normal mode analysis. The motif is illustrated Fig. 10 and consists of 22 residues of the original structure (PDB: 61yz, 129 residues), including the active site residues GLU-35 and ASP-52 (Vocadlo et al. (2001); Held & van Smaalen (2014)). To obtain the NMA target for this motif, we perform an NMA with invariant force-fiele and with 13 Á distance threshold the C. backbone of the native protein (61y2) and extract the lowest non trivial normal mode displacements for the motif residues NMA target.
17

---

Published as a conference paper at ICLR 2024
Figure 9: More samples with joint conditioning. Left column: samples for the 6lys target. Middle column: for 3adk target. Right column: for the 2hhb target.
Figure 10: Target definition for the additional experiments and to illustrate a biological application. The target motif (red) chosen the single segment connecting the two active site residues (GLU-35, ASP-52) hen-egg white lysozyme (PDB:61yz). including two residues on either side of the active site. This results in target motif of 22 residues in length. The active site residues shown with side-chains, and the motif's position in the overall sequence is marked on the bottom bar.
H.2 MODELLING
Improved motif conditioning model We modify the unconditional Genie model (Lin & AlQuraishi, 2023) order to perform the conditional training, where the model is provided with the motif coordinates for some of the training examples. We add an additional conditional pair fea- ture network that takes the target motif coordinates and frames as input with zero-padding for all non-motif coordinates and frames. The features of this motif-conditional pair feature network are fused with the output of the original unconditional pair feature network genie via concatenation along the feature dimension, followed by linear projection down to the channel size of the uncondi- tional model. The remainder of the Genie model then proceeds unchanged. This minor architectural modification means conditional Genie network has 4. 4.162M parameters while the unconditional
18

---

Published as a conference paper at ICLR 2024
Genie network has 4.087M parameters (~ 1.8% fewer). The conditional Genie model was trained for 4'000 epochs on A100 GPUs (~~ 300 A100 hours in total). We stopped training at this point, as observed almost comparable performance the publicly available model weights (which were obtained after training for 50 000 epochs). We use these model for all the additional experiments in this section. The model is trained according to algorithm 5 in Didi al. (2023) and during train- ing the model is shown a conditional sample for 80% of the time and an unconditional one for the remaining 20%.
Guidance schedule While motif scaffolding is now explicitly built into the denoiser, we still need to condition on the NMA dynamics condition. We follow a reconstruction guidance (Chung et al., 2022b) approach with modulated step-function guidance schedule
{ 7o(11 oz) if t < faaa
(t)
(24)
if 2 sttttt
with guidance scale To and starting point tstant For tmax 1000, we fixed suat 500 (i.e. conditioning starts halfway through the reverse diffusion process) and identified To = 500 as an adequate guidance scale through logarithmic scan of to values. Similar to other work diffusion models for protein backbone generation, we reduce the noise scale by a factor = 0.4, which improves the quality of generated samples (Yim al., 2023) for motif-only as well as motif+NMA conditioning.
NMA loss The presence of a functional motif defines a reference coordinate system, namely the coordinate system in which the coordinates of the to-be-scaffolded motif are given in. Notably, this means that the normal mode displacements at the motif residues are also given in the motif's coordinate system. Any designed backbone should be invariant to translations, but equivariant to rotations of this coordinate system, which correspond to rotations of the motif and the associated displacement vectors.
To better comply with these symmetry requirements, we adapt the invariant loss INMA in Eq. 15 to make use of this reference coordinate system. Using the notation of the main text, yd E RRax and J.M E mR3 represent the target motif coordinates and sample motif coordinates for a motif of m residues respectively. Similarly, wM (y) E mg3 and w (x) E m2mx3 respectively refer to the matrix of displacement vectors in the lowest non-trivial normal mode for the target and the sample. The rotation matrix R(yM .) transforms the coordinate frame of the target motif ydd to that of x M With these definitions, the updated NMA loss for the additional experiments is
KMMA= = 21dircction (R(y x )o (v), M. (z)) + Imagnitudo (R(y x Jo MM))),, R. (x))
(25)
81
U2
Adirection (U1, U2)
= 1
cos (N1, 02)
(26)
|
|
Imagnitude (U1, U2) = |||||] ||
- 12
(27)
Here, v(y) and v(z) are understood as flattened vectors in 3m and therefore ldirection directly cap- tures the relative contributions of each residuc's displacement. The factor 2 was added to align the min and max ranges of the two components of (YMA- We obtain R.y 2x'h through differentiable implementation of the Kabsch alignment algorithm (Kabsch, 1976) and v(x) from differentiable implementation of NMA on the sampled backbone x, which is then subset to the motif coordinates, as in the main text. Guidance is then performed via
Ft < xt (t) KMMA (R(y .. (x:))o (v), M (fo(x)))))
(28)
with (xx))) indicating the current estimate of the denoised structure via Tweedie's 's formula (Rob- bins, 1956) as in reconstruction guidance (Chung et al., 2022b).
Evaluation pipeline The evaluation proceeds similarly as in the main body. For each C, Co-only backbone sample, we sampled 8 sequences with ProteinMPNN (Tsampling 0.1). In those sequences, the amino acid identities of the motif residues known from the lysozyme target were kept fixed, such that only the scaffold was predicted by ProteinMPNN, Each of the 8 sequences is then fe-dolded with ESMFold, and self consisteeyy scores (scNMA, scRMSD, scTM) are calculated with respect to the original backbone sample. The original backbone sample is then paired with the ESMFold-ed design
19

---

Published as a conference paper at ICLR 2024
that had the lowest scRMSD (out of 8 ESMFold designs). We deemed the structure designable if it met the criteria of scTM>0.5, scRMSD<2A, with confidence threshold of pLDDT>70, pAE<10 for ESMFold predictions, which aligns with definitions in prior work (Watson et al., 2022; Yim et al., 2023; Lin AlQuraishi, 2023). Moreover, we evaluate an additional motif scaffolding metric, ScMOTIF-RMSH which measures the RMSD between the motif residues in the designed structure (after sequence design & ESMFold) and the target motif.
H.3 RESULTS
motif-condltioned only (n=150)
motif + NMA-conditioned (n=43)
8
75
7
65
ScMOTIF- RMSD
PLDDT
60
-55
50
2
45
1-
0
8
10
12
14
0
2
8
10
12
14
SCRMSD
SCRMSD
Figure 11: scMOTIF-RMSD scRMSD for 150 motif-only and 43 motif+NMA samples (not filtered for designability criteria.. While the chosen target (two paired anti-paralle beta sheets con- nected to the end of helix) turns out to be difficult problem for the model, the best samples that achieve lowest ScMOTIF-RMSD and scRMSD values stem from joint motif+NMA conditioning, highlighting that the dynaamis--conditoonig can successfully work the motif scaffolding.
The analysis of the NMA loss of the Genie generated Ca-only backbone and scNMA-scor of the ESSFoold design confirmed that the dynamics conditioning indeed results in the C. backbones that match the target, however there is no clear direct correspondence scNMA-scores to the original backbone NMA-loss for the dynammiss-omdittonee samples. Surprisingly, conditioning improved upon making the structure conditioning inherent to the conditional Genie model, performing the motif scaffolding still challenging the model. In the remainder of this discussion, we call all the structure-one onlly conditioned samples motif-only, and all jointly structure and dynamics conditioned Not
found while motif
Discussion of the motif scaffolding success rate As first part of the evaluation, we calculated the proportion of Genie generated backbones of the motif-only and motif+NMA samples that meet 1λ. Out of 150 motif-only sam-
the designability criteria and have backbone design ples, is designable and has motif motif-RMSD<< 1Å. while 2 out of 43 motif+NMA ones are. More- that is RMSD to the motif structure after folding inferred sequences for
over, the scMOTIF-RMSS said backbone with ESMFold, does not achieve values lower than I for any of motif+NMA and motif-only conditioned samples. Figure 11 shows in detail how ScMOTIF-RMSD correlates with scRMSD.
We believe this is combination of (1) the limited training and capacity of our model and (2) the challenging nature of our target motif, which is a segment of 2 paired, anti- anti-parallel beta-sheets connected to the end of helix. To these points into context, scaffolding task for 300 A000 GPU-hours, compared to otate-of-theeart models such as RFDiffusion, which are trained for over 25 000s of GPU hours when considering the RosettaFold2 pre-training.
model trained for the motif-
20

---

Published as a conference paper at ICLR 2024
origNMA-loss distribution
Conditioning
35
motif-only (n=150)
motif+NMA (n=43)
30
25
Count
20
10
5
0.0 0.5
1.0
1.5
2.0
2.5
3.0
origNMA-loss
Figure 12: Distribution the NMA-loss of the designed backbones. The distribution of the mo- tif+NMA conditioned backbones is strongly enriched towards low NMA-loss values when compared to the distribution motif-only backbone samples. Only 2/150 (~ 1.3%) of motif-only backbones achieve origNMA-loss
0.5, opposed to 10/43 (~ 23%) for motif+NMA corresponding to a roughly 17x-fold enrichment, while achieving comparable motif scaffolding performance (c.f.
Fig. 11). Note that the bars were stacked to avoid overlapping bars from being invisible.
Yet, despite the significantly higher model capacity RFDiffusion (42 Mio. parameters) as well as the longer training, design success rates (according the criteria outlined above) of RFDiffusion can also at or below 1% for some challenging, contiguous functional motifs (e.g. targets 5WN9 or JJHW in the RFDiffusion benchmark in the supplementary material of Watson al. (2022)). It is therefore possible that the in-silico success rates for our model with lower capacity are be below the detection threshold for this particular motif scaffolding problem.
Nonetheless, the ESMFold designed backbones achieving the lowest ScMOTIF-RMSD and the lowest scRMSD belong to the motif+NMA conditioned group, which illustrates that our NMA- conditioning approach has no discemable negative impact on the designability of samples. We believe therefore still meaningful to gleam insights from this set samples, despite the chal- lenging nature of the motif-scaffolding for our chosen target.
Discussion of the scNMA-score The distribution of NMA-loss in the motif+NMA and motif-only Genie backbone samples is consistent with previous findings that the dynamics-condition leads to the targeted dynamics in the raw backbone (Figure 12). Only 2/150 (~ 1.3%) of motif- 0.5, opposed 10/43 (~ 23%) for motif+NMA corresponding to roughly 17x-fold enrichment, while achieving comparable motif scaffolding per- formance (c.f. Fig. 11). However, much of this benefit appears to disappear in the process in inverse-folding and the subsequent re- re-folding. Joint motif+NMA conditioning still increases the relative chance of obtaining a sample with low scNMA-scorf (3/43 7% of samples below 0.5) as compared to motif-only conditioning (3/ 150 ~ 2% below 0.5), roughly 3-fold, but the difference much less pronounced than for the NMA-loss of the designed backbone (original NMA-loss). Figure 13 shows the scNMA-loss distribution for the motif+NMA and motif-only ESMFold de- signs. The best sample with low original NMA-loss is therefore not guaranteed to have similarly low scNMA-score. The pipeline inverse-folding and re-folding has also surprising effect the motif-only samples. Samples with high values original NMA-loss are occasionally corrected to better NMA scores in the pipeline and match the targeted motif's dynamics better. Still, the in-
only backbones achieve origNMA-loss
21

---

Published as a conference paper at ICLR 2024
scNMA-score distribution
18
Conditioning
motif-only (n=150)
16
NoMif+NMA (n=43)
14
12
Count
10
8
6
4-
2
0.5
1.0
1.5
2.0
2.5
ScNMA-score
Figure 13: Distribution of the self-consistency NMA score (scNMA-score of the backbone samples against structures obtained from inferring a sequence for each backbone via ProteinMPNN and refolding it via ESMFold. Joint motif+NMA conditioning increases the relative chance of obtaining sample with low scNMA-score /3/4 7% of samples below 0.5) as compared motif-only conditioning (3/150 ~ 2% below 0.5) by roughly 3-fold, however the difference is much less pronounced than for the original NMA-loss. Again, the bars were stacked to avoid overlapping bars from being invisible.
troduction of the dynamics conditioning increases the relative chance of obtaining a sample with a low scNMA-score compared motif-only sampling. We leave the interesting question of how to retain high NMA-scores through inverse folding and re-folding pipelines as an interesting future work.
Lastly, we investigate how the ScNMA correlates with the scMOTIF-RMSD While the region where ScMOTIF-RMSDD<1 1 remains unachievable for both motif+NMA and motif-only samples as previously discussed, the best samples (scMOTIF-RMSD and scNMA as low as possible) from all samples taken belong to the dynamics-condttioned subset.
H.4 ADDITIONAL ALPHAFOLD2 DESIGNS
To give visual intuition of the scores introduced above, show the AlphaFold2 (AF2) designs of the Genie backbones one motif-only conditioned, and one motif+NMA conditioned. Those back- bones were deemed designable and close designable by ESMFold their scRMSD and DDDD spectively were 1.923 A, 73.6 for motif-only conditioned sample and 1.897 Á, 68.1 for motif+NMA conditioned sample. We repeated the inverse-folding and folding steps for these selected sam- ples with state-of-the-ar art AF2, and we computed the self-consistency scores again. The Ca-only backbones derived from the AF2 designs are presented the Figures and The displacement vectors in the lowest normal mode are attached to the points of the conditioned residues.
22

---

Published as a conference paper at ICLR 2024
scNMA-score v5. ScMOTIF-RMSD for samples with ScMOTIF-RMSD < 4
Canditioning
matif-only (n=29/150)
Motif-NMA 17/43
2.0
ScNMAsscore
1.5
1.0
1.5
2.0
2.5 ScMOTIF-RMA RMSD
3.0
3.5
4.0
Figure 14: scNMA-score ScMOTIF-RMSD for the motif+NMA and motif-only samples. We focus on samples with ScMOTIF-RMSD as for structures with larger differences the motif, the scNMA-scores likely become meaningless. As consequence, 17 out of 43 and 29 out of 150 motif+NMA and motif-only samples are shown the remaining are the outliers in the region ScMOTIF-RMSD> 4Å.
Figure 15: Backbone of an AF2 ro-delded sequence obtained via ProteinMPNN from motif+NMA conditioned raw backbone. Purple arrows are the displacements of the conditioned residues in the current structure, purple are the target rotated the motif's frame of reference. Arrows are scaled up for visual clarity. Scores obtained with folding with AF2: scRMSD= 1.49, PLDDT= 80.6, SMMOTIF-RMSD-2.00 Original NMA-loss= 0.087, www.M-MMMM-----0---------
23

---

Published as a conference paper at ICLR 2024
Figure 16: Backbone of an AF2 re-folded sequence obtained via ProteinMPNN from a motif-only conditioned raw backbone. Purple arrows are the displacements of the conditioned residues in the current structure, purple are the target rotated to the motif's frame of reference. Arrows are scaled up for visual clarity. Scores obtained with folding with AF2: scRMSD= 1.52, pLDDT= 81.6, ScMOTIF-RMSD-2.077 Original NMA-loss=1 188, ScNMA-score- 0.64.
24


Reviews:
[{"review_id": "DZCW5gJkv8", "reviewer": "Reviewer_FtGW", "summary": "Tha manuscript presents a method for incorporating dynamics information in diffusion probabilistic models for protein generation. The central idea is to enforce the fluctuations in the samples to match the lowest mode of oscilation as predicted by a normal mode analysis. The authors demonstrate the general applicability of the method by combining it in a post-hoc fashion with the unconditioned Genie model to produce proteins compatible with provided dynamical properties.", "questions": "Page 3, eq (4)\nAs far as I could see, z has not been introduced. Is this just a sample from a unit normal? Please clarify.\n\nPage 3, eq (6)\nIsn't there an \"x\" missing on the right hand side?\n\nPage 3. \"Related work on Diffusion Probabilistic Models in protein context.\"\nAt this stage in the paper, after the general background on diffusion models, I would have expected a section on the specific modelling tasks that you intend to solve in this paper. Instead, you jump directly to related work, mentioning e.g. amino acid point clouds, which have not been defined. One solution could be move the related work to appear later in the paper, and go direclty to \"3 Methods\", but even that section does not give a complete description of what you are modelling (it introduces y_D, but not what x is, and how the amino acid identities appear in the model). For intance, if x is purely backbone structure, does that include all backbone atoms, or only C_alphas?\n\nPage 3. \"K ∈ R 3N×3Nis interaction constants matrix\"\nWhat is an interaction constants matrix? Is it derived from a force field? (EDIT: I see you have some info on this later, but would be good to clarify this when first mentioning NMA).\n\nPage 4. \"generate a new protein\"\nWhat exactly do you mean by \"generating\" a protein. Are we talking about both sequence and structure, conditioned on sequence length?\n\nPage 4. eq (8)\nWhy is there no weighting factor (aka temperature) on the loss term? In other words, why would you expect this to be 1? \n\nPage 5, \"x_C_M \\in ... is the expected positions of conditioned residues at t=0 as sampling progresses\"\nThis sentence was confusing to me. Doesn't \"sampling progresses\" imply a \"t\" different from \"t=0\" (and at t=0, don't you know the position?). And in this case, shouldn’t this expected position be subscripted with a time t?\n\nPage 6. \"mean chain distance\"\nThis is not clearly defined. From the reference value of 3.8, I assume this is the average CA-CA distance. Does this imply that you model the protein only at C_alpha resolution? If so, I would have expected this to be stated as part of the original model specification.\n\nPage 7. \"sample novel proteins using\"\nAgain, are you sampling both structure and sequence here?\n\nPage 7. \"For each sample, we obtain 8 ProteinMPNN generated sequences with ESMFold (Lin et al., 2022) design for each\"\nThis seems to suggest that \"generating proteins\" means only backbone coordinates, such that you need to fill in sequences using ProteinMPNN. But this sentence should probably be rewritten, since it is unclear what \"with ESMFold (Lin et al., 2022) design for each\" means. As far as I know ESMFold predicts structures, which is also consistent with the following sentences - but what does \"design\" then mean here? This should be clarified.\n\nPage 7. \"Occasionally the conditional sampling will ‘blow up’, which has been generally observed in many conditional diffusion models (Lou & Ermon, 2023)\"\nDoes the rate of blowing up depend on the \"guidance scale\" used during sampling?\n\nPage 7. \"Figure 3 shows a pair of conditional and unconditional samples for one of the strain targets (additional sampled pairs are in Appendix E)\"\nCouldn’t this be quantified rather than verified through visual inspection on just a few targets?\n\n\n### Minor comments:\n\nPage 2, \"as well as we perform the visual inspection.\"\nSomething is wrong in this sentence. Rephrase.\n\nFigure 1, caption. \"Bottom row: corresponding proteins synthesised with Genie\"\nWhat does the word “corresponding” imply? Are they conditioned on something similar?\n\nFigure 1, caption. \"Arrows are not to scale with respect to the entire protein\"\nPerhaps make this statement more precise by saying that arrows have been scaled up for increased visual clarity.\n\nPage 3, \"by the equivalence ∇x...\"\nThis identify was not immediately apparent. I realize that it just requires a few steps , but for completeness it would be helpful if you could include the derivation of this in an appendix, or otherwise include a reference where this is done.\n\nPage 5. \"and expected displacements matrix v(x)\"\nI assume this means expected according to the NMA analysis, but perhaps state explicitly.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "To my knowledge, the presented method is novel. The manuscript is well written, and the paper appears technically sound. The conducted experiments are perhaps not as comprehensive as one might hope for, but they do demonstrate the basic premise of the approach. The problem addressed by this method is of fundamental importance, and any improvements in this area of research could therefore have significant impact.", "weaknesses": "While the paper reads well overall, there are parts where the clarity could be improved, in particular in the early description of the modelling task (what is being modelled), and in the technical description of the modeling approach (some variables undefined / details missing). The first point is in my view particularly important, because as it stands, it is difficult to read from the paper whether the method is generating: 1) protein sequence and structure, 2) protein backbone structure conditioned on sequence length, 3) protein structure conditioned on sequence, 4) protein backbone structure conditioned on nothing at all. It is to some extent possible to deduce these details from the conducted experiments, but I think this information should be stated more explicitly in the paper. Regarding technical details, these are more minor things that would make the paper easier to read. For details on both, see the questions below.\n\nRelated to the above, it would also benefit the paper if the authors briefly stated how they envision their approach to be useful in practice. If we have information about dynamics that we wish to impose on generated samples, we are presumably in a fairly constrained setting, where we wish to resample only parts of a protein. What are the advantages to conditioning a diffusion model on dynamics compared to for instance just using normal mode analysis (or coarse grained molecular dynamics) to generate a structural ensemble, and then use a model to predict amino acid identities conditioned on structure (i.e. some inverse folding model)?\n\nFinally, the empirical evaluation of the method could have been stronger. Some of the results are primarily based on qualitative comparisons by-eye, or otherwise restricted to very specific motions. Clarification is also needed for some of the experimental setup (see details below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v7kkJTx9EN", "reviewer": "Reviewer_M3zP", "summary": "The authors introduce a new problem for protein generative models of conditioning on the desired flexibility of parts of the protein. They develop a way to parameterize and model this flexibility and show that their conditioning can be applied on existing pre-trained models.", "questions": "I don't have any further questions, but it would be nice if the authors could comment on the weaknesses outlined above.\n\nA small note that in Section 3.1 you state that existing neural network approaches for fining matrix eigenvectors need re-training for each new matrix. In [1] there is a proposed architecture to generate sets of eigenvectors for graph Laplacians from a given distribution. While its a slightly different problem to what you were talking about, that architecture could potentially be used to generate eigenvectors for symmetric matrices without re-training as long as the training distribution matches the test matrices you feed.\n\n[1] Martinkus, Karolis, et al. \"Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators.\"\n\n### After Rebuttal\nI read all the reviews and author answers. While I find the overall problem interesting, I do tend to agree with the other reviewers that both the methodological novelty and rigorous evaluation is lacking. I will keep my score.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors introduce a previously unaddressed problem in protein generation of modeling structure flexibility and generating proteins with desired flexibility. Which biologically is a very important feature to model.\n\nThe introduced NMA loss is sensible and the results show that the proposed conditioning pipeline indeed works well.", "weaknesses": "While in general I liked the paper, one could say its a bit light on the content. The problem itself is novel and very interesting, but the solution is more or less the standard guidance framework with a new loss. Authors say that their approach could potentially be applied to other diffusion models, it would be nice to test at least a couple for this. Most validation is done by the same NMA loss that is used for guidance. So its unsurprising that guiding using a loss decreases it. So it would be nice to have more diverse validation metrics for the flexibility. Although admittedly I don't know what would be such better metrics. Maybe molecular dynamics literature has some suggestions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jPDn07IH6E", "reviewer": "Reviewer_E2Vz", "summary": "The authors adapt a protein diffusion model to capture protein movement with an adaptation of normal model analysis.", "questions": "“We present the corresponding SDE theory as a formal justification of our approach.” In the abstract, what is SDE? Stochastic Differential Equations? If so, say that and then put SDE in ().\n\n“Remarkably, Genie outperformed other models such as ProtDiff (Trippe et al., 2023), FoldingDiff (Wu et al., 2022) or FrameDiff (Yim et al., 2023), and remains comparable to RFDiffusion (Watson et al., 2022).” Why is this remarkable? Is it to the reader to determine if this is true? Should this evidence, however remarkable, be in an Appendix figure? Is RFDiffusion statistically significantly better?\n\n“ the mean chain distance that should be close to 3.8 A ;” Where does 3.8 come from?\n\nDo the values of NMA loss differ for strain or random targets? Do you expect them to? If so, this seems like a straightforward statistical quantity to calculate.\n\n“In the analysis of the remaining samples, we considered the distributions of NMA-loss and scTM-score (Figure 5).” Isn’t the second figure RMSD? If they are the same, please define this.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "I think it is good there is a clear biological system to focus on: movement, especially hinges.\n\nThe background on diffusion-based models was thorough and helpful.\n\nI also think transferring the normal mode analysis to a tractable invariant loss is interesting and a good extension of current methods. For now, I’ve seen many invariant losses based on internal coordinates, but this is good.", "weaknesses": "Generally, I think this paper has very weak evidence to support the method works. Much of the analysis is based of heuristics or \"by eye\", which is not sufficient. If this ambiguity is expected, I anticipate the authors should generate methods, statistics, experiments, or simulations that can further support or reject the proposed hypotheses in this paper.\n\nIn detail:\n\nFigure 1 is difficult to follow. \n- What are the proteins in the lower row? They don’t seem to be the same proteins as those in the respective column as the top row. Is it the same set of motifs?\n- If the arrows aren’t to scale to the entire protein, what are they to scale to?\n\nFigure 2 - What should I be looking for? What is the range of good or bad numbers? I would strongly prefer if the dynamics of some other calibrated system could be shown in these plots, as the units and density are quite difficult to interpret.\n\nFigure 3 - I’m not sure what I’m supposed to be looking for. What does a “bad” sample look like?\n\nThe filtering in section 5.1 is very ambiguous: “At the start, we filter out the ‘low quality’ samples whose mean chain distance is outside [3.75, 3.85]” What fraction were filtered out? “Occasionally the conditional sampling will ‘blow up’, which has been generally observed in many conditional diffusion models.” What does ‘blow up’ mean? Mathematically, can you define it, or is it all by eye?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nMswtvJvmS", "reviewer": "Reviewer_65X8", "summary": "The paper introduces a method for sampling protein structures with desired dynamic properties. This method builds on prior work in protein structure diffusions and computational analysis of protein dynamics. The primary contribution is a novel loss function for dynamics guidance. This loss function using normal mode analysis (NMA) to extract target displacement vectors per residue and then uses a differentiable NMA implementation to guide sampling towards the structures with displacements close to the targets. The authors present 27 conditional samples, which display loss values considerably lower than unconditional samples and self-consistency numbers to suggest that the sampled structures are designable.", "questions": "- How were the guidance scale values selected? Were they tuned for the protein structures you evaluated on (in which case they might be overfit to the tasks)? An independent evaluation model could also be useful here.\n\n- Why is guidance only turned on in the middle of sampling? This is not standard practice to my knowledge.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The method is clearly presented If given the details of how NMA is performed, for example in a public codebase, I feel like I could straightforwardly reproduce the described method and results. The initial results also seem promising. The objective is clearly being optimized and the validity checks suggest that the sampled structures are reasonable.", "weaknesses": "The methodological and experimental contributions are both relatively minor. \n\nMethodology:\n\nThe method combines classifier guidance on a Gaussian diffusion (Ho et al. 2020) with a NMA-derived loss function. Section 3.1 could be included in Section 2, as it is established and common to use a delta function centered at the denoiser's mean to perform guidance, and it is not unique to the paper. There also doesn't appear to be anything novel about the approach to NMA besides porting numpy code to pytorch. The contribution is the synthesis of NMA with classifier guidance, and the addition of a structure guidance term. \n\nExperimental:\n\nNovel methodology is not necessary for a significant contribution when a simple method can be convincingly shown to work well. This paper could be in this category, but I think the current results are fairly limited. Showing that guidance leads to lower loss values is more of a sanity check than a full result. Ideally the authors would demonstrate that the samples have the desired dynamics with a compelling independent evaluation. In this case, such an evaluation is obviously a challenging task, as it would require something like expensive molecular dynamics simulations or actual lab work, but establishing a compelling evaluation framework could be a very impactful contribution in itself. Likewise, it's a bit hard to know what scTM > 0.5 actually signifies, even though it has been used in prior work. Ultimately these structures would have to expressed as sequences. Are there compelling samples for those sequences such that these protein hinges could actually be created? It might be helpful to show that (1) the guidance method generalizes to the outcomes we really care about, not simply the proxy objective (2) there are high likelihood sequences that can (a) be expressed (b) have the desired dynamics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)