Improved motif-scaffolding with SE(3) flow matching
Authors: Jason Yim
Andrew Campbell
Emile Mathieu
Andrew Y. K. Foong
Michael Gastegger
Jose Jimenez-Luna
Sarah Lewis
Victor Garcia Satorras
Bastiaan S. Veeling
Frank Noe
Regina Barzilay
Tommi Jaakkola
Venue: Accepted by TMLR ()

Abstract: Protein design often begins with the knowledge of a desired function from a motif which motif-scaffolding aims to construct a functional protein around. Recently, generative models have achieved breakthrough success in designing scaffolds for a range of motifs. However, generated scaffolds tend to lack structural diversity, which can hinder success in wet-lab validation. In this work, we extend FrameFlow, an SE(3) flow matching model for protein backbone generation, to perform motif-scaffolding with two complementary approaches. The first is motif amortization, in which FrameFlow is trained with the motif as input using a data augmentation strategy. The second is motif guidance, which performs scaffolding using an estimate of the conditional score from FrameFlow without additional training. On a benchmark of 24 biologically meaningful motifs, we show our method achieves 2.5 times more designable and unique motif-scaffolds compared to state-of-the-art. Code: https://github.com/microsoft/protein-frame-flow

Published in Transactions on Machine Learning Research (07/2024)
Improved motif-scaffolding with SE(3) flow matching
Jason Yim
jyóm @csall mit. edu
Computer Science and Artificial Intelligence Laboretory Massachusetts Instätute of Tecknology
Andrew Campbell
campbell@stats.or.coo
Department of Statistics
University of Ouford
Emile Mathieu
ebm3324cam.ac..
Department Engineering
University of Cambridge
Andrew Y. K. Foong Microsoft Research A1fScience
andreufoony@@mieerssol.ccom
Michael Gastegger Microsoft Research Al4Science
megssteggerrmmieerssfff.com
José Siménez-Luna Microsoft Research A14Science
jjdneneslanm@nniervsss...ocm
Sarah Lewis Microsoft Research Al4Science
seruhheatss@miccroooft.ccmmm
Victor Garcia Satorras Microsoft Research A14Science
victoryarOmmicrooofll.com
Bastiaan Veeling Microsoft Research AIşScience
besveelingg@mierroofff.com
Frank Noé
franknoe@@micrroooft.oomm
Microsoft Research A14Science
Regina Barzilay
regina mát. edu
Computer Science and Articial Intelligence Laboratory Massachusetts Institute of Technology
Tommi S. Jaakkola
tommi Commiiassal.omm mit, edu
Computer Science and Articial Intelligence Laboratory Massachusetts Institute of Technology
Reviewed on OpenReview: https: // openreview. net/ forum? id- fo faine8nDOn

---

Published in Transactions on Machine Learning Research (07 /224)
Abstract
Protein design often begins with the knowledge of a desired function from a motif which motif-scafoolding aims to construct a functional protcin around. Recently, generative models have achieved breakthrough success in designing scaffolds for a range of motifs. However, generated scaffolds tend to lack structural diversity, which can hinder success in wet-lab validation. In this work, we extend FrameFlow, an SE(3) flow matching model for protein backbone generation, to perform motif-scaffolding with two complementary approaches. The first is molif amortization, in which FrameFlow is trained with the motif as input using data augmentation strategy. The second is motif guidance, which performs scaffolding using an estimate of the conditional score from FramcFlow without additional training. On a benchmark of 24 biologically meaningful motifs, we show our method achieves 2.5 times more designable and unique motif-scaffolds compared to state-of-the-art. Code: https: frame-ffoo
//github
1 Introduction
A common task in protein design is to create protcins with functional properties conferred through pre- specified arrangement of residues known as motif. The problem is to design the remainder of the protein, called the scaffold, that harbors the motif. Motif-scaffolding is widely used, with applications to vaccine and enzyme design (Procko et al., 2014; Correia et al., 2014; Jiang et al., 2008; Siegel et al., 2010). For this problem, diffusion models have greatly advanced capabilitics in designing new scaffolds (Wu et al., 2023; Trippe et al., 2022; Ingraham et al., 2023). While experimental wet-lab validation is the ultimate test for evaluating a scaffold, in this work we focus on improving performance under computational validation of scaffolds following prior works. In-silico success is defined as satisfying the designability! criteria which has been found to correlate well with wet-lab success (Wang et al., 2021). The current state-of-theeart REdiffusion (Watson et al., 2023), fine-tunes a pre-trained RosettaFold (Back et al., 2023) neural network with SE(3) diffusion (Yim et al., 2023b) and is able to successfully scaffold the majority of motifs in a recent benchmark. 2 However, RFdiffusion suffers from low scaffold diversity which can hinder chances of a successful design. Moreover, the large model size and pre-training used in RFdiffusion makes it slow to train and difficult to deploy on smaller machines. In this work, we present a lightweight and casy-to-train model with improved performance.
Our method adapts an existing SE(3) flow matching model, FrameFlow (Yim et al., 2023a), for motif- scaffolding. We develop two approaches: (i) motif amortization, and (ii) motif guidance as illustrated in Fig. 1. Motif amortization simply trains a conditional model with the motif as additional input when generating the scaffold. We use data augmentation to amortize over all possible motifs in our training set and aid in generalization to new motifs. Motif guidance relies on a Bayesian approach, using an unconditional FrameFlow model to sample the scaffold residues, while the motif residues are guided at each step to their final desired positions. An unconditional model in this context is one that generates the full protein backbone without distinguishing between the motif and scaffold. Motif guidance was described in Wu et al. (2023) for SE(3) diffusion. In this work, we develop the extension to SE(3) flow matching.
The two approaches differ in whether to use an conditional model or to re-purpose unconditional model for conditional generation. Motif guidance has the advantage that any unconditional model can be used to readily perform motif scaffolding without the need for additional lask-specific training. To provide a con- trolled comparison, we train unconditional and conditional versions of FrameFlow on a dataset of monomers from the Protein Data Bank (PDB) (Berman et al., 2000). Our results provide a clear comparison of the modeling choices made when performing motif-scafoodding with FrameFlow. We find that FrameFlow with
AA metric based on using ProteinMPNN (Dauparas et al., 2022) and AlphaFold2 (Jumper et al., 2021) to determine the quality of a protein backbone. First introduced in REdiffusion as a benchmark of 24 single-chain motils successfully solved across prior works published.
2

---

Published in Transactions on Machine Learning Research (07/2024)
Motif amortization
Generation
Motif
Scaffold
Corruption
Noise = 0
Data
Motif guidance
t= 1
Figure We present two strategies for motil-scaffoldingg Top: motif amortization trains a flow model condition on the motif (blue) and generate the scaffold (red). During training, only the scaffold is corrupted with noise. Bottom: motif guidance re-purposes flow model that is trained to generate the full protein for motif-scaffolddng, During generation, the motif residues guided to reconstruct the true motif at while the flow model will adjust the scaffold trajectory to be consistent with the motif.
both motif amortization and guidance surpasses the performance of RFdiffusion, as measured by the number of structurally unique scaffolds that pass the designability criterion.
This work is structured follows. Sec. 2 provides background on SE(3) flow matching. We present our main contribution extending FrameFlow for motif-scaffolding in Sec. 3. We develop motif amortization for flow matching while motif guidance, originally developed for diffusion models, follows after drawing connections between flow matching and diffusion models. Next we discuss related works Sec. 4 and present empirical results Sec. 5. Our contributions are the following:
We extend FrameFlow with two fundamentally different approaches for motif-scaffolding: motif amor- tization and motif guidance. We are the first to extend conditional generation techniques with SE(3) flow matching and apply them to motif-scaffolding. With all other settings kept constant, we perform a empirical study of how each approach performs.
On benchmark of biologically meaningful motifs, show method can successfully scaffold 20 out of 24 motifs in the motif-scaffolding benchmark which is equivalent to previous stato-of-tho-art while achiev- ing 2.5 times more unique, designable scaffolds. Our results demonstrate the importance of measuring diversity to detect mode collapse.
2 Background
Flow matching (FM) (Lipman et al., 2023; Albergo et al., 2023) is simulation-free method for training continuous normalizing flows (CNFs) (Chen et al., 2018). CNFs are deep generative models that generates data by integrating an ordinary differential equation (ODE) over a learned vector field. Recently, flow matching has been extended Riemannian manifolds (Chen & Lipman, 2023), which we rely on to model protein backbones via. the local frame SE(3) representation. Sec. 2.1 gives introduction to Riemannian flow matching. Sec. 2.2 then describes how SE(3) flow matching is applied to protein backbones.
2.1 Flow matching on Riemannian manifolds
On manifold M, CNF se(-) M + M is defined via an ODE along time-dependent vector field (ee.t):M. R T.M where T.M is the tangent space of the manifold at E M and time is t [0, 1]:
dt (20) = (20) t), @o(2o) = 2g-
(1)
3Te number unique scaffolds defined the number structural clusters. See Sec. 5.1
3

---

Published in Transactions on Machine Learning Research (07/2024)
Starting with 2u Po from an casy-to-sample prior distribution Po: simulating samples according to Eq. (1) induces new distribution referred as the push-forward Pt = [é One wishes to find a vector field v such that the push-forward Pt=1 = [ot=1]+Po (at = 1) matches the data distribution P1- Such a vector field v is in general not available in closed-form, but can be lcarned by regressing conditional vector fields u(z,, 4/21) = _ 2 where 2; = Φe (=0l=1) interpolates between endpoints = ~ Po and 21 ~ P1- A natural choice for 2, is the geodesic path: 2, = exp:. (t log. (21)), where exp=, and logz, are the exponential and logarithmic maps at the point 20. The conditional vector field takes the following form: u(z,,|zz) f|21) = logz, (21)/(1 t). The key insight of conditional flow matching (CFM) (Lipman al., 2023) is that training neural network i to regress the conditional vector field is equivalent to learning the unconditional vector field v. This
corresponds to minimizing
[1 [1u(2, t|2n) 6(2,, +]][
L= (e.te
where (t: [0. 11) is the uniform distribution for [0,1] and I1-113 the norm induced by the Riemannian metric g : TM x TM + R. Samples can then be generated by integrating the ODE in Eq. (1) with Fuler steps using the learned vector field 0 in place of v.
2.2 Generative modeling on protein backbones
The atom positions of cach residue in protein backbone can be parameterized by element T SE(3) of the special Euclidean group SE(3) (Jumper et al., 2021; Yim et al., 2023b). We refer to T = (r, x) as (local) frame consisting of a rotation r E SO(3) and translation vector æ E R3. The protein backbone is made of N residues, meaning it can be paramoterized by N frames denoted as T bold face to refer to vectors of all the residues, superscripts to refer to residue indices, and subscripts refer to time. Details of the SE(3)* backbone parameterization can be found in App. B.1.
T(N) E SE(3)" We use
We use SE(3) flow matching to parameterize a generative model over the SE(3) representation of protein backbones. The application of Riemannian flow matching to SE(3) was previously developed in Yim et al. (2023a); Bose et al. (2023). Endowing SE(3) with the product left-invariant metric, the SE(3) manifold effectively behaves as the product manifold SE(3) = SO(3) x R (App. D.3 of Yim et al. (2023b)). The vector field over SE(3) then be decomposed (SE(3) (. (u (o, neural network parameterize the learned vector fields,
(,,t), U80(3) (t,t)). Our goal is train
(n) (T.) ot. (n) (T,, t)
80(3) (T.,t) log, (n) (f(1) (Tx))
(3)
1
1-t
The outputs of the neural network are denoised predictions a(n)) and p(m) which are used to calculate the vector fields in Eq. (3). The loss becomes
CSE(3) E || ([T - Van(3) (T t) |
(4)
E [| (Xe, t t(x, - &x(TT ( I1 + (T,, - V5o (To [Om)
(5)
where the expectation is taken over U(: [0, 1]), Pı (TT), (To). We have used bold-face for collections of elements, i.c. v(-)
(00-..,,6N)))) (.)- Our prior is chosen as (T ({S((3)) N( T2),, where U(S((3)) is the uniform distribution over SO(3) and (V0 is the isotropic Gaussian where samples are centered to the origin. Details of SE(3) flow matching such as architecture and hyperparameters closely follow FrameFlow (Yim et al., 2023a), details of which are provided in App. B.2.
3 Motif-scaffolding with FrameFlow
We describe our two strategics for performing motif-scaffolding with the FrameFlow model: motif amor- tization (Sec. 3.1) and motif guidance (Sec. 3.2). Recall the full protein backbone is given by T =
Unnortunately the meaning of "conditional" is overloaded. The conditionals will be clear from the context.
4

---

Published in Transactions on Machine Learning Research (07/202) 2024)
Motif Scaffold
1. Sample protein from dataset
2. Sample number and location of motif(ss.
3. Sample motif lengths.
4. Corrupt remaining residues (scaffold).
Figure 2: Motif data augmentation. Each protein in the dataset does not with pre-defined motif-scaffold annotations. Instead, we construct plausible motils at random to simulate sampling from the distribution of motifs and scaffolds.
(TU,T ..., , SE(3)* The residues can be separated into the motif TM (T()) (.., T( of length & where fi...... { f1....,NY .N are motif residue indices, and the scaffold T all the remaining residues, such that T TM U UTS The task can then be framed as the problem of sampling from the conditional distribution (T TM).
3.1 Motif amortization
We train a variant of FrameFlow that additionally takes the motif as input when generating scaffolds (and keeping the motif fixed). Formally, we model motif-conditioned CNF via the following ODE,
dt 01 (T5 = u( "), & (T
To.
(6)
The flow oi transforms a prior density over scaffolds along time, inducing a density pe( p ( We use the same prior as in Sec. 2.2: MM conditional vector field ( TM) where T is defined by interpolating along the geodesic path,
(TT). FrameFlow is trained to predict the
Ts exprę (loggg (T)) The implication is that u is conditionally independent of the motif TMM given TS. This simplifies our formulation to u(T, S TM) uTT ( that is defined in Sec. 2.2. However, when we learn the vector field, the model needs to condition on TM since the motif placement TM contains information on the true scaffold positions TTS. The training loss becomes,
USs S {/T - SE)(3) ( T |
SE(3)
(7)
where the expectation is taken over ((t:; (0, IJ), PTT Pi T po (T)) The above expectation requires access to the motif and scaffold distributions, P(T and (T (T), during training. Future work can look into incorporating known motif-scaffolds such as the CDR loops on antibodies (Dunbar et al., 2014). While some labels exist for which residues correspond to the functional motif, the vast majority of protein structures the PDB do not have labels. We instead utilize unlabeled PDB structures to perform data augmentation (see Sec. 3.1.1) that allows sampling a wide range of motifs and scaffolds.
To learn the motif-conditioned vector field ú,, use the FrameFlow architecture with 1D mask as ad- donll input with a 1 at the location of the motif and 0 elsewhere. To maintain (E(()))quivaaiann) we zero-center the motif and initial noise sample from Po (TS Zero-centering the motif also prevents the model from using the motif offsct from the origin to memorize scaffold locations which helps genoralization.
3.1.1 Data augmentation.
The flow matching loss from Eq. (7) involves sampling from M(T and pı (T) which do not have access to, but can be approximated using unlabeled structures from the PDB. Our pseudo-labeled motifs and scaflolds are generaled as follows (also depicted in Fig. 2). First, a protein structure is sampled from the PDB dataset. Second, a random number of residues are selected to be the starting locations of each motif. Third, additional residues are appended onto cach motif thereby extending their lengths. The length of each motif is randomly sampled such that the total number of motif residues is between Ymin and Tmax percent of all the residues. We Tmin 0.05 and Tmax 0.5 to ensure at least few residues used the motif
5

---

Published in Transactions on Machine Learning Research (07/2024)
but not more than half the protein. Finally, the remaining residues are treated as the scaffold and corrupted. The motil and scaffold are treated as samples from P(TM) and pı (T (TM) respectively. Importantly, each protein will be re-used on subsequent epochs where new motifs and scaffolds will be sampled. Our pseudo motif-scaffolds cover a wide range of sccnarios that cover multiple motifs of different lengths.
(TT/TMM)
The lack of functional annotations the PDB requires training over all possible motíf-scaffold annotations to handle new scenarios our method may encounter in real world scenarios. In our experiments, we evaluate how this data augmentation strategy transfers to real motiffscaffolding tasks. A similar strategy is used in image infilling where image based diffusion models are trained to infill randomly masked crops of images to approximate real image infilling scenarios (Saharia et al., 2022). Motif-scaffolding data augmentation was mentioned in RFdiffusion but without algorithmic detail. Since RPdiffusion does not release training code, we implemented our own data augmentation algorithm in Algorithm 1.
3.2 Motif guidance
We now present an alternative Bayesian approach to motif-scaffolding that does not involve learning a motif- conditioned flow model. As such it does not require having access to motifs at training time, but only at sampling time. This can be useful when an unconditional generative flow model is already available at hand and additional training is too costly. The idea behind motif guidance, first described as a special case of TDS (Wu et al., 2023) using diffusion models, is to use the desired motif TM bias the model's generative trajectory such that the motif residues end up in their known positions. The scaffold residues follow trajectory that create a consistent whole protein backbone, thus achieving motif-scalfokding.
The key insight comes from connecting flow matching to diffusion models to which motif guidance can be applicd. The following ODE describes the relationship between the vector field & in flow models - Iearned by minimizing CFM objective in Eq. (5) and the Stein score Vlogp,
AT, = ((T,, [[(Tr,))- 1 (((())Voogp((TT) dt.
(8)
The gradient is taken with respect to the backbone at time t which we omit for brevity, i.e. V Vr. Eq. (8) shows the ODE used to sample from flow models can be wrillen as the probability flow ODE used in diffusion models (Song et al., 2020) with f and as the drift and diffusion coefficients. The derivation of Eq. (8) requires standard linear algebra and calculus for our choice of vector field (sce App. D).
Our goal is to sample from the conditional (T)) from which we can extract (TSITTM. The benefit of Eq. (8) is we can manipulate the score term to achieve this goal. We modify the above to be conditioned on the motif T followed by application of Bayes rule where V (T TM) V po (T,) + + (T
dT.
((T t) 2 ,o(() 2 Vlg (T.ITM)) dt
(9)
1 [(TT,.)) {0( Vlogpt V (T (T:)) dt
(Se(s)(Tr,)) - (t)2
Vlogp.(T (T)) dt.
(10)
unconditional pred.
guidance term
We can interpret Eq. (9) as doing unconditional generation by following VSE(3) (T, while V (Tog (T,) guides the noised residues so as to be consistent with the true motif. Doob's 's H-transform ensures Eq. (9)
will sample from M) (Didi et al., 2023). The conditional score (TM (T,) is unknown, yet it can be approximated by marginalising out T, and using the neural network's 's denoised output (Song et al., 2022; Chung al., 2022; Wu et al., 2023),
pr (TMMT TT) = p M Detee (T,)dT
(11)
p (TT 16 (T.) (Tr)dT, PTTMITT (T+)).
(12)
6

---

Published in Transactions on Machine Learning Research (07/2024)
We now have the choice to define the likelihood in Eq. (12) to have higher probability the closer it is to the desired motif:
p TT (T+)) x exp (-11x (T.)) exp (-11r - ( (Tx)
[5(3) (T.))
(13)
which is inversely proportional to the distance from the desired motif. Following SE(3) flow matching, Eq. (9) becomes factorized into the translation and rotation components. Plugging p(T M ITt (T,)) in Eq. (9), we arrive at the following ODE we may sample p p(T)TTM) from
Translations: dx; (T,, + (x) || - x (TT.))//uu dt.
(14)
Rotations: dr, (socs)) (T,, + 1 |rM - (T:) /w] dll
(15)
We is a hyperparameter that controls the magnitude of the guidance towards the desired motif which we set to w/ = (1 /)2 + (1 +)²) as done in Poklc et al. (2023); Song et al. (2021). While different choices of g()) are possible, Pokle et al. (2023) proposed to use g(() (1 f)/l with the motivation that this matches the diffusion coefficient for the diffusion SDE that matches the marginals of the flow ODE. For completeness, provide the proof for g(t) in App. D. A similar calculation is non-trivial for SO(3), hence use the same g(t) as a reasonnable heuristic and observe good performance as done in (Wu et al., 2023).
4 Related work
Conditional diffusion and flows. The development of conditional generation methods for diffusion and flow models is an active area research. Two popular diffusion techniques that have been extended to flow matching are classificr-free guidance (CFG) (Dao al., 2023; Ho Salimans, 2022; Zheng et al., 2023) and reconstruction guidance (Pokle al., 2023; Ho al., 2022; Song al., 2022; Chung et al., 2022). Motif guidance is application of reconstruction guidance for motif-scaffolding. Motif amortization is most related to data-dependent couplings (Albergo et al., 2023), where flow is learned with conditioning of partial data.
Motif-scaffolding. Wang et al. (2021) first formulated motif-scaffolding using deep learning. SMCDiff (Trippe et al., 2022) was the first proposed diffusion model for motif-scaffolding using Sequential Monte Carlo (SMC). Twisted Diffusion Sampler (TDS) (Wu al., 2023) later improved upon SMCDiff using reconstruction guidance for each particle in SMC. Our motif guidance method follows from TDS (with one particle) by deriving the equivalent guidance vector field from its conditional score counterpart. RFdiffusion (Watson et al., 2023) fine-lunes pre-trained neural network with motif-conditioned diffusion training, Our approach in principle follows RFdiffusion's 's diffusion training, but differs in (i) using flow matching, (ii) not relying expensive praitngin and (iii) uses a 3x smaller neural networks. Didi et al. (2023) provides a survey of structure-basee based motif-scaffolding methods while proposing Doob's h-transform for soafiss-caffolddng. FvoDiff (Alamdari al., 2023) differs using a sequence-based diffussion model that performs motif-scaffolding with language model-style masked generation but performance falls short of REdiffusion and TDS.
5 Experiments
In this section, report the results of training FrameFlow for motif-scaffolding Sec. 5.1 describes training, sampling, and metrics. Our main results on motif-scafolding are reported in Sec. 5.2 on the benchmark introduced in REdiffusion. Additional motif-scaffolding analysis is provided in App. G.
5.1 Set-up
Training. We train two FramcFlow models. as described in Sec. 3.1 with data augmentation using perrpaaaaneerss: 7min = 0.05 so the motif is never
is trained with motif amortization
Flow uses 16.8 million parameters compared RFaiffusion's 59.8 million.

---

Published in Transactions on Machine Learning Research (07/2024)
Number of designable scaffolds
Method
75-
hamefooocanotiaation ardillusion
Count
50
18CF
6ESR, med
2KLB
BESR, long
68X2, long
IYCR
STPH
6EXZ, med
4ZYP
ceen short
STRV long
3IXT
STRV med
7MRK 85
7MRX 120
1PRW
STRV. short
7MRX 60
sexz, short
siUs
sYul
SWNB
4/HW
10jG
Number of unique designable scaffolds (clusters)
Count
20
1BCF
GEaR man
GEeR long
6EXZ long
IYCR
STPH
GEXZ, med
AZYP
6EGR, short
STAV, Jong
3IXT
STRV med
7MRX 85
TNRX, 128
1PRW
STRV short
7NIXX 60
6EXZ short
SIUs
syul
SWN9
ajlw
201G
Motif
Figure 3: Motif-scaffolding results. Top plot: RFdiffusion achioves the most designable scaffolds amongst all methods in 9/24 test. motifs compared to woameellow-mmrtizaalleoo 7/24 and TDS' 6/24; 2/24 are ties. Bottom plot: However, we observe that RFdiffusion produces the highest number of unique designable scaf- folds for only 2 out of the 24 test motifs. Therefore, previous approaches that only mcasure designability (top plot) may be misleading since those generative models that may have the best designability can also be repeatedly sampling similar scaffolds. This demonstrates the need to measure diversity alongside designabil- ity and use the number of unique designable scaffolds as the metric of success.
degenerately small and Ymax = 0.5 to avoid motif being the majority of the backbone. to be used in motif guidance, is trained unconditionally on full backbones. Since unconditional generation is not our focus, we leave the unconditional performance to App. F where we see the performance is slightly worse than REdiffusion as we will see, the motif-scaffolling performance better. Both models are trained using the filtered PDB monomer dataset introduced in FrfmeDiff. We use the ADAM optimizer (Kingma & Ba, 2014) with learning rate 0.0001. We train each model for 6 days A6000 NVIDIA GPUs with dynamic batch sizes depending on the length of the proteins in each batch technique from FrameDiff.
Sampling. We use the Euler- Marlyyaa integrator with 500 timesteps for all sampling. Following the motif-scaffolding benchmark proposed in REdiffusion, we sample 100 scaffolds for each of the 24 monomer m For cach motif, the method must sample novel scaffolds with different lengths and different motif locations along the sequence. The benchmark measures how well a method can generalize beyond the native scaffolds for set of biologically important motifs.
Hyperparameters. Our hyperparameters for neural network architecture, optimizer, and sampling steps all follow the best settings found in FrameFlow (Yim et al., 2023a). We leave hyperparameter search as a future work since it is not the focus of this work.
5.2 Motif-scaffolding results
Baselines. We consider REdiffusion and the Twisted Diffusion Sampler (TDS) baselines. RFdiffusion's 's performance is reported based on their published samples. TDS reported motif-scaffolding results with arbitrary scaffold lengths that deviated the benchmark. Therefore, we re-ran TDS with their best settings using k = 8 particles on the RFdiffusion benchmark. We refer to Trameeflow-amorizaaiin as our results with motif amortization while FrameFlow-guidance uses motif guidance.
Metrics. Previously, motif-scaffolding was only evaluated through samples passing designability (Des.). For description of designability see App. E. Within the set of designable scaffolds, we also calculate the
6Te benchmark has 25 motifs, but the motif 6VW1 involves multiple chains that FlameFlow cannot handle.
8

---

Published in Transactions on Machine Learning Research (07/202)
Motif: 1QYS
Motif: 1YCR
Motif: 5TPN
FramPllo-annorizattin diversity. In blue is the motif while red is the scaffold. For each motif (IQJG, IYCR, 5TPN), we show FramePlow-amorrization can generate scaffolds of different lengths and various secondary structure elements for the same motif. Each scaffold is in a unique cluster to showcase the samples' structural diversity.
Figure 4:
diversity (Div.) as the number of structurally unique clusters. This is crucial since designability can be manipulated to have a 100% success rate by always sampling the same scaffold with trivial changes. In real world scenarios, diversity is desired to gain the most informative feedback from expensive wet-lab experiments (Yang al., 2019). Thus diversity provides an additional data point to chock for mode collapse where the model is sampling the same scaffold repeatedly. Clusters are computed using MaxCluster (Herbert & Sternberg, 2008) with TM-score threshold set to 0.5.
Benchmark. Fig. shows how each method fares against each other in designability and diversity each motif of the motif-scaffolding benchmark. While it appears RFdiffusion gets lots of succossful scaffolds, the number of unique scalTolds is far lower than both our FrameFlow approaches. TDS achieves lower designable scaffolds on average, but demonstrates strong performance on small subset of motifs. There are some motifs that only REdiffusion can solve (7MRX_ 85, 7MRX 128) while FrameFlow is able to solve cases REdiffusion cannot (IQJG, 4JHW, 5YUT).
Table 1: Motif-scaffolding aggregate metrics
Method
Solved (t)
Div. (1)
Speed (4)
PrameFlow-amons
20
353
18s
FrameFlow-ghin
20
192
18s
RFdiffusion
20
141
50s
TDS
19
217
117s
Tab. 1 provides the number of motifs each method solves - which means at least one designable scaffold is sampled and the number of total designable clusters sampled across all motifs. Here we see each method can solve 19-20 solves motifs, but FrameFlow-amortizatee can achieve nearly double the number of unique scaffolds (clusters) as REdiffusion.
outperforms FrameFlow-guie Flow-guidacc on diversity. A potential reason for the improved diversity is the use of SE(3) flow matching in the unconditional model whereas TDS uses SE(3) diffusion (Yim et al., 2023b). Bose al. (2023) found SE(3) flow matching to provide far better designability and diversity than its diffusion counterpart. Empirically, it is known flow matching outperormm diffusion Riemannian manifolds (Chen & Lipman, 2023).
In the last. columm we give the number of seconds to sample a length 100 protein on A6000 Nvidia GPU with each method. Both FrameFlow methods are significantly faster than RFdiffusion and TDS. TDS is notably slower since its run time scales with its number of particles. We conclude that matches REdiffusion and TDS on the number of solved motifs while achieving much higher diversity and faster inference.

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Training data secondary structure
Pramellovaaooriiaaiion secondary structure
REdiffusion secondary structure
Strand composition
Strand composition
0.4
0.4
0.4
0.3
9:3
0.2
02
Strand
0.2
0.1
9.1
9.0
64
1.
Helix composition
composition
composition
Figure 5: Secondary structure analysis. 2D kernel density plots of secondary structure composition of and RFdiffusion. Here we see RFdiffusion tends to
designable motif-scaffolds from mostly generate helical scaffolds while Soaaeeloo-amortization gets much more scaffolds with strands.
Diversity analysis. To visualize the diversity of the scaffolds, Fig. 4 shows several of the clusters for motifs 1QJG, 1YCR, and 5TPN where Framel can generate significantly more clusters than RFdiffusion. Each scaffold demonstrates wide range of secondary structure elements across multiple lengths. To quantify this in more depth, Fig. 5 plot. the helical and strand compositions (computed with DSSP (Kabsch & Sander, 1983)) of designable motif-scaffolds from saaumeeloow-mmrtiaatteoe FrameFlow-amotzaatee
compared to RFdiffusion. We see achieves a better spread of sccondary structure components than RFdiffusion. A potential reason for RFdiffusion's overall lower diversity is due to its lack secondary structure diversity favoring to sample mostly helical structures. App. G provides additional analysis into the FrameFlow motif-scaffolding results. We conclude than Rddifuusion.
achieves much more structural diversity
6 Discussion
In this work, we present two methods building on FrameFlow for tackling motif-scaffolding These methods be used with any flow-based model. First, with motif-amortization we adapt the training of FrameFlow to additionally be conditioned on the motif in effect turning FrameFlow into conditional generative model. Second, with motif guidance, we use an unconditionally trained FrameFlow for the task of motif- scalfolding though without any additional lask-specific training. We empirically evaluated both approaches, wwwneeellwwmmmrtiiae
and loo--elwwwe,,,e,e, on the motif-scaffolding benchmark from REdiffusion where we find both methods achieve competitive results with state-of-the-art methods. Moreover, they are able sample more unique scaffolds and achieve higher diversity. is important note amortization and guidance complementary techniques. Amortization outperforms guidance but requires conditional training while guidance performs worse due lo approximation error in Eq. (12) from using unconditional model in conditional task. We stress the need to report both success rate and diversity to detect when a model suffers from mode collapse. Lastly, we caveat that all our results and metrics are computational, which may not necessarily transfer to wet-lab success.
unconditional flow models without further training. Guidance generally
Future directions. We have extended FrameFlow for motif-scaffolding: further extensions include binder, enzyme, and symmetric design all which REdiffusion can currently achieve. For these capabilities, we require extending FrameeFlow to handle multimeric proteins. While motif guidance does not outperform motif amortization, it is possible extending TDS to flow matching could close that gap. Related to guidance, one could explore conditioning mechanisms control properties of the scaffold such as its secondary structure. We make use of hcuristic for Riemannian roconstruction guidance that may be further improved. Despite our progress, there still remains areas of improvement to achieve success in all 25 motifs in the benchmark.
References
Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Alex Xijie Lu, Nicolo Fusi, Ava Pardis Amini, and Kevin K Yang. Protein generation with evolutionary diffusion: sequence all you need. bioRrie, pp. 2023 09, 2023.
10

---

Published in Transactions on Machine Learning Research (07/2024)
Michael S Albergo, Mark Goldstein, Nicholas M Boffi, Rajesh Ranganath, and Eric Vanden Vanden-Eijnden. Stochas- tic interpolants with data-dependent couplings. arXiv preprint or
2023.
Minkyung Baek, Ivan Auishchenko, Ian Humphreys, Qian Cong, David Baker, and Frank DiMaio. Efficient and accurate prediction of protein structure using rosettafold2. bioRriu, pp. 2023 05, 2023.
Nathaniel R Bennett, Brian Coventry, Inna Goreshnik, Buwei Huang, Aza Allen, Dionne Vafeados, Ying Po Peng, Justas Dauparas, Minkyung Back, Lance Stewart, et al. Improving de novo protein binder design with deep learning. Nature Commumications, 14(1):2625, 2023.
Helen M Berman, John Westbrook, Zukang Feng, Gary Gilliland, Talapady N Bhat, Helge Weissig, Ilya N Shindyalov, and Philip E. Bourne. The protein data. bank. Nucleic acids research, 28(1):235-22) 242, 2000.
Avishek Joey Bose, Tara Akhound-Sadege Kilian Fatras, Guillaume Huguet, Jarrid Rector- Brotor- Cheng- Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael Bronstein, and Alexander Tong. Se (3)- stochastic flow matching for protein backbone generation. arXie preprint urXiv:2310. 02391, 2023.
Ricky T. Q. Chen and Yaron Lipman. Riemannian Flow Matching on General Geometries, February 2023.
URL htttp///rxiit http:////axvvv..//a/aa.sss wttp///32....0.... 03660.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances neural information processing systems, 31, 2018.
Hyungjin Chung, Jeongsol Kim, Michael T Mecann, Mare L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv: 14687, 2022.
Bruno Correia, John T Bates, Rebecca J Loomis, Gretchen Baneyx, Chris Carrico, Joseph Jardine, Peter Rupert, Colin Correnti, Oleksandr Kalyuzhniy, Vinayak Vittal, Mary J Connell, Eric Stevens, Alexandria Schroeter, Man Chen, Skye Macpherson, Andreia M Serra, Yumiko Adachi, Margaret Holmes, Yuxing Li, Rachel E Klevit, Barney S Graham, Richard T Wyatt, David Baker, Roland K Strong, James E Crowe, Jr, Philip R Johnson, and Willian R Schief. Proof of principle for epitope-focused vaccine design. Nature, 507(7491):201-2000 2014.
Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching latent space. ar preprinl WwXiv:2307.0869. 2023.
J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte, L. F. Milles, B. M. Wicky, A. Courbet, R. J. de Haas, N. Bethel, P. J. Y. Leung, T. F. Huddy, S. Pellock, D. Tischer, F. Chan, B. Koepnick, H. Nguyen, A. Kang, B. Sankaran, A. K. Bera, N. P. King, and D. Baker. Robust deep learning-basse protein sequence design using ProteinMPNN. Science, 37865515) 56, 2022.
Kieran Didi, Francisco Vargas, Simon V Mathis, Vincent Dutordoir, Emile Mathieu, Urszula J Komorowska, and Pietro Lio. A framework for conditional diffusson modelling with applications in motif scaffolding for protein design. arXiv preprint arXiv:2312.0023 09236, 2023.
James Dunbar, Konrad Krawezyk, Jinwoo Leem, Terry Baker, Angelika Fuchs, Guy Georges, Jiye Shi, and Charlotte M Deane. Sabdab: the structural antibody database. Nucleic acids research, 42(D1):D1140- D1146, 2014.
Alex Herbert and MJE Sternberg. MaxCluster: a tool for protein structure comparison and clustering. 2008.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arxi::2007. 1100007122990 12598, 2022.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models, 2022.
John B Ingraham, Max Baranov, Zak Costello, Karl W Barber, Wujic Wang, Ahmed Ismail, Vincent Frap- pier, Dana M Lord, Christopher Thon-Hing, Erik R Van Vlack, et al. Illmmnnating protein space with a programmable generative model. Nature, pp- 1 9, 2023.
11

---

Published in Transactions on Machine Learning Research (07/2024)
Lin Jiang, Eric A Althoff, Fernando R. Clemente, Lindsey Doyle, Daniela Rothlisberger, Alexandre Zanghellini, Jasmine I, Gallaher, Jamie I, Betker, Fujie Tanaka, Carlos F Barbas III, Donald Hilvert, Kendal N Houk, Barry L. Stoddard, and David Baker. De novo computational design of retro-aldol 2008.
enzymes. Science, (19(5868)
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Polapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 2021.
Wolfgang Kabsch and Christian Sander. Dictionary of protein secondary structure: pattern recognition of hydrogen- bonded and geometrical features. Biopolymers: Original Research on Biomolecules, 22(12): 2577-2637, 1983.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprint arXiv:1422.6880 6980, 2014.
Leon Klein, Andreas Krämer, and Frank Noé. Equivariant flow matching. arXiv preprint arXiv:2306.15033 2023.
Jonas Köhler, Leon Klein, and Frank No6. Equivariant flows: exact likelihood generative learning for symmetric densities. In International conference on machine learning, pp. 5361 5370. PMLR, 2020.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling, International Conference on Learning Represenlations, 2023.
Dmitry Nikolayev and Tatjana Savyolov. Normal distribution on the rotation group SO(3). Teatures and Microstructures, 29, 1970.
Ashwini Pokle, Matthew J Muckley, Ricky TQ Chen, and Brian Karrer. Training-free lincar image inversion via flows. arXiv preprinl arXie:231000444 2023.
Erik Procko, Geoffrey Y Berguig, Betty W Shen, Yifan Song, Shani Frayo, Anthony J Convertine, Daciana Margineantu, Garrett Booth, Bruno E Correia, Yuanhua Cheng, William R Schief, David M Hockenbery, Oliver W Press, Barry I, Stoddard, Patrick S Stayton, and David Baker. A computationally designed inhibitor of an Epsttenn-Barr viral BCL-2 protein induces apoptosis in infected cells. Cell, 157(7):1644
1656, 2014.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Simo Särkkä and Arno Solin. Applied Stochastic Differential Equations. Cambridge Universily Press, 1 edition, April 2019. ISBN 978-1-108-186735 978-1-316-51008-7 676---316-. doi: 10.1017/ 9781108186735.
Neta Shaul, Ricky TQ Chen, Maximilian Nickel, Matthew Le, and Yaron Lipman. On kinetic optimal probability paths for generative models. In International Conference on Machine Learning, pp. 30883- 30907. PMLR, 2023.
Justin Siegel, Alexandre Zanghellini, Helena Lovick, Gert Kiss, Abigail R Lambert, Jennifer L StClair, Jasmine L Gallaher, Donald Hilvert, Michael Н Gelb, Barry Stoddard, Kendall N Houk, Forrest E Michael, and David Baker. Computational design of enzyme catalyst for stereoselective bimolecular Diels-Alde Alder reaction. Science, 329(9989)3309 313, 2010.
Jiaming Song, Arash Vahdat, Morteza. Mardani, and Jan Kautz. Psendoinneess--uiiee diffusion models for inverse problems. In International Conference on Learning Representations, 2022.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishck Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13.0 2020.
12

---

Published in Transactions on Machine Learning Research (07/2024)
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. arXiv preprint arXiv:2111.080000 2021.
Brian L Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. arXiv preprint ar Xrx::22006.0 149 2022.
Jue Wang, Sidney Lisanza, David Juergens, Doug Tischer, Ivan Anishchenko, Minkyung Baek, Joseph L Watson, Jung Ho Chun, Lukas F Milles, Justas Dauparas, Mare Exposit, Wei Yang. Amijai Saragovi, Sergey Ovchinnikov, and David A. Baker. Deep learning methods for designing proteins scaffolding func- tional sites. bioRrin, 2021.
Joseph Watson, David Juergens, Nathaniel Bennett, Brian Trippe, Jason Yim, Helen Eisenach, Woody Ahern, Andrew J Borst, Robert J Ragotte, Lukas F Milles, al. De novo design of protein structure and function with rfdiffusion. Nature, pp. 3, 2023.
Luhuan Wu, Brian L Trippe, Christian A Naesseth, David M Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. arXiv preprint arXiv:2306.17775 2023.
Kevin K Yang, Zachary Wu, and Frances H Arnold. Machine-learming-guiee directed evolution for protein
engineering. Nature methods, 16(8):687-694 694, 2019.
Jason Yim, Andrew Campbell, Andrew YK Foong, Michael Gastegger, José Jiménez-Luma, Sarah Lewis, Victor Garcia Satorras, Bastiaan S Veeling, Regina Barzilay, Tommi Jaakkola, et al. Fast protein backbone generation with (3) flow matching. arXiv preprint ar 0rXii:2310005222 2023a.
Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation. arXiv preprint arXiv:2302.080 02277, 2023b.
Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman, Aditya Grover, and Ricky TQ Chen. Guided flows for generative modeling and decision making. Xixiv preprint 2rxi:2211..34448 2023.
13

---

Published in Transactions on Machine Learning Research (07 (07/2023) 2024)
Appendix
A
Organisation of appendices
The appendix is organized as follows. App. B provides details and derivations for FrameFlow (Yim et al., 2023a) that we introduce in Sec. 2.2. App. provides derivation motif guidance used in Sec. 3.2. Des- ignability is an important metric in our experients, so we provide a description of it in App. E. Lastly, we include additional results unconditional generation App. F and motif-scaffolding App. G.
B FrameFlow details
B.1 Backbone SE(3) representation
A protein be described by its sequence of residues, cach of which takes discrete value from vocabu- lary of amino acids, well as the 3D structure based on the positions of atoms within each residue. The 3D structure in each residue can be separated into the backbone and side-chain atoms with the composition of backbone atoms being constant across all residues while the side-chain atoms vary depending on the amino acid assignment. For this reason, FrameFlow and previous SE(3) diffusion models (Watson et al., 2023; Yim et al., 2023b) only model the backbone atoms with the amino acids assumed to be unknown. A second model typically used design the amino acids after the backbone is generated. Each residue's backbone atoms follows repeated arrangement. with limited degrees of freedom due to the rigidity of the covalent bonds. AlphaFold2 (AF2) (Jumper et al., 2021) proposed SE(3) parameterization of the backbone atoms that we show in Fig. 6. AF2 uses mapping of four backbone atoms to single translation and rotation that reduces the degrees of freedom in the modeling. It is this SE(3) representation we use when modeling protein backbones. We refer to Appendix of Yim et al. (2023b) for algorithmic details of mapping between elements SE(3) and backbone atoms.
Casa
U2
Co
U1
Wiz
CramSchmidd(en: u2)
C, E R T (r, E SE(3)
A E SO(2)
(b) Backbone atoms of a single residue
(c) SE(3) parameterization of backbone atoms of a single residue
(a) Protein backbone atoms
Figure 6: Backbone parameeeriization with SE(3). (a) Shows the full protein backbone alomic structure without side-chains. (b) Zooms in the backbone atoms of single residue. Note the repeated arrangements of backbone atoms in each residue. (c) The transformation of turning each set of four backbone atoms into an element of SE(3).
B.2 SE(3) flow matching implementation
This section provides implementation details for SE(3) flow matching and FrameFlow. As stated in Sec. 2.1, SE(3)* N can be characterized as the product manifold SE(3) = IRN x * It follows that flow matching We will parameterize backbones with N
SE(3) N equivalent. to flow matching 3R and SO(3)
14

---

Published in Transactions on Machine Learning Research (07/2024)
by translations Rav and rotations SO(3)* N As reminder, use bold face for vectors of all the residue: T= ( T(O), = [x(11) .("),,,()], [r(1). ..... ,,())
residues T = (x,r) SE(3)* N
Riemannian flow matching (Sec. 2.1) proceeds by defining the conditional flows,
(n) 22 = (1 (1=t)x + =tal")
rt = exp, (tlog,co (w) (o) (r("))
(16)
for each residue E NJ....,N. As priors use (n) isotropic Gaussian in 3D with centering where each sample is centered to have zero mean - this is important for equivariance later ((S)(3)) is the uniform distribution over SO(3). The end points a1()) and r(()) are samples from the data distribution Pl.
(O 13) and TO (n) ((O [O 13) is the
Eq. (16) uses the geodesic path with linear interpolation; however, alternative conditional flows can be used (Chen & Lipman, 2023). A special property of SO(3) is that expr, and log,, can be computed in closed form using the well known Rodrigues' formula. The corresponding conditional vector fields are
(1))
log, (n) (r(a) (()) ,
(x) (z) [
21
(2) (/("))
(17)
S0(3)
1 1-t
1-t
We train neural networks to regress the conditional vector fields through the following parameterization,
(m) (T.) (m)
(n) (0(3) (T.,t) log log,(t) (n) (ff (T.)) (T))
21
xt
(T
i
(18)
1-t
1-t
where the neural network outputs the denoised predictions afm) and p!m) while using the noised backbone T, as input. We now modify the loss from Eq. (5) with practical details from FrameFlow,
CSE(s) = Ee(tto.)) [C T1, 6) + 2Cso (,. TT, t) 1 0.5) (T,, Tí, t)
(19)
CR Ti,() //x PR VR () (1 ||x] *
(20)
min(4, 0.9))2
log log (m (r(()) logg
( (T,))
L5o (Tr, T1, (r, - (T,, () (0(3))
(0(3)
(21)
(1 min 0.9)
We weight the SO(3) loss Lso(3) such that it is on a similar scale as the translation loss Ca. Eq. (20) is simplified to be a loss directly on the denoised predictions. Both Eq. (20) and Eq. (21) have modified denominators (1 min(t, 0.9)) instead of (1 (1-
avoid the loss blowing up near 1. In practice, we (2023b) where
sample t uniformly from Ld(e, 1] for small e. Lastly, Laux is taken from section 4.2 in Yim they apply RMSD loss over the full backbone atom positions and pairwise distances. We found using Caux for all t 0.5 to be helpful. The remainder of this section goes over additional details in FrameFlow.
Alternative (3( prior. Yim et al. (2023a) reported using the IGSO3( = 1.5) prior (Nikolayev & Savyolov, 1970) for SO(3) instead of (SO(3)) lead to improved performance. The choice of = 1.5 will shift the ra samples away from π where near degenerate solutions can arisc in the geodesic. We follow using IGSO3( 1.5) for training while using the u((O(3)) prior for sampling.
Pre-alignment. Following (Klein et al., 2023) and Shaul et al. (2023), we pre-align samples from the prior and the data by using the Kabsch algorithm to align the noise with the data to remove any global rotation that results a increased kinetic energy of the ODE. Specifically, for translation noise x, ~ (O(0 Iz) and data X2 Pi where Xg, X1 3 we solve x* arg migmiensoor) ||rx, - 112 and use the aligned noise r"x, during training. Yim et al. (2023a) found this to aid in training efficiency which we adopt.
15

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Symmetries. We perform all modelling within the zero center of mass (CoM) subspace of N as in Yim et al. (2023b). This entails simply subtracting the CoM from the prior sample Xy and all datapoints X1. As X₁ is a linear interpolation between the noise sample and data, will have 0 CoM also. This guarantees that the distribution sampled frames that the model generates is SE(3)-invariant. To see this, note that the prior distribution is SEn3)-invariant and the learned vector field VSE(3) is equivariant because we use an -quiivattvar architecture. Hence by Köhler al. (2020), the push-forward of the prior under the flow is invariant.
Auxiliary losses. We use the same auxiliary losses in (Yim et al., 2023a).
(3) inference scheduler. The conditional flow in Eq. (16) uses constant linear interpolation along the geodesic path where the distance of the current point x to the endpoint 21 is given by a pre-metric dg : :M x MM- + induced by the Riemannian metric g on the manifold. To see this, we first recall the general form of the conditional vector field with x, 21 E M is given as follows (Chen & Lipman, 2023),
d dlgg x()) d(x,,)) (x<x)/))=
V1(1,
(22)
dt
Vd(x, x1) x1)
d d k(t)
x1)
(23)
2 Vd(x, 21) |
dlog x(1)
log, (z1)
(24)
dt
[Vd(r, x1))
-d m<(t) log, (I1),
(25)
dt
with <(() monotonically decreasing differentiable function satisfying x(O) and s(1) = 0, referred as the interpolation rate" Then plugging in a the linear schedule x(t) = t, we recover Eq. (17)
(x(x)11) = d log n(t) log, (x1) = 7 (x1).
(26)
dt
However, we found this interpolation rate to perform poorly for SO(3) for inference time. Instead, we utilize an exponential scheduler x(t) = e e-st for some constant c. The intuition being that for high c, the rotations accelerate towards the data faster than the translations which evolve according to the linear schedule. The SO(3) conditional flow in Eq. (16) and vector field in Eq. (17) become the following with the exponential schedule,
= exPro ((1 ((1-e- (11--a)) e-)) log, (ri))
(27)
uf!)) = clog,(.) (m)) (n)
(28)
We find c = 10 or 5 to work well and use c = 10 in our experiments. Interestingly, we found the best performance when x(t) = 1 -t t was used for SO(3) during training while r(t) = e-ot used during inference.
We found using x(t) = -t during training made training too easy with little learning happening.
The vector field in Eq. (28) matches the vector field in FoldFlow when inference annealing is performed (Bose et al., 2023). However, their choice of scaling was attributed to normalizing the predicted vector field rather than the schedule. Indeed they proposed to linearly scale up the learnt vector field via X(1) (1 ()e at sampling time, L.e. to simulate the following ODE:
dr. = A(t)o(r,, d)tt.
However, as hinted at earlier, this is equivalent to using at sampling time a different vector field ö(re, f) induced by an exponential schedule R(t) = =t itinteeddd of the linear schedule k(t) = 11 t (that the neural
7 xx(t) as scheduler that determines the at which d(-[a1) decreases, since have that di decreases d(;,z) according die (10))1)). = k(t)d(r0, 21) (Chen & Lipman, 2023).
16

---

Published in Transactions on Machine Learning Research (07 (07/2024)
network of was trained with). Indeed we have
o(r,, b) -2 log K(t) logr. (fi) -0, log r&(t)
0r log x(t) logr, (f1)
(29)
- log x(1)
=- - 1)0, log logr, (f1) c11 - v(re,)) () = x(1) v(r,,)).
(30)
C Data augmentation
Algorithm Motif-scaffolding data augmentation
Require: Protein backbone T; Min and motif percent, Twin 0.05, Tmax 0.5.
Uniform{ [N "miil,.... [ "maxx]
Sample maximum motif size. Sample maximun nunber of motifs.
Uniform(1..... (8...,, 3: TM +
4: for E (1.....m) m do
j j~ Uniform I.... NJ TM l~ Uniform 1I.... m + i (T
P Sample location for each motif Sample length of each motif. Append existing motif.
T T U (TT (T........ mmin(+te,M)
end for
T < (Ti... [T] TM
Assign rest of residues the scaffold
10: return T T
Motif guidance details
For the sake of completeness, we derive this section the guidance term Eq. (8) for the flow matching setting. In particular, we want derive the conditional vector field v(z,, tly) in terms of the unconditional vector field t) and the correction term V Pl notation from diffusion models, i.e. i = 0 for denoised data to t = for fully noised data. We therefore need to swap + t in the end results to revert to the flow matching notations.
Beware, in the following we adopt the time
Let's consider the process associated with the following noising stochastic differential equation (SDE)
dr; = f(x, t)dt + g(t)dB,
(31)
which admits the following time-reversal denoising process
dr. [f(x,, g( log ((x)) dt g(t)dB.
(32)
Thanks to the Fokker- Planck equation, know that the the following ordinary differential equation admits the same marginal as the SDE Eq. (32):
dr. = [(m,,)) 1 90(() ((1g)VVoopp( dt
= c t)dt.
(33)
with t(z,,t) being the probability flow vector field.
Now, conditioning on some observation we have
dx. = u(xe, Ml)
1 ((,,,) t) V dt
1
f(e,,) 7 0()) (Vlg)((V + log loggi()lz))) dt
u((e,)) 1
( dt.
(34)
17

---

Published in Transactions on Machine Learning Research (07 (274))
Eq. (34) follows from the same reverse SDE theory of Eq. (32) except the initial state distribution is p(z)) v)). The drift fa 1) and diffusion g({) coefficients are unchanged while only the score reflect the new initial distribution. More details can be found in App. of Song et al. (2020). We only need to know g(t) to adapt reconstruction guidance- which estimates V log Vlogpe(y/m))- the flow matching sctting where we want to correct the vector field. Given a particular choice of interpolation by from flow matching, let's derive the associated g(t).
Euclidean setting Assume zo is data and xı is noise, with 21 ~ (O, D). In Euclidean flow matching, we assume a linear interpolation
(1 - t)xo + trj. Conditioning on 2o, we have the following conditional t) t!T). Meanwhile, let's derive the marginal density Рejo induced by
marginal density Prjo = N ((1
Eq. (31). Assuming a linear drift f(a,, t) = p(t) we know that Pho is Gaussian. Let's derive its mean
m = [R(x) and covariance Σ, Cov(2r). We have that (Särkkä Solin, 2019)
dt m, = E [f t)] = ra(tmm.
(35)
thus
[|x exp C% ((s)dd)) 20.
(36)
Additionally,
d E, E [f((,,, (mm [f(z,, t) (m, xx)] + g(t)
(37)
dt
= 2μ(t)E+ + gt))
(38)
Matching oijo and Pejo. get
exp (%, ((sosda)) xn -t)
(39)
L' u(s)ds = In([1 - -)
(40)
+ u(l) -
(41)
and
2x(())² + +g(1)² =11
(42)
1
# - 1-t 12 + g(t)² = 2t
(43)
1
+ g(1)² = 21 + 2
-- 12
(44)
2t
+ g(t)²
(45)
--6
The equivalent SDE that gives the same marginals Therefore, the following SDE gives the same conditional marginal flow matching:
2t dz, -1 xt + V dB.
(46)
SO(3) setting The conditional marginal density p.p induced by Eq. (31) with zero drift f(r,t) 0 is given by the IGSO(3) distribution (Yim et al., 2023b): prjo IGSO(3) ro. We are not aware of a closed form formula for the variance of such distribution.
On the flow matching side, we assume To is data and rı noise, with r1 ~ uSSO(3)), and a geodesic interpolation r, = expr, (tlog,, (ri)). We posit that the induced conditional marginal Pejo not an IGSO(3) distribution. As such, appears non con-trivial to derive the required equivalent diffusion coefficient g(t) for SO(3). We therefore use a heuristic the same g(() for Rdd
18

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Success (designable): if (1) and (2) satisfied. Report % of samples that designable.
1. Scaffold RMSD 2.0
SLAKEQLIBIARKLA
D'SLFOXDGDGTIVT
KELOTVCRVLSESTC
Generative model
ProtcinMPNN
KRMINEVDADGHOTE
AF2
DFPEFLTMCOLVCKE AGIPPTS
Noise + motif
Scaffold + motif
Sequence
Predicted structure
2. Motif RMSD < 1.0
Figure 7: Schematic of computing motif-scaffolding designability. 'Generative model" is a stand-in for the method used to generate scaffolds conditioned on the motif. From there, we use ProteinMPNN (Dauparas et al., 2022) design the sequence then use AlphaFold2 (AF2) (Jumper et al., 2021) in predicting the structure of the sequence. The RMSD is calculated on the scaffold (blue) and motil (red) separately with alignments. A generated scaffold passes designability if the scaffold RMSD 2.0 and motif RMSD 1.0.
E Designability
We provide details of the designability metric for motif-scaffolding and unconditional backbone generation previously usod in prior works (Watson et al., 2023; Wu et al., 2023). The quality of a backbone structure is nuanced and difficult to find single metric for. One approach that has been proven reliable in protein design is using highly accurate protein structure prediction network to recapitulate the structure after the sequence is inferred. Prior works (Bennett et al., 2023; Wang et al., 2021) found the best method for filtering backbones to use in wet-lab experiments was the combination of ProteinMPNN (Dauparas et al., 2022) to generate the sequences and AlphaFold2 (AF2) (Jumper et al., 2021) to recapitulate the structure. We choose to use the same procedure determining the computational success of our backbone samples which we describe next. As always, we caveat these results are computational and may not transfer wet-lab validation. While ESMFold (Jumper et al., 2021) may be used in place of AF2, we choose to follow the setting of RFdiffusion close as possible.
We refer to sompled backbones as backbones generated from our generative model. Following RFdilfusion, we use ProteinMPNN at temperature 0.1 to generate 8 sequences for each backbone in motif-scaffoldiig and unconditional backbone generation. In motif-scaffolding, the motif amino acids are kept fixed Protcin- MPNN only generates amino acids for the scaffold. The predicted buckbone of each sequence is obtained with the fourth model in the five model ensemble used in AF2 with 0 recycling, no relaxation, and no multiple sequence alignment (MSA) as in the MSA is only populated with the query sequence. Fig. 7 provides schematic of how compute designability for motif-scaffolding. A sampled backbone is successful referred to as designable based on the following criterion depending on the task:
Unconditional backbone generation: successful if the Root Mean Squared Deviation (RMSD)
of all the backbone atoms is 2.0Aafter global alignment of the Carbon alpha positions.
Motif-scaffolding: successful if the RMSD of motif atoms is 1Åafter alignment the motif Carbon alpha positions. Additionally, the RMSD of the scaffold atoms must be < 2Aafter alignment on the scaffold Carbon alpha positions.
19

---

Published in Transactions on Machine Learning Research (07/2024)
F FrameFlow unconditional results
We present backbone generation results of the unconditional FrameFlow model used in FrammFlow-guidance. We do not perform an in-depth analysis since this task is not the focus of our work. Characterizing the backbone generation performance ensures we are using a reliable unconditional model for motif-scaffolddng. We evaluate the unconditionally trained FrameFlow model by sampling 100 samples from lengths 70, 100, 200, and 300 done in RFdiffusion. The results shown in Tab. 2. We find that FramcFlow achieves slightly worse designability while achieving improved novelty. We conclude that FrameFlow is able to achieve strong unconditional backbone generation results that are on par with current state-of-the-art unconditional diffusion model RFdiffusion. We perform secondary structure analysis of the unconditional samples in Fig. 8.
Table 2: Unconditional generation metrics.
Method Des.(t)
Div. (t)
Nov. (4)
FrameFlow 0.86
155
0.61
RFdiffusion 0.89
159
0.65
We find FrameFlow has a tendency to sample more alpha helical structures than the data distribution but still has roughly the same coverage of structures. Future work could investigate the cause of such helical tendency and improve the secondary structure sample distribution.
Sampled secondary structure
Data secondary structure
Strand composition
0.5
Strand composition
0.5
0.4
0.4
0.3
0.1
0.1
0.0
0.0
0.0
0.2
0.4
0.6
1.0
0.0
0.2
0.4
0.8 1.0
Helix composition
Helix composition
Figure 8: Secondary structure analysis of unconditional samples. Using FrameFlow sample 100 proteins for each length 70, 100, 200, and 300. The 2D kernel density estimation plot of the secondary structure composition is shown on left. On the right, we show the secondary structure composition of all length 70, 100, 200, and 300 proteins the training set. We find FrameFlow has a lendency to sample more alpha helical structures than the data distribution.
G FrameFlow motif-scaffolding analysis
In this section, we provide additional analysis into the motif-scaffolding results in Sec. 5.2. Our focus is on analyzing the motif-scaffolding with FrameFlow: motif amortization and guidance. The first analysis is the empirical cumulative distribution functions (ECDF) of the motif and scaffold RMSD shown in Fig. We find that the main advantage of amortization is having a higher percent of samples passing the motif RMSD threshold compared to the scaffold RMSD. Amortization has better scaffold RMSD but the gap is smaller than motif RMSD. The ECDF curves are roughly the same for both methods.
Tab. 3 shows the average pairwise TM-score for all designable scaffolds per motif. We report this for each method where we find our FrameFlow approaches get the lowest average pairwise TM-score in 19 out 24 motifs. The average pairwise TM-score mcant complement the cluster critcrion for diversity case where clustering leads pathological behaviors. Both metrics have their strengths and weaknesses but together help provide details sample diversity.
20

---

Published in Transactions on Machine Learning Research (07 (224))
each motif in the benchmark. As noted in Sec. 5.2, amortization is able to solve 21 out of 24 motils in the benchmark. In Fig. 10, we visualize the generated scaffolds that are closest to passing designability for the 3 motifs it is unable to solve. We find the failure to be in the motif RMSD being over 1ÅRMSD. However, for 4JHW, 7MRX _60 and 7MRX 128 the motif RMSDs are 1.2, 1.1, and 1.7 respectively. This shows amortization is very close solve all motils the benchmark. In Fig. 11, we show designable scaffolds for each of the 21 motifs that amortization solves. We highlight the diverse range of motifs that can be solved well as diverse scaffolds.
Lastly, visualize samples from HhameFlow-mmrriizane
1.0
Method FrameFlow amortizationnn
FrameFlow-guided
0.8
0.6
0.4
0.2
Method
guided
0.0
Scaffold RMSD
Figure 9: Empirical cumulative density plot of designability RMSD over the motif (left) and scaffold (right) for maamcloo-amortizatee (blue line) and HhaneFlow-guidanee (orange line).
Table 3: Under each method name are average TM-score over all designable scaffolds for each motif. Lower is better which indicates more dissimilar pairwise scaffolds. N/A indicates no designable scaffolds were sampled. We find our FrameFlow approaches have the lowest average TM-scores among all methods.
Motif
Prameflow-amortzzation
FrameFlow-guidance
TDS
RFdiffusion
6F6R med
0.3
0.31
0.36
0.39
2KL8
0.95
0.86
0.88
0.98
4ZYP
0.55
0.44
N/A
N/A
5WN9
0.53
0.45
0.48
N/A
5TRV _hort
0.48
0.42
0.46
0.66
7MRX 60
N/A
N/A
0.29
0.59
6EXZ short
0.48
0.49
0.47
0.35
YCC
0.34
0.3
0.4
0.48
5LUS
0.6
N/A
N/A
0.73
6E6R short
0.37
0.35
0.39
0.41
3XXT
0.4
0.47
0.46
0.62
7MRX 85
N/A
0.36
N/A
0.56
QQJG
0.34
0.44
0.33
N/A
IBCF
0.76
0.69
0.5
0.83
5TRV med
0.34
0.37
0.38
0.43
SYUI
N/A
N/A
N/A
N/A
5TPN
0.48
0.54
N/A
0.61
IPRW
0.75
0.66
N/A
0.76
6EXZ med
0.37
0.38
0.36
0.49
STRV _ong
0.3
0.31
N/A
0.39
4JHW
N/A
N/A
N/A
N/A
7MRX 128
N/A
N/A
N/A
0.5
6E6R long
0.28
0.28
0.46
0.35
6EXZ long
0.3
0.3
0.38
0.38
21

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Undesignable motif-scaffolds
4JHW
7MRX_ 60
7MRX 128
SWN9 MoIIf
Motif RMSD:
Morif RMSD:
Scaffold RMSD:
Scaffold RMSD:
Scaffold RMSO:
Scaffold RMSD:
Figure 10:
on the three motifs it fails to solve. We find the > 1.0Amotif RMSD is the reason for the method failing to pass designability.
Closest motif-scaffolds from
Designable motif-scaffolds
Motif
Scaffold
1BCF
IPRW
1QJG
IYCR
2KL8
3IXT
5IUS
5TPN
STRV_ong
5TRV _med
_hort
4ZYP
5YUI
6E6R JEnR_long
6E6R_short
6EXZ_long
6EXZ med
6EXZ short
7MRX 85
Figure 11:
Designable motif-scaffolds from FrameFlow-amone ammotiiiiinnionn on 20 out of 24 motifs in the motif-
scaffolding benchmark.
22


Reviews:
[{"review_id": "DRC6zhTXQM", "reviewer": "Reviewer_awUG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nI think the approach is broadly good and the augmentation approach is clever.\n\nI think the math in applying the approach to rotations and translations is well defined.\n\nI think Figure 5 is an interesting observation in the generation bias between FrameFlow and RFdiffusion. Just curious–what does the distribution of the training set look like?\n\nWeaknesses\n\nHow realistic is the Motif data augmentation approach? Are annotated motifs close in 3D space? In your sampling algorithm, does the distribution of motif residue distances match the distribution of a known dataset?\n\nFigure 3 is very difficult to interpret. Can the Y axis be labeled? Can this data be summarized in some way? If you don’t see any clear trend, why show it? What data do you have to suggest that RF diffusion has the highest success rate–can this be backed up by a statistic, like a p-value?\n\nNeutral\n\nThe authors note their method increases diversity of generated structures. Is it explicitly defined in any objective function? What is the mechanism for this increase in diversity? Is this controllable in some way itself?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose additional methods to generate protein structures conditioned on motifs. They compare new approaches as well as additional data augmentation techniques.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think Figure 1 could be made even more clear by making all the atoms of the Scaffold red (or warm tones) and all the atoms of the Motif blue (or cool tones). While I like the concept of the figure, it is slightly confusing to me that there are blue atoms on the red Scaffold residues.\n\nTypo: “We use data augmentation to amortize over all possible motifs in our traiing set and aid in generalization to new motifs.”\n\nIt would be interesting to discuss the impact of multimers on the proposed methods.\n\nSee Weakness section about Figure 3. It needs to be made much clearer to highlight contributions.\n\nIn the Diversity analysis section, each of these generated structures is equally likely to fold? How do you ensure that the generated structures are actually valid?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "d2DL6YdVqg", "reviewer": "Reviewer_QRBg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Pros\n\n- The proposed motif amortization and motif guidance are novel and effective in the protein design domain. The paper leverages SE(3) flow matching to model protein backbones, extending it to motif-scaffolding. \n- The proposed methods aim to overcome the limitations of structural diversity in generated scaffolds, which is crucial for successful wet-lab validations. Extensive experiments have been conducted to provide a good insight into the components of the proposed method.\n- The paper is generally well-written, with clear illustrations and tables.\n\n## Cons \n\n-  In Sec. 3.1, the motif amortization introduces unlabled structure from PDB as an approximation for the $p(\\textbf{T}^M)$ and $p(\\textbf{T}_1^S|\\textbf{T}^M)$. However, the paper does not discuss the approximation error. In addition, the experiment section does not provide empirical results on the robustness of the chosen PDB.\n-  The motif guidance seems to heavily depend on the TDS, a motif-guide diffusion model, as described in Sec. 3.2. The paper claims that the diffusion model based method suffers from low scaffold diversity. However, the underlying cause for such an issue, and the reason for the proposed method to alleviate it are not well discussed in the paper.\n-  The writing can be improved by introducing more background knowledge on flow matching and ODE functions, as the current version might not be friendly enough for those who are not familiar with these methods. In addition, The paper only compare two motif-scafolding method in the experiment seciton, which might not convicinen enough.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "To design a protein, starting from a motif and leveraging its knowledge of a desired function is commonly adopted. However, such approaches tend to lack structural diversity. To alleviate this issue, the paper extends FrameFlow with motif amortization, which augments the training input motif, and motif guidance, which provides training-free conditional information. The paper conducts extensive empirical studies and justifies the effectiveness of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Provide discussion on the approximation error and experiment with different choices of PDB.\n2. Provide secondary structure analysis on FrameFlow.\n3. Provide more comparison with the SOTA method, including diffusion-based and motif-scafolding based methods.\n4. Provide discussion on the potential reason why the diffusion-based method suffers from low diversity scaffolding and how the flow matching based method resolves that.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "PaLq0Kqs8G", "reviewer": "Reviewer_8RGg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is written in a very clear way.\n- The background section is helpful in understanding the study.\n- The method description is supported by figures that demonstrate the diffusion process and data augmentation (although blue dots, presumably nitrogen atoms, may be confused with the parts of the blue motif)\n- The theoretical and methodological part of the paper is strong, including the derivations in the appendix (although I have not checked the correctness thoroughly) \n\nWeaknesses:\n- The statement in the abstract that the proposed method achieves \"a 2.5 times greater in-silico success rate of unique motif-scaffolds\" is imprecise. It does not explain what \"success rate\" is in this context.\n- The notation is inconsistent between the background and method sections: it is explicitly stated in the background section that superscripts are used to refer to residues indices, but in the next section subscripts are used.\n-  I am also a little bit concerned about the zero-centering of the motif to maintain SE(3)-equivariance. Technically this seems correct, but can it impede the generalization to other motifs? For example, if you add a few atoms or another subchain to the motif, the whole structure moves which could lead to significantly different predictions.\n- The parameters $\\gamma_{min}$ and  $\\gamma_{max}$ are not defined in Section 5.1 (only in the appendix)\n- My main concern is about the experimental section. I am unsure if the chosen evaluation protocol gives the readers any meaningful information about the model performance. It seems that the proposed method generates less designable scaffolds than the already existing methods (Figure 3). The models are thus compared using scaffold diversity, but this evaluation metric is based on clustering. The number of generated designable scaffolds may impact the number of clusters. If the generated scaffold space is sparse, it would lead to more clusters as there are no scaffolds in-between that would bridge two clusters. It can be observed in Figure 3 that for the methods producing the biggest number of designable scaffolds, the number of clusters is very small (e.g. 2KL8 or 6E6R_long). It would be more convincing if the diversity metric were defined directly on the predicted atom positions, e.g. using the average RMSD between all pairs of aligned scaffolds.\n- The experimental evaluation could have been done more thoroughly. It seems that the speed was measured only for one protein. There are only 24 motifs in the testing set. Maybe this evaluation should be at least repeated a few times to make sure that the noise related to random sampling is removed. No hyperparameter tuning was conducted. \n- Some claims in Appendix B.2 are not supported by any experiments. For example, you say that using the IGSO3 prior improved performance, and the pre-alignment of the noise resulted in improved training efficiency. These decisions seem interesting and important for your presented methods. Maybe these experiments could be described as an ablation study.\n- In appendix F, you say that FrameFlow achieves improved diversity, but Table 2 shows otherwise.\n\nIn conclusion, the approach described in the paper is very interesting, but the experiments seem rather preliminary at this stage.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces two new methods for motif-scaffolding, which is the task of building scaffolds around protein motifs. Both methods are based on FrameFlow. One, called motif amortization, starts with a pre-defined motif structure (3D positions of the alpha carbons) and generates a scaffold around it. To generate data for training, a data augmentation technique is proposed that extracts random fragments of the protein and marks them as the motif. Another method, called motif guidance, in which the motif is used to bias the generative trajectory. In the experiments, the Authors demonstrate that both methods can produce designable and diverse scaffolds, and motif amortization outperforms the other tested methods (RDdiffusion and TDS).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please add a short discussion about the impact of motif zero-centering on the generalizability.\n2. I believe more experiments should be conducted, and better evaluation metrics should be proposed (e.g. other diversity measures as suggested above).\n3. Please check carefully whether the number of clusters is not correlated with the number of designable scaffolds generated by the model.\n4. The experiment should be repeated a few times to denoise the reported metrics and to provide a standard deviation.\n5. An ablation study would help in understanding all the technical decisions related to the architecture and training procedure (see comments above).", "broader_impact_concerns": "I do not think any ethical implications need to be described in this paper. One potential implication would be the use of this research to create harmful biological agents.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All Reviewers unanimously supported accepting the paper. Among the strengths of the paper, Reviewers appreciated clarity of writing, comprehensive experimental setup, and the additional evlauation of the diversity of proposed scaffolds. Authors engaged in the rebuttal and addressed major comments.

Thank you for your submission and it is my pleasure to recommend accepting the work. 

I would like to make a minor suggestion to the work. I think readers would benefit from providing a motivation in the introduction why modifying FrameFlow is the natural way to increase diveristy of the proposed solutions. One may think that modifying other methods might or might not also bring benefit, so some elaboration on this topic could be useful. It would also seem to me that the technical introduction to CNFs might be too dense for certain audiences. Perhaps there is a way to expand it a bit and make it more intuitive to less mathematical audience?
Decision: Accept (poster)