Out of Many, One: Designing and Scaffolding Proteins at the Scale of the Structural Universe with Genie 2

## Abstract

Abstract Protein diffusion models have emerged as a promising approach for protein design. One such pioneering model is Genie, a method that asymmetrically represents protein structures during the forward and backward processes, using simple Gaussian noising for the former and expressive SE(3)-equivariant attention for the latter. In this work we introduce Genie 2, extending Genie to capture a larger and more diverse protein structure space through architectural innovations and massive data augmentation. Genie 2 adds motif scaffolding capabilities via a novel multi-motif framework that designs co-occurring motifs with unspecified inter-motif positions and orientations. This makes possible complex protein designs that engage multiple interaction partners and perform multiple functions. On both unconditional and conditional generation, Genie 2 achieves state-of-the-art performance, outperforming all known methods on key design metrics including designability, diversity, and novelty. Genie 2 also solves more motif scaffolding problems than other methods and does so with more unique and varied solutions. Taken together, these advances set a new standard for structure-based protein design. Genie 2 inference and training code, as well as model weights, are freely available at: https://github.com/aqlaboratory/genie2 .

## 1 Introduction

The design of proteins with novel structures and functions has emerged as a potent technology in therapeutic [, , ] and industrial applications [, ]. Generative AI has driven recent advances in protein design, most notably diffusion [, ] and flow matching [] models, as has the revolution in protein structure prediction sparked by AlphaFold 2 []. Proteins are one-dimensional polymers of amino acids ("sequences") that fold into three-dimensional shapes ("structures"). Generative protein models mirror this delineation, with most operating either in the sequence or structural domain. One rationale for sequence-based methods is that sequences are what ultimately get synthesized as functioning biomolecules, while structures require an additional structure-to-sequence map (inverse folding). Sequence-based models include EvoDiff [], a discrete diffusion model that uses order-agnostic autoregressive diffusion with a ByteNet-style [] architecture for denoising. EvoDiff is a promising and complementary approach to structure-based design, currently the prevalent paradigm.

Structure-based methods [, , , , , , , , ] focus on modeling structure space and typically employ separate inverse folding models such as ProteinMPNN [] to propose plausible sequences given a generated structure. Their key rationale is that structure more closely associates with protein function than sequence. Among them, Genie performs diffusion on backbone atom coordinates and uses an SE(3)-equivariant denoiser to reason over a cloud of reference frames constructed from backbone coordinates. FrameDiff [] uses a diffusion process in SE(3) on backbone frames with an AlphaFold-inspired architecture for denoising. FrameFlow [] adopts the general architecture of FrameDiff but uses flow matching instead. Chroma [] combines a correlated diffusion process that respects statistical properties of natural proteins with an efficient graph neural network. It also includes a separate design network that predicts sequences and side-chain atoms given a generated backbone. More recently, Proteus [] uses a similar diffusion process and architecture as FrameDiff but introduces graph triangle blocks that combine the expressiveness of triangle attention from AlphaFold 2 with faster runtimes by limiting attention to nearby residues.

The inter-connectedness of sequence and structure suggests that integrating their representations would advance protein design, particularly for conditional tasks that require pre-specified sequence or structural elements. Recent methods reflect this. One approach integrates sequence information as a condition of a structure-based diffusion process, as RFDiffusion [] does when designing proteins with known sequence fragments. Another approach performs diffusion or flow matching in a joint sequence-structure space, as done by MultiFlow [] when it combines an SE(3) structural flow with a discrete sequence flow. There have also been attempts [] at jointly encoding sequence and structure in a latent space and diffusing in this space; however, the approach remains nascent.

Whether encoded by sequence or structure, function is what is sought in protein design. Many functions, including interactions with small molecules and other proteins, are governed by few residues, or a motif. Achieving prescribed functions can thus often be distilled into designing a protein with a specific motif (e.g., an enzyme active site [] or antigen-binding site []), known as motif scaffolding. Diffusion models have shown success in this realm: Wu et al. [] developed a sequential Monte Carlo sampler called Twisted Diffusion Sampler and applied it to FrameDiff to scaffold motifs while RFDiffusion and an updated FrameFlow [] were explicitly trained on motif-conditioned tasks. Yet, current models cannot design proteins with multiple independent motifs, as they require inter-motif positions and orientations to be known a priori. Proteins often comprise independent functional sites, either as separate domains connected by a flexible linker or as one globular domain, such as an enzyme with multiple substrate binding sites or a scaffolding protein that engages multiple signaling ligands. The ability to design such proteins, which we term multi-motif scaffolding, would enable the development of new enzymes [], biosensors [], and therapeutics that disrupt or enhance protein-protein interactions []. Concurrent with our work, Castro et al. [] employed an established non-diffusion model, $\text{RF}_{\text{joint2}}$, to inpaint an immunogen containing three distinct epitopes. This approach appears promising but has yet to be systematically benchmarked.

In this work, we extend Genie to support single- and multi-motif scaffolding. We also improve the core Genie model through architectural modifications and enhancements to its training data and process. The resulting Genie 2 better captures protein structure space. When compared to existing models, Genie 2 sets state-of-the-art results in designability, diversity, and novelty. In addition, Genie 2 surpasses RFDiffusion on motif scaffolding tasks, both in the number of solved problems and the diversity of designs. We also curate a benchmark set comprising 6 multi-motif scaffolding problems from the literature and show that Genie 2 can propose complex designs incorporating multiple functional motifs, a challenge unaddressed by existing protein diffusion models.

## 2 Previous Genie Model

#### Diffusion with asymmetric protein representations

In contrast to all other SE(3)-equivariant diffusion models for protein generation, which use unified representations for the forward and backward diffusion processes, Genie represents proteins as point clouds of $C_{\alpha}$ atoms in the forward process and as clouds of reference frames in the reverse process. Let $\mathbf{x}=[\mathbf{x}^{1},\mathbf{x}^{2},\cdots,\mathbf{x}^{N}]$ be a sequence of $C_{\alpha}$ coordinates of length $N$. Given a sample $\mathbf{x}_{0}$ from the unknown protein structure distribution, Genie’s forward process gradually adds isotropic Gaussian noise through a cosine variance schedule $\beta=[\beta_{1},\beta_{2},\cdots,\beta_{T}]$, where $T$ is the total number of diffusion steps (set to 1,000).

$$ $q(\mathbf{x}_{t}|\mathbf{x}_{t-1})=\mathcal{N}(\mathbf{x}_{t}|\sqrt{1-\beta_{t }}\mathbf{x}_{t-1},\beta_{t}\mathbf{I})$ (1) $$

By reparameterization, we have

$$ $q(\mathbf{x}_{t}|\mathbf{x}_{0})=\mathcal{N}(\mathbf{x}_{t}|\sqrt{\bar{\alpha} _{t}}\mathbf{x}_{0},(1-\bar{\alpha}_{t})\mathbf{I})\quad\text{where}\quad\bar{ \alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}\quad\text{and}\quad\alpha_{t}=1-\beta_{t}$ (2) $$

Since the isotropic Gaussian noise added at each diffusion step is small, the corresponding reverse process could be approximated with a Gaussian distribution:

$$ $p(\mathbf{x}_{t-1}|\mathbf{x}_{t})=\mathcal{N}(\mathbf{x}_{t-1}|\mathbf{\mu}_{ \theta}(\mathbf{x}_{t},t),\mathbf{\Sigma}_{\theta}(\mathbf{x}_{t},t)\mathbf{I})$ (3) $$

where

$$ $\mathbf{\mu}_{\theta}(\mathbf{x}_{t},t)=\frac{1}{\sqrt{\alpha_{t}}}\left( \mathbf{x}_{t}-\frac{\beta_{t}}{\sqrt{1-\bar{\alpha}_{t}}}\mathbf{\epsilon}_{ \theta}(F(\mathbf{x}_{t}),t)\right)\quad\quad\mathbf{\Sigma}_{\theta}(\mathbf{ x}_{t},t)=\gamma^{2}\cdot\beta_{t}$ $$

$F(\cdot)$ is the Frenet-Serret frame construction process based on a sequence of coordinates, and $\gamma\in[0,1]$ controls the scale of injected noise in the reverse process (analogous to sampling temperature).

Figure: Figure 1: Genie 2 architecture (top), which extends Genie to enable scaffolding on (multiple) motifs. It consists of an SE(3)-invariant encoder that transforms input features into single residue and pair residue-residue representations, and an SE(3)-equivariant decoder that updates frames based on single representations, pair representations, and input reference frames. Example inputs to the model for single- and multi-motif scaffolding problems are shown (bottom-left green box), along with the corresponding generated designs (bottom-right box). In single motif scaffolding (top row), the motif may be contiguous or non-contiguous but all inter-residue positions and orientations are defined. In multi-motif scaffolding (bottom row), inter-motif geometry is left unspecified. For input sequences, white boxes denote masked out regions corresponding to the scaffold.
Refer to caption: extracted/5618193/figure_1.png

#### SE(3)-equivariant denoiser

The core of Genie is its SE(3)-equivariant denoiser $\epsilon_{\theta}(F(\mathbf{x}_{t}),t)$, which reasons over reference frames to predict the noise injected during the forward process. Figure 1 summarizes Genie’s architecture. The denoiser consists of an SE(3)-invariant encoder, which transforms individual residue and residue-residue pair features into single and pair representations, and an SE(3)-equivariant decoder, which uses Invariant Point Attention [] to update single representations that are in turn used to update input reference frames. Final noise vectors are computed as the displacement between the translation component of the updated frames and that of the input frames. For more details refer to [].

## 3 Methods

In this work we extend Genie’s architecture and training procedure to enable motif scaffolding. We also substantially improve the core unconditional model through data augmentation and scaling.

#### Motif representation for conditional generation

Genie’s architecture naturally permits integration of conditional sequence and structure information into the diffusion process. We do so by encoding the residues of each motif as one-hot vectors and concatenating these encodings to the single residue features. We encode the structure of each motif using the pairwise distance matrix of its $C_{\alpha}$ atoms. This representation is SE(3)-invariant as it does not encode the absolute position and orientation of the motif(s), and is unlike the motif conditioning procedures of other methods (e.g., RFDiffusion and FrameFlow), which fix motif coordinates and are thus sensitive to initial placement(s).

Our approach sidesteps a challenge in multi-motif scaffolding, where the design objective leaves the relative positions and orientations of motifs unspecified. By representing motif structures using pairwise distance matrices that specify intra-motif but not inter-motif distances, Genie 2 learns to satisfy the constraints of each motif while generating self-consistent configurations of inter-motif geometries. Figure 1 illustrates the types of (multi-)motif templates that can be specified. Note that even in single motif scaffolding, a motif may be non-contiguous by comprising multiple segments. What differentiates single and multi-motif scaffolding is that inter-segment geometric relationships are specified while inter-motif relationships are not. Genie 2’s formulation does require specifying sequence length separations between motifs, either by fixing them or sampling from a distribution.

#### Training

Genie 2 is trained in a purely conditional manner with every training example constituting a (single) motif scaffolding task. Tasks are constructed by first sampling structures from our training dataset to serve as ground truths. A target motif is then constructed for each structure by sampling $N_{s}$ segments totaling $N_{r}$ residues, where $N_{s}\sim\mathcal{U}(1,4)$, $N_{r}\sim\mathcal{U}(\lfloor 0.05N\rfloor,\lceil 0.5N\rceil)$, and $N$ is again the length of the protein. The starting positions and lengths of motif segments are randomly chosen subject to the number of motif residues totalling $N_{r}$. Algorithm 1 describes the task sampling procedure in more detail. We initially experimented with training on varying ratios of conditional and unconditional tasks but found that higher proportions of conditional tasks generally yielded better performance on both types of tasks, and thus switched to purely conditional training. We include an analysis of this behavior in Appendix A.4. Due to computational constraints, we limit sequence length to 256 during training; however, Genie 2 is capable of generating proteins longer than 256 residues. In addition, we do not train on multi-motif scaffolding as our input representation permits under-specification of geometric relationships as an inference-time choice. Genie 2’s performance on multi-motif scaffolding thus represents out-of-distribution generative generalization.

Figure: Algorithm 1 Motif construction for conditional training task

#### Data augmentation

Diffusion models require large datasets to robustly capture complex distributions. Generative protein models have thus far relied on training on experimentally determined protein structures from the Protein Data Bank (PDB) [, ]. Despite the enormous experimental efforts that have gone into assembling the PDB, its size remains limited to ~20,000 proteins of relevant lengths. With the development of highly accurate protein structure prediction, we hypothesized that augmenting Genie training with confidently predicted protein structures could boost its performance by expanding the space of observed folds beyond those present in the PDB. Consequently, we train Genie 2 using the AlphaFold database (AFDB) [], which consists of approximately 214M AlphaFold 2 predictions spanning nearly the entirety of UniProt []. As AFDB is highly structurally redundant, we use a subsampled version [] that applies FoldSeek [] to cluster entries based on structural similarity. We start with all cluster representatives from the FoldSeek-clustered database and then filter them using a pLDDT threshold of >80, to enrich for highly confident predictions, and a maximum sequence length of 256. This results in 588,570 structures. To our knowledge, Genie 2 is the first protein diffusion model to train on AFDB.

#### Loss function

We minimize the loss function below, which computes the mean squared error between predicted and ground truth noise:

$$ $\displaystyle L(\theta)$ $\displaystyle=\mathbb{E}_{t,x_{0},\epsilon}\left[\frac{1}{N}\sum_{i=1}^{N} \left\|\epsilon_{t}^{i}-\epsilon_{\theta}^{i}(F(x_{t}),t)\right\|^{2}\right]$ (4) $\displaystyle=\mathbb{E}_{t,x_{0},\epsilon}\left[\frac{1}{|\mathcal{M}|+| \mathcal{S}|}\left(\sum_{i\in\mathcal{M}}\left\|\epsilon_{t}^{i}-\epsilon_{ \theta}^{i}(F(x_{t}),t)\right\|^{2}+\sum_{i\in\mathcal{S}}\left\|\epsilon_{t}^ {i}-\epsilon_{\theta}^{i}(F(x_{t}),t)\right\|^{2}\right)\right]$ (5) $$

where $\mathcal{M}$ and $\mathcal{S}$ are the set of motif and scaffold residue indices, respectively. Under this construction, motifs are enforced as a soft constraint, ensuring that the model is responsive to motif specifications while also designing the protein as a whole.

## 4 Unconditional Protein Generation

To systematically assess Genie 2 and competing methods on unconditional protein generation, we conduct two sets of analyses. First, we assess methods without accounting for length while restricting the longest designed protein to 256 residues (Section 4.2). This reflects Genie 2’s in-distribution generative power since it is trained on proteins up to 256 residues long. Second, we assess methods in a length-specific manner up to 500 residues (Section 4.3) to quantify Genie 2’s out-of-distribution generative capabilities. In both analyses, we rely on the evaluation metrics described in Section 4.1.

We compare Genie 2 to Chroma, FrameFlow and RFDiffusion. The latter is widely perceived as the current state-of-the-art protein design model and has been extensively validated. A more recent model, Proteus, asserts some gains on designability over RFDiffusion at the cost of lower diversity. Unfortunately, the code for Proteus is not publicly available, precluding direct comparison. Nonetheless, based on Proteus’ reported metrics, we believe that the comparison with RFDiffusion is sufficient to establish Genie 2 as the new state-of-the-art model. We note that while Chroma contains a built-in sequence design network, we find it to underperform ProteinMPNN and so exclude it; instead we adopt the same evaluation pipeline across all methods.

### 4.1 Evaluation metrics

#### Designability

A structure that can be plausibly realized by some protein sequence is one that is designable. To determine if a structure is designable we employ a commonly used pipeline [] that computes in silico self-consistency between generated and predicted structures. First, a generated structure is fed into an inverse folding model (ProteinMPNN []) to produce 8 plausible sequences for the design. Next, structures of proposed sequences are predicted (using ESMFold []) and the consistency of predicted structures with respect to the original generated structure is assessed using a structure similarity metric (TM-score [, ]). Using this pipeline, we consider a generated structure to be designable if it is within 2Å RMSD of the most similar predicted structure ($\text{scRMSD}\leq 2$) and the structure is confidently predicted (mean $\text{pLDDT}\geq 70$). Over a set, "designability" quantifies the fraction of designable structures within it. We note that designability alone can be misleading because it does not account for structural diversity–for example, a model that has mode-collapsed into a single designable structure achieves perfect designability.

#### Diversity

Complementary to designability is the (structural) diversity of a generated protein set. To quantify diversity we start by hierarchically clustering (with single linkage) the set of designable generated structures. We exclude non-designable structures as we do not expect them to be realizable and including them would thus inflate diversity. As sequence lengths vary within a set of designable structures, we utilize TMAlign [] to compute the pairwise similarities of all structures and use a TM-score threshold of 0.6 as cutoff. This implies that any pair of structures across clusters would have a TM score of at most 0.6. We then compute "diversity" as the fraction of distinct designable clusters within a set of generated structures. As the diversity metric already enforces designability of generated clusters, we find that it better reflects the generative capabilities of a model than designability. Note that diversity depends on the number of samples generated, and tends to 0 as sample size increases. In all our experiments we use a fixed sample size to enable even comparisons.

#### F1 score

Following Lin and AlQuraishi [], we compute the harmonic mean between designability ($p_{\text{structures}}$) and diversity ($p_{\text{clusters}}$) as follows:

$$ $F_{\beta}=(1+\beta^{2})\cdot\frac{p_{\text{structures}}\cdot p_{\text{clusters }}}{\beta^{2}\cdot p_{\text{structures}}+p_{\text{clusters}}}$ (6) $$

where $\beta\in\mathbb{R}^{+}$ controls the relative weighting of designability and diversity. We set $\beta=1$ and report the metric as F1 score.

#### Novelty

Beyond designability and diversity, we also quantify the novelty of generated structures with respect to reference datasets and, by extension, the known structural universe. To compute the novelty of a generated structure we again employ TM-score as our structure similarity metric and use TMAlign to compute the TM scores between a generated structure and all structures in a reference dataset. We consider a generated structure to be novel if it is designable and its TM-score to any reference structure is at most 0.5. Similar to our diversity calculations, we apply hierarchical clustering (with single linkage and a TM-score threshold of 0.6) to the set of novel structures and define "novelty" to be the fraction of distinct novel clusters within a set of generated structures. We measure novelty with respect to both the PDB and Foldseek-clustered AFDB datasets (the latter being our training dataset) and term these measures "PDB novelty" and "AFDB novelty", respectively.

### 4.2 In-distribution performance analysis

**Table 1: Unconditional generative performance of structure-based diffusion models.**
| Method | Designability | Diversity | $F_{1}$ | PDB Novelty | AFDB Novelty |
| --- | --- | --- | --- | --- | --- |
| Chroma | 0.70 | 0.51 | 0.59 | 0.13 | 0.04 |
| RFDiffusion | 0.96 | 0.63 | 0.76 | 0.26 | 0.14 |
| Genie 2 | 0.96 | 0.91 | 0.93 | 0.41 | 0.21 |

Figure: Figure 2: Visualizations of in-distribution performance on unconditional generation. (A) Secondary structure distributions of proteins generated by Chroma, RFDiffusion and Genie 2. For reference, we also include the secondary structure distribution of 1,000 structures randomly drawn from AFDB (far right). (B) Secondary structure distributions of proteins generated by Genie 2 when sampling noise scale ($\gamma$ in equation (3)) is set to 0 (left) and 1 (right). (C) Self-consistency results on 1,000 randomly chosen structures from the PDB and clustered AFDB datasets.
Refer to caption: extracted/5618193/app_uncond_result_secondary.png

We assess Genie 2, Chroma, and RFDiffusion by generating 5 structures of every length ranging from 50 to 256 residues (1,035 structures in total). We omit FrameFlow here since it is trained using a maximum sequence length of 128, but include direct comparisons with FrameFlow in Section 4.3. Table 1 summarizes the performance of all methods on our key metrics. Relative to RFDiffusion and Chroma, Genie 2 achieves comparable designability and much higher diversity and novelty. This suggests that as a core unconditional model, Genie 2 best captures foldable protein structure space, and may thus serve as a superior engine for downstream sampling-based protein design tasks [, ].

In Figure 2A, we visualize the secondary structure distribution of generated proteins. While all methods yield a wide range of secondary structure elements, the resulting distributions are biased (relative to AFDB), with beta strand-containing structures (top left of distribution) and loop elements (bottom left) being generally underrepresented. There are multiple possible reasons for this bias. First, the high frequency of helices in the training dataset leads to models that favor generating helical structures. Second, alpha helices are likely easier to generate than beta sheets as they involve largely local interactions while sheets may involve long-range interactions. Third, we assess Genie 2 using low temperature sampling as it yields better results, but this may shift the model from its learned distribution. We test this hypothesis by visualizing the distribution of secondary structures generated by Genie 2 under a normal temperature ($\gamma=1$) in Figure 2B. We observe that the resulting distribution is in fact consistent with that of the clustered AFDB dataset.

This raises the question of whether low temperature sampling is necessary or, alternatively, why it helps improve Genie 2’s performance. To investigate this we ran our self-consistency pipeline on 1,000 randomly chosen structures from the clustered AFDB dataset. We found that only 42.4% of these structures are designable. When our designability criteria is relaxed to $\text{scTM}\geq 0.5$ and $\text{pLDDT}\geq 70$, the percentage of designable structures increases to 80.2% (Figure 2C). For comparison, among PDB structures 87.2% are considered designable by our original criteria. This suggests that while AF-predicted structures are globally reliable (with $\text{scTM}\geq 0.5$), local atomic details remain much less accurate. Since we use AFDB for training, this might explain why designability is low at normal temperature sampling, as lower temperatures appear to bias Genie 2 towards higher fidelity. We note that ProteinMPNN was exclusively trained on the PDB and thus we cannot rule out the possibility that the apparent discrepancy in designability between the PDB and AFDB is due to a bias in ProteinMPNN towards the PDB.

### 4.3 Length-based performance analysis

Figure: Figure 3: Assessment of methods by sequence length. For each method/sequence length combination, we generate 100 structures. (A) Box-and-whisker plots of scRMSDs between generated structures and their most similar ESMFold-predicted structures. Asterisks (*) indicate that sequence lengths exceed the maximum seen during training. (B-C) Plots of designability (B) and diversity (C) as a function of sequence length. (D) Example structures generated by Genie 2.
Refer to caption: extracted/5618193/figure_2.png

We next assess generative performance in a length-dependent manner. For a subset of sequence lengths ranging from 50 to 500 residues, we generate 100 structures and assess them using our design metrics. Figure 3A shows the scRMSD distribution across sequence lengths while Figures 3B and 3C plot designability and diversity as a function of sequence length, respectively. At nearly all assessed lengths, Genie 2 has comparable designability to RFDiffusion but higher diversity. For short proteins (<200 residues), Genie 2 exhibits considerably higher diversity (doubling that of RFDiffusion at 100 residues), which is noteworthy as shorter lengths constitute smaller design spaces.

Generative models generally struggle with creating larger proteins due to their increased complexity. As sequence length increases, designability decreases and in turn so does diversity, likely because diversity depends on the number of designable proteins. Larger protein lengths should in principle permit greater diversity but they are harder to generate. Nonetheless, despite having been trained on monomers of at most 256 residues, Genie 2 can generate 500-residue structures with comparable or better performance than competing methods. For reference, RFDiffusion uses a crop size of 384 during training while Chroma trains on even larger proteins (>500 residues) owing to its efficient graph neural network. Figure 3D shows examples of Genie 2 designed structures of varying lengths.

## 5 Motif Scaffolding

In this section we assess Genie 2 on single motif scaffolding and compare it to RFDiffusion on the same set of design tasks. A more recent method, FrameFlow, does assert superiority over RFDiffusion on single motif scaffolding, but its scaffolding-capable code is unfortunately unavailable, precluding direct comparison. We additionally assess Genie 2 on multi-motif scaffolding using a suite of 6 multi-motif tasks that we curated for this assessment.

Figure: Figure 4: Comparison of Genie 2 and RFDiffusion on single-motif scaffolding. (A) Performance of Genie 2 and RFDiffusion across 24 single-motif scaffolding tasks. Inset (top right) shows a scatter plot of the (unique) success rate of Genie 2 vs. RFDiffusion; each point represents a scaffolding task. Summary statistics are shown in table (left). Example designs are shown (bottom) for successful task 3IXT (green) as well as failed task 4JHW (red). Scaffolds (white), motifs (blue), and unsatisfied sought motifs (red) are overlaid. (B) Plot of number of unique successes as a function of sample size.
Refer to caption: extracted/5618193/figure_3.png

### 5.1 Evaluation metrics

Motif scaffolding problems consist of sequence and structure constraints on motif(s) plus length (min/max) constraints on scaffolds and the overall protein. To solve a motif scaffolding problem, we first sample a constraint-satisfying length for each scaffold segment while ensuring that total protein length is also within specifications. This information, together with the sequence and structure of motif(s), is passed as conditions to Genie 2. We quantify success using the criteria of RFDiffusion, which requires that generated structures achieve $\text{scRMSD}\leq 2$Å, $\text{pLDDT}\geq 70$, and $\text{pAE}\leq 5$ to be considered designable, and for designed motif(s) to have backbone $\text{RMSD}\leq 1$Å with respect to each motif to be considered constraint satisfying.

While most previous studies, including RFDiffusion, use success rate as the evaluation metric, we find that this tends to inflate performance, as it is possible to achieve high success rates by repeatedly generating only one or a few successful designs, i.e., while suffering from mode collapse. Instead, and similar to [], we cluster successful designs based on structure similarity and report the number of unique successes. This approach better balances designability with diversity when assessing motif scaffolding performance. We use hierarchical clustering with single linkage and a TM-score threshold of 0.6. For each motif scaffolding problem, we sample 1,000 structures.

Figure: Figure 5: Performance of Genie 2 on multi-motif scaffolding tasks. (A) Successful designs for task 1PRW_four (scaffolding with four $\text{Ca}^{2+}$ ion binding sites) and 4JHW+5WN9 (scaffolding with RSV-F site II epitope and RSV-G 2D10 epitope). Scaffolds are in grey and distinct motifs are colored differently. (B) (Top) Successful design for multi-epitope immunogen. (Bottom) Individual epitope designs superposed over target structures (red).
Refer to caption: extracted/5618193/figure_4.png

### 5.2 Single-motif scaffolding

For evaluation, we use a previously published motif scaffolding benchmark [] comprising 25 tasks curated from six recent publications. We exclude one task, 6VW1, as its motif consists of segments from multiple protein chains, a requirement not supported by Genie 2. Figure 4A summarizes the performance of Genie 2 and RFDiffusion across the remaining 24 tasks. For 22 tasks, Genie 2 yields a similar or larger number of unique designs than RFDiffusion. Genie 2 also solves task 5WN9 that involves scaffolding the RSV G-protein 2D10 site [], while RFDiffusion does not. We speculate that Genie 2 solves this task because it is capable of generating more diverse designs. We show examples of successful designs in Figure 4A with more in Appendix D.5. In addition, we observe that the performance gap (in terms of number of unique successes) between Genie 2 and RFDiffusion widens as sample size increases (Figure 4B), suggesting that Genie 2 is capturing a larger and more diverse structure space than RFDiffusion. Genie 2 does fail on one problem, 4JHW, that RFDiffusion also fails on, which involves scaffolding the RSV F-protein site-0. To better understand this failure case, we visualize the two closest designs (red box in Figure 4) and observe that while Genie 2 yields designable structures it does not satisfy the motif constraints.

### 5.3 Multi-motif scaffolding

To assess the multi-motif scaffolding capabilities of Genie 2, we curate from the literature a set of 6 scaffolding tasks that each require multiple motifs: designing an immunogen with two epitopes [], scaffolding two $\text{Ca}^{2+}$ binding sites (four EF hand motifs) [, ], scaffolding two binding motifs to PD-1 protein [], scaffolding $\text{Cl}^{-}$ and $\text{Ni}^{2+}$ binding sites [, ], and designing a binder of two different proteins, IL-2 receptor $\beta\gamma_{c}$ heterodimer (IL-2R$\beta\gamma_{c}$) and IL-2R$\alpha$ [, ]. More details on this set of tasks are included in Appendix B. The set is meant to reflect the breadth of potential protein design tasks, including immunogen, binder, and enyzme design.

Genie 2 solves 4 of the 6 tasks. Figure 5A shows successful designs of task 1PRW_four (scaffolding with four $\text{Ca}^{2+}$ ion binding sites) and 4JHW+5WN9 (scaffolding with RSV-F site II epitope and RSV-G 2D10 epitope). More results are included in Appendix E. In addition to our benchmark set, we apply Genie 2 to a multi-motif task proposed in a concurrent preprint []. This task scaffolds an immunogen containing three unique epitopes from the respiratory syncytial virus (RSV) fusion protein. Genie 2 solves the task with only 1,000 samples. Figure 5B shows an example design.

## 6 Limitations and Future Work

Genie 2 achieves state-of-the-art performance on both unconditional generation and motif scaffolding. Yet, its sampling time is longer than that of other methods, requiring 1,000 denoising iterations vs. 100 (FrameFlow), 500 (Chroma), and 50 (RFDiffusion). Appendix F provides a summary of sampling times across sequence lengths. One future direction for Genie 2 is to improve its sampling efficiency in both unconditional protein generation and motif scaffolding. Genie 2 also employs triangular multiplicative update layers, introduced in AlphaFold 2. These layers are computationally expensive with $O(N^{3})$ scaling, thus disproportionately affecting larger design tasks. A second future direction is thus to reduce the time and space complexity of the Genie 2 architecture, to enable generation of and training on larger proteins.

## Appendix A Additional Details on Genie 2

### A.1 Hyperparameter choices

In Table 2 we detail the key hyperparameters of the Genie 2 architecture and highlight differences from the original Genie model. For Genie 2, we increase input embedding and single representation dimensions as we found this improves performance without substantially impacting training speed. Due to the increase in model complexity, Genie 2 consists of 15.7M trainable parameters, $\sim$4x the original Genie architecture. However, Genie 2 remains four times smaller than RFDiffusion, which has 59.8M trainable parameters.

**Table 2: Key hyperparamters of Genie and Genie 2. Updated values are indicated in bold.**
| Hyperparameter | Genie | Genie 2 |  |
| --- | --- | --- | --- |
| Number of parameters | 4.1M | 15.7M |  |
| Input embedding dimension | Residue index | 128 | 256 |
| Chain index | - | 64 |  |
| Diffusion timestep | 128 | 512 |  |
| Representation dimension | Single representation | 128 | 384 |
| Pair representation | 128 | 128 |  |
| SE(3)-equivariant decoder | Number of IPA layers | 5 | 8 |

### A.2 Training

For training, we use the Adam [] optimizer with a constant learning rate of $10^{-4}$. We train Genie 2 using data parallelism on 8 Nvidia A100 GPUs with an effective batch size of 48. We train the model for 40 epochs ($\sim$5 days) for a total of $\sim$960 GPU hours. In comparison, RFDiffusion is initialized with pretrained weights from RoseTTAFold [], whose training requires 64 Nvidia V100 GPUs for 4 weeks. Training of RFDiffusion takes 3 days on 8 Nvidia A100 GPUs. Hence, Genie 2 requires much less computational resources to train than RFDiffusion.

### A.3 Sampling

To improve designability, we adjusted the sampling noise scale ($\gamma$ in Equation (3)) to trade diversity for designability. We set $\gamma=0.6$ and $\gamma=0.4$ for unconditional generation and motif scaffolding, respectively, as these settings provided the best results. Moreover, for motif scaffolding, we use the checkpoint at epoch 30 since it gives slightly better performance.

### A.4 Effect of varying conditional task ratio

We experimented with varying the frequency of conditional vs. unconditional tasks during training (0.0, 0.2, 0.5, 0.8, and 1.0). Due to computational constraints, we trained models only up to 10 epochs. Although models do not fully converge, we believe the trends are still indicative of final performance. To evaluate models, we follow the same procedure from Section 4.2 when assessing unconditional generation performance: for each model, we generate 5 samples per sequence length ranging from 50 to 256 residues. For single-motif scaffolding evaluation, we use the pipeline described in Section 5.2, but sample only 100 designs per motif scaffolding problem to conserve computational costs.

Table 3 summarizes the performance of these models on both unconditional protein generation and single motif scaffolding. As the conditional task ratio increases, motif scaffolding performance generally improves, with the best performance achieved when the conditional task ratio equals 1. Surprisingly, unconditional generation performance fluctuates but is ultimately also maximized when conditional tasks are exclusively sampled. As a result we use a conditional task ratio of 1 during all training runs.

**Table 3: Unconditional generation and motif scaffolding performance by conditional task ratio. Successes denote total number of unique successes across all problems.**
| Ratio | Unconditional Generation | Motif Scaffolding |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Designability | Diversity | $F_{1}$ | Solved | Successes |  |
| 0.0 | 0.858 | 0.771 | 0.812 | 1 | 6 |
| 0.2 | 0.892 | 0.752 | 0.816 | 14 | 101 |
| 0.5 | 0.740 | 0.649 | 0.692 | 13 | 98 |
| 0.8 | 0.865 | 0.783 | 0.822 | 18 | 179 |
| 1.0 | 0.898 | 0.802 | 0.847 | 19 | 202 |

## Appendix B Multi-Motif Scaffolding Benchmark

In Table 4, we provide detailed configurations for each multi-motif scaffolding task. We name each problem using the names of PDB structures that contain the motif(s) used in the problem. Additional postfixes are added to distinguish between problems whose motifs come from the same PDB structure. In the third column ("configuration"), we provide a detailed input specification for each multi-motif scaffolding problem. Each bolded part denotes a motif segment, including its location in the PDB structure. For example, "5WN9/A170-189{2}" in problem 4JHW+5WN9 indicates that the motif segment comes from residue 170 - 189 of Chain A in the protein 5WN9, and "2" (in curly bracket) indicates that this motif segment belongs to the second motif. Each non-bolded part denotes a scaffold segment with minimum and maximum lengths specified. At sampling time, each scaffold length is sampled within this range. For example, "10-40" in problem 4JHW+5WN9 indicates that the scaffold has a length between 10 and 40 (inclusive). The last column ("Total length") specifies the minimum and maximum length requirements for the whole sequence.

**Table 4: The benchmark set of multi-motif scaffolding problems.**
| Name | Description | Configuration | Length |
| --- | --- | --- | --- |
| 4JHW+5WN9 [] | Two epitopes | 10-40, 4JHW/F254-278{1}, 20-50, 5WN9/A170-189{2}, 10-40 | 85-175 |
| 1PRW_two [, ] | Two 4-helix bundles | 5-20, 1PRW/A16-35{1}, 10-25, 1PRW/A52-71{1}, 10-30, 1PRW/A89-108{2}, 10-25, 1PRW/A125-144{2}, 5-20 | 120-200 |
| 1PRW_four [, ] | Four EF-hands | 5-20, 1PRW/A21-32{1}, 10-25, 1PRW/A57-68{2}, 10-25, 1PRW/A94-105{3}, 10-25, 1PRW/A125-144{4}, 5-20 | 88-163 |
| 3BIK+3BP5 [] | Two PD-1 binding motifs | 5-15, 3BIK/A121-125{1}, 10-20, 3BP5/B110-114{2}, 5-15 | 30-60 |
| 3NTN [, ] | Two 3-helix bundles | 3NTN/A342-348{1}, 10-10, 3NTN/A367-372{2}, 10-20, 3NTN/B372-367{2}, 10-10, 3NTN/B348-342{1}, 10-20, 3NTN/C342-348{1}, 10-10, 3NTN/C367-372{2} | 89-109 |
| 2B5I [, ] | Two binding sites | 5-15, 2B5I/A11-23{2}, 10-20, 2B5I/A35-45{1}, 10-20, 2B5I/A61-72{1}, 5-15, 2B5I/A81-95{2}, 20-30, 2B5I/A119-133{2} | 116-166 |

For problem 3NTN, the original PDB structure is a homotrimer. It consists of three helices, which together form a binding site for $\text{Ni}^{2+}$ ion and a binding site for $\text{Cl}^{-}$ ion. When setting up this multi-motif scaffolding problem, we are interested in whether it is possible to combine two binding sites (formed by multiple chains) into a single-chain protein. One possible reason that Genie 2 fails on this task might be that this problem is not solvable given the current specification.

## Appendix C Additional Results on Unconditional Protein Generation

### C.1 Length-based performance analysis using scTM

We provide additional assessments of Genie 2 and competing methods using a second designability metric, the self-consistency TM score (scTM). scTM is computed using the same pipeline as scRMSD, described in Section 4.1, except using TM score to measure the structural distance between a generated structure and its most similar ESMFold-predicted structure. scTM is a less stringent metric than scRMSD since TM score is less sensitive to minor structural variations. Figure 6A visualizes the distribution of scTM by sequence length for Genie 2 and competing methods, while Figures 6B and 6C visualize scTM-based designability and diversity as a function of sequence length, respectively. Here, a structure is considered as scTM-based designable if it satisfies both $\text{scTM}>0.5$ and $\text{pLDDT}>70$. Diversity is computed using the same clustering procedure described in section 4.1. Overall trends remain consistent with our main results.

Figure: Figure 6: Assessment of Genie 2 and competing methods by sequence length using scTM as the designability metric. 100 structures are generated per sequence length and method. (A) Distribution of self-consistency TM between generated structures and the most similar ESMFold-predicted structures. Asterisk (*) indicates that the sampled sequence length is beyond the maximum sequence length sampled at training time. (B) Plot of scTM-based designability (percentage of scTM-designable structures) as a function of sequence length. (C) Plot of scTM-based diversity (percentage of unique scTM-based designable clusters) as a function of sequence length.
Refer to caption: extracted/5618193/app_sctm_length.png

### C.2 Additional examples of designable clusters by Genie 2

Figure: Figure 7: Examples of Genie 2 designed structures with in-distribution sequence lengths (within the maximum sequence length of 256 set at training time).
Refer to caption: x1.png

Figure: Figure 8: Examples of Genie 2 designed structures with out-of-distribution sequence lengths.
Refer to caption: x2.png

## Appendix D Additional Results on Single-Motif Scaffolding

### D.1 Evaluation details

Watson et al. [] asserts that RFDiffusion achieves a higher success rate when the noise scale is set to 0; however, this success rate does not account for the diversity of designed structures. To ensure a fair comparison, we first assessed the performance of RFDiffusion with noise scale set to 0 and 1. For each motif scaffolding problem, we sampled 100 structures per problem and evaluated them using the same pipeline as in Section 5.2. Figure 9 visualizes the number of unique successes by motif scaffolding problems. We observe that RFDiffusion solves more motif scaffolding problems with more diverse designs when the noise scale is set to 1. Thus, to maximize RFDiffusion’s performance, we compare Genie 2 with RFDiffusion with a noise scale of 1 throughout this work.

Figure: Figure 9: Performance of RFDiffusion with a noise scale of 0 and 1 across 24 single-motif scaffolding tasks. Inset (top right) shows a scatter plot of the (unique) success rate of RFDiffusion with a noise scale of 1 versus RFDiffusion with a noise scale of 0; each point represents a scaffolding task. Summary statistics are shown in table (left).
Refer to caption: extracted/5618193/app_rfd_noise.png

### D.2 Number of unique successes

**Table 5: Number of unique successes (out of 1,000 structures) generated by Genie 2 and RFDiffusion on each single-motif scaffolding task.**
| Name | Genie 2 | RFDiffusion |
| --- | --- | --- |
| 6E6R_long | 415 | 381 |
| 6EXZ_long | 326 | 167 |
| 6E6R_med | 272 | 151 |
| 1YCR | 134 | 7 |
| 5TRV_long | 97 | 23 |
| 6EXZ_med | 54 | 25 |
| 7MRX_128 | 27 | 66 |
| 6E6R_short | 26 | 23 |
| 5TRV_med | 23 | 10 |
| 7MRX_85 | 23 | 13 |
| 3IXT | 14 | 3 |
| 5TPN | 8 | 5 |
| 7MRX_60 | 5 | 1 |
| 1QJG | 5 | 1 |
| 5TRV_short | 3 | 1 |
| 5YUI | 3 | 1 |
| 4ZYP | 3 | 6 |
| 6EXZ_short | 2 | 1 |
| 1PRW | 1 | 1 |
| 5IUS | 1 | 1 |
| 1BCF | 1 | 1 |
| 5WN9 | 1 | 0 |
| 2KL8 | 1 | 1 |
| 4JHW | 0 | 0 |

### D.3 Performance as a function of sample size

Figure: Figure 10: Single-motif scaffolding performance as a function of sample size by problem. The y-axis represents the number of unique successes and is rescaled for each problem.
Refer to caption: extracted/5618193/app_single_motif_each_num_samples.png

### D.4 Scatterplot of scRMSD versus motif backbone RMSD

Figure: Figure 11: Scatterplot of scRMSD versus motif backbone RMSD by problem, where each point represents one generated structure. The green box denotes the region with $\text{scRMSD}\leq 2$Å and motif backbone $\text{RMSD}\leq 1$Å, which are the two deciding factors of a design’s success.
Refer to caption: extracted/5618193/app_single_motif_scatter.png

### D.5 Additional examples of successful motif scaffolding designs by Genie 2

Figure: Figure 12: Examples of successfully designed structures by Genie 2 for six single-motif scaffolding tasks. Scaffolds (white) and motifs (blue) are overlaid.
Refer to caption: x3.png

## Appendix E Additional Results on Multi-Motif Scaffolding

### E.1 Number of unique successes

**Table 6: Number of unique successes (out of 1,000 structures) generated by Genie 2 on each multi-motif scaffolding task.**
| Name | Number of Unique Successes |
| --- | --- |
| 3BIK+3BP5 | 17 |
| 1PRW_four | 11 |
| 1PRW_two | 8 |
| 4JHW+5WN9 | 4 |
| 2B5I | 0 |
| 3NTN | 0 |

### E.2 Additional examples of successful designs by Genie 2

Figure: Figure 13: Examples of successfully designed structures by Genie 2 for three multi-motif scaffolding tasks. Scaffolds are in grey and different motifs are colored differently. For 4JHW+4WN9, all four unique successes are shown in Figure 5.
Refer to caption: extracted/5618193/example_multi.png

Figure 13 shows the successful designs of three multi-motif scaffolding problems. The designs for 3BIK+3BP5 exhibit diverse secondary structures, including structures containing strands (first), helices (second and fourth), and loops (third). For tasks 1PRW_two and 1PRW_four, the loops of EF-hand motifs that interact with substrates are well exposed to the surface in all designs. The structures are diverse and clearly different from the original 1PRW, with the EF-hand motifs distributed asymmetrically throughout the structure. In the designs of 1PRW_two, the 4-helix bundles are also in different relative orientations compared to the original 1PRW. For example, in the second design, the loops of two bundles face the same side. These diverse and novel designs open the possibility of creating more stable or functional proteins with the desired motifs.

## Appendix F Sampling Time

In this section, we compare the generation times of Genie 2, RFDiffusion, FrameFlow, and Chroma at different lengths. We use a single A6000 GPU (48GB memory) and average the inference time of a single sample over 10 runs. For RFDiffusion, we use the self-conditioning sampler and exclude pLDDT and amino acid prediction for fair comparison. For Chroma, we use unconditional monomer sampling and exclude amino acid prediction. We use the simple profiler from PyTorch Lightning [] to profile inference function calls of FrameFlow and Genie.

**Table 7: Sampling time of different methods for proteins of different lengths.**
| Method | Parameters | Timesteps | Length |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 50 | 100 | 200 | 300 | 400 | 500 |  |  |  |
| Genie 2 | 15.7M | 1000 | 42.2 | 49.4 | 71.4 | 134 | 236 | 353 |
| RFDiffusion | 59.8M | 50 | 18.7 | 21.4 | 41.2 | 80.1 | 137 | 214 |
| FrameFlow | 17.4M | 100 | 3.45 | 4.33 | 6.50 | 9.84 | 13.8 | 18.5 |
| Chroma | 18.5M | 500 | 22.0 | 22.6 | 29.0 | 35.4 | 41.8 | 48.5 |

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. Previous methods tend to generate more alpha-helices than beta-strands. In Genie 2, what is the distribution of generated proteins among the structural classes of all-alpha, all-beta, alpha+beta, and alpha/beta? Does the designability of the generated proteins depend on the ratio of alpha-helices?

2. Why was a batch size of 256 chosen for training Genie 2? Both Proteus and Chroma use batch sizes larger than 500, yet they exhibit significantly different performance levels. Therefore, without additional experimental results, it is unclear whether batch size is the main reason for Genie 2's significantly worse performance compared to Proteus on long proteins. 

2. Please address the concerns and questions raised in the Weaknesses section.
soundness: 2
strengths: 1. For unconditional generation, Genie 2 performs better than Genie and other methods when generating short proteins.
2.  Genie 2 also supports multi-motif scaffolding.
summary: The authors introduce Genie 2, an extension of Genie that incorporates architectural innovations and extensive data augmentation. Genie 2 supports both unconditional generation and multi-motif scaffolding. The authors claim that Genie 2 achieves state-of-the-art performance, outperforming all known methods on key design metrics.
weaknesses: 1. The authors overstate the performance of Genie 2 in the Abstract. While they claim that Genie 2 achieves state-of-the-art performance and outperforms all known methods on key design metrics, Figure 3 Panel B shows that Genie 2 performs significantly worse than Proteus on long proteins.

2. The comparison benchmarking is insufficient. 

    a. Unconditional generation: The comparison lacks the inclusion of recent methods such as MultiFlow [1], CarbonNovo [2], and FoldFlow2 [3], which have demonstrated significant performance improvements. 

    b. Single-Motif Scaffolding: Methods like Chroma and FoldFlow2 should be included for comparison. While Chroma is included in the benchmarking for unconditional generation, it is excluded in the scaffolding generation comparisons. Clarification on this omission is needed.
3. The methodological contributions appear incremental. The authors extend Genie's architecture and training procedure to enable motif scaffolding, with most modifications focusing on adapting input features. Additionally, Genie 2 incorporates triangular multiplicative updates, which have already been shown to be effective in previous works such as Proteus [4], CarbonNovo [2], and FoldFlow2 [3].
4. Insufficient Analysis of Multi-Motif Scaffolding: Despite claiming multi-motif scaffolding as a key contribution, the paper provides limited analysis in this area. Interesting analysis would be :

    a. In failed cases, is the failure due to incorrect single motif design or inaccuracies in the relative positioning of motifs?

    b. Does the success rate of scaffolding depend on the number of motifs being used?

 5.  The method requires significantly more inference steps than previous methods, indicating low sampling efficiency.


[1] Campbell et al. “Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design”. ICML 2024. https://proceedings.mlr.press/v235/campbell24a.html

[2] Ren et al. “CarbonNovo: Joint Design of Protein Structure and Sequence Using a Unified Energy-based Model”. ICML 2024. https://proceedings.mlr.press/v235/ren24e.html

[3] Huguet et al. “Sequence-Augmented SE(3)-Flow Matching for Conditional Protein Backbone Generation”. https://arxiv.org/abs/2405.20313

[4] Wang et al. “Proteus: Exploring Protein Structure Generation for Enhanced Designability and Efficiency”. ICML 2024. https://proceedings.mlr.press/v235/wang24bi.html

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: At a couple places in the paper (line 806, line 872), the authors note that certain (earlier) model checkpoints were selected for evaluation due to better performance. How were these checkpoints selected?

It seems that the Chroma conditioner framework should admit single- and multi-motif scaffolding as described in this paper. Was there a reason this model was not benchmarked in these scenarios?

How important is the sequence information for motif conditioning? In the absence of this information, would the model still produce viable solutions to the motif scaffolding problems?
soundness: 3
strengths: The authors detail an extension of the Genie model for motif-conditioned protein structure generation. The incorporation of motif information is clearly explained and quite simple, and importantly seems to work quite well empirically.

Comparisons to existing methods for unconditional and motif-conditioned generation are clearly presented and fairly thorough. There are many protein structure diffusion models and the authors have done a nice job presenting results for some of the most prominent.
summary: The paper describes Genie 2, an extension of the protein structure diffusion model Genie for motif-conditioned generation. The method is extensively benchmarked against alternative methods for unconditional and motif-conditioned diffusion models, and seems perform comparatively well.
weaknesses: The authors highlight functional protein design as a motivation for this work, and assert that functional protein design tasks can often be reduced to scaffolding of known motifs. While this may be true for simple enzymes, many functional proteins of therapeutic and industrial interest are considerably more complex (e.g., involving conformational changes or inter-domain coordination). Without experimental validation of the results presented here, it is difficult to know whether the proposed method would meaningfully improve functional design outcomes.

Given the relationship between this work and the original Genie paper, it is surprising to not see any direct comparisons between Genie 2 and the original model. For instance, how would the original Genie architecture trained on the PDB compare with the results presented in Table 1? It seems that the largest improvements came from expanding the training dataset to include AFDB, but it would be useful to see more detailed ablations of the architectural, data, and training objective changes, and the interplay between these (if there exists any).

The analysis of low-temperature sampling is quite hard to follow as currently presented. It is not clear that the authors have used low-temperature sampling for the results presented in Table 1 until the following section analyzing secondary structure distributions. Further, in this section (line 323), the authors cite a finding that low-temperature sampling yielded better results, but do not provide the results of this analysis.

The result about “designability” of AFDB structures is interesting, but not very convincing in its current form. Rather than summarizing the results of changing from scRMSD to scTM, the authors should present the full distribution of values for both of these metrics. The authors note that ProteinMPNN may be biased towards the PDB; this hypothesis could be tested by using ESM-IF1, which has been trained on AlphaFold2-predicted structures.

Much of the evaluation presented in this paper depends on ESMFold as an oracle for structure prediction. While this has become a standard practice, as noted by the authors, it would be useful for readers to include some discussion of the shortcomings and potential biases introduced by this approach.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: Similar to what was mentioned in the weaknesses section.
soundness: 2
strengths: 1. Technical Innovation: The paper introduces a clever representation for multi-motif scaffolding that uses distance matrices to specify intra-motif but not inter-motif distances, allowing flexible placement of functional sites.
2. Comprehensive Evaluation: The authors provide thorough comparisons with existing methods like RFDiffusion, FrameFlow, and Chroma across multiple metrics on unconditional protein generation and single-motif scaffolding.
3. Data Innovation: The use of AlphaFold database for training data augmentation is a novel approach that expands the structural space beyond experimentally determined structures.
summary: This paper presents Genie 2, an improved protein diffusion model that extends the capabilities of its predecessor Genie. The key innovation is the addition of multi-motif scaffolding capabilities that allow designing proteins with multiple functional sites without specifying their relative positions and orientations. The model uses SE(3)-equivariant attention for the reverse diffusion process and introduces a novel motif representation approach. The authors demonstrate state-of-the-art performance in both unconditional and conditional protein generation tasks, particularly in terms of designability, diversity, and novelty.
weaknesses: 1. Training Constraints: The model is limited to training on sequences up to 256 residues in length, though it can generate longer sequences. The implications of this limitation could be better explored.
2. Multi-motif Benchmark: The benchmark set of only 6 multi-motif scaffolding problems is too small to draw statistically significant conclusions about the model's capabilities. A robust benchmark should include at least dozens of diverse cases to properly evaluate performance.
3. Multi-motif Baselines: Although previous works like RFDiffusion were not specifically designed for multi-motif scaffolding, they could still address such problems by randomly sampling relative positions and orientations between motifs. The authors should have included comparisons with these existing methods to demonstrate the advantages of their approach over such a baseline strategy.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. Table 3: The authors demonstrate the effect of conditional training ratio on small-scale, “unconverged” models. While this limitation is understandable due to time constraints, the conclusion that conditional training universally enhances performance remains speculative. More comprehensive results during the rebuttal period would strengthen the findings.
2. Low-temperature sampling (line 323): The authors state that “low temperature sampling yields better results.” Could they specify which metric(s) this “better” refers to, as it’s unclear from the context?
3. Temperature sensitivity: The authors used different temperatures (γ) for unconditional generation and motif-scaffolding, highlighting the importance of tuning this parameter. Could they include the default temperatures used for each model in the main text and clarify the impact of varying temperatures on each task? Does this adjustment impose trade-offs, and how sensitive is the model to this factor?
4. Training data balance: Is it reasonable to balance experimental structure data (PDB) and synthetic data (AFDB) for training? Are the results significantly biased towards synthetic data?
5. Secondary structure distribution (section 4.2): When evaluating secondary structure distribution, comparing to AFDB might be problematic as 1) not all baselines were trained on AFDB, and 2) AFDB may differ in structural distribution from real-world PDB data.
6. Secondary structure distribution (section 4.2): Contrary to the authors' suggestion in line 318, it seems Chroma, Proteus, and RFDiffusion appear to generate more beta strands (top left) than AFDB. Could the authors clarify their statement about underrepresented beta strands and loop elements?
7. Low-temperature sampling for AFDB (line 352): The lower designability in AFDB might explain low designability at normal temperatures for Genie 2, but it doesn’t directly explain why low-temperature sampling would mitigate this. Could the authors provide additional insights?
8. Baseline model: The authors mention the Twisted Diffusion Sampler in related work but do not include it as a baseline. Could they clarify this choice?
9. Baseline model Genie 1: Could the authors include baseline performance results for Genie 1 in unconditional generation?
10. Multi-motif scaffolding benchmark curation: This benchmark includes only proteins with <200 amino acids. Have the authors explored more complex cases with longer sequences, or are there specific challenges in curating such datasets?
11. Multi-chain generation: Although Genie 2 accepts “chain index” as an input feature, it does not support multi-chain generation with motif-scaffolding. What limits this implementation?
12. Clarifications on some claims:
    - The authors claim Genie 2 “better captures protein structure space” and covers a “larger and more diverse protein structure space.” However, this aspect has not been directly evaluated. Current results only show higher diversity, not necessarily “broader structure space coverage” compared to Genie 1, especially given the smaller sample size examined (1,035 structures, line 309).
    - In Equation (5), the authors claim that “motifs are enforced as a soft constraint, ensuring responsiveness to motif specifications while designing the protein as a whole.” However, the current loss does not distinguish explicitly between motifs and scaffolds (e.g., no separate regularization for motif losses). The authors might want to suggest the benefits of jointly learning the motif and scaffolds coordinates and further clarification could strengthen the interpretation.
soundness: 3
strengths: **[Clarity and Quality]**

- The manuscript is clearly structured and easy to follow, with a comprehensive background review covering sequence, structure, and co-design, as well as motif scaffolding. This allows readers to understand the main concepts without needing to refer to external sources.
- The authors conducted sufficient experiments to evaluate empirical performance of Genie 2 on both the unconditional generation and scaffolding, comparing it with strong SOTA baselines such as RFDiffusion and FrameFlow.

**[Originality and Significance]**

- The authors extend motif scaffolding to design multiple independent motifs, supported by a benchmark problem set, which could benefit protein generation in more complex protein machinery.
- By encoding motif conditions as node and pair embeddings, the authors provide an SE3-invariant and flexible approach for representing motif information.
summary: This work builds on Genie 1 by adding motif scaffolding, enabling the generation of protein scaffolds that support specified motifs - an important step toward the ultimate goal of designing functional proteins.

The authors developed a simple yet effective method to encode motif information as node and pair conditions, enabling the generation of protein structures that closely adhere to targeted motif cores. They also extend the single-motif problem to multi-motif generation, exploring whether the model can generate proteins with independent functions, pushing the boundaries of creating more complex and versatile protein machines. Additionally, they found that augmenting training with AFDB clusters and applying conditional training benefits both motif scaffolding and unconditional structure generation, with benchmarks against strong SOTA models, including RFDiffusion, FrameFlow, and Chroma.

In summary, this work provides an interesting perspective on protein motif design and demonstrates the improvements of Genie 2 in protein structure generation.
weaknesses: **[Quality and Clarity]**

- The current version lacks in-depth studies on key tuning parameters (e.g., sampling temperature) and critical design factors (e.g., conditional training) (see Q1/Q3).
- Some important baselines are missing (see Q8/Q9).
- Some analyses and statements need further evaluation (see Q5/Q6/Q7).
- Certain statements are vague and would benefit from clarification (see Q2/Q12).

**[Significance and Originality]**

While the independent multi-motif design presents an interesting task, the current work faces limitations that affect its impact:

- The primary improvement of Genie 2 over current SOTA appears to be in diversity (including diverse solutions in motif scaffolding). However, it remains unclear whether this improvement is due to training with AFDB or low-temperature sampling (see Q3).
- Training on AFDB is a logical approach that has already been applied in other protein design models (e.g., Bilingual Language Model for Protein Sequence and Structure, Michael Heinzinger et al., bioRxiv, https://doi.org/10.1101/2023.07.23.550085).
- Encoding protein structure as SE3-invariant node and pair features is also an established approach (e.g., AlphaFold 2 encodes template information as node and pair). Although this work shows that SE3 invariance is effective for motifs encoding, it does not demonstrate broader impact on this design choice.

Meta-review: This paper introduces Genie 2, which extends the Genie's capabilities through architectural innovations and extensive data augmentation, enabling it to explore a broader and more diverse protein structure space. Genie 2 achieves state-of-the-art performance in both unconditional and conditional protein generation, surpassing existing methods on metrics like designability, diversity, and novelty. It also excels in solving motif scaffolding problems with more unique and varied solutions, setting a new benchmark for structure-based protein design. The reviewers expressed concerns regarding the novelty and significance of the proposed method. For example, the claimed architectural innovation and the data augmentation from AFDB have been widely employed in many other works, and the method underperforms compared to Proteus in designing long protein structures (>300 residues). Despite the authors' rebuttal and additional experiments, the paper did not gain sufficient support. Therefore, I recommend rejection.

Decision: Reject