GeomMotif: A Benchmark for Arbitrary Geometric Preservation in Protein Generation
Authors: Pavel Strashnov
Andrey Shevtsov
Viacheslav Meshchaninov
Olga Kardymon
Dmitry Vetrov
Venue: ICLR 2026 Poster (2026)
Keywords: protein design
generative models
motif scaffolding
geometric preservation
deep learning
benchmark
structural biology
Primary area: datasets and benchmarks
TL;DR: We introduce GeomMotif, a new benchmark that systematically evaluates how well protein generation models preserve the 3D geometry of arbitrary structural fragments, a capability largely untested by current benchmarks that focus on functional motifs.

Abstract: Motif scaffolding in protein design involves generating complete protein structures while preserving the 3D geometry of designated structural fragments, analogous to image outpainting in computer vision. Current benchmarks focus on functional motifs, leaving general geometric preservation capabilities largely untested. We introduce GeomMotif, a systematic benchmark that evaluates arbitrary structural fragment preservation without requiring functional specificity. We construct 57 benchmark tasks, each containing one or two motifs with up to 7 continuous fragments, by sampling from the Protein Data Bank (PDB) to ensure a ground-truth, solvable conformation for every problem. The tasks are characterized by comprehensive structural and physicochemical properties: size, geometric context, secondary structure, hydrophobicity, charge, and degree of burial. These features enable detailed performance analysis beyond simple success rates, revealing model-specific strengths and limitations. We evaluate models using scRMSD and pLDDT for geometric fidelity and clustering for structural diversity and novelty. Our results show that sequence-based and structure-based approaches find different tasks challenging, and that geometric preservation varies significantly with structural and physicochemical context. GeomMotif provides insights complementary to function-focused benchmarks and establishes a foundation for improving protein generative models.

Published as a conference paper at ICLR 2026
GEOMMOTIF: A BENCHMARK FOR ARBITRARY GEOMETRIC PRESERVATION IN PROTEIN GENERATION
Pavel Strashnoy 1,st
Andrey Shevtsov!
Viacheslav Meshchaninoy
Olga Kardymon'
Dmitry Vetrov?
AXXX,, Center for Al-driven Drug Design Construccor University 'HSE University
ABSTRACT
Motif scaffolding in protein design involves generating complete protein structures while preserving the 3D geometry of designated structural fragments, analogous to image outpainting computer vision. Current benchmarks focus on functional motifs, leaving general geometric preservation capabilities largely untested. We in- troduce GeomMotif, a systematic benchmark that evaluates arbitrary structural frag- ment preservation without requiring functional specificity. We construct 57 bench- mark tasks, each containing one or two motifs with up to continuous fragments, by sampling from the Protein Data Bank (PDB) to ensure ground-truth solvable formation for every problem. The tasks are characterized by comprehensive struc- tural and physicochemical properties: size, geometric context, secondary structure, hydrophobicity, charge, and degree of burial. These features enable detailed perfor- mance analysis beyond simple success rates, revealing model-specific strengths and limitations. We evaluate models using scRMSD and pLDDT for geometric fidelity and clustering for structural diversity and novelty. Our results show that sequence- based and structure-base approaches find different tasks challenging, and that geometric preservation varies significantly with structural and physicochemical con- text. GeomMotif provides insights complementary to function-focused benchmarks and establishes a foundation for improving protein generative models.
INTRODUCTION
Deep learning is revolutionizing protein design, enabling the creation of novel enzymes, vaccines, and protein- based therapeuties. A central task in this field is motif scaffolding, where new protein is generated around a specific functional fragment, preserving its precise 3D geometry (Wang et al., 2022). This process is analogous to "image outpainting" (Saharia et al., 2022) and is a critical step in engineering new functional proteins.
However, critical diagnostic blind spot exists in current evaluation methods. Benchmarks like MotifBench (Zheng et al., 2025) focus predominantly on known functional sites, such as enzyme ac- tive sites, which inherently biases evaluation toward specific structural and physicochemical contexts. More fundamentally, this approach conflates two distinct challenges: preserving 3D geometry and sat- isfying complex functional requirements. Beyond geometric scaffolding, functional success depends critically on residue identities, charge distributions, hydrophobic packing, side-chain conformations, etc. When model fails on such tasks, is impossible diagnose whether the cause was a funda- mental inability to preserve geometry or incompatibility with these physicochemical requirements.
This conflation creates a critical diagnostic problem because in protein engineering, geometric precision is a direct prerequisite for biological function. For computationally designed protein binders, for instance, experimental success rates can plummet from nearly 50% to zero as the backbone RMSD of the binding motif deviates from its target by just 1.0 Á (Cao et al., 2022). Consequently, geometric accuracy serves as the primary computational filter used to triage designs before costly wet-lab validation. Yet, current benchmarks entangle this foundational geometric challenge with complex functional requirements, making it impossible to distinguish between a model's failure stemming from a core inability to preserve structure or from more subtle functional inconpatibilities
"Core contributor, PProject lead

---

Published as a conference paper at ICLR 2026
Figure 1: Representative geometric preservation tasks from the GeomMotif benchmark. The goal is to generate scaffold to connect predefined motil's while preserving their 3D geometry.
To address this gap, introduce GeomMotif, a benchmark designed rigorously and exclusively evaluate the fundamental capability of arbitrary geometric preservation. By sampling structural fragments from across the Protein Data Bank (PDB) without a bias toward known functions, GeomMo- tif provides comprehensive test of model's ability to generalize across diverse structural contexts. This "protein outpainting" approach establishes foundational test of generative capabilities: can model maintain local and long-range geometric relationships between arbitrarily selected residues? Success on GeomMotif offers a direct measure of a model's core generative capacity, providing insights that are complementary to function-focused benchmarks. Critically, every task in GeomMotif is guaranteed to be solvable by design, is modality-agnostii to allow fair comparison between sequence-basee based and structure-bascr approaches, and is characterized by rich set of physicochemical properties to enable fine-grate mee-graieed performance analysis.
Our main contributions
We introduce GeomMotif, modallty-agnosste benchmark of 57 diverse protein scaffolding tasks that isolates geometric preservation from functional constraints.
We adapt established evaluation protocols and the SUN (Successful, Unique, Novel) to provide a single score reflecting geometric aceuracy, structural diversity, and novelty.
Our evaluation of seven models reveals a significant performance gap, with structure-based models
like Genie2 (39.4% SUN) and REdiffusion (37.8% far outperforming sequence-based models (best at 3.5%).
We uncover counterintuitive performance patterns, notably that ESM3's multimodal (sequence + structure) mode (1.4% SUN) consistently underperforms its sequence-one only counterpart (3.5%), suggesting structure conditioning can introduce conflicting signals.
Our analysis reveals clear architectural limitations, demonstrating that sequence-based models categorically fail on tasks with spatially separated (paired) motifs, while structure- models exhibit complex, non-monotonic performance fragment complexity increases.
We provide the benchmark data at HuggingFace and the complete task construction and evaluation code GitHub.
2 RELATED WORK
Computational protein design has evolved from physics-based -aase approaches to modern machine learning methods, with motif scaffolding emerging important benchmark for evaluating generative models.
Classical Approaches. Rosetta pioneered computational motif scaffolding through physics-based energy functions and extensive conformational sampling (Kuhlman et al., 2003). The enzyme design modules introduced by the Baker lab (Rothlisberger et al., 2008; Jiang et al., 2008) enabled systematic placement of catalytic residues within pre-existing scaffolds, establishing methodologies for rational enzyme design. Subsequent developments included RosettaDesign (Kuhlman et al., 2003), enzyme design modules (Richter et al., 201 1), and the coddfrro--oopss approach for epitope grafting (Correia et al., 2014). While successful for specific applications, these methods face computational limitations that scale exponentially with design complexity, motivating the transition to machine learning approaches.
Deep Learning Methods. The paradigm shift toward neural approaches began with Wang et al. (2022), who first formalized motif scaffolding as a machine learning task. RFdiffusion (Watson al., 2023) revolutionized this field by adapting diffusion models from computer vision protein
2

---

Published as a conference paper at ICLR 2026
Success check:
Filter by novelty and diversity Novelty refTM-score
Motif(s)
Generate 100 times
Successful
SUN
Novel
Unique
Unique clusters TM-score
Generation
ESMFold
KAL
ME
MKALSLMEV
PLDDT 20
ProteinMPNN
1:MKALVEMEA 2:MKALSLMEA
&:KKALALMEV
Sequence-based generation
M
Siructure-bard braed generation
motifRMSD
Figure 2: MotmMotif's modality-agnostic evaluation pipeline. Generated proteins are assessed for geometric fidelity (Success), structural diversity (Unique), and novelty, yielding a final SUN score.
structure generation, introducing benchmark 25 functional motif's focused on enzymatic active sites and binding interfaces. This established the standard evaluation protocol: generating backbone structures, designing sequences with ProteinMPNN (Dauparas al., 2022), and validating through structure prediction. Several models have adopted the REdiffusion benchmark paradigm, including EvoDiff (Alamdari et al., 2024), DPLM (Wang et al., 2024b), DPLM-2 (Wang et al., 2024c), DiMA (Meshchaninov et al., 2025), and Genie 2 (Lin et al., 2024). FrameFlow (Yim et al., 2024) introduced SE(3) flow matching for protein backbone generation with reported success on motif scaffolding tasks. Multi-motif formulations demonstrated by Lin et al. (2024) and Liu et al. (2024) address scenarios with independently floating functional regions.
MotifBench. MotifBench (Zheng et al., 2025) a standardized benchmark for functional motif scaffolding with 30 test cases, derived from the original RFdiffusion dataset. It focuses enzymatic active sites and binding interfaces, using a fixed evaluation pipeline based on ProteinMPNN and ESMFold. While important for specific applications, this focus functional sites makes it difficult to determine a model's failure stems from an inability to preserve geometry or from not meeting complex functional requirements. Additionally, some tasks involving complex functional sites may be inherently unsolvable by current methods. GeomMotif is designed to be complementary to these efforts. provides benchmark that isolates the challenge of general geometric preservation, allow- ing for direct assessment of the core structural capabilities required for successful protein design.
METHOD
We design GeomMotif to systematically evaluate the geometric preservation capabilities of protein generation models. Our approach addresses key limitations of existing benchmarks through principled task selection, comprehensive property characterization, and modality-agnostic evaluation.
3.1 DESIGN PRINCIPLES
GeomMotif is built on four core principles. First, we ensure systematic coverage protein structure space through uniform sampling rather than focusing functional regions. Second, we categorize tasks by complexity using both the number of motifs (1-2) and total fragments (1-7). Third, we characterize each task using eight structural and physicochemical properties enable detailed, property-driven analysis. Fourth, we guarantee task solvability by extracting fragments from natural proteins, ensuring least valid solution exists.

---

Published as a conference paper at ICLR 2026
Filter and similrily
Find residue environments
Filter loops 25%) and sparse matifs (s residtues)
Calculate properties and samipe by complexity
PDB
Single motifs (3772)
Paired motifs (5364)
GeomMotif
35 single motifs 22 paired mobifs 7 complexity levels
107 proteins
Figure 3: Pipeline for GeomMotif benchmark construction. Systematic workflow for constructing the bench- mark tasks. The process includes filtering high-quality monomeric structures, clustering non-redundancy. ensuring foldability, identifying residue neighborhoods, filtering motifs based on structural properties, and organizing tasks by complexity level. This construction procedure ensures comprehensive coverage of protein structural space and guarantees the task solvability.
3.2
TASK CONSTRUCTION
We construct benchmark tasks from monomeric protein structures to ensure structural stability and avoid conformational ambiguity. Proteins in complexes may adopt different structures in isolation, potentially leading to evaluation inconsistencies when comparing generated structures against ground truth conformations.
To assemble an initial high-quality dataset and ensure systematic coverage, we filtered the Protein Data Bank (PDB) according three key criteria. First, selected only structures determined by X-ray crystallography with resolution 2.5 A or better to ensure atomic precision. Second, we included only biological monomers to avoid the conformational ambiguities that can arise when proteins are part of larger complex. Finally, we imposed a length limit of 250 residues to prevent potential misfolding artifacts, substructures from larger proteins may require their full context for proper folding. This rigorous filtering process resulted in an initial dataset of 24,001 structures.
To eliminate redundancy while preserving structural diversity, we apply two-stage clustering protocol. First, we cluster sequences at 80% identity with 90% minimum coverage using MMseqs2 (Steinegger & Söding. 2017). Second, we perform structural clustering using complete linkage hierarchical clustering with TM-score threshold 0.5 and coverage 30%. The rationale behind the choice of particular threshold values is discussed in App.
To guarantee solvability, we generate ESMFold pre- dictions for each cluster representative and retain only structures where the predicted fold aligns with the ex- perimental structure at RMSD < A. This filtering ensures all benchmark tasks are inherently solvable by the evaluation pipeline, addressing critical limitation identified in previous benchmarks (Zheng et al., 2025). This procedure yields 107 unique protein structures from which construct motifs. First, we iterate through each residue of ev- ery protein, treating it as potential center. A motif is defined as the set of all residues whose Ca atoms are within 13Å radius this central residue's Ca atom. This step yields large pool of overlapping structural neighborhoods. We then apply a series of filters to re- fine this set: (1) we exclude small motifs containing fewer than 30 residues, (2) reduce local redundancy by removing neighborhoods which share more than 20% of their residues, and (3) filter out motifs with > % loop content as determined by DSSP sec- ondary structure assignment. This process produces 3,772 single-motif candidates. For maired-motif tasks, we identify motif pairs within the same structure separated by >30A between their centers, yielding 5,364 paired motifs.
Architacturea Topologies
Class: Architactures
a/B
Topologles
of candidate structural
Architectures 15 Topologins
Figure 4: GeomMotif diverse CATH structural folds. Representation of fundamen- tal protein architccture types across the bench- mark shows balanced coverage of major strue- tural classes: mainly-alpha (a), mainly-beta (3) alpha-beta (c/33). For detailed breakdown of the CATH architectures, see App.
4

---

Published as a conference paper at ICLR 2026
Although the residues forming motif spatially close, they are often non-contiguous in the amino acid sequence. The 'ragmment complexity" of a task is therefore defined by the number of separate, continuous sequence segments that constitute the motif. To ensure the benchmark assesses model performance across a spectrum of geometric challenges, we stratify the final selection by fragment complexity. We curate the benchmark to include equal number of tasks for each fragment count (1-7 for single motifs, 3-7 for paired), sampling up to five representatives for each category. This process yields our final benchmark of 57 tasks: 35 single-motif and 22 paired-motif problems. These tasks span diverse CATH fold classes, ensuring broad coverage of protein structural space (Fig. 4).
To minimize memorization and true generative capabilities, we allow biologically plausible length variations for variable regions while preserving fixed motif geometry. This forces models to generate valid structures across diverse length contexts rather than reproducing memorized patterns. Detailed formulation is provided in App. A.1. The full specification of all the tasks is available in Tab. 3.
To enable a fine-gre ane-ggained analysis of model performance, each constructed task further defined by comprehensive set of structural and physicochemical properties, as detailed in Sec. 3.3.
3.3 PROPERTY CHARACTERIZATION
Each task in GeomMotif is characterized by eight structural and physicochemical properties that enable fine-grained analysis of model performance:
Secondary structure composition. We characterize the motif's secondary structure using three distinct properties derived from DSSP assignments.
Helical Content. The proportion of residues in a-helices, 310-helices, r-helices.
Extended Content. The proportion of residues 3-strands and 3-bridges.
Loop Content. The proportion of residues in loop or bend regions.
Motif size. the number of residues in the motif, representing the extent of geometric constraints.
Mean hydrophobicity of motif residues using the Eisenberg scale (Eisenberg al., 1984), corre- lating with the tendency of residues to be buried versus exposed.
Burial ratio, fraction of motif residues with relative solvent accessibility (RSA) < 0.2, indicating positions with stringent packing constraints.
Absolute charge density calculated absolute charge per residue based on standard assignments
(Arg/Lys: +1, Asp/Glu: -1), affecting electrostatic compatibility.
Structural context we evaluate as a ratio internal contacts (between motif residues) to external
contacts (between motif and non-motif residues) using a 4.5A distance threshold.
These properties enable analysis beyond simple success rates, revealing which structural features contribute to task difficulty for different model architectures. The extended details on the calculation of the properties is provided in App. B. The detailed breakdown of all the properties for each task is provided in Tab. 2.
3.4 EVALUATION FRAMEWORK
Our evaluation framework extends established protocols from protein motif scaffolding benchmarks (Watson al., 2023; Alamdari al., 2024; Zheng al., 2025), adapting them assess general geometric preservation capabilities across both structure-based and sequence-based generative models. The framework captures three fundamental aspects of generative performance: geometric fidelity, structural diversity, and novelty relative to known proteins (Fig. 2).
The evaluation protocol differs between sequence and structure modalities to accommodate their distinct outputs. For structure-based models, we follow the established pipeline of generating backbone structures, designing compatible sequences using ProteinMPNN (Dauparas et al., 2022), and validating through structure prediction. Sequence-based models bypass the ProteinMPNN step, as they directly generate sequences that we fold using ESMFold (Lin et al., 2023). The rationale for these pipeline choices is detailed in the App. E. In both cases, assess geometric preservation by computing the backbone scRMSD between motif residues in the predicted structure and their corresponding positions in the ground truth.
A generated protein is considered successful when it meets two criteria: (1) the motif RMSD falls below 1. 1.0A, demonstrating faithful geometric preservation, and the predicted structure exhibits
5

---

Published as a conference paper at ICLR 2026
high confidence with average pLDDT 2 70, ensuring the overall fold is well-formed. To quantity diversity among successful designs, we cluster scaffolds using TM-score based hierarchical clustering with threshold 0.8. The number of resulting clusters reflects the breadth of structural solutions each model generates. Novelty assessment involves computing the TM-score between successful scaffolds and the ground truth structures. Scaffolds with TM-score < 0.8 to the reference structures represent genuinely novel designs.
To capture the competing objectives in protein design-accuracy, diversity, and novelty-we adopt the SUN (Successful, Unique, Novel) score (Sriram et al., 2024). The SUN score represents the proportion of generated samples that simultaneously achieve all three criteria. The Uniqueness and Novelty components are assessed within the set of successful designs, ensuring that we only measure the diversity and originality of viable structures.
SUN P(Successful Unique Novel)
This metric provides stringent assessment of model capabilities, rewarding only those designs that combine geometric accuracy with structural diversity and originality. By requiring simultaneous achievement of all criteria, the SUN score naturally balances the competing objectives in protein design. A detailed discussion of this metric's interpretation and utility is provided in App. C.
Evaluation proceeds in both fixed-length and variable-length settings to distinguish memorization from true generalization. The fixed-length evaluation constrains proteins to match ground truth lengths exactly, establishing a controlled baseline. The variable-length setting permits biologically reasonable length variations, testing models' capacity to generate valid structures without strict dimensional constraints. This dual evaluation reveals whether models genuinely understand protein geometry merely reconstruct memorized patterns.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We evaluate ten protein generation models spanning different architectural paradigms: RFdiffusion (Watson et al., 2023), FrameFlow (Yim et al., 2024), Genie2 (Lin et al., 2024), La-Proteina (Geffner al.. 2025), Protpardelle-Ic (Lu al., 2025), and REdiffusion2 (Ahern et al., 2025) (structure- based approaches), ESM3 1.4B (Hayes et al., 2025), DPLM-650M, DPLM-3B (Wang et al., 2024b) (sequence- based approaches), and a random baseline. For each of the 57 tasks, we generate 100 samples per model and evaluate them. To quantify the uncertainty in our SUN score measurements, we conduct bootstrap analysis: for each of the 57 tasks, we resample the 100 generated samples with replacement across 100 bootstrap iterations, calculate the SUN score for each bootstrap sample, and compute standard deviations across iterations.
To assess model capabilities beyond potential memorization of PDB structures, we conduct exper- iments in two settings. In the primary evaluation, task lengths vary from ground truth to test true generalization (Sec. 4.2). We also include fixed-length evaluation (App. H), where protein lengths match ground truth exactly, serving as a controlled baseline to understand model behavior when length variation is removed as factor.
4.2 OVERALL PERFORMANCE ACROSS TASK CATEGORIES
Figure 5 presents the performance of evaluated models on the GeomMotif benchmark using the SUN (Successful, Unique, Novel) metric. The SUN score represents the proportion of generated samples that simultaneously achieve geometric accuracy, structural diversity, and originality relative to known proteins.
Structure- cruse models demonstrate substantially higher SUN scores compared to sequence- based approaches. Genie2, La-Proteina, REdiffusion, and Protpardelle- Ic achieve comparable performance (39.4%. 38.8% 37.8%, and 33.8%, respectively), while FrameFlow and RFdiffusion2 achieve only 23.3% and 17.9%. Interestingly, RFdiffusion2 shows notably weaker performance than the original RFdiffusion. This likely stems from RFdiffusion2 being specifically developed and tuned for enzyme design starting from precise atomic motifs of catalytic active sites, while the original RFdiffusion was trained to be broad as possible We hypothesize that this narrow focus results in high scores on the

---

Published as a conference paper at ICLR 2026
80
Not Novel
Novel, Not unique
64.9%
Novel, Unique (SUN)
13.8%
54.4%
50.4%
46.5%
9.4%
Success (%)
3.15%
3.2.4%
4.0%
7.2%
40
6.9%
27.9%
19.2%
20
39.4%
38.8%
33.8%
11.2%
23.33%
9.9%
17.9%
6.2%
6.6%
1.4%
3.5%
Genie2
La-Proteina
RFalffusion
Protpardlle-1c
FrameFlon Flow
REdiffusion2
ESM3
DRLM:38
DPLM-650M
ESM3 (seq struct)
(seg)
Figure 5: Comparative performance of protein generation models the GeomMotif benchmark. Perfor- mance metries for 10 protein generation models measured by the SUN score (Success, Uniqueness, Novelty). percentage above each indicates overall success rale al preserving motif geometry. Each segmented to show: proteins that are both novel and unique (dark teal, representing the SUN score), proteins that are novel but not structurally diverse (medium teal), and proteins that preserve geometry but are not novel (light teal). Structure based models (left) substantially outperform ssquence-based approaches (right).
Atomic Motif Enzyme Benchmark Ahern et al. (2025) (41/41 solved cases) but poor performance on GeomMotif (Success rate 19.2%. SUN 17.9%). In contrast, the original RFdiffusion was trained for broad line of tasks and shows mediocre Atomic Motif Enzyme results (16/41 Ahern et al. (2025)) but much higher GeomMotif performance (Success rate 54.4%, SUN 37.8%). This pattern highlights the complementary nature of the Atomic Motif Enzyme Benchmark and GeomMotif in evaluating different aspects of scaffolding capability.
Among sequence-based models, ESM3 in sequence-only mode shows the highest performance at 3.5%. followed by DPLM-3B (2.7%) and DPLM-650M (2.1%). Notably, the larger DPLM-3B shows only marginal improvement over DPLM-650M suggesting that simple parameter scaling may not address the underlying challenges of sequence- motif scaffolding. Surprisingly, ESM3 variant using both sequence and structure modalities achieve significantly lower scores (1 1)) compared to
Table 1: Component analysis of the SUN metric across model types and task categories. Detailed breakdown of the SUN metric into its constituent components (Success, Novelty. Uniqueness) for single and paired motif tasks. Results demonstrate
while structure-basee based models maintain high performance across all metrics for single motifs, performance degrades significantly for paired motifs, with sequence- based models showing near-zero SUN scores on paired motifs. Bold values indicate best performance per metric category.
Model
Successful, Single
Novel, %6
Unique, % Paired
SUN Score +
Paired
Single
Paired
Single
Single
Paired
Genie2
60.1 1.0
32.9 404
26.6 405
$9,9 1.0
59.9 1.0
18.8 244
La-Proteina
207
62.7 403
67.1 206
35.2 409
61.3 206
404
61.3 206
16.2
REdiffusion
65.1
43.7
65.1
25.0
62.4
20.5
62.4
13.2
Protpardelle-IC
56.2 207
44.6
56.2 207
404
53.5
22.6
53.5 407
14,
FraweFlow
30.6
25.1 404
30.6
403
30.6
20.2 105
16.0
RFdiffusion2
=06
13.5
24,9
11.4
24.9
12.7
24.9
10.8
FSM3 (seq)
+03
6.5
11.3
0.1
10.1
0.1+
6.8 63
DPLM-3B
19.3
11,0
10.2
41.2
9.8
0.6
4.9
0.5
MPLM-650M
15.9
6.5
8.7
0..
8.1
0.3
4.0
0.2
ESM3 (seq & struct)
a.1
0.7 +.2
2.0+01
0.0
2.0
0.0
2.0
0.0
Random
0.0 on
0.0 0.0
0.0 +0
0.0 0.0
0.0 +000
0.0
0.0 +00
0.0 +00

---

Published as a conference paper at ICLR 2026
Single Motif Performance (Aggregated)
Paired Motif Performance (Aggregated)
Modd Structure based
93
Sequence
03
Naam Score
40-
Figure 6: Fragment complexity vs. model performance. Structure-based models (eyan) show monotonic degradation with increasing fragment count, while sequence-based models (dark cyan) categorically fail on paired motifs, Error bars show standard deviations.
the sequence-only model. This counterintuitive result aligns with performance patterns observed on the RFdiffusion motif scaffolding benchmark reported in Wang et al. (2024c), where ESM3 in co-generation mode solved fewer tasks with lower average success rate than sequence-only mode.
Table I decomposes the SUN metric into its constituent components and shows distinct performance patterns between single and paired motif tasks. For single motif tasks, RFdiffusion achieves the highest success rate (54.4%, next is Genie2 with 46.5%), but fall short against Genie2 by overall SUN score (37.8% vs 39.35%). The performance gap widens substantially for paired motif tasks, where Genie2 demonstrates the best performance (18.8% SUN), though all models show performance degradation compared to single motif scenarios.
The decomposition reveals that performance differences primarily stem from success rates rather than novelty or uniqueness metrics. For instance, RFdiffusion maintains comparable high novelty (45.05%) and uniqueness (41.4%) for successful single motif designs, indicating that when the model succeeds, generates diverse and original solutions. However, the dramatic drop success rates for paired motifs (43.7% for RFdiffusion) constrains the achievable SUN scores regardless of downstream novelty and diversity.
Sequence-based models demonstrate uniformly lower performance across all metrics, with paired motif tasks proving particularly challenging. ESM3 in sequence-only mode achieves the best sequence-basee performance for single motifs (6.8 % SUN) but drops to near-zero for paired motifs (0.1% SUN). This categorical failure on paired motifs reveals a fundamental limitation in sequence- based approaches for maintaining geometrie relationships between spatially separated regions.
These results show that current structure-based models possess significant advantages for geometric preservation tasks, and highlight the need for architectural innovations enable sequence-based mod- els to capture long-range spatial constraints. The performance patterns validate GeomMotif's design as discriminative benchmark that reveals model-specific capabilities and limitations. These overall results, however, raise the question: what structural factors drive these performance differences? We first examine how geometric complexity affects model capabilities.
4.3 IMPACT OF FRAGMENT COMPLEXITY
To systematically evaluate how geometric complexity affects model performance, we aggregated results across all tasks by fragment count for each model type (Fig. 6). This aggregation reveals clear architectural differences in how models handle increasing structural constraints.
Structure-based models demonstrate the expected monotonic relationship between fragment complexity and performance. Table and Figure 13 (Sec. K) provide the detailed breakdown of the performance of each model. For single motifs, mean SUN scores decline as fragment count increases. RFdiffusion achieves 91.9% on single fragments but drops to 38.7% at seven fragments, Genie2 shows similar trend from 76. 76.7% to 56.8%. This consistent degradation reflects the increasing difficulty of satisfying multiple geometric constraints simultancously. While individual tasks within each complexity bin show performance variations due their specific structural and
8

---

Published as a conference paper at ICLR 2026
Theclegy
Thpology
Length
Models
Gen
Protpandelle 1e HoriiFlow
High Malk
High Steet
Medium Loep
LOW LONG
High S..00
Expoded
Meduu Mools
Long Hotlls
Figure 7: Structure-basse model performance across property categories. Average SUN scores reveal shared sensitivities: helical over 8-sheet topology, low over medium loop content, exposed over buried residues, and medium over long motifs. Sequence-based models omitted due to low overall success rates.
physicochemical properties, the aggregate trend clearly demonstrates that geometric complexity systematically challenges structure-based models.
For paired motifs, structure-based models face substantially greater challenges due to the spatial separation between motif regions. Performance across all fragment counts remains markedly lower than single-motif tasks, with mean SUN scores rarely exceeding 40%.
Sequence-basel models fail categorically spatial separation. All sequence-based models achieve near-zero performance on paired motifs, regardless of fragment complexity. This exposes fundamental limitation: linear sequence models cannot encode spatial relationships between disconnected regions. For single motifs, these models show modest improvements at fragments, possibly because highly fragmented motifs resemble the masked sequence recovery tasks used in their pre-training.
The structure modality can impair rather than enhance performance. ESM3's multimodal variant consistently underperforms its sequence-only counterpart, with the performance gap widening as fragment complexity increases. This counterintuitive result suggests that structural conditioning may introduce conflicting optimization signals, particularly when geometric constraints become more complex.
These patterns demonstrate that protein scaffolding difficulty scales with geometric complexity in expected ways when examined at aggregate level. Task-specific variations exist due to differences in physicochemical properties, but the fundamental relationship is clear: dtructure-based models struggle progressively with increasing constraints, while sequence- based models fundamentally cannot handle spatial separation. Understanding these limitations is essential for developing next-generation protein design models.
4.4 PROPERTY- PROPERTY-BASED BASED ANALYSIS OF GEOMETRIC PRESERVATION
Beyond fragment complexity, we investigated how specific structural and physicochemical properties influence model performance using our task character- ization. The analysis shows that model success rates correlate with several key properties, revealing different performance patterns across various architectures (Figs. and 8).
Helix
GenieZ
Le Proteina
50
RFdiffusion
Protpardelle-1e
FrameFlow
High
High Sheet
Secondary Structure Composition Effects Sec- ondary structure composition emerges as a dominant factor affecting geometric preservation. For structure- based models, tasks with high a-helical content consis- tently yield higher success rates. The SUN scores for Genie2, RFdiffusion and Frame tasks with 70% helical content are 84.6%, 83.4% and 54.4%, respectively, compared to 42.9%, 45.2% and 14.0% for tasks with <30% helical content. This dispar- ity is particularly pronounced in small single- motif con- texts, where purely helical motifs (>90% helix) achieve the highest performance (mean SUN score 94.6% for
single-motif
Low Charge
High Charge
Figure 8: Model sensitivity to structural properties. Average SUN scores for structure- based models across five property categories show distinct sensitivity profiles.

---

Published as a conference paper at ICLR 2026
REdiffusion). Conversely, 3-sheet-rich motifs present significant challenges across all models. Since by design GeomMotif problems contain no more than 25% of loops, tasks with high sheet content match those with low helical content discussed above. This pattern aligns with the inherent complexity of f-sheets, which require precise long-range hydrogen bonding networks spanning residues distant in sequence.
3.0
The number of fragments and their spatial arrangement dramatically influence model performance. For paired motifs, structures with high helical content in both mo- tifs yield significantly higher success rates than mixed secondary structure arrangements. When both motifs are purely helical (e-g. tasks 5XJ7, 6KFQ), Genie2 vastly outperforms other methods and achieves mean SUN scores of 46.5% compared to 13.0% for paired motif's of mixed secondary structures.
Hayn Cornect
Internalto-Exeemms Contact Ratio
-80
-60
2.0
SUN
40
000
Burial and Contact Ratios The degree of residue burial and internal contact density provides further insight into model strengths and limitations (Fig. 9). Highly buried single motifss (>60% burial) with low internal-to- toxetermall contact ratios (<0.9) prove particu- larly challenging for all models. For Genie2 and RFdif- fusion, the mean SUN score drops below 30% for such motifs, compared 80.7% for exposed motifs with low burial (<50%) and higher contact ratios (> 1.5).
1.0
Low Content
30
40
50 60
70
80
Burial Ratio (%)
Figure 9: Task performance correlates with burial and structural context. GcomMotif tasks colored by average SUN score show that high burial with low internal contacts (lower right) yields harder scaffolding problems than low burial with high contacts (upper left).
Model-Specific Property Sensitivities Different model architectures exhibit distinct sensitivities to prop- erty variations. REdiffusion demonstrates superior per- formance on single-fragment tasks with high helical content (mean SUN score 92.0%) but struggles with multi-fragmeet B-sheet arrangements (mean SUN score 40.4% for 4+ fragments with >50% 8-sheet content). FrameFlow shows particular strength paired motifs with three fragments (average of 61. 1..%% SUN), suggesting optimization for distributed spatial constraints. Meanwhile, Genie2 exhibits more balanced performance across property dimensions but shows pronounced weakness on tasks combining high fragment counts with high 3-sheet content and low contact ratios.
These property-pefformance relationships provide detailed insights for benchmark design and model development. The consistent difficulty with 3-sheet-rich motifs and complex packing environments across diverse architectures points systemic. field-wide challenges. This approach, therefore, offers a diagnostic framework that complements existing benchmarks and can guide targeted improvements to future protein design models.
5 CONCLUSION
GeomMotif establishes the first systematic benchmark for geometric preservation in protein genera- tion. It isolates this fundamental capability from the confounding effects of functional constraints. Our evaluation reveals a clear architectural divide. Structure-bart based models like Genie2, La-Proteina and RFdiffusion achieve strong geometric fidelity, however, they struggle spatial complexity increases. Sequence-hasee approaches fail categorically on spatially separated motifs. This exposes their inability to encode long-range geometric relationships.
These findings provide actionable insights for model development. Helical motifs consistently outperform A-sheet arrangements. This pronounced sensitivity suggests specific training strategies could address these weaknesses. Our comprehensive property characterization reveals that burial patterns. fragment complexity, and structural context interact in non-obvious ways. These interactions offer concrete targets for architectural improvements.
GeomMotif demonstrates that geometric preservation varies systematically with structural features. This provides the diagnostic precision that functional benchmarks cannot. This complementary perspective is essential the field advances toward sophisticated protein design challenges. The benchmark establishes a foundation for understanding generative capabilities that underpin all successful protein engineering. This enables targeted improvements that will ultimately enhance both geometric accuracy and functional success.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We have thoroughly documented our methodology to ensure the reproducibility of the GcomMotif benchmark's 's construction, evaluation, and experimental results. All necessary details are provided in the main text and expanded upon in the Appendix.
The GeomMotif construction pipeline is detailed in Sec. 3.2.App. A.1 and illustrated in Fig. 3. We specify our criteria for data filtering, structural quality, redundancy removal, and guaranteed task solvability. The process uses publicly available tools like MMseqs2 (Steinegger & Söding, 2017), TM-Align (Zhang, 2005) and ESMFold (Lin et al., 2023). Methods for calculating the eight characterizing properties for each task also provided (Sec. 3.3 and App. B).
Our modality-agnostic evaluation framework (Sec. 3.4, Fig. 2) unambiguously defines our metrics for success (motifRMSD, pLDDT), diversity, and novelty. The adapted SUN score explained in App. C. The evaluation pipeline uses public tools, including ProteinMPNN (Dauparas et al., 2022).
The baseline comparisons in Sec. 4 are reproducible, using publicly available models. Per-task performance data for each model is available in Sec. L.
Benchmark data, task construction scripts, and evaluation code are available at our GitHub, and on HuggingFace.
ETHICS STATEMENT
This research was conducted in adherence with ethical scientific practices. The GeomMotif benchmark is derived entirely from the Protein Data Bank (PDB), public and anonymized resource, and our work did not involve human or animal subjects.
We recognize the potential for dual-use applications in protein design. However, our research is foundational and focuses on creating a benchmark for geometric preservation, not on engineering proteins with specific biological functions. The intended purpose of GeomMotif is to advance protein science for beneficial outcomes, such as developing novel therapeutics and biomaterials.
By making our benchmark, code, and evaluation tools publicly available, we aim to foster transparency and support the responsible and collaborative development of protein generation models.
ACKNOWLEDGEMENTS
This research was supported in part through computational resources of HPC facilities HSE University. The work was supported by the grant for research centers in the field of Al provided by the Ministry of Economic Development of the Russian Federation in accordance with the agreement 100000C313925P4455
and the agreement with HSE University 1313--15-2025-00
LLM STATEMENT
During the preparation of this manuscript, the authors utilized a Large Language Model (LLM) to assist with editing and refining the language in certain sections. All content was carefully reviewed, edited, and revised by the authors to ensure it accurately reflects our work and conclusions. The final responsibility for the content of this paper rests entirely with the authors.
REFERENCES
Woody Ahern, Jason Yim, Doug Tischer, Saman Salike, Seth M. Woodbury, Donghyo Kim, Indrek Kalvet, Yakov Kipnis, Brian Coventry, Han Raut Altae-Tras Magnus Bauer, Regina Barzilay, Tommi S. Jaakkola, Rohith Krishna, and David Baker. Atom level enzyme active site scaffolding using rfdiffusion2. bioRxiv, 2025. doi: 101/2225 04.....00......875.... URL https: www. biorxiv. wrg. http://ooneet/eeeeeee http//aarllll/aee 04/ / 2225.002.. 04. 09. 648075. 1, 6,7
Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Neil Tenenholtz, Robert Strome, Alan M. Moses, Alex X. Lu, Nicolò Fusi, Ava P. Amini, and Kevin K. Yang. Protein generation with
11

---

Published as a conference paper at ICLR 2026
evolutionary diffusion: sequence is all you need. bioRxi,, 2024. doi: 10.1101/2003.09..1.. 556673. URL https: www.biioriv..ommmmmmo http://onteetterr..///////////// 09..115566673 3,5
Longxing Cao, Brian Coventry, Inna Goreshnik, Buwei Huang, William Sheffler, Joon Sung Park, Kevin M. Jude, Iva Marković, Rameshwar U. Kadam, Koen H. G. Verschueren, Kenneth Ver- straete, Scott Thomas Russell Walsh, Nathaniel Bennett, Ashish Phal, Aerin Yang. Lisa Kozodoy, Michelle DeWitt, Lora Picton, Lauren Miller, Eva-Maria Strauch, Nicholas D. DeBouver, Allison Pires, Asim K. Bera, Samer Halabiya, Bradley Hammerson, Wei Yang. Steffen Bernard, Lance Stewart, Ian A. Wilson, Hannele Ruohola-Bake Joseph Schlessinger, Sangwon Lee, Savvas N. Savvides, K. Christopher Gareia, and David Baker. Design of protein-binding proteins from the target structure alone. Nature, (05(7910)5551-555 May 2022. ISSN 0028-0836, 1476- 4687. doi: 00.100///4186-0000 04654-9. URL https: /www nature com/ articles/ s s41586- 022 04 654-9. 9.
Bruno E. Correia, John T. Bates, Rebecca J. Loomis, Gretchen Baneyx, Chris Carrico, Joseph G. Jardine, Peter Rupert, Colin Correnti, Oleksandr Kalyuzhniy, Vinayak Vittal, Mary Connell, Eric Stevens, Alexandria Schrocter, Man Chen, Skye MacPherson, Andreia M. Serra, Yumiko Adachi, Margaret A. Holmes, Yuxing Li, Rachel E. Klevit, Barney S. Graham, Richard T. Wyatt, David Baker, Roland K. Strong, James E. Crowe, Philipp R. Johnson, and William R. Schief. Proof of principle for epitope-focused vaccine design. Nature, 507(7491):201-206 March 2014. ISSN 0028-0836, 1476-4687. doi: 10. 1..100///ature12900 12966. URL https: www. nature..e............ com/ articles http//auuurree.......
J. Dauparas, I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte, L. F. Milles, B. I. M. Wicky, A. Courbet, R. J. De Haas, N. Bethel, P. Y. Leung, T. F. Huddy, S. Pellock, D. Tischer, F. Chan, B. Koepnick, H. Nguyen, A. Kang, B. Sankaran, A. K. Bera, N. P. King, and D. Baker. Robust deep learning- based protein sequence design using proteinmpnn. Science, 378(6615):49555 56, 2022. ISSN 0036-8075, 1095-9203. doi: 10. (0.1126/sciecee.adde add2187. URL https: / /www. science. atg//oi/110. /26 science. add2181 5,
Robbert J. de Haas, Natalie Brunette, Alex Goodson, Justas Dauparas, Sue Y. Yi, Erin C. Yang, Quinton Dowling, Hannah Nguyen, Alex Kang, Asim K. Bera, Banumathi Sankaran, Renko de Vries, David Baker, and Neil P. King. Rapid and automated design of two-component protein nanomaterials using proteinmpnn. Proceedings of the National Academy of Sciences, 121(13): c2314646121, 2024. doi: 10.1073/pnas.23 23147//66 14646121. URL https: /ww. www..pnas.a. org/ / doi/ abs//0.1073/ ppass. 2314 2314646220 61211. 22
D. Eisenberg, E. Schwarz, M. Komaromy, and R. Wall. Analysis of membrane and surface pro- tein sequences with the hydrophobic moment plot. Journal of Molecular Biology, 179(1):125- 142, October 1984. ISSN 00222836. doi:
URL https: 5, 18
//linkinghub.ce weabbbeeseeiiie com/
Tomas Geffner, Kieran Didi, Zhonglin Cao, Danny Reidenbach, Zuobai Zhang, Christian Dallago, Emine Kucukbenli, Karsten Kreis, and Arash Vahdat. La-proteina: Atomistic protein generation via partially latent flow matching, 2025. URL /ttppi..//axxivoooo/oo org/ abs/2500 09466. 6
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J. Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q. Tran, Jonathan Deaton, Marius Wiggent, Rohil Badkundri, Irhum Shafkat, Jun Gong, Alexander Derry. Raul S. Molina, Neil Thomas, Yousuf A. Khan, Chetan Mishra, Carolyn Kim, Liam J. Bartie, Matthew Nemeth, Patrick Hsu, Tom Sercu, Salvatore Candido, and Alexander Rives. Simulating 500 million years of evolution with language model. Science, https//iccncc..001....... URL https:/ /wwww science.
38766736 2025. doi: 10.1 1etll///eee..n..omee org/doi/abs 10. sciincceeecee.... ads0018. 6
Guangxing He, Jun Liu, Dong Liu, and Guijun Zhang. GraphGPSM: global scoring model for protein structure using graph neural networks. Briefings in Bioinformatics, bbb4),b July 2023. ISSN 1467-5463, 1477-4054, doi: 10. 10.1093/bib/bbad2199 URL https: // /academic.. oup. com/ haam/bbbbaae bulbb/artica aataiceeeeaaaa ht///oo//10/// 11....1.00000 (t99//ib//bba2/// //799//119977 7197734. 23
Alvaro Martin Hermosilla, Carolin Berner, Sergey Ovchinnikov, and Anastassia A. Vorobieva.
Validation of de novo designed water-soluble and transmembrane 3-barrels by in silico folding and
12

---

Published as a conference paper at ICLR 2026
melting. Protein Science, 33(7))e5033, July 2024. ISSN 0961-8368, 1469-896X, doi: 10.1002/pro. 5033. URL
www.eewwwwwww............e....e. com/ /t///oo////// http///......//.......ooo 10w.2/002//..... 5033. 24
Anna Hýskova, Eva Maršálková, and Petr Šimeček. Balancing speed and precision in protein folding: A comparison of alphafold2, esmfold, and omegafold. bioRxiv, 2025. doi: 10.1101/ 26.666. URL https:/ http:/www.t....//ot www.bbiorxi..om org/ 21/2025. 06. 20. 660709. 23, 24
Lin Jiang. Eric A. Althoff, Fernando R. Clemente, Lindsey Doyle Daniela Rothlisberger, Alexandre Zanghellini, Jasmine L.. Gallaher, Jamie L.. Betker, Fujie Tanaka, Carlos Barbas, Donald Hilvert, Kendall N. Houk, Barry L. Stoddard, and David Baker. De Novo Computational Design of Retro- Aldol Enzymes. Science, 31965868);1387-1339 March 2008, ISSN 0036-8075, 1095-9203, doi: 10. 1 126/12ieeneeeeeeeee 1152992. URL https: http://wwwwww.wwe...o wwtp://.......cc...oooo//ooooooa 11tp:///1..1..//c/cccceee
1152692. 2
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera- Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis. Highly accurate protein structure pre- diction with AlphaFold. Nature, 596(7873):583-5888 August 2021. ISSN 0028-0836, 1476- 4687. doi: 10. 1415586-21--333
URL https: /www naturaauuee.eer cww.articals/////ccoocc.cmccccet
23
Irina Kufareva and Ruben Abagyan, Methods of Protein Structure Comparison, pp. 231-257. Humana Press, Totowa, NJ, 2012. ISBN 977-1-66177--8886 doi: 10. 10100997--1-6177--588--- URL
https://dol www.orggg/ooooo 1007 117788-----777---1
61776-177988 588-779-888888 6 6_100.1110000..0 23
Brian Kuhlman, Gautam Dantas, Gregory Ireton, Gabriele Varani, Barry Stoddard, and David Baker. Design of a Novel Globular Protein Fold with Atomic-Level Accuracy. Science, 302(5649); 1364-1368, November 2003. ISSN 0036-8075, 1095-9203. doi: 10. 10.1126/cceecee. 1089427. URL
https:/ hittps://wit http:://www..cceee. scwence.innc....ommm wwwoor//ooooo. doi 10. 2 science. 089447.2
Yeqing Lin, Minji Lee, Zhao Zhang, and Mohammed AlQuraishi. Out of many, one: Designing and scaffolding proteins at the scale of the structural universe with genie 2, 2024. URL https:
//arxiv. otg. abs 2405. 15489. 3, 6. 22, 28
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam ZaraadZar,ndi, Tom Sercu, Salvatore Candido, and Alexander Rives. Evolutionary-scaae prediction of atomic-level pro- tein structure with a language model. Science, 37966637; 2023. doi: 10.1126/ science.ade2574. URL https: http://wwwww/./ www.nceceecc...c. ade2574. 5, 11
Ke Liu, Weian Mao, Shuaike Shen, Xiaoran Jiao, Zheng Sun, Hao Chen, and Chunhua Shen. Floating anchor diffusion model for multi-motif scaffolding, 2024. URL https: http:///waxwv/////eaaee/ 2406.0 03141.
Tianyu Lu, Richard Shuai, Petr Kouba, Zhaoyang Li, Yilin Chen, Akio Shirali, Jinho Kim, and Po-Ssu Huang. Conditional protein structure generation with protpardelle-Ic bioRxiv, 2025. doi: 10. 10.1100/2025.00 0tt.//2255.8888.7055555 18.670959. URL https: http:///wwwwbbbww////////p// org/ content/earllyy 2025/08/ / 2025. 08 18. 670959.
Yutaka Maruyama, Ryo Igarashi, Yoshitaka Ushiku, and Ayori Mitsutake. Analysis of Protein Folding Simulation with Moving Root Mean Square Deviation. Journal of Chemical Information and Modeling, 6359):152911541 March 2023. ISSN 1549-9596, 1549-960X. doi: 10. 20.1021/acs.jeim. 2c01444. URL https: //wpbsub.././// acs. org/ doi/10. 1021/ acs jeim. 2c01444, 23
Viacheslav Meshchaninov, Pavel Strashnov, Andrey Shevtsov, Fedor Nikolaev, Nikita Ivanisenko, Olga Kardymon, and Dmitry Vetrov. Diffusion on language model encodings for protein sequence generation, 2025. URL https: http://wwwwwwww.aaaab/oe /23 0.7726.
/arxivi
13

---

Published as a conference paper at ICLR 2026
Florian Richter, Andrew Leaver-Fay, Sagar D. Khare, Sinisa Bjelic, and David Baker. De Novo Enzyme Design Using Rosetta3. PLoS ONE, 6(5):e19330, May 2011. ISSN 1932-6203. doi: 10. 137 L/jurmall www.ll...ooo........ URL https::////ccc /ttp///xxxoo/ plos. 11..111.1111111... htt.://///oo.r/aa.ppo///// 001 9230. 2
Daniela Rothlisberger, Olga Khersonsky, Andrew M. Wollacott, Lin Jiang, Jason DeChancie, Jamie Betker, Jasmine L. Gallaher, Eric A. Althoff, Alexandre Zanghellini, Orly Dym, Shira Albeck, Kendall N. Houk, Dan S. Tawfik, and David Baker. Kemp elimination catalysts by computational enzyme design. Nature, 453(7192)
May 2008 ISSN 0028-0836, 1476-4687. doi: 10. 10.1038/nature06879, URL https:ttpp..... : www. hae wwwmart.ii. ceesiccles.ee https//nauuueeee000
Chitwan Saharia, William Chan, Huiwen Chang, Chris A. Lee, Jonathan Ho. Tim Salimans, David J. Fleet, and Mohammad Norouzi. Palette: taigee diffusion models, 2022. URL https:
//arxivv...././///// org/ abs 22112.1111111 05826.
Anuroop Sriram, Benjamin Kurt Miller, Ricky T. Q. Chen, and Brandon M. Wood. Flowllm: Flow matching for material generation with large language models as base distributions, 2024. URL
https: //axivv org/ abs 2210 23405. 6, 21
Martin Steinegger and Johannes Söding. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature Biotechnology. 35(11))1022-1023 November 2017. ISSN 1087-0156, 1546-1696 doi: 10.1038/nbt.3988 URL https: http://wwwtttttt.t.oo nature.cuooooooom com/
articles nbtttes///bbt.e.. 3988. 4, 11
Kiera H. Sumida, Reyes Núñez-Franco Indrek Kalvet, Samuel J. Pellock, Basile I. M. Wicky, Lukas F. Milles, Justas Dauparas, Jue Wang, Yakov Kipnis, Noel Jameson, Alex Kang, Joshmyn De La Cruz, Banumathi Sankaran, Asim K. Bera, Gonzalo Jiménez-Osés, and David Baker. Improving protein expression, stability, and function with proteinmpnn. Journal of the American Chemical Society, 146(3):0054-2611, 2024. doi: 10. (021//jcs..c10000 3c1094.. URL https: / /ttp//oi..o//// org.10.11111100 1021/ jacs. 3c10941. 1. PMID: 38 39192293. 22
Jue Wang, Sidney Lisanza, David Juergens, Doug Tischer, Joseph L. Watson, Karla M. Castro, Robert Ragotte, Amijai Saragovi, Lukas Milles, Minkyung Baek, Ivan Anishchenko, Wei Yang, Derrick R. Hicks, Mare Expòsit, Thomas Schlichthaerle, Jung-Ho Chun, Justas Dauparas, Nathaniel Bennett, Basile I. M. Wicky, Andrew Muenks, Frank DiMaio, Bruno Correia, Sergey Ovchinnikov, and David Baker. Scaffolding protein functional sites using deep learning. Science, 377(6604):387-394 July 2022. ISSN 8075, 1095-9203. doi: 10. N0.1116//cienceeabn222 ab02 URL httph:tpp///omm www. wwwence.ecc..........o.......e.s org/ doi 11/0/110.00 11/1111.2111 11w./11666// wwt.///cienc.... abn2100. 1,2
Tianshu Wang. Xiaocheng Jin, Xiaoli Lu, Xiaoping Min, Shengxiang Ge, and Shaowei Li. Empirical validation of proteinmpnn's efficiency in enhancing protein fitness. Fron- tiers in Genetics, Volume 14 2023, 2024a. ISSN 1664-8021. doi: 10.3389/ 2ecn..2023.134766 URL https: /tt::///wwwww /www. frontiersin. htw.iioorr/oooraalsggeeettee.eee articles/10.1 3389 fgene. 2023. 1347667. 22
Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Diffusion language models are versatile protein learners, 2024b. URL https: http:///aavvv..o/.baa/ae 24C2.18567. 6,6
Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Dplm-2: A multimodal diffusion protein language model, 2024c. URL https: http///axvvvv//// abs/ 2410.13788 8
Joseph L. Watson, David Juergens, Nathaniel R. Bennett, Brian L. Trippe, Jason Yim, Helen E. Eise- nach, Woody Ahern, Andrew Borst, Robert J. Ragotte, Lukas Milles, Basile Wicky, Nikita Hanikel, Samuel J. Pellock, Alexis Courbet, William Sheffler, Jue Wang, Preetham Venkatesh, Isaac Sappington, Susana Vázquez Torres, Anna Lauko, Valentin De Bortoli, Emile Mathieu, Sergey Ovchinnikov, Regina Barzilay, Tommi S. Jaakkola, Frank DiMaio, Minkyung Back, and David Baker. De novo design protein structure and function with RFdiffusion. Nature, 620 (7976))1089-1100 August 2023. ISSN 0028-0836. 1476 doi: 10. 06135 8. 2, 5, 6, 22, 23,
URL https: / www. na ture. com/ art
28
14

---

Published as a conference paper at ICLR 2026
Jinrui Xu and Yang Zhang. How significant is protein structure similarity with TM-score
0.5? Bioinformatics, 26(7):889-89)) April 2010. ISSN 1367-4811, 1367-4803. doi: 10.1099hhoonfommatti//bbbooo. URL ptttpp://acadeic.... oup. www/bioonoooomtiicc.... att.iclee/26/7/889/11221 23
Jason Yim, Andrew Campbell, Emile Mathieu, Andrew Y. K. Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan Veeling, Frank Noé, Regina Barzilay, and Tommi S. Jaakkola. Improved scaffl-ccafoodiig with se(3) flow matching, 2024.
URL https: /arxiv org/ abs 22s//401. 04082. 3, 6, 21, 22,
Y. Zhang. TM-align: a protein structure alignment algorithm based on the TM-score. Nucleic Acids Research, (3(7):2302-230% April 2005, ISSN 1362-4962. doi: 10.1093/mar/gki524,, URL https: // /academicc oup. com/ naw wwwppp.oo..a.aaaaaaaeaaee
11,
23
Yichuan Zhang and Wen Zhang. Rprot- deep learning approach for fast protein structure similarity calculation. BMC Bioinformatics, 26(1):171, July 2025. ISSN 1471-2105, doi: URL http::/pppe/bmmcooooooooaoaoaaa
02118866122859-02--02222 com/articles/ 125511110002222222--------2--110
biomedeenttal.
23
Wei Zheng, Qiqige Wuyun, Yang Li, Chengxin Zhang, Lydia Freddolino, and Yang Zhang. Improving deep learning protein monomer and complex structure prediction using Deep.MSA2 with huge metagenomics data. Nature Methods, 21(2):279-25 February 2024. ISSN 1548-7091, 1548- 7105. doi: 10. 152.0388/411522022-0001 30-4. https:/ //tp:///wwwwww/ http://wwww......eeee/ea nature. com. articles/
23
Zhuoqi Zheng, Bo Zhang, Kieran Didi, Kevin K. Yang, Jason Yim, Joseph L. Watson, Hai-Feng Chen, and Brian L. Trippe. Motifbench: A standardized protein design benchmark for motif-scaffolding problems, 2025, URL https: // /arxivv. wtggabs///ee /0d///5522 12479. 1, 4, 22, 23, 28
15

---

Published as a conference paper at ICLR 2026
Appendix
A Benchmark Design
A.1
Length Range Determination
B Detailed Property Characterization
B.1 Property Calculation Methods
B.2 Structural Properties
17
B.3 Physicochemical Properties
18
B.4 Property Distribution Across the Benchmark
19
C
Interpretation and Utility of the SUN Score
21
D Detailed Structural Classification of Benchmark Tasks
22
E Model Choices for the Evaluation Pipeline
22
F Rationale for Threshold Selection
23
F.1
RMSD Threshold for Geometric Fidelity (1.0 Á)
23
F22
TM-score Thresholds for Structural Similarity (0.5 and 0.8)
23
F.3
Structure Prediction Confidence (pLDDT > 70)
24
F4
Task Construction Thresholds
24
G Complete specification of all benchmark tasks
25
H Fixed-Length Evaluation
26
Benchmarking time evaluation
27
J Fairness in Comparing Structure-Bard Baste and Sequence-Bass Based Models
28
K Detailed Fragment Complexity Performance
29
L Model-specific performance across all tasks
30
16

---

Published as a conference paper at ICLR 2026
A BENCHMARK DESIGN
A. LENGTH RANGE DETERMINATION
To establish biologically plausible length ranges for the variable regions GeomMotif benchmark, we developed principled approach based on known structural properties of proteins. For region of ground truth length L, we define the allowable length range as follows:
(L, I. max(1, mound (f
(Lmin Lmax) =
(2)
(max(L LL - (0.3L. 3), L round if 3
where round(-) denotes rounding to the nearest integer.
For the total protein length, given the sum of fixed motif lengths Lmotifs, we calculate:
L
min
Limotifs + Σ Linin
(3)
Total max
min(250, max L Lmotits + Σ (mnax))
(4)
where Liii and L are the minimum and maximum lengths for the i-th variable region, and L L is the ground truth total protein length.
GeomMotif benchmark employs biologically informed approach to determining length ranges for variable regions in protein motif scaffolding tasks. We define ranges that allow approximately +30% flexibility around the ground truth length, reflecting the natural structural variation observed in homologous proteins while maintaining consistent function. This range corresponds to the typical expansion and compression possible when a protein segment adopts different secondary structures (e.g., a-helices vs. 3-strands).
For very short regions (S residues), we preserve the ground truth length as the minimum to avoid creating structurally impossible constraints, these short connections often have limited conformational flexibility. For prefix and suffix regions, we apply the same principles to ensure biologically reasonable terminal segments.
The total protein length range is derived from the sum of fixed motif lengths plus the ranges of the variable regions, with an upper bound of 250 residues ensure compatibility with models of moderate context length. This approach ensures that our benchmark presents challenges that are both structurally reasonable and computationally tractable while maintaining sufficient flexibility to explore diverse design solutions.
B DETAILED PROPERTY CHARACTERIZATION
B. PROPERTY CALCULATION METHODS
The characterization of each motif in GeomMotif involves eight properties calculated using established biophysical metrics. Below we detail the precise calculation methods:
B.2
STRUCTURAL PROPERTIES
B.2.1 SECONDARY STRUCTURE COMPOSITION
We employ DSSP (Define Secondary Structure of Proteins) to assign secondary structure elements to each residue. The assignment uses the following classification:
Helical structures: a-helices (H), helices (G), and -heliiess (I)
Extended conformations: (-strands (E) and 3-bridges (B)
17

---

Published as a conference paper at ICLR 2026
Loop regions: turns (T), bends (S), and unstructured coil (C)
The proportion of each category calculated as:
Nss
PSS- Ntotal
(5)
where Nss is the number of residues with a particular secondary structure type and Nasta is the total number residues in the motif.
B.2.2
MOTIF SIZE
We record both the total residue count in each motif and the number of fragments. Fragment count ranges from (contiguous) (highly discontinuous) for single motifs, and 3-7 for paired motifs.
B.2.3
STRUCTURAL CONTEXT RATIO
For each motif, we calculate:
Ninternal
Roontert
Nesternal
where Ninternol is the number of contacts between residues within the motif and Nexternal is the number of contacts between motif residues and non-motif residues. A contact is defined when any heavy atoms from two residues are within 4.5A of each other. This ratio indicates how self-contained versus context-dependent motif is.
B.3 PHYSICOCCIEMICAL PROPERTIES
B.3.1 HYDROPHOBICITY PROFILE
We use the Eisenberg hydrophobicity scale (Eisenberg al., 1984) to calculate the mean hydropho- bicity:
N Σ
==
H
where H, is the hydrophobicity value of residue and N is the number of residues in the motif. The Eisenberg scale values range from -2.53 (most hydrophilic) to 1.38 (most hydrophobic).
B.3.2 BURIAL RATIO
We calculate relative solvent accessibility (RSA) using DSSP, which computes the accessible surface area normalized by the maximum possible exposure for each residue type. The burial ratio is:
NRSA<0.2
Reurial
(8)
Niotal
where NRSAÇO.2 is the number of residues with RSA 0.2, indicating buried positions.
B.3.3 HYDROPHOBIC CORE CONTENT
We define hydrophobic core residues as those that are both buried (RSA < 0.2) and hydrophobic (Eisenberg score > 0.5):
Roore Nouriedhhydroopohhi
(9)
Ntotal
B.3.4 CHARGE CHARACTERISTICS
We calculate absolute charge density as:
N
Pehorge
1qil
(10)
N
where qi is the charge of residue (Arg/Lys: +1, Asp/Glu: -1, others: 0).
18

---

Published as a conference paper at ICLR 2026
B.4 PROPERTY DISTRIBUTION ACROSS THE BENCHMARK
To ensure comprehensive coverage of protein structural space, we analyzed the distribution of properties across all benchmark tasks:
Secondary structure composition spans the full range natural proteins, with helical content ranging from 0% to 100%. 8-sheet content from 0% to 93.3%, and loop content from 0% to 23.3%.
Motif sizes range from 30 75 residues, with fragment counts from 7, creating spectrum of geometric constraint complexity.
Hydrophobicity values range from -0.57 to 0.68 on the Eisenberg scale, covering both highly
hydrophilic and hydrophobic motifs.
Burial ratios span from highly exposed (0.30) to deeply buried (0.82) motifs.
Charge densities range from 0.03 to 0.57, representing varying degrees of electrostatic complexity.
Structural context internal-to-external ratios vary from 0.68 (highly context-dependent) to 2.90 (highly self-contained).
This diversity enables detailed analysis of how these properties correlate with model performance, revealing specific strengths and weaknesses of different architectural approaches protein generation.
L0
Number of tasks
tosts
Number of teoks
0.0
0.2
6.4
066
0.6
1.0
0.0
0.2
0.4
0..6
0.8
1.0
0.0
0.2
0.1
0.5
0.5
1.0
40
50
60
Loop ranlo
2 helxx ratio
8-shoot ratlo
Mott length (residues)
H
Number of tasks
Number tasts
Number of taels
Number
hydrophobiciity
Absolute
Internal
retio
Figure 10: The distribution of structural and physicochemical propertics in GeomMotif tasks. Histograms show the frequency distribution of (A) loop ratio, (B) a-helix ratio, (C) 8-sheet ratio, (D) motif size as number of residues, (E) mean hydrophobicity, (F) burial ratio, (G) absolute charge density, and (H) structural context internal-to-exteni to-external contact ratio. The diverse distribution of properties ensures comprehensive coverage of protein structural space and presents varied geometric preservation challenges.
19

---

Published as a conference paper at ICLR 2026
Table 2: The properties of protein fragments across different GeomMotif tasks
Loop (8)
(%)
Sheet
Motif(s) Length
(%)
Charge Density
Internal
Fragments
Hydrophobicit
Contact ratio
single
1.5018
12.9
$7.1
0.0
0.10
48.4
0.23
1,36
single
25.3
30,0
0.27
46.7
0.20
1.25
single
3,5X37
59
94.1
0.0
34
0.57
44.1
0.06
2.90
single 4J6KRQ
0.16
single S.5URP
22.9
77.1
0.0
35
0.12
31.4
0.31
2.39
single
10.
1.98
single 7,5018
7,5
92.3
0.0
0,15
43.6
0.13
2.52
single
M.IM2G
12.5
18.8
0.11
0.31
single
9.5CWP
0.0
100.0
0.0
0.57
40,0
0.57
1.60
single
6FFV
16.7
$3.3
0.0
0.35
30.0
0,10
1.85
single
11_176N
1.64
single
12.3P2W
6.2
78.1
15.6
32
0.22
43.8
0.16
1.38
single
6KF)
8.1
1.47
single
14,48II
12.5
70.0
17.5
40
0,02
57.1
0.25
133
single
15_1A2I
5.7
94.3
0.0
0.19
51.4
0.23
paired
1.5CPP
6.1
0.500
36.1
0.52
2.06
paired
2,5CWN
9.5
90.5
0.0
0.30
44.4
0.51
2.00
3PR9
single
17,4VV
13.6
61.4
25.0
0.25
52.1
0.16
1.83
single
single
19 1M2G
15.6
24.4
50,0
32
0.06
65.6
0.22
0.95
single
31.6
19.5
00.00
53.7
piirol 3_4K46
19.7
$4.5
25.8
0.14
40.9
0.26
1.84
pairol
4_5XI7
4.2
95.8
0.0
0.60
43.7
0.06
2.20
5018
11.6
0.04
42.0
0.19
1,97
paired
22E5
8.3
86.7
5.0
60
0.06
48.3
0.23
1.46
single
2I_ITKY
9.7
29.0
61.3
31
0.03
61.3
0.26
0.68
single
single
23 1SGW
5.7
20.0
74.3
0.10
51.4
0.26
1.29
single
24,6
6.7
single
25_4F3H
5.6
38.9
55.6
36
0.10
66.7
0.19
0.95
paired
S.1ISI
10.0
71.7
18.3
0.08
35.0
0.30
1.19
4.2
0.68
0.03
1.33
paired
10,5UNN
16.7
47.0
36.4
66
0.06
53.0
0.26
1.35
pairod
paired
12_1
18.5
$1.5
0.0
65
0.04
33.8
0.28
1663
single
single
27 AL4M
4.3
$2.2
43.5
46
0,30
56.5
0.20
1.25
single
JTCS
9.7
single
29.2LAO
12.5
37,5
50,0
0,11
68.8
0.22
0.88
single
6TCS
11.4
38.6
0.03
54.3
0.14
0.80
paize ABJI
0.23
1.54
paired
14,5KZ1.
16.7
83.3
0.0
60
0.09
50.0
0.22
1.21
paized
1SGW
8.2
47.5
44.3
61
0.05
52.5
0.28
1.21
paired
8.3
1.31
single
31JGIU
13.2
15.8
71.1
0.44
0.08
0.78
single 32.IGBG
9.5
90.5
0.19
61.9
0.21
1.27
single 33.6TCS
8.3
91.7
0.18
61.1
0.11
0.7
single
6TCS
11.4
88.6
0.05
52.3
0.11
1.32
9.4
0:23
1.48
piimo
18,SW/RR
14.5
48
80.6
-0.11
50.0
0.34
0.66
paired
20 4GVW
9.2
55.4
35.4
65
0.21
43.7
0.22
1,42
pained
1.12
parrol
22_176N
12.7
70.4
169
0.01
46.5
0.27
1.53
20

---

Published as a conference paper at ICLR 2026
C INTERPRETATION AND UTILITY OF THE SUN SCORE
The SUN (Successful, Unique, Novel) score is holistic metric we adapt from generative modeling in materials science (Sriram et al., 2024). To calculate the score, we first identify all Successful designs- those that meet the primary geometric and structural constraints. Then, conditioned only on this successful set, we assess the Uniqueness (structural diversity) and Novelty of the generated solutions. This conditional approach ensures that diversity and novelty are measured only for viable designs, methodological practice consistent with established evaluation protocols protein generation (Yim et al., 2024).
Successful
Novel
SUN
Unique
Figure 11: Conceptual breakdown of the SUN score components. The SUN score quantifies the proportion of generated designs that simultaneously satisfy the criteria for being Successful (meeting geometric and structural constraints), Unique (structurally distinct from other generated designs), and Novel (structurally distinet from known natural proteins). The final SUN score is represented by the intersection of three sets.
The SUN score represents the aggregate success rate many generation attempts and is best interpreted as a continuous value, analogous to success rates in thhhthhruughput screening or yield in chemical synthesis. It quantifies the fraction of high-quality candidates researcher expect from a generative model.
Practically, the SUN provides upper bound the experimental success rate, which helps guide resource allocation in a protein design campaign. For example, a model with a 44% SUN score may yield one promising candidate for every two three designs generated, while 4% SUN score suggests a researcher would need to screen roughly 25 candidates to find a single one of comparable quality. This metric is an upper bound because it assesses geometric plausibility- necessary but insufficient condition for the ultimate functional success that must be confirmed by wet-lab validation.
The primary utility of the SUN score is providing single, comprehensive value for high-level model comparison. For a more granular diagnostic analysis, the metric can be decomposed. By examining the individual Success, Uniqueness, and Novelty rates, and analyzing how they vary with the physicochemical properties of the GeomMotif tasks, much richer picture of model behavior emerges. This detailed breakdown reveals specific architectural strengths and limitations, enabling informed model selection for specific design challenges and providing clear targets for future research.
21

---

Published as a conference paper at ICLR 2026
D DETAILED STRUCTURAL CLASSIFICATION OF BENCHMARK TASKS
To further specify the structural diversity the GeomMotif benchmark, this section provides more granular breakdown of the included protein architectures. Figure 12 visualizes the distribution of tasks across the CATH hierarchy, extending to the Architecture and Topology Superfamily levels (CATH levels 2 and 3). This level of detail clarifies the specific types of protein folds represented within each of the major structural classes (all-alpha, all-beta, and alpha-beta) that are summarized in Fig. 4.
Gamma- Saama-Crysiallin-iie
Immunoglobulin Olfactomedin-like
Rossman fold
Jelly Rolls
Integral membrane PisY
Topoisomerase
3-Layer(aba) Sandwich
Tetatricopeptide repeat
Glutaredoxin
Rhodopsin 7TM protein
α
Up-down Bundle
Ricin
a/B
Heme Oxygenase
Repetitive alpha hairpins
D-maltodextrin-Be BBinding Protein
ARM repeat
Carbamate kinase
Alpha-Beta Complex
Ankyrin repeat
Roll
2-Layer Sandwich
Leucine Aminopeptidase
Ferredoxin-like
Ribonuclease
GroEL
Toxin rop-ribesyltransfrrae
TIM Barrel
UB roll
RuBisCO
Chitinase
Arsylsulfatase
Figure 12: Detailed CATH Classification of GeomMotif Benchmark Tasks. The inner ring displays the thrce major CATH classes: all-alpha (o), beta (5), and alpha-beta (a/3). The outer rings detail the specific Architectures and Topologies within each class.
E MODEL CHOICES FOR THE EVALUATION PIPELINE
The construction of a robust and reproducible benchmark relies on a standardized evaluation pipeline. The choice of external software for inverse and forward folding is critical decision, as any specific tool inevitably introduces systemic biases that can affect performance assessment. Acknowledging deliberate decision prioritizing methodolog- ical consistency with community standards, the computational fcasibility of a large-scale benchmark, and reliance tools with proven practical utility.
this, selection of ProNeinMPNN and ESMFoold
For structure-bar based models that generate backbone, we use ProteinMPNN to design a corresponding amino acid sequence. This choice aligns our benchmark with established best practices in the field. ProteinMPNN is the canonical inverse folding tool used in the evaluation of leading models like RFdiffusion (Watson al., 2023), Genie2 (Lin al., 2024), and FrameFlow (Yim al., 2024), as well as in the MotifBench benchmark Zheng et al. (2025). Using the same tool is crucial for methodological consistency, as allows for fair comparisons and isolates the performance of the generative models themselves. Furthermore, the reliability of ProteinMPNN is not just computational its ability to design high-fidelity sequences has been confirmed through extensive wet-lab experiments Haas al. (2024); Sumida et al. (2024); Wang et al. (2024a). Finally, from practical standpoint, ProteinMPNN well-suited for our benchmark as can operate on the Ca-only backbone traces that some structure-based generators produce.
For the validation step, we predict the structure of designed sequences using ESMFold. This choice balances predictive accuracy with the significant computational requirements of large-scale
22

---

Published as a conference paper at ICLR 2026
benchmark. The GeomMotif evaluation involves folding tens of thousands of sequences, task for which the speed of ESMFold is essential. Using a more computationally intensive predictor, such AlphaFold2 (Jumper et al., 2021), would make the benchmark prohibitively expensive and inaccessible for many researchers, whereas ESMFold offers a balance of speed and precision (Hýskova al., 2025). Our choice is also consistent with established validation pipelines, e.g. MotifBench (Zheng et al., 2025) benchmark. While all predictors have inherent biases, and discrepancies can arise between methods, adhering a single, widely-adopted standard provides the most stable foundation for comparing models. Most importantly, a cornerstone of GeomMotif's s design is ensuring every task solvable by the evaluation pipeline itself. We guarantee this by pre-filterngg our source PDBs, keeping only structures that ESMFold can accurately fold (RMSD < 1.0 Á). This critical step ensures our benchmark rigorously tests the generative model's capabilities, not the limitations of the validation tool.
F RATIONALE FOR THRESHOLD SELECTION
The construction and evaluation pipelines of GeomMotif rely on several key numerical thresholds. These values are grounded in established community standards, biophysical principles, and internal validation to ensure our benchmark is robust and interpretable. This section provides a concise justification for these choices.
F.1 RMSD THRESHOLD FOR GEOMETRIC FIDELITY (1.0 A)
Root Mean Square Deviation (RMSD) threshold of 1.0 Å is used for two purposes: first, to select solvable tasks during benchmark construction, and second, to define success in geometrie preservation for generated designs. The choice of this value is supported by several lines of evidence:
Community Standard. RMSD threshold of 1.0 À aligns with the precision demanded by other benchmarks in protein design. For instance, MotifBench employs a 1.0 Á threshold to demand "atomic precision" (Zheng et al., 2025), and the original RFdiffusion benchmark uses identical criteria as "stringent filters indicative of experimental success" validated against experimental outcomes (Watson et al., 2023).
Biophysical Plausibility. This value reflects the upper limit of natural protein ficxibility. Analyses of identical protein structures from different PDB entries show inherent RMSD variations up to 1.2 X, while molecular dynamics simulations indicate that stable regions maintain RMSD of approximately 1.0 Á (Kufareva & Abagyan, 2012; Maruyama et al., 2023). Our threshold thus ensures that benchmark tasks represent geometrically achievable conformations.
Guaranteed Solvability. By pre-filtenn neeffiltring initial PDB set to include only structures that ESMFold predicts with an RMSD < 1.0 Á relative to the experimental ground truth, we guarantee that every task in GeomMotif has least known solution verifiable by our evaluation pipeline. This addresses key limitation in prior benchmarks where task solvability was uncertain.
F.2 TM-SCORF THRESHOLDS FOR STRUCTURAT, SIMILARITY (0.5 AND 0.8)
Two distinct TM-score thresholds employed to assess structural similarity, consistent with established conventions in structural biology.
TM-score > 0.5 (for Task Construction). During the construction of the GeomMotif dataset, TM-score of 0.5 was used to cluster proteins. This is widely accepted standard for determining if two proteins share the same fold, ensuring that our benchmark samples from structurally diverse set of protein architectures (Zhang, 2005; Xu & Zhang, 2010).
TM-score < 0.8 (for Novelty and Uniqueness), In our evaluation pipeline, more stringent threshold of 0.8 is used to define both novelty and uniqueness. A TM-score above 0.8 highly likely to belong to the same folding family (Zhang &
indicates that two proteins Zhang, 2025; Xu Zhang. 2010; Zheng et al., 2024; He et al., 2023). This cutoff allows to distinguish between genuinely new structural variations and minor deviations from
23

---

Published as a conference paper at ICLR 2026
known previously generated structures. The robustness of this choice confirmed through sensitivity analysis where varying the threshold between 0.7 and 0.9 did not alter the relative performance rankings of the evaluated models.
F.3 STRUCTURE PREDICTION CONFIDENCE (PLDDT 70)
For generated design to be deemed "'ucccesfull", it must not only preserve the motif geometry (RMSD< 1.0 A) but also form a -ttuustre scaffold, as indicated by high prediction confidence score (pI.DDT 2 70) from ESMFold. This dual criterion ensures that the preserved motif is embedded within physically plausible protein structure, filtering out designs that may be disordered or misfolded. The use of confidence scores as filter for designability is standard quality control step in the field that correlates with experimental success rates (Hermosilla et al., 2024; Hýskova et al., 2025).
F.4 TASK CONSTRUCTION THRESSOLDS
The thresholds used during the construction of GeomMotif tasks were chosen to balance several competing design goals: ensuring tasks are sufficiently complex be challenging, maintaining broad structural diversity, and keeping them computationally tractable within our framework.
Motif Definition (13 Á Radius). A 13 Á radius was selected to define the spatial extent of a motif. This value was determined empirically to provide an optimal trade-off. Smaller radii (e.g., 10 A) often captured insufficient structural context, resulting in motifs that were too simple. Conversely, larger radii tended to encompass a disproportionate fraction of the total protein (which is capped 250 residues), limiting the scope for de novo scaffold generation. The Á radius consistently produced motifs with meaningful tertiary structure while leaving a substantial portion of the protein be generated.
Motif Size Filter (>30 Residues). We excluded motifs containing fewer than 30 residues to establish a baseline of geometric complexity. Preliminary analysis showed that smaller motifs were frequently composed of simple surface features single contiguous fragments. Setting a minimum size of 30 residues ensures that every task presents a non-trivial challenge. often involving multiple sequence fragments that must be correctly oriented.
Paired Motif Separation (230 Å). For paired-motif tasks, minimum distance of 30 À between motif centers was required. This ensures that the two motifs are spatially distinct and do not directly interact. This separation forces generative models construct substantial and structurally coherent scaffold to bridge the two fixed regions, directly testing their ability to handle long-range geometric constraints. The value was chosen to safely greater than the of two motif radii (13 Á 13 A 26 Å), guaranteeing clear gap between them.
24

---

Published as a conference paper at ICLR 2026
G COMPLETE SPECIFICATION OF ALL BENCHMARK TASKS
Table 3: Comprehensive listing of GeomMotif benchmark tasks. Complete specification of all 57 bench- mark tasks, including interval notation for each task. The notation follows the REdiffusion contig format: y1, m, p2, m22, Ti6n; Pn++, where ps represents ranges for generated residues and my represents motif constraints. Total protein length must fall within the specified range. This rigorous task definition ensures reprodueibility and standardization across model evaluations.
Single Motifs
Entry
Contigs
I Length
1.5028
16:30 16-30.A24-44.000 140-222-44
156-250
2,1TKY
12/22,A1847,1244
166-250
3.5XC7
15-153.A119--12228-52
145-239
6XFQ
35:101.A79-109984-156
170-250
5,5URe
22-42.433-47.944177
151-250
8.5X27
[2:22.A18-26--16.A499999----
143-241
5028
160-250
8.1M2G
183-250
3.5CWP
L27-255AA4B2-1444--2222---7-
169-250
15:27.A22-1.21-333.46--111
145-243
11.126N
22-40,AA32-500 144-1-9900000-007777444466-6
127-205
12.3F2W
163-250
13 6XFO
173-250
14.465T
18-34,A27-52,6--00 00A.477752.00000000000044441111.
146-236
15.1820
142-234
16.3PR9
115-187
17_4GVW
14-26,A21 110.:1222....2211111144444622212
148-236
18.4104
41-75,A59-633 2.-.4488800011113499000066---.-.
158-250
19_1M2G
43-81,A63-74..22 11t.4444222..111122444.111111122
183-250
20_31.86
9-17,A14-29 3-7, 4TT.://.............00000.2.....
183-250
21.1TRY
167-250
22,6ICS
3-4,44- 11.4-111..9/192255888812
23_15GW
2-3,43- 3.24553...1222222111312101111112
152-250
24 6000
181-250
25,4F38
112244008444AAA44455550044411222 17,1551995
184-25
185-250
27 4124
163-250
28.6ICS
172-250
29.2LA0
176-250
30 6ICS
188.87-161
174-250
31,1GIU
185-250
32.1GDG
163-250
33 6rCs
174-250
34 6TCS
A85-91884.911 105-109.86-660
178-250
40.74.33--61337.46688881
148-231
Paired Motifs
Entry
Contigs
Length
1.50WP
180-250
2,5CW
164-250
3,4846
170-250
4.5xC7
J26-229
5,5028
6_29R5
177-250
7.1DEX
181-250
8,1151
12.23A192213322AA0051111...22
148 222
183-250
10 5DM1
5-9.A8 155.422.2225.4407777077222222222
189-250
11.6KFQ
1-2.A2-
182-250
12.11103
158-234
13 4B21
17-31.A25-39,11 111.25589.11111--55----2------11
153-229
14.5K21
9000
162-250
15.1L04
165-250
158-243
17.1801
175-250
140-203
19.120S
189-250
20_4GVW
14277 4
A1A.844....09998119955.117777 1- 1-8.1178-
155-232
21.308X
152-228
22.126N
138-195
25

---

Published as a conference paper at ICLR 2026
H FIXED-LENGTH EVALUATION
To distinguish between genuine geometric understanding and potential memorization artifacts, we conduct our evaluation two complementary settings. The primary variable-length evaluation presented in the main text Sec. 4 allows biologically reasonable length variations for scaffold regions while preserving fixed motif geometry. This reflects realistic protein design scenarios where the total protein length is not predetermined. However, this flexibility introduces a potential confound: models might fail not due to geometric constraints but due to difficulties determining appropriate scaffold lengths.
Our secondary fixed-length evaluation addresses this concern by constraining generated proteins to match the ground truth length exactly. This controlled setting serves two critical purposes. First, it provides 100% guaranteed solvability since the target length is known to produce valid fold. Second, it isolates geometric preservation capabilities from length prediction, revealing whether models can solve the scaffolding problem when this additional complexity is removed. Comparing performance across these two settings helps identify whether model limitations stem from fundamental geometric reasoning deficits or from the additional challenge of length optimization. Models that perform well in fixed-length but poorly in variable-length scenarios may benefit from improved length prediction mechanisms, while consistent poor performance across both settings indicates deeper architectural limitations in geometric understanding.
Table 4: Fixed-length evaluation. Detailed breakdown of the SUN metric into its constituent components (Success, Novelty, Uniqueness) for single and paired motif tasks.
Model
Successful, % Paired
Novel, % f
Unique, % 1
SUN Score 1 Paired
Single
Single
Paired
Single
Paired
Single
Genie2
61.6
36,0
61.6 20.5
27.8
61.1
18.9
61.1
17.2 203
RFdiffusion
76.4
54.0
76.4 1n6
27.2 s05
70.9
11.7 202
70.9
7.1 2.1
FrameFlow
33.0
34.3
33.0 au7
22.4
33.0
27.1 206
33.0
17.8 =07
DPLM-3B
46.3 1.6
18.9 40.9
0.0
9.2
0.0
6,8 10.1
0.0
DPLM-650M
40.2
36.7
15.3 20.
0.6 201
10.2
0.5
3.7
0.1 0.0000
ESM3 (seq)
42.2 0.3
36.5 +0.4
16.5 +04
0.9 a02
21.9 x.2
0.6
8.2
0.5 +01
ESM3 (seq & struct) 70.2
60.4
27.4 0.4
2.7 a.2
42.7 a.1
17.1
1.4 1.4
In the fixed-length experiment, providing the model with the exact ground-truth protein length simplifies the generative task. This controlled setting reveals distinct behaviors between model architectures.
The performance of structure-based models shows a commensurate increase in both raw Success rates and final SUN scores. This parallel improvement indicates that the performance gain is not simply an artifact of memorization but rather result of isolating the geometric scaffolding challenge from the separate complexity of length prediction.
In contrast, sequence-based models exhibit a large increase in their Success rates but only modest improvement in their SUN scores. This pattern suggests that while these models are proficient at completing sequence within a known length constraint, they struggle to generate novel and diverse structures, highlighting a limitation in their generalization capabilities.
This setting also reveals the potential of multimodal models. For instance, the ESM3 (seq & struct) model, which performed poorly in the variable-length experiment, shows substantial improvement in its SUN score under fixed-length conditions, increasing from 1.4% to 9.25%. This suggests that providing accurate structural constraints can significantly enhance the performance of models that leverage both sequence and structure information.
26

---

Published as a conference paper at ICLR 2026
I BENCHMARKING TIME EVALUATION
To evaluate the computational requirements of running GeomMotif depending on model type, we run the evaluation pipeline 5 times calculate standard deviations. The results of the evaluation using NVIDIA A100 GPUs are reported in Tab. 5.
Table 5:
Computational requirements by model type.
Model Type
Total Time (hours)
ProteinMPNN
scRMSD cale.
pLDDT cale. Parsing
Sequence-basee
0.77 0.01
0.64
0.12 + 0001
Structure-based
6.03
0.64
4.93 1001
0.44
Structure- model evaluation requires approximately 8x longer than sequence- based models due the additional ProteinMPNN sequence design step, which necessitates sequences per generated structure followed by ESMFold prediction for each. Novelty and diversity calculations are identical for both model types and require negligible additional time (under 10 minutes total). We will include comprehensive computational requirements and runtime analysis in the revised appendix to facilitate benchmark reproduction.
27

---

Published as a conference paper at ICLR 2026
J FAIRNESS IN COMPARING SRUUCTURE-BASER AND -OOSED MODELS
potential concern in evaluation is that muucture-based models utilize 8 ProteinMPNN sequences per generated backbone, while aequence-based models generate only one sequence per sample. This difference in the number of ESMFold predictions could be perceived as providing an advantage to structure-base based approaches. To address this concern and ensure a fair comparison, we conducted additional experiments examining performance under two alternative settings: (1) structure-basse models using only ProteinMPNN sequence per backbone, and (2) sequence 8x the number sequences.
sequence-based based models generating
Table 6: Impact of sampling budget on model performance. Comparison of SUN scores when varying the number of samples: structure-based models with vs. ProtcinMPNN sequences, and sequence-based models with Ix 8x sampling. Additional compute improves all models but does not change the fundamental performance hierarchy.
Model
Successful, %
Novel, f
Unique,
SUN Score t Paired
Single
Puired
Single
Paired
Single
Paired
Single
Overall
DPLM-650M
6.5
8.7
a.1
8.1
0.3 s01
4.0
0.2
2.1
DPLM-650M, x8
19.2 a7
8.1 202
11.9 203
16
10.9.02
4.2
7.2 a03
1.4001
4.2
ESM3 (seq) xI
6.5 40.
11.3 x05
0.1 =no
10.1 =02
0.1 +00
6.8 203
0.1
x.2
ESM3 (seq) x8
24.7 ..3
9.7 40.
18.7 .04
+0
17.0-a3
as
13.5 604
2.8 .04
8.2
REdiffussin xl
a07
26.5 s:0
40.5 2077
20.
39.2 206
13.1
39.2
7.4
23.3 204
REdiffusion
204
43.7 =10
65.1 20.4
25.
62.4
20.5
62.4
13.2
37.8 204
Genie2
36.2
20.3
36,2
15.9
36.2
14.2
36.2
11.1
23.7
Genie2 x8
60.1 1.
32.9
60.1 1.0
26.6
59.9
22.5
=9.9
18.8
39.4
Table 6 presents the results of this analysis. When structure-based models are restricted to single ProteinMPNN sequence (x1), their SUN scores decrease Saatuntiill--RFFifffui drops from 37.8% to 23.3%, and Genie2 from 39.4% to 23.7%. However, under this constraint, structure- based models still dramatically outperform sequence-based models. Conversely, when sequence- based models provided with 8x computational budget (x8), their performance improves-ESM3 (seq) increases from 3.5% to 8.2%, and DPLM-650M from 2.1% to 4.3% but the fundamental performance gap remains substantial. These results demonstrate that additional sampling capacity benefits all model types, but does not eliminate the architectural advantages of structure- based approaches for geometric preservation tasks. Our primary evaluation using ProteinMPNN sequences follows the established protocol in motif scaffolding benchmarks (Watson et al., 2023; Yim et al., 2024; Lin et al., 2024; Zheng et al., 2025), which reflects standard practice the field and enables direct comparison with published results.
28

---

Published as a conference paper at ICLR 2026
K DETAILED FRAGMENT COMPLEXITY PERFORMANCE
This section provides the complete performance breakdown underlying the aggregate analysis in Sec. 4.3. Figure 13 shows individual model trajectories across fragment complexity, and Table provides the detailed numerical results for all models and tasks.
Single Motif Performance
Paired Motif Performance
Node Tpp
Gente2
La Proteina
#Foiffusion
Proppeceel--CC
RrameFlow
Rrciflusion
SUN Score
ESMS tunq and struct)
Number of Tragments
Number Fragments
Figure 13: Individual model performance by fragment complexity. SUN for each model single-motif (left) paired motif (right) tasks stratified by number of continuous fragments. Each bin contains 5 tasks.
Table 7: SUN score performance stratified by the number of continuous fragments in motifs (1-7 fragments) reveals non-monotonic relationships between fragment complexity and model performance. Structure- models exhibit different performance patterns compared sequence-brag basedd models, with the latter failing categorically on paired motifs. Bold values indicate best performance per fragment count category.
Model
Number fragments siugle motif
Number of fraguments paired motifs
Genic2
=6.7=
+1.4
18.7
33.
49.9 x23
56.8 =02
39.2..12
15.2.0
7.0
-1
La: Proteina
a2.0
a..0
45.6 223
60.5 211
11.
47.1
24.5 110
30.7
2.
21.1 211
2.1
REdiffusion
20:
22.8
83.6
$7.9 :22
42.0
53.8
38.7
14.8 216
22.8
5.9
148 202
8,0
Protpakfle-IC
21.3
s7.
69.
43.4
40.1
414
42.8
31.4
23.7
S44
12.5
8.0
FrameFlom
43.8
49.9
19.
9.8
15.
61.6
17.1
11.1
914
9,4
Ridiffusion2
314
18.0
ESMB (sog)
11
20
10
3.3 suy
4.2
8.
12.2
0.2 30z
0.0
0.0
0.0
DPLM-3B
4.6
23
2.4
4.3 a06
3.4
7.5
10.4 204
0.0 100
0.0
0.0.e
2.2 404
0.0
DPLM-650M
2.7
2.0
09
1.3 204
1.5183
7.7 212
7.0
0.0 200
0.0
0.0
1.0
0.0
ESM3 (seppstnuet)
29205
3.0
2.0
0.0000
1.1 197
288
0.0
0.0
0.0
0.0
0,0
These detailed results complement the aggregated analysis presented Figure 6, where averag- ing across models by type reveals the systematic relationship between fragment complexity and performance.
29

---

Published as a conference paper at ICLR 2026
L MODEL-SPECIHIC PERFORMANCE ACROSS ALL TASKS
Table 8: -pdel-speciffic performance on single-motif tasks. Detailed SUN scores for each model across all benchmark tasks, organized by fragment complexity (1-7 fragments). Values represent the percentage of successful, unique, and novel protein structures generated per 100 attempts. This granular performance breakdown reveals sask-specific strengths and weaknesses of different architcctural approaches.
Eatre
Genic2
La-Prstcins
REdiffiiin
Protparicle-IC
FransFlow
REdiftoion2
ESMB (oo))
DPLMEB
DPLM650
ESMB (req struct)
1,535
1000 co
96.1 60
56.0
52.55 15
$1.4
764 14
[8.1 1:92
11.0 aM
11.0 60
39.2 2
2.1TKY
1000 0.
64.6 2:17
S1.6
47.8
198
2.0 1:
24.1 1:38
4.3
11.7 1:11
2.4 1:
1,907
91.0
99.0
99.0
35.0
58
3.0
53 5
93.
5.5URP
950
63%
1:44
1:90
00:2:00
6 5300
85.2 1.
K5.1 2:4
500 44
$5.0
16. 56
160 2:24
322 1:15
0.4 93
B.D 60
322 2
7,5055
98.4 1:03
72.5
91.4
76.7
91.6 1:
69.1
4.3 1:
5.
6.9
010
S.1M2
560
44A
4600
11.6
100
3.9
0.0
0.0
0.0
2.0
10.6FFY
458 1.43
17.0 13
368 13
26.0
10.4 23
14
1:M
0.0
11,1288
98: 1:
71. 15
592 3:30
73.9 1.
18.2 4
15.61141
DO 1:M
0.0 AM
U. 80
TLD 3
17
736 1:14
NNA
M33 219
50.7 70
50.
254 1:
0.0 9:
0.0
UD t0
010
850
156
R84
41.0
18.
130
0.0
0.0
0.0
14.48J1
5966
42
15JAN
412
1:17
956
63.8
8:44
5:00
U1
18,
186 2.74
7011 2111
BAA 11
196
15
1
NK 1:
17,4GVW
400 9:33
47.4
45.1 14
2044 10
76
56 24
04 1:
0.0 9z
0.0 m0
0.0
18.4L04
$3.6
59.7
79L1
57.5
35.6
262
5.4 AM
3.5
22
0.0
2.0
0.0
21.TKKY
0.1 2:04
51A 211
10
7944 MM:1:
DO 1:0
0.0 2:
SUITES
43.0
90J1 1:
408 14
47.4 57
15.6 15
136 7.
13.3 93
7.5 14
B10
23.W
23.4 1:38
75.1
$3.6 117
62.4 1.56
82
13.6 1:34
0.0 1:M
0.0 98
0.D R0
0.D
24,0LU
48
U.D
35,F3
14.6
6:74
241
15,630
41002:41 2:
KEA 2711
16. 3:
DD 9:
0.0 2:91
B1 m0
27,4L04
846 1:33
78.6 17
72.0 3T
70.2 10
46.4 1
13.0 14
0.0 AM
0.0 92
0.0 m0
1.4
6TCS
602
67.0
514 4
36.8
K6
13.4 A:
5.7 1:33
2.2 18
1.2 12
0.0
29,2AA0
28.0
1.5
0.D
31,GUU
230 2:44
66H 2.
100 T
00.2 2:
0.022
BM 2:0
0.0 2
32 LGBG
63.2 15
844 113
31.2 27
57.9 1.
20.4 23
20:2 1:31
5.1 1:28
17.3 4
044 1:
0.D 1:
33,6TCS
694 1:18
(63 21
53. 25
45.0
20.0 16
32.6 1:53
[7.] 1:33
54
7.2
1.0 1:13
3,ATUS
182
2.
Table 9: Model-specific performance on paired-motif tasks. Detailed SUN scores for each model across all benchmark tasks, organized by fragment complexity (3-7 fragments). Values represent the percentage of successful, unique, and novel protein structures generated per 100 attempts.
AFraps
Eatry
Genic2
La-Pontii
dilisiii
Protpanidle-IC
FrameFlow
RFdifiasinn?
ESMB (os))
DPLMB
DPLMIGIM
ESMB (voq struct)
LSCWP
42
16.3
15.6 12
25.6
50, 10
19.6 7
0. 5:00
010 mn
0.0 2:
0.0 :
2,5CWN
70.7 2:4
31.6
16.4
33,5 1.
70.7
(200 1:43
10
0.0 t
0.0 1:00
0.0 1.
1.4K46
56.1
37.0
0.0
40U7
1:00
25.0 L1
-12
15. 135
1:08
00.1
0, 0:00
5,5088
0.0 1:03
0.0 1.30
55 14
58 1.36
2633 11
148 1:3
00 1:00
0.0 M0
0,0 0:00
0.0 1:0
4,7215
01/ 2:38
25.1 2:24
31.4 36
12.3 1:43
14. 244
210 2:33
00 1:00
BD tn
00 0:00
0.0
7 IDEX
34.1 4434
44.1 34
18.5 3
86 1P
46
1.2 1.0
00
0.0 s
0,0 0
0.0
9.6KTQ
04
122 13
21.0 34
0.0
10 SDNI
17.3 11
7.2 3.
182 I.
7.8 LA
19.6 12
13. 2.
0. AM
0.0 m0
0,0 0
0,0 A.
11.8KQ
25.4 1.28
00
3.1 1.
55 1T
11.4 21
10,0 1.34
00 1:08
0.0 1:00
0.0 1:
H2,1HU3
2.03
2.03
14,577
1.
11.3 8:
18.2 TM
17.5 3:
17
0.0 mm
0.0 =m
15. 15,4L04
0.0 0.
16.4 24
8.9 23
KR 24
LA 13
0.0 az
0,0 98
0.0 mm
0.0 a.
0.0
16 19OW
3.6 1.15
31.5
29.5
24.0
148 -1
11. 4:32
00 oz
0.0
0,0 a
0.0 AM
17.IBOL
30,2
58
38,0W5B
15
11.3
2:03
19,1088
11
9.5 1.
INT 7.76
K.0 8.1.1.1 11.
7.3-211
17
00 sn
0.0 2:
0.0 a:
ML4GVW
21.3 74
14.8 14
3.6 1
17.2 13
13
0.0 9.
0.0 mm
0.0 =m
0.0 9:
21,305X
23
4.4
28 1.
0.0 0.
00
0.0
0.0
0.0
22.126N
00
48
0,0
30

---

Published as conference paper at ICLR 2026
Table 10: Task-level metries for Protpardelle-Ic across experiments.
#Frags
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN Score
1 1_SCPP
paired
122
88,2 1.9
27.6
25.8
2,5CWN
paired
85.4 #411
=42
51.5
35.7 #25
4K46
paired
52.8
52.8
37.4
37.4
5XJ7
paired
70.8 +2.
70.8 2.9
44.2
44,2 1.8
5,50J8
paired
11.6 221
4.7
11.6
4.0
6.2ZE5
paired
24.2 4.2
24.2 2:2.
19.5
19.5 434
7_IDEX
paired
12.8 449
4.9
=38
9.8 #38
10 10.5DN1
paired
89.4
20.2
40.2
9.1 11.5
11 LKFQ
paired
83.2 3.6
7.
71.3 3.1
6.5 2.2
12_1HU3
paired
0.0 200
0.0
0.0 200
0,0
8 1IS1
paired
47.0 23.
0.0
0.0
0.0 2.00
9_6KFQ
paired
71.2 1.9
21.2 +53
+.11
12.5
13.4BJI
paired
3.6 2.6
3.6
3.6 =2.6
3.6
14.5KZL
paired
81.8 1.9
43.6 255
35.1 1
18.7 2.44
15_4LQ4
paired
5.8 #26
2.
=2.6
5.8 #.6
16 ISGW
paired
36.6 23.
36,2
30.1
29.8 229
17_IBOI.
paired
5.0 + 1.9
5.0 1.9
5.0 +19
5.0 1.9
18 6W5B
paired
53.2
41.6
15.6
12.2 +0.7
19.1Q0S
paired
14,0 2.7
14,0 2.7
8.6
8.6
20.4GVW
paired
58.8 #7.1
2.0 #59
22.5 2.7
18.1 +.2
21_3OSX
paired
11.6 =3,0
11.6
2.0 1.8
22_176N
paired
66.8 3.9
0.0
0.0 8.0
0.0 e 0.0
5018
single
96.6
96.6
82.7
82.7
2.ITKY
single
47.0 238
47,0 238
47.0
47.0
3.5XJ7
single
99.0 +0.
99.0 o.9
99.0 n.
99.0 +.9
4_6KFQ
single
95.2 117
95.2 1.7
93.2 1.7
93.2 1.7
5.5URP
single
84.8 2.8
84,8 3.8
84.8 1.8
84.8 3.8
6FF6FFV
single
25.4 4.3
25.4 4.3
25.4
25.4 4:4.3
6_5XI7
single
86.0 230
86,0 230
$5.0
85.0
SOJ8
single
83.6 4.11
231
76.5
76.5 2.9
8_IM2G
single
15.0 33.6
15.0 3.6
15.0 3.6
15.0 3.6
9,5CWP
single
97.4 1
97.4 1.6
8..4
88.4 1
11_IZ6N
single
94.2 2.3
94.2 2.3
74.4
12.3P2W
single
87.6.445
87.6 2.5
79,6
79.6 2441
6KFQ
single
43.0 4.0
2.40
42.0
42,0 139
14_4BJI
single
79.6 +39
79.6 3.9
77.7 3.8
77.7 3.8
15 1A2J
single
23.7
79.4 a.7
78.4
78.4
16_3PR9
single
52.0 17.
27.
52.0
52.0
17.4GVW
single
40.4
30.4 =000
29.4
29.4 2.68
18.4LQ4
single
63.8 11.
1.3
61.8
61.8 1:
19_IM2G
single
4.2
4.8 2.2
4.8 622
4.8 2:2.2
20 3L86
single
76.6 23.
76.6 2.5
66.7 43.
66.7 3.0
21_ITKY
single
5.2
2,2
5.2
5.2
22 22CSTCS
single
49.2 +48
49.2
49,2
49.2 +48
23. 23_ISGW
single
64.0
666
61.0 +33
61.0
24_6OU0
single
37.8 +2.
37,8 2.2
33.5 2:0
33.5 1:00
25 4F3H
single
45.0 230
45.0 2..
45.0 230
45,0 3.0
26JIGIU
single
27.6
238
27.6
27.6 433
27 41QQ4
single
66.6 +50
66.6 +50
66.6 500
66.6 +550
28.6TCS
single
37.2 +3.
37.2 +33
37.2
37.2 +33
29 2LA0
single
31.4
31.4
31.4
31.4
6TCS
single
51.6 +6.3
51.6 663
51.6
51.6 6:33
31JIGIU
single
24.8 #29
#29
24.8 #29
24.8 429
32 IGBG
single
66.4 4.5
445
60.5 +41
60.5 +4.1
33.6TCS
single
43.6 2.9
2.9
43.6 +9
43.6 +29
34 6TCS
single
44.8 42277
44,8 227
44.8 4277
44.8 127
35_1A2]
single
43.8 +.4.0
43.8 +40
39.7 +36
39.7 +36
31

---

Published as a conference paper at ICLR 2026
Table 11: Task-level metrics for La-Proteina across all experiments.
#Frags
Entry
Experiment
Success Rate
Novel Success
Unique Clusters
SUN Score
1 1_SCPP
paired
82,4
19.4
16.5 105
2,5CWN
paired
98.2 =12
65.8 334
47.6
31.9 117
4K46
paired
59.0
58.2
50.1
49.5
5XJ7
paired
63.6 +4.7
63.6 +47
27.4 20
27.4 2.0
5.50J8
paired
68.6 330
0.0
0.0
0.0 2.00
6.2ZE5
paired
47.2 2.4
47.2 2.4
28.9 1.5
28.9 1:5
7_IDEX
paired
67.8 66.
44.9 4.2
44.9 +422
10 10.5DN1
paired
93.4
60.6
11.1
+.3
11 LKFQ
paired
98.0 1.7
0,0 000
0.0 0.0
0.0 0.0
12_1HU3
paired
0.8 1.2
0.0
0.0
0.0 2000
8 1IS1
paired
61.0
1.7
61.0
1.2 2.7
9_6KFQ
paired
99.0 +06
+45
20.8 0.1
3.9 1.0
13.4BJI
paired
24.2
24.2
17.3
17.3
14.5KZL
paired
98.6 1.0
64.0 2.3
17.2 1.2
11.2 A.04
15_4LQ4
paired
27.0 #3.7
=37
14.5
14.5 #.0
16 ISGW
paired
60.6 239
60,6 239
10.1
40.1 226
17_IBOI.
paired
25.6 2.2
25.6 2.2
25.6 2.2
25.6 +.2
18 6W5B
paired
80.0
16.6
28.2
5.9
19.1Q0S
paired
24.2 2.4
24,2 2.4
9.5 2.9
9.5 2.9.
20.4GVW
paired
67.0 #3.3
=211
17.7 2.
14.0 #06
21_3OSX
paired
33.8 1.8
9.8
15.0
4,4 4.1
22_176N
paired
91.8 1.5
0.0 000
0.0 =0.
0.0 0.0
5018
single
100.0
100.0
96.0
96.0
2.ITKY
single
43.4
46.6 234
46.
46.6 1.4
3.5XJ7
single
99.2 +0.7
99.2 o.7
91.2 0.7
91.2 +0
4_6KFQ
single
83.2 =48
83.2 =48
80.2
80.2 +4.7
5.5URP
single
96.4 222
96.4 a 2.2
96.4 2.2
96.4 2.2
60.6FFV
single
24.0 4.5
24.0
15.4
15.4
6_5XT7
single
86.6 23.
86,6 2.3
82.6
82.6 2.1
SOJ8
single
1.0
71.6 4.77
71.6 0.0
8_IM2G
single
42.6 +2.
42.6 2.9
42.6
42.6 2
9,5CWP
single
85.6 2.44
85.6 3.4
4.44
85.6 3.4
11_IZ6N
single
92.6 1.9
= 1.9
72.5
72.5 1.5
12.3P2W
single
93.4 22.
93.4 2.3
$8.4
88.4 2.2
6KFQ
single
26,8
34.4 207
17.4
15.3 3.0
14_4BJI
single
90.8 +22
90.8 2.2
90.8 222
90.8 2.2.
15 1A2J
single
1.9
96.4 1.9
96.4
96.4 1.9
16_3PR9
single
18.0 +5.1
5.11
18.0
18.0 5.1
17.4GVW
single
54.2 +17
54.2 =
48.1 1.5
48.1 1.5
18.4LQ4
single
92.4
92.4 =2.4
59.6
59.6
19_IM2G
single
29.2
29.2 666
29.2 6.6
29.2
20 3L86
single
84.6 4.44
84.6 2.4
72.7
72,7 3.0
21_ITKY
single
57.4 43.2
252
52.4 4.7
52.4 4.4.7
22 22CSTCS
single
53.6 +3,0
53.6 330
53.6
53.6 3.0
23. 23_ISGW
single
96.8 116
96.8 1.6
75.8
75.8 1:1
24_6OU0
single
77.8 +3.2
77.8 232
72.8
72.8
25 4F3H
single
40.8 #2.1
40.8 #11
40.8 +21
40.8 a22.1
26JIGIU
single
53.6
$3.6 23.1
53.6
53.6
27 41QQ4
single
86.8 44.4
86.8 44.
76.9
76.9 +39
28.6TCS
single
63.6
63.6
63.6 +34
29 2LA0
single
37.2 1.7
37.2
37.2
37.2
6TCS
single
17.6 +4.3
17.6 443
17.6
17.6 +43
31JIGIU
single
60.4 44.4
24.4
60.4
60.4 144
32 IGBG
single
88.6 1.4
88.6
84.6
84.6
33.6TCS
single
73.0 +3.3
73.0
67.9 3.1
63.9+33
34 6TCS
single
17.2 23.5
17.2 23.5
17.2
17.2
35_1A2]
single
50.4 +.5
50.4 +
50.4 1.5
50.4
32

---

Published as a conference paper at ICLR 2026
Table 12: Task-level metrics RFdiffusion2 across all experiments.
#Frags
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN Score
1 1_SCPP
paired
182
18,4
1824
18,422
2,5CWN
paired
64.2 #52
#52
62.2 +50
62.2 +50
4K46
paired
1.0
4.4
4.4
1.0
5XJ7
paired
15.0 1.8
15.0 1.8
15.0
15.0 1.8
5.50J8
paired
22.4
25.
223
4.3
6.2ZE5
paired
23,6
23.6 23.
23,6
23,6
7_IDEX
paired
1.5
1.4
1.4
1.4
10 10.5DN1
paired
17.8 =30
15.8
14.5
12.8
11 LKFQ
paired
21.8 4.2
14.0 3.4
19.8 3.8
12.7 3.1
12_1HU3
paired
0.0 20.0
2000
0.0
0.0 2.00
8 1IS1
paired
1.7
3.6
1.7
3.6
9_6KFQ
paired
22.6 +5.
20.8 +58
20.6
19.0 53
13.4BJI
paired
+.0
0.0
0.0
0.0
14.5KZL
paired
9.4
9.2
8.4
8.2 1.8
15_4LQ4
paired
0.0 20.0
0.0
0.0
0.0 2.0
16 ISGW
paired
10.0
10,0 2.7
10.0
10.0
17_IBOI.
paired
1.8
1.8 1.3
1.8 1.3
18 6W5B
paired
16.8
16.0
10.3
9.8
19.1Q0S
paired
1.6
1.6
1.6 1.0
1.6 1.6
20.4GVW
paired
7.8 239
33
=3A
6.8 #.4
21_3OSX
paired
1.0
0.0
0.0 20.00
0.0 a.000....
22_176N
paired
30.8 2.6
5.6
30.8 2.6
5.6 e:
5018
single
74.0
74.0
74.0
2.ITKY
single
1.6
1.6
1.4
1.6 1.6.11.44
3.5XJ7
single
83.0
336
83.0 3.6
83.0 336
4_6KFQ
single
52.0 +5.
52.0 =3333
52.0 +33
52.0 +53
5.5URP
single
40.0 4.7
40.0 4.7
40.0 4.7
40.0 4.4.7
60.6FFV
single
1.2
1.2 1.9
1.2 1.9
6_5XT7
single
17.4 23,8
17.4 238
17.4 238
17.4
SOJ8
single
#2.1
#2.1
68.0
68.0 4.1
8_IM2G
single
11.0 +17
11.0
11.0
11,0
9,5CWP
single
59.0 228
59.0 2.8
59.0
59.0 2288
11_IZ6N
single
34.0 2.5
2.5
34,0
34.0
12.3P2W
single
26.6
26,6 236
26.6
26.6
6KFQ
single
128
228
13.0
13.0
14_4BJI
single
17.4 449
17.4 +49
17.4
17.4
15 1A2J
single
27.2
27.2 24.4
2..4
27.2 2:4.A
16_3PR9
single
38.2
38.2 25.
38.2
38.2
17.4GVW
single
5.4
5.4
5.4
+.4
18.4LQ4
single
+2N
=28
29.2
29,2
19_IM2G
single
2.7
2
2.6
2.6 2.7
20 3L86
single
28.8 #28
28.8 #28
28.8 228
28.8 428
21_ITKY
single
0.0
4.0
0.0
0.0 40,00
22 22CSTCS
single
26.8 +3.1
26.8 +311
26.8
26.8 3.11
23. 23_ISGW
single
13.4
13.4 +38
13.4
13.4
24_6OU0
single
25.6
25.6 2.7
25.6
25.6 2:7
25 4F3H
single
49.4
19.4
19.4
19.4
26JIGIU
single
10.0 #2.6
=2.6
10.0
10.0 4.6
27 41QQ4
single
19.2 +47
+4.
4.7
19.2 +4.7
28.6TCS
single
15.2 +4.0
+40
15.2
15.2 +40
29 2LA0
single
1.6
1.6
1.6 20.8
1.6 1.6.
6TCS
single
9.8 #28
9,8 23
9.8 =28
9.8 +288
31JIGIU
single
1.6
1.6
1.6
1.6 1.0
32 IGBG
single
28.8 4.66
28.8 4.6
28.8 466
28.8 446
33.6TCS
single
32.2 +600
32.2 +6.0
32.2 660
32.2 +60
34 6TCS
single
25.6 234
25.6
25.6
25.6
35_1A2]
single
10.2 2.4
10.2 +144
10.2
10.2
33

---

Published as a conference paper at ICLR 2026
Table 13: Task-level metrics for REdiffusion across all experiments.
#1 #Frags
Entry
Experiment
Successful, %
Novel, %
Unique, %
SUN Score
3
1_SCWP
paired
84,8 2.79
27.0 2600
48,9 1.61
15
2,5CWN
paired
83.0 335
0.50
38.3 = 1.65
18.7 #04
4
3 4K46
paired
25.8 2:14
25.8 2.14
22.6
22.6
4 SXJ7
paired
79.0
79.0 2.45
31.4 +097
31.4
5_5018
paired
20.8 3.12
6.8 2.32
a 1.95
4.2
6.2ZES
paired
41.8 2:48
41.8 2.48
38.5 2.29
38.5
7_IDEX
paired
21.2 4:26
21.2 + 4.26
15.7:
15.7 #31
5
10.5DN1
paired
57.8
44.6
22.0
17.0
11 6KFK
paired
84,4 3.44
8.8 0.98
42.2 1.72
4.5
12.IHU3
paired
0,00
0.0
1IS1
paired
13.6 23762
0.6 0.469
13.6 2.72
0.6
9_6KFQ
paired
94.0 2.00
28.8 2.79
32.5 = 0.69
10,0
13.4BJI
paired
2.6
2.6
2.6
2.6
14_5KZL
paired
54,2 5:27
45.4 5.611
19.5 1.90
16.3 220
15_4LQ4
paired
+ 3.49
#3.49
7.3 #.30
7.3
16 1SGW
puired
38.2 4.31
38.2 4.31
29.4
29.4
17_IBOL
paired
14.0
+ 3.69
3.47
13.2 3.5
6W5B
paired
30.4
29.2
9.8
19_1Q0S
paired
39.0 7.40
39.0 7.40
14.5 2.75
14,5 2:7
20.4GVW
paired
42.6
42.6 3.83
14.2 = 1.28
14.2 =..
21 30SX
paired
22.4 5.16
10.0 3,50
10.2 2:35
+.5
22_176N
paired
87.8 2:48
0.0 0.00
0.0 0.00
0.0
5018
single
100.0
100.0
98.0
2.ITKY
single
79.4 a
79.4 1:38
79.4
79.4 1M
3.5XJ7
single
100.0 = 0.00
100.0 0:00
99.0 =.00
99.0 +00
4_6KFQ
single
97.8
97.8
97.8 0075
97.8 +
5_5URP
single
82.2 3.54
82.2 a 3.54
82.2 a
82,2
10 6FFF
single
34.8
34.8
34.8 4.17
34.8
6_5XJ7
single
93.0 2:10
93.0 2.10
93.0 2.10
93.0 221
7 5018
single
0.99
a 0.89
93.1 2.85
2.088
8_IM2G
single
41.6 5:50
41.6 5.50
41.6 5.50
41.6
9.5CWP
single
99.0 0.63
99.0 0.63
97.0 a 0.62
97.0 0.6
3
11_126N
single
44.92
77.8 4.92
63.5
1.400
12.3P2W
single
87.0 = 1.67
87.0
83.0
83,0 1.6
6KFQ
single
93.8 #:00
2.04
86.7 = 1.89
86.7 #19
14_4BJI
single
96.0 2.19
96.0
95.0 2.17
95.0 +..0+22
15 1A2J
single
96.4 1.85
96.4 1.85
96.4 1.85
96.4 1.9
16_3PR9
single
70.8 3.97
70.8 3.97
70.8 a 3.97
17.4GVW
single
62.8
62.8 2:98
42.8
42.8 2.0
18 444LQ4
single
91.4 0.80
91.4 0.80
79.5 o.70
79.5 =00.
19_IM2G
single
24.8 3.06
24.8
24.8
24.8 3.1
20 3L86
single
64.0 2.37
64.0 2.37
62.1 2.30
62.1 2:3
21_ITKY
single
0.800
0.500
4..8
22.6TCS
single
47.8 4.87
47.8 4.87
47.8 4487
47.8 +.49
23,1SGW
single
52.2
52.2 + 1.94
52.2 + 199
52.2 +
24_60U0
single
41.4 4.227
41.4 a
41.4 a
41,4
25 4F3H
single
71.0 (1.0=1.. 1.10
71.0 1.10
65.7 1.01
65.7 1.0
26L1GIU
single
82.4 2:33
82.4 2.33
82.4 2:35
82.4 +23
27 41.Q4
single
78.8 3.71
78.8
77.8
77.8 +37
28 28CTTCS
single
50.4
50.4 3.88
50.4 3.88
50.4 +
29 2LA0
single
1.00
2.00
7.4 3.01
7.40
30 6TCS
single
54.4 4.22
54.4 4
54.4 4
54.4+442
31-IGIU
single
38,2 2228
38.2 + 8.28
8:28
283
32. IGBG
single
37.8 4.35
37.8
32.5 4.75
32.5 4.7
33,6TCS
single
54.2 4466
54.2 4.66
4.58
53.2 +46
34 6TCS
single
14.4 4.338
14.4 4..38
14.4
14.4 13.4
35_1A2J
single
60.4 2.65
60.4 +.65
50.7 2.23
50.7 +22
34

---

Published as a conference paper at ICLR 2026
Table 14: Entry-level metrics for Genie2 across all experiments.
#F Faag
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN Score
3
1.5CWP
paired
4.11
4.2 2211
4.2
4.2
2,5CWN
puired
116
91.6
74.
74.3 2133
3 4K46
paired
2.1
23.8
23.8
23.8 #21
4 5X17
paired
90.8 225
90.8 225
56.5
56.5 1.5
5.5018
paired
0.00
2.0
0.0
0.0
6.2ZES
paired
53.0
53.0
49.1
49.1 25.0
7_1DEX
paired
30.0 +32
30.0 5.2
28.3 449
28.3 4.9
5
10,5DNI
paired
19.0
19.0
16.1 =23
16.1
11 11.6KFQ
paired
90.2 1.7
42.6 2.2
55.5 a
26.2 1.3
12_IHU3
paired
2.0
0.0
0.0
0.0
8 IIS1
paired
0.0
0,0
0.0 1.000.0
4.0
9_6KFQ
paired
a.
97.0
=7.4=03
37.4 +0.
13.4BJI
paired
0.0
+0.0
+.0
0,0
14.5KZL
paired
14.8 1.5
14.8 5.5
10,9
10.9 4.40
15.4LQ4
paired
0.0 2.00
s0.1
0,0
0,0
16 ISGW
paired
2.4
2
2.4
2.4
17_IBOI.
paired
22.2 4.00
22.2 4.00
22.2 4.0
22.2 +.0
7
6W5B
paired
12.2
12.2
8.1
8.1
19_IQ0S
paired
21,8 6.1
21.8 2.11
18.2
18,2 250
20.4GVW
paired
6.4
39.6 6.6
22.4
#37.7
21 30SX
paired
13.2 2
13.2
7.2 2
7.2 +.2
22.176N
paired
88.6 a 3.
7.2 e 1.6
49,2 1.8
4.0 a.
1 5018
single
100.0
100.0
100.0
100.0
2.1TKY
single
4.9
4.8 L.9
4.8 1.9
4.8 4
3.5XJ7
single
100.0 =00
100.0 +00
=.0
100.0 n.0
4_6KFQ
single
92.8 20
92.8 2.0
92.8 2.0
92.8 =.0
5_5URP
single
81.6 3.5
81.6 3.5
81.6
81.6 3.5
2
10. 6FFF
single
41.8
41.8
41.8
41.8
6_5XJ7
single
87.6 6.611..
87.6
87.6
87.6 1.6
7
single
99.0 4.6
99.0 4.6
2.6
99.0 206
8_IM2G
single
54.8 2.5
54.8 2.5
54.8 2.5
54.8 2.5
9.5CWP
single
97.0
97.0 1.I
97.0 a
97.0 a 1.1
3
11.126N
single
89.4
89.4 0.5.
89.4
12.3P2W
single
73.6
73.6
73.6
73.6 20.5
6KFQ
single
#20
83.8 2:0
83.8 =20
=20
14_4BJI
single
91.2 +
91.2
90.2 +35
90.2 3.5
15. 1A2J
single
94.0
94.0 2.4
94.0 22
94.0 #.4
16 16_3PR9
single
4.3
12.6 3.3
2.3
233
17.4GVW
single
35.2 +.4
35.2 5.4
35.2
35.2 e.4
18 4LQ4
single
63.4
63.4 +33
63.4
=3.3
19_IM2G
single
+24
2.4
7.2 2:4
20 3L86
single
59.8
59.8 2.6
59.8 226
59.8 #22
21_ITKY
single
1,0
1.0
1.0
1.0
22.6TCS
single
45.6+
45.6 +34
45.6
45.6 +34.
23,1SGW
single
21.4+
21.4 +31
21.4
21.4 =3.11
24_60U0
single
34.
37.8
37.8 244
37.8 2.4.
25 4F3H
single
56.8
56.8 434
56,8
56.8 #3..4
26LIGIU
single
45.6
40.6
40.6
=5.0
27 41.Q4
single
86.2 +
86.2 3.1
85.2 30
85.2 +
28 2LCS
single
57.4 +21
57.4 +.1
57.4
$7.4 +21
29 2LA0
single
1.5
2.5
1.5
7.4
30.6TCS
single
56.2 +2
56.2 +7
56,2 227
56.2 #22
7
31.IGIU
single
24.8 +5.1
24.8 2.11
24.8
#5.1
32. 1GBG
single
66.6
66.6
66.6
66.6 +38
33.6TCS
single
66.0
66.0 +.8
66.0
28
34 6TCS
single
74.4
74.4
74.4
74.4 14.6
35.1A21
single
56.6+334
56.6 +34
55.6
55.6 +33
35

---

Published as a conference paper at ICLR 2026
Table 15: Entry-level metrics for FrameFlow across experiments.
#F FFaags
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN Score
3
1.5CPP
paired
15
1194
51,4
$1.4
2,5CWN
puired
300
80.6 3.2
77.2
75.4 =30
3 4K46
paired
13.8 39
13.8
13.8
13.8 #3.9
4 5X17
paired
24.2 33
24.2 334
24.2+34
24.2 3.4
5.5018
paired
49.0
35.0
36.4
2.3
6.2ZES
paired
14.8 231
14.8
14.8
14.8 #3.1
7_1DEX
paired
5.2 +2.
5.2
5.2
5.2
5
10,5DNI
paired
38.6
24.0
272
16.9
11 11.6KFQ
paired
40.6 448
20.2 27
25.6
12.8 1.7
12_IHU3
paired
0.0
0.0
0.0
0.0
8 IIS1
paired
13.2
11.2
13.2
11,2 1.2
9_6KFQ
paired
228
34.6 5.3
33.1 +1.
20.5 3.2
13.4BJI
paired
0.6 008
0.6
0.6
0.6
14.5KZL
paired
44.0 1
32.6
30.8 227
22.8 3.0
15.4LQ4
paired
0.6
0.6
0.6
16 ISGW
paired
13.4 4.9
13.4 4.9
13,4
13,4 2.49
17_IBOI.
paired
10.8 2.5
10.8 2.5
10.8 2.5
10.8 2.5
6W5B
paired
27.4
25.6
11,6
10.8
19_IQ0S
paired
11.2 2.7
11.2 2.7
10,0
10,0 2.4
20-4GVW
paired
+
27.0 4.3
18.8 2.6
=2.
21 30SX
paired
2.2 2.7
2.2 20.7
2.2 u.7
2.2 a.7
22.176N
paired
38.4 1.0
4.8 2.6
38.4 =
4.8 2.6
5018
single
85.2
85.2
85.2
2.1TKY
single
19.4
19.4
19.4
19.4 2.111
3.5XJ7
single
2.6
85.0 2.6
85.0 2.6
2.6
4_6KFQ
single
60.8 +4.1
60.8 +.1
60.8 =4.1
60.8 e.111
5_5URP
single
56.2 5.0
56.2 5.0
56.2 5.0
56.2 5.0
2
10. 6FFF
single
10.
10.0
10.0
10.0
6_5XJ7
single
41.8 1.9
41.8 4.9
41.8 25.
41.8 25.9
7
single
2.4
91.8 2.4
91.8 4.4
#2.4
8_IM2G
single
4.2
4.2 2.1
4.2 2.1
4.2
9.5CWP
single
67.6 a
67.6 1.7
67.6 =.7
67.6 1.7
3
11.126N
single
58.4
2.4
12.3P2W
single
55.4
55.4 438
55.4
55.4 23N
6KFQ
single
19.8 #25
19.8 #25
19.8 #2.
14_4BJI
single
57.6
57.6
57.6
57.6 4.2
15. 1A2J
single
66.6 2.6
66.6 2.6
66.6
66.6 #.2.
16 16_3PR9
single
43.0
43.0
43.0
23.2
17.4GVW
single
8.2
8.2 2.5
8.2 852
8.2
18 4L
single
31.6 +20
31.6 2.0
31.6
=20
19_IM2G
single
1.6 0.5
1.6 6.5
1.6
1.6
20 3L86
single
20.8 4:47
20.8 4.7
20.8 247
20.8 a44.7
21_ITKY
single
0.0
4.0
4,0
22.6TCS
single
13.8 5.8
13.8 5.88
13.8 =5x
13.8 558
23,1SGW
single
10.2 +
10.2 334
10.2 234
10.2 =34
24_60U0
single
6.0
6.0 1.8
6.0
6.0
25 4F3H
single
14.2
14.2 3.8
14.2 2M
14,2 a33.
26LIGIU
single
12.6 422
12.6 4:2
12.6 222
=22
27 41.Q4
single
44.4 233
44.4 2.3
44.4 -33
44.4 2.3
28 2LCS
single
9.6
9.6
9.6
9.6 1.9
29 2LA0
single
0.0
0.0
0.0
0.0
30.6TCS
single
13.8 +:27
13.8 +
13.8 227
13.8 #12
7
31.IGIU
single
12.2 +21
12.2 #2.1
12.2
#21
32. 1GBG
single
17.0
17.0 477
17.0
17.0 +47
33.6TCS
single
22.2 +28
22.2 +28
22.2
+38
34 6TCS
single
5.2
5.2 423
5.2
5,2
35.1A21
single
9.0 +28
9.0 2
9,0
9,0
36

---

Published as a conference paper at ICLR 2026
Table 16: Entry-level metrics for DPLM-3B across all experiments.
#F FFaags
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN.Score
3
1.5CPP
paired
0.00
0.0 0.0
0.0
0,0
2,5CWN
puired
=0.0
=..0
0.0
0.0
3 4K46
paired
38.6 26
0.0
0.0
0.0
4 5X17
paired
23.6 2.8
0.0 0.0
0.0 0.0
+.0
5.5018
paired
0.0
0.0
4.0
6.2ZES
paired
0,0
0.0
0.0 0.0
0.0
7_1DEX
paired
0.99
o.0
0.0 0.0
+.0
5
10,5DNI
paired
29.0
0.0
0.0
0.0
11 11.6KFQ
paired
0.6 0.5
0.0 0.0
0.0 0.0
0,0 0.0
12_IHU3
paired
2.0
0.0
2.0
0.0
8 IIS1
paired
1.6
0.0 0.0
0.0 0.0
0.0
9_6KFQ
paired
66.0 + 3.9
0.0 -0.0
0.0 0.0
0.0 +.00
6
13.4BJI
paired
0.0
0.0
0.0
0.0
14.5KZL
paired
1.5
0.0
0.0 0.0
0,0
15.4LQ4
paired
2..0
0.0
0.0
0.0
16 ISGW
paired
0,0 2.0
0,0 20.0
0.0 =
0.0
17_IBOI.
paired
22.4 + 1.9
19.8 1.5
12.3 1.0
10.9 +0.8
6W5B
paired
53.6
0.0
0.0
0,0
19_IQ0S
paired
0,0 0.00
0,0 0.0
0.0
0,0
20.4GVW
paired
0.0 4.00
0.0
0.0
0.0
21 30SX
paired
2.2
0,0 2.0
0,0
0,0
22.176N
paired
0.0
0.0 0.0
0.0 0.0
0.0 8.0
1 5018
single
100.0
100.0
11.0
11.0
2.1TKY
single
100.0
29.0
14.3
4,1 1:0
3.5XJ7
single
1.8 1.7
70.8 =.2
1.7
4_6KFQ
single
37.6
6.0 #0.6
18.8 233
3,0
5_5URP
single
0.0
0.0 0.0
0.0 a 0.0
0.0
2
10. 6FFF
single
0.0 =..0
0.0
0.0
6_5XJ7
single
2.4
0.4 20.
2.4 1.4
0.4
7
single
4.0
43.2 4,0
6.3 1.5
4..
8_IM2G
single
0.0
0.0 +..0
0.0 +.0+00
0,0+00
9.5CWP
single
3.8 227
3,8 2.7
3.8 277
3,8
3
11.126N
single
0.0
0.0
0,0
12.3P2W
single
0.0
2.0
0,0
0,0
6KFQ
single
17.2 447
0.0
0.0
0,0
14_4BJI
single
29.2 2.8
14.8 1.5
20.1 1.9
10.2
15. 1A2J
single
1.6 2088
1.0 2.6
1.6 i 0.8
1.0
16 16_3PR9
single
77.2 135
27.6
42.6 :
a 1.3
17.4GVW
single
1.9
0.0 =0.
0.0 0.0
0.0 a.
18 4LQ4
single
+35
7.0
24.8 =29
1.4
19_IM2G
single
2.2
0.0 e..0
0.0 0.0
0.0
20 3L86
single
2.0
0.0 2.0
0.0
0.0
21_ITKY
single
40.8
0.0
0.0
0,0
22.6TCS
single
11.0
11.0
11.0
11,0
23,1SGW
single
+0.7
=o.1
0.0 +.00
0.0 0.0
24_60U0
single
0.0
0.0
0.0
0.0
25 4F3H
single
7.0
4.7 + u.5
2.4
26
single
20.0
0.0
0.0
4:00
27 41.Q4
single
4.4
0.0 +0.
0.0
0.0
28 2LCS
single
3.4
3.4 +..4
3.4 2.4
3,4 +24
29 2LA0
single
0.0
0.0 2.0
0.0
4.0
30.6TCS
single
47.2 24
47.2 +
36.7 1.9
36.7 $1.
7
31.IGIU
single
400
=0.11
0.0 u.0
4.0
32 1GBG
single
22.4
6.0 +2.
14.9 2.0
4,0
33.6TCS
single
18.4 +411
18.4 +.1
18.4
18.4 +11
34 6TCS
single
4..1
32.6
29.6
28.8 239
35.1A21
single
1.4 +1.0
1.4 1.0
1.4 1.410
1.4
37

---

Published as a conference paper at ICLR 2026
Table 17: Entry-level mefrics for DPLM-650M across experiments.
#F FFags
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN.Score
3
1.5CPP
paired
2.00
0.0
0.0
0,0
2,5CWN
puired
=0.0
=.0
0.0 200
0.0
3 4K46
paired
27.6
0.0
0.0
0.0
4 5X17
paired
1.9
0.0 0.0
0.0 0.0
+.0
5.5018
paired
0.00
0.0
0.0
4.0
6.2ZES
paired
0,0
0.0
0.0 0.0
0.0
7_1DEX
paired
2.0
o..
0.0 0.0
+.0
5
10,5DNI
paired
7.4
0.0
0.0
0.0
11 11.6KFQ
paired
8.6 1.6
0.0 0.0
0.0 0.0
0,0 0.0
12_IHU3
paired
0.0
0.0
0.0
8 IIS1
paired
1.2
0.0
0.0 0.0
0.0
9_6KFQ
paired
26.6 + 38
0.0 =0.0
0.0 0.0
0.0 +.00
6
13.4BJI
paired
0.0
0.0
0.0
0.0
14.5KZL
paired
&.7
0,0
0.0
0,0
15.4LQ4
paired
0.0
0.0 2.0
0.0
16 ISGW
paired
0.0
0.0
0.0 0.0
0.0
17_IBOI.
paired
11.8 3.1
8.0 2.1
7.4 2.0
+.0
6W5B
paired
45.0
0.0
0.0
19_IQ0S
paired
0,0 2.00
2.0
0.0
0,0
20.4GVW
paired
0.0 =..
0.0
0.0
0.0
21 30SX
paired
0,0
0,0
0,0
0.0
22.176N
paired
0.0 0.0
0.0 0.0
0.0 x 0.0
0.0 8.0
1 5018
single
100.0
100.0
11.0
11.0
2.1TKY
single
100.0
37.4 +
=2.4
12. 2.05
3.5XJ7
single
85.0
7.2 2.8
56.7 3.6
1.9
4_6KFQ
single
34.2 +
3.4 =0.
22.8 2333
2.3
5_5URP
single
0.0 0.0
0.0 0.0
0.0 a 0.0
0.0
2
10. 6FFF
single
0.0
0.0
0.0
6_5XJ7
single
0,0
2.0
0.0
0.0
7
single
30.4
30.4 6.3
7.9 1:1.6
7.9
8_IM2G
single
6.0
0.0 +..0
0.0 n.
0.0
9.5CWP
single
1.7
1.2 0.7
1.2 0.7
007
3
11.126N
single
0.0
4000
12.3P2W
single
0.0
0.0 2..0
0.0
0.0
6KFQ
single
2000
0.0
0,0
14_4BJI
single
21.8 4.3
6.6 2.2
13.6 2.7
4.1
15. 1A2J
single
0.0 2.00
0.0 0.0
0.0 1 0.0
0.0
16 16_3PR9
single
11.4 2.11
3.6 + 1.9
9.1
17.4GVW
single
4.6
0.0 =..0
0.0 no
0.0 a.
18 44Q44
single
11.6 +2
= 1.2
11.6 2.
2.2
19_IM2G
single
0.0
0.0 6.0
0.0 0.0
0.0
20 3L86
single
0.0 200
0.0 2.0
0.0 a 0.0
0.0
21_ITKY
single
0.4
0.0
0.0
0,0
22.STCS
single
8.2
8.2 +
7.2 1.0
+.2
23,1SGW
single
0.0 +.0
=..1
0.0 0.0
0.0
24_60U0
single
0.0
0.0 0.0
0.0
4.0
25 4F3H
single
7.2
2.4 a 1.4
1.4
1.2
26
single
0.0 20,0
0.0 0.0
27 41.Q4
single
3.0
0.0 +0.
0.0 0.00+00
0.0+
28 2LCS
single
+0.7
0.8 +0.7
0.8 0.7
+.0
29 2LA0
single
0.0
0.0 2.00
0.0 0.00000
4.0
30.6TCS
single
31.4+29
51.4 2.9
40.3 2.3
40.3 =23
7
31.IGIU
single
4000
=0.11
u00
4:00
32 1GBG
single
10.4 4.3
0.2 +..4
10.4 443
0.2 04
33.6TCS
single
7.8 229
7.8 +.9
5.2 +00
5.2 20
34 6TCS
single
30.0
27.4
25.7
23.5 13.1
35.1A21
single
6.8 +12
5.4 1.6
4.5 +0s
3.6
38

---

Published as a conference paper at ICLR 2026
Table 18: Entry-level metrics ESM3 (seq struct) across all experiments.
#F Frags
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN.Score
3
1.5CWP
paired
0.0
0.0
0.0
0,0
2,5CWN
puired
=0.0
0.0 =.0
0.0 200
0.0
3 4K46
paired
0.0 :0.0
0.0
0.0
0.0
4 5X17
paired
5.8 1.7
0.0 0.0
0.0 0.0
+.0
5.5018
paired
0.0
0.0
4.0
6.2ZES
paired
0,0
0.0
0.0 0.0
0.0
7_1DEX
paired
0.0 0.0
o..
0.0 0.0
+.0
5
10,5DNI
paired
1.1
0.0
0.0
0.0
11 11.6KFQ
paired
0.0 0.0
0.0 0.0
0.0 0.0
0,0 0.0
12_IHU3
paired
0.0
0.0
0.0
8 IIS1
paired
0,0 2.0
0.0 0.0
0.0 0.0
0.0
9_6KFQ
paired
0.8
0.0 n.0
0.0 0.0
0.0 +.00
13.4BJI
paired
0.0
0.0
0.0
0.0
14.5KZL
paired
0.0
0.0
0.0
0,0 0.0
15.4LQ4
puired
0.0
0.0 200
0.0
16 ISGW
paired
0,0
0.0
0.0 u.0
0.0
17_IBOI.
paired
6.0
0.0 n..
0.0 + 0.0
0.0 +.00
6W5B
paired
0.0
0.0
0.0
0,0
19_IQ0S
paired
0,0
0.0
0.0
0,0
20.4GVW
paired
0.0
0.0
0.0 0.0
0.0
21 30SX
paired
0,0
0.0 2.0
0,0 3 0.0
0.0
22.176N
paired
0.0 0.0
0.0 0.0
0.0 0.0
0.0 9.0
5018
single
35.8
35.8
35.8
35.8
2.1TKY
single
2.0
2.0 2
2.0 1.3
2.0 2.0
3.5XJ7
single
0.0 n.0
0.0 =..1
0.0 + 0.0
0.0 a.00
4_6KFQ
single
0.0
0.0 e..
0,0 1,0+00
0.0
5_5URP
single
0.0 0.0
0.0 0.0
0.0 a 0.0
0.0
2
10. 6FFF
single
0.0
6_5XJ7
single
3,4
3,4 2.2
3,4 2:2
3.4
7
single
2.0
0.0
4.000
8_IM2G
single
0.0
0.0 +..0
0.0
+.0+000
9.5CWP
single
12.8 a 2.4
12.8 2.4
12.8 2.4
12.8 #.4
3
11.126N
single
0.00
0.0
0.0
12.3P2W
single
0.0
0.0 2.0
0,0
4.0
6KFQ
single
0.0
2..1
0.0
0.0
14_4BJI
single
+.2
2.2 1.2
2.2 1.2
2
15. 1A2J
single
0.0 2.00
0.0 2.0
0.0 a 0.0
4.0
16 16_3PR9
single
0.0
1.0
4:00
17.4GVW
single
0.0
0.0 e.0
0.0
0.0
18 4LQ4
single
0.0
=..1
0.0
0.0
19_IM2G
single
0.0
0.0 o.0
0.0
0.0
20 3L86
single
0.0
0.0 2.0
0.0 0.0
0.0
21_ITKY
single
4.0
0.0
0.0
0.0
22.STCS
single
0.0
0.0 o..
0,0
0.0
23,1SGW
single
+..0
=..1
0.0
0.0
24_60U0
single
0.0
0.0 0.0
0.0
0.0
25 4F3H
single
000
0.0 2..
0.0 100+00 0.0
0.0
26
single
2..0
0.0
27 41.Q4
single
1.0
1.0 1.1
1.0
1.0
28 2LCS
single
0.0 n.0
0.0 -..
0.0 + 0.0
0.0 +.00
29 2LA0
single
0.0
0.0
0.0 0.00
0.0
30.6TCS
single
9.4
9.4 2.6
9.4 +
9,4
7
31.IGIU
single
0.0 400
=0.1
0.0 u00
4.00
32 1GBG
single
0.0 +.0
0.0 +..
0.0 00
0.0
33.6TCS
single
+.8
0.6 0.8
0.6 + 0.8
34 6TCS
single
9.4
9.4
9.4 1.4
9.4 43.4
35.1A21
single
0.0 n.0
0.0 +0.0
0.0 no
0.0 +00
39

---

Published as a conference paper at ICLR 2026
Table 19: Entry-level metrics for ESM3 (seq) across experiments.
#F FFaags
Entry
Experiment
Success Rate
Novel Success (%)
Unique Clusters
SUN Score
3
1.5CPP
paired
0.00
0.0
0.0
0,0
2,5CWN
puired
1.4
1.2
1.4 =12
1.4
3 4K46
paired
39.8
0.0 20000.0
0.0
0.0
4 5X17
paired
16.6 +.7
0.0 6.0
0.0 0.0
0.0 +.0
5.5018
paired
0.0
0.0
4.0
6.2ZES
paired
0,0 2.0
0.0
0.0 1.0
0.0
7_1DEX
paired
0.0 0.0
o..
0.0 0.0
0.0
5
10,5DNI
paired
38.8 +
0.0
0.0
0.0
11 11.6KFQ
paired
4.0 0.9
0.0 0.0
0.0 0.0
0,0 0.0
12_IHU3
paired
0.0
0.0
0.0
8 IIS1
paired
2.2 1.5
0.0
0.0 0.0
0.0
9_6KFQ
paired
14.8 3.7
0.0 -0.0
0.0 0.0
0.0 +.00
6
13.4BJI
paired
0.0
0.0
0.0
0.0
14.5KZL
paired
2.8 1.2
0,0
0.0
0,0 0.0
15.4LQ4
paired
0.0
0.0 2.0
0.0
16 ISGW
paired
0,0
0.0
0.0 u.0
0.0
17_IBOI.
paired
0.0 6.0
0.0 n..
0.0 + 0.0
0.0 +.00
6W5B
paired
29.8
0.0
0.0
0,0
19_IQ0S
paired
1.4
0,0
0.0
0,0
20.4GVW
paired
0.0 =.00
+.0
0.0 0.0
0.0
21 30SX
paired
7.0
0.0
0,0 8
0.0
22.176N
paired
0.0 n.0
0.0 n.0
0.0 0.0
0.0 9.0
5018
single
100.0
95.2
18.8
17.9
2.1TKY
single
30.
30.6 2.0
26.5
26.5 a.7
3.5XJ7
single
91.2 +48
14.8 3.4
32.6 1.7
5.3 3.3++13
4_6KFQ
single
18.4 +
16.6 3.4
11.5 2.5
10.4 -.4e22.11
5_5URP
single
0.0 0.0
0.0 0.0
0.0 a 0.0
0.0 0.0
2
10. 6FFF
single
0.0 +..0
0.0
6_5XJ7
single
64.4
52.4 3.0
37.7 1.8
30.6 2
7
single
7.6 2.88
7.6 0.8
3.8
4.8
8_IM2G
single
+0.0
3.6 1.4
7.5 +11
2.7
9.5CWP
single
0.0
0.0 0.0
0.0
0,0
3
11.126N
single
0.0
1.0
12.3P2W
single
0,0
0.0
0,0
0,0
6KFQ
single
200
2.0
0.0
14_4BJI
single
4.0
4.0 2.2
4,0 4.2
4,0
15. 1A2J
single
2000
0.0 2.0
0.0 a 0.0
0.0
16 16_3PR9
single
4.3
8.4 a 1.9
55.7 2.6
17.4GVW
single
1.5
1.2 1.5
1.2 1.5
1.2
18 4LQ44
single
18.6
6.6 =
13.9 3.3
19_IM2G
single
2.7
0.0 e..
0.0 0.
0.0
20 3L86
single
2.2
0.0 2.0
0.0 3 0.0
0.0
21_ITKY
single
2.6
4.0
0,0
22.STCS
single
11.8 29
11.8 2.
11.8 2.9
11.8 2.9
23,1SGW
single
0.0 +.00
=o.1
0.0 0.0
+0000
24_60U0
single
0.0
0.0 2.0
0.0
0.0
25 4F3H
single
21.2
13.6 1.9
10.6 209
6.8 239
26
single
2.0
100
4.0
27 41.Q4
single
+.4
0.0 +0.0
0.0 0.00+00
0.0
28 2LCS
single
7.8 +30
7.8 3.0
6.7 +26
6.7 6.7+226
29 2LA0
single
0.0
0.0
0.0
4.0
30.6TCS
single
48.0
48.0 +
33.7 3.6
33.7 #36.
7
31.IGIU
single
4000
=0.11
0.0 u.0
4:00
32 1GBG
single
34.4 +
9.8 1.2
21.5 1.5
6.1 +007
33.6TCS
single
19.8 454
19.8 5.4
18.8 5.1
18.8 +5.1
34 6TCS
single
32.6 4.3
32.6
28.8
28.8 238
35.1A21
single
5.0
4.2 1.5
5.0 1.9
4.2
40


Reviews:
[{"review_id": "kySCAdBGHl", "reviewer": "Reviewer_v8Ac", "summary": "This paper presents GeomMotif, a benchmark for evaluating geometric preservation in protein generation. Unlike existing benchmarks focusing on functional motifs, GeomMotif targets general structural fragments, assessing a model’s ability to maintain 3D geometry during motif scaffolding. The authors construct 57 benchmark tasks from the Protein Data Bank, covering diverse structural and physicochemical contexts. Results show that different model classes (sequence-based vs. structure-based) have distinct strengths, highlighting the challenge of arbitrary geometric preservation. Overall, GeomMotif provides a systematic and general framework for assessing geometry-aware protein generation.", "questions": "Q1 The paper constructs a diverse set of 57 benchmark tasks and systematically evaluates existing models. Could the authors further **categorize these tasks** (e.g., by difficulty, biochemical properties, or structural complexity) to form **tiered subsets** that would facilitate more detailed and interpretable future evaluations?\n\nQ2 The selection of structure-based baselines seems limited to Genie2, RFdiffusion, and FrameFlow. Could the authors clarify the rationale behind this choice and consider including **more recent methods** such as Proteina or RFdiffusion2 to provide a more comprehensive comparison?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clearly defines an important and underexplored evaluation problem in protein generation—geometric preservation.  \n- Proposes a well-constructed and diverse benchmark with 57 tasks covering various structural and physicochemical contexts.  \n- Provides systematic evaluation metrics (scRMSD, pLDDT) that go beyond success rates and enable detailed model comparison.  \n- Offers valuable insights into the complementary strengths of sequence-based and structure-based generative models.", "weaknesses": "- Lacks direct comparison with existing motif-scaffolding benchmarks (e.g., the RFDiffusion benchmark), which would better highlight the novelty and distinct contributions of GeomMotif.  \n- The evaluation focuses mainly on general geometric cases, without testing more complex or realistic scenarios such as symmetric motifs, small functional motifs, or protein target chains as motifs, which would demonstrate broader applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ew4iqnmjsG", "reviewer": "Reviewer_TUmD", "summary": "The authors construct a motif scaffolding benchmark through selecting diverse proteins from the PDB and computationally selecting motifs. The benchmark only measures whether the motif geometry is maintained after folding the designed sequence. They then compare a wide range of current state of the art protein design models on the benchmark, finding a clear difference between structure based models and sequence based models. The authors further analyze which motifs models find most challenging in terms of structural properties such as secondary structure composition and burial ratios.", "questions": "Why not include some small motifs in the benchmark? Scaffolding the geometry of small functional sites is also an important capability for protein design models.\n\nWould there be a way to include motifs that have more loop content? Sometimes loops can mediate important interactions and scaffolding their shape would be a practically relevant task.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Compared to MotifBench, the author's benchmark has a distinct advantage in that it doesn't require the generative model to design a structure because they do not compute self-consistency RMSD and only an RMSD on the motif after folding. This means they can compare structure-based and sequence-based models on the same benchmark. In doing so the authors reveal a significant difference between these two styles of model. Given that all models were run on the same benchmark by the same authors without bias as to a specific model, I believe these kinds of unbiased results are quite beneficial to the research community.\n\nI also appreciated the analysis into which kinds of motifs are difficult for different models. For example the relative ease of scaffolding high helical content motifs and the difficulty with low helical content motifs is something that could be considered part of the folklore of protein design but not has evidence backing up this claim. The difficulty with buried motifs is also interesting and perhaps a new perspective for which motifs can be challenging to scaffold in practice.", "weaknesses": "I believe the motivation for the benchmark given in the introduction is unclear. It is claimed that MotifBench has a diagnostic blindspot in that it conflates geometric preservation with maintenance of function. I think what the authors mean here is that MotifBench requires both motif preservation and a low scRMSD to be a success whereas their benchmark only requires motif preservation and a high pLDDT of the folded structure. I don't see how low scRMSD equates to a test of 'function' for MotifBench. In my mind the main difference is that GeomMotif doesn't require a starting structure so can compare a wider array of models. I  think this should be made clearer.\n\nWith regards to comparing structure based and sequence based models, there seems to be quite an unfair advantage given to structure based models because they are allowed 8 MPNN sequences to attempt to find a good sequence that meets the success criteria. However, sequence based models only have 1 sequence to try to find success. It would therefore seem more appropriate to compare 100 structure samples with 8 MPNN sequences each with 800 sequence samples from the sequence generative model i.e. they have the same number of allowable calls to ESMFold to find a success.\n\nI also believe this benchmark missed an opportunity to reduce the noise in the evaluation criteria. The authors use purely computational filters to finally select 57 motif scaffolding tasks from the entire PDB. This seems like quite a small number of tasks given that the hand selected MotifBench benchmark has 30 problems. Computationally selecting tasks opens the possibility of selecting a very large number of tasks. I believe a more comprehensive and lower noise evalauation of different methods could have been achieved with a much large number of motif scaffolding tasks but with fewer attempts at each task. For example, you currently have 100 attempts at each task for a total of 5700 designs. You could have instead picked 5700 motif scaffolding tasks with 1 attempt each which would have covered a much broader array of problems.\n\nThis problem is exemplified in Table 2 where the authors attempt to interpret drops in performance at certain numbers of fragments in the motif with some property of a model e.g. stating that Genie2 has a drop in performance at 4-5 fragments but good performance at 1-3 and 6-7 fragments. They then conclude that 'intermediate' complexity is difficult for the model. I personally am unconvinced by this explanation as I see no fundamental reason as to why intermediate numbers of segments should be harder than a large number of segments. I would offer an alternative explanation that simply the 5 tasks with 5 fragments happened to be harder than the other tasks because there is such a small number of tasks per number of fragments bin and the random sample of tasks that had 5 fragments happened to be hard. I think this is to be expected when the total number of tasks is small. If the authors wish to make general conclusions about task difficulty versus number of fragments I think many more tasks should be included in the benchmark.\n\nIn terms of writing, I think the clarity of section 4.4 could be improved with a figure since it is hard to make conclusions as a reader just reading stated numbers in the main text.\n\nOverall, I believe the analysis of different models is of interest to the community more so than the benchmark itself due to the aforementioned issues. I am leaning towards a weak accept.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mtjR8en4uU", "reviewer": "Reviewer_aTcU", "summary": "This paper introduces **GeomMotif**, a systematic benchmark designed to address a gap in the evaluation of protein motif scaffolding models: the lack of focus on general **geometric preservation**, as opposed to just functional motif preservation.\n\n**Key Contributions:**\n\n*   **A Novel Benchmark:** GeomMotif consists of 57 tasks derived from the PDB, each with a guaranteed ground-truth solution, designed to test the preservation of arbitrary structural fragments.\n*   **Detailed Task Characterization:** Tasks are annotated with comprehensive structural and physicochemical properties, allowing for a fine-grained analysis of model strengths and weaknesses.\n*   **Multi-faceted Evaluation:** The benchmark uses metrics for geometric fidelity (scRMSD, pLDDT) and structural diversity (clustering) to provide a holistic view of model performance.\n\nThe results show that different classes of models struggle with different types of geometric challenges, highlighting the utility of this benchmark for guiding future development in protein generative modeling.", "questions": "1.  **On the Solvability Criterion and Benchmark Scale:** Regarding the construction of the benchmark, the paper employs a strict criterion for solvability (ESMFold prediction RMSD ≤ 1.0 Å to the PDB structure), which yields a final set of 107 structures. Could the authors comment on the rationale for this strictness? While it guarantees high-quality, predictable scaffolds, it also significantly constrains the diversity and scale of the benchmark. Have the authors considered an alternative definition of solvability more local to the motif itself—for instance, requiring only that the *motif region* is accurately predicted by a structure prediction model? This might allow for a much larger and more diverse set of benchmark tasks while still ensuring that the core scaffolding problem is well-posed.\n\n2.  **Incorporating Functionally Relevant Motifs:** The paper makes a strong case for a benchmark focused purely on *geometric* preservation, distinct from function-focused benchmarks. As a complementary direction, have the authors considered creating a subset of tasks within GeomMotif where the geometric fragments are specifically chosen from known functional sites (e.g., catalytic residues, binding interfaces, or other hotspots)? This could help bridge the gap between geometric and functional challenges, providing valuable insights into whether current models are better or worse at preserving fragments that are known to be functionally important, even when evaluated on purely geometric metrics.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  **Addresses a Critical Need in the Field:** The paper correctly identifies a significant gap in protein design: the lack of a systematic benchmark for the fundamental task of motif scaffolding. As most functional protein design now relies heavily on preserving specific motifs, the creation of a standardized benchmark to rigorously compare the performance of different scaffolding methods is both timely and important for the community.\n\n2.  **Well-Designed and Comprehensive Evaluation Protocol:** The evaluation protocol is a key strength of this work. It is both reasonable and thorough. The introduction of the **SUN (Successful, Unique, Novel)** metric is particularly noteworthy, as it provides a single, well-motivated score that elegantly integrates the three most critical aspects of generative design performance, moving beyond simplistic success rates.\n\n3.  **Thorough and Diverse Benchmark Construction:** The construction of the benchmark itself is exemplary. The selection and analysis of the motifs are highly detailed and thoughtfully curated to cover a wide and representative range of application scenarios. This comprehensiveness ensures that the benchmark can effectively probe the strengths and weaknesses of different models across various structural and physicochemical contexts.", "weaknesses": "### Weaknesses:\n\n1.  **Exclusion of Small Motifs Limits Scope and Practical Relevance:** The benchmark's decision to exclude motifs with fewer than 30 residues is a notable limitation. In many real-world design scenarios, particularly for active sites or epitope engineering, the critical functional motifs are often very small, sometimes comprising just a few key residues. These small, and often non-contiguous, fragments are notoriously difficult for current models to constrain geometrically. Including a dedicated set of small-motif tasks would not only better reflect practical challenges but also provide a more stringent test of model capabilities, significantly enhancing the benchmark's overall utility.\n\n2.  **Evaluation Could Be Extended to More Recent Methods and Tasks:** While the paper provides a solid comparison of established models, the field of protein generation is advancing rapidly. The benchmark would be more impactful and have greater immediate relevance if it included an evaluation of more recent state-of-the-art models, such as Protpardelle-1c, RFdiffusion2, or Proteina. Additionally, exploring performance on more nuanced motif-scaffolding tasks, like unindex motif-scaffolding, would provide a more comprehensive picture of model capabilities.\n\n3.  **Lack of Focus on Side-Chain Preservation:** The current evaluation protocol is centered on backbone geometry (Cα atoms). However, for most functional applications of motif scaffolding (e.g., designing binders or enzymes), preserving the precise 3D conformation of the motif's *side-chains* is as crucial as preserving the backbone. Many of these functionally critical motifs are the very small ones mentioned in the first point. While the evaluated models are primarily backbone-focused, the benchmark itself would be more forward-looking and comprehensive if it included metrics for side-chain fidelity. Reporting on heavy-atom RMSD for the motif region, for instance, would better assess a model's ability to preserve function and encourage the development of all-atom scaffolding methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PPed0LVEQL", "reviewer": "Reviewer_oyxS", "summary": "The authors introduce GeomMotif, a new benchmark to better evaluate the performance of current motif-scaffolding techniques in de-novo protein design. The benchmark consists of 57 modality-agnostic scaffolding tasks. To ensure systematic coverage of protein structural space, the authors uniformly sample proteins from the PDB containing varying numbers of motifs, ranging from single-motifs to multi-fragment ones with up to 7 fragments. Each motif and fragment is annotated with physicochemical metadata as well. \nFor each task, the authors generate scaffolds using multiple methods, demonstrating that the motifs or fragments are indeed \"scaffoldable\" and hence the tasks solvable. Interestingly, when comparing structure- and sequence-based models, the authors find that structure models outperform sequence-based ones. Sequence-based models struggle particularly with multi-fragment, non-local motifs. They also observe that the secondary structure and other parameters of motifs strongly influences success. Helices are easier to scaffold than β-sheets, and that burial and contact ratio are important determinants of performance. The GeomMotif benchmark is publicly available on github and hugging face for use by the broader research community.", "questions": "I have two questions regarding the benchmark creation:\n\n1. Many motifs include loops or parts of loops. Why did the authors remove these when creating the benchmark instead of including and classifying them as well? This aspect would be important for multiple applications; antibodies being the most obvious example.\n2. Does the benchmark include motifs derived from actual protein–protein interactions (PPIs) or enzymes? If so, for the PPI-derived motifs are the corresponding target proteins also included? It would be interesting to see scaffolding evaluated in the presence of the target protein, as this introduces additional constraints on how the scaffold must be generated.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "The manuscript is clearly written, and the creation pipeline of GeomMotif is well explained. The benchmark evaluation incl. its feasibility and the comparison between structure-based and sequence-only models reveals several interesting trends, highlighting both challenges and opportunities for these approaches. As expected, structure-based models outperform sequence-only models on this structurally oriented task. Moreover, motifs containing helices are easier to scaffold, a finding consistent with previous observations.", "weaknesses": "No specific weaknesses in the manuscript.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)