ProtComposer: Compositional Protein Structure Generation with 3D Ellipsoids
Authors: Hannes Stark
Bowen Jing
Tomas Geffner
Jason Yim
Tommi Jaakkola
Arash Vahdat
Karsten Kreis
Venue: ICLR 2025 Oral (2025)
Keywords: protein design
diffusion model
controllable generation
drug discovery
proteins
biology
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We develop a framework to generate protein structures conditioned on spatial protein layouts that are specified via a set of 3D ellipsoids.

Abstract: We develop ProtComposer to generate protein structures conditioned on spatial protein layouts that are specified via a set of 3D ellipsoids capturing substructure shapes and semantics. At inference time, we condition on ellipsoids that are hand-constructed, extracted from existing proteins, or from a statistical model, with each option unlocking new capabilities. Hand-specifying ellipsoids enables users to control the location, size, orientation, secondary structure, and approximate shape of protein substructures. Conditioning on ellipsoids of existing proteins enables redesigning their substructure's connectivity or editing substructure properties. By conditioning on novel and diverse ellipsoid layouts from a simple statistical model, we improve protein generation with expanded Pareto frontiers between designability, novelty, and diversity. Further, this enables sampling designable proteins with a helix-fraction that matches PDB proteins, unlike existing generative models that commonly oversample conceptually simple helix bundles. Code is available at https://github.com/NVlabs/protcomposer.

Published as a conference paper at ICLR 2025
PROTCOMPOSER: COMPOSITIONAL PROTEIN STRUCTURE GENERATION WITH 3D ELLIPSOIDS
Hannes Stark*.1,2 Bowen J Tomas Geffner', Jason Yim?, Tommi Jaakkola",
Arash Vahdat & Karsten Kreis"
IVVIDIA PCSAIL, Massachusetts Institute of Technology
ABSTRACT
We develop ProtComposer to generate protein structures conditioned on spatial protein layouts that are specified via set of 3D ellipsoids capturing substruc- ture shapes and semantics. At inference time, we condition on ellipsoids that are hand-constructed, extracted from existing proteins, or from statistical model, with each option unlocking new capabilities. Hand-specifying ellipsoids enables users to control the location, size, orientation, secondary structure, and approxi- mate shape of protein substructures. Conditioning on ellipsoids of existing pro- teins enables redesigning their substructure's 's connectivity or editing substructure properties. By conditioning on novel and diverse ellipsoid layouts from simple statistical model, we improve protein generation with expanded Pareto frontiers between designability, novelty, and diversity. Further, this enables sampling des- ignable proteins with a helix-fraction that matches PDB proteins, unlike existing generative models that commonly oversample conceptually simple helix bundles. Code is available at ht //gitttthh//// com/NViabs prot composer.
INTRODUCTION
Proteins are intricate macromolecular machines that carry out wide variety of biological and chem- ical processes. A grand vision of rational protein design is to be able to design complex and modular functions akin to those found in nature, where different spatial parts of the protein possess different properties that act in a coordinated fashion (Chu et al., 2024; Kortemme, 2024). However, current paradigms of ML-based protein structure generation are largely limited to unconditional generation (Watson al., 2023; Yim et al., 2023b), to inpainting of scaffolds and binders conditioned on known parts of the structure (Watson et al., 2023; Trippe et al., 2022), with no ability to control the higher-level spatial placement or layout of the generated protein. This leads to limited diversity and control of the generated samples and distinguishes protein generation from image generation, where such levels of control are commonplace and lead to new capabilities in the hands of human users (Li et al., 2023a; Rombach et al., 2022; Nie et al., 2024; Zheng et al., 2023a; Zhang et al., 2023a).
To bridge this gap in the protein design toolbox, we develop ProtComposer as means of controlling protein structure generative models with the protein's layout in 3D space. Specifically, we describe modular protein layouts via 3D ellipsoids augmented with annotations to provide rough "sketch" of the protein (Figure 1). Similar to blob or bounding box representations in image generation (Nie et al., 2024; Li et al., 2023a), these ellipsoids provide a level of abstraction intermediate between data-level (i.e., voxels) constraints and global conditioning. They are informative enough to control the generation of diverse proteins, but are human-interpretable: easy-to-construct, and do not con- strain the low-level details of protein structures. Hence, 3D ellipsoids facilitate a two-stage paradigm in which complex protein designs are expressed as spatial sketches by hand or via heuristic algo- rithms, and deep learning models "fill in" these sketches with high-quality, designable backbones.
In this work, we apply our philosophy to controlling Multiflow (Campbell et al., 2024), a joint sequence-structure flow-matching model with state-of-the-art designability, with 3D ellipsoid lay- outs annotated with secondary structure. Multiflow represents protein structures as a cloud of residue frames in (3) and parameterizes the flow network with Invariant Point Attention (Jumper
"Equal contribution. Work done during intersships NVIDIA. fhatark, bjing]ami edu.

---

Published as a conference paper at ICLR 2025
Sheet ellipsoid Helix ellipsoid
Novel and diverse ellipsoids from simple statistical model
From hand- specified layouts
Transformer
Protein novelty T diversity
ICA
Figure Left: ProtComposer's flow model denoises a noisy protein structure conditioned on spatial protein layout of annotated ellipsoids that we inject using our and ellipsoid tokens. Middle: At inference time, we condition our model on ellipsoids from simple statistical model (Sec. 3.4) that guarantees novel and diverse layouts. The generated proteins exhibit high novelty and diversity while maintaining designability. Right: Conditioning on hand-specified ellipsoids allows for controllable generation.
Attention (ICA)
et al., 2021; Yim et al., 2023b). To inject ellipsoid conditioning into this network, we develop an cquivariant mechanism for message passing between cllipsoids 3D space and residue frames, which we call Invariant Cross Attention. We then fine-tune Multiflow with this cross attention into a conditional model of protein structure and sequence, conditioned on ellipsoid layouts. We develop classifier-fres guidance mechanism with Multiflow, enabling interpolation between ellipsoid- conditioned and unconditional generation. Empirically, we find this family of conditional generative distributions advances the state of the art in protein generation along three axes:
Control-unlin existing models, we can prompt our method on (existing or novel) ar- rangements of secondary structure ellipsoids. We develop a family of metrics to measure adherence to ellipsoid conditioning and find strong consistency between conditioning ellip- soid layouts and generated backbones. This consistency persists well beyond the training distribution, with some particularly impressive generations shown in Figure 7.
Diversity and Novelty-by conditioning on synthetic ellipsoid layouts drawn from a fam- ily of simple statistical models (Section 3,4), we significantly increase the diversity and novelty of generations. Although there is cost to the designability of the generated pro- teins, our Pareto frontier along this tradeoff far surpasses that of adjusting inference-time parameters, the only existing option for controlling the diversity of Multiflow generations.
Compositionality-aiie although highly designable, protein generations often exhibit a low degree of architectural complexity (e.g., only composed of single alpha-helix bundle). We argue that such proteins are analogous to pathological language model outputs with high model likelihood but low information content (i.e., the sentence "and and and has high likelihood under many large language models.). We introduce a compoxitionality metric quantify this phenomenon and show that ellipsoid conditioning can improve the complexity and compositionality of generated structures.
2 BACKGROUND AND RELATED WORK
Protein Structure Generation. The primary aim of protein structure generative models (Yim et al., 2023b; Ingraham al., 2023; Bose al., 2024; Campbell et al., 2024) aiding computational design of novel proteins (Watson et al., 2023; Lauko et al., 2024). Thus, we often desire to gener- ate beyond already existing folds or secondary structure compositions, and to control generations to satisfy design specifications. To address controllability. there are two main explored avenues beyond scaffolding existing structural motifs (Yim et al., 2024a). First, conditioning on block con- tact maps and sequential secondary structure specifications (Anand & Achim, 2022; Watson et al., 2023). Second, inference time conditioning, as in Chroma (Ingraham et al., 2023), via projections onto manifold via forees from arbitrary differentiable energy function. Such inference time control enjoys high generality while ProtComposer trained for a single type of shape and semantic conditioning, but, therefore, enjoys improved adherence the control (Table 1).
2

---

Published as a conference paper at ICLR 2025
Spatial Conditioning. For image generative models, controllable generation has enabled new ap- plications that make up significant portion of their utility beyond generating impressive images. To unlock similar new capabilities for protein generation, we follow the transferrable concepts that have crystallized out as crucial. On a technical and architectural level, this includes fine-tuning strong ex- isting generative models with the principle of minimally perturbing the original model's output in the initialized conditional model (Li et al., 2023a; Zhang et al., 2023b). On conceptual level, this implies finding the right input specification: for different tasks, different levels of granularity are appropriate. In the image domain, this ranges from pixel-level specifications such as semantic seg- mentation maps or sketches to more coarse-graine specifications such as bounding-bowd boxes (Li al., 2023b; Zhang et al., 2023b), or "blobs" (Nie et al., 2024), akin to our aims for proteins.
Flow models. Flow matching (Liu et al., 2022; Lipman et al., 2022; Albergo 2022: Albergo et al., 2023) aims to learn a time-dependent vector field Vo.t that, when integrated from start time t
Vanden-Fijnde
to = 1, transports samples from a noise distribution Xg ~ Po to data distribution X1 P1. To train ta,r, we sample partially noised data from a conditional probability path (x Xv, x1) satisfying Pu(x Xb, X1) R ō(x - Xo) and pr (x(x Xo: X1) Rs ō(x - x1). A common choice is Dirac that traces out straight line between Xy and X1
geodesic for flow matching on manifolds (Chen & Lipman, 2024). At the sampled noisy datapoints Xt. we evaluate the vector field (x,)(x,) and regress it against the conditional vector field u, (x; X1, that corresponds to the conditional probability path through the continuity equation - Si = -
field
(Prwe). At convergence, UØ approximates the marginal vector field ue (x) (since the gradients are equivalent to regressing against u (x)) that evolves the prior pu to the data distribution P1 through the marginal probability path (x) J (x Xo. X1 X1)Po (Xo) p1 (x1 ]dxgdx1.
3 METHOD
3.1 ELLIPSOID REPRESENTATION OF PROTEINS
Proteins are compositional objects different regions have different properties, and we seek a lan- guage in which to succinetly describe this information to control the sampling of a generative model. To do so, we propose to represent a protein's 's spatial layout using a set of K ellipsoids, each cor- responding to a semantically coherent region of the protein. Each ellipsoid records the number of residues in the associated region, a categorical semantic feature, its position, and its shape in terms of the covariance matrix of the Ca coordinates in the region. We argue that this representation of protein spatial layouts finds a favorable tradeoff between a single global annotation, such as a text prompt or protein family, and more complex shape descriptors, such meshes voxel grids. A global annotation may be insufficient to provide the desired control over the spatial layout, and a more complex annotation could be difficult generate or specify without training an additional model. Mcanwhile, 3D ellipsoids are expressive and precise, yet easy to generate and manipulate.
Mathematically, we define a protein spatial layout consisting of K ellipsoids as an unordered set E = {E = (pR. Li. fu, me)hee (1...K} where each ellipsoid is represented as a Gaussian with mean μ4 E R covariance Σ, E R3333, count n& E N+, and feature annotation fa E X' where X' the application-dependent feature space. Viewed as Gaussian probability distributions, our ellipsoids do not have well-defined boundaries; however, for visualization and evaluation purposes, we define the ellipsoid boundary to be the surface at Mahalanobis distance V5, i...
DE = x x
R3 V
(1= (x - -px)= = v3
(1)
This is the functional form of a conventional ellipsoid. The distance V5 is chosen so that 83% of the density falls inside the surface, which yields the best visual results (Appendix Fig 12).
Ellipsoid Segmention. Provided protein structure, obtaining its ellipsoid representation (e.g., for training purposes) consists of two steps: segmentation of the protein into semantically coher- ent regions, and extraction of ellipsoid descriptions (Mu, Σ,,7,, 7l,, fx) for each region. To segment the protein, we consider a simple, non-learned segmentation algorithm that places two residues in the same region if and only if they are both spatially proximal and semantically similar. We construct a segmentation graph by drawing an edge for each such pair of residues and return the list of connected components of this segmentation graph. For each segmented region, we aggre-
3

---

Published as a conference paper at ICLR 2025
gate the residue features to obtain fk and compute the mean and covariance of the Ca positions. These steps are illustrated in Figure 2 and further detailed in Appendix Algorithm 3. We found this to be more reliable than more sophisticated variants using, e.g., K -means or spectral clustering.
Mn
In this work, we focus 3D ellipsoids speci- fying secondary structure layouts, i.e., regions of a-helices and B-sheets. Our feature space is thus two-class space of secondary structures types, f& E X = {a. .9). We featurize residues using DSSP (Kabsch & Sander, 1983) and draw edges in the segmentation graph between amino acids with the same secondary structure label and within 5 Á. The ellipsoid annotation fk then simply inherits the label of its constituent residues. We exclude all loop residues and el- lipsoids with five or fewer residues.
w
1.
2.
3.
Figure 2: Extracting ellipsoids from data. 1) In- put protein. 2) Annotate residues and draw edges if they have the same feature AND are within 5A. 3) Fit Gaussians to connected components.
3.2 ELLIPSOID CONDITIONING
Unconditional model. We inject ellipsoid conditioning into an existing protein structure genera- tive model for which we choose Multiflow (Campbell et al., 2024), which jointly generates sequence and structure, Following their framework, we generate proteins represented as an array of frames T E SE(3) where each residue's s frame T; = (R,, t.) E SE(3) has an associated translation t, R and rotation matrix R₃ I3a3 constructed from backbone coordinates following Al- phaFold2 (Jumper et al., 2021). Additionally, each residue has an amino acid type a, E {1...20),
To jointly generate the translations, rotations, and amino acids, Multiflow employs three types of flow matching procedures that iteratively update all three modalities. Translations are handled with linear flow matching from a Gaussian prior (Lipman et al., 2022), rotations with Riemannian flow matching on SO(3) (Chen & Lipman, 2024), and residue types with discrete flow matching (Camp- bell et al., 2024; Gat et al., 2024), resulting in a joint flow that transports from a prior pott, R, a) to the data distribution p1 (t, R, a) while tracing out probability path Pt. R, a) where / E [o. 1].
The flow is parameterized by single backbone architecture with translations, rotations and residue type inputs from which it predicts time dependent translation vector field (t), rotation vector field vo,, ()).( and rate matrix (a) dictating residue type updates. The architecture composed of several identical update blocks, each of which updates d-dimensional residue representations 8, E Rdd for i E (1....N] N). residue pair representations Zi, E Rdd for i, E {1...NNT .) and residue frames T, for E [1, N). The updates are (3(3)-equivvrian and accomplished with a mixture of shallow transformers and Invariant Point Attention (Jumper et al., 2021); we refer to Campbell al. (2024) for complete architectural details. After all the update blocks, the final residue tokens and frames T; are used to parameterize the flow fields (t), o Ro,((a)
Injecting Ellipsoid Conditioning, We now aim to fine-tune pre-trained unconditional Multiflow model trained to sample Pi (t, R, a) toward sampling an ellipsoid conditioned density Pi (t, R, a E) and obtain ProtComposer. At inference time, ellipsoids can be specified manually or sampled from a second distribution pEE) of novel and diverse ellipsoids (see Section 3.4) target the density PI (t, R, a EE). For fine-tuning, we only provide the conditioning information as additional input the training loss remains the unchanged loss of Multiflow. To inject the ellipsoid informa- tion, we follow best principles from semantic map conditioning for image diffusion models (Zhang et al., 2023b; Li et al., 2023b) and design architecture modifications that minimally perturb the un- conditional model at the time of initialization. That is, with an empty set of ellipsoids as input, the untrained conditional model should produce identical outputs as the unconditional model. This is accomplished by preserving the initial residue representations s, Zij, T,, and only supplying addi- tional information from 3D ellipsoids to inform their updates, described below.
We introduce additional tokens e₁ Rd for each ellipsoid k {1...K} K} that of the same dimensionality d as the residue tokens s;. These tokens are initialized with embeddings of all (F)3 quantities of ellipsoids- their size ng. squared radius of gyration tr Σ, and sec-
4

---

Published as a conference paper at ICLR 2025
Algorithm 1: Invariant Cross Attention
Algorithm 2:
Update Block
Input: Residue tokens s; and frames
Input:
Residue tokens 8;, pair reps 2ij. residue frames T ellipsoid tokens Cg. ellipsoid
T. (R,, t.); ellipsoid
parameters Eg- (Ux, 2x}
parameters E2 (Mk, 2xx)
Fik 4 T:- pig
8 +
InvariantPoom -Attetionn, z, T)
Ci - R
+
Invarian(CrosssAttnntonn T, E)
a.k 8; + Linear Sinear(PosFanbeddtaa)
S +- Concat c)
a + Linear(Flatten(Caa)
+ Transformer(s)
q: Linear(s.)
8, e < (plit((s)
kik Vik LLinear a.s))
T RigidUpdate(s, T)
S; Attontion, ( qr, kik. Vak)
Z + EdgeU (date(()
ondary structure type ft. Then, in each model layer, these tokens inform the updates of the residue representations S,, Zij. T; (and are themselves updated) via two mechanisms:
To update the residue tokens s; with information about the location and shapes of the ellip- soids, we introduce a novel invariant cross attention mechanism (Algorithm 1 and Figure 1) whereby values are aggregated from the ellipsoid tokens in an Similar IPA, this mechanism uses the residue local frames enforce invariance, although the ellipsoid tokens are not themselves updated, which we discuss further in Appendix B.
manner.
To provide a mechanism for residue and ellipsoid tokens to mutually update each other, we concatenate the tokens along the sequence dimension right before the Transformer stack, and re-split the sequence afterwards.
All other aspects of the Multiflow update blocks, such as the frame update and edge update lay- ers, remain architecturally unmodified. In Algorithms 1 and 2, we outline the new invariant cross attention alongside the modified update block with modifications colored green.
3.3 GUIDANCE FOR THE SELDICONDD AND JOINT FLOW
After fine-tuning a base protein structure generative model that samples po(t, a) 2 pi (t, R. to obtain ProtComposer's 's distribution pe (t, R, a E), we interpolate between the two distributions via classifier-free guidance (Ho & Salimans. 2022) controlled by guidance parameter À 2 0. This enables finding the optimal À to trade off between the designability of Po R. a) that is recovered with x= and the diversity, novelty, and ellipsoid adherence of (t, R, a E) corresponding to A 1. Interpolations for individual samples are visualized in Figure 3.
Implementing such guidance is complicated by the facts that we model the flow field instead of the score (as in diffusion models), that ProtComposer's conditional probability paths are not Gaussian (the guided flows of (Ho Salimans, 2022) are not directly applicable), and since we employ self- conditioning. Before elaborating on the self-conditioning difficulty, we lay out how guide the joint flow over translations, rotations, and discrete residue types by separately interpolating their flow fields at each inference step:
Translations: we interpolate the unconditional vector field u (t) and the conditioned version (t,,, (t, as X (t, E) + (1 (t)) Since the conditional probability paths for translations are Gaussian paths, this corresponds to the guided flows of Zheng et al. (2023b), which sample the same approximation of the unconditional distribution tilted by the conditional distribution guided diffusion models, i.c., an approximation of (t))((-))p(tt E)* if we were to interpolate models that only sample translations.
Rotations:
while with our non-Gaussian paths on SO(3) the results of Zheng et al. (2023b) longer hold, we find empirical success using, in analogy translations, Aug (t, E)+ +
(1 (1- --x)) (t), which follows Yim al. (2024b).
Discrete Flow: we construct the rate matrix for the discrete flow as the expectation of the conditional rate matrix (see Campbell et al. (2024)) over predicted probabilities of the denoised residues that we obtain as a combination the unconditional model's predic- tions and the ellipsoid conditioned model's predictions. Specifically, we use the uncon- ditionally predicted probabilities tilted by the ellipsoid conditioned probabilities po (a.") a ((0))11A)po(aa aa E)A, where the superscript denotes denoising time.
(1)
5

---

Published as a conference paper at ICLR 2025
lambda: Cov: UL: -5.43 Soft Acc: 0.59
lambda: Cov: Acc:
lambda-02 0.5 Acc: LL: -4.16 Soft Acc: 0.79
lambda:
lambda:
Cov: Acc:
.4.76 Soft Acc: 0.52
LL: -3.43 Soft Acc: 0.91
UL: -3.34 Soft Acc: 0.93
lambda: 0.0
lambda: Cov:
lambdai 0.5 Cov: 0.4
lambda: 0.49 0.95
lambda:
0.16 -8.47 Soft Acc:
0.63 2.29 0.96
-6.08
LL: 5.09 0.72
Two protein layouts (top and bottom) and generations for them with varying guidance strength; no guidance (A = 0) on the left, full guidance (A = 1) on the right. Ellipsoid alignment metrics are labeled as Cov=coverage, Acc=accuracy, LL=likelihood, and Soft Acc=soft accuracy.
Figure 3:
Both Multiflow and ProtComposer use self-conditioning (Chen et al., 2023), in which, during in- ference, the flow-model receives the output of the previous integration step as additional self- conditioning input. During inference, the unconditional model po (t, Ri, a) produces the self- conditioning variable X, and from the ellipsoid conditioned model po R. a E), we obtain Xg. Instead of supplying X to the unconditional and XE to the conditioned model, we use AXE + (1 - X) for both, which achieves better designability and ellipsoid adherence for all X. An exploration and ablation of self-conditioning variants is in Appendix C.2.
3.4 GENERATING NOVEL ELLIPSOIDS
In designing the ellipsoid conditioning mechanism, we have far made no assumptions about the sources of ellipsoids provided during inference. Next to using manually specified ellipsoids, there also an opportunity in sampling synthetic ellipsoids from additional generative model pa (EE) to sample an unconditional distribution of protein structures factorized as po R, a E)ps (E).
While it may be tempting use deep learning solution for pe (E). we purposefully avoid this and argue that the factorization is best leveraged with a simple statistical model over ellipsoid layouts. Instead of deep learned pe (E) that may produce layouts that are similar to the training data, simple statistical model for Po (E) guarantees sampling diverse and novel layouts, which lead to more diverse and novel protein structures from po R, a E)po (E) properties that are crucial for protein design where the aim is commonly to produce novel designs.
To generate novel ellipsoid layouts, we first sample means and covariances for ellipsoids and then assign secondary structure and residue count annotations. The model over means and covariances is
Is. vs) exp (-U(1(pk exp(-U(((t,EE)) Eo) (2)
p (((H,,)))E
II N (px, 0, 01 A=1
Ul(ln,, Ex)
1
Anglt ((x=-1y)* (uu py)]
That is, the ellipsoid means and covariances are drawn i.i.d. from isotropic Gaussian and Wishart distributions, respectively, and multiplied with the Boltzmann factor of an energy function that pe- nalizes ellipsoid overlaps. Intuitively, controls the ellipsoid's spread, i controls their volume, controls their anisotropy or "roundness" and prevents overlaps. The energy U is a simple inverse square repulsion based pairwise Mahalanobis distances. We sample this via rejection sampling. i.c., by sampling pæ D, evaluating their energy U, and rejecting with probability e -
To choose the ellipsoid annotations, we first independently annotate each ellipsoid as a with proba- bility 7 and 3 with probability 1 1-7. We then observe that for a given choice of {a, 3}, the ellipsoid volume / det E, strongly determines the residue count by a simple linear fit (Appendix 10). Hence, use this linear fit to assign the number of residues instead of modeling it independently.

---

Published as a conference paper at ICLR 2025
Table 1: Recapitulating layouts PDB proteins. Protein layout specification adherence metrics for ProtComposer conditioned on layouts of validation set proteins under different levels of guid- ance strength X. Multiflow as random baseline. The Oracle metrics are computed by treating the validation set proteins, which the layouts were drawn from, as generated proteins.
Geometric
Probabilistic
Model
Coverage
Misplacement Accuracy
Likelihood t Soft Accuracy t JSD +
Multiflow
0.49
0.40
0.59
-5.0
0.61
0.43
Chroma
0.59
0.36
0,58
-4.2
0,58
0.48
ProtComposer A=0.2
0.55
0.33
0.66
-4.5
0.65
0.36
ProtComposer A=0.4
0.62
0.28
0.75
-4.0
0.73
0.28
ProtComposer A=0.6
0.70
0.21
0.85
-3.5
0.83
0.20
ProtComposer A=0.8
0.75
0.18
0,90
-3.3
0.88
0.15
ProtComposer A=1.0
0.78
0.17
0.90
-3.2
0.90
0.13
ProtComposer A=1.6
0.86
0.13
0.93
-3.0
0.91
0.13
ProtComposer ==2.0
0.79
0.14
0.92
-3.2
0.92
0.13
Oracle
0.89
0.07
0.92
-2.8
0.93
0.15
4 EXPERIMENTS
Starting from the publicly available pre-trained checkpoint, we fine-tune Multiflow (Campbell al., 2024) on the dataset and splits supplied by the authors, where ellipsoid spatial layouts are obtained for each protein as described in Section 3.1. We train only the joint unconditional modeling task (i.e., no motif scaffolding. inverse folding, or forward folding). At inference time, we employ self- conditioning, rotational annealing, and 500 inference steps as described in Campbell et al. (2024). For guidance, we use the pretrained Multiflow checkpoint as the unconditional model.
Throughout experiments, consider three sources of ellipsoid layouts: PDB proteins from the Multiflow validation set (data ellipsoids), ellipsoids drawn from our statistical model (Section 3.4; synthetic ellipsoids). and manually specified ellipsoids. The key feature data ellipsoids is that they are associated with ground-truth proteins, providing an oracle generator for ellipsoid adherence. When using data ellipsoids, we sample proteins of equal lengths the ground-truth proteins, while for novel ellipsoids, the protein length is the sum of ellipsoid residue counts, Σx ng.. Summary statistics about both sources of ellipsoids are described in Appendix A.
4.1 ELLIPSOID CONSISTENCY
Given a 3D ellipsoid layout and protein generated based on the layout, seek quantify the degree consistency and alignment between the protein and the layout. We define two classes of metries: three geometric metrics, in which we interpret ellipsoids
ellipsoids with a definite interior and exterior based on Mahalanobis distance (Eq. 1), and three probabilistic metrics, in which we only use the Gaussian parameters associated with the ellipsoid. Layouts are provided in a fixed orientation, and the model equivariantly generates proteins in the same orientation, no noto-translational alignment is performed for any of these metrics:
Coverage t-ehe fraction of structured residues located inside least one ellipsoid.
Misplacement 4-the the sum of errors Σσ Ips pkx between Pk: = n4/ Σn Tig,, the residue fraction of ellipsoid k, and r/g- the actual fraction of structured residues inside ellipsoid k
Accuracy thehe fraction of residues located inside at least one ellipsoid with the same sec- ondary structure type the ellipsoid annotation (residues can be counted multiple times).
Likelihood t-we view the ellipsoid layout density is renormalized to integrate to Ex v! We then report the average log density of each structured residue position under this renormalized GMM.
Gaussian mixture model (GMM), whose
Soft Accuracy 1 given an ellipsoid layout viewed as GMM, distribution over secondary structure type f E {a, 3} for a residue located at arbitrary position in 3D space, given by p(J x) p(f,x) x Ekst nxN(x; p4, Ex). We
compute posterior

---

Published as a conference paper at ICLR 2025
then report the mean (over structured residues) of the normalized probability assigned to the actual secondary structure type of the generated residues.
Resegment JSD w-we we recompute the ellipsoid layout from the generated protein structure and view both ellipsoid layouts as defining Gaussian mixture models over the event space {a, (3 R We then report the lensen-Shannon divergence between the two distributions.
Note that Coverage, Misplacement, and Likelihood only quantify the alignment between the overall shape of the ellipsoid layout and the protein, whereas Accuracy, Soft Accuracy, and Resegment JSD also take into consideration the secondary structure annotations.
In Table 1, we generate proteins from layouts specified by ellipsoids drawn from the validation set of PDB proteins and we report the average for all metries. We sample for various levels of guidance À with ProtComposer, where À = 1 corresponds to the purely conditional model. We compare with Chroma (Ingraham et al., 2023) conditioned on ellipsoid layouts via its inference time ShapeCond i i oner functionality (details in Appendix A.2). For oracle and random baselines, we compute the cllipsoid alignment of the ground truth protein and of a protein generated from pre-trained Multiflow without any ellipsoid conditioning, respectively. We observe that ellipsoid alignment sharply increases when the guidance strength is increased above = 0.5 and quickly approaches the level of alignment of the oracle. Hence, 3D ellipsoids provide highly effective con- trol over protein layouts. Figure visualizes examples of protein generations and their associated alignment metrics for various guidance levels. More examples in Appendix Figure 24.
4.2 IMPROVED DIVERSITY AND NOVELTY
We now show that conditioning on ellipsoid layouts can improve the diversity (and related metries) of Multiflow generations. Without conditioning. Multiflow and related methods generate highly des- ignable proteins, but exhibit limited secondary structure diversity (73% helices instead of the 42% of PDB proteins) and low complexity by visual inspection (Figure 5). These effects can be ame- liorated by decreasing the rotational annealing, but this results in rapidly deteriorating designability (Figure 4). We thus construct a pipeline in which synthetic ellipsoids are drawn from our statistical model and provided as conditioning imput to ProtComposer. Conceptually, this be thought of as manually controlling the "ellipsoid marginal" of the generated distribution, where the ellipsoids amount to incomplete, compressed observations of the full state space (i.e., backbone structure).
To explore this pipeline systematically, we first fix K = 5 ellipsoids and 1 = 5 Á in our ellipsoid sta- tistical model, a setting that consistently produces proteins of length 120-200. We then sweep over all combinations of protein compactness a E [3, 4... 10). helix fraction 7 E [0.2, 0.4, 0.6, 0.8, 1] and ellipsoid anisotropy v E [5, 10, 20, 50, 100). and guidance strength E [0.1, 0.2....... 1.0). For each setting, we draw 100 synthetic ellipsoid layouts, generate a protein for each ellipsoid, and evaluate the set of 100 proteins on the following metries:
Designability t-ehe fraction of structures for which at least one out of 8 sequences sam- pled by ProteinMPNN (Dauparas et al., 2022) results in scRMSD < 2 when re-folded with ESMFold (Lin et al., 2023).
Diversity the Vendi score (Friedman & Dieng, 2023) of the set of structures with TM- Score (Zhang Skolnick, 2005) as the similarity kernel, ranging from to 100.
Novelty f-one minus the average TM-Score to the closest chain in the PDB as retrieved by FoldSeck (Van Kempen et al., 2024).
Helicity ththe fraction of structured residues which are assigned o-helix by DSSP. While helical proteins are not undesirable per se, we wish to increase the secondary structure diversity of Multiflow proteins be more similar to naturally observed helicity.
In Figure 4, we show the performance of all 1750 inference settings, with Pareto frontiers shown for the tradeoff between designability and each of the three other metrics. We compare with Mul- tiflow's frontier from varying its rotational inference annealing parameter (elaborated in Appendix A.4) which acts analogous to diffusion model low-temperature sampling to trade off diversity for fidelity (Yim et al., 2023a; Bose et al., 2024). Further, sweep Multiflow across protein length distributions that match the length distributions of our ellipsoid statistical model (details in Appendix A). For Chroma (Ingraham al., 2023) and RFDiffusion (Watson et al., 2023), sweep over the
8

---

Published as a conference paper at ICLR 2025
1.0
45
0.55
0.8
Diversity
40-
-Helicity
Novelty
0.50
0.6
85-
0.45
94
30-
25
0.0
0.4
0.6
0.6 0.8 1.0
0.2 0.6
0.9
Designability
Designability
Designability
Figure 4: ProtComposer Pareto frontiers traced out by our ellipsoid statistical model's family of distributions and by varying guidance strength. Multifow-vvarying rotational annealing strength. Chroma and RF Difffsionnvaariiin sampling temperature. Black dot o: metrics of PDB proteins. Each colored point (by guidance strength) one ProtComposer distribution. Shaded areas are stan- dard deviations from 4 seeds for points on the frontiers (App. A).
Multiflow
ProtComposer
Figure 5: Random samples with extracted ellipsoids segmentation threshold 6.5A from Multiflow (left) and synthetic ellipsoid pipeline with 10 Á. 100, 0.5 and À 0.6 (right). out of Multiflow proteins degenerate to a single a a-helix ellipsoid, while ProtComposer's proteins exhibit better secondary structure diversity and ellipsoid compositionality.
same length distributions and their sampling temperature. We robustly observe that ellipsoid con- ditioning enables the generation of more diverse, more novel proteins with helicity that closer to PDB proteins while retaining a higher level of designability than previously possible with Multi- flow. Appendix Figure visualizes examples of this expanded Pareto front, and Appendix Figure 23 shows scatterplots to examine the impacts of varying 1, n, and A.
Table 2: Recovering statistics of PDB proteins. The diversity, helicity, and com- positionality of PDB proteins compared with Multiflow and ProtComposer.
By increasing diversity and related metrics, ellipsoid conditioning also improves the aggregate similarity of generated proteins to proteins from the PDB. Ta- ble 2, quantitatively ascertain that proteins from pre-trained Multiflow without ellipsoid conditioning helical, less diverse, and less compositional than PDB proteins. For this, we quantify pro- tein's compesitionality by splitting into components of residues that are close to each other (6.5A as in Figure 5) and of equal secondary structure. Then, with as the count of residues in the k-tth compo- nent, we report the effective number of components exp(-Ex Pk logpx) where Ps: m&/ Σx m& can be interpreted as residue's 's probability for the k-th component. Table 2 shows that, by generating proteins based on data ellipsoids (from the Multiflow validation set), ProtComposer largely closes the gap to PDB proteins and restores the aggregate levels of helicity, diversity, and compositionality.
Model
Div.
Helíc.
Comp.
Multiflow
29
73%
1.9
Ours (A = 0.2)
32
72%
1.8
Ours (A 0.4)
36
68%
2.1
Ours () -0.6)
38
57%
2.6
Ours (A -0.8)
40
52%
2.8
Ours (A 1.0)
41
51%
2,9
Ours (A 1..)
45
48%
3.1
Ours (A 2.0)
48
47 %
3.2
PDB proteins
41
42%
4.1

---

Published as a conference paper at ICLR 2025
A
B
c
Figure 6: Controlled manipulation of ellipsoid layouts from PDB ID 8F9Y AA. (A) Original ellip- soid layout and generated protein. (B) Rotating the rightmost a-helix. (C) Translating the 3-sheets upwards. (D) Merging and expanding the B-sheet regions. (E) Secondary structure inversion.
Figure 7: Proteins generated from hand-constructed ellipsoids. Left: 4 protein-layouts with generated proteins. Right: 3 beta-barrels containing varying numbers of helices generated from large beta-sheet ellipsoid containing elongated helix ellipsoids (both top and side views shown).
4.3 FLEXIBLE CONDITIONING
To fully realize and demonstrate the potential of ProtComposer, explore manual construction or manipulation of cllipsoid layouts in various settings. First, in Figure 6 we demonstrate that by fixing the noise and changing the ellipsoid parameters, lation of generated proteins. For example, we can move individual secondary structure elements, enlarge merge them, convert between them. When applied PDB protein layouts, this capa- bility enables precise structural editing of existing proteins, an exciting and first-in-class capability. Next, in Figure 7, test the boundaries of the model's generalization ability by manually con- structing ellipsoids in eclectic layouts. These result in, for example, extremely long helix bundles, massive 3-barrels or 8-sheets, and peculiar arrangements of these artificial elements. Although these extreme structures are not always designable, they demonstrate the creative and faithful alignment of the fine-tuned model the provided ellipsoid conditioning.
impose fine-grained control and manipu-
CONCLUSION
We developed ProtComposer for conditioning protein structure generative models on 3D ellipsoid layouts. We implemented it for Multiflow and secondary structure annotated ellipsoids, using novel architectural components such as Invariant Cross Attention. We quantitatively ascertained that gen- erations tightly adhere to the input ellipsoid layouts and provided range of examples to demonstrate how this enables reliably generating proteins with desired layouts via hand-specified ellipsoids and editing existing proteins. To generate unconditionally, we condition on novel and diverse ellipsoid layouts drawn from newly developed statistical model. This produced family of protein structure distributions that far surpasses the Pareto frontiers of designability to novelty and diversity achieved by Multiflow. ProtComposer generates more compositional proteins with helicity close to that of PDB proteins while maintaining designability. We anticipate expanding the annotation types of our ellipsoids to function specifications to further push the frontier of controllable protein design.
10

---

Published as a conference paper at ICLR 2025
6
ACKNOWLEDGEMENTS
This work was primarily developed during Bowen Jing's and Hannes Stark's internships at NVIDIA. Tommi Jaakkola and Jason Yim acknowledge support and Hannes Stark and Bowen Jing knowledge partial support from the Machine Learning for Pharmaceutical Discovery and Synthesis (MLPDS) consortium, the DTRA Discovery of Medical Counterneasurre Against New and Emerg- ing (DOMANE) threats program, and the NSF Expeditions grant (award 1918839) Understanding the World Through Code.
We thank Gabriele Corso, Felix Faltings, Rachel Wu, and Weili Nie for helpful discussions and feedback.
REPRODUCIBILITY STATEMENT
Our implementations are based on the architecture, training losses, optimizers, datasets, and data processing of Multiflow (Campbell et al., 2024) for which the authors provide code at https: //github. com/ jasonkyuy im wwwummllllooooooooo All code for this paper is available at https: //github. com/NVian
composer, and we provide the following descriptions to en- able reproducing results based on the manuseript alone: We provide algorithm to specify our in- variant cross-attention mechanism, which can be implemented with invariant point attention classes https: : // thub. com aqla
openfold as a starting point. The modifica- tions of the Multiflow update block specified in Algorithm 2. Our ellipsoid extraction procedure is detailed in algorithm 3. All parameters for sweeps to produce the Pareto frontiers in Figure are specified in Section 4.2 and our Appendix Pareto frontier details A.4. We provide details how we run Multiflow as baseline in Appendix A.4. Details on running Chroma and how we implement ellipsoid conditioning for Chroma and resource requirements are in Appendix A. We describe the full form of our Ellipsoid statistical model and how sample it in Section 3.4. All our metrics are specified explicitly in Sections 4.2 and 4. with citations to tools required for their computation.
available in Appendix A.2. Training details such duration
8 ETHICS STATEMENT
We present a general method for protein structure generation that may be used to aid protein design. Proteins are versatile tools that carry biological functions. Depending on the purpose used der-denending on which protein design it is employed to aid-its use is either ethical or un- ethical. The vast majority of current use cases of similar technology are with the intent of achieving positive outcomes for humanity via, e.g., developing drugs, biomolecules that accelerate industrial processes, or proteins used as tools improve our understanding of biology itself. Thus, we esti- mate that the ethical use cases with potential for positive impact on humanity will far outweigh any negative impacts from unethical uses.
tool is
REFERENCES
Michael Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden, Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprint urXiv:2300.00877, 2023.
Michael Samuel Albergo and Eric Vinden- Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, 2022.
Namrata Anand and Tudor Achim. Protein structure and sequence generation with equivariant de- noising diffusion probabilistic models, 2022.
Avishek Jocy Bose, Tara Schound-Sadegle Guillaume Huguet, Kilian Fatras, Jarrid Rector-Brooks, Cheng-Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael Bronstein, and Alexander Tong. Se(3)-stochast) flow matching for protein backbone generation, 2024.
Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design, 2024.
11

---

Published as a conference paper at ICLR 2025
Ricky T. Q. Chen and Yaron Lipman. Flow matching general geometries, 2024.
Ting Chen, Ruixiang Zhang, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning, 2023.
Alexander E. Chu, Lucy Cheng, Gina EI Nesr, Minkai Xu, and Po-Ssu Huang. An all-atom protein generative model. bioRxiv, 2023. doi: 10. 10.100/202..05244443111
Alexander E Chu, Tianyu Lu, and Po-Ssu Huang. Sparks of function by de novo protein design.
Nature biotechnology, 42(():203-2)) 215, 2024.
Justas Dauparas, Ivan Aniscchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. Robust deep learning- based protein sequence design using proteinmpnn. Science, 378(6615):49956) 2022.
Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. Transactions on machine learning research, 2023.
Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, and Max Welling. te(3)-rransforme:::: 3d roto- translation equivariant attention networks, 2020.
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching, 2024.
Jonathan Ho and Tim Salimans. Classifier-free diffuuson guidance, 2022.
John Ingraham, Max Baranov, Zak Costello, Vincent Frappier, Ahmed Ismail, Shan Tie, Wujie Wang, Vincent Xue, Fritz Obermeyer, Andrew Beam, and Gevorg Grigoryan. Illuminating protein space with a programmable generative model. Nature, 623:1070-1078, 2023.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-58) 2021.
W. Kabsch and C. Sander. Dssp: Definition of secondary structure of proteins given a set of 3d coordinates. Biopolymers, 22:2577-2637, 1983.
Tanja Kortemme. De novo protein design-frooe new structures to programmable functions. Cell, 187(3):526-544, 2024.
Anna Lauko, Samuel J. Pellock, Ivan Anischanka, Kiera H. Sumida, David Juergens, Woody Ahern, Alex Shida, Andrew Hunt, Indrek Kalvet, Christoffer Norn, Ian R. Humphreys, Cooper Jamieson, Alex Kang, Evans Brackenbrough, Asim K. Bera, Banumathi Sankaran, K. N. Houk, and David Baker. Computational design of serine hydrolases. bioRxiv preprint, 2024.
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22511 22211,22521, 2023a.
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation, 2023b.
Yeqing Lin and Mohammed AlQuraishi. Generating novel, designable, and diverse protein struc- tures by equivariantly diffusing oriented residue clouds, 2023. URL httpt:ttt///c pttttps:/aaxxi..oo
abs/2301 12485.
Yeqing Lin, Minji Lee, Zhao Zhang, and Mohammed AlQuraishi Out of many, one: Designing and scaffolding proteins at the scale the structural universe with genie 2, 2024.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hic, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637): 123-1111):11111110, 2023.
Yaron Lipman, Ricky TQ Chen, Heli Ben- Gen-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.027447 2022.
12

---

Published as a conference paper at ICLR 2025
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint urXiv:2209.03003 2022.
Yangyang Miao and Bruno Correia. Difftopo: Fold exploration using coarse grained protein topol- ogy representations. In ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design, 2024.
Weili Nie, Sifei Liu, Morteza Mardani, Chao Liu, Benjamin Eckart, and Arash Vahdat. Compositional text-to-image generation with dense blob representations. arXiv preprint arXiv:2405.082466 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representa- tions, 2018.
Brian L Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif- scaffolding problem. arXiv preprint arXiv:2206. 041 19, 2022.
Michel Van Kempen, Stephanie Kim, Charlotte Tumescheit, Milot Mirdita, Jeongjae Lee, Cameron LM Gilchrist, Johannes Söding, and Martin Steinegger. Fast and accurate protein struc- ture search with foldseek. Nature biotechnology. 42(2): 2024.
Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eise- nach, Woody Ahern, Andrew Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature, 620(7976):1089-11000
2023.
Jason Yim, Andrew Campbell, Andrew Y. K. Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan S. Veeling, Regina Barzilay, Tommi Jaakkola, and Frank Noé. Fast protein backbone generation with se(3) flow matching, 2023a.
Jason Yim, Brian L. Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation. arXiv preprint ar arXiv:2302.0223 2023b.
Jason Yim, Andrew Campbell, Emile Mathieu, Andrew Y. Foong, Michael Gastegger, José Jiménez-Lune Sarah Lewis, Victor Garcia Satorras, Bastiaan S. Veeling, Frank Noé, Regina Barzilay, and Tommi Jaakkola. Improved motif-scaffolding with se(3) flow matching, 2024a.
Jason Yim, Andrew Campbell, Emile Mathieu, Andrew YK Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan S Veeling, Frank Noé, al. Im- proved motif-scaffolding with se (3) flow matching. ArXiv, 2024b.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836-3847, 2023a.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023b.
Yang Zhang and Jeffrey Skolnick. Tm-align: a protein structure alignment algorithm based on the tm-score. Nucleic acids research, 33(7):2302-2009) 2005.
Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12490-22499 2023a.
Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman, Aditya Grover, and Ricky T. Q. Chen. Guided flows for generative modeling and decision making, 2023b.
13

---

Published as a conference paper at ICLR 2025
A EXPERIMENTAL DETAILS
A.1 IMPLRMENTATION DETAILS
Algorithm 3: Residue Segmentation and Gaussian Fitting
Input:
Positions of alpha carbons pos, Secondary structure annotation ss, Radius threshold threshold
Output: List of ellipsoids with secondary structure type, position, and covariance.
distmat <- pairwise_distances(( ddistances(pss,ps))
edges - argwhere( (distmat < threshold (and(ss(Nooo)) == (S (t,NNone)
urgwheree(distmat
G' t graph.from.c weromeegee(edgge))
components f- extract connected components(G)
ellipsoids
for component in components do
component <- list(component)
if [scoopoonent[]]] = loop then
L continue
if le)coomponntt) < 5 then
L continue
ellissoids.pppend(
'type': ss(component(0]],
'position': : pos[component).
"covariance': covariancefossccoopoeen....
return ellipsoids
Training. We finetune Multiflow (Campbell al., 2024) starting from checkpoint provided the authors' GitHub and use their optimizers, data filtering, losses, and hyperparameters (Adam W, learning rate 0.0001). To monitor training progress, we run inference conditioned and report designability and ellipsoid adherence metrics. We do not employ early stopping based on any of the metrics. Training is carried out on 8 NVIDIA A100 GPUs for 20 hours, corresponding to 83 epochs.
data ellipsoids
Data. The training data consists PDB proteins and synthetic data. The PDB training colleted by Yim et al. (2023b) consists of 18684 proteins of length 60-384. We train on crops of size 256. Ad- ditional training data are 4179 synthetically generated proteins of Multiflow with high designability at test time are extracted from the PDB
(see Camphell et al. (2024)). The ellipsoids to condition proteins and synthetic proteins as described in Section 3.1, Figure 2 and detailed in Algorithm 3. This means that, at training time, the number of ellipsoids is determined by the data. In Figure 11, we provide a histogram
the number of ellipsoids per protein in PDB proteins. We trained addi- tional models (no results included in the paper) on the AlphaFold2 (Jumper et al., 2021) database following Lin et al. (2024), which yields a model with increased diversity and blob adherence but decreased designability.
Details of PosEmbed. Our Cwvarant-Cross-Attteee encoding PosEmbed of the relative positions of ellipsoid means to residue positions. This sinusoidal positional encoding of the relative positions (the vectors between ellipsoid means and residue positions). Each number of the 3D offset vector encoded into 64 dimensions, and all 3 concatenated.
module in Algorithm uses a positional
Inference time protein length choice. At inference time, ProtComposer (and Multiflow, RFDiffu- sion, and Chroma) take a protein length L as input, which specifies the number of residues in the generated protein. ProtComposer, each ellipsoid Eg is associated with number of residues ną that is supposed to end up in that ellipsoid. The sum of n& does not have to match the total length L. The T4 are just an additional conditioning input that the model can adhere but does not have to. Even if the sum of nış is larger than L, the model can still (and empirically does) use some of the residues for strands and to connect the ellipsoids. When generating from ellipsoids from our ellip- soid statistical model, ellipsoids, we set to be the length of the original protein which the ellipsoids were extracted from, so T. E.En nk..
set L to be equal to the of ng. When generating from data-extracted
14

---

Published as a conference paper at ICLR 2025
A.2 RUNNING CHROMA
For running Chroma (Ingraham et al., 2023), we use the author's GitHub. For Table 1, we run Chroma conditioned on the same data ellipsoids as ProtComposer, while the samples in the Pareto front (Figure 4) under varying sampling temperature are sampled unconditionally. The inverse- sampling temperatures that we sweep over are [1, 1.4, 2, 4, 8, 10, 15, 20, 40, 80] where 10 the de- fault provided by the authors. The length distributions are the same as for Multiflow described in A.4.
To condition Chroma, utilize the conditioner classes provided in the author's GitHub repository. Concretely, we use the ShapeCondit tinee sioer that is provided to condition on point clouds. The point cloud that we condition on is obtained by first sampling from Gaussian mixture model that is defined by the set of means and covariances for the ellipsoid layout we condition on. Then, we reject or accept each sample based on whether or not its Mahalanobis distance any of the ellipsoids' Gaussian is less than v5A, which is the same threshold that we use to define ellipsoid boundaries in equation I, and that is used in our ellipsoid visualizations throughout the paper. The only parameter that we alter from the ShapeCondi tinerr default settings to set the number of residues according to our length distribution and to setautosca l le=Fal FaFllas se since the desired protein volume is known and specified by the input point cloud.
A.3 RUNNING REDIFFUSION
For running REdiffusion (Watson al., 2023), we use the author's GitHub. The samples in the Pareto front (Figure 4) under varying sampling temperature are sampled unconditionally.
The sampling temperatures that we sweep over are [0, 0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 2, 2.8, 3.2, 4,4.8, where 1 is the default provided by the authors.
A.4 PARETO FRONTIERS DETAILS
The black dot in Figure 4, which corresponds the metrics achieved by PDB proteins from the PDB is computed from our validation set. This is the validation set of proteins that were deposited in PDB after 2021 used by Campbell et al. (2024).
Running Multiflow and rotational annealing. What we term "rotational annealing" is described as an exponential rate schedule for the rotation vector field in Multiflow (Camphell al., 2024) and as "inference annealing" in FoldFlow (Bose et al., 2024). What we call rotational annealing strenghts is denoted in both of these papers. We run Multiflow at the rotational annealing strenghts [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.4, 1.6, 2, 3, 4, 5, 6, 10] since 4,5, and 6 all yield essentially identical results to the annealing strength 10 used in Multiflow by default, and variation is achieved at low annealing strenghts.
Standard deviations in shaded regions. The standard deviations that we show as shaded regions in Figure are computed from parameter settings for ProtComposer" frontiers and 3 for the Multi- flow frontiers. We select the parameter settings for the point on the frontier that at a designability 0.98 for each of the three metries, For all three parameter settings, rerun inference with 4 addi- tional seeds and compute the standard deviations from the results for all four metrics (designability, novelty, diversity, and helicity). The final reported standard deviations are obtained by averaging the standard deviations across the three points.
We chose a fixed number of ellipsoids for consistency across protein lengths and the specific value of K since it is the most frequent number of ellip- soids in PDB proteins (see Figure 11). However, this still gives rise to a distribution of lengths, and we detail our considerations to provide meaningful comparisons here: To construct ProtComposer's Pareto frontiers, we sweep over all parameter combinations of a,, M, and X. The parameters a, M, are parameters of ProtComporr ProtComposer's ellipsoid statistical model, with each combination giving rise to different distribution of ellipsoids. Each ellipsoid distribution has a different length distribution (see Figure 8). Since designability is impacted by protein length and our evaluation metries of helicity, diversity, and novelty could be impacted by length as well, we also sample Multiflow at all rotational annealing strengths with several length distributions. We combine all resulting points into a single
Considering length distribution effects.
15

---

Published as a conference paper at ICLR 2025
Paramnteer combination count
combination
Average protein length
Protein length standard deviation
Figure 8: Left: Average lengths of proteins from our ellipsoid statistical model for all com- binations of 13.4...00 helix fraction ? [0.2, 0.4, 0.6, 0.8, and ellipsoid anisotropy E [5, 10, 20, 50, 100). Right: The corresponding standard deviations of the length distributions of each parameter combination.
set and compute Multiflow's Pareto frontier from that set. an attempt to cover the space of length distributions well, we sample with the following length distributions: 1) the length distribution of the a, V; 7 combination with the minimumm mean length. 2) the length distribution of the a, V, 7 com- bination with the maximum mean lenght. 3) the length distribution of the o, M, 7 combination with the average mean length. 4) a uniform length distribution between the minimum mean length of all length distributions (92) and the maximum (198).
Designability computation. We compute the fraction of structures for which at least one out sequences sampled by ProteinMPNN (Dauparas et al., 2022) results in scRMSD < 2A when folded with ESMFold (Lin et al., 2023). We use the author's ' repositories for running the tools. For ProteinMPNN the backbone version that takes N, CA, C.
B DISCUSSION
Additional Related Work. Several other protein structure generative models were developed along- side, on top of, or after the mentioned Chroma (Ingraham et al., 2023), FrameDiff (Yim et al., 2023b), and RFDiffusion (Watson et al., 2023). This includes Genie (Lin & AlQuraishi, 2023), Anand & Achim (2022), Protpardelle (Chu et al., 2023), and Genic2 (Lin et al., 2024) among others. We also note DiffTopo (Miao & Correia, 2024), which first generates skeleton protein representa- tion, related to our ellipsoid layouts, where helices and individual strands of sheets are represented three 3D points, then initializes existing secondary structure elements adhering the skeleton, noises them, and then denoises them using RFDiffusion.
User guidelines for choosing K: All specifications of numbers of ellipsoids K < 10 can be pected to yield successful generations. To see which K are the most in-distribution, please see the histogram Figure 11 where we visualize the frequency of different K.
Design of ICA and treating ellipsoids equivalent to residue frames. In our transformer layers (algorithm 2), the residue tokens update all ellipsoid tokens, and ellipsoid tokens update all residue tokens. In Invariant- EearianntCCoos-- Cross-Attentin (algorithm 1). we inject ellipsoid position and geometry infor- mation into the residues tokens without updating ellipsoid tokens.
The reason: ICA updates a residue token based on transforming the ellipsoid means and covarianc matrices into the local coordinate frame of the residue (where residue frames are defined as in AlphaFold2). The same mechanism is not applicable to updating ellipsoid tokens based on residue positions since a canonical assignment of a frame to an ellipsoid is not possible. To see this, we consider the worst-case scenario of round ellipsoid clearly, no canonical assignment of 3D orientation is possible. In the best-case scenario of an ellipsoid with three distinctly sized principal components, could choose, e.g., the two largest construct frame from. However, their sign is arbitrary, leading to 4 options among which no canonical choice exists. Thus, we opted for our ICA without ellipsoid token updates, which is empirically sufficient for strong ellipsoid adherence.
16

---

Published as a conference paper at ICLR 2025
Furthermore, only injecting relative positional encoding into the keys and values and not the queries was our default choice since is how relative positional encodings are used in language model transformers (Shaw et al., 2018) in geometric transformers such as SE3-Transformer (Fuchs et al., 2020).
Specifying the number of residues per ellipsoid. In Ellipsoid definition, each ellipsoid A: is annotated with a number residues ni that should be associated with the ellipsoid. Removing this additional feature could provide additional flexibility for the model inference time. We chose to specify the number of residues per ellipsoid to give the user the option of controlling it (or having the number be determined via linear fit if preferred).
Possible ProtComposer use cases. We envision frameworks based on ProtComposer where only secondary structure conditioning possible to enable use cases such as the following:
Example use case: We aim to scaffold therapeutically relevant functional site. The protein requires a certain shape to fit into a delivery mechanism. With ProtComposer we can specify the rough shape and size of the scaffold to still fit into the delivery mechanism
ProtComposer can redesign the connectivity of secondary structure elements: biologists aim to escape the existing space of protein topologies and discover new ones that can be used as scaffolds or for other design tasks.
Example use case: We aim to design binder for target at a flat beta-sheet region. With ProtComposer, we can specify that beta-sheet of the right size and shape should interface with the target's beta-sheet to increase the probability of success in generating strong binder.
We often know how much flexibilityrriiidity want in certain areas of the protein. With ProtComposer, we can place a rigid helix bundle, a beta-barrel, or more loosely connected substructures in those regions.
The Compositionality Metric. We note that our compositionality metric, the effective number of components, which is exp(- Σn Pk logpk) where pk = m Σx my, is based on the Diversity Index which is commonly employed in ecology demography.
C ADDITIONAL RESULTS
C.I MULTIFLOW CODESIGN SEQUENCES
Table 3: Sequence-structure co-generation. ProtComposer's self-consistency when generating sequence and structure jointly vs. dropping its generated sequence and producing the sequence with ProteinMPNN. 1-seq Designability refers to generating 1 sequence per structure, while 8-seq Designability uses the best out of 8 sequences per structure. Self-consistency RMSD abbreviated
as scRMSD.
Approach
1-seq Designability
Median scRMSDJ
Mean scRMSD)
8 8-seq Designabilityt
Joint Generation
0.75
1.76
2.15
ProteinMPNN
0.81
1.65
2.41
0.98
Multiflow codesign sequences. Whenever assessing designability we use ProteinMPNN to infer the sequence for protein structure generative model's generated sequence. Our base model Multiflow jointly generates a protein sequence and structure. Commonly we ignore this generated sequence and also produce a sequence conditioned on the structure using ProteinMPNN. Here we assess how ProtComposer's generated sequences compare with those obtained by ProteinMPNN conditioned on ProtComposer's structures.
For this purpose, we select statistical model parameters with high designability (v 50, o = 5) and draw 400 structures together with sequences from ProtComposer. Table shows that the designabil- ity of the jointly generated sequence is lower than that of a single sequence generated with Pro- teinMPNN (the default designability metric shows the best of 8 ProteinMPNN sequences). This is similar the Multiflow paper (Camphell al., 2024), where co-design did not provide designability
17

---

Published as a conference paper at ICLR 2025
Varying guidance scale from to
Separate
Conditional Both
Unonditionl Both
Ellisoold Adherence metric JSD
Seq Conditional
Struct Iterpolate Seq Conditional Beth
Interpolate
020
0.15
0.0
0.2
3.3
0.4
0.7
5.8
0.9
1.0
Designability
Figure 9: Evaluation of our different celf-conditioning variants in terms of designability and ellipsoid adherence measured in terms of Resegment JSD.. The different variants are described Appendix C.2.
improvements. Interestingly, the median self-consistency RMDSs (scRMSD) of the jointly gener- ated sequences worse, while their mean scRMSDs are better.
C.2 ABLATION OF VARIANTS FOR SELF-CONDITIONING UNDER GUIDANCE
Recall that both Multiflow and ProtComposer use self- celf-condiiioiinn (Chen et al., 2023), in which, during inference, the flow-model receives the output the previous integration step as additional self-conditioning imput. During inference, the unconditional model po R, a) produces the self- conditioning variable X, and from the ellipsoid conditioned model po(t. R, a E), we obtain XE- Instead of supplying X to the unconditional and XE to the conditioned model, we use XXE + (1 X)X. Please refer to the Multiflow paper (Campbell et al., 2024) for information on how Multiflow without classifier-fre free guidance performs self self-conditioning.
This choice is based on our exploration self cenditondi variants that are shown in Figure 9, which shows that the interpolate option performs best. The variants explore are:
Separate: supply X to the unconditional and X to the conditioned model.
Conditional Both: supply X to both models.
Unconditional Both: supply Xg to both models.
Beq-Condittinaal Both: supply X to the unconditional and XE to the conditioned model for the structure self-conditioning input while the sequence self-conditionias input is extracted from X for both models.
Struct-Iterpolat Seq- Condioniiii Both: supply AXg + (1 A)XX both models for the structure self-conditioning input while the sequence self-conditioning input is extracted from XE for both models.
Interpolate: supply XXE X)X to both models.
C.3 FURTHER RESULTS
Table 4: Compositionality of different protein structure generative models.
ProtComposer (A 2.0)
Multiflow
Chroma
RFDiffusion
Compositionality
3.2
1.9
2.5
3.3
18

---

Published as a conference paper at ICLR 2025
Beta-sheet ellipsoids
Alpha-helix ellipsoids
100
Number of residues
Number of residues
60
60
20
20
100 200
300
400
100 200
300
Ellipsoid volume
Ellipsaid volume
Figure 10: Scatter plots of the number residues in an ellipsoid and the ellipsoid's volume for beta-sheet and helix ellipsoids. The relationship is close to linear. Thus, we choose the number of residues for an ellipsoid from our synthetic ellipsoid statistical model based their volume and the linear fit to residue count. For beta sheet ellipsoids, the linear fit correlation coefficient is 0.93, and the p-auuue 0.0. For alpha-helix helix ellipsoids, the linear fit is 0.97, and the p-value is 0.0.
PDB testset length histogram
Ellipsoids per protein
120
100
Number proteins
Number of proteins
40
20
100
150
200
250
300
350
400
5.0
12.5 15.0
Residue count
Number of ellipsoids
Figure 11: Histograms for the validation set of PDB proteins. Left: Histogram of protein lengths. Right: Histogram of the number of ellipsoids per protein with our default segmentation parameters.
19

---

Published as a conference paper at ICLR 2025
Cutoff
Cutoff 2
Cutoff 3
Cutoff 4
Cutoff 5
Cutoff
Cutoff 7
Cutoff 8
Cutoff 9
Cutoff 10
Figure 12: Different cutoffs of squared Mahalanobis distance for visualizing ellipsoids.
Figure 13: Hand-specified beta-barre of increasing size.
Cutoff 4
Cutoff 4.5
Cutoff 5
Cutoff 5.5
Cutoff 6
Cutoff
Cutaff
Cutoff
Cutoff
Cutoff 10
Cutoff 4
Cutoff 4.5
Cutoff 5
Cutoff 5.5
Cutoff 6
Cutoff
Cutoff
Cutoff
Cutoff
Cutoff
Figure 14: Ellipsoids obtained with different radius cutoffs (Å units) for the residue segmentation
(Algorithm 3) for the two proteins with PDB IDs 7DG4 (top) and 7V2T (bottom).
20

---

Published as a conference paper at ICLR 2025
Color by helicity
-Mspplacmmmn
Covernge
Arcurac
0.7-
0.6-
0.6
0.4
0.5-
0.4
0.4
0.4
0.6
0.4
0.5
0.6 Designability
Designabiltt
Designability
1.0
0.9
0.9
Licellhood
Rabilb J5D
0.3
2..8
Accura
0.7
-5-
0.6
0.5
44
0.2
0.4
0.5
1.0
0.2
0.4
0.5
100
0.6 Designability
0.8
DesignabilRyy
Designabillty
Figure 15: Scatter plots of designability vs. ellipsoid adherence metrics when conditioning on synthetic ellipsoids that were drawn from ellipsoid statistical model.
21

---

Published as a conference paper at ICLR 2025
TM Score: 0.24
TM Score: 0.25
TM Score: 0.25
TM Score: 0.27
TM Score: 0.28
PDS Protein
Generated
TM Score: 0.23
TM Score: 0.22
TM Score: 0.26
TM Score: 0.26
TM Score: 0.27
POB Protein
Generated
Score:
TM 0.25
TM 0.25
0,23
POB Protein
Generated
TM Score: 0.25
TM Score: 0,24
TM Score: 0.25
TM Score: 0.25
TM Score: 0.25
POB Protein
Generated
Figure 16: Several rows of randomly chosen protein structures from the PDB validation set, their corresponding ellipsoids, and, below them, the protein we generate conditioned on those ellipsoids. Each pair of PDB protein and generated protein is annotated with the TM-Score between them. The TM-Scorre are very low and we generate novel proteins while adhering to the layouts.
22

---

Published as a conference paper at ICLR 2025
maNTT iamAN
AMMAN
Figure 17: Random samples of ellipsoids and the generated proteins from our ellipsoid statistical model with parameters = 6, v = 5, 7 = 0.4, and the number of ellipsoids K varying from (top) to (bottom).
23

---

Published as a conference paper at ICLR 2025
Facu
Figure 18: Random samples of ellipsoids and the generated proteins from our ellipsoid statistical
model with parameters a 6, v = 5, 7 0.4, and the number of ellipsoids being K 5.
24

---

Published as a conference paper at ICLR 2025
V= 5=5
== 10
V= 20
== 50
== 100
3=3
0= 4
0=5 5
o= 6
0=7
8=8
9=9
o = 10
Figure 19: Ellipsoids sampled from our ellipsoid statistical model with different parameter combi- nations of v and a, which control the ellipsoids' "roundness" or anisotropy and their compactness, respectively.
25

---

Published as a conference paper at ICLR 2025
0.75
Misplacement
9.40
0.65
Coverage
Accuracy
0.65
0.80
0.69
0.75
0.53
0.70
0.50
0.68
2.48
0.15
0.40
Guidance Strength
Guidance Strength
Guidance Strength
0.45
Soft Accuracy
0.40
Likelhoood
Reblob JSD
-6-
0.35
0.30
0.20
-10-
3
0.10
2
Guldance Strength
Guldance Strength
Guldance Strength
Figure 20: Ellipsoid adherence metrics under varying scales of guidance strength À for ellipsoids
extracted from the validation data. The highest ellipsoid adherence is attained values of X 1.
0.45
5.5
58
5.8-
Misplareeent
Accuracy
0.23
0.70
6.61
0.25
0.30
6.35
0.00
645
050
0.5
0.25
0.30
6.35
0.40 TMScare
6455
050
035
0.25
0.30
6.35
9.40 0.55 TMScare
050 055
TMScare
0.45
455
5.7
Soft Accuracy
Likelhood
-35
0.6-
0.50
6.5-
-7.5
0.40
0.25
.80
0.23
0.30
5.55
646 TMScore
CA3
030 0.5
0,30 0.23
0.25 0.40 C43 030
0.5
0.23 0.30
0.55
2.40 0.43 030 025 TMScore
TMScore
Figure 21: We generate proteins conditioned on ellipsoids extracted from the validation data and show scatter plots ellipsoid aligmment metries and the TMScore between the PDB protein and the generated protein. In general, the TMScore is very low, and the generated proteins are novel.
26

---

Published as a conference paper at ICLR 2025
lambda: 1.0
ambda: 1.4
lambda:
lambca:
lambda:
ULR
TULPO
SABARA
larbbda: 1.0
ambaaa: 2.4
lambea:
lemiiia::
lambtia
TAL LAAS
AEED
larbbda: 1.0
ambda: 1.4
lambde:
lambca:
lambda:
lambda: 1.0
ammba: 2.4
lambda:
lambci:
lambda:
larbbda: 1.0
lambda: 1.4
lambba:
lamàci::
lambda:
larbbda: 1.0
ammbaa: 2.4
lambea:
lambcii:
lambria:
lerbbds: 1.0
lambda: 1.4
lambda:
lambci:
lambda:
W
CaRRoD
Figure 22: Several protein layouts (rows) and ProtComposer generations for them with varying guidance (columns) where guidance strengths (A 2 1) including extreme values for which the model breaks down.
27

---

Published as a conference paper at ICLR 2025
Color by guidance factor
Calor by helix fraction
Color by blob compactness
Color by blob anisotropy
Diversity
4c-
40-
12-
35-
10-
25-
65
22
0.2
0.4
06
06
15
0.2
0.4
05
0.6
1.0
0.2
0A
0.5
15
Designability
Designabiiiyy
Designability
Designability
0.
08-
6.8-
Helicity
57
03
6.7-
24-
05-
6-5-
55-
05-
5.5
2.4
24
0.4
5.4
03
5.3
6.2
522
02-
6.2
05
1.0
Designability
Designability
Designability
Designabllty
Compositionalitty
30
3.5-
25.
2.5-
2.0
29
20
2.0
3.5
35
1.5
3.5
Designability
Designability
Designability
Designablliy
Novelty
40-
40-
10-
322
0.4 0a Designability
08
1.
.35 0.2
0.4 08
10
32
billggabbliiy
Designability
Designabliity
Figure 23: Scatter plots of designability and novelty, entropy, helicity, or diversity when conditioning on synthetic ellipsoids that were drawn from our ellipsoid statistical model. The colors indicate the value of the statistical model's parameter that is in the caption of each column.
28

---

Published as a conference paper at ICLR 2025
lambda: 0.0 Covs 0.36 Acc: 0.95
lambda: 0,3
lambda: 0.5
lambda: 0.7
lambda: 1.0
Cov: 0.39 Acc: 0.6
Cov: 0.39 Act: 0.78
Cov: 0.64 Acc: 0.88
Cov: 0,76 Acc: 0.92
76
0.87
CLR
lambda: Cow: 0.83 0.68 Lt: -3.40 19 Soft Ace: 0.83
0.3
lambda: Cov: 0.95 Acc: 0.69
lambda:
Cov: 0.94 Acc; 0.72
Acc: 0.7 LiL -3.2 Soft Acc: 0.89
Cov: 0.95
Cov: 0.97 Acc: 0.72
LL: -3 27 Soft Acc: 0.89
L1: -3.27 S2ftSSSftt Acc: 88
LL: -3,29 Soft Acc: 0.88
lambda: 0.0 Cov: 0.16 Acc: 0.2 LL: Soft
lambada: 0.3
lambda: 0.5
lambda: 0.7
lambda: 1.0 Cov: 0,7 Acc: 0.95 2.955
Cov: 0.24 Acc: 0.41
Cov: 0.37 Acc: 0.55
Cov: 0.61 Acc: 0.94
Soft
LL:
lambda:
lambde:
lambda:
Cav: 0.4 Acc: 0.37
Cov: 0.44
Acc: 0.92
Cov: 0.54 Acc: 0.93
Cov: 0.58 Acc: 1.0
Cov: 0.7 Acc: 1.0
LL: 5..2 Saft Ace: 0.84
LL: 4.94 Soft Acc: 0.56
LL: 4.18 Soft Acc: 0.92
LL: 4.33 Soft Acc: D 97
LL: 33.4 Soft Acc: 1.0
lambda: 0.0 Cav: 0.92 Acc: 0.8
lambada 0.3
lambsda: 0.5 Cov: 1.0
lambda: 0.7
lambda: 1.0
Cov: 0.99 1.0
Cov: 1.0 Acc: 1.0
Cov: 1.0 Acc: 1.0
Soft
lambda: 0.3
0.5
0.7
lambda:
Cow: 1.81 Acc: 0.98
Cov: 0.79 Acc: 0.98
Cov: 0.89 Acc: 0.98
Cov: 0.99 Acc: 0.89
Cov: 0.93 Acc: 0.91
LL:33.53 .3.53 Soft Acc: 0.98
LL: 21 Soft Acc: 0.98
LL 23:28 Soft Acc: 0.97
LL: .3.0 Softt Acc: 0 93
LL: 1:11 Soft Acc: 0.95
lambda: OND
lamoda: 0.3 0.53
lamoda: 0.5 Cov:
lambda: 0.7 Cov:
Cov: lamficaa 1.0
0.45 0.18
LL -1.21 Soft Acc: 0.33
LL: -4.4 Soft Acc: 0.33
-3.75 Soft Acc: 0.61
LL: -3,47
Acc: 67
LL: -3.26 Soft Acc: 0.87
Figure 24: Several protein layouts (rows) and ProtComposer generations for them with varying guidance (columns); no guidance on the left, full guidance (A 1) the right.
29


Reviews:
[{"review_id": "CHOXKi41RG", "reviewer": "Reviewer_7wor", "summary": "This paper proposes a framework to generate protein structures by conditioning on layouts specified through 3D ellipsoids. The conditions include location, size, orientation and secondary structure. These conditions are injected to flow-base protein generative models via proposed cross-attention and update modules. It shows greatly improved controllability and designability over baselines.", "questions": "Is it possible to set the order of ellipsoids? Or how complicated would it be to extend this framework to allow user to set the order of ellipsoids?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper is well motivated, formulated, written, and evaluated. \n\n1. The injection of ellipsoid information is achieved through cross attention. This allows the ellipsoids to be unordered set. i.e. user doesn’t have to specify the order or ellipsoids; model also decides the order of ellipsoids. \n2. The formulation of ellipsoid token is effective and is easy to be extended to different conditions other than secondary structure. (such as hydrophobicity)\n3. Thorough analysis of ellipsoid consistency, including both geometric and probabilistic metrics.\n4. Fig4: The comparison on designability/diversity with baseline methods are done as comparing Pareto frontiers with varying sampling temperatures/guidances. This clearly shows the tradeoff between the methods and the performance improvement from baselines.  I found this analysis insightful and believe other papers arguing increased performance could benefit from a similar evaluation scheme.\n5. The practical use-case of the method is shown in Section4.3 flexible conditioning.\n6. The controllability is greatly improved from Chroma (Table1). Accuracy and coverage is very impressive.", "weaknesses": "How is sequence design performance? What I understand is that the designability is solely based on the generated structure (i.e. generated sequence is discarded). Can you also present co-design designability value as in MultiFlow paper?\n\nOther than that, I did not find any major weaknesses from the paper. However, the ablation study can be improved. Most of the model ablations are based on guidance strength, but I am also curious about ablation study on (i) ellipsoid segementation cutoff (5A currently), (ii) allow residue token to update ellipsoid token or not. For (ii), explaining the reasoning behind the design choice may suffice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aFl2mFLnKW", "reviewer": "Reviewer_679L", "summary": "This work extends Multiflow to accept spatial conditioning of secondary structures via 3D ellipsoids, aiming to improve control in protein generation and reduce the overrepresentation of alpha-helices in current generative models. Building on Multiflow’s architecture, the authors address two main challenges:\n\n1. Integrating and updating ellipsoid conditioning with structure embeddings with minimal modifications: they introduced an *Invariant Cross Attention* module to update residue embeddings while preserving local SE3 invariance.\n2. Implementing an effective conditioning approach for flow-matching models: they used classifier-free guidance to interpolate flow vectors across translation, rotation, and amino acid spaces, and employ self-conditioning to refine predicted structures\n    \nExtensive experiments show that the proposed model can faithfully follow spatial conditioning, resulting in greater diversity, novelty, and improved secondary structure composition. This improves Multiflow by generating proteins with secondary structures more similar to natural proteins. \nOverall, this work presents a straightforward approach to control protein generation, enhancing Multiflow's diversity, novelty, and secondary structure accuracy.", "questions": "1. **Ellipsoid Representation**\n\n    a. Choosing the Number of Ellipsoids (k):\n    - *Training:* Is k determined by the structure of the training protein? If so, what is the distribution of k in natural proteins?\n    - *Evaluation based on the statistical model:* The authors appear to have used a fixed k=5 in the experiments. How was this number chosen? Have the authors tested other k values?\n\n    b. Number of Residues per Ellipsoid:\n    - The current representation specifies the number of residues in each ellipsoid, but the authors show that this number directly depends on ellipsoid volume. Could specifying the number of residues be redundant, and might removing this constraint provide the model more flexibility in generation? Have the authors examined the impact of residue count on amino acid (AA) prediction?\n\n2. **Invariance Cross-Attention (ICA) Layer Design**\n\n    a. The authors separately model the SE3 features (E_k) and scalar features (e_k) of ellipsoids using the proposed ICA and transformer to achieve SE3 invariance in the local frame. Has the team considered alternative approaches, such as modeling ellipsoids as “pseudo frames” with SE3 and scalar features and simply using IPA to update ellipsoid and residue features together?\n\n    b. In Algorithm 1:\n    - Could the authors clarify the *PosEmbed* used in line 223? Does it include distance, angles, or local coordinates?\n    - Could they also explain why the query uses un-updated $s$, while the key and value use $a$, which incorporates current ellipsoid information?\n\n3. **Results in Table 2 (natural proteins):** The model, even with the strongest guidance, tends to overestimate helices in proteins. Additionally, the authors did not present designability results, which, based on Figure 16, may be compromised with strong guidance. Could the authors elaborate on the model's performance in addressing the “overrepresented helix problem,” the trade-offs with other metrics, and its overall comparison to models like *RFDiffusion*?\n\n4. In self-conditioning (line 290), they propose supplying interpolated conditions to both conditional and unconditional models, suggesting this improves “designability and ellipsoid adherence for all $\\lambda$ values.” However, no ablation studies were provided to verify this claim.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "**[Clarity & Quality]**\n- The manuscript is well-written, with a thorough introduction and background information on protein structure generation, spatial conditioning, and flow matching for data generation. Overall, it provides a smooth reading experience.\n- The paper is of good quality, presenting clear mathematical foundations grounded in current techniques for protein modeling, diffusion-based generation, and guided sampling.\n- The problem is well-defined, and the authors designed several experiments to evaluate model performance in 1) following conditioning, 2) improving general performance, and 3) demonstrating practical use in flexible conditioning, with both quantitative and qualitative comparisons.\n\n**[Significance]**\n- The spatial conditioning approach using ellipsoids is intuitive for practical applications and has potential implications for the utility of protein generation models.\n- The proposed methods appear generalizable to various spatial conditioning scenarios. (*However, this paper focuses solely on secondary-structure conditioning.*)\n\n**[Originality]**\n- The authors introduce a novel conditioning modality using spatial ellipsoids for protein generation, along with a new layer, \"invariant cross-attention,\" to integrate this information.", "weaknesses": "**[Clarity]**\n- Some design choices and model details lack clear explanations or in-depth examination (see Q1, 2, and 4).\n\n**[Soundness]**\n- Performance on Natural Proteins: While the authors demonstrate high designability at fixed helicity levels on synthetic data, it isn't as clear if these benefits hold for natural proteins (see Q3).\n\n**[Significance]**\n- Scope: The current methods are examined only on Multiflow and for secondary structure guidance. Their practical impact on other protein generation models and types of spatial conditioning (e.g., domains, hydrophobic cores) is not extensively explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q9SPqgwx6i", "reviewer": "Reviewer_D7Wa", "summary": "This paper presents ProtComposer, which is a fine-tuned model from MultiFlow to support conditional generation based on 3D ellipsoids, showing success at controllable design and achieving SOTA on the Pareto frontier of designability and diversity.", "questions": "- Line 355: How is the length between ellipsoids determined/sampled? Also, consider an ellipsoid with beta strand annotation, how is the length between each stranded segment determined (particularly if a strand ellipsoid is formed by segments that are distant in sequence but close in structure)? \n- It would be helpful if the authors could provide ablation study results on the effect of self-conditioning, particularly the two self-conditioning schemes described in line 290.\n- Figure 9: What is the linear fit and statistical significance in both cases?\n- During training, is the ellipsoid conditioning information always provided, or only provided for a percentage of time?\n- Line 367: Why is a structured residue considered as “covered” if it is inside at least one ellipsoid instead of inside the ellipsoid it is assigned? \n- Table 1: Could the authors provide some intuition on why over-guidance (\\lambda > 1) performs better than the conditional model itself (\\lambda = 1)?\n- Table 2: Does “PDB proteins” correspond to the validation dataset or does it include the training dataset?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is well written with clean visualizations demonstrating methods and results. The concept of utilizing ellipsoids as conditions for protein generation is interesting and novel, providing a bridge between protein-level conditioning and atom-level conditioning. In addition, the authors propose an effective Invariant Cross Attention module for integrating ellipsoid conditioning and demonstrate success at achieving SOTA performance on the Pareto frontier of designability and diversity.", "weaknesses": "I have no major concerns about this paper. However, it would be helpful if the authors could elaborate on the applicability of the ellipsoid-based conditioning approach on practical protein design tasks. How would it help with or facilitate the protein design process?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A1jrejawF4", "reviewer": "Reviewer_eiq5", "summary": "Introduces ProtComposer, a generative model for proteins. ProtComposer seeks better control over the shape of generated proteins, as well as allow for greater novelty in the generation of proteins. Notably, the user has to choose between control or novelty, ProtComposer does not achieve both simultaneously. These tasks are accomplished by the introduction of 3D ellipsoid frames to guide the generation of proteins by the pre-existing Multiflow algorithm. \n\nWithout a pre-existing metric the authors feel sufficiently quantifies compositionality, they introduce their own metric. It is shown that ProtComposer outperforms existing methods in this metric.", "questions": "I am assuming that the user can redefine K during each trial as they see fit, though studies on guidelines in choosing K would be helpful. In situations where the user has only a vague idea of what they are looking for (an unfortunately common occurrence), having a guide on where to start would be beneficial.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "It addresses an important and relevant area. The results are overall quite impressive as well. Additionally, showing the ability to work with handcrafted ellipsoid frames or the more easily scalable ML generated frames shows the practicality of ProtComposer.\n\nI really like the point about using simple ML models for ellipsoid sampling as compared to NNs. I would like to see some stronger analytical or experimental justification of the claim though.", "weaknesses": "Since a custom metric is introduced in this paper and then used as justification for the performance of the model, a section (either in the main paper or the appendix) justifying this metric would be nice. Showing comparisons to other pre-existing metrics, performance of many other algorithms under this metric, or stronger domain justification would strengthen the meaning of the results. I did not reject over this since the metric intuitively looks good, but further empirical or analytical support would be nice.\n\nThe formatting of the extra results in the appendices results in very odd page layouts, where some pages are blank, others have odd whitespace gaps, etc. Adjustment to make these pages more presentable would be nice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)