Reconstructing Heterogeneous Biomolecules via Hierarchical Gaussian Mixtures and Part Discovery
Authors: Shayan Shekarforoush
David B. Lindell
Marcus A Brubaker
David J. Fleet
Venue: NeurIPS 2025 poster ()
Keywords: Cryo-EM 3D Reconstruction
Gaussian Mixture Models
Part Discovery
Primary area: machine_learning_for_sciences
TL;DR: We present a part-aware hierarchical GMM-based density model to tackle cryo-EM heterogeneous reconstruction.

Abstract: Cryo-EM is a transformational paradigm in molecular biology where computational methods are used to infer 3D molecular structure at atomic resolution from extremely noisy 2D electron microscope images.
At the forefront of research is how to model the structure when the imaged particles exhibit non-rigid conformational flexibility and compositional variation where parts are sometimes missing.
We introduce a novel 3D reconstruction framework with a hierarchical Gaussian mixture model, inspired in part by Gaussian Splatting for 4D scene reconstruction. 
In particular, the structure of the model is grounded in an initial process that infers a part-based segmentation of the particle, providing essential inductive bias in order to handle both conformational and compositional variability.  The framework, called \methodName, is shown to reveal biologically meaningful structures on complex experimental datasets, and establishes a new state-of-the-art on CryoBench, a benchmark for cryo-EM heterogeneity methods.

Reconstructing Heterogeneous Biomolecules via Hierarchical Gaussian Mixtures and Part Discovery
Shayan Shekarforoush 1.2 David Lindell Mareus A Brubaker David Fleet
'University of Toronto Vector Institute (shayan, fceet]ecs....
York University mabθeecs. boooeeccsyorru..com
wtr.otttttooooooooooooooooooooo. toronto... f edu
Abstract
Cryo-EM transformational paradigm molecular biology where computa- tional methods are used to infer 3D molecular structure at atomic resolution from extremely noisy 2D electron microscope images. At the forefront research is how model the structure when the imaged particles exhibit non-rigid conforma- tional flexibility and compositional variation where parts are sometimes missing. We introduce novel 3D reconstruction framework with a hierarchical Gaussian mixture model, inspired in part by Gaussian Splatting for 4D scene reconstruction. In particular, the structure of the model is grounded in an initial process that infers a part-based segmentation of the particle, providing essential inductive bias order to handle both conformational and compositional variability. The framework, called CryoSPIRE, is shown to reveal biologically meaningful structures on com- plex experimental datasets, and establishes a new state-of-the-art on CryoBench, a benchmark for cryo-EM heterogeneity methods. Project Webpage.
3
5
7
11
13
Introduction
14
15 Single-particle cryo-clectron microscopy (cryo-EM) is computationally driven experimental paradigm that is transforming molecular biology by enabling 3D structure determination of biomolecules, such as proteins and viruses, at near-atomic resolutions [3, 18, 38.. The core compu- tational task is estimating 3D structure from 2D images with unknown orientation and position,
18
17
19 under extremely low signal-to-noie conditions. Essential their biological function, biomolecules exhibit varying degrees of conformational flexibility, where structures deform non-rigidly, and com- positional variation, where parts of structure may be present in some images and absent others (see Fig. 1). Accordingly, major challenge in cryo-EM is the estimation of 3D structures from 21
23 such heterogeneous data and, to that end, how to infer meaningful representations of structures such parts that capture their heterogeneity. The crux of this challenge is how effectively represent 25 and regularize this variability without overfitting to the noise in cryo-EM images. Existing methods, while encouraging, generally limited in either expressiveness, interpretabilityy efficiency.
27 Here, we propose CryoSPIRE, new method for heterogeneous reconstruction. We leverage
28 part-based Gaussian mixture model (GMM) of 3D density that enables CryoSPIRE to represent
29 both conformational and compositional heterogeneity, unlike some existing deformation-bace methods [13, 33J. Further, it provides naturally interpretable and physically plausible, bast- structure contrast to existing latent variable methods based on linear density subspaces [10, 32) or neural field models [19, 20, 47]. A key challenge with bastd GMMs concerns initialization and
31
33 the discovery of parts. We propose a novel method for part discovery which estimates coarse coarse-grainet grained GMM with per-Gaussian learnalee features (c.f.. [2]) and MLP which defines Gaussian locations 35 and amplitudes. We show that these learned features naturally encode characteristics of structural heterogeneity, which we leverage to infer a part-based segmentation of the structure. Inspired in 37 part by Scaffold-GS [21], we define CryoSPIRE (Scaffold Part-Aware Mixture of Gaussians), a hierarchical model which estimates Gaussian mixture wherein the composition of components
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(A) Particle Stack
(B) Part-based Gaussian Mixture
(C) Structural States
Partial occupancy
Motion
Ga(t
Coleo)
GMM Generator
Gg((2)
(()
Letent Space
(A) Based on a stack of noisy particle images, (B) CryoSPIRE learns part-based Gaussian mixture, with parameters θ, and a latent space representing structural heterogeneity. Given a latent code 2, a generator produces 3D density map. (C) The model supports compositional variability (e.g.. Ge (2())) with missing part), and conformational flexibility (e.g., G)(2(2)) with part deformation).
Figure I:
39 and their deformation are defined terms of a set of anchors, corresponding to parts. The resulting model naturally allows for the arbitrary combination of parts which can both rigidly move and locally deform as a function of an input heterogeneity latent code (see Fig. 1).
41
To our knowledge, this is the first GMM-based model to be successfully benchmarked on Cry- oBench [15], standardized benchmark for cryo-EM heterogeneity with ground-truth labels. In particular, CryoSPIRE outperforms widely used and state-of-the-an methods [10, 19, 32, 33, 47], sometimes by a wide margin. Through ablations, we also validate key design choices, demonstrating the benefits of Gaussian features over positional encoding in DynaMight [40], and highlighting the benefits of hierarchical motion modeling. Finally, on experimental data, CryoSPIRE automatically discovers representations of 3D density maps that correspond to biologically meaningful parts.
43
44
45
46
47
49 To summarize our contributions: we propose a new method enabling part-discovery on 3D biomolec- ular structures based on a coarse-grained GMM. This part-based structure is used to initialize 51 novel, hierarchical GMM-based model for heterogeneous reconstruction with compositional and conformational variability. The resulting framework, CryoSPIRE, establishes new ffae-oof-the-ar on quantitative benchmarks and qualitative experimental datasets.
53
2 Background and Related Work
54
55
Latent Variable Models. Heterogeneous cryo-EM reconstruction methods typically introduce latent variables to represent structural variability of the 3D density map. 3DVA [32] and RECOVAR [10] learn linear subspace represent variation 3D density maps, with clever numerical and regular- ization techniques to optimize high-dimensional basis maps at high spatial resolutions. Nevertheless, model large-scale continuous motion with high dimensional subspace, memory requirements are prohibitive. Much current work has shifted to nonlinear latent models and deep learning [14, 19, 47], with Cryo-DRGN [47] and DRGN-AI [19] using auto-encoders obtain latent codes and conditional coordinate networks [24] to generate density maps. Such aatent-variabl models are hard to interpret, however, as conformational and compositional heterogeneity are not decoupled, and they provide no explicit model of motion between conformational states. By contrast, the latents in 3DFlex [33] encode flow fields that model the conformational deformation of a canonical structure. While resolv- ing detailed motion and improving the quality of density maps, 3DFlex cannot handle compositional heterogeneity, and it is highly sensitive to regularization, often requiring substantial trial and error.
56
57
58
59
60
61
62
63
64
85
66
87
88
GMM-Based Methods. Gaussian mixtures have been used to model 3D density [4, 5, 6, 40]; they provide a sparse, compact representation in which conformation and compositional variability are modeled in terms of positions and amplitudes of Gaussian components. With Gaussian components viewed as atomic primitives, such models also facilitate physics- priors [6, 40] and subsequent molecular model fitting. Nevertheless, existing GMM-based methods fall short in various ways.
89
70
71
72
73
E2GMM [4] and related methods [5, 6] generate GMM parameters with single network, which scales poorly to large numbers of Gaussians. Further, their multi-scale smoothness priors [6] are based on an arbitrary hierarchy which fails to capture part-based structures, thus resorting to manual
75
part masks to resolve and estimate local motions. DynaMight [40] is similar to CryoSPIRE in defining

---

77
an explicit motion model, but it is unable to handle compositional variations, and, as we show, its positional encodings are inferior to our learnable features.
78
79
Gaussian Splatting. Beyond cryo-EM, the effectiveness of GMMs has been demonstrated in 3DD Gaussian Splatting [16, 48], technique which provides fast approximation to the volume rendering integral [8, 23], enabling efficient high-fidelity reconstruction of 3D scenes from multi-view images [11, 17, 22, 44, 45, 46J. 3D Gaussian Splatting represents scene appearance and structure using thousands to millions of Gaussian components, each associated with parameters that control opacity and view-dependent color. CryoSPIRE is in part inspired by Gaussian Splatting [2, 21]. but tailored to cryo-EM, with different image formation model, images with signal-to-noiss ratios less than 5%, and a novel method for part discovery.
80
81
82
83
84
85
86
87
GMM Image Formation. Following [4, 5, 6, 40], we parameterize the terms of a Gaussian mixture with center E RR3, isotropic scale TR, and an amplitude E R:
88
1pp c 2
(()) = Σ m, exp
28
89 for location p E R.3 We transform the GMM into the observation space for the n-th particle image,
90 with a rotation R()) E SO(3) and translation g(r) E R2, followed by an integral projection along 91 the 2 axix of the microscope, to obtain noise-free 2D image, I(p). [4]:
[ (-11
(T( c; + (0 zy
(i)
Σ /2 exp
2s
92 where P E 1 and Hay is an operator discard coordinate of the input position. Cryo-EM images
93 are then convolved with microscope point spread function and corrupted by additive mean-zero 94 Gaussian noise, i(m) = g((") * I(() + c(*), Like other cryo-EM models, the parameters are typically 95 optimized by minimizing squared L2 reconstruction loss between model predictions and observed images. See the supplement for more details on image formation and the image likelihood.
96
3 CryoSPIRE
98
Heterogeneous cryo-EM involves non-rigid 3D reconstruction from noisy 2D images. For such an inverse problem, regularization and inductive bias are key. Local smoothness is a natural choice for regularization, however, smoothness alone not sufficient as nearby regions can deform in somewhat independent ways [33]. Further, the presence or absence of biomolecule parts is not dictated by spatial proximity alone. Macromolecular complexes, like many objects, naturally possess a part-based structure that connects to their compositional and conformational variations. But a coherent 3D part- decomposition is unavailable a priori, and estimating parts from noisy 2D observations is inherently challenging. As consequence, prior work resort to manually designed masks or meshes [25, 33).
99
100
102
104
106
Here, we propose a novel two-stage GMM-baser framework. Given particle images with correspond- ing poses {(T(m) R(,), ) and crude initial 3D structure, we first optimize coarse-grained GMM in which each Gaussian component is augmented with a learnable feature vector (c.f., [2]).
107
108
109
We observe that the learned features encode meaningful information about structural regularities. In particular, Gaussian components that coherently deform or consistently appear or disappear receive similar features, facilitating the inference of part-based segmentation of the particle. Second, based the identified parts and inspired by Scaffold-GG GS [21]. we define part-aware Gaussian mixture model in terms set of anchors, one per part, each with a corresponding set of Gaussians. Optimizing this representation recovers a high-resolution representation of 3D density maps with compositional and conformational variability. In what follows, we describe the part-based hierarchical model, (Fig. 2B-D), followed by the part discovery method and initialization scheme (Fig. 2A).
111
113
115
116
117
3.1 Part-Aware Gaussian Mixture
118 We first specify the form of the part-aware latent-conditioned mixture model; Table 1 provides a summary of the notation used. The model is conditioned on a latent coordinate E 3 C RD for each 120 image, which specifies the state of the macromolecule. The density model itself comprises a set of anchors, each associated with a meaningful part of the macromolecule (Fig, 2B). We parameterize the
3

---

(A) Part Discovery
(B) Hierarchical Model
Feature Space
(C) Anchor-GMM Modulation
Optimization
Projecton
Dlnamaiion
Anchor
Gaussian
Pese
(g
Aschon
Feee
MLP
Ghill Renderer
Renderir
Pren
A=
Dc,
MLP
Projection
Obsavition
(49 mi
Obtjcctives
1
t/
Loss
Stu.
MLP
1,11e11
Clustering
Foaturd Prior Ag. 11/11
Figure 2: Overview of CryoSPIRE. (A) To infer parts, we optimize a coarse GMM with neural networks that generate Gaussian amplitudes and centers, conditioned on image latent codes and Gaussian features. (B) Clustering on learned Gaussian features provides meaningful parts. The CryoSPIRE model comprises anchor and set of Gaussians per part. (C) MLPs generate the higidbooy motion of each anchor (top), per- ver-Gaussian displacements relative to the anchor frames (middle), and Gau-Saasi activations in (0.1) to represent occupancy (bottom). (D) A reconstruction loss compares observed images to 2D projection of the corresponding 3D GMM. Priors encourage bounded latent code and small feature offsets.
Gaussians
Anchors
Particles
Gaussian index
Anchor index
Particle index
C,
Gaussian center
a,
Anchor index of i-th Gaussian
[(]]
Observed image
m,
Gaussian amplitude
Ca
Anchor center
f(0)
Estimated projection
Gaussian scale
J.
Anchor feature
(x)
Particle latent code
Aca
Gaussian center offset
Ra
Anchor rotation
R (m)
Partiele rotation
Af
Gaussian feature offset
ta
Anchor translation
t (n)
Particle translation
Gaussian translation
Table 1: Summary of notations used to denote variables related to Gaussians, anchors or particles.
anchors as, .A {CCa f a) where C R specifies the anchor center location in canonical frame, and f. E 3 C IR E is an associated feature vector that encodes heterogeneity information of its corresponding part. The GMM has G Gaussian components associated with anchors (Fig. 2B, left), denoted by g {ff C2 8;, m2, a;) ] where f, E 3 and a; E f1...., A} specifies the anchor associated with the Gaussian that is set by the part discovery method below.
123
125
127
We parameterize the position and feature embedding of the i-th Gaussian relative to its associated anchor a; as
C₁ Co, + Aci, f.= 10.+40 +Af.
where Ae, R and Af. E E are learnable offsets. We initially set Af; all Gaussians initialized with the features of their corresponding anchors.
129
130
To enable conformational variability, we parameterize deformations at two levels. First, the large- scale motion each anchor frame is parameterized as a rigid body transformation (Fig. 2C, top). Given the latent code for n-th particle image, ((m) E 3, and the anchor feature vector Ja,, we compute the rotated and translated center of the i-th Gaussian, c(())
132
133
(n) Ac, + (Ca: + 7(m)), where (n) 7(m) (n) !e("!
La MLP ((J,, WA),
(4)
where . denotes concatenation, and the MLP with weights W returns a rotation R!!") E SO(3)
135
and translation vector {(7)) TR3 To capture fine-scale flexibility, additional shifts are applied to individual Gaussians (Fig. 2C, middle), i.e.,
196
137
x(n) (()) where E MLP ( (1)) W9)
138 Here, the network MLPC e. with separate weights W generates individual Gaussian displacements, E 1R3 which are smooth Gaussians associated with the same anchor will have similar features.

---

140
Finally, to account for compositional variability, where regions of a density map may be missing, we modulate Gaussian amplitudes (Fig. 2C, bottom), as
141
(n) m, of o!!") where o.t) MLP (ff;, W
(6)
142
Here, MLPS is an MLP with a sigmoid output activation to restrict the modulation to (0, 1). Values m close to 0 and 1, respectively, correspond to inactive (absent) and active (present) Gaussians. Con- sidering both modifications to centers and amplitudes, we obtain a modulated set of 3D Gaussians
144
for n-th particle image, g(m) {(c)"), Si, m ). Gaussian scales remain the same as they control local resolution, a factor independent of structural variability.
145
148
147
We jointly optimize the parameters θ (which includes Gaussian and anchor parameters and MLP weights), and the per-image latent coordinates, Z = (=(")), ). by minimizing the objective (Fig. 2D)
148
1 2(0,2) = N n=1
Σ c (1()), f(o)) + Ao ||
A
Σ I
149
where the reconstruction loss, L, is proportional to the negative image ligllikeli (i.e., the squared error between ((x) and g(*) i((n) where g( is the microscope point spread function and j(m)
is the 2D projection of ((m) from Eq. 2). The second term imposes zero-mean Gaussian prior
152
over the per-image latent codes, ensuring latent coordinates remain bounded [26, 33], while the
153
third term regularizes Gaussians to remain close to the anchor in the feature space. x₂ and xy are
154
hyperparameters that control the relative strength of these priors.
155
3.2
Part Discovery for Model Initialization
156
The part discovery process is illustrated in Fig. 2A. We optimize coarse-grained model without anchors and with fewer Gaussians, similarly parameterized as 9 {[f., C, 8,, m.)) Here, the Gaussian features, f., are directly learnable parameters (and randomly initialized). We use MLP (Eq. 5), to shift Gaussian centers and MLPS m (Eq. 6) to modulate Gaussian amplitudes. The parameters are estimated using the L.2 reconstruction loss and the latent prior, similar to the objective in Eq. 7. Once optimized, we find that the feature space naturally groups Gaussians into 3D parts that undergo consistent motion or appear and disappear together. Remarkably, this property emerges without any direct supervision on features.
157
158
159
160
162
184 To obtain parts, we apply clustering on the Gaussian features, thereby finding regions with reasonably consistent motion and presence. We then further divide these clusters by clustering in 3D space to ensure large parts are well-covered with anchors. For clustering we simply use k-means++ [1]. We use the position and feature vector of the Gaussian closest to the centroid of the cluster to initialize the anchor set, A = {(Ca. T From the coarse- coarse-grained model, we also compute an improved density map which is used to seed the Gaussians of the part-aware model. This provides a more robust initialization, especially in the presence of large-scale motion which can lead to blurred or over-dispersed density. Lastly, the coarse-grained model provides preliminary estimate of the image latent codes, which are used to initialize latent codes in the part-aware model.
185
186
167
168
169
170
172
173
Remark. Methods for 4D scene reconstruction [27, 31], and DynaMight [40] in cryo-EM, often use neural networks to output deformations or motion. However, they condition on positional encodings of input coordinates instead of learnable features. Such fixed conditioning strongly biases deformations to be spatially smooth, whereas our approach with learnable feature space enables a more flexible form of piecewise smoothness, allowing nearby parts to move quite differently. Through an ablation study, we show that positional encodings quantitatively underperform as well.
174
175
176
177
178
Experimental Setup
179
We quantitatively compare CryoSPIRE with the state-of-the-ar methods, namely. RECOVAR [10], CryDRRGN [47], DRGN-AI [19], 3DFlex [33] and 3DVA [32] using the CryoBench benchmark [15].
181
182
We also provide qualitative results on experimental datasets.
183
CryoBench. The sole benchmark for cryo-EM heterogeneity is CryoBench [15], a set of synthetic
datasets with trouh labels and protocol for quantitative evaluation. Two datasets, IgG-1D
5

---

IgG-ID
IgG-RL
Ribosembly
Method
Mean (std)
Med
Mean (std)
Med
Mean (sid)
Med
3D Classification [39]
0.297 (0.019)
0.291
0.309 (0.01)
0.307
0.289 (0.081)
0.288
CryoDRGN [47]
0,366 (0.003)
0,366
0.349 (0.008)
0.348
0.415 (0.019)
0.415
CryoDRGN-AI-fie [19]
0,366 (0.001)
0,366
0.355 (0.007)
0.354
0.372 (0.032)
0.374
3DFlex [33]
0.336 (0.002)
0.336
0.339 (0.007)
0.339
3DVA [32]
0.351 (0.003)
0.351
0.341 (0.006)
0.341
0.375 (0.038)
0.372
RECOVAR [10]
0.391 (0.001)
0.391
0.372 (0.008)
0.371
0.430 (0.016)
0.432
CryoSPIRE (ours)
0.402 (0.002)
0.402
0.386 (0.014)
0.389
0.427 (0.014)
0.424
Table 2: Mean (standard deviation) and median of AUC of Per-Conformation FSCs on CryoBench datasets [15]. Statistics computed across different structural states, i.e. 100 for IgG-ID and IgG-RL and 16 for Ribosembly (Best method in bold, second best underlined).
185
and IgG-RL, are based on the human immunoglobulin G (IgG) complex, simulating conformational changes by rotating the dihedral angle between the Fab domain and the IgG core (see Fig. 4D), generating 100 distinct conformations, each with 1,000 particle images. Ribosembly simulates compositional heterogencity by successively adding protein subunits and ribosomal RNA. resulting in 16 discrete structural states [35]. It has 335,240 particle images, with non- non-uniform distribution over the 16 compositional states. All particle images have 128 x 128 pixels, and are simulated with realistic point spread functions and a signal-to-noie ratio (SNR) of 0.01.
187
188
189
191
192
Experimental Datasets. We also evaluate on two real datasets: EMPIAR-10076 is dataset comprising assemblies of intermediates of the Escherichia coli large ribosomal subunit (LSU) [7], with 131,899 particle images (320 x 320 pixels, with pixel size 1.31 Å). In the original study, four major assembly states were identified [7]. with a subset of particles labeled as unassigned (non-oibosomal impurities) or 30S subunits. We also consider EMPIAR- 1 80, a conformationally heterogeneous dataset of Pre-Catalyte Spliceosome [30]. A total of 327, 490 particle images were collected (320 x 320 pixels, with pixel size 1.69 Å). Consistent with other heterogeneity methods considering this dataset [10, 47] we perform analysis on a filtered subset of 139,722 images.
193
194
195
196
197
198
199
Implementation Details. For part discovery, we seed G = 2,048 components using the rigid reconstruction and adopt lightweight MIPs with a single hidden layer of H 32 units. The latent space, 3, has dimensionality 4=4 and the feature space, 3. has dimensionality 24 We optimize the part discovery model for 15 and 50 epochs on synthetic and experimental datasets. The part-aware GMMs are optimized for 30 epochs, using = 8,19 components, except for Ribosome synthetic and experimental datasets with G 16,384, and have MLPs with three hidden layers and Hl 128 hidden units, On experimental datasets, we perform part discovery on downsampled 128 x 128 images for efficiency, while the part-aware GMM is optimized on 256 x 256 images. We use batch size B= 64 and set hyperparameters A = 0.1, Ay 0.01. The optimization runs on a single NVIDIA GeForce RTX 2080, taking 3 to 6 hours depending on the number of Gaussians in the model.
200
202
203
204
205
206
207
208
209
210
Evaluation Metrics. The quality of cryo-EM density maps are evaluated using Fourier Shell Corre- lation (FSC) [43], which is the normalized cross coos--correlation between two independently estimated density maps, as a function of frequency. Metrics for heterogeneity are less standardized, but the most common Por-Confornation FSC (or Per-Conf FSC) [15], proposed by CryoBench [15]. Per-Conf
212
213
214
FSC is the average FSC between the ground-truth 3D structure of a particle state, and the 3D structure corresponding the average latent position of images associated with that state. The Per-Conf FSC requires knowledge of ground-truth 3D structures for each image which is not available for experimental data and we instead rely on qualitative evaluation of the estimated parts and structures.
215
216
217
218
FSC results in curve which can be summarized by computing the area under the curve (AUC) to more easily compare methods. See the supplement for more details on metrics.
219
5
Results
220
221
Quantitative comparison on the three relevant CryoBench [15] datasets are provided in Table 2 and
Fig. 3. Note that CryoSPIRE outperforms 3DVA and 3DFlex which are among the most widely
223 used methods in cryo-EM&a at present. As 3DFlex cannot handle compositional changes, was not
evaluated on Ribosembly. CryoSPIRE outperforms non-linear latent variable models, Cryo-DRGN

---

1gG-1D
IgG-RL
Ribosembly
Corretation
02-
CryoSMREE CryoORGh
3DMA
RECOVAR
CryOSRIRE CryоBRGR
30WA
RECOWAR
CryoSRIRE CryoBRIN
RECOWAR
3DVA
DRONAI
30F ex
DRON-N
3011ce
DMIONAU
0.0
3.0
20.3
14.9
10.0
7.5
60
14.9
10.5
7.5
6.0
10.0
Resolution (A)
Resolution
Resolution
(A)
Figure 3: Per-Conformation FSC on CryoBench datasets. Error bars indicate standard deviation across different conformations. The highest possible resolution is 6 À these synthetic datasets.
A
C
Feature Space
Final Reconstructions
Feature Clustering
Spatial Clustering
Cluster
Archer Archer
Cluster
Rigd Reconetrution
Arstur Archar
Discovery
Amp. Modulation
3D Gaussians
Flexible Fab
Helerogeneily Latent Space
180'
Goursion
Inactive
LhMP2
Active
270
Rigid Core
III
IV
Feature Clustering (2 clusters)
Spetial Clustering (5 clusters)
IV
Figure 4: Results IgG-1D [15]. (A) Due to large motion, the Fab domain (circled) is smeared in rigid reconstruction, while our part discovery model identifies this domain and resolves its structure and motion, providing good initialization for subsequent modeling. (B) For sample structure, the histogram of amplitude modulations indicate active and inactive Gaussians. (C) Gaussian feature space, 3, shows two distinct groups (green, orange), corresponding to the flexible Fab and the rigid core; feature clustering finds these groups and divides further based on spatial proximity, yielding 5 parts. (D) Configuration of 3D Gaussians after Level-1 and Level-2 clustering. (E) The latent space, 3. captures conformation change (colored based on ground truth Fab orientation). (F) Sample structures from model corresponding to four latent points, showing rotation of the Fab domain (green).
and DRGN-AL especially IgG-ID and IgG-RL by large margin. The most competitive method is the linear subspace model of RECOVAR, which, as reported, is memory intensive due to allocation of several bases and is not interpretable without motion modeling. While CryoSPIRE significantly outperforms RECOVAR on IgG datasets, its performance on Ribosembly, where linear subspace models more favorable by design, is not statistically different from RECOVAR. Relative the
226
228
230
nominal FSC threshold of 0.5 for comparison to ground truth [36], the FSC curves in Fig. indicate
that CryoSPIRE finds higher resolution density maps. Finally, note that CryoSPIRE is the first GMM-based method to be successfully evaluated on CryoBench.
232
IgG-1D & IgG-RL (CryoBench [15]). The flexible Fab domain (circled in Fig. 4A, top) in the rigid reconstruction, used as input for part discovery, is poorly resolved. However, the part discovery model Iearns to selectively deactivate incoherent parts, shown in the histogram of the modulation
234
(n) Fig.4B. This enables more robust initialization (Fig. 4A, bottom) the hierarchical GMM. The Gaussian feature space, B. shows two clusters corresponding to the flexible Fab domain
236
factors o3
238
from the rigid core (Fig. 4C for IgG- ID and Fig. 5B for IgG-RL.). Spatial clustering produces a total
of five and six anchors for IgG-ID ID and IgG-RL, respectively. The latent heterogeneity space, 3.
7

---

Feature Space
Heterogeneity Space
C
Reconstructions
Feature Clustering
Spotial Clustering
200
IV
III
Cluster Cluster
Ancher
Ancher
230
Dihaeral Ang
209
104
150
III
S1"
Archorr
20 15
U0440-1
Figure 5: Results on IgG-RL [15]. (A) The feature space, 8. shows two parts (green and orange) corresponding to the flexible Fab domain and the rigid core. Subsequent spatial clustering yields six parts. (B) The latent space, 3. is colored with Fab orientation along with four sampled latent points that capture rotation of the Fab domain (comprising three parts). The motion of the Fab domain in IgG-RL is not as regular as that in IgG- ID, as reflected in the latent space. (C) The corresponding density maps are provided with parts illustrated in different colors.
A
Heterogeneity Latent Space
Final Reconstructions
20
State (2,3)
State 3 (2,3,5)
State 4 (2,3,4)
State (2.3,5.6)
10
State 16
-10
10
20
30
40
50
UMAP
B
Feature Space
State (1.2,3.445.667.))
State (1,2,3,45566.))
State (1,2,3.4556)
UMAP-1
Figure 6: Results on Ribosembly [15] (A) Gaussian feature space, 3. showing eight major parts identified through clustering. (B) Heterogeneity latent space, 3, colored coded with the ground-truth compositional state. (C) Visualizations of 3D density maps corresponding to seven points in latent space, with colors depicting parts (given in parentheses).
indicates circular manifold of dihedral angles for IgG-ID, see Fig. 4D. Four structures from the latent space in both datasets demonstrate that the Fab domain, covered by a few parts, undergo a large, predominantly rigid motion, while the rest of the complex remains fixed.
241
243
Ribosembly (CryoBench [15]). After part discovery, we obtain eight parts (see Fig. 6A) that are used to initialize eight anchors in the part-aware GMM. In Fig. 6B the learned latent space, 3. clearly distinguishes between the different compositional states. For seven selected states, we visualize the estimated structure (Fig. 6C), colored based on the discovered parts.
245
247
Large Ribosomal Subunit (EMPIAR- 10076 [7]). We find four major assembly states in the part discovery latent space (labeled as (I. II, V, VI) in Fig. 7A, left), which match classes (C, E, B, D) in the original study [7], with unassigned particles and 30S contaminants grouped in states III and IV, which are excluded when optimizing hierarchical model (See supplement for more details). The
248
249
250
251
Gaussian feature space, 8. (Fig. 7B) shows four distinct parts which also align with previously
252
reported structural blocks in the original study (cf. [7], Fig. 6). By analyzing the heterogeneity latent space, 3. of the part-aware model (Fig. 7A, right), we show that the major states can be further divided into subpopulations; e.g., the major state is represented with minor states (1, 2) and the major state has branched into minor states (3, 4, 5). The associated structures, shown in Fig. 7E, consistent with minor states reported in the original study [7].
253
255

---

A
Heterogeneity Latent Spaces
Major
Major Vi
Maar
Minor
Minor
Minor
Minor
Maze
MajorV VI
-23
B
Major
Major V + Minor 6
Feature space
+
A Minor 3
Minor 4
Minor 5
Ancher
UMPP
5
Anther
Anchar
Anchor 4
UMAP-1
10
15
Figure 7: Results on Large Ribosomal Subunit (EMPIAR-11076 [7]). (A) The learned heterogenity latent spaces, 3. in part discovery (left) identifies the four major assembly states (I. II, V, VT) and two groups of impurities (III, IV). After fitting the part-aware model, the major states, with impurities excluded, can be further categorized (right) into eight color-coded minor structural states. (B) The part discovery Gaussian feature space, 8. reveals four parts which are used to construct the part-aware model. (C) The structures corresponding to different states, colored by inferred part.
A Heterogeneity Latent Space
Trajectory
Tapehory 2 2
10
PCA-1 i a
PCA-2 0
-30
PCA-1 -10
PCA-1 0
PCA-1 +10
-35
B
15
Feature Space
SF3b
Trajectory
Helicase
Archer
Aether
Body
LMMP-1
Foot
PCA-2 -10
PCA-2 D
PCA-2= +10
Figure 8: Results on Pre-Catalytic Spliceosome (EMPIAR-10180 [30D. (A) PCA of the latent space, 3. is used to generate two structural trajectories. (B) The Gaussian feature space, 8. shows four parts which correspond to known helicase, SF3b, body and foot domains as shown in 3D visualization of Gaussian components configuration. (C) Three states along each trajectory. In both trajectories, body is rigid while SF3b and helicase show large-scale motion.
257
Pre-Catalytic Spliceosome (EMPIAR-1012) [30]. The feature space, 3. of the part discovery model (Fig. SB), shows four distinct clusters, which correspond coherent structural regions foot, body, helicase, and SF3b consistent with the original study [30]. Accordingly, we optimize the part-aware model with four anchors. To illustrate structural variability, we run PCA on the heterogeneity latent space, 3, and extract two principal directions illustrated in Fig. 8A. Top views of density maps along the two principal directions (Fig. 8C) show two modes of conformational heterogeneity. The first direction reflects
258
259
260
261
262
sawward-backwad rotation of the SF3b and helicase regions. The second direction captures a side-to-side rotation of SF3b, and diagonal shift of the helicase. Please see the supplement for more visualization on conformational changes.
264

---

ovensogmaniiaion
we hierarchical motion
ours
Method
1gG-1D
IgG-RI.
Ribosembly
CrysSPIRE
0.402 (0.0012)
0.386 (0.014)
0,427 (0.014)
w/o hier, motion
0.388 (0.002)
0.372 (0.010)
0.255 (0.015)
over-segment
0.384 (0.002
0.375 (0.010)
0.423 (0.016)
w/ pos. encoding
0.377 (0.002)
0.361 (0.007)
0.415 (0.023)
Figure 9: Estimated motion of Gaussians for 30 states of IgG- ID. The baselines fail to cap- ture local rigidity.
Table 3: Mean AUC-FSCs reported on datasets from CryoBench [15] for ablation study.
288
5.1
Ablation Study
267
Here, we ablate key design decisions in our framework. To demonstrate the importance of anchor- based motion modeling in CryoSPIRE, we consider baseline without anchors that uses an MIPP to directly learn deformations of individual Gaussians. Quantitative comparison on IgG-ID and IgG-RL, in Table shows that the lack of anchor based motion leads inferior results. This is less critical for Ribosembly with minor conformational changes. We also compare with model that over-segments the structure by using K = 64 anchors, which achieves worse performance. In Fig. 9, we visualize the estimated motion of Gaussians on the IgG-ID dataset. Both baselines fail to capture the locally rigid and smooth motion. Finally, we consider a baseline where the Gaussian feature space is replaced with a positional encoding, similar to previous methods, e.g., [40]. This baseline is unable identify meaningful parts and achieves inferior quantitative performance.
268
269
271
272
273
274
275
278
6
Conclusion
278
We present CryoSPIRE, a hierarchical cryo-EM density model to capture conformational and compo- sitional heterogeneity in the 3D structure of biomolecules from 2D images. This includes novel method for part discovery and a hierarchical Gaussian mixture model for which the parts provide meaningful inductive biases to regularize model fitting. CryoSPIRE establishes new state-of-the-an art on the CryoBench heterogeneous benchmark, and produces meaningful parts on experimental data.
280
282
283
While CryoSPIRE shows promising results, limitations exist. First, validation of estimated structures and variability from heterogeneous experimental data remains an open problem for all methods, including CryoSPIRE. Second, interpreting the inferred latent space remains challenging, specifically how it may relate to the biophysical energy landscape of molecular states. Third, learning per- Gaussian features is a key design choice in cryoSPIRE, it provides the inductive bias that drives features to encode local structural heterogeneity. To that end, we have only used very simple algorithms like k-means++, which requires manual selection of the number of clusters (parts). Further research will be useful to find more effective forms of clustering, perhaps incorporating principled biophysics criteria like free energy. Finally, like other methods, presume initial estimate of the structure and image poses; inaccuracies in these may limit CryoSPIRE's s efficacy. A fully ab initio method for heterogencous data remains open problem.
284
286
286
290
292
Broader Impact
294
295
Cryo-electron microscopy (cryo-EM) has emerged as a revolutionary technique in structural biology. enabling the determination of macromolecular structures with significant societal impact. Computa- tional methods, grounded in machine learning and computer vision have now been used to determine many thousands of biological structures. Notably, cryo-EM played pivotal role in elucidating the structure of the BARS-CoV.. spike protein, revealing its pre-fusion conformation and aiding in the
296
297
298
299
300
assessment of medical countermeasures. Complementing computational methods such as AlphaFold
301
for protein structure prediction, cryo-EM has revolutionized our understanding of cellular processes
and accelerated the development of novel therapeutics, including synthetic antibodies. Nevertheless,
303
we strongly condemn any usage of our proposed hierarchical 3D GMM representation for generating malicious data, improperly modifying signals, or spreading misinformation.
10

---

Acknowledgments and Disclosure of Funding
305
306 We thank Ali Punjani and John Rubinstein for helpful discussions. This research was supported in part by the Province of Ontario, the Government of Canada (through NSERC, CIFAR, and the Canada First Research Excellence Fund for the Vision: Science to Applications (VISTA) programme), and
307
308
by the companies sponsoring the Vector Institute for Artificial Intelligence.
References
311
[1] David Arthur and Sergei Vassilvitskii. K-Means++: The advantages of careful seeding. Proc. 18th Annual ACM-SIAM Symposium on Discrete Algorithms, page 1027-1035, 2007. 5,
312
313
[2] Jeongmin Bae, Scoha Kim, Youngsik Yun. Hahyun Lee, Gun Bang, and Youngjung Uh. Per- Gaussian embedding- based deformation for deformable 3d Gaussian splatting. In European Conference on Computer Vision (ECCV), pages 321-335, Springer, 2024. 1,
314
315
316
[3] Marcus A Brubaker, Ali Punjani, and David J Fleet. Building proteins in day: Efficient 3d molecular reconstruction. Proc IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2015.
317
318
319
[4] Muyuan Chen and Steven Ludtke. Deep learningbbased mixed-dimensional Gaussian mixture model for characterizing variability in cryo-EM. Nature Methods, (8(8):930-936) 2021. 2, 3
320
321
[5] Muyuan Chen, Michael F. Schmid, and Wah Chiu. Improving resolution and resolvability of single particle cryoem structures using Gaussian mixture models. Nature Methods, 21:37-40. 2023. 3
322
323
324
[6] Muyuan Chen, Bogdan Toader, and Lederman Roy. Integrating molecular models into cryoem heterogeneity analysis using scalable high-resolution deep Gaussian mixture models. Journal of Molecular Biology, 435(9):16801) 2023. 2,
325
326
327
[7] Joseph H Davis, Yong Zi Tan, Bridget Carragher, Clinton S Potter, Dmitry Lyumkis, and James R Williamson. Modular assembly of the bacterial large ribosomall subunit. Cell, 16166):160-1162.
328
2016. 6, 8, 2,
330
[8] Robert A Drebin, Loren Carpenter, and Pat Hanrahan. Volume rendering. ACM SIGGRAPH. 22(4))65-74, 1988. 3
331
332
99] Frank and L. AI-Ali. -igial-to-noise ratio of electron micrographs obtained by cross correla- tion. Nature, (56(5516):376-799 1975. 3
333
334
[10] Mare Aurèle Gilles and Amit Singer. Cryo-EM heterogeneity analysis using regularized covari-
335
ance estimation and kernel regression. Proc. National Academy of Science, 122(9):22111441122.
336
2025. 1, 2, 5, 6, 4
337
[11] Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned Gaussian splatting for efficient
338
3D mesh reconstruction and high-quality mesh rendering. In Proc. CVPR, 2024. 3
339
[12] G. Harauz and M. van Heel. Optimal determination of particle orientation, absolute hand, and contrast loss in single-pattclle electron cryomicroscopy. Optik, 73:146-156, 1986. 3
340
341
[13] David Herreros. Roy R Lederman, James M Krieger, Amaya Jiménez-Moreno, Marta Martínez, David Myška, D Strelak, J Filipovic, Carlos OS Sorzano, and José M Carazo. Estimating confor- mational landscapes from cryo-em particles by 3d zernike polynomials. Nature Communications, 14(1):154, 2023.
342
343
344
345
[14] Yue Huang, Chengguang Zhu, Xiaokang Yang, and Manhua Liu. High-resolution real-space reconstruction of cryo-em structures using neural field network. Nature Machine Intelligence, 6:892-903, 2024. 2
347
348
[15] Minkyu Jeon, Rishwanth Raghu, Miro Astore, Geoffrey Woollard, Ryan Feathers, Alkin Kaz, Sonya M. Hanson, Pilar Cossio, and Ellen D. Zhong. CryoBench: Diverse and challeng- ing datasets for the heterogeneity problem in cryo-em. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 2, 5, 6, 7, 8, 10, 3,4
350

---

352
[16] Bernhard Kerbl, Georgios Kopanas. Thomas Leimkuehler, and George Drettakis. 3D Gaus- sian splatting for real-time radiance field rendering. ACM Transactions on Graphics (Proc. SIGGRAPH), 2023. 3,
353
354
355
[17] Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Yang-Che Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3D Gaussian splatting as Markov chain Monte Carlo. In Proc. NeurlPS, 2024. 3
356
357
358
[18] Werner Kühlbrandt. The Resolution Revolution: Advances in detector technology and image processing are yielding high-resolution electron cryo-microscopy structures of biomolecules. Science, (0316617):1443-111
359
360
2014.
361
[19] Axel Levy, Michal Grzadkowski, Frederic Poitevin, Francesca Vallese, Oliver B Clarke, Gordon Wetzstein, and Ellen D Zhong. Revealing biomolecular structure and motion with neural ab initio cryo-em reconstruction. bioRxiv, 2024. 1, 2,2,5 5, 6, 4
362
383
384
[20] Axel Levy, Gordon Wetzstein, Julien NP Martel, Frederic Poitevin, and Ellen Zhong. Amortized inference for heterogeneous reconstruction in cryo-EM. Advances in Neural Information Processing Systems, 35: 15:13032 13038-33049, 2022.
365
366
367
[21]
Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold- gs: Structured 3d Gaussians for view-adaptive rendering. In Proc IEEE/CVF Conf Computer Vision and Putlern Recognition (CVPR), pages 20654-20664, 2024. 1,3
389
370
[22] Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan, Dynamic 3D Gaussians: Tracking by persistent dynamic view synthesis. In Proc. 3DV, 2024. 3
[23] Nelson Max. Optical models for direct volume rendering. IEEE Trans. Vis. Comput. Graph., 1(2)999-08,, 2002. 3
373
374
[24]
Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoor- thi, and Ren Ng. NERF: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106, 2022.
376
377
[25]
Takanori Nakane and Sjors HW Scheres. Multi-body refinement of cryo-em images in relion. In CryoEM: Methods and Protocols, pages 160- Springer, 2020. 3
378
379
[26] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proc IEEE/CVF Conf Computer Vision and Pattern Recognition (CVPR), pages 165-174, 2019. 5
380
382
[27] Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz. and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proc IEEE/CVF International Conference on Computer Vision (CVPR), pages 5865-5874, 2021. 5
383
384
385
[28] Pawel A. Penczek. Three-dimensional spectral signal-to-noise ratio for a class of reconstruction
386
algorithms. Journal of Siructural Biology, 138(1):34-46, 2002. 3
387
[29] Eric F Pettersen, Thomas D Goddard, Conrad C Huang, Elaine C Meng, Gregory S Couch, Tristan I Croll, John H Morris, and Thomas E Ferrin. Ucsf chimerax: Structure visualization for researchers, educators, and developers. Protein science, 30(1):70-82, 2021.
388
389
[30] Clemens Plaschka, Pei-Chun Lin, and Kiyoshi Nagai. Structure of a pre-catalytic spliceosome. Nature, 1421, 2017. 6, 9, 7,8
381
392
[31] Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR), pages 00318-10327 2021. 5
394
395
[32] Ali Punjani and David J Fleet. 3D Variability Analysis: Directly resolving continuous flexibility and discretcheterogeneity from single particle cryo-EM images. Journal of Structural Biology. 2 103:10770,, 2021. 1, 2, 5, 6, 4, 7,
397
398
[33] Ali Punjani and David J Fleet. 3DFlex: Determining structure and motion of flexible proteins from cryo-em. Nature Methods, 20:860-870, 2023. 1, 3, 5, 6, 4, 7,8
12

---

400
[34] Ali Punjani, John L Rubinstein, David Fleet, and Marcus A Brubaker. CryoSPARC: Algorithms for rapid unsupervised cryo-em structure determination. Nature Methods, 14:290-2900 2017.
7
[35] Bo Qin, Simon M Lauer, Annika Balke, Carlos H Vieira- Vieira, Jörg Bürger, Thorsten Mielke, Matthias Selbach, Patrick Scheerer, Christian MT Spahn, and Rainer Nikolay, Cryo-em captures early ribosome assembly in action. Nature Communicationss 14(1):898, 2023. 6
404
[36] P. B. Rosenthal and R. Henderson. Optimal determination of particle orientation, absolute hand, and contrast loss in single-particle electron cryomicroscopy. Molecular Biology,
407
333(4):721-745, 2003. 3
409
[37] Sjors W. Scheres. RELION: Implementation of a Bayesian approach to cryo-em structure determination. Journal of Structural Biology, 180(3):519 530, 2012. 2
[38] Sjors H W Scheres. Processing of structurally heterogeneous cryo-em data in RELION. In Crowther, editor, The Resolution Revolution: Recent Advances In cryoËM, volume 579 of Methods in Enzymology, pages 125-157, Academic Press, 2016.
412
[39] Sjors H W Scheres, Haixiao Gao, Mikel Valle, Gabor T Herman, Paul P B Eggermont, Joachim Frank, and Jose-Maria Carazo. Disentangling conformational states of macromolecules in 3D-EM through likelihood optimization. Nature Methods, 4(1):27-29, 2007. 6, 4
415
[40] Johannes Schwab, Dari Kimanius, Alister Burt, Tom Dendooven, and Sjors H. W. Scheres. DynaMight: Estimating molecular motions with improved reconstruction from cryo-em images.
418
419
Nature Methods, 21: 21:1855-1862, 2024. 2, 3, 10
420
[41] Amit Singer and Fred J Sigworth. Computational methods for pangle-particle electron cryomi- croscopy. Annual review of biomedical duta science, 3(1):163-190 2020.
[42] M. Unser, B. L. Trus, and A. C. Steven. A new resolution criterion based spectral signal-to- noise ratios. Ultramicroscopy, 23(1);39-51 1987. 3
423
424
[43] Marin Van Heel and Michael Schatz. Fourier shell correlation threshold criteria. Journal of structural biology, 151(3)2250-25) 2005. 6,3
[44] Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D Gaussian splatting for real-time dynamic scene rendering. In Proc. CVPR, 2024. 3
427
428
429
[45] Ziyi Yang, Xinyu Gao. Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3D Gaussians for high-fidelity monocular dynamic scene reconstruction. In Proc. CVPR, 2024.
431
432
[46] Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3D Gaussian splatting. In Proc. CVPR, 2024.
[47]
Ellen D. Zhong, Tristan Bepler, Bonnie Berger, and Joseph H. Davis. CryoDRGN: Reconstruc- tion of heterogeneous cryo-em structures using neural networks. Nature Methods, 18:1 18:175-185,
435
436
2021. 1, 2, 5, 6, 4, 8
[48] Matthias Zwicker, Hanspeter Plister, Jeroen Van Baar, and Markus Gross. Ewa splatting. IEEE Transactions on Visualization and Computer Graphics, (33):223-333) 2002. 3
13

---

NeurIPS Paper Checklist
439
1.
Claims
441
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
442
443
Answer: [Yes]
Justification: We clearly state the claims in the abstract, summarize them in list at the end of introduction section, and provide empirical evidence in the results.
445
448
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
448
449
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
450
451
NA answer to this question will not be perceived well by the reviewers.
452
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
453
454
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
458
2.
Limitations
457
Question: Does the paper discuss the limitations of the work performed by the authors?
458
Answer: [Yes]
459
Justification:
We discuss limitations in the conclusion.
Guidelines:
461
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
462
463
The authors are encouraged to create a separate "Limitations" section their paper.
464
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
466
488
489
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
470
471
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
475
475
477
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
478
479
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
480
481
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
483
485
486
487
3. Theory assumptions and proofs
488
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
489
Answer: [NA]
14

---

491
Justification:
We provide no theoretical results.
492
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
495
496
All assumptions should be clearly stated or referenced in the statement of any theorems.
497
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
499
500
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
503
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
505
508
507
Answer: [Yes]
508
Justification: In a subsection called implementation details, we included all details needed to reproduce the results.
510
Guidelines:
511
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
512
513
515
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
516
517
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
518
519
520
521
522
523
524
525
528
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
527
528
529
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily
530
model architecture, the paper should describe the architecture clearly and fully.
532
533
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
534
535
537
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
538
539
510
541
512
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
544
15

---

548
Answer: [No]
547
The datasets are publicly available and we promise to release the code in future.
Justification:
548
549
Guidelines:
The answer NA means that paper does not include experiments requiring code.
551
Please see the NeurIPS code and data submission guidelines (uttps://nips.. ec/
552
Peioooooyiii olicyy for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simxply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
554
556
557
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //n /nips. cc/ ccc/public/pee httg/guidds/ooeeee
S58
sssisinn asssnnolicy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
581
562
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
563
564
585
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
586
567
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
568
569
6. Experimental setting/details
570
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
571
572
573
Answer: [Yes]
Justification:
We provide all the necessary information in the experimental setup section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
578
The full details can be provided either with the code, in appendix, as supplemental material.
580
581
7. Experiment statistical significance
582
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
583
584
Answer: [Yes]
585
Justification:
We provide various statistics the quantitative metrics.
586
Guidelines:
587
The answer NA means that the paper does not include experiments.
588
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
590
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
582
594
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
585
The assumptions made should be given (e.g., Normally distributed errors).
16

---

597
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
596
589
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
600
601
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
603
605
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8.
Experiments compute resources
608
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
609
610
611
Answer: [Yes]
612
Justification:
We mentioned the GPU resources used to run the proposed method.
613
Guidelines:
614
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
616
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
618
619
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't It make it into the paper).
620
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: / /ttp///eerrp..eeee www.errpp.ccppuuiiiiiieececeeeee htte/public/Eetee /ttpppiceehcccccuuueeelllllll ctw.idellclllleeccccccccccc ines?
624
625
Answer: [Yes]
Justification:
We conform with the NeurIPS Code of Ethics.
Guidelines:
628
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
829
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
631
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
635
636
Answer: [Yes]
Justification:
We discuss societal impacts in the supplement.
Guidelines:
639
The answer NA means that there is no societal impact of the work performed.
840
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
642
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
644
17

---

648
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
647
649
650
651
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
654
655
658
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
658
680
681
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
663
Answer: [NA]
686
Justification: The paper poses no such risks.
687
Guidelines:
668
The answer NA means that the paper poses no such risks.
669
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
670
671
672
673
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
676
677
12. Licenses for existing assets
679
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
680
682
Answer: [Yes]
683
Justification: We mention the benchmark CryoBench throughout which the comparison with other methods is provided.
Guidelines:
686
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
689
890
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
692
689
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
694
685
696
697
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

699
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
700
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
703
704
Answer: [NA]
705
Justification:
We don't release any new assets.
Guidelines:
707
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
709
The paper should discuss whether and how consent was obtained from people whose asset is used.
712
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
714
715
14.
Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
717
Answer: [NA]
720
Justification:
No crowdsourcing or research with human subject.
721
Guidelines:
722
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
725
728
727
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
729
730
15.
Institutional review board (IRB) approvals or equivalent for research with human subjects
732
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
734
735
Answer: [NA]
737
Justification:
No crowdsourcing or research with human subject.
738
Guidelines:
739
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
742
744
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
746
747
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

750
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Question:
751
753
Answer: [NA]
Justification: The core method development in this research does not involve LLMs.
756
Guidelines:
757
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
759
Please refer to our LLM policy (https htt////eurpppp.pp http:///eeeppp....oooeeeeeeeeeee cct//ooeeeeeeceee/2222333 for what should or should not be described.
20


Reviews:
[{"review_id": "4hXxbUtODJ", "reviewer": "Reviewer_iKnc", "summary": "CryoSPIRE proposes a two-stage cryo-EM workflow:  \n1. A coarse Gaussian mixture is fitted to 2-D particles; the learned per-Gaussian feature vectors are clustered to identify “parts”.  \n2. Each part is assigned a rigid transform predicted by an MLP from a latent variable $z$, while finer per-Gaussian offsets and amplitudes model local flexibility.  \n\nAll parameters and per-image latents are optimized end-to-end with an $\\ell_2$ image loss plus Gaussian priors. Experiments on the CryoBench synthetic benchmark and several real-world complexes are reported.", "questions": "1. How does performance degrade if the initial orientations are perturbed by 5°–10°?  \n2. Why is the number of parts set to 32 for IgG and 48 for ribosome? Did you try an information-criterion or cross-validation scheme?  \n3. Can you report gold-standard FSC, local resolution maps, or atomic‐model fit statistics rather than visual montages?  \n4. Why are state-of-the-art heterogeneous refinements such as AlphaCryo4D, Bayesian multi-body, and CryoFIRE missing from Table 1?  \n5. What GPU memory and runtime would be required for a 1 Å map with $G>10^5$ Gaussians and $10^6$ particles?", "limitations": "* The method is *not* ab initio; it inherits any alignment and model bias from the initial rigid reconstruction.  \n* Successful runs require manual tuning of part count, latent size, regularization strength, and MLP width—parameters that vary across datasets with no automatic selection rule.  \n* Synthetic benchmarks dominate the results; real complexes lack objective resolution or heterogeneity metrics, so claims of biological relevance are unsubstantiated.  \n* Several competitive heterogeneous-reconstruction methods are omitted, leaving it unclear whether the reported gains hold under fair conditions.  \n* The paper demonstrates feasibility only on modest particle sets and Gaussian counts; no experiment approaches the data volumes typical of modern cryo-EM studies.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 1, "strengths_and_weaknesses": "1. The high-level idea—hierarchical Gaussian mixtures with latent-conditioned rigid motions—extends earlier neural-field approaches only marginally. The “part discovery” step is effectively a k-means clustering of per-component features; no principled criterion or unsupervised objective is introduced. \n2. The pipeline relies on *pre-estimated* orientations and a good initial rigid map; errors in either propagate unchecked. The latent prior is isotropic Gaussian even for strongly anisotropic motions, and no evidence is provided that the MLP can learn realistic energy landscapes. \n3. Quantitative evaluation is almost entirely confined to CryoBench, a synthetic dataset whose ground-truth motions are smooth and part-like. Real-data results are presented only as hand-picked snapshots without resolution statistics or FSC curves. Key baselines (e.g. Bayesian multi-body refinement, AlphaCryo4D) are omitted. \n4. Critical hyper-parameters—number of parts, latent dimension, feature-regularization weight—are tuned per dataset with no guidance. Training requires 3–6 hours on a single RTX 2080 for only 10k–20k particles, suggesting prohibitive cost at realistic data scales. \n5. The paper is dense, notation changes between sections, and several equations (Eq. 7–9) omit indices needed to implement the method. Figures lack scale bars and resolution annotations, hampering interpretation. \n\nOverall, the weaknesses outweigh the incremental strengths.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback. We address the concerns and questions below.\n\n**Hierarchical GMM vs Neural Field (SW1).** Our hierarchical Gaussian mixture model is a completely different representation than previous neural field-based approaches (eg CryoDRGN, DRGNAI) and has key advantages. Unlike neural fields defined on the Fourier domain, it allows explicit parameterization of continuous motion and compositional variability in the real-space (L28-29), facilitating use of spatial priors (L109-111), and it provides substantial gains in rendering efficiency compared to other real-space models like 3DFlex (see supplement Sec C). The use of spatial priors is greatly facilitated through learned Gaussian features that encode information about local structural variability, which would be hard in the Fourier domain. Gaussian splatting exploits similar advantages over neural fields, enabling high-fidelity real-time rendering for 3D/4D scene reconstruction from photos. Such advances are substantial.\n\n**K-means and Part Discovery (SW1).** One key result of our work is that per-Gaussian features learned with a self-supervised reconstruction loss captures useful information about structural variability, despite high levels of noise. This is unexpected and significant, as it facilitates the specification of effective priors for heterogeneity. The specific form of clustering is not key in itself. By way of context, priors for motion-based heterogeneous reconstruction methods like 3DFlex are crucial but also tedious to design in a way that enables fidelity without over-fitting. Our approach is simple by comparison; ie, we learn per-Gaussian features that condition neural networks whose output modulates Gaussian amplitude and location. This deliberate design choice provides the inductive bias driving features to encode local structural heterogeneity, enabling discovery of parts based on a simple algorithm like k-means++, where the number of parts is the only parameter. Further research may reveal better forms of clustering, perhaps incorporating principled biophysics criteria like free energy, but this is outside the scope of this paper.\n\n**Effect of Initialization (SW2, Q1, L1).** CryoSPIRE was not claimed to be ab initio (cf L286 where we note it as future work). Rather, as is common in other methods (eg, cryoDRGN, 3DVA, 3DFlex, RECOVAR), we assume an initial rigid map (which may be very blurry due to motion) and image poses as a starting point. While such initial estimates may be noisy, cryoSPIRE is able to recover from errors in the initial reconstruction, eg, with IgG-1D (Fig 4) the large motion of the Fab domain causes the initial density to be smeared out. Further, fixing initial poses is part of the CryoBench evaluation protocol to ensure that estimated structures are properly aligned with the ground truth references and to ensure fair comparisons with the baselines.\n\nHowever, per the reviewer’s request, we applied cryoSPIRE and baselines to IgG-1D but with additive noise in pose (uniform over 5 to 10 deg). Results are in the table below, showing that errors in pose cause blur in the 3D reconstructions, degrading the metrics (cf Table 1 in the paper). Nevertheless, CryoSPIRE continues to outperform.\nMethod|Per-Conf AUC-FSC\n-|-\n3DVA|0.243\n3DFlex|0.231\nCryoDRGN|0.268\nDRGN-AI|0.277\nCryoSPIRE (ours)|0.297\n\n**Isotropic Gaussian Prior, Motion Complexity and Energy Landscapes (SW2).** The sole purpose of the isotropic prior on latent codes is to loosely constrain the scale of the latent embedding. We see no obvious connection between the shape of latent prior and the form of decoded motions per se. In fact, the MLP decodes the latent codes into a variety of motions (see 3D visualizations in the supplement webpage, main.html). We also do not claim that the MLP learns the energy landscape. Rather, its purpose is to learn the mapping between latent codes and the modulation of Gaussian centers and amplitudes to capture conformational and compositional heterogeneity. While some work has aimed to infer the energy landscape (eg AlphaCryo4D or [1]), this is a challenging but distinct problem (see [2]).\n\n**Evaluation Metrics and Datasets (SW3, Q3, L3).** CryoBench (2025) is the only established benchmark for cryo-EM heterogeneity at present. Although synthetic, it contains challenging conformational and compositional variability, based on real cryo-EM studies. Our experimental datasets are widely used to evaluate methods for heterogeneity; as such they enable comparisons with SOTA methods (see supplement Figs 13&14).\n\nCryoBench provides ground-truth and FSC-based performance measures (Per-Conf & Per-Image) to quantitatively compare SOTA methods (Tables 1&3). But there is no established protocol for evaluating heterogeneous reconstruction on experimental data where ground truth is unavailable. Gold-standard FSC is a widely used protocol for homogeneous reconstruction, where one compares two 3D maps from half-sets of the data. For heterogeneity methods, this would produce two different latent conformation spaces without clear correspondences, hence FSC-based comparison is not straightforward.\n\nGold-standard FSC is even more problematic with methods that use GMM representations.  Specifically, FSC is inherently a measure of consistency of reconstruction and relies on the independence of errors in two reconstructions to provide an estimate of accuracy. This assumption is satisfied with classical cryoEM methods, but GMMs exploit a strong structural prior that induces artificial consistencies at higher frequencies, causing FSC on GMM-based methods to misleadingly over estimate resolution.\nOn experimental data, as is common practice (eg, CryoDRGN, RECOVAR), we therefore focus on qualitative comparison. As shown in the supplement pdf and webpage (main.html)  cryoSPIRE outperforms SOTA (3DFlex, 3DVA and CryoDRGN) in recovering fine-grained  detail in highly flexible domains.\n\n**Baseline Methods (SW3, Q4, L4).** We include all baselines in CryoBench (2025), including subspace models, motion-based methods, and neural field methods. The authors of CryoFIRE (2022) led the creation of CryoBench, so we expect cryoFIRE would have been included if it were competitive. Multi-body refinement (2018) is designed for particles with a small number of rigidly moving parts, for which user defined masks must be provided (but are not provided in the CryoBench evaluation protocol). It is not applicable to compositional heterogeneity or more fine-grained continuous conformational deformations. AlphaCryo4D (2022) is focused on a different but related problem of free-energy estimation and uses Relion’s 3D Classification with a large number of discrete conformational states. CryoBench evaluated 3D Classification, the results of which consistently underperform all other methods. That said, we have downloaded the AlphaCryo4D code and will try to run AlphaCryo4D on the CryoBench datasets for completeness.\n\n**Hyperparameter Selection (SW4, Q2, L2).** As reported in Sec 4 (L198), contrary to the review, for all datasets we use default settings of latent dim. $D=4$, regularization weights $\\lambda_z=0.1, \\lambda_f=0.01$ for feature and latent priors, and $H=32$, $H=128$ hidden dimensions respectively for part-discovery and hierarchical model MLPs.\nAlso contrary to the review, the number of parts is not 32 for IgG and 48 for Ribosembly. For IgG-1D and IgG-RL we used 5 or 6 parts (L237). Ribosembly used 8 parts (L241). We allow the user to choose the number of parts and Gaussians, as larger molecules (eg Ribosome) will require more components to reach higher resolutions. At present we visualize the Gaussian features with UMAP (Figs 4-6) from which the number of parts is often clear. While automatic clustering might be useful, cryo-EM practitioners often prefer to choose such parameters, so we don’t view it as critical in practice. Finally, we note our experiment (Table 2, Fig. 9) that explores the effect of different numbers of parts.\n\n**Dataset Sizes, Number of Gaussians and Runtimes (SW4, Q5, L5).** Our datasets are consistent with realistic dataset sizes. IgG-1D and IgG-RL have 100k particles and Ribosembly has >300k particles (L184-187). EMPIAR10076 and EMPIAR10180 have >100k particles each (L191, L195), not 10-20k as stated in the review. The table below reports total runtime, runtime per epoch, number of epochs, and max GPU memory for baselines on CryoBench (3DFlex does not run on Ribosembly). For cryoSPIRE we include the part discovery stage too (hence 15+30 epochs). CryoSPIRE runs significantly faster than competitive methods of CryoDRGN, DRGNAI, and 3DFlex.\n\nWe find that an order of $10^4$ Gaussians is sufficient to achieve SOTA quantitative results on CryoBench and high resolution qualitative experimental results. This is also demonstrated in prior work (see [4,5,6] in paper). \nGiven the linear image formation model, it is straightforward to share across multiple GPUs to dramatically scale the number of Gaussians. Based on the tables below and for the same number of Gaussians we used (~8k), we estimate running on $10^6$ particles approximately requires 14 min per epoch (630 min in total).\n\nIgG-1D and IgG-RL:\nMethod|Total Runtime|Max GPU Mem.|Runtime per Epoch|# Epochs\n-|-|-|-|-\nCryoSPIRE|63 min|5.10 GB|1.40 min|45\nCryoDRGN|304 min|1.34 GB|15.20 min|20\nDRGN-AI|443 min|5.55 GB|4.43 min|100\n3DVA|37 min|0.75 GB|1.85 min|20\n3DFlex|725 min|2.12 GB|-|-\n\nRibosembly:\nMethod|Total Runtime|Max GPU Mem.|Runtime per Epoch|# Epochs\n-|-|-|-|-\nCryoSPIRE|405 min|10.60 GB|9.00 min|45\nCryoDRGN|2209 min|1.36 GB|44.18 min|50\nDRGN-AI|1679 min|5.57 GB|16.79 min|100\n3DVA|120 min|0.75 GB|6.00 min|20\n\n**Notation/Figures (SW5).** We will resolve notational issues and ensure that figures are interpretable.\n\n[1] Tang et al, Ensemble reweighting using cryo-EM particle images, 2023\n\n[2] Evans et al, Counting particles could give wrong probabilities in Cryo-Electron Microscopy, 2025"}, {"review_id": "8Hd1V1AJaV", "reviewer": "Reviewer_La3P", "summary": "This paper focus on reconstructing heterogeneous biomolecules under complex conditions. Specifically, the authors aim to sovle the 3D reconstruction via cryo-EM images of a single particle under two conditions: (1) The 3D structure of biological particles is often not fixed; due to conformational flexibility, a single biomolecule may adopt multiple structural states across different functional or translational stages, while retaining similar sequences or overall functions. (2) The 2D cryo-EM projections may contain missing parts, caused by partial occupancy, structural incompleteness, or limitations in imaging and particle alignment. To solve these challenges, the authors propose CryoSPIRE, which leverages a part-based Gaussian mixture model to flexibly generte 3D partile structure.", "questions": "My main concerns are listed as following:\n1. Weakness 1: Part discovery process, whether a different structure between training and test particle will effect the reconstruction performance or not?\n2. Weakness 2: the anchor numbers and computational cost.\n*********************************************************************************************************\n\nAlso I have an extra question, but this won't affect my overall rating of the paper. I believe it is an excellent piece of work. \n\n1. Although I know the authors aim to solve the reconstruction problem in single particle scenario (subtomogram). I'm simply curious about how this method would perform on tomograms with multiple particles.", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Pros:\n1. This work aims to address a real-world biological problem, which is quite inspiring.\n2. This work is the first to introduce GMM, a generative method commonly used in AI, into the 3D reconstruction process of cryo-EM, which I believe is a highly meaningful contribution. I also believe this approach could be further extended to applications such as 3D cryo-ET generation or structural prediction.\n3. The written English of this paper makes it very easy to follow the authors' idea.\n4. The anchor-based idea proposed by the authors is very interesting, and I believe this approach could be extended to cryo-ET simulation methods.\n\nCons:\n1. Since the entire part discovery process is based on L2 loss computed from existing projection data, would the performance of missing part reconstruction be affected if the test particle is significantly different from those seen during training—for example, if it has a more complex structure?\n2. If a particle has a highly complex structure, would it require a large number of anchors? Would this introduce additional computational overhead?", "was_revised": true, "final_justification": "Thanks for the authors' response. I think the authors have addressed all my concerns. Since I already give the paper the highest rating, I will keep my original score.", "rebuttal": "We appreciate that the reviewer finds our work inspiring and our contributions meaningful. In what follows, we answer the questions.\n\n**\"...would the performance of missing part reconstruction be affected if the test particle is significantly different from those seen during training—for example, if it has a more complex structure?\"... \"whether a different structure between training and test particle will effect the reconstruction performance or not?\"** Currently, we have focused on the single particle reconstruction case, where the dataset consists of a single particle type. Here, reconstruction is framed as a model fitting to the structure of the same complex which exhibits conformational or compositional variability and the evaluation is based on FSC between output reconstruction and ground-truth structure. As such, for our work to date, we are not testing on out-of-distribution data. Rather we optimize the model from scratch on each new dataset.\n\n**\"If a particle has a highly complex structure, would it require a large number of anchors? Would this introduce additional computational overhead?\"** For a structure that exhibits more complex dynamics or consists of several compositional parts, we need to accordingly increase the number of anchors/parts. However, as the number of Gaussians is orders of magnitude higher than the number of anchors, the computational overhead to evaluate anchor-specific MLPs to learn rigid part-based motion is not a substantial barrier in our current framework. While the number of anchors is not a significant computational burden, the number of Gaussians does increase the compute cost. Fortunately, one can scale the number of Gaussians significantly, since the image formation model can be parallelized over multiple GPUs straightforwardly.\n\n**\"Although I know the authors aim to solve the reconstruction problem in single particle scenario (subtomogram). I'm simply curious about how this method would perform on tomograms with multiple particles.\"**\nInteresting question. We would like to extend the work to a more general cryoET setting (e.g., with multiple types of particles). However, we expect this would be a major undertaking that would likely require new algorithmic considerations to account for the particularities of cryoET data. As a result, this is unfortunately outside the scope of the current project but would be an excellent direction for future work."}, {"review_id": "ZSDnvHptvG", "reviewer": "Reviewer_LVtR", "summary": "Single particle cryo electron-microscopy (cryo-EM) is a field whose goal is to reconstruct 3D volumetric maps of molecules such as proteins and ribosomes from large samples of images taken with a transmission electron microscope. A key challenge in this field is heterogeneity within the sample. This includes discrete heterogeneity due to attachment and detatchment of components as well as continuous heterogeneity due to the motion of parts of the molecule.  This paper proposes a new system for 3D reconstruction in the presence of either discrete or continuous heterogeneity that is based on modeling the atomic structure of the molecule as a Gaussian mixture model with rigidly-moving components. A neural network that takes latent parameters is used to parameterize the specific configuration of the molecule in each sampled image and the model is fit to the data using SGD.", "questions": "1. Why did you not include the code in this submission?\n2. In page 4, line 133 you explain that the MLP returns a rotation in SO(3). I'm assuming that it is given as a 3x3 matrix. How do you enforce it to be a member of SO(3)?\n3. In page 9, line 268, you reference Table 1 but did you mean to reference Table 2?", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "The method is clearly motivated and explained. It is tested on CryoBench (currently the leading benchmark in this field) where it shows SOTA results.  The paper is very well written and the included main.html sets a very high bar in scientific visualization.  Overall this is a very strong and well-polished contribution.\n\nWeaknesses:\n1. The paper does not include running time benchmarks. The authors should include at least one table of runtimes (or plot), as well as memory requirements, alongside the description of the computer/s used to run the benchmarks: CPU, GPU, how many processes, how many threads, etc.\n2. For some reason, the authors did not include the code for this review but have promised to release it upon acceptance. If they follow through on their promise then this is a strong accept.\n\nMinor comments:\n1. Page 2: \"To out knowledge, this is first GMM-based...\" -> \"this is the first\".\n2. Page 2: \"Such methods are often notoriously memory intensive and have limited expressiveness\". I'm not sure what you mean by \"limited expressiveness\". Methods based on eigenvectors of the covariance matrix can reconstruct any volume, they may just require a large number of eigenvectors to do so.\n3. Page 7, line 240: \"rest of the complex is remains fixed\" -> remove the word \"is\".", "was_revised": true, "final_justification": "None of the other reviewers raised any major concerns so I maintain that this is an excellent work. I think the authors did a good job in their rebuttals, including mine.\n\nSince this is clearly high-quality work in an important and timely field, I strongly support acceptance provided that the authors will release their code.", "rebuttal": "We are grateful that the reviewer finds CryoSPIRE to be a strong and well-polished contribution. Below we answer the questions and discuss concerns raised in the review.\n\n**\"The paper does not include running time benchmarks. The authors should include at least one table of runtimes (or plot), as well as memory requirements, alongside the description of the computer/s used to run the benchmarks: CPU, GPU, how many processes, how many threads, etc.\"**\nWe thank the reviewer for pointing this out.  We will include a table with these parameters, both for our cryoSPIRE method as well as the other baseline methods on the benchmark data, where we report total runtime, runtime/epoch, number of epochs, and max GPU memory. (3DFlex does not run on Ribosembly). The GPU is a single NVIDIA GeForce RTX 2080. The CPU is Intel(R) Xeon(R) Silver 4110. For our method we include the part discovery stage too (hence 15 + 30 epochs). Note cryoSPIRE runs significantly faster than competitive methods of CryoDRGN, DRGNAI, and 3DFlex. We will include these tables in the revised paper.\n\nIgG-1D and IgG-RL:\nMethod|Total Runtime|Max GPU Mem.|Runtime per Epoch|# Epochs\n-|-|-|-|-\nCryoSPIRE|63 min|5.10 GB|1.40 min|45\nCryoDRGN|304 min|1.34 GB|15.20 min|20\nDRGN-AI|443 min|5.55 GB|4.43 min|100\n3DVA|37 min|0.75 GB|1.85 min|20\n3DFlex|725 min|2.12 GB|-|-\n\nRibosembly:\nMethod|Total Runtime|Max GPU Mem.|Runtime per Epoch|# Epochs\n-|-|-|-|-\nCryoSPIRE|405 min|10.60 GB|9.00 min|45\nCryoDRGN|2209 min|1.36 GB|44.18 min|50\nDRGN-AI|1679 min|5.57 GB|16.79 min|100\n3DVA|120 min|0.75 GB|6.00 min|20\n\n**“For some reason, the authors did not include the code for this review but have promised to release it upon acceptance.\"**\nWe will definitely release the code upon acceptance. The method has been implemented in a publicly available framework, so code release will be straightforward.\n\n**\"Page 2: Such methods are often notoriously memory intensive and have limited expressiveness\". I'm not sure what you mean by \"limited expressiveness\". Methods based on eigenvectors of the covariance matrix can reconstruct any volume, they may just require a large number of eigenvectors to do so.\"**\nWe agree that by increasing the number of principal components (e.g., as in RECOVAR), any motion could eventually be represented to a given accuracy, but continuous motions, especially large-scale ones, would require a large number of principal components.  In practice this comes at the expense of much higher (often prohibitive) memory consumption and a higher dimensional latent configuration space. We will clarify this in the revised paper.\n\n**\"Page 2: \"To our knowledge, this is first GMM-based...\" -> \"this is the first\".\nPage 7, line 240: \"rest of the complex is remains fixed\" -> remove the word \"is\".\nPage 9, line 268, you reference Table 1 but did you mean to reference Table 2?\"**\nThank you for bringing the typos to our attention. They will be corrected in the revised paper.\n\n**\"In page 4, line 133 you explain that the MLP returns a rotation in SO(3). I'm assuming that it is given as a 3x3 matrix. How do you enforce it to be a member of SO(3)?\"**\nWe follow prior work [1,2] and use the so-called S2S2 6D representation from which we compute a rotation matrix $R$ using Gram-Schmidt, thereby ensuring $R$ is a member of SO(3). In particular, we set the MLP to return a 6D vector and split it into two 3D vectors. We then normalize each to unit length and denote them by $v_1, v_2 \\in \\mathbb{R}^{3}$ . Next, we compute their cross product and normalize, ie, $v_3 = \\frac{v_1 \\times v_2}{||v_1 \\times v_2||}$, yielding a third unit vector. Note that $v_3$ is orthogonal to $v_1$. By selecting $v_1$ and $v_3$ as the first and third columns of the rotation matrix, we compute the second column as $\\tilde{v}_2 = v_3 \\times v_1$, which has unit length and is orthogonal to $v_1$ and $v_3$. Then, $R = [v_1, \\tilde{v}_2, v_3]$.  We will include these details in the revised manuscript.\n\n\n[1] Nashed, Youssef SG, et al. (2021) CryoPoseNet: End-to-end simultaneous learning of single-particle orientation and 3D map reconstruction from cryo-electron microscopy data. Proc IEEE/CVF International Conference on Computer Vision.\n\n[2] Levy, Axel, et al. (2022) CryoAI: Amortized inference of poses for ab initio reconstruction of 3d molecular volumes from real cryo-em images. European Conference on Computer Vision."}, {"review_id": "R0sNu5Zfc2", "reviewer": "Reviewer_njmf", "summary": "This paper proposes a framework for 3D reconstruction of molecular structure from noisy 2D images using hierarchical Gaussian mixture model. The method first infers a part-based segmentation using a coarse-grained GMM with learnable features. Rotations, translations and amplitude modulations are introduced to account for conformational flexibility and compositional variation. Numerical experiments on multiple datasets show the efficiency and utility of the proposed framework comparing to benchmarks.", "questions": "1. Could you please clarify the connection and difference between Eq (1)-(2) in the main paper and Eq (8)-(10) in the supplement? Do you assume ${\\bf R}^{(n)}$ and ${\\bf t}^{(n)}$ are known or unknown? They do not appear in Eq (8)-(10).\n2. Could you please clarify how do you specify the number of anchors $A$, number of Gaussians $G$ and which Gaussian is associated with which anchor? Would these affect the results of reconstruction?\n3. Could you please comment on the convergence performance of reconstruction loss minimization? \n4. I noticed there are a lot of uncertainties in FSC metric for Ribosembly dataset for all methods but much fewer for the other two datasets (Fig 3). Could you please comment on this? Where do you think the difference comes from?", "limitations": "The connection between Section 2 and Section 3 is not very clear. The new notation in Section 3 Eq(3)-(5) is confusing--it may be better to reconsider the notation or provide a summary in a table.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The authors introduce the practical problem and limitations of existing methods very well, which provides a clear motivation of the proposed method and its significance.\n2. The framework is very flexible and fits nicely with practical considerations.\n\nWeaknesses:\n\n1. Notation is confusing and writing clarity can be improved.\n2. Due to the complexity of the model, the identifiability of parameters and the robustness of the results are unclear.", "was_revised": true, "final_justification": "Dear AC, SAC, PC:\n\nI think the author partly addressed my concerns regarding sensitivity and robustness in their latest rebuttal. But since most of my training is in statistics and not in biology, I may not be able to appreciate the practical significance of the work. However, this work seems to have significant practical contributions according to other reviewers. I feel the writing clarity can be further improved, but this does not seem to be a concern to others as well.\nIn the end, I decided to raise my score by 1 and keep my confidence. I hope this can help with the final decision. Thank you!\n \nBest,\nReviewer njmf", "rebuttal": "We thank the reviewer for finding our framework flexible and practical. We will address the reviewer’s comments and questions below. \n\n**\"Notation is confusing and writing clarity can be improved ... The connection between Section 2 and Section 3 is not very clear. The new notation in Section 3 Eq(3)-(5) is confusing--it may be better to reconsider the notation or provide a summary in a table.\"**\nWe will revise the paper to clarify these sections, draw clear connections and unify notations. In short, Sec. 2, presents background on Gaussian mixture models (GMM), and it provides the well-known rendering equation (Eq. 2) to map a 3D GMM onto the image plane, with notations that makes explicit the amplitude ($m_i$), scale ($s_i$) and center ($c_i$) parameters for the i-th component. Sec. 3 presents our method, using the same notation to denote the GMM parameters. But we add a particle index $n$ in the superscript to allow us to have per-particle structural variation. The structural variation is captured by the modulation of amplitudes and the displacement of Gaussian centers (see Eqs 4-6). Following your suggestion, we will add a table that summarizes notation and rewrite these sections for improved clarity.\n\n**\"Due to the complexity of the model, the identifiability of parameters and the robustness of the results are unclear.\"**\nCryo-EM (with or without heterogeneity) is a highly non-convex problem with multiple minima. As is the case with GMM fitting, there are inherent symmetries where, for example, permutations of the parameters yield similar or even identical solutions. As such, like learning in neural networks, one should assume that the parameters are not straightforwardly identifiable, and robustness is hard to establish in any formal sense. Note that, in terms of robustness, since the introduction of SGD for ab-initio by cryoSPARC in 2017, single-particle rigid cryo-EM reconstruction has been generally successful in practice. Methods for heterogeneity, while not as reliable as methods for the single, rigid particle case, have been effective and continue to improve.\n\n**\"Could you please clarify the connection and difference between Eq (1)-(2) in the main paper and Eq (8)-(10) in the supplement? Do you assume R(n) and t(n) are known or unknown? They do not appear in Eq (8)-(10).\"**\nWe apologize for the confusion. In all experiments, we assume that pre-estimated particle poses (parameterized by rotation matrix $R$ and translation vector $t$) are given as input. For Eqs 8-10 in the supplement, to avoid clutter, we dropped the dependence on $R$ and $t$, assuming the particles are already reoriented and positioned based on the input pose. For clarity, we will modify the supplement to make this explicit and avoid such confusion.\n\n**\"Could you please clarify how do you specify the number of anchors A, number of Gaussians G and which Gaussian is associated with which anchor? Would these affect the results of reconstruction?\"**\nTo determine the number of anchors/parts, following the optimization of the part discovery GMM, as discussed in L162-170, we perform dimensionality reduction with UMAP to visualize the learned Gaussian feature space. As illustrated in Figs. 4C, 5A, 6B, 7B, and 8B, Gaussian components are clustered into well-separated groups, indicating groups of Gaussians with similar properties. We then select the desired number of parts (i.e. clusters) and use kmeans++ to obtain assignment of Gaussians to parts. Our results show a great promise for future work on automatic methods to find the number of clusters.\n\nWhile the part discovery stage uses a small number of Gaussian components (i.e. 2048), the hierarchical model in the second stage captures more fine-grained details, using many more Gaussians (8192 or 16384 based on the structure size). To assign Gaussians to anchors (part centers so to speak), we take the following procedure: For each Gaussian in the hierarchical model, we find the closest Gaussian in the part-discovery model, and use the part to which it’s assigned. Please see Sec. D in the supplement for more thorough discussion of this procedure.\n\nFinally, in an ablation study (Sec. 5.1) we show that using a high number of anchors (eg, $A=64$) leads to significant oversegmentation and degrades performance (see Table 2), as it fails to capture locally rigid and smooth motion (illustrated in Fig. 9, left). While there may be no ideal number of anchors from first principles, we find that 5-10 anchors are sufficient to help provide useful priors for heterogeneity in the datasets that we’ve considered.\n\n**\"Could you please comment on the convergence performance of reconstruction loss minimization?\"**\nIn cryoEM methods, convergence is generally assessed in terms of the stabilization of the 3D reconstruction.  Like other non-convex optimization problems in machine learning, there are few principled ways to assess convergence.  In CryoEM, because of the high level of image noise, the loss and its gradients are also very noisy. As a consequence, relatively large batches are typically used to reduce the variance of the gradient estimator during optimization, and the minibatch loss is not a very reliable indicator of convergence.  We find 30 epochs are sufficient for the map to stabilize.\n\n**\"I noticed there are a lot of uncertainties in FSC metric for Ribosembly dataset for all methods but much fewer for the other two datasets (Fig 3). Could you please comment on this? Where do you think the difference comes from?\"**\nThe visualization in Fig. 3 shows the FSC curve averaged over all structural states. The error bars depict the FSC standard deviation across different states. There are two likely causes for the FSC variability across structural states:  First, with Ribosembly, unlike IgG, particles are not uniformly distributed across compositional states. States with few particles are expected to be determined at lower resolutions (i.e. lower FSC curves) than those with many particles. A second cause may be differences in the type of heterogeneity. With conformational variability, as in IgG, if one can accurately model deformation, then one can aggregate information across many particles to resolve structure more accurately, yielding consistent FSC curves.  With compositional heterogeneity, parts of density be present in only a small number of states (e.g., consider a small subunit present in only one state) and as a result FSCs may vary significantly for different states, producing greater variance."}]

Author rebuttal:

Meta-review: This paper presents CryoSPIRE, a heterogeneous cryo-EM reconstruction method that models biomolecules with a hierarchical Gaussian mixture representation combined with part discovery. Unlike existing approaches that either rely on generic latent variables or restrict to only conformational or compositional heterogeneity, CryoSPIRE explicitly models both. It achieves state-of-the-art performance on CryoBench and demonstrates promising results on experimental systems. Overall, this is a well-motivated, technically solid, and impactful contribution that advances the state of the art in cryo-EM heterogeneity modeling.
Decision: Accept (poster)