CryoGEN: Generative Energy-based Models for Cryogenic Electron Tomography Reconstruction
Authors: Yunfei Teng
Yuxuan Ren
Kai Chen
Xi Chen
Zhaoming Chen
Qiwei Ye
Venue: ICLR 2025 Poster (2025)
Keywords: Cryo-ET
EBMs
Cryogenic Electron Tomography
Generative Energy-based Model
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Cryogenic electron tomography (Cryo-ET) is a powerful technique for visualizing subcellular structures in their native states. Nonetheless, its effectiveness is compromised by anisotropic resolution artifacts caused by the missing-wedge effect. To address this, IsoNet, a deep learning-based method, proposes iteratively reconstructing the missing-wedge information. While successful, IsoNet's dependence on recursive prediction updates often leads to training instability and model divergence. In this study, we introduce CryoGEN—an energy-based probabilistic model that not only mitigates resolution anisotropy but also removes the need for recursive subtomogram averaging, delivering an approximate *10*$\times$ speedup for training. Evaluations across various biological datasets, including immature HIV-1 virions and ribosomes, demonstrate that CryoGEN significantly enhances structural completeness and interpretability of the reconstructed samples.

Published as a conference paper at ICLR 2025
CRYOGEN: GENERATIVE ENERGY-BASED MODELS FOR CRYOGENIC ELECTRON TOMOGRAPHY RECON- STRUCTION
Yunfei Teng' Yuxuan Ren Kai Chen
Xi Chen
Zhaoming Chen
Qiwei Ye
Beijing Academy of Artificial Intelligence (BAAI) {yfteng, yxzen, kchen, xchen, zmchen, qwye)@baai.ac..o
ABSTRACT
Cryogenic electron tomography (Cryo-ET) is a powerful technique for visualiz- ing subcellular structures in their native states. Nonetheless, effectiveness is compromised by anisotropic resolution artifacts caused by the missing-wedge ef- fect. To address this, IsoNet, deep learning-basee method, proposes iteratively reconstructing the missing- wedge information. While successful, IsoNet's depen- dence on recursive prediction updates often leads to training instability and model divergence. In this study, introduce CryoGEN-an energy-based probabilis- tic model that not only mitigates resolution anisotropy but also removes the need for recursive subtomogram averaging, delivering an approximate 10x speedup for training. Evaluations across various biological datasets, including immature HIV-1 virions and ribosomes, demonstrate that CryoGEN significantly enhances structural completeness and interpretability of the reconstructed samples.
INTRODUCTION
Cryo-ET is cutting-edge imaging technique that allows the detailed examination of three- dimensional (3D) structures of biomolecules, cellular components, and whole organisms at near-atomic resolution, preserving them in near-native and hydrated environment. Pioneer- ing work by (Lucic et al. 2005) offers a com- prehensive overview of using Cryo-ET to visu- alize cellular structures within their native envi- ronments. Subsequent advancements by (Beck & Baumeister] (016)) and (Schur et al. (015) further demonstrate its capacity to resolve in- dividual proteins and their interactions at sub- nanometer resolution. These capabilities posi- tion Cryo-ET as an indispensable tool for de- ciphering the intricate organization and regula- tion of molecular machinery in living systems.
Flectron Beam
Stage
Frezen Speeimen
Transmission Imaging
Tiit Series
Fourier Transform
WBP Reconstruct
The workflow of Cryo-ET proceeds follows hydrated biological sample, intended for ob- servation, is vitrified through rapid freezing to preserve its native state, and then continuously imaged by a transmission electron microscope (TEM) as the slide is incrementally tilted through series of angles. This process will generate series of two-dimensional projections, and the collec- tion of these projections are commonly referred to as a tilt series. The tilt series are then aligned and reconstructed into 37 structure, which is known ous available tomographic reconstruction methods, weighted nook-prooection (WBP) (Radermacher 2006) is the most widely used one. This entire procedure is depicted in Figure
Missing
Reconstructed
Figure Cryo-ET imaging and reconstruction.
tomogram in Cryo-ET. Among the vari-
However, the optical constraints in the TEM stage typically limit the tilt range to approximately +600, resulting in a missing wedge of data in Fourier space. Consequently, tomograms reconstructed
t: Methodology development and algorithm implementation. Algorithm improvement and experimental validation.

---

Published as a conference paper at ICLR 2025
via WBP exhibit anisotropic resolution (see Figure 2 (a)), with the lowest resolution observed along the Z-axis (parallel to the direction of electron beam). This anisotropy leads to distorted or elongated structural features in the 3D reconstruction, hindering the structural interpretation and underscoring a persistent challenge in Cryo-ET data analysis.
Traditional methods mitigate the missing wedge problem include dual-axis tomography [Mastronarde] 1997), which acquires two per- pendicular tilt series to reduce the missing wedge smaller "missing pyramid" Com- putational approaches such as model-based it- erative reconstruction (Yan et al. 2019) and iterative compressed-sensing optimized non- uniform fast Fourier transform reconstruction Deng al.] 2016) employ statistical tools and optimization techniques to compensate for missing data. However, these methods intro- duce additional complexity to either data ac- quisition or processing, yet they remain insuf- ficient for fully restoring the missing informa- tion. Meanwhile, Cryo-ET contends with the ratios
(a) recessccodd
(b) -ooNet-corretted (baseline)
inherent challenge of low signal-to-noie (SNR) caused by the need use low electron doses to prevent radiation damage. Traditional approaches, such as tailored filtering (Tegunov & Cramer (219)) contrast transfer function deconvolution (Cohen et al. 2024), can only partially mitigate noise, leaving necessary SNR enhancement
CryoGEN-corretted (ours)
(d) Real structure
Figure 2: Reconstructed 3D structure comparison
among (a) WBP (b) IsoNet and (c) our method using simulated Cryo-ET data of (d) C13 Vippl stacked rings (EMDB:18424)
unresolved challenge.
In the past few years, deep learning- -based approaches have demonstrated promising avenues for restoring missing wedge data and improving SNR. IsoNet (Liu et al., 2022). for instance, has achieved notable success in restoring missing wedge information, inspiring subsequent methods (e.g.. (Wiedemann & Heckel 2024))- Concurrently, Noise2Noise- based denoising tools like Cry- oCARE (Buchholz et al., 2019). Topaz (Bepler et al.) (220)). and Warp (Tegunov & Cramer (019)) have significantly enhanced tomogram quality. Recent efforts (Wiedemann & Heckel 2024 Zeng et al. (024) aim to unify missing wedge correction and denoising into integrated pipelines. Despite these advances, current methods still produce artifacts and distortions, underscoring the need for more robust and refined solutions.
In this work, we advance Cryo-ET data reconstruction by introducing CryoGEN, generative deep learning framework grounded in energy-based modeling. Different from IsoNet (Liu et al. 2022), which relies on recursive training susceptible eтrоr accumulation and requiring iterative fine- tuning of predictions, CryoGEN eliminates these limitations through a direct modeling of data dis- tributions from Bayesian perspective. This approach leads to enhanced training stability, acceler- ated convergence, and immunity to pitfalls observed in IsoNet. Comparative results in Figure (b) and (c) illustrate CryoGEN's superior reconstruction fidelity.
Our contributions are: (1) Bayesian Formulation: We frame Cryo-ET reconstruction within Bayesian framework, addressing theoretical challenges and practical constraints; (2) Objective Anal- ysis: We identify potential failure mode in IsoNet's algorithm, analyze its underlying cause, and propose corrective strategies; (3) Energy-Based Modeling: We derive a novel objective function based energy-based modeling and provide detailed end-to-end training and inference protocols for CryoGEN; (4) Empirical Validation: Across multiple datasets, we show that CryoGEN achieves significantly improved accuracy and faster training compared existing methods.
2

---

Published as a conference paper at ICLR 2025
2 PRELIMINARIES
2.1 PROBLEM FORMULATION
In Cryo-ET reconstruction, let E X' represent the original data (complete tomograms with missing wedge restored), characterized by the distribution Pz- The observed data E y corresponds to WBP-reconstruuteed tomograms affected by the missing wedge, following distribution py-
We also define a missing wedge operator TM : &' + y, where y = [Tai (z) x X). This opera- tor Tas acts many-to-one linear transformation, which be conceptualized as first converting the image into the Fourier domain, masking to simulate tilt limitations, and transforming back. The imaging process for Cryo-ET is modeled
y = TM (z) + Gu Gn N0 I
(1)
where en denotes additive Gaussian noise, with o² determined experimentally. Consequently, Re- covering from y constitutes an ill-posed inverse problem.
2.2 BAYESIAN RULE
We observe that y follows Gaussian distribution y ~ (T), Applying Bayes' theorem, obtain p(a y) p(y|z)) p(x),, which leads to:
log p( v)
logp(y x))
log)(z)
constant
Posterior
Likelihood
Prior
Evidence
Thus, maximizing log p(rly) equivalent maximizing the sum of log p(y) and log ((g). While the maximization of log p(y (x) is straightforward, the prior distribution p(r) for complete tomo- grams is typically unknown. This gap not only presents challenge but also necessitates the devel- opment of an effective method for properly defining this prior.
2.3 ENERGY-BASED MODELS
To model an arbitrary distribution, we employ Energy- based models (EBMs) (Lecun et al. 2006), which provide a probabilistic framework by defining distributions in terms of an energy function. We introduce a non-negative energy function Eg, allowing the distribution to be expressed as:
p(x)
(x)),)),,
where Z is the partition function (Boltzmann) 1974). The energy function E, assigns lower en- ergies high-probabilityy states (e.g., properly reconstructed tomograms) and higher energies to implausible states, thereby sculpting the energy landscape to match the data manifold.
MOTIVATION
For each observation y E y, substituting log p(y]z) = - TM (x) yll2 into Equation 2 leads 2nz
to the optimal reconstruction x² obtained via maximum a posteriori (MAP) estimation as the following:
(y) arg log.loop(aati arg max #EX
[M(()) - -wl12 +lggp()))
(4)
20
SEN
To circumvent the computational expense of iterative sampling optimization, a natural strategy is learn a parameterized mapping ge : y + x (v). This approach, which focuses on learning the mapping directly, is not immediately obvious but is fundamentally similar to the method (Johnson) al. (0166) (i.e., training neural networks to predict the targets directly rather than generating them iteratively during runtime). This concept also forms the foundation of IsoNet (Liu et al. (022) and its variants (Buchholz et al. 2019), whose limitations we will discuss shortly, alongside the rationale for incorporating generative model to address these challenges.
Consider a simplified scenario with single observation y {yo), where two distinct ii and x2 map exactly to yo under the operation Ta. obviously:
||Tx (21) voll 0, (x2) voll3 0.
3

---

Published as a conference paper at ICLR 2025
Under the linearity of TM. the likelihood is maximized along the line segment connecting #1 and D2. Assuming a prior p(æ) modeled as Gaussian mixture:
p(x) = [N(x) 02, + +N(2 22, o2],
where N(x I) denotes single Gaussian distribution centered at with covariance matrix o² Alternaively, this can be expressed as:
|| 21112) )
(-1la
||
x2
((x) 2V2TT
exp
+ exp
20
2x²
22ll2 12. ensuring sufficient separation between modes in X, and seek to max- imize the posterior p(ge yo go). Following IsoNet's methodology (as detailed in Appendix A.I the objective simplifies solving:
We assume o² < || x2
0' = min (10) x1 112 (vo) - x2112
0
This formulation seeks the parameter 6' that minimizes the mean squared error between the func- tion's 's output go (yo) between both ia and 22. Apparently, this leads to go. (Bo) 2 (21 12). and thus
| x2l1
(z go (yo))
exp
(5)
v 2
80²
This result may be unfavorable because
| 32 1
p( = 21 or
exp
(6)
2v 2TT
20
can be shown that Equation (5) yields lower probability compared to Equation (6) under our assumptions, indicating that the result learned by go (Uo) is suboptimal compared to directly se- lecting either 21 or x2- This typically happens when the probability density function of the prior distribution is non-convex, and thus simply averaging the minima lacks meaningful interpretation, highlighting a key limitation we have observed with IsoNet. By introducing an energy function as E. (x) log p(æ), the situation is depicted in Figure 3a), where it is demonstrated that E, (T) does not correspond to a low-energy state, where F denotes the average of æ1 and Z2.
The fundamental issue arises from the inherent one-to-many mapping, which precludes the ex- istence of deterministic function capable of associating a single observation with all poten- tial minima. Mathematically, this reflects the constraint that the cardinality of ga()) cannot surpass that of V, inevitably causing undesir- able results. A simple yet effective solution is to incorporate Gaussian noise through input aug- mentation by perturbing the input as y + Eg. where Eg NO, o² Since the support of Eg spans the entire real space (an uncountably infinite set), the ambiguity in the mapping is ef- fectively resolved.
E(x)
1
X2
(a) Energy landscape
Domain y
Yo
ge (vo
yo (yo + E2)
9g (Ma)
Domain X'
Integrating the energy model E, with go(y + eg) enables the generation of distributions aligned with the target, facilitating the identi- fication a suitable minimum rather than aver- aging over multiple minima. This method also circumvents the computational burden of one-the-fly sampling, inheriting the advantages of IsoNet while resolving the earlier limitation. As illustrated in Figure 3l)). the noise cg steers the model toward distinct low-energy states during the minimization of E,, Tavoring convergence to a specific local minimum over an averaged solution. Notably, we later establish in Equation 9 that this framework be reinterpreted as generative task.
472
(b) Find a minimum
Figure Limitations IsoNet, formulation.
4

---

Published as a conference paper at ICLR 2025
4 METHODOLOGY
Our objective to determine a suitable energy model E,, but training the energy model directly on the space X' is not feasible due to the lack of labeled data, restricting training to the observation space V. However, we can assume that a sample x belongs to X, its corresponding energy should be low. Moreover, operation selected from predefined set of rotations detailed Appendix A.2 This rotation operation assumes that, aside from the missing wedge angles, the distributions are uniform in all directions, which a fundamental premise Cryo-ET reconstruction.
assume that if
X, then R(x) E X' as well, where R denotes rotation
Thus may train an energy model E on Y with (x)x) ाीे EnE (Ta x)). Inspired by (Lecun al., 2006), we do not explicitly define the energy function. Instead, could define it in manner akin to contrastive learning, where low energy is assigned to samples while higher energy attributed other undesired regions in the energy space. This approach falls under the category of implicil probubilistic models (Diggle & Gratton 1984p. Using the trained energy model, we define the distribution ps based on the Boltzmann distribution, as previously described in Equation
4.1 OBJECTIVE
We train the model with objectives: consistency loss and posterior maximization. The consis- tency loss follows the IsoNet formulation; although ultimately stabilizes, its inherent drawbacks can impede early progress and even cause divergence (see Section 3. The energy-based penalty, by contrast, supplies sharper optimization signal but is well-known for instability at the end of training. To balance these trade-offs, we combine both terms.
4.1.1 CONSISTENCY LOSS
In general, we can assume that go serves as an approximate inverse of TM. To enforce this inverse condition, we first introduce consistency loss:
Consistency Loss = Byrp N(0.0
1
Σ RER
| gor TM o R (golv)) + en) (
R
where "o" denotes function composition. While Equation largely mirrors IsoNet's objective function (Liu et al. 2022). it omits the epoch-wise recursive refinement step. If Tas is not one- to-one-ar as often the case-the problem highlighted in Section can arise, so injecting noise en becomes essential.
4.1.2 MAXIMUM A POSTERIOR
Next, our goal is to generate results that maximize the log posterior log logp(zly) by incorporating the previously discussed energy penalty term. The objective is to minimize the error X' using model trained on the y. Therefore, we must ensure that (7Tx( is low when Tax (x) E y and high otherwise. Therefore, can define the posterior as:
1 20
Posterior Eyrp,,
Σ E Ro BER
I o go kg)
v1
R
(8)
where o denotes the hyperparameter introduced in Equation 11. By overcoming the limitations of relying solely the consistency loss, our approach obtains notable performance gain in the early stage of training.
4.2
ENERGY MODEL
The energy model E can be constructed using various methods, such as (Chen et al. 2020; Du 2023) In this work, directly employ generative adversarial networks (GANs) (Goodfellow 2014 Specifically, we define our energy model as parameterized neural network Es with et al.
al.
parameters Φ. This configuration allows for the simultaneous training of both the energy model
5

---

Published as a conference paper at ICLR 2025
Eg and ge- Consequently, the energy model is learned through adversarial training, as outlined in Equation (9:
Energy max min p.pp
Es (TM o R go + +g)) - Egly +
R
RER
Furthermore, we consider a similar formulation as (Arjovsky & Bottou 2017). Let y follow the
distribution Py with support on V/ and let be absolutely continuous distribution with density Pc- Then, the distribution Pyte is also absolutely continuous with density:
Py+e = Ey-pa [pe(z - -y]1 = Pc(z - y) dpy.
(10)
Especially, If e ~ NN 1)), then (y+e(2) x Jn y1=-y day. and Equation 9 reaches Nash equilibrium when Pao (z) (y)e((), as follows:
Pgo
[1=nTaoRopo(ute) Pe, (cg) Py(y) deg dy- RER
(11)
R yEy
EyER*
This result is analogous to the conclusion presented in (Goodfellow et al. 2014). except that here, the data distribution is obtained by convolving the original data distribution with Gaussian.
4.3 ALGORITHM
The CryoGEN algorithm consists of two stages: training and inference. (1) During the training stage, a prediction model is trained by combination of consistency loss and posterior maxi- mization along with an energy model Es, using Equations (7) (9) The full training procedure is detailed in algorithm T (2) In the inference stage, the complete tomogram is first divided into multiple overlapping subtomograms. These subtomograms are then processed through the predic- tion model go generate refined versions, which are later reassembled into complete tomogram. Finally, to minimize edge effects, overlapping regions are averaged using a weighted approach.
Algorithm Training prediction model.
Require: Tomogram dataset V, noise levels 02, 0g o 0, estimated noise variance o energy model Es
Rdd + R+, prediction model go Rd + Rdd learning rate 7,, energy penalty term
A 0.
repeat
Randomly generate noise e, Eg. Eh N(000000) o(0, N/O, I..
Set [(,, 0) E.p (TM Ro go +g)) + 1/20 1|TM o goly + kg)
v11
Update 0' + 0' -1 der R o gø o TM (go(y) + - (v)
Update o < @ 11 Dz
Ex -(0
Update + nd 0 n/)) 0' 1/1 a (b)
Reduce the energy penalty term À
until convergence
return ge
EXPERIMENT
In this section, we compare our method with IsoNet, the state-of-the-art missing wedge correction technique, across various experiments. First, we validate our hypothesis using simple geometric shapes, described in Section 5.1 Next, we assess our algorithm on a simulated dataset and benchmark it against other approaches in Section 5.2 Finally, we evaluate our method on real- world examples to test its robustness. data processing details are provided in the Appendix, where we also present results from the lat- simultaneous missing wedge correction and denoising method, DeepDe (Wiedemann &
discussed in Section 5.3 Additional implementation and
Heckel 2024).
Remark. CryoGEN achieves significant speedup, completing the training process approximately 10x faster than IsoNet. For instance, with an NVIDIA V100, CryoGEN requires only two hours, whereas IsoNet takes around 20 hours Section 5.3 experiment.

---

Published as conference paper at ICLR 2025
(a)
IsoNet-corrected sphere.
IsoNet-corrected prism.
Central
Central Xi2
Central
Central KY
Central 1i2 Shoee
(c) CryoGEN-correett sphere.
(d) CryoGEN-corrected prism.
Figure 6: CryoGEN and IsoNet corrected shapes.
5.1 SIMPLE SHAPES
First, we validate the algorithm's effectiveness using sim- ple synthetic shapes with known ground truth. We gener- ate 3D sphere and triangular prism, artificially intro- ducing a missing wedge. To illustrate this, we transform the X-Z slice into the Fourier domain, clearly revealing the missing wedge, as shown in Figure 4
Our objective reconstruct the missing wedge regions, and we demonstrate that our algorithm significantly out- performs the baseline in this task. As illustrated in Fig- ure 5) the synthetic images exhibit reduced resolution in the directions affected by the missing wedge, while the X-Y slice closely matches the ground truth as designed) closely matches the ground truth by design, as illustrated in Figure
(a) Sphere.
(b) Prism.
Figure 4: Central Fourier slice of the
original missing-wedger shapes.
We first apply both IsoNet and CryoGEN the two shapes, with the results shown in Figure 6 Both methods aim to restore corrupted regions. While IsoNet successfully reconstructs the sphere, some artefacts remain, and fails restore the prism, pro- ducing distorted oval in the X-Z and Y-Z slices. contrast, CryoGEN achieves an almost perfect restoration of both shapes, with the slices closely resembling the original clean images. Our experi- ments reveal that CryoGEN can effectively recover more of the missing wedge regions and capture high- frequency signals.
(a) Original missing-wedgedt sphere.
Centri XY Siice
Central x.2 Slice
Central Si
Next. we quanticatively evaluate performance using the Peak Signal-to-Noise Ratio (PSNR) and Struc- tural Similarity Index (SSIM) between the ground
(b) Original missing-wedger prism. Figure 5: Original missing-wedged shupes. truth and the generated results. Definitions of PSNR and SSIM are provided Appendix A We compare the corrupted datasets, the results corrected by IsoNet and DeepDeWedge, and those corrected by CryoGEN, As shown Table our method consistently outperforms the baselines.
5.2 SIMULATED DATA
this section, we applied our algorithm to more complex protein assemblies. Following the ap- proach of IsoNet, we first evaluated our performance the publicly available atomic model apo- ferritin (PDB:6Z6U) (Yip et al. 2020). Additionally, we selected the recently published electron microscopy dataset of C13 Vipp1 stacked rings LEMDB:18424 (Junglas et al. 2024).. The results show that CryoGEN delivers more consistent outcomes in both the spatial and Fourier domains. Additionally, CryoGEN demands significantly less training time compared to IsoNet.

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative evaluation of image quality for tomography reconstructions using different
methods, comparing PSNR and SSIM metrics (higher values indicating better performance for both metrics) on sphere, prism and Vippl assembly datasets.
Sphere
Prism
Vipp1 assembly
Data State
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
Corrupted
21.12
0.8113
14.82
0.6931
26.68
0.8000
Iso-corrected
22.98
0.8770
19.11
0.8857
27.12
0.8191
Dewedge-corrected
23.17
0.8824
21.10
0.9278
28.75
0.8758
CryoGEN-correect 29.19
0.9706
32.69
0.9949
30.65
0.9199
Apoferritin. We made reconstructions using the atomic model of apoferritin (# PDB: 6Z6U), widely-used benchmark high-resolution CryoGEN. The simulated maps were then randomly ro- tated in ten different orientations, and a missing wedge was imposed in Fourier space, producing subtomograms with missing wedge artifacts. In this setup, CryoGEN achieved noticeably superior performance compared to IsoNet and also substantially reduced the training time. These improve- ments are particularly evident when inspecting the low-density volume ChimeraX (Goddard et al. 2018), as illustrated in Figure] 7
(a) WBP reconstructed missing- wedged apoferritin, displaying both corrupted and missing gions the low-density volume. density volume.
(b) Structure generated by IsoNet, displaying corrupted region with visible inconsistencies in the low-
(c) Structure generated by Cryo- GEN, showing much smoother and coherent low-density volume representation.
Figure 7: Comparison of low- low-density volumes generated by WBP (a), IsoNet (b) and CryoGEN (c). The results indicate that IsoNet produces synthetic artifacts stemming from the reconstruction the missing wedge, which CryoGEN effectively resolves.
C13 Vipp1 Stacked Rings. We assessed our method using the recently published C13 Vipp1 stacked rings dataset (# LMMDB:1844), which features complex molecular assemblies. This dataset was obtained from the Electron Microscopy Data Bank (wwPDB Consortium) 2023).
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(a) IsoNettcorreeted Vippl assembly.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(b) CryoGEN-corrected Vippl assembly.
Figure 8: CryoGEN and IsoNet corrected Vipp assembly. The results shown in IsoNet exhibit strong streaking artifacts, In contrast, our method does not exhibit these streaking artifacts.
8

---

Published as a conference paper at ICLR 2025
Following the approach of IsoNet, we randomly rotated the tomograms generate ten distinct sam- ples before applying missing wedge corruption. As shown in T CryoGEN outperforms IsoNet, achieving higher PSNR and SSIM values. We present the original results in Figure 2 alongside spatial and Fourier domain comparisons in Figure [ In the Fourier domain, CryoGEN accurately captures essential details and produces more consistent and symmetrical results. Furthermore, while IsoNet's reconstruction still exhibits artifacts, CryoGEN produces significantly smoother and more accurate results with higher contrast in spatial domain.
5.3 REAL-WORLD EXAMPLES
In this section, we evaluate the method's effectiveness using real-world examples. We employ well-known Cryo-ET particle selection benchmark, specifically the dataset of purified ribosomes (Zhang et al., 2016), as well as the virus-like particle dataset of immature HIV-I for both single- particle and tomographic reconstruction (Schur et al., (016). CryoGEN significantly reduces the ringing effect and achieves higher contrast, while providing better compensation for the missing wedge compared to IsoNet's irregular distribution.
(a) csoNet-corrected purified ribosomes.
(c) CryoGEN-corrected purified ribosomes.
(b) Central Fourier slice of the IsoNet-corrected purified ribosomes.
(d) Central Fourier slice of the CryoGEN- corrected purified puriffied ribosomes.
Figure 9: Comparison of IsoNet-corrected and CryoGEN-corrected purified ribosomes, including their corresponding central fourier slices. The CryoGEN-corrette images exhibit higher contrast and reduced high-frequency features. While both methods effectively fill in the missing wedge, the IsoNet correction shows an irregular distribution of high-frequency components the central re- gion, whereas CryoGEN achieves more consistent distribution.
Purified Ribosomes. The ribosomes dataset is commonly used as a Cryo-ET benchmark. We col- lected all seven tilt series from the EMPIAR- 10045 dataset and applied the same preprocessing steps as IsoNet, detailed in Appendix A.8.5 Figure 9 shows the correction results for both IsoNet and CryoGEN. Ribosomes the CryoGEN cooreetted volume appear clearer and exhibit higher con- trast, which significantly aids in particle selection. Additionally, there is less noise and fewer sharp artifacts in the background. Notably, the IsoNet-corrected volume displays a frequency spectrum with irregular concentration of high-frequency components in the central region, introducing no- ticeable noise and artifacts. In contrast, the -oorGee-oorretee volume shows a much smoother and more consistent frequency distribution, with better control over central frequencies. The more symmetrical pattern suggests reduced distortion and better alignment with the expected smooth be- havior, indicating improved data integrity.

---

Published as a conference paper at ICLR 2025
660
1.0
0
0
80
(a) Issoee-ccoreetee immature HIV capsid.
(c) CryoGEN immature HIV capsid.
(b) Central Fourier slice of the IsoNet-corrected immature HIV capsid.
(d) Central Fourier slice of the CryoGEEN- corrected inmature HIV capsid.
Figure 10: A comparison IsaNet-corrected and CryoGEN immature HIV capsids, along with their corresponding central Fourier slices, reveals distinct differences. The IsoNet-corrected images exhibit noticeable ringing effects, whereas the CryoGEN--coreecke images feature minimal noise and a smoother background. Additionally, while both methods address the missing wedge region, IsoNet correction introduces more prominent streak artifacts.
HIV Capsid. The results of HIV capsid dataset are presented Figure Following IsoNet's pre- processing procedure, collected three tilt series from the EMPIAR-ION dataset and processed the volume as detailed in Appendix 6. The CryoGEN-corrected HIV capsid is noticeably clearer than the koNettcorrected version, with minimal noise and much smoother background. In trast, the IsoNettcorrected volume exhibits a pronounced ringing effect around the gold beads, with bright rings surrounding them and unwanted white dust scattered throughout the image. Our algo- rithm effectively eliminates all these artefacts. Additionally, CryoGEN compensates for the missing wedge region more effectively than IsoNet. As shown in the top windows of Figure (a) and Fig- ure (c), the virus particle in the CryoGEN-correctt compared to the IsoNet-corrected version. In the Fourier domain, while both methods attempt to the missing wedge region, IsoNet's correction introduces more noticeable line artefacts.
volume is more intact, with fewer defects
CONCLUSION
Cryo-ET serves indispensable tool for visualizing subcellular architectures in their native envi- ronments, yet its potential is constrained by anisotropic resolution artifacts arising from the missing- wedge effect. In this study, we present CryoGEN, innovative energy-based framework engineered to resolve the missing- wedge challenge in Cryo-ET. Our method achieves faster, more stable training convergence than existing approaches while significantly enhancing reconstruction fidelity.
ACKNOWLEDGMENT
The project was led by Dr. Teng, who oversaw its progression, designed the methodology, and implemented the algorithms. Guided by Dr. Teng during the internship, Yuxuan Ren refined the algorithms and performed the experimental validation. The authors would also like to extend their sincere gratitude to Prof. Zhouchen Lin and Dr. He Zhang for their exceptional feedback on this research, well Dr. Sha Hu for her assistance in refining the biological terms during the paper writing process. The authors also express the appreciation to Dr. Hui Wang and Dr. Jiayan Zhang for their motivational discussion and biological expertise.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Martin Arjovsky and Leon Bottou. Towards principled methods for training generative adversarial networks. In International Conference on Learning Representations, 2017. URL https://
M Beck and W Baumeister. Visualizing the molecular sociology at the hela cell nuclear periphery by cryo-electron tomography. Annual Review of Biochemistry, 85:1-18, 2016.
Tristan Bepler, Kotaro Kelley, Alex J Noble, and Bonnie Berger. Topaz-denoise: general deep denoising models for cryoem and cryoet. Nature communications, 11(1):5208, 2020.
Ludwig Boltzmann. Theoretical Physics and Philosophical Problems: Selected Writings, volume 5 of Vienna Circle Collection. Springer, 1974. ISBN -88890--77--287-88
Tim-Oliver Buchholz, Mareike Jordan, Gaia Pigino, and Florian Jug. Cryo-care: Content- aware im- age restoration for cryo-transmission electron microscopy data. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 502-506. IEEF, 2019.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International conference on machine learning,
pp. 1597-1607. PMLR, 2020.
Eli J. Cohen, Tina Drobnič, Deborah A. Ribardo, Aoba Yoshioka, Trishant Umrekar, Xuefei Guo, Jose-Jesus Fernandez, Emma E. Brock, Laurence Wilson, Daisuke Nakane, David R. Hendrixson, and Morgan Beeby. Evolution of large periplasmic disk in campylohacterodt flagella enables both efficient motility and autoagglutination Developmental Cell, (9(24))3306-33211 2024.
ISSN 1534-5807, doi: https://doi.ors orgp///.///////// 10. 10166f.devcc..2224.09000
Yuchen Deng, Yu Chen, Yan Zhang, Shengliu Wang. Fa Zhang, and Fei Sun. Icon: 3d reconstruction with maising-inffrrman Biology, 2016. ISSN 1047-8477.
restoration in biological electron tomography. Journal of Structural doi: https://ooi.or////
J. Diggle and R. J. Gratton. Monte carlo methods of inference for implicit statistical models. Journal of the Royal Statistical Society. Series B (Methodological), 46(2)) 1984.
Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and MCMC. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 8489-8510. PMLR, 23-29 Jul 2023.
Thomas D Goddard, Conrad C Huang, Flaine Meng, Eric F Pettersen, Gregory Couch, John H Morris, and Thomas E Ferrin. Ucsf chimerax: Meeting modern challenges in visualization and analysis. Protein science, 27(1):14-25, 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. volume 9906, pp- 694-711, 10 2016. ISBN 978-3-319-4447499. doi: 10. 1001 978-3-319-46475-662
Benedikt Junglas, David Kartte, Mirka Kutzner, Nadja Hellmann, Ilna Ritter, Dirk Schneider, and Carsten Sachse. Structural basis for vippl membrane binding: From loose coats and carpets to ring and rod assemblics. bioRxiv, 2024. doi: 10.1101//024....08.6222..
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, San Diega, CA, USA, 2015.
Yann Lecun, Sumit Chopra, and Raia Hadsell. A Tutorial on Energy- Based Learning. MIT Press, 003 0013
/ttp//111..511111 01 2006. URL httphtttss//edt http:///......1.....eeeeee/////

---

Published as a conference paper at ICLR 2025
Yun-Tao Liu, Heng Zhang. Hui Wang. Chang-Lu Tao, Guoqiang Bi, and Hong Zhou. Isotropic reconstruction for electron tomography with deep learning. Nature Communications, 13, 10 2022. doi: 10.1038/544447--223333999
Vladan Lucie, Friedrich Förster, and Wolfgang Baumeister. Cryo-electron tomography of cells: connecting structure and function. Nature Reviews Molecular Cell Biology, 6(6):432-439, 2005.
doi: 10.1038/nmm161..
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van
Gool. Repaint: Inpainting using denoising diffusion probabilistic models, 2022. URL https:
www.xi.orr/ab///// 09865
David N. Mastronarde. Dual-axis tomography: An approach with alignment methods that preserve resolution. Journal of Structural Biology, 120(3):343-335 1997. doi: 10. 100.0//siii.997.399999
Michael Radermacher. Weighted back-projection methods. Electron tomography: methods for three-dimensional visualization of structures in the cell, pp. 245-273, 2006.
F.K. Schur, M. Obr, W.J.H. Hagen, W. Wan, and J.A.G. Briggs. Structure of the immature hiv-1 capsid in situ by cryo-electron tomography and sub-tomogram averaging. Nature, 517(7535): 505-508, 2015.
Florian KM Schur, Martin Obr, Wim JH Hagen, William Wan, Arjen Jakobi, Joanna M Kirkpatrick, Carsten Sachse, Hans-Georg Kräusslich, and John AG Briggs. An atomic model hiv-1 capsid- sp1 reveals structures regulating assembly and maturation. Science, (566629)) 2016.
Dimitry Tegunov and Patrick Cramer. Real-time eryo- eey-electron microscopy data preprocessing with warp. Nature methods, 16(11): (144-1152,,, 2019.
Simon Wiedemann and Reinhard Heckel. A deep learning method for simultaneous denoising and missing wedge reconstruction in cryogenic electron tomography. Nature Communications, 15(1): 8255, 2024.
The wwPDB Consortium. Emdb- the electron microscopy data bank. Nucleic Acids Research, 52 (D1):D456-D465, 11 2023. ISSN 0305-1048. doi: 10. 10.109//a//gka10019.
Rui Yan, Singanallur V. Venkatakrishnan, Jun Liu, Charles A. Bouman, and Wen Jiang. Mbir: A cryo-et 3d reconstruction method that effectively minimizes missing wedge artifacts and restores missing information. Journal of Structural Biology, 2019. ISSN 1047-8477. doi: https://doi.org/
2..1016/.jsb.2219.03..
Ka Man Yip. Niels Fischer, Elham Paknia, Ashwin Chari, and Holger Stark. Atomic-resolution protein structure determination by cryo-em. Nature, 587(7832):157-16)) 2020.
Xiangrui Zeng, Yizhe Ding, Yueqian Zhang, Mostofa Rafid Uddin, Ali Dabouei, and Min Xu. Dual: deep unsupervised simultancous simulation and denoising for cryo-electron tomography. bioRxiv, 2024.
Chenwei Zhang, Anne Condon, and Khanh Dao Duc. Synthetic high-resolutiinn cryo-em density maps with generative adversarial networks, 2024. URL. htt.tttp:..//rrvi.orrr//b//////s 17674
Xing Zhang, Mason Lai, Winston Chang, Iris Yu, Ke Ding, Jan Mrazek, Hwee L LNg, Otto O Yang, Dmitri A Maslov, and Z Hong Zhou. Structures and stabilization of kinetoplastid-sseciiic split rmas revealed by comparing leishmanial and human ribosomes. Nature communications, 7(1): 13223, 2016.
Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 3-11. Springer, 2018.
12

---

Published as a conference paper at ICLR 2025
A
APPENDIX
A.1
ALGORITHM FLOWCHART
y
90
NED
Tu
Consistency Loss
Tu
90
(a) IsoNer
CryoET Imaging
TM
FT o gogy + +
Es
Complementary CryoET Imaging
Ty & gold +olgg++
TM
Cg
sel te)
n
y
TM
90
Th o goly + tg)
TM goly t.gg y
Rotation
avia)
Inverse Rotation
R-1
gor
Tar R
Gaussian Noise
||
og0 TM (gr(y) + cn)
(v)
Add
Stop Gradient
g0
R-1
CryoGEN
Figure 11: Algorithm
Algorithm.flowchare flowcharts of (a) IsoNet and (b) CryoGEN.
IsoNet formulates the task of restoring y' missing wedge information to reconstruct J an inpaint- ing problem Lugmayr al. (2022), solving it in self-supervised manner as illustrated in Figure (a). It trains a neural network, denoted as gg. following these steps:
1. A sample y is processed by to obtain a missing-weenee wedge-restored i = ge
2. The signal ă is rotated by a rotation operator R (randomly chosen from predefined set R). Then, simulated missing wedge operation TM is applied, yielding ÿ = TM R(T).
3. The simulated sample ÿ fed to ge. yielding reconstructed sample 3 = go (g).
4. The inpainted part is first extracted as f Ta (&), then combined with the rotated y with missing-wedee operation to obtain Tas R(y), and finally the inverse rotation is applied to yield ÿ Rt [(a (+)) + TM R(y)).
5. 9 and form a data pair (ū. y) to train where serves as the label.
6. These steps are repeated until convergence.
Similarly, details of CroGEN are provided in Figure Figure II (b), with the complete algorithm presented in Section 4.3
13

---

Published as a conference paper at ICLR 2025
A.2 ROTATION LIST DEFINED THE ISONET
The cropped subtomograms are cube-shaped with six faces, resulting in 24 possible rotations for reorientation. However, we exclude the four rotations that maintain the same missing wedge in the X-Z direction as the original, unrotated subtomogram. Further details and a schematic diagram are provided in the supplementary materials IsoNet (Liu al. 2022).
A.3 ADDITIONAL, RESULTS
A.3.1 ADDITIONAL. RESULTS FOR SECTION 5.1
Formally, represents the predicted volume, and x² denotes the ground truth. PSNR and SSIM are defined as follows:
Im
(2patte (2) + C2) (ug + 1Lz C1)(og + o. + C2)
PSNR 10 logii
SSIM u*)
10
In the PSNR formula, In represents the maximum possible pixel value, which we define as the maximum value of the ground truth image.
In the SSIM formula:
Hg and pa- are the mean intensities images and o
o and o. are the variances images and V*
Oen the covariance between images i and y
C1 and C2 are small constants used to stabilize the division when the denominator is close to zero
The central Fourier slices of the corrected sphere and prism are displayed in Figure 12 while the original clean sphere and prism are shown in Figure 13
(a) Central fourier slice the IsoNet-correctd sphere.
(b) Central fourier slice of the IsoNet-corrected prism.
Central fourier slice of the CryoGEN-coree corrected sphere.
(d) Central fourier slice of the CryoGEN prism.
Figure 12: Central fourier slices of sphere and prism. The IsoNet's corrected has less information
both at low and freeffryyy signals with missing regions, while CryoGEN fills in most of the missing wedge. It is consistent with the spatial domain results.
14

---

Published as a conference paper at ICLR 2025
Central Fourier Slice
Central X-Y Slice
Central Slice
Central Y-Z Slice
(a) Orignal clean prism.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(b) Original clean prism.
Figure 13: Original clean shapes.
Compared to the corrected tomograms, the CryoGEN-coorected versions more closely resemble the original clean shapes.
A.3.2
ADDITIONAL RESULTS FOR SECTION 5.2
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
Original missing wedged apoferritin.
Central Fourier Slice
Central XY Slice
Central X-Z Slice
Central Y-Z Slice
IsoNet-corrected apoferritin.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(c) CryoGEN-corrected apoferritin.
Figure 14: Missing wedged, CryoGEN and Iso-Net corrected apoferritin.
15

---

Published as a conference paper at ICLR 2025
We present the central X-Y, X-Z slices, well as the central Fourier slices for the corrupted, IsoNet-corrected and CryooEN-coree cooreeced volumes in Figure [4 In the IsoNet-corrected X-Y slice, there faint white artifacts in the background, consistent with Figure 7 and line artifacts in the X- Z slice, which are absent the CryoGEN-coreette results. Additionally. the central Fourier slice of the Isoorrecoeee volume displays distinct borderline, which present in the CryoGEN- corrected slice.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central
Y-Z Slice
Figure 15: Original clean Vippl assembly.
The original clean Vippl assembly are shown in Figure 15
A.3.3 RECONSTRUCTION FROM NOISY SAMPLES
We also demonstrated the robustness of our algorithm under higher noise levels, such as SNR=0.2. The results presented in Figure T6 Even in this challenging scenario, CryoGEN outperforms IsoNet, demonstrating its strong denoising capabilities.
Central Fourier
Central Slice
Central Slice
Central Y-Z
(a) Original noisy Vippl assembly.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(b) IsoNet-corrected Vipp/ assembly.
Central Fourier
X-Y Slice Central Slice
Central Y-Z
(c) CryoGEN-correcte Vippl assembly.
Figure 16: CryoGEN and IsoNet corrected noisy Vippl assembly SNR=0.2.
16

---

Published as a conference paper at ICLR 2025
A.4 ARTIFACTS
In this section, summarize the artifacts observed during experiments, including streak arti- facts, synthetic artifacts, and ringing artifacts. Throughout all experiments, CryoGEN consistently outperforms the baseline methods.
A.4.1 STREAK ARTIFACTS
Streak artifacts commonly appear around dense materials, such as gold particles, in Cryo-ET recon- structions, primarily due to two factors: (1) the restricted tilt angle range during data acquisition, and (2) the sensitivity of reconstruction algorithms, such as WBP (Radermacher] (2066). high-intensity variations (see Figure D. These bright spots generate ripples or streaks during the back-projection process, resulting in visual distortions that degrade the overall image quality.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
IsoNet-corrected Vippl assembly.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(b) CryoGEN-corrected Vippl assembly. Figure 17: CryoGEN and IsoNet corrected Vippl assembly.
Although these artifacts are clearly depicted in Figure 17 their visibility may be constrained by the size of the gold nanoparticles and ribosomes in Figure To address this, we provide an additional view in Figure 18, where the artifacts are more noticeable.
(a) IsaNet-corrected purified ribosomes.
(b) Deeppeeeeleecooreeteed purified ribosomes.
(c) CryoGEN-correted purified ribosomes.
Figure 18: Comparison of IsoNet- and CryoGEN-corrected purified ribosomes.
17

---

Published as a conference paper at ICLR 2025
A.4.2 SYNTHETIC ARTIFACTS
Synthetic artifacts refer unintended distortions features that emerge in generated images or data, often resulting from generative models trained on datasets with incomplete distributions. As shown in Figure 19 the IsoNet method introduces synthetic artifacts caused by the reconstruction of the missing wedge, whereas the CryoGEN approach effectively resolves these issues.
(c) CryoGEN. Figure 19: Comparison of lew-iiy volumes generated by WBP (a), IsoNet (b) and CryoGEN (c).
(a) WBP.
(b) IsoNet.
A.4.3 RINGING ARTIFACTS
Ringing artifacts are undesirable distortions that appear as bands, typicallynear sharp edges to- mographic imaging. These artifacts often arise from reconstruction using band-limited signals that lack the high-frequency components required to accurately capture sharp transitions. In experiments with both purified ribosomes and immature HIV capsids, CryoGEN outperforms baseline methods, displaying minimal ringing artifacts.
(a) IsoNet-corrected purified ribo- somes.
(b) DeepDeWedge- puri- (c) CryoGEN-corrected purified ribosomes.
fied ribosomes.
Figure 20: A comparison of purified ribosomes corrected by IsoNet, De DeWedge, and CryoGEN.
(a) IsoNet-corrected immature HIV capsid. Figure 21: comparison of immature HIV capsids corrected by IsoNet, DeWedge, and CryoGEN.
(b) DeepDeWeege-corrette imma- ture HIV capsid.
CryoGEN-corrected immature HIV capsid.
18

---

Published as a conference paper at ICLR 2025
A.5 ADDTIONAL RESULTS BY DEEPDEWEDGE
First, we present the WeepDDe-edge-ooreette simple shapes in Figure 22 where defects similar those in the IsoNet-corrected versions are apparent. Both methods exhibit spatial domain artifacts, resulting in distorted volumes, and neither DeepDeWedge nor IsoNet effectively fills the missing information in Fourier space.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(a) DeepDeWedge-ooreeee sphere.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(b) DeepeeWddge-oorrette prism.
Figure 22: DeepDeeWedg--ooreetee shapes.
Next, the DeepDe weegeeeeede-cooreeeetdd simulated data is displayed in Figure 23 Similar IsoNet, DeepDeWedge encounters the same issues, generating faint shadows and distinct line artifacts in the background, well noticeable borderline in the central Fourier slice.
Finally, we test the DeepDeWedge on the real-world examples as shown in Figure 24 The DeepDeWedge-correee
ribosomes exhibit the same irregular distribution of high-frequency com- ponents in the central region and yield unsatisfactory results for the HIV capsid, with noticeable artifacts. A potential reason for the poor performance on the HIV capsid may be the loss of infor- mation caused by even-odd splits.
Central Fourier Slice Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(a) DespDeWedde-corrette apoferritin.
Central Fourier Slice
Central X-Y Slice
Central X-Z Slice
Central Y-Z Slice
(b) WeepDeeWedee-omeeete Vippl assembly.
Figure 23: DeepDeWedge-correetee simulated data.
19

---

Published as a conference paper at ICLR 2025
(a) DeepDeWedo purified ribosomes.
(c) DbepDeWedge-ooreeee HIV capsid.
(b) Central fourier slice of the DeepDeWedge- corrected purified ribosomes.
(d) Central fourier slice of the DeepDeWedge- corrected HIV capsid.
Figure 24: DeepDeWedge-coreecte ribosomes and HIV capsid, along with their corresponding central Fourier slices.
A.6
VISUALIZATION OF A SINGLE RIBOSOME
In Figure 25 we present visualization of individual ribosome. The average of all ribosomes is compared to those reconstructed by IsoNet and CryoGEN, using the fitmap command in ChimeraX for rigid-body optimization to align the maps. While the IsoNet reconstructions may appear to hibit more details in 2D grayscale images, as shown in Figure this effect likely due to undesired noise synthetic artifacts, highlighting the superior denoising capabilities of CryoGEN.
CryortN
Furthermore, we compute the corre- lation between the ribosome recon- structions generated by IsoNet and CryoGEN with the average, obtain- ing correlation coefficients of Ploatar 0.1792 and PeTeGEN 0.5134, re- spectively (PeyuGEN > PtuNa). Both qualitative and quantitative results demonstrate that CryoGEN outper- forms IsoNet, providing smoother re- constructions.
AvSTAgE
A.7 IMPLEMENTATION DETAILS
Following struct2map GAN (Zhang jet al. 2024), the architecture consists of generator and discriminator with specific design choices. The gen- erator is modified U-Net architec- ture, known as U-Net++ (Zhou et al.) 2018), which enhances the standard U-Net with dense skip connections for improved performance. The discriminator is composed of four 3D convolutional layers, each using a 3x3x3 3 kernel. Following the final convolutional layer, an adaptive average pooling layer reduces the dimensions of the feature map to 1x This output then flattened and fed into series of three fully connected layers, with ReLU activations between each layer. The final layer produces single output, which serves as the result binary classification.
Figure 25: Single Ribosome Visualization: We compared CryoGEN with IsoNet and the averaged results of all ribo- somes across two examples at different rotation angles.
20

---

Published as a conference paper at ICLR 2025
The CryoGEN are trained with the Adam optimizer (Kingma & Ba 2015) with batch size for simulated shapes and protein subtomograms and with batch size for real-world examples. The learning rate is set to 0.0004 with linear warm-up phase in the initial one-tenth steps, which is followed by a linear decay schedule thereafter. Different from IsoNet, which progressively increases the noise scale, we apply random noise levels across all training steps. Specifically, random number is sampled from a uniform distribution within the range (0, 1J and multiplied by the set noise scale for each step. Additionally, the penalty term À is kept constant during the first epoch and then decays linearly throughout the subsequent epochs.
A.8 EXPERIMENT DETAILS
A.8.1 SPHERE
hollow sphere with an outer diameter of 70 pixels and a thickness of pixels is positioned at the center of 140x x40x 140 volume. Corruption is applied by setting values zero within the missing wedge angles in Fourier space. For training, the volume is split into ten 96x96x 96 pixel subtomograms with randomly chosen origins. These subtomograms are then randomly cropped to 64x64x64 pixels before being input into the models.
A.8.2 PRISM
A prism with a thickness of 20 pixels is placed inside 96x96x 96 volume. It is randomly rotated in ten directions. Corrupted prisms are generated by setting zero values within the missing wedge angles in Fourier space. The entire volume is directly fed into the model during training.
A.8.3 SIMULATED APOFERRITIN
Ten randomly rotated apoferritin datasets are downloaded from a link provided by IsoNet and gen- erated using ChimeraX's molmap function. During training, the datasets directly fed into the model without further modifications.
A.8.4 SIMULATED STACKED RINGS
C13 Vippl stacked ring data are downloaded from the EMDB database and binned twice, resulting in x00 x00xx000000 pixels. The data is randomly rotated in ten different directions. Corrupted stacked rings are generated by setting zero values within the missing wedge angles in Fourier space. For training, the data is split into 96x96x 96 pixel subtomograms with random zero origins, then randomly cropped to 64 x x4 64 pixels before being fed into the models.
A.8.5 RIBOSOMES
Ribosome data is downloaded from the EMPIAR database and binned six times, yielding a pixel size of 13.02 A. IsoNet's deconvolution is applied, following the same procedure as described by (Liu et al. 2022). To ensure that subtomograms contain sufficient data, IsoNet's mask generation tool used extract subtomograms with at least 40% non-zero pixels based the density mask. For training, a tomogram is split into seventy 80x80 80 pixel subtomograms, resulting in a total of 490 subtomograms. These are randomly cropped to 64 x4x46 pixels before being fed into the models.
A.8.6 HIV CAPSID
Raw tilt series for the HIV capsid is downloaded from the EMPIAR database. The movie stacks drift-corrected and reconstructed using the WBP algorithm, aided by the latest tomogram processing tools such Aretomo2. The processed tomograms, TS-01, TS-43, and TS-45, are then deconvolved following the procedure outlined by (Liu et al. 2022). IsoNet's mask generation tool is applied ensure that each subtomogram contains tomogram is split into one hundred 96x 96 96 pixel subtomograms, resulting in 300 subtomograms. These are randomly cropped to 64 x x 64 pixels before being fed into the models.
least 50% non-zero pixels. During training, each
21


Reviews:
[{"review_id": "mO3q42vNzg", "reviewer": "Reviewer_dAoL", "summary": "This paper proposes a novel deep learning based method to solve the missing wedge problem, which is an inverse problem that arises in cryogenic electron tomography (cryo-ET). \n\nIn cryo-ET, the goal is to reconstruct the 3D density of a biological sample (e.g. a cell) from a set of 2D projections of the density. Due to limitations during acquisition, there is a wedge-shaped region of viewing directions where no projections can be recorded. As a result, the set of projections does not uniquely determine the 3D density of the sample. \n\nThe authors propose a method called CryoGEN that aims to fill in the missing data of the missing wedge. The method is self-supervised (does not require clean ground truth for training) and fits a missing wedge reconstruction network directly to the tomogram(s) whose missing wedge is to be filled. The fitting of this reconstruction network is regularized by an energy function, represented as another neural network, which is learned together with the reconstruction network in an adversarial manner.\n\nBased on a theoretical motivation, and experiments on synthetic data and two real-world tomograms, the authors argue that the energy function yields improved missing wedge reconstruction performance over two closely related state-of-the-art missing wedge methods (IsoNet and DeepDeWedge). \n\n\n**Recommendation:** I do not recommend publishing the paper at ICLR *in its current form*. This is mainly because I found parts of the paper (especially the motivation and training of the energy function and the construction of the losses) unclear. However, since the missing wedge problem is important, and since CryoGEN performs better than state-of-the-art baselines in some cases. \nI recognize the potential of the method and am willing to raise my score if the authors better explain their method and why it outperforms the baselines (IsoNet and DeepDeWedge).\n\n**Post rebuttal:** I have increased my score from **5 to 6**, as the authors addressed most of my concerns (see discussion below). Overall, I think that the most important contribution of the paper is the observation that equipping methods that work like IsoNet or DeepDeWedge with GAN-like energy nets to can improve the missing wedge reconstruction performance.", "questions": "- Why is the energy function learned in an adverserial way (Equation (9))?\n- Does the adverserial training of the energy function lead to any instabilities during training?\n- Why does the \"Posterior\" loss (Equation 8) involve random rotations?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- CryoGEN outperforms IsoNet on the clean synthetic data (Section 5.2) and gives a cleaner reconstruction of the (real) HIV capsids (Figure 10).\n- The missing wedge problem addressed in the paper is important and challenging.\n- The authors applied CryoGEN to two real-world tomograms, demonstrating the practical applicability of the method.", "weaknesses": "- I cannot really follow the motivation in Section 3, where the authors argue that IsoNet produces an average of two 3D densities $x_1$ and $x_2$ that produce the same measurement $y_0$ under the forward map $\\mathcal{T}_M$. The following is unclear to me:\n\t- Why do the authors \"choose\" a Gaussian mixture prior for IsoNet (lines 170 - 171)? If I understand correctly, IsoNet does not use a prior for regularization.\n\t- What is the concrete energy function ($E(x) = - \\log p(x)$) that gives a better solution?\n\t- Why does adding noise to the measurements $y$ solve the problem that there are many $x$ that map to the same $y$ (lines 194 - 199)?\n\n- I do not understand the motivation and intuition behind the consistency loss (equation (7)). Why is the reconstruction mesh $g_\\theta$ applied to y and then again after $\\mathcal{T}_M$?\n\n- The authors give no intuition as to why the energy function (which looks like a prior to me) can be learned directly on the data one wants to reconstruct. I would also appreciate more explanation as to why the energy function is learned in an adversarial manner (equation (9)).\n\n- The authors argue that CryoGEN outperforms IsoNet on the purified ribosome tomogram (Figure 9), but I find the CryoGEN tomogram visually much less appealing. It seems to contain less detail and the contrast of the ribosomes looks too strong to me. (Since there is no ground truth, it is impossible to measure which reconstruction is actually better). \n\n- The experiments on synthetic data (Section 5.2) are done on noiseless data. However, the low signal-to-noise ratio (as low as <10%) is one of the main challenges in cryo-ET. Therefore, it would be good to include an experiment that demonstrates how CryoGEN handles strong noise. Synthetic data is suitable for such an experiment because clean ground truth is available.\n\n- In the abstract, the authors state that IsoNet suffers from model collapse as it updates its own training data. This aspect is not discussed in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iUiTnRlT6m", "reviewer": "Reviewer_PzBU", "summary": "This paper introduces a new method for reconstructing 3D structures from cryo-electron tomography (Cryo-ET) data. It aims to address a common issue in Cryo-ET reconstruction known as the missing wedge problem. The proposed algorithm, CryoGEN, tackles this by using an energy-based model that learns to generate realistic 3D structures from incomplete information. The model includes two main components: an energy model, which scores how realistic a structure is, and a prediction model, which reconstructs the missing regions. CryoGEN also adds noise to the input data during training to handle the challenges posed by the one-to-many mapping issue. This approach enables the model to create accurate and clear reconstructions by filling in missing areas based on learned patterns from data. The paper uses four datasets to evaluate the effectiveness of CryoGEN, particularly in comparison to the baseline solution, IsoNet. Results show that CryoGEN produces clearer and more complete reconstructions with better preservation of structural details. On three of the datasets, quantitative evaluation is performed, demonstrating that CryoGEN surpasses IsoNet.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper addresses the missing wedge problem, a major issue in Cryo-ET that causes incomplete and blurred 3D reconstructions.\n\n2. The paper proposes a novel deep learning solution for Cryo-ET reconstruction. Using GANs and energy models is a reasonable approach.\n\n3. Diverse datasets are considered for evaluation, including both simulated and real tomograms.", "weaknesses": "1. The techniques used in CryoGEN are fairly standard in general computer vision. There doesn’t appear to be any part of the algorithm specifically motivated by the unique characteristics of Cryo-ET data, such as the missing wedge. For example, the motivation discussed in Section 3 is very common in general generative models like VAE and EBM. Also, EBM and GAN models have been widely used in general inpainting problems but not discussed as related work. \n\n2. It's unclear why Equation (8) represents the posterior distribution. Is Equation (8) an implementation of Equation (3)?\n\n3. This paper improves upon IsoNet by incorporating the energy model and noise perturbation on the input. Which of these additions is more essential to the strong performance of CryoGEN?\n\n4. Regarding computational complexity, the paper provides inconsistent information. Section 5.3 reports that CryoGEN is much faster than IsoNet. However, CryoGEN is clearly more complex than IsoNet in terms of architecture. The authors also mention that the energy model converges more slowly in Section 4.1.\n\n5. As mentioned in related work, DeepDeWedge can be used to address denoising and missing wedges simultaneously. Why is DeepDeWedge not included in the quantitative evaluation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HQi7oVZZ3l", "reviewer": "Reviewer_gbuV", "summary": "The paper addresses the limitations of Cryogenic Electron Tomography (Cryo-ET) in reconstructing 3D structures of cellular components due to the missing wedge problem, which creates anisotropic resolution in tomograms. The authors introduce CryoGEN, a generative energy-based model designed to tackle the missing wedge problem more effectively and stably, without requiring recursive prediction.", "questions": "1. What is the SNR of the data shown in Figure 7? I’m curious to see 3D reconstruction results (comparison with IsoNet) with higher noise (SNR 0.01 or 0.001) levels.\n2. On page 6: \"At the inference stage, we begin by cropping the complete tomogram into multiple overlapping subtomograms\" – Could you specify the number of crops used?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The use of energy-based models for addressing the missing wedge problem is novel and shows promise in improving reconstruction quality.\n2. CryoGenic is computationally efficient, achieving significant runtime reductions compared to IsoNet, which would be valuable in large-scale biological studies\n3. The paper is well-written and easy to follow", "weaknesses": "1. Couldn't find big technical issues.\n2. A more detailed discussion would be needed (e.g., limitations, what would be the future work).\n3. Typo: Page 1, line 32 – Repeated phrase \"insights insights into.\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DOqDRrGKjD", "reviewer": "Reviewer_wTW8", "summary": "CryoGEN's author(s) proposed a new method called CryoGEN, aimed at addressing the \"missing wedge\" problem and the challenge of low signal-to-noise ratio in cryo-electron tomography (Cryo-ET). CryoGEN combines generative adversarial networks (GANs) with energy-based models to produce more consistent and high-quality 3D reconstructions. Compared to traditional weighted back-projection (WBP) and existing deep learning methods like IsoNet, CryoGEN effectively fills in missing information and reduces blurring artifacts.\n\nKey contributions :\n\n1. The introduction of CryoGEN, which uses energy-based models to handle multiple possible solutions, avoiding the blurring effect caused by simple averaging.\n\n2. The inclusion of consistency loss and noise injection to ensure that generated images maintain fidelity to the original data while preserving diversity in the results.\n\n3. Experimental validation on multiple datasets, such as HIV viral particles and neural synapses, where CryoGEN outperforms existing baseline methods in metrics like peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Fourier shell correlation (FSC).\n\nIn all, the authors announces a more effective solution for reconstruction in Cryo-ET, showing significant improvements in quality and stability. Future work includes improving training efficiency, automating parameter adjustments, and extending its applications to other cryo-electron microscopy domains.", "questions": "1. Lack of Comparative Experiments: It is suggested to add comparisons with other recent methods, particularly with more challenging and complex models, to better validate the advantages of CryoGEN.\n\n2. Supplemental Theoretical Explanation: Could the authors provide more detailed explanations regarding the implementation details of the energy model? Adding intuitive examples and diagrams would help readers understand its complexity.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: The originality of the paper lies in its attempt to improve and combine existing methods by using generative adversarial networks (GANs) and energy-based models to address the missing wedge problem in cryo-electron tomography (Cryo-ET). \n\nQuality: The quality of the research is validated through experiments on multiple datasets, including synthetic samples and real biological samples (e.g., HIV viral particles and neural synapses). However, the experimental design lacks in-depth comparison with other state-of-the-art methods, especially the absence of sufficient ablation studies to demonstrate the importance of each model component.\n\nClarity: The paper is generally well-written with a logical flow, but some parts are explained too briefly, particularly the mathematical description and implementation details of the energy model, which might make it difficult for non-expert readers to fully understand the working principles.\n\nSignificance: The potential impact of CryoGEN in the field of cryo-electron tomography is limited. Although the method improves reconstruction quality to some extent, the degree of improvement is relatively modest, and challenges may arise when applied to more complex datasets.", "weaknesses": "Lack of Novelty: Although CryoGEN combined GANs and energy-based models, this architecture still relies on existing methods and lacks truly groundbreaking innovation. For instance, the combination of GANs and energy-based models has already been widely applied in other fields, and the paper does not sufficiently demonstrate the unique contribution of this combination in Cryo-ET. We recommend citing some recent literature, such as \"Your GAN is Secretly an Energy-based Model and You Should Use It\" by Tong Che et al. (2020), which discusses the connection between GANs and energy-based models and their effectiveness in data reconstruction. Furthermore, it is suggested to elaborate on the specific aspects where this combination lacks novelty in Cryo-ET applications, such as similarities in loss functions, model architecture redundancy, or training strategy limitations, and clarify how CryoGEN distinguishes itself from existing methods to enhance the contribution statement of the paper.\n\nIncomplete Theoretical Explanation: The mathematical description and implementation details of the energy model are overly simplified. It is recommended to provide a more detailed description of the energy model's training process, including key equations and their roles in optimizing the model. Adding intuitive examples or visual aids, such as diagrams illustrating the training dynamics of the energy model, would make the content more accessible to non-expert readers. Specifically, expanding on equations (3) and (4) would help clarify how the energy model interacts with the GAN during training. These additional details would help address potential confusion and make the feedback more actionable.\n\nLack of In-Depth Result Analysis: Although the experimental results show performance improvement, the authors do not provide in-depth discussion on the reasons behind these key performance improvements. It is recommended to provide a more detailed analysis of why CryoGEN performs better on specific datasets compared to other methods, which would better support the validity of its contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)