DRACO: A Denoising-Reconstruction Autoencoder for Cryo-EM
Authors: YingJun Shen
Haizhao Dai
Qihe Chen
Yan Zeng
Jiakai Zhang
Yuan Pei
Jingyi Yu
Venue: NeurIPS 2024 poster (2024)
Keywords: masked image modeling
foundation model
cryo-EM
denoising autoencoder
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Foundation models in computer vision have demonstrated exceptional performance in zero-shot and few-shot tasks by extracting multi-purpose features from large-scale datasets through self-supervised pre-training methods. However, these models often overlook the severe corruption in cryogenic electron microscopy (cryo-EM) images by high-level noises. We introduce DRACO, a Denoising-Reconstruction Autoencoder for CryO-EM, inspired by the Noise2Noise (N2N) approach. By processing cryo-EM movies into odd and even images and treating them as independent noisy observations, we apply a denoising-reconstruction hybrid training scheme. We mask both images to create denoising and reconstruction tasks. For DRACO's pre-training, the quality of the dataset is essential, we hence build a high-quality, diverse dataset from an uncurated public database, including over 270,000 movies or micrographs. After pre-training, DRACO naturally serves as a generalizable cryo-EM image denoiser and a foundation model for various cryo-EM downstream tasks. DRACO demonstrates the best performance in denoising, micrograph curation, and particle picking tasks compared to state-of-the-art baselines.

DRACO: A Denoising-Reconstructinn Autoencoder for Cryo-EM
Yingjun Shen 1.2"
Haizhao Da
Qihe Chen
Yan Zeng Jiakai Zhang Yuan Peit.3 Jingyi Yu²
School of Information Science and Technology, ShanghaiTTcc University.
"Cellverse Co. Ltd.
Huuman Institute, ShanghaiTech University.
(s2eny]2022, 2aahhhh20233@shanghhaa (alengh2024,zengyaaa zhangjk, peaanngjkpeiyuuueee puan, wuwinyyyljssaannnnn.y
edu.
edu, cn
Abstract
Foundation models in computer vision have demonstrated exceptional performance in zero-shot and few-shot tasks by extracting multi purpose features from large- scale datasets through self-supervised pre-training methods. However, these models often overlook the severe corruption in cryogenic electron microscopy (cryo-EM) images by high-level noises. We introduce DRACO, a Srooisinn-Recoostruutiioo Autoencoder for CryO-EM, inspired by the Noise2Noise (N2N) approach. By processing cryo-EM movies into odd and even images and treating them as inde- pendent noisy observations, we apply denoising- reconstruction hybrid training scheme. We mask both images to create denoising and reconstruction tasks. For DRACO's ' pre-training, the quality of the dataset is essential, hence build high-quality, diverse dataset from an uncurated public database, including over 270,000 movies micrographs. After pre-training, DRACO naturally serves as generalizable cryo-EM image denoiser and foundation model for various cryo-EM downstream tasks. DRACO demonstrates the best performance in denois- ing, micrograph curation, and particle picking tasks compared to state-of-the-art baselines.
Introduction
Foundation models computer vision have demonstrated remarkable capabilities in zero-shot and few-shot tasks. These models learn to extract multi-purpose visual features from large-scale, diverse datasets through text-guided 2 3 or self-supervised 4 [ pre-training methods such as masked image modeling (MIM) 6 The features can then be applied to various downstream tasks. For instance, DINOv2 (S] is trained on a large-scale curated dataset and shows significant performance improvements in classification, retrieval, segmentation, etc. The success of vision foundation models has stimulated advances across various scientific disciplines. Due to the diverse modalities of scientific imaging, training domain-specife, foundation models [7] g 9 10 is essential meet specific demands. For example, the UNI 7 foundation model for tissue imaging is pre-trained on 100 million images for 34 representative clinical downstream tasks.
In structural biology, cryogenic electron microscopy (cryo-EM) stands as pivotal bio-imaging technique I1. Unlike optical imaging methods, cryo-EM possesses several distinctive characteristics: first, eryo-EM utilizes high-energy electron beams as illumination source [2] and direct detector
'The authors contributed equally this work.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Pre-training Denoising-Reccnstrucie
Public Database
Natue-to Nchb
D
Trainable Parameters
Misk Garraralie
Frocen Paramnteers
Ocdirumes
Fituring & Verifring
Raw Mirie
Micrograph Denoising
Annotated
Adaption
Datasets
Process Nondtaa
Micrograph Cleaning
MI eongrann
Encoder
Raw Misoci
Particle Picking
Large-scale Curated Cryo-EM Dataset (530 biggst modeccess 272,000 erograghn tom)
Labels
Figure I: Overview of DRACO. For pre-training, we construct a large-scale curated dataset con- taining 529 types of protein data with over 270,000 cryo-EM movies or micrographs. Based on this, we present DRACO, a naturally serves generalizable cryo-EM image denoiser and foundation for various downstream model adaptions such as micrograph curation and particle picking.
autoencoder for cryo-EM. A pre-trained DRACO
device (DDD) captures continuous multi-frame sequence, often called a movie 13] specimen damage during exposure, the electron dose per frame is restricted, leading to extremely low signal-to-noiss ratios (SNR) in the captured images. Second, cryo-EM employs motion correction to counteract blurring induced by specimen drift during exposure to ultimately obtain a sharper single micrograph [3] Last, the acquired images comprise hundreds of thousands of target protein particles with diverse poses. To resolve 3D structures, researchers utilize pipeline composed of multiple downstream tasks including micrograph curation, micrograph denoising, particle picking, pose estimation, and ultimately high-resolution reconstruction.
To mitigate
In line with denoising autoencoders [1], which consider "robustness to partial destruction of the input" a good representation criterion, existing self-supervised learning methods, such as masked au- toencoders (MAE) [6], have been successful in learning expressive representations by reconstructing the missing patches from partially masked image. However, in cryo-EM, these methods overlook the severe corruption caused by pixel-level random noise, leading to degraded performance. To be more robust to noises, DMAE [I5] reconstructs the clean image from the masked one that is further corrupted by synthetic Gaussian noise. Nevertheless, cryo-EM clean reference images are impossible to obtain due to the fragile biological specimens, posing a significant challenge.
In this paper, we present DRACO, shown in Figure [ Inspired by Noise2Noise (N2N) [16), which learns to denoise images using only paired noisy images, we divide the original movie into two sub-movies based on odd and even frame numbers, processing them into odd and even images. We treat them as two independent, noisy observations of the underlying true signal, thus the idea of N2N can apply. During training, we partially mask both images, creating masked regions and unmasked regions corresponding to denoising and reconstruction tasks: in the unmasked region, the odd noisy patch learns recover the even noisy patch, and vice versa. In the masked region, we introduce relatively low-noise images from the complete movie to supervise the reconstructed results. This denoising- reconstrttnnn hybrid training scheme achieves the robust feature extraction of noisy cryo-EM images.
Sonooisinn-Rcconstuuttttoo Autoencoder for CryO-EM, as
The quality of training samples is crucial for the general-purpose feature extraction of cryo-EM images. Direct access to the public database leads to varying data quality, inconsistent data formats, or missing annotations. Therefore, cryo-EM image dataset by curating and manually processing 529 sets of data from EMPIAR [17), obtaining over 270,000 cryo-EM movies or micrographs in total. After pre- pre-training. DRACO naturally serves generalizable cryo-EM image denoiser and foundation for various downstream model adaptions. We hence explore the performance DRACO on three downstream tasks: micrograph curation, denoising, and particle picking. Extensive experiments show that DRACO outperforms the ofate-of-the-art baselines models, and the large-scale curated dataset.
construct a large-scale, high-quality, and diverse single-particle
downstream tasks. We will release code, mre-trainedffine-mn

---

2
Related Work
Our work aims extend the vision foundation model the field of cryo-EM. We therefore only discuss the most relevant works in respective fields.
Vision foundation models in computer vision. Vision foundation models are pre-trained on large- scale image datasets [8] 19 using self-supervised learning methods [20]. aimed at extracting general
visual signals rapidly adaptable to various downstream visual tasks [2 23 24
Techniques for pre-training vision foundation models, such as contrastive learning 26 27 and self-distillation
28 51, focus on aligning features across different models or modalities, while another method,
masked image modeling [6] 29 (30 4]. reconstructs features from masked images to capture high- level visual semanties. However, these existing vision foundation models are not directly applicable to cryo-EM imaging. In particular, their application to cryo-EM imaging is limited by the high noise levels in micrographs, which degrade signal capture. Therefore, we propose denoising- reconstruction pre-training framework that is robust to highly noisy cryo-EM micrographs, making it suitable for specific downstream tasks in cryo-EM.
Vision foundation models in life science. The remarkable success of foundation models has extended to various life science imaging domains, including applications in retinal [0), fluorescence microscopy (9]. histopathology 7 [], and radiology imaging BI]. These models have shown considerable effectiveness in tasks such as disease diagnosis, lesion detection, and image restoration within these fields. In contrast to these domains, which benefit from extensive and well-curated datasets supporting model training, the cryo-EM field lacks such resources. To fill this gap, we have developed a well-curated, large-scale dataset specifically designed to support the training of cryo-EM foundation models, ensuring that they can be effectively applied in this specialized field.
Cryo-EM image denoising. To tackle the issues of low SNR and complex noise patterns in cryo-EM images, traditional denoising techniques often employ noise models like the
model 35 to denoise. However, these methods oversimplify the noise patterns, which can lead to the loss of high-frequency signal details. Recently, NT2C
12
32
and rely on filtering methods 33 34
36 uses Generative Adversarial Network to learn the noise patterns for denoising, but it requires simulated datasets as the clean references. Another series of learning- full use of multi-frame data by generating odd and even images for denoising based on Noise2Noise (N2N) [6 39 framework. These methods do not require clean images for denoising, but they still suffer from small-scale datasets and network architectures, which limit their generalizability. In this paper, we propose DRACO, pre-trained on large-scale curated dataset, that can naturally serve as a generalizable denoiser for cryo-EM micrographs.
methods [37 38 make
Downstream tasks in single particle analysis. An effective foundational model can benefit down- stream tasks in cryo-EM, including micrograph curation and particle picking. Micrograph curation aims to ensure that only high-quality images selected for further analysis, yet current methods rely heavily on manual inspection 44) [41). Particle picking involves identifying and extracting representative particles from micrographs, which critical task in the cryo-EM single particle analysis (SPA) reconstruction pipeline. Traditional methods 42 43
44] such as template matching 45 46 and difference of Gaussians (DoG) method [47). heavily rely on prior information and require substantial ad hoc post-processing. Learning- based models 48 49 50), such as Topaz-Picking [5]], crYOL [52], and CryoTransformer [53), offer more streamlined processes but still face challenges in generalizability due to the limited data scale. Our DRACO, pre-trained on large-scale cryo-EM image datasets, can effectively adapt to these tasks and demonstrate strong generalization capahilities.
3 Preliminary: Imaging Formation Model
Cryo-EM uses a Direct Detector Device (DDD) camera 132 for their notably higher detective quantum efficiency (DQE) compared to traditional cameras. This allows recording the micrograph as multi-frame movie rather than single integrated exposure. In this setup, a movie is a series of continuous multi-frame images, denoted as M = {f. where each frame î, is an independent observation of the true signal I.
Ideally, the imaging process in cryo-EM involves two main steps: 1) projecting the 3D density volume of the region of interest V(x, ((x,y.2) R3 + R along the z-axis via weak-phase object approximation [54], and 2) modulating the projection image with the Point Spread Function (PSF) g of the cryo-EM
3

---

optical lens, expressed as:
I= g g*
V(s,
where / is considered the true signal.
However, practice, the captured frames suffer from extremely low SNR due the limited electron dosage and the high sensitivity of DDD. The main noise source is Poisson (shot) noise from the detector, denoted as Poisson(T). arising from the inherent uncertainty of the electron measurement [12]. We assume that the additional noise types like heat, readout, and dark current noise are collectively modeled as zero-mean Gaussian noise with an unknown variance r² 42
i == Poisson + g.
As the number of observations increases, the average of these movie frames converges to the true signal:
M
E[i -
4
M
We define the image noise as the difference between the captured and clean images for each frame. Thus, the expectation (mean value) of noise distribution is:
[ = E[,, 1 =.
Thus, we derive that the cryo-EM image noise is zero-mean. This conclusion is also aligned with existing cryo-EM reconstruction methods 42 41,, which directly assume that the noise distribution is an additive zero-mean Gaussian yet can still achieve high-resolution reconstruction. We are well aware that this derivation relatively trivial compared to an actual theoretical analysis [[2], but this gives us intuitive guidance to integrate the N2N [16] idea: learning to denoise images from solely noisy image pairs, into our pre-training framework.
Movie to micrograph triplets. Given an M--rame movie M, we divide it into odd frames M (/2-11) ) [M/22] and even frames M [[2)] An off-the-shelf motion correction method [3 then applied to correct cross-frame drifts in M, MM, and M By summing up the frames within each subset, we generate three micrographs with the same shape: the original micrograph i, odd micrograph f", and even micrograph ie, As aforementioned, all these micrographs are expected to reflect the true signal / but corrupted by noises.
===
Denoising-reconstruutioo Autoencoder
We introduce DRACO, denoosinggrrcconsruuce autoencoder for cryo-EM, as illustrated in Figure 2 Different from existing masked imaging modeling methods, our model uses paired odd-even micrographs as inputs for the denoising target on visible patches. Further, we utilize the original micrographs as the additional supervision signal for reconstruction masked patches.
Masking. Following the standard scheme in Vision Transformer (ViT) [55, each micrograph in triplet, consisting of one original, one odd, and one even micrograph from the same movie, is divided into regular ooe-llerlapping patches. We create patch sets { for the odd, { for the even, and (x,) ) for the original micrograph, where N represents the number of patches. For the odd and the even micrographs used inputs in model, generate two sets of binary masks, {mf), and [mf) with mask ratio 7. Here, unmasked. Additionally, we ensure that a patch can be 1) only visible in one of them, or 2) masked in both. This strategy ensures that each visible patch has no information sharing of its corresponding patch on the other micrograph. Notably, this requires 7 2 0.5 for each input micrograph.
means the i-th patch is masked, and O means
Network architecture. For DRACO's pre-training, we employ a ViT- based encoder-decode archi- tecture following the MAE framework (6). Positional embeddings are first added the input patches, which are then masked to select only visible patches, denoted as (v.)7yY The encoder, Gene, is VIT that transforms these visible patches from either odd micrographs into latent features. To

---

Encoder
Shared Weights
Decoder
yi
xf
LNNN
Encader
Xi
Frecon
Microoahh
Mirrogoph
Original Micooraph
The pipeline of DRACO. Given a pair of partially masked odd and even micrographs, the encoder takes odd-visible patches and even-visible patches as inputs. The unmasked latent patches are combined with masked latent patches together to generate the latent representation Z, Then the latent representation passes through the decoder to generate predicted patches. The N2N loss is applied to odd-visible predicted patches with corresponding even input patches, and vice versa. The reconstruction loss applied to both invisible predicted patches with higher SNR input patches.
Figure 2:
align with the original unmasked size, zeros are padded to the latent features based on the positions indicated by the corresponding masks before the encoder outputs them:
(2
Gens (1v (m 1
{ Genc (Ivi)22) := Oncc). (5)
Each of the latent features from odd and even micrographs retain part of the original micrograph's information. To reconstruct the micrograph, we generate the latent representation { J.v for further processing by the decoder:
= (1 m.) 2 + (1) 2 m m [MASK].
where the [MASK] token is shared learnable embedding representing the masked patch be predicted. Finally, the decoder Gdee takes the latent representation {z, j with another positional embedding added as input and predicts all masked and visible patches to reconstruct the complete micrograph:
(y.) = Gdee (1z. Oace).
Denoising target. Inspired by N2N and Topaz- Depoisee 337, which predict denoised images only from paired noisy images, we introduce an image-denoising target on visible patches. For any patch visible in only the odd micrographs, the model predicts its counterpart in the even micrographs, and vice versa, as illustrated Figure As mentioned earlier, the expectation (mean) values of the odd and even micrographs are the true signal. Therefore, following Topaz-Denoise, employ patch-wise L2 loss function for each visible patch, aiming for regressing the true signal:
( X ||x
yfl 1
if m 0, m 0.
LNNN (x), x..y%.
(8)
Reconstruction target. For masked patches. we let the decoder predict the pixel value of patches from the original micrograph with higher SNR compared to odd and even micrographs for better reconstruction quality. Thus, the reconstruction loss is:
Lrecon Xi: y:)
||x:
yelle.
m m 1.
Training objective. During training, we combine the N2N loss with the reconstruction loss:
L LNNN + XLecsn
(10)
where À is a dlyper-paramee set 1.0 in all our experiments.
5

---

5 Experiments
Large-scale curated dataset for pre-training. The effectiveness and robustness of DRACO depend heavily on the quality and scale of the cryo-EM pre-training dataset. However, direct access to public databases like the Electron Microscopy Public Image Archive (EMPIAR) 17 results in variations in data quality, inconsistent data formats, and inaccurate even missing annotations. To overcome these challenges, we have developed a data generation workflow. First, we select datasets with reported resolutions better than 10 A, ensuring high-quality data acquisition. Next, we collect the raw data, including metadata, movies, and micrographs, from pre-defined high-quality datasets available on EMPIAR. Finally, we re-process the raw data using cryoSPARC [41] through a custom processing pipeline designed exclude low-quality micrographs and movies, generate annotations for downstream tasks, and verify the resolutions of the reconstructed results. This workflow has allowed us to compile large-scale, curated cryo-EM dataset containing over 270,000 micrographs and more than 50,000 raw movies from 529 verified single-particle cryo-EM datasets, occupying approximately 25 TB of disk storage in total. Details of the cryoSPARC processing pipeline are provided in Appendix C.T
Data augmentation. Each micrograph in a triplet goes through the same data augmentation process: randomly cropping to between 1/16 and 1/4 of the original size (typically 4096 x 4096), resizing to 256 x 256, applying random horizontal and vertical flips, and finally normalizing based the mean and standard deviation computed from the original micrograph. We randomly crop each micrograph 16 times within single epoch for fully utilization.
Pre-training details. We explore the performance of DRACO using two ViT architectures for the encoder, ViT-B and ViT-L, denoted as DRACO-B and DRACO-L, respectively. The decoder of DRACO uses Transformer blocks with embedding dimension 512, followed by a three-layer convolution neck and linear projection layer with an output dimension 16 x 16, which is also the patch size of the input. The mask ratio for the one input micrograph is 0.75 by default. To fully utilize the original 270,000 micrographs based
our lcale-scale curated dataset. we warm up DRACO on the MAE training scheme for 200 epochs. Then we adopt our novel mnoiisingrcoonnstuuciinn pre-training for 400 epochs. The warm-up stage takes 6 hours, and the pre-training stage takes 16 hours on a GPU cluster with 64 NVIDIA A800 GPUs, requiring approximately 80 GB of memory for batch size of 4096.
5.1 Particle Picking
Particle picking aims to accurately locate particles in highly noisy micrographs, which directly related to the resolution of the final reconstructed result. For adaption, we conduct supervised fine- tuning based on the ViT-based object detection framework Detectron2 (2 on our curated dataset. The results show that DRACO is capable of accurately detecting particles of various shapes and sizes across three challenging datasets.
Dataset. To create the annotated dataset for adaptation, we employ the standard workflow of cryoSPARC to generate a high-quality annotated dataset containing over 80,000 full micrographs and approximately 8 million particles across 46 types of protein. Detailed descriptions of the annotation workflow can be found in Appendix C.2 The test dataset includes three full micrograph sets with EMPIAR ID 1) 10081: Human HCN channel protein (56), the protein structure has a similar shape to ice, which could lead to false positive picking, 2) 10350: LetB transport protein [57], this kind of protein tends to aggregate together, posing challenges in accurate picking in crowded area, and 3) 10407: 70S ribosome (58), the micrographs are in the extremely low SNR.
Baseline and metrics. We compare DRACO with existing -ftt-off-th--art Yearning-based methods for generalized particle picking, including Topaz [5T], crYOLO [52]. and CryoTransformer [53] For ViT-based baselines, including MAE, DRACO-B and DRACO-L as aforementioned. We integrate them into Detectron2 framework by loading pre-trained weights of DRACO's encoder into Detectron2's 2's encoder. Additionally, to show the effectiveness of pre-trained encoder, we compare with Detectron2 trained from scratch. More configuration details can be found Appendix [DI] We evaluate baselines in terms of conventional metrics including precision, recall, and F1 score, and the resolution of the resolved 3D structure from the picked particles, which also a crucial metric. We process the particles selected by each method with the standard cryoSPARC workflow and finally

---

Topaz
erYLLO
CryoTransformer
Detectron2
MAE
DRACO-L
LetB
Human HCNI 70S Ribosome
Figure 3: Visualization of particle picking results. We show the picking results of DRACO and baselines on the test datasets range from small transport proteins to huge ribosomes. Blue, red, and yellow circles denote true positives, false positives, and false negatives, respectively.
Table 1: Particle picking results. We report the precision, recall, FI score, and resolution on each test dataset among all baselines. The resolution is obtained from the default cryoSPARC workflow. We compare DRACO with existing state-of ofte-offtheeart methods and consistently achieve the best F1 score and resolution of reconstructed results.
Human HCNI
70S ribosome
LetBi
())
Topuz
0.162
0,956
0.623
4.20
0.382
0.325
2.30
0.3118
0.761
0.617
cYOLD
0,818
0.748
0.792
4.15
0.602
0,300
0.711
2.75
0.632
0.163
0.224
4.02
CrooTransformer
0.475
0.910
0.624
4.13
0,517
0.887
0.654
2.79
0.629
0.706
0.534
3.67
Desection
0.392
0.834
0.333
4.30
0.868
0,901
0.767
2.85
0.389
0,801
0.680
3,36
MAE
0.649
0.625
4.32
0.712
0.786
2.34
0.805
0.682
DRACO-B
0.799
0.779
produce 3D reconstruction density maps and the corresponding resolution, described in Appendix
C
Results. As illustrated in Figure both Topaz and CryoTransformer tend to pick larger number of particles, but this often results in many false positives. contrast, crYOLO achieves higher precision in picking, yet exhibits higher number of false negatives. Detectron2 trained from scratch and pre-trained MAE both have difficulties in distiguishing signal and noise, leading to a lack of generalizability. In contrast, DRACO effectively identifies correct particles, surpassing the performance of all baselines, as demonstrated in Table
5.2 Micrograph Denoising
Once pre-trained, model naturally serve generalizable denoiser by directly predicting every patch input noisy micrograph without any further fine-tuning.
Baseline and metrics. To evaluate the effectiveness of DRACO on the denoising task, we first compare DRACO with the standard MAE trained on the 270,000 micrographs from our large-scale curated dataset with VIT-B as the backbone, denoted as MAE. We further compare with a popular denoising method Topaz-Denoise (37 in cryo-EM. For a fair comparison, we train Topaz-Denoise on our odd-even micrograph dataset for 100 epochs with default settings. Last, compare with the traditional method Low-pass filtering that has already been integrated into commercial software cryoSPARC [1]] We utilize the same protocol used in cryoSPARC and set the low-pass cutoff resolution to 20 Å. As cryo-EM micrographs lack clean ground truth, following Topaz-Denoise we employ an SNR calculation method that involves manually annotated hignal-backroond region pairs references. For each i-th pair, calculate the mean and variance for both the signal region

---

Micrograph
Raw Low-pass
MAE
Topaz-Denoise
DRACO-L
Apoferritin
Raw
RNA polymerase
Low-9992
DIRACO-L
Figure 4: Qualitative comparison results of micrograph denoising. We visualize the denoising results of DRACO and state-of-the-ae baselines. Our results show the most significant SNR improve- ment without the loss of the particle structure details. In contrast, Low-pass leads to severe blur on particles, MAE introduces severe patch-wise artifacts and Topaz only shows either minor SNR improvements or blurred results.
Table 2:
Quantitative comparison of denoising results. We report the SNR calculated with Equation On our test dataset, DRACO outperforms all other baselines.
Human Apolerriia
HA Triner
Phage MS2
RNA polymerase
Raw
10.01
6.52
12.52
4.69
Low-puss
-.188
-0.84
-6.71
3.09
DenoisDenoise
-5.67
-0.83
-6.93
8.66
MAE
-0.31
-1.85
-8.45
1,27
DRACO-B
3.69
-0.13
10.13
DRACO-L
2.01
3.33
0.23
12.21
rf and the background region r/. yielding pf, of for the signal, and n&; of for the background. The average SNR then computed in dB as follows:
10 SNR
(E 12722
Σ
log10
(11)
N
uf
Dataset. To evaluate the denoising capabilities of the baseline models, we select four original micrograph datasets as test datasets, which are excluded from the training set. These sets are Human Apoferritin TEMPIAR-10421] (59], HA Trimer (EMPIAR-10096) [60], Phage MS2 (EMPIAR- [61] and RNA polymerase (EMPIAR-IT 1121) 622. For each dataset, micrographs are selected and 20 signal-backe -ackkrobbd pair regions are labeled in total. Specifically, these signal and background regions are chosen close together to ensure similar background signals across both regions.
The quantitative experiments show that DRACO achieves significant performance improve- ments in terms of SNR after denoising compared with state-of-the-an methods, as shown in Table 2. In Figure 4 the standard MAE can only recover smooth contours of the particle with severe artifacts. Low-pass filtering smooths both signal and background noise, but the background noise is still relatively high and the structure information of particles is corrupted. Topaz sometimes fails to effectively denoise micrographs but over-smooth them instead, which affects the generalizability. In contrast, DRACO outperforms all baselines in retaining the original particle signals with the lowest background noises, showing the best generalizability. We additionally reconstruct 3D density map and corresponding resolution using the denoised particles generated by each method. To ensure
Results.

---

Table 3: Quantitative comparison of reconstruction using denoised particles. We report
the resolution obtained from the standard cryoSPARC workflow. our test dataset, DRACO outperforms all other baselines.
Human Apoferritin
HA Trimer
Phage MS2
RNA polymerase
Low-pass Sopzz-Denoise
2.63
2.06
3.46
2.75
2.34
3.06
2.52
2.93
MAE
2.77
2.15
3.78
2.81
DRACO-B
2.05
2.10
2.51
2.56
Table 4: Quantitative comparison of micrograph curation. Miffi employs its own general model, while 8 trained from scratch. DRACO reports the best results on all four classification metrics.
Method
Accuracy
Precision
Recall
score
Milh
0,836
0.899
0.843
0.871
ResNei18
0.938
0.923
0.960
0.940
MAE
0,304
0.927
0.892
01:10)
DRACO-B
0.963
0.976
0.953
0.964
DRACO-L
0.983
0.976
0.992
0.984
that comparisons reflect only the impact of denoising quality, we fix the locations and poses of the picked particles, which were determined using the cryoSPARC workflow. The results are shown in Table 3 ARACO consistently achieves the highest resolution in most cases, demonstrating its effectiveness in preserving more niih-ffrequnce" signals while effectively reducing background noise. We demonstrate additional denoising results in Figure 5
5.3 Micrograph Curation
modern cryo-EM can capture thousands of micrographs in a day. However, the quality captured micrographs is unverified. Low-quality micrographs may arise from artifacts such as empty sample, ice crystals, ethane contamination, severe drifting, etc. 40). Low-quality micrographs can negatively contribute to the final reconstruction results. A reliable automated micrograph curation method can significantly improve the efficiency the data processing pipeline, resulting shorter processing time and improved final resolution. We show that DRACO can easily adapt to this 2-class classification task by linear probing, and achieving the best performance compared the state-of-the-an method. Similar to 6
we freeze the encoder backbone and train an extra linear classification head.
Dataset. We manually annotate 1,194 micrographs from original micrograph datasets, assigning binary label (accept reject) to each to indicate quality. The dataset comprises 617 high-quality and 577 low-quality micrographs. We divided these micrographs into training and evaluation datasets using an 80%/20% split ratio.
Baseline and metric. We compare DRACO-B and DRACO-L with linear probing against several baselines: a small ResNet18 [63]; an existing supervised method Miffi [40], which has been trained on 45,000 annotated data; and the standard MAE with linear probing both pre-trained and adapted on our datasets. The ResNet18 is trained from scratch to show the effectiveness of our pre-training strategy. More configuration details is provided in Appendix D..2 We report the widely used metrics for classification including precision, recall, Fl score, and accuracy our test dataset. As Miffi predicts multi-labels low-quality micrographs, we consider them all as rejections for our metric calculations.
Results. As shown in Table Miffi, limited by insufficient training data, lacks generalizability the test dataset. Both ResNet and MAE face difficulties in effectively separating noise from signal for classification for accurate classification. In contrast, DRACO extracts global information from noisy images more effectively, resulting in higher classification accuracy and demonstrating better generalizability compared to other methods.
5.4
Ablation study
We compare the performance of networks with different parameter sizes particle picking and micrograph curation tasks, shown in Table I and Table The results demonstrate that our method

---

Table 5: Evalution of mask ratios. We demonstrate the performance of DRACO with different mask ratios. The result shows that at the 0.75 mask ratio, DRACO achieves the best performance.
Micrograph Curation
Denoising
Ratio
Accuracy
Score
polymerase
Trimer
0.5
0.954
0.968
0.945
0.956
10.29
2.57
0.625
0.75
0.963
0.976
0.953
0.964
10.13
3.69
0.875
0.958
0.984
0.939
0.961
9.59
2.28
Table 6: Evalution of loss function. We demonstrate the performance of DRACO with different training objective on the 70S ribosome dataset. The result shows that with both loss, DRACO achieves the best performance.
Particle Picking
Denoising SNR(T)
Training scheme
Precision(
Recall(7)
FT Score(1)
Res.(()
DRACO-B w/o N2N
0.712
0.876
0.786
2.84
..94
DRACO-B w/o recon
0.713
0.817
0.761
2.85
-4,22
DRACO-B
0.732
0.905
0,810
2.61
-2.86
effectively scale up. Additionally, evaluate the impact of different mask ratios denoising and micrograph curation performance, as shown in Table DRACO achieves the highest SNR and curation metric 0.75 mask ratio, thus we choose 0.75 conducted an additional ablation study of loss design. Specifically, we remove either the N2N loss (w/o N2N) the reconstruction loss (w/o recon) from the training and evaluate the resulting models on particle picking and denoising tasks, as shown in Table 6 The result shows that both N2N and reconstruction losses improve performance. Without the N2N loss, DRACO struggles distinguish between signal and noise in micrographs. Without the reconstruction loss, DRACO loses its ability to extract general features. We show additional visualization of their denoising results (Figure 6 in the Appendix A
default mask ratio. We have also
Discussion
Limitations. As the first attempt to achieve robust feature extraction for cryo-EM via novel reconstruction autoencoder, our work presents opportunities for future enhancements. First, method relies heavily on the performance of motion correction algorithms. This can be improved by designing a more comprehensive denoising task for raw noisy movies. Second, although we have collected what believe be the largest curated dataset for cryo-EM, focuses primarily on mainstream single-partich datasets. To enhance dataset diversity, other types of cryo-EM datasets, such as cryo-electron tomography (cryo-ET) [64] datasets, should also be included. Finally, our current approach only supports various micrograph-level downstream tasks. For particle-level tasks, such pose estimation, more fine-grained yet robust feature extraction required. This can be achieved by developing particle-level version of DRACO.
Conclusion. We have introduced DRACO, foundation model designed specifically for cryo-EM image processing, supported by unique memoiiing-reoonstuuutt pre-training framework to enable robust feature extraction for cryo-EM micrographs. We have constructed diverse and high-quality cryo-EM image dataset from the uncurated public database, comprising over 270,000 movies and micrographs. After pre-training, performance across multiple downstream tasks, including denoising, micrograph curation, and particle picking. All code, pre-trained model weights, and datasets will be made publicly available for further research and model development.
model's versatility is evidenced by its superior adaptation
7 Acknowledgement
This work was supported by HPC Platform of ShanghaiTech University. We thank Zhenyang Xu for shaping the paper.
10

---

References
[1] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. ArXiv, abs/2204.061 2212222...,,,,,, 2022.
[2] Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichtenhofer. and Kaiming He. Scaling language image pre-training via masking. 2023 IEER/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 23390-23400, 2022.
[3] Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. Trans. Mach. Learn. Rex., 2022, 2022.
[4] Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre- pre-training with online tokenizer. International Conference on Learning Representations (ICLR), 2022.
[5] Maxime Oquah, Timoth' ce Darcet, Théo Moutakanni, Huy Q. Vo, Marc Szafraniec, Vasil Khalidow, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin Fl-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russ Howes, Po- Yao (Bernie) Huang, Shang- Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma. Gabriel Synnaeve, Huijiao Xu, Hervé Jégou. Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision. ArXiv, abs/2304.07/ 193, 2023.
[6] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll' ar, and Ross B. Girshick. Masked autoencoders are scalable vision learners. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979-15988, 2021.
[7] Richard J. Chen, Tong Ding, Ming Y. Lu, Drew F. K. Williamson, Guillaume Jaume, Andrew H. Song, Bowen Chen, Andrew Zhang. Daniel Shao, Muhammad Shaban, Mane Williams, Lukas Oldenburg, Luca L Weishaupt, Judy Wang, Anurag Vaidya, Long Phi Le, Georg Gerher, Sharifa Sahai, Walt Williams, and Faisal Mahmood. Towards a general purppsse foundation model for computational pathology. Nature medicine, 2024.
[8] Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Long Phi Le, Georg Gerber, Anil V. Parwani, Andrew Zhang. and Faisal Mahmood. foundation model for computational pathology. Nature medicine, 2024.
A
[9] Chenxi Ma, Weimin Tan, Ruian He, and Bo Yan. Pretraining a foundation model for generalizable fluorescence microscopy-baseed image restoration. Nature Methods, pages 1-10, 2024.
[10] Yukun Zhou, Mark Chia, Siegfried K. Wagner, Murat Ayhan, Dominic J. Williamson, Robbert R. Struyven, Timing Liu, Moucheng Xu, Mateo G. Lozano, Peter Woodward- et al. A foundation model for generalizable disease detection from retinal images. Nature, 622:156 163, 2023.
[11] Werner Kühlbrandt. The resolution revolution. Science, 343(6178); 1117881:1443
2014.
[12] Miloš Vulović, Raimond B.G. Ravelli, Lucas Vliet, Abraham J. Koster, Ivan Lazić, Uwe Lüccenn Hans Rullgård, Ozan Öktem, and Bernd Rieger. Image formation modeling cryo-electron microscopy. Journal of Structural Biology, 183(1):19-32, 2013.
[13] Shawn Q Zheng, Eugene Palovcak, Jean-Paul Armache, Kliment A Verba, Yifan Cheng, and David Agard. Motioncor2: anisotropic correction of beam-induced motion for improved cryo-electron microscopy. Nature methods, 14(4)3331-332 2017.
[14] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre- AntrieAntoine Manzagol, Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096-1103 110,, 2008
[15] QuanLin Wu, Hang Ye, Yuntian Gu, Huishuai Zhang, Liwei Wang, and Di Hc. Denoising masked autoencoders help robust classification. arXiv preprint arXiv:2210.069:0 2022.
[16] Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. ArXiv, abs/1803.041 189, 2018.
[17] Andrii ludin, Paul K. Korin, Sriram Somasundharam. Simone Weyand, Cesare Cattavitello. Néli José da Fonseca, Osman Salih, Gerard J. Kleywegt, and Ardan Patwardhan. Empiar: the electron microscopy public image archive. Nucleic Acids Research, 51:D1503 - D1511, 2022.
[18] Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Kumar Gupta. Revisiting unrcasonable effectiveness of data in deep learning era. pages 843-852, 2017.
[19] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kun- durthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models. ArXiv, abs/2210.08402, 2022.
11

---

[20] Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari S. Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Grégoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash. Yann LeCun, and Micah Goldblum. A cookbook of self-supervised learning. ArXiv, 00N/2304.12110 2023.
[21] Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. European Conference on Computer Vision, pages 280-296. Springer, 2022.
[22] Alexander Kirillov,, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg. Wan-Yeu Yen Lo, Piotr Dollár, and Ross B. Girshick. Segment anything. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 3992-4003, 2023.
[23] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jérôme Revaud. Dust3r: Geometric
3d vision made easy. ArXiv, abs/23 14132 2023.
[24] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. ArXiv, wtts/2108.02255 2021.
[25] Ting Chen, Simon Korbllith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. ArXiv, abs/2002.05 2020.
[26] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, ArXiv, wbs//800.0033350 00w.00378...2 2018.
[27] Wenhui Wang. Hangbo Bao, Li Dong, Johan Bjorck. Zhiliang Peng. Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, and Furu Wei. Image as a foreign language: Beit pretraining for all vision and vision-language tasks. ArXiv, abs/2208. 10442, 2022.
[28] Mathilde Caron, Hugo Touvron, Ishan Misra, Herv'e J' J'egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties supe-supe vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pages 9630-9660 9640, 2021.
[29] Hangbo Bao, Li Dong, and Furu Wei. Beit: Bert pre-training of image transformers. ArXiv, abs/2106.08254, 2021.
[30] Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Yc, and Furu Wei. Beit v2: Masked image modeling with vector-quantized visual tokenizers. ArXiv, abs/2208.0636l 2022.
[31] Chaoyi Wu, Xiaoman Zhang. Ya Zhang. Yanfeng Wang. and Weidi Xie. Towards generalist foundation model for radiology. ArXiv, abs/2308. 024663, 2023.
[32] Ludwig Reimer and Helmut Kohl. Transmission electron microscopy. 2008
[33] Xinrui Huang, Sha Li, and Song Gao. Applying a modified wavelet shrinkage filter to improve cryo- electron microscopy imaging. Journal of computational biology. a journal of computational molecular cell biology, 25 9:1050-1055 1008, 2018.
[34] Wen Jiang, Matthew L. Baker, Qiu Wu, Chandrajit L. Bajaj, and Wah Chiu. Applications of a bilateral denoising filter in biological electron microscopy. Journal of structural biology, 144 1 1-2:114-22 22, 2003.
[35] Dai Wei and Chang cheng Yin. An optimized locally adaptive non-local means denoising filter for cryo-electron microscopy data. Journal of structural biology. 172 3:211-8, 2010.
[36] Hongjia Li, Hui Zhang, Xiaohua Wan, Zhidong Yang, Chengmin Li, Jintao Li. Renmin Han, Ping Zhu, and Fa Zhang. aassetrrasfer2elcan:: denoising cryo-cm images based on noise modeling and transfer. Bioinformatics, 38(7):2022 20092222222222222 2022.
[37] Tristan Bepler, Kotaro Kelley, Alex Noble, and Bonnie Berger. models for cryoem and cryoet. Nature communications, 11(1):2208, 2020.
general deep denoising
[38] Tim-Oliver Buchholz, Mareike Jordan, Gaia Pigino, and Florian Jug. Cryo-care: Content-aware image restoration for tryo-tranmiissonn electron microscopy data. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pages 502-506, IEEE, 2019.
[39] Alexander Krull, in--Oliver Buchholz, and Florian Jug. Noise2void learning denoising from single noisy images. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2124 2132, 2018
[40] Da Xu and Nozomi Ando. Miffi: Improving the accuracy of cun-based cryo-em micrograph filtering with fine-tuning and fourier space information. Journal of Structural Biology, page 108072, 2024.
[41] Ali Punjani, John L Rubinstein, David J Fleet, and Marcus A Brubaker. cryosparc: algorithms for rapid unsupervised cryo-em structure determination.. Nature methods, 14(3):290-2%6 2017.
[42] Jasenko Zivanov, Takanori Nakane, Bjöm O Forsberg, Dari Kimanius, Wim JH Hagen, Erik Lindahl, and Sjors HW Scheres. New tools for automated high-resolution cryo-em structure determination in relion-3. elife, 7:62266,, 2018
12

---

[43] Guang Tang, Liwei Peng, Philip R Baldwin, Deepinder S Mann, Wen Jiang, Ian Rees, and Steven J Ludtke. Eman2: an extensible image processing suite for electron microscopy. Journal of structural biology. 157(1):38 2007.
[44] Dimitry Tegunov and Patrick Cramer. Real-time cryo-electron microscopy data preprocessing with warp. Nature methods, 16(11)1116-1152, 2019.
[45] AM Roseman. Findem-a a fast, efficient program for automatic selection of particles from electron micrographs. Journal of structural biology, 145(1-2):11999, 2004.
[46] K Zhang, Li,, and F Sun. Gautomatch: an efficient and convenient gpu-based automatic particle selection program. Unpublished manuscript, 2011.
[47] NR Voss, CK Yoshioka, Radermacher, CS Potter, and Carragher. Dog picker and tiltpicker: software tools to facilitate particle selection in single particle electron microscopy. Journal of structural biology,
166(2):205-22) 10000-211,222 2009.
[48] Feng Wang, Huichao Gong, Ganchao Liu, Meijing Li, Chuangye Yan, Tian Xia, Xueming Li, and Jianyang Zeng. Decppicker: A deep learning approach for fully automated particle picking in cryo-em. Journal of structural biology, 195(3):325-335 2016.
[49] Yanan Zhu, Qi Ouyang, and Youdong Mao. A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy. BMC bioinformatics, 18:1-10, 2017.
[50] Chentianye Xu, Xueying Zhan, and Min Xu. Cryomae: Few-shot cryo-em particle picking with masked autoencoders. arXiv preprint arXiv:2404, 10178, 2024.
[51] Tristan Bepler, Andrew Morin, Micah Rapp. Julia Brasch, Lawrence Shapiro, Alex J Noble, and Bonnie Berger. Positive-unlabeled convolutional neural networks for particle picking cryo-electron micrographs. Nature methods, 11:::1111111111 160, 2019.
[52] Thorsten Wagner, Felipe Merino, Markus Stabrin, Toshio Moriya, Claudia Antoni, Amir Apelbaum, Philine Hagel, Oleg Sitsel, Tobias Raisch, Daniel Prumbaum, et al. Sphire-cryolo is fast and accurate fully automated particle picker for cryo-em. Communications biology, 2(1):218, 2019.
[53] Ashwin Dhakal, Rajan Gyawali, Liguo Wang, and Jianlin Cheng. Cryotransformer: transformer model for picking protein particles from cryo-em micrographs. Bioinformatics, 40, 2023.
[54] Miloš Vulovié, Lenard M Voortman, Lucas J van Vliet, and Bernd Rieger. When to use the projection assumption and the weak-phase object approximation in phase contrast cryo-em. Ultramicroscopy, 136:61-
66, 2014.
[55] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ArXiv, abs/2010. 1929, 2020.
[56] Chia- Lee and Roderick MacKinnon. Structures of the human henl experpolarizaaioo-actiiaee channel. Cell, (68(()))
2017.
[S7] Georgia L Isom, Nicolas Coudray, Mark R MacRae, Collin T McManus, Damian C Ekiert, and Gira Bhabha. Letb structure reveals a tunnel for lipid transport across the bacterial envelope. Cell, 181(3):5336664, 2020.
[58] David Nicholson, Thomas A Edwards, Alex J O'Neill, and Neil A Ranson. Structure of the 70s ribosome from the human pathogen acinetobacter baumannii in complex with clinically relevant antibiotics. Structure, 100, 2020.
28(10):
[59] Yong 7i Tan and John I. Rubinstein. Through-grid wicking enables high-speed cryoem specimen prepara- tion. Acta Crystallographica Section D: Structural Biology, (6(11))0092-1100 2020.
[60] Yong Zi Tan, Philip R Baldwin, Joseph HI Davis, James R Williamson, Clinton S Potter, Bridget Carragher. and Dmitry Lyumkis. Addressing preferred specimen orientation in single-particle cryo-em through tilting. Nature methods, 14(8):793 7966 2017.
[61] Roman I Koning, Josue Gomez-Blanco, Inara Akopjana, Javier Vargas, Andris Kazaks, Kaspars Tars, José María Carazo. and Abraham J Koster. Asymmetric cryo-em reconstruction of phage ms2 reveals genome structure in situ. Nature communications, 7(1):12524, 2016.
[62] Andreas U Mueller, James Chen, Mengyu Wu, Courtney Chiu, B Tracy Nixon, Elizabeth A Camphell, and Seth A Darst. A general mechanism for transcription bubble nucleation in bacteria. Proceedings of the National Academy of Sciences, 120(14)ee22087412,, 2023.
[63] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[64] EV Orlova and Helen R Saibil. Structural analysis of macromolecular assemblies by electron microscopy. Chemical reviews, 111(12):7710-7444 2011.
13

---

[65] Florian KM Schur, Martin Obr, Wim JH Hagen, William Wan, Arjen J Jakobi, Joanna M Kirkpatrick, Carsten Sachse, Hans-Georg Kräusslich, and John AG Briggs. An atomic model of hiv-1 capsid-spl reveals structures regulating assembly and maturation. Science, 35366298):506-5088 2016.
[66] Emdb the electron microscopy data bank. Nucleic Acids Research, 52(D1):D456 D465, 2024.
[67] Liu, Yutong Lin, Yue Cao. Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. 2021 IEEE/CVE International Conference on Computer Vision (ICCV), pages 9992-10002, 2021.
14

---

A Additional Results of DRACO
We demonstrate that DRACO achieves the highest visual denoising quality in terms of both signal preservation and noise removal, as shown in Figure This figure serves as an extension to Figure in our main paper. Furthermore, we visualize the denoising results from our ablation study on different mask ratios in Figure 6 We observe that at 0.75 mask ratio, DRACO achieves the best denoising results, as supported by Table 2 in our main paper. Lastly, we visualize the reconstruction ability at 0.75 mask ratio in Figure DRACO demonstrates comparable reconstruction capability MAE, while significantly better denoising results on visible patches.
Micrograph
Raw
Low-pass
MAE
Demoise
DRACO-L
Raw
HA Trimer
DRAGO
Phage MS2
DRACOL
Figure 5: Additional denoising results. We have conducted additional experiments on datasets of membrane proteins and bacteriophages. DRACO achieves the highest visual denoising quality by optimally balancing signal preservation and noise reduction.
50%
62.5%
75%
87.5%
Phage MS2
RNA polymerase
Figure 6: Visualization of DRACO's results on different mask ratios. At a 0.75 mask ratio, DRACO achieves the best trade-off between signal preservation and background noise removal.
15

---

Raw
Masked
MAE
DRACO
51000
991
Figure 7: Additional results on image reconstruction. We present the reconstruction results at various image resolutions while maintaining consistent mask ratio of 0.75. DRACO demonstrates enhanced detail preservation on the visible patches compared to MAE.
B Zero-shot Capability on Cryo-ET
Though DRACO has not been pre-trained on cryo-ET datasets, we found that DRACO can be directly applied on cryo-ET tilt series. Here, we demonstrate that DRACO capable of denoising the unseen HIV tilt series (65], as shown in Figure 8 Specifically, we evaluate DRACO on both the tilt series and the volume slices, showing that DRACO effectively removes background noise and achieves higher contrast in the volume. Furthermore, we assess DRACO's 's performance in the compare the slices from both tomograms reconstructed using original
context of reconstruction. and denoised tilt series. Additionally, we compare these with the denoised slice from the original tomogram. The result shows that DRACO can improve the contrast of slices before or after the reconstruction.
16

---

(a) The Original Image of HIV Tilt Series
(b) The Denoised Image of HIV Tilt Series
(c) Slice from (a)'s Reconstruction
(d) Slice from (b)'s Reconstruction
(e) Denoised Slice of (c)
Figure 8: Denoising cryo-ET HIV tilt series with DRACO. Figure (a) and (b) show the HIV tilt series beforeand after DRACO's denoising process. Using IMOD, we reconstruct 3D volumes of HIV from both the original and denoised series, showing their slice in Figures (c) and (d). Note the horizontal stripes in these images, which are artifacts due to the missing wedge issue in cryo-ET. Figure (e) shows denoised slice from Figure (c) by DRACO.
C Workflow of CryoSPARC
C.1 Pre-training Dataset Details
a public archive for storing raw cryo-EM images and 3D reconstructions from vEM and XT experiments. It currently contains over 2,000 entries, totaling more than 2 PB of data. The EMDB [66] is an archive of 3D reconstructions derived from cryo-EM experiments, many which supplement the experimental information of the preparation and reconstruction processes.
EMPIAR [17
EMPIAR entries, such sample
In the field of 3D electron microscopy, cryo-EM micrographs undergo complex image processing to achieve 3D reconstructions with specified resolutions. Experts can model protein molecules accurately on 3D reconstructions with resolutions better than 3 Å. We define the quality of datasets from the perspective of structural biology; hence, high-quality datasets should produce high-resolution 3D reconstructions suitable for detailed analysis.
When constructing the pre-training dataset, we utilize the REST API provided by EMPIAR to obtain metadata for each dataset, such experiment type, EMDB ID, and image classification. We specifically filter for EMPIAR entries that are experimentally linked to EMDB, prioritizing those with reconstruction resolutions better than 10 Å. Subsequently, we collect as many datasets as possible that contain single-frae fraame micrographs. For datasets that include multiple frames, further process them by separating the frames into odd and even micrographs.
Cryo-EM raw data consists of multi-frame recordings known movies, which capture the number of electrons. After motion correction [33] these movies yield single frames referred to as micrographs. We download many Linle-frrmee micrographs possible, categorized by image type. Additionally,
17

---

we process multi-frame frame movies into single-framm frame micrographs using cryoSPARC's Patch Motion Correction. For each multi-framm movie, we separately process the complete frames, odd frames, and even frames to generate three types of single-frame micrographs for DRACO learning.
C.2 Particle Picking Dataset Details.
We filter and generate a dataset comprising approximately 80,000 single-frame micrographs annotated with about 8 million particles, following process based on the cryo-EM single particle analysis reconstruction pipeline software, cryoSPARC 41 Each EMPIAR public dataset [17] comes with
solved 3D density map, available on EMDB [66] Using the "create template" step in cryoSPARC, we project this map into 50 diverse poses to generate high-quality templates for template picking. Subsequent rounds of the "2D classification" step are employed to eliminate potential false positives. Finally, using these particles, we reconstruct results whose resolution did not differ by more than 20% from the reported resolution. This method allowed us to collect a high-quality annotated particle dataset.
C.3 3D Reconstruction Pipeline
The reconstruction process is also based on cryoSPARC. After picking the particles, the standard reconstruction workflow consists 2D classification, ab initio reconstruction and homogeneous refinement. 2D classification aims to remove any false positives in picked particles. Ab initio reconstruction can create an initial 3D model from certain set particles. Based on this initial model, homogeneous refinement can achieve a high-resolution result. The final resolution is determined by the Fourier shell correlation (FSC) curve. The specific method involves dividing the particles into two random halves, each undergoing homogeneous reconstruction. After reconstruction, we perform a cross-correlation on each Fourier shell in the frequency domain of two reconstructed 3D density maps. The final resolution is determined using the standard threshold of 0. 0.143 on the FSC curve.
D Downstream tasks settings
D.1 Particle Picking Settings
Particle picking baselines. We use the Topaz [51] general model with its "resnet16u64" backbone for our baseline, picking particles that score higher than 0.0 as the final results. We use the YOLO general model for our baseline from its official website Similarly,
52
CryoTransformer's open-source general model on Github, choosing particles with scores ranging from the 25th to
53
the 100th percentile.
Detectron2 configurations. For particle picking, we employ the Faster R-CNN framework within Detectron2, to fit the particle picking task with our curated dataset. The configurations for both ViT-B and ViT-L include the standard feature pyramid and window attention [67]. We also set the non-maximum suppression threshold of the region proposal network 0.6 and adjust the pooling size of the box pooler in the region of interest network to 14. Given that the particles are mostly square, the aspect ratio of the anchors is fixed 1.0. All other settings remain at their default values. The data augmentation goes the same process as described in the pre-training stage. We fine-tune Detectron2- based particle picking model on 64 NVIDIA A800 GPUs for 100 epochs with batch size of 256, requiring approximately 9 hours and consuming around 100GB of memory. After fine-tuning we process the test dataset and pick particles with scores higher than 0.1 the final results.
D.2 Micrograph Curation Settings
For micrograph curation, each ViT-based model undergoes a linear probing phase, while a ResNet18 [63] trained from scratch. We employ the miffi v [40] general model of Miffi inference on test datasets. All the model trains 50 epochs with a batch size of 128 on a single NVIDIA RTX 3090 GPU, taking about 10 minutes and utilizing around 8GB of memory.
18

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the
relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
general, answering
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We believe that the abstract and introduction accurately reflect the development of our model, its training methodology, its effectiveness various downstream tasks, and the creation of a curated dataset to support this model.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
19

---

Justification:
We have listed out the limitations of our work in Discussion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We have provided the image and the noise model use in Section 3.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have clearly described the data curation pipeline, data preprocessing, network architecture, and adaptions to downstream tasks in Experiments and Appendix A.
Guidelines:
20

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We do not provide code and data in submission. However, the code and data will be released upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nis.cc/public//uust http:///cc/giiee//eeeuuubiiiiiin
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
21

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have stated the pre-training details in Section 5 and the adaptation details in Appendix B.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We do not provide error bars or other appropriate information about the statistical significance of the experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have stated the pre-training computer resources in Experiments and the adaptation computer resources in Appendix B.
22

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compule required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
ines?
Answer: [Yes]
Justification: Our research are all conducted in line with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the positive and negative societal impacts in Introduction and Discussion.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
23

---

Answer: [NA]
Justification: We do not describle safeguards in our paper, but we will consider to prevent our model and data from misuse when we release code and dataset.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All of our code and datasets we built upon are in public with proper licenses.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the bataasess.ppprsstttt paperswi theode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification:
All of code and datasets we built upon are in public with proper licenses.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We do not have crowdsourcing experiments in our paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
We do have crowdsourcing experiments in our paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "T6woYotuVf", "reviewer": "Reviewer_RjVX", "summary": "This work proposes a foundation model for Cryo-EM tasks. The technique consists of pretraining a masked autoencoder on a curated dataset, with a Noise2Noise-like self-supervision scheme. The learned representations are then fine-tuned for various tasks, such as denoising, particle picking and micrograph curation. Experimental results show promise across all tasks compared to existing techniques.", "questions": "- How well would the method perform outside of the single-particle setting, for instance slices of cryo-ET?\n- Can the proposed technique be adapted to 3D data such as that in cryo-ET?\n- Typo: line 134 theological should be theoretical\n- Vector notation would be better in Eq (8) and (9) (using norms)", "limitations": "Limitations are addressed.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A foundation model for cryo-EM is an interesting contribution that has potential in a wide range of downstream applications in structural biology.  \n- The proposed training scheme is interesting and novel to the best of my knowledge.\n- Authors curate a pretraining dataset which, if released to the community, can greatly benefit data-driven research for cryo-EM.\n- The experimental results are promising.", "weaknesses": "- Justification for the training scheme is somewhat unclear. Why is the supervision signal from the original micrographs not used in every patch. By only utilizing the original micrograph on the masked area we discard some information. Alternatively, odd/even micrographs could be used to predict the original averaged micrograph.\n\n- It is unclear how the method generalizes to unseen real data, a common scenario where foundation models are deployed. It would be important to understand how well the technique can be adapted in such cases.\n\n- It seems like authors introduce a noise model (Gaussian + Poisson), but does not leverage the model further, beyond the zero-mean assumption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your appreciation and insighful comments.\n\n## Clarification of Supervision Target\n\nWe thank the reviewer suggests that the supervision signal of visible patches can be replaced by the original micrograph to fully ultilize the high-quality information. In our paper, we followed the practice of Topaz-Denoise [1] to use odd/even micrographs as supervision signal. However, replacing them with the original micrographs would violate the Noise2Noise (N2N) assumption that noisy pairs are **independent** since the odd-original and even-original pairs are **dependent**. This dependency arises because the original micrographs share the same odd and even frames used in the generation of odd-even-original micrograph triplets.\n\nIf space permits, we will include an ablation study that uses full micrographs to supervise the visible patches.\n\n[1] Bepler T, Kelley K, Noble A J, et al. Topaz-Denoise: general deep denoising models for cryoEM and cryoET[J]. Nature communications, 2020, 11(1): 5208.\n\n## Fully Leveraging Noise Modeling\n\nThe reviewer is correct that by far we have mainly exploited the zero-mean noise assumption. We'd emphasize that the Poisson-Gaussian noise model of cryo-EM follows the first principles [1], and the zero-mean conclusion is the theoretical basis for our use of Noise2Noise. Theoretically, we can further deploy the model to enhance 3D reconstructions via simulations schemes such as STEM [2]. We will include a detailed discussion in our revision.\n\n[1] Vulović M, Ravelli R B G, van Vliet L J, et al. Image formation modeling in cryo-electron microscopy[J]. Journal of structural biology, 2013, 183(1): 19-32.\n\n[2] Kniesel H, Ropinski T, Bergner T, et al. Clean implicit 3d structure from noisy 2d stem images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 20762-20772.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BGIKROyZUL", "reviewer": "Reviewer_ypYy", "summary": "In this paper, the authors introduce an autoencoder-based model to denoise cryo-EM micrographs. By splitting aligned movies into even and odd images, the authors obtain two images with similar signal and two different realizations of noise. Following Noise2Noise and Topaz-Denoise, the denoising target is to predict the \"even\" realization from the \"odd\" on and vice versa. The authors show that their method outputs micrographs with higher SNRs (as defined by Eq 11). The representation learned by the encoder can be used to fine-tune an object detection algorithm for particle picking or a linear layer for filtering micrographs.", "questions": "Do the authors plan to grant access to their pretraining datasets?", "limitations": "The limitations are well described at the end of the paper and the authors suggest interesting avenues for future work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "**Large-scale curated dataset.** The authors give a detailed description of the protocol they followed to preprocess the EMPIAR datasets and pretrain their model.\n\n**Description of the model.** The authors give a detailed description of the architecture of the model.\n\n**Downstream model adaptation.** I find the possibility of using DRACO for model adaptation particularly exciting and potentially impactful. The authors illustrate this capability with two examples: particle picking and micrograph cleaning.", "weaknesses": "**Denoising section.** Section 5.1 describes the method as a denoising method. I found this section slightly misleading. To me, the main contribution of DRACO is its ability to learn a compact representation of cryo-EM patches, which can be used for downstream tasks. When the method is first described as a denoiser, the reader can be misled and think that downstream tasks (e.g particle picking) use the denoised images. Furthermore, I find that the evaluation of the denoising performances lacks quantitative analysis. Table 1 only reports the SNR obtained with a heuristic definition of the SNR, but reporting the resolution obtained after homogeneous reconstruction on the denoised images would be more compelling. If the authors think the denoising capability is *not* the main contribution of the paper, my recommandation would be to move down Section 5.1 to avoid misunderstandings. Otherwise, Section 5.1 should contain further quantitative evaluations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's suggestions regarding the writing of the article. We will carefully consider your feedback and make the necessary improvements in the revision.\n\n## Clarification of Denoising Section\n\nWe apologize for the confusion and re-structure the paper. The reviewer is correct that denoising is one of the downstream tasks of DRACO, and the denoised results are not utilized in other downstream tasks.  As suggested, we will move this section further down in the paper to clarify.\n\n## Dataset Release\n\nWe are preparing to release our pre-trained dataset, which includes 529 sets of cryo-EM single-particle image data and their metadata, over 270,000 images in total. We will provide a public data server for file downloading.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FC4banW0yS", "reviewer": "Reviewer_b2KB", "summary": "In this paper, the authors introduce DRACO, a Denoising-Reconstruction Autoencoder for Cryogenic Electron Microscopy (cryo-EM) inspired by the Noise2Noise (N2N) approach. DRACO aims to address the high-level noise corruption in cryo-EM images, which are often overlooked by other foundation models in computer vision. The method involves processing cryo-EM movies into odd and even images, treating them as independent noisy observations, and applying a denoising-reconstruction hybrid training scheme. By masking both images, DRACO creates denoising and reconstruction tasks to enhance the quality of the output.\n\nFor the pre-training phase, the authors build a high-quality and diverse dataset from an uncurated public database, comprising over 270,000 movies or micrographs. This dataset is essential for ensuring the effectiveness of DRACO as a generalizable cryo-EM image denoiser and a foundation model for various downstream tasks. The experimental results demonstrate that DRACO outperforms state-of-the-art baselines in denoising, micrograph curation, and particle picking tasks.", "questions": "[Why Not Use N2C for Network Training]\nThe authors mention in the paper that the noise model for electron microscopy images is a Poisson-Gaussian noise model. Given this, why not use a Noise2Clean (N2C) approach by inputting simulated noisy images and learning the corresponding clean images for network training?\n\n[Comparison with Blind-Spot Network-Based Methods]\nThere are existing denoising methods for real noisy images based on blind-spot networks. Why can't these methods be applied to electron microscopy image denoising? Furthermore, in the comparison with other methods, the authors only compare with low-pass filtering and MAE methods, making it difficult to clearly see the effectiveness of the proposed method.\n\n[Is SNR Sufficient to Evaluate Restored Image Performance?]\nThe authors compare the SNR of restoration results from different methods. However, considering the calculation method of SNR, it may favor smoother restoration results and potentially overlook some texture details.", "limitations": "Can the authors construct a more novel paired electron microscopy image dataset?", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is well-organized and clearly written, with a logical flow of information from the problem statement to the proposed solution and experimental validation. The authors provide a clear explanation of their denoising-reconstruction hybrid training scheme, including the masking of images to create denoising and reconstruction tasks, which enhances the reader’s understanding of the methodology.", "weaknesses": "Weaknesses\nWhile the paper makes a valuable contribution, there are several areas for improvement:\n\nTraining Method Justification: The authors use a Noise2Noise (N2N) approach for training but do not justify why a Noise2Clean (N2C) approach was not considered, given the Poisson-Gaussian noise model in electron microscopy images.\n\nComparison with Blind-Spot Networks: The paper lacks a comparison with existing blind-spot network-based denoising methods. Including such a comparison would provide a clearer benchmark for evaluating the effectiveness of the proposed method.\n\nEvaluation Metrics: The use of SNR as the sole evaluation metric may not fully capture the quality of restored images, as SNR tends to favor smoother results and may overlook important texture details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their questions and suggestions on denoising. Based on this, we will further clarify the questions related to denoising baseline selection and metric evaluation.\n\n## On the Choice of Denoising Baselines\n\n**Noise2Clean baseline.** Supervised image denoising methods like Noise2Clean (N2C) are prevalent in the field of computer vision, where generating or simulating clean-noisy pairs is typically feasible. However, Topaz-Denoise has pointed out that real clean micrographs are unavailable in cryo-EM. Therefore, we use a Noise2Noise (N2N) approach to make DRACO robust to noise. In theory, one could leverage simulation techniques to produce clean-noisy image pairs in cryo-EM for N2C training. However, the current state-of-the-art methods, such as InsilicoTEM [1], still suffer from inaccurate simulations and high computational costs. We will include a detailed discussion of this in the introduction.\n\n**Blind-Spot baseline.** We appreciate the reviewer's suggestion. The Blind-Spot method is a reasonable baseline for denoising without clean data. We trained and evaluated its performance, and the results, shown in the following table, demonstrate that DRACO better distinguishes signal from background noise compared to the Blind-Spot method Blind2Unblind (B2U) [2]. For B2U, we apply the default settings and adapt them to our datasets. DRACO leverages the continuous multi-frame nature of cryo-EM images, allowing it to differentiate signal from noise more effectively than solely from noisy images. The MAE training paradigm also helps capture low-frequency signals, while Noise2Noise loss enables DRACO to focus on high-frequency signals.\n\n**Table 1**: The comparison of Blind-Spot and DRACO-B.\n|        Dataset        |   Metric   | Blind-Spot |  DRACO-B  |\n| --------------------- | ---------- | :--------: | :-------: |\n| **Human Apoferritin** |    SNR (dB, $\\uparrow$ )     |   -5.98    | **1.92**  |\n|                       | Resolution (px.,  $\\downarrow$) |    2.53    | **2.05**  |\n|     **HA Trimer**     |    SNR     |   -3.16    | **3.69**  |\n|                       | Resolution |    2.76    | **2.10**  |\n|     **Phage MS2**     |    SNR     |   -7.35    | **-0.13** |\n|                       | Resolution |    3.35    | **2.51**  |\n|  **RNA polymerase**   |    SNR     |   -1.63    | **10.13** |\n|                       | Resolution |    2.68    | **2.56**  |\n\n[1] Vulović M, Ravelli R B G, van Vliet L J, et al. Image formation modeling in cryo-electron microscopy[J]. Journal of structural biology, 2013, 183(1): 19-32.\n\n[2]Wang Z, Liu J, Li G, et al. Blind2unblind: Self-supervised image denoising with visible blind spots[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 2027-2036.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fSe1ipYSFs", "reviewer": "Reviewer_AE1Z", "summary": "This manuscript introduces a new denoising autoencoder for cryo-EM micrographs based on vision transformers (ViTs) which is trained on masked pairs of images with a noise2noise loss function paired combined a reconstruction loss. In addition to the model itself, an important contribution of the work is the curation of a large dataset of experimental micrographs used to train the model, containing a total of 270 000 micrographs from publicly available sources. The resulting model is shown to achieve state-of-the-art performance on denoising, particle picking, and micrograph curation.", "questions": "– Why use a ViT as opposed to a simpler convolutional network?\n– How does masking an image make sense on images such as micrograph which typically lack long-range structure?\n– Please define γ on page 4.\n– Why is the reconstruction target necessary for the proper training of the network? How does the dependence of the noise realization between input and target affect training here.\n– How do we expect the mask ratio to affect the results? Why does this trade off between “signal preservation and background noise removal”?", "limitations": "The authors have largely addressed the limitations of the work in the manuscript.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The manuscript describes the problem to be solved accurately and outlines why traditional methods for training autoencoders do not perform well. The overall description of the proposed model is also well illustrated through the use of figures and well-defined notation. The considerable effort expended to ensure that the training and testing datasets are of sufficient quality is also a major strength of the work. Finally, the results on the three evaluation tasks are quite impressive, even though some questions arise regarding their calculation.", "weaknesses": "The manuscript is lacking in details in several important areas. First, the description of the method is sometimes at a very superficial level, making it hard to understand how it is actually implemented. For example, the mask is given as input to the decoder in eq. (5), but it is not clear how this is given as an input to the ViT. Furthermore, the [MASK] token is simply described as a “shared learnable embedding” without any more details. Clearly spelling out the details of the architecture in the main text of in an appendix would significantly strengthen the work. This also goes for the adaptation of the network to the other two downstream tasks (particle picking and micrograph curation), where the authors simply state that they “conduct supervised fine-tuning based on Detectron2” (for particle picking) and that they “freeze the encoder backbone and train an extra linear classification head” (for micrograph curation).\n\nSecond, the motivation for the various choices made is sometimes missing. This goes for the choice of architecture but also for the construction of the loss function, the choice of patch size, data augmentation strategy, and so on.\n\nThird, the evaluation methods for the first two task need to be better motivated. For example, the SNR calculation method in eq. (7) is simply stated (with a reference to the Topaz-denoise paper) without further explanation. Similarly, the definition of true and false positives and false negatives for the particle picking task needs to be spelled out.\n\nThe final weakness is Section 5.4, which purports to be an ablation study. It refers to different results based on parameter sizes in Tables 2 and 3, but those results do not seem to be there. Furthermore, this section seems to be more of a hyperparameter study, rather than an ablation study (no steps of the pipeline are removed or replaced by others here).\n\nThere are also some issues with language, such as the reference to “theological analysis” on page 4 and “a modern cryo-EM” and “artifacts such as empty” on page 8.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Reviewer AE1Z (5)\n\nWe sincerely thank the reviewer for the detailed review of our work.\n\n## On the Clarification of Details in the Method\n\nWe truly appreciate your detailed suggestions for our method. Here, we clarify the details in below:\n\n1. Similar to MAE [He K et al, 2022], the mask ratio $\\gamma$ is a hyperparameter that represents the percentage of masked patches in the patch set. \n2. In Eq. 5, we use the binary mask $\\\\{\\mathbf{m}_i\\\\}\\_\\{i=1\\}^N$ to select visible patches (where $\\mathbf{m}_i = 1$) from input patch sets. We will make it clearer in our revision.\n3. [MASK] token is a high-dimensional learnable embedding, whose dimension is 768 for DRACO-B and 1024 for DRACO-L. As described in Eq. 6, it is used to pad the masked image tokens.\n\nWe will make all these points clear and add more details on DRACO's architecture in our revision.\n\n## On the Clarification of Design Choices\n\nThank you for your constructive suggestions.\n\n**Design choice of ViT-based DRACO.** We use ViT [Dosovitskiy A et al, 2020] as the backbone for DRACO, following MAE [He K et al, 2022]. We choose ViTs rather than CNNs due to their scalability and effectiveness in learning from large datasets [Dosovitskiy A et al, 2020]. For the long-range structure lacking issue in DRACO pre-training, as the particle size in the training dataset ranges from 100 to 200, they can serve as long-range structures compared with 16 x 16 visual token size. For data augmentation, given the high resolutions of raw micrographs (4096 x 4096), we randomly crop patches from the original micrograph with side lengths ranging from 1/16 to 1/4 of the original size and rescale them to 256 x 256, to reduce the memory and computation consumption.\n\n**Particle picking adaption.** We utilize the ViTDet [Li Y et al, 2022] architecture within the Detectron2 framework for particle picking, starting by loading DRACO's pre-trained encoder weights into Detectron2’s backbone for further fine-tuning. During adaption, a raw micrograph serves as input, with the encoder extracting a sequence of image tokens. A feature pyramid network with four parallel CNNs generates a multi-resolution feature map. This map feeds into a region proposal network, which predicts potential particle locations. Subsequently, a region of interest network refines these predictions into binary logits (particle or background) and more accurate locations. Specifically, we adapted the cross-entropy loss for 2-class classification in cryo-EM, with default training strategy and other loss functions. We will add these detailed descriptions to the revision.\n\n**Micrograph curation adaption.** We utilize DRACO's encoder as the feature extractor, coupled with a single linear layer that outputs 2-class logits indicating the probability of a micrograph being good (1) or bad (0). We employ binary cross-entropy loss to supervise this predicted value. The encoder weights remain frozen during this linear probing adaption.\n\nWe will make all these details clearer in the revision. \n\n## On the Clarification of Metrics and Notations\n\nThank you for thoroughly reading our paper, and we truly appreciate your suggestions. We will clarify below points in the revision.\n\n\nWe follow the state-of-the-art denoising method, Topaz-Denoise [Bepler T et al, 2020] to manually create particle-background pairs to calculate SNR metric for the evaluation of denoising tasks. SNR is a measure of the signal strength to the background noise that has been widely used for measuring the quality of the image in cryo-EM.\n\nWe follow Topaz [Bepler T et al, 2019] to use true positive, false positive, and false negative to represent a correctly detected particle, a wrongly detected particle, and an undetected particle, respectively.\n\n## On the Ablation Study\n\nIn the ablation study section, we followed the practices of MAE [He K et al, 2022] for parameter size and mask ratio.\n\n**Ablation study of parameter size.** In Tables 2 and 3, DRACO-B and DRACO-L refer to taking ViT-B (86M parameters) and ViT-L (307M parameters) as the encoders used on downstream tasks, respectively. The results show that a larger backbone leads to slightly better performance on downstream tasks overall. We will clarify the parameter size differences in the revision.\n\n**Ablation study of the mask ratio.** In DRACO, the mask ratio determines the proportion of the N2N loss and the reconstruction loss. This influences the network's capacity to capture both high and low-frequency signals in micrographs. As shown in Table 4 of our paper, the practically best mask ratio is 0.75 to reasonably remove background noise while preserve high-frequency signals.\n\n**Ablation study of the loss functions.**  As suggested, we have conducted an additional ablation study of loss design. In the following table, we remove either the N2N loss (**w/o N2N**) or the reconstruction loss (**w/o recon**) from the training and test the variations on particle picking and denoising tasks. The result shows that both N2N and reconstruction losses improve performance.\n\n|Task|Metric|DRACO-B w/o N2N|DRACO-B w/o recon|DRACO-B|\n| - | - |:-:|:-:|:-:|\n|**Particle Picking**|**Precision ($\\uparrow$)**|0.712|0.713|**0.732**|\n||**Recall ($\\uparrow$)**|0.876|0.817|**0.905**|\n||**F1 score ($\\uparrow$)**|0.786|0.761|**0.810**|\n||**Res. (Å, $\\downarrow$)** |2.84|2.85|**2.61**|\n|**Denoising**|**SNR ($\\uparrow$)**|-4.94|-4.22|**-2.86**|\n\nIf space permits, we will conduct the ablation study in terms of patch size and the data augmentation strategy in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Global Response

**Please see the uploaded PDF for additional DRACO results on unseen cryo-ET data and a comparison with Blind2Unblind.**

We are encouraged that all reviewers recognize DRACO as a potentially beneficial foundation model adaptable to a broad range of cryo-EM downstream tasks. We thank the reviewers for their constructive suggestions, which we will incorporate into the revision. Below, we first address the common question raised by the reviewers and then respond to their individual comments.

## Generalization Capability of DRACO (**RjVX**)

We thank the reviewer for pointing out the potential application of DRACO in cryo-ET. We have conducted an experiment and included the results in the uploaded PDF (see Fig. 1). In this experiment, we demonstrate DRACO's zero-shot denoising capability on an unseen HIV cryo-ET tilt series [1]. Specifically, we evaluate DRACO on both the tilt series and the volume slices, showing that DRACO effectively removes background noise and achieves higher contrast in the volume. If space permits, we will include one such result in the revision.

We are excited to see that DRACO shows promising preliminary results on unseen cryo-ET data. In the future, we plan to incorporate the cryo-ET dataset into DRACO's training dataset and extend DRACO to the cryo-ET 3D volumes using the following approach. We will split the cryo-ET tilt series into odd and even tilt series and then reconstruct each to obtain odd/even 3D volume pairs. By training DRACO on corresponding slices from these paired volumes, we aim to achieve effective denoising for cryo-ET 3D volumes.

We'd also like to clarify that all experiments conducted and included in the paper were on unseen datasets, demonstrating the high generalizability of DRACO. We will make this point clearer in the revision.

[1] Schur, Florian KM, et al. "An atomic model of HIV-1 capsid-SP1 reveals structures regulating assembly and maturation." Science 353.6298 (2016): 506-508.

## On the Denoising Metrics (**ypYy**, **b2KB**)

We thank the reviewers point out that the Signal-to-Noise Ratio (SNR) may not comprehensively reflect the quality of restored images. Initially, we adopted SNR as the primary quantitative metric, following the example set by Topaz-Denoise [1]. Responding to the reviewers' recommendations, we have now expanded our evaluation to include experiments that assess particle reconstruction performance across various denoising methods (**ypYy**).

Specifically, we compare the reconstruction resolution across four datasets featured in our denoising section using various denoising schemes. To ensure that comparisons reflect only the impact of denoising quality,  we fix the locations and poses of the picked particles, which were determined using the cryoSPARC [2] workflow. We employ the Filtered Back Projection (FBP) algorithm for homogeneous reconstruction to determine the pixel resolution at an FSC of 0.143 for the two half-maps. As shown in the following table, DRACO consistently achieves the highest resolution in most cases, demonstrating its effectiveness in preserving more high-frequency signals while effectively reducing background noise.

We have included visualizations of the denoising results in the uploaded PDF. For additional comparisons, please see Fig. 2, which shows the Blind-Spot denoising method (**b2KB**).

**Table 1**: The comparison of the resolution (px.) $\downarrow$  of reconstructed volume by various baselines.

|Dataset|Low-pass|Topaz-Denoise|MAE|DRACO|
|-|:-:|:-:|:-:|:-:|
|**Human Apoferritin**|2.63|2.34|2.77|**2.05**|
|**HA Trimer**|**2.06**|3.06|2.15|2.10|
|**Phage MS2**|3.46|2.52|3.78|**2.51**|
|**RNA polymerase**|2.75|2.93|2.81|**2.56**|

[1] Bepler T, Kelley K, Noble A J, et al. Topaz-Denoise: general deep denoising models for cryoEM and cryoET[J]. Nature communications, 2020, 11(1): 5208.

[2] Punjani, Ali, et al. "cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination." Nature methods 14.3 (2017): 290-296.

Meta-review: This work introduces DRACO, a foundation model for cryo-EM tasks. The model uses a hybrid denoising-reconstruction training strategy, with a Noise2Noise loss function paired with a reconstruction loss, to enhance the quality of cryo-EM images. The learned representations are fine-tuned for various downstream tasks, such as denoising, particle picking, and micrograph curation. Experimental results show that DRACO outperforms state-of-the-art techniques across these tasks, achieving higher signal-to-noise ratios (SNRs) and providing a robust solution for cryo-EM image processing. Although the reviewers pointed out some minor weaknesses (such as the motivation of choices, ablation studies), the rebuttal addresses most of them.
Decision: Accept (poster)