# Template Learning: deep learning with domain randomization for particle picking in cryo-electron tomography

Mohamad Harastani, Gurudatt Patra, Charles Kervrann, Mikhail Eltsov

► To cite this version:

Mohamad Harastani, Gurudatt Patra, Charles Kervrann, Mikhail Eltsov. Template Learning: deep learning with domain randomization for particle picking in cryo-electron tomography. Nature Communications, 2025, 16 (8833), pp.1-15. 10.1038/s41467-025-63895-0. hal-04874266

HAL Id: hal-04874266

https://inria.hal.science/hal-04874266v1

Submitted on 8 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Template Learning: Deep Learning with Domain Randomization for Particle Picking in Cryo-Electron Tomography

Mohamad Harastani¹,*, Gurudatt Patra¹, Charles Kervrann², Mikhail Eltsov¹,*

¹Department of Integrated Structural Biology, Institute of Genetics and Molecular and Cellular Biology, Illkirch, France

²Inria Center at University of Rennes, SAIRPICO Team, Cellular and Chemical Biology Unit, U1143 INSERM, UMR3666 CNRS, Institut Curie, PSL Research University, Campus universitaire de Beaulieu, Rennes Cedex, France

*Correspondence: Mohamad Harastani and Mikhail Eltsov {mohamad.harastani, mikhail.eltsov}@igbmc.fr

## Abstract

Cryo-electron tomography (cryo-ET) enables the three-dimensional visualization of biomolecules and cellular components in their near-native state. Particle picking, a crucial step in cryo-ET data analysis, is traditionally performed by template matching—a method utilizing cross-correlations with available biomolecular templates. Despite the effectiveness of recent deep learning-based particle picking approaches, their dependence on initial data annotation datasets for supervised training remains a significant limitation. Here, we propose a technique that combines the accuracy of deep learning particle identification with the convenience of the model training on biomolecular templates enabled through a tailored domain randomization approach. Our technique, named Template Learning, automates the simulation of training datasets, incorporating considerations for molecular crowding, structural variabilities, and data acquisition variations. This reduces or even eliminates the dependence of supervised deep learning on annotated experimental datasets. We demonstrate that models trained on simulated datasets, optionally fine-tuned on experimental datasets, outperform those exclusively trained on experimental datasets. Also, we illustrate that Template Learning used as an alternative to template matching, can offer higher precision and better orientational isotropy, especially for picking small non-spherical particles. Template Learning software is open-source, Python-based, and GPU and CPU parallelized.

## Keywords:

Cryo-ET; Particle Picking; Deep Learning; Domain Randomization; Physics-based Simulations; Cell Crowding; Molecular Mechanics
33

# Introduction

Recent technological advances in cryo-electron tomography (cryo-ET) expanded our capabilities to explore the three-dimensional (3D) cell architecture at the molecular scale. On one side, the automation of vitreous cryo-lamellae milling significantly enhanced the throughput of tomographic sample preparation ¹⁻³. On the other side, improvements in the hardware of transmission electron microscopes (TEM), especially new generations of direct electron detectors, enabled the routine acquisition of tomograms with the resolution that allows identification of biomolecules and determination of their structure and its variations, directly in their functional environment ⁴⁻⁸.

The cryo-ET pipeline begins with the vitrification of the sample by plunge or high-pressure freezing. In the case of bulky samples such as eukaryotic cells and tissues, this step is followed by thinning using cryo-focused ion beam milling or cryo-sectioning. A series of 2D images (micrographs) is then obtained by rotating the sample in cryo-TEM. The resulting images, referred to as a tilt series, are computationally reconstructed into a 3D volume called a tomogram ⁹,¹⁰.

Although cryo-ET allows the acquisition of high-resolution data from cellular samples, it faces serious limitations. Existing sample preparation and cryo-TEM setups limit the tilting range, typically within ±60 degrees. This angular limitation leads to a missing wedge region in Fourier space resulting in severe real-space data anisotropies commonly referred to as missing wedge artifacts ¹¹⁻¹⁴. In addition, the sensitivity of biological samples to radiation damage requires image acquisition with a limited electron dose, resulting in a compromised signal-to-noise ratio (SNR).

Locating copies of biomolecules in cryo-electron tomograms, known as particle picking, is important for understanding biomolecular distribution in cells and constitutes an initial stage in obtaining higher resolution 3D structures via averaging and classification ¹⁵,¹⁶. However, manual particle picking is labor-intensive, further complicated by the challenges posed by the missing wedge and low SNR.

A well-established method to automate particle picking using the similarity with an external template via cross-correlation is called template matching ¹⁷,¹⁸. In recent years, several deep learning-based methods have emerged and were shown capable of outperforming template matching, for some studies, in speed and accuracy ¹⁹⁻²⁴.

A specialized track of the SHape REtrieval Contest (SHREC) was launched to compare various deep learning methods for cryo-ET particle picking ²⁵. These evaluations were conducted on simulated datasets, where tomograms with ground truth annotations are available and are used for training. However, on experimental datasets, supervised deep
68 learning methods will require a training dataset, usually obtained via manual template
69 matching-assisted picking.

70 Open databases for 3D structural data of biomolecules, e.g., the Protein Data Bank (PDB)
71 ²⁶, give access to structures that can serve as templates for template matching. However,
72 despite the remarkable advancements in the physical modeling of cryo-ET data ²⁷⁻³⁰,
73 templates were not used for supervised training of deep learning models on customized
74 particle picking ²⁰,²⁵,³¹. Some methods found a use for simulated datasets in training deep
75 learning-based general-purpose particle picking (e.g. cryoYOLO ³²) or unsupervised
76 structural mining (e.g., TomoTwin ³¹). However, these methods face several limitations,
77 including molecular crowding, structural variations, or computational complexity ²²,²³,³¹,³³.

78 Using simulations derived from previously defined or inferred structures, such as those
79 obtained from *in vitro* or *in silico* studies, to train models to annotate similar structures in
80 cryo-electron tomograms would be highly advantageous over the time-consuming initial
81 annotation directly from *in situ* data. This raises the question of why such methodologies
82 are not more widespread. The literature points to a challenge known as the "synthetic-to-
83 real domain gap" ³⁴ where deep learning models trained on simulations can only operate
84 in the synthetic domain. This domain is characterized by attributes like texture,
85 illumination, noise, background, and other factors, which are notably different from those
86 in the real-world domain ³⁴.

87 Domain randomization is a state-of-the-art approach for addressing the domain gap
88 during synthetic dataset generation, specifically in the context of training models for object
89 classification and segmentation. Domain randomization is based on simulating non-
90 realistic scenarios that were proven powerful to supervised deep learning of models
91 capable of generalizing to real-world data ³⁵,³⁶. It hinges on three principles: 1) utilizing
92 models of target objects with randomized shape and pose variations; 2) incorporating a
93 mix of diverse objects termed "distractors", positioned in the background near the target
94 objects; and 3) integrating a broad spectrum of random rendering options.

95 In cryo-ET literature, addressing the synthetic-to-real domain gap for deep learning
96 training has been largely overlooked. Nevertheless, a method called CryoShift ³⁷ has
97 utilized domain adaptation and randomization techniques for subtomogram classification,
98 primarily focusing on mapping the synthetic data domain to resemble the real data during
99 classifier training. Yet, this method was limited to handling pre-isolated subtomograms of
100 molecules of interest and was not designed to process entire tomograms for particle
101 pickling or segmentation.

102 Here, we introduce "Template Learning", a domain randomization-based strategy for
generating cryo-ET simulated tomograms, incorporating considerations for molecular
crowding, structural variabilities, and data acquisition variations, with which we can learn deep learning models to achieve state-of-the-art performance in particle picking within cryo-ET experimental tomograms. In this paper, we detail our approach and systematically benchmark it against alternative methods, utilizing a recently published, exhaustively annotated *in situ* cryo-ET dataset. We also apply Template Learning to nucleosome picking within a densely populated cryo-ET sample of isolated mitotic chromosomes. We illustrate that models trained on simulated datasets, optionally fine-tuned on experimental datasets, outperform those exclusively trained on experimental datasets. Also, we illustrate that Template Learning used as an alternative to template matching, can offer higher precision and better orientational isotropy, especially for picking small non-spherical particles.

# Results

## Customized simulations to train deep learning models on particle picking

Template Learning is a streamlined pipeline for simulating cryo-ET synthetic data for training deep learning models on particle picking. The fundamental goal of Template Learning is to train deep learning models in three aspects:

1. Identifying targeted particles across diverse variations.

2. Differentiating target particles from other structures, especially in crowded environments.

3. Expanding the capabilities of the models to handle the inherent variations present in cryo-ET experimental data.

The practical application of these aspects is achieved through our proposed pipeline, illustrated in Fig.1 and further elaborated in the subsequent sections.

## Accounting for the structural and orientational variabilities

To enable trained deep learning models to effectively identify various variations of the target biomolecule, we utilize two following strategies:

1. We use multiple templates of one target biomolecule to account for compositional or significant conformational variabilities of the target particles.

2. We generate random flexible variations of each template using Normal Mode Analysis (NMA—a method for fast molecular mechanics simulations, see Methods
135 for details) ³⁸. The templates with their flexible variations are then simulated at
136 uniformly random orientations to account for orientational variations.

137 Templates in this study are typically atomic structures. However, since the availability of
138 atomic structures may vary for different target biomolecules, using lower resolution cryo-
139 EM density maps (volumes) is also possible and is explored in a subsequent section.

140 Accounting for the crowded environment

141 The existing literature on domain randomization for object recognition in natural images
142 simulates the target objects within a background containing a library of other random
143 objects ³⁹. These additional objects, distractors, obscure or cast shadows on segments
144 of the target objects. In this study, we selected 100 dissimilar protein assemblies, with
145 diverse molecular weights ranging from 30 kDa to 1 MDa to serve as elements within the
146 background (a comprehensive list of these proteins is available in Extended Data Fig.1).
147 To achieve a similar effect of distractors for cryo-ET data, i.e., overlapping at the level of
148 the tilt series and the missing wedge artifacts, these proteins must be positioned in close
149 proximity to the templates during data modeling.

150 Recent works ²⁷,⁴⁰ that involve simulating cryo-ET data for practical applications (e.g.,
151 deep learning) utilize an iterative brute-force random placement algorithm, involving the
152 rotation of a duplicate of a molecule in each iteration to find a suitable non-overlapping
153 position within the sample. Nonetheless, random placement leads to unstructured empty
154 spaces between the molecules that prevent achieving highly dense samples.

155 Two other approaches simulate molecular crowding of cryo-ET data ⁴¹,⁴². The first
156 represents molecules as spheres, based on calculating the minimum bounding sphere
157 for each molecule, and simulates crowding by optimizing a sphere-packing problem.
158 However, this approach is limited to generating high crowding only for spherical-shaped
159 molecules. The second builds on the first by including an additional step of molecular
160 dynamics simulations to enhance packing density. However, routine application of this
161 approach requires significant computational resources.

162 To achieve high molecular crowding while maintaining generality and efficiency, we drew
163 inspiration for a simplified approach from a recent concept on packing generic 3D objects
164 ⁴³, a solution we term the "Tetris algorithm." Our algorithm, illustrated in Fig.2, operates
165 on an intuitive principle: it places molecules iteratively, with each iteration positioning a
166 new molecule at a uniform random orientation as close as possible to those already
167 placed (the user only needs to choose the minimum distance). For more details on the
168 principles of the Tetris algorithm, see Methods.
169 Accounting for the real-world domain variance

170 Simulations can partially replicate the domain of real-world data; however, variations in
171 the real world can be unpredictable, and simulations simplify some complex phenomena
172 for practical reasons. Thanks to domain randomization ³⁴,³⁶, simulating an exact match of
173 the real world is not necessary for training deep learning models capable of generalizing
174 to real-world data. Instead, domain randomization focuses on training these models on
175 diverse simulated scenarios to minimize the influence of the domain on their capabilities.

176 In this work, we employ a cryo-ET physics-based simulator called Parakeet 28. Parakeet,
177 and its MULTIMEM backend 44, provide control over numerous parameters for simulating
178 cryo-ET hardware and the behavior of biological samples during data recording. While
179 many parameters related to the electron beam, lenses, detector, sample, and data
180 acquisition strategy can be controlled, it might not be necessary to randomize every
181 available parameter. On one hand, the combinations required to cover all conditions grow
182 exponentially with the number of variables. On the other hand, some variables might have
183 undesirable effects on the overall simulation. Here, we propose varying a few essential
184 parameters that significantly influence the output simulation: the electron dose, defocus,
185 tilting range, tilting step, and ice density, while giving other parameters default values.
186 The electron dose is crucial for varying the SNR and the irradiation effects on the sample.
187 The defocus plays a key role in controlling the Contrast Transfer Function (CTF) 45.
188 Varying the tilting range and step may not replicate experimental data acquisition
189 strategies, however, it can expose the deep learning model to variations that increase its
190 robustness, for example, to artifacts and imperfections resulting from tilt series alignment
191 and reconstruction. Lastly, different ice densities can simulate the variability of sample
192 thickness and solvent composition. A combination of three values for the defocus and two
193 values for each of the other variables results in 48 combinations that we found sufficient
194 for efficient domain randomization.

195 Benchmarking Template Learning for picking ribosomes *in situ*

196 In a recent study ²⁰, the first exhaustively annotated *in situ* cryo-ET dataset was published
197 (EMPIAR-10988). Here, we use this dataset to evaluate the proposed Template Learning
198 thoroughly.

199 In the following, we generate, based on the Template Learning workflow, a simulated
200 dataset to train DeepFinder ¹⁹ for ribosome annotation. We compare the performance of
201 this model (i.e., DeepFinder trained solely on simulations) to the contemporary
202 techniques, namely, DeePiCt ²⁰ and DeepFinder (trained solely on annotated
203 experimental data), and template matching.
We organized our experiments based on two principles. First, we benchmark the complete method that integrates the aforementioned concepts of Template Learning. Then, we conducted experiments where specific elements from the data simulation pipeline were intentionally omitted (e.g., the structural variations of the target template, the crowding, etc.) to assess their impact on performance and to offer guidelines for future users. Second, acknowledging that the size of a simulated dataset is user-defined (generally in deep learning, the more training data, and the more diversity, the better), we ensured the feasibility of routine use of the method by limiting its time requirement to a maximum of two days on a single GPU (within one day on a typical computing node with 4 GPUs, see Software requirements and availability section for details).

## The typical Template Learning workflow

To establish a Template Learning workflow, we proceeded with the following steps. We selected 6 eukaryotic ribosome PDB structures (PDB IDs: 4UG0, 4V6X, 6Y0G, 6Y2L, 6Z6M, and 7CPU). The 6 selected structures differ slightly in composition or conformation. By applying NMA, we generated 25 flexible variations for each structure (refer to the NMA section in Methods for details). While generating synthetic data, we employed the aforementioned general distractors (a full list of distractors is given in Extended Data Fig.1). To simulate a crowded environment, we generated volumes from the template structures (i.e., the 6 ribosome PDBs and their flexible variations) and distractors with a sampling rate of 16 Å using Eman2⁴⁶ software e2pdb2mrc. We used the generated volumes as input to populate 48 crowded volumetric distributions of size 192 x 192 x 64 voxel³ using the Tetris algorithm, with ribosome templates appearing at a frequency of one for every 5 distractors to produce a balanced volumetric density ratio balance between distractors and templates in the simulated tomograms (see Tetris algorithm section in Methods for details). Subsequently, we fed the sample information (i.e., atomic structures of templates and distractors with the positions and orientations of crowding generated using the Tetris algorithm) to Parakeet²⁸ to simulate 48 tilt-series using a dose symmetric tilting scheme using the parameters listed in Extended Data Tab.1. All other parameters adhered to default values for Volta Phase Plates (VPP) simulations within the Parakeet software. Subsequently, we binned the simulated tilt series to a similar sampling rate as the analyzed dataset (13 Å/pixel) and reconstructed them in IMOD⁴⁷ using Weighted Back Projection. We used the tomograms and their corresponding ground-truth template coordinates (around 6,500 simulated ribosomes) and segmentations (i.e., volumes of the same size as the tomograms where template instances appear in white over black background, illustrated in Extended Data Fig.2) to train DeepFinder¹⁹ using its default parameters settings (the list of parameters is given in Extended Data Tab.2).
We applied the DeepFinder simulations-trained model to the 10 VPP tomograms of EMPIAR-10988 dataset without any preprocessing, resulting in segmentation maps, i.e., ribosome segmentations based on the decisions of the model. Subsequently, we employed the MeanShift algorithm offered by the DeepFinder software, using a clustering radius of 10 voxels, to extract both the coordinates and size of these segments (count of the number of voxels of each segment), with and without the application of a mask (specifically, a cytosol mask sourced from the dataset, utilized to eliminate false positives outside the mask). We compare the extracted coordinates to the expert-validated annotations provided by the dataset at different levels of segment sizes used as a threshold (i.e., segments smaller than the threshold were removed). For this comparison, we maintained the criterion that two annotations—one from the output of the deep learning model and the other from the expert annotations—were considered to target the same particle if their spatial distance was within 10 voxels (a value consistent with the methodology of a previous study analyzed the same dataset <sup>20</sup>). An example of a tomogram, its corresponding segmentation map, and the coordinates set at a threshold that removed smaller objects compared to expert annotations are presented in Fig.3. To evaluate the performance of the method, we computed the overall Recall and Precision with and without masking (refer to the Methods section for more details on the assessment metrics). The point where Precision and Recall curves intersect was utilized to determine the F<sub>1</sub> score per tomogram for the dataset (shown in Fig.4A). The results, further illustrated in Fig.4I-J, reveal that Template Learning used to train DeepFinder exclusively on simulations, achieved state-of-the-art performance, outperforming template matching and previous deep learning methods trained solely on annotations derived from the same experimental dataset based on the median F<sub>1</sub> score (compared to values reported in <sup>20</sup>).

To reflect on these results, it is essential to highlight that the previously reported findings for DeePiCt and DeepFinder were based on models trained with 8 out of 10 fully annotated tomograms, containing approximately 20,000 ribosome annotations <sup>20</sup>. Hence, previous evaluations were conducted on 2 out of 10 tomograms, for annotating the remaining 5,000 ribosomes, utilizing three cross-validation splits (in each split, a random set of 8 tomograms was used for training and the remaining 2 tomograms for benchmarking). Additionally, each of the two models underwent a different training strategy to achieve optimal performance. In particular, in the case of DeepFinder, the training encompassed two classes—ribosomes and Fatty Acid Synthase (FAS)—as training solely on ribosomes led to suboptimal performance. On the other hand, DeePiCt was exclusively trained on ribosomes, as unlike DeepFinder, combining ribosomes and FAS during training led to suboptimal performance.

In contrast, the DeepFinder model trained solely on the simulations generated from the Template Learning workflow, not fine-tuned on any annotated experimental data, was
280 benchmarked on the complete dataset (i.e., all 10 tomograms). Hence, despite the
281 relatively modest increase in the F₁ score for the Template Learning method (0.85,
282 compared to the previous best of 0.83), it stands as the first demonstration that deep
283 learning can be trained effectively for picking a target structure, starting from only prior
284 templates, and domain-randomized simulations. Also, a model of the same network
285 architecture (DeepFinder) trained only on simulations outperforms its counterpart trained
286 only on annotated experimental datasets for cryo-ET particle picking.

287 Template Learning variations

288 We introduced several novel concepts in the Template Learning data simulation workflow,
289 demonstrating that, when combined, they achieve state-of-the-art performance for
290 supervised deep learning in annotating ribosomes within *in situ* cryo-ET tomograms. To
291 study the significance of each proposed concept and explore the potential for integrating
292 various modifications, we conducted additional experiments, which we have organized
293 into the following four categories:

294 Incorporating multiple atomic template structures and flexible variations

295 In three experiments, we progressively reduced the structural variability of input templates
296 while keeping the remaining parameters of the method unchanged. Our motivation behind
297 this series of experiments is to guide potential users regarding the outcomes of employing
298 multiple template structures and integrating flexible variations. These experiments can be
299 particularly useful to future studies, recognizing that, for certain biomolecules, the
300 presence of multiple structures and the feasibility of simulating molecular mechanics may
301 vary.

302 In one experiment, we retained from the templates a single structure and its flexible
303 variations. In a second experiment, we retained the six structures while excluding the
304 flexible variations. Finally, in a third experiment, we retained only a single structure,
305 eliminating other structures and all flexible variations. The trained deep learning models
306 resulting from the Template Learning workflows of these three experiments were used to
307 analyze the dataset as presented previously.

308 Fig.4(B-D, I) depicts the Recall-Precision curves and F₁ scores for the three experiments.
309 Notably, employing a single structure with flexible variations yielded comparable results
310 to employing multiple structures without flexible variations. We can infer from these
311 findings that introducing artificial flexible variations can serve as an effective
312 compensatory mechanism for the absence of multiple PDBs. This becomes particularly
313 relevant in studies where only a limited number of atomic structures, or even just a single
314 structure, is publicly accessible. Both approaches (i.e., a single PDB with flexibility and
multiple PDBs without flexibility) showed a slight adverse impact on the median F₁ score of the aforementioned typical template learning workflow, of approximately 5%. However, they still demonstrated comparable performance to previously reported results from supervised deep learning on annotated experimental data (compare Fig.4I/B-C and Fig.4J).

In contrast, employing a single template without any flexible variations yielded a stricter model characterized by higher Precision and lower Recall—translating to fewer false positives at the cost of more false negatives (see Extended Data Fig.3 for the output segmentation maps). Overall, the use of a single PDB template led to a relative reduction in the median F₁ score by approximately 10%. Nevertheless, it maintains a significant advantage over traditional template matching (compare Fig.4I/D and Fig.4J/Template matching).

## Incorporating distractors

In contrast to our proposed Template Learning method, previous studies did not emphasize the incorporation of a background combining a comprehensive array of dissimilar distractors. In ⁴⁸, no distractors were employed in the data simulation to train deep learning models on subtomogram segmentation. Two recent studies ²⁷,⁴⁰ utilized a small set of objects, serving as analogs to distractors in the simulation of data for training deep neural networks in regression tasks, including signal restoration and segmentation. These objects included gold fiducials, actin bundles, vesicles, and randomly placed small spheres that were relatively sparsely distributed in the volume. Our strategy is principally different; it utilizes other real biomolecules as distractors for a simple approximation of the realistic crowding of biomolecular environments.

To investigate the contribution of employing as many as 100 dissimilar distractors to annotate only a single target structure on the state-of-the-art performance of Template Learning, we conducted two additional experiments on distractors while keeping other method parameters unchanged. In one experiment, we removed all distractors from the simulated tomograms, following a methodology similar to that outlined in ⁴⁸. Fig.4(E, I) depicts the Recall-Precision curves and F₁ scores for this experiment, revealing that removing distractors from the simulated data results in hallucinations in the output segmentations leading to a high rate of false positives (refer to Extended Data Fig.3).

In a subsequent experiment, we explored if there are benefits of using many distractors compared to a limited number, similar to previous approaches ²⁷,⁴⁰. Subsequently, we reintroduced three distractors—small (PDB 1S3X), medium (PDB 5A20), and large (PDB 6UP6) proteins—while maintaining the previously established volumetric density ratio balance between distractors and templates in the simulated tomograms. Fig.4(F, I)
351 depicts the Recall-Precision curves and F₁ scores for this experiment. These results show that limiting the variability of distractors had a serious adverse impact on the picking 352 Precision. 353

354 These experiments suggest that exposing the deep learning model to a diverse range of 355 unwanted structures (i.e., distractors) alongside the target structures (i.e., templates) is 356 essential for the best performance. 357

## 358 Simulating crowding

359 To investigate the contribution of simulating high crowding, (i.e., using the Tetris algorithm 360 introduced in this work) to the state-of-the-art performance of Template Learning, we 361 aimed to design an experiment involving crowding reduction. Consequently, we continued 362 our investigation of the crowding reduction impact on Template Learning by adapting the 363 Tetris algorithm to systematically generate crowding levels aiming for 25-35% crowding 364 relative to the original settings while keeping all other parameters unchanged (to see how 365 crowding can be controlled in the Tetris algorithm, see the corresponding Methods 366 section). 367

367 Fig.4(G, I) depicts the Recall-Precision curves and F₁ scores for this experiment, 368 revealing that the deep learning model trained on a simulated dataset with less crowding 369 resulted in a 20-25% decrease in the median F₁ score. 370

## 369 Using a volumetric template

370 To the best of our knowledge, this work represents the first effort to train deep learning 371 models to utilize fully atomic structures and physics-based simulations as alternatives to 372 volume templates in traditional cryo-ET template matching. An advantage of employing 373 fully atomic structures is the utilization of rapid molecular mechanics simulations using 374 NMA. Particularly, our strategy involves coarse-graining the structures before NMA, and 375 interpolating the resulting motions to the original fully-atomic model (refer to the NMA 376 section in Methods for further details). 377

377 However, using volumetric templates, in the context of particle picking, can be 378 advantageous in specific scenarios, particularly when an initial structure can be derived 379 from the dataset. For instance, a subtomogram average at low resolution derived from a 380 partially annotated dataset can serve as an initial template to annotate more particles in 381 the dataset, enabling the generation of a higher-resolution average. 382

383 Consequently, we explored a method to incorporate a volume as a template in the 384 physics-based simulation, rather than relying solely on atomic structures. Existing 385 literature demonstrates the feasibility of generating pseudoatomic models from volumes, 386
385 primarily by estimating the volume using a set of 3D Gaussian functions 49,50. The mean 386 and standard deviations of these 3D Gaussians are calculated in a way that the 387 pseudoatomic model, when converted back to a volume (i.e., via summation of the 388 contribution of each 3D Gaussian to each voxel), estimates the volume with low error 389 (e.g., 5%). However, advanced cryo-ET physics-based simulators (e.g., Parakeet, used 390 in this work), do make use of the characteristics of these 3D Gaussians in the simulations, 391 instead, they use electron-atom interactions to model the image formation in the TEM. 392 Meaning, pseudoatomic structures based on previous works do not generate the 393 expected contrast when used in such physics-based simulators. 394

394 Hence, we devised a two-stage algorithm for real-time conversion of volumes into 395 pseudoatomic models that allows tuning the contrast by substituting some pseudoatoms 396 with actual phosphorus atoms (see Extended Data Fig.4 for illustration and Methods for 397 details). 398

398 To proceed with investigating the effect of using a volume as a template for ribosome 399 annotations, we employed the atomic structure of a ribosome (PDB 4UG0) to generate a 400 volume at 2 nm resolution using the Eman2 46 pdb2mrc software. It is noteworthy that in 401 the study 20 where the dataset (EMPIAR-10988) was published, a sub-nanometer 402 subtomogram average was obtained. Here, we explored a hypothetical scenario where a 403 2 nm resolution initial average is obtained by averaging a partially annotated subset of 404 the original dataset and investigated if the Template Learning workflow can employ this 405 structure for annotating more ribosomes. Subsequently, we transformed this volume into 406 a pseudoatomic structure, following the steps outlined in the corresponding Methods 407 section. As part of this conversion process, we introduced a substitution, replacing one- 408 third of the pseudoatoms with phosphorus atoms to approximate the contrast of 409 ribosomes observed in experimental data (Extended Data Fig.4). 410

410 We employed this pseudoatomic model to generate simulations using the Template 411 Learning workflow, by replacing the atomic structures that were previously used as input, 412 keeping the remaining parameters unchanged. 413

413 Fig.4(H, I) depicts the Recall-Precision curves and F₁ scores for this experiment, 414 indicating that the deep learning model trained on a simulated dataset with the mentioned 415 pseudoatomic structure exhibits a lower Precision and a higher Recall compared to 416 training with an actual atomic structure. We interpret these findings suggesting that lower 417 resolution template structure (2 nm instead of atomic resolution) removed certain details 418 from the simulation, resulting in a deep learning model that is more permissive but less 419 precise in its predictions. Nonetheless, the F₁ score of this experiment shows a 27% 420 increase relative to traditional template matching, despite that both methods used a 421 similar prior (i.e., a single volume ribosome template). 422
422 Adapting Template Learning to different domains

423 In the preceding section, we benchmarked Template Learning for annotating ribosomes
424 in a close-to-focus VPP cryo-ET dataset *in situ*. Hence, the previous Template Learning
425 workflow for ribosome annotation involved simulating data using VPP close-to-focus,
426 consistent with the data analyzed.

427 In addition to VPP data, the dataset (EMPIAR-10988) contains an additional set of 10
428 tomograms acquired with defocus (DEF tomograms) and without VPP. In the original
429 study ²⁰, the DeePiCt deep learning model was trained on the experimental VPP
430 tomograms and subsequently evaluated on DEF tomograms pre-processed with
431 spectrum matching (SM). SM enhanced the contrast of the DEF dataset to resemble that
432 of the VPP dataset, achieved by extracting a target spectrum from a VPP tomogram and
433 applying it to the DEF tomograms (see Extended Data Fig.5). This approach was referred
434 to as the DeePiCt "cross-domain" experiment.

435 We conducted two experiments to explore the impact of the data domain on Template
436 Learning. In the first experiment, we set up a Template Learning workflow targeting the
437 DEF domain. We accomplished this by generating a simulated dataset using a procedure
438 identical to the one outlined in the preceding section except for excluding VPP simulations
439 and employing a different range of defocus values (refer to Extended Data Tab.1 for
440 details). This simulated dataset was used to train and benchmark DeepFinder on
441 annotating ribosomes within the DEF tomograms, without undergoing SM preprocessing.
442 In the second experiment, we utilized the DeepFinder model trained on Template
443 Learning VPP simulations to annotate ribosomes within the DEF dataset that had
444 undergone SM preprocessing.

445 Fig.5 depicts the Recall-Precision curves and F₁ scores for the two aforementioned
446 experiments, compared to the previously reported result of the DeePiCt cross-domain
447 experiment. Both of our experiments yield comparable outcomes, outperforming the
448 DeePiCt cross-domain experiment by more than 10% on the median F₁ score. Notably,
449 the Template Learning-trained model for DEF data exhibited a higher Precision curve and
450 ultimately achieved a higher F₁ score, particularly when no mask (cytosol mask) was
451 applied.

452 The first experiment shows that the Template Learning workflow can be directly adapted
453 to the DEF domain avoiding the need for data preprocessing such as SM. The second
454 experiment shows that if a model trained on the Template Learning simulations of the
455 VPP domain exists, it can be reused in the cross-domain (i.e., the DEF domain) after SM,
456 and achieves a similar performance without repeating the entire Template Learning
457 workflow.
458
459
# Fine-tuning a deep learning model trained on Template Learning simulations using experimental data annotations

460
461
462
463
464
The literature on domain randomization indicates that models pre-trained on simulations and subsequently fine-tuned on real-world data outperform models trained exclusively on real-world data ³⁵. This approach can be particularly valuable to cryo-ET particle picking studies that involve challenging-to-annotate molecules, or when training on domain randomization simulations alone does not yield satisfactory results.

465
466
467
468
469
In the same dataset used for benchmarking Template Learning on ribosomes *in situ* (EMPIAR-10988), another molecule, FAS, is annotated. Previous attempts based on template matching and supervised deep learning on experimental datasets have reported challenges in annotating FAS, attributed to its distinctive shell-like structure, where particles have lower SNR compared to ribosomes ²⁰.

470
471
472
473
In the following, we present the benchmarking of Template Learning combined with DeepFinder, initially trained exclusively on a simulated dataset, on FAS annotation in EMPIAR-10988 tomograms. Subsequently, we gradually introduce fine-tuning based on the experimental data annotations to assess whether performance can be improved.

474
475
476
477
478
479
To establish a Template Learning workflow for FAS, we utilized two PDB structures (PDB IDs: 4V59 and 6QL5). By using NMA, we generated 80 flexible variations for each structure (refer to the NMA section in the Methods for details). We kept the remaining parameters for simulated data generation consistent with those aforementioned for the ribosome study, except for training the models on sphere segmentations due to the shell-like structure of FAS.

480
481
482
483
484
485
486
The F₁ score results of the simulation-trained DeepFinder model for FAS annotations are presented in Fig.7. The results show that on the VPP dataset, the DeepFinder model trained on the Template Learning workflow outperformed that trained exclusively on experimental data. The results on the DEF dataset gave an F₁ score ranging up to 22%, outperforming previous attempts that failed to pick any particle, based on the results reported in ²⁰. However, these results can still benefit from improvements compared to previously obtained results with DeePiCt trained on annotated experimental data.

487
488
489
490
491
492
We proceeded to fine-tune the simulations-trained DeepFinder model progressively using annotations from experimental data. In the following experiment, we fine-tuned it using annotations from 2 VPP tomograms containing approximately 150 FAS annotations. We kept all the DeepFinder training parameters to default (listed in Extended Data Tab.2), except for reducing the number of steps and epochs to 10 steps for 10 epochs (in place of 100 steps for 100 epochs) to avoid overfitting (since the number of training data
493 examples is low). We performed three cross-validation experiments (i.e., in each 494 experiment, the data is randomly split into 2 tomograms for training and 8 for 495 benchmarking). The results presented in Fig.6 show a significant increase in the median 496 $F_1$ score compared to training on simulated data only.

497 In a subsequent experiment, we fine-tuned the simulation-trained DeepFinder model on 498 8 VPP tomograms containing approximately 600 FAS annotations and inferred the model 499 on the remaining 2 tomograms, using 3 cross-validation splits. Again, we kept all the 500 DeepFinder training parameters to default, except for the number of steps and epochs, 501 keeping 10 steps for 30 epochs (again, to prevent overfitting). The results presented in 502 Fig.6 show a significant increase in the median $F_1$ score, outperforming previous 503 supervised deep learning methods trained solely on experimental data.

504 Finally, we performed a cross-domain experiment, where we fine-tuned the simulation- 505 trained DeepFinder model on the VPP dataset, and applied this model to annotate FAS 506 in the DEF dataset after SM preprocessing. Consistently with previous findings, the 507 results presented in Fig.6 show that training on simulations and fine-tuning on 508 experimental data outperforms training only on experimental data.

509
510

Template Learning offers higher Precision and better orientational
isotropy than template matching on nucleosome picking

511 In cryo-ET data processing, localizing a target biomolecule in a new dataset is a common
512 objective. The application of supervised deep learning to this task requires an initial
513 training dataset of particles of interest extracted directly from the new data. Such a dataset
514 is usually in the order of thousands of particles in different orientations, where more
515 particles are needed for better performance. Manually annotating these particles is time-
516 consuming, making template matching followed by manual elimination of obvious false
517 positives and further curation through subtomogram averaging and classification the most
518 common approach ¹⁹,²⁰. The principle of Template learning offers a time and computing-
519 efficient alternative to this complex procedure by annotating new data directly in a single
520 step using a model trained on synthetic data.

521 In this section, we explore the efficiency of Template Learning for nucleosome annotation
522 within a new cryo-ET dataset of partially decondensed mitotic chromosomes *in vitro* (refer
523 to the Method section for sample preparation details) and compare it with template
524 matching-based annotation routine.

525 A tomogram central slice of our data is shown in Fig.8A. Despite this dataset being
526 isolated chromosomes *in vitro*, it contains an abundance of structures other than
nucleosomes that can be false positives which are: DNA linkers, gold nanoparticles, percoll, and the non-histone components of chromatin.

## 529 Template Learning for picking nucleosomes

To establish a Template Learning workflow, we utilized six nucleosome template structures with PDB IDs: 2PYO, 7KBE, 7PEX, 7PEY, 7XZY, and 7Y00. By applying NMA, we generated 100 flexible variations for each structure (more details can be found in the NMA section of Methods). The remaining parameters of the Template Learning workflow were configured similarly to those used for the workflow established for ribosome and FAS annotation, with a few notable exceptions explained below.

Firstly, recognizing the smaller size of nucleosomes compared to ribosomes, we adjusted the frequency of appearance of nucleosome templates to one nucleosome for every two distractors (in contrast to one ribosome for every five distractors). This adjustment aimed to maintain a balanced volumetric density ratio between distractors and templates in the simulated tomograms.

Secondly, recognizing that nucleosomes have fewer atoms than ribosomes, we observed increased speed in the execution of the physics-based simulator (i.e., Parakeet). This allowed for the generation of larger tomograms for training, all within the same runtime for the ribosome study (see Software requirements section for details). In particular, the tomogram size generated in the Tetris algorithm here was set to 256x256x64 (for a pixel size of 16 Å), in contrast to 192x192x64 used for ribosome simulations.

Lastly, we adjusted the pixel size of the simulated data to 8 Å through binning, closely approximating the pixel size of our experimental tomogram, which had been previously binned four times before annotation (the unbinned pixel size was 2.075 Å).

Consequently, we trained DeepFinder on the resulting Template Learning simulations and applied it to segment our data. The corresponding segmentation map (score map), is shown in Fig.7B. Upon visual inspection, it is evident that that model has high Precision in localizing nucleosomes. Subsequently, we employed the MeanShift algorithm from the DeepFinder software to extract the corresponding annotations for the nucleosomes, utilizing a clustering radius of 6 pixels, approximately equivalent to the nucleosome's radius at this pixel size. We excluded particles close to the edges of the tomogram, the air-water interface, and the sample-carbon interface.

Following this process, we obtained 18k annotated particles and proceeded with the reference-free subtomogram averaging procedure using Relion V4.0 ¹⁵ by generating an initial model from the data, followed by two stages of refinement at binning 4 and binning
561 2, resulting in resolving the nucleosome structure at 12.8 Å (at 0.143 FSC threshold)
562 resolution shown in Fig.7B.

563 The angular distribution of the averaged particles seems isotropic, showcasing that the
564 Template Learning method is not biased to certain orientations.

565 To further verify the Precision of nucleosome identification, we classified the particles into
566 10 and 50 classes (presented in Extended Data Fig.6). All class average results showed
567 well-recognizable nucleosome shapes with DNA gyres, with the major differences being
568 in the heterogeneity of the DNA entry-exit segments. We also performed a local resolution
569 assessment using ResMap 51 (presented in Extended Data Fig.7) and observed that
570 resolution has ranged from 12-20 Å, with the highest resolution in the DNA close to the
571 core histones, and the lowest resolution around the DNA entry-exit segments.

572 Our averaging and classification results show that our Template Learning workflow
573 generated precise nucleosome annotations of uniform orientations, facilitating high-
574 throughput subtomogram averaging without the need for multiple rounds of classification
575 to eliminate false-positive particles.

576 Template matching for picking nucleosomes

577 We applied template matching starting from a nucleosome template structure generated
578 from the PDB 2PYO using the default procedure in PyTom ¹⁸. We then applied the
579 template matching procedure to our new data using an angular sampling of 7° increment
580 (45,123 rotations).

581 The score map is shown in Fig.7C. The visual inspection of the score map shows some
582 high signal for some nucleosomes also identified by the Template Learning procedure
583 (Fig.7A), but also obvious false positive signal for other objects (an example of high false
584 signal for the Percoll particles is shown at the center of the zoomed-in image in Fig.7C).

585 To have a meaningful and fair comparison with the Template Learning procedure, we
586 extracted from the same tomographic region the 18k particles showing the highest cross-
587 correlation scores. Unlike the results of Template Learning, a brief visual inspection of the
588 template matching peaks showed 50 obvious false positives corresponding to the 10 nm
589 gold and Percoll particles. The false positives were removed, and data were processed
590 Relion V4.0 ¹⁵. The reference-free initial model was generated, followed by a stage of
591 refinement at binning 4. Unlike Template Learning, the refinement did not result in a
592 significant improvement of the resolution of the initial model (judged by the shape and the
593 FSC), indicating the presence of a significant ratio of non-nucleosome particles among
594 the annotations. In agreement with this assumption, classification, and alignment into 10
595 classes at the same binning resulted in only 2 nucleosome-like classes that sum to 10.25k
picks (57% of the original picks). The particles of these 2 classes were selected for a further refinement process at binning 4 reached Nyquist resolution, and the further refinement at binning 2 led to 12.9 Å resolution which is comparable to that achieved with Template Learning.

Importantly, the angular distribution of nucleosomes annotated by template matching showed a significant imbalance towards side views compared to round top views (Fig.7C). Our simulations (Extended Data Fig.8) showed that due to the cylindrical shape of nucleosomes and the missing wedge problem of cryo-ET data, the constrained cross-correlation between a template and the particles is a function of the orientation, where side views show higher cross-correlation than top and oblique views. This problem results in extracting only side views of nucleosomes at an adequate Precision.

The orientation bias of template matching hampers its efficiency for particle detection and is prone to resolution loss in subtomogram averaging. In our case, the resolution of the nucleosome was not affected significantly, because its pseudo-symmetric cylindrical shape allowed the complete 360° orientation range of side views to have cross-correlation sufficient for detection (Fig.7C). However, this constraint may pose a more significant challenge when annotating particles with asymmetric non-spherical shapes. Our results demonstrate that Template Learning is capable of overcoming this limitation.

## Conclusion

In this study, we introduced Template Learning, a novel approach that blends the simplicity of Template Matching, requiring only template structures as input, with the advanced capabilities of Deep Learning, thereby reducing the dependence on extensive, labor-intensive annotated datasets for supervised training. Our method effectively tackles key challenges in cryo-ET particle picking, such as structural variability, cellular crowdedness, and experimental data variance, through a refined implementation of domain randomization to create simulated data for deep learning training. The efficiency of Template Learning arises from its focus on essential aspects, enabling the well-established DeepFinder model, known for its particle annotation capabilities, to train without the need for highly realistic cellular environment simulations. This approach simplifies the process for users, allowing them to generate effective simulations that train DeepFinder in annotating specific biomolecules in cryo-ET data, using only templates in a streamlined yet customizable simulation process. Template learning can utilize any atomic models, including those generated by artificial intelligence-based tools, as templates. We also confirm the feasibility of relevant template generation from low-resolution cryo-EM densities.
631 In this paper, we employed a comprehensively annotated *in situ* cryo-ET dataset for 632 ribosomes and FAS, covering defocused cryo-ET and VPP, to validate Template 633 Learning's efficacy and versatility. We demonstrated how to enhance DeepFinder's 634 supervised training by initially training on simulations, and then fine-tuning on 635 experimental data. Notably, DeepFinder, when trained solely on simulations, surpassed 636 previously established training on experimental annotations for ribosomes. Furthermore, 637 the combined approach of pre-training on simulations and subsequent fine-tuning on 638 experimental data showed improved performance for FAS. 639

640 We evaluated the efficacy of our method for the identification of a known target molecule 641 in tomographic data without any prior annotation, focusing on the case of localization of 642 nucleosomes in isolated chromosomes *in vitro*. Template Learning outperformed the 643 conventional template matching and classification routine typically employed for this type 644 of task both in terms of annotation Precision and orientational isotropy. Notably, all 645 annotations generated by Template Learning could be directly used for achieving high- 646 resolution subtomogram averaging without requiring any manual or classification-based 647 curation. 648

647 In future work, we envision adapting our framework to locate structures bound to other 648 biomolecules such as membrane proteins. Another challenge will be to reduce 649 computational issues as physics-based simulations are still resource-intensive. 650

650 Conclusively, we believe Template Learning's versatile and straightforward framework 651 marks it as a timely and potent tool for a broad spectrum of cryo-ET studies, ready to 652 make significant impacts in the field. 653

653 Software requirements and availability

654 Template Learning code is freeware, open source, GPU and CPU optimized, and fully 655 implemented in Python (https://github.com/MohamadHarastani/TemplateLearning). It 656 provides all the steps that are required to reproduce the results presented in this article 657 and can be straightforwardly applied to other studies. It provides wrappers for ProDy, 658 Eman2 and Parakeet that will be installed during the installation. The software was tested 659 on our workstations (Dell Precision 5820, Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz, 96 660 GB DDR4 RAM, 2 X NVIDIA RTX A6000 or NVIDIA Quadro RTX 8000). The time required 661 to perform the Template Learning workflow ranged from 1 to 2 days (depending on the 662 number of used GPUs). Installation and user guides are available on GitHub.
663

Data availability

664

665

666

667

668

669

The tilt series, tomogram, coordinates, and other metadata necessary to reproduce the nucleosome subtomogram averaging are deposited on EMPIAR via the accession code EMPIAR-11969. The subtomogram averages resulting from the Template Learning and template matching workflows are available on EMDB via the accession codes EMD-19823 and EMD-19825 respectively. Other benchmarking datasets employed in this work are publicly available.

670

Author contributions

671

672

673

674

675

676

677

678

Investigation, conceptualization, and methodology were done by MH and ME with contributions from all authors. Software programming and computational experiments were done by MH. Sample preparation and data acquisition were done by GP under the supervision of ME. Subtomogram averaging was done by MH and GP under the supervision of ME. Validation and results interpretation were done by all authors. Writing the original draft was done by MH with input from GP. Review and editing were done by MH, ME, and CK. Funding acquisition was done by ME and CK. Project administration was done by ME.

679

Conflicts of Interest

680

The authors declare no conflict of interest.
<figure><img src="image_2.png" /><figcaption>Fig.1: Template Learning workflow for generating simulations used for supervised deep learning used for cryo-ET picking of target particles.</figcaption></figure>

Template structures are used as input, commonly differing by some structural variations of the target biomolecule (we mean by the target biomolecule, the structure that the deep learning model is desired to learn to annotate in cryo-ET tomograms, in the figure it is the nucleosome). The input templates are augmented with flexible variations based on fast molecular mechanics simulations (see text for details). A general set of other proteins, different from the input templates, are used as “distractors” when generating synthetic data. The templates, along with their flexible variations, and distractors are placed at random orientations in close proximity using the “Tetris algorithm”— an algorithm proposed in this work to enable fast simulation of high molecular crowding. A physics-based simulator (Parakeet 28) is used to generate synthetic cryo-ET data using a set of different parameters that were chosen to produce domain randomization at the lowest cost, which are the electron dose, tilt range, tilt step, ice density, and defocus. The resulting output comprises tomograms and ground truth segmentations and coordinates corresponding to the templates. The generated data is subsequently utilized to train deep learning models (e.g., DeepFinder 19) for experimental tomogram segmentation and particle picking. The PDB IDs used for templates in this figure are
2PYO, 7KBE, 7PEX, 7PEY, 7XZY, and 7Y00, and for distractors are 3QM1, 7NIU, and 6UP6. The display of structures was performed using ChimeraX 52 and IMOD47 software.
<figure><img src="image_3.png" /><figcaption>Fig.2: The Tetris algorithm for generating crowded 3D samples.</figcaption></figure>
**A** The flowchart of the algorithm, demonstrated in 2D for simplicity. The molecules that are desired to populate the sample are used as input. In the initial iteration, a molecule is positioned at the center of an expanded volumetric sample, forming the current output. In subsequent iterations, a new molecule undergoes binarization and dilation. By multiplying the binarized version with a large positive value and subtracting the result from the dilated version, an insertion shell is generated (referred to as 'in-shell'). This insertion shell is then correlated with a binarized version of the current output volume, resulting in a correlation map (referred to as 'c-map'). Positive values in the c-map (white voxels) represent viable positions for adding the new molecule without intersection, whereas negative and zero values (black and gray voxels, respectively) represent positions where intersection with existing molecules occurs or where the distance from other molecules is too large. If the correlation map is not entirely negative, the molecule is added to the current output at the index of the maximum entry, ensuring it avoids intersections while maintaining the highest compactness. **B** An example of the Tetris algorithm input and output at each iteration, alternating between placing templates (in green) and distractors (in red). The numbers in **B** correspond to the output at different iterations of the algorithm in **A**. **C** An example of a Tetris algorithm output in 3D.
<figure><img src="image_4.png" /><figcaption>Fig.3: Template Learning trains DeepFinder to single molecule precision on segmenting and annotating ribosomes *in situ*.</figcaption></figure>

Left: Display of a 2D central slice of a VPP tomogram. Middle: Segmentation of ribosome from the Template Learning workflow shown in black and white, with a 50% transparent overlay of the mask. Right: resulting annotations within the mask, obtained by converting segmentations to annotations using the MeanShift algorithm (provided by the DeepFinder ¹⁹ software) using a clustering radius of 10 voxels, compared to expert annotations. Comparison includes True Positives (TP), False Positives (FP), and False Negatives (FN) relative to expert annotations. The tomogram, mask and expert annotations are sourced from the EMPIAR-10988 dataset.
<figure><img src="image_5.png" /><figcaption>Fig.4: DeepFinder trained on Template Learning simulations only outperforms previous techniques for ribosome annotations in cryo-electron tomograms.</figcaption></figure>

(I) Template Learning variations (A is the full method)

Deep learning models (DeepFinder) trained solely on simulations – benchmarked on 10/10 experimental tomograms (25k annotations)

<table><thead><tr><th>Principle studied</th><th>Distinction of the simulated dataset</th><th>Without mask F<sub>1</sub> median</th><th>With cytosol mask F<sub>1</sub> median</th></tr></thead><tbody><tr><td rowspan="4">Incorporating multiple atomic template structures and flexible variations</td><td>A Multiple PDB templates + flexibility</td><td>0.79</td><td><strong>0.85</strong></td></tr><tr><td>B Single PDB template + flexibility</td><td>0.76</td><td>0.80</td></tr><tr><td>C Multiple PDB structures</td><td>0.74</td><td>0.80</td></tr><tr><td>D Single PDB structure</td><td>0.72</td><td>0.74</td></tr><tr><td rowspan="2">Employing distractors</td><td>E No distractors (0 out of 100)</td><td>0.18</td><td>0.63</td></tr><tr><td>F Less distractors (3 out of 100)</td><td>0.70</td><td>0.77</td></tr><tr><td>Simulating crowding</td><td>G Less crowding (~30% of original)</td><td>0.54</td><td>0.70</td></tr><tr><td>Using a volumetric template</td><td>H Volume template (20 Å resolution)</td><td>0.65</td><td>0.76</td></tr></tbody></table>

(J) Previous results

<table><thead><tr><th>Method</th><th>Training / Benchmarking</th><th>With cytosol mask F<sub>1</sub> median</th></tr></thead><tbody><tr><td>DeePiCt (trained solely on experimental data)</td><td rowspan="2">8 tomograms (20k annotations) / 2 tomograms (5k annotations)</td><td>0.79</td></tr><tr><td>DeepFinder (trained solely on experimental data)</td><td>0.83</td></tr><tr><td>Template Matching</td><td>NA / 10 tomograms (25k annotations)</td><td>0.49</td></tr></tbody></table>

Performance benchmarking and comparative analysis of the Template Learning on ribosome annotation on the S. pombe dataset (EMPIAR-10988, 10 VPP tomograms).
**A-H** Performance measures for 8 variations of Template Learning settings. The curves depict the overall Precision and Recall against the volume of the segmented region (horizontal axis). Precision is evaluated with and without masking (±mask). Boxplots show the F₁ score per tomogram at the threshold corresponding to the highest overall F₁ score (often coinciding with the intersection of the Precision and Recall curves). **I** Table listing the differences between the experiments in **A-H** comparing their results based on their median F₁ scores (bold value is the highest among proposed and previous techniques). **J** Table of previous results with their reported performance based on the findings in ²⁰. Comparison of segmentations resulting from the different experiments on an example tomogram are shown in Extended Data Fig.3.
<caption>**Fig.5: Template Learning does not necessitate data preprocessing for DEF tomograms.**</caption>

(A)

Template Learning
Trained for DEF

<figure><img src="image_6.png" /></figure>

(B)

Template Learning
cross-domain

<figure><img src="image_7.png" /></figure>

(C)

Template Learning

Deep learning models trained on simulations - benchmarked on 10/10 DEF experimental
tomograms

<table><thead><tr><th>Experiment</th><th>Without mask<br>F<sub>1</sub> median</th><th>With cytosol<br>mask<br>F<sub>1</sub> median</th></tr></thead><tbody><tr><td><strong>Model trained on DEF simulated dataset</strong></td><td></td><td></td></tr><tr><td>Benchmarking performed on experimental DEF<br>dataset <strong>without SM</strong> preprocessing</td><td>0.61</td><td>0.70</td></tr><tr><td><strong>Model trained on VPP simulated dataset</strong></td><td></td><td></td></tr><tr><td>Benchmarking performed on experimental DEF<br>dataset <strong>with SM</strong> preprocessing</td><td>0.56</td><td>0.72</td></tr></tbody></table>

A

B

Previous results

<table><thead><tr><th>DeePiCt cross-domain experiment</th><th>With cytosol<br>mask<br>F<sub>1</sub> median</th></tr></thead><tbody><tr><td><strong>Model trained on 10 experimental VPP tomograms (25k annotations)</strong></td><td></td></tr><tr><td>Model benchmarked on experimental DEF dataset with SM<br>preprocessing</td><td>0.59</td></tr></tbody></table>

Performance benchmarking and comparative analysis of Template Learning workflow applied to ribosome annotations within the S. pombe dataset (EMPIAR-10988, 10 DEF tomograms). **A-B** The curves depict the overall Precision and Recall against the volume of the segmented region (horizontal axis). Precision is evaluated with and without cytosol masking (±mask). Boxplots show the F₁ score per tomogram at the threshold corresponding to the highest overall F₁ score (coinciding with the intersection of the Precision and Recall curves). **C** On the top, a table listing the differences between the experiments in **A-B**, comparing their results based on their median F₁ scores. On the bottom, previously reported performance based on the findings in ²⁰.
<figure><img src="image_8.png" /><figcaption>Benchmarked on VPP experimental tomograms</figcaption></figure>

<table><caption>Benchmarked on DEF experimental tomograms</caption><thead><tr><th></th><th>Method</th><th>F<sub>1</sub> median</th></tr></thead><tbody><tr><td rowspan="3">Template Learning</td><td>Using only simulated data for training and 10/10 of VPP experimental tomograms for benchmarking</td><td>0.18</td></tr><tr><td>Using simulated data for training, followed by 2/10 VPP experimental tomograms for fine-tuning and 8 VPP tomograms for benchmarking</td><td>0.38</td></tr><tr><td>Using simulated data for training, followed by 8/10 VPP experimental tomograms for fine-tuning and 2 VPP tomograms for benchmarking</td><td>0.49</td></tr><tr><td>DeePiCt</td><td>Using 8/10 VPP experimental tomograms for training and 2 VPP tomograms for benchmarking</td><td>0.46</td></tr><tr><td>DeepFinder</td><td>Using 8/10 VPP experimental tomograms for training and 2 VPP tomograms for benchmarking</td><td>0.11</td></tr></tbody></table>

<table><thead><tr><th></th><th>Method</th><th>F<sub>1</sub> median</th></tr></thead><tbody><tr><td rowspan="2">Template Learning</td><td>Using only simulated data for training and 10/10 of DEF experimental tomograms for benchmarking</td><td>0</td></tr><tr><td>Using simulated data for training, followed by 10/10 VPP experimental tomograms for fine-tuning and 10/10 DEF tomograms for benchmarking</td><td>0.20</td></tr><tr><td>DeePiCt</td><td>Using 10/10 VPP experimental tomograms for training and 10/10 DEF tomograms for benchmarking</td><td>0.15</td></tr></tbody></table>

Performance benchmarking and comparative analysis of Template Learning applied to FAS annotation within the S. pombe dataset (EMPIAR-10988, 10 DEF tomograms). The results show the performance of the Template Learning workflow used for training DeepFinder on simulations, followed by different levels of fine-tuning on experimental data, compared to results based on the findings in ²⁰.
<figure><img src="image_9.png" /><figcaption>**Fig.7:** Template Learning outperforms traditional template matching in Precision and orientational isotropy in annotating nucleosomes.</figcaption></figure>

**A** Central slice and zoomed-in 10-slice average of a cryo-ET tomogram of partially decondensed mitotic chromosomes *in vitro*. **B-C** The results of applying Template Learning (6 PDB structures as input, other parameters kept default) and traditional template matching (using PyTom with a nucleosome template) for annotating nucleosomes within the tomogram in A, followed by reference-free subtomogram averaging in Relion V4. **B** Top: score map resulting from training DeepFinder on
nucleosome annotations using simulated datasets only from the proposed Template
Learning workflow. Below: results of subtomogram averaging using 18k annotations
extracted using 6 voxel clustering radius of the MeanShift algorithm (provided by the
DeepFinder software). The results show initial model generation, followed by two stages
of auto-refinement, and the angular distribution of averaged particles. The refinement
has converged without particle curation and the angular distribution of averaged
particles is isotropic. C Top: score map resulting from traditional template matching.
Below: results of subtomogram averaging using the top 18k local maxima from the
score map. The first step was an initial model generation, followed by a stage of
refinement. The first stage of refinement did not improve the resolution of the initial
model, therefore, it was followed by simultaneous classification and alignment into 10
classes. The particles corresponding to nucleosome-like classes (57% of the original
18k particles) were used for two stages of refinement. The results show a slightly lower
resolution average than the one obtained using Template Learning and show an
anisotropic angular distribution.
695

# Methods

696

## Normal Mode Analysis (NMA)

697

698

699

700

701

702

Our study employed NMA on various atomic structures, prioritizing computational efficiency over precise molecular mechanics simulation, as the primary focus was on training the network on variations of the target biomolecule rather than precisely predicting its conformations. This emphasis on expanding the domain of the synthetic data aimed to capture a broad range of variations, not solely constrained to those witnessed in experimental data.

703

704

705

706

707

708

709

710

711

To manage computational demands, we adopted a coarse-grained modeling strategy ⁵³, focusing on Carbon Alpha and Phosphorus atoms for NMA and subsequently expanding (interpolating) the modes to the complete atomic structure. Typically, a substantial number (three times the number of atoms in the atomic structure) of normal modes are calculated simultaneously, which can be ordered based on their frequency. Low-frequency normal modes are particularly useful as they represent global movements, while high-frequency normal modes depict local movements. Prior studies have consistently highlighted the significance of these low-frequency, high-collective modes in capturing experimentally observed conformational variabilities ⁵⁴,⁵⁵.

712

713

714

715

716

717

718

719

720

The user of NMA retains control over the selection of modes and the amplitude of deformation to generate flexible variations, ensuring a balance between simulating variability and maintaining structural integrity. As a guideline, we chose the first 20 low-frequency normal modes for generating random flexible variations. Setting the amplitude range to 150 for ribosomes and 100 for nucleosomes resulted in Root Mean Square Deviation (RMSD) values of approximately 1 Å and 2 Å from the initial structures, respectively. Adjusting this parameter incurs minimal computational costs (a few minutes) by generating variations and assessing the RMSD from the initial structure, for instance using ChimeraX ⁵².

721

722

723

724

725

726

The computational time required for the calculation of NMA varies depending on the size of the input atomic structure. Specifically, for an atomic structure of 200 kDa, such as a nucleosome, the calculation typically takes approximately 1 minute. In contrast, for larger structures like a ribosome weighing around 4.5 MDa, the calculation time extends to about 1 hour when processed on a single CPU (benchmarked on Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz).

727

728

In this work, all NMA calculations were performed using ProDy ⁵⁶, a widely recognized open-source Python package for molecular mechanics simulations.
## 729 Tetris algorithm for generating highly crowded samples

730 The Tetris algorithm (flowchart shown in Fig.2) utilizes input volumes of the biomolecules, 731 essentially volumetric versions of the PDB templates and distractors, to generate a 732 densely packed sample comprising randomly rotated biomolecule copies. It 733 systematically places one randomly rotated molecule at a time in the closest proximity to 734 previously positioned molecules. The algorithm terminates when a new copy cannot be 735 placed in the sample.

736 In the initial step, the Tetris algorithm places a molecule at the center of a larger volumetric 737 sample, the size of which is set by the user, establishing the current Tetris output. 738 Subsequent iterations involve randomly rotating a new molecule, followed by its 739 conversion to binary form after applying low-pass filtering and thresholding. While the 740 values for low-pass filtering and the volume threshold are empirical and subject to 741 variation, a standard deviation sigma of 2 for a Gaussian low-pass filter, and a volume 742 threshold of 100 have generally proven effective in our trials (note that the volumes are 743 generated from PDBs using Eman2 46 e2pdb2mrc software). This copy (molecule) is then 744 diluted using a ball-like structural element of user-defined radius. Multiplying the output 745 of the binarized volume by a large positive number (integer infinity in programming) and 746 subtracting the result from the dilated volume, results in an "insertion shell", which serves 747 as a spatial guide for placing the next molecule. Notably, the radius of the structural 748 element provides direct control over the compactness of the sample.

749 The correlation between this insertion shell and a binary version of the current output (the 750 binary version is smoothed and thresholded the same way as above) generates a 751 correlation map. White voxels in the correlation map represent potential locations for 752 adding the copy, ensuring the desired intermolecular distance. In contrast, black and gray 753 voxels represent locations where the copy would intersect with previous copies or be too 754 distant from other molecules, respectively. If the correlation map is not entirely negative 755 (stopping criteria), the current copy is added at the index of the maximum entry, 756 corresponding to the location closest to the sample within the desired distance.

757 During Tetris sample generation, the oriented coordinates used for placing molecules in 758 the volumetric sample are preserved. To generate cryo-ET simulations of this sample, the 759 oriented coordinates are passed to the physical-based simulator (Parakeet).

760 Generating volumetric samples using the Tetris algorithm with molecules at a resolution 761 of 32 Å and a voxel size of 16 Å³, as implemented in our current software, achieves a 762 placement rate of approximately 1000 molecules per minute. The creation of a typical 763 Tetris, with dimensions of 3072 x 3072 x 1024 Å³, requires approximately 5 to 6 minutes 764 on a single CPU (benchmarked on Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz) and can
765 accommodate around 5000 molecules. Consequently, the generation of 48 Tetris
766 samples necessary for the tests performed in this study takes approximately 4.5 hours on
767 a single workstation.

768 Performance assessment metrics of particle picking

769 In evaluating how well our deep learning models pick particles, we used the commonly
770 employed metrics in particle localization studies, namely: the Precision, Recall, and F₁
771 score. Precision measures the percentage of correct picks out of all the particles selected
772 by the model (True Positives / Positives). Recall measures how many of the actual
773 particles were correctly identified by the model (True Positives / Ground Truth). F₁ score,
774 being the harmonic mean of Precision and Recall, provides a comprehensive measure of
775 overall model performance.

776 Converting volumes to pseudoatoms with contrast tuning

777 To convert a volume to a pseudoatomic structure that allows physics-based simulations,
778 we devised a two-stage algorithm. In the initial stage, we binarize the volume, similar to
779 the process of generating tight 3D masks in cryo-ET and cryo-EM studies. Subsequently,
780 we assign a pseudoatom to the position of every non-zero voxel, resulting in a point cloud
781 as output. We parse this point cloud into a PDB format, representing each point (i.e.,
782 pseudoatom) as a density (DENS line in PDB format). Then, the resulting file is ready to
783 be integrated into the physics-based simulations. However, projections of this structure
784 lack the desired contrast compared to those observed in experimental data. This
785 discrepancy arises from the inherent modeling of electron-atom interactions in physics-
786 based simulators. To address this contrast issue, in the second stage, we substitute a
787 fraction of pseudoatoms with actual atoms, such as phosphorus (e.g., replacing one out
788 of three pseudoatoms with phosphorus atoms). This substitution is not aimed at
789 replicating the precise chemical composition of the structure (a direction that may be
790 explored in future research) but rather to empirically enhance contrast. We determine this
791 conversion ratio (i.e., pseudoatoms to phosphorus) by projecting the pseudoatomic model
792 in 2D and comparing it to the experimental data targeted for analysis.

793 Cryo-ET of partially decondensed mitotic chromosomes

794 Chromosome Purification

795 Chicken DT40 cells were maintained at a density of 8-10 x 10⁵/ml at 39 °C and 5% CO₂.
796 The cells were synchronized to mitosis using 0.5 µg/ml of nocodazole for 13 hr resulting
797 in a mitotic index of 70 - 90%. The mitotic chromosome isolation was performed using the
798 classic polyamine-EDTA buffer-based method optimized for DT40 57. The synchronized
799 cells were harvested by centrifuging the culture at 1600 g for 5 mins at 4°C. The cells
800 were swollen at room temperature for 5 minutes in a low salt buffer in the presence of
801 polyamines and then lysed using a dounce homogenizer. The lysate was then overlaid
802 on a step sucrose gradient (15%, 60%, and 80% [w/v]) centrifugation. The 60-80%
803 interface was recovered and sedimented on a self-forming percoll gradient in the
804 presence of polyamine. The band containing the chromosomes was recovered and
805 washed to remove the excess of percoll. The isolated chromosomes are stored in Tris-
806 HCl buffer (pH 7.5) containing polyamines in 60% glycerol.

807 Sample preparation and data acquisition

808 The DT40 chromosomes were vitrified in glow-discharged (operating at 30% power with
809 a gas mixture of 80% Argon:20% Oxygen) 200 copper mesh, Quantifoil Multi Holes grids
810 (Quantifoil Micro Tools GmbH, Germany).

811 The isolated mitotic chromosomes were exposed to low ionic strength buffer by 100-fold
812 dilution in an aqueous solution containing TEEN buffer (1 mM Triethanolamine:HCl pH
813 8.5, 0.2 mM Na-EDTA pH 9 and 10 mM NaCl) for 1 hour at 4 °C. This resulted in swelling
814 of the mitotic chromosomes which were then carefully spun down onto the surface of the
815 grid. 10 nm BSA Gold Tracer (EMS, Hatfield, PA, USA) was added to the grid at a 5:1
816 ratio (chromosome:gold). The grid was blotted from the carbon side using a teflon sheet
817 and the metal side using a blotting paper respectively. The grid was blotted for 25 seconds
818 with a blot force of 10 and flash-frozen into liquid ethane using Vitrobot Mark IV (FEI) at
819 4 °C and 100 % humidity.

820 The tilt series was recorded on Titan Krios G3 (Thermo Fisher Scientific) equipped with
821 a Quantum energy filter with a slit width 20eV for higher magnifications and Gatan K2
822 detector using SerialEM Version 4.1.0 beta 58. The images were acquired at a pixel size
823 of 2.075 Å/pixel at 2–4.0 µm defocus at a dose rate of 3.2 e-/Å² per image fractionated
824 over 10 frames. A dose symmetric tilt scheme 59 was used with a 3-degree increment
825 step and the tilt range was set to ±60 degree using Serial EM.

826 Data processing

827 Individual movies are motion-corrected and averaged to form a tilt series using
828 MotionCor2 60. Gctf was used to perform CTF estimation 61. The tilt series were aligned
829 and processed with IMOD47. The position of the grid in each image is aligned by tracing
830 10 nm gold particles as fiducials throughout the tilt series. The aligned tilt series were
831 reconstructed into 3D tomograms with weighted back-projection with simultaneous
832 iterative reconstruction technique (SIRT)-like filter after 4-fold binning.
833 After picking nucleosomes with our Template Learning trained DeepFinder model, we

834 performed a reference-free subtomogram averaging following the procedure suggested

835 in Relion V4.

836
Extended Data Fig.1: The list of distractors.

<figure><img src="image_10.png" /></figure>

Template Learning employs a list of 100 dissimilar protein assemblies, termed distractors, used for domain-randomized cryo-ET data simulations. In this figure, the
distractors are displayed with corresponding PDB IDs and molecular weights. Display of these structures was done using ChimeraX⁵².
<figure><img src="image_11.png" /><figcaption>**Extended Data Fig.2: Example of synthetic data based on the Template Learning workflow for supervised deep learning on ribosome annotation.**</figcaption></figure>

Left: the positions for a crowded sample of size 3072x3072x1024 Å³ determined using the Tetris algorithm for ribosomes (in green) and distractors (in red), which are fed to Parakeet (cryo-ET physics-based simulator). Right: a central slice of simulated data and corresponding ribosome segmentations using VPP close-to-focus parameters.
<figure><img src="image_12.png" /><figcaption>**Extended Data Fig.3: Segmentation maps resulting from different Template Learning variations for ribosome annotations.**</figcaption></figure>

Central slice of a VPP tomogram from EMPIAR-10988 with its expert-validated (ground truth) ribosome segmentation and the output segmentation maps using the different Template Learning variations presented in **Fig.4**.
<figure><img src="image_13.png" /><figcaption>**Extended Data Fig.4: Template Learning can use volumes as templates via volume-to-pseudoatoms conversion.**</figcaption></figure>

Volume to pseudoatomic model conversion algorithm to produce structures that can be contrast-tuned for usage in physics-based cryo-ET simulators (Parakeet). The method starts by binarizing the volume, a process akin to creating a tight mask through low-pass filtering and thresholding. All non-zero voxels of the binarized volume are parsed into "pseudoatoms", represented in PDB file format utilizing "DENS" entries as atom types. The generated pseudoatomic structure can be used directly for cryo-ET simulations in Parakeet, but it does not necessarily produce a similar contrast to experimental data as when using fully atomic structures. Therefore, the method allows replacing gradually a portion of the pseudoatoms with phosphorus atoms to emulate more contrasted simulated projections. The optimal proportion is determined by the similarity in contrast, comparing the resulting projections to experimental data. The resultant pseudoatomic model is used for further Template Learning simulations.
<figure><img src="image_14.png" /><figcaption>**Extended Data Fig.5: Spectrum matching applied to enhance the contrast of an input tomogram obtained with defocused cryo-ET without VPP.**</figcaption></figure>

A target spectrum is extracted from a tomogram obtained with VPP, and is applied to enhance the contrast of another tomogram obtained with defocus cryo-ET without VPP, via spectrum matching as proposed in ²⁰. Input and target tomograms are sources from EMPIAR-10988.
<figure><img src="image_15.png" /><figcaption>Extended Data Fig.6: Two experiments of post-alignment classification for nucleosome particles annotated with Template Matching workflow.</figcaption></figure>

A Classification into 10 classes. B Classification into 50 classes. Both classification experiments were done in Relion V4 ¹⁵.
<figure><img src="image_16.png" /><figcaption>**Extended Data Fig.7:** Analysis of the nucleosome subtomogram average obtained via Template Learning annotations.</figcaption></figure>

**A** The nucleosome average compared to an available structure (PDB ID: 2CV5). Left: the subtomogram average. Middle: atomic structure docked in the subtomogram average displayed at 50% transparency. Right: atomic structure. **B** Angular distribution of the particles contributing to the global average. **C** Analysis of the resolution of the average. Left: Fourier Shell Correlation curves. Right: local resolution analysis using ResMap⁵¹.
<figure><img src="image_17.png" /><figcaption>**Extended Data Fig.8: Template matching for nucleosome picking can only extract certain views at an adequate Precision.**</figcaption></figure>

In this work, we utilized template matching and Template Learning techniques to annotate nucleosomes in cryo-electron tomograms. The angular distribution obtained through averaging of template matching-based annotated particles was found
anisotropic. Leveraging simulations, we explain this phenomenon. **A** Constrained cross-correlation (CCC) of nucleosome template and simulated nucleosome subtomograms as a function of the angular distribution. Left: CCC histogram of a nucleosome template and ~800 simulated nucleosome subtomograms. The simulated subtomograms were obtained using ±60 degree tilt range with a 2-degree step, 2 microns defocus, and 120 e-/Å dose symmetric scheme using Parakeet. The template was used with the ground truth angular assignment before calculating the constrained cross-correlation (CCC). Right: angular distribution colored based on the CCC values showing that side views have higher CCC compared to top views. The nucleosome atomic structure (PDB 2PYO) was oriented similarly to the volume template shown in the middle. **B** CCC histogram as a result of template matching using PyTom on an experimental tomogram. **C** Illustration based on combining the results of **A** and **B** showing that thresholding based on the CCC value can extract only side views at an adequate precision while lowering the threshold increases the number of extracted top views but at the expense of extracting many false positives. **D** Angular distribution of template matching picked particles at different thresholds in **A** showing that the particles with the highest CCC scores correspond to nucleosome side views.
848 **Extended Data Tab.1: Cryo-ET simulation parameters in Template Learning.**

849 The parameters used for domain randomization simulations in this article were based on
850 a combination of the parameters listed in this table.

<table><thead><tr><td>Parameter</td><td>Values</td></tr></thead><tbody><tr><td>Defocus for close-to-focus VPP [microns]</td><td>0, -0.5, -1</td></tr><tr><td>Defocus without VPP [microns]</td><td>-2.5, -3.25, -4</td></tr><tr><td>Electron dose [e-/Å²]</td><td>75, 150</td></tr><tr><td>Tilting range [deg]</td><td>[-40, 40], [-60, 60]</td></tr><tr><td>Tilting step [deg]</td><td>2, 4</td></tr><tr><td>Ice density [g/cm³]</td><td>0.9, 1.1</td></tr></tbody></table>

851

852 **Extended Data Tab.2: Parameters used for training DeepFinder on Template**
853 **Learning simulations.**

854 Default hyperparameters used to train DeepFinder for the different particle picking
855 experiments.

<table><thead><tr><td>Parameter</td><td>Values</td></tr></thead><tbody><tr><td>Number of epochs</td><td>100</td></tr><tr><td>Number of steps</td><td>100</td></tr><tr><td>Shifting for data augmentation [vox]</td><td>13</td></tr><tr><td>Patch size [vox³]</td><td>48</td></tr><tr><td>Batch size</td><td>25</td></tr></tbody></table>

856
857 **References**

858 1. Schaffer, M. et al. Cryo-focused Ion Beam Sample Preparation for Imaging Vitreous Cells by Cryo-electron Tomography. *Bio Protoc* **5**, (2015).

860 2. Wagner, F. R. et al. Preparing samples from whole cells using focused-ion-beam milling for cryo-electron tomography. *Nat. Protoc.* **15**, 2041–2070 (2020).

862 3. Tacke, S. et al. A streamlined workflow for automated cryo focused ion beam milling. *J. Struct. Biol.* **213**, 107743 (2021).

864 4. Obr, M. et al. Exploring high-resolution cryo-ET and subtomogram averaging capabilities of contemporary DEDs. *J. Struct. Biol.* **214**, 107852 (2022).

866 5. Danev, R. & Baumeister, W. Expanding the boundaries of cryo-EM with phase plates. *Curr. Opin. Struct. Biol.* **46**, 87–94 (2017).

868 6. Okamoto, K. et al. Structural variability and complexity of the giant Pithovirus sibericum particle revealed by high-voltage electron cryo-tomography and energy-filtered electron cryo-microscopy. *Sci. Rep.* **7**, 13291 (2017).

871 7. McIntosh, R., Nicastro, D. & Mastronarde, D. New views of cells in 3D: an introduction to electron tomography. *Trends Cell Biol.* **15**, 43–51 (2005).

873 8. Harastani, M., Eltsov, M., Leforestier, A. & Jonic, S. TomoFlow: Analysis of Continuous Conformational Variability of Macromolecules in Cryogenic Subtomograms based on 3D Dense Optical Flow. *J. Mol. Biol.* **434**, 167381 (2022).

876 9. Zheng, S. et al. AreTomo: An integrated software package for automated marker-free, motion-corrected cryo-electron tomographic alignment and reconstruction. *J Struct Biol X* **6**, 100068 (2022).

879 10. Radermacher, M. Weighted Back-projection Methods. in *Electron Tomography*:
880 *Methods for Three-Dimensional Visualization of Structures in the Cell* (ed. Frank, 881 J.) 245–273 (Springer New York, New York, NY, 2006).

882 11. Boisset, N. et al. Overabundant single-particle electron microscope views induce a 883 three-dimensional reconstruction artifact. *Ultramicroscopy* **74**, 201–207 (1998).

884 12. Zhai, X. et al. LoTToR: An Algorithm for Missing-Wedge Correction of the Low-Tilt 885 Tomographic 3D Reconstruction of a Single-Molecule Structure. *Sci. Rep.* **10**, 886 10489 (2020).

887 13. Moebel, E. & Kervrann, C. A Monte Carlo framework for missing wedge restoration 888 and noise removal in cryo-electron tomography. *J Struct Biol* **X 4**, 100013 (2020).

889 14. Guesdon, A., Blestel, S., Kervrann, C. & Chrétien, D. Single versus dual-axis cryo- 890 electron tomography of microtubules assembled in vitro: limits and perspectives. *J. 891 Struct. Biol.* **181**, 169–178 (2013).

892 15. Bharat, T. A. M. & Scheres, S. H. W. Resolving macromolecular structures from 893 electron cryo-tomography data using subtomogram averaging in RELION. *Nat. 894 Protoc.* **11**, 2054–2065 (2016).

895 16. Wan, W. & Briggs, J. A. G. Cryo-Electron Tomography and Subtomogram 896 Averaging. *Methods Enzymol.* **579**, 329–367 (2016).

897 17. Best, C., Nickell, S. & Baumeister, W. Localization of protein complexes by pattern 898 recognition. *Methods Cell Biol.* **79**, 615–638 (2007).

899 18. Hrabe, T. et al. PyTom: a python-based toolbox for localization of macromolecules 900 in cryo-electron tomograms and subtomogram analysis. *J. Struct. Biol.* **178**, 177– 901 188 (2012).

902 19. Moebel, E. et al. Deep learning improves macromolecule identification in 3D
903 cellular cryo-electron tomograms. Nat. Methods **18**, 1386–1394 (2021).

904 20. de Teresa-Trueba, I. et al. Convolutional networks for supervised mining of
905 molecular patterns within cellular context. Nat. Methods **20**, 284–294 (2023).

906 21. Hao, Y. et al. VP-Detector: A 3D multi-scale dense convolutional neural network for
907 macromolecule localization and classification in cryo-electron tomograms. Comput.
908 Methods Programs Biomed. **221**, 106871 (2022).

909 22. Genthe, E. et al. PickYOLO: Fast deep learning particle detector for annotation of
910 cryo electron tomograms. J. Struct. Biol. **215**, 107990 (2023).

911 23. Zeng, X. et al. High-throughput cryo-ET structural pattern mining by unsupervised
912 deep iterative subtomogram clustering. Proc. Natl. Acad. Sci. U. S. A. **120**,
913 e2213149120 (2023).

914 24. Martinez-Sanchez, A. et al. Template-free detection and classification of
915 membrane-bound complexes in cryo-electron tomograms. Nat. Methods **17**, 209–
916 216 (2020).

917 25. Gubins, I. et al. SHREC 2020: Classification in cryo-electron tomograms. Comput.
918 Graph. **91**, 279–289 (2020).

919 26. Berman, H. M. et al. The Protein Data Bank. Nucleic Acids Res. **28**, 235–242
920 (2000).

921 27. Martinez-Sanchez, A., Jasnin, M., Phelippeau, H. & Lamm, L. Simulating the
922 cellular context in synthetic datasets for cryo-electron tomography. bioRxiv
923 2023.05.26.542411 (2023) doi:10.1101/2023.05.26.542411.

924 28. Parkhurst, J. M. et al. Parakeet: a digital twin software pipeline to assess the impact
925 of experimental parameters on tomographic reconstructions for cryo-electron
926 tomography. *Open Biol.* **11**, 210160 (2021).

927 29. Rodríguez de Francisco, B., Bezault, A., Xu, X.-P., Hanein, D. & Volkmann, N.

928 MEPSi: A tool for simulating tomograms of membrane-embedded proteins. *J. Struct. Biol.* **214**, 107921 (2022).

929

930 30. Himes, B. & Grigorieff, N. Cryo-TEM simulations of amorphous radiation-sensitive

931 samples using multislice wave propagation. *IUCrJ* **8**, 943–953 (2021).

932 31. Rice, G. et al. TomoTwin: generalized 3D localization of macromolecules in cryo-

933 electron tomograms with structural data mining. *Nat. Methods* **20**, 871–880 (2023).

934 32. Wagner, T. et al. SPHIRE-crYOLO is a fast and accurate fully automated particle

935 picker for cryo-EM. *Commun Biol* **2**, 218 (2019).

936 33. Moebel, E. & Kervrann, C. Towards unsupervised classification of macromolecular

937 complexes in cryo electron tomography: Challenges and opportunities. *Comput.*

938 *Methods Programs Biomed.* **225**, 107017 (2022).

939 34. de Melo, C. M. et al. Next-generation deep learning based on simulators and

940 synthetic data. *Trends Cogn. Sci.* **26**, 174–187 (2022).

941 35. Prakash, A. et al. Structured Domain Randomization: Bridging the Reality Gap by

942 Context-Aware Synthetic Data. in *2019 International Conference on Robotics and

943 Automation (ICRA) 7249–7255 (2019).

944 36. Tobin, J. et al. Domain randomization for transferring deep neural networks from

945 simulation to the real world. in *2017 IEEE/RSJ International Conference on*

946 *Intelligent Robots and Systems (IROS) 23–30 (2017).*

947 37. Bandyopadhyay, H. et al. Cryo-shift: reducing domain shift in cryo-electron

948 subtomograms with unsupervised domain adaptation and randomization.
949 *Bioinformatics* **38**, 977–984 (2022).

950 38. Bahar, I., Lezon, T. R., Bakan, A. & Shrivastava, I. H. Normal mode analysis of
951 biomolecular structures: functional mechanisms of membrane proteins. *Chem. Rev.*
952 **110**, 1463–1497 (2010).

953 39. Valtchev, S. Z. & Wu, J. Domain randomization for neural network classification. *J
954 Big Data* **8**, 94 (2021).

955 40. Purnell, C. et al. Rapid Synthesis of Cryo-ET Data for Training Deep Learning
956 Models. bioRxiv (2023) doi:10.1101/2023.04.28.538636.

957 41. Pei, L., Xu, M., Frazier, Z. & Alber, F. Simulating cryo electron tomograms of
958 crowded cell cytoplasm for assessment of automated particle picking. *BMC
959 Bioinformatics* **17**, 405 (2016).

960 42. Liu, S. et al. A unified framework for packing deformable and non-deformable
961 subcellular structures in crowded cryo-electron tomogram simulation. *BMC
962 Bioinformatics* **21**, 399 (2020).

963 43. Cui, Q., Rong, V., Chen, D. & Matusik, W. Dense, Interlocking-Free and Scalable
964 Spectral Packing of Generic 3D Objects. *ACM Trans. Graph.* **42**, 1–14 (2023).

965 44. Lobato, I. & Van Dyck, D. MULTEM: A new multislice program to perform accurate
966 and fast electron diffraction and imaging simulations using Graphics Processing
967 Units with CUDA. *Ultramicroscopy* **156**, 9–17 (2015).

968 45. Glaeser, R. M. Invited review article: Methods for imaging weak-phase objects in
969 electron microscopy. *Rev. Sci. Instrum.* **84**, 111101 (2013).

970 46. Tang, G. et al. EMAN2: an extensible image processing suite for electron
971 microscopy. *J. Struct. Biol.* **157**, 38–46 (2007).
972 47. Kremer, J. R., Mastronarde, D. N. & McIntosh, J. R. Computer visualization of three-dimensional image data using IMOD. *J. Struct. Biol.* **116**, 71–76 (1996).

974 48. Che, C., Xian, Z., Zeng, X., Gao, X. & Xu, M. Domain randomization for macromolecule structure classification and segmentation in electron cryotomograms. in *2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)* (IEEE, 2019). doi:10.1109/bibm47256.2019.8983110.

978 49. Jonić, S. & Sorzano, C. Ó. S. Coarse-Graining of Volumes for Modeling of Structure and Dynamics in Electron Microscopy: Algorithm to Automatically Control Accuracy of Approximation. *IEEE J. Sel. Top. Signal Process.* **10**, 161–173 (2016).

981 50. Chen, M. & Ludtke, S. J. Deep learning-based mixed-dimensional Gaussian mixture model for characterizing variability in cryo-EM. *Nat. Methods* **18**, 930–936 (2021).

984 51. Kucukelbir, A., Sigworth, F. J. & Tagare, H. D. Quantifying the local resolution of cryo-EM density maps. *Nat. Methods* **11**, 63–65 (2014).

986 52. Goddard, T. D. et al. UCSF ChimeraX: Meeting modern challenges in visualization and analysis. *Protein Sci.* **27**, 14–25 (2018).

988 53. Doruker, P., Jernigan, R. L. & Bahar, I. Dynamics of large proteins through hierarchical levels of coarse-grained structures. *J. Comput. Chem.* **23**, 119–127 (2002).

991 54. Ma, J. Usefulness and limitations of normal mode analysis in modeling dynamics of biomolecular complexes. *Structure* **13**, 373–380 (2005).

993 55. Harastani, M., Eltsov, M., Leforestier, A. & Jonic, S. HEMNMA-3D: Cryo Electron Tomography Method Based on Normal Mode Analysis to Study Continuous
995 Conformational Variability of Macromolecular Complexes. *Front Mol Biosci* 8,

996 663121 (2021).

997 56. Bakan, A., Meireles, L. M. & Bahar, I. ProDy: protein dynamics inferred from theory

998 and experiments. *Bioinformatics* 27, 1575–1577 (2011).

999 57. Lewis, C. D. & Laemmli, U. K. Higher order metaphase chromosome structure:

1000 evidence for metalloprotein interactions. *Cell* 29, 171–181 (1982).

1001 58. Mastronarde, D. N. Automated electron microscope tomography using robust

1002 prediction of specimen movements. *J. Struct. Biol.* 152, 36–51 (2005).

1003 59. Hagen, W. J. H., Wan, W. & Briggs, J. A. G. Implementation of a cryo-electron

1004 tomography tilt-scheme optimized for high resolution subtomogram averaging. *J.*

1005 *Struct. Biol.* **197**, 191–198 (2017).

1006 60. Zheng, S. Q. *et al*. MotionCor2: anisotropic correction of beam-induced motion for

1007 improved cryo-electron microscopy. *Nat. Methods* 14, 331–332 (2017).

1008 61. Zhang, K. Gctf: Real-time CTF determination and correction. *J. Struct. Biol.* **193**, 1–

1009 12 (2016).

1010