Smart hybrid microscopy for cell-friendly detection of rare events

Domain: physics
Published: 2026-01-07
Editorial outcome: major_revision

Abstract
Fluorescence microscopy offers unparalleled access to the spatial organization and dynamics of biological events in living samples, yet capturing rare processes over extended durations remains challenging due to trade-offs between exposure to excitation light and sample health. Here, we introduce hybrid-EDA, an event-driven acquisition (EDA) framework that combines the gentleness and contextual richness of phase-contrast with the functional specificity of fluorescence. We develop surveillance for events of interest in label-free microscopy using dynamics-informed neural networks that trigger smart fluorescence acquisition upon detection. This allows us to dramatically reduce phototoxic damage while obtaining specific and functional information from fluorescence when beneficial. We demonstrate how hybrid-EDA enables improved imaging acquisitions of organelle contacts and mitochondrial divisions. We envision that hybrid-EDA will enable insights into a range of dynamic and rare biological processes, providing a powerful and general strategy for cell-friendly imaging.

Introduction
Light microscopy is an invaluable tool for investigating the dynamics of biological systems, due to its non-invasive nature. Fluorescence-based contrast offers specificity by targeting dyes to molecules or structures of interest; on the other hand, label-free microscopy exploits the natural variations in the physical properties of cellular compartments and offers broader context1. Yet, these classical approaches have their limitations – fluorescence requires a high excitation irradiance and generates cytotoxic free radicals2,3, while phase-contrast offers reduced phototoxicity but is insensitive to molecular details. Combining fluorescence and phase-contrast in correlative imaging provides both specificity and context, but maintains the phototoxicity of fluorescence microscopy.

Machine learning-based methods such as denoising and virtual staining have provided an alternative way of recovering fluorescence information with reduced phototoxicity. Denoising methods can learn a model of the noise to restore images4–7, allowing data to be collected at a lower signal-to-noise ratio, corresponding to a lower light dose. Virtual staining uses paired fluorescence and label-free images to train a model, which can then predict fluorescence patterns based on intrinsic contrast alone8–10. Thus, fluorescence-like images are generated at the low light dose of label-free imaging methods such as brightfield, differential interference contrast, or phase-contrast microscopy. However, this approach comes with major constraints: virtual staining can only reliably predict images when the label-free data contains signatures of features which should appear fluorescent. This is not generally the case for proteins or for functional readouts that are possible with fluorescent biosensors (e.g., Ca2+, membrane potential).

Another approach, complementary to denoising, is smart microscopy. Previously, we developed event-driven acquisition (EDA)11 to reduce light dose by driving our microscope with a closed-loop controller incorporating a deep learning-based detection module trained to predict specific, rare biological events. We demonstrated EDA with a detector developed to predict mitochondrial or bacterial division; this strategy allowed the microscope to toggle automatically from slow to fast imaging to capture in greater temporal detail the structural intermediates which accompanies such divisions. Additional examples of real-time adaptive light microscopy measurements have been used to lower the light dose of widefield fluorescence12, localization microscopy (MINFLUX)13, stimulated emission depletion (STED)14 or lattice light sheet microscopy15. However, the complementary combination of label-free and fluorescence modalities into a smart microscope remains unexplored, probably due to difficulties in distinguishing different subcellular features in label-free images, which makes event detection challenging.

Here, we report a hybrid method for detecting specific dynamic events in phase-contrast microscopy, and integrate it into a microscope to demonstrate real-time switching to fluorescence to capture the molecular and physiological signatures of mitochondrial contact sites and mitochondrial divisions. A key to reliable event detection in phase-contrast is including temporal information and memory in a specialized network architecture. Integrating this into the EDA paradigm allowed us to reduce the impact on sample health during surveillance by more than a 100-fold reduction in excess mortality, resulting in ten times longer experiments or, equivalently, capturing ten times more rare events. As an additional benefit, we could better exploit fluorescent sensors for live imaging despite their poor photostability.

Results
A remaining limitation of EDA-like methods is their use of fluorescence to detect events; a significant proportion of the photon budget is spent during surveillance for event detection, leading to fluorophore bleaching, phototoxicity, and their impacts on cellular physiology. Our method transcends fluorescence-only EDA by combining the strengths of two complementary imaging techniques in hybrid-EDA: phase-contrast for gentle surveillance of the sample until an event of interest is detected, and consequent correlative phase and fluorescence imaging. Neural networks perform the challenging task of detecting events in content-rich phase-contrast images to trigger the modality switch on-the-fly. Effectively, we consider the microscope acquisition as a two-state machine (Fig. 1a), with one state corresponding to gentle surveillance in phase-contrast and the other state corresponding to smart multi-channel acquisition. During each state, the microscope acquires phase-contrast images, through which the biological system is continuously monitored by an event-detection neural network whose output determines when state transitions should be actuated. Details on the response latency between event detection and state transition are provided in the Supplementary Note 2.

Transmitted light microscopy is commonly accepted to be gentler than fluorescence microscopy3, but we sought to quantify their differential impacts on cell survival during mitochondrial imaging. Mitochondria are exceptionally sensitive to phototoxicity16, and thus serve as an important reference target for imaging. We acquired widefield fluorescence or phase-contrast images of mitochondria in COS-7 cells using typical acquisition parameters (Methods, Supplementary Note 1). To assess phototoxicity, we also measured nuclear envelope integrity, a hallmark of cell death, and observed that it was compromised in 50% of cells after only 5 min of fluorescence imaging at 1 frame-per-second (fps, Fig. 1b). By comparison, in phase-contrast acquisitions with comparable contrast, nuclear envelope integrity remains indistinguishable from ‘no illumination’ conditions, even after 1 h of imaging at 1 fps. Exponential decay fits of these curves reflect an ~50-fold decrease in half-lifetimes and greater than 100-fold excess mortality of the cells in fluorescence imaging compared to phase-contrast (Supplementary Note 1, Supplementary Fig. 1). These data highlight that hybrid-EDA will be most beneficial for studying rare, dynamic events, where most time is spent in surveillance mode and fluorescence is only enabled rarely for short times.

Phase-contrast imaging naturally highlights cellular organelles due to differences in their molecular content relative to the cytoplasm, which gives rise to differences in refractive index. Organelles such as mitochondria, lysosomes, lipid droplets, and the endoplasmic reticulum can form heterotypic membrane contact sites, which play important functional roles in signaling, lipid metabolism, and organelle fission, among others17. However, many contacts are transient, lasting only seconds, and their frequency and function may be highly sensitive to cell health and state. Imaging at low phototoxicity is therefore paramount to obtain biologically relevant information. In phase-contrast movies, we observed dynamic contact events between different organelles of distinct tubular versus spherical morphologies. Based on fluorescence imaging, we identified the tubular organelles as mitochondria, and we hypothesized that the spherical organelles could be lysosomes or lipid droplets18,19, both known to form functional contact sites with mitochondria. We therefore aimed to develop an event detection framework for such apparent contacts.

The main challenge in implementing hybrid-EDA is reliably predicting or detecting biological events, which is more challenging in label-free images than in fluorescence imaging. The visibility of diverse cellular structures in phase-contrast data effectively adds segmentation and classification tasks to event detection, whereas those tasks are circumvented by the specificity of the signal in fluorescence data. To create a training dataset, we manually labeled contact events between mitochondria and spherical organelles in phase-contrast time-lapse images of COS-7 cells (Fig. 2a, Supplementary Note 3, Supplementary Fig. 2). For event detection, we trained neural networks with a U-Net architecture with the phase-contrast frames and a ground truth containing a local label consisting of a Gaussian located at each contact site. Alternative architectures such as detection-based models (YOLO20, Spotiflow21) or transformer-based approaches (LW-DETR22) could be suitable for the event detection task. However, our choice of a U-Net as our backbone was motivated by several reasons: (1) computational efficiency for real-time feedback to the acquisition, (2) ability to provide precise spatial localization through skip connections, (3) efficiency with limited training data, and (4) ease of incorporating temporal information. The U-Net architecture has additionally been shown to be on par with more recent architectures in practical settings such as cell segmentation23. We adopted the F0.1 score as our primary performance metric because it prioritizes precision (P, true positives over all predicted positives) over recall (R, ratio of actual positives recovered). In other words, it places an emphasis on reducing false positives. This is beneficial for phase-EDA, because the surveillance stage in phase-contrast is exceptionally cell-friendly, while smart acquisition during a false positive exposes the sample to fluorescence imaging without benefit (1).

We first selected models with the best F0.1 scores and used them to classify the events in the validation dataset. However, we observed a considerable spread in performance for models trained with identical parameters. To address this, we switched from cross-entropy to a soft focal loss function for training, a strategy known to be beneficial for the detection of rare positive events24,25. This stabilized the training at high performance scores. Yet, even these models yielded a high rate of false negatives—instances where genuine organelle contacts were missed (Fig. 2b). Therefore, we further improved our training approach by supplying a time series of images to the models in the form of additional image channels, enabling the networks to use dynamic information in the detection of events (Fig. 2c). With soft focal loss, this increased the recall of the best performing models by almost a factor of 2, allowing more events to be captured during adaptive acquisitions (Fig. 2d, Supplementary Fig. 3). These improved models enabled efficient detection of apparent contact events between spherical organelles and mitochondria.

We implemented a custom adaptive imaging framework based on pymmcore-plus26 to control our microscope, which allowed us to incorporate our contact detection neural network to adapt acquisition parameters on-the-fly (Methods, Supplementary Note 4, Supplementary Fig. 4). We imaged COS-7 cells in phase-contrast with live surveillance. To mark lipid droplets, cells were transfected with perilipin 5 (PLIN5) fused to mEmerald fluorescent protein. Once a contact event was detected, the framework automatically switched the microscope to correlative phase and fluorescence imaging. This allowed us to identify the contacting spherical organelle as a PLIN5-positive lipid droplet (Fig. 2e, Supplementary Fig. 5a, b, Supplementary Movies 1–3). While many detected contact partners were indeed lipid droplets, we also observed contacts with spherical organelles that were not PLIN5-positive. We therefore performed hybrid-EDA on cells stained with LysoTracker Red to mark lysosomes and LipidTOX to mark lipid droplets (Fig. 2f, Supplementary Fig. 5c, Supplementary Movies 4, 5). This enabled us to capture and distinguish mitochondria-lipid droplet and mitochondria-lysosome contact sites simultaneously, an otherwise unreliable task in phase-contrast, requiring three-color fluorescence imaging. By using the phase-contrast data to discern the mitochondrial morphology, we eliminate the need for an additional fluorescence channel and marker, enhancing the value per phototoxic effect for every channel. During labeling we identified one contact event on average every 3.1 min, with a duration of 20 s; thus, when imaging at 1 Hz hybrid-EDA captured 20 fluorescence frames per event instead of about 186 which would have been captured during continuous fluorescence. This represents a ~nine-fold reduction in phototoxicity per event, or a nine times longer accessible experiment duration. Fluorescent dyes and proteins have developed to be increasingly efficient and photostable; improvements in functional reporters are less forthcoming, perhaps because their specificity in sensing constrains optimizing other attributes beneficial for imaging. TMRE (Tetramethylrhodamine ethylester) is widely used to measure mitochondrial membrane potential. However, the membrane potential itself is very sensitive to the cell’s stress state27—meaning that even the moderate light exposure required for fluorescence imaging can induce changes. This sensitivity makes it challenging to reliably capture rapid fluctuations in membrane potential which are critical for mitochondrial quality-control, especially those which accompany mitochondrial dynamics. Inspired by recent reports showing differential bioenergetics in mitochondria interacting with lipid droplets28,29, we sought to explore the rapid dynamics of membrane potential during transient interactions between mitochondria and lipid droplets. We detected short-lived (1–3 s) membrane potential fluctuations and flickering during our acquisitions, but did not find differential behavior between the mitochondria in contact with the lipid droplet and others in the vicinity (Supplementary Figs. 5d, 6, Supplementary Movie 6)—in contrast with reported data28. Combining the low phototoxicity of phase-contrast with the specificity of fluorescence markers and sensors in this way allows us to study organelle contacts under conditions closer to unperturbed cellular physiology, leading to more biologically relevant readouts, and with additional targets per acquisition.

Mitochondrial fission mediates organelle proliferation and quality control, and is thus a crucial process in maintaining cellular homeostasis30. Fission precursors in the form of transient constrictions are subtle, and may reverse without culminating in complete division31. To tackle the challenge of detecting pre-fission constrictions by phase-contrast microscopy, we needed to employ a more complex neural network training strategy (Fig. 3a). We first adapted the training data from 11 that contained ad hoc labels for candidate mitochondrial fission precursors based on two-channel images of mitochondria and the fission factor dynamin-related protein 1 (DRP1). We refined the labels using the dynamic information inherent in the time-lapsed data. Specifically, we quantified frame-to-frame fluctuations to assess label accuracy and retained only those labels that consistently showed an event probability over 36%. With these filtered labels, we trained a U-Net to detect constrictions in single-channel images of the mitochondria. We then applied this network to the fluorescence channel of correlative time-lapses—combining mitochondrial fluorescence and phase-contrast—to identify potential division sites. Finally, we manually confirmed actual divisions and trained an additional U-Net with the verified labels and corresponding phase-contrast images (Fig. 3b).

Even with this multi-step training strategy, and when using multiple time points and soft focal loss as we had applied to detect organelle contacts, classifications of the validation data comprised significant false positives. Notably, long-lived false positives, mostly corresponding to cell membrane features such as folds or ruffles (Fig. 3c), posed a major problem by triggering hybrid-EDA over extended periods. This rendered the models unfit for informing our adaptive acquisitions.

Recognizing that the structural signatures of mitochondrial divisions are even more subtle than those of organelle contacts, we refined our approach to make more use of their dynamic signatures by developing a stateful U-Net. This architecture incorporates a state-capturing long short-term memory (LSTM) layer at the bottleneck between the encoder and decoder, equipping the model with temporal memory to track subtle, time-dependent features across image sequences (Fig. 3d)32. Using this architecture we achieved consistent improvement of model performance. As with organelle contacts, F0.1 scores and precision on the validation data improved with more input time points (Fig. 3e, Supplementary Fig. 7). Recall values remain relatively low, but we observe no specific characteristics in the missed events (e.g., peripheral vs central divisions), implying that the sampled true positive divisions are unbiased in this regard (Supplementary Fig. 8, Supplementary Movies 7–10).

DRP1 assembles at pre-constrictions, and actively further constricts mitochondria to drive membrane scission. Its low abundance and dynamicity at individual division sites makes DRP1 a challenging target for fluorescence imaging. Within seconds after division is complete, mitochondria typically recoil away from the fission site, while DRP1 disassembles, leaving as an open question whether it shares severing mechanisms with other dynamin family members33–35. We imaged COS-7 cells expressing DRP1-mEmerald in phase-contrast and detected pre-constrictions using the aforementioned networks. Switching on fluorescence allowed us to study the dynamics of DRP1 during and immediately after divisions (Fig. 3f, Supplementary Movie 11). We found that DRP1 signal is present at the tips of both daughter mitochondria for most of the divisions (Fig. 3g), distinct from dynamin-mediated fission of endocytic vesicles which occurs at the edge of the dynamin coat34. With an estimated division rate of 1 per 3.2 min for a typical cell, when imaging at 1 Hz we could image ten times longer in hybrid-EDA by imaging 20 fluorescence frames per event instead of about 192 if we were imaging continuously in fluorescence. We applied our smart imaging approach to measure membrane potential in the vicinity of constriction sites and at divisions, and captured events of temporal membrane potential loss—or ‘flickering’—associated with these sites (Fig. 3h, Supplementary Fig. 9, Supplementary Movies 12–16). While whole-mitochondrion flickering is readily observed with traditional microscopy36, this rapidly reversible subdomain loss of potential would be undetectable at the lower imaging rates typically used in these experiments to avoid reporter photobleaching. Among a total of twenty division events, we captured 4 with significant fluctuations in membrane potential, only one of which was observed to reverse constriction without going to full separation of the organelles.

Methods
Sample preparation
African green monkey kidney cells (COS-7, RRID: CVCL_0224; ECACC: 87021302) were cultured in Dulbecco’s modified Eagle medium (DMEM; Thermo Fisher Scientific, 31966021) supplemented with 10% fetal calf serum (FCS) in glass-bottomed cell culture dishes (ibidi, 81158). COS-7 cells were obtained commercially and not authenticated in house. mEmerald-PLIN5 was a gift from S. Cohen. DRP1-mEmerald was cloned from DRP1-mCherry, a gift from G. Voeltz. For lipid droplet imaging, mEmerald-PLIN547 was transfected using Lipofectamine 2000 (Thermo Fisher Scientific TFS, 11668030) and Opti-MEM (TFS, 31985062) following manufacturer’s instructions (150 ng of plasmid per dish), and allowed to incubate for 12–24 h prior to imaging. For dual-color imaging of lipid droplets and lysosomes, cells were incubated with HCS LipidTOX™ Deep Red Neutral Lipid Stain (1:1000, Invitrogen, H34477) and LysoTracker™ Green DND-26 (50 nM, Invitrogen, 1020097) for 30 min before imaging. For DRP1 imaging, cells produced mEmerald-DRP1 or DRP1-mCherry after transfection as described above, and the latter were co-stained with a 1:10000 dilution of SYBR™ Gold Nucleic Acid Stain (Thermo Fisher Scientific, S11494) for 5 min, followed by two washes with transparent media. TMRE imaging was performed after 10 min incubation with 500 nM TMRE and three washes. In all cases, FluoroBrite DMEM (TFS, 1896701) was used for washes and during imaging. For toxicity measurements, cells were incubated with SYTOX-orange (100 nM, TFS, S11368), Hoechst 33342 (0.2 μg/ml, TFS, 62249) and MitoTracker Green FM (500 nM, TFS, M46750) for 10 min and washed three times before imaging.

Microscopy
All imaging was performed on a Zeiss AxioObserver 7 with a Plan Apochromat 63x/1.40 Oil Ph3 objective for both phase-contrast and fluorescence imaging. A CoolLed pE-800 module was used for fluorescence illumination. Images were captured on a Photometrics Prime camera and cells were kept at 37 °C and 5% CO2. Adaptive acquisitions were performed using a custom software built on the pymmcore-plus environment described below. For toxicity measurements, we collected two data points for each imaging modality (fluorescence, phase, dark) in the same sample at different positions one after the other. This reduces effects from sample-to-sample variations in the survival data. Toxicity data was collected in three different samples, resulting in six data points for each laser power. A 553 high-pass filter was added to the brightfield illumination path for phase-contrast imaging. All four channels (SYTOX, Hoechst, MitoTracker, and phase-contrast) were recorded every 250 s. In between, the channel corresponding to the mentioned condition (MitoTracker, phase-contrast) was imaged at 1 fps with 100 ms exposure time. Phase-contrast imaging was performed at maximal power of the transmission illumination, while MitoTracker imaging was performed at 2% and 10% laser power (8 and 35 mW/mm2, respectively).

Neural network training
Organelle contact detection was trained on 111 events (58 positive, 53 negative), and evaluation data consisted of 29 events (18 positive, 11 negative). 596 events (319 positive, 277 negative) were used to train the detection of mitochondrial divisions with 157 events (86 positive, 71 negative) in the validation set. Training and evaluation data were split on an event basis upon registration in the event database, thus ensuring consistency of the training runs and performance scoring.

The last five frames of positive events were labeled with a Gaussian spot at the site of the event (division/contact). Organelle contact events were manually annotated based on a visible interaction of the two organelles by deformation of the mitochondrion. Additionally, both spatial proximity and temporal persistence (no visible separation for at least three frames) were considered for event selection. The divisions were followed over time and only the divisions with clear separation of the daughter mitochondria were labeled as fission events. The division site was tracked back in time and labeled as long as the future division site was distinguishable as a constriction or obvious by the mitochondrial morphology. All events used for training were checked by at least two separate people to ensure the reliability of the data-set. In a way to optimize training results, we used early models (trained on fewer events) on phase-contrast data, corrected false positives and false negatives, and added these events to the training data for subsequent training.

All networks use these five consecutive time points as input. labeled networks were trained with single images and the corresponding ground truth frames, while networks for multiple time points got the corresponding number of images plus a single ground truth frame corresponding to the last image in the time series.

Standard U-nets are of depth 2 with 16 initial filters, stateful variants used the same structure with a recurrent ConvLSTM2D layer at the bottleneck to capture spatiotemporal dependencies across the image sequence. Training was performed using TensorFlow/Keras 2.10 on NVIDIA Tesla T4 GPUs.

We used the Fβ score, which is a function of precision P and recall R, with β = 0.1 as the main metric for performance analysis.

Fβ = (1+β2)P×R / (β2P+R) (1)

This metric prioritizes the precision over the recall, and we chose it to reduce triggering on false positives. The threshold for classification was therefore optimized for the best Fβ scores and recall and precision are reported at that threshold unless declared otherwise. Prioritizing precision over recall results in the omission of some true events (false negatives), while reducing phototoxicity that would result from fluorescence imaging of false positives. This is consistent with the goal of Hybrid-EDA, to reduce phototoxicity, and the use of phase-contrast which is a nearly non-toxic monitoring modality. Missed events in validation datasets were visually inspected to and found not to introduce any evident systematic bias, and were used to fine-tune the models to increase recall.

Microscope control
We built on the capabilities of the open-source microscope control software environment pymmcore-plus to allow for adaptive acquisitions. Hardware control, image capture, and storage are performed by components of the pymmcore-plus framework. Captured phase-contrast images are transferred to an analyzer component by the integrated event system. The analyzer stores a subset of time points, runs inference of the model on this data, and forwards the result to an interpreter that implements memory and thresholding. A threshold on the event score was implemented for switching to and from correlative imaging as well as a minimum number of correlative frames to be imaged after a trigger. If a modality switch is requested, a custom scheduler is notified to adapt the active acquisition settings. This scheduler supplies acquisition events for one or the other modality (phase-contrast/correlative) to the pymmcore-plus acquisition engine at the correct times for the set frame rate (Supplementary Note 4). The execution of these events results in new phase-contrast data that is transferred to the analyzer to close the loop.

Data analysis
SYTOX-orange entry into the nucleus was measured for detecting the onset of cell death. The Hoechst channel was used to segment nuclei and both SYTOX and Hoechst were normalized on a local region around two times the size of the nuclei. The ratio of the normalized intensities in the nucleus regions was used to determine the cell state. A ratio of 0.5 was used as a cutoff for the onset of cell death per cell.

The determination of image quality parameters (signal-to-noise ratio, intensity ratio, and contrast) was realized by marking several mitochondria per field of view and adjacent regions that didn’t show any features of high contrast in brightfield and appeared dark in the fluorescence channel. The signal to noise ratio was defined as the signal in the mitochondria region divided by the standard deviation of the signal in the background region.

The analysis of mitochondrial membrane potential fluctuations during lipid droplet contacts was performed on captured event crops using CellProfiler 4.2.848. Briefly, mitochondria were segmented from the TMRE channel using the IdentifyPrimaryObjects module (adaptive minimum cross-entropy, intensity-based separation), measuring their position and mean TMRE intensity, and tracking them (TrackObjects—Overlap method). Then, these mitochondria were classified between LD-contact positive and negative based on whether they were within 1.5 μm of the event detection center at the detection time point.

For the analysis of DRP1 abundance on mitochondria after division, regions of interest of 3×3 pixels were defined around the resulting tips. Integrated intensities were calculated after the subtraction of the camera offset value and translated into the corresponding ratios by dividing the higher intensity by the lower one.

Statistics and reproducibility
No statistical methods were used to predetermine sample size. Sample size was chosen to achieve statistical significance, according to our previous experience and other studies from the field. Studies were performed on cells originating from the same cell line batch and randomly assigned to experimental conditions, and repeated on independent experimental days to ensure replicability. Data was excluded only upon signs of technical failure (poor imaging or staining quality) or cellular/mitochondrial stress (fragmentation, immobile mitochondria, apoptosis). When possible, analysis was performed in an automated manner to avoid bias.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "The hybrid event-driven acquisition imaging system developed by Willi L. Stepp and coworkers represents a significant breakthrough in biological imaging technique. This method addresses a critical challenge of real-time online fluorescence imaging for rare biological processes: Long-term high excitation irradiance generates cytotoxic free radicals that significantly affect physiological processes. The team overcame this obstacle by employing an event-driven acquisition (EDA) framework that combines the gentleness and contextual richness of phase contrast with the functional specificity of fluorescence. By integrating deep learning algorithms to detect dynamic events and trigger fluorescence acquisition, the method significantly improves imaging efficiency and cell compatibility. It holds great potential for capturing rare biological events and shows promising applications in live-cell dynamic studies. I detail specific points that should be addressed:\nMajor Concerns:\n1. The criteria used to define organelle contact or division events remain unclear—whether these are defined based on spatial proximity, temporal persistence, or subjective annotation. It is recommended that the authors provide a detailed description of the annotation procedure, including how contact events are identified and verified. The consistency of manual labeling should also be evaluated to ensure dataset reliability.\n2. The manuscript does not report the dataset size, the number of positive and negative samples, or how the data was split into training, validation, and test sets. Given that the biological events of interest are rare, the potential for class imbalance is high, which can bias training. The authors should provide these statistics to allow assessment of model generalizability.\n3. Figure 2d reveals relatively low recall values, which may result in the omission of biologically relevant events. The authors should discuss the implications of these omissions on biological interpretations and indicate whether any strategies were used to reduce false negatives.\n4. While the main advantage of hybrid-EDA lies in reduced phototoxicity, this introduces a clear trade-off with detection sensitivity. It would be beneficial to quantitatively characterize this trade-off—for example, by plotting recall and the number of fluorescence frames under varying detection thresholds—so that users can adjust the system to suit specific experimental constraints.\n5. It is not stated whether the same objective is used for both phase contrast and fluorescence imaging. The authors should clearly state the model and specifications of the objective lens used.\n6. The manuscript does not report the response latency between event detection and activation of the fluorescence imaging channel. This parameter is crucial for capturing transient events and for assessing the overall temporal precision of the hybrid-EDA system. Including this information would enhance the transparency and reproducibility of the method.\n7. The hybrid-EDA system has advantages in real-time monitoring of slow physiological processes. For example, using this system to monitor the formation of lipid droplets.\n8. The functional interactions of organelles is via membrane contact sites (MCSs) defined as areas of close proximity (from 10 to 80 nm) between the membranes of two organelles. Studying organelle interactions through conventional fluorescence microscopy can produce many artifacts. This hybrid-EDA system can be combined with super-resolution technology or fluorescence lifetime imaging technology.\n9. I am not fully convinced that the image showed events of temporal membrane potential loss or ‘flickering’ (Fig 3h). TMRE monitors the changes in mitochondrial membrane potential (mainly decreases) through a concentration diffusion process. Due to the limitations of diffusion speed and concentration gradient, TMRE hardly monitors the rapid dynamic changes of membrane potential.\nMinor points\n1. “fluorescenct” should be corrected to “fluorescent”.", "reviewer_2": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_3": "The manuscript follows a recent trend in developing smart and sample-adaptive acquisition modalities, a direction microscopy will likely turn more and more toward. The method utilizes multiple microscopy methods to monitor the sample with a gentle one and only apply the to-an-extent more phototoxic method when events are detected minimizes the influence of the microscopy on the sample, helping the sample health over a longer period. The implementation uses artificial intelligence in an appropriate way, in aiding the detection of events which with classical-computer-vision-based methods would be difficult to capture with any reasonable precision and recall. It also utilizes temporal information rather than a single frame in the event detection process, leading to more reliable detection, a key which has also previously been shown for event-driven microscopy. The implementation of the temporal information in the event detection is simple yet elegant and slightly boosts the performance.\nMy main concern with the current manuscript is that all the applications presented do not showcase the method to the fullest, as I would argue that some of the experiments could be performed at least at the same level using manual or classically automatic control of the microscope (see below). However, the application on mitochondrial division is implemented in an impressive manner, clearly building on their previous work (Mahečić 2022) but now detecting the events with only phase contrast information. When it comes to findings in the application experiments presented, the DRP1 intensity ratio between the ends of divided mitochondria is a question which the method is nicely adapted to tackle, but the investigation remains rather shallow and with a small sample size. Otherwise the experiments are presented phenomenologically without any quantification, and as such there are no further significant biological findings despite a potential for that. The method implementation nevertheless promises tackling of more complex biological questions with molecule-specific labelling, such as with DRP1, in the future, and thus overall the method holds great significance for the field of smart microscopy and applications of it and is important in its own right.\nThe adaptive combination of multiple imaging techniques in smart microscopy is not novel in the field, however, as they rightfully mention, their implementation with a label-free monitoring technique is significant for further future developments of smart microscopy methods and holds great potential for applications of the method as such. One can envision future potential combinations with higher-resolution and super-resolution fluorescence imaging techniques such as SIM or STED. I see no major flaws with the validity or robustness of the presented experiments and data interpretation, but have a few minor comments and questions below when it comes to the presentation of and claims from the data. I believe they can be resolved by improvements to clarity and presentation. With regards to the overall clarity of the manuscript, the text is well-written, figures are clear, and previous work in the field has been referenced in a satisfactory manner to provide a clear context and overview to the potential reader.\nSee below a list of specific points of concern and suggestions to improve the manuscript.\n- Most supporting movies seem to be unreferenced in the text. Furthermore, there are no channel labels in the movies and no legends in the documents received. As such, it is difficult to understand to which sections the movies belong and which movies support which claim. Properly referencing all movies in the text, or in the supplementary material, would be useful.\n- Page 2: There are claims that the impact on sample health during the monitoring phase is reduced by “greater than a 100-fold” at the end of the introduction, while in the quantification presented in Fig. 1 and Supplementary Note 1 a 50-fold improvement is noted. If this is the quantification supporting the initial statement these values should agree.\n- Page 4: A small discussion regarding the choice of a U-Net architecture over other alternatives for the neural networks, with some potential references, I believe could be interesting for the potential reader.\n- Page 4: When discussing the improvements to the original network, it is mentioned that “the recall is improved by almost a factor of 2” when adding temporal information. When looking at fig 2d, for the original BCE model there is barely an improvement of the mean and the marked best F0,1 model is improved with a factor of 1.5. For the best model in the Soft Focal case the improvement is 2-fold, but for the mean it is neither there more than seemingly a factor of 1.5. For clarity, if the mentioned factor of 2 is quantified for the best network with a soft focal loss function this should be mentioned in the text, as it currently can be interpreted as the mean and especially as an improvement factor for the BCE case due to the order of the modifications.\n- Page 4: In the experiments regarding organelle contact sites, I am struggling to understand what benefit the fluorescence imaging during the event brings in addition to the phase contrast. As the resolution seems similar in the two methods in the data presented, all contact sites can clearly be visualized in the phase contrast data. This is further indicated by the fact that only phase contrast data is used to annotate the training dataset for this event detection network. In the cases where knowing which organelle type is touching mitochondria, a single fluorescence frame recorded before or after a phase contrast time lapse, for example every minute, would seem to give the same information; in the data presented, the cellular organelles are sufficiently sparse, the phase contrast imaging is sufficiently rapid, and the organelles move sufficiently slow that a single frame of fluorescence data seems enough to identify fluorescently labelled organelles and thus allow to follow them throughout the phase contrast time lapse. If so, the manual or “classically” automatic approach of recording a single fluorescence frame every x minutes would be beneficial from a phototoxic point of view, according to what the authors mention earlier, and simplify microscope control. Furthermore, the full data could be analyzed after the experiments, with more information available for a potential neural network, and the recall could probably be improved further in that case. Further evidence for and discussion regarding that the fluorescence data during organelle contact event sites provides additional information from which any conclusions about the contact sites could be drawn would enhance this application.\n- Page 4: It is mentioned that hybrid-EDA enabled imaging for “nine times longer”, since there was “one organelle contact site detected every 3.1 minutes on average”. I miss the calculation to support this claim, and cannot follow where the value “nine times longer” comes from. A clarification of this claim in the text would be helpful. The same goes for the 1/3.2 minutes and 10x longer imaging that is mentioned for mitochondrial division events.\n- Page 6: When it comes to the experiments on mitochondrial division event sites, the same could be argued about the experiments where MitoTracker is the fluorescent target (Figure S6). In this case, the fluorescence data does not seem to add much information, as the resolution between phase contrast and fluorescence seems similar, and the tubular organelles are already distinctly identified as mitochondria, according to the discussion of the author about the organelle contact site event detection. A further discussion on what the fluorescence data adds in this case would be helpful. Experiments where DRP1 or TMRE are used as fluorescent targets instead presents clear benefits regarding the information one can gather from the experiments as opposed to pure phase contrast or fluorescence imaging.\n- Page 6, first paragraph of Smart acquisitions of mitochondrial divisions and constrictions: Fig 3e is referenced, but it should be Fig 3f.\n- Page 6: There is a claim that the mitochondria membrane potential is very sensitive to the stress state of the cell, here a reference would be good to support it.\n- Page 6: It would be beneficial to show more examples, for example in a supplementary figure, and statistics of the flickering of membrane potential at constriction sites – right now only one example is shown but in the text it is mentioned that multiple such events were captured. How often did you find such sites, and how many did you find in total? Furthermore, the example shown seems to not be connected to a division event, while the network was optimized to detect potential division sites. Is there any connection between reversible constrictions and membrane potential flickering, or does it also happen following division? How many events that you detected led to division and how many led to reversed constriction?\n- There is currently no reference to Fig S6/Supplementary Note 6 in the text – insert this at an appropriate point.\n- In the Fig S6b legend it is stated that the F1 scores are improved with additional time points for all architectures and loss functions. However, from the graph it seems relatively clear that using the U-Net there is no significant improvement, neither with bce nor focal loss function. Was the comment mentioned for the stateful architecture only?\n- Laser powers for fluorescence imaging, wherever mentioned, are mentioned in percentage. This arbitrary unit does not help the reader to understand what is going on, and is entirely subjected to e.g. the microscope and the microscope alignment at the day of experiments. This value should be replaced with a measure of power (W) or intensity (W/m2) at the sample plane, back focal plane or similar conjugate plane to ensure accurate reporting and allow any kind of comparison, especially since claims about sample health are made as a key point of the method.\nI further hope that my comments are helpful in improving the manuscript.\n(Remarks on code availability)\nThe data handling code provided (deep-events) has a limited README file with shallow installation instructions, further instructions would be helpful. Furthermore, it requires a running MongoDB server, and as this is not something I have available I cannot try out and review the code.\nAs far as I understand the microscope control software structure, it works through the pymmcore-plus environment with a few additional programs running in synchronicity. For the modules that are not part of pymmcore-plus according to Fig S3, i.e. smartRunner, Analyser, Interpreter and Actuator, I cannot find references in the manuscript or supplementary information to any availability, and as such I am unable to review it. Accessibility to such code would be helpful and I expect that this code will be shared upon publication."}, "author_response": "We thank the reviewers for their careful evaluation of our manuscript and their constructive comments. Below we provide a point-by-point response.\nReviewer 1\nGeneral comment. The hybrid event-driven acquisition imaging system developed by Willi L. Stepp and coworkers represents a significant breakthrough in biological imaging technique. This method addresses a critical challenge of real-time online fluorescence imaging for rare biological processes: Long-term high excitation irradiance generates cytotoxic free radicals that significantly affect physiological processes. The team overcame this obstacle by employing an event-drivenacquisition(EDA)frameworkthatcombinesthegentlenessandcontextualrichness ofphasecontrastwiththefunctionalspecificityoffluorescence. Byintegratingdeeplearningal- gorithmstodetectdynamiceventsandtriggerfluorescenceacquisition,themethodsignificantly improves imaging efficiency and cell compatibility. It holds great potential for capturing rare biological events and shows promising applications in live-cell dynamic studies.\nWe thank the reviewer for the very positive assessment of our work.\nQ1. The criteria used to define organelle contact or division events remain unclear—whether these are defined based on spatial proximity, temporal persistence, or subjective annotation. It is recommended that the authors provide a detailed description of the annotation procedure, including how contact events are identified and verified. The consistency of manual labeling should also be evaluated to ensure dataset reliability.\nResponse. We agree and have added details to the description of our annotation procedure in the corresponding Methods section. We also now provide all training events in a structured format including metadata in the Zenodo archive.\nAdded text [Methods, Neural Network Training]. The last five frames of positive events were labeled with a Gaussian spot at the site of the event (division/contact). Organelle contact events were manually annotated based on a visible interaction of the two organelles by deformation of the mitochondrion. Additionally,bothspatialproximityandtemporalpersistence(novisibleseparation for at least three frames) were required for event selection. Candidate events were followed in time and only those with clear separation of the daughter mitochondria were labeled as fission events. The division site was tracked back in time and labeled as long as the future division site was distinguishable as a constriction. All events used for training were checked by at least two people independently to ensure dataset reliability.\nQ2. Themanuscriptdoesnotreportthedatasetsize, thenumberofpositiveandnegativesam- ples, or how the data was split into training, validation, and test sets. Given that the biological events of interest are rare, the potential for class imbalance is high, which can bias training. The authors should provide these statistics to allow assessment of model generalizability.\nResponse. WenowprovidethisinformationintheMethodssection. Weusedearlymodels(trained onfewerevents)onphase-contrastdata,correctedfalsepositivesandfalsenegativesandaddedthese events to the training data for subsequent training. Including negative events allows us to avoid class imbalance.\nAdded text [Methods, Neural Network Training]. Organelle contact detection was trained on 111 events (58 positive, 53 negative), evaluation data was comprised of 29 events (18 positive, 11 negative). 596 events (319 positive, 277 negative) were used to train the detection of mitochondrial divisions with 157 events (86 positive, 71 negative) in the validation set. Training and evaluation data was split on an event basis upon registration in the event database, thus ensuring consistency of the training runs and performance scoring.\nQ3. Figure2drevealsrelativelylowrecallvalues,whichmayresultintheomissionofbiologically relevant events. The authors should discuss the implications of these omissions on biological interpretations and indicate whether any strategies were used to reduce false negatives.\nResponse. We agree that this point is important to discuss. The threshold for event detection was selected by maximizing the F0.1-score, which strongly favors precision over recall (β = 0.1). This choice was intentional, prioritizing high-confidence detections rather than high recall (at the cost of increasedfalsepositives)toreducepotentialphototoxicity. Thisresultsinsparsesampling;however, the random nature of the sparse sampling ensures that the detected subset is an approximately unbiased representation of events across the field of view and over time, allowing us to draw robust conclusions about temporal and spatial dynamics and to infer causal relations. To corroborate this claim, we manually identified missed events in the initial validation datasets and verified that they did not exhibit systematic structure in terms of size, shape, or spatial location. In order to further increase recall, we also fine-tuned our models by adding these false negative events to the training datasets. Finally, we note that it would also be possible to optimize the threshold to achieve higher recall if one wished to prioritize the fraction of detected events over precision. However, this would be somewhat at odds with the guiding principle of the hybrid-EDA approach, since increased recall would come at the cost of increased false positives and thus higher phototoxicity. We updated the Methods section to include these comments.\nAdded text [Methods, Neural Network Training]. This metric prioritizes precision, and we chose ittoreducetriggeringonfalsepositives. Thethresholdforclassificationwasthereforeoptimizedfor the best F scores and recall and precision are reported at that threshold unless declared otherwise. β Prioritizing precision over recall results in the omission of some true events (false negatives), while reducingphototoxicitythatwouldresultfrom fluorescenceimaging offalse positives. This isconsis- tent with the goal of Hybrid-EDA, to reduce phototoxicity, and the use of phase contrast which is a nearly non-toxic monitoring modality. Missed events in validation datasets were visually inspected to and found not to introduce any evident systematic bias, and were used to fine-tune the models to increase recall.\nQ4. While the main advantage of hybrid-EDA lies in reduced phototoxicity, this introduces a clear trade-off with detection sensitivity. It would be beneficial to quantitatively characterize this trade-off—for example, by plotting recall and the number of fluorescence frames under varying detection thresholds—so that users can adjust the system to suit specific experimental constraints.\nResponse. We found the inter-dependencies of different factors such as experimental settings, model performance, and relative phototoxicity to be fairly complicated. We explored the behavior ofthetechniqueatdifferentthresholdsforseveralofthesethroughsimulation. Wereporttheresults in the new Suppl. Fig. 4 and supply the code for these simulations on Zenodo. Panel b provides thespecificinformationrequestedhere,whilemorecontextisprovidedintheotherpanels. Detailed dataonrecallandprecisionforallmodelsusedinthemanuscriptisalsoavailableindataandplotted graphs.\nQ5. Itisnotstatedwhetherthesameobjectiveisusedforbothphasecontrastandfluorescence imaging. The authors should clearly state the model and specifications of the objective lens used.\nResponse. Wenowreportthedetailsoftheobjectiveused, andcommunicatemoreclearlythatwe used the same objective for fluorescence and phase contrast imaging.\nAdded text [Methods, Microscope]. All imaging was performed on a Zeiss AxioObserver7 with a Plan Apochromat 63x/1.40 Oil Ph 3 objective for both phase-contrast and fluorescence imaging. A CoolLed pE-800 module was used for fluorescence illumination. Images were captured on a Photometrics Prime camera and cells were kept at 37C and 5% CO2. Adaptive acquisitions were performed using a custom software built on the pymmcore-plus environment described below.\nQ6. Themanuscriptdoesnotreporttheresponselatencybetweeneventdetectionandactivation ofthefluorescenceimagingchannel. Thisparameteriscrucialforcapturingtransienteventsand forassessingtheoveralltemporalprecisionofthehybrid-EDAsystem. Includingthisinformation would enhance the transparency and reproducibility of the method.\nResponse. Weagree,inthenewSupplementaryNote2,wenowreporttheresponselatencybetween event detection and the activation of the fluorescence imaging channel. Specifically, we measured both the model inference time and the latency until the subsequent smart frame acquisition by analyzing data from 1000-frame time-lapse experiments.\nQ7. The hybrid-EDA system has advantages in real-time monitoring of slow physiological processes. For example, using this system to monitor the formation of lipid droplets.\nResponse. Weagreethathybrid-EDAcanbeappliedtoslowrareevents,suchastheonehighlighted by the reviewer, that we now mention in the Discussion. We selected specific use-cases linked to organellesvisiblebyphasecontrast,asweforeseechallengesinidentifyingtheendoplasmicreticulum (fromwhichlipiddropletsform)usingthistechnique. However,theexploitationofdifferentlabel-free imaging methods may in the future allow for the detection of different structures and interactions, perhaps including lipid droplet biogenesis.\nQ8. The functional interactions of organelles is via membrane contact sites (MCSs) defined as areasofcloseproximity(from10to80nm)betweenthemembranesoftwoorganelles. Studying organelleinteractionsthroughconventionalfluorescencemicroscopycanproducemanyartifacts. This hybrid-EDA system can be combined with super-resolution technology or fluorescence lifetime imaging technology.\nResponse. Weagreethatstudyingorganellecontactsiteswithconventionalfluorescencemicroscopy hasimportantlimitations. Inprinciple,therearenoconceptualbarrierstointegratingphasecontrast monitoring with super-resolution or Fluorescence Lifetime Imaging (FLIM) approaches, but several technical challenges currently limit such combinations. For instance, STED relies on sequential, point-by-point acquisition as opposed to widefield phase contrast. Also, single-molecule localization methodssuchas PALMorSTORM require longacquisitiontimes. Forthese reasons, inthe present workwefocusedonmaintainingtemporalresolutionandimagingspeed,whileleavingtheintegration withadvancedsuper-resolutionorlifetimemodalitiesasapromisingdirectionforfuturedevelopment.\nQ9. I am not fully convinced that the image showed events of temporal membrane potential loss or ‘flickering’ (Fig 3h). TMRE monitors the changes in mitochondrial membrane potential (mainly decreases) through a concentration diffusion process. Due to the limitations of diffu- sion speed and concentration gradient, TMRE hardly monitors the rapid dynamic changes of membrane potential.\nResponse. We agree that TMRE carries the limitations of a diffusion-based probe. Nonetheless, TMRE and TMRM are the probes originally used to characterize mitochondrial flickering, and they remain the probes of choice for membrane potential analysis3,4,7,9. Their dynamics have been shown to closely follow pH changes from a protein-encoded sensor12, thereby suggesting they are a reasonable proxy for membrane potential at the imaging speeds used in this study.\nReviewer 2 and Reviewer 3\nThe manuscript follows a recent trend in developing smart and sample-adaptive acquisition modalities, a direction microscopy will likely turn more and more toward. The method uti- lizes multiple microscopy methods to monitor the sample with a gentle one and only apply the to-an-extent more phototoxic method when events are detected minimizes the influence of the microscopy on the sample, helping the sample health over a longer period. The implementa- tion uses artificial intelligence in an appropriate way, in aiding the detection of events which with classical-computer-vision-based methods would be difficult to capture with any reasonable precision and recall. It also utilizes temporal information rather than a single frame in the eventdetectionprocess,leadingtomorereliabledetection,akeywhichhasalsopreviouslybeen shown for event-driven microscopy. The implementation of the temporal information in the event detection is simple yet elegant and slightly boosts the performance.\nMymainconcernwiththecurrentmanuscriptisthatalltheapplicationspresenteddonotshow- casethemethodtothefullest,asIwouldarguethatsomeoftheexperimentscouldbeperformed at least at the same level using manual or classically automatic control of the microscope (see below). However, the application on mitochondrial division is implemented in an impressive manner, clearly building on their previous work (Maheˇci´c 2022) but now detecting the events withonlyphasecontrastinformation. Whenitcomestofindingsintheapplicationexperiments presented, the DRP1 intensity ratio between the ends of divided mitochondria is a question which the method is nicely adapted to tackle, but the investigation remains rather shallow and with a small sample size. Otherwise the experiments are presented phenomenologically without any quantification, and as such there are no further significant biological findings despite a potential for that. The method implementation nevertheless promises tackling of more complex biological questions with molecule-specific labeling, such as with DRP1, in the future, and thus overall the method holds great significance for the field of smart microscopy and applications of it and is important in its own right.\nThe adaptive combination of multiple imaging techniques in smart microscopy is not novel in the field, however, as they rightfully mention, their implementation with a label-free monitor- ing technique is significant for further future developments of smart microscopy methods and holds great potential for applications of the method as such. One can envision future potential combinations with higher-resolution and super-resolution fluorescence imaging techniques such as SIM or STED.\nI see no major flaws with the validity or robustness of the presented experiments and data interpretation, but have a few minor comments and questions below when it comes to the presentation of and claims from the data. I believe they can be resolved by improvements to clarity and presentation. With regards to the overall clarity of the manuscript, the text is well-written, figures are clear, and previous work in the field has been referenced in a satisfactory manner to provide a clear context and overview to the potential reader.\nWethankthereviewersfortheirevaluationofourmanuscript;weaddressallcommentsandquestions below. Wewereespeciallygratefulforthesuggestionsregardingthescopeofapplicationsanddepth of quantitative analysis, which we addressed to improve the clarity and strength of the manuscript.\nQ1. Most supporting movies seem to be unreferenced in the text. Furthermore, there are no channel labels in the movies and no legends in the documents received. As such, it is difficult to understand to which sections the movies belong and which movies support which claim. Properly referencing all movies in the text, or in the supplementary material, would be useful.\nResponse. We agree, we have updated the supporting movies and now refer to them in the text.\nQ2. Page 2: There are claims that the impact on sample health during the monitoring phase is reduced by “greater than a 100-fold” at the end of the introduction, while in the quantification presented in Fig. 1 and Supplementary Note 1 a 50-fold improvement is noted. If this is the quantification supporting the initial statement these values should agree.\nResponse. We now more clearly refer to the excess mortality – that we think is the important value – in the introduction and results sections.\nAddedtext[Introduction]. IntegratingthisintotheEDAparadigmallowedustoreducetheimpact onsamplehealthduringsurveillancebyamorethan100-foldreductioninexcessmortality,resulting in ten times longer experiments or, equivalently, capturing ten times more rare events.\nAddedtext[Results]. Exponentialdecayfitsofthesecurvesreflectanapproximate50-folddecrease in half-life times and a greater than 100-fold excess mortality of the cells in fluorescence imaging compared to phase contrast.\nQ3. Page4: AsmalldiscussionregardingthechoiceofaU-Netarchitectureoverotheralterna- tives for the neural networks, with some potential references, I believe could be interesting for the potential reader.\nResponse. We have extended the text to compare to other architectures and explain our choice of a U-Net.\nAdded text [Results]. For event detection, we trained neural networks with a U-Net architecture with the phase contrast frames and a ground truth containing a local label consisting of a Gaussian located at each contact site. Alternative architectures such as detection-based models (YOLO11, Spotiflow2)ortransformer-basedapproaches(LW-DETR1)couldbesuitablefortheeventdetection task. However, our choice of a U-Net as our backbone was motivated by several reasons: (1) computational efficiency for real-time feedback to the acquisition, (2) ability to provide precise spatial localization through skip connections, (3) efficiency with limited training data, and (4) ease of incorporating temporal information. The U-Net architecture has additionally been shown to be on par with more recent architectures in practical settings such as cell segmentation13.\nQ4. Page 4: When discussing the improvements to the original network, it is mentioned that “the recall is improved by almost a factor of 2” when adding temporal information. When looking at fig 2d, for the original BCE model there is barely an improvement of the mean and the marked best F0,1 model is improved with a factor of 1.5. For the best model in the Soft Focal case the improvement is 2-fold, but for the mean it is neither there more than seemingly a factor of 1.5. For clarity, if the mentioned factor of 2 is quantified for the best network with a soft focal loss function this should be mentioned in the text, as it currently can be interpreted as the mean and especially as an improvement factor for the BCE case due to the order of the modifications.\nResponse. We revised the text to clarify according to this feedback.\nAdded text. WefirstselectedmodelswiththebestF scoresandusedthemtoclassifytheevents 0.1 in the validation dataset. However, we observed a considerable spread in performance for models trainedwithidenticalparameters. Toaddressthis,weswitchedfromcrossentropytoasoftfocalloss function for training, a strategy known to be beneficial for the detection of rare positive events5,6. This stabilized the training at high performance scores. Yet, even these models yielded a high rate of false negatives — instances where genuine organelle contacts were missed (Fig. 2b). Therefore, we further improved our training approach by supplying a time series of images to the models in the form of additional image channels, enabling the networks to use dynamic information in the detection of events (Fig. 2c). With soft focal loss, this increased the recall of the best performing models by almost a factor of 2, allowing more events to be captured during adaptive acquisitions (Fig. 2d, Suppl. Fig. 4). These improved models enabled efficient detection of apparent contact events between spherical organelles and mitochondria.\nQ5. Page4: Intheexperimentsregardingorganellecontactsites,Iamstrugglingtounderstand whatbenefitthefluorescenceimagingduringthe eventbringsinadditiontothe phase contrast. As the resolution seems similar in the two methods in the data presented, all contact sites can clearly be visualized in the phase contrast data. This is further indicated by the fact that only phase contrast data is used to annotate the training dataset for this event detection network. In the cases where knowing which organelle type is touching mitochondria, a single fluorescence frame recorded before or after a phase contrast time lapse, for example every minute, would seem to give the same information; in the data presented, the cellular organelles are sufficiently sparse,thephasecontrastimagingissufficientlyrapid,andtheorganellesmovesufficientlyslow thatasingleframeoffluorescencedataseemsenoughtoidentifyfluorescentlylabeledorganelles and thus allow to follow them throughout the phase contrast time lapse. If so, the manual or “classically” automatic approach of recording a single fluorescence frame every x minutes would be beneficial from a phototoxic point of view, according to what the authors mention earlier, and simplify microscope control. Furthermore, the full data could be analyzed after the experiments,withmoreinformationavailableforapotentialneuralnetwork,andtherecallcould probably be improved further in that case. Further evidence for and discussion regarding that the fluorescence data during organelle contact event sites provides additional information from which any conclusions about the contact sites could be drawn would enhance this application.\nResponse. Toaddressthis,weperformedadditionalexperiments(”Smartacquisitionsofmitochon- drial divisions and constrictions” section). As the reviewer mentions, fluorescence imaging provides invaluabledatafororganelleidentity(mitochondria,lipiddroplets),butthismightnotrequireEDA. However, EDA is important for collecting information on organelle function (e.g. membrane poten- tial), structures invisible in phase (e.g. nucleoids, DRP1), and rapid dynamics and fluctuations during the acquisition. Short-lived membrane potential variation (such as observed during fission but not upon vesicle contact) are fundamental for mitochondrial quality control. The rapid traf- ficking and reorganization of mitochondria and their nucleoids makes dynamic information critical for their accurate investigation. We are grateful for this feedback – these new analyses and textual changes better demonstrate the improvements of our imaging paradigm over traditional approaches for cellular health and dynamic resolution.\nQ6. Page 4: It is mentioned that hybrid-EDA enabled imaging for “nine times longer”, since there was “one organelle contact site detected every 3.1 minutes on average”. I miss the calculation tosupportthisclaim, andcannotfollowwherethevalue“ninetimeslonger”comesfrom. A clarification of this claim in the text would be helpful. The same goes for the 1/3.2 minutes and 10x longer imaging that is mentioned for mitochondrial division events.\nResponse. We now include more details to clarify this estimation.\nAddedtext[Results]. Duringlabelingweidentifiedonecontacteventonaverageevery3.1minutes, with a duration of 20 seconds; thus, when imaging at 1 Hz hybrid-EDA captured 20 fluorescence frames per event instead of about 186 which would have been captured during continuous fluores- cence. This represents a nine-fold reduction in phototoxicity per event, or a nine times longer accessible experiment duration.\nWith an estimated division rate of 1 per 3.2 minutes for a typical cell, when imaging at 1 Hz we couldimagetentimeslongerinhybrid-EDAbyimaging20fluorescenceframesperevent, compared with about 192 if we were imaging continuously in fluorescence.\nQ7. Page 6: When it comes to the experiments on mitochondrial division event sites, the same couldbearguedabouttheexperimentswhereMitoTrackeristhefluorescenttarget(FigureS6). In this case, the fluorescence data does not seem to add much information, as the resolution between phase contrast and fluorescence seems similar, and the tubular organelles are already distinctly identified as mitochondria, according to the discussion of the author about the or- ganelle contact site event detection. A further discussion on what the fluorescence data adds in this case would be helpful. Experiments where DRP1 or TMRE are used as fluorescent targets instead presents clear benefits regarding the information one can gather from the experiments as opposed to pure phase contrast or fluorescence imaging.\nResponse. A main benefit in this data is that analysis tools for segmentation, network detection and morphology of mitochondria are typically developed with fluorescence data in mind. Those toolsdonotgenerallyworkwellwithphasecontrastduetotheadditionalsignalsfromothercellular features. With this data, these established tools could be combined with the low phototoxicity of hybrid-EDA experiments. We have adjusted the caption on the supplementary Figure to explain this.\nModified caption Suppl. Fig. 8. Additional division events captured using hybrid EDA. A variety of mitochondrial morphologies triggered the adaptive fluorescence acquisitions in different cellular environments. Phase contrast imaging (top), MitoTracker fluorescence (bottom), and score obtained from the model on-the-fly as an image (crop marked as a red square at time 0). The event score is plotted below each series with a red line marking the threshold used. Capturing the fluorescence channel during the divisions allows for the use of established analysis tools for segmentation and quantification compared to the very scarce tool set available for phase-contrast data. This allows for analysis of mitochondrial length, diameter and network structure during division events (Supplementary Videos 9-12).\nQ8. Page 6, first paragraph of Smart acquisitions of mitochondrial divisions and constrictions: Fig 3e is referenced, but it should be Fig 3f.\nResponse. Corrected.\nQ9. Page6: Thereisaclaimthatthemitochondriamembranepotentialisverysensitivetothe stress state of the cell, here a reference would be good to support it.\nResponse. Added.\nAdded text [Results]. However, the membrane potential itself is very sensitive to the cell’s stress state14 — meaning that even the moderate light exposure required for fluorescence imaging can induce changes.\nQ10. Page 6: It would be beneficial to show more examples, for example in a supplementary figure,andstatisticsoftheflickeringofmembranepotentialatconstrictionsites–rightnowonly one example is shown but in the text it is mentioned that multiple such events were captured. How often did you find such sites, and how many did you find in total? Furthermore, the example shown seems to not be connected to a division event, while the network was optimized to detect potential division sites. Is there any connection between reversible constrictions and membranepotentialflickering, ordoesitalsohappenfollowingdivision? Howmanyeventsthat you detected led to division and how many led to reversed constriction?\nResponse. We agree that this is an interesting correlation. We have analyzed more data like this and report the results in a new Supplementary Figure and the relative numbers in the main text.\nAdded text. Fluorescent dyes and proteins have developed to be increasingly efficient and photo- stable; improvements in functional reporters are less forthcoming, perhaps because their specificity in sensing constrains optimizing other attributes beneficial for imaging. TMRE (Tetramethylrho- damine ethyl ester) is widely used to measure mitochondrial membrane potential. However, the membrane potential itself is very sensitive to the cell’s stress state—meaning that even the mod- erate light exposure required for fluorescence imaging can induce changes. This sensitivity makes it challenging to reliably capture rapid fluctuations in membrane potential, especially those which accompanymitochondrialdynamics. Weappliedoursmartimagingapproachtomeasuremembrane potential in the vicinity of constriction sites and at divisions and captured events of temporal mem- brane potential loss — or ‘flickering’ — associated with these sites. While whole-mitochondrion flickeringisreadilyobservedwithtraditionalmicroscopy10, thisrapidlyreversiblesubdomainlossof potential would be undetectable at the lower imaging rates typically used in these experiments to avoid reporter photobleaching (Fig. 3h, Suppl. Fig. 6). Among a total of twenty division events, we captured 4 with significant fluctuations in membrane potential, only one of which was observed to reverse constriction without going to full separation of the organelles.\nQ11. There is currently no reference to Fig S6/Supplementary Note 6 in the text – insert this at an appropriate point.\nResponse. Added.\nQ12. In the Fig S6b legend it is stated that the F1 scores are improved with additional time points for all architectures and loss functions. However, from the graph it seems relatively clear that using the U-Net there is no significant improvement, neither with bce nor focal loss function. Was the comment mentioned for the stateful architecture only?\nResponse. Yes,forthestatefularchitecture–weadjustedthecorrespondingtextaccordingly(now Fig. S7).\nQ13. Laserpowersforfluorescenceimaging, wherevermentioned, arementionedinpercentage. This arbitrary unit does not help the reader to understand what is going on, and is entirely subjected to e.g. the microscope and the microscope alignment at the day of experiments. This valueshouldbereplacedwithameasureofpower(W)orintensity(W/m2)atthesampleplane, back focal plane or similar conjugate plane to ensure accurate reporting and allow any kind of comparison, especiallysinceclaimsaboutsamplehealtharemadeasakeypointofthemethod.\nResponse. Agreed, in the revised section of the supplementary materials, we have replaced the percentage values with calibrated intensity values (mW/mm²) at the sample plane.\nQ14. The data handling code provided (deep-events) has a limited README file with shallow installation instructions, further instructions would be helpful. Furthermore, it requires a run- ningMongoDBserver, andasthisisnotsomethingIhaveavailableIcannottryoutandreview the code.\nResponse. Wethankthereviewerforpointingthisout. Intherevisedversion,wehavesignificantly expanded the README file to include detailed installation and usage instructions. Regarding the MongoDB requirement, we now provide guidance in the README on how to download and install MongoDB, with a direct reference to the official documentation.\nQ15. As far as I understand the microscope control software structure, it works through the pymmcore-plus environment with a few additional programs running in synchronicity. For the modules that are not part of pymmcore-plus according to Fig S3, i.e. smartRunner, Analyser, Interpreter and Actuator, I cannot find references in the manuscript or supplementary informa- tion to any availability, and as such I am unable to review it. Accessibility to such code would be helpful and I expect that this code will be shared upon publication.\nResponse. We have added information and links to the specific components in the Supplementary Note 4. Many of the components are derived from the versions we used in the original event-driven acquisitions paper8. The exception is the new actuator that can switch between two independent acquisitions.\nReferences\n1. Chen,Q.etal.LW-DETR:ATransformerReplacementtoYOLOforReal-TimeDetection2024.arXiv:2406.03459[cs.CV].\n2. DominguezMantes,A.etal.Spotiflow:accurateandefficientspotdetectionforfluorescencemicroscopywithdeepstereographicflowregression.Nature\nMethods22.citedBy1,1495–1504(2025).\n3. Hattori,T.,Watanabe,K.,Uechi,Y.,Yoshioka,H.&Ohta,Y.Repetitivetransientdepolarizationsoftheinnermitochondrialmembraneinducedbyproton\npumping.BiophysicalJournal88.Citedby:45;AllOpenAccess;GreenAcceptedOpenAccess;GreenOpenAccess;HybridGoldOpenAccess,2340–2349\n(2005).\n4. Lee,H.&Yoon,Y.Transientcontractionofmitochondriainducesdepolarizationthroughtheinnermembranedynaminopa1protein.JournalofBiological\nChemistry289.citedBy42,11862–11872(2014).\n5. Li,B.etal.EqualizedFocalLossforDenseLong-TailedObjectDetectionin2022IEEE/CVFConferenceonComputerVisionandPatternRecognition\n(CVPR)(IEEE,NewOrleans,LA,USA,June2022),6980–6989.\n6. Lin,T.-Y.,Goyal,P.,Girshick,R.,He,K.&Doll´ar,P.FocalLossforDenseObjectDetectionin2017IEEEInternationalConferenceonComputerVision\n(ICCV)(Oct.2017),2999–3007.\n7. Loew,L.M.,Tuft,R.A.,Carrington,W.A.&Fay,F.S.Imaginginfivedimensions:time-dependentmembranepotentialsinindividualmitochondria.\nBiophysicalJournal65.Citedby:206;AllOpenAccess;GreenAcceptedOpenAccess;GreenOpenAccess;HybridGoldOpenAccess,2396–2407(1993).\n8. Mahecic,D.etal.Event-DrivenAcquisitionforContent-EnrichedMicroscopy.NatureMethods19,1262–1267(Oct.2022).\n9. O’Reilly,C.M.,Fogarty,K.E.,Drummond,R.M.,Tuft,R.A.&Walsh,J.V.QuantitativeAnalysisofSpontaneousMitochondrialDepolarizations.Biophysical\nJournal85.Citedby:119;AllOpenAccess;GreenAcceptedOpenAccess;GreenOpenAccess;HybridGoldOpenAccess,3350–3357(2003).\n10. O’Reilly,C.M.,Fogarty,K.E.,Drummond,R.M.,Tuft,R.A.&Walsh,J.V.QuantitativeAnalysisofSpontaneousMitochondrialDepolarizations.Biophysical\nJournal85,3350–3357(Nov.2003).\n11. Redmon,J.,Divvala,S.,Girshick,R.&Farhadi,A.Youonlylookonce:Unified,real-timeobjectdetectionin.2016-December.citedBy42657(2016),\n779–788.\n12. Santo-Domingo,J.,Giacomello,M.,Poburko,D.,Scorrano,L.&Demaurex,N.OPA1promotespHflashesthatspreadbetweencontiguousmitochondria\nwithoutmatrixproteinexchange.EMBOJournal32.citedBy89,1927–1940(2013).\n13. Stringer,C.&Pachitariu,M.Cellpose3:one-clickimagerestorationforimprovedcellularsegmentation.NatureMethods22.citedBy46,592–599(2025).\n14. Zamzami,N.etal.Sequentialreductionofmitochondrialtransmembranepotentialandgenerationofreactiveoxygenspeciesinearlyprogrammedcelldeath.\nJournalofExperimentalMedicine182.citedBy1471,367–377(1995)."}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have fully addressed all comments raised in the previous review round. The revisions substantially improved the manuscript in clarity, completeness, and scientific rigor. The newly added quantitative analyses, clearer methodological descriptions, and expanded discussions on model performance and application scope have effectively resolved all major concerns.\nThe study presents a technically sound and conceptually innovative framework for hybrid event-driven acquisition microscopy, demonstrating both methodological advancement and biological relevance. I recommend the manuscript for publication in its current form.\n(Remarks on code availability)\nThe code provided is well organized and clearly documented. The revised version now includes an expanded README file with detailed installation and usage instructions, as stated in the authors’ response. The structure of the repository is clear, and dependencies are properly listed. The code is reproducible and runs as expected using the provided test datasets. It constitutes a valuable resource for the community interested in event-driven or adaptive microscopy workflows.", "reviewer_2": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_3": "Most of my questions and comments have been accurately addressed and fixed in the manuscript, in my opinion significantly improving its clarity at multiple points. See below my replies to any remaining concerns or additional comments, labelled as Q:s according to the rebuttal letter. I believe they should be easily correctable by updates to labeling throughout the text, addition of some label text in the supplementary movies, and potentially the addition of a supplementary figure consisting of existing data already used in quantification or alternatively an explanation of why this is not feasible/beneficial to the manuscript.\nQ1 – Supplementary movies\nI am unfortunately unable to see any modifications to the supplementary movies – the 13 files provided (labelled Supplementary Movie 1–13 and pertaining to figures 2e, 2f, 3f, 3h, 4a, 4b, 4c, 4d, 6a, 6b, 6c, 6d, 6e in the reviewer files) look to be identical as in the first submission. I can see references added to a few movies in the main text: “Movie” at the mention of Fig. 2e, “Movie” at the mention of Fig. 2f, “Supplementary Videos 9–12 in Suppl. Fig. 8 legend. The rest of the supplementary movies (3–8) are still unreferenced in the text, and it would be clearer if the mentions are uniform and precise (such as Suppl. Movie X), rather than just referring to “Movie”. The labels in the reviewer files where some seem to pertain to figures 4 and 6 is very unclear, as these figures do not exist. I assume that Suppl. Fig. 5 and Suppl. Fig. 8 are meant instead, but neither that adds up as Suppl. Fig. 8 does not have a panel “e”.\nI still believe the movies would be improved by adding channel labels, and potentially adding legends to the movies in the supplementary material. Seeing the rebuttal, I believe that maybe the authors have updated the movie files themselves, but that the updated files might not have been uploaded properly to the submission system?\nQ5 – Organelle contact sites fluorescence information\nI appreciate the effort to perform additional experiments to show an application where the fluorescence data can be helpful. In the new Suppl. Fig. 6, it would be good to mention in the legend how many events were considered in the quantification in panel b. I believe there has additionally been an error in the ordering of the frames, where 8 s and 10 s come before 6 s.\nQ8 – Figure reference\nThis figure reference is not corrected, it still refers to Fig. 3e rather than 3f.\nQ10 – Membrane potential flickering at constriction sites\nIn my original question, I asked about more examples shown for the membrane potential flickering at constriction and division sites, which is the topic of the paragraph. The data presented in the new Supplementary Figure referenced (Suppl. Fig. 6) refers to membrane potential at lipid droplet contact sites, which instead refers to an earlier paragraph in the text. I am not sure why this Supplementary Figure is referenced here, other than the fact that that experiment uses the same fluorescence labeling? I do still believe that showing further examples of the membrane flickering in correlation to the constriction and division sites mentioned in the statistics would make the point stronger for this application (just as with the additional data displayed in Suppl. Fig. 5 and Suppl. Fig. 8 for the other applications presented), and I do otherwise wonder if there is some specific reason as to not show additional data?\nQ11 – Reference to Suppl. Fig. S8\nDespite the reply in the rebuttal, I cannot see any reference to Suppl. Fig. S8 (previous S6) in the text or supplementary material. Has this been missed to be added?\nAdditional minor comments\n- Suppl. Fig. 4 is currently of very low quality in the supplementary file.\n- The supplementary table is labelled as “Suppl. Table 0” in its legend, but 1 in the text.\n(Remarks on code availability)\nThe updates to the repository README, mainly in terms of installation and usage instructions, as well as additional links in the supplementary material to specific code snippets, will enable a potential user to understand the work and pick it up quicker."}, "author_response": "Response to Reviewers\nReviewer 3\nWe thank the reviewer for their careful evaluation of our rebuttal and their constructive comments.\nBelow we provide a point-by-point response.\nQ1 - Supplementary Movies I am unfortunately unable to see any modifications to the\nsupplementarymovies–the13filesprovided(labelledSupplementaryMovie1–13andpertaining\ntofigures2e,2f,3f,3h,4a,4b,4c,4d,6a,6b,6c,6d,6einthereviewerfiles)looktobeidentical\nas in the first submission. I can see references added to a few movies in the main text: “Movie”\nat the mention of Fig. 2e, “Movie” at the mention of Fig. 2f, “Supplementary Videos 9–12 in\nSuppl. Fig. 8 legend. The rest of the supplementary movies (3–8) are still unreferenced in the\ntext, and it would be clearer if the mentions are uniform and precise (such as Suppl. Movie\nX), rather than just referring to “Movie”. The labels in the reviewer files where some seem to\npertain to figures 4 and 6 is very unclear, as these figures do not exist. I assume that Suppl.\nFig. 5 and Suppl. Fig. 8 are meant instead, but neither that adds up as Suppl. Fig. 8 does not\nhave a panel “e”. I still believe the movies would be improved by adding channel labels, and\npotentially adding legends to the movies in the supplementary material. Seeing the rebuttal, I\nbelieve that maybe the authors have updated the movie files themselves, but that the updated\nfiles might not have been uploaded properly to the submission system?\nResponse. We apologize for the inconvenience caused and thank the reviewer for the careful as-\nsessment. Inthisrevisedsubmission,allsupplementarymoviefileshavebeenupdatedanduploaded\nto ensure full consistency with the manuscript. Specifically, we have:\n• Ensured that all 16 supplementary movies are the updated versions, including the channel\nlabels as suggested.\n• Added clear and consistent references to all supplementary movies in the main and supple-\nmentary text (uniformly cited as Suppl. Movie X).\n• Added legends for all the movies, in the new related section of the SI (p. 12, ”List of Supple-\nmentary Movies”).\nQ5 - Organelle contact sites fluorescence information. Iappreciatetheefforttoperform\nadditionalexperimentstoshowanapplicationwherethefluorescencedatacanbehelpful. Inthe\nnewSuppl. Fig. 6, itwouldbegoodtomentioninthelegendhowmanyeventswereconsidered\nin the quantification in panel b. I believe there has additionally been an error in the ordering\nof the frames, where 8 s and 10 s come before 6 s.\nResponse. Inthisrevisedversion,wehavecorrectedtheframeorderinSupplementaryFig. 6aand\nadded the requested quantification details in the legend. Specifically, the legend now includes the\nfollowing text: “A total of 459 mitochondria were quantified across 16 movies (∼13 frames each), of\nwhich 30 were LD+ and 429 were LD−.”\nQ8 – Figure reference This figure reference is not corrected, it still refers to Fig. 3e rather\nthan 3f.\nResponse. The figure reference has been corrected to Fig. 3f in the revised manuscript.\nasked about more examples shown for the membrane potential flickering at constriction and di-\nvision sites, which is the topic of the paragraph. The data presented in the new Supplementary\nFigure referenced (Suppl. Fig. 6) refers to membrane potential at lipid droplet contact sites,\nwhich instead refers to an earlier paragraph in the text. I am not sure why this Supplementary\nFigure is referenced here, other than the fact that that experiment uses the same fluorescence\nlabeling? I do still believe that showing further examples of the membrane flickering in corre-\nlation to the constriction and division sites mentioned in the statistics would make the point\nstronger for this application (just as with the additional data displayed in Suppl. Fig. 5 and\nSuppl. Fig. 8 for the other applications presented), and I do otherwise wonder if there is some\nspecific reason as to not show additional data?\nResponse. In this revised version, the reference to Supplementary Fig. 6 (mitochondria–lipid\ndroplet contact sites) has been moved to the appropriate section of the text. Additionally, we have\nincluded and referenced a new Supplementary Fig. 9 showing examples of mitochondrial membrane\npotential during mitochondrial constriction and division events, as requested.\nQ11 – Reference to Suppl. Fig. S8. Despite the reply in the rebuttal, I cannot see any\nreference to Suppl. Fig. S8 (previous S6) in the text or supplementary material. Has this been\nmissed to be added?\nResponse. The reference to Supplementary Fig. 8 has now been added in the relevant section of\nthe manuscript and verified for consistency with the supplementary material.\nSuppl. Fig. 4 is currently of very low quality in the supplementary file.\nResponse. Ahigh-resolutionversionofSupplementaryFig. 4hasnowbeenprovidedintherevised\nsupplementary material.\nThe supplementary table is labelled as “Suppl. Table 0” in its legend, but 1 in the text.\nResponse. The supplementary table is now correctly labeled as Supplementary Table 1 in both\nthe legend and the supplementary text.\n2"}]

Decision letter