Revolutionizing EMCCD Denoising through a Novel Physics-Based Learning Framework for Noise Modeling
Authors: Haiyang Jiang
Tetsuichi Wazawa
Imari Sato
Takeharu Nagai
Yinqiang Zheng
Venue: ICLR 2025 Poster (2025)
Keywords: EMCCD
physics-based noise modeling
deep high-sensitivity imaging
fluorescence microscopy image denoising
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: A novel noise model and calibration procedure for EMCCD, synthesizing authentic training data for a neural network to achieve state-of-the-art EMCCD denoising performance.

Abstract: Electron-multiplying charge-coupled device (EMCCD) has been instrumental in sensitive observations under low-light situations including astronomy, material science, and biology. 
Despite its ingenious designs to enhance target signals overcoming read-out circuit noises, produced images are not completely noise free, which could still cast a cloud on desired experiment outcomes, especially in fluorescence microscopy.
Existing studies on EMCCD's noise model have been focusing on statistical characteristics in theory, yet unable to incorporate latest advancements in the field of computational photography, where physics-based noise models are utilized to guide deep learning processes, creating adaptive denoising algorithms for ordinary image sensors.
Still, those models are not directly applicable to EMCCD.
In this paper, we intend to pioneer EMCCD denoising by introducing a systematic study on physics-based noise model calibration procedures for an EMCCD camera, accurately estimating statistical features of observable noise components in experiments, which are then utilized to generate substantial amount of authentic training samples for one of the most recent neural networks.
A first real-world test image dataset for EMCCD is captured, containing both images of ordinary daily scenes and those of microscopic contents.
Benchmarking upon the testset and authentic microscopic images, we demonstrate distinct advantages of our model against previous methods for EMCCD and physics-based noise modeling, forging a promising new path for EMCCD denoising.

Published as a conference paper at ICLR 2025
REVOLUTIONIZING EMCCD DENOISING THROUGH A NOVEL PHYSICS-BASEL LEARNING FRAMEWORK FOR NOISE MODELING
Haiyang Jiang', Tetsuichi Wazawa Imari Sato 3, Takeharu Nagai 2, Yinqiang Zheng
'The Univerisity of Tokyo
Ossaka University
National Institute of Informatics
ABSTRACT
Electron-multipyyy muniipllin charge-coupled device (EMCCD) has been instrumental in sensitive observations under low-light situations including astronomy, material science, and biology. Despite its ingenious designs to enhance target signals overcoming read-out circuit noises, produced images are not completely noise free, which could still cast cloud on desired experiment outcomes, especially in fluorescence microscopy. Existing studies on EMCCD's noise model have been focusing on statistical characteristics in theory, yet unable to incorporate latest ad- vancements in the field of computational photography, where physics-basee noise models are utilized to guide deep learning processes, creating adaptive denoising algorithms for ordinary image sensors. Still, those models not directly appli- cable to EMCCD. In this paper, we intend to pioneer EMCCD denoising by in- troducing a systematic study on physics based noise model calibration procedures for an EMCCD camera, accurately estimatinn statistical features of observable noise components in experiments, which are then utilized to generate substantial amount of authentic training samples for one of the most recent neural networks. A first real-worlh test image dataset for EMCCD is captured, containing both im- ages of ordinary daily scenes and those of microscopic contents. Benchmarking upon the testset and authentic microscopic images, we demonstrate distinet ad- vantages of our model against previous methods for EMCCD and physics-basee noise modeling, forging a promising new path for EMCCD denoising"
INTRODUCTION
Owing to its high sensitivity photons and effective amplification of signals by impact ionization during charge transfer at high frame rates, EMCCD has, since its invention by Denvir & Conroy
2003), , become increasingly significant to multiple research areas that require experiment observa-
tions under extreme low-light conditions, including astronomy (Mallik et al.] [019] Tubbs)] (003)),
quantum imaging (Jeong al., 2023) Meda al. 2017 Kumar al. [2017] 7), and fluorescence mi-
croscopy in biology (Chao et al. 2016 Lee et al., 2017 Han et al. 2012 Kagan et al. 2022 Wang
al. 2023 Yokota et al. 2021 Yet, electron multiplication registers inside EMCCD do not guarantee immunity of noise in resulting images. As can be seen from figure I although in sub- image (a) the 16-bit raw image doesn't reveal much information P noise patterns can be observed clearly when contrast being enhanced in (b). Such noise pattern cannot be successfully removed through averaging operation, as shown in (c). Exposing for longer period, or including more frames into averaging, is not necessarily viable solution as well, especially in the field of fluores- cence microscopy, since subjects being observed might be susceptible to damage from fluorophores pointed out by Icha et al. (2017) Hopt & Neher (200I), while molecular level motions can cause
Corresponding author: yyahhng@ai. .atiiiu-oooo... ac.
"oode and dataset can found at this page
Pixel values typically controlled to he within magnitude of thousands in order protect device from EM gain aging, much lower than the maximum value of 16-hit integers, i.e., 65535.

---

Published as a conference paper at ICLR 2025
(a) Original
(b) Enhanced contrast (c) 4-frame average
(d) Denoised
Figure I:
Our denoising result. (a) Original 16-bit raw image; (b) Contrast enhanced version to
show noise patterns; (c) Averaged result over 4 consecutive frames with residual noise; (d) Denoised clean output of our system.
misalignment across long image sequences. Thus, a reliable denoising algorithm for EMCCD would be beneficial to scientific community trying harness full power of the device.
Researchers and manufacturers have been studying detailed mathematical models of EMCCD over the years. encompassing accurate gain calibration (Ryan et al. 202T, Qiao al. 2021), noise com- ponents (Basden, 2015 Hirsch et al.) 2013; Dussault & Hoess 2004). and signal-to-noise ratio (SNR) analysis (Zhang Chen 22009] Plakhotnik et al. 2006]] Basden et al., 2003), etc. With the advancements of deep learning techniques in recent years, researches involving EMCCD have em- braced this trend, developing task-specific models to better serve distinct objectives in experiments. A common denoising model paradigm, however, has yet be put forward for EMCCDs.
On the other hand, researches in the field of computer vision have progressed to simulate adequately precise noise patterns for standard optical camera sensors from physics-baseh perspective (Weil et al., 2021 Feng et al.) 22244, Cao et al. 2023 Ji et al. 2024 2023b a), in order to synthesize authentic training samples for deep neural networks that mances than those trained with real-world dataset, like See-in-the- -Dakk (SID) by Chen et al." (0018)
achieve comparable or better perfor-
Nonetheless, those models intrinsically different from physics in EMCCDs.
In this paper, we intend to derive insights from the progress of noise modeling in computer vision in order to boost image quality of EMCCD cameras fundamentally. We propose generic and ver- satile noise model for EMCCD, incorporating theoretical analysis on its physical composition and direct experiment results during calibration process on an iXon Ultra 897 EMCCD manufactured by Andor, whose products have been widely adopted in both academic and industrial research. A systematic approach for noise parameter estimation is also detailed, facilitating applicability of our pipeline onto devices from various manufacturers. With acquired statistical features of the EMCCD noise pattern, we dynamically generate massive amount of noisy-clean image pairs from ground truth of low-light raw image dataset, SID, upon which one of the latest powerful transformer model, Uformer from Wang et al. (2022). is trained to extract desirable signal from noises with dex- terity. Because SID consists only of scenes
macroscopic scale, we further finetuned model using synthetic microscopic data, proving its flexibility of being transferred from macro world to flu- orescence microscopy. For fair comparison to existing methods,
utilize EMCCD shoot 224 noisy-clean image pairs using long-short exposure method similar to SID. This world's first EMCCD testset contains 15 exposure ratios 3l ranging from 30 to 1600. Quantitative evaluations on it have demonstrated advantages that our method possesses over both physics- based noise modeling in computer vision that cannot be properly generalized EMCCD, and theoretical noise analysis for EMCCD that cannot pinpoint valuable characteristics for deep learning models.
Our contributions are three-fold:
1. A novel noise model for generating authentic noisy images specific to EMCCD is proposed, integrating fundamental analysis on structural design of the camera, as well as pixel level behaviors from images produced by EMCCD during experiments.
2. We establish a unique pipeline from noise parameter calibration to training strategy of utilizing state-of-the-arr enhancement networks on synthetic dataset, connecting physics perspectives and algorithmic designs for EMCCD denoising in fluorescence microscopy.
Quotient of long exposure time for ground truth divided by short exposure time for noisy image, indicating differences in light intensities, iee., magnitude of photon counts, as well as difficulties for denoising.
2

---

Published as a conference paper at ICLR 2025
3. Experiments are conducted, testing our model against previous methods collected first real-world dataset for EMCCD with macroscopic and microscopic scenes, demonstrating superiority of our noise model and training pipeline.
2 RELATED WORK
2.1 EMCCD NOISE
As a relatively new imaging system, EMCCD has been studied with advanced mathematical and signal processing tools since its birth by Denvir & Conroy (2003). One unique feature of EMCCD noise is multiplicative noise induced by electron multiplication registers (Basden et al. 2003), which can be modeled by Poisson- Poisson-gamma distribution (Hirsch et al. 2013). This component can be described by a noise factor (Denvir & Conroy, (003), amplifying Poisson distributed shot noise caused by guantum essence of photons to corresponding level. Theoretical analysis by Zhang &
Chen (2009) gives that noise factor of EMCCD is v2, while in practice Dussault & Hoess (2004); Araújo al. (018) observed and analyzed that existence of register voltage degradation can bring such value down to L.3. Zhang & Chen (2009) characterized 5 noise sources in EMCCD: photon shot noise, multiplicative noise, dark current noise, clock induced charge, and readout noise. Hirsch ct al. (2013) delved deeper into physical structures of noise sources, presenting a Poisson-gamms. normal (PGN) distribution model, for mathematical representation of each noise element. Ryann et al. (221) established a gain series method based the EMCCD noise model for more accurate EMCCD calibration than mean-variance intensity series method.
Another artifact in EMCCD images worth mentioning, yet not commonly discussed, is that, in- heriting from CCD sensors' blooming effect, EMCCD produces vertical/horizontal light streaks in high-contrast situations (Mallik et al., 2019). This phenomenon is believed be caused by the ab- sence of a mechanical shutter. Thus, we include algorithmic anti-blooming as part of our denoising process, to account for all possible discrepancies between noisy and clean images.
2.2 PHYSICS- IMAGE SENSOR NOISE MODEL
Noise in ordinary camera image sensors under extreme low-light can be divided into signal depen- dent and signal independent components (Nakamura) 2005). Recent noise models draw correlation between noise components and camera gain. Extreme Low-light Denoising (ELD), by Wei et al. (2021), identified 4 elements for noises in raw sensor data: shot noise, row noise, read noise, and quantization noise. PMN. by /Feng et al. (2024), further proposed the existence of dark shading in digital image sensors, depicting ixed-pattert noise (FPN) on fine-grained pixel level. Monakhova al. 2022) took inspiration from payysics-based noise formation in images, putting forward a tem- poral noise model that included periodic and a time-variant row noise components to synthesize noises for starlight video denoising. Cao et al. (2023) considered all possible noise components, and established connection between each component and camera ISO through normalizing flow-based framework, solving parameter estimation problems in a complete model.
In spite of their ability to authentically portray aspects of noises inside camera sensors, these models are fundamentally different from EMCCD's. We intend to modify their methodology order to construct best-suited noise model and calibration procedure for EMCCD denoising.
2.3 DEEP NEURAL NETWORK
U-Net (Ronneberger et al. 2015), has been popular network structure in dense prediction tasks like semantic segmentation and image restoration. SwinIR (Liang et al. (021) exploits both CNN and transformer, combining them into shifted window attention blocks to better aid image enhancement process. Restormer (Zamir et al., 2022) reduces complexity of transformer blocks through multi- head transposed attention and ford-dorward network, achieving long-range pixel connections on large images, while Uformer (Wang et al. 2022) also increases its computational efficiency by non- overlapping window, depth-wiss wiip convolution, and skip connections, maintaining high performance in image enhancement tasks at the same time. Combining the evolved power of neural networks and data synthesis from noise modeling, aim for robust algorithm for EMCCD denoising.
3

---

Published as a conference paper at ICLR 2025
3
NOISE MODEL
3.1 BASIC FORMATION
In figure 2 we present a concise yet precise noise model based on theoretical analysis and experi- mental results iXon Ultra 897 EMCCD The reason for choosing this device is two-fold: hand, its manufacturer, Andor, part of Oxford Instruments, well-regarded for its advanced EM- CCD cameras that offer high performance in research areas, making their products widely adopted in both academia and industry. On the other hand, the stability of this device compared to others allows us focus on challenging noise situations without being disturbed by components that can be avoided through hardware designs.
Fixed Pattern Noise (FPN, (1)
Blooming B(-
Photon Shot Noise N, Noise factor f
Readout Noise N, + Quantization Noise N
Paired samples for EMCCD denoising process
Clean
Noisy
Figure 2:
Outline of our EMCCD noise model, with distinct advantages: inclusion of EMCCD's unique noise factor f: a novel logarithmic logistic regression correlation for fixed pattern noise and exposure; an unprecedented numerical model for blooming effect.
A mathematical description of noise components can be formulated as:
D B KK (I Np f) FPNJ N, N,
In equation|
photon number that actually hit image sensor. D is produced image, i.e., digital number in pixels. Np is Poisson shot noise, multiplied by noise factor [1.3, v2 to form com- bination of shot noise and multiplicative noise in Dussault & Hoess (2004). 3 K is system overall gain, a value that has absorbed quantum efficiency, EM gain, umalog-to-digite. conversion (ADC) factor. It can be viewed
direct ratio between final digital number and incident photon count, if other noises are omitted. FPN stands for fixed pattern noise, a pixel-wise bias map dark current non-uniformity.B [-] represents introduction of blooming artifacts, acting upon all components that induces photoelectrons. N, is readout noise described by a zero-mean normal distribution, while Ng quantization noise described by uniform distribution.
As emphasized in figure 2 our model has 3 uniqueness: noise factor that only appears in EMCCD; precise log-logistic regression for FPN; quantitatively handling blooming effect. Different from pre- vious models that establish correlations between noise parameters and camera gain/ISO, we limited our model the largest EM gain, x00x our device, based on pragmatic viewpoint: it's the case that requires denoising the most, and the most challenging one well. We instead model noise parameters terms of exposure time. The reason for this choice is elaborated in the next subsection.
3.2 PARAMETER ESTIMATION
We utilize two categories of images for quantifying noise components: flat-field and biased frames. The former refers to a uniformly illuminated scene, while the latter is deprived of incident lights.
Flat-field frames are captured by aiming the camera at a uniformly lit white board surface with minimum roughness and unevenness, without any specular reflection. The lens camera is set to
Operating temperature were set to -60 O for all following calibration and test image acquisitions.
Boldd symbols indicate 2D images, e,g., D pM with M and N being image's 's height and width
4

---

Published as a conference paper at ICLR 2025
focus infinity to further diminish possible inconsistency among expected value of pixels. These frames are used to calibrate system overall gain K under each EM gain level.
Similar to ELD, median-variann method can be deduced for EMCCD based on properties of Poisson distributions and multiplicative noise: Var(D) = K f x Median (D) + V arr Var(N.), where N. represents unrelated noise components combined. This implies that we can acquire K by linear regression on variance and median of flat-field frames. Resulting slope would be the target K times square of noise factor f.
Ryan et al. 2 developed a gain-series approach based on EMCCD complete model uses max- imum likelihood estimation to get more accurate K. For more details, please refer to the paper and experiments illustrated in our appendix. Estimated K from two methods are not far apart, yet gain series approach's 's result has better linearity w.r.t. EM gains, which is why we adopt K from gain series as parameters for data synthesis.
Biased frames are taken with EMCCD camera's lens cap on and camera being placed in a tightly sealed darkroom, to prevent any accidental photon response on the sensor. Contained in these al- most completely dark images are systematic contribution from pixel non-uniformity and signal- independent fluctuations, i.e, FPN and readout noise. To get rid of random variations, at any given exposure time, we took 1000 biased frames, pixel-wise averaging of which would produce FPN at that specific exposure. An example of FPN is displayed in figure
As can be seen from sub-figure
Lngit2: HightA
5.77
1.20
(a) FPN@1 = 0.01s
(b) S-shaped curve of FPN against
Figure 3: Characteristics of fixed pattern noise (FPN). (a) FPN when exposure is 0.01s; (b) Loga-
rithmic logistic regression for pixel value of FPN at coordinate (195, 5) with respect to exposure.
(a) that, despite varying on small scale, FPN does have clear pattern with pixel values increasing from top to bottom, ranging from 160 to 195. In total, 52 exposure time were chosen lying between 10 10-5s to 3.0s. Even though there is no linear correlation between pixel value of FPN at any specific coordinate and exposure, we discovered that on logarithmic scale those values of a pixel and expo- sure form an S-shaped curve, namely, log (PPN:) = (log (exposure): 0 with F FNN being pixel value at coordinate (i, a being the logistic curve, and O,y being coefficients for regression. In figure (b), an example of this log-logistic regression is shown, demonstrating closeness between observed value and fit curve, having a coefficient of determinant, 2², over 0.95. One possible expla- nation for this phenomenon that, since unifformittt bias of dark current response is reflected in FPN, such component could have a temporal accumulation, possibly from heat, and a saturation point, forming into an S-shaped curve on logarithmic level. After proper removal of outliers during regression, we managed to refine coefficients from a pixel-specific map to one group of values for entire FPN, i.e., fly + 0. Using these coefficients and base FPN of 10-5,, we are able to achieve FPN predictions with average error per pixel below I digital number for exposure between 0.006s and 0s This can be formulated as follows:
FPN, exp (a (log(t):
with representing exposure.
As for readout noise, it is acquired by subtracting corresponding FPN from biased frames. Remain- ing signal variations follow closely to independent normal distributions at each pixel. In figure 4 plot (a) is a normal probability plot from one pixel calculated across 1000 samples under the same exposure. As empirical distribution (blue dots) overlaps with theoretical normal quantiles (red line), along with an ,d over 0.99, it can be proved that readout noise follow pixel-wise normal distribu- tions. However, those independent randomness does not share parameters. As is evident in plot (b)
5

---

Published as a conference paper at ICLR 2025
0.275
48
(a) Normal probability plot
(b) Standard deviation heatmap
Figure 4: Characteristics of readout noise at = 0.05s. (a) Normal probability plot of readout noise one pixel. with observed blue values perfectly aligned with red line that represents standard normal distribution; (b) A heatmap (with enhanced contrast) of std of readout noise at each pixel.
of figure a standard deviation heatmap under one exposure shows spatial fluctuations. There is no strong connection between such std maps and exposure. Therefore, we store those std maps and sample from them to generate normally distributed pixel-heterogeneous readout noises.
Blooming is another important artifact introduced by EMCCD's structural design independent of noise parameters. To verify and quantify those vertically extended light areas, we developed new category of frame for blooming factor calibration as shown in figure 5 The images are also taken
1369
300
200 400
500
(a) Calibration frame
(b) Proportionate pixel values
Figure 5:
Calibration of blooming factor. (a) Example of a calibration frame, which places a
stable light source the bottom left quadrant against completely black background inside sealed darkroom, leaving the top left quadrant to be mainly blooming pixels; (b) A plot of pixel values (y-axis) from 3 areas, light, blooming, dark, in the calibration frame under various light source intensities (x-axis), showing a proportionate correlation between blooming pixels and bright ones.
inside a darkroom, with a stable light source carefully placed to be at the bottom left quadrant of the frames against black background. forming distinct areas inside each frame: light, blooming, and dark. 1000 images were taken under each exposure time of 19 selected levels from 0.001s to 0. Following calculations are all carried out on pixel-wise averaged image from those 1000 images under each exposure in order to eliminate variations from the noises. Corresponding FPN of a given exposure is also subtracted from the averaged frame to reduce spatial non-uniformity. We sample mean values from sub-regions in areas: from bright light source, from light source excited blooming, and vg from dark area. A plot of these values under a series of varying light source intensities is shown in (b), demonstrating clear linearity for light induced blooming effect.
Based on this observation, we derived a simple estimation method to calculate a blooming factor x solely from values obtainable in the calibration frame.
C1 1/2
where C1 is column-wwie mean value of columns that contain bright area U1 and blooming area b2 shown in figure is defined the ratio between blooming and bright pixels, which can be used an inverted fashion to remove blooming in practice. Details of the derivation can found in our appendix.

---

Published as a conference paper at ICLR 2025
We demonstrate examples of blooming removal in figure 6 evidence for validity of our hypothesis on the artifact.
Blooming (b) Removed (c) Blooming
(d) Removed
Figure 6: Results of blooming removal. (a)&(b) Original image of an indoor macroscopic scene with obvious vertical streaks and its blooming removal result; (c)&(d) Microscopic scene with ver- tical light streaks and blooming removed from it.
3.3 SYNTHESIS
Inspired by Feng et al. 2024), we implement a similar preprocessing pipeline to deal with blooming and FPN outside of neural network, simplifying denoising training samples for the deep learning module. Specifically, for a given noisy input image D, our pipeline processes as follows:
D' B FPN, D
2t col mean(D) FPN,,
1+
t refers to exposure time, being corresponding blooming factor calculated from equation 3 and FPN, being corresponding fixed pattern noise described equation col mean calculates column- wise averages. D' is the direct noisy input for our neural network. Combining equation | and 4 it clear that remaining degradation includes shot noise, multiplicative noise, and readout noise. Therefore, given clean image S, we synthesize a noisy image D' by:
D' K
N, f) N + Ng; 1.3, v2 N N (0. N, U q 2 2
(5)
S I I= K : I+N, ~ P() U
(6)
P is Poisson distribution, U uniform distribution, and N normal distribution. K is system overall gain as in equation [ while std, is a pixel-wise standard deviation map from remaining readout noise in biased frames after subtracting FPN. std, is randomly sampled from all the std maps we collected across multiple exposure time to increase robustness of learning process. q is quantization step for the final digital number in each pixel.
4 EXPERIMENTS
4.1 TRAINING
We trained Uformer on synthetic data. Utilizing high definition raw images from SID, we randomly crop 512 x 512 samples from 231 images of size 4288 2848 to add noises, The GT clean images are originally in color format arranged in a Bayer-pattem, and are thus converted to single channel monochrome format before cropping. An example shown in figure 7 Also included in the figure are two other noise modeling synthetic samples, one of which, shown in sub-figure (c) and denoted (202T). is using a typical physics- based noise modeling for commercial
as ELD from Wei CMOS -SoSssensor cameras. It can be seen that, though recalibrated on our device, it emphasizes row- wise banding-patteri noise that is absent in EMCCD cameras. The other method is a theoretical complete EMCCD noise model by Ryan et al. (2021): Hirsch et al. (2013). shown in sub-figure (d). Main differences between our model and the theoretical one include:
1. preprocessing for blooming and FPN removal;
2. model of FPN and residual noise's dependency on exposure time;
3. instead computationally expensive PGN model, we focus more on tractable yet crucial components for neural networks.

---

Published as a conference paper at ICLR 2025
NIX
ENIM
END
OND
mee O
O
THEE
O
TOMNON
WO
OM TOW
TOW
(a) Clean
(b) Our noise
(c) ELD
(d) theoretical
Figure 7: Synthesized noisy images. (a) A clean GT from SID; (b) Simulated noisy image using our proposed method, adding noise pattern while maintaining content and contrast; (c) Simulated noisy image using ELD, a typical noise model for ordinary camera sensors, introducing unnatural row noise that not EMCCD; (d) Simulated noisy image using theoretical EMCCD noise model from Ryan et al. 2021), slightly altering overall contrast of the image. All noise model are calibrated using our device.
For more details ELD and theoretical EMCCD noise model, please refer to respective papers. We include other details of our training process in our appendix.
4.2 TESTSET AND FINETUNING
To the best of our knowledge. there no real-world evaluation dataset for EMCCD. Thus, to evaluate trained networks quantitatively, we collected a first-ever EMCCD test dataset. Placing the camera at stable position, and controlling exposure and gain values through cable on separated device, we attained long exposure clean images and short exposure noisy images with pixel-level alignment.
Exposure for clean GT images span range from 0.5s 8.0s depending on light intensity of the scene. Noisy input images can have exposure ranging from 1 x x000 to 0.1s, creating ratios between GT and input's exposure ranging from to 1600. GT images are all shot with EM gain disabled and averaged over 10 consecutive frames, while input images with EM gain set to 300 and averaging. Total number image pairs is 224, as one GT could correspond to multiple input images with various short exposure time. Example testset images can be found in the appendix.
Aside from quantitative measurements on a macroscopic testset, we also consider the practical us- age of our method in microscopic observations. We further finetuned model by synthesizing noise on a small dataset of 24 microscopie images including cytoplasm, endoplasmic reticulum, mi- tochondria, and nucleus. Each image obtained by taking 100 consecutive 512 512 frames with 3s exposure, followed by an averaging process and median filter to suppress noise. Example images of the finetuning dataset and additional training details included in our appendix.
RESULTS
5.1
QUANTITATIVE
We use peak signal-to-nen to-iie ratio (PSNR), structural similarity index measure (SSIM by Wang al. 22004),, ) and learned perceptual image patch similarity (LPIPS by Zhang al. (2018) as our metrics to evaluate all results.
Table Quantitative measurements of model performances. Best results are shown in red. The first row, Input, is direct evaluation on noisy inputs against clean GT, to serve as frame of reference for other results. Our model outperforms all others by at least 2dB in PSNR.
Methods
PSNR
SSIM
LPIPS
Input BM3D Dabov et
40.73
0.9173
0.4007
2007
43.69
0.9662
0.1438
ELD (ei et al. 2021
43.90
0.9724
0.1187
Theoretical Ryan et
2021
46.44
0.9788
0.0987
SRDTrans
[ Li al.
44.48
0.9677
0.2116
Ours
48.59
0.9854
0.0742
8

---

Published as a conference paper at ICLR 2025
We compare method against traditional image processing denoising algorithm, BM3D from Dabov et al. (2007), and two aforementioned noise models for learning based supervised method, ELD and theoretical EMCCD noise model, with recalibrated noise parameters and same training process. In addition, a latest self-supervised fluorescence image denoising algorithm, SRD- Trans (Li al. 2023), is compared well. Measurements tested paired macroscopic dataset reported in table From the table it can be seen that, although all algorithms introduce performance gain compared direct imput, our method has the best results among all, outperforming the second best by more than 2dB in PSNR.
We conducted ablation study on components our noise model. table by excluding each part from our full model separately, it hinders overall performances to certain extents, providing evidence for optimality of our strategy.
Table 2: Ablation study on each component of our noise model. Removal of any one would result in performance loss, proving effectiveness of our ensemble design.
Experiments Multiplicative Readout
FPN
Blooming
PSNR
47.41
X
47.46
47.44
IV
X
46.52
IV
V
48.59
Input
BM3D
ELD
Theoretical
SRDTrans
Ours
GT
Figure 8: Qualitative results of samples from our macroscopic testset. Each row represents one sample, while each column is input/result from different methods annotated at the top of the figure.
5.2 QUALITATIVE
We exhibit qualitative test results of method and other compared models in figure 8 It can be seen from the images that not only does our model handle blooming better than others, but overall remaining fluctuations in results minimum well, producing closest appearances to GTs.
In figure 9 we show results on fluorescence microscopic images, where clean GTs are not available. BM3D, ELD. theoretical model, and SRDTrans have conspicuous flaws of oversmoothing, grid-

---

Published as a conference paper at ICLR 2025
Input
BM3D
ELD
Theoretical
SRDTrans
Ours
Finetuned
Figure 9: Qualitative results of samples from fluorescence microscopic testset. Each row represents one sample, while each column is input/result from different methods annotated at the top.
pattern artifacts, inability to handle blooming, and strong remaining noise. Our model that is trained on macroscopic synthesized data produces smoothed results, while after finetuning on microscopic images, the model could reach balance between denoising and detail preserving, generating sharp edges around high contrast areas and maintaining certain level of noises for smooth transition at low contrast areas to avoid artifacts.
Zoomed-in versions of qualitative results can be found in our appendix, providing more detailed visualizations.
5.3 DISCUSSION
Despite achieving visually pleasant output, our trained network still faces inevitable overly- smoothed and residual-noise problems in some test samples, as in all other denoising tasks. Strong noise intensities entail large randomness making hard acquire accurate noise modeling, while neural networks demand samples that are representative of authentic noise distributions. The solu- tion we offered gateway to access pioneering statistical learning EMCCD denoising. Pos- sible improvements
this path include: culf-supereiee learning (Yang et al. (024) few-shot- learning (Jin et al.) 2023) for noise modeling to relieve the burden of calibration data collection; building larger microscopic dataset like Biswas & Barma (2220) that enables high capacity net- works for more accurate noise removal; incorporation of generative diffusion models (Ho et al. 2020) to fill in missing information caused by strong noises.
CONCLUSION
In this paper, we proposed a novel physics- based noise model for EMCCD cameras. A complete systematic noise parameter calibration process established based on both theoretical analysis and empirical experiment results, including unique logarithmic logistic regression for fixed- pattern noise and effective blooming removal technique. Synthesizing noisy images from a real-world low-light raw image dataset, trained state-of-the-art deep learning model. A world's first macroscopic paired testset for EMCCD collected, experiments on which demonstrated superiority of our model against preexisting methods in image processing, noise modeling in computer vision, and theoretical analysis for EMCCD noise. Our model can be further finetuned on microscopic dataset, elevating image quality of denoising output for EMCCD shot fluorescence microscopic images.
fixed-pattern
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This research was supported in part by JST-Mirai Program JPMIMI23G1, JST CREST Grand Num- ber JPMICRI5N3, JSPS KAKENHI Grant Numbers 24KK0209, 24K22318, 22H00529, 18H05410 and 22K04891.
REFERENCES
Óscar Tubío Araújo, Miguel Núñezz Cagigal, Ruben Thor Wenzel Argüelles, Roberto López López, Roberto Luis Simoes, José Marco de la Rosa, Icíar Montilla, Jesús Patrón Recio, Marta Puga An- tolín, Marcos Reyes
Luis Fernando Rodríguez Ramos, Josefina Rosich Minguell, Victor J. S. Béjar, and Alastair Basden. EMCCD in-situ periodic characterization in Shack- Hartmann wavefront sensor for GTCAO. Laird M. Close, Laura Schreiber, and Dirk Schmidt (eds.), Adaptive Optics Systems VI, volume 10703, pp. 107034W. International Society for Optics and Photonics, SPIE, 2018. doi: 10.1117/12.23126411 URL
12 23.2211
A. G. Basden, C. A. Haniff, and C. D. Mackay. Photon counting strategies with low-light-level ceds. Monthly Notices of the Royal Astronomical Society, 345(3):985-991 November 2003, ISSN 1365-2966. doi: 10.1046/j..3653 1365-8711 2.000.0.0007.00 URL htt.httpp/x..xoe www.:ddii...........oo
8711165--11--10 2003 000.7711..220.000022.222 07020.x
Alastair G. Basden. Analysis of electron multiplying charge coupled device and scientific CMOS readout noise models for Shack-Hartmann wavefront sensor accuracy. Journal of Astronomical Telescopes. Instruments, and Systems, 133):039002, 2015. doi: 10.1117/1.JATIS...3.. 10t.1117/21 3.00000
URL https::/ab...ogg//
Sumona Biswas and Shovan Barma. A large-scale optical microscopy image dataset of potato tuber for deep learning based plant cell assessment. Scientific Data, 7(1):311, Oct 2020. ISSN 2052-4463. doi: 10.
URL URL.tttss ht.lhttps://eee wtt.:/i...r.....00/0.0000.00....
s4 241597-020-00760
Yue Cao, Ming Liu, Shuai Liu, Xiaotao Wang, Lei Lei, and Wangmeng Zuo. Physics-guided iso-dependent sensor noise modeling for extreme low-light photography. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5744-5753, June 2023.
Jerry Chao, E. Sally Ward, and Raimund Ober. Fisher information theory for parameter esti- mation in single molecule microscopy: tutorial. J. Opt. Soc. Am. A. (3()):336-B57, Jul 2016. doi: 10.1364/JOSAA.33..000 000B36. /ttlttttp///pp...o wwt.ppp..pp.cc....ooo http:///t..a.rrree/aaaaaaaaaaaa.
2fm2URT=Josaa-233 33
C. Chen,, Q. Chen, J. Xu, and V. Koltun. Learning to see in the dark. In 2018 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 3291-3300, Los Alamitos, CA, USA, jun 2018. IEEE Computer Society. doi: 10. 10.1109/CVPR.D0 2018..00000004444
URL httpss//doi..
deeecoopptersooieety.. worocitty.org//0.. /00 /wW9//CPR... 2018. 00347
Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16
(8):2080-200) 2090, 2007. doi: 10. 1099TII.2007.901223
Donal J. Denvir and Emer Conroy. Electron Electron- multiplying CCD: the new ICCD. In C. Bruce Johnson, Divyendu Sinha, and Phillip A. Laplante (eds.), Low- Loow-Light-Lee and Real-Time Imaging Systems, Components, and Applications, volume 4796, pp. 164 174. International Society for Optics and
Photonics, SPIE, 2003. doi: 10.7797/12.45779. URL
10. 1117/12.
457779
David Dussault and Paul Hoess. Noise performance comparison of ICCD with CCD and EMCCD cameras. In C. Bruce Johnson, Eustace L. Dereniak, and Robert E. Sampson (eds.). Infrared Systems and Photoelectronic Technology, volume 5563, pp. 195 204. International Society for Optics and Photonics, SPIE, 2004. doi: 10.1117/12.561899, URL https: /oii oc.
1117/1 561839

---

Published as a conference paper at ICLR 2025
Hansen Feng, Lizhi Wang, Yuzhi Wang, Haoqiang Fan, and Hua Huang. Learnability enhancement for low-light raw image denoising: A data perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1))370-387, 2024. doi: 10.100/TPAMM222333305522.
Jason J. Han, Andrew P. Shreve, and James H. Werner. Super-Resolution Optical Microscopy, pp. 1-15. John Wiley & Sons, Ltd, 2012. ISBN 9780471266669.
doi: https://doiico org// 10.1002/04712666633 com. com128. htt.httss://ont wwlllinelibeaae wi ley. com do /
10. www.1002047726668 comll
Michael Hirsch, Richard J. Wareham, Marisa L. Martin-Fernanne Michael P. Hobson, and Daniel J. Rolfe. stochastic model for electron multiplication charge-coupled devices from theory to practice. PLOS ONE, 801):1-13, 01 2013. doi: 10. poou 0053367. URL
https://doi http:///ooo..gg....111 wtt.or///....///rrrrrrrr...
pone. 0053671
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances Neu- ral Information Processing Systems, volume 33, pp. 6840-6851. Curran Associates, Inc.,
ftseppprrrlees/ppe////000 2020. URL httts:://proceednngg..ceo neuripss.....uno.oo.o....rriissm http:///....aoro.eoo//////oeeeea Eile/ http://cccceeeeeaaaaaa1111111110
Alexander Hopt and Erwin Neher. Highly nonlinear photodamage in two-photon fluorescence
microscopy. Biophysical Journal, 80(4):2029 2036, 2001. ISSN 0006-3495. doi: https:
URL https::t////.. /www. www.sscienced..e rect. com/
sci scciieee///rrrcc...p/////0015111
Jaroslav Icha, Michael Weber, Jennifer C Waters, and Caren Norden. Phototoxicity in live fluores-
cence microscopy, and how to avoid it. Bioessays, 39(8), August 2017.
Junho Jeong, Changhyun Jung, Taehyun Kim, and Dongil "Dan" Cho. Using machine learning to
improve multi-qubit state discrimination of trapped ions from uncertain emced measurements. Oct 2023, doi: 10.1364/0E.49130. htttp::::///p
Opt. Express, 31(21):351 optica wrt/ http//bbstracccc www.:ei=ceememeeeeeee 31-
Xiang Ji, Zhixiang Wang, Shin'ich Satoh, and Yinqiang Zheng. Single Image Deblurring with Row-dependent Blur Magnitude. 2023 IEEE/CVF International Conference on Computer Vi- sion (ICCV), pp. 12235-12246, Los Alamitos, CA, USA, October 2023a. IEEE Computer Society. URL
doi: 10.1 10110099CCCV51000.200000222
org 1ttp///.1111///111110000
01127
Xiang Ji, Zhixiang Wang, Zhihang Zhong, and Yinqiang Zheng. Rethinking Video Frame Interpolation from Shutter Mode Induced Degradation In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12225- 12234, Los Alamitos, CA, USA, Octo- ber 2023b. IEEE Computer Society. doi: 10.1 110110//CCVV500002200000111 //doi. lew.ecompterrrcccettt.or.//////.
URL https: 2023. 01126
Xiang Ji, Haiyang Jiang, and Yinqiang Zheng. Motion Blur Decomposition with Cross-shutter Guidance In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12534-12543, Los Alamitos, CA. USA, June 2024. IEEE Computer Society, doi: 10. 10.1109CCVP3233..224401999 URL https://dol. iee.compttesssscceet.... org/ N0.1109/CVVRR2115 2024. 01191
Xin Jin, Jia- Wen Xiao, Ling-Hao Han, Chunle Guo, Ruixun Zhang, Xialei Liu, and Chongyi Li. Lighting every darkness in two pairs: A calibration-fres pipeline for raw denoising. Proceed- ings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 13275-13284, October 2023.
Barak Kagan, Adi Hendler-Neumark, Verena Wulf, Dotan Kamber, Roni Ehrlich, and Gili Bisker. Superiresolution near-infrared fluorescence microscopy of single-walled carbon nanotubes using deep learning. Advanced Photonics Research, 3(11):2000444 2022. doi: https://doi.or//00..0 adpr.202200244. URL http::s:/cniinenlirrrrre//c wiley. com/ haaam//cc/b//s///// 1002/
adpr. 202200244
12

---

Published as a conference paper at ICLR 2025
Ashok Kumar, Hayden Nunley, and A. M. Marino. Observation of spatial quantum correlations in the macroscopic regime. Phys. Rev. A, 95:053849, May 2017. doi: 10.1105/PhysRevA.0 053849. /RLttt/s:///in
http:////.........///e..eeree. 95. 053849
Antony Lee, Konstantinos Tsekouras, Christopher Calderon, Carlos Bustamante, and Steve Pressé. Unraveling the thousand word picture: An introduction super-resolution data analysis. Chem- ical Reviews, 117(11):7276-733) 7330, Jun 2017. ISSN 0009-2665. doi: 10.1022/acs.chmme org/ 1ee1c111aae chemrev 6b00729
6b00729. URL
Xinyang Li, Xiaowan Hu, Xingye Chen, Jiaqi Fan, Zhifeng Zhao, Jiamin Wu, Haoqian Wang, and Qionghai Dai. Spatial redundancy transformer for self-super fluorescence image denoising. Nature Computational Science, 3(12): Dec 2023. ISSN 2662-8457. doi: 10. 1-3588-023-00568-2
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Im- age restoration using swin transformer. In 202/ IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp. 1833-1844, Oct 2021. doi: (0.1009/CCVW544220.222
.00210.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2017. URL http://pppp..eaaattcccccco......
Corpus ID: 11::533999 53592270
U. Mallik, P. Petrone, and D. J. Benford. Maturing electron multiplying charge coupled device photon-counting with variable multiplication gain imaging for coronagraphic instrument. J Astron Telesc Instrum Sysz, 5(4), Oct 2019.
Meda, E Losero, Samantaray, F Scafirimuto, S Pradyumna, A Avella, I Ruo-Berchera, and M Genovese. nhoben-number correlation for quantum enhanced imaging and sensing. Journal of Optics, 1909)09400,, aug 2017, doi: 10. (0088/2040-996//a7777 org/10. org//0.1008/ 100000000/0000000000 8986/aa 7b27]
URL htttps:://///...c //dp. doi.
Kristina Monakhova, Stephan R. Richter, Laura Waller, and Vladlen Koltun. Dancing under the stars:
Video denoising in starlight. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16241-16251, June 2022.
Junichi Nakamura. Image Sensors and Signal Processing for Digital Still Cameras. CRC Press, Inc., USA, 2005. ISBN 0849335450.
T. Plakhotnik, A. Chennu, and A.V. Zvyagin. Statistics of single-electron signals in electron- multiplying charge-coupled devices. IEEE Transactions on Electron Devices, (5(4):618622) April 2006. ISSN 1557-9646. doi: 10. 11100/0D
Li Qiao, Mingfu Wang. and Zheng Jin. A method for emeed multiplication gain measure- ment with comprehensive correction. 2045-2322.
Scientific Reports, 11(1):6058, Mar 2021. ISSN
doi: 10. 00388441598-001-85511111 841598- 021-881121-85111-
URL https: //tpss:://d...o org / 10. 1038/
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Nassir Navab, Joachim Homngger, William M. Wells, and Alejan dro F. Frangi (eds.), Medical Image Computing and Computer-Assisted Intervention MICCAI 2015, pp. 234-241, Cham, 2015. Springer International Publishing. ISBN 978-3-319-2457444
Duncan P. Ryan, Megan K. Dunlap, Martin P. Gelfand, James H. Werner, Alan K. Van Orden, and Peter M. Goodwin. A gain series method for accurate emeed calibration. Scientific Reports, 1134) Sep 2021. ISSN 2045-2322. doi: 1038//4 10988//41598--0 5998-021-9775--66 URL https:
1038/ s41598 021007777777 97759-
Robert Tubbs. Lucky exposures: diffraction limited astronomical imaging through the atmosphere. PhD thesis, University of Cambridge, 2003.
Chi Wang, Pandi Li, Siyuan Li, and Zhanjun Lu. Application of EMCCD in the small-optical fiber endoscope imaging detection system. In Chao Zuo, Shijie Feng, Haixia Wang, and Qian Kemao (eds.), International Conference Optical and Photonic Engineering (icOPEN 2022), volume 12550, pp. 125500T. International Society for Optics and Photonics, SPIE. 2023. doi:
URL Wtt.l.tttss//.... //dopa org /tg/110. 100.1117/10 00011722226666667 2666629
13

---

Published as a conference paper at ICLR 2025
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 17683-17693, June 2022.
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004. doi: 10.1109/T1P.2003.88990
Kaixuan Wei, Ying Fu, Yinqiang Zheng, and Jiaolong Yang. Physics-based noise modeling for extreme low-light photography. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8520-8555 8537, 2021.
Zhidong Yang. Dawei Zang, Hongjia Li, Zhao Zhang. Fa Zhang, and Renmin Han. Self-supervised noise modeling and sparsity guided electron tomography volumetric image denoising. Ul- tramicroscopy, 255:1 13860, 2024. ISSN 0304-3991. doi: htps://doi.orr/1001116,.,,..ii
2023.113860. URL https: www.eecceceee...rrrecccccccc htt.://aw.....coo/ceeeeeeeeerrrr 50304399123001778
Hiroaki Yokota, Atsuhito Fukasawa, Minako Hirano, and Toru Ide. Low-light photodetectors for fluorescence microscopy. Applied Sciences, 11(6), 2021. ISSN 2076-3417. doi: 10.3390/ app11062773. htthstt::///
www. mdppi. 20tp//076---0 -00mmm20006111////211777
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5718- 5729, June 2022. doi: 10.1 10.11099/VR55268.22000 2022.00664.
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as perceptual metric. In 2018 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 586-595, Los Alamitos, CA, USA, jun 2018. IEEE Computer Society, doi: 10. 109.10/9./....0...... 200600 URL attt::://ab...eeeccmueeeeeees
org/10.. 1tt//00.1109/CVER... 2011 00068
Wenwen Zhang and Qian Chen. Signal-to-noise ratio performance comparison of electron multiply- ing ccd and intensified ced detectors. In 2009 International Conference on Image Analysis and Signal Processing, pp. 337-341, April 2009, doi: 10.1009/AASP200955555555..
A CALIBRATION DETAILS
A.1 FLAT-FIELDD FRAMES
In ELD (ei et al. 202I), a mean-variance method adopted for estimating K, leveraging follow- ing correlation based on property of Poisson distributions:
Var(D) = K² N₂) + Var(N.)
= Var
=K(KT)+ (Nr(N))
= K x Me)n(D) + Var(N.).
No represents all noise components other than shot noise combined. The proportional relation be- tween variance and mean value of flat-field frames demonstrates that linear regression could yield K. In EMCCD's case, we substitute N₂ in equation 7 with multiplicative noise N, /. updating the final linear relationship to be Var(D) = K f2 x M Man((D + VNr(NN).. i.e., the slope coefficient of regression is K = 2K. In practice, flat-field frames limited a 256 256 crop the center to avoid being affected by lens vignetting. Median, instead of mean value, is utilized to calculate independent variable in linear regression
to increase robustness of the results.
Shown in figure 10 is a comparison between estimated system overall gain K by mean-variance method and the gain-series approach from Ryan al. (202)) on flat-field frames. As mentioned
14

---

Published as a conference paper at ICLR 2025
Calibrated gain values by Mean Variance method and gain series method
MV
10
gain series
6
0
50
100
150
200
250
300
EM gain
Figure 10: System overall gain K estimated by MV and gain series method.
in the main text, the estimates of K from both methods are fairly close; however, the gain series approach exhibits better linearity with respect to the EM gains. Therefore, we adopt from the gain series for data synthesis.
A.2 BLOOMING FACTOR
We started with hypothesis that the essence of blooming is adding amount that is proportional column mean value to pixels in that column. Borrowing notations declared in the main text and figure 5 for blooming pixels can have:
1/2
C..
(8)
with b representing dark background bias being column mean without blooming, and the blooming factor. Nevertheless, real column mean value without blooming is not directly obtainable from the calibration image, as the observable column mean from the image, denoted as c1, is also increased by the blooming factor æ, meaning that C1 = (1 + x) c. Therefore, for bright and dark pixels:
C1
= b +
(9)
1+
U3 b + br.
(10)
Combining the above two equations, we can have
(11)
12
calculating the blooming factor solely from values obtainable in the calibration frame.
In practice, we empirically use window size of 32 pixels to select sub-regions for v2, 0g. and e calculation, all of which determined by sliding-window search for sub-regions with minimum standard deviation among their pixels, in order to avoid any possible spatial heterogeneity. Result estimations of the factor are shown in figure II] matching our experiment results that images taken with shorter exposure have stronger blooming effect.
Evidence of the validity of estimating blooming factor and remove such effect is included in our main text and in figure O
Despite high photon absorption rate dark surfaces in our dark room, there is still chance for photons to hounce off and fall into the image sensor, creating responses above dark current bias.
15

---

Published as a conference paper at ICLR 2025
blooming factor
fit curve
1.4
1.2
blooming factor
0.8
0.6
0.4
K=-0,9621
0.2
0.00
0.02
0.04
0.06
0.08
0.10
exposure/s
Estimated blooming factors under each exposure time. Correlations between the two are close to hyperbolic curve. A log-linear regression yields conforming result with exponential coefficient being -0.9621 (approximating -1 for reciprocal function) and an r value of 0.9996.
Figure 11:
B EXPERIMENT DETAILS
B. TRAINING PROCESS
Training process for all experiments have the same hyperparameters, using AdamW opti- mizer (Loshchilov & Hutter 2017) with initial learning rate as 2 10 and epochs for warmup training. Batch size is set to 4, training patch size to 512, number of total epochs to 1000. The rest parameters for Uformer each training process costs approximately 25 hours.
maintained their default values. Trained with 2 RTX-3090 GPUs,
B.2 TESTSET AND FINETUNING
Example pictures of our testset are shown in figure 12 Examples clean microscopic images used for finetuning process are displayed in figure 13 The finetining proceeds for another 1000 epochs synthetic noisy-clean image pairs from this dataset, continuing from previously trained checkpoints with all other hyperparameters unchanged and taking roughly 6 hours the same hardware as before.
C ADDITIONAL EXPERIMENT RESULTS
We include here enlarged qualitative results from figure g and figure 9 respectively in the follow- ing. Figure 14 shows zoomed-in areas macroscopic testset results, displaying the clearest edges around characters and shapes in our results, and at the same time with minimum remaining noise, least of noticeable artifacts.
In figure 15
apart from eliminating blooming effects, our results can preserve natural transitions between high and low contrast areas, balancing noise remnant and sharpness preservation while not introducing unwanted artifacts.
16

---

Published as a conference paper at ICLR 2025
Sample images from our collected testset. The first row is ground truth clean images, acquired by long exposure and averaging across multiple frames. The second row input noisy images, acquired by short exposure at the same position of corresponding GT.
Figure 12:
Figure 13: Examples of clean microscopic images, imaging cytoplasm, endoplasmic reticulum, mitochondria, and nucleus.
Input
BM3D
ELD
Theoretical
SRDTrans
Ours
GT
NOOR
WOOK
NDOR
NDOR
NDOR
NDOR
NDOR
ECHNOLOGY ODDY
cB
Edmi
Edmm
CD
Edm
CD
Edm
CD Edm
eO Edm
eo
Edm
Figure 14: Enlarged qualitative results figure from our macroscopic testset. Notice the clarity
of alphabets' edges produced by our results compared to others.
17

---

Published as a conference paper at ICLR 2025
Input
BM3D
ELD
Theoretical
SRDTrans
Ours
Finetuned
Figure 15: Enlarged qualitative results of figure from fluorescence microscopic testset. Notice
more natural transitions from high contrast areas to low ones and less artifact in our results.
18


Reviews:
[{"review_id": "ns6eFh9wSU", "reviewer": "Reviewer_e1uJ", "summary": "This paper proposes a physics-based noise model for EMCCD cameras. The statistical model includes some typical noise components for EMCCD sensors, and a calibration method is proposed for adaptation this noise model on each sensor. Through careful noise modeling and calibration, the authors synthesize realistic EMCCD noise data for training, and effectively improve the learning of deep denoisiers in both macroscopic testset and microscopic testset.", "questions": "Why ELD presents banding patterns in Fig. 7?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper introduces the first EMCCD denoising method utilizing physics-based noise modeling method.\n- The overall writing of this paper is good and easy to follow.", "weaknesses": "- This paper proposes the first noise modeling method for EMCCD sensors, and there are indeed some new adaptations on this sensor type. However, the main idea borrows many contributions from the similar task of CMOS noise modeling, and seems to be a EMCCD-version of ELD [1]. Specifically, the entire pipeline, i.e., physics-based noise modeling ->  calibration -> synthesis -> denoise pipeline is the same with ELD. The noise components and calibration process are also similar with ELD. In addition, the modeling of FPN and pre-processing operation comes from PMN [2] .\n- For Fig. 7, why ELD presents banding patterns, even after calibration using the target device? ELD calibrates row noise using bias frames, and the variance for row noise would be close to zero on sensors without obvious banding patterns if correctly calibrated. I wonder why ELD still causes such row patterns on EMCCD sensors.\n- There should be more comparisons with sota methods, for both noise modeling and self-supervised denoising methods. For example, [3] proposes a general noise modeling method which uses poisson sampling for signal-dependent noise and GAN for signal-independent noise. I think [3] can also handle EMCCD sensors. Stronger baselines for self-supervised methods are also recommended to compare [4].\n- I concern that it is not rigorous to use SID clean images to synthesize noisy pairs for training. Different from EMCCD sensors, SID dataset uses Sony cameras with CMOS sensors. Each sensor type has its own unique recipe for generating RAW data; even using clean images from one type of CMOS sensor to generate synthetic noisy pairs and then testing on real data from a different CMOS sensor can lead to negative effects, not to mention EMCCD data. Therefore, I believe that SID clean data is not a suitable choice for this application.\n- Section 2.3 is not necessary since no deep denoiser architecture is proposed.\n\n\n[1] Physics-based Noise Modeling for Extreme Low-light Photography. TPAMI 2021\n[2] Learnability Enhancement for Low-Light Raw Image Denoising- A Data Perspective. TPAMI 2023\n[3] Towards General Low-Light Raw Noise Synthesis and Modeling. ICCV 2023\n[4] Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios. CVPR 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HVSK9JarKG", "reviewer": "Reviewer_M8ik", "summary": "This paper focuses on EMCCD noisy data and denoising. The authors propose a physics-based noise model specifically for EMCCD cameras, which generates synthetic noisy images based on both the camera's properties and EMCCD-specific noise characteristics. This method makes the data clorser to real-life scenarios. They then train a deep learning model, Uformer, on these noisy image pairs for denoising. The Uformer model achieves better denoising results comparing to other methods.", "questions": "1. Could you provide the dimensional details of the variables in the key equations listed in the paper? It would help in understanding if you state that 'X' represents the inner product in Eq. (1).\n2. It seems that adding N_r and N_q makes the image blurry. Could you visualize both N_r and N_q?\n3. I feel that the proposed noise addition might be similar to the negative binomial low-photon noise. Could you explain the key differences between them?\n4. In line 076, could you elaborate on the differences between the EMCCD and other models, if possible?\n5. Could you provide a big-map plot or additional explanation of your Uformer model? What is the novel design aspect of this denoising model, and how does it differ from Wang's model?\n6. Did you use any method to measure whether the results indicate overfitting? Will using data augmentation techniques to generate more data improve the model's accuracy? Perhaps training the model on simulated data and testing it on the original true data could be a way to assess the quality of the simulated data.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Proposed the first dataset specific to EMCCD.\n2. Provides a detailed and clear explanation of the noise model, including settings and parameter estimation.\n3. The experiments compare the proposed method to other state-of-the-art methods.", "weaknesses": "1. For important equations, such as Eq. (1) and (5), the dimensions of each parameter are not provided, especially for N_p, f, and I.\n2. The denoising model, Uformer, should be discussed more thoroughly, with additional details explaining its design, such as the key differences compared to the Uformer model from Wang et al., 2024.\n3. The total number of image pairs is 224, which is relatively small, and the use of only 24 images for fine-tuning could lead to overfitting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CZxC3XKEYB", "reviewer": "Reviewer_V6zx", "summary": "This paper presents a novel approach to denoising images captured by electron-multiplying charge-coupled devices (EMCCDs) by introducing a physics-based noise model and a calibration procedure tailored for EMCCD-specific noise characteristics. The proposed method synthesizes authentic training data for a deep learning framework, enhancing denoising performance in fluorescence microscopy and achieving state-of-the-art results compared to existing methods. Additionally, they establish a comprehensive pipeline that connects noise parameter calibration with advanced neural network training strategies. This work paves the way for improved image quality in sensitive imaging applications across various scientific fields.", "questions": "### Original Question\nThe relationship between the ablation study and the proposed method in this paper is unclear. \n\nAs it stands, I find it difficult to directly correlate the FPNt, blooming effect, and readout noise heatmap with the ablation learning presented. Additionally, the current preprocessing appears to resemble contributions from PMN rather than from this work. \n\nI suggest clarifying the incremental contributions of the proposed method in the experiments to emphasize the original contributions of this paper.\n\n### After Rebuttal\nThe authors have addressed my concerns.\n\nI found the authors’ response to reviewer e1uJ very well-written. This work stands out because it takes a practical, problem-specific approach, using appropriate innovations to solve a real-world task. Looking at recent noise modeling research, I consider LLD [1] and PNNP [2] to be practical, while LRD [3] seems less so. LRD faces challenges with the data dependency problem, whether **paired data or noise models come first**, and the instability of the GAN-based training strategy. As a result, GAN-based noise modeling methods like LRD, CA-GAN, and Starlight often overfit the training data.  \nFor example, LRD, which includes a **Fournier Transformer Discriminator**, leaves visible row patterns in *Scene-07_IMG-0010* of the ELD dataset that are more noticeable than those in the ELD baseline, even though LRD achieves higher PSNR and SSIM scores. For this reason, I believe reviewer e1uJ’s initial rejection of this work was not well-founded.\n\nIn conclusion, I acknowledge the contributions of this paper and am inclined to keep my current rating.\n\n**Reference**  \n[1] Y. Cao, M. Liu, S. Liu, X. Wang, L. Lei, and W. Zuo, ‘Physics-Guided ISO-Dependent Sensor Noise Modeling for Extreme Low-Light Photography’, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 5744–5753.  \n[2] H. Feng, L. Wang, Y. Huang, Y. Wang, L. Zhu, and H. Huang, ‘Physics-guided Noise Neural Proxy for Practical Low-light Raw Image Denoising’, arXiv preprint arXiv:2310. 09126, 2023.  \n[3] F. Zhang et al., ‘Towards General Low-Light Raw Noise Synthesis and Modeling’, in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 10820–10830.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The introduction and the method of this paper are clear and easy to understand. Even readers who may not be familiar with EMCCD can grasp the motivation behind the noise model.\n\n2. The novelty of this work is commendable. While many key designs are inspired by existing research, they incorporate unique adjustments specific to the characteristics of EMCCD sensors. The analysis of FPN, blooming effects, and readout noise heatmaps is particularly impressive.\n\n3. The experiments presented in this paper are excellent, and I believe they will significantly contribute to sensitive imaging applications across various scientific fields.", "weaknesses": "1. In Eq. (5), D' includes $N_r$ and $N_q$; however, this seems unreasonable from a formulaic perspective. I suggest explaining why $B^{-1}$ doesn't affect $N_r$ and $N_q$. For instance, it might be beneficial to analyze the expected interactions between these two components.\n\n2. Figure 3(b) appears to exhibit some abrupt transition points (e.g., log(time) = -7, -4), and the explanation provided in L252-255 seems insufficient to cover this phenomenon. Please confirm the reproducibility of these data and clarify why an S-shaped curve is used instead of multiple piecewise functions. If these transition points are related to circuit switching, a piecewise function fitting, similar to what has been reported in PMN, should be employed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)