Optimal transport unlocks end-to-end learning for single-molecule localization
Authors: Romain Seailles
Jean Baptiste Masson
Jean Ponce
Julien Mairal
Venue: ICLR 2026 Poster (2026)
Keywords: Single Molecule Localization Microscopy
SMLM
High-Density
Learning
Deep Learning
Inverse Problems
Iterative Refinement
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We present an end-to-end learning model for single-molecule localization microscopy that combines an optimal transport loss function with an iterative neural network. It achieves state-of-the-art performance on synthetic benchmarks and real images.

Abstract: Single‑molecule localization microscopy (SMLM) allows reconstructing cellular organelles and biology-relevant structures far beyond the limited spatial resolution imposed by optics constrains, using tagged biomolecule positions. Currently, efficient SMLM requires non‑overlapping emitting fluorophores, to ensure proper image deconvolution leading to long acquisition times that hinders live‑cell imaging. Recent deep‑learning approaches can handle denser emissions, but they rely on variants of non‑maximum suppression (NMS) layers, which are unfortunately non‑differentiable and may discard true positives with their local fusion strategy. 
In this presentation, we reformulate the SMLM training objective as a set‑matching problem, deriving an optimal‑transport loss that eliminates the need for NMS during inference and enables end‑to‑end training. 
Additionally, we propose an iterative neural network that integrates knowledge of the microscope’s optical system inside our model. Experiments on synthetic benchmarks and real biological data show that both our new loss function and architecture surpass the state of the art at moderate and high emitter densities. Code is available at https://github.com/RSLLES/SHOT.

Published as a conference paper at ICLR 2026
OPTIMAL TRANSPORT UNLOCKS END-TO-END LEARN- ING FOR SINGLE MOLECLLE LOCALIZATION
Romain Séailles*
Département d' l'informatiquee, École normale supérieure (ENS-PSL), Paris, France
Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, Grenoble, France
Jean-Baptiste Masson
Decision and Bayesian Computation, Institut Pasteur, Université Paris Cité, CNRS, Inria (Epiméthée), Paris, France
Jean Ponce
Département d' d'informatique, École normale supérieure (ENS-PSL), Paris, France Courant Institute School of Mathematics, Computing, and Data Science, New York University (NYU), New York, NY, USA
Julien Mairal
Universite Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, Grenoble, France
ABSTRACT
Single-molecule localization microscopy (SMLM) allows reconstructing biology- relevant structures beyond the diffraction limit by detecting and localizing individ- ual fluorophores fluorescent molecules stained onto the observed specimen over time to reconstruct super-resolved images. Currently, efficient SMLM requires non-overlapping emitting fluorophores, leading long acquisition times that hinder live-cell imaging. Recent deep-learning approaches can handle denser emissions, but they rely on variants of non- mon-maximum suppression (NMS) layers, which are unfortunately non-differentiable and may discard true positives with their local fusion strategy. In this presentation, we reformulate the SMLM training objective set-matching problem, deriving optimal-transport need for NMS during inference and enables end-to-end training. Additionally, we propose iterative neural network that integrates knowledge of the microscope's optical system inside our model. Experiments on synthetic benchmarks and real loss function and architecture surpass
transport loss that eliminates the
biological data show that both the state of the art at moderate and high emitter densities. Code is available at https:: / com ROOOL//SSLESSHTT
INTRODUCTION
Fluorescence microscopy remains a cornerstone tool of biological research, recording photon emis- sions from fluorophores (fluorescent molecules) stained onto a specimen to characterize its structure. However, light diffraction restricts the final image resolution to approximately half the wavelength of light, preventing analysis of structures or organelles feature smaller than ~ 200 nm in practice (Mc- cutchen 1967, Schermelleh al. 2019).
Multiple experimental techniques have been developed to surpass the diffraction limit (Hell & Wichmann 1994 Gustafsson [000 Dertinger al. 2009 Laine et al.. 2023). collectively described as super-resolution microscopy methods. Among them, single molecule localization microscopy (SMLM) takes advantage of the stochastic flickering of fluorophores over a long sequence of im- ages Betzzig et al. 1991). Compared to conventional fluorescence microscopy, the laser power is
"Corresponding author: contact romat nsea fr

---

Published as a conference paper at ICLR 2026
Sparse activations over multiple frames
Conventional wide-field image
Super-resolved 3D point cloud
Figure 1: Illustration of the SMLM principle using our method. Data Feii et al. 2025) show Nup96 in human bone cancer (U2-OS) cells. (1) A conventional wide-field microscope would record an image with limit resolution of 200 nm. (2) Instead, SMLM captures many frames where only sparse subset of fluorophores actively emit in each onc. These can be detected and localized with sub-pixel precision. (3) The union of all detections is rendered 3D point cloud (color encodes depth), producing a super-resolved representation the specimen.
tuned to achieve a low density of simultaneously active fluorophores such that, with high probability, no two emitters occupy the same diffraction-limitee area at the same time (Lelek al. 2021). As modelling light propagation for point emitters in the microscope can be approximated, each emitter pixel pattern be deconvolved into point localization with positionning error massively inferior to the diffraction limit. Accumulating detections across all frames yields point cloud representation of the underlying specimen (Rust al. (006), effectively achieving super-resolution. Figure [illustrates this method. For additional details, on both experimental methods and deconvolution approaches see the review by Lelek et (2021).
However, the low-density constraint inherent to this approach limits the number of active fluorophores that can be captured in a single frame, requiring thousands of frames to reconstruct complete specimen, which hinders live-cell imaging and the observation of dynamic processes (Heilemann jet al. 2008). Consequently, high-density setups are desirable, overlapping fluorophores within the same diffraction-limites area usually lead to uncertainties in the number of fluorophores and reduced spatial resolution, yielding deteriorated reconstruction.
Deep learning methods have shown success at handling higher densities. Top methods (Speiser et al., 2021 Fei et al., 2025) predict detection map trained with pixel-wise objectives, and decide at inference whether a candidate exists not by binarizing their map using variant of non-man non-maximum suppression (NMS) (Girshick et al. 2014). This spurious local maxima while (ii) not merging nearby emitters. We three main issues with this framework. (1) These pixel-wias loss functions do not account for multiple emitters within the same pixel. (2) Objectives
uses two thresholds to (i) suppress
and (ii) are inherently in conflict, and this issue only worsens density increases, where the probability of multiple emitters activating simultancously at sub-pixel distances rises. (3) The precision-recal tradeoff is difficult tune due the two required hand-set thresholds. Figure 6 in the Appendix illustrates problems (1) and (2).
In this paper, we frame the SMLM training objective based on one-to-one matching between predicted and true emitters, using new loss function constructed from optimal transport (Peyré et al. 2019). and solve the decision problem at inference with a simple individual one-threshold filtering. These changes solve problem (1) by removing pixel-wise assignments in the training objective, problem (2) by removing decision pipelines based on spatial proximity like NMS, and problem (3) by using a single threshold during filtering, which directly controls the precision-recall tradeoff. Furthermore, NMS non non--ifferentiabiliyy prevents the model from optimizing for it: discarding it allows us to benefit from the flexibility of deep neural networks at the final model layer, unlocking end-to-end learning. Additionally, inspired by the success of iterative refinement networks for optical flow estimation (Teed Hur & Roth 2019) propose novel iterative neural network architecture that
& Deng 2020
2

---

Published as a conference paper at ICLR 2026
leverages reconstruction of the expected frame given the current estimated set of fluorophores, introducing knowledge of the microscope 's optic system into the model. We demonstrate that both our loss function and architecture choices improve the state of the art at both low- and high-density regimes on synthetic benchmarks and real data.
2 RELATED WORK
Single-molecule localization microscopy. SMLM has been enabled by the development of pho- toactivable and photoswitchable fluorophores, which allows individual molecules to emit efficiently and in a controllable manner sufficient amount of photons to be individually located (Betzig et al.) 2006; Hess et al. 2006). Early tools perform detection by locating local maxima and localizing with Gaussian estimation of the point spread function (PSF) (Patterson al. 2010; Rust al. 2006), assuming simplified light propagation in the microscope. Shortly after, the introduction of asymmetry along the z-axis of the PSF enabled 3D localization (Huang et al. 2008); the most common setup is to introduce astigmatism in the microscope optics, which we employ in this work. To improve localization accuracy, PSF models have transitioned from being only theory-derived experimentally augmented, in order to incorporate effects of real light propagation in the microscope and unmodelled effect of light propagation in the cell (Babcock et al. 2012). This usually requires a pre-calibration step using specially designed fluorescent beads, which are imaged to capture how single point of light appears at different locations. Note that while this calibration phase can be resource- and time-consuming, recent works propose live estimation of the PSF (Liu et al. 2024). 3D-DAOSTORM (Babcock et al. 2012) is a widely used classical method that uses experimentally derived PSFs, and use it as baseline in our comparisons.
SMLM can deliver high resolution (10 - 20 nm) in optimized conditions and low phototoxicity at the cost of slow acquisition speed (Lelek al. 202T SIM (Gustafsson 2000), SOFI (Dertinger et al. 2009), and eSRRF (Laine et al. (023) trade speed for resolution, while STED (Hell & Wichmann
1994) offers faster imaging but higher phototoxicity, MINFLUX (Balzarotti et al. 2017)) provides SMLM-level resolution but lacks a large field of view and requires ultra-stable hardware (Scheiderer
al. 2025). SMLM thus remains an attractive middle ground, making high-density performance a major research focus (Lelek et al. 2021).
Deep-lcarnin methods are widely used in fluorescence microscopy (Nehme et al. 2018 Ouyang ct al. 2018; [oyd et al. 2018 Cachia et al., 2023 Li et al., 2023 Mentagui et al. [024]] Fei et al.
2025). Among these, Deep (Boyd et al. 2018 uses a set formulation with loss function based on maximum mean discrepancy (Gretton et al. 2012). DECODE (Speiser et al.) (021) combines
pixel-wise detection and Gaussian-mixture localization losses, currently leading the EPFL SMLM challenge (Sage et al., 2019). More recently, LiteLoc (Fei et al. (025) refined DECODE's architecture for improved performance. We use both DECODE and LiteLoc as baselines in our benchmarks.
Optimal transport for matching. Optimal transport (Peyré et al. 2019 Villani 202 has
become a popular tool for set matching by deep learning. Recent works in object detection (Carion et al., 2020; Zhu et al., 202TT: Zhang et al. 2023 Li et al.) (022) have demonstrated success in predicting sets of variable and unknown size using bipartite matching loss functions, while other modern works have employed entropic regularization (Cuturi 2013) to achieve fully differentiable pipelines (Zareapoor et al. (202)). By framing SMLM as set matching problem, we draw direct connection to this line of work substituting objects for fluorophores - enabling the design of an end-to-end training procedure.
Iterative refinement network. Iterative refinement within neural networks has proven effective
for tasks that benefit from sequential solution improvement (Carreira et al., 2016 Yu et al. 2023). In computer vision, Putzky & Welling (2017) have applied this approach to inverse problems such as image denoising, super-resolutton, and inpainting, while Hur & Roth (2019) proposed iterative optical-flow refinement using a feedback loop with rewarping operator. As the physics of SMLM well understood (Etheridge et al., (022). we show that an accurate simulator of the microscope's physics can provide similar visual feedback, enabling progressive refinement of the solution.
3

---

Published as a conference paper at ICLR 2026
3 METHOD
3.1 PROBLEM FORMULATION
In this section, first introduce the image formation model for SMLM and formulate the corre- sponding inverse problem as a set matching task. We then present a differentiable loss function and an iterative refinement architecture that explicitly leverages the image formation process.
Image formation model. An activation is defined as an emission event from a fluorophore within a given frame (a single fluorophore may produce several activations across multiple frames). Through- out this work, an activation is represented by 4D vector æ (Ir, y, 2, n), where (x, y) denote the 2D coordinates in the camera frame (with the origin at the top-left corner), & represents the axial coordinate relative to the focal plane, and n is the photon count. Given N activations within a frame, we denote the complete set as X {>: hgig.-
Diffraction within the optical system is modeled by convolution with the point spread function (PSF) (Rossmann) 1969), which represents the image of a single point source. We define the PSF as a function P: g k outputting normalized H W image for point source at given 3D coordinates. To ensure photon count independence, the image is normalized to sum to unity in the focal plane (z 0). For a set of activations X' in a frame, the observed image (X)) is the weighted sum of PSFs, where each weight is the photon count n of the activation:
H(X) =
Σ (u.y,,,))EX'
The dependence of the PSF on depth enables 3D localization of activations from the observed image, see (Ovesný et al. 2014). Following Babcock & Zhuang (2017), we assume that the PSF is pre-calibrated on synthetic fluorescent beads and implemented as a collection of 3D splines. This
approach is standard tool in SMLM used in many works (Ries] 2020 Li et al. 2020 Speiser et al. 2021 Etheridge et al., 2022); see Babcock & Zhuang (2117) for further details.
Noise model. We adopt the noise model of SSage al. (201)), combining shot noise (Poisson distributed), amplification noise (modeled by a Gamma distribution for EM-CCD cameras), and readout noise (normally distributed). Detailed camera parameters are in Appendix A.I As noise is independent and identically distributed for each sensor (Fazel & Wester 2022).. it is applied independently to all pixels of H('). We denote by the distribution of images y produced by fluorophores X' under this model such that
y y~P
(2)
Risk minimization formulation for set matching. As no ground truth is available in most scientific
imaging applications, supervised-learning models for SMLM have be trained with a simulator, which is able to generate realistic from P(X) given sets X' of activations from a distribution D. Our approach consists of training a neural network So which directly predicts set of activations given an observation y, by minimizing the risk
0-
arg min Ex-D (C(Joyy)
dEe
Such formulation raises two major challenges: we need to design a differentiable loss function L and an architecture fo that are appropriate to the context of SMLM.
3.2 OPTIMAL TRANSPORT LOSS FUNCTION
We argue that framing SMLM as a supervised-leamiing problem leads matching formulation, for which optimal transport theory is a natural fit. To the best of our knowledge, however, this framework has not yet been applied SMLM. Figure Pravedess an overview of our method.
Let X' = {æi)ısss be the ground truth set of activations. The size of this set, N, is unknown and varies between frames, but it can be bounded by the physics of the fluorophore and the experimental protocol. We simulate an acquisition
PXX) and aim to retrieve X' from y.
Given y, the network fo outputs a set of d candidate activations d = ( and their corre- sponding detection scores $ (&s E (0. 1 The architecture of fo is detailed in Section 3.3
4

---

Published as a conference paper at ICLR 2026
Ground truths X' on frame y
Predicted set X'
Non-existent
Non-existent
Iterative refinement model
22
(1) Model maps input frame y candidate activations set X.
(2) Loss function is the optimal transport cost between & and X.
Illustration of our loss function for end-to-end training. (1) Given a simulated image and ground truth activations (see Section 3.1), D. our model (see Section 3.3) predicts candidate activations, each with detection score quantifying the plausibility of its existence. (2) We solve a regularized optimal transport problem conceptually similar to a bi-matching between ground truths and predictions over cost involving both localization and detection tasks. Our loss function is the optimal cost yielded by this solution.
Figure 2:
The fixed parameter d defines the maximum number of detectable activations; see Appendix A.2 for an analysis of its impact.
We first define L, squared cost matrix of size d x d, whose components are:
( 20-00 x,, (Ø; zz) + log det (Σ) i 10
VI S ú S d, Lis
otherwise,
where Σ = diag(o? og. of, ohh) is diagonal weighting matrix. Quadratic costs natural and principled choice for regression tasks. Extending this formulation to the negative log- eng-likelihood of multivariate normal distribution allows learn Σ end-to-end, which can be viewed an automatic weighting strategy that balances the difficulty of predicting each dimension, similar to the homoscedastic uncertainty weighting method proposed by Kendall al. (2018). Experimentally. we have found o? to be ~ 2 larger than o² and o after training, which is consistent with the optical theory of confocal microscopy (Pawley 2006).
Similarly, we define D, another d x d cost matrix whose components are:
- log(s;) [- log(1 s:) otherwise.
if S N,
V1 S iis S d, Di.
(5)
The binary cross-entropy cost is natural choice for detection tasks. It favors a high score when é paired with an element X and low score otherwise, hence promoting good detection. Finally, we define the total cost matrix C = L D, which integrates both localization and detection tasks.
For the initial set matching problem, the optimal solution (X**, S²) given the target X' consists of N' elements identical to X' with detection scores near 1, while the other d N elements have scores near 0. Ideally, each candidate in X is compared to its nearest counterpart in X' to minimize a pairwise loss. This is achieved by solving an optimal-trane -trnnpoor problem over C-effectively a bipartite matching between predictions and ground truths-whee the minimal cost aggregates all pairwise contributions. Thus, our ideal loss function the aeiimal-transpor cost with respect to C, solving:
min (T C) where B [T Rdxd 71, - =1].. TEB
(6)
and .) is the Frobenius inner product. However, while the Hungarian algorithm can exactly solve this problem in ((d3) (Kuhn 1955), algorithmic step non-differentiable which prevents end-to-end learning. We circumvent this issue by finding I' through the cntropy-reeullaine optimal transport problem, see (Cuturi 2013), and therefore define our loss function as follows:
(( (I" C) where I" arg min (T' C) - -TH(T)
TeB
5

---

Published as a conference paper at ICLR 2026
K iterations
Encoder
Image formation model
Non-existent
Refinement
X
Encoder
+
Decoder
network
Figure 3: Illustration of our iterative refinement model. Within a classic mecoder-deccder architecture, we leverage prior knowledge about known image formation model (not learned) to simulate expected frame given the current latent representation. This feedback is used to iteratively refine the model's 's inner latent representation for K steps. The encoders are identical. + and ① respectively denote element wise addition and concatenation.
H is the Shannon entropy and the entropic regularization parameter. A good approximation of r" in Eq. (7) can found efficiently with few iterations of the Sinkhorn algorithm, whose steps are differentiable with respect the elements of C, enabling its within deep learning framework, see (Genevay et al. 2018 Mialon et al.) (021). p. A more detailed analysis of the impact of and comparison with the Hungarian algorithm are available Appendix A.4
3.3 ITERATIVE REFINEMENT SCHEME
To solve Eq.
investigate architectures that explicitly leverage the image formation process. To this end, we adopt an iterative architecture, an idea that has proven successful for optical flow estimation (Hur & Roth 2019). At each iteration the network produces of candidate activations, turns those proposals into a simulated image, which then used as feedback to refine the next proposals. This iterative method is illustrated in Figure
Concretely, let be the input frame of size H x W. An encoder E H - HxW maps to latent representation (0), where C' a hyperparameter controlling the dimension of the latent space. decoder D then maps latent variables to a set of candidate activations & { and corresponding set of detection scores $ {
Given (Y, compute reconstrueted frame E As the expected image from the current proposal set, ý summarizes what the model explains in the SMLM frame. Comparing ÿ to the original frame supplies informative feedback, which helps the model correct errors and refine the candidates set over iterations.
CxH which produces a residual update of the latent representation given its current estimate, the representation of the simulated frame, and the encoded original frame. Algorithm shows how this proposal updated successively over steps. During training, the final decoded output ( ((K) used input for our loss function, see Section 3.2 Details about the decoder architecture and the computation of given X and å can be found in Appendix A.2
Concretely, define iterative refinement operator R 33
EXPERIMENTS
Implementation and training details. We construct a synthetic target activations set X' from D by uniformly sampling between 10 and 30 activations per frame, assigning each activation independent coordinates that are uniformly distributed across all dimensions. This guarantees that the network cannot learn any specific prior about the activation distribution.

---

Published as a conference paper at ICLR 2026
Algorithm 1 Iterative refinement architecture
Require: input frame E HRW encoder E, decoder D, refinement module R, camera model F()), number of iterations K E
Ensure: final proposals (
20)
E
encode original frame > decode initial proposals
2: (28(0) ((0)) .5(0)== < D (0)
3: for k = to K
do
4:
((()) + (((((((),, (
b. simulate reconstruction from current proposals
5:
2(m] t ())
> encode reconstruction
6:
z(++1] <- 2(8) 4 RR((()) 2(0, )0))
> refine latent iteratively
7:
(((k+)),((A+))) x D (22(k+1))
decode refined proposals
8: end for
9: return (g(K),8(k))
Following (Speiser et al. 2021), we augment y by the previous and the next frame into a tensor ÿ of size 3 H x W. Including these provides additional context, without introducing too complex prior about the physics of the fluorophore, about the frame of interest, and yields improved performance. We also randomly scaled each camera parameters by a coefficient e?, where p ~ (0, 0.03): this data-augmentation "trick" increases the model robustness to experimental complexities in fully controlling and characterizing experimental parameters.
The encoder E is two-layer U-Net (Ronneberger et al. 2015) with SiLU activations (Hendrycks & Gimpel 2016), and LayerNorm (Ba et | layers, with a 48- channel internal width. maps the input to latent image with C = 96 channels. The iterative refinement stage uses similar two-layer U-Net. For the decoder D, rather than the vision transformers (Dosovitskiy et al.] (202T) typical of
DETR-like detectors (Carion et al., 2020), we found light CNN yields better performance (see
Appendix A.2 The resulting network predicts d = 1 HW/4 / candidates with ~ 3 million parameters. As performance gains plateau after three iterations, we use K for all experiments.
For training, we use AdamW (Loshchilov & Hutter 2019) for 100,000 steps with a batch size of 128 on a NVIDIA-HIOO gpu, taking approximately 20h. The iterative architecture incurs higher computational burden than single-pass models like DECODE or LiteLoc; further details about our model computational footprint for training and inference are available Appendix A.3
Inference and detection-localizatio trade-off. During inference, we only retain candidate acti- vations from {() whose associated detection scores in ((K) exceed a user-defined threshold r in [0, 1]. This simple filtering strategy makes an easy lever to control the precision-reced tradc-off: discards all. By contrast, DECODE and LiteLoc use a two-threshold variant of a NMS strategy (Speiser et al. 2021 Fei et al. 2025) that may be harder to tune and harder adapt to changing dynamics during the recording.
= keeps every candidate while
For the default r. we propose to maximize the EaD metric (Section 4 on separate synthetic dataset from our simulator. This yields a threshold with the same localizonn trade-off as the EPFL challenge (Sage et al., 2019). To avoid biasing Table 1, we similarly optimized DECODE's 's and LiteLoc's 's NMS parameters. For experimental data, hyperparameters can be tuned to match specific settings, ensuring consistent precision over long recordings.
Synthetic data. Because no ground-truth annotations exist for real SMLM acquisitions, we have performed the initial evaluation on the open synthetic datasets provided by [Sage et al. (2019) on the 2016 EPFL. challenge, and have adopted their set of metrics.
To evaluate candidate activations truths and predicted activations. A prediction is considered a true positive (TP) if it lies within +50 nm in both x and y directions, and +500 nm relative to its matched ground-truth (both thresholds come from the EPFL challenge). Otherwise, predictions (resp. ground truths) are labeled as false positives (resp. false negatives). Detection performance is quantified by computing precision. recall and Jaccard Index (area under the curve is not commonly employed in this field). Localization performance is evaluated by computing the root-mean-squeh error (RMSE) for TPs, for the lateral plane (RMSF the axial dimension (RMSF.g),, and all three dimensions together (RMSF.a). A
frame, we first solve a Hungarian assignment between ground-
7

---

Published as a conference paper at ICLR 2026
Table 1:
Comparative evaluation of SMLM algorithms the EPFL 2016 challenge datasets and metrics. Densities are expressed in activations/ /um/ um/ffamee For each method, means and standard deviations are estimated over four independent training seeds (3D-DAOSTORM is deterministic). "The EPFL 2016 challenge does not include a dataset with density of 8.0; see the main text for details about its creation process.
Density SNR
Method
Precision
Recall
Jaccard
RMSE,
RMSE,
EaD
AD-DAOSTORM
0.964
0.919
0.914
11.9
16.9
0.821
High
DECODE
0 = 0.003
0.998
0.001
0.939 0.003
8.8
0.1
10.7 #.11
0.895 0.003
LiteLoc
0.996 + 0.002
0,987 2 0.001
0.983 + 0.001
9.0
0.1
11.7
0.1
0.912 = 0,001
Ours
0.998 + 0.002
0.978 1 0.016
0.980 1. 0.010
7.5
+ 0.4
10.0
+ 3.5
0.920 + 0.007
0.2
3D-DAOSTORM
0.978
0.835
0.833
19.3
29.8
0.685
Low
DECODE
0.918 = 0.002
0.978 + 0.001
0.903 :: 0.002
20.5
0.1
26.2
+ 0.1
0.737 = 0.001
LiteLoe
0.001
0.934
17.0
+
0.4
0.798 0.001
Ours
0,985 + 0.001
0.961
+ 0.001
0.947 + 0.001
18.8
4-0.2
24.5
+ 0.1
0.802 + 0.002
3D DAOSTORM
0.914
0.678
0.643
56.8
76.6
0.373
High
DECODE
0.923 + 0.003
0.946 +: 0.002
0.876 + 0.004
32.2
+.0.3
33.0
+ 0.4
0.706 0.004
LiteLoc
0.993 + 0.001
0.863 0.002
0.858 + 0.001
30.7
0.2
36.0
0.3
0.699 + 0.001
2.0
= 0.002
0.895
+
28.4
0.750 0.001
3D DAOSTORM
0,914
0.496
0.475
74.4
120.0
0.116
DECODE
0.859 + 0.034
0.874
0.006
0.756 0.027
56.4
0.3
65.3
+ 0.4
0.468 0.008
Low
LiteLoc
0.992 + 0.001
0.729
0.002
0.725
16.2
- 0.4
63.8
+ 0.2
0.500 0.002
Ours
0.973 + 0.003
0.812 1: 0.007
0.794 0.005
48.4
1: 0.6
59.5
+ 0.4
0.536 + 0.003
3D-DAOSTORM
0,910
133.9
0.009
High
DECODE
0.973 = 0.001
0.627 + 0.002
0.617
+ 0.002
59.58 + 0.02
71.5
+ 0.5
0.371 + 0.006
LiteLoe
0,988
0.001
0.008
0.533 0.003
0.2
0.5
0.319
8.0"
Ours
0.989 +: 0.001
0.578
1.788.100 0.008
0.574 0.008
$2.5
+ 0.4
64.3
+ 0.7
0.384 + 0.003
3D-DAOSTORM
0,908
0,211
0.217
92.6
175.8
-0,216
Low
DECODE
0.93 4: 0.03
0.415 d: 0.005
0.402
+ 0.009
80.25
1: 0.04
105.2
4 1.0
0.090
0.007
LiteLoc
0.986 1 0.001
0.339 1: 0.002
0.338 0.002
76.1
0.1
110.1
1 0.8
0.055
0.002
Ours
0,983 + 0.002
0.376
0.008
0.374 0.008
74.3 + 0.5
99.4
0.103 + 0.002
global performance metric called 3D efficiency (Eap) then defined as:
Eist = 1 - v(1 Jaccard)² + a RMSE Iar
Eax
Etar
EaD
where
(8)
Eax = 1
Jaccard)2 + ag R
ax.
Qia = 1.0 and Oaax 0.5 nm following definitions of the EPFL challenge. All metrics are computed frame by frame and averaged.
Our benchmark includes 3D-DAOSTORM (Babcock et al. 2012). DECODE (Speiser et al. 2021 and LotelLoc (Fei et al. 2025). All algorithms are evaluated on the open-access EPHL. 2016 challenge datasets (Sage et al. 2019), all with astigmatism PSFs. To assess performance in very high-density regime, we have synthesized a density-8.0 benchmark by temporally binning groups of frames in the original density-2.0 sequences. For each newly binned frame, we have re-sampled camera noise using the known camera parameters. This extra step prevents an artificial signal-to-noise ratio (SNR) improvement caused by the frame-averaging process.
Results are reported in Table I We observe that while our approach yields lower recall than the other methods, it preserves excellent precision and almost always achieves the lowest RMSE in all spatial dimensions. Most notably, it also outperforms all competitors on the EaD metric for all densities and SNRs, establishing itself the most balanced method with respect to this criterion.
Real data. We have evaluated our method on three publicly available datasets, all of which provide beads for calibrating their astigmatic PSFs. The Tubulin and NPC-Nup datasets from Li al.) (2018) depict, respectively, the microtubule network and nuclear pore complexes in U2OS cells. The NPC-Nup96 dataset from Fei et al. (2025) also features nuclear pore complexes in the same cell line. All datasets were acquired with conventional SMLM activation densities; therefore, to test our method's s robustness at higher densities, we applied 16-frame temporal binning to Tubulin and NPC-Nup107 and 32-frame binning NPC-Nup96. We refer the cemporally-binnes versions as T16-Tubulin, T16-NPC-Nup100 and NB2-NNCC- Note that this approach is an imperfect proxy for truly high-density imaging, it improves the SNR via noise averaging.
Figure compares 3D SMLM reconstructions, rendered with SMAP Ries 2020). for 3D-
DAOSTOMM (Babcock al. 2012), LiteLoc (Fei et al. 2025) and method. We chose to
8

---

Published as a conference paper at ICLR 2026
Ours entire image
3D-DAOSTORM zoom
LiteLoc zoom
Ours zoom
T16- T16 Tubuuiin
T32-NPC-Nuu96 T16-NPC-NupI 107
Figure 4: Qualitative comparison of SMLM methods on real data. Although ground truths are un- available, results show that our approach yields fewer grid-reconstruction artifacts (line 1), improved depth estimation consistency (line 2), and more accurate nuclear pore complex reconstruction (line 3). Refer to the main text for a more thorough discussion.
include LiteLoc over DECODE because the former delivers comparable or slightly better perfor- mance. In addition, the authors of DECODE note that their method may benefit from an extra filtering step applied to the predicted uncertainties associated with each activation. However, this post-processing requires selecting additional arbitrary thresholds that difficult to tune, making challenging to perform a fair and objective comparison with other methods.
On the T16-Tubulin dataset, DAOSTORM and eliminates the artifacts that appear with LiteLoc. On the T16-NPC-Nup10) dataset, all methods recover comparable structures; however, our method delivers more consistent depth estimates (as indicated by colors), whereas 3D-DAOSTORM and LiteLoc exhibit spatially varying detections. On the T32-NPC-Nup96 dataset, our approach reconstructed NPC's structures with clear greater fidelity than LiteLoc and 3D-DAOSTORM.
algorithm yields higher-fidelity reconstruction than 3D-
Quantitatively, the absence of ground-trutt data prevents the use of the metries introduced Section To evaluate the resolution and fidelity of reconstructed super-resolution image, we adopted two widely used metrics: Fourier ring correlation (FRC) (Banterle et al., 2013) and the ioollutionscallen Pearson's 's coefficient (RSP) Culley et al. (2018). FRC reconstructs two super-resolution images by splitting localizations into two subsets, computing their Fourier transforms, and then measuring the correlation of their spatial frequency signals against each other. The resulting curve provides estimate of the spatial frequency at which signal can no longer be distinguished from noise (Banterle et al. 2013). RSP is defined as the Pearson correlation coefficient between the reconstructed super- resolution image and reference image, typically the mean of all raw wide-field frames. Values close one indicate strong agreement between the reconstruction and the reference.

---

Published as a conference paper at ICLR 2026
Table 2:
Quantitative results real datasets. We temporally binned them simulate very high- density setups. Our method consistently scored first in those denser regimes.
Dataset
Bin size
Method
FRC (nm) +
RSP t
LiteLoc
29.7 0.3
0.708
x1
Ours
31.9 + 0.2
0.692
Tubulin (Li et al.
2018
LiteLoc
63.0 + 0.8
0.649
x 16
Ours
58.1 + 1.1
0.672
LiteLoc
19.3 = 0.3
0.696
x1
Ours
18.8 0.4
0.686
NPC-Nup107 (Li et al.
2018
LiteLoc
25.9 + 0.3
0.682
16
Ours
22.1 + 0.1
0.684
LiteLoc
29.8 0.1
0.713
x1
Ours
31.7 0.1
0.693
NPC-Nup96 Feei al. 2025)
LiteLoc
71.5 0.5
0.671
X32
Ours
44.2 0.4
0.689
Table Ablation study of our different modules over EPFL synthetic with high SNR and density of 2.0. X means we used DECODE's original loss function model architecture.
Iterative arch. OT loss func.
Jaccard
R.SS +
Eдp f
0.876 + 0.004
47.9 0.5
0.705 = 0.004
X
0.867 + 0.004
3.3
0.740 = 0.002
X
0.854 + 0.005
0.7
0.703 + 0.005
0.883 + 0.007 39.2 + 0.5
0.750 + 0.004
Results with these metrics on real datasets are reported in Table dense-activations regimes, our approach consistently yields lower FRC and higher RSP values than other methods, confirming the visual improvements illustrated in Figure 4
Ablation study. We have conducted an ablation study on synthetic data to validate the effectiveness of our loss function and our iterative architecture. Results are reported in Table It can be seen that the loss function drives most of the improvement, with our iterative architecture providing modest boost. Given the additional memory and compute overhead of our architecture, lightweight variant that retains only the optimal loss function can be considered for deployment scenarios with constrained resources.
DISCUSSION AND CONCLUDING REMARKS
We have presented novel deep-learning SMLM method that surpasses existing methods in medium and high-density regimes, all without the need for handcrafted layers. By enabling faster data acquisition, approach extends SMLM's temporal resolution, allowing more accurate observation of rapid biological processes but also stable inference precision to degrading conditions induced by evolution of recording parameters during time. Furthermore, the integration of optimal transport theory to SMLM could open path to new localization algorithms.
The main limitation of our method is the longer training and inference times that result from its iterative design. However, training is one-time cost per experimental setup, and inference remains fast enough (~ 200 fps on modern GPU) to let biologists run multiple experiments sequentially with minimal delay. Another systemic limitation shared by most top-performing methods is the dependence for precise PSF calibration (Lelek al. (02T). Future work could focus on robust methods invariant to PSF variations, pursue blind SMLM super-resolution without sacrificing precision or include PSF optimization to the microscopy setup design during training and to adapt to cellular based optical anomalies affecting the PSF inference time.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
We thank Luca Calatroni for fruitful discussions and valuable feedback. This work was supported by the French government through the "France 2030" program, particular by the PR[AI]RIE- PSAI project (ref. ANR-23-IACL-0009), Julien Mairal was also supported by an ERC grant (ref. 101087696, APHELEIA project), and Jean Ponce was supported part by the Louis Vuitton/ENS chair in artificial intelligence, the Institute of Information & Communications Technology Planning & Evaluation (ITP)) grant funded by the Korean Government (MSIT) (ref. RS-2024-004578882 National Al Research Lab Project), and Global Distinguished Professorship the Courant Institute of Mathematical Sciences and the Center for Data Science at New York University. We received access the computational resources of IDRIS from GENCI (ref. AD010616282).
ETHICS STATEMENT
In this work, we explore new model architecture and loss function to improve single-molecule localization microscopy (SMLM) reconstruction. We do not anticipate ethical or societal harms: the work is computational only and all biological data used public and were used according their licenses. We believe that by improving SMLM reconstruction and releasing our code openly, this work can broadly benefit biological research and make advanced tools accessible communities worldwide.
REPRODUCIBILITY STATEMENT
The project repository hattps://gitie:
Cean/RSLLESSSARR includes all requirements to reproduce our results. We provide the full source code and model implementation, all datasets are publicly available, training and evaluation scripts are provided with all hyperparameters set (including random seeds), and python environment specification are supplied, making all experiments from Section reproducible.
REFERENCES
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer Normalization, arXiv preprint, 1607.06450, 2016.
Hazen Babcock, Yaron M Sigal, and Xiaowei Zhuang. A high-density 3D localization algorithm for stochastic optical reconstruction microscopy. Optical Nanoscopy, 1:1-10, 2012.
Hazen P Babcock and Xiaowei Zhuang. Analyzing single molecule localization microscopy data using cubic splines. Scientific Reports, 7(552), 2017.
Francisco Balzarotti, Yva Eilers, Klaus Gwosch, Arvid H Gynnå, Volker Westphal, Fernando D Stefani, Johan Elf, and Stefan W Hell. Nanometer resolution imaging and tracking of fluorescent molecules with minimal photon fluxes. Science, 335:606-612, 2017.
Niccolo Banterle, Khanh Huy Bui, Edward A Lemke, and Martin Beck. Fourier ring correlation as resolution criterion for super-resolution microscopy. Journal of Structural Biology, 183:363-367, 2013.
Eric Betzig, JK Trautman, TD Harris, JS Weiner, and RL Kostelak. Breaking the Diffraction Barrier: Optical Microscopy on a Nanometric Scale. Science, 251:1468-1470, 1991.
Eric Betzig, George H Patterson, Rachid Sougrat, O Wolf Lindwasser, Scott Olenych, Juan S Bonifacino, Michael W Davidson, Jennifer Lippincott-Schwartz, and Harald F Hess. Imaging intracellular fluorescent proteins at nanometer resolution. Science, 313: 1442-1655. 2006.
Nicholas Boyd, Eric Jonas, Hazen Babcock, and Benjamin Recht. DeepLoco: Fast 3D Localization Microscopy Using Neural Networks. BioRxiv preprint, pp. 267096, 2018.
11

---

Published as a conference paper at ICLR 2026
Mayeul Cachia, Vasiliki Stergiopoulou, Luca Calatroni, Sébastien Schaub, and Laure Blanc-Féraud. Fluorescence image deconvolution microscopy via generative adversarial learning (FluoGAN). Inverse Problems, 39(5), 2023.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision (ECCV), pp. 213-229, 2020.
Joao Carreira, Pulkit Agrawal, Katerina Fragkiadaki, and Jitendra Malik. Human pose estimation with iterative error feedback. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 47742 2016.
Siān Culley, David Albrecht, Caron Jacobs, Pedro Matos Pereira, Christophe Leterrier, Jason Mercer, and Ricardo Henriques. Quantitative mapping and minimization of super-resolution optical imaging artifacts. Nature Methods, 15:263- 266, 2018.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in Neural Information Processing Systems (NeurIPS), 26:2292-2300 2013.
Thomas Dertinger, Ryan Colyer, Gopal Iyer, Shimon Weiss, and Jörg Enderlein. Fast, background- free, 3D super-resolution optical fluctuation imaging (SOFT). Proceedings of the National Academy of Sciences (PNAS), 106(52):22277-2239) 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. International Conference Learning Representations (ICLR), 2021.
Thomas J Etheridge, Antony M Carr, and Alex Herbert. GDSC SMLM: Single-molecne localisa- tion microscopy software for ImageJ. Wellcome open research, 7(241), 2022.
Mohamadreza Fazel and Michael J Wester. Analysis of super-resolution single molecule localization microscopy data: A tutorial. AIP Advances, 12(010701), 2022.
Yue Fei, Shuang Fu, Wei Shi, Ke Fang, Ruixiong Wang. Tianlun Zhang, and Yiming Li. Scalable and lightweight deep learning for efficient high accuracy single-molecule localization microscopy. Nature Communications, 16(7217), 2025.
Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, et al. POT Python Optimal Transport. Journal of Machine Learning Research, 22(78))1-8,, 2021.
Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning generative models with sinkhorn diver- gences. In International Conference on Artificial Intelligence and Statistics (AISTATS), volume 84, pp. 1608-1617, 2018.
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580-587, 2014.
Arthur Gretton, Karsten M Borgwardt, Malte I Rasch, Bernhard Schölkopf, and Alexander Smola. A Kernel Two-Sample Test. Journal of Machine Learning Research, 13:723-773, 2012.
Mats GL. Gustafsson. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. Journal of Microscopy, 198:82-87, 2000.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Mike Heilemann, Sebastian Van De Linde, Mark Schüttpelz, Robert Kasper, Britta Seefeldt, Anindita Mukherjee, Philip Tinnefeld, and Markus Sauer. Subdiffraattin-rrsolltii fluorescence imaging with conventional fluorescent probes. Angewandte Chemie--nteraational Edition, 47(33), 2008.
12

---

Published as a conference paper at ICLR 2026
Stefan W Hell and Jan Wichmann. Breaking the diffraction resolution limit by stimulated emission: powuulated--emissoon-eeepe fluorescence microscopy. Optics Letters, 19(11):70--88) 1994.
Dan Hendrycks and Kevin Gimpel. Gaussian Error Linear Units (GELUs). arXiv preprint, 1606.08415, 2016.
Samuel Hess, Thanu Girirajan, and Michael Mason. Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophysical Journal, 91(11):42584447 4272, 2006.
Bo Huang, Sara A Jones, Boerries Brandenburg, and Xiaowei Zhuang. Whole-cell 3D STORM reveals interactions between cellular structures with nanometer-scale resolution. Nature Methods, 5:1047-1052, 2008.
Junhwa Hur and Stefan Roth. Iterative residual refinement for joint optical flow and occlusion estimation. Conference on Computer Vision and Puttern Recognition (CVPR), pp. 5754-5763, 2019.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Conference on Computer Vision and Patlern Recognition (CVPR), pp. 7482-7491, 2018.
Harold W Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics (NRL), 2:83-97, 1955.
Romain F Laine, Hannah S Heil, Simao Coelho, Jonathon Nieon-Abell, Angélique Jimenez, Theresa Wiesner, Damián Martínez, Tommaso Galgani, Louise Régnier, Aki Stubb, et al. High-fidelity 3D live-cell nanoscopy through data-driven enhanced super-resolution radial fluctuation. Nature Methods, 20: 10:1999-1956 2023.
Mickaël Lelek, Melina T Gyparaki, Gerti Beliu, Florian Schueder, Juliette Griffié, Suliana Manley, Ralf Jungmann, Markus Sauer, Melike Lakadamyali, and Christophe Zimmer. Single-molecule localization microscopy. Nature Reviews Methods Primers, 1(39), 2021.
Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. Dn-detr: Accelerate detr training by introducing query denoising. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13619-13627, 2022.
Xinyang Li, Xiaowan Hu, Xingye Chen, Jiaqi Fan, Zhifeng Zhao, Jiamin Wu, Haoqian Wang, and Qionghai Dai. Spatial redundancy transformer for self-supervised fluorescence image denoising. Nature Computational Science, 3: 3:1067-1080, 2023.
Yiming Li. Markus Mund, Philipp Hoess, Joran Deschamps, Ulf Matti, Bianca Nijmeijer, Vilma Jimenez Sabinina, Jan Ellenberg, Ingmar Schoen, and Jonas Ries. Real-time 3D single- molecule localization using experimental point spread functions. Nature Methods, 15:367-369, 2018.
Yiming Li, Elena Buglakova, Yongdeng Zhang, Jervis Vermal Thevathasan, Joerg Bewersdorf, and Jonas Ries. Accurate 4Pi single-molecule localization using an experimental PSF model. Optics Letters, 45((3))376,-3768, 2020.
Sheng Liu, Jianwei Chen, Jonas Hellgoth, Lucas-Raphael Müller, Boris Ferdman, Christian Karras, Dafei Xiao, Keith A Lidke, Rainer Heintzmann, Yoav Shechtman, et al. Universal inverse modeling of point spread functions for SMLM localization and microscope characterization. Nature Methods, 21:1082-1093, 2024.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Conference on Learning Representutions (ICLR), 2019.
Charles Mecutchen. Superresolution in microscopy and the Abbe resolution limit. Journal of the Optical Society of America, 57(10): 190-1119, 1967.
13

---

Published as a conference paper at ICLR 2026
Hamza Mentagui, Luca Calatroni, Sébastien Schaub, and Laure Blanc-Féraud Physics-inspires generative adversarial modelling for fluctuation-base super-resolution microscopy. In IEEE International Symposium Biomedical Imaging (ISBI). pp. 1-4. IEEE, 2024.
Grégoire Mialon, Dexiong Chen, Alexandre d' Aspremont, and Julien Mairal. A trainable optimal transport embedding for feature aggregation and its relationship to attention. In International Conference on Learning Representations (ICLR), 2021.
Elias Nehme, Lucien Weiss, Tomer Michaeli, and Yoav Shechtman. Deep-STORNE super- resolution single-molecule microscopy by deep learning. Optica, 5(4):458 464, 2018.
Wei Ouyang, Andrey Aristov, Mickaël Lelek, Xian Hao, and Christophe Zimmer. Deep learning massively accelerates super-resolution localization microscopy. Nature Biotechnology, 36:460-468, 2018.
Martin Ovesný, Pavel Křtžek, Josef Borkovec, Zdeněk Švindrych, and Guy M Hagen. Thun- derSTORM: a comprehensive Imagel plug-in for PALM and STORM data analysis and super- resolution imaging. Bioinformatics, 30(16):2389-2390 2014.
George Patterson, Michael Davidson, Suliana Manley, and Jennifer Lippincott- Schwwarz.. Superres- olution imaging using single-molecule localization. Annual Review of Physical Chemistry, 61: 345-367, 2010.
James Pawley. Handbook of Biological Confocal Microscopy, volume 3. Springer, 2006.
Gabriel Peyré, Marco Cuturi, et al. Computational Optimal Transport: With Applications to Data Science. Foundations and Trends@ in Machine Learning, 11(5-6)3355-600 2019.
Patrick Putzky and Max Welling. Recurrent inference machines for solving inverse problems. arXiv preprint, 1706.04008, 2017.
Jonas Ries. SMAP: modular super-resolution microscopy analysis platform for SMLM data. Nature Methods, 17:870-872, 2020.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 1-34-241,22 2015.
Kurt Rossmann. Point greaa-function, line spread-function, and modulation transfer function: tools for the study of imaging systems. Radiology, 93(2):257-272, 1969.
Michael J Rust, Mark Bates, and Xiaowei Zhuang. Sub-diffraction-linit imaging by stochastic optical reconstruction microscopy (STORM). Nature Methods, 3:793-796, 2006.
Duncan P Ryan, Megan K Dunlap, Martin P Gelfand, James H Werner, Alan K Van Orden, and Peter Goodwin. A gain series method for accurate EMCCD calibration. Scientific Reports, (18348), 2021.
Daniel Sage, Thanh-An Pham, Hazen Babcock, Tomas Lukes, Thomas Pengo, Jerry Chao, Ramraj Velmurugan, Alex Herbert, Anurag Agrawal, Silvia Colabrese, et al. Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software. Nature Methods, 16: 387-395, 2019.
Lukas Scheiderer, Zach Marin, and Jonas Ries. MINFLUX-molese resolution with minimal photons. Nature Protocols, 19:238-247, 2025.
Lothar Schermellch, Alexia Ferrand, Thomas Huser, Christian Eggeling, Markus Sauer, Oliver Biehlmaier, and Gregor PC Drummen. Super-resolution microscopy demystified. Nature cell biology, 21:72-84 2019.
Artur Speiser, Lucas Raccasl Müller, Philipp Hoess, Ulf Matti, Christopher J Obara, Wesley R Legant, Anna Kreshuk, Jakob H Macke, Jonas Ries, and Srinivas C Turaga. Deep learning enables fast and dense single-molecule localization with high accuracy. Nature Methods, 18: 18:00 2021.
14

---

Published as a conference paper at ICLR 2026
Zachary Teed and Jia Deng. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. In European Conference on Computer Vision (ECCV), pp. 402-419, 2020.
Cédric Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021.
Changdong Yu, Xiaojun Bi, and Yiwei Fan. Deep learning for fluid velocity field estimation: A review. Ocean Engineering, 271(113693), 2023.
Masoumeh 7areapoor, Pourya Shamsolmoali, Huiyu Zhou, Yue Lu, and Salvador Gareía, Fractional correspondence framework in detection transformer. In International Conference Multimedia, volume 1, pp. 5498-5506, 2024.
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. Interna- tional Conference on Learning Representations (ICLR), 2023.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. International Conference on Learning Representations (ICLR), 2021.
A APPENDIX
A.1 IMAGE FORMATION PROCESS
SMLM experimental setups typically employ either Electron-Mulliplyingg CCD (EM-CCD) or scien- tific CMOS (sCMOS) cameras. Their sensor converts incident photons into a digital intensity value (ADU) through a sequence of physical processes, each of which introduces noise.
Let be the incident photon count the camera sensor. Initially, photon detection is modeled Poisson process - known as shot noise - with a mean proportional to and the quantum efficiency (QE), and offset known the spurious charge (c):
Re,1 P (QE QQExn++x cc.
EM-CCD cameras introduce an additional amplification stage, modeled as Gamma distribution with parameters Re,1 and the electromagnetic gain (EM):
ne.2 ((,,,,, EM) for EM-CCD camera, ne,2 nhe11 for sCMOS camera.
(10)
Subsequently, read noise is modeled by normal distribution with mean ne,2 and standard deviation RR:
ne (ne,2,
(11)
Finally, the analog- coglog-to-diggee di-ditatal conversion process yields the observed ADU, scaled by the electrons per ADU (eAdU) and offset by a baseline (B):
(1 ne,3 [ADU
min
+ B, 65535
(12)
No algebraic solution exists for the resulting distribution relating and y (Ryan al. 2021). Table 4
presents the parameters for two commonly used cameras in SMLM: the Evolve Delta 512 camera for the Tubulin and NPC-Nup107 datasets (Li et al. 2018) and the Dhyana 400BSI V3 camera for the
NPC-Nup96 dataset (Fei et al. 2025).
A.2 ARCHITECTURE DETAILS
The decoder maps a latent representation z of the image implemented C x H x W tensor set of d activations, implemented d x 5 matrix (one activation contains five elements: the three spatial coordinates (z, y, 2). the number of emitted photons n and the detection score s).
Decoder architecture.
15

---

Published as a conference paper at ICLR 2026
0.75
Range factor Jaccard 1 RMSE
E3D
0.74
1.0
0.869
47.0
0.710
1.1
0.874
45.1
0.722
0.73
1.2
0.880
42.6
0.737
1.5
0.889
39.8
0.750
0.72
2.0
0.884
40.1
0.749
0.71
3.0
0.880
41.3
0.738
1.01.11.2
2.0
3.0
Maximum offset range factor
Figure 5: Evaluation of the impact of the maximum offset range of our CNN decoder. The maximum prediction range is controlled by a range factor; the final range equals the range factor multiplied by the output pixel size (which is 2x larger than the original image pixel size). Our results show that a range factor 1.5 times the output pixel size, i.e. 3x the original pixel size, yields the best performance. Reported results computed the MTONIHDAS dataset.
As we aim predict a from image, and given the recent success of object detection by transformer architectures (Carion et al., 2020), we have considered using vision transformer (ViT) (Dosovitskiy et al., 202T). However, this architecture produced unsatisfactory results, see Table We attribute this two factors:
SMLM requires sub-pixel precision, but each ViT's token spans the whole image, so a small prediction error severely affect the output.
2. The individual localization deteeccion problem is highly local. Hence, VIT's global attention mechanism offers little benefit.
Therefore, we propose convolution decoder architecture, composed of 2 x 2 max-pooling layers and residual blocks (He et al., 2016). We experimented with different numbers of max- pooling operations and found that single max-pooling layer, followed by a residual block and an element-wise output layer yields the best results, see Table
Formally, given the latent image 2, our decoder is defined as RCCHXW
5 H/xx/22x/22
mapping latent variable to H/2 x W/2 map with 5 channels, where each pixel is an activation prototype. Consider a single pixel of D's output, and let (Ø;, 9/) be its 2D coordinates in the camera coordinate system. The five elements output for this pixel encode the characteristics of the underlying candidate activation: the detection score š,, the relative lateral coordinates (Ar;, Ağe). the depth é; and the number of emitted photons Ω4 The absolute lateral coordinates (#;; ús) reconstructed by summing (Ar; Aū;) with (#;, ū.). The magnitude of the relative coordinate offsets (A.. Aÿ) predicted by the decoder is the original image, given the final max pooling). This extended range permits multiple activations to be mapped within single pixel area, as neighbouring activations can contribute to their surroundings. Figure 5) shows experiments for various magnitude of the coordinate offsets. With factor of 1x the output pixel size, each output pixel can predict locations only on the exact surface covers. this regime, the optimal transport solution reduces to an identity pixel-wise mapping.
1.5 times the output pixel size in other words 3x the pixel size of
Table 4: Reported parameters of typical cameras used in SMLM by their manufacturer.
Parameter
Evolve Delta $12
Dhyana 400BSI V3
Camera type
EMCCD
sCMOS
Quantum efficiency (QE)
0.90
0.95
Spurious charge (c)
0.002
0.002
EM gain (EM)
300
Readout noise (σR)
74.4
1.535
Electrons per ADU (EADU)
45
0.7471
ADU baseline (B)
100
100
16

---

Published as a conference paper at ICLR 2026
Table 5:
Evaluation of different decoder architectures with different number of predicted candidates over the MTONIHDAS dataset (Sage et al., 2019), with standard deviations computed for three different training seeds. CNNs differ by the architecture of their head module, composed of alternating 2 x 2 max- max-pooling layers and double-convolution blocks.
Architecture
Parameters
Jaccard ?
RMSE Eol J
EaD
HW
2.31M
0.866 + 0.003
41.4
+ 0.7
0.732 + 0.002
CNN
HW/4
2.81M
0.883 + 0.007
39.2
- 0.5
0.750 + 0.004
HW/16
3,48M
0.875 = 0.008
40.5
+ 0.5
0.739 + 0.002
VIT
HW/4
5.95M
0.852 = 0.006
40.6
+ 0.7
0.722 + 0.001
Table 6: Muliply-Accumulate operations and number of parameters of our model subparts.
Subpart
Multiply-Accumulate operations
Parameters
Encoder
1.03 GMac
1.05
Decoder
512.56 MMac
499.4 k
Residual Network
1.87 GMac
1.26 M
Renderer
94.64 MMac
0
Finally, the output is formatted into a candidate set & {( Ds. 2, hisis and detection scores set S = {&.hsigd- We integrate this reconstruction process into the decoder, meaning =
Differentiable simulation within our model. During inference, our algorithm selects a subset of candidate detections by thresholding their confidence scores. However, this operation is non- differentiable, preventing direct gradient propagation during training. To mimic this behaviour while retaining differentiability, we replace by soft weighting that scales the photon count of each candidate by its detection confidence. For each candidate æ; X, the network outputs the 3D coordinates (æ,, Di, &,), the raw photon count îi and a detection confidence à; (0, 1). We choose to modulate the photon count by the confidence, producing the weighted activation
i; (s:, Di. 2,, 8; n.):
and the set of all such activations denoted X = { This causes activations with low detection scores to have number of emitted photons near zero, making them almost non-existent, while keeping almost untouched activations with a detection score close to one, mimicking the effect of hard threshold while remaining fully differentiable.
After derivation, the expected image ý is obtained by:
QE EM H()) + B,
(13)
CADU
and with EM = for sCMOS camera. is end-to-end differentiable approximation of the reconstructed output, and can be used inside our iterative refinement scheme.
A.3 COMPUTATIONAL FOOTPRINT
Training is performed using an NVIDIA H100 GPU using the AdamW optimizer with a learning rate of 4 10 weight decay of 0.01, and a cosine anncaling scheduler. We chose a batch size 128 to maximize GPU usage, filling all 80 GB of VRAM. It can be lowered using smaller batch sizes or gradient accumulation.
We trained for 14 hours 100 epochs of 1024 steps each, totaling approximately 100,000 steps. Excellent results (Eap 2 0.72 on EPFL's density=2.0 and high SNR dataset) are achieved after only 20 minutes of training, at around 2000 steps.
During inference, a batch size of 16 produces peak VRAM usage of 8.7 GB and processes 2500 64x64 images in s0s, 12 ms/frann frame.
17

---

Published as a conference paper at ICLR 2026
Table 7:
Evaluation of different algorithms for solving the optimal transport problem used during the computation of our loss function. Results are reported for the MTONIHDAS dataset (Sage et al. 2019, with standard deviations computed for three different training seeds.
Algorithm
Jaccard 1
RMSE vol
EgD 1
e 10
0.531 0.007
67.0
5.2
0.397 + 0.011
10
0.883 = 0.006
42.0
+ 1.3
0.739 0.002
Sinkhorn's
c 10 -4
0.883 = 0.007
39.2 0.5
0.750 = 0.004
c 10" 5
0.877 0.012
39.1
1.1
0.747
Hungarian
0.873 0.007
39.7
0.9
0.742 + 0.001
Table 8: Evaluation of the robustness of our model to domain mismatch. Multiplicative noise of increasing strength, sampled from zero-mean log-normal distribution, is applied the simulator's camera parameters. Our model demonstrates strong resilience to this perturbation.
Jitter strength
Jaceard f
RMSE
E3D
a = 0
0.956 + 0.001
34.31 + 0.26
0.811 + 0.001
0.03
0.958 = 0.002
34.52 = 0.20
0.810 + 0.001
O = 0.10
0.957 = 0.001
34.50 = 0.25
0.810 + 0.001
0.30
0.954 + 0.001
35.21 + 0.32
0.805 + 0.002
Table sshows an overview of the computational resources for each subpart of our model.
A.4 REGULARIZED OPTIMAL TRANSPORT
As explained in the main text, we solve the regularized optimal transport problem of Eq. ( with Sinkhorn's 's algorithm. Our motivations are both analytical and computational: compared to the standard bipartite matching, Sinkhorn's algorithm avoids the need for stop-gradient operations, is differentiable, and is computationally efficient GPUs. In our implementation, algorithm in log space and compute gradients automatically via PyTorch 's autograd module. Our implementation uses 20 iterations, we have found that additional iterations do not improve performance. Additionally, we have included a masking step to ensure that candidates are only assign to target activations they are capable of reaching within their limited prediction range. The observed performance boosts for increased range factors highlight the benefits of using optimal transport rather than pixel-wise assignments.
Sinkhorn's 's
Table 7 compares results for various regularization constant in regularized optimal transport problem. We also report results with bipartite matching performed by the Hungarian algorithm, that yields lower performance. We observe that smaller values for yield better performance, with no improvement beyond e = 10-4; thus our we set e = 10-4 in our implementation. Note that our implementation of the Sinkhorn's 's algorithm includes the common practical heuristic of scaling by the median of the cost matrix (Flamary et al. 2021).
A.5 ROBUSTNESS TO CAMERA PARAMETERS MISMATCII
We have conducted a study to analyze the robustness of our model to mismatch with respect to all camera parameters listed in Table 4
To this end, we have generated a synthetic dataset of 2048 frames with a mean density of 2.0, each rendered with camera parameters independently jittered by noise from a log-normal distribution, i.c. scaled by el where ~ NN o). Note that we apply a similar data augmentation process during training, with 0.03.
We have evaluated our model performance under increasing noise strength, to assess performance for increasing domain gap between training and test data. The results reported in Table Interestingly, our model appears extremely resilient to this type of mismatch. From an architecture standpoint, the of LayerNorm and 2D convolutions without additive bias makes the network insensitive to
18

---

Published as a conference paper at ICLR 2026
(1) y
(2)
Detection scores
(3) NMS
(4) Output
DECODE
Zoom in y
(2) Detection scores
(3) Simple threshold
(4) Output
Ours
Figure 6: Toy illustration of the negative effects of pixel-wise loss functions and NMS post processing. (I) A synthetic image with a zoomed version. White crosses represent the ground truth emitters. (2) DECODE's pixel-wise loss function enforces high score for the bottom-right pixel, but does not consider that is shared by two targets. By contrast, our optimal transport loss function also assigns a high score
neighboring pixel, which contribute to its neighborhood with extended localization range. (3) DECODE's NMS variant merges the two adjacent predictions in the top-left corner if one of the prediction scores isn't high enough be automatically retained. By contrast, our simple single-threshen filtering keeps all scores above the threshold, regardless of their spatial distribution. (4) Models' final outputs, showing the predicted emitters for each method with yellow crosses.
scaling. We hypothesize that this architectural choice paired with the small data augmentation during training results in remarkably stable performance with respect to this issue.
19


Reviews:
[{"review_id": "GPDRho9QbE", "reviewer": "Reviewer_VfVn", "summary": "The paper treats the problem of single molecule light microscopy reconstruction as a set matching problem, which is a sound strategy. The authors develop an optimal transport loss and a model architecture to solve this problem. The strategy and results are promising, but the robustness and generalization of the method need to be evaluated further.", "questions": "Majority of my concerns are centered on evaluation of the robustness of the method to unwanted aberrations in imaging, density of fluorophores, and quantum efficiency of fluorophores.\n\nI suggest a systematic series of simulations and experiments to address these. For example,\n1. Consider adding a learned layer to the image formation model that learns a single 3D kernel that allows the image formation model to adapt aberrations in the data.\n2. Simulate data with varying quantum efficiencies and aberrations different from training data, and evaluate the sensitivity of a trained model to these imperfections in real data.\n3. Evaluate feature activations at the end of the encoder and at the end of the decoder for specific raw data frames to test that the estimated localizations make sense, and models are truly learning a relationship between the intensity distributions and the localization of emitters.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This work introduces two key computational innovations to single-molecule localization microscopy (SMLM). First, the authors reformulate SMLM as a set-matching problem and derive an optimal transport loss function, enabling end-to-end differentiable training . This is the first application of optimal transport theory to SMLM, to the best of my knowledge, and represents a principled approach to handling variable-size sets of fluorophore detections. Second, they propose an iterative refinement architecture that explicitly integrates knowledge of the microscope's optical system by simulating expected frames from current predictions using the known point spread function (PSF), allowing computation of loss between experimental data and estimated molecular distribution.  The combined approach achieves state-of-the-art performance on the E3D metric across multiple density regimes, with particularly strong improvements in localization accuracy (lowest RMSE) at high densities (density 8.0), demonstrating the method's potential to enable faster acquisition times for live-cell imaging.", "weaknesses": "The following limitations can constrain the practical applicability of current method. First, the method exhibits consistently lower recall than competitors across all experiments (Table 1), meaning it misses more true fluorophores despite having excellent precision—this trade-off could lead to incomplete biological reconstructions. It is not clear from the paper which aspects of data or optimization influence precision and recall. \n\nSecond, the approach requires precise PSF calibration using fluorescent beads and the method's robustness to PSF miscalibration or drift remains unexplored. The iterative architecture incurs 3× higher computational cost during training (20 hours) and slower inference compared to single-pass models. \n\nThird, the high-density validation on real biological data uses temporal binning as a proxy, which artificially improves signal-to-noise ratio through averaging rather than testing true high-density, low-SNR conditions that would occur in practice. Majority of training relies on synthetic simulations with uniform priors (10-30 activations, uniform spatial distribution), which may not capture the structured, heterogeneous distributions in real biological specimens.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Oz9enGlOdW", "reviewer": "Reviewer_4CME", "summary": "The paper presents a methodology to train neural networks for detection of activations in single-molecule localization microscopy. The main premise of the paper is that optimal transport can serve as a tool to design loss functions that are fully differentiable and better performing for solving this problem. The proposed optimal transport objective matches activation points with detected observations in the images. The proposed architecture also incorporates an iterative refinement feedback which reconstructs the image with an image formation simulation given the model detections.", "questions": "* What makes the formulation novel in the context of object detection or point localization?\n* Does the SMLM problem really need a new methodology, given that other methods perform comparably?\n* What other benefits (computational, speed, etc) can be obtained from the proposed formulation?\n* Is this applicable to other microscopy imaging types?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Formulation of a differentiable loss function to detect activation points in single-molecule localization microscopy. \n* Effective use of optimal transport theory to design the proposed method.\n* Natural design of the iterative refinement strategy based on physical image-formation principles.\n* Experimental evaluation with multiple performance metrics and relevant baselines.\n* Qualitatively, the methodology also improves detections in real world data.", "weaknesses": "* While the application of optimal transport in SMLM is novel, it has been used before for similar detection and localization problems.\n* The performance metrics seem to be saturated in the selected datasets for evaluation. The performance gains seem marginal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dJfh1R4ZvD", "reviewer": "Reviewer_gn8m", "summary": "Recent deep-learning methods for high-density single-molecule localization microscopy (SMLM) typically depend on non-differentiable post-processing steps, such as non-maximum suppression, to extract the final set of emitter coordinates. This paper reformulates emitter localization as a set-matching problem and introduces an entropy-regularized optimal transport loss that enables fully differentiable, end-to-end training without the need for heuristic post-processing. Furthermore, the proposed iterative refinement architecture integrates the microscope’s image formation model during inference, making the approach physics-aware and potentially enhancing robustness in real experimental conditions. Experiments on synthetic and real datasets demonstrate state-of-the-art performance, particularly in moderate and high emitter density regimes.", "questions": "1) Could the authors clarify the motivation for adopting an entropy-regularized optimal transport loss (via the Sinkhorn–Knopp algorithm) instead of a DETR-style bipartite matching loss? In particular, what empirical or theoretical advantages (e.g., smoother optimization, differentiability, computational speed) motivated this choice, and how many Sinkhorn iterations were used in practice to approximate the transport plan?\n2) How is the gradient of the Sinkhorn-based loss computed during backpropagation? Is there a closed-form analytical expression for the gradient, or is it obtained by unrolling the Sinkhorn iterations? Providing this clarification—ideally with a short note or derivation in the appendix—would significantly improve the clarity of the method.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper is original in framing single-molecule localization as a set-matching problem and introducing an entropy-regularized optimal transport loss that removes the need for non-differentiable post-processing, enabling genuine end-to-end learning. The technical quality is high, with a carefully designed architecture that incorporates a physically accurate forward model of the imaging system, explicitly modeling shot, readout, and amplification noise. The results are clearly presented and demonstrate strong generalization from synthetic to real biological datasets, underscoring the method’s robustness and practical significance. Overall, the paper combines conceptual novelty, methodological rigor, and empirical clarity, marking a meaningful advance in the application of deep learning to super-resolution microscopy.", "weaknesses": "While the paper is strong overall, a few aspects could be clarified or expanded. First, the motivation for using an entropy-regularized optimal transport (OT) loss over standard bipartite matching approaches, such as the Hungarian assignment followed by pairwise regression losses (as employed in DETR and related works), remains insufficiently justified. A direct comparison or ablation highlighting the practical advantages of the entropic formulation—such as smoother gradients, faster convergence, or improved stability—would strengthen the claim. Second, the related work section omits several important prior efforts that jointly optimized the point-spread function (PSF) and the localization model for dense 3D SMLM, most notably DeepSTORM3D (Nehme et al., 2020). Including and discussing these works would provide a more balanced contextualization of the paper’s contributions. Finally, DECODE is excluded from the qualitative comparisons in Figure 4 under the justification that its results resemble LiteLoc; however, DECODE additionally provides per-emitter uncertainty estimates that can be leveraged for post-hoc filtering and might alter the visual comparison. A brief discussion or experiment acknowledging this aspect would help ensure a fairer and more complete evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2DY7J6qxNO", "reviewer": "Reviewer_pzpv", "summary": "This paper proposes a novel method for single-molecule localization microscopy (SMLM). The authors formulate the SMLM task as a set matching problem and employ the optimal transport cost as the loss function.\nFurthermore, the proposed method utilizes a refinement model that integrates embedded features extracted from the input image, the previous prediction step, and a reconstructed image generated through an image formation model. This design effectively incorporates prior knowledge derived from an approximate point spread function into the molecular localization process.\n\nExperimental results demonstrate that the proposed framework outperforms existing deep learning–based approaches quantitatively and qualitatively, particularly under high-density conditions.", "questions": "7-1. What is the intended meaning and benefit of “end-to-end learning” in this paper?\n\nIt is true that, in general, loss functions for set-matching problems are non-smooth, making it difficult to compute gradients through cost functions that depend on the network outputs, such as coordinates or confidence scores.\nHowever, although the authors describe SMLM as a set-matching problem, the estimation in this work appears to be performed in a pixel-wise manner, where each predicted pixel can be directly compared with its corresponding ground-truth pixel. In this case, solving a set-matching problem is unnecessary, and the network can be trained end-to-end as in DECODE.\n\nConversely, using the OT distance as a loss function may actually hinder end-to-end learning rather than enable it. It is therefore unclear why a set-based formulation is necessary or what specific benefits it is expected to provide.\n\nIn addition, the idea of formulating SMLM as a set-matching problem is closely related to [1], where the maximum mean discrepancy (MMD) is used as a loss function to quantify the difference between the estimated and ground-truth molecule sets, while the estimation is performed for the whole image rather than in a pixel-wise manner.\n\n[1] Boyd, N.; Jonas, E.; Babcock, H.; Recht, B. DeepLoco: Fast 3D Localization Microscopy Using Neural Networks. 2018. https://doi.org/10.1101/267096.\n\n\nSuggestion\n7-2. Clarify the network output and loss design.\nAlthough the authors state that the number of candidates d is a hyperparameter, it appears to depend on the input image size (H, W), with predictions made for every four pixels. From the appendix, each pixel’s offset is up to three times larger than a pixel, which markedly differs from prior works. Since the estimation is performed pixel-wise, the ground-truth coordinate set can also be converted into pixel-wise labels—indicating whether a molecule exists and, if so, its offset within the pixel. In this case, the prediction and ground-truth representations share the same structure and ordering, and the OT loss effectively reduces to a sum of pixel-wise losses when the OT map becomes the identity. The authors should clarify how this formulation differs from a conventional pixel-wise loss, possibly by analyzing the learned transport map.\n\n7-3. Analyze OT map\nThe transport cost design suggests that information from neighboring pixels might also contribute to molecular localization. Examining the OT map could help explain why, even with lower Jaccard scores, the estimated coordinates remain close in high-density cases (as in Table 1, 8.0). An ablation study comparing the OT loss with a simple pixel-wise loss would clarify the true contribution of OT and the extended offset range.\n\nMinor Comments\n\n- On page 7, line 366, the citation of DECODE refers to a different paper. The original DECODE paper is by Speiser et al., Nature Methods, 2021.\n- Plese define RMSE_vol in Table 3.\n- The reported E_{3D} values for DECODE differ by 0.001 between Table 1 and Table 3. Please clarify the reason for this discrepancy.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is clearly written and easy to follow, with a well-articulated background on single-molecule localization microscopy (SMLM). The formulation of SMLM as a set matching problem and the use of the optimal transport (OT) distance as a loss function are original and technically interesting. The refinement model is also a valuable contribution: it effectively integrates features from the input image, the previous prediction step, and a reconstructed image generated from the previous output, while maintaining a simple and lightweight network architecture.\nFurthermore, the ablation study demonstrates the effectiveness of both the loss formulation and the proposed modeling approach.", "weaknesses": "While the use of the optimal transport (OT) distance for SMLM is original, its motivation is not clearly articulated. The paper refers to the proposed framework as “end-to-end learning,” but the meaning of this term is somewhat ambiguous in this context (see comments in Section 7-1 for further detail). In addition, the paper criticizes LiteLoc in the Introduction for using an NMS layer, but LiteLoc does not actually employ such a layer. As a result, it is unclear what specific limitation or issue of LiteLoc the proposed method aims to address.\n\nIn addition, the paper presents two seemingly independent contributions—a new loss function and a new network architecture—but the connection between them is not clearly established. The architecture does not appear to leverage the properties of the proposed loss, which weakens the methodological coherence of the work. The authors should better justify why both components are included in a single framework and discuss whether one could be used meaningfully without the other.\n\nFurthermore, the training process is insufficiently described. In particular, it remains unclear what the network (especially the decoder) outputs and what training data correspond to these outputs (see also the suggestion in Section 7-2). This lack of clarity makes it difficult to assess the effectiveness of using the OT distance as a loss function.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)