Distributional Consistency Loss: Beyond Pointwise Data Terms in Inverse Problems
Authors: George Webber
Andrew J. Reader
Venue: ICLR 2026 Poster (2026)
Keywords: Inverse problems
data fidelity
denoising
image reconstruction
regularization
overfitting
Primary area: optimization
TL;DR: A new data-fidelity loss improves the quality and stability of inverse-problem reconstructions by matching the noise distribution instead of individual measurements.

Abstract: Recovering true signals from noisy measurements is a central challenge in inverse problems spanning medical imaging, geophysics, and signal processing. Current solutions nearly always balance prior assumptions regarding the true signal (regularization) with agreement to noisy measured data (data-fidelity). Conventional data-fidelity loss functions, such as mean-squared error (MSE) or negative log-likelihood, seek pointwise agreement with noisy measurements, often leading to overfitting to noise. In this work, we instead evaluate data-fidelity collectively by testing whether the observed measurements are statistically consistent with the noise distributions implied by the current estimate. We adopt this aggregated perspective and introduce $\textit{distributional consistency (DC) loss}$, a data-fidelity objective that replaces pointwise matching with distribution-level calibration. DC loss acts as a direct and practical plug-in replacement for standard data consistency terms: i) it is compatible with modern unsupervised regularizers that operate without paired measurement–ground-truth data, ii) it is optimized in the same way as traditional losses, and iii) it avoids overfitting to measurement noise without early stopping or the use of priors. Its scope naturally fits many practical inverse problems where the measurement-noise distribution is known and where the measured dataset consists of many independent noisy values. We demonstrate efficacy in two key example application areas: i) in image denoising with deep image prior, using DC instead of MSE loss removes the need for early stopping and achieves higher PSNR; ii) in medical image reconstruction from Poisson-noisy data, DC loss reduces artifacts in highly-iterated reconstructions and enhances the efficacy of hand-crafted regularization. These results position DC loss as a statistically grounded, performance-enhancing alternative to conventional fidelity losses for an important class of unsupervised noise-dominated inverse problems.

Published as a conference paper at ICLR 2026
DISTRIBUTIONAL CONSISTENCY Loss: BEYOND POINTWISE DATA TERMS IN INVERSE PROBLEMS
George Webber & Andrew J. Reader
School Biomedical Engineering and Imaging Sciences King's College London (george, andrew). (webber, reader] .ac. .u
ABSTRACT
Recovering true signals from noisy measurements is central challenge inverse problems spanning medical imaging, geophysics, and signal processing. Current solutions nearly always balance prior assumptions regarding the true signal (reg- ularization) with agreement to noisy measured data (data-fidelity). Conventional data-fidelity loss functions, such as mean-squared error (MSE) or negative log- likelihood, seek pointwise agreement with noisy measurements, often leading overfitting to noise. In this work, we instead evaluate data-fidelity collectively by testing whether the observed measurements noise distributions implied by the current estimate. We adopt this aggregated per- spective and introduce distributional consistency (DC) loss, data-fidelity objec- tive that replaces pointwise matching with distribution-lest calibration. DC loss acts a direct and practical plug-in replacement for standard data consistency terms: i) it is compatible with modern unsupervised regularizers that operate with- out paired measurement ceeel-rroundrrrre -truh data, ii) optimized the same way as traditional losses, and iii) it avoids overfitting to measurement noise without early stopping or the use of priors. Its scope naturally fits many practical inverse prob- lems where the measurement-noise distribution is known and where the measured dataset consists of many independent noisy values. We demonstrate efficacy two key example application areas: i) in image denoising with deep image prior, using DC instead of MSE loss removes the need for early stopping and achieves higher PSNR; ii) in medical image reconstruction from Poisson-noisy data, DC loss reduces artifacts in highly-iteraed reconstructions and enhances the efficacy of hand-crafted regularization. These results position DC loss as statistically grounded, performance-enhancii alternative to conventional fidelity losses for an important class of unsupervised noise-dominated inverse problems.
statistically consistent with the
INTRODUCTION
Reconstructing true signal from single of noisy measurements is prevalent challenge in scientific computing (Vogel) 2002 Tarantola 2005) Aster et al. 2018), with applications spanning medical imaging (Bertero
2017 Ribes and Schmitt 2008 Louis (992). remote sensing (Efremenko and Kokhanovsky [202] Baret and Buis 2008), time-series denoising (Gubbins) 2004), astronomical data analysis (Craig and Brown 1986 Lucy 1994), and geophysical inversion PParer 1994 Linde et al. 2015). In these problems, the reconstruction objective typi- cally combines a data-fideitty term with a regularizer: data fida-fiielity enforces consistency with the measurement process and noise model, while regularization encodes prior structure in the signal.
2222 [02]|M
While regularization has evolved substantially (Habring & Holler 2024), the underlying fita-fidelity terms have changed little. Standard objectives such as mean squared error (MSE) or negative log- likelihood (NLL) penalize residuals point-by-point. Optimizing these pointwise objectives encour- ages matching individual noise realizations rather than ensuring that the measurements are statis- tically compatible with the model. While early stopping or discrepancy-bart this behavior, they require explicit tuning and do not change the objective itself. As result, un- der a noisy realization, the ground-trute signal is not a minimizer of the pointwise data term, and regularization must compensate for noise fitting rather than focusing solely structure.
rules can mitigate

---

Published as a conference paper at ICLR 2026
We address this limitation directly by reframing the data-fidelity objective. We introduce distribu- tional consistency (DC) loss, which evaluates whether the measurements are collectively consistent with the noise distributions implied by candidate reconstruction. Each measurement is mapped to its percentile value under the cumulative distribution function (CDF) of its predicted noise model, and the resulting collection of percentile values is compared the uniform distribution expected under a correct model (Figure[]. Instead of reducing individual residuals, DC loss tests whether the entire noisy dataset behaves like typical draw from the assumed noise process.
DC loss applies whenever the noise distribution is known or can be estimated, including het- eroscedastic settings such as Poisson counts or Gaussian noise with known variance. It requires only one noise realization per measurement but many independent measurements overall. This set- ting satisfied common inverse problems such as pixel-wise imaging (Fan al., 2019), dense time-series sampling (Gubbins 2004), and tomographic reconstruction (Arridge et al. 2019).
CONTRIBUTIONS:
We introduce DC loss as a new principled data-fidelity term built from distributional cali- bration. Evaluating DC loss is simple and compatible with mnto-diifeerentiation frameworks.
We analyze how DC loss compares pointwise objectives: for parameter estimates that are far from the solution, DC loss exhibits MSE/NLL-like convergence, while near the solution it removes the incentive to fit noise, enabling stable prolonged optimization.
Across the settings we study, DC loss improves practical reconstructions: (a) Deep Image Prior (DIP) denoising (Ulyanov al., 2020) with clipped Gaussian noise, where DC loss removes the need for early stopping and yields higher peak PSNR than MSE; (b) PET image reconstruction under a Poisson model (Gourion and Noll (002), where DC loss reduces noise artifacts at high iteration numbers and pairs well with TV regularization, achieving superior noise-detail trade-offs at much smaller regularization strengths.
We demonstrate the real-world applicability of DC loss through experiments on real 3D PET brain data.
2 BACKGROUND
2.1 PROBLEM FORMULATION
We consider inverse problems where latent signal is mapped through a known forward model and corrupted by stochastic noise drawn from known distribution. No paired measurement-grouue truth data are available for learning, though a pre-trained regularizer may be provided.
Let unknown parameters 0* θ be mapped by known forward operator
θ + R
noise-fie free signal (or mean data) y = f(0') = (y1..... .NN). Measurements m = (m) mN) then modeled noisy draws from known, per-index likelihoods tied to that signal,
m: ((y))) D(((()):)
where each family Dj has ys as its single unknown parameter (e.g.. Poisson rate, Gaussian mean with known variance). Our goal is to recover an estimate i close to 0", given m, f, and (D.)
Classical data-fidelity terms enforce pointwise agreement between f(0) and m. which can promote overfitting in expressive models by tracking individual noise realizations. In Section 3 we instead assess distributional agreement: given a candidate ý = =(Ō), we ask whether the measurements m statistically consistent with the predicted noise models (D.
2.2 RELATED WORK
While most progress in inverse problems has focused regularization, some works have considered the design of data- tata-fidelity terms. Robust variants such as the Huber and Student's 's losses (Mohan et al. 2015: Kazantsev et al. 2017) can reduce outlier influence but do not prevent noise-fitting under a correct noise model. Stein's 's unbiased risk estimate (Metzler et al. 2018) penalizes the divergence of the estimator control its sensitivity noise, effectively regularizing the architecture rather than
2

---

Published as a conference paper at ICLR 2026
Poor fit (underfitting)
Optimal fit
Overfitting
Predicted (
True Sigeal orl
Signal value
Signall value
Predictediis)
Signal value
Predictas
Trus Signat by)
True Signal ln!
manusstir))
noiry peooreeeett)
Time
Time
Time
Empirical presebility denalty
Empiniss prooiiiity density
Empirical presobiilty density
Unttorm distriaution
0.8
1.
6b
c/4
CDF values
CDF values
CDF values
Figure 1: Illustration of the proposed DC loss. Top: predicted signals illustrating under-, well- and over-fitting to noisy measurements. Bottom: empirical CDF-value histograms for each case (assuming many predicted points). Poor fit histograms skew towards 0 or 1; good fits give uniform histograms; overfitting gives sharp peak near 0.5. DC loss penalizes departures from uniformity.
altering the underlying incentive to match the observed noise. DC loss operates different regime from Noise2Noise (Lehtinen et al. (2018), which requires multiple noisy observations, end-to-end data.
trained approaches (Heckel 2025) which require paired measuremeent-rround-
DC loss is instead linked most strongly to classical goodness-of-fr tests such the Kol- mogorov-Smirnov (K-S) and Cramér-von Mises (CvM) criteria (Kolmogorov T933 Cramér 1928) as well related work in emission tomography (Llacer et al. 1989). The K-S and CvM formulations provide post-hoc assessments of goodness-of-fit using L and L' discrepancies between empirical optimization-friendde analogue of these
and theoretical distributions. DC loss may be viewed tests: by changing the metric, we obtain smooth, differentiable objective suitable for optimization rather than post-hoc assessment. This avoids the limitations of post-hoe stopping criteria in iterative reconstruction, such as spatially variant convergence and ambiguity in determining when a recon- struction complete. This mechanism parallels the stabilization achieved by semi-convergence and carly-stopping methods (Elfving et al. 2014) but embeds it directly in the loss, allowing data fidelity and regularization interact synergistically.
3
METHOD
3.1 FROM POINTWISE TO DISTRIBUTIONAL CONSISTENCY
The Probability Integral Transform (PIT) provides a simple way to test whether a model's predicted distributions statistically consistent with observed data. For each measurement m, and its corre- sponding predicted noise distribution (ūr), we compute the cumulative probability
8; (m, û:) F Fo-D,( S m²).
Each value represents the percentile position of m: within its predicted distribution. The PIT says, that if the model correctly calibrated, these percentiles are uniformly distributed on [0, 1]: about 10% of data fall below the 10th percentile, 50% below the median, and so on!) Departures from uniformity therefore indicate systematic mismatch between the model and data.
Figure Tillustrates this idea. For predicted signal ý (red curve), we evaluate the CDF of each noise model (ū.)) (vertical Gaussians) at the corresponding measurement m, (black crosses), Collecting
See Appendix A.1 for full mathematical statement of this result.
3

---

Published as a conference paper at ICLR 2026
Algorithm GGenrallizdd Distributional Consistency Loss
Require: Estimate θ observed data m, number of measurements N, CDFs Fi. FN cor-
responding noise distributions D1, Dx. forward operator θ - R.
1: y f(0)
2: + (mmll))) ...., FN(mN/
Evaluate CDF of model at observed data > Apply logit transform
3: r < logit(s)
4: Sample
Logistic 1)
Generate N samples from target distribution
5: Sort r and u in ascending order
6: Loc(O)= + N Ei (ra uil
D Compute Wasserstein-1 distance
7: return CDc
these 8; values yields an empirical distribution whose shape reflects how well the model explains the data:
Underfitting:
histogram with peaks near 0 or shows that most measurements lie in improbable regions of their predicted distributions (jk).
Well-calibrated: a uniform histogram shows that each quantile contains its expected share of data.
Overfitting: a histogram with a sharp peak near 0.5 shows that most measurements lie near the center of their predicted noise distribution.
This motivates the concept of distributional consistency; a prediction is distributionally consistent when the empirical distribution of its CDF values is uniform. The next subsection formalizes this intuition into a differentiable loss.
3.2 DISTRIBUTIONAL CONSISTENCY LOSS FORMULATION
Given a current parameter estimate ê, we evaluate each measurement m, under its predicted noise distribution (0)x) via its cumulative probability
8, Fr(m; [()))) (e m²).
When the model is well-calibrated, the PIT guarantees that these percentiles follow a uniform distribution on [0, 1]. We therefore aim to make the empirical distribution close to uniform as possible.
(s1...., 8N) as
However, directly matching Uniform 00, 1] often leads vanishing gradients: when a predicted mean far from my, s; saturates near 1, giving ad, transform (inverse sigmoid)
s 0. To address this, we apply the logit
81
logit(s;) In
(3)
which stretches the endpoints to too and preserves gradient sensitivity. This maps the uniform target to Logistic(O,, 1) distribution To measure the discrepancy between the empirical {ri} and the (Og 1) reference, we compute the Wasserstein-1 (Earth Mover's) distance:
2
N
CD 0
rs ucl.
(4)
where [
N are sorted samples from Logistic(O,, 1).
This formulation encourages θ to produce predictions whose implied noise distributions make the observed data statistically typical, rather than merely close in value, offering principled alternative to pointwise losses. Algorithm summarizes the steps taken to calculate DC loss.
"ee Appendix A.2 for derivation.
4

---

Published as a conference paper at ICLR 2026
no
DC
sy maximum Ikalhhoc esimata
aignela
Worst eatimate
Ground fruth
x
Best-case DC sotimate
MSE
More reguarization
Figure 2: Interaction of DC loss and MSE loss with regularization. Crosses denote the predicted noise-free values y implied by different parameter settings. With MSE loss, the estimate is pulled toward the noisy maximum-likelihen between data-fidelity and regularity. With DC loss, many estimates that explain the data attain (near- yzero loss, forming a manifold around the MLE; adding regularization then simply selects among these data-consistent solutions rather than trading fidelity for regularity.
estimate (MLE); adding regularization forces a compromise
4 RELATION TO POINTWISE ERROR FUNCTIONS
4.1 EARLY-STAGE OPTIMIZATION BEHAVIOR
To avoid numerical precision difficulties, may wish to approximation that directly esti- mates r;, avoiding the need to compute the CDF value 8; and then apply the logit as separate steps (i.e. combining Steps 1 and 2 in Algorithm 1p. For example, when (ūc)) NNt o²), if Im--id large then 8; (p S m;; may be exactly 0 or under float precision. To resolve this issue, we may calculate T; = logit(s;) in one step, using appropriate approximations for s; and logit. When 8; ~ 1 (i.e. m, >> Úi), using the Laplace tail approximation to Gaussian distribution we obtain:
(m, -0k--9)) 0:)² +
m, -m,-): + In (V2VV
r2 = logit((ss;) Pd
(5)
2o2
σ
Note that when predicted value Di is much smaller than the measured value my is dominated by the squared term in the approximation. derivation and the s; as 0 case are given in Appendix C
Now consider the DC loss contribution for index and the gradient it gives to ÿt when ÿ; is far from yi- The per-sample term is rr uz/. where r; logit(F (F(m,, / (s)) and u, is the corresponding sorted reference value from Logistic(O,, 1). When |m.-kel is large, |rs] dominates the bounded u, so Ir. uil Iril and the gradient w.r.t. ØJ governed by Ōr:/Oū, Using the Gaussian tail approximation in equation 0r:/0î, ~ - (m; - 9r)/o². Thus, far from the solution, DC loss provides essentially the same pointwise update direction as MSE on the noisy measurement, with differences emerging only near the data-consistent regime.
A similar result holds for Poisson noise, with DC loss exhibiting similar behavior the negative Poisson log-likelihoee function when iú is far from the true signal value (see Appendix C).
4.2 LATE-STAGE OPTIMIZATION BEHAVIOR INTERACTION WITH REGULARIZZTION
We have shown that DC loss behaves similarly MSE/NLL when given estimate is far from the maximum likelihood estimate. Now, we consider how DC loss behaves close to the true solution.
Consider the simple case where the forward operator I and the noise model is Gaussian noise with variance o². Then, the ground truth free measurements y are exactly the true parameters 0" Let the noise vector be (10 o²). Then measurements m are simply m 0" +
In this setting, a "worst-case" for DC loss minimization occurs when the optimizer attributes the noise with the wrong sign, i.e. predicting -n n instead of n. Because DC loss enforces distributional (not pointwise) agreement, this estimate attains the same DC loss the ground truth. Then,
(6) Hence, in the worst case, our L2 error is ||2n||2- Our best case is, of course, zero L2 error. In comparison, using MSE the data-fidllity term leads directly to the noisy maximum likelihood
@worss-DC
(-n)
+ 0" + 2n
5

---

Published as a conference paper at ICLR 2026
True Image
0 Iterations
100 Iterations
1000 Iterations
10000 Iterations
DIP-MSE
Noisy Image
DIP-DC
Figure 3: Images denoised with DIP over time (a =
76
Top row: DIP with MSE loss; bottom
255
row: DIP with DC loss. DIP-MSE converges towards the noisy target: DIP-DC remains stable and does not display the same late-iteration degradation.
0 itaration(s)
100 Iteration]s]
1000 Iterationts)
10000 Iteration(s)
DIP-MSE
Miikeeg11.1 RDx
1S4-00 aigeraa
LEAPSL Le DMMr
DKA-N LrGin (MB
26
0.
14
94
DIP-DC
impricaa Qaeaity
000: Darftt
D--C Dmoiica
DINOS Darrayy
Velter
value
Figure 4: Histograms of the CDF values of measured data (s; values from Section 3.2 from per- 75
forming DIP with MSE (top) and DC loss (bottom) iteration increases (a
estimate, and a guaranteed L2 error of |||||2- Figure S summarizes the conclusion from this analysis, namely that DC loss can but is not guaranteed deliver a lower error estimate than MSE loss, and that the quality of estimate depends also on the prior assumptions introduced and optimization path.
In the following section, we demonstrate this behavior empirically different unsupervised inverse problem settings.
5 APPLICATIONS
5.1 DEEP IMAGE PRIOR (DIP) FOR GAUSSIAN DENOISING
popular method for unsupervised image denoising. In DIP, ran- domly initialized convolutional neural network is given fixed pure-noise input and trained, with an MSE objective, to reproduce the noisy target image. The network's inductive bias encourages the recovery of structured content, enabling unsupervised denoising, provided early stopping of training iterations is used to prevent overfitting the noise in the target image (Wang et al. 2023).
DIP (llynnov et al. 2020
Instead of the standard MSE loss, investigated using DC loss with DIP to avoid overfitting without necessitating early stopping. In our experiments, we followed the method (including the neural architecture) of Ulyanov et al. 2020 Appendix E gives the full experimental details.
In Figure 3 we show the result of the DIP method using our DC loss (DIP-DC) and MSE (DIP- MSE). It is clear that after 1,000 iterations DIP-MSE begins to overfit to noise. By 10,000 iterations, this has resulted in severe degradation of the image quality, with the noise spikes from the noisy image visible in the DIP-MSE image. In contrast, the DIP-DC image suffers such degradation.

---

Published as a conference paper at ICLR 2026
MSE Loss Trijectory
Loss Traeettrry
PSMA Tra eetettory
Peak: PSNR Noise Level
DIP-E DIP.DC
Poak PSWR (U5)
DI--MS
DIP-DC
DIP DC
Iteratione
402 2000
Recofians
2002
twodisse
(a) Loss trajectories of (b) Loss trajectories DC (c) PSNR over time (d) Peak PSNR achieved MSE loss during DIP. loss during DIP.
265
76
case.
as varies.
Figure 5: DIP-MSE and DIP-DC losses and metries. Error bars were calculated as 1.96x standard deviation on metrics over 5 random initializations of noise and network parameters.
True Image
100 iterations
1000 Iterations
2000 Iterations
5000 Iterations
10000 Iterations
NL -daam
5.0
2.5
0.0
DC-Adam
Figure 6: PET image reconstructions formed with three iterative algorithms (NLL-Adam, DC-Adam and MLEM) shown with increasing iteration number. The proposed DC loss (row 2) clearly avoids the noise overfitting with NLL as an objective, whether with Adam optimization (row 1) or with the MLEM optimization algorithm (row 3).
This effect is also seen in the histograms s; values (Figure and loss trajectories (Figure 5 for each case. Figure Pai demonstrates that the MSE loss tends to zero for DIP-MSE, while the MSE loss converges to a non-zero value for DIP-DC. Similarly, in Figure 5b) DC loss tends towards zero for DIP-DC, and converges to value away from zero for DIP-MSE.
Further, in Figure 5c we plot PSNR during optimization. As is standard in DIP, MSE requires carefully optimised early stopping avoid performance degradation. DIP-DC, by contrast, reaches its optimum PSNR and then plateaus without such intervention. Importantly, we compare against optimally carly-stopped DIP-MSE (i.e., peak PSNR), and still observe that DIP-DC achieves higher peak PSNR. Figure 5d shows this trend persists across noise levels, with larger gains at higher a. We hypothesize that MSE begins fitting noise before finer details can fully benefit from the inductive prior, whereas DC reduces this incentive and stabilizes optimization near the data-consistent regime.
show further results and trends with varied values, well results evaluating the effect of misspecifying the noise model.
In Appendix E
5.2 TOMOGRAPHIC IMAGE RECONSTRUCTION FROM POISSON NOISY MEASUREMENTS
Medical imaging features many noisy inverse problems, which a notable example positron emission tomography (PET). In PET image reconstruction, linear forward model A maps the

---

Published as a conference paper at ICLR 2026
1.5
NLL over iterations
1.50
DC loss over iterations
Normaized Root Mean Square Enror Iaveee
NLL-Adam
30
NLL-Adam MLEM
1.25
DC-Acam MLEM
NLL-Adom
0.8
DC Acam
MLEM
1.00
NRASE
0.6
NEL
23
0.75
0.50
0.2
0.25
2300
100C6
0.00
2000
4000
6000
8000
10000
2000
4000 6000
Iterations
8000
10000
Eerations
(a) NLL Joss trajectory.
(b) DC loss trajectory.
(c) Error over iterations.
Figure 7: PET reconstruction losses and metrics. As iterations increase, only DC loss avoids noise- chasing behavior that leads NLL-Adam and MLEM towards worse error.
True Image
NLL+TV
DC+TV
42.5
MLL-TV
26-
25.0
10
10
10"
12"
10"
(a) NRMSE vs. 8; DC+TV reaches lower minimum.
4
(b) Optimal-3 images; NLL+TV over-smooths, DC+TV preserves detail.
TV-reuularized PET image reconstruction with different data terms. Un- der NLL, the regularizer must both suppress noise-fitting and impose structure; with DC loss. noise- fitting already controlled, so the regularizer can concentrate on promoting desirable properties. This yields lower error and better detail at smaller 33 for DC+TV than for NLL+TV.
Figure 8:
unknown activity image i to the ideal sinogram y AA.. The detector then records counts m Poisson(y). introducing Poisson measurement noise that reconstruction algorithms must handle.
Clinical PET scans meet DC loss's assumptions of a known noise model and large number of independent measurements. We investigate the use of DC loss for PET image reconstruction from noisy measurement data. Existing unregularized algorithms optimize an image estimate with respect to the negative Poisson log- likeliiodd (NLL);
N
NLL(Ō m) = Σ m. In([A(]) + [AO]. + In(m:))
1=1
Clinically-used algorithms for PET reconstruction, such as MLEM moaximum-likelihon expecaaion-mmxiiizatton)) (Shepp and Vardi 1982), use early stopping on the number of iterations to avoid overfitting, which manifests as noise spikes and other image artifacts (Jonsson et al. 1998
Using a Brain phantom (Cocosco al.. 1997). we modeled the PET reconstruction process in 2D with single ring of a cylindrical PET scanner (Schramm and Thiclemans (2023). We then op- timized an image i with respect to the NLL and DC loss using Adam KKngmaa and Ba 2015) (with learning rate 5 x 10-3), as well as using the clinically-relevant MLEM algorithm for comparison.
In F 6 we show the visual results from each algorithm over 10,000 iterations. We see that both MLEM and NI.L-Adam overfit to noise in the image, with varying convergence rates. We further see that DC-Adam does not overfit the noise in the data. This is also observed in Figure 7a where the NLL of the DC-Adam image plateaus after 2,000 iterations, and in Figure 7b where the NLL-Adam and MLEM images do not converge to DC loss. In Figure 7c see that DC-Adam converges at the minimum error of NLL-Adam, although MLEM achieves a slightly better minimum error than both (likely due to the nen-tiiativ constraint in the MLEM algorithm).
In this setting the problem is less over parameetettrr than in Sections 5.1 and Appendix D so perfectly fitting the noise is harder. Even so, DC loss delivers clear advantage: it converges to its
8

---

Published as a conference paper at ICLR 2026
Vendor image
DC-Adam
NLL-Adam
Figure 9: An axial slice from reconstruction of real 3D PET dataset with DC-Adam or NLL-Adam, demonstrating the stable late-iteration behavior of DC loss as observed in synthetic experiments.
best solution and stays there, rather than briefly peaking and then chasing noise. In practice this yields stable late-iteration behavior without early stopping, fewer noise artifacts in highly iterated images, and better agreement with the assumed noise model. By contrast, conventional fidelity terms tend to overfit as iterations progress. Additional setup details and experiments the impact of over--araaman prerpparameteiizationn are provided in Appendix
5.2.1 REGULARIZATION WITH DC LOSS
Regularization is commonly used in medical image reconstruction to compensate for the noise-fitting tendency standard data-fidelity terms. We therefore augmented the PET image re- construction objective with regularization penalty, which introduces the regularization strength hyperparameter
E(O m) NLL(0 + + TV
For this experiment, we reconstructed from 4x lower count data while using an edge-preserving variant of total variation (TV) as the regularization term (details in Appendix [F4..
DC+TV achieved better quantitative accuracy and stronger noise suppression at its optimum 3 than NLL+TV, shown by Figure 8 Very notably,
with little delivers low NRMSE results. In contrast, as shown in Figure 8b) NLL+TV required substantially greater regularization counter noise fitting: the reconstruction with lowest NRMSE is noticeably smoother and less detailed than its DC+TV counterpart. Notably. the optimal 3 for DC+TV also was orders magniudee smaller than for NLL+TV (Figure Sa): this aligns with our hypothesis that DC loss doesn't 't place regularization in opposition to data-fidelity (see Figure 2
regularization, DC loss still
By curbing noise fitting the data- aeta-fidelity term, DC loss enables more favorable trade-off: weaker regularization suffices to suppress noise without sacrificing detail.
5.2.2 REAL PET FEASIBILITY STUDY
To assess the practical feasibility of DC loss on real measurement data, we applied it clinical 3D PET brain scan acquired on a Siemens Biograph mMR system. The dataset comprises over 70 million lines of response, and reconstruction uses full physical forward model including attenua- tion, detector normalization, scatter, and randoms. An axial slice comparing the vendor reconstruc- tion with late-iteration DC-Adam and NLL-Adam reconstructions (10,000 iterations, subsets) shown in Fig. 9 full reconstruction details are provided in Appendix F.5
In this clinical dataset, DC loss yields stable late-iteration reconstruction, whereas Nl.m-Adam dis- plays the characteristic high-frequency amplification commonly observed at large iteration counts. This experiment therefore demonstrates that DC loss is operationally feasible on real clinical PET data and behaves expected under realistic measurement and modelling conditions.
This experiment is intended as a first demonstration real-world applicability. In practice, the regimes where overfitting becomes most problematic are lower-dose acquisitions high- high-resolution reconstructions, where stronger priors are typically used. Those settings provide natural next targets for evaluating DC loss in
challenging scenarios.

---

Published as a conference paper at ICLR 2026
6 DISCUSSION
Across the settings we have studied, DC loss initially tracks pointwise losses but then shifts trajec- tory and converges once the measurements are statistically explained, at which stage, in contrast, further updates with pointwise losses mostly chase noise rather than signal. This behavior matches the theory in Section 4 once the set of CDF values is close to the uniform target, DC longer rewards per-sample noise fitting.
Crucially, combining DC with regularization, whether implicit (e.g., DIP) explicit (e.g., edge- preserving TV). yields stronger results than pointwise losses, because DC reduces the tension be- tween data-fidelity and regularization; the fidelity term stops pulling toward noise, and the regular- izer can preserve structure instead of fighting noise fitting.
As discussed in Section 4.2 DC loss effectively defines an equivalence class of solutions: any prediction that induces an (approximately) uniform distribution of CDF values across measurements attains low DC loss. This flattens the objective near good solutions. While this guards against noise-chasing, it does not by itself distinguish between plausible and implausible signals within the data-consistent regime; architectural priors or explicit penalties provide that structure.
We expect DC loss to be most useful (a) in over- peramaaramttri regimes where many viable solutions exist and noise fitting is a concern, and (b) when paired with priors that encourage interpretable or realistic outputs. In the main text, we have shown DC loss integrating well with DIP and handcrafted priors. We further demonstrate applicability to ID noisy deconvolution in Appendix and noisy image deblurring with plug-and-pu priors (Venkatakrishnan et al., 2013 in Appendix G also anticipate compatibility with score- based generative models for inverse problems (Song et al. 2022).
6.1 LIMITATIONS AND SCOPE
DC loss assumes independent measurements and known (or estimable) noise model per measure- ment, and it benefits from large enough set of measurements to reliably assess collective residual behavior. These conditions hold in many large-scale inverse problems (e.g., imaging, tomography, spatial sensing) but may limit applicability in small-data regimes or when noise is poorly charac- terized. Likewise, DC is not designed for settings where the ill ill-posedness of the forward operator, rather than noise, is the main challenge, such as inpainting or problems with large null spaces, where overfitting noisy measurements is not primary concern. For discrete noise models (e-g.. Poisson), exact uniformity of CDF values need not hold; the randomized probability integral transform could be employed (Appendix A.3). Computationally, DC loss adds overhead time relative to pointwise methods (Appendix H.. although in our experiments this was not bottleneck.
Finally, because DC loss tolerates a family of statistically valid solutions, it does not, on own, enforce structural properties such regularity, sparsity, coherence. As with pointwise losses, priors or regularizers remain important to steer reconstructions toward plausible signals.
6.2 FUTURE WORK
We focused on DC loss paired with non-learned regularization to isolate the behavior of the fidelity term itself. Appendix G shows that similar benefits also appear in a learned plug-and-play setting, suggesting that DC loss should extend naturally to learned priors. Further work should explore this integration more fully and study DC loss across additional operators, noise models, and problems.
7 CONCLUSION
We introduced distributional consistency (DC) loss, which reframes the data term in inverse prob- lems as statistical consistency with the noise model across many independent measurements in single noisy dataset. The central advance is that DC loss removes the incentive for noise-chasing in the data term, so optimization no longer rewards fitting particular noise realization and regulariza- tion can focus on structure rather than suppressing noise. Empirically, in DIP denoising and PET image reconstruction, DC loss reduced reliance on early stopping and improved
trade- offs, consistent with this picture. DC loss is simple to implement, compatible with standard priors, and provides practical data-fidelity term for many inverse problems with known noise models.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY
To ensure the reproducibility this work, further experimental details are listed in Appendices
D.2 E.T F. and [.4 Additionally, Appendix([ddetaiii the approximations used to enable practical
imp emeettatti of DC loss, with algorithm pseudocodes given as Algorithms 3 and Appendices
and B also give more mathematical derivations used support the formulation of DC loss.
Source code for the experiments is available at: httpos//gittuubb.cmm/eeeertt
Diszzibbttional--Cosssttteeee
ACKNOWLEDGEMENTS
The authors acknowledge support from the EPSRC CDT in Smart Medical Imaging [EP/S022104/1] and a GSK Studentship.
REFERENCES
Arridge. S., Maass, P., Öktem, O. & Schönlicb, C.-B. (2019) Solving Inverse Problems Using Data- Driven Models. Acta Numerica 28:1-174
Aster, R. C., Borchers, B. & Thurber, C. H. (2018) Parameter Estimation And Inverse Problems. 3rd ed. Amsterdam: Elsevier.
Balakrishnan, (1985) Order Statistics From The Half Logistic Distribution. Journal Of Statistical Computation And Simulation 20(4):287-309.
Baret, F. & Buis, S. (2008) Estimating Canopy Characteristics From Remote Sensing Observations: Review Of Methods And Associated Problems. In: Liang, S. (ed.), Advances Land Remote Springer.
Sensing, pp. 173 201. Dordrecht:
Bertero, M., Boccacci, P. & De Mol, C. (2021) Introduction To Inverse Problems In Imaging. 2nd ed. Boca Raton, FL: CRC Press.
Cramér, H. (1928) On The Composition Of Elementary Errors. Scandinavian Actuarial Journal 1928(1):13-74.
Cocosco, C., Kollokian, V., Kwan, R. K. & Evans, A. (1997) BrainWeb: Online Interface To A 3-D MRI Simulated Brain Database. Neurolmage 5(Suppl. 1):5425. In: Proceedings Of The 3rd International Conference On Functional Mapping Of The Human Brain (HBM 197), Copenhagen, Denmark.
Craig, I. & Brown, J. (1986) Inverse Problems In Astronomy. Bristol, UK: Adam Hilger.
Efremenko, D. & Kokhanovsky, A. (2021) Foundations Of Atmospheric Remote Sensing. Cham: Springer.
Elfving, T., Hansen, P. C. & Nikazad, T. (2014) Semi-Convergence Properties Of Kaczmarz's Method. Inverse Problems 30(5):055007.
Fan, I.., Zhang, F., Fan, H. & Zhang, C. (2019) Brief Review Of Image Denoising Techniques. Visual Computing For Industry, Biomedicine, And Art 2:Article 7.
Gourion, D. & Noll, D. (2002) The Inverse Problem Of Emission Tomography. Inverse Problems 1835):1435-146..
Gubbins, D. (2004) Time Series Analysis And Inverse Theory For Geophysicists. Cambridge, UK: Cambridge University Press.
Habring, A. & Holler, M. (2024) Neural-Networ--Based Regularization Methods For Inverse Prob- lems In Imaging. GAMM- M.MMM-Mittilungen 47:4202470004.
Heckel, R. (2025) Learning To Solve Inverse Problems End-To To-End. Deep Learning For Compu- fational Imaging, Oxford: Oxford University Press. doi: 10. (99X0ss/978019994777722.00.
11

---

Published as a conference paper at ICLR 2026
Jonsson, E., Huang, S. C. & Chan, T.F. (1998) Total Variation Regularization In Positron Emission Tomography. UCLA CAM Technical Report 98-48.
Kazantsev, D., Bleichrodt, F., van Leeuwen, T., Kaestner, A., Withers, P.J., Batenburg, K.J. & Lee, D. (2017) A Novel Tomographic Reconstruction Method Based On The Robust Stu- dent's Function For Suppressing Data Outliers. IEEE Transactions On Computational Imaging 3(4):682-693.
Kingma, D.P. & Ba, J. (2015) Adam: A Method For Stochastic Optimization. In: Proceedings The 3rd International Conference Learning Representations (ICLR 2015), San Diego CA, USA.
Kolmogorov, A. (1933) Sulla Determinazione Empirica Di Una Legge Di Distribuzione. Giornale Dell' Istittst Italiano Degli Attuari 4:83-91.
Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M. & Aila, T. (2018) Noise2Noise: Learning Image Restoration Without Clean Data. In Proc. 35th International Con- ference on Machine Learning (ICML) pp. 2965-2974. PMLR.
Llacer, J., Veklerov, & Nunez, J. (1989) Statistically Based Image Reconstruction For Emission Tomography. International Journal Of Imaging Systems And Technology 1:132-148.
Linde, N., Renard, P., Mukerji, T. Caers, (2015) Geological Realism In Hydrogeological And
Geophysical Inverse Modeling: A Review. Advances In Water Resources 86:86-101.
Louis, A. (1992) Medical Imaging: State Of The Art And Future Development. Inverse Problems 8(5):709-738.
Lucy, L. (1994) Astronomical Inverse Problems. Reviews In Modern Astronomy 7:31-50.
Martin, D., Fowlkes, C., Tal, D. & Malik, (2001) A Database Of Human Segmented Natural Images And Its Application To Evaluating Segmentation Algorithms And Measuring Ecological Statistics. In: Proceedings The 8th IEEE International Conference On Computer Vision (ICCV 2001), Vancouver, BC, Canada, vol. pp. 416413.4
McCann, M. T., Jin, K. H. & Unser, M. (2017) Convolutional Neural Networks For Inverse Problems In Imaging: A Review. IEEE Signal Processing Magazine 34(6):85-95.
Metzler, C. A., Mousavi, A. Baraniuk, G. (2018) Unsupervised Learning With Stein's Unbiased Risk Estimator. arXiv preprint arXiv: 1805. 10531.
Mohan, A., Venkatakrishnan, S. V., Gibbs, J. W., Gulsoy, B., Xiao, X., De Graef, M. &
Bouman, C. (2015) TIMBIR: A Method For Time-Space Reconstruction From Interlaced Views. IEEE Transactions On Computational Imaging 1(2)96-111.
Parker, R. L. (1994) Geophysical Inverse Theory. Princeton, NJ: Princeton University Press.
Pearson, S. (1938) The Probability Integral Transformation For Testing Goodness Of Fit And Combining Independent Tests Of Significance. Biometrika 30(1-2):134-144
Ribes, A. & Schmitt, F. (2008) Linear Inverse Problems In Imaging. IEFE Signal Processing Mag- azine 25(4):84-99
Schramm, G. & Thielemans, K. (2023) PARALLELPROJ Open-Source Framework For Fast Calculation Of Projections Tomography. Frontiers In Nuclear Medicine 3:Article 1324562 (2023).
Shepp, L. & Vardi, Y. (1982) Maximum Likelihood Reconstruction For Emission Tomography. IEEE Transactions On Medical Imaging 1222113-122.
Singh. R.R.D. Denker, A., Barbano, R., Kereta, 2., Jin, B., Thiclemans, K., Maass, P. & Arridge, S. (2024) Score- Bused Generative Models for PET Image Reconstruction. Machine Learning for Biomedical Imaging (Special Issue for Generative M0del5).547-585.
12

---

Published as a conference paper at ICLR 2026
Song, Y., Shen, L., Xing. L. & Ermon, S. (2022) Solving Inverse Problems In Medical Imaging With Score-Basee Generative Models. In: Proceedings Of The 10th International Conference On Learning Representations (ICLR 2022), Virtual Conference.
Tarantola, A. (2005) Inverse Problem Theory And Methods For Model Parameter Estimation. Philadelphia, PA: Society for Industrial and Applied Mathematics.
Ulyanov, D., Vedaldi, A. & Lempitsky, V. (2020) Deep Image Prior. International Journal Of Com- puter Vision 128(7):1867-11))
Venkatakrishhan, S. V., Bouman, C. Wohlberg, B. (2013) Plug-And-Play Priors For Model- Based Reconstruction. In: 2013 IEEE Global Conference On Signal And Information Processing (GlobalSIP), Austin, TX, USA, pp. 945-948.
Vogel, C. (2002) Computational Methods For Inverse Problems. Philadelphia, PA: Society for Industrial and Applied Mathematics.
Wang, H., Li, T., Zhuang, Z., Chen, T., Liang, & Sun, J. (2023) Early Stopping For Deep Image Prior. In: Proceedings Ofe 11th International Conference On Learning Reprexentations (ICLR 2023), Kigali, Rwanda.
Zhang, K., Li, Y., Zuo, W. Zhang, L., Van Gool, L. Timofte, R. (2021) Plug-and Play Image Restoration With Deep Denoiser Prior. IEEE Transactions on Pattern Analysis and Machine In- telligence, P-(99):1-1.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
ORGANIZATION
This appendix organized as follows.
Appendix presents additional mathematical details supporting the derivation of DC loss.
Appendix is more mathematically involved, and concerns the implications between whether low DC loss implies low signal error and vice versa.
Appendix derives tail approximations used in the practical implementation of DC loss, and gives the pseudocodes for implementing DC loss.
Appendix presents an illustrative example using DC loss to mitigate noise-fitring in ID signal deconvolution, with results that support the conclusions of the main paper in an additional application area.
Appendix Fe relates experimental details and further experimental results for the DIP section of the main paper, including the effect of a mismatched noise model.
Appendix presents analogous content for the PET section of the main paper.
Appendix presents an example application of DC loss
glug-and-play framework.
Appendix presents data on the time efficiency of DC loss.
Appendix declares the usage of LLMs as required by ICLR.
A MATHEMATICAL DERIVATION OF DISTRIBUTIONAL CONSISTENCY LOSS
A.1 PROBABILITY INTEGRAL TRANSFORM (PIT)
We will justify the statement in Section 3.2 that if the model parameters are close to the true parameters 0", then the values 8; ;= (f( m.) are approximately uniformly distributed on [0, 1]. This relies the classical probability integral transform (Pearson 1938), which state and prove below for convenience.
Lemma A.1 (Probability Integral Transform). Let Z be continuous random variable with CDF Fz. Then the transformed variable
U -= F2(Z)
(9)
is uniformly distributed on the interval [0, 1].
Proof. Let u E [0. 1]. We compute the CDF of U:
IPU S u) P(Fz(Z) S u) P(Z s (F7(()) (z(FF((u) u.
(10)
(1I)
(12)
Here, F7 (u) denotes the (generalized) inverse CDF of Z. which exists because Fz is continuous and strictly increasing. The last equality follows from the definition of the inverse CDF.
Thus, the distribution function of U is PUU S u) u. which the CDF of the uniform distribution on [0, 1]. Therefore, U Uniform(0, 1].
Application. In context, for each i, let be measurement assumed to be drawn from the continuous model distribution (0*),, which has CDF F. Then the probability integral transform implies:
F, m.) ~ Uniform j0, 1].
(13)
θ 0", then s; (, m,) is approximately uniformly distributed as well (assuming f and D sufficiently well-behaved). This statement is made more precise in Appendix B
14

---

Published as a conference paper at ICLR 2026
A.2 LOGIT TRANSFORM OF UNIFORM DISTRIBUTION
Let U ~ Uniform(0, 1) and define
U
Z := logit(U) log
(14)
1 1-U
We show that Z ~ Logistic(O,, 1) by deriving its CDF directly.
By definition of Z, we compute its CDF using the transformation method:
Fz(2) = (Z S 2) P log (T-U)
U
Sz
(15)
Apply the monotonic function exp to both sides:
( U
Fz(x) =
T set)
(16)
Solve the inequality for U:
U
E² US
1 Se c
(17)
1
Thus,
e²
=2
US UUS
(18)
1 e²
Since ~ Uniform(O, 1), we have
e²
Fz(2)= 1 e
(19)
1 + +
The CDF of Z matches that of standard Logistic(O, 1) distribution. Therefore, logit(U)) Logistic(O,
(20)
A.3 APPROXIMATION OF THE CDF OF A DISCRETE RANDOM VARIABLE
The probability integral transform in subsection A. applies to continuous noise. For discrete noise, Fz is a step function, so the naive PIT is not exactly uniform. In this work we use the naive approach, which is often adequate in practice; we also note a simple randomized variant that would make the transform exactly uniform if desired.
(A) Naive (non-randomized) PIT can be close to uniform when support is rich. Let Z be itd. Z. Define s; = F2(m)) and the
discrete random variable with CDF Fz and pmf pz, and let m, empirical CDF
1 N 1(s S u).
Σ F.(u) = N
(21)
As -> xo, F. converges to the distribution of (2( which places mass at the jump values of Fz- When 7 has many support points with relatively balanced probabilities, the mass of (Z) is spread across many levels in [0, 1]. and the empirical histogram of {st} can look visually close to uniform (e.g. see Figure FT). This heuristic supports the empirical usefulness of DC loss with discrete noise even without randomization.
(B) Randomized PIT makes the transform exactly uniform (discrete case). Define an indepen- dent U ~ [0 1] and the randomized PIT
grnd = Fz(Z-) + Up (Z),
where Fz(=T) = lim (z
(22)
15

---

Published as a conference paper at ICLR 2026
A direct calculation shows S
Uniform [0, 1]. Indeed, for any u [0,1],
P( I u)
Σ P(Z 2) IP 2))((F2(2 Fz(2") UU pz(z) u Z ==
(23)
Σ pz(z) (u F2(=-))
1uu S Fz(2)}
(24)
pz(2)
where (+)+ is the positive part and we used that the jumps of Fz partition [0. 1j. Applying the (monotone) logit map gives
Rrud :- logit(S Logistic(O,, 1).
(25)
Therefore, at the population level the DC loss is exactly zero.
Practical takeaway. Our experiments used the naive PIT for discrete noise (for simplicity). If exact uniformity (and hence exact logisticity after the logit map) is desired at the population level, one could instead apply the randomized PIT above with other changes to the framework.
16

---

Published as a conference paper at ICLR 2026
B RELATIONSHIP BETWEEN DC LOSS AND ERROR
Notation. Notation follows the main text. We briefty restate the pieces used below. Let 0 denote the ground-truth parameter. the forward operator, and y -(00) E R the true signal vector. For any candidate parameter 0, write y f()). For each i {l..... N), the observation my (the ith component of m E IR.-) is drawn from parametric family D; (3:)) with continuous CDF F.(: y:)- Define the probability integral transform (PIT)
81 ( Famms 0:) E (0, 1),
(26)
and set
r; r:( logit(s; (0)) E R.
(27)
Let the empirical residual-score measure be
N
P (r:(0))
Σ (.)
(28)
N
iid Independently, draw Zi.... T....,ZM Zm ~ Logistic(O, 1) and define the empirical logistic measure
M
plog M
1
Σ
òz,
(29)
M
==1
The (empirical two-sample) DC loss is
( Wi((pmm(1n) plac)
(30)
Signal errof is measured by the average squared error:
Erra (0) N
Σ U1 yx)
(31)
B. ZERO SIGNAL ERROR NEAR-ZERG EMPIRICAL DC LOsS
Proposition B.1. (0) O, then for all i, and the following hold:
(a) Expectation identity. For each and any bounded measurable test function f.
B (r(()))) EE [Ez=Logistte(UUn]/[(
(32)
(b) Empirical two-sample convergence. As N and M 20,
as In particular, this holds if M N and both tend to infinity.
(0) wi(( plag)
(33)
(c) Two-sample rate in expectation. There exists universal constant C oo such that for all
N, 21 1,
E (N. < c
(34)
M
Proof. Since Errz(0) = O, we have 0J = for every i. By the probability integral transform (with F; continuous),
s, (0) [,mm yı) Uniform(0, 1].
(35)
R is measurable and strictly increasing, the image of uniform variable is standard logistic; hence
Because logit: (0, +
ra i logit( s; (0)) ~ Logistic 1),
(36)
In this section, signal error is taken mean the error on the forward-modeled estimate.
17

---

Published as a conference paper at ICLR 2026
which yields (a): for any bounded measurable f,
[r:(r:())] [==01apisti(001:][(((
(37)
For (b), write plog for the population logistic distribution and apply the triangle inequality: (r.(0)))
Wi(P (m((0)) Phtt) I (Wi(Pir.())) pla)) + Wi(plos, Plot).
(38)
Conditionally on {ūr = ). the sample ( is i.i.d. from plos with finite first moment, and similarly {2z) M is i.i.d. from plog and independent. By standard results for empirical measures on R
Wi( (i(P())(0)) plog) (nx(0)))
O as N + 00; as M + 00;
(39)
W Wi( Plan)
(40)
hence (0) a.s. 0 as N. 000
For (c), take expectations and use known one-dimensional bounds for the empirical Wasserstein- distance (via the quantile representation and the Dvoretzky- -eefeeieeeeeffeeee inequality):
E ((Wi((((. plos)]
C
D(Wi(Plos, p S
C
(41)
VN'
VM
Combining with the triangle inequality gives
[[CN,ar(O]] S C
+
(42)
VM
B.2 SMALL SIGNAL ERROR = SMALL DC LOSS
We now argue that when the forward-model error small, the empirical DC loss must also be small. Intuitively, if Ú: is close to for each i, then the corresponding PIT values s, are close to the uniform law, hence their logit- logit-transfom r,( is close to the logistic law, Icading to small Wasserstein distance.
Proposition B.2. Suppose that for each i the CDF F.(- ()) is Lipxchitz-comtinuuus in 1, with constant K4 2 in the sense that
[i(m 7) - Fi(m| v:)l S K2/1 - ysl for all m in the support.
(43)
Then the residual scores r.) satisfy
(F, û:)) - logit( (Fim 9:)) I Li 10: Vil,
(44)
where L; is a finite constant depending on K, and on the boundedness of the logistic derivative on the relevant interval
Consequently, writing A = U, - Ys. the empirical residual-see score measure obeys
wi( Wi(P(r.(0)): P(T.(0))) S I L|A.s
(45)
Corollary B.3. If Errz(() = E. is small, then the expected DC loss is also small. In particular,
E [[N.M((]] s (VERTA VM
(46)
where constant depending on the average Lipschitz constants E. La and the logistic ref- erence distribution.
Proof sketch. The Lipschitz assumption transfers small perturbations the signal values i y: to small perturbations in the PIT values, and hence (by smoothness of the logit map on compact intervals) to the residual scores. Averaging and using the definition of the Wasserstein- distance yields the stated bound. Taking expectations and combining with the standard empirical-process bound for plud relative to its population limit then gives the corollary.
"Since logit has derivative 1/(s(1 - s)), it is Lipschitz on any compact sub-interval (e, - C (0, 1). practice, tail values contribute negligibly Wasserstein-1, this suffices.
18

---

Published as a conference paper at ICLR 2026
B.3 DOES SMALL DC LOss IMPLY SMALL ERROR?
The converse direction is more delicate. In general, while small DC loss does imply some bounds on the signal error, it is not true that DC loss tends zero that signal error must also tend to zero. DC loss is global distributional criterion: it requires the logistic residual values to be statistically indistinguishable from the logistic distribution, but this is only in aggregate and doesn't 't place strong constraint on individual data points.
For example, in the Gaussian nouus identity model discussed in Section 4.2 the estimate = - + produces residuals with the same distributional law as the true 0', and hence yields nearly zero DC loss, despite incurring La error || yl12 = ||2n||2. Thus low DC loss can coexist with high estimation error.
To obtain an implication of the form
((0,) 0
Errg (0,) 0 for a sequence of estimates (0,),
would need impose additional conditions, such as: (i) identifiability of the forward model, (ii) additional constraints to DC loss introducing uniqueness for the distributional match, or (iii) the of regularization to select among the manifold of DC-loss minimizers.
In short, while small signal error always forces small DC loss, the reverse is not automatic: DC loss controls distributional consistency, not pointwise closeness, and only under stronger structural assumptions can the two be shown to coincide exactly.
Remark (role of regularization). DC loss enforces a distributional match: many parameter set- tings can produce logit residual values that look logistic, so DC loss by itself may not pick a unique solution. Regularization (implicit or explicit) is therefore essential: not as a penalty that trades off data-fidelity (as with MSE/NLL), but selection rule within the set of DC-consistent solutions (see Figure 2). This lens explains why DC loss can yield either higher or lower estimation error than MSE, depending on the regularizer and optimization path. In practice, across the applications studied, DC loss performed reliably with standard forward models and regularizers. A plausible ex- planation is that, far from the MLE, DC loss behaves similarly pointwise losses (see Section [4.). guiding optimization toward comparable estimates; near the solution, regularization selects among the (near- (near-)zero-DD) loss candidates in way that aligns with prior structure.
B.4 FROM SIGNAL ERROR TO PARAMETER ERROR
In this appendix we have measured error in the signal space, via
NV
Errz (0)
Σ 0 v² N
y -= y f(0
This choice is natural because both the DC loss and pointwise losses are defined directly on the data, and allows us to unlink the choice forward model from the error analysis. Nevertheless, one ultimately seeks control of the parameter error ||@ 11
A simple link be stated under mild conditions the forward operator If f θ RN is injective and locally bi-Liischitz around 0*, then there exists a constant C' > such that
| 0 < y yll2 = C CV (r)
(47)
Thus small signal error implies small parameter error. In particular, our results showing that small signal error leads small DC loss extend, under such regularity assumptions, to the conclusion that parameter estimates with low error also have low DC loss.
'Note DC loss will never reward an estimate that produces residuals that are extremely far from the true signal, as it still penalizes residuals that are many from the noisy signal value. As a tends to zero, the bound that DC loss places on signal error will also tend zero.
19

---

Published as a conference paper at ICLR 2026
C EVALUATING DC LOSS IN PRACTICE: CALCULATING THE LOGIT FUNCTION OF A CDF VALUE
In subsection we motivated the direct approximation of logit(s) to avoid numerical precision difficulties sequentially caleulating and applying the logit function. In this appendix, we derive ap- propriate tail approximations to logit(s) (s) for the cases of the Gaussian, Poisson and clipped Gaussian distributions. These approximations are useful in settings where is close to 0 or 1 and separately evaluating the logit function and CDF is numerically unstable.
C.1 GAUSSIAN DISTRIBUTION
C.1.1 APPROXIMATING LOGIT(s) FOR THE TAILS OF A GAUSSIAN DISTRIBUTION
Consider drawing a measurement from Gaussian distribution NWj o²). Let
m
1
db
(48)
where & denotes the standard Gaussian CDF, and
logit(s) log
(49)
1 s
We derive asymptotic approximations for logit(s) (s) in the extreme tails, where the standardized variable:
Define
m
(50)
RIGHT TAIL: 2 >> (I.E,,
Using a classical Laplace approximation for the upper tail of the Gaussian CDF,
- ((2)
2
as 2 00,
(51)
2V2 2m
we get:
8
1 1-8 s
22 + log + log 2
logit(s) = llog
= =-log
2 log(1 - ss ss
(52)
1-8
2
Thus,
m ))
1
m - 1)
))
2
m - 1 + log 2 as a >
logit
+ log
(53)
2
LEFT TAIL: = & (I.E,, R 0)
For large negative use the lower tail approximation:
(<2) R3
1222
as + -00
(54)
1T|V2T
Then,
z² - -log [g| - llog 2
logit(s) = log(s) - log(1 - 8) ne log(s) ~
(55)
2
So,
logit
m
~
m 9
2
log
m 1
log 2 as m < 9
(56)
2
0
0
20

---

Published as a conference paper at ICLR 2026
Algorithm 2 Distributional Consistency Loss for Gaussian Noise
Require: Parameter estimate 0, observed data m, standard deviation 0, number of measurements
N, tail threshold 5
1: y + f())
D Predicted signal
2: Initialize empty vector of length N
3: for i to N do
4: 24 f- m
b. Standardized residual
5: 24 > then
6:
Tj f 2 + log(z;) + log 2
> Right tail approximation
7:
else if < - then
8:
Ti +
log(1z))) - log v 2n
> Left tail approximation
9:
else
10:
8; + ((2i)
A Evaluate Gaussian CDF
ri log 1-88
51
Standard logit transform
12: end if
13: end for
Logistic(O, 1) 15: Sort r and u in ascending order
14: Sample
Generate N samples from reference distribution
16: L L+* 1/2 EE [ra u,l
Wasserstein- distance
return
SUMMARY Let m m- then:
z²
+ log + log V
*> 0,
2 2²
logit (9(2)) x
log |2| log 2
< (
(57)
((2)
log
otherwise
((2)
C.1.2 PSEUDOCODE FOR CALCULATING DISTRIBUTIONAL CONSISTENCY LOSS WITHI
GAUSSIAN DISTRIBUTION
Algorithm 2 gives a full pseudocode for calculating the distributional consistency loss when Dz(g) =
N N(g, g²)
C.2 CLIPPED GAUSSIAN CDF
We extend our logit-CDF approximation to the case of a Gaussian distribution clipped to the interval [0, 1]. This arises in applications where measurements are inherently bounded, such as normalized image intensities. To handle this, we define smoothed approximation to the CDF that matches the Gaussian in the interior but transitions linearly near the boundaries to maintain continuity and gradient stability.
Let (m; ú. o) be the Gaussian CDF centered with standard deviation a, and width of two linear regions near and respectively. We define a modified CDF F(mlý) for [0, 1] by:
define
$(G 0) e
0 m < c,
((mli))
v(m;; o),
e e < -6
(58)
Φ(1 - e ). + +01 - -); m
1 m $1.
This construction ensures a smooth approximation to a clipped Gaussian CDF while avoiding dis- continuities in the derivative.
21

---

Published as a conference paper at ICLR 2026
For measured values that exactly 0 or uniformly resample the values be within [0, c] or [1 e, 1], respectively. This avoids degenerate gradients and provides a consistent treatment of boundary values.
To compute the logit of the CDF, we apply:
F(mlů) - F(mlū)
logit( (gii((((ml))) log
(59)
In the linear regions near 0 and 1, F(mlý) behaves approximately linearly with respect to m, and no special tail approximation is needed beyond standard numerical clamping (we clamp to small interval [Chard Chard] to avoid instability). In the central region of m values, the logit is computed directly, with the tail approximations given in Equation 57 where m and ý are far apart.
This approach allows for stable and differentiable evaluation of logit(F(m))) (mml)) across the full range of m E [0, 1], including at the boundaries,
C.2.1 PSEUDOCODE FOR CALCULATING DISTRIBUTIONAL CONSISTENCY LOSS WITH CLIPPED GAUSSIAN DISTRIBUTION
Seborithm@giveer full pseudocode for calculating the distributional consistency loss for Gaussian distribution clipped to [0, 1..
C.3 POISSON CDF
C.3.1 POISSON-GAMMA DUALITY
We consider how to efficiently evaluate the CDF of a one-dimensional Poisson random variable at measured value, i.e. compute 8; values defined in Section 3.2 Suppose N is drawn from a Poisson distribution with unknown mean, and ÿ is a predicted mean value. We consider the following Poisson-Gamme duality lemma that shows we may equivalently use an appropriately defined Gamma CDF instead of a Poisson CDF:
Lemma C.1. For all I > 0 and m N.
s(ū, m) 1 ((ū,
(60)
where
9 t(g. m) =
0 1 m! du
s(). m) =
(61)
k!
k=0
Proof. Let X ~ Poisson(j)) and Y ~ Gamma m + 1, 1). Then:
s(ğ, m) ( < m) t(ό. m) = P(Y < ()
(62)
(63)
To justify the identity P(X- < m) = P(Y > G). we appeal to the structure homogeneous
Poisson process (N())) with unit rate. Then:
The time of the (m++1) 1)th event follows a Gamma distribution: y Gamma(m + 1, 1).
The count X = Ng), the number of events up to time ÿ. is Poisson distributed with mean g.
By construction, the events of the Poisson process satisfy:
(N(ù) S m) = (Y>
(64)
since if the (m + 1) event has not occurred by time ÿ. there can be at most events up that time. Therefore,
TP(X & m) = P 0)
(65)
22

---

Published as a conference paper at ICLR 2026
Algorithm 3 Distributional Consistency Loss for Clipped Gaussian Noise (with Tail Approxima-
tion)
Require: Parameter estimate 0, observed data m, standard deviation o, number of measurements N, ramp width e, tail threshold
1: y + -(()
> Predicted signal
2: Initialize empty vector length N
3: for i = to N do
4:
q < 1i
5:
f- m,
6:
if m then
7:
f- Uniform
Perturb endpoints to avoid discrete effects
8:
else m then
9:
m f- Uniform(1 e, 1)
10:
end if
11:
m=2
Standardized residual
12:
& x 1
13:
C1-c t +
m then
15:
f m Ce
D Linear ramp left
16:
else m
then
17:
+ c1- +
m- (1-1)
(1 -c-
> Linear ramp right
else
19:
+ $(2)
Standard Gaussian CDF in central region
end
21:
if s < 8 then
Tail approximations to Logit of Gaussian CDF
2.2 + log([=]) + log v 2
22:
Fi +---
2
else if >
then
ri 2 log((=|) log 2m
24:
25: else
26: Ti + log (1-x))
end if
28: end for
29: Sample Logistic(O,, 1)
> Reference distribution
30: Sort and u in ascending order
L L+ 1 EN Ir. usl
Wasserstein- distance
32: return
Substituting equation 62 equation
63
and equation 65 gives:
s(A, m) = t(D. m)
(66)
claimed.
As a consequence of the Poisson-Gamma duality lemma, we may calculate in Section 3.2 by evaluating the CDF of Gamma(m + 1, 1) random variable. This beneficial because Porchrch does not support evaluating the CDF of a Poisson random variable, but does support evaluating the CDF of Gamma random variable. (While specifying a Poisson CDF manually is possible, is simpler to make use of optimized Torch functions where possible).
C.3.2 APPROXIMATION OF LOGIT(s) FOR POISSON CDF TAILS VIA GAMMA DUALITY AND TAIL EXPANSIONS
Building on the Poisson- Poisson-Gamma duality s(ÿ, m) -(ū, m), where
t(j. m) (d u m
m! du P(Y < i), Y Gamma(m + 1,1),
(67)
23

---

Published as a conference paper at ICLR 2026
Algorithm 4 Distributional Consistency Loss for Poisson Noise
Require: Parameter estimate 0, observed data m, number of measurements N, tail threshold
1: y + =(())
> Predicted signal
2: Initialize empty vector of length N
3: for i = 1 to N do
4: f max 0)
Ensure non-negativity
5: < 1 GammaCDF((;aa
1,
Poisson posterior tail probability
6:
i S c then
7:
rj
log(g)) q log [(m 1)
Lower tail approximation
8:
else 8 2 1 then
9:
F3 m log(() + q log [(m + 2)
Upper tail approximation
10: else
11:
T1 + log
1-8
>> Standard logit transform
12: end
13: end for
14: Sample u Logistic (0, 1)W
> Reference distribution
15: Sort and u in ascending order
L + 1/2 + Ir; uil
Wasserstein-1 distance
17: return
we now derive asymptotic approximations for logit(s) in the extreme left and right tails, i.e., when
ú >> or ij & m, respectively.
LEFT TAIL: ( >> m, SO Rs 0)
In this regime, the CDF is small, and we approximate by the leading term of the Poisson proba- bility mass function:
e
(68)
m!!
Apply logit, we obtain:
logit(s) log s m log 1 -g- --logm) ml
(69)
RIGHT TAIL: (6 < m. SO x= 1)
In this case, the upper tail 1 s t(ū m) is small. We approximate the incomplete Gamma integral by truncating its series expansion after the first term beyond the threshold m:
g
t(i. se -
(70)
(m 1)!
Then,
logit(s) R0 log(1 - 8) Rs log c-- gm+1
=-(m log + + log(m + 1)! (71)
(m + 1)!
SUMMARY:
The full approximation for logit(s), where = P(X < m) and X ~ Poisson(j), is given by
m logog ÿ - 1 - log m!!
if > m,
logit(s(), m)) x
- (m log ύ + ÿ + log(m + 1)! if m, <(..m)
m, otherwise
(72)
log I-
C.3.3 PSEUDOCODE FOR CALCULATING DISTRIBUTIONAL CONSISTENCY LOSS WITH POISSON DISTRIBUTION
Algorithm 4 gives a full pseudocode for calculating the distributional consistency loss when D,(0) =
Poisson(y) ())
24

---

Published as a conference paper at ICLR 2026
D ID DECONVOLUTION APPLICATION
We include the following example application to illustrate the use of DC loss in simple non- imaging context.
D.1 ILLUSTRATIVE EXAMPLE: DECONVOLUTION UNDER GAUSSIAN NOISE
Predicted signals (x E [0.5, 0.55))
Histograms
True signel
1D
MSE traintrindd
Signal
DC-trained
Emrircall probatiity
Noisy observations
0.50
0.51
0.52
0.53
0.54
0.55
0.0
0.2
0.4
6.6
1.9
CDF valus
Convolved signals (x [0.5, 0.55))
MSE Loss Trajectory
DC Loss Trajectory
00200
25
0.5
True signal MSE trained
00175
PSEtrraee
DC-ornned
2.0
MSE trannd
DC-tramd
Sigal
-0.5
0.0
Ang
DC-trained
Nalsy abservations
00125
00150
0.0100
1.5
0.0075
1.0
-1.0
0.0000
-1.5
1040
1:5
0.0055
0.0000
0.0
heration
Iteration
(a) Top: predicted deconvolved signals estimated with MSE and DC loss. Bottom: predicted deconvolved signals after reapplying convolution with the known operator H.
(b) Top: histograms of CDF values under each model's implied noise distributions. Bottom: training loss curves evaluating cach of MSE and DC loss on MSE- trained and DC-trained predictions.
Figure DI: 1D deconvolution with known Gaussian blur and noise level. MSE inversion amplifies noise artifacts, while DC loss yields a cleaner, artifact-free reconstruction.
To illustrate DC loss in classic inverse problem, we estimate 1D signal O* from blurred and noisy measurements
m HO + 17.
(73)
where H is a known Gaussian-blur convolution and v ~ _O o I). A single ground-truth signal, composed of low- and high-frequency sinusoids, is forward modeled and sampled N 500 discrete timesteps to produce noisy observations. We parameterize 0 directly as free vector of length N and optimize under two objectives: the standard MSE || H7 m||2 and our DC loss. To
demonstrate the properties of both losses, both reconstructions use Adam (Kingma and Ba 2015) for 20,000 iterations with learning rate 10-3
Figure Dla shows that the MSE-trained solution (red) overfits by amplifying small, sigh-frequency noise producing pronounced oscillations. In contrast, the DC-trained solution (blue) remains smooth and closely tracks the true signal (green dashed).
In Figure DTb] the top row shows histograms of CDF values under each model's implied Gaussian noise distribution: the MSE-trained fit collapses its histogram around 0.5 (indicative of overfit- ting), whereas the DC-trained remains approximately uniform (distributionally consistent). The bottom row shows the loss trajectories. In the MSE-loss plot, the MSE objective for the MSE-trained reconstruction converges towards zero, while the DC-trained reconstruction' 's MSE plateaus above zero. In the DC-loss plot, the DC objective converges
EuwLogistic(a..) [|u]] In 1.386
(74)
for the MSE-trained reconstruction (Balakrishnan 1985) (the expected distance when all CDFs col- lapse to 0.5) whereas the DC-trained reconstruction's 's DC loss converges near zero, avoiding noise amplification.
25

---

Published as a conference paper at ICLR 2026
The rest of Appendix D provides implementation details and additional experiments varying the number measurements and the noise level.
D.2 EXPERIMENTAL DESIGN
To investigate DC loss for the task of deconvolution, true signal was generated as ((x) sin + 2 sin (40Tz) and uniformly sampled at N = 500 points between 0 and 1.
To simulate degraded observation, the signal was blurred using a ID Gaussian kernel defined in domain units. Specifically, Gaussian kernel in index-space with iideex space blur = 1.0, correspond- ing signal-space blur of blur 0.002 was applied with width 31 points (i.e. 15 neighbors of the central point on each side). The resulting kernel was applied by convolution with reflective boundary conditions.
Finally, additive Gaussian noise with standard deviation Onise 0.1 was added independently to each measurement:
m, (HO)); + Th.
TK N(0, o @uoisee)
(75)
The deconvolution task was then to recover the original signal (pre-convolution) from the noisy measurements.
For each experiment, Adam (Kingma and Ba 2015) with a learning rate of 0.005 was run for 20,000 iterations. vector of zeros was used the starting set parameters. Experiments were conducted with a 24GB Nvidia GeForce RTX 3090 GPU; running deconvolution with MSE and then DC loss took ~ minutes (with many experiments able to be run parallel).
Except where otherwise defined, we took Onoise
0.1, Vblur = 0.002 and N = 500.
D.3 ASSESSING DISTRIBUTIONAL CONSISTENCY OF THE TRUE SIGNAL
We investigated the assumption that the true clean image should theoretically induce a uniform histogram of CDF values (asymptotically). In Figure [2 we compared evaluating the DC loss on the true convolved signal and on the noisy convolved signal for a varied number of measured datapoints N.
For high N, we see that evaluating the DC loss on the convolved true signal yields an approximately uniform distribution of CDF values (and correspondingly low DC loss value), while evaluating the DC loss on the noisy image yields histogram with peak at 0.5 (and correspondingly high DC loss value of approximately In 4 see Equation 74). For lower values of we obtain higher average DC loss values and less uniform histograms for the true convolved signal, although still more uniform than for the noisy signal.
D.4 VARYING NOISE LEVEL
In Figures D3 D4 and D5 we investigated varying the noise applied to the convolved signal (with the values noise 0.0T, 0.05 and 0.2 respectively). We observed that at the lowest noise level, both loss functions resulted in near-perfect fit the true signal, but only the DC-trained prediction avoided noise amplification effects in the estimate of the pre-convolved signal. This trend was seen more dramatically the noise level was increased. The fit to the true convolved signal worsened for both loss functions, but only the MSE-trained vector experienced extreme noise amplification in the pre-convolved signal (as a result of overfitting).
The loss trajectories and CDF histograms observed in these cases were consistent with those seen in
Section D.
26

---

Published as a conference paper at ICLR 2026
CDF histogram clean image
histogram of noisy Image
(DC loss value: 0.3402 = 0.2166)
(DC loss value: 1.3932 0.2450)
15
Empirical density
Empirical density
Uniform([0,1]) PDF
Uniform:(0,1]] PDF
10-
Probabiity
1.0
Probabiiity
5-
0.0
0.0
0.2
0.4
0.6 values)
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
(s) values)
(a) N 100.
histogram clean image (DC lass value: 0.1165 + 0.0772)
histogram of nolsy image (DC loss value: 1.3861 0.0749)
incpinical density Unifarm([0.1]] PDF
10-
Probabiity
Probability
0.5
Empirical density
5
Un
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
CDF values is values)
CDF values (s) values)
(b) N 1,000.
CDF histogram clean image
CDF histogram of noisy image
(DC loss value: 0.0352 = 0.0198)
(DC loss value: 1.3883 0..028)
15
Empirical density
Uniform/[0011) PDF
10-
Probabiltty
Probability
0.5
Empirical density
5-
Uniform([o,11/
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
CDF (s) values)
values (s) values)
(c) 10,000.
CDF histogram clean image
histogram of noisy image (DC loss value: 1.3864 0.0066)
(DC loss value: 0,0108 = 0.0058)
1.0
Probability densty
15
Empirica al density Uniform([O.1]] PDF
10-
Probability
0.5
impllrccal density
5
Uniform(jo,1]
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
1.0
CDF values (s; values)
(s values)
(d) N 100,000.
CDF histogram clean image
histogram of noisy image (DC loss value: 1.3862 0.0020)
(DC loss value: 0.0034 = 0.0021)
1.0
15
Empirical density
Uniform[[o.1] PDF
10-
Probabiity
0.5
Probability
Emp Empirical rical density
5-
Uniform([0.1] PDF
0.0
0.0
0.2
0.4
0.6 values)
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
(s values)
(c) N 1,000,000.
Figure D2: Investigating the DC loss values obtained and the uniformity of CDF histograms in the ideal and worst cases of evaluating the DC loss against the true convolved signal noisy convolved signal respectively. Subplots show the effect of increasing the number of measurements from = 100 N 1,000,000. Uncertainty values are given 1.96x the standard deviation over 100 evaluations of the (stochastic) loss on different instantiations of noise.
27

---

Published as a conference paper at ICLR 2026
Final CDF-value Histograms
MStAramed
DC-mmee
3.0
MGC-treine trained No ny obsurvatiios
Emplitcal pmbaaillyy
Uniform
0.5
53000
0.0
0.0
0.2
0.8
1.0
0.48
0:30
9.22
9:4
Trajectory
Loss Trajectory
3.2
Convolved signalc ix 10.45, 0.553)
0.001550
trained
1.75
1.50
Sgyst
3.0
0.5
DC-rrines
No sy obsevadiine
aat 0.000075
0.000100
1.00
1.25
0.0
-0.5
6.000025
0.25
-10
0.000000
5020 10000 :5000 20000
0.00
5000
10002 15020 20000
-12
Har aoon
Serution
0.54
(b) Top: histograms of CDF values under each model's implied noise distributions. Bottom: training loss curves evaluating each of MSE and DC loss on MSE- trained and DC-trained predictions.
(a) Top: predicted signals estimated with MSE and DC loss. Bottom: signals after convolution with the known operator H.
Figure D3: Results for ID deconvolution with known Gaussian blur with Obtur 0.02 and noise with noise 0.01.
Rinal CDF-value Histograms
Predicted signals (x [0.45, 0.55))
Empirtal proceiitty denetty
MSE-trinnd DCtraned Unilsm ret
True signal
1.0
MSE-trained
DC-trained
Sanaa
Naisy abservations
-0.5
-1.0
-1.5
0.0
0.46
0.48
0.50
0.52
0.54
Trajectory
Trajectory
Convolved signals (x E [0.45, 0.55])
0.0000
1.75
True signal MSE-trained
1.50
1.25
0.5
DC occitrined
MBEcrrined
1.00
Signal
Noisy observations
# 0.00s
DC-iremed
0.75
00005
025
-1.0
-1.5
00000
5000
10t0c 15000 20003
0.00
5000 10000 15000 20005
0.46
0.48
0.50
0.52
0.54
Feration
Earation
(b) Top: histograms of CDF values under each model's s implied noise distributions. Bottom: training loss curves evaluating each of MSE and DC loss on MSE- trained and DC-trained predictions.
(a) Top: predicted signals estimated with MSE and DC loss. Bottom: signals after convolution with the known operator H.
Figure D4: Results for ID deconvolution with known Gaussian blur with Oblur 0.02 and noise with noise = 0.05.
28

---

Published as a conference paper at ICLR 2026
Final CDF-value Histograms
Predicted signals (x [0.45, 0.55))
Empircal protabbllty denstty
MSE trained
True signal
Unitsmm set.
1.0
tisiitriined
DC-trained
Signal
Noisy absorvations
-0.5
-1.0
0.6
0.8
1.0
0.46
0.48
0.50
0.52
0.54
Trajectory
Trajectory
0.05
Convolved signals (x E [0.45, 0.55])
1.75
True signal MSE-trained ined DC-trained
0.04
3.50
1.25
0.5
MSS-trained DCtraines
3.00
Signal
0.0
Noisy abservations
0.50
0.25
-1.0
-1.5
0.00
5000 10000 15000 20000
0.00
5000
10000
15000 2000C
Barailnn
Beration
(b) Top: histograms of CDF values under each model's implied noise distributions. Bottom: training loss curves evaluating each of MSF and DC loss on MSE- trained and DC-trained predictions.
(a) Top: predicted signals estimated with MSF and DC loss. Bottom: signals after convolution with the known operator H.
Figure D5: Results for ID deconvolution with known Gaussian blur with Øbtur 0.02 and noise with noise 0.2.
29

---

Published as a conference paper at ICLR 2026
E DIP APPLICATION
E EXPERIMENTAL DESIGN
We followed the experimental design of Ulyanov et al. (2020) specify denoising task for this work. We used the same hourglass architecture with skip connections described in their paper, using their code made available at https:: / thub com/ Dmi tryülyanov/ deep- wmage www..aaaeeeeiiirraeee In all experiments, we used Adam (Kingma and Bal 2015) with learn- ing rate x 10 Experiments were conducted with 24GB Nvidia GeForce RTX 3090 GPU and run for 10,000 iterations; for the largest image size 512 x 512, running DIP-MSE and then DIP-DC took ~ 40 minutes (with ~ 4 experiments able to be run in parallel).
added independently pixel- and channel-wise to 3-channel RGB images with a fixed standard deviation a. Pixel intensities were then clipped [0,1], as in Ulyanov et al. (2020). The MSE and DC loss were then evaluated after flattening channel and spatial dimensions to a ID vector.
Images were first normalized to [O.1]. Then, Gaussian noise
The clipping function induces non-continuous noise distribution D; for each measurement (i.e. one channel of one pixel), with probability mass at O and 1. However, the CDF of clipped Gaussian such an approximation in experiments
distribution may still be well approximated, and in this section (details given in Appendix C.2).
We investigated varied noise levels from work input for experiments comparing DIP-MSE to DIP-DC; for repeatability, we ran experiments with 5 random seeds. Where error bars are shown in Figure 5d and these are with respect to 5 random initializations of random noise and network parameters. The results shown in the main body of the paper are random seed 0 and a
100 We fixed the random image and net-
75 for the image of F-16 aireraft.
255
The F-16 aircraft image is from standard datase intended for research purposes. The train, plane and castle images are from Martin et al.] (2001) (and our usage complies with the custom license, which states use is permitted "for non-commercial research and educational purposes"). The cat image is reproduced with permission from the photographer.
E.2 ASSESSING THE DISTRIBUTIONAL CONSISTENCY OF THE CLEAN IMAGE
We empirically validate the assumption that the true clean image should theoretically induce a uni- form histogram of CDF values (asymptotically). In Figure El we see that evaluating the DC loss on the clean image yields an approximately uniform distribution of CDF values (and correspondingly low DC loss value), while evaluating the DC loss on the noisy image yields histogram with a peak at 0.5 (and correspondingly high DC loss value of approximately In 4 see Equation 74).
E.3 EFFECT OF VARIED NOISE LEVEL
We varied a from a = 235
10
to a =
235
100
and performed DIP with each loss at each a.
Figure E2 shows the peak SSIM achieved by each method the F-16 image. Figure + shows the peak PSNR, the PSNR of the final image and the PSNR of the mean of the images at the t100th 9200 th iterations.
In Figures
B4
E6
and
E8
we give more detailed results (for seed 0) at different noise levels
a..
E.4 RESULTS WITH VARIED IMAGES
We additionally show some results with varied a for different images, to validate that our findings generalize beyond the F-16 aircraft image used in the main text.
Figure
F9 shows the output of DIP-DC and DIP-MSE on selection of natural images at varied a.
These results are explored in more depth in Figures EI0 EI and BI3 showing loss curves that correspond with what was observed in Section 5.1
https://sipi usc. wttp:/uceee..aaaaaaaeeaaaaaaaaee
wwpppppppllooooo ume-mi sc& m 11
30

---

Published as a conference paper at ICLR 2026
CDF histogram of clean image (DC value: 0.0040 0.0022)
CDF histogram of noisy image
(DC value: 1.3855 0.0027)
1.0
100
Empirical donsity
Uniform[(0.1]) PDF
80
Probabiiity donsity
60
Probability
40
0.2
20
Empirical density
Unifarm(jo,11 PDF
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
values)
Figure E1: Histograms of CDF values obtained during the evaluation of the DC loss on the true image (left) and the noisy image (right) (with ø = 253 standard deviation over 100 evaluations of the (stochastic) loss different instantiations of noise.
10
Uncertainty values are given as 1.96x the
Peak SSIM Noise Level
0.90
DIP-MSE
H
DIP-DC
Peak SSIM *
0.85
0.80
0.75
0.70
20
60
100
Noise level (Standard deviation a noise)
Figure E2: Peak SSIM metric values for DIP-MSE and DIP-DC. Uncertainty is shown with respect to initializations of random noise and network parameters.
31

---

Published as a conference paper at ICLR 2026
PSNR vs Noise Level
(for various choices of output image)
End averaged Images: DIP-DC
End PSNR averaged images: DIP-MSE
End PSNR: DIP-DC
End PSNR: DIP-MSE
Peak PSNR: DIP-DC
Peak PSNR: DIP-MSE
30
PSNR (dB) t
25
20
15
20
40
60
80
100
Noise level (Standard deviation a of noise)
Figure E3: PSNR metric values of variant outputs for DIP-MSE and DIP-DC. Specifically, the peak PSNR achieved, the PSNR of the image at the 1h000 iteration and the PSNR of the mean of the images at the th00th 9200th 1h000 iterations shown. Uncertainty is with respect to initializations of random noise and network parameters.
32

---

Published as a conference paper at ICLR 2026
True Image
OPP MISE
Nolsy Image
(a) Images produced by DIP-MSE and DIP-DC as function of iteration number.
DIPAKSE
DIP-DC
pelver 3o
aehver 2013
(b) Histograms associated with images produced DIP-MSE and DIP-DC function of iteration number.
DIP-MSE DIP-DC
DIP MSE
0.010
2.5
0.008
MSE
2.0
0.006
1.5
0.004
1.0
2000
4000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) MSE loss as a function of iteration number.
(d) DC loss as a function of itcration number.
0.90
32
0.85
30
0.80
28
5SIM
26
0.75
24
0.70
DIP-MSE
DIP.DC
2000
4000
6000
8000
10000
2000
4000
8000
10000
Iterations
Iterations
(c) PSNR as a function of iteration number.
(f) SSIM as a function of iteration number.
Figure E4: Results for DIP-MSE and DIP-DC
255
33

---

Published as a conference paper at ICLR 2026
True Image
OPP MISE
Nolsy Image
(a) Images produced by DIP-MSE and DIP-DC as a function of iteration number.
pelver 2010 md
(b) Histograms associated with images produced DIP-MSE and DIP-DC function of iteration number.
DIP-MSE DIP-DC
DIP MSE
0.6
0.016
0.5
0.014
DC loss
MSE
0.012
0.3
0.010
0.2
0.008
0.006
2000
4000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) MSE loss as a function of iteration number.
(d) DC loss as a function of itcration number.
30
0.80
28
0.75
PSNR
26
SSIM
0.65
0.60
DIP-MSE
0.55
DIP-MSE
0.50
DIP.DC
20
2000
4000
6000
8000
10000
2000
4000
8000
10000
Iterations
Iterations
(c) PSNR as a function of iteration number.
(f) SSIM as a function of iteration number.
Figure E5: Results for DIP-MSE and DIP-DC
255
34

---

Published as a conference paper at ICLR 2026
True Image
te ablenss
100 Iterations
1000 Itrations
S00G Reradons
10000 Rerations
Noiey Image
teration
160 teeations
1000 Iteratione
5C05 Berations
10002 herations
DP
(a) Images produced by DIP-MSE and DIP-DC function of iteration number.
i0e3
18:00 bwattonti
DIP-MSE
wirte Irrnect
1063 beateet
1931d seateb!
DIP-DC
24
34
14
(b) Histograms associated with images produced by DIP-MSE and DCP-DC as a function of iteration number.
0.0425
DIP-MSE DIP-DC
DIP-DC
0.20
0.0350
DC loss
0.15
0.0325
0.10
0.0275
0.0250
0.05
0.0225
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) MSE loss as function of iteration number.
(d) DC loss function of iteration number.
26
0.7
PSNR
24
SSIM
0.6
0.5
22
0.4
20
DIP-MSE
DIP.MSE
DIP-DC
DIP-DC
0.3
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(e) PSNR function of iteration number.
( SSIM as function of iteration number.
Figure E6: Results for DIP-MSE and DIP-DC at
50
255
35

---

Published as a conference paper at ICLR 2026
True Image
te atlensn
100 Iterations
1000 Itrations
S00G Reradons
10000 Rerations
Noley Image
teration
160 teeations
1000 Iterations
5C05 Berations
10002 herations
DP
(a) Images produced by DIP-MSE and DIP-DC function of iteration number.
Siie karsdaelir
20003 bandeo
DIP DIP-MSE
11
1003
DIP-DC
50
24
(b) Histograms associated with images produced by DIP-MSE and DIP-DC DC as a function of iteration number.
DIP-MSE DIP-DC
0.30
DIP-MSE
0.065
DIP-DC
0.25
0.060
0.20
0.055
DC loss
MSE
0.050
0.15
0.045
0.10
0.040
0.05
0.035
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) MSE loss as function of iteration number.
(d) DC loss function of iteration number.
24
0.6
PSNR
SSIM
0.5
DIP-DC
20
0.4
18
DIP-MSE
DIP-DC
16
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(e) PSNR as function of iteration number.
(f SSIM as function of iteration number.
755 Figure E7: Results for DIP-MSE and DIP-DC at
255
36

---

Published as a conference paper at ICLR 2026
True Image
te ablenss
100 Iterations
1000 Iteations
5000 ReraSons
10000 Rerations
Noiey Image
teration
160 teeations
1000 lteratione
5000 Berations
10002 herations
DP
(a) Images produced by DIP-MSE and DIP-DC function of iteration number.
bessieii
1800 bwasheeli
barttarti
Bendy
DIP-MSE
100 dnvileed
DUP- DC
344
64
(b) Histograms associated with images produced by DIP-MSE and DCP-DC as a function of iteration number.
0.30
0.09
DIP-DC
0.08
DC loss
0.20
MSE
0.15
0.06
0.10
0.05
0.05
DIP-DC
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) MSE loss function of iteration number.
(d)
DC loss function of iteration number.
24
0.7
22
0.6
0.5
20
SSIM
0.4
DIP-DC
18
16
DIP-MSE
14
DIP-DC
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(e) PSNR function of iteration number.
(f SSIM as function of iteration number.
Figure E8: Results for DIP-MSE and DIP-DC at
100
37

---

Published as a conference paper at ICLR 2026
True Image
Noisy Image
DIP-MSE
DIP-DC
(a) Cat image;
255
True Image
Noisy Image
DIP-MSE
DIP-DC
(b) Horse image;
True Image
Noisy Image
DIP-MSE
DIP-DC
(c) Train image;
25
253
True Image
Noisy Image
DIP-MSE
DIP-DC
(d) Castle image; 256
25
Figure E9: Further example images denoised with DIP-DC and DIP-MSE. Images shown are after 10,000 iterations of DIP.
38

---

Published as a conference paper at ICLR 2026
Thue Image
terations
140 trerations
1000 Iterations
5203 Beralions
10002 heralions
Noisy Image
terabirn
terations
1000 Itrationg
5002 Roration
10005 Beradion
DIP DPOC
(a) Images produced by DIP-MSE and DIP-DC as a function of itcration number.
123 Sorbe30
5008 Berbobbi
20009 lendee3)
Sixe kerddieto
tundle
S0me P1
2122
(b) Histograms associated with images produced by DIP-MSE and DIP-DC as a function of iteration number.
0.070
DIP-MSE
0.30
0.065
0.060
DC loss
0.20
0.055
0.15
0.050
0.10
0.045
DIP-MSE DIP-DC
0.05
0.040
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
MSE loss function of iteration number.
(d) DC loss function of iteration number.
0.55
DIP-MSE
22
DIP-DC
0.50
21
0.45
20
SSIM
0.40
0.35
18
0.30
17
0.25
DIP.MSE
0.20
DIP-DC
0.15
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
iterations
PSNR function of iteration number.
(f) SSTM function of iteration number.
Figure E10: Results for DIP-MSE and DIP-DC on cat image.
39

---

Published as a conference paper at ICLR 2026
True Image
te abom
100 tleratoos
1000 Haraaions
5000 Beratoos
10005 Kemalion ons
Noisy Image
to atlintnn
140 tterations
1000 Ite Iteratiind
5005 Reradions
10000 herations
(a) Images produced by DIP-MSE and DIP-DC function of iteration number.
500
Wet on SMB
Hu Sice bersdieln
Ide bendeo
LG03 herato
Ja082 berdeo
M4
24
60
(h) Histograms associated with images produced by DIP-MSE and DIP-DC as a function of iteration number.
0.050
DIP-MSE DIP-DC
DIP-DC
0.045
0.040
loss
MSE
0.035
0.030
0.025
0.1
0.020
0.015
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) MSE loss as function of iteration number.
(d) DC loss function of iteration number.
20.0
19.5
0.5
19.0
0.4
PSNR
S5IM
28.5
18.0
0.3
17.5
DIP-MSE
DIP-MSE
17.0
DIP-DC
2000
4000
600D
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(e) PSNR as function iteration number.
(f) SSIM as function iteration number.
Figure E11: Results for DIP-MSE and DIP-DC on horse image.
40

---

Published as a conference paper at ICLR 2026
Image
1000 Haralione
5000 Beraloona
DIP
Nolsy Image
terations
tterttonns
1000 Iteraioon
5000 Reradions
10000 herations
DC
(a) Images produced by DIP-MSE and DIP-DC function of iteration number.
1003
5663 tentonee
(h) Histograms associated with images produced by DIP-MSE and DIP-DC function of iteration number.
DIP-MSE DIP-DC
0.0225
DIP-MSE DIP-DC
0.0200
0.0175
DC loss
MSE
0.0125
0.4
0.0100
0.2
0.0075
0.0050
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) MSE loss as a function of iteration number.
(d) DC loss as a function of iteration number.
0.80
24
0.75
2.
SSIM
0.70
22
0.60
DIP-MSE DIP-DC
0.55
DIP-MSE
19
DIP-DIC
6000 2000
2000
6000
8000
10000
Iterations
Iterations
(e) PSNR as a function of iteration number.
(f) SSIM as function of iteration number.
Figure E12: Results for DIP-MSE and DIP-DC on train image.
41

---

Published as a conference paper at ICLR 2026
True Image
Noisy Image
to aborn
100 Iterations
1050 Iterationn
S0OG Rorations
10005 Feradions
DIPCC
(a) Images produced by DIP-MSE and DIP-DC as a function of iteration number.
L003 seaticn
seratooeo
10300 baratard)
condo
1003
19:90 aeatee)))
24 34
M4
38
04
(b) Histograms associated with images produced by DIP-MSE and DIP-DC as a function of itcration number.
0.7
DIP-MSE DIN-DC
DIPMSSE
0.6
0.016
0.5
0.014
0.4
0.012
0.3
6.010
0.2
0.006
0.1
0.004
2000
4000
6000
BOG0
10000
2000
4000
5000
8000
20000
Iterations
Iterations
(c) MSE loss as function of iteration number.
(d) DC loss function of iteration number.
0.00
0.75
26
25
0.70
24
0.65
0.60
0.55
21-
DINMMSE
DIP-MEE
20
DIP-DC
0.50
2000
4000
6000
5000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(c) PSNR as function of iteration number.
(f) SSIM as function of iteration number.
Figure B13: Results for DIP-MSE and DIP-DC castle image.
42

---

Published as a conference paper at ICLR 2026
E.5 EFFECT OF MISSPECIFIED NOISE
We considered the behavior of DC loss when the assumed noise model is systematically violated. For all experiments, DC loss assumed the same noise model of a clipped Gaussian with assumed 255 matching the setting used in the main paper.
To probe robustness, we generated data from three alternative families of noise distributions, each controlled by single scalar parameter that smoothly modulates the degree of mismatch. In every case, the clean image and optimization settings were kept fixed.
Outliers We first investigated the effect of sparse, heavy-tailed corruptions. For each pixel, with probability p. we inflated the true noise standard deviation to dre assumedd drawing the mea- surement from a heavier-tailed distribution than the one assumed by the loss.
We varied from 0% to 25%. FigureEE14shn E14ssows that the performance of DC loss degraded gracefully over this range, with similar degradation observed relative to MSE loss.
probaollity outllers
DIP
MSE
25
DIP DC
PSUR (08)
24
23
22
20
25
Outlier probability (% of pixets)
(a) Max PSNR (with respect iteration) func- tion p. the probability that pixel's measurement drawn from an outlier heavy-tailed distribution.
DIP Denoising with 25% outliers
Ground Truth
Noisy Image
DIP-MSE
DIP-DC
(b) Images corresponding p 25% on the above chart.
Figure E14: Results for outlier ablation study, investigating the effect of mispecifying the noise distribution for pixels.
Correlated noise We next examined noise with spatial correlation structure. Here, we drew an i..d. Gaussian field e (0, o2 asuume)) of the same shape as the image, and then spatially smoothed it using an isotropic Gaussian kernel. Specifically, letting 9- denote Gaussian blur with standard deviation 5 pixels, the true noise was generated
Thrme = gr * E.
The parameter f 2 0 controls the strength of correlation: J = recovers the matched i..d. model, while increasing broadens the correlation structure, thereby violating the independence assumption
43

---

Published as a conference paper at ICLR 2026
underlying the MSE and DC losses. After smoothing, Pase rescaled to have standard deviation assumedd ensuring that only the correlation structure was altered, not the noise amplitude.
We varied from 0 to 2.5. Figure E15 shows that the benefit of DC loss relative MSE loss de- grades severely when correlation is introduced. However, the performance of our method nonethe- less degrades similar rate MSE loss in this setting.
Max PSNR level noise correlation
26
DIP-MSE
DI-DD
24
22
Max
Concelation scelt [pixeb))
(a) Max PSNR (with respect to iteration) function of 7, the correlation scale.
DIP Denoising with correlated noise (T 2.5)
Ground Truth
Noisy Image
DIP-MSE
DIP-DC
(b) Images corresponding to r 2.5 on the above chart.
Figure E15: Results for correlated noise ablation study, investigating the effect of mispecifying the noise distribution by introducing correlated noise.
Heteroscedastic noise Thirdly, we tested form of noise whose variance depends on the underly- ing clean intensity.
For a pixel with clean value x E [0, 1]. we defined the true noise level as
mre (x) Osssumed (1 (2x--)))) 1)),
where [-1, 1] determines the severity of heteroscedasticity. Thus = 0 recovers the matched homoscedastic model, while increasing |a| creates regions of inflated or reduced variance depending on image brightness. For each pixel we then sampled
Thme (I) NO, oiee (z:)²,
and constructed the observed image y + Rome (x: with clipping to [0, 1]. The parameter therefore provides a single scalar that smoothly controls the degree of mismatch, analogous to outlier rate p and correlation scale 7 above.
Our ablation test again demonstrated graceful degradation in the performance both DC loss and MSE loss with respect to the noise modelling mismatch.
Conclusion Taken together, these three experiments explore mismatches in tail behavior, spatial correlation, and variance structure. Fach setting introduces controlled departure from the assumed
44

---

Published as a conference paper at ICLR 2026
Max PSNR vs level of heteroskedasticity
DIP.MSE
25.5
DIP-DC
Max PSWR (ci)
25.0
24.5
23.0
0.0C
0.05
0.31
0.25
0.20
0.25
Heleroakedesstccly putuimmter
(a) Max PSNR (with respect iteration) as function of a, the level of heteroscedasticity.
DIP Denoising with heteroscedasticity (a 0.25)
Ground Truth
Noisy Image
DIP-MSE
DIP-DC
(b) Images corresponding to 0.25 on the above chart.
Figure E16: Results for heteroscedastic noise ablation study, investigating the effect mispecifying the noise distribution by introducing heteroscedastie noise.
45

---

Published as a conference paper at ICLR 2026
noise model, allowing to assess how reconstruction quality evolves the data deviate from the idealised conditions encoded in the loss.
Under the noise model mismatch settings studied, DC loss's performance degraded gracefully and comparatively to MSE loss.
46

---

Published as a conference paper at ICLR 2026
F PET RECONSTRUCTION APPLICATION
F.1 EXPERIMENTAL DESIGN
A ground truth BrainWeb phantom was generated (Cocosco al. 1997 with width and height of N₂ = 256 voxels, and multiplied by a scale factor of 0.1 (resulting in mean voxel intensity of 2.42). forward model was specified using ParallelProj (Schramm and Thielemans 2023), approximating the dimensions of single ring of a Siemens Biograph mMR PET scanner. Lines of response were over-sampled by a factor of 4, and pairs of lines were then added together, so as to simulate thicker lines of response, ensuring the projected rays intersected all voxels in the image space (which avoided a null space in the projector for smaller voxel sizes).
This setup produced sinograms with 252 projection angles and 688 radial bins (for total of 173376 lines of response, of which 54773 intersected non-zero voxels in the N₂ = 256 ground truth phan- tom).
For the case of regularized PET image reconstruction (corresponding to Section 5.2.1 the intensity of the phantom was 25% of the unregularized casc, to simulate a lower-dose acquisition. The edge- preserving TV penalty used is defined in subsection F.4 and Na- = 256 was used only.
For each of the algorithms NLL-Adam, DC-Adam and MLEM, starting image iteration was an image of uniform intensity equal to the mean intensity of non zero voxels in the phantom.
For DC-Adam and NLL-Adam only, at each iteration, multiplied the image estimate by mask representing the convex hull of the non-zero voxels in the phantom (information that can be derived physically in real PET scan from the attenuation map). Final images were passed through a ReLU function to remove negative values (which are not physically realistic for a PET image). We pre- conditioned each gradient update by dividing by the sensitivity image (i.e. the backprojected image obtained by backprojecting a sinogram of ones).
For the experiments in subsection F33 the total sum of counts in the ground truth phantom held constant as the width of the phantom was varied from Na = 64 to Na = 512. In this appendix, learning rate 0.01 was used for N₂ = 512, 0.005 for N₂ 256 and 0.0025 for Na S 128.
Experiments were conducted with 24GB Nvidia GeForce RTX 3090 GPU and run for 10,000 iterations; for the largest image size 512 512, running DC-Adam, NLL-Adam and MLEM con- secutively took ~ 30 minutes.
F.2 ASSESSING THE DISTRIBUTIONAL CONSISTENCY OF THE CLEAN PHANTOM
In contrast to the Gaussian setting, with a Poisson noise model we have discrete CDF, As result, the theoretical guarantee of the probability integral transform does not strictly hold. However, the true image estimate should still induce approximately uniform histogram of CDF values, and investigate this assumption in this subsection.
Figure F1 explores how changing the simulated radioactivity level (often referred as the dose, affecting the total number of measured counts) influences the histogram of CDF values for both the ground-truth sinogram and the corresponding noisy sinogram. We also include an alternative reference distribution: the CDF values of noisy measurement as the Poisson mean. All count levels this figure are expressed relative the baseline count level used for the PET results in the main paper.
samples generated by treating each
These results demonstrate that targeting a uniform histogram of CDFs an appropriate approxima- tion for higher doses (analogously to how the Normal distribution becomes good approximation for the Poisson distribution for higher means). Further work is required to determine how to best ad- just for the discrete nature of the Poisson distribution at lower doses, although the alignment shown between the alternative target (blue line) in Figure FT and the true clean histogram is promising, as is the option of using the randomized PIT. (We hypothesize that the peaks at 0.5 and 1.0 are primarily due to very low measured values of 0 and 1, where the Poisson distribution is not well approximated by any continuous distribution.)
47

---

Published as a conference paper at ICLR 2026
CDF histogram of clean sinogram (DC loss value: 0.3178 0.0211)
CDF histogram noisy sinogram
(DC loss value: 1.2956 0.0095)
40
Empirical density
Uniform:((0.1) PDF
35
Probability density
25
20
0.75
15
0.50
Emairical density
0.25
Uniform((0.11) PDF
Discrete CDF alternative target
0.00
0.0
0.2
1.0
0.0
0.2
CDF values (5, values)
CDF values values)
(a) Count level 0.2.
CDF histogram of clean sinogram loss
CDF histogram of naisy sinogram
0.0205)
loss value: 1.3386 0.0095)
1.4
Empirical density
40
Uniform[[o.1]) PDF
1.2
1.0
30
0.8
Probability
Probability
0.6
0.4
Empirical density
0.2
Uniform([0.1) 1]) PDF
alternative target
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
CDF
values (s) values)
(b) Count level 1.0.
CDF histogram of clean sinogram (DC loss value: 0.1346 = 0.0204)
CDF histogram noisy sinogram
(DC loss value: 1.3596 + 0.0095)
50-
Empirical density
Uniform/[0.11]
0.8
30-
Probabiiiy
Probaability
20
Empirical density
0.2
Unifarm([0,1] PDF
Discrete CDF alternative target
0.2
0.4
0.6
0.8
0.2
CDF va lues (s) values)
CDF values (s) values)
(c) Count level 4.0
Figure Fl: Investigating the DC loss values obtained and the uniformity of CDF histograms in the ideal and worst cases of evaluating the DC loss against the true measured data or Poisson-noisy measured data respectively. Subplots show the effect of increasing the number of measured counts (and thereby reducing the impact of noise introduced by the Poisson noise model). Uncertainty values are given 1.96x the standard deviation over 100 evaluations of the (stochastic) loss on different instantiations noise.
48

---

Published as a conference paper at ICLR 2026
F.3 VARYING VOXEL SIZE AND THE NUMBER PARAMETERS TO ESTIMATE
We investigated the effect of over-parameterization on the PET reconstruction process with the DC loss, by varying the width of the phantom from Na- 64 to Na- 512. Figures F2
F3
R4
and FS
show the results of these experiments for Na- = 64, 128, 256 and 512 respectively.
It is clear that the number of voxels increases (i.e. we move into an over-paramettrized regime), the ability of traditional algorithms to overfit the noisy data increases, thereby leading to more noisy reconstructions without early stopping. This pattern is particularly obvious in the histograms for MLEM and NLL-Adam in Figures F2 which become less uniform as the voxel size decreases.
NLL-Adam and DC-Adam do not build in non-negativity as a prior, whereas MLEM does, which may explain its better peak metric values for NRMSE and SSIM.
49

---

Published as a conference paper at ICLR 2026
True Image
100 Harations
1010 Eandioons
2000 Bardlions
5000 Farilions
1000G Beralars
0.6
NLLL Atcim
0.
DC-40mm
MLEM
(a) Images produced by PET reconstruction algorithms function of iteration number.
inn)s)
1000 mnjs)
2000 Berahions)
Berationts)
10000 Iterationss
Unfommll..110o
0.00
0.75
L.00
6.58
3.75
1.00
0.25
6.50
0.75
0.25
6.50
6.75
1.00
0.50
0.25
6.75
1.00
3.0
DC-adam
Uneomm
035
6.50
0.5
L:0
80
0.00
0.25
6.59
3.75
1.00
5-
0.00
0.25
6:50
0.75
1.00
5.0
0.00
0.25
6.50
6.75
1.00
5-
0.00
0.25
6.55
0.75
1.00
Drpercal serote
MLEM
Undemi19.11000
0.55
375
1.0
0:00
3.79
0.55
0.53
0.55
0.00
0.35
0.75
1.00
C36 vilue
065)
CSP 03Me
C1P 43/0
(b) Histograms associated with PET reconstruction outputs as function iteration number.
DC iterations
NLL over iterations
1.5
NLL-Adam
3.98
DC.Adam
1.0
1.70
NLL.Adam DC-Adam
1.36
0.5
1.72
0.0
2003
4000
6200
2000
10000
2000
4000
6000
8000
10000
Iterations
(c) NLL. loss as function of iteration number.
(d) DC loss as a function of iteration number.
Nean Square (NRMSF)
Structural Simiarity Index Measure (55IM)
1.0
NLL Adam
0.8
DC-Adam
NRMSE
0.6
0.4
0.4
NLL-Adam DC-Adam
0.0
MLEM
2000
4000
6000
8000
10000
Iterations
2000
4000
6000
8000
10000
Itarations
(e) NRMSE function of iteration number.
(f) SSIM function of iteration number.
Figure F2: Results for PET reconstruction with image size N₂ = 64.
50

---

Published as a conference paper at ICLR 2026
True Image
100 Haralions
1010 Randiions
2000 Bardloons
3000 Faralions
1000 Beralars
NLLL Aazam
DC-40am
MLEM
(a) Images produced by PET reconstruction algorithms function of iteration number.
heral inn)s)
1000 mljs)
2000 Berabions)
Berationts)
10000 Iterationss
Unifermt PD?
3.5
2.5
8.5
0.00
0.22
0.50
3775
1.00
00
0.25
0.53
3.75
1.00
0.00
0.25
2:53
5.75
1.00
0.00
0.25
0.52
5.75
1.00
0.00
0.25
0.51
0.75
1.00
DC-Adam
6a
Emppicca denuity
3.0
15-
8a
0.00
0.2
3.5
Lto
9:00
2.0
0.25
0:30
6.0
0.00
0.25
1.00
6-
0.00
0.25
9.5
1.0
10
Iiproc scesty
55
3.5
L10
0:00
9.75
0.55
0.53
0.25
0.00
0.25
0.75
1.00
C aou0
C36 vilues 065)
CSP V3Me 5n Afrcc)
CTP 4210 do
(b) Histograms associated with PET reconstruction outputs as function iteration number.
DC iterations
NLL over iterations
1.5
NLL-Adam
DC.Adam
2.30
1.0
NLL.Adam DC-Adam
0.5
2.00
0.0
2002
4000
6000
2000
10000
2000
4000
6000
8000
10000
Iterations
(c) NLL loss as function of iteration number.
(d) DC loss as a function of iteration number.
Nean Square (NRMSF)
Structural Simiarity Index Measure (S5IM)
1.0
NLL. Adam
0.8
DC-Adam
NRMSE
0.6
0.4
NLL-Adam DC-Adam MLEM
0.4
0.0
2000
4000
6000
8000
10000
Iterations
2000
4000
6000
8000
10000
Itarations
(e) NRMSE function of iteration number.
(f) SSIM function of iteration number.
Figure F3: Results for PET reconstruction with image size N₂ = -128.
51

---

Published as a conference paper at ICLR 2026
True Image
100 Hanalions
1010 Eanalions
2000 Baralions
3000 Faralions
1000G Beratars
10.0
7.5
ALL Aacam
0.0
DC-40mm
MLEM
(a) Images produced by PET reconstruction algorithms function of iteration number.
mosd to to
heral inn)s)
1000 mnjs)
2000 eeabhonss
Berationts)
10000 Iberationsss
Unfernto.11 Por
14
1.5
0.00
0.75
1.0
0:00
5.59
9.75
1.00
100
0.25
6:30
0.75
0.25
6.32
6.75
0.5
0.25
6:50
6.75
1W
DC-Ndam
Emppecea denuity
0.00
025
6.5
0.5
L.0
0.00
0.25
0.59
5.75
1.00
5-0
0.25
6.50
0.79
1.00
1.0
0.00
0.25
6.50
0.75
1.00
5-
0.00
0.25
6.55
6.75
1.0
3.5
MLEE
1/
375
L10
0.25
0.55
6.52
0.25
0.00
0.55
0.75
1.0
OSP VAMKS
CDP VaMes So Aircc)
CTP walo wilest
(b) Histograms associated with PET reconstruction outputs as function iteration number.
DC iterations
NLL over iterations
1.5
322
NLL-Adam
NLL-Adam DC-Adam
DC.Adam
1.0
MLEM
2.6
0.5
0.0
2020
6022
6020
8002
10000
2000
4000
6000
8000
10000
Iterations
(c) NLL loss as function of iteration number.
(d) DC loss as a function of iteration number.
Nean Square (NRMSF)
Structural Simiarity Index Measure (55IM)
1.0
Adam
0.8
DC-Adam
NRMSE
0.6
0.4
0.6
NLL-Adam DC-Adam MLEM
0.0
2000
4000
6000
8000
10000
0.4
Iterations
2000
4000
6000
8000
10000
Itarations
(e) NRMSE function of iteration number.
(f) SSIM function of iteration number.
Figure F4: Results for PET reconstruction with image size N₂ = 256.
52

---

Published as a conference paper at ICLR 2026
True Image
100 Haralions
1010 Randiions
2000 Baraloon
500D Farilions
10000 Beratars
30-
'LLL Jazam
DC-40mm
MLEM
(a) Images produced by PET reconstruction algorithms function of iteration number.
25
180 (enettionss)
Berationss
Reretionts)
(eentioons)
10000 iterationssl
Unl PCF
10-
by
0.-
030
2.3
2.00
0.23
DC-Ndam
Emorical deratt
By
12
05-
5.00
050
035
1:00
6.50
0.25
2.51
375
1.00
0.00
6.55
0.50
6.75
3,00
0.0
5.3
325
0.50
0.55
1.50
0.0
0.00
0.35
6.54
0.75
1.00
Engrical denely
wwl.meel PCT
by
a5o
5.93
0.5 CDP viloos mass
3.25
0.55
1.50
0.50
1.00
CP woues 6 (ilcc)
CD9 volcsts values
CPP
(b) Histograms associated with PET reconstruction outputs as function iteration number.
DC iterations
NLL over iterations
1.5
NLL-Adam
4.0
NLL-Adam
DCIAdam
DC-Adam
1.0
MLEM
0.5
0.0
2002
4000
9200
2000
10000
2000
4000
6000
8000
10000
Iterations
(c) NLL loss as function of iteration number.
(d) DC loss as a function of iteration number.
Nean Square (NRMSF)
Structural Simiarity Index Measure (55IM)
1.0
Adam
0.8
DC-Adam
NRMSE
0.6
0.4
SSIM
NLL-Adam DC-Adam MLEM
0.0
2000
4000
6000
8000
10000
Iterations
2000
4000
6000
8000
10000
Itarations
(e) NRMSE function of iteration number.
(f) SSIM function of iteration number.
Figure F5: Results for PET reconstruction with image size N₂ = 512.
53

---

Published as a conference paper at ICLR 2026
F.4 FURTHER RESULTS WITH A REGULARIZED OBJECTIVE FUNCTION
We defined our edge-preserving TV prior as
(PTV
Σ W
wis (xig+1 - Xiyl + |i++I - Xisl),
(76)
N
with edge-aware weights defined as
way
(77)
V [x1 ((x.)+1-1,,)" xx,)) +x -x,,)
+
In this work, K = 0.1 and E = 1e- 8 were used. The width and height of each image was Na 256.
In Figure F6 show the pree-presr TV prior values obtained by image estimates found by DC+TV and NLL+TV against NLL, DC loss and error values (NRMSE). In line with what was expected, these charts exhibit different trends for DC+TV and NLL+TV.
Figure F6a shows that for high levels of regularization (i.e. large values of () both DC+TV and NI..+TV exhibit higher NRMSE (~ 40% or greater), as the strong regularization overpowers the data-fidelity term, seeking to drive the edge-preserving TV prior (penalty) value closer to zero. Interestingly, for DC+TV, we find many different image estimates (for range of different regular- ization strengths B) that give NRMSE close to DC+TV's minimum NRMSE, and only once an image estimate's value for adge-preserving TV (i.e. the prior or penalty value) dips too low does it come into conflict with the data- mata-fidelity constraint, resulting in worse NRMSE. In contrast, with NLL+TV, the NRMSE changes with greater sensitivity to the regularization strength 3, with only a very limited range of 3 values for which minimum NRMSE is achieved. It precisely this lack of conflict between the DC loss data-fidelity and the regularization prior which we theorize has led to the improved NRMSE values obtained by DC+TV Figures Sa and F6a
With lower regularization strengths, as expected, DC+TV retains a low value of the NRMSE while NLL+TV exhibits increasing NRMSE (as overfitting to noise occurs). Figure displass a similar chart, now showing DC loss against prior EPTV values for a range of image estimates obtained for various values. Both methods show a similar trend that seen Figure[F6a ( presumably because DC loss and NRMSE are correlated]"
For completeness, Figure F6c also shows similar trend with NLL in place of DC loss. It is note- worthy (and in line with concepts discussed previously) that the estimates found by DC+TV exhibit relatively constant NLL loss values as regularization strength increases, before leaving this "stable region" when balancing data-fidelity and regularization is no longer feasible at higher 3 values. This point occurs at very similar values in Figure F6C as in Figure F6b
The images shown with varying regularization strengths / in Figure F7 underline these trends. With low (or no) regularization strength, DC+TV finds a much better estimate than NLL+TV, and retains some improvement still at optimum regularization strengths (as in Figure 8). At high regularization strengths, both methods over-smooth and succumb to artifacts induced by the prior.
"The trend for NLL+TV surprisingly shows relatively low prior penalty for the regularization Considering images Figure F7 attribute this an imperfect prior not perfectly correlating with emror. The prior value for highly regularized images is still lower than for the unregularized case. Clearly, the prior is having the desired effect, as increasing it reduces overfitting and promotes smoothness (with edges respected). Further work could investigate this phenomenon with different regularization hyperparameters and penalty choices).
54

---

Published as a conference paper at ICLR 2026
MLL+TV
DC4TV
-.. Ground Truth TV valce
3.020
0.015
0.010
0.005
(a) Value of edge-preserving TV prior obtained against NRMSE value obtained by DC+TV estimates and NLL+TV estimates.
8-0
NLLETV DC Ground Iruth TV value
MUL TV
DC+TV
Ground Truth volue
0.000
8.020
0.015
0.015
0.010
0.005
0.005
MLL
(b) Value of edge-preserving TV prior obtained against (c) Value of edge-preserving TV prior obtained against
DC loss value obtained by DC+TV estimates and NLI. loss value obtained by DC+TV estimates and NLL+TV estimates.
NLL+TV estimates.
Figure F6: Edge-preserving TV prior values obtained by image estimates found by DC+TV and NLL+TV against NLL, DC loss and error values.
55

---

Published as a conference paper at ICLR 2026
Ground Truth
A D (no
1300
10000
2.0
1.5
p= (no TVI
==00 0=15
1.0
DEATV
0.5
0.0
Figure F7: Example images produced with DC+TV and NLL+TV for regularized PET image re- construction with an edge-preserving total variation penalty, increasing regularization strengths 3. First column: ground truth images. Second column: image results with no explicit regularization. Third column: image results with near-optimal choice of 3. Fourth column: indicative image results with 3 chosen to be "too large", resulting in regularization-induee artifacts for both methods.
56

---

Published as a conference paper at ICLR 2026
F.5 REAL PET DATA: EXPERIMENTAL SETUP
This section provides the full experimental details for the real-data reconstruction reported in Sec- tion 5.2.2 We reconstructed single 3D clinical PET brain scan acquired Siemens Biograph mMR system (non-time-of-fight), using the same Adam-based optimization framework as in the synthetic PET experiments but with an approximation of the full physical forward model used clin- ically.
Dataset and corrections. The dataset consisted of approximately 70 million lines of response, axially compressed into span-11 sinograms of dimension 252 x 344 x837. Vendor-providee software supplied the corrective factors required for forward modelling: attenuation correction L (from the MR-derived μ-map), detector normalization N, scatter estimation s, and randoms estimation
Forward model. We implemented the tomographic projection operator P using the open-source ParalleIProj library (Schramm and Thielemans, 2023), parameterised with reanner-specifte geome- try to match the real system. The complete forward model A combined the projection operator with all corrections:
Ax = NLC.,x + + s,
where C11 denotes the manufacturer's span-1 to span-11 axial compression matrix.
We restricted consideration of sinogram bins to those lines-of-response that intersect the μa-map, i.e. lines of response that actually intersect matter.
Reconstruction grid. All reconstructions were performed directly on the same voxel grid as the vendor reconstruction: 128 x 128 x 120 voxels with physical voxel dimensions (2.08626 mm) x (2.08626 mm) (2.03125 mm).
Optimization. The initial image was defined as uniform field on binary support mask derived from the j-map (after hole-fillin). Voxels outside the support were assigned a constant value equal to 1% of the mean activity inside the mask. Although not required for reconstruction, this warm start stabilised the earliest iterations and expedited prototyping.
We compared Poisson NLL and DC alternative data-fidelity terms, both optimised with Adam using learning rate of 10-4 and a positivity projection applied after each update. In line with standard practice for likelihood-basee PET reconstruction, we accelerated computation by splitting the sinogram into 21 ordered subsets and cycling through them during optimization. All methods were run for 10,000 iterations to expose the characteristic iate-iteration behavior of each loss.
For both NLL and DC, the gradient was preconditioned by dividing by stabilised sensitivity image A 10-3 matching the effective diagonal scaling used in the MI.EM algorithm.
NLL-Adam reconstruction took 0.037 sec/it compared to 0.057 sec/it for the DC-Adam reconstruc- tion.
Notes. All reconstructions operated the same PET grid, used identical physical corrections, and differed only in the choice data-fidelity term. The vendor image in Fig. 9 was the scanner's MLEM reconstruction, which may have included additional proprietary corrections.
57

---

Published as a conference paper at ICLR 2026
G PLUG-AND-PLAY WITH A LEARNED PRIOR
To demonstrate that DC loss applies cleanly to modern learned priors, we carried out an additional experiment on image deblurring/dnnoisng using a plug-and-play (PnP) framework with a pretrained denoiser prior. Concretely, we consider standard Gaussian deblurring model
y A Xime + 17.
7~ T~NN00,0"1)
(78)
where Xine a clean grayscale natural image, A is spatially invariant Gaussian blur kernel, and r is additive white Gaussian noise. Pixel values are scaled to [0. and clipped after noise is added, as in the DIP experiments.
PnP PnP-ADMM with DrCNN prior. We adopt plug-and-play ADMM scheme in which the prior
is represented by an off-the-shelf DnCNN denoiser (Zhang et al. [021), while the data term is either the usual quadratic loss or DC loss. Introducing auxiliary and dual variables v and u, we iterate
x11 = arg min { (Caua (A * xx + P ||x v² =
(79)
+1 = Dr(x 1 + u²), w = u' + +x
(80)
1+
(81)
where p > is the ADMM penalty parameter and D, is a DrCNN denoiser trained for a nominal Gaussian noise level The u-update in equation 77 is carried out by small fixed number of gradi- ent steps with Adam, while the v-update equation 80 is a pure denoising step (no backpropagation through D,).
Data terms: MSE vs. DC.
We compare two choices for the data-fidelity term:
CMSE(A + x)
A + x 1 y1B.
(82)
20
Cpc(A * x; LDC. * x, y;
(83)
where LDC-Gauss is the Glussed-Gaussian DC loss from the DIP experiments, applied the flattened residuals (A x y) with Gaussian noise model of variance o² and intensities clipped to [0, 1]. In the implementation, the DC loss is evaluated directly on the (B, C, H. W) tensors by flattening all spatial and channel dimensions.
We refer the two PnP variants as:
PnP-MSE: ADMM with quadratic data term LMSE.
. PnP-DC: ADMM with DC data term LDC-
Experimental setup. We use the same F16 image as in the DIP experiments, converted to single- channel grayscale. The forward operator A is depthwise Gaussian convolution with kernel size 15 x 15 and kernel standard deviation 1.6 pixels. We simulate clean data Yelean A * Xime and then add i.i.d. Gaussian noise 17- (000, o2 with o 25/ /55, clipping the result back to [0, 1].
The PnP-ADMM scheme is initialized with x" = o iterations, and use 5 inner gradient steps with learning rate 10-3 3 for the x-update. The denoiser D, is a standard 17-layer DnCNN trained for 25/ 25/255 and kept fixed during the experiment (Zhang al. 202T). Reconstruction quality is measured against X using PSNR and SSIM.
and u = O. We run 2500 ADMM
Results. We varied the hyperparameter p from 0.1 to 1000, carrying out PnP reconstruction for each MSE and DC loss. Figure F8 shows the PSNR achieved by each reconstruction against
PnP with DC loss achieves higher peak PSNR than with MSE (27.5 dB versus 26.9 dB; images shown in Figure(F9) and the DC curve is markedly flatter, indicating substantially greater robustness to the choice of p.
This behavior follows directly from the properties of DC loss. In PnP-MSE, higher p places more weight on the denoiser prior, while lower & places more weight the data fidelity. With MSE
58

---

Published as a conference paper at ICLR 2026
PnP: PSNR against regularization strength P
27.5
PnP-DC
-X-
22.5
PSNR (dB)
20.0
15.0
12.5
10-1
10
10
10
10a
0
Figure F8: PSNR values for PnP reconstruction with varied regularization strength for PnP-DC and PnP-MSE.
Noioy blurred Input
Prit MSE (best) p-56.2
InP-DC [besd]
Figure F9: Images obtained with maximum PSNR when reconstructing from blurred and noisy images, for PnP-MSE and PnP-DC.
data term, varying P changes this balance in way that often leads overfitting the specific noise realization in y (when the data term dominates) or oversmoothing (when the denoiser dominates). As result, PnP-MSE exhibits narrow PSNR peak.
In contrast, DC loss stops rewarding further reductions in the residual once its distribution matches the noise model. Consequently, the z-update no longer chases the measured noise, regardless of whether emphasises the data term or the prior. This makes the PnP-DC fixed point far less sensitive to p and yields both the higher optimal PSNR and the observed hyperparameter stability.
This reinforces the earlier observation that DC behaves like MSE when far from the solution but self-regularizes once noise consistency is reached, making naturally compatible with PnP meth- ods. Further work remains to explore these effects in wider range of modern learned denoising frameworks.
59

---

Published as a conference paper at ICLR 2026
H TIMING
H.1 EVALUATION SPEED OF DC LOSS
We benchmarked the computational cost of evaluating the DC loss against conventional pointwise objectives (NLL and MSE) for both Gaussian and Poisson noise models. Timings were obtained on single NVIDIA RTX 3090 GPU, averaged over 1000 runs. The same pairs of uniform random arrays (in the range [0,1]) were used to measure each pair of methods (DC) NIL or DC / MSE).
Results reported separately for forward and backward passes, and for representative dataset sizes, N 1,000 and N 1,000,000.
Smaller scale (N 1,000). At this moderate sample size, DC loss is measurably slower than NLL/MSE but still lightweight in absolute terms: forward passes are on the order of 7 x 10= versus 3 x 10-3a for NLL/MSE; backward passes show similar gap.
Table 1: Timing results (seconds) for N = 1,000 (mean = std).
Noise
Pass
Loss
Mcan
Std
Poisson
Forward
DC
0.000749
0.000476
NLL
0.000032
0.000176
Backward
DC
0.000237
0.000426
NLL
0.000042
0.000201
Gaussian
Forward
DC
0.000704
0.000478
MSE
0.000030
0.000171
Backward
DC
0.000266
0.000443
MSE
0.000026
0.000159
Larger scale (N 1,000,000). At one million samples, the gap widens: DC loss forward passes about half second. reflecting per-sample CDF/logit evaluations and sorting for the Wasserstein term, whereas NLL/MSE remain in the millisecond range. Backward passes for DC are faster than forwards but still notably slower than NLL/MSE.
Table 2: Timing results (seconds) for N 1,000,000 (mean + std).
Noise
Pass
Loss
Mean
Std
Poisson
Forward
DC
0.503176
0. 1.2922
NLL
0.002166
0.001515
Backward
DC
0.042866
0.022514
NLL
0.001780
0.001442
Gaussian
Forward
DC
0.500350
0.154613
MSE
0.000825
0.000608
Backward
DC
0.031831
0.017249
MSE
0.000965
0.000960
Context Solving inverse problems with modern unsupervised methods typically involves costly regularizers (e.g.neural network evaluations) and forward operator evaluations (c.g. tomographic projection). For context, the subsetted 3D tomography operator Section 5.2.2 required approxi- mately 0.012 per evaluation, DC loss the same problem required 0.017 per evaluation, and single rivfrse-diffusio step with the architecture and methodology of (Singh et al. 2024) required 1.05
5.22.2rrquirre
DC loss therefore introduced a noticeable runtime overhead relative to NLL/MSE, but its cost re- mained comparable to other common components in modern mrverse-problem pipelines.
60

---

Published as a conference paper at ICLR 2026
Summary. DC loss carries runtime overhead versus NLL/MSE. The overhead is small at N 1,000, but becomes more noticeable at = 1,000,000, driven chiefly by distribution evaluations and sorting in the Wasserstein step. It may be possible to reduce this overhead with approximation strategies and GPU-optimized kernels for DC loss at scale.
61

---

Published as a conference paper at ICLR 2026
1 LLM USAGE
Large-language models (LLMs) were used to sanity check ideas, polish writing and assist with converting code to algorithm pseudocode.
62


Reviews:
[{"review_id": "uo93hyDJrz", "reviewer": "Reviewer_W1WB", "summary": "This paper introduces a novel data-fidelity loss function, the Distributional Consistency (DC) loss, designed for inverse problems. The DC loss evaluates whether the entire set of observed measurements is statistically consistent with the noise distributions implied by the current signal estimate. The authors demonstrate that this method avoids overfitting, eliminates the need for early stopping in methods like Deep Image Prior (DIP) , and improves the efficacy of regularization in tasks like PET image reconstruction.", "questions": "Same as the weakness part.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper provides a comprehensive statistical analysis of the proposed DC loss.\n\n2. The idea of exploring distributional-level calibration as an alternative to data-fidelity loss is meaningful.", "weaknesses": "1. The method's core premise requires that the measurement noise distribution is known or can be well-estimated. The experiments are confined to purely synthetic data using clean Gaussian or Poisson noise models. This ideal scenario is rarely met in practice, where noise is often complex, correlated, or misspecified. The paper provides no experiments on real-world data, making it impossible to assess the method's robustness or practical generalization ability.\n\n2. The paper makes a very broad claim to be a \"performance-enhancing alternative... for inverse problems\", yet the experimental validation is narrow. It is limited to only two applications: image denoisin and PET reconstruction. Furthermore, the PET experiments are entirely simulated using a Brain Web phantom. This limited testing on \"toy problems\" and synthetic data is insufficient to support the paper's general claims.\n\n3. Follow my previous comments, another major limitation is the study's exclusive focus on unsupervised regularization methods like Deep Image Prior (DIP), a 2017 work. It does not demonstrate its effect in any state-of-the-art, end-to-end trained frameworks. Without even a proof-of-concept on generative methods or deep unrolling networks (which is way more common to use compared with DIP), the claim that this loss is a convincing alternative for modern inverse problems is unsupported and feels overclaimed.\n\n4. The practical usage of DC loss is questionable due to its computational overhead. The method requires computing a CDF for every measurement and, more importantly, sorting the entire set of $N$ transformed values at each iteration to compute the Wasserstein-1 distance. \n\n5. The paper's mathematical notation is inconsistent and confusing. For example: the unknown parameter to be estimated is introduced as $\\theta$ in the general formulation (Equation 1). In the PET application, this parameter is abruptly changed to $x$ (Equation 8) without any connection. Most critically, Equation 8 introduces a term $b$ (in $[Ax+b]$) which is never defined anywhere in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZtpyotiPKB", "reviewer": "Reviewer_KXqD", "summary": "The paper introduces a loss function for inverse problems, based on matching the distribution of residual errors with a known noise distribution.", "questions": "- How does the method perform when the noise model used does not match the data?\n\n- How does the method perform in combination with classical image reconstruction?\n\n- How does the proposed method compare with existing custom loss functions used in inverse problems?\n\n- How does the method compare with existing work based on using known noise models, for example as a stopping criterion?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Well written paper.\n\n- The overall idea of matching the noise distribution to derive a loss function is very interesting.\n\n- Results are given for several settings.", "weaknesses": "- The claim that the noise model is known or can be estimated in many practical settings seems overly optimistic. In practice, there can be many additional artifacts that might not be well captured by noise terms and/or are difficult to estimate/model. In the case of tomography for example, real-world data might include artifacts caused by not well calibrated or defective detector pixels (ring artifacts), isolated high-intensity values caused by stray X-rays, scatter components that are caused by non-linear interactions with the sample, parts of the data replaced by nonsensical values (caused for example by the X-ray beam going down during an experiment). It is unclear, but important, to assess how robust the proposed method is to such artifacts. In other words, it would be useful to know how well the method works when the noise model used does not match the data (in various degrees of severity).\n\n- The authors now use the proposed loss in combination with Deep Learning based methods. It would be interesting to see its performance for traditional image reconstruction (e.g. reconstructing the pixel values of an image directly), comparing with traditional reconstruction methods.\n\n- There is quite a bit of work in the inverse problem community about appropriate loss functions. Examples include the student's t loss [1] and the Huber loss [2]. Such works should be discussed in the introduction, and ideally compared with.\n\n- Similarly, there is quite a bit of knowledge about the effect of noise on the convergence of iterative reconstruction methods in inverse problems [3]. Based on this, several stopping criteria have been developed [4]. It would be good to discuss this prior work, and to compare directly with such stopping criteria (as you assume known noise distribution).\n\n[1] Kazantsev, D., Bleichrodt, F., van Leeuwen, T., Kaestner, A., Withers, P. J., Batenburg, K. J., & Lee, P. D. (2017). A novel tomographic reconstruction method based on the robust Student's t function for suppressing data outliers. IEEE Transactions on Computational Imaging, 3(4), 682-693.\n\n[2] Mohan, K. A., Venkatakrishnan, S. V., Gibbs, J. W., Gulsoy, E. B., Xiao, X., De Graef, M., ... & Bouman, C. A. (2015). TIMBIR: A method for time-space reconstruction from interlaced views. IEEE Transactions on Computational Imaging, 1(2), 96-111.\n\n[3] Elfving, T., Hansen, P. C., & Nikazad, T. (2014). Semi-convergence properties of Kaczmarz’s method. Inverse problems, 30(5), 055007.\n\n[4] Hansen, P. C., Jørgensen, J. S., & Rasmussen, P. W. (2021, September). Stopping rules for algebraic iterative reconstruction methods in computed tomography. In 2021 21st International Conference on Computational Science and Its Applications (ICCSA) (pp. 60-70). IEEE.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bkfKUDtFQ6", "reviewer": "Reviewer_AQeQ", "summary": "This paper proposes the Distributional Consistency (DC) Loss, a new data-fidelity objective for imaging inverse problems that goes beyond traditional pointwise losses such as MSE or negative log-likelihood. Instead of enforcing pixelwise accuracy, DC loss measures whether the residuals are statistically consistent with the assumed noise distribution. The method applies the Probability Integral Transform (PIT) to map residuals into a uniform space and computes the Wasserstein distance (after a logit transform) between the resulting distribution and a standard logistic reference. Experiments on Deep Image Prior (DIP) denoising and Poisson PET reconstruction show that DC loss effectively mitigates overfitting and improves robustness without relying on early stopping or manual regularization tuning.", "questions": "Please see the weakness section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The DC loss provides a principled alternative to traditional pixelwise data terms by enforcing consistency at the distributional level. The use of PIT and Wasserstein distance forms a coherent, probabilistically interpretable framework.\n- Experiments on DIP and PET reconstruction show that DC loss mitigates overfitting and achieves a better balance between noise suppression and structural preservation, illustrating strong potential for broader applications.", "weaknesses": "- The scope of evaluation is limited. The paper validates the proposed DC loss only on two tasks—DIP-based image denoising and Poisson PET reconstruction. While these are representative, it remains unclear whether the method generalizes to other inverse problems (e.g., deblurring, inpainting, or diffusion-based approaches) or to more complex, learned priors.\n\n- The core concept is not immediately intuitive. The key theoretical basis—the Probability Integral Transform (PIT)—is crucial for understanding why the CDF-based approach produces uniform residuals under a correct model. However, this important explanation is only presented in the supplementary material, leaving readers unfamiliar with the concept confused at first glance. As written, the method appears pointwise rather than distributional, and the main text would benefit from a concise, self-contained explanation or visual illustration of how PIT ensures distributional consistency.\n\n- The discussion of related work is insufficient. Those who are experts in imaging inverse problems might not necessarily experts in the design of data-fidelity terms. Thus, I find it surprising that the paper does not discuss any closely related works or precedents. This lack of context makes it difficult to judge whether the proposed formulation is genuinely new or an adaptation of existing statistical consistency ideas.\n\n- The approach has practical limitations:\n  - Computational cost is higher than standard MSE or NLL losses due to the sorting and Wasserstein distance computation required per iteration.\n  - The method assumes known and independent noise models for each measurement, which may not hold in real scenarios involving correlated, unknown, or spatially varying noise.\n  - The sensitivity to noise model mismatch is not studied, leaving uncertainty about the robustness of the approach in practical imaging systems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "91Pc1zQXwQ", "reviewer": "Reviewer_3iUP", "summary": "The paper introduces a new 'distributional consistency' data fidelity term that can be used for solving inverse problems. The loss cheks whether the prediction residuals follow the noise distribution by applying a variant of the probability integral transform (similar to the Von Mises statistic). Experiments on single image denoising using the deep image prior and on PET reconstruction with Poisson noise show that the proposed loss performs better than the standard negative log-likelihood setting.", "questions": "I would expect the loss to be highly sensitive to mispecified or approximate noise models - did you check the proposed method on real measurement data (not synthetically generated measurements)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- presents an interesting data fidelity term for inverse problems that exploits full knowledge of the noise distribution by checking that residuals are consistent at a distributional level.\n- the proposed loss avoids over-fitting on the deep image prior and improves performance on PET (regularized) reconstruction.\n- presents an analysis that motivates links with the standard negative log-likelihood loss", "weaknesses": "- the paper lacks a discussion of existing literature on the distributional loss, and other existing alternatives to negative log-likelihood. For example, the loss seems quite similar to the Von Mises statistic, which is well-known in statistics. It would be also good to include discussions on other related work, like robust estimators (eg Huber losses, etc).\n\n- There is no guarantee that the proposed loss should avoid overfitting in general problems, except for the empirical evidence on the two presented 'toy' examples, which involde a single image at a time. \n   - I would expect at least one experiment on learning on a dataset or its usage with modern solvers such as plug-and-play, diffusion methods, etc.\n   - I find sentences like \"it avoids overfitting to measurement noise even without the use of priors\" problematic, since there is no theoretical guarantee that this is the case - it is easy to construct counter-examples with low distributional loss and high error (as already done in the paper in 3.2) - and the results rely on 'implicit regularizations' such as the deep image prior autoencoder architecture or the reduced number of voxels in PET.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)