Score-Based Denoising Diffusion Models for Photon-Starved Image Restoration Problems
Authors: Savvas Melidonis
Yiming Xi
Konstantinos C. Zygalakis
Yoann Altmann
Marcelo Pereyra
Venue: Accepted by TMLR ()

Abstract: Score-based denoising diffusion models have recently emerged as a powerful strategy to solve image restoration problems. Early diffusion models required problem-specific training. However, modern approaches can combine a likelihood function that is specified during test-time with a foundational pretrained diffusion model, which is used as an implicit prior in a Plug-and-Play (PnP) manner. This approach has been shown to deliver state-of-the-art performance in a wide range of image restoration problems involving Gaussian and mild Poisson noise. With extreme computer vision applications in mind, this paper presents the first PnP denoising diffusion method for photon-starved imaging problems. These problems arise in new quantum-enhanced imaging systems that exploit the particle nature of light to exceed the limitations of classical imaging. The problems involve highly challenging noise statistics, such as binomial, geometric, and low-intensity Poisson noise, which are difficult because of high uncertainty about the solution and because the models exhibit poor regularity properties (e.g., exploding scores, constraints). The proposed method is demonstrated on a series of challenging photon-starved imaging experiments with as little as 1 photon per pixel, where it delivers remarkably accurate solutions and outperforms alternative strategies from the state-of-the-art.

Published in Transactions on Machine Learning Research (01/2025)
Score-Based Denoising Diffusion Models for Photon-Starved Image Restoration Problems
Savvas Melidonis
s.mekidonisdiff-julliccc
Juelich Supereomputing Centre Forschungszentrum Juelich
Yiming Xi
y.zi- Tasms.. ed.ac. uk
University of Edinburyh Heriot- Wott University
Marwell Institute for Mathematical Sciences
Konstantinos C. Zygalakis
k: aygyalaaks aet.gallaisseda....m uk
University of Edinburgh School of Mathematics
Marwell Institute for Mathematical Sciences
Yoann Altmann
y.allmann @hur. ac. uk
Heriot- Wolt Universily
School of Engineering and Physical Sciences Institute of Signals, Sensors ond Systems
Marcelo Pereyra
wwwpeeyrr@haa.oo uk
Heriot- Wall University
School of Mathematical and Compuler Sciences
Marwell Institute for Mathematical Sciences
Reviewed OpenReview: https: //
/ttps://ppenreviel/ openreview. net/ forum? id= UYXPU 7HU41
Abstract
Score-based denoising diffusion models have recently emerged powerful strategy to solve image restoration problems. Farly diffusion models required problem-specific training. However, modern approaches can combine a likelihood function that is specified during test- time with a foundational pretrained diffusion model, which is used as an implicit prior in a Elug-and-Plas (PnP) manner. This approach has been shown to deliver state-of-tlae thate-of-thh-rtt performance a wide range of image restoration problems involving Gaussian and mild Poisson noise. With extreme computer vision applications in mind, this paper presents the first PnP denoising diffusion method for photon-starved imaging problems. These problems arise in new quantum-enhanced imaging systems that exploit the particle nature of light to exceed the limitations of classical imaging. The problems involve highly challenging noise statistics, such as binomial, geometric, and low-intensity Poisson noise, which are difficult because of high uncertainty about the solution and because the models exhibit poor regularity propertics (c.g., exploding scores, constraints). The proposed method demonstrated series of challenging photon-starved imaging experiments with as little photon per pixel, where it delivers remarkably accurate solutions and outperforms alternative strategies from the state-of-the-art.

---

Published in Transactions on Machine Learning Research (01/2025)
1 Introduction
Modern computer vision systems are increasingly required to operate in extreme conditions (e.g,, ultra- fast acquisition times, low illumination, long-range, unconventional environments). This has led to the development of now quantum quantum-cnhanced imaging systems that exploit the particle nature of light to excood the limitations of classical imaging strategies (iisson al. 2020
The measurements produced by quantum-enhaneen cameras photon-starved and exhibit challenging noise statistics, such as binomial, geometric, and low-intensity Poisson noise (Altmann et al. 2017a b). The top row of Figure depicts an example of an urban scene as acquired by three different imaging systems involving Poisson, binomial, and geometric noise. Poisson-distributed measurements arise from the use of single-photon detectors capable of counting individual photons within a given time period (Altmann et al. 2017b; Rapp & Goyal 2017 Shin al., 2015; Ma et al.) (0177) or standard charge-coupled device and complementary 2008) that operate under poorly illuminated conditions. Mcasurements with binomial and geometric statistics arisc from singlo-photon detectors (that is, detectors that are not able to accurately quantify photons beyond the first detection) such quantum imaging sensors
metal oxide semiconductor cameras (Duarte
Fossum et al. 2016 Ma et al. 2019) and single-photor avalanche diodes (Altmann et al. 2017b
Eisaman
ct al. 2011
Kirmani et al. 2014 Rapp & Goyal 2017
Denoising
Deblurring
Denoising
Denoising
Inpainting
Measurements
True image
Our method
(a) Poisson
(b) Binomial
(c) Geometric
Figure Restoration examples of our method: ground truth, measurements, and restored images for five image restoration tasks involving Poisson, binomial, and geometric noise.
Performing image restoration in problems involving photon-starved measurements difficult and requires the development of specialised techniques. Farly methods for low-photon imaging performed image restoration by minimising an energy function composed of a sum of data term and handerafted prior (Oh et al.
2014 Chouzenoux et al. 2015: Figueiredo & Daoucas-Dias
2010 Harmany et al. 2012 Setzer et al. 2010 Rapp & Goyal 2017 Peng et al. 2020 Such methods are able to capture the structural properties of the image but struggle to restore fine detail. More recent methods are largely based on deep learning techniques.
End-to-end strategies train a neural network to take as input the photon-starved measurements and return
a reconstructed image (Sanghvi et al. 2022 Al-Shabili et al. 2022 Choi et al. 2018 Chi et al. 2020 Huang
et al. 2023 Kang et al. 2023). Although such strategies can achieve remarkable performance controlled environments, they are problem-speeifii and difficult to deploy flexibly. Alternatively, the so-called Plug- and-Play (PnP) approaches combine pre- trained data-driven prior encoded by a deep neural network, with
explicit data fidelity term that is specified during test time (Hurault et al. 2024 Ryu et al. 2019 Marais & Willett 2017: Rond et al. 2016 Cai et al. 2023 Luo et al.) 2024h. In particular, PnP approaches based on denoising diffusion models deliver unprecedented performance in problems involving Gaussian noise or mild Poisson noise through Gaussian approximation (Chung et al.) 2023). However, for problems involving severe Poisson noise or binomial and geometric noise as discussed this paper, the Gaussian approximation proves highly inaccurate and leads poor image reconstructions with PnP methods based on denoising diffusion models (see Section
2

---

Published in Transactions on Machine Learning Research (01/2025)
This paper presents the first PnP denoising diffusion method suitable for photon-starved image restoration problems arising in quantum-enhanced computer vision. The proposed method combines ideas of Bayesian inference and optimisation to provide accurate restoration results in a highly computationally efficient man- ner. In particular, contrary to existing state of the art methods, our approach docsn't employ the Gaussian approximation for the likelihood but instead makes appropriate use of the true one. The proposed method is demonstrated on a series of challenging photon-starved imaging problems and comparisons with alternative approaches from the state-of-the-art (see Fig. []. These experiments are carried out on the FFHQ (Karras et al. 2019 and ImageNet (Deng et al. 2009 datasets.
2
Background
2.1
Low-Photon Imaging
We consider the recovery an unknown image x' E Rd from a set of photon-starved measurements y E R related to x² by a statistical model with likelihood x + po(y))). Canonical examples include image restoration problems involving low-photon Poisson, binomial, or geometrie noise. In the case of Poisson noise, y/ , Mun] No consists of discrete photon counts, modelled as realisation of the Poisson distribution
y|æ ~ P(a H(x")),
(1)
where H is a non-negative linear operator modelling deterministic aspects of the imaging system (often distortions such as non-uniform gain, blurring operator etc.) and a > 0 is related to the strength of shot noise (the larger a, the easier the image restoration problem). The respective negative log-likelihood fu(x) 2 (y)la()) is given by
fx(x) = Σ (a. H(x)) - yi log((a H(x))) + log(y.))
(2)
where all Yr conditionally independent given x. While the Poisson noise model is widely adopted to model random photon counts, single-photon detectors can result in more complex observation models when are utilised at the edge of their ideal working conditions (Altmann et al. 2017b Fisaman et al.) 2011 Kirmani et al. 2014 Rapp & Goyal 2017 Fossum et al. 2016 Ma et al. 2019 This is typically the case for detectors such as single-photon avalauche diodes that need be reset after each photon detection, leading to dead-time during which incoming photons can be missed. If the detectors are reset regular time intervals (or acquisition period), the mcasurement image y [y1 ..., Um] E N is better modelled set of sums of binary detections {0, }} Bernoulli random variables, that is,
y|æ ~ Bin (1, 1 (
(3)
where Bin(..) stands for the product of independent binomial distributions, E N is the mumber of repetition periods for each pixel, H is again non-negative linear operator, and a >> 0 is an efficiency parameter that controls the probability of successful detection. Note that Eq. 1 is good approximation of Eq. in practice
if a in Eq. 3 is sufficiently small. The respective negative log-likelihood is given by
fz(
Σ yi log(1 - e H(a)) + a (4, - -.)((((()))))
(4)
Lastly, in first-photon imaging systems (Kirmani al. 2014 Liu et al. 2017 Peng et al. 2020), the detector records the number of acquisition periods, repetitions, until the first photon detection, for each pixel individually. Then, y ..., ym] Ny is related to x* by
y|z ~ Geo (1-e -a--=)))()))
(5)
where Geo(.) stands for the product of independent geometric distributions. The associated negative log- likelihood is
fo(z)= = Σ (u - 1) -o(x))(()))))))))))(()())))))H - log (1 e ta (c)))))))))
(6)
3

---

Published in Transactions on Machine Learning Research (01/2025)
2.2 Score- Based Denoising Diffusion Models
Score-based denoising dilfusion models are slate-of-the-art generative modelling stralegy to draw samples from a target distribution on R,, known only through sample {2 The models are constructed from a stochastic transport map between d-dimonsional Gaussian distribution (0, Ia) and æ. In particular, for an appropriate choice of the drift and diffusion coefficient 9, solutions of the stochastic differential equation (SDE) as satisfied below
dx; h(x,, t)dt + g(t)dw, Xo ~ TT
satisfy that X2 ~ La) as 00. Here is the d-dimensional Brownian motion. The reverse-time SDE associated to Eq. 7 is given by (Anderson
1982
dx: (x(x.))) g(1) log (x g())dw, Xxo ~ (a,
(8)
where (x,) is the marginal density of at time I, w is a d-dimensional Brownian motion, and now flows backwards from infinity to t= 0. Score-based denoising diffusion models use Eq. together with weighted score-matching techniques (Hyvärinen & Dayan 2005 Vincent 2011 and specialised architectures to learn
an accurate neural-network approximation of the score function s + V Pc() (Song et al., 2021) New samples from æ are then generated by using the estimated scores to approximately solve Eq. g This is achieved by using discrete-time approximation of Eq. initialised with Xy N Ia) for large finite T.. A popular specific choice of Eq.
dx. = 2 (())) + /B(t)dw,
(9)
1 3((VV dx, = - loggi()))( di + VB((jaw,
(10)
where t ++ [t) E IR,"" is scheduling function. An Euler- Halle-Maruyama approximation of Eq. 8 for the specific choice Eq. 10 leads to the popular Denoising Diffusion Probabilistic Model (DDPM) (Ho et al. 2020). A more sophisticated approximation of this backward process leads to the Denoising Diffusion Implicit Model (DDIM) (Song et al. 2020), which requires significantly less discretisa ton steps than DDPM and is therefore more computationally efficient (as it requires less neural function evaluations).
2.3 Image Restoration with Denoising Diffusion Models
Following the remarkable success of denoising diffusion models in generative modelling tasks, there has been a great interest in adapting them for image restoration. In principle, this can be achieved by modifying Eq. X to
use the conditional score s + V Po (sly). However, the resulting diffusion models are problem-speccfic and cannot be deployed flexibly. This has led to significant research on strategies that combine foundational pre- trained DDPM or DDIM prior, with explicit likelihood function specified during test time. In particular, the conditional score function can be decomposed as V log (xaly) = V logge(zz)) + V log (v)aa)) where the score V log (x,) is encoded in DDPM or DDIM targeting л. However, evaluating the likelihood score V log Pe (3)(a)) requires computing the intractable integral pr (y]lae) = (= which is not feasible even the likelihood po(y|æ)) is known.
To address this difficulty, a number of different approaches have been proposed. Notably, diffusion posterior sampling (DPS) (Chung al. 2023) introduces guidance term to modify foundational DDPM
to target (Xoly y) instead of Xg. More precisely, each iteration of DDPM is corrected by gradient step along Vx, log Po Po(y) (x,)) that promotes consistency with y, where (x(x)) is an estimate of the mean of (xo xx) derived from the scores underpinning the DDPM through Tweedic's formula. Though DPS can deliver state- of-the-art results, it requires exlensive fine-tuning and has a high computational cost. The pseudoinverse- guided diffusion model (IIGDM) (Song et al.) 2022) and the denoising diffusion restoration model (DDRM) (Kawar et al. (222)) improve on DPS by adopting DDIM strategy that is significantly more computationally efficient, and improving the approximation of V (y(lr) for the specific case of Gaussian likelihood functions. An alternative recently proposed approach (Cardoso et al. 2024) exploits sequential Monte
4

---

Published in Transactions on Machine Learning Research (01/2025)
Carlo strategy to deal with the intractable score V Vlogpr(ylz.). However, the aforementioned approaches (Kawar et al. 2022 Song et al. 2022 Cardoso et al. 2024 cannot be easily extended to the low-photon
image restoration problems considered in this paper.
Denoising diffusion models for plug-and-play image restoration (DifPPIR) (Zhu et al. 2023) modifies a foun- dational DDIM by introducing a half-quadratic relaxation to decouple the likelihood term and the prior. This relaxation leads to modified DDIM whose iterations are guided by least-squares correction step that promotes consistency with y, with the weight of the guidance increasing across iterations. DiffPIR is significantly more robust than DPS and delivers state-of-the-a performance in less iterations than compet- ing DDIM strategies. Similarly, plug-and-play split Gibbs sampler (PnP-SGS) (Cocurdoux et al. 2023 also uses half-quadratic splitting to modify a DDIM foundational prior, but relies on a stochastic sampling step to introduce consistency with y rather than a gradient step, leading to a DDIM-within- Gibbss sampler.
3 Proposed Method
Our proposed method a generalisation of DifiPIR to non-Gaussian likelihood functions that are log- concave and potcntially non-smooth. This class of likelihoods includes the Poisson, geometric, and binomial models considered above, which are log-concave and have non-negativity constraints well non-Lipschitz gradients. For presentation clarity, we recall DiffPIR in Eq. (for Gaussian likelihoods). where Bt corresponds to time discretization of the scheduling function ((t), of the implicit prior, À is a regularization controlling the balance between the foundational DDIM and the guidance term, and Ç E (0, 1) controls the stochasticity of the DDIM kernel, and a is the level of the Gaussian noise in y.
1 - Be, ar II aaa 89 is the score function
Algorithm DiIPIR
Require: {āc)T= (o, T,,,,,,
1: 2r [0)
2: for T: do
3: "0 von (rt + 0z) (0(2).))) a =0 dae/(1 at)
(()
argmin, |y - H(x)111 + m 1- =/0² 2
4: 40
5:
x4 voris!")
ar
6:
N'O, 1d)
(0) - ai-1 (V+- V(a)
7: 2t-1 1a
8: end for
9: Return
Algorithm 2 Proposed generalisation of DiffPIR (ProxDiffPIR)
Require:
{ [ S.
1: Øт
L0)
2: for T: do
20
Vin
(xt + - 80) p Aan((1 aa))
4:
a(t)
prox
Pt
(1)
arg arg f5(x) 4 le 2011 1=
p
5:
21 vāra!")
VI-ai
6:
(0
7:
21-1 = 10 (1)
+ vI- 111-0-i(1--e++
VI-
8: end for
9: Return
Note that DifIPIR is standard DDIM procedure to draw samples from the prior, with the addition of a guidance step
(((() = arg min Ily H(x) 11 Pe x - x (0)
(11)
5

---

Published in Transactions on Machine Learning Research (01/2025)
This step is equivalent to a proximal step for the Gaussian negative log-likelihood function (Bauschke et al, 2017), which suggests a natural generalisat ion of DifIPR for the general class of likelihood functions discussed with the proximal step associated with fyr that
previously, That is, we could replace line 4 in Algorithm !
is,
(2(1) 20
arg min fy + Pt 2
12 20 1
fu prox
(12)
where [y(x)= - (yl given, e.g., by Eq. 2 6 depending on the type of noise involved. Note that, unlike the Gaussian case, the solution of this optimization problem docsn't admit a closed-form solution and one needs to compule a solution iteratively by using an optimization algorithm. In addition, there are positivity constraints to be taken into account because the logarithmic terms log(-) cannot be calculated when H(xx) I O, and so unconstrained optimization tools cannot be directly applied. To address this difficulty, we introduce the approximation l of fg given by l [(H(z)+ +b,, for some small positive b obtaining the new optimisation problem
Oe $0 = prox
fb (20)) = arg min (1) + Pt 2 2o
(8)
(13)
When compared to Eq. Melidonis et al. (2023)). ) Problem 13 can then be solved with a generic solver that allows the method to be deployed flexibly. We use L-BBGG-B,, which is a version of the L-BFGS method that can deal with non-negativity constraints. Of course, it is also possible to develop specialised schemes to solve Eq. 13 more efficiently for specific forms of fy and H of interest (Figueiredo & Bioucas-Dias 2010).
12 13 allows the relaxation of the strict condition H(x) > 0 to H(xx) 2 (see
The resulting generalisation of DiffPIR, namely ProxDiffPIR, is summarised in Eq. 2 below. We view Eq. 2 as an approximation of a DDIM targeting (Xuly = y) where, in the absence of the intractable score V log Pe (ylaz), we promote consistency with y by using the unconditional score V log pr(x,) logether with a log (o is
penalised least-squares correction step derived from the likelihood po(y)z). When fy (x) log-coneave, this correction step is firmly nonexpansive (a weak form of contractivity). By comparison with DPS, which relies on an explicit gradient step that is not usually firmly non-expansive, the proposed method is more robust, we illustrate in the experiments of Section 4
4 Numerical Experiments
We illustrate the performance of the proposed method through experiments related to non-blind Poisson image deconvolution, binomial image denoising, and geometric image inpainting. For these experiments, we used 30 test images from the validation datasets of FFHQ 256 x 256 (Karras et al.) 2019) and ImageNet 256 x 256 (Deng et al. 2009), and made use of pretrained difffusion models on the aforementioned dalasets (Chung et al.J 22033 Dhariwal & Nichol 2021). For Poisson image restoration, we report comparisons with alternative ocore-based diffusion methodologies and other state-of-the-art techniques. Specifically, in all experiments, ProxDiffPIR (Algorithm 2 is compared with DPS (Chung al. 2023) as implemented with the likelihood function po(y)z) without Gaussian approximation, henceforth denoted as DPS-t. In addition, for the Poisson and binomial experiments, we also report comparisons with DPS with a Gaussian likelihood approximation of the true likelihood function Po(y(æ),, as originally proposed by the authors in Chung et al. (2023). The implementation details of DPS and DPS-t are given in Appendix B We also compare with the Plug-and-Play (PnP) ADMM scheme PIP (Rond et al. 2016)), which uses a patch-based BM3D denoiser, the unrolled Plug-and-Play (PnP) network PhD-Net (Sanghvi et al., 2022), and the Bregman proximal gradient method, namely PnP-BPG (Hurault et al. 2024). For the binomial and geometric experiments, PIP is adapted by modifying the term associated with likelihood within the ADMM scheme as proposed in Mclidonis et al. (2023). Lastly, for DPS and our method, rather than presenting a single sample as solution, we generate N samples and average them to reduce the variability of our estimates (in our experiments, we use N = 3 to achieve a compromise between accuracy and computing time). We assess reconstruction quality by computing peak signal-to-noise ratio (PSNR), structural similarity index mcasure (SSIM) (Wang et al.) 2004) and learned perceptual image patch similarity (LPIPS) (Zhang et al.) 2018).. The finetuning details of ProxDifPPR are provided in Appendix A The experiments were run an Intel(R) Core(TM)
6

---

Published in Transactions on Machine Learning Research (01/2025)
19-9940X CPU @ 3.30GHz with NVIDIA GeForee RTX 2080 Ti with 11GB memory by using Pytorch. For the LBFGS-B solver, we used the scipy library. The source code will be released upon paper's acceptance.
4.1 Non-blind Poisson image deconvolution
We aim to recover x² from noisy blurred observation y P(a H(x))), where the blur operator H. is known. In these experiments, we considered two forward operators H; truncated Gaussian blur kernel of size 9 x 9 with bandwidth 3-pixels and a motion blur kernel of size 9 x 9 (see supplementary material). We tested the performance of the methods at three different photon levels (a = 5, 10,00
4.1.1 ImageNet
We now present the quantitative results for this experiment with the ImageNet datasct, which are summarized in Table [ We observe that ProxDilPPI is very compelitive across all metrics considered, while DPS strategies can produce unstable results even after extensive finetuning (see Appendices B.1.3 and B.2.1 for more details). For example, DPS-t was numerically explosive for all images in the case of motion blur and for some images under Gaussian blur, while DPS was unstable for some experiments under photon levels and a 20 In the presence of high noise (a 5), DPS be competitive in LPIPS but not in PSNR. Tn all other cases, observe that IroxDiffPIR performs similarly better than DSP in terms of LPIPS. We have noticed that the low performance of DPS in terms of PSNR is due to noisy reconstructions in many testing images, see Figure 2 This is also the case in the presence of medium noise (a = 10); see Table D and Figure 2 In Figure 2 we also observe that alternative deterministic strategies cannot offer recoveries with fine details (which quantitatively leads to high LPIPS). In contrast, our method outperforms the DPS and deterministic methods by providing int-ietaieed and smooth reconstructions. The illustration in Figure 2 is in agreement with the quantitative results reported in Table | In this experiment, our method requires around 37 seconds per image, while DPS-t and DPS need 520 seconds per image. See Table 5
Table 1: Non-blind Poisson image deconvolution experiment: quantitative image restoration results (average over 30 images from the ImageNet validation dataset). For each quality metric, the best result. is shown in bold and the second best is underlined.
Kernel
Metrics
DPS
PmP-BM33
PhD-Net
B-PnP
PSNR
22.60
16.66
20.25
20.25
22.39
20.16
Gaussian deblur
SSIM
0.60
0.26
0.56
0.55
0.60
0.58
LPIPS
0.42
0.60
0.42
0.54
0.52
0.51
high noise
PSNR
22.86
21.01
22.05
22.41
20.23
motion deblur
SSIM
0.60
LPIPS
0.41
0.39
0.53
0.51
0.50
PSNR
23.14
16.55
17.59
22.66
23.02
21 21.58
Gaussian deblur
SSIM
0.63
0.30
0.61
0.63
0.62
= 10
LPIPS
0.39
0.62
0.56
0.51
0.49
0.48
PSNI
23.75
23.13
medium noise
motion deblur
SSIM
0.66
0.61
0.62
0.63
0.64
LPIPS
0.38
0.88
0.50
0.48
0.46
PSNK
23.63
17.87
23.20
23.52
21.41
Gaussian deblur
SSIM
0.65
0.49
0.33
0.63
0.65
0.64
20
LPIPS
0.37
0.47
0.45
PSNR
24.16
21.64
23.56
23.74
20.91
low noise
motion deblur
SSIM
0.66
0.60
0.65
0.65
0.66
LPIPS
0.36
0.40
0.47
0.45
0.11
4.1.2 FFHQ
The quantitative results for this experiment are summarized in Table 2 We observe that the diffusion- based strategies ProxDifiPIR and DPS are very competitive compared to deterministic alternatives in terms of all the metries. This is in agreement with Figure 3 where we observe that ProxDiffPIR and DPS achieve very fine-detaledd reconstructions compared to non-diffusion stralegies. On the other hand, DPS-t leads to noisy reconstructions or to exploding gradients (see, for example, the results under motion blur in Table D. Among diffusion-basod strategics, our method achieves the best performance PSNR and SSTM, while DPS slightly outperforms our method in average LPIPS by a maximum difference of 0.02. We would note that LPIPS is perceptual metric and thus has the tendency to favour images with fine details, even in

---

Published in Transactions on Machine Learning Research (01/2025)
(a) True
(b)
(c) DPS-t
(d) DPS
(e) Ours
(f) BM3D
(g) PhD-Net
(h) B-PuP
Figure 2: Poisson deblurring example (a 5 and 10) with Gaussian blur on ImageNet dataset.
8

---

Published in Transactions on Machine Learning Research (01/2025)
Table 2: Quantitative mean and standard deviation results over 30 images from the FFHQ validation dataset for the Poisson deblurring problem.
Noise Level
Kernel
Metrics PENK
ProxDIIPIR
DFS4
DPS
BMSD
Fin--Net
B-PmP
24:77
20.35
22.00
23.27
28.72
21:21
Gaussian blu
85IM
0.70
0.49
0.68
0.64
0.45
0.84
LPIPS
0.33
0.48
0.31
0.54
0.51
0.49
high noten
Motion bluc
PENR
SSIM
26.05
0.72
22.07
0.70
23.42
0.66
28.75
0.06
21.03
0.65
LPIPS
0.31
0.30
0.58
0.49
0.48
FANA
2b.12
19.94
23.00
24.20
20.30
22.79
10
Gaussian blur
LPIPS
SSIM
0.71
0.31
0.42
0.52
0.28
0.70
0.51
0.67
0.48
0.08
0.69
0:46
medium nolse
Motion blur
PSNNI
SEIM
25.12
0.72
23.70
0.71
21:21
0.68
21:01
0.85
22.61
0.69
LPIPS
0.29
0.29
0.49
0.43
0.45
PSNR
25.85
20.20
28
24.83
25-18
32.71
20 Gauesian hlur
SSIM
0.74
0.40
0.71
0.70
0:71
0:71
LPIPS
0.28
0.52
0.28
0.48
0.43
0:43
low nolee
Motion blur
PSNN
8SIM
26.30
0.77
24.76
0.74
25.13
0.71
25.32
0.71
22.00
0.71
LPIPS
0.28
0.27
0.46
0.43
0.42
the presence of sharpening artifacts; we are therefore not overly concerned about minor variations in LPIPS performance. Moreover, it should be noted that DPS required highly computationally expensive fine-tuning to produce these results (see Appendices B.1.3 and B.2.1 for more details), and despite the finc-tuning DPS is not robust. For example, Figures 3 we noticed that DPS fails correctly estimate the brightness of the true image (leading low PSNR values); this is due the bias resulting from the use of an approximate likelihood function. Also note that DPS reconstructions exhibit some artifacts (sce, for example, the last two columns in Figure 3 By comparison, method consistently delivers strong results. In addition, for this experiment, our method required 23 seconds per image, whereas the DPS methods required 110 seconds.
See Table 6

---

Published in Transactions on Machine Learning Research (01/2025)
(a) True
(b)
(c) DPS-t
(d) DPS
(e) Ours
(f) BM3D
(g) PhD-Net
(h) B-PaP
Figure 3: Poisson deblurring example (a = 5) with motion blur the FFHQ dataset.
10

---

Published in Transactions on Machine Learning Research (01/2025)
4.2 Binomial denoising
We aim to recover x* from noisy observation Bin(t, - e e-ax"). We consider two low-photon scenarios: (a = 2.5, f = 10) and (a = 0.25, i 100). We present results with the ImageNet and FFHQ datasets, as summarized below.
Table 3: Binomial denoising experiment: quantitative image restoration results (averaged over 30 images from the FFHQ and ImageNet validation datasets). For each quality metric, the best result is shown in bold and the second best is underlined.
Noise Level
Dataset
Metrics
Ours
DPS-t
DPS
PaP-BM3D
PSNR
FFHQ
SSIM
0.85
0.76
LPIPS
2.5,8 = 10
PSNR
27.97
26. 22
ImageNet
SSIM
0.81
0.73
LPIPS
0.24
0.40
PSNR
29.83
26.96
FFHQ
0.78
LPIPS
0.20
0.39
PSNR
29.45
85
ImageNet
SSIM
0.85
0.76
L.PPS
0.20
0.37
3 BATT
46 ARTES
ARTES
RUXU
TOROO
Ooo
(a) True image
(b)
(c) Ours
(d) PnP-BM3D
Figure 4: Binomial denoising example (a= 2.5, = 10) on ImageNet datasct.
4.2.1 ImageNet
The quantitative results for this experiment are summarized in Table 3 Similarly to previous experiments, our method is very competitive across all metrics. It should be noted that the quantitative results for DPS-t and DPS have not been reported in Table 3 because DPS methods led to explosion gradients for most of the tested images considered. Figure | depicts three examples of a binomial denoising experiment where we observe that our method successfully recovers fine details that PnP-BM3D is not able to capture. Figure 5) we show another three example cases of binomial denoising experiment with a 2.5, 10, where DPS-t did not lead exploding gradients. We observe that the recoveries of DPS include finer details than PnP-BM3D but they also noisy even after finetuning. For instance, using the same hyperparameter
11

---

Published in Transactions on Machine Learning Research (01/2025)
settings in DPS that yield smooth reconstruction in the third row of Figure 3 will result in noisy recoveries in the other two rows. In this experiment, our method requires 93 seconds per image average, whereas DPS-t and DPS need 520 seconds per image. See Table 5
(a) True image
(b)
(c) Ours
(d) DPS-t
(e) PnP-BM3D
Figure 5: Binomial denoising example (a = 2.5,t = 10) on ImageNet dataset.
4.2.2 FFHQ
The quantitative results for this dataset are summarized in Table 3 Again, we observe that our method is very competitive in terms of all metrics considered (outperforming competing strategies significantly noted that quantitative results for DPS-t and DPS have not been reported in Table 3 because DPS methods led exploding gradients for all the testing images. In Figure G we illustrate examples of binomial denoising problem in the case of 2.5, 10. Note that met delivers sharp restorations that clearly outperform PMP-BM3D. Our method requires 80 seconds per image, while DPS-t and DPS require 110 seconds. See Table 6
PSNR and LPIPS). should
(a) True image
(b) y
(c) Ours
(d) PaP-BM3D
Figure 6: Binomial denoising example (a 2.5, f 10) FFHQ dataset.
12

---

Published in Transactions on Machine Learning Research (01/2025)
4.3 Geometric inpainting
Lastly, we now consider an extremely challenging geometric inpainting problem, where seek to estimate x² from a measurement Geo(1 ( where H is random mask with Bernoulli random entries with probability p = 0.5. We report results for the ImageNct and FFHQ datascts, as summarized below.
Table 4: Geometric inpainting experiment: quantitative results for the FFHQ and ImageNet validation datasets (30 test. images per dataset). The best result is shown in bold and the second best is underlined.
Noise Level
Dataset
Metrics
Ours
PnP-BM3D
PSNR
22.67
21.52
FFHQ
SSIM
0.72
LPIPS
0.31
0.55
PSNR
21.03
20.50
ImageNet
SSIM
0.61
0.57
LPIPS
0.44
0.54
4.3.1 ImageNet
The quantitative results for this last experiment summarized in Table 4 Three representative examples see in Table
are presented in Figure 7 As PnP-BM3D is competitive in PSNR and SSIM since it succeeds in removing heavy noise, but it is less competitive in LPIPS since it cannot recover fine details. It is important to highlight that the original DPS method proposed by Chung et al. (2023) encountered exploding gradients. Figure 7 shows that our method provides accurate reconstructions. Note that the DPS method not applicable due to exploding gradients. In this experiment, our method requires 260 seconds per image on average. See Table
method outperforms PnP-BM3D in all metrics.
GB WESTEDOO VILLAGE MAOFTMIT
esTenn VILLAGE
VILCAGE
(a) True image
(b) y
(c) Ours
(d) PnP-BM3D
Figure 7: Geometric inpainting example (a 0.025) ImageNet dataset.
13

---

Published in Transactions on Machine Learning Research (01/2025)
4.3.2 FFHQ
The quantitative results for this dataset are summarized in Table The proposed method clearly out- performs PnP-BM3D as a competing strategy, achieving significantly better PSNR and LPIPS values. is important to highlight that the original DPS method proposed by Chung et al. (2023) encountered ex-
ploding gradients. In Figure [ we present three representative examples of the challenging inpainting task considered. We observe that the proposed method delivers accurate estimates, whereas PnP-BM3D fails to recover fine detail. In this experiment, our method requires 215 seconds per image average. See Table 6
(a) True image
(b) y
(c) Ours
(d) PnP-BM3D
Figure Geometric inpainting example (a 0.025) on FFHQ dataset.
5 Conclusion and limitations
This paper presents ProxDiffPIR, generalisation of the score-based denoising diffusion model DiffPIR for PnP image restoration problems involving photon-starved measurements. The proposed method can han- dle any likelihood function that is log-concave and potentially non-smooth. In particular, this allows the application of the method to problems involving Poisson, binomial, or geometric likelihood functions, as commonly encountered in modern quantum-enhanced imaging problems. The mothod combines a founda- tional pre-trained diffusion model as an implicit prior and likelihood function specified during test time. The likelihood is involved through a penalised least-squares step that we solve itcratively by using a quasi- Newton method. Extensive image restoration experiments in challenging scenarios with binomial, geometric, low-intensity Poisson noise show that method consistently delivers accurate solutions and outperforms alternative strategies from the state-of-the-art.
The computational lime of our proposed method ProXDidРII and the baseline DPS are summarized in Table and Table 6 As we can see the computational efficiency ProxDifiPIR directly linked to the iterative solution of the penalised lcast-squares step. As shown in the tables, computational time increases with the complexity of the problems, progressing from Poisson noise to binomial and geometric noise. Despite this, ProXRI is consistently faster than DPS in all cases, except for geometric inpainting the FFHQ
14

---

Published in Transactions on Machine Learning Research (01/2025)
dataset. However, as indicated in Table 4 DPS suffers from explosive gradients, leading to failed image reconstructions. Future work should explore more efficient solvers for specific image restoration problems of interest. Moreover, although this correction step promotes measurement consistency, there remains some bias. An interesting direction for future research would be to explore randomised correction steps that reduce this bias. Also, although the reconst quality of our approach superior to all other methods, would be interesting develop other techniques that are better able to recover subtle dctails.
Lastly, in Appendix [ explore several modifications of the original DPS methods that possibly lead to more stable algorithms. Further results based these modifications are given in Appendix D For some problems and experiments, our proposed modifications to DPS are able to improve its stability and deliver restorations that are comparable to ProxDidPPIR. However, modified DPS remains significantly slower than ProxDifFIR..
Table 5: Computational time comparison of ProxDifPII and DPS for experiments conducted on the Ima- geNet dataset.
ImageNet dataset
ProxDiffPir
DPS
Poisson
37 (s/per image)
520 (s/per image)
Binomial
93 (s/per image)
520 (s/per image)
Geometric
260 (s/per image)
520 (s/per image)
Table 6: Computational time comparison of ProxDiffPIR and DPS for experiments conducted on the FFHQ datasct.
FFHQ dataset
Prox PiffPPr
DPS
Poisson
23 1s/per image)
110 (s/ 1s/per image)
Binomial
80 (s/ /eper image)
110 //per image)
Geometric
210 (s/ per image)
110 (s/ per image)
Acknowledgments
This work was supported by the UK Research and Innovation (UKRI) Engineering and Physical Sci- ences Research Council (EPSRC) through grants EP/T007346/1, EP/V006134/1, EP EPPVV06117/1, and EP/W JEP/W00673/1, and by the EPSRC Centre for Doctoral Training in Mathematical Modelling, Analysis, and Computation (MAC-MIGS), funded by the EPSRC (grant EP/S023291/D Heriot-Walt University, and the University of Edinburgh. SM acknowledges that the research contribution on this work was undertaken while he was affiliated with Heriot- Watt University; he has since joined Forschungzeenrrum Juelich.
References
Abdullah H. Al-Shabili, Xiaojian Xu, Ivan Selesnick, and Ulugbek S. Kamilov. Bregman plug-and-plan priors. In 2022 IEEE International Conference Image Processing (ICIP), pp. 241 211-245, 2022. doi: 0Tt.://ICI6667.22222222..
Yoann Altmann, Reuben Aspden, Miles Padgett, and Steve McLaughlin. A Bayesian approach to denoising of single-photon binary images. IEEE Transactions Computational Imaging, 3(3):460 471, 2017a.
Yoann Altmann, Steve McLaughlin, and Miles Padgett. Unsupervised restoration of subsampled images con- structed from geometric and binomial data. In 2017 IEEE 7th International Workshop Computational Advances in Mulli-Sensor Adaptive Processing (CAMSAP), pp. 1-5, 2017b.
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-322 1982.
Heinz Bauschke, Patrick Combettes, Heinz H Bauschke, and Patrick L Combettes. Correction to:
Conver Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, 2017.
15

---

Published in Transactions on Machine Learning Research (01/2025)
Ziruo Cai, Junqi Tang, Subhadip Mukherjee, Jinglai Li, Carola-Bibiane Schönlieb, and Xiaoqun Zhang, NF-ULA: Normalizing flow-based unadjusted Langevin algorithm for imaging inverse problems. SIAM Journal on Imaging Sciences, 2023.
Gabriel Cardoso, Yazid Janati el idrissi, Eric Moulines, and Sylvain Le CorfI. Monte Carlo guided denoising diffusion models for Bayesian linear inverse problems. In The Twelfth International Conference on Learning Representations, 2024. URL htttps:/ wttp://wwww..eetooooniieieie..ee wwt.:eeeeotnnniiem 7id NHES
Yiheng Chi, Abhiram Guanasambandam, Vladlen Koltun, and Stanley H. Chan. Dynamic low-light imaging with quanta image sensors. In Computer Vision-ECCU EECV 2020: 16th European Conference, Glasgow, UK August 23 28, 2020, Proceedings, Part XXI 16, pp. 122 138. Springer, 2020.
Joon Hee Choi, Omar A. Elgendy, and Stanloy H. Chan, Image reconstruction for quanta image sensors using deep neural networks. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6543 6547, 2018. doi: 10.1109/1CIA (0000999IASSPPP00 201.AASS--.....61....55
Emilie Chouzenoux, Anna Jezierska, Jean- ChariChrist Pesquet, and Hugues Talbot. A convex approach for image restoration with exact Poisson- Gaussian likelihood. SIAM Journal on Imaging Sciences, 8, 03 2015,
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mecann, Mare Louis Klasky, and Jong Chul Ye. Diffu- sion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023. URL
Florentin Coeurdoux, Nicolas Dobigeon, and Pierre Chainais. Plug-and-play split Gibbs sampler: embedding deep generative priors in Bayesian inference. arXiv preprint arXiv:2304.111: 11134, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. Icce, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780 8794, 2021.
Marco F. Duarte, Mark A. Davenport, Dharmpal Takhar, Jason N. Laska, Ting Sun, Kevin F. Kelly, and Richard G. Baraniuk. Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25(2):83 91, 2008, doi: 10.1109/
MAL109/MMPP220 MSP. NSSP.2007.90 0914700.
M. Eisaman, Jingwen Fan, Alan Migdall, and Sergey Polyakov. Invited review article: Single-photon sources and detectors. The Review of scientific instruments, 82:071101, 07 2011.
Mário A. T. Figueiredo and José M. Bioucas-Dias. Restoration of Poissonian Images using Alternating Direction Optimization. IEEE Transactions on Image Processing, 19(12):3133-3144 2010.
Eric R. Fossum, Jiaju Ma, Salch Masoodian, Leo Anzagira, and Rachel Zizza. The quanta image sensor:
Every photon counts. Sensors, 16(8), 2016. ISSN 1424-8220. doi: 10.3390/s16081266. URL https:
hwww..ddp: htddpi.commii. www.eommmmmm/eee.. 12224-2220116//1100
Graham Gibson, Steven Johnson, and Miles Padgett. Single-pixel imaging 12 years on. Oplics Express, 28,
08 2020.
Zachary T. Harmany, Roummel F. Marcia, and Rebecca M. Willett. This is SPIRAL-TAP: Sparse Poisson intensity reconstruction algorithms- Seaeithmm--thoryy and practice. JEEE Transactions on Image Processing, 21 1084 1096, 2012.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 6851, 2020.
Zhuo Huang, Miaoxi Zhu, Xiaobo Xia, Li Shen, Jun Yu, Chen Gong, Bo Han, Bo Du, and Tongliang Liu. Robust generalization against photon-limited corruptions via worst-case sharpness minimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognétion, pp. 16175-16185, 2023.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Samuel Hurault, Ulugbek Kamilov, Arthur Leclaire, and Nicolas Papadakis. Convergent Bregman plug-and- play image restoration for Poisson inverse problems. Advances in Neural Information Processing Systems. 36, 2024.
Aapo Hyvärinen and Peter Dayan. Estimation non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
Xiaoyang Kang, Tao Yang, Wenqi Ouyang, Peiran Ren, Lingzhi Li, and Xuansong Xie. Ddcolor: Towards rhoto-realistic image colorization via dual decoders. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pp. 228-338, 2023.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVP conference on computer vision and pattern recognition, pp. 4401-4410 2019.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Aduances in Newral Information Processing Systems, 35:23593-2588 23606, 2022.
Ahmed Kirmani, Dheera Venkatraman, Dongeek Shin, Andrea Colago, Franco N. C. Wong, Jeffrey H. Shapiro, and Vivek K Goyal. First-photon imaging. Science, 343(6166):55 61, 2014.
Xialin Liu, Jianhong Shi, Huichao Chen, and Guihna Zeng. First-photon ghost imaging at low light level. In 2017 Conference on Lasers and Flectro- Opticss (CLEO), pp. 1-2, 2017.
Ziwei Luo, Fredrik K Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B Schön. Photo-realistic im- age restoration in the wild with controlled vision-language models. In Proceedings of the IEEE/CVF Conference Compuler Vision and Pallern Recognition, pp. 6641 6651, 2024.
Jiaju Ma, Saleh Masoodian, Dakota A. Starkey, and Eric R. Fossum. Pholoo--ummer--esll megapixel im- age sensor at room temperature without avalanche gain. Optica, 4(12):1474 1484-1481,,, Dec 2017. doi: 10.1364/ OPTICA.4.001477 URL https://opg optica.
Jiaju Ma Ma, Wing Chung, Abhiram Gnanasambandam, Stanley H Chan, and Saleh Masoodian. Photon- counting imaging with multi-bit quanta image sensor. In International Image Sensor Workshop (IISW), 2019.
Willem Marais and Rebecca Willett. Proximal-gradient methods for Poisson image reconstruction with BM3D-based rogularization. In 2017 IEEE 7th International Workshop on Computational Advances in Mulli-Senson Adaptive Processing (CAMSAP), pp. 1 1-5, 2017. doi: N0.100//AMMAAP.201...
Savvas Melidonis, Paul Dobson, Yoann Altmann, Marcelo Pereyra, and Konstantinos Zygalakis. Efficient
Bayesian computation for low-photon imaging problems. SIAM J. on Imaging Sciences, 16(3):1195-12344 2023. doi: 10.1137/22M1502220 URL https::/ddii.org/// http:///w...rrr11.111///////11// 117722212112
Albert K. Oh, Zachary T. Harmany, and Rebecca M. Willett. To or not to e in Poisson image reconstruction. Tn 2014 IEFE International Conference on Image Processing (ICIP), pp. 2829 2833, 2014. doi: 10.1109/ SCIP.2014.70P5 70255777.
Xiao Peng, Xin- Yu Zhao, Li-Jing Li, and Ming- Sun. First-photon imaging via hybrid penalty. Photonics Research, 8(3) ((3):325-330, 2020.
Joshua Rapp and Vivek K Goyal. A few photons among many: Unmixing signal and noise for photon-efficient active imaging. JEEE Transactions on Computational Imaging, 8(3):445-453 2017.
Arie Rond, Raja Giryes, and Michael Elad. Poisson inverse problems by the plug-and aong-add-lay play scheme. Journal of Visual Communication and Image Representation, 41:96-108, 2016. ISSN 1047-3203. doi: https: //doo.org/1...01//...v..... $10473203163019855
17

---

Published in Transactions on Machine Learning Research (01/2025)
Ernest Ryu, Jialin Liu, Sicheng Wang, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Plug-and-play methods provably converge with properly trained denoisers. In Kamalika Chaudhuri and Ruslan Salakhut- dinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Pro- ceedings of Machine Learning Research, pp. 5546-5557. PMLR, 09-15 Jun 2019.
Yash Sanghvi, Abhiram Gnanasambandam, and Stanley Chan. Photon limited non-blind deblurring using algorithm unrolling. IEEE Transactions on Compulational Imaging, 8:851 864, 2022.
Simon Setzer, Gabriele Steidl, and Tanja Teuber. Deblurring Poissonian images by split Bregman techniques. Journal of Visual Communication and Image Representation, 21(3):19-199, 2010.
Dongeek Shin, Ahmed Kirmani, Vivek K Goyal, and Jeffrey H. Shapiro. Photon-efficient computational 3-D and reflectivity imaging with single-photon detectors. IEEE Transactions on Computational Imaging, 1 (2):112-128) 125, 2015.
Jiaming Song, Chenlin Mong, and Stefano Ermon. Denoising diffusion implicit models. ar preprint arXiv:2010.088 022000 2020.
Jiaming Song, Arash Vahdat, Morteza, Mardani, and Jan Kautz. Pseudoinverse-guidde diffiusion models for inverse problems. In International Conference on Learning Representations, 2022.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL https:://ppenrevies wttp://ww......ooorroooioo.aaaa.
Pascal Vincent. A connection between score matching and denoising autocncoders. Neural computation, 23 (7):1661- 1674, 2011.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility structural similarity. JEEE tronsactions on image processing. 13(4):600- -610000-61,,1-0 2004.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features a perceptual metric. In CVPR, 2018.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiczhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. De- noising diffusion models for plug-and-play image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Poftern Recognition, pp. 1219-1229, 2023.
A Hyperparameters Values (ProxDiffPIR)
In Tables 7 and below, we present the hyperparameter values that have been used for the ProxDiffPIR implementation for the low-photon imaging experiments and datasets that have been discussed in the main manuscript. To select the values for À and Ç that lead to the best performance, a grid search was implemented. Specifically, for all the experiments, the ranges of the hyperparameters were selected as À E [1, 10) and E [0.1, 0.9]. The incremental step for À was set to be and for Ç to be 0.1. The search was implemented by selecting 5 random images from the FFHQ and ImageNet datasets respectively.
Table 7: Hyperparameter selection for ProxDiffPIR for the FFHQ dataset.
Pojeson
(Gaussian blur)
Poisson (Motion blur)
Bin. Demoising
Geom. Inpainting
= 5
10
20
a
=
5
a= 10
a = 20
a 2.5. 10 a 0.25,
0.025
3
5
3
3
3
18

---

Published in Transactions on Machine Learning Research (01/2025)
Table Hyperparameter selection for ProxDiffPIR for the ImageNet dataset.
Poleson (Gaussian Blur)
Poisson (Motion Blur)
Bin. Denoising
Geom. Inpainting
5
10
= 20
= 5
= 10
= 20
<r 2.5, 1. 10
0.25, 0 = 102
0.025
7
6
3
B Implementation Details for DPS
In this section, we explain in more detail how the seate-of-the-ars diffusion method DPS (Chung et al. 2023) implemented to make fair comparison with ProxDiffPIR. It should be reminded that implemented two versions of DPS; DPS with Gaussian likelihood approximation, as originally proposed by the authors in Chung et al. 2023), and DPS as implemented with the true likelihood function po(u(z) (DPS-t). It should be noted here that DPS and DPS-t proved to show instabilities in some of the experiments, especially the ones related to binomial and geometric noisc. We surmise that a primary cause of these instabilitics is that DPS and DPS-t fail account for the poor regularily properties of the low-photon likelihood terms, which for example are undefined 0 due the presence of logarithmic terms.
B.1 DPS
B.1.1 The authors in Chung al. 2023) approximate the Poisson likelihood
Poisson noise
(a [H(x)),)** exp( a [H((x)})
po())z)
(14)
yj!
by
exp (y, (a H2
Po Po m II
(15)
v 2
2y
1=1
We present DPS (Chung et al. 2023 in Algorithm measurements.
for clarity of presentation in the case of Poisson
Algorithm 3 DPS Chung et al. 2023)
Require: s8. T, v. (e:) (o.) [A] Zy;
XT 10,
for = T - 1:0 do
20
(t)
Jac
(It (1 ax.) 39 (Ir;
2 NO)
2/11 1 i0 faill-due (0r(1-04-11 + oz
It-1 2 Pt V z. 118 H(())) 13
end for
Return 20
B.1.2 Binomial noise.
The authors Chung et al. (2023) do not discuss the case of binomial distributed measurements. In cases where the number of repetition periods large (t + oo), the model can be approximated by a Gaussian distribution as
po(y)>)) qp (x)
(16)
19

---

Published in Transactions on Machine Learning Research (01/2025)
where
(y, - [tpl,)²
(x(x)
exp
(17)
2x tp(1 p)l
2[tp(1 p)]]
j=1
where we recall that p= -xH)(x)
B.1.3 Finetuning.
The stepsize Pt in Algorithm 3 is crucial hyperparameter for the stability and performance of stabilized
DPS. The authors in Chung ct al. 20223 sct Pt as
Pt c/lly - H(
(18)
where Ç > 0 is set constant. We finetune by using grid search. In Tables 9 and I0 show how Ç is selected for the several Poisson experiments discussed in the main manuscript for the FFHQ and ImageNet datasets respectively.
Table 9: Hyperparameter selection for DPS for the FFHQ dataset.
(Gaussian blur)
Poisson (Motion blur)
10
20
5
10
20
0.5
0.008
0.01
0.6
0.005
0.012
Table 10: Hyperparameter selection for DPS for the ImageNet dataset.
Polsson (Gaussian Blur)
Poisson (Motion Blur)
0.2
0.008
0.008
0.4
0.003
0.003
B.2 DPS-t
The DPS algorithm can be implemented with the true likelihood function Pn (y((x)) For clarity, we refer to this version as DPS-t. For the Poisson, binomial and geometric log- log-likeihhoods fy, as presented in the main manuscript, we can implement DPS-t as shown in Algorithm
Algorithm 4 DPS-t
Require: 80, T,y. (GT)T (6t)
XT (0,
for t- 1- do
20
(t)
(It (1 āie) so t))
Vac
4:
2 ~~ (0,
#/-1 Var-18t
l-a, .o Var(1-ār-1) + ozz
5:
6:
2i-1 21-1 (V.f(y H(2o))
7: end for
8: Return 2p
B.2.1 Finetuning.
Similarly with DPS, a stepsize needs to be included DPS-t. We assume constant stepsize Ç. see Algorithm We finetune Ç by implementing grid search on random images from the FFHQ and ImageNet datascts. In Tables II and 12 we list how the hyperparameter Ç is selected for the several experiments discussed in the main manuscript for the FFHQ and ImageNet datasets, respectively.
20

---

Published in Transactions on Machine Learning Research (01/2025)
Table 11: Hyperparameter selection for DPS-t for the FFHQ dataset.
Poisson (Gaursian blur]
Poisson (Motion blur)
5
10
20
6
10 20
450
300
Table 12: Hyperparameter selection for DPS-t for the ImageNet dataset.
Poisson (Gaussian Blur)
Poisson (Motion Blur)
450
400
150
450
150
300
C
Modifications on DPS and DPS-t
In this section, we propose stabilized versions of DPS and DPS-t that deal with the poor properties, providing more stable results than their original versions depicted in Chung et al. (2023)
C.1
Stabilized DPS
Regarding DPS, simply propose to approximate the variance term
[A f lis
(19)
2[y]; +
which leads the approximation
(_(3;
(y; laa H(x));}² 2([y]; + B)
po(y(a))
II
exp
(20)
==1 V 2
where O takes small value compared to the scale of This guarantees that instabilities will occur when y O. We show the stabilized DPS for Poisson measurements in Algorithm 5
In the case of binomial noise, we also observed in our experiments that DPS was inapplicable leading to exploding gradients. DPS can be modified for binomialclistributed measurements to give more stable results similarly with the Poisson setting by approximating p = e -a-H((x) by p = 1 H(++)())))) for B >
See Algorithm for more details and Tables 13 and 14 for the specific values of B used in further experiments in Appendix
Algorithm 5 Stabilized DPS (Poisson mcasurements)
Require: 80, T,y, (o) B, (A
2y
1: Xy ~ 10,
2: for = T 1: do
3: 20 Voc
(x) + (1 Tz)) 80 (It, 1))
N),
5: 2/-1 = 1-a; in var(l Iz + 642
āi 18
I-a,
6: Fi-1 1'-1 Pr V,, |1 H(50)) 1
7: end for
8: Return 80
C.1.1 Finetuning.
The stepsize Pt in Algorithm 5 is crucial hyperparameter for the stability and performance of stabilized
DPS. The authors in Chung et al. 2023 sct Pt
Pt C/lly H(
(21)
21

---

Published in Transactions on Machine Learning Research (01/2025)
Algorithm 6 Stabilized DPS (binomial mcasurements)
Require: $0. yy (d)==::0t)1.
1: XT ~ N))
2: for 1
20 (1)
(ot (1 āi)) se (Ic,
vāz
4: 2 ~~ N(0,1)
21-1 Va--iu
l-a; :o yarll-ar-1)
Dt 6,3
1-
2i-1 2/-1 CV No
end for
8: Return 20
where Ç > is set as constant. We finelune < by using a grid search. In Tables 13 and we show how Ç is selected for the several Poisson experiments discussed in the main manuscript for the FFHQ and ImageNet datasets respectively.
Table 13: Hyperparameter selection for stabilized DPS for the FFHQ dataset.
(Gaussian blur)
Poisson
(Motion blur)
Denoising
5
=
10
a 20
= 6
10
20
= 2.5. 10
10 0.0006 0.0015
0.25,
0.4
0.5
0.1
0.1
0.1
0.1
0.01
Table 14: Hyperparameter selection for stabilized DPS for the ImageNet dataset.
Poieson (Gaussian blur)
Poisson (Motion blur)
Bin. Denoising
a = 10
a
20
a =
a
10
a 20
a 2.5.
0.25,
0.14
0.0006
0.1
0.1
0.1
0.1
0.1
0.1
0.1
0.01
C.2 Stabilized DPS-t
As have observed in Scction DPS-t either inapplicable led to suboptimal results. Note that can approximate fy by its stabilized version f9 similar to ProxDifPIIR see Algorithm 7 this can lead to more stable samples and results.
We noticed that
A Algorithm Stabilized DPS-t
Require: sg. T, 1
1: XT ~ (0,
2: for t- = 1-1:00 do
40 Jac
(x+ - (1 T+) se (Ite; 1))
4: z ~ N(0.
r/-1 1-au in varll ( rr 0,3
1
I-see
04-1 2
V n. fly,
H(20) 5)
end for
8: Return 5o
C.2.1 Finetuning
Similarly with stabilized DPS, a stepsize needs to be included in stabilized DPS-t. We assume a constant stepsize Ç, see Algorithm 7 We finctune Ç by implementing a grid search on random images from the FFHQ and ImageNet datasets. In Tables 15 and 16 we list how the hyperparameters Ç and are selected for the several experiments discussed in the main manuscript for the FFHQ and ImageNet datasets, respectively.
22

---

Published in Transactions on Machine Learning Research (01/2025)
Table 15: Hyperparameter selection for stabilized DPS-t for the FFHQ dataset.
Poieson (Gaussian blur]
Poisson
(Motion blur)
Bin. Denoising
Geom. Inpainting
5
10
= 20
= 5
= 10
=
20
(r 2.5, 10
o. 0.25, 10a
0.025
0
0.025
0.05
0.1
0.025
0.05
0.1
0.01
0.001
0.001
Table 16: Hyperparameter selection for stabilized DPS-t for the ImageNet dataset.
Poisson
(Ganssian blur)
Polsson (Motion blur)
Bin.
Denoising
Geom. Inpainting
5
10
20
0
5
10
o
20
a
2.5.1
10
a 0.25,
0.025
8
0.025
0.05
0.05
0.025
0.05
0.1
0.01
0.001
0.001
D Further Results
D.1 Poisson noise
In Tables 17 and 18 we show the quantitative comparison between ProxDiffPIR, stabilized DPS, DPS, stabilized DPS-t and DPS-t for the FFHQ and ImageNet datasets, respectively. For the FFHQ dataset, among all DPS versions, we observe that stabilized DPS-t is the most competitive in terms of all metrics. Importantly, in cases where DPS-t not applicable (see, for example, the cases where a motion blur has been used in the inverse problem), stabilized DPS-t is not only applicable, but also improves significantly over DPS-t; see also Figure 0 f and Figure 9g for visual comparison. In Figure I0 we also observe that stabilized DPS versions correct
bias, but that might not always be the case; For example, compare Figure Ødd and Figure g where the recovery from stabilized DPS does not improve over the recovery of DPS. Regarding ProXDiffPIR, we observe that it provides very competitive results in terms of PSNR and SSIM while is slightly less competitive in terms of LPIPS than stabilized DPS-t. For the ImageNet dataset, stabilized DPS-t is not as competitive as stabilized DPS; however, it can provide more competitive results in terms of PSNR in some cases. see Table 18 As illustrated in Figure T1 the stabilized versions of DPS show significant improvements compared to the original DPS versions, which corroborates the quantitative findings detailed in Table 18 However, regardless the finetuning of all DPS versions, instabilities can still occur even by using the stabilized versions of DPS, see Figure 12 Finally, it should be observed in Table 18 that ProxDifIPIR still offers more competitive performance in terms of all metrics than any version of DPS.
Table 17: Quantitative mean results over 30 images from the FFHQ validation dataset for the Poisson deblurring problem.
Metrics
23.57
22.55
20.35
Gaussian
0.68
0.31
0.31
high
29.23
22.97
Motion blue
0.70
0.35
0.30
0.30
PSNR
25.12
23.79
23.65
24.07
19.94
Gaussian
SSEM
0.71
0:70
0.71
0.42
medium noise
Motion
25.13
23.57
23.70
0:73
24:20
0.73
0.38
0.29
PSNR
25.65
23.36
24.23
24.43
20.20
Gouselan blor
SSEM
0:71
0.72
0.40
LPIPS
0.55
0.24
0.52
PSNR
20.60
22.07
24.75
26:10
SSEM
0.77
0.62
0.74
0.74
LPIPS
0.28
0.37
0.27
0
23

---

Published in Transactions on Machine Learning Research (01/2025)
Table 18: Quantitative mean results over 30 images from the ImageNet validation dataset for the Poisson deblurring problem.
Noise Levol
Kernel
Metrics
Frox PRIFIR
Stabiizee DPS
DP8
Stabilieed DPS-
DPS4
PSNR
22.60
21
20.25
20.40
16.00
Goussian blue
88IM
0.60
0.38
0.55
0.52
0.26
LPIPS
0.41
0.41
0.42
0.44
0.00
high notse
Motion blue
FSNR
88IM
22:56
0.61
21:26
0.57
21:01
0.00
21.62
0.60
LPIPS
0.41
0.41
0.39
0.89
PSAR
23.14
20.56
17.59
20.07
16.55
10 Gaussian blur
SSIM
0.63
0.51
0:30
0.52
0. 0.24
LPIPS
0.39
0.43
0.56
0.44
0.62
medium noise
Motton bluz
PSNR
88IM
23.75
0.66
21.06
0.40
21:35
0.61
21:12
01.39
LPIPS
0.38
0.339
0.38
0.:39
PSNR
23.63
20.16
18.15
19.47
17.87
20 Gausstan blar
8SIM
0.05
0.48
0.38
0.49
0.81
LPIPS
0.87
0.45
0.64
0.47
0.07
low moise
Motion blue
PSNR
SSIM
24.10
0.60
21
0.01
21:04
0.00
21:02
0.55
LPIPS
0.36
0.38
0:00
0.43
(a) True image
(b) y
(c) Ours
(d) DPS
(o) St. DPS
(I) DPS-t
(4) St. DPS-L
Figure 9: Poisson deblurring example (a 5) with gaussian blur on the FFHQ dataset.
24

---

Published in Transactions on Machine Learning Research (01/2025)
(a) True image
(b) y
(c) Ours
(d) DPS
(e) St. DPS
(f) St. DPS-t
Figure 10: Poisson deblurring example (a = 5) with motion blur on the FFHQ dataset.
(a) True image
(b) y
(c) Ours
(d) DPS
(e) St. DPS
(f) DPS-t
(g) St. DPS-t
Figure 11: Poisson deblurring example (a = 10) with Gaussian blur ImageNet dataset.
D.2 Binomial noise
we show the quantitative comparison between ProxDifIIRR and all versions of DPS for the FFHQ and ImageNet datasets. We observe that for both datasets stabilized DPS and stabilized DPS-t do not lead to exploding gradients as DPS and DPS-t do. We also observe that ProxDiffPIR provides very competitive results terms of all considered metrics and datasets with stabilized DPS-t being competitive in terms of LPIPS. In Section 4 there were example cases for which the original versions of DPS were by making of the stabilized versions of DPS since they can provide more stable results and avoid exploding gradients, By comparison to DPS strategics, our method delivers results that are more smooth (sco, c.g., the first row of Figure 13), hence DPS-t achieves a mildly better LPIPS value. However, DPS strategies might still deliver solutions that appear realistic but are not in agreement with the truth (e.g., see the colour of the background the sccond row of Figure 13 as well as the examples in Figure 14). In Figure we visualize additional comparisons between ProXDIfPIIR, stabilized DPS, DPS-t and stabilized DPS-t (original DPS led to exploding gradients for these images). A first. observation is how stabilized DPS-t significantly
In Table 19
inapplicable see Figures [ and This is not the
Figures 13 and 14 instead.
25

---

Published in Transactions on Machine Learning Research (01/202))
(a) True image
(b) y
(c) Ours
(d) DPS
(e) Stabilized DPS
(f) DPS-t
(g) Stabilized DPS-t
Figure 12: Poisson deblurring example (a = 10) with Gaussian blur on ImageNet dataset.
improve over original DPS-t providing more stable, smooth and fine-detailed results. Stabilized DPS can also provide fine-detailed reconstructions but it prone to artifacts (scc bottom of Figure 15). A second observation is that ProxDidPIR smooths out some details compared to stabilized DPS-t for the example images. Taking into account Figure 14 though, overall ProxDiffPIR proves to be more stable.
Table 19: Binomial denoising experiment: quantitative image restoration results (averaged over 30 images from the FFHQ and ImageNet validation datasets). For each quality metric, the best result shown bold and the second best is underlined.
Noise Level
Dataset
Metrics
Ours
Stabilized DPS
DPS
Stabilized
DPS-t
PSNR
28.78
23.71
27.20
FFHQ
SSIM
0.85
0.76
0.84
2.5, 10
LPIPS
0.22
0.25
0.21
PSNR
27.97
ImageNet
SSIM
0.81
0.77
0.80
LPIPS
0.24
0.29
PSNR
29.83
23.32
28.7 72
FFHQ
SSIM
0.87
0.81
0.87
0.25,
LPIPS
0.20
0.22
PSNR
29.45
23.70
29.32
ImageNet
LPTPS
0.20
0.21
0.19
D.3 Geometric noise
In Table 20 we show the quantitative comparison between ProxDifPIR, stabilized DPS-t and DPS-t for the FFHQ and ImageNet datasets. Note that DPS not used here, as approximating a geometric distribution by using a Gaussian one is, to the best of our knowledge, DPS not applicable here) invalid. We observe that for both datasets stabilized DPS-t do not lead to exploding gradients DPS-t does. We also observe that ProxDifPPIR provides very competitive results in terms of all considered metrics and datasets, with stabilized DPS-t being competitive in terms of LPIPS. Although stabilized DPS has the potential to create intricately detailed reconstructions. it may exhibit bias; see the first row of Figure 16 and the first two rows of 17 This assertion is supported by the low PSNR and SSIM performance observed for DPS in Table 20 It should be noted that some bias can be also exhibited by ProxDiffPIR, see for example bottom rows in Figures 16 and 17 However, in our experiments, we did not notice extreme cases of bias as DPS-t exhibits, for example, in the first two rows of [7 This assertion supported by the higher PSNR and SSIM performance of ProxDifIPIR compared to DPS-1.
26

---

Published in Transactions on Machine Learning Research (01/2025)
(a) True image
(b) y
(c) Ours
(d) St. DPS
(e) St. DPS-t
Figure 13: Binomial denoising example (a= 2.5, = 10) on FFHQ dataset.
14 BATT 96 ARTES
AATE
ROXU
ROOU
(a) True image
(b)
(c) Ours
(d) St. DPS
(e) St. DPS-t
Figure 14: Binomial denoising example (a= 10) on ImageNet dataset.
Table 20: Gcometric inpainting experiment: quantitative results for the FFHQ and ImageNet validation datasets (30 test images per dataset). The best result. is shown in bold and the second best is underlined.
Noise Level
Dataset
Metrics
Ours
Stabilized DPS-t
DPS-t
PSNR
22.67
19.41
PFHQ
SSIM
0.72
0.63
0.025
LPIPS
0.31
0.33
PSNR
21.03
16.60
ImageNet
SSIM
0.61
0.19
LPIPS
0.44
27

---

Published in Transactions on Machine Learning Research (01/2025)
True image
(b) y
(c) Ours
(d) St. DPS
(e) DPS-t
(f) St. DPS-t
Figure 15: Binomial denoising example (a= 2.5, t 10) on ImageNet dataset.
(a) True image
(b) y
(c) Ours
(d) St. DPS-t
Figure 16: Geometric inpainting example (a 0.025) FFHQ dataset.
28

---

Published in Transactions on Machine Learning Research (01/2025)
NESTRSDO VILLAGE COMTTRSINN
eETTaD KILLAGE
(a) True image
(b) y
(c) Ours
(d) St. DPS-t
Figure 17: Geometric inpainting example (a 0.025) on ImageNet dataset.
29


Reviews:
[{"review_id": "4UweSAdyX7", "reviewer": "Reviewer_g6Hg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Weaknesses:\n- Unclear motivation on improving the effectiveness of photon imaging. Although, this work can tackle various scenarios of problems, but what is the research problem that hinders the performance of existing works is unclear. A further justification on why the proposed method can outperform other baselines should be more clearly presented.\n- Lack of computational efficiency comparison with other baseline methods. Although this work conduct test time adaptation which removes the burden of pre-training, the efficiency for each step could still be studied to justify the efficiency.\n- Missing several closely-related references:  \nHuang et al., Robust generalization against photon-limited corruptions via worst-case sharpness minimization, in CVPR 2023  \nKang et al., Ddcolor: Towards photo-realistic image colorization via dual decoders, in ICCV 2023  \nLuo et al., Photo-Realistic Image Restoration in the Wild with Controlled Vision-Language Models, in CVPR 2024.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Summary:\nThis paper introduces a plug-and-play denoising diffusion method which aims to tackle photon-starved imaging problems. Because previous studies rely on hand-crafted priors to decide the type of photon noise, this paper aims to tackle the problem in an automated manner. To achieve this goal, the authors extends the foundational DiffPIR to handle non-Gaussian scenarios such as Poisson, Binomial, and Geometric distributions. Further, a proximal optimization step for non-Gaussian log-concave likelihoods is proposed to address non-negativity and non-Lipschitz gradient constrains. Moreover, an efficient DDIM-based diffusion is proposed to reduce computational costs meanwhile maintaining learning performance. Through high-quality visual results and quantitative comparisons, the effectiveness of the proposed method is carefully justified.\n\nStrengths:\n- The proposed method is quite adaptive and tackle various situations of photo imaging problems, which is more practical than many other existing works. Plus, the plug-and-play learning manner makes the proposed method easy to apply.\n- Rigorous mathematical formulation is provided to demonstrate the proposed method, which strengthens the theoretical credibility of the approach.\n- The efficiency improvement is quite impressive, which significantly surpasses other baseline methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see weaknesses part.", "broader_impact_concerns": "No enthic concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ykrCQTNdLE", "reviewer": "Reviewer_Abd5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The paper shows a technique to perform image restoration leveraging diffusion models, that works with generic noise models.\n- The experimental results are solid in that the proposed method beats existing ones when measuring PSNR and SSIM. Qualitatively, the image restoration performance of the method is also strong.\n\nWeaknesses:\n\n- From a methodological perspective, the contribution of the work is incremental, as the algorithm is heavily based on DiffPIR. In that sense, the work should be categorized as a computer vision paper rather than a machine learning paper. Although the work falls within the scope of TMLR, it would arguably fit better in a computer vision journal or conference, and it would receive more accurate reviews in such venues.\n\n- What is the reason that the proposed method does not beat DPS with respect to LPIPS?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents ProxDiffPIR a new Plug-and-Play denoising diffusion model designed for image restoration in extremely low-light, \"photon-starved\" conditions, where noise statistics are complex due to quantum effects. Unlike previous models, which struggle with high uncertainty and poor regularity in these settings, this approach combines a pre-trained diffusion model with Bayesian inference for adaptable and efficient restoration. Tested on challenging datasets, it outperforms other methods, achieving accurate results.\n\nThe proposed method is a direct generalization of DiffPIR, a previous work which is essentially ProxDiffPIR under the assumption that the noise distribution is Gaussian.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "No requested changes. The analysis seems complete.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DeklBjEFSJ", "reviewer": "Reviewer_6eck", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS\nIn comparison to most articles on diffusion based image restoration, one strength of this work is the use of a realistic noise model for photon-starved images. The results on measurement conditional image generation are impressive and look better, in some cases, than existing methods.\n\nWEAKNESSES\nBoth image and quantitative results are only slightly better than the existing state of the art. There were some errors or lack of clarity in the theoretical methods.  The article would benefit from a better explanation of the significance of photon-starved imaging.  It would greatly improve the work to apply the method to some physical measurements from a real photon-starved imaging system.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a diffusion based image restoration method using diffusion models to restore low-exposure or photon-starved imaging systems. Their method combines score based diffusion priors with a probabilistic measurement model of images as jointly independent Poisson, Binomial, and Geometric random variables. They modified the existing DiffPIR method for plug-and-play image restoration with a log-likelihood term corresponding to the photon-starved imaging system. Their results show effective measurement conditioned image generation with some improvements in some cases over the state-of-the art methods such as diffusion posterior sampling.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Section 1 Introduction\n\n\" photon-resolving single-photon detectors capable of counting individual photons \" \n\nThis language is too redundant.\n\n\n\"consists of discrete photon counts, modelled as a realisation of the stochastic\nprocess y|x⋆ ∼ P(α · H(x))\"   \n\n\nTo be precise, I recommend referring to this as a Poisson distribution, not a Poisson process. A stochastic process refers to a collection of random variables indexed by time or space. See references below\n\nKao, Edward PC. An introduction to stochastic processes. Courier Dover Publications, 2019.\n\nTodorovic, Petar. An introduction to stochastic processes and their applications. Springer Science & Business Media, 2012.\n\nGallager, Robert G., and Robert G. Gallager. \"Poisson processes.\" Discrete stochastic processes (1996): 31-55.\n\n\n\nIn equation (2), it looks like you have assumed the elements of the vector, y, are jointly independent. I recommend making this statement explicitly. \n\n\nIn equation (3), I think understand what you are trying to say, but the language needs to be clarified. \n\n\" , the measurement image, y = [y1 . . . , ym] ∈ Nm0 is better modelled as a set of sums of binary detections {0, 1} \"\n\nThis part is clear, however I would recommend explicitly using the term Bernoulli random variable.\n\n\"where Bin(·, ·) stands for the product of independent binomial distribution\"\n\nThis part is not clear because you are using notation with an argument for the time,  t. A Binomial distribution is defined by the sum of independent Bernoulli trials, but it is not a function of time, t.  If I understand what you are saying correctly, you have a Bernoulli process, taking value zero or one at each time point.  Then, the time integral is a Binomial random variable. However, this Binomial random variable will not be a function of time after integrating over time.\n\n\"Note that Eq. 2 is a good approximation of Eq. 3 in practice if α in Eq. 3 is sufficiently small.\"\n\nEq (2) is a Poisson log-likelihood and Eq (3) is a distribution. I recommend comparing Eq (1) and Eq (3) or comparing Eq (2) and Eq (4). \n\nSection 2.2  \n\n\n\"Denoising diffusion models use Eq. 7 together with weighted score-matching techniques\"\n\nRecommend re-phrasing to make this claim about score-based generative models or score-based diffusion models. \"Denoising diffusion models\" will make the reader assume you mean \"Denoising diffusion probabilistic models (DDPMs)\" which use a discrete-time framework and do not use score-matching techniques. \n\n\nSection 3\n\n\"That is, we could replace line 4 in Eq. 1...\" \n\nI think you mean Algorithm 1 and not Eq. 1\n\n\n\"...with the proximal step associated with f_y, that is...\"\n\nDo you need to invoke the notion of a proximal step at all? I think it would be more clear to the reader if you simply justify your method using Bayes rule, breaking a log-posterior into the sum of a log-prior (diffusion based) and log-likelihood (Poisson, Binomial, or Geometric). The notation of \"prox\" with subsrcipt f_y which is a probability density function, and superscript \\rho_t which is a scalar weight on the log-prior is confusing. My recommendation is to remove proximal operator notation from the entire article. \n\n\n\nSection 4.1\n\n\"Gaussian blur kernel of size 9 × 9 with bandwidth 3-pixels\"\n\nThe width of a Gaussian kernel is roughly six times the bandwidth, so this implementation will result in a truncated Gaussian kernel.", "broader_impact_concerns": "The improvement over the existing state-of-the-art such as DPS-t is modest, which limits the impact.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agree that the contribution presented in this paper is valuable, that the approach is clearly motivated and derived, and that the results are compelling. I concur and recommend this paper be accepted, provided the authors address the reviewers' comments in their camera-ready version.
Decision: Accept (poster)