Title: Convergent Bregman Plug-and-Play Image Restoration for Poisson Inverse Problems
Venue: NeurIPS 2023
Accepted: yes
Keywords: Plug-and-Play

Poisson Inverse Problems

Bregman distance

Proximal Gradient Descent

nonconvex and nonsmooth optimization

Poisson inverse problems
Abstract: Plug-and-Play (PnP) methods are efficient iterative algorithms for solving ill-posed image inverse problems. PnP methods are obtained by using deep Gaussian denoisers instead of the proximal operator or the gradient-descent step within proximal algorithms. Current PnP schemes rely on data-fidelity terms that have either Lipschitz gradients or closed-form proximal operators, which is not applicable to Poisson inverse problems. Based on the observation that the Gaussian noise is not the adequate noise model in this setting, we propose to generalize PnP using the Bregman Proximal Gradient (BPG) method. BPG replaces the Euclidean distance with a Bregman divergence that can better capture the smoothness properties of the problem. We introduce the Bregman Score Denoiser specifically parametrized and trained for the new Bregman geometry and prove that it corresponds to the proximal operator of a nonconvex potential. We propose two PnP algorithms based on the Bregman Score Denoiser for solving Poisson inverse problems. Extending the convergence results of BPG in the nonconvex settings, we show that the proposed methods converge, targeting stationary points of an explicit global functional. Experimental evaluations conducted on various Poisson inverse problems validate the convergence results and showcase effective restoration performance.

Convergent Bregman Plug-and-Play Image Restoration for Poisson Inverse Problems

Abstract

Plug-and-Play (PnP) methods are efficient iterative algorithms for solving ill-posed image inverse problems. PnP methods are obtained by using deep Gaussian denoisers instead of the proximal operator or the gradient-descent step within proximal algorithms. Current PnP schemes rely on data-fidelity terms that have either Lipschitz gradients or closed-form proximal operators, which is not applicable to Poisson inverse problems. Based on the observation that the Gaussian noise is not the adequate noise model in this setting, we propose to generalize PnP using the Bregman Proximal Gradient (BPG) method. BPG replaces the Euclidean distance with a Bregman divergence that can better capture the smoothness properties of the problem. We introduce the Bregman Score Denoiser specifically parametrized and trained for the new Bregman geometry and prove that it corresponds to the proximal operator of a nonconvex potential. We propose two PnP algorithms based on the Bregman Score Denoiser for solving Poisson inverse problems. Extending the convergence results of BPG in the nonconvex settings, we show that the proposed methods converge, targeting stationary points of an explicit global functional. Experimental evaluations conducted on various Poisson inverse problems validate the convergence results and showcase effective restoration performance.

1 Introduction

Ill-posed image inverse problems are classically formulated with a minimization problem of the form

where f is a data-fidelity term, g a regularization term, and λ > 0 a regularization parameter. The data-fidelity term is generally written as the negative log-likelihood f ( x ) = -log p ( y | x ) of the

probabilistic observation model chosen to describe the physics of an acquisition y ∈ R m from linear measurements Ax of an image x ∈ R n . In applications such as Positron Emission Tomography (PET) or astronomical CCD cameras [Bertero et al., 2009], where images are obtained by counting particles (photons or electrons), it is common to use the Poisson noise model y ∼ P ( αAx ) with parameter α > 0 . The corresponding negative log-likelihood corresponds to the Kullback-Leibler divergence

The minimization of (1) can be addressed with proximal splitting algorithms [Combettes and Pesquet, 2011]. Depending on the properties of the functions f and g , they consist in alternatively evaluating the proximal operator and/or performing a gradient-descent step on f and g .

Plug-and-Play (PnP) [Venkatakrishnan et al., 2013] and Regularization-by-Denoising (RED) [Romano et al., 2017] methods build on proximal splitting algorithms by replacing the proximal or gradient descent updates with off-the-shelf Gaussian denoising operators, typically deep neural networks trained to remove Gaussian noise. The intuition behind PnP/RED is that the proximal (resp. gradientdescent) mapping of g writes as the maximum-a-posteriori (MAP) (resp. posterior mean) estimation from a Gaussian noise observation, under prior p = exp( -g ) . Aremarkable property of these priors is that they are decoupled from the degradation model represented by f in the sense that one learned prior p can serve as a regularizer for many inverse problems. When using deep denoisers corresponding to exact gradient-step [Hurault et al., 2021, Cohen et al., 2021] or proximal [Hurault et al., 2022] maps, RED and PnP methods become real optimization algorithms with known convergence guarantees.

̸

However, for Poisson noise the data-fidelity term (2) is neither smooth with a Lipschitz gradient, nor proximable for A = Id (meaning that its proximal operator cannot be computed in a closed form), which limits the use of standard splitting procedures. Bauschke et al. [2017] addressed this issue by introducing a Proximal Gradient Descent (PGD) algorithm in the Bregman divergence paradigm, called Bregman Proximal Gradient (BPG). The benefit of BPG is that the smoothness condition on f for sufficient decrease of PGD is replaced by the "NoLip" condition " Lh -f is convex" for a convex potential h . For instance, Bauschke et al. [2017] show that the data-fidelity term (2) satisfies the NoLip condition for the Burg's entropy h ( x ) = -∑ n i =1 log( x i ) .

Our primary goal is to exploit the BPG algorithm for minimizing (1) using PnP and RED priors. This requires an interpretation of the plug-in denoiser as a Bregman proximal operator.

where D h ( x, y ) is the Bregman divergence associated with h . In Section 3, we show that by selecting a suitable noise model, other than the traditional Gaussian one, the MAP denoiser can be expressed as a Bregman proximal operator. Remarkably, the corresponding noise distribution belongs to an exponential family, which allows for a closed-form posterior mean (MMSE) denoiser generalizing the Tweedie's formula. Using this interpretation, we derive a RED prior tailored to the Bregman geometry. By presenting a prior compatible with the noise model, we highlight the limitation of the decoupling between prior and data-fidelity suggested in the existing PnP literature.

In order to safely use our MAP and MMSE denoisers in the BPG method, we introduce the Bregman Score denoiser, which generalizes the Gradient Step denoiser from [Hurault et al., 2021, Cohen et al., 2021]. Our denoiser provides an approximation of the log prior of the noisy distribution of images. Moreover, based on the characterization of the Bregman Proximal operator from Gribonval and Nikolova [2020], we show under mild conditions that our denoiser can be expressed as the Bregman proximal operator of an explicit nonconvex potential.

In Section 4, we use the Bregman Score Denoiser within RED and PnP methods and propose the B-RED and B-PnP algorithms. Elaborating on the results from Bolte et al. [2018] for the BPG algorithm in the nonconvex setting, we demonstrate that RED-BPG and PnP-BPG are guaranteed to converge towards stationary points of an explicit functional. We finally show in Section 5 the relevance of the proposed framework in the context of Poisson inverse problems.

2 Related Works

Poisson Inverse Problems A variety of methods have been proposed to solve Poisson image restoration from the Bayesian variational formulation (1) with Poisson data-fidelity term (2). Although

̸

f is convex, this is a challenging optimization problem as f is non-smooth and does not have a closedform proximal operator for A = Id , thus precluding the direct application of splitting algorithms such as Proximal Gradient Descent or ADMM. Moreover, we wish to regularize (1) with an explicit denoising prior, which is generally nonconvex. PIDAL [Figueiredo and Bioucas-Dias, 2009, 2010] and related methods [Ng et al., 2010, Setzer et al., 2010] solve (2) using modified versions of the alternating direction method of multipliers (ADMM). As f is proximable when A = Id , the idea is to add a supplementary constraint in the minimization problem and to adopt an augmented Lagrangian framework. Figueiredo and Bioucas-Dias [2010] prove the convergence of their algorithm with convex regularization. However, no convergence is established for nonconvex regularization. Boulanger et al. [2018] adopt the primal-dual PDHG algorithm which also splits A from the f update. Once again, there is not convergence guarantee for the primal-dual algorithm with nonconvex regularization.

Plug-and-Play (PnP) PnP methods were successfully used for solving a variety of IR tasks by including deep denoisers into different optimization algorithms, including Gradient Descent [Romano et al., 2017], Half-Quadratic-Splitting [Zhang et al., 2017, 2021], ADMM [Ryu et al., 2019, Sun et al., 2021], and PGD [Kamilov et al., 2017, Terris et al., 2020]. Variations of PnP have been proposed to solve Poisson inverse problems. Rond et al. [2016] use PnP-ADMM and approximates, at each iteration, the non-tractable proximal operator of the data-fidelity term (2) with an inner minimization procedure. Sanghvi et al. [2022] propose a PnP version of the PIDAL algorithm which is also unrolled for image deconvolution. Theoretical convergence of PnP algorithms with deep denoisers has recently been addressed by a variety of studies [Ryu et al., 2019, Sun et al., 2021, Terris et al., 2020] (see also a review in Kamilov et al. [2023]). Most of these works require non-realistic or sub-optimal constraints on the deep denoiser, such as nonexpansiveness. More recently, convergence was addressed by making PnP genuine optimization algorithms again. This is done by building deep denoisers as exact gradient descent operators ( Gradient-Step denoiser ) [Cohen et al., 2021, Hurault et al., 2021] or exact proximal operators [Hurault et al., 2022, 2023]. These PnP algorithms thus minimize an explicit functional (2) with an explicit (nonconvex) deep regularization.

Bregman optimization Bauschke et al. [2017] replace the smoothness condition of PGD by the NoLip assumption (4) with a Bregman generalization of PGD called Bregman Proximal Gradient (BPG). In the nonconvex setting, Bolte et al. [2018] prove global convergence of the algorithm to a critical point of (1). The analysis however requires assumptions that are not verified by the Poisson data-fidelity term and Burg's entropy Bregman potential. Al-Shabili et al. [2022] considered unrolled Bregman PnP and RED algorithms, but without any theoretical convergence analysis. Moreover, the interaction between the data fidelity, the Bregman potential, and the denoiser was not explored.

3 Bregman denoising prior

The overall objective of this work is to efficiently solve ill-posed image restoration (IR) problems involving a data-fidelity term f verifying the NoLip assumption for some convex potential h

PnP provides an elegant framework for solving ill-posed inverse problems with a denoising prior. However, the intuition and efficiency of PnP methods inherit from the fact that Gaussian noise is well suited for the Euclidean L 2 distance, the latter naturally arising in the MAP formulation of the Gaussian denoising problem. When the Euclidean distance is replaced by a more general Bregman divergence, the noise model needs to be adapted accordingly for the prior.

In Section 3.1, we first discuss the choice of the noise model associated to a Bregman divergence, leading to Bregman formulations of the MAP and MMSE estimators. Then we introduce in Section 3.2 the Bregman Score Denoiser that will be used to regularize the inverse problem (1).

Notations For convenience, we assume throughout our analysis that the convex potential h : C h ⊆ R n → R ∪ { + ∞} is C 2 and of Legendre type (definition in Appendix A.1). Its convex conjugate h ∗ is then also C 2 of Legendre type. D h ( x, y ) denotes its associated Bregman divergence

3.1 Bregman noise model

We consider the following observation noise model, referred to as Bregman noise 1 ,

We assume that there is γ > 0 and a normalizing function ρ : dom( h ) → R such that the expression (6) defines a probability measure. For instance, for h ( x ) = 1 2 || x || 2 , γ = 1 σ 2 and ρ = 0 , we retrieve the Gaussian noise model with variance σ 2 . As shown in Section 5, for h given by Burg's entropy, p ( y | x ) corresponds to a multivariate Inverse Gamma ( IG ) distribution.

Given a noisy observation y ∈ int dom( h ) , i.e. a realization of a random variable Y with conditional probability p ( y | x ) , we now consider two optimal estimators of x , the MAP and the posterior mean.

Maximum-A-Posteriori (MAP) estimator The MAP denoiser selects the mode of the a-posteriori probability distribution p ( x | y ) . Given the prior p X , it writes

Under the Bregman noise model (6), the MAP denoiser writes as the Bregman proximal operator (see relation (3)) of -1 γ (log p X + ρ ) . This acknowledges for the fact that the introduced Bregman noise is the adequate noise model for generalizing PnP methods within the Bregman framework.

Posterior mean (MMSE) estimator The MMSE denoiser is the expected value of the posterior probability distribution and the optimal Bayes estimator for the L 2 score. Note that our Bregman noise conditional probability (6) belongs to the regular exponential family of distributions

̸

with T ( y ) = γ ∇ h ( y ) , ψ ( x ) = γh ( x ) -ρ ( x ) and p 0 ( y ) = exp ( γh ( y ) -γ ⟨∇ h ( y ) , y ⟩ ) . It is shown in [Efron, 2011] (for T = Id and generalized in [Kim and Ye, 2021] for T = Id ) that the corresponding posterior mean estimator verifies a generalized Tweedie formula ∇ T ( y ) . ˆ x MMSE ( y ) = -∇ log p 0 ( y ) + ∇ log p Y ( y ) , which translates to (see Appendix B for details)

Note that for the Gaussian noise model, we have h ( x ) = 1 2 || x || 2 , γ = 1 /σ 2 and (9) falls back to the more classical Tweedie formula of the Gaussian posterior mean denoiser ˆ x = y -σ 2 ∇ ( -log p Y )( y ) .

Therefore, given an off-the-shelf "Bregman denoiser" B γ specially devised to remove Bregman noise (6) of level γ , if the denoiser approximates the posterior mean B γ ( y ) ≈ ˆ x MMSE ( y ) , then it provides an approximation of the score -∇ log p Y ( y ) ≈ γ ∇ 2 h ( y ) . ( y -B γ ( y )) .

3.2 Bregman Score Denoiser

Based on previous observations, we propose to define a denoiser following the form of the MMSE (9)

with g γ : R n → R a nonconvex potential parametrized by a neural network. Such a denoiser corresponds to the Bregman generalization of the Gaussian Noise Gradient-Step denoiser proposed in [Hurault et al., 2021, Cohen et al., 2021].

When B γ is trained as a denoiser for the associated Bregman noise (6) with L 2 loss, it approximates the optimal estimator for the L 2 score, precisely the MMSE (9). Comparing (10) and (9), we get ∇ g γ ≈ -1 γ ∇ log p Y , i.e. the score is properly approximated with an explicit conservative vector field. We refer to this denoiser as the Bregman Score denoiser .

Is the Bregman Score Denoiser a Bregman proximal operator? We showed in relation (7) that the optimal Bregman MAP denoiser ˆ x MAP is a Bregman proximal operator. We want to generalize this

property to our Bregman denoiser (10). When trained with L 2 loss, the denoiser should approximate the MMSE rather than the MAP. For Gaussian noise, Gribonval [2011] re-conciliates the two views by showing that the Gaussian MMSE denoiser actually writes as an Euclidean proximal operator.

Similarly, extending the characterization from [Gribonval and Nikolova, 2020] of Bregman proximal operators, we now prove that, under some convexity conditions, the proposed Bregman Score Denoiser (10) explicitly writes as the Bregman proximal operator of a nonconvex potential.

Proposition 1 (Proof in Appendix C) . Let h be C 2 and of Legendre type. Let g γ : R n → R ∪ { + ∞} proper and differentiable and B γ ( y ) : int dom( h ) → R n defined in (10) . Assume Im( B γ ) ⊂ int dom( h ) . With ψ γ : R n → R ∪ { + ∞} defined by

suppose that ψ γ ◦ ∇ h ∗ is convex on int dom( h ∗ ) . Then for ϕ γ : R n → R ∪ { + ∞} defined by

we have that for each y ∈ int dom( h )

Remark 1. Note that the order in the Bregman divergence is important in order to fit the definition of the Bregman proximal operator (3) . In this order, [Gribonval and Nikolova, 2020, Theorem 3] does not directly apply. We propose instead in Appendix C a new version of their main theorem.

Remark 2. As shown in Appendix C (equation (64) ), B γ ( y ) can be written as B γ ( y ) = ∇ ( ψ γ ◦ ∇ h ∗ ) ◦ ∇ h ( y ) so that the Jacobian J B γ ( y ) = ∇ 2 h ( y ) . ∇ 2 ( ψ γ ◦ ∇ h ∗ )( ∇ h ( y )) . The hypothesis of convexity of ψ γ ◦ ∇ h ∗ on int dom( h ∗ ) is thus equivalent to the fact that the Jacobian of B γ is positive semi-definite on int dom( h ) .

This proposition generalizes the result of [Hurault et al., 2022, Prop. 1] to any Bregman geometry, which proves that the Gradient-Step Gaussian denoiser writes as an Euclidean proximal operator when ψ γ ◦ ∇ h ∗ ( x ) = 1 2 || x || 2 -g γ ( x ) is convex. More generally, as exhibited for Poisson inverse problems in Section 5, such a convexity condition translates to a constraint on the deep potential g γ .

To conclude, the Bregman Score Denoiser provides, via exact gradient or proximal mapping, two distinct explicit nonconvex priors g γ and ϕ γ that can be used for subsequent PnP image restoration.

4 Plug-and-Play (PnP) image restoration with Bregman Score Denoiser

We now regularize inverse problems with the explicit prior provided by the Bregman Score Denoiser (10). Properties of the Bregman Proximal Gradient (BPG) algorithm are recalled in Section 4.1. We show the convergence of our two B-RED and B-PnP algorithms in Sections 4.2 and 4.3.

4.1 Bregman Proximal Gradient (BPG) algorithm

Let F and R be two proper and lower semi-continuous functions with F of class C 1 on int dom( h ) . Bauschke et al. [2017] propose to minimize Ψ = F + R using the following BPG algorithm:

Recalling the general expression of proximal operators defined in relation (3), when ∇ h ( x k ) -τ ∇ F ( x k ) ∈ dom( h ∗ ) , the previous iteration can be written as (see Appendix D)

With formulation (15), the BPG algorithm generalizes the Proximal Gradient Descent (PGD) algorithm in a different geometry defined by h .

Convergence of BPG If F verifies the NoLip condition (4) for some L F > 0 and if τ < 1 L F , one can prove that the objective function Ψ decreases along the iterates (15). Global convergence of the

iterates for nonconvex F and R is also shown in [Bolte et al., 2018]. However, Bolte et al. [2018] take assumptions on F and h that are not satisfied in the context of Poisson inverse problems. For instance, h is assumed strongly convex on the full domain R n which is not satisfied by Burg's entropy. Additionally, F is assumed to have Lipschitz-gradient on bounded subset of R n , which is not true for F the Poisson data-fidelity term (2). Following the same structure of their proof, we extend in Appendix D.1 (Proposition 3 and Theorem 3) the convergence theory from [Bolte et al., 2018] with the more general Assumption 1 below, which is verified for Poisson inverse problems (Appendix E.4).

Application to the IR problem In what follows, we consider two variants of BPG for minimizing (1) with gradient updates on the data-fidelity term f . These algorithms respectively correspond to Bregman generalizations of the RED Gradient-Descent (RED-GD) [Romano et al., 2017] and the Plug-and-Play PGD algorithms. For the rest of this section, we consider the following assumptions

Assumption 1.

(i) h : C h → R ∪ { + ∞} is of class C 2 and of Legendre-type.

(ii) f : R n → R ∪ { + ∞} is proper, lower-bounded, coercive, of class C 1 on int dom( h ) , with dom( h ) ⊂ dom( f ) , and is subanalytic.

(iii) NoLip : L f h -f is convex on int dom( h ) .

(iv) h is assumed strongly convex on any bounded convex subset of its domain and for all α > 0 , ∇ h and ∇ f are Lipschitz continuous on { x ∈ dom( h ) , Ψ( x ) ≤ α } .

(v) g γ given by the Bregman Score Denoiser (10) and its associated ϕ γ obtained from Proposition 1 are lower-bounded and subanalytic.

Even though the Poisson data-fidelity term (2) is convex, our convergence results also hold for more general nonconvex data-fidelity terms. Assumption (iv) generalizes [Bolte et al., 2018, Assumption D] and allows to prove global convergence of the iterates. The subanalytic assumption, defined in Appendix A.3, is a sufficient condition for the Kurdyka-Lojasiewicz (KL) property to be verified [Bolte et al., 2007]. The latter, also defined in Appendix A.3, can be interpreted as the fact that, up to a reparameterization, the function is sharp. The KL property is widely used in nonconvex optimization [Attouch et al., 2013, Ochs et al., 2014, Bolte et al., 2018]. As detailed in Appendix A.3, the subanalytic assumption of f , g γ and ϕ γ allows for the Kurdyka-Lojasiewicz (KL) property to be verified by the objective functions f + g γ and f + ϕ γ .

4.2 Bregman Regularization-by-Denoising (B-RED)

We first generalize the RED Gradient-Descent (RED-GD) algorithm [Romano et al., 2017] in the Bregman framework. Classically, RED-GD is a simple gradient-descent algorithm applied to the functional λf + g γ where the gradient ∇ g γ is assumed to be implicitly given by an image denoiser B γ (parametrized by γ ) via ∇ g γ = Id -B γ . Instead, our Bregman Score Denoiser (10) provides an explicit regularizing potential g γ whose gradient approximates the score via the Tweedie formula (9). We propose to minimize F λ,γ = λf + g γ on dom( h ) using the Bregman Gradient Descent algorithm

which writes in a more general version as the BPG algorithm (14) with R = 0

As detailed in Appendix E.4, in the context of Poisson inverse problems, for h being Burg's entropy, F λ,γ = λf + g γ verifies the NoLip condition only on bounded convex subsets of dom ( h ) . Thus we select C a non-empty closed bounded convex subset of dom( h ) . For the algorithm (17) to be well-posed and to verify a sufficient decrease of ( F λ,γ ( x k )) , the iterates need to verify x k ∈ C . We propose to modify (17) as the Bregman version of Projected Gradient Descent, which corresponds to the BPG algorithm (14) with R = i C , the characteristic function of the set C :

For general convergence of B-RED, we need the following assumptions

Assumption 2.

(i) R = i C , with C a non-empty closed, bounded, convex and semi-algebraic subset of dom( h ) such that C ∩ int dom( h ) = ∅ .

̸

(ii) g γ has Lipschitz continuous gradient and there is L γ > 0 such that L γ h -g γ is convex on C ∩ int dom( h ) .

In [Bolte et al., 2018, Bauschke et al., 2017], the NoLip constant L needs to be known to set the stepsize of the BPG algorithm as τL < 1 . In practice, the NoLip constant which depends on f , g γ and C is either unknown or over-estimated. In order to avoid small stepsize, we adapt the backtracking strategy of [Beck, 2017, Chapter 10] to automatically adjust the stepsize while keeping convergence guarantees. Given γ ∈ (0 , 1) , η ∈ [0 , 1) and an initial stepsize τ 0 > 0 , the following backtracking update rule on τ is applied at each iteration k :

Using the general nonconvex convergence analysis of BPG realized in Appendix D.1, we can show sufficient decrease of the objective and convergence of the iterates of B-RED.

Theorem 1 (Proof in Appendix D.2) . Under Assumption 1 and Assumption 2, the iterates ( x k ) given by the B-RED algorithm (18) with the backtracking procedure (19) decrease F λ,γ and converge to a critical point of Ψ = i C + F λ,γ with rate min 0 ≤ k ≤ K D h ( x k +1 , x k ) = O (1 /K ) .

4.3 Bregman Plug-and-Play (B-PnP)

We now consider the equivalent of PnP Proximal Gradient Descent algorithm in the Bregman framework. Given a denoiser B γ with Im( B γ ) ⊂ dom( h ) and λ > 0 such that Im( ∇ h -λ ∇ f ) ⊆ dom( ∇ h ∗ ) , it writes

We use again as B γ the Bregman Score Denoiser (10). With ψ γ defined from g γ as in (11) and assuming that ψ γ ◦ ∇ h ∗ is convex on int dom( h ∗ ) , Proposition 1 states that the Bregman Score denoiser B γ is the Bregman proximal operator of some nonconvex potential ϕ γ verifying (12). The algorithm B-PnP (20) then becomes x k +1 ∈ Prox h ϕ γ ◦∇ h ∗ ( ∇ h -λ ∇ f )( x k ) , which writes as a Bregman Proximal Gradient algorithm, with stepsize τ = 1 ,

With Proposition 1, we have B γ ( y ) ∈ Prox h ϕ γ ( y ) i.e. a proximal step on ϕ γ with stepsize 1 . We are thus forced to keep a fixed stepsize τ = 1 in the BPG algorithm (21) and no backtracking is possible. Using Appendix D.1, we can show that B-PnP converges towards a stationary point of λf + ϕ γ .

Theorem 2 (Proof in Appendix D.3) . Assume Assumption 1 and ψ γ ◦ ∇ h ∗ strictly convex on int dom( h ∗ ) . Then for Im( ∇ h -λ ∇ f ) ⊆ dom( ∇ h ∗ ) , Im( B γ ) ⊆ dom( h ) and λL f < 1 (with L f specified in Assumption 1), the iterates x k given by the B-PnP algorithm (20) decrease λf + ϕ γ and converge to a critical point of λf + ϕ γ with rate min 0 ≤ k ≤ K D h ( x k +1 , x k ) = O (1 /K ) .

Remark 3. The condition Im( B γ ) ⊆ dom( h ) and the required convexity of ψ γ ◦ ∇ h ∗ come from Proposition 1 while the condition Im( ∇ h -λ ∇ f ) ⊆ dom( ∇ h ∗ ) allows the algorithm B-PnP (20) to be well-posed. These assumptions will be discussed with more details in the context of Poisson image restoration in Section 5.

5 Application to Poisson inverse problems

We consider ill-posed inverse problems involving the Poisson data-fidelity term f introduced in (2). The Euclidean geometry ( i.e. h ( x ) = 1 2 || x || 2 ) does not suit for such f , as it does not have a Lipschitz gradient. In [Bauschke et al., 2017, Lemma 7], it is shown that an adequate Bregman potential h (in the sense that there exists L f such that L f h -f is convex) for (2) is the Burg's entropy

for which dom( h ) = R n ++ and L f h -f is convex on int dom( h ) = R n ++ for L f ≥ || y || 1 . For further computation, note that the Burg's entropy (22) satisfies ∇ h ( x ) = ∇ h ∗ ( x ) = -1 x and ∇ 2 h ( x ) = 1 x 2 . The Bregman score denoiser associated to the Burg's entropy is presented in Section 5.1. The corresponding Bregman RED and PnP algorithms are applied to Poisson Image deblurring in Section 5.2.

5.1 Bregman Score Denoiser with Burg's entropy

We now specify the study of Section 3 to the case of the Burg's entropy (22). In this case, the Bregman noise model (6) writes (see Appendix E.1 for detailed calculus)

For γ > 1 , this is a product of Inverse Gamma ( IG ( α, β ) ) distributions with parameter β i = γx i and α i = γ -1 . This noise model has mean (for γ > 2 ) γ γ -2 x and variance (for γ > 3 ) γ 2 ( γ -2) 2 ( γ -3) x 2 . In particular, for large γ , the noise becomes centered on x with signal-dependent variance x 2 /γ . Furthermore, using Burg's entropy (22), the optimal posterior mean (9) and the Bregman Score Denoiser (10) respectively write, for y ∈ R n ++ ,

Denoising in practice As Hurault et al. [2021], we chose to parametrize the deep potential g γ as

where N σ is the deep convolutional neural network architecture DRUNet [Zhang et al., 2021] that contains Softplus activations and takes the noise level γ as input. ∇ g γ is computed with automatic differentiation. We train B γ to denoise images corrupted with random Inverse Gamma noise of level γ , sampled from clean images via p ( y | x ) = ∏ n i =1 IG ( α i , β i )( x i ) . To sample y i ∼ IG ( α i , β i ) , we sample z i ∼ G ( α i , β i ) and take y i = 1 /z i . Denoting as p the distribution of a database of clean images, training is performed with the L 2 loss

Denoising performance We evaluate the performance of the proposed Bregman Score DRUNet (B-DRUNet) denoiser (26). It is trained with the loss (27), with 1 /γ uniformly sampled in (0 , 0 . 1) . More details on the architecture and the training can be found in Appendix E.2. We compare the performance of B-DRUNet (26) with the same network DRUNet directly trained to denoise inverse Gamma noise with L 2 loss. Qualitative and quantitative results presented in Figure 1 and Table 1 show that the Bregman Score Denoiser (B-DRUNet), although constrained to be written as (25) with a conservative vector field ∇ g γ , performs on par with the unconstrained denoiser (DRUNet).

Table 1: Average denoising PSNR performance of Inverse Gamma noise denoisers B-DRUNet and DRUNet on 256 × 256 center-cropped images from the CBSD68 dataset, for various noise levels γ .

Bregman Proximal Operator Considering the Burg's entropy (22) in Proposition 1 we get that, if η γ : x → ψ γ ◦∇ h ∗ ( x ) = ψ γ ( -1 x ) is convex on dom( h ∗ ) = R --, the Bregman Score Denoiser (25) satisfies B γ ( y ) = Prox h ϕ γ , for the nonconvex potential ψ γ ( y ) = ∑ n i =1 log( y i ) -g γ ( y ) -1 . The convexity of ψ γ ◦ ∇ h ∗ can be verified using the following characterization, which translates to a condition on the deep potential g γ (see Appendix E.2 for details)

(a) Clean

(b) Noisy ( 17 . 85 dB)

(c) DRUNet ( 32 . 56 dB)

(d) B-DRUNet ( 32 . 56 dB)

It is difficult to constrain g γ to satisfy the above condition. In Appendix E.2, we proceed to a detailed experimental validation of this assumption. We empirically observe on images that the trained deep potential g γ satisfies the above inequality. This suggests that the convexity condition for Proposition 1 is true at least locally, on the image manifold. Indeed, we prove in Appendix E.3 that the convexity condition holds for the theoretical MMSE denoiser (24). As explained below, our denoiser is trained by minimizing the L 2 cost for which the optimal Bayes estimator is the MMSE. The MMSE is then a theoretical denoiser that our denoiser tries to approximate. This explains why our denoiser, after training, naturally satisfies this condition without necessitating supplementary constraints.

5.2 Bregman Plug-and-Play for Poisson Image Deblurring

We now derive the explicit B-RED and B-PnP algorithms in the context of Poisson image restoration. Choosing C = [0 , R ] n for some R > 0 , the B-RED and B-PnP algorithms (18) and (20) write

Verification of the assumptions of Theorems 1 and 2 for the convergence of both algorithms is discussed in Appendix E.4.

Poisson deblurring Equipped with the Bregman Score Denoiser, we now investigate the practical performance of the plug-and-play B-RED and B-PnP algorithms for image deblurring with Poisson noise. In this context, the degradation operator A is a convolution with a blur kernel. We verify the efficiency of both algorithms over a variety of blur kernels (real-world camera shake, uniform and Gaussian). The hyper-parameters γ , λ are optimized for each algorithm and for each noise level α by grid search and are given in Appendix E.5. In practice, we first initialize the algorithm with 100 steps with large τ and γ so as to quickly initialize the algorithm closer to the right stationary point.

Note that the constraint λL f < 1 for convergence of the B-PnP algorithm (Theorem 1) may not be respected. The global NoLip constant L f can indeed be locally very lose. As explained in the Appendix (32), we can adopt a backtracking-like strategy on the regularization parameter λ to ensure convergence. Nevertheless, with the proposed default value of λ , this backtracking algorithm was never activated over the variety of blur kernels and noise levels experimented.

Weshow in Figures 2 and 3 that both B-PnP and B-RED algorithms provide good visual reconstruction. Moreover, we observe that, in practice, both algorithms satisfy the sufficient decrease property of the objective function as well as the convergence of the iterates. Additional quantitative performance and comparisons with other Poisson deblurring methods are given in Appendix E.4.

6 Conclusion

In this paper, we derive a complete extension of the plug-and-play framework in the general Bregman paradigm for non-smooth image inverse problems. Given a convex potential h adapted to the geometry

(a) Clean

(c) B-RED (

23

.

01

dB)

(d) B-PnP (

22

.

96

dB)

(e)

(

λf

+

g

γ

)(

x

k

)

B-RED

(f)

(

λf

+

ϕ

γ

)(

x

k

)

B-PnP

(g)

||

x

i

+1

-

x

i

||

2

(a) Clean

(b) Observed (

16

.

21

dB)

(c) B-RED (

24

.

50

dB)

(d) B-PnP (

24

.

51

dB)

(e)

(

λf

+

g

γ

)(

x

k

)

B-RED

(f)

(

λf

+

ϕ

γ

)(

x

k

)

B-PnP

(g)

||

x

i

+1

-

x

i

||

2

+1.2e3

24060

B-RED

0.2

0.4

B-PnP

10-6

B-RED

B-PnP

24040

0.0

0.2

10-7

24020

0.4

24000

-0.6

10-8

0.8

050100 150 200 250300350 400

10-9

23900

B-RED

1182.2

→B-PnP

B-RED

1182.0

10-6

B-PnP

23850

1181.8

23800

1181.6

10-7

1181.4

10-8

23750

1181.2

050100150200250300350 400

050 100 150 200 250 300 350 400

10-9

050100150200250300350 400

of the problem, we propose a new deep denoiser, parametrized by h , which provably writes as the Bregman proximal operator of a nonconvex potential. We argue that this denoiser should be trained on a particular noise model, called Bregman noise, that also depends on h . By plugging this denoiser in the BPG algorithm, we propose two new plug-and-play algorithms, called B-PnP and B-RED, and show that both algorithms converge to stationary points of explicit nonconvex functionals. We apply this framework to Poisson image inverse problem. Experiments on image deblurring illustrate numerically the convergence and the efficiency of the approach.

The central significance of our work stems from its theoretical study but we recognize certain limits within our experimental results. First, when applied to deblurring with Poisson noise, our proposed algorithms do not outperform existing methods in terms of PSNR. Second, while we prove that B-RED is convergent without restriction, the convergence of B-PnP depends on a specific convexity condition. Despite being confirmed with experiments and having robust theoretical foundations, this assumption could potentially be not verified when applied to non-natural images that significantly differ from those in the training dataset. Finally, due to the nonconvex nature of our proposed prior, the practical performance of the algorithms can be sensitive to their initialization.

Acknowledgements This work was funded by the French ministry of research through a CDSN grant of ENS Paris-Saclay. It has also been carried out with financial support from the French Research Agency through the PostProdLEAP and Mistic projects (ANR-19-CE23-0027-01 and ANR19-CE40-005). It has also been supported by the NSF CAREER award under grant CCF-2043134.

References

Abdullah H Al-Shabili, Xiaojian Xu, Ivan Selesnick, and Ulugbek S Kamilov. Bregman plug-and-play priors. In 2022 IEEE International Conference on Image Processing (ICIP) , pages 241-245. IEEE, 2022.

Hédy Attouch, Jérôme Bolte, Patrick Redont, and Antoine Soubeyran. Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the kurdykałojasiewicz inequality. Mathematics of operations research , 35(2):438-457, 2010.

Hedy Attouch, Jérôme Bolte, and Benar Fux Svaiter. Convergence of descent methods for semialgebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized gauss-seidel methods. Mathematical Programming , 137(1-2):91-129, 2013.

Arindam Banerjee, Srujana Merugu, Inderjit S Dhillon, Joydeep Ghosh, and John Lafferty. Clustering with bregman divergences. Journal of machine learning research , 6(10), 2005.

Heinz H Bauschke, Jérôme Bolte, and Marc Teboulle. A descent lemma beyond lipschitz gradient continuity: first-order methods revisited and applications. Mathematics of Operations Research , 42(2):330-348, 2017.

Amir Beck. First-order methods in optimization . SIAM, 2017.

Mario Bertero, Patrizia Boccacci, Gabriele Desiderà, and Giuseppe Vicidomini. Image deblurring with poisson data: from cells to galaxies. Inverse Problems , 25(12):123006, 2009.

Jérôme Bolte, Aris Daniilidis, and Adrian Lewis. The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM Journal on Optimization , 17(4):1205-1223, 2007.

Jérôme Bolte, Aris Daniilidis, Olivier Ley, and Laurent Mazet. Characterizations of łojasiewicz inequalities: subgradient flows, talweg, convexity. Transactions of the American Mathematical Society , 362(6):3319-3363, 2010.

Jérôme Bolte, Shoham Sabach, Marc Teboulle, and Yakov Vaisbourd. First order methods beyond convexity and lipschitz gradient continuity with applications to quadratic inverse problems. SIAM Journal on Optimization , 28(3):2131-2151, 2018.

Jérôme Boulanger, Nelly Pustelnik, Laurent Condat, Lucie Sengmanivong, and Tristan Piolot. Nonsmooth convex optimization for structured illumination microscopy image reconstruction. Inverse problems , 34(9):095004, 2018.

Regev Cohen, Yochai Blau, Daniel Freedman, and Ehud Rivlin. It has potential: Gradient-driven denoisers for convergent solutions to inverse problems. Advances in Neural Information Processing Systems , 34, 2021.

Patrick L. Combettes and Jean-Christophe Pesquet. Proximal splitting methods in signal processing. In Fixed-Point Algorithms for Inverse Problems in Science and Engineering , pages 185-212. Springer, 2011.

Michel Coste. An introduction to semialgebraic geometry, 2000.

Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Association , 106(496):1602-1614, 2011.

Mario AT Figueiredo and Jose M Bioucas-Dias. Deconvolution of poissonian images using variable splitting and augmented lagrangian optimization. In 2009 IEEE/SP 15th Workshop on Statistical Signal Processing , pages 733-736. IEEE, 2009.

Mário AT Figueiredo and José M Bioucas-Dias. Restoration of poissonian images using alternating direction optimization. IEEE transactions on Image Processing , 19(12):3133-3145, 2010.

Rémi Gribonval. Should penalized least squares regression be interpreted as maximum a posteriori estimation? IEEE Transactions on Signal Processing , 59(5):2405-2410, 2011.

Rémi Gribonval and Mila Nikolova. A characterization of proximity operators. Journal of Mathematical Imaging and Vision , 62(6):773-789, 2020.

Samuel Hurault, Arthur Leclaire, and Nicolas Papadakis. Gradient step denoiser for convergent plug-and-play. arXiv preprint arXiv:2110.03220 , 2021.

Samuel Hurault, Arthur Leclaire, and Nicolas Papadakis. Proximal denoiser for convergent plugand-play optimization with nonconvex regularization. In International Conference on Machine Learning , pages 9483-9505. PMLR, 2022.

Samuel Hurault, Antonin Chambolle, Arthur Leclaire, and Nicolas Papadakis. A relaxed proximal gradient descent algorithm for convergent plug-and-play with proximal denoiser. In Scale Space and Variational Methods in Computer Vision: 9th International Conference, SSVM 2023, Santa Margherita di Pula, Italy, May 21-25, 2023, Proceedings , pages 379-392. Springer, 2023.

S. Kamilov, C. A. Bouman, G. T. Buzzard, and B. Wohlberg. Plug-and-play methods for integrating physical and learned models in computational imaging. IEEE Signal Process. Mag. , 40(1):85-97, January 2023.

Ulugbek S. Kamilov, Hassan Mansour, and Brendt Wohlberg. A plug-and-play priors approach for solving nonlinear imaging inverse problems. IEEE Signal Process. Lett. , 24(12):1872-1876, December 2017.

Kwanyoung Kim and Jong Chul Ye. Noise2score: tweedie's approach to self-supervised image denoising without clean images. Advances in Neural Information Processing Systems , 34:864-874, 2021.

Steven G Krantz and Harold R Parks. A primer of real analytic functions . Springer Science & Business Media, 2002.

Anat Levin, Yair Weiss, Fredo Durand, and William T Freeman. Understanding and evaluating blind deconvolution algorithms. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 1964-1971. IEEE, 2009.

Michael K Ng, Pierre Weiss, and Xiaoming Yuan. Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods. SIAM journal on Scientific Computing , 32(5):2710-2736, 2010.

Peter Ochs, Yunjin Chen, Thomas Brox, and Thomas Pock. ipiano: Inertial proximal algorithm for nonconvex optimization. SIAM Journal on Imaging Sciences , 7(2):1388-1419, 2014.

R Tyrrell Rockafellar. Convex analysis , volume 11. Princeton university press, 1997.

Yaniv Romano, Michael Elad, and Peyman Milanfar. The little engine that could: Regularization by denoising (red). SIAM Journal on Imaging Sciences , 10(4):1804-1844, 2017.

Arie Rond, Raja Giryes, and Michael Elad. Poisson inverse problems by the plug-and-play scheme. Journal of Visual Communication and Image Representation , 41:96-108, 2016.

Ernest Ryu, Jialin Liu, Sicheng Wang, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Plug-andplay methods provably converge with properly trained denoisers. In International Conference on Machine Learning , pages 5546-5557. PMLR, 2019.

Yash Sanghvi, Abhiram Gnanasambandam, and Stanley H Chan. Photon limited non-blind deblurring using algorithm unrolling. IEEE Transactions on Computational Imaging , 8:851-864, 2022.

Simon Setzer, Gabriele Steidl, and Tanja Teuber. Deblurring poissonian images by split bregman techniques. Journal of Visual Communication and Image Representation , 21(3):193-199, 2010.

Masahiro Shiota. Geometry of subanalytic and semialgebraic sets , volume 150. Springer Science & Business Media, 2012.

Yu Sun, Zihui Wu, Xiaojian Xu, Brendt Wohlberg, and Ulugbek S Kamilov. Scalable plug-and-play admm with convergence guarantees. IEEE Transactions on Computational Imaging , 7:849-863, 2021.

Matthieu Terris, Audrey Repetti, Jean-Christophe Pesquet, and Yves Wiaux. Building firmly nonexpansive convolutional neural networks. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 8658-8662. IEEE, 2020.

Singanallur V Venkatakrishnan, Charles A Bouman, and Brendt Wohlberg. Plug-and-play priors for model based reconstruction. In 2013 IEEE Global Conference on Signal and Information Processing , pages 945-948. IEEE, 2013.

Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. Learning deep cnn denoiser prior for image restoration. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3929-3938, 2017.

Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, and Radu Timofte. Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021.

A Definitions

A.1 Legendre functions, Bregman divergence

Definition 1 (Legendre function, Rockafellar [1997]) . Let h : C ⊆ R n → R ∪{ + ∞} be a proper lower semi-continuous convex function. It is called:

(i) essentially smooth, if h is differentiable on int dom( h ) , with moreover ||∇ h ( x k ) ||→∞ for every sequence { x k } k ∈ N of int dom( h ) converging towards a boundary point of dom( h ) .

(ii) of Legendre type if h is essentially smooth and strictly convex on int dom( h ) .

We also recall the following properties of Legendre functions that are used without justification in our analysis (see [Rockafellar, 1997, Section 26] for more details):

h is of Legendre type if and only if its convex conjugate h ∗ is of Legendre type.

For h of Legendre type, dom ( ∇ h ) = int dom( h ) , ∇ h is a bijection from int dom( h ) to int dom( h ∗ ) and ( ∇ h ) -1 = ∇ h ∗ .

Examples We are interested in the two following Legendre functions

L 2 potential : h ( x ) = 1 2 || x || 2 , C = dom( h ) = dom( h ∗ ) = R n , D h ( x, y ) = 1 2 || x -y || 2

Burg's entropy : h ( x ) = -∑ n i =1 log( x i ) , C = dom( h ) = R n ++ , dom ( h ∗ ) = R n --and D h ( x, y ) = ∑ n i =1 x i y i -log ( x i y i ) -1 .

A.2 Nonconvex subdifferential

Following Attouch et al. [2013], we use as notion of subdifferential of a proper, nonconvex function Φ the limiting subdifferential

with ˆ ∂ Φ the Fréchet subdifferential of Φ . We have

A.3 Kurdyka-Lojasiewicz (KL) property and subanalytic functions.

Definition 2 (Kurdyka-Lojasiewicz (KL) [Attouch et al., 2013]) . A function f : R n -→ R ∪ + ∞ is said to have the Kurdyka-Lojasiewicz property at x ∗ ∈ dom ( f ) if there exists η ∈ (0 , + ∞ ) , a neighborhood U of x ∗ and a continuous concave function ψ : [0 , η ) -→ R + such that ψ (0) = 0 , ψ is C 1 on (0 , η ) , ψ ′ > 0 on (0 , η ) and ∀ x ∈ U ∩ [ f ( x ∗ ) < f < f ( x ∗ ) + η ] , the Kurdyka-Lojasiewicz inequality holds:

Proper lower semicontinuous functions which satisfy the Kurdyka-Lojasiewicz inequality at each point of dom ( ∂f ) are called KL functions.

This condition can be interpreted as the fact that, up to a reparameterization, the function is locally sharp around its critical points i.e. we can bound its subgradients away from 0. For more details and interpretations, we refer to [Attouch et al., 2010] and [Bolte et al., 2010]. As shown in Bolte et al. [2007], a large class of functions that have the KL-property is given by continuous subanalytic functions .

Definition 3 (Subanalytic [Bolte et al., 2007]) .

A subset S of R n is a semianalytic set if each point of R n admits a neighborhood V for which there exists a finite number of real analytic functions f i,j , g i,j : R n → R such that

Using we get

As h is strictly convex, ∇ 2 h ( y ) is invertible and

C Proof of Proposition 1

A new characterization of Bregman proximal operators We first derive in Proposition 2 an extension of [Gribonval and Nikolova, 2020, Corollary 5 a)] when h is strictly convex and thus ∇ h is invertible. We will then see that Proposition 1 is a direct application of this result.

Proposition 2. Let h of Legendre type on R n . Let ζ : int dom( h ) → R n . Suppose Im ( ζ ) ⊂ dom( h ) . The following properties are equivalent:

(i) There is ϕ : R n → R ∪ { + ∞} such that Im( ζ ) ⊂ dom( ϕ ) and for each y ∈ int dom( h )

(ii) There is a l.s.c ψ : R n → R ∪ { + ∞} proper convex on int dom( h ∗ ) such that ζ ( ∇ h ∗ ( z )) ∈ ∂ψ ( z ) for each z ∈ int dom( h ∗ ) .

A subset S of R n is a subanalytic set if each point of R n admit a neighborhood V for which

where U is a bounded semianalytic subset of R n × R p with p ≥ 1 .

A function f : R n → R ∪ { + ∞} (resp. f : R n → R m ) is called subanalytic if its graph { ( x, y ) ∈ R n × R , y = f ( x ) } (resp. { ( x, y ) ∈ R n × R m , y = f ( x ) } ) is a subanalytic subset of R n × R (resp. R n × R m ).

Subanalytic functions encompass both analytic functions and semialgebraic [Coste, 2000] functions [Shiota, 2012]. Stability results of subanalytic functions by sum, product, and composition are given in [Shiota, 2012, Section 1.2].

B More details on the Bregman Denoising Prior

We detail here the calculations realized in Section 3.

We first verify that the Bregman noise conditional probability belongs to the regular exponential family of distributions . Indeed for T ( y ) = γ ∇ h ( y ) , ψ ( x ) = γh ( x ) -ρ ( x ) and p 0 ( y ) = exp ( γh ( y ) -γ ⟨∇ h ( y ) , y ⟩ )

which corresponds to the Bregman noise conditional probability introduced in Equation (6). The Tweedie formula of the posterior mean is then [Kim and Ye, 2021]

When (i) holds, ψ can be chosen given ϕ with

When (ii) holds, ϕ can be chosen given ψ with

Remark 4. [Gribonval and Nikolova, 2020, Corollary 5 a)] with Y = int dom( h ) states that (i) is equivalent to

However (ii) and (iii) are not equivalent. We show here that (ii) implies (iii) but the converse is not true. Let ψ convex defined from (ii). Thanks to Legendre-Fenchel identity, we have that

Therefore (ii) implies (iii) with g = ψ ∗ . However, the last lign is just an implication.

Proof. We follow the same order of arguments from [Gribonval and Nikolova, 2020]. We first prove a general result reminiscent to [Gribonval and Nikolova, 2020, Theorem 3] for a general form of divergence function and then apply this result to Bregman divergences.

Lemma 1. Let a : Y ⊆ R n → R ∪ { + ∞} , b : R n → R ∪ { + ∞} , A : Y → Z bijection from Y to Z (with Z ⊂ R n ). Consider ζ : Y → R n . Let D ( x, y ) := a ( y ) -⟨ x, A ( y ) ⟩ + b ( x ) . Suppose Im( ζ ) ⊂ dom( b ) . The following properties are equivalent:

(i) There is ϕ : R n → R ∪ { + ∞} such that Im( ζ ) ⊂ dom( ϕ ) and for each y ∈ Y

(ii) There is a l.s.c ψ : R n → R ∪{ + ∞} proper convex such that ζ ( A -1 ( z )) ∈ ∂ψ ( z ) for each z ∈ Z .

When (i) holds, ψ can be chosen given ϕ with

When (ii) holds, ϕ can be chosen given ψ with

Before proving the Lemma, we can directly see that Proposition 2 is the specialization of Lemma 1 with Bregman divergences. Given h of Legendre-type, the divergence D ( x, y ) = a ( y ) -⟨ x, A ( y ) ⟩ + b ( x ) becomes the Bregman divergence D h ( x, y ) defined in (5) for Y = int dom( h ) , Z = int dom( h ∗ ) , A ( y ) := ∇ h ( y ) , a ( y ) := ⟨∇ h ( y ) , y ⟩ -h ( y ) and b ( x ) := h ( x ) if x ∈ dom ( h ) and + ∞ otherwise. For h of Legendre-type, ∇ h is a bijection from int dom( h ) to int dom( h ∗ ) and ( ∇ h ) -1 = ∇ h ∗ .

Proof. We now prove Lemma 1. We follow the same arguments as the proof of [Gribonval and Nikolova, 2020, Theorem 3(c)]. (i) ⇒ (ii) : Define

As we assume Im( ζ ) ⊂ dom( b ) and Im( ζ ) ⊂ dom( ψ ) , we have ρ : R n → R ∪ { + ∞} and dom( ρ ) = Z . Let z ∈ Z and y = A -1 ( z ) . From (i), ζ ( y ) is the global minimizer of x → D ( x, y ) + ϕ ( x ) or of x →-⟨ x, A ( y ) ⟩ + b ( x ) + ϕ ( x ) and ∀ z ′ ∈ Z , y ′ = A -1 ( z ′ ) ,

By definition of the subdifferential, this shows that

̸

The previous definition is valid because by (ii), Im( A ) = Z ⊂ dom( ∂ψ ) ⊂ dom( ψ ) and therefore ψ ( A ( y )) < + ∞ . By (ii), ∀ z, z ′ ∈ Z ,

Let ˜ ρ the lower convex envelope of ρ (pointwise supremum of all the convex l.s.c functions below ρ ). ˜ ρ is proper convex l.s.c. and ∀ z ∈ Z , ∂ρ ( z ) = ∅ . By [Gribonval and Nikolova, 2020, Proposition 3], ∀ z ∈ Z , ρ ( z ) = ˜ ρ ( z ) and ∂ρ ( z ) = ∂ ˜ ρ ( z ) . Thus for ψ = ˜ ρ , we get (ii).

(ii) ⇒ (i) :

Define η : Y → R by which gives ∀ y, y ′ ∈ Y ,

This yields

We define θ : R n → R ∪ { + ∞} obeying dom( θ ) = Im( ζ ) with

For y, y ′ ∈ ζ -1 ( x ) , as ζ ( y ′ ) = ζ ( y ) , we have by (59) η ( y ′ ) -η ( y ) ≥ 0 and η ( y ) -η ( y ′ ) ≥ 0 and thus η ( y ′ ) = η ( y ) . The definition of θ is thus independent of the choice of y ∈ ζ -1 ( x ) .

For x ′ ∈ Im ( ζ ) , x ′ = ζ ( y ′ ) . Using the previous inequality with η , we get

that is to say, ∀ x ′ ∈ Im ( ζ )

Given the definition of θ , this is also true for x ′ / ∈ Im ( ζ ) .

We set ϕ = θ -b . As Im( ζ ) ⊂ dom( b ) , b ( ζ ( y )) < + ∞ and ϕ : R n → R ∪{ + ∞} . Adding a ( y ) on both sides, we get

As this is true for all y ∈ Y , we get the desired result.

Eventually, Proposition 1 is a direct application of Proposition 2 with ζ = B γ : R n → R n defined on int dom( h ) by

The function B γ verifies ∀ z ∈ int dom( h ∗ ) ,

and ψ = ψ γ ◦ ∇ h ∗ is assumed convex on int dom( h ∗ ) . From Proposition 2, we get that there is ϕ γ : R n → R ∪ { + ∞} such that for each y ∈ int dom( h )

Moreover, for y ∈ int dom( h ) ,

As h is assumed strictly convex on int dom( h ) , for y ∈ int dom( h ) , the Hessian of h , denoted as ∇ 2 h ( y ) is invertible. By differentiating

we get so that

With the definition (11), we directly get

Finally, Proposition 2 also indicates that ϕ γ can be chosen given ψ γ with

(74)

This gives ∀ y ∈ int dom( h ) ,

D The Bregman Proximal Gradient (BPG) algorithm

D.1 Convergence analysis of the nonconvex BPG algorithm

We study in this section the convergence of the BPG algorithm

for minimizing Ψ = F + R with nonconvex functions F and/or R .

For the rest of the section, we take the following general assumptions.

Assumption 3.

(i) h : R n → R is of Legendre-type.

(ii) F : R n → R is proper, C 1 on int dom( h ) , with dom( h ) ⊂ dom( F ) .

̸

(iii) R : R n → R is proper, lower semi-continuous with dom R∩ int dom( h ) = ∅ .

(iv) Ψ = F + R is lower-bounded, coercive and verifies the Kurdyka-Lojasiewicz (KL) property (defined in Appendix A.3).

(v) For x ∈ int dom( h ) , T τ ( x ) is nonempty and included in int dom( h ) .

Note that, since R is nonconvex, the mapping T τ is not in general single-valued.

̸

Assumption (v) is required for the algorithm to be well-posed. As shown in [Bauschke et al., 2017, Bolte et al., 2018], one sufficient condition for T τ ( x ) = ∅ is the supercoercivity of function h + λ R for all λ > 0 , that is lim || x ||→ + ∞ h ( x )+ λ R ( x ) || x || = + ∞ . As T τ ( x ) ⊂ dom( h ) , T τ ( x ) ⊂ int dom( h ) is true when dom( h ) is open (which is the case for Burg's entropy for example).

The convergence of the BPG algorithm in the nonconvex setting is studied by the authors of [Bolte et al., 2018]. Under the main assumption that Lh -F is convex on int dom( h ) , they show first the sufficient decrease property (and thus convergence) of the function values, and second, global convergence of the iterates. However, as we will develop in Appendix D.3, Lh -F is not convex on the full domain of int dom( h ) but only on the compact subset dom( R ) .

One can verify that all the iterates (76) belong to the convex set

where Conv ( E ) stands for the convex envelope of E . We argue that it is enough to assume Lh -F convex on this convex subset with the following assumption.

Assumption 4. There is L > 0 such that, Lh -F is convex on Conv (dom R ) ∩ int dom( h ) .

We can now prove a result similar than [Bolte et al., 2018, Proposition 4.1].

Proposition 3. Under Assumptions 3 and 4, let ( x k ) k ∈ N be a sequence generated by (18) with 0 < τL < 1 . Then the following properties hold

(i) (Ψ( x k )) k ∈ N is non-increasing and converges.

Proof. We adapt here the proof from [Bolte et al., 2018] to the case where Lh -F is not globally convex but only convex on the convex subset Conv (dom R ) ∩ int dom( h ) .

Sufficient decrease property We first show that the sufficient decrease property of Ψ( x k ) holds. This is true because the following characterisation of C 1 convex functions holds on a convex subset. We recall this classical proof for the sake of completeness.

Lemma 2. Let f : X → R n be of class C 1 , then f is locally convex on C a convex subset of X = dom ( f ) if and only if ∀ x, y ∈ C , D f ( x, y ) = f ( x ) -f ( y ) -⟨∇ f ( y ) , x -y ⟩ ≥ 0 .

Proof. ⇒ Let x, y ∈ C , for all t ∈ (0 , 1) , x + t ( y -x ) ∈ C and by convexity of f

i.e.

and

⇐ Let x, y ∈ C , t ∈ (0 , 1) and z = y + t ( x -y ) ∈ C . We have

Combining both equations gives

Therefore, we have Lh -F convex on C , if and only if, ∀ x, y ∈ C , D Lh -F ( x, y ) ≥ 0 , i.e. D F ( x, y ) ≤ LD h ( x, y ) .

The rest of the proof is identical to the one of [Bolte et al., 2018] and we recall it here. Given the optimality conditions in (76), all the iterates x k ∈ C satisfy

using D F ( x, y ) ≤ LD h ( x, y ) , we get

which, together with the fact that Ψ is lower bounded, proves (i). Summing the previous inequality from k = 0 to K -1 gives

Thus ( D h ( x k +1 , x k )) k is summable and converges to 0 when k → + ∞ . Finally

To prove global convergence of the iterates upon the Kurdyka-Lojasiewicz (KL) property, [Bolte et al., 2018, Theorem 4.1] is based on the hypotheses (a) dom( h ) = R n and h is strongly convex on R n and (b) ∇ h and ∇ F are Lipschitz continuous on any bounded subset of R n . These assumptions are clearly not verified for h being the Burg's entropy (22) or F the Poisson data-fidelity term (2). Indeed, in that case, dom( h ) = R n ++ , and h is strongly convex only on bounded sets. Moreover F and h are not Lipschitz continuous near 0 .

However, thanks to the proven decrease of the iterates and as Ψ is assumed coercive, the iterates remain bounded. We can adopt the following weaker assumptions to ensure that the iterates do not tend to + ∞ or 0 .

Assumption 5.

(i) h is strongly convex on any bounded convex subset of its domain.

(ii) For all α > 0 , ∇ h and ∇ F are Lipschitz continuous on { Ψ( x ) ≤ α } .

Under these assumptions, we prove the equivalent of [Bolte et al., 2018, Theorem 4.1].

Theorem 3. Under Assumption 3, 4 and 5, the sequence ( x k ) k ∈ N generated by (18) with 0 < τL < 1 converges to a critical point of Ψ .

We follow the proof given in [Bolte et al., 2018] with some updates to adapt to our weaker Assumption 5. In particular, we show in Lemma 3 that the sequence ( x k ) k ≥ 1 is still "gradient-like" i.e. it verifies the assumptions H1, H2 and H3 from [Attouch et al., 2013]. Once these conditions are verified, the result directly follows from [Bolte et al., 2018, Theorem 6.2].

Let us first note that by coercivity of Ψ and decrease of the iterates Ψ( x k ) (see Proposition 3), the iterates remain in the set

Lemma 3. The sequence ( x k ) k ≥ 1 satisfies the following three conditions:

H1 (Sufficient decrease condition)

H2 (Relative error condition) ∀ k ≥ 1 , there exists ω k +1 ∈ ∂ Ψ( x k +1 ) such that

H3 (Continuity condition) Any subsequence ( x k i ) converging towards x ∗ verifies

Proof. H1 : Sufficient decrease condition . From (85), we get

Besides, h is assumed to be strongly convex on any bounded convex subset of its domain. Furthermore, notice that Conv ( C ( x 0 )) ∩ dom( h ) is a convex subset of dom( h ) as intersection of convex sets. Therefore, there is σ h > 0 such that

With the convention D h ( x, y ) = + ∞ if x / ∈ dom( h ) or y / ∈ int dom( h ) ,

As ∀ k ≥ 1 , x k ∈ C ( x 0 ) , we get

which proves (H1).

H2 : Relative error condition . Given (76), the optimality condition for the update of x k +1 is

For we have

and

By assumption, ∇ F and ∇ h are Lipschitz continuous on C ( x 0 ) = { Ψ( x ) < Ψ( x 0 ) } . As seen before, ∀ k ≥ 1 , x k ∈ C ( x 0 ) . Thus, there is b > 0 such that

H3 : Continuity condition . Let a subsequence ( x k i ) converging towards x ∗ . Using the optimality in the update of x k we have

From (95) and the fact that (Ψ( x k )) k converges, we have that || x k -x k -1 || → 0 . Thus ( x k i -1 ) i also converges towards x ∗ . In addition, since h is continuously differentiable, D h ( x ∗ , x k -1 ) = h ( x ∗ ) -h ( x k -1 ) -⟨∇ h ( x k -1 ) , x ∗ -x k -1 ⟩ → 0 . Passing to the limit in (103), we get

By lower semicontinuity of R and continuity of F , we get the desired result:

Backtracking The convergence actually requires to control the NoLip constant. In order to avoid small stepsizes, we adapt the backtracking strategy of [Beck, 2017, Chapter 10] to the BPG algorithm.

Given γ ∈ (0 , 1) , η ∈ [0 , 1) and an initial stepsize τ 0 > 0 , the following backtracking update rule on τ is applied at each iteration k :

Proposition 4. At each iteration of the algorithm, the backtracking procedure (106) is finite and with backtracking, the convergence results of Proposition 3 and Theorem 3 still hold.

Proof. For a given stepsize τ , we showed in equation (85) that

Taking τ < 1 -γ L , we get 1 τ -L > γ τ so that

Hence, when τ < 1 -γ L , the sufficient decrease condition is satisfied and the backtracking procedure ( τ ←-ητ ) must end.

Replacing the former sufficient decrease condition (107) with (108), the rest of the proofs from Proposition 3 and Theorem 3 are identical.

D.2 Proof of Theorem 1

Proof. We recall the B-RED algorithm

It corresponds to the BPG algorithm (14) with F = F λ,γ = λf + g γ and R = i C .

Theorem 1 is a direct application of Proposition 4 that is to say of the convergence results of Proposition 3 and Theorem 3 with backtracking. Given Assumptions 1 and 2, we verify that Assumptions 3, 4 and 5 are verified:

̸

Assumption 3 . R = i C verifies dom( R ) ∩ int dom( h ) = C ∩ int dom( h ) = ∅ . Moreover, R is semi-algebraic and thus subanalytic as the indicator function of a closed semi-algebraic set. g γ and f

being both assumed subanalytic and lower-bounded, by [Shiota, 2012, I.2.1.9], their sum λf + g σ is then subanalytic (up to adding a constant to make f and g σ non-negative). Ψ = F λ,γ + i C is then also subanalytic, and thus KL. Ψ is also lower-bounded and coercive as f is lower-bounded and coercive and g γ is lower-bounded. Finally, for x ∈ int dom( h ) , T τ ( x ) is non-empty as h + λi C is supercoercive.

Assumption 4 . By summing convex functions, using Assumption 1(iii) and Assumption 2(ii), L = λL f + L γ verifies Lh -( λf + g γ ) convex on Conv (dom R ) ∩ int dom( h ) = C ∩ int dom( h ) .

Assumption 5 . As g γ is assumed to have globally Lipschitz continuous gradient, this follows directly from Assumption 1(iv).

D.3 Proof of Theorem 2

It corresponds to the BPG algorithm (14) with F = λf and R = ϕ γ .

Theorem 2 is a direct application of the convergence results of Proposition 3 and Theorem 3. We now denote Ψ = λf + ψ γ . Given Assumptions 1, we verify that Assumptions 3, 4 and 5 are verified.

̸

Assumption 3 . For R = ϕ γ , we have Im ( B γ ) ⊂ dom( ϕ γ ) and as for y ∈ int dom( h ) , Im ( B γ ) ⊂ int dom( h ) (by equation 13), we get dom( R ) ∩ int dom( h ) = ∅ . ϕ γ and f being both assumed subanalytic and lower-bounded, by [Shiota, 2012, I.2.1.9], their sum λf + ϕ σ is then subanalytic (up to adding a constant to make f and ϕ σ non-negative) and thus KL. Ψ is coercive because f is coercive and ϕ γ is lower-bounded. Finally, T τ ( x ) well-posed is ensured via Proposition 1.

Assumption 4 . This is the L-smad property of f given by Assumption 1.

Assumption 5 . This is directly given by Assumption 1(iv).

E Application to Poisson Inverse Problems

E.1 Burg's entropy Bregman noise model

For h being the Burg's entropy (22), the Bregman noise model (6) writes for x, y ∈ R n ++ as

E.2 Inverse Gamma Bregman denoiser

Derivation of (29) We first derive here the condition for convexity of η γ := ψ γ ◦ ∇ h ∗ introduced in Section 5.1.

We have and

which gives

For Burg's entropy and y = ∇ h ∗ ( x ) = -1 x with x < 0 , this writes

Using and

we get

For η γ to be convex, a necessary condition is that ∀ y ∈ R n ++ and d ∈ R n

Training details For N γ , we use the DRUNet architecture from [Zhang et al., 2021] with 2 residual blocks at each scale and softplus activation functions. We condition the network N γ on γ similarly to what is done for DRUNet. We stack to 3 color channels of the input image an additional channel containing an image with constant pixel value equal to 1 /γ . We use the same training dataset as [Zhang et al., 2021]. Training is performed with ADAM during 1200 epochs. The learning rate is initialized with learning rate 10 -4 and is divided by 2 at epochs 300 , 600 and 900 .

Validation of the convexity hypothesis Given the trained Bregman Score Denoiser B γ = y -y 2 ∇ g γ ( y ) , we propose to verify validity of the hypothesis of convexity from Proposition 1 i.e. the convexity of η γ : x → ψ γ ◦ ∇ h ∗ ( x ) = ψ γ ( -1 x ) on int dom( h ∗ ) = R n --. ψ γ is defined in (11). We showed above that for z ∈ R n --and y = -1 /z , ∀ d ∈ R n ,

In order to verify this assumption around the image manifold, we represent in Figure 4

where the minimum is taken over

{ X i } i : the images from an evaluation dataset of clean images, here the 68 images from CSBD68.

{ D i } : 100 random images realizations of a uniform random variable D i ∼ U [0 , 1] n .

and y ξ is obtained from x by interpolating between ˆ y ξ ∼ p ξ ( y | x ) a noisy version of x and B γ (ˆ y ξ ) with α ∼ U [0 , 1] as

In the figures, we represent c ( γ, ξ ) with respect to:

in the x -axis: the denoiser strength, i.e. the γ parameter of the denoiser.

in the y -axis: the noise level ξ in the input image. Figure 4(a), the noise model is Inverse Gamma noise p ξ ( y | x ) = IG ( ξ -1 , ξx ) i.e. the noise model used for training the denoiser. In figure 4(b), the noise model is Poisson p ξ ( y | x ) = P ( ξx ) i.e. the noise model of the degradation on which our plug-and-play algorithms are evaluated, and in Figure 4(c), the noise model is Gaussian p ξ ( y | x ) = N ( x, ξ 2 Id) .

We observe that for a large variety of noise levels γ and even far away from the image manifold i.e. for large noise, we verify the convexity assumption.

(a) Inverse Gamma noise

p

ξ

(ˆ

y

ξ

|

x

) =

IG

(

ξ

-

1

, ξx

)

.

(b) Poisson noise

p

ξ

(ˆ

y

ξ

|

x

) =

IG

(

ξx

)

.

180

80

1440

160

1360

70

140

1280

60

Enoiselevel

120

noiselevel

1200

100

50

1120

80

40

1040

09

30

40

096

20+

20+

880

20

30

40

50

60

70

80

06

100

20

30

40

50

60

70

80

06

100

ydenoiser

ydenoiser

80

2600

70

2400

2200

60

2000

noise level

50

1800

40

1600

30

1400

20

1200

10

1000

0

800

20

30

40

50

60

70

80

06

100

ydenoiser

(ˆ

p

ξ

y

ξ

|

x

) =

IG

(

x, ξ

2

Id)

.

E.3 The MMSE denoiser verifies the convexity hypothesis from Proposition 1

Lemma 4. For h Burg's entropy, and thus for the noise model (23) , with the MMSE denoiser ˆ x MMSE ( y ) = E [ x | y ] , we have η γ : x → ψ γ ◦ ∇ h ∗ is strictly convex on int dom( h ∗ ) , where ψ γ is defined in (11) from g γ = -1 γ log p Y .

Proof. Recall that the MMSE denoiser corresponds to the Bregman Score Denoiser

in the particular case where we have exactly

Note that in the Euclidean case (i.e. for the L 2 Bregman divergence h = 1 2 || . || 2 and p Y = p σ = p ∗N (0 , σ 2 Id) ), it is shown in Gribonval [2011] to be verified by the MMSE denoiser. We now show the same result for Burg's entropy Bregman potential. For x ∈ R n ++

Recall that, in this case, the Bregman noise model writes

where α ( x ) = exp( ρ ( x ) + nγ ) ∏ n i =1 x γ i > 0 . We verify that ∀ x ∈ int dom( h ∗ ) = R n --

We showed in (119) that, ∀ y ∈ R n ++

Similar to Gribonval [2011], for simplicity, we first write the proof in the single variable case ( n = 1) .

We have ∇ g γ ( y ) = -1 γ p ′ Y ( y ) p Y ( y ) and ∇ 2 g γ ( y ) = 1 γ [ p ′ Y ( y )] 2 -p ′′ Y ( y ) p y ( y ) p 2 Y ( y ) , and thus

where we denote I (1) ( y ) = ∫ z zu ( z, y ) dz .

where we denote I (2) ( y ) = ∫ z z 2 u ( z, y ) dz .

Eventually, we get, after simplification,

̸

As α ( x ) > 0 , the previous term is 0 if and only if p X ( z ) p X ( z ′ ) = 0 when z = z ′ , which would imply p X ( z ) = 0 ∀ z . This is impossible since p X is a proper pdf.

The extension for n > 1 is straightforward and ( z -z ′ ) 2 becomes ⟨ z -z ′ , d ⟩ 2 .

Moreover

E.4 Convergence of B-RED and B-PnP algorithm for Poisson inverse problems

In this section we verify that the Burg's entropy h in (22) and the Poisson data likelihood f defined in (2) verify the assumptions required for convergence of B-RED and B-PnP. We remind the expression of f and h .

̸

for A ∈ R m × n . Note that as done in Bauschke et al. [2017], denoting ( a i ) 1 ≤ i ≤ n the columns of A , we assume that a i = 0 m and ∀ 1 ≤ j ≤ m, ∑ n i =1 a i,j > 0 such that Ax ∈ R m ++ if x ∈ R n ++ . This is verified for A representing the blur with circular boundary conditions with the (normalized) kernels used in Section 5.2.

We first check Assumptions 1 .

(ii) f is real analytic and thus subanalytic on R n ++ .

(iii) It is shown in [Bauschke et al., 2017, Lemma 7] that f verifies the NoLip assumption, i.e. L f h -f is convex on R n ++ , for L f ≥ || y || 1 . y stands for the Poisson degraded observation appearing in the definition of f .

(iv) First, h is strongly convex everywhere on its domain except in + ∞ . For C a bounded subset of R n ++ , as ∇ 2 h ( x ) = 1 x 2 Id , we have ∀ x ∈ C, ∀ d ∈ R n , ⟨∇ 2 h ( x ) d, d ⟩ > 1 sup x ∈ C x 2 || d || 2 indicating that h is strongly convex on bounded subsets of R n ++ . Second, h and f are Lipschitz continuous everywhere on R n ++ except close to 0 . For both B-RED and B-PnP Ψ( x ) → + ∞ when x → 0 and { Ψ( x ) ≤ α } avoids the case x → 0 .

(v) We remind the parameterizations B γ = Id -∇ g γ and g γ ( y ) = 1 2 || x -N γ ( x ) || 2 with a U-Net N γ (with softplus activations). The softplus activation function being real analytic, using the fact that the composition and sum of real analytic functions are real analytic [Krantz and Parks, 2002], N γ and then subsequently g γ are real analytic functions.

For the convergence of B-PnP, we also need to prove that ϕ γ is real analytic (and thus subanalytic) on its domain. For this assumption to be verified, we require in Theorem 2, ψ γ ◦ ∇ h ∗ not only to be convex, but to be strictly convex . Indeed, using the expression (64) B γ ( y ) = ∇ ( ψ γ ◦ ∇ h ∗ ) ◦ ∇ h ( y ) , we have for y ∈ int dom( h )

By bijectivity of ∇ h ∗ between int dom( h ∗ ) and int dom( h ) , we have

Now, using the fact that ∇ 2 h ( y ) is positive definite, it follows

By the inverse function theorem [Krantz and Parks, 2002], as ∀ y ∈ int dom( h ) , J B γ ( y ) > 0 , B -1 γ is then real analytic on Im ( B γ ) . We finally obtain, using the expression (12) of ϕ γ on Im ( B γ ) , again by sum and composition, that ϕ γ is real analytic (and thus subanalytic) on its domain.

Eventually, g γ is non-negative and we now prove that we have ∀ y ∈ R n , ϕ γ ( y ) ≥ g γ ( y ) . If y / ∈ int dom( h ) , as Im( B γ ) ⊂ int dom( h ) , ϕ γ ( y ) = + ∞ , this is verified. If y ∈ dom( h ) ,

as D h ( y, y ) = 0 , we have and

where the inequality comes from (13) and the last equality from (12). Therefore ϕ γ is also lower-bounded.

Second, we verify Assumption 2 required for the convergence of B-RED.

(i) [0 , R ] n is a non-empty closed, bounded, convex and semi-algebraic subset of R n ++ .

(ii) With the parametrization g γ ( y ) = 1 2 || x -N γ ( x ) || 2 with a neural network N γ . g γ can be shown to have Lipschitz gradient (see Hurault et al. [2021, Appendix B] for a proof). We can not show a global NoLip property for g γ . However, as ∇ g γ is Lip ( g γ ) -Lipschitz, we have ∀ x ∈ (0 , R ] n , ∀ d ∈ R n ,

which proves that, for L γ = Lip ( g γ ) R 2 , L γ h -g γ is convex on (0 , R ] n .

B-PnP additional assumptions We finally discuss the additional assumptions required for the convergence of B-PnP in Theorem 2.

Im( B γ ) ⊆ dom( h ) = R n ++ . We train the denoiser B γ to restore images in [ ϵ, 1] n (with ϵ = 10 -3 ), the denoiser is thus softly enforced to have its image in this range. In practice, we empirically verify during the iterations that we always get x k > 0 .

ψ γ ◦ ∇ h ∗ convex on int dom( h ∗ ) . This is discussed in detail in Appendix E.2.

Im( ∇ h -λ ∇ f ) ⊆ dom( ∇ h ∗ ) . We now show that this condition is true if λ || y || 1 < 1 . For x > 0

Thus we need to verify that ∀ 1 ≤ i ≤ n , 1 + λx i ∇ f ( x ) i > 0 . For f Poisson data-fidelity term, using ( Ax ) j = ∑ n k =1 a j,k x k , we have ∀ 1 ≤ i ≤ n ,

We assumed that A has positive entries and ∑ m j =1 a j,i = r i > 0 . Therefore, using 0 ≤ a j,i x i ∑ n k =1 ,a j,k x k < 1 , we get

which is positive if λ || y || 1 < 1 .

The stepsize condition λL f < 1 . Using the NoLip constant proposed in [Bauschke et al., 2017, Lemma 7] L f = || y || 1 , the condition boils down to λ || y || 1 < 1 . The condition λ || y || 1 < 1 is too restrictive in practice, as || y || 1 could get very big, especially for large images. This is due to the fact that the NoLip constant L f ≥ || y || 1 can be largely overestimated. Indeed, in the proof of [Bauschke et al., 2017, Lemma 7] as well as in the proof of the previous point, the upper bound

can be very loose in practice. For B-RED this is not a problem, as we use automatic stepsize backtracking. However, for B-PnP, backtracking is not possible as the stepsize is fixed to τ = 1 . In order to still guarantee the convergence, we adopt the following backtracking-like strategy to adjust the regularization parameter λ :

-Choose an initial value for λ > 0 .

-At each iteration of the B-PnP algorithm, check sufficient decrease of the objective function F = λf + ϕ γ i.e.

If at some iteration, this condition is not satisfied before convergence, we alert the user and restart the algorithm with λ ←-ηλ for some η ∈ (0 , 1) . We also let the user know that, for optimal performance, it might be necessary to adjust the regularization parameter γ of the denoiser, in order to compensate for this decrease of λ .

In our deblurring experiments, for the proposed value of λ , over the variety of blur kernels and noise levels experimented, the sufficient decrease property was always verified and this backtracking algorithm was never activated. This illustrates that || y || 1 is a rough approximation of the NoLip constant.

E.5 Experiments

We give here more details and results on the evaluation of B-PnP and B-RED algorithms for Poisson image deblurring. We present in Figure 5 the four blur kernels used for evaluation. Initialization is done with x 0 = A T y . The algorithm terminates when the relative difference between consecutive values of the objective function is less than 10 -8 or the number of iterations exceeds K = 500 .

(a)

(b)

(c)

(d)

Choice of hyperparameters For B-RED, stepsize backtracking is performed with γ = 0 . 8 and η = 0 . 5 .

When performing plug-and-play image deblurring with our Bregman Score Denoiser trained with Inverse Gamma noise, for the right choice of hyperparameters λ and γ , we may observe the following behavior. The algorithm first converges towards a meaningful solution. After hundreds of iterations, it can converge towards a different stationary point that does not correspond to a visually good reconstruction. We illustrate this behavior Figure 6 where we plot the evolution of the PSNR and of the function values f ( x k ) and g γ ( x k ) along the algorithm.

(a) PSNR

(

x

k

)

(b)

f

(

x

k

)

(c)

g

γ

(

x

k

)

20

B-RED

18

16

14

12

0

1500 3000 4500 6000 7500 900010500

10700

B-RED

10600

10500

10400

10300

0

1500 3000 45006000 7500 9000

24000

B-RED

23500

23000

22500

22000

0

1500 3000 4500 6000 7500 9000

This phenomenon can be mitigated by using small stepsize, large γ and small λ values at the expense of slowing down significantly the algorithm. To circumvent this issue, we propose to first initialize the algorithm with 100 steps with initial τ , γ and λ values and then to change this parameters for

the actual algorithm. Note that it is possible for B-RED to change the stepsize τ but not for B-PnP which has fixed stepsize τ 1 = . For B-PnP, as done in Hurault et al. [2022] in the Euclidean setting, we propose to multiply g θ by a parameter 0 < α < 1 such that the Bregman Score denoiser becomes B α γ ( y ) = y -α ( ∇ 2 h ( y )) -1 . ∇ g γ ( y ) . The convergence of B-PnP with this denoiser follows identically. The overall hyperparameters λ , γ , τ and α for B-PnP and B-RED algorithms for initialization and for the actual algorithm are given in Table 2.

Additional experimental results We provide in Table 3 a quantitative comparison between our 2 algorithms B-RED and B-PnP and 3 other methods. (a) PnP-PGD corresponds to the plug-and-play proximal gradient descent algorithm x k +1 = D σ ◦ (Id -τ ∇ f ) with D σ the DRUNet denoiser (same architecture than B-RED and B-PnP) trained to denoiser Gaussian noise. (b) PnP-BPG corresponds to the B-PnP algorithm x k +1 = D σ ◦∇ h ∗ ( ∇ h -τ ∇ f )( x k ) with again the DRUNet denoiser D σ trained for Gaussian noise. For both (a) and (b) the parameters σ and τ are optimized for each noise level α . (c) ALM Unfolded [Sanghvi et al., 2022] uses the Augmented Lagrangian Method for deriving a 3-operator splitting algorithm that is then trained specifically in an unfolded fashion for image deblurring with a variety of blurs and noise levels α . The publicly available model being trained on grayscale images, for restoring our color images, we treat each color channel independently.

Note that contrary to the proposed B-PnP and B-RED algorithms, the 3 compared methods do not have any convergence guarantees. We observe that our algorithms performs the best when the Poisson noise is not too intense ( α = 40 and α = 60 ) but that PSNR performance decreases for intense noise ( α = 20 ). We assume that this is due to the fact that the denoising prior trained on Inverse Gamma noise is not powerful enough for such a strong noise. A visual example for α = 20 is given Figure 7. As a future direction, we plan on investigating how to increase the regularization capacity of the deep inverse gamma noise denoiser to better handle intense noise.

(a) Clean

(b) Observed (

14

.

91

dB)

(c) B-RED (

22

.

44

dB)

(d) B-PnP (

22

.

40

dB)

口

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nInspired by the No-Lips literature on the optimization of convex objectives which are not globally L-smooth, this paper adds to the PnP literature the “Bregman score denoiser” which extends the BPG algorithm by a Bregman-based prox-map along with convergence conditions despite NN-parametrized non-convex potentials are involved.\n\nSTRENGTHS:\nOriginality:\nThis is a plausible extension of recent related work on PnP networks based on state-of-the-art theory from the field of numerical optimization.\n\nQuality:\nIt is apparant that the authors do not both fields very well.\n\nSignificance: \nThe paper adds a new concept to the PnP literature.\n\nWEAKNESSES:\nClarity: \nThe presentation intersperses references, top-level arguments and technical details in a confusing manner. I had to read few times forth and back in order to get an idea what this paper is about.\n\nAuthors criticize “unrealistic assumptions” (e.g. line 92) in related work but have to admit later on that their own assumptions are hard to check as well, even for a simple scenario (lines 275-277). The need for backtracking line search is not convenient either.\n\nSignificance: \nRegarding the theory, I did not get if some `generic’ properties of the Bregman-prox-maps exist, that play the very same role like, say, the firmly-nonexpansiveness of Euclidean prox-maps, to achieve convergence in the considered generalized scenarios. The concrete scenario (Poisson noise) is classical. A comparison to a related approach shows no improvement (Fig.1, (c), (d)). In particular, artificially corrupting images looks like old-style image denoising papers.\n\nQUESTIONS:\nBefore eq. (14), the convexity assumption regarding F and R, adopted in [Bauschke et al 2017] to which you refer, is missing.\n\nline 92: why is nonexpansiveness considered as an “unrealistic requirement” to ensure convergence pf PnP, in view of your own assumptions which are not easy to check either?\n\nHow is the set C in (18) determined?\n\nLIMITATIONS:\nThis point has not been explicitly addressed in the paper, apparently. Please comment.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies an extension of the Plug-and-Play (PnP) framework for solving inverse imaging problems by considering descent schemes in metrics different from L2: Motivated by the fact that some data fidelity terms such as the Kullback-Leibler divergence allow for an efficient minimization with the Bregman Proximal Gradient (BPG) method (an extension of proximal gradient descent to arbitrary Bregman distances instead of squared L2), the authors develop a parametrization of a learnable denoiser which can be interpreted as a proximal operator (or descent step) of a cost function w.r.t. the underlying Bregman distance that combines well with a particular data fidelity term. Under some additional assumptions, this allows proving the convergence of the resulting Bregman PnP framework. Numerical experiments illustrate that the resulting scheme can successfully solve deblurring problems with Poisson noise.\n\nSTRENGTHS:\nThe paper is technically sound and presents the technical construction of the Bregman PnP approach very well. It is an elegant solution that closes a gap missing in the PnP (and RED) framework with learnable priors. I found it very convincing in terms of its theory and it even derives some general (smaller) missing pieces for convergence beyond the seminal works by Bolte, Bauschke, Teboulle and co-authors. \n\nDespite this strengths section being short in comparison to the weaknesses, I think that the strong theoretical contribution along with an illustration of the implementation outweighs the weaknesses in terms of (benchmark) results, such that I am leaning towards accepting the paper.\n\nWEAKNESSES:\nThe numerical experiments/results are not very convincing from a practical point of view. \n- First, the denoising performance of the dedicated network is not better than that of a plain network (might not be very important).\n- Second, the main paper does not compare to the plain PnP approaches with L2 fidelity. The supplementary material reports tiny differences only (with the proposed approach being slightly worse for large noise and slightly better for small noise)\n- According to line 275, the desired condition of $\\phi_\\gamma \\circ \\nabla h^*$ being convex is not enforced but seems to hold empirically when training the network. Thus, any convergence guarantee is lost.\n- Figure 5 in the supplementary material shows a concerning risk of ending up with a bad result. The mitigation strategy of first running 100 iterations with a first set of parameters and then switching to a different set of parameters represents a significant amount of fine-tuning (possibly exceeding the number of hyperparameters and amount of fine-tuning used for the standard L2 PnP approach), such that even the small improvements in table 3 of the supplementary material need to be viewed with care. \n\nMinor aspects:\n- In 225 the author decide to do backtracking line search to avoid estimating the NoLip constant, but would backtracking on the (differentiable but not L-smooth) data fidelity term considered here not work in the L2 case?\n- The authors mention that there is no result on the convergence of the PDHG method for nonconvex regularizers (line 81). As I was curious, I briefly searched online and found \"Precompact convergence of the nonconvex Primal–Dual Hybrid Gradient algorithm\" by Sun et al., Journal of Computational and Applied Mathematics, 2018. Precompactness of the primal variable (the image) would be easy to ensure if one restricts every value to [0,1]. Is their result applicable? (Honestly, I have not read the paper yet). \n- The condition $\\lambda L_f <1 $ in Theorem 2 seems to limit the amount of data fidelity one can use in order to still have a convergent algorithm - is this a limitation?\n\nQUESTIONS:\nConsidering a difficult (2-stage) optimization with different parameters to avoid bad minimizers as shown in Fig. 5 of the supplement, a lack of strict convergence guarantee as the convexity condition cannot be enforced, and negligible difference to a plain PnP or RED approach in terms of the PSNR, what is the advantage of the proposed method?\n\nLIMITATIONS:\nI don't think there is a potential negative societal impact of this work. In terms of general limitations, I think the authors should be more open about the fact, that the use of the Bregman distance framework did not result in improved results over prior algorithmic schemes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-02"}

comments: SUMMARY:
This paper develops a Bregman Plug and Play image restoration algorithm for solving under-determined inverse problems in the presence of Poisson measurement noise. The framework trains the image denoising algorithms used within PnP iterations (the "Bregman Score Denoiser") to remove noise with an exponential distribution that depends on the distribution of the measurement noise (as opposed to the Gaussian noise used for training previous PnP denoisers). The paper integrates the proposed denoiser into iterative algorithms to form B-PnP and B-RED. Both algorithms effectively deblur images in the presence of Poisson noise and are provably convergent to fixed points.

STRENGTHS:
-Generally well-written

-Well motivated

-I believe technically sound

-Evaluated on several different blur kernels, including those based on camera shake. (Blur kernels were known)

WEAKNESSES:
-Based on table 3 in the appendix, the proposed method does not meaningfully improve performance over existing methods (though it does come with convergence guarantees)

-Comparisons with existing methods are not particularly comprehensive

-In practice, PnP algorithms are quite sensitive to how hyperparameters are chosen and setting them correctly can be a challenge. This algorithm introduces another parameter, gamma.

QUESTIONS:
Typos
Line 90: "see also a review Kamilov et al." --> "see also a review by Kamilov et al."
Line 197: Puting (i) on a separate line would read cleaner

LIMITATIONS:
Not discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06