Original: G RADIENT S TEP D ENOISER FOR CONVERGENT P LUGAND -P LAY
Revised: G RADIENT S TEP D ENOISER FOR CONVERGENT P LUGAND -P LAY

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Plug-and-Play (PnP) methods constitute a class of iterative algorithms for imaging problems where regularization is performed by an off-the-shelf denoiser.
Revised: Plug-and-Play (PnP) methods constitute a class of iterative algorithms for imaging problems where regularization is performed by an off-the-shelf denoiser.

Original: Although PnP methods can lead to tremendous visual performance for various image problems, the few existing convergence guarantees are based on unrealistic (or suboptimal) hypotheses on the denoiser, or limited to strongly convex data-ﬁdelity terms.
Revised: Although PnP methods can lead to tremendous visual performance for various image problems, the few existing convergence guarantees are based on unrealistic (or suboptimal) hypotheses on the denoiser, or limited to strongly convex data-ﬁdelity terms.

Original: We propose a new type of PnP method, based on half-quadratic splitting, for which the denoiser is realized as a gradient descent step on a functional parameterized by a deep neural network.
Revised: We propose a new type of PnP method, based on half-quadratic splitting, for which the denoiser is realized as a gradient descent step on a functional parameterized by a deep neural network.

Original: Exploiting convergence results for proximal gradient descent algorithms in the non-convex setting, we show that the proposed
Revised: Exploiting convergence results for proximal gradient descent algorithms in the nonconvex setting, we show that the proposed

Original: PnP algorithm is a convergent iterative scheme that targets stationary points of an explicit global functional.
Revised: PnP algorithm is a convergent iterative scheme that targets stationary points of an explicit global functional.

Original: Besides, experiments show that it is possible to learn such a deep denoiser while not compromising the performance in comparison to other state-of-the-art deep denoisers used in PnP schemes.
Revised: Besides, experiments show that it is possible to learn such a deep denoiser while not compromising the performance in comparison to other state-of-the-art deep denoisers used in PnP schemes.

Original: We apply our proximal gradient algorithm to various ill-posed inverse problems, e g. deblurring, superresolution and inpainting.
Revised: We apply our proximal gradient algorithm to various ill-posed inverse problems, e g. deblurring, superresolution and inpainting.

Original: For all these applications, numerical results empirically conﬁrm the convergence results.
Revised: For all these applications, numerical results empirically conﬁrm the convergence results.

Original: Experiments also show that this new algorithm reaches state-of-the-art performance, both quantitatively and qualitatively.
Revised: Experiments also show that this new algorithm reaches state-of-the-art performance, both quantitatively and qualitatively.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Image restoration (IR) problems can be formulated as inverse problems of the form
Revised: Image restoration (IR) problems can be formulated as inverse problems of the form

Original: [Equation]
Revised: [Equation]

Original: where f is a term measuring the ﬁdelity to a degraded observation y , and g is a regularization term weighted by a parameter λ ≥ 0 .
Revised: where f is a term measuring the ﬁdelity to a degraded observation y , and g is a regularization term weighted by a parameter λ ≥ 0 .

Original: Generally, the degradation of a clean image ˆ x can be modeled by a linear operation y
Revised: Generally, the degradation of a clean image ˆ x can be modeled by a linear operation y

Original: =
Revised: =

Original: A ˆ x + ξ , where A is a degradation matrix and ξ a white Gaussian noise.
Revised: A ˆ x + ξ , where A is a degradation matrix and ξ a white Gaussian noise.

Original: In this context, the maximum a posteriori (MAP) derivation relates the data-ﬁdelity term to the likelihood f ( x ) = − log p ( y | x ) = 12 σ 2 ||
Revised: In this context, the maximum a posteriori (MAP) derivation relates the data-ﬁdelity term to the likelihood f ( x ) = − log p ( y | x ) = 12 σ 2 ||

Original: Ax − y
Revised: Ax − y

Original: || 2 , while the regularization term is related to the chosen prior.
Revised: || 2 , while the regularization term is related to the chosen prior.

Original: Regularization is crucial since it tackles the ill-posedness of the IR task by bringing a priori knowledge on the solution.
Revised: Regularization is crucial since it tackles the ill-posedness of the IR task by bringing a priori knowledge on the solution.

Original: A lot of research has been dedicated to designing accurate priors g .
Revised: A lot of research has been dedicated to designing accurate priors g .

Original: Among the most classical priors, one can single out total variation (Rudin et al., 1992), wavelet sparsity (Mallat, 2009) or patch-based Gaussian mixtures (Zoran & Weiss, 2011).
Revised: Among the most classical priors, one can single out total variation (Rudin et al., 1992), wavelet sparsity (Mallat, 2009) or patch-based Gaussian mixtures (Zoran & Weiss, 2011).

Original: Designing a relevant prior g is a difﬁcult task and recent approaches rather apply deep learning techniques to directly learn a prior from a database of clean images (Lunz et al., 2018; Prost et al., 2021; Gonz´alez et al., 2021).
Revised: Designing a relevant prior g is a difﬁcult task and recent approaches rather apply deep learning techniques to directly learn a prior from a database of clean images (Lunz et al., 2018; Prost et al., 2021; Gonz´alez et al., 2021).

Original: Generally, the problem (1) does not have a closed-form solution, and an optimization algorithm is required.
Revised: Generally, the problem (1) does not have a closed-form solution, and an optimization algorithm is required.

Original: First-order proximal splitting algorithms (Combettes & Pesquet, 2011) operate individually on f and g via the proximity operator
Revised: First-order proximal splitting algorithms (Combettes & Pesquet, 2011) operate individually on f and g via the proximity operator

Original: [Equation]
Revised: [Equation]

Original: Among them, half-quadratic splitting (HQS) (Geman & Yang, 1995) alternately applies the proximal operators of f and g .
Revised: Among them, half-quadratic splitting (HQS) (Geman & Yang, 1995) alternately applies the proximal operators of f and g .

Original: Proximal methods are particularly useful when either f or g is nonsmooth.
Revised: Proximal methods are particularly useful when either f or g is nonsmooth.

Original: Plug-and-Play (PnP) methods (Venkatakrishnan et al., 2013) build on proximal splitting algorithms by replacing the proximity operator of g with a generic denoiser, e.g. a pretrained deep network.
Revised: Plug-and-Play (PnP) methods (Venkatakrishnan et al., 2013) build on proximal splitting algorithms by replacing the proximity operator of g with a generic denoiser, e.g. a pretrained deep network.

Original: These methods achieve state-of-the-art results (Buzzard et al., 2018; Ahmad et al., 2020; Yuan et al., 2020; Zhang et al., 2021) in various IR problems.
Revised: These methods achieve state-of-the-art results (Buzzard et al., 2018; Ahmad et al., 2020; Yuan et al., 2020; Zhang et al., 2021) in various IR problems.

Original: However, since a generic denoiser cannot generally be expressed as a proximal mapping (Moreau, 1965), convergence results, which stem from the properties of the proximal operator, are difﬁcult to obtain.
Revised: However, since a generic denoiser cannot generally be expressed as a proximal mapping (Moreau, 1965), convergence results, which stem from the properties of the proximal operator, are difﬁcult to obtain.

Original: Moreover, the regularizer g is only made implicit via the denoising operation.
Revised: Moreover, the regularizer g is only made implicit via the denoising operation.

Original: Therefore, PnP algorithms do not seek the minimization of an explicit objective functional which strongly limits their interpretation and numerical control.
Revised: Therefore, PnP algorithms do not seek the minimization of an explicit objective functional which strongly limits their interpretation and numerical control.

Original: In order to keep tractability of a minimization problem, Romano et al.
Revised: In order to keep tractability of a minimization problem, Romano et al.

Original: (2017) proposed, with regularization by denoising (RED), an explicit prior g that exploits a given generic denoiser D in the form g ( x ) = 12 (cid:104) x, x − D ( x ) (cid:105) .
Revised: (2017) proposed, with regularization by denoising (RED), an explicit prior g that exploits a given generic denoiser D in the form g ( x ) = 12 (cid:104) x, x − D ( x ) (cid:105) .

Original: With strong assumptions on the denoiser (in particular a symmetric
Revised: With strong assumptions on the denoiser (in particular a symmetric

Original: Jacobian assumption)
Revised: Jacobian assumption)

Original: , they show that it veriﬁes
Revised: , they show that it veriﬁes

Original: [Equation]
Revised: [Equation]

Original: Such a denoiser is then plugged in gradient-based minimization schemes.
Revised: Such a denoiser is then plugged in gradient-based minimization schemes.

Original: Despite having shown very good results on various image restoration tasks, as later pointed out by Reehorst & Schniter (2018) or Saremi (2019), existing deep denoisers lack Jacobian symmetry.
Revised: Despite having shown very good results on various image restoration tasks, as later pointed out by Reehorst & Schniter (2018) or Saremi (2019), existing deep denoisers lack Jacobian symmetry.

Original: Hence, RED does not minimize an explicit functional and is not guaranteed to converge.
Revised: Hence, RED does not minimize an explicit functional and is not guaranteed to converge.

Original: Contributions.
Revised: Contributions.

Original: In this work, we develop a PnP scheme with novel theoretical convergence guarantees and state-of-the-art IR performance.
Revised: In this work, we develop a PnP scheme with novel theoretical convergence guarantees and state-of-the-art IR performance.

Original: Departing from the PnP-HQS framework, we plug a denoiser that inherently satisﬁes equation (3) without sacriﬁcing the denoising performance.
Revised: Departing from the PnP-HQS framework, we plug a denoiser that inherently satisﬁes equation (3) without sacriﬁcing the denoising performance.

Original: The resulting ﬁxed-point algorithm is guaranteed to converge to a stationary point of an explicit functional.
Revised: The resulting ﬁxed-point algorithm is guaranteed to converge to a stationary point of an explicit functional.

Original: This convergence guarantee does not require strong convexity of the data-ﬁdelity term, thus encompassing ill-posed IR tasks like deblurring, super-resolution or inpainting.
Revised: This convergence guarantee does not require strong convexity of the data-ﬁdelity term, thus encompassing ill-posed IR tasks like deblurring, super-resolution or inpainting.

Original: [Equation]
Revised: [Equation]

Original: ELATED WORKS
Revised: ELATED WORKS

Original: PnP methods have been successfully applied in the literature with various splitting schemes:
Revised: PnP methods have been successfully applied in the literature with various splitting schemes:

Original: HQS (Zhang et al., 2017b; 2021), ADMM (Romano et al., 2017; Ryu et al., 2019), Proximal Gradient Descent (PGD) (Terris et al., 2020).
Revised: HQS (Zhang et al., 2017b; 2021), ADMM (Romano et al., 2017; Ryu et al., 2019), Proximal Gradient Descent (PGD) (Terris et al., 2020).

Original: First used with classical non deep denoisers such as BM3D (Chan et al., 2016) and pseudo-linear denoisers (Nair et al., 2021; Gavaskar et al., 2021), more recent PnP approaches (Meinhardt et al., 2017; Ryu et al., 2019) rely on efﬁcient off-the-shelf deep denoisers such as DnCNN (Zhang et al., 2017a).
Revised: First used with classical non deep denoisers such as BM3D (Chan et al., 2016) and pseudo-linear denoisers (Nair et al., 2021; Gavaskar et al., 2021), more recent PnP approaches (Meinhardt et al., 2017; Ryu et al., 2019) rely on efﬁcient off-theshelf deep denoisers such as DnCNN (Zhang et al., 2017a).

Original: State-of-the-art IR results are currently obtained with denoisers that are speciﬁcally designed to be integrated in PnP schemes, like IRCNN (Zhang et al., 2017b) or DRUNET (Zhang et al., 2021).
Revised: State-of-the-art IR results are currently obtained with denoisers that are speciﬁcally designed to be integrated in PnP schemes, like IRCNN (Zhang et al., 2017b) or DRUNET (Zhang et al., 2021).

Original: Though providing excellent restorations, such schemes are not guaranteed to converge for all kinds of denoisers or IR tasks.
Revised: Though providing excellent restorations, such schemes are not guaranteed to converge for all kinds of denoisers or IR tasks.

Original: Designing convergence proofs for PnP algorithms is an active research topic.
Revised: Designing convergence proofs for PnP algorithms is an active research topic.

Original: Sreehari et al.
Revised: Sreehari et al.

Original: ﬁrst used the proximal theorem of Moreau (Moreau, 1965) to give sufﬁcient conditions for the denoiser to be an explicit proximal map, which are applied to a pseudo-linear denoiser.
Revised: (2016) used the proximal theorem of Moreau (Moreau, 1965) to give sufﬁcient conditions for the denoiser to be an explicit proximal map, which are applied to a pseudo-linear denoiser.

Original: The convergence with pseudo-linear denoisers havethen been extensively studied (Gavaskar & Chaudhury, 2020; Nair et al., 2021; Chan, 2019).
Revised: The convergence with pseudo-linear denoisers have been extensively studied (Gavaskar & Chaudhury, 2020; Nair et al., 2021; Chan, 2019).

Original: However,in the PnP literature, state-of-the-art  results are obtained with deep denoisers.
Revised: However, state-of-the-art PnP results are obtained with deep denoisers.

Original: Various assumptions have been proposed to ensure the convergence of the related PnP schemes.
Revised: Various assumptions have been made to ensure the convergence of the related PnP schemes.

Original: With a “bounded denoiser” assumption, Chan et al.
Revised: With a “bounded denoiser” assumption, Chan et al.

Original: ; Gavaskar & Chaudhury (2019)ﬁrst showed convergence of PnP-ADMM with stepsizes decreasing to 0.
Revised: ; Gavaskar & Chaudhury (2019) showed convergence of PnP-ADMM with stepsizes decreasing to 0.

Original: Cohen et al. (2021) considers the wider class of demi-contractive mappings, but this condition is hard to verify in practice. 
Revised: RED (Romano et al., 2017) and RED-PRO (Cohen et al., 2021) respectively consider the classes of denoisers with symmetric Jacobian or demicontractive mappings, but these conditions are either too restrictive or hard to verify in practice. In

Original:  Many works focus onthe Lipschitz properties ofthe PnP operator, in order to rely on Banach ﬁxed-point theorem.
Revised: Appendix A.3, more details are given on RED-based methods. Many works focus on Lipschitz properties of PnP operators.

Original: Depending on the splitting algorithm in use, convergence canthen be obtained by assuming the denoiser averaged (Sun et al., 2019), ﬁrmly nonexpansive (Sun et al., 2021; Terris et al., 2020) or simplya nonexpansive (Reehorst & Schniter, 2018; Liu et al., 2021).
Revised: Depending on the splitting algorithm in use, convergence can be obtained by assuming the denoiser averaged (Sun et al., 2019b), ﬁrmly nonexpansive (Sun et al., 2021; Terris et al., 2020) or simply nonexpansive (Reehorst & Schniter, 2018; Liu et al., 2021).

Original: These settings are unrealistic as deep denoisers do not generally satisfy such properties.
Revised: These settings are unrealistic as deep denoisers do not generally satisfy such properties.

Original: Methods like Ryu et al.
Revised: Ryu et al.

Original: (2019) or Terris et al.
Revised: ; Terris et al.

Original: (2020)thus propose different ways to train deep denoisers with constrained Lipschitz constants in order to ﬁt the technical properties required for convergence.
Revised: (2020) propose different ways to train deep denoisers with constrained Lipschitz constants, in order to ﬁt the technical properties required for convergence.

Original: But imposing hard Lipschitz constraints on the network alters its denoising performance (Bohra et al., 2021;
Revised: But imposing hard Lipschitz constraints on the network alters its denoising performance (Bohra et al., 2021;

Original: Hertrich et al., 2021).
Revised: Hertrich et al., 2021).

Original: Yet, Ryu et al.
Revised: Yet, Ryu et al.

Original: (2019) manages to get a convergent PnP scheme without assuming the nonexpansiveness of D .
Revised: (2019) manages to get a convergent PnP scheme without assuming the nonexpansiveness of D .

Original: This comes at the cost of imposing strong convexity on the data-ﬁdelity term f , which excludes many IR tasks like deblurring, super-resolution and inpainting.
Revised: This comes at the cost of imposing strong convexity on the data-ﬁdelity term f , which excludes many IR tasks like deblurring, super-resolution or inpainting.

Original: As a matter of fact, given the ill-posedness of IR problems, looking for a unique solution via contractive operators is a restrictive assumption.
Revised: Hence, given the ill-posedness of IR problems, looking for a unique solution via contractive operators is a restrictive assumption.

Original: In this work, we do not impose contractiveness, but still obtain convergence results with realistic hypotheses.
Revised: In this work, we do not impose contractiveness, but still obtain convergence results with realistic hypotheses.

Original: One can relate the ideal deep denoiser to the “true” natural image prior p via Tweedie’s Identity.
Revised: One can relate the ideal deep denoiser to the “true” natural image prior p via Tweedie’s Identity.

Original: In (Efron, 2011), it is indeed shown that the Minimum Mean Square Error (MMSE) denoiser D ∗ σ (at noise level σ ) veriﬁes D σ ( x ) = x
Revised: In (Efron, 2011), it is indeed shown that the Minimum Mean Square Error (MMSE) denoiser D ∗ σ (at noise level σ ) veriﬁes D σ ( x ) = x

Original: + σ 2 ∇ x log p σ ( x ) where p σ is the convolution of p with the density of N (0 , σ 2 Id) .
Revised: + σ 2 ∇ x log p σ ( x ) where p σ is the convolution of p with the density of N (0 , σ 2 Id) .

Original: In a recent line of research (Bigdeli et al., 2017; Xu et al., 2020; Laumont et al., 2021; Kadkhodaie & Simoncelli, 2020), this relation is used to plug a denoiser in gradientbased dynamics.
Revised: In a recent line of research (Bigdeli et al., 2017; Xu et al., 2020; Laumont et al., 2021; Kadkhodaie & Simoncelli, 2020), this relation is used to plug a denoiser in gradient-based dynamics.

Original: In practice, the MMSE denoiser cannot be computed explicitly and Tweedie’s Identity does not hold for deep approximations of the MMSE.
Revised: In practice, the MMSE denoiser cannot be computed explicitly and Tweedie’s Identity does not hold for deep approximations of the MMSE.

Original: Finally, let us mention that the form of prior considered here was already suggested in Bigdeli & Zwicker (2017) and Romano et al. (2017, Section 5.2) but not exploited for convergence analysis.
Revised: In order to be as exhaustive as possible, we detailed the addressed limitations of existing PnP methods in Appendix A.1.

Original:  T HE G RADIENT S TEP P LUG - AND -P LAY
Revised:  T HE G RADIENT S TEP P LUG - AND -P LAY

Original: The proposed method is based on the PnP version of half-quadratic-splitting (PnP-HQS) that amounts to replacing the proximity operator of the prior g with an off-the-shelf denoiser D σ .
Revised: The proposed method is based on the PnP version of half-quadratic-splitting (PnP-HQS) that amounts to replacing the proximity operator of the prior g with an off-the-shelf denoiser D σ .

Original: In order to deﬁne a convergent PnP scheme, we ﬁrst set up in Section 3.1 a Gradient Step (GS) denoiser.
Revised: In order to deﬁne a convergent PnP scheme, we ﬁrst set up in Section 3.1 a Gradient Step (GS) denoiser.

Original: We then introduce the Gradient Step PnP  algorithm in Section 3.2.
Revised: We then introduce the Gradient Step PnP (GS-PnP) algorithm in Section 3.2.

Original: 3. G RADIENT S TEP DENOISER
Revised: 3. G RADIENT S TEP DENOISER

Original: We propose to plug a denoising operator D σ that takes the form of a gradient descent step
Revised: We propose to plug a denoising operator D σ that takes the form of a gradient descent step

Original: [Equation]
Revised: [Equation]

Original:  with g σ : R n → R .
Revised: (4) with g σ : R n → R .

Original: Contrary to Romano et al.
Revised: Contrary to Romano et al.

Original: (2017), our denoiser exactly represents a conservative vector ﬁeld.
Revised: (2017), our denoiser exactly represents a conservative vector ﬁeld.

Original: The choice of the parameterization of g σ is fundamental for the denoising performance.
Revised: The choice of the parameterization of g σ is fundamental for the denoising performance.

Original: As already noticed in Salimans & Ho (2021), we experimentally found that directly modeling g σ as a neural network ( e.g. a standard network used for classiﬁcation) leads to poor denoising performance.
Revised: As already noticed in Salimans & Ho (2021), we experimentally found that directly modeling g σ as a neural network ( e.g. a standard network used for classiﬁcation) leads to poor denoising performance.

Original: In order to keep the strength of state-of-the-art unconstrained denoisers, we rather use
Revised: In order to keep the strength of state-of-the-art unconstrained denoisers, we rather use

Original: [Equation]
Revised: [Equation]

Original: where N σ : R n → R n is parameterized by a neural network. The resulting denoising operator is
Revised: which leads to D

Original: [Equation]
Revised: [Equation]

Original: where  J N σ ( x ) refers to the Jacobian of N σ at point x .
Revised: where N σ : R n → R n is parameterized by a neural network and J N σ ( x ) is the Jacobian of N σ at point x .

Revised: As discussed in Appendix A.2, the formulation (5) for g σ has been proposed in (Romano et al., 2017, Section 5.2) and (Bigdeli & Zwicker, 2017) for a distinct but related purpose, and not exploited for convergence analysis.

Original: Let us highlight that, thanks to this deﬁnition of D σ , we can parameterize N σ with any differentiable neural network architecture R n → R n that has proven efﬁcient for image denoising.
Revised: Thanks to our deﬁnition (6) for D σ , we can parameterize N σ with any differentiable neural network architecture R n → R n that has proven efﬁcient for image denoising.

Original: Although the representation power of the denoiser is limited by the particular form (6), we show (see Section 5.1) that such parameterization still yields state-of-the-art denoising results.
Revised: Although the representation power of the denoiser is limited by the particular form (6), we show (see Section 5.1) that such parameterization still yields state-of-the-art denoising results.

Original: We train the denoiser D σ  by minimizing the MSE loss function
Revised: We train the denoiser D σ for Gaussian noise by minimizing the MSE loss function

Original: [Equation]
Revised: [Equation]

Original: Using equation (4), it can be written in terms of g σ : [Equation]
Revised: when written in terms of g σ using equation (4).

Original: Remark 1.
Revised: Remark 1.

Original: By deﬁnition, the optimal solution g ∗ σ ∈ arg min L is related to the MMSE denoiser D ∗ σ , that is, the best non-linear predictor of x given x + ξ σ .
Revised: By deﬁnition, the optimal solution g ∗ σ ∈ arg min L is related to the MMSE denoiser D ∗ σ , that is, the best non-linear predictor of x given x + ξ σ .

Original: Therefore, it satisﬁes Tweedie’s formula and ∇ g ∗ σ = − σ 2 ∇ log p σ (Efron, 2011) i.e. g ∗ σ = − σ 2 log p σ + C , for some C ∈ R .
Revised: Therefore, it satisﬁes Tweedie’s formula and [Equation] (Efron, 2011) i.e. g ∗ σ = − σ 2 log p σ + C , for some C ∈ R .

Original: Hence approximating the MMSE denoiser with a denoiser parameterized as (4) is related to approximating the logarithm of the smoothed image prior p σ with − 1 σ 2 g σ .
Revised: Hence approximating the MMSE denoiser with a denoiser parameterized as (4) is related to approximating the logarithm of the smoothed image prior p σ with − 1 σ 2 g σ .

Original: This relation was used for image generation in the context of “Denoising Score Matching” by Saremi & Hyvarinen (2019); Bigdeli et al.
Revised: This relation was used for image generation in the context of “Denoising Score Matching” by Saremi & Hyvarinen (2019); Bigdeli et al.

Original: 3. A P LUG - AND -P LAY METHOD FOR EXPLICIT MINIMIZATION
Revised: 3. A P LUG - AND -P LAY METHOD FOR EXPLICIT MINIMIZATION

Original: The standard PnP-HQS operator is T PnP-HQS =
Revised: The standard PnP-HQS operator is T PnP-HQS =

Original: D σ ◦ Prox τf , i.e. (Id −∇ g σ ) ◦ Prox τf when using the
Revised: D σ ◦ Prox τf , i.e. (Id −∇ g σ ) ◦ Prox τf when using the

Original: GS denoiser as D σ .
Revised: GS denoiser as D σ .

Original: For convergence analysis, it proves fruitful to ﬁt the proximal gradient descent algorithm by switching the proximal and gradient steps.
Revised: For convergence analysis, we wish to ﬁt the proximal gradient descent (PGD) algorithm.

Original: We also propose to  relax the denoising step with a parameter λ ≥ 0 .
Revised: We thus propose to switch the proximal and gradient steps and to relax the denoising step with a parameter λ ≥ 0 .

Original: Our PnP algorithm with GS denoiser (GS-PnP) then writes x k +

Revised: Our PnP algorithm with GS denoiser (GS-PnP) then writes x k +

Original: =
Revised: =

Original: T τ,λ GS-PnP ( x k ) with T τ,λ GS-PnP =
Revised: T τ,λ GS-PnP ( x k ) with T τ,λ GS-PnP [Equation]

Original: Prox τf ◦

Original: [Equation]
Revised: [Equation]

Original: Under suitable conditions on f and g σ (see Lemma 1 in Appendix A), ﬁxed points of the  operator T τ,λ GS-PnP correspond to critical points of 
Revised: Under suitable conditions on f and g σ (see Lemma 1 in Appendix C), ﬁxed points of the PGD operator T τ,λ GS-PnP correspond to critical points of a classical objective function in IR problems

Original: [Equation]
Revised: [Equation]

Original: Therefore, using the GS denoiser from equation (4) is equivalent to include an explicit regularization and thus leads to a tractable global optimization problem solved by the PnP algorithm.
Revised: Therefore, using the GS denoiser from equation (4) is equivalent to include an explicit regularization and thus leads to a tractable global optimization problem solved by the PnP algorithm.

Original: Our complete
Revised: Our complete

Original: PnP scheme is presented in Algorithm 1.
Revised: PnP scheme is presented in Algorithm 1.

Original: It includes a backtracking procedure  that will be detailed in Section 4.2.
Revised: It includes a backtracking procedure on the stepsize τ that will be detailed in Section 4.2.

Original: Also, after convergence, we found it useful to apply an extra gradient step Id − λτ ∇ g σ in order to discard the residual noise brought by the last proximal step Prox τf .
Revised: Also, after convergence, we found it useful to apply an extra gradient step Id − λτ ∇ g σ in order to discard the residual noise brought by the last proximal step Prox τf .

Original:  C ONVERGENCE ANALYSIS
Revised:  C ONVERGENCE ANALYSIS

Original: Param.
Revised: Param.

Original: : init.
Revised: : init.

Original: z 0 , λ > 0 , σ ≥ 0 , (cid:15) > 0 , τ 0 > 0 ,
Revised: z 0 , λ > 0 , σ ≥ 0 , (cid:15) > 0 , τ 0 > 0 ,

Original: K ∈ N ∗ , η ∈ (0 , 1) , γ ∈ (0 , 1 / 2) .
Revised: K ∈ N ∗ , η ∈ (0 , 1) , γ ∈ (0 , 1 / 2) .

Original: Input : degraded image y .
Revised: Input : degraded image y .

Original: Output: restored image ˆ x . k = 0 ; x 0 =
Revised: Output: restored image ˆ x . k = 0 ; x 0 =

Original: Prox τf ( z 0 ) ; τ = τ 0 /η ; dif > (cid:15) ; while k < K and dif > (cid:15) do z k = λτD σ ( x k )
Revised: Prox τf ( z 0 ) ; τ = τ 0 /η ; ∆ > (cid:15) ; while k < K and ∆ > (cid:15) do z k = λτD σ ( x k )

Original: + (1 − λτ ) x k ; x k +

Revised: + (1 − λτ ) x k ; x k +

Original: =
Revised: =

Original: Prox τf ( z k ) ; if F ( x k ) − F ( x k +1 ) <
Revised: Prox τf ( z k ) ; if F ( x k ) − F ( x k +1 ) <

Original: γτ || x
Revised: γτ || x

Original: k − x k +

Revised: k − x k +

Original: || [-
-] ; then τ = ητ ; else dif = F ( x k ) − F ( x k +1 ) F ( x 0 ) ; 
Revised: || [-
-]2 ; then τ = ητ ; else ∆ = F ( x k ) − F ( x k +1 ) F ( x 0 ) ; k

Original: k = k + 1 ; end ˆ x = λτD σ ( x K )
Revised: = k + 1 ; end ˆ x = λτD σ ( x K )

Original: + (1 − λτ ) x K ;
Revised: + (1 − λτ ) x K ;

Original: In this section, we introduce conditions on f and D σ that will ensure the convergence of the
Revised: In this section, we introduce conditions on f and D σ that will ensure the convergence of the

Original: PnP iterations (9) towards a solution of (10).
Revised: PnP iterations (9) towards a solution of (10).

Original: For that purpose, we make use of the literature of convergence analysis (Attouch et al., 2013; Beck, 2017; Beck & Teboulle, 2009; Gu et al., 2018) of the proximal gradient descent algorithm in the non-convex setting.
Revised: For that purpose, we make use of the literature of convergence analysis (Attouch et al., 2013; Beck, 2017; Beck & Teboulle, 2009) of the PGD algorithm in the nonconvex setting.

Original: 4. C ONVERGENCE RESULTS
Revised: 4. C ONVERGENCE RESULTS

Original: A common setting for image restoration is the convex smooth L 2 data-ﬁdelity f ( x ) =
Revised: A common setting for image restoration is the convex smooth Ldata-ﬁdelity f ( x ) =

Original: ||
Revised: ||

Original: Ax − y ||
Revised: Ax − y ||

Original: In order to cover the noiseless case or to deal with a broader range of  degradation models, like Laplace or Poisson noise, we only assume f to be proper, lower semicontinous and convex.
Revised: In order to cover the noiseless case or to deal with a broader range of common degradation models, like Laplace or Poisson noise, we only assume f to be proper, lower semicontinous and convex.

Original: On the other hand, the regularizer g σ is assumed to be differentiable with Lipschitz gradient, but not necessarily convex. 
Revised: Next, the regularizer g σ is assumed to be differentiable with Lipschitz gradient, but not necessarily convex. This assumption on g σ is reasonable from a practical perspective.

Revised: Indeed, using a network N σ with differentiable activation functions, the function g σ introduced in Section 3 is differentiable with

Revised: Lipschitz gradient (details and proof are given in Appendix B).

Original: Without further assumptions on f and g σ , the following theorem establishes the convergence of  the objective function values andconvergence of the residual.
Revised: Without further assumptions on f and g σ , the following theorem establishes the convergence of both the objective function values and the residual for a large variety of IR tasks.

Original: Theorem 1 (Proof in Appendix A) .
Revised: Theorem 1 (Proof in Appendix C) .

Original: Let f : R n → R ∪ { + ∞} and g σ : R n → R be proper lower semicontinous functions with f convex and g σ differentiable with L -Lipschitz gradient.
Revised: Let f : R n → R ∪ { + ∞} and g σ : R n → R be proper lower semicontinous functions with f convex and g σ differentiable with L -Lipschitz gradient.

Original: Let λ > 0 ,
Revised: Let λ > 0 ,

Original: F = f + λg σ and assume that F is bounded from below.
Revised: F = f + λg σ and assume that F is bounded from below.

Original: Then, for τ < 1 λL , the iterates x k given by the iterative scheme (9) verify
Revised: Then, for τ < 1 λL , the iterates x k given by the iterative scheme (9) verify

Original: (i) ( F ( x k )) is non-increasing and converges.
Revised: (i) ( F ( x k )) is non-increasing and converges.

Original: (ii)
Revised: (ii)

Original: The residual || x k +1 − x k || converges to 0 .
Revised: The residual || x k +1 − x k || converges to 0 .

Original: (iii) All cluster points of the sequence ( x k ) are stationary points of (10).
Revised: (iii) All cluster points of the sequence ( x k ) are stationary points of (10).

Original: Remark 2.
Revised: Remark 2.

Original: In the non-convex setting, the quantity γ k = min 0 ≤ i ≤ k
Revised: In the nonconvex setting, the quantity γ k = min 0 ≤ i ≤ k

Original: || x
Revised: || x

Original: i +1 − x i || 2 is commonly used to analyze the convergence rate of the algorithm (Beck & Teboulle, 2009; Ochs et al., 2014).
Revised: i +1 − x i || 2 is commonly used to analyze the convergence rate of the algorithm (Beck & Teboulle, 2009; Ochs et al., 2014).

Original: Following the proof of Theorem 1 in Appendix A, we can obtain γ k ≤ 1 k F ( x 0 ) − lim F ( x k ) 12 τ − Lthat is to say a O ( 1 k ) convergence rate for the squared L 2 residual.
Revised: Following the proof of Theorem 1 in Appendix C, we can obtain γ k ≤ 1 k F ( x 0 ) − lim F ( x k ) 12 τ − Lthat is to say a O ( 1 k ) convergence rate for the squared L 2 residual.

Original: Remark 3.
Revised: Remark 3.

Original: Even if most data-ﬁdelity terms f encountered in image restoration are convex, Theorem 1 can be extended to nonconvex f .
Revised: Even if most data-ﬁdelity terms f encountered in image restoration are convex, Theorem 1 can be extended to nonconvex f .

Original: The proof of (iii) requires technical adaptations that can be found in (Li & Lin, 2015, Theorem 1) (in particular the 1-Lipschitz property of Prox τf does not hold anymore). 
Revised: The proof of (iii) requires technical adaptations that can be found in Li & Lin (2015, Theorem 1) (as the 1-Lipschitz property of Prox τf does not hold anymore). Such nonconvex f appear for example in the context of phase retrieval (Metzler et al., 2018).

Original: We can further obtain convergence of the iterates by assuming that the generated sequence ( x k ) is bounded and that f and g σ verify the Kurdyka-Lojasiewicz (KL) property.
Revised: We can further obtain convergence of the iterates by assuming that the generated sequence ( x k ) is bounded and that f and g σ verify the Kurdyka-Lojasiewicz (KL) property.

Original: The boundedness of ( x k ) is discussed in Appendix B. The KL property (deﬁned in Appendix C) has been widely used to study the convergence of optimization algorithms in the non-convex setting (Attouch et al., 2010; 2013;
Revised: The boundedness of ( x k ) is discussed in Appendix D. The KL property (deﬁned in Appendix E) has been widely used to study the convergence of optimization algorithms in the nonconvex setting (Attouch et al., 2010; 2013;

Original: Ochs et al., 2014).
Revised: Ochs et al., 2014).

Original: Very large classes of functions, in particular all the semi-algebraic functions, satisfy this property.
Revised: Very large classes of functions, in particular all the semi-algebraic functions, satisfy this property.

Original: In practice, in the extent of our analysis, the KL property is always satisﬁed.
Revised: In practice, in the extent of our analysis, the KL property is always satisﬁed.

Original: Theorem 2 (Proof in Attouch et al.
Revised: Theorem 2 (Proof in Attouch et al.

Original: Theorem 5.1) .
Revised: Theorem 5.1) .

Original: Let f : R n → R ∪ { + ∞} and g σ : R n → R be proper lower semicontinous functions with f convex and g σ differentiable with
Revised: Let f : R n → R ∪ { + ∞} and g σ : R n → R be proper lower semicontinous functions with f convex and g σ differentiable with

Original: L -Lipschitz gradient.
Revised: L -Lipschitz gradient.

Original: Let λ > 0 , F = f + λg σ and assume that F is bounded from below.
Revised: Let λ > 0 , F = f + λg σ and assume that F is bounded from below.

Original: Assume that F verify the KL property.
Revised: Assume that F verify the KL property.

Original: Suppose that τ < 1 λL .
Revised: Suppose that τ < 1 λL .

Original: If the sequence ( x k ) given by the iterative scheme (9) is bounded, then it converges, with ﬁnite length, to a critical point x ∗ of F .
Revised: If the sequence ( x k ) given by the iterative scheme (9) is bounded, then it converges, with ﬁnite length, to a critical point x ∗ of F .

Original: Remark 4.
Revised: Remark 4.

Original: As explained in Attouch et al.
Revised: As explained in Attouch et al.

Original: (2013, Remark 5.2), the continuity of f is not required since we use the “exact forward-backward splitting algorithm”.
Revised: (2013, Remark 5.2), the continuity of f is not required since we use the “exact forward-backward splitting algorithm”.

Original: We can thus deal with non-continuous data-ﬁdelity terms, as it is the case with the inpainting application in Appendix G.4.
Revised: We can thus deal with non-continuous data-ﬁdelity terms, as it is the case with the inpainting application in Appendix J.3.

Original: As detailed before, the assumptions in Theorems 1 and 2 on f and g σ are reasonable from a practical perspective.
Revised: A more detailed description of all the assumptions of Theorems 1 and 2 is given in Appendix F.

Original: Indeed, using a network N σ with differentiable activation functions, the function g σ introduced in Section 3 is differentiable with Lipschitz gradient (details and proof are given in Appendix D).

Original: Therefore, for a large variety of image restoration tasks, our PnP scheme (9) is guaranteed to converge to a stationary point of the targeted objective function F .

Original: 4. B ACKTRACKING TO HANDLE THE L IPSCHITZ CONSTANT OF ∇ g σ
Revised: 4. B ACKTRACKING TO HANDLE THE L IPSCHITZ CONSTANT OF ∇ g σ

Original: The convergence of Algorithm 1 actually requires to control the Lipschitz constant of ∇ g σ only on a small subset of images related to { x k } .
Revised: The convergence of Algorithm 1 actually requires to control the Lipschitz constant of ∇ g σ only on a small subset of images related to { x k } .

Original: Therefore, estimating L for all images and setting the maximum stepsize τ accordingly will lead to sub-optimal convergence speed.
Revised: Therefore, estimating L for all images and setting the maximum stepsize τ accordingly will lead to sub-optimal convergence speed.

Original: In order to avoid small stepsizes, we use the backtracking strategy (Beck, 2017, Chapter 10).
Revised: In order to avoid small stepsizes, we use the backtracking strategy of Beck (2017, Chapter 10) and Ochs et al.

Original: The convergence study in the proof of Theorem 1 is based on the sufﬁcient decrease property of F established  equation (22).
Revised: The convergence study in the proof of Theorem 1 is based on the sufﬁcient decrease property of F established in equation (33).

Original: Without knowing the exact Lipschitz constant L , backtracking aims at ﬁnding the maximal stepsize τ that yields the sufﬁcient decrease property.
Revised: Without knowing the exact Lipschitz constant L , backtracking aims at ﬁnding the maximal stepsize τ yielding the sufﬁcient decrease property.

Original: Given γ ∈ (0 , 1 / 2) , η ∈ [0 , 1) and an initial stepsize τ 0 > 0 , thebacktracking procedure consists in applying at each iteration k the following update rule on τ  :
Revised: Given γ ∈ (0 , 1 / 2) , η ∈ [0 , 1) and an initial stepsize τ 0 > 0 , the following update rule on τ is applied at each iteration k :

Original: [Equation]
Revised: [Equation]

Original: Proposition 1 (Proof in Appendix E) .
Revised: Proposition 1 (Proof in Appendix G) .

Original: Under the assumptions of Theorem 1, at each iteration of the algorithm, the backtracking procedure (11) is ﬁnite ( i.e. a stepsize satifying
Revised: Under the assumptions of Theorem 1, at each iteration of the algorithm, the backtracking procedure (11) is ﬁnite ( i.e. a stepsize satifying

Original: F ( x k ) − F ( T τ,λ GS-PnP ( x k )) ≥
Revised: F ( x k ) − F ( T τ,λ GS-PnP ( x k )) ≥

Original: γτ || T τ,λ GS-PnP ( x k ) − x k || 2 is found in a ﬁnite number if iterations), and with backtracking, the convergence results of Theorem 1 and Theorem 2 still hold.
Revised: γτ || T τ,λ GS-PnP ( x k ) − x k || 2 is found in a ﬁnite number if iterations), and with backtracking, the convergence results of Theorem 1 and Theorem 2 still hold.

Original: Remark 5.
Revised: Remark 5.

Original: In practice, as explained in Section 5, we choose to initialize the stepsize as λτ 0 =and backtracking hardly ever activates.
Revised: In practice, as explained in Section 5, we choose to initialize the stepsize as λτ 0 =and backtracking hardly ever activates.

Original: In this particular case, our results prove convergence of the standard PnP-HQS scheme x k +

Revised: In this particular case, our results prove convergence of the standard PnP-HQS scheme x k +

Original: =
Revised: =

Original: Prox τf ◦ D σ ( x k ) applied with our speciﬁc denoiser.  
Revised: Prox τf ◦ D σ ( x k ) applied with our speciﬁc denoiser.  E XPERIMENTS

Revised: In this section, we ﬁrst show the performance of the GS denoiser.

Revised: Next we empirically conﬁrm that our GS-PnP method is convergent while providing state-of-the art results for different IR tasks.

Original: E XPERIMENTS 5. G RADIENT - DESCENT - BASED DENOISER
Revised: 5. G RADIENT - DESCENT - BASED DENOISER

Original: Denoising Network Architecture We choose to parameterize N σ with the architecture
Revised: Denoising Network Architecture We choose to parameterize N σ with the architecture

Original: DRUNet (Zhang et al., 2021)) (represented in Appendix H), a U-Net in which residual blocks are integrated.
Revised: DRUNet (Zhang et al., 2021)) (represented in Appendix H), a U-Net in which residual blocks are integrated.

Original: One ﬁrst beneﬁt of DRUNet is that it is built to take the noise level σ as input, which is consistent with our formulation.
Revised: One ﬁrst beneﬁt of DRUNet is that it is built to take the noise level σ as input, which is consistent with our formulation.

Original: Also, the U-Net models have previously offered good results in the context of prior approximation via Denoising Score Matching
Revised: Also, the U-Net models have previously offered good results in the context of prior approximation via Denoising Score Matching

Original: (Ho et al., 2020).
Revised: (Ho et al., 2020).

Original: Furthermore, Zhang et al.
Revised: Furthermore, Zhang et al.

Original: (2021) showed that DRUNet yields state-of-the-art results for denoising but also for PnP image restoration.
Revised: (2021) showed that DRUNet yields state-of-the-art results for denoising but also for PnP image restoration.

Original: In order to ensure differentiability w.r.t the input, we change RELU activations to ELU.
Revised: In order to ensure differentiability w.r.t the input, we change RELU activations to ELU.

Original: We also limit the number of residual blocks to 2 at each scale to lower the computational burden.
Revised: We also limit the number of residual blocks to 2 at each scale to lower the computational burden.

Original: Training details We use the color image training dataset proposed in Zhang et al.
Revised: Training details We use the color image training dataset proposed in Zhang et al.

Original: (2021) i.e. a combination of the Berkeley segmentation dataset (CBSD) (Martin et al., 2001), Waterloo Exploration Database (Ma et al., 2017), DIV2K dataset (Agustsson & Timofte, 2017) and Flick2K dataset (Lim et al., 2017).
Revised: (2021) i.e. a combination of the Berkeley segmentation dataset (CBSD) (Martin et al., 2001), Waterloo Exploration Database (Ma et al., 2017), DIV2K dataset (Agustsson & Timofte, 2017) and Flick2K dataset (Lim et al., 2017).

Original: During training, the input images are corrupted with a white Gaussian noise ξ σ with standard deviation σ randomly chosen in [0 , 50 / 255] .
Revised: During training, the input images are corrupted with a white Gaussian noise ξ σ with standard deviation σ randomly chosen in [0 , 50 / 255] .

Original: With our parameterization (6) of D σ , the network N σ is trained to minimize the L 2 loss (7).
Revised: With our parameterization (6) of D σ , the network N σ is trained to minimize the L 2 loss (7).

Original: While the original DRUNet is trained with L 1 loss, we stick to the L 2 loss to keep the interpretability of g σ as an approximation of the log prior (see Remark 1).
Revised: While the original DRUNet is trained with L 1 loss, we stick to the L 2 loss to keep the interpretability of g σ as an approximation of the log prior (see Remark 1).

Original: For each batch, the gradient ∇ g σ is computed with PyTorch differentiation tools.
Revised: For each batch, the gradient ∇ g σ is computed with PyTorch differentiation tools.

Original: We train the model on 128 × 128 patches randomly sampled from the training images, with batch size 16 , during 1500 epochs.
Revised: We train the model on 128 × 128 patches randomly sampled from the training images, with batch size 16 , during 1500 epochs.

Original: We use the ADAM optimizer with learning rate 10 − 4 , divided by 2 every 300 epochs.
Revised: We use the ADAM optimizer with learning rate 10 − 4 , divided byevery 300 epochs.

Original: It takes around one week to train the model on a single Tesla P100 GPU.
Revised: It takes around one week to train the model on a single Tesla P100 GPU.

Original: Denoising results We evaluate the PSNR performance of the proposed GS denoiser (GS-DRUNet) on 256 × 256 color images center-cropped from the original CBSD68 dataset images (Martin et al., 2001).
Revised: Denoising results We evaluate the PSNR performance of the proposed GS denoiser (GS-DRUNet) on 256 × 256 color images center-cropped from the original CBSD68 dataset images (Martin et al., 2001).

Original: In Table 1, we compare, for various noise levels σ , our model with the simpliﬁed DRUNet (called “DRUNet light ”) that has the same architecture as our GS-DRUNet (2 residual blocks) and that is trained (with L 2 loss) without the conservative ﬁeld constraint.
Revised: In Table 1, we compare, for various noise levels σ , our model with the simpliﬁed DRUNet (called “DRUNet light ”) that has the same architecture as our GS-DRUNet (2 residual blocks) and that is trained (with L 2 loss) without the conservative ﬁeld constraint.

Original: We also provide comparisons with the original DRUNet (Zhang et al., 2021)
Revised: We also provide comparisons with the original DRUNet (Zhang et al., 2021)

Original: (with 4 residual blocks at each scale and trained with
Revised: (with 4 residual blocks at each scale and trained with

Original: L 1 loss) and two state-of-the-art denoisers encountered in the PnP literature: FFDNet (Zhang et al., 2018) and
Revised: L 1 loss) and two state-of-the-art denoisers encountered in the PnP literature: FFDNet (Zhang et al., 2018) and

Original: DnCNN (Zhang et al., 2017a).
Revised: DnCNN (Zhang et al., 2017a).

Original: For each network, we indicate in Table 1 the average runtime while processing a 256 × 256 color image on one Tesla P100 GPU.
Revised: For each network, we indicate in Table 1 the average runtime while processing a 256 × 256 color image on one Tesla P100 GPU.

Original: [Table]
Revised: [Table]

Original: for various noise levels While keeping small runtime GS-DRUNet slightly outperforms its constrained light and outdistances the deep denoisers FFDNet and DnCNN.
Revised: light and outdistances the deep denoisers FFDNet and DnCNN.

Original: Our GS-DRUNet denoiser, despite being constrained to be an exact conservative ﬁeld, reaches the performance of (and even slightly outperforms) its unconstrained counterpart
Revised: Our GS-DRUNet denoiser, despite being constrained to be an exact conservative ﬁeld, reaches the performance of (and even slightly outperforms) its unconstrained counterpart

Original: DRUNet light .
Revised: DRUNet light .

Original: Second, departing from the latter, we are able to reduce the processing time by a large margin ( ÷ 7 ) while keeping close PSNR to the original DRUNet (around -0.05dB) and maintaining a signiﬁcant PSNR gap (around +0.5dB) with other deep denoisers like DnCNN and FFDNet.
Revised: Second, departing from the latter, we are able to reduce the processing time by a large margin ( ÷ 7 ) while keeping close PSNR to the original DRUNet (around -0.05dB) and maintaining a signiﬁcant PSNR gap (around +0.5dB) with other deep denoisers like DnCNN and FFDNet.

Original: Note that the time difference between GS-DRUNet and DRUNet light is due to the computation of ∇ g σ via backpropagation.
Revised: Note that the time difference between GS-DRUNet and DRUNet light is due to the computation of ∇ g σ via backpropagation.

Original: These results indicate that GS-DRUNet is likely to yield a competitive and fast PnP algorithm.
Revised: These results indicate that GS-DRUNet is likely to yield a competitive and fast PnP algorithm.

Original: 5. P LUG - AND -P LAY IMAGE RESTORATION
Revised: 5. P LUG - AND -P LAY IMAGE RESTORATION

Original: We apply our PnP Algorithm 1 to deblurring in Section 5.2.1, super-resolution in Section 5.2.2 and  inpainting in Appendix G.4.
Revised: We show in this section that, in addition to being convergent, our PnP Alg. 1 reaches state-of-the-art performance in deblurring and super-resolution (Sections 5.2.1 and 5.2.2) and realizes relevant inpainting (Appendix J.3).

Original: In all cases, we seek an estimate x∈ R n of the original clean image ˆ x ∈ R n , from an observationy ∈ R m obtained as y =
Revised: In all cases, we seek an estimate x of a clean image ˆ x ∈ R n , from an observation obtained as y =

Original: A ˆ x + ξ ν  , with A a m × n degradation matrix and ξ ν a white Gaussian noise with zero mean and standard deviation ν .
Revised: A ˆ x + ξ ν ∈ R m , with A a m × n degradation matrix and ξ ν a white Gaussian noise with zero mean and standard deviation ν .

Original: The objective function minimized by Algorithm 1 is
Revised: The objective function minimized by Alg. 1 is

Original: [Equation]
Revised: [Equation]

Original: In practice, for ν > 0 , we multiply F in equation (12) by ν 2 and consider F = f
Revised: In practice, for ν > 0 , we multiply F in equation (12) by ν 2 and consider F = f

Original: + λ ν g σ with f ( x ) =
Revised: + λ ν g σ with f ( x ) =

Original: 12 ||
Revised: 12 ||

Original: Ax − y
Revised: Ax − y

Original: || 2 , g σ ( x ) =
Revised: || 2 , g σ ( x ) =

Original: 12 || N σ ( x ) − x || 

Revised: 12 || N σ ( x ) − x ||

Original: and λ ν = λν 2 .
Revised: and λ ν = λν 2 .

Original: With this formulation, the convergences of iterates and objective values are guaranteed by Theorems 1 and 2.
Revised: With this formulation, the convergences of iterates and objective values are guaranteed by Theorems 1 and 2.

Original: We  demonstrate in Appendix G.4 that our framework can be extended to other kinds of objective functions.
Revised: We also demonstrate in Appendix J.3 that our framework can be extended to other kinds of objective functions.

Original: For example, inpainting noise-free input images leads to a non differentiable data-ﬁdelity term f .
Revised: For example, inpainting noise-free input images leads to a non differentiable data-ﬁdelity term f .

Original: Due to the large computational time of some compared methods, we use for evaluation and comparison a subset of 10 color images taken from the CBSD68 dataset (CBSD10) together with the three famous set3C images (butterﬂy, leaves and starﬁsh).
Revised: Due to the large computational time of some compared methods, we use for evaluation and comparison a subset of 10 color images taken from the CBSD68 dataset (CBSD10) together with the 3 famous set3C images (butterﬂy, leaves and starﬁsh).

Original: Quantitative results run on the full CBSD dataset arealso given in Appendix G. Allinput images are center-cropped to the size 256 × 256 .
Revised: Quantitative results run on the full CBSD68 dataset are given in Appendix J. All images are center-cropped to the size 256 × 256 .

Original: For each IR problem, we provide default values for the parameters σ and λ that can be used to treat successfully a large class of images and degradations.
Revised: For each IR problem, we provide default values for the parameters σ and λ that can be used to treat sucessfully a large class of images and degradations.

Original: The inﬂuence of both parameters is analyzed in Appendix G.5.
Revised: The inﬂuence of both parameters is analyzed in Appendix J.5.

Original: Performance could be marginally improved by tuning λ for each image, for example with the method of Wei et al.
Revised: Performance can be marginally improved by tuning λ for each image, for example with the method of Wei et al.

Original: based on reinforcement learning.
Revised: based on reinforcement learning.

Original: Inall our experiments, backtracking is performed with η = 0 .
Revised: In our experiments, backtracking is performed with η = 0 .

Original: 9 and γ = 0 .
Revised: 9 and γ = 0 .

Original: We estimated (see Appendix D) that on a majority of images, the Lipschitz constant L of ∇ g σ is slightly larger than 1 .
Revised: We observe (see Appendix B) that on a majority of images, the Lipschitz constant L of ∇ g σ is slightly larger than 1 .

Original: As convergence is ensured for λ ν τ = ν 2 λτ < L , wechoose to set the initial stepsize to τ 0 = ( ν 2 λ ) − 1 .
Revised: As convergence is ensured for λ ν τ = ν 2 λτ < L , we set the initial stepsize to τ 0 = ( ν 2 λ ) − 1 .

Original: At the ﬁrst iteration, the gradient step in equation (9) is thus exactly D σ .
Revised: At the ﬁrst iteration, the gradient step in equation (9) is thus exactly D σ .

Original: We observe that in the majority of our experiments, backtracking is never activated.
Revised: In the majority of our experiments, backtracking is never activated.

Original: Finally, the algorithm is initialized with a proximal step and terminates when the relative difference betweentwo consecutive values of the objective function is less than (cid:15) or the number of iterations exceeds K .
Revised: The algorithm is initialized with a proximal step and terminates when the relative difference between consecutive values of the objective function is less than (cid:15) or the number of iterations exceeds K .

Original: 5.2. D EBLURRING
Revised: 5.2. D EBLURRING

Original: For image deblurring, the degradation operator A
Revised: For image deblurring, the degradation operator A

Original: = H is a convolution performed with circular boundary conditions.
Revised: = H is a convolution performed with circular boundary conditions.

Original: Therefore, H = F ∗ Λ F , where F is the orthogonal matrix of the discrete
Revised: Therefore, H = F ∗ Λ F , where F is the orthogonal matrix of the discrete

Original: Fourier transform (and F ∗ its inverse), and Λ is a diagonal matrix.
Revised: Fourier transform (and F ∗ its inverse), and Λ is a diagonal matrix.

Original: The proximal operator of the data-ﬁdelity term f ( x ) = 12 || Hx − y
Revised: The proximal operator of the data-ﬁdelity term f ( x ) = 12 || Hx − y

Original: ||
Revised: ||

Original: 2 involves only element-wise inversion and writes
Revised: 2 involves only element-wise inversion and writes

Original: Prox τf ( z ) =
Revised: Prox τf ( z ) =

Original: F ∗ ( I n + τ Λ ∗ Λ) − 1 F ( τH T y + z ) .
Revised: F ∗ ( I n + [Equation] Λ ∗ Λ) − 1 F ( τH T y + z ) .

Original: We demonstrate the effectiveness of our method on a large variety of blur kernels (represented in
Revised: We demonstrate the effectiveness of our method on a large variety of blur kernels (represented in

Original: Table 2) and noise levels.
Revised: Table 2) and noise levels.

Original: As in Zhang et al.

Original: (2017b) ; Pesquet et al. ; Zhang et al. (2021), we use the 8 real-world camera shake kernels proposed in Levin et al.
Revised: As in (Zhang et al., 2017b; Pesquet et al., 2021; Zhang et al., 2021), we use the 8 real-world camera shake kernels proposed in Levin et al.

Original: (2009) as well as the 9 × 9 uniform kernel and the 25 × 25 Gaussian kernel with standard deviation 1 .
Revised: (2009) as well as the 9 × 9 uniform kernel and the 25 × 25 Gaussian kernel with standard deviation 1 .

Original: 6 (as in Romano et al.
Revised: 6 (as in (Romano et al., 2017)).

Original: We consider Gaussian noise with 3 noise levels ν ∈ { 2 . 55 , 7 . 65 , 12 . 75 } / 255 i.e. ν ∈ { 0 . 01 , 0 .
Revised: We consider Gaussian noise with 3 noise levels ν ∈ { 2 . 55 , 7 . 65 , 12 . 75 } / 255 i.e. ν ∈ { 0 . 01 , 0 .

Original: For all noise levels, we set σ = 1 .
Revised: For all noise levels, we set σ = 1 .

Original: 8 ν , λ ν = ν 2 λ = 0 .
Revised: 8 ν , λ ν = ν 2 λ = 0 .

Original: 1 for motion blur (kernels (a) to (h)) and λ ν = 0 .
Revised: 1 for motion blur (kernels (a) to (h)) and λ ν = 0 .

Original: 075 for static blur (kernels (i) and (j)).
Revised: 075 for static blur (kernels (i) and (j)).

Original: Initialization is done with z 

Revised: Initialization is done with z

Original: =
Revised: =

Original: y
Revised: y

Original: but we show in
Revised: but we show in

Original: Appendix G.6 the robustness to the initialization.
Revised: Appendix J.6 the robustness to the initialization.

Original: The stopping criteria are (cid:15) =
Revised: The stopping criteria are (cid:15) =

Original: 10 − 5 and K = 400 .
Revised: 10 − 5 and K = 400 .

Original: We compare in Table 2 our method (GS-PnP) against the patch-based method EPLL (Zoran & Weiss, 2011; Hurault et al., 2018), the deep PnP methods IRCNN (Zhang et al., 2017b), DPIR (Zhang et al., 2021), MMO (Pesquet et al., 2021), and the “RED-FP” algorithm (Romano et al., 2017)
Revised: We compare in Table 2 our method (GS-PnP) against the patch-based method EPLL (Zoran & Weiss, 2011; Hurault et al., 2018), the deep PnP methods IRCNN (Zhang et al., 2017b), DPIR (Zhang et al., 2021), MMO (Pesquet et al., 2021), and the “RED-FP” algorithm (Romano et al., 2017)

Original: (with TNRD denoiser (Chen & Pock, 2016)) referred to as RED.
Revised: (with TNRD denoiser (Chen & Pock, 2016)) referred to as RED.

Original: Both IRCNN and DPIR use PnP-HQS with a fast decrease of τ and σ in a few iterations (8 iterations for DPIR) without guarantee of convergence.
Revised: Both IRCNN and DPIR use PnP-HQS with a fast decrease of τ and σ in a few iterations (8 iterations for DPIR) without guarantee of convergence.

Original: DPIR uses the DRUNet denoiser from Table 1.
Revised: DPIR uses the DRUNet denoiser from Table 1.

Original: MMO is the only compared method that guarantees convergence by plugging a DnCNN denoiser trained with Lipschitz constraints, but the onlypretrained network given is for very low noise level.
Revised: MMO is the only compared method that guarantees convergence by plugging a DnCNN denoiser trained with Lipschitz constraints (but the only given network was trained for very low noise level).

Original: Finally, as RED only treats the Y channel in the YCbCr color space, we also indicate in Appendix G.2, for RED and the proposed method, the PSNR evaluated on the Y channel only.
Revised: Finally, as RED only treats the Y channel in the YCbCr color space, we also indicate in Appendix J.1, for RED and the proposed method, the PSNR evaluated on the Y channel only.

Original: Among all methods, GS-PnP closely follows DPIR in terms of PSNR for low noise level but performs equally or better for higher noise levels.
Revised: Among all methods, GS-PnP closely follows DPIR in terms of PSNR for low noise level but performs equally or better for higher noise levels.

Original: Other comparisons are conducted in Appendix G. on the Set3cdataset (Table 4) andon the full CBSD68 dataset (Table 5).
Revised: Other comparisons are conducted in Appendix J.1 on the Set3c and the full CBSD68 datasets (Tables 4 and 5).

Original: These results all conﬁrm that GS-PnP reaches state-of-the-art in PnP deblurring, for a variety of kernels and noise levels.
Revised: These results exhibit that GS-PnP reaches state-of-the-art in PnP deblurring for a variety of kernels and noise levels.

Revised: We underline that the convergence of GS-PnP is guaranteed, whereas DPIR can asymptotically diverge (see Appendix J.7).

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: For qualitative comparison, we show in Figure 1(c-f) the deblurring obtained with various methods on the image “starﬁsh” (from set3C).
Revised: For qualitative comparison, we show in Figure 1(c-f) the deblurring obtained with various methods on the image “starﬁsh” (from set3C).

Original: Note that our algorithm, compared to competing methods, can recover the sharpest edges.
Revised: Note that our algorithm, compared to competing methods, can recover the sharpest edges.

Original: We also give convergence curves that empirically conﬁrm the convergence of the values F ( x k ) (g) and of the residual min 0 ≤ i ≤ k
Revised: We also give convergence curves that empirically conﬁrm the convergence of the values F ( x k ) (g) and of the residual min 0 ≤ i ≤ k

Original: || x
Revised: || x

Original: i +1 − x i
Revised: i +1 − x i

Original: || 2 /
Revised: || 2 /

Original: ||
Revised: ||

Original: x 0 || 2 (h).
Revised: x 0 || 2 (h).

Original: These observations are supported by the additional experiment shown in Appendix G.2, Figure 6.
Revised: These observations are supported by the additional experiment shown in Appendix J.1, Figure 6.

Original: [Figure]
Revised: [Figure]

Original: 5.2. S UPER - RESOLUTION
Revised: 5.2. S UPER - RESOLUTION

Original: For single image super-resolution, the low-resolution image y ∈ R m is obtained from the highresolution one x ∈ R n via y =
Revised: For single image super-resolution, the low-resolution image y ∈ R m is obtained from the highresolution one x ∈ R n via y =

Original: SHx + ξ ν where H ∈ R n × n is the convolution with anti-aliasing kernel.
Revised: SHx + ξ ν where H ∈ R n × n is the convolution with anti-aliasing kernel.

Original: The matrix S is the standard s-fold downsampling matrix of size m × n and n = s 2 × m .
Revised: The matrix S is the standard s-fold downsampling matrix of size m × n and n = s 2 × m .

Original: In this context, we make use of the closed-form calculation of the proximal map for the data-ﬁdelity term f ( x ) = 12 || SHx − y || 2 , given by Zhao et al.
Revised: In this context, we make use of the closed-form calculation of the proximal map for the data-ﬁdelity term f ( x ) = 12 || SHx − y || 2 , given by Zhao et al.

Original: [Equation]
Revised: [Equation]

Original: (14) where ˆ z τ = τH T S
Revised: (14) where ˆ z τ = τH T S

Original: T
Revised: T

Original: y
Revised: y

Original: + z and Λ =
Revised: + z and Λ =

Original: [Λ 1 , . .
Revised: [Λ 1 , . .

Original: , Λ s 2 ] ∈ R m × n , with Λ = diag(Λ 1 , . .
Revised: , Λ s 2 ] ∈ R m × n , with Λ = diag(Λ 1 , . .

Original: , Λ s 2 ) a blockdiagonal decomposition according to a s × s paving of the Fourier domain.
Revised: , Λ s 2 ) a blockdiagonal decomposition according to a s × s paving of the Fourier domain.

Original: Note that I m + τd ΛΛ ∗ is a m × m diagonal matrix and its inverse is computed element-wise.
Revised: Note that I m + τ d ΛΛ ∗ is a m × m diagonal matrix and its inverse is computed element-wise.

Original: As expected, with s = 1 , equation (14) comes down to equation (13).
Revised: As expected, with s = 1 , equation (14) comes down to equation (13).

Original: As in Zhang et al.
Revised: As in Zhang et al.

Original: (2021), we evaluate super-resolution performance on 8 Gaussian blur kernels represented in Table 3: 4 isotropic kernels with different standard deviations ( 0 . 7 , 1 . 2 , 1 . 6 and 2 . 0 ) and 4 anisotropic kernels.
Revised: (2021), we evaluate super-resolution performance on 8 Gaussian blur kernels represented in Table 3: 4 isotropic kernels with different standard deviations ( 0 . 7 , 1 . 2 , 1 . 6 and 2 . 0 ) and 4 anisotropic kernels.

Original: Results are averaged between isotropic and anisotropic.
Revised: Results are averaged between isotropic and anisotropic.

Original: We consider downsampled images at scale s = 2 and s = 3 and Gaussian noise with 3 different noise levels ν ∈ { 0 . 01 , 0 .
Revised: We consider downsampled images at scale s = 2 and s = 3 and Gaussian noise with 3 different noise levels ν ∈ { 0 . 01 , 0 .

Original: Our method (GS-PnP) is compared against bicubic upsampling, RED,
Revised: Our method (GS-PnP) is compared against bicubic upsampling, RED,

Original: IRCNN (“IRCNN+” from Zhang et al. (2021)) and DPIR.
Revised: IRCNN (“IRCNN+” from (Zhang et al., 2021)) and DPIR.

Original: We give again in Appendix G.3 the results obtained on the Set3C dataset (Table 7).
Revised: We give again in Appendix J.2 the results obtained on the Set3C dataset (Table 7).

Original: All our results are obtained with λ
Revised: All our results are obtained with λ

Original: ν = ν 2 λ = 0 .
Revised: ν = ν 2 λ = 0 .

Original: 065 and σ = 2 ν .
Revised: 065 and σ = 2 ν .

Original: Initialization z 0 is done with a bicubic interpolation of y (with a shift correction (Zhang et al., 2021)) and the stopping criteria are (cid:15)
Revised: Initialization z 0 is done with a bicubic interpolation of y (with a shift correction (Zhang et al., 2021)) and the stopping criteria are (cid:15)

Original: = 10 − 6 and K = 400 .
Revised: = 10 − 6 and K = 400 .

Original: Our method  outperforms in PSNR all other PnP methods over the considered range of blur kernels, noise levels and scale factors.
Revised: Besides being the only compared PnP method with convergence guarantee, GS-PnP outperforms in PSNR all other PnP algorithms over the considered range of blur kernels, noise levels and scale factors.

Original: We show in Figure 2 the super-resolution of the image “leaves” downsampled by 2, with an isotropic kernel and noise level ν = 0 .
Revised: We show in Figure 2 the super-resolution of the image “leaves” downsampled by 2, with an isotropic kernel and noise level ν = 0 .

Original: GS-PnP (f) recovers more accurately structures and color details than competing approaches (c-e), while converging in terms of function values (g) and residual (h).
Revised: GS-PnP (f) recovers more accurately structures and color details than competing approaches (c-e), while converging in terms of function values (g) and residual (h).

Original: Additional visual comparisons are presented in Appendix G.3.
Revised: Additional visual comparisons are presented in Appendix J.2.

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure] [Table] [Figure] Super-resolution with various methods on “leaves” (set3C) downsampled by 2 , with the indicated blur kernel and input noise level ν = 0 .
Revised: ) downsampled by 2 , with the indicated blur kernel and input noise level ν = 0 .

Original: Note that our algorithm is the one that recovers sharpest leaves.
Revised: Note that our algorithm is the one that recovers sharpest leaves.

Original: In (g) and (h), we show the evolution of F ( x k ) and γ k = min 0 ≤ i ≤ k || x
Revised: In (g) and (h), we show the evolution of F ( x k ) and γ k = min 0 ≤ i ≤ k || x

Original: i +1 − x i
Revised: i +1 − x i

Original: || 2 /
Revised: || 2 /

Original: ||
Revised: ||

Original: x 0 || 2 and in AppendixG.1 the evolution of the PSNR..
Revised: x 0 || 2 and in AppendixJ.4 the evolution of the PSNR..

Original: The empirical convergence rate is faster than the O ( 1 k ) theoretical worst case rate established in Remark 2.
Revised: The empirical convergence rate is faster than the O ( 1 k ) theoretical worst case rate established in Remark 2.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we introduce a new PnP algorithm with convergence guarantees.
Revised: In this work, we introduce a new PnP algorithm with convergence guarantees.

Original: A denoiser is trained to realize an exact gradient step on a regularization function that is formulated through a neural network.
Revised: A denoiser is trained to realize an exact gradient step on a regularization function that is formulated through a neural network.

Original: This denoiser is plugged in an iterative scheme closely related to PnP-HQS, which is proved to converge towards a stationary point of an explicit functional.
Revised: This denoiser is plugged in an iterative scheme closely related to PnP-HQS, which is proved to converge towards a stationary point of an explicit functional.

Original: One strength of this approach is to simultaneously allow for a non strongly convex (and not smooth) data-ﬁdelity term with a denoiser that may not be non-expansive.
Revised: One strength of this approach is to simultaneously allow for a non strongly convex (and non smooth) data-ﬁdelity term with a denoiser that may not be nonexpansive.

Original: Experiments conducted on ill-posed imaging problems (deblurring, super-resolution, inpainting) conﬁrm the convergence results and show that the proposed PnP algorithm reaches state-of-the-art image restoration performance.
Revised: Experiments conducted on ill-posed imaging problems (deblurring, super-resolution, inpainting) conﬁrm the convergence results and show that the proposed PnP algorithm reaches state-of-the-art image restoration performance.

Original: This work also opens several research perspectives.
Revised: This work also opens several research perspectives.

Original: One could ﬁrst examine which information is encoded in the proposed prior.
Revised: One could ﬁrst examine which information is encoded in the proposed prior.

Original: For example, based on the sharp visual results, one can question if a relation can be drawn between this prior and the gradient energy or sparsity.
Revised: For example, based on the sharp visual results, one can question if a relation can be drawn between this prior and the gradient energy or sparsity.

Original: Also, it would be interesting to see if the recovery analysis developed in Liu et al. (2021) can adapt to the proposed framework.
Revised: Also, it would be interesting to see if the recovery analysis developed in (Liu et al., 2021) can adapt to the proposed framework.

Original:  R EPRODUCIBILITY S TATEMENT
Revised:  R EPRODUCIBILITY S TATEMENT

Original: Anonymous source code is given in supplementary material.
Revised: Anonymous source code is given in supplementary material.

Original: It contains a README.md ﬁle that explains step by step how to run the algorithm and replicate the results of the paper.
Revised: It contains a README.md ﬁle that explains step by step how to run the algorithm and replicate the results of the paper.

Original: Moreover, the pseudocode of our algorithm is given Algorithm 1.
Revised: Moreover, the pseudocode of our algorithm is given Algorithm 1.

Original: In Section 5 it is precisely detailed how all the hyper-parameters are chosen and, for each experiment, which dataset is used.
Revised: In Section 5 it is precisely detailed how all the hyper-parameters are chosen and, for each experiment, which dataset is used.

Original: As for the theoretical results presented in Section 4, complete proofs are given in the appendixes.
Revised: As for the theoretical results presented in Section 4, complete proofs are given in the appendixes.

Original:  E THICS S TATEMENT
Revised:  E THICS S TATEMENT

Original: We believe that this work does not raise potential ethical concerns.
Revised: We believe that this work does not raise potential ethical concerns.

Original: A  P ROOF OF T HEOREM
Revised: A C OMPARISON WITH THE P N P LITERATURE A. L IMITATIONS OF PREVIOUS P N P METHODS

Revised: In the existing literature, PnP approaches have one of the following limitations: • They are not able to provide proof of convergence when non strongly convex data-ﬁdelity terms are involved (Ryu et al., 2019), which is the case of some classical IR problems such as deblurring, super-resolution or inpainting. • They are restricted to (nearly) nonexpansive denoisers (Reehorst & Schniter, 2018; Ryu et al., 2019;

Revised: Sun et al., 2019b; Xu et al., 2020) or denoisers with a symmetric Jacobian

Revised: (Romano et al., 2017).

Revised: But it has already been shown that imposing symmetric Jacobian or Lipschitz property on a deep denoiser network alters its denoising performance (Bohra et al., 2021;

Revised: Hertrich et al., 2021).

Revised: We highlight that it was already empirically observed (Romano et al., 2017; Zhang et al., 2021) that the performance of the denoiser directly impacts the performance of the corresponding PnP scheme for IR.

Revised: • They show convergence of iterates thanks to decreasing time steps (Chan et al., 2016), but there is no characterization of the obtained solution (it is not a minima or a critical point of any functional).

Revised: On the other hand, our method is proved to converge to a stationary point of an explicit functional including a non strongly convex data-ﬁdelity term.

Revised: It also relies on a (possibly expansive) denoiser that, although being constrained to be a conservative vector ﬁeld, allows to produce state-of-the-art results for various ill-posed IR problems.

Revised: A. O N THE REGULARIZATION g σ .

Revised: We ﬁrst underline that the main point of our method is to deﬁne the denoiser as D σ = Id −∇ g σ .

Revised: The choice for g σ is important for the denoising performance.

Revised: With respect to the convergence properties of GS-PnP, this is nevertheless a secondary issue, as our method would converge for other differentiable regularizers g σ .

Revised: The proposed regularization g σ ( x ) = 12 || x − N σ ( x ) || 2 was previously mentioned in the RED original paper (Romano et al., 2017)

Revised: (but explicitly left aside) and used in the DAEP paper (Bigdeli &

Revised: Zwicker, 2017).

Revised: The main difference between our regularizer and the one alternately proposed in

Revised: RED and DAEP is the following:

Revised: • RED and DAEP

Revised: both consider a generic given pretrained denoiser D σ : R n → R n , which is then associated with the regularizer g σ ( x ) =

Revised: 12 || x − D σ ( x ) || 2 and used as such in IR problems.

Revised: • In our method, we set g σ ( x ) =

Original: {+12 || x − N σ ( x ) || 
+}
Revised: {+12 || x − N σ ( x ) || 
+}

Revised: (with N σ : R n → R n differentiable) and then we train the denoiser as D σ = Id −∇ g σ with the loss function ||

Revised: D σ ( x + ξ ) − x || 2 for clean images x and additive white Gaussian noise (AWGN) ξ .

Revised: With this new formulation, we are ensured that D σ = Id −∇ g σ is inherently a conservative vector ﬁeld, without further assumptions on N σ .

Revised: Thanks to this relation, the (slightly modiﬁed) PnP-HQS given in relation (9) becomes a proximal gradient descent (PGD).

Revised: We can then make use of convergence results of the PGD algorithm in the nonconvex setting to show the convergence of PnP-HQS.

Revised: In contrast to the original RED paper, we aimed at ﬁnding one setting of Plug-and-Play image restoration that allows for a convergence proof with sufﬁciently general hypotheses.

Revised: For this purpose, we had to consider this very particular form of regularization.

Revised: A. R ECENT LITERATURE ON R EGULARIZATION BY DENOISING

Revised: We here provide a more detailed discussion on the follow-up literature on RED.

Revised: In parallel to the RED method (Romano et al., 2017), the authors of Bigdeli & Zwicker (2017) propose to use the regularization, mentioned but not exploited in RED, g ( x ) =

Revised: || D σ ( x ) − x || 2 , where D σ is a pretrained denoising autoencoder.

Revised: Next, Bigdeli et al.

Revised: (2017) extended this work with a new prior, which is the Gaussian-smoothed version of the natural image prior.

Revised: Inspired by

Revised: Tweedie’s formula, they approximate the gradient of this log smoothed prior with the residual of a pretrained denoising autoencoder.

Revised: With this new formulation, it is possible to optimize on the restored image but also on other parameters (e.g. the noise level and the used blur kernel).

Revised: Initially designed in the context of convex data-ﬁdelity term, RED Romano et al.

Revised: (2017) has been applied in the nonconvex setting for phase retrieval problems in prDeep Metzler et al.

Revised: Regularization by Artifact-Removal (RARE) (Liu et al., 2020) extends the RED framework by replacing the denoiser by a more general artifact-removal operator.

Revised: The main advantage of this operator is that it can be trained without groundtruth data, but only by mapping pairs of artifact and noise contaminated images obtained directly from undersampled measurements.

Revised: This is particularly useful for medical imaging applications where it is difﬁcult to acquire fully-sampled training data.

Revised: The convergence of the original RED algorithm is discussed in Reehorst & Schniter (2018).

Revised: The authors provide a convergence proof for RED-PGD which requires the denoiser to be nonexpansive, which, as detailed in the previous sections, is a restrictive hypothesis.

Revised: The authors of Liu et al.

Revised: (2021) provide a recovery guarantee for the PnP framework, meaning convergence to a x ∗ that satisﬁes y

Revised: =

Revised: Ax ∗ while being in the set Fix ( D ) of the ﬁxed points of D .

Revised: More precisely, they show the convergence of the PnP-PGD method towards such a true solution x ∗ under the assumptions that the denoiser residual R = Id − D is bounded and Lipschitz, and that the measurement operator satisﬁes a “set-restricted eigenvalue condition” (S-REC, which can be understood as strong convexity on the image of the denoiser Im ( D ) ).

Revised: Under the additional assumptions that the denoiser is nonexpansive and that there exists

Revised: x ∈ Fix ( D ) that is also critical for the regularizer g , they show that PnP and RED have the same solutions.

Revised: As mentioned by the authors, it is nevertheless difﬁcult to verify the S-REC condition for a given measurement operator: since Im ( D ) is not explicit, it is not clear how much S-REC relaxes the strong convexity.

Revised: As explained in Sectionsand 3, our results do not require strong convexity of the data-ﬁdelity term.

Revised: Instead of including an explicit regularization in the functional, RED-PRO (Cohen et al., 2021) aims at minimizing the data-ﬁdelity term on the set Fix ( D ) of ﬁxed points of a generic denoiser

Revised: D .

Revised: The study is conducted under the hypothesis that the denoiser is demicontractive, which implies that Fix ( D ) is convex, thus leading to a convex optimization problem.

Revised: However, this assumption seems difﬁcult to verify in practice and the existence of ﬁxed points for the RED-PRO operator does not appear straightforward.

Revised: In contrast, the ﬁxed points of the GS-PnP operator are directly related to the stationary points of the global functional F = f + λg σ (Lemma 1 in Appendix C), whose existence is guaranteed as soon as F is coercive (see the discussion in Appendix D).

Revised: ASYNC-RED (Sun et al., 2020) enables faster computation of RED by decomposing the inference into a sequence of partial (block-coordinate)

Revised: updates on x which can be executed asynchronously in parallel over a multicore system.

Revised: As in their previous work BC-RED (Sun et al., 2019a), the authors propose to further reduce the computational time by using only a random subset of measurements at every iteration.

Revised: Convergence of ASYNC-RED is shown, provided the denoiser is nonexpansive.

Revised: A possible future extension of our work is the integration of the ASYNC framework to accelerate

Revised: GS-PnP for large scale imaging inverse problems.

Revised: As our GS-PnP converges without assuming nonexpansiveness of the denoising operation, it would be interesting to see if one can adapt the

Revised: GS-PnP convergence properties to an ASYNC-GSPnP algorithm.

Revised: B L IPSCHITZ CONSTANT OF ∇ g σ

Revised: First, let us give a result which ensures that a large class of neural networks trained with differentiable activation functions have Lipschitz gradients with respect to the input image.

Revised: Proposition 2.

Revised: Let H =

Revised: h p ◦ . . .

Revised: ◦

Revised: h 1 be a composition of differentiable functions

Revised: h i : R d i − 1 → R d i .

Revised: Let us assume

Revised: that for any i the differential map h (cid:48)

Revised: i is bounded and Lipschitz.

Revised: Then H (cid:48) is Lipschitz.

Revised: Proof.

Revised: Let us denote H

Revised: i

Revised: = h

Revised: i ◦ . . .

Revised: ◦

Original: {+h 1 (and by convention, H 
+}
Revised: {+h 1 (and by convention, H 
+}

Revised: = Id ).

Revised: Let (cid:107) h (cid:48)

Revised: i (cid:107) ∞ be the best uniform bound on the operator norms (cid:107) h (cid:48)

Revised: i ( x ) (cid:107) , x ∈ R d i − 1 (which is also the best Lipschitz constant of h i ).

Revised: Let us also denote (cid:107) h (cid:48)

Revised: i (cid:107) Lip the Lipschitz constant of h (cid:48)

Revised: i .

Revised: The chain rule gives that for any x ,

Revised: H

Revised: [Equation]

Revised: ( x ) can be expressed as a composition of linear maps

Revised: [Equation]

Revised: Therefore, for any x

Revised: [Equation]

Revised: y ,

Revised: [Equation]

Revised: We can thus bound the operator norms

Revised: [Equation]

Revised: 19) and thus

Revised: [Equation]

Revised: 20) which concludes because the chain-rule ensures that (cid:107) H (cid:48)

Revised: i (cid:107) ∞

Revised: ≤ (cid:107) h

Revised: [Equation]

Revised: i (cid:107) ∞ . . .

Revised: (cid:107) h

Revised: [Equation]

Revised: 1 (cid:107) ∞ .

Revised: Proposition 2 applies for a neural network obtained as a composition of fully-connected layers with

Revised: ELU activation functions, that is, by composing functions of the form

Revised: [Equation]

Revised: 21) where A is a matrix, b a vector and E is the element-wise ELU deﬁned by

Revised: [Equation]

Revised: It is easy to see that E is differentiable and that E (cid:48) is 1 -Lipschitz with (cid:107) E (cid:48) (cid:107) ∞ ≤ 1 .

Revised: Therefore

Revised: [Equation]

Revised: is also bounded and Lipschitz.

Revised: Let us also mention that this proposition encompasses the case of U-nets which, in addition to composing fully-connected layers, also integrates skip-connections.

Revised: For example, taking a skipconnection on a composition h 3 ◦ h 2 ◦ h 1 amounts to deﬁning

Revised: [Equation]

Revised: This can be simply rewritten H =

Revised: h 3 ◦ ˜ h

Revised: 2 ◦ h 1 where

Revised: [Equation]

Original: We ﬁrst remind that a function f : R n −→ R ∪ + ∞ is proper if its domain
Revised: It is then clear that ˜ h 2 has bounded Lipschitz differential as soon as h 1 and h 2 do. The bound obtained in the proof of Proposition 2 is exponential in the depth of the neural network. We now provide some experiments showing that, in practice, the Lipschitz constant of ∇ g σ does not explode. With the parameterization proposed in equation (5), g σ is twice differentiable with the right choice of the activation functions in the CNN N σ ( e.g. ELU).

Original: [Equation]
Revised: We show in Figure 3, for various noise levels σ , the distribution of the spectral norms ||∇ 2 g σ ( x ) || S on the training image set X , estimated with power iterations. The computed value varies a lot across images. Hence approximating the Lipschitz constant of ∇ g σ with L = max x ∈ X ||∇ 2 g σ ( x ) || S would lead to underestimated stepsizes and slow convergence on most images. Backtracking solves this issue by ﬁnding at each iteration the optimal stepsize allowing sufﬁcient decrease of the objective function.

Revised: C P ROOF OF T HEOREM

Revised: We ﬁrst remind that a function f : R n −→ R ∪ + ∞ is proper if its domain dom ( f ) = { x ∈ R , f ( x ) <

Original: 15) is non empty.
Revised: + ∞} (26) is non empty.

Original: Also, recall that f is lower semicontinuous at x ∗ if lim inf x → x ∗ f ( x ) ≥ f ( x ∗ ) .
Revised: Also, recall that f is lower semicontinuous at x ∗ if lim inf x → x ∗ f ( x ) ≥ f ( x ∗ ) .

Original: Proof.
Revised: Proof.

Original: (i) For ease of notation, we consider λ = 1 .
Revised: (i) For ease of notation, we consider λ = 1 .

Original: The generalisation for any λ > 0 is straightforward by rescaling g σ (and L ) accordingly.
Revised: The generalisation for any λ > 0 is straightforward by rescaling g σ (and L ) accordingly.

Original: We denote the proximal gradient ﬁxed point operator T τ =
Revised: We denote the proximal gradient ﬁxed point operator T τ =

Original: Prox τf ◦ (Id − τ ∇ x g σ ) , the objective function F = f + g σ
Revised: Prox τf ◦ (Id − τ ∇ x g σ ) , the objective function F = f + g σ

Original: and we introduce
Revised: and we introduce

Original: [Equation]
Revised: [Equation]

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: By deﬁnition for the arg min , x k +1 =
Revised: By deﬁnition for the arg min , x k +1 =

Original: T τ ( x k ) ⇒ Q
Revised: T τ ( x k ) ⇒ Q

Original: τ ( x k +1 , x k ) ≤ Q τ ( x k , x k ) .
Revised: τ ( x k +1 , x k ) ≤ Q τ ( x k , x k ) .

Original: Moreover, with g σ being L -smooth, we have by the descent lemma, for any τ ≤ 1 L and any x, y ∈ R n , 
Revised: Moreover, with g σ being L -smooth, we have by the descent lemma, for any τ ≤ 1 L and any x, y ∈ R n , [Equation]

Original: [Equation]
Revised: [Equation]

Original: so that for every x, y ∈ R n ,
Revised: so that for every x, y ∈ R n ,

Original: Q τ
Revised: Q τ

Original: [Equation]
Revised: [Equation]

Original: x, x ) = F ( x ) and Q τ ( x, y ) ≥
Revised: x, [Equation] Q [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Therefore, at iteration k ,
Revised: Therefore, at iteration k ,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: ( F ( x k )) is thus non-increasing.
Revised: ( F ( x k )) is thus non-increasing.

Original: Since F is lower-bounded, ( F ( x k )) thus converges to a limit F ∗ .
Revised: Since F is lower-bounded, ( F ( x k )) thus converges to a limit F ∗ .

Original: (ii) Note that Q τ ( x k +1 , x k ) ≤ Q τ ( x k , x k ) implies 
Revised: (ii) Note that Q τ ( x k +1 , x k ) ≤ Q τ ( x k , x k ) implies [Equation]

Original: [Equation]
Revised: [Equation]

Original: Using also (18) with stepsize 1 L , we get
Revised: Using also (29) with stepsize 1 L , we get

Original: [Equation]
Revised: [Equation]

Original: Summing over k = 0 , 1 , ..., m givesm (cid:88)
Revised: Summing over k = 0 , 1 , ..., m gives

Original: [Equation]
Revised: [Equation]

Original: Therefore, lim k →∞ || x k +1 − x k ||
Revised: Therefore, lim k →∞ || x k +1 − x k ||

Original: = 0 .
Revised: = 0 .

Original: (iii)
Revised: (iii)

Original: We begin by the two following lemmas characterizing the proximal gradient descent operator
Revised: We begin by the two following lemmas characterizing the proximal gradient descent operator

Original: T τ =
Revised: T τ =

Original: Prox τf ◦ (Id − τ ∇ x g σ ) .
Revised: Prox τf ◦ (Id − τ ∇ x g σ ) .

Original: Lemma 1.
Revised: Lemma 1.

Original: With the assumptions of Theorem 1, for x ∗ ∈ R n , x ∗ is a ﬁxed point of the proximal gradient descent operator T τ =
Revised: With the assumptions of Theorem 1, for x ∗ ∈ R n , x ∗ is a ﬁxed point of the proximal gradient descent operator T τ =

Original: Prox τf ◦ (Id − τ ∇ x g σ ) , i.e. T τ ( x ∗ ) =
Revised: Prox τf ◦ (Id − τ ∇ x g σ ) , i.e. T τ ( x ∗ ) =

Original: x ∗ , if and only if x ∗ is a stationary point of problem (10), i.e. −∇ g σ ( x ∗ )
Revised: x ∗ , if and only if x ∗ is a stationary point of problem (10), i.e. −∇ g σ ( x ∗ )

Original: ∈ ∂f ( x ∗ ) .
Revised: ∈ ∂f ( x ∗ ) .

Original: Proof.
Revised: Proof.

Original: By deﬁnition of the proximal operator, we have
Revised: By deﬁnition of the proximal operator, we have

Original: [Equation]
Revised: [Equation]

Original: Lemma 2.
Revised: Lemma 2.

Original: With the assumptions of Theorem 1, T τ is 1 + τL Lipschitz.
Revised: With the assumptions of Theorem 1, T τ is 1 + τL Lipschitz.

Original: Proof.
Revised: Proof.

Original: Using the fact that for f convex, Prox τf is 1-Lipschitz ((Bauschke & Combettes, 2011,
Revised: Using the fact that for f convex, Prox τf is 1-Lipschitz (Bauschke & Combettes, 2011,

Original: Proposition 12.28)), and by the Lipschitz property of ∇ x g σ 
Revised: Proposition 12.28), and by the Lipschitz property of ∇ x g σ ,

Original: [Equation]
Revised: [Equation]

Original: Note that, by nonconvexity of g σ , the ﬁxed point operator T τ is not necessarily non-expansive, but we can still show the convergence of the ﬁxed-point algorithm towards a critical point of the objective function.
Revised: Note that, by nonconvexity of g σ , the ﬁxed point operator T τ is not necessarily nonexpansive, but we can still show the convergence of the ﬁxed-point algorithm towards a critical point of the objective function.

Original: We can now turn to the proof of (iii).
Revised: We can now turn to the proof of (iii).

Original: Let x ∗ be a cluster point of ( x k ) k ≥ 0 .
Revised: Let x ∗ be a cluster point of ( x k ) k ≥ 0 .

Original: Then there exists a subsequence ( x k j ) j ≥ 0 converging to x ∗ .
Revised: Then there exists a subsequence ( x k j ) j ≥ 0 converging to x ∗ .

Original: We have ∀ j ≥ 0 ,
Revised: We have ∀ j ≥ 0 ,

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] Using (ii), the right-hand side of the inequity tends to 0 as j → ∞ .
Revised: Using (ii), the right-hand side of the inequality tends to 0 as j → ∞ .

Original: Thus || x ∗ − T τ ( x ∗ ) ||
Revised: Thus || x ∗ − T τ ( x ∗ ) ||

Original: = 0 and x ∗ =
Revised: = 0 and x ∗ =

Original: T τ ( x ∗ ) , which by Lemma 1 means that x ∗ is a stationary point of problem (10).
Revised: T τ ( x ∗ ) , which by Lemma 1 means that x ∗ is a stationary point of problem (10).

Original: B O N THE BOUNDEDNESS OF ( x k )
Revised: D O N THE BOUNDEDNESS OF ( x k )

Original: In order to obtain convergence of the iterates, in Theorem 2 the generated sequence ( x k ) is assumed to be bounded.
Revised: In order to obtain convergence of the iterates, in Theorem 2 the generated sequence ( x k ) is assumed to be bounded.

Original: In the experiments (Section 5), we observed that under the rest of assumptions of
Revised: In the experiments (Section 5), we observed that under the rest of assumptions of

Original: Theorem 2, boundedness was always veriﬁed.
Revised: Theorem 2, boundedness was always veriﬁed.

Original: A sufﬁcient condition for the boundedness of the iterates is the coercivity of the objective function, that is, lim | x |→∞ F ( x ) =
Revised: A sufﬁcient condition for the boundedness of the iterates is the coercivity of the objective function, that is, lim | x |→∞ F ( x ) =

Original: + ∞ (because the non-increasing property gives F ( x k ) ≤ F ( x 0 ) ).
Revised: + ∞ (because the non-increasing property gives F ( x k ) ≤ F ( x 0 ) ).

Original: Similar to Laumont et al.
Revised: Similar to Laumont et al.

Original: (2021), we can constrain F to be coercive by choosing a convex compact set C ⊂ R n where the iterates should stay and by adding an extra term to the regularization g σ :
Revised: (2021), we can constrain F to be coercive by choosing a convex compact set C ⊂ R n where the iterates should stay and by adding an extra term to the regularization g σ :

Original: [Equation]
Revised: [Equation]

Original: with Π C the Euclidian projection on C .
Revised: with Π C the Euclidian projection on C .

Original: As g σ is differentiable, the gradient step becomes
Revised: As g σ is differentiable, the gradient step becomes

Original: [Equation]
Revised: [Equation]

Original: In our experiments, we choose the compact set C as C =
Revised: In our experiments, we choose the compact set C as C =

Original: [ − 1 , 2] n .
Revised: [ − 1 , 2] n .

Original: In practice we observe that all the iterates always remain in C and that the extra regularization term || x − Π C ( x ) || 2 is never activated.
Revised: In practice we observe that all the iterates always remain in C and that the extra regularization term || x − Π C ( x ) || 2 is never activated.

Original: Therefore, we don’t present this technical adaptation in Algorithm 1 but we let the reader aware that boundedness of ( x k ) is not a limiting assumption.
Revised: Therefore, we don’t present this technical adaptation in Algorithm 1 but we let the reader aware that boundedness of ( x k ) is not a limiting assumption.

Original: C KL PROPERTY
Revised: E KL PROPERTY

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: Kurdyka-Lojasiewicz (KL) property (taken from Attouch et al. (2010))
Revised: Kurdyka-Lojasiewicz (KL) property (taken from Attouch et al. (2010))

Original: (a) A function f : R n −→ R ∪ + ∞ is said to have the Kurdyka-Lojasiewicz property atx ∗ ∈ dom ( f ) if there exists η ∈ (0 , + ∞ ) , a neighborhood U of x ∗ and a continuous concave function ψ : [0 , η ) −→ R + such that ψ (0) = 0 , ψ
Revised: (a) A function f : R n −→ R ∪ + ∞ is said to have the Kurdyka-Lojasiewicz property at

Original: is C 1 on (0 , η ) , ψ (cid:48) > 0 on ( η ) 
Revised: x ∗ ∈ dom ( f ) if

Original: and
Revised: there

Original:  ∀ x ∈ U ∩ [ f ( x ∗ ) <
Revised: exists η ∈ (0 , + ∞ ) , a neighborhood U of x ∗ and a continuous concave function ψ : [0 , η ) −→ R + such that ψ (0) = 0 , ψ is C 1 on (0 , η ) , ψ (cid:48) > 0 on (0 , η ) and ∀ x ∈ U ∩ [ f ( x ∗ ) <

Original: f < f ( x ∗ )
Revised: f < f ( x ∗ )

Original: + η ] , the Kurdyka -Lojasiewicz inequality holds :
Revised: + η ] , the Kurdyka-Lojasiewicz inequality holds:

Original: [Equation]
Revised: [Equation]

Original: (b) Proper lower semicontinuous functions which satisfy the Kurdyka-Lojasiewicz inequality at each point of dom∂f are called KL functions.
Revised: (b) Proper lower semicontinuous functions which satisfy the Kurdyka-Lojasiewicz inequality at each point of dom ( ∂f ) are called KL functions.

Original: This condition can be interpreted as the fact that, up to a reparameterization, the function is sharp i.e. we can bound its subgradients away from 0.
Revised: This condition can be interpreted as the fact that, up to a reparameterization, the function is sharp i.e. we can bound its subgradients away from 0.

Original: A big class of functions that have the KL-property is given by real semi-algebraic functions.
Revised: A big class of functions that have the KL-property is given by real semi-algebraic functions.

Original: For more details and interpretations, we refer to Attouch et al.
Revised: For more details and interpretations, we refer to Attouch et al.

Original: (2010) and Bolte et al.
Revised: (2010) and Bolte et al.

Original: D L IPSCHITZ CONSTANT OF ∇ g σ
Revised: F O N THE ASSUMPTIONS OF T HEOREMS 1 AND

Original: First, let us give a result which ensures that a large class of neural networks trained with differentiable activation functions have Lipschitz gradients with respect to the input image.
Revised: In this section, we explicitly list and comment all the assumptions required by Theorems 1 and 2. These assumptions are standard in nonconvex optimization. We now detail why each assumption is veriﬁed for our plug-and-play image restoration algorithm.

Original: Proposition 2. Let H = h p ◦ . . .
Revised: Assumptions of Theorem 1 :

Original: ◦ h 1 be a composition of differentiable functions h i : R d i − 1 → R d i . Let us assume
Revised: • Data-ﬁdelity term f : R n → R ∪ { + ∞} proper lower semicontinous and convex.

Original:  that for any i the differential map h (cid:48) i is bounded and Lipschitz. Then H (cid:48) is Lipschitz. Proof.
Revised: This is a general assumption that includes most of the data-ﬁdelity terms classically used in IR problems.

Original: Let us denote H i = h i ◦ . . . 
Revised: Note that we do not require differentiability of f . Degradations with Gaussian, Poisson or Laplacian noise models fall into this hypothesis.

Original: [-◦ h 1 (and by convention, H 
 = Id ). Let (cid:107) h (cid:48)-]
Revised: [-◦ h 1 (and by convention, H 
 = Id ). Let (cid:107) h (cid:48)-]As noticed in Remark 3, our results can even be easily extended to a nonconvex data-ﬁdelity term f , which encompasses applications like phase retrieval Metzler et al.

Original: i (cid:107) ∞ be the best uniform bound on the operator norms (cid:107) h (cid:48) i ( x ) (cid:107) , x ∈ R d i − 1 (which is also the best Lipschitz constant of h i ). Let us also denote (cid:107) h (cid:48) i (cid:107) Lip the Lipschitz constant of h (cid:48)
Revised: In practice, it is helpful to have f proximable,

Original: i.
Revised: i

Original: The chain rule gives that for any x , H (cid:48) ( x ) can be expressed as a composition of linear maps [Equation]
Revised: e. Prox f with closed-form formula.

Original: Therefore, for any x, y ,
Revised: Otherwise, Prox f needs to be

Original: [Equation] We can thus bound the operator norms [Equation] [Equation]
Revised: calculated at each iteration with an optimization algorithm.

Original: 34) and thus (cid:107) H [Equation] H [Equation]
Revised: • Regularization function g σ : R n → R proper lower semicontinous and differentiable with L -Lipschitz gradient .

Original: (cid:107) [Equation] 35) which concludes because the chain-rule ensures that (cid:107) H (cid:48) i (cid:107) ∞
Revised: We parametrize as g σ ( x ) =

Original: ≤ (cid:107) h (cid:48) i (cid:107) ∞ . . . (cid:107) h (cid:48) 1 (cid:107) ∞ . Proposition 2 applies for a  neural network obtained as a composition of fully-connected layers with
Revised: 12 || x − N σ ( x ) || 2 with N σ a differentiable neural network.

Original: ELU activation functions, that is, by composing functions of the form [Equation] where A is  a matrix, b a vector and E is the element-wise ELU deﬁned by [Equation]
Revised: This assumption on g σ is thus reasonable from a practical perspective.

Original: It is easy to see that E is differentiable and that E (cid:48) is 1 -Lipschitz and (cid:107) E (cid:48) (cid:107) ∞ ≤ 1 .
Revised: Indeed, using a network N σ with differentiable activation functions, our function g σ is differentiable with Lipschitz gradient (details and proof are given in Appendix B).

Original: Therefore [Equation]
Revised: • Functional F = f

Original: is also bounded and Lipschitz.
Revised: + λg σ bounded from below.

Original: Let us also mention that this proposition encompasses the case of U-nets which, in addition to composing fully-connected layers, also integrates skip-connections. For example, taking a skipconnection on a composition h 3 ◦ h 2 ◦ h 1 amounts to deﬁning [Equation] This can be simply rewritten H =
Revised: This is straightforward as all the terms are positive.

Original: h 3 ◦ ˜ h 2 ◦ h 1 where
Revised: • The stepsize τ < 1 λL .

Original: [Equation] It is then clear that ˜ h 2 has bounded Lipschitz differential as soon as h 1 and h 2 do. The bound obtained in the proof of Proposition 2 is exponential in the depth of the neural network. We now provide some experiments showing that, in practice, the Lipschitz constant of ∇ g σ does not explode.
Revised: This is handled by backtracking (see Section 4.2).

Original: With the parameterization proposed in equation (5), g σ is twice differentiable with the right choice of  the activation functions in the CNN N σ ( e.g. ELU). We show in Figure 3, for various noise levels σ , the distribution of ||∇ 2 g σ ( x ) || S on the training image set X , estimated with power iterations.
Revised: Assumptions of Theorem 2 : • Assumptions of Theorem• F verify the KL property. The KL property (deﬁned in Appendix E) has been widely used to study the convergence of optimization algorithms in the nonconvex setting (Attouch et al., 2010; 2013; Ochs et al., 2014).

Original: The computed value varies a lot across images. Hence approximating the Lipschitz constant of ∇ g σ with L = max x ∈ X ||∇ 2 g σ ( x  ) || S would lead to under-estimated stepsizes and slow convergence on most images.
Revised: Very large classes of functions, in particular all the semialgebraic functions, satisfy this technical property. It encompasses all the data-ﬁdelity and regularization terms encountered in inverse problems. • The sequence ( x k ) given by the iterative scheme (9) is bounded.

Original: Backtracking solves this issue by ﬁnding at each iteration the optimal stepsize allowing sufﬁcient decrease of the objective function.
Revised: As discussed in Appendix D, the boundedness can be ensured with a potential additional projection at each iteration. This is just a theoretical guarantee, as we observed that such a projection is never activated in practice.

Original: E P ROOF OF P ROPOSITION
Revised: G B ACKTRACKING AND P ROOF OF P ROPOSITION

Original: Proof.
Revised: Before giving the proof of Proposition 1, we ﬁrst point out that our backtracking line search is a classical Armijo-type backtracking strategy, already used for nonconvex optimization in (Beck, 2017, Chapter 10) or Ochs et al. Other procedures could be investigated in future work. For instance, Li & Lin (2015) uses a Barzilai-Borwein rule to initialize the backtracking line search. Scheinberg et al.

Revised: and Calatroni & Chambolle (2019) have also proposed a backtracking strategy that allows for both decreasing and increasing of the stepsize.

Original:  For a given stepsize τ , we showed in Appendix A, equation (22) that
Revised: We now give the proof of Proposition 1. Proof. For a given stepsize τ , we showed in Appendix C, equation (33) that

Original: [Equation]
Revised: [Equation]

Original: Taking τ < 1 − 2 γ L , we get 12 ( 1 τ − L ) > γτ
Revised: Taking τ < 1 − 2 γ L , we get 12 ( 1 τ − L ) > γτ

Original: so that
Revised: so that

Original: [Equation]
Revised: [Equation]

Original: Hence, when τ < 1 − 2 γ L , the sufﬁcient decrease condition equation (42) is satisﬁed and the backtracking procedure ( τ ←− ητ ) must end.
Revised: Hence, when τ < 1 − 2 γ L , the sufﬁcient decrease condition equation (42) is satisﬁed and the backtracking procedure ( τ ←− ητ ) must end.

Original: In the proof of Theorem 1, we can replace the sufﬁcient decrease (22) by (42) and ﬁnish the proof with the same arguments.
Revised: In the proof of Theorem 1, we can replace the sufﬁcient decrease (33) by (42) and ﬁnish the proof with the same arguments.

Original: In the same way, in the proof of Theorem 2 given in Attouch et al.
Revised: In the same way, in the proof of Theorem 2 given in (Attouch et al., 2013,

Original: (2013)[Theorem 5.1], our sufﬁcient decrease (42) replaces Attouch et al.
Revised: Theorem 5.1), our sufﬁcient decrease (42) replaces (Attouch et al., 2013, Equation (52)).

Revised: H DRUN ET light ARCHITECTURE

Revised: The architecture of the DRUNet light denoiser of (Zhang et al.

Original: (2013, Equation (52)).
Revised: (2021)) is given in Figure 4.

Original: F E XPANSIVENESS OF THE DENOISER
Revised: I E XPANSIVENESS OF THE DENOISER

Original: As g σ is not necessarily convex, our GS-DRUNet denoiser D σ = Id −∇ g σ is not necessarily nonexpansive and neither is the gradient step Id − λτ ∇ g σ .
Revised: As g σ is not necessarily convex, our GS-DRUNet denoiser D σ = Id −∇ g σ is not necessarily nonexpansive and neither is the gradient step Id − λτ ∇ g σ .

Original: This is not an issue as, unlike previous theoretical PnP studies (Terris et al., 2020; Reehorst & Schniter, 2018), our convergence results do not require a non-expansive denoising step.
Revised: This is not an issue as, unlike previous theoretical PnP studies (Terris et al., 2020; Reehorst & Schniter, 2018), our convergence results do not require a nonexpansive denoising step.

Original: To advocate that our method converges without this assumption, we show in Figure 4 the evolution of|| D σ ( x k +1 ) − D σ ( x k )
Revised: To advocate that our method converges without this assumption, we show in Figure 5 the evolution of

Original: || ||
Revised: [Figure]

Original: x k +1 − x k || along the algorithm that was run to obtain the super-resolution results of Figure 7.
Revised: along the algorithm that was run to obtain the super-resolution results of Figure 7.

Original: In this experiment, backtracking did not get activated and stayed ﬁxed at λτ = 1 .
Revised: In this experiment, backtracking did not get activated and stayed ﬁxed at λτ = 1 .

Original: The gradient step in the PGD algorithm was thus simply a denoising step D σ = Id − λτ ∇ g σ .
Revised: The gradient step in the PGD algorithm was thus simply a denoising step D σ = Id − λτ ∇ g σ .

Original: Note that the Lipschitz constant of D σ goes above 1 but convergence is still observed as shown by the two convergence curves in Figure 7.
Revised: Note that the Lipschitz constant of D σ goes above 1 but convergence is still observed as shown by the two convergence curves in Figure 7.

Original: G A DDITIONAL E XPERIMENTS G. PSNR CONVERGENCE CURVES

Original: We ﬁrst plot Figure 5 the evolution of the PSNR along the iterations of the PnP algorithm during the experiments of Figure 1 and Figure 2.

Original: This illustrates that the minimization of F coincides with the maximization of the PSNR, which supports the interest of the optimized functional F = f + λg σ .

Original: [Figure] G. D EBLURRING
Revised: J A DDITIONAL E XPERIMENTS J. D EBLURRING

Original: We give here additional image deblurring experiments.
Revised: We give here additional image deblurring experiments.

Original: We ﬁrst present the PSNR performance comparison on the Set3c dataset in Table 4.
Revised: We ﬁrst present the PSNR performance comparison on the Set3c dataset in Table 4.

Original: We also provide an evaluation of the 3 best methods (GS-PnP, DPIR and IRCNN) on the full CBSD68 dataset in Table 5.
Revised: We also provide an evaluation of the 3 best methods (GS-PnP, DPIR and IRCNN) on the full CBSD68 dataset in Table 5.

Original: For fair comparison with RED, we also expose Table 6 the PSNR calculated on the Y channel only.
Revised: For fair comparison with RED, we also display in Table 6 the PSNR calculated on the Y channel only.

Original: An additional visual comparison is ﬁnally shown in Figure 6.
Revised: An additional visual comparison is ﬁnally shown in Figure 6.

Original: Details and comments are given in the corresponding captions.
Revised: Details and comments are given in the corresponding captions.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: our method in this set of conditions.
Revised: our method in this set of conditions.

Original: However, over the variety of kernels and noise levels, and in particular for motion blur, our method generally outperforms RED.
Revised: However, over the variety of kernels and noise levels, and in particular for motion blur, our method generally outperforms RED.

Original: [Figure]
Revised: [Figure]

Original: G. S UPER - RESOLUTION
Revised: J. S UPER - RESOLUTION

Original: We also present additional super-resolution experiments.
Revised: We also present additional super-resolution experiments.

Original: We realize a full PSNR performance comparison on the Set3c dataset Table 7.
Revised: We realize a full PSNR performance comparison on the Set3c dataset Table 7.

Original: We show additional visual comparisons between methods
Revised: We show additional visual comparisons between methods

Original: Figure 7 and Figure 8.
Revised: Figure 7 and Figure 8.

Original: Details and comments are given in the corresponding captions.
Revised: Details and comments are given in the corresponding captions.

Original: [Table]
Revised: [Table]

Original: all methods at levels
Revised: all methods at levels

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: G. I NPAINTING ( WITH NON - DIFFERENTIABLE DATA - FIDELITY TERM )
Revised: J. I NPAINTING ( WITH NON - DIFFERENTIABLE DATA - FIDELITY TERM )

Original: We now propose to apply our PnP scheme to image inpainting with the degradation model
Revised: We now propose to apply our PnP scheme to image inpainting with the degradation model

Original: [Equation]
Revised: [Equation]

Original: where A is a diagonal matrix with values in { 0 , 1 } .
Revised: where A is a diagonal matrix with values in { 0 , 1 } .

Original: For inpainting, no noise is added to the degraded image.
Revised: For inpainting, no noise is added to the degraded image.

Original: In this context, the data-ﬁdelity term is the indicator function of A − 1 ( { y } )
Revised: In this context, the data-ﬁdelity term is the indicator function of A − 1 ( { y } )

Original: = { x | Ax = y } : f ( x ) =
Revised: = { x | Ax = y } : f ( x ) =

Original: ı A − 1 ( { y } ) (which, by deﬁnition, equals + ∞ on A − 1 ( { y } ) and 0 elsewhere).
Revised: ı A − 1 ( { y } ) (which, by deﬁnition, equals + ∞ on A − 1 ( { y } ) and 0 elsewhere).

Original: Despite being non differentiable, f still veriﬁes the assumptions of Theorems 1 and 2 and convergence is theoretically ensured.
Revised: Despite being non differentiable, f still veriﬁes the assumptions of Theorems 1 and 2 and convergence is theoretically ensured.

Original: The proximal map becomes the orthogonal projection
Revised: The proximal map becomes the orthogonal projection

Original: Π A − 1 ( { y } )
Revised: Π A − 1 ( { y } )

Original: [Equation]
Revised: [Equation]

Original: In our experiments, the diagonal of A is ﬁlled with Bernoulli random variables with parameter p = 0 .
Revised: In our experiments, the diagonal of A is ﬁlled with Bernoulli random variables with parameter p = 0 .

Original: We run our PnP algorithm with σ = 10 / 255 .
Revised: We run our PnP algorithm with σ = 10 / 255 .

Original: Given the form of f , we do not use the backtracking strategy and keep a ﬁxed stepsize.
Revised: Given the form of f , we do not use the backtracking strategy and keep a ﬁxed stepsize.

Original: Even if we do not exactly know the Lipschitz constant of ∇ g σ , we observed in Figure 4 that, for small noise, it was almost always estimated as slightly larger than 1 .
Revised: Even if we do not exactly know the Lipschitz constant of ∇ g σ , we observed in Figure 5 that, for small noise, it was almost always estimated as slightly larger than 1 .

Original: We thus choose λτ = 1 and empirically conﬁrm convergence with this choice in follow-up experiments (see Figure 9).
Revised: We thus choose λτ = 1 and empirically conﬁrm convergence with this choice in follow-up experiments (see Figure 9).

Original: The algorithm is initialized with x 0 = y + 0 .
Revised: The algorithm is initialized with x 0 = y + 0 .

Original: 5 ∗ (Id − A )
Revised: 5(Id − A )

Original: y (masked pixels with value 0 . 5 ) and terminates when the number of iterations exceeds K = 100 .
Revised: y (masked pixels with value 0 . 5 ) and terminates when the number of iterations exceeds K = 100 .

Original: We found it useful to run the ﬁrst 10 iterations of the algorithm at larger noise level σ = 50 / 255 .
Revised: We found it useful to run the ﬁrst 10 iterations of the algorithm at larger noise level σ = 50 / 255 .

Original: As y does not have noise, we found preferable not to run the last extra gradient pass from Algorithm 1.
Revised: As y does not have noise, we found preferable not to run the last extra gradient pass from Algorithm 1.

Original: We show inpainting results on set3C images Figure 9.
Revised: We show inpainting results on set3C images Figure 9.

Original: Our PnP restores the input images with high accuracy, including its small details.
Revised: Our PnP restores the input images with high accuracy, including its small details.

Original: Furthermore, convergence of the residual at rate O ( 1 k ) is empirically conﬁrmed. 
Revised: Furthermore, convergence of the residual at rate O ( 1 k ) is empirically conﬁrmed. [Figure] J. PSNR CONVERGENCE CURVES

Revised: We ﬁrst plot Figure 10 the evolution of the PSNR along the iterations of the PnP algorithm during the experiments of Figure 1 and Figure 2.

Revised: This illustrates that the minimization of F coincides with the maximization of the PSNR, which supports the interest of the optimized functional F = f + λg σ .

Original: [Figure]
Revised: [Figure]

Original: G. I NFLUENCE OF THE PARAMETERS
Revised: J. I NFLUENCE OF THE PARAMETERS

Original: In this section we study more deeply the inﬂuence of the parameters involved in the GS-PnP algorithm.
Revised: In this section we study more deeply the inﬂuence of the parameters involved in the GS-PnP algorithm.

Revised: Three parameters are involved: the stepsize τ , the denoiser level σ and the regularization parameter λ .

Revised: • The stepsize τ is automatically tuned with backtracking and is not tweaked heuristically, contrary to other competing methods based on PnP-HQS.

Revised: The ﬁrst regularization parameter σ is linked to the used denoiser.

Revised: • The second regularization parameter λ is introduced so as to target the objective function f + λg σ , which is the main purpose of our method.

Revised: It is a classical formulation of inverse problems, and the trade-off parameter λ is usually tuned manually.

Original: Figure 10 and Figure 11 respectively plot the average PSNR when deblurring the CBSD images with different values λ ν and σ/ν , and  ﬁxed  ν= 0 .
Revised: Thus, like PnP-HQS, we have two parameters that we are free to tune manually. One additional motivation for keeping both λ and σ as regularization parameters is to be able to use our PnP algorithm with noise-blind denoisers like DnCNN that are independent on σ . In practice, in our experiments, we ﬁrst roughly estimated σ proportionally to the input noise level ν and tweaked λ more precisely. Note that, for each inverse problem, our parameters λ and σ are ﬁxed for a large variety of kernels, images and noise levels ν .

Revised: The parameters are not optimized for each image.

Revised: Figure 11 and Figure 12 respectively plot the average PSNR when deblurring the CBSD10 images with different values λ ν and σ/ν , and ﬁxed ν = 0 .

Original: Both parameters control the strength of the regularization.
Revised: Both parameters control the strength of the regularization.

Original: We observe that λ ν and σ have a similar inﬂuence on the output: for small λ ν or small σ , the regularization involved by the denoising pass is not sufﬁcient to counteract the noise ampliﬁcation done by the proximal steps with large τ (recall that when τ → ∞ , Prox τf tends to the pseudo-inverse of A ).
Revised: We observe that λ ν and σ have a similar inﬂuence on the output: for small λ ν or small σ , the regularization involved by the denoising pass is not sufﬁcient to counteract the noise ampliﬁcation done by the proximal steps with large τ (recall that when τ → ∞ , Prox τf tends to the pseudo-inverse of A ).

Original: On the contrary, as expected, increasing λ and σ tends to over-smooth the output result.
Revised: On the contrary, as expected, increasing λ and σ tends to over-smooth the output result.

Original: On a single image, we also vary the main parameters of both GS-PnP and RED (Figure 12).
Revised: On a single image, we also vary the main parameters of both GS-PnP and RED (Figure 13).

Original: For fair comparison, the PSNR is computed on the luminance channel only.
Revised: For fair comparison, the PSNR is computed on the luminance channel only.

Original: This experiment conﬁrms that, when manually optimizing the parameters for both methods, the PSNR results obtained with
Revised: This experiment conﬁrms that, when manually optimizing the parameters for both methods, the PSNR results obtained with

Original: GS-PnP and RED remain close, as already observed in Table 6. 
Revised: GS-PnP and RED remain close, as already observed in Table 6. [Figure] [Figure]

Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original:  [Figure]
Revised: [Equation] [Figure] [Equation] [Figure]

Original: [Figure]
Revised: [Figure]

Original: G. I NFLUENCE OF THE INITIALIZATION
Revised: J. I NFLUENCE OF THE INITIALIZATION

Original: In Figure we examines the robustness of the method to the initialization.
Revised: In Figure we examine the robustness of the method to the initialization.

Original: As can be seen on this experiment, the output image does not change much even for relatively large perturbation of the initialization.
Revised: As can be seen on this experiment, the output image does not change much even for relatively large perturbation of the initialization.

Original: We thus observe a robustness to the initialization, both in terms of visual aspect and
Revised: We thus observe a robustness to the initialization, both in terms of visual aspect and

Original: PSNR.
Revised: PSNR.

Original: We also observed that initializing with a uniform image does not change the output of the algorithm.
Revised: We also observed that initializing with a uniform image does not change the output of the algorithm.

Original: We suggest that the robustnessto the initialization comes from the ﬁrst proximal steps on the data-ﬁdelity term (with a large τ ), which prevents the algorithm to be stuck in a poor local minimum. 
Revised: We suggest that this robustness comes from the ﬁrst proximal steps on the data-ﬁdelity term (with a large τ ) which prevent the algorithm to be stuck in a poor local minimum. Note that the use of large τ beginning the possible to [Figure] [Figure]

Original: Note that it is the backtracking procedure that allows to use large τ in the beginning of the algorithm.
Revised: [Figure] C ONVERGENCE OF DPIR (Z HANG ET AL . (2021)) In this section, we illustrate that, contrary to our method, the DPIR algorithm is not guaranteed to converge and can even easily diverge.

Original: [Equation]
Revised: In Figure 15, we plot the convergence curves of both DPIR and our GS-PnP when deblurring the “starﬁsh” image degraded with a motion kernel and ν = 0 . In the original DPIR paper Zhang et al. (2021), only 8 iterations are used with decreasing τ and σ . More precisely, σ decreases uniformly in log-scale from 49 to the input noise level ν , and τ is set proportional to σ 2 . In order to study the asymptotic behaviour of the method, we propose two strategies to run DPIR with 1000 iterations:

Original: H DRUN ET light ARCHITECTURE{+(i) Decreasing σ from 49 to ν over 1000 iterations instead of 8 (Figure 15, row 1). (ii) Decreasing σ from 49 to ν in 8 iterations, and then keep the last values of σ and τ for the the remaining iterations (Figure 15, row 2). As illustrated by the plot of (cid:80) i ≤ k || x i +1 − x i || 
+}
Revised: {+(i) Decreasing σ from 49 to ν over 1000 iterations instead of 8 (Figure 15, row 1). (ii) Decreasing σ from 49 to ν in 8 iterations, and then keep the last values of σ and τ for the the remaining iterations (Figure 15, row 2). As illustrated by the plot of (cid:80) i ≤ k || x i +1 − x i || 
+}

Revised: (third column), DPIR fails to converge with both strategies, even if the residual || x k +1 − x k || 2 tends to decrease with the second strategy.

Revised: This divergence also involves a loss of restoration performance in terms of PSNR (ﬁrst column).

Revised: On the other hand, as theoretically shown in this paper, the residual || x k +1 − x k || 2 with GS-PnP tends to(reaches ∼ 10 − 13 before the activation of backtracking, versus ∼ 10 − 4 for DPIR) and its series converges.

Revised: [Figure]

Revised: convergence of GS-PnP when deblurring the “starﬁsh” image.

Revised: The two ﬁrst rows display results obtained with DPIR with two different strategies used for decreasing σ : in the ﬁrst row, σ is decreased from 49 to ν over 1000 iterations; in the second row, σ is decreased in 8 iterations and then kept ﬁxed for the remaining iterations.