Title: Block Coordinate Plug-and-Play Methods for Blind Inverse Problems
Venue: NeurIPS 2023
Accepted: yes
Keywords: inverse problems

plug-and-play priors

computational imaging

nonconvex optimization
Abstract: Plug-and-play (PnP) prior is a well-known class of methods for solving imaging inverse problems by computing fixed-points of operators combining physical measurement models and learned image denoisers. While PnP methods have been extensively used for image recovery with known measurement operators, there is little work on PnP for solving blind inverse problems. We address this gap by presenting a new block-coordinate PnP (BC-PnP) method that efficiently solves this joint estimation problem by introducing learned denoisers as priors on both the unknown image and the unknown measurement operator. We present a new convergence theory for BC-PnP compatible with blind inverse problems by considering nonconvex data-fidelity terms and expansive denoisers. Our theory analyzes the convergence of BC-PnP to a stationary point of an implicit function associated with an approximate minimum mean-squared error (MMSE) denoiser. We numerically validate our method on two blind inverse problems: automatic coil sensitivity estimation in magnetic resonance imaging (MRI) and blind image deblurring. Our results show that BC-PnP provides an efficient and principled framework for using denoisers as PnP priors for jointly estimating measurement operators and images.

Block Coordinate Plug-and-Play Methods for Blind Inverse Problems

Abstract

Plug-and-play (PnP) prior is a well-known class of methods for solving imaging inverse problems by computing fixed-points of operators combining physical measurement models and learned image denoisers. While PnP methods have been extensively used for image recovery with known measurement operators, there is little work on PnP for solving blind inverse problems. We address this gap by presenting a new block-coordinate PnP (BC-PnP) method that efficiently solves this joint estimation problem by introducing learned denoisers as priors on both the unknown image and the unknown measurement operator. We present a new convergence theory for BC-PnP compatible with blind inverse problems by considering nonconvex data-fidelity terms and expansive denoisers. Our theory analyzes the convergence of BC-PnP to a stationary point of an implicit function associated with an approximate minimum mean-squared error (MMSE) denoiser. We numerically validate our method on two blind inverse problems: automatic coil sensitivity estimation in magnetic resonance imaging (MRI) and blind image deblurring. Our results show that BC-PnP provides an efficient and principled framework for using denoisers as PnP priors for jointly estimating measurement operators and images.

1 Introduction

Many problems in computational imaging, biomedical imaging, and computer vision can be formulated as inverse problems involving the recovery of high-quality images from low-quality observations. Imaging inverse problems are generally ill-posed, which means that multiple plausible clean images could lead to the same observation. It is thus common to introduce prior models on the desired images. While the literature on prior modeling of images is vast, current methods are often based on deep learning (DL) , where a deep model is trained to map observations to images [1-3].

Plug-and-play (PnP) priors [4, 5] is one of the most widely-used DL frameworks for solving imaging inverse problems. PnP methods circumvent the need to explicitly describe the full probability density of images by specifying image priors using image denoisers. The integration of state-of-the-art deep denoisers with physical measurement models within PnP has been shown to be effective in a number of inverse problems, including image super-resolution, phase retrieval, microscopy, and

medical imaging [6-13] (see also recent reviews [14,15]). Practical success of PnP has also motivated novel extensions, theoretical analyses, statistical interpretations, as well as connections to related approaches such as score matching and diffusion models [16-29].

Despite the rich literature on PnP, the existing work on the topic has primarily focused on the problem of image recovery where the measurement operator is known exactly. There is little work on PnP for blind inverse problems, where both the image and the measurement operator are unknown. This form of blind inverse problems are ubiquitous in computational imaging with well-known applications such as blind deblurring [30] and parallel magnetic resonance imaging (MRI) [31]. In this paper, we address this gap by developing a new PnP approach that uses denoisers as priors over both the unknown measurement model and the unknown image, and efficiently solves the joint estimation task as a block-coordinate PnP (BC-PnP) method. While a variant of BC-PnP was proposed in the recent paper [21], it was never used for jointly estimating the images and the measurement operators. Additionally, the convergence theory in [21] is inadequate for blind inverse problems since it assumes convex data-fidelity terms and nonexpansive denoisers. We present a new convergence analysis applicable to nonconvex data-fidelity terms and expansive denoisers. Our theoretical analysis provides explicit error bounds on the convergence of BC-PnP for approximate minimum mean squared error (MMSE) denoisers under a set of clearly specified assumptions. We show the practical relevance of BC-PnP by solving joint estimation problems in blind deblurring and accelerated parallel MRI. Our numerical results show the potential of denoisers to act as PnP priors over the measurement operators as well as images. Our work thus addresses a gap in the current PnP literature by providing a new efficient and principled framework applicable to a wide variety of blind imaging inverse problems.

All proofs and some details that have been omitted for space appear in the supplementary material.

2 Background

Inverse Problems. Many imaging problems can be formulated as inverse problems where the goal is to estimate an unknown image x ∈ R n from its degraded observation y = Ax + e , where A ∈ R m × n is a measurement operator and e ∈ R m is the noise. A common approach for solving inverse problems is based on formulating an optimization problem

where g is the data-fidelity term that quantifies consistency with the observation y and h is the regularizer that infuses a prior on x . For example, a widely-used data-fidelity term and regularizer in computational imaging are the least-squares g ( x ) = 1 2 ‖ Ax -y ‖ 2 2 and the total variation (TV) functions h ( x ) = τ ‖ Dx ‖ 1 , where D is the image gradient, and τ > 0 a regularization parameter.

The traditional inverse problem formulations assume that the measurement operator A is known exactly. However, in many applications, it is more practical to model the measurement operator as A ( θ ) , where θ ∈ R p are unknown parameters to be estimated jointly with x . This form of inverse problems are often referred to as blind inverse problems and arise in a wide-variety of applications, including pralellel MRI [32-37], blind deblurring [30,38,39], and computed tomography [40-43].

DL. There is a growing interest in DL for solving imaging inverse problems [1-3]. Instead of explicitly defining a regularizer, DL approaches for solving inverse problems learn a mapping from the measurements to the desired image by training a convolutional neural network (CNN) to perform regularized inversion [44-48]. Model-based DL (MBDL) has emerged as powerful DL framework for inverse problems that combines the knowledge of the measurement operator with an image prior specified by a CNN (see reviews [3, 49]). The literature of MBDL is vast, but some wellknown examples include PnP, regularization by denoising (RED), deep unfolding (DU), compressed sensing using generative models (CSGM), and deep equilibrium models (DEQ) [50-54]. All these approaches come with different trade-offs in terms of imaging performance, computational and memory complexity, flexibility, need for supervision, and theoretical understanding.

The literature on DL approaches for blind inverse problems is broad, with many specialized methods developed for different applications. While an in-depth review would be impractical for this paper, we mention several representative approaches adopted in prior work. The direct application of DL to predict the measurement operator from the observation was explored in [55, 56]. Deep image prior (DIP) was used as a learning-free prior to regularize the image and the measurement operator

in [57, 58]. Generative models, including both GANs and diffusion models, have been explored as regularizers for blind inverse problems in [59, 60]. Other work considered the use of a dedicated neural network to predict the parameters of the measurement operator, adoption of model adaptation strategies, and development of autocalibration methods based on optimization [34-37,61-64].

PnP. PnP [4, 5] is one of the most popular MBDL approaches based on using deep denoisers as imaging priors (see also recent reviews [14,15]). For example, the proximal gradient method variant of PnP (referred to as PnP-ISTA in this paper) can be formulated as a fixed-point iteration [65]

where D σ is a denoiser with a parameter σ > 0 for controlling its strength and γ > 0 is a step-size. The theoretical convergence of PnP-ISTA has been explored for convex functions g using monotone operator theory [20,22] as well as for nonconvex functions based on interpreting the denoiser as a MMSE estimator [23]. The analysis in this paper builds on the convergence theory in [23] that uses an elegant formulation by Gribonval [66] establishing a direct link between MMSE estimation and regularized inversion. Many variants of PnP have been developed over the past few years [6-12], which has motivated an extensive research on its theoretical properties [16,18,20,22,23,27-29,67-70].

Block coordinate regularization by denoising (BC-RED) is a recent PnP variant for solving large-scale inverse problems by updating along a subset of coordinates at every iteration [21]. BC-RED is based on regularization by denoising (RED), another well-known variant of PnP that seeks to formulate an explicit regularizer for a given image denoiser [17,19]. BC-RED was applied to several non-blind inverse problems and was theoretically analyzed for convex data-fidelity terms.

PnP was extended to blind deblurring in [71,72] by considering an additional prior on blur kernels that promotes sparse and nonegative solutions. PnP was also applied to holography with unknown phase errors by using the Gaussian Markov random field model as the prior for the phase errors [73]. Calibrated RED (Cal-RED) [43] is a recent related extension of RED that calibrates the measurement operator during RED reconstruction by combining the traditional RED updates over an unknown image with a gradient descent over the unknown parameters of the measurement operator. However, this prior work does not leverage any learned priors for the measurement operator and does not provide any theoretical analysis.

Our contributions. (1) Our first contribution is in the use of learned deep denoisers for regularizing the measurement operators within PnP. While the idea of calibration within PnP was introduced in [43], denoisers were not used as priors for measurement operators. (2) Our second contribution is the application of BC-PnP as an efficient method for jointly estimating the unknown image and the measurement operator. While BC-RED was introduced in [21] as a block-coordinate variant of PnP, the method was used for solving non-blind inverse problems by using patch-based image denoisers. (3) Our third contribution is a new convergence theory for BC-PnP for the sequential and random block-selection strategies under approximate MMSE denoisers. Our analysis does not assume convex data-fidelity terms, which makes it compatible with blind inverse problems. Our analysis can be seen as an extension of [23] to block-coordinate updates and approximate MMSE denoisers. (4) Our fourth contribution is the implementation of BC-PnP using learned deep denoisers as priors for two distinct blind inverse problems: blind deblurring and auto-calibrated parallel MRI. Our code-which we share publicly-shows the potential of learning deep denoisers over measurement operators and using them for jointly estimating the uknown image and the uknown measurement operator.

3 Block Coordinate Plug-and-Play Method

We propose to efficiently solve blind inverse problems by using a block-coordinate PnP method, where each block represents one group of unknown variables (images, measurement operators, etc). The novelty of our work relative to [21] is in solving blind inverse problems by introducing learned priors on both the unknown image and the uknown measurement operator. Additionally, unlike [21], our work proposes a fully nonconvex formulation that is more applicable to blind inverse problems.

Consider the decomposition of a vector x ∈ R n into b ≥ 1 blocks

Algorithm 1 Block Coordinate Plug-and-Play Method (BC-PnP)

1: input: initial value x 0 ∈ R n , parameters σ ∈ R b + , and step-size γ > 0 .

3: Choose an index i k ∈ { 1 , · · · , b }

2: for k = 1 , 2 , 3 , · · · do

4: x k ← x k -1 -γ U i k G i k ( x k -1 )

5: end for

For each i ∈ { 1 , · · · , b } , we define a matrix U i ∈ R n × n i that injects a vector in R n i into R n and its transpose U T i that extracts the i th block from a vector in R n . For any x ∈ R n , we have

Note that (4) directly implies the norm preservation ‖ x ‖ 2 2 = ‖ x 1 ‖ 2 2 + · · · + ‖ x b ‖ 2 2 for any x ∈ R n . We are interested in a block-coordinate algorithm that uses only a subset of operator outputs corresponding to coordinates in some block i ∈ { 1 , · · · , b } . Hence, for an operator G : R n → R n , we define the block-coordinate operator G i : R n → R n i as

We are in-particular interested in two operators: (a) the gradient ∇ g ( x ) = ( ∇ 1 g ( x ) , · · · , ∇ b g ( x )) of the data-fidelity term g and (b) the denoiser D σ ( x ) = ( D σ 1 ( x 1 ) , · · · , D σ b ( x b )) , where the vector σ = ( σ 1 , · · · , σ b ) ∈ R b + consists of parameters for controling the strength of each block denoiser. Note how the denoiser acts in a separable fashion across different blocks.

When b = 1 , we have U 1 = U T 1 = I and BC-PnP reduces to the conventional PnP-ISTA [23,65]. When b > 1 , we have at least two blocks with BC-PnP updating only one block at a time

/negationslash

As with any coordinate descent method (see [74] for a review), BC-PnP can be implemented using different block selection strategies. One common strategy is to simply update blocks sequentially as i k = 1 + mod ( k -1 , b ) , where mod ( · ) denotes the modulo operator. An alternative is to proceed in epochs of b consecutive iterations, where at the start of each epoch the set { 1 , · · · , b } is reshuffled, and i k is then selected consecutively from this ordered set. Finally, one can adopt a fully randomized strategy where indices i k are selected as i.i.d. random variables distributed uniformly over { 1 , · · · , b } .

Throughout this work, we will assume that each denoiser D σ i is an approximate MMSE estimator for the following AWGN denoising problem

where i ∈ { 1 , · · · , b } and z i ∈ R n i . We rely only on an approximation of the MMSE estimator of x i given z i , since the exact MMSE denoiser corresponds to the generally intractable posterior mean

Approximate MMSE denoisers are a useful model for denoisers due to the use of the MSE loss

for training deep denoisers, as well as the optimality of MMSE denoisers with respect to widely used image-quality metrics such as signal-to-noise ratio (SNR).

As a simple illustration of the generality of BC-PnP, consider b = 2 with the least-squares objective

where v ∈ R n 1 denotes the unknown image and θ ∈ R n 2 denotes the unknown parameters of the measurement operator. BC-PnP can then be implemented by first pre-training a dedicated AWGN denoiser D σ i for each block i and using it as a prior within Algorithm 1. It is also worth noting that the functions g in (10) is nonconvex with respect to the variable x ∈ R n . In the next section, we present the full convergence analysis of BC-PnP without any convexity assumptions on g and nonexpansiveness assumptions on the denoiser D σ .

4 Convergence Analysis of BC-PnP

In this section, we present two new theoretical convergence results for BC-PnP. We first discuss its convergence under the sequential updates and then under fully random updates. It is worth mentioning that BC-RED with fully random updates was theoretically analyzed in [21]. The novelty of our analysis here lies in that it allows for nonconvex functions g and expansive denoisers D σ i . The nonconvexity of g is essential since most data-fidelity terms used for blind inverse problems are nonconvex. On the other hand, by allowing expansive D σ i , our analysis avoids the need for the spectral normalization techniques that were previously suggested for PnP methods [21,22].

In the following, we will denote as D ∗ σ := ( D ∗ σ 1 , · · · , D ∗ σ b ) the exact MMSE denoiser in (8). Our analysis will require five assumptions that will serve as sufficient conditions for our theorems.

Assumption 1. The blocks x i are independent with non-degenerate priors p x i over R n i .

As a reminder, a probability distribution p x i is degenerate over R n i , if it is supported on a space of lower dimensions than n i . Assumption 1 is required for establishing an explicit link between the MMSE denoiser (8) and the following regularizer (see also [23,66] for additional background)

where each function h i is defined as (see the derivation in Section D.2 of the supplement)

where γ > 0 is the step size, ( D ∗ σ i ) -1 : Im ( D ∗ σ i ) → R n i is the inverse mapping, which is well defined and smooth over Im ( D ∗ σ i ) , and h σ i ( · ) := -log( p z i ( · )) , where p z i is the probability distribution over the AWGN corrupted observations (7). Note that the function h i is smooth for any x i ∈ Im ( D ∗ σ i ) , which is the consequence of the smoothness of both ( D ∗ σ i ) -1 and h σ i .

Assumption 2. The function g is continuously differentiable and ∇ g is Lipschitz continuous with constant L > 0 . Additionally, each block gradient ∇ i g is block Lipschitz continuous with constant L i > 0 . We define the largest block Lipschitz constant as L max := max { L 1 , · · · , L b } .

Lipschitz continuity of the gradient ∇ g is a standard assumption in the context of imaging inverse problems. Note that we always have the relationship ( L/b ) ≤ L max ≤ L (see Section 3.2 in [74]).

Assumption 3. The explicit data-fidelity term and the implicit regularizer are bounded from below

Assumption 3 implies that there exists f ∗ > -∞ such that f ( x ) ≥ f ∗ for all x ∈ R n . Since Assumptions 1-3 correspond to the standard assumptions used in the literature, they are broadly satisfied in the context of inverse problems.

Our analysis assumes that at every iteration, BC-PnP uses inexact MMSE denoisers on each block. While there are several ways to specify the nature of 'inexactness,' we consider the case where at every iteration k of BC-PnP the distance of the output of D σ i to D ∗ σ i is bounded by a constant ε k .

Assumption 4. Each block denoiser D σ i in D σ satisfies

where D ∗ σ i is given in (8) and z k i = x k -1 i -γ ∇ i g ( x k -1 ) .

For convenience, we will define quantities ε 2 := max { ε 2 1 , ε 2 2 , · · · } and ε 2 t := (1 /t ) ( ε 2 1 + · · · + ε 2 t ) that correspond to the largest and the mean squared-distances between the inexact and exact denoisers. Assumption 4 states that the error of the approximate MMSE denoiser used for inference is bounded relative to the exact MMSE denoiser, which is reasonable when the approximate MMSE denoiser is a CNN trained to minimize the MSE.

It has been shown in the prior work [23, 66] that the function h in (11) is infinitely continuously differentiable over Im ( D ∗ σ ) . Our analysis requires the extension of the region where h is smooth to include the range of the approximate MMSE denoiser, which is the goal of our next assumption.

Assumption 5. Each regularizer h i in (12) associated with the MMSE denoiser (8) is continously differentiable and has a Lipschitz continuous gradient with constant M i > 0 over the set

We will define M max := max { M 1 , · · · , M b } to be the largest Lipschitz constant and Im ε ( D ∗ σ ) := { x ∈ R n : x i ∈ Im ε ( D ∗ σ i ) , i ∈ { 1 , . . . , b }} to be the set over which h is smooth. Assumption 5 expands the region where the regularizer associated with the exact MMSE denoiser is smooth by including the range of the approximate MMSE denoiser. For example, this assumption is automatically true when the exact and approximate MMSE denoisers have the same range, which is reasonable when the approximate MMSE denoiser is trained to imitate the exact one.

Our first theoretical result considers the sequential updates, where at each iteration, i k is selected as i k = 1 + mod ( k -1 , b ) with mod ( · ) being the modulo operator. We can then express any iterate ib produced by BC-PnP for i ≥ 1 as

Note that x ib ∈ Im ε ( D ∗ σ ) since each block is an output of the denoiser. We prove the following result.

Theorem 1. Run BC-PnP under Assumptions 1-5 using the sequential block selection and the step 0 < γ < 1 /L max . Then, we have

where C 1 > 0 and C 2 > 0 are iteration independent constants. If additionally the sequence of error terms { ε i } i ≥ 1 is square-summable, we have that ∇ f ( x tb ) → 0 as t → 0 .

Our second theoretical result considers fully random updates, where at each iteration, i k is selected as i.i.d. random variables distributed over { 1 , · · · , b } . In this setting, we analyze the convergence of BC-PnP in terms of the sequence { G ( x k ) } k ≥ 0 . Note that it is straightforward to verify that Zer ( G ) = Zer ( ∇ f ) , which makes this analysis meaningful. We prove the following result.

Theorem 2. Run BC-PnP under Assumptions 1-5 using the random i.i.d. block selection and the step 0 < γ < 1 /L max . Then, we have where D 1 > 0 and D 2 > 0 are iteration independent constants. If additionally the sequence of error terms { ε i } i ≥ 1 is square-summable, we have that G ( x t ) a . s . - - → 0 as t →∞ .

The expressions for the constants in Theorems 1 and 2 are given in the proofs. The theorems show that if the sequence of approximation errors is square-summable, BC-PnP asymptotically achieves a stationary point of f . On the other hand, if the sequence of approximation errors is not square-summable, the convergence is only up to an error term that depends on the average of the squared approximation errors. Both theorems can thus be viewed as more flexible alternatives for the convergence analysis in [21]. It is also worth mentioning that the theorems are interesting even when the denoiser errors are not square-summable, since they provide explicit error bounds on convergence. While the analysis in [21] assumes convex g and nonexpansive D σ , the analysis here does not require these two assumptions. It instead views the denoiser D σ as an approximation to the MMSE estimator D ∗ σ , where the approximation error is bounded by ε k at every iteration of BC-PnP. This view is compatible with denoisers trained to minimize the MSE loss (9).

0

250

500

10

-

1

10

-

2

10

-

3

10

-

4

||

x

k

-

x

k

-

1

||

2

2

sequential

iid

0

250

500

10

-

2

10

-

1

||

θ

k

-

θ

k

-

1

||

2

2

0

250

500

0

.

45

0

.

35

0

.

25

RMSE

(

x

)

0

250

500

0

.

95

0

.

85

0

.

75

SSIM

(

x

)

0

250

500

0

.

335

0

.

340

0

.

345

0

.

350

0

.

355

RMSE

(

θ

)

5 Numerical Validation

We numerically validate BC-PnP on two blind inverse problems: (a) compressed sensing parallel MRI (CS-PMRI) with automatic coil sensitivity map (CSM) estimation and (b) blind image deblurring . We adopt the traditional /lscript 2 -norm loss in (10) as the data-fidelity term for both problems. We will use x to denote the unknown image and θ to denote the unknown parameters of the measurement operator. We use the relative root mean squared error (RMSE) and structural similarity index (SSIM) as quantitative metrics to evaluate the performance.

We experimented with several ablated variants of BC-PnP, including PnP, PnP-GD θ , and PnPoracle θ . PnP and PnP-oracle θ denote basic variants of PnP that use pre-estimated and ground truth measurement operators, respectively. PnP-GD θ is a variant of PnP based on [43], where θ is estimated without any DL prior. It is worth noting that PnP-oracle θ is provided as an idealized reference in our experiment. As discussed in the following subsections, we also compare BC-PnP against several widely-used baseline methods specific to CS-PMRI and blind image deblurring.

5.1 Compressed Sensing Parallel MRI

The measurement operator of CS-PMRI consists of complex measurement operators A ( θ ) ∈ C m × n that depend on unknown CSMs { θ i } in C n . Each sub-measurement operator can be parameterized as A i ( θ i ) = PF diag ( θ i ) , where F is the Fourier transform, P ∈ R m × n is the sampling operator, and diag ( θ i ) forms a matrix by placing θ i on its diagonal. We used T2-weighted MR brain acquisitions of 165 subjects obtained from the validation set of the fastMRI dataset [75] as the the fully sampled measurement for simulating measurements. We obtained reference θ i from the fully sampled measurements using ESPIRiT [76]. These 165 subjects were split into 145, 10, and 10 for training, validation, and testing, respectively. BC-PnP and baseline methods were tested on 10 2D slices, randomly selected from the testing subjects. We followed [75] to retrospectively undersample the fully sampled data using 1D Cartesian equispaced sampling masks with 10% auto-calibration signal (ACS) [76] lines. We conducted our experiments for acceleration factors R = 6 and 8 . We adopted DRUNet [12] as the architectures of D σ for training both the image and CSM denoisers. BC-PnP and its ablated variants are initialized using CSMs θ 0 pre-estimated using ESPIRiT [76] and images x 0 ← A ( θ 0 ) H y , where A H denotes the Hermitian transpose of A .

We considered several baseline methods, including ENLIVE [35], ESPIRiT-TV [76], Unet [77], and ISTANet+ [51]. ENLIVE is an iterative algorithm that jointly estimates images and coil sensitivity profiles. ESPIRiT-TV is an iterative algorithm that applies TV reconstruction method in (1). Unet is trained to map raw measurements to desired ground truth without the knowledge of measurement operator. ISTANet+ denotes a widely-used DU architecture. We tested ESPIRiT-TV and ISTANet+ using CSMs pre-estimated using ESPIRiT.

Figure 1 illustrates the convergence behaviour of BC-PnP on the test set for the acceleration factor R = 8 . Figure 2 illustrates reconstruction results for the acceleration factor R = 6 . Table 1 summarizes the quantitative evaluation of BC-PnP relative to other PnP variants and the baseline methods. These results show that joint estimation can lead to significant improvements and that BC-PnP can perform as well as the idealized PnP-oracle θ that knowns the true measurement operator.

0.201 / 0.90

ESPIRiT-TV

0.113 / 0.95

0.112 / 0.95

PnP

0.084 / 0.97

BC-PnP (Ours)

0.064 / 0.97

PnP-oracle

θ

RMSE / SSIM

Ground-truth

0

.

00

0

.

25

0

.

50

5.2 Blind Image Deblurring

The measurement operator in blind image deblurring can be modeled as A ( θ ) x = θ ∗ x , where θ is the unknown blur kernel, x is the unknown image, and ∗ is the convolution. We randomly selected 10 testing ground truth image from CBSD68 [80] dataset. We generated 25 × 25 Gaussian kernels with σ = 10 1 . We tested the algorithms on 2 Gaussian kernels. We used a pre-trained image denoiser, as in the experimental setting of [12]. The kernel denoiser was trained on 10,000 generated kernels at several noise levels. We adopted DnCNN with 17 layers as the architectures of D σ for training kernel denoisers. BC-PnP and its ablated variants are initialized with the blur kernels θ 0 pre-estimated using Pan-DCP [39] and images x 0 ← A ( θ 0 ) T y .

We compared BC-PnP against several baseline methods, including Pan-DCP [39], SelfDeblur [58], DeblurGAN [78], USRNet [79]. Pan-DCP is an optimization-based method that jointly estimates image and blur kernel. SelfDeblur trains two deep image priors (DIP) [81] to jointly estimate the blur kernel and the image. DeblurGAN is a supervised learning-based method that lacks the capability for kernel estimation, but can reconstruct images via direct inference. USRNet is a DU baseline that was tested using blur kernel estimated from [39]. The results of DeblurGAN and USRNet are obtained by running the published code with the pre-trained weights.

Figure 3 illustrates the reconstruction results with a Gaussian kernel. Figure 3 demonstrates that BC-PnP can reconstruct the fine details of the snake skin, as highlighted by the white arrows, while both Pan-DCP and PnP produce smoother reconstructions. Additionally, BC-PnP generates a more accurate blur kernel compared to the ground truth kernel, whereas Pan-DCP and SelfDeblur yield blur kernels with artifacts. Table 2 presents the quantitative evaluation of the reconstruction results using

0.10/0.41/0.36

Pan-DCP

0.20/0.19/1.20

SelfDeblur

0.10/0.45/-

PnP

0.09/0.53/0.14

BC-PnP (Ours)

0.09/0.52/0.00

PnP-oracle

θ

RMSE

x

/SSIM/RMSE

θ

Ground-truth

0

.

000

0

.

375

0

.

750

a Gaussian kernel, indicating that BC-PnP outperforms the baseline methods and nearly matches the SSIM and RMSE values of PnP-oracle θ that is based on the ground truth blur kernel.

6 Conclusion

The work presented in this paper proposes a new BC-PnP method for jointly estimating unknown images and unknown measurement operators in blind inverse problems, presents its theoretical analysis in terms of convergence and accuracy, and applies the method to two well-known blind inverse problems. The proposed method and its theoretical analysis extend the recent work on PnP by introducing a learned prior on the unknown measurement operator, dropping the convexity assumptions on the data-fidelity term, and nonexpansiveness assumptions on the denoiser. The numerical validation of BC-PnP shows the improvements due to the use of learned priors on the measurement operator and the ability of the method to match the performance of the oracle PnP method that knows the true measurement operator. One conclusion of this work is the potential effectiveness of PnP for solving inverse problems where the unknown quantities are not only images.

Limitations

The work presented in this paper comes with several limitations. The proposed BC-PnP method is based on PnP, which means that its performance is inherently limited by the use of AWGN denoisers as priors. While denoisers provide a convenient, principled, and flexible mechanism to specify priors,

they are inherently self-supervised and their empirical performance can thus be suboptimal compared to priors trained in a supervised fashion for a specific inverse problem. PnP running over many iterations can also have higher computational complexity compared to some end-to-end alternatives, such as DU with a small number of steps. Our analysis is based on the assumption that the denoiser used for inference computes an approximation of the true MMSE denoiser. While this assumption is reasonable for deep denoisers trained using the MSE loss, it is not directly applicable to denoisers trained using other common loss functions, such as the /lscript 1 -norm or SSIM. Finally, as is common with most theoretical work, our analysis only holds when our assumptions are satisfied, which might limit its applicability in practice. Our future work will investigate ways to improve on the results presented here by exploring new PnP strategies for relaxing assumptions for convergence, considering end-to-end trained variants of BC-PnP based on DEQ [53,54], and exploring BC-PnP using explicit regularizers [26-28].

Broader Impact

The expected impact of this work is in the area of imaging inverse problems with potential applications to computational imaging. There is a growing interest in computational imaging to leverage pretrained deep models for estimating the unknown image as well as the unknown parameters of the imaging system. The ability to better address such problems can lead to new imaging tools for biomedical and scientific studies. While novel DL methods, such as the proposed BC-PnP approach, have the potential to enable new technological capabilities, they also come with a downside of being more complex and requiring higher-levels of technical sophistication. While our aim is to positively contribute to humanity, one can unfortunately envisage nonethical use of imaging technology.

Acknowledgments and Disclosure of Funding

Research presented in this article was supported in part by the NSF CCF-2043134. This work is also supported by the NIH R01EB032713, RF1AG082030, RF1NS116565, and R21NS127425.

References

M. T. McCann, K. H. Jin, and M. Unser, 'Convolutional neural networks for inverse problems in imaging: A review,' IEEE Signal Process. Mag. , vol. 34, no. 6, pp. 85-95, 2017.

A. Lucas, M. Iliadis, R. Molina, and A. K. Katsaggelos, 'Using deep neural networks for inverse problems in imaging: Beyond analytical methods,' IEEE Signal Process. Mag. , vol. 35, no. 1, pp. 20-36, Jan. 2018.

G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis, and R. Willett, 'Deep learning techniques for inverse problems in imaging,' IEEE J. Sel. Areas Inf. Theory , vol. 1, no. 1, pp. 39-56, May 2020.

S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg, 'Plug-and-play priors for model based reconstruction,' in Proc. IEEE Global Conf. Signal Process. and Inf. Process. , Austin, TX, USA, Dec. 3-5, 2013, pp. 945-948.

S. Sreehari, S. V. Venkatakrishnan, B. Wohlberg, G. T. Buzzard, L. F. Drummy, J. P. Simmons, and C. A. Bouman, 'Plug-and-play priors for bright field electron tomography and sparse interpolation,' IEEE Trans. Comput. Imaging , vol. 2, no. 4, pp. 408-423, Dec. 2016.

C. Metzler, P. Schniter, A. Veeraraghavan, and R. Baraniuk, 'prDeep: Robust phase retrieval with a flexible deep network,' in Proc. Int. Conf. Mach. Learn. , Stockholmsmässan, Stockholm Sweden, Jul. 10-15 2018, pp. 3501-3510.

K. Zhang, W. Zuo, S. Gu, and L. Zhang, 'Learning deep CNN denoiser prior for image restoration,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , Honolulu, USA, July 21-26, 2017, pp. 3929-3938.

T. Meinhardt, M. Moeller, C. Hazirbas, and D. Cremers, 'Learning proximal operators: Using denoising networks for regularizing inverse imaging problems,' in Proc. IEEE Int. Conf. Comp. Vis. , Venice, Italy, Oct. 22-29, 2017, pp. 1799-1808.

W. Dong, P. Wang, W. Yin, G. Shi, F. Wu, and X. Lu, 'Denoising prior driven deep neural network for image restoration,' IEEE Trans. Pattern Anal. Mach. Intell. , vol. 41, no. 10, pp. 2305-2318, Oct 2019.

K. Zhang, W. Zuo, and L. Zhang, 'Deep plug-and-play super-resolution for arbitrary blur kernels,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , Long Beach, CA, USA, June 16-20, 2019, pp. 1671-1681.

K. Wei, A. Aviles-Rivero, J. Liang, Y. Fu, C.-B. Schönlieb, and H. Huang, 'Tuning-free plug-and-play proximal algorithm for inverse imaging problems,' in Proc. Int. Conf. Mach. Learn. , 2020.

K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte, 'Plug-and-play image restoration with deep denoiser prior,' IEEE Trans. Patt. Anal. and Machine Intell. , 2022.

R. Liu, Y. Sun, J. Zhu, L. Tian, and U. S. Kamilov, 'Recovery of continuous 3D refractive index maps from discrete intensity-only measurements using neural fields,' Nat. Mach. Intell. , vol. 4, pp. 781-791, Sept. 2022.

R. Ahmad, C. A. Bouman, G. T. Buzzard, S. Chan, S. Liu, E. T. Reehorst, and P. Schniter, 'Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery,' IEEE Signal Process. Mag. , vol. 37, no. 1, pp. 105-116, 2020.

U. S. Kamilov, C. A. Bouman, G. T. Buzzard, and B. Wohlberg, 'Plug-and-play methods for integrating physical and learned models in computational imaging,' IEEE Signal Process. Mag. , vol. 40, no. 1, pp. 85-97, Jan. 2023.

S. H. Chan, X. Wang, and O. A. Elgendy, 'Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,' IEEE Trans. Comp. Imag. , vol. 3, no. 1, pp. 84-98, Mar. 2017.

Y. Romano, M. Elad, and P. Milanfar, 'The little engine that could: Regularization by denoising (RED),' SIAM J. Imaging Sci. , vol. 10, no. 4, pp. 1804-1844, 2017.

G. T. Buzzard, S. H. Chan, S. Sreehari, and C. A. Bouman, 'Plug-and-play unplugged: Optimization free reconstruction using consensus equilibrium,' SIAM J. Imaging Sci. , vol. 11, no. 3, pp. 2001-2020, Sep. 2018.

E. T. Reehorst and P. Schniter, 'Regularization by denoising: Clarifications and new interpretations,' IEEE Trans. Comput. Imag. , vol. 5, no. 1, pp. 52-67, Mar. 2019.

Y. Sun, B. Wohlberg, and U. S. Kamilov, 'An online plug-and-play algorithm for regularized image reconstruction,' IEEE Trans. Comput. Imag. , vol. 5, no. 3, pp. 395-408, Sept. 2019.

Y. Sun, J. Liu, and U. S. Kamilov, 'Block coordinate regularization by denoising,' in Proc. Adv. Neural Inf. Process. Syst. , Vancouver, BC, Canada, Dec. 2019, pp. 382-392.

Ernest K. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang, and W. Yin, 'Plug-and-play methods provably converge with properly trained denoisers,' in Proc. Int. Conf. Mach. Learn. , Long Beach, CA, USA, Jun. 09-15 2019, vol. 97, pp. 5546-5557.

X. Xu, Y. Sun, J. Liu, B. Wohlberg, and U. S. Kamilov, 'Provable convergence of plug-and-play priors with mmse denoisers,' IEEE Signal Process. Lett. , vol. 27, pp. 1280-1284, 2020.

J. Liu, S. Asif, B. Wohlberg, and U. S. Kamilov, 'Recovery analysis for plug-and-play priors using the restricted eigenvalue condition,' in Proc. Adv. Neural Inf. Process. Syst. , December 6-14, 2021, pp. 5921-5933.

Z. Kadkhodaie and E. P. Simoncelli, 'Stochastic solutions for linear inverse problems using the prior implicit in a denoiser,' in Proc. Adv. Neural Inf. Process. Syst. , December 6-14, 2021, pp. 13242-13254.

R. Cohen, Y. Blau, D. Freedman, and E. Rivlin, 'It has potential: Gradient-driven denoisers for convergent solutions to inverse problems,' in Proc. Adv. Neural Inf. Process. Syst. , December 6-14, 2021, pp. 18152-18164.

S. Hurault, A. Leclaire, and N. Papadakis, 'Gradient step denoiser for convergent plug-and-play,' in Proc. Int. Conf. Learn. Represent. , 2022.

S. Hurault, A. Leclaire, and N. Papadakis, 'Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization,' in Proc. Int. Conf. Mach. Learn. , 2022, pp. 9483-9505.

R. Laumont, V. De Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra, 'Bayesian imaging using plug & play priors: When Langevin meets Tweedie,' SIAM J. Imaging Sci. , vol. 15, no. 2, pp. 701-737, 2022.

P. Campisi and K. Egiazarian, Blind image deconvolution: theory and applications , CRC press, 2017.

J. A. Fessler, 'Optimization methods for magnetic resonance image reconstruction,' IEEE Signal Process. Mag. , vol. 1, no. 37, pp. 33-40, Jan. 2020.

L. Ying and J. Sheng, 'Joint image reconstruction and sensitivity estimation in SENSE (JSENSE),' Magn. Reson. Med. , vol. 57, no. 6, pp. 1196-1202, June 2007.

M. Uecker, T. Hohage, K. T. Block, and J. Frahm, 'Image reconstruction by regularized nonlinear inversion-Joint estimation of coil sensitivities and image content,' Magn. Reson. Med. , vol. 60, no. 3, pp. 674-682, Sept. 2008.

Y. Jun, H. Shin, T. Eo, and D. Hwang, 'Joint deep model-based MR image and coil sensitivity reconstruction network (joint-icnet) for fast MRI,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 2021, pp. 52705279.

H C. M Holme, S. Rosenzweig, F. Ong, R. N Wilke, M. Lustig, and M. Uecker, 'ENLIVE: An efficient nonlinear method for calibrationless and robust parallel imaging,' Scientific reports , vol. 9, no. 1, pp. 1-13, 2019.

M. Arvinte, S. Vishwanath, A. H. Tewfik, and J. I. Tamir, 'Deep J-Sense: Accelerated MRI reconstruction via unrolled alternating optimization,' in Proc. Med. Image. Comput. Comput. Assist. Intervent , Apr. 2021.

A. Sriram, J. Zbontar, T. Murrell, A. Defazio, C L. Zitnick, N. Yakubova, F. Knoll, and P. Johnson, 'End-to-end variational networks for accelerated MRI reconstruction,' in Proc. Med. Image. Comput. Comput. Assist. Intervent , 2020, pp. 64-73.

D. Kundur and D. Hatzinakos, 'Blind image deconvolution,' IEEE Signal Process. Mag. , vol. 13, no. 3, pp. 43-64, 1996.

J. Pan, D. Sun, H. Pfister, and M. Yang, 'Deblurring images via dark channel prior,' IEEE Trans. Pattern Anal. Mach. Intell. , vol. 40, no. 10, pp. 2315-2328, 2017.

E. Malhotra and A. Rajwade, 'Tomographic reconstruction from projections with unknown view angles exploiting moment-based relationships,' in Proc. IEEE Int. Conf. Image Proc. , 2016, pp. 1759-1763.

D. C Lee, P. F Hoffmann, D. L Kopperdahl, and T. M Keaveny, 'Phantomless calibration of ct scans for measurement of bmd and bone strength-inter-operator reanalysis precision,' Bone , vol. 103, pp. 325-333, 2017.

C. Cheng, Y. Ching, P. Ko, and Y. Hwu, 'Correction of center of rotation and projection angle in synchrotron x-ray computed tomography,' Scientific reports , vol. 8, no. 1, pp. 9884, 2018.

M. Xie, J. Liu, Y. Sun, W. Gan, B. Wohlberg, and U. S. Kamilov, 'Joint reconstruction and calibration using regularization by denoising with application to computed tomography,' in Proc. IEEE Int. Conf. Comp. Vis. Workshops , October 2021, pp. 4028-4037.

S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng, and D. Liang, 'Accelerating magnetic resonance imaging via deep learning,' in Proc. Int. Symp. Biomedical Imaging , April 2016, pp. 514-517.

K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, 'Deep convolutional neural network for inverse problems in imaging,' IEEE Trans. Image Process. , vol. 26, no. 9, pp. 4509-4522, Sep. 2017.

E. Kang, J. Min, and J. Ye, 'A deep convolutional neural network using directional wavelets for low-dose x-ray CT reconstruction,' Medical Physics , vol. 44, no. 10, pp. e360-e375, 2017.

H. Chen, Y. Zhang, M. K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou, and G. Wang, 'Low-dose CT with a residual encoder-decoder convolutional neural network,' IEEE Trans. Med. Imag. , vol. 36, no. 12, pp. 2524-2535, Dec. 2017.

X. Xu, J. Pan, Y. Zhang, and M. Yang, 'Motion Blur Kernel Estimation via Deep Learning,' IEEE Trans. on Image Process. , vol. 27, no. 1, pp. 194-205, Jan. 2018.

V. Monga, Y. Li, and Y. C. Eldar, 'Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,' IEEE Signal Process. Mag. , vol. 38, no. 2, pp. 18-44, Mar. 2021.

A. Bora, A. Jalal, E. Price, and A. G. Dimakis, 'Compressed sensing using generative priors,' in Proc. Int. Conf. Mach. Learn. , Sydney, Australia, Aug. 2017, pp. 537-546.

J. Zhang and B. Ghanem, 'ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 2018, pp. 1828-1837.

A. Hauptmann, F. Lucka, M. Betcke, N. Huynh, J. Adler, B. Cox, P. Beard, S. Ourselin, and S. Arridge, 'Model-based learning for accelerated, limited-view 3-d photoacoustic tomography,' IEEE Trans. Med. Imag. , vol. 37, no. 6, pp. 1382-1393, 2018.

D. Gilton, G. Ongie, and R. Willett, 'Deep equilibrium architectures for inverse problems in imaging,' IEEE Trans. Comput. Imag. , vol. 7, pp. 1123-1133, 2021.

J. Liu, X. Xu, W. Gan, S. Shoushtari, and U. S. Kamilov, 'Online deep equilibrium learning for regularization by denoising,' in Proc. Adv. Neural Inf. Process. Syst. , New Orleans, LA, 2022.

Y. Han and J. C. Ye, 'Framing U-Net via deep convolutional framelets: Application to sparse-view CT,' IEEE Trans. Med. Imag. , vol. 37, no. 6, pp. 1418-1429, 2018.

X. Peng, B. Sutton, F. Lam, and Z. Liang, 'Deepsense: Learning coil sensitivity functions for sense reconstruction using deep learning,' Magn. Reson. Med. , vol. 87, no. 4, pp. 1894-1902, 2022.

E. Bostan, R. Heckel, M. Chen, M. Kellman, and L. Waller, 'Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network,' Optica , vol. 7, no. 6, pp. 559-562, 2020.

D. Ren, K. Zhang, Q. Wang, Q. Hu, and W. Zuo, 'Neural blind deconvolution using deep priors,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 2020, pp. 3341-3350.

M. Asim, F. Shamshad, and A. Ahmed, 'Blind image deconvolution using deep generative priors,' IEEE Trans. Comput. Imaging , vol. 6, pp. 1493-1506, 2020.

H. Chung, J. Kim, S. Kim, and J. Ye, 'Parallel Diffusion Models of Operator and Image for Blind Inverse Problems,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 2023.

Y. Hu, W. Gan, C. Ying, T. Wang, C. Eldeniz, J. Liu, Y. Chen, H. An, and U. S Kamilov, 'SPICE: Self-supervised learning for mri with automatic coil sensitivity estimation,' arXiv:2210.02584 , 2022.

A. Gossard and P. Weiss, 'Training Adaptive Reconstruction Networks for Blind Inverse Problems,' arXiv:2202.11342 , 2022.

Y. Huang, E. Chouzenoux, and J.-C. Pesquet, 'Unrolled variational bayesian algorithm for image blind deconvolution,' IEEE Trans. Image Process. , vol. 32, pp. 430-445, 2022.

Y. Li, M. Tofighi, J. Geng, V. Monga, and Y. C Eldar, 'Efficient and interpretable deep blind image deblurring via algorithm unrolling,' IEEE Trans. Comput. Imaging , vol. 6, pp. 666-681, 2020.

U. S. Kamilov, H. Mansour, and B. Wohlberg, 'A plug-and-play priors approach for solving nonlinear imaging inverse problems,' IEEE Signal. Proc. Let. , vol. 24, no. 12, pp. 1872-1876, Dec. 2017.

R. Gribonval, 'Should penalized least squares regression be interpreted as maximum a posteriori estimation?,' IEEE Trans. Signal Process. , vol. 59, no. 5, pp. 2405-2410, May 2011.

T. Tirer and R. Giryes, 'Image restoration by iterative denoising and backward projections,' IEEE Trans. Image Process. , vol. 28, no. 3, pp. 1220-1234, Mar. 2019.

A. M. Teodoro, J. M. Bioucas-Dias, and M. A. T. Figueiredo, 'A convergent image fusion algorithm using scene-adapted Gaussian-mixture-based denoising,' IEEE Trans. Image Process. , vol. 28, no. 1, pp. 451-463, Jan. 2019.

Y. Sun, Z. Wu, B. Wohlberg, and U. S. Kamilov, 'Scalable plug-and-play ADMM with convergence guarantees,' IEEE Trans. Comput. Imag. , vol. 7, pp. 849-863, July 2021.

R. Cohen, M. Elad, and P. Milanfar, 'Regularization by denoising via fixed-point projection (red-pro),' SIAM J. Imaging Sci. , vol. 14, no. 3, pp. 1374-1406, 2021.

M. Ljubenovi´ c and M. A. Figueiredo, 'Blind image deblurring using class-adapted image priors,' in Proc. IEEE Int. Conf. Image Proc. IEEE, 2017, pp. 490-494.

M. Ljubenovi´ c and M. A. Figueiredo, 'Plug-and-play approach to class-adapted blind image deblurring,' Int. J. Doc. Anal. Recognit. , vol. 22, no. 2, pp. 79-97, 2019.

C. J Pellizzari, M. F Spencer, and C. A Bouman, 'Coherent plug-and-play: digital holographic imaging through atmospheric turbulence using model-based iterative reconstruction and convolutional neural networks,' IEEE Trans. Comput. Imaging , vol. 6, pp. 1607-1621, 2020.

S. J. Wright, 'Coordinate descent algorithms,' Math. Program. , vol. 151, no. 1, pp. 3-34, June 2015.

F. Knoll et al. , 'fastMRI: A publicly available raw k-space and DICOM dataset of knee images for accelerated MR image reconstruction using machine learning,' Radiology: Artificial Intelligence , vol. 2, no. 1, pp. e190007, 2020.

M. Uecker, P. Lai, M. J. Murphy, P. Virtue, M. Elad, J. M. Pauly, S. S. Vasanawala, and M. Lustig, 'ESPIRiT-an eigenvalue approach to autocalibrating parallel mri: where sense meets grappa,' Magn. Reson. Med. , vol. 71, no. 3, pp. 990-1001, 2014.

O. Ronneberger, P. Fischer, and T. Brox, 'U-Net: Convolutional networks for biomedical image segmentation,' in Proc. Med. Image. Comput. Comput. Assist. Intervent. , 2015, pp. 234-241.

O. Kupyn, T. Martyniuk, J. Wu, and Z. Wang, 'Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better,' in Proc. IEEE Int. Conf. Comput. Vis. , 2019, pp. 8878-8887.

K. Zhang, L. V. Gool, and R. Timofte, 'Deep unfolding network for image super-resolution,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , Jun. 2020, pp. 3217-3226.

D. Martin, C. Fowlkes, D. Tal, and J. Malik, 'A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,' in Proc. IEEE Int. Conf. Comp. Vis. , Vancouver, Canada, July 7-14, 2001, pp. 416-423.

D. Ulyanov, A. Vedaldi, and V. Lempitsky, 'Deep image prior,' in Proc. IEEE Conf. Computer Vision and Pattern Recognition , Salt Lake City, UT, USA, June 18-22, 2018, pp. 9446-9454.

R. Gribonval and P. Machart, 'Reconciling 'priors' & 'priors' without prejudice?,' in Proc. Adv. Neural Inf. Process. Syst. , Lake Tahoe, NV, USA, December 5-10, 2013, pp. 2193-2201.

D. P. Bertsekas, 'Incremental proximal methods for large scale convex optimization,' Math. Program. Ser. B , vol. 129, pp. 163-195, 2011.

J. Bolte, S. Sabach, and M. Teboulle, 'Proximal alternating linearized minimization for nonconvex and nonsmooth problems,' Math. Program. , vol. 146, pp. 459-494, 2013.

J. Mairal, 'Incremental majorization-minimization optimization with application to large-scale machine learning,' SIAM J. Optim. , vol. 25, no. 2, pp. 829-855, Jan. 2015.

E. K. Ryu and W. Yin, Large-Scale Convex Optimization: Algorithms and Analyses via Monotone Operators , Cambridge University Press, 2023.

A. Kazerouni, U. S. Kamilov, E. Bostan, and M. Unser, 'Bayesian denoising: From MAP to MMSE using consistent cycle spinning,' IEEE Signal Process. Lett. , vol. 20, no. 3, pp. 249-252, March 2013.

R. Gribonval and M. Nikolova, 'On Bayesian estimation and proximity operators,' Appl. Comput. Harmon. Anal. , vol. 50, pp. 49-72, Jan. 2021.

D. Kingma and J. Ba, 'Adam: A method for stochastic optimization,' in Proc. Int. Conf. on Learn. Represent. , 2015.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a block-coordinate optimization strategy for plug-and-play (PnP) methods to solve blind inverse problems in imaging. Empirical validation is performed with blind deblurring and parallel CS-MRI. Theoretical study on the convergence of the proposed BC-PnP was extended from the theoretical result of PnP with MMSE denoisers.\n\nSTRENGTHS:\n1. To the best of my knowledge, this is one of the first works to extend PnP methods to blind inverse problems\n\n2. Theory is sound and proves proper convergence properties of the proposed method.\n\n3. The method seems to be robust and generally applicable to inverse problems arising in imaging, as validated in the experiments.\n\nWEAKNESSES:\n1. I do not find much novelty in the theory of the paper. While the components are all sound, both components of the extension (extending to block coordinate update and extending to denoisers that are optimal only up to a constant error) seems to be quite trivial. I would appreciate it if the authors could elaborate on this point.\n\n2. BC-PnP seems to rely quite heavily on initialization for the imaging operators (i.e. sensitivity maps, blur kernels) that are close to ground truth. Such initialization strategy is not always possible, and for many cases even when it is possible, is often computationally demanding. Does BC-PnP converge properly also for cases when there is no such initialization strategy? Adding on this point, it might not be fair to compare other blind inverse problem solver that do not leverage any initialization strategy.\n\n3. For blind deblurring, it seems that the authors only tested on Gaussian blur kernels, which are known to be relatively easy to fit. Does the method scale to more complex blur kernels, such as in [1,2]?\n\n\n**Referneces**\n\n[1] Levin, Anat, et al. \"Understanding and evaluating blind deconvolution algorithms.\" 2009 IEEE conference on computer vision and pattern recognition. IEEE, 2009.\n\n[2] Chung, Hyungjin, et al. \"Parallel diffusion models of operator and image for blind inverse problems.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\nQUESTIONS:\nPlease see weaknesses\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new plug-and-play algorithm for solving blind inverse problems, i.e., where the forward sensing operator is not fully known. The method leverages pretrained denoisers for both the signals and the unknown parameters of the sensing model. A theoretical analysis of the proposed algorithm is presented, showing that under certain conditions on the denoisers and data fidelity functions, the algorithm is guaranteed to converge to a local minimizer.\n\nSTRENGTHS:\n- The paper presents a novel plug-and-play algorithm for blind inverse problems, with some theoretical guarantees under relatively mild assumptions.\n\n- The experimental results show a good performance with respect to other competing methods in two different blind inverse problems.\n\nWEAKNESSES:\nI find it a bit surprising that the PnP method outperfoms other unrolled algorithms trained for a specific inverse problem. Even the authors express that \"While denoisers provide a convenient, principled, and flexible mechanism to specify priors, they are inherently self-supervised and their empirical performance can thus be suboptimal compared to priors trained in a supervised fashion for a specific inverse problem\"\nI wonder if the comparison presented in the paper is fair, do the competing methods use architectures that have a similar expressive power than the DRUNet denoiser used for PnP? Moreover, there exist end-to-end architectures that can estimate both the forward operator parameters and the underlying image (e.g. \"Deep algorithm unrolling for blind image deblurring\" by Li et al.), which haven't been evaluated here.\n\nQUESTIONS:\n- What is the performance of the proposed method on the blind deblurring problem with non-isotropic kernels? It would be good to show an example in this setting.\n\nLIMITATIONS:\nThe paper discusses some of the limitations of the proposed method. However, there is no discussion related to the inherent large computational complexity of PnP methods, that require many more iterations than unrolled networks to reach convergence.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

comments: SUMMARY:
The paper generalizes the plug-and-play framework from single-block to a multi-block case by embedding unknown inverse operators. It provides convergence analysis for the proposed method and conducted various experiments on two imaging applications, i.e., parallel MRI and blind image deblurring. Numerical results are sufficiently convincing to show that the proposed method works better than the other comparable PnP methods in the blind measurement operator case.

STRENGTHS:
The paper reads well and is well-organized from theoretical discussions to numerical experiments. The proposed method is original with clear motivation of unknown inverse operators from the realistic setting. Thus, it may also shed some light on other imaging applications.

WEAKNESSES:
1. The computational cost of this framework is not analyzed or even mentioned. Likewise, the running times for all the experimental results could be listed and compared. Some other end-to-end methods may also be tested. 
2. Some minor notational confusion exists, e.g., lines 198-199. 
3. Theorems 1-2 require square-summable condition for the error sequences to ensure the convergence of gradients. Further discussions may be added about how this condition can be satisfied or checked in practice.

QUESTIONS:
In all the numerical experiments, are the assumptions 1-5 and conditions in Theorems 1-2 verified or checked? If yes, the details could be included in the supplement.

LIMITATIONS:
The authors have listed the limitations and provided some potential solutions in future works.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

comments: SUMMARY:
Develops a plug-and-play technique for blind (unknown forward model) imaging inverse problems. Key idea is to use two denoising algorithms: One as a prior on the image and one as a prior model on the measurement operator (e.g., blur kernel). Also analyzes the convergence of the algorithm. Successfully applies (in simulation) the proposed algorithm to MRI with unknown coil sensitive maps and deblurring with unknown blur kernel.

STRENGTHS:
Investigates important and understudied problem

Proposed technique is well-justified and seemingly effective

I like the paper's approach to framing the blind image reconstruction problem (equation (10)).

Easy to read

Proposed method seems generally applicable to lots of problems and opens up and opportunity for lots of follow-up work

Code will be shared publicly

WEAKNESSES:
Lacks comparisons with closest competitors. Particulary [60].

Missing related work: There's a line of work on PnP for holography with unknown phase errors (e.g., [A]) that may be worth mentioning.

[A] Pellizzari, Casey J., Mark F. Spencer, and Charles A. Bouman. "Coherent plug-and-play: digital holographic imaging through atmospheric turbulence using model-based iterative reconstruction and convolutional neural networks." IEEE Transactions on Computational Imaging 6 (2020): 1607-1621.

QUESTIONS:
Diffusion models are essentially denoisers by a different name, therefore a comparison with [60] seems obligatory. They are essentially solving the same problem (though the current method has more theory). How do they compare?

LIMITATIONS:
Adequately discussed
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-08