Statistical image recovery: A message-passing perspective

We review MAP and MMSE-based approaches to image recovery and their implementation via generalized approximate messagepassing (GAMP), highlighting recent results on GAMP convergence for general measurement operators. We consider the recovery of image x ∈ C from noisy outputs y ∈ C of known linear measurement operator Φ ∈ C . The “statistical” approach to image recovery models the image x as a realization of random x ∼ px and the measurements as a realization of random y whose statistics are governed by a likelihood function of the form py|z(y|Φx). Here, py|z is the pdf of y conditioned on the (hidden) transform outputs z = Φx and x is a hypothesis of the image. For clarity, we denote random quantities in san-serif font. In the maximum a posteriori (MAP) approach to statistical image recovery, one computes the most probable estimate of x given y, i.e., xMAP = argmaxx px|y(x|y) = argmaxx py|x(y|x)px(x)/py(y) = argmin x { − log py|z(y|Φx)− log px(x) } (1) which can be interpreted (from a non-statistical viewpoint) as regularized loss minimization, i.e., xMAP = argminx { L(Φx) +R(x) } (2) using the loss L(z) , − log py|z(y|z) and regularization R(x) , − log px(x). By choosing py|z and px so that both L(·) and R(·) are convex, one can readily apply convex optimization algorithms to the image recovery problem. In image recovery, it is popular to use regularizations of the form R(x) = ‖Ωx‖1 for a given matrix Ω. In the minimum mean-squared error (MMSE) approach to statistical image recovery, the objective is to compute xMMSE = E{x |y =y} = ∫ CN x px|y(x|y)dx, (3) with the hope of mean-square optimal performance. Unfortunately, the high-dimensional integral in (3) is computable in closed-form for only a very narrow class of priors and likelihoods (e.g., Gaussian) and even then may require the inversion of a very large matrix. For problems with separable loss and regularization, i.e., L(ẑ) = ∑M i=1 Li(ẑi) and R(x) = ∑N j=1 Rj(xj), (4) a computationally efficient inference methodology that supports either MAP or MMSE recovery was recently proposed under the name of “generalized approximate message passing” (GAMP) [1]. GAMP is an extension of the AMP algorithm [2] from quadratic loss (i.e., Li(ẑi) = 1 νw (yi − ẑi) 2 for some ν > 0) to generic loss Li(·), as needed for phase retrieval, Poisson noise, or quantized measurements. Interestingly, the behavior of GAMP for large i.i. d Φ is rigorously characterized by a state evolution whose fixed points, when unique, are MAP or MMSE optimal [3]. Still, important questions remain about the convergence of GAMP for generic Φ, and whether GAMP can be applied to non-separable regularizers like ‖Ωx‖1, which are commonly used in image recovery. ∗With support from NSF grants CCF-1018368 and CCF-1218754. In this talk, we review recent results on the convergence of GAMP for generic Φ. First, for any Φ, we recall that the fixed points of MAP-GAMP are known to coincide with the critical points of the optimization (2) [4]. Meanwhile, the fixed points of MMSE-GAMP are known to coincide with the critical points of the optimization [4] (fx , fz) = argminbx ,bz J(bx , bz) s.t. E{z|bz}=ΦE{x |bx} (5) J(bx , bz) , D ( bx‖px ) +D ( bz‖py|zZ −1+H ( bz ;ν p ) , (6) where J is a high-dimensional approximation of the Bethe freeenergy [5]. In (6), bx(x) = ∏ j bxj (xj) and bz(z) = ∏ i bzi(zi) are pdfs, D(·‖·) denotes KL divergence, and H(bz ;ν ) , ∑M i=1 var{zi|bzi}/ν p i + lnπν p i for ν p i = ∑N j=1 |Φij | 2 var{xj |bxj}. But these fixed points don’t tell the whole story, because GAMP may diverge. For quadratic Li(·) and Rj(·), however, the convergence of GAMP has been fully characterized, and global convergence can be ensured by “damping” [6]. Damping can also be used to ensure local convergence under strictly convex Li(·) and Rj(·) [6]. We also review recent results on connections between GAMP and convex optimization algorithms. For example, with MAP-GAMP, the variable updates coincide with those of the primal-dual hybrid gradient (PDHG) approach to (2) while the stepsizes are adapted in accordance with the local cost [6]. Meanwhile, with MMSE-GAMP, the mean updates coincide with an application of PDHG to (5) under a local convexification of the augmented Lagrangian, while the variance updates adapt that local convexification. Finally, we describe a recent variant on MMSE-GAMP that guarantees global convergence with generic Φ for strictly convex F,G with bounded derivatives. Finally, we describe how GAMP can be configured to use “analysis” non-separable regularizers R(x) = ∑D d=1 Gd([Ωx]d) [7].