Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration
Authors: Guy Ohayon
Tomer Michaeli
Michael Elad
Venue: ICLR 2025 Poster (2025)
Keywords: Image restoration
blind face image restoration
inverse problems
perception-distortion tradeoff
image processing
computer vision
machine learning
generative models
optimal transport
rectified flow
flow matching
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose Posterior-Mean Rectified Flow (PMRF), a simple image restoration algorithm. Unlike previous works, our method approximates the optimal estimator that minimizes the MSE under a perfect perceptual index constraint.

Abstract: Photo-realistic image restoration algorithms are typically evaluated by distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures (e.g., FID, NIQE), where the desire is to attain the lowest possible distortion without compromising on perceptual quality. To achieve this goal, current methods commonly attempt to sample from the posterior distribution, or to optimize a weighted sum of a distortion loss (e.g., MSE) and a perceptual quality loss (e.g., GAN). Unlike previous works, this paper is concerned specifically with the *optimal* estimator that minimizes the MSE under a constraint of perfect perceptual index, namely where the distribution of the reconstructed images is equal to that of the ground-truth ones. A recent theoretical result shows that such an estimator can be constructed by optimally transporting the posterior mean prediction (MMSE estimate) to the distribution of the ground-truth images. Inspired by this result, we introduce Posterior-Mean Rectified Flow (PMRF), a simple yet highly effective algorithm that approximates this optimal estimator. In particular, PMRF first predicts the posterior mean, and then transports the result to a high-quality image using a rectified flow model that approximates the desired optimal transport map. We investigate the theoretical utility of PMRF and demonstrate that it consistently outperforms previous methods on a variety of image restoration tasks.

Published as a conference paper at ICLR 2025
POSTERIOR-MEAN RECTIFIED FLOW: TOWARDS MIN- IMUM MSE PHOTO-REALISTIC IMAGE RESTORATION
Guy Ohayon, Tomer Michaeli, Michael Elad
Technion - Israel Institute of Technology (ohayonguy@cs, tomer mmée,, elad@ os) wwwchiinn...ne.ecce ac.
ABSTRACT
Photo-realistic image restoration algorithms are typically evaluated by distortion measures (e.g., PSNR, SSIM) and by perceptual quality measures (e.g.. FID, NIQE), where the desire is to attain the lowest possible distortion without com- promising on perceptual quality. To achieve this goal, current methods commonly attempt to sample from the posterior distribution, to optimize a weighted sum of distortion loss (e.g., MSE) and perceptual quality loss (e.g., GAN). Un- like previous works, this paper concerned specifically with the optimal esti- mator that minimizes the MSE under a constraint perfect perceptual index, namely where the distribution of the reconstructed images is equal to that of the ground-truth ones. A recent theoretical result shows that such an estimator can be constructed by optimally transporting the posterior mean prediction (MMSE estimate) to the distribution of the ground-truth images. Inspired by this result, we introduce Posterior-Mert Rectified Flow (PMRF), a simple yet highly effec- tive algorithm that approximates this optimal estimator. In particular, PMRF first predicts the posterior mean, and then transports the result to high-quality image using rectified flow model that approximates the desired optimal transport map. We investigate the theoretical utility of PMRF and demonstrate that it consistently outperforms previous methods variety of image restoration tasks.
INTRODUCTION
Photo-realistic image restoration (PIR) is the task of reconstructing visually appealing images from de- graded measurements (e.g., noisy, blurry). This is
Distortion xlla X(IP]
x MMS
a long-standing research problem with diverse ap- plications in mobile photography, surveillance, re- mote sensing, medical imaging, and more. PIR
PX(y (posterior sampling)
No (our goal)
algorithms commonly evaluated by distortion
MMSE
Unattainable
8 (posterior mesan)
measures (e.g.. PSNR, SSIM (Wang et al. 2004).
LPIPS (Zhang al. 2018)), which quantify some type of discrepancy between the reconstructed im-
Perceptual index d(px.Pg)
ages and the ground-truth ones, and by perceptual quality measures (e.g.. FID (Heusel et al., (017). KID (Binkowski al. 2018). NIQE (Mittal al. 2013). NIMA (Talebi & Milanfar 2018)), which are intended to predict the extent which the recon- structions would look natural to human observers. Since distortion and perceptual quality are typically at odds with each other (Blau & Michaeli, 1018), the core challenge PIR is to achieve minimal distor- tion without sacrificing perceptual quality.
Figure 1: Illustration of the distortion perception tradeoff, where distortion is mea- sured by MSE. Many photo-realistic image restoration methods aim for posterior sam- pling. Theoretically, this approach achieves perfect perceptual index (Dx px) but its MSE is twice the MMSE. contrast, we aim for the estimator X, that minimizes the MSE under a perfect perceptual index con- straint (Eq. (3), which typically achieves smaller MSE than posterior sampling.
A common way to approach this task through pos- terior sampling (Bendel et al. 2023] Chung al.
2023 Daras et al.] 2024; Kawar et al. 202
2022
Man et al. 2023
Murata al. 2023
Ohayon et al.
2021 Saharia et al.
2022
2023 Song

---

Published as a conference paper at ICLR 2025
Ground-truth
Degraded
PMRF output
Ground-truth
Degraded
PMRF output
Figure 2:
Visual results of PMRF (our method) on the CelebA-Test blind face image restoration data set. Our algorithm produces sharp and visually appealing details while maintaining incredibly low distortion according to a variety of measures simultaneously. See Table
Table 1: Quantitative evaluation of state-of-the-art blind face image restoration algorithms on the CelebA-Test benchmark. Red, blue and green indicate the best, the second best, and the third best scores, respectively. Our method achieves the best FID, KID, PSNR and SSIM, and the second or third best scores in the rest of the perceptual quality and distortion measures. A visual comparison provided in Figure 2 and Figure 6 in the appendix.
Perceptual Quality
Distortion
Method
FID
KIDI
NIQEL
Precisiont
PSNRT
SSIM
LPIPS).
Degl.
LMD
DOT
100.2
0.0914
6.462
0.1600
21.32
0.6636
0.4756
43.87
2.876
RestoreFormer++
41.15
0.0290
4.187
0.6877
25.31
0.6703
0.3441
29.63
2.043
RestoreFormer
42.30
0.0301
4.405
0.7010
24.62
0.6460
0.3655
32.13
2.299
CodeFormer
53.16
0.0425
4.649
0.6940
25.15
0.6700
0.3432
37.28
2.470
VQFRv1
41.79
0.0297
3.693
0.6593
24.07
0.6446
0.3515
35.75
2.429
VQFRv2
46.77
0.0346
4.169
0.6590
23.23
0.6412
0.3624
44.38
3.053
GFPGAN
46.72
0.0350
4.415
0.6970
24.99
0.6774
0.3643
36.05
2.443
DiffBIR
59.06
0.0509
6.084
0.5643
25.39
0.6536
0.3878
32.94
2,006
DifFace
38.43
0.0258
4.288
0.7413
24.80
0.6726
0.3999
45.79
2.965
BFRffusion
41.53
0.0301
4.966
0.6623
26.21
0.6917
0.3619
30.98
1.992
PMRF (Ours)
37.46
0.0257
4.118
0.7073
26.37
0.7073
0.3470
30.67
2.030
et al. 2023 Wang et al. 2023a Zhu et al. 2023 Specifically, letting and Y denote the ran- dom vectors corresponding to the ground-truth image and its degraded measurement, respectively, posterior sampling generates a reconstruction X' by sampling from Pxjy (such that P Pxly). This solution is appealing as it theoretically guarantees perfect perceptual index" (fx Px)) In- terestingly, however, the Mean Squared Error (MSE) that this solution achieves is not the minimal possible under the perfect perceptual index constraint. Indeed, the MSE achieved by posterior sam- pling is precisely twice the Minimum MSE (MMSE) that can be achieved without constraint on the perceptual index (Blau Michaeli 2018). This while the minimal MSE achievable under perfect perceptual index constraint is typically strictly smaller (Blau & Michaeli, 2018 Freirich jct al. 2021), as illustrated Figure I We denote by X, the estimator that minimizes the MSE under perfect perceptual index constraint. Its formal definition is provided in Section 2.2
Another common way to solve PIR tasks is to train a model by minimizing a weighted sum of distortion loss (e.g.. MSE) and GAN loss (Goodfellow et al.) 2014 Gu al. 2022
Ledig
ct al. 2017 Wang et al. 2013 2021 2022 2023b Yang et al. 2021 Zhang et al.
202
Zhou
Formally, the perceptual index of X is defined the statistical divergence hetween py and px-
2

---

Published as a conference paper at ICLR 2025
al. 2022). As explained by Blau & Michaeli (2018), this is a principled way to traverse the distortion pereetptonn tradeoff, where the GAN loss coefficient acts as Lagrange multiplier that controls the desired perceptual index. Thus, in principle, one approximate Xo by selecting sufficiently large such coefficient. Despite the elegance of this approach, diffusion methods that aim for posterior sampling tend to perform better in practice, both in terms of distortion and perceptual quality (see Table D. implying that current GAN-based methods fail to approximate Xo. Such a shortcoming can be partially attributed to the fact that GANs are extremely difficult to optimize, especially when the GAN loss coefficient is significantly larger than that of the distortion loss.
In this paper, we propose Posterior- Mean Rectified Flow (PMRF), straightforward framework to directly approximate Xo. Interestingly. Freirich et al. (2021) proved that Xo can be constructed by first predicting the posterior mean X = E
and then optimally transporting the result to the ground-truth image distribution (see Section 2.2 for formal explanation). Motivated by this result, PMRF first approximates the posterior mean by using a model that minimizes the MSE between the reconstructed outputs and the ground-truth images. Then, we train a rectified flow model (Liu al.) 2023) to predict the direction of the straight path between corresponding pairs posterior mean predictions and ground-truth images. Given degraded measurement at test time, PMRF solves an ODE using such a flow model. with the posterior mean prediction set as the initial condition. As explain in Section 3 PMRF approximates the desired estimator Xo, aiming for solution that minimizes the MSE under a perfect perceptual index constraint.
Our paper is organized as follows. In Section 2 we provide the necessary background and set math- ematical notations. In Section 3 we describe our proposed method, and provide intuition via theo- retical results and a toy example with closed-form solutions. In Section 4 we discuss related work. In Section 2 we demonstrate the utility of PMRF on a variety of image restoration tasks, including denoising, reseerrrsolltion, inpainting, colorization, and blind restoration. We show that PMRF sets a new state-of-ther ohettr-artttttt on several benchmarks in the challenging blind face image restoration task, and is either on-par or outperforms previous frameworks the rest of the tasks. Finally, in Section we conclude our work and discuss its limitations.
BACKGROUND
We adopt the Bayesian perspective for solving inverse problems (Davison 2003 Kaipio & Som- ersalo, 2005), where a natural image x is regarded as realization of a random vector X with probability density function px. The degraded measurement y (e.g., a noisy or low-resolution im- age) is a realization of a random vector Y, which is related to X via the conditional probability density function Py Given a degraded measurement y, an image restoration algorithm generates prediction á by sampling from P (-ly), such that X adheres to the Markov chain X Y ->> X (i.e. X and X are statistically independent given Y).
2.1 DISTORTION AND PERCEPTUAL INDEX
Image restoration algorithms are typically evaluated by their average distortion (XA X)), where (zx, 2) is some distortion measure that quantifies the discrepancy between x and a, and the ex- pectation is taken over the joint distribution Px Common examples for A (xc, .) are the absolute error || - f l1, the squared error ||æ 2||2 and LPIPS (Zhang et al. 2018 Moreover, as the goal in PIR is to produce reconstructions that would look natural to humans, PIR algorithms are also evaluated by perceptual quality measures. The ideal way evaluate perceptual quality is assess the ability of humans to distinguish between samples of ground-truth images and samples of recon- structed ones. This is typically done by conducting experiments where human observers vote on whether the generated images are real or fake (Dahl et al., 2017, Denton et al. 2013 Guadarrama al. 2017 lizuka et al., 2016), Isola et al., 2017, (alimans al. 2016) Zhang et al., 2016 2017). ) However, such experiments are too costly and impractical for optimizing models. A practical and sensible alternative to quantify the perceptual quality is via some perceptual index xxfxx..p)... where d(.,.) is statistical divergence between probability distributions (e.g., Kullback-Leibler, Wasser- stein) (Blau & Michaeli 2018). Quantifying the perceptual index for high- nigh-dimensional distributions is both statistically and computationally intractable, so it is common to resort to approximations. Popular examples include the Fréchet Inception Distance (FID) (Heusel et al. 2017) and the Kernel
Inception Distance (KID) (Binkowski et al. 2018).
3

---

Published as a conference paper at ICLR 2025
2.2 OPTIMAL ESTIMATORS FOR THE SQUARED ERROR DISTORTION
Due to the distortion-perccetiion tradeoff (Blau & Michaeli 2018), it has become common practice compare image restoration algorithms on the distortion-perceppee plane, where the goal is to obtain optimal estimators with the lowest possible distortion given a prescribed level of perceptual index. This goal can be formalized by the distortion- pptetpioonp function (Blau & Michaeli 2018
D(P) min E ((X(
s.t. d(px,Px) S.P.
PRIY
Perhaps the most common points of interest on D(P) are ((oo) and D()), where the first point cor- responds to the estimator achieving minimal average distortion under no constraint, and the second corresponds to the estimator achieving minimal average distortion under a perfect perceptual index constraint. Considering the squared error distortion, these points are defined by
min B|||X
1 and
PXIY
min B||| 1 s.t. PX = PX.
PRIY
the posterior mean X = [[X]], which typically produces overly-smooth reconstructions (Blau & Michaeli] 2018). Therefore, PIR tasks, is more appropriate to aim for the solution to Problem (3). Interest- ingly. |Friiichh et al. (202T) proved that a solution to Problem (3) can be obtained by solving the optimal transport problem
respectively. It is well-known that the unique solution to Problem
PUV E arg min
B[IIU" - VV
where IIpxx ((-x (pu.v' : Pt" = Px is the set of all joint probabilities pvr, v with marginals pt px and pv Px.. Namely, the optimal solution to Problem (3) can be constructed as follows: Given a degraded measurement y. first predict the posterior mean f = E[X Y y). and then sample from p=]j(- which is the optimal transport plan from Px- to px- Similarly to Freirich et al. (202T), we denote such a solution to Problem (3) by Xo.
As discussed before, of the most common and appealing solutions for PIR tasks is the estima- tor X that samples from the posterior distribution Pxjy, such that Pxıy = Pxly- While such an estimator always attains perfect perceptual index (Blau & Michaeli 2018), its MSE is typically larger than that of Xo (Blau & Michaeli] 2018: Freirich et al. (021) (see Figure In other words, to design an algorithm with minimal MSE under perfect perceptual index constraint, one should often not resort to posterior sampling, but rather to solving Problem 33). This is our goal in this paper. Lastly, one may wonder whether sampling from PxI&- instead of using the optimal transport
plan from Equation (4) may also be effective in terms of MSE. However, in Appendix A. we prove that such an approach leads to precisely the same MSE as sampling from the posterior.
2.3 FLOW MATCHING AND RECTIFIED FLOWS
Flow matching. Flow matching algorithms (Albergo & Vanden-Eijjdenn 2023: Lipman et al., 2023
Liu et al., 2023) are generative models defined via the ODE
d7, = v),,,
where e is often called a vector field, and Z, some forward process such that Pz, is the source distribution, from which we can easily sample (e.g., isotropic Gaussian noise), and pz, is the target distribution from which we aim to sample (e.g., natural images). In principle, one can generate samples from the target distribution pz, by solving Equation (5). where samples from the source distribution pz, are set as the initial conditions for the ODE solver. Nevertheless, given a particular forward process Z, there are possibly many different vector fields that satisfy Equation (5). The goal in flow matching is to somehow find an appropriate vector field with desirable practical and theoretical properties, e.g., where the solution Equation (5) is unique.
Rectified flow. Rectified flow (Liu et al. 2023 is a flow matching algorithm defined via the par- ticular forward process
2 1Z + (1 t)Zu,
4

---

Published as a conference paper at ICLR 2025
Algorithm 1: Posterior-Mean Rectified Flow (PMRF)
Training
Stage 1: Solve w* < arg min.. X - (Y)
Stage 2: Solve 8 +- arg min, E [| Zo) - ug (2 +)
Z, = LX (1 (X)-))))) + o,c), where 1 is sampled from U'O, 1].
Inference (using Euler's method with K steps to solve the ODE)
Sample (0,
② x fw- (y) + GgC
11 is the given degraded measurement
for O..... K 1 do
a + 2 + vo- (#, k
Return it
which connects samples from pz, and pz, with straight lines. Here, Zo and Z1 can be statistically independent, as is typically the case when learning a flow model from Gaussian noise to image data, but they can also have any joint distribution 2,. This forward process clearly adheres to the ODE dZ. = (Z) - Zo), where Zi - Zo is the corresponding vector field. However, this is not a practical generative model, since it requires knowing the "destination" realization of Zi at any time step / < (i.e., the solution is not causal). To solve this issue, Liu et al. (2023) offer instead to use
URF(Zz, E Zo|Z,).
(7)
which is a causal vector field that generates the target distribution, given that the solution Equa- tion (5) exists and is unique when adopting such a vector field (Theorem 3.3 in (Liu et al.) 2023
Interestingly, solving the ODE in Equation (5) with URF often approximates the optimal transport map from the source distribution to the target one, especially when the process is repeated several times (i.e., reflow) or when pz; .Zo is close to the optimal transport plan between pz, and pz, (Liu
ct al. 2023 Tong et al. (024). To learn URF, one can simply train model og by minimizing the loss
L
[I(2]
Zo) vg vo (Ze (Ze, 1) de,
where the expectation is taken over the joint distribution ,7, Liu al. 2023).
3 POSTERIOR-MEAY RECTIFIED FLOW
We now describe our proposed algorithm, which we coin Posterior-Mean Rectified Flow (PMRF) (Algorithm D}. Our method consists of two simple training stages. First, we train model fa to predict the posterior mean by minimizing the MSE loss,
w arg min E [[|X [Y())
Note that this training stage can often be skipped, whenever there exists an off-the-shelf algorithm that attains sufficiently small MSE (high PSNR) in the desired restoration task. In the second stage, we train a rectified flow model uo (a vector field) to solve
J.' IE [[[(X Zo) vo t|? dt,
0t = arg min
(10)
T.t, where NNO,, 7) is statistically in- dependent of Y and X, and a, is hyper-parameter that controls the level of the Gaussian noise added to the posterior mean prediction. As shown by AAlergo et al. (2023). adding such a noise is critical when the source and target distributions lie on low and high dimensional manifolds. respec- tively. Specifically, it alleviates the singularities resulting from learning deterministic mapping between such distributions. Note, however, that adding noise to fwr (Y) may harm the MSE of the reconstructions produced by PMRF, and or should be taken to be sufficiently small.
where Z: :- LX + (1 Z) Here, Zo fa- +o,t
To explain why PMRF approximates the desired estimator Xo. prove important proposition and demonstrate it on a simple example with closed-form solutions. Specifically, let
dz, URF t)dt with 2. Zu
(11)
5

---

Published as a conference paper at ICLR 2025
be the ODE in PMRF, where URF (2, (
E[X Zo|Z, = 2 and Z, is the random vector generated we prove the following:
by PMRF at time stept t E [0, 1]. In Appendix A.2
Proposition I. Suppose that a = O, and let us assume that the solution of the ODE in Equation 11 exists and is unique. Then,
(a) 7n attains a perfect perceptual index (Pz, = p
(b) The MSE of Zi cannot be larger than that of the posterior sampler.
(c) If the distribution of (X - -X*) = 24 is degeneeeee for almost every 2 E supp pz, and
t E [o, 1), then the MSE of Z is strictly smaller than that of the posterior sampler.
Note that the assumption in (a) and (b) is the same as the one in (Liu et al., 2023), so it is not more limiting. Whether the additional assumption in (c) holds depends on the nature of the restoration task. For example, if X can be restored from Y with zero error (i.e,, Px]x(-11) is a Dirac delta function for almost every y), then X - X 0 almost surely and the assumption in (c) does not hold. Yet, this is not an interesting setting as the degradation is not invertible in most practical scenarios. To gain intuition into a more common scenario, consider the following example:
Example 1. Let Y = = N, where X ~ ~N(0, 1) and N ~ NO, x are statistically independent and N 0. Then, the MSE of Xo is strictly smaller than that of the posterior sampler. Moreover, when 0g 0, all the assumptions in Proposition hold, and we have 21 Xy almost surely.
See Appendix A..3 for the proof of Example This example shows that PMRF not only outperforms posterior sampling, but may even coincide with the desired estimator X, in certain cases.
4 RELATED WORK
Before moving on to demonstrate the effectiveness of our approach, it is instructive to note the difference between our PMRF method and existing techniques that may superficially seem similar.
Diffusion and flow-based posterior samplers. Diffusion or flow-based image restoration algo- rithms often attempt to sample from the posterior distribution by training conditional model that takes Y (or some function of Y, like X*) as an additional input (Lin et al.) [024] /Zhu et al., 2024). Some works avoid training a conditional model for each task separately, and rather modify the sam- pling process of a trained unconditional diffusion model (Chung et al.) 2023 Kawar et al., 2022) In Section 5.2 we perform controlled experiment on various inverse problems, which shows that our PMRF method consistently outperforms posterior samplers with the same architecture.
Flow from degraded image. Some diffusion//low models are trained on corresponding pairs of ground-truth images and degraded measurements (Albergo al., |023) Delbracio Milanfar 2023
Li al,, 2023). In this approach, the idea is to obtain a high-quality image by solving an ODE/SDE with the degraded measurement set as the initial condition. For example, Albergo et al. (2023) trained rectified flow model for the forward process Z, = LX + (1 ()Y where Y is up- sampled version of Y such that matches the dimensionality of X. These algorithms are closely related to PMRF, in the sense that they learn to transport an intermediate signal (instead of pure noise) to the ground-trutt image distribution. Yet, they have two critical disadvantages compared the degraded
to PMRF. First, the flow model's design is not agnostic to the type of degradation, signals can have varying dimensionalities or lie in a different domain than that of the ground-truth images (e.g., in MRI image reconstruction). Thus, the task of the flow model may be harder than necessary, as it needs to translate signals from one domain to another. On the other hand, in PMRF the flow model always operates in the image domain, where the dimensionalities of the source and target signals are the same. Second, the theoretical motivation for flowing from Y is not clear, least from reconstruction performance standpoint (e.g., distortion). In contrast, the theoretical motivation underlying PMRF is clear: it approximates Xo. which achieves the minimal possible MSE under the constraint of perfect perceptual index. As we show in Section 5.2 PMRF always either outperforms or is on-par with the solution that flows from Y (see Figure
Methods that aim for X, directly. To the best of our knowledge, Deep Optimal Transport (DOT) (Adrai et al. 2023 is the only existing method that, like PMRF, attempts to approximate

---

Published as a conference paper at ICLR 2025
Xy directly. Specifically, DOT approximates the desired optimal transport map (Equation (4) via a linear transformation in the latent space of a variational auto-encoder (VAE) (Kingma & Welling 2014). This transformation is computed in closed-form using the empirical means and covariances (in latent space) of the source distribution (that of the posterior mean predictions) and the target distribution (that of the ground-truth images), under the assumption that both are Gaussian. This method is computationally efficient, but the use of a VAE imposes a performance ceiling. Moreover, the optimal transport in DOT occurs in latent space and assumes that the source and target distri- butions are Gaussians, unlike Equation 4 which occurs in pixel space and does not make such an assumption. In contrast, PMRF does not use VAE, and approximates the optimal transport directly we show that PMRF significantly outperforms DOT (see Figure 4
in pixel space. In Section 5
5 EXPERIMENTS
5.1 BLIND FACE IMAGE RESTORATION
We train PMRF to solve the challenging blind face image restoration task, and compare its perfor- mance with leading methods. As in previous works (e.g.. (Wang al.] 202I)), we use the FFHQ
data set (Karras et al. 2019) with images of size 512 x 512 to train our model. Similarly to previous works, we adopt a complex and random degradation process to synthesize the degraded images.
Y [(X kr) tR +NolleG
(12)
where @ denotes convolution, ho is Gaussian blur kernel of size 41 x 41 and variance 0², LR is bilinear down-sampling by factor R. Ng is white Gaussian noise of variance 8², and [ JPE is JPEG cumpression-deccopressttt
with quality factor Q. Similarly to (Yue & Loy (024), we synthesize the degraded images by sampling a. R, ô and Q uniformly from [0.1, 15), [0.8, 32], [0, 20), and [30, 100), respectively. See Appendix B.1 for additional implementation details.
5.1.1 EVALUATION SETTINGS
For evaluation, we consider the common synthetic CelebA-Test benchmark, as well as the real- world data sets LFW-Test (Huang et al. 2008 Wang et al., 202T). WebPhoto- Test (Wang et al. (Wang et al. 2021). and WIDER - (hhou al.) 2022). CelebA- Test
2021]. CelebAdult consists of 3,000 high-quality images taken from the test partition of CelebA-HQ (Karras et al., 2018), and the degraded images were synthesized by Wang et al. (2021). For the real-world data sets, the degradations are unknown and there is no access to the clean ground-truth images. We compare our performance with DOT (Adrai et al. 2023) and leading blind face restoration models, including BFRffussion (Chen et al. 2024), DIfIBIR (Lin et al. 2024). DifFace (Yue & Loy| 2024). CodeFormer (Zhou et al.) 2022). GFPGAN (Wang et al. 2021 VQFRy1 and VQFR>2 (Gu et al. 2022). RestoreFormer and RestoreFormer++ (Wang et al. 2022 2023b). We do not compare with FlowIE (Zhu et al. 2024]. as the official checkpoints of this method are currently unavailable. How- ever, note that FlowlE is conditional method that employs ControlNet (similarly DiffBIR). Namely, it falls under the category of methods that attempt to sample from the posterior distribution, which are fundamentally different from PMRF Notably, the restoration methods that we compare against also use the degradation model from Equation (2). though the ranges of a, R, 8, and Q differ across methods. The ranges we choose, those from Yue & Loy 2024). are the most severe among all the compared methods. For example, the range of R we use is [0.8, 32), whereas Wang et al. (202T) use 8). Thus, PMRF attempts to solve a more difficult restoration task than some of the compared methods. In the following experiments, we use K = 25 flow steps in PMRF (Algo- rithm D. Refer to Appendix B.2 for an evaluation of additional values of K, and to Appendix B.3 for the implementation details of DOT.
5.1.2 RESULTS ON CELEBA-TESE
For the CelebA-Test benchmark, we measure the perceptual quality by FID (Heusel et al. 2017
KID (Binkowski et al., 2018), NIQE (Mittal et al. | (013)), and Precision (Kynkaänniemi et al. 2019
and measure the distortion by the PSNR, SSIM (Wang et al., (2004), and LPIPS (Zhang al. 2018
Similarly to previous works (Gu et al. 2022; Wang et 2021), we also compute the identity metric Deg (using the embedding angle of ArcFace (Deng et al. 2019)) and the landmark distance LMD.
7

---

Published as a conference paper at ICLR 2025
Input (real-wozid)
DIBBIR
DifFace
BFRifusion
RestoreForzeer
GFPGAN
PMRF (Ours)
LFW-Test
WIDER Test
CelebAdull- Test
WebPhoto-
17.0
17.9
19.4
11.0
11.6
12.6
12.6
5.11
5.4
5.9
5.8
38.7
55.7
72.7
89.7
33.1
33.8
74.2
94.8
96.5
113.0
129.5
146.1
72.7
93.5
114.3
135.0
FID
FID
FID
FID
PMRF (Ours)
RestoreFormerr+
VQPRv1 VQPRv2
GPPGAN
DifFace
DifBBIR
Posterior mean
BFRifusion
Figure 3: Real-world face image restoration. Top: Qualitative results on inputs from the WIDER- Test data set. Bottom: Comparison on the "distortion" piisootionn"ppereptioon plane (IndRMSE vs. FID), where IndRMSE indicates the RMSE of each method (the true distortion cannot be computed there is no access to the ground-truth images). Our algorithm outperforms all other methods IndRMSE, while achieving on-par perceptual quality compared to the state-of-the-are
Both of these can be considered as distortion measures, as they quantify some type of discrepancy between each reconstructed image and its ground-truth counterpart.
The results are reported in Table Notably, PMRF outperforms all other methods in FID, KID, PSNR, and SSIM, achieves the second best scores in NIQE, Precision and Deg, and the third best scores LPIPS, and LMD. Interestingly, no other method attains such consensus in performance like PMRF, namely, where none of the measures are significantly compromised compared to the state-of-the-art. For example, while DifFace achieves the highest Precision, it attains worse LMD, Deg, LPIPS, SSIM. and PSNR compared to the third best method in each of these metrics. This demonstrates that PMRF produces robust reconstructions, in the sense that it does not "over-fit" particular perceptual quality distortion measures, but rather achieves high performance in all of them simultaneously. Visual results are provided in Figure 2 and Figure in the appendix.
5.1.3 RESULTS ON REAL-WORLLD DEGRADED IMAGES
Evaluating the distortion for real-world degraded images is impossible, as there is no access to ground-truth images. Consequently, previous works conduct only perceptual quality evaluation (e.g., FID) on real-world data sets such as WIDER-Tes. and LFW-Test. Yet, high perceptual quality alone is clearly not indicative of reconstruction performance (to attain high perceptual quality, one may simply ignore the inputs and generate samples from px). Thus, we consider a measure which indicates the Root MSE (RMSE) and allows ranking algorithms according to their (approximate) RMSE, without access to the ground-truth images. Specifically, for any estimator X' holds that
E|||X - &1111 E[J[ f(Y + m,
(13)
where f(Y) 2e X is an approximation of the true posterior mean predictor X', and m is a con- stant that does not depend on X (see Appendix F for an explanation). Thus, the square root of E|| - [[) which we denote by IndRMSE, indicates the true RMSE. We utilize the posterior mean predictor trained by Yue & Loy 2024 as f, and compute the IndRMSE of all the evalu- ated algorithms on the LFW-Test, WebPhoto- Test, CelebAdult-Test, and WIDER- Test data sets. As
IImprrtattly, the exact same posterior mean predictor model (and weights) is also used by other methods such DifFace and DiftBIR, this is fair evaluation.
8

---

Published as a conference paper at ICLR 2025
Grund-truch
X
Flow from Y
(Ours)
Colorzation (ImugeNed)
Super-resoluiee
(ImageNer)
Colorization
(Celebu))
Super-rpanlann
(CelehA)
Colorization
Super-resolution
Denoising
RMSE
26.2
20.5
18.3
ImageNet
24.1
18.5
15.3
Celeb
21.9
16.5
12.3
19.8
14.5
9.3
2.4
5.9
9.5
13.1
2.2
10.0
17.8
25.6
0.1
3.6
7.0
10.5
FDeLIP
FDCLIP
FDeLIP
Inpainting
RMSE
20.7
18.0
13.0
16.5
19.7
16.5
11.8
14.8
18.7
15.0
10.5
13.0
17.8
13.5
9.2
11.3
35.1
53.7
72.3
90.9
36.5
58.2
79.9 101.61
36.9
52.6
68.4 84.1
35.5
53.0
70.5
88.0
4-
FID
FID
FID
4-
FID
PMRF (Ours)
X
Flow cond. 8 (post. sampling)
Flow from Y
DOT
Posterior mean
Flow
Y (post. sampling)
Figure 4: A controlled experiment comparing PMRF (our method) with several baseline methods, where the models are trained with the same architecture, hyper-parameters, etc. (see Section 5.2 Top: Qualitative comparison of PMRF and the baseline methods on several tasks. Bottom: Quan- titative comparison on the eustortion-perception plane, DOT is not flow model, but rather another approach that attempts approximate Xo (like PMRF). These experiments demonstrate that PMRF is either superior or is on-par with previous frameworks (i.e., posterior sampling or flowing from Y) on a variety of image restoration tasks. See Section 5.2 for more details.
before, we evaluate perceptual quality by FID, KID, NIQE, and Precision. In Figure 3 we provide visual results inputs from the WIDER-Test data set, and compare the algorithms on "distortion". perception plane (IndRMSE vs. FID). DOT is not plotted as it achieves far worse FID compared to other methods. Our algorithm attains the best (smallest) IndRMSE on all data sets, while achiev- ing on-par perceptual quality compared to the state-of-the-art This indicates that PMRF achieves superior distortion on such real-world data sets, while not compromising perceptual quality. In the appendix, we report the rest of the perceptual quality measures in Tables to 10 provide visual results in Figures 7 to 9
and also report the performance DOT.
5.2 COMPARING PMRF WITH PREVIOUS FRAMEWORKS IN CONTROLLED EXPERIMENTS
One may wonder whether the performance of PMRF is attributed the framework itself (Algo- rithm M]. or, maybe it is attributed to the model architecture, the rectified flow training approach, the chosen hyper-parameters, Could have done better by training flow sample from the

---

Published as a conference paper at ICLR 2025
posterior, or by adopting the approach of (Albergo et al. 2023) and flow directly from Y? Here, we conduct a controlled study where we demonstrate that the high performance of PMRF is indeed attributed to the proposed framework itself (Algorithm D. Specifically, we consider several im- age restoration tasks (denoising, super-resolution, etc.), where we train PMRF and several baseline methods on the "same grounds", using the ImageNet (Deng al.] 2009) (128 128) and FFHQ (256 x 256) data sets. In each task we train two conditional rectified flow models, where one is conditioned on the degraded measurement (we call this method flow conditioned on Y), and the other is conditioned on the posterior mean predictor fw. (Y) (we call this method flow conditioned on )) The first model represents posterior sampling methods, and the second model allows for a fair comparison of model capacity with PMRF (since PMRF is comprised of (Y) and a flow model). In fact, theoretically speaking, the second approach achieves precisely the same MSE as the posterior sampler (see Appendix [. and is often used in practice (e-g., in (Lin et al.) 2024] Zhu et al. 2024)). In addition, we train an unconditional rectified flow model, where the forward process is defined as Z. = f.X + (1 Zo. Zo- = Y + JgC. E e~ (N0 f). and Y is the up-scaled version of the degraded measurement Y such that it matches the dimensionality of X (we call this method flow from Y). This method represents the frameworks in (Albergo al. 2023 Delbracio & Milanfar 2023 Li et al. 2023), which we discuss in Section[]] All of the models are trained with the PMRF, using the same architecture, learning rate, weight decay, number of training epochs, etc. Moreover, for PMRF and flow conditioned on X method, we use the exact same architecture and weights for fw- (Y). To clarify the differences between the mathematical for- mulations of the baseline methods, in Table IT in the appendix we summarize the definitions of the training loss and the forward process of all methods. Moreover, in Algorithms to we disclose a pseudo-code for the training and inference procedures of the baseline methods. While DOT is not flow method, we still evaluate its performance as it is related to PMRE
same hyper-parameters
plane (RMSE FID for face restoration, and RMSE vs. FDCLIP (Stein et al.) 2023) for ImageNet restoration), using K = 100 flow steps for each flow algorithm. We clearly PMRF dominates all other methods in most tasks, achieving notably smaller RMSE without compromising (and sometimes even improving) perceptual quality. This demonstrates that PMRF achieves our desired goal, which is to attain low distortion without compromising on perceptual quality. For the image denoising tasks, we observe that PMRF and flow from Y attain relatively similar performance, and both dominate the posterior sampling approaches. We hypothesize that, in some tasks (e.g.. denoising), flowing from Y may be as effec- tive PMRF in terms of approximating Xo. To demonstrate this, we prove in Appendix D that flowing from Y is optimal in the toy problem in Example | (just like PMRF). Yet, our experiments demonstrate that PMRF generally leads to better performance compared to previous frameworks. To assess the effectiveness of each method given different inference time constraints, in Figure in the appendix we vary the number of flow inference steps K for each method. Interestingly, we observe that PMRF is still either on-par dominates the other methods for any given number of inference steps. These results further demonstrate that the superior performance of PMRF is attributed to our framework itself, rather than to the chosen hyper-parameters. See Appendix Cfor more details, and refer to Figures 10 to 16 in the appendix for visual comparisons.
In Figure 4 we compare the algorithms on the distortion-perceptiin
CONCLUSION AND LIMITATIONS
We presented method that directly approximates X. the estimator that minimizes the MSE under a perfect perceptual index constraint (Equation 3 We showed that our approach, coined PMRF, simple yet highly effective image restoration algorithm that outperforms previous frameworks (e.g., posterior sampling, flow from Y, and GAN-based methods) in a variety of image restoration tasks. As we explained in Section PMRF alleviates the issues resulting from solving the ODE by adding Gaussian noise to the posterior mean predictions. We note that the noise level o, should be carefully tuned, as taking to be too large or too small may cause the MSE or the perceptual quality of PMRF to degrade, respectively. While the flow from Y method (A (Algorithm 4 suffers from the same limitation (though it does not provide a theoretical guarantee on the MSE, like PMRF), this may be considered disadvantage of PMRF compared to posterior sampling methods (e.g.. Algo- rithm 2). which do not require such a under some conditions, PMRF is guaranteed to achieve smaller MSE than the posterior sampler. However, as in (Liu et al. 2023). one could argue that the assumptions in Proposition I may be too limiting in some cases.
Finally, we proved in Proposition I that
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
Our codes are available at /t/ttps://gitti com/ chayonguy/PMRF We provide all the ex- planations and checkpoints necessary to reproduce our results, including training. inference, and the computation of the distortion and perceptual quality measures in Section [ Besides our code, our paper discloses all the implementation details required to reproduce the results, including ar- chitecture details, training hyper-parameters, elc. Refer to Sections 5.1 and 5.2 and appendices B
and C for implementation details, and to Table 12 in the appendix for summary of our training hyper-parameters.
ACKNOWLEDGMENTS
This research was partially supported by the Israel Science Foundation (ISF) under Grants 2318/22, 951/24 and 409/24, and by the Council for Higher Education - Planning and Budgeting Committee.
REFERENCES
Theo Adrai, Guy Ohayon, Michael Elad, and Tomer Michaeli. Deep optimal trans- port: A practical algorithm for photo-realistic image restoration. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 61777-61791. Curran Associates, Inc., 2023. URL httpps://procedddnn// htt.://rrreeddd............r.. www.rnsu.i.... cc http://seeccpppeeeeleeeeeeeeeee
ftt.sf//aapeee/22/////// file FiLe//c2lccaa11addeSSeeeee Fite::/ccccaaaseeeeslaccea222119 Paper Conference, pdff
Paperr
Michael S. Albergo, Mark Goldstein, Nicholas M. Boffi, Rajesh Ranganath, and Eric Vanden- Eijnden. Stochastic interpolants with data-dependent couplings. arXiv, 2023. URL https:
//arxivv.or//as//23/1/ 03725
Michael Samuel Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference Learning Representations, 2023. URL
https::/cpenreeviie net/ forum dit.ummmddd1111ee......
Matthew C Bendel, Rizwan Ahmad, and Philip Schniter. A regularized conditional GAN for poste- rior sampling in image recovery problems. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: /tttps://opentee http///pprrevveee wwt/ffooume d=z4 700
Mikotaj Binkkwski, Dougal J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018. URL https:
//openzeview net//ta wor wt
Yochai Blau and Tomer Michaeli. The perception-distortion tradeoff. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2018.
Xiaoxu Chen, Jingfan Tan, Tao Wang, Kaihao Zhang, Wenhan Luo, and Xiaochun Cao. Towards real-world blind face restoration with generative diffusion prior. IEEE Transactions Circuits and Systems for Video Technology, pp- 1-1, 2024. doi: 10. 199/0CSTT22222243366999
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffiusion posterior sampling for general noisy inverse problems. In The Eleventh Interna- tional Conference on Learning Representations, 2023. URL https::/ pettps://openrevee.
Forum? Srrum?1d=OnDD2GACTO
Katherine Crowson, Stefan Andreas Baumann, Alex Birch, Tanishq Mathew Abraham, Daniel Kaplan, and Enrico Shippole. Scalable high-resolution pixel-space image synthesis with hourglass diffusion transformers. Forty-first International Conference on Machine Learning, 2024. URL
https: https://epenrevieet net f 2 WdW.IIn
Ryan Dahl, Mohammad Norouzi, and Jonathon Shlens. Pixel recursive super resolution. In Pro- ceedings of the IEFE International Conference Computer Vision (ICCV), Oct 2017.
11

---

Published as a conference paper at ICLR 2025
Giannis Daras, Hyungjin Chung, Chich-Hsin Lai, Yuki Mitsufuji, Jong Chul Ye, Peyman Milan- far, Alexandros G. Dimakis, and Mauricio Delbracio. A survey on diffusion models for inverse problems. arXiv preprint arXhv:2410.00083, 2024.
00083
A. C. Davison. Statistical Models. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2003. doi: 10.1017/CB09780511 11815810111
Mauricio Delbracio and Peyman Milanfar. Inversion by direct iteration: An alternative to denois- ing diffusion for image restoration. Transactions on Machine Learning Research, 2023, ISSN 2835-8856. URL tification.
wwwp:/rrewweeeeeeeeeeeeeeeeeeeee forum: www.ooornniiimoosssoee.. 1d=VmyFF51L33 Featured Cer-
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, doi: N0.110//VPR.22000522666
pp. 248-24882 248-255, 2009.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos 7afeiriou. Areface: Additive angular margin loss for deep face recognition. In 2019 IEEE/VVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4685 4685-4694, 2019. doi: 10.1109/VPP2201990004
Emily L Denton, Soumith Chintala, arthur szlam, and Rob Fergus. Deep genera- tive image models using laplacian pyramid of adversarial networks. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural In- formation Processing Systems, volume 28. Curran Associates, Inc., 2015. fttts::/prrceedingg wwwpeeediinnnnnnrniineeee neurips wwtppp/peapleppeeeeeeeeeeepppppe /espr////e//ppper//////1111 aa
URL
Papee Paper..a pdf
Dror Freirich, Tomer Michaeli, and Ron Meir. A theory of the distortion peeeottonnperreptioon tradeoff in wasserstein space. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural Information Process- ing Systems, volume 34, pp. 25661-25672. Curran Associates, Inc., 2021. URL http://pprr_.ileeeappr////221111
d 77766696clee33302233aa11///883 Paper. pdf
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sher- jil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Ad- vances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL. https::/ ngs www.inuurrs..m wttpiisss.ccppeeee wc/paper.pppeeee.eeeee cempper_fllesss filess/paper/2014) File/ /ttp://cceeea2222211111001011ee. Paierff--pppp..... pdf
Yuchao Gu Xintao Wang, Liangbin Xic, Chao Dong, Gen Li, Ying Shan, and Ming-Ming Cheng. Vqfr:
Blind face restoration with vector-quantized dictionary and parallel decoder. In Com- puter Vision ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVIII, pp. 126-143, Berlin, Heidelberg, 2022. Springer- ISBN 978- 3-031-19796-3, doi: 10. (11007797783-01--999 978-3 118-3-0311 3-10--3----- 17731-19797-02
8. URL https:://dooo http:///oo...o/10.101010000
Sergio Guadarrama, Ryan Dahl, David Bieber, Jonathon Shlens, Mohammad Norouzi, and Kevin Murphy. Pixcolor: Pixel recursive colorization. In British Machine Vision Conference 2017, BMVC 2017, London, UK, September 4-7, 20/7. BMVA Press, 2017. URL httpp:://ww. dropbox. com/
/ http://wwkkk11mmmmm www.exxxiinddxxxeeeeeeo 0447.pdf
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by two time-scale update rule converge to a local nash equilibrium. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: ottpo///o/o///epooee.oo.oooooooo
In I. Guyon, U. Von Luxburg,
files/
12

---

Published as a conference paper at ICLR 2025
Gary B. Huang, Marwan Mattar, Tamara Berg, and Eric Leaned-Miller. Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments. In Workshop on Faces Detection, Alignment, and Recognition, Marseille, France, October 2008. Erik Learned-Mlller and Andras Ferencz. and Frédéric Jurie. URL httph:://///.. /tttps://inria hal.
in 'Real-Life" Images:
sci accincc//nnnie Sance/inria-00::0 1233
Satoshi lizuka, Edgar Simo- Serra, and Hiroshi Ishikawa. Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultancous Classification. ACM Transactions on Graphics (Proc. of SIGGRAPH 2016), 35(4), 2016.
Phillip Isola, Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. CVPR, 2017.
Jari Kaipio and Erkki Somersalo. Statistical and Computational Inverse Problems. Springer, Dor- drecht, 2005. doi:
CtLbttts:: BRLtttts:://cs.co wttp:/d..ccc...eeeecceeeeeeeeeee 1338000
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for im- proved quality, stability, and variation. In International Conference on Learning Representations, 2018. URL btRpsss:::
openreveiivvooooo ew net. wwweeeeoouuee forum? d=Hk
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative ad- versarial networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396-4405, 2019. doi: 10.1 109/CCVR.2019900455
Bahjat Kawar, Gregory Vaksman, and Michael Elad. Stochastic image denoising by sampling from the posterior distribution. In 2021 IEEE/CVF International Conference Computer Vision Work- shops (ICCVW). pp. 1866-1875, 2021a. doi: 10. 1011000/CCV3:41202222...000
Bahjat Kawar, Gregory Vaksman, and Michael Elad. Snips: Solving noisy inverse problems stochas- tically. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang. and Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 21757-21769. Curran Associates, Inc., 2021b. URL htt
www.::iiiiinnnnneinaaeeeeeee.e.e www.weww..........eeeeeeeeeeeee. files/
Bahjat Kawar, Michael Flad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, 2022.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL TRLLattp:/ /awxia..ovo..................
abs/ 00ss/1112 1312.111222 6114
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Im- proved precision and recall metric for assessing generative models. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Buc, E. Fox, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https::/pproceednnsssc.co cc/pppr files//aper/201s/ pdf
F1le/02340m 0 2ew.:2222222c0000222220112222213
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo- realistic single image super-resolution using generative adversarial network. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 105-114, 2017. doi: 10.11.09/VP.
Bo Li, Kaitao Xue, Bin Liu, and Yu-Kun Lai. Bbdm: Image-to-image translation with brownian bridge diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1952-1961, 2023.
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF International Confer- on Computer Vision (ICCV) Workshops, pp. 1833-1844, October 2021.
13

---

Published as a conference paper at ICLR 2025
Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Bo Dai, Fanghua Yu, Wanli Ouyang, Yu Qiao, and Chao Dong. Diffbir: Towards blind image restoration with generative diffusion prior. arXiv, 2024. URL wttp://rrxiivv.oooooo org/ abs 200..100001011000
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Repre- sentations, 2023. URL httphttts///ppeeeeeeeee
Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Repre- sentations, 2023. URL https: http//ppeereeeee.eeeeeee n wor.moiduuioooo wttp://teeeooooooi.diiii.i....o.
Iya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019. URL
http://rweeewwweeeeeeeee nww.:nruttffouuoooo ?id
Bkg6RiCqY7
Sean Man, Guy Ohayon, Theo Adrai, and Michael Elad. High-perceptual quality jpeg decoding via posterior sampling. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1272-1282, 2023. doi: (0.1199CCPRR555222.2222 202.392222222200122222
Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. Making "completely blind" image qual- ity analyzer. IEEE Signal Processing Letters, 20(3):209-22) 212, 2013. doi: 10.1 10./
2227726.
Naoki Murata, Koichi Saito, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Yuki Mitsufuji, and Stefano Ermon. GibbsDDRM: A partially collapsed Gibbs sampler for solving blind inverse problems with denoising diffusion restoration. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 15501-255222 PMLR, 23-29 Jul 2023. URL https::/ppoceeedinng press/v2021 http://222mmrrrata..2. html
Anton Obukhov, Maximilian Seitzer, Po-Wei Wu, Semen Zhydenko, Jonathan Kyl, and Elvis Yu- Jing Lin. High-fidelity performance metrics for generative models in pytorch, 2020. URL https: :tttps::/gte / htwpeittt......tttoeooeeeeeeeeee com www//otsaa/toooeooooooooooooooee fodeliideiyyy Version: 0.3.0, DOI: 10.5281/zen- 0d0.4957738.
Guy Ohayon, Theo Adrai, Gregory Vaksman, Michael Elad, and Peyman Milanfar. High per- ceptual quality image denoising with a posterior sampling cgan. In 202/ IEEE/CVF Inter- national Conference on Computer Vision Workshops (ICCVW), pp. 1805-1813, 2021. doi:
10. 10.1100/0CVW4441220220.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning. volume 139 of Proceedings of Machine Learning Research, pp. -768.8763. PMLR, 18-24 Jul 2021. html
/ttp:///no......eee//////////ada URL. https: /ttpoc///edde./eoooooooocc ngs. mlr wrress//l1//rraforrrreee
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEF/CVF Con- ference Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, June 2022.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, SIGGRAPH '22, New York, NY. USA. 2022. Association for Computing Machinery, ISBN 9781450393379.
doi: 10.1145/52822333530755
URL https:
7/doi. oaailorg//in 10. 45/ 3w2.333223322 3530757
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, (74)))71134726 2023. doi: 0000TPPAMM..0022224441
14

---

Published as a conference paper at ICLR 2025
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.). Advances in Neural Information Processing Systems, volume 29. Cur- ran Associates, Inc., 2016. URL /RRl/tttp::///rreeeeddiii n
neur htwp:nngsniiiii..ceeeeeeeeeeeeee wttppppr.ccpppppeppppppp files/
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023.
URL https::/
net htt///oorrmrsssee __00s_SKKMMMMRK
George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Vil- lecroze, Zhaoyan Liu, Anthony L. Caterini, Eric Taylor, and Gabriel Loaiza-Ganen Expos- ing flaws of generative model evaluation metrics and their unfair treatment of diffusion mod- els. In Thirly-seventh Conference on Neural Information Processing Systems, 2023. URL htttss://openrevie... 1 ftte://oooommmo000
forum?
Christian Szegedy, Vincent Vanhoucke. Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 2818-2826, 2016.
doi: 10. 1099CVPR.2016...M.
Hossein Talebi and Peyman Milanfar. Nima: Neural image assessment. IEEE Transactions
Image Processing, 27(8):3998-4011 2018. doi: 10. 10.1109/1P.22111288111
Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative mod- els with minibatch optimal transport. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL cation.
Expert Certifi-
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In International Conference on Computer Vision Work- shops (ICCVW).
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In The European Con- ference on Computer Vision Workshops (ECCVW), September 2018.
Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. Towards real-world blind face restoration with generative facial prior. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2021.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. The Eleventh International Conference on Learning Representations, 2023a.
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004. doi: 10. 101110911.2003.81988880
Zhouxia Wang. Jiawei Zhang, Runjian Chen, Wenping Wang, and Ping Luo. Restoreformer: High- quality blind face restoration from undegraded key-value pairs. 2022.
Zhouxia Wang, Jiawei Zhang, Tianshui Chen, Wenping Wang, and Ping Luo. Restoreformer+4: Towards real-world blind face restoration from undegraded key-value paris. 2023b.
Tao Yang, Peiran Ren, Xuansong Xie, and Lei Zhang. Gan prior embedded network for blind face restoration in the wild. In IEEE Conference on Computer Vision and Patlern Recognition (CVPR), 2021.
Zongsheng Yue and Chen Change Loy. Difface: Blind face restoration with diffused error contrac- tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-15, 2024. doi: N0.1109/PPAM..2024...8
15

---

Published as a conference paper at ICLR 2025
Kai Zhang. Jingyun Liang, Luc Van Gool, and Radu Timofte. Designing practical degradation model for deep blind image super-resolution. In IEEE International Conference on Computer Vision, pp. 4791-4800, 2021.
Richard Zhang, Phillip Isola, and Alexei Efros. Colorful image colorization. In ECCV, 2016.
Richard Zhang, Jun- Yan Zhu, Phillip Isola, Xinyang Geng, Angela Lin, Tianhe Yu, and Alexei A Efros. Real-time user-guided image colorization with learned deep priors. ACM Transactions on Graphics (TOG), 94), 2017.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrcasonable effectiveness of deep features perceptual metric. In CVPR, 2018.
Shangchen Zhou, Kelvin C.K. Chan, Chongyi Li, and Chen Change Loy. Towards robust blind face restoration with codebook lookup transformer, In NeurlPS, 2022.
Yixuan Zhu, Wenliang Zhao, Ao Li, Yansong Tang, Jie Zhou, and Jiwen Lu. Flowie: Efficient image enhancement via rectified flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 13-22, June 2024.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and and-play image restoration. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (NTIRE), 2023.
16

---

Published as a conference paper at ICLR 2025
A SUPPLEMENTARY EXPLANATIONS FOR PMRF
A.1 PROOF THAT CONDITIONING ON X ACHIEVES THE SAME MSE AS POSTERIOR SAMPLING
Proposition 2. Let X be the estimator which, given any degraded measurement first predicts the posterior mean 3= = =E[[XY = y) and then samples from P (-13" Then, the MSE of X' equals twice the MMSE, which is the MSE attained by the posterior sampler.
Proof. The MSE of X' is given by
E|| X
B X 11],
(14)
where this equality follows from Lemma 2 in (Freirich et al. 2021 (Appendix B.1). By the defini- tion of X' we have 8
P..
E||| X-1 E||
11].
(15)
Substituting this result into Equation
14 we get
B|| - X' X - -8113
(16)
Namely, X' attains precisely the same MSE as the posterior sampler, which is equal to twice the MMSE (Blau & Michaeli (018). Thus, theory, one should not expect to improve the MSE of conditional diffusion/flow model by supplying X condition instead of Y.
A.2 PROOF OF PROPOSITION 1
For completeness, we first restate Proposition I and then provide its proof.
Proposition 1. Suppose that a 0, and let us assume that the solution the ODE in Equation 11 exists and is unique. Then,
(a) ž1 attains perfect perceptual index (p2, px).
(b) The MSE of 2, cannot be larger than that of the posterior sampler.
(c) the distribution of (X X*) 2, is non-degenerate for almost every 2, E supp Pz, and
t [0, 1), then the MSE of 2, is strictly smaller than that of the posterior sampler.
Proof. We first prove (a) and (b) assuming that the solution for the ODE in Equation 11
ists and is unique for a. = 0. Then, we will prove (c) by also assuming that the distribution of (X X') 2, is non-degenerate for almost every 2, and & [0, 1].
From Theorem 3.3 in (Liu et al. 2023) we have P2, = Pz, for every E [0, 1). This implies that P2. pz, px. i.e., PMRF attains a perfect perceptual index when 0, 0. This proves (a).
Next, without additional assumptions, we will prove (b) by showing that
E 2 - X-11 < - X-11,,
(17)
which will imply that the MSE of 2 can only be smaller than that of the posterior sampler. Since o, = O, we have Zn = X + O.C = X". Following similar arguments to those in the proof of
Note that X is "posterior sampler" which is conditioned X Thus, Algorithm represents such an algorithm, which is of the baseline methods evaluate in Section 5.2
17

---

Published as a conference paper at ICLR 2025
Theorem 3.5 in (Liu al. 2023). it holds that
21 - 1
URF Ze, t)ctt
(18)
27
URF( Ze: 1)dl
(19)
| (2,, t]]? dt
(20)
B|X -x*12a1 1 dt
IE
(21)
S
BIIX 1111212,jt
(22)
E
BBIIX X 11112 dl
(23)
|=||X 1
(24)
[E|| X 111
(25)
where Equation 188 follows from the definition of 21 and X-, Equation 19 follows from the fact
that P2, pz, Equation (20) follows from Jensen's 's inequality, Equation (1 follows from the def-
inition of URF Z,, t), Equation (22 follows from Jensen's 's inequality, Equation 23) follows from the
linearity of the integral operator, and Equation (24) follows from the law of total expectation. Thus, we have |[|| [| x x S E||X -X-1| 11). Combining this result with Lemma 2 from (Freirich
et al. 2021
(Appendix B.1), conclude that
E[|| - 1111 B[||X
- 811
S | - 11).
(26)
where the left hand side is the MSE of PMRF, and the right hand side is the MSE of the posterior sampler, which always equals twice the MMSE (Blau & Michaeli 2018).
Finally, to prove (c), let us further assume that (X - {') = 2, is a non-degenerate ran- dom vector for every 21 E pz, and
[0, 1]. Thus, the inequality in Equation (22)
becomes strict (from Jensen's 's inequality for strictly convex functions), and hence we have
2 - X1113 [12] < [|||X 11] Combining this result with Lemma 2 from (Freirich al. 2021)
(Appendix B. I), we conclude that
[||| - 111 < -
- 11.
(27)
Namely, the MSE of 2n (left hand side) is strietly smaller than that of the posterior sampler (right hand side).
A.3 PROOF OF THE RESULTS IN EXAMPLE
From Blau & Michaeli 2018 Freirich et al. 2021) ) we know that Xo in Example attains MSE that is strictly smaller than that of the posterior sampler (assuming that ON > 0). Specifically, the closed-form solution of X, in Example is given by Freirich et al. 2021):
X
Y.
(28)
V1+
Moreover, in this example, it is well known that the posterior mean E [[X|Y] is given by
X
V
(29)
1 + N
Next, we will prove that:
(a) All the assumptions in Proposition hold.
(b) ž, = -o almost surely.
18

---

Published as a conference paper at ICLR 2025
Proof of (a). Since 0s O, we have (R, E[X [z] and Ze = =x + (1- x)x*
Below, we show that
NN
Cov(X- - Zz) =t
and
(30)
1 + N
Var(Z;) 12
NN
(31)
1 N
1 1
Since X - -x and Ze are jointly Gaussiar we have
URF (Z,,) t) = =E[X- 1z
=E(x- x+1 Cov(X -8,, Ze)
(Ze E((z))
(ar
Cov(X 88, Zi)
Bin
(32)
(ar(
03 1+o%
Zr.
82 ON +
1
1
1+03s
1
1 Po Zt:
(33)
where Equation (32 follows from the fact that E - X = and E[Z,] = 0. One can verify that
the solution of dZ, = URF(2 (Ze, t)dt for any initial condition Zo = c is unique and is given by
2: = c 1 120
(34)
To show that the distribution of (X 8*) z is non-degenerate for almost every = and t [0, 1], note that
Var(X - -Co(X-- - X- 82)
= CC)((X,X) - -2Cov(X, + (0,
1
1 2Cov X,
" o zr
Y
+ Cov
1+ Y. 1 ox Y
2
1
= 1
Cov(X,V) +
(1+03) Cov(V,
1 o
2
1
1 1 NN
1 o
1
= 1
1 1+ N
ON
(35)
1+
'x- X and Z, can be written as linear transformation (X, Y), which are jointly Gaussian random variables. Thus, X x and Z, are jointly Gaussian.
19

---

Published as a conference paper at ICLR 2025
Thus, for any t > O, and assuming ON > 0, the correlation between X X
and Ze is given by
o 1+ox
Cov(X- - -x Zz)
(ar(Z)) Var(X - -X+)
a f2
1To )
(
a3 (To0k
+ 10
1
< 1.
(36)
Namely, the correlation between X - 8 and Ze is strictly smaller than for every t E (0, 1]. Moreover, for t = 0 the correlation between X-x* and Zi clearly equals zero, so such a correlation
is smaller than 1 for every & E [0, 1]. This implies that the distribution of (X - | = is non- degenerate for almost every 2 and E [0, 1], and so all the assumptions in Proposition hold.
To prove Equations (30) and 31p. first note that Cov(X, = Cov(X',8') = 1+o3 and so
Cov(X, x= Cov(X" X = 0. Thus,
Cov(X- Ze) Cov(X 8",, tX + (1 (1-t)8")
= (Cov((X, X) Cov(X, X²)) + (1 (C) 8*) - Cov(X",
=1
1
1+ o.
NN
1 2
(37)
1 ON
and,
(2) = Cov(Z,, Eτ)
Cov(IX + (X
= X) ()(xxtt)ooxxxx'))))))) + -1- C) .8))
1
1 + (21(1 - () (1 - (1=1)²) + N
= 2 + (2t - 21 11 21 + 12)
1 o
=t +(1 - P)
1+
(38)
1 N
1 1
2
Proof of (b). The proof follows directly from Equation 34p. Specifically, for the initial condition Zo X we have
2, = 11 3
11 2z Y
N
y
(39)
1 N
8o-
(40)
Thus, in Example 1
PMRF with 0, coincides with the desired optimal estimator Xo-
20

---

Published as a conference paper at ICLR 2025
A.4 REFLOW (OPTIONAL)
To potentially improve the MSE of PMRF further, one may conduct reflow procedure (Liu et al. 2023). where a sequence of flow models are trained, and the flow model at index k learns to flow from the source distribution to the distribution generated by the flow model at index k. Specifically,
let 21+1 be the random vector generated by PMRF (Algorithm [ where 2 replaces the role of X in Algorithm [ and 0 X (Zo remains unchanged). Thus, from Theorem 3.5 in (Liu et al.)
2023), we have E[e( 2 2o)] S E[c( Zo)], which implies the reflowing may only improve the MSE of PMRF, and hence improve the approximation of the desired optimal transport map
(Equation We leave this possibility for future work.
B SUPPLEMENTARY DETAILS AND EXPERIMENTS IN BLIND FACE IMAGE RESTORATION
B. IMPLEMENTATION DETAILS OF PMRF
During training, we only use random horizontal flips for data augmentation. We use the SwinIR (Liagg et al. 202T model trained by Yue & Loy (224) as the posterior mean predictor fwr in Algorithm I and use d3 = 0.1. This model was trained using the same synthetic degrada- tion as in Equation [12, with the same ranges for R, 8, and Q we mentioned in Section 5.1 The SwinIR model's weights are kept frozen during the vector field's s training stage, and the same weights are utilized during inference well. The vector field up is a HDiT model (Crowson et al.) 2024). which we train from scratch. As (Crowson et al. 2024), we sample t uniformly from 010 using stratified sampling strategy. The vector field trained for 3850 epochs using the AdamW optimizer (Loshchilov & Hutter 2019), with learning rate of 5 10 10-4 (81, B2) = (0.9. 0.95), and weight decay of 10 (as in (Crowson et al. 2.24)). In the last 350 epochs,
reduce the learning rate gradually, multiplying it by 0.98 at the end of every epoch. The training batch size is set to 256 and is kept fixed. We compute the exponential moving average (EMA) of the model's weights, using a decay of 0.9999. The EMA weights of the model are then used in all evaluations. Our model is trained using bfloat16 mixed precision. A summary of the vector field training ssper-parameters provided in Table 12
B.2 VARYING THE NUMBER OF FLOW STEPS K IN PMRF
In Tables 2 to we evaluate the performance of PMRF for various choices of K (the number of in- ference steps in Algorithm As expected, increasing K generally improves the perceptual quality while harming the distortion.
B.3 DETAILS OF DOT
We use the official codes of DOT (Adrai al.] 2023) as provided by the authors. This method performs optimal transport between the source and target distributions in latent space, using the closed-form solution for the optimal transport map between two Gaussians. As in (Adrai et al. 2023), we use the VAE (Kingma & Welling 2014) of stable-diffusion (Rombach et al. 2022). For computing the latent empirical mean and covariance of the target distribution, we provide to the code the first 1000 images from FFHQ, with images of size 512 x 512 (the default is 100 images, so using 1000 images instead ensures that the performance of DOT not compromised, as explain by Adrai et al. (2023)) For computing the latent empirical mean and covariance of the source distribution, we randomly synthesize degraded images according to Equation (12) from the first 1000 images in FFHQ, and reconstruct each image using the SwinIR model with the pre-trained weights from (Yue & Loy 2024) (the same weights we use in PMRF). Given a degraded image y at test time, the code Adrai et al. (223) first predicts the posterior mean using the SwinIR model, encodes it to latent space, optimally transports the result using the pre-computed empirical means and covariances, and finally uses the decoder to obtain the reconstructed image.
21

---

Published as a conference paper at ICLR 2025
B.4 COMPUTATION OF FID, KID, AND PRECISION
For each data set and algorithm, the FID, KID, and Precision are computed between the entire FFHQ 512 x 512 training set, and the reconstructed images produced for the degraded images in the test data set (as in previous works). For example, for the evaluations the CclebA-Test data, this means that the FID is computed between the 70,000 FFHQ images, and the 3,000 CelebA-Tess reconstructed images.
C SUPPLEMENTARY DETAILS ON SECTION 5.2
C.I DEGRADATIONS
Face restoration. The degraded images in each face restoration task in the controlled experiments are synthesized according to the following degradations:
1. Denoising: We apply additive white Gaussian noise with standard deviation 0.35.
2. Super-resolution: We use the 8x bicubic down-sampling operator, and add Gaussian noise with standard deviation 0.05.
3. Inpainting: We randomly mask 90% of the pixels in the ground-trute image, and add Gaussian noise with standard deviation 0.1.
4. Colorization: We average the color channels in the ground-truth image (with weight of 3 for each color channel), and add Gaussian noise with standard deviation 0.25.
ImageNet restoration. For the general-content (ImageNet) image restoration tasks in the con- trolled experiments, we consider the following degradations:
1. Denoising: We apply additive white Gaussian noise with standard deviation 0.2.
Super-resolution: We use the 4x bicubic down- wwwn-samllin operator, and add Gaussian noise with standard deviation 0.05.
Colorization: We average the color channels the ground-truth image (with a weight of for each color channel), and add Gaussian noise with standard deviation 0.05.
C.2 IMPLEMENTATION DETAILS OF THE FLOW METHIOSS
C.2.1 TRAINING
Face restoration. For all the face restoration tasks in Section 5.2 the models trained on the FFHQ data set with images of size 256 x 256 (we down-sample the original 1024 x 1024 images to 256 256). Unlike in the blind face image restoration experiments, where the model is trained on images of size 512 x 512, here we choose to use smaller image resolution to save computational resources and achieve shorter training times. We use random horizontal flips for data augmentation.
ImageNet restoration. The general-content image restoration models are trained on the Ima- geNet (Deng al.) (209) training data, after resizing the images 128 128 pixels. To obtain these images, we first resize the original images to have a shorter side of 128 pixels, and then perform random cropping obtain the desired size. We use random horizontal flips for data augmentation.
C.2.2 CHOICE OF a.
As expected, observe that using as 0 in both PMRF (Algorithm T and the flow from Y method (Algorithm(4) leads to blurry results with small MSE and large FID. Thus, for fair comparison, we use the same value of a. in both methods. We use a, 0.025 for all the ImageNet restoration tasks and for the face image denoising task. For the rest of the face restoration tasks (inpainting, colorization, and super-resolution, we use og 0.1
'Note that the "optimal" value of a. depends on the severity of the restoration task. For example, in mild image denoising task, the posterior mean X may already be close to the ground-truth image, should be smaller compared to where the noise severe.
22

---

Published as a conference paper at ICLR 2025
C.2.3 VECTOR FIELD
Face restoration. Similarly to Appendix B.1 the vector field is HDiT model. The time in Al- gorithms 1 and 2 to is sampled from 0 using a stratified sampling strategy. For all baseline methods and PMRF, we train the vector field for 1000 epochs, use fixed batch size of 256, adopt the AdamW W optimizer with learning rate of 5 x 10-1, (B1, .2) (0.9, 0.95), and weight decay of 10-2 As in (Crowson et al. 2024), we do not apply learning rate scheduling. Finally, we use the EMA weights for evaluation, using a decay of 0.9999. A summary of the hyper-pane ppper-prramtters is provided in Table 12
ImageNet restoration. The vector field remains the exact same HDiT model as in the face restora- tion experiments. Here, the model is trained for 100 epochs. and the rest of the hyper-parameterr (optimization, EMA for evaluation, etc.) remain the same as before.
C.2.4 POSTERIOR MEAN PREDICTOR
Face restoration. The posterior mean predictor fw is a 4.4M parameters SwinIR mode which we train from scratch for each task. In all tasks, this model is trained for 1000 epochs, with fixed batch size of 256, using the AdamW optimizer with a learning rate of 5 x 10 (31, B₂) (0.9, 0.95). without weight decay, and without learning rate scheduling. When utilizing this model in the flow process (e.g., in PMRF), we use the EMA weights computed with a decay of 0.9999.
ImageNet restoration. For the posterior mean predictor fws we use the exact same HDiT model as in appendix C.2.3] Namely, for the general-content restoration experiments, the posterior mean predictor and the vector field models are the same. This model trained for 100 epochs in all tasks. The rest of the training hyper- apper-parameters (optimization, EMA, etc.) remain the same the those of the SwinIR model described above.
C.2.5 EVALUATION
Face restoration. We test all models on the CelebA-Test data set, with images of size 256 x 256. We utilize the torch-fidelisn lity package (Obukhov et al. 2002 to compute FID, using the default inception-tee -on--v--omme -33mooopat- image feature extractor (Szegedy et al. 2016). The FID of each method is computed between the entire FFHQ 256 x 256 training set, and the images produced by the algorithm for the synthesized CelebA-Tess degraded images.
ImageNet restoration. We test all models on the ImageNet validation data set (50,000 images), with images resized to 128 128 pixels. To obtain these images, first resize the original images to have a shorter side of 128 pixels, followed by center cropping to the desired size. We again utilize the torch de delity package to compute FDCLIP, which is the Fréchet distance in the latent space image feature extractor (Radford et al., (02T) (using this model instead
of the cl wall--v-----22 of inception-v3- -3-omppt---- ensures better alignment with human opinion scores Stein et al. 2002)). The FDcuP of each method is computed between the entire ImageNet validation data set Iground-truth images) and the images produced by the algorithm for the corresponding synthesized degraded images.
C.3 DETAILS OF DOT
Face restoration. We utilize DOT (Adrai et al. 2023) similarly to Appendix B.3 using images of size 256 256 instead of 512 x 512, and adopting the official codes of the authors. To compute the latent empirical mean and covariance of the target distribution, we provide the first 1000 from FFHQ to the official code of DOT. For the source distribution, we randomly synthesize degraded images according to the degradation of each task (Appendix C.T) from the first 1000 images in FFHQ, reconstruct each image using the SwinIR model we trained for each task (the same weights use in PMRF), and finally compute the empirical mean and covariance of the reconstructions in latent space.
We use the official code for the SwinIR architecture from https: / / http:/gitthb.ooooo/./ wtw....com/inggyun..iaat SwiniB Implementation details and hyper-parameter provided in our code.
23

---

Published as a conference paper at ICLR 2025
ImageNet restoration. We select the first image from each class in the ImageNet training data. To compute the latent empirical mean and covariance of the target distribution, we provide the gathered 1000 images the official code of DOT. For the source distribution, we degrade each of the collected 1000 ground-truth images according to the degradation of each task (described in appendix [C.1) reconstruct the results using the trained ImageNet posterior mean predictor model (described in ap- pendix C.2.4., and finally compute the empirical mean and covariance of the reconstructions in latent space.
D PROVING THAT FLOW FROM Y IS ALSO OPTIMAL IN EXAMPLE
In Section 5.2 we show that, for the denoising task, PMRF and flow from Y are on-par in terms of both perceptual quality and MSE. To provide intuition for this result, show that flow from Y leads to the desired estimator X, in Example (just like PMRF does).
Specifically, as in Example[] suppose that X ~ -N 1) ~ NO (N) ON 0, and Y = X+N. In flow from Y with d, = we have Z, = iX + (1 t)Y, and thus URr 1) = E[ - Y|Z]).
Below, we show that
Cov(X - Y, 72 = (1 - 1)o2: and Var(Z;) (12 - 21 + 1) +
(41)
(42)
Hence,
URF (R, E[X Z|Z
E|x r Cov(X - -Y,Z0) Var(Zr)
(Z, E(Z,)
Cov(X Y 2r) Zr
(43)
Var
(t 11 2122 + 1)+
Zt.
(44)
where Equation 443 holds since E[X- - Y]=
= 0 and B[Zz] 0. One can verify that the solution of
dz. URF Z,, dtt for any initial condition 2.8- is given by
- c v (1 21 +1+1)+1
(45)
VI+oX
Namely, we have
21 1 Y
V1+02 N
Xo,
(46)
where the last equality follows from Equation in Example D
28
It follows that flow from Y is also optimal
just like PMRF.
Demonstrating Equations
(1 and 42 is straightforward. We have
Cov(X- Y, Cov(X Y,t + (1-t))
=Coov(X, + (1-)/ooxx - (Coo((,Y) - (1-t)Cov(Y,y)
=t+(1 (1])) -t (1 (1-t) +)(+)o
=(t
(47)
24

---

Published as a conference paper at ICLR 2025
and
Var(Z,) = Cov(tX +OV))x+-(1-),,xxx++11111
= + (1ot))Co
+2 (1-()=))
(1-
=tº(1 ((112+1+0))+211
+1
= =Po0 210 + +
+1
2t²2 2t + +)++1. +1.
(48)
E INDICATOR RMSE (INDRMSE) DERIVATION
The MSE of any estimator X can always written as
B||| - 81 = B||| X
+ 1-1
(49)
Bi||
m,
(50)
where X
E[X is the MMSE estimator, Equation (49 follows from Lemma 2 in (Freirich 2021 (Appendix B.1), and m is some constant that does not depend on X. Thus,
J(Y) we have
B|| X11 ~ EiJ S(X))) + m,
(51)
so E||
1 may be used as an indicator for V /B[IX - 11.. Future works should in- vestigate the effectiveness of this measure.
25

---

Published as a conference paper at ICLR 2025
Varying the number of flow steps K in PMRF (Algorithm 1b on the CelebA-Test blind face image restoration benchmark. Red, blue and green indicate the best, the second best, and the third best scores, respectively. Increasing the number of steps improves the perceptual quality while hindering the distortion. These results are expected due to the distortion periiiotooereppioon tradeoff.
Table 2:
Perceptual Quality
Distortion
K
FID
KID|
NIQEL
Precision
PSNR1
SSIM1
LPIPS
Degl
LMD
3
81.81
0.0811
8.9012
0.2820
27.668
0.7669
0.3582
31.41
2.0340
5
63.77
0.0581
7.4568
0,4563
27.498
0.7601
0.3401
30.80
2.0294
10
44.39
0.0342
5.2648
0.6427
27.017
0.7388
0.3314
30.49
2.0215
25
37.46
0.0257
4.1179
0.7073
26.373
0.7073
0.3470
30.67
2.0303
50
36.63
0.0244
3.8492
0.7050
26.028
0.6896
0.3591
30.89
2.0409
100
36.57
0.0240
3.7311
0.7010
25.810
0.6787
0.3662
31.06
2.0409
Table 3: Varying the number of flow steps K in PMRF (Algorithm 1 the LFW-Test blind face image restoration benchmark. Red, blue and green indicate the best, the second best, and the third best scores, respectively. Increasing the number of steps generally improves the perceptual quality while hindering the IndRMSE. These results are expected due to the distortion- petortionnpercepttoon tradcoff.
K
FID
KIDI
NIQE,
Precisiont
IndRMSE,
3
78.2331
0.0692
8.2315
0.3477
3.3934
64.3121
0.0524
6.8733
0.5143
3.8008
10
51.9845
0.0387
4.9896
0.6546
4.8648
25
49.3151
0.0366
4.0028
0.6692
6.1382
50
49.5581
0.0375
3.7126
0.6826
6.7960
100
49.6561
0.0377
3.6242
0.6710
7,2004
Table 4: Varying the number of flow steps K PMRF (Algorithm I on the WIDER- -est blind face image restoration benchmark. Red, blue and green indicate the best, the second best, and the third best scores, respectively. Increasing the number of steps generally improves the perceptual quality while hindering the IndRMSE. These results expected due to the distortion-perception tradeoff.
K
FID4
KID|
NIQEJ
Precisiont
IndRMSEJ
85,0361
0.0704
9.9988
0.2742
5,3486
5
65.2563
0.0451
8.4650
0.5381
5,7665
42.5002
0.0179
5.5677
0.7144
7.1134
25
41.2685
0.0160
4.0726
0.7144
9,2164
50
41.4446
0.0174
3.6953
0.6845
10.3403
100
42.9437
0.0183
3.5704
0.6907
11.0674
26

---

Published as a conference paper at ICLR 2025
Table Varying the number of flow steps K in PMRF (Algorithm) D on the WebPhoto- blind face image restoration benchmark. Red, blue and green indicate the best, the second best, and the third best scores, respectively. Increasing the number of steps generally improves the perceptual quality while hindering the IndRMSE. These results expected due to the distortion-perception tradeoff.
K
FID
KID
NIQEL
Precision
IndRMSEL
128.7858
0.0996
9.1626
0.3907
3.2961
113.4734
0.0782
7.5893
0.5553
3.7371
10
91.3677
0.0484
5,4199
0.6413
4,8369
25
81.0642
0.0347
4,2402
0.6462
6.3098
78.7174
0.0324
3,9512
0.6265
7.0159
100
79.1239
0.0313
3.7990
0.5602
7.6887
Table 6: Varying the number of flow steps K in PMRF (Algorithm) 1 on the CelebAdult-Tees blind face image restoration benchmark. Red, blue and green indicate the best, the second best, and the third best scores. respectively. Increasing the number of steps generally improves the perceptual quality while hindering the IndRMSE. These results are expected due to the iestortion-perception tradeoff.
K
FID4
KID
NIQEJ
Precisiont
IndRMSEJ
122.8780
0.0551
6.6818
0.3944
3.7339
113.7837
0.0426
5.5810
0.4444
4.3313
105.7426
0.0319
4.4119
0.6111
5,4908
25
102.8914
0.0293
3.7367
0.5500
6.7145
102.1454
0.0276
3.5609
0.6278
7.3004
100
102.0568
0.0279
3,4878
0.5944
7.7286
Table 7: Quantitative evaluation of blind face restoration algorithms on the LFW-Test data set.
Method
FID
KID
NIQEJ
Precisiont
IndRMSE
SwinIR ( Posterior mean)
87.34
0.0808
8.595
0.2513
DOT
97.09
0.0891
5.705
0.1806
26.24
FestoreFormer++
50.80
0.0386
3.911
0.6330
9,429
RestoreFormer
49.04
0.0355
4.168
0.6674
12.21
CodeFormer
52.82
0.0387
4.484
0.6756
9.534
VQFRv1
51.31
0.0399
3.590
0.6014
11.26
VQFRv2
51.16
0.0378
3.761
0.6154
16.15
GFPGAN
47.59
0.0308
4.554
0.6400
9,842
DifIBIR
40.97
0.0234
5.738
0.5804
9.105
DifFace
46.48
0,0329
4.024
0,7411
11.33
BFRffusion
50.93
0.0377
4.963
0.6850
7,210
PMRF (Ours)
49.32
0.0366
4.003
0.6692
6.138
27

---

Published as a conference paper at ICLR 2025
Table 8: Quantitative evaluation of blind face restoration algorithms on the WIDER- -eet data set.
Method
FID
KID|
NIQEJ
Precisiont
IndRMSE
SwinIR ( Posterior mean)
91.96
0.0780
10.16
0.1649
0
DOT
82.15
0.0618
7.633
0.4082
14.900
Fosmore
45.41
0.0209
3.759
0.6505
14.466
RestoreFormer
50.23
0.0251
3.894
0.6505
14.200
CodeFormer
39.27
0.0138
4.164
0.7227
12.185
VQFRv1
44.21
0.0192
3.055
0,5959
17.042
VQFRv2
38.70
0.0157
3.995
0.6381
16,368
GFPGAN
41.28
0.0182
4.450
0.7876
11.840
DifiBIR
35.87
0.0114
5.659
0.6361
11.106
DifFace
37.38
0.0131
4.383
0.7856
10.418
BFRffusion
56.82
0.0307
4.647
0.5825
11.759
PMRF (Ours)
41.27
0.0160
4.073
0.7144
9.2164
Table 9: Quantitative evaluation of blind face restoration algorithms the WebPhoto-Tees data set.
Method
FID
KIDI
NIQE
Precisiont
IndRMSE
SwinlR ( Posterior mean)
132.1 1
0.1022
9.638
0.2383
DOT
125.6
0.0865
7.397
0.3071
20.69
RestoreFormert+
75.60
0.0291
4.080
0.6143
18.43
RestoreFormer
77.80
0.0334
4.460
0.6265
11.55
CodeFormer
84.17
0.0406
4.709
0.6830
8.952
VQFRv1
75.57
0.0312
3.608
0.5774
12.53
VQFRv2
83.52
0.0411
4.620
0,5848
14.48
GFPGAN
88.43
0.0494
4.941
0.6781
9.240
DiAIBR
92.82
0.0541
6.069
0.5307
9.152
DifFace
80.05
0,0341
4.405
0,7273
10.31
BFRffusion
84.83
0.0388
5.612
0.5872
7.222
PMRF (Ours)
81.06
0.0347
4.240
0.6462
6.310
Table Quantitative evaluation of blind face restoration algorithms on the CelebAdult- data set.
Method
FIDU
KIDJ
NIQEL
Precision
IndRMSEL
SwinIR (Ar Posterior mean)
143.80
0.0811
7.477
0.4222
0
DOT
208.54
0.1634
6.018
0.0444
44.24
RestoreFormert+
103.81
0.0313
4.006
0.5167
11.43
RestoreFormer
103.96
0.0315
4.320
0.5556
14.97
CodeFormer
111.62
0.0427
4.544
0.5722
10.49
VQFRv1
105.59
0.0336
3.756
0.5944
11.14
VQFRv2
104.72
0.0337
3.999
0.6056
18.51
GFPGAN
109.19
0.0395
4.423
0.5111
11.90
DIfBBIR
109.74
0.0411
5.650
0.5000
9.853
DifFace
98.780
0.0243
3.901
0.6833
12.66
BFRifusion
103.06
0.0290
4.702
0.6056
8.037
PMRF (Ours)
102.89
0.0293
3.737
0.5500
6.715
28

---

Published as a conference paper at ICLR 2025
Inpainting
Super-resolution
Denoising
Colorization
271.6
324.1
169.5
429.6
RMSE
223.2
276.7
141.1
390.3
174.8
229.3
112.6
351.0
126.4
181.9
84.1
311.7
35.5
53.0
70.5
88.0
36.5
58.2
79.9
101.6
36.9
52.6
68.4
84.1
35.1
53.7
72.3
90.9
FID
FID
FID
FID
PMRF (Ours) & Xo
Flow cond. X (post. sampling)
A
Flow from Y
DOT
Posterior menn 8
Flow Y (post. sampling)
Face image restoration.
Colorization
Super-resolution
Denoising
685.6
419.5
331.5
RMSE
586.9
349.5
248.9
488.1
279.5
166.4
389.3
209.5
83.8
2.4
5.9
9.5
13.1
2.2
10.0
17.8
25.6
0.1
3.6
7.0
10.5
FID
FID
FID
PMRF (Ours) x Xo
Flow cond. X (post, sampling) A
Flow from Y
DOT
Posterior menn
8
Flow Y (post. sampling)
ImageNet restoration.
Figure 5: A controlled experiment comparing PMRF with previous methodologies, where we vary the number of steps K in each algorithm (Algorithms [ and 4. Specifically, we
K E {5, 10, 20, 50, 100), where larger marker size corresponds to larger value of K. See Sec- tion 2 for more details.
Table 11: comparison the forward process and training loss of PMRF and the baseline methods from Section 5.2 For the flow from Y algorithm, have yt Y for all tasks besides super- resolution. For the super-resolution task, we up-scale Y using nearest-neighhor interpolation.
Forward process
Flow training loss
Z, tX + (1 t)Z, Zo = (Y) + a.e
PMRF (Ours)
mine So E [I((X Zo) vo(Zi, dt
(0,11
Z, = LX +1 1)2
Flow cond. on Y
ming So [][(X Zo) 00 (Z,, t, Y) dt
Zo No(,
Z, tX + (1 Z)
Flow cond. on X
mins fo E [[[(X Zo) v0 vo t, fw (2) dt
Zo ~ NO00
Z, tX + (1 t)
Flow from Y
Zo yt +
ming 10 E [[(X- Zo) - no (2 1)
c~ (0.
29

---

Published as a conference paper at ICLR 2025
Algorithm 2: Flow conditioned on Y
Training
Solve l < arg min E [|l(X Zo) vo Zt,,, Y)
2 = LX (1 Zo, Ze ~ (N, I). is sampled uniformly from U00,11 1].
Inference (using Euler's method with K steps to solve the ODE)
Initialize (0 for i f O..... + f + vo- (t, R v)
do
11 y is the given degraded measurement
Return &
Algorithm 3: Flow conditioned on X
Training
Stage 1: Solve w + arg min., X (Y)
Stage 2: Solve +- arg ming || Zo) - uo oo,Z,, t, fw. (Y))
Z. LX
Zu, Ze NO, I). is sampled uniformly from U10, 1].
Inference (using Euler's method with K steps to solve the ODE)
Initialize [N
for i + 0 K 1 do
② i + K u0- (f, k fu-
11 is the given degraded measurement
Return $\
Algorithm 4: Flow from Y
Training
Solve l" + arg ming E [[(X Zo) - uu (e( t)13
// : =x=xx t)Zo, = Yo=Y+ e~ (0, I), and yt is the up-scaled version of Y that sampled uniformly from
matches the dimensionality of X. U].
Inference (using Euler's method with K steps to solve the ODE)
Initialize ~Nly Io²) /1 y² is the up-sca version of the degraded measurement for i ... K 1 do + f + vo- (σ, k
Return &t
30

---

Published as a conference paper at ICLR 2025
Table 12: HDiT architecture (Crowson al. 2024) details and training
Blind face restoration (Section 5.1
Controlled experiments
Hyper-parameter
(Section 5.2
Parameters
160M
121M
44.83 (face restoration)
GFLOPs forward
100.67
11.21 (ImageNet restoration)
Memory consumption
612MB
464MB
1000 (face restoration)
Training epochs
3850
100 (ImageNet restoration)
Batch size
256
256
256x256 (face restoration)
Image size
512x512
128 x 128 (ImageNet restoration)
Precision
6 mixed
bfloat16 6 mixed
Training hardware
16 A100 40GB
L40 48GB
Training time
12 days
2.5 days
Patch size
4
4
Levels
2 +
+
(local global attention)
Depth
(2,2,8)
(2,11)
Widths
(256,512,1024)
(384,768)
Attention heads
(4. 8, 16)
(6,12)
(width/ / head dim.)
Attention head dim.
64
64
Neighborhood
7
7
kernel size
Mapping depth
1
1
Mapping width
768
768
Optimizer
AdamW
AdamW
Learning rate
5 10-4
5 10
Learning rate
Multi-step
Not applied
scheduler
last 350 epochs
AdamW betas
(0.9, 0.95)
(0.9, 0.95)
AdamW eps.
10
10-8
Weight decay
10-2 2
10-2 2
EMA decay
0.9999
0.9999
31

---

Published as a conference paper at ICLR 2025
Gromil-rath
Engett (yyithetic)
Figure Comparison with state-of-the-ard blind face restoration methods on inputs from the CelebA-Test data set. Our method produces high perceptual quality while achieving lower dis- tortion overall. Zoom for best view.
(roal-world)
DERBIR
DifFace
BFRffusion
Rostore Pormer-i-1
GFPGAN
PMRF (Ours)
Figure 7: Qualitative results on the real-world LFW-Test data set. Our algorithm produces constructions with either better or on- perceptual quality compared to the ofate-of-the-arr, while maintaining very high consistency with the input measurements. Zoom in for best view.
32

---

Published as a conference paper at ICLR 2025
Input wreal-worid)
DiRBIR
DifFace
BF Rifiusion
RestoreFormees
GFPGAN
PMRF (Ours)
Figure 8: Qualitative results on the real-world WebPhoto-Tess data set. Our algorithm produces reconstructions with either better on-par perceptual quality compared to the saate-o--the-aee while maintaining very high consistency with the input measurements. Zoom in for best view.
Inpat (real-word world)
DiMBIR
Difface
BFRiiusion
Restore Pormer-+++
GFPGAN
PMRF (Ours)
Figure 9: Qualitative results on the real-world CelebAdult-Tee data set. Our algorithm produces reconstructions with either better on-par perceptual quality compared to the state-of-the-art, while maintaining very high consistency with the input measurements. Zoom in for best view.
33

---

Published as a conference paper at ICLR 2025
Degrades Geound truts
Posterior
x
K-6
PMRP (Ours)
K 100
K 5
Flow cora.
K 100
K-5
Flow cond. f
K 100
X 5
Hlo from Y
K - 100
DOT
Figure 10: Visual results on the face image colorization task from Section 5.2 Our method outper- forms all baselines for any number of inference steps K. Zoom in for best view.
34

---

Published as a conference paper at ICLR 2025
Gronnd-rrut
Degraded
Posterio
x
K K-6
PMRF (Ours)
X - 100
K-5
Flow cocd. Y
K 100
--6
X
Flow
R 1001
X 5
Flow frous
K - 100
DOT
Figure 11: Visual results on the face image denoising task from Section 5.2 Our method is on-par with flow from Y. and outperforms the posterior sampling methods for any number of inference steps K. Zoom in for best view.
35

---

Published as a conference paper at ICLR 2025
Geound truts
Degraded
Posterior
*
K-6
PMRP (Ours)
K 100
K 5
Flow cora. Y
K 100
K-5
Flow cond. f
K 100
X 5
Hlo from Y
K - 100
DOT
Figure 12: Visual results on the face image inpainting task from Section 5.2 Our method outper- forms all baselines for any number of inference steps K. Zoom in for best view.
36

---

Published as a conference paper at ICLR 2025
Goonnd-rrut
Degraded
Posterio
x
K K-6
PMRF (Ours)
X - 100
K-5
Flow cocd. Y
K 100
--6
X
Flow
K 1001
X -
Flow Srors
K - 100
DOr
Figure 13: Visual results from Section 5.2 on the face image super-resolution task. Our method on-par with flow from Y. and outperforms the posterior sampling methods for any number of inference steps K. Zoom in for best view.
37

---

Published as a conference paper at ICLR 2025
Ground truth
Degrded
Pustecioe X
K G
PMRE (Ouz))
X 100
K-5
Flow Y
K 100
X
K
K- 100
5
Naive Bow
K 100
DOT
Figure 14: Visual results from Section 5.2 on the ImageNet denoising task. Our method outperforms all baselines for any number of inference steps K. Zoom in for best view.
38

---

Published as a conference paper at ICLR 2025
Ground truth
Deguded
Pustecioe
X
PMRE (Ouzs)
X
X 100
K-5
Flow Y
K 100
K
K- 100
--6
Naive Bow
K 100
DOT
Figure 15: Visual results from Section 5.2 on the ImageNet super-resolution task. Our method
outperforms all baselines for any number of inference steps K. Zoom in for best view.
39

---

Published as a conference paper at ICLR 2025
Ground truth
Deguded
Pustecioe
X
X G
PMRE (Ours)
X 100
K-5
Flow cood. Y
K 100
X
K
K- 100
Naive Bow
K 100
DOT
Figure 16: Visual results from Section 5.2 on the ImageNet colorization task. Our method outper-
forms all baselines for any number of interence steps K. Zoom in for best view.
40


Reviews:
[{"review_id": "KdhzC7jPtq", "reviewer": "Reviewer_Zdeq", "summary": "This paper starts from the distortion-perception tradeoff in image restoration. It first generates the posterior mean and then transports the result to a high quality image following the ground-truth image distoration. The first stage is done by a traditional end-to-end model SwinIR trained with L1 loss, while the second stage is implemented with a rectified flow model by approximating the desired optimal transport map.", "questions": "See weakness", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1, It minimizes the MSE under a perfect perceptual index constraint and achieves a smaller MSE (higher fidelity) than posterior sampling methods (e.g., diffusion-based methods directly restoraing the image from noises).\n\n2, With such a two-stage model, it could be easy-to-train (training a L1 loss-based model is stable) and fast-to-inference (both of the two stages are few-step models).\n\n3, This paper is well-motived, well-written, well-supported and easy-to-understand.", "weaknesses": "1, What’s the performance of the baseline model trained the L1 loss? Will it achieve the best PSNR?\n\n2, In addition to distortion metrics, could you please the model’s ability for human face identity perservation?  \n\n3, What are the inference speed, memory consumption and FLOPs?\n\n4， The core idea of generating a L1 loss result and then transport it to visually-pleasing image is somewhat similar to the idea of [1] (using the normalizing flow to generate high-quality images conditional the low-quality input). What are their relations?\n\n[1] Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pr8T7NyLxX", "reviewer": "Reviewer_PKin", "summary": "This paper presents a novel approach to photo-realistic image restoration (PIR) that aims to minimize mean squared error (MSE) while maintaining perceptual quality. The authors introduce the Posterior-Mean Rectified Flow (PMRF) algorithm, which optimally transports the posterior mean prediction to the distribution of ground-truth images. This method contrasts with traditional approaches that often prioritize perceptual quality at the expense of distortion. By leveraging a rectified flow model, PMRF effectively approximates the optimal estimator that minimizes MSE under a perfect perceptual index constraint. Experimental results demonstrate that PMRF outperforms existing methods across several restoration tasks.", "questions": "There are noticeable reflective spots or areas on the forehead, inner eye corners, and cheeks. Are these artifacts a result of the PMRF ? Please clarify the cause of these effects. Additionally, are there any limitations to the proposed method? Could this design be adapted for more general image restoration tasks beyond photo-realistic images?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ PMRF utilizes an optimal transport approach to align the posterior mean with the ground-truth distribution, enhancing the accuracy of image restoration.\n+ The paper provides theoretical foundations that demonstrate PMRF’s ability to achieve lower MSE compared to traditional posterior sampling methods.\n+ The authors conduct extensive experiments across multiple image restoration tasks, showcasing PMRF’s effectiveness and robustness compared to state-of-the-art techniques.", "weaknesses": "- The recent FlowIE work also employs rectified flow for image restoration. I noticed that this paper references FlowIE in the supplementary material. It would be beneficial to include a discussion of this related work and highlight the differences in the main paper.\n- In Figures 1 and 4, there are some unnaturally abrupt reflective spots or areas on the forehead, inner eye corners, and cheeks. Are these artifacts caused by the Posterior-Mean Rectified Flow? Please explain the reason behind these effects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2bqSC7dU6k", "reviewer": "Reviewer_amj7", "summary": "The work introduces Posterior-Mean Rectified Flow (PMRF) for image restoration, addressing the challenge of minimizing Mean Squared Error (MSE) while maintaining high perceptual quality. PMRF operates in two stages: first predicting the posterior mean, then using a rectified flow model to transport the result to match the ground-truth image distribution. The authors provide theoretical foundations showing that PMRF can achieve better MSE than posterior sampling methods while maintaining perfect perceptual quality under certain conditions. Through extensive experiments on various image restoration tasks (including blind face restoration, denoising, super-resolution, inpainting, and colorization), they demonstrate that PMRF consistently outperforms or matches state-of-the-art methods in both distortion metrics and perceptual quality measures.", "questions": "If I understand correctly, the idea proposed by the authors should also be effective on general images, beyond the specific tasks tested. It would be interesting to know if the authors have considered conducting experiments or making attempts on general image datasets to further validate the versatility of their method. Of course, the paper is still completely acceptable without these additional experiments. However, including them would significantly enhance the impact of the work.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "+ The paper provides comprehensive experiments and solid theoretical proof, supporting the effectiveness of the proposed method.\n+ It is well-written, with clear and informative figures and tables that enhance the reader's understanding. \n+ The motivation behind the approach is convincing, and the experimental results substantiate the motivation to a significant extent, demonstrating the practical impact of the proposed framework.", "weaknesses": "- While the method shows strong results in blind face image restoration, its evaluation is limited to this domain. Given the theoretical generality of the proposed framework, it would be valuable to see experiments on more diverse and general image datasets to assess the broader applicability of PMRF. Extending their experimental evaluation to include general image restoration would demonstrate the versatility of their method.\n- Including ablation studies on the rectified flow model, posterior mean prediction, noise levels, and other architectural choices would provide deeper insights into the importance of each component and help the reader understand which parts of the framework drive the performance improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N0sB9f4itD", "reviewer": "Reviewer_rJJP", "summary": "The paper introduces posterior-mean rectified flow framework, which approximates optimal estimator under perceptual constraint. The method includes predicting posterior mean and subsequent distribution transport using rectified flow. The experiments on face image restoration tasks validate its efficacy.", "questions": "I suggest the authors add experiments on other image domains rather than face domain since the main topic is image restoration problem.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.\tThe motivation is clearly stated. The main idea that considering image restoration problem using MSE optimization under perceptual constraint is interesting and essential.\n2.\tThe experiments on face image restoration demonstrate superior performance over previous methods.", "weaknesses": "1. The method seems like such a pipeline: Supervised restoration model trained using MSE loss + SDEdit method (with added $\\sigma_s$ noise). Can authors provide further interpretations?\n2. The experiments focus on specific the face image domain. The extension to other image domains (such as natural images) is unclear and unexplored.\n3. The structure of the paper should be reorganized. The main body does not contain any quantitative results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)