Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Authors: Xinmin Qiu
Chen Gege
Bonan Li
Congying Han
Tiande Guo
Zicheng Zhang
Venue: ICML 2025 poster (2025)
Keywords: Blind Face Restoration
Schrödinger Bridge
Primary area: applications->computer_vision

Abstract: Blind face restoration (BFR), which involves converting low-quality (LQ) images into high-quality (HQ) images, remains challenging due to complex and unknown degradations. 
    While previous diffusion-based methods utilize feature extractors from LQ images as guidance, using raw LQ images directly as the starting point for the reverse diffusion process offers a theoretically optimal solution. 
    In this work, we propose Pseudo-Hashing Image-to-image Schrödinger Bridge (P-I2SB), a novel framework inspired by optimal mass transport problems, which enhances the restoration potential of Schrödinger Bridge (SB) by correcting data distributions and effectively learning the optimal transport path between any two data distributions.
    Notably, we theoretically explore and identify that existing methods are limited by the optimality and reversibility of solutions in SB, leading to suboptimal performance. 
    Our approach involves preprocessing HQ images during training by hashing them into pseudo-samples according to a rule related to LQ images, ensuring structural similarity in distribution.
    This guarantees optimal and reversible solutions in SB, enabling the inference process to learn effectively and allowing P-I2SB to achieve state-of-the-art results in BFR, with more natural textures and retained inference speed compared to previous methods.

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Xinmin Qiu Gege Chen
Bonan Li'
Congying Han Tiande Guo Zicheng Zhang'
Abstract
of extensive prior information during the restoration pro- cess. In recent years, generative models (Ho et al., 2020; Rombach et al., 2022; Goodfellow et al., 2014; Liu et al., 2022; Karras et al., 2022; Li et al., 2025) based on diffu- sion processes have demonstrated promising results in BFR. These models have evolved from focusing on the transport between Gaussian noise and real data distributions to facili- tating transport between arbitrary real data distributions. In contrast, bridge-based methods address the transport paths between arbitrary distributions. An increasing number of image processing tasks in the visual domain, including im- age inpainting, and image segmentation, are capitalizing on this capability. Diffusion-basee generative approaches offer robust data priors and a comprehensive solution framework. Particularly in image restoration tasks, which require trans- forming low-quality (LQ) images into high-quality (HQ) counterparts with clear details and consistent content.
Blind face restoration (BFR), which involves con- verting low-quality (I-Q) images into high-quality (HQ) images, remains challenging due to com- plex and unknown degradations. While previous diffusion-based methods utilize feature extractors from LQ images as guidance, using raw LQ im- ages directly as the starting point for the reverse diffusion process offers a theoretically optimal so- lution. In this work, we propose Pseudo-Hashing Image-to-image Schrödinger Bridge (P-12SB), a novel framework inspired by optimal mass trans- port problems, which enhances the restoration potential of Schrödinger Bridge (SB) by correct- ing data distributions and effectively learning the optimal transport path between any two data dis- tributions. Notably, we theoretically explore and identify that existing methods are limited by the optimality and reversibility of solutions in SB, leading to suboptimal performance. Our approach involves preprocessing HQ images during training by hashing them into pseudo-sample according to rule related L.Q images, ensuring structural similarity in distribution. This guarantees optimal and reversible solutions in SB, enabling the in- ference process to learn effectively and allowing P-12SB to achieve state-of-the-art results in BFR, with more natural textures and retained inference speed compared to previous methods.
Recent strategies to address these issues can be categorized into two primary approaches. The first approach involves utilizing non-diffusion based methods for preliminary pro- cessing, followed by refinement through diffusion- based techniques. Examples include DifFace (Yue & Loy, 2022) and DR2 (Yue & Loy, 2022), which employ CNN-based models like SwilR (Liang et al., 2021) for initial restoration, enhancing texture clarity using diffusion- based methods. The second approach involves designing sophisticated fea- ture extraction and fusion techniques to refine condition guidance. Methods such as DiffBIR (Lin al., 2023) and PGDiff (Yang et al., 2024) incorporate external modules to improve the integration of L.Q images as condition guid- ance, which indirectly controls the inverse process. How- ever, these approaches introduce additional complexities. The preliminary models are constrained by their lightweight nature, potentially leading to the loss of critical informa- tion. Meanwhile, the latter requires meticulously designed modules to effectively guide the path, which can weaken the inherent transport capabilities of diffusion models.
1. Introduction
Blind face restoration (BFR) (Yang et al., 2021; Wang et al., 2021; Chen et al., 2021a) presents a significant challenge in the field of image restoration due to the complexity of degra- dation and the unknown causes of such degradation, along with high demands for the quality of restoration results. As a blind inverse problem, BFR necessitates the incorporation
Both these methods share a focus learning generative models for HQ images by using feature extractors and fusion to derive and embed content features from IQQ images into the model as guidance conditions. However, the forward and reverse processes of diffusion models are not inherently linked LQ images, limiting restoration performance due to the constraints of LQ feature extraction. Thus, robust feature
Equal contribution 'School of Mathematical Sciences, Univer- sity of Chinese Academy of Sciences, Boijing. Correspondence to: Congying Han <hancy @ucas.ac.cn>>
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
extraction and embedding for LQ images are essential to fully exploit diffusion models, such as DDPM (Ho al., 2020; Nichol & Dhariwal, 2021), which leverage learned transformations between Gaussian noise and HQ images. Consequently, we explore the direct construction of optimal transport paths directly between LQ and HQ images, where Schrödinger Bridge (SB) based methods can effectively achieve this goal.
Ro (X).
di, Us U FlgggoO.. +B.a...
Xn
In this work, we propose P-I2SB, novel restoration frame- work, which consists of two stages: the Pseudo-Hashing Module (PHM) for preprocessing image pairs using the seeudo-hashing strategy, and the Schrödinger Bridge Mod- ule (SBM) for directly finding the optimal path between two data distributions without the need for indirect guid- ance through Gaussian noise. In the second stage (SBM), employ 12SB (Liu al., 2023), robust restoration framework based on SB. 12SB leverages the forward and backward stochastic differential equation models inherent in SB, connecting with stochastic gradient models (SGM) through conditional configurations. While effective in single-degradation image restoration tasks, the performance in blind tasks suboptimal. We address this by examining how the optimal transport problem in image restoration can establish the forward and backward stochastic differential equation models of SB. Notably, blind tasks do not conform to the typical Monge optimal transport problem, where the optimal transport between images is not unique. A sin- gle high-quality image may correspond to multiple optimal paths from various LQ images, each facilitating restoration back the high-quality image. This contradicts the optimal- ity principle of reversible SB, hindering the development of a consistent I2SB model. We explore this theoretical foundation, as illustrated in Figure 1, and introduce novel pseudo-hashing preprocessing strategy, which constitutes the first stage (PHM) of our method. This technique pro- cesses training image pairs in blind tasks by transforming the associations between high-quality and low-quality dis- tributions, ensuring the transformation is reversible.
(Ra. dx, Is & VlogW(X tijdr + d.di
Figure I. Theoretical Foundations. We first pscudo-hashing zo to to using the information about 21 that &p has similar distribu- tion structure with æ1, and then build Schrödinger Bridge Module (SBM) between do and x1 Both two processes reversible.
ing previous diffusion-based and flow-based methods. By directly identifying optimal transport paths between LQ and HQ distributions, our framework aligns more closely with the nature of reverse problems, achieving superior results with natural and clear textures and reduced distortions.
2. Related Work
2.1. Blind Face Restoration
Blind face restoration (Li et al., 2020; Wang et al., 2021; 2022; Qiu et al., 2023; 2024) is a crucial subfield within image restoration. Sigmificant advances have been pro- posed, including various methodological approaches such as CNN-based (Menon et al., 2020; Li et al., 2020), GAN- based (Yang et al., 2021; Wang et al., 2021; Chen et al., 2021a), dictionary-based (Gu et al., 2022: Zhou et al., 2022: Wang et al., 2022), and diffusion-based (Yue & Loy, 2022: Wang et al., 2023) methods. GFPGAN (Wang et al., 2021) and GPEN (Yang et al., 2021) leverage GAN-based gen- erative models within an encoder-decoder framework to embed face prior. RestoreFormer (Wang et al., 2022) in- tegrates classical dictionary-based methods with contem- porary vector quantization (VQ) techniques (Esser al., 2021). DiABIR (Lin et al., 2023) enhances prior knowledge by employing pre-trained stable diffusion as generation pri- ors. FlowIE (Zhu et al., 2024) and PMRF (Ohayon et al., 2024) are image enhancement methods based on rectified flow (Liu et al., 2022).
Instead of relying on external methods or designing complex networks, our work focuses on correcting unreasonable end- point constraints to fully exploit the potential of SB models for learning optimal transport paths compared to previous. We identify that conflicts arise from the forward process definition in Vanilla-I2SB theoretically, where single HQ image can correspond to multiple degraded versions, result- ing in divergent forward processes. The core of our research is the exploration of these improvements from both theo- retical and experimental perspectives, demonstrating their effectiveness, which confirms the practical efficiency and economic viability of the proposed enhancements. Com- pared to guidance- baaed methods, our approach achieves state-of of-the-ar art results within the same inference time, par- ticularly on datasets with complex degradations, outperform-
2.2. Schrödinger Bridge in Diffusion
Schrödinger Bridge (SB) problem (Schrödinger, 1932; Léonard, 2013; Chen et al., 2016) represents an entropy- regularized formulation of optimal transport. Given two arbitrary probability distributions, p(zo) and p(11). the SB
2

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
framework seeks determine an optimal coupling path between these distributions. To address this challenge, De- noising Diffusion Bridge Models (DDBMs) (Zhou et al., 2023) have been introduced, which leverage the concept of driving the diffusion process to establish a diffusion bridge. Image-to-Imal
Therefore, the more extensively studied OMT problem is relaxed version introduced by Kantorovich in the 1940s This problem seeks an optimal joint distribution within von-empty joint distribution space. Its dynamic formulation was elegantly developed by Benamou and Brenier (Ben- amou & Brenier, 2000), and is expressed as:
Schrödinger Bridge (12SB) (Liu et al., 2023) adopts a computational framework analogous to standard score-based models. This approach has demon- strated exceptional performance across variety of image- to-image translation tasks. Unlike conventional diffusion models (Ho et al., 2020; Nichol & Dhariwal, 2021; Song et al., 2020; Rombach et al., 2022), I2SB constraints neces- sitate careful consideration to ensure optimal performance in practical applications.
inf (a.0) /
"fe [l((tx))1/Hx(dd
+ V (w/))= = 0, 14 Vo V1.
Ot
Here, the flow (nz: S t 1} represents a family of continuous mappings from the interval [o, 1] to the set of probability measures on Rd each possessing finite sec- ond moment. The variable denotes smooth vector fields associated with these mappings.
3. Preliminaries
We begin by establishing some notation and providing an overview of the problem. From Bayesian perspec- tive (Davison, 2003; Kaipio& & Somersalo, 2006), any image I E RR whether perceived as clear or degraded, can be viewed as a realization of random variable X. We consider two probability distributions, denoted as My and P1, which correspond the measures of the spaces of HQ and LQ images, respectively. These distributions are expressed as Po(r)dz= = vo(dz) and p1 (v)dy = 1/2 (dy). Given a LQ im- age sampled from p1 (y). the objective of an image restora- tion algorithm is to generate a prediction that closely approximates the corresponding clear image x ~ po()).
3.2. Schrödinger Bridge Problem
Let Ω = C((O, 1), RR) denote the space of all continuous R -valued paths over the unit time interval [0, 1]. The sets (2) and P(2) represent all positive measures and prob- ability measures on the space Q2 respectively. Given a ref- erence path measure R E (S2), the Schrödinger bridge problem (Schrödinger, 1932; Léonard, 2013; Chen al., 2016) is formulated as:
minimize H(P[R) Ep(In dP dR s.t. PEP(O) Po vu. P1 P1=11
(3)
3.1. Optimal Mass Transport Problem
The image restoration problem can be viewed as a type of optimal mass transport (OMT) problem. To begin, we consider the general framework of optimal mass transport, specifically Monge's OMT problem (Chen et al., 2021b). Let Thar := {T', _dd + ded T#o 11) represent a family of measure- preserving maps, where T#m = v indicates that is the forw-forward of 14 under the map T. Additionally, let c: Rd x Td + (0, +o0) be a cost function that quantifies the cost of transporting a unit mass from a location (z) to a location y (1()).
where Vo. E P(Rd) represent the prescribed initial and final marginal distributions. The term H(P R) denotes the relative entropy of the measure P with respect to the reference measure R. This problem admits at most one solution, denoted P E P(L), which is referred to as the Schrödinger bridge from Mo 1/1 over the measure R.
In general, we consider a reference family denoted as D C (S2), which consists of Markov finite energy diffu- sion processes possessing various practical propertics. The coordinate process (,) = =t( can be represented:
inf I c(x, (()))ooddd).. TETrg JRd
dx,- f((x)ddt + v B.dW,,
(4)
dX,- = (x.( - log + V.dW,
If the map 7' is reversible, then for given y ~ Pi (v), we have x = -y) ~ po(x) as the restored image y in the context of image restoration. However, Monge's problem does not always admit a solution. For instance, consider the case where Po is Dirac distribution and P1 is the sum of two Dirac distributions, each with half the magnitude. Since Vo must be "split" to be transported to two distinct locations, no map exists between them, rendering Tagy, an empty set.
where 3, IR represents diffusion coefficients, and Js. (TRed, [0, 1)) + TR. denotes drift function. The term p(x, t) is the mareinaa density function of X, at time t. W, is the Wiener process, and We is its reverse process.
Given R E ID, we can obtain specific Schrödinger bridge P E from vn to v1 over R, whose coordinate process
'More details can seen Appendix
3

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
l
LQ image
Pseudo Strategy
1 pes
t~,
luq HQ image
Ies
Pseudo image
Cross-Atta
+ concatenate
Ipes
H H110 .2O)
?
Image-to-Image Schrödinger Bridge
C1. Ipes
U1 !HQ
Schrödinger Bridge Module M
Pseudo- Pseudo-Hashing Module (PHM) H
(a) Training
Inverse-PHM == )
(b) Inference
T
Origin:
8= 8
Low-quality Image y
Pscudo Image &
Cat-12SB: = 8[:3]
Inference
Image X
Res-12SB:
M()
Noise-12SB:
Denoise
Figure 2. Framework of the proposed P-12SB for blind face restoration task. P-12SB a bridge-based diffusion model incorporating a pseudo-hashing strategy. During training (a), given LQ and HQ face pairs, Pseudo image is generated via the Pseudo Pseudo-Hashing Module (PHM), including the strategies of Origin, Cat-12SB, Res-12SB 12SB, and Noise- 12SB. In inference (b), the model produces a Pscudo Image from LQ imput, which is then converted high-quality result using the inverse pseudo-hashing method.
X can be represented as follows (Chen et al., 2023):
4. Methodology
Here, introduce the Pseudo-Hashing-I225 (P-12SB). We (5) begin by theoretically exploring how SB retrieves clearer HQ images, presenting theorem to justify the rationale of I2SB and demonstrating its limitations for BFR through Corollary (Sec. 4.1). To resolve this issue, we propose the Pseudo-Hashing Module (PHM) with three implemen- tations: Res-I2SB, Cat-I2SB, and Noise-I2SB (Sec. 4.2), alongside the Schrödinger Bridge Module (SBM). Addition- ally, perform a toy experiment the MNIST dataset for quick verification (Sec. 4.3).
dX, = [Ja 3 log V(X.. ()dd V dX,- = [fe V log (X,, d) + B.dW,
s.t. Xo ~ po: X1 pi.
(6)
The functions V & E (R. [0, 1l) represent time-
varying energy potentials that solve set known coupled partial differential equations (PDEs).
The density function p" (t, of X; serves the solution to the following optimal transport problem (OTM) (Chen et al., 2021b):
4.1. Practical Application and Improvement of SB
Building the previous discussion, we derive optimal reversible Markov diffusion process X; that describes the evolution of the image state from clear to degraded and vice versa between two boundary distributions. In practice, the explicit forms of Po and p1 are unknown. Instead, we have access sample pairs { y:) x3 Po. Vo Pi Pi where 2; denotes clear image and ys its corresponding degraded counterpart.
('' inf (p.0)
/ G (1) a(t, x:1 J
3 log p(1,2x)
(12 (1)p(t,z)drdl
p(t, x)
Öp
8.t.
+ V (rp) = 0,
01
(8)
p(0,)) = (o).) p(1,-) = p.()).
For image processing tasks, a practical approach involves learning the reverse process from the forward process of a self-designed reversible Markov diffusion process X,. As for image restoration, we posit that X, is optimal when its
where ū(t x) = fe(x) lB, log p(t, and p(t, x r)dr
(dzr), just similar as equation (2).

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
forward process transforms a clear image into its correspond- ing degraded counterpart, which is stringent condition.
Algorithm Training
1: Input: clean pa(-) and degraded ps((1æ)
This leads us to consider the following question: given stochastic process defined over the time interval (0, 1):
2: repeat
t~ ur,r x ~ PA . pss(X
== H(x, y), according (11)
dx; (x()))dt + BBddW,
(9)
5: X, ~ qXx, (X, X1), according (15)
6: L= wa Il (X2,, t; X ||. take SGM
To ensure that the process X; can be described by Equa- tion (9) for t E (0, 1). it is necessary to appropriately select time boundary distributions. x represents the stochastic process subject to the conditions Xo- and X1 = y.
7: until converges
A Algorithm 2 Inference
Input: degraded XN ~ ps. trained c(, (1::;)) Step N
Generally, for any positive measure Q E M+ (9t) on Ω, its disintegration formula is expressed (I.éonard, 2013):
2: for n N to do
3: Predict X6 using c(i,; 0),
4: X ~ p(Xx-1 xn), according to (16)
20 = / 12a
(.)
(10)
5: end for
6: T = T(xo, XN), according to (18)
where Q01 (1) E M+ (I²2), and Q Q(- Xu = x, X1 = y) E P(Ω), referred to as the bridge of Q between the pairs (xr, y).
7: return: i
ing to address the infeasibility issue highlighted in Corol- lary 4.2. The unified training process detailed in Algo- rithm where x and y represent HQ and LQ images, respec- tively. Specifically, considering the feasibility of image pair configurations, we propose three detailed pseudo-hashing strategies within PHM, as illustrated in Formula (11).
Theorem 4.1. For P E D whose coordinate process Xx is the solution of the boundary value problem (5,6), then the coordinate process X," of pay is also subject to the equation (5) with boundary condition Xo = xx X1 = y
if pa(.) 5.(.) centered at E Rdd and pi(-) (c) centered E gdd
x,
Origin,
Theorem 4.1 indicates that an identical stochastic process over the time interval (0, 1) can be learned using a large number of sample pairs, which are assumed to follow Dirac delta distributions.
x. + y, I y,
Cat-12SB,
f == =(x(zx))
(11)
Res-12SB,
x + a (y)e,
Noise-12SB.
Take 12SB for example, which effectively utilizes the SGM framework construct SB for specific known degradation types. However, complexities arise in BFR, such as a sin- gle HQ image corresponding to multiple types of LQ im- ages. In this scenario, all LQ images are assumed to follow a certain distribution, resulting in complex sample pairs like {(2,, Vty) 24 ~ Po. Po, ~ pı}. (i 1.....,.J = 1...., .). Despite this, constructing a unified SB, as an- ticipated by I2SB, not feasible. The underlying reason is derived directly from Theorem 4.1, which we present corollary:
The first strategy is sample concatenation, referred to as Cat- I2SB. Given a high-quality image In, and its corresponding randomly degraded LQ image !td = Degradation, (4ng): where denotes the random degradation type and parame- ters, we perform channel concatenation. Specifically, Ing and Tt are concatenated to form new image, while 1g is concatenated with itself. Consequently, the training image pair becomes:
(e, v) = (Ing + lig
Id. $ In.)
(12)
Corollary 4.2. If we have two pairs like (1) and (xr, V2), then we can not build a unified Schrödinger bridge P E ID) between them by means shown in theorem 4.1.
The second strategy is relative residual prediction, termed Res-I2SB. From a spatial geometric perspective, the direc- tion of Rd Idd Ing aligns with 4ng. as illustrated in Figure 3(a). By setting the two endpoint distributions of the image pair as:
This follows from the uniqueness of the solution to a stochas- tic differential equation given an initial value.
(i. = -RR Ii Ing: Itg)-
(13)
4.2. Pseudo-Hashing,
Exeudo-Hashing Module Initially, for the given high- quality (HQ) and low-quality (LQ) image pairs, we pro- pose the Pseudo-Hashing Module (PHM) as preprocess-
We achieve the goal of hashing the endpoint distributions while maintaining a consistent direction and distance be- tween the hashed endpoints.
5

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
8
1
/
Correspondingly, the results of the pseudo-hashing need be restored in the inference process to obtain the correspond- ing high-quality image of the final repair. We give unified restoration process:
Y1g
Y1
6
3
3
3
3
6
6
5
5
5
5
3
8
8
8
8
y1, Kg
y1,
9
9
9
9
it
Origin,
LQ
128B
Audiv
.t: 3), 1 y,
Cat-12SB,
(4)
(b)
ã = J
Res-I2SD (18)
Figure 3. Toy Experiments. (a) illustrates beeudo-hashing of to Res-12SB within the R space. For single HQ image zo with two LQ counterparts, 91, and Vig. is split into do, and foz using residuals between (Zu. Vt,) and (zu, V1,). This allows the construction of unified SBM, depicted by bluc arrows between (Zog M1y) and (Zog, 91,). (b) presents results of toy experiments.
Denoise(E;;of)
Noise-I2SB.
where Denoise uses DDIM (Song et al., 2020) at the initial number of steps to denoise the image. Consequently, we obtain the final clear restoration result, denoted as T. The inference process is detailed in Algorithm 2.
The third strategy involves lightly adding noise, referred to as Noise-I2SB. A small amount of Gaussian noise is added to hag. which does not cause significant visual degradation but directly alters the distribution of the image vector Inq- se
4.3. Toy Exploration and Analysis
This is expressed as Ingise = l/g + Ade, where Xa represents the noise level, directly related to lig The image pair is then set as:
To directly investigate the impact of various hashing strate- gies within I2SB, we conducted comparative experiments on the MNIST dataset. The MNIST dataset consists of 28x28 grayscale images of handwritten digits. We denote HQ im- ages Ing and generate LQ images 1 by applying random manual degradation. Both training and inference processes utilize lightweight model implemented at resolution of 64x64. The comparative methods include Vanilla-12SB and variations with different hashing strategies: Res-12SB, Cat- 12SB, and Noise-12SB. As depicted in Figure 3 (b), while Vanilla- 12SB and the three hashing strategies can all restore degraded images of handwritten digits to clear images, there are discrepancies in label classification accuracy and sim- ple texture representation. This indicates that the improved methods preserve the structural integrity of image content more accurately during the restoration of LQ images.
( y) = (Ihgian Itg)-
(14)
This method leverages the most common Gaussian noise correction in SGM, applying minimal noise to the image to redistribute its numerical components into new distribution without compromising image content, as shown in Figure 2.
Schrödinger Bridge Module After preprocessing in the Pseudo-Hashing Module (PHM), we obtain the image pair (&,v). which is used to design the Schrödinger Bridge Module (SBM) for training these predefined image pairs. These pairs are treated as two new data distributions for SBM to learn. Inspired by 12SB (Liu et al., 2023), we directly employ the forward and reverse processes of 12SB. The forward process is defined as follows:
We recognize a strong correlation between the task con- straints of image-to-image translation and the data distribu- tions. In previous methods like DDPM (Ho et al., 2020), LQ images are often used guidance conditions to ensure ac- curate content inference. Techniques such as SR3 (Saharia et al., 2022) and IDM (Gao et al., 2023) enhance the embed- ding of LQ images to improve the ability of diffusion models to preserve image content during inference, which modifies the transmission path between noise and HQ images. We thoroughly explored the theoretical optimality of using the raw LQ image directly as one endpoint of diffusion process, instead of Gaussian distributions. In image-to to-imaae trans- mission, directly altering the input form of endpoint images, rather than adding complex network structures for feature extraction, is more straightforward and efficient, Both theo- retical and experimental results confirm that this approach effectively aids the SB in identifying the optimal path.
X xX,
X1) = (NNxiiMMXXX X1), Ex),
t.2 2
o²² X1, E X, = 0,2 + 0²
0.207
P4 =
-,² Xo +
0² o
(15)
where o. and oe is the scheduled parameter. The reverse pro- cess follows the design of the SGM framework. as shown:
X p(X.-11X6 Xt),
(16)
where denotes the trained model using a Unet network. Additionally, during training, we incorporate the loss as follows:
L Xo 11,t~ ((,,T, where the analysis of losses is detailed in Appendix F.
N.
(17)
oe
6

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
GPEN
RestoroFormer
DMDNet
DAEER
DifFace
DiATBIR
PGDiff
PMRF
FlowlE
P-12SB
Figure 4. Qualitative comparisons on the CelebA-Test for blind face restoration. Taking Res-I2SB example, our P-12SB demonstrates strong performance detail enhancement, hue preservation and attitude preservation compared to these latest GAN-based, dictionary-basee and diffusion-basee based methods. Zoom in for best view.
5. Experiments
generalization ability. All these datasets have no overlap with our training dataset.
Training Datasets We choose FFHQ (Karras et al., 2019) as training dataset, which contains 70,000 high-quality PNG format face images with 1024 1024 resolution. In this experiment, we resize all images 512 512 for training.
5.1. Comparisons with State-of-the-art Methods
Comparison Methods We compare Pseudo-Hasse Hasudng with recent BFR methods, including PSFRGAN (Chen al., 2021a), GFPGAN (Wang et al., 2021), GPEN (Yang et al.,
Preprocessing Since our P-12SB is supervised training, the corresponding LQ-HQ image pairs are required. We use generated random degradation model to simulate LQ images in the real world. Its generation formula (Zhang al., 201801 8a; Wang et al., 2021) is shown in (19), where xr HQ image, kn is Gaussian blur kernel, represents down- sampling scale factor, and g represents JPEG compression with quality factor q. The parameters a, r, 5, q are randomly sampled from 10.1: 10), {0.8: 8). {0: 20}, {60: 100). to align with the experimental environment recent methods for BFR, as shown in Appendix E. We also add gray color probability during the training process for color adaptation and augment data with the horizontal flip.
2021), VQFR (Gu et al., 2022), CodeFormer (Zhou et al., 2022), RestoreFormer (Wang et al., 2022), DMDNet (Li et al., 2022), DAEFR (Tsai et al., 2023), DifFace (Yue & 2022), DR2 (Wang et al., 2023), PGDiff (Yang et al., 2024), DifIBIR (Lin et al., 2023), PMRF (Ohayon et al., 2024). FlowIE (Zhu et al., 2024) and 12SB (Liu al., 2023).
Metrics We quantitatively compare the differences be- tween our method and SOTAs using five widely-used metrics, including SSIM (Wang et al., 2004), PSNR, FID (Heusel et al., 2017), NIQE (Mittal et al., 2012), and LPIPS (Zhang et al.. 201 (018b). Among them, NIQE a no- reference metric. SSIM and PSNR are pixel-wise similarity measures, while FID, NIQE and LPIPS are perceptual.
= [(x @ ki)) L. +ISJIPGG
(19)
Evaluation on Synthetic Dataset. We compared the restoration performance of P-12SB, using Cat-12SB as an example, with existing SOTA methods on synthetic CelebA- Test. Figure 4 shows visual comparison results, demonstrat- ing that approach can restore colors, and local features in severely degraded facial images. VQFR and RestoreFormer exhibit noticeable color artifacts in their restoration outputs, while CodeFormer, DR2, PMFR, and FlowIE produce overly smooth results, losing local tex- ture details. In contrast, our improved method achieves more
Testing Datasets We choose CelebA-Test, Real-World LFW, CelebChild and WedPhoto-Tess as the testing datasets. CelebA-Test contains 3,000 HQ images randomly sampled from CelebA-HQ (Karras et al., 2018) with the resolution of 512 x 512. Similarly, the corresponding random LQ images are generated for evaluation by using the degrada- tion model in (19) and the same set of parameters used in the training dataset. Real-World LFW, CelebChild and WedPhoto- Test are different real- world datasets to test the
accurate textures,

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
LQ
GPEN
RestoreFormer
DMDNer
DAEFR
DifFace
DifTBIR
PGDiff
PMRF
FlowTE
12SB
P-128B
Figure 5. Qualitative comparisons on real-world datasets. Our P-12SB demonstrates superior performance in both detail enhancement and hue preservation, particularly inputs with severe degradation. Zoom in for best view.
Table 1. Quatitative comparison on CelchA-Test with 3000 randomly selected images for blind face restoration. Bold and underline indicate the best and the second best performance. Our P-12SB excels in naturalness and perceptual metrics. GFP denotes GFPGAN and Restore denotes RestoreFormer. We compare with recent BFR methods, including GPEN (Yang ct al., 2021), GFPGAN (Wang ct al., 2021), RestoreFormer (Wang et al., 2022), DMDNet (Li et al., 2022), DAEER (Tsai et al., 2023), DifFace (Yue & Loy, 2022), DifIBIR (Lin al., 2023), DR2 (Wang et al.. 2023). PGDiff (Yang et al., 2024), PMRF (Ohayon et al., 2024), FlowIE (Zhu et al., 2024)
and 12SB (Liu et al., 2023).
Methods
Metries
Input
GPEN
GFP
Restore
DMDNet
DAEPR
DifFace
DifIBIR
DR2
PGDiff
PMRF
FlowEE
12SB
P-12SB
CVPR
CVPR
CVPR
TPAMI
ICLR
TPAMI
ECCV
CVPR
NIPS
ICML
CVPR
ICML
SSIM
0.6460
0.6777 0.6827 0.6219
0.6727
0,5892
0.6494
0.6570
0.6554
0.6220
0.6815
0.6479
0.7047
0.658I
PSNR
24.921
25.423 25.401
24.206
25.318
22.439
24.055
25.297
24.194
22.920
26.001
24.594
26.174
25.405
F
93.564
22.508 20.676
17.080
22.790
18.295
19.654
19.288
32.628
22.547
14,248
21.393
25.6026
13.910
NIQE!
9.1407
6.7775
6.7324
5.3440
6.7038
5,3992
6.1638
6.4053
8.1487
5,4556
5.6228
6.3571
6.5709
5,3300
LPIPS
0.5953
0,2956 0.2823
0.2702
0.2965
0.2695
0.3052
0.2689
0.3447
0.3011 0.2413
0.2623
0.2851
0.2395
natural and accurate restoration in both overall color reten- tion and local texture generation, attributed to the effective use of high-performance generative models. Quantitative comparisons are presented in Table 1. Our method demon- strates competitive performance on perceptual metrics such as FID, NIQE, and LPIPS. Throughout the restoration pro- cess, our approach achieves results that are more natural and consistent with facial priors.
the challenging Wider dataset, which features severe and complex degradations, our method achieves leading results on no-reference metrics such as FID and NIQE. These find- ings underscore the robustness and generalizability of our approach in a wide range of real-world scenarios.
5.2. Ablation Studies
Different between PHM and data augmentation. Data augmentation involves using certain properties of images, such as rotation and brightness, to augment the actual nu- merical distribution of training data without altering the semantic information of the images. In contrast, the strat- egy design in pseudo-hashing module (PHM) is intended to satisfy the boundary constraints in SB models, requiring the direct or indirect involvement of degradation representa- tions, and this hashing design is reversible.
Evaluation on Real-world Datasets. As illustrated in Figure 5, our method demonstrates exceptional robustness in handling real-world degradations, consistently producing HQ face images. In contrast, biffusion-basee methods like FlowIE and PMRF often result in unnatural facial structures. While codebook- based approaches can sometimes generate natural-looking faces, they frequently introduce substantial color and texture artifacts. Our method, however, excels in predicting local textures with fewer imperfections com- pared to other diffusion- based techniques. This advantage is attributed to our pseudo-hashin module, which enables the modeling of a more continuous optimal learning path. Ta- ble 2 presents a detailed comparison of our approach against existing SOTA methods, highlighting our superior perfor- mance across perceptual quantitative metrics. Particularly
Guidance Module. In this study, we systematically vali- date the efficacy of the guidance module, encompassing both conditional and degradation-aware components, through series of ablation experiments. As delineated in Table 3, experiments (a)-(c) rigorously analyze the impact of incor- porating these guidance mechanisms on restoration perfor-
8

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Table Quantitative comparison on real-world datasets. Bold indicates best performance. The non-reference metric NIQE
assesses the quality of restoration, while the distribution gap between FFHQ and results is quantified by FID.
Datasct
LFW
CelebChild
WebPhoto Test
Wider
Type-based
Method
NIQEL
FID
NIQEJ
FID
NIQE,
FID.
NIQEL
FID.
LQ
6,4582
140.64
6.6542
140.46
10,0806
174.52
12.0099
174.52
GPEN
GAN
6.0327
54,51
6.6999
118.66
6.6234
97.42
6.2069
97.42
GFPGAN
GAN
6.2330
53.53
7.0313
118.76
7.1427
104.63
6,9093
104.63
VQFR
Codebook
5.5092
54.08
6.5187
116.65
6.1202
88.52
5,4927
88.52
CodeFormer
Codebook
5.3945
54.82
6.4258
114.67
5.8507
87.49
5.2382
87.49
DifFace
Diffusion
6.8339
66.89
6.7321
119.07
7.3431
97.60
7.9364
97.60
DiIIBR
Diffusion
5,8180
44.42
6.2220
110.48
6.3345
91.95
5,6994
91.95
DR2
Diffusion
6,2578
$4.29
7.0488
123.53
8.2475
116.40
8.6105
116.40
PGDifT
Diffusion
5.1049
47.19
5,8718
111.72
5,3651
89.69
4,9502
89.69
PMRF
Flow
5.1464
54.97
5,5044
108.30
5,4790
83.76
5.2207
83.76
FlowlE
Flow 5.7388
51.03
6.3734
112.69
6,5751
92.94
5,5997
92.94
Vanilla-12SB
Bridge
5.8074
64.43
7.1276
123.82
6,9031
108.03
6.0377
108.03
P-I2SB
Bridge
5,2125
54.39
5.6425
108.07
5.2329
84.68
4,9056
69.62
Table Ablation studies CelebA-Test. (a) donates Vanilla- 12SB as the baseline, (b)-(c) compare different condition guidance, and (f)-(f) compare different Pseudo-Hashing strategies. "Iq" do- nates LQQ images as the condition in forward and reverse process and "da" donates degradation-awarc representation condition.
termed P-I2SB. This strategy circumvents the conflict be- tween blind inverse problems and the original definition of SB, which often leads to lack of feasible solutions. By deeply exploring and leveraging the capability of bridge- based models learn optimal transport, P-I2SB improves restoration performance. Extensive experiments the prac- tical BFR demonstrate that P-I2SB significantly improves the restoration capabilities of the Schrödinger Bridge.
Condition
Hashing strategy
Metries
Method
lq da
NIQEL
FIDJ
LPIPS
(a)
25.6026
6.5709
0.2851
(b)
25.9522
6.4849
0.2904
(c)
25.4932
6.3898
0.2945
Limitations and Future Work. Our approach is limited by the inability of artificially generated degradation pro- cesses to fully replicate real-world scenarios, such as water- marks, where restoration remains suboptimal. To address this, we propose three strategies: using segmentation to isolate primary subjects and reduce interference, expanding degradation types training datasets to enhance restora- tion through supervised learning, and employing contrastive learning improve degradation-aware information extrac- tion from real- world datasets via unsupervised learning. Additionally, the dominance of frontal face images in the training data limits the model's ability to restore faces in extreme poses. Future work will integrate methods like 3D Morphable Models (3DMMs) to extract pose informa- tion, aiding the learning from less represented samples and improving robustness across diverse facial orientations.
(d) ours
Noise
18.2646
5,7595
0.2678
(e) ours
13.9109
5,3300
0.2395
ours
14.2941
5.4401
0,2431
mance. Leveraging three established perceptual metrics, including FID, NIQE, and LPIPS, we quantify the con- tributions of each guidance component. The model that integrates both types of guidance achieves the best restora- tion results, indicating that learning the process between two distributions directly BFR using SB is not optimal. Appropriate guidance conditions can enhance the ability of the model to fit the inverse process.
Finally, and most importantly, we demonstrate the significant improvements brought by the pseudo-hashing strategy in the comparisons between periments (c) and (d)-(f). There marked enhancements in NIQE, FID and LPPPS. The visual results in Appendix G also reveal that the degradation artifacts are noticeably re- duced in (a)-(c), with our P-12SB exhibiting superior content preservation and local texture generation.
Pseudo Hashing Strategy.
7. Conclusion
In this paper, we introduced P-12SB for the challenge of blind face restoration, structured across two stages: the Pseudo-Hashing Module (PHM) and the Schrödinger Bridge Module (SBM). Our theoretical analysis identified funda- mental limitations of traditional Schrödinger Bridge (SB) solutions in handling blind problems and led to the devel- opment of three feasible ceauudnh strategies. Experi- mental results demonstrate the superior restoration perfor- mance on both synthetic and real-world datasets. This work offers a novel perspective on resolving challenges in blind problems by effectively computing the relationship between two distributions.
6. Discussion and Future Work
Advantages. This work investigates why image-to- to--imaee transfer methods like 12SB fail in blind inverse problems. It identifies the core issue as the limitations imposed by the optimality and reversibility of SB solutions, which result in poor restoration. Inspired by spatial geometry, we propose a straightforward correction strategy for training image pairs,

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Acknowledgements
Gao, S., Liu, X., Zeng, B., Xu, S., Li, Y., Luo, X., Liu, J., Zhen, X., and Zhang, B. Implicit diffusion models for continuous super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10021-10030, 2023.
This paper supported by National Key R&D (Research and Development) Program of China (2011YFA)000033) the National Natural Science Foundation of China (Nos. 12431012, U23B2012, 12471 (47)308) and Beijing Natural Sci- ence Foundation (1254050).
Goodfellow, I., Pouget-Abadic, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In NeurlPS, 2014.
Impact Statement
This project aims to provide an effective generative approach address the problem of blind face restoration (BFR). Our method theoretically investigates the ill-posedness of Schrödinger Bridge solutions in the context of BFR, propos- ing novel framework that directly uses degraded images as conditions for restoration through preprocessing of data distributions, eliminating the need for any feature extraction design. However, like other generative technologies, there is potential for misuse. Malicious entities could combine this method with target generative models to produce mislead- ing or harmful 2D facial images. Addressing these ethical issues is crucial, and future research in 2D generation must prioritize understanding and mitigating these risks to ensure the responsible application of these technologies.
Gu, Y., Wang, X., Xie, I.., Dong, C., Li, G., Shan, Y., and Cheng, M.-M. Vqfi: Blind face restoration with vector- quantized dictionary and parallel decoder. In ECCV, pP.
126-143. Springer, 2022.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. In NeurIPS, pp. 6840-6851, 2020.
Kaipio, J. and Somersalo, E. Statistical and computational inverse problems, volume 160. Springer Science & Busi- ness Media, 2006.
References
Karras, T., Aila, T., Laine, S., and Lehtinen, J. Progres- sive growing of gans for improved quality, stability, and variation. In ICLR, 2018.
Benamou, J.-D. and Brenier, Y. computational fluid me- chanics solution to the monge-kantorovin mass transfer problem. Numerische Mathematik, 84(3)375-333, 2000.
Karras, T., Laine, S., and Aila, T. A style-based genera- tor architecture for generative adversarial networks. In CVPR, pp. 4401-4410, 2019.
Chen, C., Li, X., Yang, L., Lin, X., Zhang, L., and Wong, K.-Y. K. Progressive semantic- aware style transformation for blind face restoration. In CVPR, pp. 11896-11905, 2021a.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space diffusion baatud generative models. Advances neural information processing systems, 35: 26565-26577, 2022.
Chen, T., Liu, G.-H., and Theodorou, E. A. Likelihood training of schrödinger bridge using forward-backward sdes theory, 2023.
Léonard, C. A survey of the schrödinger problem and some of its connections with optimal transport. arXiv preprint arXiv: /308.0215, 2013.
Chen, Y., Georgiou, T. T., and Pavon, M. On the relation between optimal transport and schrödinger bridges: A stochastic control viewpoint. Journal of Optimization Theory and Applications, 169:671-691, 2016.
Li, B., Zhang, Z., Nie, X., Han, C., Hu, Y., Qiu, X., and Guo, T. Styo: Stylize your face in only one-shot. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 4625-4633, 2025.
Chen, Y., Georgiou, T. T., and Pavon, M. Stochastic control liaisons: Richard sinkhorn meets gaspard monge on a schrodinger bridge. Siam Review, 63(2):249-313, 2021b.
Davison, A. C. Statistical Models. Cambridge Series in
Li, X., Chen, C., Zhou, S., Lin, X., Zuo, W., and Zhang, L. Blind face restoration via deep multi-scale component dictionaries. pp. 415, 2020.
Statistical and Probabilistic Mathematics. Cambridge Uni- versity Press, 2003.
Esser, P., Rombach, R., and Ommer, B. Taming transformers for high-resolution image synthesis. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pp.
Li, X., Zhang, S., Zhou, S., Zhang, L., and Zuo, W. Learn- ing dual memory dictionaries for blind face restoration. IEEE Transactions on Pattern Analysis and Machine In- telligence, 45(5):5904-59) 2022.
2021.
10

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. Swinir: Image restoration using swin trans- former. In Proceedings of the IEEE/CVF international 2021.
Schrödinger, E. Sur la théorie relativiste de l'électron I'interpnétation de la mécanique quantique. In Annales de l'institul Henri Poincaré, volume 2, pp. 269-310, 1932.
conference on computer vision, pp.
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprini arXiv:2010.025002 2020.
Lin, X., He, J.. Chen, Z., Lyu, Z., Fei, B., Dai, B., Ouyang. W., Qiao, Y., and Dong, C. Diffbir: Towards blind image restoration with generative diffusion prior. arXiv preprinl WrXiv:2308.15070 2023.
Tsai, Y.J.,, Liu, Y.-L., Qi, L., Chan, K. C., and Yang. M.- H. Dual associated encoder for face restoration. arXiv preprint arXiv:2388.07314 2023.
Liu, G.-H., Vahdat, A., Huang, D.-A., Theodorou, E. A., Nie, W., and Anandkumar, A. 12sb: srage-to-image schrödinger bridge. In Proceedings of the 40th Interna- tional Conference on Machine Learning, ICML 2023, pp. 22042-22062, 2023.
Villani, C. Topics in optimal transportation, volume 58.
American Mathematical Soc., 2021.
Wang, X., Li, Y., Zhang, H., and Shan, Y. Towards real- world blind face restoration with generative facial prior. pp. 9168-9178, 2021.
Liu, X.,
Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint ar arXiv:2209.033
Wang. Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, P. Image quality assessment: from error visibility struc- tural similarity. IEEE transactions on image processing, 13(4):600-61) 2004.
2022.
Menon, S., Damian, A., Hu, S., Ravi, N., and Rudin, C. Pulse: Self Self-supervised photo upsampling via latent space exploration of generative models. In CVPR, pp. 2437- 2445, 2020.
Wang. Z., Zhang, J., Chen, R., Wang, W., and Luo, P. Re- storeformer: High-quality blind face restoration from undegraded key-value pairs. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 17512-17521, 2022.
Mittal, A., Soundararajan, R., and Bovik, A. C. Making "completely blind" image quality analyzer. IEEE Signal processing letters, 20(3):209 212, 2012.
Wang, Z., Zhang, Z. Zhang, X., Zheng, H., Zhou, M.. Zhang, Y., and Wang, Y. Dr2: Diffusion- robust degradation remover for blind face restoration. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 1704-1713, 2023.
Nichol, A. Q. and Dhariwal, P. Improved denoising diffusion probabilistic models. In ICMI., pp. 8162-8171, 2021.
Ohayon, G., Michaeli, T., and Elad, M. Posterior-meat mean rec- tified flow: Towards minimum mse photo-realistic image restoration, arXiv preprint arXiv:2410.004115 2024.
Yang. P., Zhou, S., Tao, Q., and Lay, C. C. Pgdiff: Guiding diffusion models for versatile face restoration via partial guidance. Advances in Neural Information Processing Systems, 36, 2024.
Qiu, X., Han, C., Zhang, Z., Li, B., Guo, T., and Nie, X. Diffbfr: Bootstrapping diffusion model for blind face restoration. In Proceedings of the s1 ACM International Conference on Multimedia, pp. 7785-7795, 2023.
Yang, T., Ren, P., Xie, X., and Zhang, L. Gan prior em- bedded network for blind face restoration in the wild. In CVPR, pp. 672-681, June 2021.
Qiu, X., Li, B., Zhang, Z., Han, C., and Guo, T. Dr-bfr: Degradation representation with diffusion models for blind face restoration. arXiv preprint arXiv:2411.1053 2024.
Yue, 7. and Loy, C. C. Difface: Blind face restoration with diffused error contraction. 2022.
Zhang, K., Zuo, W., and Zhang, L. Learning a single con- volutional super-resolution network for multiple degrada- tions. In CVPR, pp. 3262-3271, 201 8a.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolee resouutosottion image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Zhang, R., Isola, P., Efros, A. A.. Shechtman, E., and Wang. O. The unreasonable effectiveness of deep features perceptual metric. In CVPR, pp. 586-595, 2018b.
Saharia, C., Ho, J., Chan, W. Salimans, T., Fleet, D. J., and Norouzi, M. Image super-resolution via iterative refinement. IEEE Transactions on Pattrnn Analysis and Machine Intelligence, 45(4):4713 2022.
Zhou, L., Lou, A., Khanna, S., and Ermon, S. Denoising dif- fusion bridge models. arXiv preprint arXiw:2309.16948, 2023.

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Zhou, S., Chan, K., Li, C., and Loy, C. C. Towards robust blind face restoration with codebook lookup transformer. Advances in Neural Information Processing Systems, 35: 18599-306111 2022.
Zhu, Y., Zhao, W. Li, A., Tang. Y., Zhou, J., and Lu, Flowie: Efficient image enhancement via rectified flow. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13-22, 2024.
12

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
A. Overview
this Appendix, additional proof, experimental details and experimental results are provided, including:
Details about the connection between different versions of OMT problem (in Sec. B);
Details about the proof of theorem 4.1 (in Sec. C);
Details about experiments setting (in Sec. D);
Details about preprocessing data (in Sec. E);
Details about loss analysis (in Sec. F);
More ablation results (in Sec. G);
Qualitative comparisons with SOTA methods in CelebA-Test and real-world datasets (in Sec. H and Sec. I);
Qualitative results about failure samples (in Sec. J).
B. OMT
In the 1940s, Kantorovich studied "relaxed" version of the Monge's problem (Chen et al., 2021b):
inf / cxz, v)r(drdy). =Ell(po.11) gad
(20)
Here, [[ v1) is the probability distributions on Id2d with marginals vo and v1. Indeed, [I v1) always contain the product measure Vo V/-
Though there are some different vrsions of OMT problem, they have some connection. If [f y) || y/1 and if 1 does not give mass to sets of dimension < nı, there exists unique optimal n² (Kantorovich) induced by unique (Monge) map T", such that T" = (I x TT) #vo where I is the identity map (Villani, 2021).
Let
/ ||x y1l2
W = inf
(21)
TEIl(Po.P5)
called as - quadratic distance. W3 is also the minimum value of the following optimal problem (Be- namou & Brenier, 2000) written as problem (2). Let {uz: 0 S t < 1} and {u" (t, E [0, x Rd) be optimal for (2). And now, 14 = [[1 - I + tT*
C. Proof
The proof of theorem 4.1. We proof the theorem for common Markov path measure P E D whose coordinate process has the SDE representation (9). The evolution its density function (1( can be characterized by the Fokker Plank equation on time interval (0, 1):
0p((x) - (at(z)p((z)) + 3 (x). Øt
(22)
By disintegration formula (10), we have
Pt(2) Por
(23)
We can get the disintegration formula of the density function:
(z(2) ( p (z) (dady). (ddddy)
(24)
13

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Suppose all the density functions are good enough, we do some differential operations on equation (24):
Öp((z)
Spza Pi rM Po (drdy)
(a) 01
Ōl dpp
(25)
f
0 (z)
Poi (dzdy),
paa
Ot
V (ax())p((z)) = V . (<(=)p; 2y (2) Po)
(6)
(26)
-fpu V (ac(z)p, TM (z) Pordddy).
=
Apr(z) = V (pr(2)
/ Raa
=V
(z) (drdy)
(c)
(27)
= / gad
Apt Pu (drdyy.
Then, let (a) + (b) (c):
Op)(2) V (a)(())))(())) 1 - Td
Øt
(28)
0pt (2)
1 Api (o) O.
(
Until now, we stress that what we consider a boundary value problem. That is, we want to find some boundary conditions subject our requirement which is to find suitable Poi so that the following equation can hold:
Op," T (2)
1 - (2).
(a)())
(29)
One obvious choice is the Dirac distribution. Let pon (Ir yddddy == Poi (dzdy). In the sense of Dirac distributions where po())= = 5zp (x) == 6(x x0) and pi (y) = Ey (v) = 8(y - 3n), we can get:
xo)5 v1) Pun (x, y) = Po()) (v) Seg (1)) (v).
(30)
We concluded that the bridges of path between pairs (x, v) can induce an identical SDE over time interval (0, 1) when take the boundary distribution as Dirac distributions centered at and y.
D. Implementation Details
We implement P-12SB using PyTorch leveraging eight 32GB Tesla V100 GPUs. The training process employs the Unet network as the predictor. The Adam optimizer is utilized with learning rate x 10-4, The training process comprises 40000 iterations with batch size of 64.
E. Pre-processing data
Since our P-12SB relies on supervised training, it necessitates paired low-quality (LQ) and high-quality (HQ) images. To simulate LQ images representative of real-world scenarios, we employ a degradation model generated through random sampling. Its generation formula (Zhang et al., 2018a; Wang et al., 2021) is presented in Eq.(1.), where ø is the HQ image, k, is the Gaussian blur kernel, represents the down-sampling scale factor, and q represents the JPEG compression of
14

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
Table 4. Degradation parameters in training stage.
Method
Blur list
Blur size
Blur sigma
Kernel prob
Noise range
Down sampling Jpeg quality
Gray prob
GPEN
'iso", 'aniso"
41
[0.1, 10]
[0.5, 0.5]
[0, 20]
[0.8, 8)
[60, 100]
0.2
GFPGAN
'iso", "aniso"
41
[0.1, 10]
[0.5, 0.5]
[0, 20]
[0.8, 8]
[60, 100]
0.01
P-12SB
'iso", "aniso"
41
[0.1, 10]
[0.5, 0.5]
[0, 20]
[0.8, 8)
[60, 100]
0.2
inference with [0,200]
Loss inference
Can 253
Neve
1.285
Step
Stop
Figure 6. Loss in Inference. The right shows the loss across different steps during inference for 64 batch images. from slep = T to step 0. The left resents zoomed-in view of the curve from step 200 step O.
the image with quality factor q. To ensure direct comparability with the experimental results of recent BFR methods, we randomly sample the parameters a, r, 8, q from {0.1: 10), (0.8: 8), {0: 20). {60: 100). respectively.
[(x @ ka) 4r +nolIPEG,
(31)
This parameter selection is consistent with the established experimental settings in the field, as shown in Table 4. To enhance color adaptation and mitigate overfitting, we introduce gray color probability mechanism and apply horizontal flip transformations to the training data. The specific degradation handling functions are included in the python file.
F. Numerical Experiments
Figure illustrates the variation in loss at different steps during the reverse inference process. It can be observed that the loss for Vanilla-I2SB exhibits significant fluctuations, with substantial loss occurring at the initial stages of inference. This indicates that, under complex degradation conditions, the Schrödinger Bridge model fails to learn feasible solution, leading to severe oscillations at the endpoints of the constraints. In contrast, the three loss curves following the pseudo-hashing strategy preprocessing show a noticeably smoother loss at the early stages of inference.
G. More Ablation Results
Figure 7 illustrates the results of the ablation study through visualizations. Panel (a) depicts the Vanilla-I2SB method, serving as the baseline for comparison. Panel (b) utilizes low-quality (LQ) images as conditional guidance, while panel (c) incorporates both LQQ images and degradation representation for enhanced guidance. Panels (d) through (f) demonstrate the application of three distinct pseudo-random strategies. The visualizations indicate that although panels (a) to (c) effectively restore the general outlines of the images, they fail to capture fine local texture details, leading to an overall smooth and averaged appearance. This inadequacy is particularly evident in the restoration critical facial features, such as the eyes, where accuracy is compromised.
15

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
LQ
(a)
(c)
(d)
(e)
GT
Figure 7. Ablation studies on CelebA-Test. (a) donates Vanilla- 12SB as baseline, (b)-(c) compare different condition guidance, and (d)-(f) compare different Pseudo-Hashing strategies. Zoom in for best view.
H. Qualitative Comparisons in CelebA-Test
In Figure 8, present comprehensive comparison of P-I2SB with several state-of-the-art methods, including GFP- GAN (Wang et al., 2021), GPEN (Yang al., 2021), VQFR (Gu et al., 2022), CodeFormer (Zhou et al., 2022), Restore- Former (Wang et al., 2022), DMDNet (Li et al., 2022), DAEFR (Tsai et al., 2023), DifFace (Yue & Loy, 2022), DifBBIR (Lin et al., 2023), DR2 (Wang et al., 2023), PGDiff (Yang et al., 2024), FlowIE (Zhu et al., 2024) and PMRF (Ohayon et al., 2024). Our results indicate that P-I2SB excels in preserving the original identity and facial expressions of the images while generating natural and plausible texture details. This method surpasses other approaches in terms of visual fidelity and detail accuracy, offering superior performance in maintaining both the overall appearance and the intricate nuances of facial features.
I. Qualitative Comparisons in Real-world
Figures 9, 10, 11, and 12 present a comparative analysis of P-12SB across four real-world datasets: CelebChild, LFW, WedPhoto-Tees Test, and Wider. Figure shows comparative results low-quality images with serious degradation in real- world datasets. These comparisons are made against several state-of-the-art methods, including GPEN (Yang et al., 2021), GFPGAN (Wang et al., 2021), VQFR (Gu et al., 2022), CodeFonner (Zhou al., 2022). RestoreFormer (Wang et al., 2022). DMDNet (Li al., 2022), DAEFR (Tsai et al., 2023), DifFace (Yue & Loy, 2022), DifBBIR (Lin al., 2023), DR2 (Wang al., 2023), PGDiff (Yang et al., 2024), FlowlE (Zhu al., 2024), PMRF (Ohayon et al., 2024) and Vanilla-12SB (Liu et al., 2023). The results demonstrate that P-12SB generates more coherent and detailed outputs by effectively utilizing the Pseudo-Hashing Module (PHM) and Schrödinger Bridge Module (SBM). This approach circumvents the indirect process of extracting features from LQ images by directly using the raw LQ images in place of Gaussian noise as one endpoint of the data distribution the generative diffusion model. This ensures the existence of a theoretically optimal solution. The pseudo- heaudonhashing strategy significantly improves the feasibility of solutions in blind inverse problems based on the Schrödinger Bridge method. The generated images excel in preserving facial structure and generating local texture details.
J. Failure Samples
Figure 14 illustrates the failure cases of P-12SB. Specifically, when real-world images contain watermarks, the inference results of P-I2SB still exhibit residual artifacts. This issue primarily arises because the model is trained on synthetically generated degradation datasets, which fail to fully capture the complexity of real-world degradations. To address this
16

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
limitation, our future research will focus enhancing the robustness of the Schrödinger Bridge method by utilizing unsupervised learning augment the pseudo-hashing strategy, thereby extending its applicability to a broader range of blind inverse problems.
17

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
LQ
GPEN
RestoreFormer
DMDNet
DifFace
DiMBIR
PGDiff
PMRF
FlowIE
DAEFR
P-12SB
GT
LQ
GPEN
RestoreFormer
DMDNet
DifFace
DiABBR
PGDifT
PMRF
FlowIE
DAEFR
P-12SB
GT
LQ
GPEN
Restore
DMDNet
DifFace
DiABBR
PGDiff
PMRF
FlowlE
DAEER
P-I2SB
GT
Figure 8. Qualitative comparisons on CelebA-Test. Our P-12SB demonstrates superior performance in both detail enhancement and hue preservation, particularly inputs with severe degradation. Zoom in for best view.
18

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
LQ
GPEN
GFPGAN
VQFR
CodeFormer
RestoreFormer
DMDNet
Difface
D
PGDiff
PMRF
FlowlE
DAEFR
12SB
P-12SB
LQ
GPEN
GFPGAN
VOFR
CodeFormer
RestoreFormer
DMDNet
DifFace
DiMBIR
DR2
PGDif
PMRF
FlowiL
DAEER
12SB
P-12SB
LQ
GPEN
GFPGAN
VQFR
CodeFoommer
RestoreForreer
DMDNet
Diff'ace
DimBIR
DR2
PGDiff
PMRF
FlowiE
DAEFR
12SB
P-12SB
Figure Qualitative comparisons on Wider. terms both detail enhancement and hue preservation, our P-I2SB outperforms, particularly on inputs suffering from severe degradation. Zoom in for best view.
19

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
GPEN
GFPGAN
VQFR
CodeFormer
RestorcFormer
DMDNet
DifFace
DiMBIR
DR2
PGDif
PMRF
FIowIE
DAEFR
12SB
P-12SB
1QQ
GPEN
GFPGAN
VQFR
CodeFormer
RestoreFormer
DMDNet
DifFace
DimBIR
DR2
PGDiff
PMRF
FlowlE
DAEFR
12SB
P-12SB
LO
GPEN
GFPGAN
VQFR
CodeFormer
RestoreFormer
DMDNet
DifFace
DiABIR
DR2
PGDiff
PMRF
FlowlE
DAEFR
12SB
P-12SB
Figure 10. Qualitative comparisons WebPhoto. Demonstrating superior capability, P-I2SB significantly improves detail and huc retention, even on inputs with extreme degradation. Zoom in for best view.
20

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
LO
GPEN
GFPGAN
VOFR
CodeFormer
RestorFFormer
DMDNet
Difface
D
DR2
PGDiff
PMRF
FlowlE
DAEFR
12SB
P-12SB
1QQ
GPEN
GFPGAN
VQFR
CodeFormer
RestoreFormer
DMDNet
DifFace
DiMBIR
DR2
PODif
PMRF
FlowIE
DAEFR
D2SB
P-12SB
LQ
GPEN
GFPGAN
VQFR
CodeFormer
RestorcFormer
DMDNet
DifFace
DiABIR
DR2
PGDiff
PMRF
FlowlE
DAEFR
12SB
P-12SB
Figure 11. Qualitative comparisons CelebChild-Tess Our P-12SB excels in both enhancing fine details and preserving hues, especially when dealing with severely degraded inputs. Zoom in for best view.
21

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
PPA
PPA
IPA
PPA
PPa
LQ
GPEN
GFPGAN
VQFR
CodeFormer
RestorcFormer
DMDNet
DifFace
PA
tpa
DiABIR
DR2
PGDif
PMRF
FlowlE
DAEFR
12SB
P-12SB
LQ
GPEN
GEPGAN
VQFR
CodeFormer
RestoreFormer
DMDNet
DifFace
DiABIR
DR2
PGDiff
PMRF
FlowlE
DAEER
12SB
P-12SB
GPEN
GFPGAN
VQFR
CodeFormer
RestoreFormer
DMDNer
DifFace
Di
DR2
PGDiff
PMRF
FlowlE
DAEFR
12SB
P-12SB
Figure 12. Qualitative comparisons on LFW. The P-12SB showcases exceptional performance in detail refinement and color fidelity. particularly on heavily deteriorated inputs. Zoom for best view.
22

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
LQ
GPEN
GFPGAN
VQFR
CodeFormer
Restore
DMDNet
DifFuce
M
N
DiMBIR
DR2
PGDif
PMRF
FlowlE
DAEFR
12SB
P-12SB
LQ
GPEN
GFPGAN
VQFR
CodeFormer
RestoreFormer
DMDNet
DifFace
DiABIR
DR2
PGDifT
PMRF
FlowIF
DAEFR
12SB
P-12SB
LQ
GPEN
GEPGAN
VQFR
CodeFormer
RestoreFormar
DMDNet
DifFace
DiABIR
DR2
PGDiff
PMRT
FlowlE
DAEFR
I2SB
P-12SB
Figure 13.. Qualitative comparisons on serious degradation in real-world dataset. The P-I2SB showcases exceptional performance in detail refinement and color fidelity, particularly heavily deteriorated inputs. Zoom in for best view.
23

---

Feature out! Let Raw Image as Your Condition for Blind Face Restoration
LQ
GPEN
RestoreFormer
DMDNet
DifFace
DifBBIR
PGDiff
PMRF
FlowlE
DAEER
P-12SB
GT
IR
O IC D
m
OI D
LQ
GPEN
Restore
DMDNet
DifFace
DiABIR
DK
O N
AFFIC I
D
OFF Ch
PGDiff
PMRF
FlowlE
DAEFR
P-12SB
GT
Figure Failure samples in CelebA-Test. Degradation artifacts such as watermarks can seriously affect the restoration results. Zoom in for best view.
24


Reviews:
[{"review_id": "V9fJVKADkH", "reviewer": "Reviewer_7utM", "summary": "This paper proposes P-I2SB, a novel framework for blind face restoration that leverages a pseudo-hashing strategy to preprocess image pairs and a Schrödinger Bridge Module (SBM) to learn optimal transport paths between LQ and HQ distributions. The key innovation lies in directly using raw LQ images as endpoints in the diffusion process, addressing limitations in existing Schrödinger Bridge (SB) methods related to solution optimality and reversibility. Experiments demonstrate state-of-the-art performance on synthetic and real-world datasets.\n\n## update after rebuttal\nThank you for your reply. After considering the comments from the other reviewers, I have decided to increase the score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the thorough review and insightful comments you have provided.\n> Q1. The paper does not thoroughly analyze the computational overhead of the pseudo-hashing strategies (Cat/Res/Noise-I2SB) compared to baseline methods, despite claiming retained inference speed.\n\n- **Computational Complexity**: In comparison to the baseline methods, our approach incorporates an additional pseudo-hashing module (PHM). The computational complexity of this module is as follows: for Cat-I2SB, it is $\\mathcal{O}(n)$; for Res-I2SB, it is $\\mathcal{O}(n)$; and for Noise-I2SB, it is $\\mathcal{O}(nT)$, where $T=10$ denotes the local number of steps in DDIM, and $n$ denotes the number of input images. This computational complexity is calculated with respect to processing $n$ input images. During inference, our method only adds the Inverse-PHM process relative to the baseline methods, without significantly increasing the complexity of this model or inference time.\n\n> Q2. What about the sensitivity to degradation types and the scalability to non-face domains?\n\n- **Sensitivity to Degradation Types**: The pseudo-hashing module (PHM) is specifically designed for image restoration under various unknown degradation. The three hashing strategies presented in the paper introduce degradation feature representations either directly or indirectly. This enables the module to perceive both the type and degree of degradation, thereby effectively handling low-quality image restoration across different degradation types.\n- **Scalability to Non-Face Domains**: Our method possesses scalability, and we have verified its effectiveness primarily in blind face restoration (BFR), where the demand for naturalness is exceedingly high. As noted in *\"Limitations of Face Image Generation\" [1]*, face restoration is more challenging than non-face tasks due to the need for naturalness and identity consistency. The restoration of identity, micro-expressions and other details is crucial, significantly validating the effectiveness of our method. Meanwhile, non-face models emphasize robustness, and currently, no general model can be directly applied to face restoration. We plan to extend our method to general restoration tasks in the future. The advancement of BFR is also critical to the development of face restoration field, and we will continue to advance BFR and expand our research into general domains.\n\n_reference: [1] Rosenberg H, Ahmed S, Ramesh G, et al. Limitations of face image generation. In AAAI, 2024, 38(13): 14838-14846._\n> Q3. Res-I2SB assumes that the degradation process can be modeled as a linear residual from HQ to LQ, but in reality, degradation (such as JPEG compression and non-uniform blurring) is mostly a nonlinear transformation, which may lead to insufficient fitting ability of the model for complex degradation.\n\n- It is instructive for us. The hashing module in Res-I2SB is not designed to construct a linear residual from LQ to HQ. Instead, it aims to model the transformation relationship between two data distributions using a SB-based model. This transformation is often a complex nonlinear one, which necessitates the use of the SB model for resolution. In Res-I2SB, $(I_{lq}, I_{lq} - I_{hq})$ is treated as a new image pair. We do not further explore its linear relationship but require that $I_{lq} - I_{hq}$ serve as a new boundary distribution to help find the optimal path.\n- In this context, the SB model aims to find the optimal joint distribution between given distributions, not to map low-quality to high-quality images. This joint distribution perspective is a broader generalization. Our goal is to establish the optimal dynamic diffusion path between these images, without assuming any specific mapping relationship, such as a linear constraint. Preliminaries (Sec.3) in main paper shows the evolution from a mapping to a joint distribution perspective in optimal transport problems.\n\n> Q4. Cat-I2SB, Res-I2SB, Noise-I2SB still require manual prior selection. I wonder if it would be better to introduce pre-trained degeneration classifiers to guide strategy selection?\n\n- Since we are dealing with a blind task where the degradation parameters and types are random and uncertain, potentially involving multiple combinations, classifiers cannot be used to clearly distinguish each category.\n- In terms of strategy design, Noise-I2SB handles differences caused by various degradations using degradation representation. We employ the formula $I_{hq}^{noise} = I_{lq}^d + \\lambda_d \\epsilon$, where $\\lambda_d$ represents the noise level, which is directly related to $I_{lq}^d$. Here, we set $\\lambda_d$ as the degeneration representation feature of $I_{lq}^d$ with an MLP layer.\n- Looking forward, we will focus on researching how to design PHM strategies without relying on manual prior selection. This will require further exploration of the relationship between the conditions for the optimal solution in SB and the boundary constraints.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "n/a", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "Strengths:\n\n1. The pseudo-hashing module (PHM) is a novel and theoretically grounded approach to ensure distribution alignment, enabling direct use of LQ images without complex feature extraction.\n\n2. Comprehensive theoretical analysis justifies the framework’s design and improvements over vanilla SB methods.\n\nWeaknesses:\n\nThe paper does not thoroughly analyze the computational overhead of the pseudo-hashing strategies (Cat/Res/Noise-I2SB) compared to baseline methods, despite claiming retained inference speed.", "other_comments_or_suggestions": "n/a", "questions_for_authors": "- What about the sensitivity to degradation types and the scalability to non-face domains?\n\n- Res-I2SB assumes that the degradation process can be modeled as a linear residual from HQ to LQ, but in reality, degradation (such as JPEG compression and non-uniform blurring) is mostly a nonlinear transformation, which may lead to insufficient fitting ability of the model for complex degradation.\n\n- Cat-I2SB, Res-I2SB, Noise-I2SB still require manual prior selection. I wonder if it would be better to introduce pre-trained degeneration classifiers to guide strategy selection?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DShRnKKZyH", "reviewer": "Reviewer_TmMa", "summary": "This paper proposes the Pseudo-Hashing Image-to-Image Schrödinger Bridge (P-I2SB), a novel framework for blind face restoration (BFR). The key insight of this paper is that using raw LQ images directly as the starting point for the reverse diffusion process is theoretically optimal.\nThe authors argue that Schrödinger Bridge (SB)-based approaches offer a better alternative to conventional diffusion-based BFR methods by explicitly learning the optimal transport path between the HQ and LQ distributions. To address limitations in existing SB models (such as optimality and reversibility issues), the paper introduces a Pseudo-Hashing Module (PHM) that preprocesses HQ images into pseudo-samples, ensuring a structurally similar distribution to LQ images. This facilitates an optimal and reversible transformation in the SB framework.\nExtensive experiments demonstrate that P-I2SB outperforms prior BFR methods in terms of texture realism, and preservation of facial details.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and constructive suggestions. Your input is crucial to refining and enhancing the quality of our paper.\n> Q1. The computational complexity of PHM preprocessing is not well analyzed. What is the computational cost of PHM preprocessing relative to standard diffusion models?\n\n - **Computational Complexity**: In comparison to the baseline methods, our approach incorporates an additional pseudo-hashing module (PHM). The computational complexity of this module is as follows: for Cat-I2SB, it is $\\mathcal{O}(n)$; for Res-I2SB, it is $\\mathcal{O}(n)$; and for Noise-I2SB, it is $\\mathcal{O}(nT)$, where $T=10$ denotes the local number of steps in DDIM, and $n$ denotes the number of input images. This computational complexity is calculated with respect to processing $n$ input images. During inference, our method only adds the Inverse-PHM process relative to the baseline methods, without significantly increasing the complexity of this model or inference time.\n\n> Q2. More ablations on different hashing strategies are required. The paper could include more ablation studies to quantify the impact of PHM on transport efficiencyexplicitly.\n\n- We compared the ablation experiments of different strategies in **Table 3** in main paper and **Figure 7** in the appendix. The related comparison results are provided here again. Additionally, we considered combining all three strategies. Due to time constraints, this was only validated in a toy experiment, as shown in **Sec.4.3 (Toy Exploration and Analysis)**, and more results are shown at the following link: [https://anonymous.4open.science/r/P-I2SB](https://anonymous.4open.science/r/P-I2SB). \n\n- **Table.3 Ablation studies** on CelebA-Test. (a) donates Vanilla-I2SB as the baseline, (b)-(c) compare different condition guidance, and (d)-(f) compare different Pseudo-Hashing strategies. \"lq\" donates LQ images as the condition in forward and reverse process and \"da\" donates degradation-aware representation as condition.\n\n| Method          | Condition | Condition | Hashing | Metrics | Metrics | Metrics |\n|-----------------|:---------------:|:---------------:|:--------------------:|:----------------:|:---------------:|:-----------------:|\n|          | lq | da | strategy | NIQE↓ | FID↓ | LPIPS↓ |\n| (a)             |                 |                 |                      | 25.6026          | 6.5709          | 0.2851            |\n| (b)             | ✓               |                 |                      | 25.9522          | 6.4849          | 0.2904            |\n| (c)             | ✓               | ✓               |                      | 25.4932          | 6.3898          | 0.2945            |\n| **(d) ours**    | ✓               | ✓               | Noise                | 18.2646      | 5.7595          | 0.2678            |\n| **(e) ours**    | ✓               | ✓               | Cat                  | 13.9109          | 5.3300      | 0.2395        |\n| **(f) ours**    | ✓               | ✓               | Res                  | 14.2941          | 5.4401          | 0.2431            |\n\n> Q3. Limited real-world evaluations—most experiments use synthetically degraded images. Could adaptive pseudo-hashing improve restoration in real-world scenarios?\n\n- We conducted tests on four datasets in total. Among them, CelebA-Test consists of synthetically degraded images, while ***LFW, CelebChild, WebPhoto-Test, and Wider*** are real-world image datasets. **Table 2** and **Figure 5** in main paper quantitatively and qualitatively compare the restoration performance on these four real-world datasets. Additionally, in **Appendix Sec. J**, we provide a detailed comparison of the restoration effects across the four real-world datasets using Figures 9, 10, 11, and 12.\n- The reason for selecting these four datasets for testing is that previous SOTA methods have been validated on these publicly available datasets. They include various low-quality images with different levels of degradation found in real-world scenarios. This allows for a fairer comparison between our method and the SOTAs.", "claims_and_evidence": "The authors claim:\n1.Schrödinger Bridge-based approaches can offer theoretically optimal transport paths for image restoration.\n2.The Pseudo-Hashing Module (PHM) improves SB-based restoration by ensuring reversibility and distribution alignment between LQ and HQ samples.\n3.P-I2SB achieves state-of-the-art (SOTA) results in BFR, outperforming existing methods in quality and efficiency.\nthese claims are supported by theoretical analysis and quantitative results.", "methods_and_evaluation_criteria": "The paper presents a strong methodological foundation, leveraging optimal transport theory to redefine BFR as a Schrödinger Bridge problem. The proposed PHM preprocessing step ensures that LQ and HQ distributions align better, improving reversibility in the transformation.", "theoretical_claims": "The paper provides a mathematical justification for using Schrödinger Bridge models in BFR. The derivations appear correct, but the paper could include more ablation studies to quantify the impact of PHM on transport efficiency explicitly.", "experimental_designs_or_analyses": "The experimental setup is solid", "relation_to_broader_scientific_literature": "This work builds upon Diffusion-based image restoration and Schrödinger Bridge methods in generative modeling.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n(1) Theoretical novelty: Reformulates BFR as an optimal transport problem, leading to a more principled approach.\n(2) Practical improvements: Achieves better texture details and inference efficiency than prior methods.\n(3) Conceptually elegant: The pseudo-hashing mechanism is a clever solution to the non-reversibility issue in SB models.\nWeaknesses:\n(1) The computational complexity of PHM preprocessing is not well analyzed.\n(2) More ablations on different hashing strategies are required.\n(3) Limited real-world evaluations—most experiments use synthetically degraded images.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. Could adaptive pseudo-hashing improve restoration in real-world scenarios?\n2.What is the computational cost of PHM preprocessing relative to standard diffusion models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BW2QwJZjbc", "reviewer": "Reviewer_hmwC", "summary": "The authors present Pseudo-Hashing Image-to-Image Schrödinger Bridge (P-I2SB), a novel framework inspired by optimal mass transport. By correcting data distributions and effectively learning the optimal transport path between them, it enhances the restoration capabilities of Schrödinger Bridge (SB). Experimental results demonstrate that P-I2SB achieves state-of-the-art performance in blind face restoration (BFR), producing more natural textures compared to previous methods.\n\n## Update after rebuttal\n\nThe authors have addressed most of my concerns. However, one point remains unclear: whether commonly used data augmentation techniques in image restoration—when applied alone, without the involvement of SwinIR or other external components—can help the Vanilla-I2SB model better identify the optimal path. Clarifying this would further illuminate the distinction between the effects of the PHM module and those of data augmentation. I will keep my score unchanged.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comprehensive review and valuable insights. \n> Q1. Their approach resembles a data augmentation technique, like in DiffBIR, suggesting that pairing I2SB with it might yield similar results, indicating potential simplicity.\n- **Our P-I2SB is not a data augmentation method.** Data augmentation involves using certain properties of images, such as rotation and brightness, to augment the actual numerical distribution of training data without altering the semantic information of the images. In contrast, the strategy design in pseudo-hashing module (PHM) is intended to satisfy the boundary constraints in SB models, requiring the direct or indirect involvement of degradation representations, and this hashing design is reversible.\n- **DiffBIR** consists of two stages: first, the removal of degradation information, followed by generating a high-quality image that balances quality and identity. This is **completely different** from PHM. In our first stage (PHM), we do not remove any degradation information; instead, we rely on an SB-based model to learn indirect restoration relationships.\n\n> Q2. They provide limited evidence that the paths are optimal; additional analyses, like those in Appendix Figure 6, would help substantiate these claims.\n- **Existence of Optimal Path in P-I2SB**: We first establish the existence of the optimal path in our P-I2SB. Our model is constructed based on SB theory, as outlined in the Preliminaries (Sec.3). The consistency between Equation (7) and the optimal transport problem in Equation (2) reveals why the solution of the SB model is the optimal transport path.\n- **Monge's Problem**, linked to image restoration, seeks the optimal mapping $T$ between two distributions under a certain cost metric. While it can theoretically map low-quality to high-quality images, its practical scope is limited by mapping constraints. Therefore, the optimal transport problem introduces a version that finds the optimal joint distribution, ensuring a solution always exists. This is depicted in Equation (2). In BFR problems, the relationship between low- and high-quality images is more complex. To leverage the existing I2SB model, as discussed in Preliminaries (Sec.3), we propose PHM to address these complexities.\n- **Additional Experiments and Analyses**: To analyze the superiority of the path found by our P-I2SB, we compared P-I2SB with the baseline method on the loss analyses along the path $\\{x_t\\}, t \\in [0,1]$. The results can be found at the following link: [https://anonymous.4open.science/r/P-I2SB](https://anonymous.4open.science/r/P-I2SB).\n$$\nforward: X_t\\sim q(X_t|X_0, X_1)=\\mathcal{N}(X_t;\\mu_t(X_0, X_1), \\Sigma_t),\ninverse: \\hat{X_t} \\sim p(X_t|X_0^\\epsilon,X_{t+1})=\\mathcal{N}(X_t;\\mu_\\sigma(X_0^{\\epsilon}, X_{t+1}), \\Sigma_\\sigma).\n$$\n\n> Q3. Lines 73 and 147 contain incorrect double quotation marks.\n\nThank you for pointing this out. Due to compilation issues, we have used **bold** and *italic* formatting instead to avoid errors related to the use of double quotation marks. The revised version is as follows:\n- the ***optimal transport*** between images is not unique \n- a ***relaxed*** version introduced by Kantorovich\n\n> Q4. Line 164 references “problem (2),” but the text does not provide a corresponding explanation.\n- Thank you for your insightful question, which helped us identify an error in Equation (7). The corrected version is as follows, where $\\bar{v}(t,x):=f_t(x)-\\frac{\\sqrt{\\beta_t}}{2}\\nabla\\log\\bar{p}(t,x)$ and $p(t,x)\\mathrm{d}x = \\mu_t(\\mathrm{d}x)$.\n$$\n\\inf_{(p,v)} \\int_0^1\\int_{\\mathbb{R}^{d}} [\\frac{1}{2}||v(t,x)-\\bar{v}(t,x)||^2 + \\frac{\\beta_t}{8}||\\nabla\\log\\frac{p(t,x)}{\\bar{p}(t,x)}||^2]p(t,x)\\mathrm{d}x\\mathrm{d}t. \\tag{7}\n$$\n- **The text \"as shown in problem (2)\" means it is actually the same as equation (2)**. This implies that the SB problem is linked to the optimal transport problem. Furthermore, $p(t,x), (t\\in[0,1])$ obtained by solving the SB model represents a form of optimal transport path.\nThe phrase \"as shown in problem (2)\" is incorrect, and we will correct it to \"just like equation (2)\".\n$$\n\\inf_{(\\mu,v)} \\int_0^1{\\int_{\\mathbb{R}^{d}} ||v(t,x)||^2 \\mu_t(\\mathrm{d}x)}\\mathrm{d}t. \\tag{2}\n$$\n\n> Q5. What is the significance of designing Cat-I2SB? If only the first three channels are retained, equation (12) can be simplified to $(I_{hq}, I_{lq}^d)$.\n- The Cat-I2SB is designed to **implement PHM strategies by hashing training image pairs** from $(I_{hq}, I_{lq}^d)$ to $(I_{hq} \\oplus I_{lq}^d, I_{lq}^d \\oplus I_{lq}^d)$, enabling the creation of varied training pairs for different degradations $d$.\n- In Equation (12), only the first three channels are used in the final data. However, this hashed data is utilized during P-I2SB training, where the diffusion network's input and output channels are adjusted. The original pairs $(I_{hq}, I_{lq}^d)$ remain unhashed. This modification increases the parameter size of Unet, approximately 558.6MB.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes, the proposed method achieves state-of-the-art performance on the BFR task.", "theoretical_claims": "Yes, The theoretical claims and proofs appear similar to those of I2SB. As I'm not an expert in this area, I did not notice any specific issues.", "experimental_designs_or_analyses": "Yes, I have carefully reviewed the experimental design and the corresponding ablation studies, and I did not encounter any issues.", "relation_to_broader_scientific_literature": "This paper builds upon the I2SB (ICML 2023) framework by integrating a novel pseudo-hashing preprocessing strategy.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper is well-structured and clearly presented.\n2. The experiments are comprehensive, with thorough ablation studies.\n3. The authors provide ample discussion, addressing both the method’s advantages and limitations.\n\nWeaknesses:\n1. While the authors claim that blind tasks do not align with the typical Monge optimal transport framework—due to the non-uniqueness of “optimal transport” between images—this purportedly contradicts the optimality principle of a reversible SB and complicates constructing a consistent I2SB model. However, their proposed approach appears more akin to a data augmentation technique. Similar to the data augmentation method in DiffBIR, pairing I2SB with such an approach could potentially achieve similar results, suggesting the method may be overly simplistic.\n2. Although the authors assert that their approach learns more optimal transport paths than existing methods, they offer limited analyses or evidence to validate that the paths are indeed optimal. Additional experiments or analyses, such as those found in Appendix Figure 6, would help substantiate these claims.", "other_comments_or_suggestions": "1. Lines 73 and 147 contain incorrect double quotation marks. \n2. Line 164 references “problem (2),” but the text does not provide a corresponding explanation.", "questions_for_authors": "What is the significance of designing Cat-I2SB? After performing the cat operation, the resulting number of channels differs from other methods. If only the first three channels are retained, equation (12) can be simplified to $\\(I_{hq}, I_{lq}^{d})$.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "juqlysPZTb", "reviewer": "Reviewer_UKRe", "summary": "This paper proposes the Pseudo-Hashing Image-to-image Schrodinger Bridge (P-I2SB) framework to enhance the restoration potential of Schrodinger Bridge (SB) by correcting data distributions and effectively learn the optimal transport path between any two data distributions. This approach preprocesses HQ images during training by hashing them into pseudo-samples according to a rule related to LQ images. This guarantees optimal and reversible solutions in SB, enabling the inference process to learn effectively and allowing P-I2SB to achieve state-of-the-art results in BFR.\n\n## update after rebuttal: The rebuttal has addressed certain formatting issues in the manuscript. I respectfully maintain my original rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review of the paper structure and formatting, which is greatly appreciated.\n> Q1. In Table 1, the reference numbers for SoTA methods should be listed.\n\n- Due to the ICML citation format not using numbers, the references were too lengthy to include directly in the table. Instead, we noted the references in the Comparison Methods section of Sec. 5.1. We have now supplemented the references and adjusted the font size in the table to ensure that they can be properly displayed in the main paper in future versions.\n> Sec.5.1 (Line 361 - Line 370) **Comparison Methods** We compare Pseudo-Hashing with recent BFR methods, including PSFRGAN (Chen et al., 2021a), GFPGAN (Wang et al., 2021), GPEN (Yang et al., 2021), VQFR (Gu et al., 2022), CodeFormer (Zhou et al., 2022), RestoreFormer (Wang et al., 2022), DMDNet (Li et al., 2022), DAEFR (Tsai et al., 2023), DifFace (Yue & Loy, 2022), DR2 (Wang et al., 2023), PGDiff (Yang et al., 2024), DiffBIR (Lin et al., 2023), PMRF (Ohayon et al., 2024), FlowIE (Zhu et al., 2024) and I2SB (Liu et al., 2023) .\n\n| Metrics        | Input  | GPEN | GFP | Restore | DMDNet | DAEFR | DifFace | DiffBIR | DR2 | PGDiff | PMRF | FlowIE | I2SB | **P-I2SB** |\n|----------------|--------|------|-----|---------|--------|-------|---------|---------|-----|--------|------|--------|------|------------|\n|                |        | *CVPR* | *CVPR* | *CVPR* | *TPAMI* | *ICLR* | *TPAMI* | *ECCV* | *CVPR* | *NIPS* | *ICML* | *CVPR* | *ICML* |            |\n|                |        | (Yang et al., 2021) | (Wang et al., 2021) | (Wang et al., 2022) | (Li et al., 2022) | (Tsai et al., 2023) | (Yue & Loy, 2022) | (Lin et al., 2023) | (Wang et al., 2023) | (Yang et al., 2024) | (Ohayon et al., 2024) | (Zhu et al., 2024) | (Liu et al., 2023) |            |\n| SSIM ↑         | 0.6460 | 0.6777 | _0.6827_ | 0.6219  | 0.6727 | 0.5892  | 0.6494  | 0.6570  | 0.6554 | 0.6220  | 0.6815 | 0.6479 | **0.7047** | 0.6581     |\n| PSNR ↑         | 24.921 | 25.423 | 25.401   | 24.206  | 25.318 | 22.439  | 24.055  | 25.297  | 24.194 | 22.920  | _26.001_ | 24.594 | **26.174** | 25.405     |\n| FID ↓          | 93.564 | 22.508 | 20.676   | 17.080  | 22.790 | 18.295  | 19.654  | 19.288  | 32.628 | 22.547  | _14.248_ | 21.393 | 25.6026 | **13.910** |\n| NIQE ↓         | 9.1407 | 6.7775 | 6.7324   | _5.3440_ | 6.7038 | 5.3992  | 6.1638  | 6.4053  | 8.1487 | 5.4556  | 5.6228 | 6.3571 | 6.5709 | **5.3300** |\n| LPIPS ↓        | 0.5953 | 0.2956 | 0.2823   | 0.2702  | 0.2965 | 0.2695  | 0.3052  | 0.2689  | 0.3447 | 0.3011  | _0.2413_ | 0.2623 | 0.2851 | **0.2395** |      |", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "It is the first work to introduce the Pseudo-Hashing Module (PHM) and the Schrodinger Bridge Module (SBM) theories into blind face restoration.\nExperimental results demonstrate the superior restoration performance on both synthetic and real-world datasets.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "The authors conducted extensive experiments to validate their proposed method.", "relation_to_broader_scientific_literature": "This paper complements the exploration of blind face restoration, making a certain contribution.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "In Table 1, the reference numbers for SoTA methods should be listed.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes P-I2SB, a novel blind face restoration framework based on Schrödinger Bridge and inspired by optimal transport theory. By hashing HQ images into pseudo-samples aligned with LQ distributions, it ensures optimal and reversible restoration, achieving state-of-the-art results with better textures and efficiency. The paper receives all positive reviews, with one reviewer upgrading their score from weak accept to accept after the rebuttal. Based on the overall enthusiasm and the strengthened justification provided during the rebuttal, the paper is recommended for acceptance. The authors are encouraged to incorporate the clarifications and discussions from the rebuttal into the final version.
Decision: Accept (poster)