Self-Supervised Selective-Guided Diffusion Model for Old-Photo Face Restoration
Authors: Wenjie Li
Xiangyi Wang
Heng Guo
Guangwei Gao
Zhanyu Ma
Venue: NeurIPS 2025 poster ()
Keywords: Old-Photo Face Restoration
Primary area: deep_learning

Abstract: Old-photo face restoration poses significant challenges due to compounded degradations such as breakage, fading, and severe blur. Existing pre-trained diffusion-guided methods either rely on explicit degradation priors or global statistical guidance, which struggle with localized artifacts or face color. We propose Self-Supervised Selective-Guided Diffusion (SSDiff), which leverages pseudo-reference faces generated by a pre-trained diffusion model under weak guidance. These pseudo-labels exhibit structurally aligned contours and natural colors, enabling region-specific restoration via staged supervision: structural guidance applied throughout the denoising process and color refinement in later steps, aligned with the coarse-to-fine nature of diffusion. By incorporating face parsing maps and scratch masks, our method selectively restores breakage regions while avoiding identity mismatch. We further construct VintageFace, a 300-image benchmark of real old face photos with varying degradation levels. SSDiff outperforms existing GAN-based and diffusion-based methods in perceptual quality, fidelity, and regional controllability. Code link: https://github.com/PRIS-CV/SSDiff.

Self-Supervised Selective-Guided Diffusion Model for Old-Photo Face Restoration
Wenjie Li", Xiangyi Wang', Heng Guo Guangwei Gao?, Zhanyu Ma'
|Beijing University of Posts and Telecommunications Nanjing University of Science and Technology
[cswjli, guoheng, mazhanyu/@bupt. edu. csggao@gmail.com
Abstract
Old-photo face restoration poses significant challenges due compounded degra- dations such as breakage, fading, and severe blur. Existing pre- pre-trained diffusion- guided methods either rely on explicit degradation priors or global statistical guidance, which struggle with localized artifacts or face color. We propose Self- Supervised Selective- Guided Diffusion (SSDiff), which leverages pseudo-reference faces generated by pre- rre-trained diffusion model under weak guidance. These pseudo-labels exhibit structurally aligned contours and natural colors, enabling region-specifie restoration via staged supervision: structural guidance applied throughout the denoising process and color refinement in later steps, aligned with the coarse-to-fine nature of diffusion. By incorporating face parsing maps and scratch masks, our method selectively restores breakage regions while avoiding identity mismatch. We further construct VintageFace. 300-image benchmark of real old face photos with varying degradation levels. SSDiff outperforms exist- ing GAN-based and diffusion- methods in perceptual quality, fidelity, and regional controllability. Code link: pettps:/ggitae com/ Wwtp/PISSCCUSSDDII.
Introduction
Recent image restoration methods may causing misaligned features. To address this. face restoration
excel in generic scenes but struggle with facial regions,
4 has been extensively studied, R 9 10 for real scenarios. However, the
particularly in blind face restoration (BFR) restoration of old face photographs remains relatively underexplored, as such low-quality (LQ) images often suffer from compounded degradations, including fading, breakage, and severe blurring, which pose challenges for constructing large-scale paired datasets for supervised learning.
Although existing learning-baad based BFR methods, such as GPEN 77. CodeFormer 9 DifFace [10] and DifBBR [1I. can alleviate blurring in old face photos, they struggle to address degradations like breakage and color fading that are present in the training data, as illustrated in Fig. T Recently, the powerful generative capability of diffusion priors 12 13 has opened up new possibilities for face restoration. By designing effective guidance strategies for pre-trained diffusion models, the denoising trajectory can be steered toward task-specifi objectives, enabling adaptation to various [6) in train-free manner. Such flexibility and low cost make pre-trained diffusion-guiin guided methods well-suited for old-photo face restoration.
zero-shot restoration tasks [12 14 [5]
Nevertheless, existing pre-trained ieftusiionguiider methods 14
for old-photo face restoration are limited by rigid, pre-defined priors. For example, DDNM 14 constructs a closed-form solution space using the pseudo-inverse of linear degradation model and adjusts the denoising trajectory through backpropagation. However, as shown in Fig. degradation in old photos is complex and
16]
"Correpooding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

LQ
CodeFormer
DifFace
10
DiABIR
DDNM
PGDiff 16
Ours
Figure 1:
Existing face restoration methods face challenges with complex degradations in old face photos, which may result in residual breakage unnatural facial colors. In contrast, method restores sharp facial structures and natural facial color, and no visible breakage regions.
nonlinear, resulting in DDNM only being able to process old photos with relatively simple degradation. PGDiff [16] improves generalization by introducing pre-defined global attribute priors, such color statistics and smoothing semantics. However, it faces two key limitations: i) smoothing semantic priors is less robust to large breakage regions and may cause the model collapse facial structure consistency during early sampling, leading to noticeable artifacts. ii) enforcing global color statistics (i.e., matching the average mean and variance of color channels in FFHQ [17] dataset) may induce sampling bias toward local extrema, resulting in significant image color inhomogeneity.
Motivated by these limitations, investigate the generative behaviors of mre-trained diffusion models under different guidance and discover a key insight. Our Insight: As shown in Fig. 2 we observe that, under weak guidance from degraded input with appropriate 8, a pre-trained generative face diffusion model can produce face images that, despite deviating from the original identity (low feature IOU, e.g., eyes, mouth, efc), exhibit structurally similar facial contours (high contour IOU), natural color tones (saturation distribution close to HQ faces in FFHQ), and effective restoration in breakage regions (low edge strength variation). This motivates us use such images as gveudo-referenee to guide the restoration of facial colors and breakage regions. Mcanwhile, facial components like eyes and mouth are not involved in guiding features prevent low-fidelity facial results.
Building on this insight, we introduce Supf-Supe Selective Guidedd Diffusion (SSDiff), a training- free framework that performs self-suuprv by using pseudo-referrecce generated under weak guidance. To fully exploit the guidance potential of pseudo-references, SSDiff introduces a staged guidance scheme aligned with the coarse-to-fine nature of diffusion: restoration-oriented guidance, including covering both structure-aware and breakage completion, is applied throughout the process, while color refinement is introduced in later steps, discussed in Section 3.2 Additionally, we integrate face parsing maps and scratch masks obtained from inputs to selectively guide breakage completion and face coloring without disturbing ddentity-sensitiv features. Through our design, our restorations exhibit high IOU of face contour and components, smooth breakage regions, and consistent color saturation, as shown in Fig. 2 Therefore, SSDiff produces high- figh-quality results even under complex degradations, as shown in Fig. Moreover, benefiting from face parsing- based region selection, SSDiff can also perform region-specce stylized restoration, such as hair and lips.
Contributions. i) We propose SSDiff, training- free framework that introduces reference learning paradigm, where pseudo-references generated under weak guidance from a pre-trained face diffusion model are suitable to steer the reverse process for high-fidelity old-photo face restoration. ii) We design a staged, region-specific guidance scheme that combines structure-aware and breakage-aware restoration and late-stage color guidance, enabling robust restoration across diverse degradation types. iii) We construct VintageFace, a real-world benchmark of 300 old face photos with varied degradations, and demonstrate that SSDiff consistently outperforms existing GAN-baser and zero- shot diffusion methods in perceptual quality and fidelity of restorations.
2
Related Work
Blind Face Restoration. Blind face restoration aims to recover HQ face images from LQQ face im- ages with complex degradations. To enhance visual quality, previous approaches integrate pre-trained

---

Frosde
HC antur (thrno
100
Peteetaa N7 L22 Trred
Gradimnt Ci
Stong
D0
Arg Cheat
2000
Oerratie
-2
20
Pre-Trained Face Generation Diffusion Model
Costear
Pestore
Elgye Strongts
158
201
Figure 2: (Left) LQ inputs provides supervision via a loss gradient scaled by 8, aligning reverse diffusion outputs toward it. Strong guidance improves fidelity but harms perceptual quality, while weaker guidance enhances perceptual quality but reduces fidelity. (Right) Statistical analysis on our dataset shows that pseudo-labels generated under weak guidance (s 1e 3) preserve facial contours similar to LQ inputs, exhibit low similarity in semantic regions (e.g., eyes, mouth, nose), and achieve smooth breakage regions with color saturation distributions close HQ faces.
generative adversarial networks 18 trained on HQ faces into the restoration pipeline. Methods such as GFPGAN [6. GPEN [7]. GLEAN 19]. SGPN [20), and FREx [2]] leverage facial structure cues
extracted from degraded inputs and inject them into a pre-trained StyleGAN [17] for restoration. To mitigate the uncertainty of StyleGAN in continuous feature space, approaches like VQFR (8),
[ and DAEFR [22 adopt pre-trained VQGAN 23 retrieve semantically similar facial features from a discrete codebook. However, GAN-based methods suffer from mode collapse. With the rise of diffusion probabilistic models 24), diffusion-based methods like DR2 (25) employ diffusion models to resist complex degradations, while DifFace [10] models the posterior distribution from LQ HQ images to generate high-fidelity results. PFStorer (26). PLTrans (27). and Wave- Face (28] further introduce high-frequency features to constrain the denoising process for improved fidelity. Nevertheless, these methods remain limited in handling old-photo face restoration, which may involve breakages and fading that don't It appear in training. Our method, situated within the blind face restoration paradigm, is more robust to such challenging degradations in old face photos.
CodeFormer
Diffusion Prior for Restoration. Methods leveraging diffusion priors for restoration can be broadly categorized into two groups. The first group, including DitBBIR (II) StableSR [29] OSEDiff (30], and OSDFace 31,, incorporates pre-trained diffusion models such as Stable Diffusion B into the restoration pipeline and fine-tunes controllable modules for generative restoration. These approaches perform well on degradation types seen during training but fail to generalize to unseen ones. The second group focuses on zero-shot restoration by developing efficient guidance strategies for pre- trained diffusion models. Among them, DDRM [32], DDNM 14), GDP [15, and T21 (3 adjust pre-trained diffusion models by estimating degradation process each iteration, using fixed linear 14 33 or parameterized degradation models [5] to steer intermediate outputs towards the input LQ images. PGDiff [16] and AGLLDiff (34) guide restoration by modeling desired attributes of HQ images, such as color statistics and structural information from clean datasets. Our method belongs to the second category but differs by using pseudo-referent guidance instead of explicit attribute modeling. Furthermore, unlike applying global priors, SSDiff selectively guides facial sub-regions in stages, enabling finer control under complex degradation.
operators [32
generated through weak
3 Methodology
3.1 Preliminaries
Denoising Diffusion Probabilistic Models (DDPM). DDPM generative models [24] consist of two processes: i) the forward process progressively adds Gaussian noise / to the input wo with a predefined variance schedule { producing a noisy sample æ, at timestep / according to:
[ D2 = N xz: V - 1..1-1...T).
3,01-1 + Bie, C~N(00,)
(2)
ii) The reverse process is parameterized by learned denoising network cs. typically UNet [35 architecture, which iteratively parameterize the mean value #4p (z,, t) by network Eg:
1 1 - B,I,
Po (2x- = N (#:-1: 1: M4 (xt, t), Eo(xi, Eo(x,, = 1 1- a
(3)
3

---

Ye
y/
Selective Coloring
Restore
ys
P
SJ
27
Style Transfer
Face
Denoise
Face Parsing P
Mask M
Strong
Sw
TT)
Label
Xy
X₁
X1
Output Xg
Pre- Pre-Trained Face Generation Diffusion Model
Figure 3: Overview of our method. Face parsing maps P and scratch masks M are estimated from input Yo using pre-trained face parsing network [36] and pre-trained scratch detection network 37 "Weak Guide" refers to the case shown in Fig. 2 at = le-3, i.e., Sw = le
Mg (Dt,
1 Vot
ax Eo (1)2,)
- ai
at 11= Be,
04 II 0z.
VI 1t
t=0
In practice, we can directly estimate 2n = xt/ vāa Ea (2 (ode..tv(/e (1-V/(11a)p))) (1-ā+)/āg from Eg.
=0:0////
Classifier Guidance. Classifier guidance [12 introduces semantic control into the reverse diffusion
process by leveraging gradients from a pre-trained classifier C.p (y|z). Instead of training a conditional diffusion model, this approach modifies the denoising step of unconditional model by shifting the predicted mean based on the classifier gradient Va log (ylæ)) evaluated at the intermediate prediction. This guides the generation trajectory toward regions image space that are more likely to match the target semanties:
Pa.á ( y) RJ .N (Ea (zt: Eg(3r, t) v log co (z)lz) Iz=)o(mr.))) Ex( f))
(5) guidance signal that biases the sampling
where the gradient g = v log Co (y) (r) distribution toward the target class y. Egg (æt, t) denotes the degree of diffusion the sample.
Self-Supervised Selective-Guided Diffusion Model (SSDiff)
As shown in Fig aim to restore an old face photo LQ input Yo E CC*Hx which may suf- fer from breakage, fading, and blurring, into HQ face photo output wy E RCx xW, To achieve this,
Image Gradient Magaltie (1)
Later Stage
0.8
unify self-supervised loss gradient guidance der generalized reverse diffusion update, where task-specific losses L1, La and L3 as pseudo clas- sifiers to replace traditional classifier gradients, i.e.,
Early Stage
0.6
s - -v lyy a: 12 in (5. To enable stage-aware supervision, SSDiff integrates two parts: Selective Restoration Guidance, which is applied across the entire reverse trajectory recover face and breakages, and Selective Coloring Guidance, which is applied later steps to refine facial color. This temporal separation is empirically motivated by the gradient behavior observed Fig where image gradient magnitude remains steady in early steps but drops sharply later steps, indicating shift from structural restoration to detail refinement, Our design naturally aligns with the coarse-to- fine denoising curriculum [38, 39 diffusion models, where early reverse steps reconstruct spatial structures while later steps refine details. Combined with Section 3.1 Algorithm I summarizes the full sampling process SSDiff. Below, we elaborate on its two key components: Selective Restoration Guidance and Selective Coloring Guidance.
V. log (y)l)) X (y-
0.2
SSDIff w/o Selective Coloring
0.0
0,27
Reverse DifTusion Step
early stage, gradient drops are minor, as updates focus on coarse structures like facial contours, which occupy a small region. In the later stage, gradients drop sharply due refine- ments face details, which cover most face.
Figure

---

Algorithm 1 : Staged Self-Supervised Sampling in our SSDiff.
Input: Old face photo yo- face parsing map P and mask M obtained from Yo-
Require:
Pre-trained diffusion model (Ho( (), Eg(ar, ()), face selector S, style transfer ST,
pseudo label y, under weak guidance Sw, edge magnitude D, strong guidance s, for restoration.
Output: High-quality face photo Vo.
2T ~ NO0,, I); y. + (vp: M), yn + - M. P). 0, + S (yp, P)
for t from to do
3t - vur 21- V I- -vaae 10 (x()
di
Se f S (tt, M). In t (d), M, P)
repeat
if
> Ti then V C Va, Li + Va L2 = Va.la
D Stage Restoration Guidance (D(y,) - -a(,,)
1V. 2e + V
F- ~ N (14g (TZ, 1) 82 (2 1) VL, Eg Ft. 1))
else
Stage II: Restoration & Coloring Guidance
iT, f
va: Dr; - V 1- ED (2,, t): 2t + S (ăr, P). + ST(@ 0p)
VL = VL + (V s. Li3 = -a,ll 1,, - . 12
x ~ N (No( 1) 8, ( 1) L. Eo ()),
end if
until N steps
T--1 N (ua( s, Sar V) Eo(ar,
end for return 20
Selective Restoration Guidance. Selective restoration guidance is applied throughout the reverse diffusion process, with its main role being to accurately restore the face image and fill in the breakage areas. Given the input Po. we first use an arbitrary pre-trained face restoration model [9] 7 to obtain an initially restored semantics, resulting in an output Dn that remains broken. We treat non- broken regions y. of Dn as plausible and use them to guide the frozen pre- pre-trained face generation model to perform reverse diffusion in non-broken regions of faces. We define this part loss L between mon-masked face regions ye of Do and the reverse diffusion-guided output i.,, formulated as L [l(1 M) (y, -2) Its loss gradient Va, Lı provides supervision to the reverse diffusion process, ensuring faithful restoration in non-broken face regions. M E {0, 13 is a mask obtained by a pre-trained seratch detector (37 from yo- @ is Hadamard product.
For breakage regions, previous methods [1 16 typically set gradients to zero and rely on the pre-trained face generation model to complete them automatically. However, this often results in incomplete completion, especially in areas such as background and skin. To address this issue, we introduce smoothness-guiie
loss C2 which explicitly enforces surface smoothness consistency between pscudo-labels and guided features within breakage regions, mitigating color inconsistency caused by direct guidance. We first face parsing maps 36 P and scratch masks (37] M derived from input Yo to select plausible breakage guide regions &n and y, within breakage parts from reverse diffusion and pseudo-labels This selection focuses on part of the breakage regions M guide C M such as background, skin, and hair, while excluding semantically sensitive areas (e.g., eyes, eyebrows, mouth) that may differ from original identity. Furthermore, we extend (E) the mask of breakages along the horizontal and vertical dimensions of M guide. to alleviate incomplete scratch detection 337]_ The loss L2 for the breakage regions is defined as:
L = ||D (y... E guide)) - D (ă,,. (MM guMde))))
(6)
where the function D (y, M) E IR HxW computes the average per-pixel gradient magnitude within the masked area. Let Yi.; denote the pixel value at the spatial location (i, then D is defined as:
D My.M) Vio y (M,, @ Mij+1)+ |y,., Mitiol (M,,, @ M,+1,). (7)
The above operations ensure smooth consistency of gradients in breakage regions to restore breakages. Then, we leverage the loss gradient V*+ C2 as guidance signal during the reverse diffusion process, enabling our model to actively refine breakage regions. For the remaining small portion of breakage, we still allow models to complete them automatically, as pre-trained face generation models generally perform well in restoring semantically critical facial regions such eyes, eyebrows, and mouth.
5

---

Table Quantitative comparison old-photo face images. which are categorized simple, medium, and hard based on degradation degree. Bold and underlined indicate best and second best results. Our Appendix provides results on public test sels, including
(0), and CelebA- -Chlid 16/.
Type
Metric
GAN-based
Diftusion-based (Learning)
Diffusion-based (Train-free)
BOPB
Code 9
DifFace [T0]
DímBIR (TT
DDNM [14]
PGDiff [T6]
Ours
FID4
177.19
151.01
132.14
163.47
177.82
136.73
129.13
BRISQUEL
22.76
6.69
23.13
17.08
51.74
8.77
6.71
Simple
TOPIQT
0.4444
0.6005
0.5954
0.5474
0.3976
0.6216
0.6494
MAN-IQA1
0.3124
0.3876
0.3676
0,3568
0,2742
0.3646
0.40M5
FID
176.11
189.37
142.22
198.84
173.96
142.56
128.31
BRISQUE!
22.54
8.29
23.45
12.27
52.78
10.22
7.29
Medium
TOPIQT
0.4337
0.6123
0.5917
0.5684
0.3929
0.6060
0.6412
MAN-IQA-
0.2975
0.3789
0.3511
0,3509
0,2721
0.3456
0.3949
FIDL
218.35
173.89
145.85
200.38
203.13
166.42
122.59
BRISQUEL
28.91
10.51
26.41
14.36
54.17
13.60
8.95
Hard
TOPIQ
0.3865
0.5670
0,5450
0,5424
0,3447
0.5747
0,5977
MAN-IQAT
0,2454
0.3379
0.3159
0.3140
0.2243
0.3159
0.3575
LQ
BOPB 37
CodeFormer
!
DifFaace 10
DiABIR 11
DDNM 14
PGDiff 16
Ours
Figure 5: Qualitative comparisons of old-photo face images with existing methods.
Selective Coloring Guidance. Selective coloring guidance is applied only after facial identity features such as eyes, nose, and mouth have stabilized during the reverse diffusion process. At this stage, as shown in Fig identity features essentially fixed, and remaining steps focus refining details, which reduces the risk of introducing identity inconsistency through color guidance. Given intermediate images IT, from the reverse diffusion and pseudo-label Yp- first extract regions related to the skin, including face and neck, using face parsing maps M, resulting in æ. and 9p However, direct supervision from D to &. is suboptimal due potential variations in facial components, To address this, we employ a pre-trained color style transfer network 40) to transfer the color style of p to I obtain color-adapted intermediate result y. which subsequently guides the remaining reverse diffusion steps for faithful and coherent face coloring. To supervise this process, we define color consistency loss C3 between &, and y., which enforces global color coherence while allowing structural flexibility in regions sensitive to identity. Specifically, L3 is defined as:
(8) In this phase, by using the loss gradient Vg to guide tones and shadows skin regions, and combining this with the earlier selective restoration guidance, facial coloring realism, and identity preservation are improved while supporting further fine-grained facial detail reconstruction.
C3 13 22 y. Transfrr (æ., Up.)
Overall Guidance. Our guidance strategy staged to align with the progression of the reverse diffusion. In the early phase of denoising, where i E [Ti, models focus solely on structural restoration. We apply guidance losses L Li + L2, where Li encourages fidelity in intact regions and C2 enforces smoothness in breakage regions. Once the facial identity stabilizes, in the later phase t [0, Ti], we introduce a selective coloring loss L3 and formulate the full loss as L L1 Le C3-
In this framework, pscudo-labels generated under a weak guidance scale sw are used to supervise the actual restoration process, which is conducted under stronger guidance scale s,, To ensure coherent interaction between the two stages, we adopt joint optimization perspective:
L= arg min
E (+t, a (x7) (, tx (+)) + (ly - -s(())) (9)

---

LQ
BOPB 37
CodeFormer
9
DifFace 10
DiffBIR
[I]
DDNM 14
PGDiff 16
Ours
Figure 6: Quantitative comparisons of results on no breakage and large-region breakage images.
Theoretically, optimal pair of weak and strong guidance scales (s achieves equilibrium, where neither under- nor -erddgud dominates, enabling effective perceptual restoration.
4
Experiments
4.1 Experiments Setting
Datasets and Metrics. Our method training-free and does not require training dataset. For evaluation, we randomly collect 300 old face photographs from the Internet as our benchmark, called VintageFace. All images are cropped and aligned to 512 x 512 using the open-source FaceLib library'. We further categorize them into three levels of degradation: simple, medium, and hard, based image quality, 100 images each. Details the benchmark are provided our Appendix. Since ground-truth is unavailable, we follow previous works [1O] [1] and adapt no-reference metrics, like FID [I]. BRISQUE (4), TOPIQ [43], and MAN-IQA
to evaluate perceptual quality. Furthermore, we provide qualitative results and CLIP-based 45 identity distance assess restoration fidelity.
Implementation Details. We use the pre-trained real-time model BiseNet (36) and the scratch detection model from (37 to obtain face parsing maps and scratch masks from inputs, respectively. Our pre-trained diffusion model is unconditional denoising network trained on FFHQ [17 datasets, which learns to reconstruct high-quality faces from pure noise over T 1000 steps. We restore breakage facial regions during the first 600 steps, and apply color migration in the remaining 400 steps (7₁ 400). The strong gradient factor of our SSDiff is 8, = 3.5e PGDiff [16] is also set to this value for fairness. For the style transfer process shown in Fig. 3 we adopt pre-trained lightweight model CAP-VSTNet 40), without using its built-in segmentation module for boundary invariance. All experiments implemented in PyTorch framework NVIDIA RTX 4090 GPU.
4.2 Comparisons with Existing Methods
We compare SSDiff with BOPB (37], which is specialized for old photo restoration; Code- Former [9]. DifFace [10], and DiffBIR [T.. which are designed for blind face restoration; and
DDNM [14] and PGDiff [16), which guide pre-trained diffusion for zero-shot old-photo restoration.
As shown Table a quantitative compar- ison of our SSDiff with the above methods demonstrates that our approach not only significantly outperforms existing methods in FID, BRISQUE, and TOPIQ, which reflect image features quality, but also achieves more natural image quality in the cross-modal human perceptual metric MAN-IQA. However, these metrics alone do not fully capture the fidelity of the re- sults. Therefore, Figure. presents qualita- tive comparisons across three types of old face photographs: simple, medium, and hard. Our method produces face images with natural color, clear texture, and minimal damage, while maintaining high fidelity and preserving the identity of the input face images. Furthermore, we provide more old-face photo restoration results in our Appendix.
Offferet
17
Different Up Color
Lobe
UP
LQQ
Pseudo-1 Label Ours (Orginal)
Ours (Style)
Figure 7: Our method allows stylized restoration of facial components specified by reference to pseudo-labels on inputs.
Facelib: https:/giitubbooomssjjjado/////p

---

Evaluation on Diverse Old Face Photos. As shown in Fig. 6 we validate the generality of our SSDiff on a broader set of old face photos. This includes no-breakage images where the mask is all zeros, and large-region breakage images where the mask indicates missing regions to be filled. In all scenarios, our method performs robustly, achieving high-quality face coloring and region completion while restoring sharp facial contours. It significantly outperforms existing methods that are limited to a single function, such as coloring, completion, or suffer from overall poor results.
Region-Specifie Stylized Restoration. Since the face parsing map is insensitive to scratches and highly structured features of faces, we can reliably select specific facial components for targeted restoration using fixed semantic labels. As shown in Fig. [7] our restored faces exhibit unnatural violet around the lips. This issue can be mitigated by explicitly selecting the lip region via the parsing map and increasing the guidance strength for this component. Furthermore, the same strategy can be applied to adjust hair color, for example, modifying yellowish hair to dark tone by reference to pseudo-labels This property allows us to fine-tune facial local components with suboptimal details and achieve more visually appealing face restoration results.
Weal Daklo
Stoing
Overk Gedanne 23
100
60
80
Input
Pscudo-1
Ours-1
Pscudo-2
Ours-2
Ours-1
Ours-4
fx Se
60
Optimal
Optimal
25
35
45
26
Stoongth
Pseudo-Ours
Ours
Pscudo-3
Ours-3
Ours
Ours-4
Figure 8: Ablation studies on the effect of different weak guidance factors sw for generating pscudo labels and strong factors s, in restoration results. When 8, Le and sw 3.5e our results show the best guidance loss, facial tones, facial details, and fidelity of identity.
4.3 Ablation Studies
Self-Supervised Strategy. When the self-supervised strategy is removed, namely when the pseudo-label v, is no longer used, we disable both the selective restoration guidance and selective coloring guidance associated with yp in breakage regions. As shown in Table results show significant drop in visual qual- ity without the self-supervision strategy. Furthermore.
Table 2: Ablation studies of our self- supervised strategy on "medium" type data.
Superviee
FIDJ
TOPIQ"
BRISQUEL
145.7
0.618
7.92
128.3
0.641
7.29
based Fig. 2 we explore the appropriate range for weak and strong guide factors sw. and sa. which determines the reliability of generated pseudo-labels and results. As shown in Fig. 8 an excessively large s.. leads to incomplete removal of pseudo-label scratches, leaving visible artilacts in restorations. Conversely, an overly small causes some identity feature mismatch, resulting in distorted eyes and dark facial tones. Thus, we sw
1e and s, 3.5e to obtain best results.
Selective Guidance. We first evaluate the effective- ness of the three guided losses: L1. L2, and L3- As shown in Table all three gradient-based losses con- tribute to improved restoration. As shown in Fig. 10 visualization results further reveal that L1 and La en- hance the sharpness of facial structures, reducing vis- ible breakage artifacts in old photos. In contrast, L3 plays critical role in facial coloring, ensuring a natural tone in results. Furthermore, since C3 is applied after the facial structure has stabilized, we investigate the appropriate value for Ti. As shown in Fig, letting T denote the total number of diffusion steps, setting Ti 0.4T (fluctuating with degradations) stabilizes the facial structure and enables the guided coloring to produce more natural skin tones.
Table 3: Ablation studies of our selective guidance strategy on "medium" type data.
Guidance
FID
TOPIQ
BRISQUE]
Ours
128.3
0.641
7.29
L
142.8
0.602
8.18
w/o La
132.1
0.630
7.43
w/o Ls
136.3
0.624
7.65
Next, we justify the design of selective restoration. The pseudo-label y, is not used to guide high- semantic facial components such as the eyes, nose, or mouth, as these regions are identity-sensitee and pscudo-label inconsistencies may degrade fidelity, shown in Fig. 10 For selective coloring,

---

IQQ
Pscudo-Label
Ti 0.67
Ti = 0.57
T 0.47
= 0.37
0.27
Ti 0.17
Figure 9: Ablation study on different I3 values in reverse diffusion, i.e., when to start guiding face coloring, on restoration results. Best facial tones and details are observed T₁ = 0.47.
we examine why guidance is applied only to skin regions, rather than full faces, why we don't 't just color transfer. As shown in Fig- [0] due to the limited performance of the pre-trained color transfer network and large color variations of overall faces, the overall direct migration full face guidance leads to unsatisfactory results. Therefore, we restrict color guidance to skin regions.
Fidelity of Restoration. Due the presence of severely degraded facial features in a part of inputs, it is challenging reliably evaluate identity consistency across all samples. To ensure meaningful measurement, we select 50 LQQ in- puts with relatively well-preserved facial features and com- pute CLIP-based [45] identity feature distance between the restorations and inputs. As shown in Table T our method achieves low identity distances, indicating good fidelity in restoring identity-relevan relevant features.
Table 4: Quantitative comparison on identity distances (L) with LQ inputs.
BOPB [
DifFace [T0]
DifTBIR (
0.2348
0.2451
0,2223
DDNM [14]
PGDif [16
Ours
0.2386
0.2859
0.2280
4.4 Limitations and Future Works
Our method improves the robust- ness of face restoration for old photos and performs well even under complex breakage. How- ever, as shown in Fig. [T] when large stained regions are present, our method may mistakenly re- store these stains part of the face, such as skin, leading to un- clean results. This issue stems from the diffusion model's ten- dency over-rely surround- ing context in severely degraded regions, leading to incorrect restorations when visual cues ambiguous. In the future, plan to incorporate fine-grad -ea-ggriidd region-level annotations for such cases to reduce failure modes.
Psoudo
urtind .5
Our
in b2
w/o bg
Y a facid fontones Whe eyes eyebrows or mouch in facia! dreokoge replons.
Style Tronsley
Figure 10: (Left) Ablations of L1, La. L3. and why can't 't guide "re" in breakages in Lz. (Right) Ablations of color-guided areas, and why don't we just color style transfer (ST) instead of guiding coloring?
IQQ
BOPB 37
CodeFormer
DifFace
TO
DiMBTR
ID]
DDNM LB
PGDif 16
Ours
Figure 11: Our Failures. When large stains appear, they may be misinterpreted as a part of facial or hair structures and color by our method, leading to unnatural artifacts and distorted local details.
5
Conclusion
In this work, we present SSDiff, specialized framework for old-photo face restoration that leverages the generative capacity of pre-trained diffusion models through self-supervised pseudo-reference guidance. Unlike existing approaches that rely explicit degradation models or pre-defined high- quality face attributes, SSDiff guides the denoising process using features selected from generated pseudo-references and facial geometry priors such as parsing maps and scratch masks at different sampling stages. This design enables the selective restoration of plausible facial regions of old photos while maintaining high visual fidelity. Extensive experiments demonstrate that SSDiff not only handles complex old-photo degradations effectively but also enables region-specins stylization of faces, offering flexible and robust solution for challenging old-photo face restoration tasks.

---

6 Acknowledgments
This work was supported by National Natural Science Foundation of China (Grant No. 62472044, U24B20155, 62225601, U23B2052), Deijing-Tianjin-Hebee Basic Research Funding Program No. F2024502017, Hebei Natural Science Foundation Project No. 242Q0101Z, Beijing Natural Science Foundation Project No. L242025.
References
[1] Shangquan Sun, Wenqi Ren, Xinwei Gao, Rui Wang, and Xiaochun Cao. Restoring images in adverse weather conditions via histogram transformer. In ECCV, pages 111-129. Springer, 2024.
[2] Rongyuan Wu, Tao Yang. Lingchen Sun, Zhengqiang Zhang. Shuai Li, and Lei Zhang. Seesr: Towards semantics-aware real-world image super-resolution. In CVPR, pages 25456-25467, 2024.
[3] Marcos V Conde, Gregor Geigle, and Radu Timoftc. Instructir: High-quality image restoration following human instructions. In ECCV, pages 1-21. Springer, 2024.
[4] Wenjie Li. Mei Wang. Kai Zhang, Juncheng Li, Xiaoming Li. Yuhang Zhang, Guangwei Gao, Weihong Deng, and Chia- Wen Lin. Survey on deep face restoration: From non-blind to blind and beyond. arXiv preprint 2rXiv:2309.
2023.
[5] Wenjie Li, Heng Guo, Xuannan Liu, Kongming Liang, Jiani Hu, Zhanyu Ma, and Jun Guo. Efficient face super-resolation via wavelet-based feature enhancement network. In ACM MM, pages 4515-4523, 2024.
[6] Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. Towards real-world blind face restoration with generative facial prior. In CVPR, pages 9168 9178, 2021.
[7] Tao Yang, Peiran Ren, Xuansong Xie, and Lei Zhang. Gan prior embedded network for blind face restoration in the wild. In CVPR, pages 672-681, 2021.
[8] Yuchao Gu, Xintao Wang. Lianghin Xie, Chao Dong, Gen Li, Ying Shan, and Ming-Ming Cheng. Vqfr: Blind face restoration with vector-quantized dictionary and parallel decoder. In ECCV, pages 126-143. Springer, 2022.
[9] Shangehen Zhou, Kelvin Chan, Chongyi Li. and Chen Change Loy. Towards robust blind face restoration with codebook lookup transformer. In NeurlPs, volume 35, pages 30599 1000306111, 2022.
[10] Zongsheng Yue and Chen Change Loy. Difface: Blind face restoration with diffused error contraction. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2024.
[11] Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Bo Dai, Fanghua Yu, Yu Qiao, Wanli Ouyang, and Chao Dong. Diffbir: Toward blind image restoration with generative diffusion prior. In ECCV, pages 430-448. Springer, 2024.
[12] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurlPs, volume 34, pages 9780-8794, 2021.
[13] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, pages 10684-10695, 2022.
[14] Yinhuai Wang. Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In ICLR, 2023.
[15] Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, and Bo Dai. Generative diffusion prior for unified image restoration and enhancement. CVPR, pages 9935-9946. 2023.
[16] Peiqing Yang, Shangchen Zhou, Qingyi Tao, and Chen Change Loy. Pgdiff: Guiding diffusion models for versatile face restoration via partial guidance. In NeurlPs, 2023.
[17] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. CVPR, pages 0400- 2019.
[18] Ian J Goodfellow, Jean Pouget-Abadic, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurlPs, volume 27, 2014.
[19] Kelvin CK Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, and Chen Change Loy. Glean: Generative latent bank for large-facte factor image super-nesolution. In CVPR, pages -4245-1425, 2021.
10

---

[20] Feida Zhu, Junwei Zhu, Wenqing Chu, Xinyi Zhang. Xiaozhong Ji, Chengjie Wang, and Ying Tai. Blind face restoration via integrating face shape and generative priors. In CVPR, pages 7662-7671, 2022.
[21] Zhengrui Chen, Liying Lu, Ziyang Yuan, Yiming Zhu, Yu Li. Chun Yuan, and Weihong Deng, Blind face restoration under extreme conditions: leveraging 3d-2d prior fusion for superior structural and texture recovery. In AAAI, volume 38, pages 1263-1271, 2024.
[22] Yu-Ju Tsai, Yu-Lun Liu, Lu Qi, Kelvin CK Chan, and Ming-Hsuan Yang. Dual associated encoder for face restoration. ICLR, 2024.
[23] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873-12883 2021.
[24] Jonathan Ho, Ajay Jain, and Pieter Abheel. Denoising diffusion probabilistic models. In NeurlPs, volume 33, pages 6840-6851, 2020.
[25] Zhixin Wang. Ziying Zhang, Xiaoyun Zhang, Huangjie Zheng, Mingyuan Zhou, Ya Zhang, and Yanfeng Wang. D12: Diffusion-basse robust degradation remover for blind face restoration. In CVPR, pages 1704 1713, 2023.
[26] Tuomas Varanka, Tapani Toivonen, Soumya Tripathy, Guoying Zhao, and Erman Acar. Pfstorer: Personal- ized face restoration and super-resolution. In CVPR, pages 2372-2381, 2024.
[27] Lianxin Xie, Csbingbing Zheng, Wen Xue, Le Jiang, Cheng Liu, Si Wu, and Hau San Wong. Learning Regradation-unawaan representation with prior-based latent transformations for blind face restoration. In CVPR, pages 9120-9129, 2024.
[28] Yunqi Miao, Jiankang Deng, and Jungong Han. Waveface: Authentic face restoration with efficient frequency recovery. In CVPR, pages 6583-6592, 2024.
[29] Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin CK Chan, and Chen Change Loy. Exploiting diffusion prior for real-world image super-resolution. International Journal of Computer Vision, pages 21, 2024.
[30] Rongyuan Wu, Lingehen Sun, Zhiyuan Ma, and Lei Zhang. One-step effective diffusion network for real-world image super-resolution. In NeurlPy, volume 37, pages 92529-92555 2024.
[31] Jingkai Wang, Jue Gong, Lin Zhang, Zheng Chen, Xing Liu, Hong Gu, Yutong Liu, Yulun Zhang, and Xiaokang Yang. Osdface: One-step diffusion model for face restoration. In CVPR, 2025.
[32] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In NeurlPs, volume 35, pages 23593-23606, 2022.
[33] Kanchana Vaishnavi Gandikota and Paramanand Chandramouli. Text-guided explorable image super- resolution. In CVPR, pages 25900-25911, 2024.
[34] Yunlong Lin, Tian Ye. Sixiang Chen, Zhenqi Fo, Yingying Wang. Wenhao Chai, Zhaohu Xing, Wenxue Li, Lei Zhu, and Xinghao Ding. Aglldiff: Guiding diffusion models towards unsupervised training-free real-world low-light image enhancement. In AAAI, volume 39, pages 5307-5315, 2025.
[35] Olaf Ronneherger, Philipp Fischer, and Thomas Brox, U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pages 234-241. Springer, 2015.
[36] Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In ECCV, pages 325-341, 2018.
[37] Ziyu Wan, Bo Zhang, Dongdong Chen, Pan Zhang, Dong Chen, Jing Liao, and Fang Wen. Bringing old photos back to life. In CVPR, pages 2747-2757, 2020.
[38] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML, pages 41 41-48, 2009.
[39] Erdi Sayar, Giovanni lacca, Ozgur S Oguz, and Alois Knoll. Diffusion-basse based curriculum reinforcement learning. In NeurIPs, volume 37, pages 97587-97617 2024.
[40] Linfeng Wen, Chengying Gao, and Changging Zou. Cap-vstnet: Content affinity preserved versatile style transfer. In CVPR, pages 18300-18309, 2023.
[41] Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Making a "completely blind" image quality analyzer. IEEE Signal Processing Letters, 20(3):209 2012 2012.

---

[42] Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing, 21(12))4695-4708, 2012.
[43] Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Topiq: A top-down approach from semantics distortions for image quality assessment. IEEE Transactions on Image Processing. 2024.
[44] Sidi Yang. Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. Maniqa: Multi-dimension attention network for no-reference image quality assessment. In CVPR, pages 1191-1200, 2022.
[45] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In /CML, pages 8748-8766 8763, PmLR, 2021.
[46] Jiankang Deng. Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In CVPR, pages 4690-4699, 2019.
[47] Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super- resolution with pure synthetic data. In ICCV, pages 1990019144 2021.
[48] Wenjie Li. Heng Guo, Yuefeng Hou, Guangwei Gao, and Zhanyu Ma. Dual-domain modulation network for lightweight image super-resolution. IEEE Transactions Multimedia, 2025.
[49] Zongsheng Yue, Jianyi Wang, and Chen Change Loy. Efficient diffusion model for image restoration by residual shifting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims the abstract and introduction accurately reflect the paper's contributions and scope by emphasizing our contributions and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In Sec. 4.4 of the main paper, we discuss the limitations of the proposed method.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
13

---

Answer:
[NA]
Justification:
This paper does not include theoretical proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: In Sec. we extensively delineate all the details of our methodology, facilitat- ing the reproduction of all experimental content based on our method description.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
14

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification: To prevent potential plagiarism before the paper is accepted, we assure that upon acceptance, we will promptly provide link to the open-source code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
public/guides/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
Coaee//dddeSebmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Sec. 4.1. we provide a detailed description of the experimental hyperparam- eter settings, evaluation metrics, and the computational resources required.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[NA]
Justification: This is not the experimental direction of this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Sec. 4.1 we provide a detailed description of the experimental hyperparam-
eter settings, evaluation metries, and the computational resources required.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http://wwwwp..ccobb.b...ccccceee
Answer: [Yes]
Justification: Yes, our research fully adheres to the NeurlPS Code of Ethics guidelines.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our method aids old photos restoration.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
16

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
The paper does not involve the release of models or datasets that pose highrisk of misuse. The collection of old photographs faces are taken from the Internet.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper uses publicly available methods, all of which are properly credited in the main text.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
17

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not include new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This is not the experimental direction of this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
This is not the experimental direction of this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
18

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our paper does not include the usage of LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy battps://neune http://arrrieecc/oeeerrreee///// cc/Coonferences/2225//LL. for what should or should be described.
19

---

Appendix
A Details of Our Benchmark
First, illustrated in Fig. SS22 we present our criteria for categorizing our proposed VintageFace benchmark into simple, medium, and severe degradation levels. Specifically, we employ frozen CLIP model to compute the similarity between each old photo and textual description of degradation severity. Images are then ranked by their similarity scores and assigned to categories accordingly. To ensure accuracy, we further manually corrected small number of misclassified samples.
Second, as shown in Fig S13 we display representative examples from each degradation level in the VintageFace benchmark. These examples demonstrate varying degrees of blurring, fading, and structural damage, and are largely consistent with the classification criteria established in Fig S12
Scores
"an old-face photo with severe blur, heavy fading, and large visible breakage"
0.02
Simple 1/3
Simple 1/3
0.35
Old-face Photos
Pre-Trained CLIP $
Calculate Similarity
0.43
Medium 1/3
Manual Screening
Medium 1/3
0.52
0.62
Hard
Hard 1/3
1/3
0.94
Figure S12: Method for categorizing degradation types into simple, medium, and hard levels in our benchmark VintageFace for testing.
Simple
Medium
Hard
Figure S13: A showcase of representative facial images with varying degradation types from VintageFace benchmark. The benchmark includes faces across diverse genders, ages, and ethnicities.
B More Comparisons
First, we provide additional visual results in the appendix (e.g., ST6 Fig and Fig to complement the main text. These figures showcase the restoration performance old face
20

---

Table S5: Quantitative comparison on real- BFR benchmarks that contain old face photos, like WebPhoto-Tese and CelebA Child. Bold and underlined indicate best and second best results.
GAN-based
Dilfusion- bbased (Learning)
Diffusion-based (Train-free)
Dataset
Metric
GPEN 7
Code 9
DifFace [T0]
DitBBIR [
DDNM [T4]
PGDiff [[6
Ours
FID
101.3
83.2
89.1
91.8
165.6
96.1
86.9
WebPhoto-Tess
NIQE.
6.326
4.705
4.831
6.069
9.259
5.117
4.406
CelebA-Child
FIDI
113.0
T16.2
113.1
118.9
151.2
121.0
112.7
NIQE
4.945
4,983
4,818
5,549
6.576
5.070
4.524
photographs across diverse genders, ethnicities (Asian, European-American and Indian), and age groups. Our method effectively balances perceptual quality and identity preservation: the restored images exhibit minimal artifacts or breakage while maintaining faithful facial identity.
Second, we observe that widely used blind face restoration benchmarks, such as LFW-Test and CelebChild, also include substantial number of old face photos. However, these differ from our dataset in that they primarily exhibit blurring, with no significant structural damage and limited fading. To demonstrate the generalization and effectiveness of our method, compare it with state-of-the--e approaches on these two benchmarks. Following previous works (9] [6], we adopt FID and NIQE evaluation metrics. As shown in Table S5 our method achieves good quantitative results on both benchmarks. Furthermore, we provide visual comparisons in Fig. ST9, which reveal that our method not only effectively addresses blurring but also excels restoring facial color. This perceptual advantage, particularly in color restoration, is not fully captured by quantitative metrics.
Thirdly, VintageFace primarily consists of frontal photos, as portrait photos decades ago were typically studio-based, focusing on clearly capturing facial features, making profile shots rare. Additionally, eyewear was less common, resulting in fewer photos with glasses. Consequently, our data has fewer such samples. Nevertheless, as shown in Fig. S14, SSDiff performs robustly across these scenarios, including glasses, profile shots, and severe degradations, consistently yielding favorable results.
C
More Ablation
Robustness of Pre-trained Networks. Our SSDiff is generally robust to inaccuracies in external components (face parsing, scratch detection, style transfer). These networks only provide coarse directional signals during reverse diffusion, similar to classifier-guided diffusion, and are not strict constraints. As long as the guidance is not severely misleading, the strong generative prior of the frozen diffusion model dominates reconstruction. To quantify this robustness, as shown in the Table S6
Table S7 and Table S8 conduct ablations on the Medium type subset:
For parsing map networks, we introduce inaccuracies by replacing the original parsing maps with pseudo-label parsing maps of different strengths s, where the resulting errors are even larger than those observed in parsing maps under severe degradations (79% IOU). The resulting IoU with the original parsing map is: for s=2.5e-4, IoU=76% (24% discrepancy): for s=le-4, loU=70% (30% discrepancy); For scratch detection networks, we randomly flip 10%, 20%, and 30% of the masks of breakage regions to simulate situation where some of the breakages have not been detected; For style transfer networks, we weaken the style transfer guidance by reducing the style factor a from to 0.1 and 0.2, slightly affecting color and content.
Table S6: Parsing Maps.
Table S7: Scratch Masks.
Table S8: Style Transfer.
Ours 76% 70%
Ours 10%flip 20%ffip 30%flip
Ours (a=0) a=0.1 a=0.2
FID(L)
128.3 131.1 133.4
128.3 129.2
130.7
132.5
128.3
129.2 128.8
MAN-IQA(1) 0.395 0.391 0.382
0.395
0.391
0.379
0.381
0.395
0.396 0.392
Face (t)
0.985 0.955
0.974
0.953
0.937
0.977 0.959
Here, Face Similarity (range [0, 1J) denotes the cosine similarity between features (extracted with AreFace (46)) of the perturbed restoration and the original restoration (Ours). These results show that errors in the pre-trained networks are not severe, and the strong generative prior of the diffusion model can propagate the correct cues to other regions, preventing significant performance drops. This demonstrates that SSDiff is robust to these pre-trained networks.
21

---

Wearing Glasses
Different Pose
Hard Case
Input
Ours
Input
Ours
Input
Ours
Figure $14: Visualization of SSDiff under wearing glasses, different poses, and severe degradation.
Lateney. Our method is built upon existing pre-trained diffusion face generation models, where additional inference overhead mainly comes from four components: simple restore, face parsing network, scratch detection network, and a style transfer network. All components are lightweight and are executed at a single denoising step rather than throughout the entire process. Moreover, except for the style migration network, the other three can optionally be pre-processed offline. When all four components are executed online, the average latency for processing single old face photo is 95 ms on an NVIDIA GeForce RTX 4090. If the three offline-optional components are pre-processed. the average latency is reduced to 24 ms. In contrast, PGDiff [16] requires semantic information extraction each denoising step, introducing latency of about 10 s. Therefore, our method only introduces minimal latency existing pre-trained generation diffusion frameworks.
Computational Cost. As shown Ta- ble [S further compare the number of Parmas, FLOPs, and inference time of our method with existing diffusion-based face restoration methods 10 IT [6] We let the restore be performed offline; our method performs excellently. Our method is smaller in terms of the Parmas and FLOPs counts, especially compared stable diffusion-baan basodd methods like DifiBIR ..].
Table S9: Quantitative comparison on computational
costs with existing diffusion based BFR methods.
Costs
DifFace [1O
DitBIR [1]
PGDiff [16]
Ours
Parmas.
175.4M
1717M
47.7M
45.4MI
FLOPs.
268.8G
24234G
127.5G
120.7G
Crop
Redine
face Bestonation Method
Alig
Method
Non face
Regione
Method
Combine
Figure S15: (Left) Face images highly sensitive to artifacts, directly restoring photos containing faces with image restoration methods may results in visually disturbing results. (Right) A common strategy involves cropping and aligning facial regions, followed by restoration using face restoration methods, while non-facial regions are enhanced with image restoration methods to ensure visual perception. Therefore, old photo face restoration holds practical value for old photo restoration.
D Necessity of Old-Photo Face Restoration
While general image restoration methods ① aim to restore the entire image holistically, we argue that dedicated face restoration (6] is necessary and beneficial, especially in the context of severely degraded old portraits. As shown in Si3 directly applying general real-world image restoration models [47] to facial regions may introduce noticeable artifacts, even when these methods perform reasonably well on background areas. This is because facial regions are typically small size, contain rich structural priors (e.g., eyes, nose, mouth), and highly sensitive to local distortions. Artifacts in these regions are particularly perceptible and detrimental to human perception.
Similarly, old face photos suffer from unique degradation patterns such as heavy blurring, fading, and structural damage. Applying global restoration methods [47 to these faces without region-specific modeling frequently leads distorted identity features unnatural textures. Therefore, advocate for face-specific old photo restoration approaches [14] [6] that focus on preserving facial identity and fidelity, while allowing general old photo restoration techniques [37] handle the surrounding non-facial regions. This targeted strategy ensures high-quality restoration where perceptual sensitivity highest and complements broader restoration pipelines. Therefore, respectfully believe the task of old photo face restoration holds specifically practical value.
22

---

LQ
BOPB 37
GPEN 7
CodeFormer 9
DifFace [T]
DiffBIR II
ResShift [49]
DDNM [14
PGDiff [66
Ours
LQ
ВОРB (37
GPEN 7
CodeFormer 9
DifFace 100]
DiffBIR I1]
ResShift [49]
DDNM [14]
PGDiff [16]
Ours
LQ
ВОРB 37
GPEN 7
CodmFrrmer 9
DifFace
DifBBIR III
ResShift 49
DDNM [14]
PGDiff [6
Ours
LQ
BОPB 37
GPEN 7
CodeFormer 9
DifFace [10]
DilIIR |||
ResShift 49
DDNM [[4]
PGDiff [16]
Ours
Figure S16: Qualitative comparisons with existing methods VintageFace.
23

---

LQ
BOPB 377
GPEN 7
CodeFormer 9
DifFace TO]
DifIBIR II]
ResShift [49]
DDNM [14
PGDiff [6]
Ours
LQ
ВОРB (37
GPEN 7
CodeFormer h
DifFace [10]
DiffBIR II
ResShift (49
DDNM [14
PGDiff [16]
Ours
LQ
ВОРB 37
GPEN 7
CodeFormer 9
DifFace [0]
DifBBIR 11]
ResShift [49]
DDNM [14)
PGDiff [16]
Ours
LQ
ВОРВ 37
GPEN 7
CodeFormer 9
DifFace LO
DifiBIR |||
ResShift 49
DDNM [44]
PGDiff [66]
Ours
Figure S17: Qualitative comparisons with existing methods VintageFace.

---

LQ
BОPB 37
GPEN 7
CodeFormer 9
DifFace [T]
DifIBIR [I]
ResShift [49]
DDNM [14]
PGDiff [16]
Ours
LQ
ВОРB (37
GPEN 7
CodeFormer 9
DifFace 10]
DiffBIR [1]
ResShift [49]
DDNM [14]
PGDiff [16]
Ours
LQ
ВОРB 37
GPEN 7
CodeFrrmer (
DifFace 100
DiffBIR 11]
ResShift [49]
DDNM [14]
PGDiff [16]
Ours
LQ
BОPB 37
GPEN ?
CodeFormer 9
DifFace [I]
DifIBR ID|
ResShift 44
DDNM [14)
PGDiff [16]
Ours
Figure S18: Qualitative comparisons with existing methods VintageFace.
25

---

LQ
BOPB 377
GPEN 7
CodeFormer 9
DifFace [T]
DifIBIR I
ResShift [49]
DDNM [14
PGDiff [66
Ours
LQ
BOPB (37
GPEN 7
CodeFormer 9
DifFace 10]
DiffBIR [T]
ResShift (49]
DDNM [1]
PGDiff [16]
Ours
IQQ
BOPB 37
GPEN 7
Cormerrmer
DifFace 10]
DifBBIR [XI]
ResShift [49
DDNM [14]
PGDiff [16]
Ours
LQ
BОPB 37
GPEN ?
CodeFormer 9
DifFace [O
DilIBRR |
ResShift 49
DDNM [14]
PGDiff [66]
Ours
Figure S19: Qualitative comparisons with existing methods WebPhoto- and CelebA-Child.
26


Reviews:
[{"review_id": "mCITvxuQh2", "reviewer": "Reviewer_FoKK", "summary": "This paper introduces SSDiff, a training-free framework designed to address specific degradations in old face photos, such as breakage and color fading. SSDiff leverages a pre-trained face diffusion model to generate a pseudo-label, which encapsulates both structural and color information. Based on this pseudo-label, a staged, region-specific guidance scheme is proposed to enhance the restoration process. Additionally, a real-world benchmark, VintageFace, is introduced, and SSDiff demonstrates superior performance compared to existing methods on this dataset.", "questions": "1. The caption for Figure 3 should be revised for clarity and precision.\n2. It is unclear what \\hat{x_{t}} represents.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. The idea of the paper is straightforward: it uses a clean pseudo-label to guide the inverse process. Additionally, the staged approach proposed by the authors for handling different types of information is quite reasonable. These concepts are supported by analytical results presented in Fig. 2 and Fig. 4, making the overall logic sound.\n2. From both quantitative and qualitative results, SSDiff demonstrates good performance on the VintageFace benchmark.\n\nWeaknesses：\n1. The experimental settings and scenarios are somewhat incomplete:\n\ta. The results in Table 2 are quite puzzling. Without incorporating the self-supervised strategy proposed by the authors, this experimental setup achieves near state-of-the-art performance. Please provide corresponding explanations and clarifications.\n\tb. The paper only presents results for front-facing photos, without showing results for other poses, such as profiles. Additionally, the paper does not include results for the hard-case photos introduced in the VintageFace benchmark. Please clarify how SSDiff performs on these two common cases, as well as on cases involving individuals wearing glasses.\n\tc. Are the results in the table of Fig. 8 based on a single image? If so, the selection of these two scale parameters may not be representative.\n\td. In many of the loss calculations, the authors exclude the eyes, nose, and mouth. How does SSDiff perform if these areas contain significant scratches or degradation?\n\te. How does the author ensure identity preservation? There doesn’t appear to be any constraint on identity during the supervision process.\n\tf. The handling of the background regions is suboptimal (see Fig. 5, Fig. 9, and Fig. S4, third case).\n\n2. Some of the claims are puzzling and may not be entirely accurate.\n\ta. What is the author attempting to convey in lines 191-195? The equation presented appears to be an optimization formulation, but in the experiments, s_{w}sw​ and s_{s}ss​ are empirically chosen constants.\n\tb. In line 241, the authors claim that the face parsing network is robust to degradations. I suggest that the authors include results for some hard cases, such as those with significant degradations or profile poses, to better support this claim.\n\tc. In line 38 of supp, Why does the style transfer model only need to be applied in a single step?\ta. The caption for Figure 3 should be revised for clarity and precision.\n It is unclear what \\hat{x_{t}} represents.", "was_revised": true, "final_justification": "After reading the rebuttal and  other reviewers' opinion, most of my concerns have been addressed. \n\nThe authors are suggested to include the rebuttal's results and explanations.", "rebuttal": "Thank you for recognizing the clear idea of our method, the staged guidance design, and the soundness of our analysis, as well as the strong performance of SSDiff on benchmarks. We also appreciate your suggestions for adding more visual examples. **While the rebuttal system does not allow us to include additional figures**, we provide quantitative clarifications where possible and will incorporate your recommended visualizations in our final version.\n\n* **Weak1 (a): Explanation of Table 2.**\nThe baseline without our self-supervised strategy still benefits from two factors: 1. We retain the L1 loss, which provides a coarse directional signal that makes restorations visually clearer. 2. Mask-guided diffusion explicitly marks unreliable breakages so that frozen diffusions can use its generation to hallucinate missing content. Although this can only recover simple breakages, it naturally leads to better results than methods without any breakage repair. However, these results should not be confused with the self-supervised strategy. By introducing pseudo-reference guidance, it substantially improves color, texture, and fine details.\n\n* **Weak1 (b): Results in more scenarios.** \nThank you for this suggestion, which makes our evaluation more comprehensive. Below, we evaluate our SSDiff with existing diffusion-based methods:\n\n    **More poses:** We collect 20 relatively profile-view faces (fully side profiles are rare) from VintageFace and the web, and 20 frontal faces with comparable degradation levels (measured using Fig. S1 in the supplementary). The metrics show no significant gap between the two settings, demonstrating that such cases do not affect the robustness.\n\n    |Poses|Front|Profile-view|\n    |-|-|-|\n    |FID($\\downarrow$)|128.6|132.9|\n    |MAN-IQA($\\uparrow$)|0.3975|0.3920|\n\n    Furthermore, we evaluate 100 fully profile-view faces from real benchmarks (LFW-Test, WebPhoto-Test, Wider-Test); SSDiff surpasses DiffBIR, demonstrating generalization to pose variations. (DiffBIR has shown strong profile-view visual results in its paper.)\n\n    |Real profile-view|DifFace|DiffBIR|PGDiff|Ours|\n    |-|-|-|-|-|\n    |FID($\\downarrow$)|151.5|*146.0*|158.1|**143.5**|\n    |NIQE($\\downarrow$)|*4.116*|5.803|6.225|**4.035**|\n    |MAN-IQA($\\uparrow$)|0.3603|*0.3912*|0.3712|**0.3955**|\n\n    **Hard cases:** Some visualizations already come from hard-degradation subsets (e.g., **first row in Fig. 1, the third row in Fig. 5, and the second row in Fig. 6 in our main paper**). We understand that this question may refer to more extreme degradations. Therefore, we evaluate on the first 200 faces of the real benchmark Wider-Test (severely degraded but no breakages); SSDiff remains competitive.\n\n    |Severe degradation|DifFace|DiffBIR|PGDiff|Ours|\n    |-|-|-|-|-|\n    |FID($\\downarrow$)|*82.60*|82.75|91.80|**81.02**|\n    |NIQE($\\downarrow$)|*4.403*|5.891|4.930|**4.092**|\n    |MAN-IQA($\\uparrow$)|0.3386|**0.3781**|0.3415|*0.3697*|\n\n    **Wearing glasses:** Due to scarcity, we collect 10 pairs of old-photo faces (glasses vs. non-glasses) with similar degradation levels from the web; the performance gap is negligible, demonstrating that such cases do not affect the robustness.\n\n    |Glasses|Wear|Not wear|\n    |-|-|-|\n    |FID($\\downarrow$)|128.4|127.6|\n    |MAN-IQA($\\uparrow$)|0.3838|0.3814|\n\n    We further evaluate 100 faces with glasses from real benchmarks (LFW-Test, WebPhoto-Test, and Wider-Test), where SSDiff performs competitively with only a minor gap to DiffBIR, which is built upon a fine‑tuned SD 2.1 model with heavy training costs, while our method is training-free and inference on a frozen DDPM generator.\n\n    |Real data (wear glasses)|DifFace|DiffBIR|PGDiff|Ours|\n    |-|-|-|-|-|\n    |FID($\\downarrow$)|136.3|**119.8**|126.9|*121.7*|\n    |NIQE($\\downarrow$)|*3.871*|5.567|5.241|**3.843**|\n    |MAN-IQA($\\uparrow$)|0.3558|**0.4020**|0.3284|*0.3962*|\n\n    Finally, VintageFace mainly consists of front-facing photos because decades ago, portraits were usually taken in studios to clearly record one's appearance, so profile views are rare. Similarly, glasses were uncommon at that time, so faces with glasses are rare. Therefore, our results don't contain examples of this. We will add these results in the final version.\n\n* **Weak1 (c): Clarification on Fig. 8.**\nAll ablations, including those summarized in Fig. 8, are conducted on our Medium subset, which we consider representative because it spans a wide range of degradations from mild to near-severe. The single input image shown in the figure is only for illustration due to layout constraints and may have caused confusion. We will make this clear in the final version.\n\n* **Weak1 (d): Performance when eyes, nose, and mouth are with heavy scratches.**\nAs shown in **Fig. 3 and in the second and third rows of Fig. 5 in the main paper**, SSDiff is able to recover scratches around eyes, nose, and mouth with natural visual quality. To validate more heavy scratches, we select 50 moderate‑scratch faces and 50 severe‑scratch faces of these regions from VintageFace. We evaluate the mean smoothness (e.g., Edge strength) before and after restoration. The table shows that although severe scratches produce poorer smoothness in inputs of these regions, smoothness of restorations with no significantly degraded areas, demonstrating the robustness of our SSDiff for eyes, nose, and mouth. This robustness comes from generative priors of diffusion, which use contextual facial cues and pseudo-reference guidance to recover these regions. Visualization can also prove this, in Fig. 11 of the main paper, where even heavily degraded, eyes, nose, and mouth are recovered relatively well.\n\n    |Type|Medium scratches|Severe scratches|\n    |-|-|-|\n    |Edge Strength($\\downarrow$)|Input:114/Restored:46|Input:140/Restored:51|\n\n* **Weak1 (e): Identity preservation.** \nIdentity consistency is preserved by combining the fidelity signal from a restorer with diffusion guidance. The restorer first produces a coarse but aligned structure and texture, and in reliable regions without breakages, this output is close to the original identity. These reliable regions are enforced through our L1 loss in all reverse diffusion processes, so that the guidance passed to diffusion models retains the original appearance, preventing identity drift in generation.\n\n    we avoid explicit constraints like landmarks or 3D face shape because such constraints are highly sensitive to errors and even fail under moderate degradations. Instead, our design follows the same philosophy as generation-based restoration methods for controlling fidelity: using a reliable restoration module to preserve original facial textures as a constraint. However, the difference is that we then use diffusion guidance rather than training optimisation to refine images. This avoids the need for end-to-end training while effectively constraining the diffusion so that the results do not drift away from the true identity.\n\n* **Weak1 (f): Handling of background regions.** \nThe main reason for the less satisfactory background restoration is that the detected scratch fails to fully cover all breakage areas, and the diffusion itself has relatively weak generative ability for backgrounds compared to faces. Nevertheless, resulting backgrounds remain visually reasonable, showing the robustness of SSDiff for such detection error. We appreciate your observation, which indeed points out a potential direction for future work; we plan to explore region-adaptive guidance, where different areas (e.g., face vs. background) are guided with adaptive strengths to address this issue.\n\n* **Weak2 (a): Line 191-195.** \nThe current formulation may cause a misunderstanding. The intention of Eq. 9 was not to suggest that $s_w$ and $s_s$ are learned during network optimization. We wish to express that empirically determining these two coefficients by manual search can minimize the overall guidance loss, achieving optimal restoration. In the final version, we will revise Eq. 9 to make clear that these are fixed hyperparameters rather than parameters obtained through optimization.\n\n* **Weak2 (b): Robustness of Face Parsing.** \nOur claim is based on extensive observations: under heavy degradations, facial landmarks and 3D face estimation often fail completely, whereas parsing networks still produce approximate facial contours. This is also supported by our analysis in \"Weak2\" of Reviewer pXcG, where the IoU value between severely degraded and GT facial contours is high (79%), and we further show that SSDiff remains robust even with imperfect parsing maps in that response. We will add more visual examples to better support this.\n\n* **Weak2 (c): Single-step use of style transfer.** \nStyle transfer is applied only once, at the moment when face structures become relatively stable ($T=T_1$). The color distribution obtained at this moment provides a coarse but reliable color reference, which is then propagated and refined by the diffusion during the remaining reverse steps. Because the Selective Restoration Guidance is applied throughout, this single-step color initialization doesn't harm structural fidelity. Repeating style transfer at every diffusion step offers little additional benefit while incurring prohibitive computational cost ($T_1$ times longer runtime), so a single application is enough.\n\n* **Weak2 (d) & Q1 & Q2: Unclear of $\\hat{x}_{t}$.** \nIn Fig. 3, **$\\hat{x}_{t}$** denotes the intermediate restored estimate of clean images at timestep t in the reverse diffusion. Specifically, at each step, the model predicts a clean image from the current noisy state (as formulated in Line 7 of Algorithm 1 in our main paper), and **$\\hat{x}_{t}$** is this predicted clean sample. It indicates how images progressively become accurate as reverse diffusion proceeds. We will revise the figure caption in the final version."}, {"review_id": "6FQVgqyZVI", "reviewer": "Reviewer_bLnM", "summary": "This paper introduces a novel framework, SSDiff, designed to address the unique challenges of restoring old face photographs, which typically suffer from complex and compounded degradations such as breakage, fading, and blur. The authors propose generating pseudo-references using a pre-trained diffusion model under weak guidance. These pseudo-labels, although not identity-preserving, exhibit realistic facial contours and natural color tones, which can effectively guide restoration. SSDiff applies selective restoration guidance early in the denoising process for structural and breakage recovery, and selective coloring guidance later for natural skin tone refinement. This aligns with the coarse-to-fine nature of diffusion models and avoids identity mismatches. The framework leverages face parsing maps and scratch masks to selectively apply supervision, avoiding over-correction in identity-sensitive regions (like eyes or mouth), while enhancing degraded areas like skin and background.\n\nAlso, the paper introduce VintageFace, a curated set of 300 real old face photos categorized into simple, medium, and hard degradation levels, enabling robust evaluation under diverse degradation conditions.\n\nThis paper also demonstrates strong qualitative and quantitative performance, outperforming existing GAN-based and zero-shot diffusion-based methods in perceptual quality (FID, BRISQUE, TOPIQ, MAN-IQA) and identity preservation (CLIP-based distance).", "questions": "- The proposed framework depends on multiple external models (face parsing, scratch detection, style transfer). How sensitive is SSDiff’s performance to the quality of these components?\n- How is the 50 LQ selected for measuring fidelity of restoration?\n- Have you considered evaluating SSDiff on existing open-source datasets related to degraded or historical face images (e.g., CelebA-HQ with synthetic degradations, or subsets from LFW with added artifacts)? Even a small-scale cross-dataset test or synthetic degradation benchmark could help demonstrate that the method generalizes beyond your self-built data.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Innovative Self-Supervised Framework: The paper proposes SSDiff, a training-free framework that uses self-generated \"pseudo-reference\" faces to guide restoration, avoiding the need for paired training data.\n- Staged Guidance: The model uses a staged guidance system, focusing first on structural restoration and then on color refinement in later steps, which aligns with the coarse-to-fine nature of diffusion models.\n- Selective Restoration: By using face parsing maps and scratch masks, the method selectively restores damaged areas while avoiding alterations to identity-sensitive features like eyes and mouth.\n- New Benchmark: The authors constructed \"VintageFace,\" a new benchmark of 300 real old face photos with varied degradation levels, to evaluate performance.\n\nWeaknesses:\n- Limited Benchmark Scale: The newly introduced VintageFace benchmark, while a valuable contribution, is relatively small, with 300 images.\n- Narrow Identity Preservation Test: The evaluation of identity preservation was conducted using CLIP-based distance on a select subset of 50 inputs with well-preserved features, which may not represent performance on more severely degraded images.\n- Specialized Focus: The paper's scope is highly specialized for the compounded degradations common in old photographs, such as breakage, fading, and severe blurring. While the performance is superior, the real-world impact is limited since such degraded old photos are relatively rare.", "was_revised": true, "final_justification": "The additional experiments and enriched dataset have addressed majority of my concerns. Thus I’m happy to adjust the score accordingly.", "rebuttal": "Thank you for recognizing the novelty of our SSDiff, including the self-supervised design using pseudo-references, the staged coarse-to-fine guidance strategy, and the selective restoration mechanism that preserves identity-sensitive regions. Below, we address your concerns point by point. **As the current rebuttal system does not provide a way to include figures**, some clarifications are given quantitatively, and the corresponding visualizations will be added in the final version.\n\n* **Weak1: Limited Benchmark Scale.**\nThe VintageFace benchmark contains 300 images, but this benchmark is not a primary contribution; rather, it was introduced because no such dataset currently exists to provide an initial evaluation set for restoration performance under extreme old-photo degradations, including breakage, fading, and severe blur. We plan to further expand this benchmark in future work. Moreover, as shown in **Table S1 in our supplementary**, we have also validated SSDiff on broader datasets containing old photos (WebPhoto-Test and CelebA-Child), demonstrating that our method is not limited by the current scale of VintageFace.\n\n* **Weak2: Narrow Identity Preservation Test.**\nTo address your concern, we evaluate on the subset of CelebA-HQ (1000 images) with synthetic degradations generated following the degradation model in GFPGAN [1], where degradation parameters (blur kernel σ, downsample scale r, Gaussian noise δ, JPEG quality factor q) are randomly sampled from (7:9, 8:16, 20:40, 30:40) to form a **severe-degradation set** (measured using Fig. S1 in the supplementary). We recompute CLIP-based identity distances on this dataset, and we further adopt face landmark distance (LMD) [2] as the face fidelity metric. PSNR or SSIM are not used as they primarily measure pixel consistency rather than face identity preservation. As shown in the table below, our method remains consistent with the paper's findings: SSDiff keeps competitive among diffusion-based methods, only behind DiffBIR on the LMD metric. Note that DiffBIR is trained on large paired datasets synthesized using the same degradation model, whereas SSDiff is training-free.\n\n    |Degradation Type|DifFace|DiffBIR|DDNM|PGDiff|Ours|\n    |-|-|-|-|-|-|\n    |Identity Distance ($\\downarrow$)|0.171|*0.154*|0.187|0.202|**0.153**|\n    |LMD ($\\downarrow$)|4.20|**3.88**|4.85|5.19|*4.12*|\n\n\n* **Q1: Robustness of pre-trained networks.**\nSSDiff is generally robust to inaccuracies in external components (face parsing, scratch detection, style transfer). These networks only provide coarse directional signals during reverse diffusion, similar to classifier-guided diffusion, and are not strict constraints. As long as the guidance is not severely misleading, the strong generative prior of the frozen diffusion model dominates reconstruction. Across many visual results in the paper, there are inevitably cases where these networks are imperfect, yet the restorations remain stable and of high quality.\n\n    **Parsing maps:** As shown in **Fig. 6 (second row) in our main paper**, even with occluded contours leading to inaccurate parsing, the results remain visually faithful. Furthermore, as discussed in our analysis of \"Weak2\" for Reviewer pXcG, even under severe degradations, the IoU between degraded facial contours and GT contours remains around 79%, demonstrating its robustness.  In addition, the results of the parsing map do not directly supervise the guidance, but rather in an indirect manner, also leading to better robustness.\n\n    **Scratch detection:** As shown in **Fig. S4 (second row) in our supplementary**, large stains on the head are only partially detected, yet the restoration remains natural. Similarly, **in the failure case in Fig. 11**, only a fraction of the stains are detected, but the restored results are still better than other methods. This demonstrates that the model uses detected regions as a rough guide and can roughly recover nearby structures even with incomplete masks.\n\n    **Style transfer:** Although the main paper does not include intermediate style-transfer outputs, we observe the intermediate style‑transfer result for Fig. 7 (first row). The result shows that transferred images initially have a yellowish facial tint; however, once used as a coarse color prior, the diffusion process refines and propagates realistic colors, producing a natural complexion.\n\n    To quantify this robustness, we conducted additional experiments on the Medium type subset:\n\n    1. **Parsing maps:** We introduced inaccuracies by replacing the original parsing maps with pseudo-label parsing maps of different strengths $s$, where the resulting errors are even larger than those observed in parsing maps under severe degradations (79% IOU). The resulting IoU with the original parsing map is:\n        * for s=2.5e-4, IoU ≈ 76% (24% discrepancy);\n        * for s=1e-4, IoU ≈ 70% (30% discrepancy);\n\n        |Parsing Maps|Ours|IOU(76%)|IOU(70%)|\n        |-|-|-|-|\n        |FID($\\downarrow$)|128.3|131.1|133.4|\n        |MAN-IQA($\\uparrow$)|0.395|0.391|0.382|\n        |Face Similarity ($\\uparrow$)|1|0.985|0.955|\n\n    2. **Scratch masks:** Randomly flip 10%, 20%, and 30% of the masks of breakage regions to simulate a situation where some of the breakages have not been detected.\n\n        |Scratch Masks|Ours|10% flip|20% flip|30% flip|\n        |-|-|-|-|-|\n        |FID($\\downarrow$)|128.3|129.2|130.7|132.5|\n        |MAN-IQA($\\uparrow$)|0.395|0.391|0.379|0.381|\n        |Face Similarity ($\\uparrow$)|1|0.974|0.953|0.937|\n\n    3. **Style transfer:** We weaken the style transfer guidance by reducing the style factor α from 0 to 0.1 and 0.2, slightly affecting color and content. \n\n        |Style Transfer|Ours (α=0)|Poor-1 (α=0.1)|Poor-2 (α=0.2)|\n        |-|-|-|-|\n        |FID($\\downarrow$)|128.3|129.2|128.8|\n        |MAN-IQA($\\uparrow$)|0.395|0.396|0.392|\n        |Face Similarity ($\\uparrow$)|1|0.977|0.959|\n\n    Here, Face Similarity (range [0, 1]) denotes the cosine similarity between features (extracted with ArcFace [3]) of the perturbed restoration and the original restoration (Ours). **These results show that errors in the pre-trained networks are not severe (Less than 40%), and the strong generative prior of the diffusion model can propagate the correct cues to other regions, preventing significant performance drops.** This demonstrates that SSDiff is robust to these pre-trained networks. We will include these quantitative results and additional visual examples in the final version.\n\n* **Q2: How to select 50 LQ samples for measuring fidelity.** \nThe 50 LQ images used for fidelity evaluation are chosen from samples with relatively mild degradation, particularly those without breakage or severe blur in key facial components such as eyes, nose, and mouth. Since in the absence of ground truth, identity distances have to be computed with input old photos. For heavily degraded images with severe blur, fading, or breakage, a successful restoration even incorrectly increases the identity distance (where a smaller distance indicates better fidelity) because the restoration corrects degradations that are inadvertently treated as part of the original face identity. Therefore, to mitigate this bias, we select old photos with lighter degradation and better-preserved facial features, providing a more reliable evaluation of identity fidelity.\n\n\n* **Weak3 & Q3: Generalization to other datasets and degradations.** \nAs shown in **Table S1 and Fig. S7 of our supplementary**, we have evaluated SSDiff on popular benchmarks beyond our VintageFace, including WebPhoto-Test (407 LQ faces) and CelebA-Child (180 LQ faces). These sets contain a large number of historical face images with varying degrees of blur and fading but no breakage, where our masks automatically degenerate to standard restoration (e.g., all black). SSDiff achieves superior quantitative (FID, NIQE) and qualitative results on both benchmarks, demonstrating strong generalization.\n\n    Following your valuable suggestion, we further performed additional evaluations on a 1000-image subset of CelebA-HQ using the same degradation model in GFPGAN [1], where degradation parameters (blur kernel σ, downsample scale r, Gaussian noise δ, JPEG quality factor q) are randomly sampled from (0.2:10, 1:8, 0:15, 60:100) to form a **mixed-degradation set**. We further evaluate on the real benchmark you mentioned (LFW-Test with 1711 LQ faces).\n\n    |CelebA-HQ (synthetic)|DifFace|DiffBIR|DDNM|PGDiff|Ours|\n    |-|-|-|-|-|-|\n    |LPIPS ($\\downarrow$)|0.2722|**0.2659**|0.4112|0.3104|*0.2695*|\n    |DISTS ($\\downarrow$)|0.1441|*0.1431*|0.3281|0.1636|**0.1320**|\n    |FID ($\\downarrow$)|69.3|*51.2*|115.8|58.1|**50.3**|\n    |NIQE ($\\downarrow$)|*4.29*|6.08|8.83|4.837|**4.16**|\n\n    |LFW-Test (real)|DifFace|DiffBIR|DDNM|PGDiff|Ours|\n    |-|-|-|-|-|-|\n    |FID ($\\downarrow$)|46.4|**40.4**|112.3|47.3|*42.2*|\n    |NIQE ($\\downarrow$)|4.30|5.74|9.64|*4.11*|**4.03**|\n    |MAN-IQA ($\\uparrow$)|0.368|**0.423**|0.275|0.373|*0.411*|\n\n    As shown in these tables, SSDiff consistently achieves the best or second-best results across both reference and non-reference metrics. Note that our focus is on old-photo face restoration rather than real-world face restoration. Nevertheless, as shown in the above tables and in our paper, SSDiff demonstrates strong robustness to real-world degradations. These results indicate that SSDiff is capable of handling both old face photos and real-world degraded faces, whereas methods such as DiffBIR, though effective for real degradations, struggle with old face photos.\n\n[1] Wang X, Li Y, Zhang H, et al. Towards real-world blind face restoration with generative facial prior. CVPR 2021.\n\n[2] Gu Y, Wang X, Xie L, et al. Vqfr: Blind face restoration with vector-quantized dictionary and parallel decoder. ECCV 2022.\n\n[3] Deng J, Guo J, Xue N, et al. Arcface: Additive angular margin loss for deep face recognition. CVPR 2021."}, {"review_id": "YC1CwWLmOn", "reviewer": "Reviewer_DyPr", "summary": "This article presents a novel approach to old photo face restoration, termed Self-Supervised Selective-Guided Diffusion (SSDiff). The proposed method leverages a pre-trained diffusion model to generate pseudo-reference faces under weak guidance, which are then utilized to guide the restoration process through staged supervision. The method applies structural guidance throughout the denoising process and color refinement in later steps, aligned with the coarse-to-fine nature of diffusion. The experimental results demonstrate that SSDiff outperforms existing methods in terms of metrics such as FID, BRISQUE, TOPIQ and MAN-IQA. The qualitative comparison results also indicate that SSDiff is capable of producing natural colors, clear textures, and minimal artifacts, while maintaining high fidelity and identity consistency.", "questions": "1. Since the test set lacks ground truth (GT), I wonder how FID is evaluated. My understanding is that it's calculated with respect to the FFHQ dataset. In that case, is the FID computed on 300 images reliable?\n2. The authors only conducted quantitative tests on their self-collected dataset, which lacks ground truth (GT), and non-reference metrics are often unreliable in many cases. It would be beneficial to test the quantitative results on some datasets with GT or to manually simulate degradation on FFHQ and evaluate the performance.\n3. The authors utilize pre-trained face parsing and scratch detection networks. It is unclear whether the generated images will be substantially degraded if these networks are not accurate. Furthermore, the incorporation of these models may lead to a notable increase in inference time. Providing quantitative results on computational cost and inference time would be beneficial.\n4. The efficacy of the proposed method is heavily dependent on the accuracy of the pseudo labels. Nevertheless, it is a known issue that diffusion models tend to produce inconsistent images under weak supervision. Does the model possess robustness to handle such situations?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nThis article's analysis of images generated under different guided intensities is insightful, and the idea of utilizing images generated under weak supervision as a reference for color and texture is interesting.\n\n**Weaknesses**\n\nThe method has several limitations, including reliance on accurate pseudo-labels and pre-existing networks, and limited quantitative evaluation. See the question section for details.", "was_revised": true, "final_justification": "I have carefully read the reviewers' comments and the authors' responses to them. The authors have clarified these concerns directly and provided rich and convincing experiments, so I have decided to increase my score.", "rebuttal": "Thank you for highlighting the insightful analysis of guided intensities and the novel idea of using weakly supervised generated images as references for color and texture. We address your concerns point by point below. **As the current rebuttal system does not provide a way to include figures**, some clarifications are given quantitatively, and corresponding visualizations will be added in the final version.\n\n* **Q1: Calculation of FID.** \nThe calculation of FID in our experiments follows official codes of VQFR [1] rather than computing statistics only on 300 images in our VintageFace. Specifically, we use the pre-trained inception weight released with VQFR, which is trained on 70k high-quality FFHQ images. FID is then computed between the feature distribution of our restored images and that of the reference FFHQ distribution. Therefore, the reliability of FID does not depend on the number of images in VintageFace itself, but on the fixed statistics from the large-scale FFHQ reference.\n\n* **Q2: Reference metrics on synthetic data.**\nDue to time constraints, we randomly select 1000 face images from CelebA-HQ for evaluation. Synthetic degradations are applied using the same degradation model as GFPGAN [2], where degradation parameters (blur kernel σ, downsample scale r, Gaussian noise δ, JPEG quality factor q) are randomly sampled from (0.2:10, 1:8, 0:15, 60:100) to form a **mixed-degradation** set. We report LPIPS and DISTS to assess perceptual quality and include face landmark distance (LMD) as a fidelity metric (as in VQFR [1]). PSNR and SSIM are not used as they primarily measure pixel similarity rather than face identity preservation.\n\n    |CelebA-HQ (synthetic)|DifFace|DiffBIR|DDNM|PGDiff|Ours|\n    |-|-|-|-|-|-|\n    |LPIPS ($\\downarrow$)|0.2722|**0.2659**|0.4012|0.3104|*0.2695*|\n    |DISTS ($\\downarrow$)|0.1441|*0.1431*|0.3281|0.1636|**0.1320**|\n    |LMD ($\\downarrow$)|3.37|**3.01**|3.52 |4.15|*3.29*|\n\n    Compared with existing diffusion-based methods, our method achieves competitive performance, only behind DiffBIR, which is trained on large paired datasets generated with the same degradation model.  Note that our focus is on old-photo face restoration rather than blind face restoration. Nevertheless, as shown in the above tables and in our paper, SSDiff shows strong robustness to synthetic degradations. These results indicate that SSDiff is capable of handling both old face photos and synthetic degraded faces, whereas methods such as DiffBIR, though effective for synthetic degradations, struggle with old face photos.\n\n\n* **Q3_1: Robustness of pre-trained networks.**\nSSDiff is generally robust to inaccuracies in external components such as face parsing and scratch detection networks. These pre-trained networks provide only coarse directional guidance during reverse diffusion—similar to classifier-guided diffusion—rather than strict constraints. As long as their outputs are not severely erroneous, the strong generative prior of the frozen diffusion model dominates reconstruction and recovers plausible structures. Consequently, even when these auxiliary outputs are imperfect, the restorations remain stable and of high quality. Below, we analyze visual cases present in our paper to prove this point: \n\n    **Parsing maps:** As shown in **Fig. 6 (second row) of the main paper**, even with occluded contours, leading to inaccurate parsing, the results remain visually natural. Furthermore, as discussed in our analysis of \"Weak2\" for Reviewer pXcG, even under severe degradations, the IoU between degraded facial contours and GT contours remains around 79%, demonstrating the robustness of the parsing network.\n\n    **Scratch detection:** As shown in **Fig. S4 (second row) of our supplementary**, large stains on the head are only partially detected, yet the restoration remains natural. Similarly, **in the failure case in Fig. 11**, only a fraction of the stains are detected, but the restored results are still better than other methods. This demonstrates that the model uses detected regions as a rough guide and can roughly recover nearby features even with incomplete masks.\n\n    To quantify this robustness, we conduct additional experiments on the Medium degradation subset:\n\n    1. **Parsing maps:** We introduce inaccuracies by replacing the original parsing map with the pseudo-label parsing map of different strengths $s$, where the resulting errors are even larger than those observed in parsing maps under severe degradations (79% IOU). The resulting facial contours IoU with the original parsing maps is:\n        * for s=2.5e-4, IoU ≈ 76% (24% discrepancy);\n        * for s=1e-4, IoU ≈ 70% (30% discrepancy);\n\n        |Parsing Maps|Ours|IOU(76%)|IOU(70%)|\n        |-|-|-|-|\n        |FID($\\downarrow$)|128.3|131.1|133.4|\n        |MAN-IQA($\\uparrow$)|0.395|0.391|0.382|\n        |Face Similarity ($\\uparrow$)|1|0.985|0.955|\n\n    2. **Scratch masks:** Randomly flip 10%, 20%, and 30% of the masks of breakage regions to simulate a situation where some of the breakages have not been detected.\n\n        |Scratch Masks|Ours|10% flip|20% flip|30% flip|\n        |-|-|-|-|-|\n        |FID($\\downarrow$)|128.3|129.2|130.7|132.5|\n        |MAN-IQA($\\uparrow$)|0.395|0.391|0.379|0.381|\n        |Face Similarity ($\\uparrow$)|1|0.974|0.953|0.937|\n\n        Here, Face Similarity (range [0, 1]) denotes the cosine similarity between features (extracted with ArcFace [3]) of the perturbed restoration and the original restoration (Ours). This result shows that errors in the parsing map network are not severe (Less than 40%), and the strong generative prior of the diffusion can propagate the correct cues to other face regions, preventing significant performance drops. This demonstrates that SSDiff is robust to these pre-trained networks. We will include these quantitative results and additional visual examples in the final version.\n\n\n* **Q3_2: Computational Costs & Speed.**\nAs detailed in **Sec. C.2 and Sec. C.3 of our supplementary**, we have analyzed the impact of the auxiliary components (e.g., pre-trained face parsing and scratch detection networks) on both computational cost and inference time:\n\n    **Params and FLOPs:** As shown in **Table S2 of our supplementary**, our SSDiff requires fewer parameters and FLOPs than existing diffusion-based methods. Because the face parsing and scratch detection networks can be executed offline, reported values in the supplementary do not include them. For completeness, if we include these components, the total becomes 105.4 M parameters and 140.6 G FLOPs, still significantly lower than DifFace (175.4 M, 268.8 G) and DiffBIR (1717 M, 23234 G). PGDiff also depends on scratch masks; using the same scratch detection model, its cost would be roughly 84.3 M parameters and 146.8 G FLOPs, comparable to ours. These results indicate that SSDiff does not suffer from a computational disadvantage.\n\n    **Inference Speed:** As reported in **Sec. C.2 of our supplementary**, all auxiliary networks are executed once in reverse diffusion rather than at every sampling step. Even when all networks run online, the added latency on an NVIDIA RTX 4090 is only about 95 ms, which is negligible compared to the tens of seconds required by the diffusion sampling itself.\n\n\n* **Q4: Robustness to pseudo-label inconsistency.**\nWe appreciate your concern. This issue was carefully considered in the design of SSDiff. As shown in the statistical analysis in Fig. 2 of the main paper, by controlling the strength of weak supervision, we ensure that the pseudo-labels (generated by a frozen diffusion model) closely align with facial contours, with only small deviations (reflected in the reported IoU values). Moreover, these pseudo-labels are not used as pixel-level constraints; they serve only as coarse guidance for color style and smoothness in damaged regions, so moderate inaccuracies have limited influence on the final restoration.\n\n    To further assess robustness under less accurate pseudo-labels, we conducted an additional experiment on the Medium degradation subset using pseudo-labels generated with a smaller sampling step size (s=1e-4), where the average IoU of facial contours with the original input drops from 87% to 70%. As shown below, compared to the original pseudo-labels (s=1e-3), the resulting image quality and Face Similarity (defined in \"Q3_1\") degrade slightly. This confirms that SSDiff remains relatively robust even when pseudo-labels are partially less reliable.\n\n    |Pseudo-label quality|MAN-IQA($\\uparrow$)|FID($\\uparrow$)|Face Similarity($\\uparrow$)|\n    |-|-|-|-|\n    |IOU:87% (Orginal)|0.395|128.3|1|\n    |IOU:70%|0.374|136.6|0.929|\n\n[1] Gu Y, Wang X, Xie L, et al. Vqfr: Blind face restoration with vector-quantized dictionary and parallel decoder. ECCV 2022.\n\n[2] Wang X, Li Y, Zhang H, et al. Towards real-world blind face restoration with generative facial prior. CVPR 2021.\n\n[3] Deng J, Guo J, Xue N, et al. Arcface: Additive angular margin loss for deep face recognition. CVPR 2021."}, {"review_id": "Tqn39we65L", "reviewer": "Reviewer_pXcG", "summary": "SSDiff combines weak guidance to generate pseudo-labels, and employs selective restoration with selective coloring using structural awareness to extend the paradigm of pre-trained zero-shot diffusion guidance, expanding the new paradigm of zero-shot diffusion guidance. In addition, the authors present a benchmark for real-world old photographs of human faces. Experiments demonstrate its advanced performance from perceptual quality and fidelity of restoration results.", "questions": "Please refer to the Strengths And Weaknesses!", "limitations": "Please refer to the Strengths And Weaknesses!", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Paper strength: The authors propose a novel perspective by generating pseudo-labeled faces as references using a pre-trained diffusion model. This novel motivation is supported by comprehensive experiments, which validate the effectiveness of pseudo-labeled references. Additionally, paper writing on this paper is clear and easy to understand. \nThe design of selective guidance using pseudo-labels is interesting, making SSDiff support the region-specific stylized restoration, like hair and lips. Meanwhile, the proposed guidance strategy is effective, as demonstrated by extensive quantitative and qualitative experiments. \n\nMajor Weaknesses: \nI have observed the pipeline of the method is mainly used for faded old photos with damage, and the three components of the method correspond to the blurred, damaged, and faded cases. However, old face photographs may not always have damage, and when this happens, is the method still applicable? Figure 6 demonstrates this to some extent, but can there be quantitative evidence?\nThe guidance process relies somewhat on face-parsing maps, but the robustness of the face-parsing maps is not explored. \nWhen the face structure stabilizes, SSDiff guides coloring, but for old photos with different degrees of degradation, this moment T_1 should not be deterministic, it will be in a range, and the authors should clarify this.\nIn the Selective Restoration Guidance subsection, the authors do not describe how to select the breakage regions of images that do not contain the face component areas.\nMinor Weaknesses:\nThe visualization shows good fidelity, but regarding the quantitative analysis of fidelity, why use CLIP-based face identity distances instead of normal identity distances?\nThe s = 3.5e-3 in Implementation Details does not match Figure 3; it should be S_s.", "was_revised": true, "final_justification": "Overall, I appreciate the author's efforts during the rebuttal and consider this a high-quality submission. Incorporating the aforementioned results will further strengthen the methodological rigor and reproducibility. Therefore, I am raising my score from Accept to Strong Accept.", "rebuttal": "Thank you for recognizing the novelty and rationality of our SSDiff framework, including the idea of pseudo-labeled references, the clear writing, and the effectiveness of the selective guidance strategy. Below, we address your concerns point by point. **As the current rebuttal system does not provide a way to include figures**, some clarifications are given quantitatively, and the corresponding visualizations will be added in the final version.\n\n* **Weak1: Quantitative evidence.** Yes, our method is applicable even when old photos have no visible breakage. When masks are empty, these modules act as normal restoration, and the later-stage color refinement still benefits faded images. **Table S1 in our supplementary** reports results on WebPhoto-Test (407 faces) and CelebA-Child (180 faces), both consisting of old face photos without breakage. SSDiff achieves the best or second-best scores across quantitative results, including FID and NIQE on the above benchmarks, confirming its effectiveness beyond broken cases with quantitative evidence.\n\n* **Weak2: Robustness of face-parsing maps.** \nThe performance of SSDiff is generally robust to inaccuracies in the face parsing network. In our framework, the pre-trained parsing model is used only to provide a coarse directional signal during the reverse diffusion process, similar to the loose guidance mechanism in classifier-guided diffusion, rather than as a strict constraint. As long as this signal is not severely misleading (i.e., without gross parsing failures), the strong generative prior of the frozen diffusion model dominates the reconstruction and can recover plausible facial structures.\n\n    As discussed in **Sec. C.1 of the supplementary**, face parsing is considerably more robust than facial landmarks or 3D face models. Based on extensive experiments, we have observed that parsing maps almost never fail completely, even for degraded faces, and typically provide a sufficiently accurate spatial prior. To quantify this, we manually degraded 100 randomly selected high-quality CelebA‑HQ faces using the degradation model in GFPGAN [1], where degradation parameters (blur kernel σ, downsample scale r, Gaussian noise δ, JPEG quality factor q) are randomly sampled from (0.2:10, 12:16, 0:15, 60:100). These degraded images can be classified as hard-degraded following the standard of Fig. S1 in the supplementary, still achieve an average **face contours IoU of 79%** between their parsing-map and those of original high-quality images.\n\n    Qualitatively, across the visual results in the main paper, some parsing maps are clearly imperfect, yet the restored faces remain visually stable and of high quality. For example, **in Fig. 6 (second row)**, part of the face contour is occluded, leading to an inaccurate parsing map, but the final restoration remains natural, demonstrating that SSDiff does not rely on precise parsing boundaries.\n\n    To further evaluate robustness against parsing-map inaccuracies, we conducted a controlled experiment on the Medium subset of VintageFace by replacing the original parsing maps with pseudo-label maps generated with different guidance strengths $s$. The resulting face contours IoU with respect to the original parsing map are:\n\n    * for s=2.5e-4, IoU ≈ 76% (24% discrepancy);\n    * for s=1e-4, IoU ≈ 70% (30% discrepancy);\n\n    The resulting performance is summarized below:\n\n    |Parsing Maps|Ours|IOU(76%)|IOU(70%)|\n    |-|-|-|-|\n    |FID($\\downarrow$)|128.3|131.1|133.4|\n    |MAN-IQA($\\uparrow$)|0.395|0.391|0.382|\n    |Face Similarity ($\\uparrow$)|1|0.985|0.955|\n\n    Here, Face Similarity (range [0, 1]) denotes the cosine similarity between features (extracted with ArcFace [3]) of the perturbed restoration and the original restoration (Ours). This result shows that errors in the parsing map network are not severe (Less than 40%), and the strong generative prior of the diffusion can propagate the correct cues to other face regions, preventing significant performance drops. This demonstrates that SSDiff is robust to these pre-trained networks. We will include these quantitative results and additional visual examples in the final version.\n\n* **Weak3: $T_1$ range.** Thank you for this suggestion. In extensive experiments on our whole benchmark, we also find that the transition point $T_1$ is not a fixed value but lies within a range. However, for simplicity and consistency across all experiments, we set $T_1=0.4T$, which corresponds well to most cases. In practice, depending on the degree of degradation, $T_1$ typically falls between $0.35T$ and $0.45T$; lighter degradations tend to shift $T_1$ slightly bigger. We will clarify this in the final version and include additional visual examples to illustrate the effect of this variation.\n\n* **Weak4: Selection of non-face breakage regions.** For selecting breakage regions outside facial components, our method combines the scratch mask with the labels from the face parsing map. The parsing map identifies non-face areas such as background, skin, and hair, while the scratch mask (binary: white=1 for breakage, black=0 for intact) localizes breakage regions. Taking the intersection of these two masks gives the breakage regions that do not contain facial components, which are then guided for restoration. We will incorporate these instructions into our final version.\n\n* **Weak5: Why CLIP-based face identity distances?** Traditional identity distances metrics rely on facial landmark detection (e.g., ArcFace [2]) and domain-specific face recognition embeddings. In input old photos with breakages, landmarks are often missing or heavily distorted, which leads to unstable or even invalid identity features and thus unreliable distances. To avoid this bias, we adopt CLIP-based face identity distances. CLIP embeddings are computed directly from the entire image without requiring landmark alignment, and they are trained on very diverse, cross-domain image-text data. This makes them more robust to blur, fading, and partial breakage, allowing us to measure identity similarity more stably and fairly under the challenging conditions of old-photo restoration. We will incorporate the above discussion into our final version.\n\n* **Weak6: Details error.** Thank you for your careful review. We will correct $s$ = 3.5e-3 to $s_s$ = 3.5e-3 in the implementation details of our main paper.\n\n[1] Wang X, Li Y, Zhang H, et al. Towards real-world blind face restoration with generative facial prior. CVPR 2021.\n\n[2] Deng J, Guo J, Xue N, et al. Arcface: Additive angular margin loss for deep face recognition. CVPR 2021."}]

Author rebuttal:

Meta-review: This paper presents a training-free framework for old-photo face restoration using pseudo-reference faces from weak diffusion guidance with staged supervision (structural then color) and selective restoration. Reviewers praised the novel approach, technical soundness, strong results on the VintageFace benchmark, and clear presentation. Reviewers' initial concerns about robustness, limited evaluation scenarios, and computational costs were addressed during the rebuttal through experiments showing the algorithm's robustness to 30-40% parsing errors and better performance across diverse conditions. The rebuttal led to score increases, with all reviewers satisfied. Given the technical soundness, comprehensive evaluation, and consensus positive reviews, the AC recommends Acceptance.
Decision: Accept (poster)