Title: Self-Supervised Selective-Guided Diffusion Model for Old-Photo Face Restoration
Venue: NeurIPS 2025
Accepted: yes
Keywords: Old-Photo Face Restoration
Abstract: Old-photo face restoration poses significant challenges due to compounded degradations such as breakage, fading, and severe blur. Existing pre-trained diffusion-guided methods either rely on explicit degradation priors or global statistical guidance, which struggle with localized artifacts or face color. We propose Self-Supervised Selective-Guided Diffusion (SSDiff), which leverages pseudo-reference faces generated by a pre-trained diffusion model under weak guidance. These pseudo-labels exhibit structurally aligned contours and natural colors, enabling region-specific restoration via staged supervision: structural guidance applied throughout the denoising process and color refinement in later steps, aligned with the coarse-to-fine nature of diffusion. By incorporating face parsing maps and scratch masks, our method selectively restores breakage regions while avoiding identity mismatch. We further construct VintageFace, a 300-image benchmark of real old face photos with varying degradation levels. SSDiff outperforms existing GAN-based and diffusion-based methods in perceptual quality, fidelity, and regional controllability. Code link: https://github.com/PRIS-CV/SSDiff.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSSDiff combines weak guidance to generate pseudo-labels, and employs selective restoration with selective coloring using structural awareness to extend the paradigm of pre-trained zero-shot diffusion guidance, expanding the new paradigm of zero-shot diffusion guidance. In addition, the authors present a benchmark for real-world old photographs of human faces. Experiments demonstrate its advanced performance from perceptual quality and fidelity of restoration results.\n\nSTRENGTHS AND WEAKNESSES:\nPaper strength: The authors propose a novel perspective by generating pseudo-labeled faces as references using a pre-trained diffusion model. This novel motivation is supported by comprehensive experiments, which validate the effectiveness of pseudo-labeled references. Additionally, paper writing on this paper is clear and easy to understand. \nThe design of selective guidance using pseudo-labels is interesting, making SSDiff support the region-specific stylized restoration, like hair and lips. Meanwhile, the proposed guidance strategy is effective, as demonstrated by extensive quantitative and qualitative experiments. \n\nMajor Weaknesses: \nI have observed the pipeline of the method is mainly used for faded old photos with damage, and the three components of the method correspond to the blurred, damaged, and faded cases. However, old face photographs may not always have damage, and when this happens, is the method still applicable? Figure 6 demonstrates this to some extent, but can there be quantitative evidence?\nThe guidance process relies somewhat on face-parsing maps, but the robustness of the face-parsing maps is not explored. \nWhen the face structure stabilizes, SSDiff guides coloring, but for old photos with different degrees of degradation, this moment T_1 should not be deterministic, it will be in a range, and the authors should clarify this.\nIn the Selective Restoration Guidance subsection, the authors do not describe how to select the breakage regions of images that do not contain the face component areas.\nMinor Weaknesses:\nThe visualization shows good fidelity, but regarding the quantitative analysis of fidelity, why use CLIP-based face identity distances instead of normal identity distances?\nThe s = 3.5e-3 in Implementation Details does not match Figure 3; it should be S_s.\n\nQUESTIONS:\nPlease refer to the Strengths And Weaknesses!\n\nLIMITATIONS:\nPlease refer to the Strengths And Weaknesses!", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article presents a novel approach to old photo face restoration, termed Self-Supervised Selective-Guided Diffusion (SSDiff). The proposed method leverages a pre-trained diffusion model to generate pseudo-reference faces under weak guidance, which are then utilized to guide the restoration process through staged supervision. The method applies structural guidance throughout the denoising process and color refinement in later steps, aligned with the coarse-to-fine nature of diffusion. The experimental results demonstrate that SSDiff outperforms existing methods in terms of metrics such as FID, BRISQUE, TOPIQ and MAN-IQA. The qualitative comparison results also indicate that SSDiff is capable of producing natural colors, clear textures, and minimal artifacts, while maintaining high fidelity and identity consistency.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThis article's analysis of images generated under different guided intensities is insightful, and the idea of utilizing images generated under weak supervision as a reference for color and texture is interesting.\n\n**Weaknesses**\n\nThe method has several limitations, including reliance on accurate pseudo-labels and pre-existing networks, and limited quantitative evaluation. See the question section for details.\n\nQUESTIONS:\n1. Since the test set lacks ground truth (GT), I wonder how FID is evaluated. My understanding is that it's calculated with respect to the FFHQ dataset. In that case, is the FID computed on 300 images reliable?\n2. The authors only conducted quantitative tests on their self-collected dataset, which lacks ground truth (GT), and non-reference metrics are often unreliable in many cases. It would be beneficial to test the quantitative results on some datasets with GT or to manually simulate degradation on FFHQ and evaluate the performance.\n3. The authors utilize pre-trained face parsing and scratch detection networks. It is unclear whether the generated images will be substantially degraded if these networks are not accurate. Furthermore, the incorporation of these models may lead to a notable increase in inference time. Providing quantitative results on computational cost and inference time would be beneficial.\n4. The efficacy of the proposed method is heavily dependent on the accuracy of the pseudo labels. Nevertheless, it is a known issue that diffusion models tend to produce inconsistent images under weak supervision. Does the model possess robustness to handle such situations?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel framework, SSDiff, designed to address the unique challenges of restoring old face photographs, which typically suffer from complex and compounded degradations such as breakage, fading, and blur. The authors propose generating pseudo-references using a pre-trained diffusion model under weak guidance. These pseudo-labels, although not identity-preserving, exhibit realistic facial contours and natural color tones, which can effectively guide restoration. SSDiff applies selective restoration guidance early in the denoising process for structural and breakage recovery, and selective coloring guidance later for natural skin tone refinement. This aligns with the coarse-to-fine nature of diffusion models and avoids identity mismatches. The framework leverages face parsing maps and scratch masks to selectively apply supervision, avoiding over-correction in identity-sensitive regions (like eyes or mouth), while enhancing degraded areas like skin and background.\n\nAlso, the paper introduce VintageFace, a curated set of 300 real old face photos categorized into simple, medium, and hard degradation levels, enabling robust evaluation under diverse degradation conditions.\n\nThis paper also demonstrates strong qualitative and quantitative performance, outperforming existing GAN-based and zero-shot diffusion-based methods in perceptual quality (FID, BRISQUE, TOPIQ, MAN-IQA) and identity preservation (CLIP-based distance).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Innovative Self-Supervised Framework: The paper proposes SSDiff, a training-free framework that uses self-generated \"pseudo-reference\" faces to guide restoration, avoiding the need for paired training data.\n- Staged Guidance: The model uses a staged guidance system, focusing first on structural restoration and then on color refinement in later steps, which aligns with the coarse-to-fine nature of diffusion models.\n- Selective Restoration: By using face parsing maps and scratch masks, the method selectively restores damaged areas while avoiding alterations to identity-sensitive features like eyes and mouth.\n- New Benchmark: The authors constructed \"VintageFace,\" a new benchmark of 300 real old face photos with varied degradation levels, to evaluate performance.\n\nWeaknesses:\n- Limited Benchmark Scale: The newly introduced VintageFace benchmark, while a valuable contribution, is relatively small, with 300 images.\n- Narrow Identity Preservation Test: The evaluation of identity preservation was conducted using CLIP-based distance on a select subset of 50 inputs with well-preserved features, which may not represent performance on more severely degraded images.\n- Specialized Focus: The paper's scope is highly specialized for the compounded degradations common in old photographs, such as breakage, fading, and severe blurring. While the performance is superior, the real-world impact is limited since such degraded old photos are relatively rare.\n\nQUESTIONS:\n- The proposed framework depends on multiple external models (face parsing, scratch detection, style transfer). How sensitive is SSDiff’s performance to the quality of these components?\n- How is the 50 LQ selected for measuring fidelity of restoration?\n- Have you considered evaluating SSDiff on existing open-source datasets related to degraded or historical face images (e.g., CelebA-HQ with synthetic degradations, or subsets from LFW with added artifacts)? Even a small-scale cross-dataset test or synthetic degradation benchmark could help demonstrate that the method generalizes beyond your self-built data.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SSDiff, a training-free framework designed to address specific degradations in old face photos, such as breakage and color fading. SSDiff leverages a pre-trained face diffusion model to generate a pseudo-label, which encapsulates both structural and color information. Based on this pseudo-label, a staged, region-specific guidance scheme is proposed to enhance the restoration process. Additionally, a real-world benchmark, VintageFace, is introduced, and SSDiff demonstrates superior performance compared to existing methods on this dataset.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1. The idea of the paper is straightforward: it uses a clean pseudo-label to guide the inverse process. Additionally, the staged approach proposed by the authors for handling different types of information is quite reasonable. These concepts are supported by analytical results presented in Fig. 2 and Fig. 4, making the overall logic sound.\n2. From both quantitative and qualitative results, SSDiff demonstrates good performance on the VintageFace benchmark.\n\nWeaknesses：\n1. The experimental settings and scenarios are somewhat incomplete:\n\ta. The results in Table 2 are quite puzzling. Without incorporating the self-supervised strategy proposed by the authors, this experimental setup achieves near state-of-the-art performance. Please provide corresponding explanations and clarifications.\n\tb. The paper only presents results for front-facing photos, without showing results for other poses, such as profiles. Additionally, the paper does not include results for the hard-case photos introduced in the VintageFace benchmark. Please clarify how SSDiff performs on these two common cases, as well as on cases involving individuals wearing glasses.\n\tc. Are the results in the table of Fig. 8 based on a single image? If so, the selection of these two scale parameters may not be representative.\n\td. In many of the loss calculations, the authors exclude the eyes, nose, and mouth. How does SSDiff perform if these areas contain significant scratches or degradation?\n\te. How does the author ensure identity preservation? There doesn’t appear to be any constraint on identity during the supervision process.\n\tf. The handling of the background regions is suboptimal (see Fig. 5, Fig. 9, and Fig. S4, third case).\n\n2. Some of the claims are puzzling and may not be entirely accurate.\n\ta. What is the author attempting to convey in lines 191-195? The equation presented appears to be an optimization formulation, but in the experiments, s_{w}sw​ and s_{s}ss​ are empirically chosen constants.\n\tb. In line 241, the authors claim that the face parsing network is robust to degradations. I suggest that the authors include results for some hard cases, such as those with significant degradations or profile poses, to better support this claim.\n\tc. In line 38 of supp, Why does the style transfer model only need to be applied in a single step?\ta. The caption for Figure 3 should be revised for clarity and precision.\n It is unclear what \\hat{x_{t}} represents.\n\nQUESTIONS:\n1. The caption for Figure 3 should be revised for clarity and precision.\n2. It is unclear what \\hat{x_{t}} represents.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}