Title: HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidance
Venue: NeurIPS 2025
Accepted: yes
Keywords: Low-level Vision

Diffusion Model

Human Body Restoration
Abstract: Human-centered images often suffer from severe generic degradation during transmission and are prone to human motion blur (HMB), making restoration challenging. Existing research lacks sufficient focus on these issues, as both problems often coexist in practice. To address this, we design a degradation pipeline that simulates the coexistence of HMB and generic noise, generating synthetic degraded data to train our proposed HAODiff, a human-aware one-step diffusion. Specifically, we propose a triple-branch dual-prompt guidance (DPG), which leverages high-quality images, residual noise (LQ minus HQ), and HMB segmentation masks as training targets. It produces a positive–negative prompt pair for classifier‑free guidance (CFG) in a single diffusion step. The resulting adaptive dual prompts let HAODiff exploit CFG more effectively, boosting robustness against diverse degradations. For fair evaluation, we introduce MPII‑Test, a benchmark rich in combined noise and HMB cases. Extensive experiments show that our HAODiff surpasses existing state-of-the-art (SOTA) methods in terms of both quantitative metrics and visual quality on synthetic and real-world datasets, including our introduced MPII-Test. Code is available at: https://github.com/gobunu/HAODiff.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a two-stage framework for human body restoration with a focus on addressing human motion blur. The first stage constructs a realistic degradation pipeline that simulates spatially-aware motion blur using part-level segmentation and morphological operations, enhancing data diversity. The second stage introduces a Dual-Prompt Guidance (DPG) module to extract semantic priors from degraded inputs, which are used to guide a one-step diffusion model for high-fidelity restoration. The methodology is technically solid and well-motivated, combining detailed degradation modeling with efficient generative restoration. However, more analysis on the effectiveness of individual degradation components and a deeper understanding of the prompt embeddings would further strengthen the work.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The restoration task is well-structured into two clear stages: Dual-Prompt guidance for semantic extraction followed by One-Step Diffusion for efficient detail restoration, effectively addressing the challenge of generating high-quality images from low-quality inputs.\n- The use of diffusion inversion with classifier-free guidance, latent VAE sampling, and Performer-based pooling significantly improves sampling efficiency by reducing the need for multiple diffusion steps, resulting in a focused and technically sound approach.\n\nWeaknesses:\n- The method’s heavy reliance on precise body part segmentation via Sapiens limits practical applicability, with no discussion on obtaining accurate masks in real-world or unsupervised settings.\n\n- Human motion blur modeling remains overly simplistic, ignoring complexities like varying limb velocities, camera exposure, and depth-of-field effects, which may hinder real-world generalization.\n\n- The Dual-Prompt architecture lacks clear motivation and detailed explanation on why joint training of image, residual, and mask predictions is necessary.\n\n- The approach primarily assembles existing modules (SwinIR, Stable Diffusion, Performer, CFG), limiting novelty to engineering integration.\n\nQUESTIONS:\n- How well does the HMB pipeline generalize to more complex human motion scenarios, such as fast multi-limb movements in sports or dancing with occlusions? Please provide results or analysis on such challenging cases.\n- Are there any training stability issues with the three-branch Dual-Prompt setup? Clarify training details and convergence.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel one-step diffusion model, HAODiff, which addresses the challenges of human motion blur and generic degradation in human-centric image restoration through a dual-prompt guidance mechanism. The model leverages a degradation pipeline that simulates realistic degradation scenarios and employs adaptive positive-negative prompts to enhance restoration quality. Experimental results demonstrate its superiority over existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The design of the triple-branch DPG mechanism is innovative. It leverages multiple training targets to generate adaptive positive-negative prompts, which enhances the effectiveness of CFG.\n2. The design of the degradation pipeline and the dual-prompt guidance mechanism is well-founded and addresses a gap in existing research related to human motion blur. The integration of classifier-free guidance through adaptive positive-negative prompts enhances the model's robustness against diverse degradations.\n3. The paper provides extensive experimental results on both synthetic and real-world datasets, including the newly introduced MPII-Test benchmark. The results demonstrate that HAODiff outperforms existing state-of-the-art methods in terms of quantitative metrics and visual quality.\n4. The paper is well-organized, with a logical flow from problem statement to methodology, experiments, and discussion.\n\nWeakness:\n1. The paper lacks evaluations on classic metric like PSNR, SSIM or MAE, which affects reliability of the model's performance.\n2. Fig 8 shows limb poses appear unnatural, and overlapping objects may be misrestored. Can the authors provides the reason and potential solutions?\n\nQUESTIONS:\nsee weakness\n\nLIMITATIONS:\nThe authors present failure case but lack detailed interpretation and potential solutions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the challenging problem of human body restoration, with a specific focus on handling the co-occurrence of generic image degradation and human motion blur (HMB), a scenario often overlooked by existing methods. The authors propose a comprehensive solution with three main components: A new degradation pipeline that synthetically generates realistic training pairs by first simulating HMB on specific body parts and then applying a sequence of generic degradations. A novel prompt generation module, named Dual-Prompt Guidance (DPG), which is a triple-branch network trained to predict the high-quality (HQ) image, the residual noise (LQ - HQ), and an HMB segmentation mask. These outputs are then used to form adaptive positive and negative prompts for the diffusion model. A one-step diffusion model, HAODiff, that leverages these adaptive prompts via classifier-free guidance (CFG) to achieve efficient and high-fidelity restoration. The authors also introduce MPII-Test, a new benchmark dataset curated for evaluating HBR performance on real-world images with significant motion blur. Extensive experiments show that HAODiff surpasses existing state-of-the-art methods quantitatively and qualitatively.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nThe paper correctly identifies a significant gap in existing blind image restoration (BIR) and HBR research: the lack of explicit handling for human motion blur, which is a very common artifact. The proposed degradation pipeline (Sec 3.1) is a logical and direct way to address this, and its design, which places HMB in the first-order degradation stage, is physically plausible.\n\nThe paper is very well-written and easy to follow. The methodology is explained clearly, and the figures, are illustrative and aid in understanding the proposed framework.\n\nWeaknesses\n\n1. While the guidance mechanism (DPG) is novel, the second stage of the model—the one-step diffusion (OSD) process described in Section 3.3, is largely built upon existing frameworks like OSDHuman and other one-step models. The use of a standard UNet with LoRA, a VAE encoder/decoder, and the standard CFG formulation means the contribution is more about how to guide an existing diffusion framework rather than advancing the diffusion process itself. **This is not a major flaw but slightly reduces the fundamental nature of the contribution.**\n\n2. The proposed solution involves a two-stage training process: first training the DPG module, and then training the OSD model using the frozen DPG as a prompt generator. This introduces **significant complexity** in terms of training procedure and pipeline management compared to end-to-end trainable models.\n\n3. In Section 3.2, the DPG is built with a backbone of two RSTBs (with six Swin Transformer Layers) and three reconstruction branches of another two RSTBs (with three STLs). While this is inspired by SwinIR, it is a rather heavy architecture for a guidance module. It is unclear if this level of complexity is necessary. A lighter backbone might have achieved similar guidance quality while improving overall efficiency.\n\nQUESTIONS:\nRegarding the HMB-R metric, the paper mentions it is calculated using a fine-tuned YOLO model (Sec 4.1), with details in the supplementary material. Given that this is a key metric for validating the central claim of the paper, could the authors provide a brief summary of the detector's performance (e.g., mAP on a hold-out set) in the main paper? The reliability of this metric is crucial for interpreting the results in Table 2.\n\nThe DPG architecture (Sec 3.2) features dedicated branches for residual noise and HMB masks. How would this framework generalize to other types of structured degradation, such as heavy rain, snow, or haze? Would it require adding a new, specialized branch for each new degradation type, potentially making the model cumbersome? What are the authors' thoughts on the scalability of the DPG's multi-branch design?\n\nSome other concerns are listed in Weaknesses\n\nLIMITATIONS:\nThe authors have provided a good discussion of limitations in Section 5, acknowledging failure cases in scenarios with extreme degradation and the modest efficiency penalty from the dual-pass UNet required for CFG. To build on this, I would add:\n\nThe paper does not discuss the limitation of its two-stage training pipeline (training DPG, then training OSD), which can be more complex and resource-intensive to manage and tune compared to end-to-end approaches.\n\nAs raised in the questions, the current design of DPG with specialized branches for specific degradations might not scale well to a wider variety of real-world distortions, which could be considered a limitation of the current framework's generalizability", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a solid solution pipeline for image restoration tasks under human body scene, which consists of a degraded samples generation pipeline, an image restoring pipeline and a human body restoration metrics dataset. The degradation pipeline generate degraded images from high quality images through three paths, which cover the usual image degradation scenes (blur, erosion, dilation, noise, down-sampling, compression), and specially HMB scenes (Human Motion Blur). The restoration pipeline adopts a novel dual-prompt-guidance strategy that split and convert features into positive and negative prompt of CFG. And it outperforms previous methods numerically and visually. Also, the methods is relative light-weight and less performance-consuming, which makes it possible for application on light-weights devices like phones, tables and drones.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper is well organized and the experimental results (metrics and examples) are sufficient to support its claims.\n\nThe method achieves SOTA and outperforms previous method from both numerical metrics and visual quality.\n\nThe method adopts light-weight modules (e.g. swin-transformer-blocks and performer) and one-step diffusion schema, which makes it fast and less performance-consuming. So, it has the potential availability for inferencing on light-weight devices like phones (which is under the most common scene that users need human blur restoration), tablets and drones. \n\nThe proposed strategy dual-prompt guidance is interesting, which logically split and convert additional features into negative and positive prompt of classifier free guidance, has the potential usage for other tasks.\n\nThe ablation study is detailed and convincing.\n\nWeakness:\n\nThe method is very complex, including training and fine-tuning of multiple models and multiple stages, and also generative adversarial training.\n\nOther: See Questions.\n\nQUESTIONS:\nFor the sample generation pipeline:\n\nIt seems that all the samples finally pass through the compression process (Figure 2). But under the real scene, devices (phones and cameras) usually store uncompressed images. Does the pipeline takes this into consideration and generates samples without compression?\n\nUsually, different parts of human bodies can simultaneously have motion blur in different directions. Does the pipeline takes this into consideration?\n\nDoes the pipeline disentangles the motion blur of camera and motion blur of human body, which could happens at same time?\n\nFor the restoration pipeline:\n\nHow much is the performance gain from adversarial training?\n\nLIMITATIONS:\nThe limitations have been thoroughly and clearly discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-17"}