Title: One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution
Venue: NeurIPS 2025
Accepted: yes
Keywords: Real-world Video Super-Resolution

Diffusion Models
Abstract: It is a challenging problem to reproduce rich spatial details while maintaining temporal consistency in real-world video super-resolution (Real-VSR), especially when we leverage pre-trained generative models such as stable diffusion (SD) for realistic details synthesis. Existing SD-based Real-VSR methods often compromise spatial details for temporal coherence, resulting in suboptimal visual quality. 
We argue that the key lies in how to effectively extract the degradation-robust temporal consistency priors from the low-quality (LQ) input video and enhance the video details while maintaining the extracted consistency priors.
To achieve this, we propose a Dual LoRA Learning (DLoRAL) paradigm to train an effective SD-based one-step diffusion model, achieving realistic frame details and temporal consistency simultaneously.
Specifically, we introduce a Cross-Frame Retrieval (CFR) module to aggregate complementary information across frames, and train a Consistency-LoRA (C-LoRA) to learn robust temporal representations from degraded inputs.
After consistency learning, we fix the CFR and C-LoRA modules and train a Detail-LoRA (D-LoRA) to enhance spatial details while aligning with the temporal space defined by C-LoRA to keep temporal coherence. 
The two phases alternate iteratively for optimization, collaboratively delivering consistent and detail-rich outputs. During inference, the two LoRA branches are merged into the SD model, allowing efficient and high-quality video restoration in a single diffusion step. Experiments show that DLoRAL achieves strong performance in both accuracy and speed. Code and models will be released.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new diffusion-based video super-resolution pipeline that synthesizes high-resolution videos in one-step diffusion sampling. The author proposes a dual-LoRA learning scheme, consisting of one for retrieving information across frames and one for enhancing video consistency. Extensive results show the effectiveness of their pipeline.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper is well-organized and easy to follow. The results in the video demo are convincing and impressive. However, at the same time, I have some concerns about it:\n1. My main concern about this paper is the novelty. The core contribution of this work is the dual-stage LoRA tuning. However, the loss functions of the proposed dual-stage training are very similar, except that the consistency stage utilizes an additional classifier score distillation loss. However, this loss term is not well introduced in the manuscript, and I cannot easily find how and why such a design brings performance gain against existing methods.\n\n2. Missing ablation study. Because the dual-stage LoRA fine-tuning is a core contribution, more visual and numeric comparisons should be provided to demonstrate that such a design leads to prior performance. I see one ablation in the supplementary, but hope to see with/ and w/o each stage.\n\n\nMinor suggestions:\n1. Please add \"Ours\" to the proposed method in the figures when comparing with other methods (e.g., Figure 1).\n\n2. Please keep the method's name and the position of zoomed regions in the video demo consistently. Please do not use brown color that is similar to the background, try some vivid ones like red, light yellow, or green for better visualization. Try to stop at images with the best visualization in the demo to show differences more clearly. \n\nThe paper can be better organized to make it more complete and convincing. I give my current rating based on the assumption that the dual-stage training mechanism brings performance improvements and is unique compared with existing techniques. I will change my rating according to authors' feedback.\n\nQUESTIONS:\nI wonder why the author selects image diffusion model as their solution instead of video diffusion models, which are effective in solving video tasks.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method called DLoRAL to achieve faster, more consistent, and detail-enhanced video super-resolution. DLoRAL consists of a CFR module, C-LoRA, and D-LoRA, which are designed to extract cross-frame complementary information, maintain temporal consistency, and restore image details, respectively. Experiments are conducted to demonstrate the effectiveness of DLoRAL. Compared to previous methods, DLoRAL not only achieves improved performance but also requires less inference time.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well-written and easy to understand.\n2. The design of Cross-Frame Retrieval is interesting, utilizing the top-k most similar K V pairs to compute cross-frame attention.\n3. The proposed method employs a single-step diffusion model, resulting in faster inference speed compared to baselines.\n\nWeaknesses:\n1. Why use the image model SD for video super-resolution tasks? There are already several advanced video generation models, such as Hunyuan Video and Wan2.1 1.3B, with acceptable LoRA training costs. It seems more straightforward to perform video super-resolution tasks on video generation models.\n    - Furthermore, based on my experience, the temporal inconsistency issue may stem from using SD for video generation. Using video generation models might significantly alleviate this issue and does not seem to require specific fine-tuning to enhance temporal generation capabilities, which is necessary when using SD. Therefore, I am concerned about the practical significance of the motivations behind this paper.\n2. The proposed method is based on the UNet architecture, and I am uncertain if it can be adapted to the current DiT architecture.\n3. How can it be ensured that the two designed LoRA implementations achieve true decoupling? The paper claims that D-LoRA is intended to learn high-frequency details, but since the supervision signal comes from video, it will inevitably learn motion information. Moreover, D-LoRA's loss also uses the loss from the Temporal Consistency Stage (C-LoRA).\n4. Will training on constructed pseudo-video data for the enhancement stage affect performance? Since pseudo-videos lack reasonable motion information.\n5. I noticed that the appendix includes some ablation studies on the training strategy; however, the paper lacks detailed ablations for each proposed module, as well as explanations of how these modules contribute to overall performance.\n6. Does the proposed method maintain its faster inference speed when applied to multi-step diffusion models? Or, if the compared baseline uses a single-step diffusion model, is there a significant difference in inference speed compared to the proposed method?\n\nQUESTIONS:\nSee Weaknesses.\n\nI may consider increasing the score once all concerns are addressed.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of Real-World Video Super-Resolution (Real-VSR), where the goals of generating rich spatial details and maintaining temporal consistency are often in conflict. The authors propose a novel framework, Dual LoRA Learning (DLORAL), which leverages a pre-trained Stable Diffusion model to perform high-quality VSR in a single diffusion step. The core idea is to decouple the learning of temporal consistency and spatial detail into two specialized, low-rank adaptation (LoRA) modules: a Consistency-LoRA (C-LORA) and a Detail-LoRA (D-LORA). The model first learns robust temporal priors from degraded inputs using a Cross-Frame Retrieval (CFR) module and the C-LORA. Subsequently, it enhances spatial details using the D-LORA while keeping the consistency modules frozen. These two training stages are alternated iteratively to find an optimal balance. During inference, the LoRAs are merged, enabling efficient, one-step restoration that, as experiments show, outperforms existing methods in both perceptual quality and speed.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The central contribution—decoupling the optimization of temporal consistency and spatial detail via a dual-LoRA, alternating training paradigm—is highly original and significant. This method directly confronts the fundamental trade-off that plagues many generative VSR models. The resulting ability to produce both temporally stable and detail-rich videos marks a notable advancement in the field.\n\n- The paper demonstrates state-of-the-art performance across a wide range of quantitative metrics and on multiple challenging real-world and synthetic datasets. The qualitative results presented in Figures 3 and 4 are particularly impressive, showing clear superiority over competing methods in restoring facial structures and fine textures. Critically, the model achieves this while being remarkably efficient, performing inference in a single step, which is approximately 10x faster than other diffusion-based VSR models. This combination of quality and speed is a major strength.\n\n- The technical execution is solid. The Cross-Frame Retrieval (CFR) module is well-designed, employing mechanisms like top-k attention and a learnable gating threshold to robustly aggregate information from noisy adjacent frames, rather than using a naive fusion. The dynamic, iterative training process with a smooth loss transition strategy is also a thoughtful detail to ensure stable convergence.\n\nWeaknesses:\n\n- The proposed training pipeline is notably complex, involving two separate stages, two different training datasets, alternating optimization, and a carefully scheduled loss interpolation for transitions. While this yields impressive results, the paper lacks an ablation study to justify this complexity. It is unclear how sensitive the final performance is to this specific training curriculum. A simpler joint-training approach with a well-weighted composite loss might potentially achieve comparable results with greater ease of implementation and training stability.\n\n- The Cross-Frame Retrieval (CFR) module, which is critical for establishing consistency, operates with a sliding window of only two frames (the current and the immediately preceding one). This architectural choice may limit the model's ability to handle more complex temporal dynamics, such as occlusions, dis-occlusions, rapid non-linear motion, or re-emerging objects after several frames. The reliance on only the previous frame for temporal priors is a potential bottleneck for long-term consistency.\n\n- For the detail enhancement stage, the authors generate \"video sequences\" by applying random pixel-level translations to single high-quality images from the LSDIR dataset. While this enables the use of an optical flow loss, this synthetic motion is highly simplistic and not representative of real-world camera or object motion. It is unclear if training D-LORA on this artificial data is truly beneficial or if it might bias the model in subtle ways.\n\nQUESTIONS:\nI believe the paper is strong, but my final evaluation could be improved if the authors address the following points:\n\n- Regarding training complexity, could you please provide an ablation study or at least a discussion on the necessity of the alternating, dual-dataset training scheme? Specifically, how does this approach compare to a simpler baseline that performs joint training on a combined dataset, using a single, fixed loss function (e.g., $L_{enh}$ from Eq. 5) from the start? This would help clarify whether the performance gains are truly from the decoupled optimization or from other factors.\n\n- The CFR module's reliance on only the single preceding frame (t−1) seems potentially fragile. How does the model perform in video scenes with significant challenges to this assumption, such as rapid motion, severe occlusions, or scene cuts where the t−1 frame provides little useful information? Have you considered extending the CFR module's temporal window to incorporate information from more distant frames (e.g., t−2, t−5)?\n\n- The generation of motion via random pixel shifts on still images is an interesting choice. How crucial is this simulated motion for the performance of the Detail-LoRA? What happens if D-LORA is trained on high-quality still images alone, without the optical flow regularization term ($L_{opt}$)? Does this significantly degrade temporal consistency, or does the frozen C-LORA and CFR module provide sufficient temporal guidance on their own? Answering this could potentially simplify the data preparation pipeline significantly.\n\nLIMITATIONS:\nYes, the authors have included a limitations section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is interesting. The authors add two LORA to enhance the temporal consistency and details of the original one step diffusion SR network.\n\nSTRENGTHS AND WEAKNESSES:\nExperimental results show obvious improvments with slight computational increments.\n\nStrengths\n1. Decoupled Learning for Conflicting Objectives: By separating temporal consistency (C-LoRA) and spatial detail (D-LoRA) into dedicated modules, DLoRAL avoids the trade-off inherent in joint optimization, allowing each component to specialize in its target.\n2. Effective Extraction of Temporal Priors: The CFR module leverages inherent consistency in LQ videos (despite noise/blur) to aggregate structure-aligned cross-frame information, providing stable guidance for both modules.\n3. Efficient One-Step Inference: Built on a one-step diffusion framework, DLoRAL reduces inference time by ~10× compared to multi-step diffusion methods (e.g., Upscale-A-Video, MGLD), with comparable or better quality.\n4. Strong Performance Across Metrics: DLoRAL achieves state-of-the-art results in perceptual metrics (MUSIQ, CLIPIQA), fidelity (PSNR, SSIM), and temporal consistency (warping error), balancing detail and coherence effectively.\nAlternate Training Strategy: Iterative optimization of consistency and detail stages, with smooth loss transitions, ensures each module converges without interfering with the other.\n\nWeaknesses\n1. Limitations of VAE Compression: Inherits the 8× downsampling VAE from Stable Diffusion, which struggles to restore ultra-fine details (e.g., small text) due to aggressive compression.\n2. Temporal Coherence Disruption: The VAE’s heavy compression may disrupt temporal consistency in LQ frames, making it harder to extract robust temporal priors. A VAE specifically designed for Real-VSR could mitigate this.\n3. Dependence on Pre-trained Models: Relies on the pre-trained Stable Diffusion backbone, which may limit generalization to highly specialized video degradation types not covered by the original model’s training data.\n\nQUESTIONS:\nPerhaps ablation studies can be given to show the contribution of each LORA.\n\nLIMITATIONS:\nCan such design be applied to other image-based diffusion networks? such as image generation network.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}