Vivid-VR: Distilling Concepts from Text-to-Video Diffusion Transformer for Photorealistic Video Restoration
Authors: Haoran Bai
Xiaoxu Chen
Canqian Yang
Zongyao He
Sibin Deng
Ying Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Video Restoration
Diffusion Transformer
Text-to-Video
ControlNet
Concept Distillation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We present Vivid-VR, a DiT-based generative video restoration method.

Abstract: We present Vivid-VR, a DiT-based generative video restoration method built upon an advanced T2V foundation model, where ControlNet is leveraged to control the generation process, ensuring content consistency. However, conventional fine-tuning of such controllable pipelines frequently suffers from distribution drift due to limitations in imperfect multimodal alignment, resulting in compromised texture realism and temporal coherence. To tackle this challenge, we propose a concept distillation training strategy that utilizes the pretrained T2V model to synthesize training samples with embedded textual concepts, thereby distilling its conceptual understanding to preserve texture and temporal quality. To enhance generation controllability, we redesign the control architecture with two key components: 1) a control feature projector that filters degradation artifacts from input video latents to minimize their propagation through the generation pipeline, and 2) a new ControlNet connector employing a dual-branch design. This connector synergistically combines MLP-based feature mapping with cross-attention mechanism for dynamic control feature retrieval, enabling both content preservation and adaptive control signal modulation. Extensive experiments show that Vivid-VR performs favorably against existing approaches on both synthetic and real-world benchmarks, as well as AIGC videos, achieving impressive texture realism, visual vividness, and temporal consistency. The codes and checkpoints are publicly available at https://github.com/csbhr/Vivid-VR.

Published as a conference paper at ICLR 2026
VIVID-VR: DISTILLING CONCEPTS FROM TEXT-TO- VIDEO DIFFUSION TRANSFORMER FOR PHOTOREAL- ISTIC VIDEO RESTORATION
Haoran Bai, Xiaoxu Chen, Canqian Yang, Zongyao He, Sibin Deng Ying Chen' Alibaba Group, China
Real-world video
AIGC video
Restored video
Restored video
(a) Bicubic
(b) SUPIR
(C) UAV
(d) MGLD
(c) ReedVRR-715
(1 Vivid-VR
Figure I: Video restoration results on both real-world and AIGC videos. To mitigate drift during fine-tuning of the controllable generation pipeline, we propose concept distillation strategy that preserves both texture realism and temporal coherence in generated videos. Leveraging this strategy, Vivid-VR achieves impressive texture realism and visual vividness. (Zoom-in for best view)
ABSTRACT
We present Vivid-VR, DiT-based generative video restoration method built upon an advanced T2V foundation model, where ControlNet is leveraged to control the generation process, ensuring content consistency. However, conventional fine- tuning of such controllable pipelines frequently suffers from distribution drift due to limitations imperfect multimodal alignment, resulting in compromised tex- ture realism and temporal coherence. To tackle this challenge. propose con- cept distillation training strategy that utilizes the pretrained T2V model to syn- thesize training samples with embedded textual concepts, thereby distilling its conceptual understanding to preserve texture and temporal quality. To enhance generation controllability,
redesign the control architecture with two key com- ponents: 1) control feature projector that filters degradation artifacts from input video latents minimize their propagation through the generation pipeline, and 2) new ControlNet connector employing a dual-branch design. This connector synergistically combines MLP-based feature mapping with cross-attention mecha- nism for dynamic control feature retrieval, enabling both content preservation and adaptive control signal modulation. Extensive experiments show that Vivid- VR performs favorably against existing approaches on both synthetic and real-world benchmarks, as well as AIGC videos, achieving impressive texture realism, vi-
"Corresponding author

---

Published as a conference paper at ICLR 2026
sual vividness, and temporal consistency. The codes and checkpoints publicly available at http:://gitubb.co///bb///iiiiiii
INTRODUCTION
Video restoration aims to recover lost textures, fine details, and structural information from low- quality (LQ) input videos to produce high-quality (HQ) ones. Traditional acconstructoon-baee methods typically employ CNNs Wang et al. (2019); Pan et al. (202T); CChan et al. (2021 2022a b)
and Transformers Liang et al. (2024 2022) extract visual cues for quality enhancement. How-
ever, these approaches face inherent limitations due to insufficient prior knowledge and the ill-posed nature of the inverse problem, reconstructing high-quality textures directly from severely degraded inputs remains extremely challenging. While GAN-based methods Wang et al. (2018) 2021 can generate some textures to certain extent, their generative capacity remains limited.
Recent years have witnessed significant advancements in diffusion-based generative models Rom- bach et al. (2022): Podell al. (2023); Blattmann al. (2023), which can now synthesize pho- torealistic content. This progress has established generative video restoration as a promising new paradigm. While initial explorations using text-to-image (T2I) diffusion models have shown im- pressive results in image restoration tasks Wang et al. (024); Yu et al.] (2024); (Chnn et al. (2025a), their direct application to video sequences suffers from significant temporal inconsistencies due to inadequate motion modeling. Early attempts to address this limitation typically incorporate tempo- ral enhancement mechanisms, including adding trainable temporal layers to diffusion denoisers and VAE decoders Zhou et al. (2024), employing optical flow-based motion compensation Yang et al. 2024a). However, these post- cost-modification during model fine-tuning are insufficient for achieving robust temporal coherence. The advent of Diffusion Transformers (DiT) Peebles & Xie (2023) has enabled a significant leap forward, with text-to-vides (T2V) models Yang et al. (2024b) now capable of generating both high-quality and temporally stable video content. This has spurred the develop- ment of T2V-based restoration approaches. For instance, SeedVR Waan et al.] (2025b) integrates the shift-window attention mechanism with DiT for computational efficiency, and STAR Xie et al. (0255) proposes a dynamic frequency loss for enhanced fidelity, both achieving decent results.
Despite their advancements, current restoration methods still underperform native T2V models in both texture realism and temporal coherence. This performance gap stems primarily from distribu- tion drift induced by imperfect multimodal alignment during the fine-tuning process. This issue is not prominent in the T2V model pretraining phase because of the large, diverse training dataset. But the challenge becomes significantly amplified when fine-tuning these models for video restoration, manifesting as unrealistic textures and compromised temporal consistency.
To overcome this challenge, we propose a concept distillation training strategy that leverages syn- thetic data generated by a pre-trained T2V model. The proposed approach begins with source video and its corresponding text description obtained through a vision-language model (VLM). We first corrupt the source video with noise, then employ the pre- pre-trained T2V model to perform denois- ing while incorporating the text description. This process yields a vidco that encapsulates the T2V model's s semantic understanding of the textual concepts, ensuring inherent modal alignment between the generated video and text description in the T2V model's latent space. By blending these synthe- sized data with real training samples during fine-tuning, our method successfully transfers the T2V model's conceptual knowledge to the video restoration model, thereby mitigating the distribution drift problem while preserving both texture realism and temporal coherence.
Furthermore, we use ControlNet Zhang et al. (2023a) for generation control and introduce two key innovations. First, we develop a control feature projector, which effectively filters degradation ar- tifacts to minimize their propagation through the generation pipeline. While Faith Chen et al. (202a)) achieves similar functionality by jointly fine-tuning the VAE encoder which expensive to train, our solution implements this feature projector as a lightweight CNN-based extension to the VAE encoder. Second, we redesign the ControlNet connector with a dual-branch architecture. Dif- ferent from existing connectors Yu et al. (2024) which fail to properly consider DiT features during fusion, we combine an MLP branch with a cross-attention mechanism, enabling dynamic feature retrieval that preserves the generation quality and realism of native T2V models. Benefiting from these improvements, the proposed method, named Vivid-VR. achieves impressive texture realism and visual vividness (see Figure
2

---

Published as a conference paper at ICLR 2026
In summary, main contributions follows:
We propose a novel concept distillation training strategy that leverages a pre-trained T2V model to synthesize aligned text-video pairs, effectively mitigating distribution drift during fine-tune tuning and preserving texture and temporal quality.
We improve the ControlNet architecture by introducing lightweight control feature pro- jector and a dual-branch connector, enabling degradation artifact removal and dynamic control feature retrieval.
The proposed Vivid-VR performs favorably against existing methods on both synthetic and real-world benchmarks, as well AIGC videos.
2 RELATED WORK
Reconstruction-basee Video Restoration. Early approaches focused on architecture design and loss functions for direct HQ reconstruction from degraded inputs. CNN-based methods employed various strategies for temporal information integration, including optical flow estimation Caballero al. (2017); Pan al. (2020). deformable convolutions Wang al.] (2019). bidirectional feature propagation Chan al. ). and optical flow-guides deformable alignment modules Chan et al. (2022a). Transformer- based methods Liang al. (2024 2022) improved performance through at- tention mechanisms for long-term spatio-temporal modeling. Meanwhile, some studies Wang et al. Z0ann: et al. (202I) have introduced more complex degradation simulations to improve real- world generalization. To produce richer textural details, GAN-based frameworks Wang et al. (2018) 2021 are consequently adopted that incorporate adversarial training. Despite these advances, meth- ods relying solely input-derived cues without strong priors still produce overly smoothed results when handling severely degraded content.
Diffusion-based Video Restoration. Diffusion-based generative models Rombach al. (2022) Podell et al. (2023); Yang et al. (2024b) have made significant progress, which introduce a new paradigm for restoration tasks. Initial explorations focused on image restoration Wang et al. (2024); Yu et al. )0224; Chen et al. (2025a), and they achieved remarkable results. However, these ap- proaches fundamentally lack temporal modeling capabilities, resulting in severe frame inconsis- tencies when directly applied to video sequences. Early solutions Zhou et al. (2024); Yang et al. (2024a) attempted to mitigate this through temporal enhancement techniques, such as incorporating trainable temporal layers or implementing optical flow-based motion compensation, yet these post- hoc adjustments proved inadequate for ensuring robust temporal coherence. Diffusion Transformers (DiT) Peebles & Xie (2023) enabled high-quality T2V generation Yang al. (2024b) with superior temporal stability, inspiring video restoration methods. ScedVR Wang et al. (2055b) combines the shift-window attention mechanism with DIT to improve computational efficiency. STAR Xie et al. 2025) designs a dynamic frequency loss function to improve fidelity. Concurrently, efforts to im- prove inference efficiency have led to the design of one-step diffusion models WWng al.) (2025a); Chen et al.] (2025b). Nevertheless, existing methods still exhibit noticeable gaps in texture realism and temporal consistency compared to native T2V models, due distribution drift from imperfect multimodal alignment of training data. To bridge this gap, our work introduces concept distillation training strategy that effectively preserves the texture and temproal quality of the base T2V model.
3 METHOD
The proposed Vivid-VR leverages the advanced T2V model (i.e., CogVideoX1.5-SD Yang et al. (2024b) as its foundation, incorporating the ControlNet /hang et al. (2023a) to condition the gener- ation process on input videos. Figure 2 shows an overview of the proposed method. In this section, we first present the model architecture of the proposed method, and then explain the proposed con- cept distillation training strategy.
3.1 MODEL ARCHITECTURE
Text Description Generation. Building upon the T2V-based framework, the proposed method re- quires both LQ input video and corresponding text descriptions. We employ CogVLM2- Vog--ii-Video Yang et al. (202bb) for text generation maintain consistency with Vog 1.5-5B's training config-
3

---

Published as a conference paper at ICLR 2026
Trainable
Add
xP-1
CogVLMM Video
proven halee drnood
Tinatt
Cress
xpor 1.
Attentoon
Text bae
Contrel
MLP
Malr Srazt
Connector
Parchify
Blocks x
LQ ispul video
ControlNet
ControlNiet
HQ cutput video
Figure 2: An overview of the proposed method. Vivid- VR first processes the LQ input video with CogVLM2-Videe generate text description, which is encoded into text tokens via T5 encoder. Simultaneously, the 3D VAE encoder converts the input video into latent, where our control feature projector removes degradation artifacts. The video latent is then patchified, noised, and combined with text tokens and timestep embeddings as input to DiT and ControlNet. For enhanced control- lability, we introduce a dual-branch connector: an MLP for feature mapping and a cross-attention branch for dynamic control feature retrieval. After T denoising steps, the 3D VAE decoder recon- structs the HQ output. Only the control feature projector, ControlNet, and connectors are trained via the proposed concept distillation strategy, and other parameters remain frozen.
uration. Given the input LQ video, Cog CLgVLMM-VIL Video produces an aligned text description, subse- quently encoded into text tokens through the T5 Raffel et al. (2020) text encoder.
parallel with text tokens generation, we preprocess the LQ input video to generate corresponding visual tokens for DiT and ControlNet. The preprocessing pipeline begins by encoding LQ video through the VAE encoder, producing the latent representation that contains both content information and degradation artifacts. Since these degradation artifacts may negatively impact generation quality, we propose a lightweight Control Feature Projector to eliminate them. The proposed projector consists three cascaded spatiotemporal residual blocks that effectively filter the degraded features, outputting cleaner latent representation. The video latent then patchified and noise injected form the visual tokens for subsequent processing.
Control Feature Preprocessing.
ControlNet Pipeline. Given the text tokens, the visual tokens and the timestep embedding, DiT and ControlNet both perform T denoising steps. DiT comprises N DiT blocks, while ControlNet contains N/7 blocks initialized from DiT's 's' first N/7 ones. During denoising process, ControlNet's visual tokens are integrated into DiT through N proposed mral-branch Connectors. For the 7th connector, the fusion process of the control visual tokens is:
-'= (.I++MMP(clT7))
CA('', (l/7))))/)))
where 1 denotes the visual tokens from the 7th DIT block; cl$/7] represents the corresponding ControlNet block visual tokens aligned with the 5th DiT block; MLP((.) and C'((-) are the MLP layer and cross attention module respectively; f is the fused visual tokens. After T denoising steps, the visual tokens are unpatchified and fed into the VAE decoder to generate the final HQ outputs.
3.2 CONCEPT DISTILLATION TRAINING STRATEGY
Training Data Collection. Effective training of DiT-based video restoration models demands exten- sive high-quality text-video pairs, but existing public datasets et al. (2017); ; Nah al. (2019); Xue et al. (2019)][Serggen & Poppe (202)) lack in both scale and diversity. To address this, we collected a large-scale video pool consisting of million videos with resolutions higher than 1024 x 1024, frame rates higher than 24, and durations higher than 2 seconds. These videas cover wide range of scenes, including portraits, natural landscapes, plants and animals, urban landscapes, etc. To en- sure video quality, we further screened these videos using the no-reference video quality assessment metrics Wu et al. (202)); y Zhang et al. (2023b) to remove low-quality videos. For the remaining HQ videos, generated text descriptions using CogVLM2-Vide consistency with Cog VideoX1. 1.5-5B configuration. The final curated multimodal training dataset comprises 500K text-video pairs with exceptional quality and variety.
Yang al. (2024b), maintaining
4

---

Published as a conference paper at ICLR 2026
Figure 3: Example videos generated by the proposed concept distillation train- ing strategy. The top row presents source videos, and the second row shows corresponding generated videos after embedding textual concepts via the T2V model. Due to VLM cap- tioner limitations, the source videos exhibit imperfect alignment with their text descriptions, while the generated videos have better modality alignment. (Zoom-in for best view)
an alanddondd industrial with lighthunse... parkod white
a peseeful countryside... ... is dutted with trees with leavess...
foncal landsespe.. it revials rocky outinop and gentle sky...
Concept Distillation. Due to limitations of the VLM captioner, the constructed text-video data pairs are not perfectly aligned (see Figure row). This may lead to distribution drift during fine- tuning, degrading output video quality. While developing a more accurate VLM captioner could enhance text-video alignment, it has two drawbacks: (1) it is costly, and (2) discrepancies may per- sist in the T2V model's 's latent space, potentially still leading to distribution drift. Instead, we address this issue by distilling the text-concept understanding capabilities of the T2V base model into the video restoration model. To this end, we employ the T2V model itself (i.c., 1og to perform text-guided rideo-t-vvidoo translation, generating training data for distillation. Specifi- cally, given a text- video pair, we perturb the source video by adding noise with a standard deviation corresponding to T/2 time steps. We then apply Cog VogVieXoXI.5551 steps, conditioned on the text description, yielding a synthesized video with inherent alignment to text concepts in this T2V model's latent space. As illustrated in Figure (second row), the generated video largely retains the source content, but some concepts have been modified to better align with those in the text description. We randomly extract tixt- pairs from the constructed multimodal training dataset and employ the aforementioned process to generate 100K sample pairs. These gen- erated samples are then combined with the original training dataset to facilitate fine-tuning of the control module in our DiT-based video restoration model.
denoise the video over T/2
Model Training. Following the settings of Cog V.deo Yang et al. (002b)), we employ e- prediction for training, and the loss function is:
L = Exx.t..
|||0 (10,, _² x text 10)l .01"
(2)
where Tu is the HQ video sampled from training dataset; zent and x are the corresponding text description and the synthesized LQ video using the degradation model from Wang al. (202I); and are the time step and noise; x; Vārzo + v1 die is the noised latent of xp, and or is the cumulative multiplication of the variance corresponding to time step 4; ni denotes the networks of DiT and ControlNet (including the control feature projector and the connectors); is the optimiza- tion target, which is defined as t = Vage - VI 0, During training, only the control feature projector, ControlNet, and connectors are trained, and other parameters remain frozen.
EXPERIMENTAL RESULTS
In this section, we evaluate the proposed Vivid-VR on both synthetic and real-world benchmark datasets and compare it with ofateoof-the-art methods.
4.1 IMPLEMENTATION DETAILS
The overall training dataset includes 500K real videos and 100K generated videos, well as their
corresponding text descriptions. During training, we resize the short side of these videos to 1024 pixels and then center-crop them to 1024 1024 resolution. The number of training video frames is randomly selected between 17 and 37. We use the A Lossmizer & Hutter (2017) with learning rate of 0.0001, and adopt cosine annealing learning rate scheme Wang et al. (2019)
Loshchilov
5

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparisons on benchmarks, including synthetic (SPMCS, UDMIO, YouHQ40), real-world (VideoLQ, UGC50), and AIGC (AIGC50) videos. The best and second performances are marked in red and blue, respectively.
Dalasets
Metrics
Real- ESRGAN
SUPIR
MGLD
LAV
STAR
DOVE
SecdVR
SeedVR2
Vivid-VIR
PSNR
23.19
21.86
21.02
23.01
24.18
24:80
24.08
21.73
SSIM
0.690
0.609
0.595
0.606
0.720
0.754
0.689
0.777
0.604
LPIPS
0,230
0.304
0.281
0.277
0.301
0.168
0,263
0.191
0.278
SPMCS
NIQE
5,393
3.494
3,790
3,503
7.058
4.031
4,514
4.969
3.457
MUSIQ 1
51.39
65.23
58.02
66.11
30.62
63.29
56.99
53.23
70.03
CLIP-IQA
0.469
0.357
0.427
0.254
0,410
0.347
0.483
DOVER
8,235
10.07
7,981
8.987
4.266
9.898
9,779
8.625
11:35
MD-VQA
79.16
82,88
78,92
81,90
74.87
83,07
79.56
78.78
86.55
PSNR
27.57
27.02
28.97
28.20
27.29
30.53
27.80
29.04
24.54
SSIM
0,857
0.816
0.873
0.826
0.855
0.894
0.848
0.884
0.761
LPIPS
0.187
0.208
0.158
0.196
0.167
0.101
0.148
0.117
0,243
UDMIO
NIQE
5,835
4,438
4.827
5.109
6.072
5.055
5,345
5.641
4,046
MUSIQ
56.19
$5.17
50.29
64.71
CLIP-IQA
0.418
0.333
0.289
0.340
0.273
0.272
0.426
DOVER
9.402
10.49
9.319
9.774
9.454
10.41
9,349
8.752
11.97
MD-VQL VQA
83.51
85.21
83.89
83.14
82,10
83.99
80.15
79.88
90.05
PSNR
23.02
21.57
23.24
22.31
22.92
24.10
22.46
24.00
21.31
SSIM f
0.655
0.585
0.639
0.592
0.657
0.688
0.621
0.693
0.579
LPIPS
0.347
0.350
0.340
0.433
0.283
0.240
0.357
YouHQ40
NIQE
3.299
3.127
4,456
4.243
MUSIQ
60.03
68.46
59.40
65.97
36.36
60.65
61.91
59.34
70.55
CLIP-IQA
0.389
0.485
0.362
0.427
0.279
0.356
0,360
0.336
0.447
DOVER
12.60
12.93
11.01
12.36
7.868
12.52
14.00
12,80
14.61
MD-VQL IDA
88.85
89.44
86.24
87.35
76.89
86.51
87.51
85.82
92.92
NIQE L
5.014
4.628
4.565
4.591
5.789
5.049
4,994
5.674
4.371
MUSIQ
54.45
57.70
55.82
50.52
55.
62.47
VidcoLQ
CLIP-IQA
0,287
0.299
0.297
0.262
0.265
0.271
0.229
0.220
0.338
DOVER
8.453
8.609
8,830
7.777
8.758
8.780
7,240
6.331
9.743
MD-VQA
80.50
77.32
80.67
78.02
78.56
79.33
74.80
73.52
83.14
NIQE
5,866
5,396
4.633
5,350
5.754
5.493
5.662
6,230
4,361
MUSIQ f
52.22
58.25
61.42
54.71
55.01
57.82
49.76
46.12
67.61
UGC50
CLIP-IQA
0,318
0,382
0.396
0.353
0.353
0.353
0.305
0.276
0.450
DOVER
10.25
12.01
11.78
10.44
10.92
11.84
10.47
MD-VQA
80.85
82,27
81.12
81.93
82.30
75.49
88.89
NIQE
5,680
5,206
4,953
5.579
5.737
5.278
5.029
5.973
4.184
MUSIQ
54.26
58.11
61.39
57.62
51.66
62.07
61.61
49.35
67.18
AIGCS0
CLIP ILIP-IQA
0,349
0,380
0.391
0.376
0.309
0.379
0.378
0.290
0.445
DOVER
12.36
13.33
12.70
12.28
12.10
14.49
14.46
11.34
14.51
MD-VQA
84.56
84.80
85.45
83.06
86.97
85.54
81.47
80.37
89.69
We train Vivid-Va WWid-VRRan on 32 NVIDIA H20-96G GPUs, with batch size of per GPU. The number of training iterations is 30K, and the entire training process takes approximately 6K GPU hours. For inference, we set the number of denoising steps to and used the DPM solver Lu et al. (2022). To maintain consistency with training settings, run inference on videos at 1024 x 1024 resolution. For higher resolution inputs, we employ aggregation sampling Wang et al. (2024) with direct block concatenation rather than Gaussian beegshen-weighhed averaging to prevent overlapping region artifacts.
4.2 QUANTITATIVE RESULTS
To evaluate the performance of the proposed algorithm, we compare Vivid-VR against state-of- the-art approaches, including reconstruction--asse methods (Real-ESRGAN Wang et al. (202 generative image restoration methods (SUPIR Yu et al. (2024)). and generative video restoration methods (UAV Zhou et al. (2024), MGLD Yang et al. (0244)) STAR.Xc et al. (2025), DOVE al. (0225). SeedVR-75 Wang et al. (0255)). SeedVR2-7B Wang al. (2025a)). The evaluation covers synthetic (SPMCS Tao et al. (2017). UDM10 Yi et al. (2019), YouHQ40 Zhou et al. (2024) and real-world (VideoLQ Chan et al. (022b)) benchmarks. Furthermore, we construct two test- sets, containing real-world UGC videos (UGC50) and AIGC videos (AIGC50). For real-world and AIGC videos lacking of ground truth, we employed no-reference image (NIQE Mittal et al. (2012).
MUSIQ Ke et a (2021), CLIP-IQA Wang al. (203)) and video quality assessments (DOVER Wul ct al.] (2023), MD-VVV VQA Zhang et al. (2023b)). For synthetic benchmarks, we supplemented these metrics with full-reference evaluations (PSNR, SSIM, and LPIPS /Zhan al. (008)).

---

Published as a conference paper at ICLR 2026
Bicubie
SUPIR
MGLD
Inpul
(gl) DOVE
SeaVVR-7
Vivid-VR
(b2) Ricubic
(e2) SUPIR
(d2) UAV
(c2) MGLD
(+2) Input
(2) STAR
(g2) DOVE
(h2) BoodVR-7
(2) Vivid-VR
(b3) Bicubic
SUPIR
(d3) UAV
MGLD
Input
(g3)
SeadVR-7B
Vivid-VR
Qualitative comparison results on synthetic (first row), real-world (second row), and AIGC (third row) videos. The proposed Vivid-VR produces the frames with more reasonable struc- tures, as well as more realistic and vivid textures. (Zoom-in for best view)
Figure 4:
10
DR
2016 10
DNL
1821
910
110
410
410
410
11
110
(6) Input (85 frame)
(b) Bécubic
(c) SUPIR
(4) STAR
(6) SeedVR-7B
( mol
Vivid-VR
Figure 5: Visual comparison results on temporal consistency. (a) displays the #5 frame of the input video, and (b)-(g) present the outputs of the compared methods at frames 0, 5, and 10, where "CD" denotes the proposed concept distillation. Vivid-VR demonstrates superior temporal coherence, as evident from the consistent structure of windows and doors throughout the sequence. (Zoom in the red arrow area in each frame)
Table presents quantitative comparisons on 6 benchmark testsets. The proposed Vivid- VR signif- icantly outperforms existing methods in no-reference metrics, achieving the best results in almost all metrics. At the same time, we also note its advantages in full-reference metrics appear less pronounced. We argue that this arises primarily from the inherent limitations of these metries, which often fail to align with human perceptual preferences. For example, the LPIPS values of Figure gl) and (il) are 0.3112 and 0.4297, respectively, while Figure 4il) is more prefered by human. This phenomenon becomes particularly evident in generative restoration scenarios where severe imput degradation allows multiple plausible HQ outputs. making full-reference metries inad- equate for quality assessment. This has also been mentioned in Yu et al. (2024) and has been noticed by quality assessment studies Blau&& & Michaeli 2018); Jinjin et al. (2020): (uu et al. (2022).
7

---

Published as a conference paper at ICLR 2026
Table 2: Ablation studies of the proposed method on the UGC50 testset, where "FT"" denotes "fine-tuning", "CA" denotes "cross attention", "SK" denotes "replacing MLP with skip connec- tion", "QW" denotes "using Qwen2.5-VV VL as VLM captioner", "From scratch" denotes "synthesiz- ing videos from scratch in concept distillation", and (i) is the setting of the proposed Vivid-VE
Control Feature Preprocessing
Conroolee Connectors
Concept Distillation
NIQP.
MUSIQ
CLIP-IQA
DOVER
Enc
Projector
(a)
4,622
63.06
0.414
13.98
(b)
4,632
64,31
0.408
14.40
(G)
X
5.183
59.78
0.374
13.04
(d)
X
SK
4,730
63.91
0.401
13.71
V
4.771
61.21
0.389
13.77
V
X
5,364
57.36
0.363
12.99
<
QW
5.253
60.88
0.354
13.45
(b)
4.3
QUALITATIVE RESULTS
Figures I and present visual comparisons with existing methods synthetic, real-world, and AIGC videos. The proposed Vivid-VR achieves remarkable texture realism and visual vividness. Notably, Vivid-VR is able to generate reasonable and clear structures, such the house shown in Figure ill), while existing methods exhibit structural distortions, artifacts, and loss of fine details (see Figure4ell ())(()))) Moreover, the proposed Vivid-VR produces more realistic and delicate tex- tures on human portraits and animal fur (see Figure and (i3)), while existing methods frequently yield either overly blurred or oversharpened outputs that are perceptually unrealistic.
In addition, Figure 5 shows visual comparisons on temporal consistency. As shown in Figure 5ig), the proposed Vivid-VR demonstrates superior coherence. For example, the structures of the win- dows and doors are well-consistent throughout the sequence. In contrast, SUPIR shows frame-wiss wise inconsistency it is an image-based based restoration approach (see Figure (fc)). While STAR and SeedVR-7B leverage T2V frameworks, their fine-tuning- cuningg-nduued distribution drift compromises temporal consistency (see Figure de))- Notably, Vivid-VR exhibits similar degradation when without using the proposed concept distillation strategy (see Figure
5 ANALYSIS AND DISCUSSIONS
have shown that the proposed Vivid-VR performs favorably against state-of-the-ar methods. To better understand the proposed algorithm, we perform further analysis on the key components.
5.1 EFFECT OF THE CONTROL FEATURE PROJECTOR
The T2V base model is trained on HQ videos, making direct use latent features from LQ videos detrimental generation quality. To address this issue, we propose control feature projector to remove degradation artifacts from the LQ video latents. To validate its effectiveness, we conducted an ablation study by disabling this module while keeping other settings unchanged. As shown in Table ((a) vs (i)), the proposed control feature projector is able to improve video restoration. SUPIR attempts tackle this challenge by independently fine-tuning the VAE encoder. However, this decoupled optimization creates feature incompatibility with subsequent DiT and ControlNet, leading to suboptimal results (Table (b)). While this problem can be solved through joint VAE encoder and video restoration optimization, this approach is expensive train. In contrast, proposed lightweight control feature projector can achieve similar effects at lower training cost.
In addition, visualize the features before and after passing through the control feature projec- tor. As shown Figure 6. features from the low-quality input exhibit degradation artifacts, such as blurry contours. In contrast, after being processed by projector, these features become signifi- cantly sharper and more defined. This provides evidence that the proposed control feature projector effectively filters out degradation artifacts from the input latent signal.
8

---

Published as a conference paper at ICLR 2026
(a) Input
(b) Belore CFP
(c) Alter CFP
(a) Sourve video
(b) From hall-noised
[c] From scratch
Figure 6: Visualizations of features before and af- ter passing through the control feature projector, where "CFP" denotes "control feature projector".
Figure 7: Example videos generated from half- noised real samples and from scratch in the pro- posed concept distillation process.
Bicubic
ZeooFF
Bieubie
ScedVR-78
(c) MLP
SK-CA
MLP
Input
(e) CD
(i QW
Vivid-VR
Figure Effect the dual-branch connector, where "SK" denotes "replacing MLP with skip connection" " and "CA" denotes "cross attention".
Figure 9: Effect of the concept distillation, where "CD" denotes "concept distillation" and "QW" denotes "using Qwen2.5- VL as VLM captioner".
5.2 EFFECT OF THE DUAL- BRANCH CONNECTOR
For the ControlNet connector, we propose a dual-branch architecture combining an MLP for feature mapping with cross attention mechanism for dynamic feature retrieval. One may wonder whether this design helps video restoration. To answer this question, we conduct three ablation studies: 1) disabling the cross attention branch; 2) replacing the MI.P branch with a skip connection; 3) adopting the ZeroSFT connector Yu et al. (2024). Table Figure show the comparative results of the ablation experiments. When the cross attention branch is disabled, the MLP connector does not perform well and produces results lacking in detail (see Table 2] and Figure R()). When the MLP branch is simply disabled, the video restoration model fails to converge due to its exclusive selection of DiT-like features from control inputs, resulting in output results that do not match the input content. To ensure model convergence, we replace the MLP with a skip connection. The results in Table d) and Figure f) show that without the MLP feature mapping, the recovered results are well. These experiments demonstrate the necessity of our dual-branch design.
In addition, the results in Table e show that the performance of ZeroSFT connector Yu et al. (2024) is inferior our proposed dual-branch connector. Furthermore, the normalization operation in ZeroSFT architecture often causes residual artifacts of adjacent frames to appear in the output frames (see Figure(]d)) while removing the normalization leads to gradient explosion during train- ing. In contrast, our proposed connector avoids these problems.
5.3 EFFECT OF THE CONCEPT DISTILLATION STRATEGY
To mitigate distribution drift caused by imperfect multimodal alignment in training data, we intro- duce concept distillation training strategy, that leverages the T2V base model generate training data. To verify its effectiveness, we disable the generated training data and train this baseline method only the collected videos. Table f and Figure 9 e) show that the baseline method without the concept distillation strategy fails to achieve high-quality results, showing overly sharp textures due distribution drift. For the similar reason, textures generated by STAR and SeedVR also less

---

Published as a conference paper at ICLR 2026
Table 3: Influence of text-visual alignment in the concept distillation, where strategy enhances text-visual alignment, and better alignment brings better restoration quality and semantic accuracy.
Training Data Text Visual Aligument
Restored Quality (DOVER1)
Restored Semantic Accuracy
(FGA BI (I2)))
(PGA-BI (CAA-IIJP2()
w/o Concept Distillation
3.49
12.99
3.69
(b) w/ Concept Distillation (Ours) (c) w/ Shuffled Text during Distillation
3,97
14.46
3.78
1.77
11.88
3.21
realistic, addition, the baseline method without the concept distillation also suffers from de- cline in temporal coherence (see Figure 5tt). This verifies that our concept distillation method can facilitate video restoration in terms of both perceptual quality and temporal consistency.
Furthermore, to verify whether using more accurate VLM captioner could resolve the distribution drift problem, we conduct an ablation study: using the more advanced Qwen2.5-VL as VLM cap- tioner for training data annotation. As evidenced by Table 2 g) and Figure fi), the more accurate Qwen2.5- VL also fails to completely eliminate the modality gap in the T2V model's latent space, demonstrating the persistence of distribution drift even with superior captioning models.
In addition, one might wonder if it's possible to generate synthetic videos from scratch (starting with noise). As shown in Figure 7 found that videos generated directly from noise often contain no- ticeable flaws, such as distorted human figures. Synthesizing samples from half-noised real samples significantly alleviates this issue. We further conducted an ablation study, and the results in Ta- ble (h) show that training a model on data generated "from scratch" degrades the video restoration performance, leading to 1.19 point drop on the DOVER metric.
5.4 INFLUENCE OF TEXT- VISUAL. ALIGNMENT CONCEPT DISTILLATION
The alignment between text and visual content is eritical for concept distillation, and our method is specifically designed to enhance it. To quantitatively validate this, we employ
et al. (2024) metric for semantic consistency between text and video. We randomly selected 10, 000 samples from the training videos generated by our concept distillation to evaluate the training data text-visual alignment. As shown in Table (()) (b)), the proposed concept distillation strategy boosts the training data alignment score (with the FGA-BLIP2 score increase of 0.48), which directly translates to higher quality and better semantic accuracy in the final restored videos. is worth noting that the source videos used to generate synthetic samples are randomly sampled from our original collected dataset. This means the addition of synthetic samples does not expand or alter underlying distribution of the training data, indicating that the performance gains stem from better text-visual alignment rather than from easier distribution of synthetic data.
Furthermore, we validate the influence of significant text-visual misalignment, where we simulated misalignment by randomly shuffling text-video pairings. As shown in Table C(), this drastically lowered the training data alignment (FGA-BLIP2 score dropped to 1.77). Using this data during the concept distillation process caused a significant decline in both the quality and semantic accuracy of the final output, indicating that significant text- visual misalignment degrades performance.
CONCLUSIONS AND LIMITATIONS
We have proposed Vivid-VR, a DiT-based generative video restoration method built upon an ad- vanced T2V foundation model. To mitigate distribution drift during fine-tuning, we have introduced a concept distillation training strategy that leverages the pre-trained T2V model to synthesize train- ing samples with embedded textual concepts. Regarding the model architecture for controllable generation, we have proposed two key components: 1) a control feature projector that removes degradation artifacts from latent video features, and 2) dual-branch connector combining MLP and
mechanism for control feature mapping and dynamic retrieval. Both quantitative and qualitative experimental results demonstrate the effectiveness of the proposed method.
The proposed method builds upon the Vog T2V model and inherits its inference com- plexity, which results in lengthy inference times. Future work will explore ways to enhance the algorithm's 's efficiency, such as applying one-step diffusion fine-tuning to achieve comparable video restoration quality in single forward pass.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311. 2023.
Yochai Blau and Tomer Michaeli. The perception-distorioon tradeoff. In CVPR, pp. 2018.
Jose Caballero, Christian Ledig, Andrew Aitken, Alejandro Acosta, Johannes Totz, Zehan Wang, and Wenzhe Shi. Real-time video super-resolution with spatio-temporal networks and motion compensation. In CVPR, pp. 4778-4787, 2017.
Kelvin CK Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Basicvsr: The search for essential components in video super-resolution and beyond. CVPR, pp. 4947-4956, 2021.
Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Basicvsr++: Improving video super-resolution with enhanced propagation and alignment. In CVPR, pp. 5972-5981, 2022a.
Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Investigating tradeoffs in real-world video super-resolution. In CVPR, pp. 0262555697777 5971, 2022b.
Junyang Chen, Jinshan Pan, and Jiangxin Dong, Faithdiff: Unleashing diffusion priors for faithful image super-resolution. In CVPR, pp. 28 28188-28197, 2025a.
Zheng Chen, Zichen Zou, Kewei Zhang, Xiongfei Su, Xin Yuan, Yong Guo, and Yulun Zhang.
Dove: Efficient one-step diffusion model for real-world video super-resolution. arXiv preprint arXiv:2505.16239, 2025b.
Junyuan Deng, Xinyi Wu, Yongxing Yang, Congchao Zhu, Song Wang, and Zhenyao Wu. Acquire and then adapt: Squeezing out text-to-image model for image restoration. In CVPR, pp. 23195- 23206, 2025.
Jinjin Gu, Haoming Cai, Chao Dong, Jimmy Ren, Radu Timofte, Yuan Gong, Shanshan Lao, Shuwei Shi. Jiahao Wang. Sidi Yang, et al. Ntire 2022 challenge on perceptual image quality assessment. In CVPRW, pp. 951-9671 2022.
Shuhao Han, Haotian Fan, Jiachen Fu, Liang Li, Tao Li, Junhui Cui, Yunqiu Wang, Yang Tai, Jingwei Sun, Chunle Guo, et al. Evalmuse-40k: A reliable and fine-graines benchmark with comprehensive human annotations for text-to-image generation model evaluation. arXiv preprint arXiv:2412.18150, 2024.
Gu Jinjin, Cai Haoming, Chen Haoyu, Ye Xiaoxing, Jimmy S Ren, and Dong Chao. Pipal: large- scale image quality assessment dataset for perceptual image restoration. In ECCV, pp. 633-651, 2020.
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In ICCV, pp. 51 5148-5157, 2021.
Jingyun Liang, Yuchen Fan, Xiaoyu Xiang, Rakesh Ranjan, Eddy Ilg, Simon Green, Jiezhang Cao, Kai Zhang, Radu Timofte, and Luc V Gool. Recurrent video restoration transformer with guided deformable attention. In NeurlPS, pp. 378-393,7 2022.
Jingyun Liang, Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, and Luc Van Gool. Vrt: A video restoration transformer. IEEE Trans. Image Process., 33:2171-2182, 2024.
Ilya Loshchilov and Frank Hutter. Decoupled 17.1:05000000 2017.
weight decay regularization.
arXiv preprint
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint a.0009221..01095, 2022.
11

---

Published as a conference paper at ICLR 2026
Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Making "completely blind" image quality analyzer. IEEE Sign. Process. Letters, 20(3):209 212, 2012.
Seungjun Nah, Sungyong Baik, Seokil Hong, Gyeongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee. Ntire 2019 challenge video deblurring and super-resolution: Dataset and study. In CVPRW, pp. 0-0, 2019.
Jinshan Pan, Haoran Bai, and Jinhui Tang. Cascaded deep video deblurring using temporal sharpness prior. In CVPR, pp. 3043-3051, 2020.
Jinshan Pan, Haoran Bai, Jiangxin Dong, Jiawei Zhang, and Jinhui Tang. Deep blind video super- resolution. In ICCV, pp. 4014 1-4200, 2021.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In CVPR, pp. 4195-4205, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv::307.01195, 2023.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Mach. Learn. Res., 21(140))1-67, 2020.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer: High- resolution image synthesis with latent diffusion models. In CVPR, pp. 10684-10695, 2022.
Alexandros Stergiou and Ronald Poppe. Adapool: Exponential adaptive pooling for information- aaaaaaaaaaiiiia retaining downsampling. IEEE Trans. Image Process., 32:251-266, 2022.
Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, and Oliver Wang. Deep video deblurring for hand-held cameras. In CVPR, pp. 1279-1288, 2017.
Xin Tao, Hongyun Gao, Renjie Liao, Jue Wang, and Jiaya Jia. Detail-revealing deep video super- resolution. In ICCV, pp. 4472 2017.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint 2rX3v:2503.20314 2025.
Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In AAAI, pp. 2555-2563, 2023.
Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin CK Chan, and Chen Change Loy. Exploiting diffusion prior for real-world image super-resolution. Int. J. Comput. Vis., 132((2)):5929-54) 2024.
Jianyi Wang, Shanchuan Lin, Zhijie Lin, Yuxi Ren, Meng Wei, Zongsheng Yue, Shangehen Zhou, Hao Chen, Yang Zhao, Ceyuan Yang, et al. Seedvr2: One-step video restoration via diffusion adversarial post-training arXiv preprint arXiv:2506.05301, 2025a.
Jianyi Wang, Zhijie Lin, Meng Wei, Yang Zhao, Ceyuan Yang, Chen Change Loy, and Lu Jiang. Seedvr: Seeding infinity in diffusion transformer towards generic video restoration. In CVPR, pp. 2171-2172, 2025b.
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In ECCVW, pp. 0-0, 2018.
Xintao Wang, Kelvin CK Chan, Ke Yu, Chao Dong, and Chen Change Loy. Edvr: Video restoration with enhanced deformable convolutional networks. In CVPRW, pp. 0-8, 2019.
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In CVPR, pp. 1905-1914, 2021.
12

---

Published as a conference paper at ICLR 2026
Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from 2023.
aesthetic and technical perspectives. In ICCV, pp.
Rui Xie, Yinhong Liu, Penghao Zhou, Chen Zhao, Jun Zhou, Kai Zhang, Zhenyu Zhang, Jian Yang. Zhenheng Yang, and Ying Tai. Star: Spatial-temporal augmentation with text-to-video models for real-world video super-resolution. arXiv preprint arXiv:2501.02976, 2025.
Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with task-oriented flow. Int. J. Comput. Vis., 127(8):1106-1122 2019.
Xi Yang, Chenhang He, Jianqi Ma, and Lei Zhang. Motion-guides latent diffusion for temporally consistent real-world video super-resolution. In ECCV, pp. 242.242. Springer, 2024a.
Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvideox: Texcotovideo diffusion models with an expert transformer. arXiv preprint hil::2408.06772. 2024b.
Peng Yi, Zhongyuan Wang, Kui Jiang, Junjun Jiang, and Jiayi Ma. Progressive fusion video super- resolution network via exploiting non-local spatio-temporal correlations. In ICCV, pp. 3106- 3115, 2019.
Fanghua Yu, Jinjin Gu, Zheyuan Li, Jinfan Hu, Xiangtao Kong, Xintao Wang, Jingwen He, Yu Qiao, and Chao Dong. Scaling up to excellence: Practicing model scaling for photo-realistic image restoration in the wild. In CVPR, pp. 25669-25680, 2024.
Kai Zhang, Jingyun Liang, Luc Van Gool, and Radu Timofte. Designing practical degradation model for deep blind image super-resolution. In CVPR, pp. 4791-4800, 2021.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. ICCV, pp. 3836-3847, 2023a.
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In CVPR, pp. 586-5955 2018.
Zicheng Zhang. Wei Wu, Wei Sun, Danyang Tu, Wei Lu, Xiongkuo Min, Ying Chen, and Guangtao Zhai. Md-vqa: Multi-dimensional quality assessment for uge live videos. In CVPR, pp. 1746- 1755, 2023b.
Shangchen Zhou, Peiqing Yang. Jianyi Wang, Yihang Luo, and Chen Change Loy. Upscale-a-video: Temporal-consisee diffusion model for real- real-world video super-resolution. In CVPR, pp. 2535- 2545, 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1
USE LLMS
In this paper, we used LLMs to assist with gramiar and writing polish. All LLLM-generate sugges- tions were carefully reviewed and edited. The authors take full responsibility for the final content.
A.2 ADDITIONAL ANALYSIS OF THE PROPOSED METHOD
A.2.1 TRADE-OFF BETWEEN FIDELITY AND QUALITY
As Yu et al. (2024) points out, powerful generative prior is a double-edged sword, as excessive generative capacity may in turn affect the fidelity of the restored video. To address this, we introduce Restoration Resioration-Guided Sampling into Vivid- AR's inference sampling process to balance the quality and fidelity:
1. " x(at) 2est x/ + (())
(3)
where zest is the denoised latent time step 1, and I the original input latent; denotes the total number of denoising steps; is the guidance coefficient; gest is the output latent after the eestorationguuider sampling. Figure I0 demonstrates this trade-off: higher guidance coefficient yield more realistic results, while lower r preserve greater fidelity to the original input content.
A.2.2 EFFECT OF THE NUMBER OF GENERATED TRAINING VIDEOS
As mentioned in the main paper, the proposed method employs the concept distillation strategy to generate 100K videos for training. A natural question arises: does the number of generated training videos impact restoration performance? To investigate this, we conducted the ablation studies here. Rable(]demonstratee that increasing the number of generated training videos from 0 to 100A yields significant performance gains, while expanding from 100K 150K shows diminishing returns. Considering the cost of generating training videos, we therefore adopt 100K generated videos as our standard configuration. Furthermore, we verified that relying solely on generated training data (without source videos) leads to suboptimal results. This occurs because the T2V base model's outputs contain inherent imperfections. Training exclusively on such data ultimately compromises model performance.
Table 4: Effect of the number of training videos generated by the proposed concept distillation.
Concept Distillation Training Stratogy
Methods
NIQB 1
MUSIQ
CLIP IQA
DOVER 1
# souree training videos
generated training videos
500K
5.364
57.36
0.363
12.99
(b)
S00K
50K
4.562
63.00
0.408
13.46
(c)
500K
100K
4.361
67.61
0.450
14.46
500K
150K
4.292
67.19
0.443
14.51
A.2.3 EFFECT OF THE NUMBER OF CONTROLNET BLOCKS
To reduce the parameter count, employ N/7 DiT blocks in ControlNet. In Deng et al. 2025 only one block is used, and all connectors share the same control feature. To further investigate whether N/7 DiT blocks are indeed necessary, set the number of blocks to 1 and retrain using the same settings. The results Table show that using only one block does not perform well.
Table 5: Effect of the number of controlNet blocks.
NIQE J.
MUSIQ
CLIP-IQA
DOVER
DrT block ComtrolNet
4.855
66.85
0.442
14.17
N/7 DiT blocks in ControlNet (Vivid-VR)
4.361
67.61
0.450
14.46
14

---

Published as a conference paper at ICLR 2026
A.2.4 EXTENDING TO OTHER PRE-TRAN -REAIIDD T2V MODELS
We selected Cog Vog 5BB5BB the T2V base model because it a state-of-the-art, publicly avail- able T2V model at the time of our research, demonstrating exceptional generation quality and tem- poral stability. To assess the generalizability of our approach, we replaced Cog VideoX1..-5B with more recent open-soune source model, Wan2.2-T12V-5BB T12V-B Wan et al. (2025). By applying the same training strategy, we found that our proposed concept distillation and ControlNet improvements transferred effectively to this new model, also achieving SOTA results (see Table 6 We also experimented with a smaller model, Wan2.1-T2V-1.33 Wan al. (025), and observed a corresponding drop in restoration performance. This suggests that leveraging larger and more powerful T2V models could yield superior outcomes. Extending this to even larger models (e.g.. Wan2.1-T2V-14B) remains direction for future work due to significant computational demands.
Table Extending to other pre-trained T2V models.
Pretrained Model
Parameters
NIQEL
MUSIQ
CLIP-IQAT
DOVERT
CigVideoXI
4.361
2
5B
4,101
67.88
0.461
14.01
Wan2.1-12V-L.3B
1.38
5.078
63.21
0.339
12.98
A.2.5 MORE QUALITATIVE COMPARISON
In the main paper, we have demonstrated that the proposed Vivid-VR achieves state-of-the-ar art performance. Due to the submission file size limit, we further provide more visual comparisons with state-of-the-ard methods and video examples in the Supplementary Material, where Vivid-V VR demonstrates superior structural clarity, texture richness, visual vividness, and temporal consistency.
15

---

Published as a conference paper at ICLR 2026
Fidelity
Quality
Inpur
-
r-
Figure 10: Trade-off between fidelity and quality. Higher guidance coefficient in the Restoration- Guided Sampling yield more realistic results, while lower 7 preserve greater fidelity to the original input content. (Zoom-in for best view)
16


Reviews:
[{"review_id": "lmDNtunu1S", "reviewer": "Reviewer_7MWW", "summary": "The paper presents a DiT-based video restoration model trained with a novel concept-distillation strategy that uses a pre-trained T2V generator to produce aligned text–video pairs, which eliminates distribution drift. It also proposes a lightweight ControlNet projector and dual-branch connector that further suppress artifacts and enable dynamic control.", "questions": "I don’t have any further questions at the moment; I’m waiting to discuss with the other reviewers.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It proposes concept distillation with a pre-trained T2V model to generate aligned text–video pairs.\n2. It markedly outperforms prior methods.\n3. A high-quality dataset is created that should significantly benefit the video-generation community.", "weaknesses": "Lack of video supplementary results: As a video-oriented work, without video results as supplementary material, it is difficult for the public to intuitively evaluate the model's performance, especially the quality of temporal consistency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "npXZOUIlOz", "reviewer": "Reviewer_vgeK", "summary": "Vivid-VR is a generative video restoration method. To address the issue of distribution drift during fine-tuning, the authors propose a concept distillation strategy that uses the pre-trained T2V model to synthesize aligned text-video pairs, thereby preserving texture realism and temporal coherence. The control mechanism is further enhanced with a novel feature projector to filter degradation artifacts and a dual-branch connector for dynamic control feature retrieval. Extensive experiments demonstrate that Vivid-VR achieves superior performance in texture realism and temporal consistency compared to existing methods on various benchmarks.", "questions": "1. For the concept distillation process, what will happen if the generated textual descriptions have a major difference from the visual content? Then the pretrained T2V model may generate videos that are quite different from the original low HQ videos. Will it influence the semantic accuracy of the final high HQ videos?\n\n2. Why CogVideoX1.5-5B is selected as the pretrained T2V model? What will the model perform when selecting other alternatives? Besides, what will the video restoration perform when adopting pretrained T2V models in different sizes?\n\n3. What is the rationale of selecting DiT as the main architecture of the video restoration model, considering there are many alternatives, such as MMDiT?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Leveraging the capabilities of pretrained T2V models to enhance the video restoration performance is an interesting approach. Using the textual description as a connector, the authors find an effective way to transfer the T2V model’s pretrained knowledge to the video restoration model, which I believe benefits the community.\n\n2. The proposed method outperforms several previous advances by a large margin in a wide range of benchmarks. The experiments are comprehensive, and the qualitative demos are also impressive.\n\n3. The ablation studies are also exhaustive. The effectiveness of each branch is clearly verified.", "weaknesses": "1. Though the idea of transferring the T2V model's capability to downstream tasks is interesting, the proposed method seems to be trivial and similar to other methods. Using a pretrained model to corrupt and reconstruct the visual content is a common way, especially in image enhancement and restoration tasks, and it is also widely adopted to add the textual description during the reconstruction. It is more likely to be a transition from the image restoration task to the video restoration task, which makes the technical contribution less competitive.\n\n2. How the concept distillation really works remains ambiguous. Why putting the textual description into the DiT block can ''transfer the T2V model’s conceptual knowledge to the video restoration model''? What concept is transferred to the video restoration model? What will happen if we use a video captioner to generate captions and add a text encoder to the DiT block instead of leveraging the pretrained T2V model?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UapNsNWjmQ", "reviewer": "Reviewer_BT4i", "summary": "This paper proposes a video restoration method. The core contributions include training with mixed real/synthesized data (termed “concept distillation”) and two architectural modifications.", "questions": "See weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The method demonstrates strong empirical performance according to the reported evaluations.", "weaknesses": "Technical novelty and depth are limited. The architectural modifications, while effective, are relatively standard and offer limited insight; it is unlikely that these design choices will significantly influence future research. Showing that synthesized videos can benefit video restoration is a useful observation, but this point alone, at the current level of investigation, does not meet the novelty threshold to me.\n\nThe proposed components are not sufficiently analyzed. For example, although the projector is effective, there is no evidence that its efficacy is due to “filtering degradation artifacts from input video latents” (line 49). Given the presented experiments, the safe conclusion is only that having a trainable projection layer at that location is beneficial; any stronger claims require further evidence. Regarding concept distillation, can we directly generate synthesized samples from scratch instead of starting from half-noised real samples? Moreover, how do we know that the gains from concept distillation stem from better image–text alignment rather than other factors, such as the intrinsically easier distribution of synthesized data?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)