Dimension-Reduction Attack! Video Generative Models are Experts on Controllable Image Synthesis
Authors: Hengyuan Cao
Yutong Feng
Biao Gong
Yijing Tian
Yunhong Lu
Chuang Liu
Bin Wang
Venue: NeurIPS 2025 poster ()
Keywords: video generative models
controllable image generation
image customization
conditional image generation
Primary area: applications
TL;DR: We explore the application of video generation models in controllable image generation tasks.

Abstract: Video generative models can be regarded as world simulators due to their ability to capture dynamic, continuous changes inherent in real-world environments. These models integrate high-dimensional information across visual, temporal, spatial, and causal dimensions, enabling predictions of subjects in various status. A natural and valuable research direction is to explore whether a fully trained video generative model in high-dimensional space can effectively support lower-dimensional tasks such as controllable image generation. In this work, we propose a paradigm for video-to-image knowledge compression and task adaptation, termed \textit{Dimension-Reduction Attack} (\texttt{DRA-Ctrl}), which utilizes the strengths of video models, including long-range context modeling and flatten full-attention, to perform various generation tasks. Specially, to address the challenging gap between continuous video frames and discrete image generation, we introduce a mixup-based transition strategy that ensures smooth adaptation. Moreover, we redesign the attention structure with a tailored masking mechanism to better align text prompts with image-level control. Experiments across diverse image generation tasks, such as subject-driven and spatially conditioned generation, show that repurposed video models outperform those trained directly on images. These results highlight the untapped potential of large-scale video generators for broader visual applications. \texttt{DRA-Ctrl} provides new insights into reusing resource-intensive video models and lays foundation for future unified generative models across visual modalities. The project page is \url{https://dra-ctrl-2025.github.io/DRA-Ctrl/}.

Dimension-Reducctonn Attack! Video Generative Models are Experts on Controllable Image Synthesis
Hengyuan Cao Zhejiang University
Yutong Fengt Kunbyte AI
Biao Gong Ant Group
Yijing Tian Hangzhou Normal University
Yunhong Lu Zhejiang University
Chuang Liu Hangzhou Normal University
Bin Wang Kunbyte Al
{caohy, yunhonglu)@zju. edu. en tianyijing200201633 com liuchuangOhznu, edu, cn (fengyutong. fyt, biao. gong, binwang39310gmail com
Video Generation
Iransition
Controllable
Dimension-Reduction Attack!
Image Generation
Subject- Generation
Style Transferring
Spatiol-aligned Control Generation
Camp-io--mag
Caloriatkes
Dekurriu
Dest--todrrgg
Denla Prodiction
Cuipuiniru
Saper Rosolatar
Figure 1:
This paper leverages high-level prior of video generative models unify controllable image generation in low-level. Bottom results show various types of task supported by DRA-Ctr1.
Abstract
Video generative models can be regarded as world simulators due to their abil- ity to capture dynamic, continuous changes inherent real-world environments. These models integrate high-dimensional information across visual, temporal, spatial, and causal dimensions, enabling predictions of subjects in various status. A natural and valuable research direction is to explore whether fully trained video generative model sigh-dimensional space can effectively support lower- dimensional tasks such controllable image generation. In this work, we propose a paradigm for indo-tt--image knowledge compression and task adaptation, termed Dimension- Reduuctio Attack (DRA-Ctr1), which utilizes the strengths of video models, including long-range context modeling and flatten atll-ateentonn, to per- form various generation tasks. Specially, to address the challenging gap between
I Project leader.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

continuous video frames and discrete image generation, we introduce a mixup- based transition strategy that ensures smooth adaptation. Moreover, we redesign the attention structure with a tailored masking mechanism to better align text prompts with image-level control. Experiments across diverse image generation tasks, such as subject-driven and spatially conditioned generation, show that repurposed video models outperform those trained directly on images. These results highlight the untapped potential of large-scale video generators for broader visual applications. DRA-Ctrl provides new insights into reusing resource-intensive video models and lays foundation for future unified generative models across visual modalities. The project page is https: //dra- ctrl- 20225.02550000 github. io CtrrR
1
Introduction
have significantly improved the quality of image synthesis from natural language prompts. To enhance controllability. researchers
Recent advances in text-to-image (T2I) generative models 38
34
9
23
have introduced auxiliary conditions into the context of generation [54] 24
49
32
60
10
18),
such as subject reference images, edge maps and depth cues. This has given rise to the paradigm of controllable image generation, where both textual and visual conditions collaboratively guide the synthesis process. While early methods relied on additional image adapters or cross-attention mechanisms [54
6
15
recent approaches leverage full-attention architectures 661 22 50
[3] 7 [25 12
47 that treat all input tokens as unified sequence. However, these models are all built upon image generative models, thus remain limited by the static nature of image data, which lacks the continuous temporal and causal structures transformation present in the real world.
29
Video generative models 2 53 20 45). in contrast, are trained to predict sequences of frames with rich spatiotemporal dependencies. The prior knowledge learned by these models incorporates long- range context, consistent object transitions, non-rigid transformation and high-level scene dynamics. These capabilities align closely with the goals of controllable image generation. This observation inspires a new direction, i.e., repurposing pretrained video models to support image-level tasks by transferring their high-ding nigh-dimensional knowledge into lower-dimensional setting. This work dives into this idea and presents a framework termed DRA- Ctrl that efficiently adapts video generators for diverse controllable image generation scenarios.
However, directly adapting video generative models to controllable image generation presents non- trivial challenges. A naive baseline would be to gather the condition image and target image into the frame sequence of video generators. The key hindrance confronted here is that the video data inherently consists of temporally continuous frames with smooth transitions, while the condition- target image pairs represent a discrete, abrupt change between two states. In detail, we investigate to adapt two variants of video generative models treating the image pairs as two-framed video. For image-to-video (I2V) model consuming the condition image as the first frame. it suffers to over-constrain the output to mimic the condition image. While for text-to-video (T2V) model, it is inevitable to inject the condition image as non-noisy frame tokens into the sequence. Thus the model takes much efforts to readapt the new paradigm, and tends to forget its pre-training knowledge with suboptimal performance. These baseline solutions expose the fundamental discrepancy between the continuous dynamics learned by video models and the discrete transition required by controllable image generation. Therefore, is essential for DRA-Ctrl to conduct stable transferring when repurposing the video models without forgetting their high-dimensional capabilities.
To address these challenges in DRA-Ctr1, we propose a mixup-based transition strategy, inspired by the mixup (57) principle in representation learning, serving as a bridge connecting the diverse intermediate gaps in videos and images. The core idea to treat the condition and target images as boundary frames of a synthetic shot transition sequence, with intermediate frames generated using a temporal position-aware mixup. Each intermediate frame is weighted by its relative position between the two endpoints, enabling smooth interpolation while preserving key visual characteristics. We implement the mixup transition with the 12V model. When integrating with these augmented frames, the constraint from condition to target images is significantly relaxed, making it easier to adapt to discrete image generation. Despite this, real video transitions generally require dozens of intermediate frames, resulting in dramatically increased computation cost. To mitigate this, we introduce Frame- Skip Position Embedding, a positional encoding scheme that expands temporal intervals the latent space, allowing large image transformations with only a few frames. Additionally, to distinguish
2

---

complex combination of subjects and environments in multiple images, we adapt the condition and target prompts into the full-attention mechanism together with a masking strategy.
We evaluate DRA-Ctrl on a wide range of controllable image generation tasks, including subject- driven image synthesis, spatially aligned condition generation (e.g., canny-to-image translation, colorization, deblurring, depth-based generation and depth prediction), masking image generation (inpainting and outpainting) and style transferring. Our experiments demonstrate that video generative models can be effectively re-purposed for these tasks, consistently outperforming methods built upon image generative models. This surprising effectiveness highlights a compelling "Dimension- Reduction Attack", where high-dimensional video priors offer enhanced control when adapted to -wwer-dimensional image tasks, encouraging more efforts to further investigate the extending capability of video generative models.
Related Works
Subject-driven Image Generation. Subject-driven image generation with diffusion models typically follows two paradigms: tuning-based and tuning-frree methods. Tuning-based methods [10 21
achieve strong identity consistency but require per-subject fine-tuning, limiting scalability and introducing non-trival computational overhead. Tuning-free methods instead enhance generalization
through training on large-scale datasets, climinating inference-time time tuning. Early works [54] 24 49]
32 60 18
extract subject information from reference images using an image encoder. and inject these features into the generation process via cross- attesttntnntion mechanisms. Then, Hu et al. [15] propose using a ReferenceNet which is architecturally identical to the denoising UNet the image feature extractor, providing detailed and accurate control information for controllable generation. Later adyancements in tuning-free methods leverage the model's inherent in-context learning capabilities [77, , treating the model itself as an image feature extractor to provide subject-specific information for generation. Zeng et al. (56) proposes to model the joint distribution of multiple text-image pairs sharing the same subject, investigating in-context learning within UNet-based diffusion models for subject-drive image generation. With the introduction of DiT architectures [33] recent works 61 22 50)
51 7 25
have explored the full-attention mechanism, where reference images and generated images jointly participate in self-attention, to facilitate subject feature extraction and enable in-context learning for subject-driven generation. We propose leveraging video diffusion models' inherent frame-level full-attention mechanism for subject-driven generation.
Spatially-aligned Image Generation. cantially-aligned control signals for fine-grained image generation have emerged critical research direction. Early conditional Generative Adversarial Networks (GANs) 19 63) and transformers [4] achieve image-to-image translation by learning the mapping from conditional images to target images. Recent diffusion models enable tighter integration of such controls. SDEdit (30 guides generation process by first adding noise to stroke paintings and then denoising them. In contrast, T2I-Adapter (31] trains an adapter network to enable more diverse and precise control signals. ControlNet (58) reuses the encoding layers of pre-trained diffusion models as backbone for learning control signals. UniControl (36) further advances this direction by integrating multiple tasks within a unified framework via a task-aware HyperNet, demonstrating zero- shot capabilities on unseen tasks and combined tasks. Subsequent works 7 51 43 25
12
29 47
have unified subject-driven and ppatially-alignee image generation within one framework that maps control images target outputs, which DRA-Ct:t also follows.
Image Generation with Video Models. While existing works employ video generative models for image editing (requiring pixel-aligned partial modifications) that are methodologically naive, our framework targets controllable image generation that enables comprehensive transformations including background replacement, subject pose/state alteration, and holistic content regenera- tion. FramePainter [59] injects interactive editing signals extracted by the control encoder into the generation process via cross-attention mechanisms and synthesizes a two-frame video where the first frame reconstruct the condition image and the second one produces the edited output. Object- Mover (55) addresses the object relocation task by fine-tuning a video generative model through frame-wise concatenation of condition images with various control signals. Rotstein et al. [39 proposes a direct 12V approach for image editing, where condition images and Vision Language Model prrocesedd prompts are jointly fed into the model, with edited results obtained through a specialized frame selection strategy. While Lin et al. [27] and Chen et al. [7] similarly employ video models for controllable image generation editing tasks, primarily motivated by their ability
3

---

Cundition Image
Mixup-based Transition
Tarnet Emsane
Trainable
Prozen
G Condition Image
B 2el
Tirget
Condition
Ti Target Image
Prompp
Prompe
Tx: Target Image Prompi
Gy Condition Image Prompe
3D VAE Encoder
23D VAE
Eneoder
MLLM
.day
Maxked
Unmakeed
Qurry'
3,
Atterition
Video Diffusion Transformer Blocks
Mask
SSNSS
SANS
Lous
Reveighting
drrbuitly to ongst
Figure 2: The training framework of DRA-Ctr1, We propose mixup-based transition strategy to construction shot transition videos to adapt the video model for abrupt image changes, with FSPE strategically reducing transitional frames. The loss function adaptively reweighted according the proportion of target image in the token sequence. Besides, to align text prompts with image-level control, we design an attention masking mechanism.
perform full attention in the temporal dimension, work further introduces strategies like mixup to better exploit the rich priors inherent in video models.
Method
Given that video generative models' inherent temporal full-attention and rich dynamics priors, we argue they can be efficiently re-purposed for controllable image generation tasks. To successfully video generative models for handling abrupt and discontinuous image transitions, we propose multiple strategies, shown in Figure Specifically, in Section [.1] we introduce our foundational model, Video-122,, detailing its architecture and objective function; in Section 3.2 we present our mixup-based shot transition strategy that construct shot transition video with condition and target images; in Section 3.3 we propose new position embed- ding method that reduces the required number of transition frames; in Section 3.4 we describe an attention masking strategy to properly guide information interaction.
adapt
3.1 Preliminaries
Our method builds upon Video-122 (00). which consists of three key components: (1) causal 3DVAE that compresses videos in both spatial and temporal dimensions, (2) a text encoder built upon Multimodal Large Language Model (MLLM), which processes not only textual information but also partial conditioning image features, (3) transformer employing unified full-attention mechanism to jointly process image and text signals.
The 3DVAE maps video sequence E (1T++1)xxx1HXx10 into a compact latent representation E I2 (g(T+))x6x2Hxx) x2W which is subsequently patchified and unfolded to yield visual tokens Zvisual of length (T + 1) H x W. Meanwhile, the textual tokens Ztextaal are obtained by processing target prompt Tp and condition image C through the MLLM. Then concatenated sequence Zuisual Zteztual is fed into the transformer, where unified full-attention mechanism is applied to effectively fuse information across both modalities. To enhance the model's ability to capture positional relationships, 3D Rotary Position Embedding (RoPE) 41 introduced in each transformer block. To achieve I2V generation, Hunyuan Video-12V employs token replacement technique, where the visual tokens of the first frame are replaced with the condition image tokens. In addition, CLIP- (37] text features and the diffusion timestep & are adopted global guidance signals and incorporated into the transformer. The objective function follows flow matching (28):
L = 1l t, Tp) (E 11-
where denotes Gaussian noise, y: = (1 t)y + Le, and and stand for the neural network and its corresponding parameters respectively.

---

consetency
prampe
prompt adherrnnc
10MA
Decader
10x/ Decide
T2V Model
T2V based Subject-driven Model
12V Model
12V-based Subject-driven Model
aottinttaae Seee
3trAE becoder
becoder
32WAE becode
MJUM
sat cuniing
perpele
Figure 3: The inference process of T2V/12V models and their finetuned subject-driven image generation models. By treating the condition and target images directly two-frame video and fine-tuning T2V/12V models accordingly, the corresponding T2V/12V baselines can be obtained.
3.2 Mixep- Shot Transition
The simplest approach for controllable image generation using video generative models is to treat condition and target images two-frame video. During training, the condition image remains noiseless and excluded from loss calculation, while the target image is noise-corrupted and included in loss calculation. During inference, the condition image maintains noiseless provide complete control signals. Empirical tests with Hunyuan Video-T2V/12V on subject- generation task, as shown in Figure and Table demonstrate that neither model meets the requirements for subject- driven generation: the T2V model lacks subject consistency, while the I2V model over-preserves similarity to the condition image and exhibits poor prompt adherence. The observed results are expected because T2V model does not enforce consistency as strictly as I2V model, while the I2V model's s strong inter-frame consistency preservation limits prompts' controllability.
To address these limitations, we draw inspiration from cinematic shot transitions by treating condition and target images as storyboard endpoints. Then, we fine-tune the I2V model generate transition frames and target image according to condition image. This approach maintains consistency and enhances controllability through smooth visual transitions. Specifically, we observe that certain 12V models [20] [55 can naturally produce fade-in-fade-- fade-out transitions similar to those in PowerPoint presentations. Therefore, we propose constructing transition frames Fa, with condition image Fa=o
and target image Fa=1 by interpolation, Fa = ((1 - 3) P 3F 1/7 where a E [0, 1] and y set to 2.2 ensure smooth inter-fram transitions. During training, we keep the condition image Fo=o noise-free and exclude it from loss calculation, while applying noise and including Fo-co5i 1 in the loss calculation. The contribution weight each latent frame in the loss is determined by its proportional content from the target image, yielding the final loss function:
1//0 B a² (3 2a),
1
L
w(k) C1, Cp, (ee)))((e)llll K k=0
(1 t) Encode Fo k)) te,
if k = 0, 1,..., K 1,,,
y
(2)
(1 t) Encode Pa=l, -1) Le,
if = K,
4k 2
4h +1 2
2
w(k) = Σ
4K +1
4K +
where Encode(:, (., k) is the encoder of the 3DVAF, which maps 47 + 1 frames in pixel space to T + 1 latent representations and returns the (k:22)ttt latent representation, Cy is the prompt of the condition image. We encode the target image separately to ensure the independence of the corresponding latent representation during inference. During inference, the condition image's latent representation is concatenated with K + Gaussian noise in latent space and perform progressive denoising while keeping the condition image's 's latent representation unchanged throughout the process, ultimately decoding the last frame of the denoised latent representations through the decoder Decode(-) of the 3DVAE to obtain the final generated result Past Decode
5

---

3.3 Frame Skip Position Embedding
Achieving smooth shot transition often requires dozens or even hundreds of frames. Since we only aim to obtain the final frame, inserting so many transition frames between condition and target images would severely degrade the efficiency both training and inference. In Hunyuan Video, the model incorporates both temporal and spatial information (n, i.j) into tokens through RoPE, where n = 0,
represents the latent frame index of the tokens in temporal dimension and i = 0, 1, H - 1 and 0, 1,... W denote the height and width coordinates of the tokens in spatial dimensions, respectively. To achieve long-term effects with minimal latent frames, we enhance RoPE by incorporating skip intervals along the temporal dimension, called Frame Skip Position Embedding (FSPE), ("') = (n x 6, j), where represents the skip interval. This approach constructs long-term sparse representation of latent frames using minimal latent frames, significantly reducing computational overhead.
3.4
Attention Masking Strategy
Due to the absence of textual descriptions for shot transition videos, we jointly input the prompts from both condition and target image into the network subject-driven generation task. This approach enables the model to acquire all textual information corresponding to the shot-transition videos. However, in this way, there four distinct token sequences during full-attention computation, iee, condition image tokens C₁ generated frame tokens 77, target image prompt tokens Tr, and condition image prompt tokens Cr. To prevent unintended information blending across these token sequences, we design an attention masking strategy as illustrated in Figure 2 Specifically, our designed attention mask assigns extremely negative value to similarity scores between incompatible token sequences (e.g., condition image tokens and target image prompt tokens) to effectively block unintended interactions while maintaining necessary information flows,
Mp. {- -c,, if (p. E (C, x TT) (T, x CPP x CI) CP) x T)). 0, otherwise.
Furthermore, during inference, we enhance the differentiation between Tp and Cp influences by augmenting the attention mask region corresponding to (T, x Tp) with an offset of w times its absolute mean value, where we set w = 0.6.
4 Experiments
4.1 Experimental Setup
Tasks. We extensively evaluate the effectiveness of method across multiple tasks, including spatially-aligned generation, subject-driven generation and style tranferring. For spatially-aligned image generation, we specifically design five distinct sub-tasks: canny-to-image generation, depth-to- image generation, image colorization, image deblurring and image in/out-painting.
adopt subset of the Text-to- Text-to-Image-- 2M dataset (64) for training, consisting of around 160K samples, where the condition images are extracted from the corresponding ground-truv -trth images. The models trained with batch size of 8 and gradient accumulation over steps, resulting in an effective batch size of 16. We employ the A optimizer and conduct training on 2 NVIDIA H800 GPUs (80GB memory each). For subject-driven image generation, we utilize the high-quality subset the Subjects200K dataset (33, comprising approximately 110K image pairs for training. This model trained using NVIDIA H800 GPUs.
Training. For spatially-aligned image generation,
Benchmarks. For spatially-aligned generation, we employ the COC02017 validation dataset [26 comprising 5,000 images resized to 512x512 resolution the test set, where the corresponding prompts are randomly selected from multiple candidate captions associated with each image. For subject-driven generation, we evaluate method on DreamBench (40) by generating images for 25 text prompts per subject, using one reference image for each of the 30 subjects in the benchmark.
Metrics. For spatially-aligned generation, we evaluate methods in terms of controllability and generation quality. Controllability assessed by the similarity of the extracted condition images from generated and ground-truth image. Specifically, we employ the Fl score for tony-tt-imaae

---

Table Quantitative results on COC02017 validation set.
The best results are bold.
Controllability F1
General
Quality
Condition
Model
Method
FID.
SSIM
ControlNet [58]
0.34
18.74
0.35
SDI.5
38
T21-Adapter []]
0.22
20.06
0.35
Uni-ControlNet 62
0.20
17.38
Canny
ControlNet
0.21
98.68
0.25
FLUX.1
OminiControl
0.38
20.63
0.40
EasyControl 61
0.31
16.07
Hunyuan HunyuanVidco-12 Video-12V 20
DRA-Ctr1
0.42
19.44
0.38
ControlNet
923
23.02
0.34
SD1.5
T21-Adapter
1560
24.72
0.27
Uni-ControlNet
1685
21.79
Depth
ControlNet
2958
62.20
0.26
FLUX.1
OminiControl
903
27.26
0.39
EasyControl
1092
20.39
MinnyunnVideo-PP
DRA-Ctr1
76
20.83
0.33
ControlNet
572
30.38
0.74
FLUX.1
Deblur
OminiControl
132
11.49
0.87
HlunyuanVideo-12
DRA-Ctr1
11
9.08
0.64
ControlNet
351
16.27
0.64
FLUX.1
Colorization
OminiControl
24
10.23
0.73
Hunyuan Video-12V
DRA-Ctr1
30
8.39
0.85
SD1.5
ControlNet
7588
13.14
0.40
Mask
FLUX.1
OminiControl
6248
15.66
0.48
Hunyuan Vidco -2V
DRA-Ctr1
16
9.87
0.59
task and use Mean Squared Error (MSE) for other tasks. Generation quality is quantified using Fréchet Inception Distance (FID) [3] and Structural Similarity Index Measure (SSIM) [48] between generated and ground- images. For subject-driven generation, we evaluate methods by standard automatic metrics and Vison- Vison-Language (VL) Model. We measure subject consistency by DINO and CLIP-I scores, which compute the cosine similarity between the condition image and the generated image in DINO (3) and CLIP (37) embedding spaces. Prompt adherence is quantified by the cosine similarity between the CLIP embeddings of the prompt and the generated image, referred to as CLIP-T score. However, these metrics have inherent limitations: DINO and CLIP-I measure global image similarity rather than directly evaluating subject consistency, while CLIP-T struggles with fine- grained semantic alignment and other challenges [37]. To address this, propose VI. score, novel metric based on QWen2.5-VI [I]. which evaluates generated images for subject consistency and prompt adherence via tailored prompts. The VL model outputs discrete scores (0-4) per dimension. with the final score computed as their average.
4.2 Spatiall Image Generation Results
To validate DRA-CtrI's effectiveness for spatially-aligned generation tasks, we conduct compre- hensive comparisons with multiple competitive approaches. As shown in Figure 4b] our method demonstrates superior performance in several aspects: compared to OminiControl, our approach generates more realistic traffic light images for canny-to-image: produces images with more vivid details for depth-to-image; image; achieves richer color variations in the blue-boxed regions for colorization; better preserves original image details in red-boxed areas for deblurring: and creates more authentic results for inpainting. These qualitative comparisons consistently highlight our method's advantages in maintaining spatial alignment while generating high-quality images across diverse generation scenarios. Quantitative results presented in Table further demonstrate the superiority of DRA- DRA-Ctrl. DRA-Ctrl achieves significant advantages controllability, attaining the best results across all tasks except colorization, while maintaining highly competitive performance in general quality.

---

Canditoo imag
Oun
UNO
Denintontrrl MS. Diflaioo
cenry to iruge depth-dc-immgg
silerization
dedouriing
Iepaisting
stuffee
Conditin Irage
arinal
the Eifeet Tawey
candle with
Orign image
dag soaring and nonecle
dag
Oum
outiil
(a) Subject- driven image generation.
(b) ppatially-aligaan image generation.
Figure 4: Qualitative results comparing different methods.
Table 2: Quantitative results on DreamBench. The best and
Origin Bitmoj-sttll
Origin
Bitmoji-style
second best values of each metric highlighted.
Method
VL Scoret DINO! CLIP-I CLIP-TI
Oracle
0.774
0.885
Textual Inversion
0.569
0.780
0.255
Figure 5:
Qualitative results of
DreamBooth 40
0.668
0.803
0.305
DRA-Ctrl style transferring.
BLIP-Diffusion
0.670
0.805
0.302
ELITE
49
0.647
0.772
0.296
Re-Imagen
0.600
0.740
0,270
BootPIG
35
0.674
0.797
0.311
SSR-Encoder
60
0.612
0.821
0.308
Tace out
OmniGen
51
0.693
0.801
0.315
OminiControl
4
2.21
0.559
0.765
0.310
FLUX.1 IP-Adapter
54
0.582
0.820
0.288
dide Lmn
MS-Diffusion 46
1.94
0.655
0.782
0.307
UniReal 7
0.702
0.806
0.326
Figure 6: Different mixup- based shot transition types.
UNO 50
2.43
0.657
0.786
0.315
DRA-Ctr1
2.56
0.722
0.825
0.302
4.3
Subject-driven Image Generation Results
To validate the effectiveness of DRA-Ctrl for subject-driven generation, we conduct comprehensive comparisons with multiple state-of-the-art approaches. Qualitative results are presented in Figure 4al where our method demonstrates superior subject consistency. As shown in the third row, our approach generates dog that even preserves details like the neck tag, while competing methods exhibit inconsistent breeds or fail to generate the subject altogether. The quantitative results are presented in Table where we compare various tuning-basse based and tuning- ffee approaches. Under all comparison methods, our approach achieves the highest VL Score (2.56), DINO (0.722), and CLIP-I (0.825), along with competitive CLIP-T score of 0.302.
4.4 Style Transfer
We employ GPT-4o to generate 100 original-to-Biimoj-sstt- Bitmoji-style image pairs, which are subsequently used to fine-tune our subject-driven model for achieving style transfer effects. The results are demon- strated in Figure where our model successfully captures the distinctive aesthetic characteristics of Bitmoji-style animation while preserving the original content's structural integrity.
4.5 Ablation Studies
To validate the effectiveness of our proposed strategies, we conduct comprehensive ablation studies on our method from multiple perspectives, including comparisons with T2V/IVV baselines, analysis of different shot transition types, ablation on the number of transition frames, and module ablation.

---

Table 3: Comparison with baselines.
Table Ablation on modules DRA-Ctrl.
VL DINOf CLIP-I CLIP-T
VLT DINO CLIP-11 CLIP-T
Oracle
0.774
0.885
Oracle
0.774
0.885
T2V baseline 2.01
0.658
0.787
0.306
w/o loss reweighting
2.32
0.744
0.839
0.292
12V bascline 2.34
0.803
0.874
0.291
w/o FSPE
2.28
0.777
0,853
0.287
DRA-CtrL
2.44
0.715
0.821
0.298
w/o mixup strategy
2.38
0.900
0.918
0.271
w/o attention masking
2.41
0.777
0,856
0.292
Table 4: Ablation on transition types.
full version
2.42
0.742
0.834
0.295
VI. DINO CLIP-I CLIP-T
slide away fade in fade out 2.42 0.742
2.19 0.708
0.822
0.292
0.834
0.295
Table 7: Generation efficiency analysis.
latent VL, DINO1 CLIP-I CLIP---CLIPTTE CLIP-T Time/s! frames
Table 5: Ablation on frame numbers.
number of transition frames
Oracle
0.774
0.885
VL DINO CLIP-I CLIP-T
12V
2
2.34
0.803
0.874
0.291
10.8
4
2.19
0.692
0.820
0.292
baseline
8
2.42
0.742
0.834
0.295
PRA-Ctrl
4
2.44
0.715
0.821
0.298
24.0
12
2.09
0.715
0.826
0.283
12V
37
1.09
0.698
0.810
0.257
251
Comparison between T2V/1VV baselines. Quantitative results3onn DreamBench align with Figure and Section 3.2 The T2V baseline, whose base model is unable to accept images as control signals, achieves high CLIP-T score but suffers from low DINO and CLIP-I scores. The I2V baseline produces condition image-like outputs, with the DINO real images, but suffers from low prompt adherence. Under identical experimental configurations, DRA-Ctze achieves balanced performance, with DINO, CLIP-I and CLIP-T positioned between the two baselines and the highest VI. Score, exhibiting superior performance.
surpassing the result measured on
Different mixup-based shot transition types. In addition to the fade-in-fade-out approach for constructing transition frames, we also experimented with slide-away transitions, with examples illustrated in Figure Quantitative results in Table demonssratee that the fade-in-fade-out mixup strategy outperforms slide-away across all three metrics. This observation aligns with our findings that video models tend to exhibit stronger priors for fade-in-fade-out shot transitions, while showing weaker priors for more complex transition types.
Number of transition frames. We investigate the impact of varying numbers of transition frames on experimental results, as shown in Table Both insufficient and excessive transition frames harm performance. This phenomenon may stem from two factors: too few frames create excessively large inter-frame variations that increase learning difficulty, while too many frames introduce unnecessary computational overhead and slower convergence under the same training budget.
Module ablation. We conduct ablation studies on our proposed modules, including loss reweighting, FSPE, mixup strategy, and attention masking, with experimental results summarized in Table Since our method employs an I2V model the base architecture, all proposed modules aim to address its inherent limitations of excessive similarity to the condition image and poor prompt adherence. The results demonstrate that FSPE, mixup strategy, and attention masking significantly mitigate these issues, while loss reweighting primarily accelerates model convergence.
4.6 Generation Effiency Analysis
To analyze DRA-CtrI's generation efficiency, we compare against the 12V baseline and the 12V model on DreamBench, assessing generation quality and efficiency. The 12V model generates videos from prompts and condition images, using the final frames outputs. With å = 12 in FSPE (corresponding to 48 pixel-space frames), we set the I2V model produce 145-frame videos. Table 7 results show method achieves 90.4% faster generation than the I2V model with the highest VL score.

---

5 Conclusion
Leveraging the rich high- nighddimensional information priors inherent video models, we propose to repurpose them for low-dimensional controllable image generation, demonstrating advantages akin to "DDimennionnllyy-Reddiictti
Attack" effect compared to conventional image generation models. Specifically, to bridge the gap between video models' native capability for modeling continuous smooth transitions and the requirement for discrete abrupt changes in controllable image generation, introduce novel mixup-based transition strategy that constructs smooth transition between condition image and target image. Moreover, we redesign the attention masking mechanism that precisely aligns text prompts with image-level control signals. Our work establishes new paradigm for activating high-dimensional video models to solve low-dimensional image generation tasks, while paves the way for future development of unified generative models across visual modalities.
Limitations. Our method employs a video model not optimized for image generation, resulting in slightyy inferior performance image quality metrics (FID, SSIM) compared to image-specific approaches. Besides, since Hunyuan Video-12V primarily uses LLa VA 8 for prompt understanding, our CLIP-T scores are marginally lower than competing methods. Additionally, the requirement for transitional frames leads to reduced generation efficiency.
References
[1] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijic Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang. Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. URL httpppp:///ax....// org/
abs/2502. 13923
[2] Tim Brooks. Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman. Eric Luhman, Video generation models as world simulators. OpenAl Blog, 1:8, 2024.
[3] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers, 2021. (ttpsstss://ax org/ abs/2104. 14294
Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer, 2021. URL, tttpp:ttt:///riii org/ abs/2012 203220001100044
[5] Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W. Cohen. Re-imagen: Retrieval-augmeettdd text-to-image generator, 2022. URL /ttp:://////r org/ 2twp/2299.... 14491
[6] Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6599-6602, 2024.
[7] Xi Chen, Zhifei Zhang, He Zhang, Yuqian Zhou, Soo Ye Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, Hui Ding, Zhe Lin, and Hengshuang Zhao. Unireal: Universal image generation and editing via learning real-world dynamics, 2024. URL/https://annt org/ wwwg////////////eee/ee/ee///e//e abs/ 07774
[8] XTuner Contributors. Xtuner: A toolkit for efficiently fine-tuning 11m. sttp::///tttubb.com/ttttee
xtuner 2023.
[9] Patrick Esser, Sumith Kulal, Andreas Blattmann. Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[10] Yutong Feng, Biao Gong, Di Chen, Yujun Shen, Yu Liu, and Jingren Zhou. Ranni: Taming text-to-image diffusion for accurate instruction following. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4744-4753, 2024.
[11] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing toxt-to-image generation using textual inversion. 2022. URL pectts::/arxtte org/abs.////o.o..ooooo...o atwpaab///....///////////// 01618
10

---

[12] Zhen Han, Zeyinzi Jiang. Yulin Pan. Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou. Ace: All-round creator and editor following instructions via diffusion transformer, 2024. URL
https://arxiv. wtg//b//////aaaeoooeeeee 2000000000000
[13] Martin Heusel. Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium, 2018. URL Uttptttss:////// https://axxxx
htg/abb/1706... 08500
[14] Edward J. Hu. Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhi Yuanzhi Li. Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https://:///x org/ abs/2106. 09685
[15] Li Hu. Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-vine vodvideo synthesis for character animation, 2024. htt/ttts:://rxxw.orrr
2311.
[16] Miao Hua, Jiawei Liu, Fei Ding, Wei Liu, Jie Wu, and Qian He. Dreamtuner: Single image is enough for subject-driven generation, 2023. URL pestps://arxxe www.oogg////////.ooooooo/oee /212.. 13691
[17] Lianghua Huang, Wei Wang, Zhi-Fan Wu, Yupeng Shi, Huanzhang Dou, Chen Liang, Yutong Feng, Yu Liu. and Jingren Zhou. In-context lora for diffusion transformers, 2024. URL https://arxiv.. oraeziii.rrr//ae org/abs/ 2410. 23775
[18] Linyan Huang, Haonan Lin, Yanning Zhou, and Kaiwen Xiao. Flexip: Dynamic control of preservation and personality for customized image generation, 2025, URL httts: wwwpg///gg/aaaeeeeeeeee 07405
[19] Phillip Isola, Jun- Yan Zhu, Tinghui Zhou, and Alexei A. Efros. image-to-image translation with conditional adversarial networks, 2018. URL pettts::/arxiv..oo abt////1...11/.//.. 07004
[20] Weijie Kong. Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang. Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang. Changlin Li, Duojun Huang. Fang Yang. Hao Tan, Hongmei Wang. Jacob Song, Jiawang Bai, Jianbing Wu, Jinbao Xue, Joey Wang. Kai Wang. Mengyang Liu, Pengyu Li. Shuai Li, Weiyan Wang. Wenging Yu, Xinchi Deng, Yang Li, Yi Chen, Yutao Cui, Yuanbo Peng. Zhentao Yu, Zhiyu He, Zhiyong Xu, Zixiang Zhou, Zunnan Xu, Yangyu Tao, Qinglin Lu, Songtao Liu, Dax Zhou, Hongfa Wang, Yong Yang, Di Wang. Yuhong Liu, Jiang, and Caesar Zhong. Hunyuanvideo: A systematic framework for large video generative models, 2025. URL http::tttt///aai wwwxxxiv.orggaaae hang/abs/2411 2412. 03603
[21] Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion, 2023. httppp/te:://rriv wwwp:/wwwww.oo.bba.aob/aabbeaae 2ttpp./222........
[22] Nupur Kumari, Xi Yin, Jun- Yan Zhu Ishan Misra, and Samanch Azadi. Generating multi- image synthetic data for text-to-image customization, 2025. URL
http://aaiiv..r////a/5/.00.../ 01720
[23] Black Forest Labs. Flux. httts:///itthbb com/ wttp://hwww...ooooaaaoooreeeeeee fooes--oorestlaaa///
2024.
[24] Dongxu Li. Junnan Li, and Steven C. H. Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing, 2023. /ttt/tttta//ariii/ org/ abs/ 2205. 14720
[25] Zhong- Yu Li. Ruoyi Du Juncheng Yan, Le Zhuo, Zhen Li, Peng Gao. Zhanyu Ma. and Ming- Ming Cheng. Visualcloze: universal image generation framework via visual in-context learning. 2025. URL htgggbbb/250 07960
https://arxiv.
[26] Tsung- Yi Lin, Michael Maire, Serge Belongie. Lubomir Bourdev, Ross Girshick, James Hays, Pictro Perona, Deva Ramanan, C. Lawrence Zitmick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015. URL httpps://aaxxi..rrrrter org/ http//bs/1400 abs51405.11122
[27] Yijing Lin, Mengqi Huang, Shuhan Zhuang, and Zhendong Mao. Realgeneral: Unifying visual generation temporal in context learning with video models, 2025. Uttpsttps///aax http://aaarvvvv...////// abs///ba////// 2203. 10406
[28] Yaron Lipman. Ricky T. Q. Chen, Heli Hen- Maximilian Nickel, and Matl Le. Flow matching for generative modeling, 2023, URL htttps://arxiiii www.oogg/aab..... absaaas/1112 02747
[29] Chaojie Mao, Jingfeng Zhang. Yulin Pan. Zeyinzi Jiang, Zhen Han, Yu Liu, and Jingren Zhou. Ace++ Instruction- based image creation and editing via context-aware content filling, 2025. URL https
//arxiv.org/ abs//501. 024877
[30] Chenlin Meng. Yutong He. Yang Song, Jiaming Song. Jiajun Wu, Jun Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. 2022. URL https
//arxiv , org/ wwwpaa/s....//////////./aaaaaaaa

---

[31] Chong Mou, Xintao Wang, Liangbin Xie. Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models, 2023, URL https://arxiv.. org/ abs/a202. 08453
[32] Xichen Pan, Li Dong. Shaohan Huang. Zhiliang Peng. Wenhu Chen, and Furu Wei, Kosmos-g: Generating images in context with multimodal large language models. 2024. URL htttps:///// 02992
[33]
William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. URL https //axivv.orp 2212/222222.. 09748
[34] Dustin Podell, Zion English, Kyle Laccy, Andreas Blattmann, Tim Dockhorn Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint (3XX:::307.01955 2023.
[35] Senthil Purushwalkam, Akash Gokul, Shatiq Joty, and Nikhil Naik. Bootpig: Bootstrapping shot personalized image generation capabilities in pretrained diffusion models, 2024. URL https://aaxxv org/ aaaaabaa/4440 /200. 13974
[36] Can Qin, Shu Zhang, Ning Yu. Yihao Feng, Xinyi Yang. Yingbo Zhou, Huan Wang. Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, Stefano Ermon, Yun Fu, and Ran Xu. Unicontrol: A unified diffusion model for controllable visual generation in the wild, 2023,
wwtp:////rg/b//////////ooo/s http://as/2300.../ 11147
[37] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transfcrable visual models from natural language supervision, 2021. URL http:: //te:://aaxxveee/ wwwpiii.rrrgaaa org/abs/ abs/
2103. 00020
[38] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent difffusion models, 2022. URL https://arxiv, wwwgg/////////g/ee 10752
[39]
Noam Rotstein, Gal Yona, Daniel Silver, Roy Velich, David Bensaid, and Ron Kimmel. Pathways on the image manifold: Image editing via video generation, 2025. (ttps://:: .rg/ aba/
16819
[40] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation, 2023. URL https://arxiv org/ /bsp//200 12242
[41] Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, 2023. ftt.psttt:://arzii .rg. http//aaa/224...00 2////11144.009 09864
[42] Ke Sun, Jian Cao, Qi Wang, Linrui Tian, Xindi Zhang, Lian Zhuo, Bang Zhang, Liefeng Bo, Wenbo Zhou, Weiming Zhang, et al. Outfitanyone: Ultra-high quality virtual try-on for any clothing and any person. arXiv preprint arXiv:2407. /6224, 2024.
[43] Zhenxiong Tan, Songhua Liu, Xingyi Yang. Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer, 2025. URI. htttss///aariiii www.//rgg//aaaa abs// babss224/ /2111. 15098
[44] Linrui Tian, Qi Wang. Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions. In European Conference on Computer Vision, pages 244 260. Springer, 2024.
[45] Team Wan, Ang Wang. Baole Ai. Bin Wen, Chaojie Mao, Chen-War Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. Jianyuan Zeng. Jiayu Wang. Jingseng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng. Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang. Tianyi Gui, Tingyu Weng. Tong Shen, Wei Lin, Wei Wang. Wei Wang. Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang. Xin Xu, Yan Kou, Yangyu Lv, Yifci Li, Yijing Liu, Yiming Wang. Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong. Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models, 2025. htIps::s:: // wtw.axiiivvv... org/ abs/ 2503, 20314
[46] Xierui Wang, Siming Fu, Qihan Huang, Wanggui Hc, and Hao Jiang. Ms-diffusion: Multi-subject zero-shot image personalization with layout guidance, 2025. _ //taxvv.///iv./eaa org/ abs/ 07209
[47] Xinlong Wang, Wen Wang. Yue Cao. Chunhua Shen, and Tiejun Huang. Images speak in images: A generalist painter for in-tontex visual learning, 2023. URL https: //ta:///aa/ii///aa www.wiwwww/waaaaaaa..aooaoo awt.bb//2/22 02499
12

---

[48] Zhou Wang, A.C. Bovik. H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from enror visibility structural similarity. IEEE Transactions Image Processing, 13(4):600-612, 2004. doi: 10.1109/TIP222038899990
[49] Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation, 2023. URL https:
//arxiv. //axivvorg/abs/ wrw..///3////........../ 2302. 13848
[50] Shaojin Wu, Mengqi Huang. Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian Hc. Less-to-moree generaliza- tion: Unlocking more controllability by in-context generation, 2025. htl.:ttpp///arxwi.rr//aaaa 0206002100
[51] Shitao Xiao, Yueze Wang. Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huane, and Zheng Liu. Omnigen: Unified image generation, 2024. URL https://arxiv org/ abs/ 2409. 11340
[52] Lihe Yang. Bingyi Kang. Zilong Huang. Xiaogang Xu. Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data, 2024.
[53] Zhuayi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng. Da Yin, Yuxuan Zhang, Weihan Wang. Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with expert transformer,
2025, URL https://axxxx org/ wwtggb//////e/o..eo.ooooegeeeee/ 2aabb/2/400 /ttp/4488.0000 06072
[54] Hu Yc, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models, 2023. URL htt.:ttpp///// htappp:////xxxvo//////// hetr//abs/3308.... 221./333880080 06721
[55] Xin Yu, Tianyu Wang, Soo Ye Kim, Paul Guerrero, Xi Chen, Qing Liu, Zhe Lin, and Xiaojuan Qi. Objectmover: Generative object movement with video prior, 2025. URL https://arxix: org/abs/ 2503. 08037
[56] Yu Zeng. Vishal M. Patel, Haochen Wang, Xun Huang, Ting-Chun Wang, Ming-Yu Yu Liu, and Yogesh Balaji. Jedi: Joint-image diffusson models finetuning-free personalized text-to-image generation, 2024. URL 20070
[57] Hongyi Zhang, Moustapha Cisse, Yamn N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 2017.
[58] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion 05543
models, 2023. URL http//tttte///aaii.oor//////////
[59] Yabo Zhang, Xinpeng Zhou, Yihan Zeng, Hang Xu, Hui Li, and Wangmeng Zuo. Framepainter: Endowing interactive image editing with video diffusion priors, 2025. URL htttps://arxi..oo wttp://iiwrrr/a///////a//// abs/2501 08225
[60] Yuxuan Zhang, Yiren Song. Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, and Zhongliang Jing. Ssr-encoder: Encoding selective subject representation for subject-driven generation, 2024. URL htttss:///aaiiii wttp://wwww.ggg///////....
wrgg 2312 16272
[61] Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Liu. Easycontrol: Adding efficient and flexible control for diffusion transformer, 2025. htt.::tttp:/arattt. wttp:///rr//bbb//b//aaaa .07027
[62] Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee K. Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models, 2023. URL https //arxiv. //aax/v..rg/ab//22222 wttp:or/////......// 16322
[63] Jun- Yan Zhu, Taesung Park, Phillip Isola. and Alexei A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks, 2020. URL
[64] zk. text-to-image-2m (revision e64fca4), 2024. URL. htap:://hugiigfaaccco/aatat
jackyhate/text-tor image-2M
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction (Section I our main claims reflect the paper's contribution and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our experimental details are provided in Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
We provide open access to data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee 7/nips.com / siiniioiscyp for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We provide experimental details in Section
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: settings.
All comparison experiments conducted under the same experimental
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Related experimental details are provided in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
Our research fully complies with the NeurlPS Code of Ethics in all respects.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss societal impacts of the work in the Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We describe the safeguards in the Appendix.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets used in the paper are properly credited in compliance with academic standards
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have released our assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification: We use Vision- Vingua-Language model for evaluation, and details are provided in Section[4]
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
20

---

A More Experimental Details
In this section, we provide additional experimental details, including the configurations of LoRA and other hyperparameters. For different tasks, we employ distinct settings: Section A describes the spatially-aligned image generation tasks, Section A.2 covers the subject-driven image generation task, and Section A.3 presents the experimental details for style transfer.
DRA-Ctrl employs LoRA 14] to fine-tune the base model with rank of 16. Since our method needs to simultaneuusly process noiseless condition image token sequences and noisy generated image token sequences, we set the LoRA scale to 0 when handling the generated image token sequences to distinguish between them. Additionally, we set ó to 12 in the Frame Skip Position Embedding (FSPE). This configuration enables frames the latent space effectively emulate frames, corresponding to + 36 x 4 = 145 frames in pixel space approximately equivalent to 5-sonnnd short video at 30 frames per second (fps), which sufficiently achieves the shot transition effect.
A.1 Spatially-llinned Image Generation
Key
C7
T
Tp
Query
C
Attention
Tr
Mask
Masked
Unmasked
Tp
Figure 7: Attention masking strategy in spatially-aligned tasks.
In spatially-aligned image generation tasks, the condition image is directly extracted from the ground-truth image without a corresponding prompt. Therefore, we do not employ the condition image prompt Cp in our experiments, but we still utilize the attention masking strategy, with the corresponding attention mask illustrated in Figure [ Besides, we train the model for 6,000 steps. In septh-to-image and depth prediction tasks, the depth image extracted from the ground image using Depth Anything [52). For the depth prediction task, we prepend "[depth] " to the prompt to guide the model to generate depth maps rather than regular images. In the deblurring task, we apply Gaussian blur to the images with randomly selected integer blur radius between 1 and 10 during training. For the in/out-painting task, we randomly select a rectangular region the image during training, then mask either the selected region (with 0.5 probability) the area outside it (with 0.5 probability) to create the condition image. In the super-resolution task, the condition image is obtained by downsampling the original image by a factor of 4.
A.2 Subject-driven Image Generation
For the subject-driven image generation task, we train the model for 9,000 steps. During inference, while employing attention masking, the simultaneous presence of both target image prompts Tp and condition image prompts Cp may still cause information blending. To address this, we strengthen the interaction between target image tokens T7 and Tp while suppressing Cp's influence the generated output. Specifically, within the (Tr x Tp) attention mask region, we augment the attention weights by adding 0.6 x l (where I denotes the mean absolute value of the original weights). The modified attention computation for this region is formulated as:
Q2K7 Vd
+ 0.6 mean (1 122KZ Vd
)) Vz.
Attention(Z) softmax
(4)
A.3 Style Transfer
21

---

Figure 8: Bitmoji-style example images in our dataset.
[USER PROMPT]
bitmoji M
R+*0300:
Figure 9: The prompt format used for generating Bitmoji-stybe images with GPT-4o.
We collected 100 diverse images containing subjects such as humans, animals and buildings from the web. Using carefully designed prompts, we guided ChatGPT-4o to generate corresponding Bitmoji-style images, which formed our training set. The subject-driven image generation model is fine-tuned for 2,600 steps with batch size of 8
NVIDIA H800 GPU to obtain the final model. Example images from our dataset are shown in Figure 8 and the prompt format we employed is where the image dimensions determined by their original resolutions.
shown in Figure 9
B
More Details about the VL Score
Current evaluation metrics for subject-driven image generation primarily employ DINO and CLIP-I to assess subject consistency, and CLIP-T for prompt adherence. However, two critical limitations exist: first, there lacks comprehensive metric to directly evaluate subject-driven generation quality; second, these existing metrics exhibit notable shortcomings - both DINO and CLIP-I are sigmificantly influenced by background interference, while CLIP-T struggles with fine- ine-grained semantic alignment.
To address these issues, we propose leveraging an advanced Vision-Language (VL) model, such evaluator produce a holistic metric. Our approach consists of three First, we provide the VL model with prompt instructing it to score (prompt, reference image, generated image) triplets based multiple fine-grained criteria for both subject consistency and prompt adherence. Next, we have the model summarize its task to confirm proper understanding. Finally, we input each triplct and collect the model's scores. Since both metrics are discrete scores ranging from 0 to 4, we average them to derive comprehensive metric termed the VL Score. An example input-output demonstration of the VL model is shown Figure TO
QWen2.5-VL [1].
steps:
C More Visualization
This section presents additional qualitative experimental results across all tasks, including transition frames generated by our model. The spatially- spatially-aligned aligned image generation results are detailed in Section C.I while the subject-driven image generation outcomes are presented in Section C.2 and the style transfer performance is analyzed in Section C.3 Unless otherwise specified, all image generation this paper uses 50 sampling steps by default, including both qualitative results and quantitative evaluations, and generated images maintain consistent resolution of 512 x 512 pixels.
22

---

[USER PROMPT)
[Task Instruction] professional
your role reference inage and an generated inage based on a given text proupt. Focus on t2o aspecta:
the semantic consistency
Subject Consietency: Coupare zey visual attributes (Shape: main body outline, structure, and proportions. Color: caturation, hue, brightness, and distribation. Texture: realisn, clarity, and aesthetic appeal. Facial: facial features, only apply when subject
person or sninal) between the
reference and generated inage.
2. Prompt Adherence: Verify if the generated image accurately reflects all elements described in the text prompt: (Ralevance: ubether
shoun. Accuracy: objacts, settings, actions, characteristics Completenees: whetber the geuerated inage onits significant details. Context: apprepriate enviromeett, interactions, You need to provide your scores Subject Consistency and Prompt Adherence based on the folloring criteria. [Subject Coneistency Scoring Criteria] Very Poor (0): No resemblance. The generated inage's 's subject has relation the reference.
elementa
portrayal
background elesente).
Poor (1): Mininal resemblance. The subject falls xithin the same broad category but diffars significantly. Fair (2): Moderate resemblance. The subject shove likeness to the reference with notable variances. (3): Strong resemblance. The subject closely
with only discrepancies. Excellent (4): Mear-identical. The subject of the generated image virtually indistinguccable from the reference,
[Prompt Adberence Scoring Criteria] Very Poor (0): resenblance. The generated image's subject has relation to the reference.
(1): Mininal resenblance. subject
broad category but differs
+ significantly.
(2): Moderate resemblance. subject shoos
Good (3): Strong resenblance. The subject closely natches the refarence with only minor discrepancies. Excellent (4): Near idaa-iienti The subject of the generated inage is virtually indistinguichable from the reference.
[Input Fornat] Every time you will receive two inages and prompt, the first inage is a reference image, and the
inage the generated inage, and
the prompt describing generated
image.
[Output Format] return scores (0-4) (Vaabjecctcconssstaneyyl 3, \ prrhppt_adherrnce""" 4).
format, strictly follouing example:
[VL NODEL O[TPUT]:
Sure, I underatand the task. Please provide ne with the reference image, the generated inage, and the proupt
according provided
[USER PROMPT]:
Did you underatand the task above? Please sumarize the taska need do and show how you will execute the detailed plan for the taek.
[VL NODEL DUTPUT]:
understood
clearly. long, onitted
[USER PROMPT]:
(the first inage) {the second Image) {proupt)
NODEL DUTPUT]:
json'an(" "eteeeeccc_ccssittette 3, "prompt_edherrncee" adherence": 2"
Horo...
Figure 10: An example of VL Score evaluation process.
C.1 Spatially-aligned Image Generation Results
Our method performs image-to-on to-iie-- generation conditioned on input images, where the state of these condition images significantly impacts the output quality. We found that directly using canny edges, depth maps with black representing maximum depth, or black masks in in/out-painting tasks often resulted in unnaturally dark generated images. To address this, we implemented color normalization scheme that remaps the darkest values (0, 0, 0) to medium-gray (128, 128, 128) while linearly scaling all other color values proportionally, preventing extreme darkening.
23

---

C.1.1 Canny-to-image
orange rosy brown hat, smiling brightly against plain lght-colored background
Mosquito frazen clear ice cube on sand, glowing sunset casting go den light with misty holo around ice
Playful black dog peers through a rustic wooden fence bright, curious eyes and an excited expression
Playful orange tabby cat running Jostulyy through sunlit green aroes, surrounded by lush follage and soft sparkles
Serene lotus with pink petals floats near a glowing candle on delkrellectvee water surrounded by stones
Figure 11: More canny-to to-image generation results.
C.1.2 Colorization
Avibrant rainbow yellow and orful feather yellow background
Brown- floral shirt holds colorful
Iridescent rainbow metaliic dark pink resembling
Majestic bird with striking
ruttled soft blurred background
Vibrant orange-red Inclyhug black spats perched bluree green background
Figure 12: More colorization generation results.

---

C.1.3 Deblurring
Adorable sunglass-wearing penguin floral mangarita sunny
Feisty stylized glrrig eyes, sharp plumage, orange against
striking blue-green eyes, makeup, gray background
popples face, green eyes, orange eyeshadow, citthe,, green background
creates water, against background
Figure 13: More deblurring generation results.
C.1.4 Depth-to-image
Adorable fuzzy chick soft pink stands dever potch dapp forest sunlight
Blonde bold red puffer jacket stands confidently snow loden forest, black gloves controsting winter whites
Golden brown bread loat cicsed tabbe, kichenn
green-tippen glow against
Vintage stripes cobblestone alley, details gleaming meedy lighting
Figure 14: More depth-to-image generation results.
25

---

C.1.5 Depth Prediction
Chubby hedgehog eyes
quills glowsng
wings cockpit glowing against stary cosmos swirlngg
egz, greens,
cats perch roottop, gazing pastel soft
Vibrant ditys gray
Figure 15: More image-to-depth generation results.
C.1.6 In/out-painting
with long brown beams bathed
against bluree home backdrop
bold gleaming backdrrp
Mona medical onigmatic boneath
left emerges right, delicately crading
against
lower half imago, holding adorable orange
Figure 16: More in/out-painting generation results.
26

---

C.1.7 Super-resolution
Crisyy butfalo and golcen fries wed-and-white checkered lining gleaming metal with
Iridescent bluebird perches branch blossoms, chest dlowing soft follage
waterfall cascades framed wilderness
rugzed snow-capoed moody, cloud-streaked sky.
blonde black suspenders gazes against bright fluffy
Figure 17: More super-resolution generation results.
C.2 Subject-driven Image Generation Results
condition bion backpack
backpack
cond tion
cat chef outfit
condition
white
condition
sunflowers around
prompt: stuffed
Farget prompt stuffed background
Figure 18: More subject-driven generation results on DreamBench.
27

---

conditio image prorpe: yeiAA wooden vied rests ground bees Nowers and dock
target rrage prompei A woodar vidlin les on sandy beach ocsan
cundition Cher with zuchiors irage grompt, Chair seforeooo colling window, aescrapers glowing trrough glos suniigt croces polisted Framz
mivaralest backstop
condionniggg image
besiee
Indoors
Cool
beside strckedaancaees
peoepe
glatenning
mage
image prompt: Woman cream somaterat calmiyy cracding firelace, sumounddd bycsem ght natte shelees
target image
snowy
Figure 19: More subject-driven generation results.
Interestingly, we discover that during subject-driven image generation, DRA-Ctrl can occasionally control two subjects in the condition image simultaneously. shown in the third row Figure [9] our method successfully makes the tiger wear sunglasses while placing the stacked pancakes on the
grass.
28

---

C.3
Style Transfer
contt en image sromet: An neange or ats quiletly the stooe can Recide the growe gravaes With ihe som parted up. locks one cide.
Oageet Irage grompe blnnji stalt An orange cat sis quiclyy on the stane sibb. Beside itare the groar grasses. Wit it cars perked up, & ools to ond side.
pramat: The an
The hat. They selking hund the coverre Surrounded luth They snalking hand hand leaf covered road. Surrounded by trees.
mmge prompt beinaj style The back view
cadit image sramat delph lnaping mage prompe binojj Gray leaping
consit on image sromst calorfu flowers in the fron, light ollone house with chimrey in the back, surrounded by trees. bagge mage prompe benaj stale Coloriul flowers in the front, a colored house with chimney the back surfounded trees
Figure 20: More style transfer generation results.
D Failure Cases
While DRA-Ctr' successfully achieves controllable image generation in most cases, it may occasion- ally fail the image-to-depth task, primarily manifesting as the presence of colored regions in the generated depth images. We attribute this limitation to the inherent nature of video models, which predominantly generate color data. failure case is presented in Figure 21
Origin Image
Generated Image
cheerful smiling cartoor cactus mised arms stands in vibrant desert landscape under blue sky
Figure 21: A failure case of DRA-Ctrl.
E
Societal Impact
Our work advances controllable image generation with significant societal implications, offering both opportunities for innovation and risks requiring proactive mitigation. Below, we outline the potential positive and negative impacts, alongside measures to address the latter.
On the positive side, our high-quality, controllable generation method empowers creative and practical applications. Artists and designers can leverage it to produce imaginative content efficiently, while

---

educators benefit from dynamically generated visual aids for teaching. The fine-grain control also enables ethical uses in journalism and advertising, enhancing productivity and accessibility across domains.
However, negative impacts must be acknowledged. Malicious actors could exploit the technology to create convincing fake images for disinformation, fraud, or impersonation; to mitigate this, we adopt a gated release of models restrict access. Bias in training data might lead stereotypical or discriminatory outputs, disproportionately harming marginalized groups - addressed through rigor- ous bias testing during development. Further, misuse for non-consensual imagery (e.g.. deepfakes) necessitates monitoring mechanisms and legal safeguards to protect privacy.
In summary, while our technology unlocks creative and educational potential. its risks- pass-particularly around misinformation, bias, and privacy-deman deliherate countermeasures. By combining techni- cal safeguards with poliey-oriented solutions, aim to foster responsible use and maximize societal benefit.
Safeguards
To mitigate potential misuse risks associated with our controllable image generation technology, we will implement a gated release strategy when making the models publicly available. This will include: comprehensive usage guidelines explicitly prohibiting malicious applications such disinformation campaigns and non-consensual imagery generation; an access control mechanism requiring users to agree to ethical use terms before obtaining the model. While we recognize no safeguards can eliminate all risks, these measures represent our proactive commitment to responsible AI development and deployment.
30


Reviews:
[{"review_id": "OzgJQkdwms", "reviewer": "Reviewer_hSh7", "summary": "- This work proposes Dimension-Reduction Attack (DRA-Ctrl), a method that utilizes the long-range context modeling and full attention in video generative models to perform various controllable image generation tasks.\n- Specifically, authors leverage priors from Image-to-Video (I2V) models for the tasks of 1) subject-driven generation, image-to-image translation and style transfer, and demonstrate that such repurposed video models outperform those trained only on images.\n- The main idea of the paper is to construct a synthetic video with the condition image as the first frame and the target image as its last frame. The intermediate frames are generated using a temporal position-aware mixup strategy between the first and the last frames. An I2V model is then trained on these synthetic videos to solve several image-to-image translation tasks. To avoid computation costs due to processing a lot of intermediate frames, they propose Frame-skip position embedding (FSPE) to augment the 3D RoPE embeddings. FSPE \"simulates\" a longer video in the latent space by expanding the PEs.\n- Experiments demonstrate that video generative models can be repurposed to solve controllable image generation tasks and consistently outperform methods built upon image generation tasks", "questions": "- Why is the method called dimension-reduction \"attack\"? What is the attack that is being done in the proposed method? The rest of the title conveys the workings of the proposed method but the name is very misleading.\n- For the VL score, how do you ensure that the model returns the same score for a particular image pair in a randomized validation set? In my experience a VLM outputs different scores for the same input when asked at a later point in time. This makes the metric very unreliable. \n- What is the Oracle score in Table 2? If it is computed using the ground truth images, why can't the VL-Score metric be computed for this row?\n- Please comment on my issue with the experimental setup described in the weaknesses section.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- The paper is well written and easy to follow. The paper has adequate details for fair reproduction.\n- The results on image-to-image translation (Table-1) and subject-driven generation are impressive.\n- The use of mixup, a data augmentation strategy widely used in self-supervised learning, for media generation is novel and this line of work can potentially improve the \"mixup\" of ideas from both the domains.\n- The idea that leveraging video generation models for improved controlled image generation tasks is very interesting and can potentially open up several valuable research problems.\n\n**Weaknesses**\n- My main issue with this work is the evaluation setup. Evaluation setup compares DRA-Ctrl with contemporary work. It fails to normalize the results with compute. A more fair way to evaluate DRA-Ctrl would be to report FLOPs and GPU hours spent by the pre-training models and the finetuning step and then comparing results. \n- Here is the setup that I believe is fair. 1) Image vs Video pre-trained model initialization: Compare a method that uses an image generation model's (trained in N gpu hours, M Flops) weights as initialization to DRA-Ctrl, that initializes from a video generation model trained with the same GPU hours and FLOPs. This experiment would be convincing that the priors learned by video generation model can indeed improve image generation. With the current setup it is not clear if the compute budget of pre-trained model or the video generation task itself is helping. 2) Number of parameters/compute budget in the fine-tuning stage: Compare with methods that use similar compute budgets/parameters in the 2nd stage to fully understand where the improvements are coming from.", "was_revised": true, "final_justification": "I thank the authors for their thorough rebuttal. Authors have satisfactorily answered my questions about computational cost, VLM output stability and the oracle score. AFter reading through other reviews and author response, I vote to keep my initial score.", "rebuttal": "We sincerely appreciate the reviewers' recognition of our work, particularly the acknowledgment that the mixup strategy in DRA-Ctrl is novel and extensive, as well as the assessment that our approach is interesting and has the potential to open up new research directions. We are also deeply grateful for the highly professional and insightful questions raised by the reviewers. Below, we address each of these points in detail.\n\n**Q1:** Concerns that the differences in computational costs may lead to an unfair comparison, when comparing the video model-based method (DRA-Ctrl) and image model-based methods.\n\n**A1:** We sincerely appreciate the reviewer's insightful observations regarding the evaluation setup. We have taken note of these concerns during our experiments and have made every effort to align the evaluation setup when comparing these two categories of methods. \n    \n1) About pre-trained models. The reviewer’s point about controlling computational budgets is well taken. Matching resources between the pretrained image and video models would clarify if DRA-Ctrl’s advantage comes from extra compute or the video task itself. However, video generation is inherently more complex than image generation, which naturally requires more GPUs and time. Due to limited resources, strict alignment during pretraining was not feasible. To address this, we chose HunyuanVideo as our backbone because its architecture closely matches FLUX. This choice helps us align the pretrained models as much as possible.\n\n2) About number of parameters/compute budget in the fine-tuning stage. Since UNO uses FLUX as its backbone, we compare computational budget between DRA-Ctrl and UNO. As demonstrated in Table 1, DRA-Ctrl achieves better performance with less computational budget.\n\nTable 1: Computational budget comparison between DRA-Ctrl and UNO.\n|  | DRA-Ctrl | UNO |\n| :--- | :---: | :---: |\n| number of parameters | 92 M | 956 M |\n| total batch size | 32 | 16 |\n| training steps | 9 k | 10 k |\n| VL Score $\\uparrow$ | 2.56 | 2.43 |\n\n**Q2:** The meaning behind the name (DRA-Ctrl) of our method.\n\n**A2:** We apologize for any confusion our method's naming may have caused the reviewer. The name is inspired by the science fiction novel *The Three-Body Problem*, which introduces \"Dimension-Reduction Attack\", a devastating attack employed by advanced civilizations against lesser ones, characterized by forcibly reducing three-dimensional space to two dimensions with overwhelming power.\n\nIn our work, we leverage a 3D video generation model to tackle 2D image generation tasks, achieving superior performance. Drawing an analogy to the novel’s concept, we refer to our approach as DRA-Ctrl to highlight its capability to bridge and dominate across dimensions.\n\n**Q3:** Concerns about VLM output instability.\n\n**A3:** The reviewer raises a valid observation regarding the inherent variability in VLM outputs. Our VL Score remains statistically reliable for evaluation purposes, supported by the following evidence. 1) Our assessment is conducted on DreamBench, which requires evaluating $30\\times25=750$ images per method and averaging the results. This extensive sampling significantly reduces the impact of random fluctuations. 2) We systematically tested each method with random seeds 0-4 (5 independent runs). As shown in Table 2, while absolute scores show minor variations across seeds, the relative ranking between methods remains consistently stable within each seed. 3) Furthermore, the use of VLM for evaluation has been widely adopted in recent literature[1][2].\n\nTable 2: VL Scores across different random seeds.\n| Random Seed | 0 (rank) | 1 (rank) | 2 (rank) | 3 (rank) | 4 (rank) | Average (rank) |\n| :--- | :--- | :--- | :--- | :--- | :--- | :---: |\n| OminiControl | 2.21 (3) | 2.21 (3) | 1.98 (3) | 2.50 (3) | 2.51 (3) | 2.28 (3) |\n| MS-Diffusion | 1.94 (4) | 1.94 (4) | 1.75 (4) | 2.27 (4) | 2.30 (4) | 2.04 (4) |\n| UNO | 2.43 (2) | 2.43 (2) | 2.13 (2) | 2.73 (2) | 2.75 (2) | 2.49 (2) |\n| DRA-Ctrl | **2.56** (1) | **2.56** (1) | **2.23** (1) | **2.83** (1) | **2.82** (1) | **2.60** (1) |\n\n**Q4:** The Oracle score of VL Score.\n\n**A4:** Thanks for the comment. Unlike DINO and CLIP-I metrics, which only require image pairs [3], computing the VL Score also needs prompt information. Since DreamBench does not provide ground-truth images for every subject under each prompt, we do not report oracle values for the VL Score in our paper. However, to establish a meaningful benchmark, we manually annotate prompts for each image in DreamBench. For each subject, we select one image as the reference and treat the others as ground truth. This enables us to compute a provisional VL Score oracle, as shown in Table 3.\n\nTable 3: The oracle score of VL Score.\n|  | Random Seed = 0 | Average over Random Seeds 0-4 |\n| :--- | :---: | :---: |\n| Oracle | 2.61 | 2.60 |\n| OminiControl | 2.21 | 2.28 |\n| MS-Diffusion | 1.94 | 2.04 |\n| UNO | 2.43 | 2.49 |\n| DRA-Ctrl | **2.56** | **2.60** |\n\n[1] Zhuang, C., Huang, A., Cheng, W., Wu, J., Hu, Y., Liao, J., ... \\& Zhang, C. (2025). Vistorybench: Comprehensive benchmark suite for story visualization. arXiv preprint arXiv:2505.24862.\n\n[2] Tan, Z., Liu, S., Yang, X., Xue, Q., \\& Wang, X. (2024). Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint arXiv:2411.15098.\n\n[3] Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., \\& Aberman, K. (2023). Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 22500-22510)."}, {"review_id": "7vr2mRnyHF", "reviewer": "Reviewer_MS8a", "summary": "This work proposes using video generation models (e.g., Hunyuan Video) to achieve spatial structure-controlled image generation.\nA mix-based transition strategy is introduced to bridge the gap between videos and images.\nThe authors also propose a Frame-Skip Position Embedding, which expands temporal intervals in the latent space.\nExperiments are conducted under various types of conditions to verify the effectiveness of the proposed approach.", "questions": "How does the image generation speed compare to that of Flux and SD?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Weaknesses   \nThe generation quality is sometimes worse than that of Flux, but I think it is acceptable.  \nI wonder whether the inference speed is significantly slower when using video models for image generation.   \nThe authors are encouraged to provide an analysis of why using video models can greatly improve controllability, and to include any failure cases where the method struggles to provide effective control.    \n\nStrengths    \nThe improvement in controllability is impressive.    \nThe topic explores an interesting and promising direction.  \nThe experiments are comprehensive.", "was_revised": true, "final_justification": "Thanks for the author's response. Most of my concerns are resolved and I will maintain my initial score as borderline accept.", "rebuttal": "We sincerely appreciate the reviewer's positive feedback on our work, particularly the recognition of DRA-Ctrl's exceptional controllability, the intriguing and promising nature of our research topic, and the thoroughness of our experiments. Regarding the reviewer's questions, we make the following responses.\n\n**Q1:** The generation quality is sometimes worse than that of Flux, though acceptable.\n\n**A1:** Thanks for the comment. We fully agree that visual quality is an important evaluation dimension. The slight degradation observed in some cases results from our use of HunyuanVideo as the backbone, which does not match Flux in overall visual performance. We chose it deliberately to ensure a fair and controlled comparison, as it shares architectural similarities with Flux (both based on the MM-DiT family) and was state-of-the-art at the time of submission. This setup allows us to isolate the effects of our proposed paradigm and architectural innovations without conflating them with differences in backbone design. While newer models like WAN 2.2 offer better visual quality, their architectures differ significantly from Flux, making them less suitable for direct comparison. we have observed further improvements when applying our method to such models, and we will continue to adopt the strongest open-source backbones and contribute our advances to the community to support broader progress in this area.\n\n**Q2:** Generation speed compared to image generation model and other controllable image generation methods.\n\n**A2:** We thank the reviewer for the highly constructive comments. We evaluate the average single-image generation time on a single H800 GPU with 50 inference steps for generating $512\\times512$ images on DreamBench, and the results are presented in Table 1. The results demonstrate that controllable image generation methods typically require around $2\\times$ to $6\\times$ inference time of their backbone models. Our DRA-Ctrl doubles the number of latent frames of UNO, a strong baseline (4 latent frames v.s. UNO's 2 latent frames), while maintaining reasonable computational overhead, the image generation time increases by only 63% and demonstrating superior performance.\n\nTable 1: Generation speed of image generation models and controllable image generation methods.\n|  | Backbone | Generation Time/s | VL Score $\\uparrow$ |\n| :--- | :---: | :---: | :---: |\n| SDXL | - | 2.15 | - |\n| FLUX | - | 4.09 | - |\n| MS-Diffusion | SDXL | **4.65** | 1.94 |\n| OminiControl | FLUX | 7.93 | 2.21 |\n| UNO | FLUX | 14.7 | 2.43 |\n| DRA-Ctrl | HunyuanVideo | 24.0 | **2.56** |\n\n**Q3:** Analysis of why video models can improve controllability. Failure cases of DRA-Ctrl.\n\n**A3:** Thanks for the comment. Video models are trained on a large number of frames with spatiotemporal dependencies, thereby encoding rich prior knowledge about long-range context, consistent object transitions, non-rigid transformations, and high-level scene dynamics. These priors align well with goals of controllable image generation, as discussed in L32-35 in our paper. DRA-Ctrl successfully leverages these inherent priors from video models for controllable image generation, achieving superior performance. In contrast, image models are primarily trained with text-image pairs and lack such prior knowledge relevant to controllable image generation. They struggle to understand object pose variations, background changes, and other image transformations. Therefore, adopting video models as the backbone demonstrates clear advantages over using image models. Furthermore, we visualized the attention maps of DRA-Ctrl and observed that during subject-driven image generation, the attention weights between the final frame and the condition image are significantly higher in the subject regions. This clearly demonstrates DRA-Ctrl's strong controllability in preserving subject fidelity.\n\nWhile DRA-Ctrl successfully achieves controllable image generation in most cases, it may occasionally fail in the image-to-depth task, primarily manifesting as the presence of colored regions in the generated depth images. We attribute this limitation to the inherent nature of video models, which predominantly generate color data. The limited LoRA fine-tuning appears insufficient to completely prevent the model from producing partial color outputs in these cases. We have included these failure cases in the revised version."}, {"review_id": "7T3BegmNZ4", "reviewer": "Reviewer_h3gK", "summary": "This work targets at controllable image generation by leveraging existing video generation models. Multiple image generation is formulated into keyframe inbetween task in video generation. By carefully designning the keyframe transition between condition image and target one, the video capabilities in smooth transition over time are reused. Experimental results demonstrate the effectiveness of reusing video generative prior in image generation.", "questions": "- In terms of number of transition frames, what if we have no transition frames? That is, we directly feed the condition image and target one.\n- As mentioned at L159, I2V model over-preserves similarity to the condition image and exhibits poor prompt adherence. After keyframe fine-tuning, is prompt following better?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This paper is well-organized and clearly presented. \nThe motivation for reusing the video generative prior is insightful for image applications.\nThe experimental details are thoroughly provides including training and evaluation, which helps reproduction.\nThe comparison against prior efforts is comprehensive. Ablations further help the audience better understand every components.", "was_revised": true, "final_justification": "Thanks for the response that addressed my concerns.", "rebuttal": "We gratefully acknowledge the reviewer's positive assessment of our work, including conceptually insightful motivation for image applications and comprehensive experimental analysis. In response to the reviewer's questions, we address each point as follows.\n\n**Q1:** What if we have no transition frames?\n\n**A1:** We appreciate the reviewer's valuable suggestions for improving our work. We have conducted additional ablation studies on the number of transition frames, including an experiment without any transition frames. The results are presented in Table 1. The experimental results align with the I2V baseline findings: when no transition frames are used, the generated images maintain excessive similarity to the condition images (as evidenced by the high DINO and CLIP-I scores in the table), while demonstrating poor prompt adherence (indicated by the lower CLIP-T scores).\n\nTable 1: Ablation study on the number of transition frames.\n| number of transition frames | VL Score $\\uparrow$ | DINO $\\uparrow$ | CLIP-I $\\uparrow$ | CLIP-T $\\uparrow$ |\n| :---: | :---: | :---: | :---: | :---: |\n| 0 | 2.37 | 0.736 | **0.837** | 0.291 |\n| 4 | 2.19 | 0.692 | 0.820 | 0.292 |\n| 8 | **2.42** | **0.742** | 0.834 | **0.295** |\n| 12 | 2.09 | 0.715 | 0.826 | 0.283 |\n\n**Q2:** After finetuning, does the I2V model show improvement in 1) reducing excessive similarity between generated images and condition images, and 2) enhancing prompt adherence?\n\n**A2:** Yes. We clarify that the I2V model in L159 is derived from HunyuanVideo-I2v, finetuned using two-frame data from the Subjects200K dataset (referred to as I2V baseline in our paper). As demonstrated in Table 3 in the paper (shown in the following Table 2), I2V baseline exhibits two critical limitations: 1) excessive similarity to condition images, evidenced by very high DINO score surpassing even the oracle and very high CLIP-I score approaching oracle, and 2) poor prompt adherence, reflected in low CLIP-T score. However, under the same experimental setting, DRA-Ctrl exhibits high DINO and CLIP-I scores, yet significantly lower than I2V baseline, and improved CLIP-T score compared to I2V baseline. This result demonstrates that the keyframe-finetuned DRA-Ctrl avoids the two limitations of the I2V baseline.\n\nTable 2: Comparison with baselines.\n|  | VL Score $\\uparrow$ | DINO $\\uparrow$ | CLIP-I $\\uparrow$ | CLIP-T $\\uparrow$ |\n| :--- | :---: | :---: | :---: | :---: |\n| Oracle | - | 0.774 | 0.885 | - |\n| T2V Baseline | 2.01 | 0.658 | 0.787 | **0.306** |\n| I2V Baseline | 2.34 | **0.803** | **0.874** | 0.291 |\n| DRA-Ctrl | **2.44** | 0.715 | 0.821 | 0.298 |"}, {"review_id": "S6nr6Znih0", "reviewer": "Reviewer_A4pM", "summary": "The paper proposes a shift for the video generation models to use them for controllable image generation. In the paper, the framework, DRA-Ctrl, is proposed for task adaptation and knowledge compression. DRA-Ctrl employs two strategies: 1. Mixup-based transition to bridge the gap between videos and images and 2. Frame-Skip Position Embedding to allow large image transformations without generating many intermediate results. DRA-Ctrl is evaluated on various controllable image generation tasks such as subject-driven generation, style transferring, spatial-aligned controllable generation. The proposed model uses HunyuanVideo-I2V as backbone model.", "questions": "Please have a consistent style on the tables. Table 1 highlight the proposed model with yellow, but other tables don’t and they have DRA-Ctrl. In table 1, the best scores are in bold, while, in table 2 the best scores are highlighted red and the second-best scores are blue. For the rest of the tables the best score are in bold for VL results only. Moreover, some tables have arrow near name of metrics, some don’t. Please make tables consistent.\n\nThank you for your acknowledgment and detailed discussion on safeguards  and societal impact with both the potential positive and negative impacts of the proposed model!", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Writing is clear and the flow is easy to follow.\n\n2. The proposed model can be used for various condıtional image generation tasks.\n\n3. The paper provides comprehensive evaluation and ablation studies.\n\nWeaknesses:\n\n1. Comparison with SD1.5 in Table 1 can be useless because it is the initial stable diffusion-based model and there are SDXL, SD2, SD3, and SD3.5. I find comparison with the recent models more interesting.\n\n2. Possible long inference time: This is a known limitation of many diffusion-based image generation models, and it is likely a weakness of the proposed model as well. However, the paper provides limited analysis of inference time. While it includes an efficiency comparison with baseline models, this analysis should be extended to include state-of-the-art (SoTA) image editing and generation models. Without this broader comparison, it is difficult to be convinced of the proposed model’s efficiency.", "was_revised": true, "final_justification": "I decided to maintain my score as Borderline Accept.", "rebuttal": "We thank the reviewer for recognizing our method's versatility across conditional image generation tasks, thorough experiments, and responsible discussion of safeguards and societal impact. Below we provide answers to the reviewer's questions.\n\n**Q1:** Comparison with the recent models such as SDXL, SD2, SD3, and SD3.5.\n\n**A1:** Thanks for this excellent point. In Table 1 in our paper, we focus our comparison on SD1.5 and FLUX-based methods because we consider FLUX's performance highly competitive with models like SDXL, SD3, etc. We completely agree that broader comparisons are essential, which is why we include them in Table 2, benchmarking against methods like MS-Diffusion (built on SDXL) and BootPIG (on SD2.1). The reviewer's feedback highlights that our presentation could be clearer. In the revised version, we will restructure our experimental tables to better integrate these results, ensuring the comprehensiveness of our evaluation is immediately apparent and avoids any misunderstanding.\n\n**Q2:** Possible long inference time.\n\n**A2:** We sincerely appreciate the reviewer's constructive suggestions. In response, we have supplemented our analysis with comparative results between DRA-Ctrl and other state-of-the-art (controllable) image generation methods, as presented in the Table 2. All experiments are conducted on a single H800 GPU with consistent settings: 50 inference steps for $512\\times512$ image generation, evaluated on DreamBench. The results demonstrate that controllable image generation methods typically require around $2\\times$ to $6\\times$ inference time of their backbone models. Our DRA-Ctrl doubles the number of latent frames of UNO, a strong baseline (4 latent frames v.s. UNO's 2 latent frames), while maintaining reasonable computational overhead, the image generation time increases by only 63\\% and demonstrating superior performance.\n\nTable 2: Generation speed of image generation models and controllable image generation methods.\n|  | Backbone | Generation Time/s | VL Score $\\uparrow$ |\n| :--- | :---: | :---: | :---: |\n| SDXL | - | 2.15 | - |\n| FLUX | - | 4.09 | - |\n| MS-Diffusion | SDXL | **4.65** | 1.94 |\n| OminiControl | FLUX | 7.93 | 2.21 |\n| UNO | FLUX | 14.7 | 2.43 |\n| DRA-Ctrl | HunyuanVideo | 24.0 | **2.56** |\n\n**Q3:** Inconsistent style on the tables.\n\n**A3:** We sincerely appreciate the reviewer's careful review of our manuscript's formatting. In the revised version, we will implement the following improvements: 1) adopt a unified highlight color scheme throughout, 2) emphasize all optimal results by applying bold formatting (previously unbolded), and 3) consistently annotate all metrics with directional arrows for clarity."}, {"review_id": "PAedP9WTuY", "reviewer": "Reviewer_1JJJ", "summary": "In this paper, the authors focus on the problem of image generation via a video generative model. They build on top of the HunyuanVideo-I2V model, training it with image pairs set as two frame videos with a mixup transition. They train their model on the Subjects200K and Text-to-Image-2M datasets and evaluate their approach on a number of diverse visual tasks: Canny to Image, Depth to Image, Deblurring, Colorization, and Masked out inpainting, and style transfer. They show quantative results on these tasks, showing improvement over baselines. Finally they perform an ablation analysis.", "questions": "1. Can you elaborate on the novelty of the approach? It seems that this is just finetuning a video model for image generation.\n\n2. It isn't surprising that video priors would improve image generation. Could you elaborate more on the significance of this result?", "limitations": "Yes, the authors have addressed the limitations of the work.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper focuses on a problem with immediate applications to computer graphics and computational photography: Image Generation\n\n2. The paper shows promising results on a diverse number of tasks, not simply general image synthesis.\n\n3. The approach is simple and makes intuitive sense: Video priors should strengthen image generation performance.\n\nWeaknesses:\n\n1. The paper's result is not particularly surprising. Video models probably outperform strict image models as the data domain of video is much richer.\n\n2. The approach does not have many novel technical contributions, it's largely taking an existing video model and tuning it for better image generation performance. \n\n3. There are some minor grammatical and stylistic issues (such as the choice to put a figure before the abstract), although these do not dramatically detract from readability.", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal. Considering also the comments of other reviewers, I keep the score at 4.", "rebuttal": "We are deeply grateful for the reviewer's positive assessment of our work, particularly immediate applications of our method, consistently strong performance across diverse tasks, and conceptually intuitive design. Below we provide comprehensive answers to the reviewer's questions.\n\n**Q1:** It is not surprising that video models outperform image model on controllable image generation tasks.\n\n**A1:**  We fully agree with the reviewer's perspective that video priors would significantly improve controllable image generation. Through our carefully designed mixup strategy, attention masking strategy, and FSPE, we have successfully transformed this intuitive concept into practical algorithms. The effectiveness of our method has been thoroughly validated through extensive experiments including comparative studies, ablation experiments, and qualitative evaluations on 7 different tasks.\n\nWhile the idea of using video priors to enhance controllable image generation may appear straightforward, we would like to clarify that effectively activating a model's prior knowledge for new tasks remains an important, challenging, and widely studied problem in computer vision research. This challenge has become particularly prominent given the abundance of powerful open-source models available in the community, making the alignment of pre-trained models to desired outcomes a highly active research area. Representative works in this direction include ControlNet [1], which leverages the priors of text-to-image generative models combined with external perceptual control conditions to accomplish new tasks, and CLIP [2], whose prior knowledge has been widely applied to image retrieval and semantic matching tasks, among others.\n\nIn our work, the key research focus has been on how to effectively activate video priors. As detailed in Section 3.2 of our paper, **directly applying simple fine-tuning of T2V or I2V models leads to significant issues, either in terms of poor subject consistency or weak prompt adherence** (the quantitative experimental results are presented in Table 3 of our paper). To address these challenges in utilizing video priors, we proposed the mixup strategy that transforms the image generation problem into a very short video generation task through constructed transition frames. Simultaneously, we incorporated the attention masking strategy to ensure proper information interaction and FSPE to reduce the number of required transition frames, ultimately enhancing our method's performance. The significance of our work lies in its ability to effectively harness the inherent yet previously difficult-to-utilize prior knowledge in video models.\n\n**Q2:** Our method does not have many novel technical contributions.\n\n**A2:** We sincerely apologize for any confusion caused and will provide clarification on this matter. Our main technical contributions are summarized below.\n\n1. Mixup strategy. Inspired by our observation that video generation models often transition between scenes using fade-in-fade-out effects, we propose a mixup strategy to construct transition frames, naturally transforming the controllable image generation task into a short video generation problem. To the best of our knowledge, we are the first to apply video models to controllable image generation via this mixup approach.\n\n2. Novel attention masking strategy. To enhance the model's understanding of the condition image content, DRA-Ctrl occasionally incorporates condition image prompts. To ensure proper information flow, we introduce a novel attention masking strategy.\n\n3. Frame Skip Position Embedding (FSPE). To reduce the number of required transition frames and improve efficiency, we propose FSPE, which enables a small number of frames to simulate longer sequences, achieving smooth transitions from the condition image to the target image.\n\n4. VL score. For more accurate evaluation in subject-driven image generation tasks, we design a new metric leveraging powerful vision-language models.\n\n**Q3:** Some minor grammatical and stylistic issues.\n\n**A3:** We sincerely appreciate the reviewer's careful review, and we will address these grammatical and stylistic issues in the revised version.\n\n[1] Zhang, L., Rao, A., & Agrawala, M. (2023). Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 3836-3847).\n\n[2] Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021, July). Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763). PmLR."}]

Author rebuttal:

Meta-review: The paper proposes a dimension-reduction-based approach for control in video generative models. The reviewers found the method to be sound and to improve on existing approaches.
Decision: Accept (poster)