Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy
Authors: Yuan Zeng
Yujia Shi
Yuhao Yang
Dongxia Liu
Zongqing Lu
Wenming Yang
Qingmin Liao
Venue: ICLR 2026 Poster (2026)
Keywords: Generative Models
Human Motion Synthesis
Representation Alignment
Pose-Free Animation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Human image animation aims to generate a video from a static reference image, guided by pose information extracted from a driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such signals are prone to errors under occlusion or complex poses. Building on these observations, we present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos. We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning dependable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross-ID features with those learned from same-ID data, regularizing optimization and accelerating convergence. 
Extensive experiments demonstrate that DirectAnimator attains state-of-the-art visual quality and identity preservation while remaining robust to occlusions and complex articulation, and it does so with fewer computational resources. Our project page is at https://directanimator.github.io/.

Published as a conference paper at ICLR 2026
BEYOND SKELETONS: LEARNING ANIMATION DI- RECTLY FROM DRIVING VIDEOS WITH SAME2X
TRAINING STRATEGY
Yuan Zeng' Zongqing Lu Tiigghu University
Yujia Shi2.3
Yuhao Yang'
Dongxia Liu
Wenming Yang'
Qingmin Liao
Harbin Institute of Technology *Pengcheng Laboratory
zengy22, yangyh23, 3, 1 1iudx24 wouudddeeeaaaa @maalls...........lll.m...mmlllm memails.tsinghhae edu..eduuc.. cn, 24b951075estu hit .ddu. cn [luzq, yang. wenming szz. tsinghua. edu. et,, liaogm@tsinghenne edu.cn
D
(o)
stage2 wio. SameZX Same2X
Reference
6:50-
Training Loss
DirectAmimator
0.30
faster
0.00
20c
Training Iteration
Figure Our proposed driving cue provides more robust representation for complex motions and self-occlusions. Left: Errors in skeleton maps such as front-back confusion, inaccurate hand localization, and missing limbs result noticeable artifacts in StableAnimatoo outputs. In contrast, DirectAnimator uses raw pixels from the driving video as driving signals, generating accurate and realistic frames. Right: The Same2X training strategy significantly improves training efficiency in cross-ID scenarios (Stage 2), reaching the same loss level 6.7x faster than training without it.
ABSTRACT
Human image animation aims to generate a video from a static reference image, guided by pose information extracted from driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such sig- nals are prone errors under occlusion complex poses. Building on these ob- servations, we present DirectAnimator, framework that bypasses pose extraction and directly learns from raw driving videos. We introduce Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning de- pendable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross- ID features with those learned from same-ID data, regularizing optimization and accelerating convergence. Extensive experi- ments demonstrate that DirectAnimator attains state-of-the-art visual quality and identity preservation while remaining robust to occlusions and complex articula- tion, and it does with fewer computational resources. Our project page is https:/ddirecaammattogggtthhl

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Human Image Animation (HIA) aims to generate a photorealistic video of a reference identity per- forming poses and expressions sourced from driving video. This task has witnessed growing interest due to its wide range of applications in virtual avatars, video editing, and entertainment content creation. In recent years, numerous HIA methods have emerged, leveraging different de- noising models, including SD-based (Rombach et al. 2022a) approaches such as AnimateAnyone Hu 2024), SVD-based (Blattmann et al.) (203) methods like StableAnimator (Tu et al. 2022), and DiT-b (Peebles & Xie| X (023) frameworks such as UniAnimate-DiT (Wang et al., 2025) Most of these methods rely on intermediate representations such as skeleton maps (Yang et al. 2023),
DensePose (Güler et al. 2018), or SMPI. parameters (Loper et al. 2023) as driving signals to guide the animation process. However, existing driving signals introduce several limitations. First, pose conditioning is often noisy and unreliable. Due to challenges like occlusion complex body artic- ulation, existing pose estimators often produce erroneous or incomplete results, leading to distorted generations. Second, pose representations often struggle to capture facial expressions accurately, as facial landmarks or sparse keypoints provide limited semantic richness, often leading to expression- less unnatural animations. To address these limitations, more intuitive approach is directly use the raw driving video as driving signal, leveraging its rich information without relying on in- termediate abstractions. This better mirrors how humans learn motion patterns, relying on holistic visual demonstrations rather than simplified pose representations. Nevertheless, such direct con- ditioning is non-trivial and introduces several critical challenges. First, in raw driving frames the cues that matter most for animation, namely body pose and facial expressions, are not explicitly encoded but are entangled with appearance details. Unlike skeletons, where these cues are modeled by explicit keypoints and lines, the model must learn to extract and control them directly from raw pixels while keeping the reference identity intact. Without an appropriate representation and injec- tion mechanism, the model is prone to failing to follow the driving motion. Second, even if motion and expression cues can be extracted in same-ID settings, training directly on cross-ID pairs further complicates optimization. When the reference and driving identities differ, the model must simulta- neously (i) follow the motion and expressions in the driving video and (ii) preserve the appearance of the reference identity. This dual objective makes gradients noisy and slows convergence, and naive cross-ID training often leads to unstable training dynamics.
To overcome these challenges, we propose DirectAnimatorr, which animates reference images di- rectly from raw driving videos. To address the first challenge that motion and expression cues are buried in raw pixels, DirectAnimator abstracts motion, expression, and spatial correspondence into a structured driving cue triplet and injects these cues into the denoising process via a CueFusion DiT block. The driving cue triplet comprises three complementary signals. (i) Pose Cue, which captures the foreground region of the driving video; (ii) Face Cue, which crops and centers face regions to guide expressive synthesis; and (iii) Location Cue, consisting of grid-based softened body and face masks, provides spatial pri- ors to facilitate aligmment between the driving video and the reference image. In the CueFusion DiT block, the reference image is encoded into vision embeddings that flow through the DiT backbone, while the three driving cues are encoded into separate control embeddings. These control embed- dings modulate each DIT block via adaptive layer normalization (AdaLN) (Perez al.. 2018 and gated residual connections, so that "who the person is" (identity) is carried by the main denoising path, and "how the person moves and emotes" is injected through conditional modulation. To fur- ther mitigate the second challenge, namely the optimization difficulty of direct video conditioning in cross-ID scenarios, we introduce the Same2X training strategy. The model is first pretrained on same-ID data to establish its ability to animate the reference image under the guidance of the driving cue, and then adapted to cross-ID scenarios by aligning internal representations with same-ID model via a novel Same2X alignment loss. This strategy accelerates convergence and improves generation quality in challenging cross-II settings.
temporally coherent pose sequences by segmenting and frequency-flteriin
Figure I demonstrates the effectiveness of our proposed driving cue and Same2X training strategy. On the Teft, skeleton-based methods suffer from (a) front-back ambiguity, (b) hand misplacement due to occlusion, and (c) missing keypoints, leading to anatomically incorrect results by StableAni- mator (Tu et al., 2024). In contrast, our Driving Cue mitigates these issues and enables more robust animation. On the right, we show training curves of denoising loss, where our Same2X training strategy significantly accelerates convergence in the cross-ID ID training stage (stage 2). achieves the same loss level 6.7x faster than training without the strategy and reaches final loss comparable
2

---

Published as a conference paper at ICLR 2026
to that of the same-ID training stage (stage 1). Moreover, Same2X is generalizable framework that reduces reliance on real data by leveraging pseudo data, improving training efficiency and animation quality. Our contributions are summarized as follows: (i) We reformulate human image animation to use the raw driving video instead of an estimated pose proxy as the driving signal, enabling the model to learn holistic motion and expression patterns without noisy intermediate estimators. (ii) We instantiate this paradigm in DirectAnimator, framework that abstracts raw videos into a structured Driving Cue Triplct and injects these cues via a CueFusion DiT block, together with the Same2X training strategy that transfers supervision from same-ID to cross-ID settings. (iii) Exten- sive experiments and ablation studies validate the effectiveness of the DirectAnimator framework and demonstrate that it achieves state-of-the-art animation quality.
2 RELATED WORK
Driving Signals in Human Image Animation. Recent HIA methods explore various driving sig- nals to improve identity preservation, motion accuracy, and generalization. Most diffusion-basee methods directly adopt 2D skeleton maps as pose conditions. Disco (Wang et al., 2024b) is among the first to apply diffusion models to HIA, improving compositionality and generalization through disentangled control and human-centric pretraining. AnimateAnyone (Hu) 22224) enhances appear- ance consistency and motion smoothness via ReferenceNet and pose-guided temporal modeling. UniAnimate- DiT (Wang et al., 2025) extends the UniAnimate (Wang et al. (024c) framework by replacing the SVD denoising model (Blattmann et al. 2023) with Wan2.1 [Wan et al. 2025), a DÝT- based architecture for improved video synthesis. DynamiCtrl (Zhao al. 2025) augments the driv- ing signal with detailed textual prompts describing appearance, improving identity preservation via the text branch in the MM-DiT framework. Although each method introduces unique architectural components improvements, they all rely 2D skeleton maps as pose conditions. To improve the reliability of pose guidance, MimicMotion (Zhang et al., 2024) encodes keypoint confidence directly into the visual skeleton map. allowing the model to focus reliable skeletons under occlu- sion and noise. StableAnimator (Tu et al.) 2024) further incorporates ArcFace-based (Deng et al., 2019) face embeddings into the driving signal enhance identity preservation. MagicAnimate (Xu et al. 2024) replaces sparse skeleton maps with DensePose maps (Güler et al. (0018), enabling more robust pose conditioning for complex or rotational motions. Champ (Zhu al. 20224) leverages the SMPL model (Loper et al.) 2023) along with rendered depth, normal, and semantic maps to provide geometry-awarc motion guidance, enabling accurate 3D shape and pose aligmment. RaalesDance (Zhou et al. 2024) incorporates HaMeR-based 3D hand representations (Pavlakos et al.) 2024) to enhance realism hand animation. HumanDiT (Gan et al. 2025) replace skeleton maps with key- point maps, reducing spatial overlap and ambiguity in pose conditioning. Unlike prior methods that use intermediate representations (c.g., skeleton maps, DensePose, or SMPL) as driving sig- nals, DirectAnimator directly conditions on raw driving video pixels, which provides richer motion information and mitigates the accumulation of pose estimation errors.
Facial Animation without Explicit Pose Estimation. There exists a line of work on facial and portrait animation that also avoids explicit pose or keypoint estimation by learning motion repre- sentations directly in latent or image space. LIA (Wang al. 2022) trains self-supervised auto- encoder and animates images via linear trajectories in the latent space, without extracting explicit structural representations. AniTalker (Liu et al. 2024) targets talking-face generation and learns an identit--disentangled motion representation together with a diffusion model for diverse facial mo- tions. X-Portrait (Xie et al. 2024) further leverages a pre-trained reenactment network to synthesize cross-identity driving frames and uses these synthetic signals to train its motion control module, improving generalization without relying on explicit pose or keypoints. All these methods bypass explicit pose estimation but are designed for single-face or upper-body talking-head scenarios. In contrast, our method targets full-body human image animation, where large body articulation, ex- tensive spatial extent, and cluttered backgrounds make direct video guidance more challenging than in single-face or talking-heat scenarios.
key Representations Alignment for Training DIT. Representation alignment has emerged technique to stabilize and accelerate the training of diffusion transformers. REPA YYu et al. 2024) first introduced an alignment mechanism that regularizes the feature space of the diffusion model to match that of a pretrained vision encoder (c.g.. DINOv2 (Oquab et al.] (023)). This approach improves sample quality and accelerates convergence by anchoring training to semantically mean-
3

---

Published as a conference paper at ICLR 2026
ingful features. REPA-E (Leng al. 2025 builds REPA by extending representation alignment to enable end-to-end training of the VAE and diffusion model. By aligning intermediate features across the two modules, REPA-E accelerates training, and enhances latent space quality. Most cently, SRA (Jiang et al. 2025) argues that meaningful representations can emerge naturally from the generative process itself. It proposes self-distillation approach that aligns the denoised latent features of earlier layers with those of later layers, enabling implicit representation learning without any auxiliary encoders. Inspired by these approaches, integrate representation alignment into the HIA training paradigm and propose the Same2X training strategy, which leverages same-ID features as internal guidance for cross- training stage.
Facee Detrctor
Frnquetcy Dernate
Lotest Yeatare
Grenatennte
Eeieediing
Element-stle ool
Gate
Golh
Phobylascnt
Prompt
Text Encoder
VAEE Enco *
Reference
AdsLN & Modulatt
FueeFusion DTT Block
Patchify
Frese
Gate
Eyeet
Attention
C Poes
Lacacion
Evisoon
MaLLN & Modulate
AdalX Modulate
Eyras
Erect
Fpass
Fuss
E
Exiiion
Epace
()) Framowekk of DirectAnimator
(b) CucFusion DIT Block
Figure 2: Overview of DirectAnimator. (a) We replace the skeleton maps with our proposed driving cue triplet: Pose Cue (Crose, Face Cue (CRace), and Location Cue (Crocation). frozen VAE encoder maps the reference image, pose cue, and face cue into the latent space. Pose and face latents are each concatenated with their corresponding masks from the location cue. These features are then patchified and fed into the CF-DiT Block. (b) The CF-DiT Block injects pose and face cues via Adaptive LayerNorm with immeconditioned modulation, and uses gated residuals to ensure stable and controllable denoising.
3 METHODOLOGY
3.1 NETWORK ARCHITECTURE
The overall architecture of DirectAnimator is illustrated in Figure 2 (a), the input com- prises a reference image and a driving video sequence DI:N = [D1,... DN We first preprocess the driving video extract driving cues that capture essential pose and expression information. These cues are then encoded into latent features using a 3D Variational Autoencoder (3D VAE), and subsequently transformed into sequence of visual patch embeddings via patchify module (Peebles & Xie 2023). The resulting embeddings are fed into the proposed CueFusion DiT Block, which performs multi-source feature fusion and denoising to generate the animated frames. In the following subsections, we detail the construction of the driving cues and the design of the CueFusion DiT Block.
Overview.
Driving Cue Extraction. Due to the inherent limitations of skeleton based driving signal, such as instability and limited expressiveness, we propose bypass explicit keypoint extraction and instead drive human image animation directly using raw information from the driving video. To this end, we design three complementary driving cues, namely Pose Cue, Face Cue, and Location Cue, to replace traditional skeleton maps and other pose representations. Examples of the driving cues shown in Figure with additional examples provided in the appendix.
To capture the identity's pose information, SAM (Ren et al., 2024)) to isolate the foreground regions from the driving video. Compared to pose estimators (e.g., OpenPose (Cao et al., 2017), DWPose (Yang al. (223)), segmentation is generally more stable under occlusion, motion blur, and other real-world conditions. Neverthe- less, segmentation may still produce noisy incomplete results. Inspired by the random dropping
use an image segmentation model (e.g., Grounded

---

Published as a conference paper at ICLR 2026
Drising Frome
Pose Cue
Face Cae
Location Cue (Puoe&/ecee)
Briviin Frome
Pose Cur
Pace Ose
Locadion Cue (Pose&Face)
Figure 3: Examples of driving cues.
strategy, we discard low-quality segmentation results, forcing the model to rely on adjacent results for temporal reasoning. However, the segmented foreground contains rich appearance details (c.g., clothing and hair textures), such high-frequency information may distract the model from focusing on the pose information. Therefore, we apply low-pass filtering in the frequency domain to sup- press irrelevant details and emphasize pose dynamics. The resulting detail-suppressed foreground sequence serves as our Pose Cue.
Although previous methods have shown strong performance in pose transfer, their ability to capture facial expressions remains limited. We attribute this to the insufficient expressiveness of 68 68-point facial landmarks (Yang et al. 2023), which struggle to represent complex expressions. Inspired by X-Dyna (Chang et al. 2025), we directly use face regions cropped from the driving video as conditions for facial animation. Specifically, we first localize the face regions, then crop, resize, and center them to preserve maximal expressive detail. This process produces a Face Cue that enable more expressive facial animation.
In cross-ID settings, the location and scale of the identity in the reference image may differ signif- icantly from those in the driving video. Thus, spatial alignment is necessary before feature fusion. Prior methods typically rely on affine transformations of skeleton maps to achieve pose alignment (Tu et al. 22224 Xu et al. (0024). However, in DirectAnimator, since the pose cue is represented by dense foreground pixels rather than sparse skeleton maps, applying affine transformations directly may lead to distortion or structural artifacts. Thus, we introduce Location Cue, consisting of body and face masks. We generate body masks for the driving identity and align them to the reference identity using the spatial alignment strategy proposed in Animatr (Tu et al. 2024). To mit- igate the risk of identity leakage, we apply grid-based softening to the mask boundaries. Similarly, generate and align the face mask using the same strategy.
Overall, our proposed Driving Cue, which consists of the Pose Cue (motion), Face Cue (expression), and Location Cue (alignment), not only avoids the limitations of pose estimation but also provides a more robust and semantically rich representation for animation control. Extensive experiments demonstrate its effectiveness in achieving high-quality pose transfer and expressive facial synthesis.
CueFusion DiT Block. To enable the driving cue to effectively guide the denoising process of DiT 2023 Yang al. 2024), introduce the CueFusion DIT (CF-DiT) Block, whose architecture is Ilustrated on Higure 2 (b). With the recent success of transformer diffusion models in video generation (e.g., 1 (Wan al. 2025). Cog (Yang al., (024)), series of methods have adopted DiT as the denoising model for human video generation. These methods (i) concatenating the driving features with the noisy latents as inputs to the DiT block, as in DreamActor-Me (Luo et al., 2025| (ii) injecting the driving features into the vision branch via cross-attention, as in ConsisID Yuaan et al. 2024); (iii) modulating the driving features using adaptive LN before injecting them into the DIT block, as DynamiCtrl (Zhao al. 2025).
(Peebles & Xie
typically incorporate driving signals using one of the following strategies:
Our CF-DIT Block adopts the third strategy, which has proven to be practical approach due to its low computational overhead and effective conditioning. Specifically, we apply adaptive LN to inject the patch embeddings of the pose cue en and face ep into the DiT block. Specifically, we first use the time embedding e, to learn modulation and gating factors for the cues via an MLP:
ap, Sp, Tpp a f5,77 MLP(SiLU(())),
(1)
where Op. Sp, and 7pp are the scale, shift and gating factors for modulating pose features. Similarly, af. B3, and 7 are factors for modulating face features. We then the scale (a) and shift (9) factors to modulate the normalized cues:
c LN((p)) (1 (1+aa)+ +
(2)
LN((e)) (1+ +8,
5

---

Published as a conference paper at ICLR 2026
ID Setting
DIT Block
DVT Block
Beference Image
DVT
Driving Fromes (Same-ID)
HIA &
Same2X Alignment
Driving Prames (Cross-1D)
Poeudo Reforence
?
9901 Haggoouun
DIT Blnck
DV Block
Referencc
Driving
Cross-ID Setting
(a) Same-ID and Cross-ID Settings
(b) Cross-ID Training Stage
Figure 4: (a) Comparison between different settings. In the Same-ID setting, the reference image and driving video share the same identity. In the more practical Cross-II) setting, they feature dif- ferent identities.
(b) Overview of the cross-II) training pipeline. First, a model trained under the Same-ID setting. Then, in the Cross-ID training stage, new model is trained using pseudo driving cues generated from same-ID data to simulate cross-ID conditions. The training is supervised by both standard denoising loss and proposed Same2X Alignment Loss. This S2X loss aligns the feature embeddings of the cross-ID model with those from the pre-trained same- ID model, transfer- ring knowledge from the simpler setting mitigate errors in the more challenging cross-ID task.
The modulated embeddings egd and of are added element- to the text and vision embeddings before entering the 3D full attention layers, preserving the original channel dimension while effec- tively integrating the driving cues into the denoising process. To further ensure the stability and continuity of the driving cue throughout the denoising process, we leverage shorteut connections and gating mechanisms. The initial embeddings ep and ef are combined with the modulated ones via gated residual connections:
c = Ep 7pp Cp cf = ef + 7f ef
(3)
M
This design ensures that each DiT block receives both the raw and modulated driving cues, enhanc- ing controllability of the denoising process.
3.2 SAME2X TRAINING STRATEGY
The HIA task is commonly studied under two settings: same-identity (same-ID) and cross-identity (cross-ID), as shown in Figure 4 (a). While same-ID setting is casier to train and requires less data, cross-ID setting more practical for real-world applications. In the cross-II setting, where the driving and reference identities differ, most methods follow three-step pipeline: (i) extract pose from the driving video, (ii) spatially align with the reference identity, and (iii) use as the driving signal. However, this approach is suboptimal, as both pose extraction and alignment introduce unavoidable errors.
To address this, DirectAnimator bypasses pose extraction by using raw pixels from the driving video as the driving signal. While conceptually simple and flexible, directly training a model from cross- ID data is inherently challenging: the model must first understand the poses and expressions of driving identity, and then transfer them to the reference identity, making the training significantly harder. Inspired by recent advances in training diffusion transformer (e.g.. REPA (Yu et al.) 2024), REPA-E (Leng et al., 2025), and SRA (Jiang et al. 2025)), we propose Same2X training strategy that eases training in the cross-ID setting. Same2X training strategy consists of same-ID and a cross-ID training stage, with the cross-ID's pipeline illustrated in Figure 4 (b). In the same-ID training stage, train the model using pairs of reference images and driving videos from the same video clip. In the cross-ID training stage, we generate pscudo driving cues using driving videos from same-ID stage. Specifically,
niimaor (Tu al. 2024) generate pseudo pose

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparison on the TikTok and Unseen datasets. Bold text indicates the best result, while underlined text indicates the second-best. 1 denotes that higher values are better. FIS and FTS stand for Face Identity Similarity and Face Temporal Similarity, respectively. In the table, a/b denotes results TikTok and Unseen, respectively.
Method
FIDI
SSIM
PSNRT
LPIPS.
LI(E-04))
FIST
FIST
FYDJ
Training Steps
MagicAnimate
12.09/4235 0.714/0435
29.16/ 23.17
0.239/ /0453
3,13/643
0.545 70.522
0.629 0.603
179:07/833.11 /833.10
AnimateAnyone
-/36.49
0,516/0.227
4*A10000KK
Champ
-/34.62
0.802/0.553
/0.53
/0.589
S*A100K8KK
MimicMation
28.03/37667
0.601 /0.578
-/25.53
0.416/03318
5.85 4.91 0.621
/0.563
0.669)
/0.621
326.57 / 433.20
8 A 100k6M
StablcAnimator
-/3..99
0,801 / 0.603 30.81
/2.11 0.232 /02273
2,87 4,39 0.662 /0.653
0.7701
140.62 365.52
4" A100033MM
UniAnimate-DNe
29.992
0.787
0.6619
29.76/
27.89
0.226
3,37 0.643/0.647
0.695
/0.684
306.17
217/299.55
DynamiCnt
-/36.0
0.766
/0633 30.22
27.43
0.166/ 0.266
2.34 3.52 8.63//0.11 0.725
0.7002
152.31 339.80
8*H20650K
DireciAnimator
25.87 27.62 0,806 0.708 /0.12/29.41 0.225
0,249 2,12/ 3,22 0.682 0.682/0.661 0.661 0.730 0,723 142.60 276.34
4*H20=10K
Reference
Skeleton Map Anlmate-Anyon MimicMotion StableAnimator UniAnimateDiT
Pase Oue
DirectAnimator
Figure 5: Qualitative comparisons between DirectAnimator and baselines on the TikTok (Row 1) and Unseen (Row 2) datasets, with visual artifacts highlighted in yellow dashed boxes for clearer comparison. User IDs are obscured for privacy protection.
cues and Face-Adapter (Han et al. 2024 to obtain pseudo face cues. For each reference identity, we generate 0~3 pseudo driving cues simulate cross-ID conditions. For more details on pseudo driving cue generation, please refer to the appendix.
During Cross-ID training stage, the model supervised not only by denoising loss but also by feature alignment signals from the model trained under same-ID setting. To this end, we introduce a Same2X Alignment Loss (S2X Loss) to guide the feature dynamics:
Cszx(Os, 0x) = -E
sim
(1. (D.)) A(D..))
(4)
N
n=1
Here, 0g and 0x denote the model trained under same-ID and cross-ID ID settings, respectively. h! and 12!" w] represent the patch embeddings at the th DiT block for the same-ID and cross-ID models, and indexes the patch tokens. The function sim(-,-) measures cosine similarity. We combine the S2X Loss and denoising loss to train DirectAnimator, with the overall loss function in L Demoissng + XLs2x, where À is factor controlling
the ID training stage formulated the contribution of the S2X loss. As demonstrated in Figure [] the Same2X training strategy sig- nificantly accelerates convergence in the cross-ID setting by alleviating the learning difficulty. To our knowledge, this is the first approach to leverage feature alignment across different settings for training HIA models.
EXPERIMENTS
4.1 EXPERIMENTS SETTING
Implementation Details and Dataset. We construct our model based on CogVideoXIs. (Yang
al. 2024). All parameters of the DiT block updated, while the Text Encoder and VAE Encoder

---

Published as a conference paper at ICLR 2026
Refferencs
Figure 6: Sample results of DirectAnimator. User IDs are manually obscured for privacy protection.
are kept frozen. accommodate video predictions of varying sizes and resolutions, we utilize a bucket sampler during training. The model is first trained for 10K iterations on same-ID setting with a learning rate of 2e-5, followed by 30K iterations on cross-ID setting using the same learning rate. conducted 4 H20 GPUs. All text prompts were generated using Qwen2- VL (Wang et al. (024a) model. Previous works have generally not released training data due to privacy policies. Therefore, collected 4,000 video clips (5~20 seconds) from the internet to train DirectAnimator..
All experiments
addition, the training data includes video clips from the TikToo training set (I~334). All of these data are used in the same-ID training stage. For the cross-ID training stage, we synthesized a large number of pseudo data. After a rigorous filtering process, we retained 3,300 high-quality [reference, pseudo driving cue] pairs for training. To thoroughly evaluate the generation performance of DirectAnimator, we use video clips from the Tok test set (335~340), along with 50 additional video clips collected from the internet (referred set. To ensure fair comparison, our method is conditioned on the pseudo driving cues generated by StableAnimator,, rather than using driving cues that share the same identity as the reference image.
"Unseen"), ), as the test
Baselines and Metrics. We conduct broad comparisons to validate our method's 's superiority. Specif- ically, we compare against SD-based methods including AnimateAnyone (Hu) 2024). MagicAni- mate (Xu et al., 2024), and Champ (Zhu et al. (2024), SVD-based methods such as MimicMotion (Zhang et al.j (224) and StableAnimator (Tu et al.) 2024), and DiT-based methods including Dy- namiCtrl (Zhao et al. 2025) and UniAnimate- DiT (Wang al. (0255). We evaluate image quality using FID (Heusel et al. 2017), SSIM (Wang et al.
2004). LPIPS (Zhang et al. 2018). PSNR (Hore) & Ziou 2010), and LI error. For video quality, adopt FVD (Unterthiner et al. 2018). We use four metrics to evaluate identity, temporal consistency, and motion transfer: Face Identity Similar- ity (FIS) for identity preservation, Face Temporal Similarity (FTS) for temporal consistency, Pose Landmark Consistency (PLC) for body pose following, and Facial Landmark Consistency (FLC) for facial expression and mouth-shape shape transfer. More detailed definitions and implementation de- tails are provided in Appendix A..4 Since DynamiCtrl generates only 2-second clips and is highly sensitive text prompts, we include it in quantitative comparisons only.
4.2 COMPARISON WITH SOTA METHODS
Quantitative Comparison. Table presents the quantitative comparison results. As shown, Di- rectAnimator achieves state-of-the-an animation performance while using fewer computational re- sources. On the TikTTok test set, method obtains the best results in FID, SSIM, LI, and FIS, and also performs competitively in L.IISS, FVD, and FTS, indicating high visual quality and strong identity preservation. On the more challenging Unseen test set, DirectAnimator consistently outper- forms all baselines across all metrics. This dataset, featuring fast-paced and complex dance motions, often leads inaccurate pose estimations, and DirectAnimator maintains robust performance by by- passing pose extraction. We further evaluate pose and expression transfer using PLC and FLC. As reported Table 2 DirectAnimator achieves the lowest landmark errors on both test set, confirm- ing more accurate body pose following and more faithful facial expression transfer than existing methods. Finally, on a single H20 GPU 5122 resolution with 49-frame clips, Table reports an end-to-end runtime comparison with key baselines to illustrate the inference cost of our design.
Qualitative Comparison. Figure 5 presents a qualitative comparison across different methods. In Row 1, the skeleton map produces inaccurate estimation of the driving frame: the subject's left hand and arm should be occluded by the body and therefore not visible in the skeleton. While AnimatcAnyone and MimicMotion successfully generate plausible body structures, StableAnima- tor mistakenly renders an extra left hand. contrast, DirectAnimator not only produces anatom- ically correct results but also better preserves the subject's identity. In Row the skeleton map
8

---

Published as a conference paper at ICLR 2026
Table 2: Pose and facial landmark consis- Table 3: Ablation study for driving cues and Same2X tency on TikTok / Unseen.
training strategy.
Method
PLCL
FLCJ
ID Settings
[FDD,
SSIM;
PSNR
FIST
FVD1
MagicAnimate
0.094/0 / 128
0.058/ /0.079
S1
Face Cue
28.21
0.729
29.13
0.418
245.
AnimateAnyone
0.083/0.1 115
0.09/0.000
52 w Face Cue (no enlargedcentter
28.02
0.742
29.45
0.610
185.93
Champ
/0.134
0.054/0.082
S3 w Pose Cuc fayh)
29.32
0.715
29.13
0.591
191.92
MimicMation
0.081 00.102
0.045/ 0.064
S4 w Pose Cue ignay, no low-pass)
28.10
0.744
29.40
0.625 184.07
StubleAnimator
0.075 /0.097
0.042/ 0.000
S5 w Skeleton Map
29.74
29.01
0.578 216.38
86
w/o Location Cue
30.89
0.682
29.17
0.529
203.22
UniAnimate-DDi
0.082/0 0.2//0.100 102
0.058 / /.0044
57
w Location Cue (lord musk only)
29.13
0.718
29.25
0.582 228.54
DynamiCrrl
0.075/0.116
0.042 0.069
S8
w/o Same2X
32.21
0.691
28.67
0.530 290.43
DirectAnimator (Ours)
0.071/ 0.092
0.037/ 0.057
DirectAnimator (Ours)
27.61
0.752
29.53
0.638
180.52
introduces front-back ambiguity, leading most methods to generate incorrect body orientations. Al- though UniAnimate-DDe generates a correct pose, it fails to model the face direction. In contrast, DirectAnimator, guided by the pose cue, produces accurate results. Figure ( shows multi-frame animation results generated by DirectAnimator, which demonstrate strong identity preservation and background consistency across frames. More results are available in the appendix.
4.3 ABLATION STUDY
All ablation studies were performed on a training data subset, with both same-ID and cross-II) training stage using 500 videos each. The TikTok test set was used for evaluation.
Driving Cues. To improve animation stability and reduce sensitivity to pose estimation errors, we design Driving Cue Triplet consisting of Pose Cue (motion), Face Cue (expression), and Location Cue (alignment). We conduct detailed ablation study to evaluate the impact of the driving cue design on overall animation quality. As shown in Table 3 removing the Face Cue (SI) causes a substantial drop in facial quality: FIS decreases from 0.638 to 0.418 and FVD increases from 180.52 to 44, showing that a dedicated face branch crucial for identity and expression preservation. Using Face Cue without enlarging and centering the detected face (S2) recovers most of the gain, but still lags behind our full design in FIS and FVD, indicating that the enlarged and centered Face Cue yields more consistent facial details. For the body motion signal, replacing our filtered Pose Cue with raw RGB foreground (S3) degrades all metrics, since irrelevant appearance details are introduced into the driving signal. A gray-scale foreground without low-pass filtering (S4) performs better than raw RGB but remains slightly worse than our full Pose Cue, confirming that frequency- domain low-pass filtering further stabilizes pose modeling by suppressing high-frequency textures. Substituting the entire triplet with skeleton maps (S5) yields only moderate results and is consistently inferior to our method in FID, FIS, and FVD, which highlights the advantage of learning from carefully processed pixel cues rather than explicit pose estimators. For spatial alignment, removing the Location Cue (S6) leads to weaker performance, especially in FIS and FVD. Using only a hard foreground mask as the Location Cue (S7) partially improves over the no-Location baseline yet still falls behind our soft, grid-based Location Cue, suggesting that the softened alignment reduces artifacts and identity leakage. These ablations demonstrate that each component of the Driving Cue Triplet contributes meaningfully to robust and high- HIA, and that the specific design choices are important rather than interchangeable.
Table 4: Ablation study for alignment depth D) and regularization coefficient X.
Table 5: Ablation study for the design of CF-DiT block.
Metric|
D=10
D=20
D=30
X=0.1
A=0.5
A=1
Settings
FIDJ
SSIMI
PSNRT
FISt
FVDJ
FID L
27.61
28.41
31.84
27.81
27.61
27.76
DC Injection
30.68
0.682
29.09
0.544
372.80
FIS 7
0.638
0.591
0.503
0.634
0.638
0.627
CA Injection
32,30
0.675
28.85
0.502
453.73
FVD 4
180.52
230.95
423.51
189.72
180.52
191.78
DirectAnimator
27.61
0.752
29.53
0.638
180.52
Same2X Training Strategy. To address the challenge of learning robust cross-identity human image animation, we introduce Same2X training strategy, a two-stage framework that first trains the model on iime-identity data and then adapts to cross-identity samples using a Same2X alignment loss. As shown in Table 3 (S8), removing the Same2X strategy leads consistent degradation across

---

Published as a conference paper at ICLR 2026
Table 6: Effect of pseudo data quality.
Settings
FIDJ
SSIM
PSNRT
FISt
FVD.
Unfiltered pscudo set (3,000)
29.93
0.712
29.17
0.584
218.30
MimicMotion-onl), pseudo eues (500)
28.17
0.739
29.43
0.635
191.00
Stuble-nimator-onll pseudo cues (500)
28.22
0.745
29.40
0.630
187.40
Filtered pseudo set (Ours) (500)
27.61
0.752
29.53
0.638
180.52
Table 7: Inference time comparison on an H20 GPU at 512" resolution (49 frames).
Method
Preproc
Backbone
Preproc (ms)
Gen (s)
AnimateAnyone
DWpose
SD
9.6
102.59
Stable/nnimtor
DWpose
SVD
9.6
91.63
DirectAnimator (Ours)
G-SAM LPF
Cod
31
152.80
all metrics, including an increase in FID (+4.6), noticeable drop in FIS (from 0.638 to 0.530), and a substantial increase in FVD (+110). These results underscore the contribution of Same2X training strategy during cross-ID training stage. We further conduct ablation studies on two key hyperparameters: alignment depth (D) and regularization coefficient (A), As shown in Table 4 increasing the alignment depth (e.g.. D=20 and D=30) leads significant performance drops. This aligns with REPA (Yu et al., 2024), indicating that shallow blocks capture geometric structure, while deeper blocks focus on high-freyuency details. Applying the Same2X alignment loss to shallow blocks enhances regularization without hindering fine-grained learning in deeper blocks. We also evaluate the impact of X. As the results show, the model relatively robust to different values X, and adopt À 0.5 the default setting. Overall, these ablation studies validate the effectiveness and efficiency of the Same2X training strategy.
CueFusion DiT Block. To validate the effectiveness of proposed CueFusion DiT (CF-DiT) Block, we conduct an ablation study comparing it against two commonly used injection strategies: (i) DC Injection, which Directly Concatenates the driving features with the noisy latent inputs to the DiT block; (ii) CA Injection, which injects the driving features into the vision branch via Cross- Attention. As shown in Table 5 our CF-DiT Block consistently outperforms both DC and CA Injection across all metrics. This confirms the advantage of our design in effectively integrating driving cues into the denoising process.
Pseudo Data Quality and Generator Choice. To further understand the impact of pscudo driv- ing cues on Cross-ID training. we conduct an ablation study on pseudo data quality and generator choice. As shown in Table 6 using large unfiltered pseudo set of 5,000 samples leads to noticeably worse performance compared to our filtered setup, indicating that simply increasing the amount of pseudo data without quality control can hurt both generation fidelity and temporal coherence. When training with 500 pscudo cues from a single source, both MimicMotion-onll and StableAnimator- only settings improve over the unfiltered baseline and achieve similar performance, suggesting that the model is not overly sensitive to the choice of pscudo generator and mainly relies on the shared motion and expression patterns. Our Filtered pseudo set achieves the best results across all metrics, demonstrating that our filtering strategy effectively controls pseudo bias while retaining the benefits of pseudo-driven training.
CONCLUSIONS
We proposed DirectAnimator, HIA framework that directly learns from driving videos without relying on pose estimators. By introducing a structured driving cue triplet and integrating it through the CueFusion DiT block, our method achieves high animation quality with improved robustness. To address the challenge of cross-ID training, we designed the Same2X strategy, which aligns internal representations between same-ID and cross-ID settings to guide learning. Extensive experiments confirm that DirectAnimator outperforms prior methods in visual quality and identity preservation. While DirectAnimator eliminates the need for pose estimators and curated annotations, limitations like pseudo cue quality, and explicit pose supervision remain. Future work aims to tackle these issues, paving the way for more intuitive and human-level animation learning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We thank the reviewers for the thoughtful discussion and feedback. This work was supported by the National Natural Science Foundation of China (U23B2030, Nos. 62311530100 and 62171251) and the Special Foundations for the Development of Strategic Emerging Industries of Shenzhen (No. KJZDD202310230047000000
REFERENCES
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models large datasets. arXiv preprint arXiv:231..51127, 2023.
Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7291-7299, 2017.
Di Chang, Hongyi Xu, You Xie, Yipeng Gao, Zhengfei Kuang, Shengqu Cai, Chenxu Zhang, Guox- ian Song, Chao Wang. Yichun Shi, et al. X-dyna: Expressive dynamic human image animation. arXiv preprint arX1022501.0022, 2025.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4690-4699, 2019.
Qijun Gan, Yi Ren, Chen Zhang, Zhenhui Ye, Pan Xie, Xiang Yin, Zehuan Yuan, Bingyue Peng, and Jianke Zhu. Humandit: Pose-guides guided diffusion transformer for long-form human motion video generation, arXiv preprint arXiv:2502.00 048477 2025.
Rıza Alp Güler, Natalia Neverova, and lasonas Kokkinos. Densepose: Dense human pose estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7297-7306, 2018.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffu- sion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023.
Yue Han, Junwei Zhu, Keke He, Xu Chen, Yanhao Ge, Wei Li, Xiangtai Li, Jiangning Zhang, Chengjie Wang. and Yong Liu. Face-adapter for pre-trained diffusion models with fine-grained id and attribute control. In European Conference on Computer Vision, pp. 20-36. Springer, 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Alain Hore and Djemel Ziou. Image quality metrics: Psnr ssim. In 2010 20th international conference on pattern recognition, pp. 2366-2369. IEEE, 2010.
Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character anima- tion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8153-8163, 2024.
Dengyang Jiang, Mengmeng Wang, Liuzhuozheng Li, Lei Zhang, Haoyu Wang, Wei Wei, Guang Dai, Yanning Zhang, and Jingdong Wang. No other representation component is needed: Diffusson transformers can provide representation guidance by themselves. arXiv preprint arXiv:2505.02831, 2025.
Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint ar5iv:2504. 10483, 2025.
11

---

Published as a conference paper at ICLR 2026
Tao Liu, Feilong Chen, Shuai Fan, Chenpeng Du, Qi Chen, Xie Chen, and Kai Yu. Anitalker: animate vivid and diverse talking faces through decouuldee facial motion encoding. In Proceedings of the 32nd ACM International Conference on Multimedia, pp.
2024.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 851-866. 2023.
Yuxuan Luo, Zhengkun Rong, Lizhen Wang, Longhao Zhang. Tianshu Hu, and Yongming Zhu. Dreamactor-ml: Holistic, expressive and robust human image animation with hybrid guidance. arXiv preprint arXiv:2504.01224 2025.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafranice, Vasil Khalidov, Pierre Fermandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.071 193, 2023.
Georgios Pavlakos, Dandan Shan, Iijaa Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3d with transformers. In Proceedings of the IEEE/CVF Confer- on Computer Vision and Pattern Recognition, pp. 9026-982688 9836-983,, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 4195-4203 2023.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with general conditioning layer. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open- spen-world models for diverse visual tasks. arXiv preprint 1eXi::2401.141555 2024.
Robin Rombach, Andreas Blattmann, Dominík Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022a
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10000 10695, 2022b.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint ar arXiv:2010.02502. 2020.
Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stableanimator: High-quality identity- preservin human image animation. arXiv preprint arXiv:2411.1769) 2024.
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: new metric & challenges. arXiv preprint arXiv:1812.01717, 2018.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. 2025.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing Aisionllangaag model's 's perception of the world at any resolution. arXiv preprint 240.::2409.12111 2024a.
Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Chinn Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- 2024b.
nition, pp.
12

---

Published as a conference paper at ICLR 2026
Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion models for consistent human image animation, 2024c. httptttt//aaaat http:///xxxvvv...gaaaa.. htrgg//bs/2a0
01188
Xiang Wang, Shiwei Zhang, Longxiang Tang, Yingya Zhang, Changxin Gao, Yuehuan Wang, and Nong Sang. Unianimate-dit: Human image animation with large-scale video diffusion trans- former. arXiv preprint arXiv:2504. 11289, 2025.
Yaohui Wang, Di Yang. Francois Bremond, and Antitza Dantcheva, Latent image animator: Learn- ing to animate images via latent space navigation. arXiv preprint arXiv:2203.09044 2022.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(:6000 612, 2004.
You Xie, Hongyi Xu, Guoxian Song, Chao Wang, Yichun Shi, and Linjie Luo. X-portrait: Expres- sive portrait animation with hierarchical motion attention. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Zhongcong Xu, Jianfeng Zhang. Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1481-1490, 2024.
Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effective whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4210-4220, 2023.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint wwxXv:2440800677. 2024.
Sihyun Yu, Sangkyung Kwak, Huiwon Jang. Jongheon Jeong. Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06944 2024.
Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge. Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan. preaetippreeriiigggggg text-to-video generation by frequency decomposition. arXiv preprint arXiv:2411. 1. 17440, 2024.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018
Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang. Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mimicmotion: High-quality human motion video generation with confidence-aware pose guid- ance. arXiv preprint arXiv:2406. 196000 2024.
Haoyu Zhao, Zhongang Qi, Cong Wang, Qingping Zheng, Guansong Lu, Fei Chen, Hang Xu, and Zuxuan Wu. Dynamictrl: Rethinking the basic structure and the role of text for high-quality human image animation. arXiv preprint arXiv:2503.2 2025.
Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. 2024.
arXiv preprint ar arXiv:2412.2044
Jingkai Zhou, Benzhi Wang, Weihua Chen, Jingqi Bai, Dongyang Li, Aixi Zhang, Hao Xu, Mingyang Yang, and Fan Wang. Realisdance: Equip controllable character animation with real- istic hands. arXiv preprint arXiv:2409.06202. 2024.
Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. arXiv preprint 200 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
In this appendix, we first present the foundational concepts and diffusion-based architectures in Section A.T] Section A.2 then provides an in-depth description of our Driving Cue representation, including the effect of low-pass filtering on pose cues, how spatial aligmment is learned from pseudo cues, and the foreground-based spatial alignment used inference time. In Section A..3 we elab- orate on the Same2X training strategy, including the generation and utilization of pseudo driving cues. Section A.4 introduces our pose and expression evaluation metrics. Section AA.5 presents ad- ditional visualizations, including more qualitative comparisons and further animation results, while Section A.6 highlights representative failure cases. Finally, Section A.7 discusses the current limi- tations of our method and outlines promising directions for future work.
A.1 PRELIMINARIES
Diffusion Models. Diffusion probabilistic models have emerged as a powerful class of generative models, framing data generation the reversal of gradual noise-adding process (Ho et al. 2020). These models define a forward process q( q(#1r1120) that incrementally corrupts a clean data point Zo with Gaussian noise over T discrete steps, followed by a reverse process (x that learns to denoise the noisy inputs back to the original data distribution. In the forward diffusion process, each step defined as:
(5) where 3r is the noise variance schedule at time I. This forms Markov chain from æo to ær. with at approaching an isotropic Gaussian distribution as + T. For computational efficiency, we can directly sample noisy version H; any time t from 2o using the closed- form expression:
((x,(.,,11 = (a v1- -, 3,T),
Tz Varao + VI- E e~ I),
where ā. I1-- B.) is the cumulative product of noise factors up step t. The reverse process is parameterized by a neural network, often a time-aware U-Net, which estimates the mean and variance of the posterior pe (
Pa ( xx)= = (( t), Ee (2x,,)) t)).
In practice, the model is trained to predict the noise e added at each step, using the simplified loss:
Emu., [||e cs t)]?
Csimple
where ce(-) predicts the noise component in the sample given the timestep t. To enable condi- tional generation, diffusion models can be extended with context vectors e (e.g., text embeddings, segmentation masks), modifying the noise prediction to (xt c). This enables controllable gen- eration while maintaining sample diversity. Despite their strong generation quality, diffusion models typically require hundreds of denoising steps for high-fidelity samples, leading to slow inference. To address this, several acceleration methods have been proposed, such as DDIM (Song et al., (020) deterministic sampling) and fast samplers based on knowledge distillation noise schedule truncation.
Diffusion Transformers. Diffusion Transformers (DiT) (Peebles & Xie 2023) present fully
Transformer-basee backbone for diffusion models, replacing the conventional convolutional U-Net architecture. Built upon the latent space framework of Stable Diffusion (Rombach al., 2022b). DiT processes image representations encoded by a fixed VAE encoder into low-dimensional fea- ture maps. These latent tensors are segmented into non-overlapping patches and transformed into sequences of tokens via linear projection. Each token is enriched with temporal embeddings to rep- resent the noise schedule step, along with optional conditioning inputs (e.g., class labels). The model architecture comprises a stack of Transformer layers equipped with standard attention mechanisms, feed-forward networks, and normalization, all interleaved with residual paths. These components work together to model long-range dependencies and contextual relationships among latent patches. After traversing the Transformer pipeline, the output tokens are mapped back to the latent grid for- mat, producing both noise and variance predictions. This dual- head output is then used to iteratively refine the denoising process across multiple diffusion steps. The design demonstrates strong scala- bility and performance.
14

---

Published as a conference paper at ICLR 2026
CogVideoX Model. Recent advances in diffusion-basee generative models have significantly pushed the boundaries of video synthesis (Yang et al. 2024; Wan et al. 2025 Zheng et al. (024).
Among these, CogVidcoX (Yang et al.] (024) emerges scalable and tigh--pefforing diffusion Transformer architecture tailored for long-duration, text-conditioned video generation. Built upon the Diffusion Transformer (DiT) backhone (Peebles & Xie 2023), Cog VideoX integrates several critical innovations that address longstanding challenges in temporal coherence and cross-modal alignment. To efficiently encode the spatio-temporal redundancy in videos, Cog VogVideoX adopts a 3D Causal Variational Autoencoder (VAE) which compresses video clips across both spatial and tem- poral axes, enabling tractable sequence lengths for Transformer modeling. Unlike prior works that employ frame-wise 2D VAEs, this 3D design notably improves motion continuity and reduces flick- ering artifacts. To bridge the semantic gap between text prompts and video content, Cog VideoX introduces an Expert Transformer module, incorporating modality-speciff adaptive LayerNorms that allow the model to conditionally modulate text and video representations. This design not only enhances alignment but also mitigates scale disparities in multimodal embeddings. Moreover, Videox utilizes a 3D full attention mechanism to jointly capture spatial and temporal depen- dencies, in contrast to factorized attention used in prior approaches (Guo et al. 2023). thus better preserving global scene dynamics.
Training VideoX involves a progressive paradigm including mixed-duration training, resolu- tion scaling, and an innovative technique called explicit uniform sampling to stabilize loss conver- gence across diffusion timesteps. In addition, comprehensive video-text data pipeline leveraging dense video captioning contributes to better semantic grounding. Empirical results demonstrate that Cog VideoX outperforms previous state-of-the-ar models across multiple automated metrics and human evaluations, particularly excelling in generating coherent, high- figh-fideiity, and instruction- following videos.
A.2 MORE DETAILS FOR DRIVING CUE
In the main paper, we introduced a novel Driving Cue representation that replaces conventional pose estimation with three complementary and semantically disentangled signals: Pose Cue, Face Cue, and Location Cue. In this section, we provide detailed descriptions of the data processing pipelines used generate each cue from raw driving videos. Additionally, we include more visual examples to illustrate the driving cue.
To facilitate learning of pose-related information, we first remove background from the driving video. Specifically, we employ the Grounded SAM model (Ren et al.) 2024) to segment out the foreground human subject. To further suppress redundant information, we apply a low-pass filter in the frequency domain to eliminate high-frequency image details. The resulting foreground image is used the Pose Cue. While most prior methods adopt 68 facial landmarks as the driving signal for expression transfer, such sparse representations often fail to capture complex facial expressions ac- curately, making it difficult to faithfully transfer expressions from the driving video to the reference image. Inspired by X- X-Dyna (Chang et al., 2025), we instead propose directly use the raw facial image as a more expressive input. However, in many driving frames, the face occupies only a small region of the image. To ensure the model can access high-resolution facial features, we utilize the InsightFace to detect the face and then enlarge and center it. The resulting image is used as the Face Cue. The generated pose cue and face cue are not aligned to the spatial position of the reference identity, which may hinder the model's training process. To address this, we introduce an auxiliary signal for pose and face alignment. Specifically, we apply the alignment strategy from StableAnima- tor (Tu et al.) 2024) to the pose and face masks in the driving video, aligning their spatial layout and scale with that of the reference identity. To prevent potential information leakage during training, we further apply grid-based softening operation on the pose mask blurring the mask boundaries while retaining the coarse silhouette. These aligned pose and face masks together form the Location Cue.
Effeet of Low-Pass Filtering on Pose Cues. To better illustrate the effect of the low-pass filter, Figure 7 visualizes a foreground frame before and after filtering in both the spatial and frequency do- mains. We first convert the segmented RGB frame to grayscale, which removes chromatic informa- tion but preserves most structural and textural details. Applying the low-pass filter in the frequency domain smooths fine textures on the hair and clothing while maintaining the global silhouette and coarse shading. This can be clearly seen from the zoomed-in patches, where high-frequency wrin-
15

---

Published as a conference paper at ICLR 2026
kles and fabric patterns noticeably attenuated in the filtered result. The uog-magnitude Fourier spectra of the grayscale and filtered images further confirm this behavior: the filtered spectrum exhibits strong concentration of energy around the low- www-frquencyy center and substantial reduc- tion of high-frequency components. These observations show that the pose cue indeed suppresses figh-frequency appearance details while preserving the body structure required for motion control.
Filtered
Color
Gray
Filtered Gray
Gray Spectrum
Spectrum
From left to right: original foreground color frame, grayscale frame with zoomed-in local patches, low-pass filtered grayscale frame with corresponding patches, and the log-magnitude Fourier spectra of the grayscale and filtered images. The spatial zooms reveal that fine hair and clothing textures are smoothed while the global body shape preserved, and the spectra show that high-frequency components are strongly attenuated after low-pass filtering.
Figure 7:
Learning Spatial Alignment from Pseudo Cues. To learn robust spatial alignment, we do not directly use the raw outputs of StableAnimator Tu al. (2024) or MimicMotion Zhang et al.) (2024) as driving cues. First, we generate spatially aligned pseudo driving videos by selecting pseudo reference images whose body scale and location are close to the original driving videos, ensuring that these methods produce high-quality motion with only mild or no pose alignment. Then, af- ter foreground segmentation, we apply additional scaling and translation to the segmented person to synthetically create spatially misaligned pseudo pose cues, while the Location Cue (pose and face masks) is extracted from the original driving video and remains aligned with the reference im- age. Training on such pairs of misaligned pseudo pose cues and aligned Location Cues explicitly teaches DirmctAnimator to recover spatial alignment from the Location Cue, rather than inheriting any alignment behavior from StableAnimator or MimicMotion.
Foreground-basse Spatial Alignment at Inference Time. Inspired by the global similarity align- ment in StableAnimator Tu et al. (2024) align the driving sequence to the reference image di- rectly at the foreground -maso level. without relying on skeleton keypoints, Let M E {0, 1] 1] xW be the binary foreground mask of the reference image and M, E {0, 1 be the foreground mask of the 1-th driving frame. For each mask M, we compute the tight person bounding box
B Tmin, Tmax Vmins Vmax
and define its center and height as
c(B) = ((Imin + fmaa)/2, (ymin + Vmmx)) /2) h(B)- Vmax Ymin
Denote by Bref the bounding box of Mref and by B, the bounding box of M,, with centers cee c(B and c, = c(Bt), and heights he = hi h(rref) and h, (...). We first estimate global isotropic scale by matching the person height over the whole driving sequence:
he
81
8 = median,(st),,
(9)
h
and then align the centers by a single 2D translation vector E R.?:
Σ Ct2 t= Je CC
(10)
This yields global similarity transform in the image plane,
= t. x RER
(11)
16

---

Published as a conference paper at ICLR 2026
which is shared by all driving frames. We obtain the aligned foreground masks M, by warping each Mr with the inverse transform T-1:
(x)x) = (x)) Yx
W) x (I.....,1) H).
(12)
In other words, the foreground person region in each driving frame is resized and shifted that its location and scale are spatially aligned with the reference foreground region, while keeping single, temporally stable similarity transform for the entire driving sequence.
A.3 MORE DETAILS FOR SAME2X TRAINING STRATEGY
The pseudo driving cues play central role in the cross-ID IDD training stage. They enable us to simulate diverse driving conditions for a given reference identity without relying on manually annotated or true cross-ID pairs. By leveraging motion and expression signals extracted from same-ID videos, these pseudo cues provide semantically meaningful supervision while avoiding identity leakage. In this section, we detail the process for generating pseudo driving cues used in the cross-II) training stage of our Same2X strategy. We generate two types of cues: pseudo pose cues and pseudo face cues.
To generate pseudo pose cues for the cross-ID ID training stage, we leverage StableAnimator (Tu et al, 2024) and MimicMotion (Zhang al. 2024). two of the most competitive human image animation methods to date. In addition to the data used for same-ID training. we collect an extra set of 1,000 images featuring diverse identities as the pseudo reference set. For each driving video sampled from the same-ID training set, we randomly select 0 to 3 images from the pseudo reference set as reference images. Using these pairs, we generate animated videos via the above animation models, followed by manual filtering to ensure quality. The filtered results serve as pseudo driving videos. We then apply foreground segmentation and low-pass filtering to obtain the corresponding pseudo pose cues. Each pseudo driving cue thus preserves the same motion pattern as the original driving video but appears with a different identity and background. These samples can be interpreted as synthetic cross-ID driving signals for training. enabling the model to simulate cross-ID scenarios during training.
However, both Stable/nnimator and MimicMotion rely skeleton maps drive animation, which often results in inaccurate facial expressions compared to those in the original driving video. To address this limitation and obtain higher-fidelity facial dynamics, we adopt the face reenactment method Face-Adapter (Han et al., 2024) to generate candidate facial animations. We then select the most visually consistent and expressive results for further processing. Face reenactment aims to transfer facial expressions and head motion from a source video to target identity, while pre- serving the target's appearance and identity traits. In our case, pseudo reference images are used as target identities, and the driving videos serve source inputs. We first crop the face regions from both inputs and apply Face-Adapter to transfer expressions from source to target. The result- ing face regions are then centered and enlarged to construct the pseudo face cues, which provide complementary supervision for facial animation in the cross-ID training stage.
For the Location Cue, we directly reuse the cues generated during the same-ID training stage. This is feasible because, in the cross-ID training stage, the reference images are sampled from the same- ID training data, and the driving signals (pscudo driving cues) are synthesized using driving videos from the same-ID stage. As result, the pose and face alignment remain perfectly consistent, allowing us to reuse the same-ID location cues without additional processing. Figure 8 shows visual examples of the generated pseudo driving cues.
A.4 POSE AND EXPRESSION EVALUATION METRICS
To comprehensively evaluate pose and facial expression transfer, we combine identity-based met- rics with aandmark-based geometric metrics. Face Identity Similarity (FIS) measures how well the identity of the reference image is preserved in the generated images. Concretely, we extract face em- beddings from generated and reference images using ArcFace (Deng et al. 2019) and compute the average cosine similarity between corresponding pairs. Face Temporal Similarity (FTS) measures how temporally consistent the facial appearance remains within generated video. We compute face embeddings for each frame using ArcFace and average the cosine similarity between embeddings of adjacent frames. Pose Landmark Consistency (PI.C) measures how closely the generated body pose
17

---

Published as a conference paper at ICLR 2026
Pseudo Reference Image
Pscudo Driving
Pscudo
Pseudo Face Cue
Location Cue (Pase&Face)
Driving
Pseudo Driving Frame
Pseudo Pose Cur
Location Cue (Pose&Face)
Reference Image
Face Cue
Driving Frame
Figure 8: Examples of pseudo driving cues.
follows the driving pose. On the TikTTok and Unseen test sets, we extract body keypoints from both the driving and generated vidcos using DWpose, and compute the normalized distance between corresponding body landmarks. Facial Landmark Consistency (FLC) measures how accurately fa- cial expressions and mouth shapes transferred from the driving video. Similarly, extract 2D facial keypoints from driving and generated frames and compute the normalized distance between corresponding facial landmarks.
Taken together, FIS/FTS and PLC/FLC offer a complementary view of model behavior: the former focuses identity preservation and temporal smoothness, while the latter directly evaluates the
18

---

Published as a conference paper at ICLR 2026
geometric accuracy of pose and facial expression transfer. We report PLC and FLC in Table 2 and use them, alongside FIS/FTS and qualitative visualizations, to assess DirectAnimator on both TikTok and Unseen.
A.5 MORE VISUALIZATION RESULTS
More Qualitative Comparison. In Figure 9 we present more comprehensive comparison results, where the artifacts produced by baseline methods are highlighted with yellow bounding boxes for clarity. Specifically, in the first row, MimicMotion /Zhang et al. (2024) fails to capture fine facial de- tails, while StableAnimator Tu et al. (2024), driven by an erroneous skeleton map, produces incorrect hand structures. In the second row, both MimicMotion and Stimle generate unnatural wrist structures, with wrist contours extending beyond the forearm. By contrast, our method faithfully reproduces anatomically correct wrist details. Moreover, StableAnimator exhibits temporal incon- sistency in accessory generation: the bracelet is absent in the second row but reappears in the third row. In the fourth row, our approach achieves superior identity preservation, producing faces more consistent with the reference image. The fifth and sixth rows show that nearly all baselines struggle with body rotations, manifesting structural errors and poor temporal coherence. We attribute this to the inherent limitations of skeleton map-based driving signals. In contrast, our method leverages the newly proposed driving cues, enabling more stable generation under such challenging scenarios. Finally, the seventh row demonstrates the stronger expression transfer capability of our model, with generated facial expressions that align more closely with those in the driving frames.
These comparisons clearly demonstrate that the proposed DirectAnimator outperforms existing methods across multiple dimensions, including structural fidelity, identity preservation, and tem- poral consistency. In particular, our method exhibits superior robustness and realism in scenarios involving complex body poses and expression changes, providing a more reliable solution for pose- free human image animation.
More Animation Results. Figure I0, we present the animation results of DirectAnimator, The first row demonstrates the model's capability in pose alignment: even when the driving frame and the reference image differ significantly in scale and position, DimectA is able to generate high-quality animations. The second row highlights the strength of identity preservation, where the generated video frames maintain strong consistency with the reference image in the facial region and avoid distributional biases, faithfully retaining the phenotypic traits of the subject. The third row illustrates the model's advantage in expression transfer, as the generated animations closely replicate the fine-grained facial expressions of the driving frame. Finally, the fourth row shows that DirmctA provides more robust modeling of complex motion structures: the generated video frames preserve the correct occlusion relationships between the two arms, whereas skeleton map-based approaches often confuse such relationships and degrade animation quality. These results demonstrate that DirectAnimator consistently achieves superior performance across key dimensions, including pose alignment, identity preservation, expression transfer, and complex motion modeling.
A.6 FAILURE CASES
We visualize and analyze several failure cases of DirectAnimator under challenging conditions. The corresponding videos are available mild motion blur, the generated frames often lose fine details in the affected regions. This issue is especially pronounced in hand articulation, shown in Case 1 of Figure 11] Second, when the driv- ing video exhibits severe motion blur to the extent that human structure becomes ambiguous, such as during fast gymnastic movements, the generated animation may produce anatomically incorrect poses, as illustrated in Case 2. Third, low video quality also degrades performance. For example, poor lighting conditions as in Case 3(1) or low spatial resolution as in Case 3(2) make it difficult to accurately infer the subject's motion, resulting in noticeably lower animation quality.
project page. First, when the driving video contains
A.7 LIMITATIONS AND FUTURE WORK
While DirectAnimator demonstrates strong performance across various benchmarks, several limi- tations remain. First, since raw pixels from the driving video the driving signal, motion
19

---

Published as a conference paper at ICLR 2026
blur in the input frames affect the model's ability to learn clean and precise features. As sult, the model may reproduce motion blur artifacts the generated outputs, particularly under fast or low-light conditions. Second, the visual fidelity is still bounded by the capacity of the under- lying generative backbone. In some challenging cases, we occasionally observe imperfect identity preservation and noticeable artifacts, especially for fine-grained texture details. Third, method still relies on an explicit location cue to guide pose alignment during training, In contrast, humans naturally learn motion patterns through implicit alignment without requiring explicit location cues. Inspired by this observation, future work will explore removing the explicit pose alignment step. By improving training paradigms and incorporating larger and more diverse datasets, we aim to enable the model to learn implicit pose alignment in end-to-end fashion, bringing HIA methods closer to human-level reasoning. future work, we also plan enhance the pseudo cue generation process through more robust motion synthesis and automated quality filtering. Improving hand ani- mation fidelity is another priority, potentially by integrating specialized hand pose estimators. Since DirectAnimator and Same2X model-agnostic, another promising direction is to combine them with larger and higher-resolution video generation backbones, which we expect will further improve texture sharpness, clothing details, and identity preservation.
B ETHICS STATEMENT
This work focuses advancing human image animation through diffusion-bart models. All train- ing data were collected from publicly available sources, with user identities anonymized or obscured ensure privacy protection. No personally identifiable information sensitive attributes were used. Our method designed for applications such as virtual avatars and creative content generation. Nev- ertheless, we acknowledge the potential for misuse producing misleading or non-consensual con- tent. To mitigate these risks, we encourage responsible use aligned with the ICLR Code of Ethics, and we explicitly discourage any application of our model that infringes upon personal rights or violates legal and ethical standards.
C REPRODUCIBILITY STATEMENT
We have taken multiple steps to ensure reproducibility of our work. The main paper and appendix describe the model architecture, Driving Cue construction, and Same2X training strategy in detail. Implementation details, datasets, baselines, evaluation metrics, and ablation studies are also pro- vided. In addition, we include the source code in the appendix files and provide an anonymous project page facilitate independent verification of our results.
D LLM USAGE STATEMENT
In this work, GPT-5 was utilized tool for language refinement. Specifically, the model was employed to improve the clarity, coherence, and readability of the manuscript. While GPT-5 played important role in enhaneing the presentation of the text, it was not involved in the ideation, perimental design, or data analysis processes. All content generated by the model was thoroughly reviewed and revised ensure adherence to academic integrity and the originality of the work.
20

---

Published as a conference paper at ICLR 2026
Reference
Skeleton Map
Animute Anyone
MimicMotion
Stable-
UniAnimate-DiT
Pose Cuee
DirectAnimator
Animator
(0urs)
MirroMition
POZKAR
(1)
Ori Crwinghhos
NimrcMittoo
SOOCIAAY
UNIAN
PoaeCUs
Siracake mater
(2)
DrideyPore
MiricMectan
(4)
AiriiMistton
PoeeCLe
66)
Criingg Foee
Mimicirction
PoeeCLe
Figure 9: Qualitative comparisons with baseline methods, highlighting artifacts and showing the superiority of DirectAnimane in structural fidelity, temporal consistency, and expression transfer.
21

---

Published as a conference paper at ICLR 2026
(1)
22)
33)
44)
Figure 10: Animation results of DirectAnimator, demonstrating (1) pose alignment, (2) identity
preservation, (3) expression transfer, and (4) complex motion modeling.
22

---

Published as a conference paper at ICLR 2026
Reference
Care1
Case?
CuseJ (1)
Cuse
Figure 11: Failure cases of DirectAnimator. Case shows loss of detail under mild motion blur, particularly in hands. Case 2 demonstrates structural distortion when motion blur obscures body configuration. Case includes two subcases: inaccurate motion transfer under poor lighting (3(1)) and under low resolution (3(2)).
23


Reviews:
[{"review_id": "vGqv4D2gGO", "reviewer": "Reviewer_8a4X", "summary": "This paper presents DirectAnimator, a human image animation framework that bypasses explicit pose estimation by directly learning from raw driving videos. The method introduces a Driving Cue Triplet (pose, face, location cues) integrated via a CueFusion DiT block, and a Same2X training strategy that aligns cross-identity features using same-identity representations. Experiments demonstrate state-of-the-art performance on TikTok and Unseen datasets in visual quality and identity preservation.", "questions": "- How does the model perform when the driving video contains heavy motion blur or low resolution, given that the pose cue relies on segmentation and filtering?\n\n- What are the limitations of this work, are there any failure cases analyses?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ This paper is well written and easy to follow.\n\n+ Directly using raw video pixels as driving signals effectively avoids error accumulation from pose estimation in the Human Image Animation (HIA) domain.\n\n+ The Same2X alignment loss significantly accelerates cross-identity convergence (6.7× faster) and enhances generalization compared to previous work like REPA.\n\n+ Experimental results show DirectAnimator outperforms recent SOTA methods across multiple metrics (FID, FIS, FTS, FVD).", "weaknesses": "- While innovative in full-body HIA, the paper overlooks prior face animation works that already bypass explicit pose estimation, such as LIA [1*] which uses latent space motion trajectories, and AniTalker [2*] which employs identity-disentangled motion representations. These methods demonstrate alternative \"pose-free\" approaches that should be discussed.\n\n- Components of the Driving Cue Triplet build on existing ideas, face cropping resembles X-Dyna [3*], and foreground segmentation has been used in prior video generation works.\n\n- Cross-ID training relies on pseudo cues from StableAnimator and Face-Adapter, potentially inheriting their biases without sufficient analysis of error propagation.\n\n- Cross-dataset evaluation on diverse real-world benchmarks (e.g., UCF101, FineGym) with complex motions is missing and limits claims of generalizability.\n\n- Analysis of inference speed compared to skeleton-based methods is missing, despite the additional preprocessing costs of segmentation and filtering.\n\nReferences:\n\n[1*] Learning to Animate Images via Latent Space Navigation. ICLR, 2022.\n\n[2*] AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding. MM, 2024.\n\n[3*] X-Dyna: Expressive Dynamic Human Image Animation. CVPR, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XCI4JZyMpV", "reviewer": "Reviewer_ebkb", "summary": "This paper introduces a novel method for human animation, with its primary contribution being an alignment training approach that facilitates the disentanglement of motion and identity (Same2X training strategy). Furthermore, to enable fine-grained motion transfer including detailed elements such as facial expressions, the method incorporates several carefully designed visual cues (face cue, location cue and pose cue). Comprehensive experiments validate the effectiveness of the proposed approach.", "questions": "See the weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "+ The idea to disentangle the motion and the idnetity in the feature space by distillation is interesting.\n\n+ The experimental results show that the proposed method is promising. The videos presented in the supplementary material are SOTA.", "weaknesses": "+ I appreciate the insight behind the Same2X training strategy, though I have concerns regarding the reliability of the pseudo data used to support the cross-identity training process.\n\n+ while the transfer of facial expressions is highlighted as a key contribution in the Introduction (Lines 62–65), there appears to be no dedicated metric evaluating pose or facial landmark accuracy. It is important to note that identity preservation (measured by FIS and FTS) is not equivalent to expression or pose transfer. Therefore, relying solely on FIS and FTS is insufficient to fully validate this aspect of the method.\n\n### Some suggestions\n\n+ The contribution of the paper is not clearly stated (this is also one of the major concern). Although the method demonstrates strong performance, the novelty remains unclear. The introduction of visual cues seems more like an engineering refinement rather than a core research contribution. It would be helpful to explicitly distinguish what fundamentally makes this approach apart from prior human animation methods.\n\n+ The presentation could also be improved. The connection between the proposed method (Lines 74–90) and the challenges outlined earlier (Lines 60–73) is not well corresponded. The Introduction section would benefit from further refinement to better link the **problem statement** with the **proposed solution**.\n\nI will increase the score if the presentation is improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a5tk2KLSIx", "reviewer": "Reviewer_e8ax", "summary": "This paper introduces DirectAnimator, a framework for Human Image Animation (HIA) that seeks to overcome the instability and limited expressiveness of traditional skeleton-based driving signals. The core contribution is the replacement of explicit pose estimators with a Driving Cue Triplet (Pose Cue, Face Cue, and Location Cue) derived directly from raw driving video frames. The method also proposes a Same2X training strategy to stabilize cross-identity animation by aligning internal feature representations. The authors report state-of-the-art performance on several quantitative metrics. Despite the promising results, the methodological concerns outlined in the weaknesses section are fundamental and require a major overhaul and deeper analysis.", "questions": "1. Please provide examples and analysis of the specific failure modes of DirectAnimator, particularly under conditions where the input segmentation model (Grounded SAM) produces poor results (e.g., severe occlusion, complex background, or extreme motion blur).\n2. Could you provide a clear and obvious sample of the spatial mismatch challenge, such as a case with a large-scale driving video paired with a small-scale reference (source) image? My understanding is that, during the cross-ID training stage, the pseudo driving cues generated by the StableAnimator are designed to follow the body shape of the original driving video (which shares the identity of the reference image). If this is the case, how does this process fundamentally address the spatial alignment problem? If the StableAnimator already effectively resolves the spatial mismatch problem, how does the proposed method (DirectAnimator) differ, and is its primary function essentially a distillation of the StableAnimator architecture or its capabilities?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The idea of replacing sparse, error-prone keypoint estimation with a dense, segmented, and filtered foreground (Pose Cue) is a clever and potentially more robust approach to motion transfer. It directly addresses artifacts common in skeleton-based methods。\n2. The quantitative results (lower FVD, higher SSIM, etc.) reported in the ablation and comparison studies suggest that DirectAnimator achieves high visual quality and identity preservation, outperforming several recent strong baselines。", "weaknesses": "1. The Same2X training strategy is somewhat similar to the approach used in X-Portrait, which also involves generating pseudo driving samples for cross-identity training. While effective, this core idea of leveraging synthetic data for generalization is not entirely novel in the field.\n2. There appears to be a low visual quality in both the video demonstrations and the illustrated figures within the paper. Specifically, I observe that identity preservation in the figures is insufficient, and the clothing in the demo video still exhibits noticeable artifacts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Jus0IU5oT3", "reviewer": "Reviewer_wkJD", "summary": "Human image animation generates videos from static images using motion cues from driving videos, but traditional pose-based methods suffer from errors inherent in pose extraction.\nDirectAnimator eliminates pose extraction and learns directly from raw videos through a Driving Cue Triplet—pose, face, and location cues—fused via a CueFusion DiT block.\nA Same2X training strategy aligns features, enabling stable and efficient learning.\nDirectAnimator achieves state-of-the-art performance on the Unseen (internet-crawled) dataset and competitive results on the TikTok dataset.", "questions": "1. Regarding the dependency on StableAnimator [1] mentioned in Weakness 2, what is the reason that the proposed method is not considered to be bound by the performance of StableAnimator?\n\n2. Even though cross-ID training starts from a model pretrained on Same-ID data, why does using the S2X loss in Eq. (4)—which guides the latent representations to follow those of the Same-ID trained model—result in faster and better training? Could you provide an explanation?\n\n3. In L207, a low-pass filter is used to remove high-frequency information. However, in Figures 2–5, the pose cues appear to have only lost color information while retaining most appearance details. Could you clarify why this happens?\n\n4. In L222, spatial alignment is adopted from StableAnimator, but I could not find an explanation of this in the StableAnimator paper after a quick scan. Could you specify the exact section or provide a brief description? Also, with the location cue, would your method be capable of performing HIA on chibi characters, similar to what TCAN[2] demonstrated?\n\n[1] Tu, Shuyuan, et al. \"Stableanimator: High-quality identity-preserving human image animation.\", CVPR 2025.\n\n[2] Kim, Jeongho, et al. \"Tcan: Animating human images with temporally consistent pose guidance using diffusion models.\", ECCV 2024.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "1. By conditioning directly on raw videos, the model effectively addresses errors that arise during pose extraction.\n2. Inspired by REPA, REPA-E, and SRA, it performs latent alignment, which improves both training speed and overall performance.\n3. Incorporating a face cue helps maintain facial identity consistency, leading to more natural and expressive animations.", "weaknesses": "1. It is difficult to identify clear advantages over StableAnimator[1] from the comparison videos presented on the project page.\n\n2. Although the paper claims to propose a Human Image Animation (HIA) method that overcomes the limitations of skeleton-based pose extraction, it appears that the training process relies on results generated by StableAnimator, which itself depends on skeleton pose extraction.\n\n3. Minor Suggestion: In Equation (4), it seems that the parameter D, which is mentioned in Table 3, is missing.\n\n[1] Tu, Shuyuan, et al. \"Stableanimator: High-quality identity-preserving human image animation.\", CVPR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)