DanceTogether: Generating Interactive Multi-Person Video without Identity Drifting
Authors: Junhao Chen
Mingjin Chen
Jianjin Xu
Xiang Li
Junting Dong
Mingze Sun
Puhua Jiang
Hongxiang Li
Yuhang Yang
Hao Zhao
Xiao-Xiao Long
Ruqi Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Controllable video generation
Multi-person Interactive Video Generation
Multi-person Pose Estimation
Multi-object Tracking
Primary area: generative models
TL;DR: DanceTogether is an end-to-end diffusion framework for controllable multi-actor video generation with identity preservation, enabled by a novel adapter, new datasets, and a benchmark supporting human–robot and cross-domain scenarios.

Abstract: Controllable video generation (CVG) has advanced rapidly, yet current systems falter when more than one actor must move, interact, and exchange positions under noisy control signals. We address this gap with DanceTogether, the first end-to-end diffusion framework that turns a single reference image plus independent pose-mask streams into long, photorealistic videos while strictly preserving every identity. A novel MaskPoseAdapter binds “who” and “how” at every denoising step by fusing robust tracking masks with semantically rich but noisy pose heat maps, eliminating the identity drift and appearance bleeding that plague frame-wise pipelines. To train and evaluate at scale, we introduce (i) PairFS-4K, 26 h of dual-skater footage with more than 7 000 distinct IDs, (ii) HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain transfer, and (iii) TogetherVideoBench, a three-track benchmark centred on the DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a significant margin. Moreover, we show that a one-hour fine-tune yields convincing human-robot videos, underscoring broad generalisation to embodied-AI and HRI tasks. Extensive ablations confirm that persistent identity-action binding is critical to these gains. Together, our model, datasets, and benchmark lift CVG from single-subject choreography to compositionally controllable, multi-actor interaction, opening new avenues for digital production, simulation, and embodied intelligence.

Published as a conference paper at ICLR 2026
DANCETOGETHER: GENERATING INTERACTIVE MULTI-PERSON VIDEO WITHOUT IDENTITY DRIFTING
Junhao Chen
Mingjin Chen?
Jianjin Xu
Xiang Li Yuhang Yang*
Junting Dong
Mingze Sun" 1
Puhua Jiang'
Hongxiang Li
Hao Zhao 10,t
Xiaoxiao Long" Ruqi Huang
Shenzhen International Graduate School, Tsinghua University PAIR, Tsinghua University
The Hong Kong Polytechnic University *Carnegie Mellon University
6 Peking University Shanghai AI Laboratory Uniierssit Science & Technology of China
"The Hong Kong University of Science and Technology
"Nanjnng University BAAI
Project Page:
https: //w/cntann....oooooooooooooo.oo. gi github. io/
Driving
Video
Image
Figure 1: DanceTogether generates complex two-person interaction videos with interactive details and consistent identity preservation from single reference image (see the left-most of each row), using independent
pose and mask sequences as control signals.
ABSTRACT
Controllable video generation (CVG) has advanced rapidly, yet current systems falter when more than one actor must move, interact, and exchange positions under noisy control signals. We address this gap with DanceTogether, the first end-to-end diffusion framework that turns single reference image plus inde- pendent pose-mask streams into long, photorealistic videos while strictly pre- serving every identity. A novel MaskPoseAdaater binds "who" and "how" every denoising step by fusing robust tracking masks with semantically rich but noisy noisy-pose meat-maps, eliminating the identity drift and appearance bleed- ing that plague frame-wise pipelines. To train and evaluate at scale, we intro- duce (i) Pa SairFs--44 K, 26 h of dual-skater footage with 7, 7,000+ distinet IDs, (ii) HumanRob-300 300, a one-hour humanoid-robot interaction set for rapid cross- domain transfer, and (iii) TogethervideoBench. a three-track benchmark centred on the DanceTogEval 100 test suite covering dance, boxing, wrestling, DanceTogether outper-
yoga, and figure skating. On rogetherri forms the prior arts by significant margin. Moreover, we show that one-hour fine-tune yields convincing human-robot videos, underscoring broad generaliza- tion to embodied-Al and HRI tasks. Extensive ablations confirm that persistent identity-action binding is critical to these gains. Together, our model. datasets, and benchmark lift CVG from single-subject chorcography to compositionally controllable, multi-actor interaction, opening new avenues for digital production, simulation, and embodied intelligence.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Controllable video generation (CVG) Ma al. 2024) seeks translate explicit control sig- nals-e.g. per-frame human poses, body masks, or trajectory commands into photorealistic human- motion videos. Compared AI generation tasks that use single conditioning (reference images or text) (Podell ct al. 2024 OpenAl 2024, Khachatryan et al. 2023), some controllable generation tasks typically combine multi-modal conditions as input (Zhang al,, 2023b Peng et al., 2024: Hu 2024; (Cen et al. (025a)). Such tasks using multi-modal control signals have broad and important applications in film production (Bugliarello et al. 2025 Song et al. 2024). digital human (Chen et al. 2025cl TTan et al., 2025; Miao et al. 2026 Wang et al. 2025c Sun et al. 2024b), and embodied Al (Albaba et al. 2025). In particular, we investigate the task of CVG with multi multi-person
person interactions, which is highly challenging as it simultaneously requires (i) preserve the identities of multiple ac- tors over hundreds of frames, (ii) maintain the apatioltemporal coherence of complex interactions such hand-holding, lifts, position exchanges, and synchronous choreography, and (iii) faithfully obey noisy control signals in the presence of occlusion, motion blur, and rapid viewpoint changes.
Most existing systems adopt a frame- synthesis followed by temporal smoothing paradigm: each image is generated independently from pose or text conditions and then stitched into video via interpolation, optical-flow warping, or temporal convolutions (Di Chang et al. 2023 Ma et al. 2024 Zhu et al. 2024b). Nearly all of these models are trained solely on single- person dance datasets (Zhu
ct at. 2024a Xu et al., [2024c Hu 2024 Zhang et al. [024b]] Waan et al.) (202cc). A handful of works incorporate multi-person footage (Wang et al. [024e] Zhang et al. [024a]] Xue et al. 2024). but they exhibit pronounced identity drift and appearance bleeding when the actors exchange positions. In
general, of-te-of-thh-ar methods struggle with identity inconsistency, cross-subject contamination, and missing interaction details-issues that rapidly worsen once more than one performer is involved.
We present DanceTogetlen the first end-to-end diffusion framework expressly tailored for controllable multi-person interaction video generation. Our guiding hypothesis is that robust multi-actor synthesis requires an explicit, persistent binding between identity and motion throughout the diffusion process. To this end, we deliberately disentangle identity from action and then re-couple them: instead of relying solely on fragile pose estimates, we fuse stable tracking masks with semantically rich pose cues. This fusion is realised by a novel conditional adapter, MaskPoseAdapter which combines the reliable, easy-to-obtain body masks with the informative yet noisy poses into bimodal control signal. By integrating each subject's 's mask and pose into a unified representation, the adapter enforces precise identity-to -t-iittttnn alignment every generative step.
Our framework operationalizes the identity actionn binding principle through three tightly coupled modules. (i) MultiFace Encoder distills a compact set of identity tokens from a single image and injects them into every cross-attention layer, ensuring subject appearance is held constant throughout the sequence. (ii) MaskPoseAdapter fuses robust per-person tracking masks with semantically tich-but noisy- pose maps to deliver a bimodal conditional signal that aligns "who" and "how" at every diffusion step, thereby safeguarding both identity integrity and motion fidelity. (iii) Video Diffusion Backbone leverages these aligned signals to synthesize high-resolution clips whose multi- actor motions remain coherent, physically plausible, and free of identity drift.
Extensive evaluation on the new shows that DanceTogether decisively advances controllable multi-person video generation. Video Quality) it raises the bar over the strongest prior (StahleAnimatoo (Tu et al. 2024) +swing dance data (Maluleke et al. 2024) finetune) by +12.6 HOTA, +7.1 IDF1, +5.9 MOTA, trims MPJPE,p by 69 % (1555 - 492 px), and boosts OKS/POseSSIM to 0.83/0.93. 93. Visual fidelity also improved accordingly: human mask region FVD/FID decreased from 29.0/66.7 to 1/41/0 without sacrificing CLIP alignment effect. Fine-tuning on our proposed one-hour HumanRob-23 300 dataset can generate convincing buman-robor interaction videos, which highlights the framework's 's broad generalization capability and prospects in embodied AI research.
built around our 100-clip DanceToggval- 100
Across the three core tracks Ceeenttty-CCnssstennyy Interaction-Coherenee
To summarize, our main contributions include:
1. Dance Together framework. We present the first end-to-end diffusion framework for controllable multi-person interaction video generation. Our novel MaskPoseAdaaper fuses stable tracking masks with pose cues to enforce identity-action binding throughout the generation process.
2

---

Published as a conference paper at ICLR 2026
2. Data curation pipeline and datasets. We develop monocular-RGB pipeline for extracting tracking-awer aware human poses and masks. Using this, we curate PairFS- 4K (26h dual dual-person figure skating) and HumanRob-300 (Ih robot interaction) datasets.
3. Together Bicee benchmark. We introduce comprchensive evaluation bench- mark with three tracks Eidentry-Consiitenny DanceTogEval 10000 containing 100 dual-actor clips across diverse activities.
setruaction-Chhrenne Video Quality) and
4. Superior performance and generalization. Our method achieves significant improvements: +12.6 HOTA, +7.1 IDF1, +3.9 MOTA over the strongest baseline, 69% reduction in pose error, and enhanced visual fidelity (FVD: 29.0+177.1 Cross-domain fine-tuning demonstrates strong generalization to human-robot scenarios.
2 RELATED WORK
2.1
DIFFUSION MODELS FOR VIDEO GENERATION
Diffusion models have achieved significant progress in video generation (Poole al.
2022
et al.
2023 Kong et al.
2024
Li al. 2024 Wang et al. 2024a Chen et al. 2024 Lin et al. [224
Han et al. 2024|| Liu et al.
2025| Yang et al. (025b). Early works used UNe for spatiotemporal
fusion (Singer et al.] 2022 Ho et al. 2022p. Building on this, (Blattmann al. 2023b) introduced temporal dimensions into latent diffusion models; (Ho et al. 2022) employed cascaded spatial-
temporal super-resolution; (Xu et al., (2024a) enabled multimodal control through spatiotemporal
consistency modeling: (Blattmann et al.] (023a) proposed temporal interpolation strategies for 2D-
to-video adaptation. Despite advances commercial models (Kong et al. 22024, Kuaishou 2025
OpenAl, 2024) achieving good temporal consistency and resolution, they remain inadequate for
fine-grained human motion control tasks.
2.2 CONTROLLABLE HUMAN VIDEO GENERATION
Diffusion models (Rombach et al. 2022 Blattmann et al. 2023a) have significantly advanced
controllable human video generation. Most methods build on pre- traitrained Stable Diffusion with pose
guidance, using keypoints or skeletons as conditions (Zhang et al. 2023b; Hu 2024). Disco (Wang
et al. 2024d) separates background and pose control, a strategy extended by (Hu [2024] Xu et al.
2024c
Other approaches use geometric priors from 3D models (Wang et al. | et al.
2024b or SMPL models (Zhu al.) [2024a] Li al. (025a)), but remain limited single-person scenarios. Despite these advances, most methods struggle with multi-person interactions and identity consistency. Recent works (Wang al. 2024e Luo al., 2025 Ma et al., 2024, Xue al., 2024 Karras et al. 2023 Feng et al.) 2023 Tu et al. 2024) explore identity preservation using encodings or masks Yoon et al. 2021), but are limited to short videos. Some methods (Wang et al. 2024c Zhang
et al. 2024b) use local masks or attention mechanisms, yet lack explicit identity-action binding,
causing drift long sequences.
2.3 RECENT MULTI-PERSON ANIMATION WORKS
Multi-person controllable animation (Huang et al. 2024 Xu et al. 2024b Chang et al. 2025) has recently advanced through works like Multi-Humaa Vid (Wang al. (025d) and Follow-your- multipose (Zhang et al.) 2024a) demonstrating spatial guidance for motion synthesis. Concurrent mask-based approaches include MagicMotion (Li et al., (0256). Multi Multi-Human'
(Wang al.)
2025d), EverybodyDance (Ling et al.). and ReMask-Amiman
(Xiang et al. 2025). which leverage masks, geometric priors, or identity correspondence for controllable generation. DiT-based models like Wan2.2 (Cheng et al. [025 Diang et al.) (025) and commercial systems like Kling (Kuaishou 2025) and Veo3. (Google) 20225) enable text-driven animation with strong temporal consistency. However, these methods primarily handle scenarios with fixed relative positions and lack fine-grained per-person control, making them inadequate for identity drift during position exchanges wheen when people cross paths heavily occlude each other. Our approach addresses this gap through explicit identity-action binding via gated mask-pose fusion and cross personneoonnnnooo attention (Sec. 3.2). preserving consistent identities throughout dynamic interactions and position swaps.
3

---

Published as a conference paper at ICLR 2026
MaskPoss Adapter
Σ = Edl (Epr 20((1 Mffce Moony) 19)
U-Net
FEW SHEATO
Decodar
Faremition
LOMO TrIER
Praren
Jos SECSOLS
VAE Encoder
Reportt frand
Caucatanccs Speratice
Mrightee
Face Eanddings
Pace bookiter
Inceder
Antasted frames
Figure 2:
DunceTogether pipeline overview: A single reference image and per-person pose/mask sequences enter the system; the MaskPoseAdapter fuses these control signals, the MultiFace Encoder injects identity tokens, and the rideo-diffusien backbone synthesizes interaction video that preserves consistent identities for actors. The implementation details of AasperseAdapter and MuldiFaceEncoder be found in Sec. A
3
METHOD
3.1 VIDEO DIFFUSTON BACKBONE
Starting point - StableAnimator. Our backbone follows the StableAnimator architecture (Tu et al. 2024): 16-frame latent UNet fe derived from Stable Video Diffusion (SVD). For every training clip we take input (Iref: Pi.r, M1.r) where F E 3 XXW is a reference image, and P, M, are the pose map and tracking mask at frame f.
Three conditioning streams. The UNet is conditioned by three streams, each of which begins with frozen pretrained encoder and then refined by trainable adapters (see Fig.
Latent image stream. A frozen SVD VAE encoder maps both the reference image Leef and each input video frame their respective latent representations. The reference latent Zref E 6x64x64 is tiled along the temporal axis and concatenated with the per-frame latents Zyt. This concatenated tensor is then fused with the trainable PasskPossAdapter condition latents via element-wise addition, producing the final latent input to the UNet.
CLIP image embeddings. frozen ViT-H/14 encoder ΦCLIP produces eclip E 1024 These embeddings serve keys/values in every trainable cross-attention block.
Refined face embeddings. A frozen ArcFace model éID outputs e E 12522, which is then refined by the trainable MultiFaceEneoder Ig:
fface g eclip) E Kxd
implemented four Perceiver-IO layers (K 4, d 768). The resulting identity tokens modulate the same trainable cross-attention layers.
Distribution-aware ID Adapter. To prevent feature-distribution shift when injecting identity tokens, StableAnimator inserts an ID Adapter before each temporal block. Given input features h, we first apply spatial self-attention and two cross- antsstattttiinn steps, then align and fuse the face branch to the image branch in a single fused update:
h SA(h), hang
CA(hi Celip),
hiace CA(h, Ejace):
hince
Pface
hSace
Oung + Ming
hout
hing
haaee
Face
(p, the per-token mean and standard deviation. and Eface the set of K identity tokens. By matching the first and second moments of the face and image features, this adapter preserves identity information consistently across all frames.
Here SA/CA denote
4

---

Published as a conference paper at ICLR 2026
Human-tracking masked reconstruction loss. Building upon StableAnimator's face-focused loss, we incorporate per-person binary masks for face and body regions. Original 512 x 512 masks are downsampled via nearest-neighbor interpolation to the latent resolution 64 x 64. Given N individuals with binary masks Mf
body E {0, 1x61x61,
we optimize
Lree Σ B (Zge 2.) (1 M body 2 M
face
Here body masks have weight 1 and face masks weight 2, encouraging the model to focus capacity on identity-critical regions while preserving overall reconstruction fidelity.
3.2 MASKPOSEADAPTER BINDING IDENTITY TO ACTION
Motivation. Pose keypoints alone cannot reliably distinguish individuals when they overlap or exchange positions. Conversely, binary masks lack semantic motion information. We propose to fuse these complementary signals in a learned feature space, explicitly binding each person's 's identity (via masks) their motion (via poses).
Architecture overview. For each person i, we process pose maps P. R3x512x512 and masks M, E {0, 1) x512x512 through separate encoders, producing pose features f.00e 1220x64x64 and compact mask features fmusk g4x66464 The key innovation lies in three stages:
(1) Gated fusion. We learn per-pixel gates that adaptively weight pose vs. mask contributions:
f; = A- -u poee + f (1 - A) w mask @ fmmask + residual,
where p maskk E [0, are gate activations and À 0.8 biases toward pose. This allows the network to emphasize identity (mask) when poses are ambiguous, and vice versa. A lightweight
PoseEnhancer further refines local details.
(2) Cross-person attention. To handle occlusions and position exchanges, apply LayerNorm to each f, and concatenate across persons. shallow network produces attention logits, normalized via (emperature-scaled softmax:
Oan softmax()((LN(()) (N(x)))))).
(5)
where 7 is learnable. This reweights person importance dynamically-ess e.g., emphasizing the fore- ground actor during occlusion.
(3) Integration. The final conditioning F Rx61X66 combines attention-weighted features with small residual mean:
N
= 0.95 Convix1 Σ an O... + 0.05
N Σ f
(6)
which is reshaped (B, T, C, 64, 64) and injected into the UNet via element-wise addition.
Why it works. By fusing masks and poses at the feature level with learned gating and cross- person reasoning, MaskPose creates unified representation where identity and motion are inseparable. This eliminates the identity confusion observed when processing poses alone (see ablations in Sec. B). Full implementation details including gate architectures, channel dimensions, and layer specifications are provided in Sec. A.1
3.3 MULTIFACE ENCODER: COMPACT IDENTITY INJECTION
To handle multiple identities efficiently, we extend single-person identity encoding to multi-identity regime. Each ArcFace embedding ea 5 is projected to K 4 tokens via two-layer MLP, then refined through a lightweight FacePerceiver (4-layer cross-attention with CLIP embeddings as memory). The resulting tokens t; I. K 768 for all N persons are concatenated:
T [t:: te: tN] E (NNK) x68
This extended token sequence modulates every cross- aassetttentonn layer in the UNet without architectural changes. Compared to naively duplicating face encoders, this -eiiht design reduces parame- ters while enabling the model to learn inter-identity relationships. Detailed layer configurations and attention mechanisms are in Sec. A.2

---

Published as a conference paper at ICLR 2026
3.4 LIGHTWEIGHT EXTENSION TO MULTIPLE PERSONS
MaskPose modular design enables effortless scaling to arbitrary person counts. To extend from two to N persons, we instantiate N independent branches with shared weights. The cross-person attention (Equation(5) naturally generalizes without modification.
Training strategy. We leverage single and dual-person data through adaptive branch activation: for single-person clips, randomly activate one branch (zero others); for two-person clips, randomly activate two branches; for N N-person clips, activate branches. New branches initialize by copying weights from trained branches. This curriculum requires architectural changes, additional losses, or multi-person training data.
3.5 DATA CURATION PIPELINE
To address the lack of two-person interaction datasets with diverse identities, static backgrounds, and fixed cameras, we propose a comprehensive data curation pipeline that recovers poses and mask annotations from monocular RGB videos. As shown in Fig. our pipeline segments videos into scenes, detects and tracks individuals using YOLOv8x (Jocher et al. 2023 and "SNee-based RelD (Zhou et al., 2019 2021). and selects primary subjects based on coverage and consistency. We then generate high-quality per-person masks and 133-point pose annotations using SAMURAI (Yang) et al. 2024), DWPose (Yang et al. 2023), and MatAnyone (Yang al., 2025a). followed by automatic and manual filtering to ensure data quality. We aggregate wide range of single- and two-person motion datasets cataaets-innludiig including TikTokDataset (Jafarian & Park 2021). Champ (Zhu et al.] 2024a), DisPose (Li ct al., 2025a). Human Vid (Wang et (004e), Swing Dance (Maluleke et al. (2024). Harmony4D (Khirodkar et al., 2024). CHI3D (Fieraru et al. 2023), Beyond Talking (Sun et al. 2025), and our newly collected PairF$-48 to maximize identity diversity and interaction types. PairFS-4K comprising 4.8K figure skating segments and over 7,000 unique identities, is the first large-scale two-person figure skating video dataset. All datasets summarized in Tab. providing a rich foundation for controllable human interaction video generation in real-world scenarios. More details the Data Curation Pipeline can be found in Sec. [
For specifics on the collection and processing of PairFS- SairFS-4K, 4K, please refer to Sec. C.5 The segmentation module can easily be
substituted with newer segmentation models Ravi et 2024)
Tian et al. (2023); Chen al. (2022)
Liu et al. (202aa)
Curation
Key BBox Extraction
Grounded Segmentation
Foregroundi Background Segmentation
Milcc Fransox
DWPose Extraction
Person Tracking Mask
Person Tracking Pose
Person Face Mask
Main Target Tradking
Segmentaiion
Video Splitting
SAMURAI
Ery Wer Moy
Figure 3: Data Curation Pipeline Overview. Our pipeline processes raw videos through human tracking, mask generation with SAMURAI (Vlasic al. 20088. pose estimation with DW-Pose (Yang et al. 2023). and alpha matting produce per-person annotations.
3.6 TOGETIERRVIDRBBENCI BENCHMARK
We introduce Together VigetherVideoBenchh comprehensive benchmark for controllable multi-person video generation, which systematically evaluates three orthogonal tracks: Interaction-Coherence and Video Quality. Please refer details Sec. D in the Appendix.

---

Published as a conference paper at ICLR 2026
Table I: Summary of datasets used in Dance'Together training. "Static competition background;
Static laboratory background; MMulti-view setup.
Dalaset
Type
Action
IDs
Total
Avg.
Scene
Camera
TikTokDataset Jafarian & Park 2021
Single
Dance
332
1.03 hrs
s
Static
Fixed
Champ Zhu
Single
Dance
832
9.73
Static
Fixed
DisPose [Drera ara 20220
Single
Dance
8,636
38. 12 hrs
1
Statie
Fixed
Human Wang
2024g
Single
Dance
16,310
89
1
Dynamic
Moving
Hi4D Yin al. 2023
Double
Interact
40
0.10 hrs
3.6
Static!
Fixed
Harmony4D Khirodkat Kar et
2024
Deuble
Interact
24
0.58 hrs
12.s
Static!
Fixed
CHT3D (Fierami di al: 21755
Double
Interact
1.75
4
Static
Fixed!
Swing Dance MMaluikke
2024
Double
Dance
1,356
23.36 hrs
122 s
Static"
Moving
HoCo (Sun et 2175
Double
Talking Head
26
45 hrs
7
Static!
Fixed
PairFS.4K
Double
Figure Skating
7,273
26.87 87
Static"
Moving
HumanRobb-300
Single
Robot Interact
336
0.83 has
9
Dynamic
Moving
DanceTogEEva-1100
Double
Interact Dance
Static
Fixed
4 RESULTS
4.1
EXPERIMENTAL SETUP
We collect several publicly available video datasets, as detailed in Sec. C.I We utilize DWPose (Yang et al. 2023) and ArcFace (Deng et al. 2019) to extract skeletal poses and facial embeddings/masks. To evaluate the robustness of our model, we conduct experiments on TanceTogEvas 1000 curated set of 100 previously unseen two-person interaction videos from the internet. Following recent advances in animation generation, we initialize our U-Net, PoseNet, and Face Encoder with the pre-trained weights from StableAnimator, then further train them on large-scale single-person datasets (Tu et al.) 2024
Li et al. 2025a Zhu et al. 2024a Wang et al.) 2024e). We subsequently transfer the pre-trained weights to our proposed MaskPoseAdaptee and MultiFace Encoder, and perform full tine-tuning using multi-person datasets innluudnng our proposed PairFS dataset (Maluleke et al. (2024] Sun et al.
2025 Khirodkar al. 20244 Fieraru al.
2023). Our model is trained for 20 epochs on NVIDIA A100 80G GPUs, with batch size of per GPU and learning rate set to le 5.
4.2 BASELINES
We compare approach with state-of-the-art pose-conditioned human video generation mod- els, including AnimateAnyone, Champ, MimicMotion, Human Vid, UniAnimate, UniAnimate-DD DisPose, and StimaeAnimator. In particular, we fine-tune StableAnimator for 40 epochs on the dual-person dancing subset from the Swing Dance dataset (Maluleke et al. (022)), and include this fine-tuned variant as a new baseline in our evaluation. Fig. 4 compares our proposed DanceTog with four strong baselines, AnimateAnyone Human Vid, UniAnimate, and StableAnimator all of which achieve relatively high scores in the quantitative evaluation. All baselines exhibit severe identity drift, loss of interaction details, or even missing subjects when dealing with position exchanges and complex interactive poses. Additional comparisons, including more baselines and dual- -erson interaction examples, are provided in the Sec. [ Video comparison results with baselines are provided in the Supp. Mat. User study results are provided in Sec. demonstrating that our proposed benchmark aligns with human preferences.
Table 2: Multiple Object Tracking results on TogetherVideeBBeee because the sum of false positives (FP) and false negatives (FN) exceeds the number of ground truth objects. This happens when the frames only contain a single person.
* Negative values occur
HOTA family
CLEARMOTA family
Identity
Method
Venue
HOTA
DetA1
AssA
MOTA!
MOTP!
IDFI1
Animate Anvone Hu 2024
CVPR 2024
41.26
39.99
43.21
26.67
75.73
51.54
Champ Zhu et all 20/4
BCCV 2024
19.32
14.78
26.32
-19.54*
67.92
17.84
MimicMMmmi Zisag Q
2024bb
Arxiv 2024
21.14
16.06
30.50
-55,77
62.13
15.24
HumanVid Win 0
NeurPP 2024
56.12
58.89
53.69
58.86
84,20
68.84
UniAnimate Wang
SCIS
46.71
42.33
80.74
UniAnimate Di Wang
2025b
Arxiv 2025
35.02
31.15
40.65
10.66
77.99
39.51
DisPose (Li
20.68
15.91
-52.49"
2024
CVPR 2025
67.75
67.91
67.70
69.62
87.67
79.37
StableAnimttor 12000
CVPR 2025
71.35
70.91
71.89
73.89
88.22
82.53
Dance' Tog w. Duttacinn
80.26
74.44
86.57
73.68
95.45
86.28
DanceTog Tog Datayult
81.79
86.69
77.04
95.69
87.73
Dance Tog w. Dataral + DatapairP?
83.94
79.48
88.68
79.80
95.49
89.59
7

---

Published as a conference paper at ICLR 2026
Image
Animate Anyone
HumanVid
UniArimate
Stable Animator
Danoa Tag (Ours)
Figure 4: The RGB image in the "Ref. Image" row is the input reference frame, and the two pose maps in that row correspond to the inference results shown immediately below.
Table 3: Comparison of models across interaction coherence metrics.
MPIPEa1
OKS1
PoseSSIM1
SmacthRMS (x10")
TimeDnngassd (x10"))
FVMD]
Method
(x10")
Animate Anyone
3255.07
0.27
0.67
1.26
2.43
1.87
Champ
4117.88
0.06
0.78
MimicMotion
5542.99
0.09
0.74
1.02
1.94
1.15
Human Vid
3480.74
0.48
0.78
1.09
2.10
1.11
2286.26
UniAnimate DIT
2184.81
0.22
0.71
1,53
2.92
3.72
StabicAninator StabicAnimstor Datnawiin
1571
0.63
0.82
0.96
1.84
1.00
1555.16
0,70
0.84
0.89
1.72
Dance Tog w. Dataysing
858.99
0.75
0.88
0.84
1.62
0.51
TunceTog Data fuil
557.60
0.81
0.92
0.85
1.64
0.66
DanceTog w. Data + Detemarrss
492.24
0.83
0.93
0.83
4.3 QUANTITATIVE RESULTS
Track Cdentity-Cnniistenyy Tab. reports multiple-ohjeer-tracking (MOT) scores on DanceTogEval- 100. Across all eight published baselines, StableAnimator fine-tuned on SwingDance (StableAnimator + Dataswing) is the previous best performer, reaching 71.35 HOTA and 82.53 IDF1. BonceTogether markedly exceeds this strong baseline on every single metric: with full training data lifts HOTA from 71.35 to 81.79 79 (+10.44 %) and IDF1 from 82.53 to 87.73 (+6.3 %), while pushing AssA 86.69. Adding the proposed PairFS- 4K dataset provides further gain, culminating in 83.94 HOTA, 89.59 IDF1, and a 79.80 MOTA. These results establish a new state of the art for long-range identity preservation under frequent occlusions and position exchanges.
Track 2: Interaction-Coherenee Tab. Jevaluates how faithfully each method follows the target
motion and how smoothly the interaction unfolds. Our model slashes MPJPEsp by 68 % relative to the top baseline (from 1555 px to 492 px) and attains the highest OKS (0.83) and PoseSSIM (0.93). At the same time, DanceTogether records the lowest motion disioniiiittt scores- Saoeess-moothRMS and TimeDynRMsE -indicating physically plausible, temporally consistent choreography. Champ achieves high scores on SmoothRMS and TimeDynRMSE due to its use estimated SMPL as guidance, which incorporates smoothing methods in the process of generating SMPL sequences. These two metrics only compare the motion continuity of each individual person without applying weights to the pair. Champ's '' inference results typically contain only single person. The FVMD halved compared with StableAnimator (0.54 vs. 1.00), further corroborating superior interaction quality.
8

---

Published as a conference paper at ICLR 2026
4 present full-frame and mask-aware appearance metrics. Benefiting from dense identity- action binding and the high-diversity PairFS-4K corpus, Dancelogether delivers the best perceptual fidelity in both settings. In full-frame evaluation it attains the lowest FVD (76.3) and FIID 175.1), alongside the highest CLIP score (0.95) and ST-SSIM (0.70). Within the human- masked regions- the areas most sensitive to identity drift- drift-mask-ane awak- FVD plunges from 29.0 to 17.1, and C-FID shrinks from 12.5 to 7.9, highlighting crisp texture reproduction and identity accuracy. Notably, these improvements are achieved without sacrificing low-level reconstruction fidelity: LI and LPIPS fall in tandem, while PSNR and SSIM increase. Complete quantitative evaluation results can be found in Sec.F)
Track 3: Video Quality. Tab.
Table 4: Comparison of models on Full Frame and Human Masked Region.
Method
PSNR1
SSIM1
LPIPS
Masked
Misked
Full
Masked
Masked
Full
Misked
Full
Masked
AnimotccAyynee
37.33
59.92
13:23
10.45
0.49
0.92
0.56
0.06
108.2
44.8
118.1
101 4
Champ
43,70
83.35
11,93
8.36
0.49
0.92
0.56
0.07
125.7
69.2
114.6
178.7
MinicMetion
77.02
0.07
65.5
116.6
180.9
HumanVis
38.93
47.97
13.67
12:15
0.52
0.93
0.50
0.05
97.2
34.9
90.2
72.4
UniAnimure
37.95
56.34
13.62
11:05
0.55
0.92
0.53
0.06
132.0
45.0
151.2
1098
UniAnimare-Dil
43.11
64.48
12.34
9.89
0.50
0.91
0.53
0.06
111.9
514
100.3
119.6
DisPose
127.9
StuhleAnimator
48.51
0.57
0.93
0.05
85.7
84.1
71.8
StableAnimmon w. Datovumy
30.31
41.41
15.27
13:06
0.60
0.93
0:42
0.04
78.8
29.0
79.3
66.7
Dance Tog w. Dang
32.62
34.49
15.12
14:76
0.59
0.94
0.44
0.03
79.3
21.5
82.1
57.5
Dance Tog Dataysii pil
29.94
32.80
15.15
0.61
0.94
0.03
76.9
20.6
77.6
56.1
Dance Tog Dataysa + DotapaisPs
15.82
0.61
0.94
0.42
0.03
76.3
17.1
75.1
48.0
4.4 ABLATION STUDY
We perform ablation studies demonstrating that each proposed component, MaskPoseAdapter for pose-mask fusion, MuldeaacEncoderr for multi-identity encoding, and PairFS-4K dataset-cconrrinnn contributes significantly to performance improvements (see Sec. for details).
4.5 INFERENCE EFFICIENCY
Table 5: Inference speed comparison (fps).
DanceTog achieves competitive inference speed (0.88 fps) while maintaining superior genera- tion quality, significantly faster than DiT-based methods like UniAnimate-Dii (0.03 fps), and comparable to baselines such as StableAnimator (0.89 fps) and AnimateAnyone (0.97 fps). Like all curtusion-basee human animation methods, DanceTog is not real-time but provides prac- tical balance between quality and efficiency for controllable multi-person video generation. All
Method
FPS
DanceTog (Ours)
0.88
Champ
1.42
Animate Anyone
0.97
StableAAnimator
0.89
HimanVid
0.81
Dispose
0.42
UniAnimate
0.25
MimicMotion
0.06
baseline methods are tested on single A800
UniAnimate-DiT
0.03
GPU, See Sec. Gor details on training/inference costs and multi multi-person person training support of baselines.
CONCLUSION
We present DanceTogether, the first end-to-end diffusion framework for generating long, photoreal- istic multi-actor videos from single reference image and independent pose-mask streams, while strictly preserving each identity. Our method integrates a novel MaspPose for persistent identity-action alignment and a MultiFace Encoder for compact appearance encoding. Trained on our newly curated multi-actor datasets and evaluated on a comprehensive benchmark, DanceTogether outperforms all existing pose-conditioned video generation models by significant margin. It gener- alizes well across domains, as demonstrated by convincing numan-robo interactions after minimal adaptation. This work marks step forward toward compositionally controllable, adentity-awan video synthesis, laying a foundation for future advances in digital content creation, simulation, and embodied AI.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported in part by the National Natural Science Foundation of China under contract No. 62171256, and in part by the Shanghai Artificial Intelligence Laboratory.
REFERENCES
Mert Albaba, Chenhao Li, Markos Diomataris, Omid Taheri, Andreas Krause, and Michael Black. Nil: No-data imitation learning by leveraging pre-trained video diffusion models. arXiv preprint arXiv:2503. 10626, 2025.
Keni Bernardin and Rainer Stiefelhagen. Evaluating multiple object tracking performance: the clear mot metries. EURASIP Journal on Image and Video Processing, 2008:1-168 1-00, 2008.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:231..55127, 2023a.
Andreas Blattmann, Robin Rombach, Huan I. Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis, Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 22563 -2575,, 2023b.
Emanuele Bugliarello, Anurag Arnab, Roni Paiss, Pieter-Jan Kindermans, and Cordelia Schmid. What you doing? a closer look at controllable human vidco generation. arXiv preprint arXiv:2503.0455 2025.
Zhe Cao, Tomas Simon, Shib-En En Wei, and Yaser Sheikh, Realtime multi-person 2d pose estimation using part affinity fields. In CVPR, 2017.
Di Chang, Hongyi Xu, You Xic, Yipeng Gao, Zhengfei Kuang, Shengqu Cai, Chenxu Zhang, Guoxian Song, Chao Wang, Yichun Shi, et al. X-dyna: Expressive dynamic human image animation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5499-5509, 2025.
Yupeng Chang, Xu Wang. Jindong Wang, Yuan Wu, Linyi Yang. Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1 45, 2024.
Junhao Chen, Xiang Li, Xiaojun Ye, Chao Li, Zhaoxin Fan, and Hao Zhao. Idea23d: Collaborative Imm agents enable 3d model generation from interleaved multimodal inputs. In Proceedings of the 31 International Conference on Computational Linguistics, pp. 4149-4166, 2025a.
Junhao Chen, Jingbo Sun, Xiang Li, Haidong Xin, Yuhao Xue, Yibin Xu, and Hao Zhao. LLMsPark: A benchmark for evaluating large language models in strategic gaming contexts. In Christos Christodoulopoulls, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 182-194, 194, Suzhou, China, November 2025b. Association for Computational doi: 10. 12285////22255.mnnn/l ac1 antaan egy o 2025. f ndi
Linguistics. ISBN 979-8 8
URL ht https://
emr 12.
Mingjin Chen, Junhao Chen, Huan-ang Gao, Xiaoxue Chen, Zhaoxin Fan, and Hao Zhao. Ultraman Ultra-fast and high-resolution texture generation for 3d human reconstruction from single image. 2025c.
Qihua Chen, Yue Ma. Hongfa Wang. Junkun Yuan, Wenzhe Zhao, Qi Tian, Hongmei Wang, Shaobo Min, Qifeng Chen, and Wei Liu. Follow-your your-canvas. Higher-resolution video outpainting with extensive content generation. arXiv preprint 210i:::240900033. 2024.
Xiaoxue Chen, Tianyu Liu, Hao Zhao, Guyue Zhou, and Ya-Qin Zhang. Cerberus transformer: Joint semantic, affordance and attribute parsing. Proceedings of the IEFF/CVP Conference on Computer Vision and Pattern Recognition, pp. 19649-19658, 2022.
Gang Cheng, Xin Gao, Li Hu, Siqi Hu, Mingyang Huang. Chaoman Ji, Ju Li, Dechao Meng, Jinwei Qi. Penchong Qiao, et al. Wan-animate: Unified character animation and replacement with holistic replication. arXiv preprint arXiv:2509 14055, 2025.
Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente. Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object understanding. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 21 121121161112 1366, 2022.
10

---

Published as a conference paper at ICLR 2026
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2019.
Yichun Shi Di Chang, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Xiao Yang. and Mohammad Soleymani. Magicdance: Realistic human dance video generation with motions & facial expressions transfer. arXiv preprint arXiv:2311. 2(3):4 2023.
Keyan Ding, Kede Ma, Shiqi Wang. and Eero P Simoncelli. Image quality assessment: Unifying structure and texture similarity, IEEE transactions on pattern analysis and machine intelligence, 44(5):2567 2581, 2020.
Mengyang Feng, Jinlin Liu, Kai Yu, Yuan Yao, Zheng Hui, Xiefan Guo, Xianhui Lin, Haolan Xue, Chen Shi, Xiaowen Li, ct al. Dreamoving: A human video generation framework based on diffusion models. arXiv preprint arXiv:2312.05107, 2023.
Mihai Fieraru, Mihai Zanfir, Elisabeta Oncata, Alin-lonut Popa, Vlad Olaru, and Cristian Sminchisescu. Reconstructing three-dimensional models of interacting humans. arXiv preprint arXiv:308.00184, 2023.
Google. Veo 3.1: Video generation, 2025. URL wtt.:httss//aistudiiiii gow.aaittttooooooooooooooo www.c/ooooooocccce http:///..oooooo.coooooooo//////
Hongeheng Guo, Wei Zhang, Junhao Chen, Yaonan Gu, Jian Yang, Junjia Du, Binyuan Hui, Tianyu Liu, Jianxin Ma, Chang Zhou, and Zhoujun Li. Iw-bench: Evaluating large multimodal models for converting image-to-web. 2024.
abs/ 2409.1899999
Haonan Han, Rui Yang. Huan Liao, Jiankai Xing, Zumnan Xu, Xiaoming Yu, Junwei Zha. Xiu Li, and Wanhua Li. Reparo: Compositional 3d assets generation with differentiable 3d layout alignment. arXiv preprint arXiv:2405.18525, 2024.
Yuze He, Yushi Bai, Matthieu Lin, Wang Zhao, Yubin Hu, Jenny Sheng. Ran Yi, Juanzi Li, and Yong- Jin Liu. T @ bench: Benchmarking current progress in text-to-3d generation. arXiv preprint arXi:22110009977, 2023.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscorc: A reforence-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7514-7528, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge a local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, Systems, volume 30. Curran Associates, Inc., 2017.
S. Vishwanathan, and R. Garnett (eds.). Advances Neural Information Processing
Jonathan Ho. Tim Salimans, Alexey Gritsenko. William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 01:8633-8666, 2022.
Li Hu. Animate anyone: Consistent and controllable image-to- video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8153-8163, 2024.
Jiancheng Huang. Mingfu Yan, Songyan Chen, Yi Huang, and Shifeng Chen. Magicfight: Personalized martial arts combat video generation. In Proceedings of the 32nd ACM International Conference on Multimedia, pp- 10833-10842, 2024.
Kaiyi Huang. Kaiyue Sun, Enze Xie, Zhenguo and Xihui Liu. Tži-compeench: A comprehensive benchmark for worn-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-78747, 2023.
Yasamin Jafarian and Hyun Soo Park. Learning high fidelity depths of dressed humans hy watching social media dance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 12753-12762, 2021.
Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17191-17202, 2025.
Glenn Jocher. Ayush Chaurasia, and Jing Qiu. Yolo by http://s///tttt..oo///tt//sa///s ultralytics 2023.
Johanna Karras, Aleksander Holynski. Cing-Chun Wang, and Ira Shhiacheer--Sieemma... Dreampose: Fashion image-to-video synthesis via stable diffusion. In 2023 /EFF/CVF International Conference on Computer Vision (ICCV), pp. 22623-22633 IEEF, 2023.

---

Published as a conference paper at ICLR 2026
Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to-image diffusion models are zero-shot vidco generators. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15954-15964, 2023.
Rawal Khirodkar, Jyun-Ting Song. Jinkun Cao, Zhengyi Luo, and Kris Kitani. Harmony4d: A video dataset for in-the-wild close human interactions. Advances in Neural Information Processing Systems, 37:107270- 107285, 2024.
Weijie Kong, Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 2024.
Kuaishou. Kling: A multimodal diffusion transformer for high-fidelity video generation. Technical report and product release, 2025. Kling video generation model (Kuaishou).
Hongxiang Li, Yaowei Li, Yuhang Yang. Junjie Cao, Zhihong Zhu, Xuxin Cheng. and Long Chen. Dispose: Dis- entangling pose guidance for controllable human image animation. In The Thirteenth International Conference on Learning Representations, 2025a. URL bottps://pperrev/// www.iiewwweeeeeeeeeeeeeeeeeeeeee nwt. forrmm d-AumOa 10MKG
Quanhao Li, Zhen Xing, Rui Wang, Hui Zhang, Qi Dai, and 7uxuan Wu. Magicmotion: Controllable video generation with dense-to-sparse trajectory guidance. arXiv preprint arXiv:2503. /6421, 2025h.
Ronghui Li, Junfan Zhao, Yachao Zhang, Mingyang Su, Zeping Ren, Han Zhang, Yansong Tang, and Xiu Li. Finedance: A fine-grained choreography dataset for 3d full body dance generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10234-10243, 2023.
Ronghui Li, YuXiang Zhang, Yachao Zhang, Hongwen Zhang, Jie Guo, Yan Zhang, Yebin Liu, and Xiu Li. Lodge: coarse to fine diffusion network for long dance generation guided by the characteristic dance primitives. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
1524-1534, 2024.
Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa. Ai choreographer: Music conditioned 3d dance generation with aist++. In Proceedings of the IEEE/CVF international conference on computer vision, pp.
13401-134412 2021.
Mingxiang Liao, Qixiang Ye, Wangmeng Zuo, Fang Wan, Tianyu Wang. Yuzhong Zhao, Jingdong Wang, Xinyu Zhang, et al. Evaluation of text-to-video generation models: A dynamics perspective. Advances in Neural Information Processing Systems, 37:109790-109886 2024.
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.0013/ 2024.
Tsung-Yr Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision (ECCV), pp. 740-755, 2014.
Haotian Ling, Zequn Chen, Qiuying Chen, Donglin Di, Yongjia Ma, Hao Li, Chen Wei, Zhulin Tao, and Xun animation. In The
Yang. Everybodydance: Bipartite graph based identity correspondence for Thirty-ninth Annual Conference on Neural Information Processing Systems.
Jiahe Liu, Youran Qu. Qi Yan, Xiaohui Zeng, Lele Wang, and Renjie Liao. Fr\, 'echet video motion distance: metric for evaluating motion consistency in vidcos. arXiv preprint arXiv:2407. 16/24, 2024a.
Tianqi Liu, Zhaoxi Chen, Zihao Huang, Shaocong Xu, Saining Zhang, Chongjie Ye, Bohan Li, Zhiguo Cao, Wei Li. Hao Zhao, et al. Light-x: Generative 4d video rendering with camera and illumination control. arXiv preprint 2512v2251..555 051 2025.
Xinyu Liu, Beiwen Tian, Zhen Wang. Rui Wang, Kehua Sheng, Bo Zhang, Hao Zhao, and Guyue Zhou. Delving into shape-aware zero-shot semantic segmentation. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2999-300900 3009, 2023a.
Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng. Raymond Chan, and Ying Shan, Evalcrafter: Benchmarking and evaluating large video generation models. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 22139-22149, 2024b.
12

---

Published as a conference paper at ICLR 2026
Yuanxin Liu, Lei Li, Shuhuai Ren, Rundong Gao, Shicheng Li, Sishuo Chen, Xu Sun, and Lu Hou, Fetv: benehmark for fine-grained evaluation of open-domain text-to-video generation. Advances in Neural Information Processing Systems, 36:62352 62387, 2023b.
Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Laura Leal- Taixé, Daniel Cremers, lan Reid, Stefan Roth, Simone Milani, Alexander Kirillov, and Paul Voigtlaender. Hota: A higher order metric for evaluating multi-object tracking. International Journal of Computer Vision, 129())5588 578- 2021.
Yuxuan Luo, Zhengkun Rong. Lizhen Wang, Longhao Zhang, Tianshu Hu, and Yongming Zhu. Dreamactor- ml: Holistic, expressive and robust human image animation with hybrid guidance. arXiv preprint arXiv:2504.01724, 2025.
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 4117-4125, 2024.
Neelu Madan, Andreas Mogelmose, Rajat Modi, Yogesh S Rawat, and Thomas B Moeslund. Foundation models video understanding: A survey. Authorea Preprints, 2024.
Vongani Maluleke, Lea Müller, Jathushan Rajasegaran, Georgios Pavlakos, Shiry Ginosar, Angjoo Kanazawa, and Jitendra Malik. Synergy and synchrony in couple dances. arXiv preprint arXiv:2409.044H0 2024.
Thomas Mclistas, Nikos Spyrou, Nefeli Gkouti, Pedro Sanchez, Athanasios Vlontzos, Yannis Panagakis, Giorgos Papanastasiou, and Sotirios Tsaftaris. Benchmarking counterfactual image generation. Advonces in Neural Information Processing Systems, 37:133207-133223 2024.
Xingyu Miao, Junting Dong. Qin Zhao, Yuhang Yang. Junhao Chen, and Yang Long. From frames to sequences: Temporally consistent human-centric dense prediction, 2026. URL htttpe://awx....rg/////
01661
Anush K Moorthy and Alan C Bovik. Efficient motion weighted spatio-temporal video ssim index. In Human Vision and Electronic Imaging XV, volume 7527, pp. 440-448. SPIE, 2010.
OpenAL. Sora: Creating video from text.
2024. Accessed: 2024-06-01.
Bohao Peng, Jian Wang. Yuechen Zhang, Wenbo Li, Ming-Chang Yang, and Jiaya Jia. Controlnext: Powerful and efficient control for image and video generation. arXiv preprint arXiv:2408. 06070, 2024.
Dustin Podell, Zion English, Kylc Laccy, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Nikhila Ravi, Valentin Gabeur, Yuan- Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam 2: Segment anything in images and videos, 2024, URL http://awaww.....////a//aaaaaaa/ /200 00714
Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance measures and data set for multi-target, multi-camera tracking. In European Conference on Computer Vision (ECCV), pp. 17-35, 2016.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolttion image synthesis with latent diffusion models. In CVPR, 2022.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-viden data. arXiv preprint arXiv:2209.14799 14792. 2022.
Kunpeng Song, Tingbo Hou, Zecheng Hc, Haoyu Ma, Jialiang Wang, Animesh Sinha, Sam Tsai, Yaqiao Luo, Xiaoliang Dai, Li Chen, et al. Directorlim for human-centric video generation. arXiv preprint 24X.::2412.1448. 2024.
Tomás Soucek and Jakub Lokoe. Transnet v2: An effective deep network architecture for fast shot transition detection. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 11218-11221, 2024.
13

---

Published as a conference paper at ICLR 2026
Kaiyue Sun, Kaiyi Huang. Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2x-compbench: A comprehensive benehmark for compositional text-to-vidoo generation. arXiv preprint arXiv:2407.14505 2024a.
Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang. Aojun Zhou, Zipeng Qin, Yi Wang, et al. Journeyd:: A benchmark for generative image understanding. Advances in neural information processing systems, 36:49659-49663 2023.
Mingze Sun, Junhao Chen, Junting Dong, Yurun Chen, Xinyu Jiang, Shiwei Mao, Puhua Jiang, Jingho Wang. Bo Dai, and Ruqi Huang. Drive: Diffusion-basse rigging empowers generation of versatile and expressive characters. arXiv preprint arXiv:2411. 177.23,,,,, 2024b.
Mingze Sun, Chao Xu, Xinyu Jiang. Yang Liu, Baigui Sun, and Ruqi Huang. Beyond talking- generating holistic 3d human dyadic motion for communication. International Journal of Computer Vision, 133(5):2910-22 2926, 2025.
Yumlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang, Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, et al. Vidco understanding with large language models: survey, IEEE Transactions on Circuits and Systems for Video Technology, 2025.
Beiwen Tian, Mingdao Liu, Huan-ang Gao, Pengfei Li, Hao Zhao, and Guyue Zhou. Unsupervised road anomaly detection with language anchors. In 2023 IEEE international conference on robotics and automation (ICRA). pp. 7778-7785 IEEE, 2023.
Linrui Tian, Siqi Hu, Qi Wang, Bang Zhang, and Liefeng Bo. Emo2: End-effector guided drivee avatar video generation. arXiv preprint arXiv:2501.1068) 2025.
Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stablcanimator: High-quality ileenttty-presering human image animation. arXiv preprint arXiv:2411.17699 2024.
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metrie for video generation. 2019.
Daniel Vlasic, Ilya Baran, Wojeiech Matusik, and Jovan Popović. Articulated mesh animation from multi-view silhouettes. In Acm Siggraph 2008 papers, pp. 1-9. 2008.
Jiangshan Wang, Yue Ma. Jiayi Guo, Yicheng Xiao, Gao Huang, and Xiu Li. Cove: Unleashing the diffusion feature correspondence for consistent video editing, arXiv preprint 20x0::2406008800, 2024a.
Jiangshan Wang. Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang. Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprint 1..442241..0476,, 2024b.
Qilin Wang, Zhengkai Jiang, Chengming Xu, Jiangning Zhang, Yabiao Wang, Xinyi Zhang, Yun Cao, Weijian Cao, Chengjic Wang, and Yanwei Fu. Vividpose: Advancing stable video diffusion for realistic human image animation. arXiv preprint arXiv:2405. 18156, 2024c.
Tan Wang. Linjee Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang. Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9326-9336, June 2024d.
Xiang Wang. Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang. Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion models consistent human image animation. Science China Information Sciences, 2025a.
Xiang Wang, Shiwei Zhang, Longxiang Tang, Yingya Zhang, Changxin Gao, Yuehuan Wang. and Nong Sang. Unianimate-dit: Human image animation with large-scale video diffusion transformer. arXiv preprint arXiv:2504.1 1289, 2025h.
Yuchi Wang, Junliang Guo, Jianhong Bai, Runyi Yu, Tianyu He, Xu Tan, Xu Sun, and Jiang Bian. Instructavatar: Text-guided emotion and motion control for avatar generation. In Proceedings of the AAAI Conference on Artificill Intelligence, volume 39, pp. 81 8132-8140, 2025c.
Zhenzhi Wang. Yixuan Li, Yanhong Zeng. Youqing Fang, Yuwei Guo, Wearan Liu, Jing Tan, Kai Chen, Tianfan Xue, Bo Dai, et al. Humanvid: Demystifying training data for camera-controllable human image animation. In The Thirt Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024c.
Zhenzhi Wang, Yixuan Li, Yanhong Zeng, Yuwei Guo, Dahua Lin, Tianfan Xue, and Bo Dai. Multi-identity human image animation with structural vidco diffusion. arXiv preprint arXiv:2504.04/222 2025d.
14

---

Published as a conference paper at ICLR 2026
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from erfor visibility structural similarity, IEEE Transactions on Image Processing, 13(4);600- 612, 2004.
Xunzhi Xiang, Haiwei Xue, Zonghong Dai, Di Wang. Minglei Li, Ye Yue, Fei Ma, Weijiang Yu, Heng Chang. and Fei Richard Yu. Remask-animate: Refined character image animation using mask-guided adapters. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 8628-8628 2838-866.222 2025.
Jiaqi Xu, Xinyi Zou, Kunzhe Huang. Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, and Jun Huang. Easyanimate: perforreformance long video generation method based on transformer architecture. arXiv preprins arXiv:2405. 18991, 2024a.
Zhe Xu, Kun Wei, Xu Yang, and Cheng Deng. Do you guys want dance: zero-shot compositional human dance generation with multiple persons. arXiv preprint 2rXiv:2401.33300 2024b.
Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Wia Liu, Chenxu Zhang, Jiashi Feng. and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1481-1449 1-490, 2024c.
Jingyun Xue, Hongfa Wang, Qi Tian, Yue Ma, Andong Wang, Zhiyuan Zhao, Shaobo Min, Wenzhe Zhao, Kaihao Zhang, Heung-Yeur Shum, et al. Follow-your-pnse v2: Multiplc-condition guided character image animation for stable pose control. arXiv preprint arXiv:2406.03303 2024.
Peng Yan, Xuanqin Mou, and Wufeng Xue. Video quality assessment via gradient magnitude similarity deviation of spatial and spatiotemporal slices. In Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2015, volume 9411. pp. 182- 191. SPIE, 2015.
Cheng- Yen Yang, Hsiang- Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang, Samurai: Adapt- ing segment anything model for zero-shot visual tracking with motion-aware memory. arXiv preprint arXiv:2411.11922 2024.
Peiqing Yang, Shangchen Zhou, Jixin Zhao, Qingyi Tao, and Chen Change Loy. MatAnyone: Stable video matting with consistent memory propagation. In CVPR, 2025a.
Xiuyu Yang. Bohan Li, Shaocong Xu, Nan Wang, Chongjie Ye. Zhaoxi Chen, Minghan Qin. Yikang Ding, Zheng Zhu, Xin Jin, et al. Orv: 4d occupancy-centric robot video generation. arXiv preprint 27X71225506.0379, 2025b.
Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effective whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2210-4220, 2023.
Xiaojun Ye, Junhao Chen, Xiang Li, Haidong Xin, Chao Li, Sheng Zhou, and Jiajun Bu. Mmad: Multi-modal movie audio description. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 11415-11428, 2024.
Wanqi Yin. Zhongang Cai, Ruisi Wang, Fanzhou Wang, Chen Wei, Haiyi Mei, Weiye Xiao, Zhitao Yang. Qingping Sun, Atsushi Yamashita, Lei Yang, and Ziwei Liu. Whae: World-grounde humans and cameras. In European Conference on Computer Vision pp. 20-37. Springer, 2024.
Yifei Yin. Chen Guo, Manuel Kaufmann, Juan Jose Zarate, Jie Song, and Otmar Hilliges. Hi4d: instance segmentation of close human interaction. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 17016-17027, 2023.
Jae Shin Yoon, Lingjie Liu, Vladislav Golyanik, Kripasindhu Sarkar, Hyun Soo Park, and Christian Theobalt. Pose-guided human animation from a single image in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15039 15048, 2021.
Baoli Zhang, Haining Xie, Pengfan Du, Junhao Chen, Pengfei Cao, Yubo Chen, Shengping Liu, Kang Liu, and Jun Zhao. Zhujiu: A multi multi-dimeesionnll, In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 479-494, 2023a.
chinese benchmark for large language models.
Beiyuan Zhang, Yue Ma, Chunlei Fu, Xinyang Song, Zhenan Sun, and Ziqiang Li. Follow-yourrmultiosse Tuning-free multi-character text-to-video generation via pose guidance. arXiv preprint arXiv:2412.16495 2024a.
Lumin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 3836-3847, 2023b.
15

---

Published as a conference paper at ICLR 2026
Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mimiemo- tion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406. 19680, 2024b.
Kaiyang Zhou and Tao Xiang. Torchreid: A library for deep learning person re-identification pytorch. arXiv preprint arxiv:99100 10093, 2019.
Kaiyang Zhou, Yongxin Yang. Andrea Cavallaro, and Tao Xiang. Omni-scale feature learning for person re-identification. In ICCV, 2019.
Kaiyang Zhou, Yongxin Yang. Andrea Cavallaro, tions for person re-identification. TPAMI, 2021.
Xiang. Learning generalisable omni-scale representa-
Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu. Jie Hu, and Yunhe Wang. Genimage: A calllonn-scale benchmark for detecting ai- ai-generated image. Advances in Neural Information Processing Systems, 36:77771-777 177782, 2023.
Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision. pp. 145 162. Springer, 2024a.
Tianyi Zhu, Dongwei Ren, Qilong Wang, Xiaohe Wu, and Wangmeng Zuo. Generative inbetweening through frame-wise wisc conditions-driven vidco generation. arXiv preprint arXiv:2412.11755, 2024b.
16

---

Published as a conference paper at ICLR 2026
A DETAILS OF THE PROPOSED COMPONENTS
A.1 MASKPOSEADAPTER
Relying solely pose keypoints (pose maps) makes it difficult to distinguish different indi- viduals multi-person scenarios; directly treat- ing binary tracking masks as additional chan- nels would compromise the translational in-
MaskPose Adapter
Σ
Residual Blend 0.95-Inte
Reshape Pase Lentens
a-Mean
variance of the pose encoder. We therefore propose MaspPoseeddaptr:: first performing masks in the "pose feature space," then injecting them into pose latents using gated-weighting strategy, and finally applying cross-person soft-attention to reorder per-person importance. Fig. il- lustrates MaskPose/Aaptte which fuses inde- pendent pose streams and masks into single pose-mask latent F E BxC61x661
or
lightweight transformations
Softmax
Attention Pre-Conv
Intogration
LayerNora
C
LayerNorm
R
Pose Enhancer
branch
Pose Enhancer
pose Tcatures
Σ
Raw pose Teatures
Σ
Pose Gate
Mask Gate
Pose Gate
Plask Gate
Per-person Pose Encoding. For each person an independent PoseNet processes the RGB pose map P; 1R355126522 PoseNet con- sists of eight convolutional layers, expanding the channels from 3 128, followed by a convolution, with weights shared across all per- sons. The output pose features are then scaled by learnable factor s. The final output can be expressed as:
Posellet
Mask processer
Posellet
Plak processer
Person 1 Masks
Person Masks
Person 1 Poses
Person Poses
Figure 5: Details of the MaskPoseAdapter
ppose s Convixi (PoscNet(P:))
R Cx61x61 ==320,
Binary human/facial masks M, E f0, 1) 11x11xx2122512 are processed through two convolutional layers to produce a 3-channel feature map:
Light Mask Processor.
fmask = (M,) 3x3x44644
(9)
which preserves contour information while avoiding mask features from dominating the pose features.
Gate-based Fusion. We apply per-pixel gates control how much of the pose and mask features to trust. These gates are implemented as convolutional layers followed by Sigmoid activations. The gate outputs are:
pose (F w mask an((f )):
(10)
where 7 and n are each a Conv=SiLU~Con==S smoidd sequence. The gated features are then combined with learnable weight À RJ 0.8
i = Aw woooe
pose
prose +
(1 A) wi
mask @ pmask
(11)
ID-dominant
line mask
where fpose is the pose feature reduced to 3 channels for gating. residual link is added refine the fusion, where the coefficient Otes controls the strength of the residual term:
f, f, + Orres ((1 A) w mask
f.mas)).
Ores 0.5.
(12)
Pose Enhancer. The fusion output passed through a lightweight PoseEnhancer module consisting of 3 convolution, followed by SiLU activation and BatchNorm, and 1 convolution:
h, Posennnaacer(f).
(13)
To further refine the pose features, scaling factor Sp 1.5 is applied to the raw features before final integration:
f 8p (1 Orres) h;-
(14)
17

---

Published as a conference paper at ICLR 2026
LayerNorm and Attention. Each of the enhanced pose features f; is normalized per-channel using LayerNorm, resulting f. The normalized features are concatenated along the channel dimension and processed through lightweight attention mechanism consisting of three x 1 convolution layers, each followed by BatchNorm and ReLUU. This generates attention logits & for each person:
t = Φ LayerNorm(f]) ..., LayerNormn(xx) E I Nx64x64
(15)
These logits are normalized across the person dimension using temperature softmax function:
art Softmax Deitmap (C), SoftmaxWithTemp,. = softmax(x/r),
(16)
where 7 is a learnalee temperature parameter.
Cross-Person Integration. The normalized features are integrated via attention- weighted sum
N i=1 Oan, f,, followed by 1 x convolution and residual averaging:
0.95 Convixi(S) 0.05
f
(17)
where F E Cx64x64 is reshaped to (B, C, 64, 64) and injected into the UNet.
A.2 MULTIFACE ENCODER
MultiFaceEncoder
Fig. Gillustrates MultiFace Encoder. For every mini-batch receive E RNxBx with Daa 512 and Delip = 1024, where the first axis enumerates the N identities and the second the samples in the batch. Each sample also carries a length- CLIP embedding eclip E Bx1xxxpp which is used key/value memory in all cross-
Refined Face Emeddings
C
attention steps.
Repeat for each
Stage Per-identity token projection. For identity E (L...., ......,, NN and sample b we transform the ArcFace vector e. with a two-layer MLP (Linear (512, 1024) + GELU + Linear (1024, KD) and reshape into K learnable tokens
person
FFN
FacePereiv er
x4
of width D = 768:
% MLP MLP2XGELU(E,)) RRD.
(18)
(0)
LN (eeehhppexx((X..)) E KXD (19)
Image Embeddings ings
FFN
Stage II FacePerceiver refinement. The K latent tokens ti
Cross Attn
query a lightweight FacePerceiver with depth Lp = 4:
((+1) L0 +FFNF + (1 (0 e ) ..... 3.
LayerNorm
1.6
(20)
Queries originate from the latent tokens, whereas keys/values are the concatenation of the projected CLIP embedding and the tokens (cf. veicceiveetteees in on in the code). A residual shorteut controlled by the flags shortcut, scale (A) reproduces the exact behaviour of MultiFace Encoder:
FFN
ID embeds list
Figure 6: Details of the Multi-
shortcut = 0,
(21) Face Encoder.
t
(4) 1bb
A (4)
shortcut 1.
Stage III Multi-person concatenation. After processing all identities with shared weights, the refined tokens are stacked along the sequence axis:
Tp t1.6i t2 ...; tNa E (NK))x
TE BBBKxXX for the batch.
(22)
The UNet's cross-attention layers can therefore read T directly, gaining NK extra tokens without any architectural change.
18

---

Published as a conference paper at ICLR 2026
B ABLATION STUDY
B.I DATASET ABLATION STUDY
Ablation study on the datasets have been compared in the main text in Tabs. 2 and StablcAnimator (Tu et al., (024)) fine-tuned for 40 epochs on the swing dance dataset (Maluleke et al. 2024) (StableAnimator
9
Dataswing) shows significant improvement over the original pre-trained weights provided by the authors, but still performs noticeably worse than Dance Tog trained for 20 epochs on the same Swing dance dataset (DanceTog w. Dataswing)- Using training data except PairFS-4K (Dance Tog w. Data clearly performs better than the model trained only on the swing dance dataset (Danee' Tog w. Dallswing): but still underperforms compared to DanceTog trained on all data including PairFS-44 SSK (denoted as Dance'Tog Dalafut + Dala PairPaiiFS)
B.2 ABLATION STUDY ON SUB-MODULES AND INPUTS
Tab. 6 presents ablation results for the new model and multi-input approaches proposed in Dance
The following configurations evaluated:
w/o mask input: The model does not use a separate mask input during the input process.
w/o pose input: The model does use a separate pose input during the input process.
w/o MaskPoseeAapter The original PoseNet is used instead, where poses of all individuals are directly fed as condition inputs to the model.
w/ SimpleFusion: Instead of using MaskPose Adapter's fusion strategy, the Pose Enhancer latents from individual person branches are directly summed element-wise, bypassing the multi-person attention mechanism.
w/o MultiFaceEncode The original FaceEncoder is used instead, where only the embedding of the largest detected face in the reference image serves the condition input.
Track I: Siaatty-Consistang
Track 2: Interaction-Cohceeee
Track 3: Vidco Quality
Model Variant
HOTAT
MOTA?
IDF1
MPJPE
OKSt
Pos:SSIMt
FVMDx10*1
PSNRT
FVDJ
PIDJ
C-FID]
w/o mask input
33.63
15.48
42.49
1625.04
0.28
0.85
2.97
11.02
40.4
73.1
14.7
wko pase input
81.88
74.23
86.38
1292.33
0.46
0.85
4.91
14.98
19.7
58.1
w/o Aosppere
48.95
1692.55
11.19
SimpleFasion
58.23
46.15
65.71
1487.26
0.39
0.84
3.82
12.85
31,2
64.8
12.4
w/o FaleFeece
83.31
78.81
88.55
893.32
0.74
0.89
1.26
15.67
17.9
49,2
8.4
DanceTog
83.94
79.80
89.59
492.24
0.83
0.93
0.54
15.82
17.1
48.0
Table 6: Module ablation study.
Fig. 7 and present qualitative comparisons of our ablation studies. DanceTTogether is compatible with StableAnimator's "Inference with task and test samples focus full-body two-person interaction video generation rather than large-area face-mask talking heads or single-person half-body dance sequences, the benefit of HJB-based Face Optimization is less pronounced. In tests, inference without HJB-based Face Optimization at approximately 0.8 s/iteration, whereas enabling HJB-baseh Face Optimization reduces throughput about 15 s/iteration. Furthermore, our ablation study indicates that applying HJB-based Face Optimization does not significantly impact the quality of two-person interaction video generation. Consequently, all experiments reported in the main text for StableAnimmaor and Dance'Tog were performed without HJB bbsed Face Optimization.
Face Optimization" (Tu et al.] (2024). Since our
B.3 COMPARISON BETWEEN POSENET AND MASKPOSEADAPTER
Fig. @ishows the feature maps obtained by the original PoseNet and our proposed MaskPoseAdapter from consecutive frames with the same input. It can be clearly observed that the output of MaskPose strongly binds pose and mask information, enabling clear identification which ID each pose corresponds to, and still providing sufficient mask information even when input poses are missing in some occluded frames. In contrast, the original PoseNet's s output makes it difficult to distinguish each individual pose, and pose features may be lost in occluded frames.
19

---

Published as a conference paper at ICLR 2026
GT
w/o mask input
w/o pose input
w/o
MaskPose
Adapter
w/o
MultiFace
Encoder
Full
DanceTog
Figure 7: Ablation study animation results (1/2).
GT
w/o mask input
w/o pose input
w/o
MaskPose
Adapter
w/o
MultiFace
Encoder
Full
DanceTog
Figure 8: Ablation study animation results (2/2).
20

---

Published as a conference paper at ICLR 2026
StableAnimator finetune
MaskPoseAdapter (transfer from PoseNet pretrain)
MaskPoseAdapter (after full data training)
Stable Leaattom pretrain PoseNet
aatg PoseNet
Ai
Figure 9: Comparison of PoseNet and MaspPose outputs under identical frame inputs.
21

---

Published as a conference paper at ICLR 2026
residual 1 1ph==00. 001
residual, Lattual_alppha aaappphp-00...mm 00
residual a appa-ppp.0...mm 05
residual, Laadual__ppp Lapph==0. 1
L_apph=0.2 .2
residual Leatalllllpaa 1pph=00.3 33
residual, [
residual L Lpha=00. 4
residua residual_alphin _aalaa==00p 5
residua Lapph=00. 6
residual L_a 1pha=0.7
residual Lasuallallphinn 1pha=0.. 8
residual 1pha=0.. 9
Figure 10:
The effect of residual alpha MaskPoseAddapter output.
mask processor out channe 1s=1
mask processor out channe 1s=3
mask processor out channels=33
Figure 11: The effect of different output channel numbers in the Light mask processor MaskPoseAdapter output.
22

---

Published as a conference paper at ICLR 2026
B.4 EXPERIMENTS ON RESIDUAL ALPHA AND MASK PROCESSOR
Fig. 10 and Fig. illustrate the influence of various Light mask processors and the parameter Ores the feature maps generated by MaskPoseAdapter. Through extensive experimentation, determined the optimal number of output channels for the Light mask processor and the value of a that effectively balances the pose and mask features in the output feature maps of MaskPoseAdapter In practice, when training on the full dataset, we set Opes 0.5 and employ a Light mask processor with 3-channel output.
C DETAILS OF DATA CURATION PIPELINE
Due to the limitations of existing two-person interaction datasets (Maluleke et al. 2024: Sun et al. 2025), which fail to simultancously provide identity diversity, static backgrounds, and fixed camera positions, propose novel data processing pipeline that recovers tracked human pose estimations from monocular RGB videos. Our pipeline extracts independent pose sequences, human silhouette masks, and facial masks for distinct individuals. We collected over 170 hours of paired figure skating videos from the internet and curated more than 26 hours of high-quality two-person figure skating segments, providing tracking masks, pose estimations, and facial masks for each individual subject ID. Additionally, we compiled a 1 1-hour humanoid robot dataset for fine-tuning our model to support controllable video generation tasks involving humanoid robots.
C.I DATASET COLLECTION
We collected various sanglepersone motion videos from existing research enrich identity information. including TiTtaDatts (Jafarian & Park (02T]) Champ (Zhu et 2024a). DisPose (Li et al.. (025a) and Human Vid (Wang et al. 2024e). Additionally, we gathered two-person interaction videos from existing research, including partner dancing, dual talking heads, and laboratory-recordae interactions from Swing Dance (Maluleke et al. 2024), Harmony4D (Khirodkar et al. (024), HI4D (Yin et al., 2023), CHI3D (Fieraru et al. 2023). and Beyond Talking (Sun et al.) (025). ). While synthetic data has been used for video generation training in prior work (Yin et al. 2024 Wang et al. (202e)), our method focuses controllable human interaction video generation in real-world scenarios,
did not use any synthetic data during training.
C.2 HUMAN TRACKING AND SUBJECT SELECTION
We first segment raw videos into scenes using TransNetV2 (Soucek & Lokoe 2024) and detect
humans using YOLOv8x (Jocher al. 2023). For each person crop p!. we extract 512-dimensional
identity features I using pre-trained OSNer (Zhou & Xiang 2019 Zhou et al. 2019 2021 }. Our enhanced tracking algorithm combines spatial proximity with Rell similarity
maintain consistent
identities across frames. From all tracked identities, we select the two main subjects based on coverage (appearance frequency > 40%), consistency, and quality score Q 0.7 Coverage, + 0.3 Consistency,
C.3 ANNOTATION GENERATION
Starting from the bounding boxes b; of key frames, SAMURAI Yang et al. 2024) bidirectionally propagates masks throughout the video sequence. We extract pose information of 133 keypoints using DWPose (Yang et al. 2023) and assign each pose to independent subject IDs via an IOU matching approach utilizing the masks generated by SAMURAL MatAnyone (Yang et al. 2025a) produces high-quality alpha mattes from SAMURAI masks, providing data for tasks requiring background replacement (Sun et al. 2025). The complete data processing pipeline is illustrated in Fig. B] where our Data Curation Pipeline generates four outputs from RGB video input: independent mask sequences, pose sequences, and facial mask sequences for each individual, as well as alpha mask sequences for all subjects.
23

---

Published as a conference paper at ICLR 2026
C.4 DATA FILTERING
Clips are automatically filtered based on: bbox overlap (max IoU < 0.1), size validation (2% < bbox area < 80% of frame), exact 2 primary subjects with 2 40% coverage, and temporal consistency (> 90% successful tracking). For PairFS-4K, we additionally perform manual curation to ensure high-quality two-person interactions with clear visibility and balanced representation of skating movements.
ArTHKA
$
Figure 12: Data curation pipeline results (1/4).
C.5 PAIRFS-4K DATASET PREPARATION PROCESS
We collected 932 figure skating videos from the internet, including numerous Olympic figure skating compilation videos with multiple shots. Using TransNetV2 (Soucek & Lokoc 2024), we developed an automatic segmentation script and employed HumanRelD and Yolox for identification and tracking of the main subjects. After manually filtering out segments that did not conform single-person or pair figure skating criteria, we obtained 4.8K figure skating segments with a total duration of approximately 26 hours, and average segment length of about 20 seconds. We train model on Dataset (Jafarian & Park 2021), Champ (Zhu et al., 2024a). DisPose (Li et al. 2025a). Human Vid (Wang al.) (024e). Swing Dance (Maluleke al.) 2024), Harmony4D (Khirodkar ct al. 2024). CHID (Fieraru et al. 2023), Beyond Talking (Sun et al.) 2025), and PairFS-4K, using resolutions of 512 x 512. Due to the limited number of unique identities in HI4D, we exclude from our training set. A detailed summary of all datasets is provided in Tab. [ PairFS-4K is the first two-person figure skating video dataset with over 7,000 unique identities.
D TOGETHERVIDEOBENCH BENCHMARK
D.1 VIDEO GENERATION BENCHMARK OVERVIEW
There have been many benchmarks for evaluating large generative models (Liu et al. 20236 Zhang
al. 2023a Sun al. 2024a Liu et al. 2024b Chen et al.! 2025b Collins et al.) 2022 He et al.
24

---

Published as a conference paper at ICLR 2026
P
# 
Figure 13: Data curation pipeline results (2/4).
Figure 14: Data curation pipeline results (3/4).
25

---

Published as a conference paper at ICLR 2026
Figure 15: Data curation pipeline results (4/4).
2023, Sun et al. 2023; Zhu et al. 2023 Mclistas et al. 2024 Huang et al. 2023 Guo et al.
2024
Chang et al. 2024). Recently, some video understanding methods have also been used evaluate the quality of generated videos (Tang et al., 2025; Liao et al. 2024 Liu et al. 2024b Ye et al.) 2024; Madan al. 2024). Despite this, the field of controllable video generation has lacked reliable evaluation benchmark. Recent controllable video benchmarks (AIST++ (Li et al.) 2021] )), Tok-alE (Jafarian & Park (20TT) have mainly focused on single-person dance or static portrait animations, overlooking the three key challenges faced by realistic multi-person generation: multi- identity consistency (avoiding identity confusion in long sequences), interaction coherence (ensuring physically reasonable and temporally smooth interactions), and strict conditional fidelity (precisely following pose, mask, or text control inputs). To systematically evaluate these dimensions, we propose Together VideBBehh featuring three orthogonal tracks- delis--dddntiiy-Cnssstennyy Interaction- Coherence, and Video suported by a unified, automated parsing pipeline that extracts per- person pose, mask, face-crop, and bounding-box representations for fair and reproducible assessment.
edentity-Consistenee To evaluate the ability of models to maintain consistent appearance and identity for each individual across long video sequences, we adopt standard multi-object tracking metrics, including HOTA (Luiten et al., 202T). MOTA (Bernardin & Stiefelhagen 2008), and IDF1 (Ristani al. 2016). These metrics comprehensively assess detection accuracy, association accuracy, and identity preservation, and are computed using the TrackEval toolkit (Luiten et al.) 2021). This track crucial for ensuring that generated videos do not suffer from identity switches or appearance confusion, especially in multi-person scenarios.
This track focuses on the temporal smoothness and physical plausibility of interactions between multiple humans, well the adherence to external control signals. We employ pose adherence (MPJPE-2D) (Cao et al., 2017). object keypoint similarity (OKS) (Lin et al. 2014). and the following metrics: pose structure similarity (PoseSSIM), motion smoothness (SSoothRMS) temporal dynamics error (SIMMS)) and Fréchet Video Motion Distance (FVMD) (Liu et al. 2024a) comprehensively evaluate the quality of human motion and interaction.
Interaction-Cohereent
Video Quality: To assess the overall visual fidelity and semantic consistency of generated videos, suite of widely adopted metrics, including SSIM (Wang al. 2004), FVD (Unterthiner
26

---

Published as a conference paper at ICLR 2026
al. 2019), FID (Heusel et al. 2017). CLIP (Hessel et al. 2021). and the following metrics: LPIPS. L1. PSNR, DISTS (Ding et al. 2020), ST-SSIM (Morrhy & Bovik 2010). GMSD-T Yan et al. 2013). These metrics collectively measure both the perceptual quality and the alignment of generated content with the intended conditions. We calculate the metrics for both the overall frame and the human mask region of each frame separately, shown in Fig. [6] Since the backgrounds of some evaluation data exhibit slight jittering, we believe that the quantitative evaluation of the human mask region is more indicative of human ID consistency and video quality in the generated videos than the quantitative evaluation of the full frame.
GT
Stable Animator
Error map person
Error map
person
We use individual human masks for each person to conduct quantitative evaluation. The eтror map shown in the figure is the L1 Loss error map, which calculates the pixel-level absolute difference between the GT and predicted images.
Figure 16:
All tracks share a unified Data Curation Pipeline that automatically extracts per-person pose, mask, face-crop, and bounding box for both ground truth and generated videos, ensuring reproducibility and fair comparison. For each video, we compute the relevant metrics for every individual and report the average across all videos in each group.
D.2 EVALUATION DATASET
While laboratory-recorded datasets such as Harmony4D (Khirodkar et al. (0224), HI4D (Yin et al. 2023), and CHI3D (Fieraru et al. 2023) provide precise annotations, their videos are typically limited to 3-12 seconds, feature single scenes, and involve minimal position exchanges between subjects. As result, they are insufficient for evaluating long-duration, multi-position, and realistic human interactions. To address this gap, have manually curated and edited 100 high-quality two-person interaction videas from public competitions, films, documentaries, and social media, forming the core evaluation set of Recheeecchh. These videos encompass wide range of real-world
27

---

Published as a conference paper at ICLR 2026
interaction patterns, including tethanng-inttnssvee swing and Lindy-Hop routines, Latin ballroom duets, pair figure skating, boxing, wrestling and combat sequences, partner acrobatics and acro-yoga throws, everyday social gestures (such handshakes and hugs), and two-person conversations. Each clip features exactly two performers, with nearly static cameras and backgrounds. Frequent occlusions, position exchanges, and physical contact between subjects introduce long-range motion, viewpoint changes, and identit--swichhng pallengessffcttonn making it suitable testbed.
D.3 METRICS
Below are the evaluation metries and computation procedures used the three tracks of Togeth- er erVideoBench. To ensure reproducibility, both ground-truth and generated videos are first processed by our Data Curation Pipeline (Sec. C which yields for each subject:
Pose sequences: 133 keypoints per frame via DWPose (Yang et al.
2023
Human masks: per-frame human masks via SAMURAI (Yang et al.
2024).
Bounding boxes: tight boxes around each human mask (for MOT eval Luiten et al. 2021
D.3.1 TRACK -OOTIISTECYYSS
IDFI1:
After frame-level association with the Hungarian algorithm, let IDTP, IDFP and IDFN be identity-trus true positives, false positives and false negatives.
2 IDTPI
IDF1 2 JUTPI + IDFP + JIFN
(23)
It is the harmonic mean of identity precision and recall and therefore measures how often the correct ID label maintained.
IDP IDR T:
Precision and recall components of IDFI.
IDTP
IDTP|
IDP
IDR
(24)
IDTP + IDFP
IDTP + [IDEN
HOTA 1:
Higher-Order Tracking Accuracy (Luiten 2021) decomposes into DetA (detection accuracy).
AssA (association accuracy) and LocA (localisation accuracy):
HOTA VDetA x AssA, LocA =1
Σ (1 IoU()))
(25)
TP
ЫЕТP
MOTA MOTP1: CLEAR-MOT summary:
MOTA = FP +
+N FN + IDSW
Erp (1 (1- IoU)
MOTP =
(26)
GT dets
TP
IDSW 4. FP FN
Absolute counts of identity switches, false positives and false negatives.
D.3.2 TRACK 2 CEEELACTION-CHHEEENN
All keypoints are first temporally aligned and isotropically scale-shift aligned via similarity transform.
MPJPF-2D J:
Let Stpj and Xtyj be the predicted and ground-truth pixel coordinates of joint of person p at frame t, after SIM3 alignment; T, P. denote total frames, persons, and joints. Then
MPJPE-2D TPJ
Keps Xxps llg
(27)
28

---

Published as a conference paper at ICLR 2026
OOKSS For each frame t. flatten over P x J valid keypoints. Let dy be the Euclidean error of the kth keypoint, σG its COCO standard deviation, and A the estimated person area. Then
OKS, 1 K
K
exp
20 [ 10-60)0)))
dk 10-6))
1 OKS
Σ OKS. T
(28)
Pose-Heat SSIM t:
Rasterise the set of keypoints at each frame into a Gaussian heatmap H() size H x W with = 4 px, then
1 T Σ (X.M( H(X.))
PoseHHeatSSIM
(29)
where X,, X, E PxJx2 are the keypoint arrays.
SmoothRMS Compute the third-order temporal derivative (jerk) of each trajectory, scaled by frame rate f:
d3
X
tpj
dt
Xtpj
x f
(30)
Then
SmoothRMS
X kpp 1
(31)
TPJ
Time-Dyn RMSE.::
With the second-order derivative (acceleration)
d² dtz
Ktpj =
Xtpj X f
(32)
define
TiMeDynRMSF.
1 TPJ
X X 11
(33)
FVMD L:
Model the velocity vectors of all keypoints as 2D Gaussians N(ps Ep) for prediction and N(p, Eg) for ground truth, where μ = E[x] and = Cov(x). Then
FVMD ||m My 11 + Tr(2ip En (2ip 21)
(34)
D.3.3 TRACK 3 VIDEO QUALITY
-I1::
Let [e c) and (x. y, c) be the ground-truit -tuth and predicted RGB pixel values at frame t, spatial location (x. y) and channel c, over T frames of size H x W and C channels. Then
TW H
L1 THWC
ΣΣΣΣ
T(,,y,c) f,(,,,,))-
(35)
PSNR t:
Compute the fer-frame mean squared error
MSE 1 W c
HWC ΣΣΣ (. -((x.,.))((22,...))))
(36)
then
255
PSNR 20 logıol
(37)
VMSE
29

---

Published as a conference paper at ICLR 2026
SSIM f:
For each frame and each channel c, compute
SSIM; SSIM (() Ie(.),))).
(38)
then average:
T TC
SSIM
ΣΣ SSIM,
(39)
LPIPS L:
On 256 256 crop, let Φe(-) be the f-th layer feature map and wy learned weights. Then
LPIPS T. Σ
H.W. we o - -
(40)
DISTS 4:
Let fe(-) be VGG16 feature maps, f their normalized versions, and G(-) the Gram matrix. Define
structure; (fe(1), f(()))
texture; MSE(G()(()), G(L())).
(41)
Then
DISTS 1 L Σ 0.5 (0.5 structurct 0.5 (1- (-texturce))
(42)
CLIPScore
We encode each frame from the ground-truth and generated videos into CLIP image embeddings Ut, ig E Rd, normalize them to unit vectors, and compute the frame-wise cosine similarity:
(43)
|eel |
The final CLIPScore is obtained by averaging all T frames:
T Σ
CLIPScore = T 8t
(44)
ST-SSIM t:
With window length w 3, define for each spatio-tempora block
SSIMsp SSIM (Je+((e I
(45)
then
1 T T-
ST-SSIM
SSIMap
(46)
GMSDDTemppral.. For each = 2..... T. let
(x
v) = VI
(o(=,)) vi
(47)
and
2 91 +
(M,
(48)
9 9
+E
Then
1
GMSD-Temporal
Var., Varz,(GMSS((r, (GM))
(49)
[T (7-1) W
t=
30

---

Published as a conference paper at ICLR 2026
FVD J:
Extract I3D features for each non-overlapping 16-frame clip, compute means Jry Hy and covari- ances E,. Sy then
FVD = 1 Tr(2,, + Mi - -2(2,
(50)
FID J:
On all frames, extract Inception- V3 features, form (1,, E,) and (Ky. Ey), and
FID ||,, 2 1 Tr(E, + -2(22,,))
(51)
CLIP-FID J:
Identical to FID but using CLIP embeddings instead of Inception features:
CLIP-FID = ||sr - | Tr(E, + E - 2(2))
(52)
All Track-3 metrics are reported both on the full frame and on each human mask (per-person); the final masked score is the arithmetic mean over the two performers.
E
USER STUDY
To assess perceptual quality, we conducted user study involving 50 participants (25 general users and 25 domain experts). Each participant viewed 20 video pairs in Pance'TogEvall 100 dataset (ground-trutt vs. generated) and rated the generated video using 5-point Likert scale across three dimensions. The evaluation criteria were defined follows:
Track 1: Identity-Consistnney "To what extent are the identity characteristics (face, clothing, body shape) of the individuals consistently preserved throughout the generated video?"
5: Identity is perfectly consistent at all times; individuals remain clearly distinguishable
even after position exchanges.
4: Identity is mostly consistent, with only minor and infrequent confusion.
3: Moderate consistency; noticeable identity confusion or drift occurs.
2: Frequent confusion; the two individuals are often hard to distinguish.
1: Identity consistency completely fails; individuals are indistinguishable.
"How natural and temporally coherent are the motions and interactions (e.g., holding hands, lifts, synchronized movements) between the individuals in the generated video?"
Track 2: Interaction-Coherees
5: Motions are highly natural and fluid; interactions are physically plausible and temporally smooth.
4: Motions are generally natural; interactions are mostly reasonable with occasional minor glitches.
3: Motions acceptable but somewhat stiff; interactions appear unnatural at times.
2: Motions are clearly unnatural; interactions are implausible and exhibit abrupt changes.
1: Motions extremely rigid; interactions completely unrealistic.
Track Video Quality "What the overall visual quality of the generated video? (e.g., sharpness, detail preservation, presence of artifacts)"
5: Visually sharp with rich details and almost no artifacts; quality approaches real footage.
4: Good quality with minor artifacts or slight blur in some frames.
3: Acceptable quality, but noticeable artifacts distortions
present.
2: Poor quality with frequent artifacts, blurriness, or unnatural textures.
1: Very poor quality with severe artifacts and obvious Al-generated appearance.
31

---

Published as a conference paper at ICLR 2026
E. RESULTS AND ANALYSIS
The average scores (out of 5) across 1,000 ratings per track are summarized in Tab. Our full model achieves the highest scores across all tracks, with average of 4.61/5.0, significantly outperforning all baselines.
Table 7: User study results (mean scores out of 5). Higher is better.
Identity Consistency
Interaction Coherence
Video Quality
Method
Average
Animate Anyone
2.31
2.47
2.68
2.49
Champ
1.89
2.21
2.34
2.15
MimicMotion
1.92
2.18
2.29
2.13
Human Vid
3.14
3.29
3.42
3,28
UniAnimate
2.78
2.91
3.05
2.91
Aiit
2,42
2.56
2.71
2.56
DisPose
1.95
2.23
2.37
2.18
Stimatnn
3.38
3.51
3.45
3,45
Atimatr w. Dataswing
3.89
4.02
4.11
4.01
Dance w. Dataswing
4.28
4.35
4.19
4.27
Dance w. Datafall
4.47
4.52
4.38
4,46
DanceTog Data full + Data
4.61
4.68
4.53
4.61
Analysis of Human Preference vs. Artifacts. Although quantitative metrics are strong, we acknowledge the visual challenges inherent in this task. The qualitative superiority of DanceTog is further evidenced by the user preference margin: specifically in Identity Consistenci, users preferred method over the second-best baseline (StablcAnimator) by margin of approximately 36.4% (4.61 vs 3.38).
While some generated samples may exhibit minor boundary artifacts synthetic appearance, attribute this primarily to two factors:
(i) Extreme Task Difficulty: The task involves complex multi-person interactions (e.g., oc- clusion, contact, position swapping) where baselines often suffer catastrophic failures (as shown in Fig. Fig. 29
contrast, our model maintains structural integrity during these
critical moments.
(ii) Data Pre-processing Artifacts: To train on diverse motion backgrounds, we utilized background replacement strategy based on extracted human masks. Occasional failures in the segmentation model (e.g., MatAnyOne) can introduce boundary noise in the training data, which propagates the generated results.
Despite these isolated artifacts, the user study conclusively demonstrates that human evaluators prioritize the semantic correctness (identity preservation and interaction logic) of our method over the pixel-perfect but identity- drifting results of baseline methods.
F ADDITIONAL QUANTITATIVE RESULTS
All Track3 metrics shown in Tab 8 and Tab The experimental results prove that DanceTog achieves better performance than StableAnimator when trained on the identical Swing Dance dataset.
G INFERENCE AND TRAINING EFFICIENCY
As shown in Tab. 10 Dance'Tog achieves competitive inference speed (0.88 fps) while maintaining superior generation quality. Our method is significantly faster than most recent baselines such as UniAnimate (0.25 fps) and MimicMotion (0.06 fps), and comparable to lightweight methods like Animate Anyone (0.97 fps) and StableAnimator (0.89 fps). Note that, like all human animation
32

---

Published as a conference paper at ICLR 2026
Table Comparison of models using Full Frame evaluation metrics.
Method
L14
PSNR1
SSIM?
LPIPS.
DISTS!
CLIP
ST-SSIM
GMSD-T] TL
FVD]
FID!
C-FID)
AnimateAnyose
37.32
13.23
0.49
0.56
0.27
0.54
0.42
108.2
118.1
27.7
Champ
43.70
0.36
125.7
25.6
MimicMotion
52.08
11.04
0.47
0.58
0.32
0.91
0.37
0.39
121.0
116.6
26.2
HumanVid Vid
38.95
13.67
0.:52
0.50
0.26
0.91
0.53
0.35
97.2
90.2
18.6
UniAnimate
37.95
13.62
0.55
01.53
0.29
0.89
0.61
0.42
132.0
151.2
42.8
UniAnimate-DDil
43.11
12.34
0.50
0.53
0.28
0.45
0.42
111.9
100.3
20.8
DisPose
SaabicAnimator
33:14
14.60
0.57
0:44
0.24
0.91
0.66
0.40
65.7
84.1
18.1
Subbickninattr w. Dalasenn
30.31
15.27
0.60
0/42
0.22
0.94
0.69
0.42
78,8
79.3
16.1
DanceTog Datitooing
32
15.12
0.59
0.44
0.23
0.91
0.68
0.38
79.3
82.1
14.7
Dance/Tog Togee/ogge Dalafall
29
DaneeTog Datarut Dotoparrs
15.85
0.22
0.70
0.39
76.3
75.1
Table 9: Comparison of models using Human Masked Region evaluation metrics.
Method
L1|
PSNRT
SSIM?
LPIPS)
DISTS)
CLIP
ST:SSIM
GMSD-T]
FVD]
FIDJ
C-FID)
Anyoee
59.92
10.45
0.92
0.06
0.70
0.18
44,8
101.4
19.2
Champ
MimicMotion
77.02
8.75
0.92
0.07
0.16
0.90
0.56
0.17
65.5
180.9
33,9
FumaaVid
47.97
12.13
0,93
0.05
0.12
0.91
0.76
0.15
31,9
72.4
14.2
UniAnimate
56. 34
11.05
0.92
0.06
0.13
0.92
0.70
0.17
45.0
109.8
21A
UnlAnimate-IDT"
0.14
21.5
76.75
0.16
0.17
64.7
364
StablcAnimator
48.51
12.00
0.93
0.05
0.12
0.93
0.75
0.15
38.4
71.8
15.7
StablcAnimator Datawwing
41.41
13.06
0.93
0.04
0.11
0.94
0.80
0.14
29.0
66.7
12.5
Dance Data..ing
34.49
14,76
0.94
0.05
0.09
0.85
0.14
21.5
57.5
9.5
DanceTog Data
20.6
Dance Tog Data Dalaparrs
30.14
15.82
0.94
0.03
0.08
0.95
0.87
0.14
17.1
48.0
7.9
methods, Dance Tog is not a real-time approach, but provides a practical balance between quality and efficiency for controllable multi-person video generation.
H APPLICATIONS: HUMAN-ROBOT INTERACTION VIDEO GENERATION
After fine-tuning on our HumanRob-300 humanoid-robot video dataset, DanceTogether can generate realistic interaction videos between a humanoid robot and a human (see Fig. 17). This demonstrates the effectiveness and generalization ability of Dance TonceToge,,er, offering new insights for embodied-Al and human-robot interaction research. After the robot and the human exchange positions, both agents retain their original identities. The method also handles fine-grained interactions, such as handshakes and sparring, remarkably well. This part of the video results can be found on the supplementary webpage.
I ADDITIONAL QUALITATIVE RESULTS
Fig. [ Fig. 18 and Fig. showwaaee the generalization capability of DanceTTog. With the same driving video and different reference images, DanceTog generates diverse outputs. Fig. 20 Fig. 21
Fig. 22 and Fig. 23 present qualitative comparisons across consecutive frames for different cases. The top row in each figure shows the input reference image and the corresponding pose sequence. The pose sequence is estimated from ground truth video, and the first frame is used as the reference image input for each baseline. Our proposed DanceTog method consistently outperforms all baselines in generating video frames with rich interaction details. Notably, it preserves individual identity features even when the two subjects exchange positions. For qualitative video comparisons, please refer the supplementary webpage.
Figs. 24 show qualitative comparisons of all baselines. We extracted consecutive frames where position swapping occurs. The leftmost column is the GT video. We used the first frame of the GT video as the reference image (not the first frame shown in the figures), and the dwpose results estimated from the GT video the pose condition input for each baseline (corresponding the GT images in the first column). Due to file size limitations, the images below are compressed. Please refer the webpage in the supplementary materials for the original videos.
33

---

Published as a conference paper at ICLR 2026
Figure 17: Using the first frame as the reference image, we perform inference on human-robot
interaction sequences conditioned on independent pose maps and human masks.
34

---

Published as a conference paper at ICLR 2026
Driving Video
Ref. Image
Figure 18: Additional motion transfer results (1/2).
Driving Video
Ref. Image
Figure 19: Additional motion transfer results (2/2).
35

---

Published as a conference paper at ICLR 2026
GT
Animate Anyone
HumanVid
UniAnimate
Stable
Animator
DanceTog
(Ours)
Figure 20: Additional animation results (1/4). The image with red borders is the reference images.
GT
Animate Anyone
HumanVid
UniAnimate
Stable
Animator
DanceTog
(Ours)
Figure 21: Additional animation results (2/4). The image with red borders is the reference images.
36

---

Published as a conference paper at ICLR 2026
GT
Animate Anyone
HumanVid
UniAnimate
Stable
Animator
DanceTog
(Ours)
Figure 22: Additional animation results (3/4). The image with red borders is the reference images.
GT
Animate Anyone
HumanVid
UniAnimate
IW
Stable
Animator
DanceTog
(Ours)
Figure 23: Additional animation results (4/4). The image with red borders is the reference images.
37

---

Published as a conference paper at ICLR 2026
Animate Anyone
Stable Animator
DanceTog (Ors))
GT
Champ
Mimicmotion HumanVid UniAnimate
Dispose
Figure 24: Additional animation results (1/6).
38

---

Published as a conference paper at ICLR 2026
Animate Anyone
Stable Animator
DanceTog
GT
Champ
Mimicmotion HumanVid UniAnimate
Dispose
(Ours)
Figure 25: Additional animation results (2/6).
39

---

Published as a conference paper at ICLR 2026
Animate Anyone
Stable Animator
DanceTog (Ours)
GT
Champ Mimicmotion HumanVid UniAnimate
Dispose
Figure 26: Additional animation results (3/6).
40

---

Published as a conference paper at ICLR 2026
Animate Anyone
Stable Animator
DanceTog
GT
Champ
Mimicmotion HumanVid UniAnimate
Dispose
(Ours)
Figure 27: Additional animation results (4/6).
41

---

Published as a conference paper at ICLR 2026
Animate Anyone
Stable Animator
DanceTog (Ours)
GT
Champ Mimicmotion HumanVid UniAnimate
Dispose
Figure 28: Additional animation results (5/6).
42

---

Published as a conference paper at ICLR 2026
Animate Anyone
Stable Animator
DanceTog (Ours)
GT
Champ
Mimicmotion HumanVid UniAnimate
Dispose
Figure 29: Additional animation results (6/6).
43

---

Published as a conference paper at ICLR 2026
Table 10: Inference speed and training configuration comparison. All inference speeds measured on a single A800 GPU (excluding model loading time). Training configurations indicate the computa- tional resources and time required. FPS: frames per second.
Method
Venue
Training Config
Open Source
Inference Speed (fps)
Our Method
Dance w. Daiis Dance Tog fula
8xA100 4 days (20 sp.)
0.88
nodes 8 A800, days (20 ep.) 9 nodes 8 A800, days (20 ep.)
0.88
Dance w. Dalaral Pu
0.88
Baseline Methods
Animate Anyone
2024
CVPR 2024
0.97
StablcAnimater [u
2024
CVPR 2025
8xA100, 4 days (20
0.89
Champ Zhu et nl. 2024n
ECCV 2024
1.42
Human
Wang al.
2024
Dispose Li
M125a
ICLR 2025
0.42
UniAnimale Wang clal.
2025a
SCIS 2025
MimicMoticm
22224
ICML 2025
0.06
UniAnimate-DDi Wang eral. 2
Training code available, supports -ual-peroon data input. : Open source but data format incompatible requires special parameters (e.g.. camera parameters).
x: Not open source.
ep.: epochs.
J LIMITATIONS
While DogeeTog achieves state-of-the-art performance on two-person interaction benchmarks, it has several limitations. First, our framework is optimized for up to two actors; extending it to handle larger groups would incur substantial computational and memory overhead and may require hierarchical factorized conditioning mechanisms. Second, the quality of generated videos depends heavily on the accuracy of the input pose and mask sequences. Severe occlusions, fast motion blur, failures in the underlying detectors (e.g., DWPose (Yang al. 2023), SAMURAI (Yang al., 2024)) can degrade identity preservation and interaction fidelity. Third, we assume a mostly static camera and relatively simple backgrounds; dynamic camera motion or highly cluttered scenes may introduce artifacts or identity confusion. Fourth, like most diffusion- methods, Dance DanceTogetherr Together computationally intensive and incurs non- non-trivial latency, limiting real-time applications.
K FUTURE WORKS
DanceTogether enables diverse applications in creative industries (film, gaming, VR/AR) and embodied-Al research. A particularly promising direction is robot imitation learning, where our framework can synthesize large-scale training data for human-robot collaboration (Albaba et al. 2025) without expensive real-world collection, as demonstrated by our HumanRob-306 results (Fig.
L THE USE OF LARGE LANGUAGE MODELS (LLMS)
Large language models (LLMs) were used only to assist with language editing and minor text improvements the preparation of this manuscript. They were not involved in the design of the research, the development of methods, the execution of experiments, or the interpretation of results. All scientific content, including analyses, conclusions, and contributions, remains the work of the authors.
44


Reviews:
[{"review_id": "X5ydHbrCoK", "reviewer": "Reviewer_9khT", "summary": "This paper presents DanceTogether, an end-to-end diffusion framework designed to generate interactive, multi-person videos without the identity drift that plagues existing methods. The system takes a single reference image and independent pose-mask control streams to produce long, photorealistic videos while strictly preserving each actor's identity. The primary technical contribution is the novel MaskPoseAdapter, which explicitly binds identity to motion by fusing robust tracking masks with noisy pose-maps at every denoising step. The authors also introduce significant contributions for training and evaluation: the PairFS-4K and HumanRob-300 datasets, and the TogetherVideoBench benchmark. The method demonstrates state-of-the-art results on their benchmark and shows strong generalization to human-robot interaction.", "questions": "1: Why is there a 0.95 factor in the Residual Blend shown in Figure 2?\n\n2: What does the PoseNet0401 mean? What's 0401?\n\n3: There seems to be a typo:  \"Pose Gete\" in Figure 2.\n\n4: What does \"Intg\" in Figure 2 mean?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1: The proposed DanceTogether aims to address a critical and well-defined problem: the failure of existing models to maintain actor identities during complex interactions. By explicitly fusing robust tracking masks (the \"who\") with semantic pose maps (the \"how\"), the model establishes a persistent identity-action binding throughout the diffusion process.\n\n2: The well-curated PairFS-4K (26 hours of dual-skater footage) and HumanRob-300 (1 hour of human-robot interaction) datasets are helpful for the community to dive into the area of multi-person interaction video generation, addressing a clear lack of large-scale, diverse multi-person interaction data.\n\n3: The proposed TogetherVideoBench is a comprehensive, three-track benchmark that thoughtfully evaluates the distinct challenges of Identity-Consistency, Interaction-Coherence, and Video Quality. \n\n4: Extensive experiments show the effectiveness of the proposed framework and datasets on multi-person interaction human animation.", "weaknesses": "1: **The presentation could be significantly improved.** For instance, Figure 2, which illustrates the core pipeline, is overly crowded, making it difficult for a reader to easily follow the main architectural components and data flow. Furthermore, the method description, particularly in Section 3.3 on pages 5 and 6, is saturated with low-level implementation details like specific activation functions (e.g., Sigmoid) and normalization layers (e.g., LayerNorm). This focus on minutiae tends to obscure the high-level innovations that differentiate the method from related works. This confusion is amplified by a disconnect between the diagram and the text. For example, terms like \"Attention Pre-Conv\" are used in Figure 2 but are not clearly defined in the text, while key steps described in the text, such as the \"Cross-Person Integration\" (Equation 13), are difficult to locate within the figure. This lack of clear mapping makes the overall method presentation somewhat confusing to follow.\n\n2: **The clarity of the core methodological innovation is limited.** The ablation study (Table 5) demonstrates that removing either the \"mask input\" or the \"MaskPoseAdapter\"  severely degrades performance. This highlights the importance of mask conditioning, a point the paper also emphasizes. However, using per-person tracking masks and pose sequences as conditional inputs is a relatively common practice in human animation, so the use of \"mask input\" itself isn't a significant novelty. The ablation \"w/o MaskPoseAdapter\" (which uses the original PoseNet) also performs poorly, suggesting the fusion method is key. Yet, the current experiments do not clearly disentangle the performance gains of the novel fusion strategy within the adapter from the more straightforward, non-novel benefits of simply using mask conditioning in the first place. This makes it difficult to pinpoint the most critical novel design choice responsible for the performance gains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QUiTsPa8Jh", "reviewer": "Reviewer_rryZ", "summary": "This paper addresses a limitation in controllable video generation (CVG): the inability of existing systems to generate long, photorealistic multi-person interaction videos while preserving identity and respecting noisy control signals (e.g., poses, masks). The authors propose DanceTogether, the first end-to-end diffusion framework tailored for this task, which synthesizes videos from a single reference image and independent per-person pose/mask sequences.\nKey technical innovations include:\n1. MaskPoseAdapter: A novel conditional adapter that fuses stable tracking masks with semantically rich but noisy pose heatmaps, enforcing \"identity-action binding\" at every denoising step to eliminate identity drift and appearance bleeding.\n2. MultiFace Encoder: Distills compact identity tokens from the reference image and injects them into cross-attention layers, ensuring consistent subject appearance across frames.\n3. Curated Datasets:\nPairFS-4K: 26 hours of dual-skater footage with 7,000+ unique identities.\nHumanRob-300: 1 hour of humanoid-robot interaction data for cross-domain generalization.\n4. TogetherVideoBench: A comprehensive benchmark with three tracks (Identity-Consistency, Interaction-Coherence, Video Quality) and the DanceTogEval-100 test suite (covering dance, boxing, skating, etc.).", "questions": "1. Authors note that extending to >2 actors would require hierarchical conditioning. Could you elaborate on potential designs (e.g., grouping actors into sub-pairs, using graph-based attention) and share any preliminary results or challenges (e.g., computational cost, identity confusion)?\n2. You note DanceTogether is computationally intensive. Are there opportunities to optimize inference speed (e.g., model distillation, smaller backbones) without sacrificing quality? What is the current inference latency per frame, and what are your targets for real-world use?\n3. How might DanceTogether be modified to handle low-quality pose/mask signals (e.g., from occluded frames or low-resolution videos)? Would integrating a denoising module for poses/masks (e.g., using diffusion to refine inputs) improve performance, and have you tested this?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Experiments are comprehensive: 8 baselines, ablation studies for all key modules, and cross-domain validation. Metrics are carefully chosen to measure identity, interaction, and quality—avoiding overreliance on visual subjective assessments.\n2. Dataset curation: The data pipeline uses state-of-the-art tools to extract high-quality annotations, and PairFS-4K is the first large-scale dual-skating dataset with diverse identities.\n3. The work advances CVG from single-subject to multi-actor scenarios, a critical step toward real-world applications (e.g., virtual film sets, AI-driven animation).\n4. TogetherVideoBench provides a standardized evaluation framework, which will accelerate progress in the field by enabling fair comparisons between methods.", "weaknesses": "1. DanceTogether’s performance degrades under severe occlusions, motion blur, or detector failures. The paper does not propose solutions to robustify against noisy inputs (e.g., denoising modules for poses/masks), which is a practical limitation for real-world use.\n2. The model assumes mostly static cameras and simple backgrounds. Dynamic camera motion or cluttered scenes introduce artifacts, but the paper does not explore adaptations (e.g., camera motion estimation) to address this.\n3. The HumanRob-300 dataset is only 1 hour long, and the paper provides little detail on interaction types (e.g., handshakes vs. collaborative tasks) or robot models. More diverse human-robot data and experiments would better validate cross-domain generalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tNBg0axRVX", "reviewer": "Reviewer_sBvg", "summary": "The paper introduces DanceTogether, an end-to-end diffusion framework for controllable two-person interaction video generation from a single reference image plus independent per-person pose and mask sequences. Key ideas are a MaskPoseAdapter that fuses pose heatmaps with tracking masks to bind “who” and “how” at every denoising step, and a MultiFace Encoder that injects compact identity tokens throughout the U-Net. The authors also curate PairFS-4K (about 26hrs figure-skating pairs), HumanRob-300 (about 1hr human-robot), and propose TogetherVideoBench to evaluate identity consistency, interaction coherence, and video quality.", "questions": "1. The authors acknowledges that they \"assume a mostly static camera and relatively simple backgrounds; dynamic camera motion or highly cluttered scenes may introduce artifacts or identity confusion.\" How the moving cameras will impact the results? Could this pipeline involves camera control (e.g., by estimating and conditioning on camera motion or via background stabilization)?\n\n2. For human-robot, the visualization results is not strong enough to clarify the contribution. The cross-domain section shows qualitative success after ≈1 h fine-tune but lacks quantitative comparisons or baselines in that domain. As this paper only uses the appearance of a robot from images, what is the major difference between human-robot and human-human? Is it just for showing generalization ability to non-human appearances? Apparently, the robot movement in this paper is not related to embodied AI training or world model.\n\n\n3. Will it be possible to train this paper's idea on a DiT backbone? I understand that the reason choosing U-net and StableAnimator as the baseline could be limited GPU resources. However, some problems in results from U-net could just vanished with a DiT backbone. It is hard to assess the contribution of this paper from a bad backbone or pretrained model.\n\nOverall, it is a good paper with adequate content for publication. The authors have shown as many experimental results as possible, which will be beneficial to this domain. However, concerns about scope limited to two people and other novelty concerns motivate me to rate it as marginally below the acceptance threshold. I could adjust my rating during author rebuttal and discussion period.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Clear target & mechanism. The paper diagnoses identity drift in multi-person controllable video and proposes a concrete architectural answer (pose–mask fusion + identity tokens) rather than only post-hoc temporal smoothing. The design is described with implementation-level detail. \n\n2. Comprehensive evaluation. Three-track benchmark with MOT-style identity metrics, motion/pose adherence, and perceptual quality, along with ablations that remove each module/input. Reported gains are consistent across tracks. \n\n3. Useful data curation. PairFS-4K fills a gap for dual-actor interactions with many IDs; TogetherVideoBench focuses evaluation on human-masked regions to reduce background bias.", "weaknesses": "1. The primary concerns about this paper is that the scope is limited to two people. The method is explicitly “optimized for up to two actors,” and scaling to larger groups is acknowledged as non-trivial in both compute and design; the datasets/benchmark are also two-person-only. This narrows impact for wider multi-human scenes and social group interactions. How would the architecture and compute scale to N>2 actors? Any preliminary 3–4 person results?\n\n2. Besides, I also have concerns about the novelty. Many components (e.g., StableAnimator backbone, identity adapters) are adapted/extended; the contribution feels like a strong systems integration with a new adapter and data pipeline rather than a fundamentally new generative principle.\n\n3. A common problem for methods in this task is heavy reliance on external controls. Quality hinges on the quality of provided mask/pose. However, it is not easy to get high-quality mask/pose pair in in-the-wild applications. Did the author investigate how to get mask/pose pair easily, or how to handle the noise in the input mask/pose pair?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2N5dqRHPpE", "reviewer": "Reviewer_E8i4", "summary": "This paper introduces DanceTogether, an end-to-end diffusion framework for controllable multi-person video generation that addresses the challenges of identity drift and appearance bleeding in complex interactions. The method utilizes a MaskPoseAdapter to fuse robust tracking masks with noisy pose heatmaps, ensuring persistent identity-action binding throughout the denoising process. The authors also contribute two new datasets—PairFS-4K and HumanRob-300—and a benchmark called TogetherVideoBench for evaluation.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clearly structured and easy to follow. \n\n2. Experimental results show improvements in identity consistency, interaction coherence, and video quality over compared methods. The results are further bolstered by the provided webpage showcasing the generated videos, which offers valuable qualitative evidence.", "weaknesses": "1. The technical contributions appear incremental. Multi-person video generation is an established research topic, and the use of conditional diffusion models for this task is now a relatively conventional approach. \n\n2. The experimental comparison may be unfair. The proposed model was trained on significantly more data (e.g., the PairFS-4K dataset) than the baseline methods. It is therefore unclear whether the performance gains stem from the novel architecture or simply from the increased scale of the training data. \n\n3. The evaluation is insufficient for claiming superiority in multi-person generation. The authors primarily compare against methods designed for single-person video generation. To properly validate their claims, comparisons against existing state-of-the-art multi-person video generation models on established benchmarks are necessary. [a] Follow-your-multipose: Tuning-free multi-character text-to-video generation via pose guidance, arXiv 2024. [b] Magicfight: Personalized martial arts combat video generation, ACM MM 2024.\n\n4. The paper introduces several hyperparameters but provides no ablation studies or sensitivity analysis for them. The impact of these design choices on the final performance remains unquantified, making it difficult to assess their necessity.\n\n5. Despite the strong quantitative results, the qualitative evidence from the provided video samples is less convincing. The generated videos often exhibit noticeable artifacts, temporal inconsistencies, and an overall synthetic appearance that is easily discernible to the human eye. \n\n6. The scope of the work is limited to two-person interactions. The method's scalability and effectiveness on videos involving three or more persons remain an open question, which limits the generalizability of the claimed contributions to broader multi-person scenarios.\n\n7. The paper lacks any analysis of computational cost, inference speed, or model complexity. Given that the proposed framework involves multiple encoders and a diffusion model, its practical efficiency is a significant concern for real-world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)