EverybodyDance: Bipartite Graph–Based Identity Correspondence for Multi-Character Animation
Authors: Haotian Ling
Zequn Chen
Qiuying Chen
Donglin Di
Yongjia Ma
Hao Li
Chen Wei
Zhulin Tao
Xun Yang
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion model
Character animation
Primary area: applications

Abstract: Consistent pose‐driven character animation has achieved remarkable progress in single‐character scenarios. However, extending these advances to multi‐character settings is non‐trivial, especially when position swap is involved. Beyond mere scaling, the core challenge lies in enforcing correct Identity Correspondence (IC) between characters in reference and generated frames. To address this, we introduce EverybodyDance,  a systematic solution targeting IC correctness in multi-character animation. EverybodyDance is built around the **Identity Matching Graph (IMG)**, which models characters in the generated and reference frames as two node sets in a weighted complete bipartite graph. Edge weights, computed via our proposed Mask–Query Attention (MQA), quantify the affinity between each pair of characters. Our key insight is to formalize IC correctness as a graph structural metric and to optimize it during training. We also propose a series of targeted strategies tailored for multi-character animation, including identity-embedded guidance, a multi-scale matching strategy, and pre-classified sampling, which work synergistically. Finally, to evaluate IC performance, we curate the **Identity Correspondence Evaluation** benchmark, dedicated to multi‐character IC correctness. Extensive experiments demonstrate that EverybodyDance substantially outperforms state‐of‐the‐art baselines in both IC and visual fidelity.

EvenybodyDance: Bipartite Graph-Based Identity Correspondence for Multi-Character Animation
Haotian Ling Zequn Chen
Qiuying Chen
Donglin Di
Yongjia Ma² Hao Li
Chen Wei
Zhulin Tao Xun Yang
University of Science and Technology of China
$Communication University of China "MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, USTC
Lii Auto
paotianling@man www.ustc.ee. edc...dduccco xyang21@ustc.. edu. {chenzequn, didonglin, mayongjia, lihao43, thenwei10/@1lxiiane com chenqydeuc.edi eet.y/@uuu.euu..... taozl@cuc. edu.
Abstract
Consistent pose-driven character animation has achieved remarkable progress in single-character scenarios. However, extending these advances to multi-character settings is non-trivial, especially when position swap is involved. Beyond mere scaling, the core challenge lies in enforcing correct Identity Correspondence (IC) between characters in reference and generated frames. To address this, introduce EverybodyDance a systematic solution targeting IC correctness in multi-character animation. EverybodyDance built around the Identity Matching Graph (IMG), which models characters in the generated and reference frames as two node sets in weighted complete bipartite graph. Edge weights, computed via our proposed Mask-Query Attention (MQA), quantify the affinity between each pair of charac- ters. Our key insight to formalize IC correctness as a graph structural metric and to optimize it during training. We also propose a series of targeted strategies tailored for multi-character animation, including adentity-embeddee guidance, a multi-scale matching strategy, and pre-classifidd sampling, which work synergisti- cally. Finally, to evaluate IC performance, we curate the Identity Correspondence Evaluation benchmark, dedicated multi-character IC correctness. Extensive experiments demonstrate that serybodyDaacc substantially outperforms state-of- the-art baselines in both IC and visual fidelity.
1 Introduction
Character animation aims to generate video sequences from still images guided by specific pose sequences [[2 Unlike text-driven generation focusing mainly high-level semantic alignment 33,4,, it requires a dual fidelity: maintaining consistent visual appearance including fine- fine-graing grained details and accurately performing complex motion sequences (5.). This requirement has generated
significant research interest 7
9 T0 12:
Despite significant advances in single character animation generation (e.g.
8
9 13
14
12
extending these methods to multi-character scenarios introduces unique challenges (see Figure I). The key challenges are twofold. First, in multi-character scenarios, characters can swap relative
"Work done during internship at Li Auto. Project Leader.
"Coresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Ref Ing
Relotive
Positions
Fixed
Changed
Ref Ing
Ours
Magic Animate
Magic Dance
Existing Issues
Performance
Figure 1: The left panel highlights the challenges of extending existing methods multi multi-character scenarios. The yellow box indicates feature interference between characters, while the red box marks identity mismatches. The right panel illustrates our method's accurate identity correspondence.
positions, leading to identity confusion. Furthermore, the appearances of different characters can interfere with one another. Existing single-character animation approaches [14] 13: 9 I0 12 are mainly based implicit data-driven paradigms. In multi-character scenarios, such paradigm struggles to guarantee accurate one-to-one correspondence between generated and reference characters (see
Section 4.3 Empirical results demonstrate that state-of-the-at methods often struggle achieve satisfactory Identity Correspondence (IC) under these conditions (see Section 4.2)
To address these limitations, we propose explicit modeling framework, which directly captures the correspondence between generated characters and their reference counterparts. Our method enforces correct IC between characters during training. Specifically, we introduce weighted complete bipartite graph, Identity Matching Graph (IMG), whose two node sets represent generated/reference characters. Edge weights, derived by our proposed Mask-Query Attention (MQA), quantify the affinity between each generated/reffrrecce pair. A global matching score derived from the IMG provides direct, optimizable objective for IC correctness. Integrating IMG into training achieves disentanglement of multiple characters and yields more accurate IC for multi-character animations. The construction process of IMG is dynamic, making it scalable for any number of characters.
To resolve the ambiguity of motion guidance in nulti-character scenarios, we designed Identity Embedded Guidance (IEG). IEG provides clear anchors for each character throughout both training and inference. During the training phase, IEG and IMG work synergistically to create a guidance- supervision loop. To further strengthen the robustness of IC, employ a suite of targeted im- provements. First, to enforce the correct correspondence across the entire feature hierarchy, we introduce multi-scale matching strategy. In addition, to address the long-tail distribution of the multi-character dataset, we propose a pre-classified sampling strategy to ensure that difficult and in- frequent position-swap samples receive sufficient emphasis during training. To rigorously evaluate IC performance under complex multi-character conditions, we also present the Identity Correspondence Evaluation (ICE) benchmark, designed to challenge and compare SOTA methods on their ability to maintain correct IC.
Our main contributions summarized follows: (1) Graph-Based IC Modeling: We propose the Identity Matching Graph (IMG), a weighted complete bipartite graph that explicitly models IC in multi-character animation, whose edge weights computed via our proposed Mask- Mask-Query Attention (MQA). (2) Targeted Strategies: We propose a series of targeted strategies, including identity-emeedded pose guidance, multi-scale matching strategy and a pre-classified sampling strategy, all tailored to multi-character animation. (3) ICE Benchmark: We curate the benchmark, ICE, for comprehensive evaluation in multi-character animation. Extensive evaluations demonstrate that our approach significantly outperforms SOTA baselines in both IC accuracy and visual fidelity.
2 Related Work
2.1 Diffusion Models
Diffusion Models gradually corrupt data by adding Gaussian noise and learn a reverse denoising process to model complex distributions [15] 16, 17 18). At inference, samples generated by

---

starting from pure noise and iteratively denoising with the trained model [19; 20). Extensions to latent diffusion operate compressed feature spaces for efficient ligh-resolution and text-to-image generation [2I] Beyond images, diffusion frameworks produce temporally coherent videos for facial
expression and dance generation [22 9 [1] Conditional diffusion enables flexible generation by
guiding the reverse process with model-free or classifier-free cues [20] 23
2.2 Video Generation
Early video synthesis relied on GAN-based 22 frameworks (e.g., TGAN (21), which introduced temporal shift modules to enforce frame- to to-frame coherence. Diffusion- approaches [26] extend image diffusion models by integrating spatio-temporal conditioning specialized temporal attention layers, as seen in Cune-A-Video s s(27) tailored spatio-temporal attention and MagicVideo's Video's [28] directed temporal attention module in latent space. Transformer-centriic models such as Video Diffusion Transformer (VDT) (29) and Matten (30] leverage modular temporal and spatial attention (e.g.. Mamba-Attention (31) capture long-range dependencies and global video context. Training- free extensions such as FreeLong [32] employ SpectralBlend temporal attention mechanism to adapt pretrained short-clip diffusion models for long-video generation, maintaining both global consistency and local detail without additional training. Recent video super-resolution and editing techniques employ temporal-conss consistent diffusion priors to reduce flicker and preserve object appearance, further enhancing smoothness in tasks from animation to real-world scene synthesis [33
2.3 Human Image Animation
used appearance flow for feature warping but suffered from adversarial training issues such as mode collapse and motion inaccuracy [9]. More recent work 112 9 10
Early GAN-based methods
34
35
36
37
38
39
40
46) based diffusion models, which offers -or-forgrround-backgroonn control. ReferenceNet 9 improves fine-detailed consistency by injecting the appearance of reference frame into the denoising UNet. Recent work has also made valuable contributions to multi-character scenarios. Ingredients [49] focuses text-controlled multi-character layout, Follow- Your-Pose- V2 [50] focuses on scenes where characters maintain fixed relative positions.
7
8
3
42
43
44
45
stable training 47. Disco S uses ControlNet
48 for disentangled
3 Method
Identity Correspondence (IC), one-to-one matching between each generated character and its counterpart in the reference frame, becomes especially critical when characters swap positions. Existing character animation methods [8] 9 [[: typically rely on end-to-end training losses that capture only global similarity, often failing enforce correct IC in such scenarios (see Section 4.3
Section 3.1 introduces our formulation of the Identity Matching Graph (IMG). Section 3.2 explains how the IMG is constructed. Section 3.3 presents our targeted strategies for multi-character animation.
3.1 Problem Formulation
Concretely, we construct weighted complete bipartite graph between the reference (ref) and the generated (gen) characters in each frame. The node set R = [ri..... I'm represents m characters ordered from left to right in the reference frame (numbered 1 to m)) while the node set G = {g1..... In with n < m describes characters in the generated frame. We define the Identity Matching Graph (IMG) as the following bipartite graph:
BID = (R, 9, E, w), E = (r,,, 93) | Sisis m, 1sjsn), where the edge weight w(r;: 93)
(1)
represents the affinity (potential correspondence) between r; and gj (see left panel of the Figure Z}. During training, for each generated frame we construct its edges (i.c., all possible correspondences between characters). We
IMG, yielding the set É of denote the edge set of n ground-truth correspondences by M Since the edges in M represent the correct IC, our objective is to increase their weights by training the UNet [51]. Therefore, we use the following ratio C to quantify the correctness of IC as:
E(rogeem w(rigg))
C = E E(ragee w(ri,g))
E 1) É.
(2)
3

---

Identity Natching Groph
Zeput
Nasked Region
Mode
Hadarnard product
Tgt
Attention
Edge Construction
Mask-Query Attention (MQA)
Rf ''X"""
Fail
Agureyettin
0.47
X" [0r. (1avs1))
9+
10.53
1..52
E = (r..g))) U.M
Node Construction
Formulation
IMG Construction
Figure 2: The left panel illustrates whal is the IMG. The target (tgt) frame indicates the ground truth correspondence, indicating which edges belong to the set M The right panel shows how we build the IMG. Since the regions in R do not overlap spatially, we sum all {r. /m representations into Tall-
Lower C' indicates a more severe ambiguity. Optimizing C will force the model to learn correct IC, which will serve as a loss term during the training of the diffusion model. For example, in the left panel of Figure 2 if the left generated character !1 has a higher affinity to reference character 71 than its true counterpart r2. by the IMG construction Section 3.2 the edge weight for (r2, 91) will be a low value. Under the total loss defined in Equation 8 this low-weighted pairing incurs a penalty, thereby driving the model to learn the correct inter-character correspondence.
3.2 Identity Matching Graph Construction
Node Construction. During training, the reference and generated frames are encoded into the latent space [47). Therefore, we propose to use the corresponding masked regions in the latent space to represent each character, and then build the IMG nodes from those regions. We denote {M)) as J the instance segmentation [52] masks of the m reference characters, and {M,})= represents the masks of the (n & m) generated characters. Since instance segmentation is performed offline on the training set, we extract the masks for the generated frames from their corresponding ground -ruth target frames. This approach circumvents potential drawbacks in both efficiency and accuracy.
In chosen UNet [51] layer f, we extract the intermediate reference and generated feature, denoted as fr E Rexh' xnd and generated latent maps. Each mask is interpolated to match the spatial resolution of the correspond- ing latent feature map: M Interp( (M,) E {0, 1]
f Re x respectively. (h', w") and (h w²) are the sizes of the reference
M Interp(M E {0, 1) xx 1 The latent regions corresponding to these segmentation masks are treated as graph nodes ri. as:
T f M,
9j = f MS.
C denotes the Hadamard product, yielding reference nodes {r:) 1am and generated nodes {m
Edge Construction. To compute the edge weight w(ri. gy) of the IMG, we propose the Mask-Query Attention to estimate the affinity between 9j and r,. explooiss the ability of the attention mechanism's
to capture spatial dependence. Each generated character {9, 1an is transformed into a query matrix Q; E IR xx and transfer each reference character {re)m to a key K; E R!N w") xd The attention map A between r, and gj is calculated as:
553 9
A
softmax
Q: K. Vd.
R (((+
A? A[[p.) 4) denotes the dependence from the p-th patch of gj to the g-th patch of r.. We use the score S/ to reflect the affinity between generated character gj and reference character ri:
Σ Σ Alpp.gl.
Dタ= {pl Mfipl 1}, v== {g M4 = 1}
(5)
VEV PEV

---

ReferenceNet
Pose
Spatial-Ates -Attn
Enc
Cross-Attn
Terporal-Attn
Ref
VAE
Internediate Feature
Enc
Gradlent from e
Construction
(Training only)
CLIP
Ref Ing
Selected for MSN
DengesingNet
Pose
VAE
Enc
Tgt Pose
Noise
Mask
c
XT
Figure 3: Training pipeline EverybodyDance. We only construct the IMG during training. We additionally imput the IEG of the reference image. ReferenceNet binds character identity by fusing the reference image's appearance to create identity-ave aware features that guide the DenoisingNet.
Equation aggregates patch-level dependency A into a node-level affinity score S which reflects the overall correlation between the generated character 9j and the reference character rı. In the attention map A². each row corresponds to a patch in the generated latent 9j. and each column corresponds to patch in the reference latent r; To quantify the relative affinity between 9j and each reference character {r:)
first compute the of affinity scores )m Each score s aggregates patch-level attention scores from A² across the masked regions defined by M3 and M,. We then normalize these scores to obtain the final edge weights {w(ri, 9
S Σ Σ 5 +2
weri, 93)
E (0,1), 1), = 10-8
Computing the pair-wise attention A for all n pairs is inefficient. As shown in Figure aggregate all reference nodes into a single representation, rau = -n rt. Each generated node Oj then computes attention against Fau- This optimization reduces the computational complexity from (m n)) to (n). Since m and are dynamically determined by the number of segmentation masks, the IMG is built in fully dynamic process that can be extended any number of characters.
3.3 Targeted Strategies
Identity Guidance Existing methods rely solely on pose guidance without incorporating explicit identity information [54] 55) 35 [56]. which complicates multi-character animation by lacking
reliable identity cues for correct Identity Correspondence (IC). To resolve this, introduce Identity- Embedded Guidance (IEG), which embeds identity into DWPose (54) by color-coding each skeleton (see Appendix for details). The IEG from each reference frame is also injected into its feature space.
These colored skeletons serve to mark reference characters and guide the placement of corresponding characters during generation, thereby enabling the model to differentiate identities during training and inference. We provide explicit input cues (IEG) and a matching loss (IMG). Both originate from the segmentation masks ensure alignment between guidance and supervision.
Multi-Scale Matching To improve training robustness and ensure correct across different feature spaces, perform Multi-Scale Matching (MSM) at N selected UNet [5]] layers { At each
5

---

Table 1: Quantitative comparison the ICE benchmark. Arrows indicate optimal direction (!=lower better, 1=higher better). AnimateAAnynee denotes the model fine-tuned on our dataset.
Frame Quality
Video Quality
Method
SSIM
P
LPIPS!
L14
FID
FID-VID
FVD
AnimateAnyone
0.616
14.97
0.339
5,16E-05
59.19
32.057
364.85
Anyonee
0.596
14.67
0.342
5,35E-05
54.71
31.274
358.31
MimicMotion
[4
0.621
15.00
0.338
5,48F-05
60.77
26.490
381.69
MagicDance
12
0.508
13.81
0.424
1.33E-04
53.30
47.127
471.71
MagicAnimate [10
0.614
13.95
0.369
6.36E-05
76.28
42.257
521.67
UniAnimate 13
0.623
15.66
0.328
3,41E-05
44.38
26.696
295.56
Everybody Dance
0.654
16.93
0.304
2.86E-05
40.19
23.584
225.06
B and compute layer-level IC score according to Equation 2
layer& we construct its IMG
C ((raa)eM w(x) (ri,
E [0, 1],
E(maa))e w (ri.
where (r:: fJ) are the edge weights at layer l, M are the ground-truth correspondences, and E(4) is the full bipartite edge set at that layer. We then use the average of J over all N layers to the matching loss Lmatch- Minimizing Lmatch thus encourages the model to maximize IC correctness across all chosen scales, yielding more robust multi-character generation with accurate IC. The full pipeline illustrated Figure 3 The final training objective consists of standard diffusion reconstruction loss Caitt and Lmatch. denoted as:
L Laiff + Ammathh A> 0,
(8)
where À balances the frame quality against IC correctness.
Pre- Fre-Classified Sampling Existing methods [T] 9 TO typically select reference-targel frame pairs randomly from a training video. However, in multi-character scenarios, challenging sample pairs, such as those involving position swaps, are relatively rare. To address this, we extract the position of each character. Then, with probability we draw from the pre-classified challenging swap pairs, and with probability P we conduct random sampling.
4 Experiment
4.1 Settings
Quantitative Metrics. quantitatively evaluate the performance of different methods, we employ several widely used metrics, including L1 [57], PSNR [58 6 SSIM [59], LPIPS [60]. FID [24].
FID-VID 24), and FVD 61 These metrics jointly provide comprehensive evaluation.
Baselines. To validate the superiority of our method, we conduct extensive comparisons against
several SOTA methods: MagicAnimate [10), AnimateAnyone (9) MagicPose [[2], MimicMotion
14]. Follow- Yoolo---our-Pos--22 (5)) and UniAnimate 13
Dataset and Other Details. We curated custom multi-character dataset comprising approximately 800 video clips. For IC correctness evaluation, we introduce the ICE-bench, which contains 3,200 video frames. Our model is fine-tuned based on the AnyonteAnyone framework using this dataset. For full descriptions the training dataset and ICE-bench, other experiments, please refer to the Appendix.
4.2 Comparison Study
We evaluate our method, EverybodyDance on the ICE-Bench using both quantitative metries and qualitative showcases. As reported in Table EverybodyDance achieves substantial improvements over backbone model, Anyone: reduces the FVD score by 38.3% %, indicating signifi- cantly improved video fidelity. To ensure these gains stem from our proposed targeted enhancements

---

Nagic Aninate
Img
Tgt
Anyone
Motion
NagicDance
Ours
03
79
JEST
HER
HHH
Figure We compare our method with several oftte-of-the-ar baselines. The last three rows illustrate three particularly challenging scenarios: (I) reference images exhibiting complex, non-standard poses; (2) target poses involving fewer character than the corresponding reference images; and (3) reference characters undergoing severe occlusion. Under these difficult conditions, our method consistently outperforms existing approaches, demonstrating accurate IC.
rather than -ptaset-specific biases, additionally fine-tuned AnimateAnyone so, it still fails to match the performance of Everybody Dance. Qualitatively, as shown in Figure our approach consistently achieves accurate character identity correspondences in challenging scenarios such as position swaps, where existing methods often produce identity confusion or mismatches.
dataset;
4.3 Ablation Study
To elucidate how our method enforces correct IC, we conduct a series of ablation experiments, summarized in Table The experiments are categorized into the following three groups:
Effectiveness of the Identity Matching Graph We compare our IMG-based approach against several ablation variants: 1) vw IEG: We fine-tune the backbone model using IEG rather than DWPose. 2) End2End: We provide the IEG of reference image, allowing the model to learn IC in an end-to-end scheme. 3) End2End-N We further use masks over each character's region enforce the model to focus on the corresponding region (see details in the Appendix).
As shown in the first group, introducing IEG alone yields some gains, while embedding identity (End2End and End2End-M) into the reference image's feature space enables partial performance improvements but remains insufficient. Only when IMG is incorporated to explicitly supervise Character-to-chracche correspondence, the model achieves a dramatic improvement.
We also present qualitative comparison results in Figure 5 Figure 6 visualizes attention maps for both the IMG-based and end-to-end paradigms. We present visualizations rall alongside each generated character 9, in Section 3.2. For each 91 and 92, arranged from left to right, we display its affinity scores with all reference characters. To enable direct comparison, we include the corresponding attention map visualizations from the End2End-N model. In the table, the columns labeled IMG-9; and End2End-Mr M-g respectively illustrate the attention maps 9i and 92 over the reference image.
Effectiveness of Multi-Scale Matching As demonstrated in the second group of experiments, progressive increase in the number of matching layers leads to consistent improvements in overall performance.

---

Table 2: To facilitate analysis, the table is divided into three groups. MSM-N refers to building the IMG using the last layers of the UNet, while PCS-p denotes selecting pre-classified hard samples with sampling ratio of p. In the Full setting, set N 5 and p 0.3.
Experiment
Experiment Settings
Frame Quality
Video Quality
Group
SSIM
1
LPIPS
L1
FID.
FID-VIDY
FVD.
IMG Effectivenesss
Full
0.654
16.93
0.304
2.86E-05
40.19
23.584
225.06
Finetune
0.596
14.67
0.342
5.35E-05
54.71
31.274
358.31
t/w IEG
0.615
15.42
0,340
3.85E-05
48.78
29.804
319.96
End2End-M
0.634
15.84
0.338
3.32E-05
45.05
28.464
285.09
End2End
0.630
15.74
0.337
3.41E-05
45.23
28.789
289.69
MSM-4
0.649
16.81
0.308
2.96E-05
40.54
23.704
232.59
MSM Settings
MSM-3
0.644
16.80
0,313
2.98E-05
40.49
24.566
232.47
MSM-2
0.641
16.68
0.317
3.02F-05
42.58
24.935
236.
w/o MSM
0.637
16.50
0.321
3.14E-05
41.26
25,507
256.01
PCS-0.5
0.654
16.88
0.311
2.95E-05
40.56
24.603
228.19
PCS Settings
PCS-0.4
0.650
16.89
0.312
2.99E-05
41.73
23.708
234.56
PCS-0.2
0.652
16.72
0.311
2.96E-05
40.99
24.072
226.89
PCS-0.1
0.654
16.94
0,309
2.95E-05
40.47
23,592
227.32
w/o PCS
0.632
16.23
0.329
3.07E-05
43.79
25.146
252.33
X-0.05
0.637
16.59
0.322
3.01E-05
41.76
23.243
233.34
Settings
A-0.10
0.653
16.80
0.309
2.99E-05
42.04
23.691
234.35
A-0.15
0.649
16.77
0.308
2.89E-05
42.62
24.093
234.44
X-0.20
0.654
16.93
0.304
2.86E-05
40.19
23.584
225.06
X-0.25
0.653
16.71
0.316
2.98E-05
40.47
23.156
230.18
Ref Ing
Tgt Pose
Finetune
w/o MSM
w/o PCS
End2End- -M
End2End
Full
GT
Figure 5: Qualitative comparison against different variants. Red boxes highlight cases of identity switch, while yellow boxes indicate instances of feature contamination.
Effectiveness of Pre-Classified Sampling. By comparing PCS under different sampling ratios, we find that ratio of 0.3 achieves an optimal trade-off between hard sample abundance and diversity. This setting yields the best performance and effectively alleviates the long-tail data problem.
Syype-Parameterr Analysis on the X. We conduct a sensitivity analysis on the introduced in Equation (8. Setting 0.20 achieves the best overall performance. This value offers an optimal trade-off between the diffusion reconstruction loss and the identity matching loss. Higher values cause the IC accuracy plateau while slightly degrading the visual quality.
Fffeetiveness of MQA. We conduct an experiment the Appendix to compare MQA with other similarity-baan based affinity calculation methods.

---

Ref
Ing
Tgt Img E,
IMG-E: End2End-N-
IMG
End2End Bonkeend-M-B- -M-8
Figure 6: Ideally, the attention distribution should be primarily concentrated on the corresponding reference character located the opposite side of gj-
Table 3:
Quantitative comparison on frame and video quality metrics. For more details about settings of this benchmark, please refer to [50].
Frame Quality
Video Quality
Method
SSIM
PSNR
LPIPS|
L14
FID.
FID-VID
FVD.
DisCo 8
0.793
29.65
0.239
7.64E-05
77.61
104.57
1367.47
MagicAnime
10
0.819
29.01
0.183
6,28E-05
40.02
19.42
223.82
MagicPose
12
0.806
31.81
0.217
4.41E-055
31.06
30.95
312.65
AnimatcAnyone
0.795
31.44
0.213
5.02F-05
33.04
22.98
272.98
Follow-Your-Poss
50
0.830
31.86
0.173
4.01E-05
14.56
142.76
Everybody Dance (Ours)
0.879
32,49
0.151
0.92F-05
26.01
12,68
127.36
4.4 Generalizability
On Public Multi-character Benchmark. To validate the generalizability of our method, we also conducted comparisons with the publicly available benchmark provided by Follow- Hollow-Your-oe This benchmark is distinguished by frequent inter-person occlusions. As shown in Table our method outperforms Follow-Your- Your-Pose-Ve and other SOTA methods across all quality metrics. should be noted that these metrics primarily reflect overall video fidelity. Since our method lacks explicit occlusion modeling, the bengrounddbackgrron
(50)
order during occlusions will be determined randomly.
In Diverse Scenarios. We conducted a comprehensive quantitative evaluation to assess our method's capabilities in diverse scenarios. Specifically, benchmarked its performance challenging multi- character videos containing to 5 individuals, and on the widely-used single-character TikTok [62] benchmark. As shown in Table 4 our proposed method demonstrates superior performance all competing methods across all key metrics. For qualitative results, please refer to the Appendix.
Cross- Video Motion Transfer. To assess the generalizability of our method for real-world applications, we conduct a cross-video motion transfer experiment. In this setting, a source video provides the motion template used to animate diverse of reference images. Moreover, test the model's flexibility by reassigning character positions. We reorder the color-coded identities in the target IEG. The result, depicted in Figure 7 is correctly rendered sequence where the characters' relative positions are swapped, underscoring our model's capacity for robust and flexible identity control.
Conclusion and Limitation
In this work, we introduce Everybody Dance, framework that addresses the critical challenge of Identity Correspondence (IC) in multi-character animation. The core of our method is the Identity Matching Graph (IMG), which formalizes the ambiguous problem IC correctness into an explicit, optimizable graph-structural metric. To construct this graph, our Mask-Query Attention (MQA) efficiently computes edge weights. This graph-based loss works in synergy with our Identity- Embedded Guidance (IEG) together, they form cohesive guidance-supervisee architecture. Finally,

---

Table 4: Quantitative comparison multi-character and single-character benchmarks. We group metrics into Frame Quality and Video Quality. Best results are in bold.
Frame Quality
Video Quality
Scene
Method
SSIM
PSNR*
LPIPS!
FID
FID-VIDU
FVD
Character
AnimatcAnyone
9
0.606
14.70
0.370
56.66
35.356
401.51
More
AnimateAnyone"
0.607
14.92
0.363
64.34
36.308
406.08
UniAnimate [13
0.640
15.62
0.338
57.75
29.030
348.91
Ours
0.671
16.68
0.315
42.84
23.571
261.01
Single Character
AnyoneeAnyone
0.768
17.85
0.280
52.15
25.864
209.14
AnimateAnyone
0.764
17.19
0.291
62.52
25.943
213.68
End2End
0.770
17.65
0.288
45.25
22.515
186.34
Ours
0.772
17.78
0.279
40.56
20.294
163.85
ARTTA
Tgt
JEHI
JUST
JENK
Tgt Pose
Figure 7: We employ multi-person video clip the source pose and various references to generate animations. We also swap the relative positions of characters the target pose.
we enhance IC robustness through two targeted strategies: Multi-Scale Matching (MSM) enforces correctness across multiple feature hierarchies, while Pre- rre-Classified Sampling (PCS) addresses challenging, rare training samples. These contributions enable our model significantly improve identity consistency and visual quality in complex multi-character scenes.
However, current method is unable to effectively handle scenarios with severe inter-character occlusion. Incorporating 3D datasets [63 64) 65) presents a promising direction for future work to address this. Furthermore, the performance of method depends on the accuracy of the upstream instance segmentation model. Finally, our current quantitative evaluation still relies on proxy metrics that measure overall video fidelity. Designing dedicated metrics that directly and quantitatively evaluate IC correctness remains a significant open problem.
Acknowledgement
The authors appreciate the generous support Li Auto, which provided the financial backing and essential computational resources that made this research possible. The authors also thank our colleagues the University of Science and Technology of China, Auto, and Communication University of China for their insightful discussions and support throughout this project.
10

---

References
[1] da Silva, A. G., M. V. Mendes Gomes, I. Winkler. Virtual reality and digital human modeling for ergonomic assessment in industrial product development: patent and literature review. Applied Sciences, 1084):0084, 2022.
[2] Wohlgenannt, I., A. Simons, S. Stieglitz. Virtual reality. Business & Information Systems Engineering, 62:455-400 2020.
[3] Loeschcke, S., S. Belongie, S. Benaim. Text-driven stylization of video objects. In European Conference on Computer Vision, pages 594-609. Springer, 2022.
[4] Jiang. Y., S. Yang, T. Koh, et al. Text2performer: Text-driven human video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22747- 22757. 2023.
[5] Chan, C.,, S. Ginosar, T. Zhou, et al. Everybody dance now. In Proceedings of the IEEE/CVF international conference computer vision, pages 5933-5942. 2019.
[6] Siarohin, A., Lathuilière, S. Tulyakov, al. First order motion model for image animation. In
H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, R. Garnett, eds., Advances Neural Information Processing Systems, vol. 32, Curran Associates, Inc., 2019.
[7] Karras, J.. A. Holynski, T.-.. Wang, et al. Dreampose: Fashion video synthesis with stable diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22680-22690. 2023.
[8] Wang. T. L. Li, K. Lin, al. Disco: Disentangled control for realistic human dance generation. arXiv preprint 23Xv:2307.0004) 2023.
[9] Hu, L. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 113-8163.. 2024.
[10] Xu, Z., J. Zhang. J. H. Liew, et al. Magicanimate: Temporally consistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1481-149) 2024.
[11] Zhu, S., J. L. Chen, Z. Dai, et al. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision (ECCV). 2024.
[12] Chang, D., Y. Shi, Q. Gao, et al. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion. In Forty-first International Conference on Machine Learning. 2024.
[13] Wang, X.. S. Zhang, C. Gao, et Unianimate: Taming unified video diffusion models for consistent human image animation. ArXiv, abs/2406.01 188, 2024.
[14] Zhang. Y., J. Gu, L.W.. Wang, et al. Mimicmotion: High-quality human motion video generation with confidence-aween pose guidance. arXiv preprint arXiv:2406.19680, 2024.
[15] Ho, J., A. Jain, P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-668 68511, 2020.
[16] Sohl-Dickstein J., E. Weiss, N. Maheswaranathan, et al. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2263 PMLR, 2015.
[17] Luo, C., D. Di, X. Yang, et al. Trame: anaccctory-anchoredd multi-view editing for text-guided 3d gaussian manipulation. IEEE Transactions on Multimedia, 2025.
[18] Di, D., Yang, C. Luo, et al. Hyper-3dg: Text-to-3d gaussian generation via hypergraph. International Journal of Computer Vision, 13365))2886-2909 2025.

---

[19] Song, Y., J. Sohl- Sohl-Dickstein, D. P. Kingma, et al. Score-based generative modeling through stochastic differential equations. arXiv preprint arX1v:2011.13456, 2020.
[20] Song, J., C. Meng, S. Ermon. Denoising diffusion implicit models. arXiv preprint
arXiv:2010.02502, 2020.
[21] Liu, L.,, Y. Ren, Z. Lin, et al. Pseudo numerical methods for diffusion models manifolds. In Interaational Conference on Learning Representations. 2022.
[22] Tian, L., Q. Wang, B. Zhang, al. Emo: Emote portrait alive- stwe-genneraing expressive portrait videos with audio2video diffusion model under weak conditions. arXiv preprint arXiv:2402. 17485, 2024.
[23] Lu, C., Y. Zhou, F. Bao, et al. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[24] Heusel, M., H. Ramsauer, Unterthiner, al. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[25] Saito, M., E. Matsumoto, S. Saito. Temporal generative adversarial nets with singular value clipping. In ICCV. 2017.
[26] Xie, Q. Y. Ma, D. Di, et al. Moca: -lentity-preserving text-to-video generation via mixture of cross attention. In ACM Multimedia Asia. 2025.
[27] Wu, J. Z., Y. Ge, X. Wang, et al. Tune-a-video: One-shot tuning of image diffusion models for text-to- video generation. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7589-7599, 2023.
[28] Zhou, D., W. Wang, H. Yan, et al. Magicvideo: Efficient video generation with latent diffusion models. ArXiv, abs/2211.11013 2022.
[29] Lu, H., G. Yang, N. Fei, et al. Vdt: General-purpose video diffusion transformers via mask modeling. In International Conference on Learning Representations. 2024.
[30] Gao, Y., J. Huang, X. Sun, et al. Matten: Video generation with mamba- ArXiv, abs/2405.0300 2024.
[31] Li, K., Li, Y. Wang, et al. Videomamba: State space model for efficient video understanding. European Conference on Computer Vision. 2024.
[32] Lu, Y., Y. Liang. L. Zhu, et al. Freelong: Training- free long video generation with spectralblend temporal attention. ArXiv, abs/2440.119113 2024.
[33] Zhou, S., P. Yang. J. Wang, al. Upscale-a- viieeo Temporal-consister diffusion model for real-world video super-resolution. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages -335-2545,00 2024.
[34] Siarohin, A., O. Woodford, J. Ren, et al. Motion representations for articulated animation. In CVPR. 2021.
[35] Li, Y., C. Huang, C. C. Loy. Dense intrinsic appearance flow for human pose transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3693-3702. 2019.
[36] Kissel, C., Kümmel, D. Ritter, et al. Pose-guided sign language video gan with dynamic lambda. arXiv preprint arXiv: 105.00242, 2021.
[37] Zablotskaia, P., A. Siarohin, B. 7hao, et al. Dwnet: Dense warp-based network for pose-guided human video generation. arXiv preprint arXiv:1910.09139 2019.
12

---

[38] Yoon, J. S., L. Liu, V. Golyanik, et al. Pose-guides human animation from a single image in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15039- 15048. 2021.
[39] Fu, J. S. Li, Y. Jiang, et al. Stylegan-humant A data-centric odyssey of human generation. In European Conference on Computer Vision. 2022.
[40] Jiang, S., H. Jiang, Z. Wang, et al. Humangen: Generating human radiance fields with explicit priors. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12543-12554, 2023.
[41] Zhang, P., L.. Yang. J. Lai, et al. Exploring dual-task correlation for pose guided person image generation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
pages 7703-7712, 2022.
[42] Sarkar, K., D. Mehta, W. Xu, al. Neural re-rendering of humans from single image. In European Conference on Computer Vision. 2022.
[43] Albahar, B., S. Saito, H.-Y. Tseng, et al. Single-image 3d human digitization with shape-guided diffusion. SIGGRAPH Asia 2023 Conference Papers, 2023.
[44] Shao, R., Y. Pang. Z. Zheng, et al. Human4dit: 360-degree human video generation with 4d diffusion transformer. arXiv preprint arXiv:2405.17405, 2024.
[45] Yuan, S., Huang, X. He, et al. Identity-prent preserygn text-to-video generation by frequency decomposition. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 12978-12988. 2025.
[46] Yin, X., D. Di, L. Fan, et al. Grpose: Learning graph relations for human image generation with pose priors. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pages 9526-9534. 2025.
[47] Rombach, R., A. Blattmann, D. Lorenz, et al. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695. 2022.
[48] Zhang, L., A. Rao, M. Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836-3847. 2023.
[49] Fei, Z., D. Li, D. Qiu, et al. Ingredients: Blending custom photos with video diffusion transformers, 2025.
[50] Xue, J., H. Wang, Q. Tian, et al. Towards multiple character image animation through enhancing implicit decoupling. In The Thirteenth International Conference Learning Representations. 2025.
[51] Ronneberger, O., P. Fischer, T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MICCAl 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
[52] Ravi, N., V. Gabeur, Y.-T. Hu, al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.000 14, 2024.
[53] Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
[54] Yang, Z., A. Zeng, C. Yuan, et al. Effective whole-body pose estimation with two-stages distillation. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 4212-4222 2023.
[55] Fang, J. Li, H. Tang, et al. Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
13

---

[56] Loper, M., N. Mahmood, J. Romero. et al. Smpl: A skinned multi person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 851-866. 2023.
[57] Isola, P., J.-Y. Zhu, T. Zhou, et al. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference computer vision and paltern recognition, pages 1125-1134 1134. 2017.
[58] Netravali, A. N. Digital pictwres: representation, compression, and standards. Springer, 2013.
[59] Wang, Z., A. C. Bovik, H. R. Sheikh, et al. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-612, 2004.
[60] Zhang, R., Isola. A. A. Efros, et al. The unreasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595. 2018.
[61] Unterthiner, T., van Steenkiste, K. Kurach, et al. Fvd: A new metric for video generation. 2019.
[62] Jafarian, Y., H. S. Park. Learning high fidelity depths of dressed humans by watching social media dance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12753-12762. 2021.
[63] Ng. E., D. Xiang, Joo, et al. You2me: Inferring body pose in egocentric video via first and second person interactions. CVPR, 2020.
[64] Liang, H., W. Zhang, W. Li, et al. Intergen: Diffusion-based multi-human motion generation under complex interactions. International Journal of Computer Vision, 132(9):3463-3443, 2024.
[65] Siyao, L., T. Gu, Z. Yang, al. Duolando: Follower gpt with off- reinforcement learning for dance accompaniment. In The Twelfth International Conference on Learning Representations.
[66] Kingma, D. P., M. Welling, Auto-encoding variational bayes. In International Conference on Learning Representations (ICLR). 2014.
[67] Radford, A., J. W. Kim, C. Hallacy, et al. Learning transferable visual models from natural language supervision. International conference on machine learning, pages PMLR, 2021.
[68] Kingma, D. J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations. 2014.
[69] Ho, J., T. Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications. 2021.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly discuss the limitations of existing methods in multi-character scenarios and analyze their shortcomings conceptual level. These points align well with the paper's actual contributions and set the scope appropriately for the proposed approach.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, the paper discusses the limitations of the proposed method, primar- ily highlighting the lack of high-quality datasets specifically curated for multi-character scenarios.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
15

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include formal theoretical results or proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the suppiemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: The implementation details are thoroughly described in the main text (Sec- tion 3.2), and all hyperparameter configurations are provided. This ensures that the main experimental results can be fully reproduced.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We will release the code after publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. ec/
sublic/guidess httae//guiiee/oeeebbiimeaeee
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips cc/ httpppullllplppiee https//ooooouaaassssooooooool111
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We conducted extensive ablation studies and hyperparameter sensitivity exper- iments. All relevant training and testing details, optimizer types, and parameter settings, are clearly specified in the experimental section and further elaborated in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
We don't have experiment need to report error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide detailed information regarding the computational resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification:
We ensure full compliance with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This work offers significant positive societal impacts. It greatly reduces production costs and time, enabling more creators to produce high-quality animations efficiently and fostering innovation in digital entertainment. The technology enhances user experience across gaming, virtual reality, and film industries. While there are potential risks, such as misuse for creating deceptive or deepfake content, these concerns can be mitigated through access control.
18

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymenn of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: We have considered potential misuse risks of our models and data. To ensure responsible release, we plan to implement usage guidelines and access restrictions.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes].
We have properly cited the original papers for all the code, data, and models used in our work, respecting their licenses and usage terms.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
19

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
patasess.paperssitt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: New assets will be released after publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We provide detailed instructions given to participants and related information in the appendix.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: We conducted only low-risk anonymous questionnaires, which do not involve sensitive personal data pose risks to participants.
Guidelines:
20

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The research does involve the of LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
21

---

Appendix
A Preliminary: ReferenceNet Based Character Animation
Character animation aims to synthesize realistic character videos from a single reference image and a driving pose sequence. Formally, given reference image Zref and target pose sequence {p: ) the goal is to generate a video {Ir)t=1 where each frame Tt maintains visual consistency with Tref while conforming the pose Pc-
[3] builds upon the Stable Diffusion [47 framework, which performs denoising in the latent space. Let Enc and Dec denote the encoder and decoder of the latent diffusion model. The reference image is first encoded into a latent representation Zret Enc( (err, and each frame is generated from noisy latent inputs 24 (N through conditional denoising process:
Most of character animation approach [2:
10: 9
20 Denoise(zr, (p:)/=1, Tnet);
where the denoising process is iteratively performed by UNet-based network to recover 2o, which then decoded by Dec(zg) to reconstruct the video frame.
To preserve the appearance consistency of Tyefs [9] introduce ReferenceNet, a UNet-like structure
R- Net designed to extract spatially detailed features from the reference image. Specifically,
R- - Net produces intermediate features fiet E RR which are fused into the main denoising
UNet co via apatial-attentin mechanism:
Attmspatial
( (I1, Ø2) = SelfAttention (Concat(x1, (2))
(10)
Here, E IR [g+xhxxxxe is the feature from the denoising UNet, and x2 E Rxxuxe from ReferenceNet. The operation repeats 22 along the temporal axis and performs attention, then extracts the first half as the refined output, ensuring that spatial detail flows from the reference into each frame.
In addition, high-level semantic features from the CLIP image encoder used in cross-attention to condition the denoising on global content, complementing ReferenceNet's local details.
By aligning both low-level and high-level cues from the reference image, and integrating them into the diffusion denoising pipeline, the ReferenceNet significantly enhances the temporal and spatial fidelity of character animations. Its design ensures efficient inference, R - Net is executed only once per sequence, while maintaining consistency across all video frames.
B
Other Details
B.1 Edeeeity-Embedded Guidance
Although AlphaPose (55) provides built-in identity tracking alongside multi-character pose estimation, we observed that its tracking module is not sufficiently reliable for complex multi-character scenarios. In particular, we encountered the following common failure cases: (1) Identity Confusion: Different individuals are incorrectly assigned the same identity label due to visual similarity (e.g., similar clothing); (2) Cruss-identity Misassignment: Skeletal data from multiple individuals are incorrectly associated, resulting in identity mismatches; (3) Temporal Inconsistency: The identity assigned to person changes from frame to frame, often due to occlusion or tracking errors.
These identity-related failures can severely hinder the downstream learning of accurate identity correspondence by introducing noisy supervision and temporal jitter. To mitigate such tracking errors, decouple pose estimation and identity assignment: multi-character pose maps extracted using DWPose [54], while consistent identity anchors are derived from instance-level bounding boxes produced by SAM2 [52]. This hybrid approach yields pose embeddings that are structurally faithful and identity dinrriiyddisrimmiatieee, reducing errors caused by tracking failures.
We utilize DWPose to extract multi-character poses from individual image frames, where each detected pose is represented by a set of 2D keypoints with corresponding confidence scores. However, the output poses are unordered and lack explicit identity labels. To address this limitation, we use SAM2 to generate instance-level bounding boxes, with each bounding box assigned a unique identity label, which will serve persistent identity label between frames.
22

---

Time
63
131
190
196
279
Frame
soquert Prquent
Alphapose
IEG
Figure 8: AlphaPose exhibits severe identity inconsistency. Using the skeleton color from frame I as identity reference, the same female character is mistakenly reassigned from green (frame 1) to blue (frame 131), and the male character from red to blue (frame 279). In contrast, our IEG maintains consistent identity assignment throughout the sequence.
For each candidate pose, we calculate the proportion of its keypoints that fall within each bounding box. pose-box pairing is accepted only the ratio of enclosed keypoints exceeds a predefined threshold, ensuring robustness in cluttered or partially occluded scenes. This strategy effectively preserves the temporal continuity of identity embeddings (see Figure 8)
Each matched skeleton is labeled with an identity-specific color to distinguish different characters. This color-encoded representation serves as our Ddentity-Embeede Guidance (IEG). This strategy obviates the need for additional training or hyperparameter tuning, exhibits strong robustness in complex multi-character scenarios involving position swaps, and retains high generalizability across different pose estimation backbones and instance-level segmentation frameworks.
B.2 Training Pipeline
To formally revisit the core training pipeline described in the main text, summarize the overall procedure as follows. During training, the model learns to generate multi-character video frames with correct IC by jointly optimizing the diffusion reconstruction loss and the identity matching graph (IMG)
IC loss. Initially, the target frame sequence X, is passed through a VAE encoder obtain the latent representation 2o, and instance segmentation masks (MT)? ) extracted via SAM [52]_ At each diffusion timestep I,, noise & is injected into Zo to produce z/, which, alongside identity-embedded pose guidance c", cf and semantic features s" of the reference image (according to 1 we additionally include c" to inject identity information), is processed by the UNet backbone predict Concurrently, V selected UNet layers, the intermediate features f" f and interpolated masks M", .M9 are used to construct IMG nodes rss Py- Mask-Query Attention computes affinities w (r:, 9j), from which the layer-wise consistency score c( is derived and aggregated into the matching loss Lmatch. With probability P. challenging swap-pair samples selected via pre-classified sampling to emphasize identity-switel scenarios. As shown in the Algorithm []
B.3 Inference Pipeline
During inference, EverybodyDance generates multi-character video frames conditioned on reference frame, reference pose, and a driving pose sequence, without constructing the IMG or computing any matching loss. The pose guidance is pre-processed in IEG c", c' to indicate character identities. The driving pose et is added with initial noise to predict The iterative denoising process follows the standard DDIM schedule, gradually refining 2; back zo. Finally, the VAE decoder reconstructs
23

---

Algorithm Training Pipeline of EverybodyDance
Require:
Target frame Xg, IEG c², c', masks (Mf), sampling ratio p. reference image, total training
steps
Ensure: Model parameters 0
1: for step to S do
2: Sample challenging pair w.p- / else random
3: Zg < VAE.Encodeff
4: Extract reference semantic features s" by R Net
5:
g ~ (O, I). Ze f Var 2o UNe 24, c², ,,,, ,s,,",, Caitt || 1 El?
v1 8
6:
7:
8:
for f = N do Extract f" f interpolate masks, construct nodes fr, = f" @ M. MM Jiz
9:
for j = to n do
11:
9j = fd o M Tall = E
12:
Compute {w (ri, (ri,g;)) mm) MQA(ran..))) 99))
13:
end for Compute (()) end for
14:
15:
Lmatch \/ 5. -C()
17:
Update - VoVon + XLrmatc))
18:
end for
Algorithm 2 Inference Pipeline EverybodyDance
Require: Reference image, driving poses IEG {ct) reference IEG c²
Ensure: Generated frames 17
1: Z, +- +-N(0,1) I)
2: Extract semantic features s" by Net
3: for ==
do
4: Sample ê < UNets c²,, """c''s"" s",,)
5: Compute 2 via DDIM 200
6: end for
7:
VAE.Decode(zo)
the RGB frame, producing temporally coherent multi-character video with accurate IC. As shown in the Algorithm
B.4 End2End End2End-M
We introduce two alternative training strategies assess the effectiveness of the Identity Matching Graph (IMG). End2End refers fully end-to-end training pipeline where the IMG construction is involved, and the model is trained solely with standard video generation losses. End2End-M, on the other hand, replaces the original Identity Correspondence (IC) loss derived from the IMG with a simplified constraint: an L2 loss computed over the masked region the reference image. This provides coarse form of identity guidance without constructing the full identity matching graph. These two strategies represent training without identity constraints (End2End) and with weak identity constraints (End2End-M), respectively.
C ICE-Bench
Although Follow- Hoello--Yuu-PPe Your- Pour-Pose-VV V2 (50) introduced the Mulli- Character benchmark for multi-character animation, it features relatively simple character interactions and only limited occlusion scenarios. To address this critical gap, we introduce Identity Correspondence Eyaluation benchmark (CFn-,, the first multi multi-character benchmark to evaluate IC performance in multi-characte animation tasks.

---

ICE-Bench consists of carefully curated video clips dance video clips totaling more than 3,200 frames. ICE-Bench is deliberately designed stress test identity correspondence challenging interaction scenarios. We adopt multi-criteria filtering strategy to ensure that the benchmark presents sufficient challenges. We first select clips that exhibit clear multi-character interactions, such as positional exchanges and occlusions, discarding those with minimal interaction. Second, we retain videos with stable lighting, minimal blur, and clear joint visibility to ensure visual quality and reliable pose extraction. Finally, we balance diversity and difficulty by including varied dance genres and environments.
D
More Experiment Details
D.I Implementation Details
We employ the Animate Anyone [9] the backbone for animation pipeline. In our setup, both the ReferenceNet and DenoisingNet utilize shared pose guider. The construction the IMG does not involve adding any additional modules; directly leverage the original spatial attention blocks already present in the DenoisingNet. The training process divided into two stages. Stage focuses on Linglefrrmme spatial quality, while Stage 2 prioritizes temporal coherence in video sequences. Stage was trained for 5,000 steps while Stage 2 was trained for 1,000 steps, with the VAE [66] encoder and CLIP [67] encoder frozen throughout. In Stage 1, the ReferenceNet, DenoisingNNtt, and Pose Guider are trainable. A batch size of 32 is applied, using center-cropped 768x768 resolution images. For Stage 2, only the temporal attention modules are trainable, with batch size of 8 and training sequences comprising 24 consecutive frames sampled at 3-frame intervals. Video frames are processed at 5 51x55 12 resolution. We set the learning rate 2.0 e- and use the Adam [68] optimizer. During the inference phase, we employ the DDIM scheduler with 50 denoising steps. We set the classifier-free free guidance [69 scale to 3.5.
D.2 Dataset
We construct MultiDance dataset, a dataset tailored for multi-character animation. Existing datasets predominantly designed for single-person motion transfer and lack both rich multi-character interaction. MultiDance comprises 814 multi-character dance video clips at 1080p resolution, totaling 257K frames. It covers diverse range challenging scenarios, including dynamic position exchange and partial occlusion. Notably, approximately 30% of the frames contain relatively complex interactions (e.g. positional swaps or occlusion). Dance styles span various categories such as pop dance, street dance, aerobics, and ballet, filmed in both indoor (well lit) and outdoor (evenly illuminated) environments.
E
More Experiments
E.1
Training Curve
We record the IC score (cooresponding to the matching loss Lmarch) curve throughout the training process. shown in Figure which clearly reflects the model's progressively improving ability to capture identity correspondences- - increasing from 0.55 to approximately 0.70.
E.2 More Characters
To further evaluate the effectiveness of our method in more complex settings, we increase the number of characters involved in the generation process. To assess generalization capability, we employ video clip as the source of the pose sequence and use single reference image to guide the generation of an entire video. Importantly, the relative spatial arrangements of the characters differ between the reference image and the target poses, presenting a more challenging scenario for preserving accurate identity correspondences. The qualitative results presented in Figure [I
25

---

Tgt Pose
Ref Ing
Gen Frunes
Tgt Pose
Ref Ing
Gen Frunes
Pose
Ref Ing
Gen Frunes
Tgt
Ref 1ng
Gen Frunes
1gt Pose
Ref ING
Franes
Tgt Pose
Ref Ing
Franes
Tgt Pose
Ref Ing
Gen Franes
Figure 9: Our performance on Yollow-Your-Pose-22 Multi-Character pulic benchmark.
26

---

IC Score During Training
0.725
0.700
0.675
Score
0.650
0.625
0.600
0.575
0.550
1000
2000
3000
4000
5000
Step
Figure 10: Identity correspondence score during stage training.
Source
Target Pose
Gen Sequence
Ref Ing
Source
Pose
Pose
Target
Pose
Sequence
Ref Ing
Figure 11: Even with more character identities, our method consistently maintains accurate identity correspondence.
27

---

Table Quantitative comparison of different affinity computation methods on our ICE benchmark.
Our proposed IMG+MQA approach demonstrates superior performance in both frame-level quality and temporal video consistency.
Method
SSIM
PSNR*
LPIPS 4 FID 4
FID-VID
FVD
CrossAttn- -aaee
0.648
15.69
0.315
45.82
29.22
306.75
Sim-Based
0.629
16.25
0.329
47.09
24.35
269.14
Ours (IMG+MQA)
0.654
16.93
0.304
40.19
23.58
225.06
F
Further Analysis on Affinity Computation
While our primary approach utilizes the Identity Matching Graph (IMG) with Mask-Query Attention (MQA), we also explored alternative paradigms for affinity computation to further validate design choices. One inspiring direction comes from methods like Ingredients [49], which use per-character CLIP embeddings for layout control. We designed and evaluated two alternative variants optimized with a standard cross-entropy classification loss. The implementation details are as follows:
CrossAtin- In this variant, we first segment the reference image and encode each of the m characters into distinct identity features using CLIP encoder. Within the Cross- Attention layers of our model, instead of attending to a single CLIP embedding for the entire image, we compute the affinity between the latent features of the generated frame and each of the m individual character embeddings. After applying a softmax function and masking the background, this process yields affinity map of shape (HW, m.]. where each entry represents the probability given latent patch belonging to one of the reference characters.
Sim- - This approach relies on direct feature similarity. We use segmentation masks to extract intermediate features for the reference characters and generated characters. We then compute an [n, m] affinity matrix by taking the dot-product between each generated and reference character's feature representation, followed by a softmax operation. Each entry (i.j) in this matrix indicates the similarity score between the i-th generated character and the j-th reference character.
As shown in Table 5 our proposed IMG+MQA framework outperforms both variants across all quantitative metrics. The CrossAttn Based method, while reasonable, relies on an external CLIP space that is not inherently tied to the generative model's internal feature representation, creating a disconnect. The Sim-Based approach performs worse, particularly struggling to differentiate between characters with visually similar features. We chose the IMG+MQA paradigm because computes affinity using the model's native spatial attention scores, which are intrinsically coupled with the denoising and generation process. This inherent coupling ensures that the supervision signal for identity correspondence directly influences the most relevant parts of the network responsible for spatial layout and appearance, leading superior performance.
G
User Study
To comprehensively evaluate the perceptual quality of the generated multi-character videos, we conduct user study from three complementary perspectives: fidelity, coherence, and identity correspondence (IC) accuracy.
For the study, we collected approximately 200 sets of human feedback from group of university- educated participants. In each evaluation session, participants were shown the video clips generated by all competing methods for the same input. They were then asked to score each video on scale from (worst) to (best) for the three criteria independently. The final user preference scores are aggregated provide quantitative comparison, shown in T
Specifically, fidelity measures how realistic and visually appealing each frame appears; coherence assesses the temporal consistency and motion smoothness across frames; and IC accuracy evaluates whether the identity correspondences between the generated video and the reference images are correctly maintained as expected.
28

---

Table 6: User study results based on three quality dimensions: Fidelity, Coherence, and Identity Correspondence (IC) Accuracy.
Fidelity
Coherence
IC Accuracy
Method
Mean
Var.
Mean
Var.
Mean
Var.
Everybody Dance
4.40
0.48
4.32
0.55
4.57
0.38
UniAnimate
3.01
1.22
3.61
2.12
0.80
MagicDance
3.73
1.20
2.80
1.34
1.99
0,87
AnymateAnyone
3.05
1.31
3.25
1.28
2.33
1.07
MagicAnimate 2.13
1.14
2.25
1.26
1.83
0.69
In the user study, participants are shown series of video clips generated by different methods and are asked to score them on scale from I (worst) to (best) for each of the three criteria independently. The final user preference scores then aggregated to provide quantitative comparison across different methods, as shown Table
29


Reviews:
[{"review_id": "BRXvaTWcL1", "reviewer": "Reviewer_acUv", "summary": "This paper aims to extend the one-character pose-driven animation task to multi-characters and propose the EverybodyDance method to solve this problem.  EverybodyDance explicitly builds the character correlation between the reference image and target video clips instead of simply implicitly mapping like ReferenceNet. Specifically, EverybodyDance introduces the Identity Matching Graph (IMG) to connect the characters from different sources and uses character latent features to represent the node and Mask–Query Attention (MQA) to represent the match weights of each character pair.  Other techniques like identity-embeded guidance, a multi-scale matching method and pre-classified sampling are proposed to improve model robustness and IC accuracy. Finally, a new Multi-Character Animation  benchmark dataset is introduced to evaluate the effectiveness of EverybodyDance  and other baselines. EverybodyDance achieves superior performance than other baselines.", "questions": "Please mainly address the questions in the weakness section.", "limitations": "The paper slightly mentioned the limitations in the end of the paper.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths:\n  - This paper proposes an explicit identity mapping method to solve the task of multi-character animations. \n  The key techniques Identity Matching Graph (IMG) and Mask-Quey Attention(MQA) sound novel and reasonable.\n  - The quantity and quality results in a multi-character benchmark demonstrate the effectiveness of EverybodyDance. \n\n# Weaknesses:\n  - To my best of knowledge, EverybodyDance is not the first paper to solve the Multi-Character pose-driven Animation task. Previous work like [1] has proposed a method to solve this task. More discussions and comparisons are necessary to added to the main paper.\n  - From the visualization of figures in this paper, there are almost two characters in reference images and target video clips. Does the training and test dataset consist of samples with only two characters? Can EverybodyDance also be extended to reference images of more than 2 characters? More distribution descriptions about the multi-character dataset and target visualization would be better to show the effectiveness of EverybodyDance.\n  - EverybodyDance is finetuned with a custom multi-character dataset based on the AnimateAnyone and achives better performance than baselines in ICE benchmark. Because of the finetuning paradigm, we doubt that the FT process may introduce multi-character  bias and further destroy generalization for reference images with single characters. It is also necessary to make comparisons in test dataset involving single characters.\n\n\n[1] Xue J, Wang H, Tian Q, et al. Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control[J]. arXiv preprint arXiv:2406.03035, 2024.", "was_revised": true, "final_justification": "The rebuttal well addressed some of my concerns. After a few rounds of discussion with the authors, most of concerns are resolved. Thus, I would raise the final rating to borderline accept.", "rebuttal": "## **I. Concern about Missing Related Work**\n---\nThank you for pointing out this important related work and for emphasizing the necessity of a direct comparison. We completely agree that this discussion was not highlighted sufficiently in the main body of our initial submission. We are grateful for your feedback, as it gives us the opportunity to clarify the detailed comparison and core differences between our work and \"Follow-Your-Pose v2\" (FYP-v2).\n\n### *A. On Experimental Comparison:*\n\nWe directly compared with FYP-v2, as shown in **Appendix E.1 (Table 1 and Figure 2)** of supplementary material. Although the official code and weights for Follow-Your-Pose v2 have not been released, their publicly available benchmark enabled us to perform a fair and direct comparison. For your convenience and clarity, we summarized the comparison results below. As shown in the table, our method achieves superior performance on FYP-v2's official multi-character benchmark across all image and video quality metrics.\n\n| Method | SSIM↑ | PSNR↑ | LPIPS↓ | L1↓ | FID↓ | FID-VID↓ | FVD↓ |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| DisCo| 0.793 | 29.65 | 0.239 | 7.64E-05 | 77.61 | 104.57 | 1367.47 |\n| DisCo+| 0.799 | 29.66 | 0.234 | 7.33E-05 | 73.21 | 92.26 | 1303.08 |\n| MagicAnime| 0.819 | 29.01 | 0.183 | 6.28E-05 | 40.02 | 19.42 | 223.82 |\n| MagicPose| 0.806 | 31.81 | 0.217 | 4.41E-05 | 31.06 | 30.95 | 312.65 |\n| AnimateAnyone| 0.795 | 31.44 | 0.213 | 5.02E-05 | 33.04 | 22.98 | 272.98 |\n| AnimateAnyone†| 0.796 | 31.10 | 0.208 | 4.87E-05 | 35.59 | 22.74 | 236.48 |\n| FYP-V2 | 0.830 | 31.86 | 0.173 | 4.01E-05 | 26.95 | 14.56 | 142.76 |\n| **Ours** | **0.879** | **32.49** | **0.151** | **0.92E-05** | **26.01** | **12.68** | **127.36** |\n\n### *B. On the Fundamental Difference in Problem Formulation:*\n\nBeyond the advantages in quantitative metrics, we wish to further highlight the **fundamental difference** in problem formulation between our work and FYP-v2, which constitutes our core innovation. The challenges of multi-character animation can be divided into two levels:\n\n* Level 1: Feature Interference: This refers to avoiding interference of appearance features (e.g., clothing) when multiple characters are positioned side-by-side in the same frame. FYP-v2 made significant progress in addressing this challenge through its depth order guider.\n\n* Level 2: **Spatial Controllability:** This refers to a model's fundamental ability to control *which* character appears at *which* spatial location. A failure in this controllability could lead to the most critical error: e.g., character A appearing with character B's appearance after they swap positions. This is a more profound and difficult challenge than mere feature interference.\n\nOur work, \"Everybody Dance,\" is the **first** framework to systematically address this Level 2 challenge of spatial controllability. Our primary contribution is ensuring robust Identity Correspondence (IC), accurately tracking and controlling character identities as their spatial layout changes.\n\nIn fact, it is candidly acknowledged that spatial controllability remains a limitation of their method in **Appendix D of FYP-v2 paper**. Our work directly addresses this key problem that the prior work didn't cover.\n\nIn summary, our work not only surpasses FYP-v2 in performance on its own benchmark, but, more importantly, defines and systematically solves the profound challenge of spatial controllability. We will integrate this important discussion and comparison in the final version. Thank you again for your valuable suggestion.\n\n## **II. Concern about More Character Generalization**\n---\nWe are grateful for your insightful questions regarding our method's scalability. We appreciate the opportunity to clarify this crucial point.\n\nAbout our training data, the original MultiDance dataset consists of two-person videos. This makes your question about generalization to more characters a critical test for our method. Our goal was to demonstrate that our model learns the *general principle* of identity correspondence, rather than simply remembering two-person patterns.\n\nFor visual illustration, we kindly invite you to refer to **Appendix Figure 4** in our supplementary material. This figure showcases a challenging three-person scene, which serves as a 'zero-shot' test since the model was not trained on such configurations. We hope it provides qualitative evidence for the robustness of our method.\n\nHowever, to make this evaluation more comprehensive and to further address your concern, we constructed a new, challenging benchmark as a supplement to our original ICE benchmark. This new benchmark contains approximately 2,000 frames containing complex interactions between **3 to 5 characters**. On this new benchmark, our method demonstrates significant and consistent improvements, quantitatively confirming its scalability:\n\n| Method | PSNR*↑ | LPIPS↓ | SSIM↑ | FID↓ | FVD↓ | FID-VID↓ |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| AnimateAnyone | 14.70 | 0.370 | 0.606 | 56.66 | 401.51 | 35.356 |\n| AnimateAnyone*| 14.92 | 0.363 | 0.607 | 64.34 | 406.08 | 36.308 |\n| UniAnimate | 15.62 | 0.338 | 0.640 | 57.75 | 348.91 | 29.030 |\n| **Ours** | **16.68** | **0.315** | **0.671** | **42.84** | **261.01** | **23.571** |\n\nFurthermore, a key advantage of our framework is that this scalability comes with **no additional** computational overhead at inference time, as the Identity Matching Graph (IMG) is only constructed during training.\n\nIn summary, both our original qualitative evidence and these new quantitative results on a challenging 3-5 person benchmark support our method's generalization performance. We will include these new results and benchmark to our final version.\n\n## **III. Concern about Single-Character Generalization**\n---\nThank you for this insightful suggestion. Your concern about our multi-character fine-tuning might affect single-character performance is reasonable. We found that our method achieves an overall performance improvement on the single-character benchmark, and we conducted an ablation study to verify the reason of this improvement. The experiments were conducted on the **public** TikTok (Jafarian et al., 2021) benchmark.\n\n| Method | PSNR*↑ | LPIPS↓ | SSIM↑ | FID↓ | FVD↓ | FID-VID↓ |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| Moore-AnimateAnyone | **17.85** | 0.280 | 0.768 | 52.15 | 209.14 | 25.864 |\n| AnimateAnyone* | 17.19 | 0.291 | 0.764 | 62.52 | 213.68 | 25.943 |\n| w/ IEG | 17.41 | 0.289 | 0.766 | 51.52 | 195.42 | 24.515 |\n| End2End | 17.65 | 0.288 | 0.770 | 45.25 | 186.34 | 22.515 |\n| End2End-M | 17.62 | 0.285 | 0.769 | 43.17 | 182.16 | 22.881 |\n| **Ours** | 17.78 | **0.279** | **0.772** | **40.56** | **163.85** | **20.294** |\n\nOur analysis is as follows:\n\n* First, simply fine-tuning the backbone on our multi-character dataset (`AnimateAnyone*`) leads to a performance drop on the single-character task.\n* To isolate the effect of different driving pose guidance, we fine-tuned the backbone using only our Identity-Embedded Guidance (`w/ IEG`), which is slightly better than `AnimateAnyone*`.\n* Next, both `End2End` and `End2End-M` introduce reference IEG of the reference image for labeling identity (see our Appendix B.4 for details). The results show that introducing IEG for identity guidance is beneficial.\n* Finally, our full method (`Ours`) achieves the best overall results. This demonstrates the effectiveness of our **Identity Matching Graph (IMG)**."}, {"review_id": "y7vH9GOqdD", "reviewer": "Reviewer_SZsZ", "summary": "The paper proposes a method to address the identity preserving issue in multi-character animation video generation. It proposes a bipartite matching between reference character and generated character frame-wisely. By formulating the correspondence matching into a matching loss and applied during training, the method is able to better maintain identity in the generation and outperform existing approaches consistency under different settings. Experiments show improved generated video quality on the proposed ICE benchmark.", "questions": "1.  **Instance segmentation**\n\nIn order to segment the generated character in each generated frame, does that mean during each training iteration, the method has to run segmentation on the generated results? In the early state training, the generation could be poor and how to ensure it could segment the generated character correctly. Also, what is training overhead by applying such segmentation per iteration. \n\n2. **How to determine the number of person**\n\nI would like to know how to determine m and n, the number of characters for reference and generated. For different inputs with different number of person, how does the current method count number of character in the generation process and do the matching. \n\n3. **Occlusion**\n\nI am curious how does the IMG address occlusion issue in occluded frames, without consider temporal information, the per-frame setting will suffer from bad matching for occluded frames.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n**Important task**\n\nMaintain identity consist across videos is a important task. The paper does the weakness analysis of existing methods and show the importance of addressing such issue in multi character animation.\n\n\n**Good quantitative results compare to existing approaches**\n\nThe method outperform other methods under the same setting. I appreciate the author conducts a strict comparison and remove the influence of different datasets. \n\nWeakness \n\n**No temporal correspondence in the proposed IMG**\n\nThe bipartite graph is constructed per frame and there is no consistent person match across time, which loss the temporal consistency. I would like to know why not consider the temporal information in IMG and how current IMG could improve the temporal consistency by only considering the matching framewisely.\n\n**Affinity computation**\n\nI am not fully convinced the affinity could reflect the person identity information. As the feature contains the pose information, how to ensure the affinity score has remove the pose information for both reference and generated character. \n\n**Scaling person**\n\nI understand the current dataset doesn't contain too much person. But i am highly doubt the proposed method could work for larger number of person input. As most of results only showcase two person (which is quite simple in matching), I wonder how does the paper work for a group of person and does the training still applicable to such larger number, as the current training needs instance segmentation per iteration.", "was_revised": true, "final_justification": "After reading the rebuttal, I believe the authors have addressed most of my concerns. The experimental results on both multi-character and occlusion datasets outperform all other methods across all metrics. However, I still have some reservations about the evaluation, as it relies on indirect metrics that are not fully convincing (not directly judge the character consistency).  The method achieves the best results even without explicitly handling occlusion, which raises slight questions about the robustness of the evaluation. \n\nThe original paper lacks experiments, while the rebuttal gives sufficiently large experiments and all those tables should be reported in the final version. Considering this, I will raise my score to borderline accept.", "rebuttal": "## **I. General Response**\n---\nThank you for your valuable feedback. You've raised excellent points that helps us clarify our system design. Our method is a **systematic solution**. Our core innovation is the Identity Matching Graph (IMG). However, it needs to cooperate with other components (e.g., Identity-Embedded Guidance), which are also designed to address some of important challenges you concerned. The core of our task, identity correspondence for multi-character animation, lies in **spatial controllability**—*determining which character appears at which location and performs what pose*. This differs from identity preservation. \n\n## **II. Concern about Temporal Correspondence**\n---\nThank you for your valuable question. To clarify upfront, this key to achieve temporal consistency is not handled by IMG, but a simple yet critical component of our system: the **Identity-Embedded Guidance (IEG)**. Your observation is correct: our frame-wise Identity Matching Graph (IMG) doesn't handle this role directly. Its purpose is instead to supervise the model's ability to follow the IEG's identity guidance during training:\n* **What IEG is:** IEG is an identity guidance signal that embeds a **persistent** Identity(ID) (represented by different color) into the pose skeleton. This ID remains consistent for a specific character throughout the whole video, even if the characters swap positions. As shown in our **Appendix's Figure 1**.\n* **When IEG is Used:** IEG is utilized as the guidance signal during both training and inference.\n* **What IEG Offers:** A reference IEG specifies each character's ID in the reference image, and a target IEG sequence provides the corresponding motion trajectory for each character.\n* **How IEG works:** IEG guides the model to *generate the correct character at the correct position*. As shown in the **Figure 7**, by swapping the ID in the input target IEG sequence, our model correctly generates the characters in their new, swapped positions while maintaining their unique appearances.\n* **How IEG collaborates with IMG:** During training, the IEG functions as the **ID instructor**, providing identity labels and pose guidance, while the IMG serves as a **supervisor**, ensuring the model learns to follow the IEG's ID instruction.\n* **How IEG generated**: IEG is generated using cross-frame identity tracking and pose estimation (see our Appendix B.1).\n\n| Method            | SSIM↑     | LPIPS↓     | FID↓     | FVD      |\n| ----------------- | --------- | ---------- | -------- | -------- |\n| w/o Reference IEG | 0.619     | 0.331      | 42.84    | 251.05   |\n| w Reference IEG   | **0.654** | **0.304**  | **40.19** | **225.06** |\n\nWe designed a simple experiment. When using the same model for inference, one version does not mark the identity of the reference character (`w/o Reference IEG`), while another version does (`w Reference IEG`). We see that without identity instruction, the model’s performance decreases significantly, especially on the FVD metric, which reflects temporal coherence.\n\nFurthermore, the **Temporal Attention** blocks within our model provide temporal smoothness during the generation process. However, we agree that your suggestion about incorporating temporal correspondence within the IMG is a promising direction for future research.\n## **III. Concern about Instance Segmentation**\n---\nThank you for your reasonable question regarding our implementation about segmentation.\n\nTo clarify directly, the segmentation is **not** performed on the `generated` image during training or inference. Instead, it is performed offline on the ground-truth (GT) `target` and  `reference` image from the training video samples. During training, the `target` image serves as the GT of the  `generated` image. Therefore, we only use the clean, pre-computed mask of the `target` image. Since this is a one-time, offline pre-process procedure of the dataset, the training loop is free from expensive SAM2 segmentation **computational overhead**. Furthermore, the segmentation **accuracy** is reliable because we only extract masks from GT `target` frames, not the potentially poor `generated` images.\n\n## **IV. Question about Determining the Number of Persons**\n---\nThank you for this critical question. We will clarify how the number of characters is determined during the construction of the Identity Matching Graph (IMG). For IMG construction, it **dynamically** determines node number by the training sample:\n\n- `m` is the number of skeletons for the reference image, each paired with a pre-computed unique SAM2 mask for IMG node construction.\n- `n` is the number of skeletons for the target image, each also paired with a pre-computed unique SAM2 mask for IMG node construction.\n\nIn short, `m` and `n` are determined by the number of skeletons, without an explicitly counting mechanism.  A case can be found in **Appendix's Figure 4**, where the target IEG contains three skeletons, our model adaptively generates three characters. As for \"**how to do the matching**\", IMG is dynamically constructed from various `m` and `n`. If you have any further questions, we are willing to discuss then with you.\n\n## **V. Concern about Scaling Person Number**\n---\nThank you for this crucial question on scalability. You are correct that our training data consists mainly of two-person scenes, which makes generalization to more characters critical test for our method.\n\n### *A. Generalization to More Characters*\nOur Appendix includes a qualitative result on a challenging three-person scene (**Appendix Figure 4**). To provide comprehensive quantitative proof, we further conducted new experiments on a challenging benchmark comprising **3-5 characters** scene. The results below confirm that our method's effectiveness and generalization performance:\n\n| Method | PSNR*↑ | LPIPS↓ | SSIM↑ | FID↓ | FVD↓ | FID-VID↓ |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| AnimateAnyone | 14.70 | 0.370 | 0.606 | 56.66 | 401.51 | 35.356 |\n| AnimateAnyone*| 14.92 | 0.363 | 0.607 | 64.34 | 406.08 | 36.308 |\n| UniAnimate | 15.62 | 0.338 | 0.640 | 57.75 | 348.91 | 29.030 |\n| **Ours** | **16.68** | **0.315** | **0.671** | **42.84** | **261.01** | **23.571** |\n\nFurthermore, a key advantage of our framework is that this scalability comes with no additional computational overhead at inference time, as the Identity Matching Graph (IMG) is only constructed during training.\n\n### *B. Does the Training Still Applicable?*\nYes, the training process remains fully applicable. As discussed in our response to Question IV, the construction of the IMG is an entirely dynamic process. The number of nodes for any given training sample is determined by the number of character masks pre-computed from that video clip. Consequently, our training framework supports a variable number of characters without modifications.\n\n## **VI. Concern about Affinity Disentanglement**\n---\nThank you for this question, as it touches upon the core of our learning mechanism. First, let's assume that the \"pose\" you mentioned is the \"posture\" such as sit or stand. If this is not the case, please don't hesitate to further inform us. We don't explicitly remove pose information from the features. Instead, our training framework is designed to **mitigate** its influence.\n\nThis is demonstrated in **Figure 6** (third row). In this example, both reference characters have similar postures. If posture information were the important factor, the affinity map would show a strong response for both. However, as can be seen, our method's attention is sharply focused on the correct reference character, proving that our training effectively weakens the influence of pose.\n\nIf the model were to rely on posture similarity and assign high affinity to a wrong reference character, our IMG loss (`L_match`) would penalize this. Therefore, to minimize the loss, the model is forced to follow the IEG's identity instruction (as we discussed in Response II), downplaying the role of posture information.\n\n## **VII. Concern about Occlusion**\n---\n### *A. How to Operate*\n\nThank you for your valuable question on occlusion. During training, IMG rely on ground-truth masks, which can be extracted even when characters are occluded. Our IEG provides the correct identity label for that character, and the IMG uses this information to compute a matching loss. IMG teaches the model to associate an occluded appearance with the correct, full identity from the reference image. We agree that your suggestion to incorporate more direct temporal modeling is an excellent direction for future work, especially for handling severe occlusion.\n\n### *B. Performance on Occlusion Benchmark*\n\nWe quantitatively validated our method's performance on the **public** Follow-Your-Pose-V2 benchmark, which is specifically noted for its frequent and challenging inter-person occlusions. As reported in the **Appendix (Table 1)**, EverybodyDance achieves SOTA results, outperforming all baseline methods across quality metrics on this benchmark.\n| Method | SSIM↑ | PSNR↑ | LPIPS↓ | L1↓ | FID↓ | FID-VID↓ | FVD↓ |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| DisCo| 0.793 | 29.65 | 0.239 | 7.64E-05 | 77.61 | 104.57 | 1367.47 |\n| DisCo+| 0.799 | 29.66 | 0.234 | 7.33E-05 | 73.21 | 92.26 | 1303.08 |\n| MagicAnime| 0.819 | 29.01 | 0.183 | 6.28E-05 | 40.02 | 19.42 | 223.82 |\n| MagicPose| 0.806 | 31.81 | 0.217 | 4.41E-05 | 31.06 | 30.95 | 312.65 |\n| AnimateAnyone| 0.795 | 31.44 | 0.213 | 5.02E-05 | 33.04 | 22.98 | 272.98 |\n| AnimateAnyone†| 0.796 | 31.10 | 0.208 | 4.87E-05 | 35.59 | 22.74 | 236.48 |\n| FYP-V2 | 0.830 | 31.86 | 0.173 | 4.01E-05 | 26.95 | 14.56 | 142.76 |\n| **Ours** | **0.879** | **32.49** | **0.151** | **0.92E-05** | **26.01** | **12.68** | **127.36** |"}, {"review_id": "0W2ESFMu4B", "reviewer": "Reviewer_QHRK", "summary": "The paper introduces a multi-identity feature association graph during the training phase to address the issue of confusion in multi-person video generation, and builds corresponding evalution benchmark. In detail, a mask query attention is proposed to optimize the identify graph. The experiments demonstrate the outperformances in identify matching and visua fidelity", "questions": "1. Lack of related work listed above.\n\n2. Instead of using ICG, a more straightforward approach, similar to the method described in \"Blending Custom Photos with Video Diffusion Transformers,\" could be to use the softmax results of the attention between the reference identity features and the target features?\n\n3.The foundational video generation models have evolved significantly in recent times. It would be valuable to explore this topic based on models like Wan2.1 or HunyuanVideo, or even a purely text-to-image model", "limitations": "It is recommended to supplement experimental comparisons on single-person videos to demonstrate the model's generalization performance, and what about more than two persons?", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength\nThis paper is a systematic solution for ensuring Identity Correspondence correctness in multi-character animation, a challenging problem when extending pose-driven animation to multi-character settings with position swaps. and the benchmark is provided to facilitate academic progress\n\nWeakness\nThe highly relevant papers listed below are not mentioned, and these methods should be analyzed or compared accordingly. The performance analysis of single-id video is also necessary in the paper.\na、Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control\nb、DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation\nc、Identity-preserving text-to-video generation by frequency decomposition\nd、Blending Custom Photos with Video Diffusion Transformers", "was_revised": true, "final_justification": "The authors have provided the required reference papers and comprehensive comparative experimental results. Furthermore, they have supplemented their work with experiments under single-person dancing scenarios, which further validate the effectiveness of their algorithm. The experimental results convincingly demonstrate the superiority of the proposed method. I also acknowledge that multi-person dance generation is an interesting direction for future research. Overall, my final recommendation is a borderline accept.", "rebuttal": "## **I. Concern about Missing Related Work**\n---\nThank you for your valuable feedback on missing related works. We divide the four related work you mentioned into two categories: (1) works on **Pose-Driven Character Animation**, which address the same task as ours, and (2) works on **ID-Preserving Text-to-Video (T2V)**, which tackle a different problem. We will cite and discuss these four excellent works in our final version.\n\n### *A. Works on Pose-Driven Character Animation*\n\nPose-driven character animation aims to generate a video by driving a source character with a reference image and a pose sequence. However, the core contribution of our method is **spatial controllability** — determining *which character appears at which location*.\n\n\n#### *a. \"Follow-Your-Pose v2\" (FYP-v2)*\n\nWe appreciate FYP-v2's valuable exploration in tackling multi-character animation task. We kindly invite you to check the reference and comparison of FYP-v2 in our Appendix E.1. \n\nFor a fair comparison, we also evaluate our method on the **official** FYP-v2 benchmark, which focuses on occlusion scenarios without involving position swapping. As shown in **Appendix E.1, Table 1** of our supplementary material, our method outperforms all baselines across all metrics. This indicates that our approach not only offers spatial controllability but also achieves state-of-the-art performance in occlusion scenarios. We also listed comparison results with FYP-v2 below.\n| Method | SSIM↑ | PSNR↑ | LPIPS↓ | FID↓ | FID-VID↓ | FVD↓ |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: |\n| FYP-V2 | 0.830 | 31.86 | 0.173 | 26.95 | 14.56 | 142.76 |\n| **Ours** | **0.879** | **32.49** | **0.151** | **26.01** | **12.68** | **127.36** |\n\nFurthermore, it is candidly acknowledged that  handling \"position swap\" remains a limitation of their method in **Appendix D** of FYP-v2 paper. Our work addresses this problem. As shown in the Figure 7, by swapping the ID in the input skeleton, our model correctly generates the characters in their new, swapped positions while maintaining their unique appearances.\n\n#### *b. \"DanceTogether\" (DanceTog)*\n\nAs a concurrent work, DanceTog was released on arXiv after our submission deadline and without publicly available code, so it was not included in our original discussion. We believe that our method and DanceTog represent two distinct yet equally valuable technical approaches. The core of DanceTog lies in its `MaskPoseAdapter`, which integrates mask and pose information at the input level to provide strong guidance. We make contributions at both the input and supervision levels: Identity-Embedded Guidance(IEG) is used to provide ID instruction and pose sequence, while Identity Matching Graph (IMG) supervises the model to generate according to the ID instructions from IEG.\n\nOur core advancement lies in **scalability and flexibility**. As noted in **Appendix A** of the DanceTog paper, extending their method to more than two characters leads to significant computational overhead. In contrast, our approach incurs no additional inference cost when handling more than two characters (see **Figure 4** in our Appendix E.5), highlighting our method's scalability. Furthermore, as shown in **Figure 7** of our main paper, our method also exhibit strong flexibility (swapping character positions by changing the skeleton).\n\n### *B. Regarding Works on ID-Preserving T2V*\n\nID-Preserving T2V models create scenes from text while embedding an ID, without precise control over position and pose. In contrast, pose-driven character animation requires maintaining visual elements from the entire reference image, with precise control over character position and pose. Despite the differences in task formulation, we recognize that discussing the underlying methodologies remains valuable.\n\n* `ConsisID`'s core idea about frequency decomposition to solve **single-ID preservation** problem is inspiring. However, it doesn't involve identity correspondence between multiple characters.\n* `Ingredients` (Blending Custom Photos with Video Diffusion Transformers): It handles multi-ID scenarios using a pixel-level classification loss through an ID Router, delivering impressive performance on multi-ID video customization. In contrast, our method constrains character-level identity correspondence by capturing model's native attention maps to enforce generating the correct character at the correct position. This is a very interesting and insightful comparison, and we will include more detailed discussion in the final version.\n\n## **II. Suggestion on Affinity Computation**\n---\nThank you for your inspiring suggestion. Given the differences between the two tasks, directly transferring the `Ingredients` method is difficult. `Ingredients` incorporates per-character CLIP embeddings with a text prompt for **relative** position control. In contrast, the pose-driven character animation task requires generating the correct character at the correct position and is guided by pose for precise motion and **absolute** position control. Before explaining why we adopted the IMG paradigm, we designed two variants of our method based on `Ingredients`. If there are any detailed setting you'd like us to implement, please further discuss with us. Here is our implementation details:\n\n* `CrossAttn-Based`: We segment the reference image to extract `m` distinct identity features using a CLIP encoder for each character. In the Cross Attention layer of the model (**Figure 3**), instead of interacting with the CLIP embedding of the whole image as usual, we compute the affinity between the features of the generated frame and the `m` CLIP embeddings for each character. After applying softmax, we obtain an `[HW, m]` affinity map, where `H` and `W` are the latent shape. Each entry indicates the probability of a latent patch belonging to one of the `m` reference characters, with the background masked out.\n\n* `Sim-Based`: This approach directly measures similarity between characters. We use masks to extract features for `m` reference characters and `n` generated characters from the model's intermediate feature. By computing their dot-products and applying softmax, we generate an `[n, m]` affinity matrix, where each entry `(i, j)` indicates the similarity between the `i`-th generated character and the `j`-th reference character. \n\nEach variant is optimized using cross-entropy loss for classification.\n\n|Method | SSIM ↑ | PSNR* ↑ | LPIPS ↓ | FID ↓ | FID-VID ↓ | FVD↓ |\n|:---|:---|:---|:---|:---|:---|:---|\n| CrossAttn-Based | 0.648 | 15.69 | 0.315 | 45.82 | 29.22 | 306.75 |\n| Sim-Based | 0.629 | 16.25 | 0.329 | 47.09 |24.35| 269.14 |\n| **Ours** | **0.654** | **16.93** | **0.304** | **40.19** | **23.58** | **225.06** |\n\nWe chose IMG+MQA for affinity computation as it directly utilizes the model's native attention scores, tightly coupled with the generation process. In contrast, `CrossAttn-Based` relies on additional features unrelated to the model’s behavior, while `Sim-Based` struggles to distinguish visually similar characters—issues our method inherently avoids. However, this is a very inspiring suggestion, and we are willing conduct further experiment in this direction.\n\n## **III. Concern about Different Number of Characters**\n---\nThank you for your critical questions regarding our model's generalization capabilities. Our method demonstrates superior generalization on both single-character and multi-character (N>2) video generation, outperforming strong baselines in both scenarios. Due to space constraints, we kindly invite you to check some details in our response to Reviewer `acUv`.\n\n### *A. Generalization to Single-Character Task*\n\nOur method not only maintains performance on single-character tasks but also improves upon the single-character backbone (AnimateAnyone's public version implemented by Moore Threads), demonstrating its generalization capability. As detailed in our response to Reviewer `acUv`, we conducted a comprehensive ablation study. Below are the results on the **public** single-character TikTok (Jafarian et al., 2021) benchmark.\n\n| Method | PSNR*↑ | LPIPS↓ | SSIM↑ | FID↓ | FVD↓ | FID-VID↓ |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| AnimateAnyone | **17.85** | 0.280 | 0.768 | 52.15 | 209.14 | 25.864 |\n| **Ours** | 17.78 | **0.279** | **0.772** | **40.56** | **163.85** | **20.294** |\n\n### *B. Generalization to More Characters*\n\nOur method effectively scales to multi-character scenes, as shown by the qualitative results in **Appendix Figure 4** of the supplementary materials. We also present quantitative results for 3-5 character scenarios (see Reviewer `acUv` for details), demonstrating that our method consistently outperforms baselines in these challenging cases.\n\n| Method | PSNR*↑ | LPIPS↓ | SSIM↑ | FID↓ | FVD↓ | FID-VID↓ |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| AnimateAnyone | 14.70 | 0.370 | 0.606 | 56.66 | 401.51 | 35.356 |\n| AnimateAnyone*| 14.92 | 0.363 | 0.607 | 64.34 | 406.08 | 36.308 |\n| UniAnimate | 15.62 | 0.338 | 0.640 | 57.75 | 348.91 | 29.030 |\n| **Ours** | **16.68** | **0.315** | **0.671** | **42.84** | **261.01** | **23.571** |\n\nWe will integrate these results into the final version.\n\n## **IV. Suggestion about Foundational Models**\n---\nThank you for your insightful suggestion. Leveraging foundational models is indeed a promising area for future research. Our chosen backbone, AnimateAnyone, is built on the text-to-image model Stable Diffusion 1.5, aligning with your perspective on using such models. Extending our method to the latest video generation models is a valuable direction for future exploration. In the future, we are open to transferring our method to more advanced foundational models."}, {"review_id": "467jmkMina", "reviewer": "Reviewer_Mm46", "summary": "This paper proposes EverybodyDance, which aims to address the issue of identity consistency in pose-conditioned multi-person dance videos. Specifically, EverybodyDance adopts the backbone of AnimateAnyone. Its input includes a reference image of multiple humans (typically two or three people), a pose map (with different subjects marked in different colors), and target poses, while the output is a video sequence.\n\nThe highlight of this work lies in a simple yet reasonable training strategy designed to enhance the network's identity awareness. Put simply, it extracts the deep features of all subjects from both the ground truth and the generated frames using masks, and applies a matching loss that encourages low similarity between different identities and high similarity within the same identity. This idea (at least seemingly) is straightforward and practical, and effectively improves identity alignment.\n\nThe authors also collect a new multi-person (mainly two-person) dance dataset to train and validate their method.\n\nAlthough the core idea of the paper is quite simple and straightforward, it shows significant improvements and could be highly beneficial to the field of multi-subject video generation.", "questions": "On the design of affinity computation, the authors take an equal and unique treatment to different kind of features into single \"attention map\" (Line 150). However, is there any possibility to treat different parts (e.g., hair, clothes, pants, decorations...) separately? Will such \"equal and unique\" way lose identity details?", "limitations": "There is a short limitation. But I didn't see \"potential negative social impact\" of this work. The authors do need to append it further. \n\nMeanwhile, this work includes a new dataset&benchmark. **Do the collected data involve copyright issues and privacy issues**? I am not sure about it.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n+ The core idea is very simple, reasonable, and easy to extend.\n\n+ The authors introduce a dual-person dance dataset (albeit a small one) and propose a benchmark based on it.\n\n+ Experiments show that the proposed identity matching method significantly improves identity consistency. The ablation study is also fairly comprehensive.\n\n+ The writing is clear and easy to understand.\n\n**Weaknesses**\n\n- In the authors’ demo, when occlusion occurs between subjects, the pose and generated video exhibit abrupt changes. Does this indicate that the authors intentionally avoided evaluating performance under occlusion? Does this imply that their method may not generalize well to scenarios involving strong interactions?\n\n - When discussing the paper’s limitations, the authors cite the lack of high-quality data. However, there exist many high-quality 3D human-human interaction datasets, such as You2Me (Ng et al., 2020), InterHuman (Liang et al., 2023), and DD100 (Siyao et al., 2024). The authors neither discuss whether training on such high-quality 2D/3D labeled data could improve performance, nor do they cite these works.", "was_revised": true, "final_justification": "The rebuttal generally resolve my questions. I keep the original score.\n\nHowever, the reviewers DO need to revise details I mentioned (including data occlusions, using/discussing 3D interaction data...).", "rebuttal": "## **I. Concern about Potential Loss of Identity Details**\n---\nThank you for your insightful question concerning the granularity of our affinity computation. To answer this question, we first provide an analysis based on the training process, and subsequently design an experiment to verify it.\n\n### *A. Principle Analysis*\nDuring training, the matching loss and the diffusion model’s L2 loss work together; therefore, the loss of identity details would be penalized by the L2 loss. As shown in Figure 1, our method accurately preserves the identity details such as hairstyles and shoes. For example, the woman in white and the man in orange. Your suggestion to treat different parts separately is very insightful and could be achieved with more advanced segmentation models, potentially improving the identity appearance details. Below, we design a simple experiment to analyze whether our affinity computation captures fine-grained visual feature correspondences.\n\n### *B. Experimental Verification*\nThe overall procedure is similar to the visualization experiment shown in Figure 6. The experimental setup is as follows:\n1.  First, we use a combination of a face segmentation model to extract a precise face mask for each character.\n2.  Next, we use only the features from the `generated` character's **face region** as a query to attend to the entire feature map of the corresponding `reference` character.\n3.  We then analyze the distribution of the resulting affinity scores. Specifically, we compare the **Area Ratio** (AR) (face mask area of the `reference` character / full character mask area of the `reference` character) to the **Affinity Ratio** (AF) (affinity score on the face region of the `reference` character / total affinity score for the `reference` character).\n\n| Metric | Mean | Variance |\n| :--- | :--- | :--- |\n| AF | 0.7874 | 0.0245 |\n| AR | 0.1038 | 0.0020 |\n\nOn average, for the reference character, the facial region accounts for only **10.38%** of the area but holds **78.74%** of the total affinity. The `generated` character’s face shows a high affinity with the `reference` character’s face, while its affinity with the remaining area is much lower. This demonstrates that most of the affinity is concentrated in this fine-grained facial region, indicating that the affinity computation retains a significant level of fine detail.\n\n## **II. Concern about Occlusion**\n---\nThank you for your critical question regarding occlusion handling. Our response consists of two parts: first, we will explain the reason of \"abrupt changes\", and second, we will show that our model still achieves a comprehensive SOTA performance on a public benchmark containing occlusion scenarios.\n\n### *A. Cause of the Phenomena*\n\nUnder severe occlusion, the accuracy of existing pose estimation algorithms can degrade dramatically, making it difficult to extract reliable driving signals. According to our automated data processing script, these unreliable skeletons are discarded. We thought these extreme cases would cause unpredictable behavior and interfere with the evaluation of the models' performance. The focus of our work is to evaluate model performance under reliable driving signals. To ensure fairness, this data processing pipeline was applied equally to all baselines. We recognize that handling such \"abrupt changes\" in driving signals caused by upstream tool is important for future research.\n\n### *B. Performance on Occlusion Benchmark*\n\nDespite the aforementioned data processing challenges, our model SOTA performance on a **public** benchmark with frequent occlusions.\n\nWe conducted a full quantitative evaluation on the **public multi-character benchmark from Follow-Your-Pose-V2 (FYP-V2)**, which contains frequent occlusions. As shown in **Appendix Table 1**, our method significantly outperforms all SOTA methods, including `Follow-Your-Pose-V2`.\n\nThe complete quantitative comparison is as follows:\n\n| Method (Method) | SSIM ↑ | PSNR↑ | LPIPS ↓ | L1 ↓ | FID ↓ | FID-VID ↓ | FVD ↓ |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| DisCo | 0.793 | 29.65 | 0.239 | 7.64E-05 | 77.61 | 104.57 | 1367.47 |\n| DisCo+ | 0.799 | 29.66 | 0.234 | 7.33E-05 | 73.21 | 92.26 | 1303.08 |\n| MagicAnime| 0.819 | 29.01 | 0.183 | 6.28E-05 | 40.02 | 19.42 | 223.82 |\n| MagicPose| 0.806 | 31.81 | 0.217 | 4.41E-05 | 31.06 | 30.95 | 312.65 |\n| Animate Anyone| 0.795 | 31.44 | 0.213 | 5.02E-05 | 33.04 | 22.98 | 272.98 |\n| Animate Anyone*| 0.796 | 31.10 | 0.208 | 4.87E-05 | 35.59 | 22.74 | 236.48 |\n| Follow-Your-Pose-V2| 0.830 | 31.86 | 0.173 | 4.01E-05 | 26.95 | 14.56 | 142.76 |\n| **Everybody Dance (Ours)** | **0.879** | **32.49** | **0.151** | **0.92E-05** | **26.01** | **12.68** | **127.36** |\n\n## **III. Concern about Related Dataset**\n---\nThank you for your valuable suggestion regarding these relevant datasets. We agree they are high-quality works and appreciate the opportunity to kindly explain what data is our task required:\n\n* **You2Me dataset:** You2Me contains high-quality and complex two-person interactions. However, the videos in this dataset are primarily filmed from a first-person perspective, meaning that typically only one complete person appears in the frame. The core of our work is to address identity confusion and preservation when multiple characters co-exist in the same frame. Therefore, the data paradigm of You2Me does not fully align with our \"multi-character co-presence\" task setting.\n\n* **InterHuman and DD100 datasets:** These datasets provide precise 3D human pose and shape parameters (e.g., SMPLx models). However, they primarily focus on the body's geometric structure and dynamics, lacking the detailed and diverse character appearances (e.g., clothing, hairstyles, faces) that are essential for our task. Our method aims to animate a character with a specific appearance from a reference image, which requires training data to contain both rich poses and their corresponding realistic appearance information. This was our original intention when we mentioned the \"scarcity of high-quality multi-character video datasets\" in our paper.\n\nNevertheless, your suggestion is very helpful. We recognize that incorporating 3D priors into our pipeline is a valuable future direction, especially for challenges like severe occlusion. We will include this discussion about the datasets mentioned in the final version. Thank you again for the constructive feedback.\n\n## **IV. Concern about Copyright and Privacy**\n---\nThank you for your important question regarding the privacy and copyright. We took this very seriously from the beginning. To construct and release our dataset and benchmark responsibly, we have strictly followed the principles below:\n\n### *A. Privacy-First Manual Curation:* \n\nWe did not engage in any automated or indiscriminate data scraping. Instead, all data was carefully and manually selected by our researchers according to strict criteria:\n*   **Source and Context:** We focused on videos created by professional performers (based on their subscriber count) on public platforms, filmed in public spaces like dance studios or stages. We purposely avoided content from private spaces, such as personal rooms. Besides, we didn't include any minors in our dataset.\n*   **Respect for Creators:** We strictly adhered to the creators' explicit wishes. Any video or creator account with statements like \"Do not download,\" \"No reposting,\" or \"Not for AI training\" was carefully excluded from our collection.\n*   **Data Minimization:** We followed the principle of data minimization at every stage. We only obtained information that was necessary to achieve our research objective. For instance, we extracted only the relevant video segments and discarded any unrelated content such as the creators’ profiles.\n\n### *B. Protecting Creators with a 'Notice and Takedown' Policy*\n\nThis policy is central to our ethical approach, aiming to protect and support anyone who appears in our video data.\n*   **No Raw Data Distribution:** We will never publicly release or redistribute original video files. For research purposes, we only provide links to publicly available sources.\n*   **Accessible Opt-Out Channel:** We provide a clear and straightforward way to contact us. Any creator appearing in a video can reach out at any time to request that we remove their data. We commit to immediately and permanently removing any related links from our dataset.\n\n### *C. Restrictive Licensing for Academic Research:*\nWe will release our curated dataset links, not the actual video data, under the Creative Commons CC BY-NC 4.0 license. This ensures the dataset is prohibited from being used for any commercial purposes by any individual or organization.\n\nWe believe the measures we’ve outlined reflect our respect and effort to protect privacy and copyright, and that we do our best to advocate such respect and protection in the research community."}]

Author rebuttal:

Meta-review: After the rebuttal, all reviewers are consitently positive. Thus, it is a clear acceptance. Since many experiments was conducted during rebuttal period, authors need to include them in the final version.
Decision: Accept (poster)