Title: EverybodyDance: Bipartite Graph–Based Identity Correspondence for Multi-Character Animation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion model

Character animation
Abstract: Consistent pose‐driven character animation has achieved remarkable progress in single‐character scenarios. However, extending these advances to multi‐character settings is non‐trivial, especially when position swap is involved. Beyond mere scaling, the core challenge lies in enforcing correct Identity Correspondence (IC) between characters in reference and generated frames. To address this, we introduce EverybodyDance,  a systematic solution targeting IC correctness in multi-character animation. EverybodyDance is built around the **Identity Matching Graph (IMG)**, which models characters in the generated and reference frames as two node sets in a weighted complete bipartite graph. Edge weights, computed via our proposed Mask–Query Attention (MQA), quantify the affinity between each pair of characters. Our key insight is to formalize IC correctness as a graph structural metric and to optimize it during training. We also propose a series of targeted strategies tailored for multi-character animation, including identity-embedded guidance, a multi-scale matching strategy, and pre-classified sampling, which work synergistically. Finally, to evaluate IC performance, we curate the **Identity Correspondence Evaluation** benchmark, dedicated to multi‐character IC correctness. Extensive experiments demonstrate that EverybodyDance substantially outperforms state‐of‐the‐art baselines in both IC and visual fidelity.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes EverybodyDance, which aims to address the issue of identity consistency in pose-conditioned multi-person dance videos. Specifically, EverybodyDance adopts the backbone of AnimateAnyone. Its input includes a reference image of multiple humans (typically two or three people), a pose map (with different subjects marked in different colors), and target poses, while the output is a video sequence.\n\nThe highlight of this work lies in a simple yet reasonable training strategy designed to enhance the network's identity awareness. Put simply, it extracts the deep features of all subjects from both the ground truth and the generated frames using masks, and applies a matching loss that encourages low similarity between different identities and high similarity within the same identity. This idea (at least seemingly) is straightforward and practical, and effectively improves identity alignment.\n\nThe authors also collect a new multi-person (mainly two-person) dance dataset to train and validate their method.\n\nAlthough the core idea of the paper is quite simple and straightforward, it shows significant improvements and could be highly beneficial to the field of multi-subject video generation.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n+ The core idea is very simple, reasonable, and easy to extend.\n\n+ The authors introduce a dual-person dance dataset (albeit a small one) and propose a benchmark based on it.\n\n+ Experiments show that the proposed identity matching method significantly improves identity consistency. The ablation study is also fairly comprehensive.\n\n+ The writing is clear and easy to understand.\n\n**Weaknesses**\n\n- In the authors’ demo, when occlusion occurs between subjects, the pose and generated video exhibit abrupt changes. Does this indicate that the authors intentionally avoided evaluating performance under occlusion? Does this imply that their method may not generalize well to scenarios involving strong interactions?\n\n - When discussing the paper’s limitations, the authors cite the lack of high-quality data. However, there exist many high-quality 3D human-human interaction datasets, such as You2Me (Ng et al., 2020), InterHuman (Liang et al., 2023), and DD100 (Siyao et al., 2024). The authors neither discuss whether training on such high-quality 2D/3D labeled data could improve performance, nor do they cite these works.\n\nQUESTIONS:\nOn the design of affinity computation, the authors take an equal and unique treatment to different kind of features into single \"attention map\" (Line 150). However, is there any possibility to treat different parts (e.g., hair, clothes, pants, decorations...) separately? Will such \"equal and unique\" way lose identity details?\n\nLIMITATIONS:\nThere is a short limitation. But I didn't see \"potential negative social impact\" of this work. The authors do need to append it further. \n\nMeanwhile, this work includes a new dataset&benchmark. **Do the collected data involve copyright issues and privacy issues**? I am not sure about it.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a multi-identity feature association graph during the training phase to address the issue of confusion in multi-person video generation, and builds corresponding evalution benchmark. In detail, a mask query attention is proposed to optimize the identify graph. The experiments demonstrate the outperformances in identify matching and visua fidelity\n\nSTRENGTHS AND WEAKNESSES:\nStrength\nThis paper is a systematic solution for ensuring Identity Correspondence correctness in multi-character animation, a challenging problem when extending pose-driven animation to multi-character settings with position swaps. and the benchmark is provided to facilitate academic progress\n\nWeakness\nThe highly relevant papers listed below are not mentioned, and these methods should be analyzed or compared accordingly. The performance analysis of single-id video is also necessary in the paper.\na、Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control\nb、DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation\nc、Identity-preserving text-to-video generation by frequency decomposition\nd、Blending Custom Photos with Video Diffusion Transformers\n\nQUESTIONS:\n1. Lack of related work listed above.\n\n2. Instead of using ICG, a more straightforward approach, similar to the method described in \"Blending Custom Photos with Video Diffusion Transformers,\" could be to use the softmax results of the attention between the reference identity features and the target features?\n\n3.The foundational video generation models have evolved significantly in recent times. It would be valuable to explore this topic based on models like Wan2.1 or HunyuanVideo, or even a purely text-to-image model\n\nLIMITATIONS:\nIt is recommended to supplement experimental comparisons on single-person videos to demonstrate the model's generalization performance, and what about more than two persons?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method to address the identity preserving issue in multi-character animation video generation. It proposes a bipartite matching between reference character and generated character frame-wisely. By formulating the correspondence matching into a matching loss and applied during training, the method is able to better maintain identity in the generation and outperform existing approaches consistency under different settings. Experiments show improved generated video quality on the proposed ICE benchmark.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n**Important task**\n\nMaintain identity consist across videos is a important task. The paper does the weakness analysis of existing methods and show the importance of addressing such issue in multi character animation.\n\n\n**Good quantitative results compare to existing approaches**\n\nThe method outperform other methods under the same setting. I appreciate the author conducts a strict comparison and remove the influence of different datasets. \n\nWeakness \n\n**No temporal correspondence in the proposed IMG**\n\nThe bipartite graph is constructed per frame and there is no consistent person match across time, which loss the temporal consistency. I would like to know why not consider the temporal information in IMG and how current IMG could improve the temporal consistency by only considering the matching framewisely.\n\n**Affinity computation**\n\nI am not fully convinced the affinity could reflect the person identity information. As the feature contains the pose information, how to ensure the affinity score has remove the pose information for both reference and generated character. \n\n**Scaling person**\n\nI understand the current dataset doesn't contain too much person. But i am highly doubt the proposed method could work for larger number of person input. As most of results only showcase two person (which is quite simple in matching), I wonder how does the paper work for a group of person and does the training still applicable to such larger number, as the current training needs instance segmentation per iteration.\n\nQUESTIONS:\n1.  **Instance segmentation**\n\nIn order to segment the generated character in each generated frame, does that mean during each training iteration, the method has to run segmentation on the generated results? In the early state training, the generation could be poor and how to ensure it could segment the generated character correctly. Also, what is training overhead by applying such segmentation per iteration. \n\n2. **How to determine the number of person**\n\nI would like to know how to determine m and n, the number of characters for reference and generated. For different inputs with different number of person, how does the current method count number of character in the generation process and do the matching. \n\n3. **Occlusion**\n\nI am curious how does the IMG address occlusion issue in occluded frames, without consider temporal information, the per-frame setting will suffer from bad matching for occluded frames.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to extend the one-character pose-driven animation task to multi-characters and propose the EverybodyDance method to solve this problem.  EverybodyDance explicitly builds the character correlation between the reference image and target video clips instead of simply implicitly mapping like ReferenceNet. Specifically, EverybodyDance introduces the Identity Matching Graph (IMG) to connect the characters from different sources and uses character latent features to represent the node and Mask–Query Attention (MQA) to represent the match weights of each character pair.  Other techniques like identity-embeded guidance, a multi-scale matching method and pre-classified sampling are proposed to improve model robustness and IC accuracy. Finally, a new Multi-Character Animation  benchmark dataset is introduced to evaluate the effectiveness of EverybodyDance  and other baselines. EverybodyDance achieves superior performance than other baselines.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:\n  - This paper proposes an explicit identity mapping method to solve the task of multi-character animations. \n  The key techniques Identity Matching Graph (IMG) and Mask-Quey Attention(MQA) sound novel and reasonable.\n  - The quantity and quality results in a multi-character benchmark demonstrate the effectiveness of EverybodyDance. \n\n# Weaknesses:\n  - To my best of knowledge, EverybodyDance is not the first paper to solve the Multi-Character pose-driven Animation task. Previous work like [1] has proposed a method to solve this task. More discussions and comparisons are necessary to added to the main paper.\n  - From the visualization of figures in this paper, there are almost two characters in reference images and target video clips. Does the training and test dataset consist of samples with only two characters? Can EverybodyDance also be extended to reference images of more than 2 characters? More distribution descriptions about the multi-character dataset and target visualization would be better to show the effectiveness of EverybodyDance.\n  - EverybodyDance is finetuned with a custom multi-character dataset based on the AnimateAnyone and achives better performance than baselines in ICE benchmark. Because of the finetuning paradigm, we doubt that the FT process may introduce multi-character  bias and further destroy generalization for reference images with single characters. It is also necessary to make comparisons in test dataset involving single characters.\n\n\n[1] Xue J, Wang H, Tian Q, et al. Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control[J]. arXiv preprint arXiv:2406.03035, 2024.\n\nQUESTIONS:\nPlease mainly address the questions in the weakness section.\n\nLIMITATIONS:\nThe paper slightly mentioned the limitations in the end of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}