Title: PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancement
Venue: NeurIPS 2025
Accepted: yes
Keywords: video generation

video customization
Abstract: Despite recent advances in video generation, existing models still lack fine-grained controllability, especially for multi-subject customization with consistent identity and interaction.
In this paper, we propose PolyVivid, a multi-subject video customization framework that enables flexible and identity-consistent generation. To establish accurate correspondences between subject images and textual entities, we design a VLLM-based text-image fusion module that embeds visual identities into the textual space for precise grounding. To further enhance identity preservation and subject interaction, we propose a 3D-RoPE-based enhancement module that enables structured bidirectional fusion between text and image embeddings. Moreover, we develop an attention-inherited identity injection module to effectively inject fused identity features into the video generation process, mitigating identity drift. Finally, we construct an MLLM-based data pipeline that combines MLLM-based grounding, segmentation, and a clique-based subject consolidation strategy to produce high-quality multi-subject data, effectively enhancing subject distinction and reducing ambiguity in downstream video generation.
Extensive experiments demonstrate that PolyVivid achieves superior performance in identity fidelity, video realism, and subject alignment, outperforming existing open-source and commercial baselines.  More comprehensive video results and comparisons are shown on the project page in the supplementary material.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nText-to-video generation has attracted great attention recently. Despite the significant developments, existing text-to-video generation methods still lack fine-grained controllability, especially for multi-subject customization with consistent identity and interaction. To solve the problem, the paper proposes PolyVivid, a multi-subject video customization framework enabling controllable multi-subject video generation. Specifically, PolyVivid utilizes a VLLM-based text-image fusion module to encode subject images and uses a 3D-RoPE-based identity-interaction enhancement module to further enhance identity preservation and enable richer subject interaction. Besides, to efficiently inject text embeddings and image embeddings, PolyVivid proposes an attention-inherited identity injection module. Experiments have shown the effectiveness of the proposed methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n\n* Video generation involving fine-grained interactions between multiple subjects is a critically important and interesting problem in the field.\n\n* The proposed 3D-RoPE mechanism is novel.\n  \n\nWeaknesses:\n\n* The organization of the manuscript has led to an inadequately substantiated experimental component, lacking the depth and rigor required to fully evaluate the proposed approach.\n* The sharing MM Attention mechanism illustrated in Figure 3 lacks a detailed explanation in the main text, which significantly confuses the reader, making it difficult to understand the methodological intent when reading Section 4.3\n\nQUESTIONS:\nPlease see weaknesses. Besides, here are some other questions and suggestions below:\n\n* A comprehensive comparative analysis with the backbone model (HunyuanVideo) is required, encompassing both qualitative and quantitative evaluations to assess the relative improvements of the proposed method. On the one hand, whether the approach demonstrates improvements in established evaluation metrics, and on the other hand, whether it enables the generation of previously failed cases that the backbone model could not produce.\n* In Figure 6 of the Appendix, the case ``A person is sitting on a sofa, petting a cat.\" The generated video seems to have a blurry mosaic at the woman's elbow. What could this be due to?\n* The paper mentioned that 'existing works lack fine-grained controllability, especially for multi-subject customization with consistent identity and interaction'. I would like to know if the method is able to generate some complex counterfactual relations. For example, \"A horse rides on an astronaut.\"\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes PolyVivid, a novel inference-time multi-subject video customization framework composed of several components. It introduces a design that leverages VLLMs for semantic understanding of both reference images and texts, along with a novel 3D-RoPE-based identity interaction enhancement module, which effectively preserves the sequential relationship between text and images through bi-directional fusion. Additionally, the paper presents an identity injection attention module featuring a new adaptation method for handling image tokens, and a data construction pipeline that consider subjects appearing only briefly in the video. This pipeline demonstrates superior performance even compared to recent closed-source models, such as Pika and Kling.\n\nSTRENGTHS AND WEAKNESSES:\n**S1**. The paper is easy to read and follow, and the presentation is clear. The supplementary material is also well organized.\n\n**S2**. Clique-based subject consolidation is interesting. This simple design effectively improves the robustness of multi-subject personalization datasets.\n\n**S3**. 3D-RoPE-based identity-interaction enhancement module is a novel and straightforward design for text-image fusion within the attention module. This structure presents a promising direction for future research in personalization, or in broader in-context image or video generation.\n\n**S4**. The method also demonstrated superior performance compared to closed-source models such as Kling and Pika.\n\n**W1**. (minor) I think the model name Keling should be Kling. (Kling appears to be the official name)\n\n**W2**. Image and text fusion via VLLMs is not a novel idea. I think citing [1] and [2] would be helpful.\n\n[1] Song et al., MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation, ECCV 2024\n\n[2] Li et al., BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing, NeurIPS 2023\n\nQUESTIONS:\n**Q1**. While the qualitative samples suggest that identity preservation is well handled, some samples give a slight copy-and-paste impression, e.g., Figure 5 in the supplementary material. Was there any attempt to filter out overly similar segmentations in the dataset or to refine the data construction process for selecting frames?\n\n**Q2**.. How does the model perform on more outlier-like concepts (e.g., the monster toy in DreamBooth) or with highly similar subjects, such as two male individuals? Additionally, I’m curious whether the model can handle more complex compositions—beyond side-by-side placements—such as a scene featuring a can with the monster toy illustrated on it. (This isn’t meant as a negative comment, I understand that video models still have limitations in editability.)\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method for multi-subject personalized video generation, built on top of HunyuanVideo (a video diffusion model). To enhance identity preservation and interaction control, the authors introduce three key components:\n\n- A new multi-subject customization dataset, preprocessed using MLLM, SAM, CLIP, and a clique-based subject grouping strategy to support diverse and coherent interactions.\n\n- A 3D-RoPE-based identity interaction module, which encourages information exchange between embeddings from the VAE (capturing detailed identity) and a VLLM (LLaVA) responsible for interaction modeling.\n\n- An attention-inherited identity injection mechanism for more effective subject-specific conditioning throughout the video sequence.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is well written and easy to follow.\n\n- The generated videos show better performance in terms of visual quality, identity preservation, and prompt alignment, compared to baselines (including commercial models) —quantitatively and qualitatively.\n\n- The use of 3D RoPE embeddings for identity interaction enhancement is novel and effective, contributing significantly to performance.\n\n\nWeaknesses:\n- Since the model accepts a clique length longer than one-third of the total sequence, it may struggle to handle more than three subjects.\n\n- The model may fail to disambiguate identities when subjects are described using similar word, potentially leading to identity permutation.\n\nFor example, ``a man is standing next to a chair, and an another man is sitting on the chair.`` It will be difficult to assign each identity, as it use same natural language as description word.\n\n- Although the qualitative results are strong, the paper lacks a detailed evaluation of video quality. Metrics such as those from VBench (e.g., motion consistency) or human preference studies are not reported, so, it's difficult to determine whether fine-tuning introduces any trade-offs in visual quality.\n\nQUESTIONS:\n1. Can the model reliably disambiguate identities when subjects belong to the same category?\n\nFor example, in a prompt like ``a man on the left, and a man on the right``, does the model consistently assign the correct identity across frames? I reviewed Figure 3 of the appendix (the two-giraffe example), but it does not provide sufficient evidence to assess whether the model can reliably control identity in multi-subject scenarios where all subjects belong to the same category.\n\n2. What are the limitations of the model with respect to subject scale(size) and visibility?\n\n Can it simultaneously handle very small objects (e.g., a ring) alongside full-body subjects? Additionally, how does the model behave when a subject should be occluded or temporarily disappears from prompt? It's because I suspect the model tend to maintain the subjects visible in frames across video.\n\n3. Token Dimensions \n\nIn Equations (5) and (6), both the <image> token embedding $z_{T,I}$ and the identity token $z_I$ are said to have spatial dimensions of $w \\times h$. However, if my understanding is correct, $z_{T,I}$ comes from LLaVA and $z_I$ from the Hunyuan VAE, which may produce different token grid sizes. Are these embeddings resized or projected to enforce matching spatial shapes?\n\nLIMITATIONS:\nThe model may struggle to handle more than three reference images effectively.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a method for multi-reference condition to a text-to-video model. The method encodes both the video textual caption and the reference images into llava embedding space. The authors propose to modify the rope module in order to improve the interaction between the llava embeddings and the VAE embeddings of the image references. The authors propose an additional modification to the attention modules aiming to improve the contextualization of the image embedding with the video embedding sequence.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* Relevant works are discusses.\n* The results seem appealing. \n\nWeaknesses:\n* Overall, the pipeline seems overcomplicated. I think that most of the new design choices that the paper introduces are unnecessary for properly integrating multi reference condition into the video model.\n* The VLLM-based Text-image Fusion (Sec 4.1) seem redundant to me, and also very specific to Hunyuan which uses llava as opposed to T5, this also reduced efficiency. To my understanding, all the experiments in the ablation study feed in the images into llava, so this component is not motivated + may contaminate the other experiments in the ablation study. I am missing the simple baseline of text-only llava embeddings (the caption may refer to the reference images!) and concatenation of the image VAE embedding to the sequence, without any other modifications to the overall architecture.\n* It's unclear to me why a special rope is needed, what is the problem with simply treating the reference images as additional frames and use the existing rope module of the video model? \n* The authors claim that simply concatenating the image vae embedding to the vae embedding sequence has a limitation as distant frames will be affected less by the condition image. However, I am not convinced that this speculation is valid. It is also worth mentioning that the authors use Hunyuan which uses DiT with full sequence attention, so the reference image directly interacts with each frame in the video. Indeed, the ablation study shows that the identity preservation increases with simple token concatenation.\n\nQUESTIONS:\n* The Attention-inherited Identity Injection is not entirely clear. While Figure 3 shows the proposed modification, it can be helpful to also specify more clearly in the text what is the difference compared to the baseline architecture. Specifically, add to Equations 7-8 the entire flow of the attention as shown in the Figure.\n\nLIMITATIONS:\nThe authors don't specifically addressed potential negative societal impact of their work. However, I think the limitations of their method are not fundamentally different from limitations of similar papers accepted to neurips and similar conferences.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-04"}