Title: MEGADance: Mixture-of-Experts Architecture for Genre-Aware 3D Dance Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: AI for Art

Creative AI

Multimedia Learning

3D Motion Generation

Music-to-Dance Generation
Abstract: Music-driven 3D dance generation has attracted increasing attention in recent years, with promising applications in choreography, virtual reality, and creative content creation. Previous research has generated promising realistic dance movement from audio signals. However, traditional methods underutilize genre conditioning, often treating it as auxiliary modifiers rather than core semantic drivers. This oversight compromises music-motion synchronization and disrupts dance genre continuity, particularly during complex rhythmic transitions, thereby leading to visually unsatisfactory effects. To address the challenge, we propose MEGADance, a novel architecture for music-driven 3D dance generation. By decoupling choreographic consistency into dance generality and genre specificity, MEGADance demonstrates significant dance quality and strong genre controllability. It consists of two stages: (1) High-Fidelity Dance Quantization Stage (HFDQ), which encodes dance motions into a latent representation by Finite Scalar Quantization (FSQ) and reconstructs them with kinematic-dynamic constraints, and (2) Genre-Aware Dance Generation Stage (GADG), which maps music into the latent representation by synergistic utilization of Mixture-of-Experts (MoE) mechanism with Mamba-Transformer hybrid backbone. Extensive experiments on the FineDance and AIST++ dataset demonstrate the state-of-the-art performance of MEGADance both qualitatively and quantitatively. Code is available at https://github.com/XulongT/MEGADance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors introduce MEGADance, a novel two-stage architecture for music-driven 3D dance generation that emphasizes genre-aware choreography. This is achieved by decoupling dance generality and genre specificity across two stages. In the first stage, MEGADance uses Finite Scalar Quantization to learn a diverse motion codebook. Combined with kinematic and dynamic constraints, FSQ improves reconstruction fidelity and motion plausibility. In the second stage, the MEGADance uses a Mixture-of-Experts (MoE) model, where Universal Experts model genre-invariant motion patterns and Specialized Experts model genre-specific patterns, via hard routing genre labels. Each expert is implemented as an autoregressive Mamba-Transformer, enabling both local and global temporal modeling. In part because of its strong inductive biases and genre sensitivity, MEGADance outperforms prior methods across quantitative metrics as well as human evaluations.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality**\n- S: Overall this is a high-quality paper. It is well-written, proposes novel and strong methodology, and performs extensive experiments with convincing quantitative and qualitative results.\t\n- W: See question 1 about complexity analysis; this is important information that appears to be totally left out. Also, the experiments don't show any error bars, although this appears to be the case for other papers in the field. Could the authors expand on this?\n- W: I would have appreciated if the authors could share the code for the purposes of evaluating how reproducible it seems. I understand anonymity concerns prevent a GitHub link, but there are other ways of sharing code anonymously, such as using a code anonymizing website like https://anonymous.4open.science/.\n\n**Significance / Originality**\n- S: The model carries real-world relevance from a creative standpoint, as it enables genre-aware music-generated choreography, albeit only for genres cleanly labelled in training data.\n- S: MoE + FSQ + Mamba-Transformer is novel in this domain.\n- W: It remains unclear how well MEGADance generalizes or adapts to new genres (i.e., how simple would this be? how expensive would this be?). This imposes limits on the architecture and its use cases in any applied scenario. See Q2 / Q3 on this point as well.\n\n**Clarity**\n- S: The text is well written overall and the math notation is consistent.\n- W: This is minor, but I would appreciate better figure captions. For example, the master architecture figure (Fig. 2) should have a clear caption that walks the reader through Stage 1, then Stage 2, and breaks down the components of the figure. It is only clear to me where the blue and pink boxes fit in with respect to the left box after I have studied the methodology, whereas it should be pretty obvious to someone just skimming the paper.\n\nQUESTIONS:\n1. Could authors provide any information on model parameter complexity? The stated generation runtime of 0.19 second is helpful but it would be better to understand the computational cost of the model as a whole, especially as a comparison to previous models. More broadly, quoting the NeurIPS Paper Checklist (line 607), authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.\n\n2. How well does MEGADance scale when new genres are added?\nSince the Specialized Experts are tied to specific genres, it's unclear how the model handles genre extension without full retraining. Could experts be modularly added or fine-tuned?\n\n3. Can MEGADance generate dances that blend multiple genres?\nIn its current form, hard routing limits the model to one genre at a time. Authors mention in App4 that free form text would be an interesting next step to allow for more creative generation... but I am curious to hear if there could be an even simpler next step. For example, would it be possible to mix Specialized Experts' latents? This reminds me of the field of style transfer, that studies this exact sort of exploration of the latent space, ex. Yin et. al https://arxiv.org/abs/2208.09406.\n\nLIMITATIONS:\nA limitations section is included in App4, although I would like to see it padded with information that I have asked about above, such as computational cost, scaling with new data, and more flexible generation modes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a two-stage music-to-dance generation framework that integrates a Mixture-of-Experts (MoE) architecture, Finite Scalar Quantization (FSQ) with kinematic-dynamic dual constraints, and a Mamba-Transformer hybrid backbone. The model is designed to generate high-fidelity and genre-controllable dance motions.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper introduces a two-stage music-to-dance generation model that combines MoE, FSQ, and a Mamba-Transformer backbone to improve motion fidelity and genre controllability.\n\nStrengths\n\n(1)\tBy using a Mixture-of-Experts (MoE) to separate genre-specific and shared motion patterns, and combining it with Finite Scalar Quantization (FSQ) and a Mamba-Transformer hybrid backbone, the model achieves better choreographic consistency and motion fidelity. This combination is both original and novel, offering an effective solution for controllable and high-fidelity dance generation.\n\n(2)\tThe paper presents solid experimental comparisons and ablation studies, supported by comprehensive visualizations to validate the effectiveness of each component.\n\nWeaknesses:\n\n(1)\tWhile the authors claim to compare against multiple methods, Table 1 only includes two recent baselines: FineDance (2023) and LODGE (2024), with no broader comparison to other relevant or competitive approaches. Moreover, it is unclear why Table 2 omits standard metrics FID_s and DIV_k for the AIST++ evaluation.\n\n(2)\tAlthough the paper claims that both the MoE design and the Mamba-Transformer backbone improve computational efficiency, only the inference time of the proposed model is reported. It would strengthen the argument to provide FLOPs or latency comparisons with other models, or at least include ablation results showing the impact of these modules on efficiency.\n\n(3)\tThe loss formulation involving velocity and acceleration (Eq. 3) lacks clarity. Notations such as S’ and S’’ are introduced without explanation. It would be helpful to rewrite the equation in a more explicit form.\n\nQUESTIONS:\n1.\tMany prior works have pointed out that autoregressive frameworks often suffer from error accumulation and motion freezing [8], how did the authors address this issue?\n\n2.\tPlease refer to the weaknesses section.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a system for music-conditioned and genre-aware dance motion generation. It contains some key innovative components. Finite scalar quantization (FSQ) is used to encode motions into latent space representations. Mixtures of genre-specialized and universal experts are then used to generate genre-aware dance motion sequences. To capture both long-range dependencies and finer local variations in motion, these experts are implemented using combinations of Mamba and Transformers. Through a number of experiments, the proposed system was evaluated with key characteristics highlighted.\n\nSTRENGTHS AND WEAKNESSES:\nThe proposed system is shown to be very competitive compared to the state-of-the-art when tested on the two most common datasets. The best results are shown in the genre controllability evaluation, which is indeed what the paper aims to improve. A rather extensive ablation study further highlights the importance of the different components of the system.\n\nThe paper is very well written and structured, which includes illustrations. Even if it contains a lot of content, it is still rather easy to read and understand. The proposed architectural components are both easy to understand and make perfect sense. One of the more intriguing components of the proposed system is FSQ for quantization. FSQ is more lightweight than VQ-VAE and doesn't require a codebook to be trained, a codebook that may be poorly utilized. In the ablation study provided in the paper, FSQ is shown to perform better than VQ-VAE, despite its simplicity.\n\nAnother characteristic component that makes sense, and is supported by positive results, is the distinction between specialized and universal experts. Even if there are many different genres in dance, there are few enough to motivate a specialized expert for each, rather than having a single generic generator and the genre as an argument that conditions the output.\n\nThe combination of Mamba and Transformers is not novel, but appears to be suitable for the task. Transformers are known to be good at handling long-range dependencies and interaction between music and motion, while Mamba may be better at capturing local variations in motion. The benefit of the combination is clear from the reported results.\n\nIn the comparison between different methods, the number of tested methods is quite low. There are more methods in the literature than those shown in the paper, especially if you look back in time. Those included, however, are representative of the current state-of-the-art. The numbers seem to have come from the original papers, except for Bailando++, where the numbers are worse than what has been reported earlier. There ought to be a good reason for this to be clarified.\n\nMinor things: The captions of Fig. 4a and Fig. 4b seem to be switched. Sometimes there are spaces before citations, and sometimes not. It would be better to make it more uniform. There is a missing bold value for FACT in Table 2. In the same table, it ought to be $DIV_k$ and $DIV_g$, not $DIV_g$ and $DIV_s$.\n\nQUESTIONS:\nThe paper includes some rather specific qualities of what the specialized and universal experts are expected to capture, but how can you know that these are true? The only difference seems to be a switch based on the genre when selecting the specialized expert.\n\nWhere do the numbers for Bailando++ come from, given that they seem to be different from the original paper or what has been published in other papers? Is it related to the fact that this method is not genre-aware?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper proposes an MOE-based architecture MEGADance for 3D music-to-dance generation by decoupling it into the dance content part and the genre style part. MEGADance consists of two stages - the High-Fidelity Dance Quantization Stage (HFDQ) and the Genre-Aware Dance Generation Stage (GADG). HFDQ learns the motion representation as discrete tokens through 3D motion reconstruction. Then the GADG generates motion tokens based on music features, genre features, and previously generated motion tokens in a generative way. To enhance the autoregressive generation process, MoE and Mamba-Transformer are applied. Experiments on two benchmark datasets show MEGADance outperforms current baselines, especially in genre controllability.

STRENGTHS AND WEAKNESSES:
Strength:
- Experiments show that the proposed framework outperforms baselines on two datasets (FineDance and AIST++).
- The MoE design to decouple genre and dance content sounds plausible.

Weakness:
- It's a little bit difficult to judge the novelty of the proposed framework. MEGADance is based on Transformer autoregressive decoder backbone with the generation target being discrete motion tokens, which is a relatively common practice for vision generation task using autoregressive Transformer. Although MEGADance proposes to include MoE and Mamba-Transformer for the autoregressive backbone, the core idea still remains the same and the proposed module/strategy themselves are not novel either. Motivation and analysis of the necessity of the proposed strategies are needed to help better understand the contribution.
- The complexity analysis in time and memory of the proposed framework is missing.

QUESTIONS:
Please see weakness.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03