Title: WaveAR: Wavelet-Aware Continuous Autoregressive Diffusion for Accurate Human Motion Prediction
Venue: NeurIPS 2025
Accepted: yes
Keywords: Human motion prediction
Abstract: This work tackles a challenging problem: stochastic human motion prediction (SHMP), which aims to forecast diverse and physically plausible future pose sequences based on a short history of observed motion. While autoregressive sequence models have excelled in related generation tasks, their reliance on vector‐quantized tokenization limits motion fidelity and training stability. To overcome these drawbacks, we introduce \textbf{WaveAR}, a novel AR based framework which is the first successful application of a continuous autoregressive generation paradigm to HMP to our best knowledge. WaveAR consists of two stages. In the first stage, a lightweight Spatio‐Temporal VAE (ST-VAE) compresses the raw 3D-joint sequence into a downsampled latent token stream, providing a compact yet expressive foundation. In the second stage, we apply masked autoregressive prediction directly in this continuous latent space, conditioning on both unmasked latents and multi‐scale spectral cues extracted via a 2D discrete wavelet transform. A fusion module consisting of alternating cross-attention and self-attention layers adaptively fuses temporal context with low- and high-frequency wavelet subbands, and a small MLP‐based diffusion head predicts per-token noise residuals under a denoising loss. By avoiding vector quantization and integrating localized frequency information, WaveAR preserves fine‐grained motion details while maintaining fast inference speed. Extensive experiments on standard benchmarks demonstrate that our approach delivers more accurate and computationally efficient predictions than prior state‐of-the-art methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new approach for stochastic human motion prediction. The proposed approach presents multiple components:\n- First, the motion is encoded as a sequence of continuous tokens using a spatio-temporal VAE (ST-VAE). In comparison  with the recent autoregressive approaches that rely on vector-quantized representations, this continuous latent representation allows for preserving more details.\n- Second, predictions are made in the latent space in an autoregressive fashion, conditioned on wavelet features extracted from the motion. This step combines a masked generative approach with a diffusion process. The wavelet transforms preserve more fine-grained features than the commonly used DCT, while the designed generation process allows for realistic, diverse predictions.\n\nExperiments on two widely used benchmarks demonstrate good performance compared to state-of-the-art approaches.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\nI feel like this paper has many strengths:\n- The approach is very well motivated: the introduction shows the current limitations of prior works and introduces the contributions that will address them.\n- The approach is novel: to my knowledge, this is the first that uses continuous autoregressive generation for human motion prediction.\n- The method is clear and well-explained. Despite the many components of the approach (tokenization, wavelet features, autoregressive generation with masking, diffusion process), the paper still reads nicely and is very understandable.\n- The masking allows the use of the same model for motion prediction and motion in-betweening.\n- The experiments follow the common practice, and the method obtains good results compared to the state of the art (better accuracy and very decent diversity).\n- Ablations are well-designed to demonstrate the interest of the technical contributions.\n- Visualisations given in the supplementary materials show good qualitative performance, with accurate yet diverse outputs.\n\n# Weaknesses\n\nI do not find major weaknesses in this paper. However, here are some points that I think could still be improved:\n\n- There is no discussion at all about the limitations of the method and future lines of research.\n- Additional ablation studies could be performed. In particular, it would be interesting to see how the model would perform with DCT features instead of wavelets.\n- The method section could be even clearer with improved notations. For instance, the symbol sigma is used for different purposes in Eqs. 1 and 10 (even if its meaning is clearly explained in both equations). L187, wouldn't it be clearer to use $L'$ instead of $T'$ as the sequence of $T$ poses is encoded to $L$ tokens before masking?\n- I think there is a small mistake L227-228: The ST-VAE is supposed to reduce the temporal dimension, but here, the length of the input and the tokenized representation is the same (H).\n- L145: The reference to the figure should be given (not fig. X).\n- From my understanding, at training time, the wavelet features contain information about the entire sequence, including the tokens that are masked. Wouldn't it be better to have something more similar to the inference settings at training time, with wavelet features computed only on the past, and masking done solely on the future?\n\n# Overall recommendation\n\nThis paper presents many strengths: the approach is well motivated and explained, novel, and obtains good results. Additionally, the ablations give insights into the most important parts of the model, and qualitative results are as impressive as quantitative ones. While I don't find major weaknesses in this paper, I feel that discussions on the current limitations and future works, as well as additional ablations, could improve it. I recommend \"Accept\".\n\nQUESTIONS:\nIn the rebuttal, I would like the authors to discuss the following points:\n- What are the limitations of the proposed approach, and what could be future improvements over WaveAR?\n- Ablations on the DCT vs. Wavelets\n- Discussion on the difference between training and inference settings (past always fully visible at inference, wavelet features computed only on the past at inference).\n\nLIMITATIONS:\nLimitations and potential negative impacts are not mentioned at all (the checklist says it is in the supplementary materials, but I could not find it).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents WaveAR, a wavelet-aware autoregressive model for human motion prediction. The approach integrates frequency-domain features extracted via a discrete wavelet transform into masked latent representations using an attention-based fusion mechanism. A subsequent MLP module serves as a noise predictor within a diffusion process to estimate future motion sequences. The model is evaluated on the Human3.6M and HumanEva-I datasets and is compared against state-of-the-art methods using standard diversity and accuracy metrics.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. The paper includes thorough experiments including various ablations that analyze the contribution of each component offering insights into the model’s design choices.\n\n2. The integration of wavelet transforms allows the model to capture motion information at multiple temporal scales, which can help in representing fine-grained motion patterns.\n\n\n\nWeaknesses:\n\n1. While the authors evaluate on standard benchmarks such as Human3.6M and HumanEva-I, recent works (e.g., BeLFusion, HumanMAC, and CoMusion) have demonstrated results on AMASS—a larger and more diverse dataset that better captures a wide range of human motions. The absence of evaluation on AMASS limits the assessment of the model’s robustness and weakens the strength of its generalizability claims.\n\n2. While WaveAR emphasizes accurate motion prediction and achieves the best ADE and FDE scores, the improvement over previous methods is marginal and comes with a large drop in diversity, as indicated by significantly lower APD scores (e.g., compared to HumanMAC on HumanEva-I and CoMusion on Human3.6M). In the context of stochastic human motion prediction, diversity is critical to reflecting the multimodality of plausible future movements. For instance, in autonomous driving scenarios, a pedestrian turning right might vary subtly in trajectory or speed, and capturing these variations within each motion mode is crucial. This raises concerns about the model's ability to capture diverse and plausible future motions within each mode, which is essential for robust stochastic prediction. \n\n3. Although the authors mention that WaveAR is computationally efficient and fast, they do not provide discussion of computational cost, inference speed, or model size relative to other baselines.\n\nQUESTIONS:\n1. Could the authors provide a demonstration of the model’s robustness and generalizability on a recent, large-scale, and diverse dataset—such as AMASS, which has been adopted by several recent baselines?\n\n2. The authors are encouraged to further analyze the trade-off between accuracy and diversity, and to discuss whether their model still preserves meaningful diversity within each mode, rather than collapsing to a limited set of dominant motions. \n\n3. Could the authors provide more details or comparisons regarding the computational cost, inference speed, or model size of WaveAR relative to other baselines?\n\n4. Minor typos: L.145\n\nLIMITATIONS:\nThe paper lacks a discussion of limitations. An analysis of issues such as the trade-off between accuracy and diversity, as well as potential failure cases, would contribute to a more balanced and transparent presentation of the work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents WaveAR, a method for stochastic human motion prediction by autoregressively making predictions in a continuous latent space. By doing so, the paper claims to be avoiding potential performance degradations due to discretized tokens. Furthermore, this neural motion representation is further extended by Discrete Wavelet Transform (DWT) features that are intended to capture multi-scale low- and high-frequency motion details. The model first uses a Spatio-Temporal VAE to compress raw 3D joint sequences into a continuous latent token stream, eliminating quantization errors. Next, a masked autoregressive diffusion process predicts future tokens, by leveraging DWT features. The proposed approach is evaluated on Human3.6M and HumanEva-I benchmarks, demonstrating a better or comparable performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1- Model Flexibility and Versatility: The model demonstrates significant flexibility. Beyond its primary function of forecasting, it can be effectively adapted for other tasks, such as motion in-betweening, by leveraging its flexible masking strategy.\n\n2- Competitive Performance: The model performs better than or on-par with the baselines on standard benchmarks. \n\n3- Integration of Techniques: A key strength lies in the novel adaptation of the MAR framework by guiding the autoregressive process with multi-scale features from a Discrete Wavelet Transform (DWT).\n\n**Weaknesses:**\n\n1- Limited Scope of Evaluation Benchmarks: The experimental evaluation is confined to the Human3.6M and HumanEva-I datasets. While these are standard benchmarks, the absence of tests on larger and more diverse datasets (e.g., CMU, AMASS) limits the assessment of the model's scalability and generalizability to more complex, in-the-wild motion.\n\n2- Omission of a Key Baseline Comparison: The paper acknowledges that its use of DWT features is inspired by MotionWavelet, yet MotionWavelet is not included as a baseline in the quantitative comparisons. Given that MotionWavelet also achieves strong performance, it is difficult to determine the precise advantage of WaveAR's more complex architecture (which includes the ST-VAE, fusion module, and diffusion head) over a potentially simpler, wavelet-focused approach. This omission makes the contribution of the additional components less clear.\n\n3- Potentially Overstated Novelty Claim: The central claim that WaveAR is \"the first successful application of a continuous autoregressive generation paradigm to [the] SHMP domain\" may be overstated. Earlier autoregressive models in human motion synthesis operated in continuous latent spaces before diffusion models became popular. For instance, HuMoR (Rempe et al., 2021) also predicts motion autoregressively in a continuous latent space. The paper's novelty claim should be refined to more accurately position the work within the existing literature, perhaps by emphasizing its combination of continuous autoregression with a wavelet-guided diffusion process.\n\nQUESTIONS:\nSee my weaknesses section for the main questions. I have one extra question:\n\n1- (in line 3, abstract) “While autoregressive sequence models have excelled in related generation tasks, their reliance on vector-quantized tokenization limits motion fidelity and training stability.” Is there any evidence for this statement? The proposed ST-VAE can be similarly limited due to the inherent properties of the VAE formulation (e.g., reconstructing high-frequency details in the signal while ignoring the low-frequency ones).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes WaveAR, which is a novel autoregressive (AR) framework for stochastic human motion prediction (SHMP). It applies the continuous autoregressive generation paradigm to the field of human motion prediction. It aims to solve the problem of detail loss and training instability caused by Vector Quantization (VQ) in the existing AR model when dealing with continuous input, and captures fine motion details by introducing wavelet transform.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n- The experimental results are truly impressive. WaveAR consistently achieves the most accurate predictions (lowest ADE and FDE) on standard HMP benchmarks like Human3.6M and HumanEva-I, significantly outperforming existing state-of-the-art methods. \n- WaveAR's flexible masking strategy extends its utility beyond just motion prediction; it excels in motion in-betweening tasks as well, showcasing its strong generalization capabilities across different application scenarios.\n\n**weakness**\n\n- In the introduction section, the author claims that WavAR is one of the \"first successful application\" as a contribution point, but the first one should not be an academic contribution, Instead, it should highlight the advantages of continuous autoregressive generation paradigm in this task. I think the author should re-summarize this innovation point.\n\n- The paper mentions that WaveAR maintains \"fast inference speed\" and \"computational efficiency\". However, beyond comparative results with existing methods in Table 1, specific inference time data are not explicitly provided. A detailed discussion on the model's performance and potential bottlenecks in practical real-time applications such as robotics control or VR would greatly enhance the paper's completeness.\n\nQUESTIONS:\n-  In the ablation study \"w/o DWT\" results indicate the effectiveness of the DWT module. Could the authors provide a more in-depth analysis, perhaps through visualizations or other quantitative metrics, to demonstrate how DWT specifically captures and leverages high-frequency details to improve motion prediction accuracy.\n\n-  The paper emphasizes WaveAR's computational efficiency, but specific inference time data (e.g., milliseconds per frame) or model size (number of parameters) compared to SOTA baselines are not provided.\n\nLIMITATIONS:\nThe authors acknowledge some limitations in their supplementary material , specifically mentioning the trade-off between accuracy and diversity, which results in relatively lower diversity scores. Building on these points. \nAdditional limitations can be seen in Questions and Weakness sections.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}