Title: CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow
Venue: NeurIPS 2025
Accepted: no
Keywords: Human motion prediction

probability density

fast inference
Abstract: Many density estimation techniques for 3D human motion prediction require a significant amount of inference time, often exceeding the duration of the predicted time horizon. 
To address the need for faster density estimation for 3D human motion prediction, we introduce a novel flow-based method for human motion prediction called CacheFlow.
Unlike previous conditional generative models that suffer from time efficiency, CacheFlow takes advantage of an unconditional flow-based generative model that transforms a Gaussian mixture into the density of future motions.
The results of the computation of the flow-based generative model can be precomputed and cached.
Then, for conditional prediction, we seek a mapping from historical trajectories to samples in the Gaussian mixture.
This mapping can be done by a much more lightweight model, thus saving significant computation overhead compared to a typical conditional flow model.
In such a two-stage fashion and by caching results from the slow flow model computation, we build our CacheFlow without loss of prediction accuracy and model expressiveness. 
This inference process is completed in approximately one millisecond, making it 4$\times$ faster than previous VAE methods and 30$\times$ faster than previous diffusion-based methods on standard benchmarks such as Human3.6M and AMASS datasets.
Furthermore, our method demonstrates improved density estimation accuracy and comparable prediction accuracy to a SOTA method on Human3.6M.
Our code and models will be publicly available.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a generative model for human motion prediction based on past motion frames. Existing methods have two significant limitations that inhibit their applications in real-time applications:\n- inability to estimate the probability distribution of the motions,\n- significant computational overhead that requires a high-end GPU to perform inference.\n\nThe first issue has been solved by incorporating a flow-based model that is capable of modeling the direct probability distribution of the data. However, such an approach is insufficient to tackle the second concern. To maintain the pipeline in the probabilistic models regime, the authors precompute triplets (latent motion representation, density change between the motion and its latent representation, the motion itself). Then, the authors train a light-weight RNN that takes an input motion and predicts Gaussian Mixture Model parameters. From there, one can sample a new future motion by sampling from GMM directly. As the GMM is a continuous model, the model retrieves the prediction by finding the closest sample from the triplets.\n\nTo further optimize the model, the authors suggest using the factorization of the latent space that reduces the latent dimensionality\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The authors leverage the properties of a CNF-based model and GMMs in a neat way that enables them keeping the data probability distribution tractable. Therefore, one can use likelihood directly to sample novel motions in an interpretable way.\n- At the same time, the method description is easy to follow and easy to understand where the benefits stem from, including its inference efficiency.\n- The proposed approach is general and can be applied to other generative models where efficiency is of high importance.\n\n**Weaknesses**\n- The main contribution of the paper—its efficiency—brings little value given the conducted experiments.  For example, GSPS and DivSamp, the authors compare to, achieve better results in terms of the diversity of quality, while maintaining the real-time inference. On the other hand, CacheFlow requires storing the whole dataset in memory and loading it to perform inference. Given that the authors motivate their method to be used in real-world applications, storing AMASS on embedded systems may be beyond capabilities in actual robotics. \n- One could argue that a simpler baseline that involves, for example, Multidimensional Scaling (for embedding the motions) and running GMM on those embeddings could achieve similar results while providing even higher efficiency and maintaining the ability to compute samples' likelihood.   \n- Given above, the paper calls for more principled evaluation of the efficiency that would answer the following questions:\n    - How much memory storing the embedded representation of triplets requires,\n    - How applicable is the solution to CPU-based systems (robotics) where having energy-hungry GPUs is impossible,\n    - What other benefits the likelihood estimation brings? If sampled several samples in the latent space, are they similar to each other. The APD achieved with CacheFlow suggests the model is unable to generate diverse samples.\n\nQUESTIONS:\n- The paper calls for reframing the narrative—instead of comparing the approach to models which try to maximize the diversity and quality, the authors should focus on the benefits of the likelihood estimation and the inference efficiency, especially in the constrained computation regimes.\n- Although not as actionable, I suggest comparing quality of different encoding/decoding models and their effect on the inference speed since little was mentioned in the paper about the actual models used during experiments.\n- The paper should present also more qualitative results.\n\nLIMITATIONS:\n- The authors properly mention the limitation that CacheFlow requires the precomputation of triples for efficient inference.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focuses on improving computational efficiency of stochastic motion prediction. Based on normalizing flow, this paper proposes CacheFlow that partitions conditional motion generation procedure into independent lightweight conditional prediction and heavy unconditional prediction. The connection between conditional prediction and unconditional prediction is built by an introduced latent variable that is Gaussian mixture of training points. The heavy unconditional predictions of motions can be precomputed and cached. During inference, CacheFlow maps input motion sequence into a latent variable z, fetches its nearest training point, and use cached unconditional information to predict future motions.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. The idea of separating conditional generation into lightweight conditional generation and unconditional generation to improve sampling efficiency is interesting.\n2. This paper is well written and organized, with clear description for their motivation and proposed method.\n3. Experiments show that the proposed method significantly improves sampling efficiency for motion prediction, while preserving accuracy.\n\n**Weaknesses**:\n1. Generalization capability. Considering that the intermediate variable $z$ is a Gaussian mixture of motions in the training set and the sampling approach in Eq. (13) uses nearest neighbour, the generation outcome of the proposed method will always be a random sample of training motions. This means that the proposed method cannot generalize to new motions, which may not be acceptable for any generative methods.\n2. Large performance gap on diversity metric APD. As in Table 1, GSPS and DivSamp presents significantly better diversity in their prediction than the proposed method, while with a relatively low inference latency. The proposed method seem not good at producing diverse motions. Is this a negative effect of nearest neighbour reduction?\n3. Experiment setting. The result that using training motions for prediction achieves good performance indicates a large overlap between training set and test set, making the experimental results less convincing.\n4. Typos. For example, \"four-fold\" in line 58, \"ecoder\" in line 125.\n\nQUESTIONS:\nPlease see the weaknesses.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes CacheFlow, a framework designed for fast 3D human motion prediction. The key idea is to decouple the expensive density estimation stage of flow-based models by precomputing a large set of latent-motion-density triplets using an unconditional flow-based model. At inference time, a lightweight conditional GMM is used to match the context to cached samples, allowing for fast retrieval and sampling. This approach significantly reduces inference time while maintaining competitive accuracy with state-of-the-art methods on Human3.6M and AMASS datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper clearly articulates the computational bottleneck in current flow-based and diffusion models and positions CacheFlow as a practical solution for real-time applications. \n\n2. By leveraging an unconditional model and precomputing triplets, the method sidesteps expensive computation during inference.\n\n3. The experiments provide quantitative comparisons with prior work and include an ablation study that analyzes the contribution of individual components of the proposed method.\n\nWeaknesses:\n\n1. The use of nearest-neighbor retrieval from a fixed precomputed set raises concerns about its potential impact on both the diversity and accuracy of generated motions. While this design offers clear computational advantages, additional discussion of the diversity-accuracy trade-off would help clarify how well the model captures a broad and accurate range of plausible future motions.\n\n2. Despite emphasizing the realism of generated motions, the paper lacks qualitative results such as videos of generated motion clips, which limits the ability to assess temporal coherence and physical plausibility. Moreover, unlike several prior works that incorporate physically motivated constraints (e.g., joint velocity smoothness or symmetry), the paper does not appear to include or discuss such priors. Providing qualitative examples and analyzing these aspects would help strengthen realism of the generated sequences.\n\n3. While the reliance on a precomputed cache is acknowledged as a limitation, it would be helpful to include targeted evaluations such as cross-action generalization tests or ablations on unseen action categories to better understand how well CacheFlow extrapolates beyond training-distribution motions.\n\nMinor weakness: Highlighting (e.g., bold fonts) is missing in Table 1, making comparisons harder to parse.\n\nQUESTIONS:\n1. What does \"FlowPrecomp\" refer to in Table 1? I guess it’s a typo.\n\n2. Have the authors considered augmenting the cache or incorporating mechanisms to update it dynamically at inference time, in order to improve generalization to out-of-distribution conditions?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses a key limitation in probabilistic human motion prediction: the high computational latency of state-of-the-art generative models, which hinders their use in real-time applications like interactive robotics and virtual reality. The central goal is to drastically reduce inference time without sacrificing the quality and diversity of the predictions.\n\nTo achieve this, the paper introduces CacheFlow, a novel framework that trades significant offline computation for extremely fast online inference. The method consists of two main stages. First, in an offline \"slow path,\" a powerful but computationally expensive normalizing flow model generates a large and diverse cache of high-quality future motion sequences. Second, for online prediction, a separate, lightweight conditional model is trained. This model takes an observed motion history as input and quickly outputs a latent vector, which is then used to rapidly retrieve the most suitable motion sequence from the pre-computed cache via a fast nearest-neighbor search. This approach effectively bypasses the slow generation step at inference time.\n\nThe primary contribution is a new paradigm for motion prediction that decouples generation from retrieval. Experimental results on the Human3.6M and AMASS datasets demonstrate that CacheFlow achieves inference speeds around 1.3 ms, approximately 30 times faster than prior state-of-the-art methods, while maintaining comparable prediction accuracy. This work presents a practical and effective solution for enabling high-fidelity, real-time human motion prediction in latency-sensitive systems.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths*\n- The paper does an excellent job of motivating the research. It clearly identifies a real-world bottleneck (inference latency) that prevents the deployment of otherwise powerful generative models in interactive applications like robotics and VR. The goal is well-defined and undeniably important.\n\n- The high-level concept of CacheFlow is presented clearly, especially in Figure 1. A reader can quickly grasp the core idea of trading offline computation for online speed by pre-generating a cache of motions.\n\n*Weaknesses*\n\nThe paper's strengths are unfortunately overshadowed by deep and fundamental weaknesses that undermine its scientific validity.\n\n- The central and fatal flaw of this paper is its complete failure to analyze the costs of its own method. The authors present the dramatic inference speed-up as a \"free lunch,\" which it is clearly not. A rigorous scientific paper must provide a balanced analysis of its proposed method, including its drawbacks. This paper omits:\n\n- Analysis of Computational Cost: The entire \"Slow Path\" (Figure 1), which is the foundation of the method, is un-analyzed. How much time and what computational resources (e.g., GPU-days) are required to generate the cache? Without this information, the \"30x speed-up\" claim is misleading. The method may simply be shifting a prohibitive computational burden from inference time to a one-time, but equally prohibitive, offline cost.\n\n- Analysis of Memory Cost: The cache consists of N_z = 2^15 motion sequences. This is not a trivial amount of data. What is the memory footprint (in MB or GB) of this cache? This is a critical parameter for the very applications the paper targets (robotics, AR/VR), which are often deployed on resource-constrained hardware. A method that requires gigabytes of RAM is not a practical solution.\n\n- Analysis of Inflexibility: The static cache means the model's predictive capability is permanently frozen. The paper fails to discuss the severe practical limitation this imposes. How does one update the system to handle new motions? The apparent need to regenerate the entire cache is a massive drawback that is ignored.\nThis lack of analysis is not a minor omission; it is a failure to conduct basic scientific due diligence. Without it, the community cannot fairly evaluate the method's utility or reproduce the work under comparable cost constraints.\n\n- The core idea is a straightforward space-time tradeoff (caching), a classic technique in computer science. While applying it to this domain is new, the paper presents it without any novel theoretical insight or principled analysis. It comes across as a brute-force engineering solution. Given the lack of analysis of the \"space\" (memory) and \"pre-computation time\" costs, the contribution feels hollow and lacks the depth expected for a top-tier publication.\n\n(1) The experimental evaluation is conducted in a sterile, best-case scenario that does not support the paper's claims of real-world applicability.\n\n(2) The model is tested on clean, high-quality MOCAP data. However, its stated applications (e.g., assistive robots) will rely on real-world sensor data, which is inevitably noisy. The paper provides no evidence that the method is robust to such noise. The nearest-neighbor lookup is likely to be highly sensitive to perturbations in the input, a critical failure mode that is never tested.\n\n(3) The reliance on datasets like Human3.6M, which lack diversity, means the resulting cache is inherently biased. The model would likely fail for individuals whose movements (due to age, disability, or cultural background) are not represented. This isn't just a generic AI bias issue; it's a fundamental architectural flaw of a system that cannot predict anything outside its fixed cache.\n\n- The paper's claims of significance are predicated on its suitability for real-world interactive systems. However, given the un-analyzed (and likely prohibitive) memory costs and the lack of demonstrated robustness, these claims are unsubstantiated. The paper shows the method works in a vacuum but fails to provide the evidence needed to believe it is a viable solution for the problems it claims to solve.\n\n\n- 3DPW, BEHAVE, RICH, GIMO are also the representive and recently-used datasets. Please provide the results on more broader datasets.\n\nQUESTIONS:\n1. What are the full computational and memory costs of the CacheFlow method?\n\nYour paper's central claim is a ~30x inference speed-up, but this speed comes at the cost of a significant offline computation and an online memory footprint, neither of which are quantified. To properly evaluate the method's practicality, it is essential to understand these costs.\n\nActionable Request: Please provide the following metrics for the Human3.6M and AMASS experiments:\n\n(a) Cache Generation Time: The total wall-clock time required to run the \"Slow Path\" and generate the full cache (N_z = 2^{15}), specifying the hardware used (e.g., number and type of GPUs).\n\n(b) Cache Memory Footprint: The total size of the pre-computed cache in Megabytes or Gigabytes.\n\n(c) Online Memory Usage: The total RAM required during online inference (including the cache and the lightweight conditional model).\n\nHow This Could Change My Score:\n\nIncrease: If you can demonstrate that these costs are reasonable for your target applications (e.g., cache generation takes hours, not weeks, and the memory footprint is manageable for a typical robot or AR device), this would substantially strengthen your claim of practicality and I would be inclined to raise my score.\n\nDecrease/Remain: If the costs are prohibitive (e.g., weeks of computation on a large cluster, or a multi-gigabyte cache), or if you are unable to provide these numbers, it would confirm my suspicion that the method is not a practical solution, solidifying my recommendation for rejection.\n\n2. How robust is CacheFlow to noisy input data?\n\nThe paper claims applicability to real-world systems, yet it is only evaluated on clean, laboratory-grade motion capture data. Real-world motion tracking is imperfect. The nearest-neighbor search at the core of your method seems particularly susceptible to noise, where a small perturbation in the input could lead to retrieving a completely incorrect motion from the cache.\n\nActionable Request: Please provide an analysis of the model's robustness. The ideal response would be a new experiment where you add varying levels of controlled noise (e.g., Gaussian noise with increasing variance) to the input motion history and report how the prediction error (MPJPE) degrades. A substantive discussion of this failure mode and potential mitigation strategies would also be valuable.\n\nHow This Could Change My Score:\n\nDemonstrating that the prediction quality degrades gracefully with increasing noise would significantly bolster your claims of real-world applicability. Even if performance degrades sharply, a transparent analysis of this limitation would improve the paper's scientific quality and could lead me to raise my score.\n\nIgnoring this question or stating that it is out of scope would reinforce my view that the work is a proof-of-concept that is not ready for real-world consideration.\n\n3. How is the cache meant to be updated or adapted?\n\nThe proposed system is static. Its knowledge is frozen in the pre-computed cache, which seems highly impractical for applications that require adaptation to new users, tasks, or environments (e.g., assistive technology that must adapt to a specific user's gait).\n\n Please discuss the procedure and associated costs for updating the cache. Consider two scenarios: (a) incrementally adding a few new motion types, and (b) adapting the system to a new domain with substantially different motions. Is a full, costly regeneration of the entire cache the only option?\n\nHow This Could Change My Score:\n\nIf you can propose a principled and computationally feasible method for updating or fine-tuning the cache/model, it would address a major architectural weakness of the current framework. This would make the system far more practical and would likely cause me to raise my score.\n\nIf the only solution is a complete and costly regeneration, this is a severe limitation that must be acknowledged as a major drawback. A failure to address this question would confirm that the method's flexibility has not been considered.\n\nLIMITATIONS:\nThe model is evaluated on Human3.6M and AMASS.\n\n- hese datasets, while standard, primarily consist of motion capture data from a limited number of actors in controlled, laboratory settings. They lack diversity in terms of age, body type, mobility (e.g., motions of elderly people or people with disabilities), and cultural context.\n\n- The authors should acknowledge that the model's performance will likely degrade significantly when faced with populations underrepresented in the training data. This is not just a general AI problem; it is a specific limitation of this system, as its cache will not contain these diverse motions.\n\n- The method maps a past trajectory to a future one. Real-world motion capture (e.g., from a single RGB camera) is often noisy and imperfect. The paper does not discuss how sensitive the lightweight conditional model is to noisy input data. Does slight noise in the input lead to drastically different (and incorrect) predictions from the cache?", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-04"}