Title: MotionRAG: Motion Retrieval-Augmented Image-to-Video Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image-to-Video Generation

Motion Transfer

Retrieval-Augmented Generation
Abstract: Image-to-video generation has made remarkable progress with the advancements in diffusion models, yet generating videos with realistic motion remains highly challenging. This difficulty arises from the complexity of accurately modeling motion, which involves capturing physical constraints, object interactions, and domain-specific dynamics that are not easily generalized across diverse scenarios. To address this, we propose MotionRAG, a retrieval-augmented framework that enhances motion realism by adapting motion priors from relevant reference videos through Context-Aware Motion Adaptation (CAMA). The key technical innovations include: (i) a retrieval-based pipeline extracting high-level motion features using video encoder and specialized resamplers to distill semantic motion representations; (ii) an in-context learning approach for motion adaptation implemented through a causal transformer architecture; (iii) an attention-based motion injection adapter that seamlessly integrates transferred motion features into pretrained video diffusion models. Extensive experiments demonstrate that our method achieves significant improvements across multiple domains and various base models, all with negligible computational overhead during inference. Furthermore, our modular design enables zero-shot generalization to new domains by simply updating the retrieval database without retraining any components. This research enhances the core capability of video generation systems by enabling the effective retrieval and transfer of motion priors, facilitating the synthesis of realistic motion dynamics.


Peer reviews:
comments: SUMMARY:
This paper introduces MotionRAG, a retrieval-augmented framework designed to improve motion realism in image-to-video generation. The authors propose several contributions, including Retrieval-based motion adaptation and Seamless motion injection. The method enhances motion realism across diverse domains without retraining, supporting zero-shot generalization by updating the retrieval database.

STRENGTHS AND WEAKNESSES:
Strengths:
- The concept that "motion can be inherently transferred across domains" is well-justified and intuitive.
- The authors test their method on three different base models, demonstrating broad applicability.

Weaknesses:
- The retrieval database and test sets both come from OpenVid-1M and SkillVid, meaning the test videos likely share a similar distribution with the retrieved references. This could inflate performance compared to real-world scenarios where videos may differ significantly from the database. A more convincing evaluation would involve out-of-distribution or in-the-wild videos.
- Lack of physics-based evaluation: While the abstract and introduction emphasizes physical realism, the paper does not quantitatively assess whether generated videos adhere to physical laws. Recent benchmarks (e.g., "Do generative video models understand physical principles? arxiv '25") could provide more rigorous metrics for evaluating motion plausibility and dynamics.

QUESTIONS:
1. Please address the two concerns in the weakness section.
2. Further disussion between the proposed implicit motion conditioned mechanism and explicit optical-flow-based motion condition mechanism (Motion-I2V and MOFA-Video) would be helpful to understand the contribution of this submission.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-29

comments: SUMMARY:
This paper proposes MotionRAG, a retrieval-augmented framework for video generation that enhances motion realism by transferring motion priors from semantically similar videos retrieved via text prompts.
Experiments across general and domain-specific datasets (OpenVid-1K and SkillVid) show MotionRAG consistently improves motion realism, semantic alignment, and video quality with minimal computation overhead. Ablations further validate the design.

STRENGTHS AND WEAKNESSES:
stengths:
1. This paper is well-written and easy to understand.
2. The paer propose a RAG based video generation method which is interesting and meaningful for video generation community.
3. The expertiments are extensive enough to demonstrate the performance.

weaknesses:
One of the potential weaknesses of the proposed method is its strong dependence on the quality of the retrieved reference videos. It is unclear whether the model will be able to automatically filter or reduce the impact of these sub-optimal references if there are errors in the search results. Although the Motion Context Transformer is designed to have context-aware motion adaptation capabilities, its robustness to low-quality or erroneous search results is not explicitly analyzed in this paper, and mechanisms such as confidence weighting or retrieval screening are not proposed to alleviate this problem. Further exploration of denoising or robustness mechanisms related to retrieval quality will help improve the reliability of the system in practical applications.

QUESTIONS:
See Weaknesses.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

comments: SUMMARY:
This manuscript presents MotionRAG, a retrieval-augmented framework designed to improve motion realism in image-to-video generation. The core contributions are:

-A pipeline that uses sentence embeddings to retrieve semantically relevant reference videos from a large corpus, providing real-world motion priors for generation.
-Lightweight adapters inserted after cross-attention layers in frozen diffusion models, seamlessly integrating adapted motion features to guide video synthesis.

STRENGTHS AND WEAKNESSES:
Strengths
-By explicitly leveraging retrieved video examples, MotionRAG grounds motion dynamics in real data 
-The framework’s modular design allows for zero-shot transfer.
-The paper thoroughly investigates retrieval quality (random vs. semantic), number of references (5 vs. 9), and integration strategies (Top-1, Avg-9, MCT-9)

Weaknesses:
-Experiments focus on short clips (~2 seconds) and a fixed set of general and instructional domains; the approach’s scalability to long-range motions or fine-grained action nuances (e.g., subtle gestures) remains untested.
-Large-scale retrieval (e.g., nearest-neighbor search over millions of embeddings) can be nontrivial; reporting end-to-end latency including retrieval indexing would be necessary.
-MotionRAG’s performance hinges on the diversity and relevance of the retrieval corpus; can rare or unique motions perform well? better to add analysis and failure cases
-Will the work be open-source or public? It will be necessary to provide temporary/anonymous repo and demo code during the review.

QUESTIONS:
see comments above.
It will be necessary to address the comments above to maintain positive score.

LIMITATIONS:
see comments above
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a retrieval-augmented framework, MotionRAG, to enhance motion realism in video generation by transferring motion priors from relevant reference videos. The framework addresses the challenge of generating physically plausible and semantically coherent motion. MotionRAG contains a key component, Context-Aware Motion Adaptation (CAMA), that adapts retrieved motion priors to the target image context with in-context learning. Experiments demonstrate that MotionRAG can improve motion quality across multiple base models and domains with negligible computational overhead.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- This paper applies the RAG technique to video generation to improve motion realism, which is meaningful for the video generation task.\n- This paper achieves zero-shot effective motion transfer across different domains with negligible computational overhead.\n- The writing of this paper is clear and easy for the reader to follow.\n\nWeaknesses\n- The RAG technique itself is not novel and has been proven to be effective in other related generation tasks. Therefore, the contribution of applying RAG to motion transfer is relatively weak.\n- One of the crucial points of RAG is to construct the dataset, but this paper lacks the necessary data analysis. How such datasets are constructed is important to the community.\n- This paper lacks a detailed analysis and visualization of the semantic similarity of the retrieved reference videos. Based on the current version, MotionRAG relies on the existence of videos with highly similar motion in the dataset and cannot accurately achieve motion transfer.\n- Existing evaluation metrics cannot fully reflect the quality of motion, especially the physical constraints claimed in this paper.\n\nQUESTIONS:\n- The captions given by the existing video captioners are bound to be rough and unable to accurately describe the physical motions. When constructing a dataset, is it necessary to manually select homogeneous reference videos?\n- I highly doubt the robustness of filtering out reference videos based solely on text similarity. Text cannot convey the details of physical motion and may lead to incorrect retrieval. Even for the examples shown in the supplementary material, the text descriptions are inaccurate. There are also large differences in physical motion between the retrieved reference videos. What about the reference video retrieved when the text description is out of domain?\n\nLIMITATIONS:\nI would suggest adding more analysis and visualization of the dataset in the revised version.\n\nAnd more metrics for measuring physical motion need to be considered.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}