Title: Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding
Venue: NeurIPS 2025
Accepted: yes
Keywords: large vision language model

temporal video grounding

reinforcement learning

post-training
Abstract: Temporal Video Grounding (TVG), the task of locating specific video segments based on language queries, is a core challenge in long-form video understanding. While recent Large Vision-Language Models (LVLMs) have shown early promise in tackling TVG through supervised fine-tuning (SFT), their ability to generalize remains limited. To address this, we propose a novel post-training framework that enhances the generalization capabilities of LVLMs via reinforcement learning (RL).
Specifically,  our contributions span three key directions:
(1) Time-R1: we introduce a reasoning-guided post-training framework via RL with verifiable reward to enhance capabilities of LVLMs on the TVG task. 
(2) TimeRFT: we explore post-training strategies on our curated RL-friendly dataset, which trains the model to progressively comprehend more difficult samples, leading to better generalization.
(3) TVGBench: we carefully construct a small but comprehensive and balanced benchmark suitable for LVLM evaluation, which is sourced from available public benchmarks.
Extensive experiments demonstrate that Time-R1 achieves state-of-the-art performance across multiple downstream datasets using significantly less training data than prior LVLM approaches, while improving its general video understanding capabilities. 
Project Page: https://xuboshen.github.io/Time-R1/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper applies GRPO (Guided Reinforcement Policy Optimization) to the problem of Temporal Video Grounding, which involves identifying the precise time interval during which a specified event occurs in a video. The authors also make strong contributions on the data side by constructing a benchmark tailored to this task through careful modification of existing datasets. Their approach demonstrates strong performance across multiple downstream tasks.\n\nHowever, I feel the post-training component is less compelling. The proposed RL method is largely similar to standard GRPO, differing mainly in the introduction of a new reward function (tIOU), which is effectively a form of reward shaping—a fairly common practice in RL. This makes it feel like an application of GRPO to a new task, but with limited technical contribution and novelty.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* The paper is clearly written and easy to follow.\n* The contributions are broad, covering multiple aspects including dataset construction, benchmarking, and post-training.\n\n**Weaknesses**\nMost of my concerns are focused on the novelty and technical contributions of the post-training component:\n\n* The proposed post-training approach using RL is very similar to GRPO, differing primarily in the choice of reward function (the proposed tIOU). This is a fairly minor contribution on its own, since GRPO is designed to allow simple task-specific reward shaping through human-defined or heuristic reward functions.\n* Relatedly, no ablation or analysis is demonstrating why the proposed reward function is better than simpler, more intuitive alternatives.\n\nQUESTIONS:\n- Can the authors ablate the reward function proposed? Here are some simple reward functions that I can think of. In interest of time it may not be easy to complete all, but at least some ablations are important to analyze the impact of the reward function:\n   - A simple GRPO style validation, i.e., is the proposed time interval an exact match?\n   - Absolute Error or RMSE\n   - The second term in the proposed tIoU\n- After seeing the use of tIoU vs IoU in the Appendix, there seems to be no clear quantitative reason why the proposed reward function should be used. Are there any qualitative examples?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Time-R1, a post-training framework based on reinforcement learning (RL) designed to enhance the temporal video grounding (TVG) capabilities of large vision-language models (LVLMs).\nTo enable efficient RL-based training, the authors propose an appropriate data curation and filtering strategy, as well as a reward model, and further introduce a compact benchmark to validate their approach.\nThe method achieves state-of-the-art performance across various benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. Time-R1 and TimeRFT\n\nThis work pioneers the use of reinforcement learning (RL)-based post-training for large vision-language models (LVLMs) in the context of temporal video grounding (TVG). \n\n2. Data Efficiency and Strong Generalization\n\nThe proposed method demonstrates that with a curated, moderately sized RL-friendly dataset (TimeRFT), it is possible to surpass the performance of large-scale supervised fine-tuning approaches.\n\n3. TVG Benchmark\nThe paper constructs a new, systematic benchmark (TVGBench) specifically designed for temporal video grounding. TVGBench reflects real-world task complexity and includes a balanced distribution of query types (human, object, environment, etc.), supporting objective and reproducible evaluation across diverse model architectures.\n\nWeaknesses\n\n1. Attribution of SOTA Performance to Backbone Quality\nIt remains unclear whether the observed SOTA performance is primarily due to the use of a stronger LVLM backbone (Qwen2.5-VL-7B), rather than the proposed training and reward strategies. In contrast, the SFT baselines utilize a variety of vision encoders and language models (e.g., TimeChat: ViT-G/14 from EVA-CLIP + LLaMA-2-7B; TIMESUITE: UMT-L + Mistral-7B; VideoChat-Flash: UMT-L + Qwen2-7B; TRACE: ViT-L/14 + Mistral-7B). The main results do not directly demonstrate whether Time-R1 yields significant performance improvements over these baselines when using the same vision encoder and language model architecture. A controlled head-to-head comparison on identical backbones would be necessary to fairly isolate the effectiveness of the proposed method.\n\n2. Effectiveness of the tIoU Reward\nSection 3.2 introduces the tIoU reward to promote deeper temporal understanding. However, in the ablation study (Table 2, Row 1 vs. Row 2), adding tIoU appears to decrease performance. The authors should provide a more detailed analysis or hypothesis as to why incorporating tIoU does not consistently yield gains, and under what circumstances (e.g., data distribution, filtering, w/ Multi-Epoch) it becomes effective.\n\n3. Ablation of Reasoning Template Reward\nWhile the paper claims that the reasoning template reward encourages better intermediate reasoning and thus benefits temporal grounding, it is not empirically clear to what extent this mechanism enhances the model's reasoning capability or TVG performance. A direct ablation comparing models with and without the reasoning template reward on reasoning quality, as well as an assessment of its impact across various benchmarks (as in Table 1 and Table S1), would greatly strengthen the claims.\n\n4. Quantitative Comparison of Inference Speed\nThe supplementary material mentions the slow inference speed as a limitation. However, a quantitative comparison of inference time or throughput between Time-R1 and relevant baselines is missing. Such a comparison would help practitioners assess the real-world trade-offs between accuracy and efficiency.\n\n5. (minor) typo in Table 2, trainning -> training\n\nQUESTIONS:\n1. Efficiency of Training: SFT vs. Time-R1\nHow does the computational cost and convergence speed of training a model using large-scale supervised fine-tuning (SFT) compare to training with the curated RL-friendly data in Time-R1? Specifically, is Time-R1 more efficient in terms of GPU hours, memory usage, or time to convergence, given the smaller yet carefully selected dataset?\n\n2. (Minor) In Supplementary Table S5, does “TimeZero-7B” refer to the Time-R1 method with the Qwen2.5-VL-7B backbone, or is it a different model?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of Temporal Video Grounding (TVG) with Large Vision-Language Models (LVLMs). The authors claim that existing LVLM-based TVG methods are sub-optimal (sometimes even perform worse than conventional feature-based small models) due to the over-penalization of false negatives during supervised fine-tuning (SFT). Therefore, to effectively supervise the model through task-specific metrics (e.g., temporal IoU), the authors proposed to leverage RL to allow direct optimization, thereby reducing rigid penalties of autoregressive losses and encouraging plausible timestamp predictions. The contributions of this work include Time-R1, TimeRFT, and TVGBench.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Overall, the paper is well-written and easy to follow.\n2. The motivation for using RL is clear and reasonable -- SFT overly penalizes false negatives even when the predicted timestamps are close to the ground truth, and adopting additional timestamp prediction heads would sacrifice the pretrained language capabilities of LLMs.\n3. I like the idea of dynamic hard sampling strategy during training. The experimental results also demonstrated the effectiveness of this scheme.\n\nWeaknesses:\n\n1. It is unclear why applying GRPO (DAPO) with temporal grounding data & tIoU rewards can improve general video QA performance. To my understanding, training solely on such data would improve performance on TVG, but would also lead to lower performance on other tasks. The authors should provide more clarification for it.\n2. In Figure 4, why does SFT significantly underperform even the base model on all tasks?\n3. For the SFT-based cold start, the authors did not provide an in-depth analysis on why using dense video captioning-style thinking as CoT.\n4. Several recent important baselines / benchmarks shall be carefully discussed / compared to justify both the effectiveness of Time-R1 and TVGBench. To name some, the missing methods in Table 1 and related work are [1-5], the similar benchmarks requiring further discussions are [2, 7, 8].\n\n[1] Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning. ICML 2024.\n[2] E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding. NeurIPS 2024.\n[3] Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment. CVPR 2025.\n[4] VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Grounding. AAAI 2025.\n[5] VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning. arXiv 2025.\n[6] VideoExpert: Augmented LLM for Temporal-Sensitive Video Understanding. arXiv 2025.\n[7] ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos. NeurIPS 2024.\n[8] CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding. ICLR 2025.\n\nQUESTIONS:\nPlease refer to the weakness section for my questions. My major concern is the lack of clarifications and important baselines.\n\nLIMITATIONS:\nThe limitations are addressed in the supplementary material, but the potential negative societal impacts are not.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The work introduces RL post-training framework into Temporal Video
Grounding and achieves SOTA performance across various TVG benchmarks.
In addition, the work also curates a dataset and benchmark for TVG task,
contributing to the development of the research community.

STRENGTHS AND WEAKNESSES:
Strengths：
1. This work achieves SOTA performance across TVG benchmarks 
2. This work effectively integrates RL into TVG task and innovatively proposes two rewards curated for TVG task to assist the post training

Weaknesses：
1. TVGBench has only 800 samples and comes from existing datasets, so the
necessity of building this benchmark is questionable
2. This work is solid, but it is just the RL post-train paradigm commonly used in LLM
adapted to the TVG task, which is novel but not much

QUESTIONS:
1. The paper only uses Qwen2.5-VL-7B as the base model. Can the same conclusion be reached by using other series models or models of different sizes?
2. Figure 4 shows that the VQA capability is improved after RL post training. Is this because the dataset used in the TimeRFT stage contains VQA data? If only TVG CoT data is used for training in the post training stage, can the VQA capability be kept on par with the base model?
3. The paper does not conduct ablation experiments on the two rewards of the TVG task. Is it possible to use only one of the rewards to observe the experimental results?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-02