MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal LLMs

## Abstract

Abstract Despite advancements in Multimodal Large Language Models (MLLMs), their proficiency in fine-grained video motion understanding remains critically limited. They often lack inter-frame differencing and tend to average or ignore subtle visual cues. Furthermore, while visual prompting has shown potential in static images, its application to videos’ temporal complexities, particularly for fine-grained motion understanding, remains largely unexplored. We investigate whether inherent capability can be unlocked to boost MLLMs’ motion perception and enable distinct visual signatures tailored to decouple object and camera motion cues. In this study, we introduce 𝙼𝚘𝚝𝚒𝚘𝚗𝚂𝚒𝚐𝚑𝚝 \mathtt{MotionSight} , a novel zero-shot method pioneering object-centric visual spotlight and motion blur as visual prompts to effectively improve fine-grained motion understanding without training. To convert this into valuable data assets, we curated 𝙼𝚘𝚝𝚒𝚘𝚗𝚅𝚒𝚍 − 𝚀𝙰 \mathtt{MotionVid-QA} , the first large-scale dataset for fine-grained video motion understanding, with hierarchical annotations including SFT and preference data, Θ ​ ( 40 ​ K ) \Theta{(40K)} video clips and Θ ​ ( 87 ​ K ) \Theta{(87K)} QAs. Experiments show 𝙼𝚘𝚝𝚒𝚘𝚗𝚂𝚒𝚐𝚑𝚝 \mathtt{MotionSight} achieves state-of-the-art open-source performance and competitiveness with commercial models. Using 𝙼𝚘𝚝𝚒𝚘𝚗𝚅𝚒𝚍 − 𝚀𝙰 \mathtt{MotionVid-QA} , we fine-tuned 𝙼𝚘𝚝𝚒𝚘𝚗𝙲𝚑𝚊𝚝 \mathtt{MotionChat} on Qwen2.5VL-7B, which attains 48.3% overall accuracy on FAVOR-Bench that is comparable to Qwen2.5VL-72B’s 48.1%.
In summary, we present a novel zero-shot method and a large-scale, high-quality dataset specifically for fine-grained motion understanding. All the code and annotations will be publicly available.

## 1 Introduction

Benefiting from high-quality video-text datasets (Chen et al., 2024b; Liu et al., 2024a; Chen et al., 2024a; Wang et al., 2023; Yang et al., 2024) and large model parameters (Zhu et al., 2025b; Hurst et al., 2024; Bai et al., 2025; Zhu et al., 2025a; DeepMind, 2025), Multimodal Large Language Models (MLLMs) have exhibited impressive performance on a wide range of video understanding tasks.
Unlike static images, videos possess a temporal dimension characterized by continuous frame-to-frame changes over time. These changes, originating from object or camera motion, create dynamic and expressive motion patterns, distinguishing videos from images, as illustrated in Figure [1](https://arxiv.org/html/2506.01674v2#S0.F1)(a).

However, even with rapid progress and development in video understanding, the task of fine-grained video motion understanding still lacks necessary attention and exploration. While MLLMs acquire broad semantic knowledge from large-scale data pre-training (Bai et al., 2025; Chen et al., 2024d; Zhang et al., 2024b; Wang et al., 2025a; Zhang et al., 2025), their direct application to motion understanding is often suboptimal. This stems from their tendency to process spatial regions with uniform importance and a lack of inherent mechanisms to explicitly discern the subtle inter-frame variations critical for nuanced motion analysis. These predispositions mean their potential for fine-grained understanding remains largely untapped (Tu et al., 2025) as shown in Figure [1](https://arxiv.org/html/2506.01674v2#S0.F1) (b). Given this, how can we boost the latent capabilities of MLLMs derived from large-scale data to achieve fine-grained understanding of local motion cues and enhance the modeling of subtle inter-frame dynamics? Furthermore, how can we transform the augmented implicit understanding from the models into structured data assets that can be used for training other models and for in-depth analysis?

Motivated by these questions, we conducted extensive experiments and explorations on how to boost MLLMs’ inherent fine-grained motion understanding capability through zero-shot approaches, without relying on additional training data. Previous studies in the image understanding domain (Yao et al., 2022; Shtedritski et al., 2023; Yang et al., 2023b; a) have demonstrated strong interest in visual prompting techniques, but their extension to address the intricate temporal complexities of video, especially for nuanced fine-grained motion understanding, still requires further investigation.
To demonstrate the inadequacy of naively adapting prompting methods from static images to encapsulate the intrinsic temporal dynamics of events in video, we conducted transfer evaluations on recent motion-specific benchmarks, and found that even background blur, the best-performing image-based visual prompt (Yang et al., 2023b), paradoxically exhibited the poorest performance in fine-grained motion understanding. This approach tends to decrease the model’s ability in fine-grained motion understanding due to the loss of contextual information, as shown in Figure [3](https://arxiv.org/html/2506.01674v2#S2.F3).

To address this, we propose $\mathtt{MotionSight}$, a novel video visual prompting method that decouples object and camera motion tailored for fine-grained motion understanding. For object motion, we apply a spotlight-like visual prompt on bounding boxes correlated with the queried motion to enhance motion perception (aiming at focusing the model’s attention on the core motion, as illustrated in Figure [7](https://arxiv.org/html/2506.01674v2#S5.F7)). This is inspired by the fact that pre-training data inevitably includes scenarios like stage performances and TV shows, where important moving subjects are often highlighted while the background is dimmed.
For camera motion, which necessitates the MLLM’s perception of subtle scene changes—a capability where MLLMs often exhibit limitations (Hong et al., 2025; Tu et al., 2025)—we designed a method to synthetically introduce motion blur into video frames. Interestingly, our experiments reveal that this addition of motion blur significantly benefits camera motion determination.
Through our carefully designed configuration, $\mathtt{MotionSight}$ enables MLLMs to achieve enhanced results for fine-grained motion understanding without additional training data.

To further convert this capability into explicit, actionable data assets, we collected and annotated $\Theta(40K)$ video clips, $\Theta(87K)$ question-answer pairs. Through a rigorous filtering mechanism that enhanced the quality of annotated data, we developed an SFT dataset and a preference dataset for training strategies. This process distilled the fine-grained motion understanding capability of MLLMs and aligned it with human preference, and constructed $\mathtt{MotionVid-QA}$, the first large-scale open-source dataset for fine-grained video motion understanding to date, encompassing diverse scenes and high-quality video footage. Benefiting from this dataset, our trained $\mathtt{MotionChat}$ significantly outperformed the original Qwen2.5VL-7B model, thus validating the effectiveness of $\mathtt{MotionVid-QA}$.
Extensive experiments demonstrated that $\mathtt{MotionSight}$ and $\mathtt{MotionVid-QA}$ effectively contribute to enhancing fine-grained motion understanding.

## 2 Related Work

Figure: Figure 2: Overview of the interaction process. Left: Our $\mathtt{MotionSight}$ pipeline captions high-quality data, transforming it into data assets. Right: This data undergoes rigorous filtering to align with human preferences, resulting in our high-quality dataset $\mathtt{MotionVid-QA}$.
Refer to caption: x2.png

#### MLLMs for video understanding.

As MLLMs continue to advance, a growing body of research focuses on applying them to video understanding (Bai et al., 2025; Hong et al., 2024; Yao et al., 2024; Xu et al., 2024; Liu et al., 2024c; Zhang et al., 2024a; Wang et al., 2024a; Zhang et al., 2024b; Li et al., 2024; Wang et al., 2024c; 2025a; Zhang et al., 2025; Wang et al., 2024b; Chen et al., 2024d).

Figure: Figure 3: Comparison of our method with other existing methods. Directly applying image visual prompts can lead to misinterpretation. By employing decoupled object-guided motion focusing and inter-frame information enhancement, our method addresses the challenge faced by previous methods.
Refer to caption: x3.png

Video understanding models often use keyframes as samples, which are then encoded for LLMs. Several approaches develop specialized connectors (such as improved Q-Formers for video) (Wang et al., 2024c; 2025a; Zhang et al., 2025), while models like QwenVL (Bai et al., 2025; Wang et al., 2024b) and InternVL (Chen et al., 2024d) encode videos frame-by-frame through a vision encoder before feeding them to the LLM. Although these methods excel at event-level video representation, they tend to struggle with fine-grained motion understanding due to limited perception of inter-frame dynamic differences, resulting in inaccurate information retrieval. To overcome this limitation, we propose a training-free pipeline that decouples object motion and camera motion, leveraging novel visual prompting methods to improve the model’s understanding of fine-grained motion.

#### Fine-grained motion understanding datasets.

Early action recognition datasets (Soomro et al., 2012a; Kuehne et al., 2011; Caba Heilbron et al., 2015) had limited fine-grained motion understanding due to simplistic categorical labels.
Recent works use MLLMs for auto-annotation (Chen et al., 2024b; Wang et al., 2023; Yang et al., 2024; Nan et al., 2024), but granularity remains limited.
Structured video captions (Ju et al., 2024; Fan et al., 2024a; Wu et al., 2025) respond to the need for fine-grained semantics. However, deficiencies persist in motion semantics delineation due to the lack of a well-designed approach for obtaining fine-grained semantic representations.
Benchmarks like MotionBench (Hong et al., 2025) and FAVOR-Bench (Tu et al., 2025) have datasets with insufficient sample sizes, limiting scene diversity and semantic richness. To overcome these limitations, we propose $\mathtt{MotionVid-QA}$, the first large-scale dataset for fine-grained motion understanding, featuring extensive scene coverage and high video quality.

## 3 MotionSight

Figure: Figure 4: The detailed pipeline of $\mathtt{MotionSight}$. Our method includes query-based motion decoupling, gating based on object motion and camera motion. Subsequently, it selectively passes through modules based on the decoupled type. Then, we carefully designed a template prompt for MLLMs to understand our enhanced input and make final decisions.
Refer to caption: x4.png

This section introduces $\mathtt{MotionSight}$ (Figure [4](https://arxiv.org/html/2506.01674v2#S3.F4)) with enhanced fine-grained motion perception. Our method decouples object and camera motion and discusses techniques to enhance MLLM input videos. We sample the input video $\mathbf{V}=\{\mathcal{I}_{i}\}_{i=1}^{L}$ ($\mathcal{I}_{i}\in\mathbb{R}^{3\times H\times W}$) to $\mathbf{V}_{s}=\{\mathcal{I}_{s_{j}}\}_{j=1}^{T}$, with $L$ and $T$ denoting original video length and sampled frames length, respectively. Our approach is as follows:

$$ $\mathcal{R}_{obj}=\text{MLLM}(\Phi_{obj}(\mathbf{V}_{s}))),\quad\mathcal{R}_{cam}=\text{MLLM}(\Phi_{cam}(\mathbf{V}_{s},\mathbf{V})).$ (1) $$

Here, $\mathcal{R}_{obj}$ and $\mathcal{R}_{cam}$ are object and camera motion understanding, respectively. $\Phi_{obj}$ and $\Phi_{cam}$ are the corresponding visual prompting functions.

### 3.1 Object Referring

Initially, the MLLM processes sampled frames $\mathbf{V}_{s}$ and the query $\mathcal{Q}$ to infer a set of semantically relevant object categories $\mathcal{C}=\{c_{1},c_{2},...,c_{n}\}$. This inferred set $\mathcal{C}$ guides our subsequent visual perception modules for object localization and trajectory estimation. Formally, the process of obtaining tracked object trajectories $\mathcal{O}$ is defined by the composition:

$$ $\mathcal{O}=\mathcal{M}_{track}(\mathcal{M}_{detect}(\mathcal{I}_{s_{t}},\mathcal{C};\theta_{det}),\{\mathcal{I}_{s_{j}}\}_{j=t+1}^{T};\theta_{track}),$ (2) $$

where $\mathcal{M}_{detect}$ (Liu et al., 2024b) identifies bounding boxes for categories $\mathcal{C}$ in key frame $\mathcal{I}_{s_{t}}$. $\mathcal{M}_{track}$  (Ravi et al., 2024) then propagates these detections across subsequent frames $\{\mathcal{I}_{s_{j}}\}_{j=t+1}^{T}$ yielding trajectories $\mathcal{O}$. While direct action inference can hallucinate, robust object identification, even with initial errors, is refinable by lower-confidence detections (Chen et al., 2024c).

### 3.2 Action Focusing

Given tracked objects $\mathcal{O}=\{(\mathbf{b}_{s_{t},i})_{t=1}^{T_{i}}\}_{i=1}^{m}$ ($\mathbf{b}_{s_{t},i}$: $i$-th object’s bounding box at frame $s_{t}$; $T_{i}$: trajectory length), we use a dynamic temporal aggregator $\mathcal{A}$ to derive refined spatial regions $\mathcal{B}=\{b_{t}\}_{t=1}^{T}$, which merge and stabilize bounding boxes against jittering. $\mathcal{A}$ adaptively adjusts its temporal aggregation window based on intra-trajectory positional variance $\mathcal{V}$. Let $\mathcal{X}=\mathcal{U}_{i=1}^{m}(\mathbf{b}_{s_{1:T_{i}},i})$ denote the union of bounding boxes in each frame:

$$ $\mathcal{B}=\mathcal{A}\left(\mathcal{X},\,\mathcal{V}(\mathcal{X})\right)=\{b_{t}\}_{t=1}^{T}.$ (3) $$

Here, $\mathcal{V}(\cdot)$ quantifies bounding box positional variance along a trajectory. Specifically, with low positional variance, $\mathcal{A}$ favors a union of bounding boxes over longer temporal spans; with high variance, it focuses on localized regions in shorter temporal windows. To quantify positional variance, we measure Manhattan distance between pairs of bounding box centers as $\|\text{center}(\mathbf{b}_{s_{t_{1}},i})-\text{center}(\mathbf{b}_{s_{t_{2}},i})\|_{1}$.

The object motion enhancement function $\Phi_{obj}$ then applies visual prompting techniques to the original frames using these dynamically aggregated object regions:

$$ $\Phi_{obj}(\mathbf{V_{s}})=\mathcal{F}_{VP}(\mathbf{V_{s}},\mathcal{B}),$ (4) $$

where $\mathcal{F}_{VP}$ represents our visual spotlight approach that darkens the background outside $\{b_{t}\}_{t=1}^{T}$ while preserving the detected objects in their original positions, enhancing focus on the relevant moving elements. In this way, our visual prompt considers object focus and smooth transition.

### 3.3 Motion Blur

To overcome the inherent limitations of existing MLLMs in perceiving inter-frame changes, particularly subtle camera motions, we introduce a dedicated Motion Blur Transformation $\mathcal{T}_{MB}$ as our camera motion core enhancement function $\Phi_{cam}$. This function, $\Phi_{cam}(\mathbf{V},\mathbf{V}_{s})=\mathbf{V}^{\prime}$, operates on sampled timestamps $\{s_{t}\}_{t=1}^{T}$ using the entire video sequence $\mathbf{V}=\{\mathcal{I}_{t}\}_{t=1}^{T}$ to generate a sequence of motion-enhanced frames $\mathbf{V}^{\prime}=\{\mathcal{I}^{\prime}_{s_{t}}\}_{t=1}^{T}$, thereby amplifying temporal motion cues. For a given frame $\mathcal{I}_{s_{t}}$, the enhanced frame $\mathcal{I}^{\prime}_{s_{t}}$ is derived through a temporally weighted aggregation of its $N$ preceding frames from the original video. This process can be formally expressed as:

$$ $\displaystyle\Phi_{cam}(\mathbf{V},\mathbf{V}_{s})=\left\{\mathcal{T}_{MB}(\mathbf{V}_{s},N,t)\right\}_{t=1}^{T},\quad\text{where }\mathcal{T}_{MB}(\cdot)=\sum_{k=0}^{N-1}w_{k}(\gamma)\cdot\mathcal{I}_{s_{t}-k}.$ (5) $$

Here, $N$ is the temporal window size (zero-padding applied for ${s_{t}-k}<1$). The temporal kernel we use has a temporally increasing trend, with $\sum_{k}w_{k}(\gamma)=1$. This temporal aggregation within $\Phi_{cam}$ accentuates motion trajectories by inducing motion blur effects across $\mathbf{V}^{\prime}$, enhancing the MLLM’s capacity to perceive and interpret subtle camera movements.

**Table 1: The comparison of existing motion-specific datasets with ours. Our dataset significantly surpasses existing methodologies in both scale and annotation granularity. Furthermore, the quality of our dataset generally exceeds that of currently prevalent motion-specific datasets used for comparison.**
| Dataset | #Videos | #Text | Annotation Types | Subject | Usage | Scenario |
| --- | --- | --- | --- | --- | --- | --- |
| UCF101 (Soomro et al., 2012b) | 13K | N/A | Class Labels | Human | Action recognition | Human-centric |
| ActivityNet (Caba Heilbron et al., 2015) | 20K | N/A | Class Labels | Human | Action recognition | Human-centric |
| Kinetics-700 (Carreira et al., 2019) | 650K | N/A | Class Labels | Human | Action recognition | Human-centric |
| Charades (Sigurdsson et al., 2016) | 9K | 27K | Captions | Human, Indoor objects | Video understanding | Indoor |
| Charades-Ego (Sigurdsson et al., 2018) | 8K | 68K | Captions | Human, Indoor objects | Video understanding | Indoor |
| MotionBench-train (Hong et al., 2025) | 5K | 5K | Captions | Diverse objects, Camera | Motion understanding | Open domain |
| FavorBench-train (Tu et al., 2025) | 17K | 17K | Captions | Diverse objects, Camera | Motion understanding | Indoor |
| $\mathtt{MotionVid-QA}$ (ours) | 40K | 87K | Mixed QAs | Diverse objects, Decoupled camera | Fine-grained motion understanding | Open domain |
| - SFT | 35K | 80K | QAs | - | Enhance motion understanding | - |
| - DPO | 5K | 7K | Chosen/Reject QAs | - | Align with human preferences | - |

Figure: Figure 5: $\mathtt{MotionVid-QA}$: High-quality filtering. Construction of a high-quality video dataset via filtration. Extensive data distribution. Diverse sources yield varied scenes, subjects, and camera perspectives. Human preference comparison. Our preference data annotation significantly surpasses baseline, particularly for camera motion (Zoom in for best view).
Refer to caption: figure/chart2.png

Figure: Figure 6: A comparative visualization of $\mathtt{MotionVid-QA}$ against existing data. In contrast to antecedent methodologies reliant upon class labels or captions, our approach facilitates the provision of substantially richer and more diverse informational content. Notably, even in scenarios characterized by the absence of salient principal objects, our methodology consistently yields high-quality annotations pertaining to object and camera dynamics.
Refer to caption: x5.png

## 4 MotionVid

In this section, our work yields two key data resources: instruction/preference subsets from public data tailored for two-stage model refinement (SFT/DPO) towards high-quality motion understanding.

#### Dataset collection and processing.

$\mathtt{MotionVid-QA}$ is curated from a variety of sources to ensure multiple types of motion understanding tasks are covered, and rich scenarios ensure the diversity of the dataset, including ActivityNet (Caba Heilbron et al., 2015), Kinetics-700 (Carreira et al., 2019), Charades (Sigurdsson et al., 2016), Charades-Ego (Sigurdsson et al., 2018), Tarsier2-Recap-585K (Yuan et al., 2025), OpenVid-1M (Nan et al., 2024), and MotionBench-train (Hong et al., 2025).
To ensure the quality of the videos, data processing steps we outline in Figure [2](https://arxiv.org/html/2506.01674v2#S2.F2) are applied.

#### SFT and DPO.

SFT aims to produce a specialized model $\pi^{\text{SFT}}$ capable of effectively capturing spatiotemporal dynamics and semantic motion patterns inherent in video data, thereby enhancing its performance on specific video understanding applications.
DPO aims to simplify Reinforcement Learning from Human Feedback (RLHF) by utilizing the log-likelihood of the learning policy. Instead of learning an explicit reward model, it implicitly expresses the reward function through pair-wise preference data $D=\{(x_{i},y^{chosen}_{i},y^{reject}_{i})\}_{i=1}^{M}$ to optimize the policy model. Let $\pi_{\theta}$ is the language model policy which always initialized to $\pi^{\text{SFT}}$, and $\pi_{\text{ref}}$ is also initialized from $\pi^{\text{SFT}}$. The objective function is defined as:

$$ $\mathcal{L}_{\text{DPO}}({\theta})=-\mathbb{E}_{(x,y^{chosen},y^{reject})\sim\mathcal{D}}\left[\log\sigma\left(\beta\log\frac{\pi_{\theta}(y^{chosen}|x)}{\pi_{\text{ref}}(y^{chosen}|x)}-\beta\log\frac{\pi_{\theta}(y^{reject}|x)}{\pi_{\text{ref}}(y^{reject}|x)}\right)\right]$ (6) $$

The DPO loss aims to maximize the reward difference between preferred and non-preferred samples, using human preference comparisons as signals (see Figure [5](https://arxiv.org/html/2506.01674v2#S3.F5)).

#### 𝙼𝚘𝚝𝚒𝚘𝚗𝚅𝚒𝚍 − 𝚀𝙰 \mathtt{MotionVid-QA} : a large-scale dataset for fine-grained motion understanding.

For the pre-filtered dataset, we selected $\Theta(40K)$ clips annotated them with $\mathtt{MotionSight}$. Using VQAScore (Lin et al., 2024) and human thresholds for categorization, high-quality clips became preference dataset candidates, low-quality ones were eliminated, and the rest formed our instruction dataset.
In the SFT phase, to enhance motion understanding capabilities via SFT, we used $\mathtt{MotionSight}$ annotations as text data. For the preference dataset, aiming to align fine-grained motion understanding with human preferences via DPO. With a balance between efficiency and quality, we re-annotated this portion of the data using Tarsier2 (Yuan et al., 2025) as the baseline. High-quality preferences data was then developed by incorporating human preference signals from multiple, guided, and well-educated individuals.

Our curated dataset (Table [1](https://arxiv.org/html/2506.01674v2#S3.T1), Figure [6](https://arxiv.org/html/2506.01674v2#S3.F6)) significantly advances fine-grained video motion understanding. Its key contributions are:
(1) Pioneering scale and scope. The first large-scale, open-source dataset for this task, offering diverse scenes and high-quality footage.
(2) Diversity and quality. Rigorous filtering enhances clarity (mitigating annotation hallucinations), text-video consistency and pronounced dynamics (cf. Figure [5](https://arxiv.org/html/2506.01674v2#S3.F5)).
(3) Hierarchical data composition. Comprising SFT and preference subsets, it enables multi-faceted learning: general motion understanding (SFT) and refined, human-aligned fine-grained comprehension.
Overall, this large-scale, high-quality, hierarchically structured resource will significantly support future model training and evaluation.

## 5 Experiments

### 5.1 Datasets and Comparisons

**Table 2: Quantitative results on MotionBench. We compared our $\mathtt{MotionSight}$ with both proprietary MLLMs and open-source MLLMs on MotionBench, all of which have been trained on large-scale video data. The best results of open-source methods are marked in bold.**
| Model | # Frames | Overall | AVG. | MR | LM | CM | MO | AO | RC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Proprietary MLLMs |  |  |  |  |  |  |  |  |  |
| Gemini 2.0 Flash (DeepMind, 2025) | 1fps | 56.1 | 52.6 | 60.9 | 57.1 | 50.9 | 74.1 | 37.6 | 35.0 |
| Gemini 1.5 Pro (Team et al., 2024) | 1fps | 51 | 48 | 51 | 52 | 54 | 67 | 40 | 22 |
| GLM-4V-Plus-0111 (ZhipuAI, 2025) | 2fps | 62.8 | 60.3 | 64.1 | 67.0 | 67.4 | 73.5 | 46.7 | 42.8 |
| Open-source MLLMs |  |  |  |  |  |  |  |  |  |
| Oryx-34B (Liu et al., 2024c) [ICLR’25] | 64 | 49 | 47 | 48 | 52 | 44 | 65 | 42 | 32 |
| LLaVA-NeXT-Video-34B (Zhang et al., 2024a) | 32 | 48 | 44 | 53 | 45 | 36 | 66 | 39 | 23 |
| Qwen2.5VL-7B (Bai et al., 2025) | 1fps | 53.0 | 48.8 | 58.3 | 55.3 | 34.0 | 71.5 | 39.5 | 34.0 |
| Qwen2.5VL-72B (Bai et al., 2025) | 1fps | 58.3 | 54.3 | 64.0 | 60.3 | 48.6 | 73.2 | 46.8 | 33.0 |
| InternVL3-8B (Zhu et al., 2025a) | 16 | 58.1 | 53.7 | 65.1 | 63.0 | 47.8 | 74.1 | 39.7 | 32.3 |
| InternVL3-78B (Zhu et al., 2025a) | 16 | 61.5 | 57.6 | 67.2 | 63.9 | 55.8 | 78.1 | 44.9 | 35.8 |
| TE Fusion (Hong et al., 2025) [CVPR’25] | 16 | 58 | 54 | 64 | 59 | 51 | 69 | 41 | 39 |
| Qwen2.5VL-7B + $\mathtt{MotionSight}$ | 1fps | 55.6 | 52.2 | 59.7 | 58.1 | 48.3 | 73.6 | 40.1 | 33.5 |
| InternVL3-78B + $\mathtt{MotionSight}$ | 16 | 63.0 | 59.3 | 68.5 | 65.4 | 58.7 | 78.6 | 47.6 | 37.0 |

#### Datasets.

We evaluated our method $\mathtt{MotionSight}$ on the benchmark of MotionBench (Hong et al., 2025) and FAVOR-Bench (Tu et al., 2025) using their publicly available dev set and close-ended evaluation (encompassing six types of question-answer pairs), respectively. They are both fine-grained motion-level benchmarks encompassing a wide range of video types.
We report accuracy for each problem type, with Overall representing accuracy across all problems. We also calculate AVG. as the average accuracy across question categories, giving equal weight to each problem type regardless of sample distribution. It’s worth noting that our $\mathtt{MotionSight}$ is an untrained zero-shot enhancement scheme, while $\mathtt{MotionChat}$ is trained on the $\mathtt{MotionVid-QA}$ dataset.

Figure: Table 3: Quantitative results on FAVOR-Bench. We selected representative MLLMs as baselines for comparison. We computed the OM (object motion) metric by averaging all metrics excluding the CM (camera motion) metric in FAVOR-Bench.

#### Quantitative evaluation.

Table [2](https://arxiv.org/html/2506.01674v2#S5.T2) and Table [5.1](https://arxiv.org/html/2506.01674v2#S5.SS1.SSS0.Px1) presents the quantitative results on MotionBench and FAVOR-Bench, respectively. Our $\mathtt{MotionSight}$ consistently enhances the performance of base MLLMs. When using Qwen2.5VL as the backbone, our method achieves a 3.4% improvement on MotionBench and a 3.0% improvement on FAVOR-Bench in category average (AVG.), while camera motion improves by 14.3% on MotionBench. Effective performance improvements were also observed in metrics tightly coupled with object motion, including MR, LM, MO and AO. Furthermore, InternVL3-78B augmented with $\mathtt{MotionSight}$ demonstrates a significant performance enhancement, achieving state-of-the-art results among open-source models like TE Fusion (Hong et al., 2025) and exhibits strong competitiveness against leading proprietary models like GLM-4V-Plus-0111 (ZhipuAI, 2025).

#### Qualitative Evaluation.

As shown in Figures [7](https://arxiv.org/html/2506.01674v2#S5.F7) and [9](https://arxiv.org/html/2506.01674v2#S5.F9), the visual spotlight of $\mathtt{MotionSight}$ enhances the model’s ability to focus more on motion information, while the motion blur method significantly enables the model to perceive changes in camera motion in videos. Our approach to decoupling object motion and camera motion leads to a significant improvement in accuracy. Figure [6](https://arxiv.org/html/2506.01674v2#S3.F6) presents our dataset, which we believe will make substantial contributions to the research community.

Figure: Table 4: Quantitative results for $\mathtt{MotionChat}$ based on Qwen2.5VL-7B across different training stages. We analyzed the impact of different training strategies by selectively including or excluding SFT and preference datasets on FAVOR-Bench. Green areas indicate best performance. “✔” indicates the presence of a training stage, while “✘” indicates its absence.

Figure: Figure 7: The difference between using visual spotlight and the original MLLM. We used Grad-CAM and selected the same layer for gradient computation. After incorporating the visual spotlight, the model pays more attention to the core region. Prompt: “What are the people doing?”.
Refer to caption: x6.png

#### MotionChat.

We conducted a fine-tuning experiment on our self-built $\mathtt{MotionVid-QA}$ dataset to demonstrate its core effectiveness. Based on Qwen2.5VL-7B, we first performed a full-parameter SFT training, followed by DPO training on the SFT model, with the vision module frozen during DPO training. As shown in Table [5.1](https://arxiv.org/html/2506.01674v2#S5.SS1.SSS0.Px3), after the two-stage fine-tuning on $\mathtt{MotionVid-QA}$, our $\mathtt{MotionChat}$ significantly outperformed the original model. Furthermore, it achieved an overall accuracy of 48.3% on FAVOR-Bench, which is comparable to the motion understanding ability of Qwen2.5VL-72B.

**Table 5: Evaluation on VideoMME. We present the core general-purpose tasks.**
| Model | Overall | Temporal | Action | Object | Temporal | Spatial | Action | Information |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Perception | Recognition | Recognition | Reasoning | Reasoning | Reasoning | Synopsis |
| Qwen2.5VL-7B | 73.7% | 83.3% | 77.1% | 74.4% | 76.9% | 74.1% | 68.1% | 86.6% |
| +Ours | 76.0% | 88.9% | 80.2% | 76.2% | 84.6% | 77.8% | 70.2% | 87.8% |

Figure: Figure 8: We compiled the metrics related to object motion, camera motion, and other motion from the benchmark and averaged the experimental model and task metrics. Other motion refers to tasks with low correlation to object motion, for which we also used the visual spotlight to focus on core regions. Our method shows significant advantages after decoupling motion.
Refer to caption: x7.png

#### General Video Understanding.

To verify the broader impact of our approach, we tested VideoMME. The results, shown in Table [5](https://arxiv.org/html/2506.01674v2#S5.T5) and Figure [8](https://arxiv.org/html/2506.01674v2#S5.F8), indicate that $\mathtt{MotionSight}$ improves performance on several general-purpose tasks, particularly those related to object motion, and does not exhibit significant global information loss. Our method outperforms the baseline on most metrics and achieves performance comparable to the LLaVA-OneVision-72B model. We attribute this enhancement to the visual spotlight, which helps the model focus on task-relevant regions, thereby yielding consistent performance gains.
For more general video understanding experiments, please refer to Section [B](https://arxiv.org/html/2506.01674v2#A2).

**Table 6: Experiments of several visual prompt methods specialized for motion decoupling on MotionBench. Green areas indicate best performance and red areas show lowest scores.**
| Method | Object Motion | Camera Motion |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| OM AVG. | MR | LM | MO | AO | RC | CM AVG. |  |
| Qwen2.5VL-7B (Bai et al., 2025) | 51.7 | 58.3 | 55.3 | 71.5 | 39.5 | 34.0 | 34.0 |
| + Visual Spotlight | 53.0 | 59.7 | 58.1 | 73.6 | 40.1 | 33.5 | - |
| + Object Crop | 52.5 | 59.0 | 55.9 | 71.3 | 40.1 | 36.5 | - |
| + Background Blur (Yang et al., 2023b) | 49.3 | 56.2 | 53.9 | 65.4 | 39.9 | 31.3 | - |
| + Object Motion Blur | 50.2 | 60.2 | 52.5 | 68.8 | 40.4 | 29.3 | - |
| + Pose Estimation | 50.6 | 56.4 | 54.6 | 70.1 | 40.1 | 32.0 | - |
| + Global Motion Blur | 50.8 | 57.2 | 57.0 | 69.1 | 39.3 | 31.5 | 48.3 |

Figure: Figure 9: Qualitative example of $\mathtt{MotionSight}$. The upper panel represents the baseline, while the lower panel shows the result enhanced by our method, which possesses fine-grained inter-frame difference perception, enabling precise capture of subtle motions.
Refer to caption: x8.png

### 5.2 Ablation Study

#### Object motion understanding.

We evaluate visual prompting strategies to improve object motion understanding.
As detailed in Table [6](https://arxiv.org/html/2506.01674v2#S5.T6), our proposed visual spotlight yields the highest average object motion score (OM AVG.).
Background blur, however, negatively impacted performance, contrasting its effectiveness in static image prompting (Yang et al., 2023b).
We attribute this failure to blurred object boundaries, increasing the demand for robustness and misleading MLLMs.
Other visual prompts, including object crop, object motion blur (applied solely to the object mask) and pose estimation (applied to the entire video), also provided marginal or negative impacts.
These findings underscore the efficacy of visual spotlight in directing model attention to pertinent object movements.

#### Camera motion understanding.

For camera motion, our primary evaluation focused on applying motion blur to the entire video frame, also referred to as global motion blur. Table [6](https://arxiv.org/html/2506.01674v2#S5.T6) shows that our motion blur synthesis approach facilitates the model’s perception of subtle inter-frame differences, thereby yielding a substantial improvement, significantly outperforming the baseline. The results also demonstrate the effectiveness of our decoupled object and camera motion method. Please refer to Section [B](https://arxiv.org/html/2506.01674v2#A2) in the appendix for more ablation studies.

### 5.3 More Applications

The $\mathtt{MotionVid-QA}$ can not only be used for MLLM fine-tuning but also support video generation, enhancing motion quality in generated videos. We constructed video-text pairs from the large-scale, high-quality data in the SFT subset, and conducted fine-tuning on Wan2.1-T2V-1.3B, followed by evaluation on VBench. As shown in Table [7](https://arxiv.org/html/2506.01674v2#S5.T7) and Figure [10](https://arxiv.org/html/2506.01674v2#S5.F10), after our efficient fine-tuning, the model achieves significant improvements in motion-related generation, which can be attributed to the high quality and rich fine-grained motion content of our dataset.

**Table 7: Fine-tuning results on $\mathtt{MotionVid-QA}$ for video generation.**
|  | Human Action | Motion Smoothness | Dynamic Degree | Temporal Style |
| --- | --- | --- | --- | --- |
| Wan2.1-T2V-1.3B | 64.2 | 97.9 | 47.7 | 20.4 |
| + $\mathtt{MotionVid-QA}$ | 71.0 | 98.3 | 54.3 | 21.5 |

Figure: Figure 10: Qualitative example of our fine-tuned Wan2.1-T2V-1.3B.
Refer to caption: x9.png

## 6 Conclusions

In this work, we address the challenge of fine-grained video motion understanding, a task that has been relatively underexplored despite the advancements in MLLMs. We introduce $\mathtt{MotionSight}$, a novel zero-shot approach that decouples object motion from camera motion. By employing visual spotlight and motion blur, it enhances the ability of MLLMs to perceive subtle motion cues. Furthermore, we curate $\mathtt{MotionVid-QA}$, the first large-scale open-source dataset designed for fine-grained video motion understanding. Our experiments demonstrate the effectiveness of $\mathtt{MotionSight}$ and $\mathtt{MotionVid-QA}$ to facilitate future research in this domain.

## References

- Anthropic (2025)
Anthropic.
Claude 3.7.
[https://www.anthropic.com/claude/sonnet](https://www.anthropic.com/claude/sonnet), February 2025.
- Bai et al. (2025)
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al.
Qwen2. 5-vl technical report.
*arXiv preprint arXiv:2502.13923*, 2025.
- Caba Heilbron et al. (2015)
Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles.
Activitynet: A large-scale video benchmark for human activity understanding.
In *Proceedings of the ieee conference on computer vision and pattern recognition*, pp. 961–970, 2015.
- Carreira et al. (2019)
Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zisserman.
A short note on the kinetics-700 human action dataset.
*arXiv preprint arXiv:1907.06987*, 2019.
- Chen et al. (2024a)
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, Li Yuan, Yu Qiao, Dahua Lin, Feng Zhao, and Jiaqi Wang.
Sharegpt4video: Improving video understanding and generation with better captions.
In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), *Advances in Neural Information Processing Systems*, volume 37, pp. 19472–19495. Curran Associates, Inc., 2024a.
URL [https://proceedings.neurips.cc/paper_files/paper/2024/file/22a7476e4fd36818777c47e666f61a41-Paper-Datasets_and_Benchmarks_Track.pdf](https://proceedings.neurips.cc/paper_files/paper/2024/file/22a7476e4fd36818777c47e666f61a41-Paper-Datasets_and_Benchmarks_Track.pdf).
- Chen et al. (2024b)
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al.
Panda-70m: Captioning 70m videos with multiple cross-modality teachers.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 13320–13331, 2024b.
- Chen et al. (2024c)
Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou.
Halc: Object hallucination reduction via adaptive focal-contrast decoding.
*arXiv preprint arXiv:2403.00425*, 2024c.
- Chen et al. (2024d)
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al.
Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 24185–24198, 2024d.
- DeepMind (2025)
Google DeepMind.
Gemini: Our most intelligent ai models, built for the agentic era.
[https://deepmind.google/technologies/gemini/](https://deepmind.google/technologies/gemini/), 2025.
- Fan et al. (2024a)
Tiehan Fan, Kepan Nan, Rui Xie, Penghao Zhou, Zhenheng Yang, Chaoyou Fu, Xiang Li, Jian Yang, and Ying Tai.
Instancecap: Improving text-to-video generation via instance-aware structured caption.
*arXiv preprint arXiv:2412.09283*, 2024a.
- Fan et al. (2024b)
Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li.
Videoagent: A memory-augmented multimodal agent for video understanding.
In *European Conference on Computer Vision*, pp. 75–92. Springer, 2024b.
- Hong et al. (2024)
Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, et al.
Cogvlm2: Visual language models for image and video understanding.
*arXiv preprint arXiv:2408.16500*, 2024.
- Hong et al. (2025)
Wenyi Hong, Yean Cheng, Zhuoyi Yang, Weihan Wang, Lefan Wang, Xiaotao Gu, Shiyu Huang, Yuxiao Dong, and Jie Tang.
Motionbench: Benchmarking and improving fine-grained video motion understanding for vision language models.
*arXiv preprint arXiv:2501.02955*, 2025.
- Hurst et al. (2024)
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al.
Gpt-4o system card.
*arXiv preprint arXiv:2410.21276*, 2024.
- Ju et al. (2024)
Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan.
Miradata: A large-scale video dataset with long durations and structured captions.
*Advances in Neural Information Processing Systems*, 37:48955–48970, 2024.
- Kuehne et al. (2011)
Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre.
Hmdb: a large video database for human motion recognition.
In *2011 International conference on computer vision*, pp. 2556–2563. IEEE, 2011.
- Li et al. (2024)
Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al.
Videochat-flash: Hierarchical compression for long-context video modeling.
*arXiv preprint arXiv:2501.00574*, 2024.
- Lin et al. (2024)
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan.
Evaluating text-to-visual generation with image-to-text generation.
*arXiv preprint arXiv:2404.01291*, 2024.
- Liu et al. (2024a)
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee.
Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a.
URL [https://llava-vl.github.io/blog/2024-01-30-llava-next/](https://llava-vl.github.io/blog/2024-01-30-llava-next/).
- Liu et al. (2024b)
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al.
Grounding dino: Marrying dino with grounded pre-training for open-set object detection.
In *European Conference on Computer Vision*, pp. 38–55. Springer, 2024b.
- Liu et al. (2024c)
Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao.
Oryx mllm: On-demand spatial-temporal understanding at arbitrary resolution.
*arXiv preprint arXiv:2409.12961*, 2024c.
- Nan et al. (2024)
Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai.
Openvid-1m: A large-scale high-quality dataset for text-to-video generation.
*arXiv preprint arXiv:2407.02371*, 2024.
- Ravi et al. (2024)
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al.
Sam 2: Segment anything in images and videos.
*arXiv preprint arXiv:2408.00714*, 2024.
- Shtedritski et al. (2023)
Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi.
What does clip know about a red circle? visual prompt engineering for vlms.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pp. 11987–11997, 2023.
- Sigurdsson et al. (2016)
Gunnar A Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta.
Hollywood in homes: Crowdsourcing data collection for activity understanding.
In *Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14*, pp. 510–526. Springer, 2016.
- Sigurdsson et al. (2018)
Gunnar A Sigurdsson, Abhinav Gupta, Cordelia Schmid, Ali Farhadi, and Karteek Alahari.
Charades-ego: A large-scale dataset of paired third and first person videos.
*arXiv preprint arXiv:1804.09626*, 2018.
- Soomro et al. (2012a)
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah.
Ucf101: A dataset of 101 human actions classes from videos in the wild.
*arXiv preprint arXiv:1212.0402*, 2012a.
- Soomro et al. (2012b)
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah.
Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012b.
URL [https://arxiv.org/abs/1212.0402](https://arxiv.org/abs/1212.0402).
- Team et al. (2024)
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.
*arXiv preprint arXiv:2403.05530*, 2024.
- Tu et al. (2025)
Chongjun Tu, Lin Zhang, Pengtao Chen, Peng Ye, Xianfang Zeng, Wei Cheng, Gang Yu, and Tao Chen.
Favor-bench: A comprehensive benchmark for fine-grained video motion understanding.
*arXiv preprint arXiv:2503.14935*, 2025.
- Wang et al. (2024a)
Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun.
Tarsier: Recipes for training and evaluating large video description models.
*arXiv preprint arXiv:2407.00634*, 2024a.
- Wang et al. (2024b)
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al.
Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.
*arXiv preprint arXiv:2409.12191*, 2024b.
- Wang et al. (2023)
Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al.
Internvid: A large-scale video-text dataset for multimodal understanding and generation.
*arXiv preprint arXiv:2307.06942*, 2023.
- Wang et al. (2024c)
Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al.
Internvideo2: Scaling foundation models for multimodal video understanding.
In *European Conference on Computer Vision*, pp. 396–416. Springer, 2024c.
- Wang et al. (2025a)
Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al.
Internvideo2. 5: Empowering video mllms with long and rich context modeling.
*arXiv preprint arXiv:2501.12386*, 2025a.
- Wang et al. (2025b)
Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal.
Videotree: Adaptive tree-based video representation for llm reasoning on long videos.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 3272–3283, 2025b.
- Wu et al. (2025)
Shengqiong Wu, Weicai Ye, Jiahao Wang, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, Shuicheng Yan, Hao Fei, et al.
Any2caption: Interpreting any condition to caption for controllable video generation.
*arXiv preprint arXiv:2503.24379*, 2025.
- Xu et al. (2024)
Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng.
Pllava: Parameter-free llava extension from images to videos for video dense captioning.
*arXiv preprint arXiv:2404.16994*, 2024.
- Yang et al. (2024)
Dongjie Yang, Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao.
Vript: A video is worth thousands of words.
*Advances in Neural Information Processing Systems*, 37:57240–57261, 2024.
- Yang et al. (2023a)
Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao.
Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v.
*arXiv preprint arXiv:2310.11441*, 2023a.
- Yang et al. (2023b)
Lingfeng Yang, Yueze Wang, Xiang Li, Xinlong Wang, and Jian Yang.
Fine-grained visual prompting.
*Advances in Neural Information Processing Systems*, 36:24993–25006, 2023b.
- Yao et al. (2022)
Yuan Yao, Ao Zhang, Zhengyan Zhang, Zhiyuan Liu, Tat-Seng Chua, and Maosong Sun.
Cpt: Colorful prompt tuning for pre-trained vision-language models, 2022.
URL [https://arxiv.org/abs/2109.11797](https://arxiv.org/abs/2109.11797).
- Yao et al. (2024)
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al.
Minicpm-v: A gpt-4v level mllm on your phone.
*arXiv preprint arXiv:2408.01800*, 2024.
- Yuan et al. (2025)
Liping Yuan, Jiawei Wang, Haomiao Sun, Yuchen Zhang, and Yuan Lin.
Tarsier2: Advancing large vision-language models from detailed video description to comprehensive video understanding, 2025.
URL [https://arxiv.org/abs/2501.07888](https://arxiv.org/abs/2501.07888).
- Zhang et al. (2025)
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al.
Videollama 3: Frontier multimodal foundation models for image and video understanding.
*arXiv preprint arXiv:2501.13106*, 2025.
- Zhang et al. (2024a)
Y Zhang, B Li, H Liu, Y Lee, L Gui, D Fu, J Feng, Z Liu, and C Li.
Llava-next: A strong zero-shot video understanding model.
2024a.
- Zhang et al. (2024b)
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li.
Video instruction tuning with synthetic data.
*arXiv preprint arXiv:2410.02713*, 2024b.
- ZhipuAI (2025)
ZhipuAI.
Glm-4v.
[https://bigmodel.cn/dev/howuse/glm-4v](https://bigmodel.cn/dev/howuse/glm-4v), 2025.
- Zhu et al. (2025a)
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al.
Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.
*arXiv preprint arXiv:2504.10479*, 2025a.
- Zhu et al. (2025b)
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang, Jiapeng Luo, Yi Wang, Conghui He, Botian Shi, Xingcheng Zhang, Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu, Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang.
Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models, 2025b.
URL [https://arxiv.org/abs/2504.10479](https://arxiv.org/abs/2504.10479).

## Appendix A Implementation Details

### A.1 Experimental Settings

We performed all experiments of $\mathtt{MotionSight}$ on 8 NVIDIA 4090 GPUs with 48GB of memory each. For GroundingDINO, the box threshold and text threshold for post-processing grounded object detection are both set to 0.25. For SAM 2, the mask dictionary model uses an IOU threshold of 0.8 for updating masks. In motion blur part we use decay factor $\gamma=0.65$ and temporal window $N=7$. For action focusing we use darken factor $\beta=0.9$. To address potential detection loss due to frequent object entry and exit (e.g., FAVOR-Bench Tu et al. (2025)), we perform re-detection using $\mathcal{M}_{detect}$ at fixed intervals $\Delta t$, ensuring comprehensive object capture. The weighted mean (temporal kernel) in Sec [3.3](https://arxiv.org/html/2506.01674v2#S3.SS3) is implemented as follows:

$$ $w_{N-1-k}(\gamma)=\gamma^{k}\cdot\prod_{j=k+1}^{N-1}(1-\gamma^{j})$ (7) $$

For the fine-tuning experiments, we utilized 32 A100 GPUs with 80GB of memory each. The SFT process was conducted on the Qwen2.5-VL-7B-Instruct model, employing a global batch size of 128. During SFT, the vision tower, LLM, and merger components were all trainable. During DPO training, we exclusively trained the LLM part.

### A.2 Prompt Template

We present our prompt details. For the Object Referring stage, we use the prompt shown in the upper image to enable the MLLM to locate the most critical objects based on the video and question content. We utilize the concept of action groups , allowing the MLLM to identify relevant objects—and even their components—in a fine-grained manner.

The code below defines a key part of our final configuration template. Based on enhanced inputs such as Action focus or motion blur, we route the input into the MLLM using a routing mechanism. In this template, description_for_video_type refers to descriptions tailored for different video types.

video_descriptions = {
’original’: ’Original video:\n’,
’spotlight’: ’Spotlight video:\n’,
{internallinenumbers*}
’motion_blur’: ’Original video with motion blur to more clearly determine the type of motion (such as whether the camera is moving, as one frame combines information from multiple frames. If static objects in the background appear noticeably blurry, there is a good chance that the camera is moving!):\n’
}

## Appendix B More Experiments

### B.1 Applications to general video understanding

We conducted additional evaluations with Qwen2.5VL-7B being the backbone on the TempCompass benchmark to prove the general effectiveness of $\mathtt{MotionSight}$ and impact of global information.

**Table 8: Evaluation results on the main TempCompass benchmark.**
| Model | Avg | Action | Attribute Change | Direction | Order | Speed |
| --- | --- | --- | --- | --- | --- | --- |
| Qwen2.5VL-7B | 68.50 | 94.40 | 76.20 | 50.42 | 71.62 | 50.65 |
| +Ours | 71.32 | 94.72 | 79.03 | 52.55 | 74.47 | 58.40 |

**Table 9: Performance on the captioning subset of TempCompass.**
| Model | Avg | Action | Attribute Change | Direction | Order | Speed |
| --- | --- | --- | --- | --- | --- | --- |
| Qwen2.5VL-7B | 54.6 | 92.7 | 68.5 | 32.8 | 51.2 | 29.9 |
| +Ours | 64.0 | 93.2 | 77.1 | 38.1 | 67.0 | 47.4 |

**Table 10: Breakdown on TempCompass’s fine-grained captioning sub-dimensions. AS: Absolute Speed; CM: Camera Motion; CGA: Coarse-grained Action; CLC: Color & Light Change; CC: Combined Change; FGA: Fine-grained Action; OC: Other Change; RS: Relative Speed; SSC: Size & Shape Change.**
| Model | AS | CM | CGA | CLC | CC | FGA | Other | OC | RS | SSC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Qwen2.5VL-7B | 43.9 | 15.0 | 95.1 | 76.9 | 70.0 | 90.1 | 51.2 | 66.7 | 22.7 | 63.1 |
| + $\mathtt{MotionSight}$ | 62.9 | 31.7 | 95.6 | 87.5 | 75.0 | 90.6 | 67.0 | 75.0 | 39.5 | 72.2 |

For TempCompass (as shown in Table [10](https://arxiv.org/html/2506.01674v2#A2.T10), Table [8](https://arxiv.org/html/2506.01674v2#A2.T8), Table [9](https://arxiv.org/html/2506.01674v2#A2.T9)), we compared our method with Qwen2.5VL-7B under the same settings, using “gpt-4o-mini-2024-07-18” as the LLM evaluator for TempCompass. The results show that our method significantly outperforms the baseline on TempCompass, demonstrating strong advantages even in more general temporal understanding tasks. Notably, we observed that our method achieves far superior performance on subtasks with higher output freedom, such as captioning. This reflects a deeper enhancement in motion understanding: objective questions are essentially simple discriminative tasks, often requiring only the judgment of isolated motion facts. In contrast, captioning demands more profound and fine-grained descriptions, which aligns well with the core design of $\mathtt{MotionSight}$ for refined motion understanding. In summary, our approach has greater universality and potential.

### B.2 More Ablation Studies on Robustness

To clarify that our method demonstrates strong robustness to hyperparameters, we conducted ablation studies on essential parameters: the degree of background darkening in the bounding-box-based spotlight (from slight dimming to completely black background), temporal window size, and the decay factor discussed in Section [3.3](https://arxiv.org/html/2506.01674v2#S3.SS3).

To evaluate the robustness of background darkening in the visual spotlight, we performed denser sampling within the selected parameter range from 0.1 to 1.0. The results are shown in Table [12](https://arxiv.org/html/2506.01674v2#A2.T12). We perform separate analyses for the temporal window size and the decay factor to assess their individual robustness on model performance, as shown in Table [13](https://arxiv.org/html/2506.01674v2#A2.T13) and Table [14](https://arxiv.org/html/2506.01674v2#A2.T14).

Overall, the ablation results show that our method significantly outperforms the baseline under different parameter settings. Thus, our method $\mathtt{MotionSight}$ has strong robustness. We supplemented more visualizations of $\mathtt{MotionSight}$, as shown in the Figure [13](https://arxiv.org/html/2506.01674v2#A2.F13).

Figure: Figure 11: Ablation study on model scalability.
Refer to caption: x10.png

**Table 12: Ablation study on the degree of background darkening.**
| Darken | Baseline | 0.1 | 0.5 | 0.8 | 0.85 | 0.9 | 0.95 | 1 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| OM AVG. | 51.7 | 52.5 | 52.5 | 51.9 | 52.1 | 53.0 | 52.1 | 52.0 |

**Table 13: Ablation study on temporal window size ($N$ in Equation [5](https://arxiv.org/html/2506.01674v2#S3.E5)), with fixed decay factor: 0.65.**
| Temporal Window | Baseline | 5 | 7 | 9 | 11 |
| --- | --- | --- | --- | --- | --- |
| CM | 34.0 | 48.1 | 48.3 | 46.0 | 41.0 |

**Table 14: Ablation study on the decay factor ($\gamma$ in Equation [5](https://arxiv.org/html/2506.01674v2#S3.E5)), with fixed temporal window: 7.**
| Decay Factor | Baseline | 0.4 | 0.65 | 0.85 |
| --- | --- | --- | --- | --- |
| CM | 34.0 | 46.2 | 48.3 | 45.2 |

### B.3 Scalability of 𝙼𝚘𝚝𝚒𝚘𝚗𝚂𝚒𝚐𝚑𝚝 \mathtt{MotionSight}

We conducted experiments based on InternVL3, ranging from 1B to 78B parameters, demonstrating that our method exhibits good scalability across popular model sizes. The results on MotionBench are shown in Figure [11](https://arxiv.org/html/2506.01674v2#A2.F11). The lighter portion within each bar represents the performance improvement of our method over the baseline model.

### B.4 Qualitative results of 𝙼𝚘𝚝𝚒𝚘𝚗𝙲𝚑𝚊𝚝 \mathtt{MotionChat} on the Dataset

Figure: Figure 12: Quantitative results between baseline and our fine-tuned models trained using both SFT and DPO. For each case, the upper one is the Qwen2.5VL-7B baseline, and the lower one is our model after fine-tuning.
Refer to caption: x11.png

Figure: Figure 13: Another qualitative example of $\mathtt{MotionSight}$.
Refer to caption: x12.png

As shown in Figure [12](https://arxiv.org/html/2506.01674v2#A2.F12), $\mathtt{MotionChat}$ exhibits enhanced fine-grained motion perception, more accurately interpreting complex motion narratives compared to the baseline model. Our fine-tuned model, trained on our dataset, demonstrates superior fine-grained motion perception capabilities and outperforms the baseline.

## Appendix C Visual Spotlight’s Motivation

Visual spotlight is designed to help the model better focus on the most crucial regions of temporal changes when processing video frames. In scenarios like TV shows and movies, important moving subjects are often highlighted while the background is dimmed. Additionally, some performance shows often use techniques such as stage spotlights. MLLMs inevitably encounter such data during training (e.g., datasets like Panda-70M, which contain a large amount of TV program content). Also, using the visual spotlight method can indirectly help MLLMs concentrate their visual attention on core motion regions, intuitively and effectively improving the model’s ability to capture motion information.

Figure: Figure 14: Another Grad-CAM visualization of our visual spotlight method after vision module. Despite using a smaller darken factor, the model can still provide more attention to moving objects.
Refer to caption: x13.png

Figure [7](https://arxiv.org/html/2506.01674v2#S5.F7) and Figure [14](https://arxiv.org/html/2506.01674v2#A3.F14) shows the Grad-CAM visualization results after applying our visual spotlight method to Qwen2.5VL-7B. The first figure uses a higher darken factor, while the second figure uses a slightly lower one. As can be seen, our method allows the model to focus more on the motion-related visual parts relevant to the question across different darken factors, thereby improving its fine-grained motion understanding capabilities.

Furthermore, we have implemented a comparative experiment where bounding box coordinates are directly provided as prompts, as shown in Table [11](https://arxiv.org/html/2506.01674v2#A2.F11). The results show that although providing bounding box coordinates is also an intuitive prompting method, it does not lead to noticeable performance improvement. This is because it is challenging for the MLLMs to precisely apply the coordinate information to the video, and coordinates require significantly longer text context, demanding higher parsing capabilities from the model. This phenomenon further validates the reasonableness and effectiveness of our method in terms of engineering implementation and model adaptation.

## Appendix D Discussions on computational overhead

We acknowledge that performance improvements often come with a trade-off in time or resources. This is a common engineering challenge for many current vision-prompting enhancements and workflows, such as InstanceCap (Fan et al., 2024a) [CVPR’25], VideoTree (Wang et al., 2025b) [CVPR’25], and VideoAgent (Fan et al., 2024b) [ECCV’24].

To address this, we propose an alternative solution that achieves superior performance on fine-grained motion understanding tasks with a computational cost equivalent to the native Qwen2.5VL-7B model. This solution involves fine-tuning the model on our meticulously curated dataset. Meanwhile, we have optimized the $\mathtt{MotionSight}$ pipeline, reducing the average increase in inference latency to less than 75%. Our overall inference time overhead is relatively lightweight compared to other workflows.

In summary, for users who seek the highest performance, we’ve optimized the $\mathtt{MotionSight}$ pipeline, which is a specific and valuable workflow improvement; for users who prioritize efficiency, we offer $\mathtt{MotionChat}$—an end-to-end model fine-tuned on a new dataset, with a computational cost comparable to the base model.

## Appendix E Dataset Filter and Curation for 𝙼𝚘𝚝𝚒𝚘𝚗𝚅𝚒𝚍 − 𝚀𝙰 \mathtt{MotionVid-QA}

In this section, we focus on the methodological aspects and specific steps involved in filtering and curating the instruction and preference subsets for SFT and DPO, providing supplementary details to the dataset construction process of $\mathtt{MotionVid-QA}$ described in Section [4](https://arxiv.org/html/2506.01674v2#S4). Figure [15](https://arxiv.org/html/2506.01674v2#A5.F15) showcases the examples we filtered.

Figure: Figure 15: More examples of $\mathtt{MotionVid-QA}$. We filtered out low-quality videos and kept the high-quality ones. Our $\mathtt{MotionVid-QA}$ includes varied scenes, subjects, and camera perspectives.
Refer to caption: figure/chart3.png

### E.1 Initial Data Collection and Pre-processing

$\mathtt{MotionVid-QA}$ was aggregated from multiple sources (Section [4](https://arxiv.org/html/2506.01674v2#S4)), resulting in an initial set of video clips, denoted as $C_{R}$.
These clips underwent an initial data processing pipeline, $\mathcal{P}_{initial}$. The pipeline $\mathcal{P}_{initial}$ filters the raw clips $C_{R}$ such that only clips satisfying specific quality metrics are retained. Specifically, a clip $c\in C_{R}$ is included in the pre-filtered set $C_{P}$ if its optical flow score $s_{f}(c)$ is above a threshold $\tau_{f}$ AND its clarity score $s_{c}(c)$ is above a threshold $\tau_{c}$.
The set of pre-filtered clips is thus defined as:

$$ $C_{P}=\{c\in C_{R}\mid s_{f}(c)>\tau_{f}\land s_{c}(c)>\tau_{c}\}$ (8) $$

### E.2 Detailed Filtering and Subset Creation for SFT and DPO

The methodology for creating SFT and DPO subsets from the pre-filtered set $C_{P}$ is introduced in Section [4](https://arxiv.org/html/2506.01674v2#S4) of the main paper.

#### Initial selection and annotation.

A subset of clips was chosen from $C_{P}$ for annotation with $\mathtt{MotionSight}$. Let $C_{A}$ denote the set of successfully annotated clips.

#### Annotation quality-based categorization.

Each clip $c\in C_{A}$ was evaluated using VQAScore Lin et al. (2024). To ensure the rationality of the chosen VQAScore thresholds for categorization (detailed below and in Table [15](https://arxiv.org/html/2506.01674v2#A5.T15)), we manually checked multiple samples at the boundaries of these thresholds. Based on this evaluation, the clips were categorized into three distinct groups and the text-video visualization results satisfying different threshold conditions are shown in Figure [16](https://arxiv.org/html/2506.01674v2#A5.F16).

- •
High-Quality Clips ($C_{H}$): These clips were designated as high-quality and served as candidates for the DPO dataset. To ensure scenario diversity and account for varying annotation precision across different original data sources, clips were selected if their VQAScore Lin et al. (2024) exceeded a specific threshold $\tau_{j,k}$ defined for its original data source $j$ and motion aspect $k$ (e.g., $k\in\{\text{'object', 'camera'}\}$). These VQAScore thresholds $\tau_{j,k}$ are detailed in Table [15](https://arxiv.org/html/2506.01674v2#A5.T15). The set $C_{H}$ is formally defined as:
$C_{H}=\{c\in C_{A}\mid\text{ExceedsVQAScoreThreshold}(c)\}$
(9)
where $\text{ExceedsVQAScoreThreshold}(c)$ holds if the VQAScore of clip $c$ (from source $j$, for aspect $k$) is greater than the specific threshold $\tau_{j,k}$ for that source-aspect pair, as given in Table [15](https://arxiv.org/html/2506.01674v2#A5.T15).
- •
Low-Quality Clips ($C_{L}$): Clips that failed to meet the minimum quality criteria were eliminated. This includes clips whose VQAScore was below a human-set minimum threshold $\tau_{vL}=0.3$. The set $C_{L}$ is defined as:
$C_{L}=\{c\in C_{A}\mid\text{VQAScore}(c)<0.3\}$
(10)
- •
Instruction Dataset Clips ($C_{S}$): The remaining clips formed the SFT instruction dataset. This set is defined as:
$C_{S}=C_{A}\setminus(C_{H}\cup C_{L})$
(11)

#### SFT dataset construction and question types.

The SFT dataset is constructed using the Instruction Dataset Clips ($C_{S}$). For each clip in $C_{S}$, we generate a question-answer pair to fine-tune the model’s ability to understand and describe motion. To cover diverse aspects of motion understanding, we categorize our questions into three types: Object-centric, Camera-centric, and Mixed-focus. During the SFT data generation, one question is randomly selected from the pool of questions corresponding to the primary motion aspect (object, camera, or mixed) identified in the clip’s annotation. These SFT dialogues (question-answer pairs) are crucial as they also form the foundation for constructing the preference data for DPO.

Figure: Figure 16: The results corresponding to our three different thresholds are presented separately. Top: High consistency between text and video. The camera movement and changes in viewpoint are strictly described in chronological order, resulting in extremely high quality. Middle: Fairly good consistency between text and video. The actions of the main characters are described with reasonable accuracy, but some imprecise areas exist. Bottom: Relatively poor consistency between text and video, providing limited or erroneous information.
Refer to caption: x14.png

Figure: Figure 17: Interactive annotation interface for DPO focused on fine-grained video motion understanding. This Python-based front-end allows annotators to choose between two textual descriptions (“Option A” and “Option B”) for the same video clip, selecting the one that more accurately captures the nuanced motion in the video. The interface supports loading data in JSONL format and records annotator preferences, thereby providing data for the model’s preference learning.
Refer to caption: x15.png

##### Object-centric Questions:

These questions focus on the movement, actions, and interactions of objects within the video. Examples include:

##### Camera-centric Questions:

These questions probe the camera’s movement, techniques, and perspective. Examples include:

##### Mixed-focus Questions:

These questions require a comprehensive understanding of the interplay between object motion and camera work. Examples include:

#### Preference dataset construction.

The preference dataset, consists of preference pairs of the form $(x_{i},y^{chosen}_{i},y^{reject}_{i})$. These pairs were generated from the high-quality clips in $C_{H}$. The process involved re-annotating these clips using Tarsier2 Yuan et al. (2025) and then incorporating human preference signals, as illustrated in Figure [5](https://arxiv.org/html/2506.01674v2#S3.F5) of the main paper and more details in Section [F](https://arxiv.org/html/2506.01674v2#A6).

This rigorous, multi-stage curation methodology ensures the high quality of the $\mathtt{MotionVid-QA}$ subsets, which are crucial for robust model training and evaluation in fine-grained motion understanding.

**Table 15: VQAScore Thresholds ($\tau_{j,k}$) for High-Quality Clip Selection ($C_{H}$), per Source and Motion Aspect. A clip $c$ from source $j$ and aspect $k$ is included in $C_{H}$ if its VQAScore$(c)>\tau_{j,k}$.**
| Data Source ($j$) | Object Motion ($\tau_{j,\text{object}}$) | Camera Motion ($\tau_{j,\text{camera}}$) |
| --- | --- | --- |
| Kinetics-700 | 0.75 | 0.70 |
| ActivityNet | 0.75 | 0.72 |
| Charades | 0.72 | 0.70 |
| Charades-Ego | 0.72 | 0.70 |
| SSV2 | 0.68 | 0.68 |
| OpenVid-1M | 0.70 | 0.70 |

**Table 16: Key criteria for human preference annotation in selecting textual descriptions of fine-grained motion. Annotators chose the description that better satisfied these aspects.**
| Criterion | Guideline for Selection | Key Questions |
| --- | --- | --- |
| 1. Accuracy | Prefer more accurate identification & description of primary motion(s). | - Core action correctly identified?<br>- Agents/objects in motion correct?<br>- Avoids misinterpreting actions? |
| 2. Granularity | Prefer more fine-grained & detailed account of motion, capturing nuances. | - Complex movements broken down?<br>- Specific body/object details?<br>- Overly general or specific? |
| 3. Temporal Dynamics | Prefer better capture of temporal aspects (sequence, duration, speed, rhythm). | - Sub-actions order correct?<br>- Pace/intensity conveyed?<br>- Speed/tempo changes reflected? |
| 4. Camera Movement | Prefer description that accurately identifies & describes significant camera movements (e.g., pan, tilt, zoom, tracking). | - Camera movement (pan, tilt, zoom, dolly, static) correctly identified?<br>- Effect of camera movement on scene understanding clear?<br>- Distinguished from object motion? |
| 5. Factual Correctness | Prefer response factually grounded in visual evidence, no hallucinations. | - Only visible elements/actions?<br>- Contradicts visual information?<br>- Infers unobservable intent? |

## Appendix F Guidelines for Human Preference Annotation

For DPO, annotators chose between two textual descriptions for a video clip, selecting the one that better captured its fine-grained motion. To ensure fairness, the order in which these two descriptions were presented was randomized. The following guidelines ensured consistent, high-quality annotations: To facilitate this process, we developed an interactive, python-based front-end for user-friendly annotation, as shown in Figure [17](https://arxiv.org/html/2506.01674v2#A5.F17). The relevant code can be found in the supplementary materials.

Annotators selected the preferred response based on a holistic evaluation of the criteria in Table [16](https://arxiv.org/html/2506.01674v2#A5.T16), prioritizing superior understanding and articulation of fine-grained motion.

Annotators were advised to review clips multiple times and compare descriptions against these criteria. When responses excelled in different areas, they selected the one most helpful for understanding fine-grained motion.

## Appendix G Limitations

Given that the accuracy of $\mathtt{MotionSight}$ is partially dependent on object detection methods, it requires fine-tuning the detection model for domain-specific instances, and its benefits diminish in scenes without objects. Furthermore, we plan to expand our $\mathtt{MotionVid-QA}$ and improve its quality to train more powerful fine-grained motion understanding models to enhance its impact.

## Appendix H LLMs Usage

LLMs were used as a general-purpose assist tool to refine the writing of this paper. This usage was strictly limited to improving the clarity, grammar, and overall readability of the text. Specifically, LLMs were employed to rephrase sentences, suggest alternative wording, and correct grammatical errors.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: Just curious whether the authors tried applying MotionSight on top of the fine-tuned MotionChat model to see if it yields additional improvements. This could establish a potential self-improving VLM pipeline.

---
Overall, I’m borderline on this submission (currently rating it 4, since no borderline is available, and because I still have some concerns and would like to see the authors’ reply and additional results). I’d be happy to update my rating if my questions are well answered.
soundness: 3
strengths: - The paper is well-motivated and clearly written, addressing an important gap in motion understanding for MLLMs.

- The proposed visual prompting (spotlight + motion blur) is intuitive and effective, yielding consistent zero-shot gains.

- The experiments are comprehensive, and the curated MotionVid-QA dataset provides a useful resource for future work.
summary: The paper presents MotionSight, a zero-shot visual prompting framework that enhances MLLMs’ fine-grained motion understanding through two complementary visual cues, Visual Spotlight for object motion and Motion Blur for camera motion.
Building on this, the authors further construct MotionVid-QA, a large-scale video QA dataset automatically annotated by MotionSight and refined via multi-stage filtering and human preference alignment. Experiments demonstrate consistent zero-shot improvements on MotionBench and FAVOR-Bench, and show that fine-tuning on MotionVid-QA (MotionChat) can yield additional gains with competitive efficiency.
weaknesses: - **Limited novelty of Visual Spotlight.**
   The proposed visual spotlight is conceptually similar to existing image-level attention prompting methods (e.g., *Attention Prompting on Image for Large Vision-Language Models*, ECCV 2024), which also use soft masks to highlight salient regions. A comparison with such methods in Table 6 would strengthen the claim of novelty.

- **Potential ambiguity in Motion Blur.**
   Although the temporal weighting distinguishes different frames, the resulting blur may introduce directional ambiguity. For instance, in Figure 4 (bottom right), both lower-left and upper-right motion directions could be plausible, suggesting that motion blur may not always provide a reliable cue.

- **Dataset quality and generalization concerns.**
   MotionVid-QA is generated using MLLM-based annotations (Qwen2.5-VL-7B), meaning its quality and difficulty are inherently tied to the model’s capacity. While filtering ensures correctness, it does not increase complexity, so the dataset might mainly contain simple QA pairs. As a result, MotionVid-QA may improve weaker models but offer limited benefit for stronger ones (e.g., InternVL3-78B). Evaluating SFT effects on multiple models would better validate its generality.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Can the method improve performance on the spatial benchs?
2. Why choose to use the detection model instead of enhancing MLLM detection capability through RL?
soundness: 2
strengths: 1. The paper introduces a novel zero-shot method, MotionSight, which is the first to apply visual prompting techniques specifically tailored for fine-grained video motion understanding. This includes the innovative use of a visual spotlight to highlight moving objects and motion blur to emphasize camera movements, both of which are unique adaptations from static image prompting.
2. The idea of decoupling object and camera motion is novel and addresses a significant gap in current MLLMs, which often struggle with subtle inter-frame dynamics.
summary: The paper introduces a novel zero-shot method called MotionSight to enhance fine-grained motion understanding in MLLMs without requiring additional training. It addresses the limitations of existing MLLMs in perceiving subtle inter-frame dynamics by decoupling object and camera motion using visual prompting techniques. Specifically, it applies a visual spotlight to highlight moving objects and introduces motion blur to emphasize camera movements. To support training and evaluation, the authors present MotionVid-QA, the first large-scale open-source dataset for fine-grained video motion understanding, containing ~40K video clips and ~87K question-answer pairs with hierarchical annotations. Extensive experiments show that MotionSight significantly improves performance on benchmarks like MotionBench and FAVOR-Bench, and when used to fine-tune MotionChat, it achieves competitive results with much larger models.
weaknesses: 1. More Experiments: Can you provide more results on video perception benches, such as mvbench, TOMATO bench, etc.
2. The accuracy of MotionSight is partially dependent on object detection methods. This means that the performance of the model can be significantly affected by the quality of the object detection algorithm used.

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: 1. What is the failure-mode policy when Grounding DINO or SAM2 misses or hallucinates boxes? When a video contains both salient object motion and complex camera motion, why route to a single path instead of feeding both augmentations simultaneously?
2. What is the head-to-head win-rate between MotionSight captions and Tarsier2 captions in the preference round? Please report overall and per-motion-type ratios.
soundness: 4
strengths: 1. Clear problem definition and strong motivation: Addresses a notable shortcoming of MLLMs in fine-grained motion understanding.
2. Simple yet effective method: As a zero-shot approach, MotionSight improves model performance without training, offering broad applicability.
3. Notable dataset contribution: MotionVid-QA is the first large-scale open-source dataset focused on fine-grained motion understanding, with high-quality annotations and significant community value.
summary: This paper proposes MotionSight, a zero-shot approach to enhance fine-grained motion understanding in multimodal large language models (MLLMs). The method decouples object and camera motion, introducing visual spotlight and motion blur as visual prompts to improve perception of action details and camera changes. Additionally, the authors construct the first large-scale open-source dataset, MotionVid-QA, containing 40K videos and 87K Q&A pairs for SFT and DPO training. Experiments show MotionSight outperforms existing open-source models on several benchmarks and rivals commercial models on certain metrics.
weaknesses: 1. Conservative innovation: While extending visual prompting to video is useful, "spotlight" and "motion blur" are traditional enhancements lacking methodological breakthroughs. Although the motion decoupling strategy is reasonable, the process of "detecting first and then focusing" relies on existing detection/tracking models. If the detection fails or is missed, the effect will be greatly reduced.
2. Limited generalization analysis: Insufficient discussion of failure cases in complex scenes (e.g., multi-object interactions, low-light, non-rigid motions like smoke or water).
3. Dataset distribution and quality: MotionVid-QA is largely sourced from existing human-action datasets, potentially limiting diversity; non-human or synthetic motion videos are underrepresented. Human preference annotation introduces subjectivity; no inter-annotator agreement or statistical significance reported.
4. Incomplete efficiency analysis: Mentions ~75% latency increase but lacks detailed time/memory profiling across resolutions or frame counts.

---

confidence: 1
contribution: 2
initial_rating: 6
presentation: 3
questions: 1.See weakness.

2.Though the ablation demonstrates strong robustness to several hyperparameters, I wonder whether the proposed method is sensitive to the parameters of selected models of MLLM.
soundness: 3
strengths: 1.Fine-grained video motion understanding is under explored and worth studying. The paper analyze this task from both method and data sides by designing tailored model and constructing a valuable dataset.

2.The experiment result is solid with high performance on several evaluation benchmarks for fine-grained video motion understanding.

3.Convincing visualization examples are provided to prove the effectiveness of the proposed method.
summary: This paper propose MotionSight, a novel visual prompting method for fine-grained video motion understanding by tailored decoupling object and camera motion. The experiments reveal the enhanced performance for fine-grained motion understanding without additional training data. The authors also collected and annotated the first large-scale fine-grained video motion understanding dataset MotionVid-QA.
weaknesses: 1.The overview of the framework and proposed framework may raise the concern about the processing cost as it contains many designed modules compared for fine-grained  video motion understanding task compared to single MLLM methods. I encourage the authors to introduce the training/inference cost and speed compared with the provided baseline methods in more detailed.

2.Although the performance of the model has improved to some extent after training with MotionVid-QA, the results in Table 4 have certain limitations. The author may compare the results of the model trained on datasets of similar size.


Decision: Accept (Poster)