MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal LLMs
Authors: Yipeng Du
Tiehan Fan
Kepan Nan
Rui Xie
Penghao Zhou
Xiang Li
Jian Yang
Zhenheng Yang
Ying Tai
Venue: ICLR 2026 Poster (2026)
Keywords: Video Understanding
Multimodal LLMs
Fine-grained Motion
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Despite advancements in Multimodal Large Language Models (MLLMs), their proficiency in fine-grained video motion understanding remains critically limited. They often lack inter-frame differencing and tend to average or ignore subtle visual cues. Furthermore, while visual prompting has shown potential in static images, its application to videos' temporal complexities, particularly for fine-grained motion understanding, remains largely unexplored. We investigate whether inherent capability can be unlocked to boost MLLMs' motion perception and enable distinct visual signatures tailored to decouple object and camera motion cues. In this study, we introduce $\mathtt{MotionSight}$, a novel zero-shot method pioneering object-centric visual spotlight and motion blur as visual prompts to effectively improve fine-grained motion understanding without training. To convert this into valuable data assets, we curated $\mathtt{MotionVid-QA}$, the first large-scale dataset for fine-grained video motion understanding, with hierarchical annotations including SFT and preference data, $\Theta{(40K)}$ video clips and $\Theta{(87K)}$ QAs. Experiments show $\mathtt{MotionSight}$ achieves state-of-the-art open-source performance and competitiveness with commercial models. Using $\mathtt{MotionVid-QA}$, we fine-tuned $\mathtt{MotionChat}$ on Qwen2.5VL-7B, which attains 48.3\% overall accuracy on FAVOR-Bench that is comparable to Qwen2.5VL-72B's 48.1\%. In summary, we present a novel zero-shot method and a large-scale, high-quality dataset specifically for fine-grained motion understanding. All the code and annotations will be publicly available.

Published as a conference paper at ICLR 2026
MOTIONSIGHT: BOOSTING FINE- -RAANGR MOTION UNDERSTANDING IN MULTIMODAL LLMs
Yipeng Du Tichan Fan Kepan Nan
Rui Xie
Penghao Zhou
Xiang Li
Jian Yang'
Zhenheng Yang²
Ying Tai 3 Nankai University
1 Nanjing University 2 ByteDance
AVG
Overall
13.9
FAVOR-Bench
?
Static image
Full video with more Fromes
The buy night to seging Batk alless
GV bus obsiesaly eyniit, rather diplavin somowatil comsooot
CM
CM
OM
CM
What is the police offeer in black doing Hox the careera moviing?
Fine-grained motion retrieve
MationBench
jansking ooer the The ganera
the whitz
Overall
AVG
He seems to koiling patricegger commuliig recess pres when the can the while auil in headeey The carera a passsuy so the reEst
The pelice otfice in back a Walling thee fikking to sponap of pesopll Thx carsor mostig lift
MotianSight (Durs)
Gemini
InternV.3
kramiLa
Matieesigght (Durc)
(b)
Figure 1: Motivation and approach overview. (a) Temporal dynamics inherent in motion distinguish videos from static images. (b) Existing MLLMs show limitations in mne-grained motion detection,
whereas approach excels in accurately understanding object and camera motion. (c) Our approach shows superior performance on MotionBench and FAVOR-Benee compared to SOTA.
ABSTRACT
Despite advancements in Multimodal Large Language Models (MLLMs), their proficiency in fine-grained video motion understanding remains critically lim- ited. They often lack inter-frame differencing and tend average or ignore subtle visual cues. Furthermore, while visual prompting has shown potential in static images, its application to videos' temporal complexities, particularly for fine-grained motion understanding, remains largely unexplored. We investigate whether inherent capability can be unlocked to boost MILMs' motion perception and enable distinct visual signatures tailored decouple object and camera mo- tion cues. In this study, we introduce MotionSight a novel zero-shot method pioneering object-centric visual spotlight and motion blur visual prompts effectively improve fine-grained motion understanding without training. To convert this into valuable data assets, we curated MotionVid QA, the first large-scale dataset for fine-gratd fine-grained video motion understanding, with hicrarchical annotations including SFT and preference data, (440K) video clips and (887)) QAs. Ex- periments show Moti MotionSight achieves state-of-the- ahe- open spprceource performance and competitiveness with commercial models. Using MotionVid - QA, we fine- tuned Moti onChat on Qwen2.5VL-7B, which attains 48.3% overall accuracy on FAVOR- -eoocBBecch that is comparable to Qwen2.5VL-72B ' 48.1% 1%. In summary, we present novel zero-shot method and large-scale, high-quality dataset specifically for fine- granned motion understanding. All the code and annotations are available at httttts::/// ju-pca ab. qi thub. 1o/.ooo//oroeecooooo jectss soott. Mot. MnnS ight
INTRODUCTION
Benefiting from high- high-quality video-text datasets CChn et al.
2024b Liu et al. 2024a Chen et al.
2024a
Wang et al. 22023 Yang et al., (0244 and large model parameters (Zhu al. 2025b Hurst
et al.
2024 Bai et al. 2025 Zhu et al. 2025a DeepMind, 2025), Multimodal Large Language
Equal contributions. Ying Tai is the corresponding author.

---

Published as a conference paper at ICLR 2026
Models (MLLMs) have exhibited impressive performance wide range of video understanding tasks. Unlike static images, videos possess a temporal dimension characterized by continuous frame- to-frame changes over time. These changes, originating from object or camera motion, create dynamic and expressive motion patterns, distinguishing videos from images, as illustrated in Figure aa).
However, even with rapid progress and development in video understanding, the task of fine-grained video motion understanding still lacks necessary attention and exploration. While MLLMS acquire broad semantic knowledge from large-scale data pre-training (Bai et al. 2025 Chen et al. 2024d Zhang et al. 2024b; Wang et al. 2025a Zhang et al., 2025). their direct application to motion understanding is often suboptimal. This stems from their tendency to process spatial regions with uniform importance and lack of inherent mechanisms to explicitly discern the subtle frtam- variations critical for nuanced motion analysis. These predispositions mean their potential for fine- grained understanding remains largely untapped (Tu et al., 2025) as shown in Figure (b). Given this, how can we boost the latent capabilities of MLLMS derived from large-scale data to achieve fine-grained understanding of local motion cues and enhance the modeling of subtle inter-frame dynamics? Furthermore, how can we transform the augmented implicit understanding from the models into structured data assets that can be used for training other models and for in-depth analysis?
Motivated by these questions, we conducted extensive experiments and explorations on how to boost MLLMs' inherent fine-grained motion understanding capability through scro-shot approaches, without relying on additional training data. Previous studies in image understanding (Yao et al. 2022]
Shtedritski et al. 2023) Yang et al. 2023b Yu al.. (024) have demonstrated strong interest in visual prompting methods, but their extension to address the intricate temporal complexities of video, especially for nuanced fine-grained motion understanding, still requires further investigation. To demonstrate the inadequacy of naively adapting prompting methods from static images to encapsulate the intrinsic temporal dynamics of events in video, we conducted transfer evaluations on recent motion- sprlion-p benchmarks, and found that even background blur, the best- best-performing image-based visual prompt (Yang et al., 2023b), paradoxically exhibited the poorest performance in fine-grained motion understanding. This approach tends to decrease the model's ability in fine-grained motion understanding due to the loss of contextual information, as shown in Figure 3
To address this, we propose WetionSightt a novel video visual prompting method that decouples object and camera motion tailored for fine-grained motion understanding. For object motion, we apply a spotlight-like visual prompt on bounding boxes correlated with the queried motion to enhance motion perception (aiming at focusing the model's 's attention on the core motion, as illustrated in Figure 7. This is inspired by the fact that pre-training data inevitably includes scenarios like stage performances and TV shows, where important moving subjects are often highlighted while the background is dimmed. For camera motion, which necessitates the MILM's perception of subtle scene changes- capability where MLLMs often exhibit limitations (Hong et al. 2025 Tu al. 2025/ww we designed a method synthetically introduce motion blur into video frames. Interestingly. our experiments reveal that this addition of motion blur significantly benefits camera motion determination. Through our carefully designed configuration, MotionSight enables MLLMs to achieve enhanced results for fine-grained motion understanding without additional training data.
To further convert this capability into explicit, actionable data assets, we collected and anno- tated @(40K) video clips, ((87K) question-answer pairs. Through a rigorous filtering mecha- nism that enhanced the quality of annotated data, we developed an SFT dataset and a preference dataset for training strategies. This process distilled the fine-grained motion understanding ca- pability of MLLMS and aligned it with human preference, and constructed MotionVid QA, the first large-scale open-source dataset for fine-grained video motion understanding to date, encompassing diverse scenes and high-quality video footage. Benefiting from this dataset, our trained MotionChat significantly outperformed the original Qwen2.5VL--BB model, thus validating the effectiveness of MotionVid QA. Extensive experiments demonstrated that MotionSight and MotionVid QA effectively contribute enhancing fine-grained motion understanding.
2 RELATED WORK
MLLMs for video understanding. As MI.MSs continue to advance, a growing body of research
focuses on applying them to video understanding (Bai et al. 2025 Hong et al. 2024 Yao et al. 2024
Xu et al.] 2024 Liu et al.] 2024d Zhang et al.) 2024a Wang et al. 2004g Zhang et al. 2004b
et al. 2024b Wang et al. 2024c 2025a
Zhang et al. 2025 Wang et al. 2024b Chen et al. 2024d).
2

---

Published as a conference paper at ICLR 2026
Caption Pipeline
Video Source
Annotations
Glabal Description
Object Mation
Comera Mation
Boseline MLLM
Chosen
Preference Dataset
Videos with motion
GeReeg Purpose"
MotionChat
MationSight Pipolins
Data Sourees
Text- Video Algnamunt
Object Referring
Optccll Intersity
Inisersit Lendl Appropstir
Reject
Decision MLLM
Action Focusing
Answer refinement
Motion
Motion Specific
SFT Dataset
Data Sources
(<Global Description>, -Object Motion Motion>))
Drop
Our MotionSight pipeline captions high- quality data, transforming it into data assets. Right: This data undergoes rigorous filtering to align with human preferences, resulting in our high-quality dataset MotionVid QA
Figure 2: Overview of the interaction process. Left:
Video understanding models often use keyframes as samples, which are then encoded for LLMs. Several approaches develop specialized connectors, while models encode videos frame-by-frame through a vi- sion encoder before feeding them the LLM. Al- though these methods excel at event-level video rep- resentation, they tend struggle with fine-g fine-grained motion understanding due to limited perception of
Lack inter-frame differencing
Video
Sample
Lack of training une-ggained motion dataset
METM
Query
(a) Training-based based MLLMs
Motion-misseouding
iiter-frame dynamic differences. Directly migrating existing image-level visual prompts to video under-
Video
augment
Visual prompt from image
MILM
standing (ao et al. 2022 Shtedritski et al., 2023]
Yang et al. [023b Yu et al.) (024) performs poorly due to their failure to account for motion information. To overcome this limitation, we propose to decouple object motion and camera motion, leveraging novel visual prompting methods to improve the model's s understanding of fine-grained motion.
Query
am not sure what he doing
because of bhurry background!
(b) Naive visual prompts from image
frame simulate
Motion-aware
Video
Obj-focus
augmenn
Fine-grained motion understanding datasets. Early action recognition datasets (Soomro et al. 2012a| Kuehne et al., [001]] Caba Heilbron
Cam-hint
WILL
Query
2015) had limited fine-grained motion understanding due to simplistic categorical labels. Recent works
use MLLMs for auto-annotation (Chen et al. 2024b
Decouple
Motion
understand he is swinging his
And cumera
Wang et al. 2023 Yang et al.) (024)Nan et al. 2024).
(c) MotionSight (Ours)
but granularity remains limited. Structured video cap-
Figure Comparison of our method with other existing methods. Directly applying image visual prompts can lead to misinter- pretation. By employing decoupled object- guided motion focusing and inter-frama infor- mation enhancement, method addresses the challenge faced by previous methods.
tions (Ju et al. 2024, Fan et al. 2024a] Wu et al. 2025) respond to the need for fine-grained semantics.
However, deficiencies persist in motion semantics de-
lineation due to the lack of a well-designed approach for obtaining fine-grained semantic representations.
Benchmarks like MotionBench (Hong et al. (2025)
and FAVOR-Bentd (Tu et al.) (025) have datasets
with insufficient sample sizes, limiting scene diversity and semantic richness. To overcome these limitations, we propose MotionVid QA, the first large-scale dataset for fine-grained motion under- standing, featuring extensive scene coverage and high video quality.
MOTIONSIGHT
This section introduces MotionSight (Figure 4 with enhanced fine-grained motion perception. Our method decouples object and camera motion and discusses techniques enhance MILLM input
3

---

Published as a conference paper at ICLR 2026
Original Video
Answer!
Object motioo: ridor approuches a dump os brown korse...
Camera motton: ou catirr nils plgght upward to follow the ascre's
Sample
Decision
Fine grained S
Reasening
Send revision to
Decision MLLM
Config Templa
MLLM
Action Focusing
Decaba 10
Question- based Deccuppe
Object Motion
Referring
① Object
Bounding-hox
Aggregation
501, 598)
Frene 4-8-31111111010
0-3:3.60
Ursal Spoctighe
Usuaa
Clpss with action Focus
What the rider on the hurse doing?
groupe and rite", backgrard troes jumpitg costaa
Original Video
Careero metion fini
Camera Mation
Graundng
Syncheni'
Is the cortera
Toiting not!
Weight
docay
Sliding woindow
Motion Blür
Object Mation
Figure 4:
The detailed pipeline of MotionSight. Our method includes query-based motion decoupling, gating based on object motion and camera motion. Subsequently, it selectively passes through modules based on the decoupled type. Then, carefully designed a template prompt for MLLMS to understand our enhanced input and make final decisions.
videos. We sample the input video V= = (I.) (I, E R to = = (I,,), with L. and T denoting original video length and sampled frames length, respectively. Our approach is as follows:
Robi (L (V)))). Ream MLLM( cam (V., V)).
(1)
Here, Robj and Ream are object and camera motion understanding, respectively. Pobj and Peom are the corresponding visual prompting functions.
3.1 OBJECT REFERRING
Initially, the MLLM processes sampled frames V, and the query Q to infer a set of semantically relevant object categories C = {cr, C2,..., Cn) This inferred set C guides our subsequent visual perception modules for object localization and trajectory estimation. Formally, the process of obtaining tracked object trajectories O is defined by the composition:
Merack
(I
0raak))
(2)
where Malttot (Liu et al.
2024b) identifies bounding boxes for categories C in key frame Izy-
Merack (Ravi al. 2024 then propagates these detections across subsequent frames {Ie, }=(++
yielding trajectories O. While direct action inference can hallucinate, robust object identification,
even with initial errors, is retinable by lower-confidence detections (Chen et al. 2024c)
3.2 ACTION FOCUSING
Given tracked objects O {(b.,.)) (b... i-th object's bounding box at frame s; T:: trajectory length), we use dynamic temporal aggregator A to derive refined spatial regions B = (b.) which merge and stabilize bounding boxes against jittering. A adaptively adjusts its temporal
aggregation window based on intra-trajectory positional variance V. Let X' = 24 (b (bstrr.)) denote the union of bounding boxes in each frame:
B = A(Y, V(X)) |T
(3)
Here, V(.) quantifies bounding box positional variance along a trajectory. Specifically, with low positional variance, AA favors a union of bounding boxes over longer temporal spans; with high variance, focuses on localized regions in shorter temporal windows. To quantify positional variance, we measure Manhattan distance between pairs of bounding box centers as [|center(bs...) center( :)ll
The object motion enhancement function Posy then applies visual prompting techniques to the original frames using these dynamically aggregated object regions:
o (v.) = Fvr(Vs,
where Fv represents our visual spotlight approach that darkens the background outside (by IT while preserving the detected objects in their original positions, enhancing focus on the relevant moving elements. In this way, our visual prompt considers object focus and smooth transition.
4

---

Published as a conference paper at ICLR 2026
Table The comparison of existing motion-specific datasets with ours. Our dataset significantly surpasses existing methodologies in both scale and annotation granularity. Furthermore, the quality of our dataset generally exceeds that of currently prevalent notion--spccite datasets used for comparison.
AVidioss
#Tont
Type
Servato
UCP101 peganes al 2011115
13K
N6A
Labck
Harzar
Actun recegptiion
Itaan sonte Humon coonio
Aotiviyhi 2a1e Hritiiee
2015
10K:
Class Labels
Harrar
Artion recogration
recogritionn
(handos
Hemar, Infoce
Video
Infoce
Chandes
2016
63K
Captios
Hortas, indocr chiects
Vidco andersiaaiiu
Iadove
of 1015
Captios
Divene objecs, Curcers olgels. Curerr
Motton ustersarding
Open dansin
RoweBocc
Capioos
Motoon usterzerding
NottentVi (1 (HIP))
10K
Moval
lieene Decuppel
Fineo graind musion instrvaeeting
dimroin
-STT
19K
90K
DAe
Exiiesss mosicn indeniending
DPO
5K
7X
CaserRoeet QAA
Alige with hurean pioferccces
100%
Total Results
100%
Object Motion
3976
Camera Moting
10.1%
67.9%
wedert KMN
62%
60%
60.0%
40%
40%
99.4%
32.1%
3.0%
Oc
Quelira
Dute
Human Preference Comparison
Figure 5: MotionVid - QA: High-quality filtering. Construction high-quality video dataset via filtration. Extensive data distribution. Diverse sources yield varied scenes, subjects, and camera perspectives. Human preference comparison. Our preference data annotation significantly surpasses baseline, particularly for camera motion (Zoom in for best view).
3.3 MOTION BLUR
To overcome the inherent limitations of existing MLLMs in perceiving inter-frame changes, partic- ularly subtle camera motions, we introduce a dedicated Motion Blur Transformation TArB as our camera motion core enhancement function 2eam. This function, Dcam (V, V.) = V', operates on sampled timestamps { using the entire video sequence V = {I+)== to generate a sequence of motion-enhancee frames V = ) thereby amplifying temporal motion cues. For given frame I,,, the enhanced frame I,, is derived through a temporally weighted aggregation of its N preceding frames from the original video. This process can be formally expressed as:
T cam (V, V.) (V.,
where TMB(.)-
Σ k=0
wk (7) II,
(5)
Here, N is the temporal window size (zero-padding applied for k 1). The temporal kernel we use has a temporally increasing trend, with Σx wx(7)= = This temporal aggregation within + camm accentuates motion trajectories by inducing motion blur effects across V', enhancing the MI.LM's capacity to perceive and interpret subtle camera movements.
4 MOTIONVID
In this section, our work yields two key data resources: instruction/preffrrece subsets from public data tailored for two-stage model refinement (SFT/DPO) towards high-quality motion understanding.
Dataset collection and processing. MotionVid QA is curated from variety of sources to ensure multiple types of motion understanding tasks are covered, and rich scenarios ensure the diversity of the dataset, including ActivityNet (Caba Heilbron al. 2015), Kinetics-700 (Carreira al.) 2019). Charades (Sigurdsson et al. 2016), Charades-Fgge Higo (Sigurdsson et al.) (018). Reeaa- 585K (Yuan al. 2025), Open Vid- IM (Nan et al. 2024), and BotionBenchhtrain (Hong al.) 2025). To ensure the quality of the videos, data processing steps we outline in Figure 2 are applied.
SFT and DPO. SFT aims to produce a specialized model π SFT capable of effectively capturing spatiotemporal dynamics and semantic motion patterns inherent in video data, thereby enhancing its performance on specific video understanding applications. DPO aims to simplify Reinforcement Leamning from Human Feedhack (RLHF) by utilizing the log-likelihhod of the learning policy. Instead
5

---

Published as a conference paper at ICLR 2026
Chesen/Reject QAs (Ours)
Labels
Object Matian!
NSA obvieus abject notion
Describe trejectory path taken by moving object.
of overnedd Throughout the sequence, yellow verizus cirectione, inflancedd creating dynomic yet peaceful atnespieee. The backgreund renairs condistant. with the dark couds and eccosiandl beoms of light piarting through, adding a dramatic effect to the scene The felling leaves cont mus to be the primory mot lan, arhancing the tranqul yet sumber noed es they flutter and settic on the ground and umong the flowers.
[Chosen] The
wbbrant
White flowers line the
foll from the trees drifting slowly the The lanves
subtle
dynemic This filsstpp desicts path, florked white natural scone,
Captions
flowers, winds its way towards grove of trees with virrant yellow folloge The sky dbove is dramatic comes of dark, swirlng clouds, sugpesting on impending storm. the fromes progr raid, yallow leaves begnn the frees, cought is breeze, adding the
[Reject] The scene degicts patraaay flonked by loth greerery and white flowors on both sides, with a now frees bearing yellow lesves the left. day is evercest with dark, sminous clouds, and
ternado is forming distance, creating serse of impending storm. Yellow Icoves are being blown by wind, scottering through the oir fonding on parhh surrounding ny vegetatiin.
Comero Mation:
passage still landscape. The beauty
Question: What is the dominant camero movement technique employed this vides seyment?
[Choson) The comere steadlly noves forward olong the parhwey, montoiing consistent focus on the scene anead, The yellow looves continue to be blown by the aind creating dynonic and immerstv vieuel effect The dark clouss and the
nafure.
tomaae the beckground remsin promisent, adsing to the cramatic atmasphere of the scene.
(Saject) video begins with a screns some of dirt poth lending through a grave of trees with vibrant yellow lonwes Throughout secuence, yellow leaves gently from frees, drifting slowly oit The camera remoins static. copturing the gertle novenent the looves and the stillness the surruunding envrronent.
Lost comera motion; Lark human signal, QA pains!
Figure 6: A comparative visualization of MotionVid - QA against existing data. In contrast to antecedent methodologies reliant upon class labels or captions, our approach facilitates the provision of substantially richer and more diverse informational content. Notably, even in scenarios characterized by the absence of salient principal objects, our methodology consistently yields high- quality annotations pertaining to object and camera dynamics.
of learning an explicit reward model, it implicitly expresses the reward function through pair-wise M to optimize the policy model. Let xe is the language preference data D = { yf gchosen y/ ) Ji= model policy which always initialized to n SPT and Tet is also initialized from x SFT The objective function is defined as:
grrjeett) loog
To (ychosen (x)
Teat (y rejje m } (6)
To (y
(f) =
9og
Teat (y (yrhosnn (z) B log
The DPO loss aims to maximize the reward difference between preferred and non-preferred samples, using human preference comparisons signals (see Figure
MotionVid - QA: a large-scale dataset for fine-grained motion understanding. For the pre- filtered dataset, we selected O (40K) clips annotated them with MotionSight. Using VQAScore (Lin) et al. 2024) and human thresholds for categorization, high-quality clips became preference dataset candidates, low-quality ones were eliminated, and the rest formed our instruction dataset. In the SFT phase, to enhance motion understanding capabilities via SFT, we used MotionSight annotations as text data. For the preference dataset, aiming to align fine-grained motion understanding with human preferences via DPO. With a balance between efficiency and quality, we re-annotated this portion of the data using Tarsier2 (Yuan et al. 2025) as the baseline. High-quality preferences data was then developed by incorporating human preference signals from multiple, guided, and well-educated individuals.
Our curated dataset [Table Figure significantly advances fine- thee-rrined video motion understanding. Its key contributions are: (1) Pioneering scale and scope. The first large-scale, open-source dataset for this task, offering diverse scenes and high-quality footage. (2) Diversity and quality, Rigorous filtering enhances clarity (mitigating annotation hallucinations), text-video consistency and pronounced dynamics (cf. Figure 5) (3) Hierarchical data composition. Comprising SFT and preference subsets, it enables multi-faceted learning: general motion understanding (SFT) and refined, numan-aligned fine-grained comprehension. Overall, this large-scale, high-quality, hierarchically structured resource will significantly support future model training and evaluation.
EXPERIMENTS
5.1 DATASETS AND COMPARISONS
Datasets. We evaluated method Moti MotionSight on the benchmark of MotionBench (Hong et al., 2025) and LAVOR-Bench (Tu et al. (025) using their publicly available dev set and close-ended evaluation (encompassing six types of question-answer pairs), respectively. They are both fine-grained

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative results MotionBench. We compared our MotionSight with both pro- prietary MLLMs and open-source MLLMs on MotionBench, all of which have been trained on large-scale video data. The best results of open-source methods are marked in bold.
Model
# Frames
Overall
AVG.
MR
LM
CM
MO
AO
RC
Proprietary MLLMs
Gemini 2.0 Flash (DeepMind 2025
1ips 1ips
56.1
52.6
60.9
57.1
50.9
74.1
37.6
35.0
Gemini Pro (Team]
51
67
GLM-IV-Plus-O OITI ZhipuA 2025
21ps
62.8
60.3
64.1
67.0
67.4
73.5
46.7
42.8
Open-source
Oryx-34B Liu al. 2024d
64
49
47
48
52
44
65
42
32
LLaVA-NeX --800---888 Zhang ot 2024a
53
66
Qwen2.5V1 (Rai al. 21.4
lfps
53.0
48.8
58.3
55.3
34.0
71.5
39.5
34.0
Qwen2.5VL-7288 Barse
Ifps
64.0
48.6
73.2
46.8
InternVI.3-88 (7hir mi 20000400240
16
58.1
53.7
65.1
63.0
47.8
74.1
39.7
32.3
IntemVVL37781
21p55
67.2
78.1
TF. Fusion (Hong of al. 21A5
16
58
54
64
59
51
69
41
39
Qwen2.5VL-7B Mottonnighht InterVVL3-78B- NotionSight
1fps
55.6
52.2
59.7
58.
48.3
73.6
40.1
33.5
16
63.0
89.3
68.5
65.4
58.7
78.6
47.6
37.0
motion-level benchmarks encompassing wide range of video types. We report accuracy for each problem type, with Overall representing accuracy across all problems. We also calculate AVG. the average accuracy across question categories, giving equal weight to each problem type regardless of sample distribution. It's worth noting that our MotionSight is untrained zero-shot enhancement scheme, while MotionChat is trained on the MotionVid QA dataset.
Quantitative evaluation. Table and Table 3 presents the quan- titative results on MotionBench and FAVOR- BenOh, respectively. Our MotionSight consistently enhances the performance of base MLLMs. When using Qwen2.5VL. as the backbone, method achieves 3.4% improvement on MotionBench and 3.0% improvement FAVOR- Bench category average (AVG.), while camera motion improves by 14.3% on MotionBench. Effective performance improvements observed in metrics tightly coupled with object motion, including MR, LM, MO and AO. Furthermore, InternLL--78B augmented with MotionSig demonstrates significant performance enhancement, achieving state-of-the-art results among open-source models like TE Fusion (Hong et al. 2025) and exhibits strong competitiveness against leading proprietary models like GLM-4V-Plus-011 0111 (ZhipuAl 2025).
Table 3: Quantitative results on FAVOR-Bench. We se- lected representative MILMs as baselines for comparison. We computed the OM (object motion) metric by averaging all metrics excluding the CM (camera motion) metric in
Model
# Frames
Overall
AVG.
OM
CM
Pruprictary MLLMs
GPT-4o (Huss ct
20241
lps
42.1
43.1
Gomini- P--Po Tein e 2124
1pps
49.9
50.7
52.6
41.6
Clande-3, 3.7-Somee Anntroii 2025
Tpps
43.7
44.0
45.0
39.1
Open-snuzce MLLMS
LLaVA-NeXT
(Zhan dell 2103
1fps
41.5
41.5
43.4
31.5
Qmen2.5VL-77
20 2013
1fps
42.3
41.6
43.7
30.9
LL37VL
52.8
52.6
56.2
34.3
also
Quen2.5VL-778 5LL-7B +MotionSightt
InternV1.37888
Motlon8ighht
53.8
53.5
Qualitative Evaluation. As shown in Figures 7 and 9 the visual spotlight of MotionSight hances the model's ability to focus more on motion information, while the motion blur method significantly enables the model perceive changes in camera motion in videos. Our approach to decoupling object motion and camera motion leads to significant improvement in accuracy. Figure 6 presents our dataset, which believe will make substantial contributions to the research community.
Original
VVisual
Spotlight
Figure 7: The difference between using visual spotlight and the original MILM. We used Grad-CAM and selected the same layer for gradient computation. After incorporating the visual spotlight, the model pays more attention to the core region. Prompt: "What are the people doing?".
7

---

Published as a conference paper at ICLR 2026
MotionChat. We conducted fine-tuning experiment on our self- built MotionVid QA dataset to demonstrate its core effectiveness. Based Qwen2.5VI.-7B.
Table Quantitative results for MotionChat based on Qwen2.5VL-7B across different training stages. We ana- lyzed the impact of different training strategies by selectively including excluding our SFT and preference datasets FAVOR-Bench. Green areas indicate best performance. "V" indicates the presence of a training stage, while "x" indicates its absence. "+ShareGPT4Vidder denotes that we fine-tuned Qwen2.5VL-7B solely on the ShareGPT4Video dataset, demon- strating the superior quality of our data.
first performed full-parameter SFT training, followed by DPO training on the SFT model, with the vision module frozen dur- ing DPO training. As shown in Table after the two-stage fine-tuning on MotionVid - QA, our MotionChat significantly out- performed the original model. Fur- thermore, achieved an overall ac- curacy of 48.3% on FAVOR- ReBcechh, which is comparable to the motion understanding ability of Qwen2.5VL-72B.
SET
DPO
Overall
AVG.
AS
HAC
SAD
MAD
CM
NSM
48.3
46.9
49.6
54,9
45.9
55.1
32.1
43.8
xx
*
x
45.8
44,5
47.3
51.6
43.7
52.3
30.1
42.2
43.8
42.3
45.6
45,8
44,9
49.2
28.9
39.1
General Video Understanding. To verify the broader impact of our approach, tested VideoMME. The results, shown in Table 5 and Figure [8] indicate that NotionSight, improves performance on several general-purpose tasks, particularly those related object motion, and does not exhibit significant global information loss. Our method outperforms the baseline on most met- rics and achieves performance comparable to the LLaVA-One's OneVinion-722 model. We attribute this enhancement to the visual spotlight, which helps the model focus on task-relevant regions, thereby yielding consistent performance gains. To further investigate the generalizability and extended value of our approach, we conducted additional experiments with Moti MotionSight on MVBench (Li et al.) 2024a), TOMATO (Shangguan et al. 2024). STI-Bench (Li et al., 2025) and TempCompass (Liu et al. 2024c), shown in Table 7 Notably, results also show that our decoupling strategy brings additional gains in spatial perception and understanding.
5.2 ABLATION STUDY
Object motion understanding. We evaluate visual prompting strategies to improve object motion understanding. As detailed in Table our proposed visual spotlight yields the highest average object motion score (OM AVG.). Background blur, however, negatively impacted performance, contrasting its effectiveness in static image prompting (Yang et al. 2023b). We attribute this failure to blurred object boundaries, increasing the demand for robustness and misleading MILMs. Other visual prompts, including object crop, object motion blur (applied solely to the object mask) and pose estimation (applied to the entire video), also provided marginal or negative impacts. These findings underscore the efficacy of visual spotlight in directing model attention to pertinent object movements.
Table 5: Evaluation on ViE We present the core general-purpost tasks.
Model
Overall
Temporal Perception
Action Recognition
Object Recognition
Temporal Reasoning
Spatial Reasoning
Action Reasoning
Information
Synopsis
Qwen2.5VV VVB.2.
73.7%
77.1% 80.2%
74.4% 76.2%
76.9%
68.15 70.2%
86.6%
+Ours
76.0%
88.9%
84.6%
77.8%
87.8%
Baadine
$2.4
910
Qwen2.5VL-78 MMotionblight
$7.8
Performance
Performance
86.6
50
45
80
79.6
60
75.6
75
35
Camera Motion
Object Motion
Other Motion Average performance improvement on general video benchmark (VidesMMEE.
Object Motion
(a)
Average performance improvement Fine-grained
mation benchmarks (MotionBench and (WORRBBench..
Figure 8: We compiled the metrics related to object motion, camera motion, and other motion from the benchmark and averaged the experimental model and task metrics. Other motion refers to tasks with low correlation to object motion, for which we also used the visual spotlight focus on core regions. Our method shows significant advantages after decoupling motion.
8

---

Published as a conference paper at ICLR 2026
Table 6: Experiments of several visual prompt methods specialized for motion decoupling on
MotionBench. Green areas indicate best performance and red areas show lowest scores.
Object Motion
Camera Motion
Method
OM AVG.
MR
LM
MO
AO
RC
CM AVG.
Qwen2.5V1. 7B (Bai et al. 2025
51.7
58.3
55.3
71.5
39.5
34,0
34.0
Visual Spotlight
53.0
59.7
58.1
73.6
40.1
33.5
Object Crop
52.5
59.0
55.9
71.3
40.1
36.5
+ Background Blur (Yang et ml. 2023b
49.3
56.2
53.9
65.4
39.9
31.3
Objectt Motion Blur
50.2
52.5
40.4
29.3
Pose Estimation
50.6
56.4
54.6
70.1
40.1
32.0
+ Global Motion Blur
50.8
57.2
$7.0
69.1
39.3
31.5
48,3
API Prompting (Yu et 2024
51.4
57.9
55.5
73.6
39.3
30.5
The vidko suparros bugind with a stalionary frome, fealuring a man stording next to black oor with its trunk open The man is holding a block soocter and a changing cable, which later placed inside the cor trunk. The comena then tronsitions to close-up up shot of the man cs he holds the charging cable and tolks, implying that the video is demonstrating how to charge on electni sceater using the provided cable. The comara briefly remoins sheody, focusing on the mon os he exploins the process. At one point, the mon benos down, likely to retrieve or place the otble the trunk.
Thers is no no notioeable comera motion, suggesting that the comare remains stationary, providing o clear view of events unfolding in front it. The mon stonds front of the car, and the comera positionsl itauf to copture actions clearly.
Object Mation: The person stards
open
They gesture their hand while talking gesture. meving black bay cround They a black still holding the dbject talking
places
ground begins orgonize insice
ttchedd
ooject
adjusts bog
oyain. Mation
The organizing items trunk, the black ooject cables shows the pr slight zoom in to highlight the object The comero then follows the person movemun they place sbjectt back into the trunk and adjust the beg, maintaining ateyydy handheld mation. Finelly as the person cleses the trunk and steps back, the comera zooms slightly to capture the enfine soene, includingg the survounding street and passing from.
shot the
the ground truck. holding
remains
slightly the person gestures and talks, person places the
focus closely their
Figure 9: Qualitative example of MotionSight. The upper panel represents the baseline, while the lower panel shows the result enhanced by our method, which possesses fine Aine-graied inter-frame difference perception, enabling precise capture of subtle motions.
Camera motion understanding. For camera motion, our primary evaluation focused on applying motion blur to the entire video frame, also referred to as global motion blur. Table TJble that our motion blur synthesis approach facilitates the model's perception of subtle inter- differences, thereby yielding a substantial improvement, significantly outperforming the baseline. The results also demonstrate the effectiveness of our decoupled object and camera motion method. Please refer to Section B in the appendix for more ablation studies.
Table 7: Performance of MotionSight on more video understanding benchmarks.
Method
MVBench
TOMATO
STI-Bench
TempCompass
Qwen2.5-VL-7B
60.5
26.8 27.3
31.6
68.5
+ MotiinSiiiioo
64.4
36.5
71.3
5.3 MORE APPLICATIONS
Applications to the video generation. The MotionVid QA not only be used for MILM fine-tuning but also support video generation, enhancing motion quality in generated videos. We constructed video-text pairs from the large-scale, high-quality data in the SFT subset, and conducted fine-tuning on Wan2. 1-2VV-L33 followed by evaluation on VBench. As shown in Table & and Figure TO after our efficient fine-tuning, the model achieves significant improvements in motion- related generation, which can be attributed to the high quality and rich fine- grained motion content of our dataset.
Self-improving VLM pipeline. We tested the further application of MotionSight on top of MotionChat and found that it can further improve the model's performance, shown in Table 9 Therefore, MotionSight has the potential to serve as a self-improving optimization approach. This introduces novel paradigm for unique self-enhancing motion understanding capabilities: we can

---

Published as a conference paper at ICLR 2026
Prompt: A bost sailing leisurely along the Seine River with Eiffel Tower in background.
Prompt: person stretching leg.
Original W.n21- 1,30
MitionVid
Figure 10: Qualitative example of our fine-tuned Wan2. 1-n2----2
Table 8: Fine-tuning results on MotionVid QA for video generation.
Human Action
Motion Smoothness
Dynamic Degree
Temporal Style
Wan2.1-T2V-1.3B
64.2
97.9
47.7
20.4
+ -MotionVid Q4
71.0
98.3
54.3
21.5
Table 9: Results after applying MotionSight on top of the fine-tuned Motion model.
Method
Overall
AVG.
AS
HAC
SAD
MAD
CM
NSM
MotionChat
48.3
46.9
49.6
54.9
45.9
55.1
32.1
43.8
MotionChat MotionSight
49.7
48.3
50.7
55.3
46.0
56.0
38.2
43.8
leverage "MotionChat+ MotionSight" as a more powerful teacher to generate next-generation datasets, thereby continuously pushing the ceiling of VLMs in fine-grained motion understanding.
DISCUSSIONS ON NOVELTY AND VALUE
We are the first in the motion understanding field to propose a methodology that decouples object motion from camera motion. The challenge fine- aen-ggrined motion understanding lies in the difficulty MLLMs have in perceiving subtle inter-frame changes:
Existing video understanding models perceive motion holistically and struggle to separate object motion from camera motion. In contrast, our specially designed decoupling strategy makes it casier for the model to perceive subtle camera motion and object motion separately,
Simply applying static image prompting methods commonly used in previous work to videos can lead to misunderstandings (Figure 3.. Our designed algorithm possesses dynamic stability, which helps enhance the MLLM's ability to perceive fine-grained motion.
Existing datasets are limited to indoor human-centric scenarios with simple labels (Ta- ble D.. In contrast, we constructed the first large-scale open-source fine- ine-grained video motion understanding dataset, MotionVid QA, which includes open-domain motion and fine-grained labels such as decoupled motion and preference question pairs.
Moreover, MotionChat offers strong practical efficiency (as shown Table 21: outperforms the 72B baseline while being approximately 5-6x faster and far more memory-efheienn
7 CONCLUSIONS
In this work, we address the challenge of fine-grained video motion understanding, task that has been relatively underexplored despite the advancements in MLLMs. We introduce MotionSight, novel zero-shot approach that decouples object motion from camera motion. By employing visual spotlight and motion blur, it enhances the ability of MLLMs to perceive subtle motion curate MotionVid QA, the first large-scale open-source dataset designed
cues. Furthermore, for fine- fine-graines grained video motion understanding. Our experiments demonstrate the effectiveness of Mo MotionSight and MotionVid QA facilitate future research in this domain.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work was supported by the Gusu Innovation and Entrepreneur Leading Talents: No. ZXL2024362, Natural Science Foundation of Jiangsu Province: BK20241198, and Natural Sci- ence Foundation of China: No. 62406135.
REFERENCES
Anthropic. Claude 3.7.
February 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025.
Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the ieee conference on computer vision and pattern recognition, pp. 961-970, 2015.
Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zisserman. A short note on the kinetics-700 human action dataset. arXiv preprint arXiv: 1907.06987.7 2019.
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong. Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang. Li Yuan, Yu Qiao, Dahua Lin, Feng Zhao, and Jiaqi Wang. Sharegpt4video: Improving video understanding and generation with better captions. In A. Globerson, L.. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 19472-19495. Curran Associates, Inc., 2024a. URL httpttss//prrcceingss.tott http://eeeeiineeeeeeissossooo/eo
and Bencheanchhhsseeeeee Bwaicrks_srraaacea Track. pdf
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wel Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple coos-moo teachers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13320-13 11331,, 2024b.
Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Hale: Object hallucination reduction via adaptive focal-contrast decoding. arXiv preprint arXiv:2403.0042 2024c.
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguise tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 24185 24185-24198, 2024d.
Google DeepMind. Gemini: Our most intelligent ai models, built for the agentic era.
https
7/depmindd wlleepmind.googlee/eeeeeeeeeeeee www.ggoooooggooooooooo pegggiiis/ggmnnn///////ggg 2025.
Tiehan Fan, Kepan Nan, Rui Xie, Penghao Zhou, Zhenheng Yang, Chaoyou Fu, Xiang Li, Jian Yang, and Ying Tai. Instancecap: Improving text-to-video generation via instance-aware structured caption. arXiv preprint arXiv:2412.09283: 2024a.
Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Videoagent: memory- augmented multimodal agent for video understanding. In European Conference Computer Vision, pp. 75-92. Springer, 2024b.
Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, et al. Cogvlm2: Visual language models for image and video understanding. arXiv preprint 440iv:2408. 16500, 2024.
Wenyi Hong, Yean Cheng, Zhuoyi Yang, Weihan Wang, Lefan Wang, Xiaotao Gu, Shiyu Huang, Yuxiao Dong, and Jie Tang. Motionbench: Benchmarking and improving fine- -ra-gea video motion understanding for vision language models. arXiv preprint arXiv:2501.02955, 2025.
11

---

Published as a conference paper at ICLR 2026
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.2127,, 2024.
Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng. Yu Xiong, Qiang Xu, and Ying Shan. Miradata: A large-scale video dataset with long durations and structured captions. Advances in Neural Information Processing Systems, 37:48955-48970, 2024.
Hildegard Kuehne, Hueihan Jhuang, Estíhaliz Garrote, Tomaso Poggio, and Thomas Serre. Hmdb: a large video database for human motion recognition. In 2011 International conference on computer vision, pp. 2556-2563, IEEE, 2011.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11195-22206,, 2024a.
Xinhao Li, Yi Wang. Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al. Videochat-flash: Hierarchical compression for long-context video modeling. arXiv preprint arXiv:2501.000 .00574, 2024b.
Yun Li, Yiming Zhang, Tao Lin, XiangRui Liu, Wenxiao Cai, Zheng Liu, and Bo Zhao. Sti-bench: Are miims ready for precise spatial-temporal world understanding? arXiv preprint arXiv:2503.23765, 2025.
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. arXiv preprint arXiv:2404.01291, 2024.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lec. Llava-next: Improved reasoning, oer, and world knowledge, January 2024a. URL https
//1lava-v1. gww.l..gtttu.....mm httpbbiio/oogg/22-000000 -0-a--3---aaa-eettt----
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pp. 38 55. Springer, 2024b.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video iims really understand videos? arXiv preprint 000600060000 2024c.
Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen La, and Yongming Rao. Oryx milm: On- demand seatial-temporal understanding at arbitrary resolution. arXiv preprint 1rXiv12409.12961, 2024d.
Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid- A large-scale high-quality dataset for text-to-video generation. arXiv preprint 1xXX:22407.02371, 2024.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.007 084400000714, 2024.
Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, and Arman Cohan. Tomato: Assessing visual temporal reasoning capabilities in multimodal foundation models. arXiv preprint urXiv:2410..222 2024.
Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about red circle? visual prompt engineering for vlms. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11987-11997, 2023.
12

---

Published as a conference paper at ICLR 2026
Gunnar A Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. Hollywood in homes: Crowdsourcing data collection for activity understanding. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14, pp- 510-526. Springer, 2016.
Gunnar A Sigurdsson, Abhinav Gupta, Cordelia Schmid, Ali Farhadi, and Karteek Alahari. Charades- ego: A large-scale dataset of paired third and first person videos. arXiv preprint arXiv: 1804,09006 2018.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: dataset 101 human actions classes from videos in the wild. arXiv preprint /2l:::222.04400 2012a.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: dataset 101 human actions classes from videos in the wild, 2012b. URL httth:::: http/aaxivi./o org/ abs 11212.0400
Gemini Team, Petko Georgiev, Ving lan Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.055500 2024.
Chongjun Tu, Lin Zhang, Pengtao Chen, Peng Ye, Xianfang Zeng, Wei Cheng, Gang Yu, and Tao Chen. Favor-bench: A comprehensive benchmark for fine-grained video motion understanding. arXiv preprint arXiv:2503. /4935553..499,,,, 2025.
Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun. Tarsier: Recipes for training and evaluating large video description models. arXiv preprint urXiv:2407.00633 2024a.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-lane language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12199 12191, 2024b.
Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint ar arXiv:2307.06944 2023.
Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang. Yansong Shi, et al. Internvideo2: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pp. 396-416. Springer, 2024c.
Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvide22. 5: Empowering video milms with long and rich context modeling. arXiv preprint arXiv:2501.12388 2025a.
Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jachong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for Ilm reasoning long videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
3272-3283, 2025b.
Shengqiong Wu, Weicai Ye, Jiahao Wang, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, Shuicheng Yan, Hao Fei, et al. Any2caption: Interpreting any condition to caption for controllable video generation. arXiv preprint arXiv:2503.24:: 24379, 2025.
Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. Pllava: Parameter- free llava extension from images to videos for video dense captioning, arXiv preprint arXiv:2404. 16994, 2024.
Dongjie Yang, Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao. Vript: A video is worth thousands of words. Advances in Neural Information Processing Systems, 37:57240-57261 2024.
Jianwei Yang. Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.1 11441, 2023a.
13

---

Published as a conference paper at ICLR 2026
Lingfeng Yang. Yueze Wang. Xiang Li, Xinlong Wang, and Jian Yang. Fine- tine-grained visual prompting. Advances in Neural Information Processing Systems, 36:24993-25000 2023b.
Yuan Yao. Ao Zhang, Zhengyan Zhang, Zhiyuan Liu, Tat-Seng Chua, and Maosong Sun. Cpt: Colorful prompt tuning for pre-trained vision-language models, 2022. URL BRL.htpp:: whthhhtppt///aat //arxiv
org/abs/ 21 09. 11797
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint arXiv:2408.00 01800, 2024.
Runpeng Yu, Weihao Yu, and Xinchao Wang. Attention prompting on image for large vision- lisoun-language models. In European Conference on Computer Vision, pp. 251-268. Springer, 2024.
Liping Yuan, Jiawei Wang, Haomiao Sun, Yuchen Zhang, and Yuan Lin. Tarsier2: Advancing large vision-language models from detailed video description to comprehensive video understanding, 2025. URL htttttt/////xxiiii http:///awx.v....aaa///bbaa/
07888
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang. Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.33106, 2025.
Zhang, Li, H Liu, Y Lee, L Gui, D Fu, J Feng, Z Liu, and C Li. Llava-next: strong zero-shot video understanding model. 2024a.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713, 2024b.
ZhipuAI. Glm-4v,
2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. 10479, 2025a.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang. Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang, Jiapeng Luo, Yi Wang. Conghui He, Botian Shi, Xingcheng Zhang. Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu. Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models, 2025b. URL http:tt:::/ //tp/vvavvvv///////ave/eaaeaawwv org/ abs 2504. 10479
A IMPLEMENTATION DETAILS
A.1 EXPERIMENTAL SETTINGS
We performed all experiments of MotionSigh ihtt on 8 NVIDIA 4090 GPUs with 48GB of memory
each. For GroundingDINO, the box threshold and text threshold for post-processing grounded object detection are both set to 0.25. For SAM 2, the mask dictionary model uses an IOU threshold of 0.8 for updating masks. In motion blur part we use decay factor & = 0.65 and temporal window N = 7. darken factor 8 = 0.9. To address potential detection loss due to frequent
For action focusing object entry and exit (e.g.. FAVOR- -ench Tu et al. (2025)), we perform re-detection using Maetect fixed intervals At, ensuring comprehensive object capture. The weighted mean (temporal kernel) in Sec 3.3 is implemented as follows:
WN-1-4(7) = *
II j=k+1
(1 7d)
14

---

Published as a conference paper at ICLR 2026
For the fine-tuning experiments, we utilized 32 A100 GPUs with 80GB of memory each. The SFT process was conducted on the Owen2.5-VL-7B-Imnst During SFT, the vision tower, LLM, and merger components were all trainable. During DPO training, we exclusively trained the LLM part.
model, employing global batch size of 128.
A.2 PROMPT TEMPLATE
We present prompt details. For the Object Referring stage, the prompt shown in the upper image to enable the MLLM to locate the most critical objects based on the video and question content. We utilize the concept of action groups allowing the MILM to identify relevant objects- and even their components-in in a fine-grained manner.
The code below defines key part of final configuration template. Based on enhanced inputs such as Action focus or motion blur, we route the input into the MLLM using a routing mechanism. In this template, iescrippiinn.io for_vorreeereoooooooo video_type refers descriptions tailored for different video types.
vi video perp_ddscripiiinss =
'origina'' 'Original video: In',
spotlight'; 'Spotlight video:\n',
mottonnbbur' 'Original video with motion blur to more clearly determine the type of motion (such as whether the camera is moving, as one frame combines information from multiple frames. If static objects in the background appear noticeably blurry, there is good chance that the camera is moving!):\r!
Action group analyze.
<video><t iti.e_ifeee..
have question: "<quest on> " need you analyze above question step by step. this step, you don't need to directly answer the question.
Please provide your response in the following JSON format without any comment:
"action.objects" ["object1", "object2", ...l,
strings, each describing a specific entity that is involved the main action or motion. Each entity should a single object or group of objects. For example, if the question is about person cating, include both the person and the rice bowl. If the question is about object motion, make they interact with affect. You also provide fine-grained components of larger objects when relevant. Each string represents different object. All items must physical entities that can he visually identified, not abstract concepts. Only keep the moving objects that are highly relevant to the question and reduce the background objects. You must provide at least one action object.
the "action oobjectt" field, provide
include both the moving objects (actors/performerr) and the objects
Config Template.
seldee><<tim.inne
The video contains [len_frames] frames sampled (sec) seconds.
(description video type 1)
[IMAGE.TOKEN]
Frame www.fffaammmmmmma [IMAGE_OOKEN] (description_for for www.vvioo.type....
15

---

Published as a conference paper at ICLR 2026
Frame(frame_num) (MMAGE_TOKEN;
Frame (frame : (IMAGE TOKEN
Here is the question: "<question>".
Reply based on the above information. Answer only the answer letter without showing your process.
B MORE EXPERIMENTS
B. APPLICATIONS TO GENERAL VIDEO UNDERSTANDING
We conducted additional evaluations with Qwen2.5VL. VLL-7B heing the backbone on the TempCompass benchmark prove the general effectiveness of Moti onSight and impact of global information.
Table 10: Evaluation results on the main TempCompass benchmark.
Model
Avg
Action
Attribute Change
Direction
Order
Speed
Qwen2.5VL-7B
68.50
94.40 94.72
76.20
50.42 52.55
71.62
50.65
+Ours
71.32
79.03
74.47
58.40
Table 11: Performance on the captioning subset of TempCompass.
Model
Avg
Action
Attribute Change
Direction
Order
Speed
Qwen2.5VI.7B
54.6
92.7
68.5
32.8
51.2
29.9
+Ours
64.0
93.2
77.1
38.1
67.0
47.4
Table 12: Breakdown on TempCompass's fine-grained captioning sub-dimensions. AS: Absolute
Speed; CM: Camera Motion; CGA: Coarse-grained Action; CL.C: Color & Light Change; CC:
Combined Change; FGA: Fine-grained Action; OC: Other Change; RS: Relative Speed; SSC: Size
Shape Change.
Model
AS
CM
CGA
CLC
CC
FGA
Other
OC
RS
SSC
Qwen2.5VL-7B
43.9
15.0
95.1
76.9
70.0
90.1
51.2
66.7
22.7
63.1
+ *MotionSight
62.9
31.7
95.6
87.5
75.0
90.6
67.0
75.0
39.5
72.2
For TempCompass (as shown in Table 12 Table TO Table 11 we compared our method with
Qwen2.5VL-7B under the same settings, using "gpt-4o-ny mini-2024-077 2024-07-18 as the LLM evaluator for TempCompass. The results show that method significantly outperforms the baseline TempCompass, demonstrating strong advantages even in more general temporal understanding tasks. Notably, observed that our method achieves far superior performance subtasks with higher output freedom, such as captioning. This reflects a deeper enhancement in motion understanding: objective questions are essentially simple discriminative tasks, often requiring only the judgment of isolated motion facts. In contrast, captioning demands more profound and fine-grained descriptions, which aligns well with the core design of MotionSight for refined motion understanding. In summary, our approach has greater universality and potential.
B.2 MORE VALIDATION OF DATASET QUALITY AND GENERALIZATION CAPABILITY
To further verify the generalization capability of VidionVid-QAA on stronger models, we also fine- tuned the Qwen3--L--BBIInitteee
model. Notably, the original Qwen3-VL-8D model (overall score of 53.0) has already surpassed the VL3-778 model (52.8 Table 3). As shown in Figure []] and Table [3] the results demonstrate that even already strong models, method can still significantly improve performance, which validates the excellent generalizability and complexity of our dataset.
16

---

Published as a conference paper at ICLR 2026
Qwen2 SVL-1B
Qwani-V1-58
Qwaii-V1.4B
50-
57-
ImaavVL3-788
48.3
56.3
48-
55.8
FAVOR- FAVOR-Bench Reuult
46
44-
42
41.6
$2.8
51-
Overall
AVG.
Overall
AVG.
The training results on Qwen2.5-VL-77B and Qwen3-VL-8B using our dataset show that our data can also enhance even stronger models, demonstrating its high quality, high complexity, and strong generalization capability.
Figure 11:
Table 13: Results of Fine-tuning Qwen3-VL-8B-Innstee with Motion Vid-QA on FAVOR-Bend'
Method
Overall
AVG.
HAC
SAD
MAD
CM
NSM
Qwen3-VI-B8
53.0
52.8
52.9
58.5
52.7
56.4
41.7
54.7
+Motion Vid FT
55.8
56.3
55.4
64.3
53.0
61.5
42.5
60.9
After fine-tuning with Motion the overall accuracy increased by 2.8%, and the average accuracy increased by 3.5%. Our fine-tuned 8B model (55.8%) also significantly outperforms the Intern VL3-78B model (52.8%). The substantial gains on complex tasks such as HAC (+5.8%) and NSM (+6.2%) further validate that the filtered dataset retains challenging fine-grained motion semantics, which remain applicable even to powerful bascline models.
B.3 MORE ABLATION STUDIES ON ROBUSTNESS
To clarify that our method demonstrates strong robustness hyperparameters, we conducted ablation studies on essential parameters: the degree of background darkening in the bounding boo-bboo-beeeeeee based spotlight (from slight dimming completely black background), temporal window size, and the decay factor discussed in Section 3.3
To evaluate the robustness of background darkening in the visual spotlight, performed denser sampling within the selected parameter range from 0.1 to 1.0. The results are shown Table [5] We perform separate analyses for the temporal window size and the decay factor assess their individual robustness on model performance, as shown in Table 16 and Table 17
Overall, the ablation results show that our method significantly outperforms the baseline under differ- ent parameter settings. Thus, our method Moti iettonSight iight has strong robustness, We supplemented more visualizations of MotionSight, as shown in the Figure 14
Table 15: Ablation study the degree of background darkening.
Darken
Baseline
0.1
0.5
0.8
0.85
0.9
0.95
1
OM AVG.
51.7
52.5
52.5
51.9
52.1
53.0
52.
52.0
17

---

Published as a conference paper at ICLR 2026
Boeciin Deraal Perfomma: mece
Dvadd Guin AVG. Gais
63.0
Boseiine AVG. Perfumtaace
60
$9.3
Table 14: Ablation study of different input methods.
$5.6
Per Performance
55
Model
OM AVG.
Qwen2.5V1.-7B
51.7
47,5
Our MotionSight
53.0
45
Direct Input Coords
51.8
40
8B
7SB
Figure 12: Ablation study on model scalability.
Table 16: Ablation study on temporal window size (N in Equation 5)
with fixed decay factor: 0.65.
Temporal Window
Baseline
5
7
11
CM
34.0
48.1
48.3
46.0
41.0
Table 17: Ablation study on the decay factor (y in Equation
5.
with fixed temporal window:
Decay Factor
Baseline
0.4
0.65
0.85
CM
34.0
46.2
48.3
45.2
Table 18: Performance under different frame sampling parameters.
Method
Baseline
fps
8 frames
4 frames
MotionSight
53.0
55.6
55.4
54.1
Table 19: Performance under different max max_pixels parameters.
Method
Baseline
360p
720p
native
MotionSight
53.0
55.6
55.8
55.9
B.4 ABLATION STUDIES ON MORE PARAMETERS
To further investigate the robustness of our method, we conducted experiments with a wider range of parameters. The results show that Our method exhibits strong robustness to the parameter choices the selected MILMM.
Robustness to frame sampling parameters. Sampling at fps, frames, or 4 frames all yield stable improvements over the baseline, shown in Table 18
Robustness to max_pixede resolution parameters. For different mix-pixels settings in the MLLM (where "native" indicates no scaling), our method consistently and stably outperforms the baseline, shown in Table 19
Architectural robustness. The architectures used in experiments (Q5en2.5-VL. and Intern VI.3) represent the two mainstream typical MILMM architectures, which in itself demonstrates the robustness of our approach to different selected models.
18

---

Published as a conference paper at ICLR 2026
In the video, there are two individunls standing on the porch house. One person walking towards the left side of the frame, while the other remaing stationary near the deorway, The porch adomned with severdi potted plants and decorative items, including large blue pet white furniture. The appears to during the daytime, indicated by the bright lighting and shadows by the they are elements
objects and people.
significant actions being performed by the objects
environment.
The video begins with th a person in a black outfit walking towards the house entrance from left side the frame. As they approach, on individual stonding of the deorway roises their hand gesture, The first person continues to walk forword, and upon reaching the steps, they pouse briefly before entering the house. Throughout the the individuals remain mostly stationary except for the movement person walking the subtle gestures of the individual doorway. surrounding envirotment, indl uding plant chairs, remains unchanged.
In the video, person exiting rustic building, holding coiled traditional attire, suggesting a rural agriculuurl setting they step out, they seem ergaged form work activity neloted to forming or outdoor lobor, given the tools they are corrying. The envirommnt around them includes stone walls and wooden structures, reinforcing the impression of countryside village location, The person's is movements suggest they might be heading towards these such tending livestock preparing for fire.
bundle of sticks finewaod, The individual appeors be
task that
The video begins with they walks forward, awinging slightly they move. Throughout the sequence, the person maintains steady pace, continuing to gesture with their left arm while holding the rope. their night hand. The background remains consistent, featuring the wooden and brick structure of the house, with significant changes in the scene or additional interactions.
exiting doorway, holding rope in their hand and small object in their left hand, they outward, pointing or gestuning towards something screen.
arm
Figure 13: Quantitative results between baseline and fine-tuned models trained using both SFT and DPO. For each case, the upper one the Qwen2.5VI.-7B baseline, and the lower one is our model after fine-tuning.
Table 20: Performance across MLLM models of different parameter scales.
Intern
Overall
AVG.
IB
47.6
43,9
+Ours
50.8 (+3.2)
47.5 (+3.6)
8B
57.2
52.7
+Ours
59.6 (+2.4)
55.6 (+2.9)
78B
61.5
57.6
+Ours
63.0 (+1.5)
59.3 (+1.7)
B.5 SCALABILITY OF MotionSight
We conducted experiments based on InternV33, ranging from 1B to 78B parameters, demonstrating that method exhibits good scalability across popular model sizes. The results MotionBench are shown in Figure 20 2nd and Table 20) The lighter portion within each bar represents the performance improvement of our method over the baseline model.
B.6 QUALITATIVE RESULTS OF MotionChat ON THE DATASET
As shown in Figure 13 MotionChat exhibits enhanced fine- ine-grained motion perception, more accu- rately interpreting complex motion narratives compared to the baseline model. Our fine-tuned model, trained on our dataset, demonstrates superior fine-grained motion perception capabilities and outperforms the baseline.
19

---

Published as a conference paper at ICLR 2026
The video begins with a women holding a young child by the hands, lifting them dlightly off the ground. The child is loughing and oppears to be enjoying the moment. In the background, another child is seen interacting with a small brown horse, gently petting or brushing The continued lift lower the child, creating playful swinging motion. The child's loughter and excitement evident os they The woman's movements gentle and riythmic, ensuring the child's safety while adcing to the
Object Mation:
lifted higher and then brought
playful atmesphere.
Camera Motion: The camera remoins relatively steady throughout, focusing on the interaction tion between the womon and children. The womon seen holding young child by the hands, lifting them slightly off the ground in playful swinging mo motion, The comero follows the motion slightly, maintaining a handheld feel that adds to the dynamic and lively atmesphere. The camera subtly pans to keep the main subjects focus while allowing the backir sund activity remain visible. The coptures joyful moment with dynamic interactions tions between the individuals slight
the horse,
enhancing the
liveliness spontoneity.
Figure 14: Another qualitative example of MotionSight.
C VISUAL SPOTLIGHT'S MOTIVATION
Visual spotlight is designed to help the model better focus on the most crucial regions of temporal changes when processing video frames. In scenarios like TV shows and movies, important moving subjects are often highlighted while the background is dimmed. Additionally, some performance shows often use techniques such as stage spotlights. MLLMs inevitably encounter such data during training (e.g., datasets like Panda-70M, which contain large amount of TV program content). Also, using the visual spotlight method can indirectly help MLLMs concentrate their visual attention on core motion regions, intuitively and effectively improving the model's ability to capture motion information.
Question: What's the horse's walking direction?
Original
-Visual
Spodight
Figure 15: Another Grad-CAM visualization of our visual spotlight method after vision module. Despite using smaller darken factor, the model can still provide more attention to moving objects.
Figure 7 and Figure 15 shows the Grad-CAM visualization results after applying our visual spotlight method to Dwen2.5VL-77 The first figure uses higher darken factor, while the second figure uses a slightly lower one. As can be seen, our method allows the model to focus more on the motion-related visual parts relevant to the question across different darken factors, thereby improving its fine-grained motion understanding capabilities.
Furthermore, we have implemented a comparative experiment where bounding box coordinates directly provided as prompts, as shown in Table 14 The results show that although providing bounding box coordinates also an intuitive prompting method, it does not lead to noticeable performance improvement. This is because it is challenging for the MLLMs to precisely apply the coordinate information to the video, and coordinates require significantly longer text context, demanding higher parsing capabilities from the model. This phenomenon further validates the reasonableness and effectiveness of our method in terms of engineering implementation and model adaptation.
D DISCUSSIONS ON COMPUTATIONAL OVERHEAD
Performance is common engineering challenge for many current training-free enhancement mod- ules and workflows, such as InstanceCap (Fan et al. 2024a) (oveR'25,, VideoTree (Wang et al.
2025b) (CvPR/25), and VideoAgent (Fan al. 2024b) (BOCV"34).
20

---

Published as a conference paper at ICLR 2026
Table 21: Taking 1080p input an example, performance testing across different frame numbers. Each setting is tested 5 times and the average is reported.
Method
FAVOR- Bench
8 frames,
frames, 1080p
32 frames,
1080p
1080p
Time
Mem
Time
Mem
Time
Mem
Owen2.5-VL-70
42.3
8.15s
27.8GB
15.5s
28.2GB
31.0s
29.0GB
Qwen2.5-VI.-72B
39.2s
320.8GB
93.1s
322.9GB
207.7s
330.4GB
MotionChat-7B (Ours)
48.3
7.14s
27.8GB
13.5s
28.2GB
32.
29.0GB
To address this, we propose an alternative solution that achieves superior performance on fine-grained motion understanding tasks with a computational cost equivalent to the native Qwen2.5VL. LL model. This solution involves fine-tuning the model on our meticulously curated dataset. Detailed computational time and memory overhead are shown in the Table 21 demonstrating that our approach not only outperforms the 72B results in effectiveness but also approximately 5-6x faster. Meanwhile, we have optimized the MotionSight pipeline, reducing the average increase in inference latency to less than 75%.
In summary, for users who seek the highest performance, we' ve optimized the MotionSight pipeline, which specific and valuable workflow improvement; for users who prioritize efficiency, we offer MotionChat- end-to-end model fine-tuned on dataset, with computational cost comparable to the base model.
E DATASET FILTER AND CURATION FOR MotionVid QA
In this section, focus the methodological aspects and specific steps involved in filtering and curating the instruction and preference subsets for SFT and DPO, providing supplementary details to the dataset construction process of MotionVid described in Section Figure 16 showcases the examples we filtered.
E.I INITIAL DATA COLLECTION AND PRE-PROCESS
MotionVid - QA was aggregated from multiple sources (Section 4 resulting in initial set of video clips, denoted as CR. These clips under- went an initial data processing pipeline. Penatiol. The pipeline Pinitial filters the raw clips Cn such that only clips satisfying specific quality metrics are retained. Specifically, clip E CR included in the pre-filtered set Cp its optical flow score (c) is above threshold Ty AND its clarity score se(c) is above a threshold тe. The set of pre- pre-filtredd clips is thus defined as:
Low Clarity
High Clanty
Motion
Notroabie Mation
Figure 16: More examples of MotionVid - QA We filtered out low-quality videos and kept the high-quality ones. Our MotionVid QA includes varied scenes, subjects, and camera perspectives.
Cp = =c CR sy(c) Ty A As((c) > Te)
(8)
E.2 DETAILED FILTERING AND SUBSET CREATION FOR SFT AND DPO
The methodology for creating SFT and DPO subsets from the pre-filtered set Op is introduced in Section of the main paper.
of
Initial selection and annotation. A subset of clips was chosen from Cp for annotation with MotionSight. Let CA denote the set of successfully annotated clips.
21

---

Published as a conference paper at ICLR 2026
Annotation quality-based categorization. Each clip C CA was evaluated using VQAScore Lin ct al. (2024). To ensure the rationality of the chosen VQAScore thresholds for categorization (detailed below and Table 22). we manually checked multiple samples at the boundaries of these thresholds. Based on this evaluation, the clips were categorized into three distinct groups and the text-video visualization results satisfying different threshold conditions are shown in Figure T7
High-Quality Clips (C#): These clips were designated as high-quality and served as candidates for the DPO dataset. To ensure scenario diversity and account for varying annotation precision across different original data sources, clips were selected if their VQAScore Lin et al. 2024) exceeded specific threshold тj defined for its original data source j and motion aspect ls (e.g., & E {" ('object', 'camera' }). These VQAScore thresholds Ty,k are detailed Table 22 The set Cи is formally defined as:
C# (c E Ca Exceeds [QAScone)hressholllcc)
where Exceeds WWA.SoreeTrreeeeeeeoeoooeeee aspect is greater than the specific threshold J for that source-aspect pair, given in Table 22]
holds if the VQAScore of clip (from source j. for
Low-Qualin Clips (Cr,): Clips that failed to meet the minimum quality criteria were elimi- nated. This includes clips whose VQAScore below human-set minimum threshold TuL = 0.3. The set CL is defined as:
CL
{c E CA VQAScore(c) < 0.3)
(10)
Instruction Dataset Clips (Cs): The remaining clips formed the SFT instruction dataset. This set is defined as:
Cs = CA (CH U..)
(11)
SFT dataset construction and question types. The SFT dataset is constructed using the Instruction Dataset Clips (Cs). For each clip in Cs, we generate question-answer pair to fine-tune the model's ability to understand and describe motion. To cover diverse aspects of motion understanding, categorize our questions into three types: Object-centric Camera-centric, and Mixed-focur During the SFT data generation, one question randomly selected from the pool questions corresponding to the primary motion aspect (object, camera, or mixed) identified in the clip's annotation. These SFT dialogues (question-and -nnswer pairs) are crucial as they also form the foundation for constructing the preference data for DPO.
OBJECT-CENTRIC QUESTIONS: These questions focus on the movement, actions, and interactions of objects within the video. Examples include:
Object-centric Questions.
"What objects moving in this video?"
"Can you describe the motion of objects this video?"
"What is happening to the objects in this scene?"
"How are the objects moving in this video?"
"Describe the movements of the main subjects in this clip."
"What actions
being performed by the objects in this video?"
"How would you characterize the object motion
scene?"
"What kind movement do you observe from the objects in this video?"
"Describe the trajectory of the moving objects in this clip."
"How do the objects interact with each other in this video?"
COMERA-CENTRIE QUESTIONS:
These questions probe the camera's movement, techniques, and
perspective. Examples include:
Camera-centric Questions.
"How is the camera moving in this video?"
"Describe the camera motion in this video."
22

---

Published as a conference paper at ICLR 2026
High Condstency
The video opens with a close-up of person on yellow inflatable Tube navigating through rapids, with the camera slightly zooming copture the individual's movements, As people similar tubes join, the forming and floating together. The comera then zooms out to show the group moneuvering through the water, occasionally tilting and stabilize. The scene tronsitions to wider shot, with the
pons to the right, revealing the group
to follow their movements they right to capture the group moving through both turbulent and calmer sections of the river. The comera occasionally zooms in on individuals they navigate the ropids, then zooms out to show the group' dynamic movement. The camera follows the group downstream, using handheld mation to capture the spleshing water and rocky surroundings It pans to show individuals standing on rocks, possibly prepering to jump ossist others, before returning to focus the group fleating together. The video concludes with a straddy shot of group continuing downstream, interacting with the river's flow and each other, with comera maintaining wide view to encomposs the entire scene.
ponning left and
Standard d TV Consistency
The video begins with a person holding a white, multi-sided die in their hand, positioned in front of a child sitting on wooden purple shirt, seated and
deck. The die nemains stationary in the person's hand throughout the sequence. The child, dressed appears to be engaged in conversation or interaction with the person holding the die. The background remains consistent, showing the wooden deck and part of house. There throughout the frames, incicattng a static scene focused on the interaction between the person and the child.
significant changes in the positioning or actions of the objects or individuals
Low TV Consisency
The video features person stonding in a stadium with pink seats and green field in the bac background. Throughout the video, the individual remoins stationory, facing forword. The background consistently shows bonners and advertisements, significant changes or movements occurning in the environment. The person's posture and pasition do not alter across the frames, maintaining a steady and static scene.
The results corresponding to our three different thresholds are presented separately. Top: High consistency between text and video. The camera movement and changes in viewpoint strictly described in chronological order, resulting in extremely high quality. Middle: Fairly good consistency between text and video. The actions of the main characters are described with reasonable accuracy, but some imprecise areas exist. Bottom: Relatively poor consistency between text and video, providing limited or erroneous information.
Figure 17:
"What camera techniques are used in this video?"
"Is the camera stationary or moving in this clip?"
"How does the camera angle change throughout this video?"
"What kind of camera movements
you identify in this footage?"
"How would you characterize the camera work this video?"
"Does the camera follow any specific subject in this video?"
"What perspective does the camera provide in this scene?"
"How does the camera movement contribute the viewing experience?"
LIXED-FOCUS QUESTIONS: These questions require comprehensive understanding of the inter- play between object motion and camera work. Examples include:
Mixed-focus focus Questions.
"Describe the primary object's specific action, including its fine-grained motion. How does the camera's 's movement (e.g., tracking, zoom, pan) follow or frame this object's action, and what are the object's i's key visual attributes highlighted by this interplay?"'
"Considering the primary object's 's movement and its interaction with other elements, what is its implied goal? How does the camera's perspective (e.g.. close-up, wide shot, pfint-of-vieet view) and any dynamic changes its movement contribute "Analyze significant change in the primary object's motion or behavior. How does the camera's operation (e.g.. a sudden zoom, a switch to slow motion, a change in focus) coincide with and emphasize this specific change in the object's action?"
obscure this implied intention?"
"Discuss the overall pattern of the primary object's movement throughout key segment of the video. Correlate this with the dominant camera movement strategy used that segment. How does this
23

---

Published as a conference paper at ICLR 2026
J5ONL Fil:: jod
Open.SOM
Sartieeume
Prevous
enkaseed
(Right Arrow)
Use LEFT ARROW select Optice RIGHT ARROW oelect Optinn Phess to loggle Milowenn ESC est
Figure 18: Interactive annotation interface for DPO focused fine-grained video motion under- standing. This Python-based front-end allows annotators to choose between two textual descriptions ("Option A and "Option B") for the same video clip, selecting the that more accurately captures the nuanced motion in the video. The interface supports loading data in JSONL format and records annotator preferences, thereby providing data for the model's preference learning.
Table 22: VQAScore Thresholds (Ty.k) for High-Quality Clip Selection (CH), per Source and Motion Aspect. A clip from source j and aspect k included CH if VQAScore(c) > Tj.k-
Data Source (j)
Object Motion (Tjjabject)
Camera Motion (j.cmerr))
Kinetics-700
0.75
0.70
ActivityNet
0.75
0.72
Charades
0.72
0.70
Charades-Fgo
0.72
0.70
SSV2
0.68
0.68
Open Vid-IM
0.70
0.70
combined object-camera choreography affect scene's narrative the information conveyed about the object's activity?"
Preference dataset construction. (Wi,yf ychosen 3/ reject). These pairs were generated from the high-quality clips in Ch. The process involved re-annotating these clips using Tarsier2 Yuan et al. (222)) and then incorporating human preference signals, as illustrated in Figure 5 of the main paper and more details in Section
The preference dataset, consists of preference pairs of the form
This rigorous, multi-stage curation methodology ensures the high quality of the MotionVid QA sub- sets, which are crucial for robust model training and evaluation in fine-grained motion understanding.
F GUIDELINES FOR HUMAN PREFERENCE ANNOTATION
For DPO, annotators chose between two textual descriptions for a video clip. selecting the one that better captured its fine-grained motion. To ensure faimeess, the order in which these two descriptions were presented was randomized. The following guidelines ensured consistent, high- quality annotations: To facilitate this process, we developed interactive, python-based front-end for
24

---

Published as a conference paper at ICLR 2026
Table 23: Key criteria for human preference annotation in selecting textual descriptions of fine-grane tine-grained motion. Annotators chose the description that better satisfied these aspects.
Criterion
Guideline for Selection
Key Questions
1. Accuracy
Prefer more accurate identifica-
Core action correctly identified?
tion & description of primary
Agents/obbects in motion correct?
motion(s).
Avoids misinterpreting actions?
2. Granularity
Prefer more fine-grained & de-
Complex movements broken down?
tailed account of motion, captur- ing nuances.
-Specific body/object details?
-Overly general or specific?
3. Temporal Dynamics
Prefer better capture of tempo-
Sub-actions order correct?
ral aspects (sequence, duration, speed, rhythm).
Pace/intensity conveyed?
Speed/tempo changes reflected?
4. Camera Movement
Prefer description that accurately identifies & describes significant camera movements (e.g., pan, tilt, zoom, tracking).
Camera movement (pan, tilt, zoom, dolly, static) correctly identified? Effect of camera movement on scene understanding clear?
Distinguished from object motion?
Factual Correctness
Prefer response factually grounded in visual evidence,
Only visible elements/actions?
Contradicts visual information?
hallucinations.
Infers unohservable intent?
user-friendly annotation, shown in Figure T8 The relevant code can be found in the supplementary materials.
Annotators
selected the preferred response based holistic evaluation of the criteria in Table 23 prioritizing superior understanding and articulation of fine-grained motion.
Annotators
advised review clips multiple times and compare descriptions against these criteria. When responses excelled in different areas, they selected the one most helpful for understanding fine-grained motion.
Furthermore, we randomly selected 100 samples for independent re-annotation by different annotators. The calculated Cohen's coefficient is 0.731. This demonstrates high consistency in qualitative annotation tasks, and our approach achieves clearly usable level.
G FAILURE CASE
In scenarios where clear visual effects cannot be obtained, such as low-light conditions or extreme weather (e.g., heavy fog or torrential rain), the performance all vision- perponn-eeeptton-bbeee based MLLMs will degrade, including current SOTA models and our method.
H LIMITATIONS
Given that the accuracy of MotionSight is partially dependent on object detection methods, it requires fine-tuning the detection model for dppain-specific instances, and its benefits diminish in scenes without objects. Furthermore, we plan to expand our MotionVid QA and improve its quality to train more powerful fine-grained motion understanding models to enhance its impact.
I LLMS USAGE
LLMs were used as general-purpose assist tool to refine the writing of this paper. This usage was strictly limited to improving the clarity, grammar, and overall readability of the text. Specifically, LLMS were employed to rephrase sentences, suggest alternative wording, and correct grammatical errors.
25


Reviews:
[{"review_id": "pS1BdOI9TL", "reviewer": "Reviewer_p68Z", "summary": "The paper presents MotionSight, a zero-shot visual prompting framework that enhances MLLMs’ fine-grained motion understanding through two complementary visual cues, Visual Spotlight for object motion and Motion Blur for camera motion.\nBuilding on this, the authors further construct MotionVid-QA, a large-scale video QA dataset automatically annotated by MotionSight and refined via multi-stage filtering and human preference alignment. Experiments demonstrate consistent zero-shot improvements on MotionBench and FAVOR-Bench, and show that fine-tuning on MotionVid-QA (MotionChat) can yield additional gains with competitive efficiency.", "questions": "Just curious whether the authors tried applying MotionSight on top of the fine-tuned MotionChat model to see if it yields additional improvements. This could establish a potential self-improving VLM pipeline.\n\n---\nOverall, I’m borderline on this submission (currently rating it 4, since no borderline is available, and because I still have some concerns and would like to see the authors’ reply and additional results). I’d be happy to update my rating if my questions are well answered.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is well-motivated and clearly written, addressing an important gap in motion understanding for MLLMs.\n\n- The proposed visual prompting (spotlight + motion blur) is intuitive and effective, yielding consistent zero-shot gains.\n\n- The experiments are comprehensive, and the curated MotionVid-QA dataset provides a useful resource for future work.", "weaknesses": "- **Limited novelty of Visual Spotlight.**\n   The proposed visual spotlight is conceptually similar to existing image-level attention prompting methods (e.g., *Attention Prompting on Image for Large Vision-Language Models*, ECCV 2024), which also use soft masks to highlight salient regions. A comparison with such methods in Table 6 would strengthen the claim of novelty.\n\n- **Potential ambiguity in Motion Blur.**\n   Although the temporal weighting distinguishes different frames, the resulting blur may introduce directional ambiguity. For instance, in Figure 4 (bottom right), both lower-left and upper-right motion directions could be plausible, suggesting that motion blur may not always provide a reliable cue.\n\n- **Dataset quality and generalization concerns.**\n   MotionVid-QA is generated using MLLM-based annotations (Qwen2.5-VL-7B), meaning its quality and difficulty are inherently tied to the model’s capacity. While filtering ensures correctness, it does not increase complexity, so the dataset might mainly contain simple QA pairs. As a result, MotionVid-QA may improve weaker models but offer limited benefit for stronger ones (e.g., InternVL3-78B). Evaluating SFT effects on multiple models would better validate its generality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qaIJaFeznp", "reviewer": "Reviewer_9ap8", "summary": "The paper introduces a novel zero-shot method called MotionSight to enhance fine-grained motion understanding in MLLMs without requiring additional training. It addresses the limitations of existing MLLMs in perceiving subtle inter-frame dynamics by decoupling object and camera motion using visual prompting techniques. Specifically, it applies a visual spotlight to highlight moving objects and introduces motion blur to emphasize camera movements. To support training and evaluation, the authors present MotionVid-QA, the first large-scale open-source dataset for fine-grained video motion understanding, containing ~40K video clips and ~87K question-answer pairs with hierarchical annotations. Extensive experiments show that MotionSight significantly improves performance on benchmarks like MotionBench and FAVOR-Bench, and when used to fine-tune MotionChat, it achieves competitive results with much larger models.", "questions": "1. Can the method improve performance on the spatial benchs?\n2. Why choose to use the detection model instead of enhancing MLLM detection capability through RL?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a novel zero-shot method, MotionSight, which is the first to apply visual prompting techniques specifically tailored for fine-grained video motion understanding. This includes the innovative use of a visual spotlight to highlight moving objects and motion blur to emphasize camera movements, both of which are unique adaptations from static image prompting.\n2. The idea of decoupling object and camera motion is novel and addresses a significant gap in current MLLMs, which often struggle with subtle inter-frame dynamics.", "weaknesses": "1. More Experiments: Can you provide more results on video perception benches, such as mvbench, TOMATO bench, etc.\n2. The accuracy of MotionSight is partially dependent on object detection methods. This means that the performance of the model can be significantly affected by the quality of the object detection algorithm used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FG6WNwNOnZ", "reviewer": "Reviewer_aBZB", "summary": "This paper proposes MotionSight, a zero-shot approach to enhance fine-grained motion understanding in multimodal large language models (MLLMs). The method decouples object and camera motion, introducing visual spotlight and motion blur as visual prompts to improve perception of action details and camera changes. Additionally, the authors construct the first large-scale open-source dataset, MotionVid-QA, containing 40K videos and 87K Q&A pairs for SFT and DPO training. Experiments show MotionSight outperforms existing open-source models on several benchmarks and rivals commercial models on certain metrics.", "questions": "1. What is the failure-mode policy when Grounding DINO or SAM2 misses or hallucinates boxes? When a video contains both salient object motion and complex camera motion, why route to a single path instead of feeding both augmentations simultaneously?\n2. What is the head-to-head win-rate between MotionSight captions and Tarsier2 captions in the preference round? Please report overall and per-motion-type ratios.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Clear problem definition and strong motivation: Addresses a notable shortcoming of MLLMs in fine-grained motion understanding.\n2. Simple yet effective method: As a zero-shot approach, MotionSight improves model performance without training, offering broad applicability.\n3. Notable dataset contribution: MotionVid-QA is the first large-scale open-source dataset focused on fine-grained motion understanding, with high-quality annotations and significant community value.", "weaknesses": "1. Conservative innovation: While extending visual prompting to video is useful, \"spotlight\" and \"motion blur\" are traditional enhancements lacking methodological breakthroughs. Although the motion decoupling strategy is reasonable, the process of \"detecting first and then focusing\" relies on existing detection/tracking models. If the detection fails or is missed, the effect will be greatly reduced.\n2. Limited generalization analysis: Insufficient discussion of failure cases in complex scenes (e.g., multi-object interactions, low-light, non-rigid motions like smoke or water).\n3. Dataset distribution and quality: MotionVid-QA is largely sourced from existing human-action datasets, potentially limiting diversity; non-human or synthetic motion videos are underrepresented. Human preference annotation introduces subjectivity; no inter-annotator agreement or statistical significance reported.\n4. Incomplete efficiency analysis: Mentions ~75% latency increase but lacks detailed time/memory profiling across resolutions or frame counts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h5DTxl7t3t", "reviewer": "Reviewer_Am5g", "summary": "This paper propose MotionSight, a novel visual prompting method for fine-grained video motion understanding by tailored decoupling object and camera motion. The experiments reveal the enhanced performance for fine-grained motion understanding without additional training data. The authors also collected and annotated the first large-scale fine-grained video motion understanding dataset MotionVid-QA.", "questions": "1.See weakness.\n\n2.Though the ablation demonstrates strong robustness to several hyperparameters, I wonder whether the proposed method is sensitive to the parameters of selected models of MLLM.", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.Fine-grained video motion understanding is under explored and worth studying. The paper analyze this task from both method and data sides by designing tailored model and constructing a valuable dataset.\n\n2.The experiment result is solid with high performance on several evaluation benchmarks for fine-grained video motion understanding.\n\n3.Convincing visualization examples are provided to prove the effectiveness of the proposed method.", "weaknesses": "1.The overview of the framework and proposed framework may raise the concern about the processing cost as it contains many designed modules compared for fine-grained  video motion understanding task compared to single MLLM methods. I encourage the authors to introduce the training/inference cost and speed compared with the provided baseline methods in more detailed.\n\n2.Although the performance of the model has improved to some extent after training with MotionVid-QA, the results in Table 4 have certain limitations. The author may compare the results of the model trained on datasets of similar size.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)