Video-Skill- C o T : Skill-based Chain-of-Thoughts for Domain-Adaptive Video Reasoning (2025)

Abstract: Recent advances in chain-of-thought (CoT) reasoning have improved complex video understanding, but existing methods often struggle to adapt to domain-specific skills (e.g., temporal grounding, event detection, spatial relations) over various video content. To address this, we propose Video-Skill-CoT (aka Video-SKoT) a framework that automatically constructs and leverages skill-aware CoT supervisions for domain-adaptive video reasoning. First, we construct skill-based CoT annotations: We extract domain-relevant reasoning skills from training questions, cluster them into a shared skill taxonomy, and create detailed multi-step CoT rationale tailored to each video question pair for training. Second, we introduce a skill-specific expert learning framework. Each expert module specializes in a subset of reasoning skills and is trained with lightweight adapters using the collected CoT supervision. We demonstrate the effectiveness of the proposed approach on three video understanding benchmarks, where Video-SKoT consistently outperforms strong baselines. We also provide in-depth analyses on comparing different CoT annotation pipelines and learned skills over multiple video domains.

## introduction understanding complex video content requires integrating rich spatiotemporal cues and adapting to diverse domain-specific reasoning needs from cinematic narratives, egocentric recordings, to indoor scenes @xcite @xcite @xcite . models should acquire and integrate a wide range of distinct reasoning skills, such as temporal grounding, spatial relationship recognition, and multi-step planning. ## related work video understanding with mllms. prior video understanding models focused on pretraining strategies @xcite @xcite . recent work incorporates cot reasoning @xcite ) from the nlp domain and studies how to collect and learn to generate such cot reasoning for different video understanding tasks @xcite @xcite . unlike these methods, which often struggle with comprehending videos without explicit skill-specific guidance, our approach introduces a skill-aware reasoning framework incorporating question-adaptive skill selection and skill-guided cot supervision. skill-specific expert learning. modular and expert-based architectures have been widely explored to improve parameter efficiency and mitigate interference in multi-task and multi-domain settings, where each expert learns different knowledge. mixture-of-experts (moe) frame-works dynamically route inputs to expert subnetworks @xcite , while adapterbased methods introduce lightweight, task-specific modules into pretrained models @xcite . @xcite studies learning skill-specific expert diffusion models for the text-to-image generation task. a concurrent work, @xcite studies a multi-agent system where each agent is implemented as a lora @xcite expert. while @xcite relies on predefined expert roles (planner, grounder, verifier, and answerer), specific architectures, and manually curated role-specific annotations, our expert framework flexibly adapts to any video understanding dataset by automatically discovering and leveraging relevant reasoning skills. ## conclusion we propose video-skot, a novel video understanding framework for effective domain adaptation of mllms. we propose to automatically collect skill-specific cot annotations from video qa datasets and construct a skill-based reasoning pipeline that combines a lightweight skill assigner with a collection of lora-based expert adapters. empirical results on three diverse benchmarks demonstrate consistent gains of video-skot over strong baselines, highlighting the enhanced quality of our reasoning traces.