MetaCaptioner: Towards Generalist Visual Captioning with Open-source Suites
Authors: Zhenxin Lei
Zhangwei Gao
Changyao Tian
Erfei Cui
Guanzhou Chen
Danni Yang
Yuchen Duan
Zhaokai Wang
Wenhao Li
Weiyun Wang
Xiangyu Zhao
Jiayi Ji
Yu Qiao
Wenhai Wang
Gen Luo
Venue: ICLR 2026 Poster (2026)
Keywords: Visual Captioning
Multimodal Large Language Model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Generalist visual captioning goes beyond a simple appearance description task, but requires integrating a series of visual cues into a caption and handling various visual domains.  In this task, current open-source models present a large performance gap with commercial ones, which limits various applications such as data synthesis.  To bridge the gap,  this paper proposes CapFlow, a novel multi-agent collaboration workflow.  CapFlow demonstrates for the first time that, by capitalizing on open-source models, it is possible to achieve caption quality on par with GPT-4.1 in various domains with an 89.5\% reduction in costs.   By leveraging CapFlow as the data synthesizer, we produce high-quality visual captions from image and video domains at scale, and obtain a generalist visual captioner via fine-tuning, namely MetaCaptioner. Through extensive experiments, we show that MetaCaptioner not only achieves comparable captioning capabilities with commercial models but also reaches top-tier multimodal performance in the open-source community. We hope CapFlow and MetaCaptioner can benefit future multimodal research by providing a strong and cost-effective visual captioning solution. Our source code and models will be publicly released.

Published as a conference paper at ICLR 2026
META CAPTIONE:: TOWARDS GENERALIST VISUAL CAP- TIONING WITH OPEN-SOURCE SUITES
Zhenxin Lei Zhangwei G Changyao Tian 8.1, Erfei Cui
Guannhhu Chen 2.1
Danni Yang"., Yuchen Duan6.1, Zhaokai Wanga1, Wenhao Lis.1, Weiyun Wang 3,1 Xiangyu Zhao?.,, Jiayi Ji Yu Qiao', Wenhai Wang".,, Gen Luo'
Shanghaii AI Laboratory SShangaii Jiao Tong University FFudan University
"University of Chinese Academy of Science Xiamen University
TThe Chinese University of Hong Kong
ABSTRACT
Generalist visual captioning goes beyond simple appearance description task, but requires integrating a series of visual cues into a caption and handling various visual domains. In this task, current open-source models present a large performance gap with commercial ones, which limits various applications such as data synthesis. To bridge the gap, this paper proposes CapFlow, a novel multi-agent collaboration workflow. CapFlow demonstrates for the first time that, by capitalizing on open- source models, it is possible to achieve caption quality on par with GPT-4.1 in various domains with an 89.5% reduction in costs. By leveraging CapFlow as the data synthesizer, we produce high-qualing visual captions from image and video domaíns at scale, and obtain generalist visual captioner via fine-tuning, namely MetaCaptioner. Through extensive experiments, we show that MetaCaptioner not only achieves comparable captioning capabilities with commercial models but also reaches top-tier multimodal performance in the open-source community. We hope CapFlow and MetaCaptioner can benefit future multimodal research by providing strong and cost-effective visual captioning solution. Our source code and models can be found at htttpss: //q1 thub. wttpc////peeeeilaa//eeetaaaaaaaa onner
GPI-4.11 V: Certainly! Her is detailed description and analysis of the image you pol lllath Tide: "he dack aaaeerallaa Overiiw:ialhe infographic visually presents the valu difeeent type seles millions
Metrics
Factual
Profesdon
sujor durk entegortescinint. **illegal drage** (yrllaworaagee circlesja2.. "*Prseription drags** (Muue-gren circles)in3. "*Non-drugs* ** (pin pupppe cireles) in'o Fach category is represented by chastr of circle, where the size each citcle is propurtional to the sales value for that specific teem grop Detrildd [-)
Accurncy:
marketplaces over specified period. [-.] chart divided into
Esllonr
Good
Infarmation
Reasening
Completenes:
Exilews
Exflent
OmniCaptioner: The inforrapic titled The Dark Webl" presents data en sales across various platforms such Silk Road 2, Evolution, and Agara from December 2013 to Jnly 2015. [-] it divided inte main rections: one for illezal drugs another for drags, winthh [.Jillegal Drugs Seccion's ""Cocains*", $5.2 millien'n- *MMarijsaaa** Bestasy**; 57.7 cntegoryYe-[-. **plls** millionin- "Furabppins* $1 millloni's **sSs** $1.5
Mecrics
Bepi
150w
Accuracy:
-alism::
Bat
Good
larges:
Information
Reasoning
millionin: **Bloner Toby $1.4 millioo'n *DDM**> 50.4 millionin **Ketamine** 50.8 millionin- "Appanooom"** ** *Vaaiu""].
Completenens
Rigor:
Good
Mattonationer: The inage is bubble chart that visually encapsulates value of sales on the dark web markerpheees Silk Road 2, Evvolution and Agorn, spanning from December 2013 July 2015. The chart's purpose is to illustrate the economic scale and diversity of illlgal and non illegal goods traded these platforms. [..] composed drups, non-drugs,
Metrics
Factual
Prefession
Proccidia 144
Arcuracy:
-alism:
Eallent
Exflent
three categories:
monetary value prescription drags.
catogory subeategoey, color scheme of dark gray [.. The chart's design allows a quick and easy understanding of the data, making an ellictive wol for communicuiing comples information. [..]
varying sizes to repeesent
Information
Compliencee
Rigors
Reasoning
Eallent
Ealleni
Natural Porception
Infographic & Document
Chart Math
Medical Hio-imaging
Code Programmming
Knowledge Education
Video Temporal
Symthetic
Interaction
Image
Visual Caption Domains MetaCaptioner
Figure 1: Comparison of caption quality between GPT-4.1, OmniCaptioner (Lu al. 2025 and MetaCaptioner. These metrics are computed by GPT-5, MetaCaptioner is able to generate informative captions for a variety of visual domains with comparable quality to GPT-4.11.

---

Published as a conference paper at ICLR 2026
1 INTRODUCTION
Visual captioning, a fundamental task in computer vision, aims to describe images or videos using natural sentences [Wang al. all, 2020, eu al. 2015). Over the past decade, this field has received an influx of interest in the community (Chen et al. 2004bb Li et al. [024d] You et al., [016 Anderson ct al., 2018), with the majority of research focusing on natural scene description (You et al., 2016
Yao et al. 2017. [Anderson et al. 2018). With the advent of multimodal large language models (MLLMs) (aang et al.) 2025 2024b; Li et al. 2024a), visual captioning has evolved beyond a simple multimodal task and now plays increasingly crucial role in model pre-training (Wang
ct al. 2025; Betker et al. 2023) and data synthesis (Chen et al. 2202bb Li et al. 2024d: Yu et al.,
2024 This progression imposes higher demands on existing visual captioners handling more
diverse and complex scenarios, such as mathematical figures (Zhang et al. 2022bb Lu et al. 2023h),
diagrams (Mathew al. 2022), and documents (Mathew et al. 2021).
To overcome this limitation, increasing efforts have been devoted to producing informative visual captions. Among them, a promising method is to design domain-specific prompt and leverage the advanced commercial MLLM, e.g., GPT-4 (Hurst et al. 2024)) to generate detailed captions (Chen al. 2024b). While effective, such approach becomes prohibitively expensive as the data scale increases. Recent attempts also explore open-source MIL.Ms to synthesize the generalist visual captions (Xing et al. 2025; Lu et al., 2025), yet their caption quality still lags far behind that of commercial ones due to the limited capabilities. Therefore, one critical issues naturally raised: Is possible to bridge the gap in generalist visual captioning through open open-source suites?
To answer this question, we observe that the main difficulty of generalist visual captioning with open-source MLLM lies in the large domain gap. In particular, most MLLMs excel at natural scenes, and they tend to describe the subject, texture, and object relationship in an image. Nevertheless, the description requirements become quite different and challenging in other visual domains. For instance, for knowledge-based scenario, e.g., math, the description should not only contain visual appearance, but also the related visual knowledge and underlying visual logic. As shown in Fig. open-source MLLMs like OmniCaptioner (Lu et al. (2025) struggle to seamlessly integrate these capabilities into one description, often lacking fine-grained details in some aspects.
To address this issue, we propose CapFlow, novel multi-agent collaboration workflow for produc- ing generalist visual captioning with open-source MLLMs. The main principle of CapFlow is to decompose visual captioning into sub-taske and synthesize evidence from diverse aspects. As shown in Fig. 2 CapFlow consists of a hierarchical workflow, where each agent performs as a distinct role perception, visual knowledge extraction, visual reasoning, summary, etc. CapFlow, visual descriptions from different perspectives are produced by the corresponding agents and ultimately aggregated into a single caption. To further climinate the domain gap, we design a domain routing mechanism to dynamically assign the suitable workflow of CapFlow for different visual domains. Through close collaboration, CapFlow can achieve general visual captioning capabilities comparable to the top commercial models i.e. GPT-4.1 (OpenAl 2025a), at only 10.5% of the cost.
Based on CapFlow, we aim to equip generalist visual captioning capabilities to common MILMs (Wang al. 2025), thus yielding an efficient and general visual captioner, namely MetaCap- tioner. In particular, we leverage CapFlow as a strong data engine to produce informative captions for images and videos from various domains, with strict reject sampling pipeline filter out low-quality captions. Thanks to the cheap yet effective pipeline of CapFlow, we can efficiently scale up the data size to 4.1 millions, enabling a seamless convert of the open-source MLLM into Meta- Captioner through fine-tuning, Thanks to the high-quality captions from CapFlow, MetaCaptioner not only demonstrates powerful visual captioning capabilities a lower cost (0.7% of GPT-4.1), but also achieves top-tier multimodal performance against existing open-source MILMMs.
To validate our approach, we conduct extensive experiments on 13 benehmarks of two common settings: (1) Visual reasoning with large language models (LLMs) (Lu et al.] (025):
: The detailed captions generated by CapFlow and MetaCaptioner are used as input to LLM for visual reasoning. thus diagnosing visual caption capabilities. (2) Downstream evaluations (Wang et al., 2025 2024b Xie
al. ) We directly evaluate the performance of MetaCaptioner on understanding and reasoning tasks across various domains, thereby validating the benefits of training with generalist caption data. Our experimental results show that our MetaCaptioner, even with a lightweight parameter size, can already achieve comparable captioning capabilities with the commercial model GPT-4.1 (OpenAl
2

---

Published as a conference paper at ICLR 2026
MMV-
MichVista
A12D
05.0
03.5
Data Pool
Data synthesis
Captions
VQA
Router
Grounding
WorkFlow
OCR
Pereciver
Reasoner
MMMU
MMBench-Y
Vidko-MMMF
Summarizer
25.1
$4.8
1.25 125
1:23
MetaCaptioner
0.92
Captions
8B
CapFlow Framwork
MetaCaptioner
CPT-4IV
Carffon
DerrCCaptonnn-53
MesCopskaee (rard)
Figure 2: Comparison of existing captioning systems with our our methods. Our CapFlow adopts multi-agent collaboration workflow to produce high-quality captions for training MetaCaptioner 8B. Under the setting of visual reasoning with L.Mss, CapFlow and MetaCaptioner-88 achieve comparable performance with GPT-4.1 using open-source suites.
2025ah on several benchmarks, and its downstream performance also benefits from the synthetic data from CapFlow, e.g. +3.7 on MathVerse Verse. CapFlow and MetaCaptioner provide cheap and strong visual captioning solution that will potentially benefit future multimodal research. summary, our contributions three folds:
We propose CapFlow, an innovative multi-agent collaboration workflow for producing generalist captioning various image and video domains. CapFlow is the first open-source framework that reaches comparable captioning performance with GPT-4.1.
By leveraging CapFlow as the data synthesizer, we present the strongest open-source visual captioner the community, namely MetaCaptioner. MetaCaptioner not only possess powerful generalist captioning capabilities, but also maintains top-tier multimodal performance.
We conduct extensive experiments to ablate CapFlow and MetaCaptioner, providing invaluable hints for future research. In addition, source models and codes will be publicly released, further advancing the development generalist visual captioning and the broader multimodal research.
RELATED WORK
2.1
MULTIMODAL LARGE LANGUAGE MODELS
With the development of multi-modal large language models (MLLMs), visual-language alignment has attached more attention. Most approaches improve the alignment visual and language by pre-training large-scale image-text pairs (Radford et al. 2021 Alayrac et al. 2022 Li et al.
2022 2023). Subsequent research attempt to achieve this challenge through incorporating high quality data for pre- pre-training and supervised fine-tuning stage (Liu et al. 2023b Li et al. 2024a Guo ctt al. 20246; Zhu et al., 2023 Wang et al.] 22024b] Chen et al.) 2024cc Wang et al. (025). Some
models further introduce more fine-grained image-fext annotation data (Li al. 2024d 2025 Deng et al. (e.g grounding and mask annotations) to reduce hallucinations and strengthen visual grounding ability. Nevertheless, due to the domain coverage and annotation costs, constructing multi-domain, high-quality image-caption pairs remains a significant challenge.
2.2 GENERALIST VISUAL CAPTIONING
Recent progress multimodal community has significantly increased the demand for generalist visual captioning. In particular, existing methods can be broadly categorized into two categories. The first category often prompts powerful MLLMs (e.g., GPT-4o (Hurst al., (024) and GPT-4V (Achiam et al. 2023) through carefully designed prompts to generate high-quality captions (Chen et al. 2024b d al, 2025). Some works also adopt coarse-to coaisse-tt-ffngggree fine-grained optimization like introducing visually differentiated descriptions (Chen al. 2024d) and multi-turn caption optimization (Betker al. 2023 Xing et al. 2025] You et al. 2025). The second category centers on muuti-source caption synthesis, typically incorporating multiple domain expert models (e.g., SAM (Kirillov al. 2023
3

---

Published as a conference paper at ICLR 2026
Dirverse data sourcer
Siructure Domain
The imago detailed and educational diagram Cycle*", illustrating the continnous movement water betweeni ...]. eyele is depicted in circular flow, emphasizing its eyclical nature and interconnected processes. layout is organized into distinct sections representing stmospheric, terrestrial. subterranean components, with clear labels, arrows, and color coding to guide
Captions:
Water
Agent-1
Agent-M
Tisk-solving Layer Workflow
Cycle
Summnarizer
Domain Router
workflows
through each [..]The inclusion of e'lements like sus, tree, and mountain range adds depth the regrescntation. emplasizing the rales of [...] Overall, the image serves excellent educational tool teaching principles hydrology and environmental science.
Natural Domain
Agent
Agent-N
Task sulving Layer
Workflow
Floe Frumework
Summarizer
Workflows in CapFlow
Natural workflow
Document workflow
Chart & Math
UI workflow
Medical workflow
Code
Knowlodge workflow
Video workflow
Synthetic workflow
workflow
workflow
Agents CapFlow
Natural
Structure Perception Agent
Infogruphic
Video
General
Perception Agent
Perception A
Perception
Perception Agent
Reasoning
Agent
CoderAAgon
Medical
Knowledge
Video
General
Reusoning Agent
Reasoning Agent
Reusoning Agent
Summary Agent
Figure 3: Overview of CapFlow Framework. CapFlow can dynamically select an appropriate caption workflow for the imput image, and each workflow is equipped with multiple agents for close collaboration. With this pipeline, CapFlow can produce high-quality captions various visual domains with open-source MILMs, e.g., Qwen2.5-VL [Bai et al. 2025)
Uni (Guo et al. 2024a). and Paddle OCR (Cui al. 2025) and merging the domain expert results (Li et al. 2024d 2025 Wang et al. 2023a Lian et al.) 2025 Deng et al., 2025). However, both categories face limitations content granularity and comprehension depth, often exceeding the capabilities of open-source MLLMs.. Therefore, approaching generalist captioning with spen-source models remains significant challenge in the community.
3 CAPFLOW
3.1 OVERVIEW
A core challenge in generating diverse visual descriptions across different domains lies in the varying requirements for granularity and semantic depth (Xing et al.) 2025 Lu et al. 2025). In this work, we propose CapFlow. novel multi-agent collaboration framework for automated visual description. As illustrated in Fig. 3 CapFlow consists of two main parts: Domain Routing and Hierarchical Captioning Workflow The design of CapFlow features in: 1) high-quality caption generation that integrates various visual cues, 2) collaborative agent framework applicable to a wide range of visual
domains, and 3) cheap deployment cost and high scalability.
3.2
DOMAIN ROUTING
The motivation for the domain routing originates from an intuitive principle: various visual domains demand varying levels of cognitive complexity (Zhong et al. 2024, Chen et al., (025). Therefore, CapFlow needs to first determine the domain of the visual input and then assign a workflow suitable for visual captioning. To approach this target, we first adopt an MILM as a domain router, with a carefully-designed prompt to query the visual domain of images. This process is similar to solving visual muestion-answert our pre-defined domain based on the visual input and the prompt. For data with a known domain, can skip the domain routing part.
problem, where the MLLM should select the best option from
4

---

Published as a conference paper at ICLR 2026
Data synthesizer
Metrics
Accuracy:
Factual
Profession
-alism:
Excellent
Caption Pool
Caption Pool
Excellent
Excellent
MctaCaptioner -SB
CapFlow
Information
Reasoning
Bad
Completeness:
Rigor:
Discard
VQA
Excellent
Pool
Reject sampling
Training Engine
Figure 4: Training pipeline of MetaCaptioner. We leverage CapFLow as the data synthesizer to generate various captions, upon which strict reject sampling pipeline is used filter out low-quality ones. The obtained captions combined with common instruction data to train MetaCaptioner.
3.3 HIERARCHICAL CAPTIONING WORKFLOWS
Based on the routing results, we design different routing paths for visual captioning of corresponding domains, and each routing path is customized for a visual domain. As shown in Fig. 3 the routing path is designed as a hierarchical workflow, with two distinct layers for task solving and information summarization, respectively.
In particular, the task-solving layer aims decompose complex captioning task into several simple sub-tasks and perform divide-and -ndee--u-- through multiple agents. Each domain is equipped with distinct set of agents solve domain-specifee tasks, as shown in Tab. [T] Therefore, we design a variety of functional agents that are responsible for different tasks, where most of the agents are based LLMs. In particular, there be roughly divided into four categories: guideline, perception, reasoning, and tools. Specifically, the guideline agents aim provide an overview description of the visual input, mainly containing global information like style, structure, and geography. In contrast, the perception agents depict the fine-grained details of images and videos such textures, colors, and lighting. These two types of agents mainly focus on visual appearance. To compensate for high-level semantics, the reasoning agent responsible for capturing the underlying visual knowledge, logic, and relationships. In addition, we notice that some domains often require additional visual tools to accommodate their requirements. For example, the document-relatte data often requires OCR parsers to obtain textual words. Therefore, also design the tool agent to perform specific tools like the bas-dased parser and the code parser. The detailed prompts are provided in the Appendix.
total of 30 functional agents in the workflow, which can
The visual cues generated by the task-solving layer ultimately aggregated into caption through the information summarization layer. In this layer, an LLM agent aims to summarize the textual contents from the task-solving layer bne-by- analyze their contributions the final captions, and organize into an informative, detailed and structured caption.
METACAPTIONES
Based on Capflow, we aim to train a strong generalist captioning model. To this end, we first collect images and videos from various domains, then employ Capflow the data synthesizer generate massive high-quality captions, and ultimately obtain MetaCaptioner-8B through fine-tuning.
4.1 DATA COLLECTION
Data Source and Processing. We first collect approximately 70 million raw images and videos from 140+ open-source image and video datasets. These sources span wide range of domains, including natural images (e.g., SAM (Kirillov et al. 2023) and LAION (Schuhmann al. 2022 ), structured images (e.g., ArxivQA (Li et al. 2024c) and NovaChart (Hu et al. (0244), knowledge-basee images (e.g., PMC-VQA (Zhang et al. 2023a) and iNat (Cui et al. 2018)), and aesthetic images (e.g., PixArt (Chen et al. 2024a)). Due to the imbalance in different domains, we further collect large mount of structured images (e.g., charts, posters, GUI screenshots), text recognition images (e.g., documents, reports), and commonsense images (e.g., natural science, human activities). Following the protocol of (Zhu et al. 2025), we process these open-source datasets and obtain approximately 18 million structured images. To ensure the visual quality, apply filtering based image resolution
5

---

Published as a conference paper at ICLR 2026
Visual Domain
Functional Agent
Summary Agent
Natural
Natural Perception, General Reasoning. Visual Guideline
Structure & Math
Structure Perception, Infographic Perception, General Reasoning, Visual Guideline
Infographic & Document
Infographic Perception, OCR, General Reasoning. Visual Guideline
Medicul Bio-Imaging
Natural Perception. Medical Reasoning, Visual Guideline
General Summary
U1 & Interaction
UI Perception, OCR, General Reasoning
Code Programming
Coder, General Reusoning, Visual Guideline
Knowledge & Educution
Infographic Perception, Knowledge Reasoning. Visual Guideline
Synthetic
Texture Perception, General Reasoning, Visual Guideline
Video & Temporal
Video Perception, Video Reasoning, Video Guideline
Video Summary
Table 1:
The agentic workflow of various visual domains. We decomposite the visual description task into visual guideline, perception, reasoning, and tool task.
and aspect ratio (Xing et al. 2025 Li et al. 2024d 2025). retaining images with a short edge larger than 512 pixels, an aspect ratio less than and videos with resolution higher than 480p. As result, we construct a high-quality dataset containing 5 million semantically rich and diverse samples from broad range of sources. More details are provided in the Appendix.
CapFlow as the Data Engine. To accommodate data from diverse domains, we adopt CapFlow as the data engine for annotating the collected data construct the dataset called MetaCaption-SM For the functional agents, we adopt powerful open-source MI.MM Qwen2.5-V1L-725 as the functional agent and carefully design task-specific prompts to guide the agent toward fine-grain fine-grained visual perception and understanding. For the summary agent, we leverage the strong contextual processing capabilities of Qwen2.5-72B to integrate and summarize information from various functional agents, ultimately generating high-quality visual captions. The prompts for large-scale data synthesis are provided in the Appendix.
4.2 REJECT SAMPLING PIPELINE
As illustrated in Fig. 4 we further develop strict reject sampling strategy to filter out low-quality data from the MetaCaption-Sh dataset generated by CapFlow. Specifically, we employ carefully designed prompts to ask MILMSs to score the quality of each caption from multiple aspects. For the image modality, captions are evaluated across five dimensions: Factual Accuracy, Information Completeness, Reasoning Rigor, Core Intent Capture, and Professionalism. For the video modality, captions are assessed based on Temporal and Factual Accuracy, Event and Detail Coverage, Temporal Causal Logic, Core Intent, and Professionalism. To minimize the influence of the model's subjective judgment on the evaluation, we adopt a 3-point rating scale for scoring the results and only sample those that achieve professional levels (e.g., rating 3) across all sub-domains for subsequent model training. As result, we obtain a high-quality dataset called MetaCaption-4.1N.
4.3 TRAINING PIPELINE
Based MetaCaption-4.1M. we trained powerful multimodal large language model, namely MetaCaptioner-88. Specifically, MetaCaptioner-88 adopts InternViTT-00OM (Chen et al. 2024e) and Qwen3-8B-Instruct (Yang et al. (2025) as the vision encoder and the language model, respectively. In the pre-training stage, we follow Intern to perform native pre-training on various visual and text data (Wang al., 2025). For instruction tuning, we mix our generalist captioning data (MetaCaption-4.IM with instruction data from Intern VL3.5 (Wang et al.) 2025) to optimize the entire model. Training schedule and steps are kept consistent with Intern VL3.5 (Wang al. 2025).
5 EXPERIMENT
5.1 IMPLEMENTATION DETAILS
In CapFlow, we utilize Qwen2.5-VL-722 (Bai al. 2025) and Qwen2.5-72B Bai et al. 2025 as functional agents and summary agents, respectively. The entire annotation process requires approximately 480 H200 GPU days. For reject sampling, to provide the metric score, taking around 0.1 H200 GPU days. For pre- of MetaCaptioner, it is trained for one epoch with a learning rate of le-5 and a batch size of 256. For supervised
employ Qwen2.5-VL-7B
the judger

---

Published as a conference paper at ICLR 2026
Math
Math
Chart
Info
Video
Model
MMMU
MMVet
AI2D
Cost
Verse
Vista
QA
VQA
MME
GPT-4.1
55.7
61.7
56.8
65.0
62.3
63.2
75.5
26.8
$1.47
Baseline (7B)
50.7
47.2
42.1
57.6
54.4
49.0
64.5
23.9
$0.01
+ Hierarchical Workflow
51.6
48.6
44.0
59.0
57.8
44.2
66.0
26.1
$0.02
Domain Routing
54.7
50.5
43.9
58.7
58.0
45.0
67.4
26.2
$0.02
+ Scale up to 72B
55.1
57.8
53.1
62.5
59.2
50.2
74.2
27.6
$0.14
Table 2: Ablation study of CapFlow. Our baseline is Qwen2.5-VL-7B (Bai et al.
2025). Through
our careful design, CapFlow can achieve comparable performance GPT-4.1 (OpenAl 2025a) on most benchmarks, but at a much lower cost. We calculate the average inference cost on 100 samples.
fine-tuning (SFT), we train MetaCaptioner for 160k iterations with a learning rate of 2c-4. The complete training process takes approximately 192 H200 GPU days.
5.2 EVALUATION BENCHMARKS
We evaluate methods on 13 comprehensive multimodal benchmarks. Specifically, visual question answering benchmarks include Infographic VQA test (Mathew et al.) (002), ChartQA test (Masry
al. 2022). Document VQA (Mathew al. 2021). and A12D test (Kembhavi et al. 2016 MLLM
benchmarks encompass MMBench VI.1 (Liu et al. (2024), MMMU val (Yue et al., 2024 MMVet
YYu al., 2023), Vista fextmini (Lu al. 2023a), Math Verse vision only (Zhang et al. 2024b),
Merhe
SEEDBench2 Plus (Li et al.) 2024b), and MMStar (Chen et al. 2024c). Video understanding
benchmarks contain VidcoMME (Fu et al., 2025) and MMBBnnc Video (Fang et al. 2024). The
evaluation metrics follow existing methods using VLMEvalkit Muuan et al. 2024
5.3 EVALUATION SETTINGS
Caption Quality Evaluation. Similar to the metrics in reject sampling, we define five metrics that can comprehensively reflect the caption quality: factual accuracy, completeness, reasoning rigor, intent capture, and professionalism. For these metrics, GPT-5 to assign a score from (bad) to 3 (excellent) to each metric. Additionally, we conducted a thorough human evaluation to verify the aligmment of different judges. Please refer to the Appendix for more details.
design strict evaluation prompts for
Visual Reasoning with LLMs. In this setting, visual captioners will generate description the visual prompt for existing reasoning-enhanced L.LMs, enabling them to seamlessly perform various multimodal downstream tasks. This setting decouples visual understanding from textual reasoning, where downstream performance highly depends on the generated captions. Therefore, we can quantitatively diagnose the informativeness of captions from the downstream performance.
Direct Multimodal Evaluation. Direct multimodal evaluation is the most common setting to assess the capabilities of MLLMs downstream tasks (Wang et al. 2025 2024b). Through this setting, we aim to diagnose the impact of high-qualit, synthetic data (i.e., MetaCaption IM) on the multimodal capabilities of MetaCaptioner.
5.4
ABLATION STUDY
5.4.1 ABLATION STUDY OF CAPFLOW
In Tab. 2 we ablate the design of CapFlow and compare it with GPT-4.1 (OpenAl 2025a) under the setting of visual reasoning with LLMs. In this table, we first observe that starting from the Qwen2.5-VL--7B baseline, the introduction of the Hierarchical Workflow brings moderate gains on most benchmarks, e.g.,+1.4 on MMVet and +1.9 on MathVerse The subsequent integration of the Domain Routing categorize different domains yields more pronounced improvements, particularly on MMMU (+4.0 over baseline), MMVet (+3.3 gains), indicating its critical role in addressing domain gaps. Notably, scaling the base model of CapFlow to 72B parameters leads to substantial performance boosts across all tasks, nearing GPT-4,1 on several fronts: MMMU (55.1 vs. 55.7), Math Vista (62.5 vs. 65.0), and A12D (74.2 vs. 75.5). On VideoMME, CapFlow-72B even exceeds

---

Published as a conference paper at ICLR 2026
Pre-training
SFT Data
Math
Math
Chart
Info
Video MME
Model
MMMU
MMVet
AI2D
Average
Data
Verse
Vista
VQA
ShareGPT4V-550K
53.8
55.3
22.0
56.9
76.2
63.8
75.2
60.6
58.0
Intern VI1-600
DenseFusion-450K
Vanilla(3M
55.2
53.2
31.9
58.7
78.3
66.2
75.6
62.0
60.4
Qwen3-8B
MetaCaption-450k
56.1
62.5
26.9
61.2
80.0
68.3
77.5
62.1
61.8
ShureGPT4Y-4550
55.8
54.7
24.2
59.3
79.2
67.0
76.5
61.0
59.7
Intern VIT-600M
Vanilla IM)
DemseFusion 450K
55.2
62.1
27.1
59.0
78.4
68.3
77.2
60.7
61.0
Qwen3 SB
+MetaCaption 450K
56.9
62.8
27.9
60.9
79.6
68.4
77.4
61.7
62.0
The impact of synthetic captions generated by CapFlow on pre-training and fine- tuning. The vanilla data is randomly sampled from that of VL-335 (Wang al. is sampled from our synthetic data (MetaCaption
Table 3:
2025
Model
Factual Ace
Completeness
Reasoning Rigor
Intent Capture
Professionalism
Average
GPT-4.1
2.17
2.69
2.66
2.97
1.24
2.35
CapFllo (ours)
2.36
2.42
2.60
2.80
Qwen2,5-VL--B
1.82
2.10
1.70
2.22
2.04
1,97
OmniCaptinnr
1.27
1.93
1.43
2.33
1.17
1.63
MetuCuptioner (ours)
1.35
2.29
1.51
2.45
2.59
2.04
Table 4: Caption quality of CapFlow and existing methods on 5 metrics. We randomly sample 250 samples from multimodal benchmarks ZZhng et al. 2024b), and prompt GPT-5 to rate the five metrics.
GPT-4.1 (27.6 vs. 26.8), highlighting strong capabilities in video domains. Most importantly, these results achieved at only 10.5% of the cost of GPT-4.1 solutions, i.e., $0.14 vs. $1.5 per image, making high-quality caption synthesis feasible for large-scale applications. In summary, the ablation study confirms that both the workflow design and model scaling essential to the performance of generalist captioning.
5.4.2 IMPACT oF SYNTHETIC CAPTIONS ON TRAINING
In Tab. we systematically evaluate the impact of integrating different types of synthetic caption data (Chen et al.) 2024b; Li et al. 2024d) pre-training and fine-tuning. As shown in Tab. when augmenting pre-training data, CapFlow demonstrates clear advantages several key benchmarks, with the highest scores MMVet (62.5), InfoVQA (68.3), MMMU (56.1), and A12D (77.5). Considering the very competitive baselines, these gains confirm the benefits of captions generated by CapFlow to MILMM pre-training. In the fine-tuning phase, incorporating CapFlow data also leads to notable improvements, particularly on MMVet (62.8) and MathVista (60.9), where it achieves the best average performance among all configurations. Notably, Video-MME results (61.7 for MetaCaption) highlight its competence in temporal visual reasoning, outperforming other synthetic data. In conclusion, the synthetic captions produced by CapFlow consistently enhance model performance across both training stages, demonstrating their richness, accuracy, and broad applicability.
5.5
MAIN RESULTS
5.5.1
COMPARISON CAPTION QUALITY
In Tab. we compare the caption quality of our methods with several state-of-the-art captioning models the complex multi-mode modal scene. Among open-source models, Qwen2.5-VL-7B (Bai
et al. 2025) performs respectably (average score: 1.97), although it lags behind commercial models factual and reasoning aspects. OmniCaptioner-77 (Lu et al. 2025) trails behind with an average of 1.63, indicating room for improvement in complex captioning scenarios. In contrast, our CapFlow framework achieves superior performance on reasoning rigor, intent capture, and professionalism with average score of 2.33, slightly below the strong commercial baseline GPT-4.1 (2.35).
Our MetaCaptioner, fine-tuned with synthetic data from CapFlow, also demonstrates a much stronger generalist captioning ability, with performance comparable GPT-4.1. These results confirm the strong captioning capabilities of both CapFlow and MetaCaptioner, effectively closing the perfor- mance gap with commercial models by using open-source tools.
8

---

Published as a conference paper at ICLR 2026
MMB
Video
Math
Math
Math
SEED2
MM Star
Captioner
LLM
MMMU
MMB
AVG
Video
MME
Vista
Verse
Vision
Plus
VQA
Qwen2-VL-7B
DS- Ma-Qwen-77 7B
0.57
20.8
47.7
40.5
31.6
56.6
46.0
43.8
42.4
54.7
37.4
InterVVL..5-8B
DS- Qe-n-wen-B
0.88
26.2
60.6
44.7
34.8
61.8
48.6
52.7
52.8
55.4
38.7
OmniCaptioner-77
Qwen-7B
0.62
22.9
51.7
38.6
32.2
53.1
41.2
51.4
47.5
53.1
42.5
MetaCaptioner-88
DS DS-Q 3B-n-enn77
1.23
27.2
61.5
47.8
37.2
62.7
49.0
53.3
54,8
57.8
49,4
OmniCaptioner-79 Qwen-32B
0.64
24.7
56.0
39.3
33.1
57.4
48.0
55.3
59.2
48.3
MetaCaptioner-88 DS- DS-Qwen-32B
1.49
26.7
65.1
49,9
38.5
66.5
57.0
57.5
66.8
74.4
55.3
Table 5: Comparison of MetaCaptioner and existing captioners under the setting of visual
reasoning with LLMs (Lu et al.) 2025). For each benchmark, we generate captions as visual prompts for LLM reasoning. We utilize Deepseck-RI
Qwen (Guo et al. 2025) as the LLM.
MMB Video
Video MME
Math Vista
Math Verse
Math Vision
Doc
Chart
Info VQA
Model
MMMU
MMB
AVG
VQA
GLM4.IV-9B
1.63
68.2
80.7
68.4
54.4
93.3
70.0
80.3
68.0
72.4
KL-e-VL-8B
67.7
80.7
54.8
50.8
87.0
72.5
63.0
72.8
71.4
76.3
Qwen2.5
1.79
65.1
67.8
41.1
25.4
95.3
87.3
82.6
63.9
55.0
82.6
67.1
MiniCPM-V2.6-88
1.70
60.9
73.3
35.0
21.7
90.8
82.4
57.5
50.9
78.0
60.9
VL3
1.69
66.3
71.6
39.8
29.3
92.7
86.6
76.8
68.2
62.7
81.7
66.6
InternV1.3.5 Instruct
55.8
76.2
MetaCaptioner-88
1.76
64.2
75.8
56.5
52.6
93.0
86.8
76.6
66.7
69.5
80.8
71.1
Table 6: Direct performance comparison between MetaCaptioner and existing MLLMs. For fair comparison, models with reinforcement learning are marked in gray.
5.5.2 RESULTS OF VISUAL. REASONING WITH LLMs
In Tab. 5 we compare MetaCaptioner with existing captioners under the setting of visual reasoning with LLMs. In this setting, the LLM solves the multimodal task through the text prompt and captions produced by the captioner. When using Deepseek-R R --Distill-Qwen---- (Guo at. | (025) as the LLM, MetaCaptioner-88 outperforms both Intern VL..5- and OmniCaptioner-778 across all reported metrics. Notably, it achieves significant gains on MMBench-Video (1.23 vs 0.88 and 0.62), Video- MME (27.2 vs 26.2 and 22.9), and Math Vision (37.2 vs 54.8 and 32.2). The average performance (AVG) of MetaCaptionnr-888 reaches 49.4 4, substantially higher than OmniCaptioner- 7B' 42.5. The performance gap further widens when using the larger LLM, i.e., Bropseek-RR-Disttiie Rii--istill---ee-32E -wenl3BB.. MetaCaptioner-8B achieves significantly higher scores on challenging benchmarks such as Math Vista (65.1 vs 56.0), MMMU (66.8 vs 59.2), and SEEDBench2 Plus (66.5 vs 57.4). The significant performance gains indicate that MetaCaptioner generates more informative and structurally accurate captions that better leverage the enhanced reasoning capabilities of larger LLMs.
5.5.3
RESULTS DIRECT MULTIMODAL EVALUATION
In Tab, 6 we evaluate the general multimodal capability of MetaCaptioner and existing MLLMs (Yao et al. 2024 Hong et al. 2025 Team et al. 2025). From this table, MetaCaptioner demonstrates com- petitive performance across wide range of multimodal benchmarks against leading open-source source MLLMs. Notably, MetaCaptioner achieves strong results on several key benchmarks, including Math (56.5), Math Vision (52.6), and MMMU (69.5), outperforming most compared instruct models in these domains. Compared to models with reinforcement learning, MetaCaptioner also demonstrate better performance on some tasks, e.g., +1.5 and +16.8 over GLM4. on MMMU and ChartQ,, separately. This indicates its capability in handling thowledge-intensing and diagram- matic reasoning tasks, likely attributable to the high-quality caption data synthesized via CapFlow. These results affirm again that MetaCaptioner, trained with MetaCaption-4.1M, achieves top-tier multimodal capabilities among open-source models.
6 CONCLUSTON
In this work, we introduce CapFlow, a novel multi-agent collaboration framework designed to bridge the performance gap between open-source and commercial generalist visual captioning systems. By efficiently orchestrating multiple specialized agents, CapFlow achieves comparable caption quality with GPT-4.1 across diverse visual domains with an 89.5% reduction in costs. Leveraging this

---

Published as a conference paper at ICLR 2026
scalable pipeline, synthesize large-scale high-quality caption dataset spanning both image and video modalities, which turn enable the training of our generalist captioner, namely MetaCaptioner. Extensive experiments confirm the strong generalist captioning ability and multimodal capabilities of MetaCaptioner against existing MI.MMs. We believe that CapFlow and MetaCaptioner will serve valuable tools for future research and applications in visual captioning and reasoning and the broader multimodal community.
7
ETHIC STATEMENT
This study exclusively utilizes images and videos from open-source datasets, which have been rigorously filtered and screened to mitigate potential biases and ethical issues. Furthermore, all pre-trained models employed are sourced from the open-source community. However, despite these precautions, large models are inherently susceptible hallucinations, which may lead to the generation of misleading or incorrect content.
8 REPRODUCIBILITY STATEMENT
For reproducibility, our paper provides sufficient implementation details, hyperparameters and a complete set of prompts in the main text and the Appendix. To further facilitate replication, all source codes and models are now available at ht //qi thub. com htt..///pennllaaaaaaeaaaaaaaaaaa iner oner
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint 230.:2:3000087744 2023.
Jean-Bas Maan-Baptsste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-2377 2022.
Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6077-6086, 2018.
Chongyang Bai, Xiaoxue Zang, Ying Xu, Srinivas Sunkara, Abhinav Rastogi, Jindong Chen, and Blaise Aguera y Arcas. Uibert: Learning generic multimodal representations for understanding. 2021.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e3. pdf. 2(3):8, 2023.
Tanvir Bhathal and Asanshay Gupta. Websight: A vision-fire architecture for robust web agents. arXiv preprint arXiv:2508 16987, 2025.
Yuxiang Chai, Siyuan Huang, Yazhe Niu, Han Xiao, Liang Liu, Dingyu Zhang, Peng Gao, Shuai Ren, and Hongsheng Li. Amex: Android multi-annotatio expo dataset for mobile gui agents, 2024.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao. Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Weak-to-stron, training of diffusion transformer for 4k -at-too-mmae generation. In European Conference on Computer Vision, pp. 74-91. Springer, 2024a.
10

---

Published as a conference paper at ICLR 2026
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi multi-modal models with better captions. In European Conference on Computer Vision, pp. 370-387. Springer, 2024b.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang. Yu Qiao, Dahua Lin, al. Are we on the right way for evaluating large vision-language models? Advances in Neural Information Processing Systems, 111227556-278777 2024c.
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong. Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpi4video: Improving video understanding and genera- tion with better captions. Advances in Neural Information Processing Systems, 37:19472-19495, 2024d.
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning A survey of long chain-of- thought for reasoning large language models. arXiv preprini arXiv:2503.09567 2025.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open- source multimodal models with model, data, and test-time scaling. arXiv preprint Xii::2412..02211, 2024e.
Gheorghe Comanici, Eric Bieber, Mike Schaekermano, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, el. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 2025.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, Yue Zhang, Wenyu Lv. Kui Huang, Yichao Zhang, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. Paddleoer 3.0 technical report, 2025. URL Pattps://aran
org/ abs 2507. 05595
Yin Cui, Yang Song, Chen Sun, Andrew Howard, and Serge Belongie. Large scale fine- ane-grained categorization and domain-specific transfer learning. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp. 4109-4118, 2018.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang. Jae Sung Park, Moham- madreza Salchi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for stateoffttheaat multimodal models. arXiv preprint arXiv:2409. 17146, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. leee, 2009.
Xueqing Deng, Qihang Yu, Ali Athar, Chenglin Yang, Linjie Yang, Xiaojie Jin, Xiaohui Shen, and Liang-Chich Chen. Coconut-pancap: Joint panoptic segmentation and grounded captions for fine-grained understanding and generation. arXiv preprint arXiv:2502.02588, 2025.
Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. VImevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM international conference multimedia, pp. 11198-11222 12011 2024.
Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. Mmbench-vidde video: A long-ford form multi-shot benchmark for holistic video understanding. Advances in Neural Information Processing Systems, 17.8908--89122 2024.
11

---

Published as a conference paper at ICLR 2026
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video- mme:: The first-ever comprehensive evaluation benchmark of multi-modal Ilms in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24108-244 24118, 2025.
Xiaotang Gai, Jiaxiang Liu, Yichen Li, Zijie Meng, Jian Wu, and Zuozhu Liu. 3d-rad: A compre- hensive 3d radiology med-vqa dataset with multi-temporal analysis and diverse diagnostic tasks. arXiv preprint arXiv:2506. / 147, 2025.
Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, pp. 5842-5850, 2017.
Shuhao Gu, Jialing Zhang, Siyuan Zhou, Kevin Yu, Zhaohu Xing, Liangdong Wang, Zhou Cao, Jintao Jia, Zhuoyi Zhang, Yixuan Wang, et al. Infinity-mm: Scaling multimodal performance with large-scal scale and high-quality instruction data. arXiv preprint arXiv:2410. 18558, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Yongxin Guo, Jingyu Liu, Mingda Li, Qingbin Liu, Xi Chen, and Xiaoying Tang. Trace: Temporal grounding vidco I1m via causal event modeling. arXiv preprint arXiv:2410.05643, 2024a.
Zonghao Guo, Ruyi Xu, Yuan Yao, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, and Gao Huang. Llava-uhd: an Imm perceiving any aspect ratio and high-resolution images. In European Conference on Computer Vision, pp. 390-406. Springer, 2024b.
Xuehai He, Zhuo Cai, Wenlan Wei, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathological visual question answering. arXiv preprint arXiv:2010. 12435, 2020.
Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:250.001066, 2025.
Linmei Hu, Duokang Wang, Yiming Pan, Jifan Yu, Yingxia Shao, Chong Feng, and Liqiang Nie. Novachart: A large-scale dataset towards chart understanding and generation of multimodal large language models. In Proceedings of the 32nd ACM International Conference on Multimedia, pp- 3917-3925, 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXi::2410.21274 2024.
Baoxiong Jia, Ting Lei, Song-Chun Zhu, and Siyuan Huang. Egotaskqa: Understanding human tasks in egocentric videos. Advances in Neural Information Processing Systems, 35:3343-3360, 2022.
Mehran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut. Geomverse: evalutttooo of large models for geometric reasoning. arXiv preprint 12224
A systematic evaluation 2023.
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Sco. Hannanch Hajishirzi, and Ali Farhadi. A diagram worth a dozen images. In European conference on computer vision, pp. 235-251. Springer, 2016.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander C Berg, Wan-Yen al. Segment anything. In Proceedings of the IEEE/CVF international conference computer vision, pp.
2023.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li. Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint 24X8:2408.03326 2024a.
12

---

Published as a conference paper at ICLR 2026
Bohao Li, Yuying Ge, Yi Chen, Yixiao Ge, Ruimao Zhang, and Ying Shan. Benchmarking multimodal large language models with text-rich visual comprehension. arXiv preprint arXiv:2444. 247992404..6790, 2024b.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language ainuage-image pre- training for unified vision- language understanding and generation. In International conference machine learning, pp. 12888-12900. PMLR, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language rrguuge-image pre pre-train training with frozen image encoders and large language models. In International conference machine learning, pp. 19730-19742. PMLR, 2023.
Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang. Xiachong Feng, Lingpeng Kong, and Qi Liu. Multimodal arxiv: A dataset for improving scientific comprehension of large vision- rision-lanuuage models. arXiv preprint vxxx:2443300033 2024c.
Xiangtai Li, Tao Zhang, Yanwei Li, Haobo Yuan, Shihao Chen, Yikang Zhou, Jiahao Meng, Yueyi Sun, Shilin Xu, Lu Qi, et al. Denseworld-Im: Towards detailed dense grounded caption in the real world. arXiv preprint wrXiv2550.224022 2025.
Xiaotong Li, Fan Zhang, Haiwen Diao, Yueze Wang. Xinlong Wang, and and Lingyu Duan. Densefusion- Im: Merging vision experts for comprehensive multimodal perception. Advances in Neural 2024d.
Information Processing Systems,
Long Lian, Yifan Ding, Yunhao Ge, Sifci Liu, Hanzi Mao, Boyi Li, Marco Pavone, Ming-Yu Liu, Trevor Darrell, Adam Yala, et al. Describe anything: Detailed localized image and video captioning. arXiv preprint 2XXi:2554.16777 2025.
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Ya- coob, and Dong Yu. Mme: Advancing multimodal chart understanding with large-scale instruction tuning. arXiv preprint arXiv:2311.10774 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances 2023b.
in neural information processing systems,
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model all-around player? In European conference on computer vision, pp. 116-233. Springer, 2024.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXhv:2310002255, 2023a.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hanmaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023b.
Yiting Lu, Jiakang Yuan, Zhen Li, Shitian Zhao, Qi Qin, Xinyue Li, Le Zhuo Licheng Wen, Dongyang Liu, Yuewen Cao, et al. Omnicaptioner: One captioner to rule them all. arXiv preprint 2rX:::2004.07089 2025.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424, 2023.
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A bench- mark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203. 10244, 2022.
13

---

Published as a conference paper at ICLR 2026
Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 2021.
Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1697-1706, 2022.
OpenAl. Introducing gpt-4.1 modal family, 2025a. gpt 1-1
URL htpp://openai..om/nneee
OpenAl. Introducing gpt-5 modal family, 2025b. URL
Nathaniel Pinckney, Christopher Batten, Mingjie Liu, Haoxing Ren, and Brucek Khailany, Revisiting verilogeval: Newer Ilms, in-context learning, and specification-tort tasks, 2024. URL hetps:
wwwxxiv.rrg/aba 2408 11053
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PmLR, 2021.
Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, and Bernt Schiele. Movie description. International Journal of Computer Vision, 123(1)994-120, 2017.
Tanik Saikh, Tirthankar Ghosal, Amish Mittal, Asif Ekbal, and Pushpak Bhattacharyya. Scienceqa: A novel resource for question answering on scholarly articles. International Journal on Digital Libraries, 23(3):289-301, 2022.
Naganand Yadati Sanket Shah, Anand Mishra and Partha Pratim Talukdar. Kvqa: Knowledge-aware visual question answering. In AAAI, 2019.
Eldon Schoop. Xin Zhou, Gang Li, Zhourong Chen, Bjoern Hartmann, and Yang Li. Predicting and explaining mobile ui tappability with vision modeling and saliency analysis. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1-13, New Orleans, LA. USA, May 2022. Association for Computing Machinery. BtLhttps: /oii org/10. 1145/3491102 3502042]
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278 222227222224, 2022.
Shuai Shao, Zijian Zhao, Boxun Li, Tete Xiao, Gang Yu, Xiangyu Zhang, and Jian Sun. Crowdhuman: A benchmark for detecting human in a crowd. arXiv preprint 1 00123, 2018.
Share. Sharegemini: Scaling up video caption data for multimodal large language models, June 2024. URL/attps://qittt
com/ Sharelaa S/aa///eeaarrriiinn
Kwai Keye Team, Biao Yang, Bin Wen, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, et al. Kwai keye-vl technical report. arXiv preprint arXiv:2507.01949, 2025.
Haoran Wang, Yue Zhang, and Xiaosheng Yu. An overview of image caption generation methods. Computational intelligence and neuroscience, 2020(1):3062006 2020.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Mcasuring multimodal mathematical reasooniing with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095-95166 2024a.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge. et al. Qwen2-vl: Enhancing vision- language model's perception of the world at any resolution. arXiv preprint a409 12/11, 2024b.
14

---

Published as a conference paper at ICLR 2026
Teng Wang, Jinrui Zhang, Junjie Fei, Hao Zheng, Yunlong Tang, Zhe Li, Mingqi Gao, and Shanshan Zhao. Caption anything: Interactive image description with diverse multimodal controls. arXiv preprint S00.:::330000677.., 2023a.
Weiyun Wang, Min Shi, Qingyun Li, Wenhai Wang, Zhenhang Huang, Linjie Xing, Zhe Chen, Hao Li, Xizhou Zhu, Zhiguo Cao, et al. The all-seeing project: Towards panoptic visual recognition and understanding of the open world, arXiv preprini arXiv:2308 01907, 2023b.
Weiyun Wang. Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang, Qingyun Li, Lewei Lu, Xizhou Zhu, et al. The all-seeing project v2: Towards general relation comprehension of the open world. arXiv preprint arXiv:2402. 19474, 2024c.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint (XXi:2508.18865 2025.
Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, et al. Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer, 2025. URL
http://axxvv..roo wtt.vv..rrg/bb//511.111
18427
Long Xing, Qidong Huang, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Jinsong Li, Shuangrui Ding, Weiming Zhang. Nenghai Yu, et al. Scalecap: Inference-time scalable image captioning via dual-modality debiasing. arXiv preprint arXiv:2506.198 2025.
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual PMLR, 2015.
attention. In International conference on machine learning, pp.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, al. Qwen3 technical report. arXiv preprint arXiv:2505.099) 093388, 2025.
Ting Yao, Yingwei Pan, Yehao Li, Zhaofan Qiu, and Tao Mei. Boosting image captioning with attributes. Proceedings of the IEEE international conference on computer vision, pp. 4894 1902,2017,
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Miniepm-v: gpt-4v level milm on your phone. arXiv preprint arXiv:2408.01800 2024.
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. Clevrer: Collision events for video representation and reasoning. arXiv preprint arXi::1910.01444 2019.
Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with semantic attention. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4651-4659, 2016.
Zuyao You, Junke Wang, Lingyu Kong, Bo He, and Zuxuan Wu. Pix2cap-coco: Advancing visual comprehension via pixel-level captioning. arXiv preprint arXiv:2501. 2025.
Qiying Yu, Quan Sun, Xiaosong Zhang, Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, and Jingjing Liu. Capsfusion: Rethinking image-text data at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14022-14032, 2024.
Weihao Yu, Zhengyuan Yang. Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.0 02490, 2023.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang. Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal under- standing and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVE Conference on Computer Vision and Patlern Recognition, pp. 9556-9561, 2024.
15

---

Published as a conference paper at ICLR 2026
Jiwen Zhang, Jihao Wu, Yihua Teng, Minghui Liao, Nuo Xu, Xiao Xiao, Zhongyu Wei, and Duyu Tang. Android in the zoo: ceiii--f-action-thooeee for gui agents. arXiv preprint
arXhv:2403.02713, 2024a.
Lin Zhang, Xianfang Zeng, Kangcong Li, Gang Yu, and Tao Chen. Sc-captioner: Improving image captioning with self-correction by reinforcement learning. arXiv preprint arXiv: 06/25, 2025.
Renrui Zhang. Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, al. Mathverse: Does your multi-modal Ilm truly see the diagrams in visual math problems? In European Conference Computer Vision, pp. 169-186. Springer, 2024b.
Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pme-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305. 10415, 2023a.
Xiaotian Zhang, Chunyang Li, Yi Zong, Zhengyu Ying, Liang He, and Xipeng Qiu. Evaluating the performance of large language models on gaokao benchmark. 2023b.
Mingyu Zheng, Yang Hao, Wenbin Jiang, Zheng Lin, Yajuan Lyu, QiaoQiao She, and Weip- ing Wang. IM-TQA: A Chinese table question answering dataset with implicit and multi- type table structures. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 5074-5094, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18663/11/2222acllloooo org.
URL
https::/aclLantho..oo wtwp://wwww..oo.oo.oo2r...oo.. aal- 278
Tianyang Zhong, Zhengliang Liu, Yi Pan, Yutong Zhang, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, et al. Evaluation of openai ol: Opportunities and challenges of agi. arXiv preprint arXiv:2409. 18486, 2024.
Yangyang Zhou, Xin Kang, and Fuji Ren. Tual at imageclef 2019 vqa-med: a classification and generation model based transfer learning. In CLEF (working notes), 2019.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.1052 10592, 2023.
Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye. Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. 10479, 2025.
16

---

Published as a conference paper at ICLR 2026
Parameters
Pre-training Stage
SFT Stage
batch size
256
256
iterations
300, 000
160, 000
token length
32, 768 packed
32, 768 packed Adam
optimizer
A lc-5
learning rate
2c-5
Intern VTT-300M
Intern VIT-330M
training module
MLP Projecter
+ MLP Projecter
+ Qwen3-8B
Qwen3-8B
Table 7: Training details for MetaCaptioner-SB.
A APPENDIX
A.1 THE USE OF LaRge LANGUAGE MODELS (LLMs)
We only use the LLM (Deepseck-RI) to polish papers, but not for idea generation or to participate in the actual writing process.
A.2 TRAINING DETAILS
We have trained the Qwen-3-8B: -8-n----e-nntt model for two days on 128 H200 GPUs, resulting in the MetaCaptioner-88 model. The distributed training process is orchestrated using Xpuyu, employing the InternVL3.5 (Wang et al., 2025) optimization strategy. Additional prrramppraeeters utilized during training detailed in Tab.
A.3 ANALYSIS METACAPTION DATASET
In this section, we provide comprehensive overview of our dataset, including the data sources, domain partitioning methodology, and thorough statistical analysis.
A.3.1 DATA SOURCE
As illustrated in Fig. the MetaCaptioner-4.1 1M dataset is derived from over 140 open-source datasets. Data acquisition is conducted through combination of automated collection and manual sampling, with the entire process spanning one year. The specific sources for these datasets enumerated as follows:
Natural. Within the domain of natural images, we primarily focus on urban landscapes, natural scenery, human activities, animals, plants, still life, dynamic objects, and remote sensing images. Our data sources in these categories include: SAM (Kirillov et al.) 2023). COCO (Lin al. 2014).
All Seeing project (Wang et al. 2023b 2024c). Laion (Schuhmann et al. 2022), Crowdhuman (Shao
al. 2018), Inifinity-mm (Gu al. 2024). CitySeapes (Cordts al. 2016). etc.
Structure & Math. Within the domain of structured and mathematical images, our focus encom-
passes charts, tables, diagrams, equations, and geometric figures, of which are characterized by clear mathematical and numerical information. We collect raw data from the following open-source
datasets, e.g., Pixmo (Deitke et al. 2024), IM-TQA (Zheng et al.) 2023), MMC (Liu et al. (023a), Geomverse Kazemi et al. 2023). KVQA (Sanket Shah & Talukdar 2019), etc.
Infographic Document. We gather infographic and deetment-related data from the sources listed below: Info VQA Mathew al. (022),, Pixmo(Deitke et al. 2024), ArxivVQA(Li et al.) (024c), Laion(Schuhmann et al.) 2022), etc. Compared with the Structure Math domain, this domain is assigned specifically for those images with large amount of text information.
Medical & Bio-Ingging. For medical and biology images, which require specialized domain collect and augment data from open-source medical datasets. The specific data sources are as follows: Red-VQA (Gai et al. (025), ImageClef (Zhou
knowledge for accurate interpretation,
al. 2019) PMC-VQA (Zhang et al. 2023a), and Path-VQA (He et al. 2020), etc.
17

---

Published as a conference paper at ICLR 2026
Data Domains
34 Subdomains from 140 datasets
Know
Strectare
Nemse Karvingg
Chat
Tible 20/5
Nevia 0U%h
Natural
All
Tauaktn-2.0 23.5%
Erevndig 52%
Synthetie
Eireeet
Octanty
Elkarde 7.0%
Melicall
lafographic
Spacs
Nettanl OCK 17.2%
Vidse
Cualagen
Infographic
Pubilag
Strecture
Wid Cale 11.3%
Natucal
Tols
Coke Ualeskaiing 2.%
Scmar Ciject 13.5%
Asemd
Syathetic
Acaring
Teathhngg--
Data Domain Distribution
Data Source Distribution
Figure 5: Category distribution of MetaCaption-411.. a) The distribution of main domain; b) The data distribution in each subdomain. We report the percentage of cach subdomain of its corresponding main domain.
UT & Interaction. In the UT & Interaction domain, we collect web page screenshots, mobile screen captures, and other images related to web user interfaces. The following datasets are utilized for further data collection: Taperception (Schoop et al. 2022). Amex (Chai et al. 2024), Android in the
zoo (Zhang et al.) 2024a). UlBert (Bai et al. 2021). ), etc.
collect screen snapshots, augmented code through transformations, as well as structured code diagrams. The following datasets are utilized for further
Code & Programming. In this domain,
data collection: MMMU (Yue et al. 2024), WehSight (Bhathal & Gupta 2025). and Verillog
(Pinckney et al. 2024), etc.
Knowledge & Education. To address the need for comprehensive world knowledge, construct knowledge dataset divided into social science and natural science domains. The dataset is sampled from variety of sources, including high school-level examinations, natural science questions, cultural festivals, and real-world social scenarios, e.g., ScienceQA (Saikh et al. 2022), ImageNet (Deng 2009), Inat (Cui et al. 2018). GAOKAOBench (Zhang et 2023b), MMMU Yuue et al. 2024). etc.
Synthetic Asthetic. For synthetic and aesthetic images, we collect data from open-source AIGC datasets, such as LAION, PixArt, and openly licensed photography websites. Specifically, we collect data from the followin datasets, e.g., PixArt-a (Chen et al. 2024a). COCO-T21 (Lin et al. 2014).
and Laion-T21 (Schuhmann et al. 2022). etc.
Video & Temporal. We collect temporal data with semantic-rich information from human ac- tivities, educational videos, movies, egocentric videos, reasoning videos, and sports videos, e.g., LSMDC (Rohrbach et al.) 2017). ShareGemini (Share 2024). ShareGPTVideo (Chen ct al. 2024d), VideoChatGPT (Maaz et al. )023), 3 SSV2 Gooaa et al. 2017). Clevrer (Yi al. 2019), Egotask (Jia ct al. 2022) etc.
Finally, meticulously select high-quality samples from each dataset, and, following the protocols of (Zhu et al. 2025). perform additional filtering, augmentation, and deduplication to ensure data quality.
A.3.2 DOMAIN SPLIT
As summarized in Tab the MetaCaptioner dataset encompasses nine principal categories: Natural, Structure Math, Infographic & Document, Medical & Bio-Imaging, UI & Interaction, Code & Programming, Knowledge & Education, Synthetic & Aesthetic, and Video & Temporal. Each major
18

---

Published as a conference paper at ICLR 2026
Domain
Subdomain
Avg. Caption Length
Natural
Scenes Object, Social Activity, Animal and Plants, Remoting Sense,
863
Structure & Math
Chart, Table, Equation, Geometry, Diagrum Natural OCR, Document, Poster, Tahle Docs.
868
Infographic & Document
836
Medical & Bio-Imaging
Radiology, Pathology. Clinical, Case Report Website, Mobile, Tablet
729
UI & Interaction
1033
Code Programming
Code OCR, Cude Understanding, Web Code Science Knowledge, Art & Culture, Celebrity
1150
Knowledge Education
916
Synthetic & Aesthetic
Text2Image, AIGC Human Activity, Education, Movies, Egocentric, Reasoning, Sports
679
Video & Temporal
904
Table 8: Domain split and static results for MetaCaption-4.IM. We divide the data into nine
primary domains, guided by their visual characteristics and knowledge requirements. The subdivision into specific subdomains is primarily based on the data sources and their original tasks, We further static the average caption length for each domain.
Model
Factual Ace
Completeness
Rensening Rigor
Intent Capture
Professionalism
Human Preference
Average
GPT-4.1
2,80
CapFlow (ours)
2.82
2.95
2,93
2,85
2.95
2.96
2,90
Qwen2.5-V1-7B
2.41
2.32
1.97
2.54
2.31
2.31
Om
2.42
2.37
2.53
2.51
2.44
2.33
2.46
MetaCaptioner (ours)
2.88
2.94
2.86
2.90
2.90
2.90
Table 9: Caption quality of CapFlow and existing methods on 5 metries with human evaluation.
We randomly sample 250 samples from multimodal benchmarks (Yue et al.
2024 Wang et al.)
2024a Kembhavi et al., 2016 Mathew et al.) 2022; Chen et al.) 2024c) and invite eight human annotators to evaluate the captions according to five metrics.
category further subdivided into 34 subdomains based on the visual content and thematic focus. The corresponding visual understanding tasks include classification, detection, segmentation, visual grounding, common VQA, image captioning, multi-round conversation, OCR, document OCR, chart QA, chart OCR, table OCR, table QA, mathematical problem solving, visual reasoning, diagram reasoning, medical VQA, knowledge VQA, video captioning, video VQA, video highlight detection, video conversation, among others.
A.4 HUMAN EVALUATION CAPTION QUALITY
We conduct an additional experiment to further evaluate caption quality based on human judgment (Zhang al. 2025 Chen al. (2024b). Specifically, we select 250 data samples, illustrated in Tab. to construct the evaluation dataset. Eight human annotators are invited to assess the quality of the generated captions. The assessment include: factual accuracy, completeness, reasoning rigor, intent capture, professionalism, and human preference score. Annotators instructed to follow a rigorous three-point rating protocol for all criteria (Fang et al. 2024). During the evaluation, we adhere to strict couble-blind protocol: each annotator randomly assigned captions generated by Qwen2.5-VL-78 OmniCaptioner, MetaCaptioner, Capflow-728 and GPT-4.1, with model identities concealed.
The results, summarized in Tab. demonstrate strong consistency with the findings in Tab. thereby validating the effectiveness of reject sampling strategy and mitigating the potential model bias introduced by LLM-as-judge. Moreover, we find that human annotators assigned higher reasoning rigor scores the fine-grained captions produced by MetaCaptioner and CapFlow, compared to those generated by GPT-4.1 and OmniCaptioner. We argue that detailed captions with clear visual cues and explicit reasoning processes are more favored by humans.
A.5 ANALYSIS ON THE ROBUSTNESS oF DOMAIN ROUTER
We conducted ablation study to assess the robustness of the domain router. Specifically, four human annotators carefully selected 500 images from the original training data, covering eight major visual domains. These images feature complex scenes with mixed-domain characteristics, such as geometric figures accompanied by surrounding text and equations, or posters that combine natural scenes with overlaid captions. To ensure annotation quality, blind labeling and cross-validation were
19

---

Published as a conference paper at ICLR 2026
Visual Domain
Accuracy
Human Average
97.0%
Average
84.8%
Natural
85.7%
Structure & Math
87.5%
Infographic & Document
83.3%
Medical & Bio-Imaging
100.0 %
UI & Interaction
77.2%
Code & Programming
83.2%
Knowledge & Education
82.9%
Synthetic
80.0%
Table 10: Robustness analysis on domain routing mechanism. We carefully selected 500 samples from the MetaCaptioner-4. 1 dataset and invited four human annotators to conduct standard ground truth annotation and subsequent evaluation.
Model
LLMs
MMMU
Math Vista
A12D
Overall
Cost
GPT-5
DS DS-Qwen-7BB
58.2
64.0
74.2
65.4
1.56$
Gemini-2.5-pro
DS- DS-Qwen-7B
56.8
63.8
76.2
65.6
2.33$
GPT-4.1
DS WS-Qwen-7B
55.7
65.0
75.5
65.4
1.47$
Gemini-2.0-Flash
DS-Qwent wS-Qwen-7B
56.0
56.6
74.0
62.2
0.37$
Capflow-72B
DS-Qwen-7B
55.1
62.5
74.2
64.0
0.14$
Table 11: Caption quality comparison between Capflow-72B and the latest strong commercial models. Capflow-72B demonstrates a comparable result with the latest commercial models. We calculate the average inference cost on 100 samples.
performed, with consensus required to resolve any disagreements. Following the captioning stage, all four annotators evaluated both the routing and captioning results for each misrouted sample to further assess any performance degradation. The detailed evaluation results for each visual domain provided in Tab. 10
As shown Tab. 10 we observed misrouted samples in our experiments, which were subsequently subjected to additional human evaluation. Despite routing errors, the captions generally maintained a high professional standard: four annotators rated 64 out of the 76 misrouted samples as acceptable (84.2%)) During the final quality filtering stage, 83.3% of misrouted samples exhibiting clear hallucinations or errors were accurately excluded.
Our findings suggest that, in most cases, misrouting leads only to sub-optimal outputs rather than significant quality degradation. Only small proportion of misrouted samples adversely affect caption quality, and these can be effectively filtered out during the reject sampling stage.
For mixed-domain inputs, the domain router assigns workflows based on the most prominent visual features. In general. Qwen2.5-VL-7B demonstrates strong performance in this task. However, our analysis of misclassified cases indicates that rich-text content and numerical tables can substantially interfere with the model, resulting in misclassification among the Knowledge, Document, and Struc- ture Image domains. Furthermore, AIGC- -enerreeddt images and natural scenes can also introduce confusion in certain real-world scenarios.
A.6 COMPARISON WITH COMMERCIAL BASELINES
We conduct a thorough comparison between Capflow-72B. Gemini-2.00 2.0mini-2.0-Flssh Flaashh, Gemini-2.5-Par manici et al. 2025), GPT-4.1 (OpenAl 2025a), and GPT-5 (OpenAl 2025b) in the setting of visual reasoning with LLMs. This setting is designed to evaluate the quality of the caption in an LLM- driven manner. Specifically, the whole evaluation process decouples the visual understanding into
(Co-
20

---

Published as a conference paper at ICLR 2026
two stages: VLLM perception and LLM reasoning. First, the caption model is prompted generate the caption for images, and then the LLM will use the caption to answer the question directly. Thus, higher score in this setting effectively reflect the completeness, accuracy, and visual consistency of the caption.
As shown in Tab. 11 Capflow-72B outperforms Gemini-2.0-Fllsh (64.0 62.2) and achieves performance comparable to GPT-5 (64.0 vs. 65.6). We observe that Capflow demonstrates a notable advantage the domains of mathematical and structured images. For example. Math Vista benchmark, Capflow, Gemini-2.0-Flashh, and GPT-5 achieve scores of 62.5, 56,6, and 64.5, respectively. On the AI2D benchmark, Capflow even matches the performance of GPT-5 (74.2 vs. 74.2), underscoring its strong capability in structured image captioning. Although there remains a performance gap between Capflow and the strongest commercial baseline, Gemini-2.5-PP and GPT-5 (64.0 vs. 65.5 vs. 65.4), Capflow-72B exhibits significantly lower inference costs (0.02$ vs. 2.33$ vs. 1.56$ for 100 samples).
the
21

---

Published as a conference paper at ICLR 2026
A.7 PROMPT TEMPLATES
In this section, we first present the prompt templates used in Capflow for constructing functional agents and summary agents. Then, we provide the prompt templates for quality evaluation in the image domain and the video domain.
A.7.1 PROMPT TEMPLATES USED CAPFLOW
The prompt templates used in Capflow contain three parts: domain router agent, functional agent, and summary agent. We will detail these prompt templates in the following.
Domain Router Agents. The domain router agents in Capflow are designed assign workflows based on the primary characteristics of the visual input. To mitigate potential misclassification risks, we explicitly define the distinguishing features of each visual domain in the prompt design and provide clear boundaries for features that are prone to confusion. Furthermore, we introduce a three-point confidence assessment mechanism to further reduce routing errors. The agent is required to evaluate its own routing confidence, assigning lower confidence scores to mixed-domain or ambiguous cases. With these two explicit guidance strategies, the domain router becomes capable of efficieetlly allocating workflows across large-scale, multi-source visual inputs. The detailed prompt template utilized to activate domain router agent presented as follows:
Prompt for Domain Router Agent
You are a rigorous visual domain classification agent. Your task is to classify the input image or video (optionally with the following predefined categories and return class-JSON object with the following structure: Output requirements:
nser-provided text)
The output must be a single-line class-I8ON object. The keys and their order must be strictly: {class, explanation, comfidence.scoree
One predefined English category strings below.
explenation ion: beief. objective analysis (1-3 sentences), indicating key visual features and reasons for not choosing
similar categories.
con score: integer from (1, 2, 3), following scoring criteria helow
ficlds, markup. line breaks, apologics, disclaimers allowed.
Predefined Categories (exact strings):
Natural
Structure & Math
Inlographicc Document
Medical & Bio- Bin-Inaging
UI & Interaction
Code & Programming
Knowledge & Educatiin
Synthetic & Acsthctic
Video Temporal
Category definitions and classification hints:
Natural: Real-world scenes, human social activities, plants/animals nature, natural landscapes, panoramas, satellite/drone imagery.
Structure & Math: Mathematical charts, labular images, geometric ligures. formulas, mathematical flowchurts, diagrams emphasizing quantitative or formal relationships. The core feature is clear representation of muthematicallssienn content.
Infographle & Document: Natural scenes with promincnt text, document pages (c.g.. PDPs), posters with large blocks or
prominent billes of locusing layout understanding and text extraction.
Medical & Bio-ing Radiology (CT/MRI/X-rar ray). pathology slides, clinical photos, multi image case reports, images featuring pathological traits humans, animals, or plants.
Interaction: Screenshots of webbites/mohhlchhbbeeaaaap interfacess, pages, or interactive characterized by app icons clear web/Ul elements.
Code & Programming: Screenshots of code, code analysis, WD/editor terminals, code fragments- -eatures clear code covering more than of the image.
Knowledge & Education: Images with biological, physical, chemical, other scientific diagrams, educational illustrations with text, scientific visualizations, articulture, muscum exhibits relevant for leamiin or general knowledge.
Synthetic & Aesthetic: All Al Aeggrrerte images, typically with stylized features such as anime. synthesized humans,
22

---

Published as a conference paper at ICLR 2026
Video & Temporal: Content emphasizing lempuraal dynamics, such as human activities, instructional videos, film clips, first- pist-person views, motion analysis, temporal reasoning.
Disambiquation rules:
Video frames: If the care task is cleadl about time/motiioa/cclaviie recognition, choose "Vidco & Temporal", otherwise, clussily by content visible in the frame.
Charts documents: Layour dominates, choose "Inlographicc Document"; the chart contains quantitative data,
chouse "Structure & Math".
IDE code interfaces: Choose "Code& & Programming" rather than "UI & Interaction".
Educational postershaadouts: If containing clear biology. physies, chemistry, culture, or history knowlodge, choose "Knowledge & Education" rather than "Inlographic & Document".
Natural scenes: If ohvicusly styized, consider "Synthetic & Aesthtiic" instead of "Natural".
Mixed-domain images: Choose the most appropriate category based on the dominant content, and explain the reasoning (e E. a table mixing numbers and text, if inferable by numbers, choose "Structure & Math").
Confidence scoring (strict 3-point scale):
3 Clear, unambiguous match: multiple streng leatures indicate a single category: low overlap with others.
2 Reasonable match but some ambiguity or overlap; fentures present but not complete (e.g., large text blocks Diccuent Math?).
1 Low confidence; poor image quality, heavy occlusion, mixed domains with no dominant type, missing key features.
Functional Agents. The functional agents employed in Capflow can be broadly categorized into two groups: visual perception agents and visual reasoning agents. The visual perception agents comprise the natural perception agent, structural perception agent, infographic perception agent, OCR agent, coder agent, texture agent, and video perception agent. The reasoning agents include the general reasoning agent, medical reasoning agent, knowledge reasoning agent, and video reasoning agent. The detailed prompt templates utilized to activate each functional agent are presented follows:
Prompt for Natural Perception Agent
You are an expert specializing in ultra-high-detaal image description and style interpretation. Your task is to craft exceptionally detailed, lugically clear, fluent descriptions every image, identify ing its artistic medium, decorative features, rendering techniques, to serve as figure captions for profesiinnl academic publications.
You should descrihe a variety of image types, including hut not limited to classical and contemporary artworks, classical musical scores, science posters, web interfaces, natural landscapes, close-ups of plants animals, human activities, sports, street scenes, ancient and modern architecture, literary works, economic statements, mathematical charts, chemical formulas, physical model diagrams, film posters, and other images rich in information.
Please follow these rules wh when describing:
1. For all concrete objects and artistic representations (e.g., road signs, pedestrians, animals, cups, painted figures, etc.): Accurately identify all visual subjects the image and provide precise and comprehensive description of their eategory, key features, color (type. brightness, saturation), geometric characteristies (square, round, irregular shapes), quantity, size. local details (such the jucket and pants of a person, accessories on clothing. etc.). lexture features, absolute and relative position in the image, and spatial relationships (such objects stacked or laid flat)..
For visual elements with abstract referential meaning requiring domain-spectte specifie knowledgs (e.g. charts, design elements in posters, stacks of geometric hodies in space, quadratic function grahhs, etc.): Use domain-appaoeriate vocahulary and sentences to accurately and thoroughly describe their type, key features, geometric shape, quantity, size, local details (such accessories on clothing. inflection points of quadratic function. points, Lines, planes in geomettic images, circular berders. etc.), absolute position within overall imuge, relative position to other objects, all decorative features (such icen outlines).
readable form, including text and numbers in documents, formulas, tables, charts, and artistic typugraphy. Additionally, accurately identily the features these texts (such as color, size, layout characteristics). and separately describe any distinctive text features, including bold, underlined, special fonts, different colors, or highlighted text.
all visible text elements: Accurately and completely identify visible text and its features, presenting
For image backgrounds: Accurately indicate the arrangement of environmental elements in concrete scenes as well as the dynamic and armospheric characteristics those seenes express (e.g.. soccer field during a match. clean and tidy classroom, or rapidly Blowing waterfall), For abstract artistically colored image backgrounds, provide an overall summary (e.g.. pure white background. poster with hand drawn style globe).
Describe style features all visible objects (including but not limited to lighting. brightness and darkness, color tone, realism, contrast, saturation, etc.). and for images with signifieant acsthetic characteristics, moderately integrate the stmosphere created by the image and the possible sensory experience (such as light and shudow, texture, ambience, and sense of movement). All stylistic features should be naturally incorporated into the descriptions of visual elements and the overall background, and as indopendent tags.
be listed
If there are special logical relationships in the image (such as spatial arrangement, inclusion, nesting, flow, cansality, etc.), describe these order according their logical charncteristics.
For obscured, blurred, extremely small elements: Only describe the identifiable portions; parts that cannot be accurately recognized can be omitted.
23

---

Published as a conference paper at ICLR 2026
8. All visual and texiual elements in the natural and social sciences must be described using prulessional terminology: for artistic, aesthetic, and literary content, you are encouraged to use elevated adjectives and expressive vocahulary to convey the emotiinna impact and aesthetic qualities of the image.
9. If common knowledge or world knowledge involved (such as plants and animals, celehrities, famous landmarks, historical sites, etc.). you must state it explicitly and avoid vague expressions like "somcone" or "somewhere.""
10. Output should be complete, coherent paragraphs, allowing multiple sentences per paragraph, avoiding lists, structured subheadings. bullei points, or semantic tags.
11. When describing objects, data, or structures, you are encouraged to use measurable language or relative references (such as "in lower corner the frame," "as
figure," "in sharp contrast with the background") )
12. Deseribe the content the image directly, without using phrases like "the image shows" expressions, do not use speculative language such as "possihly" or "ahout," do not describe content not present in the image, and do not use phrases like "there is text in the image."
Avoid repetition and uncertain
13. NOT infer analyze any content
image; only describe what is visibly present.
14. For complex images, keep total description within 500 words, with priority given complete presentation main structures and prominent leatures; details may be appropriately condensed.
Prompt for Logical Perception Agent
are an expert with outstanding professional competence in spatial understanding, logical reasoning, mathematical analysis. Your task is produce highty information-dense, detail-rich, and rigorous academic deseription in the style of research paper for images that require professional analysis and logical reasoning in the natural sciences and interdisciplinary fields, including but not limited to mathematics, physies, chemistry, electronic information, economics, psychology, pharmacy. and medicine. Your description will serve as the figure cuption for images in professional neademic publications.
The descripit should adhere to academic conventions, employ appropriately embed LaTeX formulas, provide stop-by-stop explanations of formulas, structures, and data the context the relevant discipline.
empley strict logical structure, precise terminology, and fluent written expression,
Image content may include but is not limited to mathematical formulas, geometric figures, data tables, visual charts, chemical structure diagrams, logic problems, model structure diagrams, physical model schematies, olympiad math problems, cconomics and psychology charts, medical data tables. and other images that require specialized knowledge to interpret.
Please strictly follow the rules below:
The first sentence must concisely summarize the overall information and subject of the image. using standard academic terminology and expressions commonly lound in reseurch papers (e.g. "This figure illustrates..." "), accurately specifying the type of image (such function graph, geometric proof, data analysis, flowchart, etc.). and precisely summarizing the content.
Systematically and progressively identify and deseribe in detail all visible elements, specifically including the following:
Text, Numbers, and Formulus:
Accurately identify all visible text, numbers, formulas, and symbols (including variable names, units. titles, annotations, and table contents). All formulas must be returned in standard LaTeX format and embedded scamlessly in the text, with each explained context and with relevant disciplinary buckground.
For symhols with different meanings across disciplines, clarify their meaning according to context. Additionally, descrihe the foot, color, font size, decoration, spatial position, layout, and artistic features all character or numeric elements.
Geometric Figures and Structural Elements:
Provide a detailed description of all geometric figures and structural elements, with a focus on the layout the whole and substructures, shapes, line segments, ares, connections, angles, intersection points, tangeney, parallelism and containmeut, spatial distribution, auxiliary constructions. mathenatical properties, and their spatial or logical relationships with other elements.
For abstraci complex geometric ferms, prioritize the use of professional matherattcal language describe their spatial relationships, symmetries, transformations, and topological features. For properties such as concurrency, collinearity, or coplanarity, provide precise characterization
on the relative or absolute spatial positions image.
Data Tables and Visual Charts:
Fully extract all visible data, text, and decorative features.
Descrihe hasic attributes visual elements (such as har charts, line graphs, scatter plets, pie charts, heatmaps, etc.): color (as precisely possible, e.g., by subjective color name or RGB), shape, size, value, absolute or relative position, intersections lines and figures, spatial or interactive selationships with text and other elements, as well as any decorative features (such as axes, tick marks, legends, borders, color blocks, partitions, etc.).
All values, variables, coordinates, scales, etc. must have their units or dimensions elearty defined.
Baractura/Trameeoo Flowchurtss Diagrams, and Model Structure Diagrams:
Describe in detail all structural units, modules, relations, grouping, enclosure or nesting structures, and their spatial distribution as well as logical, causal, or dependency relationships.
according to their spatial layour, specifying their attributes, hicrarchical
For all connecting lines, arrows, edges, flow indicators, describe in detail their direction, and end points, style (such as solid, dashed, curved, etc.). thickness, color, and any attached labels or symbols, and analyze the processes and logical relationships indicated arows or connecting lines.
Describe the size, color, line type, and spatial position of borders, frames, and special shapes (such restanglos, circles, cllipses, diamonds, parallelograms, polygons, etc.).
All annotations and markers that add supplementary meaning to the diagram must be identilied and explained.
Professional Strueture Dingrams in Chemistry, Biology, Materials Science, Pharmacy, and Medicine:
24

---

Published as a conference paper at ICLR 2026
Accuralely describe mulecular structural formulas, cellular structures, drug molecules, etc., including their constituent units, connections, spatial configuration, symbols, numbering, elements, groups, and all relevant attributes, and explain the function and significance of each within the structure.
3. For all stractured images, clearly descrihe the spatial distribution, structural hierarchy, grouping, nesting, and their logical or cansal
relationships among the elements.
information emphasized or highlighted in the image, provide more detuiled description.
DO NOT infer analyze any content beyond what is visible in the image; only deseribe directly visible content.
DO NOT speculate about facts not present in the image (e.g. intersecting lines not necessurily perpendicular). If any information is unclear or ambiguous, this should he explicidly stated.
7. The oufgut must be a complete, coherent natural paragraph, with mullippe sentences forming one or more natural paragraphs as appropriate. Except cases of extremely complex structure, only concise lists organized in natural language are permitted. The of structured subbscdings. bullet points, or scmantic tags is strictly prohibited.
All text and formulas must he output in LaTeX or Markdown format, and each must be smoothly ex explained in fluent prose within the main The overall deseriptive style conform academic paper standards, with rigorous expression, clear logic, and standard terminology. You must self-check for coverage. logical consistency, and compliance. If nothing is missing, there is no need state explicilly.
Each image description should be limited to 500 words; the content extremely complex, slight extension allowed, but information density, logical clarity, and academic completeness must be ensured.
Prompt for Reasoning Agent
are an expert with extensive global knowledge reasoning, and image analysis. Your fask is provide reasonable and accurate interpretation and analysis of image based strictly on directly visible elements within the image, and to construct a knowledge document in professional and academic language, suitable for When analyzing the image, your analysis must be strictly grounded visible information, starting from global features and progressively moving towards specific details:
high level of professional expertise in spatial understanding. logical
illustration explanation in academic journals and maguzines.
1.
First, use precise, domain- specifie terminology to accurately identify the basic information, structural hierarchy, and all logical relationships in the image. In 2-3 sentences, provide an overview of the image's 's layout, layers, partitions, envirenmeur, states, and modular churacteristics.
Based the image's intrinsic properties and features, choose relevant rules from the following for further analysis:
For images with concrefe objects or natural scenes (e.g.. human activities, sports events, etc.), focus on the correlation between event:
object, time, environment,
What are the attributes and characteristics of the object itsel??
Object-enviromeent relations: What features do the object's position, distribution, and in the environment demonstrate? Object-object relations: What are the spatial arrangements, ordering. stacking. interaction features, or cultural connections between objects?
Object-time/eeer sime/event relations: At what time is the object undergoing what event?
images involving text, with cultural, historical, geographic, or artistic significance. draw on your extensive world knowledge for thorough interpretation and deep reflection:
What message does the image seek to convey?
By what means is this conveyed? Doess this appronch reference other knowledge?
Why express this way? What are the advantages of this appronch?
For abstruct, geometric, and reasoning images (such as formulas, geometric ligures, logic puzzles, etc.). lirst identify the busic visual Information and perform mathematical ahstraction before analyzing:
formulas, equations, and function expressions:
Analyze type, characteristics, variables, and parameter meanings. Decompose and summarize the structure and properties of the formula, including but not limited to:
Extremum analysis: maximum, minimum, extremum points, and their conditions for existence and uniqueness.
Monotonicity and interval properties: bow the function/ecppeessinn increases, decreases, is conves/concave, or periodic on different intervals.
Special point identification: intersections, zeros, inflection points, asymptotes, axes of symmetry, and their calculations
and significance.
Trends and invariants: such as conservation laws, symmelry, periodicity, recurrence relations, parity, and other mathematical thcoretical meanings.
For complex formalas, hriefly explain key components appropriate.
For geometric structures and complex graphical structures:
Comprehensivelyy and accurately identify all pussible structural labels in the image, auxiliary lines, segment lengths, and key geometric features.
Strictly base all inferences about hidken matemattccal properties and domain knowledge on visible visual information only. For gualitative and guantilative inferences, start from local features and gradually generalize to the whole.
For images involving text, data, symbols, analyze their corresoondence or mapping with external information model structure, and employ mathematical tools for deeper analysis when necessary.
For pattern evolution, path planning, and inductive reasoning problems:
Summurize evolution laws, change patierns, or optimal stralegies based on visible sequences, evolutions, palhs, arrangements. combinations, recursions, etc.
For dynamic processes, analyze the mathematical meaning and objectives of each step.
25

---

Published as a conference paper at ICLR 2026
For logic puzzles, inductive reasoning. and sequences, use rigurous logical induction and deduction to derive general rules conclusions.
interences must be strictly based on directly visible and highly certain information in image. is forbidden introduce theurems progerties not directly supported by visihle elements.
complex procedural or demonstrative images (such flowcharts, diagrams, posters, and UI interfaces), focus inter- preting the execution logic. procedural Ilow, and intent of visual information communication:
Thoroughly examine overall functions, partitioned lunctions, and local functions, Analyze the execation sequence. dependen- cies between modules and processes, and the embedded professional knowledge and information by considering the imuge'
visual elements.
For inductive, summary, and statistical images (such as charts, tables, etc.):
Start from the row and column attributes and visual features of the chart to analyze the characteristics of the datn step by step.
Parse mathematical and statistical laws within the data and interpret them appropriately in conjunction with the chart's title.
Pay attention to outliers and missing values, and attempt to analyze these aspects.
Content Requirements:
1. All reusoning and analysis must he strictly hased on visible information in the image, and only moderately and reasonahly extended from direcily visible information. Avoid unnecessary statements listing.
For formulas, and tables that can analyzed. please incorporate LaTeX formulas improve readahility.
For highly correlated image text information, jointly amalyze both image and
4. Please rigoroully check the information in your outpot, remove redundant statements, and only present reasoning processes and conclusions that are cleadly supported by visual information.
Format Requirements:
relevant information present, not provide any description, Avoid using statements as "There no image."
2. Use coherent and well cornected narrative paragraphs rather tham frugmeeted or disjointed sentences. Approgriately use lists to explain inferences as needod.
complex images, reasoning and analysis should within 600 words. For simple figures, keep analyses concise and only state direcily deducible inferences.
Prompt for Infographic Perception Agent
an expert in the analysis posters, infographies, document images. Your to generate exceptionally detailed, precise, and vell-structured descriptions, strictly based the visible content of the image, expert visual designer and technical communicator.
For each image, follow these guidelines:
1. First, succinetly summarize the core message or theme the image conveys, clearly stating its main content and overall visual style. Use precise descriptors for the visual style, such "bright." "minimalisu" " "vinlage"" " "modern,, "technological" "matural," "artisticc" or others. Explicidly describe the color scheme, dominant colors, and the atmosphere or mood created by these choices.
Next, methodically describe all textual and visual elements in order, proceeding from top bottom and left to right. For each legends, captions). provide detailed information
element (text, icons, diagrams, shapes, images, lines,
The spatial and contextual relationships between elements, including arrangement, overlap, grouping. separation, symmetry. alignment, layering (fregroondbnckground),, and logical or communicative links.
All visible text, formulas, and chart contents, specifying the full extracted content, font type (e.p. sans-serif, serif, calligraphy), color, position (such top. corners, center), alignment, any occlusion. orientation (horizontal, vertical, rotated). and decorative leatures fe.g. geometric shupes, gradients, shudows, outlines, borders, background fills).
All visible visual components and backgrounds, precisely noting color, geometric form (rectangle, cirele, rounded edges, dashed or solid lines, etc.). size, position. texture, motion or static state, layering. decorative effects (such as borders, drop shadows, any other observable characteristics.
gradients. textures),
For all lines or arrows, specify their type (solid, dasbed, curved, struight). color, thickness, direction, and how they connect or scparate diffcrent elements.
For any legends, captions, or explanatory pancls, detail their placement, content, icon shapes, color coding. and how they relate the main disgram.
there are any zoomed-in in views, insets, or callouts, describe pusition. connection main image, and the specific details they highlight.
For all icons graphical symbols, specify their shape, color, position, and they are explained used koys within the Layout.
Finally, analyze how textual and visual elements work together reinforce the main message or theme, and assess the clarity and effectiveness of the design based strictly the actual content and layout. Discuss the intended audience based the image's design complexity, style, and information density.
Formats and requirements:
DO NOT use any lists, structured subleadings, bullet points, semantic tags.
2. Begin directly with the description of the image content without using opening phrases like "'The image shows....
DO NOT use speculative language such as "possibly" or "probably". Avoid repetition and uncertain statements.
26

---

Published as a conference paper at ICLR 2026
DO NOT describe information not requested in the content requirements, nor use phrases like "This image contains no lext content"..
For obszured, blurry, or very small elements, describe only what can be confirmed and ignore parts that cannot he accurately identified.
If there them explicity instcad of using phrases like "a person", "a place", otc.
world knowledge, for example, species, celebrities, scenic spots and historical sites, you
Prompt for OCR Agent
You advanced OCR model cupable of accurately extracting and reconstructing textual content from various types images. Your tack is to comprehensively analyze the provided image and return all detected textual content in clear, readable, and well-structured format.
Your capabilities include:
Extracting from diverse image types such photographs, scanned documents, posters, screenshots, handwritten notes, forms, tables, diagrams, technical drawings. and multilingual content.
Preserving the reading order and logical structure of text the image, including titles, paragraphs, lists, tables, and annotations.
3. Ignoring purely non-textual visual elements unless they are directly associated with text (e.g., labeled diagrams or annolated charts).
Handling complex backgrounds, mulfi-column layouts, or overlapping elements images.
Supporting multilingual and mixed-langnage extraction.
For bold, underlined, or colored text, return the content in LaTeX formal specifying these font styles.
Formats and requirements:
1. Outpot the extracted results in clear, readable manner, restoring the original structure as much as possible (e.g. using line breaks for paragraphs, indentation for lists, aligned text for tables).
add any explanatory notes comments: only oulpui recugnized textual content.
Directly output
LaleX format without introductory text.
Prompt for UI Agent
an advanced assistant specializing analysis Graphical User Interface (GUI) images, capable converting them into highly detailed, logically structured natural paragraphs. Your task is to provide comprehensive, fluent, and professional annotations for each GUT image, accurately reflecting its visual composition, layout, and functional elements.
You must be able describe GUTs from various domains, including web pages, mobile upplications, desktop software, dachboards, and embedded device interfaces. GUI images may contain backgrounds, navigation indicators, popups, modal dialogs, lists, tables, charts, as well as various interactive and decorative elements.
menus, icons, buttons, input fields, labels,
Conduct your analysis according to the following requirements, presenting the output in logically connected, fluent paragraphs not lists, bullet points, headings):
right and top to bottom, ensuring completeness and For each element, specify its type, position, size, color, fomt, alignment, style, and relevant relationships groupings with precision. Accurately extract all visible text, specifying its content, font properties, color, and spatial placement. Deseribe the background in detail, including overall color schemes, background images, gradicnts, textures, and dynamic effects, any. Emphasize layout stracture, spacing, alignment, and visual hierarchy.
Detail Extraction:
Describe all visible elements spatial order from
spatial
2. Description of Interactive Elements: Identify and describe all interactive components, indicating type, function, state (enabled, indication. Integrate contextual detail about their within
disabled, focused, bovered, pressed), any visual leedback the user workflow and their contribution to user interaction.
3. Overall Description: Summarize the overall purpose. primary function, and visual style of the GUI in concise, informative statement. Integrate observations overall palette, visual style (minimalis, skenomorphic, material, modern, etc.), and visible thematic or branding features, All descriptions must be strictly based on observable facts, avoiding subjective or generic statements.
Formats and requirements:
only natural paragraphs; not use bullet points, headings. must be logically
Ensure descriptions highly consistent actual appearance layout GUL.
If cignificant inconsiseencies detected, adjust the description needeed, but strictly adhere to visible details.
DO NOT use semantic markdown headings; present all content complete, natural paragraphs.
5. Maintain professional and analytical tone, using precise technical terminology for positions, relationships, and styles.
27

---

Published as a conference paper at ICLR 2026
Prompt for Coder Agent
You senior software engineer and code analysis expert with a solid theoretical foundation in programming languages and extensive software development experience. Your task to provide professional captious images containing code snippets, algurithu illustrations, software architecture diagrams, program flowcharts, data structure diagrams, website screenshols, or development tool interfaces.
These images typically include source code in various programming languages, psenlocode,, algorithm vismlliaations system design charts, debugging interfaces, IDE screenshots, and similar content requiring you to apply somprehensive computer science knowledge for accurate identification and technical unalysis. Other relevant website screenshots may also appear and should be included in the description.
the image content, provide a detailed technical description as follows:
1. Accurately identify programming languages, code structures, algorithmic logic, and system architecture components, using standardized computer science terminology.
Provide detailed descriptions visible code syntax elements, function delinitions, variable declarations, control flows, data structures, and architectural modules, ensuring accuracy and specificity in technical descriptions.
Objectively analyze observable functional characteristics, algorithumic complexity, or system design patterns. without specu- lating code performance runlime effects.
Strietly base all analysis visually observable code and interface content, without making asumptions about complete implementationn, business lagic, or overall system functionality.
Formats and requirements:
Use professional technical langunge to compose coherent paragraphs, ensuring that descriptions are both technically accurate readily understandable programmers.
2. The output should support code learning, technical documentation, and software developmennt
3. DO NOT use semantic lags, lists, or any bullet points. Formut the response single coberent paragraph.
Prompt for Knowledge Reasoning Agent
are professional image analysis expert with extensive world knowledge, specializing geography, history, culture, art, architecture, and society. Your task is to depict the visual information present in images and to infer and identify any famous figures. landmark buildings, artworks, historical events, or other world knowledge elements that may appear in the image. When generating descriptions, please follow these guidelines:
1. Provide a comprehensive and defailed description of the main content in the image, including the overall characteristics and recugnizable object
background the scene, visible objects their precise spatial distribution, detailed features (such as key figures, architectural subjects, natural landscapes, background elements, lighting conditions, spatial layout. dynamic or static context, object types, colors, quantities, actions, exact locations, textual content, the relative positions between objects). Accurately convey all stylstic features of the image, including color palette, artistic style, and visual atmosphere.
Based visual cues and your professional knowledge, make explicit and precise judgments shout the speeific people, places, buildings, artworks, or cultural references involved in the image whenever possible.
Formats and requirements:
multiple plausible interpretations, explain the most likely option.
presented fluent, professional, lugically coberent paragraphs. For images aesthetic qualities, more advanced and expressive vocabulary.
outpul
3. Avoid vagueness or generalizations. Focus on direct insights from the image and knowledge, and provide high-value information in your outpur.
Prompt for Medical Reasoning
You clinical expert with extensive professional knowledge in clinical medicine, public health, and biology. Your task to provide highly uccurate descriptions for medical and biological images. including but not limited to: medical imaging (such X-rays, CT images, medical specimens, images related to pharmacy, biochemistry, microbielogy experiments, as well as statistical charts in the field of public health.
MRT images, ultrusound, ECGs, etc.), endoscopic and surgical images, histopathological and anatomical
images require high degree expertise precision. must your solid foundation medicine. biology. clinical theory to deliver objective and accurate descriptions.
When generating descriptions of medical and biological images, strictly adhere to the following principles (only when such content is truly visible the image):
1. Use precise and stanardizzd medical and biological terminology to describe observable anatomical structures, tissue characteristics, imaging features, experimental elements, ensuring terminooogy professional and standardized.
Describe the morphologicl,, radiological, experimental charucteristies shown in the image, including structure, color, distribution, signal intensity, density, shape, size, spatial levels, statistical distriburion, etc.
28

---

Published as a conference paper at ICLR 2026
3. Analyze and infer in detail possible lesions, abnormalities, or prominent features presented in the image, locusing on clinical reference
If image contains technical elements (such equipment models, imaging parameters, staining types, axes, scale bars, legends, data units, etc.), describe them truthfully.
For issues that impact interpretation iisiion--siaa such as obstructions, blur, artifacts, uneven staining, outliers, missing information describe accurately. but do not speculate on causes or consqueenes.
Formats and requirements:
Write coherent pangraahs using professional medical and biological language, ensuring all descriptions hased directly visible image or data evidence, and are scientific, objective, and accurate. Do not describe nonctistent or indeterminable content.
DO NOT use semantic labels, bullet points, or lists. Output in clear, logically connected natural paragraph format.
Prompt for Visual Guideline Agent
You expert at synthesizing and summarizing complex visual information. Your fask is provide concise, insightful summary that cuptures the essence, main messuge, and key features insights. Condense the image's 's content, relationships, and significance into coherent, high-level overview. Highlight the theme, main visual elements, and any notable stylistic, cultural, or scientific characteristies. Express the overall atmosphere, language, suitable
image, integrating both olserved details and analy tical
impact the image clear, Formats and requirements:
summary or conclusion.
DO NOT repeat exhaustive visual details or stop-by-step reasoning.
on synthesis, clarity, insight, articulate image's essence makes it distinctive or meaningful.
your summary as fluent, elegant paragraph, without headings, or introductory phrases.
Prompt for Video Perception Agent
You are an expert specializing in ultra-detailed video description and style interpretation. Your task to produce Ingically clear and fluent deseriptions for the input video for frames obtained average sampling, with extremely fine-grained and comprehensive conteut descriptions of key scenes.
Task Setup:
You need to identify all visible subjects, background features, ont-screen text, temporal structure, camera and lens movements, decorative features, and rendering techniques the video, producing explanatory captions suitable for academic publication.
Applicable Scope:
documentary videos, news, tutorial demonstrations, sports, surveillance, scientific recording. animation, slideshows/crren recordings with transtioos, UI demonstrations, posters and charts within videos, mapstimelines,, split-screen and multi panel layouts, and other information rich videos.
Description Rules:
concise and natural paragraph (2-3 sentences) to summarize overall content of the video, subjects and their spatiotemporal dynamics, background information. and identify changes in shots and scenes time; in the main text, expund on these using time anchors or shot indices (Imm:s)), [S)].
General Introduction:
square brackets. Prefer absolute time [mn:ss] or [hh::mmsss]] ss]; the timeline unstable, is livestream, or absolute time cannot be obtained, is permissible to use approximate time I mmsss], shot indices [SI], [82], coarse segments (eeginnnnn/miklleeenn]]]] or relative anchors [TO+00:15][+ 15{] Choose main scheme throughout. if mixing necessary, explain the logic first Indicate time intervals as "[a-b]" 1' (in line with the chosen scheme). Cleadly mark cransitions/cuts at the relevant anchor. For the first cleadly legiblc on-screen text/chart, give
Temporal Organization
Anchors: Depict video content
chronological order, uniformly marking
determined; not, use approximate time indes.
Key Scene Description:
focus idlettfying primary and secondary subjects, describing their visual characteristies in great detail, including subject type, appearance, color (hue, brightness, saturation), geometric shape, quanrity absoluto/rclative position, relationships (occlusion, stacking, alignment),, cntering/cxiting the frame, and changes in visibility:
Main Subject Description: For each
or range),
Events and Actions: Identify and mark key
progresses, focusing subject actions and internctions, including direction of movement (described using screen coordinates and reference objects), speod/thythm (slow//ncdiun/fss or estimated froquency. e.g.. "abeut
arrangement changes. Clearly describe interactions the environmentt objects, and other subjects, as well as the sequence of events:
per second"), ). posture,
Scene Environment: Describe in detail overall layout and characteristies of the and the arrangement subjects within the sozne:
general world knowledge (such animals, plants, famous people. famous landmarks historical sites) is involved, must he clearly specified; do not use vague expressions such as "somenne" or "somewhere".
29

---

Published as a conference paper at ICLR 2026
and other clemenis, use technical terms to de- surihe their type, geometric structure, quantity local details, layout, relative position, legends/avevscaless units, horders, color blocks, and encoding; formulas may be presented in LaTeX (keep descriptive, do not derive or analyze).
Abstract/Tschnical Elements: For (idtm///whmagg/llotattttttnnnend
Visible Text: Identify and extract visible text on screen (subtilles or backgrond text), naturally embedkiing into the overall deseription of the frame, and try to specify font size, color, and spatial position.
Camera Movement and Changes: Attempt to ideatify shot scale (long/medium/closc) viewpoint (hightlowleye-lovel,, camera movements (pan, zoom, tracking) with direction and intensity, composition and depih cues (such widettelephoto appearance), transitions (cut, dissolve, wipe), and lighting changes; only describe what is supported by visible evidence. For sereen record- ings, slides, still or static images. prefer terms like "viow movemeu//crollin/UUU UI zoom/element fadeldigital zoom//weeeing" ; use camera/lens lerminology only for vet-action lootage, and avoid mistnterpreting UI
optical zoom.
Style Features: Naturally incorporate the stylistic features of lighting. lone, contrast, saturation. color grading, realism, and strmosphere of the subjects and background into the paragraph, du not list them as lags or in bullet points.
Blur/Oecllasio and Uncertainty: For blurre,, partially nocluded, or fleeting elements, only descrihe what can confirmed; qualifiers like pusgetollposhhheiuinniiaall when uncertain, and do not output spoculative details; avoid introducing information from outside the Irame.
Formats and requirements:
Only use coherent natural paragraphs; not use any lists, headings, semantic labels; not include invisible content or state missing elements (e-g., "no text screen"). "..e""".
Oulput format:
2. Natural entry: Directly descrihe the scene content, without opening phrases like "this scene shows..." Avaid repetition and uncertainty, do not use speculative language such as "might", : "probably", etc., do not describe information not present the frame, and do not phrases like "no text scroen".
3. Time anchors: Use square brackets to consistently mark [mm:ss] or [bh:mm:ss] key points (consistently throughout), c.g., "|00:12-00:281": for very short segments, use "loogiaiin/mmdddleeeddd Clearly indicate transitionscuuts at the corresponding time.
Coordinate reference: By default, describe position movemmett using screen coordinates and structural references (leftrighu/ top/hottom, quadrant, centerline, edgelcomerr relative to another object).
5. Quantification and units: Use qualifiers such us "about/ut lenst/at most" for quantity, duration, speed, frequency, and progortion; provide calculated values only when they can measured from frame, and report with visible precision and units.
Uncertainty: Only state conclusions with visible evidence; use "posihlesuppccedmmdeeiifii external facts unless explicifly shown in the frame or visible text.
uncertainty; not introduce
7. Multi-thousplitssceee and overlays: At Atllsss specify shot/scene and split panel position (e.g.. lefbrigguhhebbctto///iidd index), mark the appearance duration overlays such scorebourds, UI panels, subiile burs.
Academic cultural scenarios:
technical terms for scientificheehhnica visual elements; for ariisticaesttetiic content, more
expressive vocabulary can be used, bui maintain an objective description
9. Length: Detailed description should be no less than 500 words and no more than 800 words.
Prompt for Video Reasoning Agent
lop-level anuly lical expert with extensive world knowledge, deep spatiotemporal understanding, and rigorous logical reasoning skills, particularly adept at transforming complex viden information into concise and insightfull knowledge documents. Task Setup:
Your task is to transform video (ur sampled frames) into independent, citable video knowledge documeett. This documennt shoold unfold in logical hierarchy from overall to detail, from deseription to interpretation, with a level of depth and rigor suitable for academic illustration. The outpot should stand alone and be suitable explanatory material research reports or publications.
Analytical Structure:
1. Structural Overview (2-3 sentences): Summarize the video's 's temporal structure (such as scene shot distribution), main subjects, environment, partitions or overlays (such as scoreboards, UI pancls), as well as main states and any possible repetitive structures.
Inference Body:
Detail Perception:
This step focuses on precise capture of objective facts (What thene?)
Accurately identify describe key subjects (people, objects), environmental features, and any visible text or symbols. For tables, charts, and other visualizations, accurately extract data, axcs, units, and legends, and describe their basic distribution and treads. All information must be based on visible labels.
Action Understanding and Causal Inference: This step focuses on logically organizing dynamic processes (What happening why?)
Using temporal anchors, construct a "subject time location event causality" chain.
Analyze in detail the stages of actions, changes of and variations in spatial position. For process demonstrations or tutorials, follow visual cues such as cursors or arrows to clarify steps, dopendencics, and
execution order.
Explain direct causal relationships between events (e.g.. A knocks over B, causing B to fall).
Semantic and Thematic Analysis: Try reasonably interpret meanings (What does mcan?)
Basod on visible evidence. analyze the intentions of subjects, the purposes behind actions, and possible social relationships
(such cooperation or confrontation).
Interpret the symbolic meaning of cultural, historical, or artistic elements in the video, and explain how they serve the overall theme of the video.
30

---

Published as a conference paper at ICLR 2026
Analyze the narralive or emphatic function of camera languuge (such push, pull. pun, track, or angle switch). Summarize core theme,
atmosphere conveyed, argument attempis make.
Formats and requirements:
based visual textual evidence visible vidco; world knowlodge may only be used to explain or supplement such evidence, and never to invent details not supported by the lostage.
Evidence-based inference: inferences
Naturalized content: Organize your unalysis as coherent, fluent natural language article; is permitted to split content into multiple paragraphs stepwise explanation. Avoid rigid
headings excessive bullet points maintain overall narrative unity. Brief lists may be used with caution only when enumerating purallel items (such as technical parameters or procedural sleps) clarity.
Length limit: For simple scenes, koop analysis needed to ensure all key information is covered.
than 800 words); for information-dense. complex soenes, expand
Professional formatting: Use appropeiate formatting (such as LaTeX for formulas, code, chemical structures) for technical content ensure accuracy.
specialized vocabulary professional while expressive vocabulary may used for aesthetic or artistic scenes.
Vocabulary expression:
Prompt for Video Guideline Agent
You expert syntheskzing and distilling video information. Please, in single natural paragraph, concisely and insightfully summarize the video's main themee, core narrative or process, main subjects and notable events, as well as visible stylistic features. Focus comprehensive overview, highlighting the video's uniqueness and significance. Formats and requirements:
Aveid shot shot-by-shie recounting stepwise reasoning:
DO NOT lists headings;
Length should 100 words;
Replics must be strictly based visible content, avoiding subjective speculation.
31

---

Published as a conference paper at ICLR 2026
Summary Agents. The Summary Agents consist of two variants: general summarizer and video summarizer. The summarizer is prompted to integrate visual cues, reasoning processes, and other relevant information into highly professional document. Distinct guidelines are applied for static images and videos to ensure that spatial and temporal elements are properly incorporated according to predefined rules. The detailed prompt templates for the general summary agent and the video summary agent are shown as follows:
Prompt for General Summary Agent
You are an expert in writing professional academic imuge description documents at the highest level. Your lask is to synthesize multiple expert-level image description documents provided by user, and generate comprehensive analytical document that fully incorporates fine-grained image details and demonstrates expert-level image understanding. The focus is the accurate depiction and summarization of fine-grained visual details.
The provided image description documents are mainly divided into two parts: A. Image descriptions from different perspectives (which may include text recognition results, general summaries, and various line graigggandd details from multiple angles); B. An interpretative inference ahout the image content based visual information.
Refore composing your output, carefully review and understand all descriptive documents ensure you capture all visual elements and their characteristies. Analyze the inference text to extract key information that nids in understanding the image, and integrate this information your comprchensive description.
output should consist of coherent, label-free sentences and paragraphs, organized as follows:
1. Begin with highly concise paragraph summarizing the image type, theme, purpose, composition, layout, color scheme, and visual style, enabling readers to
grasp the core information and function of the image.
2. The main body should consist of detailed description and reasoning analysis, each presented in several logically connected and fluent paragraphs. The detailed description must be as thorough as possible, while the reasoning section should emphusize logical consistency and causality.
For the detailed description:
Organize and merge fine- gne-grained information from all description documents by categorizing according to objects described, then reconstruct the information into semantically coherent description;;
Integrate isolated or fragmented information, ensuring that all artribates, decorative features, appearance characteristics, spatial relations, functional relations, textual data, and any other types of information prssen the documeats are fully explained:
Data from documents, tables and charts should be described in fully connected paragraphs or clearly structured lists, and examples should he avoided.
For the reasoning analysis:
TThe reasoning content must immediately follow the detailed description and should thoroughly reference the key elements described g. "According to the data in columm of Table the company is shown be operating loss"): RReusnnngg causal unalysis, or structural explanations must be constructed only based on the inference document and the detailed descriptions, no information outside the provided documents should introduced;
When numerical. data analysis, or causal inference is involved, important reasoning processes may be elearly presented using latex-style inline formulas.
For multi-image or complex images, provide separate, thorough description of the key features of each sub- aah-inage, and supplement the overview with internal relationships, causality, and logical connections between images. For flowcharts, time series, sequence images. the order of description may be adjusted appropriate to fit the structural characteristics.
Formats and requirements:
1. For scientific, engineering, or clinical images, use precise, prufessional, and logically rigorous languuge consistent with domuin. specific terminology and reasoning. artistic, aesthetic, knowledge, cultural images, use more expressive and sophisticated vocabulary and sentence structures.
2. All visual information should be strictly consistent with the original content, especially the texts, numbers, symbols, etc.
3. State all content and interpretations directly, without using introductory phrases such as "In the image description section," "For reasoning analysis." similar expressions. Avoid repetition and uncertain statements.
"For the paragraph breaks, connecting words, and structured lists or inline formulas is allowed to enhance readability and rigur.
Maintain logical coherence and clarity between sentences, paragraphs,
overall document. Appropriate natural
5. Avaid information redundancy by ensuring that the same object is not described repeatedly, and eliminate meaningless or excessive statements.
If any image details analyses uncertain, conflicting, unclear, explicitl point such discrepancies the analysis, and priocitize adopting the more detailed or contextually consistent explanation from the reference documents.
7. The use of semantic labels, ballet points, or heading markers (such as "Tirle", "Detailed Deseription". "Reasoning Analysis", etc.) strictly prohibited.
Prompt for Video Summary Agent
You top-level document integration expert, skilled merging fragmented video information from different dimensions and constructing a logically rigorous, detail-rich, and deeply insightful professional-level analysis document.
Task Setup:
32

---

Published as a conference paper at ICLR 2026
Your lask is to inlegrate mulliple user provided video description and inference documents into single, unilied, cohereni, and logically consistent video description report. This report must achieve comprehensive perception and deep understanding video content, meeting the standards required for direct academic publication professional reporting.
Merging Principles:
1. Information fidelity and de-duplication: The primary goal is to ensure no loss of information. Ensure that all key information (subjects, atributes, actions, spatiotenmporal relationships, text, etc.) input documents accurately represented the final report.
The final output should smooth, narrative article, not structured data list. Strietly prohibit the use of any semantic labels, ballet points, titles (such as "Detail Description", "Inference Analysis"), All content should he naturally integrated into paragraphs.
Narrative flow priority:
Evidence-based reasoning: All analysis inferences must originate from visual details explicitly mentioned in the input documents.
Output Structure and Execution Process:
1. Begin with a highly condensed introduction (2-3 sentences) to establish macro level understanding for the reader. Content should cover
core narrative, subjects, scenes, key events, and visual style armosphere created by shots, lighting, and color.
2. The main section should locus on summarizing the key scenes of the video, with each scene including both content perception and understanding.
Detail Description (Content Perception):
Use precise time anchors to explain important secnes and key events the video shot by shot. For each shot scene, strictly reference the relevant content from the documents, and categorize and merge descriptions by object and event, restructuring them into semantically coherent, vidco-level description; organize and integrate isolated or scattered visual information, casuring key points covered while avoiding redundant repetition
element.
All key information points must be aligned with precise time anchors frecommenneed format: [mnmass] or [mmmess)]]. Ensure consistent naminn entities thmughout the documentt
If there are uncertainties, differences, unclear information in image details or analysis, these differences should be clearly idcatified in the analysis, and the more detailed part of the reforenced documents should be adopted.
Inference and Analysis (Content Understanding:
closely follow the detail description, thoroughly citing elements from the perception content for analysis (c.g. At [02:13), 13), the door closes, and at [02:16]. the light turns off, forming a sequence that triggers the alarm), focus natural progression from "detail perception" "action understanding causal inference" "semantic themattc anulysis":
Inference content
explanations hased inference documents detailed content; introducing information not present the documents prohibited.
Only inferences, causal,
3. Finally, use a highly condensed paragraph (2-3 sentences) to reinopecttielyy summarize the core logic of the key scenes and important events that appear video, and synthesize and explain cxisting inferences.
Formals and requirements::
videos with scientific, engineering. or clinical significance, use accurate, professional, logical, and domain-ssocitte vocabulary and sentence structures; for artistic, aesthetic, knowledge, and cultural content, more expressive and advanced wording and sentences.
visual information
strictly consistent
original content, especially texts, numbers, symbols, elc.
Directly state facts and inforences, avoiding guiding phrases such "This section describes..." "Te following inference.....
paragraphs, the overall document. Moderate natural paragraphing, connectors, and appropriate lists or inline formulas is allowed to enhance readability and rigor.
Maintain logical coherence and structural clarity between
The use of any semantic labels, bullet points, or title markers (such as "Tite"" "Detail Description" "Logical Reasoning", etc.) is prohibited.
33

---

Published as a conference paper at ICLR 2026
A.7.2 QUALITY EVALUATION PROMPT TEMPLATES
In the reject sampling stage and caption quality evaluation experiments, we use the following prompt templates to conduct quality evaluation.
Prompt for Image Quality Evaluation
You are an expert in multimodal content evaluation responsible for the rigorous quality assessment of comprehensive image-text description. Task Sctup:
You must strictly evaluate the candidate description based solely on visual information provided by the image, scoring precisely from I to points the following live cure dimensions, Every must have cleur and objective supporting reasons.
Core Evaluation Dimensions:
This dimension assesses the absolute consistency between the description and the factual content of the provided image or images. Chock whether all entity types, attributes, quantitics, locations, OCR text, and chart data are accuratc and casure that there internal contradictions within the description.
Factual Accuracy:
Information Completenes:: This dimension evaluates the coverage of the image's information. Check whether all salient elements, important relationships (such as interactions and hieranchies), and key details (such ases usewand legends) are comprehensively covered.
logical inferences. cansal relationships, or conclusions are fully and directly supported by visual evidence. Over-interpretation or hallucinated knowledge without evidence is strictly prohibited.
Reasoning Rigor: dimension assesses the rigor reasoning.
Core Intent Capture: This dimension evaluates ability the description capture the core intent of the image. Assess whether the description successfully distill and conveys the image's 's main message. purpose, theme. rather than merely listing details.
5. Professionalisn & Expression:
This dimension evaluates the professionalism and standardization of the language. Assess if the language fluent, coberent, logically clear, strictly Kollows natural language paragraph format (ltem lists or subheadings before the paragraph strictly probibited), and whether word choice and formatting meet professional domain standards.
Scoring Standard:
Serious issues or completely inconsistent.
2: Basically meets requirements but with obvious deficiencies.
Highly consistent, obvious
dimension.
Issue Tagging & Explanation:
1. explanation: Explanations must be concise and to the point. For deductions, directly point "what is wrong": for full marks, hriefly state "what is good".
issues: You must select
labels following preset precisely tag identified problems: l'Entity Error'. 'Altribute Emorr, "Quntity Error', 'Pusition Relation Errer', 'Hallucinated Existence', "OCR Error, 'Reasoning Fallasy'. FFectull Eroor', SormmtureFoomat Violation', 'Core Missing')
Formats and requirements:
Strictly oulput in the following ISON farmat:
["Eaczual haa.allaccccccyy "competenees""; 'reasoning_rigor": ccor_intenn.ccppuure "professis onna canliammexgresssion. "averallaccore"; "isques";, "explanation":)
Prompt for Video Quality Evaluation
senior multimedia content analyst, specializing in the evaluation dense, long-form video descriptions. Your task provide rigorous, nulti-dimmnnional quality assessment of caudidate video cuption against the source video.
Task Setup:
Strietly evaluate the caption based on the vidco's visual and temporal information. Score each dimension from 1 to Your output must be single, valid ISON object.
Core Evaluation Dimensions:
This dimension the absolute correctness facts within their spatial and temporal context. It examines if all described objects, atiributes, actions, and on screen text are accurate at their specified timestamps ([mmuss]) also checks for consistency in entity identification across different
Temporal & Factual Accuracy:
This dimcasion assesses the comprelensivenes of the description. It examines if the caption covers all key events, primary character actions, significant scene chunges, and critical details like camera movements (pans, zocms, cuts) or interactions with L. elements.
Event & Detail Coverage:
Temporal & Causal Logic: This dimension assesses the logical eoherence of the narrative, examines the sequence of events is described in the correct chromological order and any stated cause and effect relationships are directly supported by the visual evidence the video.
Core Narrative & Intent: This dimension assesses the caption's ability to capture the video's main purpose. It examines the description succesfully synthesizes details to convey the core story, main process, or ceniral theme, rather than jusi listing disconnected ohservations.
34

---

Published as a conference paper at ICLR 2026
Coherence & Formatting: This dimension usesses the quality of the language and siructure. It examines the caption's clarity, narrative flow,
required formatting (e-g,, correct use listsheadings).
Scoring Standard:
Severe issucs completely incorrect.
2: Largely cvrrect but with noticeable llaws.
3: Highly accurate and well-formed with no signilicant issues.
Issue Tagging Explanation:
concise directly pinpoint (timestamp possible)". For full marks, briefly state "What good".
explanation: explanation
deductions, state wrong
issues: You must select one or more lags from predelined list below to categorize identilied problems: I'Entity Error, 'Attribute Error,, "Temporal Error', Soquence Error', 'Causality Error', 'Motion/Action Error,, "Coverage Omission', "Formatting Ervor', Knowledge Ervor, "Core Intent Missing'!
and requirements:
Provide your ovaluation strictly in the following ISON format: ["cemporal.f *aora-rarrat va-intent" "ccherance formatting" ;, "averall Lccors "isques":, "explanation"
;, "event.detad wtt..tteeee..eeae...............
35

---

Published as a conference paper at ICLR 2026
A.8 VISUALIZATION
Comparsion between Omnicaptioner and MetaCaptioner-88. As illustrated in Fig. and Fig. 7 compare the performance of MetaCaptioner with the previous state-of-the-art method, OmniCap- tioner(Lu et al. 2025). on tasks involving multimodal data and flowchart descriptions. We marked the correctness and consistency of the content green, and mistakes and hallucinations in red.
Compared to OmniCaptioner, MetaCaptioner demonstrates superior ability to generate highly spe- cialized visual descriptions for content exhibiting intricate visual structures. Notably, MetaCaptioner achieves significantly better performance in the perception and understanding of abstract and complex images. Furthermore, benefiting from high-quality data annotation, MetaCaptioner exhibits pro- nounced advantage in the logical coherence of its descriptions. Specifically, when confronted with complex visual elements and procedural information, MetaCaptioner able logically describe and analyze visual components, thereby effectively mitigating the hallucination and over-interpretation phenomena that often occur in visual-language models when processing complex graphics.
present the visualization results GPT-4. (OpenAl, 2025a) and Capflow on the geometry problem and video description. We mark the correctness and consistency of the content in green, and mistakes and hallucinations red.
Comparison between GPT-4.1 and our CapFlow. As shown in Fig, 8 and Fig.
Solving complex geometry problems places extremely high demands model's ' reasoning ability. Our Capflow achieves the same level of perception as GPT-4.1, 1, while offering finer granularity the perception of structured knowledge, and its responses exhibit high degree of logical consistency. Although some errors still occur during the inference process, identifying intersection points not marked in the diagram (such as the intersection of line LM and the circle) remains highly challenging- even for GPT-4.1.
In the video description task. Capflow demonstrates significant reduction in excessive inference and analysis of visual elements compared to GPT-4.1. This improvement can be attributed to Capflow's decoupling mechanism for visual tasks. Furthermore, we observe that Capflow provides more detailed and comprehensive descriptions of specific timestamps. Unlike GPT-4.1, which often produces formalized expressions, the captions generated by Capflow are semantically richer and more diverse. These findings highlight Capflow's 's substantial potential as powerful data engine.
36

---

Published as a conference paper at ICLR 2026
Infographic Image
Elrenicte
Please provide a detailed description for this poster:
Own i Capt i onen
image
With white Uustratio deplcting soene Chron Namba: paga. Above the
laea, skyf figure
ciegendlly
680
60D
200
head wings vasible
To the loft the sword, two charactors are engaged in combot. One character appears to be youngg male weeting whilte shirt and darkk pantl, while the oltwe charactee tree beardde heir and weens ned lop. Buil fiuuns hare tierr arms exdandeed lowards each other as if they ane attempting to canch something mid air.
668
616
On right side the image, another character depicted motion, seomingly loaping forward juckel and
leg
Al the botton the imape small perloriish dag-lkiee crealure with u round body and dicambble facial featurea alands find next the langer sward.
The background consists of simple geomaticc shapos, p pradommanndy triangles, which do not disract from hhe main action taking place. The overal composition is dynamic, with the clagonal plecement the wword creating i werse movemnntt theoughoot the Essttattoo
384
256
389
66
DO
Factual Erroc lack logic and professionalism.
Me MetaCaptioner
The image vixuatly enpaging and informative foggaaphic Iiled *The Chroride Ancient Olympic Games Classical Graeco." which chrorides with key everts
evolution of Clympic o Games their inception 778 BC thair ccourence
The infographie is structree design, using deecrptone. employe almosohore, Smeline veusa compestion,
years, accompanied symbolic Bustritione schome orange issed chronclogically highrihtt affectiely convey trater progression of the Olympic Gamex, appaniing to suchence inlewalee hodoryy sports, and classicll stuciee.
ensuting clariy
of
designed to
intograhic begins with the first Clymplc Games in 775 BC, where only the un or stade race was held. The Inttll event represented runningg Spure, symbbolznng the simpkeity of the early games. As the smmlnn prograsses the intioducion of the diaute (384.5 meleri) in 728 BC is depicled with an alusration of awo-slage race, followed by the sevan-stage race (1345.0 meters) in 720 Bc, Ilsttratdd by a nunting figure on a marked track. The progression of noents is futher detelled with intorlationn the fnurhhone chancol recing 880 IKC, meprseeted by chwriet with four horees, and bocing lih IC: Bustrsted by ton bosers in manch The penteltion, indluding unelling the stedum, long jump, jeosin frow and diass throw, was inteaduaed 708 BC, depicted by a sihouette man portorming these actiities.
it houetee of
smarine cont uns with introduction pariiration riding ilustraeed
reproserreed combol
and
20 in 832
nunnrr marked contesta 568 BC are represented teo figures halding homms while two-toons charott rading was introuoodd 408 BC, Blusratted by charlot with beo horees. A two stage race in armar was rndoduted in 520 BC, depioted by man in amoor running. The four foal charist racing was introduood in 384 BC, represented by cheriol with for foels and swo-hall chorce recting was moddlaed 284 BC, Blserated by chwint with two loads Horwebek raong was infrddsded 256 RCC. ceepictdd by man riding a hase The finel anerts inclade the hed Cames in snlquiyy in 360 AD. reprosentedd by a cheriot recs, and Emparor Nero postporing the Gomes 66 AD. Blstirttd by chanes raco. The smeline concludes with Sul decres 80 BC to move the Olymplcs to Rome, dep atdd by a mon riding a horse.
Besing young
introduced 616 BG, depicted
bosers match The herald and trumpeter
The visual alements the infoprattic are carefuly chosen to complement simple, houetees atrietes
infomation, antandingg the understanding
hastoiicll events. dear steua appealing ftsmes marking years. micnng of infographic. further emphosizo
of
againet backgrund, orange
nirning figun
the toxt bottom image inciades cwelts for eddor designet dinecoor dieedtrr and temploie suthor, widiing professiore touch the ovenel tesign.
ilustations. timeline framed orange
chronologica
The time the effectively ulssrates the grocual ovolution the Olympic Games, nrom single event in 770 BC to complex senes events the oth century BC, The introduction or new events, such as the deaulss, the seven stage rooe and the perrathion refeett the growing sooh diccaton of the Games. The inclusion of charkct racina and harseback racing in the lale years indisates shi't towarda more diranse and challeng linginn events. The dimelne also highlghts the culural and passcall fnctors that inflreceed Gamen, such as Emppor Nem''s poepponemnnt and Sulli's decwe to mooe the Olyppiss Home
finel woent 309 wheen the Gares ware probiitted Fmpparor Therdissiu merks and of the andenn Olymeic fned tion infogreshic provides compreheratiee coendem of Andent Olympic Games, effectively conveying historical progression combiaatin of visual and textual elements creates cohesive informatvve narratie that both engaging educational
Games and the cuttural poliicall factors that shaped their deve opment. The
All descriptions are consistent with the Images; factual inferemces are accurrat;; the understanding of visual elements thorough: the captions meet the professional standards the fteld.
Figure 6: Comparison between Omnicaptioner and MetaCaptioner-8B in the Inforgraphic Image. The correctness is marked in green, and the incorrectness is marked in red.
37

---

Published as a conference paper at ICLR 2026
Diagram
The spring force does negative work, decreasing speed and kinetic energy.
Frictionless
Please provide detailed description for this poster:
Stop
First touch
Own i Capt oner
Diagram physis illutrraiing the effect spring spring
moving from right frictionless image mass depicted green rectangle, moving spring constant
inwards right. A pink arme lébeled indicates the uebcity the mass diagram incudes annotations: spring force does negative work, decreasing speed and kinetic energy" in the upper left comerr "Frictionles" near the mass, and "Stop" and "First touch" at the bottom left and right.
respectively. The background a light beige tolor, with blue gradient under mass indicating the surface, Technical ilustration, clear and educational, minimaleeicc design, precise line work, muted color palette, informative, scientific diegram.
Few errors; the description is not detailed enough; the image recognition basically accurate.
Me MetaCaptioner
image is schematic regrosentation physics scenario insthving the interaction between mass os spring with spring constant purpose the diagram to illustrate the piiciples of snergy conservation and work in mecharical system, emphasiing the rale of the spring force decolerating the moving object. The compositio induudes fritionless hor contaaaal surface, a block libeled "mm" colled spring on the left, and various annotstions that clar distances, foroes, and motion. The color scheme soft pastel tones- light blue for the surface, light teal for the block, and pink for certain elements like educational approachable atmosphere.
image begins with vectical pink indi coting its spring constant. The spring is attoched to brown restangular block on the right, symbolising the fixed end. To the right of the spring a light teol rectangular black labeled "m," representing the moss. purple arow above the block points the left, indicating the direction of motion, while a black ourved line connects the bleck the text "Rrichionesss" signiying the absence of frictional forces the surface. Below these elements, label- "Stop" and First the lobel).
the left, acts boundary or refeeence point. Adjacent to is tightly wound gray spring labeled "
touch" elaced poottion where the block
double headed contact the spring (the'
labeled "d'"
spring compresiing (the
A key feature of the diagram the explonation provided a white box the top left comnar: "The soring force does negative work, decreasing speed and kinetic energy. The statement suctinctly captures the essence of the physical process depicted, highiighonn the interplay between the spring's restoring force and the block's lonetic
mrangement of elements dearly illustrates the sequence of events: the block toward the darity and logical making
force. The alignmentt "First touch' to "Stop."
elements
the progression the block's
The detailed analyis of the image reveals several important relationships and interactions, The block as Initially moving with velocity u. aporoaches the spring, inttaaing a dynamic interaction. on. As the block touches the sprinn for the first time, the spring begins compress, storing potential energy. The compression continues until the block momentarlly fly halts ( (Sino"), ", at which point all of the block's initial kinatic enengy has been converted into the spring's potential Throughout
blodk's mosion, serforming nogative (W and thereby reducing block's speed and kinetic energy. According the principle work energy, the change kinetic energy AK the ecual forces acting on
the
opposhton
done all
AK Waperng Watter
scenario, since surface fnctionless, other signFicant energy salely attributed the done by the spring force. The distance represents the total dispacement the block from initisl postiion the point where stops, directly correlating with the amount of work done by the spring force and the resulting reduction in the block's knnetic energy.
except spring force.
decrease
The image effectively conveys the fundamental concept energy transfer conservation within simple harmonic ascilistor system By visuall depicting block's motion, the
associated trandformations diagram compression. layout, dear lobeling, and colors enhance compreheneton, ensuring that the intended sudience can mechanics energy prtnciples esotly grosp the underlying physics concepts.
excalient educational bool for understanding standing
This description is overall clear and well-structured, accurately conveying the physical meaning and educational purpose of the image.
Figure 7: Comparison between Omnicaptioner and MetaCaptioner-8B in the Diagram. The correctness is marked in green, and the incorrectness is marked in red.
38

---

Published as a conference paper at ICLR 2026
Geometry Image
K
X
M
17
X
10
L
Please provide detailed description for this Geometry.
GPT-4.1V 1V
This image a geometric diagram invalving
triangle outside circle. Here detailed deseription: Circle: There is single cirdle on the side of the image center la labeled M.
Hadius: line segment extends from the center point the circumference. This segement labeled Indicating length.
Chord and Tangent: From point the circle, there right angle (denoted by a small square at and another line extendss outwards
to point outside the circle,
Triangle KML: Points K and l. are consected, forming triangle MKL. The side KL (opposite the right angle at K) is labeled 17, and the side ML labeled 10.
Right Angle: right
suggests
perpendicular
Labels and Style: All points (M, K, I) are lebeled with uppercase letters. The lines are straight and the circle smooth. The lengths
written cleur; bold numeric text beside the respective line segments.
Functiorality: The diagram appears
designed for intthematical or geometric problem, possibly involving the Pythagorean
theorem, properties of tangeents, or circles
Overall, this clean, simple geometric illustrattin depect scenarto involving ctrcle. tangent, and thetr respective lengths, likely for educational or problem-selving purposes.
CapFZow
The Image presents classic problem Enclidean geometry involving circle and secaut line. The overall composition and layout of the image are designed to elacidate the application of geametric theorems, particularly the Power of Point Theorem and the Pythagorean theorem. The diagram employs clean, instructional visual style
as tangent
triangle and text, a pink square indicating right angle. This simplicity onhances clarity and focus, making it ideal for educational purposes.
outline
black
diagram shows circle at K. forming right angle with the radius MK, as indicated by the pink square. Additionally, the secant line ML of the ctrcle
centered at
a radius MK, where the poins tangency: tangent
Intersects
segment and extending outside the ctrcle. The length of the secent line ML outside the circle given as 10 units, whille the total length from external point to the point of tangency is 17 units. The spatial arrangement is logical, with the circle occupying the central area and the triangle extending outward, creating a clear visual representation the geometric relationssips.
intersects the circle
points, dividing the
In detail, the triangle MKL has vertices M, and Side KM marked as which is also the radius of the circle. Side KL. labeled 17 units,
annotated bold, black uppercase letters respective points, and the vartable x appeans twice near the two radii and once above the side KM. The lengths 17 and 10 are written adjacent sides and ML. respectively, in standard sans-serif font, black color and positioned horizontally. The right- angle symbol distinctly marks the perpendicularity ML. to the circle, reinforcing the tangential relationshij.
units.
The geometric stracture reveals that MK perpendicular the tangent line the where they of tangents to circle The Line segment KL acts as secant line, dividing into two segments: one external (10 units) and one Internal (7 units) to the dircle. These segments establich the conditions for applying the Power of Point Theorem, which states that product the lengtle external point. This relationship can mathematically expressed ası
fundamental property
two segments the line equals square length langent
10 17
determine the radius of x, 10, and Mt. 17, we can express the relationship
we analyze right triangle MKL using Pythagorean theorem, Given
x² 10a 17a
Simplifying this equation yields:
x² 100 = 289
x² 189
Therej ore, The effectively combines visual and textual elements to cunvey the mathematical concept the geometry of circles and triangles It targets students or individuals studying geometry by providing a clear and focused presentation of problem, eliminating distractions and unhancing understanding and problem solving efficiency: The logical and systematic spatial aranngement, alonn with clear definition lines angles, facilitates easy interpretation application of goometric theurems. The solution obtained, x = v189, is derived purely from the provided geometric relationships and visual information, contirming the utility of the diagram for educational purposes.
189
Figure 8: Comparison between powerful GPT-4.11 and our Capflow in describing a geometry
image. The correctness is marked in green, and the incorrectness is marked in red.
39

---

Published as a conference paper at ICLR 2026
Video
TAy Al
Anwa
10
GPT-4 4_ 1V
Cartainy! Here's a decalled description and breskdown the prowided video, which appears to be countoown hightight real of the best goals from the FIFA Women's World Cup 2023:
General Overview.
The video dynamic countdown of the geals scored during the FIFA Women's World Cup 2023, featuring clips from various matches rounds. Each segment dedicctted to soectiic goad, shawing muttple angles moments from the bullcup, exccution, lower with renking number, and eech goel is introducee with an on-screen caption
graphic overlay namnin the scorer. the opposing team, and the metch phase (e.g g. group stage, semitinals.
aftermath.
Detalled Seane-by-Scone Description
England's Lauren James vs. Denmark (Group Stage)
Clips 1-3: The tirst throe image depic: England in white and Denma ik red during a group stage match. The action is near Donmank's gool. The scoreboard shows 0-0 at 5:14, Lauren James is shown scoring goel from outside the box. One shot shows the bail heading toward the top corner and the Danish goalkeepee diving in vain. Multiple camera angles: widee, behind the goal, and aerial.
England's is Lauren James vs. China (Group Stage)
40-43: Brazil yellow, Panama goalkeepars Action her dell css control.
score 2-0 47:37. Zaneratto is involved
attack, finishing with powerful past
Colombla's Linda Ca cedo Germany (Group Stage)
Cipps4- 48: Germany in white Colombla purpie/ycllow 0-0 at 51:47, Ca codo dribb into the box, evacies several defondors, and curis shot into the net. Celebrations with fans and teammates, and a shot of the ball in the
Australia's a Sam Ker va. England (Semifinaiss
Clips 49-53: Australaa yellow, England in blue, England at 62:20, Kerr makes a salo run, unieashos long-range shot, scores stunning Vanous show the trajectory the ball the reaetion fans and players. Ireland's Katie McCabe ve, Canada (Group Stage)
Clipe 54-58; tre and
Carads whits/red, 0-0 1:34, McCabe scores directly from corner kick (an "Olympica" gool), with the ball reactions.
curling
with dosse ups the ball the Ponamals Marta Cox vs. France (Group Stage)
Celeb
Clips 59- 64: Panama in red, France in white, 0-0 at 1:06. Cox takes a longrannge range free kick, sending the ball into the not for speclaculer goal. Teammates pile on in celebration as the crowd erupts.
CapFlow
This video is a vibrant montage of gigh from the FIFA Women's is World Cup 2023, showcasing series intense soccer matchos and remorkable goel-scoring moments festures dynanic plays. dtrilgicc teem efforis, and pivotal indivdual performence by notable players such as Lauren James, Teresa Abilera, and Kate McCabe. The videa captures the compotitive spint and global craitement of the toumement, with scenes high-quail quality toctago, amphasizing
vivily depicled stadiume filed with enthusisttic crowds. The visual style combinee cleer color scheme and vibrant
arrictictsm
players against the
settings.
The video begins by setting the stage with sequence from a match between Panama and France during the Group Stage. The lush green field is teeminng with activit players from both teems position themeelves stralegicall. The focus then narros onlo Marta Cox, Paniema' intlates ottensive the teneion builds, makes
display, disrupting French defensive Inc. move towarde penally area, engeging a confrontation the gealkeeger. With doft outmansuver, scores a gea that sends the stadium into a frensy colebration. This pivotal moment ancapsulatess the bland ind vidual brilliance and teemwork that charaderizes the loumement.
Her arbblligg and passing
c goa subsequent celebrations. players' unforms numbers prominently displayed, fecilaating essy ddeniffiation of key figuree Lauren James, Terees Abilleira, and Kale McCabe. The bal''s trajeciory and Interaction with players ard meticulously captured, adding depth to the portrayal of each critical play. Goalkespers are ottnn seen in dynamic poslions, either diving intercept or srrlegicaly defending the goel. The sucience, diverse in their slire and enthusisem, contribules to the lvvey ambiance of the event.
Expending further, video actions, the precise execution,
together collection similar moments
metches. Each highlights
capturing the
The lighting conditions acroos the clps vary sightly, reflecting different times of day and stadium sclups, yet all sequences are uniformly bright, ensuring optimal visibilly of the unfoiding adion. Brand advertisements from sponsors such as Power adw Dove and Hyundsii adom the perpheetes of the pitch, situating the matches within aroader commercial and global context. The vide''s eciting maintains seamieess flow, with smooth transitons that keep the viewer engaged, enhancing the nerrativ the tourement's progression from group quartertinals beyond.
The video's thematic essence less its celebration of women's football, spotlighting themes of teamwork, individual talent, and the competitive elhoo of the sport. The complation skilfully Illustrstes the universal appeel and indusivity of the tourement, featuring diverse array of teams and players. Camera plays crucial rale conveying message. employing wide angle shots during build-up phases coee-up during each play.
goal moments.
varistion framing
appreciate the tacical intricacies
highs
Analyzing the video's structure and content is evident that each segment corefully crafted culminate climactic goal. For example. one sequence unsuscesaful dive to stop The celebratory reactions is that follow each goel, ranging from hugs and jumps to juyfu sprints, emphesize the sfgcificenee of these ach evements within the context of the tournament.
free that elagently These moments underscore players' technical prowess and drama inherent footbal.
defensive
the net Another showssees goalksepar's
spectalors. Their reactions contribute the overal nerrative, refecting the global appea and importance the FIFA Women's World Cup 2023. Through Its dynamic visuars, meticulous attention to detail and thematic depth, the video invites viewers to not ony witness but also deeply apprec atc the amteticiimm passion, and artistry showssied by the fomale athieles competing on the world stage. The hamonious bland of indiviuaal aniliance and collective effort createss compellng trbute to the spint women's football, casturing the heart of this prestigious global event.
emphasis
emotional aspects
game
reinforced by animated responses
Figure 9: Comparison between powerful GPT-4.11 and our Capflow in describing a sports video.
The correctness is marked in green, and the incorrectness is marked in red.
40


Reviews:
[{"review_id": "jCz1C0jtza", "reviewer": "Reviewer_5uKz", "summary": "This paper introduces CapFlow, a multi-agent workflow for generating visual captions using open-source multimodal large language models (MLLMs). CapFlow employs domain routing and specialized agents to produce multi-domain visual captions, which are filtered via a reject sampling pipeline. The synthesized data is then used to fine-tune MetaCaptioner, a general-purpose captioning model.", "questions": "* The paper attributes notable performance improvements to the proposed domain routing mechanism. Could the authors provide more analysis or ablations demonstrating its robustness? Specifically, I wonder how sensitive is the system to routing errors or misclassification of domains. Is there a significant drop in caption quality when the routing assigns incorrect workflows?\n\n* While Table 5 shows that captions generated by MetaCaptioner substantially enhance downstream reasoning when used as inputs to external LLMs, Table 6 indicates relatively modest gains in its direct multimodal evaluation. Does this imply that MetaCaptioner mainly improves as a captioning front-end rather than as a genuinely stronger multimodal learner? If so, how does this affect its practical utility in end-to-end multimodal understanding tasks?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The manuscript is clearly written, with a logical flow that makes the overall pipeline easy to follow. The problem statement is well-motivated and practically relevant, making the work engaging and interesting to read.\n* If the dataset and models are publicly released as claimed, they would provide a valuable resource for the open-source multimodal learning community, enabling further research on generalist captioning and multimodal alignment.", "weaknesses": "* Technical novelty is limited. Although the agent composition and hierarchical workflow are carefully designed, the core components are incremental integrations of existing methods rather than fundamentally new algorithmic contributions. The work introduces no novel learning architecture or objective formulation.\n\n* Although the paper claims that MetaCaptioner achieves performance comparable to GPT-4.1, the presented benchmark results do not fully substantiate this claim.\n\n* Qualitative or diagnostic discussion of where prior models fall short and how the proposed method specifically addresses those limitations is needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CYZdV90EKI", "reviewer": "Reviewer_U7rU", "summary": "This paper introduces CapFlow, a multi-agent, hierarchical workflow for generalist visual captioning across nine domains. A domain router selects a domain-specific workflow; functional agents produce evidence; a summary agent aggregates it into a dense caption. Using CapFlow as a data engine, the authors synthesize MetaCaption-4.1M and fine-tune MetaCaptioner-8B. They report caption quality comparable to GPT-4.1 while cutting average inference cost to 10.5%, with further gains when scaling functional agents to 72B.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Concrete workflows for nine visual domains with explicit agent roles. \n2. A multi-agent pipeline used to synthesize 4.1M high-quality captions. \n3. Ablations show gains from hierarchical workflows and domain routing.\n4. ~89.5% cost reduction vs. GPT-4.1.\n5. Implementation details for data and training are provided.", "weaknesses": "1. Evaluation leans heavily on LLM-as-a-judge. Most headline claims rely on model-based scoring. This makes the results vulnerable to prompt/model bias. I would expect at least a small but representative human preference study per domain and a few task-grounded metrics to triangulate quality.\n2. Router robustness is unclear. The domain router is main contribution, yet there’s no analysis on mixed-domain inputs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NqKUTkxDuU", "reviewer": "Reviewer_tFTQ", "summary": "The paper introduces CapFlow, a novel multi-agent framework for generalist visual captioning using open-source models, and presents MetaCaptioner, a fine-tuned captioner trained on high-quality data synthesized by CapFlow. The work demonstrates competitive performance with commercial models (e.g., GPT-4.1) at significantly lower cost.", "questions": "Please see Weakness part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  MetaCaptioner achieves state-of-the-art results among open-source captioners and approaches GPT-4.1 in quality, especially in reasoning-intensive tasks.\n2.\tCapFlow demonstrates a substantial reduction in inference and data generation costs compared to commercial models.\n3.\tA well-motivated hierarchical, domain-specialized multi-agent design that directly tackles domain diversity in visual captioning.", "weaknesses": "1. Lack of Direct Human Evaluation. Caption quality is judged by GPT-5; potential model-based evaluation bias is unaddressed.\n2. Missing Gemini caption or GPT5 caption comparison. No caption-quality comparison to a closed-source baseline like Gemini, weakening “near-commercial quality” claims.\n3. There are hallucinations in reject sampling process. The MLLM (qwen2.5vl-7B) used in the reject-sampling pipeline may exhibit notable hallucinations.\n4. It is unclear how specialist and reasoning agents are instantiated and validated (e.g., structural perception, infographic perception, OCR, coder, texture, video perception; general/medical/knowledge/video reasoning). Are these just prompt-specialized variants? How is accuracy ensured relative to visual specialists? Is there cross-agent verification?\n5. The workflow appears relatively simple, so tightly binding the approach to a multi-agent framing can feel overstated. A leaner design—e.g., a single controller agent that orchestrates tools and drives self-evolving caption refinement (plan–act–reflect loops)—might achieve similar quality with less complexity and overhead.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Lb4TQErlDR", "reviewer": "Reviewer_V8hc", "summary": "The paper proposes a method for generating a small model which is capable of producing high quality captions for real world image data. This requires a complex synthetic data pipeline called CapFlow. CapFlow uses an MLLM domain router along with domain specific agents to generate initial high quality captions for a large amount of data. This is followed by rejection sampling to remove low quality captions. The data is then used to train MetaCaptioner an 8B parameter model which produces high quality captions. The paper compares with several recent open source MLLMs along with the closed GPT4.1 and shows an improvement over the baseline QWEN model at cheaper cost than GPT 4.1.", "questions": "I think there is a good contribution here but it is a bit hard to understand exactly what the practical impact of the work here is. On the good side I like the design of the CapFlow pipeline from a systems standpoint. The parts all make sense and automating it reduces human expense. I also like what the paper achieves with an 8B model, it's a good goal. That said it wasn't clear to me from the improvements in Table 2 on 8B how much of a difference the model makes in practice. What do the higher scores mean? How do I interpret them? This is coupled with what I would call an extremely expensive training time of 480 + 192 H200 days. That is not cheap, and it may be misleading for Table 2 to show 2c per 100 samples when the actual model training consumed significant energy cost and GPU purchasing cost. \n\n**Specific Questions**\n1. How do I understand the results in Table 2 in terms of their practical impact?\n2. What is the true cost of MetaCaptioner after accounting for training cost?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper makes a clear improvement over the baseline QWEN even at 8B\n- The automatic captioning pipeline is well motivated and carefully designed\n- The method could be useful for data generation tasks of its own", "weaknesses": "- Although the paper improved over QWEN the improvement does seem small unless scaling to 72B (Table 2)\n- Significant training time (480 + 192 H200 days)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)