Composition-Grounded Data Synthesis for Visual Reasoning
Authors: Xinyi Gu
Jiayuan Mao
Zhang-Wei Hong
Zhuoran Yu
Pengyuan Li
Dhiraj Joshi
Rogerio Feris
Zexue He
Venue: ICLR 2026 Poster (2026)
Keywords: Compositional data synthesis
compositional generalization
visual reasoning
chart understanding
Visual language model
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: We propose COGS (COmposition-Grounded data Synthesis), a data-efficient framework compositionally generate reasoning data to equip MLLM with complex visual reasoning in Chart and WebGUI understanding tasks.

Abstract: Pretrained multi-modal large language models (MLLMs) demonstrate strong performance on diverse multimodal tasks, but remain limited in reasoning capabilities for domains where annotations are difficult to collect. In this work, we focus on artificial image domains such as charts, rendered documents, and webpages, which are abundant in practice yet lack large-scale human annotated reasoning datasets. We introduce COGS (COmposition-Grounded data Synthesis), a data-efficient framework for equipping MLLMs with advanced reasoning abilities from a small set of seed questions. The key idea is to decompose each seed question into primitive perception and reasoning *factors*, which can then be systematically recomposed with new images to generate large collections of synthetic question-answer pairs. Each generated question is paired with subquestions and intermediate answers, enabling reinforcement learning with factor-level process rewards. Experiments on chart reasoning show that COGS substantially improves performance on unseen questions, with the largest gains on reasoning-heavy and compositional questions. Moreover, training with a factor-level mixture of different seed data yields better transfer across multiple datasets, suggesting that COGS induces generalizable capabilities rather than dataset-specific overfitting. We further demonstrate that the framework extends beyond charts to other domains such as webpages. We release the code and data at https://cogsynthesis.github.io.

Published as a conference paper at ICLR 2026
COMPOSITION-GROUNDEE DATA SYNTHESIS FOR VI- SUAL REASONING
Xinyi Gu Jiayuan Mao' Zhang-Wer Hong'
Zhuoran Yu?
Pengyuan Li
Dhiraj Joshi
Rogerio Feris" Zexue Hel.3
MIT UWW-Madison MIT-IBB Watson Al Lab
ABSTRACT
Pretrained multi-modal large language models (MLLMs) demonstrate strong per- formance on diverse multimodal tasks, but remain limited in reasoning capabili- ties for domains where annotations are difficult to collect. In this work, we focus on artificial image domains such as charts, rendered documents, and webpages, which are abundant in practice yet lack large-scale human annotated reasoning datasets. We introduce COGS (COmposition--rrunnde efficient framework for equipping MLLMs with advanced reasoning abilities from small set of seed questions. The key idea is to decompose each seed question into primitive perception and reasoning factors, which can then be systematically recomposed with new images generate large collections of synthetic question- answer pairs. Each generated question is paired with subquestions and interme- diate answers, enabling reinforcement learning with factor-level process rewards. Experiments on chart reasoning show that COGS substantially improves perfor- mance on unseen questions, with the largest gains on reasoning-heavs and com- positional questions. Moreover, training with factor-level mixture of different seed data yields better transfer across multiple datasets, suggesting that COGS in- duces generalizable capabilities rather than dataset-specific overfitting. We further demonstrate that the framework extends beyond charts to other domains such as webpages. Project page: https//coogyythhssissggo github.io.
data Synthesis), data-
INTRODUCTION
Pretrained multi-modal large language models (MILMM)) have achieved impressive performance across wide range of multimodal tasks (Liu et al., 2023c; Bai et al., 2025; Wang al., 2025a; Agrawal et al., 2024; OpenAl et al., 2024; Comanici et al., 2025; Anthropic, 2024), yet advanced reasoning capabilities remain underdeveloped, especially in domains where user reasoning-intensse query-answer data is difficult to collect. In this work, we consider reasoning capability over artificial image domains, including charts, tables, information graphs, rendered documents, webpages, etc. While such images are abundant on the web, datasets containing reasoning questions over them are scarce. However, developing MLLMs that can handle these reasoning queries is critical for downstream applications, such as building agents that interpret and edit documents or take actions in digital environments.
In this paper, we aim to equip MLLMs with these missing capabilities using only small set of seed questions in target domain. Our goal is to bootstrap from these seed questions to generate a large, diverse dataset of synthetic question-answer pairs, leveraging additional unlabeled images such as online charts or webpages. The key insight of our approach is compositionality: although a seed question set may contain only limited number of surface forms, each question can be decomposed into a set of smaller subquestions, which we call factors in this paper. Factors may capture primitive perception and reasoning steps, such as reading number from chart, comparing two entries in a table, or performing arithmetic. Crucially, these factors can be recombined systematically and compositionally to produce much larger space of complex questions.
We propose a data-efficient framework, COGS (COmposition-Grouniie data Synthesis), that oper- ationalizes this idea in three stages. First, given a seed dataset of questions in the target domain, we
WWrk done during an internship at MIT-IBM Watson AI Lab. Corresponding to pxy@mit.edv.. zexueh @mi t. edu.

---

Published as a conference paper at ICLR 2026
Seed Dataset
Factors
COGS Synthetic Dataset
What was..?
N.. True?
Which
Whatls What's 7
Is ''mmore.. than.. True or False?
Which... is greater?
subtract...
what
es
ndin
What wet.
More.: True Fabe??
What Chosa ABCD
Another Image Source
Wher k. If 1% what
What excluaigg... -.. nighest?
sta starting perception
calculation
What Whetl Whet it.. releett
fact checking
dittronee.)
Figure I: COGS: Starting from small set of crasonnng-innensive seed questions, COGS decomposes them into primitive perception and reasoning factors, which are then recombined with new image sources synthesize question- -answer answer pairs. This process expands both the quantity and diversity of reasoning types beyond the original seeds. Fig. 2 shows an illustrative example.
decompose each question into its constituent perception and reasoning factors. Second, we recom- bine subsets of discovered factors with new images to generate novel compositional questions, each paired with intermediate subquestions and intermediate answers as bonus. Finally, we finetune pretrained MILM using Group Relative Policy Optimization (GRPO; Shao et al., 2024), augmented with process rewards derived from the factor annotations for fine-grained supervision.
We begin our evaluation on chart reasoning, a domain that exemplifies the scarcity of annotated reasoning questions despite the ubiquity of images. Our experiments show that COGS signifi- cantly improves the reasoning capabilities of base MLLMs, with the largest gains observed on reasoning- heavy and compositional questions. Moreover, tures of datasets: training jointly on multiple datasets yields positive transfer, demonstrating that the model acquires transferable capabilities rather than overfitting to a partieular dataset. Finally, illustrate that the same framework extends to the webpage reasoning domain, highlighting its broad applicability.
framework naturally supports mix-
In summary. this work introduces a principled approach to bootstrapping new reasoming skills in pretrained MLLMs from a small seed query set, by exploiting the factorized structure of questions to unlock scalable synthetic data generation and process-level reinforcement learning.
RELATED WORK
Understanding artificial image such charts and web GUIs demands grounded perception and substantial visual reasoning. General-purpose MLLMS make this feasible through large-scale pre- training and instruction tuning (Liu al., 2023e; Bai et al., 2025; Wang et al., 2025a; Agrawal al., 2024: OpenAl et al., 2024; Comanici et al., 2025; Anthropic, 2024). Prior work on automatic in- struction generation (Wang et al., 2023c; Yuan et al., 2024), refinement, and evolutionary methods (Xu et al., 2024a; Zeng et al., 2024) increases the complexity of synthetic data for text reasoning. While these methods mainly search for reasoning trajectories in text space, we aim to analyze and augment the seed dataset by automatically detecting reasoning components grounded in visual fea- tures. Unlike approaches that rely on hand-crafted heuristies (Xu al., 2024a) or strong pretrained language models (Zeng et al., 2024), COGS extracts component groups from the seed data and uses them customize the dataset for the target task. In parallel with generalist MLLMs, special- ist models have been introduced to target these domains more directly, prioritizing structured text extraction, numeric grounding, and compositional reasoning. New benchmarks and data-synthesis methods based on human-defined heuristics have followed, and specialist models have been trained accordingly.
Chart understanding including description and reasoning tasks. Benchmarks emphasize diverse image sources have been designed with different templates or task formulation to evaluate the mod- els performance (Kahou et al., 2018; Kafle et al., 2018; Methani et al., 2020; Masry et al., 2022; Xu et al., 2024b; Xia et al., 2025; Shi et al., 2024; He et al., 2024). Recent human-curated evalu- ations emphasize scientific real figures, multi-chart settings, and open-vocabulary answers, and
2

---

Published as a conference paper at ICLR 2026
(a) Soad Data
(b) Factors in Seed Data
(c) Re-composed Complex Qs
Decompasition
1.4. B. fo on
Re-Composition
I. Un fa.nn (R,, Pp ...d
Fine-tuning
Reasoning
MLLMs
human-curated seed data into factors
factors into
Original Q Uemilkania
analysietellatio
(demiiiction) (Beerfiictionn
Sampled Factors:
Carcuation
(Bland mmn
Iident/fiation staray growth.J
predictio
Sub Qs
calculation
[ddentfiiaaion:: populittion in.. for Niperio7, A: 43.54)
[[dbntiifcarinn22 public
comple
+jdentiiicatin2: populittion in.. for Indo. A 0.761
/alldattion:: femerma butoosn..... ... 42.78%)
Quittire Wiat ia the atadiins offermnce and the porcenaag public senvirs powthis prencier
[Calcutationt:: annlutaallue dfferrece 99)
ranking
selection
what the cifference between the ... Nigoria and Indi??
Re-composed
Seed Data
Decomposed Factors
Factor Pool
Re-Corl Pe-Composed Questions New Charts
Figure 2: The framework COGS consists of three stages: seed question decomposition, factor recomposition, and model fine-tuning.
they raise the bar with inference- heavyy questions that demand reasoning before answering (Masry et al., 2025a; Liu et al., 2024a; Wang et al., 2024b; Tang et al., 2025; Huang et al., 2025). Datasets and training strategies to specialist MILMM have been developed. Specialist pipeline methods first convert charts into structured intermediates and utilize an LLM answer on top of that representation, which improves numerical fidelity when extraction is accurate (Lee et al., 2023; Liu et al., 2023b;a; Xia et al., 2025). Specialist model can also be trained end-to-end for unify perception and reasoning inside a VLM, often by aligning multi-format inputs and instruction-tune tuning (Han et al., 2023; Car- bune al., 2024; Meng et al., 2024; Ye et al., 2023b;e; 2024; Chen al., 2023; Hu et al., 2024; Liu et al., 2024c; Wang et al., 2023a; Liu et al., 2024a; Zhang et al., 2024c; Yan et al., 2024; Masry et al., 2025b; Chen et al., 2024; Zhao et al., 2025; Jia al., 2025; Wu et al., 2025; Xu et al., 2025). Data synthesis approaches including question-level template and in-context example has been designed for specialist fine-tune a pretrained model (Li et al., 2024b; Chen et al., 2025; He al., 2023).
GUI understanding. GUI understanding has been studied through a growing set of benchmarks for page comprehension and reasoning over website and app UIs (Awal et al., 2025; Liu al., 2024b; Chen et al., 2021; Hsiao et al., 2025; Li et al., 2020; Chang et al., 2022; Wang et al., 2024a), as well as for grounding and agentic predictions (Liu al., 2024b; Li et al., 2025; Cheng al., 2024). Models have been developed for specialist tasks such information extraction (Back et al., 2019), detection//ocalization
UI elements for agentic use (Hu et al., 2024; Lee et al., 2023; Hong et al., 2024: Zheng et al., 2024; Gou et al., 2025) and general-purpose UI reasoning. (Ye et al., 2023a; Baechler et al., 2024; You et al., 2024; Liu et al., 2025; Wang et al., 2025b).
3 COGS
Fig. 2 provides overview of COGS, which consists of three stages. First, given a seed dataset of questions in the target domain, we decompose each question into its underlying perception and reasoning factors. Second, the framework collects all discovered factors across the domain and, together with an image collection, generates new questions by recomposing randomly sampled subset of these factors. Finally, the newly generated questions are used to finetune pretrained MLLM. During this stage, we leverage the factor decompositions associated with each generated question to define process-level rewards.
In this section, we begin with the problem formulation in Section 3.1, then describe factor decom- position in Section 3.2 and question generation via factor recomposition in Section 3.3, and finally discuss the process reward design for RL finetuning in Section 3.4. Prompts for decomposition and recomposition are presented in Appendix E.
3.1 PROBLEM FORMULATION
Our goal is to fine-tune multimodal large language model (MLLM) to acquire new capabilities in answering complex, compositional questions in a target domain. Let Q denote the set of natural language questions in this domain, and let I denote the corresponding set of images. A question E Q can often be interpreted as requiring a sequence of perception factors (e.g.. identifying a number in chart or localizing factors (e.g., logic, arithmetic, or spatial reasoning). We denote the factorized representation of a question q (f,, f2,..., {&), fi E where F is the set of possible factors.
element by its relation to another element a webpage) and reasoning
3

---

Published as a conference paper at ICLR 2026
Our objective is to use a small seed question dataset Q to bootstrap a process that can (i) discover the relevant set of factors F in the target domain, (ii) generate novel and valid questions by recom- posing subsets of factors, and (iii) use these generated questions to improve pretrained MLLM through reinforcement learning. Notably, we do not require ground-truth answers for the seed ques- tions, which makes the data collection process more scalable.
3.2
SEED DATA DECOMPOSITION
The first stage of our framework COGS is the decomposition of seed questions into a set of inter- pretable factors. As illustrated in Fig. 2, complex question that asks for the energy growth and pub- lic service growth can be broken down into distinct perception parcettion.cctorss and reasoning factors, In this example, the question requires (i) identifying the percentage of energy growth (Per cept on1), )), (ii) identifying the percentage of public services growth verception2), and (iii) computing their absolute difference (Calculation.)
We obtain such decompositions by prompting a MLLM. Specifically, we provide the MLLM with a natural language description of the decomposition task, a set of in-context examples (each consists of a paired question and its list of factors), the target question to be decomposed, and the image associated with this question to ensure each factor is fisuullgroundee
This step essentially recovers fi E F. For each
the factorized representations of the given question q - f2.,.2..... factor, the MILLM outputs a category label (e.g., Ca culat on, Count i ng) and corresponding subquestion that describes the role of this factor in the original question. These subquestions serve as exemplars of target categories that will later be used during the factor recomposition stage.
We then aggregate all factors discovered from Q" to form the space of possible factors F. Each factor is represented by a category name (e.g., Calculation, Count counting, Comparison and is associated with set of exemplar subquestions extracted from the seed dataset. The obtained fac- tor set F serves two purposes. First, it builds a compositional representation of the latent structure underlying complex questions, making it possible to recombine factors into new questions in the domain. Second, it provides fine-grained supervision for reinforcement learning: since each gener- ated question is associated with its underlying factors, we can define process rewards that provide intermediate signals for accomplishing individual reasoning steps.
3.3 QUESTION GENERATION VIA FACTOR RECOMPOSITION
The second stage of our framework COGS is generate new questions by recomposing previously discovered factors. As illustrated in Fig. 2, the input to this stage includes: (i) a textual description of the recomposition task together with a single question recomposition example, (ii) a new image I from any source, (iii) a list of factors subsampled from F. Each factor is specified by its category name and a sampled subset of subquestions from the seed dataset Q".
We prompt a MLLM with this input to generate new subquestions of similar kinds but grounded on the new image. The MLLM then composes these subquestions into coherent overall question. Alongside question generation, the MLLM is also responsible for producing answers: answers to subquestions are generated first, which are then combined to form the answer to the recomposed overall question. Therefore, the generated data pairs consist of both the overall question-answer pair (q, a) and its associated factor-level subquestions and answers. Formally, each data point is as a tuple (L, q, a, (fi). {a+3) where q + {f1, f2, ...., fk and a, Answer(fi 1).
An additional advantage arises in artificial domains such as charts, where images are often accom- panied by underlying metadata (e.g., tables of data associated with the figures). In such cases, leverage this auxiliary metadata during question generation to improve answer precision. This idea is consistent with prior work in synthetic data generation for structured domains (Masry et al., 2022).
Overall, this recomposition procedure enables us to expand the training distribution compositionally. generating diverse questions grounded solely from dataset of unlabeled images without requiring additional question-answer annotations.
3.4 REINFORCEMENT LEARNING- BASED FINE-TUNING
The final stage of COGS is reinforcement learning fine-tuning, where we adopt Group Relative Pol- icy Optimization (GRPO; Shao et al., 2024) to fine-tune a pretrained MLLM with the generated question-answer data. A key advantage of our recompositional design is that each complex ques-
4

---

Published as a conference paper at ICLR 2026
tion is automatically paired with its corresponding subquestions and sub-answers during the data generation phase. This structure enables richer reward modeling beyond answer correctness.
In RL-based fine-tuning for MLLMs, the most common choice of reward model is to assign rewards based on exact or approximate answer matching (c.g.. Fl string score). However, since COGS generates both overall questions and their factor-level subquestions, we define additional process rewards that assess whether intermediate reasoning steps are carried out correctly. Concretely, given data point (1, 9, a, {fe], {a:3), for each factor fi with subquestion 8; and ground-truth answer ag we prompt an LLM-based reward model to verify whether the model's thain-of-thoughee reasoning produced the correct sub-answer. This yields a binary score o₁ {0. 1} for each factor.
Formally, let y ( E {0, } denote the correctness of the final answer for output y, -N the number of subquestions, and X a weighting hyperparameter. We define the subquestion hit rate as pou (1) / Σ 6₁ In this work, we consider three reward models:
StandardRM: r(y) = (y), (y), which only evaluates final-answer correctness. This is the de- fault option when subquestion supervision not available.
ProcessRM-sun r(y) ( (y) + X x., (y), which combines correctness of the final answer
with the average subquestion accuracy, encouraging faithful reasoning at the factor level.
ProcessRM-mas r(y) max fial (y): A puu (y)) which prioritizes the final answer but still provides reward shaping when the intermediate reasoning is largely correct.
The summation- bassed process reward is common choice. However, because question may admit multiple valid decompositions and the resulting factor-level signals are noisy, the summed reward can misrank policies. By contrast, Process preserves policy orders. In contrast, our analysis shows that the max-based reward is order-preseevvi with respect to final-answer accuracy.
Proposition 3.1 Assume final E {0, 1J", À E (0,1) and yuu E [0, 1J. yuu is noisy shaping reward: paub a final + and with a [0, 1]. Note that Ez[e] may vary with x. Define Vy(T))
Egg (inal))
ProcessRM-max preserves policy orders. For any policies Ti; T2.
( Vy((T2) = sign ma [m]] - Eay. rTa);
ProcessRM-summ does not necessarily preserve policy orders. That is, there exist policies TT, n2 with Vy(m1) (22), E[r | E(r n2
Proof sketch. Using final E {0. 1}. max = y finul + Sy-ub (1 fnall With Ez final = 0 =c and Pr, finnll = 0) = 1 (T). we get m ाr] ()(T)) (-)) = which is an affine, strictly increasing transform of Vy if Ac < 1.
(1- (1-Xc)V
To see that ProcessRM-sum does not preserve orders, note that
E m1]
Err [2] (1 + Xa (Vz(m1) Vi +λ
(E Exgl=l)
can be <- <1(1+---AV))))
This theoretical insight is further empirically verified in our experiments.
EXPERIMENT
We evaluate COGS across multiple artificial image domains to assess its effectiveness in equipping pretrained MLLMs with new reasoning capabilities. We begin with, in Section 4.1, the chart reason- ing domain. Using& small subset of questions from the ChartQAPro dataset (Masry et al., 2025a), held-out questions. Extending to mix-
show that COGS substantially improves performance ture of datasets, ChantQAro + MMC (Liu et al., 2024a), we observe consistent improvements on both datasets, indicating that our framework enables transferable reasoning skills rather than overfit- ting to a single dataset. Next, in Section 4.2, we evaluate COGS on the Visual DesualWeBBench dataset (Liu et al., 2024b), demonstrating that the same approach generalizes beyond the chart domain.
"When final takes values from 00, 1). ProcessRM-nnt preserves policy orders iff. À suh
5

---

Published as a conference paper at ICLR 2026
Model
Factoid
MCQ
Convers.
FactChk.
Hypoth.
Overall
Proprietary Models
GPT-S-nano
45.95
63.64
49.40
63.58
49.82
50.74
GPT-4o-mini
43.63
66.43
45.48
59.88
45.20
48.32
Gemini 2.5 Flash-Lite
40.42
19.96
48.77
37.43
16.66
38.72
Claude Haiku 3.5
43.44
65.03
39.84
61.79
38.77
46.74
Opensource Models (7B+)
Qwen2.5-VL-72 (basc)
42.07
62,59
44,88
60.78
50.72
47.36
LL3.5--G--SS
43.02
58.74
42.86
58.02
54.48
46.86
Phi-4-14B
23.18
34.27
40.93
46.91
36.31
31,61
Chart Specialist Models
Laa
8.11
23.08
18.37
45.06
29.55
17.19
ChartMoE
19.03
35.66
32.97
45.68
27.08
27.28
Prompting Strategies: over
Qwen2.5-VI-77
Self-Consistency
43.44
61.54
44,00
59.82
41.76
47.22
Tree of Thoughts
40.01
$7.94
41.55
54.13
53.35
44.44
Decompositional CoT
42.08
65.03
42.57
56.53
45.55
46.36
Data Synthesis Approuches:
Qwen2.5-VL-7B
ChartQA-Traie
38.77
60.14
49.72
61.11
53.12
46.64
Chart-RI R1
42.17
46.85
50.53
61.11
55.55
47.32
In-Context Q Example
46.33
62.94
46.91
61.11
61.72
50.58
COGS (Ours)
46.88
65.73
51.16
61.85
58.25
52.02
Table
Accuracy (%) on ChartQAPro grouped by question type. COGS performs the best.
Finally, in Section 4.3, conduct a series of ablation studies better understand the sources of these improvements. Specifically, we examine: (i) which categories of questions benefit the most, and (ii) the comparative effectiveness of different reward models.
4.1 CHART UNDERSTANDING
4.1.1
GENERALIZATION FROM SEED TO TARGET DATASET
Chart Question Answering (CQA) requires interpreting visual representations in charts and rea- soning over their spatial relation and underlying data. The recently released ChartQAPro bench- mark (Masry et al., 2025a) consists of 1,948 human-curated question-answe pairs targeting com- diverse chart types. Unlike earlier chart QA datasets that often rely syn- thetic or templated questions, ChartQAPro emphasizes natural, high-quality queries that demand multi-step reasoning and interpretation. This makes a rigorous testbed for evaluating the visual reasoning ability of multimodal language models.
plex reasoning
not publicly available, we randomly select 33% of the released test set as validation data and treat them as seed questions for data syn- thesis. The remaining 67% held as fully unseen test set for all experiments. We com- pare COGS against state-of-the--rt pretrained multimodal large language models (MLLMs), chart- specialist models, and recent data synthesis approaches. For all data synthesis methods, including COGS, we use the training set of ChartQA (Masry al., 2022) the image source, in order to avoid any contamination from the evaluation data.
Setup. Since ChartQAPro's training and validation
Baselines. We consider the following models when evaluating COGS:
Proprietary Models: We include representative proprictary models as reference baselines, focus- ing on small but competitive variants: GPT-5 nano, GPT-4o mini, Gemini 2.5 Flash, and Claude Haiku 3.5.
We compare against recent open-source general-purpose MILMMs of com- parable sizes, including Qwen2.5-VL-78 (Bai et al., 2025), Intern IntemVLL33.-GPT-oo. (Wang et al., 2025a), and thi-4-14B (Abdin al., 2024).
General MILMSs:
Chart Specialist Models We consider models specifically designed for chart understanding, in- cluding LaaA (Han al., 2023), which improves chart QA performance after training

---

Published as a conference paper at ICLR 2026
high-quality synthetic instruction data, and ChartMoB (Xu et al., 2025) which integrates Mixture-of-Experte (MoE) architecture to facilitate chart understanding.
Prompting Strategies We compare against 3 prompting strategies at inference time, including ceff-oonsitenccy (Wang et al., 2023b) and Tree of Thoughts prompting (Yao et al., 2023). We also introduce inference-time decomposition as an additional variants of our method. In this setting, the LLM is prompted at inference time to decompose a complex question into simpler percep- tion and reasoning subquestions, using the same decomposition instructions as in our factor pool construction.
Data Synthesis Approaches. We compare COGS against other data synthesis methods in- cluding (1) the original QA pairs in ChartQA training set which is generated by machine, (2) ChartR1 (Chen et al., 2025) which programmatically synthesizes chart reasoning data and con- duct reinforcement finetuning: and (3) In- In-Context Question Examples where we follow question synthesis convention to generate questions with in-context question examples. Example questions are sampled from the seed dataset. Notably, all baselines are fine-tuned with GRPO using the same base model, Qwen2.5-VL... for same training effort, to enable fair comparisons. We used StandardRM due to the absence of subquestions and corresponding answer in these datasets.
(Bai et al., 2025), and image source (the training set of ChartQA)
Result. Table 1 shows the performance on different question types on ChartQAPro. Among open- source MILMs. Qwen2.5-VL-7B achieves the strongest overall accuracy (47.36%). Proprietary models such as GPT models and Haiku 3.5 perform reasonably well, but remain slightly below the our fine-tuned Qwen2.5-VV -V-- using the COGS framework. Chart-specialist models, while tailored to chart understanding, perform poorly compared to COGS. This is largely because they are typically constrained by specially designed architecture and trained on relatively narrow datasets, which do not fully cover the distribution of ChartQAPro and therefore suffer from domain gaps.
All data synthesis approaches demonstrate minor benefits over the base model likely due to domain gaps well. COGS achieves the highest overall accuracy of 52.02%, outperforming both base- lines and all open-source MILMs by a significant margin. We provide in-depth analysis of the performance gains in Section 4.3.
We observe a substantial performance gap between inference-time decomposition and COGS, largely due error accumulation across sub-questions. Instead, COGS mitigates this issue by rewarding correct intermediate substitutions in training, reducing error compounding. Moreover. RL training in COGS enables the model flexibly integrate decomposition signals without being constrained to a single reasoning path, unlike inference-ting -imen decomposition where the provided examples the context may restrict the model's ability to explore novel reasoning paths.
4.1.2 GENERALIZATION OVER MIXTURE OF DATASETS
Setup. We extend COGS to multi-dataset setting by incorporating the MultiModal Chart Bench- mark (MMC-Bench) (Liu et al., 2024a), a recent CQA dataset with reasoning-intensive,, annotated QA pairs. Similar to our ChartQAPro (noted as seed A) setup, split the MMC-Bench test set into 33% validation questions (used as seed B) and 67% held-out test questions.
intensive, human-
Variants. We compare two strategies for synthesizing data across domains: 1. Data-level mix- ture: decompose and recompose A and B independently, then combine the synthesized data, i.e., Recompose(Decomposs (A)) Recompose (Decompose (BB)) 2. Factor-level mixture: decompose A and B separately, merge all extracted factors into a joint pool, and recompose using this combined RecomppseeDecomppss
pool, i.e., Recompose( (Deoosseeeeemseeeeeeeeeeee (A) U Decompose( (B). In addition, we include two "specialist models" trained only with augmented data from a single domain (e.g., trained on augmented A and evaluated on A). These serve as "upper- bound references" for in-domain data augmentation. All methods use Qwen2.5-VI.-7B as the base model and are trained with GRPO and ProcessR
COGS shows transferrable benefits across datasets. As shown in Table 2, both data-level and factor-level mix- tures substantially improve performance in both domains,
Model
ChartQAPro MMC
Qwen2.5VL
47.36
85.65
demonstrating that COGS facilitates positive transfer across datasets rather than simply overfitting to one. Cru- cially, the factor-level mixture consistently outperforms the data-level mixture, suggesting that factor recomposi- tion better captures shared structures between domains.
+ -CharQQAPoo
52.02
85.69
+ MMC
49.93
88.10
Deta-Tevei Mix
"50.72
86.99"
+ Factor-level Mix
52.33
87.55
Table 2: Multi-data co-training results.
7

---

Published as a conference paper at ICLR 2026
+4.47
+4
Geen
coant
compans
cakcuaaion entrapoitiin ched
Figure 3: Accuracy (%) on ChartQAPro by reasoning Figure 4: Accuracy (%) ChartQAPro factor numbers and question types. COGS generally by reasoning factor types with complexity yields the larger gains on questions with more factors. from low high.
Factor-level mixture is a better strategy for data mixing. We observe that factor-level mixture consistently outperforms data-level mixture, and achieves performance both domains compa- rable to specialist models trained exclusively in-domain. This suggests that factorization offers more effective way leverage multiple datasets. Prior research on data mixing and multi-dataset training has shown that simply combining heterogeneous datasets often fails to yield the best gen- eralization, as models may overfit dominant distributions or under-utilize complementary signals. By breaking down questions into primitive factors before recomposition, COGS provides a com- mon representational ground across datasets, enabling more transferable training. This suggests promising direction for the long-standing challenge of data mixture in foundation model training.
4.2 WEBPAGE GUI UNDERSTANDING
To demonstrate the generality of COGS, also evaluate it the webpage question answering domain, which requires visual, semantic, and structural reasoning over graphical user interfaces (GUIs). We adopt VisualWebBende (Liu al., 2024b), benchmark consisting of diverse real- world webpages paired with ixstoning-intessivee human-curated questions. We use questions from VisualWebBench as seeds and screenshots from MultiUI (Liu et al., 2025) as the image source.
Setup. We evaluate the same set of proprietary and general-purposee MLLMs as in the chart understanding experiments. In addition, we compare against the GUI specialist UIX -wwe22 and data synthesis approach in Liu et al. (2025). We sampled 10k webpage QA data from MultiUI (Liu et al., 2025), and fine-tuned Qwen2.5-VL- 7B (Bai al., 2025) with GRPO (Shao et al., 2024). The results are reported as MultiUI-QA in Table 3.
Model
WebQA
Proprietary Models GPT-5-nano GPT-4o-mini
89.47
81.34
Gemini 2.5 Flash-Liite
81.85
Claude Haiku 3.5
80.86
Opensource Models (~7B)
Result. Table 3 shows the result. Owen2.5-VL-7B achieves 85.65% accuracy, outperforming most open- source baselines, while specialist models such as UiX- Qwen2 lag behind. Inference-time decomposition yields minor gain (86.124 12%) Among these, COGS achieves the best non propprrpyyy result at 88.04%. These findings confirm that COGS generalizes beyond charts, effectively boosting reasoning capability over complex webpages.
Qwen2.5-VL-7B (base model)
85.65
Intera MMeenVL335-GPT-OS.
74.64
Phi-4-14B
74.16
Specialist Models
UiX-Qwen2
68.90
Inference-time decomposition
Decompositional CoT
86.12
Data Synthesis Approaches
MultUI-WQA
86.60
4.3 ADDITIONAL ANALYSIS
COGS (Ours)
88.04
In this section, we attribute the reasoning capability gains of COGS to two factors: (1) enhanced performance on reasoning-inennsiee questions, including multi-hop rea- soning (Fig. 3) and complex reasoning factors (Fig. 4), and (2) the impact of different reward models.
Table
Accuracy (%) on Visual- WebBench. COGS performs the best among all non-proprietary models.
COGS improves multi-hop reasoning. Fig. shows model performance grouped by the number of factors. Overall, the performance improvement becomes more pronounced questions have longer reasoning chains. This trend holds across factoid, multiple-choice, and fact-checking questions. For hypothetical questions, however, the trend is less salient: conjecture that their difficulty is already
8

---

Published as a conference paper at ICLR 2026
Factold
Qwee 5VL.
COGS
stage requiies less than
perseption: Project Plan: 144 50)
perception: 1. Froject Project
00
Project int: 40 (2)
Closurer
Ansewar: 3
80)4, Eowc: 164 60) S. Cloune 58
Decomposition
60)
comparison: 164 cit, 40 60), 36 60), 154 60y, d0)
iperosption: project phose stages and their
count the phones that requ e no
comesperding ding hours)
hours: Project Init: 40 hours M&C.
count 3
toomparison: each stage - 50 hours)
56, phosee triel
The srower is
(count numbe of stages when fesure 90))
The attoome 8
Hypothetical
Queetion
Qwen2 5VLL
CoOS
It average age of the athictes was 1 year lower. how many Jupi new wiitei would have
perception: The curment avorage age
perception: The team
the Japariose atlinas is 27 .808 yoars.
calculation. New
aga 17.08-
been yourger than the avarage of all the oll leles?
calculation. the average age were
28 Ds.
Answer: 3
year lower the new average would be
comparteon Amongg A Scchaler 23 DD
Decomposition:
years.
Tanaka: 29 o. Adhledos younger than 28.08:
(peroeption: ourrent overage age
count Wazanabe (22). G Edwards (23)
A Schater (23) H Watonate (22)
average age -1g
countt Tharw new affleee your get then o8.
(camparison Japarass
age
Togasti
average
athietes yourger
answer
(couné, number of Japeress altietas yourger
Ihnn 26,
than the new average ago)
The anawer 8
Multi- Choicc
Question
Qwen2 VVLL
COGS
the frend from 2010 2020 continues, what
perception Decressed by
perception From 2010 2020,
would the projected greathouse gas
approsdmataly 150 million tonnes iTom
omisssions decreased from agoroxmatoly E0C
ssiion
-d00 to -480)
million tonnes to about 400 million sonnes.
estrapolation: Ouer years,
The ustal decswaoe 200 over 10 years
decrease about por year.
extrapolation: From 2020 to 2030 another
Decompositon:
continues, can project for
10 years, the emtesions wou d decrease by
UK ghweehouow seriiations
2000
another
the projected
Iaad from 2010 2020;
level, 450 16 435
amissioons
be
200
testrapolation: continure rntion of the frend to
compartson: The diseest option 435
comparison a)
exiirate 2050 emissions)
is 370
The
toomparison: match with the given optins)
The answer i
Factold
Question:
Qwen2.5VL
COOS
Bwowwe 2015 and 2017 what the scoenege
parcaption: inmignent popurittio for
parcaption: immigrent populition for each
innual growth of the inmignnt popurttion Tho houddddd?
such -01 41.5 2016: 45
your - 2013: 41 S................ 2018 43.7 ----20 2017: 46.5
2017: 44.5:44.0
calculation: dfferonce between each consen consecuttre year. 2014 2013 42.4 -41.3
Answer: BDC
calculations Total growth - 44,5
Decomposiãon:
(2017)- 4113 (2013) a 8.2, Average
2016 2014.... 2017 2016...
(percaption: immigrant popuration valuee
annual gmotth 3 millinn years
and diiide 3.2
million
ywors 2015 in 2017)
0.64 milionn thousands
icaloulation: average annual growth these yes) The answer 640
answeris
Figure 5: Qualitative evaluation examples. COGS-RL improved base models questions that contain multiple factors and from different question types in CharQAArr.
dominated by the first counterfactual reasoning factor ("ifxxx happens,...") Therefore, adding more factors does not compound the hardness in the same way.
This observation is further illustrated by the substantial gains on the Count (+4.25%) and Compare (+4.47%) factors in Figure 4. These two factors frequently co-occur as essential steps in multi-hop reasoning, such as Counting values based on results of Comparison operations. As illustrated in Figure (Rows a and b), models trained with COGS better capture such compositional structures, whereas bascline models tend to shortcut the process and directly produces the answer (which can be flawed, c.g., being number-insensitive and incorrectly concluding 56 > 60, as shown in Row a).
COGS supports advanced reasoning factors. At the factor level, we also observe strong gains on advanced reasoning factors such as Extrapolation (+7.62%) and Calculation (+3.04%) in Figure 4. These factors require models not only to execute operations but also to decide which operations are appropriate (e.g., whether to add, divide, or apply another function). The complexity is illustrated in Figure 5 (Rows and d). By training models positional data generation framework, we can improve factor-level reasoning performances. For example, in Row d. COGS correctly identifies that the average annual growth should be computed over 4 intervals, rather than mistakenly dividing the difference between the first and last year by 5.
diverse reasoning trajectories with com-
Ablation: reward model. We conduct an ablation study on the reward models under three settings pro- posed in Section 3.4, with GRPO and COGS data seeded from CharQQAro. As shown in Table 4: ProcessRM M- sum slightly worsens performance, while ProcessRM- max consistently improves it compared to StandardRM. This is consistent with our thcoretical analysis in Propo- sition 3.1, which shows that ProcessRM- preserves policy order under noisy sub-reward signals, whereas ProcessRM-sum does not.
Reward Model
Overall Acc.
StandardRM Process&M-suu ProcessRM-max
50.96
50.35
52.02
Alt. Training Setting
SFT+ProcessRM-mas
46.62
Table 4: Ablation study on reward mod- els shows that ProcessRM-mal maxi- mally boosts the model performance.
We further ablate the training strategy by an additional su- pervised fine-tuning (SFT) phase with 35k COGS examples prior GRPO. To mitigate overfitting.

---

Published as a conference paper at ICLR 2026
the datasets for SFT and GRPO are non-overlapping. Results show that while SFT helps regulate
output format, does not enhance reasoning ability, consistent with findings from Chu et al. (2025).
Ablation: size of seed questions. We also ablate the seed set size used for COGS data synthesis on ChartQAPro. We held out 67% of ChartQAPro a fixed evaluation set for fair comparison and sam- pled I 1%, 5%, 15%, 25%, and 33% of the original data size from the remaining 33%. We then trained Qwen2.5-VL-7B on data generated from each seed set. As shown in Fig. 6, performance increases with seed size. Questions generated from very small seed sets are less representative, resulting in relatively poor performance, whereas a reasonable subset such as 33% already yields a substantial boost.
Accuracy (%)
15
33
Size seed questions ChartQAPro)
Figure 6: Ablation study the size of seed questions for COGS on ChartQAPro. Perfor- mance improves as the size grows.
5 CONCLUSION
We have introduced COGS, a data-efficient frame-
work for equipping pretrained multi-modal large language models with new reasoning capabilities in domains where annotated question- question-answer answer data is scarce. The key idea is decompose seed questions into primitive factors, and then systematically recompose these factors with new images to generate diverse, compositional training data.
Future Work. This work opens several future directions. First, our experiments focus on single charts and single webpage screenshots; extending COGS to long-context reasoning over visually rich documents will broaden its scope. Second, it is important to study how our data synthesis can be integrated into the pretraining stage MLLMs or combined with search algorithms to further boost process reward guidance (Zhang et al., 2024a; Park et al., 2025). Third, future work may con- sider investigating how the reasoning capabilities acquired through COGS transfer to downstream tasks such as chart code editing web agent applications.
ETHICS STATEMENT
The primary contribution of this work is an efficient data augmentation pipeline that factorizes a small set of seed data into diverse reasoning question-ansen answer pairs. All data used in our work that are already publicly released and open-sourced under their respective licenses, which we carefully followed. Our method does not generate new images introduce additional modalities. We make sure that the synthesized question- answer pairs focus only on reasoning over charts and web pages, avoiding offensive. biased, or sensitive content. As such, the ethical considerations remain consistent with those already established for the underlying datasets and models. We hope we further promote reproducibility and transparency through the release of code and augmented data.
REPRODUCIBILITY STATEMENT
To ensure transparency, we rely solely on publicly available resources: all open-source LLM weights are downloaded from their official repositories, and proprietary models are accessed via their doe- umented code and APIs. We describe all experimental configurations, including prompt templates, hyperparameter choices, and softwareehardware environments, in Section 3, Section 4, and the Ap- pendix D. Our experiments do not involve any private or sensitive data. We release the code and data at https:/coogyyntheissggihht github.io.....
ACKNOWLEDGMENTS
We thank Lei Xu, Bowen Pan. Pingchuan Ma and members of MIT-IBM Watson AI Lab for helpful discussions and valuable feedback.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck. Ronen Eldan, Suriya Gunasckar, Michael Harrison, Russell J Hewett, Mojan Javaheripi. Piero Kauffmann, al. Phi-4 techni- cal report. arXiv preprint 2xXi::24/2.0890. 2024.
Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jes- sica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. Pixtral 12b. arXiv preprint arXiv:2410.07073. 2024. 1,2
Anthropic. The claude 3 model family: Opus, sonnet, haiku. https//www-cchaatttopi.coomm /8tp:///a000000000000/0//2/////0 18857677//Moeel www.anthropiccco//ewww//eoo faait////ww//lldddd-alll.l. f-ammll.. Model card. 1,2
Card.Clauud..3pdd. 2024. URL https://
Rabiul Awal, Mahsa Massoud, Aarash Feizi, Zichao Li, Suyuchen Wang, Christopher Pal, Aish- warya Agrawal, David Vazguez, Siva Reddy, Juan A Rodriguez, al. Webmmu: A bench- mark for multimodal multilingual website understanding and code generation. arXiv preprint arXiv:2508. /6763, 2025.
Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Vic- tor Cärbune, Jason Lin, Jindong Chen, and Abhanshu Sharma. Screenai: a usion-language model for ui and infographics understanding. In Proceedings of the Thirty-Third International Joint Con- ference on Artificial Intelligence, pp. 3058-3068, 2024. 3
Youngmin Back, Bado Lee, Dongyoon Han, Sangdoo Yun, and Hwalsuk Lee. Character region awareness for text detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9365-9374 2019.
Shuai Bai, Keqin Chen. Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaobai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025.
URL htpps//arriv.orr/abs///000 139233. 1, 2, 6, 7,8
Victor Carbune, Hassan Mansoor, Fangyu Liu, et al. Chart-basee reasoning: Transferring capabili- ties from Ilms vlms. In proceddings of NAACI, 2024.
Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. Webqa: Multihop and multimodal In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16495-16504, 2022.
Jinyue Chen, Lingyu Kong, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Onechart: Purify the chart structural extraction via one auxil- iary token. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 147- 155, 2024. 3
Jun Chen, Deyao Zhu, Xiaoqian Shen, al. Minigpt-v2: large language model as a unified interface for vasion-language multi-task learning. arXiv preprint:2310.094788 2023,
Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Yufeng Zhong, and Lin Ma. Chart-rl: Chain- of-thought supervision and reinforcement for advanced chart reasoner, 2025. URL https://arxiv. org/abs/2507.15509 3,
Xingyu Chen, Zihan Zhao, Lu Chen, JiaBao Ji, Danyang Zhang, Ao Luo, Yuxuan Xiong, and Kai Yu. WebSRC: A dataset for web-based structural reading comprehension. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 4173 4173-4185, Online and Punta Cana, Dominican Republic, 2021. Association for Computational Linguistics. doi: 10. N0.18653/v1/2000 http///22.2.emmmmmii.... wawmmmmmai.n.44
URL https::/cclnthholg..oo////
wwn.p-main.neee ma.n.343. 3
Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 9313-9332, 2024.
11

---

Published as a conference paper at ICLR 2026
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training In Forty-second International Conference on Machine Learning, 2025. 10
Gheorghe Comanici, Eric Bieber, and et al. Gemini 2.5:
Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. URL 206.66661.... 1,
https//arxix.org/ab/2557...
Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the digital world as humans do: Universal visual grounding for GUI agents. In The Thirteenth International Conference on Learning Representations, 2025. URL https:// 3
openreview openreview.net/formmmf met/fforummidd-kxooqqiii
Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. Chartllama: A multimodal I1m for chart understanding and generation, 2023, URL https:
Srrriv.org/abs231111655
3.
Wei He, Zhiheng Xi, Wanxu Zhao, Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, and Xuanjing Huang. Distill visual chart reasoning ability from Ilms to milms. CoRR, 2024. 2
Zexue He, Marco Tulio Ribeiro, and Fereshte Khani. Targeted data generation: Finding and fixing model weaknesses. In The s1 Annual Meeting Of The Association For Computational Linguis-
fics, 2023. 3
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang. Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 14281-142991 2024. 3
Yu- Yu-Chung Hsiao, Fedir Zubach, Gilles Baechler, Srinivas Sunkara, Victor Cärbune, Jason Lin, Maria Wang, Yun Zhu, and Jindong Chen. Screenqa: Large-scale question-anse answer pairs over mobile app screenshots. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume Long Papers), pp. 9427-9452, 2025. 3
Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp- 3096-3120, 2024. 3
Muye Huang, Han Lai, Xinyu Zhang, Wenjun Wu, Jie Ma, Lingling Zhang, and Jun Liu. Evochart: A benchmark and self-training approach towards real-world chart understanding. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp.
2025. 3
Caijun Jia, Nan Xu, Jingxuan Wei. Qingli Wang, Lei Wang, Bihui Yu, and Junnan Zhu. Chartrea- soner: Code-driven modality bridging for long-chain reasoning in chart question answering, 2025.
URL https//arxivvorr/aas//55000
/ttp//aaa///.....///// 10116.
Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan, Dvqa: Understanding data visual- izations via question answering. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5648-5656, 2018. 2
Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Akos Kadar, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning, 2018. URL https:
//ardv.org/ab///100007300 2
Kenton Lee, Mandar Joshi, Iulia Raluca Ture, et al. Pix2struct: Screenshot parsing as pretraining for visual language understanding. proceedings of ICML., pp. 18893-18912, 2023. 3
Bo Li, Peiyuan Zhang, Kaichen Zhang, Fanyi Pu, Xinrun Du Yuhao Dong, Haotian Liu, Yuanhan Zhang, Ge Zhang, Chunyuan Li. and Ziwei Liu. Lmms-eval: Accelerating the development of large multimoal models, March 2024a. URL https//gtthbb.com//vooinnLLLLLL Lab/lmms-ewe 18
12

---

Published as a conference paper at ICLR 2026
Kaixin Li, Ziyang Meng. Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro Gui grounding for professional high-resolution computer use, 2025. URL https//aarii.oor//bb2555.......0
3
Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, and Zhiwei Guan. Widget captioning: Gener- ating natural language description for mobile user interface elements. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5495-5510, Online, 2020. Association for Computational Linguistics. doi: 10.1865331122220.mmmg main.
443. URL httpss//cclambbolog.orr// w0200020 wtwpa//222.....2--1.e 1alip--mai..44...
Zhuowan Li, Bhavan Jasani, Peng Tang, and Shabnam Ghadar. Synthesize step-by-step: Tools tem- plates and ulms as data generators for reasoning- basdd chart vqa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 13613-13623, June 2024b.
3
Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, et al. Deplot: One- shot visual language reasoning by plot-to-table translation. In Findings of ACL, pp. 10381-10399, 2023a. 3
Fangyu Liu, Francesco Piccinno, Syrine Krichene, al. Matcha: Enhancing visual language pre- training with math reasoning and chart derendering. In proceedings of ACL, pp. 12756-12770, 2023b.
Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, and Dong Yu. Mme: Advancing multimodal chart understanding with large-scale in- struction tuning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 1: Long Papers), pp. 1287-1310, 2024a. 5,
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34999 2023c. 1,2
Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, and Xiang Yue. Visualwebbench: How far have multimodal lims evolved in web page understanding and grounding?, 2024b. URL https://arxiv.ot wttp://ww...r///////////.. 405555.5....... 3, 5,8
Junpeng Liu, Tianyue Ou, Yifan Song, Yuxiao Qu, Wai Lam, Chenyan Xiong, Wenhu Chen, Graham Neubig, and Xiang Yue. Harnessing webpage UIs for text-rich visual understanding. In The Thirteenth International Conference on Learning Representations, 2025. URL URL. https//////pperr net/forum TT=1IsT44333. 3, 3
Mengsha I Liu, Daoyuan Chen, Yaliang Li, Guian Fang, and Ying Shen. Chartthinker: A contex- tual thain-of-thoughtt approach optimized chart summarization. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evalua- tion (LREC-COLING 2024), pp. 3057-3074, 2024c.
Ahmed Masry, Xuan Long Do. Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: bench- mark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pp. 2263-2279, 2022. 2,6
Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mchrad Shahmo- hammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, and Shafig Joty. Chartqapro: A more diverse and challenging benchmark for chart question answering, 2025a. URL. https: //ariiv.rrgaab//5500055555 00556.0666....6..... 3, 5,
Ahmed Masry, Megh Thakkar, Aayush Bajaj, Aaryaman Kartha, Enamul Hoque, and Shafiq Joty. Chartgemma: Visual instruction- tuning for chart reasoning in the wild. In Proceedings of the 31st International Conference Computational Linguistics: Industry Track, pp. 625-643, 2025b. 3
Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. Chartassisstant: A universal chart multimodal language model via chart-to table pre- training and multitask instruction tuning. CoRR, 2024. 3
13

---

Published as a conference paper at ICLR 2026
Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. Plotqa: Reasoning over scientific plots. In Proceedings of the ieee/cvf winter conference on applications of computer vision, pp. 1527-1536, 2020. 2
OpenAI, Josh Achiam, Steven Adler, and al. Gpt-4 technical report, 2024. URL https://axxiv.or// abs/2303.087744 1,2
Sungjin Park, Xiao Liu, Yeyun Gong, and Edward Choi. Ensembling large language models with process seward-guiden tree search for better complex reasoning. In Proceedings of the 2025 Con- ference of the Nations of the Americas Chapter of the Association for Compulational Linguisties: Human Language Technologies (Volume 1: Long Papers), pp. 10256-10277 2025. 10
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint 230v:2402.033000, 2024. 2, 8, 18
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rihf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pp. 1279-1297, 2025. 18
Chufan Shi, Cheng Yang, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, et al. Chartmimic: Evaluating Imm's cross-modal reasoning capability via chart-to-code generation. CoRR, 2024. 2
Liyan Tang, Grace Kim, Xinyu Zhao, Thom Lake, Wenxuan Ding, Fangcong Yin, Prasann Singhal, Manya Wadhwa, Zeyu Leo Liu, Zayne Sprague, Ramya Namuduri, Bodun Hu, Juan Diego Ro- driguez, Puyuan Peng, and Greg Durrett. Chartmuseum: Testing visual reasoning capabilities of large vision-language models, 2025. URL https://ariiv.rr/aas/5555.. 13444. 3
Maria Wang, Srinivas Sunkara, Gilles Baechler, Jason Lin, Yun Zhu, Fedir Zubach, Lei Shu, and Jindong Chen. Webquest: benchmark for multimodal qa on web page sequences. arXiv preprint 137111 2024a.
Peifang Wang, Olga Golovneva, Armen Aghajanyan, al. DOMINO: dual-system for multi-step
visual language reasoning. arXiv preprint:2310.02880 2023a. 3
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang. Zhe Chen, Hongjie Zhang, Ganlin Yang. Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding. Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang. Yuchen Duan, Xuehui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang. Songze Li, Xiangyu Zhao, Haodong Duan. Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui Hc, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang. Huipeng Deng. Biqing Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang. Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu, Huanze Tang, Rui Wang, Haijun Lv, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, and Gen Luo. Internv13.5: 5: Advancing open-source multimodal models in versatil- ity, reasoning, and efficiency, 2025a. URL https://arxivvorr//aaaa wttp:////.././/......oo/ 18265. 1,2,6
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023b. 7
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self self-generated instructions. In Proceedings of the 61st anmual meeting of the association for computational linguistics (volume 1: long papers). pp. 13484-13500 2023c. 2
Zinu Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladì, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal 1lms. Advances in Neural Information Processing Systems, 37:113569- -136977,
2024b.
14

---

Published as a conference paper at ICLR 2026
Ziwei Wang, Weizhi Chen, Leyang Yang, Sheng Zhou, Shengchu Zhao, Hanbei Zhan, Jiongchao Jin, Liangcheng Li, Zirui Shao, and Jiajun Bu. Mp-gui: Modality perception with milms for gui understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 2971 1-2221,,, 2025b. 3
Yifan Wu, Lutao Yan, Leixian Shen, Yinan Mei, Jiannan Wang, and Yuyu Luo. Chartcards: A chart-metadata generation framework for multi-task chart understanding. arXiv preprint arXiv:2505. /5466 3025.
Renqiu Xia, Zhang. Hancheng Ye, Xiangehao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Peng Ye, Min Dou. Botian Shi, Junchi Yan, and Yu Qiao. Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning, 2025. URL https//aaxix.or///a24400...1 12185. 3,3
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. Wizardlm: Empowering large pre-trained language models to follow com- plex instructions. In The Twelfth International Conference on Learning Representations, 2024a.
Zhengzhuo Xu, Sinan Du, Yiyan Qi, Chengjin Xu, Chun Yuan, and Jian Guo. Chartbench: A benchmark for complex visual reasoning in charts, 2024b. URL https///ariivargga22222..111
Zhengzhuo Xu, Bowen Qu, Yiyan Qi, SiNan Du, Chengjin Xu, Chun Yuan, and Jian Guo. Chart- moe: Mixture of diversely aligned expert connector for chart understanding. In The Thirteenth International Conference Learning Representations, 2025. 3.7
Pengyu Yan, Mahesh Bhosale, Jay Lal, Bikhyat Adhikari, and David Doermann. Chartreformer: Natural language-driven chart image editing. In International Conference on Document Analysis and Recognition, pp. 453-469. Springer, 2024. 3
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems, 36: 36:11809 180991811182 2023. 7
Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. Ureader: Universal ocr-free visually language understanding with multimodal large language model. Findings of the Association for Computational Lin- guistics: EMNLP 2023, pp. 2841-2858. 2023a.
Jiabo Ye, Anwen Hu, Haiyang Xu, et al. Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model. In Findings of ACL, 2023b. 3
Qinghao Ye, Haiyang Xu, Guohai Xu, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint:2304. 14178, 2023c. 3
Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug-owl2, Revolutionizing multi-modal large language model with modality collabo- ration. In Proceedings of the ieeelcvf conference on computer vision and pallern recognition, pp.
1300011001, 2024. 3
Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang. and Zhe Gan. Ferret-ui: Grounded mobile ui understanding with multimodal llms. In European Conference on Computer Vision, pp. 240-255. Springer, 2024. 3
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E Weston. Self-rewarding language models. In Forty-first International Conference on Machine Learning, 2024.
Weihao Zeng, Can Xu, Yingxiu Zhao, Jian MtanGGuang Lou, and Weizhu Chen. Automatic instruction evolving for large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 6998-7018, 2024. 2
15

---

Published as a conference paper at ICLR 2026
Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mets"; Llm self-training via process reward guided tree search. Advances in Neural Information Processing Systems, 37:64735 64772, 2024a. 10
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models, 2024b. URL https://arriv.org//b//477... 12772.
Liang Zhang. Anwen Hu, Haiyang Xu, Ming Yan, Yichen Xu, Qin Jin, Ji Zhang, and Fei Huang. Tinychart: Efficient chart understanding with visual token merging and program-of-thought learning. CoRR, 2024c. 3
Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, and Maosong Sun. Chart- coder: Advancing multimodal large language model for chart-to-code generation, 2025. URL hutps///arxiv.org/abb//501 .66598. 3
Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. Gpt-4v (ision) is a generalist web agent, if grounded. In International Conference on Machine Learning, pp. 61349 61385, PMLR, 2024. 3
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
We use ChatGPT as grammar checker for the writing of this paper. We also use small proprictary language models as evaluation baselines to compare performance in our experiments as described in Section We use open-sourced MLLM in our synthetic data generation pipeline follow corre- sponding license.
B QUALITATIVE EVALUATION ON VISUALWEBBENCH
We provide some visualization examples of evaluation on Visual WehBench. COGS has been largely improved in questions involving reasoning like comparison, as well as spatial relation.
Question:
Owen2 8VL
COGS
Which team has the moat Issses the NFC West division es per tha standings shown?
perception: Angeles Rams (LAC) The answer is LAC
perception: SPO: L-LAR: 7-
SEA: 6 ARt 13
Answer: ARI
comparison ARI has the highest number of icessss 13.
Decomposition:
The answer ARI
(perception the NFC West division standings)
(comparison: the team with the nighest number losses)
Question:
Qwen2.5VL
COGS-RL
ahortwo langth among the local fiverites
parcaption Elysion Loop
percaption: the lengtha of
rear Los Angeles?
tralls latec:
Answer: [1.2 ml, '1.9 mile]
answer is Elysian Park Short Loop
1. Elysian Park West Loop Trask 2.4
mi Elysinn Park Short Loop: 1 m
Decomposition:
3. Silver Lake Reservor Trail 2.2 mi
(parcaplion local favaribes near Los Angelea)
4. Portola Trail Loop: 2.8 mi
(comparison: compare the trail langths of the lssted tralls)
comparison: The shortast longth Is 1.9 miles for the Elyslan Park
(minimum: densly the shoriest trail length)
Short Lonp.
The answer
Question:
Qwan2.5Vi
COOS-RL
What type of deals is offered next 'Small Pet Deals'
The answer is Dog Deals
The answer is Bird Deals
50% Off
Answer: Bird Deals
Decomposition:
(identifcation: the section labolad 'Small Pet Deals)
(spatial relation: find the deal type noxt 'Small Pet Deals))
(identification: the devil offered neat 'Small Pwt Dewis)
Figure 7: Example of evaluation on VisualWebBenci
C ABLATION STUDY ON BASE MODELS
We evaluate whether the gains from COGS depend on model capacity within a family or on the choice of model family at a fixed parameter scale. Concretely, we fine-tune DW-33.5-VL-33B (same family as our main experiments but smaller size) and using the same COGS dataset and the same RL configuration hyperparameters, reward settings, and evaluation protocols are kept identical to isolate the effect of the base model. We then evaluate on the fixed ChartQAPro test split using the same metries reported in the main table.
(different family at 7B scale) the main experiments. All training
Across both comparisons, COGS consistently improves the corresponding base model. These results indicate that the benefit of COGS is not specific to particular parameter count to single model family, and that the decomposition guided synthesis remains effective under changes in backbone capacity and architecture. Full results reported in Table 5.
Model
Overall (Base)
Overall (+COGS)
Qwen2.5-VL-3B
36.22
38.68
I.LVV-v1..577
13.47
22.04
Table 5: Accuracy (%)
ChartQAPro comparing base models and +COGS.
17

---

Published as a conference paper at ICLR 2026
D MORE DETAILS ABOUT COGS IMPLEMENTATION AND REPRODUCIBILITY
Hyper-parameters We use verl(Sheng et al., 2025) for GRPO training. We ran with epoch = using a large-scale distributed setup with 8 GPUs per node across nodes. The model was trained with batch size 1024 and maximum input/output lengths = 4096 tokens for prompts and 2048 tokens for responses, respectively. Optimization used learning rate- 1e-6, with GRPO updates performed on bini-batches of 256. To stabilize training, we applied a K -peenalyy loss with coefficient 0.001, while disabling KL in the reward. Gradient checkpointing was enabled for memory efficiency, and tensor model parallelism was set to size 2. Rollouts used 16 samples per step, with GPU memory utilization capped 0.6.
Running Software/Hardware Environment and Training Time Our implementation is based on Python, with Transformers v4.51.3, PyTorch v2.6.0, and CUDA 12.4. We use VERL v0.5.0.dev0 fine-tune models with Reinforcement Learning via GRPO Shao et al. (2024). All experiments are distributed across 4 nodes, each equipped with NVIDIA H100 GPUs (80GB). Training data is generated using wwvn2.5-v1-72B.. Fine-tuning the base Qwen2.5-v1-7B model takes approximately 10 hours with GRPO and 2 hours with SFT on the Chart Understanding task with 10k reasoning examples, while the WebGUI QA task (also 10k examples, but web images are larger) requires about 21 hours of GRPO.
We sampled 67% from each benchmark dataset for evaluation. For evaluation on chart question answering benchmarks, We adopted official prompt templates for each question category under the chain of thought setup released in original paper. For webpage GUI question answering, enable the chain of thought by following prompt: You will be given image and question that you need to answer based on the provided image. You need to think step-by-step and format the final answer in separate sentence like "The answer is X". The final answer should be in the fewest words possible. We use Imms-eval for evaluation(Zhang et al., 2024b; Li et al., 2024a).
Evaluation
Prompt for Seed Question Decomposition and New Question Recomposition are included in the next section.
18

---

Published as a conference paper at ICLR 2026
E PROMPTS
This section specifies the prompt templates we use to decompose questions from seed dataset for the factor pool and re-composition of questions for COGS dataset.
E.1 PROMPTS FOR QUESTION DECOMPOSITION
Decomposition Prompt.
We can decompose each question into subquestions from one of the general types. Here are examples:
[in-context example: Chart/Web]
Please do the same for the following questions in the same format without explana- tion.
Check the information in the attached image carefully. If the question can be easily answered with a simple identification step, avoid unnecessary decomposition.
Remember to strictly follow the format of the example, and don't 't provide the answer. <image>
Question: {query}
In-context question-decompoittinn example: Chart for both ChartQAPro and MMC in this paper
(Question: How many times has the satisfied rate been above 25%?) = (identification: satis- fied rate of each year) (comparison: each instance 25%) (count: number of instances where satisfied rate > 25%
(Question: Is the following statement True False? Gen X has experienced steeper pop- ulation increase than baby boomers did between 1990 and 2015.) (identification: Gen X's population increase curve) + (identification: baby boomers' increase curve) + (comparison: which one has a steeper curve) (fact checking: given the finding from the previous step, is the statement true?)
(Question: if a multi-college district served 10,000 students, how many students termined eligible using EFC criteria?) = (identification: percentage students determined eligible using EFC criteria in multi-college district) (calculation: number of students based on that percentage)
de-
(Question: if the actual Avg ACA Premium in 2017 had turned +30% label accurately reflected the difference compared to the Low Est. projection for that hypothetical $5,000 value, what would be the implied Avg Individual Mrkt Premium Without ACA Low Est. in 2017?) = (identification: Avg ACA Premium in 2017) + (identification: +30% label that reflects the difference of Avg ACA Premium in 2017 compared Low Est.) + (calculation: implied Low Est. value based on the given 30% difference and hypothetical $5,000 ACA Premium)
be $5,000, and the
In-context question-decompositioo example: Webpage GUI for VisualWehBench in this paper
(Question: How many times has the satisfied rate been above 25%?) = (identification: satis- fied rate of each year) + (comparison: each instance 25%) (count: number of instances where satisfied rate > 25%)
(Question: the following statement yes or Gen X has experienced a steeper population increase than baby boomers did between 1990 and 2015.) = (identification: Gen X's popula- tion increase curve) (identification: baby boomers' increase curve) + (comparison: which one has steeper curve) (fact checking; given the finding from the previous step, decide yes or no)
(Question: According this chart, what is the revenue of Retailer D Month 6?) (identi- fication: revenue of Retailer D at Month 6)
19

---

Published as a conference paper at ICLR 2026
E.2 PROMPTS FOR QUESTION RE-COMPOSITION
Given the following chart:
chart: <image>
Your Task is to generate 5 sets of question-answe answer pairs for instruction tuning. In each set of QA pairs, you need to first identify {perception count] entities, and then compose treasoning.count } level of reasoning questions related to them. The 2-nd order rea- soning questions should be based on the answers of the 1-st order reasoning questions, and Each question must meet ALL these conditions:
1. Content Source: Only use data present in the given chart.
2. Structure: Each question must include exactly {count1} {factorl].
and { {countN] {factorN]. Each identification question should ask about one and only one entity/concept, the following (reasoning factors Jsbbquestions } should be the question related and only related to the entities/concept mentioned in the previous subquestions. The specific example of each subquestion type will be provided in the following text.
3. Content: Each question must be based the chart data, and can be answered using natural language. Avoid asking about the size of an object that not relevant to the data (e.g.. font size of a label).
4. Relevance: If there a reasoning subquestion, it must operate on the entities or values identified in the observation subquestion. [in-context example 1]
5. Conciseness: After writing the detailed question, provide a natural concise version. This concise version should still look like question, and can be asked independently without the previous question. [in-context example 2]
6. Answer: Provide step-by-step reasoning for how you found the answer.
7. Final Answer: Provide just the concise final answer to the concise question, without any explanation or reasoning.
Reference Examples: [factors} (sampled subquestion of the factor)
Here are some examples of the concise questions:
Expected Output Format for the generated questions: Use the following structure for each pair:
[in-context example 3]
Instructions:
1. Follow the example strictly. If the question contains reasoning subquestions, make it
is relevant to the observation questions.
2. Use only the given data in the chart.
3. Provide exactly 5 unique Q&A pairs. [question types]
4. Validate each answer. The answer must be grounded to the data shown in the chart.
5. Each pair must include both detailed step-by-ster reasoning and the final result.
Generate Now:
Please proceed with generating your question-anss ansoerr pairs now.
In-Context Example 1
For example, if the observation subquestion asked about the value of A and B, and you asked to generate a calculation subquestion after them, must be some calculation between A and B. If there tions should be based on the answers of the previous subquestions. Do not ask irrelevant questions. For example, if the first subquestion is "what's the difference between A and B", an acceptable next-level reasoning question would be "what's the difference between A and B compared to C". You should avoid an unacceptable question like "what's the difference between A and B and what's 's the difference between A and C".
multiple levels of reasoning questions, the later reasoning subques-
20

---

Published as a conference paper at ICLR 2026
In-Context Example 2
For example, if the detailed question asks about new value if A is changed, the concise question cannot simply refer to "new value" without mentioning it depends on being changed. An explicit example: detailed question: "What is A, what is B, what is the new value of A+B is changed to 10, what is the difference between the new value and C?" bad concise question is: "What is the difference between new value and C?", because it does not mention A is changed to 10, and it does not mention A+B is the new value. A correct concise question should be: "What is the difference between the new value of A+B and C, if A is changed to 10?"
In-Context Example 3
{
"Question"; "<Full question with two identifications and one comparisen>",
"Concise question"; "<Concise versien of the question>", "Answer": "<step-by-step reasoning and calculation>", "Final Answer": "<The final answer to the concise questien>"
},
2: {
"Question": "...",
"Coneise question"; "...",
"Answer": "...", "Final Answer": "..."
}
Example (not actual data):
1: {
"Question": "What was the percentage for Technology, what the percentage for Finance, and what them?",
difference between
"Concise question"; "What is the difference between Technelogy and Finance's percentages?",
"Answer": "Step Technology's percentage is 23.7%. Step Finance's percentage is 28. step 3: The di fference is 123.7% 26.381 2.68.", "Final Answer": "2. 68"
21

---

Published as a conference paper at ICLR 2026
F COGS DATA EXAMPLE
F.1 VISUALIZATION OF SELECTED COGS-CHARTQAPRO
We provide visualization examples of COGS-CharQQAPro Datasets.
Complex Question:
U.S. Construiing to World Sipatoe
Which
following question: if the pereentage of 'DK' respondents added 'Not essential', what would be pereentage of 'Not essential"? 42% b) 44% 46% 48%
Answer:
Ieceoial 50%
42%
30%
Sub-Questions:
QI: What is the pereentage of respondents who responded 'DK'? A1:4
Q2: What is the percentage of respondents who think it is not essential?
A2: 38
Q3: The percentage of 'DK' respondents is 4%, and the percentage of those who think it is not essential is 38%. What would be the new total percentage of 'Not essential' if the percentuge of 'D' respondents was added to 'Not essential"? A3: a) 42%
Complex Question:
the following statement True or False: score Republicuns in Dec 2008 is higher the Democrats Jan 2010?
Answer:
True
Sub-Questions:
Q1: What the perception score of Republicans in Dec 2008?
A1: 82
Q2: What was the perception score of Democrats in Jan 2010?
A2: 24
Q3: the score of Republicans in Dev 2008 higher than the score of Democrats in Jan 2010?
A3: True
Complex Question:
Which country has a higher adjusted net attendance rate 2006, Brazil or Djibouti?
Answer:
Brazil
Sub-Questions:
Q1: What the adjusted net uttendance rate for Bruzil in 2006?
A1: 97.91
Q1: What the adjusted net attendance rate for Djibouti in 2006?
A2: 79.87
Q3: Which country has higher adjusted net attendance rate, Brazil or Djibouti?
A3: Brazil
Figure 8: Example of complex questions with 3 subquestions.
22

---

Published as a conference paper at ICLR 2026
Complex Question:
Internalggius mingling differs oorosa eountries
Is the following statement True False? The result of subtracting swapped Rever/ percentage from swupped Ccasionally.FFequeem percentage is positive South Africa.
Answer: Falsc
Pesportt
Sub-Questions:
Q1: What is the percentage of people who interact 'Never/Rarely' in South Africa?
Me4
Al:
Q2: What is the percentage of people who interact Occasionally.Frqquentty in South Aftica?
Vnain ove Jomr Linegin coneeto
A2: 69
Q3: Swap two pereentages identified. The percentage people who interact "Never/arelly' South Africa becomes 69%, the percentage people who interact Oecasionall/Feeeeentty in South Africa becomes 31%. What is the result of subtracting swapped 'Never/arelly' percentage from the swapped Oocaasionally/ Frouuulyyyyyyy pereentage? A3-
Q4: Is the following statement True False? The result of subtracting the swapped 'Never/Rurely' percentage from the swupped Occasionaly//Freeeeeyy percentage positive for South Africa.
A4: False
Complex Question:
What is the sum of Sweden and Czechia's shares of population with alcohol drug disorders. if the share in Sweden decreases by 10%?
Answer:
4,703
Sub-Questions:
Q1: What is the share of the population with ulcohol or drug use disorders in Sweden?
A1: 2.57
Q2: What is the share of the population with alcohol or drug use disorders Czechia?
A2:
Q3: the share in Sweden decreases by 10%, what is the share Sweden?
A3: 2.313
Q4: What is the new of Sweden and Czechia's shares of population with alcohol drug disorders, if the share in Sweden decreases by 10%?
A4: 4.703
Complex Question:
Majority of Mobile Display Ad Dollars Go To Five Tech Companies
Does the adjusted value Google's surpass Facebook's increuse?
Answer:
False
Share 2013 display revenues
Sub-Questions:
Q1: What is the share of mobile display ad revenues for Facehook?
A1: 35.7
Other
Q2: What is the share of mobile display ad revenues for Google?
26 9%
A2: 15.4
6.0%
Q3: What is the difference between the shares of mobile display ad revenues for Facebook and Google?
7.3%
Google 13.4%
Apple
8.7%
A3: 20.3
Twitter
Pandora
Q4: Apply a 5% increase in Google's share and check the adjusted value surpasses Facebook's share
Source Markker
PEW RESEARCH CENTER
A4: udjusted value Google's share 16.17, which does not surpass Facebook's share 35.7
Figure 9: Example of complex questions with 4 subquestions.
23

---

Published as a conference paper at ICLR 2026
Complex Question:
Feeling the Recovery Blahs
Is the combined percentage of 'Nor recovering all' and 'Reooveiing strongly' more than double the 'DK/refused" percentage?
Which best deseribes your opinion: The economy is...
Answer:
True
Not
Sub-Question:
recoverigg
at
20%
Q1: What is the pereentage of people who think the cconomy is "Not recovering at all'?
DK
Recovering: ringe but not strongly 68%
A1: 26
refused-
Q2: What is the pereentage of people who think the cconomy is 'Recovering strongly"?
Recovering strongl
A2:6 6
6%
Q3: The combined percentage of those who think the economy either "Nor recovering at all'
Source: Rew Research Uen
or 'Reovveiig strongly"?
TODAY conducted April 23-27,
A3: 32
2014
Q4: What the percentage people who think economy 'DK/refused" doubled?
A4:4
Q5: combined percentage recovering and Recovering strongly than
double the "DK/refused" percentage?
A5: True
Complex Question:
Is the difference between Tertiary and Primary GPI greater smaller than the difference between Primary and Socondary GPT?
Answer:
Greater
Sub-Questions:
QI: What is the GPI for Tertiary education? Al: 1.01
Q2: What is the GPI for Primary education?
A2: 0.98
Q3: What is the GPI for Sccondary education? A3: 0.98
Q4: The GPI for Tertiary education and the GPI for Primary education is 0.98, What difference between these two values? A4: 0.03
Q5: GPI for Primary education and the GPI for Secondary education 0.98, Whut the difference between these values A5:
Q6: The difference between Tertiary Primary GPI is 0.03, and the difference between Primary and Secondary GPI is Is the difference between Tertiary and Primury GPI greater smaller than the difference between Primary and Secondary GPI? Greater
Figure 10: Example of complex questions with more than 4 subquestions.
24

---

Published as a conference paper at ICLR 2026
F.2 VISUALIZATION OF SELECTED COGS- BIBUALWEBBENCHH
We provide visualization examples of COGS-Visua Datasets.
1011-91--001
Complex Question:
What would be the total cost of Kryptek and Trucker hats if the quantity is increased by
Answer:
Eack Home
78
Subquestions:
Q1: What is the price of the Kryptek and Trucker hats?
A1: 26
Q2: Calculate the total cost the quantity increased by
A2: 78
Kryptek and Trucker
hats
$26.00 CD
Quarryy
Comserry inurryy ivere FLEVE Iol
Complex Question:
What is the full zip code of the city where James Kutten's office located?
Cerre oudo 501, chenriest
Answer:
63017
JAMES KUTTEN
Subquestions:
ATTORNEY
Tox Controversy Attorney Fhan Ye exjeriiero
Q1: Find the city next to the address.
A1: Chesterfield
Krowledge Experience.
Experise,
Q1: What is the full code of this city?
A2: 63017
Legal Disclaimers
Artile
Complex Question:
What is the difference in years between the publication dates of the first and third articles listed under Related Articles?
Services
Answer:
Subquestions:
Relssed Artiies
Q1: What is the publication dates of the first articles listed under Related Articles?
Pettema fure demende erhatiue arouee Brbite
A1: 2013
Dut haail Jevueru
Q2: What is the publication dates of the third articles listed under Related Articles?
A2: 2013
Q3: What is the difference A3: 0
Figure 11: Example of complex questions seeded from 33% WesBenhh.
25


Reviews:
[{"review_id": "8xcO7kkiE2", "reviewer": "Reviewer_hVuu", "summary": "This paper introduces COGS, a data-efficient framework for enhancing visual reasoning capabilities in multimodal large language models  for artificial image domains like charts and webpages. The key innovation is decomposing seed questions into primitive perception and reasoning \"factors,\" then systematically recombining these factors with new unlabeled images to generate synthetic training data. Each generated question includes subquestions and intermediate answers, enabling reinforcement learning with process-level rewards. The authors evaluate COGS primarily on chart reasoning (ChartQAPro, MMC-Bench) and webpage understanding (VisualWebBench), demonstrating substantial improvements over baseline models, with particularly strong gains on reasoning-heavy and compositional questions.", "questions": "Can factors learned from one domain (e.g., charts) transfer to another domain (e.g., webpages) without redecomposition? Would a shared factor pool improve multi-domain performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Novel compositional approach: The factor decomposition and recomposition strategy is intuitive and well-motivated, providing a principled way to scale up training data from limited seed examples while maintaining diversity and complexity.\n\n- Strong empirical results: COGS achieves meaningful improvements across multiple benchmarks (52.02% on ChartQAPro vs. 47.36% base model; 88.04% on VisualWebBench vs. 85.65% base), outperforming both general-purpose models and domain-specific baselines.\n\n- Transferability across datasets: The factor-level mixture experiments (Section 4.1.2) demonstrate that COGS induces generalizable reasoning capabilities rather than dataset-specific overfitting, with factor-level mixing outperforming data-level mixing.\n\n- Theoretical contribution: Proposition 3.1 provides valuable theoretical insight into why ProcessRM-max preserves policy ordering while ProcessRM-sum does not, backed by empirical validation.\n\n- Comprehensive evaluation: The paper includes thorough ablations (reward models, question complexity, factor types) and extends beyond charts to webpages, demonstrating generalizability.", "weaknesses": "- The paper focuses exclusively on artificial image domains (charts, webpages). It's unclear whether this approach would transfer to natural images or other multimodal reasoning tasks. The restriction to domains with \"abundant unlabeled images\" may limit applicability.\n\n- Dependency on high-quality decomposition: The entire framework relies on an MLLM's ability to accurately decompose questions into factors. The paper doesn't thoroughly analyze decomposition quality or failure modes. What happens when decomposition is incorrect or incomplete?\n\n- Limited baseline comparisons: While the paper compares against several data synthesis approaches, it doesn't compare against other compositional reasoning methods or more sophisticated prompting techniques (e.g., self-consistency, tree-of-thoughts).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PiLiIC0AwT", "reviewer": "Reviewer_ceC7", "summary": "This paper presents COGS, a framework for augmenting the reasoning capabilities of multimodal large language models (MLLMs) in domains lacking large annotated datasets, such as charts and webpages. The key idea is to decompose seed questions into primitive factors, and then systematically recompose these factors with new images to generate diverse, compositional training data.", "questions": "1. Does the manuscript describe any validation mechanism for the generated subquestion–answer pairs to prevent intermediate errors from propagating and compounding through the pipeline?\n\n2. Can the authors provide empirical results or error analyses examining the quality/diversity of the generated synthetic data? \n\n3. Is there any analysis of how the choice/size of the initial seed set $\\mathcal{Q}^0$ affects performance or coverage? For instance, how does model generalization degrade when the seed is minimal or unrepresentative?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "COGS factorizes a small seed set into reusable perception/reasoning factors, recomposes them with new images to create diverse, grounded QA pairs , and uses the factor structure to supply process-level rewards, yielding richer supervision than final-answer matching alone.", "weaknesses": "1. The authors state that the framework *does not require ground-truth subquestion answers*, but this appears to hold only for the *seed set*: during data synthesis, each sample includes LLM-generated sub-answers that are then used for process-level supervision in RL. If these pseudo-labels are noisy, errors may accumulate and be amplified through factor-level recomposition.\n\n2. The manuscript does not isolate the factor pool’s *direct* contribution, emphasizing cross-dataset ablations instead. Clarifying its substantive, incremental value, and distinguishing it from direct decomposing/recomposing method, would make the contribution more transparent.\n\n3. The title is *COMPOSITION-GROUNDED **INSTRUCTION SYNTHESIS**  FOR **VISUAL REASONING***, yet the manuscript does not clearly foreground *instruction synthesis* and *visual reasoning*; it predominantly centers on chart QA. The related-work section likewise emphasizes task descriptions rather than surveying these two threads. Moreover, while the introduction mentions tables and documents, the experiments are confined to a relatively narrow setting. Chart QA can serve as a proxy for visual reasoning, but the heavy focus on charts leaves the broader visual-reasoning aspect underdeveloped.\n\n4. The approach is mainly applicable when a task admits a reliable decomposition into subquestions; for complex cases that are not readily decomposable, its utility is limited.\n\n5. Minor editorial issues: on lines 71 and 205, ***Grouped Rollout Policy Optimization*** should be ***Grouped Relative Policy Optimization***; on line 184, the enumerator should be ***(i)***.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OHPsVmIxre", "reviewer": "Reviewer_jHKn", "summary": "This paper proposes COGS (Composition-Grounded Instruction Synthesis), a data-efficient pipeline to endow MLLMs with visual reasoning skills in artificial image domains like charts, documents, webpages. The method (i) decomposes a small set of seed questions into atomic perception/reasoning factors, (ii) recomposes sampled factors with new images to synthesize QA pairs with sub-questions and sub-answers, and (iii) fine-tunes an MLLM via GRPO using process rewards computed at the factor level.  Experiments on ChartQAPro and VisualWebBench show consistent gains over strong open baselines, ablations on mixing factor pools across datasets, reward model, factor break down further demonstrate the effectiveness and scalability of the method.", "questions": "1. Decomposition quality: What is the inter-run agreement for factor labels/sub-questions on the same seed? Any filtering or self-consistency checks?\n\n\n2. Mixture strategy: In factor-level mixing, how often do cross-dataset factors actually co-occur in recomposed questions? Can you provide some domain drift failure cases?\n\n\n3. Factor generation: When MLLM decomposes the main question and generate the subquestion with the Factor, how to make sure the number of the generated Factor is within a reasonable range?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clear, modular idea with practical upside. Factorizing seed questions then reusing those factors across unlabeled images is straightforward and domain-agnostic. The pipeline is well-motivated and broadly applicable.\n2. Process reward design with both theoretical and experimental justification. The ProcessRM-max objective is motivated by a simple but persuasive order-preservation argument and turns out to be effective.\n3. Transfer via data-level and factor-level mixing. The author provides a valid ablation on generalization over mixture of datasets, indicating the method captures reusable structure, not just surface patterns.", "weaknesses": "**1. Positioning in prior data-synthesis work is thin.**\n\n The introduction and related work don’t really situate the paper within the broader line of data-synthesis methods that first generate sub-questions/functions and then recombine them into new examples. It would help to spell out what’s borrowed vs. what’s new, why existing approaches fall short for this setting, and how the paper’s factorization addresses those limits. Please cite a few representative strands to make the attribution explicit. See [1-3].\n\n**2. Comparisons skip broader synthesis and RL frameworks.**\n\n The experiments compare against chart/GUI-specific generators, but there are notable data-synthesis pipelines in RL and process-supervised training that should be part of the picture. Even a small, controlled head-to-head with one or two representative frameworks (same base model, same budget) would clarify what the proposed method adds beyond existing synthesis strategies. Some works can be found in [4-5].\n\n**3. Reliability of synthetic sub-questions/answers isn’t audited.**\n\nDecompositions can be inconsistent or non-minimal, and recomposition relies on an LLM to produce sub-answers. The order-preserving argument assumes a particular noise structure, but we don’t see concrete diagnostics. Although the results demonstrate the effectiveness of the pipeline, we still need more diagnostics like sub-answer error rates; factor-label drift across seeds/images; sensitivity of GRPO to sub-reward noise. Right now, reliability is assumed rather than demonstrated.\n\n**4. Model ablation is narrow.**\n\nResults are limited to Qwen2.5-VL-7B. It’s unclear whether the gains persist across families (and weaker/stronger baselines) or scale with model size. A light grid—e.g., a smaller and a larger open model, plus a different family—would make the real-world applicability much clearer.\n\n[1]. Self-Instruct: Aligning Language Models with Self-Generated Instructions\n\n[2]. WizardLM: Empowering large pre-trained language models to follow complex instructions\n\n[3]. Automatic Instruction Evolving for Large Language Models\n\n[4]. ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search\n\n[5]. Self-Rewarding Language Models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8LbqS3GM2A", "reviewer": "Reviewer_dvz5", "summary": "The paper proposes COGS (COmposition-Grounded instruction Synthesis), a three-stage pipeline to improve VLMs’ reasoning in artificial image domains (charts, rendered documents, webpages). Stage 1 decomposes a seed set of questions into interpretable perception and reasoning factors; Stage 2 recomposes sampled factors with new images to synthesize new questions; Stage 3 fine-tunes VLMs using GRPO with the generated questions. Experiment demonstrates the effectiveness of COGS on ChartQAPro and MMC-Bench.", "questions": "Please see Weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tThe pipeline of COGS is clear and easy to follow. The method is intuitive.\n\n2.\tFactor-level sub-questions make the supervision more transparent and are practical for error analysis.", "weaknesses": "1.\tEvaluation protocol risks leakage. The use of 33% of the test set as seeds violates the test-only usage and can tune the pipeline to the test distribution even without answer leakage. I checked the ChartQAPro benchmark, and it has 1341 charts for 1948 questions. Different questions may target the same image chart; this makes image-level leakage likely.\n\n2.\tThe process reward uses LLM-as-a-judge, which can cause significant training overhead. The paper should report efficiency statistics", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)