FlowGen: Synthesizing Diverse Flowcharts to Enhance and Benchmark MLLM Reasoning
Authors: Kaiwen Shi
Sichen Liu
Ziyue Lin
Hangrui Guo
Gong Cheng
Venue: ICLR 2026 Poster (2026)
Keywords: flowchart parsing
flowchart QA
flowchart synthesis
Primary area: datasets and benchmarks
TL;DR: A controllable synthesizer that generates flowcharts that have customizable structural features and supports multiple renderer backends.

Abstract: Flowcharts are widely used to represent processes and relationships through intuitive visual representations. However, accurately interpreting these diagrams remains challenging due to their structural complexity and high visual diversity. Existing flowchart datasets often lack fine-grained control over key properties such as graph complexity and rendering style, limiting their utility for training and testing of multimodal large language models (MLLMs) on visual reasoning tasks. To address these limitations, we introduce FlowGen, a controllable synthesizer that generates flowcharts that have customizable structural features and supports multiple renderer backends. FlowGen enables fine-grained control over graph properties such as graph order and size, branched arrows, and nested subgraphs, facilitating systematic evaluation of MLLMs' capabilities. Extensive experiments on open-source and proprietary MLLMs show that training on FlowGen substantially improves flowchart parsing and question answering (QA), while also enhancing generalization to other public datasets. Furthermore, FlowGen provides challenging test datasets that expose consistent weaknesses in current MLLMs, particularly related to high structural complexity and varied rendering styles. Our code and data are publicly available at https://github.com/nju-websoft/FlowGen.

Published as a conference paper at ICLR 2026
FLOWGEN: SYNTHESIZING DIVERSE FLOWCHARTS TO ENHANCE AND BENCHMARK MLLM REASONING
Kaiwen Shi, Sichen Liu, Ziyue Lin, Hangrui Guo, Gong Cheng*
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China (ka wenshi, sichenliu, 221240094, 221240069 smail. ammail.ng w.w....jj......e................ edu. fereengejuuedu....
ABSTRACT
Flowcharts widely used to represent processes and relationships through in- tuitive visual representations. However, accurately interpreting these diagrams remains challenging due to their structural complexity and high visual diversity. Existing flowehart datasets often lack fine-grained control over key properties such graph complexity and rendering style, limiting their utility for training and testing of multimodal large language models (MLLMs) on visual reason- ing tasks. To address these limitations, we introduce FlowGen, controllable synthesizer that generates flowcharts that have customizable structural features and supports multiple renderer backends. FlowGen enables fine-graine fine-grained con- trol over graph properties such as graph order and size, branched arrows, and nested subgraphs, facilitating systematic evaluation MILMs' capabilities. Ex- tensive experiments on open-source and proprietary MILMS show that train- ing on FlowGen substantially improves flowchart parsing and question answer- ing (QA), while also enhancing generalization to other public datasets. Further- more, FlowGen provides challenging test datasets that expose consistent weak- nesses in current MLLMs, particularly related to high structural complexity and varied rendering styles. Our code and data are publicly available at https: //githuu.. com n websoft. /Fwooeen.....
INTRODUCTION
Flowcharts, as illustrated in Figure 1, are widely used to represent processes, relationships, and workflows, prevalent in domains such scientific communication, business process modeling (BPMN), software engineering documentation, and education (Bhushan et al., 2024). As graph- ical language, flowcharts integrate symbolic information with spatial layout, enabling humans to quickly understand complex processes. However, this same structural complexity makes automatic parsing highly challenging (Montalvo, 1990). Building models to extract structured representations from flowcharts is crucial for enabling downstream applications, including code generation, knowl- edge extraction from documents, automated reasoning workflows, and multimodal question answering (Vasudevan et al., 2008; Bashir & Giri, 2013; Hu et al., 2024).
Motivation. Recent advances in vision language models (VLMs) and MLLMs have shown potential to understand flowcharts, enabling end-to-end reasoning from images (Ye et al., 2024). However, the development of such capabilities is hindered by the limitations of existing datasets. On the one hand, existing test sets provide limited control over complex structures, such as branching factor and hierarchical nesting, They are also usually
solada Delvery Trusiin
Starlgy Tuzts
Paymert
Cuus sellig Recommercaton
Prica Achurment
penentee
Sodal Iragratoo
Upeiii)
Doncuuton
Marksing Compatgn Launch
Figure Example flowchart synthesized by FlowGen.
"Corresponding Author: Gong Cheng.

---

Published as a conference paper at ICLR 2026
Table 1: Comparison between our proposed synthesizer (FlowGen) and existing flowchart datasets.
Danaset
Complexity
Renderen(s)
Source
#Train
dTes
RowchanQA (Tannert 2023)
Graphaviz
Synthetic
released
PQo (Singh ot al., 2024)
NA NA
Mermaid
Synthetic
1,319
CBD (Bhashan & Lec, 2022)
NA
Web crawling
300
96
(Awal 2011)
Branching Branching
Hand-drwn flowcharts
2016)
Hand-drawn
IMR (Schiller et al., 2021bb
Branching & Nesting
NA
Hand-diww RPMN diagrams
173
Flodd cam (Pan al. 2024)
Branching
Mermaid
Synthetic
8,000
2,000
Mermaid, Graphviz, PlantUML, Diagrums
FlowGen (ours)
Branching Nesting
Synthetic
Unlimited
Unlimited
rendered in single visual style, which prevents evaluating the robustness stylistic variation, while real-world flowcharts are highly diverse (Awal et al., 201 I; Bresler et al., 2016; Schäfer et al., 2021b; Bhushan & Lee, 2022; Tannert et al., 2023). On the other hand, existing training sets are small, narrowly focused specific domains,
missing important structural features (Sun et al., 2022: Singh et al., 2024; Bhushan et al., 2024; Pan et al., 2024). These issues hinder effective model training, and models fail when confronted with high structural complexity cross-renderer settings. These limitations call for a more principled way to generate both training and test resources. More related work is discussed in Appendix B.
Our Work. To bridge both training and test gaps in previous work, compared in Table 1, we introduce FlowGen, controllable flowchart synthesizer that generates diagrams with tunable struc- tural features and supports multiple rendering styles. Figure I presents example flowchart syn- thesized by FlowGen, demonstrating split arrows, merge arrows, nested structures, and variations in colors, node shapes, and edge styles. This diversity enriches the distribution of generated flowcharts to better approximate real-world practices.
With FlowGen, we: (1) build large-scale symthetic training set and demonstrate that fine-tuning open-source MILLMs on it substantially boosts their ability of flowchart parsing and flowchart QA, with strong transfer gains multiple existing datasets, in some cases even surpassing proprietary MLLMs; (2) construct synthetic test sets with controlled complexity and renderer diversity, and use them to comprehensively test both open-source and proprietary MLLMs, revealing their consistent weaknesses in handling nested structures and cross-renderer generalization. Together, these results establish controllable flowchart synthesis as a practical and effective strategy to advance the machine understanding of such diagrams.
Our contributions are summarized as follows:
We introduce controllable flowchart synthesizer capable of generating floweharts with adjustable structural complexity and multiple renderer styles, enabling the principled con- struction of both training and test resources.
We synthesize large training sets and show that fine-tuning with them significantly im- proves MLLM reasoning on flowcharts, with strong transferability to wide range of ex- isting flowchart parsing and QA datasets.
We synthesize challenging test sets that systematically expose weaknesses of both open- source and proprietary MLLMs, providing insights into their limitations under high com- plexity and cross-renderer variation.
2 SYNTHESIZER FRAMEWORK
Figure 2 provides overview of our FlowGen synthesizer pipeline. The synthesizer converts compact, user-specified configuration into structurally valid graphs, then semantically annotates and renders them with multiple, stylistically diverse backends. The pipeline is organized into three stages: (1) configuration, (2) graph construction, and (3) rendering. These stages jointly enable fine-grained control structural complexity, semantic richness, and visual appearance.
2

---

Published as a conference paper at ICLR 2026
Stage 2: Graph Construction
Stage 1:
Stage 3:
Configuration
Rendering
Stage 2.1:
Stage 2.2:
Stage 2.3:
Topology Generation
Semantic Annotation
Style Definition
40
Customization
Stylistiic Diversity
Human Verification
Graph Order (v)
Sampled Parameters
Style Dictionary
Mermaid
Split Arrows (s.)
120 Real-world Application Domains
Merge Arrows
Spanning Tree db 0000
Visual Richness
Grophviz
(rx)
Branching Factor (dg)
Add Additional Edges
40 Node Nomes
40 Edge Labels
Node/Edge Shapes
PlantUML
Density (o)
Diagrams
Unlabeled Edge Ratio (2)
Border/Fill Color
Nested Subgraphs (n)
Introduce Nested
Node/Edge Annoation
Optional:
Subgraphs
Scanned Style
Figure 2: Overview of our Geo synthesizer pipeline.
2.1
CONFIGURATION ATION
The synthesizer begins by sampling a set of structural parameters that govern the topological and semantic complexity of generated flowcharts. We define seven controllable parameters.
Graph Order (v). This specifies the total number of nodes in a flowchart, including those con- tained in nested subgraphs. We categorize the scale into three levels: small (8-12 nodes), medium (13-20 nodes), and large (21-30 nodes). This parameter the primary factor controlling the overall length and complexity of the synthesized flowcharts.
Split Arrows (c.). Split arrows introduce virtual nodes that model the divergence of a single pro- cess into multiple concurrent branches. Although they not carry domain-specific semantics, they substantially increase structural variability. The parameter é, specifies the number of such virtual nodes, which regulates the branching complexity the graph structure.
Merge Arrows (cm.). Merge arrows capture the convergence of multiple parallel streams into a single unified process. Each merging structure introduces virtual node with multiple incoming edges, and the total number of such virtual nodes is controlled by em Together with split arrows, these constructs enable simulation of complex control-flow patterns.
Branching Factor (Aw). This governs the maximum fan-in and fan-out degrees of virtual nodes created for the split and merge arrows. During construction, up to As outgoing edges are assigned split node, and up to As incoming edges to a merge node. This provides fine-grained control over the degree of divergence and convergence.
Density (p). This is defined as the edge-to-node ratio e/v, where denotes the total number of edges and y the number of nodes. This parameter controls the overall visual density of graph. Higher values of indicate denser connectivity patterns, leading to visually cluttered graphs, whereas lower values produce sparser and more tree-like structures.
Unlabeled Edge Ratio (2). This parameter À E [0 specifies the proportion edges that do not carry textual labels. A higher increases semantic ambiguity and makes the reasoning task
3

---

Published as a conference paper at ICLR 2026
harder. When À = 0, all edges are annotated with labels; when À = 1, edges contain labels. This parameter allows for controlled adjustment of semantic sparsity.
Nested Subgraphs (r)). This determines the number of hierarchically embedded subgraphs, which introduce structural depth in addition the global node count Each nested subgraph is assigned depth 1, and the edges incident to a subgraph connect directly to its internal nodes.
2.2 GRAPH CONSTRUCTION
Given the sampled parameters, the synthesizer constructs an abstract directed acyclic graph with se- mantically coherent labels
the structural backbone of the flowchart. The process consists of three stages: topology generation, semantic annotation, and style definition.
2.2. TOPOLOGY GENERATION
We begin by generating random spanning tree nodes to establish acyclic backbone for node positioning. To increase connectivity, additional edges are then introduced according to the sampled Eg. cm and Ag. If the resulting number of edges is still below the target density further random acyclic edges are added until the specified ratio is met. To incorporate hierarchical depth, nested subgraphs are created by randomly selecting to 5 connected nodes and grouping them into higher-level node, controlled by the nesting count Nested subgraphs are constrained to be disjoint, preventing overlap and preserving clarity. If exceeds the feasible number of disjoint groupings for a given graph size, the generator halts further nesting to maintain structural validity.
2.2.2 SEMANTIC ANNOTATION
Once the abstract topology is constructed, the synthesizer assigns semantic domain by sampling a topic from 120 predefined application domains. Each topic provides 40 node names and 40 edge labels, initially generated with GPT-4o and refined through human verification. The nodes and edges are annotated with topic-specific labels to maintain semantic coherence, and the edges are labeled with probability 1 A, where À E [0, 1] denotes the unlabeled edge ratio. Additional details of the application domains and representative node and edge vocabularies are provided in Appendix C.
2.2.3 STYLE DEFINITION
After topology generation and semantic annotation, the visual appearance of nodes and edges is determined using renderer-specific definitions from a predefined style dictionary. The node and edge shapes are randomly sampled from the available ones, with possible repetition to emulate the variability of real-world diagrams. To introduce color variation, five palettes are randomly drawn from pool of 90 predefined schemes for each flowehart topic instance, and node borders, fills, and edge colors are sampled from these palettes. This rendering strategy enhances stylistic diversity and visual richness while ensuring thematic consistency within each flowchart synthesized.
2.3 RENDERING
The final stage of the synthesizer converts abstract annotated graph codes into visual flowcharts. To capture the diversity of real-world styles, we integrate four complementary rendering backends: Mermaid', Graphviz², PlantUML3, and Diagrams". Each backend provides its own syntax, layout algorithms, and visual conventions. For instance, Mermaid emphasizes lightweight web-native dia- grams, Graphviz offers fine-grained control over hierarchical layouts, PlantUMI. supports modular subgraph definitions, and Diagrams provides programmatic composition of clusters and entities.
During rendering, abstract nodes and edges, which have been annotated with textual labels and style attributes, are translated into renderer-specific elements such as shapes, connectors, and nesting sub-
"nttps://mermaadd-iis thub. io/
"nttps://plantur com/ hnttps://diagran inngannmeeecmmmmmm com/
4

---

Published as a conference paper at ICLR 2026
graphs. By applying the user-specified renderer, synthesizer produces visually rich and diverse flowcharts that closely resemble diagrams created real- scenarios.
FLOWGEN FOR MLLMS TRAINING
In this section, we systematically evaluate the performance gains of MLLMs fine-tuned on the FlowGen-synthesiiz training set for flowehart parsing and flowchart QA tasks. We further con- duct ablation experiments to dissect the impact of key synthesis components.
3.1 EXPERIMENTAL. SETUP
3.1.1 EVALUATION DATASETS
We evaluate wide range of MLLMs on datasets covering two task categories. For flowchart pars- ing, use Flow (Singh al.. 2024), CBD (Bhushan & Lee, 2022), FC A (Awal al., 201 I), FC (Bresler et al., 2016), hdBPMN (Schäfer et al., 2021b), and FlowLLearn (Pan et al., 2024). These datasets provide flowcharts with structural annotations, enabling direct assessment of parsing accuracy. For flowchart QA, we adopt Flow VQA, FlowLearn, AI2D (Kembhavi et al., 2016), and MISS-QA QA (Zhao al., 2025), which contain diagram-basse questions that require reasoning over both structure and semantics. Details of these datasets are provided in Appendix D.I.
3.1.2 EVALUATION METRICS
For flowchart parsing, we represent the graph structure as triplets that capture both topology and semantics. We distinguish: (1) labeled edges (A, B), where X denotes directed relation from node A to node B; (2) unlabeled edges (A, connectedTo, B), which use the predefined re- lation connectedT to indicate an unlabeled relation between and B; and (3) nested subgraphs (A, partO f, B), where node A belongs to a container group B, capturing hierarchical containment.
Strict Precision, Recall, and F1. Based on triplets, we compute standard parsing metrics:
Precision iTreed (1 T gotal [Tpreal
[Tpred O good [Tgotal
2 Precision Recall
Recall
F1 Precision + Recall
(1)
These strict metrics require string matches between predicted (Tpred) and gold-standard triplets (Tgota), thus capturing topological fidelity but sensitive to textual mismatches.
Relaxed Precision, Recall, and F1. To tolerate minor textual variations, we define a relaxed matching rule based on the normalized Levenshtein distance (Levenshtein, 1965). The edit simi- larity (ES) between two strings S₁ and S2 is given by:
EdidDistance(Sj,S. S2) max (IS)) (S2))
S2)
predicted triplet (hp, p tp) matches a gold-standard triplet (hy. gg ty) if and only min min(ES( hg), (S(pp.rg) ES((p: (g)) 0.85.
Using this relaxed criterion, we recompute precision, recall, and FI, which provide robustness. to minor OCR errors while still strictly evaluating structural correctness.
Accuracy. For flowchart QA datasets (e.g., AI2D, MISS-QA), report Accuracy, defined the proportion of questions for which the model prediction exactly matches the gold-standard answer. This metric captures end-to-end reasoning performance and complements the triplet-based metrics.
3.1.3 PARTICIPATING MODEL.S
We evaluate the performance of 10 MLLMs across two categories: (1) open-source models, in- cluding Qwen2.5-VL-33B7BB VL-3B/7B (Bai et al., 2025), VL3-2B (Zhu et al., 2025), MiniCPM-V2.6- 8B (Yao al., 2024), Llava-V1. V.1.6--isttal--BB-HH (Liu et al., 2024), and the Gemma3 series (4B,
5

---

Published as a conference paper at ICLR 2026
Table 2: Configurations for synthesizing the training Ranges indicate values to sample.
Difficulty
Ca
Fm
A
Easy
(8, 12)
(0, 0)
(0, 0)
(1. 2)
0
(1.0, 1.1)
(0.3, 0.6)
Medium
(13,20)
(1,
(1,2)
(2,
1
(1.1,
(0.6, 0.8)
Hard
(21,30)
(2,
(2.3)
(3,4)
2
(1.2, 1.3)
(0.8, 1.0)
Table 3: Performance comparison of MLLMs flowchart parsing datasets. We report strict Fl and
relaxed F1 (rF1) scores. We compare each model with its variant fine-tuned on FlowGen (+ SFT).
Flow VQA
CBD
FCA
FCB
1dBPMN
FlowLearn
Model
F1
(F
F1
#FI
F1
IF1
FI
(FI
FI
iFI
F1
iFl
Proprictary Multimodal Large Language Models
GPT-4o
67.1
52.6
55.3
35.8
53.9
Gomini-2.5-Flash
62.3
63.3
63.2
66.2
38.5
40.9
52.9
55.8
8.1
10.5
82.1
86.2
GL-M-4V-Plus
74.0
74.5
53.6
58.7
33.0
34.0
46.8
49.3
8.6
11.3
31.6
40.8
Open Source Multimodal Large Langunge Models
Qwen2.5-VL-3B
12.7
13.0
17.9
18.3
2.8
3.0
8.6
9.1
2.3
3.3
20.6
25.4
SFT
51.3
52.9
49.8
52.0
22.4
23.0
34.6
37.2
8.4
11.3
41.6
55.5
Qwen2.5-VL-7B
59.4
59.7
49.1
51.2
25.8
27.2
32.0
33.2
16.6
20.6
43.2
48.3
SFT
70.9
55.4
57.7
31.4
40.8
43.7
18.3
24.2
60.1
IntemVL3-2B
15.7
17.0
18.4
19.5
4.6
5.7
8.3
9.1
3.2
4.3
21.8
32.6
34.6
37.8
21.6
2.1
MiniCPM--V2.--8B
17.6
18.5
20.4
20.9
7.5
8.7
8.8
10.2
3.4
5.3
9.1
12.7
SFT
44.0
47.2
37.9
39.5
13.4
14.3
22.9
24.9
3.6
5.7
20.0
31.0
2.6
3.3
12.2
13.1
2.8
3.7
1.0
8.9
SFT
4,9
8.7
14.0
17.4
2.3
2.7
4.8
5,4
1.1
1.5
4.4
13.5
Gemma3-4B-11
17.8
3.6
SFT
19.6
29.8
28.6
35.7
15.3
18.8
7.5
11.2
1.2
1.7
11.4
21.0
Gemma3- 12B-IT
36.4
40.9
36.2
39.0
11.6
14.8
12.7
19.0
10.1
13.7
29.7
37.3
SFT
38.9
46.3
39.3
45.5
13.1
15.8
17.9
23.1
3.7
4.6
24.4
39.0
12B) (Team al., 2025a); and (2) proprietary models, including GPT-4o (OpenAI, 2024), Gemini- 2.5-Flash (Comanici et al., 2025), and GLM4V-Plus (Team et al., 2025b). Appendix D.2 summarizes the details of each model. All experiments are carried out on standardized software and hard- ware stack, using PyTorch 2. 6. Transformert RTX 5880 Ada Generation 48GB GPUs. The training and test code is implemented in Python 3.10, with reproducibility ensured by fixed random seeds and deterministic dataloaders. Detailed experimental settings, including prompts, are provided Appendix D.3 and Appendix F.
4 5.53.3. and CUDA 12.4 on NVID IA
3.2 SYNTHESIZED TRAINING SET
To synthesize training set with FlowGen, systematically define three configurations of param- eters representing different levels of difficulty: easy, medium, and hard, as shown in Table 2.
For each renderer (Mermaid, Graphviz, PlantUM,, Diagrams), we generate flowcharts across all three difficulty levels. Each difficulty level is combined with 120 topics, with eight random instan- tiations per topic. This yields total of 1,,520 flowcharts in the synthesized training set.
3.3
EXPERIMENTAL RESULTS
3.3.1
MAIN RESULTS
Table reports the flowehart parsing results on six benchmarks. Models fine-tuned on Flow- Gen (+ SFT) generally yield substantial improvements across models and datasets. For instance, Qwen2.5-VL-3B improves from 12.7% to 51.3% on Flow and from 2.8% to 22.4% on FC_A, while Intern VL3-2B increases from 15.7% to 34.6% FlowVQA. Qwen2.5-VL-77 im- proves from 43.2% to 60.1% F1 on FlowLeam, demonstrating strong cross- generalization rather than overfitting to synthetic data. Importantly, these improvements also manifest under edit similarity. Although performance on the challenging hdBPMN benchmark remains modest, the gains confirm that synthetic training substantially boosts generalization to real-world datasets.

---

Published as a conference paper at ICLR 2026
Moreover, open-source models fine-tuned with FlowGen comparable proprietary models some datasets. For instance, Qwen2.5-VL-7B attains 60.1% on FlowLearn and 18.3% on hdBPMNN, outperforming GLM4V-Plus (31.6% and 8.6%). These findings highlight FlowGen's 's potential to broaden access to high-performing models with limited resources.
We further evaluate models flowchart QA benchmarks, as reported in Table 4. All QA evalua- tions are conducted under zero-shot prompting (see Appendix E). As expected, prompting with gold- standard triplets establishes an upper bound on achievable gains. No- tably, triplets extracted by FlowGen consistently achieve performance close to gold-standard annotations across models and datasets, while substantially outperforming triplets extracted by the base model itself. For example, Qwen2.5-VL-3B reaches 88.2% on FlowLearn with eextreeee triplets, ap- proaching the gold-standard result of 89.1%, and markedly exceeding the 77.1% obtained with self- extracted triplets. Similarly, on AI2D, MiniCPM-V2.6-8B attains 42.8% using FlowGen-extracter triplets, compared 45.0% with gold-standard triplets and only 32.6% with self-extraction. Similar trends are observed on MISS QA and Flow VQA.
Table 4: Performance comparison (accuracy) of MLLMs flowchart QA datasets. We compare each model with its variants prompted with gold-standard triplets (+ Gold-Standard Triplets), with triplets extracted by itself (+ Self-Extracted Triplets), and with triplets extracted by Qwen2.5-VL-7B fine-tuned on FlowGen (+ FlowGen- Extracted Triplets).
Model
FQA
FlowLearn
A12D
MISS-QA
GPT-4o
90.2
83.2
71.7
63.0
Gemini-2.5-BBa
88.0
83.4
63.2
67.3
GLM
86.4
89.8
74.6
$7.0
Owen2.5-VL-3B
64.6
72.3
50.7
35.6
Goldd-tandard Triplets
75.6
89.1
53.5
47.3
Sellf-Extracted Triplcts
66.2
51.2
37.2
Floo Triplets
73.4
88.2
51.4
38.7
Owen2.5-VL-7B
Gold Stamdard Triplets
83.2
73.9
62.5
55.3
Sell Extracted Triplets
77.1
70.7
60.5
44.6
BhowGen-Exttacee Triplets
81.1
61.1
49.2
MiniCPM- V2.6
61.6
50.9
32.0
31.1
Gold-Standard Triplets
71.4
45.0
37.8
Self Extratted Triplets
61.0
$1.1
32.6
31.5
Extrate-Exttaatted Triplets
Besides triplet-augmented QA, we also fine-tune QA models on FlowGen- synthesized flowchart parsing tasks. Specifically, on FlowVQA and FlowLearn, fine-tune each model under set- tings: (i) SFT on the original QA training set, and (ii) SFT on an equal-sized mixed training set where half of the original QA samples are replaced with FlowGen- synthesized flowchart parsing tasks. As shown in Table 5, incorporating Flow- Gen generally improves end-to-end QA accuracy across models and datasets. For instance, Qwen2.5-VL-7B increases from 85.3% to 89.1% on FlowLearn, while MiniCPM-V2.6 improves from 73.2% to 77.6% on FQo These gains indicate that FlowGen enhances models' intrinsic ability to understand flowcharts, even when evaluated directly on downstream QA tasks using their fine-tuned checkpoints. Together with the triplet-aummented QA results, this end-to-end evaluation demonstrates that FlowGen delivers genuine performance improvements by effectively enhancing models' structure- aware visual reasoning capabilities, rather than merely providing auxiliary intermediate representations.
Table 5:
Performance comparison (accuracy) of MLLMs on flowehart QA datasets. We compare each model with its variants fine-tuned on the original train- ing set SFT on the original training set) and on mixture of the original training set and FlowGen (+ SFT on mixed training set).
Model
VQA
FlowLearn
Qwen2.5 VLLBBB
64.6
72.3
SFT on the original training set
81.3
85.1
-FT on mixed training set
85.6
84.7
Qwen2.5-VL-7B V-VB-B
74.6
71.1
SHT on the original triningg set
88.2
85.3
SFT on mixed training set
89.2
89.1
MiniCPM-V2 6-.6-88
61.6
80.9
SFT on the original training set
73.2
86.2
SFT mixed training
77.6
87.5
We also investigate whether fine-tuning on FlowGen transfers to broader multimodal reasoning abil- ities, with detailed results reported in Appendix F.
7

---

Published as a conference paper at ICLR 2026
Table 6: Ablation results (strict F1) of FlowGen under different settings.
Training Variant
FlowVQA
CBD
FC A
FC
bdBPMN
FlowLearn
Full FLOW (All Features)
70.9
55.4
29.5
40.8
18.3
60.1
w/o
(Only Mermaid)
76.2
53.3
28.5
38.1
17.0
60.9
w/o Nested Subgraphs
70.0
54.6
29.2
40.6
16.6
59.2
w/o Split/Merge Arrows
69.6
54.3
28.4
38.8
17.7
58.3
3.3.2 ABLATION STUDY
We perform ablation experiments on Qwen2.5-VL-77 using synthetic data generated under dif- ferent constraints while keeping the total training data size identical the main experiment. We evaluate each variant on six flowchart parsing benchmarks, reported in Table 6.
Restricting training data single renderer (Mermaid) substantially reduces cross-domain generalization, as shown by drops on CBD (from 55.4% to 53.3%) and FC_B (from 40.8% to 38. %). In contrast, Flow VQA and FlowLeearn slightly improve under this setting, since they exclusively use Mermaid and thus align better with the single-renderer distribution. In general, these results highlight the importance style variation for learning representations that generalize between renderers and improve robustness.
(1) Multi-renderer:
(2) Nested subgraphs: Removing hierarchical nesting leads to markedly reduced accuracy on hdBPMN, the only benchmark among the six that extensively features nested grouping structures, where performance decreases from 18.3% to 16.6%. Smaller declines are also observed on other datasets. This suggests that additional exposure to nested structures provides the model with richer structural signals, enabling it to better capture flowehart organization even when explicit nesting is absent. In general, these results underscore the value of modeling hierarchical layouts.
(3) Split/merge arrows: Excluding split/merge arrows results performance degradation on sev- eral benchmarks, including FC_B (from 40.8% 38.8%), and FlowLearn (from 60. 1 to 58.3%). This confirms that such structures are critical for accurately modeling branching logic and parallel workflows, which are common in real-world flowcharts.
To further isolate the contribution of structural information. we perform an additional ablation where all node names and edge labels are replaced with random meaningless strings, preserving the original graph structure while removing semantic annotation. The results are reported in Appendix G.
3.3.3 ERROR ANALYSIS
We analyze 50 cases where models fine-tuned with FlowGen fail, including 25 cases from Flow and 25 from hdBPMN. Note that single failure case may contain multiple error types; the percent- ages reported in the following can added to more than 100%. Three main error types emerge.
(1) Complex nesting 4)%%: Deeply nested structures, particularly in hdBPMN, often lead to incor- rect handling of hicrarchical groupings and subgraph boundaries. Models often flatten or misplace partof relations. Examples are presented in Appendix H.I.
(2) Edge ambiguity 44%%) For high edge density or overlapping connectors, models sometimes confuse edge directionality misinterpret relation labels. Examples presented in Appendix H.2.
(3) OCR-driven failures (56%)) Although public benchmarks are primarily composed of visually clear diagrams, covering both digitally rendered and handwritten flowcharts, occasional recognition emors still occur (e.g., due to small font or stylized text in node labels). These OCR-related errors lead to incorrect missing node labels. Illustrative cases are shown in Appendix H.3.
FLOWGEN FOR MLLMS TESTING
In this section, construct challenging test set using FlowGen evaluate the robustness of MLLMs in flowchart parsing. The evaluation is designed to assess model capabilities in cross- renderer generalization, comprehension of nested structures, and parsing from noisy scanned inputs.
8

---

Published as a conference paper at ICLR 2026
4.1 EXPERIMENTAL SETUP
4.1.1
DATASETS
The FlowGen test set is designed to evaluate model performance along two challenging dimensions: graph complexity and scanned document degradation. Accordingly, we construct six evaluation subsets by systematically configurating these two factors.
(1) Graph subsets (Basy/Meduun/Hard) Following the definition Section 3.2, the structural difficulty of a flowchart is determined by a set of parameters. These configurations yield three levels of complexity: Easy, Medium, and Hard. For each difficulty level, we synthesize 1,440 flowcharts (4 renderers x 120 topics x 3 instances), resulting in total of 4,320 flowcharts across the Graph- Fasy/Medium/Hann subsets.
(2) Scanned subsets (Easy/Medium/Hard). To evaluate robustness against simulated scanning effects, we further rasterize flowcharts into scanned-like images with degradations such as blur, perspective distortion, and lossy compression using pillow 10. 4. 0 $ and opencv-python 4.10.0 O. 84 6. 'Tee severity of these perturbations increases from Easy to Hard. To avoid confound- ing structural variation, each scanned subset is generated by sampling one instance per topic from Craph-Easy/Medumm/EE
Each subset again contains 1,440 diagrams, leading to 4,320 flowcharts in total. The full configuration ranges for scanned degradations are provided in Appendix I.
The training set is synthesized with the same pipeline and configurations as the test set, ensuring matched distributions, while remaining strictly disjoint to prevent overlap.
4.1.2 EVALUATION METRICS
For evaluation the test set, we follow the strict precision, recall, and F1 score metrics introduced in Section 3.1.2, thus maintaining consistency with the criteria used during the training experiment.
4.1.3
PARTICIPATING MODELS
We evaluate the same set of models described in Section 3.1.3.
4.2
EXPERIMENTAL RESULTS
4.2.1
MAIN RESULTS
Table summarizes model performance on the FlowGen test set. Overall, we observe that both open-source and proprietary MsLMs struggle on this benchmark: even state-of-the-arn proprietary systems such as GPT-4o and Gemini-2.55Flaas achieve less than 25% F1 on most subsets. Open- source base models also perform poorly, highlighting the structural and stylistic challenges posed flowcharts are used for both training and by FlowGen. In this experiment, FlowGen-synthesizee under this potentially favorable condition, testing deliberate stress-test setting. Notably, where renderer- level regularities could in principle be exploited during training, model performance remains far from saturation across all subsets. This suggests that the challenge of FlowGen primarily arises from ity complex and diverse topologicul structures (e.g., branching, nesting, and heteroge- neous layouts), rather than superficial rendering effects.
We further observe that training on the subset combining samples from all the six subsets (+ SFT on Combined) consistently improves performance across all test subsets, demonstrating a degree of cross-subset generalization. However, this unified training strategy typically underperforms subset- specific fine-tuning, indicating that different subsets capture distinct characteristics. Even with com- bined supervision, the absolute performance on hard subsets (i.e., Graph Hard and Saaneed-Hard remains limited. These results demonstrate that FlowGen constitutes a challenging benchmark for current MILMs: even when sample distributions overlap and unified training is applied, models still struggle to robustly parse complex flowchurt structures, highlighting the diversity and difficulty of the underlying graph distributions. We also conduct an extra experiment to quantitatively assess the diversity flowcharts synthesized by FlowGen, with details provided in Appendix J.
10.0.84/

---

Published as a conference paper at ICLR 2026
Table 7: Performance comparison (strict F1) of MILMS FlowGen test subsets.
Test Subsets
Graph Easy
Graph Medium
Graph Hard
Scanned Easy
Scanned-Medium Medium
Scanned Hard
GPT-4o Flash
38.4
16.2
12.0
24.0
23.2
20.5
Gemini F.s.-lshh
43.2
13.7
9.7
22.9
23.1
21.6
GLM-4V-Plus
30.0
9.4
5.3
15.3
19.3
18.3
Qwen2.5. 2..e2255VVL-31
SFT on Graph Easy
50.2
24.0
16.0
33.0
30,5
29.0
SFT on Graph Medium
36.3
32,9
28.2
33.1
33.1
31.3
SFT Graph Hard
26.9
30.5
30.3
29.4
29.8
28.4
SFT on Scanned Easy
45.1
31.8
29.0
34.8
35.3
34.1
SFT Scanned Medium
43.6
32.2
35,5
SFT on Scanned Hard
43.7
32.6
29.3
35.6
35,5
35.0
SPT Combined
49.9
33.1
Qwen2.5-V1.777
35.7
10.9
7.7
18.9
18.5
17.2
SFT on Graph Easy
74,9
35.0
21.2
45.3
44.3
41.9
SFT Graph Medium
SFT on Graph Hard
39,1
43.6
43.3
42.6
42,9
40.8
SFT on Scanned Easy
66.8
47.8
41.6
53.9
53.1
49.7
SFT Scanned Medium
68.8
48.1
42.0
54.4
53.6
51.0
SFT on Scanned Hard
66.0
47.1
42.0
$3.0
52.6
90.2
SFT Combined
66.5
47.2
42.6
52.8
51.7
30.2
MiniCPM-VV.6--5
7,3
2.1
2.1
4,0
4,3
3.5
SFT on Graph Easy
33.0
13.1
9.0
18.1
17.7
SFT Graph Medium
23,9
19.6
15.2
20.0
19.9
18.1
SFT on Graph Hard
17.5
17.
17.4
17.8
17.9
16.8
SFT Scanned Easy
29.6
19.1
22.2
22.1
20.8
SFT on Scamned Medium
29.6
19.0
16.1
22.3
22.3
20.4
SFT Scanned Hard
SFT on Combined
39,5
24.2
20.6
29.0
28.8
27.1
4.2.2 ERROR ANALYSIS
We analyze 50 failure cases on the FlowGen's test set and identify four major error categories: Cross- renderer generalization, Complex nesting, Edge ambiguity, and OCR-driven failures. Compared to previous datasets, FlowGen highlights additional challenge Cross-renderee generalization, where stylistic differences between renderers (e.g., PlantUML vs. Mermaid) frequently lead to structural misinterpretations. Detailed error breakdowns and examples provided in Appendix K.
CONCLUSION
In this work, present FlowGen, controllable flowchart synthesizer that exposes explicit struc- tural configurations and supports diverse rendering styles, providing both a scalable training re- source and systematic test benchmark. Our experiments show that current MILMs, including leading proprietary models, perform poorly on the FlowGin test set, particularly under high struc- tural complexity and varied rendering styles. However, fine-tuning on FlowGen substantially boosts the robustness and generalization of open-source models, highlighting controllable synthesis as an effective strategy for advancing flowchart understanding and its downstream reasoning tasks. By releasing FlowGen, we provide the community with reproducible and extensible framework for training. testing, and advancing models for graph-structured visual reasoning.
To address the limitations of our work, our future work includes exploring diverse implementa- tions of split/merge arrows beyond introducing virtual nodes, incorporating more features of real flowcharts into our synthesizer pipeline, and experimenting with more complex flowchart-based tasks such as root cause analysis in IT operations.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
The detailed experimental settings of synthesizer, models, hyper-parameter settings, and computa- tional resources can be found in Section 3.1.3, Appendix D and Appendix E. The codes and datasets for reproducing our evaluation results are provided in the GitHub repository.
ACKNOWLEDGEMENTS
This work was supported by the National Science and Technology Innovation 2030 Major Program (2025ZD0544900) and sponsored by CCF-Lenovo Blue Ocean Research Fund.
REFERENCES
Ahmad-Montaser Awal, Guihuan Feng, Harold Mouchere, and Christian Viard-Gaudin. First exper- iments new online handwritten flowchart database. In Document Recognition and Retrieval XVIII, volume 7874, pp. 81-90. SPIE, 2011. 2, 5, 15
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding. local- ization, text reading, and beyond, 2023. URL https: //axxiv/ www.voorg/abbaa hta///bb//23000 12966. 15
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025.
URL https:tt////mmm /arxiviiiv org/ abss2502.... 13923.
Rumaan Bashir and Kaiser J. Giri. Diagram recognition: Domain knowledge based approach. In 2013 International Conference on Machine Intelligence and Research Advancement, pp. 445- 449, 2013. doi: 10.110//CMIRA.213..99
Shreyanshu Bhushan and Minho Lee. Block diagram-to-text: Understanding block diagram im- ages by generating natural language descriptors. In Yulan He, Heng Ji, Sujian Li, Yang Liu, and Chua-Hui Chang (eds.), Findings of the Association for Computational Linguistics: AACL- IJCNLP 2022, 153-168, Online only, November 2022. Association for Computational Lin- windinns-aacl.... 2022. findings----ii aacl. 15/. 5, 15
guistics. doi: 10.18653/v1/2022.
URL https: ptteps::/aclaanhhologoo /org/
Shreyanshu Bhushan, Eun-Soo Jung, and Minho Lee. Unveiling the power of integration: Block diagram summarization through local-global fusion. In Findings of the Association for Computa- fional Linguistics ACL 2024, pp. 13837-13856, 2024. 1,2
Martin Bresler, Daniel Pržša, and Václav Hlaváäž.
Online recognition of sketched arrow- connected diagrams. Inf. J. Doc. Anal. Recognit., (93))253-267, September 2016. ISSN URL https: //t::///oo..// ortg/0..1/0000001
1433-2833. doi: 00.1007/110032-06--0226 110032-016-001
5,
Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Car- los Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa De- hghani, Daniel Salz, Mario Lucic, Michael Tschannen, Arsha Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang. Ceslee Montgomery, Paulina Pietrzyk, Marvin Ritter, AJ Piergiovanni, Matthias Minderer, Filip Pavetic, Austin Waters, Gang Li, Ibrahim Alabdulmohsin, Lucas Beyer, Julien Amelot, Kenton Lee, Andreas Peter Steiner, Yang Li, Daniel Keysers, Anurag Arnab, Yuanzhong Xu, Keran Rong. Alexander Kolesnikov, Mojtaba Seyedhosseini, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut. Pali-x: On scaling up multilingual vision and language model, 2023. URL https: / arxiv. www.aaiww..wbbaa.eeeeaaaa www/aba/////e.ooaoooao....t....r /230. 65. 15
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the
11

---

Published as a conference paper at ICLR 2026
frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06661, 20200 6
Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. How abilities in large language mod- els are affected by supervised fine-tuning data composition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 177-198, Bangkok, Thailand, Au- gust 2024. Association for Computational Linguistics. doi: 10. 12118/8///222222ll longlong.1.. 12. URL bttps://aclanne
act..//////22...1ooo......... 12/.
Anwen Hu, Yaya Shi, Haiyang Xu, Jiabo Ye, Qinghao Ye, Ming Yan, Chenliang Li, Qi Qian, Ji Zhang, and Fei Huang.
Scientific diagram analysis with the multimodal large language model. In Proceedings of the 32nd ACM International Conference on Multime- dia, MM '24, pp. 6929-6938, New York, NY, USA, 2024. Association for Computing Machin- ery. ISBN 97984007068688, doi: 10.1145/3664644.3668 294. URL www.h:tttp/.omm /ww.i//oi/./// orgg /10. 1145/366461 3681294.
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images, 2016.
Vladimir Levenshtein. Binary codes capable correcting deletions, insertions, and reversals. So- viet physics. Doklady, 10:707-710, 1965. URL https: /api...p... www.nticsccooooooo...o..o /rrg http::////ap.ssssttsssssciiieeee
CorpusID: 60827152.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-
training with frozen image encoders and large language models, 2023. URL https: /arxiv.
arg/abs// 2301. 12597. 15
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URL wttp://wwwwwor//ooooooa/oeossose abs 2304 08485. 15
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee, Improved baselines with visual instruction tuning, 2024. URL https:///xxxii...aat/// org/ abs 22300 03744. 5
Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning, 2022. URL ptattps://axxv..../ httt:p://arxiiv.orr////////11/1
13214. 15
Fanya S. Montalvo. Diagram Understanding pp. 5-27. Springer US, Boston, MA, 1990. ISBN 978- 1-4613-056--9. doi:
URL htt.lattts///ci..oo/o//////
OpenAl. Hello gpt-4, 2024. URL http://ppaaaaaaemmmeeeeeeeelllee
6
Huitong Pan, Qi Zhang, Cornelia Caragea, Eduard Dragut, and Longin Jan Latecki. Flowlearn:
Evaluating large vision-language models on flowchart understanding, 2024. URL https://
arxiv.org// abs/a/bs//4400 05183. 2, 15
Bernhard Schäfer, Margret Keuper, and Heiner Stuckenschmidt. Arrow r-cnn for handwritten dia- gram recognition. International Journal on Document Analysis and Recognition (IJDAR), 24(1): 3-17, 2021a. 15
Bernhard Schäfer, Han van der Aa, Henrik Leopold, and Heiner Stuckenschmidt. Sketch2bpmn: Automatic recognition of hand-drawn bpmn models. Marcello La Rosa, Shazia Sadiq, and Ernest Teniente (eds.), Advanced Information Systems Engineering, pp. 344-360, Cham, b021 Springer International Publishing. ISBN 978-3-030-79382-1. 2, 15
Shubhankar Singh, Purvi Chaurasia, Yerram Varun, Pranshu Pandya, Vatsal Gupta, Vivek Gupta, and Dan Roth. Flowvqa: Mapping multimodal logic in visual question answering with flowcharts, 2024. URL https: /wwxiv..vvv.. wwwiiioorrrrgeeeee /bbs//////// /aala/ba/44000 6 9237. 2, 15
12

---

Published as a conference paper at ICLR 2026
Lianshan Sun, Hanchao Du, and Tao Hou. Fr-detr: End-to-end flowchart recognition with precision and robustness. IEEE Access, 10:64292 64301, 2022. doi: D0.100/AACCESSS222......80 2, 15
Simon Tannert, Marcelo G. Feighelstein, Jasmina Bogojeska, Joseph Shtok, Assaf Arbelle, Peter W. J. Staar, Anika Schumann, Jonas Kuhn, and Leonid Karlinsky. FlowchartQA: The first large- scale benchmark for reasoning over flowcharts. In Piush Aggarwal, [["))zge (au()e(c))ann Carina Silberer, Sina Z sss) and Torsten Zesch (eds.), Proceedings of the 1st Work- shop on Linguistic Insights from and for Multimodal Language Processing, pp. 34-46, Ingol- stadt, Germany, September 2023. Association for Computational Lingustics. URL https: / //aclanthology wwg/2023. 1imo- 1. 1.5.1.1.... 2, 15
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Cas- bon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xi- aohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Cole- man, Yi Gao, Basil Mustafa, lain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abhcesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi. András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivaku- mar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eu- gene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron. Hadi Hashemi, Hanna Klimezak-Plucinske Harman Singh. Harsh Mchta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wi- eting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma. Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Ni- lay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Ruben- stein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Pöder. Sijal Bhatnagar, Sindhu Raghuram Panyam. Sivan Eiger. Susan Zhang. Tianqi Liu, Trevor Yacovone, Tyler Liechty. Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero. Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Bar- ral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean-Baptia Baptiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma technical report, 2025a. URL https: / a/////axi//// rxiv. org/ abs 2503 1 19786. 6
GLM-V Team, :, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi. Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang. Ziyang Pan, Aohan Zeng, Baoxu Wang, Bin Chen, Boyan Shi, Changyu Pang, Chenhui Zhang, Da Yin, Fan Yang, Guoqing Chen, Jiazheng Xu, Jiale Zhu, Jiali Chen, Jing Chen, Jinhao Chen, Jinghao Lin, Jinjiang Wang, Junjie Chen, Leqi Lei, Letian Gong, Leyi Pan, Mingdao Liu, Mingde Xu, Mingzhi Zhang, Qinkai Zheng, Sheng Yang, Shi Zhong, Shiyu Huang. Shuyuan Zhao, Siyan Xue, Shangqin Tu, Shengbiao Meng. Tianshu Zhang, Tianwei Luo. Tianxiang Hao, Tianyu Tong, Wenkai Li, Wei Jia, Xiao Liu, Xiaohan Zhang, Xin Lyu, Xinyue Fan, Xuancheng Huang, Yanling Wang, Yadong Xue, Yanfeng Wang, Yanzi Wang, Yifan An, Yifan Du, Yiming Shi, Yiheng Huang, Yilin Niu, Yuan Wang, Yuanchang Yue, Yuchen
13

---

Published as a conference paper at ICLR 2026
Li, Yutao Zhang, Yuting Wang, Yu Wang, Yuxuan Zhang, Zhao Xue, Zhenyu Hou, Zhengxiao Du, Zihan Wang, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Minlic Huang, Yuxiao Dong, and Jie Tang. Glm-4.5v and fon-44.v-thinknng. Towards versatile multimodal reasoning with scalable reinforcement learning, 2025b.
Bintu G. Vasudevan, Sorawish Dhanapanichkul, and Rajesh Balakrishnan. Flowchart knowledge extraction on image processing. 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 40754444400 4082, 2008. URL https:
/ //api. semant www.coolw....coooooopoooeeceeeee 124499911.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm your phone. arXiv preprint arXiv:2408.01800 000000000.00,,,, 2024. 5
Junyi Ye, Ankan Dash, Wenpeng Yin, and Guiling Wang. Beyond end-to-end vlms: Leveraging intermediate text representations for superior flowchart understanding, 2024. URL https://
arxiv. orgo abs 2412 16420. 1
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang. Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of CVPR, 2024. 20
Yilun Zhao, Chengye Wang, Chuhan Li, and Arman Cohan. Can multimodal foundation models understand schematic diagrams? an empirical study on information-sseking qa over scientific papers. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 18598- 18631, Vienna, Austria, July 2025. Association for Computational Linguistics. URL https: //aclanthology crg/2025 findings-acl. 9
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan Hc, Tan Jiang, Jiapeng Luo, Yi Wang, Conghui Hc, Botian Shi, Xingcheng Zhang, Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu, Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. In- ternv13: Exploring advanced training and test-time recipes for open-source multimodal models, 2025. URL https: / arxiv. org. abs aabs/2500 2504.0000 11.00.1040400 1
14

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
As indicated in the submission form, we answer "Large Language Models: Yes, to aid or polish writing." LLMs were only used for language refinement, such polishing wording, improving clarity, and ensuring consistency. They were not involved in research ideation, methodology design, or experimental analysis. All scientific contributions and results are solely due to the authors.
B RELATED WORK
B. EXISTING DATASETS
Research on flowchart understanding has traditionally relied fixed datasets rather than control- lable synthesizers. Existing datasets such as IconQA (Lu et al., 2022), FlowchartQA (Tannert et al., 2023), FlowVQA (Singh et al., 2024), and FlowLearn (Pan et al., 2024) provide important bench- marks for evaluating models on tasks ranging from flowchart parsing to flowchart QA. Other datasets such as CBD (Bhushan Lee, 2022). FC_A (Awal al., 2011), FC_B (Bresler et al., 2016), and hdBPMN (Schäfer et al., 2021b) provide domain-specific diagram test resources. However, these resources share several limitations. First, they lack controllability: users cannot systematically vary structural properties such node count, edge density, or nesting depth. Second, stylistic diver- sity is limited, since most datasets rely on a single renderer or annotation style, whereas real-world flowcharts come from heterogeneous sources. Finally, most existing datasets designed for eval- uation only, and do not support large-scale, controllable synthesis for training purposes. To date, there no general-purport flowehart synthesizers that flexibly generate both training and test sets with controllable complexity and rendering diversity, leaving an important gap in the field. Ta- ble provides systematic comparison between our proposed synthesizer and existing datasets. Our FlowGen is unique in combining controllable structural features, multi-renderer diversity, and scalability both training and test, filling a critical gap in the field.
B.2 EXISTING METHOOS
Alongside datasets, range of methods have been explored for flowchart-hased visual reasoning. Early approaches adopt a deteecioon-nnd-asscciin and-association paradigm, such as Arrow R-CNN (Schäfer et al., 2021a) and FR-DETR (Sun al., 2022), which first localize symbols and arrows before recon- structing the underlying graph. While these pipelines can handle relatively simple diagrams, they typically depend on handcrafted heuristics and degrade under high structural complexity stylistic variation. More recent work has shifted toward end-to-end VLMs and MLLMs, including BLIP- 2 (Li et al., 2023), LLaVA (Liu et al., 2023), PaLI-X (Chen al., 2023), and Qwen-VL. (Bai al., 2023). These models integrate perception and reasoning within a unified framework, achieving strong performance on natural-image VQA tasks. However, they remain limited in structural pars- ing of flowcharts, where precise recovery of hierarchical and relational information is required. In this work, we focus exclusively on evaluating representative MI..Ms. By systematically analyzing their performance on controlled test datasets, we aim to reveal their strengths, limitations, and the specific challenges posed by flowchart understanding.
C APPLICATION DOMAINS AND SEMANTIC VOCABULARIES
This appendix provides additional details on semantic annotation in the construction of FlowGen. Specifically, FlowGen covers 120 distinct application domains, spanning a wide range of real-world scenarios. For each domain, we curate vocabulary consisting of 40 node names and 40 edge labels, which are initially generated using GPT-4o. These domain -eenaff vocabularies are used to annotate graph nodes and edges, ensuring semantic cohesiveness within graphs and domain diversity across graphs. Representative examples from several domains are presented below. The complete lists are available in /Synthesizer/my dictionary.py in our GitHub repository.
15

---

Published as a conference paper at ICLR 2026
Application Domains and Semantic Vocabularies
APPLICATIONS
'Finance', 'Healthcare', 'Manufacturing', "dducation', 'E_commerce', 'Transportation', 'Energy', "Legal', 'Logistics', 'Telecom', 'Agriculture', 'Tourism', Real Estate', 'Public Safety', 'EEtertainment',
Environment', "'overmmentt,,.. 'Retail', 'Automotive', Acrospace', 'Insurance', 'cybersecurity', 'Smart Heme', Smart City', "Biotechnology', 'Pharnaccuticals', 'Rebetics', Construction', "Media', 'Sports', 'Fashion', 'Human Resources',
NODE NAMES
'Finance':
Accouutt Verification', 'Credit Scoring', Approval Decision', 'Fraud Check', , 'Transaction Audit', Investment Planning', 'Financial Advisory',
:ealthcare':
'aatient Intake', 'Lab Test', 'Diagnosis', Prescription', Medical History', 'Health Screening', 'Allergy Test', Genetic Counseling', 'Pain Management', 'Patient Transfer', },
EDGE LABELS
'Finance's
'verifies', 'approves', 'rejects', 'checks', cclccllatss', 'recommenás', ccnsuuls', 'plans', aallocates', 'monitors', 'medels', 'settles',
Heal thcare' [
diagnoses', 'examines', 'tests', 'treats', 'scans', mmonitors', 'records', 'administers', 'researches', "prevents', 'educates', 'surveys', 'immunizes',
},
D IMPLEMENTATION DETAILS
D.1 SUMMARY OF EVALUATION DATASETS
Table summarizes the public datasets used in our evaluation experiments, providing their task, size, source, and license facilitate reproducibility and transparency.
D.2 MODEL VERSIONS AND SOURCES
Table summarizes the proprietary and open-source MLLMs evaluated in this study, organized by provider with release dates and official access sources. Open-source models are linked to their Hugging Face repositories, while proprietary models are accessed through their respective APIs.
16

---

Published as a conference paper at ICLR 2026
Table 8: Summary of evaluation datasets used in experiments.
Dataset
Task
Size
Source
License
Flow
Parsing & QA
956
Synthetic flowcharts
MIT License
CBD
Parsing
96
Computerized block diagrams
Rescarch-only
FC_A
Parsing
145
Hand-written flowcharts
CC-BY-NC-SA
FC_B
Parsing
196
Hand-written flowcharts
NA
hdBPMN
Parsing
Hand-drawn BPMN diagrams
CC-BY-4.0
Feaw
Parsing & QA
2,000
Scientific literature & Synthetic
CC-BY-NC-SA-4.0
AI2D
QA
4,060
Scientific Diagrams
CC BY-SA
MISS-QA
QA
173
Schematic Diagrams
CC-BY-4.0
Table 9: Evaluated proprietary and open-source MLLMs.
Provider
Model
Releuse
Source
Proprictury Multimodal Large Language Models
OpenAl Google
GPT-4o
2024-08
OpenAl AP1
Gemini-2 F-Faaa
2025-04
Google Al Stadio API
Zhipo Al
GLM-4V-Plus
2024-08
Zhipa
Open-Source Multmodul Large Language Models
Alibaba
Qwen2.5-VL-38
2025-01
https://hugg www.gigggggee o wttp:///////eeooot hte////e////eeeeeeeeeeeeeeeeeeee Sevee2.5-VEL --VL-3B- 38-Instruct
SLL-2
2025-01
huipst /bugg
ww/Qeenn/Qwnn2
VL-7B- Instruct.
OpenGVLLah
IntemV113-28
hct //hugg Caiee, o0/ wtt.://///pee..e
Mp
MiniCPM-VV2 V2.6.8B
2025.04
hetpa: //hugg/ faace........................... baade.coo ooppppmm./iiinnnnmp CPX-V-2__
LLaVA
LLaVA-vI 6-Mistral-7B
2024-12
but Lin Cace httllllaaat/lllaaa//
Google
Gemmañ-44
2025.03
httpp: //hugg/ ng ta.e. ot/../o///e ww/.gogg//////o/o/oooo/o//oo.o/o
hatpl////gmmaa//eeeeeeee/ 3-4b-1t
Germa3- 12B
2025.03
httpa://hugginge
haagggll//nmmaaeeeee 3-12b-it
D.3 FINE-TUNING HYPERPARAMETERS
For open-source models, we perform supervised fine-tuning (SFT) on FlowGen-synthesized train- ing set using the configuration shown in Table 10. The training employs LoRA adapters on both language and vision components, with other modules frozen for efficiency.
Table 10: Hyperparameter configuration for SFT on FlowGen-syntthsizeed training set.
Hyperparameter
Value
Framework
PyTorch 2.6.0 DeepSpeed (ZcRO-3)
Precision
BF16 (FP16 disabled)
Gradient checkpointing
Enabled
Optimizer
AdamW (8) = 0.9, 32 0.999, weight decay=0.1)
Learning rale
5 10
(cosine scheduler)
Warmup ratio
0.03
Global batch size
32
Per-device batch size Gradient accumulation steps
4
Epochs
LoRA rank / alpha/ / dropout
64 / 64 / 0.05
Vision tower
Frozen (with vision LoRA enabled)
LLM backbone
Frozen
Merger
Frozen
Image resolution range
256 x 28 x 28 1280 x 28 x 28
This configuration ensures efficient training with limited GPU resources while still adapting both textual and visual representations through LoRA.
17

---

Published as a conference paper at ICLR 2026
E
PROMPT TEMPLATES
Prompt Template for Flowchart Parsing
[System Input]
You are an expert assistant specialized in flowchart understanding.
Your task is to parse flowchart images into structured knowledge triples that capture both topology and semantics.
[User Input]
{image input: flowchart)
(text imput: <image>>Pleass describe all the information in the flowchart image the form of triples:
For each arrow labeled with X directed from node A to node B, output triple: <A, X, B>; For each unlabeled arrow directed from node A to node B, output triple: <A, connectedTo, B>;; For each node fully inside node group output triple <A, partff, B>.
The following triples are example outputs:
<Food Packaging Improvement, secures, Quantum Computing Integration
<Graphene Production, catalyzes, Nanoscnsors>>
<Graphene Production, partOf, Drug Delivery Systems>
<Nanocomposites, connectedToo, Spectroscopy>>
Nanodevice Fabrication, connectedTo, Graphene Production
Nanosensors, connectedTo, Nanocomposits>>
Spectroseopy, educates, Nanodevice Fabrication>>)
An Example Prompt for Answering Flowchart-based Yes/No Questions
[System Input]
You are expert assistant specialized in flowchart understanding. Your task is to parse flowchart images into structured knowledge triples that capture both topology semantics,
[User Input]
(image input: flowchart)
(text input:
Determine whether the following description of the image correct (just answer
Yes or No)
"Consistent failures through this type of intervention constitute challenging group for the perception model as seen on right." connectedto "Perception Model is used discovery of failures at the scene level."
The following description, extracted in the form of triples from the image, provided for reference and may contain errors. <BERT, connectedTo, Coca-Coo Truck>
<Coca-Cola Truck, connectedTo, Perception Model>
<Coca-Cola Truck, connectedTo, Perception Model
<Police Car. partOf, Interventions>>
Police Car, partOf, Scene Understanding>> Scene Understanding, connectedTo, Police C
The answer is:
18

---

Published as a conference paper at ICLR 2026
Example Prompt for Answering Flowchart-based Multiple-Choice Questions
[System Input]
You are an expert assistant specialized in flowchart understanding. Your task is to parse flowchart images into structured knowledge triples that capture both topology and semantics.
[User Input]
{image input: flowchart)
(text input: <image>>leant Please answer the following multiple-choice question about the image:
The sexual asexual process by which organisms generate new individuals of the same kind called?
(A).eerrouuction
(B).polination
(C).growth
(D).gemiination
following description, extracted the form triples from the image, provided reference and may contain errors.
<Fertilization, connectedTo, Green Bean>> <Growth, connectedTo, Germination>> <Growt,, connectedTo, Reproduction
<Reprddutionn, connectedTo, Pollination> Reproduuction, connectedToo, Growth> GGeen Bean, connectedTo, Germination>>
The answer is: }
Prompt for Topic and Node/Edge Names Generation
[System Input]
You tasked with helping build synthetic flowchart dataset. Please generate 120 distinct
application domains (topics).
each topic:
(1) Provide the topic name.
(2) List 40 concise node names that could appear in flowcharts within this topic.
(3) List 40 concise edge names that describe interactions or relationships among the nodes.
Constraints:
Use short phrases (1-4 words) for node and edge names.
Avoid redundancy across different topics.
Ensure that node names are realistic the chosen topic.
Ensure that edge names reflect meaningful relationships.
Output in structured JSON format:
(GPTT40 Output]
("Topic": "Finance".
"Nodes names": ['Account Verification". "Credit Scoring", "Risk Assessment". ...].
"edges names": ["verifies", "approves", "budgets",
"Topic": "Healtheare".
"Node names": ["Patient Intake", "Medical History", "Emergency Response".....
"Edge names": ("diagnoses", "treats", "examines". }.
19

---

Published as a conference paper at ICLR 2026
F EXPERIMENTS ON MMMU
Recent studies have shown that the composition of SFT data plays a critical role determining the resulting capabilities of multimodal models. particular, Dong et al. (2024) demonstrate that naively mixing heterogeneous tasks often leads to optimization conflicts, while sequential fine- tuning can induce catastrophic forgetting. Motivated by these findings, and following their inter- leaved training paradigm, we investigate whether fine-tuning on FlowGen transfers beyond flowchart parsing to broader multimodal reasoning.
Specifically, we examine whether incorporating FlowGien-syttheszzed flowchart parsing tasks dur- ing SFT improves performance on MMMU (Yue et al., 2024), a diverse benchmark covering general- domain multimodal reasoning tasks that are not directly related to flowcharts. This experiment aims to assess whether FlowGen only induces task-specific gains, or it promotes more generalizable struc- tural reasoning abilities.
Since MMMU does not provide an official training split, we follow common practice and use its 900 validation samples for SFT and 150 development samples as test cases. For each model, consider two SFT settings:
Fine-tuning solely on MMMU samples.
Fine-tuning equal-sized mixed dataset, where 50% of MMMU samples replaced with -essyyeeyyteeee flowchart parsing samples. We adopt an interleaved training strategy the batch level, ensuring that FlowGen and MMMU samples are jointl opti- mized throughout training.
Table I1: Performance comparison (accuracy) of MLLMs MMMU. We compare each model with its variants fine-tuned on the original training set (+ SFT on the original training set) and on mixture of the original training set and FlowGen (+ SFT mixed training set).
Training Variant
Qwen2.5-VL-3B
Qwen2.5-VL-7B
MiniCPM-V2.6-8B
Base Model
53.3
58.6
49.4
+ SFT on the original training set
59.3
62.6
53.3
+ SFT on mixed training set
60.2
65.3
54.0
As shown in Table 11, models fine-tuned with the mixed dataset consistently outperform their MMMU-only counterparts on the MMMU benchmark. Notably, these improvements are observed despite replacing half of the MMMU supervision with FlowGen data, indicating that FlowGen does not dilute task-relevant learning but instead provides complementary signals.
We attribute this behavior to the structural nature of FlowGen supervision. Feo emphasizes plicit modeling of entities, relations, and spatial layouts through triplet extraction, which encourages the model learn transferable representations of graph structure and visual organization. Such in- ductive biases are broadly applicable across multimodal reasoning tasks, including those MMMU that require compositional understanding, spatial grounding, multi-step inference.
20

---

Published as a conference paper at ICLR 2026
G ABLATION ON SEMANTIC ANNOTATION
To further isolate the contribution of structural information flowchart parsing, we conduct an additional ablation in which all node names and edge labels replaced with random meaningless strings. Specifically, node names are substituted with random strings starting with uppercase letters, while edge labels are replaced with random lowercase strings of length 5-12. This preserves minimal formatting conventions while eliminating meaningful semantics. This experiment evaluates whether FlowGen-traine models can still leverage structural cues when textual semantics are removed.
Table 12: Ablation results (strict F1) of FlowGen using semantic annotation (+ SFT on Full Flow- Gen) random strings (+ SFT FlowGen w/o Semantic Annotation) for node names and edge labels.
Model
Flow VQA
CBD
FC A
FC
hdBPMN
FlowLearn
Qwen2.5-VL-3B
12.7
2.8
8.6
2.3
20.6
SFT Full FlowGen
51.3
49.8
22,4
34.6
8.4
41.6
SFT on FlowGen w/o Semantic Annotation
50.9
44.1
13.6
24.1
5.9
39.5
Qwen2.5-VLa
59.4
49.1
25.8
32.0
16.6
43.2
SFT Full FlowGen
70.9
55,4
29.5
40.8
18.3
60.1
+ SFT on FlowGen w/o Semantic Annotation
64.5
52.9
28.4
34.8
17.4
56.0
MiniCPM-V2.6-25
17.6
20.4
7.5
8.8
3,4
9.1
+ SFT on Full FlowGen
44.0
37.9
13.4
22,9
3.6
20.0
+ SFT on FlowGen w/o Semantic Annotation
35.2
29.8
9.7
13.9
3.6
19.1
Table 12 reports the results. Compared to models trained with FlowGen's original random but se- mantically meaningful labels, performance with meaningless labels shows moderate degradation. Importantly, the performance gains SFT remain consistent and significant across different models and datasets. These results indicate that the benefits from using FlowGen primarily driven by learning structural patterns, topological relations, and spatial layouts from flowcharts, rather than relying solely textual semantics.
21

---

Published as a conference paper at ICLR 2026
H ERROR ANALYSIS ON PUBLIC DATASETS
H.1 COMPLEX NESTING ERROR PUBLIC DATASET
An Example of Complex Nesting Error on hdBMPN Dataset
[System Input]
You are an expert assistant specialized in flowchart understanding. Your task is to parse flowehart images into structured knowledge triples that capture both topology and semantics.
[User Input]
(image input: flowchart)
(text input: <image>>Plaase describe the information the flowchart image the form of triples: For each arrow labeled with X directed from node A to node B, output triple: <A, X, B>; For each unlabeled arrow directed from node node B, output triple: <A, connectedTo, B>; each node fully inside node group output triple <A, partOf, B
The following triples are example outputs:
<Food Packaging Improvement, secures, Quantum Computing Integration>>
<Grahhnne Production. catalyzes, Nanosensors>
<Graphene Production, partOf, Drug Delivery Systems>
Nannocomposites, connectedToo, Spectroscopy>>
Nanodevice Fabrication, connectedToo, Graphene Production
Nanosensors, connectedToo, Nanocomposites>
Spectroscopy, educates, Nanodevice Fabrication>>
Qwen2.5-VL-7B Output:
<Claim Handling Process, connectedTo, End Claim Payers>
<Claim Handling Process, mark as Okay, Write Settlement Recommendation>
<Check Settlement Recommendation, connectedTo,, Claim Handling Process>>
<Customer Insurance Claim, connectedTo, Senior Claim Officer>
<Customer Insurance Claim, connectedTo, Write Settlement Recommendation>>
<Examine Claim, connected Check Settlement Recommendation>>
Examine Claim, TonnectedTo, Write Settlement Recommendation>
End Claim Payers, connectedTo, Claim Handling Process>
Rewrites Claim, connected Examine Claim>
Senior Claim Officer, claim receives, Rewrites Claim>>
Write Settlement Recommendation. connectedTo, Check Settlement Recommendation
Error Analysis:
Qwen2.5-VL--BB's prediction fails to capture the hierarchical containment between roles and tasks. In the gold standard, operations such as "register claim". "examine claim", and "write settlement recommenda- tion" explicitly nested under "claim officer" and further within "insurance". The model instead flattens this structure, inventing entities like "End Claim Payers" producing spurious links (e.g.. (Senior Claim Officer, claim receives, Rewrites Claim)), while omitting key partOf relations. This collapse of nested dependencies leaves the explanation structurally inconsistent and semantically incomplete.
22

---

Published as a conference paper at ICLR 2026
H.2
EDGE AMBIGUITY ERROR PUBLIC DATASET
Example of Edge Ambiguity Error on Flow VQA Dataset
[System Input]
Configured Appendix
[User Input]
Configured Appendix
Qwen2.5-VL-77B Output:
<Begin the first loop iteration for connectedTo, Call the Hto__Note function with the current and с arguments>
<Begin the first loop iteration x, connectedTo, Initialize the second nested loop
<Call the Ntoe function with the current x and as arguments, Input x, c, Execute stop_Note function using and <>>
<Execute the stop Note function using and Output, End the first loop iteration for e>>
<End, No. For each value of from 0 117>
<End the loop iteration
c, connectedTo, For each value of e from to 15> <End the first loop iteration for x, connectedTo, For each value of from 0 to 15>
<oor each value of e from 0 to 15, Yes, Begin the first loop iteration for c>>
<For each value of from 0 to 117, Yes, Begin the first loop iteration for x>>
<niitialize nested loop structure, connectedTo, For each value x from to 117>
Initialize second nested loop, connectedToo, each value c from to 15>
Start, connectedTo, Initialize nested loop structure
Error Analysis:
Swen2.5-VL-7B- prediction suffers from edge ambiguity, misrepresenting the dircction and semantics of key transitions in the flow loop. In the gold standard, control flow edges clearly encode how each loop iteration branches back to its condition (e.g.. (End first loop iteration for c, connectedTo, For each value с from to 15)) and how termination is handled (e.g.. (For each value of x from 0 to 117, No, End)). The model instead reverses or distorts these relations, producing edges such as (End, No, For each value of x from 0 117) and (End the first loop iteration for connectedTo, For each value of from 0 15), which invert intended logic. While many node labels are preserved, the predicted edges fail capture whether they represent continuation, termination, or loop-back connections, resulting structurally inconsistent flow and an inaccurate representation of the looping process.
23

---

Published as a conference paper at ICLR 2026
H.3
OCR DRIVEN ERROR ON PUBLIC DATASET
An Example of -CR-Driven Error hdBPMN
Sdvaniie 134 #5/ Rnd
11sk
Prquest rerived
Stsple rish Cc1e5sn ead.
in
orgpe di 100
Corten
Cold (r
[System Input]
Configured Appendix H.1
[User Input]
Configured Appendix H.I1
Qwen2.5-VI-77B Output:
Advanced Risk Assessment, connectedTo, Simple Risk Assessment>>
<Assess Risk, above threshold, Advanced Risk Assessment>
AAsesss Risk. below threshold. Simple Risk Assessment>
Advanced risk asessmend, connectedToo, Intram customer>>
<Customer Notification, connected'oo, Urgent Wire Dishursement>>
<Simple Risk Assessment, connected'oo, Customer Notification
UUrgent Wire Disbursement, connectedTo, End Procesy>
<End Procesy, connectedTo, Customer Notification>
Request Received, connectedTo, Assess Risk>>
Error Analysis:
Qwen2.5-VL-7BBss outputs exhibit typical OCR-driven emrrs: misrecognized text such "intram tomer" (likely intended as "inform customer") and "End Procesy" (likely "End Process") results in incor- rect node labels like "Customer Notification" and "Urgent Wire Disbursement" that deviate from the source diagram. Consequently, the model generates spurious edges (c.g.. (End Procesy, connectedTo, Customer Notification)) while omitting valid dependencies, such the direct sequence from "Inform customer" "End Process" implied by the original workflow. These DCR-induced distortions compromise both the semantics and topology into structural parsing errors.
predicted graph, illustrating how recognition noise propagates directly
SCANNED STYLE CONFIGURATIONS
Table 13 summarizes the configuration used synthesize scanned-style flowcharts with three diffi- culty levels.
Table 13: Configurations for scanned-style flowcharts generation with three difficulty levels.
Difficulty
Blur radius
Vignette
Rotation
Perspective
Color
Noise
JPEG quality
Easy
0.8
0.1
1.0º
No
Yellowish
0.005
90
Medium
1.0
0.3
1.2º
No
Yellowish
0.008
70
Hard
1.2
0.5
1.5
Yes
Gray
0.015
40
24

---

Published as a conference paper at ICLR 2026
J ANALYSIS OF INTRA-TOPIC FLOWCHART DIVERSITY
To quantitatively assess the diversity of graph structures synthesized under the same topic, mea- sure the structural dissimilarity between flowcharts using graph distance (GED). GED is defined as the minimum number of elementary graph operations, including node or edge insertion, deletion, and substitution, required to transform one graph into another.
edit
Table 14:
Mean graph edit distance (GED) with standard deviation (SD) over intra-topic graphs constructed by FlowGen.
Mean GED
SD
Within the same difficulty level
Easy
18.0
2.2
Medium
28.5
4.1
Hard
36.8
6.4
Between difficultyy levels
Easy-Medium
38.0
5.1
Easy-Hard
76.8
8.9
Medium-Han Hard
48.5
7.2
Table 14 reports the mean GED with standard deviation (SD) between flowcharts from the same topic, evaluated both within the same difficulty level and between different difficulty levels. The results show substantial structural variation among intra-topic flowcharts: mean GED values range from 18.0 to 36.8 for flowcharts of the same difficulty level, and further increase to 76.8 when comparing flowcharts across difficulty levels. These findings demonstrate that FlowGen produces highly diverse flowchart structures even within single topic, supporting the diversity and challenge of the test subsets.
K ERROR ANALYSIS ON FLOWGEN TEST SET
We further analyze 50 failure cases on FlowGen's test sets, which incorporates renderer diversity and simulated scanning artifacts. Note that a single failure case may exhibit multiple error types, so the percentages reported below can sum to more than 100%. Four major eтror types emerge.
(1) Cross-renderer generalization (48%); Visual conventions differ substantially across renderers (e.g., PlantUML vs. Mermaid). As result, models often fail to generalize across styles, misinter- preting arrowheads, connector boundaries, or nesting layouts. Such renderer-specific artifacts cause structural discrepancies cases are shown in Appendix K.I.
when the underlying abstract topology is identical. Representative
(2) Complex nesting (38%) % ): Similar to the errors observed on other benchmarks, models frequently mishandle deeply nested partOf relations. Examples are provided in Appendix K.2.
(3) Edge ambiguity (54%): When multiple edges overlap connectors visually congested, models still confuse edge directionality or labels. These cases resemble the ambiguity observed in Section 3.3.3 but are further exacerbated by renderer-induced variations. Examples are given in Appendix K.3.
(4) OCR-driven failures (66 %) In scanned test subsets, text recognition errors often corrupt node labels or generate extra nodes that do not exist in the original flowchart. Examples are provided in Appendix K.4.
25

---

Published as a conference paper at ICLR 2026
K.1
ERROR ON FLOWGEN TEST SET
An Example of Cross-Rendera Error on FlowGin Test Set
[System Input]
Graph Medium From PlantUML
Graph Medium From Mermaid
[System Input]
Configured Appendix
[User Input]
Configured Appendix H.1
Qwen2.5-VI-77B For PlantUML Flowchart Output:
<Behavior Change Support, connectedToo, Holistic Hcalth <asessment>><<Bhhavve Change Support, protects, Social Connection Enhancement>> < Detox Programs, assesses. Emotional Well-being><<Det Programs, eats, Gut Health for Wellness> Detox Programs, moves, Environmental Wellness>> Wellness><<Deto <etoox Programs, stress-reduces, Physical Activity Planning><Digital Detox, connectedTo, Resilience Build- ing><Digita Detox, optimizes, Social Connection Eahansementt><<Dggiin <Digital Detox, promotes, Holistic Health Assesmment><Diiitall Detox, tests, Physical Activity Taaniing><Emmttionn Well-being, connect- edTo, Gratitude Practice>>< <eseriie>><EEiirommment Wellness, connectedTo,, Nature Connection <errriinn>><Eiiiimmeenen Wellness, partff, Health Coaching><<Gu Health for Wellness, certifies, Work-Life Balance Coach- ing><<Gut Health for Wellness, coaches, Physical Activity Planning><<Holistee Health Assessment, connectedTo, Behavior Change Support> Hcalth Assessment, connectedToo, Social Connec- tion Fnhancement>><<Meeee
Coaching, connectedTo, Work-Life Balance Coaching><<Mediattin Coaching, motivates, Holistic Health Assessment> Massage Therapy. connectedTo, Gratitude Prac- tice <assage Therapy. partOf. Health Coaching Nature Connection, connectedTo, Environmental Wellness><<Nature Connection, partOf, Health Conching>~<Ptysscal Activity Planning. connectedTo. Digital Detox>><PPysical Activity Planning, connectedTo, Work-Life Balance Coaching><<Physiccl Ac- tivity Planning, cats, Detox Programs><<Resilienss
Building, complics, Quantum Computing Integra- tion><Resillienee Building, connectedTo, Digital Detox><<Reilienn Building, connectedTo, Holistic Health Assessment> -eeeinennt--RRiileeeeee Building, connected Meditation Staching>><Resilenne Building, connectedTo, Physical Activity Plamning><<Resllience Building, personalizes. Physical Activity Plan- ning><Resiliincee Building, researches. Gratitude Practice><<Rsilieeee Building, supports, Holistic Health <reessment><<Socinn Connection Enhancement, balances, Holistic Health > Connection Enhancement, connectedTo, Holistic Health <ssessment><<Sciinn Connection Enhance- optimizes. Digital Detox>>< Social Connection Enhancement, protects, Behavior Change Sup- Port>><Sccial Connection Enhancement, recommends, Bchavior Change Support> Lfok- Balance Coaching, connectedTo, Mcditation Coaching><<orkkll Balance Coaching, connected'To, Physical Ac- Branninn><WWrr--LLne Work-L Balance Coaching, connects, Detox Programs>> < <ookk-L.iff Balance Coach- ing. secures. Physical Activity Planning>
tivity Planning>><
Qwen2.5-VL-7B For Mermaid Flowchart Output: <Aromatherapy Usage, advises, Burnout Pre- vention> < Aromatherapy Usage, connectedloo, Alternative Therapies>- < Aromatherapy Usage, partOf,
26

---

Published as a conference paper at ICLR 2026
Resilience Building,> < Alternative Therapies, connectedTo, Aromatherapy Usage>> <lteget><Altemaaiivv Ther- apies, partOf, Resilience <eilding><<Bhaviorrr Change Support, connected[o, Personal Growth Plan- ning><<Rehavior Change Support, connected Physical Activity Plamingg><Burrout Prevention, connectedTo, Detox Programs><<Bume <Bumout Prevention, partOf, Resilience Building><DDttxx Programs, connectedTo, Burnout Trevention>><<ettoo Programs, partOr, Resilience Building>><Emmlonn Well- ness Programs, complies, Behavior Change Support><<Emplony Wellness Programs, eats, Health Coaching>>< Entiiommttall Wellness, connectedTo, Holistic Health Assessment> < <Envirrommenaal Well- ness, connectedTo, Mindfulness Practice><<Environmeer Wellness, connectedTo, Physical Activity Planning><<Enirrome
Wellness, sleeps, Physical Activity Planning><<Heltth Coaching, connect- edTo, Detox <rograms><<Helle Coaching, connectedTo, Envirommental Wellness><<Hellth Coach- ing, connectedTo, Personal Growth <aaning>><<ealle Coaching, eats. Employee Wellness Pro- grams>-<Holistic Health Assessment, recommends, Personal Growth Tlaninn>><Jouraaling for Well- ness, connectedTo, Mindfulness Practice>< <Journaiinn for Wellness, connectedoo, Personal Growth Plan- ning><Journaling for Wellness, connectedTo, Physical Activity Planning> <aaniing><Jorralligg for Wellness, for Wellness, optimizes, Massage Therapy Therapy><Jourrrr ><Jonnaalnng for enhances, Massage Environmental Wellness> MMnnduunss Practice, connectedTo, Environmental Wellness, stress-reduces, Melnnues>sMinnfuunness Practice, educates, Speetroscopy>><<asaapg Therapy, connectedTo, Personal Growth Baamingg><MMassaee Therapy, engages, Health Coaching><Massagee Therapy, meditates, Envi- ronmental <hellness><<Masaee Therapy, mindfulness, Personal Growth P Therapy, personalizes, Alternative Therapies> <errapie>><<eeronaall
<Personal Growth Planning, connectedTo, Behavior Change Sup- port> <Persnnlll Growth Planning, connectedTo. Holistic Health Assessment>> <Personal Growth Plan- ning, connectedTo, Physical Activity Planning><Personal
<Personal Growth Planning, recommends, Holistic Health Assessment><<Phyyicee Activity Planning, connectedTo, Environmental Wellness> Wellness>><Physical Activity Planning. rescarches, Holistic Health Assessment><<hhyicee Activity Planning, sleeps, Environmental Wellness>"
Error Analysis:
Qwen2.5-VI--78 exhibits inconsistent predictions across different renderers for the same underlying topic. For example, in the PlantUML rendering, the otaddataddard graph emphasizes role-task relations such (Detox Programs, cats, Gut Health for Wellness) and (Social Connection Enhancement, protects, Holistic Health Assessment). However, the model output drifts into spurious links like (Resilience Building, com- plies, Quantum Computing Integration), which are entirely unrelated. In contrast, when the same topic is rendered with Mermaid, the gold standard encodes alternative therapy structures (e.g., (Aromatherapy Usage, advises, Burnout Prevention). (Alternative Therapies, partOf, Resilience Building)), yet the model prediction introduces inconsistent or shallow associations, failing to capture the intended containment and functional dependencies. These examples illustrate a clear gross-renderer sensitivity: even for the same se- mantic topic, the model's outputs diverge substantially depending on the renderer, resulting in structurally mismatched and semantically incoherent graphs
27

---

Published as a conference paper at ICLR 2026
K.2 COMPLEX NESTING ERROR ON FLOWGEN TEST SET
An Example of Complex Nesting Error on FlowGen Test Set
[System Input]
Configured Appendix H.1
[User Input]
Configured Appendix H.l
Qwen2.5-VI--7B Output:
<AfterSShhool Programs, connectedToo, Special Needs Accommodation>>
<After-School Programs, disciplines, Field Trip Organization
<After-Schoo Programs, guides. Hygiene Practices>>
<Field Trip Organization, connectedTo, Playground Safety Inspection>>
<Field Trip Organization, maintains, Playground Safety Inspection>>
<Felld Trip Organization, safeguards, Nutritional Meal Preparation
<Hygiene Practices, assesses, Nutritional Meal Preparation
<yyiene Practices, communicates, Field Trip Organization
Hygiene Practices, connectedTo, Field Trip Organization>>
Hygiene Practices, connectedTo, Playground Safety Inspection>>
Hygiene Practices, supervises, Field Trip Organization>
Music and Movement Activities, connectedoo, Special Needs Accommodation>>
Music and Movement Activities, partOf, Child Protection Policies>>
Nutritional Meal Preparation, connectedTo, Field Trip Organization>>
Nutritional Meal Preparation, partff, Child Protection Policies>>
Parent Communication, advises, Story Time Reading
Parent Communication, connected Music and Movement Activities>>
Parent Communication, partOr, Child Protection Policies>>
Playground Safety Inspection, connectedTo, Field Trip Organization>>
Special Needs Accommodation, cares for. Hygiene Practices>>
Special Needs Accommodation, connected Music and Movement Activities>>
Special Needs Accommodation, connectedToo, Playground Safety Inspection>>
Special Needs Accommodation, prepares, Field Trip Organization
Story Time Reading, advises, Parent Communication>>
< Story Time Reading, partOf, Child Protection Policies>>
Transmission Electron Microscopy, connectedToo, Nanodevice Fabrication>>
Transmission Electron Microscopy, connectedTo, Solar Cell Efficiency Enhancement>
Error Analysis:
Qwen2.5-VL-7B fails to capture hierarchical containment and role-task dependencies present in the gold standard. While the gold-standard graph encodes explicit partOf relations e.g.. situating "Music Movement Activities", "Parent Communication", "Story Time Reading". and "Nutritional Meal Prepara- tion" under "Child Protection Policies", the model's prediction, however, flattens or distorts these struc- tures: it omits key containment relations, incorrectly shifts responsibilities (c.g., assigning (Nutritional Meal Preparation, ParTOf, Child Protection Policies) while failing to preserve its other nested partOf re- lations), and even hallucinates irrelevant nodes such as Transmission Electron Microscopy, demonstrating difficulty in preserving complex hierarchies across multiple functional domains.
28

---

Published as a conference paper at ICLR 2026
K.3 EDGE AMBIGUITY ERROR ON FLOWGEN TEST SET
An Example of Edge Ambiguity Error FlowGen Test Set
[System Input]
Configured as Appendix
[User Input]
Configured Appendix
Qwen2.5-VI--7B Output:
<Biocontainment Strategies, partOf, Biofoundry Operations>>
Biocontainment Strategies, regulates, Cellular Reprogramming>
Biossfety Level Management, builds, Regulatory Frameworks
Biosafety Level Management, partff, Biofoundry Operations>>
Biocontainment Strategies, simulates, Biosafety Level Management>>
<Cellular Reprogramming, partOf, Biofoundry Operations>>
<Cellular Reprogramming, purifics, Public Engagement>>
<Computational Design of Biological Systems, clones, Modular Biology Design'>
<Computational Design of Biological Systems, connectedTo, Public Engagement>>
<Drug Delivery Systems, circuits-desinns, Synthetic Organism Design>
<Drug Delivery Systems, materials-develops, Metabolic Pathway Engineering>>
Gene Synthesis Platforms, connectedTo, Metabolic Pathway Engineering>>
<Gne Synthesis Platforms, ferments, Regulatory Frameworks>>
<Gnn Synthesis Platforms, security- manages, Metabolic Pathway Engineering>>
Gene Synthesis Platforms, tests, Synthetic Organism Design>>
Metabolic Pathway Engineering, connectedTo, Drug Delivery Systems>
Metabolic Pathway Engineering, cchical-considers, Public Engagement>>
Metabolic Pathway Engineering, materials-devellps, Drug Delivery Systems>>
Modular Biology Design, connectedTo. Synthetic Organism Design>>
<ubblc Engagement, connectedTo, Computational Design of Biological Systems>>
<Public Engagement, connected'oo, Synthetic Organism Design>>
Public Engagement, purifics, Cellular Reprogramming'
<eggulatory Frameworks, builds, Biosafety Level Management
<Regulatory Frameworks, connectedTo, Metabolic Pathway Engineering>>
Regulatory Frameworks, licenses, Public Engagement>>
Transmission Electron Microscopy, connectedTo, Nanodevice Fabrication
29

---

Published as a conference paper at ICLR 2026
Error Analysis:
Qwen2.5-VI.-7B3 prediction exhibits clear edge ambiguity errors, where the directions of edges are fre- quently misrepresented. In the gold standard, edges encode regulatory constructive relations (e-g., (Biosafety Level Management, simulates, Biocontainment Strategies)), while model frequently reverses them (e.g.. (Biocontainment Strategies, simulates, Biosafety Level Managemeat)), inverting the causal logic. In addition. spurious edges and nodes are introduced (e.g.. (Transmission Electron Microscopy, con- To,tedTo, Nanodevice Fabrication)), which do not exist the imput diagram. As result, the predicted graph fails preserve the structural consisteney the scanned flowchart, producing ambiguous and logi- cally inconsistent edge relations.
K.4 OCR-DRIVEN ERROR ON FLOWGEN TEST SET
An Example of OCR-Driven Error on FlowGen Test Set
[System Input]
Configured Appendix H.1
[User Input]
Configured Appendix H.l
Qwen2.5-VL-77 Output:
<Employee Engagement Surveys, connectedTo, Offer Management
<Employee Recognition Programs, connectedTo, Change Management>>
<Employee Recognition Programs, connectedToo Global Mobility Programs
Employee Recognition Programs, connectedlTo, Nanodevice Fabrication>>
<Employee Recognition Programs, connectedTo,, Talent Management>>
Analytics, connectedTo, Performance Appraisal>
Offer Management, connectedTo, Recruitment Planning >
Performance Appraisal, connectedTo, HR Analytics
Recruitment Planning, connectedToo, Talent Management>>
<Change Management, connectedTo, Employee Engagement Surveys>>
Change Management, connectedTo, Global Mobility Programs>>
<Change Management, connectedTo, Nanodevice Fabrication
<Global Mobility Programs, connectedTo, Onboarding Process>>
Onboarding Process, connectedTo, Compensation and Benefits>>
Talent Management, connectedTo, Global Mobility Programs>>
30

---

Published as a conference paper at ICLR 2026
Error Analysis:
Owen2.55VII-7BB output reveals typical OCR-driven erross: misrecognized text leads to hallucinated nodes such "Nanodevice Fahrication", which do exist in the gold standard. As result, the model gener- ates spurious edges (e.g., (Employce Recognition Programs, connectedTo, Nanodevice Fabrication)) while omitting valid dependencies like (HR Analytics, connectedTo, Global Mobility Programs). These OCR- induced distortions corrupt both the semantics and the topology of the predicted graph, illustrating how recognition noise directly propagates into structural parsing
31


Reviews:
[{"review_id": "Lb87ZXnKpT", "reviewer": "Reviewer_i9ZK", "summary": "The paper introduces a system called FlowGen, which is a controllable flowchart synthesizer that generates diverse flowcharts with adjustable structural complexity and visual styles. The system enables large-scale, customizable datasets for both training and benchmarking multimodal large language models (MLLMs) on flowchart understanding and visual reasoning tasks. FlowGen supports multiple renderers (Mermaid, Graphviz, PlantUML, Diagrams) and configurable parameters (e.g., graph order, branching factor, nested subgraphs). Experimental results show that models trained with FlowGen data (e.g., Qwen2) achieve significant improvements in flowchart parsing and flowchart QA, with strong generalization to public datasets such as FlowVQA and FlowLearn. Moreover, FlowGen serves as a challenging evaluation benchmark, revealing weaknesses in even top proprietary models (GPT-4o, Gemini-2.5-Flash).", "questions": "What could happen if the authors generate data with different LLMs (open-source vs. closed-source MLLMs)? What is the impact of a less performant MLLM? To what extent the noise hurts the overall performance?\n\nDoes a combination of MLLM generated and human generated data could improve performance and avoid potential biases that could be brought by the MLLM?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation of the paper is strong. The authors motivate that prior work (including FlowchartQA, FlowVQA, FlowLearn) does not have control over structural and stylistic complexity, whereas FlowGen specifically introduces a parametric generation framework to address this limitation. The generation of synthetic data resembles the data augmentation which show indeed improvement in performance. \n\n2. The paper presents a good experimental design with many MLLMs and two types of evaluation metrics (exact and relaxed metrics). The Easy/Medium/Hard complexity of the dataset seems interesting and allows for various analysis. \n\n3. The paper contains good ablation studies that systematically isolate contributions from multiple perspectives: multi-renderer training; nested subgraphs; and split/merge arrows\nshowing their distinct impacts on performance.", "weaknesses": "1. Semantic labels (node and edge names) are generated with LLMs and verified by humans. I wonder if this could introduce stylistic or linguistic biases, especially if similar LLMs are later tested on FlowGen data. \n\n2. FlowGen by design enables controllable generation, but it would be interesting to see if synthetic diagrams may or may not fully capture the diversity and imperfections of real-world flowcharts (e.g., hand-drawn figures, inconsistent layouts, textual clutter). \n\n3. Since training and test distributions share the same generative pipeline, models might implicitly learn renderer-specific biases, even if content instances differ. Did the authors observed such issues?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i6FTH3u7Qz", "reviewer": "Reviewer_9ofn", "summary": "The paper presents FlowGen, a controllable synthesizer that generates flowcharts with adjustable structural and visual complexity to evaluate and fine-tune multimodal large language models (MLLMs). The system defines parameters for graph order, branching, density, unlabeled edges, and nesting depth, and renders diagrams using multiple backends (Mermaid, Graphviz, PlantUML, Diagrams). The authors generate 11,520 flowcharts across 120 topic domains and three difficulty levels, fine-tune ten MLLMs, and evaluate on multiple flowchart parsing and QA datasets. Fine-tuning with FlowGen consistently improves performance, particularly on structurally complex diagrams. In addition, the authors use FlowGen to create challenging validation sets designed to probe model robustness along two controlled axes (graph complexity and scanned document degradation) which systematically expose consistent weaknesses in both open-source and proprietary models.", "questions": "**Questions for Authors**\n\n1. Are the reported results on flowchart parsing and QA datasets obtained in zero-shot mode, or were the models fine-tuned on these evaluation datasets? Please clarify to properly interpret generalization performance.\n2. When using FlowGen-generated training and test sets, how do the authors ensure no overlap or near-duplicates exist between them?\n3. What is the source and diversity of the 120 topic domains used for generation? Are they drawn from existing ontologies?\n4. Have the authors computed any quantitative metrics to measure variability within the same topic and across difficulty levels? This could help to assess the true flexibility of FlowGen and potential limitations in coverage.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- **Novel and practical contribution:** FlowGen fills a significant gap in controllable synthetic flowchart generation for multimodal reasoning.\n- **Controlled and systematic flowchart synthesis:** The modular three-stage architecture (configuration, graph construction, rendering) allows manipulation of structural and stylistic parameters, producing flowcharts with diverse properties.\n- **Comprehensive experimentation:** Evaluations span multiple models, datasets, and difficulty levels, with well-designed ablations and detailed error analyses.\n- **Robustness dimension:** The introduction of degraded and complex test sets is a valuable step toward stress-testing MLLMs.", "weaknesses": "- **Limits of flexibility and variability:** While FlowGen enables controlled chart variability, the actual range of possible flowcharts likely depends on the set of semantic topics and predefined parameter ranges. A quantitative study of generated flowchart diversity within the same topic would help clarify the boundaries and potential limitations of the method. Moreover, the 120 topic domains used for across the experiments are insufficiently described; the source and diversity of these domains need more explanation.\n- **Unclear evaluation protocol:** The authors do not specify whether reported results on flowchart parsing and QA datasets are in zero-shot settings or if models were fine-tuned on the target evaluation datasets. This ambiguity makes it difficult to interpret the performance gains and assess true generalization.\n- **Dataset independence concerns:** In section 4, training and test sets are both generated by FlowGen, raising questions about evaluation fairness and true generalization. Clarifying how overlap and near-duplicates are prevented is essential.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cRcL1WfEnF", "reviewer": "Reviewer_3UeF", "summary": "This paper presents FlowGen, a controllable flowchart synthesizer for training and benchmarking multimodal LLMs. The generator exposes seven knobs (graph order, split/merge arrows, branching factor, density, unlabeled-edge ratio, nested subgraphs) and renders with Mermaid, Graphviz, PlantUML, and Diagrams to induce style diversity (Fig. 2, p.3). Using FlowGen, the authors build a 11,520‑image training set (Table 2, p.6) and a six‑subset test suite spanning structural difficulty and scanned‑image degradation. Across six parsing benchmarks, fine‑tuning open‑source MLLMs on FlowGen yields large F1 gains (e.g., Qwen2.5‑VL‑7B: FlowLearn 43.2→60.1 strict F1; hdBPMN 16.6→18.3; Table 3, p.6), and supplying triplets extracted by a FlowGen‑tuned model improves flowchart QA accuracy for multiple models (Table 4, p.7). A new FlowGen test set exposes low base performance for both open‑source and proprietary models (e.g., GPT‑4o ≤25% F1 on most subsets) and shows that FlowGen‑SFT substantially helps (Table 6, p.9). Ablations indicate multi‑renderer training and explicit modeling of nesting/split‑merge structures matter (Table 5, p.7).", "questions": "- Can you report QA with (a) gold triplets, (b) no triplets, and (c) per‑model self‑extracted triplets? How sensitive are Table 4 gains to extractor accuracy?\n- Have you tried ontology mapping, lemmatization, or embedding‑based matching to reduce synonym penalties in triplet evaluation? How do rankings change? \n- What is performance when training on one renderer and testing on another (full cross‑matrix), beyond the single‑renderer ablation in Table 5?\n- Can you quantify the gap between virtual‑node split/merge and a gateway/edge‑semantics encoding (e.g., BPMN‑style gateways) on hdBPMN?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Combines controllable graph parameters with multi‑renderer outputs; introduces scanned‑style perturbations and a matched train/test pipeline\n- Extensive, multi‑model evaluation; clear ablations demonstrating the role of renderer diversity, nesting, and split/merge \n- Well‑specified metrics and prompts (Appendix D) and transparent training setup.\n- Offers a scalable path to stress‑test MLLM reasoning over structured diagrams; highlights consistent weaknesses", "weaknesses": "1. Evaluation Design Masks True Model Capabilities\n\nTable 4's approach—feeding triplets extracted by a FlowGen-tuned Qwen-7B to all models—introduces two fundamental biases. First, it conflates extraction ability with reasoning capacity, making it impossible to isolate genuine question-answering performance. Second, it privileges models that happen to align with the chosen extractor's error profile. The paper should instead report: (i) QA performance without triplets as a baseline, (ii) QA with gold-standard triplets to measure pure reasoning, and (iii) QA using each model's self-extracted triplets to properly separate parsing from comprehension.\n\n2. Virtual Node Encoding Introduces Unvalidated Structural Bias\n\nThe split/merge virtual-node representation deviates from standard BPMN conventions (gateways, arrow semantics) in ways that could fundamentally bias what models learn. While the authors acknowledge this as future work (Conclusion, p.9), no quantitative analysis compares this encoding against alternatives. Without understanding how much this choice shapes results, we cannot assess whether findings generalize to real-world diagram understanding.\n\n3. Underwhelming Performance on Real Data Raises Generalization Questions\n\nThe best FlowGen-SFT result on hdBPMN remains modest—just 18.3% F1 for Qwen-7B (Table 3). This gap between synthetic training performance and real-world evaluation suggests the approach may not transfer beyond stylized, template-generated data. The paper needs substantially larger, truly out-of-distribution real-world test sets to validate practical applicability.\n\n4. Train-Test Splits Are Distribution-Matched, Not Truly Adversarial\n\nFlowGen's test sets are disjoint from training but deliberately distribution-matched (Sec. 4.1.1, p.8). This design choice likely inflates apparent robustness compared to unconstrained deployment scenarios. Cross-renderer evaluations (where training and test use different rendering engines) and unseen-topic tests would provide more realistic difficulty estimates.\n\n5. \"Scanned\" Augmentations Are Synthetic, Not Ecological\n\nTable 10's scanned-style effects (p.20) come from Pillow/OpenCV filters, not actual scanning or photography. Real documents exhibit sensor noise, lighting variation, physical degradation, and artifacts that synthetic perturbations cannot fully capture. Evaluation on genuinely scanned, photographed, and hand-drawn diagrams is essential for validating real-world robustness.\n\n6. Vocabulary Verification Lacks Transparency\n\nNode/edge vocabularies are generated by GPT-4o then \"human-verified\" (Sec. 2.2.2, p.4), but verification coverage and inter-annotator reliability are not reported. Without knowing what percentage was checked, by how many annotators, and with what agreement rates, we cannot assess label quality or rule out systematic template artifacts.\n\n7. \"Unlimited Diversity\" Claim Is Unsupported\n\nThe assertion that FlowGen generates unlimited diverse flowcharts contradicts the method's reliance on 120 static domain dictionaries with fixed vocabularies. Examination reveals repetitive structures, especially in \"complex\" configurations. The system samples from finite pools—it does not model true domain variation or generate genuinely novel structures.\n\n8. Random Semantic Assignment Undermines Reasoning Validity\n\nLabels are randomly sampled from GPT-generated word lists without semantic validation or causal coherence. Many examples show nonsensical flows (e.g., \"Health Coaching → Detox Program → Journaling Wellness\"). This fundamentally undermines the benchmark's value as a reasoning test: models learn to parse arbitrary structures, not understand real-world logic or causal relationships.\n\n9. Methodologically Inappropriate Baseline Comparisons\n\nComparing fine-tuned open-source MLLMs to proprietary reasoning-focused models like Gemini-2.5 is not methodologically sound. Gemini is architecturally designed for multi-step reasoning chains, not direct diagram parsing, making performance comparisons misleading. The evaluations conflate architectural differences with actual diagram understanding capability.\n\n10. Missing Critical Semantic Ablation\n\nThe central claim—that FlowGen improves visual reasoning—depends on semantically labeled nodes and edges. Yet the paper never evaluates performance when semantic labels are removed or randomized. This omission is critical: observed improvements could stem entirely from textual cues in the image rather than genuine spatial or structural reasoning. Without this ablation, we cannot determine what models actually learn.\n\n11. Limited Conceptual Contribution\n\nBeyond reporting performance numbers, the paper provides little insight into why MLLMs struggle with flowcharts or how they might fundamentally improve. The work essentially demonstrates that synthetic fine-tuning increases scores on similar synthetic data—an expected result that doesn't advance our understanding of diagram comprehension or suggest paths toward more robust visual reasoning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZPuREXbJHJ", "reviewer": "Reviewer_TxLZ", "summary": "This paper introduces FlowGen, a controllable synthesizer designed to generate diverse and structured flowcharts for training and evaluating multimodal large language models (MLLMs). The authors conduct comprehensive experiments showing that fine-tuning on FlowGen significantly improves flowchart parsing and question answering, while also exposing weaknesses in current MLLMs under complex or cross-renderer conditions. However, some aspects of the dataset design and evaluation pipeline need further clarification and empirical support.", "questions": "Please see weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper addresses an important gap in multimodal reasoning benchmarks by introducing a controllable and scalable flowchart synthesis framework, which allows explicit control over graph complexity and rendering diversity.\n\nThe proposed FlowGen synthesizer is technically sound and provides a reproducible pipeline for generating both training and testing data, with adjustable parameters such as graph order, branching patterns, and rendering style.\n\nThe experiments are comprehensive, covering multiple flowchart-related tasks and a variety of open-source and proprietary MLLMs, which strengthens the empirical validity of the results.", "weaknesses": "Weaknesses and Questions\n\nThe semantic content of the synthesized flowcharts is said to be drawn from 120 predefined application domains, each providing 40 node names and 40 edge labels. However, the paper does not explain the distribution and diversity of these domains, nor does it provide examples of the node and edge vocabularies. A more detailed breakdown would help readers assess how representative and balanced these semantic domains are. Additionally, the process of generating content “with GPT-4o and refined through human verification” is mentioned but not described in sufficient detail—clarifying this step would strengthen the reproducibility and credibility of the dataset.\n\nWhile the experiments focus on several flowchart-specific tasks, flowchart understanding is arguably a fundamental multimodal reasoning skill. It would be valuable to evaluate whether models fine-tuned on FlowGen also exhibit transfer improvements on broader benchmarks such as MathVista or MMMU, to demonstrate that FlowGen enhances general reasoning rather than only task-specific performance.\n\nIn Section 3.2, the authors mention synthesizing a training set of 11,520 flowcharts but do not explain how the task components of each sample are structured. Are all samples associated with parsing tasks, or do they also include QA tasks or other forms of supervision? Beyond Table 2, more detailed statistics, such as the task type distribution and average tokens per sample, would help readers better understand the dataset’s design and scope.\n\nThe evaluation setup involves prompting models with triplets extracted by Qwen2.5-VL-7B fine-tuned on FlowGen data. This design is not fully end-to-end, and may introduce error propagation from the extraction stage. A cleaner approach might be to evaluate each model’s direct performance on FlowGen-derived tasks (e.g., QA) using its own fine-tuned checkpoint, without intermediate feature extraction.\n\nThe paper also introduces a synthetic test set generated using the same pipeline. However, it is unclear whether the test set underwent any manual inspection or quality assurance. Since the same generation process is used for both train and test data, some discussion of data overlap prevention and human verification would be helpful to ensure a fair and meaningful evaluation.\n\nTable 6 shows that models trained on specific subsets (e.g., Graph Easy, Medium, Hard) achieve their best results on the corresponding test subsets. This suggests possible limited cross-subset generalization. Have the authors tested a model trained on the entire training set (combining all difficulty levels) to examine whether it performs more robustly across test subsets? Such results would provide a clearer picture of the model’s generalization ability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)