Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation
Authors: Lei Chen
Xuanle Zhao
Zhixiong Zeng
Jing Huang
Liming Zheng
Yufeng Zhong
Lin Ma
Venue: ICLR 2026 Poster (2026)
Keywords: Chart-to-Code Generation
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs

Abstract: While reinforcement learning (RL) has proven highly effective for general reasoning in vision-language models, its application to tasks requiring deep understanding of information-rich images and structured output generation remains underexplored. Chart-to-code generation exemplifies this challenge, demanding complex reasoning over visual charts to produce structured code. Supervised fine-tuning (SFT) alone is often insufficient, highlighting the need for effective RL strategies tailored to structured outputs. In this paper, we systematically investigate the performance plateau of SFT through large-scale experiments and propose Multimodal Structured Reinforcement Learning (MSRL) for chart-to-code generation. We construct the largest training corpus to date, with 3 million chart-code pairs curated from real-world tables in arXiv papers, addressing the limitations of previous synthetic datasets. Despite achieving state-of-the-art performance, our experiments show that simply increasing SFT data eventually leads to diminishing improvements. To break this plateau, MSRL employs a multi-granularity reward system that integrates both textual and visual feedback. At the textual level, rule-based rewards validate fine-grained code details, while at the visual level, a model-based reward assesses the structural similarity between rendered code and ground-truth charts. We implement a two-stage curriculum training strategy, first optimizing the model with textual rewards and then incorporating visual signals for further enhancement. Experimental results demonstrate that MSRL substantially breaks the SFT plateau, improving high-level metrics by 6.2% and 9.9% on ChartMimic and ReachQA benchmarks, respectively. Notably, our method outperforms all existing approaches in the chart domain and achieves competitive results with advanced closed-source models.

Published as a conference paper at ICLR 2026
BREAKING THE SFT PLATEAU: MULTIMODAL STRUC- TURED REINFORCEMENT LEARNING FOR CHART-TO- CODE GENERATION
Lei Chen
Xuanle Zhao
Zhixiong Zeng'
Jing Huang
Liming Zheng
Yufeng Zhong
Lin Ma"
Meituan
zengzhix ongemiiingemeiiiiiiiiimm tuan. com
forest. 11 nma @gma com
Project: httts:://gtthiii htw.aa/...oooooocoooooooo//o///
ABSTRACT
While reinforcement learning (RL) has proven highly effective for general reason- ing in vision-language models, its application to tasks requiring deep understand- ing of information-rich images and structured output generation remains under- explored. Chart-to-code generation exemplifies this challenge, demanding com- plex reasoning over visual charts to produce structured code. Supervised fine- tuning (SFT) alone often insufficient, highlighting the need for effective RL strategies tailored to structured outputs. In this paper, we systematically investi- gate the performance plateau of SFT through large-scale experiments and propose Multimodal Structured Reinforcement Learning (MSRL) for chart-to-code gener- ation. We construct the largest training corpus to date, with 3 million chart-code pairs curated from real-world tables in arXiv papers, addressing the limitations of previous synthetic datasets. Despite achieving state-of tfcc-of-ttherrttt art performance, our experiments show that simply increasing SFT data eventually leads to diminish- re-
ing improvements. To break this plateau, MSRL employs ward system that integrates both textual and visual feedback. At the textual level, rule-based rewards validate fine-grained code details, while at the visual level, a model-based reward assesses the structural similarity between rendered code and ground-truth charts. We implement two-stage curriculum training strategy, first optimizing the model with textual rewards and then incorporating visual signals for further enhancement. Experimental results demonstrate that MSRL substan- tially breaks the SFT plateau, improving high-level metrics by 6.2% and 9. 9.9% on ChartMimic and ReachQA benchmarks, respectively. Notably, our method out- performs all existing approaches in the chart domain and achieves competitive results with advanced closed-source models.
INTRODUCTION
Large language models (LLMs) have demonstrated impressive reasoning capabilities complex textual problems, including code generation and mathematical problem-solving OpenAl (2025b);
Guo et al. (2025). Pivotal to this success is the implementation of reinforcement learning (RL) to optimize the capacities of LLM to generate step-by-stey reasoning Wei et al. (2022) and verify final Snswers/Shao al. (2024). While multimodal large language models (MLLMs) have achieved impressive performance on established visual reasoning tasks like visual question answering Zhang al. 2024a): Huang al. 2025). their core reasoning capabilities remain limited, especially when tasks require processing information-eene images like charts Xu et al. [hen) Chen et al. (205aa)
and generating structured outputs like code Xia al. (2024); Zhao et al.
(022b))
Among these challenges, chart-to-code generation Yang et al. (2024a); Zhang et al. (002cc) stands as task of both high complexity and significant practical value. Charts are widely used to communicate scientific findings, business insights, and statistical data, yet their information is often
Project leader. Corresponding author.

---

Published as a conference paper at ICLR 2026
100
+3.39
2.8M SFT
+35k MSRL
Performance (%)
+6.22
80-
12
70
Execution Rate
Soor
Hih-lovel Score
400K
B0OK
1600K 2000K
2800K
Exec. Rate
Low-lovel
Hig-leval
Figure 1: The SFT plateau and RL performance gain from our experiments. The left figure illustrates that scaling SFT data from 200k to 2.8M leads to a performance plateau after exceeding 2M data points. The right figure denotes the performance gain from our proposed MSRL training strategy.
locked in visual formats that are difficult for automated systems to interpret or repurpose. Enabling MLLMs accurately convert charts into executable code not only advances their visual understand- ing and structured reasoning capabilities, but also opens up impactful applications, such as assisting researchers in automattig the reproduction of scientific visualizations. Although previous works have attempted to solve this task by using methods like Supervised Fine-TTuning (SFT) /Zhnn et al. (2024b); Zhao et al. (2025b) or Direct Preference Optimization (DPO) Rafailov (DPO))Rafailo et al. (2023); /Zhag et al. (2025) on synthetic data, their performance remains limited due to the simplistic patterns of the corpus and ineffective generalization strategies. Therefore, a promising direction is to leverage more realistic data and robust strategies to enhance model performance.
In this work, we first systematically investigate the SFT performance plateau through experiments at various data scales. To enable this analysis, construct the largest training corpus to date for chart-to-code generation, covering 24 distinct chart types and comprising million chart-code pairs. Our two-stage data pipeline curates high-quality tables from arXiv papers and then utilizes LLMs to generate diverse and complex plot codes, ensuring realistic distributions and avoiding monotonous trends. Unlike prior works Ni al. (2025); [Chen et al. (2025b) that apply RL directly after SFT, we establish saturated SFT baselines to isolate the true effectiveness of our proposed multimodal struc- tured reinforcement learning (MSRL) strategy. Through experiments with varying SFT data scales, we demonstrate that simply increasing the volume of SFT data leads to performance plateau that cannot be overcome by data quantity alone. To break through this plateau, we propose the MSRL training strategy, which incorporates multi-g multi-graauulaiity reward functions from both textual and visual aspects. For textual rewards, we standardize code formats the training corpus and design a com- prehensive reward function that verifies code correctness from five key perspectives. Recognizing that text-based rewards alone may overlook overall chart structure, we introduce a visual reward mechanism: generated code is rendered into images and MLLM evaluates their similarity to the original input chart. By jointly leveraging these multi-granulariin rewards, MSRL optimizes both global chart context and fine- ine-grained code details. We evaluate models on various chart-to-code benchmarks Yang et al. (2024a): He et al. 2024).. The results demonstrate that it establishes a new state-of-the--rt, it outperforms all previous open-source models and rivals advanced proprietary MLLMs. In summary, the main contributions of this work are as follows:
We propose an MSRL training strategy that employs a multi-level reward function, which combines a rule-based component for assessing textual code correctness with a model- based evaluator for evaluating the visual fidelity of the rendered chart. Benefits from the MSRL training strategy, our proposed model achieves state-of-the-an (SOTA) performance compared to all other open-source MLLMs.
We construct the first large-scale chart-to-code training corpus, generated using real-world tables as a data source. After applying several filtering methods, we curate a final dataset of 2.8 million samples for SFT and 33 thousand for RL.
Through SFT experiments at various data scales, we identify the performance bottleneck, confirming that merely increasing SFT data quantity is insufficient to improve performance.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
2.1 CHART MLLMS
Chart understanding is crucial research area encompassing both low-level tasks, such as data ex- traction Liu al. 2202), and high-level tasks like question answering Masry et al. (2022) and summarization Kantharaj et al. (2022). Recent work has focused on training MLLMs on extensive, chart-specific datasets enhance their understanding capabilities Xia et al. (2024); Zhang et al. (204bb): Xia et al. (2023), leading to superior performance across various chart-related tasks, For example, ChartMoE Xu al. (2024) proposes mixture-of-expert. (MoE) structure for multi-task pretraining. More recently, leveraging reinforcement learning CCenn et al. (2025a); Masry et al.) (2025); |Rodriguez al.) (2025b) to enhance reasoning capacities has garnered significant interest. Both Chart-RI and BigCharts-RI optimize MLLMs using Chain-of-Thought (CoT) Wei et al. (2022) reasoning data and reinforcement learning with verified rewards (RLVR) Shao et al. (2024).
2.2 REASONING MLLMS
The success of large reasoning models (LRMs) like DeepSeek-RI Guo et al. (2025) spurs signif- icant research into enhancing LLM reasoning through reinforcement learning with rule-based re- wards [Shao et al. (2024). This paradigm extends to the Dision-language domain, where numerous works aim improve the long-chain reasoning capabilities of MLLMs Shen et al. (2025); Wang iet al. (2025b). Following this trend, initial efforts such as Vision-RI Huang et al. (2025) and RI- One Vision Yang et al. (2025b) adapt the Group Relative Policy Optimization (GRPO) algorithm with multimodal reasoning data to enable long-form reasoning in VLMs. Subsequent works, in- cluding MMEureka Meng et al. (2025) and RI-Zero Liu et al. (2025), further advance this area by introducing improved reinforcement learning strategies for visual long-term reasoning, Recently, approaches like Point-RFT T Ni et al. (2025) utilize task-specific thinking frameworks to enhance reasoning capabilities further.
2.3 MLLMS FOR CODE
In the rapidly advancing research area of multimodal code generation, foundational work has fo- cused on establishing benchmarks to assess the capabilities of MLLMs. These benchmarks evaluate performance on tasks such as generating execution code for solving visual problems Li et al. (2024); Wang et al. (2025a); [Rdriguez et al. (2025a) 2024); Yang et al. (2024b) and HTMI. code from web- page screenshots (Si et al. (2024); Yun et al. (2024): Xiao al. (2024); Awal et al. (2025). Among these emerging tasks, chart-to-code generation has attracted significant interest due the complex- ity of its visual inputs. This task challenges an MLLM to generate plotting code that accurately reproduces the chart image. Recently, several benchmarks have been proposed to evaluate MLLMs in this context, assessing a range of capabilities. For instance, benchmarks such as ChartMimic Yang et al. (2024a), Plot2Code Wu et al. (2025), and ChartX Xia et al. (2024) assess chart-to-code generation capabilities, while others like ChartEdit Zhao et al. (2025a) and ChartM Yang et al. 2025a) evaluate the editing functionalities.
3 METHODS
3.1 TRAINING CORPUS CONSTRUCTION
While several chart-to-code generation datasets have been proposed, they are generally constrained by reliance on purely synthetic data and limited scale, resulting in charts that exhibit simplistic trends and lack diversity. To overcome these challenges, we construct world tables from arXiv papers and then leveraging LLMs to generate the corresponding plotting code. To ensure separation from the ChartMimic benchmark, we exclusively papers published in 2023 and earlier as our corpus. We ensure data diversity and quality through several strategies, such using high-quality seed code and filtering out con-executable code.
dataset by crawling real-
Through this process, we construct the largest training corpus to date, comprising 3 million chart- code pairs, to investigate the performance limits of SFT. However, previous research Chen et al.
3

---

Published as a conference paper at ICLR 2026
Prop
Prange
Gowcrtted Codks
Sample EE0
CUrt S.OEN
Gentii--5 Pl-h
1o Rusa Trnil
SPT-4
Tebla from
Irege stality Encketer
Code Examplat
RL Chart2Code Generation
Generated Code
Ground Truth
bana Scare
partt
Type
MSRL Medel
Testul Sooe
Setir/ebert Score
Textual
Lebel Sroen
Mar
vasd
Execution Soone
Visud Evolutter
Generttdd Zeage
(b) Our MSRL Framework
Figure 2: The data generation pipeline and proposed MSRL framework. (a) Our pipeline prompts Gemini-2.5-Flast with tables from arXiv papers and example codes to generate plotting code. After execution and filtering, the chart-to-code dataset of 3M pairs is obtained. We then sam- ple 130k of these pairs and apply three-stage filters (chart, data, vision) to curate a final, high-quality dataset of 33k examples for RL. (b) The framework of our proposed MSRL strategy. The textual reward is derived from a rulc-based evaluation of the generated code across five distinct dimensions. An MILM quantifies the visual reward based on the rendered image.
2025aa; Huang et al. (2025) shows that separating the SFT and RL training corpora prevents over- fitting to the SFT data format and enhances the exploration capacity of RL. To curate a high-quality dataset for RI., we begin with a candidate corpus of 130 thousand samples from the overall training corpus. These candidate data samples are then subjected to two-stage filtering strategy. The first filtering stage focuses on the code content, filtering based on chart and data type to ensure code diversity and that the output code is suitable for data extraction. Specifically, the chart types filtered according to plot functions. To handle complex charts containing multiple chart types, we employ tree- ree-structured parsing method to identify each type present progressively. The data values are filtered according to the data definition format, retaining only data structured as one-dimensional arrays and non-nested dictionaries facilitate extraction during the RL stage. This step narrows the candidate corpus to 45 thousand samples. The second stage addresses visual quality, employing GPT-4.1-mini OpenAl (2025a) as visual LLM judge filter for high-fidelit chart images. To mitigate potential bias from the MLLM judger, we manually inspect 100 samples that are assigned a score of 0 by the model, and observe over 90% consistency with human judgment. This process yields the final RL training corpus of 33 thousand high-quality samples. The remaining samples are utilized for SFT. The comparisons of our dataset with relevant chart-to-code datasets are listed in Table [ Our dataset features a completely realistic data source, more challenging multi-chart content, and richer visual diversity enabled by more API types.
3.2 SUPERVISED FINE-TUNING
Firstly, we perform SFT on Qwen2.5-VL as the base model using our generated chart-to-code dataset. To explore the effectiveness of data scaling and identify the SFT plateau, we fine-tune model six distinct training corpus sizes, created by partitioning our main dataset into sub- sets of 200k, 400k, 800k, 1.6M, 2M and 2.8M samples. The model is trained by minimizing the standard autoregressive objective, which is the negative log-likelihood of the target sequence. As
4

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of existing chart-to-code generation datasets. A indicates that ChartMoE uses both data from real CSV tables and data generated by LLMs. "API types" different Matplotlib API types covered by the dataset.
refers to the number of
Realistic data source
Dataset
Multi-chart
Chart types
Data samples
API types
Chartl
x
10
11k
83
ChartMoE
<20
800k
Chart2Code
X
15
168
ChartCoder
X
27
115k
187
Ours
24
3M
1,555
demonstrated in Section
4.2 and Figure T1 merely scaling the SFT data volume ultimately results in
a performance plateau. Increasing the SFT dataset from 2M to 2.8M samples results in no further performance gains.
3.3 MULTIMODAL STRUCTURED REINFORCEMENT LEARNING
Although SFT can significantly enhance the chart-to-code capabilities of MILMs, our experiments reveal that this approach ultimately reaches performance plateau, likely due to the limitation that SFT restricts the model's capacity restore detailed information. This limitation stems from the core drawback of the SFT objective that it treats every token in the target sequence with uniform importance. Therefore, SFT is ineffective in optimizing plotting code generation, as the plotting code comprises largely of boilerplate snippets such plt. plot, with critical information like specific data values or styling parameters appearing infrequently. Consequently, employing RL with reward function designed to prioritize the accuracy of such critical content presents promising approach for enhancing code generation capacity. To this end, we propose a two-stage RL training strategy utilizing multi-granularity reward functions that combine both textual and visual feedback.
Textual Reward While the RLVR method provides a powerful mechanism to guide models by measuring the correctness of detailed information, its practical application is challenged by the stylistic diversity of generated code, particularly in data definitions and function calls. This stylistic heterogeneity leads difficulty in comprehensively extracting and parsing the critical information for reward computation. To mitigate this, we introduce a code normalization step that maps each generated output canonical representation before calculating the reward. This process ensures the textual reward function is invariant to syntactic variations, providing more suitable method for RLVR. Specifically, the textual reward is granular rule- accuracy score, which a weighted average derived from evaluating key aspects of the generated code. We assess data values using soft value matching, chart types with hard string matching, layout with hard value matching and elements like titles and labels via edit distance. The soft value matching allows relative error tolerance of +5%. Separately, we compute an execution reward, which a binary score indicating whether the generated code can be executed successfully. These two rewards ensure that both the fidelity of critical information and the executability of the code are considered during RL optimization.
Visual Reward However, analysis reveals that purely textual rewards generally focus fine- grained details, while the overall structure of the rendered chart image is not considered. The chart- to-code task requires generating code that replicates the overall style of charts. To this end, we propose visual reward feedback Gu et al. (2025). Specifically, the generated code is executed to render a chart image, and we then utilize MLLMs to score the visual similarity between the generated chart and the input. This score is then normalized to serve as the visual reward. For the visual reward, we employ Qwen2.5-VE VL- Bai al.) (2025) as the evaluation model, which compares the chart image rendered by the generated code against the ground-truth chart across six key aspects: chart type, layout, text content, data, style, and clarity. The resulting scores are then normalized to form the final reward. Crucially, code that fails render an image receives reward of 0. The framework of our proposed MSRL is denoted in Figure 2
Two-stage RL Obtaining visual rewards through model evaluation requires substantial computa- tional time. To ensure training efficiency, our two-stage RL strategy first optimizes the model using only textual reward. In the second stage, introduce hybrid reward, which combines the tex- tual reward with a visual one, to fine-tune the model for visual fidelity using a reduced number of samples. This approach allows balance computational cost and model performance, achiev-
5

---

Published as a conference paper at ICLR 2026
Table 2: Evaluation results various models on the ChartMimic Direct Mimic and ReachQA bench- marks. $ denotes the updated result with code_ pass constraints reported in its repository.
ChartMimic
ReachQA
Model
Params
Exec.Rate Rate Low-Level High-Level
Exec Rate Low-Level Hivh-Level
Proprietary
GeminiPro Vision
68.2
53.8
53.3
74.0
67.0
67.8
Claude-3-opus
83.3
60.5
60.1
89.0
51.7
61.1
GPT-4V
91.2
76.4
78.9
88.0
69.5
78.6
GPT-4o
93.2
79.0
83.5
92.8
81.8
84.0
Open Open-Source Source Ceneral--omaae
Qwen2-VL-7B
7B
67.0
32.9
35.0
55.4
22.6
29.3
Qwen2.5-VI--7B
7B
73.2
44.6
41.6
62.2
36.9
37.6
InternVL2-8B
8B
61.8
34.4
38.9
50.8
24.1
24.2
InternVL2-26B
26B
69.3
41.4
47.4
55.4
29.0
28.8
Qwen2-VL-72B
72B
73.3
54.4
50.9
77.2
50.0
48.1
Open-Source Chart-Domain
ChartLlama
13B
57.5
24.8
28.1
54.8
11.1
8.1
ThnyChart
3B
42.5
26.3
25.9
34.4
11.6
11.2
C.artVL.M-L
14B
19.5
15.8
13.9
8.2
2.1
3.9
Chart2Code
7B
62.1
42.9
33.3
63.6
52.3
49.7
ChartCoder
7B
91.4
72.5
74.0
83.8
67.9
69.4
MSRL-SFT
7B
93.2
73.0
77.6
92.2
78.6
80.0
MSRL
7B
96.5
78.6
83.8
98.2
86.1
89.9
ing strong visual quality without excessive resource consumption. To properly balance the different training objectives, the total reward R for completed trajectory is calculated weighted sum of the following three components:
R = Beet + w. Rvis + Rexes
(1)
where Wt, Wy. and we are hyperparameters that balance the contribution of each component. We adopt the Group Relative Policy Optimization (GRPO) Shao al. (2024) algorithm for RL. GRPO optimizes policy by leveraging group group-wise relative advantages among sampled responses, which is
well-suited for our customized reward designs of charts.
EXPERIMENT
4.1 BASELINES AND BENCHMARKS
We evaluate MSRL against three categories of state-of-the-art models. The first category comprises
general-domain, open-source Multimodal Large Language Models (MLLMs): Intern VL2 (8B, 26B) Chen et al. (2024b), Qwen2-VLL (7B, 72B) Wang et al. (2024). and Qwen2. VL Bai et al. (2025). The second includes proprictary models: GemimPProViss Vision Team al. (2023), Claude-3- opus Anthropic 2024), GPT-4V OpenAl (2023), and GPT-4o
The third category
consists of chart-specific MILMs: ChartLlama Han
(2023), Tinychart Zhang et al. (2024b),
ChartVLM Xia et al. )2024), Chart2Code Zhang et al. (2025) and ChartCoder Zhao et al. (2025b).
models were evaluated in a zero-shot setting on two distinet benchmarks: ChartMimic Yang et al. (2024a) and ReachQA He et al. (2024). For ReachQA, we adopt the evaluation setting of Chart2Code Zhang et al. (2025), using the 500 plotting scripts from its test set. For ChartMimic, we utilize the 600 examples from the latest version of the Direct Mimic task.
4.2 MAIN RESULTS
Our proposed MSRL establishes a new state-of saee--ff-theeee art among all open-source MLLMs, as detailed in Table 2 On the ChartMinie benchmark Yang al. (2024a), it achieves execution rate of

---

Published as a conference paper at ICLR 2026
Table Detailed results of low-level scores on the ChartMimic Direct Mimic and ReachQA bench-
marks. The ChartCoder performance metrics have been corrected due to a flawed evaluation setup.
ChartMinic
ReachQA
Model
Params
Text
Layout
Type
Color
Text
Layout
Type
Color
GPT-4o
81.5
89.8
77.3
67.2
84.4
91.2
81.3
70.5
Qwen2-VL-72B
72B
43.2
80.5
54.6
39.4
41.0
52.2
59.1
47.8
Qwen2.5-VL-7B
7B
37.7
65.6
42.6
32.4
29.6
40.9
44.2
32.8
ChartCoder
7B
65.9
85,4
72.3
66.4
58.2
80.0
70.4
63.2
MSRL
7B
78.0
93.0
75.3
68.2
87.5
96.6
83.1
77.2
Todaa Howarss
Teckel Heweorh Stnggeet
Eace Exccation Nowarts
Figure Comparison of textual reward and execution rate changes between baseline and SFT models during the RL stage.
96.5, low-level score of 78.6, and a high-level score of 83.8. Notably, its performance is not only superior to its open-source models but also comparable to that of GPT-4o, significantly larger pro- prietary model. On the ChartMimic benchmark, MSRL outperforms CharCooder, the previous lead- ing Chart-domain MLLM specifically designed for chart-to-code generation. Unlike other works that yield suboptimal performances from fine-tuning open-source models, our initial MSRL-SFT model already establishes a new state-of-the-an among open-source models. However, MSRL-SFT still falls short of GPT-4o on both low-level and high-level performance metrics. Building upon this strong foundation, the MSRL strategy further boosts the performance, especially in high-level scores, demonstrating the effectiveness of our integrated approach. The performance gap between SFT and RI. models emphasized Figure D
We also conduct detailed evaluation of low-level performance metrics, with the results presented in Table 3 Notably, MSRL outperforms all open-source models across low-level evaluation metrics and surpasses GPT-4o on the majority of evaluation metrics, which shows the strong capacity of MSRL chart-to-code generation. As the GPT-4o is much larger than MSRL, the results demon- strate the effectiveness of our proposed methods.
SFT Plateau To investigate and demonstrate the effectiveness and efficiency of RL, we utilize vari- data scales for SFT to explore the upper limit of it. As illustrated in Figure the results reveal clear trend, whereby model performance improves sharply as the data size increases to 400k samples, followed by period of slower growth up the 1.6M samples. Beyond this data scale, performance saturates and reaches a distinct plateau, showing negligible growth with any further increase in data scale. Based this trend, even if the data multiplied, the benefits brought by SFT will be mini- mal. Applying our proposed MSRL method yields significant performance gains across all metrics. On the ChartMimic benchmark, we observe an average improvement of 6%, with notable increases in execution rate as well as in both low-level and high-level scores.
RL Plateau To investigate whether RL training exhibits performance plateau, we present the per- formance curves of the two-stage RL training with the increase of data size, as shown in Figure In the first stage using only textual rewards, the model breaks through the SFT plateau and achieves significant performance improvement of approximately 5%, converging at maximum of 22k training samples. In the second stage with multimodal rewards, the model's performance is further improved by about 1.5%, and almost converges maximum of 11k training samples. Our exper- iments effectively capture the performance plateau and improvements within a practical data size range, providing strong evidence for the effectiveness of our two-stage RL strategy.

---

Published as a conference paper at ICLR 2026
5FT Stage
H. Seggi Sonual W Rewards
Stapps
Semple
Sanple
Figure 4: Performance curve of MSRL with increasing data scale. From left to right, the curves correspond to the SFT stage, RL stage 1, and RL stage 2. Avg Score denotes the average of the Low- and High-level scores in ChartMimic.
4.3 ABLATION STUDY
SFT/RL Ablation
To understand the impact of different training settings, we conduct ab- lation studies on both the SFT and RL train- ing stages. In this ablation study, we use only not
Table 4: Ablation studies on SFT and RL stages. For two RL experiments, only a textual reward is used.
ChartMimic
SET RL
Exec. Low-Level High-Level
textual rewards to ensure the results influenced by our improved RL algorithm, al-
73.16
44.58
41.55
lowing us demonstrate the interplay be- tween SFT and RI.. Using Qwen2.5-VI--BB- Instruct the baseline model, the results are presented in Table 4 As the RL-only model underperforms on low-level and high-level scores compared to the SFT-only approach, only the ex- ecution rate is slightly higher due to the inclusion of an execution- reward. Furthermore, the significantly larger performance gain from RL on suboptimal baseline, compared an SFT model, indicates that this improvement is partly attributable to the unsaturated state of baseline models. The performance gain on a saturated SFT model demonstrates that our proposed multi-granularity reward function can surpass the upper limit of SFT, achieving significantly superior performance. Figure compares the reward and exccution rate for the baseline and SFT models during the RL training, showing that applying RL to the SFT model achieves much higher textual reward.
93.17
73.02
77.61
93.83
65.60
62.29
97.00
78.06
82.69
Table 5: Comparison of single-stage and two-stage RL training with different reward strategies. "T" and "V" represent textual and visual reward, respectively. All training is conducted H800 GPUs.
ChartMimic Low-Level High-Level
Rewards
Samples
GPU Hours
Exec.Rate
93.17
73.02
77.61
(T. -)
(22k, -)
97.00
78.06
82.69
240
(V. -)
(22k, -)
97.37
79.13
84.01
1344
(T, T V)
(22k, 11k)
97.50
79.50
83.81
912
T V)
(22k, 5.5k)
96.50
78.62
83.83
576
Reward and RL Training Strategy Ablation To thoroughly evaluate the effects of reward de- sign and RL training strategies, conduct ablation experiments comparing textual rewards, visual rewards, and their combinations. As shown in Table 5, both reward types improve model perfor- mance, but differ notably in resource consumption and efficiency. Visual rewards provide notable enhancements in code details and visual fidelity, but require more computational resources due to the overhead of image rendering and model evaluation. In contrast, textual rewards achieve competitive performance with much lower resource requirements. To balance efficiency and effectiveness, we adopt a two-stage RL strategy. The ablation results yield several important insights: 1) Both textual and visual rewards lead to significant improvements over the SFT model, with visual rewards being more effective for visual fidelity. 2) The high resource consumption of visual rewards highlights the importance of balancing performance and efficiency. 3) The two-stage RL strategy achieves the trade-off by first using low-cost textual rewards, followed by high-cost visual rewards with a limited number of samples. Based these findings, empirically report the two-stage training strategy with 50% of the training samples in our experiments, as it offers a better trade-off between performance and consumption efficiency.
8

---

Published as a conference paper at ICLR 2026
Ground Truth
GFT-4o
ChartCader
Quenn 5-VL-88
Curd
Figure 5: Showcasing charts generated by MSRL compared to proprictary and open-source MLLMs. The first column shows the ground-truth chart images. Columns 2-4 display charts rendered from the code generated by GPT-40, ChartCoder, and Qwen2.5-VL-7B, while the last column shows charts rendered by our MSRL model.
Generalization to Unseen Plotting Libraries To evaluate whether MSRL can transfer its capabil- ities to other plotting libraries. we collect 107 Seaborn and 150 Plotly images from their official websites. Since we only use Matplodib-style code to generate charts, the new test data are out- of-domain. We perform inference with both Qwen2.5-VL VLL-7B and MSRL on these two small test sets, calculating execution rates and high-level scores according to the ChartMimic protocol. As shown in Table 6 the results demonstrate that MSRL exhibits generalization to unseen Seaborn- and Plotly-style images.
Table Performance Qwen2.5-VL. VL-7B and MSRL on Seaborn and Plotly test sets.
Seaborn
Plotly
Model
Exec.Rate High-Level Exec.Rate High-Level
Qwen2.5-VL-7B
69.2
25.7
62.7
22.6
MSRL
85.1
30.5
90.0
35.9
4.4 VISUALIZATION
To provide a qualitative perspective on performance, we showcase several illustrative examples. As visually demonstrated in Figure the charts generated by MSRL exhibit a markedly higher fidelity to the ground truth when compared with leading open-source MLLMs, even GPT-4o. In particular, the data values and layouts of charts generated by MSRL are significantly more precise.
CONCLUSION
In this work, we investigate the performance plateau of Supervised Fine-Tuning (SFT) in chart-to- code generation and propose a Multimodal Structured Reinforcement Learning (MSRL) strategy to overcome We begin by constructing 3pir training corpus from real-world tables to sys- tematically demonstrate that merely increasing SFT data leads to diminishing returns. To address this, we introduce the MSRL, which optimizes the saturated SFT model using a two-stage training process guided by multi-granularity reward function that assesses both textual and visual correct- ness. Our approach achieves new ffate-of-the-a performance on standard benchmarks, surpassing all open-source models and rivaling powerful proprietary ones. We make significant contributions to the field, including the curation of a large-scale dataset for a definitive empirical analysis of the SFT performance bottleneck, and the development of the MSRL framework.

---

Published as a conference paper at ICLR 2026
6 REPRODUCIBILITY STATEMENT
For datasets, we provide a detailed description of the data generation process in Section 3.1 of the paper, along with dataset statistics and examples in Appendix A and the exact prompts used for dataset construction in Appendix [ For code implementation, we include the SFT and RL train- ing code for MSRL the supplementary materials, which contains complete training scripts and implementations of the multimodal reward functions.
REFERENCES
Anthropic. Introducing the next generation of claude, 2024. URL com/ nwws/claudseeeeee family
Rabiul Awal, Mahsa Massoud, Aarash Feizi, Zichao Li, Suyuchen Wang, Christopher Pal, Aish- warya Agrawal, David Vazquez, Siva Reddy, Juan A Rodriguez, et al. Webmmu: A benchmark for multimodal multilingual website understanding and code generation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 25129 23129-25156, 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Yufeng Zhong, and Lin Ma. Chart-rl: Chain-of-thought supervision and reinforcement for advanced chart reasoner. arXiv preprint arXiv:2507.1550, 2025a.
Yang Chen, Yufan Shen, Wenxuan Huang, Shen Zhou, Qunshu Lin, Xinyu Cai, Zhi Yu, Botian Shi, and Yu Qiao. Learning only with images: Visual reinforcement learning with reasoning, rendering, and visual feedback. arXiv preprint arXiv:2507.20766, 2025b.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024a.
Zhe Chen, Jiannan WU, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24185-24198, 2024b.
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on llm-as-a-judge, 2025. URL https::/ http:://axxvvvorr/../e/
2411. 15594
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, arXiv preprint arXiv:2501.122
2025.
Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. Chartllama: multimodal 11m for chart understanding and generation. arXiv preprint 23X1v:2311.16488 2023.
Wei He, Zhiheng Xi, Wanxu Zhao Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, and Xuanjing Huang. Distill visual chart reasoning ability from uims to milms. arXiv preprint arXiv:2410. 187.98888. 2024.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao. Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint 25X3v:2503.067499 2025.
10

---

Published as a conference paper at ICLR 2026
Shankar Kantharaj. Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. Chart-to-text: A large-scale benchmark for chart summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4003-4023, 2022.
Kaixin Li, Yuchen Tian, Qisheng Hu, Ziyang Luo, Zhiyong Huang, and Jing Ma. Mmcode: Bench- marking multimodal large language models for code generation with visually rich programming problems. In Findings of the Association for Computational Linguisties: EMNLP 2024, pp. 736- 783, 2024.
Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. Deplot: One-shot visual lan- guage reasoning by plot-to-table translation. In Findings of the Association for Computational Linguistics: ACL 2023, pp.
2023.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl rl-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20785 2025.
Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: bench- mark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, pp. 2263-2279, 2022.
Ahmed Masry, Abhay Puri, Masoud Hashemi, Juan A. Rodriguez, Megh Thakkar, Khyati Mahajan, Vikas Yadav, Sathwik Tejaswi Madhusudhan, Alexandre Piche, Dzmitry Bahdanau, Christopher Pal, David Vazquez, Enamul Hoque Prince, Perouz Taslakian, Sai Rajeswar Mudumba. and Span- dana Gella. Bigcharts-rl: Enhanced chart reasoning with visual reinforcement finetuning. In Conference on Language Modeling (COLM), 2025.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, Ping Luo, Yu Qiao, Qiaosheng Zhang, and Wenqi Shao. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule- based reinforcement learning. arXiv preprint arXiv:2503.07365, 2025.
Minheng Ni, Zhengyuan Yang, Linjie Li, Chung-Ching Lin, Kevin Lin, Wangmeng Zuo, and Li- juan Wang. Point-rft: Improving multimodal reasoning with visually grounded reinforcement finetuning, arXiv preprint arXiv:2505.19700 2025.
OpenAl.
Gpt-4v(ision) system card 2023.
URL
gpt - Sy-system-card//
OpenAl. Gpt-4o, 2024. URL htt.htttpp//ppeea.ccom//// 1ttp:/hhhllo-p---000000000 Accessed: 2024-05-13.
OpenAl. Gpt-4.1, https:: httpt::t/opeeeeteeee htt///ppeaacoom//e
atttp:://ppeaa..cmmm www.o//owwnoooo.o.oooeeoooee 2025a.
OpenAl. Introducing openai o3 and 04-mini. httpss:/opentt cww/ iddx/ introducing- and- 04 maimmmii// April 2025b. Accessed: 2025-07-14.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193. 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728-53747 2023.
Juan A Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte Suresh, François Savard, Ahmed Masry, Shravan Nayak, et al. Bigdoes: An open dataset for training multimodal models on document and code tasks. In The Thirteenth International Conference Learning Representations, 2024.
11

---

Published as a conference paper at ICLR 2026
Juan A Rodriguez, Abhay Puri, Shubham Agarwal, Issam H Laradji. Pau Rodriguez, Sai Rajeswar, David Vazquez, Christopher Pal, and Marco Pedersoli. Starvector: Generating scalable vector graphies code from images and text. In Proceedings of the Computer Vision and Pattern Recog- nition Conference, pp. 16175-16186, 2025a.
Juan Rodriguez, Haotian Zhang, Abhay Puri, Aarash Feizi, Rishav Pramanik, Pascal Wichmann, Arnab Mondal, Mohammad Reza Samsami, Rabiul Awal, Perouz Taslakian, et al. Rendering- aware reinforcement learning for vector graphics generation. In Advances in Neural Information Processing Systems (NeurIPS 2025), 2025b.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qiangian Zhang, et al. Vlm-rl: A stable and generalizable rl-style large vision-language model. arXiv preprint arXiv:2504. 07615, 2025.
Chenglei Si, Yanzhe Zhang. Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2code: How far are we from automating front-end engineering? arXiv preprint arXiv:2403.03 163, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- can-BBatistee Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint 12Xiv:2312./18000 2023.
Ke Wang. Junting Pan, Linda Wei, Aojun Zhou, Weikang Shi, Zimu Lu, Han Xiao, Yunqiao Yang, Houxing Ren, Mingjie Zhan, et al. Mathcoder-vl: Bridging vision and code for enhanced multi- modal mathematical reasoning. arXiv preprint arXiv: 10557, 2025a.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing usion-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
Yaoting Wang. Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal Shain-of-thougee reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12605, 2025b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. th-thuut prompting elicits reasoning in large language models. Advances in neural information processing systems, 0122824-244877 2022.
Chengyue Wu, Zhixuan Liang, Yixiao Ge, Qiushan Guo, Zeyu Lu, Jiahao Wang, Ying Shan, and Ping Luo. Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. In Findings of the Association for Computational Linguistics: NAACL 2025. pp. 3028, 2025.
Renqiu Xia, Bo Zhang, Haoyang Peng, Hancheng Ye, Xiangchao Yan, Peng Ye, Botian Shi, Yu Qiao, and Junchi Yan. Structchart: Perception, structuring, reasoning for visual chart understanding. arXiv preprint arXiv:2309. 1268, 2023.
Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, al. Chartx & chartvim: versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12118 12185, 2024.
Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang. Xinyi Xu, Wenxuan Wang. Zhiyao Xu, Yuhang Wang, and Michael R Lyu. Interaction2code: Benchmarking mllm-based interactive webpage code generation from interactive prototyping. arXiv preprint arXiv:2411.03292 2024.
Zhengzhuo Xu, Bowen Qu, Yiyan Qi. SiNan Du, Chengjin Xu, Chun Yuan, and Jian Guo. Chart- moe: Mixture of diversely aligned expert connector for chart understanding. In The Thirteenth International Conference Learning Representations, 2024.
12

---

Published as a conference paper at ICLR 2026
Cheng Yang, Chufan Shi, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran XU, Xinyu Zhu, Siheng Li, Yuxiang Zhang, et al. Chartmimic: Evaluating Imm's 's gross-modee reasoning capability via chart-to-code generation. In The Thirteenth International Conference Learning Represen- tations, 2024a.
Donglu Yang, Liang Zhang, Zihao Yue, Liangyu Chen, Yichen Xu, Wenxuan Wang, and Qin Jin. Chartm3: Benchmarking chart editing with multimodal instructions. In Proceedings of the 33rd ACM International Conference on Multimedia, pp. 5001-5009, 2025a.
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang. Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. -o-ovviis:::: Advancing generalized multimodal rea- soning through cross-modal formalization. arXiv preprint arXiv:2503. 2025b.
Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, et al. Matplotagent: Method and evaluation for Ilm-based agentic scientific data visualization. In Findings of the Association for Computational Linguistics ACL 2024, pp. 11789-11804, 2024b.
Sukmin Yun, Rusiru Thushara, Mohammad Bhat, Yongxin Wang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, et Web2code: large-scale webpage- to-code dataset and evaluation framework for multimodal llms. Advances in neural information processing systems, 37:1 12154-112157, 2024.
Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. Mm- Ilms: Recent advances in multimodal large language models. In Findings of the Association for Computational Linguistics ACL 2024, pp. 12401-12430, 2024a.
Liang Zhang, Anwen Hu, Haiyang Xu, Ming Yan, Yichen Xu, Qin Jin, Ji Zhang, and Fei Huang. Tinychart: Efficient chart understanding with visual token merging and program-of carrram-of-hhhurnee of-thouughs learning. arXiv preprint arXiv:2404. 16635, 2024b.
Zhehao Zhang, Weicheng Ma, and Soroush Vosoughi. Is gpt-4v (ision) all you need for automating academic data visualization? exploring vision-language models' capability in reproducing aca- demic charts. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp-
8271-8288 2024c.
Zhihan Zhang, Yixin Cao, and Lizi Liao. Enhancing chart-to-code generation in multimodal large language models via iterative dual preference learning. arXiv preprint arXiv:2504. 022000000 2025.
Xuanle Zhao, Xuexin Liu, Yang Haoyue, Xianzhen Luo, Fanhu Zeng. Jianling Li, Qi Shi, and Chi Chen. Chartedit: How far are miims from automating chart analysis? evaluating milms' capability via chart editing. In Findings of the Association for Computational Linguistics: ACL 2025, pp- 3616-3630, 2025a.
Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang. Zhiyuan Liu, and Maosong Sun. Chart- coder: Advancing multimodal large language model for chart-to-code generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguisties (ACL), 2025b.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye. Zheyan Luo, Zhanghhi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguis- tics. URL/attp://arxiv org/ http:/bb///423333333
A CHART2CODE DETAILS
We count the quantity and distribution of 24 chart types used in the SFT and RL stages respectively, as detailed in Table A The distribution among the various types is relatively balanced. For the RL data, filtered out two chart types, Graph and Funnel, which are not suitable for textual reward calculation. Furthermore, we present three visualization examples from the Chart2Code dataset, with increasing data density and complexity, as shown in Figures G H and
13

---

Published as a conference paper at ICLR 2026
Table A: The detailed chart types and corresponding quantities used in the SFT and RL. phases.
Split
Bar
Line
ErrorBar
Hcatmap
Box
Scatter
Histogram
Radar
3D
SFT
131,623 2,669
131,596
130,818 1,067
123,795
125,541 756
127,612 2,639
132,180 1,510
124,160 776
128,744
RL
2,743
1,984
1,699
Split
Pic
ErrorPoint
Violin
Area
Bubble
Multi-axes
Ring
Rose
Treemap
SFT
125,812
123,774
120,180 900
120,038
127,207 1,093
116,438 1,691
105,912 960
118.047 674
119,176
RL
1,446
2,277
1,558
518
Split
Bar_num
Contour
Density
Quiver
Graph
Funnel
Total
SFT
128,333
123,461 1,040
130,769 1,899
128,987 1,700
108,150
1,152
2,853,505 34,356
RL
2,757
B IMPLEMENTATION DETAILS
We utilize 2.8M chart2code data for SFT and 33k curated high-quality chart2code data for RL.
as the baseline model and LLaMA-Factory Zheng al. (024) for SFT. The SFT stage is trained for one epoch with learning rate of le-5 and a batch size of 32. The training process takes 60 hours on 16 H800 GPUs. Built upon the SFT model, MSRL undergoes two-stage RL process using the MM-EUREKA framework Meng al. (2025). MSRL consists of two RL stages. Stage one utilizes 22k data pairs and a textual-only reward (we = 1, the remaining 11k pairs with a hybrid reward (w, 0.5, We = 0.5, We = 0.5). The textual reward components are weighted as = 0.4, We = 0.3, wi 0.1, Wt- 0.1, wis 0.1. We deploy Dwwen.55-VL-72--nnstie
MSRL-SFT employs Owen2.5VL--B-Innste
we 0, We 0.5). Stage
as the evaluation model with max model len set to 12,288 for calculating visual rewards. For both RL stages, we train for one episode with a learning rate of le-6 and a batch size of 128. We set the generation temperature to 1, generate 8 rollouts per sample, and omit the KL divergence from the loss computation. The first and second stages required and 24 hours of training, respectively, on 24 H800 GPUs.
C MORE ABLATION STUDIES
The Use of Visual Evaluator To verify that the improvement in performance results from effective visual reward design rather than overfitting to the preferences of the VLLMM judger, we conduct an ablation experiment to observe the distribution of sample scores when the visual evaluator changes. Specifically, we randomly sample 200 instances from the RL data and use the MSRL-SFT model to perform chart-to-code prediction, rendering the resulting Python code into images. Following the ChartMimic scoring guidelines, human ratings of rendered samples serve the gold scores. Next, we use 1PT-4.1, Sive2.55VV--70 Qwen2.5-VL. VL-7B, and Vn3e to score the images. Ad- ditionally, compute the visual similarity between the ground-truie trutth and rendered images using DINO-L. We calculate the Pearson correlation coefficients between the scores from these models and the human scores, and plot the distributions of model scores versus human scores. As shown in the Figure A GPT-4, achieves the highest correlation with human judgment and has the most sim- ilar score distribution, followed by Qwen2.5-VL-728 while Owen2.5-VL-7B performs the worst. Considering both performance and budget, we select Qwen2.5-VL-72B as the visual evaluator, as it fairly reflects model capability and prevents reward hacking.
In addition, we introduce two vision models, Intern VIT Chen et al. (2024)) and DINOV2-L al.) (2023). as evaluators for visual rewards, calculating the visual similarity between ground- truth and generated image features. As shown in Table B these vision models yield improved performance, but there remains a substantial gap compared to models employing VL.Ms as visual evaluators. This indicates that vision models trained on image-text matching assess similarity at coarse level and do not adequately capture fine-grained differences in chart elements. Compared to (wee2.55-VL-725 smaller vision models such DINO-L exhibit greater diserepancies in score distributions relative to human judgment.
GRPO vs DPO Intuitively, GRPO is naturally more suitable for chart-to-code tasks than DPO. The degree to which code restores a chart can be quantitatively measured by a set of well-defined rules, making it ideal for designing the reward function in GRPO. In contrast, DPO learns from preference
14

---

Published as a conference paper at ICLR 2026
Figure A: Scatter plots of model scores versus human gold scores for five visual evaluators (GPT-4.1, Dwen2.5-VL-722B Qwen2.5-VL-7B, Intern VL3-3LL3-38, and DINO-L). Pearson denotes the Pearson correlation coefficient between model and human scores.
Table B: The ablation study visual evaluators. exclude textual rewards.
"_" denotes the SFT model. All experiments
ChartMimic LeveLevel High-Level
Visual Evaluator
73.0
77.6
Intern VIT DINOv2-L Qwen2.5-VL-72B
75.1
78.
75.5
79.8
77.8
81.9
pairs, and it is challenging to clearly define what constitutes "bad" code in chart-to-code tasks, which can lead ambiguity in the model's s optimization direction. To ensure thorough evaluation, we conduct ablation study comparing GRPO and DPO. Since DPO requires the construction of positive and negative sample pairs, we follow the Chart2Code /Zhang et al. (2025) and randomly perturb 2-3 attributes of the chart code generate negative samples using GPT-4.1, resulting in RL dataset comparable size to GRPO for DPO training on the SFT model. As shown Table C the performance improvement of DPO is significantly less than that of GRPO. This indicates both the superiority of the GRPO algorithm and the effectiveness of our multimodal structured reward design and two-stage curriculum learning.
Table C: The ablation study RL optimizers. "." denotes the SFT model.
ChartMimic Low-Level High-Level
RL Optimizer
73.0
77.6
DPO
74.0
78.9
GRPO
78.6
83.8
Comparison with Existing Datasets To highlight the value of our synthetic data, we compare the results of using data and open-source data (i.e., ChartCoder) in both the SFT and RL stages, as shown in Table D The results show that models trained with our data achieve better performance in both stages. On the one hand, sufficient data enable SFT to reach its full potential. On the other hand, our charts are generated from real tabular data, featuring more challenging multi-chart content and greater visual diversity, which are key factors contributing the effectiveness of our dataset.
D QUALITATIVE ANALYSIS
Figures E and show two examples of generated code from different models and their correspond- ing rendered images. Compared to the initial Qwen2.5-VLL-7B model, both MSRL-SFT and MSRL show significant improvements in chart type recognition, numerical extraction, and color identifi- cation. Compared to MSRL-SFT, MSRL performs better in chart type recognition and clarity. In Figure MSRL-SFT incorrectly identifies the line chart as a scatter plot, displays incorrect leg- ends, and has overlapping issues with image boundaries. In Figure MSRL-SFT has layout syntax errors, where only the last subplot is successfully displayed out of three subplots. MSRL's results more consistent with the original images, though it still has minor issues in detail. For
15

---

Published as a conference paper at ICLR 2026
Table D: Performance comparison with the ChartCoder dataset. We use Qwen2.5-VL-7B and MSRL-SFT as base models for SFT and RL for fine-tuning on the two datasets, respectively.
ChartMimic Low-Level High-Level
Stage
Dataset
ChartCoder Ours
68.5
68.0
SFT
73.0
77.6
ChartCoder
76.5
81.2
RL
Ours
78.6
83.8
MSRL arbitrarily adds title and scatter data points to the subcategory "Ours (LoSA)". In F while MSRL successfully renders three subplots, the layout recognition not sufficiently accurate, leaving room for further improvement.
example, in Figure
E PROMPT TEMPLATE
To enhance transparency and reproducibility, we provide the exact prompts used for dataset genera- tion and visual reward feedback.
Figure B shows the prompt used for code generation. We use real table data input, select one chart type from 24 predefined chart types, and sample code examples the selected chart type to generate plotting code. Specifically, for chart types Pie, Ring, and Treemap. require the model to display actual values on the image, as these chart types typically only visualize percentage values in their code, making impossible to read the true values from the image.
Figure C shows the prompt used for evaluating image quality. When processing RI data, we use this prompt filter out visually low-quality samples, such as those with overlapping text elements.
Figure D shows the prompt used to calculate visual rewards. Following ChartMimic Yang et al. 2024a),
the match between ground truth and model predictions across 6 dimensions, with this score normalized as the visual score and incorporated into the overall reward computation.
Prompt Template of Code Generation
Generate high quality python code for platting (chart_typp) chart from the following table data: (table_data)
Requirements:
The code must present table data reasonable
The code example of (chart_type) chart (given in JSON format) is:
[code_exomple]
You must be limited the code sample and drow different styles of dials.
The generated code should not be too complicated and all text elements (labels, titles, legends) must be fully visible without overlap or truncation.
Pie/Ring Pie/Ring/ Treemap chart visualization: always display the actual numerical values on each segment. Percentages ore optional, but values must be clearly visible. IMPORTAN Generate only ONE figure with all necessory information. If multiple plots needed, subplots ppit.subplots) to arrange them in single figure. Output format: python...
Figure B: Prompt template for Chart2Code code generation.
F THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this work, we utilized L.LMs primarily as writing assistance tools to polish language and improve clarity. They helped refine sentence structures and enhance readability throughout the manuscript. The core research ideas, experimental design, implementation, and analysis were all conducted by the authors, with LLMs serving only editorial aids.
16

---

Published as a conference paper at ICLR 2026
Prompt Template of Image-quality Evaluation
You are expert data visualization and chart analysis. Your task is to evaluate if the provided chart image is properly rendered and of high quality.
Please analyze chart check for the following issues:
1. Rendering problems (missing elements, incorrect display)
2. Data anomalies (outliers that don't make sense, inconsistent scales)
Truncated or cut-off text or elements
Overlapping text elements that reduce readability
Poor color choices that make the chart hard read
Missing labels or legends that necessory for understanding
7. Any other quality issues that affect the chart's s usefulness
After your analysis, provide a single score:
Scoore The chart properly rendered, data oppears normal, and there significant quality issues.
Score O: The chart has one
significant quality issues that affect its usefulness.
Respond with ONLY the score (1 or 0) and nothing else.
Figure C: Prompt template for image-quality evaluation during RL data filtering.
Prompt Template of Visual Reward Feedback
You are an excellent judge at evaluating visualization chart plots. The first image (reference image) is created using ground truth matplotlib code, and the second image (AI-generated image) created using matplotlib code generated by AI assistant. Your task score how well the AI-generated plot matches the ground truth plot.
### Scoring Methodology:
The AI-generated image's 's score is based on the following criteria, totaling score out of 100 points:
1. "*Chart Types (20 points)"*
Does the AI ge-ernaeddd image include all chart types present the reference image
(e.g.. line charts, bar charts, etc.)?
**Loyout (10 points)**
Does the arrangement of subplots the AI AI-generated image match the reference image
(e.g., number of rows and columns)?
3. **ext Content (20 paints)**
Does the AI-generated image include all text from the reference image
(e.g., titles, annotations, axis labels), excluding axis tick labels?
**Daa (20 points)***
How accurately do the data trends in the AI-generated image resemble those in the original image and is the number of data groups the same as in the reference image?
Style (20 points)"*
AI-generated image match the original in terms of colors colors, fill colors, etc.). marker types (point shapes, line styles, etc.), legends, grids, and other stylistic details?
6. *Clarity (10 points)**
Is the AI-generated image clear and free of overlapping elements?
### Evaluation:
Compare the two images head to head and provide detailed assessment. Use the following format for your
response:
Comments:
Chart Types: $(your comment and subscore)
Layout: $(your comment and subscore)
Text Content: $[your comment and subscore)
Data: $(your comment and subscore)
SStyle: $(your comment and subscore)
Clarity: $(your comment and subscore)
Score: $(your final score out of 100)
Please use the above format to ensure the evaluation is clear and comprehensive.
Figure D: Prompt template for visual reward feedback during the RI. training.
17

---

Published as a conference paper at ICLR 2026
Ground Truth Image
MSRL-SI SFT
MSRL Image
Qwen2 5-VL-78 Image
MSRL-SFT Output
MSRL Output
Natholhe gertiay Taame: Troage Twmel, Pol tneeweed 1664 (o
Owen2.5-VL-22 Output
Tran
byy ET
matk ands.
Die wnth sagidle ynd/
natton wills
2007 Your Aambar Mardeesi
smomnt
andinet
cooa ensttopfoa (oor ode der conernig
laal This Mherryy
Figure E: An example of comparing the code generated by different models with rendered images.
18

---

Published as a conference paper at ICLR 2026
Ground Truth Image
MSRL-SFT Enagee
MSRL Image
Qwen2. 5-v-2- Image
MFT Output
MSRL Output
lend
2wen2.5-VL-78 5-VL-78 Output
comp. (et
2IL2 22--8L (L
Ae-C
codeen 2010
prrgee Jowr)
Figure F; An example of comparing the code generated by different models with rendered images.
19

---

Published as a conference paper at ICLR 2026
Ground Truth Image
suerag Cuality Metris of Ceomereesce tot
Tetter Lood
Beat
Ground Truth Code
import matplotlib pyplot
import numpy
# Data from the table methods
"Base (GPT-4)'
LLossesss (GPT-4).
Lossless (GPT-3.5)
Semantic (SPT-4)
Semontic (SPT.3.5)
Ziib Deflate Least
ZZiib Deflate Most
6s_ volues (0.923, 0.76, 0.743, 0.936, 0.93, 10, 1.0]
ere. ere_ values [0.61, 0,945, 0, 0,786, 0.622, 0.623, 10,0000,,0000000 0.911
sne voluss (1.0, 0.542, 0,374, 0, 0.949, 0.937, 0.594, 0,94, 0,615]
Create figure axes fig, ax pttsuiploot((iigiiee=(22 B)) # Convent methods to numerical x-oxis volues
== crange((en(methods)
Plot area chart
ox stackepoo((x, cs volues, ere, values, see_vel ('C's 'ERE', 'SRE') alpha=0.
# Add labels and title
t_lobel Method')
ox.set ex.set_ylebel( Metric Value')
ox.set fitiel Average Quality Matrics of Decompressed Text')
set st.setttiick(()
sise___ticckaeelsmeetodss,, rotation:45, rright)
legend
ax www.eeend(lac==loe lupplelcc=puperrroee left') # Set y-axis limits
ex.set_ylim(0,
Adjust based sun
# Add a grid for better readability
ax.gni((xiis: y linestyle=' - alpha=0.7)
layout tight prevent labels everlapping pit.tight Inyout0 # Show the plot pit.shaw()
Figure G: A case of chart-code pairs for area charts in our Chart2Code dataset.
20

---

Published as a conference paper at ICLR 2026
Ground Truth Image
Eancition 0'g27wilfh eps reprosetaaiin))
(Pg27N reprocotiaaionn
26-
Serve
26-
34
2.0
Ground Truth Code
import matplotib pyplot os pit import numpy sp
first
Fig826lUThups))
rparroy(1, 2,
data pointst groupt approoy111 antoy[10,, 12,
errorsi groupl - rparro())) 1.2. 1.2, 11, 11,,1,,1e]
data conn__oooits_rouupupe sparroy([s 9,88,,9,110 7,
erronst groups mp.ooo[0066003 0.9,0 1.1,0.1,,771 10,,12 12p
second plat (comesponding to Flg22bU3k.eps) x <_vvoess2 rearoy(l), data paiteppoitts2 2oints2_ aeriess mgaaroy([20,
if bata
26]
Frrois__eerissA aparroyt[2, 2.5, 2.6])
rearoyll1..
19..22.
data periesC an errors? servecd arroy( 5. 18, 1.9)
Il Createe figure with two supplets (arranged verthoolly) fig. (ax), mx2) = pubpllolots 1, figeize:100 8))
af Piot the first errorbar chart the first susplet (ex))
axd errorborrxx.eeee verlerr color blue")
arr__pontst_graee groupt, yerreerorssggrrwe groupp. labels Group 1', fritee copsizeets
aereerbrbbxx (xrlaas volusss peintp group,, yerrarross group2, lobel- Group 2", fmtt copsizee
rinlaa Condition")
(f FigtZoWUheee representction))
vlobel Measured Value') legend) grie((rrs))
Ji Plat the second emvrbar chart on the second subplot (ax2) ux2 errorsarbx, velues, in___oonss__eeeee. copsize-5, colors greer")
label 'Series A', fmit'"").
axž errorbor(x volues2, deto_ Sartu_oonntt__eeee seriesf,, yermermors? ospeizenfi celors purple')
pormenors?_eerriddd servesd, lobel: Series fintt'
errobborrrr velluee data_pootss2_eneeee yerrsemors2_eeness
xX,
oopsizesh, orange
Lille
(FigaZbbUTh.qp represienttiio)))
xlabel Time
ylobel( Weosured Value') ax2 legend( ax2 Trid(re)
at Adjust loyoul to provent everlap pit.tihtttoo -oyout) show the plo? pit show)
Figure H: case of chart-code pairs for errorbar charts in our Chart2Code dataset.
21

---

Published as a conference paper at ICLR 2026
Ground Truth Image
Ground Truth Code
motplatiib pyplot
deportments: F
"Alta Verrpaa' Bojo Veropaz', "Chimd 'Chiuuimdas' Progreso", "Escunt lo (Gunt Suntemals", Huchuctenerge" Izebal "Jllpp" 'Tericp'. Petén'. Guezzlhennrgg: "Quiché", 'Retahules, Socotepequez" 'Son Meroos" "Sonte Rode Solali" Suchitapaquez Totonicapdn', Zacape"
e- (28,88 1016, 10100
1,020, 1.004.1005 1.064, 1,05,, 1011, 1,006, 1.006, 1,009, 1002,, 1.004, 1.004, 1.004, 1.003, 1010, gnowth rote (0 0.023, 0,030,
0,030, 0.041, 0.033, 0.053, 0.006, 0.049, 0.032)
0.075,
0.038,
corrying_com 1446, 5574, 1087. 697. 1598, 2142, 723, 1518]
[1685, 1258, 1233, 941, 400, 27230, 2509, 2374, 496, 2415, 3048, 1783,
infiection ppoit [142, 159, 96, 306, 57, 82, 85, 171, 129, 146, 82, 156, 120, 169, 125, 102, 78, 111. 141, 129)
Create subplots fiy. uxes pli adggiettttttreeee nools-2, Figsize-(18, 123) axes flatter))
Corrying Capacity
axes(0)
barl daportettsssn cerrying_cspccthy colore skyblle')
oxt set._tttee ylobelf Carrying Capocity (K) axd set hitiee Corrying Capcoity of COVID 19 Coses by Department')
po sxchhcc_ssronsseee and tick
x (ntoohooes0)
exd griddaxisss y. linestyle -.. dipha-0..7 for bor
ccc:
height get.get heightO)
height
Only show widhk/22 height.
txctfrct get al)
"Sd'
fontsizen8)
Platting Inflection Point ex2 oxes[1]
bers_ip ox2 bertckepermments tments, inflection_poitt celors lightcoral))
nxt set ylobel Leffection Point (dnys))
ax2 set_ Tiie inflecttion Point of COVID Cases by Department')
ex2 tick fack_saronsfoxtss x, (etorssiix.ootooooee. rotaaherss0))
axd gridloniee 'y linestyle: diphar0.7)
height bet. height)hhh)))
textibar
birtget_wiitth/2... height.
"Ad" inttheigh"), hez center" bottom'. fontsizes))
Platting Growth Rete ax3 a oxaste]
bere, or ax3 ber(departments, growth_rote. color lightgnsen")
ex3 sel ylabel Growth Rate (-))
exd set hitlel Snowth Rete of COVID-19 dases by Deportment")
tick pank_panass rotatiione0))
ex3 ara.giddoviss' y. linestyle:" dipho:0.7) gr.
height baarget_hhighh?
texttartt
ban.get wicth//2, height,
"0.3r 2. height, har center botton', fomisize-8)
Platting 'o porometer
cxll = ovests)
bens aed sew.berrikpprrtmertt color= goldd)
aod set ylabel Porameter (o))
coo Fithee horometer Vol Richards Model by Department")
awd.rick_panemsfoxerr notatiers90)
324.grijjoiie)" by linestyle d.pha-0.7)
height bon
height) hooight
axe textfkar get _x( wirgget_wwcttg22,,. height,
"2.3r 2. height, has center. betton', fontsize:8)
tight loyout) pit.show()
Figure I: A case chart-code pairs for nar charts in our Chart2Code dataset.
22


Reviews:
[{"review_id": "DGhcQoCxRU", "reviewer": "Reviewer_pbDs", "summary": "This paper introduced Multimodal Structured Reinforcement Learning (MSRL) for the chart-to-code task. The authors first analyze the scaling of the supervised finetuning (SFT) approach and show that the performance plateaus beyond certain data/compute scale. After that, they propose an RL-based solution that breaks this plateau and shows further improvements in performance. The authors propose two sets of rewards: textual and visual. The textual reward focuses on aspects in the generated code while the visual rewards measures the similarity between the reference and rendered chart (from the generated code). To create data for SFT and RL, the authors propose a data generation pipeline that first sources tables from arXiv papers. Then, an LLM uses these tables along with code examples to generate code for new chart images. After that, the codes are rendered into chart images which are also further filtered using GPT4o to keep only high fidelity charts.  The authors evaluate their model on two benchmarks: ChartMimic and ReachQA, and show nice performance improvements over existing methods.", "questions": "see weaknesses above", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* The proposed RL approach shows performance improvements on the chart-to-code task that can’t be achieved by scaling the SFT data/training on its own. These experiments are quite interesting and could have valuable insights. \n\n\n* The proposed model achieves the state-of-the-art results on two chart-to-code benchmarks: ChartMimic and ReachQA.  The authors have provided detailed ablation experiments to show the benefits from each of their proposed techniques/reward functions.", "weaknesses": "* Limited Visual diversity: using a set of example codes and forcing the code to follow a specific structure may significantly limit the visual diversity of the dataset. There’s also no analysis to support the claim of visual diversity compared to existing datasets/approaches. \n\n\n* The evaluation is only limited to two benchmarks: ChartMimic and ReachQA. Furthermore, it’s limited to the niche chart-to-code task. It would strengthen the contribution of the paper if the RL approach can be expanded to other popular chart tasks (e.g., QA) to truly verify the “Breaking the SF Plateau” claim in the title and the paper. \n\n\n* The visual evaluator in the RL approach is quite massive, 72B params. I am not sure why the authors didn’t simply use a small CNN/ViT and just measure the visual similarity between the extracted features. \n\n\n* In the Intro, the authors claim that DPO does not generalize well but they haven’t conducted any experiment to prove that GRPO generalizes better than DPO in this task using the same dataset.\n\n\n* Potential Data Leakage: The ChartMimic benchmark was constructed by scaping data from arXiv and the proposed dataset here also starts by scraping Tables from arXiv. I am concerned that the performance improvements could be tied to data leakage.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9R0L0n0hWU", "reviewer": "Reviewer_vxZ1", "summary": "The paper studies why SFT saturates on chart-to-code and proposes a two stage RL recipe with multi-granularity rewards (textual + visual). The authors build a large chart-code corpus to show SFT plateaus beyond 2M examples, and then use GRPO with (a) execution + rule-based textual checks and (b) a render and compare visual reward scored by a strong MLLM. On ChartMimic and ReachQA, MSRL (7B) establishes new SOTA among open-source models and approaches GPT-4o.", "questions": "- How do results change when visual reward/evaluator is swapped (e.g., InternVL (latest), Gemini, GPT-4.1) and on a small human-scored subset?\n- What exact de-duplication and topical overlap checks were done to ensure arXiv-derived tables don’t appear in or unduly resemble test items in ChartMimic/ReachQA\n- curious to see if MSRL trained on Matplotlib code generalizes to other plotting libraries? Perhaps a notable few?\n\n- Why GRPO? Is it the first thing you tried and worked decently?\n\n\nLastly, some relevant key papers on visual code generation that may have been missed in citations:\n[1] Rodrigez et al, Generating Scalable Vector Graphics Code From Images And Text. https://arxiv.org/abs/2312.11556\n[2] Xia et al, StructChart: Perception, Structuring, Reasoning for Visual Chart Understanding.\n[3] Rodriguez et al, BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks. https://arxiv.org/abs/2412.04626\n[4] Awal et al. WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation https://arxiv.org/abs/2508.16763\n\nHappy to relook at the scores if authors consider some of the points raised.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Clear empirical story about the SFT ceiling. authors isolate SFT scaling curve and convincingly shows a plateau beyond 2M samples before introducing RL, this sort of strengthens the causal claim that RL brings the next jump, not just under-tuned SFT.\n2. The textual reward normalizes code and scores specific fields (data, type, layout, titles/labels, exec) while visual reward compares rendered images against ground truth via an MLLM.. the two-stage schedule is sensible imo and empirically validated.\n3. Strong results: MSRL-7B beats open-source baselines and seems competitive with GPT-4o on both datasets.. detailed low-level metrics suggest real gains, not just execution hacks.\n4. Repruducible details specified", "weaknesses": "- Clearly the visual reward depends on Qwen2.5-VL as judge. If the policy aligns to judge's biases or defects, improvements could reflect evaluator gaming rather than genuine fidelity. Perhaps a judge-swap test (e.g., different MLLM/human) is needed to rule out any judge overfitting.\n- The entire pipeline centers on Matplotlib-style code and a fixed rendering toolchain. It is unclear whether the learned behaviors transfer to other plotting libraries (Seaborn/plotly or vega) or different runtimes\n- Real-world tables are scraped from arxiv, but the paper doesn’t quantify overlap with benchmarks. If time allows, data contamination studies are a must these days.\n- Also, some light on whether the gains are from the reward design vs. the specific optimizer (grpo vs dpo) would be great.\n- [Optional but important] The abstract is quite long, and the third paragraph in the introduction (starting around line 75) is similarly lengthy. Try to simplify where possible. Avoid using multiple wrap figures unless absolutely necessary.. if you can, place Table 3 and Table 4 side by side. Also the figure and table captions are too brief and not self-contained. Figure 5 is impossible to read at first glance. It would be good to do a dedicated pass focused on improving writing clarity and presentation quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C8eOJ8PizH", "reviewer": "Reviewer_iWJj", "summary": "This paper targets the task of chart-to-code generation, where the model produce excuteable plotting code given a chart image.\n\nThe authors observe a performance plateau with supervised fine-tuning (SFT), To overcome this, they propose Multimodal Structured RL (MSRL), a GRPO based reinforcement learning approach with multi-granularity rewards. Training follows a two-stage curriculum: RL on textual feedback, then on combined textual+visual feedback. \n\nThe paper claims state-of-the-art results on the ChartMimic and ReachQA benchmarks, with a ~6–10% gain in high-level accuracy over SFT alone.", "questions": "* Did you try cross-judge or human judge is done to visual rewards, (e.g., swapping the 72B judge with a different model or use gpt-4o or human to cross-judge qwen2.5-vl-72B, especially at late stage RL, where reward hacking is most possible?\n* What's the reason for RL with visual rewards for only 100 steps where the rewards are still gaining fast, is the limit stability, compute, or hacking?\n* [weakness #3] Could you disentangle curriculum effects from training budget by reporting a compute-matched comparison?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Clear diagnosis of the SFT plateau. The scaling curve explicitly plateaus after ~2M SFT examples, and shows visual RL breaks the curve.\n* Large curated and balanced chart‑code corpus from real‑world arXiv is great contribution to the community.\n* MSRL (7B) outperforms ChartCoder and is competitive with GPT‑4o on component scores such as layout and text (Tables 1–2, p. 6), with qualitative cases showing better fidelity and execution reliability than open and proprietary models", "weaknesses": "* The visual reward (and RL data filtering) rely on a single MLLM judge (Qwen2.5‑VL‑72B). Without cross‑judge verification or human studies, there is a very risk of reward hacking or bias toward that evaluator’s preferences.\n* Possible double‑counting of execution success in stage 2, in Sec 3.3 R = w_t*R_text + w_v*R_vis + W_e*R_exec, R_text also contains R_exec\n* Compute-normalized ablation for two-stage vs single-stage: It looks possible that the two-stage curriculum outperforms single-stage primarily because it uses a larger update budget (trained on more steps).\n* Several relevant papers are not cited :\n    - [1] **Rendering-Aware Reinforcement Learning for Vector Graphics Generation**, Neurips 2025.\n    - [2] **MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization**, ACL Findings 2024.\n    - [3] **A Survey on LLM-as-a-Judge, arXiv:2411.15594**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sYRgZ2ellU", "reviewer": "Reviewer_npBT", "summary": "The paper focuses on chart-to-code generation. The authors built a large chart-code corpus and found that SFT hits a performance plateau when training data is scaled, with more data bringing little improvement. To address this, the authors used MSRL to fine-tune MLLMs and design task-specific rewards. The MSRL method achieved better performance than SFT. Moreover, the trained MLLMs outperform most baselines.", "questions": "Tthere are some questions and minor issues do not affect my overall assessment:​\n1. For Figure C, is there any reference for the selection of these scores?​\n2. Regarding citations, the authors have cited a large number of arXiv papers. In fact, many of these papers have been published in conferences, so attention should be paid to the correctness of citations.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The methods of data synthesis and reinforcement learning are intuitive and interesting, providing a certain basis for solving the problems in chart interpretation tasks.​\n2. In the chart-to-code task, the model's performance has obvious advantages compared with open-source models, demonstrating the effectiveness of some designs in the paper.​\n3. Sufficient experiments are conducted, including comparative experiments, ablation studies, and qualitative experiments, which help to verify the related hypotheses and results.​\n4. The preliminary study on the performance of SFT has certain reference value within the scope of this task, and it is natural to introduce RL based on this experiment, which ensures the logical continuity of the research.", "weaknesses": "1. On line 52, the description of the significance of the chart-to-code task is not convincing enough. The authors state that this task is \"complex\" but fail to highlight its strong \"significance\". Although it is based on previous research, it may arouse readers' questions about its application value (e.g., help MLLMs better understand charts or assist researchers). In addition, the connection between this part and the previous paragraph is not smooth.​\n2. In the preliminary study (Figure 1), the SFT plateau and RL performance gain are shown, but the study on the change of RL performance with the increase of data size is missing. Since SFT will converge when the training data increases to a certain extent, according to the authors' motivation, RL should converge later, but this comparison is lacking. Although I noticed Figures 3 and 4, they only show the change curves of reward. The scale law of specific training data on performance is important and can emphasize the motivation of this paper.​\n3. For methods, the main contributions of this paper lie in data synthesis and reward design. The GRPO used is in a standard form (lines 259-269). This not only reduces the innovation of the paper in terms of contributions but also makes the excessive space spent on introducing non-original content unnecessary in writing.​\n4. The experimental results support the point 3 that the contribution of synthetic data is significant, which should be the core of this paper. However, the authors do not produce detailed discussion and analysis on data synthesis, usage, scaling law, etc.​\n5. In Table 4, the effects of Textual and Textual+Visual are close, but there is no in-depth discussion on this in the paper, which weakens the rationality and effectiveness of the method design.​\n6. Table 5 has a similar problem. The results of each variant are close (~1%), but the paper only lists the results without in-depth analysis.​\n\nIn general, **the research motivation of this paper needs to be strengthened, the experiments supporting the motivation need to be more completely supplemented, the focus of the designed method is biased, and although the experiments are sufficient, the support and analysis for the method design and problem solving are insufficient.**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)