Title: Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation
Primary area: foundation or frontier models, including LLMs
Keywords: Chart-to-Code Generation, Reinforcement Learning

Abstract: While reinforcement learning (RL) has proven highly effective for general reasoning in vision-language models, its application to tasks requiring in-depth understanding of information-rich images and the generation of structured outputs remains underexplored. Chart-to-code generation is a prime example of this challenge, as it demands complex reasoning over visual charts to generate structured code. Therefore, supervised fine-tuning (SFT) alone is often insufficient, highlighting a critical need for effective RL strategies that appropriately reward the structured outputs.
In this paper, we systematically investigate the performance plateau in SFT through large-scale experiments and propose Multimodal Structured Reinforcement Learning (MSRL) for chart-to-code generation, which substantially breaks through this plateau. To this end, we first construct the largest training corpus to date, containing 3 million chart-code pairs. To mitigate the simplistic patterns of prior synthetic data, we curate real-world tables from arXiv papers as the data source. Despite reaching a state-of-the-art performance, our experiments demonstrate that simply scaling the SFT data eventually hits a performance plateau where further increases yield negligible improvements. To break this plateau, our MSRL method leverages a multi-granularity structured reward system that uses multimodal textual and visual feedback to guide the RL training. At the textual level, rule-based rewards validate the fidelity of fine-grained code details. At the visual level, a model-based reward is derived by first rendering the generated code into an image and then employing an evaluator model to assess its structural similarity to the ground-truth chart. To ensure training stability, we implement this multimodal reward within a two-stage curriculum, where the model initially trains on textual rewards alone before progressing to the combined textual and visual signals.
Experimental results demonstrate that our MSRL framework significantly breaks the SFT plateau, improving the high-level metrics by 6.2% and 9.9% on ChartMimic and ReachQA benchmarks respectively. Notably, our method significantly outperforms all methods in the chart domain and achieves competitive performance with advanced closed-source models for the first time.