Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach (2025)

Abstract: Large language models can translate natural-language chart descriptions into runnable code, yet approximately 15% of the generated scripts still fail to execute, even after supervised fine-tuning and reinforcement learning. We investigate whether this persistent error rate stems from model limitations or from reliance on a single-prompt design. To explore this, we propose a lightweight multi-agent pipeline that separates drafting, execution, repair, and judgment, using only an off-the-shelf GPT-4o-mini model. On the Text2Chart31 benchmark, our system reduces execution errors to 4.5% within three repair iterations, outperforming the strongest fine-tuned baseline by nearly 5 percentage points while requiring significantly less compute. Similar performance is observed on the ChartX benchmark, with an error rate of 4.6%, demonstrating strong generalization. Under current benchmarks, execution success appears largely solved. However, manual review reveals that 6 out of 100 sampled charts contain hallucinations, and an LLM-based accessibility audit shows that only 33.3% (Text2Chart31) and 7.2% (ChartX) of generated charts satisfy basic colorblindness guidelines. These findings suggest that future work should shift focus from execution reliability toward improving chart aesthetics, semantic fidelity, and accessibility.

## introduction natural language chart generation turns a request such as plot quarterly revenue by region into runnable code and a complete figure. a reliable solution would let non-programmers explore data, shorten the loops of professional analysts, and (potentially) create alternative renderings for lowvision users that are more accessible to screen readers. two conditions decide whether the community can call the task solved: the generated script must execute without error, and the visual output must match the description. one public benchmark has guided much of the recent progress. the dataset text2chart31 @xcite ) measures exact failure rates across thirty-one plot types along with various code similarity metrics. zero-shot prompts for large language models (llms) initially failed on about forty percent of inputs. supervised finetuning and reinforcement learning reduced that number below fifteen percent and brought pairwise plots close to six percent. prompt engineering improved results a little more @xcite @xcite . closed-source models such as gpt-4-turbo and claude3opus achieved similar levels, yet stubborn errors remained for complex surfaces and volumetric plots. work on chart captioning and visual question answering @xcite @xcite @xcite shows that even a small execution failure can break an entire downstream pipeline. instead of one large prompt, agent-based systems use several small prompts that plan, call tools, and verify interim results @xcite . code-repair agents update faulty scripts after runtime errors and often succeed within a few steps @xcite @xcite . this study asks two research questions. first, whether chart generation, defined in its present benchmark setting, can be improved using a simple agent-based framework. if the answer is yes, the field would need to revisit its evaluation criteria, shift resources away from training larger models for pure execution, and retire the current benchmarks in favour of more difficult scenarios such as noisy data or multi-step analytic sessions. second, if using traditional metrics and our approach effectively "solves", what objectives should guide new work when execution errors are already rare? for general image generation, this could extend chart evaluation work to move beyond a binary "runs or not" view. structural similarity (ssim) scores reflect low-level alignment. at the same time, multimodal llm judges rate perceptual fidelity and semantic match @xcite @xcite . furthermore, a focus on aesthetics, readability, and accessibility could produce charts that follow colorcontrast guidelines for color-blind users, place legends to avoid overlap, and include alt-text or tactile representations for screen readers and embossing devices. such qualities matter for journalism, education, and government dashboards, where the chart must inform a wide audience rather than only run without crashing. to answer these questions, we developed a lightweight multi-agent pipeline that uses an offthe-shelf gpt-4o-mini model without any additional training. the pipeline was also tested with three open-source models (llama 3.1, gemma 3, and qwen 3). a draft agent generates python/-matplotlib code from natural-language chart descriptions, and a repair agent iteratively debugs and rewrites the code-up to three times-when execution fails. on both text2chart31 and chartx, the system reduces execution errors by nearly 5 absolute percentage points, outperforming fine-tuned baselines while preserving image quality as measured by ssim and multimodal llm judgment, and similar patterns were evident with vistext. a manual analysis of 100 sampled outputs found that 83% were accurate, with most remaining issues related to minor stylistic differences rather than semantic or data errors. these findings suggest that agentic pipelines are not only more robust than single-shot prompting but also capable of producing high-quality, visually faithful charts. overall, we make the following contributions. ## related work chart generation and evaluation. large language models (llms) are increasingly used to automate chart generation from natural language, but major challenges persist in quality control and evaluation. datasets such as chartqa, plotqa, chartllama, and charxiv @xcite @xcite establish benchmarks for chart reasoning, captioning, and data extraction. these corpora focus primarily on chart understanding and question answering. ford et al. (2025) and ye et al. (2024) also examine chart captioning and visual qa. to address text-to-chart generation directly, text2chart31, chartx, and vistext pair instructions with both code and image outputs, allowing complete evaluation pipelines @xcite @xcite . still, these datasets remain small and often require human references. prompt engineering, cot prompting @xcite , and imagebased analysis methods (e.g., ssim) have been proposed to assess generated charts @xcite . however, hallucinated or stylistically inconsistent outputs remain common @xcite . moreover, recent work questions the reliability of multimodal llm judges for chart evaluation @xcite . this motivates the development of task-specific evaluation protocols combining low-level visual similarity (e.g., ssim) as well as using llm-as-a-judge @xcite . ## data we use three benchmark datasets. the statistics for the datasets used in our study are shown in text2chart31. the text2chart31 dataset (pesaran zadeh et al., 2024) consists of 11,128 data points with 31 types of plots, providing a robust environment to test the benefits of agentic visualizations. the dataset contains text plot descriptions, python code to create a charts, image files of the created charts, and (for 8,166 data points) csv data files as well as reasoning steps. the dataset is seg-mented into 9,705 data points in the training set and 1,423 in the test set (for the purpose of this current study, only the test set was used). the dataset was created synthetically through the use of gpt-3.5turbo and gpt-4. topics span nearly 50 scientific, political, economic, and popular culture subjects. the 31 chart types are distributed in five categories, including pairwise, statistical, gridded, irregularly gridded, and 3d/volumetric data. chartx. the second dataset is chartx @xcite . this benchmark data consists of 4,848 synthetically-generated chart examples in the validation set and 1,152 in the test set (similarly, for the purpose of this current study, only the test set was used), spanning 18 chart types. a total of 22 topics are represented in the charts, with similar cultural, economic, political, and scientific themes as the first dataset. the data file contains text descriptions of the charts, python code, the raw data points, and chart image files. vistext. the third dataset is vistext @xcite . this benchmark data consists of 12,441 bar, line and area charts constructed using real-world data from the statista.com pertaining to topics such as technology, economics, and sports. the test set was used in this study, totaling 1,764 charts. the data file consists of text descriptions of the charts including the data points and chart image files. note that the text descriptions of the charts were created automatically rather than human-generated. ## methodology figure 1 provides a high-level overview of our paper. our framework has three main components. first, we use a drafting agent that transforms the text into code to generate the chart. second, the code is evaluated using a python interpreter, and any resulting errors are passed to a re-writer agent; otherwise, if there is an error, the final chart is returned. if there was an error, the re-writer agent will try to fix the error. this process is repeated until the code runs or a maximum number of iterations is hit. we describe each part in detail below. step 1: drafting agent. first, we generated baseline python code from the provided text descriptions in the dataset using the closed-source gpt4o-mini llm. we conducted both zero-shot and few-shot runs for the baseline. in the zero-shot setting, the llms receive only the task instructions and the data without examples. the prompt follows the format from the text2chart31 paper @xcite . specifically, the system prompt instructs the model to generate python code for data visualization as follows: ## conclusion we presented a lightweight agentic framework that significantly improves text-to-chart generation by reducing execution errors while preserving image quality, all without fine-tuning or reinforcement learning. our analysis shows that single-prompt llms (whether zero-or few-shot) still fail frequently, whereas multi-agent repair loops offer more robust performance at low cost. beyond execution, we highlight the need to shift focus toward semantic fidelity, visual clarity, and accessibility. paired with image quality analysis, agent-based approaches can help address these broader challenges. future work should explore how these methods can enhance real-world applications, especially for users with visual or cognitive impairments @xcite .