T ext2 C hart31: Instruction Tuning for Chart Generation with Automatic Feedback (2024)

Abstract: Large language models (LLMs) have demonstrated strong capabilities across various language tasks, notably through instruction-tuning methods. However, LLMs face challenges in visualizing complex, real-world data through charts and plots. Firstly, existing datasets rarely cover a full range of chart types, such as 3D, volumetric, and gridded charts. Secondly, supervised fine-tuning methods do not fully leverage the intricate relationships within rich datasets, including text, code, and figures. To address these challenges, we propose a hierarchical pipeline and a new dataset for chart generation. Our dataset, Text2Chart31, includes 31 unique plot types referring to the Matplotlib library, with 11.1K tuples of descriptions, code, data tables, and plots. Moreover, we introduce a reinforcement learning-based instruction tuning technique for chart generation tasks without requiring human feedback. Our experiments show that this approach significantly enhances the model performance, enabling smaller models to outperform larger open-source models and be comparable to state-of-the-art proprietary models in data visualization tasks.

## introduction recently, a range of nlp tasks has been addressed by leveraging the remarkable ability of large language models (llms). this advancement has been possible largely through the process of instructiontuning @xcite , which fine-tunes llms to rely on intuitive natural language instructions and skillfully solve intricate tasks, encompassing fields like question answering @xcite , summarizing @xcite , and sentiment analysis @xcite . however, available llms continue to suffer from the difficult tasks of visualizing complex, fact-based, real-world data through charts and plots, mainly because of two challenges. firstly, the current datasets @xcite @xcite @xcite primarily focus on qa in the chart domain rather than chart generation, and they rarely cover a full range of chart types and their varied applications. several chart forms like 3d, volumetric, gridded, and irregularly gridded remain largely unexplored or insufficiently studied. these forms are important for evaluating the capabilities of llms in under-standing multidimensional data, spatial data, and vector field data. developing such instructional datasets typically entails significant expenses due to the complex nature of text-to-chart processes, incorporating various data components such as text, code, and data tables. this complexity, along with the lack of specific online sources containing these plot types, makes their collection difficult and timeconsuming. it necessitates human expert intervention to ensure quality, which drives up costs. secondly, existing instruction-tuning methods based on supervised fine-tuning do not fully utilize the potential of rich datasets; for example, chart data include multiple components like text descriptions, code, and figures. supervised fine-tuning struggles to effectively extract and leverage all the intricate information and relationships within these components, leading to suboptimal performance. to address the first challenge, we propose a novel hierarchical pipeline for chart generation by leveraging the advanced linguistic skills of gpt-3.5-turbo @xcite and code generation and data analysis capabilities of gpt-4-0613 @xcite . we contribute a dataset encompassing 31 unique plot types from the matplotlib library @xcite , featuring 11.1k tuples that combine descriptions, code, data tables, and plots, covering a wide range of use cases. our pipeline is structured into the following steps: topic generation, description creation, code production, data table and reasoning step formulation, and cycle consistency verification. this approach reduces biases towards common topics or plot types, and ensures consistent and accurate generation of multiple data elements. by minimizing the human supervision in our proposed pipeline, we can generate a highquality large-scale dataset that includes comprehensive descriptions, codes, data tables, reasoning steps, and illustrated graphs. we further propose a novel reinforcement learning-based instruction tuning technique to address the second challenge. this method is tailored for chart generation tasks without costly human feedback. we propose two different reward functions: the preference reward and alignment reward. for the preference reward, we construct a preference dataset from the supervised fine-tuned model's output and the ground truth code. for the alignment reward, we optimize the model to increase the similarity between ground truth description and regenerated description from the code, exploiting the cycle consistency between code and description. we jointly optimize two sequential policy models using the ppo @xcite . finally, we make the following contributions: ## text2chart31 dataset our newly contributed text2chart31 dataset supports 31 plot types based on matplotlib with 11.1k data points. we outline its key characteristics in for the dataset, we develop a hierarchical plot generation pipeline leveraging gpt-3.5-turbo and gpt-4. despite their impressive capabilities for text and code generation, collecting high-quality data points is challenging for two primary reasons: ## instruction tuning approach we discuss our proposed instruction tuning methods for fine-tuning llms to tackle the three data visualization tasks: (1) description-to-chart, (2) raw-data-to-chart, and (3) code-to-description, using the text2chart31 dataset. we respectively denote three specialized models for the three tasks: π θ 1 , π θ 2 , and π θ 3 . we train these models with two phases: supervised fine-tuning (sft), followed by reinforcement learning (rl) with two types of reward that are specifically tailored to improve chart generation performance. initially, all three tasks undergo supervised fine-tuning. afterward, using ppo algorithm @xcite , we jointly optimize π θ 1 with the preference reward and π θ 3 with the alignment reward that ensures cycle consistency and coherence of outputs. algorithm 1 summarizes the overall procedure. ## experiments baselines. for the evaluation of the three target tasks, we compare with the state-of-the-art opensource baseline models as follows: (i) descriptionto-chart: code llama instruct (rozière et al., 2024), llama 3 instruct (meta ai, 2024), starcoder @xcite , and instruct codegen @xcite , (ii) raw data-to-description: llama 2 chat @xcite and llama 3 instruct model, and (iii) code-to-description: code llama, llama 2 chat, and llama 3 instruct models. we also compare with proprietary models including gpt-3.5-turbo @xcite , gpt-4-0613, gpt-4-turbo-2024-04-09 (openai, 2023), gpt-4o-2024-05-13 (openai, 2024), and claude 3 opus @xcite . evaluation metrics. for the three target tasks, we report the following evaluation measures. ## related work chart datasets. there are several existing chart datasets, including plotqa @xcite , chartqa @xcite , figureqa @xcite , unichart @xcite , autochart @xcite , chart-to-text @xcite . these datasets primarily focus on question and answer (qa) tasks on a limited range of plot types. more recently, chartllama @xcite proposes a text-to-chart dataset that includes qa tasks and generates visualization code from provided descriptions. however, these datasets still lack coverage in certain plot categories such as 3d/volumetric plots and vector field plots, and they do not cover the use case of analyzing the raw data and predicting the most suitable plot types. on the other hand, our text2chart31 dataset encompasses 31 plot types with 11.1k tuples that combine descriptions, code, data tables, and plots, thereby covering a wide range of use cases. instruction tuning. employing reinforcement learning with human feedback is a prevalent strategy for enhancing (un)supervised finetuned models, whether by integrating human feedback into the learning loop @xcite or by leveraging preference data generated by human @xcite @xcite . however, we argue that this methodology might not offer the most practical solution for plot visualization tasks, given the intricate and fact-intensive nature of plot types. moreover, considering the limitations of human cognition, there is a risk of overlooking crucial small details essential for validating the accuracy of generated plots. to address this, we propose a novel automatic method that constructs a preference dataset using supervised fine-tuned output. cycle consistency. exploiting cycle consistency to enhance the performance of the generative model has been mainly studied in the image domain @xcite . recently, ddpo @xcite adopts the llava model @xcite to increase the alignment between the image and the text. following this line of research, we propose an alignment reward that exploits cycle consistency between description and code to improve llm for chart generation tasks. this is made possible because of the rich nature of our text2chart31 dataset, which consists of diverse textual modalities, including visualization code and description. ## conclusion we introduce a novel hierarchical pipeline and a comprehensive dataset for chart generation. the proposed text2chart31 dataset, encompassing 31 unique plot types, provides a robust foundation for diverse visualization tasks with its 11.1k tuples of descriptions, code, data tables, and plots. additionally, we proposed an rl-based instruction tuning technique employing preference and alignment rewards, improving llms in data visualization.