PlotGen: Multi-Agent LLM-based Scientific Data Visualization via
  Multimodal Feedback
Authors: name: Kanika Goswami; name: Puneet Mathur; name: Ryan Rossi; name: Franck Dernoncourt
Published: 2025-02-03 02:00:29

Abstract: Scientific data visualization is pivotal for transforming raw data into
comprehensible visual representations, enabling pattern recognition,
forecasting, and the presentation of data-driven insights. However, novice
users often face difficulties due to the complexity of selecting appropriate
tools and mastering visualization techniques. Large Language Models (LLMs) have
recently demonstrated potential in assisting code generation, though they
struggle with accuracy and require iterative debugging. In this paper, we
propose PlotGen, a novel multi-agent framework aimed at automating the creation
of precise scientific visualizations. PlotGen orchestrates multiple LLM-based
agents, including a Query Planning Agent that breaks down complex user requests
into executable steps, a Code Generation Agent that converts pseudocode into
executable Python code, and three retrieval feedback agents - a Numeric
Feedback Agent, a Lexical Feedback Agent, and a Visual Feedback Agent - that
leverage multimodal LLMs to iteratively refine the data accuracy, textual
labels, and visual correctness of generated plots via self-reflection.
Extensive experiments show that PlotGen outperforms strong baselines, achieving
a 4-6 percent improvement on the MatPlotBench dataset, leading to enhanced user
trust in LLM-generated visualizations and improved novice productivity due to a
reduction in debugging time needed for plot errors.