D ata N arrative: Automated Data-Driven Storytelling with Visualizations and Texts (2024)

Abstract: Data-driven storytelling is a powerful method for conveying insights by combining narrative techniques with visualizations and text. These stories integrate visual aids, such as highlighted bars and lines in charts, along with textual annotations explaining insights. However, creating such stories requires a deep understanding of the data and meticulous narrative planning, often necessitating human intervention, which can be time-consuming and mentally taxing. While Large Language Models (LLMs) excel in various NLP tasks, their ability to generate coherent and comprehensive data stories remains underexplored. In this work, we introduce a novel task for data story generation and a benchmark containing 1,449 stories from diverse sources. To address the challenges of crafting coherent data stories, we propose a multi-agent framework employing two LLM agents designed to replicate the human storytelling process: one for understanding and describing the data (Reflection), generating the outline, and narration, and another for verification at each intermediary step. While our agentic framework generally outperforms non-agentic counterparts in both model-based and human evaluations, the results also reveal unique challenges in data story generation.

## introduction visual data stories have emerged as a powerful medium for communicating data, effectively combining the strengths of visualizations and text to convey contextual information and causal relationships @xcite . ranging from data scientists to business analysts to journalists, people frequently write data-driven reports that integrate charts and text to present information to readers in a clear, coherent and visually engaging manner @xcite . the essence of a visual data story involves identifying compelling insights within data ("story pieces"), presenting figure 1 : an example data story in our corpus extracted from gapminder @xcite them through visualizations and texts, and arranging these representations into a coherent narrative that communicates an overarching message @xcite . well-crafted visual stories have the potential to significantly enhance data understanding, even for those without specialized technical backgrounds. by combining narrative with data visualization, authors can illustrate trends, highlight correlations, and uncover hidden insights that might be lost in dense tables or reports. for example, fig. 1 shows a gapminder data story @xcite in which renowned storyteller hans rosling explained how birth rates in the world have changed over time using text and charts. data storytelling is widely used across various companies, including microsoft and tableau to effectively communicate insights and drive decisionmaking. in business intelligence, it can help present sales trends and performance metrics, while in healthcare, it can help illustrate patient outcomes and track disease outbreaks. marketers can lever-age it to optimize strategies through customer behavior visualization, and financial analysts use it for investment performance and risk assessments. in education, it can help track students' performance highlighting areas where they excel and where they might need additional support, and in public policy, it can communicate the impact of policies on social issues, for instance, a data story could show how a new housing policy affected homelessness rates, providing evidence-based insights to policymakers and the public alike. despite the popularity of data-driven stories, crafting them remains challenging and timeconsuming, requiring skills in data analysis, visualization, graphic design, and storytelling. to facilitate data-driven storytelling, extensive research has introduced new concepts, theories, and tools. for instance, @xcite explored different design spaces from a narrative structure point of view, while others @xcite @xcite focused on visual representations for crafting visual stories, tailoring their approaches based on specific tasks and communication objectives. while insightful and coherent, manually created data stories require significant human effort and time. in response, efforts have been made to develop automated methods for generating data stories @xcite @xcite , but these often produce simple facts lacking in quality and engaging narratives. the rise of llms has prompted researchers to explore their effectiveness in tasks like chart summarization @xcite , chart question answering @xcite and natural language story generation @xcite . however, the ability of llms to generate stories from data tables and to understand their effectiveness remains largely unexplored partly because of the lack of a benchmark dataset. to address the research gap, we first develop a new task and the corresponding benchmark consisting of 1,449 data stories collected from realworld sources. motivated by the impressive performance of llm-based agents in various planning tasks @xcite @xcite @xcite , we then propose an agentic framework which takes data tables as inputs and employs two llm agents -a generator or actor and an evaluator or critic -to mimic the human process of data story generation through writing and revising based on critic's feedback (figure 2 ). the process includes a planning step (reflection and outline generation) and a story generation step (narration), with each step verified and revised by the critic llm, creating a feedback loop to ensure coherence and factual consistency. experimental results show that our agentic framework outperforms non-agentic llm counterparts in terms of generating more insightful and coherent stories with better resemblance to human-written narratives. our main contributions include: (i) a new automatic data story generation task and a corresponding benchmark dataset, (ii) a multi-step llm-agent framework for data story generation. (iii) extensive automatic and human evaluations that demonstrate the state-of-the-art performance of data-narrative. we make our code and data story corpus publicly available at here. ## benchmark construction given the lack of a benchmark for automated data storytelling, we started by exhaustively searching across diverse online sources such as news sites, visualization repositories, and data blog sites. at the end, we chose three suitable sources that contain data stories covering a series of visualizations and texts as we described below. ## conclusion and future work we present datanarrative, a new benchmark for multimodal data story generation that combines text generation, data analysis, and information visualization. our benchmark includes 1,449 diverse data stories with open-ended multimodal outputs, each featuring various charts and related texts. we then propose an llm-agent-based story generation framework that mimics the human process of creating data stories by using a generator and an evaluator agent. our experiments show that this framework generally outperforms the direct method in both automatic and human evaluations. the study also highlights unique challenges in multimodal long-form data story generation, such as the difficulty of building open-source models that generate long, coherent stories with rich narratives. to address this, we release a training corpus for the community to explore fine-tuning opensource models for this task. additionally, our agentic framework can serve as a foundation for humanin-the-loop co-authoring of data stories with llms, where humans act as critics, collaborating and coediting with the llm to create coherent and informative stories. we hope our research inspires further work in multimodal data storytelling.