QUIS : Question-guided Insights Generation for Automated Exploratory Data Analysis (2024)

Abstract: Discovering meaningful insights from a large dataset, known as Exploratory Data Analysis (EDA), is a challenging task that requires thorough exploration and analysis of the data. Automated Data Exploration (ADE) systems use goal-oriented methods with Large Language Models and Reinforcement Learning towards full automation. However, these methods require human involvement to anticipate goals that may limit insight extraction, while fully automated systems demand significant computational resources and retraining for new datasets. We introduce QUIS, a fully automated EDA system that operates in two stages: insight generation (ISGen) driven by question generation (QUGen). The QUGen module generates questions in iterations, refining them from previous iterations to enhance coverage without human intervention or manually curated examples. The ISGen module analyzes data to produce multiple relevant insights in response to each question, requiring no prior training and enabling QUIS to adapt to new datasets.

## introduction exploratory data analysis (eda) is the process of discovering meaningful insights from vast amounts of data, and it is a complex task requiring careful data exploration. there are various eda techniques to uncover insights by analyzing patterns in the data. automated data exploration (ade) systems accelerate the eda process through automation. ade literature includes statistics-based @xcite @xcite @xcite and interactive methods @xcite @xcite , where users explore data through natural language queries or receive suggestions for subsequent actions. visualization-based * work done as part of internship at ibm research, india. ade using reinforcement learning is studied @xcite @xcite @xcite to achieve full automation. while these systems minimize user involvement, they often demand dataset-specific training and substantial computational resources, particularly as the number of features, categorical values, or patterns increases, making the process increasingly challenging. ## preliminaries although it is challenging to precisely define the notion of an insight due to variations in users' objectives, for this work, we adopt the definition of an insight consistent with previous studies @xcite . consider a tabular dataset @xmath0, x 2 , . . . x n } where each x i is an attribute (column) of the dataset. an insight, denoted by insight(b, m, s, p ), consists of the following: the quis system incorporates the following insight types as candidates for our patterns: ## methods the eda process is often guided by the questions that arise from the semantic context and the statis-tical properties of the dataset. hence, we propose an approach, quis (question-guided insight generation), that employs a two-stage process (refer to figure 1 ). the first stage, qugen, leverages llms to formulate questions based on the dataset schema, basic statistics, and iteratively updated in-context examples. the second stage, questiondriven insight generation (isgen), systematically analyzes the tabular data statistics based on the questions to uncover meaningful insights. ## experimental evaluation in our study, we evaluated the quis pipeline's effectiveness using human assessment and insight scores on three datasets: sales @xcite , adidas sale @xcite and employee attrition @xcite . human evaluation focused on the individual insights assessing relevance, comprehensibility, and informativeness (details in appendix c). we tested two conditions: replicating prior work to establish robust baselines @xcite @xcite is challenging due to the lack of available code, datasets, and implementation details. additionally, the differences in insight types and presentation formats across existing approaches make direct comparisons difficult. therefore, our main focus is on comparing quis, against the baseline onlystats. for further information about the parameters of the experimental conditions, please refer to appendix d. the insights were evaluated by six participants who are well-versed in data analysis, with each insight assessed by three different evaluators. each criterion -relevance, comprehensibility, and informativeness -was rated on a scale of 1 to 5; where 1 indicated the insight was not relevant, comprehensible, or informative; and 5 indicated the insight was highly relevant, comprehensible, or informative. ## conclusion & future work eda systems often rely on user-generated, goaloriented questions, which means the quality of the generated insights depends solely on these input questions, introducing potential overhead. to address this limitation, we propose a fully automated eda system that generates dataset-specific questions automatically and performs insight discovery. this system operates in a data-agnostic manner, requiring no prior training, thereby minimizing the dependency on user input and streamlining the overall insight discovery process. as a future work, we propose to enhance qugen to generate questions in chunks where isgen processes each chunk of questions before qugen generates the next chunk. this would enable qugen to use insights and their scores from previous chunks to inform the generation of subsequent chunks. additionally, we will explore incorporating other types of insights as future work. for example, we aim to include outlier in time-series, anomaly detection, predictive insights and trend reversal to further enhance the variety and depth of insights generated by the quis system.