QTS umm: Query-Focused Summarization over Tabular Data (2023)

Abstract: People primarily consult tables to conduct data analysis or answer specific questions. Text generation systems that can provide accurate table summaries tailored to users’ information needs can facilitate more efficient access to relevant data insights. Motivated by this, we define a new query-focused table summarization task, where text generation models have to perform human-like reasoning and analysis over the given table to generate a tailored summary. We introduce a new benchmark named QTSumm for this task, which contains 7,111 human-annotated query-summary pairs over 2,934 tables covering diverse topics. We investigate a set of strong baselines on QTSumm, including text generation, table-to-text generation, and large language models. Experimental results and manual analysis reveal that the new task presents significant challenges in table-to-text generation for future research. Moreover, we propose a new approach named ReFactor, to retrieve and reason over query-relevant information from tabular data to generate several natural language facts. Experimental results demonstrate that ReFactor can bring effective improvements to baselines by concatenating the generated facts to the model input. Our data and code are publicly available at https://github.com/yale-nlp/QTSumm.

## introduction in the era of data-driven decision-making, tabular data plays a crucial role in facilitating data analysis, serving as a concise and structured representation of information @xcite @xcite @xcite . people often consult tables to extract valuable insights and make informed decisions. for example, sales managers typically explore large tables with specific business questions to gain insights about clients and processes. sports coaches will analyze performance figure 1 : an example of qtsumm. given the numerous data points in the table, different users may be interested in various aspects for their own informationseeking or decision-making purposes. the system needs to perform human-like reasoning and analysis over relevant table regions to generate a tailored table summary. existing work and datasets on table -to-text generation @xcite @xcite @xcite have mainly focused on converting tabular data into coherent statements, aiming to present the structured data in a humanreadable format. however, these approaches have overlooked the fundamental goal of addressing users' information-seeking purposes. table -to-text generation systems should adopt a more flexible and interactive approach that allows people to obtain a user-customized summary tailored to their information needs @xcite @xcite @xcite , as illustrated in figure 1 . while table question answering (qa) @xcite @xcite @xcite has made significant progress in answering fact-based questions, the primary focus of their approaches is on extracting relevant facts or entities from the table and composing short-form answers. nevertheless, in real-world scenarios, users often have more complex and diverse information needs that extend beyond simple fact retrieval. they expect models to perform human-like reasoning and provide trustworthy explanations or analyses that accompany the extracted insights. with comprehensive consideration of the realworld information needs of users when consulting tabular data, we propose a new task, query-focused table summarization. in this task, the model is required to generate a user-customized summary given the table and user query. to enable research in this area, we construct a human-annotated tableto-text generation dataset named qtsumm foot_0 , that contains 7,111 query-summary pairs over 2,934 wikipedia tables covering diverse topics. we provide a comprehensive evaluation of current state-of-the-art models, including text generation @xcite @xcite , table-to-text generation @xcite @xcite , and large language models @xcite @xcite @xcite . our results and analysis from different perspectives reveal that the existing models struggle in solving this new task, highlighting the challenges the models face when performing human-like reasoning and analysis to generate summary tailored to users' information needs. to improve both text generation systems for qt-summ, we propose refactor. given a user query, refactor can retrieve and reason over query-relevant facts from the source table to generate multiple data insights in natural language sentences. our results illustrate that directly concatenating the original input sequence with refac-tor's generation can bring effective improvements to state-of-the-art baseline systems. we conclude our main contributions as follows: ## refactor qtsumm requires models to perform human-like reasoning in generating summaries that provide comprehensive and precise analysis of the source table to fulfill the user's information need. however, existing end-to-end text generation models rely on error-prone implicit reasoning processes for generating text, leading to diminished explainability and challenges in addressing user queries that necessitate complex human-like reasoning @xcite . to address this, we present refactor, to retrieve and reason over query-relevant information from tabular data to generate several nl data insights (i.e., facts) as explicit reasoning results. as shown in figure 3 , the generated facts is concatenated to the model input to mitigate the implicit reasoning issues, enhancing the comprehensiveness and faithfulness of generated summary. we next discuss the implementation of refactor. ## conclusion this paper defines a new query-focused table summarization task, and constructs a large-scale benchmark, qtsumm. we investigate a set of strong baselines, including text generation, table -to-text generation, and large language models. experimental results and manual analysis reveal that the new task presents significant challenges in table -to-text generation. moreover, we propose a novel approach named refactor, to retrieve and reason over query-relevant information from tables, improving the faithfulness of generated summary.