TAP 4 LLM : Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning (2024)

Abstract: Table reasoning tasks have shown remarkable progress with the development of large language models (LLMs), which involve interpreting and drawing conclusions from tabular data based on natural language (NL) questions. Existing solutions mainly tested on smaller tables face scalability issues and struggle with complex queries due to incomplete or dispersed data across different table sections. To alleviate these challenges, we propose TAP4LLM as a versatile pre-processor suite for leveraging LLMs in table-based tasks effectively. It covers several distinct components: (1) table sampling to decompose large tables into manageable sub-tables based on query semantics, (2) table augmentation to enhance tables with additional knowledge from external sources or models, and (3) table packing & serialization to convert tables into various formats suitable for LLMs’ understanding. In each module, we design and compare several common methods for usage in various scenarios, aiming to shed light on the best practices for leveraging LLMs for table-reasoning tasks. Our experiments show that our method improves LLMs’ reasoning capabilities in various tabular tasks and enhances the interaction between LLMs and tabular data by employing effective pre-processing.

## introduction the extensive and complex characteristics of the data are commonly represented in the format of structured data. table is one of those fundamental and widely used semi-structured data types in different areas, such as relational databases, spreadsheet applications, and programming languages * equal contribution. ## tap4llm: table provider for llms the overall architecture of tap4llm is defined as follows (as illustrated in figure 2 ): given a natural language query/utterance q from applications (e.g., excel copilot) and a table t from table manager (e.g., table cache or database connection), our system incorporates three core components as follows: • table sampling: decompose a large table t into a sub-table t ′ with specific rows and columns. • table augmentation: incorporate relevant external knowledge, metadata, and attributes about the original table t explicitly. • table packing: convert tables into various formats suitable for llms' understanding while control the token allocation for table sampling and augmentation. ## experiments in this section, we first present the experimental setup, followed by an extensive comparison between baselines within each module in tap4llm. additionally, we conduct an ablation study and provide a thorough evaluation of the performance of tap4llm. for further details on the experimental setup and additional experiments, please refer to appendix d and e. ## conclusion table provider for llm) as a powerful toolkit designed to enhance the interaction between llms and structured table data. it provides optimized prompt designs and robust functionalities to ensure highquality outputs when llms process table-related inputs. we believe that tap4llm has the potential to significantly improve table modeling and exploratory data analysis (eda), with applications in various domains.