T ab SQL ify: Enhancing Reasoning Capabilities of LLM s Through Table Decomposition (2024)

Abstract: Table reasoning is a challenging task that requires understanding both natural language questions and structured tabular data. Large language models (LLMs) have shown impressive capabilities in natural language understanding and generation, but they often struggle with large tables due to their limited input length. In this paper, we propose TabSQLify, a novel method that leverages text-to-SQL generation to decompose tables into smaller and relevant sub-tables, containing only essential information for answering questions or verifying statements, before performing the reasoning task. In our comprehensive evaluation on four challenging datasets, our approach demonstrates comparable or superior performance compared to prevailing methods reliant on full tables as input. Moreover, our method can reduce the input context length significantly, making it more scalable and efficient for large-scale table reasoning applications. Our method performs remarkably well on the WikiTQ benchmark, achieving an accuracy of 64.7%. Additionally, on the TabFact benchmark, it achieves a high accuracy of 79.5%. These results surpass other LLM-based baseline models on gpt-3.5-turbo (chatgpt). TabSQLify can reduce the table size significantly alleviating the computational load on LLMs when handling large tables without compromising performance.

## introduction tables serve as the most prevalent forms of structured information across diverse domains, ranging from databases and spreadsheets to open data repositories, web pages and document collections. developing natural language interfaces for tabular data poses a significant challenge, primarily in terms of effectively interpreting the semantics of table cells and understanding the relationships between cell values in response to a user query. this challenge is accentuated when tables are enveloped in text, such as titles, captions, and contextual text ## methodology our approach capitalizes on the proficiency of llms in parsing natural language text and generating sql to enhance their capabilities in table reasoning. large language models face challenges in accommodating extensive contextual information, especially when dealing with large tables that exceed their token limits. increasing this size for large tables is unrealistic due to the quadratic time and memory complexities of self-attention mechanism in input length. furthermore, llms are more likely to produce errors when handling lengthy contexts @xcite . to overcome these challenges, our work efficiently identifies and extracts relevant table parts, optimizing the input prompt size without sacrificing performance. ## conclusion our proposed decomposition approach has shown promise across different table reasoning tasks, achieving remarkable performance compared to models that require the use of a full table . our method is novel in leveraging text-to-sql generation to decompose tables into smaller and relevant sub-tables tailored for table reasoning tasks. this approach provides a new perspective and direction for table reasoning research, and we hope it will inspire more future work on combining natural language understanding and structured data processing.