T able W ise at S em E val-2025 Task 8: LLM Agents for T ab QA (2025)

Abstract: Tabular Question Answering (TabQA) is a challenging task that requires models to comprehend structured tabular data and generate accurate responses based on complex reasoning. In this paper, we present our approach to SemEval Task 8: Tabular Question Answering, where we develop a large language model (LLM)-based agent capable of understanding and reasoning over tabular inputs. Our agent leverages a hybrid retrieval and generation strategy, incorporating structured table parsing, semantic understanding, and reasoning mechanisms to enhance response accuracy. We fine-tune a pre-trained LLM on domain-specific tabular data, integrating chain-of-thought prompting and adaptive decoding to improve multi-hop reasoning over tables. Experimental results demonstrate that our approach achieves competitive performance, effectively handling numerical operations, entity linking, and logical inference. Our findings suggest that LLM-based agents, when properly adapted, can significantly advance the state of the art in tabular question answering.

## introduction the task of question answering on tabular data (tabqa) involves answering natural language queries using structured tabular data. given a natural language query, the goal is to generate accurate responses based on the tabular data. conventional tabqa methods typically involve providing large language models (llms) with the entire table alongside the query, under the assumption that full-table context enables models to reason over any potentially relevant information. however, this strategy faces significant challenges, including limited scalability due to context length constraints, poor generalization and high computational overhead. retrieval-augmented generation (rag) methods have partially tried to address these issues by retrieving semantically similar chunks of the table. however, they rely on embeddings that often fail to capture relational structures and numerical precision, resulting in poor retrieval performance and difficulty in handling diverse column types. to overcome these limitations, we propose an llm-agent-based framework that abstracts tables † the authors contributed equally to this work. ## related work early approaches for tabqa primarily relied on semantic parsing, translating queries into executable programs such as sql, as seen in models like neural programmer @xcite and sqlnet @xcite . while effective, these approaches required annotated sql queries and struggled to generalize across diverse table schemas. later on, models such as tapas @xcite and tabert @xcite ) used weakly supervised learning over table-text pairs, removing the dependence on explicit sql annotations. while these models advanced the state of the art, they exhibited limitations in numerical reasoning capabilities and scalability to long or complex tables. to further address these challenges, retrieval-augmented generation (rag) techniques, exemplified by models like rasat @xcite , incorporated retrieval mechanisms to identify relevant table segments before answer generation. however, despite improved scalability, these methods often failed to capture the structured and relational aspects inherent to tables. recent developments in agentic llms, such as toolformer @xcite and react @xcite , introduced frameworks where language models interact with external tools (e.g., sql engines, calculators) to perform more accurate and interpretable reasoning. building upon these advancements, we propose a novel framework that integrates agentic reasoning with sql-based execution. by abstracting tables into sql databases and enabling llms to interact with them, our approach enhances generalization across domains, and strengthens numerical reasoning capabilities in tabqa tasks. ## dataset each table is provided in two formats: all.parquet, containing the complete set of rows, and sample.parquet, containing the first 20 rows of the corresponding all.parquet file. the collection of all all.parquet files across every table constitutes the databench dataset, while the set of all sample.parquet files forms the databench-lite dataset. additionally, for each table, a set of natural language queries has been provided, which can be answered using the table's information. ## system overview we introduce a novel pipeline for question answering over tables that allows llms to interact with a set of specialized tools to deliver precise, reliable responses. the end-to-end architecture is depicted in figure 1 and comprises two primary stages: ## results and analysis we integrated four open-source llms-llama-3.3-70b, codestral-22b-v0.1, mistral-7b-v0.3, and llama-3.2-3b-into our agentic framework and evaluated each on both the databench and databench-lite datasets. ## conclusion our research highlights the advantages of using an agentic approach for qa on tabular data. it demonstrates its ability to dynamically construct and refine queries for precise information retrieval, which is essential for qa performance. by outperforming rag-based and full-table context methods in handling mixed data types and scalability challenges, it proves more adaptable and efficient. additionally, its iterative reasoning surpasses direct sql querying and reinforces the potential of llm-powered agents in table-aware ai systems.