LLM -Symbolic Integration for Robust Temporal Tabular Reasoning (2025)

Abstract: Temporal tabular question answering presents a significant challenge for Large Language Models (LLMs), requiring robust reasoning over structured data—a task where traditional prompting methods often fall short. These methods face challenges such as memorization, sensitivity to table size, and reduced performance on complex queries. To overcome these limitations, we introduce TEMPTABQA-C, a synthetic dataset designed for systematic and controlled evaluations, alongside a symbolic intermediate representation that transforms tables into database schemas. This structured approach allows LLMs to generate and execute SQL queries, enhancing generalization and mitigating biases. By incorporating adaptive fewshot prompting with contextually tailored examples, our method achieves superior robustness, scalability, and performance. Experimental results consistently highlight improvements across key challenges, setting a new benchmark for robust temporal reasoning with LLMs. Code and TEMPTABQA-C dataset: https:// coral-lab-asu.github.io/llm_symbolic.

## introduction large language models (llms) have revolutionized natural language processing, achieving remarkable success across a variety of tasks. however, answering questions about temporal tabular data task that requires precise reasoning over structured information with time-based attributes remains a significant challenge. this capability is crucial in fields such as finance, healthcare, and policymaking, where actionable insights often depend on understanding and analyzing evolving datasets. yet, existing methods often fall short, struggling with complex queries, large datasets, and scenarios that require nuanced reasoning. an example * work done during internship at upenn † primary mentor and corresponding author. equally important is the need for robust methods. many existing approaches rely on direct prompting, which often depends on heuristics and memorized patterns rather than true reasoning. this results in semantic biases and poor performance in demanding scenarios, such as counterfactual reasoning, large table contexts, or multi-step queries. this leads to a second critical question: how can we develop methods that remain robust across diverse table structures, dynamic data, and complex queries? to address these challenges, we propose a comprehensive framework that reimagines how llms approach temporal tabular data. at its core is temptabqa-c, a synthetic dataset generation method designed to fill the gaps in existing benchmarks. temptabqa-c provides precise control over data characteristics, enabling consistent and systematic evaluation across a wide range of scenarios, including counterfactual reasoning and intricate temporal queries. building on this foundation, we introduce a symbolic intermediate representation approach that transforms unstructured tables into structured database schemas. llms are guided to generate sql queries based on these schemas, which are executed to produce accurate answers. e.g. in figure 1 , the sql query serves as a symbolic representation and provides the correct answer, whereas direct prompting fails on given query. this structured pipeline reduces semantic biases, enhances interpretability, and significantly improves the generalization of models across different table configurations. additionally, we incorporate adaptive few-shot prompting, a dynamic approach that selects contextually relevant examples tailored to each query. this method overcomes the limitations of static examples, further improving the robustness of the system in complex scenarios. our experiments demonstrate that this frame-work delivers substantial improvements over direct prompting methods. it excels in critical areas such as counterfactual reasoning, scalability to larger datasets, and the handling of complex queries. beyond these technical advancements, our work establishes a new benchmark for temporal tabular question answering by addressing fundamental weaknesses in existing approaches and introducing innovative tools for evaluation and reasoning. these contributions pave the way for building more interpretable, scalable, and robust ai systems with implications for critical real-world applications. our contributions are as follows: ## the temptabqa-c dataset the temptabqa-c dataset is a large-scale, semiautomatically generated resource designed for evaluating temporal reasoning in large language models (llms). it provides a benchmark for analyzing the tabular temporal abilities of llms by enabling controlled variations in data characteristics, making it superior to traditional human-curated datasets. this section describes the dataset creation process, its schema, and key characteristics. ## answering questions about tables in this section, we describe two high level approaches for answering questions about structured data. ## experimental setup we designed experiments to address the following research questions: to answer these questions, we conducted experiments to evaluate the two core approaches: direct prompting and symbolic intermediate representation. ## results and analysis in this section, we present the results for gpt-4o and gemini 1.5 pro. additionally, we evaluated gemini 1.5 flash, gpt-4o mini, mixtral, llama 3.1 70b, code llama, and sql coder, which demonstrated similar trends. the results of these additional experiments are included in the appendix. ## comparison with related work temporal reasoning in llms is an evolving field intersecting with advancements in tabular reasoning, logic, and symbolic methods. our work advances this area by introducing the temptabqa-c dataset for detailed evaluation of temporal reasoning in tabular contexts. key advancements in related areas are discussed below. tabular reasoning. applying llms to semistructured tables has been studied for question answering, semantic parsing, and table-to-text generation @xcite @xcite . approaches like tapas @xcite , @xcite , and tabbie @xcite enhance table understanding through joint tabulartextual embeddings, while table2vec (zhang et al., 2019) and tabgcn (pramanick and bhattacharya, 2021) investigate alternative representations. recent work also explores symbolic reasoning for structured tables with fixed schemas @xcite @xcite . building on these advances, we use sql-based symbolic methods for temporal queries in semistructured data. our temptabqa-c dataset further enables fine-grained evaluation of temporal reasoning across diverse tabular characteristics. temporal reasoning. temporal reasoning is central to question answering and event-centric tasks, with datasets like time-sensitiveqa @xcite and torque @xcite addressing time-sensitive comprehension in text, and tempqa-wd @xcite and cronquestions @xcite focusing on temporal links in knowledge graphs. models like cronkbqa @xcite further enhance performance by incorporating temporal reasoning during training. our work extends these efforts to structured tabular datasets. while datasets such as temptabqa @xcite and tram @xcite tackle similar challenges, temptabqa-c advances the field by introducing counterfactual reasoning, scalable table sizes, and diverse question difficulties, offering a broader framework for evaluating temporal reasoning. ## conclusion this work investigates temporal tabular question answering with llms, tackling key challenges in counterfactual robustness, data sensitivity, and question complexity. we introduced temptabqa-c, a controlled benchmark designed for systematic evaluations. by combining symbolic intermediate representations with adaptive fewshot prompting, our approach leverages database schemas and sql query generation to address the limitations of direct prompting. our experiments demonstrate that symbolic representations improve generalization, counterfactual robustness, and scalability, especially when handling larger tables. additionally, adaptive prompting enhances reasoning for complex queries. these results highlight the necessity for the model to be data-blind, reasoning exclusively on metadata-such as the table schema-to ensure robust reasoning rather than relying on mere memorization. immediate future work can focus on conducting detailed error analysis and exploring finetuning techniques. a deeper analysis of the results will further illuminate the strengths and limitations of the approach. temptabqa-c lays a strong foundation for advancing structured temporal reasoning in llms and encourages future efforts to develop interpretable and robust temporal reasoning in ai systems.