SCITAT : A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types (2025)

Abstract: Scientific question answering (SQA) is an important task aimed at answering questions based on papers. However, current SQA datasets have limited reasoning types and neglect the relevance between tables and text, creating a significant gap with real scenarios. To address these challenges, we propose a QA benchmark for scientific tables and text with diverse reasoning types (SCITAT). To cover more reasoning types, we summarize various reasoning types from real-world questions. To reason on both tables and text, we require the questions to incorporate tables and text as much as possible. Based on SCITAT, we propose a baseline (CAR), which combines various reasoning methods to address different reasoning types and process tables and text at the same time. CAR brings average improvements of 4.1% over other baselines on SCITAT, validating its effectiveness. Error analysis reveals the challenges of SCITAT, such as complex numerical calculations and domain knowledge.

## introduction scientific question answering (sqa) plays a crucial role in addressing research questions based on scientific papers @xcite . advancing sqa development can significantly accelerate knowledge acquisition @xcite . the dense technical terms and heterogeneous data representations in papers present challenges for the sqa task @xcite . to evaluate and enhance the model capabilities in sqa, numerous datasets are proposed @xcite @xcite . however, existing datasets exhibit the following limitations, as shown in ## scitat dataset the input for our task consists of scientific tables, text, and a question, and the output is the answer. moreover, we annotate the rationale of each question. for brevity, we refer to each question, its corresponding rationale, and answer, as an instance. we begin by describing the construction process of scitat. we employ a framework combining automatic generation with manual annotation to enhance both the quality and efficiency of the annotation process, as illustrated in figure 2 . ## car car is designed to address the questions on the context of scientific tables and text. given that sc-itat combines diverse reasoning types, car is composed into two modules: calculator and rea-text: tables 2 and 3 soner, as illustrated in figure 4, which focus on different reasoning types. to process tables and text simultaneously, calculator extracts and computes the numerical information from the context and reasoner derives the final answer based on the calculated information. the prompts we use are presented in appendix b.2. ## conclusion to address the limitations of previous scientific qa datasets, which involve limited reasoning types and fail to consider the relevance between tables and text, we propose scitat, the qa benchmark for scientific tables and text with diverse reasoning types. to incorporate diverse reasoning types, we analyze the questions posed by researchers and combine the types in prior works, summarizing 4 reasoning types with 13 subtypes. to ensure that the questions encompass both tables and text, we require the questions to include both elements when-ever possible. for scitat, we introduce car, a baseline that combines reasoning methods to enhance the performance across various reasoning types, with handling both tables and text. experimental results show that car outperforms other baselines by an average of 4.1%, demonstrating its effectiveness. error analysis reveals the challenges in scitat, such as grounding relevant context, complex numerical reasoning, and the need for domain-specific knowledge.