Open- W iki T able : Dataset for Open Domain Question Answering with Complex Reasoning over Table (2023)

Abstract: Despite recent interest in open domain question answering (ODQA) over tables, many studies still rely on datasets that are not truly optimal for the task with respect to utilizing structural nature of table. These datasets assume answers reside as a single cell value and do not necessitate exploring over multiple cells such as aggregation, comparison, and sorting. Thus, we release Open-WikiTable, the first ODQA dataset that requires complex reasoning over tables. Open-WikiTable is built upon WikiSQL and WikiTableQuestions to be applicable in the open-domain setting. As each question is coupled with both textual answers and SQL queries, Open-WikiTable opens up a wide range of possibilities for future research, as both reader and parser methods can be applied. The dataset is publicly available.

## introduction tables have played a prominent role as a source of knowledge in question answering (qa). they contain various types of data such as numeric, temporal, and textual information in a structured manner. early table qa datasets @xcite @xcite have focused on complex questions that exploit the structure of tables via aggregation, comparison, or sorting. however, these datasets assume that the relevant table is always given for each question @xcite , limiting their applicability in real-world scenarios. for more practical use, recent works extend tableqa to the open-domain setting, where the evidence table should be retrieved solely from using the question itself. the first research of open-domain qa over tables is @xcite . they released a dataset, nq-table, by extracting questions from natural figure 1 : open-wikitable is built by revising wikisql and wikitablequestions. through decontextualization, the question provides the necessary information to retrieve the grounding table. as the dataset is labeled with textual answers and sql queries, both reader and parser approaches can be used. questions @xcite whose answers reside in a table. all questions, however, are answered by extracting a single cell and do not necessitate any extensive reasoning across multiple cells. it is also notable that 55% of the evidence tables consist of only a single row, which has little structure. another work for open-domain table qa is @xcite . they presented e2e-wtq and e2e-gnq datasets, extensions of wikitablequestion @xcite and gnqtables @xcite datasets is their small size, containing only around 1k examples in total. this makes it challenging to train language models as there may not be enough data. given that there is currently no dataset that fully considers the structural property in the opendomain setting, we present open-wikitable. it extends wikisql and wikitablequestions to be more applicable in the open-domain setting. open-wikitable is a large-scale dataset composed of 67,023 questions with a corpus of 24,680 tables. the key features of our dataset are listed below. in this work, we thoroughly explain the data construction process of open-wikitable. we perform open domain question answering by incorporating a retrieval task with qa over tables (see figure 1 ). then, we evaluate the performance of the retriever and the qa models with both reader and parser approaches. although the questions in wikisql and wik-itablequestions require more advanced table reasoning than those of existing open-domain table qa datasets (see ## experiments first of all, we split tables into segments so that models can handle long tables within the limited input sequence length. inspired by @xcite , tables are split row-wise into 100-word chunks. around 52% of tables in our corpus are split into multiple chunks, which resulted in a total of 54,282 table segments. for the retrieval task, each table is flattened and appended with the table descriptions, and then fed to a retriever. when a grounding table is split into multiple tables, all table segments that are relevant to an answer should be retrieved. then, we perform end-to-end table qa where the model should answer the question given retrieved tables. more details about experimental settings are in appendix b. ## conclusion we present open-wikitable, the first odqa dataset that requires complex reasoning over wikipedia tables. the dataset is constructed by revising wikitablequestions and wikisql to be fully functional in the open-domain setting through decontextualization and paraphrasing. the dataset provides both textual and logical form answers for each question so that end-to-end reader and parser models can be trained. we hope that open-wikitable can provide new opportunities for future research such as investigating the effectiveness of leveraging both reader and parser approaches in the retrieval and generation phase.