Representations for Question Answering from Documents with Tables and Text (2021)

Abstract: Tables in web documents are pervasive and can be directly used to answer many of the queries searched on the web, motivating their integration in question answering. Very often information presented in tables is succinct and hard to interpret with standard language representations. On the other hand, tables often appear within textual context, such as an article describing the table. Using the information from an article as additional context can potentially enrich table representations. In this work we aim to improve question answering from tables by refining table representations based on information from surrounding text. We also present an effective method to combine text and table-based predictions for question answering from full documents, obtaining significant improvements on the Natural Questions dataset (Kwiatkowski et al., 2019).

## related work most work on qa with tables prior to bert involves first converting the table to a knowledge graph (kg) where cell entries are entities with row/column relations, then using entity linking to identify spans in the question that match an entity in the knowledge graph, and finally parsing the question to generate a sql query using some variant of a sequence-to-sequence model @xcite . due to the advances in contextualized word embeddings, more recent work proposed a modification of the bert transformer architecture to be used for representing tables. @xcite proposed the usage of additional @xcite tokens between headers of the table to make a bert model more suitable for the tables. recently, @xcite introduced a pretraining procedure for joint representation of tabular data paired with an utterance, where the approach is to linearize the structure of tables to be compatible with a bert model. our approach for table encoding is most similar to that of @xcite , where the authors generalized the bert architecture similarly to @xcite with new types of relations to encode table-specific relationships. the main differences between our table representation and @xcite is that in our representation we use 5 types of relations, cell-column, cell-row, in-cell, cross-column and cross-row (more details in section 3.1), while in their work the authors use cell-column and cell-row relations only for the table representation, but in addition use question-cell relations for marking matches between tokens in the question and corresponding cell values. finally, the two most recent works on table representation learning, tapas @xcite and grappa @xcite , also use pretraining on the wikitables dataset @xcite that we use in our work. therefore, our table representations based on transformers and our pretraining method are comparable to those in recent and concurrent work. leveraging tables is a hard problem. however, most studies on table-based qa omit an important additional information source: the text in the article discussing the table. prior attempts at integrating a kb and text use early fusion of document text and kg information @xcite , where they integrate text and a kg sub-graph in a single graph, from which an entity is selected to answer the question. structured kgs are often easier to interpret than tables, which have a wide variety of possible schemas. infotabs @xcite introduced a dataset for the natural language inference task based on premises that are tables, where the authors explore multiple table representations, including a key-value approach and linearized representations with table rows corresponding to "sentences." hypothesis representations are calculated separately. recently, tabert @xcite introduced a joint table-utterance representation approach, where a table row is concatenated with a short text utterance, such as the query in question answering, and passed as an input to a bert-based model. such an approach relies on the initial table representation to select the table rows most relevant to the query. in contrast, we enrich the table representation using an attention mechanism with the representations of the most relevant parts of the context of the article in which the table appears. the natural questions is a large corpus that contains real user queries along with their corresponding wikipedia articles, which may or may not contain an answer anywhere in the article. alberti et al. (2019) provided a bert-based baseline that treats both table and text segments like text: a sequence of tokens with word and position embeddings. re-cently, liu et al. (2020) improved this baseline by using dynamic dual-attention over paragraphs and cascade answer predictor. in another direction, ravula et al. (2020) used an extended transformer architecture that models extra-long documents with limited propagation of information among different segments. all three approaches did not distinguish between text and table input, treating tables as text while not taking into account table structure. to the best of our knowledge, this is the first work that focuses on table-based qa for the real user queries in the natural questions corpus, and shows that table-based and overall qa performance can be improved by building on state-of-the-art pre-trained representations of table structure, additionally enriched via attention to related article text. ## methods this section describes two methods that improve table representations for qa from tables: an extension of the bert architecture for table encoding to better capture the relationships between table elements, and a mechanism that incorporates related unstructured text of an article as context to further improve the table representation. we then describe an approach for question answering from both text and tables that combines predictions from text qa and table qa models using a late fusion approach. ## experiments we evaluate our model on the natural questions (nq) corpus @xcite that consists of questions and paired wikipedia pages, with the task of finding the exact location of the answer that is present in the article, if any. the nq dataset is large (300k training samples) and answers (if present) can appear in either or both text and tables. nq contains one human annotation for every question in the training set, and 5 annotations for every question in the development and test sets. based on the statistics of the development set, around 14% of questions contain a short answer in a table. only 48% of the questions have a long answer annotation (a paragraph or a table that contains an answer), while only 33% contain a short answer annotation (an exact location of a short answer phrase). in this work we first evaluate our models on a subset of questions that contain at least one short answer in tables, referred as nqtables, and then further evaluate the model on the full dataset, referred as fullnq. for the questions in nqtables, we further evaluate in two settings: (i) nqtables tab , where systems are limited to predict answer spans only from tables, and (ii) nqtables, where systems can predict answer spans from both text and tables. note that although all questions in nqtables have at least one answer in a table, they might also have answers in text, and systems operating in the full nqtables setting have more chances to arrive at a correct answer than systems in nqtables tab setting. while our primary focus in this work is on improving the short answer prediction, we also report the long answer prediction results for our best model used for the full nq dataset, according to official metrics. the development set (dev) for nqtables used in this work contains 1118 questions where the short answer can be found in a table. since the official test set of the corpus is not public, all our experiments use the official development set as our test set, while splitting the training set into training (90%) and 2 validation sets (each contains 4% of the data). the first validation set (val-1) is used for tuning the parameters of the table-based models, while the second validation set (val-2) is used for tuning the parameters to combine textbased and table-based models. for clarity, in all the experiment variations we use the notation defined above where nqtables tab -dev/val-1/val-2 is limited to predict answer spans only from tables, and nqtables-dev/val-1/val-2 is able to predict answer spans from both text and tables. the official evaluation script computes f1 scores and considers any questions that have at least 2 annotated answers as being answerable, while questions with 1 or no answer are unanswerable. an f1 score is calculated on the ability to predict a span that matches at least one of the annotations for the answerable questions, and to correctly predict unanswerable ones. since the validation sets contain only a single annotation, the f1 score measure based on 5-way annotation cannot be used directly on val-1 or val-2. therefore, in the table-based experiments on the val-1, we report both accuracy (percent of correctly predicted answers) and modified f1 score (f1*), where modified f1 is based on the match of the predicted answer to a single annotation. finally, we report a string-based f1 score that accepts any exact string match of a predicted span to an answer when combining text and table models. ## conclusion tables in web documents are pervasive and can be directly used to answer many search queries. in this work, we presented an approach to enrich table representations using information from article text, and showed that it improves a state-of-the-art pretrained structure-aware table representation for question answering from tables. we also studied how to effectively combine text-based and tablebased approaches. finally, we performed the first study focusing on table qa for the natural questions dataset, and showed that improved representations of tables lead to performance gains. in future work, our methods can be applied to other qa datasets, such as wikitablequestions @xcite and hybridqa @xcite . the correct answer "bowling for soup" was correctly predicted from the table by the model that does not use context attention, while the context-aware model predicts the answer that has higher number of links to the textual context.