Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills (2022)

Abstract: Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning. In this work, we propose to leverage semi-structured tables, and automatically generate at scale question-paragraph pairs, where answering the question requires reasoning over multiple facts in the paragraph. We add a pre-training step over this synthetic data, which includes examples that require 16 different reasoning skills such as number comparison, conjunction, and fact composition. To improve data efficiency, we sample examples from reasoning skills where the model currently errs. We evaluate our approach on three reasoning-focused reading comprehension datasets, and show that our model, PReasM, substantially outperforms T5, a popular pre-trained encoder-decoder model. Moreover, sampling examples based on model errors leads to faster training and higher performance.

## introduction large pre-trained language models (lms) @xcite @xcite have become the backbone of natural language processing in recent years. however, recent work has shown that they struggle in performing symbolic reasoning operations, such as composition or conjunction of facts @xcite , numerical operations @xcite , and quantification @xcite , without substantial amounts of additional data. past work on improving reasoning in pre-trained models has taken two flavors: (a) adding specialized components for specific skills, like numerical and temporal reasoning @xcite @xcite , or (b) generating synthetic examples at scale, for example, by using grammars or templates (rozen † work done while working at the allen institute for artificial intelligence. figure 1 : an example table and question-contextanswer triplets generated from the table as synthetic data. each color corresponds to a different reasoning skill and colored cells are necessary to answer the question. the contexts shown are partial, such that the actual context contains the necessary information to answer the question and additional distractors. answers are not necessarily extractive (e.g., date difference). @xcite @xcite @xcite , and question generation models @xcite @xcite . in this work, we take the latter approach and argue that semi-structured tables are a valuable resource for automatic generation of training data that can endow lms with reasoning skills. tables can be crawled from the web at scale, and cover a wide range of domains and topics. moreover, their structured nature makes them amenable to automatic processes of data generation. specifically, given a table, we use templates to generate reading comprehension (rc) examples, that is, questioncontext-answer triplets, where answering the question requires diverse types of reasoning over facts mentioned in the context. fig. 1 shows an example table, and three generated question-context-answer examples, which require fact composition, number comparison, and computing a date difference. unlike prior work where semi-structured data was used for reasoning over tables or knowledge-bases @xcite @xcite , here we harness tables to allow lms to reason over text directly. fig. 2 provides an overview of our approach. we generate data by crawling tables from wikipedia, and applying 16 different example generators (egs) on each table. each eg corresponds to a particular reasoning skill (composition, numerical comparison, see we add a pre-training step over this generated data, where we perform multi-task training over the 16 task corresponding to the egs. since each eg can generate vast numbers of examples, it is important to focus training on reasoning skills that the model lacks. thus, we use error-driven sampling @xcite to construct training batches, where most examples are sampled from egs that the model currently struggles with. we fine tune our pre-traind for reasoning model, preasm, on three rc datasets that require reasoning: drop @xcite , iirc @xcite , and mmqa @xcite . preasm outperforms the original pre-trained t5 @xcite ) model by significant margins: 7.6, 4.1, and 1.2 f 1 points, respectively. our results set a new state-of-the-art on mmqa and are the best results on iirc for models where the retriever and reader are trained separately. our analysis shows that preasm leads to improvements of up to 40 f 1 points on specific question types, such as computing the difference between two dates, without causing a drop in other question types. in conclusion, our results suggest that tables are a viable and untapped source of information for automatically generating large amounts of data that can be used to endow lms with skills that are not captured using current pre-training approaches. our code, data, and models are publicly available and can be downloaded from https://github. com/oriyor/turning_tables . ## data generation our goal is to train a rc model that given a question q and textual context c returns an answer a, given a training set @xmath0, c i , a i )} n @xmath1. we focus on questions that require reasoning over the context, e.g., composing two facts. to endow lms with reasoning skills, we want to generate a large synthetic training set d @xmath2, c j , a j )} m @xmath3, before finetuning on a target dataset. ## training since our egs generate large quantities of examples, one can think of each eg as providing an infinite stream of examples. in this setup, a natural question is how to construct training batches such that the model learns the required skills as quickly as possible. after briefly describing our model, we will detail our training framework, where we sample examples from egs in an error-driven manner. model we use a standard encoder-decoder architecture @xcite ). given a training example (q, c, a), the model takes as input the sequence of tokens 'q [sep] c', and the task is to autoregressively decode the answer a token-by-token. we train to maximize the maximum likelihood objective log p (a | q, c). ## experimental results we present results on the downstream rc datasets ( §5.1) and on the synthetic data ( §5.2). ## analysis reasoning skills in drop to check which reasoning skills preasm has, we use a proposed split of a subset of drop to reasoning skills @xcite . the neural module network (nmn) used in @xcite . nmn was trained only on a subset of the original drop dataset. when comparing to t5, preasm dramatically improves performance on date-difference, and also leads to sizable gains in number-compare, extract-number and count. accuracy vs. training cost trade-off we evaluate preasm-base models on drop and iirc oracle as we vary the number of pre-training steps on d syn (fig. 6 ). most of the improvement happens in the first 100k steps, and error-driven sampling outperforms uniform sampling throughout training. error sampling outperforms momentum sampling in the latter part of training. a possible reason is that the reasoning skills in the downstream tasks are correlated with the harder tasks during pre-training (arithmetic addition and date difference). this provides an advantage for error sampling, since it will focus on these tasks even if the improvement during pre-training is small. ## related work template-based data generation has been previously used for data augmentation, for example to inject numerical skills @xcite , and to improve consistency @xcite , and zero-shot accuracy @xcite . in addition, templates were used for dataset construction @xcite @xcite , and to analyse model generalization @xcite . in this work, we automatically generate examples by instantiating templates using structured data. since our method relies solely on tables as input, it is highly scalable, has rich lexical diversity, and can be easily extended to new skills and domains. recently, @xcite introduced the wikinldb dataset, which includes queries that require reasoning over a set of textual facts. queries are instantiated with values from a knowledge graph (kg), and facts are generated by a lm. unlike this work, wikinldb is focused on evaluating reasoning skills. we, on the other hand, show that generated examples can be used to endow a pretrained lm with new reasoning skills. moreover, tables are much easier to collect at scale compared to kgs, which tend to have limited coverage. data augmentation techniques have been extensively explored in rc, qa, and dialogue @xcite @xcite @xcite . here, we focus on tables as a valuable source for data generation. pre-training over tables has focused in the past on reasoning over tables and knowledge-bases @xcite @xcite @xcite . here, we use pre-training over tables to improve reasoning over text. we leave evaluation on tasks beyond rc to future work. error-driven sampling has been considered in the past in the context of active learning @xcite , reinforcement learning @xcite @xcite , transfer learning @xcite , and distributionally robust optimization @xcite , where the goal is to perform well over a family of distributions. similar to @xcite , we compute heterogeneous batches based on error rates, and show that this improves efficiency and performance. ## conclusion we propose semi-structured tables as a valuable resource for generating examples that can endow pretrained language models with reasoning skills. we generate 5m examples that correspond to 16 reasoning skills from wikipedia tables and add a pretraining step over this data. to improve data efficiency we use error-driven sampling, which focuses training on reasoning skills that the model currently lacks. we evaluate our model, preasm, on three reasoning-focused rc datasets and show that it leads to substantial improvements in all cases.