Few-Shot Table-to-Text Generation with Prototype Memory (2021)

Abstract: Neural table-to-text generation models have achieved remarkable progress on an array of tasks. However, due to the data-hungry nature of neural models, their performances strongly rely on large-scale training examples, limiting their applicability in real-world applications. To address this, we propose a new framework: Prototype-to-Generate (P2G), for table-to-text generation under the few-shot scenario. The proposed framework utilizes the retrieved prototypes, which are jointly selected by an IR system and a novel prototype selector to help the model bridging the structural gap between tables and texts. Experimental results on three benchmark datasets with three state-of-the-art models demonstrate that the proposed framework significantly improves the model performance across various evaluation metrics.

## introduction generating natural language from structured table @xcite , i.e. table-to-text generation, is an important research problem for various nlp applications, such as biographical descriptions @xcite , restaurant information @xcite , basketball game summaries @xcite , and open-domain question answering @xcite . the main challenge of table-to-text generation stems from the structural difference between the table and the natural language text. with recent advances in neural networks, many sophisticated neural models @xcite @xcite have been proposed to address this problem. while achieving impressive results, such neural models are data-hungry, i.e. large amounts of training data are required for them to learn the mapping between tables and texts. this can prohibit these models from being applied to real-world applications due to the huge data curation overhead @xcite . this motivates us to investigate fewshot table-to-text generation @xcite , that allows the model to learn a satisfactory table-to-text mapping with limited labelled training data. in this work, we propose to address this problem by augmenting data-to-text generation models with prototype memory acquired from a large unlabelled corpus. our motivation is two-fold: (1) relevant human-authored texts, termed "prototypes", are informative and can teach the model how to better describe the table when limited training data is available. (2) however, traditional lexical-based ir systems, e.g. bm25, are inaccurate and the quality of their results are not guaranteed. therefore, a bert-based prototype selector is required to further select the prototypes, from the results retrieved by the ir system, that are closely related to the table for better guiding the neural generation model. figure 1 illustrates the proposed prototype-to-generate (p2g) framework. given the table, an ir system is first applied to retrieve candidates that are potentially related to the table from a large unlabelled corpus. based on the retrieved candidates, a prototype selector then selects the top n prototypes based on the table-text pairwise similarity. lastly, a sequence generator takes the table and the selected prototypes as input to produce the output. to prevent the model from uncritically copying the information contained in the prototypes that is irrelevant to the table, we introduce a content-aware learning objective when training the generator. in recent years, retrieval-based (i.e. templatebased) text generation has been studied in different nlp areas, including machine translation @xcite , unconditional text generation @xcite , dialogue systems @xcite , paraphrase generation @xcite , and question answering @xcite . despite their differences, we identify two major limitations in previous studies compared to our approach. firstly, most previous research @xcite @xcite build their retrieval corpus based on data consisting of aligned source-target pairs, which precludes the use of abundant unlabelled data. secondly, current retrieval mechanisms are either based on lexical similarity (e.g. bm25) where its accuracy cannot be guaranteed, or large neural networks @xcite which require a large amount of data to train. notably, our framework is independent of the choice of generation model. for a comprehensive evaluation, we test our approach on three representative models, including the current state of the art. the experimental results on three datasets show that our framework leads to remarkable performance improvements across all evaluation metrics. ## methodology figure 1 depicts an overview of our framework. given a linearized table @xmath0, ..., t |t | }, where t @xmath1, v i } is an attribute-value pair, an ir system first retrieves a set of m candidates r from the large unlabelled corpus. then, a prototype selector f ( §2.1) selects the top n prototypes s from r that are most related to t . lastly, a sequence generator g ( §2.2) takes t and s to produce the output y. ## case study in ## conclusion in this study, we introduced a new retrievalbased framework, prototype-to-generate (p2g), which augments table-to-text models with prototype memory from unlabelled data. extensive experiments and analysis on three benchmark datasets show that our approach can significantly improve the performance of various strong generation models on all evaluation metrics. our code, models and other related resources can be found in https://github.com/yxuansu/ few-shot-table -to-text-generation