Text-to-Text Pre-Training Model with Plan Selection for RDF -to-Text Generation (2020)

Abstract: We report our system description for the RDFto-Text task in English on the WebNLG 2020 Challenge. Our approach consists of two parts: 1) RDF-to-Text Generation Pipeline and 2) Plan Selection. RDF-to-Text Generation Pipeline is built on the state-of-the-art pretraining model, while Plan Selection helps decide the proper plan into the pipeline.

## introduction natural language generation from data, or data-totext, aims to generate natural language text that describes the input data such as tables and graphs. webnlg @xcite is one of the data-to-text tasks, where the given input data is a set of triples (a tree or a graph). a triple consists of entities and a relationship between them in the form of (subject, predicate, object) describing one fact. for example, (donald trump, birthplace, new york) describes the facts "donald trump was born in new york." in the webnlg task, up to 7 triples are given as the input data. figure 1 presents an instance of the webnlg dataset 1 , where 3 triples are given as the rdf input graph and the output text is the natural language description describing the fact from triples. generating text from triples is useful for many applications such as question and answering @xcite , where a set of triples retrieved as an answer can be organized and translated into natural language text. many approaches have been proposed to deal with the webnlg task. neural machine translation (nmt) approach is one of the popular methods for this task @xcite . in webnlg 2017, a neural machine translation-based approach, achieved the highest score in the automatic evaluation @xcite . in this approach, 1 an example is from https://webnlg-challenge. loria.fr the delexicalization, where entities are replaced by placeholders, with n-gram search and the linearization using dbpedia type enrichment are used together with the standard encoder-decoder with attention model. due to the simple linearization, the relationship between entities as the graph in triples might lessen. to preserve such information, a graph-based triple encoder (gtr-lstm) @xcite , is introduced. later, @xcite argued that splitting the generation process into plan selection and text realization could help generate the description text that is faithful to the triples. planenc uses a graph convolution network-based model to predict the order of the triples and then an lstm with attention and the copy mechanism is employed to generate the text corresponding to the order of triples @xcite . recently, kale (2020) investigated textto-text transfer transformer (t5) model for the data-to-text task. with transfer learning ability, the t5 model is the current state-of-the-art for rdf-totext generation. to further investigate the t5 model from the study @xcite , we conduct experiments with the t5 model on webnlg 2020 challenge foot_0 . based on our preliminary results, we found that changing the order of the triples before the linearization affected the bleu score. in this paper, we therefore present a text-to-text pre-training model for the data-to-text generation with plan selection. in our approach, we follow the study @xcite , to build the rdf-to-text generation pipeline. then, we develop the graph neural network (gnn) based model to score the plan associating with the triple order. the triple order with the highest plan score is linearized and feed to generator in the rdf-to-text generation pipeline. ## approach in our approach, there are two steps: 1) rdf-to-text generation and 2) plan selection. rdf-to-text generation aims to transform input triples into natural language text describing them, while plan selection is the pre-processing process to select the order of the triples for the rdf-to-text generation pipeline. note that plan selection is performed before rdf-to-text generation in the testing phase. ## conclusion in this paper, we introduce rdf-to-text generation with plan selection for webnlg 2020 challenge. the approach comprises two parts: 1) the pipeline for rdf-to-text generation and 2) plan selection. in the pipeline, we follow the state-of-the-art model for rdf-to-text generation @xcite , while in plan selection we propose a method for selecting the order of triples for the pipeline. overall, we observed a slight improvement in the bleu score when applying plan selection. nevertheless, plan selection only optimized the bleu score in this study. in the future, we aim to investigate optimize plan selection in other metrics.