Machine Translation Aided Bilingual Data-to-Text Generation and Semantic Parsing (2020)

Abstract: We present a system for bilingual Data-ToText Generation and Semantic Parsing. We use a text-to-text generator to learn a single model that works for both languages on each of the tasks. The model is aided by machine translation during both pre-training and fine-tuning. We evaluate the system on WebNLG 2020 data 1 , which consists of RDF triples in English and natural language sentences in English and Russian for both the tasks. We achieve considerable gains over monolingual models, especially on unseen relations and Russian.

## introduction text corpora and structured data are important to practical nlp applications. they complement each other in content and topic coverage, therefore, it is important to convert from one to the other depending on the downstream task. rdf is a common format used to store structured data as triples of @xcite . many template based, pipeline based, statistical and neural systems have been proposed for converting triples into natural text @xcite @xcite . recently, an end-to-end text generation model t5 was shown to achieve stateof-the-art performance on this task @xcite . t5 has an encoder decoder framework pretrained on web pages by masking spans of text randomly and finetuned on task specific data. several systems have also been proposed for the reverse task of semantic parsing that involves extracting rdf triples from natural text @xcite , however t5 has not been utilized for it yet. we use t5 as the initial model, augmenting both pretraining and finetuning with several parallel cor-pora in a multi-task setup. this multi-task multilingual (pre-)training regimen improves the quality of both text generation and semantic parsing. in this paper, we present the description of our multilingual multi-task system and its performance on the webnlg 2020 challenge. this dataset consists of rdf triples in english and natural language sentences in english and russian. we demonstrate that multitask learning across languages provides significant improvement on both the tasks, especially on unseen relations and russian. furthermore, incorporating parallel corpora improves the quality of text generation. ## experiments we experimented with a variety of models for both the pretraining and the finetuning stages. here, we present the results of the different pretrained models with the best finetuned model and the different finetuned models with the best pretrained model. ## final results in prior sections, all the experiments were evaluated on the seen and reserved unseen dev sets. in this section, we train the model on the full training set and report the results on the test sets, as communicated by the challenge organizers. we use our best pre-trained model bt5+wmt and our best finetuned model bilingual+wpc for text generation and bilingual for semantic parsing. the results are shown in ## conclusion we developed a system for bilingual data-to-text generation and semantic parsing, using t5, aided by machine translation during both pretraining and finetuning. we evaluated this system on webnlg 2020 and showed improvements, especially on unseen relations.