The UPC RDF -to-Text System at W eb NLG Challenge 2020 (2020)

Abstract: This work describes the end-to-end system architecture presented at WebNLG Challenge 2020. The system follows the traditional Machine Translation (MT) pipeline, based on the Transformer model, applied in most text-totext problems. Our solution is enriched by means of a Back Translation step over the original corpus. Thus, the system directly relies on lexicalise format since the synthetic data limits the use of delexicalisation.

## introduction natural language generation (nlg) can be divided into: text-to-text generation or data-to-text generation, according to @xcite . the webnlg challenge 2020 consists in mapping data-to-text. more specifically, the data is a set of resource description framework (rdf) triples extracted from dbpedia and the corresponding text is a verbalisation of these triples 1 . the information structure is based on rdf, which consist of three elements: subject, predicate, object . thus, it establishes relations (predicate) between entities (subject, object). this can be appreciated in figure 1 . however, this information structure is not easy readable neither understandable, hence, it is hard for people to comprehend the meaning of such data. there has been a lot of previous work in the nlg domain @xcite in the past two decades. @xcite work in the medical domain, where they use a traditional nlg approach to generate sentences from rdf data filtering repetitive rdf, and then group coherent triples aggregating the generated sentences in order to produce the final ones. @xcite generate cooking recipes from semantic web 1 https://webnlg-challenge.loria.fr/ challenge_2020/ data, using a large corpus to extract lexicon in the cooking domain, which is then used in conjunction with a traditional nlg approach to generate cooking receipts. @xcite use a method which works well on rdf triples in a seen domain but fails with unseen ones. their aim is to learn sentence templates from parallel rdf data and text corpora by means of aligning entities in rdf triples with entities mentioned in sentences, and then extracting these templates from the aligned sentences by replacing the entity men-tion with a unique token. we decided to only participate in the english version of the rdf-to-text challenge @xcite . we used a model based on the transformer encoded-decoder architecture @xcite . moreover, inspired by previous work in the mt field, we enlarged the original corpus by means of back translation (bt) @xcite . the rest of the document is organised as follows. first, in section 2 we take a deeper dive into the task formulation. next, in section 3 the preprocessing plan is explained. then, in section 4 we depict the transformer model architecture adapted to our problem. thereafter, we briefly describe postprocessing in section 5. finally, in section 6 we summarize the implementation of bt over the original challenge followed by brief results and conclusions in sections 7 and 8 respectively. ## task formulation the goal of the rdf-to-text task is to generate text from a set of triples, which are words establishing relations between them. the input to our system is data in the form of triples that can be denoted as a set of rdf, i.e. k := {r 1 , ..., r n }. each rdf r i can be defined as s i , p i , o i , these elements stand for subject, predicate and object, respectively. notice that each element can contain more than one word, there is no prior restriction in that sense. for instance, the subject 'barack obama' would be encoded as finally, we aim to generate a discourse s, which consists of a sequence of words [w 1 , ..., w m ]. the resulting discourse in s should be grammatically correct and should also contain all the information present in the triples. ## preprocessing in this section we describe the first steps performed on data. there is a common step, delexicalisation, that has been performed since the very beginning of this challenge, back to 2017 foot_0 . we decided to avoid this step due to the implementation of our bt method that does not contain the required mapping: from individual entities to generic words. the very first data processing guide is defined as follows, and it is exemplified in ## the transformer model the transformer @xcite is considered a state-of-the-art encoder-decoder architecture, with great success in a vast field of applications such as mt. following this success and taking advantage of its simpler architecture, we proposed a simple transformer approach trained in an end-toend fashion. one of the main traits that enables these models to attain such surprising results, is the attention mechanism, that allows to model dependencies regardless their distance in the input or output sequences. this capability is a fundamental feature for rdf-to-text, as automated generation of text takes into account the relationship between words that may not appear consecutively. ## results in ## open discussion an interesting capability that was not implemented would have been to obtain delexicalised synthetic data, so the model learned more generic representations. furthermore, it would be interesting to enlarge the synthetic corpus and use this synthetic corpus to train more relevant models in the rdfto-text domain.