Neural Surface Realization for Italian
Authors: Valerio Basile, Alessandro Mazzei (Università di Torino, 2018)
Topic: Syntax and Dependency Treebanks

Abstract
We present an architecture based on neural networks to generate natural language from unordered dependency trees. The task is split into the two subproblems of word order prediction and morphology inﬂection. We test our model gold corpus (the Italian portion of the Universal Dependency treebanks) and an automatically parsed corpus from the Web.

Introduction
Natural Language Generation is the process of producing natural language utterances from an abstract representation of knowledge. As opposed to Natural Language Understanding, where the input is well-deﬁned (typically a text or speech segment) and the output may vary in terms of complexity and scope of the analysis, in the generation process the input can take different forms and levels of abstraction, depending on the speciﬁc goals and applicative scenarios. However, the input structures for generation should be at least formally deﬁned. In this work we focus on the ﬁnal part of the standard NLG pipeline deﬁned by Reiter and Dale (2000), that is, surface realization, the task of producing natural language from formal abstract representations of sentences’ meaning and syntax. We consider the surface realization of unordered Universal Dependency (UD) trees, i.e., syntactic structures where the words of a sentence are connected by labeled directed arcs in a treelike fashion. The labels on the arcs indicate the syntactic relation holding between each word and its dependent words (Figure 1a). We approach the surface realization task in a supervised statistical setting. In particular, we draw inspiration from Basile (2015) by dividing the task into the two independent subtasks of word order prediction and morphology inﬂection prediction. Two neural network-based models run in parallel on the same input structure, and their output is later combined to produce the ﬁnal surface form. A ﬁrst version of the system implementing our proposed architecture (called the DipInfo-UniTo realizer) was submitted to the shallow track of the Surface Realization Shared Task 2018 (Mille et al., 2018). The main research goal of this paper is to provide a critical analysis for tuning the training data and learning parameters of the DipInfo-UniTo realizer.

Conclusion
Conclusion and Future Work Conclusion and Future Work In this paper, we considered the problem of analysing the impact of the training data and parameters tuning on the (modular and global) performance of the DipInfo-UniTo realizer. We computationally proved that the DipInfo-UniTo realizer can gives competitive results (i) by augmenting the training data set with automatically annotated sentences, and (ii) by tuning the learning parameters of the neural models. In future work, we intend to resolve the main lack of our approach, that is the impossibility to realize non-projective sentences. Moreover, further optimization of both neural models will be carried out on a new high-performance architecture (Aldinucci et al., 2018), by executing a systematic gridsearch over the hyperparameter space, namely the regularization factor and weight initialization for ListNet, and the speciﬁc DNN hyperparameters for the morphology module.