Tree Kernels-based Discriminative Reranker for Italian Constituency Parsers
Authors: Antonio Uva, Alessandro Moschitti (Università di Trento, Qatar Computing Research Institute, 2016)
Topic: Syntax and Dependency Treebanks

Abstract
This paper aims at filling the gap between the accuracy of Italian and English constituency parsing: firstly, we adapt the Bllip parser, i.e., the most accurate constituency parser for English, also known as Charniak parser, for Italian and trained it on the Turin University Treebank (TUT). Secondly, we design a parse reranker based on Support Vector Machines using tree kernels, where the latter can effectively generalize syntactic patterns, requiring little training data for training the model. We show that our approach outperforms the state of the art achieved by the Berkeley parser, improving it from 84.54 to 86.81 in labeled F1.

Introduction
Constituency Syntactic parsing is one of the most important research lines in Computational Linguistics. Consequently, a large body of work has been also devoted to its design for Italian language AUTHOR. However, the accuracy reported for the best parser is still far behind the state of the art of other languages, e.g., English. One noticeable attempt to fill this technological gap was carried out in the EvalIta challenge, which proposed a parsing track on both dependency and constituency parsing for Italian. Among the several participant systems, the Berkeley parser AUTHOR gave the best result AUTHOR. At the beginning, the outcome for constituency parsing computed on TUT AUTHOR was much lower than the one obtained for English on the Penn Treebank AUTHOR. In the last EvalIta edition, such gap diminished as the Italian parser labeled F1 increased from 78.73% (EvalIta 2009) to 82.96% (EvalIta 2011). Some years later the parser F1 improved to 83.27% AUTHOR. However, the performance of the best English parser AUTHOR, i.e., 92.1%, is still far away. The main reason for such gap is the difference in the amount of training data available for Italian compared to English. In fact, while Penn Treebank contains 49,191 sentences/trees, TUT only contains 3,542 sentences/trees. In presence of scarcity of training data, a general solution for increasing the accuracy of a machine learning-based system is the use of more general features. This way, the probability of matching training and testing instance representations is larger, allowing the learning process to find more accurate optima. In case of syntactic parsing, we need to generalize either lexical or syntactic features, or possibly both. However, modeling such generalization in state-of-the-art parser algorithms such as the Bllip AUTHOR is rather challenging. In particular, the space of all possible syntactic patterns is very large and cannot be explicitly coded in the model. An easier solution consists in using such features in a simpler model, which can be trained to improve the outcome of the main parser, e.g., selecting one of its best hypotheses. In particular, tree kernels (TKs) by AUTHOR can be used for encoding an exponential number of syntactic patterns in parse rerankers. In this work, we aim at filling the gap between English and Italian constituency parsing: firstly, we adapted Bllip parser, i.e., the most accurate constituency parser for English, also known as Charniak parser, for Italian and trained it on TUT. We designed various configuration files for defining specific labels for TUT by also defining their type, although we did not encode head-finding rules for Italian, needed to complete the parser adaptation. Secondly, we apply rerankers based on Support Vector Machines (SVMs) using TKs to the k-best parses produced by Bllip, with the aim of selecting its best hypotheses. TKs allow us to represent data using the entire space of subtrees, which correspond to syntactic patterns of different level of generality. This representation enables the training of the reranker with little data. Finally, we tested our models on TUT, following the EvalIta setting and compare with other parsers. For example, we observed an improvement of about 2%, over the Berkeley parser, i.e., 86.81 vs. 84.54.

Conclusion
This work was inspired by AUTHOR and AUTHOR, who explored discriminative approaches for ranking problems. Their studies were limited to WSJ, though, and did not explore the use of max-margin classifiers, i.e., SVMs. The first experiments with SVMs and TKs were conducted by AUTHOR, who proposed a new SVM-based voting algorithm making use of preference reranking. In this paper, we adapted the Charniak parser for Italian gaining an improvement of 1.07\% over the Berkeley model (indicated by EvalIta as the state of the art for Italian). Then, our TK-based reranker further improved it up to 2 absolute percent points. It should also be noted that our best reranking result is 3.54 absolute points better than the best outcome reported in AUTHOR, i.e., 83.27. In the future, we would like to integrate (i) the features developed in the reranking software available by AUTHOR in our model for further improving it, (ii) generalizing lexical features (e.g., embeddings, brown clusters) and including similarity measures in PTK, i.e., SPTK AUTHOR. A special thank is due to Alberto Lavelli and Alessandro Mazzei for enabling us to carry out an exact comparison with their parser.