Parsing All: Syntax and Semantics, Dependencies and Spans (2020)

Abstract: Both syntactic and semantic structures are key linguistic contextual clues, in which parsing the latter has been well shown beneficial from parsing the former. However, few works ever made an attempt to let semantic parsing help syntactic parsing. As linguistic representation formalisms, both syntax and semantics may be represented in either span (constituent/phrase) or dependency, on both of which joint learning was also seldom explored. In this paper, we propose a novel joint model of syntactic and semantic parsing on both span and dependency representations, which incorporates syntactic information effectively in the encoder of neural network and benefits from two representation formalisms in a uniform way. The experiments show that semantics and syntax can benefit each other by optimizing joint objectives. Our single model achieves new state-of-the-art or competitive results on both span and dependency semantic parsing on Propbank benchmarks and both dependency and constituent syntactic parsing on Penn Treebank.

## introduction this work makes the first attempt to fill the gaps on syntactic and semantic parsing from jointly considering its representation forms and their linguistic processing layers. first, both span (constituent) and dependency are effective formal representations for both semantics and syntax, which have been well studied and discussed from both linguistic and computational perspective, though few works comprehensively considered the impact of either/both representation styles over the respective parsing @xcite . second, as semantics is usually considered as a higher layer of linguistics over syntax, most previous studies focus on how the latter helps the former. though there comes a trend that syntactic clues show less impact on enhancing semantic parsing since neural models were introduced @xcite . in fact, recent works @xcite propose syntax-agnostic models for semantic parsing and achieve competitive and even state-of-the-art results. however, semantics may not only benefit from syntax which has been well known, but syntax may also benefit from semantics, which is an obvious gap in explicit linguistic structure parsing and few attempts were ever reported. to our best knowledge, few previous works focus on the relationship between syntax and semantic which only based on dependency structure @xcite @xcite . to fill such a gap, in this work, we further exploit both strengths of the span and dependency representation of both semantic role labeling (srl) @xcite and syntax, and propose a joint model foot_0 with multi-task learning in a balanced mode which improves both semantic and syntactic parsing. moreover, in our model, semantics is learned in an end-to-end way with a uniform representation and syntactic parsing is represented as a joint span structure @xcite relating to head-driven phrase structure grammar (hpsg) @xcite which can incorporate both head and phrase information of dependency and constituent syntactic parsing. we verify the effectiveness and applicability of the proposed model on propbank semantic parsing foot_1 in both span style @xcite and dependency style @xcite and penn treebank (ptb) @xcite for both constituent and dependency syntactic parsing. our empirical results show that semantics and syntax can indeed benefit each other, and our single model reaches new stateof-the-art or competitive performance for all four tasks: span and dependency srl, constituent and dependency syntactic parsing. ## structure representation in this section, we introduce a preprocessing method to handle span and dependency representation, which have strong inherent linguistic relation for both syntax and semantics. for syntactic representation, we use a formal structure called joint span following @xcite to cover both constituent and head information of syntactic tree based on hpsg which is a highly lexicalized, constraint-based grammar @xcite . for semantic (srl) representation, we propose a unified structure to simplify the training process and employ srl constraints for span arguments to enforce exact inference. ## experiments we evaluate our model on conll-2009 shared task @xcite for dependency-style srl, conll-2005 shared task @xcite for span-style srl both using the propbank convention @xcite , and english penn treebank (ptb) @xcite for constituent syntactic parsing, stanford basic dependencies (sd) representation (de @xcite converted by the stanford parser 8 for dependency syntactic parsing. we follow standard data splitting: for both two scores, but it makes the training process become more complex and unstable. thus we employ a parameter to balance two different scores in joint decoder which is easily implemented with better performance. for the predicate disambiguation task in dependency srl, we follow @xcite and use the off-the-shelf disambiguator from @xcite . for constituent syntactic parsing, we use the standard evalb 9 tool to evaluate the f1 score. for dependency syntactic parsing, following previous work @xcite , we report the results without punctuations of the labeled and unlabeled attachment scores (las, uas). ## related work in the early work of srl, most of the researchers focus on feature engineering based on training corpus. the traditional approaches to srl focused on developing rich sets of linguistic features templates and then employ linear classifiers such as svm @xcite . with the impressive success of deep neural networks in various nlp tasks @xcite @xcite @xcite @xcite @xcite @xcite @xcite , considerable attention has been paid to syntactic features @xcite @xcite . @xcite @xcite @xcite modeled syntactic parsing and srl jointly, @xcite jointly modeled srl and ccg parsing, and @xcite combined the supertags extracted from dependency parses with srl . there are a few studies on joint learning of syntactic and semantic parsing which only focus on dependency structure @xcite @xcite . such as @xcite based on dependency structure only focus on shared representation without explicitly analyzing whether syntactic and semantic parsing can benefit each other. the ablation studies results show joint learning can benefit semantic parsing while the single syntactic parsing model was insignificantly worse (0.2%) than the joint model. @xcite only made a brief attempt on chinese semantic treebank to show the mutual benefits between dependency syntax and semantic roles. instead, our work focuses on whether syntactic and semantic parsing can benefit each other both on span and dependency in a more general way. besides, both span and dependency are effective formal representations for both semantics and syntax. on one hand, researchers are interested in two forms of srl models that may benefit from each other rather than their separated development, which has been roughly discussed in @xcite . @xcite is the first to apply span-graph structure based on contextualized span representations to span srl and @xcite built on these span representations achieves state-of-art results on both span and dependency srl using the same model but training individually. on the other hand, researchers have discussed how to encode lexical dependencies in phrase structures, like lexicalized tree adjoining grammar (ltag) @xcite and headdriven phrase structure grammar (hpsg) @xcite . ## conclusions this paper presents the first joint learning model which is evaluated on four tasks: span and dependency srl, constituent and dependency syntactic parsing. we exploit the relationship between semantics and syntax and conclude that not only syntax can help semantics but also semantics can improve syntax performance. besides, we propose two structure representations, uniform srl and joint span of syntactic structure, to combine the span and dependency forms. from experiments on these four parsing tasks, our single model achieves state-of-the-art or competitive results.