Dependency Parsing via Sequence Generation (2022)

Abstract: Dependency parsing aims to extract syntactic dependency structure or semantic dependency structure for sentences.Existing methods for dependency parsing include transition-based method, graph-based method and sequence-to-sequence method.These methods obtain excellent performance and we notice them belong to labeling method.Therefore, it may be very valuable and interesting to explore the possibility of using generative method to implement dependency parsing.In this paper, we propose to achieve Dependency Parsing (DP) via Sequence Generation (SG) by utilizing only the pre-trained language model without any auxiliary structures.We first explore different serialization designing strategies for converting parsing structures into sequences.Then we design dependency units and concatenate these units into the sequence for DPSG.We verify the DPSG is capable of parsing on widely used DP benchmarks, i.e., PTB, UD2.2, SDP15 and SemEval16.In addition, we also investigate the astonishing low-resource applicability of DPSG, which includes unsupervised cross-domain conducted on CODT and few-shot cross-task conducted on SDP15.Our research demonstrates that sequence generation is one of the effective methods to achieve dependency parsing.Our codes are available now.

## introduction dependency parsing (dp), which aims to extract the structural information beneath sentences, is fundamental in understanding natural languages. it benefits a wide range of natural language processing (nlp) applications, such as machine translation @xcite , question 1 https://github.com/timelessling/dpsg-code/tree/main ‡ boda lin and zijun yao make equal contribution * corresponding author sentence ms. haag plays elianti . syntactic dependency semantic dependency ms. plays haag elianti . nn nsubj dobj punct ms. plays haag elianti . arg1 arg2 compound syntactic dependency structure in sequence semantic dependency structure in sequence 1 2 3 4 5 2 ms. haag plays elianti . 3 3 3 3 haag haag plays elianti . 1 3 3 3 n [nn] [nsubj] [root] [dobj] [punct] [compound] [arg1] [root] [arg2] [rel-no] @xcite and the dm semantic dependency structure @xcite . they are further converted into unified serialized representations. we use the position id to replace the head word in flattened sequence. in order to verify the feasibility of generativeparsing and explore suitable design solutions, we propose dependency parsing via sequence generation (dpsg). the core idea is to find a unified unambiguous serialized representation for both syntactic and semantic dependency structures. then an encoder-decoder plm is learned to generate the parsing results following the serialized representation, without the need for an additional decoder. that is, our parser can achieve its function using one original plm (without any modification). in particular, dpsg consists of three key components. the serializer is responsible for converting between the dependency structure and the serialized representation. the positional prompt pattern provides supplementary word position information in the input sentence to facilitate the sequence generation process. the encoder-decoder plm with added special tokens performs the parsing task via sequence generation. we conduct experiments on 5 popular dp benchmarks: ptb, ud2.2, codt, sdp15, and se-meval16. dpsg performs generally well on different dp tasks. it significantly outperforms the baselines on cross-domain (codt) and chinese sedp (semeval16) corpora, and achieves comparable results on the other three benchmarks. in addition, the few-shot cross-task experiments also demonstrate the applicability of dpsg . we also design a series of pilot experiments to explore the rationality of the designing of the dpsg. our research shows that generative sequence-tosequence method has the potential to be an ef-fective sequence-to-sequence approach for dependency parsing. ## preliminaries we formally introduce the dependency parsing task and the encoder-decoder plm, and the corresponding notations. this paper uses bold lower case letters, blackboard letters, and bold upper case letters to denote sequences, sets, and functions, respectively. elements in the sequence and the sets are enclosed in parentheses and braces, respectively. ## analysis this section studies whether there is better implementation for dpsg. we are particularly interested in: 1) the designing of the serializer, 2) the effect of the introduced special tokens, and 3) the choice of the plm model. we use ptb as the benchmark and compare dpsg introduced in section 3 with many other possible choices. the results of these exploratory experiments are shown in ## conclusion this paper proposes dpsg, a generative sequenceto-sequence dependency parsing method. by serializing the parsing structure to a flattened sequence, plm can directly generate the parsing results in serialized representation. dpsg not only achieves good results in different parsing representations, but also performs surprisingly well on unsupervised cross-domain dp. the few-shot transfer experiments also suggest that dpsg is capable of investigating the inner connection between sedp and sydp. the exploratory experiments and analyses demonstrate the rationality of the designing of dpsg. considering the unity, indirectness, and effectiveness of dpsg, we believe the sequence generative is one of the effective sequence-to-sequence methods for dependency parsing.