Improving the Extraction of Supertags for Constituency Parsing with Linear Context-Free Rewriting Systems (2022)

Abstract: In parsing phrase structures, supertagging achieves a symbiosis between the interpretability of formal grammars and the accuracy and speed of more recent neural models.The approach was only recently transferred to parsing discontinuous constituency structures with linear context-free rewriting systems (LCFRS).We reformulate and parameterize the previously fixed extraction process for LCFRS supertags with the aim to improve the overall parsing quality.These parameters are set in the context of several steps in the extraction process and are used to control the granularity of extracted grammar rules as well as the association of lexical symbols with each supertag.We evaluate the influence of the parameters on the sets of extracted supertags and the parsing quality using three treebanks in the English and German language, and we compare the best-performing configurations to recent state-of-the-art parsers in the area.Our results show that some of our configurations and the slightly modified parsing process improve the quality and speed of parsing with our supertags over the previous approach.Moreover, we achieve parsing scores that either surpass or are among the state-of-the-art in discontinuous constituent parsing.

## introduction discontinuous constituency parsing deals with the task to find hierarchies of -possibly noncontiguous -phrases (constituents) in a given sentence and assigns a label (constituent symbol) to each phrase. traditional approaches use grammar formalisms such as linear context-free rewriting systems (lcfrs) to model these hierarchies @xcite @xcite . statistical parsing with these grammars is remarkably slow and inaccurate by today's standards. but they still find some attraction as both, the grammars and parsing with them, are easily interpretable. more recent parsers use neural classifiers and either leverage the parsing process into a linear task @xcite @xcite or score constituent labels for selected phrases @xcite . although, in the latter approaches, the occurring discontinuities are restricted to a small degree. in supertagging-based parsing @xcite , a grammar is accompanied by a classifier that selects and scores a small sample of rules. after that, these rules and their scores are interpreted as a weighted grammar and used for statistical parsing in the usual manner. this remaining statistical parsing process is significantly faster than the traditional approach, as the grammar is much smaller. the approach was investigated in combination with tree adjoining grammars (tag; @xcite and combinatory categorial grammars (ccg; @xcite , often in the context of dependency parsing. since the introduction of recurrent neural networks (rnn) as classifiers, the accuracy of the supertag prediction has improved by far @xcite @xcite . a recent publication @xcite showed that supertagging improves the quality and speed of parsing constituency structures with lcfrs, bringing it close to recent discontinuous parsing methods. however, their extraction process for supertags is rather convoluted and uses hard-wired strategies for, e.g., the lexicalization and binarization. in this work, we investigate if the quality of predictions and parsing with lcfrs supertags can be improved by introducing parameters that replace these hard-wired configurations in the extraction. section 3 presents a formulation of supertags and an extraction algorithm that is -in our eyes -easier to grasp than the previous definition, bewhere the survey was carried out. the tree is illustrated with crossing branches, so that the leaves appear ordered. for each constituent, the path to its lexical head is double-struck. ## notation a discontinuous constituent tree is a tuple (ξ, pos, w) as follows: w is a sequence of termi- figure 2 : a binary lexical lcfrs derivation for the string where the study was carried out. we sometimes consider lexical heads for the constituent structure. for each inner node, the lexical head is the critical sentence position that determines its symbol; and we call each of the node's children that does not contain the lexical head a modifier. in fig. 1 , for each inner node in ξ, the path to its lexical head is double struck. e.g. for the inner node with symbol s, the lexical head is the position 3. the subtree starting at the np node is its only modifier. we briefly cover the notation for binary lexical lcfrs (in the following just lcfrs). ## contributions this section presents all concepts involved with the processes for the extraction from (discontinuous) constituency treebanks and parsing with lcfrs supertags. we will start in section 3.1 with a short motivation for our notation for supertags, which deviates from the previous formulation. section 3.2 describes our process for the extraction of supertags and highlights parts that require sets of parameters. these parameters are described in detail in the following section 3.3. section 3.4 concludes with an overview for the parsing process. all the described steps and concepts are illustrated along the arrows in fig. 3 , their labeling conforms with the numbering in the para-graph headings of sections 3.2 and 3.4. ## conclusion we generalized the extraction of supertags from treebanks by introducing parameters for previously fixed parts of the construction. the parameters allow us to control the part of the constituent tree that is associated with a terminal for each supertag (guide constructor) as well as the granularity of the grammar rules (nonterminal constructor). at the same time, the extraction process was re-ordered so that its description is less convoluted while retaining the same functionality. some introduced guide and nonterminal constructors performed better than the vanilla variants. specifically, we observe the following: while the highly ambiguous grammars extracted from dptb benefit from finer nonterminal granularity with greater markovization window, the large and more specific grammars extracted from tiger improve with coarser granularity; the grammar for negra lies somewhere in between. compared to the previous implementation of supertagging with lcfrs, we could improve the parsing quality across all three discontinuous treebanks. the improvements close the gap between the quality of parsing with lcfrs supertagging and the most recent discontinuous parsing strategies. in case of the two german treebanks, we could even surpass them, most notably in terms of the f1-score for discontinuous constituents.