A New Representation for Span-based CCG Parsing (2021)

Abstract: This paper proposes a new representation for CCG derivations. CCG derivations are represented as trees whose nodes are labeled with categories strictly restricted by CCG rule schemata. This characteristic is not suitable for span-based parsing models because they predict node labels independently. In other words, span-based models may generate invalid CCG derivations that violate the rule schemata. Our proposed representation decomposes CCG derivations into several independent pieces and prevents the span-based parsing models from violating the schemata. Our experimental result shows that an off-the-shelf span-based parser with our representation is comparable with previous CCG parsers.

## introduction combinatory categorial grammar (ccg) (steedman, 2000) is a mildly context-sensitive grammar formalism. several neural ccg parsing methods have been proposed so far (lewis and steedman, 2014; xu et al., 2015; lewis et al., 2016; vaswani et al., 2016; lee et al., 2016; xu, 2016; yoshikawa et al., 2017; stanojević and steedman, 2019, 2020; bhargava and penn, 2020; tian et al., 2020; prange et al., 2021; liu et al., 2021). currently, neural span-based models (cross and huang, 2016; stern et al., 2017; gaddy et al., 2018; kitaev and klein, 2018) have been successful in the field of constituency parsing. however, we cannot directly apply this technique to ccg parsing. span-based models assume that each node label in parse trees can be predicted independently, while, in ccg, each node label (category) is strictly restricted by ccg rule schemata. the independence assumption of span-based models implies that the models are not guaranteed to generate valid ccg derivations. to solve this problem, we propose a method of representing ccg derivations in a way suitable for span-based parsing models. our proposed repre- ## ccg and span-based parsing this section gives an overview of combinatory categorial grammar (ccg) (steedman, 2000) and explains why we cannot directly apply the spanbased approach to ccg parsing. ## span-based representation to overcome the problem described in the previous section, we propose a new representation for ccg derivations. we call the new ones span-based representations (sbrs for short), which decomposes ccg derivations into several independent pieces to prevent the span-based parsing model from violating the ccg rule schemata. figure 2 shows an example of ccg derivation and its sbr version. we realize span-based ccg parsing as follows: ## generating oov categories in our proposed representation, lexical categories are not directly assigned to words. lexical categories are decomposed into several node labels. this means that lexical categories are not defined by a finite set and that the span-based parsing model learned from sbrs may generate oov lexical categories that do not appear in the training data. algorithm 2 recover(n, p ) 1: n is a node in an sbr. 2: n is a node in a ccg derivation. 3: p , l and r are categories. 4: ul(s) is a ul tag if exists. 5: ur(s) is a ur tag if exists. 6: 7: if n is a terminal (word) node then 8: @xmath0, n ) 9: else 10: we conducted an experiment using the ccgbank (hockenmaier and steedman, 2007) @xcite to evaluate the performance of our method. @xcite we used the berkeley neural parser (kitaev and klein, 2018) with bert (devlin et al., 2019) as a span-based parser. we converted the training (sections 02-21) and the development (section 00) data into sbrs and learned the model from the data. the number of sbr's labels in the training data was 486. @xcite the hyperparameters for training were identical to those of kitaev et al. (2019). we evaluated the parsing performance by labeled f 1 on the test data (section 23). we obtained labeled dependencies using the c&c parser's generate program (clark and curran, 2007). as a baseline model, we trained a model directly using the ccg derivations. ## conclusion this paper proposed a new representation for ccg derivations. our proposed representation realizes a span-based ccg parser that follows the ccg binary rule schemata. furthermore, the parser can generate oov categories. one remaining problem in the proposed method is to treat unary rule schemata in ccg. our method encodes unary rules using the additional information described in section 3.1.1, but this approach may violate the unary rule schemata. in the future, we will extend the method to treat ccg unary rules validly.