Holographic CCG Parsing (2023)

Abstract: We propose a method for formulating CCG as a recursive composition in a continuous vector space. Recent CCG supertagging and parsing models generally demonstrate high performance, yet rely on black-box neural architectures to implicitly model phrase structure dependencies. Instead, we leverage the method of holographic embeddings as a compositional operator to explicitly model the dependencies between words and phrase structures in the embedding space. Experimental results revealed that holographic composition effectively improves the supertagging accuracy to achieve state-of-the-art parsing performance when using a C&C parser. The proposed span-based parsing algorithm using holographic composition achieves performance comparable to state-of-the-art neural parsing with Transformers. Furthermore, our model can semantically and syntactically infill text at the phrase level due to the decomposability of holographic composition.

## introduction combinatory categorial grammar (ccg; steedman 2000) is a highly lexicalized grammar formalism comprising syntactically rich lexical categories and a limited number of combinatory rules. in principle, ccg is suitable for modelling complicated syntactic structures and operates as a natural interface connecting syntax to semantics because of its isomorphism with lambda calculus @xcite @xcite . in this paper, we propose a method to formulate ccg (a discrete symbol system) as an operation between distributed representations in a continuous vector space, demonstrating its contribution to improved supertagging performance and span-based parsing. prior studies on pcfg, compositional vector grammar (cvg; @xcite , and its generalization, latent vector grammar (lveg; zhao in this study, we introduce recursive vector composition in the embedding space illustrated in figure 1 by employing holographic embeddings (hole; @xcite to incorporate syntactic structures into the supertagging and parsing model explicitly. similar methods for embedding tree structures into fixed-length vectors, cvg, and kernel-inspired encoders with recursive mechanisms for interpretable trees (kermit; zanzotto et al. 2020a) have been proposed. our model differs from cvg, as it does not require a large number of matrix parameters for nonlinear compositions, and directly optimizes parsing, enabling the construction of phrase-level representations by dynamically exploring phrase structures, whereas kermit requires an external parser. experiments revealed that phrase-level dependency modelling with holographic composition can induce correct supertagging, achieving state-of-the-art performance in supertagging and parsing with a c&c parser @xcite , and further improved performance with a novel span-based parsing algorithm. 1 additionally, we focused on the fact that the inverse operation of holographic composition is easily available. this property can be applied to text-infilling tasks, predicting missing parts of sentences consistent with the rest syntactically and semantically. this task is difficult to accomplish using the existing neural architectures. the main contributions of this research can be summarized as follows: we further demonstrate the applicability of decomposability to phrase-level text-infilling. ## holographic ccg our research objective is to compute phrase-level representations to capture dependencies and hierarchical relationships among its internal components for supertagging and parsing in ccg. we describe the mechanism for composing these representations and discuss their application to supertagging. we then introduce a novel span-based parsing that utilizes these representations. ## conclusion in this paper, we proposed a novel method for formulating ccg as a recursive composition operation on a continuous vector space and constructing phrase/sentence-level representations from word representations. we demonstrated its utility for supertagging and parsing. experimentation demonstrated the effectiveness of holographic compositions in explicitly modelling dependencies between sentence components, resulting in improved performance and state-of-the-art results in supertagging and parsing using the c&c parser. in addition, we validated that phrase-level text-infilling is possible by applying the decomposable property of the holographic representation in the proposed model.