Exploring the Constructicon: Linguistic Analysis of a Computational C x G (2023)

Abstract: Recent work has formulated the task for computational construction grammar as producing a constructicon given a corpus of usage. Previous work has evaluated these unsupervised grammars using both internal metrics (for example, Minimum Description Length) and external metrics (for example, performance on a dialectology task). This paper instead takes a linguistic approach to evaluation, first learning a constructicon and then analyzing its contents from a linguistic perspective. This analysis shows that a learned constructicon can be divided into nine major types of constructions, of which Verbal and Nominal are the most common. The paper also shows that both the token and type frequency of constructions can be used to model variation across registers and dialects.

## introduction construction grammar (cxg) is a usage-based approach to language which views grammatical structure as a set of form-meaning mappings called a constructicon @xcite . from this usagebased perspective, a construction could belong in the grammar either (i) because it is sufficiently entrenched (i.e., frequent) that it is stored and processed as a unique item or (ii) because it is sufficiently irregular (i.e., idiomatic) that it requires a unique grammatical description @xcite . the advantage of cxg from this perspective is that it focuses on explaining the creativity, the flexibility, and the idiosyncrasy of actual language use in real-world settings @xcite . given this focus of cxg as a linguistic theory, the ideal computational implementation must be data-driven and unsupervised. for example, approaches which rely on manual annotations derived from individual introspection @xcite fail to capture the usage-based foundations of cxg, in addition to being unreproducible and difficult to scale. for this reason, most recent work on com-putational cxg has taken an unsupervised learning approach to forming constructicons @xcite . such an unsupervised approach has its own challenges, however, especially the challenge of evaluation. grammars from other syntactic paradigms can be evaluated by annotating a gold-standard corpus and then measuring the ability of both supervised and unsupervised models to predict those same sets of annotations (cf., @xcite . given its usage-based foundations, this approach to evaluation is simply not feasible for computational cxg because the standard for what counts as a construction depends to some degree on the corpus or the community of speaker-hearers that is being observed. for this reason, recent work on computational cxg has undertaken both internal and external evaluations for determining which one of a set of posited constructicons is better. an internal metric measures the fit between a grammar and a given corpus to determine which alternative constructicon offers a better description @xcite . this work has drawn on minimum description length @xcite as an evaluation metric because it combines both descriptive adequacy (i.e., the fit between the grammar and the test set) and model complexity (i.e., the number and the type of constructions in the grammar). an external metric evaluates and compares constructicons using their performance when applied to a specific prediction task. recent work has focused on the use of computational cxg for modelling individual differences @xcite , register variation @xcite , and population-based dialectal differences @xcite @xcite . because cxg is a usage-based paradigm, the definition of a construction that is referenced above depends on both entrenchment and idiomaticity. both of these are properties of a corpus of usage rather than properties of a language as a whole. in other words, it is only meaningful to describe entrenchment relative to a particular individual, dialectal community, or context of production. these external tasks have therefore focused on the degree to which computational cxg can in fact account for differences in usage across these dimensions. the contribution of this paper is to undertake a detailed qualitative and quantitative evaluation of a learned grammar. while it is not possible to start with gold-standard linguistic annotations of constructions, it is possible to apply a linguistic analysis to the output of an unsupervised, usagebased framework. we start by describing the model and the data which are used to learn the constructicon (section 2) before presenting examples of types of constructions that it contains (section 3). we then proceed to a quantitative analysis of the grammar (section 4). finally, we end with a discussion of the challenge of parsing a nested and hierarchical grammar which contains representations at different levels of abstraction (section 5). ## methods and data computational cxg is a theory in the form of a grammar induction algorithm that provides a reproducible constructicon given a corpus of exposure @xcite . the theory is divided into three components, each of which models a particular aspect of the emergence of constructicons given exposure to a corpus of usage. first, a psychologically-plausible measure of association, the ∆p , is used to measure the entrenchment of potential constructions @xcite . these potential constructions are sequences of lexical, syntactic, and semantic slotconstraints. the problem of category formation is to define the inventory of fillers that are used for slot-constraints. in this implementation, lexical constraints are based on word-forms, without lemmatization. syntactic constraints are formulated using the universal part-of-speech tagset @xcite and implemented using the ripple down rules algorithm @xcite . semantic constraints are based on distributional semantics, with k-means clustering used to discretize fasttext embeddings @xcite . the semantic constraints in the examples in this paper are formulated using the index of the corresponding clusters, a simple notational convention. second, an association-based beam search is used to identify constructions of arbitrary length by finding the most entrenched representations in reference to a matrix of ∆p values @xcite . the beam search parsing strategy allows the grammar to avoid relying on heuristic frames and templates for producing potential constructions. third, a measure of fit based on the minimum description length paradigm is used to balance the increased storage of item-specific constructions against the increased computation of more generalized constructions @xcite . the point is that any construction could become entrenched but more idiomatic constructions come at a higher cost. the contribution of this paper is to evaluate this existing model of cxg @xcite rather than to alter its overall method of learning a constructicon. we therefore apply the model without further discussion of its implementation and focus instead on a linguistic analysis of the resulting constructicon. the data used to learn grammars is collected from three sets of corpora: social media (twitter), nonfiction articles (wikipedia), and web pages (from the common crawl) drawn from the corpus of global language use @xcite . this training corpus contains 2 million words per register for a total of 6 million words. from a usage-based perspective, exposure to language continues after the grammar has been acquired and such exposure might change the entrenchment of particular constructions. the model thus undertakes a second pruning stage which updates the constructicon given an additional 2 million words of exposure @xcite . the model observes sub-corpora from each of the three registers in increments of 100k words. each construction in the grammar receives an activation weight with an initial value of 1. for each sub-corpus in which a construction is not observed, its weight decays by 0.25. for each sub-corpus in which a construction is observed, its weight is returned to 1. when a construction's weight falls below 0, it is forgotten and removed from the grammar. this is a simple model of the way in which continued exposure leads to the forgetting of previously entrenched constructions. while somewhat arbitrary, the decay rate (0.25) is chosen to ensure that a construction is not forgotten simply because it occurs primarily in a specific register: this decay rate means that a construction must be absent from four successive sub-corpora, thus ensuring that each of the three registers has been observed. thus, this pruning method removes unproductive constructions given additional exposure while ensuring that all three registers remain represented. a package for reproducing this grammar induction algorithm is available foot_0 as well as the specific grammars used in this study. 2 this method produces a constructicon that contains 12,856 constructions. the analysis in this paper is based on using this constructicon to annotate samples of 1 million words from 12 independent corpora: project gutenberg @xcite , wikipedia @xcite , european parliament proceedings @xcite , news article comments @xcite , product reviews @xcite , blogs @xcite , and tweets from six countries (with 1 million words representing each country; dunn 2020). this range of corpora allows us to consider both register (different contexts of production) and dialect (different populations using the same register) when measuring the frequency and the productivity of individual constructions in the grammar. ## categorizing constructions in this section we categorize the learned constructions to aid our quantitative analysis of the contents of the constructicon. we annotate a random sample of 20% of the constructions using the categorization described below, thus allowing an estimate of the overall composition of the grammar. the primary categories are verbal, nominal, adjectival, adpositional, transitional, clausal, adverbial, sentential, and fixed idioms. these categories are defined and exemplified in this section. the first category consists of verbal constructions. as shown in (1), we notate the construction using its slot-constraints, with each slot separated by dashes. lexical constraints are shown in italics; syntactic constraints are shown in small caps; and semantic constraints are shown using the index of their distributional cluster (e.g., <521>). using this notation, the construction in (1) is a simple passive verb phrase in a continuous aspect, defined using primarily syntactic constraints. the verbal construction in (2) now contains a semantic constraint (<521>). this domain contains lexical items like house and carriage, all locations that can be moved into or out of. the construction thus captures a meaning-based pattern of movement in relation to some area. ## distribution of construction types the first step in quantifying the contents of the constructicon is to estimate the relative distribution across these nine categories. this is shown in figure 1 using annotations of 20% of the grammar to estimate the overall distribution. the y-axis contains a bar chart for each category of construction and the x-axis shows the percent of the constructicon which falls into that category. thus, for example, the most frequent type of construction is verbal at 33.7% of the grammar, followed by nominal at 21.7% and sentential at 18.3%. this distribution is not surprising given that verbs and nouns are the most common open-class lexical items and that sentential clauses form the basic structure of sentences. the next step is to measure the frequency of each construction and the number of its unique types, thus capturing its productivity. these measures of frequency and productivity are corpus-specific in the sense that different constructions are more likely to be used in specific contexts or by specific populations. we thus consider 12 distinct corpora of 1 million words each, six representing distinct registers and six representing distinct populations within the same register. starting with a comparison across registers, the frequency of each category of construction (i.e., the mean number of tokens) also provides a view of the grammatical differences between these six registers. for instance, blogs contain fewer adpositional constructions than other registers while published books and speeches in parliament contain approximately twice as many overall. wikipedia articles contain many fewer cases of clasual and transitional constructions, indicating a register with fewer embedded clauses. further, blogs have nearly twice as many sentential constructions (i.e., base main clauses) as wikipedia, but many fewer adpositional phrases. this would indicate that information can be packaged in short sentences or in additional adpositional constructions, depending on the register. note that another set of wikipedia corpora was available during the grammar learning process, so that the reduced frequencies of these types are not simply a matter of under-fitting the register. the next question is whether the differences in frequency of individual constructions across corpora are random or whether they reveal underlying relationships between the corpora themselves. in other words, given the frequencies of each construction in the grammar, we would expect a meaningful grammar to create meaningful relationships between conditions. a condition in this case refers to the register or the population represented by the corpus. this is shown in figure 2 using burrow's delta to calculate the distances between corpora and then hierarchical clustering to visualize relationships based on these distances. the figure shows relationships between registers on the top. the two core clusters are with modern formal documents (eu and wikipedia) and digital crowd-sourced documents (comments and blogs and reviews). the books from project gutenberg, from a different historical period, are an outlier. on the bottom the figure shows relationships between different dialects within the same register (tweets). the core pairs are the countries which are closest in geographic terms: ireland and the uk together with australia and new zealand, with canada and the us as a distant pair. in both cases, we see that the frequencies of constructions in the grammar provide meaningful relationships between both registers and dialects. this is important because it shows that the differing frequencies of constructions are not simply arbitrary patterns from this particular model but also reproduce two sets of real-world relationships. ## clipping: the problem of parsing the analysis in this paper has categorized and described the kinds of constructions that are contained in a learned constructicon, has quantified the frequency and productivity of each kind, and has shown that the usage of these constructions can reconstruct meaningful relationships between corpora. the analysis of construction types in section 3, however, reveals a major challenge in this approach to computational cxg: the unification or clipping together of these constructions into complete utterances during parsing @xcite . the idea in cxg is that word-forms are not the basic building blocks of grammar. rather, the types of constructions analyzed in this paper form the basic units, themselves built out of slot-constraints that depend on basic category formation processes. with the exception of short utterances, however, no single construction provides a complete description of a linguistic form. these constructions must be clipped together: a sentential construction, for example, joined with a verbal construction and then a nominal construction. cxg posits a continuum between the lexicon and the grammar, so that the constructicon contains basic units at different levels of abstraction. we must distinguish, however, between first-order constructions of the type discussed in this paper and second-order constructions which are formed by clipping together these lower constructions. a complete constructicon would thus also contain emergent structures formed from multiple first-order constructions. as a desideratum for future developments, we can conceptualize two types of second-order constructions: first, slot-recursion would allow a higher-order construction to contain first-order constructions as slot-fillers. for example, the set of sentential constructions could be expanded by allowing verbal constructions to fill verbal slots. second, slot-clipping would allow two overlapping constructions to be merged, for instance connecting a transitional construction with a verbal construction. an overlapping shared slot-constraint would license such slot-clipping unifications. ## conclusions the main contribution of this paper has been to provide a qualitative linguistic analysis of a learned construction grammar, providing a new perspective on grammars which have previously been evaluated from a quantitative perspective. we presented a division of construction types into nine categories such as verbal and nominal, with those two openclass categories the most common. the discussion of examples shows both the range and the robustness of computational construction grammar. this linguistic analysis does point to two current weaknesses: first, not all constructions fit nicely into the categories used for annotation (c.f., section 3.1). a truly usage-based grammar does not necessarily align with introspection-based analysis, especially in regards to boundaries between constructions. introspection often focuses on constructions which are complete or self-contained units, while the computational constructions place common pivot points at boundaries. second, these constructions do not generally describe entire utterances, so that we must consider a form of clipping to provide complete parses (c.f., section 5). from a quantitative perspective, the analysis of register and dialectal differences shows that the productivity of these constructions also reproduces expected relationships between corpora. this is important for providing an external evaluation of the grammar: the differences between registers, for example, show how functions which are salient in a given communicative situation ultimately drive constructional frequencies. in other words, the frequencies of different types of constructions reflect meaningful patterns in real-world usage.