Decomposing and Recomposing Event Structure (2022)

Abstract: We present an event structure classification empirically derived from inferential properties annotated on sentence- and document-level Universal Decompositional Semantics (UDS) graphs. We induce this classification jointly with semantic role, entity, and event-event relation classifications using a document-level generative model structured by these graphs. To support this induction, we augment existing annotations found in the UDS1.0 dataset, which covers the entirety of the English Web Treebank, with an array of inferential properties capturing fine-grained aspects of the temporal and aspectual structure of events. The resulting dataset (available at decomp.io) is the largest annotation of event structure and (partial) event coreference to date.

## introduction natural language provides myriad ways of communicating about complex events. for instance, one and the same event can be described at a coarse grain, using a single clause (1), or at a finer grain, using an entire document (2). similar in spirit to this prior work, but different in method, our work aims to develop an empirically derived event structure classification. where prior work takes a top-down approachhand-engineering an event classification before deploying it for annotation-we take a bottom-up approach-decomposing event structure into a wide variety of theoretically informed, crosscutting semantic properties, annotating for those properties, then recomposing an event classification from them by induction. the properties on which our categories rest target (i) the substructure of an event (e.g., that the building described in (1) consists of a sequence of subevents resulting in the creation of some artifact); (ii) the superstructure in which an event takes part (e.g., that laying a house's foundation is part of building a house, alongside framing the house, installing the plumbing, etc.); (iii) the relationship between an event and its participants (e.g., that the contractors in (1) build the house collectively through their joint efforts); and (iv) properties of the event's participants (e.g., that the contractors in (1) are animate while the house is not). to derive our event structure classification, we extend the universal decompositional semantics dataset (uds; @xcite . uds annotates for a subset of key event structure properties, theoretical approaches vendler categorizes event descriptions into four classes: statives (3), activities (4), achievements (5), and accomplishments (6). as theoretical constructs, these classes are used to explain both the distributional characteristics of event descriptions as well as inferences about how an event progresses over time. ## annotation protocol we annotate for the core event structural distinction not currently covered by uds, breaking our annotation into three subprotocols. for all questions, annotators report confidence in their response to each question on a scale from 1 (not at all confident) to 5 (totally confident). 3 event-subevent annotators are presented with a sentence containing a single highlighted predicate followed by four questions about the internal structure of the event it describes. q1 asks whether the event described by the highlighted predicate has natural subparts. q2 asks whether the event has a natural endpoint. the final questions depend on the response to q1. if an annotator responds that the highlighted predicate refers to an event that has natural parts, they are asked (i) whether the parts are similar to one another and (ii) how long each part lasts on average. if an annotator instead responds that the event referred to does not have natural parts, they are asked (i) whether the event is dynamic, and (ii) how long the event lasts. all questions are binary except those concerning duration, for which answers are supplied as one of twelve ordinal values (see @xcite : effectively no time at all, fractions of a second, seconds, minutes, hours, days, weeks, months, years, decades, centuries, or effectively forever. together, these questions target the three vendler-inspired features (dyn, dur, tel), plus a fourth dimension for subtypes of dynamic predicates. in the context of uds, these properties form a predicate node subspace, alongside factuality, genericity, and time. event-event annotators are presented with either a single sentence or a pair of adjacent sentences, with the two predicates of interest highlighted in distinct colors. for a predicate pair (p 1 , p 2 ) describing an event pair (e 1 , e 2 ), annotators are asked whether e 1 is a mereological part of e 2 , and vice versa. both questions are binary: a positive response to both indicates that e 1 and e 2 are the same event; and a positive response to exactly one of the questions indicates proper parthood. prior versions of uds do not contain any predicate-predicate edge subspaces, so we add document-level graphs to uds ( §6) to capture the relation between adjacently described events. this subprotocol targets generalized event coreference, identifying constituency in addition to strict identity. it also augments the information collected in the event-subevent protocol: insofar as a proper subevent relation holds between e 1 and e 2 , we obtain additional fine-grained information about the subevents of the containing event-for example, an explicit description of at least one subevent. event-entity the final subprotocol focuses on the relation between the event described by a predicate and its plural or conjoined arguments, asking whether the predicate is distributive or collective with respect to that argument. this property accordingly forms a predicate-argument subspace in uds, similar to protoroles. ## validation experiments we validate our annotation protocol (i) by assessing interannotator agreement (iaa) among both experts and crowd-sourced annotators for each subprotocol on a small sample of items drawn from existing annotated corpora ( §4.1-4.2); and (ii) by comparing annotations generated using our protocol against existing annotations that cover (a subset of) the phenomena that ours does and are generated by highly trained annotators ( §4.3). ## corpus annotation we collect crowd-sourced annotations for the entirety of ud-ewt. predicate and argument spans are obtained from the predpatt predicate-argument graphs for ud-ewt available in uds1.0. the total number of items annotated for each subprotocol is presented in event-subevent these annotations cover all predicates headed by verbs (as identified by ud pos tag), as well as copular constructions with nominal and adjectival complements. in the former case, only the verb token is highlighted in the task; in the latter, the highlighting spans from the copula to the complement head. event-event pairs for the event-event subprotocol were drawn from the uds-time dataset, which features pairs of verbal predicates, either within the same sentence or in adjacent sentences, each annotated with its start-and endpoint relative to the other. we additionally included predicate-argument pairs in cases where the argument is annotated in uds as having a word-net supersense of event, state, or process. to our knowledge, this represents the largest publicly available (partial) event coreference dataset to date. event-entity for the event-entity subprotocol, we identify predicate-argument pairs in which the argument is plural or conjoined. plural arguments are identified by the ud number attribute, and conjoined ones by a conj dependency between an argument head and another noun. we consider only predicate-argument pairs with a ud dependency of nsubj, nsubjpass, dobj, or iobj. our goal in inducing event structural categories is to learn representations of those categories on the basis of annotated uds graphs, augmented with the new uds-e annotations. we aim to learn four sets of interdependent classifications grounded in uds properties: event types, entity types, semantic role types, and event-event relation types. these classifications are interdependent in that we assume a generative model that incorporates both sentence-and document-level structure. 5 document-level uds semantics edges in uds1.0 represent only sentence-internal semantic relations. this constraint implies that annotations for cross-sentential semantic relations-a significant subset of our event-event annotationscannot be represented in the graph structure. to remedy this, we extend uds1.0 by adding document edges that connect semantics nodes either within a sentence or in two distinct sentences, and we associate our event-event annotations with their corresponding document edge (see figure 1 ). because uds1.0 does not have a notion of document edge, it does not contain @xcite fine-grained temporal relation annotations, which are highly relevant to event-event relations. we additionally add those attributes to their corresponding document edges. generative model algorithm 1 gives the generative story for our event structure induction model. we assume some number of types of events t event , roles r role , entities t ent , and relations r rel . figure 4 shows the resulting factor graph for the semantic graphs shown in figure 1 . ## comparison to existing ontologies to explore the relationship between our induced classification and existing event and role ontologies, we ask how well our event, role, and entity types map onto those found in propbank and verbnet. importantly, the goal here is not perfect alignment between our types and propbank and verbnet types, but rather to compare other classifications that reflect top-down assumptions to the one we derive bottom-up. implementation to carry out these comparisons, we use the parameters of the posterior distributions over event types θ (ev) p for each predicate p, over role types θ (role) pa for each argument a of each predicate p, and over entity types θ (ent) pa for each argument a of each predicate p as features in an svm with rbf kernel predicting the event and role types found in propbank and verbnet. we take this route, over direct comparison of types, to account for the possibility that information encoded in role or event types within verbnet or propbank is distributed differently in our more abstract classification. we tune l2 regularization (λ ∈ {1, 0.5, 0.2, 0.1, 0.01, 0.001}) and bandwidth (γ ∈ {1e-2, 1e-3, 1e-4, 1e-5}) using grid search, selecting the best model based on performance on the standard ud-ewt development set. all metrics reflect ud-ewt test set performance. role type comparison we first obtain a mapping from uds predicates and arguments to the propbank predicates and arguments annotated in ewt. each such argument in propbank is annotated with an argument number (a0-a4) as well as a function tag (@xmath0, @xmath1, etc.). we then compose this mapping with the mapping given in the propbank frame files from propbank rolesets to sets of verbnet classes and from propbank roles to sets of verbnet roles (agent, patient, theme, etc.) to obtain a mapping from uds arguments to sets of verbnet roles. because a particular argument maps to a set of verbnet roles, we treat predicting verbnet roles as a multi-label problem, fitting one svm per role. for each argument a of predicate p, we use as predictors [θ (ev) p ; θ (role ## conclusion we have presented an event structure classification derived from inferential properties annotated on sentence-and document-level semantic graphs. we induced this classification jointly with semantic role, entity, and event-event relation types using a document-level generative model. our model identifies types that approximate theoretical predictions-notably, four event types like vendler's, as well as proto-agent and proto-patient role types like dowty's. we hope this work encourages greater interest in computational approaches to event structural understanding while also supporting work on adjacent problems in nlu, such as temporal information extraction and (partial) event coreference, for which we provide the largest publicly available dataset to date.