A uto A spect: Automatic Annotation of Tense and Aspect for Uniform Meaning Representations (2021)

Abstract: We present AutoAspect, a novel, rule-based annotation tool for labeling tense and aspect. The pilot version annotates English data. The aspect labels are designed specifically for Uniform Meaning Representations (UMR), an annotation schema that aims to encode crosslingual semantic information. The annotation tool combines syntactic and semantic cues to assign aspects on a sentence-by-sentence basis, following a sequence of rules that each output a UMR aspect. Identified events proceed through the sequence until they are assigned an aspect. We achieve a recall of 76.17% for identifying UMR events and an accuracy of 62.57% on all identified events, with high precision values for 2 of the aspect labels.

## introduction as the field of natural language processing advances, there are increasing demands for more sophisticated applications and richer representations. abstract meaning representations (amr; @xcite , and their more recent crosslingual incarnation as uniform meaning representations (umr; @xcite , are a response to that demand. amr/umrs provide an abstract, directed acyclic graph representation of a complete sentence, focusing on the underlying "who" did "what" to "whom" elements of the events being described. the more information that can be associated with those events, in terms of whether they have been completed, or whether they have achieved their intended results, the better. the increased richness of umr tense, aspect and modality annotations, as described below, can more clearly identify the completion and achievement of events in a cross-lingual context, provid-ing a firmer baseline for comparing typologically distinct languages. automating such a complex semantic processing task provides valuable qualitative and temporal crosslingual features that applications like translation models and virtual assistants can utilize to more accurately capture the semantic nuances of events. given the substantial amounts of english amr annotation, the question immediately arises of how to efficiently add these new annotation features to pre-existing english amrs. this paper describes an implementation of an automatic system that relies on verbnet, a rich lexical resource, as the basis for categorizing event descriptions according to the aspect guidelines discussed below. our initial results are quite promising, and there are obvious next steps to take. ## background information previous automatic annotation models operate on different definitions of aspect. in @xcite , clauses are annotated for situation entity types, which capture some of the same semantic distinctions as the umr aspect annotation scheme, including state and habitual. @xcite also include modal distinctions in their annotation, such as questions and imperatives. unlike @xcite , the umr aspect annotation distinguishes between different types of dynamic (non-stative) clauses (activity, endeavor, and performance); these are all annotated as event in @xcite . @xcite annotate a binary telicity distinction: telic vs. atelic. the umr aspect annotation annotates a three-way distinction for non-stative events, which takes both the qualitative and temporal dimensions of event semantics into account. the umr aspect annotation consists of a feature assigned to events that indicates their internal qualitative and temporal structure @xcite . every node classified as an "event" in umr receives an aspect annotation. umr defines "event" based on the typological prototype of a verb as described in @xcite , and croft (in press). umr events exhibit either the prototypical information-packaging of a verb, predication (as opposed to modification and reference), or the prototypical semantic class of a verb, a process (as opposed to a property or an entity). the aspectual distinctions made in umr are based on @xcite 's two-dimensional analysis of aspect and build on the aspect annotations from @xcite . @xcite analyzes aspectual structure as having both a temporal dimension and a qualitative dimension; the temporal dimension measures out the event's unfolding over time and the qualitative dimension measures out the change that occurs (or does not occur) during the event. the umr aspectual distinctions do not have a direct correspondence with either specific verbs or specific constructions in a language; instead, they annotate the aspectual structure of an event in its context. in order to ensure the maximum cross-linguistic comparability of annotation values, umr uses lattices of compatible annotation values for certain annotation categories, including aspect @xcite . in addition to the lattices, umr also considers a certain level of specificity in annotation as the base level, corresponding to distinctions that tend to be straightforward to annotate in a majority of languages. for aspect, there are four base-level annotation values: state, activity, endeavor, and performance. in addition, there is a ha-bitual value. the annotation of these categories manually relies on a series of decisions that distinguish event types. first, event nominals are annotated as process, one of the more coarse-grained categories on the lattice. processes in reference (i.e., event/action nominals) lack grammatical clues as to their aspectual structure in english and many other languages; therefore, they are simply annotated as process. next, the habitual value applies to all events that are repeated on a regular basis, regardless of the internal aspectual structure of each individual (repeated) event. this means that further aspec-tual distinctions (including between process and state) are collapsed for habitual events. the rest of the aspectual values make a distinction between stative and non-stative events. state is the base-level value that captures all stative events; non-stative events are divided into activ-ity, endeavor, and performance. umr defines stative events as those in which no change occurs on the qualitative dimension during the event @xcite . in addition to prototypical stative events, umr extends the state value to "nonverbal predication" (he is a teacher. / there is a sandwich in the kitchen.), events modalized by ability modals (this car can go 100mph.), and modal complement-taking predicates (she wants to eat at noon.). in addition, umr uses the state value for a class of events termed "inactive actions" in @xcite . inactive actions are semantically intermediate between states and processes; this class includes verbs of sensation, perception, cognition, emotion, and position. these types of events can variably be construed as either states or processes, even within the same language. since the construal of an inactive action can be hard to ascertain in context, umr annotates all inactive actions with the state value. the rest of the base level aspect annotations (activity, endeavor, performance) work to characterize non-stative events (processes in umr). these events are defined as those that involve change on the qualitative dimension. the distinction between the activity annotation value and the endeavor and performance values is based on whether the event is still ongoing at document creation time (dct), or whether it has ended. events annotated as activity indicate that the event may be ongoing at dct. both endeavor and performance characterize non-stative events that have ended prior to dct. the endeavor and performance values differ from each other in whether the event has been terminated or completed. events annotated as en-deavor signal that the event has terminated, without reaching completion. this means that the event has not reached a distinct result state on the qualitative dimension. the performance label indicates that an event has been completed, reaching a distinct result state. the umr performance value corresponds to vendler's achievements and accomplishments. umr also incudes a number of more finegrained aspectual types on its aspect lattice (see @xcite . many of these finegrained aspect values make distinctions that crosscut the base-level annotation values. these include whether an event is punctual or durative, which cross-cuts the endeavor/performance distinction. for durative endeavors and per-formances, in addition to all activities, there is also the cross-cutting distinction of incremental vs. nonincremental change @xcite . the base-level umr aspectual values were selected because they capture the most salient pieces of aspectual structure and they can be consistently annotated, even in languages with minimal grammatical aspect marking like english. the presence or absence of change on the qualitative dimension is captured by the process vs. state distinction. boundedness on the temporal dimension is captured for processes by the distinction between activity and endeavor/performance. finally, boundedness on the qualitative dimension is captured by the distinction between endeavor and performance. ## rule-based classification the rule-based classifier foot_0 follows the sequential manual annotation steps as closely as possible, immediately exiting the sequence as soon as an annotation label has been assigned by a numbered step. the annotation loop processes text by sentence, such that every identified event in a sentence receives an annotation. ## results many of the annotation steps are themselves complex semantic processing tasks. therefore, the performance of this rule-based model depends heavily on the limits of the classifier components being utilized. given the lack of substantial training data that would be required for a machine learningdriven implementation, we focus on specific examples that reveal the linguistic blind spots of our model. ## conclusion the linguistic blind spots discovered from analyzing the results highlight many areas for future development for the autoaspect annotator. it is clear that the primarily syntax-driven rules are unable to capture semantic properties like stativity, disambiguation of prepositional phrases, and genre of text. feeding in input sentence-by-sentence also prevents autoaspect from properly analyzing multi-sentence events. the tool currently has a recall of 76.17% for identifying events and a 62.57% accuracy for identified events. as more gold umr data is processed, further linguistic blind spots can be identified for development past the pilot version.