Conjunct Resolution in the Face of Verbal Omissions (2023)

Abstract: Verbal omissions are complex syntactic phenomena in VP coordination structures. They occur when verbs and (some of) their arguments are omitted from subsequent clauses after being explicitly stated in an initial clause. Recovering these omitted elements is necessary for accurate interpretation of the sentence, and while humans easily and intuitively fill in the missing information, state-of-the-art models continue to struggle with this task. Previous work is limited to small-scale datasets, synthetic data creation methods, and to resolution methods in the dependency-graph level. In this work we propose a conjunct resolution task that operates directly on the text and makes use of a split-and-rephrase paradigm in order to recover the missing elements in the coordination structure. To this end, we first formulate a pragmatic framework of verbal omissions which describes the different types of omissions, and develop an automatic scalable collection method. Based on this method, we curate a large dataset, containing over 10K examples of naturally-occurring verbal omissions with crowd-sourced annotations of the resolved conjuncts. We train various neural baselines for this task, and show that while our best method obtains decent performance, it leaves ample space for improvement. We propose our dataset, metrics and models as a starting point for future research on this topic.

## introduction natural language is economic, and many elements in the message are not spelled out but omitted by speakers, left for the receiver of the message to complete. the hearer, either a human or an algorithm, should then complete the missing information and recover the intended meaning. this kind of omission and recovery occurs at all levels of conversation, from syntax to pragmatics, and is performed naturally and intuitively by humans, to the extent that they often don't even realize that something was missing in the message they received. of that amount, about 75% was based on natural gas, 18% on oil, and 7% on hydroelectric power. of that amount, about 75% was based on natural gas. of that amount, about 18% was based on oil. of that amount, about 7% was based on hydroelectric power. figure 1 : a demonstration of the conjunct resolution task. at the top, the input sentence with the omitted words in bold and the coordinating element is underlined. below, the expected rewrite, consisting of a single fully-specified standalone statement for each of the conjuncts. the rewrite must not include the coordinating element. an important class of omission phenomenaand the focus of this work -involves verbs and their arguments, especially around coordination structures. a verb and some of its arguments that appear in an initial clause may be omitted in subsequent clauses. for example, in the sentence "josh likes wine and jane water", the verb likes is omitted from the phrase "jane likes water" and has to be inferred. we find that state-of-the-art syntactic parsers @xcite fail consistently even on toy examples such as this one, and assign a structure in which "jane water" is interpreted as a noun-compound which is the object of josh's liking, together with water (figure 2 ). downstream applications, such as, google translate 1 also fail. google translate translates the above sentence to french as "josh josh likes wine and jane water. aime le vin et l'eau de jane". when back-translated to english using the same engine, it results in "josh likes jane's wine and water". significantly more complex sentences involving verbal omissions are of course also possible, and nlp systems routinely fail around them. linguistically, such verbal omissions are studied under the terms ellipsis and gapping, and the research broadly aims at categorizing the verbal omission cases into sub-categories, carefully documenting the condition under which different omissions can or cannot occur @xcite @xcite . for language technology purposes, however, we would like to focus on detection, resolution and usability, not categorization. the need to recover empty elements automatically and robustly in large-scale calls for a unifying approach, by which to consider the different cases of verbal omissions as instances of a single phenomenon, that can be effectively addressed by a broad-coverage nlp system. due to failures of even language technology applications on such constructions, in this paper, we suggest that these verbal omission phenomena should be addressed explicitly by systems, rather than hoping it will be caught on as a side product of end-to-end neural training. to this end, our aim is to establish a verbal-omission recovery task, and to create a supporting large-scale corpus documenting such verbal omissions in naturally occurring english sentences, together with automatic annotation models that recover the implicit information and make it explicit. how should a system that resolves such missing information look like? what are its outputs? in the syntactic parsing and treebanking literature, there has been an ongoing debate regarding the best way to represent recovery of omissions in coordinated structures @xcite @xcite @xcite @xcite @xcite @xcite . however, none of the solutions are fully satisfactory. first, all of them are highly technical in nature, and require significant linguistic expertise in order to even understand the notation. often they assume some formal tree or graph notation, which makes their adoption unlikely by people who work on nlp systems but who may lack the specific syntactic expertise. moreover, how can one feed such theory-loaded graph-based annotation into nlp systems for downstream, user-facing tasks? to mitigate this, our proposed solution is strictly within the text-to-text paradigm, and involves a rewriting task where the input is a sentence exhibiting a coordinated structure, and the output is the set of sentences which contain the same information, but where the implicit information is made explicit. we illustrate an input and output example in figure 1 . as we further discuss below, this text-enrichment approach has several appealing properties: first, it is natural and easily comprehensible to any competent speaker of the language. second, this caters for both large-scale annotation efforts and task adoption by potential users. lastly, it produces an output which can trivially be fed into any language processing system which takes natural language text as input. we collect a corpus of 10,206 sentences involving a wide range of verbal omissions in coordinated structures, annotated via this text-rewriting task to recover the missing elements. using this corpus, we conduct experiments on the omission-recovery task. training a t5-based model to perform the task shows that the performance saturates after seeing only 10% of the data, but the accuracy of the neural model is far from being perfect, leaving ample room for future modeling and improvements. foot_2 ## related work previous research on verbal omissions in coordination structures has classified such phenomena into (at least) six categories: (1) conjunction reduction @xcite ; (2) gapping @xcite @xcite ; (3) vp ellipsis @xcite ; (4) sluicing @xcite ; (5) pseudo-gapping @xcite ; and (6) argument clusters. all of these phenomena are forms of ellipsis and are considered as ways to use language efficiently. how do nlp systems cope with such complex syntactic phenomena? work around gapping in the syntactic parsing community has focused on representation schemes for this phenomena in dependency graphs. various representation has been proposed. for instance, the universal dependencies (ud) framework @xcite introduces the concept of an "orphan" dependency to indicate the presence of an ellipsis, and @xcite provides a detailed analysis of how gapping constructions could be represented using ud. related work @xcite utilize such schemes by promoting a new head of the clause in cases of gapping and attaching all remnants to it. however, having a representation schema for a construction does not mean that an automatic parser is able to accurately predict it. to bridge this gap (no pun intended), several efforts have been made to increase the training data size for these constructions by enriching existing data or creating artificial datasets. @xcite proposed data enrichment methods that utilize existing annotated parse trees to mimic the structure of elliptical constructions, and @xcite trained a parser on artificial elliptical treebanks, achieving an f1 score of 36% on a small dataset (an improvement relative to prior work). moreover, @xcite ; kato and matsubara (2020) proposed a reconstruction algorithm at the dependency graph level, but relied on an oracle to identify the gapped sentences. despite these efforts, current approaches do not fully address the issue of verbal omissions in coordination structures, and remain scattered. additionally, these methods are not suitable for large-scale applications and are not ready for use in downstream tasks. in this work we take a more realistic, consistent and unified approach, wherein all verbal omissions are treated within the same framework, for which we provide a gold benchmark, a crowdsourcing interface, models for the tasks demonstrating feasibility and efficacy. ## data collection process we collect a dataset of 10,206 examples of a wide array of omission cases, which can serve for both training and evaluation. we aim for the collected sentences to cover a wide range of omission cases. the underlying data was sourced from three publicly available datasets: squad 2.0, dailymail, and cnn @xcite , with a roughly equal proportion of sentences from each one. our proposed collection protocol involves two steps: (1) automatic collection of sentences that are likely to contain interesting omission phenomena; (2) manual annotation of the sentences via crowd-sourcing. ## conjunct resolution dataset our conjunct resolution dataset consists of 10,206 verbal omission sentences, each paired with one of the conjunctions: "and", "or", and "but" ( non-rewritable sentences. 491 of the 10,206 (4.9%) were marked as non-rewritable. of these, 445 contain an explanation. 6 the reasoning behind deciding whether a sentence is rewritable or not seems to be non-trivial. for instance, consider the following two sentence: despite that both sentences are cases of gapping, only the second is rewritable. to recognize this, the reader (human or model) needs to be able to identify when two events are bound together by another piece of information. in the first sentence, the phrase "but i wouldn't be surprised if it was the other way round" lacks context when appearing in the rewritten sentences. however, for the second sentence, there is no such issue, and is thus rewritable. foot_7 ## evaluation metric as no task is complete without an evaluation metric, we propose an automatic evaluation metric for the proposed conjunct resolution as rephrasing task. for evaluation, we are interested in measuring three things: (1) how accurate the model is at resolving verbal omissions, (2) how often does the model omit other information after resolution, and (3) how often the model generates extra information. to address these three criteria, we propose to measure recall and precision over the predicate-argument relations recovered by a dependency parser on the generated compared to the gold sentence set. ## experiments we evaluate neural models on the task, both in supervised fine-tuning and in in-context learning setups. for the supervised fine-tuning case, we measure both task performance as well as the dependence of performance on dataset size. as a concluding experiment, we take the best-performing model and manually evaluate it against the task definition. dataset split and preprocessing. we shuffle and then split our dataset to train (80%), validation (10%), and test (10%) sets. each input instance contains a sentence and a marked conjunction. each output instance is a sequence of sentences. for feeding the text to the models, we mark the conjunction using a special token foot_9 and separate the output sentences using special tokens. 10 supervised fine-tuning. we fine-tune ten t5large @xcite ) models, using increasingly larger subsets of the training data, from 10% to 100% in increments of ten. all models were trained with the same hyperparameters and finetuned for five epochs, with the best performing model on the validation set being saved and subsequently evaluated on the test-set (for the detailed training configuration, see a.2). in-context learning / prompting. we evaluate the state-of-the-art gpt text-davinci-3 model from openai, in an in-context learning (prompting) fashion @xcite . to obtain in-context examples, we randomly sampled for each test instance three re-writable and one non-rewritable sentence, sharing the conjunction with the test instance. further details of the prompt and parameters are available in the appendix a.2. ## conclusions we present a novel approach for studying verbal omissions in coordination structures. previous research in this area has been fragmented, focusing on individual phenomena. in contrast, we propose a unified approach which considers all conjunction related verbal omissions under the same framework, by introducing a text-to-text conjunct-resolution task, to resolve omitted verbs and their arguments. we compiled and curated a large dataset of conjunction related verbal omissions, consisting of over 10,000 sentences and human annotations, which serves as a valuable resource for further research in this area. our results using state-of-the-art models as neural baselines demonstrate that this task is challenging and merits further work.