Ellipsis Resolution as Question Answering: An Evaluation (2021)

Abstract: Most, if not all forms of ellipsis (e.g., so does Mary) are similar to reading comprehension questions (what does Mary do), in that in order to resolve them, we need to identify an appropriate text span in the preceding discourse. Following this observation, we present an alternative approach for English ellipsis resolution relying on architectures developed for question answering (QA). We present both single-task models, and joint models trained on auxiliary QA and coreference resolution datasets, clearly outperforming the current state of the art for Sluice Ellipsis (from 70.00 to 86.01 F1) and Verb Phrase Ellipsis (from 72.89 to 78.66 F1).

## introduction ellipsis resolution is a hard, open problem in nlp, and an important source of error in machine translation, question answering, and dialogue understanding @xcite @xcite @xcite . there are no large annotated text corpora for this phenomenon, even for english, and we only have annotations for a subset of the known ellipsis constructions. since annotation is expensive and cumbersome, any synergies with existing nlp tasks could be useful and enable us to leverage auxiliary data when learning models for ellipsis resolution. this paper presents a simple yet strong approach to ellipsis resolution based on a straightforward observation, depicted in figure 1 , that ellipsis resolution can be converted to a qa problem. ellipsis and questions put in focus referentially dependent expressions @xcite , or free variables @xcite , that need to be resolved in order to comprehend the discourse. for similar observations about different tasks, see @xcite and @xcite . ## methodology in this section, we briefly describe the various datasets used for training, and explain how they are converted into qa format. we then move on to the choice of model architectures and the reasoning behind their selection. ## experiments & results we conduct two sets of experiments: (i) the single-task experiments, in which we train and evaluate separate models for the two ellipsis resolution tasks; and (ii) the joint modelling experiments, where we train on the best possible combination of ellipsis resolution, coreference resolution and qa data, as determined on the validation set. the results can be seen in ## dataset ablations we determine the best task combinations on heldout validation data for each ellipsis resolution task. 5 for sluice ellipsis, the best results are obtained by training the models on a combination of sluice and vp ellipsis data. for vp ellipsis, the best performance is attained when the models are trained with a combination of all datasets. when training a model for a particular task, we sample auxiliary data from other datasets to match the size of the main task's dataset. for each dataset, the variations in its f 1 scores of the best performing architecture when combined with other datasets are shown in figure 2 . the most interesting findings from these ablations are mentioned below. when the two ellipsis datasets are combined, the overall performance of the models increase for both tasks by around 1% each. this shows that the two types of ellipsis are similar, and that when learning ellipsis resolution models, there is considerable synergy between the two resources. if we add subsampled coreference data when training these models, the verb phrase ellipsis models gain up to 2.9%. one possible explanation then at 10:15, the dow suddenly started to rebound, and when it shot upward it did so even faster than the early-morning fall. ## error analysis we now look at some errors made by our best performing models. first, we compare the errors made by our single-task and joint sluice ellipsis resolution models before moving on to vp ellipsis. 6 sluice ellipsis the joint sluice ellipsis results improve modestly over the single-task sluice ellipsis results. this is noteworthy, since the added vp ellipsis data is quite small compared to the size of the sluice data. these models consistently select an antecedent of the right syntactic form, which is normally a complete sentence. many of the errors consist of empty outputs: single-task sluice ellipsis produces 58 empty outputs, while joint sluice ellipsis produces 63. another source of error is discontiguous antecedents. it is not unusual for the gold antecedent to be a discontiguous span @xcite , but our models are not permitted to produce such antecedents, so these cases will always be a source of error. all the systems have problems when the antecedent follows the ellipsis, as in the following example: i don't know why, but they seem to need a story. we also compared the right and left periphery scores of sluices, and found better results predicting the right periphery: for single-task sluice ellipsis, there were 678 matches on the left edge, and 733 on the right edge; for joint sluice 6 we also briefly discuss how coreference resolution benefits from synergies with ellipsis in appendix c.1. ellipsis, there were 703 left matches and 734 right matches. ## related work we are not the first to use question answering to redefine a set of tasks. recently, @xcite showed that semantic role labeling annotations could be solicited by asking simple questions that implicitly target predicate-argument relations in a sentence. parallel to our work, hou (2020) cast bridging anaphora resolution as question answering based on context. @xcite and @xcite 2020) also reformulate coreference resolution and named entity recognition as qa. in the realm of re-framing relation extraction as a qa problem, levy et al. (2017) and abdou et al. (2019) create monolingual and multilingual template based qa datasets respectively, which yield relation extraction models which were better at generalizing in the zero-shot setting. extending this idea, mccann et al. ( 2018 ) introduced the de-canlp challenge, which casts 10 core tasks in nlp as question-answering problems. similar to our work, their architecture jointly learns across all of these tasks. decanlp includes pronoun resolution, a subset of coreference resolution, but it does so only on a small, hand-crafted dataset; it does not address ellipsis. limitations of our approach one limitation of our approach is that, like most previous work, we assume ellipsis and coreference resolution amount to finding antecedent spans that corefer with the target mention. this is not always the case; the elided material can: (i) have extra-linguistic antecedents, and (ii) refer to something that is contextually implied. ## conclusion we present strong models for sluice and verb phrase ellipsis resolution problems, by reformulating them as machine reading comprehension problems, significantly outperforming the previously best reported results. we also empirically show that training these models jointly and with auxiliary data from coreference resolution and question-answering further improves their performance. our code is publicly available at https://github.com/rahular/ellipsis-baselines .