Evaluation of Coreference Resolution Systems Under Adversarial Attacks (2020)

Abstract: A substantial overlap of coreferent mentions in the CoNLL dataset magnifies the recent progress on coreference resolution. This is because the CoNLL benchmark fails to evaluate the ability of coreference resolvers that requires linking novel mentions unseen at train time. In this work, we create a new dataset based on CoNLL, which largely decreases mention overlaps in the entire dataset and exposes the limitations of published resolvers on two aspects—lexical inference ability and understanding of low-level orthographic noise. Our findings show (1) the requirements for embeddings, used in resolvers, and for coreference resolutions are, by design, in conflict and (2) adversarial approaches are sometimes not legitimate to mitigate the obstacles, as they may falsely introduce mention overlaps in adversarial training and test sets, thus giving an inflated impression for the improvements.

## introduction resolution of coreferring expressions is a natural step for text understanding, but coreference resolvers appear to have a negligible effect in downstream nlp tasks @xcite @xcite . for instance, @xcite rewrite pronouns with their antecedents (e.g., he is replaced by dominick dunne), using the berkeley entity resolution system @xcite . however, this fails to improve the cross-sentence coherence of system summaries, although the resolver performs well on the ontonotes 4.0 dataset @xcite . the conll benchmark @xcite reflects the recent advances of coreference resolution systems. nevertheless, previous work @xcite indicates that the progress on the conll benchmark is inflated, as the training and test sets share a large size of mentions. this may test example: iraqi leader saddam has given a speech to mark the tenth anniversary of the gulf war. the iraqi leader said the gulf war was a confrontation... train example: there were other signs today that iraq's leaders have few regrets over the action that precipitated the gulf war. the gulf war began 10 years ago... as opposed to evaluating on standard benchmarks, recent work @xcite @xcite investigates the generalization ability of nlp systems under adversarial attacks. for instance, @xcite show that natural language inference systems fail blatantly when lexical changes, e.g., replacing a word by its synonym, occur in premises and hypotheses. @xcite observe that spelling errors distract text classification systems from correct prediction. inspired by these works, we investigate published coreference resolvers in two realistic adversarial setups, which challenge (a) lexical inference ability to resolve coreferent mentions, where one mention is, e.g., synonymous or in a type-of relationship with its antecedent and (b) denoising ability against typographic (low-level) noise. to do so, we construct a new benchmark dataset by modifying the mention spans from @xcite . this can mitigate lexical overlaps between the conll training and test sets, as illustrated in our analysis yields several findings: (1) we show that the lexical inference ability of published resolvers, including the state-of-the-art resolver based on bert, is poor, i.e., the failure to properly resolve the coreference of a mention and its hy-pernymous (or hyponymous) antecedent within the same synset. (2) we identify an important reason for this failure: a mismatch, by design, between the requirements of coreference resolution and embeddings (used in resolvers). while a plausible coreference resolver anticipates ignoring the semantic difference of a word and its hypernym and linking them as coreferent mentions, embeddings capture the nuanced and fine-grained meanings well. (3) further, we show that coreference resolvers fail to generalize to the conll benchmark dataset with minor low-level (orthographic) noise. as a remedy, we use a common adversarial approach @xcite to incorporate lexical changes and low-level noise in coreferent mentions at train time, which appears to largely address the obstacles. however, we reveal that it introduces a large size of mention overlaps in the adversarial training and the test sets. this indicates an unrealistic situation where resolvers are only robust to what has been seen during training. these findings indicate potential directions for future work, which may benefit coreference resolvers in downstream tasks and in real-world applications with natural occurring noise (e.g., usergenerated texts). ## adversarial data collection our goal is to construct a benchmark dataset on which we evaluate the ability to resolve coreference that requires lexical inference and understanding of low-level noise. ## shielding via adversarial training shielding setup. we measure to what extent adversarial training @xcite can improve lexical inference ability and the robustness to low-level noise for the baseline systems. we include the adversarial training set at train time, but do not augment the training data, i.e., only replace 50% clean examples using our text manipulations. we split our evaluation into two setups: (1) indomain evaluation, e.g., the training and test set used for training and evaluation are modified by swapping characters and (2) out-of-domain evaluation, e.g., we use adversarial training that trains a baseline system from scratch on a modified training set of one noise, denoted as at-noise, and evaluates on the adversarial test sets of the remaining noise. ## conclusions coreference resolution have the potential to help downstream nlp systems solve problems that require text understanding. however, the performance scores on the conll benchmark are inflated, because mentions are largely overlapping in the whole dataset, and the evaluation in a constrained domain fails to expose the limitations of coreference resolvers in the wild. our experiments show that published resolvers fail to link coreferent mentions involving minor low-level noise and lexical changes. beyond that, we show a caveat when mitigating the obstacles via adversarial approaches: lexical overlaps introduced by data augmentation must be removed from adversarial training and test sets so as to see how the approaches perform realistically.