Decontextualization: Making Sentences Stand-Alone (2021)

Abstract: Models for question answering, dialogue agents, and summarization often interpret the meaning of a sentence in a rich context and use that meaning in a new context. Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window. We isolate and define the problem of sentence decontextualization: taking a sentence together with its context and rewriting it to be interpretable out of context, while preserving its meaning. We describe an annotation procedure, collect data on the Wikipedia corpus, and use the data to train models to automatically decontextualize sentences. We present preliminary studies that show the value of sentence decontextualization in a user-facing task, and as preprocessing for systems that perform document understanding. We argue that decontextualization is an important subtask in many downstream applications, and that the definitions and resources provided can benefit tasks that operate on sentences that occur in a richer context.

## introduction many applications of natural language processing need to be able to interpret, or present, text independently from the rich context in which it occurs. for example, summarization systems extract salient information from documents and present it in a reduced context. many systems also segment documents prior to interpretation of retrieval for computational efficiency. in all of these cases, we would like the context-reduction step to be meaning preserving but, to date, there has been no independent method of ensuring this. ## linguistic background we start with the following definition: ## task definition an annotator is provided with an entire document d with a target sentence within the document, represented as a start and end index s st , s end . first, the annotator decides whether the target sentence can be decontextualized or not, labeling it as feasible or infeasible. if the example is marked as feasible, the annotator decontextualizes the sentence, producing y, a new sentence that satisfies the conditions in definition 1. ## two applications we present two demonstrations of the utility of decontextualization. first, we argue that the decontextualized sentences can be valuable in themselves in question answering, and show that they can be useful as a preprocessing step. ## related work prior literature in summarization studied how article context affects the understanding of sentences within an article. it has been observed that disambiguating entity mentions and correctly resolving anaphora is crucial for automatic summarization @xcite and for evaluation of summarization systems @xcite . @xcite identified that information missing from a sentence could be identified in the article context in newswire text 60% of the time. this is considerably less frequent than for the encyclopedic text studied here, but nevertheless hints that decontextualization for newswire text could be feasible. it remains unclear whether information accessible in newswire contexts can be readily incorprated into sentences using controlled edits of the type we employ. successful decontextualization models must resolve entity and event coreferences @xcite as well as other forms of anaphora @xcite . these are necessary but insufficient for decontextualization however, which also involves discourse marker removal, acronym expansion, and fluent and grammatical sentence generation. the term decontextualization was introduced in a recent table-to-text generation dataset @xcite where a sentence from a wikipedia document was decontextualized such that it can be interpretable when presented with a table alone. they cover only the sentences that are relevant to the table, and adapt it to the table context. in a recent image captioning dataset @xcite , sentences are re-written such that information that cannot be inferred from the image is removed. for example, entity names are replaced with generics (e.g., -tom cruz , +a man is waiting.''). ## conclusion we define decontextualization, the task of rewriting a sentence from a document to be interpretable in an empty context, while preserving its meaning. we build a crowdsourced dataset and a model for decontextualization, and demonstrate how decontextualization can be used in a user-facing task and as a sub-component of an application system. we believe that decontextualization will also be helpful in a wide range of other applications. for example, in multi-document summarization @xcite , co-referring entities and events must be resolved across different documents and removing ambiguous references may help; extractive summarization @xcite could benefit from the type of pre-processing that we presented for open-domain qa; anaphora resolution is crucial for both summarization and machine translation @xcite ; and decontextualizing sentences may help in recovering explicit mentions of entities and relations which can help information extraction @xcite . the current formulation focuses on the english encyclopedic corpus and rewriting for an empty context, and future work can explore different domains of text as well as mapping to a different context.