Unsupervised Entity Linking with Guided Summarization and Multiple-Choice Selection (2022)

Abstract: Entity linking, the task of linking potentially ambiguous mentions in texts to corresponding knowledge-base entities, is an important component for language understanding. We address two challenge in entity linking: how to leverage wider contexts surrounding a mention, and how to deal with limited training data. We propose a fully unsupervised model called SumMC that first generates a guided summary of the contexts conditioning on the mention, and then casts the task to a multiple-choice problem where the model chooses an entity from a list of candidates. In addition to evaluating our model on existing datasets that focus on named entities, we create a new dataset that links noun phrases from WikiHow to Wikidata. We show that our SumMC model achieves state-of-the-art unsupervised performance on our new dataset and on exiting datasets.

## introduction entity linking (el) is an important natural language processing (nlp) task that associates ambiguous mentions to corresponding entities in a knowledge base (kb, also called knowledge graph). el is a crucial component of many nlp applications, such as question answering @xcite and information extraction @xcite . although there have been significant and continuous developments of el, most work requires sufficient labeled data and a well-developed kb @xcite @xcite . however, many real-world applications, especially those in specific domains, suffer from scarcity of both training data and a fully-populated kb. previous research has tackled this problem by learning el models without data labeled entity links, but requires indirect supervision in the form of textual descriptions attached to entities in kbs, drawn from sources such as wikipedia @xcite . however, such descriptions may not be available in kbs in low-resource domains such as medicine or law. thus, we focus on fully unsupervised el, which only has access to the entities' names and their kb relations like subclass-of @xcite . one challenge of unsupervised el is leveraging useful information from potentially noisy and misleading context @xcite . specifically, a local context (the sentence containing the mention) may not be sufficient for disambiguating the target mention without the global context (other sentences in the document). for example, in figure 1 , the target mention 'band' cannot be disambiguated solely with the local context "this band is so lovely", but needs to consider the global context that also includes "i can't wait for my wedding." to address this problem, we introduce an unsupervised approach to el that builds on the strengths of large neural language models like gpt-3 @xcite . we use zero-shot gpt-3 prompting for two sub-tasks. first, we perform guided summarization, which summarizes the input document conditioned on the target mention and outputs a condensed global context. then, we cast el to a multiple-choice selection problem where the model chooses an entity from a list of candidates. we refer to our unsupervised el model as summc (summarization+multiple-choice). with a few exceptions @xcite , the majority of el work targets named entities, such as names of people and organizations @xcite , neglecting entities such as physical objects or concepts. to comprehensively evaluate our model, we create the first el dataset on procedural texts, wikihow-wikidata, which links noun phrases from wikihow foot_0 to wikidata foot_1 entities @xcite . our summc model outperforms current stateof-the-art (sota) unsupervised el models on our new wikihow-wikidata data, as well as existing benchmarks including aida-conll @xcite , wned-wiki and wned-clueweb dataset @xcite . in addition, we also provide ablation studies to show the positive influence of generating guided summaries. 3 ## methodology fully unsupervised el is the task that links a target mention from a given document to some entities in a kb without requiring any text data to be labeled with explicit links to the kb. the only available information in the kb is the names of the entities and the relations among them. in this paper, we follow previous work @xcite and use wikidata as our target kb, which defines instance-of and subclass-of relations between entities. wikidata can be seen as a knowledge graph with entities as nodes and relations as edges, and the popularity of an entity can be represented by its degree. we now introduce summc, our proposed unsupervised el model which consists of two instances of a generative language model. the first performs guided summarization by generating a summary of the document conditioned on a mention. the second casts el to a multiple-choice selection problem and chooses an appropriate entity from a list of candidates generated by some heuristics. in our work, we use gpt-3 as the language model due to its superior performance on various nlp tasks @xcite . candidate generation. following previous work @xcite , we first select all entities from wikidata whose name or alias contains all tokens in a mention. then, we narrow it down to the top 20 entities with the highest degree (in-degree + out-degree) in the kb. for each entity in the final list, we produce a textual representation by concatenating the names of all related entities. for example, the representation of the candidate ribbon in figure 1 is ribbon: costume component, textile. summc. the first application of gpt-3 performs a guided summarization of the input document. with zero-shot prompting, gpt-3 summarizes the texts using the prompt "[d] summarize the text above in one sentence: [m]", where [d] is the input document and [m] is the target mention. here, we force gpt-3's summarization to start with the mention to ensure that the conditioned summary contains both the target mention and related global context. at this point, the generated summary serves as a global context while the sentence containing the mention serves as a local context, both of which help disambiguate the target mention. the second application of gpt-3 casts the task to multiple-choice selection following many successful cases @xcite . with the two contexts, gpt-3 transforms el to a multiple-choice question using the prompt "according to the context above, which of the following best describes [m]?", followed by the representations of the mention [m]'s candidates as choices. ## wikihow-wikidata dataset most work on el has targeted named entities, especially in the news. to account for more diverse entities in different styles of texts, we create a humanannotated dataset called wikihow-wikidata that links noun phrases in procedural texts to wikidata. the research revolving around entities in procedural texts have long received much attention in the community @xcite @xcite , without existing large-scale datasets of entity links in such a style of texts. to create the dataset, we first extract 40,000 articles from the wikihow corpus @xcite detailing everyday procedures. to select mentions to link, we choose the top 3 most-frequentlyoccurring nouns from each article using a part-ofspeech tagger, assuming that most mentions in a document share the same word sense @xcite . then, we ask students from a university in the u.s. to manually link these mentions to some wikidata entity. finally, to measure and control annotation quality, we manually annotate a subset of examples beforehand as control questions. details about our data collection process, interface, and measures for quality control can be found in appendix b. eventually, wikihow-wikidata consists of 11,287 triples of a wikihow article, a target mention, and a wikidata entity. ## experiments we evaluate our summc model along with other strong baselines on some widely used el datasets and our wikihow-wikidata dataset. ## results and discussion we show our results in ). eigentheme assumes that gold entities in a document are topically related @xcite . it captures global context only using the relations between mentions while neglecting the texts in the document. however, this assumption might not always hold. our model, in contrast, removes this assumption by producing a guided summary of texts in the document. ## conclusion we introduce summc, a fully unsupervised entity linking model that first produces a summary of the document guided by the mention, and then casts the task to a multiple-choice format. our model achieves new state-of-the-art performance on var-ious benchmarks, including our new wikihow-wikidata, the first el dataset on procedural texts. notably, our approach of guided summarization may be applied to other tasks that benefit from global contexts. future work might also extend our methods to supervised settings.