Robust Candidate Generation for Entity Linking on Short Social Media Texts (2022)

Abstract: Entity Linking (EL) is the gateway into Knowledge Bases. Recent advances in EL utilize dense retrieval approaches for Candidate Generation, which addresses some of the shortcomings of the Lookup based approach of matching NER mentions against pre-computed dictionaries. In this work, we show that in the domain of Tweets, such methods suffer as users often include informal spelling, limited context, and lack of specificity, among other issues. We investigate these challenges on a large and recent Tweets benchmark for EL, empirically evaluate lookup and dense retrieval approaches, and demonstrate a hybrid solution using long contextual representation from Wikipedia is necessary to achieve considerable gains over previous work, achieving 0.93 recall.

## introduction entity linking (el) is the task of linking mentions to their corresponding entities in a knowledge base (kb) such as wikidata. el is commonly formulated in three sequential steps: named entity recognition (ner), where mentions are identified, candidate generation, where a list of possible entity candidates is generated, and entity disambiguation, where a final candidate is selected. earlier el works relied on alias tables (dictionary from strings to possible wikidata entities; often associated with a score) and key-word based retrieval methods @xcite @xcite . however, these approaches suffer on noisy text, such as short-form tweets. an example of a difficult tweet would be "liam is a gr8 ml researcher" where the desired span to link would be "liam". here, an alias-based approach would only retrieve entities based on the span "liam", of which there are 8,350 different wikidata entities containing that name. without the context of "gr8 ml researcher", it quickly becomes unfeasible to find the correct candidate. furthermore, alias based approaches are also heavily dependent on the spans retrieved, where the retrieved span must be exactly present in the alias table in order to be found @xcite @xcite . this presents a challenge due to the difficulties of ner systems on noisy social media text @xcite . more recently, bert-based dense entity retrieval approaches have shown to produce sota results on news datasets such as tackbp-2010 and mewsli-9 @xcite @xcite . dense retrieval approaches rely on relevant context around the mention, which is abundant in long and clean documents such as news, but often absent or brief in noisy and short user-generated text, such as that found on twitter. prior works that focus on social media linking, such as tweeki (harandizadeh and singh, 2020), used small, annotated datasets and did not study the more recent dense retrieval approaches. recently, twitter researchers released an endto-end entity linking benchmark for tweets called tweetnerd. it is the largest and most temporally diverse open-sourced dataset benchmark on tweets @xcite . excited by the availability of this benchmark, we study the application of recent linking methods on this large and noisy user generated data. we empirically evaluate sparse and dense retrieval approaches on this data and describe the challenges and design choices of building a robust linking system for tweets. our main contributions are as follows: (a) to the best of our knowledge, we are the first study to compare dense retrieval, sparse retrieval, and lookup based approaches for entity linking in a social media setting, which makes our work relevant for the research community interested in processing noisy user generated text. (b) we assess the robustness of dense retrieval techniques in the presence of span detection errors coming from ner systems for social media text. this is a common problem for social media datasets as the top ner f1 score for social media datasets is significantly lower than other domains @xcite . (c) we assess the impact of using short wikidata entity descriptions against the longer wikipedia descriptions for representing candidates, and highlight the significant loss in performance from using shorter descriptions for social media text. this is relevant as many recent dense retrieval methods for generic entity linking have proposed using short descriptions from wikidata for candidate representations. (d) our analysis is the first to explore sparse and dense retrieval on the largest and most temporally diverse entity linking dataset for tweets called tweetnerd @xcite . (e) finally, through quantitative and qualitative analysis, we assert the complimentary nature of candidates generated by lookup and dense retrieval based approaches. this asserts the validity of our hybrid approach towards candidate generation and is reflected in significant performance improvement by using hybrid candidate generation for entity linking. ## methodology 2.1 knowledge base to represent our kb, we followed prior work and retrieved a july 2022 download of english wikipedia foot_0 @xcite . however, wikipedia also includes miscellaneous pages or pages that refer to multiple entities, such as disambiguation pages and "list of" pages. an example of such a page is "list of birds of canada" foot_1 , which describes 696 distinct birds, each with their own respective wikipedia page. to detect these pages, we retrieve the "instance of" category of each entity from wikidata, which classifies each wikipedia entity into distinct categories. using this information, we reduce the entity set from 56.8m to 6.5m wikidata entities. ## conclusion in this work, we have evaluated the usage of sparse and dense retrieval techniques towards candidate generation on social media text. in our qualitative and quantitative experimentation, we have highlighted the complementary strengths of both methods. combined, our hybrid approach achieves significant improvements on tweetnerd, a large temporally diverse dataset for entity linking on tweets. we also demonstrate the improvements that dense retrieval translates to improved downstream entity linking performance using both gold and ner based spans. there are also a few directions for future work. first, in this work we focused on the candidate generation step for entity linking. while we report preliminary results for the entity disambiguation step in appendix section a, future work could explore efficient ways to disambiguate the candidates retrieved from our hybrid approach. second, future work could expand our evaluation beyond the english tweets found in tweetnerd and develop a multi-lingual solution. third, it is important to note that there are significant linguistic differences between the formal text found on wikipedia and informal speech on twitter. recent work has explored leveraging mentions as entity descriptions, which could be applied to twitter text to bridge this gap @xcite . overall, our work highlights the best practices for improving entity linking on short and noisy social media text. we hope this work inspires future entity linking efforts on this challenging domain.