Tweeki: Linking Named Entities on T witter to a Knowledge Graph (2020)

Abstract: To identify what entities are being talked about in tweets, we need to automatically link named entities that appear in tweets to structured KBs like WikiData. Existing approaches often struggle with such short, noisy texts, or their complex design and reliance on supervision make them brittle, difficult to use and maintain, and lose significance over time. Further, there is a lack of a large, linked corpus of tweets to aid researchers, along with lack of gold dataset to evaluate the accuracy of entity linking. In this paper, we introduce (1) Tweeki, an unsupervised, modular entity linking system for Twitter, (2) TweekiData, a large, automatically-annotated corpus of Tweets linked to entities in WikiData, and (3) TweekiGold, a gold dataset for entity linking evaluation. Through comprehensive analysis, we show that Tweeki is comparable to the performance of recent state-of-the-art entity linkers models, the dataset is of high quality, and a use case of how the dataset can be used to improve downstream tasks in social media analysis (geolocation prediction).

## introduction popularity and steady increase in adoption of social media makes it a ripe domain for understanding and analyzing world events, with twitter as one of the largest social media platforms. as a result, tweets now have become a rich source of information, and twitter analysis has been widely applied for many applications such as trend detection @xcite , opinion mining @xcite , election politics @xcite , and many others. however, short length of the text, casual and error-prone writing style, and evolving topics over time is extremely challenging for existing text analysis tools @xcite . one of the common approaches is to bridge the gap between unstructured text (e.g. a tweet) and let's go to california for the disney internship structured, machine-readable knowledge bases (e.g. wikidata) using entity linking (el) that grounds named mentions to a unique, real-world entity, i.e., an entry in the knowledge base (see figure 1 for an example). entity linking has been widely applied in natural language processing @xcite @xcite on domains including news, biographical text, movie/show plots, amongst others. although entity linking has been used in social media applications as well @xcite , it is much less common. there are a number of reasons by existing entity linking systems are not commonly used for social media analysis. one of the primary concerns is that many of the existing entity linking systems are supervised @xcite , which not only makes them suitable for the domains they are trained on @xcite , but also makes them excessively reliant on context around the mention. for these reasons, supervised el systems tend to be inaccurate on noisy and short text @xcite . even unsupervised systems are heavily-engineered and complex in nature @xcite , containing rules, obso-lete lexicons, low coverage kbs, heuristic scoring, etc., making them difficult to maintain, extend, and adapt. this is especially a problem for social media where relevant (and new) entities, writing styles, and vocabulary change completely and frequently. finally, we currently lack linked datasets to use for social media analysis, such as manually annotated data to evaluate, compare, and benchmark these entity linking systems. these shortcomings need to be addressed before entity linking can be used widely for social media analysis. in this paper, we propose tweeki, a system and resource for entity linking twitter to wikidata, that addresses the above challenges. the tweeki entity linking pipeline is unsupervised, simple, and modular, thus mitigating the problems arising from the supervised setup and complexity design. the pipeline components are chosen based on their performance on short, noisy texts, and can be easily extended with more/new entities and improved taggers. further, we use wikidata foot_0 as the kb, benefiting from regular updates and higher coverage compared to wikipedia and other kbs. we run the tweeki system on a large collection of tweets to provide the first large, automatically-linked corpus of tweets, that we call tweekidata. finally, we also manually annotate a small set of tweets to provide gold annotations for entity links to create the tweekigold dataset, which can be used to evaluate and compare entity linking systems on social media text. the implementation of tweeki, and the accompanying datasets (tweekidata and tweekigold) are available publicly foot_1 . we provide a comprehensive evaluation of the pipeline, using the manually annotated tweekigold data and other tweet-based datasets. we show that tweeki performs comparably to leading sophisticated entity linking systems and proposed models for neel2016 challenge @xcite . further, we carry out analysis on mention extraction and candidate generation for different types of mentions, and show the variety and diversity of mentions and entities that get linked. finally, as a potential downstream application of tweeki, we develop a preliminary model for geolocation prediction from tweet text, and show that leveraging tweeki leads to increased the accuracy of such systems for social media analysis. in this section we propose a simple, unsupervised entity linker called tweeki that, as we will show later, performs similar to the state of the art. first, we introduce the task definition and some assumptions that we made to build tweeki, then describe the pipeline components in more detail. ## tweekidata and tweekigold in this section, we will describe tweekidata, a massive automatically-linked corpus created by running tweeki on a number of source datasets (3.1). we also introduce tweekigold, a small, manuallyannotated dataset for measuring the quality of the linker and the automatically linked dataset. ## conclusions although entity linking for social media text has many potential applications, it has not been widely adopted by the community due to presence of supervised and complex entity linking systems that are hard to maintain, extend, and apply to new entities and different writing styles. further, there is no large-scale linked corpus of tweets available for researchers to use for social media analysis, and very few gold annotated tweet datasets to evaluate and compare different entity linking systems. our proposed work, collectively called tweeki, consists of an unsupervised, extensible entity linking pipeline, a massive automatically-linked dataset of tweets (tweekidata), and a small, manually annotated dataset of gold links (tweekigold). our experiments show that the linker is accurate, and the dataset can be used to obtain improvements in downstream applications. we have released the source code and datasets from this paper at https://ucinlp.github.io/tweeki .