ENTYFI : A System for Fine-grained Entity Typing in Fictional Texts (2020)

Abstract: Fiction and fantasy are archetypes of long-tail domains that lack suitable NLP methodologies and tools. We present ENTYFI, a web-based system for fine-grained typing of entity mentions in fictional texts. It builds on 205 automatically induced high-quality type systems for popular fictional domains, and provides recommendations towards reference type systems for given input texts. Users can exploit the richness and diversity of these reference type systems for fine-grained supervised typing, in addition, they can choose among and combine four other typing modules: pre-trained real-world models, unsupervised dependency-based typing, knowledge base lookups, and constraint-based candidate consolidation. The demonstrator is available at: https://d5demos.mpi-inf.mpg.de/entyfi .

## introduction motivation and problem. entity types are a core building block of current knowledge bases (kbs) and valuable for many natural language processing tasks, such as coreference resolution, relation extraction and question answering @xcite @xcite . context-based entity typing, the task of assigning semantic types for mentions of entities in textual contexts (e.g., musician, politician, location or battle) therefore has become an important nlp task. while traditional methods often use coarse-grained classes, such as person, location, organization and misc, as targets, recent methods try to classify entities into finergrained types, from hundreds to thousands of them, yet all limited to variants of the real world, like from wikipedia or news @xcite @xcite . entity type information plays an even more important role in literary texts from fictional domains. fiction and fantasy are core parts of human culture, spanning from traditional folks and myths into books, movies, tv series and games. people have created sophisticated fictional universes such as the marvel universe, dc comics, middle earth or harry potter. these universes include entities, social structures, and events that are completely different from the real world. appropriate entity typing for these universes is a prerequisite for several end-user applications. for example, a game of thrones fan may want to query for house stark members who are faceless men or which character is both a warg and a greenseer. on the other hand, an analyst may want to compare social structures between different mythologies or formations of different civilizations. state-of-the-art methods for entity typing mostly use supervised models trained on wikipedia content, and only focus on news and similar real-world texts. due to low coverage of wikipedia on fictional domains, these methods are thus not sufficient for literary texts. for example, for the following sentence from lord of the rings: "after melkor's defeat in the first age, sauron became the second dark lord and strove to conquer arda by creating the rings" state-of-the-art entity typing methods only return few coarse types for entities, such as person for sauron and melkor or location for first age and arda. moreover, existing methods typically produce predictions for each individual mention, so that different mentions of the same entity may be assigned incompatible types, e.g., arda may be predicted as person and location in different contexts. contribution. the prototype system presented in this demo paper, entyfi (fine-grained entity typing on fictional texts, see @xcite for full details) overcomes the outlined limitations. entyfi supports long input texts from any kind of literature, as well as texts from standard domains (e.g., news). with the sample text above, entyfi is able to predict more specific and meaningful types for entity mentions: outline. the following section describes the architecture of entyfi with the approach underlying its main components. the demonstration is illustrated afterwards through its graphical user interface. our demonstration system is available at: https://d5demos.mpi-inf.mpg. de/entyfi . we also provide a screencast video demonstrating our system, at: https://youtu . ## system overview entyfi comprises five steps: type system construction, reference universe ranking, mention detection, mention typing and type consolidation. figure 1 shows an overview of the entyfi architecture. ## web interface the entyfi system is deployed online at https: //d5demos.mpi-inf.mpg.de/entyfi . a screencast video, which demonstrates entyfi, is also uploaded at https://youtu.be/g_esaonagfq . ## demonstration experience a common use of entity typing is as building block of more comprehensive nlp pipelines that perform tasks such as entity linking, relation extraction or question answering. we envision that entyfi could strengthen such pipelines considerably (see also extrinsic evaluation in @xcite ). yet to illustrate its workings in isolation, in the following, we present a direct expert end-user application of entity typing in fictional texts. suppose a literature analyst is doing research on a collection of unfamiliar short stories from fanfiction.net. their goal is to understand the setting of each story, to answer questions such as what the stories are about (e.g. politics or supernatural), what types of characters the authors create, finding all instances of a type or a combination of types (e.g. female elves) or to do further analysis like if female elves are more frequent than male elves and if there are patterns regarding where female villains appear mostly. due to time constraints, the analyst cannot read all of stories manually. instead of that, they can run entyfi on each story to extract the entity type system automatically. for instance, to analyze the story time can't heal wounds like these 3 , the analyst would paste the introduction of the story into the web interface of entyfi. after computing the similarity between the input and the reference universes from the database, entyfi would then propose the lord of the rings, vampires diaries, kid icarus, twin peaks and crossfire as top 5 reference universes, respectively. the analyst may consider the lord of the rings and vampires diaries, top 2 in ranking, of particular interest, and in addition, select the universe forgotten realms, because that is influential in their literary domain. the analyst would then run entyfi with default settings, and get a list of entities with their predicted types as results. they could then see that elladan and elrohir are recognized as living thing, elves, hybrid people and characters, while redhorn as living thing, villains, servants of morgoth, and imladris as location, kingdoms, landforms and elven cities. they could then decide to rerun the analysis with reference universes the lord of the rings and vampires diaries but without running type consolidation. by ignoring this module, the number of predicted types for each entity increases. especially, elladan & ehrohir now are classified as living thing, elves, characters, but also location and organization. similarly, redhorn belongs to both living thing and places, while imladris is both a kingdom and a devastating event. apparently, these incompatibilities in predictions appear when the system does not run type consolidation. the analyst may wonder how the system performs when no reference universe is being used. by only selecting the real-world typing module @xcite , the predicted types for el- mention settings default (ref. universes + all modules) default without type consolidation only real-world typing elladan & elrohir men, hybrid peoples, elves of rivendell, real world, elves, characters, living thing, antagonists, supernatural, species, etc. the results show not only incompatible predictions, but also that the existing typing model in the real world domain lacks coverage on fictional domains. by using a database of fictional universes as reference, entyfi is able to fill these gaps, predict fictional types in a fine-grained level and remove incompatibilities in the final results. from this interaction, the literature analyst could conclude that the story is much related to the lord of the rings, which might help them to draw parallels and direct further manual investigations. ## related work earliest approaches for entity typing are based on manually designed patterns (e.g., hearst patterns (hearst, 1992)) to extract explicit type candidates in given texts. these pattern-based approaches can achieve good precision, but their recall is low, and they are difficult to scale up. traditional named-entity recognition methods used both rule-based and supervised techniques to recognize and assign entity mentions into few coarse classes like person, location and organization @xcite @xcite . recently, fine-grained namedentity recognition and typing are getting more attention @xcite @xcite . @xcite use a classic linear classifier to classify the mentions into a set of 112 types. at much larger scale, @xcite uses 16k types from the wordnet taxonomy as the targets for entity typing. finet is a combination of pattern-based, mention-based and verb-based extractors to extract both explicit and implicit type candidates for the mentions from the contexts. with the development of deep learning, many neural methods have been proposed @xcite @xcite . @xcite propose a neural network with lstm and attention mechanisms to encode representations of a mention's contexts. recently, @xcite use distant supervision to collect a training dataset which includes over 10k types. the model is trained with a multi-task objective function that aims to classify entity mentions into three levels: general (9 types), fine-grained (112 types) and ultra-fine (10201 types). while most existing methods focus on entity mentions with single contexts (e.g. a sentence), entyfi attempts to work on long texts (e.g., a chapter of a book). by proposing a combination of supervised and unsupervised approaches, with a following consolidation step, entyfi is able to predict types for entity mentions based on different contexts, without producing incompatibilities in predictions. many web demo systems for entity typing have been built, such as stanford ner foot_1 , displacy ner foot_2 and allennlp foot_3 . however, these systems all predict only a few coarse and real world types (4-16 types). entyfi is the first attempt to entity typing at a fine-grained level for fictional texts. in a related problem, the richness of wikia has been utilized for entity linking and question answering @xcite . ## conclusion we have presented entyfi, an illustrative demonstration system for domain-specific and long-tail typing. we hope entyfi will prove useful both to language and cultural research, and to nlp researchers interested in understanding the challenges and opportunities in long-tail typing.