F antasy C oref: Coreference Resolution on Fantasy Literature Through Omniscient Writer’s Point of View (2021)

Abstract: This paper presents a new corpus and annotation guideline for a novel coreference resolution task on fictional texts, and analyzes its unique characteristics. FantasyCoref contains 211 stories of Grimms’ Fairy Tales and 3 other fantasy literature annotated in the omniscient writer’s point of view (OWV) to handle distinctive aspects in this genre. This task is more challenging than general coreference resolution in two ways. First, documents in our corpus are 2.5 times longer than the ones in OntoNotes, raising a new layer of difficulty in resolving long-distant referents. Second, annotation of literary styles and concepts raise several issues which are not sufficiently addressed in the existing annotation guidelines. Hence, considerations on such issues and the concept of OWV are necessary to achieve high inter-annotator agreement (IAA) in coreference resolution of fictional texts. We carefully conduct annotation tasks in four stages to ensure the quality of our annotation. As a result, a high IAA score of 87% is achieved using the standard coreference evaluation metric. Finally, state-of-the-art coreference resolution approaches are evaluated on our corpus. After training with our annotated dataset, there was a 2.59% and 3.06% improvement over the model trained on the OntoNotes dataset. Also, we observe that the portion of errors specific to fictional texts declines after the training.

## introduction coreference resolution is a core nlp task to link all mentions that refer to the same entity together in a document @xcite . a few coreference datasets have been created for several genres @xcite @xcite ; however, coreference resolution on literary texts has been comparatively underexplored. this may be due to the nature of this entertainment genre being not as important. nonetheless, with the latest advancement in neural-based coreference systems @xcite @xcite , the scope of this task is getting broader and likely emerges as a key component to teach children how to read in education applications @xcite or to conduct engaging conversations in chatbots. unlike non-fictional texts, referents in literary texts can be interpreted quite differently, depending on which point of view that the annotator takes (e.g., which character's point of view, which part of the story that the reader is at). this unique property in literature is more frequently noticeable in the fantasy genre. for instance, in the famous story snow white, the evil queen disguises herself as an old woman and gives snow white a poisonous apple. from snow white's point of view, she does not know that the old woman is her step-mother at this point, but the writer knows it. moreover, the three mentions, evil queen, old woman, and step-mother, all refer to the same entity but appear differently, which can be confusing even in the reader's point of view before one finishes reading the entire story. to accomplish the main purpose of coreference resolution, language understanding, this issue of inconsistent interpretation needs to be addressed with specific guidelines. distinguished from most of previous work, this study conducts coreference annotation in the omniscient writer's point of view, which can reflect the author's intention of how the text should be understood, and therefore, is proper for reading comprehension. also, ontonotes, the most widely known coreference dataset @xcite , does not ana-lyze this issue in its guidelines. moreover, previous studies of coreference resolution on literary texts @xcite @xcite present only limited analyses on dynamic issues that occur in fictional texts (e.g., asymmetry of knowledge, comprehensive physical or status change in entities). the lack of a high-quality dataset in terms of size and consistency is the major hindrance in this task. this inspires us to create a new coreference corpus on literature and evaluate a state-of-the-art coreference system on this genre to confirm the feasibility of this research. contributions of this work are as follows: to the best of our knowledge, this is the first publicly available corpus that provides full coreference annotation on fantasy literature. we believe that this work will lead a new perspective of coreference resolution on this unexplored domain. foot_0 2 annotation the source materials used for our coreference annotation are household tales by brothers grimm by jacob grimm and wilhelm grimm (grimms' fairy tales; henceforth, gft), which consists of 211 stories foot_1 . three additional fantasy texts (henceforth, aft) have also been annotated to be used as a separate test set. aft includes the story of aladdin and the story of ali baba and the forty thieves from the arabian nights: their best-known tales by nora archibald smith and kate douglas foot_2 with alice's adventures in wonderland by lewis carroll foot_3 . all sources are available on the project gutenberg. the materials consist solely of fantasy texts, from which literary-specific characteristics such as metamorphoses of a character, metaphorical expressions, and asymmetry of knowledge are abundantly found. meanwhile, these features hardly appear in non-fictional contexts, and thus, the existing corpora (which mostly choose non-fictions as their materials) have paid less attention to these literary features. ## error analysis for both ontonotes model and our model, the predicted results of 9 stories, randomly selected from the test set, have been examined based on a number of error types. the error categories were added or modified by the annotators throughout the analysis, and the finalized error types are found in ## conclusion in this work, we present fantasycoref, a new english annotated dataset of 367,891 tokens from grimms' fairy tales and additional fantasy literature. fantasycoref is larger in size compared to the existing coreference corpora in literature, and takes into consideration a number of issues specific to this genre. these issues are organized as guidelines having four categories of the following: 1) asymmetry of knowledge, 2) changes in entities, 3) foretelling or wishes, and 4) lexical variations. the state-of-the-art coreference system trained on our annotated corpus results in a significant improvement on fictional texts and a decrease in genre-specific errors. however, the best performance with 76.88% still has room for improvement compared to the those shown in the ontonotes dataset. hence, we believe that fantasycoref is a highly valuable resource that can be utilized to develop the current coreference resolution models and, by extension, language comprehension tasks. in the gold-children, 'the two pieces (of a golden fish)' is buried, then grows into 'two golden lilies'. although the appearance has changed throughout the growth, no further mention of 'the two pieces' appear after their growth. hence, we consider 'the two pieces' and 'two golden lilies' to be the referents. a.3 foretelling and wishes (8) (prophecy) "you must not drink [the wine which will be brought to you at night] x , and must pretend to be sound asleep." (...) the eldest came and brought him [a cup of wine] x (...) in the shoes that were danced to pieces, 'an old woman' foretells and warns 'a poor soldier' that he must not drink 'the wine' that will be served that night, and 'a cup of wine' is indeed offered to him as the warning said. we link the first appearance of an entity in a prophecy, curse, or dream (in this example, 'the wine which will be brought to you at night') to its realization in the real-world (in this example, 'a cup of wine'). in the poor man and the rich man, the man wishes for 'a house' and this wish is fulfilled as a form of 'a new clean-looking house with red tiles and bright windows'. when an entity that is wished or requested for in a story is realized into a real-world entity, we link the two as referents. in the three little men in the wood, the mother makes the girl fetch her some strawberries and later, the girl says about strawberries. in this case, since 'a basketful of strawberries' are general, we do not annotate them as the same entity. if the mother asked the girl to find some 'special strawberries', strawberries should be annotated. a.4 lexical variations (11) (variations in terms of address) when firtwister was busy cooking, [a little shrivelledup old mannikin] x came to him (...) "be off, [sly hypocrite] x ," (...) but how astonished fir-twister was when [the little insignificant dwarf] x sprang up at him, (...) "(...) they may just try their chance with [the little scrubbingbrush] x ;" (...) then [the malicious dwarf] x wanted to spring on him (...) in strong hans, 'a little shrivelled-up old manikins' is referred to as various expressions according to how other characters perceive him in the story or how the narrator depicts his appearance or personality. regardless of the variation, when a reader can perceive that several anaphoric terms of address point to a single character, these terms are grouped under the same entity. in the water-nix, 'the birds' is used as an metaphoric expression referring to 'the children' who have run away. we link metaphoric expressions to its referent when the connection between them can be perceived by the reader.