The F airy N et Corpus - Character Networks for G erman Fairy Tales (2021)

Abstract: This paper presents a data set of German fairy tales, manually annotated with character networks which were obtained with high inter rater agreement. The release of this corpus provides an opportunity of training and comparing different algorithms for the extraction of character networks, which so far was barely possible due to heterogeneous interests of previous researchers. We demonstrate the usefulness of our data set by providing baseline experiments for the automatic extraction of character networks, applying a rule-based pipeline as well as a neural approach, and find the neural approach outperforming the rule-approach in most evaluation settings.

## introduction the creation and publication of data sets that can be shared without restrictions is a driving factor in the progress of digital humanities. in this paper we describe a new resource, comprising 40 german fairy tales by the brothers grimm. our corpus contains the annotation of all fictional entities and their references as well as a manually created character network for each of the texts. the main contribution of this work is a publicly available resource foot_0 that serves the purpose of supporting the creation of automatic algorithms for character network extraction. the paper is structured as follows: first, we give a small overview of the most prominent works that deal with automatic extraction of character networks as well as the very heterogeneous ways the authors evaluate the networks. we then describe the data and its origin, our guidelines and the interrater agreement we achieved on the different layers of annotation in more detail in section 3. in section 4 we present preliminary experiments that can be referred to as baselines in future projects using the corpus, before concluding the paper. since to the best of our knowledge no freely available data set comprising character networks for prose texts that originate a standardized set of guidelines exist, we cover the different approaches that dealt with the automatic extraction of character networks and depict how they went around this issue of a missing data set. for a wide overview of this field we refer to @xcite . for textual media, the methods to create character networks share many similarities, especially in regard to the lexical preprocessing involved. the similiarities end, when we compare how character networks are evaluated. the field is still very heterogeneous, rendering direct comparisons of different approaches almost infeasible. by publication of this data set we hope to address this point and help to support convergence in terms of character network evaluation. most previous work only evaluates their extracted networks by proxies. the system of @xcite evaluate by assuming the character and co-occurence frequencies (between characters) are distributed by a power law, which they were able to recover. the work of @xcite uses modern and classical literature and compare their character networks using social network analysis (sna), but found that there is no significant difference between these two sets. the work of coll ardanuy and sporleder (2014) extracted networks and subsequently clustered them to examine whether the networks can be used as proxies for unsupervised genre and authorship attribution, but could only slightly exceed their baselines. the work of @xcite can be seen as one of the first papers that dealt with the automatic extraction of character networks. they validated their networks by the verification of different literary theories (e.g. they suspected that the more characters are involved in a story, the less dialogue is con-tained). @xcite and @xcite extracted interactions between characters using relations of the categories interact and observe which they learn using a kernel support vector machine. they evaluated their system based on sna metrics, such as the node-in-degree-centrality or out-centrality. the method of @xcite evaluated the networks by measuring the overlap between nodes and edges. their results suggest that coreference resolution is the limiting factor of their approach, only reaching f1-scores of 0.3. the work of @xcite , extracted gold character networks from labeled expert summaries of novels, and compared the gold networks with system networks extracted from the full text novels using ranking metrics. ## data and annotation the texts that we plan to annotate are the fairy tales of the seventh edition of the children's and household tales by the brothers grimm for the german language foot_1 . currently, we finished annotating 40 fairy tales alongside their character networks. each fairy tale now consists of two files: the first file is the text (modernized with a tool from deutsches textarchiv foot_2 @xcite ) with markup representing the references of the characters (including coreference annotations) and the second file contains a manually annotated character network in the form of a ".xlsx" file (for automatic usability we will release .json as well), containing several layers of annotations, described in the following section. ## baseline experiments in order to demonstrate the usefulness of our data set, both for the automatic extraction and evaluation of character networks, we conduct baseline experiments using a rule-based pipeline and an approach based on neural networks. both approaches detect and cluster character references. the rulebased version uses the named entity recognition of @xcite and the coreference module of @xcite , while the neural network c2f @xcite conducts both steps in an endto-end fashion. c2f was pre-trained on droc and fine-tuned on the fairy tales using 5 folds, so that the entirety of our corpus could be labeled by the system. 6 the resulting entities form the nodes in the networks. as a proxy for interactions (which represent the edges in the networks) we used cooccurence counts for a given discours (in our work we differentiate between a sentence or a paragraph as discours). the corresponding edge weights are the number of times the two entities appear in the same discours. currently, we do not infer relation types, which is left for future work. ## conclusion we present a corpus of 40 german fairy tales by the brothers grimm with hand-annotated coreference information and character networks. this data set can be used to train and compare different approaches for the automatic extraction of character networks and helps to homogenize the evaluation of research on this topic. we demonstrated its usefulness by comparing two different systems available for the automatic extraction of character networks. we found that even state-of-the-art approaches can not reliably extract character networks, even for a seemingly easy domain such as fairy tales, evaluated on the overlap of characters in the network and the general shape of the resulting graph. for future work, we plan to conduct experiments for the automatic prediction of all further annotated layers, especially sentiment, attributes and relations of/between the characters.