Dr. Livingstone, I presume? Polishing of foreign character identification in literary texts (2022)

Abstract: Character identification is a key element for many narrative-related tasks. To implement it, the baseform of the name of the character (or lemma) needs to be identified, so different appearances of the same character in the narrative could be aligned. In this paper we tackle this problem in translated texts (English–Finnish translation direction), where the challenge regarding lemmatizing foreign names in an agglutinative language appears. To solve this problem, we present and compare several methods. The results show that the method based on a search for the shortest version of the name proves to be the easiest, best performing (83.4% F1), and most resource-independent.

## introduction character identification is both a complex and a difficult task that can be solved using different methods, from manual @xcite to automatic @xcite . one of the necessary steps for character identification is to detect which exact character appears in the text @xcite . for such detection, lemmatization is required. lemmatization is a process of assigning to a wordform its lemma @xcite . it is one of the important tasks in natural language processing (henceforth nlp), since many other nlp methods require it during the preprocessing stage. for agglutinative languages, such as finnish, correct lemmatization can turn out to be a difficult task because one word can have many wordforms (e.g. a finnish word may have more than 50 wordforms. consider an example for english name lizzy in finnish translation: lizzy, lizzystä (from / about lizzy), lizzylle (to lizzy), lizzyn (lizzy's)). current state-of-the-art models that use neural networks can help with solving this task. for example, such a lemmatization model is implemented as part of the turku neural parser pipeline, which currently yields the best results for finnish lemmatization @xcite . however, their accuracy, though close to 100% foot_0 , is not perfect, so lemmatization may require further refinement which would help to enhance the end result for character identification. in this paper we discuss enhancing foreign characters' identification for english characters in finnish texts, via improving lemmatization of characters' names. the structure of the paper is as follows: first we provide an overview of the related work (section 2), subsequently we describe our data (section 3), after which we discuss the creation of the gold standard for our methods and a definition of character in the context of our research (section 4). we continue the paper with describing the methods (section 5) that we introduced and used. finally, we present our results and analyze them (section 6). we conclude our paper in section 7. code for the paper is available at https: //github.com/aleksanko/naacl2022 . ## related work lemmatization for agglutinative languages, such as the one targeted in our study (finnish), has been tackled from different perspectives. the first attempts to solve the problem for finnish were the fintwol tool @xcite and morfo @xcite . around three decades later one of the most known non-neural methods, omorfi @xcite was developed. omorfi uses finite state transducers and can be used further for enhancing lemmatization @xcite . the current state-of-the-art is represented by turku neural parser pipeline @xcite , @xcite that treats lemmatization as a sequence-to-sequence problem using the opennmt neural machine translation toolkit @xcite and yields 95%-97% accuracy. in our research, we are focusing on retrieving canonical forms of foreign names. this may be challenging since foreign names are not typically expected by the lemmatizer, so it may be prone to errors. however, this step is necessary in case of agglutinative languages: otherwise one character may split into two or more characters (for example, instead of catherine, we would have three entities: catherine, catherinea and catherinen), which affects further the results for building character networks or narrative. ## data the data used in our experiments is a corpus of finnish translations made by kersti juva (a subcorpus of the classics of english and american literature in finnish corpus, or ceal foot_2 ). the corpus consists of the short novel washingtonin aukio, 2003 ("washington square", 1880) by henry james and the novels ylpeys ja ennakkoluulo, 2013 ("pride and prejudice", 1813) by jane austen and kolea talo, 2006 ("bleak house", 1853) by charles dickens. the corpus is stored as text files, 3 files and 384,053 words in total. ## creation of gold standard before applying our methods (see section 5), we had to choose a gold standard character names' list, so that we can evaluate our methods. to perform this task, we got the information from different internet sources that contain information about characters from the novels in our dataset (see appendix a). while creating a gold standard character names' list, we also faced many questions about characters, such as: what is a literary character? who do we consider a character from the point of the narrative? who do we consider a character from the point of character extraction where we are forced to filter the results of automatic named entity recognition foot_3 ? do we take into consideration off-screen characters (characters that are only mentioned in the text and do not participate in the plot)? to answer these questions, we need to define what / who the character is. the literary character can be seen as a construct whose definition and features depend on the study area @xcite . jannidis (2013) considered a character "a text-or media-based figure in a storyworld, usually human or human-like" or "an entity in a storyworld". overall, characters are intertwined with narrative and storyworld, contributing to their development from many aspects. we considered a literary character every figure that was relevant for the narrative development (thus, e.g. names of famous persons that are mentioned but do not appear in the novel were not included). so we decided to include both onscreen (entities that are actively participating in the storyworld) and off-screen (entities that are passively contributing to the construction of the storyworld) characters (e.g. in case of washington square, it was the mother of the main character that gets mentioned only twice). we also included all possible names that can be used for naming a certain character by splitting the full name (e.g. elizabeth bennet would also get versions elizabeth and bennet) and by analyzing possible versions (lizzy for elizabeth bennet) that were mentioned in the internet sources (see appendix a). so elizabeth bennet would get the following names: bennet, eliza, eliza bennet, elizabeth, elizabeth bennet, lizzy. the creating of the gold standard was carried out only by one annotator. ## methods to apply our methods, we have to carry out the preprocessing first. this includes the following workflow: for named entity recognition on finnish texts and further lemmatization, a python library named stanza @xcite was used, because it provided a state-of-the-art level of named entity recognition for this language. finnish language models are represented in stanza by turku neural parser pipeline @xcite , so we will be using the turku neural parser pipeline's lemmatizer @xcite as a baseline. we have used and compared three methods of finding correct names' lemmas. these methods were applied on the output of the preprocessing, i.e. lists of names that were results of applying named entity recognition on finnish translations, then filtering only person-type entities, and finally lemmatizing them. the methods were implemented using python and are as follows: ## results we evaluated the results of our methods and the baseline according to the following criteria: the overall count of results can be found in in it is quite noticeable from method 1 performed well on both a short text (washington square) and a significantly longer novel (bleak house). pride and prejudice, but precision for this text was lower than for other two: 63%. both external sources that were used for method 2 (wikipedia and wiktionary) showed the worst recall results on bleak house (46% and 60%) but scored over 90% on pride and prejudice. in terms of precision, checking in wiktionary (method 2.2) performed better than using wikipedia for both washington square and pride and prejudice, while the usage of both resources led to the same result for bleak house. we assume that this result can be attributed to the difference between the names, surnames and nicknames used in these novels. method 3 achieved the second best precision overall (and the best precision for washington square), but did not show good results in terms of recall (worst for two texts out of three). while applying this method, we also noticed that, without applying additional checks, it seems to filter out a certain amount of true positive cases, since the suffixes in question (partitive and genitive) contain one or two letters and can easily be just parts of correct lemmas. in only one method can be considered languageindependent: search for the shortest version of lemma (method 1). it can also be considered the only method that does not require a lot of external sources of knowledge, since even searching for the suffixes requires knowledge of finnish grammar. the only knowledge that is required for the first method is knowledge about the type of language (agglutinative / fusional), but since the problem with wrongly lemmatized names is mostly the problem of agglutinative languages, this knowledge can be considered basic. it is worth noting that lemmatization and scrupulous study of extracted names has also shown changes in translation regarding the original text. thus, there is no guster (the servant of mr. snagsby and mrs. snagsby) in the finnish version of bleak house but molly, due to the wordplay. such changes made the creation of the gold standard more difficult since it was based on the original namings of characters. we suggest that there were also some issues related to misprints in the finnish translations (e.g. in the translation of washington square sometimes names lavinia and catherine were misprinted as lavina and catherna) which lead to additional wrong results. such errors were fixed, so the final version of results contained only right versions of names. ## conclusion perhaps surprisingly, a rather simple method that searches for the shortest version of the character's name (method 1) yielded one of the best results with average precision of 78.7%, the best overall recall (88.7%) as well as the best overall f 1 (83.4%). searching for a name in wikipedia (method 2.1) led to slightly lower precision (77.6%). searching for a name in wiktionary (method 2.2) was overall slightly worse than method 2.1 (f 1 76.1% vs 77.3%), but almost on the same level as checking if the name contains suffixes (method 3): average precision for both was about 71%. in addition, method 1 did not require any additional resources and it was relatively languageindependent which would allow it to be used without any modifications for other agglutinative languages. we suggest that a combination of these methods (for example, simple combination of method 1 and method 3 should help e.g. in case when the characters do not have common names in genres like fantasy or sci-fi) will further improve the search for the right lemmas for foreign names in texts written in agglutinative languages and thus enhance the character identification.