A Data-driven Approach to Named Entity Recognition for Early M odern F rench (2022)

Abstract: Named entity recognition has become an increasingly useful tool for digital humanities research, specially when it comes to historical texts. However, historical texts pose a wide range of challenges to both named entity recognition and natural language processing in general that are still difficult to address even with modern neural methods. In this article we focus in named entity recognition for historical French, and in particular for Early Modern French (16th-18th c.), i.e. Ancien Régime French. However, instead of developing a specialised architecture to tackle the particularities of this state of language, we opt for a data-driven approach by developing a new corpus with fine-grained entity annotation, covering three centuries of literature corresponding to the early modern period; we try to annotate as much data as possible producing a corpus that is many times bigger than the most popular NER evaluation corpora for both Contemporary English and French. We then fine-tune existing state-of-the-art architectures for Early Modern and Contemporary French, obtaining results that are on par with those of the current state-of-the-art NER systems for Contemporary English. Both the corpus and the fine-tuned models are released.

## introduction named entity recognition (ner) is an extensively studied task in natural language processing (nlp) that consists in identifying and classifying named entities mentions in unstructured text. these named entities often are real-world objects such as a person, a location, an organisation name or even a product. ner has been an important task in natural language processing for some time now. it was the focus of the muc conferences and associated shared tasks @xcite , and later that of the conll 2003 and the ace shared tasks @xcite . ner has quickly established itself as a pillar of the new methods of reading texts promoted by the digital humanities (dh), based on the analysis of large sets of literary or historical data via computational methods @xcite . these sources being not only contemporary, the need for tools dealing with medieval or early modern states of language is now increasing. ner interests researchers in dh for numerous reasons since the application can be quite broad, from genealogy or history studies for which finding mentions of persons and places in texts is very useful; to applications in digital literature where researchers can use ner to highlight the path of different characters in a book or in a series of publications. both the research in ner and dh can benefit from one another as it has already been suggested particular properties of literature can help to build better ner systems @xcite and even study how much diachronic variation influences ner systems @xcite . for the present study, we will focus on developing both an annotated corpus as well as a ner system for early modern french. we loosely define early modern french as a state of language following middle french in 1500-adopting here the terminus ad quem used by the dictionnaire de moyen français @xcite )-and ending with the french revolution in 1789. in consequence, it encompasses three centuries (16 th , 17 th and 18 th c.), or two linguistic periods: the français préclassique or "preclassical french" (1500-1630) and the français classique or "classical french" (1630-1689); both periodisations which are currently used in french linguistics (e.g. by @xcite . early modern french poses some particular challenges for ner systems, and mainly two. first, the spelling was not fixed and place names could be written differently from one text to another, but also in the same text. in early modern french, the name of the city of lyon could be written lyon, but also lion, creating in this case a homograph that has today disappeared (the lion being, like in english, an animal). second, cities have changed their names, states have appeared, empires have disappeared, etc. and it is therefore impossible to use tools available for contemporary french. in this paper we develop a system that tries to tackle these specific challenges posed by early modern french, however, instead of developing a specialised architecture for this, we opt for a data-driven approach in which we try to annotate as much text as possible of an heterogeneous corpus covering several centuries and a vast range of genres and styles. we produce a fine-grained ner annotated corpus for early modern french that is many times bigger than some of the most popular ner annotated corpora for contemporary english and french @xcite . we then fine-tune existing state-of-the-art architectures d'alembert @xcite and camembert @xcite for early modern and contemporary french respectively obtaining results that surpass the current state of the art ner systems for contemporary french @xcite , and that are on par with ner systems for contemporary english @xcite @xcite . we release both the corpus and the fine-tuned model in order to insure reproducibility of our experiments. foot_0 ## related work if many evaluation campaigns for the recognition of named entities have been carried out since the end of the nineties foot_1 , most of the corpora produced have until recently dealt with contemporary documents, particularly taken in the press (articles, dispatches. . . ). in recent years, however, research has begun to focus on "historical" documents, but the diachronic depth of the language remains imperfectly treated, with a very clear concentration on the most recent textual sources: the 19 th c. and 20 th c. are by far over-represented @xcite . if the older states of language, linguistically more complex because of the instability of their spelling, remain left aside, we do note some attempts to extract entities from texts written before the 19 th c. previous research concerns 17 th c. english (ocrised versions of the journals of the house of lords, cf. @xcite , medieval latin (charters, cf. torres aguilar et al. 2016), german and french (legal documents written between the 14 th and the 18 th c., cf. gwerder 2017). with the emergence of data-driven approaches, new corpora keep emerging for niche languages such as middle high german and old norse @xcite . french is a typical case regarding ner, with resources and solutions focusing on documents written after the french revolution. one of the oldest dataset is the one produced during the ester-2 evaluation campaign @xcite , dealing with of radio broadcast transcripts. for the older documents, we have the quaero @xcite , europeana @xcite and impresso @xcite corpora, going back the 19 th c., but again with an almost unique focus on the press. non-journalistic and/or nonrecent french, however, seem to have attracted researchers in recent years. we have already mentioned the study of @xcite , whose data has unfortunately not been manually annotated and is therefore far from being optimal, and is limited to place and person names. if older rule-based approach keep being used (for place names, cf. kogkitsidou and gambette 2020), only one project has produced a manually annotated corpus, but limited to toponyms and using normalised versions (i.e. aligned with contemporary french) of 17 th c. plays @xcite . an ambitious manually annotated corpus for pre-revolutionary non-normalised french is still needed to give the means to researchers in history, literature or linguistics to offer new interpretation, relying on quantitative approaches such as "distant reading" @xcite . if possible, this would corpus would need cover several centuries, and to offer more entities than just place and person names, such as quantities or events. ## corpus rather than designing a new corpus, we have decided to use a subpart of the "core corpus" of the presto project @xcite , namely the text written during the french ancien régime 3 . this choice is driven by our will to limit the number of annotated corpora for historical french, the same set of documents having already been abundantly corrected to train a lemmatizer @xcite , but also to avoid a complex selection of works supposed to ensure a relative representativeness of literary documents from the ancien régime, already perfectly done by our colleagues. the number of genres covered is extremely large: poetry, drama, novel, correspondence, grammar, philosophy, short stories, encyclopedic literature, etc. and guarantees, here again, a reasonable representativeness of the range of possibilities of belles-lettres foot_3 . the corpus is balanced regarding the distribution per century (c. 10/century) but not regarding the length of the texts, which increases over time (cf. fig. 1 ), following a possible trend in literature. ## conclusion in this paper we have produced a significantly big, fine-grained ner annotated corpus for early modern french, as well as state-of-the-art models for coarse ner annotation in early modern french. we showed that adopting a data-driven approach in which one focuses on producing as much annotated data as possible as opposed to producing highly specialised machine learning architectures for ner, is a quite successful approach as we have obtained results for early modern french that far surpass the current state of the art for contemporary french and that are on par with the current state-ofthe-art specialised architectures for contemporary english. the corpus that we have produced also opens many future perspectives of research, for instance, we hope that in the future we will be able to study the impact of the size of the fine-tuning data in the fine-tuning of transformer-based models, something that could be easily achieved by iteratively fine-tuning different transformer-based with subsets of our corpus of incremental size. furthermore, one could also use all the other levels of annotation of our corpus to develop a specialised architecture capable of predicting all annotation layers at once. in the end, we hope that both our corpus and our fine-tuned models will be useful to researchers in both natural language processing and digital humanities.