Exhaustive Entity Recognition for C optic: Challenges and Solutions (2020)

Abstract: Entity recognition provides semantic access to ancient materials in the Digital Humanities: it exposes people and places of interest in texts that cannot be read exhaustively, facilitates linking resources and can provide a window into text contents, even for texts with no translations. In this paper we present entity recognition for Coptic, the language of Hellenistic era Egypt. We evaluate NLP approaches to the task and lay out difficulties in applying them to a low-resource, morphologically complex language. We present solutions for named and non-named nested entity recognition and semi-automatic entity linking to Wikipedia, relying on robust dependency parsing, feature-based CRF models, and hand-crafted knowledge base resources, enabling high accuracy NER with orders of magnitude less data than those used for high resource languages. The results suggest avenues for research on other languages in similar settings.

## introduction recent developments in high quality nlp have been likened to a tsunami @xcite , powered largely by big data for tasks such as named entity recognition (ner), and continuous meaning representations in the form of word embeddings, for english and other languages @xcite . meanwhile, low resource and historical languages have not been able to take advantage of these advances for several reasons: 1. gold datasets for most tasks are much smaller -english ontonotes 5.0 ner @xcite has 1.7m words, more than all digitized text available in many ancient languages; 2. work on embeddings often assumes at least ∼30 million words of training data @xcite ); 1 3. for historical and non-standardized languages, orthographic variation, regional differences, lacunae and other phenomena make learning from unlabeled data exceptionally hard. in this paper, we report on a series of experiments, including successes and failures, in creating historical language resources for the coptic language. the main contributions of this work are threefold: the coptic language coptic is the last stage of ancient egyptian, written in the first millenium ce in a modified greek alphabet with additional letters derived from egyptian demotic script. coptic writings are abundant, ranging from religious writings, such as hagiographies, homilies, magical texts, and ascetical treatises to social documents including letters, legal documents, and administrative records @xcite . a vast amount of original compositions survive next to translated texts, especially of religious writing. many early monastic centers wrote extensively in coptic and many manichaean and gnostic works survive only or primarily in coptic. although it was rarely the most common bureaucratic language, coptic papyri provide crucial information about official and economic affairs in egypt during roman, byzantine, and arab rule. as such, coptic literature is important to general history and the history of christianity and other religions in and outside egypt in antiquity. coptic grammar presents challenges to automated processing, including agglutinative morphology (fusion of multiple affixes to content words, as in (1)), incorporation (e.g. compound verbs which contain fused verbal arguments, as in (2)), and spelling variation which characterizes many ancient texts, including for widespread greek loanwords (3). in (1), the sequence corresponding to 'she had made him hear' is spelled in coptic without spaces (or hyphens), meaning segmentation is needed before individual words, and then entities, can be recognized. in (2), a compound verb fuses to 'soul', changing the form of the verb and again requiring splitting. in (3), spelling variation makes the greek kollēgion hard to recognize. these challenges are not the focus of this paper, and our results below will be based on normalized, gold segmented text. however they mean that in practice, entity recognition will degrade when applied to automatically segmented and normalized text. since humanists often have very high standards of accuracy, we therefore require robust solutions, as well as possibilities of incorporating semi-automatic correction steps, which we discuss below. ## experiments in automatic entity recognition mention detection before we can evaluate entity classification and linking, we must first find entity span candidates in texts. as baselines we consider: a. using all and only nouns as entity spans (noun); and b. using all and only sequences of words attested as an entity in the training data (lookup). the noun strategy will only recall single token mentions, and include incorrect non-referring expressions. lookup should have few false positives, but low recall, since novel strings in the test data will be missed. as competitive solutions we consider two families of methods: 1. in parse, entities are assumed to cover the spans of phrases headed by nouns, as identified by the syntax tree; 2. sequence: sequences of words in the corpus are scanned and classified as entities using a neural sequence tagger. since we have reference syntax trees for our data, we can evaluate performance with gold and predicted trees from an automatic parser. as a 'sequence' based system, we train a state of the art nner system @xcite , which relies on a bidirectional recurrent neural network (rnn) with biaffine attention. the system scores spans based on start and end token indices, considering all possible spans, including nested mentions, and outputs a probability for each span category, or 'no-entity' (='o(outside)' in bio encoding) for spans not predicted as mentions. the system relies on word embeddings, which we provide using word2vec @xcite based on ∼1 million tokens of unannotated, automatically segmented coptic text from coptic scriptorium @xcite with a vocabulary size of ∼11,000 types and 50 dimensional representations. ## applications distant reading entity information can help address a variety of research questions, as well as making data more accessible and easier to discover in the larger dh ecosystem @xcite . as a first way of looking at entities in coptic text, we can consider how to visualize the global picture of entity frequencies in our data as a kind of distant reading @xcite . two interactive visualizations we can use for this purpose are entity term networks and recursive treemaps @xcite . entity term networks visualize a head word's relationships with other words in its entity spans. figure 2a captures part of the network for ma ma 'place'. larger nodes represent more frequently recurring terms, and broad arrows correspond to frequent transitions. the network for ma gives us a clearer idea of its potential semantic relationships: often preceded by pek/petn pek/petn 'your', almost always followed by n n 'of', continuing to nouns indicating purpose ('place of dwelling', or 'lavatory' with rmh rmē 'urination'), events (yeleet šeleet 'wedding'), directions (ya ša 'east') and more. similarly, the approach using dependency trees to identify nested noun phrases is viable and substantially more accurate for data in the several 10k-100k tokens range. although our approach relies on the existence of such syntactically annotated data to train a parser, the size of the data set is likely to be a more realistic target for projects in similar settings to ours than the size of datasets underlying standard approaches to (n)ner in modern languages. for comparison, the universal dependencies treebanks for ancient greek and latin include over 400,000 and 800,000 tokens respectively, meaning that the treebank used here is very modestly sized; and even with larger treebanks, those languages too lack highly expressive word embeddings based on hundreds of millions, or even billions of words, which are commonly available for modern european languages. our methods therefore show promise for mention detection in annotating other resources for classical languages, such as freely available ancient greek and latin texts available in the perseus digital library @xcite . for entity classification, our approach shows the robustness of feature based classifiers (section 3), while also revealing the added value of a knowledge base (kb) architecture. a kb derived strictly from the training data is already helpful for the hybrid approach taken here (kb+crf in section 3), while an even broader coverage kb can be constructed with relatively low effort, which makes it possible to capture some of the rarer, but often lexically unambiguous entity types, such as names of animals or plants. for entity linking, we show modest results in terms of recall, but quite good precision which can facilitate larger manual annotation efforts by offering reliable suggestions for human review. in section 4 we outlined some of the applications that wide-coverage entity annotations can bring for humanists, including style and genre studies, highlighting differences between documents that are otherwise similar from the text type perspective, and bird's eye-view visualizations which allow us to examine how texts talk about entities, and how often. the data used for the example studies in this paper comes from the high quality, but small, manually annotated corpus prepared for this work. we plan to publish a much larger automatically annotated corpus of coptic texts using the tools described here, which we expect to deliver much more comprehensive capabilities in exploring the contents of coptic texts, many of which still have no translations that might allow exploring the content in english. such larger scale data could also be used to link to other projects featuring entity identifiers (in coptic or otherwise), and to lexicographic projects, such as the coptic dictionary online @xcite , or the database and dictionary of greek loanwords in coptic (ddglc) @xcite .