Enhancing State-of-the-Art NLP Models for Classical A rabic (2023)

Abstract: Classical Arabic, like all other historical languages, lacks adequate training datasets and accurate “off-the-shelf” models that can be directly employed in the processing pipelines. In this paper, we present our in-progress work in developing and training deep learning models tailored for handling diverse tasks relevant to classical Arabic texts. Specifically, we focus on Named Entities Recognition, person relationships classification, toponym sub-classification, onomastic section boundaries detection, onomastic entities classification, as well as date recognition and classification. Our work aims to address the challenges associated with these tasks and provide effective solutions for analyzing classical Arabic texts. Although this work is still in progress, the preliminary results reported in the paper indicate excellent to satisfactory performance of the fine-tuned models, effectively meeting the intended goal for which they were trained.

## introduction arabic chronicles and biographical collections preserve a plethora of information on long-term environmental and societal processes that shaped and molded islamic societies. numerous and extensive, these written texts are the richest "mine" of information and are particularly valuable for the period before the 15th century, for which exceptionally few archival documents are available. our work focuses on constructing the social history of the islamic world from historical and biographical texts that constitute a significant part of the arabic written tradition. the project studies a vast corpus of digitized texts and relies on a series of computational methods for identifying and linking relevant information from the corpus. currently, the project is at the stage of fine-tuning relevant nlp-based approaches. the work described in this paper is the methodological foundation for the creation of the network of knowledge. this network will serve as the main research framework of the project for the study of the social history of the islamic world. classical arabic, like all other historical and ancient languages, lacks adequate training datasets and accurate "off-the-shelf" models that can be directly employed in the processing pipelines. in light of this, our objective is to make a valuable contribution to the field by creating comprehensive training datasets for various tasks related to classical arabic text processing and analysis. furthermore, we aim to develop, train, and fine-tune models that can be easily integrated and shared with fellow researchers in the community, facilitating their work and promoting further advancements in the field of classical arabic language processing. in the following sections, we present our inprogress work in developing and training deep learning models tailored for handling diverse tasks relevant to classical arabic texts. specifically, we focus on ner, person relationships classification, toponyms classification, onomastic section boundaries detection, onomastic entities classification, as well as date recognition and classification. ## related work recent advancements in deep learning and language modeling have significantly propelled the development of natural language processing (nlp) models for the arabic language. several transformer-based language models are currently available and provide state-of-the-art performance in various downstream tasks. arabert (antoun et al., 2020) is the first transformer-based language model for the arabic language. the camel lab (obeid et al., 2020) introduced a collection of pre-trained models for several arabic nlp tasks such as part-ofspeech (pos) tagging, named entity recognition (ner), sentiment analysis, and text classification. these models have been trained using different corpora, namely, classical arabic ca, dialectal arabic da, modern standard arabic msa, and the mixed corpus which comprises all available corpora. farasa foot_0 (abdelali et al., 2016) offers diverse solutions and models for arabic text processing. it also provides a restful api, allowing users to access its functionalities and leverage languageindependent solutions. further, several pre-trained models have been trained for different downstream tasks such as arat5 ( nagoudi the majority of the models mentioned in this context have been trained primarily on modern arabic texts. however, their applicability to classical arabic texts varies in terms of performance. no-tably, the camelbert-ca model foot_1 is the only model that is specifically trained on classical arabic texts. it offers the highest initial performance, if compared to all the other models. serving as a cornerstone for our research, this model formed the basis for our initial annotations and subsequent fine-tuning, allowing us to adapt it to our specific tasks and requirements. ## corpus texts utilized in our project are a sub-corpus of the openiti corpus (nigst et al., 2023). @xcite at the moment, our sub-corpus includes 101 multi-volume texts (c. 71 million tokens), which include approximately 495 thousand biographical records. most of these texts-about 70 of them-come from the period of 1000-1600 ce and from all the major regions of the islamic world, spanning from spain (al-andalus) and north africa (al-maġrib), to egypt (mis . r) and syria (al-šām), to iraq (al-'irāq), iranian provinces (fārs, khurasān, etc.) and central asia (mā-warā'-l-nahr). figure 1 illustrates a typical structure of biographies in our corpus (figure 2 offers a translation for additional clarity). the onomastic section, which provides details about the biographee's name, genealogy, origins, as well as some social and religious background, is typically located at the beginning of the biography. the onomastic section may also mention members of the immediate and extended family. this section is usually followed by information about the biographee's education: with whom they studied, in which cities, and, sometimes, what specifically they studied. the section on teachers is often followed by a section on biographee's students, who are listed in a structurally similar manner. in some biographies, descriptions of the biographee's characteristics are given as well, either as the opinion of the main text's author or as opinions of other earlier biographers. in the middle, biographies often include other important facts from the life of biographees. usually, concluding sections of biographies provide information on the date and place of biographee's death, and, occasionally, the location of biographee's burial. ## methodology manually created data plays a crucial role in the training process of machine learning models. large and accurate training datasets are particularly important to the development of more precise models with improved performance. historical and classical languages pose a unique challenge as there is often a lack of readily available training datasets. creating such datasets requires domain experts with specialized knowledge to perform accurate data annotation. given the limited resources available, we have decided to employ the active learning process (wang and hua, 2011) as a solution to generate training data for the various tasks we aim to tackle. figure 3 illustrates the active learning paradigm we adapted as an efficient strategy to produce accurate training data to be used for model training. figure 4 illustrates a lightweight annotation scheme that we developed to increase the easiness, speed, and accuracy of manual annotation. @xcite after preparing our sub-corpus, we began working with a random subset of biographies. initially, we utilized camel lab models for lemmatization, named entity recognition (ner), and partof-speech (pos) tagging to perform the initial annotation steps. also, in the initial stage, we employed our own rule-based models for date recognition and classification. following this, we proceeded with the first round of manual refinement and correction conducted by domain experts, resulting in the successful correction of 1,100 biographies. to ensure consistency and accuracy, annotators performed cross-validation to ensure the correctness and consistency of the annotations. using this refined dataset, we fine-tuned the ner model and trained a model specifically designed to detect boundaries of the onomastic section. subsequently, we employed these models to annotate a subset of biographies and repeated the cycle of manual correction and fine-tuning. this iterative process will continue until we achieve a stable performance that meets our expectations, enabling the models to accurately perform the intended tasks. ## ner for classical arabic named entities recognition aims to identify entities within the biographies and classify them into three main categories, namely, toponym, per-son, and misc. this task can be viewed as a token classification task, where each token in the text is assigned a specific label. there is a notable distinction between modern arabic names and traditional arabic (islamic) names. modern names often follow a similar structure to western names, including a given name and a surname or family name. traditional arabic names typically consist of up to six different elements, though not all of these elements have to be present in each case and they may appear in any order. traditional arabic names and their elements are explained in more detail in the section 6.2. ## onomastic entity recognition for classical arabic each biography starts with a robust onomastic section on the biographee. the onomastic section is particularly valuable as it gives information on various backgrounds of the biographee. to identify the respective descriptors in the text, two different models are required. the first model identifies the boundaries of the onomastic section in the text of a biography. applied to the identified onomastic section, the second model recognizes and classifies different onomastic elements. ## date recognition, classification, and parsing to numerical values. the research focus of the project is on the period of c. 600-1600 c.e. and information from the texts is assigned to a certain point in time somewhere within this period. we are particularly interested in what kind of informationhistorical events-can be associated with specific points in time. first, the model searches the arabic text with a regular expression (regex) which will match phrases reporting on dates. the regex matches days, days of the week, months, and years. additionally, it captures ten preceding tokens, which are considered the context of a date. at the moment, we are primarily interested in years and their thematic contexts. @xcite when the regex finds an occurrence of a date phrase, it usually returns two main groups of elements. one of the groups is the context; another one is a series of spelled-out numerals of the date statement, including ones, tens (decades), hundreds (centuries), and, in late texts, a thousand (for the first millennium). the model then uses a dictionary that returns numerical values of date statement element. summing up these numerical values gives us the actual value of the date. additional regex is then applied to the date context to check if it has any of the most common contextual vocabulary. for example, tokens like wulida (he was born) or wulidat (she was born) are used to classify dates as dates of birth. if the context contains more than one term from the date classification dictionary, we use the one closest to the date statement. we defined six main classes of dates, which are birth, death, knowledge transfer, appointment or termination of an office, pilgrimage, and undefined dates. so far, the model only returns information about a date if it's explicitly stated in the recognized phrase. however, not all information is always presented explicitly. a common instance is the omission of the century, as authors often expect readers to infer the exact century from the context. consequently, we need to enhance our model to derive any missing data from other dates provided in the biography, headers of chapters containing the biography (especially when biographies are grouped into periods-a common practice in our another challenge involves handling date statements that refer to periods or approximate years. for example, instead of exact numbers, we may find words like ba'd . and nayyif, which refer to an unspecified year within a specified decade. while these date statements cannot be translated into precise numerical values, this limitation does not severely impact our project. given the extensive period we are studying, we typically operate on the granularity of decades, rounding exact years to the nearest decade when necessary. additionally, authors sometimes report alternative dates for the same event, either by detailing both dates fully or by abbreviating the second date. in such cases, we make an effort to collect and process both dates. ## conclusions and future work in the preceding sections, we have outlined our efforts in adapting existing state-of-the-art arabic nlp models to specific research tasks. we fine-tuned an ner model, specifically tailored for historical and biographical texts in classical arabic, which exhibits excellent performance in detecting persons and toponyms. moreover, we trained models to further classify detected persons, based on how they are related to the biographee. this model achieved good performance, particularly in the classification of teachers and students within the biographical context. the model for toponym classification is still under development as we are lacking sufficient training data. we are currently working on increasing our training dataset for this task. further, we trained a boundaries detection model to locate the onomastic section inside biographies, and yet another model that identifies onomastic elements within that section. the model for date recognition, classification, and parsing achieved promising results for the main goals of the project. still, date recognition is not a trivial task and we are researching ways to overcome limitations such as the missing centuries. although this work is still in progress, the preliminary results reported in the paper indicate the excellent-to-satisfactory performance of the finetuned models, effectively meeting the intended goal for which they were trained. however, our ongoing efforts involve expanding the training datasets and further fine-tuning the models with the aim of achieving even better results. finally, our contribution extends beyond the trained models themselves. we have also developed and curated valuable training datasets that can serve as a resource for other researchers and contribute to the advancement of work in the field of classical arabic. these datasets provide a foundation for further exploration and improvements in the current models. our future work can be summarized as follows. in the short term, our primary focus lies in generating additional training data to facilitate further fine-tuning of the models. this iterative process is aimed at continuously improving the performance of the models until reaching a stage where they can effectively annotate the entire corpus. figure 4 illustrates an exemplar output of our annotation pipeline, wherein the annotated text has undergone processing by all the discussed models. after annotating all biographies in our corpus and extracting all relevant metadata (onomastic elements, persons, toponyms, dates, etc.), our subsequent objective is to organize this information into networks comprised of overlapping thematic clusters. these thematic clusters will serve as an analytical framework, enabling us to explore and derive insights from the interconnections and relationships among various social, professional, and religious groups, within extensive historical and geographical contexts as they are recorded in our vast corpus. additionally, the project explores the development of these networks through spatial and temporal analysis, which is grounded in the recognition of dates and toponyms. overall, this network will serve as the main research framework of the project for the study of the social history of the islamic world. further, this project will help to identify weakly researched topics in the field of arabic studies and at the same time provide a new research tool for fellow researchers to start working on these topics.