Code-Switching and Back-Transliteration Using a Bilingual Model (2024)

Abstract: The challenges of automated transliteration and code-switching–detection in Judeo-Arabic texts are addressed. We introduce two novel machine-learning models, one focused on transliterating Judeo-Arabic into Arabic, and another aimed at identifying non-Arabic words, predominantly Hebrew and Aramaic. Unlike prior work, our models are based on a bilingual Arabic-Hebrew language model, providing a unique advantage in capturing shared linguistic nuances. Evaluation results show that our models outperform prior solutions for the same tasks. As a practical contribution, we present a comprehensive pipeline capable of taking Judeo-Arabic text, identifying non-Arabic words, and then transliterating the Arabic portions into Arabic script. This work not only advances the state of the art but also offers a valuable toolset for making Judeo-Arabic texts more accessible to a broader Arabic-speaking audience.

## introduction judeoarabic is a family of ethnolects spoken and written by various jewish communities living in arabicspeaking countries, from geonic times (9th century) down until the late 20th century. the lan guage is typically written in hebrew letters, en riched with diacritic marks that relate to the under lying arabic. however, inconsistencies in render ing arabic words in the hebrew alphabet increase the level of ambiguity of a given written word. fur thermore, judeoarabic texts usually include non arabic words and phrases, such as quotations or borrowed words from hebrew and aramaic. on judeoarabic, see, for instance, @xcite . fig ure 1 is an example of an original text written in judeoarabic in the eleventh century. a wealth of judeoarabic works (philosophy, bible translation, biblical commentary, and much more) is already available on the internet. how ever, most speakers of arabic are unfamiliar with the hebrew script, let alone the way it is used to render judeoarabic. thus, our primary goal in this endeavor is to allow arabic readers, who are unfamiliar with hebrew, to nevertheless read and understand these texts. a very large quantity of ancient texts written in judeoarabic was found in the cairo geniza. this treasure trove of handwritten documents, treatises, and books-mostly fragmentary-was discovered in the late 19th century in the attic of old cairo's benezra synagogue, and has profoundly impacted the fields of jewish studies, mediterranean and in dian history, and semitic linguistics. this unique collection spans over a millennium, from the 9th to 19th century ce, offering invaluable insights into the daily lives, religious practices, commerce, and intellectual pursuits of the jewish communities and their neighbors in egypt and the mediterranean world. comprising letters, legal documents, reli gious texts, and fragments of various languages, including hebrew, aramaic, arabic, and judeo arabic, the geniza illuminates the dynamic intercul tural exchanges and adaptations within this diverse jewish diaspora. its discovery significantly ex panded understanding of medieval mediterranean society and continues to be a rich source for schol arly research, shedding light on a fascinating and variegated tapestry of human history and culture @xcite . images of virtually all this material are viewable on the internet as part of the friedberg genizah project. 1 other digital projects and libraries have made additional judeoarabic texts readily accessible. the ktiv project of the national library of israel links to scans of thousands of pages of medieval codices. 2 the princeton geniza project provides access to images and transcriptions of thousands of documents. 3 the friedberg judeoarabic project provides digital texts for more than 100 important works. 4 plus there are several additional resources for judeoarabic available. 5 we focus on two main tasks: (1) automatic iden tification of the language of morphemes (not just words) in the text, judeoarabic or not (in which case it is virtually always either hebrew or ara maic); and (2) automatic transliteration of judeo arabic into arabic letters (of the arabic parts only). code switching is the act of changing language while speaking or writing, as often done by bilin guals @xcite . in our case, with cross language inflections (e.g. when a hebrew word is inflected following arabic morphological rules) in addition to the rich morphology of arabic, code switching turns out to be nontrivial. we use a lan guage model of both arabic and hebrew, written in hebrew script (we elaborate on the model below), finetuned on the codeswitching task. transliteration is the process of converting a text from one (input) script into another (target script). transliteration differs from translation and is con siderably easier, since semantics play only a small role in decipherment. our primary objective in this study is to de velop tools that enable the automatic conversion of judeoarabic texts into arabic, thus rendering many books and texts readily accessible to arabic readers. it could also facilitate intertextual studies like @xcite , as well as enabling computa tional processing of judeoarabic texts once they are converted into the arabic script, for which nu merous tools already exist. for instance, tirosh becker et al. (2022) could benefit from using arabic partofspeech taggers upon transliterating the texts into arabic. ## related work there have been several prior attempts to transliter ate texts written in judeoarabic into arabic script. for other languages and some of the difficulties involved, see @xcite . modern stud ies focused on transliteration include @xcite for romanized arabic (arabizi) to arabic, (jaf and kayhan, 2021) for ottoman to the modern latin turkish script, and @xcite for romanized bengali (banglish) to ben gali. the first attempt at automated transliteration of judeoarabic texts @xcite employed a method inspired by statistical machine translation, which had been state of the art until deep neural networks took over. this was followed by @xcite who took a similar approach combined with a recurrent neural network (rnn) that was applied to the transliterated arabic text to handle specific errors, notably those associated with tamarbuta, hamza, and shadda. in both of those studies, the transliteration procedure is based on a loglinear model, where the main component is a phrase table that captures the number of occur rences of each character in the training data. they used relatively short parallel texts for training the model, which they evaluated on a small test set of 500 words. in a more recent work @xcite , the au thors trained a model to automatically transliterate judeoarabic texts into arabic using an rnn, com bined with the connectionist temporal classification (ctc) loss to deal with unequal input and output lengths. they increased the size of the training set by generating some parallel texts synthetically. that brought some improvement over the baseline. to the best of our knowledge, no previous work has proposed using a pretrained language model for transliteration, as we introduce here. ## methodology to transliterate a judeoarabic text into arabic, we employ a twostep approach. the first step involves code switching, where we identify non arabic words that are not required for translitera tion in the subsequent step. in the second step, we convert each arabic word from the judeoarabic hebrew script to the arabic script. before delving into the details of each step, we provide a summary of the data sources utilized in both processes. ## conclusions we have established a pipeline that integrates the two models we introduced in this work: code switching detection and transliteration. 11 this pipeline processes judeoarabic text by first iden tifying nonarabic words, which do not require transliteration into arabic, followed by the translit eration of words recognized as arabic. in the original text has apostrophes and punctuation. as explained in section 3. dedicated models per genres. most of our train ing and test work was performed with a specific, literary genre of data. classical authors, like halevi and saadia whose works we used for training, each follow fixed transcription rules and were consis tent in their transliterations from arabic to hebrew script. accordingly, the conversion tool that we cre ated is somewhat crippled when dealing with texts from other genres. inventory lists, prescriptions, newspapers, and other quotidian documents, writ ten by a large variety of people, may be too diverse in style and too varied in spelling. this leads to the question whether there can be a perfect compre hensive tool that will be able to transliterate every judeoarabic text. without answering the ques tion, we suggest that, with prior semiclassification, these texts could be transliterated better. one po tential enhancement can be done by sampling some specific words, which contain "hard" letters, and determining parameters for the map from arabic to hebrew script, consistency in letter mapping, and variety of vocabulary that is used. armed with this information, we could build downstream post processors to provide text corrections, or we may even finetune individual models for different styles and genres. other languages. judeoarabic is not the only language written in a different script than usual for its base language. other jewish languages, like judeopersian, judeoyemenite, ladino, or even yiddish, are similarly written in hebrew charac ters. various languages of countries in the former ussr and its sphere of influence have undergone russification. texts in polish, romanian, serbian, mongolian, and many other languages have been published in the cyrillic alphabet, or an extension thereof. in the internet and socialmedia age, texts in many languages have been shoehorned into us ing the latin alphabet, leading to informal writ ten forms like arabizi and romanized hindi. the ideas we developed should help inform efforts to reexpress such texts as well.