Transliteration for Low-Resource Code-Switching Texts: Building an Automatic C yrillic-to- L atin Converter for T atar (2021)

Abstract: We introduce a Cyrillic-to-Latin transliterator for the Tatar language based on subword-level language identification. The transliteration is a challenging task due to the following two reasons. First, because modern Tatar texts often contain intra-word code-switching to Russian, a different transliteration set of rules needs to be applied to each morpheme depending on the language, which necessitates morpheme-level language identification. Second, the fact that Tatar is a low-resource language, with most of the texts in Cyrillic, makes it difficult to prepare a sufficient dataset. Given this situation, we proposed a transliteration method based on subword-level language identification. We trained a language classifier with monolingual Tatar and Russian texts, and applied different transliteration rules in accord with the identified language. The results demonstrate that our proposed method outscores other Tatar transliteration tools, and imply that it correctly transcribes Russian loanwords to some extent.

## introduction modern tatar has two orthographies: cyrillic and latin. the two alphabets are mostly mutually compatible when an input string consists of only tatarorigin words. effectively, however, modern tatar has a massive amount of russian loanwords, and, in colloquial texts, even a whole phrase may be switched to russian. this linguistic phenomenon is known as code-switching or code-mixing. a difficulty of transliteration from cyrillic to latin lies in the following two facts. first, a different set of transliteration rules has to be applied to tatar and russian words. this requires a language detection for each token, or worse, for each morpheme, which would additionally require morphological analysis. it is expected that a full implementation of such a system will produce heavy processes. second, because modern tatar frequently * equal contribution mixes russian words, it is not easy to obtain a pure tatar dataset for developing a language detector. existing methods are based on either tatar monolingual rules or a huge bundle of ad-hoc rules aimed to cover russian-origin words @xcite . the experimental results in section 6 demonstrate that the former monolingual rulebased transliterators show low accuracy because russian words are not supported. the latter extensively rule-based transliterator has better accuracy, but still misses a certain amount of words. this implies that a strictly rule-based method requires an ever-lasting process of adding rules ad hoc for exceptional words to further improve the accuracy. this is obviously unrealistic and inefficient. in this study, in contrast, we pursue a simple yet high-accuracy automatic transliteration system from cyrillic tatar to latin tatar. we prepare two sets of simple rule-based monolingual transliteration rules for tatar and russian each. in addition, we train a classifier that automatically determines tatar or russian for each subword. each token is then transliterated to latin tatar by applying the rules of the detected language to its subwords. the results demonstrate that our proposed method achieves higher accuracy than the previous tools. also, our proposed method demonstrates higher accuracy than the transliterator with only tatar-based rules, indicating that the method can correctly predict and transcribe russian words to some extent. through the long history of language contact with russian, modern tatar contains a large amount of russian loanwords. most of the speakers are bilingual along with russian, and younger generations living in urbanized cities tend to have better competence in russian than in tatar. this bilingualism leads to frequent code-switching (cs) in text and speech, particularly of colloquial genre @xcite . ## method we transliterate cyrillic tatar to latin tatar word by word, as each word does not affect other words in tatar transliteration 3 . taking into account the fact that tatar has intraword cs, we created a classifier that detects a language (tatar or russian) for each subword. to implement the language classifier, we prepared two monolingual corpora of tatar and russian. given the lack of pure tatar texts without cs in modern texts foot_3 , we employed tatar translation of qur'an foot_4 (19,691 words with duplication) translated in 1912 that contains no russian loanwords in order to avoid noise to train the classifier. its russian counterpart foot_5 (21,256 words with duplication) was translated by the ministry of awqaf, egypt. the training process is as follows. first, the words collected from the dataset were automatically divided into subwords by the byte pair encoding algorithm @xcite . banerjee and bhattacharyya (2018) reports that, unlike morfessor, bpe can flexibly solve the oov problem because some subwords are character-level segments. in our case, due to the meagerness of the monolingual training data, we employed bpe to avoid the oov problem. then, assuming that longer subwords are less ambiguous with respect to labels to be assigned, we took the longest match to make it easier to distinguish between russian and tatar; for this reason, the subword merge operation was repeated until no further merge was possible. the obtained subwords are then represented in subword embeddings using fasttext foot_6 @xcite . the classification model is the supervised classifier provided by fasttext, with the following hyperparameters that are known to per-form high accuracy foot_7 : the number of dimensions is 16, the minimum and maximum character n-gram sizes are 2 and 4. hierarchical softmax is used as the loss function. together with this representation, the subword embeddings are annotated with a language label @xcite . it is worth noting that, unlike @xcite , we do not employ deep learning approach. while the task in @xcite is multi-labeling, our language identification task is a binary classification with a low-resource training dataset; for this reason, deep learning is too superfluous and heavy for achieving our task. to evaluate the performance of our model, we apply bpe also to the test data in the same manner, and predict a language label for each subword. adjacent subwords are, when possible, combined to form a longer subword with a single language label for the sake of better accuracy; that is, for example, when two consecutive subwords are both labeled as tt, then they are combined into one subword labeled as tt. finally, each subword is converted to the latin alphabet with the transliteration rules of the predicted language, and combined into a word as an output. ## experimental setup for the evaluation of the performance, we prepared 700 sentences (shuffled; 8,466 words with duplication, 5,261 without duplication) from the corpus of written tatar @xcite and their latin counterpart as the gold data that was manually transcribed by us and verified by a native speaker. also, we annotated the cyrillic text data so that cs russian morphemes are tagged. according to this, the text data contains 1,009 words (with duplication) with a russian morpheme, and 598 words (with duplication) with intra-word cs. for evaluation metrics, we calculated bleu and longest common sequence (lcs) f-measure for each letter in a word as well as word accuracy (acc) and character error rate (cer). the calculation of lcs f-measure and acc is based on @xcite . to compare the performances of tatar monolingual transliteration and of tatar-russian bilingual transliteration (i.e., our proposed method; "tt-ru hybrid" henceforth), we also evaluated the data with solely tatar monolingual transliteration rules ("tt-based" henceforth). for the comparison with the existing transliteration tools, we computed the scores of speak.tatar, ttt, and aylandirow 9 . ## results as shown in note that the perfection is not necessarily the ultimate goal of this transliteration. as described in detail in section 7, there may be several ways of spelling in actual language use, particularly of the spelling of proper nouns. this variance in spelling foreign words is common among languages. ## analysis the results illustrated that the bilingual transliterators generally have higher accuracy than the 9 although we mentioned fintat in section 2, we excluded it from the evaluation because it is based on a spelling system slightly different from 2013latin. monolingual transliterators. however, it needs to be examined that tt-based also gained higher scores than the other monolingual ones, even though it does not support russian loanwords. this is due to the fact that their transliteration rule sometimes does not follow the rules of 2013latin. for example, tәnky t "criticism" is transliterated as tänqit' (with a hamza at the end) by the ttt, where it should be transliterated as tänqit according to 2013latin. in fact, due to the de-facto absence of any institution that regulates the latin orthography, different varieties in spelling styles can be observed on the internet. for this reason, the seemingly high scores in tt-based are merely a product of the orthographical consistency. keeping this in mind, the improvement in the scores between tt-based and tt-ru hybrid indicates that the bilingual transliteration method designed to be adaptive to russian loanwords successfully predicted the language and transcribed them. examples of successful transliteration in our ttru hybrid are zakon and sovet shown in the upper example of however, tt-ru hybrid wrongly transliterated 116 words that were correct in tt-based; for example, in the lower example of ## conclusion in this paper, we proposed a new transliteration system that converts cyrillic tatar to latin tatar. taking into account the facts that different transliteration rules are applied to russian loanwords and that intra-word cs is frequently observed, our proposed method involved language identification for each subword. even though tatar resources available at hand for training were limited, the results were significantly better than existing transliteration tools. the simple architecture of language detection employed in this approach is languageagnostic does not need detailed analyses such as syntactic parsing and pos tagging, our method is applicable to other low-resource languages that have intra-word cs.