Towards Machine Translation for the K urdish Language (2020)

Abstract: Machine translation is the task of translating texts from one language to another using computers. It has been one of the major tasks in natural language processing and computational linguistics and has been motivating to facilitate human communication. Kurdish, an Indo-European language, has received little attention in this realm due to the language being less-resourced. Therefore, in this paper, we are addressing the main issues in creating a machine translation system for the Kurdish language, with a focus on the Sorani dialect. We describe the available scarce parallel data suitable for training a neural machine translation model for Sorani Kurdish-English translation. We also discuss some of the major challenges in Kurdish language translation and demonstrate how fundamental text processing tasks, such as tokenization, can improve translation performance.

## introduction since the early advances in the field of natural language processing (nlp), one major task that motivated researchers to use machines for natural languages has been machine translation (mt). natural language is notoriously complex and irregular with a great variability in the type of morphemes and their meanings as well as their syntactic and semantic dependencies in the context. therefore, manual translation has proved to be inviable for such a task. over half a century, mt techniques have been getting more efficient and less languagedependent. dictionary-based and rule-based approaches, which are deemed traditional approaches in the field, carry out the task of translation using manually-defined rules and resources @xcite . later, statistical machine translation further paved the way for dimin-ishing the role of a linguist in the loop and decrease language dependency @xcite . with the current advances in machine learning, particularly recurrent neural networks, neural machine translation (nmt) has been successfully used with state-of-the-art results for many languages @xcite . that being said, mt is not similarly challenging for all languages due to language-specific features. for instance, the translation of a morphologicallyrich language, such as czech or german, represents further challenges in alignment in comparison to a less morphologically-rich language like english @xcite . moreover, mt systems require reasonably large aligned datasets and performant language processing tools such as tokenizer, stemmer and morphological analyzer @xcite . such resources and tools are not always available, particularly for less-resourced languages. languages are classified as less-resourced where general-purpose grammars and raw internet-based corpora are the only existing resources, lacking manually crafted linguistic resources and large monolingual or parallel corpora. except the richer-resourced languages, the majority of the human languages are considered less-resourced @xcite . this is also the current status of the kurdish language, an indo-european language spoken by 20-30 million speakers @xcite . in this paper, we discuss the major challenges in mt for sorani kurdish, including the lack of basic language processing tools such a tokenization. to further highlight the challenges, we report the performance of two nmt models in various experimental setups based on the tokenization methods and resources. despite the scarcity of parallel corpora for kurdish, there are a few parallel resources which can be used for the task, partic-ularly the tanzil corpus @xcite which contains 92,354 parallel sentences, the ted corpus @xcite and kurdnet-the kurdish wordnet @xcite . ## related work there have been very few previous studies that address the kurdish language in the mt realm. one of the outstanding projects in creating a rule-based machine translation system for kurmanji and sorani is the apertium project @xcite . in this open-source project, various tools and resources are developed for the kurdish language, including bilingual and morphological dictionaries, structural transfer rules and grammars. another initial attempt to create a machine translation system for kurdish is inkurdish foot_0 which uses dictionary-based methods for translation. @xcite report that this system fails to translate based on the length of the input sentences and the degree of idiomaticity. as the two major existing machine translation tools for sorani kurdish, kaka-khan (2018) states that although the rule-based method of apertium performs significantly better, limitations of the lexicon and transfer rules lead to incorrect translations and therefore, generalization across domains remains a difficult task. kurdish language translation has been also of interest to many humanitarian organizations due to the refugee crisis in the current years, translators without borders (twb) foot_1 and tarjimly foot_2 , to mention but a few @xcite . some of these organizations use mobile applications to enable refugees to get in touch with translators for their translation needs such as appointments with authorities. in the case of twb, a machine translation system is created based on the apertium project. however, no experiment regarding the performance of the tool is reported. more recently, there has been an increasing number of resources created for the kurdish language, such as dictionaries @xcite , domain-specific corpora @xcite , folkloric corpus @xcite and kurdnet-the kurdish wordnet @xcite . however, parallel corpora are more scarcely available. bianet @xcite is a parallel news corpus containing 6,486 english-kurmanji kurdish and 7,390 turkish-kurmanji kurdish sentences. opus foot_3 also contains parallel translations in kurmanji and sorani for the gnome and ubuntu localization files foot_4 @xcite . more importantly, the tanzil corpus provides translation of qoranic verses in sorani kurdish. in 2016, the translation service of google, i.e. google translate foot_5 , added kurmanji kurdish to its list of languages foot_6 . motivated to explore this field, in this paper, we focus on creating an nmt system for the sorani dialect of the kurdish language. ## sorani kurdish sorani kurdish is one of main dialects of kurdish along with kurmanji kurdish and southern kurdish @xcite . this dialect is mainly spoken by the kurdish populations in the kurdish regions of iran and iraq. unlike kurmanji dialect for which a latin-based script is used, sorani kurdish is mostly written in the arabic-based script of kurdish with no universally accepted orthography upon which scholars agree and is used by the public @xcite . kurdish has a subject-object-verb word order with a system of tense-aspect-modality and person marking @xcite . moreover, sorani kurdish is a split-ergative language where transitive verbs in the past tenses are marked with an agentive case different from the nominative case @xcite . the agentive case in kurmanji kurdish is the oblique case while sorani kurdish only uses different pronominal enclitics for ergative-absolutive alignment @xcite . for further clarification, a few examples in sorani kurdish are provided below. in example 1 in the past tense, the pronominal @xmath0. in contrast, in example 3, the same patient marker -in (in green) is used with a present tense as the subject marker with a nominative-accusative alignment and the pronominal @xmath1. on the placement of agent markers, unlike patient markers, i.e. -in, which always appear immediately after the verb, the agentive markers follow an erratic pattern where they tend to appear immediately after the first prefix in verb forms, e.g. example 2, or they attach to the leftmost morpheme in verbal phrases, if present, as in example 1 @xcite . moreover, sorani kurdish morphology is known to be complex, particularly due to the variety of affixes, clitics and the pattern in which they appear within the word and the phrase @xcite . moreover, the stringing property of the arabicbased script along with the lack of a unified orthography creates further complexity in a way that many word forms are concatenated into a single one. this is particularly the case of copula when emerges as an enclitic, as shown in example 4. this yields further complications in the alignment of kurdish and other languages. figure 1 illustrates the alignment of the english sentence "this is a woman from canada" with its sorani kurdish translation eme jinêke le kanadawe. the alignment is carried out at various levels, namely word-level, token-level and morpheme-level. this demonstrates how the granularity of the alignment varies depending on the level, ranging from a coarse-grained alignment at word-level where "is a woman" is aligned with only one word jinêke, to a more fine-grained alignment at token-level. ultimately, at morphemelevel, circumposition le ... ewe and indefinite article -êk are alignable with their english equivalents, 'from' and 'a', respectively. to facilitate the reading, this example is provided in the latin-based script of kurdish. ## data description given the scarcity of parallel corpora for kurdish, we used all publicly-available parallel corpora, despite their limited topic coverage and size. in this section, we describe the data used for this study. ## results and analysis to analyze the performance of the models and evaluate the impact of tokenization on the translation quality, we create various datasets based on the tokenization techniques, namely bpe, unigram, wordpiece and wordpunct. as the tanzil corpus is remarkably larger, we create two sets of testing scenarios where initially 10% of each dataset is set aside as the first testing set (test 2 in ## conclusion and future work in this paper, we present our efforts to develop an nmt system for the sorani dialect of kurdish. we describe how due to scarcity of parallel corpora, we used translations of religious texts as the material for training a machine translation system. moreover, we created basic language processing tools, such as tokenization, by figure 2 : the performance of model 1 in translating kurdish to english with respect to certain length-limited sentences in the kurdish ted corpus in terms of various bleu scores using unsupervised techniques, namely word-piece, byte-pair-encoding and unigram language model. we train two nmt models using different hyper-parameters and evaluate the models based on the datasets and the tokenization techniques. although the imbalanced data makes the models over-fit, our qualitative analysis indicates that some syntactic and lexical properties of kurdish are correctly learnt in the translation outputs. there are two major limitations in the current project which could not be addressed due to our focus being on the preprocessing steps: a baseline system and further experiments with respect to hyper-parameters. given the current state of lack of parallel corpora, we were not able to extend the study the other dialects. we strongly believe that this should be a motivation to create more resources for the kurdish language 13 . moreover, as an initial work of its kind for the machine translation of kurdish, we dealt with many basic language processing tasks which were not properly addressed. developing such tools should be a priority in the field of kurdish language processing. regarding future work, we would like to suggest morpheme-based translation @xcite . as kurdish is a morphologically-rich language, it might be beneficial to go beyond tokens 13 shortly after this project, @xcite present a parallel corpus created based on multilingual news websites content. and carry out the alignment task at morphemelevel. we also believe that lexicons can be efficiently incorporated for compensating the scarcity of resources for kurdish @xcite . we also propose the usage of other dialects of kurdish, such as kurmanji, and closely-related languages, like persian, for improving the performance of future machine translation models @xcite . another recent promising direction for a low-resource setup like kurdish is monolingual sequence-to-sequence pretraining techniques, such as masked sequence to sequence pre-training (mass) @xcite or mbart @xcite .