Rule-based machine translation from K azakh to T urkish (2018)

Abstract: This paper presents a shallow-transfer machine translation (MT) system for translating from Kazakh to Turkish. Background on the differences between the languages is presented, followed by how the system was designed to handle some of these differences. The system is based on the Apertium free/open-source machine translation platform. The structure of the system and how it works is described, along with an evaluation against two competing systems. Linguistic components were developed, including a Kazakh-Turkish bilingual dictionary, Constraint Grammar disambiguation rules, lexical selection rules, and structural transfer rules. With many known issues yet to be addressed, our RBMT system has reached performance comparable to publicly-available corpus-based MT systems between the languages.

## introduction in this paper we present a prototype shallowtransfer rule-based machine translation system using the apertium free/open-source machine translation platform @xcite for translating from kazakh to turkish. one of the most common criticisms towards rule-based machine translation (rbmt) regards the amount of work necessary to build a system for a new language pair @xcite . in fact, in a traditional scenario, linguists with expertise in the source and target language need to manually build all the dictionary entries and transfer rules. conversely, in a corpus-based/statistical mt approach @xcite , no such effort is required as the system can be automatically built from parallel corpora providing they exist. if parallel corpora do not exist, then we see two options that remain. the first is to create a new parallel corpus, either by translating millions of words from scratch (requiring effort from translators),¹ or by finding parallel text online and processing it (requiring effort from programmers). the second option is to build a rule-based machine translation system (requiring effort from linguists). the most labour-intensive of these approaches is to translate the data from scratch, although this might be practical in certain situations. building a rule-based machine translation system and finding and processing parallel texts from the internet are, given equal available expertise, around equally time consuming. as large, freely available parallel corpora are not known to exist between kazakh and turkish and we were interested in structural differences between these two languagesand producing new linguistic resources, we chose to build a rule-based mt system. kazakh and turkish are different enough that native speakers are not able to make sense of the other language, but also share similar enough structure that an rbmt system is feasible with some level of linguistic knowledge. this paper demonstrates that, with many known issues yet to be addressed, our rbmt system has already reached performance comparable to publically available smt systems between the languages. this has been accomplished solely with open source tools and some level of linguistic knowledge about ¹the millions of words number is taken from @xcite who compare neural mt against phrase-based smt for english-spanish for 0.4 million to 385.7 million words. even for these morphologically-poor languages, even with one million words the performance is poor. the two languages, and without large parallel corpora or machine learning algorithms (although some of the components in our opinion could be significantly improved by using them; see section 6 for more details). the paper will be laid out as follows: section 2 gives a short review of some previous work in the area of turkic-turkic language machine translation and an overview of other publically available kazakh-turkish machine translators; section 3 introduces kazakh and turkish and compares their grammar; section 4 describes the system and the tools used to construct it; section 5 gives a preliminary evaluation of the system; section 6 describes our aims for future work; and finally section 7 contains some concluding remarks. ## previous work within the apertium project, there is ongoing work on building mt systems (and thus underlying components such as morphological transducers) for translating between two turkic languages, between a turkic language and russian or between a turkic language and english. among released mt systems there are: kazakh-tatar @xcite , tatar-bashkir @xcite , crimean tatar-turkish and english-kazakh @xcite mt systems. several other mt systems have been reported that translate between turkish and other turkic languages, including turkish-crimean tatar @xcite , turkish-azerbaijani @xcite , turkish-tatar @xcite , and turkish-turkmen @xcite mt systems. as for the systems for translating to/from kazakh (besides the ones already mentioned above), there is a bidirectional kazakh-english machine translation system @xcite which uses a link grammar and statistical approach. none of these mt systems to our knowledge have been released to a public audience. @xcite propose a segmentation system for inflectional affixes of kazakh. @xcite , @xcite and @xcite present work on kazakh morphological analysis. both kazakh and turkish are among the languages supported by google translate² and yandex translate³ tools. ## the languages kazakh is classified as a member of the northwestern (or kypchak) branch of the turkic language family. it is primarily spoken in kazakhstan, where it is the national language, sharing official status with russian. large communities of native speakers also exist in china, neighbouring central eurasian republics, and mongolia. the total number of speakers is at least 10 million people @xcite . the present-day kazakh cyrillic alphabet consists of 42 letters, 33 of which are letters found in the russian alphabet. there are controversial plans to transition to a latin alphabet by 2025. turkish is classified as a member of the southwestern (or oghuz) branch of turkic language family. with over 70 million l1 speakers @xcite , it is the turkic language spoken by the most people. the turkish latin alphabet contains 29 letters. kazakh and turkish exhibit agglutinative morphology, meaning that word forms may consist of a root and a series of affixes. an mt system between kazakh and turkish is potentially of great use to the language communities. automatic mt can save time and money over going through e.g. russian or english (and the system is much easier to develop). we continue with a brief overview of some differences in phonology, orthography, morphology and syntax of the languages. a complete and detailed comparison is out of scope of this work. ## system our machine translation system is based on the apertium mt platform @xcite .⁴ the platform was originally aimed at the romance languages of the iberian peninsula, but has also been adapted for other, more distantly related, language pairs. the whole platform, both programs and data, are licensed under the free software foundation's general public licence⁵ (gpl) and all the software and data for the completed supported language pairs (and the other pairs being worked on) is available for download from the project website. ## evaluation the system has been evaluated in two ways. the first is its coverage. the second is translation qualitythe error rate of two pieces of text produced by the system when comparing with postedited versions of them. ## future work we intend to continue the development of the system to improve the quality of the translations. there are a number of areas where we believe that more work would yield better results, among which are the following: ## conclusion to our knowledge we have presented the first free/open-source mt system between kazakh and turkish. the performance is similar to other translators created using the same technology, and in srl for low resource languages isn't needed for semantic smt meriem beloucif and dekai wu human language technology center hong kong university of science and technology clear water bay, hong kong mbeloucif|dekai@cs.ust.hk