An E nglish- S wahili parallel corpus and its use for neural machine translation in the news domain (2020)

Abstract: This paper describes our approach to create a neural machine translation system to translate between English and Swahili (both directions) in the news domain, as well as the process we followed to crawl the necessary parallel corpora from the Internet. We report the results of a pilot human evaluation performed by the news media organisations participating in the H2020 EU-funded project GoURMET.

## introduction large news media organisations often work in a multilingual space in which they both publish their material in numerous languages and monitor the world's media across video, audio, printed and online sources. as regards content creation, one way in which efficient use is made of journalistic endeavour is the republication of news originally authored in one language into another; by using machine translation, and with the appropriate user interfaces, a journalist is able to take a news story or script, in the case of an audio or video report, and quickly obtain a preliminary translation that will be then manually post-edited to ensure it has the quality required to be presented to the audience. concerning news gathering, expert monitors and journalists have to currently perform a lot of manual work to keep up with a growing amount of broadcast and social media streams of data; it is becoming imperative to automate tasks, such as translation, in order to free monitors and journalists to perform more journalistic tasks that cannot be achieved with technology. in order to cope with these requirements, promoting both the reach of the news published to underserved audiences and the world-wide broadcasting of local information, the h2020 eu-funded project gourmet (global under-resourced media translation), foot_0 aims at improving neural machine translation (nmt) for under-resourced language pairs with special emphasis in the news domain. the two partner media organisations in the gourmet project, the bbc in the uk and deutsche welle (dw) in germany, publish news content in 40 and 30 different languages, respectively, and gather news in over 100 languages. in particular, both media partners gather news in and produce content in swahili. according to wikipedia, swahili has between 2 and 15 million first-language speakers and 90 million second-language speakers. as one of the largest languages in africa and the recognised lingua franca of the east african community, bbc and dw see swahili as an important language in which to make content available. the nmt systems described and evaluated herein can be deployed to support them in this domain specific context. the rest of the paper is organised as follows. next section describes the corpora we used to train our english-swahili nmt systems in both translation directions. section 3 then describes the crawling of the additional corpora we used and made publicly available. section 4 describes the main linguistic contrasts between english and swahili and the challenges they pose for building mt systems between them. section 5 describes the resources, other than corpora, that we used to build our own systems and the technical details of the training of the nmt systems. section 6 discussed the results of automatic evaluation measures, describes a manual evaluation we are conducting and provides preliminary results. the paper ends with some concluding remarks. ## monolingual and bilingual corpora parallel data is the basic resource required to train nmt. additionally, it is common practice to use synthetic parallel corpora obtained by back-translating monolingual data @xcite . this section describes the corpora we used to train the nmt systems described in section 5. tables 1 and 2 describe the parallel and monolingual corpora we used, respectively. as regards parallel corpora, with the exception of gourmet and sawa, all of them were downloaded from the opus website, 2 one of the largest repositories of parallel data on the internet. 3 we used two additional parallel corpora: the sawa corpus @xcite , that was kindly provided by their editors, and the gourmet corpus, that was crawled from the web following the method described in section 3. as regards monolingual data, only three corpora were used: the newscrawl @xcite for english (en) and for swahili (sw), 4 and the gourmet monolingual corpus for sw. the first two corpora were chosen because they belong to the news domain, the same domain of application of our nmt systems. given that the size of the sw monolingual corpus is much smaller than the size of the en monolingual corpus, additional monolingual data in sw was obtained as a by-product of the process of crawling parallel data from the web. ## crawling of additional corpora the amount of data for en-sw is clearly low, even if one compares it to the amount of data available for other under-resourced language pairs, such as english-maltese or english-icelandic. 5 for this reason, a new corpus was crawled from the internet (see the gourmet corpus in crawling. crawling is the first step of the pipeline implemented in bitextor and consists of downloading any document containing text from the websites specified by the user. we used wget foot_3 to crawl documents from 3 751 websites; foot_4 these websites were obtained by leveraging automaticlanguage-identification metadata from the com-moncrawl corpus: foot_5 we consider those websites with at least 5 kb of text in en and in sw. every website was crawled during a period of 12 hours and only documents in en or sw were kept; cld2 10 was used for automatic language identification. plain text was extracted from html/xml and, after this, sentence splitting was applied to every document. from the collection of 3 751 preselected websites, 519 were not available at the time of crawling and, from the remaining 3 232, only 908 ended up containing data in both languages. document alignment. in this step, documents that are likely to contain parallel data are identified. bitextor supports two strategies for document alignment: one based on bilingual lexicons and another based on mt. the last option was not feasible in this work as no high-quality mt system between sw and en was available; therefore, the first one was used. this method combines information from bilingual lexicons, the html structure of the documents, and the url to obtain a confidence score for every pair of documents to be aligned @xcite . the bilingual lexicon used was automatically obtained from the word alignments obtained with mgiza++ @xcite for the following corpora: eubookshop v2, ubuntu and tanzil (see sentence alignment. in this step, aligned documents are segmented and aligned at the sentence level. two sentence-alignment tools are supported by bitextor: hunalign @xcite and bleualign @xcite . we used hunalign because bleualign requires an mt system to be available. the same bilingual dictionary used for document alignment was provided to hunalign in order to improve the accuracy of the alignment. after applying hunalign, 2 051 678 unique segment pairs were obtained. cleaning. bicleaner foot_7 @xcite was used to clean the raw corpora obtained after sentence alignment. cleaning implies removing the noisy sentence pairs that are either incorrectly aligned or not in the expected languages. 12 bicleaner cleaning models require some languagedependent resources: bicleaner was used to score all the sentence pairs in the raw corpus with two different scores: one coming from the regressor, which may be interpreted as the probability that the pair of sentences are parallel, and one coming from the language model, which is the probability that one of the sentences in the pair is malformed. after sampling a small fraction of the corpus, the score thresholds were set to 0.68 and 0.5, respectively. the resulting parallel corpus consisted of 156 061 pairs of segments. in addition to the parallel corpus obtained after cleaning, a large amount of swahili monolingual data was obtained as a by-product of crawling and released as a monolingual corpus. monolingual data cleaning consisted of discarding those sentences not deemed fluent enough to be used for nmt training. sentences were ranked by perplexity computed by a character-based 7-gram language model and only the 6 million sentences with the lowest perplexity were kept. the language model was trained foot_9 on the concatenation of the sw side of the parallel corpora listed in ## contrasts and challenges for mt swahili belongs to a very large african language family, the niger-congo family, and more specifically to the bantu group. swahili is currently written in the latin script, with no diacritics; the apostrophe is used in the seldom-occurring combination ng' which represents the sound of ng in singer (not finger); one common example is ng'ombe, ('cow'). swahili is morphologically and syntactically quite different from english, in spite of the fact that both are subject-verb-object languages. swahili verb morphology is rich and agglutinative, and a large number of morphologically-marked nominal genders participate in nominal and verbal agreement. the challenges to build an mt system for news translation between en and sw are twofold. on the one hand, parallel corpora are rather scarce. on the other hand, a number of challenges stem from the linguistic divergences between the two languages: ## neural machine translation model this section describes the steps followed to build @xmath0. we firstly describe corpora preprocessing and give details about the nmt architecture used and the process followed to choose it. secondly, we present the strategies followed in order to take advantage of monolingual corpora and to integrate linguistic information into the nmt systems. ## evaluation this section reports the scores obtained on the test corpus using automatic evaluation metrics. it then describes the manual evaluation we are conducting at the time of writing these lines and provides preliminary results. ## concluding remarks we have described the development and evaluation of an nmt system to translate in the news domain between english and swahili in both directions. we have also described the crawling of a new parallel corpus from the internet which we have made publicly available. we performed an automatic evaluation of both systems. according to it, the @xmath1, whereas the @xmath2. in addition, the @xmath3, and the @xmath4.65 ± 3.97 70.80 ± 4.30 evaluator 2 47.30 ± 6.20 52.94 ± 5.97 evaluator 3 48.42 ± 3.28 60.20 ± 3.75 evaluator 4 54.40 ± 3.92 56.53 ± 4.02 as future work, and in view of the scarcity of bilingual resources available, we plan to try approaches based on monolingual corpora @xcite . we also plan to study if a correct segmentation of verbs, which are very rich and complex (see