Machine Translation for Nko: Tools, Corpora, and Baseline Results (2023)

Abstract: Currently, there is no usable machine translation system for Nko, a language spoken by tens of millions of people across multiple West African countries, which holds significant cultural and educational value. To address this issue, we present a set of tools, resources, and baseline results aimed towards the development of usable machine translation systems for Nko and other languages that do not currently have sufficiently large parallel text corpora available. (1) Fria \parallel el: A novel collaborative parallel text curation software that incorporates quality control through copyedit-based workflows. (2) Expansion of the FLoRes-200 and NLLB-Seed corpora with 2,009 and 6,193 high-quality Nko translations in parallel with 204 and 40 other languages. (3) nicolingua-0005: A collection of trilingual and bilingual corpora with 130,850 parallel segments and monolingual corpora containing over 3 million Nko words. (4) Baseline bilingual and multilingual neural machine translation results with the best model scoring 30.83 English-Nko chrF++ on FLoRes-devtest.

## nko corpora for machine translation this section discusses the extension of flores-200 and nllb-seed to nko, which included the multilingual alignment of nllb-seed, and the use of fria∥el to translate those corpora to nko. this section also introduces nicolingua-0005, a collection of monolingual corpora and bi-and trilingual parallel corpora donated by nko community members. ## baseline machine translation experiments this section describes transformer @xcite based encoder-decoder neural machine translation models built using the fairseq toolkit @xcite . both bilingual and multilingual translation models are explored. at present, results remain quite modest, with the best model achieving a 30.83 @xmath0. eight models were trained: the bilingual unidirectional models 200.11 and 200.16, the multilingual model 201.16, and its variant that is trained to also autoencode nko segments 202.16, and models 206.19, 207.19, 208.19 and 209.19, which explore three different ways of specifying language tokens. ## conclusion this work presented fria∥el, a collaborative parallel text curation system with copyediting-based quality workflows. fria∥el enabled the extension of existing multilingual corpora, flores-200 and nllb-seed with high-quality nko translations. those, and a new corpus we introduced, nicolingua-0005, served to build baseline bilingual and multilingual nmt systems for nko, with the best model achieving the accuracy of 30.84 @xmath1. we have released fria∥el to facilitate the development and extension of multilingual parallel corpora to more languages. we have also released resources and tools to enable the reproducibility of our results, and further progress towards usable mt systems for nko. were there any disagreements regarding language standards? b. how were those disagreements resolved? b fria∥el software engineering diagrams on the next three pages appear: • workflow and task management sequence diagrams • workflow and task state-transition diagrams • logical data model a translation workflow entity in the active state is created for each dataset entry. the workflow manager creates related unassigned tasks as needed, per the rules of the workflow. the task manager assigns tasks to users as appropriate. uncompleted tasks are moved back to the unassigned status when not completed within the lease period. the workflow manager moves workflows to the completed status when all related tasks are completed and there is no need to create additional tasks.