Improved N orwegian B okmål Translations for FLORES (2025)

Abstract: FLORES+ is a collection of parallel datasets obtained by translation from originally English source texts. FLORES+ contains Norwegian translations for the two official written variants of Norwegian: Norwegian Bokmål and Norwegian Nynorsk. However, the earliest Bokmål version contained non-native-like mistakes, and even after a later revision, the dataset contained grammatical and lexical errors. This paper aims at correcting unambiguous mistakes, and thus creating a new version of the Bokmål dataset. At the same time, we provide a translation into Radical Bokmål, a sub-variety of Norwegian which is closer to Nynorsk in some aspects, while still being within the official norms for Bokmål. We discuss existing errors and differences in the various translations and the corrections that we provide.

## introduction this paper describes our submission to the wmt 25 open language data shared task, where participants were asked to contribute to open dataset collections such as flores+, the mt seed dataset or other parallel datasets. we have chosen to focus on the norwegian bokmål part of the flores+ dataset, as the authors notice non-fluencies in the dataset in 2024, and notified the original authors. these issues were attempted resolved in a process that lead to additional errors, which are the ones that form the basis of this paper. in addition to correcting these translations, we translate the resulting norwegian bokmål dataset into a version of a specific variety of written norwegian called radical bokmål. having these two normed varieties can be beneficial for experiments where variation in norwegian spelling norms is important. we summarize some of the encountered errors in the newest bokmål translations, and show some results on several machine translations baselines for the new and existing norwegian versions. ## the norwegian language and its norwegian is one of the official languages of norway, along with sámi languages and norwegian sign language. it is a north-germanic language historically descendant of western norse, but following large saxon and east norse influences, is largely mutually intelligible with its neighbors swedish and danish, and more different from icelandic and faroese. however, following centuries of having danish as norways national language, nationalist movements in the late 19 th century lead to the establishment of two written standards: landsmål (today nynorsk), which was based on dialects "untainted" by danish, and rigsmål (riksmål, today bokmål), which was norwegianized danish. nynorsk historically aimed at preserving norwegian-specific features, which means that saxon and danish influences are less pronounced in nynorsk than in bokmål. ## the flores dataset the flores dataset is an evaluation dataset for multilingual machine translation, consisting of a dev and a devtest part with about 1000 sentences each. the dataset is multiparallel and englishcentric: the original sentences are in english, and all other language variants were produced by translation. several versions were made available over time, reflecting efforts to increase language coverage and address quality issues. flores101 was the first version of flores, covering 101 languages, including norwegian (goyal et al., 2021). while the authors claimed that the sentences were "[...] translated in 101 languages by professional translators through a carefully controlled process", we observed severe quality problems with the norwegian bokmål translations. see further discussions in 3.1. flores200 was a continuation from both flo-res101 and guzmán et al. (2019), with an increased coverage of 200 languages (nllb team, 2022). the norwegian sentences appear to be unchanged between flores101 and flores200. the flores200 translations were used, among others, in the belebele (bandarkar et al., 2024) benchmark. quality problems in the former therefore directly affect results reported on the latter dataset. we have used the bokmål sentences from flores200 both as an aid in correcting the translations, and as a point of comparison agains the new dataset as a whole. these sentences initially struck the authors as unnatural, with exampled reported such as translating iron (the metal) as strykejern (eng. 'clothes iron'), and (judicial) court as hoff (eng. royal court). flores+ the responsibility for the flores datasets was eventually moved to the open language dataset initiative. @xcite as a result, the updated versions are referred to by flores+ and published on huggingface. @xcite in january 2024, the authors of this paper reached out to the original flores101 authors to express concern over the quality of the norwegian bokmål dataset, based on flores200. following this, the dataset was updated, as indicated by a changelog note from november 11 th 2024 foot_4 . this note informs that the norwegian version has been updated after quality assessment, but with no further information. going through these changes, we see, however, that not all errors were corrected, and that new ones were introduced. correcting these errors is the main focus of this paper. ## translation correction we introduce our methodology and discuss some of the encountered errors. see appendix a for selected example sentences in all languages involved in this process. ## conclusion even after its initial correction, several obvious and non-native-like mistakes remained in the flores+ bokmål dataset. our attempt has corrected the most obvious mistakes, making sure that there are at least no grammatical or lexical mistakes in the dataset, without introducing excessive changes to the work done by the professional translators. we hope that these corrections make results from these datasets more reliable. on a more personal note, this is not the first time the authors experience problems with context and understanding coming in the way when translating datasets that are supposed to be the basis of massive-parallel datasets. we urge the creators of such original datasets to perhaps add clarifying remarks where there might be misunderstandings. following the observation that close to 70% of all sentences in the corrected dataset contained at least one lexical or grammatical error, we recommend earlier users of the dataset to reevaluate results used on this dataset. there is also some reason to doubt the claims that all these translations were indeed done by professional translators, and we hope that future dataset creators will use the native professional communities to gain valuable feedback in these situations.