PROMT Systems for WMT 23 Shared General Translation Task (2023)

Abstract: This paper describes the PROMT submissions for the WMT23 Shared General Translation Task. This year we participated in two directions of the Shared Translation Task: English to Russian and Russian to English. Our models are trained with the MarianNMT toolkit using the transformer-big configuration. We use BPE for text encoding, both models are unconstrained. We achieve competitive results according to automatic metrics in both directions.

## introduction the wmt shared general translation task is an annual event where different companies and researchers build and test their systems on the test sets provided by the organizers. this year we decided to participate in two directions: english to russian and russian to english. we use the standard transformer-big configuration for our models. the english-russian model is basically the same as last year, whereas the russian-english model is a new one built for wmt23. the rest of the paper is organized as follows: in section 2 we describe in detail the systems we submitted to the shared task. in section 3 we present and discuss the results. we conclude the paper in section 4 with discussion for possible future work. ## systems overview all of our wmt22 submissions are @xcite transformer-big @xcite systems. we use the opennmt toolkit @xcite version of byte pair encoding (bpe) @xcite for subword segmentation. our bpe models are case-insensitive, we use special tokens in the source and target sides to process case (see @xcite for details). all of the systems are unconstrained, i.e. we use all data provided by the wmt organizers, all publicly available data and some private data crawled from different web-sources. we also augment our training data with two types of synthetic data: 1) back-translations @xcite and 2) synthetic data with placeholders as described in @xcite . the back-translations are obtained using the previous versions of our nmt models which are baseline transformers trained with less data (and without some up-to-date data like the news 2021 corpora from statmt.org). we also tag all our synthetic data with special tokens at the beginning of the source sentences as described in @xcite . all models are trained with guided alignment which is used at translation time to handle named entities and document formatting. we obtain alignments using the fast-align @xcite tool. the data statistics for the russian-english language pair are presented in the details regarding different directions can be found in the next section. ## results and discussion the results are presented in as we can see, we outperform our baselines (i.e. previous versions of the models). the gains we observe, however, are not that large. however, other test sets, such as the tico-19 evaluation set 3 @xcite , show more substantial improvements. the bleu score on that test set has grown from 33.8 to 35 points. poor performance on the generaltest2023 set can be due to the problems that our submitted models have with translation of colloquial content. this can be explained by our data preparation scheme. as we have already mentioned above, we want our models to translate formal text better and thus 'sacrifice' colloquial data. the examples of such mistranslations are presented in we made a thorough investigation into the generaltest2023 sets. thus, we found out that there are four major topics for the russian-english test set: 1) movie reviews; 2) news of any the forecast of gdp components for 2019 slightly changed, which is primarily due to the release of actual data for the second quarter of 2019. ## conclusions and future work in this paper we presented our submissions for the wmt23 shared general translation task. we show good results in both directions we participate. we clearly outperform our baselines in both directions. a detailed analysis of the translations shows us that we lose quality in translation of colloquial speech. we have already started to work in this direction. we have synthesized data where, e.g., 'please' is substituted with 'plz' and so on. we plan to train our model on this synthetic data so that it could deal with such colloquial examples.