CUNI at WMT 23 General Translation Task: MT and a Genetic Algorithm (2023)

Abstract: This paper presents the contributions of Charles University teams to the WMT23 General translation task (English to Czech and Czech to Ukrainian translation directions). Our main submission, CUNI-GA, is a result of applying a novel n-best list reranking and modification method on translation candidates produced by the two other submitted systems, CUNI-Transformer and CUNI-DocTransformer (document-level translation only used for the en \rightarrow cs direction). Our method uses a genetic algorithm and MBR decoding to search for optimal translation under a given metric (in our case, a weighted combination of ChrF, BLEU, COMET22-DA, and COMET22-QE-DA). Our submissions are first in the constrained track and show competitive performance against top-tier unconstrained systems across various automatic metrics.

## introduction our submission for this year's wmt general translation task @xcite is based on the previous submissions of our team @xcite and mbr decoding in combination with genetic algorithm (ga). we describe the method in separate work @xcite . the main goal of our submission is to find out whether our approach improves the translation quality perceived by humans. for this reason, we submitted both the base system translations and the mutated and reranked (i.e. ga-processed) translations for the human evaluation. as all the parts of the approach are described in detail in the mentioned papers (as well as all the related work), we will restrict ourselves to providing a short overview of the main points in section 2. in section 3, we describe the datasets, tools and parameters used to obtain results presented in section 4. finally, we draw conclusions from the results. ## methods our submissions make use of two features that are not typical for current mt systems: documentlevel context and translation refinement through a genetic algorithm. ## system description our models are based on submissions of our team from previous years @xcite . we resubmit those (cuni-transformer and cuni-doctransformer submissions) and we also submit an additional translation: the outputs of these models combined, mutated and rescored by the ga described in section 2.2 (cuni-ga submission). ## results this section presents automatic metric scores on validation sets and the official test set. ## future work our setting allows many straightforward modifications to potentially improve the results of our method. first of all, mbr decoding works well on a large, diverse set of initial candidates, obtained for example by sampling. in our experiments, we only use short n-best lists produced by beam search. an additional benefit stemming from the diversity of the initial candidates is a more dive diverse set of possible tokens for replacement mutations. second, we did not run any search for the parameters of the ga process (crossover and mutation rates, number of generations, population size, selection method), due to the large computational costs of this approach. we believe a set of better parameters could be found easily by, for example, a grid search. finally, the metrics used for the fit-ness function are combined by a simple weighted sum. multi-criterion genetic algorithms can be explored for a better approach to combine multiple evaluation scores for the translations. also, reranking and modifying the translations on a sentence level can introduce inconsistencies previously mitigated by using document-level mt, losing the advantages of document-level processing. @xcite show that using sentencelevel metrics for whole document-level segments might be a viable option for avoiding this issue. ## conclusion we confirm that using mbr decoding in combination with a genetic algorithm can improve scores in selected evaluation metrics, while creating original novel translations. we show that our systems are competitive in both submitted language pairs, winning among constrained systems based on automated evaluation metrics.