KYB General Machine Translation Systems for WMT 23 (2023)

Abstract: This paper describes our approach to constructing a neural machine translation system for the WMT 2023 general machine translation shared task. Our model is based on the Transformer architecture’s base settings. We optimize system performance through various strategies. Enhancing our model’s capabilities involves fine-tuning the pretrained model with an extended dataset. To further elevate translation quality, specialized pre- and post-processing techniques are deployed. Our central focus is on efficient model training, aiming for exceptional accuracy through the synergy of a compact model and curated data. We also performed ensembling augmented by N-best ranking, for both directions of English to Japanese and Japanese to English translation.

## introduction in the context of the wmt 2023 general machine translation shared task for japanese to/from english, we tackle the inherent challenges posed by the diverse linguistic structures of these languages. the transformative impact of the transformer model on neural machine translation is undeniable. while current trends prioritize larger models and extensive datasets, our focus remains on achieving efficient translation with modest resources. this study underscores our use of a compact model and limited computational assets to enhance translation quality. built upon the transformer model's base settings, our approach uses pre-trained models trained on japanese-english parallel data @xcite . a previous study involved fine-tuning on various datasets, yielding excellent translation within specific domains @xcite . in this study, we refined our fine-tuning dataset and systematically tuned hyperparameters to optimize results. post-fine-tuning, we harnessed model ensembling techniques to amalgamate multiple model outputs, leading to better translation quality. our study highlights the specifics of our system configurations and methods, offering a concise overview of our strategies. ## data selection and preprocessing in this section, we elaborate on the process of creating our fine-tuning dataset for the neural machine translation (nmt) system, with a focus on enhancing translation quality for the wmt competition. our approach involved meticulous data selection and preprocessing to ensure the effectiveness of our system. we describe the details behind selecting the base dataset, incorporating additional parallel corpora, and performing data cleaning to curate a high-quality training dataset. ## model training this section details the training process of our translation models using the fairseq toolkit. the selection and configuration of models, as well as the optimization parameters, are presented. additionally, our model training strategy (figure 2 ), including the utilization of advanced techniques such as mixed-precision training and beam search during decoding, is outlined. ## model ensembling with n-based reranking in this section, we delve into the intricacies of our advanced model ensembling approach (figure 3 ) coupled with n-based reranking, a technical strategy inspired by @xcite . our objective is to optimize translation quality through a combination of models and a refined reranking mechanism. ## post-processing of translation we found specific tendencies of mistranslation, such as adding double quotation marks, deleting part of the quotation marks, or repeating a specific word endlessly in our translation results. these phenomena are typical problems in machine translation tasks, so we added post-processing to reduce the mistranslations. after the post-processing, we submitted our results. our final submission was scored as shown in ## conclusion in this study, we embarked on an extensive exploration of high-efficiency model training strategies, leveraging limited computational resources alongside a streamlined model architecture rooted in the transformer framework's base settings. our investigation yielded crucial insights and techniques that converge to create a high-quality translation system. through our experimentation, we identified data cleaning, model averaging, ensembling, beam search, finetuning, parameter-tuning, and post-processing as pivotal techniques, enhancing the quality of our compact model and modest dataset.