Yishu: Yishu at WMT 2023 Translation Task (2023)

Abstract: This paper introduces the Dtranx AI translation system, developed for the WMT 2023 Universal Translation Shared Task. Our team participated in two language directions: English to Chinese and Chinese to English. Our primary focus was on enhancing the effectiveness of the Chinese-to-English model through the implementation of bilingual models. Our approach involved various techniques such as data corpus filtering, model size scaling, sparse expert models (especially the Transformer model with adapters), large-scale back-translation, and language model reordering. According to automatic evaluation, our system secured the first place in the English-to-Chinese category and the second place in the Chinese-to-English category.

## introduction this year, the dtranx ai team participated in the wmt2023 universal translation sharing task and focused on enhancing the performance in the zh-en and en-zh language directions. for data preprocessing, we employed various methods, including knowledge-based rules, language detection, and language modeling, to clean parallel, monolingual, and back-translated data. our data primarily comprised large-scale data mining and backtranslation. additionally, we applied punctuation regularization, byte pair encoding (bpe) @xcite , and subword regularization methods @xcite for processing the data, which yielded excellent results across all languages. in the modeling section, we have made enhancements to fairseq @xcite by increasing the model's depth and width. specifically, we augmented the transformer model et al. @xcite by significantly increasing the number of layers and widening the model architecture.this modification allows the model to capture more com-plex patterns and dependencies in the data. additionally, we have embraced the concept introduced by bapna et al. @xcite to expand the transformer model by incorporating languagespecific adapters, thus bridging the gap between diverse languages. lastly, we integrated the dense transformer model with the sparse adapter model and leveraged the language model to reranking the final results, leading to further improvements in system performance. for both english and chinese translation tasks, we have developed separate systems. we have enhanced the model capacity and applied adapter fine-tuning techniques, while also incorporating additional proprietary data for system training purposes.during the model inference phase, we have implemented reordering techniques to select more optimal translations. based on automatic evaluation, our system achieved first place in english to chinese translation and second place in chinese to english translation. ## conclusion in this paper, we present dtranx ai's submission to the wmt2023 universal translation shared task. for the chinese-english and english-chinese language pairs, we adopt a bilingual model as the fundamental structure and enhance it through various strategies. these include increasing the model capacity, fine-tuning with adapters, incorporating private relevant corpus, and optimizing the translation output by reordering. our experimental results demonstrate the effectiveness of these optimization techniques.