Advances and Challenges in Unsupervised Neural Machine Translation (2021)

Abstract: Unsupervised cross-lingual language representation initialization methods, together with mechanisms such as denoising and back-translation, have advanced unsupervised neural machine translation (UNMT), which has achieved impressive results. Meanwhile, there are still several challenges for UNMT. This tutorial first introduces the background and the latest progress of UNMT. We then examine a number of challenges to UNMT and give empirical results on how well the technology currently holds up.

## relevance to the computational linguistics community this tutorial makes an attempt to review the latest progress on unmt by introducing advances and challenges for unmt. mt is a classic topic in the nlp community. recently, unmt has attracted great interest in the researchers in both the mt/nlp community and industry. this tutorial is primarily towards researchers who have a basic understanding of deep learning based nlp. we believe that this tutorial would help the audience more deeply understand unmt. presenter: rui wang presenter: hai zhao 1. introduction of mt 2. methods for unmt 3. challenges in unmt 4. summary (30 min) (70 min) (60 min) (20 min) 1.1 statistical mt 2.1 usmt and unmt 3.1 distant language 4.1 conclusion (smt) pairs 1.2 neural mt 2.2 advanced pre-training 3.2 domain adaptation 4.2 future trends (nmt) technologies 2.3 multilingualism 3.3 training efficiency -coffee break -(30 min) we introduce the cutting-edge technologies. this tutorial is primarily towards researchers who have a basic understanding of deep learning based nlp, and it is supposed to widen and deepen the understanding of cutting-edge nlp for the audience. ## tutorial outlines we will present our tutorial in three hours. the detailed tutorial outlines are shown in ## specification of any prerequisites for the attendees this tutorial is primarily aimed at researchers who have a basic understanding of nlp and deep learning. his research focuses on machine translation (mt), a classic task in nlp. his recent interests are traditional linguistic based and cutting-edge machine learning based approaches for mt. he (as the first or the corresponding authors) has published more than 40 mt papers in top-tier nlp/ml/ai conferences and journals, such as acl, emnlp, iclr, aaai, ijcai, tpami, taslp, etc. he has also won several first places in top-tier mt shared tasks, such as wmt-2018, wmt-2019, wmt-2020, etc. he has given several tutorials, such as eacl-2021, emnlp-2021, ccmt-2019, etc. he served as the area chairs of iclr-2021 and naacl-2021. 2. dr. hai zhao, professor, department of computer science and engineering, shanghai jiao tong university, china. his research interest is natural language processing. he has published more than 120 papers in acl, emnlp, coling, iclr, aaai, ijcai, and ieee tpami/tkde/taslp. he won the first places in several nlp shared tasks, such as conll and sighan bakeoff and top ranking in remarkable machine reading comprehension task leaderboards such as squad2.0 and race. he has taught the course "natural language processing" in sjtu for more than 10 years. he has given several tutorials, such as eacl-2021, emnlp-2021, etc. he is acl-2017 area chair on parsing, and acl-2018/2019 (senior) area chairs on morphology and word segmentation.