JUST System for WMT 20 Chat Translation Task (2020)

Abstract: Machine Translation (MT) is a sub-field of Artificial Intelligence and Natural Language Processing that investigates and studies the ways of automatically translating a text from one language to another. In this paper, we present the details of our submission to the WMT20 Chat Translation Task, which consists of two language directions, English –> German and German –> English. The major feature of our system is applying a pre-trained BERT embedding with a bidirectional recurrent neural network. Our system ensembles three models, each with different hyperparameters. Despite being trained on a very small corpus, our model produces surprisingly good results.

## introduction the language of chat texts is considered a common language where people are rarely paying attention to correct spelling. therefore, using the traditional methods of machine translation (mt), like dictionaries, is insufficient @xcite . as deep learning (dl) models are becoming more evolved and complex, this motivates the natural language processing (nlp) community researchers to employ them for challenging tasks such as mt of informal language, such as what is used in chat. techniques like contextual word embeddings and pre-trained dl models are becoming very common in natural language generation (nlg) tasks such as mt @xcite @xcite . the chat translation task is a new task in the fifth conference on machine translation (wmt20). 1 translating chat text, specifically the chats of customer support, is a main and exciting task in the field of mt. this kind of tasks has not been widely considered in previous mt studies, 1 http://www.statmt.org/wmt20/chat-task.html mostly because of the absence of openly existing datasets. the target of this new chat translation task is to translate the customer support chat text from english to german and vice versa. the essential goal of this task is to develop models that can translate conversational text and study the use of multilingual models. we take part in the wmt20 shared chat translation task in two language directions: @xmath0. in this paper, we discuss our submission for this task, which is based on the bidirectional recurrent neural networks (bi-rnn) @xcite and using the pre-trained bert embedding, known as bert-base-multilingual-cased @xcite . this paper is constructed as follows. in section 2, the task and data descriptions are provided. section 3 discusses our proposed model. section 4 shows the experiments we conduct and their results. finally, the conclusion is in section 5. ## task and data description the chat translation shared task of wmt20 offers participants the opportunity to address a challenging problem faced by many companies today as they expand their customer support units to multiple different languages. the shared task provides a dataset consisting of a set of conversations between agents and customers. the organizers supplied a corpus for the english-german language pair. specifically, the task involves translating the chat text of an agent speaking english and a customer speaking german. we are asked to translate the agent's chat text from english to german, and the customer's from german to english. the dataset used for this shared task depends on the corpus of taskmaster-1 @xcite , which has the english language, and it consists of dialogues in six fields. a small part of this dataset was chosen and translated to german. the shared task has been provided with train, development, and test sets in json format. each chat in the data file has a specific structure. each conversation contains a speaker (who is either an agent or a customer), a source chat text, and a target chat text. for the test set file, we are asked to translate the source chat text to target depending on the speaker. if it is an agent, the translation is from english to german. otherwise, the translation is from german to english. for evaluating the participating models, the task organizers employ both automatic metrics (bleu @xcite and ter @xcite ) as well as human evaluation. ## just system our system follows the sequence of steps shown in figure 1 . in the following subsections, we discuss each step in details. ## results the results based on the dev set are show in for evaluation on the test set, we combine the train and dev dataset of each group into one file. we train each group separately and then we ensemble the three models into one. this model is used to get the target of each sentence in the test set of each group. it is worth mentioning that we only use the small dataset provided with the shared task.