Reference Free Domain Adaptation for Translation of Noisy Questions with Question Specific Rewards (2023)

Abstract: Community Question-Answering (CQA) portals serve as a valuable tool for helping users within an organization. However, making them accessible to non-English-speaking users continues to be a challenge. Translating questions can broaden the community’s reach, benefiting individuals with similar inquiries in various languages. Translating questions using Neural Machine Translation (NMT) poses more challenges, especially in noisy environments, where the grammatical correctness of the questions is not monitored. These questions may be phrased as statements by non-native speakers, with incorrect subject-verb order and sometimes even missing question marks. Creating a synthetic parallel corpus from such data is also difficult due to its noisy nature. To address this issue, we propose a training methodology that fine-tunes the NMT system only using source-side data. Our approach balances adequacy and fluency by utilizing a loss function that combines BERTScore and Masked Language Model (MLM) Score. Our method surpasses the conventional Maximum Likelihood Estimation (MLE) based fine-tuning approach, which relies on synthetic target data, by achieving a 1.9 BLEU score improvement. Our model exhibits robustness while we add noise to our baseline, and still achieve 1.1 BLEU improvement and large improvements on TER and BLEURT metrics. Our proposed methodology is model-agnostic and is only necessary during the training phase. We make the codes and datasets publicly available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html#DomainAdapt for facilitating further research.

## related work neural machine translation (nmt) has made significant progress in the past decade and has even reached human-level performance in certain domains and language pairs. however, research in the field of question-answering (qa) is still in its early 1 technically, we supply the synthetic reference to the model. however, the references were used to sort the data according to their lengths to keep the training data order consistent among different models. the synthetic reference was not used to calculate the loss of the model. ## background the neural machine translation (nmt) task can be divided into two major components: fluency: ensuring grammatical correctness in the generated output for the target language. adequacy: preserving the meaning of the source text in the generated output. nmt systems often produce translations that are fluent but may lack adequacy. @xcite suggested that this is partly because the models tend to prioritize partially translated output over the source sentences during the decoding stage. achieving a balance between fluency and adequacy remains a challenging task. most nmt systems use maximum likelihood estimation (mle) @xcite ) objective during training. in the equation 1, where s represents the number of training samples in a batch, n is the number of tokens on the target-side of a training sample, y (s) n is the ground truth token on the target-side at step n, x (s) is the source sentence, y (s) <n represents the target-side tokens from previous steps, and θ denotes the model parameters. note that during training, the teacher forcing @xcite ) method is used for faster convergence and stable training. in teacherforcing method, ground truth tokens y (s) <n are used instead of partially translated output, ŷ(s) <n . major disadvantages of teacher forcing include • the trained model is exposed only to the training distribution but not its output. however, the reference is not supplied to the model during testing. this makes the model completely rely on its (possibly wrong) partially translated output, creating a discrepancy between training and testing @xcite . the challenges in nmt can be addressed through various approaches. for instance, a) gradually exposing the model to partially translated output as training progresses has been proposed as a solution @xcite . alternatively, b) one can pretrain the model using mle objectives before finetuning it with a desired evaluation metric, such as bleu. these techniques have demonstrated their effectiveness in improving results up to a certain point. nonetheless, it is worth noting that the mle objective often performs sufficiently well, especially when applied to clean, extensive datasets alongside suitable regularization techniques. in real-world use cases, many organic datasets, including conversations, question-answers, and reviews, are primarily monolingual. while backtranslation is recognized as an effective technique for leveraging monolingual data, it cannot be utilized when only source-side monolingual data is available. given that numerous datasets and websites are predominantly in english, and the construction of machine translation systems often involves translating from english to other languages, generating high-quality synthetic parallel datasets through forward-translation can be challenging. in back-translation, the source-side text is synthetic, while the target-side is considered the gold standard. back-translation aids in robust training, as it introduces errors on the source-side due to the synthetic nature of the text, while the target-side remains correct. this is the opposite of forwardtranslation, making it somewhat less effective but valuable in situations where better alternatives are lacking in standard machine translation systems. however, employing synthetic data through forward-translation can lead to adverse effects when dealing with noisy text translation. because the source-side is inherently noisy, forwardtranslated synthetic data will inevitably contain a substantial amount of errors. this, in turn, results in the propagation of errors during model training. furthermore, using alternative evaluation metrics as loss functions is often not very helpful, as they rely on gold-standard references, such as the bleu score, which may not be available. therefore, we propose a novel loss function that relies solely on source-side sentences, a target-side language model, and a source-side grammatical error correction model. ## methodology in this section, we first delve into mlm score and bertscore and explain why we have chosen to incorporate them into our loss function. subsequently, we detail our proposed training procedure for the translation of noisy questions. ## experiments and results in this section, we will provide an overview of the datasets used, discuss the baseline models, and present the results achieved by our model. ## conclusion we have developed a robust nmt system tailored for translating questions. our focus is on addressing the unique challenges posed by noisy questions, which are often presented in the form of statements due to the limited grammatical knowledge of users. the mle-based fine-tuning with synthetic data has several limitations, specifically when the source is noisy. we propose an mlm and bertscore-based training method to balance adequacy and fluency instead of using synthetic references for training data. our method improves translations for noisy questions compared to mle fine-tuning with synthetic data, and it also enhances translations on non-noisy data compared to the pre-trained model. we achieve up to 47.2 bleu and 35.4 ter scores, based on different settings. we conducted human evaluations with annotators from an e-commerce organization and observed a clear improvement in translation quality. we believe that the approach of balancing fluency and adequacy during training can be applied to other domains and languages. in the future, we plan to explore the use of quality estimation metrics, capable of scoring both fluency and adequacy during training. further, we would like to explore extending the method for other lowresource languages.