E volve MT : an Ensemble MT Engine Improving Itself with Usage Only (2023)

Abstract: This work proposes a method named EvolveMT for the efficient combination of multiple machine translation (MT) engines. The method selects the output from one engine for each segment, using online learning techniques to predict the most appropriate system for each translation request. A neural quality estimation metric supervises the method without requiring reference translations. The method’s online learning capability enables it to adapt to changes in the domain or MT engines dynamically, eliminating the requirement for retraining. The method selects a subset of translation engines to be called based on the source sentence features. The degree of exploration is configurable according to the desired quality-cost trade-off. Results from custom datasets demonstrate that EvolveMT achieves similar translation accuracy at a lower cost than selecting the best translation of each segment from all translations using an MT quality estimator. To the best of our knowledge, EvolveMT is the first MT system that adapts itself after deployment to incoming translation requests from the production environment without needing costly retraining on human feedback.

## introduction machine translation (mt) has experienced substantial progress in recent years, resulting in improving accuracy and more human-like translation output. despite these advancements, challenges remain, particularly in ensemble modeling. ensemble models integrate predictions from multiple individual models to achieve a more accurate final output. however, the effective combination of these models is often a complex task that requires thoughtful consideration of factors such as the model architecture, training data, and prediction combination methods. one of the significant challenges in mt ensembling is that the training data used to train the ensemble model, may not be fully representative of the data to be translated later, leading to a mismatch between the model and the data. this paper presents evolvemt, a method that addresses data drift in ensemble models by continual self-adaptation for optimal performance during usage. in the subsequent section, we review existing machine translation (mt) quality estimation metrics in the literature, which have been trained on human evaluation or post-editing datasets. in the approach section, we present a comprehensive explanation of the proposed method. in the experiments section, we describe our experimental design and provide quantifiable results demonstrating the enhancement resulting from the application of the proposed method, as compared to state-of-the-art quality estimation metrics. finally, we discuss the obtained results and present our conclusions. ## related work in the wmt20 metrics shared task @xcite , four reference-free metrics were submitted to evaluate machine translation outputs in the news translation task. these metrics use bilingual mapping of contextual embeddings from language models such as xlm-roberta @xcite to assess cross-lingual semantic similarity. however, they often struggle to accurately differentiate between human and machine translations, except for comet-qe @xcite , the only reference-free metric capable of doing so. the study by @xcite evaluated top mt systems from wmt 2020 using multidimensional quality metrics (mqm) and professional translator annotations. their results showed a low correlation between crowd worker evaluations and mqm, leading to different rankings and questioning previous conclusions. the study also found that automatic metrics based on pre-trained embeddings can outperform human crowd workers, suggesting that models trained with crowd-sourced human evaluations may have higher accuracy. the wmt21 metrics shared task @xcite , used mqm expert-based human evaluation to acquire reliable ratings, and evaluate metrics on news and ted talk translations produced by mt systems. results showed reference-free metrics comet-qe and openkiwi performed well in scoring human translations but not as well with mt outputs, and were strong at segment-level human translation evaluation while competitive with reference-based metrics in system-level evaluation. regemt @xcite is a referencefree metric in wmt21 that uses an ensemble of surface, syntactic, and semantic similarity metrics as input to a regression model. as demonstrated by cushlepor @xcite , it allows customization, outperforming lexical semantic similarity-based metrics with a higher computational cost. onception @xcite used active learning to converge an mt ensemble in a production environment to the best mt with evaluations acquired online. @xcite used bandit-learning to adapt mt policies based on simulated user feedback, outperforming the best single mt in mixeddomain settings. a contextual bandit strategy was proposed to make instance-specific decisions, but the system still required a human-in-the-loop (hitl) process. ## approach evolvemt is a quasi mt ensemble technique. in contrast to the traditional multi-system mt approach, which combines outputs from multiple mt systems to enhance translation accuracy and fluency, evolvemt prioritizes the selection of the most optimal translation from a finite set of mt systems, as we demonstrate in this section. figure 1 below shows the system architecture of evolvemt. the system is centered around a multi-class classification model that drives multiple processes to select the best mt model for translation requests. for each incoming machine translation request, we use spacy @xcite and stanza @xcite frameworks to extract morphological and lexical features. these features include the count of tokens, characters, and the average word length, as well as the frequency of part-of-speech labels (such as nouns, verbs, adjectives, etc.), the frequency of named entity recognition labels (including entities such as persons, locations, organizations, etc.), and the frequency of morphological features (e.g. gender and aspect). these features are combined with the 1024-dimensional embedding vector generated by the xlm-roberta encoder of the comet-qe model and stored alongside the input sentence in the ranked batch requests queue. this queue serves the purpose of prioritizing translation requests that necessitate precedence in processing. at the outset, requests in the ranked batch requests queue are ranked based on the order in which they are added. the highest-ranked machine translation (mt) request is selected for translation. the multi-class mt classifier employs the extracted features of the selected mt request to determine the mt systems to be utilized. the classifier prioritizes mt systems with a higher probability of having a higher comet-qe value. exploration of additional mt systems becomes more likely only if the probabilities from the classifier's prediction exhibit high entropy. this enables evolvemt to minimize the cost of exploration when the best mt is predicted with high certainty. finally, the selected mt systems are utilized to translate the mt request, and the comet-qe score is calculated for each translation. the translation with the highest score is chosen and returned in response to the mt request. the multi-class mt classifier is then updated online with the best mt system, as determined by the comet-qe score, serving as a label for the extracted features of the mt request. the online machine learning (ml) functionality of flaml automl framework @xcite is utilized for online learning. this capability enables the optimization of model hyper-parameters during the iterative course of ml, facilitating continual ml without repetitively hyper-tuning the classifier from scratch. subsequently, the classifier is employed to rerank the ranked batch requests queue, based on the uncertainty of the classifier, with requests having higher entropy being placed at the top of the queue for prioritized translation. getting the mt request with the maximum entropy from the queue after each iteration, helps prioritize the most informative sample for the iterative training of the classifier. as the classifier improves its ability to predict the best mt model for mt requests via learning, it reduces the likelihood of exploring other mt(s). the driving algorithm in evolvemt, which out-lines the primary process of the proposed method, is presented in pseudo-code form in algorithm 1. ## discussion the cost-benefit analysis of evolvemt highlights the trade-off between run-time efficiency and training expenses. while the run-time cost of evolvemt may be higher than that of best mt, it does not require the extensive and time-consuming training process required for traditional mt ensemble methods. this training process involves obtaining translations from all mts and scoring them using references generated by annotators. however, the increased run-time cost of evolvemt is offset by its ability to achieve superior production quality and adapt to changes in the data domain with a minimum amount of overhead. as the data domain changes, traditional mt ensemble techniques require costly retraining to accommodate the new domain, whereas evolvemt can adapt ## limitations the performance of the evolvemt system is contingent upon the reliability of the comet-qe model in providing accurate labels for the mt requests. utilizing the encoder's embeddings as features necessitates that the comet-qe model performs effectively on blind mt requests. the batch reranking of mt requests after each learning step may result in a computational bottleneck if the queue size is substantial. to mitigate this issue, an asynchronous re-ranking process could be implemented, whereby the queue is only reorganized once the re-ranking is completed. additionally, before the re-ranking process, a diverse subset of the queue can be selected based on the xlm-roberta embeddings, which reflect the novelty of the requests relative to previously processed mt requests. the source embeddings from the xlm-roberta model can be cached in parallel during the batch feature extraction process utilizing gpu capabilities, thus facilitating efficient comet-qe inference. evolvemt could also be optimized for cost-effectiveness by incorporating the cost of each mt in the ensemble into the algorithm. ## conclusion and future work this study presents a novel approach called evolvemt for ensembling machine translation (mt) engines, focusing on minimizing the number of engines required to be queried to achieve optimal quality. to evaluate the efficacy of the proposed method, a series of experiments were conducted, wherein evolvemt was implemented with varying levels of granularity in terms of the maximum number of engines permitted for each individual mt request. the quantitative results of the experiments indicate that, compared to the traditional method of querying all available mt engines, evolvemt offers a more cost-effective solution for the ensembling process without compromising the quality of the resulting translations. evolvemt presents a unique advantage in terms of cost efficiency compared to comet-qe ensemble. this is achieved by utilizing a stochastic exploration approach that selectively queries additional mt engines based on predicted probabilities, which are also employed in an active-learning framework by re-ranking mt requests after each learning step. furthermore, unlike traditional mt ensemble techniques, evolvemt can adapt in realtime to changes in customers' translation requests, without incurring the cost of acquiring human references or undergoing costly re-training or finetuning. in conclusion, this paper presents four significant contributions to the field of machine translation: (1) the introduction of the first self-improving mt system that operates without the need for human feedback; (2) the capability of adaptively optimizing the mt ensemble in response to production environment translation requests through online machinelearning; (3) the development of a novel approach for selectively querying mt engines rather than relying on translations from all available engines; and (4) the implementation of an active-learning framework that leverages uncertainties from the ensemble for batch translation.