B ayesian Model-Agnostic Meta-Learning with Matrix-Valued Kernels for Quality Estimation (2021)

Abstract: Most current quality estimation (QE) models for machine translation are trained and evaluated in a fully supervised setting requiring significant quantities of labelled training data. However, obtaining labelled data can be both expensive and time-consuming. In addition, the test data that a deployed QE model would be exposed to may differ from its training data in significant ways. In particular, training samples are often labelled by one or a small set of annotators, whose perceptions of translation quality and needs may differ substantially from those of end-users, who will employ predictions in practice. Thus, it is desirable to be able to adapt QE models efficiently to new user data with limited supervision data. To address these challenges, we propose a Bayesian meta-learning approach for adapting QE models to the needs and preferences of each user with limited supervision. To enhance performance, we further propose an extension to a state-of-the-art Bayesian meta-learning approach which utilizes a matrix-valued kernel for Bayesian meta-learning of quality estimation. Experiments on data with varying number of users and language characteristics demonstrates that the proposed Bayesian meta-learning approach delivers improved predictive performance in both limited and full supervision settings.

## introduction quality estimation (qe) models aim to evaluate the output of machine translation (mt) systems at run-time, when no reference translations are available @xcite . qe models can be applied for instance to improve translation productivity by selecting high-quality translations amongst several candidates. a number of approaches have been proposed for this task @xcite @xcite @xcite , and a shared task yearly benchmarks proposed approaches @xcite . different users of mt output have varying quality needs and standards, depending for instance on the downstream task at hand, or the level of their knowledge of the languages involved. thus, the perception of the quality of mt output can be subjective, and therefore the quality estimates obtained from a model trained on data from one set of users may not serve the needs of a different set of users. in order to be able to make the most of these models, it is thus desirable to be able to efficiently adapt them to the needs and preferences of the end-user and with as little supervision as possible. however, most existing qe models are trained and evaluated in a fully supervised setting which assumes access to substantial quantities of labelled supervision data, which may not be available and can be expensive and time-consuming to obtain. in order to endow qe models with the ability to learn to adapt efficiently with limited supervision data, this work proposes a bayesian meta-learning framework for the training and evaluation of qe models that are able to adapt to the needs of endusers with limited supervision data. we further improve the performance of bayesian meta-learning for the task of quality estimation by extending the state-of-the-art bayesian model-agnostic meta-learning (bmaml) approach of @xcite to utilize stein variational gradient descent @xcite with matrix-valued kernels @xcite , and demonstrate that this leads to enhanced predictive performance in both limited and full supervision settings. ## bayesian model-agnostic meta-learning with matrix-svgd in this work we propose to improve the predictive performance of bmaml for quality estimation with the use of the matrix-svgd, which uses matrix-valued kernels for more effective parameter updates, in place of the original svgd algorithm used in @xcite . as pre-conditioning information, we use p , the average of the fisher information matrix of the particles: the full algorithm, which we refer to as matrix-bmaml, is outlined in algorithm 1. we use machine translation quality estimation as a case study in this work, and so assume access to a distribution of quality estimation tasks p(t ) (each qe task can be a qe user/annotator/post-editor with their corresponding data), and a quality estimation model f w parameterized by w , though the approach can also be applied to other natural language processing or computer vision tasks. algorithm 1 bayesian model-agnostic meta-learning with matrix-svgd require: distribution of qe tasks p(t ) require: qe model fw , number of update steps k require: learning rates α, β 1: initialize w 2: while not done do 3: sample batch of qe tasks ti ∼ p(t ) 4: for each ti do 5: sample d train t i from t train i 6: sample d val t i from t val i 7: w i 0 ← w 8: for @xmath0,..k do 9: w i @xmath1; d train t i , α) 10: end for 11: end for 12: w ← w -β∇w t i ∼p(t ) l f w i k ; d val t i 13: end while we first initialize the parameters of the quality estimation model (line 1). then in each iteration, we sample a batch of qe tasks (line 3), and for each qe task, we sample instances from its training and validation sets (lines 4-6). thereafter, task-specific parameters are initialized from the model's parameters (line 7), and then updated with k steps of matrix-svgd (using equations ( 2 ) and ( 4 ) to ( 7 )) (lines 8-10). at the end of each iteration, a metaupdate is performed on the model's parameters w . ## experiments and results we conduct experiments in two settings: in a limited supervision setting, where we provide all models access to only a limited number of training instances per qe task; and in a full-supervision setting, where we provide the models with access to all available training instances for each qe task. the qt21 dataset we evaluate our approach with the publicly available qt21 @xcite , a large-scale dataset containing translations from both statistical (smt) and neural (nmt) machine translation systems in multiple language directions 1 . this is the largest dataset with annotator information available. we make use of data from the english-latvian (en-lv) and english-czech (encs) language directions. the languages were chosen as they contain the largest number of annotators. each instance in the dataset is a tuple of source sentence, its machine translation, the corresponding post-edited translation by a professional translator (post-editor), a reference translation and other information such as (anonymized) post-editor identifier. we construct a qe dataset from this corpus by computing the hter @xcite values between each source sentence and its post-edited translation. we thereafter split the data into train, dev and test splits for each post-editor, which constitutes a qe task. a breakdown of the number of train, dev and test instances per qe task/post-editor is available in ## qe model the quality estimation model used by all methods is based on multi-lingual distilbert @xcite , a smaller version of multi-lingual 1 http://www.qt21.eu/resources/data/ bert @xcite trained with knowledge distillation @xcite . it accepts as input the source and machine translation outputs concatenated as a single text, separated by a '[sep]' token and prepended with a '[cls]' token. the representation of the '[cls]' token is then passed to a linear layer to predict hter @xcite values as regression targets. ## conclusions we proposed a bayesian meta-learning framework for adapting machine translation quality estimation models to the quality needs and preferences of each user with limited supervision data. we further extend a state-of-the-art bayesian metalearning method with the use of matrix-valued kernels, which enables the incorporation of preconditioning information for more effective svgd updates. using data from two language directions, we demonstrate improved predictive performance in both limited and full-supervision settings over recent state-of-the-art bayesian and non-bayesian meta-learning methods.