How Transferable are Attribute Controllers on Pretrained Multilingual Translation Models? (2024)

Abstract: Customizing machine translation models to comply with desired attributes (e.g., formality or grammatical gender) is a well-studied topic. However, most current approaches rely on (semi-)supervised data with attribute annotations. This data scarcity bottlenecks democratizing such customization possibilities to a wider range of languages, particularly lower-resource ones. This gap is out of sync with recent progress in pretrained massively multilingual translation models. In response, we transfer the attribute controlling capabilities to languages without attribute-annotated data with an NLLB-200 model as a foundation. Inspired by techniques from controllable generation, we employ a gradient-based inference-time controller to steer the pretrained model. The controller transfers well to zero-shot conditions, as it is operates on pretrained multilingual representations and is attribute- rather than language-specific. With a comprehensive comparison to finetuning-based control, we demonstrate that, despite finetuning’s clear dominance in supervised settings, the gap to inference-time control closes when moving to zero-shot conditions, especially with new and distant target languages. The latter also shows stronger domain robustness. We further show that our inference-time control complements finetuning. Moreover, a human evaluation on a real low-resource language, Bengali, confirms our findings. Our code is in the supplementary material.

## introduction pretrained multilingual translation models with massive coverage @xcite @xcite @xcite have become of the backbone of many translation systems. while their off-theshelf translation quality has been constantly improving @xcite @xcite , the flexibility of customization towards desired attributes, such as formality or grammatical gender, is another important metric. adapting generic systems for attribute-controlled translation relies on training data with attribute information. creating such annotated data often requires language-specific knowledge and manual curation. this makes data acquisition challenging even for single languages. when scaling to the numerous directions served by massively multilingual models, it quickly becomes impractical, as shown in figure 1 . while prior works @xcite @xcite showed promising results of finetuning on limited attribute-annotated data, to allow other languages without supervised data to similarly benefit from the customization possibilities, the transferability of the attribute controllers remains to be studied. a straightforward way to achieve attribute control is finetuning on attribute-specific data. recent works @xcite have shown that finetuning with just hundreds of attribute-specific sentences is sufficient. however, small finetuning data also brings the risk of overfitting and catastrophic forgetting @xcite . it is especially relevant when generalizing to new languages, where finetuning on some languages may erase the knowledge of others from pretraining @xcite @xcite . while these issues may be mitigated by partial finetuning @xcite , domain mismatch between the finetuning data and the test domain can still degrade translation quality. we will validate these concerns in various zero-shot conditions with different language relatedness and domains. on the other end of the spectrum, inference-time customization is another paradigm of attribute control. in this case, the pretrained model is fully unchanged in the training stage. at inference time, the generation process is steered towards desired attributes by e.g. re-weighting entries in the output distribution @xcite @xcite or editing model activations @xcite . to enable cross-lingual transfer, the controller must be trained on features that are shared across languages. this precludes methods that operate on the surface vocabulary level. in this work, we will extend an activation-based approach @xcite originally for decoder-only models to cross-lingual transfer on pretrained translation models. task formalization we focus on the following task: given a pretrained many-to-many multilingual translation model covering n languages and n (n -1) translation directions, along with parallel data on k (k ≪ n (n -1)) translation directions where the target translation corresponds to specific attributes (e.g., formality level), we aim to customize the pretrained model to translate with desired attributes for as many directions as possible. we refer to the subsequent model as an attribute controller. specifically, after learning on the k sets of parallel data with attribute annotation, to what extent can we transfer the attribute controller to the remaining n (n -1)k translation directions? ## transferring attribute controllers for multilingual translation to generalize to new translation directions, an ideal controller should be attribute-rather than languagespecific. that is, its representation for different attribute labels varies little with specific languages. inference-time control by classifier guidance: our first approach builds upon the observation that the activations of pretrained multilingual models capture commonalities of different languages @xcite ). an attribute classifier trained on these activations can then potentially transfer across languages, which we use at inference time to steer the generation for languages without attribute-annotated data. the control takes effect on inference-time model activations instead of parameters, as shown in figure 2 . specifically, we first train an attribute classifier while freezing the pretrained model, and then edit the model activations towards the wanted attribute based on the predicted label at inference time. this idea has shown success in controllable image synthesis @xcite ) and text generation @xcite . to the best of our knowledge, no prior work has explored it for cross-lingual transfer. specifically, we extend the approach by @xcite to encoder-decoder models. for machine translation, given a frozen pretrained model, we run forward passes with attributeannotated 1 parallel data (x, y) c for @xmath0, where x and y are the source and target sentences with individual sentence pairs (x, y) @xmath1, y), and c is the number of attribute labels. 1 only the target side needs attribute labels. while freezing the translation model's parameters, we train a classifier that maximizes p (c | h), where c is the ground-truth attribute label and h is the last decoder layer's hidden states after forceddecoding parallel data (x, y): @xmath2, encoder(x)). like with a standard model, the output distribution is then softmax(wh), where w maps the hidden states h to the vocabulary distribution. at inference time step t, the hidden state is: based on all available decoder states till t-1, we predict an attribute label: argmax c p (c | h 1,...,t-1 ). following @xcite , we meanpool the states from timestep 1 to t -1 for the prediction. it also empirically showed better performance than 1) using a token-level classifier without pooling and 2) operating on the cumulative sum of hidden states from all time steps so far. 3 as h 1,...,t-1 is only determined by a t-1 , we can rewrite p (c | h 1,...,t-1 ) as p (c | a t-1 ). comparing the prediction to the desired attribute c * , we can derive gradients measuring how much the current activations satisfy the desired c * . the gradients, ∇ a t-1 p (c * | a t-1 ), are then back-propagated for several iterations with given step sizes, resulting in updated activations ãt-1 , which further leads to modified decoder hidden state: @xmath3, ãt-1 ). a new output token y t (that more likely satisfies the control) is generated from ht by softmax(w ht ). finetuning-based control: a more common way to realize control is finetuning the pretrained model on attribute-specific parallel data, as done in domain adaptation @xcite . to transfer to directions without annotated data, the adaptation step must mostly learn the desired attributes rather than the specific languages in finetuning, so as not to forget the languages without annotated data. on our tasks, naive finetuning already works effectively: we finetune the full model on each attribute, resulting in one specialized model per attribute as shown in figure 2 . 4 partial finetuning e.g. with adapters @xcite ) is a more parameter-efficient approach. we do not explore partial finetuning in this work, as it does not fully align with our focus on the transferability of attribute controllers. ## experimental setup we experiment on two attribute control tasks: formality and grammatical gender control. as outlined in ## supervised conditions finetuning more effective than classifier guidance in supervised conditions: a comparison of scores in classifier guidance more robust to domain mismatch: as motivated in §4.2, the gender control results in gap between finetuning and classifier guidance shrinks in zero-shot conditions: while finetuning was consistently leading in supervised conditions ( §5), now under zero-shot conditions with unseen target languages, the gap shrinks. for formality control, on korean, the most distant language, cg consistently achieves stronger control results than finetuning, indicating more robustness when transferring to unfamiliar settings. overall in multilingual controllers help when the base model is not massively multilingual: in classifier guidance is complementary with finetuning: when applying cg on top of the finetuned models, we see the strongest control accuracy for both formality and gender control. this obser- on the more challenging case of gender control which involves domain mismatch, adding cg to finetuning does not impact translation quality on fr and causes a slight degradation on it. this is likely linked to poor hyperparameter choices in cg: due to time constraints we directly used the hyperparameters when applying cg alone, which are too strong for models already finetuned for attribute control. we are optimistic for improved scores under more fitting hyperparameters. ## conclusion to generalize attribute-controlled translation to data-scarce conditions, we asked the question "how transferable are attribute controllers on pretrained multilingual translation model?". we use a novel classifier guidance method to extend a pretrained nllb-200 model for attribute control and contrast its performance to finetuning-based control. our results led to the following recommendations for upgrading existing multilingual translation systems with attribute control capabilities: 1) given in-domain target sentences annotated with attributes, even as few as the lower hundreds, finetuning is the primary choice. 2) in case of distant new target languages or strong domain mismatches between the attribute-annotated data and test data, decoding with classifier guidance is more promising. otherwise finetuning is recommended. 3) in case specific resource constraints preclude finetuning or hosting multiple specialized variants of the underlying model, we then recommend inferencetime control by classifier guidance. 4) in case the underlying translation model is not massively multilingual, finetuning the model or training the controller on multiple target languages is beneficial.