Inference-Time Control by Classifier Guidance:
Our first approach builds upon the observation that the activations of pretrained multilingual models capture commonalities of different languages Pires et al. ( 2019 ); Liu et al. ( 2020 ) .
An attribute classifier trained on these activations can then potentially transfer across languages,
which we use at inference time to steer the generation for languages without attribute-annotated data.
The control takes effect on inference-time model activations instead of parameters , as shown in Figure 2 .
Specifically,
we first train an attribute classifier while freezing the pretrained model,
and then edit the model activations towards the wanted attribute based on the predicted label at inference time.
This idea has shown success in controllable image synthesis Dhariwal and Nichol ( 2021 ) and text generation Li et al. ( 2022 ) .
To the best of our knowledge, no prior work has explored it for cross-lingual transfer.

Specifically,
we extend the approach by Dathathri et al. ( 2020 ) to encoder-decoder models.
For machine translation,
Given a frozen pretrained model,
we run forward passes with attribute-annotated 1 1 1 Only the target side needs attribute labels. parallel data (\mathbf{X},\mathbf{Y})^{c} for c\in[C] , where \mathbf{X} and \mathbf{Y} are the source and target sentences with individual sentence pairs (\mathbf{x},\mathbf{y})^{i}\in(\mathbf{X},\mathbf{Y}) , and C is the number of attribute labels.

While freezing the translation model’s parameters,
we train a classifier that maximizes P(c\mid\mathbf{h}) ,
where c is the ground-truth attribute label and \mathbf{h} is the last decoder layer’s hidden states after forced-decoding parallel data (\mathbf{x},\mathbf{y}) :

\mathbf{h}=decoder(\mathbf{y},encoder(\mathbf{x}))

Like with a standard model, the output distribution is then \text{softmax}(\mathbf{W}\mathbf{h}) ,
where \mathbf{W} maps the hidden states \mathbf{h} to the vocabulary distribution.

At inference time step t , the hidden state is:

\mathbf{h}_{t}=decoder(y_{t-1},\mathbf{A}_{t-1}), (2)

where y_{t-1} is the token from the previous step,
and \mathbf{A}_{t-1} is the model activations. \mathbf{A}_{t-1} contains activation key-value pairs 2 2 2 Note these are not the key/value projection weights of the Transformer, but the activations after applying the projections. from the decoder self-attention and cross-attention for steps 1 to t-1 ,
and is cached in most Transformer decoding implementations Ott et al. ( 2019 ); Wolf et al. ( 2020 ) .

Based on all available decoder states till t-1 ,
we predict an attribute label: \mathrm{argmax}_{c}P(c\mid\mathbf{h}_{1,...,t-1}) .
Following Dathathri et al. ( 2020 ) , we meanpool the states from timestep 1 to t-1 for the prediction.
It also empirically showed better performance than 1) using a token-level classifier without pooling and 2) operating on the cumulative sum of hidden states from all time steps so far. 3 3 3 In initial experiments training an English-German formality classifier, the accuracy on the dev set was 86.7% (meanpool), 66.1% (token-level) and 73.0% (cumulative sum).

As \mathbf{h}_{1,...,t-1} is only determined by \mathbf{A}_{t-1} ,
we can rewrite P(c\mid\mathbf{h}_{1,...,t-1}) as P(c\mid\mathbf{A}_{t-1}) .
Comparing the prediction to the desired attribute c^{*} ,
we can derive gradients measuring how much the current activations satisfy the desired c^{*} .
The gradients, \nabla_{\mathbf{A}_{t-1}}P(c^{*}\mid\mathbf{A}_{t-1}) ,
are then back-propagated for several iterations with given step sizes,
resulting in updated activations \tilde{\mathbf{A}}_{t-1} ,
which further leads to modified decoder hidden state:

\tilde{\mathbf{h}}_{t}=decoder(y_{t-1},\tilde{\mathbf{A}}_{t-1})

A new output token y_{t} (that more likely satisfies the control) is generated from \tilde{\mathbf{h}}_{t} by \text{softmax}(\mathbf{W}\tilde{\mathbf{h}}_{t}) .