Controlling the Focus of Pretrained Language Generation Models (2022)

Abstract: The finetuning of pretrained transformer-based language generation models are typically conducted in an end-to-end manner, where the model learns to attend to relevant parts of the input by itself. However, there does not exist a mechanism to directly control the model’s focus. This work aims to develop a control mechanism by which a user can select spans of context as “highlights” for the model to focus on, and generate relevant output. To achieve this goal, we augment a pretrained model with trainable “focus vectors” that are directly applied to the model’s embeddings, while the model itself is kept fixed. These vectors, trained on automatic annotations derived from attribution methods, act as indicators for context importance. We test our approach on two core generation tasks: dialogue response generation and abstractive summarization. We also collect evaluation data where the highlight-generation pairs are annotated by humans. Our experiments show that the trained focus vectors are effective in steering the model to generate outputs that are relevant to user-selected highlights.

## introduction transformer-based models pretrained on largescale text data have become the dominant paradigm for natural language generation (nlg) tasks @xcite @xcite . the attention module @xcite , which aggregates information via a weighted average over word-level embeddings, plays a vital role in these models. the attention mechanism serves two major purposes: (1) it captures linguistic phenomena in the input @xcite @xcite ; (2) it helps the model focus on relevant portions of the input (e.g., alignment in machine translation @xcite and abstractive summarization @xcite ). the attention module is particularly useful as it does not require any explicit supervision: the figure 1 : illustration of our motivation: different highlights in the input (including persona) lead to different generations. this example is from our collected dialogue data for evaluation (section 3). model learns to attend to relevant parts of the input by itself through end-to-end training. however, this property makes it difficult to explicitly control the model's focus. if the model happens to put focus on some span of context that the user thinks is not so important, we currently do not have a mechanism to correct it. this is especially sub-optimal in some nlg applications involving a relatively long input such as dialogue or summarization: focusing on different spans of the input could result in completely different generations (illustrated in figure 1 ). it would be attractive to give the user an option to control the model's focus. in this work, we aim to develop a mechanism to steer the model to generate output relevant to some user-specified input spans (which we term as highlights). 1 this goal, however, brings about significant challenges. for one, many popular nlg datasets are collected in an end-to-end manner, i.e., without annotations of which spans of input are most relevant to the reference target. it would also be ideal for the proposed approach to be compatible with existing pretrained transformer models, as retraining such models is often costly. in this work, we propose an focus vector framework to address the challenges outlined above. our contributions are as follows: ## model formulation we assume the target model is a standard pretrained transformer encoder-decoder model @xcite that has already been finetuned on end-to-end task-specific data (e.g., dialogue or summarization) with the standard negative log-likelihood (nll) loss. our goal is to establish a control mechanism whereby the user can highlight several spans of the input, and the model is supposed to generate outputs relevant to the highlighted text. crucially, this mechanism should not change the base model, in order to allow the user to default back to the original model if desired. we begin by establishing notation. we denote the end-to-end training data by tx, yu, where x " tx 1 , ..., x n u refers to the input token sequence, and y refers to the corresponding reference target token sequence. during evaluation, some spans of the input x will be highlighted, and we use a binary indicator c i to indicate whether the i th input token is to be highlighted during generation. in this work we only consider a set of complete sentences as a valid highlight span. this design choice is mainly for convenience during our human-annotated evaluation data collection, and our framework can readily be generalized to phrase-level highlights. suppose the encoder model is composed of l transformer layers. we denote the d-dimensional output embedding of the i th position on the l th encoder layer by h l i . we use th 0 i u to denote the input embeddings. each decoder layer performs multi-head cross-attention on the outputs of the encoder, where the attention weight computation for the h th head on the l th decoder layer is formulated as below: here kp¨q is a linear transform, and α i,j is the attention weight assigned to encoder output h l i , for the j th position decoder query vector q j . we use p m py|xq to denote the probability assigned to y given input x by the original target model. for more details of the transformer encoder-decoder architecture, we refer readers to @xcite . our proposed framework involves two stages. we first obtain automatic highlight annotations using attribution methods. then, these annotations are used to train the focus vectors. in the next section, we review the attribution methods. ## evaluation data collection we test our method on two nlg tasks: dialogue response generation and abstractive summarization. for the dialogue task, we adopt the personachat dataset @xcite . it is an open domain multi-turn chit-chat dataset, where two participants are required to get to know each other by chatting naturally. each of them is given a persona: several pieces of personal information such as "i major in computer science", serving as background information. the participants are required to reflect their assigned persona in the conversation. for summarization, we adopt the cnn/dailymail dataset @xcite , which is a standard dataset for end-to-end abstractive summarization. to save space, we defer details and statistics of the datasets to appendix a. both personachat and cnn/dailymail are created end-to-end and do not contain annotated highlight spans. for principled evaluation, we utilize the amazon mechanical turk (amt) platform to collect evaluation sets where the highlightgeneration pairs are annotated by humans. for personachat, each turker foot_1 is shown a dialogue history and the corresponding persona of the speaker. the dialogue history is randomly selected from the original test set of personachat. then the turker is required to choose 1-3 sentences as highlights (for example, one sentence in persona, and one sentence in dialogue history), and write down a dialogue response that not only continues the current dialogue, but also is relevant to the chosen highlights. finally, we ask the turker to repeat the above process, but select a different set of highlights and provide another response. after a few preliminary trials and modifications to our instructions / rewards, we find that turkers comply nicely with our instructions and provide highquality highlight-response pairs. for cnn/dailymail however, we first found that turkers had difficulty writing a high-quality summary for a given news article, with many turkers giving random responses even after we increased the reward. this is perhaps unsurprising given that writing a good summary is challenging and the reference summaries are written by experts. after a few disappointing iterations, we turn to a compromise: we directly provide the turkers with the reference summary, and only ask them to select 2-5 relevant sentences in the article. this simplifies the task, and we are able to collect high-quality labels. this compromise is not ideal, as it reverses the order of highlighting and generation. however, we find that in most cases, the reference summaries in cnn/dailymail are well covered by several "key" sentences in the article, which are highlighted by the turkers. therefore, we believe this compromise does not hurt the soundness of our evaluation. in order to ensure high data quality for both dialogue and summarization, we design a qualification test that turkers need to pass before conducting the actual tasks. several automatic checks and a minimal time limit are added in the scripts to prevent trivial answers. we also manually monitor the incoming submissions, and ban misbehaving turkers and discard their submissions. more details about our amt setup are provided in appendix b. our final collected datasets include 3,902 highlight-generation pairs for personachat, and 4,159 pairs for cnn/dailymail. they are randomly split 50/50 into dev/test sets. we in-clude a number of samples of our collected data in the supplementary materials. our code and the collected dataset will be released in https://github.com/question406/ learningtofocus . we hope that this evaluation data could facilitate future research in this direction. ## conclusion in this work we propose the focus vector framework as a light-weight solution to control the focus of pretrained transformer models. it has two major advantages: (1) focus vectors act as simple transformations to the embeddings in the encoder, and the transformer model is kept fixed; (2) attribution methods are utilized to get automatic highlight labels for training focus vectors. we test our approach on two tasks: dialogue response generation, and abstractive summarization. for evaluation, we collect data where the highlight-generation pairs are annotated by humans. experiments show that the trained focus vectors are effective in steering the model to generate output text that is relevant to the specified highlights.