Discourse-Aware Soft Prompting for Text Generation (2022)

Abstract: Current efficient fine-tuning methods(e.g., adapters, prefix-tuning, etc.) have optimized conditional text generation via training a small set of extra parameters of the neural language model, while freezing the rest for efficiency. While showing strong performance on some generation tasks, they don’t generalize across all generation tasks. We show that soft-prompt based conditional text generation can be improved with simple and efficient methods that simulate modeling the discourse structure of human written text.We investigate two design choices: First, we apply hierarchical blocking on the prefix parameters to simulate a higher-level discourse structure of human written text. Second, we apply attention sparsity on the prefix parameters at different layers of the network and learn sparse transformations on the softmax-function. We show that structured design of prefix parameters yields more coherent, faithful and relevant generations than the baseline prefix-tuning on all generation tasks.

## introduction recent advances in pre-trained langauge models (plms) @xcite @xcite have made great impact on text generation research, especially when they are finetuned on downstream tasks such as summarization, data-to-text generation, long-question answering, etc. consequent research have shown that plms' impact can further be improved when trained with more parameters, on more data and with more compute (gpt-3 @xcite , megatron @xcite ). on the flip side, storing larger lms or fully fine-tuning them (updating all the parameters) on downstream tasks usually causes resource or over-fitting issues. to mitigate fine-tuning issues, recent work have proposed prompt-based learning @xcite , which focus on learning textual prompts to steer plms' continuation towards desired output while keeping the model parameters frozen. while providing strong control of the plms, such prompt engineering could be time consuming requiring manual crafting. there is a growing research direction under prompt learning towards lightweight finetuning @xcite , which update only a small number of existing or extra parameters while keeping most of the original pre-trained parameters frozen. among them is prefix-tuning (li and liang, 2021), which prepends tunable continuous task-specific prompt vectors called prefixes to the input and only trains these continuous prompts during fine-tuning. although prefix-tuning can yield comparable results to full fine-tuning on some generation tasks, it did not generalize to tasks like abstractive summarization. in this work, we focus on prefix-tuning and investigate ways to improve its generalization on text generation tasks. we start asking the following questions that motivates our design choices: (1) do different parts of the transformer network process the prefix parameters more efficiently?; (2) do prefix parameters capture high-level discourse structure of the input text?; (3) can constraining prefix attention distribution to be structurally sparse enable better transfer of the task features? to address (1), we conduct empirical analysis on prefix-tuned bart @xcite , by varying the size of the prefix parameters at the encoder and decoder networks on text generation tasks. we find that the prefix parameters at higher layers impact the performance the most, while sparse prefixes can be sufficient at the lower layers ( § 6.1). motivated by this finding and to address (2), we investigate discourse-aware soft prompting via hierarchical blocking of prefix parameters. previous text generation work (e.g., abstractive summarization) has shown that abstraction can be better modeled with hierarchically structured architectures @xcite @xcite . to simulate a hierarchical discourse structure while only tuning additional prefix parameters, we first split the input and output text into segments and then assign sets of prefix parameters to each segment at different layers. with this structure, a set of prefixes can only be reached by their designated input or output segments during self-attention. we argue that for conditional generation tasks with hierarchically structured blocking of prefixes, we can simulate the structure of human writing styles: in input text each paragraph is a distinct section of related sentences and in output text (e.g., summary) each output sentence outlines salient concepts. thus, a set of prefixes designated to each input and output segment at different layers can learn levels of abstractions from each section. we show performance improvements over baseline prefix tuning, yielding comparable results to full fine-tuning in several generation tasks in § 6.2. inspired by these findings, we address (3) by applying a suite of known sparse attention alternatives to standard full-attention matrix during prefixtuning. our goal is to analyze whether sparse prefix-tuned models can encode important features better than dense prefix-tuned models. prior work have shown that sparsity in self-attention not only improves training efficiency, but also focusing on salient features while pushing down unrelated features and relations can impact the model performance. this improves language modeling @xcite , language understanding @xcite ) and text generation @xcite @xcite . motivated by previous work we apply sparsity into the self-attention by substituting the softmax function with a sparse alternative under encoder prefix-tuning (without introducing any additional model parameters). with spectral analysis we show that sparse prefix parameters can identify important features compared to dense prefix parameters. our quantitative analysis yield performance improvements on automatic metrics over best prefix-tuning models, while human judges generally prefer our sparse prefix model generations on factuality and coherence criteria ( § 6.3 and § 6.4). efficient tuning of plms offers a promising new direction for many nlp tasks including text generation, which we study in this work. our results suggest that prompt design with hierarchical structure and sparsity in prefix parameters: (i) generate more coherent and faithful text than baseline prefix-tuning across several summarization and data-to-text generation tasks, (ii) trail the performance of fine-tuning on most summarization tasks with a small margin, while at par with fine-tuning on data-to-text generation tasks, (iii) improve all the baselines in low-resource settings. ## prefix-tuning following the intuition of the text-based prompt tuning methods @xcite , prefix-tuning (li and liang, 2021) introduces task-specific prompt parameters with the goal of triggering the desired response of the lm without updating any of the original lm parameters. at each layer, it prepends tunable prefix parameters (also called softprompts) as additional keys and values to the multihead self-attention. prefix-tuning defines h (l) i as the activation at the i-th token (@xmath0: introducing sparsity. secondly, the weaker prefix attention in longer inputs (figure 1 -cnn/dm attention matrices) may imply that the attention neglects important connections, and potentially dis-turbed by many unrelated words. this issue can be attributed to the softmax function at attention score calculation @xcite . softmax produces attention distribution with dense dependencies between words, and fails to assign near/exactly zero probability to less meaningful relations. thus, the model neglects to put more attention to important connections while also being easily disturbed by many unrelated words @xcite . this issue is more pronounced in tasks like abstractive summarization, since only a handful of salient input aspects is needed to compose a coherent summary. sparse attention mechanisms @xcite can remedy this issue by learning to put more emphasis on the important features. below we describe ways to apply a suite of blocking schemes and sparsity in prefix-tuning models as sketched in figure 2 . each block represents attention matrix @xmath1, while each row vector a @xmath2, @xmath3, are the attention weights of p -prefix and t -input-key features. ## conclusion we have described simple but effective prompt design options for prefix-tuning of text generation tasks. we enrich prefix parameters with structural biases by way of: prefix-blocking at different layers of the network, sparsity on prefix-parameters and an ensemble of both biases. we show with quantitative and human evaluations on metrics such as coherence and faithfullness that discourse aware prefix improves prefix-tuning across all text generation tasks even at low data settings.