SlowGuess/NAACL_2022_OCR huggingface dataset:
Junyi Li $^{1,3,4}$ , Tianyi Tang $^{1}$ , Jian-Yun Nie $^{3}$ , Ji-Rong Wen $^{1,2,4}$  and Wayne Xin Zhao $^{1,4*}$
$^{2}$ School of Information, Renmin University of China
<sup>3</sup>DIRO, Université de Montréal
<sup>4</sup>Beijing Key Laboratory of Big Data Management and Analysis Methods
{lijunyi, steven_tang, jrwen}@ruc.edu.cn
nie@iro.umontreal.ca batmanfly@gmail.com
Pretrained language models (PLMs) have made remarkable progress in text generation tasks via fine-tuning. While, it is challenging to fine-tune PLMs in a data-scarce situation. Therefore, it is non-trivial to develop a general and lightweight model that can adapt to various text generation tasks based on PLMs. To fulfill this purpose, the recent prompt-based learning offers a potential solution. In this paper, we improve this technique and propose a novel prompt-based method (PTG) for text generation in a transferable setting. First, PTG learns a set of source prompts for various source generation tasks and then transfers these prompts as target prompts to perform target generation tasks. To consider both task- and instance-level information, we design an adaptive attention mechanism to derive the target prompts. For each data instance, PTG learns a specific target prompt by attending to highly relevant source prompts. In extensive experiments, PTG yields competitive or better results than fine-tuning methods. We release our source prompts as an open resource, where users can add or reuse them to improve new text generation tasks for future research. Code and data can be available at https://github.com/RUCAIBox/
Transfer-Prompts-for-Text-Generation.
In natural language processing (NLP), text generation is an important research topic that aims to automatically produce understandable text in human language from input data (Li et al., 2022). In recent decades, various approaches have been widely applied to a variety of text generation tasks (Li et al., 2019; Gehring et al., 2017; Li et al., 2021a), especially the emergence of pretrained language models (PLMs) (Li et al., 2021c). By involving large-scale parameters pretrained on massive general corpora,
PLMs such as GPT-3 (Brown et al., 2020) have achieved substantial progress in text generation. Through the fine-tuning paradigm, PLMs can adapt to various text generation tasks by directly adjusting the model parameters with labelled datasets.
However, in real-world scenarios, we are inevitably confronted with tasks having only limited labelled data (e.g., new domains). It is often difficult to fine-tune text generation models in a data-scarce situation (Chen et al., 2020; Li et al., 2021b). Although the input and output formats are different for various text generation tasks, these tasks essentially adopt similar learning and generation mechanism (e.g., Seq2Seq (Sutskever et al., 2014)). Furthermore, the success of PLMs sheds light on the possibility of developing general or transferable text generation models. For example, Radford et al. (2019) framed generation tasks as language modeling by predicting the next token given previous tokens. Based on these studies, we aim to devise a general and lightweight text generation approach that can effectively adapt to various new tasks and datasets, based on PLMs.
To fulfill this purpose, the recently proposed prompt-based learning offers a potential technical solution (Liu et al., 2021b). In this paradigm, a text generation task can be solved with the help of a prompt containing task-specific information. For example, T5 (Raffel et al., 2020) framed summarization and question answering into a text-to-text format by utilizing prompts "summarize:" and "answer the question:". Based on learned or manually designed prompts, PLMs can be leveraged to perform existing or new generation tasks without being tuned (Brown et al., 2020; Li and Liang, 2021), which provides a unified approach to utilizing PLMs for various generation tasks. Furthermore, to quickly adapt PLMs to new NLU tasks, several works directly used a soft prompt learned from source NLU tasks to initialize the prompt for a target NLU task (Vu et al., 2021; Su et al., 2021).
Human Language Technologies, pages 3506 - 3518
Inspired by these studies, we aim to apply prompt-based methods to data-scarce text generation tasks in a transferable setting.
Despite promising, there are still two major challenges for transferring prompts in text generation. Firstly, it has been found that prompts are highly task-specific (Gao et al., 2020), and it is difficult to effectively transfer or reuse existing prompts for new tasks. Second, for a single task, even a welllearned prompt may not be suitable for all the data instances from a large population (Scao and Rush, 2021), and hence it is non-trivial to design effective transferring strategy considering both task- and instance-level characteristics.
To address the above issues, we propose PTG: Prompt Transfer for Text Generation, a novel prompt-based transfer learning approach for text generation. PTG is built upon a transfer learning setting. Specifically, we learn source prompts from a number of representative source generation tasks and then transfer these prompts as target prompts to perform target generation tasks. The core idea is that these learned source prompts serve as representation bases (i.e., value vectors in self-attention mechanism). For each data instance from a new task, we learn a specific target prompt by attending to highly relevant source prompts. To support such an approach, we construct a multi-key memory network storing both source prompts and prompt clusters for key-value prompt finding, and then design an adaptive attention mechanism considering both task- and instance-level information to derive the target prompt. Instead of using a fixed prompt for a new task, our approach is able to effectively learn the most suitable prompt representation from source prompts for a specific data instance. Such an adaptive mechanism considers the specific instance-level features, making our approach more flexible to transfer to new text generation tasks.
To the best of our knowledge, we are the first to introduce the idea of prompting in transfer learning to address text generation tasks. For evaluation, we test PTG on 14 datasets from three sets of text generation tasks: i) compression to express salient information in concise text such as summarization; ii) transduction to transform text while preserving content precisely such as style transfer; and iii) creation to produce new content from input context such as story generation. In both fully-supervised and few-shot experiments, PTG yields competitive or better results than fine-tuning PLMs.
Besides performance benefits, more importantly, we release our source prompts to serve as an open-source prompt library. Researchers can train new task prompts added to our library and reuse these learned prompts to improve unseen text generation tasks. Our library can further act as an analysis tool, such as analyzing what factors influence prompts' transferability across generation tasks and interpreting the task similarity by measuring the corresponding prompt similarity.
Prompt-based Language Models. Prompt-based learning is a way of leveraging PLMs by preponding task-specific instructions to the task input when feeding into PLMs. Early approaches mainly utilized hand-crafted prompts to adapt to different generation tasks (Brown et al., 2020; Raffel et al., 2020; Zou et al., 2021). However, manually designed prompts are not flexible and cannot be applied to more kinds of new tasks. Thus, recent works have focused on automating the learning of discrete prompts (Shin et al., 2020; Gao et al., 2020). However, learning prompts over discrete space is hard to optimize and likely to be sub-optimal. To address these problems, many works proposed to optimize continuous prompts (Liu et al., 2021c; Li and Liang, 2021), which are more flexible to many kinds of tasks. Among these studies, prefix-tuning (Li and Liang, 2021) presupposed a sequence of vectors to the input for text generation tasks. By contrast, we utilize soft prompts to investigate transfer learning for text generation and demonstrate that generation tasks can often help each other via prompt transfer.
We are also closely related to existing works on transfer learning in NLP tasks (Jeong et al., 2020; Wiese et al., 2017; Liu et al., 2019). Prior studies have shown that cross-task transfer can address the data scarcity issue (Wiese et al., 2017), enhance the ability to complex reasoning and inference (Jeong et al., 2020), or learn effective word representations (Liu et al., 2019). Efforts to transfer prompts for addressing NLU tasks have also been developed (Vu et al., 2021; Su et al., 2021). As a representative work, Vu et al. (2021) used the learned prompt to directly initialize the prompt for a target task while not considering the specific input. Our work focuses on challenging text generation tasks by utilizing prompts to extract implicit task-related
knowledge and considering specific model inputs for the most helpful knowledge transfer.
Generally, the objective of text generation is to model the conditional probability  $\operatorname{Pr}(y|x)$ , where  $x = \langle w_1, \ldots, w_n \rangle$  and  $y = \langle z_1, \ldots, z_m \rangle$  denote the input text and output text respectively and consist of sequences of tokens from a vocabulary  $\mathcal{V}$ .
Prompting is a technique for injecting extra task information to PLMs as a condition during the generation of output text (Brown et al., 2020). Typically, prompting is conducted by prepending a series of tokens (discrete prompts) or continuous vectors (continuous prompts) to the input  $x$ . In our paper, we adopt continuous prompts. Specifically, given a series of  $n$  input tokens,  $x = \langle w_1, \dots, w_n \rangle$ , we first utilize PLM to embed the tokens, forming a matrix  $E_x \in \mathbb{R}^{n \times e}$ , where  $e$  is the dimension of the embedding space. Then, our continuous prompt  $p$  is represented as a parameter matrix  $E_p \in \mathbb{R}^{l \times e}$ , where  $l$  is the number of prompt vectors. The prompt  $p$  is then prepended to the embedded input forming a single matrix  $[E_p; E_x] \in \mathbb{R}^{(l + n) \times e}$  which is encoded by PLMs as an ordinary sequence, such that the model maximizes the likelihood of the ground-truth  $y$ , i.e.,  $\operatorname*{Pr}(y | [p; x])$ .
3.2 Prompt-based Transfer Learning
In a general transfer learning framework, we define a set of source generation tasks  $\mathcal{S} = \{\mathcal{S}_1,\dots ,\mathcal{S}_T\}$  where the  $t$ -th task  $S_{t} = \{(x_{i}^{t},y_{i}^{t})\}_{i = 1}^{n_{t}}$  contains  $n_t$  tuples of the input text  $x_{i}^{t}\in \mathcal{X}$  and its corresponding output text  $y_{i}^{t}\in \mathcal{V}$ . For a target generation task  $\mathcal{T}$ , the goal of transfer learning is to use the previously learned task-specific knowledge of the source tasks  $\mathcal{S}$  to help improve the performance of a learned model  $f_{\theta}$  (parameterized by  $\theta$ ) in the target task  $\mathcal{T}$ .
In this paper, we consider a new transfer learning setting based on prompting. Specifically, the parameters of the underlying PLM are frozen, and the text generation tasks have to be fulfilled by prepending prompts (continuous vectors) to input as described in Section 3.1. Formally, we will learn an independent source prompt  $p_t$  for each source generation task  $S_t$  based on a shared frozen PLM by maximizing the likelihood  $\operatorname*{Pr}(y_i^t|[p_t; x_i^t])$ . Our core idea is to transfer these learned source prompts to a new (target) text generation task, such that the
Figure 1: Overview of our proposed model PTG.
target generation task can be performed in zero or few shot settings.
Our proposed method, Prompt Transfer for Text Generation (PTG), is depicted in Figure 1. Our approach first learns a number of source prompts for various representative source generation tasks, and then derive the prompt for the target generation task with a novel adaptive attention mechanism. Next we will describe each part in detail.
4.1 Learning Transferable Source Prompts
To extract task-related knowledge from source generation tasks, we learn a set of source prompts and store them in a source prompt pool. The motivations for introducing the prompt pool are twofold. First, we expect to identify the similarity between source generation tasks. Second, the pool stores task-specific prompts for every source task, which can be shared by all target tasks.
Constructing Source Prompt Pool. For each source generation task  $S_{t}$ , we aim to learn a source prompt  $p_{t}$  given its training data  $\{(x_i^t,y_i^t)\}_{i = 1}^{n_t}$ . Following the learning steps in Section 3.1, we learn an independent source prompt  $p_{t}$  for each source task  $S_{t}$  based on a shared frozen PLM, i.e., BART. These source prompts are stored in a prompt pool  $\mathcal{P} = \{p_1,\dots ,p_t,\dots ,p_T\}$ , where  $T$  is the total number of source text generation tasks.
To construct the source prompt pool, a key point lies in the selection of source text generation tasks. According to the literature (Deng et al., 2021), text
generation tasks can be categorized as performing compression, transduction, or creation based on changes in conveyed information from input to output. Moreover, recent studies have shown that few but diverse source tasks/domains also lead to remarkable transfer learning performance (Friedman et al., 2021; Zhuang et al., 2021). Therefore, we select six text generation tasks (including 14 public datasets) within the three types of generation tasks for learning their corresponding source prompts.
Clustering Source Prompts. As described above, the source tasks are diverse in the prompt pool. It is challenging for PLMs to effectively transfer or reuse existing prompts for new tasks. Thus, to identify the similarity between source tasks (prompts), we construct a source prompt pool for more effective cross-task knowledge transfer. In particular, via spectral clustering algorithm (Ding et al., 2001), we group these source prompts into several prompt clusters. Under this algorithm, each prompt  $p_t$  is regarded as a node in a weighted undirected graph  $\mathcal{G}$ . The similarity degree (weight) between node (prompt)  $p_i$  and  $p_j$  is computed via the position-agnostic Euclidean distances (Su et al., 2021):
$$
w _ {i, j} = \frac {1}{1 + \frac {1}{l ^ {2}} \sum_ {k _ {1} = 1} ^ {l} \sum_ {k _ {2} = 1} ^ {l} \left| \left| p _ {i , k _ {1}} - p _ {j , k _ {2}} \right| \right|}, \tag {1}
$$
where  $p_{i,k_1}, p_{j,k_2}$  denote the  $k_1$ -th and  $k_2$ -th vector of prompt  $p_i$  and  $p_j$ , respectively. We then adopt the min-max cut strategy (Ding et al., 2001) to partition the graph  $\mathcal{G}$  into several subgraphs representing different prompt clusters  $\mathcal{C} = \{\mathcal{C}_1, \dots, \mathcal{C}_m\}$ , where  $m$  is the total number of clusters. When transferring the source prompts, it will be better to identify the suitable prompt cluster and select the most relevant source prompt. By contrast, previous works considered each source prompt equally and ignore the differences between different tasks (Vu et al., 2021; Su et al., 2021).
Multi-Key Memory Network. With source prompts encoding task-related knowledge, the second motivation is to share them with every target generation task. To facilitate the prompt transfer from source tasks to target tasks, we build a multi-key memory network to store these clustered prompts. Specifically, for a source prompt  $p_t$  from the prompt cluster  $\mathcal{C}_z$ , i.e.,  $p_t \in \mathcal{C}_z$ , it is associated with a learnable cluster key  $k_z^c$  and a learnable prompt key  $k_t^p$ , as follows:
$$
\tilde {\mathcal {P}} = \left\{\mathcal {C} _ {z}: \left\langle \boldsymbol {k} _ {z} ^ {c}, \boldsymbol {k} _ {t} ^ {p}, p _ {t} \right\rangle \right\} _ {z = 1} ^ {m}, \tag {2}
$$
where  $\pmb{k}_z^c,\pmb{k}_t^p\in \mathbb{R}^d$  , and  $d$  is the key embedding size. In our memory network, these learned source prompts serve as representation bases, i.e., value vectors, which can be transferred to target generation tasks through key-value prompt finding.
4.2 Transferring Instance Adaptive Prompts
Previous works (Li and Liang, 2021; Vu et al., 2021) usually consider only the task information but ignore the specific input data when deriving prompts. However, for a single task, even a welllearned prompt may not be suitable for all the data instances (Scao and Rush, 2021), and thus it is non-trivial to design effective transferring strategy considering both task- and instance-level characteristics. In our model, we design an adaptive attention mechanism to incorporate the instance feature for constructing the target prompt.
Adaptive Attention Mechanism. Specifically, for an instance  $(x,y)$  of the target task  $\mathcal{T}$ , we use both task-level and instance-level queries to adaptively lookup and select the source prompts for transferring the previously learned task-related knowledge. The task-level query aims to select the overall information related to the specific target task, which is defined as a learnable task query vector  $q^{task} \in \mathbb{R}^d$ . However, the source prompts in the pool are diverse but limited, thus the task-level prompt may not well adapt to all the data instances of the target generation task. Therefore, we design an instance-level query to learn the target prompt by attending to the highly relevant source prompts to help improve the model performance in specific instances. The instance-level query is computed as the input encoding  $q^{ins} \in \mathbb{R}^d$  through a frozen PLM such as BERT (Devlin et al., 2019):
$$
\boldsymbol {q} ^ {\text {i n s}} = \operatorname {A v e r a g e} (\operatorname {B E R T} (x)), \tag {3}
$$
where we average the top-layer representations of every input tokens encoded by BERT.
For a source prompt  $p_t \in \mathcal{C}_z$ , we use  $q^{task}$  and  $q^{ins}$  to lookup its corresponding cluster key and source key respectively, following multi-head attention (Vaswani et al., 2017). Thus, the final matching score between the instance  $x$  and prompt  $p_t$  is calculated as:
$$
s _ {t} = \operatorname {s o f t m a x} \left(\lambda \cdot \boldsymbol {q} ^ {\text {t a s k} \top} \boldsymbol {k} _ {z} ^ {c} + (1 - \lambda) \cdot \boldsymbol {q} ^ {\text {i n s} \top} \boldsymbol {k} _ {t} ^ {p}\right), \tag {4}
$$
where  $\lambda$  is a hyper-parameter. Finally, according to the weight score, the selected source prompt is computed as:  $\tilde{p} = \sum_{t=1}^{T} s_t \cdot p_t$ .
Compared to other prompt-based transfer learning methods that used only a fixed prompt for a new task (Vu et al., 2021; Li and Liang, 2021), our adaptive attention mechanism is able to effectively learn the most suitable prompt representation from source prompts for a specific data instance. Such a mechanism makes our model more flexible to transfer to new text generation tasks.
Prompt-based Text Generation. Based on the above adaptive attention mechanism, we retrieve the prompt  $\tilde{p}$  encoding the most useful and relevant knowledge to help the model perform the specific generation instances. As described in Section 3.1, we prepend the prompt  $\tilde{p}$  to the input embedding of  $x$ , which then flows through a generative PLM such as BART (Lewis et al., 2020) for generating text. The generative PLM is optimized via maximum likelihood estimation (MLE) as:
$$
\mathcal {L} _ {\mathrm {M L E}} (\theta) = \mathbb {E} _ {(x, y) \sim (\mathcal {X}, \mathcal {Y})} \log \Pr (y | [ \tilde {p}; x ]). \tag {5}
$$
During the learning process of the target task, the retrieved prompt  $\tilde{p}$  is adaptive to different instances and is frozen because it encodes the previously learned task-related knowledge.
4.3 Model Discussion
For prompt-based transfer learning in text generation, the key point lies in how to effectively transfer or reuse existing prompts (encoding task-specific knowledge) for new generation tasks considering both task- and instance-level characteristics.
To achieve this goal, we first learn a set of source prompts encoding task-specific knowledge from a number of representative source text generation tasks (Section 4.1). These source prompts serve as representation bases, i.e., value vectors in the multi-key memory network. Moreover, we design an adaptive attention mechanism considering both task- and instance-level information for constructing the target prompt (Section 4.2). Each data instance from a new generation task can learn a specific prompt by attending to the most highly relevant source prompts.
Compared with typical transfer learning methods, our model utilizes a lightweight technique, i.e., prompting, to learn task-specific knowledge from source tasks. Our pretrained source prompts can help PLMs perform more effective and useful knowledge transfer.
In this section, we first set up the experiments, and then report the results and analysis.
Datasets. We select 14 public datasets divided into three types of text generation tasks: i) compression to express salient information in concise text including summarization (CNN/Daily Mail (See et al., 2017), XSum (Narayan et al., 2018), MSNews (Liu et al., 2021a), Multi-News (Fabbri et al., 2019), NEWSROOM (Grusky et al., 2018)) and question generation (SQuAD (Rajpurkar et al., 2016)); ii) transduction to transform text while preserving content precisely including style transfer (Wiki Neutrality (Pant et al., 2020)) and text paraphrase (Quora (Wang et al., 2017)); and iii) creation to produce new content from input context including dialog (PersonaChat (Zhang et al., 2018), TopicalChat (Gopalakrishnan et al., 2019), DailyDialog (Li et al., 2017), DSTC7-AVSD (Alamri et al., 2019), MultiWOZ (Budzianowski et al., 2018)) and story generation (WritingPrompts (Fan et al., 2018)). Dataset statistics are in Appendix A.
Baselines. We compare our proposed PTG to the following baselines:
- GPT-2 (Radford et al., 2019), BART (Lewis et al., 2020), and T5 (Raffel et al., 2020): These are three representative PLMs for text generation, where all pretrained parameters are fine-tuned on each target task dataset separately. We adopt the LARGE version of these PLMs.
- PRIXTUNING (Li and Liang, 2021): It is the recent state-of-the-art prompt-based PLM for text generation by concatenating a sequence of vectors and the input, which keeps PLM parameters frozen but optimizes a set of continuous prefix vectors.
- SPoT (Vu et al., 2021): It also adopts a prompt-based transfer learning method which first trains a prompt on source tasks and then uses the resulting prompt to initialize the prompt for a target task.
- MULTI-TASK MODELTUNING: This strong multi-task baseline first fine-tunes BART on the same source tasks used for PTG and then fine-tunes on each target task dataset individually.
We conduct all methods in the same setting to obtain their results without special tricks such as label smoothing. Compared with other baselines, our model is extremely lightweight, i.e., when solving target generation tasks, we freeze the transferred
Target Task	SUMMARIZATION (CNN/Daily Mail)	DIALOG (PersonaChat)
#Metrics	ROUGE-1	ROUGE-2	ROUGE-L	BLEU-1	BLEU-2	Distinct-1	Distinct-2
GPT-2LARGE	30.30	7.66	26.40	36.07	22.64	1.57	8.54
BARTLARGE	41.37	21.16	38.36	40.48	26.48	1.42	7.60
T5LARGE	40.47	20.30	37.57	42.23	27.36	1.39	7.63
PREFIXTUNING	41.79	20.69	38.50	41.87	27.28	1.33	7.20
SPOT	39.38	17.24	36.71	39.74	26.52	1.33	7.81
MT MODELTUNING	41.43	21.17	38.40	40.47	26.49	1.45	7.83
PTG	42.40	21.35	39.14	45.46	29.52	1.46	8.34