Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer in Prompt Tuning (2024)

Abstract: In real-world scenarios, labeled samples for dialogue summarization are usually limited (i.e., few-shot) due to high annotation costs for high-quality dialogue summaries. To efficiently learn from few-shot samples, previous works have utilized massive annotated data from other downstream tasks and then performed prompt transfer in prompt tuning so as to enable cross-task knowledge transfer. However, existing general-purpose prompt transfer techniques lack consideration for dialogue-specific information. In this paper, we focus on improving the prompt transfer from dialogue state tracking to dialogue summarization and propose Skeleton-Assisted Prompt Transfer (SAPT), which leverages skeleton generation as extra supervision that functions as a medium connecting the distinct source and target task and resulting in the model’s better consumption of dialogue state information. To automatically extract dialogue skeletons as supervised training data for skeleton generation, we design a novel approach with perturbation-based probes requiring neither annotation effort nor domain knowledge. Training the model on such skeletons can also help preserve model capability during prompt transfer. Our method significantly outperforms existing baselines. In-depth analyses demonstrate the effectiveness of our method in facilitating cross-task knowledge transfer in few-shot dialogue summarization.

## introduction automatic text summarization @xcite is one of the most important and challenging problems in nlp. among the different forms the text to be summarized could take, dialogues have been serving as a critical part of human-human and humanmachine interaction. there has been significant progress made in dialogue summarization these ## method: skeleton-assisted prompt transfer (sapt) the existing non-task-specific general-purpose prompt transfer technique @xcite relies solely on the source and target task supervision to train the soft prompt, without the help of any intermediate task-specific medium. even though dst and dialogue summarization are closely related tasks, the intrinsic domain shift between them should still not be ignored. therefore, having an intermediate task-specific medium should conceivably be helpful for better connecting the distinct source and target task. such a medium can take algorithm 1 skeleton extraction with perturbation-based probes input: a collection of dialogues x containing n dialogues: @xmath0, x 2 , . . . , x n }, where dialogue x i contains p i dialogue turns: x @xmath1, t i2 , . . . , t ip i ], 1 ≤ @xmath2; a trained dst model lm dst ; a textual similarity metric sim(•, •) (higher means more similar). output: also, as the model capability of processing source task data is closely associated with the knowledge it has gained during the source task pretraining, it needs to be effectively preserved during the prompt transfer to facilitate the target task. nonetheless, the capability per se is admittedly a bit abstract and thus hard to concretely model in practice. inspired by recent advances in interpretable nlp, we argue that the model sensitivity to perturbation-based probes should arguably be a concretization of model capability @xcite . thus, maintaining model sensitivity during the prompt transfer should logically benefit the preservation of model capability. and notably, the aforementioned extra task supervision can ex-actly create conditions for (source-task) modelsensitivity information to be explicitly passed to the (target-task) model during the prompt transfer. to these ends, we propose skeleton-assisted prompt transfer (sapt), a dialogue-specific prompt transfer technique. sapt provides the model with extra supervision during its prompt transfer by training it to perform skeleton generation along the way (detailed in subsection 3.1). this extra supervision (i.e. skeleton generation) is separately incorporated into both the source and target task supervision, and thus can essentially function as an intermediate task-specific medium (because of the increased overlap between the updated source and target task) that is beneficial for the cross-task knowledge transfer. to get the supervised training data for skeleton generation, we design a novel automatic skeleton extraction approach that requires neither annotation effort nor domain knowledge (detailed in subsection 3.2). specifically, we observe the model's output variation to perturbation-based probes and extract the dialogue turns to which the model displays the highest sensitivity as skeletons. training the model on such skeletons can also help preserve model capability during prompt transfer. this is because those skeletons (extracted with perturbationbased probes) embody the model sensitivity to perturbation-based probes which is a concretization of model capability. on the whole, sapt creates an intermediate task-specific medium using skeleton generation as extra supervision ( §3.1), and preserves model capability during prompt transfer by training the model on the skeletons extracted with perturbation-based probes ( §3.2). as a result, the distinct source and target task is able to be better connected because they have got semantically closer to each other, and the target task is able to be facilitated because the model has been discouraged from forgetting the knowledge it has gained during the source task pretraining. §3.3 describes sapt's overall workflow. ## conclusion we focus on improving the prompt transfer in prompt tuning from dialogue state tracking to few-shot dialogue summarization, and propose sapt, a dialogue-specific prompt transfer technique, which uses skeleton generation as extra supervision by training the model on the dialogue skeletons extracted with perturbation-based probes. in this way, a beneficial intermediate task-specific medium is created between the source and target task, and the model capability is able to be better preserved during the prompt transfer, resulting in the model's better consumption of dialogue state information from the source task. significantly stronger empirical performance and in-depth analyses on two dialogue summarization benchmarks demonstrate the effectiveness of our method in fewshot dialogue summarization.