Improving Abstractive Dialogue Summarization with Speaker-Aware Supervised Contrastive Learning (2022)

Abstract: Pre-trained models have brought remarkable success on the text summarization task. For dialogue summarization, the subdomain of text summarization, utterances are concatenated to flat text before being processed. As a result, existing summarization systems based on pre-trained models are unable to recognize the unique format of the speaker-utterance pair well in the dialogue. To investigate this issue, we conduct probing tests and manual analysis, and find that the powerful pre-trained model can not identify different speakers well in the conversation, which leads to various factual errors. Moreover, we propose three speaker-aware supervised contrastive learning (SCL) tasks: Token-level SCL, Turn-level SCL, and Global-level SCL. Comprehensive experiments demonstrate that our methods achieve significant performance improvement on two mainstream dialogue summarization datasets. According to detailed human evaluations, pre-trained models equipped with SCL tasks effectively generate summaries with better factual consistency.

## introduction dialogue summarization aims to condense the essential information in the dialogue into a brief text. compared with text summarization, the conversations are semi-structured data and contain multiple participants who shall be distinguished @xcite . furthermore, dialogues are characterized by informal language, coreference, and repetition @xcite . all of these bring new challenges to the existing text summarization methods. although pre-trained models have achieved great success in natural language processing especially text summarization @xcite @xcite , how * ## method 2.1 probing test to investigate how well pre-trained seq2seq models can distinguish speakers, we conduct a simple probing experiment on samsum, a widely-used dialogue summarization corpus. concretely, we first encode the integral dialogue text with the bart @xcite encoder and randomly sample k tokens to obtain their hidden states. then, in pairs, we aggregate and feed these hidden states into mlp to determine whether they are from the same speaker. to investigate the compatibility of the pre-trained seq2seq model with flat dialogue text, we freeze the parameters of the bart encoder and solely fine-tune the mlp classification layers during training stage. then we evaluate the classification accuracy on the test set. intuitively, the binary classification task is easy, but the accuracy is only 58.1% for vanilla bart. even after fine-tuning bart with the summarization task (parameters of bart are not frozen) on samsum before the probing test , the accuracy is still only 60.2% foot_0 . the result indicates that pretrained seq2seq models can not identify speakers well from flat dialogue text, and simply fine-tuning with the dialogue summarizing task is unhelpful. we need more explicit methods to help models understand flat dialogue text. ## experiment in this section, we conduct experiments and human evaluations on the popular datasets samsum and ami. more descriptions of the datasets and the implementation details can be found in the appendix. ## conclusion in this paper, we focus on the speaker identification problem in the dialogue summarization task. through the probing test and manual analysis, we find that the existing pre-trained model can not identify different speakers well in the conversation, leading to factual errors. therefore, we propose three speaker-aware scl tasks to address this problem. experimental results and human evaluations illustrate the effectiveness of our methods.