M edia S um: A Large-scale Media Interview Dataset for Dialogue Summarization (2021)

Abstract: This paper introduces MediaSum, a large-scale media interview dataset consisting of 463.6K transcripts with abstractive summaries. To create this dataset, we collect interview transcripts from NPR and CNN and employ the overview and topic descriptions as summaries. Compared with existing public corpora for dialogue summarization, our dataset is an order of magnitude larger and contains complex multi-party conversations from multiple domains. We conduct statistical analysis to demonstrate the unique positional bias exhibited in the transcripts of televised and radioed interviews. We also show that MediaSum can be used in transfer learning to improve a model’s performance on other dialogue summarization tasks.

## introduction dialogue summarization can provide a succinct synopsis for conversations between two or more participants, based on human-transcribed or machine-generated transcripts. dialogue summaries are useful for participants to recap salient information in the talk and for absentees to grasp the key points. as a result, several models have been recently proposed to summarize daily conversations @xcite , meeting transcripts @xcite and customer support conversations @xcite . however, compared with the abundance of text summarization datasets, there are very few public datasets for dialogue summarization. and existing datasets are limited to their small sizes. for example, the benchmark datasets for meeting summarization, ami @xcite and icsi @xcite , only contain transcripts and abstractive summaries for 137 and 59 business meetings, respectively. while recently some larger dialogue summarization datasets have been proposed, they are either built from a narrow domain, e.g. the crd3 dataset @xcite which is built from conversations in a live-streamed show for the dungeons and dragons game, or not publicized due to privacy reasons, e.g. the didi dataset @xcite ) from customer service conversations. this lack of large-scale dialogue summarization datasets is due to a higher labeling cost compared with news articles and privacy issues with many real daily dialogues and business meetings. on the other hand, media interview transcripts and the associated summaries/topics can be a valuable source for dialogue summarization. in a broadcast interview, the host discusses various topics with one or more guests. as many interviews proceed with pre-defined topics, the accompanying summaries are of a relatively high quality. also, the wide variety of topics, different backgrounds of speakers, and the colloquial form of chat make these interviews very close to daily conversations and business meetings. therefore, we collect public interview transcripts and the associated summaries/topics from npr and cnn to build a large-scale dialogue summarization dataset, mediasum. in npr, each transcript comes with an overview of the interview, which is used as the summary in our dataset. we leverage the interview dataset @xcite to get transcripts and crawl the associated descriptions. we end up with 49.4k npr transcripts with summaries. we then collect 269.4k cnn interview transcripts from 2000 to 2020, each with a list of topic descriptions. as many cnn interviews contain multiple topics, we conduct segmentation at the boundary of commercial breaks to assign each topic to the most relevant interview segment via lexical matching. in this way, we not only obtain tran-scripts with a more concentrated topic but also enlarge the total number of instances. we end up with 414.2k cnn transcript segments with topic descriptions as summaries. thus, in total, our me-diasum dataset contains 463.6k transcripts with summaries. we show that compared to existing public dialogue summarization datasets, media-sum contains more speakers, longer conversation and is an order of magnitude larger. also, we demonstrate the unique positional bias in interview dialogues: while a televised interview often mentions keywords in the summary at the beginning of the program, a radio interview usually mentions these keywords at both the beginning and the end of the program. in experiments, we evaluate several benchmark summarization models on our dataset. we then show that after fine-tuning on mediasum, models' performance can be improved on other dialogue summarization tasks like ami, icsi and samsum, demonstrating the transfer learning capability of our dataset. ## related work due to the success of corpus-based methods, the past decade saw the emergence of many dialogue datasets on various domains @xcite . however, very few of these datasets contain corresponding summary text. as human dialogues have very different structures and language patterns from written articles, dialogue summarization models can only limitedly benefit from the largely available news summarization data @xcite . current public datasets for dialogue summarization are either very small or in a specific domain. @xcite and icsi @xcite contain 137 and 59 meeting transcripts with abstractive summaries. ami meetings are recorded in an artificial environment with actors and icsi contains meetings of a speech group. multiwoz @xcite ) is a multi-domain taskoriented dialogue dataset where the instructions have been used as summaries @xcite . all dialogues are conducted between one user and one agent on the topic of booking and inquiry. samsum @xcite hires linguists to write messenger-like daily conversations. although the dialogues are open-domain, they are not from real human conversations. crd3 (rameshkumar and bailey) contains 159 episodes from the critical role show with transcribed conversations between dungeons and dragon players. additionally, there are non-public dialogue summarization datasets in th domains of customer support @xcite and medical conversation @xcite . ## conclusion we introduce mediasum, a large-scale media interview dataset for dialogue summarization, consisting of 463.6k transcripts and summaries from npr and cnn. we conduct transcript segmentation to align topic descriptions to segments for cnn interviews. the mediasum dataset is an order of magnitude larger than existing corpora and contains complex multi-party conversations from { "id": "npr-11", "program": "day to day", "date": "2008-06-10", "url": " https://www.npr.org/templates/story/story.php?@xmath0, "title": "researchers find discriminating plants", "summary": "the 'sea rocket' shows preferential multiple domains. we also show that mediasum can be used as a dataset for transfer learning to improve a model's performance on other dialogue summarization tasks.