Towards Abstractive Grounded Summarization of Podcast Transcripts (2022)

Abstract: Podcasts have shown a recent rise in popularity. Summarization of podcasts is of practical benefit to both content providers and consumers. It helps people quickly decide whether they will listen to a podcast and/or reduces the cognitive load of content providers to write summaries. Nevertheless, podcast summarization faces significant challenges including factual inconsistencies of summaries with respect to the inputs. The problem is exacerbated by speech disfluencies and recognition errors in transcripts of spoken language. In this paper, we explore a novel abstractive summarization method to alleviate these issues. Our approach learns to produce an abstractive summary while grounding summary segments in specific regions of the transcript to allow for full inspection of summary details. We conduct a series of analyses of the proposed approach on a large podcast dataset and show that the approach can achieve promising results. Grounded summaries bring clear benefits in locating the summary and transcript segments that contain inconsistent information, and hence improve summarization quality in terms of automatic and human evaluation.

## introduction podcasts are one of the most popular forms of new media. as of today, over 155 million people listen to a podcast every week @xcite . with the growing interest, there is an increased demand for textual summaries that foretell the content of podcasts. those summaries help people decide, in a few seconds, if they will listen to a podcast or subscribe to the channel. they are helpful for users who want to find podcasts previously listened to. furthermore, they can be re-purposed for social media posts or email marketing campaigns, enabling content creators to make their podcasts accessible to a larger audience. it is desirable to generate grounded summaries from podcast transcripts, where spans of summary text are closely tethered to the original audio. figure 1 provides an example of a grounded abstractive summary. when a user clicks on a summary segment, she will be directed to an audio clip that gives further detail of the conversational context. grounded summaries give us a preview of notable podcast clips @xcite and they may further release summarization service providers from potential legal claims by directing users to the original audio. this is because, speech recognizers induce transcription errors and abstractive summarization models may hallucinate facts that are not entailed by the original @xcite , both can cause podcast summaries to contain misleading or inaccurate information. with grounded summaries, users are able to frame, interpret, and place into context any system-generated summaries, thus reducing the barriers to deploy podcast summarization technology. one may attempt to align summary text and podcast transcripts in a post-processing step to generate grounded summaries. unfortunately, hallucinations do not allow for proper alignments as they are not found in the transcripts @xcite . hierarchical attention models may seem promising for this task @xcite . however, the excessive length of the transcripts makes it difficult to produce attention distributions over the entire transcripts. recent evidence suggests that attention weights are not reliable indicators of the relative importance of inputs @xcite , thus it remains an open question whether attention can be used to find alignments between transcripts and summary segments. in this paper, we seek to generate grounded summaries from podcast transcripts by exploring an on-demand abstractive summarizer. it mimics how a human might approach a lengthy transcript -the expert would identify a portion of the transcript that is deemed most important and relevant to the existing summary, use it as a ground to produce a new piece of the summary, and that process is repeated until the summary is finished. our summarizer employs a novel regularization technique that enables it to visit portions of the transcript in chronological order, while allowing zigzags in order to produce a coherent summary. this has another implication. it implies that we may estimate what percentage of a podcast transcript is covered by the summary and thus adjust that when necessary. distinguishing our work from earlier research on extract-then-abstract methods @xcite @xcite @xcite , we require selected transcript chunks to have high salience, but also those salient content must appear at the beginning of the selected chunks, so that the corresponding audio clips can provide good jump-in points for users to start listening. our experiments are performed on a large podcast summarization dataset containing over 100,000 english podcasts @xcite . we show that our proposed grounded summarizer can perform competitively or better than the state-of-the-art methods, including the recent methods that leverage large, pretrained models @xcite as judged by automatic metrics and human evaluation. our contributions in this paper are as follows. with the rapid rise of podcasts comes the need for automatic summarization of podcast transcriptions. while comparatively understudied, recent work has shown great progress. @xcite present the spotify dataset that was adopted in trec 2020 for the podcast summarization task. 2 our participating system in trec 2020 focuses on identifying salient segments from transcripts and using them as input to an abstractive summarizer @xcite . @xcite develop classifiers to detect and eliminate extraneous marketing materials in podcasts to aid summarization. in this paper, we explore techniques that generate grounded podcast summaries where pieces of summary text are tied to short podcast clips. one of the most serious problems of neural abstractive summarization is that the summaries can contain factually incorrect information and hallucinations @xcite @xcite . without grounded summarization, users have to listen to the full episodes to find connections between details of the summaries and the original podcasts. if successful, grounded summaries will benefit a number of summarization tasks where the input involves lengthy transcripts, including meetings @xcite @xcite , medical conversations @xcite , interviews @xcite , livestreams @xcite and more. an extract-then-abstract strategy could be used to produce grounded abstractive summaries @xcite @xcite @xcite . most of these approaches are tailored to written documents, e.g., news, wikipedia, and scholarly articles. they extract sentences from the documents and use them as input to an abstractive summarization model to produce a summary. nevertheless, transcripts of spoken language lack essential document structure such as sentence, paragraph and section boundaries, making it unclear how these approaches will perform on podcasts. attention provides another mechanism for aligning the summary and transcript segments. the use of sparse attention allows a summarization model to potentially scale to longer documents @xcite @xcite . hierarchical transformer encodes multiple paragraphs in a hierarchical manner to allow them to exchange information @xcite @xcite . however, it is shown that attention weights are not reliable indicators of the relative importance of inputs, as alternative attention distributions would have yielded similar results @xcite . our approach in this paper is to better align summary segments with chunks of the transcripts to allow easy tracing of inconsistent information. it features a generator that writes a summary from beginning to end, and a savvy selector that knows when to switch to a new transcript chunk and where to switch to. differing from pg networks @xcite and retrieval-augmented generation @xcite , our selector places heavy emphasis on modeling and selection of tran-script chunks. a desirable chunk is expected to be about 2 minutes long and places important information at the beginning to enable easy user verification. in the following section, we present details of the model implementation. ## our approach a major challenge facing podcast summarization is the dramatic length difference between source and target sequences. at a speaking rate of 122 words per minute for spontaneous speech @xcite , the full transcript of a 1-hour long episode contains roughly 7,000 words and that of a 1.5-hour long episode could reach 10,000 words. in contrast, a podcast summary is short, containing on average 61 words according to @xcite . the ratio of their lengths could reach as high as 100-to-1, and this motivates our study of abstractive grounded summarization where summary segments are grounded to selected chunks of transcripts as a way of combating the inevitable errors that occur in podcast summarization. let x be the sequence of tokens in the source transcript and y be the sequence of tokens in the summary. these tokens share the same vocabulary v. we use x c to denote a chunk of the transcript, and c gives the indices of tokens that belong to the chunk. the full transcript can be decomposed into a sequence of chunks, denoted by {c 1 , • • • , c m }. the chunks may have varying sizes and overlap with each other; they are the grounds for generating a podcast summary. our assumption is twofold. firstly, we assume a summary segment is produced by conditioning on the previously generated tokens (@xmath0. secondly, there exists a function g(x, @xmath1. ( 1 )) that determines the most appropriate grounding chunk for generating all tokens of the segment. particularly, when the entire transcript is treated as a single chunk, it reduces to the standard conditional generation model p θ (y j |@xmath2, x). thus, the crucial point is a coarse segmentation of the source transcript and an alignment between the transcript chunks and summary segments. in this work we use a sliding window to produce transcript chunks, with window size w and stride size s. 3 the sizes can be measured in terms of tokens. e.g., @xmath3. the rationale for using overlapping chunks is to find those that serve both as grounds for summary generation and good jump-in points for user verification. the sizes can also be measured by the number of sentences. e.g., @xmath4, non-overlapping chunks. in spoken language, a series of consecutive short sentences often indicates the content is relatively unimportant @xcite . given a summary segment y, we designate x c as a grounding chunk if it attains the highest score s(x c , y) (eq. ( 2 )). this position-biased coverage score favors the transcript chunk that covers summary bigrams and puts summary content at the beginning to aid humans in performing content verification. it measures the percentage of unique summary bigrams b( y) covered by a chunk x c . particularly, i [b @xmath5. each bigram b k has an associated weight w k (eq. ( 3 )). if it appears in the first position of x c (pos @xmath6, it receives a weight of one. otherwise, the weight is decayed according to the relative position of the bigram's first occurrence in the chunk (pos k ) and γ is a coefficient for the decay. 4 we proceed by training a neural encoder-decoder model to generate an abstractive summary from the grounding transcript chunks. each segment of the summary (= sentence) 5 is generated conditioned on its grounding chunk x c and all the previously generated tokens @xmath7. the process starts from the first 3 discourse segmentation is beyond the scope of this work. there is little to no data available to build a discourse segmentation tool and little existing work on discourse analysis of podcasts. we refer the reader to @xcite for recent advances in discourse processing research. 4 if a summary segment cannot be mapped to a chunk using eqs. (2-3), we perform the following: y is assigned to the first chunk c1 if it is the first segment of the summary. otherwise, y is assigned to the same chunk as the previous summary segment to improve coherence. we require xc and y to have a minimum of four shared bigrams (stopwords-only bigrams are excluded). future work may consider aligning transcripts and summaries based on propositions @xcite . 5 we use sentences as summary segments; other sentencelike segments are possible in future work. transcript decode summary transcript decode summary encode e n c o d e en co de encode text selected but not used text too long to fit into gpu memory text is selected but not used in the summary due to strong position bias transcript chunks are progressively exposed to the summarizer to consolidate information a b switch switch figure 2: strong position bias can cause the abstractor to use only content at the beginning of the input to generate a summary. by exposing the chunks progressively, our approach makes use of this characteristic to consolidate information from multiple transcript chunks. there is a notable difference between our approach and most extract-then-abstract approaches that select important sentences from the document and provide them to the abstractor all-at-once. as illustrated in figure 2 , strong position bias causes the abstractor to use only content at the beginning of the input to generate a summary. by exposing the chunks progressively, our approach naturally makes use of this characteristic to consolidate information from multiple source chunks. it reduces the amount of computation necessary to train the encoder-decoder model, as only selected transcript chunks are encoded which is equal to the number of summary segments. moreover, it is possible to encourage the summary to have a good coverage of the source content by specifying a minimal set of grounding chunks to be used for generation. ## results and analysis experimental settings. our encoder-decoder model uses bart-large as the base model before fine-tuning it on the podcast dataset. we use the adamw @xcite optimizer, where the momentum parameters are set to 0.9 and 0.999. the regularizing coefficient α is tuned on the validation set in the range of {0, 0.01, 0.1, 1}. for summary decoding, we use beam search with a beam size @xmath8. ## conclusion in this paper, we investigate podcast summarization to produce textual summaries for podcast episodes that help listeners to understand why they might want to play those podcasts. we present a new kind of podcast summary where spans of summary text are tethered to the original audio to allow users to interpret system-generated abstracts in context. experiments on a benchmark dataset demonstrates the utility of our proposed approach.