Detecting Extraneous Content in Podcasts (2021)

Abstract: Podcast episodes often contain material extraneous to the main content, such as advertisements, interleaved within the audio and the written descriptions. We present classifiers that leverage both textual and listening patterns in order to detect such content in podcast descriptions and audio transcripts. We demonstrate that our models are effective by evaluating them on the downstream task of podcast summarization and show that we can substantively improve ROUGE scores and reduce the extraneous content generated in the summaries.

## introduction podcasts are a rich source of data for speech and natural language processing. we consider two types of textual information associated with a podcast episode: the short description written by the podcast creator, and the transcript of its audio, both of which may contain content that is not directly related to the main themes of the podcasts. such content may come in the form of sponsor advertisements, promotions of other podcasts, or mentions of the speakers' websites and products. while such content is tightly integrated into the user experience and monetization, it is a source of noise for many natural language processing and information retrieval applications which utilize podcast data. for example ( in the description. information extraction models looking for entities may mistakenly retrieve sponsor names from advertisements. in this paper, we introduce the problem of detecting extraneous content (which we sometimes shorthand as ec) in episode descriptions and audio transcripts. we produce an annotated corpus by taking advantage of podcast listening data, construct models to detect extraneous content, and evaluate our models for accuracy of detection, as well as for the downstream task of summarizing podcast transcripts. we also discuss some of challenges that arise while annotating and classifying extraneous content in this domain. ## previous work a related, well-studied problem is boilerplate detection on web pages, mainly involving the detection of templates, navigational elements, and advertisements @xcite . such models tend to rely on the specific structure of web page boilerplate markup and characteristics. there has also been work on detecting promotional content on wikipedia @xcite . there are primarily two lines of work in advertisement detection and discovery in multimedia. one computes acoustic features over the entire audio to discriminate between the segments of con-tent and the segments of advertisements @xcite @xcite . the other fuses multimodal features such as visual cues to segment ad clips from televised and online videos @xcite @xcite . our work is closely related to @xcite , who analyze consumer engagement on audio advertisements when compared to topical content, as we utilize the user engagement signals to predict extraneous content segments. ## datasets and annotation to create an annotated dataset, we selected a random set of podcast episodes out of the spotify podcast dataset, a corpus of 105,360 episodes @xcite . each episode in the dataset has an automatically generated transcript and a short text description of the episode written by the podcast creators. we annotate both sources, creating ground-truth labels for the extraneous content detection task, using the open source software doccano @xcite . annotators were instructed to select spans that correspond to extraneous content, which we defined as ads, social media links, promotions of other podcasts, and show notes that are not directly related to the episode. respecting sentence boundaries was encouraged, but not required, to allow for cases where extraneous content starts or ends mid-sentence. ## application to podcast summarization we address the problem of automatically generating episode descriptions from podcast transcripts, a task similar to abstractive summarization. within this problem, we evaluate the downstream effect of removing extraneous sentences from the training and/or test data. alternatives to removal (such as using the model's predictions as auxiliary inputs in the downstream system) are left for future work. we experiment with two supervised abstractive summarization models both built using bart @xcite . the first experiment uses a model pretrained for summarization on the cnn/dailymail dataset 4 (hermann et al., 2015). this model (which we refer to as bart-cnn) evaluates the extent to which extraneous text in the transcripts contribute to the presence of extraneous content in the generated descriptions. in the second experiment, we fine-tune bart-cnn on our corpus of podcasts, similar to the work of zheng et al. (2020), using the episode transcripts as inputs and descriptions as targets. we refer to this model as bart-podcasts; with it, we can evaluate the effects of ec as realistic noise which may contaminate the training data of summarization models. ## conclusion we introduced the problem of detecting extraneous content in podcast descriptions and transcripts, presented models that leverage textual and listener data, and evaluated them on a downstream summarization task. we consider our models to be baselines for a new problem with several opportunities for future work. although we used two separate models for the descriptions and transcripts with the view that the language patterns are different, a joint model or shared components may be able to take advantage of some of the common vocabulary. one could leverage the 'boilerplate' nature of some types of extraneous content like ads by detecting repeated sentences and phrases across the corpus. a language model that is robust to noisy speech transcripts @xcite may improve accuracy on podcast transcripts. given that extraneous content may appear as pre-recorded audio, or with a different speaking pitch and cadence, acoustic features alongside textual ones may be helpful. thomas wolf, lysandre debut, victor sanh, julien chaumond, clement delangue, anthony moi, pierric cistac, tim rault, rémi louf, morgan funtowicz, joe davison, sam shleifer, patrick von platen, clara ma, yacine jernite, julien plu, canwen xu, teven le scao, sylvain gugger, mariama drame, quentin lhoest, and alexander m. rush. 2020. transformers: state-of-the-art natural language processing. in proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38-45, online. association for computational linguistics. chujie zheng, harry jiannan wang, kunpeng zhang, and ling fan. 2020. a baseline analysis for podcast abstractive summarization.