From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions (2024)

Abstract: Text segmentation is a fundamental task in natural language processing, where documents are split into contiguous sections. However, prior research in this area has been constrained by limited datasets, which are either small in scale, synthesized, or only contain well-structured documents. In this paper, we address these limitations by introducing a novel benchmark YTSeg focusing on spoken content that is inherently more unstructured and both topically and structurally diverse. As part of this work, we introduce an efficient hierarchical segmentation model MiniSeg, that outperforms state-of-the-art baselines. Lastly, we expand the notion of text segmentation to a more practical “smart chaptering” task that involves the segmentation of unstructured content, the generation of meaningful segment titles, and a potential real-time application of the models.

## ytseg dataset as part of this work, we introduce a new benchmark, ytseg, to evaluate text segmentation systems on less structured and more diverse content than previous benchmarks aimed for. the dataset consists of 19,299 english youtube videos with their transcripts and chapters. an example of a youtube video organized into chapters can be found in the screenshot provided in figure a1 . we processed the data to adapt it for the text segmentation task, aligning the sentences in the transcription with video chapters. in addition, we release yt-seg[titles] for the training and evaluation of generative models predicting the chapter titles. by including video and audio data, the benchmark also paves the way for multi-modal approaches. the dataset, along with the instructions and scripts, is available online and released under a cc by-nc-sa 4.0 license foot_0 . ## methodology in the following, we present the models designed for the text segmentation and title generation task, which are used in our experiments and applied to the newly introduced benchmark. we also elaborate on the modifications we have made to adapt them for online implementation. sentence encoder document encoder mean pooling t1 sentence tokens sentence representations segment boundaries t2 t3 t4 t5 tk ... o1 s1 o2 o3 o4 o5 ok ... s1 s1 s2 s3 s4 s5 sl ... ... 0 0 0 0 0 1 figure 2 : the hierarchical architecture of the segmentation model consists of a sentence encoder and a document encoder returning the binary segment boundaries. ## related work a number of benchmarks have been proposed for the text segmentation task. however, the majority are either small in size or synthetically constructed, often by concatenating documents or sections of documents @xcite @xcite . larger-scale benchmarks are scarce and limited to a narrow type of documents such as wikipedia articles @xcite or specific domains like news @xcite foot_4 . equally, these datasets have in common that they only encompass documents that are structured in nature. in contrast, research on spoken, conversational, or generally unstructured or semi-structured content is still in its infancy. @xcite and @xcite separately proposed segmentation for lecture video transcripts. although these works are related to our task, they are only confined to a single domain (lecture videos), while segments are artificially constructed based on the presentation slides. in the context of conversational content, @xcite constructed a dataset by joining excerpts of different chat conversations, while @xcite provided a small set of meeting transcriptions segmented by topic shifts. regarding title generation, this study focuses on generating section headings and chapter titles for video transcripts. while research exists in title generation across various domains, including news headlines @xcite @xcite , product titles @xcite , video titles @xcite , stackoverflow posts (liu et al., configuring cloud operations on google cloud -google cloud tech (xiaagef1qvi) intro intro introduction & agenda intro how to know what's going on in the cloud how the operations components play together operational use cases exploring the products of interest for the operations team google cloud operations suite site reliability engineering google sre sre practices customer success story: krikey customer success story: krikey customer story -krikey wrap-up wrap up wrap up & additional resources generated (no context) generated (previous titles) reference (a) an exemplary output showing duplicate section titles. 8 email etiquette tips -how to write better emails at work -harvard business review (1xctnf7c74s) intro intro why bother with email etiquette? step 1: have a call to action if appropriate 1. have a call to action when appropriate include cta in subject line stick with one email thread for the same topic 2. stick with one email thread for the same topic one email thread per topic tip #3 3. explain why you added in or took out recipients manage recipients tip #5: summarize the sender's main points 4. include your main point first followed by context summarize in your reply tip #2: include the context 5. summarize the sender's main points in your reply start with the main point 6. hyperlink whatever possible 6. hyperlink whatever possible hyperlink whenever possible change default setting to reply instead of reply all 7. change default setting to reply instead of reply all change default setting to "reply" (not "reply all") change undo send option to 30 seconds 8. change undo send option to 30 seconds change undo send options outro outro outro generated (no context) generated (previous titles) reference (b) an exemplary output showing inconsistent numbering and formatting. 2022a; @xcite , and pull requests @xcite , our work addresses the unique challenges of video content structuring. one distinctive aspect lies in ensuring that all section headings of one document not only serve as informative signposts but also maintain a coherent and seamless flow between them. the closest work to ours has been conducted by @xcite , whose proposed model generates hierarchical outlines for wikipedia documents. ## conclusion in this work, we present a novel benchmark for smart chaptering. the task aims to segment unstructured content, in particular speech, conversations, and transcriptions, in a linear sequence of chapters and provides each chapter with a title. we think this benchmark is a valuable addition to the text segmentation landscape as larger-scale, non-synthetic benchmarks are scarce, and previous research focused primarily on well-structured, homogeneous documents. as part of this, we propose an efficient and state-of-the-art hierarchical segmentation model and a corresponding title-generating model, both of which have also been architected to work online. by combining our proposed segmentation and title generation models, various practical applications are conceivable. for example, content creators, podcasters, and educators could use it to structure their content for their audience. we see our work also as a stepping stone to support even more unstructured content and speech in a broader scope, such as meetings.