NUTSHELL : A Dataset for Abstract Generation from Scientific Talks (2025)

Abstract: Scientific communication is receiving increasing attention in natural language processing, especially to help researches access, summarize, and generate content. One emerging application in this area is Speech-to-Abstract Generation (SAG), which aims to automatically generate abstracts from recorded scientific presentations. SAG enables researchers to efficiently engage with conference talks, but progress has been limited by a lack of large-scale datasets. To address this gap, we introduce NUTSHELL, a novel multimodal dataset of *ACL conference talks paired with their corresponding abstracts. We establish strong baselines for SAG and evaluate the quality of generated abstracts using both automatic metrics and human judgments. Our results highlight the challenges of SAG and demonstrate the benefits of training on NUTSHELL. By releasing NUTSHELL under an open license (CC-BY 4.0), we aim to advance research in SAG and foster the development of improved models and evaluation methods.

## introduction abstracts are essential in scientific communication, allowing researchers to quickly grasp the key contributions of a paper. with the ever-growing number of publications, abstracts help researchers stay informed without reading full papers. beyond their practical utility, abstracts also pose a significant challenge for natural language generation models: abstracts are a specialized form of summarization that not only condenses content but also promotes the work, often using domain-specific terminology and structured language. medical domain @xcite @xcite , generating abstracts from citations @xcite , summarizing specific paper sections @xcite , and leveraging knowledge graphs for abstract generation @xcite . with the growing availability of recorded conference talks, a new challenge emerges: generating abstracts from spoken content or speech-to-abstract generation (sag). the abstracts offer researchers a quick way to assess relevant talks without watching entire recordings. additionally, as conferences include more virtual content, automatically generated summaries enable efficient engagement with recorded talks @xcite . while speech summarization has been explored in domains like news @xcite , youtube videos @xcite , and meeting minutes @xcite , large-scale datasets for scientific talk abstract generation are lacking. existing work @xcite aligns transcripts with the corresponding papers and extracts overlapping textual segments as summaries. however, these segments are drawn from the paper rather than the talk itself, failing to capture the distinct contributions, framing, and nuances conveyed in spoken presentations. other studies have focused on summarizing ted talks @xcite @xcite , which target a broad audience and prioritize inspiration and engagement over technical content. to bridge this gap, we introduce nutshell a new multimodal dataset for abstract generation from scientific talks. built from recorded presentations of *acl conferences, the dataset pairs abstracts with their corresponding spoken content and video, offering a valuable resource for future research. to validate the quality of the abstracts as concise and well-structured summaries of the talks -i.e., capturing the essence of the presentations in a nutshell -we performed a human assessment, which confirmed their effectiveness and suitability for the sag task. to establish baselines for sag using our dataset, we evaluate three model types: (1) a cascaded model combining automatic speech recognition (asr) with text-based summarization, (2) a stateof-the-art speech-language model (speechllm) without fine-tuning, and (3) a speechllm finetuned on our dataset. our contributions are three-fold: ## the nutshell dataset in this section, we introduce the new nutshell resource. we chose to build our corpus upon the the acl anthology foot_1 since it provides a rich collection of multimodal resources (talks and abstracts) and open-access licensing. starting from 2017, a significant number of papers published in the main *acl conferences (acl, emnlp, and naacl) include a video of the presentation, all released under the creative commons attribution 4.0 license. this makes *acl an ideal resource for building a multimodal dataset for the sag task. in the following, we present a feasibility assessment of sag through human evaluation ( §2.1). then, we describe the collection process performed to create nutshell, together with the final dataset statistics ( §2.2). ## analysis to demonstrate the quality and usability of our corpus, as well as provide baselines for future works, we develop and evaluate four different models using both automatic metrics and human evaluation. ## conclusion in this work, we introduce nutshell, a novel dataset for sag from recorded *acl conference talks. by releasing this dataset under an open license, we hope to foster further advancements in sag research and encourage the development of more effective models and evaluation techniques. future work could explore the integration of the video content provided in the corpus, offering an additional modality for enriching the generation process and further improving abstract quality.