S tand U p4 AI : A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos (2025)

Abstract: Aiming towards improving current computational models of humor detection, we propose a new multimodal dataset of stand-up comedies, in seven languages: English, French, Spanish, Italian, Portuguese, Hungarian and Czech. Our dataset of more than 330 hours %, is at the time of writing the biggest available for this type of task, and the most diverse. The whole dataset is automatically annotated in laughter (from the audience), and the subpart left for model validation is manually annotated.% Contrary to contemporary approaches, we do not frame the task of humor detection as a binary sequence classification, but as word-level sequence labeling, in order to take into account all the context of the sequence and to capture the continuous joke tagging mechanism typically occurring in natural conversations. As par with unimodal baselines results, we propose a method for e propose a method to enhance the automatic laughter detection based on Audio Speech Recognition errors. Our code and data are available online: https://tinyurl.com/EMNLPHumourStandUpAnonym

## introduction and related works humor detection remains a challenging tasks for computer systems @xcite . yet, such mechanisms could be a massive improvement, in particular for conversational interactive systems such as chatbots and socially interactive agents. these kind of systems, which are designed to simulate a natural humanlike conversation and its structure @xcite , often struggle to identify or handle humorous attempts from the user, leading to inefficient and frustrating experiences @xcite . while different theories of humor exist, most of them have in common the idea that humor emerges when the current situation surprisingly deviates from our expectations (warren and * same supervision @xcite . generally, a joke or funny story is based on the following sequence : the setup of a joke introduces some expectations on how a story usually ends and the punchline reveals the reality and the unexpected (and funny) twist of the story @xcite . sometimes, additional funny comments called tags can be added around and after the punchline to maintain the momentum of the laughter. conversational humor often stems from unexpected deviations in content, behavior, or context, with timing and intensity being critical yet unpredictable triggers @xcite . despite theoretical models, comedians rely on live testing to refine timing, phrasing, and delivery for audience engagement @xcite , as responses depend on cultural and contextual factors. this highlights the complexity of modeling humor computationally, necessitating diverse datasets to capture its multifaceted dynamics. stand-up comedy, due to its nature aiming at recreating the spontaneity of everyday conversational humor, is a great context for studying these mechanisms and structures with computers. many previous works investigating computational techniques to process humor relied on corpus of people speaking in less natural and more conventional and standardized ways. for instance, in @xcite @xcite , the authors relied on acted data from sitcoms. the ur-funny and ted laughter @xcite datasets are composed of ted talks, which contain less outbursts of laughter and poorer language diversity than stand-up comedy. most of the work participating in the muse challenges for the automatic estimation of humor are relying on public interviews @xcite , using the passau-spontaneous football coach humour dataset @xcite . additionally, while some of the previous work explored other languages (chauhan et al., 2021), most of them are investigating english humorous content only. another early work on stand-up humor is the one of turano and strapparava (2022), which analyzes 90 scripts of 68 comedians, in english only. the closest work from ours would be the one described in kuznetsova and strapparava (2024), which proposed a 40 hours dataset in russian and english. most humor detection models in videos treat humor as a sequence classification task, identifying punchlines only at the end of a sequence @xcite @xcite . however, multiple laughs can occur within a single sentence, sometimes consecutively. to address this, we reframe the task as sequence labeling, enabling continuous prediction of audience laughter throughout the joke, rather than relying on end-only classification. in this article, we are presenting multiple contributions towards the development of humor detection models. first, we collected and annotated a dataset of stand-up comedy performance in different languages extracted from online videos. this dataset is the largest and most linguistically diverse multilingual dataset of live comedy performances. it has the ambition to be a reference dataset for any type of humor modeling tasks. second, we propose a original methodology for the task of humor de-tection by using a sequence labeling approach we adapted to automatically predict laughter during a performance. third, this led us to came up with new techniques for handling errors in automatic transcription and automatic laughter detection, validated on a manually laughter annotated test set. fourth, we present first results of sequence labeling models built on our dataset and applied to predict laughter to be used as baselines by the community. ## dataset the standup4ai dataset is composed of 3,617 standup videos in 7 languages. it contains the associated transcriptions and audience laughters that have been automatically refined, of comedians during stand-up comedy performances in various languages. to build the dataset, we first collected a specific set of videos of stand-up comedy from the internet, we then performed automatic transcription on these videos and we finally fixed some errors in the outputs by developing improved transcription and automatic laughter annotation techniques (overview in figure 1 ). ## experiments and results we conducted two types of experiments. the first validate the proposed technique to find new outbursts of laughter that were not detected using the off-the-shelf model of @xcite . the second is the sequence labeling task. ## conclusion in this article we presented the most diverse dataset of multilingual stand-up comedy performance at the date of today, standup4ai. we propose baseline results on the tasks of laughter prediction approached as a sequence labeling task, highlighting the interest of the diversity contained in our dataset. on top of this, we show the interest of a simple yet efficient technique enhancing a state-of-the-art automatic laughter detection method, that we successfully validate with manual annotations and by using it to train a humor detection model. the results highlighted the potential of our dataset for the development of computational models of humor.