A kan Cinematic Emotions ( ACE ): A Multimodal Multi-party Dataset for Emotion Recognition in Movie Dialogues (2025)

Abstract: In this paper, we introduce the Akan Cinematic Emotions (AkaCE) dataset, the first multimodal emotion dialogue dataset for an African language, addressing the significant lack of resources for low-resource languages in emotion recognition research. AkaCE, developed for the Akan language, contains 385 emotion-labeled dialogues and 6162 utterances across audio, visual, and textual modalities, along with word-level prosodic prominence annotations. The presence of prosodic labels in this dataset also makes it the first prosodically annotated African language dataset. We demonstrate the quality and utility of AkaCE through experiments using state-of-the-art emotion recognition methods, establishing solid baselines for future research. We hope AkaCE inspires further work on inclusive, linguistically and culturally diverse NLP resources.

## introduction emotion recognition in conversation (erc) is a rapidly evolving subfield of natural language processing (nlp) that focuses on detecting or classifying the emotional states expressed by speakers in multi-turn conversations @xcite . unlike traditional emotion recognition tasks that aim to identify emotions from isolated text or speech snippets or speech utterances such as @xcite , erc seeks to leverage contextual cues from prior dialogue, speaker relationships, and conversational flow to infer emotional states more accurately @xcite . in recent years, erc has garnered significant attention within the nlp community, driven by its growing relevance to a range of real-world applications. notable examples include empathetic chatbot systems @xcite , call-center dialogue systems @xcite , and mental health support tools @xcite 2018). these systems rely on erc to capture the evolving emotional dynamics of conversations, enabling more contextually appropriate and emotionally aware responses. developing robust erc systems often requires multimodal data integration @xcite , which is challenging due to the need to jointly model diverse inputs like scene context, discussion topics, conversational history, and speaker personalities @xcite @xcite . however, comprehensive multimodal erc dialogue datasets remain scarce, with benchmark resources like iemocap @xcite , msp-improv @xcite , meld @xcite , and m³ed @xcite ) being notable exceptions. a major limitation of existing erc datasets is their focus on high-resource languages, particularly english (iemocap, msp-improv, meld) and chinese (m³ed). this lack of linguistic diversity hinders the development of erc systems for lowresource languages, especially in africa. to our knowledge, no multimodal erc dataset exists for any african language, despite the continent being home to approximately 3,000 of the world's 7,000 languages (leben, 2018) and 18.3% of the global population @xcite . to address this gap, we introduce the akan con-versation emotion (akace) dataset, a multimodal emotion dialogue dataset for akan, a major west african language spoken by about 20 million people @xcite . akan is the most widely spoken language in ghana, with around 80% of the population using it as a first or second language, and approximately 44% identifying as native akan speakers. it is also natively spoken in parts of ivory coast and togo. the language primarily comprises three main dialects: asante, akuapem, and fante. akace contains 385 emotion-labeled dialogues from 21 akan movies, covering diverse scenes and topics. it includes 6,162 utterances from 308 speakers (155 male, 153 female), ensuring a genderbalanced dataset. as a tonal language, akan's prosodic features are crucial for emotion recognition, so akace includes word-level prosodic prominence annotations to support research on prosody in erc. our baseline experiments validate the dataset's quality and utility for low-resource and cross-cultural emotion recognition research. our main contributions are: single-modality datasets focus exclusively on a single modality, such as text, including emorynlp (zahiri and choi, 2018), emotionlines @xcite , and dailydialog @xcite . these are useful for text-based emotion recognition but lack the multi-modal richness needed for comprehensive analysis involving vocal or visual cues. 1 our data and code are available at this github repo. multi-modal datasets combine text, audio, or video, such as cmu-mosei @xcite , afew @xcite , mec @xcite , and ch-sims @xcite . while offering richer features, they are not conversational and miss the dynamic, context-dependent expressions seen in natural dialogues. conversational multi-modal datasets integrate text, audio, and video with conversational context, such as iemocap @xcite , msp-improv @xcite , meld @xcite , and m³ed @xcite . but these datasets mainly focus on high-resource languages, leaving gaps for low-resource languages. existing datasets lack resources for low-resource languages, such as akan, and prosodic annotations critical for tonal languages. akace fills this gap by offering a multi-modal resource with prosodic labels and conversational data, enabling robust ser for akan and advancing cross-cultural ser research. ## akace dataset we construct the akace dataset by collecting and annotating dialogues from akan-language movies, with examples illustrated in figure 1 . the dataset includes transcriptions with speaker identifications, emotion labels, and word-level prosodic prominence annotations. ## experiments and analysis we conduct a series of experiments to establish baseline performance for emotion recognition on akace using unimodal and multimodal approaches. we first evaluate text, audio, and vision separately with state-of-the-art models, then explore modality combinations through feature concatenation and transformer-based fusion. these results serve as a foundation for future research on multimodal emotion recognition in akan.