Afrispeech-Dialog: A Benchmark Dataset for Spontaneous E nglish Conversations in Healthcare and Beyond (2025)

Abstract: Speech technologies are transforming interactions across various sectors, from healthcare to call centers and robots, yet their performance on African-accented conversations remains underexplored. We introduce Afrispeech-Dialog, a benchmark dataset of 50 simulated medical and non-medical African-accented English conversations, designed to evaluate automatic speech recognition (ASR) and related technologies. We assess state-of-the-art (SOTA) speaker diarization and ASR systems on long-form, accented speech, comparing their performance with native accents and discover a 10%+ performance degradation. Additionally, we explore medical conversation summarization capabilities of large language models (LLMs) to demonstrate the impact of ASR errors on downstream medical summaries, providing insights into the challenges and opportunities for speech technologies in the Global South. Our work highlights the need for more inclusive datasets to advance conversational AI in low-resource settings.

## introduction deep learning approaches have revolutionized and yielded significant performance gains across several natural language tasks @xcite , especially for high-resource languages @xcite . while conversational speech recognition has continued to make significant strides in task automation in different domains such as medical @xcite @xcite , voice assistants @xcite , call center @xcite , and robotics @xcite , much of the research in automatic speech recognition 1 authors marked with ♠ are senior authors. in anglophone african contexts, regional accents further complicate the development of asr systems resulting in poor generalization from stateof-the-art (sota) models @xcite in both general @xcite @xcite and medical @xcite contexts. in more developed countries, performant speech recognition models are particularly useful in overburdened healthcare systems, where they can help to reduce the documentation workload for overwhelmed clinicians @xcite . recent advancements in medical conversation transcription and summarization using llms have led to wide adoption of these technologies in hospitals in developed countries @xcite helping to improve clinical documentation quality and productivity @xcite . given the heavy patient loads @xcite as well as the recent integration of ai/nlp/asr systems into clinical workflows in african clinics 12 , performant ambient transcription and summarization systems are highly desired. however, the generalizability of such systems to african contexts remains underexplored @xcite . to tackle this problem, our contributions are as follows: we lay the groundwork for more inclusive asr and nlp technologies in healthcare and beyond. ## methodology figure 1 shows an overview of the dataset creation and benchmarking process, illustrating how afrispeech-dialog supports tasks like speaker diarization, asr, and summarization. below, we describe the dataset creation process and the evaluation of state-of-the-art (sota) models on these tasks to demonstrate their applicability and highlight challenges in african-accented conversational speech. ## conclusion this study highlights several challenges with accented conversational asr, especially its impact on downstream medical summarization. we benchmark speaker diarization, asr, and medical summarization, drawing attention to gaps and opportunities to improve accented conversational asr in the african context.