A fri S peech-200: Pan- A frican Accented Speech Dataset for Clinical and General Domain ASR (2023)

Abstract: Africa has a very poor doctor-to-patient ratio. At very busy clinics, doctors could see 30+ patients per day—a heavy patient burden compared with developed countries—but productivity tools such as clinical automatic speech recognition (ASR) are lacking for these overworked clinicians. However, clinical ASR is mature, even ubiquitous, in developed nations, and clinician-reported performance of commercial clinical ASR systems is generally satisfactory. Furthermore, the recent performance of general domain ASR is approaching human accuracy. However, several gaps exist. Several publications have highlighted racial bias with speech-to-text algorithms and performance on minority accents lags significantly. To our knowledge, there is no publicly available research or benchmark on accented African clinical ASR, and speech data is non-existent for the majority of African accents. We release AfriSpeech, 200hrs of Pan-African English speech, 67,577 clips from 2,463 unique speakers across 120 indigenous accents from 13 countries for clinical and general domain ASR, a benchmark test set, with publicly available pre-trained models with SOTA performance on the AfriSpeech benchmark.

## introduction the african continent and the nearby islands constitute one-fourth of the land surface of the earth @xcite . approximately 1.3 billion people live in africa, which is about 18% of the world's population (wikipedia contributors, 2023a). of the estimated 7,000+ languages and dialects in the world, over 3,000 languages are found in africa † corresponding author: tobi@intron.io. despite its large and predominantly young population, africa bears a significant proportion of the global disease burden @xcite with multiple socioeconomic factors contributing to high mortality and morbidity rates @xcite . healthcare systems are overburdened and underfunded in many african countries @xcite @xcite , struggling to cope with the increasing demand for services, while at the same time facing significant shortages of trained health workers @xcite @xcite @xcite . a recent study conducted by @xcite in 47 african countries shows that the region has a ratio of 1.55 health workers (physicians, nurses, and midwives) per 1000 people-3x less than the who-recommended density of 4.45 health workers per 1000 people. while technology can help mitigate some of these problems, @xcite and @xcite aptly show that although africa has enjoyed massive growth in mobile technology, telecommunication, and internet penetration over the past two decades, healthcare technology lags significantly. a 2019 systematic review on the use of automatic speech recognition (asr) for clinical documentation in the us from 1990 to 2018 by @xcite and other similar studies @xcite @xcite showed that the use of speech recognition led to a 19-92% decrease in mean documentation time, 50.3-100% decrease in turnaround time, and 17% improvement in documentation quality. however, in the african context, the lack of training datasets for many of the 3000+ languages and accents in the continent remains an obstacle in developing and adopting robust speech recognition systems for the general domain and for clinical asr in particular @xcite @xcite . while recent efforts have begun to turn this tide for the majority of african languages like swahili, kinyarwanda, and yoruba @xcite @xcite , over a thousand african languages and accents remain excluded from global speech research advancements. recent single-digit word error rates (wer) @xcite @xcite in multiple sota publications and benchmarks on librispeech @xcite , ted-lium3 @xcite , and other datasets using architectures like wav2vec2 @xcite , conformer @xcite , transducer, and whisper @xcite contrast significantly with asr performance for african accented speech @xcite (see figure 2 ). we explore whether curating a large pan-african speech corpus might unlock comparable single-digit performance on african accents. we restrict this investigation to accented speech in english because english is the official language for the medical record in most anglophone african countries, expanding the utility of this work to multiple anglophone african countries. our contributions are as follows: ## related work with the advent of large multilingual speech datasets @xcite @xcite @xcite , various research groups have proposed large self-supervised speech models such as wav2vec @xcite , vq-wav2vec @xcite , wav2vec 2.0 @xcite , hubert @xcite , xlsr @xcite , and xls-r @xcite . these models achieved stateof-the-art performance on many downstream tasks such as automatic speech recognition (asr), automatic speech translation (ast), and language identification. however, most existing systems still perform poorly on accented speech @xcite . @xcite further showed that popular commercial asr systems-like amazon, apple, google, ibm, and microsoftexhibit substantial racial disparities in their speech recognition capabilities. most asr systems work best for native english speakers and their accuracy plummets dramatically with non-native english speakers @xcite . to enhance the performance of accented speech recognition, various methods have been proposed, which can be categorized into modeling and dataset approaches. on the modeling front, there have been efforts such as dialect-aware asr models @xcite , domain adversarial training (dat) @xcite , combining dat with transfer learning @xcite , using voice conversion (vc) @xcite , combining vc with speed perturbation @xcite , and accent pre-training (acc-pt) @xcite . these efforts, however, produced marginal improvements and still exhibit poor generalization capabilities. datasets have played a major role in improving asr performance. the current sota in asr @xcite demonstrated the superior utility of large supervised datasets. therefore, to bridge the asr performance gap for african accented speech, multiple dataset creation efforts @xcite @xcite @xcite @xcite @xcite have been established. however, many of these datasets are limited in size and diversity. for example, common voice @xcite ) contains less than 10 hours of african english speech, @xcite evaluate on 50 hrs of african accented english (not released), @xcite provide 40 hrs of accented english, less than 20% is african. @xcite @xcite are limited to a few south african accents, and @xcite include less than 8 hours, while @xcite include less than 2 hours of accented african english speech. furthermore, there are no available benchmarks for clinical asr for african languages, creating a need for evaluation datasets that help identify areas of improvement in this domain. while previous works have primarily focused on adapting western accents to african accents, to the best of our knowledge, there has been limited research specifically addressing domain adaptation from a general domain to the clinical domain in the african context. in this regard, our work is the first attempt to bridge this gap and tackle the unique challenges associated with adapting accented african english asr systems to the clinical domain. ## afrispeech dataset we introduce afrispeech, a pan-african accented english speech dataset for clinical and general domain asr crowd-sourced from 2,463 african speakers, 200.70 hrs with an average audio duration of 10.7 seconds. speaker, gender, age ## ethical considerations clinical asr models can improve productivity for clinicians, they can also increase documentation errors, especially through incorrect transcription of numbers, fractions, dates, and proper nouns which have legal, safety, and prognostic implications in healthcare. we caution clinicians to use asr with full discretion and review transcripts carefully before final submission into the medical record. we release afrispeech hoping that it will be beneficial to clinical and non-clinical use cases within and outside africa, improving asr performance for accented speech and it may contain biases due to publicly available datasets. we do not have access to reviewers who are native speakers of most of the languages covered in afrispeech who can provide a rigorous review of self-reported accents. this hinders our ability to investigate samples from all languages. we hope that future users of the dataset will further investigate afrispeech's utility and quality for their languages.