On the Role of Speech Data in Reducing Toxicity Detection Bias (2025)

Abstract: Text toxicity detection systems exhibit significant biases, producing disproportionate rates of false positives on samples mentioning demographic groups. But what about toxicity detection in speech? To investigate the extent to which text-based biases are mitigated by speech-based systems, we produce a set of high-quality group annotations for the multilingual MuTOX dataset, and then leverage these annotations to systematically compare speech- and text-based toxicity classifiers. Our findings indicate that access to speech data during inference supports reduced bias against group mentions, particularly for ambiguous and disagreement-inducing samples. Our results also suggest that improving classifiers, rather than transcription pipelines, is more helpful for reducing group bias. We publicly release our annotations and provide recommendations for future toxicity dataset construction.

## introduction with the growing prevalence of machine learning systems capable of processing and generating speech, there is rising interest in speech-aware toxicity detection @xcite @xcite . traditional cascaded approaches to speech toxicity detection use automated speech recognition (asr) to convert speech to text, before applying a standard text classifier. this strategy has two main issues. first, it eliminates rich prosodic and contextual information present in speech, which could degrade model performance. second, text-based toxicity detection systems are well known to exhibit significant biases against minoritized groups @xcite . for instance, many systems are more likely to consider african american english (aae) as toxic @xcite , while others denote the mere mention of identities such as "gay" and "lesbian" as toxic @xcite . often, these issues are attributed to biases in the training data. because minoritized communities are overwhelmingly the subject of online toxicity @xcite , classifiers misinterpret benign group mentions as toxic, producing a disproportionate rate of false positives for marginalized groups @xcite . given these limitations, recent research has sought to develop toxicity classifiers that operate directly on speech. in this work, we perform a systematic comparison of speech-based and cascaded text-based toxicity detection systems. specifically, we hypothesize that access to speech audio provides useful contextual information, which could reduce false positives. to investigate this, we produce a new set of annotations for a multilingual speech toxicity dataset, mutox @xcite , annotating for both toxicity and group mentions while also correcting automated transcripts. to ensure consistent and accurate data, annotations were performed by the authors using a rigorous multi-stage process of cross-checking and discussion. we leverage these annotations to produce critical new insight into both the efficacy and biases of speech-based and text-based toxicity detection models. our work reveals that incorporating speech data at inference time improves performance and reduces false positives on samples mentioning group identities, and eliminates false positives on ambiguous samples. furthermore, we find that this bias is not the result of transcription error, but of the classifier itself. we make our annotations publicly available to facilitate future research into the fairness and efficacy of speechbased toxicity detection. 1 contributions. to summarize our main contributions, we: ## annotating mutox the foundational contribution of this work is a new, high-quality set of annotations for the mu-tox test partition, allowing us to evaluate classifier bias against group mentions. we believe this represents the first fairness audit dataset for multilingual speech toxicity detection. ## the role of speech context we compare four representative toxicity classifiers to evaluate the utility of using speech data directly as opposed to cascaded asr-based systems, and to isolate the role of speech during training from during inference (see ## effect of transcription error one potential root cause of the failures observed in some cascaded asr-based systems could be the asr process. in other words, to what extent are the performance differences between the text-based classifiers a result of transcription failures rather than biases in the classifier itself? to address this question, we re-evaluate each classifier using the annotator-corrected transcripts. in fig. 5a , we observe that correcting the transcripts leads to a predictable improvement in the overall performance of the text-based classifiers. at the same time, fig. 5b shows that the effect on the false positive rate (fpr) specifically for group mentions was minimal. this suggests that transcription errors alone do not account for the observed biases in toxicity detection when group mentions are present and that refining transcription pipelines is unlikely to be a productive strategy for reducing bias in speech toxicity detection systems. ## ambiguity in toxicity annotation our hypothesis that speech context can support less biased toxicity detection is predicated on the idea that toxicity itself is often highly subjective and context-dependent, making it hard to detect from outside of the initial conversation. indeed, our annotation process is a testament to this fact. while we intentionally designed the annotation process with multiple stages to support interactive discussion and consensus building, an analysis of annotator disagreement demonstrates the extent to which toxicity judgments can vary. after the first stage of annotation, annotators only unanimously agreed on toxicity in 66% of samples. for 32% of samples, at least two annotators agreed, producing a majority vote, but for the remaining 2% of samples, every annotator voted differently. a total of 7% of samples were flagged for stage 2 discussion (see fig. 6a ). these samples tended to be challenging to annotate, often requiring some degree of inference to determine what was left unsaid. after review, annotators could not agree ("no consensus") on 8 samples, whereas 40 samples resulted in a "cannot say" (see fig. 1a ). from the selection of flagged samples in annotators also exhibited similar levels of disagreement when annotating for group mentions (see fig. 6b ) despite our detailed and iterative shared guidelines. a particular challenge for annotators was identifying whether certain group mentions corresponded to the category "racial or ethnic groups," as speakers rarely disambiguate between nationalities, ethnicities, or linguistic groups. ultimately, annotators reached a consensus after extensive discussion for all but one sample (see fig. 1b ). during transcription correction, annotators unanimously agreed more frequently-about 85% of samples, with only 5% requiring stage 2 review. annotators failed to reach a consensus on the correct transcription for 7 samples, highlighting the difficulty inherent in cascaded approaches. ## discussion & conclusion leveraging our new, high-quality set of group annotations for the mutox test partition, we compared the performance and biases of text-and speechbased toxicity classifiers. our analysis revealed that models that make use of speech data during both training and inference exhibit reduced fpr bias against group mentions. for ambiguous samples, we found that models trained on speech but without speech access at inference time exhibit an increased fpr, suggesting that the multimodal models rely on spurious correlations when lacking an informative modality. finally, we found that improving the quality of automated transcripts does little to reduce bias in english and spanish, but this may change with lower-resourced languages where asr systems exhibit poorer performance @xcite .