An Investigation into Explainable Audio Hate Speech Detection (2024)

Abstract: Research on hate speech has predominantly revolved around the detection and interpretation from textual inputs, leaving verbal content largely unexplored. Moreover, while there has been some limited exploration into hate speech detection within verbal acoustic speech inputs, the aspect of interpretability has been overlooked. As such, we introduce a new task within the audio hate speech detection task domain - we specifically aim to identify specific time frames of hate speech within audio utterances. Towards this, we propose two different approaches, cascading and End-to-End (E2E). The first cascading approach initially converts audio to transcripts, identifies hate speech within these transcripts, and subsequently locates the corresponding audio time frames. Conversely, the second E2E approach processes audio utterances directly, which allows it to pinpoint hate speech within specific time frames. Moreover, due to the lack of explainable audio hate speech datasets that include frame-level rationales, we curated a synthetic audio dataset to train our models. We further validate these models on actual human speech utterances and we find that the E2E approach outperforms the cascading method in terms of audio frame Intersection over Union (IoU) metric. Furthermore, we observe that the inclusion of frame-level rationales significantly enhances hate speech detection accuracy for both E2E and cascading approaches.

## introduction online platforms such as youtube, dailymotion, and tiktok have undoubtedly experienced a notable surge in popularity over the years. while this * equally contributed has led to an increased dependence on audio as a primary mode of communication, this phenomenon has also brought the issue of hate speech in audio content to the forefront. youtube, for instance, has consistently been proactive in removing hateful content since its inception, aligning with its hate speech policy foot_0 . nevertheless, it is worth noting that out of a total of 10,501,072 channels removed from the youtube platform within the period of july to september 2023, 26,130 channels were specifically taken down due to their association with hate speech foot_1 . these statistics underscore the unequivocal importance and the imperative need for the development of effective methodologies to precisely identify hate speech within verbal expressions. an important point to note, however, is that most hate speech datasets are exclusively text-based. consequently, research endeavors pertaining to hate speech detection @xcite as well as investigations into hate speech explainability @xcite are confined to textual inputs. in other words, despite the explosive increase of hate speech on audiobased online social platforms, there is a notable absence of research that addresses hate speech in verbal data. a few studies related to auditory hate speech detection have been proposed. for example, @xcite ; @xcite curated respective audio-visual multi-modal datasets. yet, to the best of our knowledge, no research addresses explainable hate speech detection in the audio domain, that is, the understanding of the rationale behind the model's decisions. therefore, we first introduce the new task of explainable audio hate speech detection, which encompasses two sub-tasks: audio hate speech classification (ahs-cls) and audio hate speech frame detection (ahs-fd). the former involves determining whether an audio utterance is hate speech, while the latter identifies the specific time frames containing hate speech. in addition, since there is a lack of interpretable audio hate speech datasets that include audio frame-level rationales, we curated a dataset called audiohatexplain, which annotates which part of human and synthetic audio recordings pertains to hate speech. moreover, we propose cascading and e2e models, which are able to elucidate the underlying reasons for classifying speech as hate or not by identifying relevant audio rationales. the cascading approach first transforms audio into text transcripts, detects hate speech within these transcripts, and then matches the detected hate speech to the corresponding audio time frames. in contrast, the e2e approach directly processes the audio input, enabling it to identify the specific time frames containing hate speech accurately. we validated these models on actual human speech utterances and found that the e2e approach outperforms the cascading method regarding the audio frame intersection over union (iou) metric. this superiority is attributed to the bottlenecks arising from conversion between audio and text, such as automatic speech recognition (asr) errors, and disarrangement between word tokens and time frames. furthermore, we observed that including frame-level rationales significantly enhances hate speech detection accuracy for the e2e approach. ## dataset generation given the absence of existing explainable audio hate speech datasets, we created a synthetic dataset using a text-to-speech (tts) model. this section details the methods used to convert text transcripts samples avg. length (sec.) train 14,183 6.62 dev. 1,771 6.60 test-synth. 300 8.97 test-human 300 10.52 to delineate the process of audio rationale generation, it is imperative first to comprehend the foundational structure of the original text-based hatexplain @xcite dataset. this text-based hatexplain dataset, represented as @xmath0, w (1) , y (1) ), . . . , (x (l) , w (l) , y (l) )}, comprises l samples. each sample consists of a textual sentence x paired with its corresponding binary class label @xmath1, 1}, denoting whether the sentence qualifies as hate speech (1) or as normal discourse (0). moreover, each textual sentence x is supplemented by a set of wordlevel annotations w , which is defined as @xmath2, δ (1) w ), . . . , (w (n ) , δ (n ) w )}. here, n signifies the position of a word w within the sentence, with each word linked to its word-level rationale δ @xmath3, 1}. specifically, a word w is assigned a rationale of 1 if it contributes to the classification of the sentence as hate speech, and 0 otherwise. these word-level rationales serve as discernible evidence aiding in identifying and classifying hate speech within textual content. text-to-speech from the above-mentioned text dataset, we convert each text-based transcript x into audio samples of sample rate 22050 hz. this conversion is achieved by leveraging the nonautoregressive fastspeech @xcite tts model in conjunction with the hifi-gan @xcite vocoder. to ensure the coherence of audio samples, we expand abbreviations, remove emojis, and exclude sentences in languages other than english, as well as those that contain semantically vacuous words like the placeholder "<@xmath4. rationale labeling each tts-generated audio sample a is then paired with its binary classification label @xmath5, 1} to identify whether it is hate speech (1) or normal audio (0). moreover, it is imperative to provide acoustic rationales to facilitate the explainability of hate speech detection in the audio domain. to accomplish this, we employ a pretrained montreal forced aligner @xcite to identify the timestamps, i.e., the starting and ending times in milliseconds of each spoken word within a given sentence. subsequently, we divide audio samples into m 10mslong audio frames f , and each audio frame f is annotated with its audio frame-level rationale δ @xmath6, 1}. it is possible to annotate each audio frame with rationales as text-based word-level rationale δ w are previously provided. our au-diohatexplain dataset can thus be represented as @xmath7, f (1) , y (1) ), . . . , (a (l) , f (l) , y (l) )} of l tts-generated audio samples. in addition, the set of frame-level audio annotations f can be reorganized as human recordings in addition to the synthetic audio dataset, we curated a separate collection of authentic human recordings for evaluation purpose. the transcript used for these recordings were derived from the original text-based hatexplain dataset, but underwent a two-step post-processing procedure. specifically, among the 1,779 original test samples, we initially use chatgpt (appendix a) to select texts that were suitable for spoken format. this process enabled us to sample 695 spokenform texts from the hatexplain test set. these texts were then manually filtered to ensure that the final utterances adhered to syntactic and lexical choices appropriate for spoken language @xcite . ultimately, 300 utterances were selected for the test set. we synthesized these samples using tts models and also recorded them with human participants. the participant group consisted of 10 individuals, 6 males and 4 female speakers. each participant read an average of 30 utterances, including hate speech and normal texts. recordings were conducted in silent environments. for ethical considerations, all participants were fully informed about the nature of the transcripts, which included hateful language. moreover, all recordings were conducted with the explicit consent of the volunteers for research purposes only. the statistics of the audiohatexplain is provided in ## methodology in order to classify entire audio clips as hate speech or normal, as well as precisely pinpoint the audio frames associated with hate speech, we introduce two models. the first model uses a cascading framework (figure 1 ), which transcribes audio into text, predicts hate speech in the text, and then maps the word-level rationale onto a time grid. the second model (figure 2 ) employs an e2e design, directly classifying and predicting hate speech frames from the audio. ## experimental setup cascading models we adapted whisperx @xcite for asr and for word-level time stamp-ing and utilized the bert model from @xcite for hate speech detection within transcribed text. the bert model foot_2 was trained using the adam optimizer (kingma and ba, 2014) with a learning rate of 2e-5, a batch size of 64, a threshold θ set to the mean value of attention scores, and a coefficient λ of 0.1. we selected the best checkpoint with the highest ahs-cls f1 after 10 training epochs. the asr model was not finetuned, using whisper-large-v2 foot_3 as the checkpoint. to enhance the robustness of the cascading model against asr transcription errors, we trained the bert-based model with both asr transcriptions and golden texts. the model trained with asr transcriptions is called cas. (asr text), while the model trained with golden texts is referred to as cas. (gold text). the bert model is fine-tuned using either golden transcriptions paired with corresponding word-level rationales or asr transcriptions with word-level rationales that account for potential inaccuracies. e2e models for the end-to-end (e2e) model, we utilized wav2vec 2.0 foot_4 as the shared speech encoder. the audio-level classification head and the frame-level detection head were fine-tuned simultaneously or separately (e2e cls-only and e2e fd-only). unless specified otherwise, the e2e model was trained with both tasks. we employed the adam optimizer with a learning rate 4e-3 and a batch size of 64, training the model for 50 epochs. the best model was selected based on the highest ahs-cls f1 score. ## conclusion in this paper, we introduced the new task of explainable audio hate speech detection, which encompasses two sub-tasks: audio hate speech classification (ahs-cls) and audio hate speech frame detection (ahs-fd). furthermore, we introduced e2e and cascading models. these models are capable of not only classifying hate speech directly from verbal speech, but also identifying hate rationales within audio frames. in particular, the proposed e2e model consistently outperforms the cascading model on the ahs-fd task. this superiority is attributed to the bottlenecks arising from conversion between audio and text within the cascading model. this suggests that, for the task of explainable audio hate speech detection, is it more effective to directly process audio inputs. upon acceptance, we plan to make our dataset and code publicly available to encourage further research for the important topic of explainable audio hate speech detection.