Recent Advances in Online Hate Speech Moderation: Multimodality and the Role of Large Models (2024)

Abstract: Moderating hate speech (HS) in the evolving online landscape is a complex challenge, compounded by the multimodal nature of digital content. This survey examines recent advancements in HS moderation, focusing on the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) in detecting, explaining, debiasing, and countering HS. We begin with a comprehensive analysis of current literature, uncovering how text, images, and audio interact to spread HS. The combination of these modalities adds complexity and subtlety to HS dissemination. We also identified research gaps, particularly in underrepresented languages and cultures, and highlight the need for solutions in low-resource settings. The survey concludes with future research directions, including novel AI methodologies, ethical AI governance, and the development of context-aware systems. This overview aims to inspire further research and foster collaboration towards responsible and human-centric approaches to HS moderation in the digital age.

## introduction in the era of rapid information exchange and digital connectivity, the rise of hate speech (hs) presents a significant challenge with profound implications for global societies. hs, which is any communication demeaning a person or a group based on social or ethnic characteristics, undermines social harmony and individual safety, both online and offline @xcite . the recent israel-hamas conflict has notably escalated both anti-muslim and anti-semitic sentiments worldwide, evidenced by the trending of hashtags such as #hitlerwasright and #deathtomuslim on the social media platform x. 2 moreover, the council on american-islamic 0:32 0:46 0:54 "you don't belong in the usa" "and we're not patience for ingrates" "no point to immigrate" figure 1: examples of an anti-migrant hs in different forms, encompassing text, image and/or audio modalities. the text-based, vision-language and video-based hs are taken from the social bias inference corpus (sbic) dataset, the facebook hateful memes (fhm) dataset and the bitchute website, respectively. relations reported receiving 774 help requests and bias reports from muslims in the usa within a 16day period. 3 while digital interconnectivity facilitates swift information sharing, it simultaneously amplifies the spread and the impact of hs, transcending geographical boundaries. technological advancements have transformed the expression of hs, leading to its manifestation in various novel forms. traditionally, hs was predominantly text-based, found in written materials @xcite , or verbalized in posts, broadcasts, and public speeches @xcite . the digital era has ushered in more complex and subtle variants of hs, engaging multiple sensory modalities. a notable instance is vision-language hs, which fuses visual elements with text, commonly disseminated through captioned images and memes @xcite . videobased hs, another emerging form, amalgamates text, visuals, and audio, creating a multi-faceted and potentially more influential mode of communication @xcite . figure 1 shows various hs forms targeting immigrants, underscoring animosity towards individuals of diverse nationalities. the text-based approach overtly projects hostile attitudes towards them in the host country. in visionlanguage hs, visual (e.g., a person preparing to shoot) and textual elements (e.g., sighting an illegal immigrant mowing the lawn) jointly convey antagonism. the figure also includes a music parody, integrating derogatory visuals with discriminatory audio lyrics, to showcase contempt for immigrants. while existing research surveys @xcite @xcite have largely focused on text-based hs, they often overlook the complexity of multimodal content. our survey addresses this gap by offering a comprehensive analysis of hs across various digital platforms, including text, visual, auditory, and combined multimodal expressions. we explore the distinct ways hs manifests in these formats, providing insights into their characteristics and moderation challenges. additionally, we emphasize the critical role of large language models (llms) and large multimodal models (lmms) in moderating hs, given their ability to process and interpret diverse data types. this survey critically evaluates existing solutions, identifies areas for improvement, and advocates for a shift towards multimodal approaches in hs moderation. in summary, our paper not only bridges the gap in the existing literature by providing a detailed exploration of multimodal hs but also paves the way for future research in this area. we aim to inspire advancements in hs moderation technology, particularly in the development and refinement of large models, which are imperative for tackling the complex and ever-changing nature of online hs. ## hate speech hs takes various forms -written text, images, spoken words, and multimedia content -each posing risks of violence, animosity, or prejudice against specific groups. this section reviews existing literature on hs, categorizing it into text-based, image-based, video-based, and audio-based types. for each hs form, we provide a detailed categorization across four tasks: detection, explanation, debiasing, and counter-speech. detection identifies hateful content, forming the basis for further actions. explanation promotes transparency by clarifying why content is flagged, building trust in automated systems. debiasing is essential to refine detection systems, ensuring fairness and reduc- mod. dataset task labels source # records text wz-ls (waseem and hovy, 2016) det. [m.c.] sexism, racism, neither twitter 16,914 ghc (kennedy et al., 2018) det. [m.c.] vo, hd, cv [b] implicitness [m.c.] hate targets forums 27,665 stormfront (de gibert et al., 2018) det. [b] hateful stormfront 9,916 dt (davidson et al., 2017) det. [m.c.] hateful, offensive, neither twitter 24,802 founta (founta et al., 2018) det. [m.c.] offensive, abusive, hateful speech, aggressive, cyberbullying, spam, normal twitter 80,000 dynahate (vidgen et al., 2021) det. [b] hateful, [m.c.] hate targets, [m.c] animosity, derogation, dehumanization, threatening, support h-m adv 41,134 sbic (sap et al., 2020) det. [b] offensive [m.c] hate targets [b] intent [b] lewd [b] group [b] in-group mixed 44,671 expl. [f.t.] implied statement ihc (elsherief et al., 2021) det. [m.c.] implicit, explicit, non-hate [m.c.] grievance, incitement, inferiority, irony, stereotypical, threatening, others twitter 22,584 expl. [f.t.] implied statement hatexplain (mathew et al., 2021) det. [m.c] hate, offensive, normal [m.c.] hate targets mixed 20,148 expl. [m.l] text rationales/snippets nacl (masud et al., 2022) det. [m.c.] hate intensity [m.l.] hate spans mixed 4,423 ctr. finegrained fhm @xcite det. crisishatemm @xcite det. ## methodology this section reviews the state-of-the-art methodologies that have significantly contributed to primary areas of hs research, particularly those involving large models. first, we discuss the recent capabilities of large models (section 3.1). subsequently, we explore studies in four important hs areas: detection (section 3.2), explanation (section 3.3), debiasing (section 3.4), and counter-speech (section 3.5), focusing on works using large models. this review highlights the emerging trends, providing insights into how large models can be used to understand and address hs in its various forms. ## challenges in the dynamic realm of research, especially in areas related to user-generated content and online harmfulness, numerous challenges persist that shape the trajectory and emphasis of scholarly in-vestigations. these challenges, ranging from technical to ethical, define the landscape in which research on hs moderation and detection operates. data complexity, quality, and sourcing. the subtlety of some hate speech, known as implicit hate speech, presents a considerable challenge in identifying and understanding the underlying intent, as these intents can hide within seemingly neutral language or actions @xcite @xcite . this difficulty highlights the complexity of human communication and biases, where hateful messages can be conveyed indirectly or through coded language. furthermore, sourcing data from diverse platforms such as gab, youtube, and 4chan introduces difficulties in standardization and interpretation @xcite . additionally, the uneven distribution of hate instances across datasets poses significant obstacles for accurate model training. @xcite . these challenges underscore the need for advanced methods capable of navigating the intricate and multifaceted nature of data. ## conclusion we highlighted the advancements in hs moderation, underscoring the pivotal role of llms and lmms. despite these strides, challenges remain, particularly in inclusivity and nuanced detection. future research should focus on developing ai methodologies that are more context-aware and ethically governed. this endeavor is not only a technological challenge, but also a moral imperative, necessitating interdisciplinary collaboration. as we advance, it is crucial to ensure that technological advancements are matched with a commitment to responsibility, striving for a digital environment that is secure and welcoming for everyone.