Enhancing LLM -based Hatred and Toxicity Detection with Meta-Toxic Knowledge Graph (2025)

Abstract: The rapid growth of social media platforms has raised significant concerns regarding online content toxicity. When Large Language Models (LLMs) are used for toxicity detection, two key challenges emerge: 1) the absence of domain-specific toxicity knowledge leads to false negatives; 2) the excessive sensitivity of LLMs to toxic speech results in false positives, limiting freedom of speech. To address these issues, we propose a novel method called *MetaTox*, leveraging graph search on a meta-toxic knowledge graph to enhance hatred and toxicity detection. First, we construct a comprehensive meta-toxic knowledge graph by utilizing LLMs to extract toxic information through a three step pipeline. Second, we query the graph via retrieval and ranking processes to supplement accurate, relevant toxicity knowledge. Extensive experiments and case studies across multiple datasets demonstrate that our MetaTox boosts overall toxicity detection performance, particularly in out-of-domain settings. In addition, under in-domain scenarios, we surprisingly find that small language models are more competent. Our code is available at https://github.com/YiboZhao624/MetaTox.

## introduction online social media platforms have become a major source of information for people worldwide. meanwhile, they also provide a communication tool for spreading toxic content including harassment, trolling, cyberbullying, and hate speech, which poses a serious and continual threat to the harmony of society @xcite and harms children's mental health @xcite . it thus becomes a the statement you provided does not contain hate speech. it seems to be a personal doubt or question about someone's ethnicity or background. ...... ## related work previous methods can mainly be classified into rulebased, embedding-based, and transformer-based, and llm-based approaches. rule-based methods @xcite rely on predefined rules that match specific patterns within the text. @xcite uses the insults and swears words to form a dictionary for hatred and toxicity detection. embedding-based approaches, on the other hand, combine text representations obtained by methods like word2vec @xcite . transformer-based methods @xcite , typically perform fine-tuning on pre-trained models like bert. recent studies have explored the use of llms, which typically serve as a powerful data augmentation tool. some methods @xcite directly prompt llms to generate explanatory information about the text, which is then combined with the original text to train a detection model. furthermore, some approaches @xcite employ structured reasoning methods such as chain-of-thought (cot) @xcite and tree-of-thought (tot) @xcite to infer hidden semantics of input text. some studies aim to enhance the robustness of hatred and toxicity detection. for example, textattack @xcite improves robustness through data augmentation, while @xcite proposes the toxictrap modular, leveraging adversarial training to strengthen model resilience. however, previous approaches have either overlooked related real-world knowledge or relied solely on distilling the limited internal knowledge of llms. in contrast, our method is the first to build a hate-related kg and introduce it into hatred and toxicity detection, offering interpretive knowledge injection and multi-hop inference ability. ## methodology in this section, we present metatox, our proposed method for enhancing hatred and toxicity detection using a meta-toxic knowledge graph. meta-tox consists of two stages: (1) construction of the meta-toxic knowledge graph through a three-step pipeline, and (2) querying the graph to enhance the downstream task of binary classification for toxicity detection. we begin with the data collection process, which lays the foundation for constructing the knowledge graph. ## conclusion in this paper, we proposed a novel method called metatox to address both false positive and false negative misjudgments caused by the lack of domain-specific knowledge and llms' extreme sensitivity. first, we leverage llms to extract toxic content through a three-step pipeline, which builds the meta-toxic knowledge graph. next, we query the graph with retrieval and ranking processes to provide additional, relevant toxic knowl-edge. extensive experiments and detailed case studies across various datasets show that metatox significantly lowers the false positive rate while improving overall toxicity detection performance in ood scenarios. in addition, under in-domain scenarios, we surprisingly find that small language models are more competent.