Implanting LLM ’s Knowledge via Reading Comprehension Tree for Toxicity Detection (2024)

Abstract: Toxicity detection plays a crucial role in maintaining the peace of the society. Existing methods can be roughly categorized as small language model (SLM) based and large language model (LLM) based. However, due to the limitation of SLMs on general knowledge and the potential embedded bias in LLMs despite their large amount of knowledge, it is not a good idea to detect toxicity only with either SLM or LLM based method.In this work, we propose to implant LLM’s knowledge into SLM based methods such that we can stick to both types of models’ strengths. To this end, we develop a reading comprehension (RC) tree to transfer knowledge between two models. Specifically, we first construct the RC tree, from an extensive to intensive reading perspective, to capture the local and global information in the text. We then model samples encoded by SLM and knowledge extracted from LLM as two distributions using the constructed RT tree. We finally transfer knowledge via optimal transportation between two distributions. Extensive experiments prove the effectiveness of our method on real-world and machine-generated datasets.

## introduction with the prevailing of online communication, toxic content has been growing in recent years on the websites. in addition, the malicious use of large language models makes toxic language more common and difficult to detect. toxic language may result in serious hazards such as the promotion of violent crimes and discrimination against marginalized groups. due to its harmfulness to individuals and society @xcite , toxic language has become a serious concern. the harm of toxic content can be prevented either by pre-detection before online release or by post-detection before spreading to a broader audience, and thus the detection of toxic content plays a crucial role in controlling toxicity. existing methods for this purpose can be roughly categorized into small language model (slm) based and large language model (llm) based types. the finetuned slm based methods @xcite are the mainstream. due to the limited knowledge, the performance of these methods is not very satisfying. recently, a llm based method @xcite ) is presented for this task. however, the potential of llms has not been fully exploited. more importantly, llms have their own bias which may lead to new safety issues (liyanage and ranaweera, 2023; @xcite . as fig. 1 (a ) and (b) show, neither the roberta @xcite based method nor chat-gpt (openai, 2022) can make a correct prediction about the given sample. to have a close look, we let two language models paraphrase 'cockroaches' and 'scorpions' by masking these two words for roberta and directly asking chatgpt. it can be seen that roberta associates 'cockroaches' with 'everywhere' and 'scorpions' with 'poisonous' while chatgpt actually knows they are both 'insects'. we hence infer that the wrong prediction of roberta based method is caused by the limited knowledge and that of chatgpt might be due to the potential bias embedded in the model @xcite . in view of the shortcomings of slm and llm based methods, a natural idea is if we can combine both types of language models to leverage their strengthes. however, this is not a trivial task since toxicity detection requires the deep understanding of the text. a simple combination by treating llm's knowledge as the context of slm or concatenating embeddings from two models does not improve slm much, as shown in fig. 1 (c ). this drives us to solicit an effective way to comprehend the text. fortunately, humans have well established the theory for enhancing reading skills @xcite @xcite , and a pioneering work @xcite has solved the summarization problem by simulating the human reading process. in this paper, we propose to implant llm's knowledge into slm based method via a reading comprehension tree for toxicity detection. (1) we first construct the reading-comprehension (rc) tree to gain a deep understanding of the text, i.e., we first form local rough understanding via scanning words and phrases in the text and then incorporate the global comprehensive semantics of the entire text. (2) we then use the rc tree to model the sample encoded by slm and knowledge from llm as two distributions, i.e., we add task-specific features to the root of the tree to get multi-grained information distributions in a top-to-down manner. (3) we finally introduce the optimal transport method to fulfill knowledge implanting task, i.e., we apply optimal transport to llm's knowledge distribution and the sample's distribution. our main contributions are as follows: reading comprehension is a cognitive and constructive process about text @xcite . it is necessary to get a comprehensive understanding of the text for further applications @xcite . in the field of natural language processing, reading comprehension is usually modeled as question answering @xcite and applied to summarization tasks @xcite @xcite . existing methods mainly focus on evaluating the importance of text spans to specific questions or summarizations @xcite @xcite @xcite . in contrast, our method aims at comprehending text spans themselves and captures the multi-granularity semantics into a reading-comprehension tree, which is suitable for diversified downstream tasks including the knowledge transfer in our work. ## methods this section illustrates the details of our reading comprehension tree (rct) based method. ## conclusion in view of the limitation of knowledge in slm based methods and the potential bias and harmfulness in llm based methods, we propose a novel framework to implant knowledge from llms into slm based methods to leverage their strengthes for toxicity detection. specifically, we design a reading comprehension (rc) tree to mimic the cognitive process, which gets a deep understanding of texts in the sample and knowledge and also facilitates the transportation of the knowledge. experiments on three widely used datasets prove that our method achieves significantly better performance than existing detection tools and methods.