SLM -Mod: Small Language Models Surpass LLM s at Content Moderation (2025)

Abstract: Large language models (LLMs) have shown promise in many natural language understanding tasks, including content moderation. However, these models can be expensive to query in real-time and do not allow for a community-specific approach to content moderation. To address these challenges, we explore the use of open-source small language models (SLMs) for community-specific content moderation tasks. We fine-tune and evaluate SLMs (less than 15B parameters) by comparing their performance against much larger open- and closed-sourced models in both a zero-shot and few-shot setting. Using 150K comments from 15 popular Reddit communities, we find that SLMs outperform zero-shot LLMs at content moderation -11.5 % higher accuracy and 25.7% higher recall on average across all communities. Moreover, few-shot in-context learning leads to only a marginal increase in the performance of LLMs, still lacking compared to SLMs. We further show the promise of cross-community content moderation, which has implications for new communities and the development of cross-platform moderation techniques. Finally, we outline directions for future work on language model based content moderation.

## introduction content moderation has become a growing area of interest for the nlp community @xcite due to the rapid growth and use of social media. the primary challenge in content moderation lies in detecting undesirable, norm-violating behavior amidst vast amounts of content posted by users. in order to deal with large volumes of content, most platforms rely on automated tools to either directly remove norm-violating content or triage undesirable content for manual review by human moderators @xcite . although moderation involves dealing with a wide range of undesirable behaviors, current computational approaches tend to adopt narrow definitions of abuse (e.g., toxicity, hate speech) @xcite . however, content moderation is a nuanced and contextual task that varies across platforms, communities, and over time-e.g., a post deemed desirable in one community may be undesirable in another @xcite . moderation requires an understanding of community norms, and depend on the manual labor and judgment of human moderators, who are often overworked and uncompensated @xcite . despite these variations between communities, current approaches do not effectively incorporate community norms when detecting undesirable content. due to the natural language understanding capabilities of large language models (llms) @xcite , recent work has explored the use of llms for content moderation @xcite . although llms can be used off-the-shelf and generate explanations for moderation decisions, moderators would still require the ability to dynamically adapt these models to suit the preferences and norms of their community. in other words, there is a need for specialized models that provide moderators with more control and configurability via fine-tuning. current methods for fine-tuning llms are constrained by the substantial computational resources and costs required, making it nearly impossible for moderators to tailor llms for community-specific content moderation. given these limitations, a viable alternative could be the use of small language models (slms), which provide a lightweight and cost-effective option compared to llms. recent research has shown that despite their smaller size, slms can be fine-tuned to achieve performance comparable to llms on various natural language understanding tasks @xcite . this work explores the potential for using slms in content moderation, evaluating their performance relative to llms in terms of accuracy, recall, and precision across multiple online communities on reddit. in particular, we focus on whether finetuned slms can offer a more resource-efficient approach without compromising on moderation quality. we aim to determine if slms can strike a balance between cost-effectiveness and moderation accuracy, ultimately providing a viable alternative to the computationally expensive llms for largescale, community-specific moderation. findings: our findings reveal that fine-tuned slms outperform zero-shot llms at in-domain content moderation tasks having both higher accuracy and recall, with slightly lower precision. we present a case study of r/changemyview and conduct an error analysis on false positives and false negatives to identify trade-offs when using slms over llms. we show that even under a fewshot setting, llms lack performance compared to slms. moreover, slms have a higher auc score compared to llms on more realistic imbalanced datasets. next, we highlight the potential of slms for cross-domain content moderation tasks and investigate possible reasons for their high performance. finally we discuss the implications of our findings for content moderation and highlight future directions for improving cross-community approaches to moderation as well as dynamically adapting community-specific models over time. ## discussion and implications in this section, we discuss the implications of our work for online content moderation. ## conclusion this paper examined the effectiveness of small language models (slms) and large language models (llms) in content moderation tasks. we found that slms with less than 15b parameters, such as gemma-2-9b and mistral-nemo-instruct, consistently outperformed significantly larger llms like gpt-4o at identifying undesirable content. slms showed a higher recall, indicating their superior ability to flag a wider range of undesirable content, which can be crucial for effective moderation on large-scale platforms. we also found that slms have higher auc scores than llms under realistic imbalanced data conditions, and that even with in-context examples llms fail to match slms. in addition to in-domain moderation tasks, we uncovered the potential of slms to be used in crosscommunity moderation tasks. cross-community moderation can benefit smaller communities with fewer resources to train their own in-domain models or newer communities where community norms are still emerging. we provided qualitative insights into the trade-offs arising from content length and topic that could impact slms and llms differently. overall, slms offer an effective, scalable, and costeffective solution for content moderation, achieving a strong balance between accuracy, recall, and stability. future work can further improve crossdomain performance by training models that learn shared notions of norms and values, and explore frameworks that leverage complementary strengths of slms and llms to enhance moderation capabilities across diverse online communities.