It Is Not Only the Negative that Deserves Attention! Understanding, Generation & Evaluation of (Positive) Moderation (2025)

Abstract: Moderation is essential for maintaining and improving the quality of online discussions. This involves: (1) countering negativity, e.g. hate speech and toxicity, and (2) promoting positive discourse, e.g. broadening the discussion to involve other users and perspectives. While significant efforts have focused on addressing negativity, driven by an urgency to address such issues, this left moderation promoting positive discourse (henceforth PositiveModeration) under-studied. With the recent advancements in LLMs, Positive Moderation can potentially be scaled to vast conversations, fostering more thoughtful discussions and bridging the increasing divide in online interactions.We advance the understanding of Positive Moderation by annotating a dataset on 13 moderation properties, e.g. neutrality, clarity and curiosity. We extract instructions from professional moderation guidelines and use them to prompt LLaMA to generate such moderation. This is followed by extensive evaluation showing that (1) annotators rate generated higher than professional moderation, but still slightly prefer professional moderation in pairwise comparison, and (2) LLMs can be used to estimate human evaluation as an efficient alternative.

## introduction moderation is helpful and at times crucial for maintaining healthy and constructive discussions. professional moderators not only police unproductive behavior, but also ask questions that help clarify and enhance the quality of discussions, as seen in fig. 1 (a ). they also encourage greater engagement from different participants. in political deliberation, moderators act as facilitators @xcite , making sure different perspectives are heard and productively debated @xcite . moderation is also vital in other contexts @xcite . b a ...the real issue with remote work for me isn't just the logistics; it's the lack of social interaction... user comment llm a b thank you for your comment! could organizing a mix of virtual and in-person social events help in this... great point! let's hear from others...how important is social interaction to your overall job satisfaction... instructions guidelines moderator professional moderation generated moderation in its various forms, the target of moderation can be either countering the negative or fostering the positive, or sometimes both. while content moderation deals with the policing of negative behavior in online settings (e.g. hate speech, misinformation, or abuse), positive moderation is implemented with interventions that target or foster positive behavior (e.g. constructiveness, inclusivity, empathy, cf. sec. 2). traditionally, much of the focus in nlp has been on studying content moderation that deal with negative aspects determined harmful or detrimental. much less focus has been put on positive moderation despite its importance in reducing conflicts among users @xcite , increasing participant satisfaction @xcite , and ultimately enhancing users' perception of discussion quality @xcite @xcite . recent advances in large language models (llms) allow for moderation to be conducted at scale @xcite , opening new avenues for enhancing online discussions. moderation can then act as a transparent signal to guide discussions and preemptively counter negativity (e.g. by ensuring that all perspectives contribute to the discussion). while positive moderation remains a task with limited data available, leveraging task-specific instructions to prompt models has proven effective in addressing such tasks @xcite . in this context, our research analyzes positive moderation and its generation with a special attention given to the evaluation. we investigate generating positive moderation using instructions derived from professional moderation guidelines to prompt llama models (fig. 1 ). we complement our modeling experiments with two annotation studies which allow us to better understand the phenomenon and evaluate the generation. the first (figure 2 : a) annotates moderation properties identified in professional guidelines, such as neutrality, clarity, and curiosity. we use data from two sources, one with professional moderation comments @xcite , and the other derived from the subreddit r/changemyview which covers a variety of topics @xcite . the annotation is done on professional moderation comments from the first data source mixed with generated moderation from both, resulting in 400 samples with 13 fine-grained annotation layers. the annotators score generated moderation higher than professional on all properties, demonstrating the success of using instructions based on moderation guidelines for prompting. further analysis shows that the score for professional moderation is more highly correlated with it addressing actual issues in the user comments as compared to generated moderation. this suggests the high scores for the latter might be assigned based on form and not content. we validate this in a second annotation study, a pairwise preference evaluation (figure 2 : a ′ ), showing that annotators still slightly prefer professional moderation in pairwise comparisons. relying only on human evaluation creates a bottleneck for further generation experiments. to address this, we explore automatic evaluation @xcite . experiments that use the data collected from annotators as gold scores demonstrate the effectiveness of prompting smaller-scale llms to estimate those scores. additional experiments show that we can estimate the individual evaluation of annotators by building persona prompts based on data from a questionnaire taken by annotators (figure 2: b ). the questionnaire collected data about annotators' expectations in online discussions and their expectations from moderators. employing auto-eval, we conduct further experiments on moderation generation that show better generation performance when incorporating chain-of-thought (cot) prompting or when incorporating more details about moderation properties. the contributions of this work encompass multiple levels. first, at the level of the investigated phenomenon, this is the first study targeting generation for positive moderation -and it additionally integrates recent development in llms with a thorough human evaluation. second, at the level of resources, it contributes to alleviate the low-resource limitations in the study of positive moderation by collecting a significant amount of annotations in two different scenarios (assessment vs. preference); releasing data with non-aggregated, fine-grained annotations, and user preferences, as well as generation and evaluation instructions foot_0 . third, on the methodological level, our extensive evaluation experiments comparing human annotators and llms (including annotator personas) contribute to current research on alleviating evaluation bottlenecks with llms, and on a better understanding of the potential and limitations of llms in subjective tasks. ## related work work focused on the generation of content moderation interventions includes counter-narrative generation @xcite and the employment and evaluation of llms as moderators @xcite . formulation of explainable moderation decisions @xcite @xcite is also related. the research landscape on nlp-supported facilitative and positive moderation, on the other hand, has focused generally on analysis rather than generation, e.g., investigating the effect of human moderation on deliberation qualitatively @xcite or empirically @xcite , or trying to predict the need for moderator intervention from comment features @xcite @xcite , with @xcite considering the link between different argument and deliberative quality dimensions and the need for moderation. @xcite also highlights the greater similarity between approved and removed content compared to other content that moderators did not review. when we turn to related work on interventions, it immediately appears clear that the space of actions (and corresponding tasks) for positive moderation is broader than for content moderation. related work ranges over forms of intelligent nudging of inactive participants @xcite , summarizing arguments and generating key points of a discussion @xcite , displaying comments with high constructiveness rankings to facilitate engagement @xcite , using llms to facilitate the establishment of common ground among users @xcite . taken together, these works represent first steps towards tasks that are related to positive moderation. there is, however, a gap in the generation of moderation interventions that are targeted specifically at the discourse quality, such as supporting participants in improving the (deliberative) quality of their comments, broadening the scope of the discussion, and keeping the deliberative goal present: this is precisely the gap that this work aims at filling. evaluation: while human evaluation remains the gold standard for generation assessment @xcite , automatic evaluation methods have recently gained prominence. increasingly, llms are used to approximate human evaluation, as seen in bertscore @xcite and other prompting-based methods utilizing gpt models @xcite 3 data we conduct our experiments on two datasets: one with professional moderator comments from a deliberative online discussion, and the other with moderation-like comments made by users. regulation room we use data from regulation room 2 (regroom), a deliberation platform that aims to get citizens involved in the regulatory decision-making process @xcite . participants can review detailed information provided 2 http://regulationroom.org/ by federal agencies about proposed regulations and share their comments and opinions. human moderators oversee and support the discussion. while their role includes moderation of negative aspects in the conversation (i.e. policing), it also focuses on promoting positive aspects such as improving the quality of a comment, and broadening the discussion by encouraging participation from other users. in our experiments, we focus on these two moderator functions and filter out any samples that deals with the moderation of negativity. moderators follow a moderator protocol @xcite , which defines the role of the moderator and theoretical properties and grounds for a moderator intervention. the regroom dataset contains 3k comments spanning various topics, with approximately 25% being followed by a moderation comment. user moderation the user moderation (usermod) dataset @xcite foot_1 contains a sample from changemyview dataset @xcite . the dataset is based on discussions from the /r/changemyview subreddit, where each thread is a discussion that aims to effectively change the view of the original poster (op) in the thread about a specific topic. the dataset has more variety of domains but lacks the professional moderation that aims at promoting the positive aspects of a discussion as the platform moderation in reddit is focused on handling rule violations. this dataset has been collected based on the observation that in online discussions, users themselves sometimes act as moderators without being instructed to, based on their general intuitions (e.g., asking for clarifications, suggesting points for improvement to other users). @xcite collected 1000 comments annotated for their potential to be a (user) moderation of their preceding comment. ## annotation we collect data and run an annotation study to identify moderation properties, evaluate the generation of moderator comments, and establish the properties that differentiate generated from professional moderation. annotators were recruited from prolific. 4 the annotators met specific criteria, including english fluency, a high school diploma or higher, and a 100% approval rate on previous submissions. two annotation studies are conducted: the first (sec. 4.1) targets the assessment of individual (generated and professional) moderation comments and the fine-grained scoring of moderation properties; the second (sec. 4.2) is a preference task directly comparing two moderation replies to the same comment. in both, the annotators were first asked a set of introductory questions to identify their own expectations in online discussions and in relation to moderation (cf. fig. b .2). ## models & instructions for results to be more reproducible, we use an open source model for our experiments (llama-3, the newest model in the llama family of models @xcite ). we use nested 4bit quantization @xcite and top-p (nucleus) sampling with the default parameters from the original llama codebase. experiments are done with two model sizes 8b and 70b and the instruction fine-tuned version is used. ## results & analysis analyzing and evaluating moderation is challenging due to its subjective and ambiguous nature. we use human evaluation as foundation @xcite and complement it with llm-based automatic evaluation to streamline further experiments. ## conclusion this work made first steps toward the automatic generation of positive moderation, which facilitates the discussion by encouraging positive user behaviour. our work showcases the potential for using llms to achieve this purpose in real-world settings. we envision scenarios where human moderators receive initial suggestions that, though requiring some adjustment to better address issues in user comments, offer a valuable starting point as demonstrated in our experiments. by leveraging the capabilities of llms, we can assist in fostering more thoughtful and inclusive discussions online, and aim to bridge the divide in increasingly polarized interactions online.