Detoxifying Online Discourse: A Guided Response Generation Approach for Reducing Toxicity in User-Generated Text (2023)

Abstract: The expression of opinions, stances, and moral foundations on social media often coincide with toxic, divisive, or inflammatory language that can make constructive discourse across communities difficult. Natural language generation methods could provide a means to reframe or reword such expressions in a way that fosters more civil discourse, yet current Large Language Model (LLM) methods tend towards language that is too generic or formal to seem authentic for social media discussions. We present preliminary work on training LLMs to maintain authenticity while presenting a community’s ideas and values in a constructive, non-toxic manner.

## introduction certain topics, beliefs, and views can lead to polarized and inflammatory discourse, often with little headway made in conveying these views to an opposing side in a constructive manner. as such topics are often strongly tied to emotional experiences, beliefs, and identity, the expression of such views likewise carry heavy emotional weight which can hinder constructive discourse. we thus explore the potential for natural language generation through large language models (llms) to mediate such discourse by expressing polarizing views and opinions in a manner with reduced toxicity and with a style more closely mimicking that of the target audience. consider the following human authored reddit post: there's no shame in abortion. only repressed and self loathing people like you, feel that way. i've had two and don't regret anything. your shame tactics don't work with me, cupcake. the text is associated with computed levels of interaction measures: toxicity, sentiment, and attitude. from this information, language generation can be applied to the goal of encouraging constructive discussion through an application of large language models to attempt to retain the underlying meaning while removing inflammatory or divisive language. specifically, a 3-layer system couples an encoder-decoder model and a language generation model, where the encoder-decoder identifies content parameters from a post. these, taken together with the original subreddit and the interaction measures to fill in blanks to create a synthetic post: i have had two abortions and stand by my choices. your attempts to shame me are ineffective, my friend. it is not productive to sham people for their own decisions. you should be ashamed of your own choice. abortion is a personal choice that should not carry any negative stigma. everyone is entitled to their own beliefs, but shaming others for their own decisions is not productive. personally, i have had two abortions and stand by my choices. if you want to know more about me, please feel free to reach out to me. many current llm methods, such as chatgpt foot_0 , tend towards a tone of summarization while also generating text that is prosaic without a prompt specifically asking for a specific style. this tone is likely to be seen as that of a moderator or automated system, and we believe would less effective in improving discussions in communities. our approach of learning community style and opinions while mediating toxic expression supports our ultimate goal of an echo-chamber burster, where we generate posts resembling a subreddit's style but presenting an opposing view point. the assumption here is that by constructively expressing an opposing view, the community may be more open to listening to the opposing view. we anticipate this system could be used for social media platforms, moderators, or widgets to suggest alternative phrasings of posts that maintain the underlying views of an individual while moving towards more constructive community engagement. to this end, we make our system available for public use foot_1 . ## prior work one can partially frame the problem of reducing toxicity while retaining semantic content as a style transfer problem. tokpo and calders (2022) perform style transfer to mitigate bias training on nonparallel texts by mapping from the latent space of biased text to non-biased text. @xcite explores the efficacy of zero-shot, one-shot, and few-shot prompts for style transfer. adversarial approaches @xcite have also shown strong results when applied to parallel data. however, these methods tend to prioritize fluency and sometimes formality, which could be seen as inauthentic in social media discussions. several recent studies have explored various aspects of large language models (llms) and their applications to similar problems. sadasivan et al. (2023) investigated the automated detection of llm-generated text and found that such detection can be obfuscated by paraphrasing the llm output using a lighter t5 model. @xcite demonstrated that gpt-3 provides an inherent level of "factualness" and "genericity" when summarizing collections of reviews. moreover, the adaptability of chatgpt to different cultural contexts was assessed by @xcite . they determined that while chatgpt is capable of adapting to an american cultural context, it encounters difficulties with other cultural contexts. this limitation poses challenges when attempting to adapt the model to individual communities, especially those that are orthogonal to each other, such as r/prochoice and r/prolife. @xcite examined the capabilities of llms in zero-shot learning scenarios. while llms exhibit impressive zero-shot learning abilities, their findings suggest that fine-tuned models, when combined with tailored prompts, are more effective at generating the desired outputs. this insight is particularly relevant to our detoxification task, where we employed a fine-tuned model with crafted prompts to guide response generation. we observe that major large conversational models (eg chatgpt, bard, claude) are both closed sourced and constantly undergoing improvement, rendering prompts unstable. our approach does not rely on sophisticated prompt engineering and can work on controversial domains with profanity. ## data collection data was gathered from reddit using a data collection and enrichment framework that combined multiple collection methods to ensure coverage over different timescales and moderator activity. we used a combination of the pushshift api foot_2 and the python reddit api wrapper (praw) foot_3 for collecting posts and comments from reddit communities. while pushshift can efficiently return cached comments and posts, it does not provide updated upvote/downvote data -we thus obtain revised scores using praw. additionally, we collect data repeatedly each day to determine which comments and posts have been removed by moderators using the method described in @xcite . we used data gathered from r/prochoice and r/prolife over the span of a year in order to gather contrasting and opposing viewpoints. including posts and comments which were deleted, we gathered 116,293 items from r/prochoice. each comment and post is enriched with off-theshelf tools for classifying text based on emotion foot_4 , sentiment foot_5 , and toxicity (hanu and unitary team, 2020). a summary of each post was also included using a fine-tuned version of flan-t5-xxl foot_6 . for the final experiments a randomly selected training (70%) and test (20%) split was constructed. in order to avoid cross-contamination between training and test data, child posts were only allowed in the training or test set if the parent post was also in the same set. the training set contained 65,292 posts labelled low toxicity and 11,936 posts labelled as high toxicity while the test set contained 18,631 low toxicity posts and 3,435 high toxicity posts. ## methods we conducted a detoxification task aimed at rephrasing posts with high toxicity scores to reduce their toxicity while preserving the author's original intent. to ensure that the appropriate context was provided for generating the target post, we incorporated summaries of parent post and the target post as a part of the prompt for our model. we experiment prompt text parent-child with summaries (pcs) post summary: ?parent_summary. a post: ?parent_post. reply summary: child_summary a reply: parent-child with toxicity and summaries (pcts) post summary: ?parent_summary. a parent_toxicity post: ?parent_post. reply summary: ?child_summary a ?child_toxicity reply: ## future work while augmenting and guiding response generation with summaries was essential in binding the output to the original text, summarization alone is insufficient in maintaining deeper meaning. certain opinions or statements can lead to uncivil discourse in a community even absent profanity or negative language directed towards individuals. we aim to address this using additional guidance in the form of stance predicates @xcite to improve the faithfulness of detoxified posts to their original text.