APPDIA : A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversations (2022)

Abstract: Using style-transfer models to reduce offensiveness of social media comments can help foster a more inclusive environment. However, there are no sizable datasets that contain offensive texts and their inoffensive counterparts, and fine-tuning pretrained models with limited labeled data can lead to the loss of original meaning in the style-transferred text. To address this issue, we provide two major contributions. First, we release the first publicly-available, parallel corpus of offensive Reddit comments and their style-transferred counterparts annotated by expert sociolinguists. Then, we introduce the first discourse-aware style-transfer models that can effectively reduce offensiveness in Reddit text while preserving the meaning of the original text. These models are the first to examine inferential links between the comment and the text it is replying to when transferring the style of offensive Reddit text. We propose two different methods of integrating discourse relations with pretrained transformer models and evaluate them on our dataset of offensive comments from Reddit and their inoffensive counterparts. Improvements over the baseline with respect to both automatic metrics and human evaluation indicate that our discourse-aware models are better at preserving meaning in style-transferred text when compared to the state-of-the-art discourse-agnostic models.

## introduction disclaimer: due to the nature of this work, figures and examples may contain offensive phrases. the spread of offensive and hateful content on social media can be detrimental to users' psychological well-being @xcite . anonymity on platforms such as reddit can further embolden users to post such hateful content @xcite . further, the sheer volume of content on popular social media platforms can render the human moderation process ineffective @xcite figure 1 : example of instances where pretrained language models either fail to remove offensiveness (bart/t5) or drastically alter the intended meaning (dialogpt) when fine-tuned on our style transfer task et al., 2022) or psychologically damaging for moderators @xcite and calls for ai systems that can mitigate this problem. ai moderation of social media by simply removing content classified as offensive @xcite may reduce diversity in online conversations and deter users from using the platform @xcite . our exploration reveals that many comments removed by moderators on reddit contain contributions to the discourse beyond their offensive content. rather than simply removing these comments from social media platforms, they can be turned into inoffensive statements by using alternative words, removing profanity, or paraphrasing certain parts, while preserving the overall semantic content. we approach the problem of eliminating offensiveness from text while preserving original semantic content as a supervised style-transfer task, where offensive text is transferred to inoffensive text. as a first step towards this goal, we create the first publicly-available, expert-annotated style transfer corpus for reddit data, which contains offensive comments that include certain lexical items and more subtle instances that are implicit and grounded in context. this differentiates our work from unsupervised approaches are mostly good at handling instances with explicit lexical cues @xcite . although large pretrained transformer models have been successfully deployed for generation tasks, these models come with the risk of either failing to generate desired output or obfuscating the source passage's meaning while still producing coherent text @xcite . in our work, we target the issue of content preservation using discourse frameworks, which have been successfully employed for various generation tasks @xcite @xcite , but have not been employed in style transfer models. we hypothesize that integrating discourse coherence frameworks within transformer-based style transfer models can contribute to better preservation of semantic content, specifically for short social media comments. we study our hypothesis with a small pilot annotation of style-transferred text produced by pretrained transformer models. figure 1 shows examples of the issues described above in our style transfer task, where bart @xcite and t5 @xcite do not remove offensiveness from the original comment, but dialogpt @xcite significantly alters the original semantic content. we observe that coherence relations between a comment and its reply are not preserved under style transfer in cases where offensiveness is removed. for instance, the removed comment refers to "emotionally manipulative behavior" in the parent comment with "this is evil", exhibiting the behavior of "same-unit" discourse relation, which is not preserved in the style-transferred text generated by dialogpt. to test our hypothesis, we provide the following contributions: the results for both discourse-aware approaches indicate improvement in content preservation over the pretrained baselines, providing support for our hypothesis and for the use of discourse frameworks to preserve meaning in style-transfer tasks. ## related work paraphrase generation is a well-studied problem that has yielded large datasets such as the pdtb paraphrase database @xcite , wikianswer @xcite , paranmt (wieting and gimpel, 2018), and the mscoco dataset @xcite . recent works in the related but relatively new field of style transfer primarily target sentiment transfer @xcite , formality transfer @xcite or expertise transfer @xcite . very few works have targeted transferring style from offensive to inoffensive text, with nogueira dos @xcite and @xcite being notable exceptions. our dataset differs from the aforementioned works in multiple ways. ours is the first publicly available dataset that contains offensive reddit comments that are rewritten by experts, paired with parent comment/post, and automatically tagged with discourse relations. further, both nogueira dos @xcite and @xcite derive their datasets from political subreddits @xcite , while our data encompasses subreddits on personal views, question-answer discussions, and gender rights in addition to political subreddits. development of pretrained language models (plm) such as bart @xcite has changed the landscape of natural language generation research and we are witnessing a shift toward controllable text generation @xcite @xcite . discourse relations have been proposed as a possible controlled generation method, shown to aid extractive and abstractive summarization @xcite ), text generation from meaning representations @xcite , and question answering with logical reasoning or complex answers @xcite . discourse-aware models have also been shown to generate more coherent texts @xcite within a reinforcement learning setting. our work integrates both rst-dt and pdtb frameworks with pretrained transformer models and provides a comparison of the relative efficacy of the two frameworks for a generation task. within the context of style transfer, recent works have focused on classification and reconstruction loss (nogueira dos santos et al., 2018; chawla and yang, 2020) in semi-supervised/unsupervised setting, use of copy mechanism @xcite , and coherence classifier @xcite to guide the style transferred text. to our knowledge, our work is the first to utilize discourse coherence frameworks for style transfer. ## data collection and annotation in order to reduce offensiveness in text, we create an expert-annotated dataset of offensive comments and their style-transferred counterparts. in this section, we first describe our pipeline for collecting and curating a set of offensive comments from reddit. then, we describe our annotation process for reducing offensiveness in these comments. ## discourse-aware models we propose two approaches for integrating the pdtb and rst-dt discourse frameworks into pretrained transformer models, as described below. pdtb within-comment relations to extract pdtb relations at the comment level, we parse the comment text in isolation, first using the lin et al. 2020 ) xlnet-large model to extract implicit discourse relations (not signaled by a discourse connective) from adjacent sentence pairs. because there is no pdtb-tagged reddit corpus available, we run these models trained on the pdtb-2 corpus. for the l2 classification task (the more difficult of the tasks we run), @xcite report an f-1 score of 80.61, and @xcite report an accuracy of 57.74 (they do not report f-1 for the l2 classification task) on the pdtb-2. we then use the positions of the argument pairs, and their discourse relations, in our input. ## experiments in this section, we first describe the experiments with pretrained transformer models, followed by the experiments with discourse-aware models. ## results below, we describe the results of our experiments. we split our dataset into an 80-10-10 split for training, development, and test sets respectively. we first calculate automatic metrics, reporting the bleu @xcite and rescaled bertscore @xcite on our test set. in addition, we compute the safescore -percentage of style transferred comments predicted as inoffensive by the bert classifier that was initially used to identify potential candidates. further, we ask a human annotator to compare style-transferred text generated by baseline model and our proposed discourse-aware model. ## conclusion and future work in this paper, we have demonstrated that utilizing discourse frameworks and parsing models can help pretrained transformer models preserve original content when transferring style from offensive to inoffensive. we have shown that combining different discourse frameworks can further improve content preservations. the improvements we observe in this paper are significant; however, we hypothesize that utilizing discourse relations for these tasks can be even more impactful if the performance of existing discourse parsers is improved. discourse parsing is a very challenging task @xcite , and the largest and most widely-used corpora are composed of news texts over a short time span. thus, there is a need for further research (and additional annotated corpora) on discourse relations within the context of social media. we hope our publicly available code and data will motivate other researchers to build on the groundwork laid out in this paper. further research is also necessary in the context of style-transferring for offensive text. after further improving these language models and evaluating their safety, future systems that are proven to be robust and effective can potentially help social media moderators or be deployed in a human-in-the-loop or assistive technology capacity. we expect these models to have the potential to not only improve the psychological well-being of users but also to motivate healthy engagement on social media.