Towards Toxic Positivity Detection (2022)

Abstract: Over the past few years, there has been a growing concern around toxic positivity on social media which is a phenomenon where positivity is used to minimize one’s emotional experience. In this paper, we create a dataset for toxic positivity classification from Twitter and an inspirational quote website. We then perform benchmarking experiments using various text classification models and show the suitability of these models for the task. We achieved a macro F1 score of 0.71 and a weighted F1 score of 0.85 by using an ensemble model. To the best of our knowledge, our dataset is the first such dataset created.

## introduction toxic positivity can be defined as the overgeneralization of a positive state of mind that encourages using positivity to suppress and displace any acknowledgement of stress and negativity @xcite . the popularity of the term "toxic positivity" peaked during the covid 19 pandemic (refer to figure 1 ) where it was used to identify advice that focused on just looking at the positive at a time when people were hurting due to loss of life, loss of jobs and other traumatic events. toxic positivity results in one minimizing one's own negative feelings and suppressing negativity instead of acknowledging, processing and working through it. some examples of toxic positivity include telling someone to focus on the positive aspects of a loss, telling someone that positive thinking will solve all their problems, suggesting that things could be worse and shaming someone for expressing negative emotions. this suppression of emotions is not only unhelpful but also leads to poorer recovery from the negative effects of the emotion. accepting and working through one's emotions is the better route to take while dealing with negative emotions @xcite . macro level events like covid 19 and climate change disasters have distressed many people in the past few years @xcite . social media is used by people having mental health issues or going through a tough time to find community, support, advice and encouraging messages @xcite . however, it becomes important to be able to differentiate between messages that may help uplift an individual and those that may look positive but promote suppression of emotions and cause great harm in the long term recovery from negative emotions. the harms of toxic positivity are not only limited to its deleterious mental health outcomes but it can also be used to uphold oppression by making people ignore the oppression that is going on and encouraging them to "just be positive". in this paper, we aim to create a dataset for toxic positivity and perform text classification using various transformer based models to establish the baseline results for this task. ## related work there have been studies that show the ineffectiveness and deleterious effects of emotion suppression. @xcite showed that people who suppressed their emotions had a greater experience of negative emotions while also expressing lesser positive emotion. they also showed that using suppression is related negatively to well being. a study done by @xcite involved dividing 60 participants diagnosed with anxiety and mood disorders into two groups. one group was given a rationale for suppressing their emotions while the other was given a rationale for accepting emotions. it was found that suppression was ineffective in reducing distress while watching an emotion-provoking film. it was also seen that the suppression group showed a poorer recovery from the changes in negative affect after watching the film compared to the acceptance group. a similar observation is seen in the case of physical pain as well. @xcite divided participants into three groups during a cold-pressor pain induction (cpt) where participants would dip their hands in cold water for as long as tolerable. the first group was told to pay attention to the pain, the second was told to focus on their room at home as a distraction, and the third was told to suppress the sensations they felt. it was seen that the group that focused on the pain had a faster recovery from the pain and the suppression group had the slowest recovery from pain. suppressing pain has shown to have negative outcomes, while accepting it is observed to be as a better strategy. @xcite through longitudinal and lab studies showed that habitually accepting mental experiences broadly predicted psychological health and that it reduced negative emotional response and experience. hence toxic positivity, with its overemphasis on thinking positively and having a positive state of mind, encourages emotion suppression rather than emotional acceptance which has negative consequences for the person who engages in it. lecompte-van poucke (2022) conducted a critical discourse analysis of toxic positivity as a discursive construct on facebook. two corpora of posts from organizations that promoted endometriosis awareness (an invisible chronic condition) were analyzed using systematic functional linguistics, pragma-dialectics and critical theory. the study showed that users on social media platforms of-ten engage in toxic positivity or forced positive discourse which is inspired by the neoliberal "positive thinking" ideology, leading to a less inclusive online community. in the field of nlp, there have been many papers focusing on hate speech detection using support vector machine (svm), long short term memory networks (lstm),convolutional neural network (cnn), transformers and other machine learning models @xcite @xcite . these works use twitter posts (tweets) to create datasets. youtube and reddit comments have also been used in some works @xcite . there have been recent efforts in hope speech detection as well @xcite . the hopeedi dataset (chakravarthi, 2020) is a hope speech dataset that contains youtube comments that have been marked for hope and not-hope speech. there has been a shared task on this dataset where participants have used various machine learning models for hope speech detection like multilingual transformer-based models, recurrent neural networks (rnn) and @xcite . however, to the best of our knowledge, there has been no prior work on creating datasets and classification models for toxic positivity. ## methodology we used the following transfomer-based models for text classification: we have fine-tuned the "bert-base-uncased" model in our implementation. we also experimented with an ensemble based classifier for which we additionaly used the following: ## experiments and results we experimented with 3 transformer models bert, roberta, and albert. each of the classification models utilizes a pretrained transformer encoder, i.e. bert-base, roberta-base, and albert-base. the pooled output layer from each encoder is passed through respective dropout layers (@xmath0.3) for further regularization and linear layers (mapping from a vector size of 768 to the number of classification categories, i.e. 2). a softmax function is applied to each of the size-2 vectors for normalized likelihoods of the two classes. the results from these models are provided in we also experimented with an ensemble-based classifier. the classifier is an ensemble of three predictors with a random forest classifier on top (as shown in figure 2 ). the predictors were the three text classification transformer based models as mentioned above. the likelihoods from each of the predictors were concatenated and passed as features to an xgboost random forest classifier to generate an ensemble class prediction. after a bayesian search for the classifier parameters on the validation set, the number of tree estimators w set to 149, subsample ratio of the training samples to 0.50, and subsample ratio of columns for each split to 0.33. each of the transformer encoder predictors were trained using adamw optimizer (β 1 = 0.9, β 2 = 0.999, ϵ = 10 -8 ), with cross entropy loss, using a linear training scheduler. the encoder pipelines were trained with an initial learning rate of 2e -5 and the xgboost ensemble classifier with a learning rate of 1.0. the predictors were trained for 6 epochs . the predictions from the epoch with the best validation weighted macro f1 score were utilized for the ensemble classification. the overall batch size for the pipeline was set to 16. the ensemble model generalized better than the individual models producing the highest macro f1 score of 0.71 and a weighted f1 score of 0.85 as seen in ## conclusion and future work in this work, we created a dataset for toxic positivity detection. we scraped 4,250 sentences from twitter and the inspirational quote website brainyquote. we then annotated them and achieved a kappa score of 0.82 for toxic positivity classification. we then performed experiments using transformer-based models for text classification. our ensemble model gave us the best results achieving a macro f1 score of 0.71 and a weighted f1 score of 0.85. as more people turn to social media to get help when they are going through a tough time, it becomes important for them to be able to differentiate between positive and toxic positive messages. furthermore, being able to recognize toxic positivity is also important for chatbots and other automated systems that aim to provide mental health assistance. we hope that our work contributes to further research in this field. in the future, we plan to extend the study by introducing a larger dataset in english as well as other languages.