Say ‘ YES ’ to Positivity: Detecting Toxic Language in Workplace Communications (2021)

Abstract: Workplace communication (e.g. email, chat, etc.) is a central part of enterprise productivity. Healthy conversations are crucial for creating an inclusive environment and maintaining harmony in an organization. Toxic communications at the workplace can negatively impact overall job satisfaction and are often subtle, hidden, or demonstrate human biases. The linguistic subtlety of mild yet hurtful conversations has made it difficult for researchers to quantify and extract toxic conversations automatically. While offensive language or hate speech has been extensively studied in social communities, there has been little work studying toxic communication in emails. Specifically, the lack of corpus, sparsity of toxicity in enterprise emails, and well-defined criteria for annotating toxic conversations have prevented researchers from addressing the problem at scale. We take the first step towards studying toxicity in workplace emails by providing (1) a general and computationally viable taxonomy to study toxic language at the workplace (2) a dataset to study toxic language at the workplace based on the taxonomy and (3) analysis on why offensive language and hate-speech datasets are not suitable to detect workplace toxicity.

## introduction studies have shown that more than 80% of the issues affecting employees' productivity and satisfaction are related to negative work environment behaviors such as harassment, bullying, ostracism, gossiping, and incivility @xcite . moreover, workplace gossiping results in distracted * most of the work was done while the first author was an intern at microsoft research figure 1 : an example of workplace communication. the highlighted sentence was annotated as toxic and gossip by annotators. this instance has a confidence score of 0.15 on perspective api foot_0 employees and low morale. @xcite and @xcite find that workplace incivility leads to social undermining of employees which could lead to trust issues, difficulty in establishing cooperative relationship, lower job satisfaction and attitudinal outcomes such as gaining personal power and reputation @xcite @xcite . many organizations enact policies that prohibits practicing extremely toxic behaviors like bullying, verbal threats, profanity, harassment and discrimination; yet detecting more subtle forms of toxicity like negative gossiping, stereotyping, sarcasm, and microaggressions in conversations remains a challenge. toxicity can be manifested in different ways. it spans a wide spectrum that includes subtle and indirect signals; that can often be no less toxic than overly offensive language @xcite . while the research community has made enormous progress in detecting overly offensive language and hate speech @xcite @xcite , there has been less focus on computationally evaluating other subtle expressions of toxicity. qualitative studies have found these subtle signals to have long lasting negative effect @xcite . as figure 1 shows, currently popular toxicity detection tools cannot detect subtle yet hurtful conversations as harmful. we argue that it is equally important to detect these subtle aggressive conversations and educate employees for a healthy workplace. detecting wider aspects of toxic text can be challenging. subtle signals like stereotyping, mild aggression can be contextsensitive, sparse, highly subjective and do not have well defined annotation guidelines; whereas overly toxic language and hate speech are rarely contextsensitive @xcite and have welldefined guidelines @xcite . in this paper, we take first steps towards (1) defining a taxonomy for studying toxic language in workplace setting by analyzing the definitions from impoliteness theory and psychology (2) building a dataset of human annotations on publicly available email corpus (3) providing computational methods to establish baselines for detecting toxic language in enterprise emails, and (4) analyzing why current datasets and tools for detecting hate speech do not work in our setting. ## related work offensive language detection: perspective api is a popular toxicity detector for detecting offensive conversations. @xcite devised a taxonomy and created a dataset to detect hate speech and discrimination. @xcite studied bullying, @xcite released a dataset to study bullying in online posts, and @xcite released a corpus for offensive posts named offenseval which has been encouraging researchers to study offensive contents. recently, safi samghabadi et al. (2020) released a dataset with emojis for identifying sexually profane language and @xcite showed joint model of emotion and abusive language detection helps model performance. however, toxic language in workplace has often subtle aggressive conversations and lesser offensive text. subtle aggressive conversations can be covert faux pas or unintentional whereas offensive text is overt and includes intentional choice of words. also, a conversation in a workplace is more formal than the social media text. due to their fundamental different structure, current datasets and models trained on these datasets are not able to properly detect workplace toxicity. ## toxicity in enterprise email our goal is to study and understand workplace toxic communications through one of the most frequently used ways of communication in organizational settings, emails (the radicati group, 2020). the distribution of our dataset (section 3.2) demonstrates the significant presence of the implicit and subtle toxic language in workplace email communications contrary to social media and open source communities. we created a taxonomy (section 3.1) and a crowd sourced annotation task (section 3.2) to manually annotate toxic language in the avocado research email collection @xcite . this collection contains corporate emails from an information technology company referred to as "avocado". the collection contains an anonymized version of the full content of emails, and different meta information from outlook mailboxes of employees' emails. the full collection contains 279 employees and 938,035 emails. in addition, we perform analysis of different emotional affects for each category of toxic language. from previous work, we understand that toxic language has a strong correlation with negative emotions. we also studied whether using context was beneficial in determining toxicity. to this end, we conducted an analysis to study whether humans benefit from context in detecting toxic language in emails. we assume that to determine toxicity in a text, humans read the entire email body and previous emails and not only the given text. we quantify these observations through annotations before using context aware representations in our modeling. ## detecting toxic conversations in emails we design our experiments with the following goals: (1) investigate if contextual information (email body, the parent email) helps in determining toxicity. we also study which categories of toxic language benefit from adding context to the sentence. (2) we also test our hypothesis that current toxic language datasets cannot identify indirect aggressive or impolite sentences. we consider current state-of-the-art toxic language detectors for this task. (3) evaluate our baseline models on other datasets including wiki comments @xcite and github @xcite to study if understanding subtle signals help in determining overly toxic language. we experimented with publicly available state- ## results and analysis generalization to other domains: to investigate how other domains can lever our dataset, we trained the baseline models for toxic language detection @xcite and context aware sentence classification @xcite on toxiscope. then, we tested these models against different toxic language datasets. since we did not find any dataset studying toxic language in workplace (with implicit and explicit toxic text), we picked the datasets that overlap with one or few categories of our interest. the results are presented in leveraging social media and open source communities data to detect workplace toxicity: offensive language is widely studied on social media language and there are several datasets and methods available for this task. tables 8 presents the performance of the publicly available models and api foot_2 on toxiscope. the model from @xcite has a reasonable performance on toxiscope. their method uses lexicons for microaggressions from external sources. leveraging these external sources as weak supervision signals might help in boosting performance of models for toxiscope as well. next, we investigated if these datasets can be helpful in training models for detecting workplace toxicity. we fine-tuned and trained bert based models over microaggression, github, and wiki comments and ran the inference on toxiscope. as we expected, ## conclusion previously, we saw a gap in available resources to detect workplace negative communications and based on our observations, microaggression dataset was the only resource applicable to this domain which did not show promising performance. hence, we created toxiscope to close this gap. we presented a taxonomy and annotation guidelines to study toxic language in workplace emails. we also provided baseline methods to detect toxic language in toxiscope. further, we demonstrated the necessity of new dataset to detect workplace toxicity since the models trained on existing overly toxic datasets and on microaggression dataset do not detect subtle toxic text. in addition, we observed that context help bert based models to detect subtle toxic sentences. however, our results indicate that we need more sophisticated models and better representation of context to detect implicit toxic sentences. in future, we will explore other methods like weak supervision from other sources and going forward, we will also investigate other research questions pertaining to the likelihood of an individual using toxic language repeatedly, correlation of power and gender dynamics with respect to toxicity, presence of the bias (racial/gender) in toxiscope, understanding the degree of severity of toxic text. we hope our work will encourage the researchers in the community to study and develop methods to detect workplace toxicity.