All Models are Wrong, But Some are Deadly: Inconsistencies in Emotion Detection in Suicide-related Tweets (2024)

Abstract: Recent work in psychology has shown that people who experience mental health challenges are more likely to express their thoughts, emotions, and feelings on social media than share it with a clinical professional. Distinguishing suicide-related content, such as suicide mentioned in a humorous context, from genuine expressions of suicidal ideation is essential to better understanding context and risk. In this paper, we give a first insight and analysis into the differences between emotion labels annotated by humans and labels predicted by three fine-tuned language models (LMs) for suicide-related content. We find that (i) there is little agreement between LMs and humans for emotion labels of suicide-related Tweets and (ii) individual LMs predict similar emotion labels for all suicide-related categories. Our findings lead us to question the credibility and usefulness of such methods in high-risk scenarios such as suicide ideation detection.

## introduction each year over 700,000 people die by suicide worldwide, where for each suicide there are many more attempts foot_0 and often numbers are underestimated due to under-reporting or misclassification foot_1 . however, the majority of affected people also deny having suicidal thoughts when asked by a mental health professional @xcite . in recent years, there has been tremendous growth in using natural language processing (nlp) to not just identify but also understand suicidal behavior. many works have looked at developing methods to detect suicidal ideation with varying degrees of success and applicability to the real-world. nlp methods have been utilized to identify relevant features and more recently language models (lms) have shown remarkable performance on a variety of tasks. the widespread availability of lms via huggingface foot_2 has also enabled researchers to make quick emotion and sentiment predictions. using information available on social media, sentiment analysis has been used to detect early signs of suicidal ideation and prevent suicide attempts (). one drawback of such an approach in suicide ideation is that there is no 'quality check' to ensure that emotion and sentiment labels are correct. this may be specifically dangerous in critical applications such as suicide ideation detection. research gap: state-of-the-art methods in suicidal ideation from social media currently focus on binary classification tasks, categorizing posts as either positive or negative sentiment, without considering real-time application scenarios. however, tweets often contain a spectrum of emotions tailored to specific contexts, and the absence of such nuanced analysis can affect the model's ability to identify context, leading to higher rates of false positives and false negatives. there is still a gap in understanding the consistency and robustness of these models in inferring emotions from suiciderelated text. this shows a need for comprehensive examination of diverse sources of tweets related to suicide, a multifaceted approach, and the model's ability to identify embedded emotions in tweets. moreover, this also demands a quality check on the state-of-the-art models which can only identify the presence or absence of suicide-related words or binary sentiments rather than capturing the emotions based on contexts. contributions: in this paper, we examine the results of three lms that are fine-tuned to predict emotion labels from suicide-related tweets from diverse contexts and draw comparisons to human expert's emotion annotations. our main contribu-tions can be summarized as: after exploring the emotional nuances between human annotations and lm predictions in identifying emotions in tweets that contain suicide-related content, our study brings critical insights with broader implications for nlp for mental health: detecting suicide-related language and emotions: detection methods for suicidal intent, ideation, or risk based on deep and machine learning have evolved significantly over the past decades, and various techniques have been employed to enhance model accuracy. traditionally, feature engineering has been a crucial component of these methods, where features extracted from text using dictionaries play a pivotal role in training machine learning models. to overcome these limitations researchers have incorporated human annotation to obtain more finegrained labels, e.g., on risk-levels @xcite , distinctions between worrying language and flippant references to suicide @xcite , content and affect of suicide-related posts @xcite , or from clinical contexts @xcite . several methods have been proposed to detect suicide intent and ideation, including feature-based models with combinations of lexical features @xcite , and psychological and affective features @xcite . work at the intersection of sentiment analysis and suicide has looked at augmenting neural networks with emotional information for ideation detection, @xcite , introduce both psychological and affective features @xcite or distinguishing suicide notes from other types of content @xcite . in @xcite , a joint learning framework has been proposed with an additional knowledge mod-ule and claimed to have the highest cross-validation score. @xcite explored the accumulated emotional data from blogs and examined these emotional traits that are predictive of suicidal behaviors. ## methods in this section, we detail the dataset description and composition, annotation process, and annotation categories present in the tweets. ## experimental results and discussions in this section, we provide a detailed description of the lms employed in our study, along with their predictions and comparisons with human annotations. additionally, we also extend our discussion of tweets that are misclassified by all the lms in ecosystem analysis. choice of language models: language models including bert @xcite , roberta @xcite , and distilbert @xcite , have been proven effective in detecting suicidal ideation from social media platforms like twitter @xcite . these models have shown superiority over traditional rnn-based methods and have proven robust performance in health surveillance tasks from tweets. the goal of this study is to understand the ability of models that were trained on a more general corpus when attempting to infer emotions from suicide-related texts. therefore, we have chosen three lms: distilbert foot_4 , twitter roberta foot_5 , and distilroberta (hartmann, 2022), because they contain the closest matching emotion labels, are most frequently downloaded, and have been trained on similar data (e.g., tweets), and fine-tuned on similar applications. we fine-tune each lm to predict a single emotion label per tweet. in ## conclusions in this work, we explored the variance between emotions annotated by humans and those predicted by language models from suicide-related tweets. we found that (i) across all three lms there was limited consensus among models and between models and humans, (ii) lms make the same predictions for minority categories that are related to suicide, (iii) the models are biased towards certain emotions in most of the categories, and (iv) the models cannot identify the correlation between emotions and psychological process for certain instances, that are prone to systemic failure as evidenced by liwc breakdown. this enforces the shortcomings of lms in mirroring the human cognitive abilities in comprehending the context of tweets and shows that there is an increased need for a 'quality check' when using ai-powered solutions in critical and sensitive application areas such as mental health. limitations and future directions this is a first study on using lms to predict emotions in critical applications (e.g.: suicide-related content) and there are several limitations: (i) the emotion labels do not align across all lms and with the original corpus, (ii) the dataset itself is relatively small and an analysis over other similar dataset would be beneficial to see if these initial findings generalize across datasets, and (iii) we only used a limited number of lms and a comparison across more systems would be helpful to validate initial patterns. however, distil-roberta aligns fully with twisco's annotated emotion labels and also failed to capture the emotional content compared to human annotations. therefore, we would like to see future research to further (i) investigate if these patterns generalizes over different datasets, (ii) include multiple other lms into ecosystem analysis, and (iii) conduct a more in-depth qualitative ecosystem analysis over multiple label categories. in addition to this, we would like to uncover the rationale behind the variations in distributions observed across the lms, incorporating explainability across various categories and lms would be a potential way to comprehend the emotion distribution disparities. finally, providing external guidance to make lms aware of the context of tweets would be an interesting dimension to explore.