Towards Intention Understanding in Suicidal Risk Assessment with Natural Language Processing (2022)

Abstract: Recent applications of natural language processing techniques to suicidal ideation detection and risk assessment frame the detection or assessment task as a text classification problem. Recent advances have developed many models, especially deep learning models, to boost predictive performance.Though the performance (in terms of aggregated evaluation scores) is improving, this position paper urges that better intention understanding is required for reliable suicidal risk assessment with computational methods. This paper reflects the state of natural language processing applied to suicide-associated text classification tasks, differentiates suicidal risk assessment and intention understanding, and points out potential limitations of sentiment features and pretrained language models in suicidal intention understanding.Besides, it urges the necessity for sequential intention understanding and risk assessment, discusses some critical issues in evaluation such as uncertainty, and studies the lack of benchmarks.

## introduction warning: this paper contains text examples that are negative, depressive, or adverse. suicide is a global problem, with most industrialized countries and many emerging markets seeing exceptionally high rates. the latest who publication on suicide study, foot_0 entitled suicide worldwide in 2019, reported that more than 700,000 people die by suicide every year around the world, with one suicide case for every 100 deaths. according to a report on mental health from the world health organization, 2 one out of every four persons in the world lives with mental disorders to some extent. and 3 out of 4 people with severe mental disorders do not receive treatment, worsening the problem. previous research has found a link between mental problems and the likelihood of suicide @xcite . more people live with mental health issues during particular periods, such as the pandemic. many may not seek care from mental health practitioners due to insufficient mental health services. social networking sites offer an essential forum for communication and information sharing online. online discussions also include much harmful information and can result in issues like cyberstalking or cyberbullying. the result is severe and dangerous because the wrong information is frequently engaged in social cruelty, causing rumors or even mental harm. research shows a link between cyberbullying and suicide @xcite . victims exposed to too many negative messages or events may become depressed and desperate. some of them are likely to choose suicide as their option and ask for suicide methods on online social network websites @xcite , which is called cybersuicide. furthermore, some social groups even persuade other individuals to commit suicide together, namely cybersuicide pact. thus, it is necessary to understand users' intentions by mining their conversations, modeling their profiles, and analyzing their social groups. on the other hand, social networking also offers a channel for peer support among those living with mental illnesses, allowing social workers to provide proactive social care and early intervention. people experiencing a mental health condition sometimes post their feelings or experience on online discussion forums like reddit or social networking websites such as twitter and weibo. that user-generated content is an essential portal to facilitate automated suicidal risk assessment and allow able via http://www.who.int/mental_health/ action_plan_2013/mhap_brochure.pdf?@xmath0. computational methods using natural language processing techniques have been studied to classify mental disorders and suicide from various data types, mainly social media posts. see the recent review of nlp applied to mental illness detection @xcite . suicidal risk assessment in social media is a task that categorizes given social posts into different levels of suicidal risks. early detection of suicide paves the way for suicide prevention. research on suicide with nlp is booming. suicide attempt or completed suicide usually starts from suicidal ideation. thus, early assessment of the intensity of suicidal ideation and effective intention understanding can help predict later suicidal risk better. a recent review @xcite and a comparative analysis of recent techniques @xcite discussed many publications for suicidal risk classification. for example, some construct sentiment lexicons for posts regarding suicidality and build machine learning classifiers @xcite . recent deep learning-based models build comprehensive neural architectures to improve the classification performance, including attentive relation network @xcite , enhanced word embedding @xcite , transformer networks @xcite , and hyperbolic user representation learning @xcite . however, most existing works that approach suicidal risk assessment as a text classification problem consider less about human intention understanding. human intention or mental state understanding is a complex cognitive process. this position paper argues that suicidal risk assessment with natural language processing needs a better intention understanding of the social posts. it points out several unresolved issues and open questions, including the limitation of sentiment features and pretrained language models, the importance of sequential intention understanding, the critical ingredients in evaluation, and the need for more benchmarks for visual-grounded intention understanding and multilingualism. this paper is organized as follows. section 2.1 highlights the challenges of suicidal intention understanding and differentiates it from suicidal risk assessment. section 3 proposes the setting of intention understanding in sequence (e.g., the timeline of user's posts). we discuss the obstacles of evaluation and benchmarks in section 4 and 5. section 6 reviews additional related work. we conclude this paper in section 7. ## suicidal intention understanding many factors can lead to suicide, i.e., users' personality (such as hopelessness, severe anxiety, schizophrenia, alcoholism, and impulsivity), social factors (like social isolation, too much exposure to deaths), and adverse life events (including traumatic events, physical illness, affective disorders, and previous suicide attempts). due to the prevalence of social networks, online users may be exposed to risk factors such as talking about suicide, worsening mood, and cybersuicide pacts. suicide is also related to other matters, such as access to lethal suicide methods in the physical world. adequate measurement of suicidality is vital to assessing people at risk for suicide attempts. the scale of suicidal ideation (ssi) @xcite , one of the classic measures of suicidality, develops a 19-item instrument for quantifying suicidal intention in clinical research. its variables include race, age, education, civil status, employment status, and psychiatric diagnosis. as an instrument of clinical psychology, ssi was found capable of assessing the subtle changes in levels of depression and hopelessness. whether the nlp model can mimic clinical professionals' screening practice and understand the inherent intention of people living with mental conditions and suicidality becomes a challenging problem. this section discusses intention understanding foot_2 in suicidal risk assessment with natural language processing. ## intention understanding in sequence the suicidal ideation could be chronic, including duration and strength at different stages. more recent works focus on identifying people with suicidal risks based on a single post or a collection of posts, i.e., post or user classification. for example, several works conducted user-level suicidal risk classification @xcite . however, people's mood changes over time @xcite and the level of suicidal ideation experiences fluctuations as time goes by @xcite . the communicative nature of social networks and the chronic nature of suicide ideations require nlp models to detect people's suicidal intentions in post sequences or over online conversations. recent research developed temporal-aware models to represent the sequence of social posts and predict the social user's suicidal risk. for example, historic tweets are used to improve the prediction of an individual's suicidal risk with time-aware model @xcite and phase-aware model @xcite . the retrospective evaluation designated for a specific period fails to monitor the dynamics of suicidal ideation. we need longitudinal suicidal risk monitoring over social posts' timelines, enabling dynamic social care and timely clinical intervention. figure 5 shows a pseudo case of suicide severity assessment in post sequence (similar to sequence labeling for social posts), where an individual post at each time stamp is associated with a severity scale and the severity changes when the user lives with various mental conditions in the timeline or 9 the whole sentence does not make sense. some kind of intervention such as social support or clinical intervention has been taken. higher levels of suicidal risk are more critical than low risks, and different levels of suicidal risks should be allocated with different clinical resources. dynamic detection can, in a timely manner, prioritize people needing better mental healthcare resource allocation. it can also integrate with the human-in-the-loop system for more reliable suicidal intention understanding. the ordinal relationship of the severity scale contains critical information for fine-grained classification. @xcite approached user-level suicidal risk assessment as an ordinal regression problem. the ordinal constraints and sequence dependency between time stamps play an essential role in the sequential modeling of suicidal risk monitoring. ## evaluation of suicidal risk assessment an evaluation must be carefully considered when computational approaches are used in suicide research. most publications on suicidal risk assessment or suicidal ideation detection use aggregated evaluation scores such as accuracy, f1 score, and auc-roc to evaluate the predictive performance. the sensitivity or recall, calculated by the ratio of true positives to the total positives, is an essential metric that needs great attention because some lowrisk cases might be deprived of social care and die by suicide if a model has a low recall. for some safety-critical applications like suicidal risk prediction, incorrect predictions may cause ethical costs and even the loss of life. thus, uncertainty estimation becomes an important task. dusenberry et al. (2020) analyzed the model uncertainty in medical applications of mortality and diagnosis prediction. no current suicidal ideation detection literature estimates data (aleatoric) or model (epistemic) uncertainty. epistemic uncertainty is required to understand examples different from training data for the life-critical task @xcite . intelligent intention understanding systems should be able to assign how confident is the model prediction in some erroneous predictions of fine-grained suicidal ideation. human uncertainty and disagreement in annotation are ubiquitous. we need to learn with disagreement @xcite . learning with humansin-the-loop (zanzotto, 2019) can also be utilized to select more reliable instances or samples with less ambiguous intentions. ## benchmarks a recent survey on mental health research @xcite reviewed studies published between january 2012 and december 2019 in conferences, workshops, and journals focusing on nlp and healthcare research. this survey investigated the availability of mental health-related text data, as shown in figure 6a . a review on suicidal ideation detection @xcite investigated thirteen published datasets, with the availability shown in figure 6b . due to the sensitivity of suicide data, all available datasets require data usage agreement or need to be requested with the authors' permission. more are unavailable, removed, or out of maintenance among those published datasets. most datasets contain binary classes (i.e., a positive class with suicide messages and a negative class from control groups). these datasets simplify the setting. the severity of suicide and suicidal ideation are fine-grained. the scale for suicide ideation introduces a fine-grained suicidal risk rating scale to measure the intensity of suicide in interview-based screening @xcite . the self-monitoring suicide ideation scale (clum and curtin, 1993) designed for self-report measures on a daily basis includes the intensity and duration of ideation and the level of control in making a suicide attempt. check out the review by @xcite for more suicide assessment measures. we need more datasets like @xcite and more annotation about intention and ideation for special social care. most datasets contain self-reported posts from social websites like reddit, where users seek peer support or express personal feelings or experiences. @xcite is a remarkable dataset of suicide events from electronic health records in the mimic iii database @xcite by filtering the suicide-associated icd codes and introducing domain experts' annotation. it consists of suicidal behavioral evidence of suicidal attempts and ideations, where suicidal ideation is defined as text mentions about self-harming or taking one's own life in clinical notes. social posts are not as reliable as clinical notes written by clinicians. current distantly supervised mental health models lack the generalization ability across different domains (e.g., data platforms and populations) @xcite . moreover, to our knowledge, no datasets have been introduced from the perspective of intention understanding. finally, we still lack benchmarks for visualgrounded intention understanding and multilingualism. a picture of cutting the wrist can be suicidal intention or sharing self-harm behavior of other people. with social text, the visual signal can help better understand the intention. many datasets were collected from reddit and weibo, which are used mainly in the us and china. figure 7 shows that english datasets dominate the field of mental health research, which may cause demographic biases. ## additional related work suicidal detection has drawn much research attention since the suicide rate has increased these years. the cause of suicide is complicated and related to many complex interactions of risk factors (o'connor and nock, 2014). feature engineering-based nlp models use n-gram features, knowledge-based features, syntactic features, context features, and class-specific feature @xcite . machine learning classifiers include regression analysis @xcite , boosting, and svm classification @xcite . @xcite proposed a model to identify signs of distress in transcripts of helpline conversations. @xcite and @xcite compared the effectiveness of several multivariate methods. other research and methods have been conducted to stop cybersuicide such as speech pattern recognition, mobile phone network analysis, social media content detection @xcite , and reply bias assessment for online suicide prevention (huang and bashir, 2016). however, there was not much discussion of suicide intent in those works. intention classification is helpful for commercial applications such as target advertising in social media @xcite . one of the early studies by @xcite formulates the intention identification from posts in multi-domain discussion forums as a binary classification task (i.e., explicit intent and non-intent posts with a specific focus on buying intention). similarly, @xcite classified purchase intention in questionand-answer websites. @xcite studied a multi-class intention classification on social posts on twitter social networks and proposed the def-inition of the intent tweet if a tweet follows the following three conditions: 1) containing at least one verb; 2) with an explicit description of the user's intent to perform an activity; 3) in a recognizable way. however, this definition does not fit early detection of suicidal intention because the suicidal intention is usually implicit and early detection requires recognizing suicidal thoughts as early as possible before the suicidal attempt. ## conclusion intention understanding is an essential aspect of suicidal risk assessment. robust intention understanding in suicidal risk assessment is needed to provide efficient triage support to social workers and psychiatrists. this position paper discusses suicidal intention understanding and conducts a case study on sentiment analysis and fine-tuning pretrained language models in the context of suicide research. it further points out critical aspects in the task settings and evaluation and the lack of benchmarks.