C hat GPT for Suicide Risk Assessment on Social Media: Quantitative Evaluation of Model Performance, Potentials and Limitations (2023)

Abstract: This paper presents a novel framework for quantitatively evaluating the interactive ChatGPT model in the context of suicidality assessment from social media posts, utilizing the University of Maryland Reddit suicidality dataset. We conduct a technical evaluation of ChatGPT’s performance on this task using Zero-Shot and Few-Shot experiments and compare its results with those of two fine-tuned transformer-based models. Additionally, we investigate the impact of different temperature parameters on ChatGPT’s response generation and discuss the optimal temperature based on the inconclusiveness rate of ChatGPT. Our results indicate that while ChatGPT attains considerable accuracy in this task, transformer-based models fine-tuned on human-annotated datasets exhibit superior performance. Moreover, our analysis sheds light on how adjusting the ChatGPT’s hyperparameters can improve its ability to assist mental health professionals in this critical task.

## introduction according to the world health organization (who) 1 , more than 700,000 people die due to suicide every year. for every suicide, there are many more people who attempt suicide. furthermore, suicide is the fourth leading cause of death among 15-29 year-olds. according to the us centers for disease control and prevention (cdc) 2 , the rate of suicides per 100,000 increased from 13.5 in 2020 to 14.0 in 2021. social media platforms are becoming a common way for people to express their feelings, suffering, and suicidal tendencies. one of the most effective methods recommended by the who for preventing suicide is to obtain information from social media and report suicidal ideation to healthcare providers 1 the world health organization 2 the us centers for disease control and prevention to enable early identification, assessment, and follow-up with affected individuals. hence, social media provides a significant means for obtaining information to identify individuals who are at risk of committing suicide, allowing for timely detection and intervention @xcite in recent years, there has been a growing interest in using natural language processing (nlp) techniques for suicide prevention @xcite . researchers have developed suicide detection systems to analyze and interpret social media data, including text data. by detecting linguistic markers of distress and other risk factors, these systems can help identify individuals with a risk of suicidality and provide early interventions to prevent such incidents @xcite . nlp techniques, therefore, offer a promising avenue for suicide prevention efforts, enabling more proactive and effective interventions to support those in need. this paper investigates the strengths and limitations of chatgpt, an advanced language model created by openai @xcite , as a tool for suicidal ideation assessment from social media posts. the chatgpt api provides access to a powerful natural language processing tool that can generate human-like text, answer questions, and perform a variety of other languagerelated tasks. with chatgpt, developers can build conversational interfaces, chatbots, and virtual assistants to interact with users and provide informative responses. however, some studies have highlighted the potential risks and ethical concerns associated with the use of chatgpt and other language models in sensitive domains, such as mental health and suicide prevention @xcite . therefore, it is crucial to carefully evaluate the use of chatgpt in such settings to better appreciate its potential and limitations. our two research questions to assess the reliability of chatgpt in the suicide prevention task are as follows: the main contributions of this study are as follows: these contributions provide a comprehensive evaluation of chatgpt's performance in this critical application domain and highlight it's potential to be used to assist suicide prevention experts. our code is available at github 3 . ## background and related work in this section, we review the related work in suicide ideation detection as well as the generative language technology of the chatgpt model. ## dataset we utilize the university of maryland reddit suicidality dataset(umd) @xcite , which is collected from the reddit platform. reddit is an online website and forum for anonymous discussion on a wide variety of topics. it is made up of millions of collective forums or groups called subreddits, including the depression foot_0 and sucidewatch foot_1 subreddits. the umd dataset is a collection of reddit posts and comments created by individuals who expressed suicidal thoughts or behaviors. the dataset contains over 100,000 posts and comments collected from various subreddits, including those related to mental health and suicide prevention, such as "r/suicidewatch". the data was collected over a period of several years and includes the content of the post and comments as well as the location and timing of the posts. umd has been repeatedly used by researchers to develop and test natural language processing algorithms and machine learning models to identify and analyze patterns in online communication related to suicide risk @xcite . @xcite proposed a method for improving text representation through the incorporation of sentiment scores based on lexicon analysis and latent topics. additionally, they introduce the use of relation networks for the detection of suicidal ideation and mental disorders, leveraging relevant risk indicators. @xcite utilized two pretrained masked language models, mentalbert and mentalroberta, specifically designed to support machine learning in the mental healthcare research field. the authors assess these domainspecific models along with various pretrained language models on multiple mental disorder detection benchmarks. the results show that utilizing language representations pretrained in the mental health domain enhances the performance of mental health detection tasks, highlighting the potential benefits of these models for the mental healthcare research community. this dataset contains annotations at the user level, utilizing a four-point scale to indicate the severity of the suicide risk: (a) no risk, (b) low risk, (c) moderate risk, and (d) high risk. according to @xcite , the dataset is divided into three subsets, each containing annotations for a distinct task. in this study, we utilized the subset designated for task a. this task focuses on risk assessment and involves simulating a scenario in which an individual is suspected to require assistance based on online activity, such as posting to a relevant forum or discussion (e.g., r/suicidewatch). the objective of the task is to evaluate the individual's risk level based on their online activity. this task requires minimal data, with each user typically having posted no more than a few times on suicidewatch. ## methodology this paper evaluates the ability of chatgpt to predict the level of suicidal ideation on the umd dataset @xcite and compares it with two fined-tuned classifiers. ## results in this section, we present the results of our study in accordance with the research questions presented in section 1 based on the test set described in section 3. for evaluation, we report four widely-used metrics in this task, accuracy, precision, recall, and fscore to provide a comprehensive and informative evaluation of the performance of the classification models @xcite . for chatgpt, we also report the inconclusiveness rate described in section 4. zero-shot learning: in this section, we present the results of chatgpt for suicidal ideation prediction with the zero-shot learning approach. the goal of our project is to evaluate the performance of chatgpt in assessing the level of suicidality of a written text. furthermore, we use five different temperature values to evaluate the impact of temperature on generated response, and report the inconclusiveness rate of chatgpt at each temperature. the rest of the metrics are used to evaluate the performance of chatgpt for the instances in that chatgpt was able to generate a conclusive answer. as presented in for further evaluation, we present the precision-recall (pr) graph of the model at each temperature. the pr graph displays the trade-off between precision and recall for different thresholds used to classify instances. figure 1 shows the pr curve of chatgpt for each class. moreover, it shows the impact of increasing temperature values on predicting the suicidality of the text in each class. as the temperature increases, the area under the pr graph declines. in other words, the graph shows lower values for both precision and recall measures. moreover, figure 1 shows that the average precision (ap) of chatgpt in predicting the no risk and high risk classes is higher, compared to the two middle classes, low risk and moderate risk. the bar chart depicted in figure 2 illustrates which classes are more challenging for chatgpt for suicidality assessment. figure 2 shows that at the temperature of 0.1, 3 out of 5 inconclusive instances belong to moderate risk and 2 out of 5 instances belong to low risk categories. few-shot learning: we use prompt engineering to implement few-shot learning with chatgpt. the prompt consists of a few examples from the training dataset, and the model is trained to assess the suicidality level of the text based on the given figure 2: number of instances for which chatgpt refrains from making a decision, at different temperature values and for classes no risk, low risk, moderate risk, and high risk criteria. similar to zero-shot learning, we begin by providing the definitions of each risk category in the prompt, followed by eight training examples and their corresponding labels, with each example and label being separated by a comma and placed in individual paragraphs. the prompt concludes with a request for chatgpt to provide an assessment based on the given criteria. chatgpt achieves the highest f1-score at the temperature of 0.1. furthermore, we observed that the inconclusiveness rate of chatgpt in few-shot learning was significantly lower compared to zero-shot learning. additionally, the inconclusiveness rate remained almost constant at different temperature values, indicating that chatgpt is more confident in generating responses when it is provided with a few examples. figure 3 presents the pr graph of chatgpt for two extreme temperature values.as presented in ## discussions and conclusion this study focuses on the evaluation of the accuracy and quality of response generated by chatgpt for the assessment of suicidal ideation levels. the performance of chatgpt was assessed in zero-shot and few-shot learning scenarios. zero-shot learning can be particularly useful when obtaining labeled data is difficult or expensive. in zero-shot learning, chatgpt achieved an f1-score of 0.73, on our test set (@xmath0.1). these findings demonstrate the potential of chatgpt as a tool for data annotation, particularly when utilizing a simple prompt design. however, it is important to note that in sensitive tasks such as suicidal ideation detection or assessment, caution must be exercised to ensure accuracy and ethical considerations should be prioritized. we conducted a few-shot learning experiment to assess the performance of chatgpt when a few labeled examples of the training data are appended to the prompt. we achieved an f1-score of 0.71 in few-shot learning (@xmath1.1). in zero-shot learning, the model is able to leverage its existing knowledge to make predictions for new tasks. this approach can be particularly effective when the model needs to generalize to a wide range of possible new tasks. on the other hand, few-shot learning requires the model to learn from a limited amount of training data for each new task. this approach can be more challenging, as the model has to generalize from a small set of examples and may struggle to identify patterns or relationships that are important for the new task. in this study, we carried out an experiment to examine the impact of the temperature hyperparameter on the performance of chatgpt. in zero-shot learning, our findings indicate that there is a negative correlation between the f1-score and the temperature hyperparameter. in other words, as the temperature increases, the model's performance tends to decrease. these results suggest that careful optimization of hyperparameters, such as temperature, is crucial for achieving optimal performance of chatgpt. in few-shot learning, there is still a negative correlation between the f1-score and the temperature hyperparameter. however, the change in the f1-score value is subtle, indicating that the impact of temperature tuning on model performance may not be significant. another discovery highlighted in this paper pertains to the examination of the inconclusiveness rate of chatgpt. there is a trade-off between the inconclusiveness rate and the f1-score in order to optimize the performance of the chatgpt. in sensitive tasks such as suicide assessment risk, it is crucial to have a highly accurate model that can provide reliable predictions. in some cases, it may be preferable for the model to provide an "i do not know" response rather than providing unreliable predictions about suicidality. careless responses from a suicidal assessment model can have serious consequences, including false positives or false negatives, which can harm individuals at risk. figure 2 indicates that the inconclusive instances mostly belong to two middle classes low risk and moderate risk. these two classes are highly subjective due to the vague boundaries of definitions. for example, the zero-shot learning model was not able to provide an assessment for the following instance: "i have ups and downs, i've had them for a long time and i don't know why, since december i've been going to therapy, i've been getting meds too and at first they helped suppress the storm of thoughts that won't let me sleep eat and think, i keep finding myself trying to sleep". on the other hand, the few-shot learning predicts high risk suicidality level for this instance, and the human experts annotated this instance as low risk. this example clarifies that generating an "i do not know" answer here can be preferable to a wrong assessment. to evaluate how well chatgpt performs compared to other transformer-based models, we conducted an experiment where we fine-tuned two other models, albert and distilbert, with the train set of the umd dataset. the results of this experiment, shown in data collection and annotation are essential but expensive processes in supervised machine learning. obtaining high-quality labels can be specifically costly and time-consuming in sensitive tasks such as suicide detection. based on our results, one possible approach to reduce the cost and increase the quality of data annotation is to use chatgpt in an expert-in-the-loop setting. chatgpt can assist a human annotator in providing faster and more accurate feedback for a given task. for example, in the case of suicide detection, a human annotator can use chatgpt to generate responses to various prompts related to suicidal behavior. the annotator can then review the model's output and provide corrections or feedback to refine the output.