Incorporating Word Count Information into Depression Risk Summary Generation: INF @ U o S CLP sych 2024 Submission (2024)

Abstract: Large language model classifiers do not directly offer transparency: it is not clear why one class is chosen over another. In this work, summaries explaining the suicide risk level assigned using a fine-tuned mental-roberta-base model are generated from key phrases extracted using SHAP explainability using Mistral-7B. The training data for the classifier consists of all Reddit posts of a user in the University of Maryland Reddit Suicidality Dataset, Version 2, with their suicide risk labels along with selected features extracted from each post by the Linguistic Inquiry and Word Count (LIWC-22) tool. The resulting model is used to make predictions regarding risk on each post of the users in the evaluation set of the CLPsych 2024 shared task, with a SHAP explainer used to identify the phrases contributing to the top scoring, correct and severe risk categories. Some basic stoplisting is applied to the extracted phrases, along with length based filtering, and a locally run version of Mistral-7B-Instruct-v0.1 is used to create summaries from the highest value (based on SHAP) phrases.

## introduction with the ability to use large language models (llms) to classify people's suicide risk level comes the need for transparency: artificial intelligence (ai) has been known to learn incorrect patterns and make incorrect generalizations @xcite . to this end, especially in a sensitive domain such as mental health, insight into the reasons for the prediction made is required to allow an expert to look through the output and correct it as needed. in this work, we employ shap values to extract phrases contributing to the llm's decision regarding suicide level risk which we further summarize using locally run generative ai to offer an explanation for the suicide risk level assigned. the clpsych 2024 shared task @xcite explores the use of llms in order to find evidence within text supporting an assigned suicide risk level. the university of maryland reddit suicidality dataset version 2 dataset, which was made available to participants, contains user-linked posts from reddit annotated for level of suicide risk labelled on a four point scale (no risk, low, moderate, and severe risk) as described in @xcite and @xcite . the evidence supporting the risk level could be supplied in one of two ways: for the first task, we fine-tune a pre-trained reddit based mental health model for suicide risk level classification and extract shap value based phrases which represent the highest contributors to the decision. for the second task, a subset of the phrases extracted from the first task is used as part of a prompt to a generative ai algorithm which is instructed to produce a summary focusing on the aspects highlighted in the task definition, namely: emotions, cognitions, behaviour and motivation, interpersonal and social support, mental health related issues and additional risk factors. ## related work the approach is composed of two distinct phases: (1) fine-tuning of a suicide risk classifier, and (2) generation of a summary. the work also explores the integration of additional psycholinguistic based information and transparency via explainability. ## method the creation of short summaries describing the mental health state and depression risk of users is of a two step design: (1) a deep learning classifier is built for risk level prediction, which provides access to important phrases in each post, and (b) a subset of such phrases is summarized by a generative ai system. ## results and discussion optimization, with early stopping, was performed over learning rate, the quantity of data used in training and inclusion or exclusion of liwc information. the training data was balanced and the best performing model, at 51% (over a balanced evaluation dataset which included all 4 classes), was found to be using expert data only with liwc information included with a learning rate of 2 -e6. the pipeline, starting from risk level classification, through extraction of important phrases and ending with summary generation, was run for all 125 users in the evaluation set. while important phrases were extracted from all posts, only the highlights that were used in summary generation were submitted, alongside summaries, resulting in some submissions having empty highlights for specific posts (but having a non empty summary, as this was generated from posts which were deemed more informative). fourteen users were therefore submitted with empty highlights for at least one post (21 posts, out of 166 posts, in total): this affects the overall metrics for the system shown in after the competition, manual analysis was performed of the summarized evidence and the extracted highlights. note that the official summaries and highlights were not released, so the results presented are only our judgements. the evaluation set contained 125 users: 39% of the submitted summaries were complete sentences summarizing the requested aspects of its inputs, 39% were also good summaries, but rather than sentences, they consisted of lists (such as "emotions: hopelessness, loneliness"). 4% answered each point with an exact quote from the shap phrases and 8% were a mix of quotes from posts and generated text. also relevant were 3% of summaries which in addition contained information which wasn't linked to the required points -such as a basic sentence containing only the person's age (e.g. "they are 30 years old."). the remaining summaries were either partial (2%), or probably too general, appearing to outline importance of the various aspects for suicide risk evaluation. only one summary was nonsensi-cal and all summaries were within the permitted length, with a mean of 85 words. since the summarized evidence is generated from shap extracted phrases, 25% of these (136 highlights) were also manually explored. while 88% appeared ok (in this we include highlights which were not clearly supporting suicide risk judgement alone, but they complemented other selected highlights), a large portion contained fragments within the highlight: such as portions of a previous or following sentence, or ending at a point where is was clear how the fragment continued but with the end missing (e.g. ". . . one way or"). some fragment highlights were also not entire sentences from the original post, but they were a self contained sentence. the remaining sources of error were either fragments that were too short to carry enough meaning (5%), fragments that -due to their selection -were inconsistent (1%), and highlights which didn't appear pertinent to the assessment of someone's risk of suicide (6%). ## conclusion and future work we have shown the utility of shap explainability for the extraction of important phrases from text for the purpose of transparency within a text based suicide risk level classifier. mistral 7b performs well for summary generation in this domain, retaining text integrity and producing minimal hallucinations. further investigations are required as to the contribution of the liwc tool to the changes of shap extracted phrases, alongside a comparison with a one step (rather than the two step, shap + mistral) summary generation process. ablation tests evaluating changes to each step of the pipeline would also bring more insights. in future work, the quality of highlights selected by shap could be improved by ensuring complete sentences surrounding the highlight are extracted.