Detecting Moments of Change and Suicidal Risks in Longitudinal User Texts Using Multi-task Learning (2022)

Abstract: This work describes the classification system proposed for the Computational Linguistics and Clinical Psychology (CLPsych) Shared Task 2022. We propose the use of multitask learning approach with bidirectional long-short term memory (Bi-LSTM) model for predicting changes in user’s mood and their suicidal risk level. The two classification tasks have been solved independently or in an augmented way previously, where the output of one task is leveraged for learning another task, however this work proposes an ‘all-in-one’ framework that jointly learns the related mental health tasks. The experimental results suggest that the proposed multi-task framework outperforms the remaining single-task frameworks submitted to the challenge and evaluated via timeline based and coverage based performance metrics shared by the organisers. We also assess the potential of using various types of feature embedding schemes that could prove useful in initialising the Bi-LSTM model for better multitask learning in the mental health domain.

## introduction mental illness has greatly affected a vast majority of world's population due to covid-19 and its resulting economic recession. according to the world health organisation (who), global prevalence of anxiety and depression has increased by a massive 25% raising concerns about providing mental health and psychosocial support to the population as a covid-19 response plan 1 . many social media platforms have risen to the challenge by offering space to online users to self report their mental health issues, receive counselling support and resolve their mental health issues. this activity has resulted in two research trends: (1) the surge in development of machine learning algorithms that can automatically detect mental health issues from the language used in social media platforms and (2) the development of new and better diagnostic measures and mental health monitoring tools suitable for the clinical community. most of the research tasks revolve around classifying individuals on the basis of suicide risk or having a mental health condition @xcite , however a few have thought of monitoring individual's mood and mental health in real time @xcite . despite the growing interest in this interdisciplinary space, there are challenges regarding the availability, use and validity of mental health data gathered from social media platforms and decisions drawn from it. this paper describes our work identifying moments of change in user's mood (task a) and suicidal risk level (task b) in the clpsych shared task 2022. we have experimented with several different sentence and word embedding techniques to draw semantically meaningful features for initialising the multitask sequential model. the model utilised for sequential representation of data is bidirectional long short-term memory (bi-lstm) @xcite , trained jointly for multiple tasks (task a and task b). the multi task outputs determine bilstm model post 1 post 3 post 2 post n-1 post n post 1 ie/is/0 post 3 ie/is/0 post 2 ie/is/0 post n-1 ie/is/0 post n ie/is/0 task a (moment of change) timeline i post embedding task b (classification) timeline i user risk multi-head attention layer the moments of change in user's mood as well as assess the level of suicidal risk from their posts. we have participated in two tasks introduced by the organisers: the first task (task a) is to predict the changes in user's mood over time based on the linguistic content gathered from their posting activity shared on online social media platforms. this is a post-level sequential classification task that aims to detect those sub-periods where a user's mood deviates from their baseline mood. sequence of an individual's posts over a time span of two months is collected for this shared task @xcite . the progression in user's mood is categorised as follows: (1) switch (is), which signifies a sudden change in user's mood, (2) escalation (ie), which denotes a gradual shift in user's mood and (3) none (o), denoting no change in user's mood over time. the mood shifting is graded on a scale from positive to negative. this information is further used for task b where user's suicidal risk level is predicted as low, moderate and severe based on the longitudinal mood changes of the user @xcite . the class distribution of the data for each of these labels is shown in in order to tackle data imbalance issues, the 'no risk' and 'low risk' label instances were merged and represented as 'low risk' examples in the data set for task b. the task participants were required to sign data use agreements and abide by ethical practice during the competition. ## methodology this section demonstrates the stages involved in developing the proposed multi-task model for determining moments of change in mood and user's suicidal risk determined through a sequence of posts in user's timeline. figure 1 shows the high-level model architecture for both the tasks. ## experiment and results in this work we have used two different combinations of feature embeddings for the user posts. for the ease of reference, we consider naming them as p emb which is the concatenation of fast-text and sbert embeddings (p f t ⊕ p sbert ) and p task-emb which is the concatenation of fast-text, sbert, and task-specific scores of the post (p f t ⊕ p sbert ⊕ p score ). models: the efficacy of the proposed model is evaluated on two types of post embeddings (p emb , p task-emb ), with and without the attention layers. this, eventually leads us to four different types of models for evaluation: (i) multitask: model using p emb , (ii) multitask-score: model using p task-emb , (iii) multitask-attn: model with attention layer using p emb , and (iv) multitask-attn-score: model with attention layer using p task-emb . ## conclusion and future work this work demonstrates the power of using various feature embeddings for multi task learning with bi-lstm on the clpsych shared task 2022 data set. we have tried several different textual embeddings to represent the content of user's posts. these embeddings are passed on to the bi-lstm which is trained to learn two labels jointly. the model has shown to give promising results on the test set when attention layer is incorporated and complete set of feature embeddings (fasttext+sbert+taskscore) is utilised. on task a, our team topped the postlevel classification problem based on the window based and coverage based statistics, whereas for task b, we showed second best results in the competition. in future, we would like to compare our proposed model with other single task learning models trained using separate loss functions. given the correlation between the shared tasks, multi-task learning is expected to yield good results as shown in this paper, however it will be interesting to explore the underlying user information (e.g. age, gender, etc) that could be explicitly added to support tasks for mental health and suicidal risk prediction. also in order to mitigate the effects of imbalanced classes, we would like to improve our developed pipeline using resampling techniques.