Approximate Nearest Neighbour Extraction Techniques and Neural Networks for Suicide Risk Prediction in the CLP sych 2022 Shared Task (2022)

Abstract: This paper describes the participation of our group on the CLPsych 2022 shared task. For task A, which tries to capture changes in mood over time, we have applied an Approximate Nearest Neighbour (ANN) extraction technique with the aim of relabelling the user messages according to their proximity, based on the representation of these messages in a vector space. Regarding the subtask B, we have used the output of the subtask A to train a Recurrent Neural Network (RNN) to predict the risk of suicide at the user level. The results obtained are very competitive considering that our team was one of the few that made use of the organisers’ proposed virtual environment and also made use of the Task A output to predict the Task B results.

## introduction clpsych 2022 shared task @xcite introduces the problem of assessing changes in a person's mood over time on the basis of their linguistic content @xcite . the purpose of the organisers is to focus on posting activity in online social media platforms. in particular, given a user's posts over a certain period in time, the aim of the task is to capture those sub-periods during which a user's mood deviates from their baseline mood and to use this information to predict the suicide risk at user level. thus, the clpsych 2022 shared task consists of the two subtasks: (1) identify mood changes in users' posts over time and; (2) show how subtask a can help to assess the risk level of a user. this paper presents our participation in the subtasks t1 and t2. ## methods our team's participation in the task has been based on a system for capturing changes in mood over time and the information generated by this system has been used by another system that allows the prediction of the level of suicide risk in social network users. ## results apart from the difficulty of the tasks themselves described in previous sections, another difficulty of the task was working in the environment that the organisers managed to access and work with the data. instead of distributing the annotated dataset for training, the norc data enclave environment was used. the norc data enclave provides a confidential and protected environment in which only authorized participants could securely access and analyze remotely the data. however, due to the problems that some participants had in working in this environment, the datasets (training and test) were distributed to these groups. for this reason, a column in the results tables indicates the use of the data enclave environment to obtain these results. ## conclusions in this work, we introduce the approximate nearest neighbour (ann) extraction technique and the use of recurrent neural networks (rnn) to automatically capture changes in mood over time and the prediction of the suicide risk at the user level. the shared task had the added challenge of working in a virtual environment that the organisers had prepared to preserve the privacy of the real data with which we had to work. however, due to the problems of some participants in working in this environment, the data were distributed among these groups and could be processed outside the virtual environment. this fact, from our point of view, prevents a fair comparison among the systems that used the environment and those that did not. this is due to the fact that the virtual environment has no internet connection and therefore the resources available to process the data were only the libraries that previously had been installed at the beginning of the shared task. leaving this consideration aside, our system performed acceptably, having a high score in recall. the low precision we obtained is an aspect that we need to improve on, for future work. as for the analysis of the organisers in terms of window size, it can be said that our system performed remarkably well for window sizes 2 and 3, obtaining the best recall scores in these cases. another challenge of the task was set by the organisers when planning the two sub-tasks. in this case, participants were encouraged to use the output of sub-task a as input for sub-task b to predict the suicide risk at the user level. in our case, we took up this challenge and together with just another team we were the only ones to use the output of subtask a to predict the suicide risk in subtask b. moreover, by a very small margin with the other team, we obtained the best scores in f1-measure and recall.