Suicidal Risk Detection for Military Personnel (2020)

Abstract: We analyze social media for detecting the suicidal risk of military personnel, which is especially crucial for countries with compulsory military service such as the Republic of Korea. From a widely-used Korean social Q&A site, we collect posts containing military-relevant content written by active-duty military personnel. We then annotate the posts with two groups of experts: military experts and mental health experts. Our dataset includes 2,791 posts with 13,955 corresponding expert annotations of suicidal risk levels, and this dataset is available to researchers who consent to research ethics agreement. Using various fine-tuned state-of-the-art language models, we predict the level of suicide risk, reaching .88 F1 score for classifying the risks.

## introduction suicide is one of the major causes of death in the military. in some countries where military service is compulsory because of a conscription system, active-duty military personnel live in physical separation from their family and friends for an extended period of time, often against their will. in the republic of korea, for example, most men have the obligation to serve in the military for about a year and half, leading to a large population of about 600,000 in active duty as of this year. many of them experience difficulty adapting to the isolated environment, and some of them are at risk of suicide. one approach to detect the suicide risk signs of active-duty soldiers is the analysis of social media posts, similar to the approach used for detecting suicide risk of the general public @xcite @xcite . however, research on military suicide finds that there are distinct risk factors, such as combat exposure, injury, bereavement, and negative unit climate associated only with military service @xcite . for this reason, we cannot directly apply the findings of suicide risk research of the general public to the military personnel. in this paper, we take on the challenge of collecting social media posts related to military service in the republic of korea, annotating and analyzing them using nlp methods for detecting suicide risk of active-duty soldiers. the first and most challenging step is to create an annotated dataset of military-related social media posts written by active-duty soldiers. we collect posts from a popular social question and answering (q&a) platform. anonymous posts are allowed, so we find that there is a considerable amount of military-related posts that contain possible suicide risk and other mental health issues. annotation poses a challenge, as the mental health and suicide risk of soldiers should be analyzed by mental health experts experienced in the military setting. it is difficult, though, to find such experts, so we reached out to two separate groups of experts, military experts and mental health experts. we asked both groups for annotation, and our analysis includes the results of the annotation, as well as the results of the prediction of suicide risk. our focused contribution is in building a dataset of 2,791 social media posts written by military personnel in korean with corresponding 13,955 expert annotations of suicidal risk levels. we finetune various state-of-the-art language models to classify the risks for developing simple yet effective baselines, achieving up to .88 f1 score. ## constructing annotated dataset we describe the steps in collecting relevant posts, preprocessing, and annotations. we also explain ## experiments we classify the posts by annotated risk levels at the post-level. we first compute the maximum value of risk annotations to aggregate multiple annotations for each post with the aim to give alert if there is any possibility of suicidal risk. this might increase false positives, but the experts view that in practice, false positives are better than false negatives. classification types. we classify posts in three ways: 1) the four risk levels, 2) flagged or not, and 3) urgent or not. for binary classification, we consider low, high, imminent risk posts as flagged and no risk as not flagged, and high, imminent risk as urgent and low, no risk posts as not urgent @xcite . models. we leverage two type of models: 1) convolutional neural network (cnn) and 2) pretrained language models, which are used in the relevant shared task @xcite . the teams that participated in the shared task demonstrated that cnn is effective for the risk classification task @xcite . also, asu @xcite shows fine-tuning pre-trained language model is highly effective. note that our dataset contains korean posts with post-level risk annotations, so these previous models should be adjusted to our dataset. specifically, we use cnn with pre-trained korean subword-level word embeddings for the input of the two convolution layers @xcite . in case of using pre-trained language models, we have a choice to use multilingual models trained with a corpus that includes korean (multilingual-bert @xcite , xlm-r @xcite ) , or a model pre-trained over only korean corpus (kobert 2 ). since we classify the risk at the post-level, these models are fine-tuned by post-level supervisions without aggregation of posts to the user-level. detailed experimental settings for reproducibility are in the appendix. ## discussion and conclusions in this paper, we tackle the problem of suicidal risk of military personnel from their social media posts. we focus on the specific population of military personnel in compulsory service because it requires a unique approach to fully understand their suicidal risk. as our first step, we collect 2,791 militaryrelevant posts in a social q&a platform that are written by at-risk active-duty soldiers and remove any identifying information from the data. then five annotators (three military experts and two clinicians) evaluate the degree of suicidal risk of the posts. after the dataset is constructed, we fine-tune a pre-trained language models, achieving at most 0.88 f1 score. our research can be the first step toward proper intervention programs and institutional support for soldiers with mental health issues. such follow-up would maximize the value of our model and data. we also plan to add domain-specific features to our model, collect more data, integrate existing suicidal risk datasets with various languages to improve performance.