A unified framework for cross-domain and cross-task learning of mental health conditions (2022)

Abstract: The detection of mental health conditions based on an individual’s use of language has received considerable attention in the NLP community. However, most work has focused on single-task and single-domain models, limiting the semantic space that they are able to cover and risking significant cross-domain loss. In this paper, we present two approaches towards a unified framework for cross-domain and cross-task learning for the detection of depression, post-traumatic stress disorder and suicide risk across different platforms that further utilizes inductive biases across tasks. Firstly, we develop a lightweight model using a general set of features that sets a new state of the art on several tasks while matching the performance of more complex task- and domain-specific systems on others. We also propose a multi-task approach and further extend our framework to explicitly capture the affective characteristics of someone’s language, further consolidating transfer of inductive biases and of shared linguistic characteristics. Finally, we present a novel dynamically adaptive loss weighting approach that allows for more stable learning across imbalanced datasets and better neural generalization performance. Our results demonstrate the effectiveness of our unified framework for mental ill-health detection across a number of diverse English datasets.

## introduction depression is a mental health condition characterized by low mood, energy and self-esteem @xcite . one of the most serious effects of depression is the loss of joy in life, which leads to an increased suicide risk among people with depression. 1 however, due to the social stigma surrounding depression, many * work done while at the university of cambridge 1 https://www.who.int/en/news-room/factsheets/detail/suicide people who suffer from it hesitate to seek help. suicide is one of the leading causes of death globally, especially among young people: it is the second most common cause of death among people aged 15-24. 2 post-traumatic stress disorder (ptsd), which is characterized, among others, by symptoms of emotional outburst and negative thought, may also co-occur with depression, and can be a common response to ptsd sufferers. there has thus been interest in the development of natural language processing (nlp) models for detection and/or prevention intervention. for example, this has been the focus of multiple shared tasks at the computational linguistics and clinical psychology (clpsych) workshops @xcite @xcite as well as the audio/visual emotion challenge (avec) @xcite @xcite . however, previous work has tended to focus primarily on a single domain and/or mental health condition at a time. each of the shared tasks listed above were focused on a single dataset from one domain; for example, the clpsych 19 shared task used only forum posts from reddit. the top systems at these shared tasks also frequently made use of domain-specific meta features such as the number of reddit posts per time period, which were found to be among the most informative in suicide risk detection @xcite . meanwhile, research has shown a lack of generalizability across datasets in classification models for mental health nlp @xcite . the goal of our research is to develop models that can capture domain-independent and interrelated characteristics of different mental ill-health detection tasks, and generalize better. the novelty of our work is in proposing an alternative way of formulating the modeling of mental health conditions, which is more robust and effective compared to existing approaches, and we believe can benefit future research in this important task. we use english data from the clpsych 2015 and 2019 shared tasks, which were obtained from twitter and reddit respectively, as well as the distress analysis interview corpus -wizard of oz (daic-woz) @xcite , which consists of interview transcripts. our open-domain setup precludes the use of domain-specific features such as the meta properties previously mentioned, as well as audiovisual cues from daic-woz interview recordings which may not always be available (e.g., due to user privacy concerns over voice and speech analysis). to validate the general applicability of our approach, we experiment with two different types of approaches: 1) we develop novel multi-task learning architectures using a dynamically adaptive loss weighting scheduler that we show can lead to more effective learning across tasks/domains; and 2) we develop lightweight and interpretable models that, in contrast to the task-specific architecture and feature engineering used by many top shared task submissions @xcite @xcite , utilize a cross-task and cross-domain linguistic space that sets a new state of the art on several tasks while matching the performance of more complex task-and domainspecific systems on others. to the best of our knowledge, this is the first approach towards a unified framework for opendomain (cross-domain) detection of different types of mental health conditions (cross-task). ## data & related work we use data from two clpsych shared tasks @xcite , as well as the daic-woz corpus @xcite used in the avec challenges @xcite @xcite , summarized below. while there has been little research in the development of cross-domain mental health models, there has been some effort to develop multi-task ones. these include models for different mental health conditions (anxiety, schizophrenia, panic, eating disorders) @xcite , or the use of auxiliary linguistic tasks such as figurative language detection @xcite . however, all of the methods focus on a single domain (twitter) and therefore capture a limited part of semantic space, whereas we focus on cross-domain methods that generalize across datasets. ## results experimental setting for daic-woz and goemotions, we use the published train/dev/test splits. since clpsych 15 and 19 did not include a development set, we put aside 10% of the training sets (randomly selected and stratified by class) for development. both approaches use the same data and are tuned across the task development sets. we evaluate model performance using macro f1. for clpsych 15, we also report precision separately, as this was the primary evaluation metric of the shared task. we also report f1*, the official metric for clpsych 19 task c, which is f1 computed without the lowest risk class ('a'). feature-based model in f1) compared to both the single-task and constant counterparts, contributing to an effective unified approach. on the other hand, adapt-variant is on par with constant. the effectiveness of adapt-fixed can be attributed to the fact that it enforces learning to a certain (high) level of performance for each task, as opposed to adapt-variant that has a less strict approach to learning performance thresholds. adding emotion detection (goemo) as an auxiliary task leads to overall improvements (avg f1). comparing the constant variants with and without goemo, we see the largest improvements in tasks b and c of 7.1 and 8.4 f1 points respectively. this can be explained by the fact that both the goemotions and clpsych 19 datasets were collected from reddit (however, it seems that the dataset generalizes more poorly to task a, which was collected only from one specific subreddit). overall, we observe again that adapt-fixed achieves the best performance across the neural models, with particularly large improvements in task a as well as dw of 10.5 and 13.5 points respectively. adaptive loss weighting analysis the adaptive loss weighting algorithm is motivated by the very different learning schedules of the different tasks. as can be seen in in in section 4, we noted that the adapt-fixed models generally improved both single task as well as average f1 after 30 epochs of training. in qualitatively, we also observe that the adaptive loss weighting algorithm has a smoothing effect on training (figure 1 ). comparing the constant model (dotted lines) to adapt-fixed (solid lines), we can see that, using the latter, we obtain a more stable version which empirically converges faster. ## conclusion we presented two approaches to cross-domain and cross-task mental ill-health detection. the first involves the development of a general set of features; the second uses a multi-task model, utilizing bert as a shared encoder @xcite . we found the former to perform well across all domains and tasks, demonstrating that they share a common set of linguistic cues. in comparison to shared task submissions which use complex neural models @xcite @xcite , our approach either matches their performance or improves over 15 four classes -confusion, realization, pride, and neutralare excluded as they are not overtly positive or negative. we furthermore investigated the use of affective features, as well as examined negative emotion features in isolation, as a useful inductive bias for the detection of different types of mental health conditions, extending previous work that examines the effect of emotion features in a single domain and single task setting (depression detection in reddit posts; @xcite ). emotion detection, as an auxiliary objective, increased the average f1 score by 1.7 points, with the most substantial improvements observed in tasks from the same domain as the emotion dataset (clpsych 19). finally, we presented an adaptive loss weighting algorithm which successfully balances performance across tasks with different learning schedules while increasing the overall performance. a comparison of model results with and without adaptive weighting revealed that it not only led to improved performance, but also outperformed the best average f1 score achieved over all epochs with constant weighting. our feature-based approach outperformed the neural counterpart, indicating that there is scope for further research towards a unified framework for open-domain detection of various mental health conditions. however, our feature-based results experimentally demonstrate that such an approach is feasible and effective, and achieves a new state of the art on several tasks. to the best of our knowledge, this is the first approach towards a unified framework for opendomain (cross-domain) detection of different types of mental health conditions (cross-task). our paper aims to lay a platform for future research, facilitating progress in this important effort.