Privacy Aware Question-Answering System for Online Mental Health Risk Assessment (2023)

Abstract: Social media platforms have enabled individuals suffering from mental illnesses to share their lived experiences and find the online support necessary to cope. However, many users fail to receive genuine clinical support, thus exacerbating their symptoms. Screening users based on what they post online can aid providers in administering targeted healthcare and minimize false positives. Pre-trained Language Models (LMs) can assess users’ social media data and classify them in terms of their mental health risk. We propose a Question-Answering (QA) approach to assess mental health risk using the Unified-QA model on two large mental health datasets. To protect user data, we extend Unified-QA by anonymizing the model training process using differential privacy. Our results demonstrate the effectiveness of modeling risk assessment as a QA task, specifically for mental health use cases. Furthermore, the model’s performance decreases by less than 1% with the inclusion of differential privacy. The proposed system’s performance is indicative of a promising research direction that will lead to the development of privacy-aware diagnostic systems.

## introduction in recent years, natural language processing (nlp) has emerged as a powerful field of study that focuses on the interaction between human language and computational systems @xcite . mental health is a crucial aspect of overall well-being, and gaining insights into individuals' mental states has become an increasingly important area of study. nlp techniques have been useful in identifying text markers that indicate an individual's mental well-being @xcite . social media websites, such as twitter and reddit, provide a wealth of textual data that offers a unique opportunity to analyze the mental health status of their users at scale, enabling the exploration of patterns, trends, and potential interventions @xcite . assessing users' mental health risk can be reduced to a basic text classification task, where the transformer architecture @xcite has demonstrated state-of-the-art performance. bert @xcite encodings have been utilized for training a variety of mental health risk detection systems @xcite @xcite . bert models fine-tuned on social media data @xcite are able to classify at-risk individuals with high accuracy. however, advances in text classification models have stagnated with the advent of bert encodings. posing the risk assessment problem as a qa task is more analogous to consulting a trained clinician @xcite . qa systems built using bert have been used for public education on topics in mental health @xcite . nearly 30% of qa healthcare systems focus on mental health applications such as workplace empowerment, screening, effecting behavior change, and reducing smoking/alcohol dependence @xcite . multiple-choice qa models demonstrate a promising alternative approach to depression severity estimation even with low amounts of training data @xcite . further development of qa models could lead to better autonomous diagnostic systems. this work proposes the use of allenai's unified-qa model @xcite to assess the mental health risk of users from their social media posts. the research objective is to explore whether qa transformer models are better than text classification transformers at assessing the risk to mental health and modeling language markers that are indicative of specific mental illnesses. we compare unified-qa to stateof-the-art pre-trained language models that perform text classification on the same data. training models on sensitive user data in their raw form makes them non-compliant with data privacy rules which can have serious legal ramifications in the case of unexpected data breaches @xcite . by using differential privacy, language models can be trained such that they do not memorize the training data, leading to data security and better model generalization @xcite . this work also studies the impact of differential private training on qa model performance. the contributions of the paper are as follows. ## proposed methodology the proposed work consists of fine-tuning the unified-qa model on two large mental health datasets in a differentially private manner. ## conclusion & future work this work aimed to explore the effectiveness of question-answering models for mental health risk assessment. the unified-qa model performed ∼2% better than low-parameter pre-trained language models and produced competitive results when compared to larger state-of-the-art language models. with differential privacy, the model only suffers a sub-one percent drop in performance. this study contributes to the growing body of research on mental health analysis using social media data. it highlights the potential of artificial intelligence techniques to improve our understanding of mental some limitations in the design of this study offer directions for further work to improve performance. since the redditmh dataset uses posts from general mental health subreddits, the binary-choice qa model was not exposed to finer-grained data from illness-specific subreddits, which could lead to better assessments. also, the multiple-choice qa model forces an answer from the list of five diseases even if there are no distinct language markers of any disease. future work will combine binary and multi-choice qa models to filter at-risk individuals. two-stage qa models will serve as effective screening tools to ultimately provide care to individuals who need it the most.