Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data (2021)

Abstract: Mental health conditions remain underdiagnosed even in countries with common access to advanced medical care. The ability to accurately and efficiently predict mood from easily collectible data has several important implications for the early detection, intervention, and treatment of mental health disorders. One promising data source to help monitor human behavior is daily smartphone usage. However, care must be taken to summarize behaviors without identifying the user through personal (e.g., personally identifiable information) or protected (e.g., race, gender) attributes. In this paper, we study behavioral markers of daily mood using a recent dataset of mobile behaviors from adolescent populations at high risk of suicidal behaviors. Using computational models, we find that language and multimodal representations of mobile typed text (spanning typed characters, words, keystroke timings, and app usage) are predictive of daily mood. However, we find that models trained to predict mood often also capture private user identities in their intermediate representations. To tackle this problem, we evaluate approaches that obfuscate user identity while remaining predictive. By combining multimodal representations with privacy-preserving learning, we are able to push forward the performance-privacy frontier.

## introduction mental illnesses can have a damaging permanent impact on communities, societies, and economies all over the world @xcite . individuals often do not realize they are at risk of mental disorders even when they have symptoms. as a result, many are late in seeking professional help and treatment @xcite , particularly among adolescents where suicide is the second leading cause of death (curtin first two authors contributed equally. ## multimodal mobile dataset intensive monitoring of behaviors via adolescents' frequent use of smartphones may shed new light on the early risk of suicidal thoughts and ideations @xcite . smartphones provide a valuable and natural data source with rich behavioral markers spanning online communication, keystroke patterns, and application usage. learning these markers requires large datasets with diversity in participants, variety in features, and accuracy in annotations. as a step towards this goal, we recently collected a dataset of mobile behaviors from high-risk adolescent populations with consent from participating groups. we begin with a brief review of the data collection process. this data monitors adolescents spanning (a) recent suicide attempters (past 6 months) with current suicidal ideation, (b) suicide ideators with no past suicide attempts, and (c) psychiatric controls with no history of suicide ideation or attempts. passive sensing data is collected from each participant's smartphone across a duration of 6 months. participants are administered clinical interviews probing for suicidal thoughts and behaviors (stbs), and self-report instruments regarding symptoms and acute events (e.g., suicide attempts, psychiatric hospitalizations) are tracked weekly via a questionnaire. all users have given consent for their mobile data to be collected and shared with us for research purposes. this study has been carefully reviewed and approved by an irb. we follow the nih guidelines, with a central irb (single irb) linked to secondary sites. we have irb approval for the central institution and all secondary sites. ## mood prediction methods in this paper, we focus on studying approaches for learning privacy-preserving representations from mobile data for mood prediction. our processed data comes in the form of {(x t,i , x k,i , x a,i , y i )} n @xmath0, x @xmath1, and x @xmath2. y denotes the label which takes on one of our 3 mood categories: negative, neutral, and positive. in parallel, we also have data representing the corresponding (one-hot) user identity x id which will be useful when learning privacypreserving representations that do not encode information about user identity x id and evaluating privacy performance. ## experiments we perform experiments to test the utility of text, keystroke, and app features in predicting daily mood while keeping user privacy in mind. ## conclusion in this paper, we investigated the learning of language and multimodal representations of typed text collected from mobile data. we studied the challenge of learning markers of daily mood as a step towards early detection and intervention of mental health disorders for social good. our method also shows promising results in obfuscating user identities for privacy-preserving learning, a direction crucial towards real-world learning from sensitive mobile data and healthcare labels. in addition, our findings illustrate several challenges and opportunities in representation learning from typed text as an understudied area in nlp. limitations & future work: while our approach shows promises in learning representations for mood prediction, several future directions on the modeling and nlp side include: 1) better models and pre-training algorithms for nlp on typed text, 2) algorithms that provide formal guarantees of privacy @xcite , and 3) federated training from decentralized data @xcite to improve privacy @xcite and fairness @xcite of sensitive data. we describe more limitations and future social implications of our work in our broader impact statement in appendix a. applications in mental health: suicide is the second leading cause of death among adolescents. in addition to deaths, 16% of high school students report seriously considering suicide each year, and 8% make one or more suicide attempts @xcite . despite these alarming statistics, there is little consensus concerning imminent risk for suicide @xcite . current research conducts clinical interviews and patient self-report questionnaires that provide longterm assessments of suicide risk. however, few studies have focused on imminent suicidal risk, which is of critical clinical importance as a step towards adaptive real-time interventions @xcite . given the impact of suicide on society, there is an urgent need to better understand the behavior markers related to suicidal ideation. to realize this goal, we need accurate and timely methods that predict when interventions are most needed. monitoring (with participants' permission) mobile data to assess mental health and provide early interventions is, therefore, a rich opportunity for scalable deployment across high-risk populations. our data collection, experimental study, and computational approaches provide a step towards data-intensive longitudinal monitoring of human behavior. however, one must take care to summarize behaviors from mobile data without identifying the user through personal (e.g., personally identifiable information) or protected attributes (e.g., race, gender). this form of anonymity is critical when implementing these technologies in real-world scenarios. our goal is to be highly predictive of mood while remaining as privacy-preserving as possible. we outline some of the potential privacy and security concerns below. limitations: while we hope that our research can provide a starting point on the potential of detecting mood unobtrusively throughout the day in a privacy-preserving way, we strongly acknowledge there remain methodological issues where a lot more research needs to be done to enable the realworld deployment of such technologies. we emphasize that healthcare providers and mobile app startups should not attempt to apply our approach in the real world until the following issues (and many more) can be reliably resolved: privacy and security: there are privacy risks associated with making predictions from mobile data. to deploy these algorithms across at-risk populations, it is important to keep data private on each device without sending it to other locations. even if data is kept private, it is possible to decode data from gradients (zhu and han, 2020) or pretrained models @xcite . in addition, sensitive databases with private mobile data could be at-risk to external security attacks from adversaries @xcite . therefore, it is crucial to obtain user consent before collecting device data. in our exper-iments with real-world mobile data, all participants have given consent for their mobile device data to be collected and shared with us for research purposes. all data was anonymized and stripped of all personal (e.g., personally identifiable information) and protected attributes (e.g., race, gender).