Machine Learning Methods for Quantification of Depression Severity and Prediction of Recovery Trajectory using Longitudinal Video and Audio Data, with Applications to Deep Brain Stimulation Treatment Optimization By

Machine Learning Methods for Quantification of Depression Severity and Prediction of Recovery Trajectory using Longitudinal Video and Audio Data, with Applications to Deep Brain Stimulation Treatment Optimization By Sahar Harati Predictive analytics and computational phenotyping techniques have shown promising results in several areas of medicine, including automated classification of radiology imaging, survival analysis in cancer patients, and prediction of life-threatening events in hospitalized patients. In recent years, computational psychiatry has emerged as a field that combines multiple levels and types of data and computational modeling to improve understanding, prediction, and treatment of mental illness. Mental health patients often undergo a variety of non-invasive (e.g., cognitive counseling) and invasive (e.g., surgery) therapies before finding an e↵ective treatment plan. Improved prediction of treatment response can shorten the duration of clinical trials and improve patient experience and outcomes. A key challenge of applying predictive modeling to this problem is that often, the e↵ectiveness of a treatment regimen remains unknown for several weeks. In this thesis, we propose Machine Learning approaches to extracting audio-visual features for predicting the likely outcome of Deep Brain Stimulation (DBS) treatment several weeks in advance for patients suffering from major depressive disorder, a common psychiatric illness for which there are no objective, non-verbal, automated markers that can reliably track treatment response. We first explore the use of video analysis of facial expressivity in a cohort of severely depressed patients before and after DBS. We introduce a set of variability measurements to obtain unsupervised features from muted video recordings. We then leverage the link between short-term emotions and long-term depressed mood states and use a neural network model on the top of emotion-based audio features. The results show that unsupervised features extracted from these audio and video recordings, when incorporated in classification models, can discriminate di↵erent levels of depression severity during ongoing DBS treatment. Moreover, for the long term prediction and in the absence of immediate treatment-response feedback, we utilize a joint state-estimation and temporal di↵erence learning approach to model both the trajectory of a patient’s response and the delayed nature of feedbacks using deep neural networks. The results based on longitudinal recordings of patients with depression show that the learned state values are predictive of the long-term success of DBS treatments. Our findings suggest that Machine Learning models can discover objective biomarkers of depression and patient response to treatments, which have the potential to standardize treatment protocols and enhance the design of future clinical trials. Machine Learning Methods for Quantification of Depression Severity and Prediction of Recovery Trajectory using Longitudinal Video and Audio Data, with Applications to Deep Brain Stimulation Treatment Optimization