Hybrid deep learning for predicting hypertensive disorder onset using temporal and non-temporal data

The aim of this proof-of-concept study is to demonstrate the extension of the PatientLevelPrediction R-Package with deep learning models using temporal and non-temporal data. We present a hybrid deep-learning based method to predict hypertensive disorder onset in patients with pharmaceutically treated depression using temporal measurement data fed into a deep convolutional neural network, and non-temporal data fed into a multi-layer perceptron. We compare the hybrid method with other algorithms and show a higher discriminative performance. The developed deep learning pipeline can easily be expanded to cover more advanced network structures and hybrids. We aim to evaluate the deep learning methods on more cohorts at risk and outcomes in multiple databases in the near future. Introduction The success of machine learning largely depends on the selection of an optimal feature representation. In the early days, the machine learning community mainly focused on algorithm development, while currently the field is shifting to more powerful feature engineering. Deep learning models are widely used to automatically learn highlevel features from both temporal and static raw data, and have achieved remarkable results in image processing and speech recognition [1,2]. Recently, interesting results have been shown in healthcare applications [3-5]. The applied model architectures are composed of multiple non-linear neural networks, e.g. convolutional neural network. However, these methods have not been assessed at large scale, i.e. many cohorts at risk and many outcomes on EHR data. This would require a systematic approach against a common data model. We believe, the Observational Health Sciences and Informatics (OHDSI) initiative is in the best position to expand the insights of the global machine learning community in the use of deep learning for patient-level predictive modelling. Methods and Results We implemented different several deep learning models in the PatientLevelPrediction Package using PyTorch (http://pytorch.org ), e.g., Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Multilayer Perceptron (MLP). The framework can easily be extended with more advanced network structures. As a proof-ofconcept, we demonstrate the use of the deep learning implementation for predicting hypertensive disorder in patients with pharmaceutically treated depression in the Integrate Primary Care Information (IPCI) GP database. The target population consists of n=133,349 patients with pharmaceutically treated depression (PTD) without a history of psychosis, mania, or dementia. In total 3,894 PTD patients developed a hypertensive disorder. We used a backward observation window of 365 days and a time at risk window of 365 days following the start of the cohort. In this study, we only considered measurement as temporal data and other patient information (procedures, drugs, conditions, observation and demographic data) are treated as non-temporal data. In total 224 types of measurements and 5648 types of other non-temporal data where used as candidate predictors. The temporal data, i.e. the measurements, are fed into the CNN, wherein we employ a similar CNN architecture as used by Razavian et al. [3], i.e. two convolutional layers and two fully connected layers, the dropout probability between each layer is 0.5. The non-temporal data, i.e. all except for the measurements, are fed into the a MLP with one hidden layer. The final prediction is averaging the output probabilities from the two individual models. We use a weighted loss function to overcome the class imbalance problem when training the model using Adam with regularization [6]. We compared the performance with Lasso as implemented in R (Cyclops) using only measurements and Lasso using the full feature set, CNN using only measurements, logistics regression (LR) and MLP as implemented in PyTorch on the full features set, and a hybrid of CNN and MLP/LR. Table 1. Discriminative performance of multiple algorithm for predicting hypertensive disorder in PTD patients. Algorithm Features Train AUC Test AUC Lasso (measurements) Measurements 0.71 0.68 CNN (measurements) Measurements 0.87 0.86