DCMN: Double Core Memory Network for Patient Outcome Prediction with Multimodal Data More and more healthcare data are becoming readily available nowadays. These data can help the healthcare professionals and patient themselves to better understand the patient status and potentially lead to improved care quality. However, the analysis of these data are challenging because they are largescale and heterogeneous, highdimensional and sparse, temporal but irregularly sampled. In this paper, we propose a method called Double Core Memory Networks (DCMN) to integrate information from different modalities of the longitudinal patient data and learn a joint patient representation effective for downstream analytical tasks such as risk prediction. DCMN is designed not only to disentangle the temporal and nonlinear intramodal dependencies for the data within each modality but also to capture the longterm intermodal interactions. DCMN models are the endtoend memory networks with two external memory cores where each modality of data is compressed and stored. Each memory core has an informationflow controller named query to interact with an external memory module. In addition, we incorporate a gating mechanism into basic DCMN model to perform dynamic regulation of memory interaction. DCMN models have multiple computational layers (hops) allowing data of different modalities interacting with each other recurrently along with a mechanism of alternating access of external memory for each memory core hopbyhop. We evaluate DCMN models on two outcome prediction tasks, including a mortality prediction on the public Medical Information Mart for Intensive Care III (MIMICIII) database and a cost prediction on the Hospital Quality Monitoring System (HQMS) dataset. Experimental results demonstrate that our DCMN models are more competitive over the baseline methods in the multimodal prediction setting.

Predictive Multilevel Patient Representations from Electronic Health Records The advent of the Internet era has led to an explosive growth in the Electronic Health Records (EHR) in the past decades. The EHR data can be regarded as a collection of clinical events, including laboratory results, medication records, physiological indicators, etc, which can be used for clinical outcome prediction tasks to support constructions of intelligent health systems. Learning patient representation from these clinical events for the clinical outcome prediction is an important but challenging step. Most related studies transform EHR data of a patient into a sequence of clinical events in temporal order and then use sequential models to learn patient representations for outcome prediction. However, clinical event sequence contains thousands of event types and temporal dependencies. We further make an observation that clinical events occurring in a short period are not constrained by any temporal order but events in a long term are influenced by temporal dependencies. The multiscale temporal property makes it difficult for traditional sequential models to capture the shortterm cooccurrence and the longterm temporal dependencies in clinical event sequences. In response to the above challenges, this paper proposes a Multilevel Representation Model (MRM). MRM first uses a sparse attention mechanism to model the shortterm cooccurrence, then uses intervalbased event pooling to remove redundant information and reduce sequence length and finally predicts clinical outcomes through Long ShortTerm Memory (LSTM). Experiments on realworld datasets indicate that our proposed model largely improves the performance of clinical outcome prediction tasks using EHR data.