  The wide adoption of Electronic Health Records (EHR) has resulted in large
amounts of clinical data becoming available, which promises to support service
delivery and advance clinical and informatics research. Deep learning
techniques have demonstrated performance in predictive analytic tasks using
EHRs yet they typically lack model result transparency or explainability
functionalities and require cumbersome pre-processing tasks. Moreover, EHRs
contain heterogeneous and multi-modal data points such as text, numbers and
time series which further hinder visualisation and interpretability. This paper
proposes a deep learning framework to: 1) encode patient pathways from EHRs
into images, 2) highlight important events within pathway images, and 3) enable
more complex predictions with additional intelligibility. The proposed method
relies on a deep attention mechanism for visualisation of the predictions and
allows predicting multiple sequential outcomes.
