Cluster-based Feature Importance Learning for Electronic Health Record Time-series
Authors: Henrique Aguiar
Mauro Santos
Peter Watkinson
Tingting Zhu
Venue: ICLR 2022 Submitted (2022)
Keywords: Clustering
Electronic Health Records

Abstract: The recent availability of Electronic Health Records (EHR) has allowed for the development of algorithms predicting inpatient risk of deterioration and trajectory evolution. However, prediction of disease progression with EHR is challenging since these data are sparse, heterogeneous, multi-dimensional, and multi-modal time-series. As such, clustering is used to identify similar groups within the patient cohort to improve prediction. Current models 
have shown some success in obtaining cluster representation of patient trajectories, however, they i) fail to obtain clinical interpretability for each cluster, and ii) struggle to learn meaningful cluster numbers in the context of the imbalanced distribution of disease outcomes. We propose a supervised deep learning model to cluster EHR data based on the identification of clinically understandable phenotypes with regard to both outcome prediction and patient trajectory. We introduce novel loss functions to address the problems of class imbalance and cluster collapse, and furthermore propose a feature-time attention mechanism to identify cluster-based phenotype importance across time and feature dimensions. We tested our model in over 100,000 unique trajectories from hospitalised patients with Type-II respiratory failure to predict five different outcomes. Our model yielded added interpretability to cluster formation and outperformed benchmarks by at least 5% in mean AUROC.

CLUSTER-BASED FEATURE IMPORTANCE LEARNING FOR ELECTRONIC HEALTH RECORD TIME-SERIES
Anonymous authors Paper under double-blind review
ABSTRACT
The recent availability of Electronic Health Records (EHR) has allowed for the development of algorithms predicting inpatient risk of deterioration and trajectory evolution. However, prediction of disease progression with EHR is challenging since these data are sparse, heterogeneous, multi-dimensional, and multi-modal time-series. As such, clustering is used to identify similar groups within the pa- tient cohort to improve prediction. Current models have shown some success obtaining cluster representation of patient trajectories, however, they i) fail ob- tain clinical interpretability for each cluster, and ii) struggle to learn meaningful cluster numbers in the context of the imbalanced distribution of disease outcomes. We propose a supervised deep learning model to cluster EHR data based on the identification of clinically understandable phenotypes with regard both outcome prediction and patient trajectory. We introduce novel loss functions to address the problems of class imbalance and cluster collapse, and furthermore propose a feature-time timere attention mechanism to identify causter-based phenotype importance across time and feature dimensions. We tested our model in over 100,000 unique trajectories from hospitalised patients with Type-ll respiratory failure predict four different outcomes. Our model yielded added interpretability to cluster for- mation and outperformed benchmarks by at least 5% in mean AUROC.
INTRODUCTION
Chronic conditions such as Chronic Obstructive Pulmonary Disease (COPD) and Cardiovascular Disease (CVD) describe a broad spectrum of medical ailments, and affect significant percentage of the overall population (Adeloye et al.) 2015). Such diseases are characterized by the existence of multiple distinct patient subgroups, largely distinguished by differences in pathology and in the response to different treatments and medical interventions Turner et al. (2015); Vogelmeier et al. (2018). Exacerbation of COPD, condition respiratory failure, can result in emergency hospital admission and mortality if it is not well treated and managed. Early identification of COPD patients' subgroups is therefore of high medical importance and relevance. EHR time-series data are typically used determine clinically relevant COPD inpatient subgroups, and have been applied to detect risk of deterioration (Pikoula et al. 2019).
However, modelling discase progression and risk prediction is challenging due to the extreme data heterogeneity nature of EHRs. Firstly, EHR data contains a mixture of demographic or static vari- ables (i.e. time independent such age and sex), and multi-dimensional time-series (e.g Heart Rate, HR, and laboratory measurements, such as blood tests). Secondly, EHR time-series are multi-modal as different features are collected from different devices, representing distinct clinical properties of relevance. Similarly, time-series features are sampled at different times and have low and distinct sampling rates, as well as different missing value properties. Furthermore, each feature is associated with different noise and evolution patterns.
Recent advances in deep learning (DL) approaches have shown promising results in EHR modelling due to their capacity to handle complex data (Rajkomar et al. 2018). Nonetheless, DL approaches lack relevant interpretability frameworks to be scaled and applied in hospital settings. Several such models have since been proposed tackle this issue (Mayhew et al. 2018), however, most of them focus on a subset of EHR features (usually vital signs only) and fail to provide a clinically-focused phenotypic analysis of learnt patient sub-groups (via clustering).

---

This work builds on previous research in literature to introduce cluster-basee feature-tine attention mechanism to predict patient outcomes based on EHR data. Our method also leverages phenotypic information to aid in clinical interpretabilityy not only making use of demographies and vital-signs information but also of relevant laboratory measurements (all present in the EHR) to provide a more complete patient physiological status. Our contributions include the following:
An end-to-end DL supervised model to cluster EHR patient data based on the identification of clinically understandable cluster phenotypes with regard to both outcome prediction and patient trajectory in a multi-class setting;
A weighted loss to address data imbalance for both tasks of clustering and prediction, a common issue in the medical domain;
The incorporation of a novel loss mechanism in the model, to address the issue of cluster collapse and promote sample assignment to all available clusters;
Finally, the inclusion of novel interpretability framework, derived from cluster-based feature-time attention layer, aiming to identify relevant timestamps and feature variables to represent the patient physiology, cluster assignment and, ultimately, outcome prediction.
This paper is structured as follows. In Section 2 we describe previous research in EHR time-series modelling, clustering and attention methods. Section 3 introduces the dataset used for analysis and provides description of the proposed model. The experimental setup and results of our analysis are presented in Section E and discussion takes place in Section 5 Finally, concluding remarks and future work are available in Section G
2 RELATED WORK
EHR data comprise complex time-series data, being high-dimensional, multi-modal and heteroge- neous, and thus presenting challenges when used in machine learning models (Keogh & Kasetty (2003; Rani & Sikka (2012)). An important goal in a medical setting is to identify phenotypically separable clusters with distinct phenotypic profiles (which we denote as phenotypic clustering here- after). For the purpose of this work, cluster phenotypes result from the combination of two distinct components: a) the evolution profile patient trajectories' within the cluster, and b) the character- isation of the cluster with regards to clinical variables of interest. The latter may include features not used for clustering and may provide information about the underlying or future health status. Traditional clustering models such as K-Means or hierarchical clustering have been shown to fail to capture the existing time-dependent feature relationships. As such, variants have been proposed to mitigate this problem. A temporal version of the K-Means algorithm, Time-Series K-Means (TSKM, Tavenard et al. (2000), models the distance between time-series of different datapoints, using the Euclidean distance (which is equivalent considering all temporal observations as an independent feature value for the corresponding patient admission), or time-series alignment strate- gies such as Dynamie-Time Warping (DTW, Berndt & Clifford (9944) and soft-I)TW (Cuturi & Blondel 2017)).
Recent DL architectures, ranging from Auto-Encodee Encoderss (AE, Ma et al. (2019)), Convolutional Neural Networks (CNN, Munir et al. (2018)) and others, have shown great promise when applied to time- series data across variety of domains. Fortuin et al. 2019) proposed a Self-Organising Map- Varia- tional Auto-Encoder (SOM-VEB), is a state-of-the-art, unsupervised, DL clustering algorithm which extends a variational auto-encoder architecture (Kingma & Welling 2013) for observation learning and representation, through the addition of a Markov model (Gagniuc (017)), to infer temporal evo- lution within the latent space. Clustering performed in the low dimensional latent space through the use of self-organising maps (Kohonen) 1982) to obtain a discrete, oopologicalll-interreeaa latent representation of the learnt clusters. a supervised setting, AC-TPC (Lee Van Der Schaar 2020) serves as the current state-of-the-arr for identifying phenotypically separable clusters in pa- tient trajectories in EHR data. AC-TPC maps EHR data into a latent space via an encoder, and uses an actor-critic network (Konda & Tiitsiklis 2000) which leverages clinical outcomes to aid in clus- ter formation and obtaining cluster phenotypes. Neither SOM-VAE and AC-TPC provide clinically meaningful interpretation of feature-time importance or outcome of interest.
2

---

Attention mechanisms have recently been proposed provide greater interpretability Recurrent
Neural Networks (RNN) and to aid in dealing with long-term dependencies (Vaswani et al. 201 Xu
et al. 2015). and have also been used in modelling EHR time-series (Schwab et al. [017:] Shashiku- 2016) proposes a two-level reverse attention mechanism to
mar et al. 2018). RETAIN (Choi et al. mimic physician's 's decision process and predict future diagnosis. In other recent works, attention mechanisms based on bi-directional RNN and CNN outperformed standard classification models in predicting high risk vascular diseases with the addition of medication information as input data (Kim et al. 2017). A drawback of such attention mechanisms is the focus on temporal interpretabil- ity only, and inability to look at individual features, which key in a medical setting. To solve this issue, (Shamout et al. (2019) considered independent RNN per feature, with concatenation of the resulting latent vectors. However, the latter does not allow the joint modelling across both feature and time dimensions. Alternatively. (Kji et al. (2019); Gandin et al. (202T) proposed learning atten- tion weights directly on the original inputs, prior to being transformed by a RNN, which does not allow modelling of the resulting latent representations. To the best of our knowledge, existing models have been proposed that jointly leverage both feature and time dimensions (feature-time) to determine clinical observation relevance clustered EHR data.
METHODS
3.1
DATASET AND PRE-PROCESSING
Our dataset was retrieved from retrospective database of routinely collected observations from concluded hospital admissions between March 2014 and March 2018 (the HAVEN project, REC reference: 16/SC/0264 and Confidential Advisory Group reference 08/02/1394). The database in- cludes EHR measurements of adult patients admitted to four hospitals from the Oxford University Hospitals NHS Foundation Trust. Note that the HAVEN dataset does not include data from Inten- sive Care Units (ICU), and we have excluded observations taken in the Emergency Department. Key characteristics of HAVEN cohort data include a) heterogeneity, b) multi-modality, and difference in: c) noise distributions, d) sampling rates, e) missing values, etc. Such properties are common across EHR settings, and are challenging with respect to learning useful representations and predictions.
We used the protocol defined in/Pimente et al. (019) to subset the cohort to those patients at risk of developing Type-ll II Respiratory Failure (T2RF) in hospital (a diagram of the data selection steps can be found in Figure A[iin in the Appendix). Four patient outcomes were considered in our analysis: i) no event during hospital stay, leading to successful discharge from the hospital, or the first instance of one of three possible events, ii) unplanned entry to ICU, iii) cardiac arrest (also named "Cardiac" hereafter) and iv) death. Outcome groups are not clearly separable (see Tables AØ A in the Appendix), so patient clusters will naturally contain a mix of different admission outcomes. In this setting, the clinically relevant component of a cluster phenotype (henceforth denoted as cluster outcome propensity or cluster outcome) is represented as categorical distribution indicating the corresponding propensity for cluster-assigned patients to each corresponding outcome.
For each admission, observations were grouped according mean window observation value into 4 hour blocks based on the time to outcome (discharge in the case of no event during stay) only observations within 24 and 72 hours before the outcome were considered. This time window was se- lected based on those traditionally used for validating Early Warning Score (EWS) systems (baseline models used by UK NHS staff to track inpatient physiology, (Royal College of Physicians] 2017) and clinical input, such that the target phenotype represents the patient status in the subsequent 24 hours. Features were transformed according to max normalization due to skewness and het- erogeneity in their distributions. Patient admissions were randomly split into train, validation and test sets. Missing values were imputed based on the previously observed time block all remaining missing observations were imputed according to the feature median from the aggregated validation and test data (see Section 4 for description of train-test data split). Imputed values were flagged in a three-dimensional mask matrix.
After processing. input data contained over 100,000 unique patient trajectories corresponding to 4,266 unique patient admissions (only last patients admissions were considered in our analysis). Original trajectories for the patient cohort are shown in the Appendix in Figures A A A for different variables/features. A lack of clear outcome group separability can be observed across
3

---

temporal and static variables. Furthermore, note the high degree of imbalance in the data admissions with no event account for over 86.8% of the total number of admissions, while event classes correspond to 10.3% Death, 1.8% ICU and 1.1% Cardiac.
3.2 PROPOSED MODEL
We propose a novel model, which we denote by Cluster-bAsed iMportancE Learning fOr Time- series (CAMELOT). Our proposed methodology displayed Figure || Our model builds on previous literature on 3 key items: a) modified loss function to target the multi-class imbalance, b) a novel loss function ensure cluster assignment and phenotype are representative, and c) novel feature-time attention-level framework to boost representation and introduce feature-time in- terpretability for cluster assignment.
Figure 1: Diagram of proposed model. MLP Multilayer Perceptron neural network blocks; RNN- Recurrent Neural Network.
Let N denote the number of patients and Dy the number of input features. Input data consists of a set of patient trajectories = {(xm.)) )Tn \ J==1, where Tn the maximum number of temporal observations for patient and set of patient outcomes Y = {yx } Input trajectory data for the n-th patient is represented as X,, = ...,,,...,,TTTTI where each Xn,t E RP, is referred to as an observation (vector), with a maximum of observed Dy feature values. The corresponding patient outcome one-hot encoded vector yn IR. (more generally, the dimension of yn equals the number of possible outcomes).
Our DL model can be decomposed into 3 neural network blocks: an Encoder, Identifier and Predic- tor. We refer the action of each network respectively as E, I and P (for example, [(x) denotes the output of the Identifier given some input vector x). Both the Identifier and Predictor are Multilayer Perceptrons (MLP), networks of stacked feed-forward dense layers. On the other hand, the Encoder block can be further sub-divided into a) stack of RNN layers and b) our proposed custom attention layer (see Section 3.3 for further details). Separately, also consider set of trainable cluster representation vectors, C {cI,..., We assign the outcome for cluster as P(c,).
A model call is as follows: Given the n-th patient input trajectory data X.. the Encoder network returns a latent representation Zn == E(X,), E R Consequently. the Identifier network computes cluster assignment probabilities, π. [(z,, E R Each element of n,, _. represents the probability assignment of 2, to cluster i, given a total of K clusters. A cluster, kiump is selected according to categorical sampling (kigmmp ~ (a)(),,)), and the corresponding cluster representation, caapp Ckn,, is then selected from C. The output of the model is 1pred == P(clam)) IR... We note that sammp is only sampled during a training phase; at prediction stage, cluster selection follows the equation kppcd arg max n
i=I....K
3.3 ENCODER NETWORK AND A CUSTOM ATTENTION LAYER
The diagram of our proposed Encoder network is presented in Figure 2 The Encoder contains (i) a Recurrent Neural Network (RNN) block of stacked Long Short-Term Memory (LSTTM) layers, and (ii) a customised attention layer, which computes a latent representation by comparing input data with the sequence of output states from the RNN block. We use the same notation above,
code will be shared online after review
4

---

kpak
LSTN Encecar
Ry-1
Viarigg Induce
Oieewero ather
UEVT SOOU 0.8 helpps
EDt Layons
Figure 2: Diagram of the Encoder network composed of an LSTM Encoder and custom attention layer.
and write the sequence of output states of the final LSTM layer as On.1y TT,TT. with On, E R Theoretically, each On,t corresponds to a representative summary of input patient information up until time 1. We propose approximate On, each individual feature, thereby allowing the separation of output states into contributions from each feature. Note that important feature transformations be fime-independent parametrising the model, over- over-fitting and ensure feature representation maps are similar across time.
linear combination of latent representations of
avoid over-
Our attention layer behaves as a set of D feed ferd-rd neural network layers, U1,..., ... UD, jointly represented by: (i) matrix of learnable kernel weights D E RRx Dy- We write D [Dı,... DD (i) matrix of learnable bias vectors B E: D, Similarly, can write B = [B,,... .... BD, J; and (iii) an activation function, a which matches the output activation of the RNN block.
Input data for patient X,, is fed as input to the RNN block, which outputs sequence of latent output states (on,t){= For t 1, .,.. Tni we compute Dj feature representations in latent space
RM, o(D Xn.t + B)
where R,,, [R] [RI,,,.... Dij is our collection of feature representations, is applied element- and A = D Xn,1 is matrix satisfying Ar, Di. Equivalently, R is the output of dense layer, U, with kernel D., bias B,, activation o and input (Xn.)).. We approximate On, & Ed a R = a4 This approximation is minimised following a least squares criterion, which has a well-known solution, ar, and corresponding optimal approximation Ön,t- En 3:On, where
Given, Ô,,,, similar procedure is used to compute context vector weights (3 are learned to provide a more representative context vector.
3.4 ATTENTION MAP VISCALISATION
Given cluster representation vectors, Cj., we can compute cluster-wise feature-time attention visu- alisation map follows. First, normalise feature-weights a; according to softmax function, 8; (ā.)) E RRD; where a the softmax function: (xx) [Tepp
exp
Secondly, we compute X [
cluster-wise weights, ot according to a Icast-square approximation of C ze Σt= 2n and solved as before. We similarly normalise > to obtain cluster temporal scores, et ( Finally, we can compute K scoring matrices, M,,,.,, M,KKK RT,xDDy (M e, = ck of, Note that: || = E, eh. E, sf = E. ci, = 1. Given that Matrices M are normalised, they may be consequently, visualised as a normalised feature- time map for cluster assignment relevance and provide further model interpretability.
5

---

3.5 LOSS OPTIMISATION
The model is optimised through consideration of three distinct loss functions. We introduce a weighted erosspentropy loss function:
Ixpred Utrues Upred)
Σ we Weyime log (3frea) = - Wee log
(2)
where e' is the true outcome for particular patient. We propose inversely proportional normalised weights: We= Wi and w. is inversely proportional to the class distribution, i.e., we N. with N being the number of patients and No being the number of patients with outcome label c. Class weighting penalises misclassification more heavily on less sampled classes.
A
We also propose a novel distribution loss function, (T). We define the average cluster prob- ability of assignment as TC := N 2n. Tr. Then we introduce Ldis(TT) = -H (Tc), where H denotes entropy. Note that Laia is minimised when тe is uniform, ensuring all clusters are 'ex- plored' and have comparable number of samples. While clusters should not necessarily be assigned the same number of samples, this loss helps to overcome concern of cluster collapse, where clus- ters do not separate and samples are assigned to a very small number of non-representative clus- ters. Finally, separate cluster representation vectors, define the cluster separation loss as (Clus()) = - Eis l|c, cy
To optimise our model, iterative gradients are applied according to weighted combinations of the above loss functions with hyper-parameter weights ax 3::
1. Outcome Predictor is updated according to Tpred:
2. Encoder and Identifier are trained according to Ixpred + aT
3. Cluster representation vectors are updated with regards to Topred + 3T T.cuus
3.6 INITIALISATION
Our proposed model also follows a set of initialisation pre-training procedures. Firstly, the Encoder and Outcome Predictor pre-trained according to classification task (ý P(F(x))), with cor- responding loss [predd Latent state representations E(x) are clustered through a K-means algorithm with K clusters across the whole training set. Cluster representation vectors initialised as given by the resulting cluster centroids, and finally the Cluster Identifier network is pre-trained to identify clusters as predicted by the K-Means algorithm with categorical cross-entropy loss. Implementation was completed in Python, with TensorFlow 2, scikit-learn and NumPy. All experiments were run with Tesla v100 GPU, and CPUs Intel(R) Xeon(R) Gold 6246 3.30GHz.
RESULTS
For Benchmark purposes, we considered TSKM as a classic clustering benchmark, and SOM- VAE and AC-TPC as state-of-the-art phenotypic clustering methods. AC-TPC considers temporal subse- quences of a complete patient set of observations for comparison purposes, we consider only the model output for the complete patient sequence. For simplicity, we present results with all input features considered.
All models were trained on the same training set (60% of the complete input data) and evaluated against the same test set (remaining 40%). For DL models, we further split the training set into purely training and validation sets. All experiments with varying syper-parameters were repeated 10 times with fixed set of 10 distinct seeds, and results are reported according to average metric and standard deviation. complete list the hyper-parameters considered for each model included in Table A7 in the Appendix. In bold, top-performing typer-parameters are indicated. Optimal integer hyper-paraman parameters (K, 1) were selected according to an "Occam's 's Razor" approach for each parameter, assign it the highest value such that increasing this amount does not lead to significant increase in performance according to mean AUROC and a Friedman's 's hypothesis test. Neural network size parameters were kept consistent across all DL. models where applicable. All

---

selected according to highest AUROC performance conditional on the model predicting each class (c.g. not making predictions solely for no event or Death Events).
other optimal
We evaluated clustering performance through standard clustering metrics, including Silhouette score (SIL, Rousseeuw )) Pavies-Bouldin Index (DBL, Davies (1979)), Variance Ratio Criterion
(VRI. Calinski & Harabasz (1974). Results for all clustering models are displayed in Table 3
In Table 4 we evaluated the (multi-class) prediction performance with regards to Area-under-the (AUROC), unweighted mean Fl-score, unweighted mean Recall, and Normalised Mutual Information (NMI). For purely unsupervised models 1SOM-VAE and TSKM), an outcome predictive pipeline was constructed by assigning patient admissions clusters, and consequently to the empirical outcome distribution in the corresponding cluster. The prediction task was also benchmarked against traditional classifiers for outcome prediction in EHR data, namely Support Vector Machines (SVM), XGBoost (XGB) and NEWS2 (i.e., the National Early Warning Score used in the UK hospitals). Furthermore, to evaluate other Neural Network models as bench- marks and also justify both proposed mechanisms, we furthermore considered two other bench- marks: a) ENC-PRED: a stacked LSTM Encoder, followed by MLP network for outcome predic- tion, and ATTEP; model equivalent CAMELOT, except the original entropy loss is considered over the clustering dist loss. Supervised performance for all the above models is included in Table4 As noted, convergence was particularly difficult for ATTEP due cluster collapse.
Receiver-Operating-Cur
Metric
TSKM
SOM-VA
AC-TPC
CAMELOT (prposed)
SIL
0.35 (=0.01)
0.25 (E 0.08)
0. 0.00)
0.11 (10.04)
DBI
1,19 (8)
399 0.3)
4,3 0.80)
3T2
VKI
554.6 (+2.50)
[=
(6.3 18.7)
216.7 (1).22)
Table 3: Clustering separability results by the different clustering methodologies given input data with all available features (static, vital-signs, serum and haematological variables). For each metric and model, the average score and standard deviation are returned. The best values for each metric are indicated in bold.
Metra
AURICC
Fi-score
Recail
NMI
SVM
0.50 0,02)
0,23 0.00)
0,25 (L 000)
DOT (3I 0.02)
XGB
NEWS2
0.61
0.29
0,34
0.01
TSKM
0.5 (#: 0.01)
0.24 (.33)
(:26 0.02)
101 (#t 0.D3)
SOM
0.6
0.05)
0.27 0.2)
0.3 0.03)
AC-TPC
0.68 (=) 0.01)
0.38 (1:0001)
(.36 0.01)
0.17 0.02)
ENC FRED
0.7 0,02)
(2:0000))
0.36 (::: 0.02)
U106 (:l: 0.03)
ATTEP
0.67 (= 0.002
(15.00 (2)
(:36 0.02)
0.16 0.05)
CAMELUT (proposed)
0.73 (00.0
(-00000))
0.38 (#:0.02)
0.20 (#00.03)
Table 4: Outcome prediction scores across all models, displayed with an average and standard deviation of a set 10 seeds (except NEWS2, which deterministic). The best values for each metric are indicated bold. For clustering algorithms, cluster outcome distributions were taken to be the empirically observed distribution each cluster.
On top of performance evaluation with regards to clustering separability and outcome prediction. display a comparison between the learnt cluster phenotypes of the proposed model and that of phenotypic clustering benchmark AC-TPC, For each cluster, the corresponding outcome propensity P(c) is shown as bar plot over the possible outcomes with corresponding probability value. We also display cluster outcome propensity plots for both TSKM and SOM-VAE in the Appendix (Fig- ures A and A A We note that the cluster outcome propensity distributions learnt by CAMELOT also align with the empirical outcome relevance in the learnt clusters (Table A (2). Lastly, we also display feature-time cluster relevance attention maps in Figure 6 For each cluster, a patient was randomly selected from the set of patients in the corresponding cluster, and corresponding feature- time attention matrix and visualised as a heatmap.
5 DISCUSSION
Our proposed model shows an improvement in clustering performance ( see Table when compared to the current phenotypic clustering benchmark (AC-TPC), and outperforms SOM-VAE according
7

---

Proposed Model
AC-TPC
Popmert Outer
Pepaoet Ouatte NN-330
ACTRC omer Medd
AC Outte
Proceed 10-30
Outer
AC 1-3-03
ACOI
Figure 5: Comparison bar plots of cluster outcome propensity distributions for the proposed model and benchmark AC-TPC. On the left (blue), distributions displayed for each cluster (out of total of 6), and each phenotype corresponds to the probability of an outcome. Similar results are shown on the right (yellow) for AC-TPC. The title each sub- sub-plot indicates the cluster considered, well number of patients as- signed to given cluster.
Attertiee Naps for Sampled Patlents fromod Chittrr
Adand werp Chato
0000
Nitt serch Oueie
0,00
Figure 6: Feature-Tiin Time Cluster Relevance Map. Each Heatmap represents feature time relevance matrix for random patient assigned to a given cluster. Vertical Axis indicates time to outcome, in hours, while horizontal axis indicates different input fcatures.
to VRI. Although the cluster separability metrics are superior in the case of TSKM, this is expected given metric bias towards convex clusters, and the convexity of the K-Means based algorithm. In particular, DL clustering occurs in latent space, which, unfortunately. is not easily comparable with algorithm targetting the input space (such K-Means). Furthermore, argue clusters learnt by TSKM clusters extremely hard to distinguish with regards to outcome propensity (as evidenced by very low performance on prediction task (Table 4D), and b) TSKM clusters are less separable with regards to trajectory evolution, there is less separation of mean HR trajectories, and less cluster separation when data is projected to two-di- mensional domain with t-stochastic neighbour embedding (ISNE)- both figures are in the Appendix,
TSKM are less relevant that our model's
Figures 10 and All
With regards to predictive power. can be seen in Table that our model outperforms both standard classifiers (at least 8%) and benchmarking clustering methods (at least 5%) according to mean AU- ROC. similar increase can be seen in other classification metrics, with the exception of F1-score, where model performance is slightly below to that of AC-TPC. Our model is able to more accu- rately determine patterns in the data than the previously proposed models, as EHR data is extremely complex and heterogeneous. It is particularly promising that the model obtains good predictive task results despite a clustering bottleneck (i.e. sample predicted outcomes are done through the assigned cluster, opposed to tailored the precise input data). While it possible that other models could show better performance on the direct task of outcome prediction given EHR input, such models can potentially be associated with lack of robustness or input sensitivity difficulties. Furthermore, it
8

---

is likely they would struggle with identifying relevant trends and properties of clinical interest. As such, for new admissions, such models could provide a prediction for the overall outcome, but no robust understanding of how this outcome will occur, and how to prevent potential risks of deterio- ration, let alone the ability to pool data from other similar patients.
Figure shows the advantage of two key aspects of methodology. Our model identifies clear, separable cluster outcome distribution and provides a useful layer of interpretability to clinicians to understand a potential risk of deterioration. Our model also identifies more diverse of cluster outcomes than AC-TPC, which only picks up 3 different cluster outcome distributions, and doesn't t identify the presence of the "ICU" and "Cardiac" classes. We also show other cluster-phenotype benchmark results in Appendix A.8 and A.9. With regards to clusters learnt by CAMELOT. clusters and are the clusters with most ill cohort- they are largely representative of death and cardiac events on the subsequent 24 hours. On the other hand, while cluster 2, 5 are healthier, with smaller chance of adverse events. Clusters and 4 are largely "healthy" cluster, with reduced risks of the most intense adverse events. We note cluster outcome propensity distributions learnt by AC-TPC are unable to provide this level of detailed information. Furthermore, the propensity distribution learnt by our model matches with the empirical number outcome events observed in each cluster (displayed in Appendix Table A12). Note, furthermore, that the model managed to successfully navigate a heavy blass-imbalance setting. Representative clusters are able capture different-sized sub-populations, yet still identify potential risks of deterioration.
On the other hand, learnt cluster attention maps introduced in Figure 6 introduce yet another layer of interpretability to our proposed clustering model. The personalised attention maps highlight the relevant feature-time pairs driving patient cluster assignment, and can be used to identify the most important clinical variables. For instance, analysing Figure 5 suggests clusters with highest propen- sity for either of Death or Cardiac Arrest events are Cluster 0, 3 (and 2 to slightly smaller extent). This reflects in the resulting attention maps, where SBP and FIO2 are highlighted as key clinical variables for cluster assignment. This conclusion is further corroborated when considering descrip- tive statistics of CAMELOT clusters (Table 133), as SBP and FIO2 are some of the few variables with some significant separation across clusters, and when considering trajectory evolutions (Figure A44. Lastly, note that feature-time weights are also relevant if potential deterioration events did not take place so that we are more confident on a patient's 's health status. As an example, attention maps for clusters I and 4 (reasonably healthy clusters) indicate FIO, as very relevant towards the latter stages of the admission this is likely due to these patients not showing an increase in oxygen intake (as they did not need it). Thus, attention maps can be very versatile.
CONCLUSION AND FUTURE WORK
In this work, we propose a novel deep learning model for the task of identification of phenotypically separable clusters applied to EHR data for. As part of proposed model, propose 2 distinct loss functions and introduce a novel feature-time attention layer to better represent patient data and to introduce feature-time relevance map for each cluster. Our experiments show promising results with the addition of both methodological tools above, on both cluster separability and outcome prediction performance. The addition of the feature-time layer has the added benefit of introducing key interpretability tools for researchers to understand relevant regions for good patient physiology representation as well as an indication of what can lead to patient deterioration.
There are multiple interesting avenues of investigation building on this work. On the other hand, the current attention layer mechanisms could potentially be improved with the addition of temporal weight smoothness, or, alternatively, weight regularization to encourage exploration of the complete feature-time space. Alternatively, cluster selection through a neural network mechanism introduces high capacity at the potential cost of robustness and cluster collapse. Potentially, more traditional methods incorporated into a similarly complex pipeline can achieve better performing through a clearer identification of cluster regions in latent space. Furthermore, methodological improvements will also benefit from a more extensive testing across other diverse datasets and other potential areas of application.

---

REFERENCES
Davies Adeloye, Stephen Chua, Chinwei Lee. Catriona Basquill, Angeliki Papana, Evropi Theodor- atou, Harish Nair, Danijela Gasevic, Devi Sridhar, Harry Camphell, et al. Global and regional estimates of copd prevalence: Systematic review and meta-analysis. Journal of global health, 5 (2), 2015.
Donald J Berndt and James Clifford. Using dynamic time warping to find patterns in time series, In KDD workshop, volume 10, pp. 359-370. Seattle, WA, USA:, 1994.
Tadeusz Califiski and Jerzy Harabasz. A dendrite method for cluster analysis. Communications in Statistics-theory and Methods, 3(1):1-27, 1974.
Edward Choi, Mohammad Taha Bahadori, Joshua A Kulas, Andy Schuetz, Walter F Stewart, and Jimeng Sun, Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. arXiv preprini 1rXiv: 2016.
Marco Cuturi and Mathieu Blondel. Soft-dtw: a differentiable loss function for time-series. In International Conference Machine Learning, pp. 894-903. PMLR, 2017.
DL Davies. et dw bouldin. a cluster separation measure. IEEE Trans. Pattern Anal. Mach. Intell, (2), 1979.
Vincent Fortuin, Matthias Hüser, Francesco Locatello, Heiko Strathmann, and Gunnar Rätsch. Deep self-organization: Interpretable discrete representation learning on time series. In International Conference on Learning Representations, 2019, URL //pprreieew nettttiiweeeeee...........e..e.e net forum? id=rygjcsR9Y7
Paul A Gagniuc. Markov chains: from theory to implementation and experimentation. John Wiley & Sons, 2017.
laria Gandin, Arjuna Scagnetto, Simona Romani, and Giulia Barbati. Interpretability of time- series deep learning models: study in cardiovascular patients admitted to intensive care unit. Journal of Biomedical Informatics. 121:103876, 2021. ISSN 1532-0464. doi: https://doi.
URL https: www. www.sceeieeeeec.c...c.oe...o com/ soom/scienee/
aat.ccle/p1//1155466222002
Deepak A Kaji, John R Zech, Jun S Kim, Samuel K Cho, Neha Dangayach, Anthony B Costa, and Eric K Oermann. An attention based deep learning model of clinical events in the intensive care unit. PloS one, 14())<0211007, 2019.
Eamonn Keogh and Shruti Kasetty. On the need for time series data mining benchmarks: survey and empirical demonstration. Data Mining and knowledge discovery, 7(4):349-371, 2003.
You Jin Kim, Yun-Geun Lee, Jeong Whun Kim, Jin Joo Park, Borim Ryu, and Jung-Woo Ha.
Highrisk prediction from electronic medical records via deep attention networks. arXiv preprint arXiv:1712.000010 2017.
Diederik P Kingma and Max Welling. arXiv:1312.6114 2013.
Auto-encoding variational bayes.
arXiv preprint
Teuvo Kohonen. Self Self-organized formation of topologically correct feature maps. Biological cyber- netics, 43(1):59-69, 1982.
Vijay R Konda and John N Tsitsiklis. Actor-critic algorithms. In Advances in neural information processing systems, pp. 1008-1014, 2000.
Changhee Lee and Mihaela Van Der Schaar. Temporal phenotyping using deep predictive cluster- ing of disease progression. Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference Machine Learning, volume 119 of Proceedings of Machine Learn- ing Research, pp. 5767-5777 PMLR, 13-18 Jul 2020. URL press/v119//er wwtp11999eeeleeee html html
mmrr.
10

---

Qianli Ma, Jiawei Zheng, Sen Li, and Gary W Cottrell. Learning representations for time series clustering. Advances in neural information processing systems, $2:3781-37911 2019.
Michael B Mayhew, Brenden K Petersen, Ana Paula Sales, John D Greene, Vincent X Liu, and Todd S Wasson. Flexible, cluster-based analysis of the electronic medical record of sepsis with composite mixture models. Journal of biomedical informatics, 78:33-44 42, 2018.
Mohsin Munir, Shoaib Ahmed Siddiqui, Andreas Dengel, and Sheraz Ahmed. Deepant: A deep learning approach for unsupervised anomaly detection in time series. leee Access, 7:1991-2005, 2018.
Maria Pikoula, Jennifer Kathleen Quint, Francis Nissen, Harry Hemingway, Liam Smeeth, and Spiros Denaxas. Identifying clinically important copd sub- sub-ttyes using data- approaches in primary care population based electronic health records. BMC medical informatics and decision making, 19(1):1-14, 2019.
Marco AF Pimentel, Oliver C Redfern, Stephen Gerry, Gary S Collins, James Malycha, David Prytherch, Paul E Schmidt, Gary B Smith, and Peter Watkinson. A comparison of the ability of the national early warning score and the national early warning score 2 to identify patients risk of in-hospital mortality: a multi-centre database study. Resuscitation, 134: 2019.
Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M Dai, Nissan Hajaj, Michaela Hardt, Peter Liu, Xiaobing Liu, Jake Marcus, Mimi Sun, et al. Scalable and accurate deep learning with electronic health records. NPJ Digital Medicine, 1(1):1-10, 2018.
Sangeeta Rani and Geeta Sikka. Recent techniques of clustering of time series data: survey. International Journal of Computer Applications, 52(15), 2012.
Peter Rousseeuw. Silhouettes: graphical aid to the interpretation and validation of cluster analy- sis. Journal of computational and applied mathematics, 20:53-65, 1987.
Royal College of Physicians. National early warning score (news) 2. Standardising the assessment of acute-illness severity in the NHS, 2017.
Patrick Schwab, Gaetano C Scebba, Jia Zhang, Marco Delai, and Walter Karlen. Beat by beat: Classifying cardiac arrhythmias with recurrent neural networks. In 2017 Computing in Cardiology (CinC), pp. 1-4. IEFF, 2017.
Farah E Shamout, Tingting Zhu, Pulkit Sharma, Peter J Watkinson, and David A Clifton. Deep interpretable early warning system for the detection of clinical deterioration. IEEE journal of biomedical and health informatics, 24(2):437-446, 2019.
Supreeth P Shashikumar, Amit J Shah, Gari D Clifford, and Shamim Nemati. Detection of parox- ysmal atrial fibrillation using attention-base based bidirectional recurrent neural networks. In Pro- ceedings of the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 715-223, 2018
Romain Tavenard, Johann Faouzi, Gilles Vandewiele, Felix Divo, Guillaume Androz, Chester Holtz, Marie Payne, Roman Yurchak, Mare Rußwurm, Kushal Kolar, et al. Tslearn, machine learning toolkit for time series data. J. Mach. Learn. Res., 21(118):1-6, 2020.
Alice M Turner, Lilla Tamasi, Florence Schleich, Mehmet Hoxha, Ildiko Horvath, Renaud Louis, and Neil Barnes. Clinically relevant subgroups in copd and asthma. European respiratory review, 24(136):283-29) 2015.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 6998-6008. 2017.
Claus F Vogelmeier, Kenneth R Chapman, Marc Miravitlles, Nicolas Roche, Jørgen Vestbo, Chau Thach, Donald Banerji. Robert Fogel, Francesco Patalano, Petter Olsson, al. Exacerbation heterogeneity in copd: subgroup analyses from the flame study. International journal of chronic obstructive pulmonary disease, 13:1 2018.

---

Kelvin Xu, Jimmy Ba. Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In International conference machine learning, pp. PMLR, 2015.
A APPENDIX
A.1 DATA
A description of the complete pipeline of data re-processing, following the protocol defined in Pi- mentel et al. (2019). is shown in Figure A
Ocnvelion Ancystods
Ly 3333
spi ALCA0
A.1: HAVEN processing
A total of 26 input features were considered. Firstly, 4-hourly vital-sign sets which included features: Heart Rate (HR), Respiratory Rate (RR), Systolic Blood Pressure (SBP), Diastolic Blood Pressure (DBP), peripheral Oxygen Saturation (SpO2), Temperature (TEMP), level of consciousness via the AVPU scale Alert, Verbal, Pain, Unresponsive and estimated Fraction of Inspired Oxygen (FiOg, available when an oxygen mask applied to the patient). Each set consisted of timestamp and the vital-sign numerical values. Secondly, 4 demographic variables were selected (modelled as static variables): age, sex, and admission type (elective or surgical). Thirdly, we included fea- tures resulting from biochemistry blood tests, denoted 'Serum': Scrum levels of urea, albumin, creatinine, sodium, potassium and C-reactive protein. Finally, haematological blood test features also included: white and haemaglobin cell counts, concentration of eosinophils, basophils, and too-pphooo---yypooyyyttt ra-
neutrophils, and lymphocytes, as well as dosinophil-to-base tios. These features were selected based on domain knowledge of features related to severity in the prognosis and outcome of T2RF inpatients.
Descriptive statistics for all input variables is described in Table A 2 Median and inter-quartile range (IQR) is displayed for continuous and categorical variables, while binary variables are shown according number of counts in the dataset and corresponding cohort proportion. Statistics are displayed for the complete data ("All"), but also for each sub-cohort defined by the overall outcome. We can observe that these sub-cohorts are not clearly separable and are hard identify solely from this information.
summary of the patient cohort in relation to outcomes and target phenotypes can be seen in Table A Challenges with regards to obtaining phenotypically separable clusters can similarly be observed there no clear significant difference between the target outcome sub-cohorts with regards to demographic input variables. With regards to outcome distribution, we also note the high degree of imbalance in the dataset- the large majority the patients in our dataset suffered from no adverse events (over 86%), while only 48 had Cardiae event, and 76 were re-directed to the ICU.
The lack of outcome sub-cohort separability can further observed in a temporal domain. Figures A A o plot the mean trajectories of different temporal variables sets for each outcome sub- cohort, respectively, according to vital signs, haematological and serum features. Mean calculated based the time outcome, and missing observations are disregarded and ignored.
12

---

34 to
20d0 350
0 10
1: 76
1::0 :220
030..% 1260 1388
731-55-100 1010
128655 220
10 31.h
15000 15000 L.0000
2012
2010 20101200:
2500
44.
11:00
144 3M
120 O2 140
31
4.0%%
maRege 1540
8. 128
00000 018
100,000
53,0 33.0 320
0S000 128
10 11:30
39008800,
05 19539
AINCE 4300
:300 -200015400 -0
in in no
14 20
320 00 139
(4go 1469-7230 7230
6N 30 W
Wot 12:0
10 1:0g
0akdte 015
1:460--13 11
100 180
2013 21 20300
11333 2000
ABCC 4000
36000000000 -2
1re0 u.
15.
1o00
48
1,201
15:00
12 0. 120
6000 13
200R 46
(
1222 1240
00303:4-15 1530
03-0310000
19008-43
68863. 1813
1998--12
2.0000 1:55
2L23 23 3-- 11380
185.0 15500122000 R200E 15540
A200 110
12 281-1-114 1413
1500 555
100..
1200 10
1199 04
11 0:00 11:00
1001 (:60 13:28
01 QU0 010
7AL 7A.69-11 11:00
112 00.1007--1 149
0. 1.60
28 20.00
137000 137000 033000 100000
A 17
A3
423
0002
1ypa
Istrer
tange
10053
1099
10053
Steer www. tharee
Bana Sho Sosss Poscaae
"e teweeroee atteett
Shat BaalB 2. com (lu0 53thighi 'our
NEIl yiu own (lao
L7k Litighorett rezz Mond)
Oftzznn Pou iis lod INd urmuai cert sttmm) 10Dum tetoppoo
Ergacier 2oa Soon
rogunaay fars
Fuirstog
116
D3P
59:13
100
RBV
K00
1231
SER
902
A.2: Descriptive statistics and information input data features. Variables are displayed with type, descrip- tion, units and average statistics. We separate all features according to medical literature, including vital-sign, static, serum and haematological variables, and we also display statistics outcome sub-cohort, defined as cohort with those patients assigned to given outcome.
13

---

No Event
Death
1:0
Cardinc
(QQR)
71 (61
(74
(6l
76 (69 825
Gender, M
1810 (48. 9%6)
247 (56.0%)
38 (50.0%)
28 (58.33%)
CCI (IQR)
4133
(4 (4--1)
74 17)
(4 23)
Elective
T126 (30.44))
(10.5%)
(4.2%)
Surgical
1054 (28.556)
48 (10.1%)
22 (29.0%)
(12.5%)
A.3: Descriptive demographic variable information for each outcome sub-cohort.
trajectorics signs per cstcome group
150.-
Haalthy ICL N8-78 Cara Hi 48
55.0
Fine Outcomec
line Ousuoeecooos
132.
06.0
$0.0
91.
1107
.0.-
35.8
28-
15
12-
181
179 150
225
130
175
150
125
300
A.4: Plot of mean vital-sign trajectories (median with respect SpO₂) solid line as given by outcome groups: admissions with a) Cardiac-, b) Death- or c) ICU-, and d) No-events. The respective standard errors are represented by the dashed lines. We visualised trajectories from up to days prior to an outcome event or discharge- the black lines represent the time window (72 24 hours prior an event discharge) considered for imput models.
A.2 MODEL TRAINING
list indicating the grid-search range of cated in Table A For simplicity, we define TP := (0.001, 0.01, 0.1, 10), = {32, 64, 128, 256} and K : (3. ..., 20). bold, tor-performing hyper-parameters according target metrics defined
considered our experiments are indi-
in Section 4 are highlighted.
A.3 RESULTS COMPARISON
In Figures A and Figures A we display cluster outcome propensity distributions for some of our experiments with benchmark clustering models COM-VAE and TSKM, respectively. Both mod- els do not naturally associate clusters with a distribution we estimate the cluster outcome as the empirical outcome distribution for the patient cohort assigned to the corresponding cluster.
14

---

ICA Wh
641
200
50.0
Dours
thonn)
-2-
100.0
40-
600
4.-
144.0
135
250
135
350
25 175 120
125
300
50
A.5: Plot of mean haematological trajectories in solid line as given by the 4 outcome groups: admissions with Cardiac-, b) Death- ICU-, and d) No-events. The respective standard errors are represented hy the dashed lines. We visualised trajectories from up to 7 days prior outcome lines represent window (72 24 hours prior to an event or discharge) considered for input to all models.
discharge the black
2034
184
0.1
0.0
20c
8
10.0
2.-4
20c
275 130
225
1.0
115
220
135
A.6: Plot of mean serum trajectories in solid line given by the 4 outcome groups: admissions with a) Cardiac- b) Death-, e) ICU-, and d) No-events. The respective standard visualised trajectories from up to 7 days prior to an outcome event discharge the black lines represent the time window (72 24 hours prior to an event or discharge) considered for input to all models.
represented by the dashed lines.
We note that clusters learnt by both clustering benchmark models have identical outcomes, which provides no useful clinical interpretability to the cluster-defined populations, as well as likely not assisting models to learn relevant cluster representations.
15

---

SOM VAE Cluster Phenotypes
No-113
3.400
2.5
2.00
235
2.00
S0N WI Clutter 4: N=1120
S0M WIE Cluster S: N-65
A.8: plots of learnt cluster phenotypes for SOM- VAE optimal Each plot represents a cluster its phenotype is corresponding empirical outcome distribution cluster-assigned patient cohort.
We go further in comparing clusters learnt by TSKM and by our proposed model. We argue clusters learnt by CAMELOT are much more relevant towards our goal. We show this through two distinct plots. Firstly, in Figure A AIO, we display a scatter plot of patients each cluster (CAMELOT on right and TSKM clusters on the left) after projection to two dimensions. Projection was com- pleted through principal component analysis reduction to 50 dimensions, followed by 1-stochastic neighbour embedding dimensionality projection two.
Furthermore, also demonstrated that TSKM does not learn separable cluster trajectory evolu- tion profiles CAMELOT. This is shown in Figure A11 where Heart- Rate mean trajectories for each cluster (i.e., average HR observations aligned to the same time until end of observations for patients in the clusters) are displayed. It is clear that CAMELOT cluster trajectories easier to separate.
complete description of the number patient admissions with given outcome per learnt cluster in our proposed model can be seen in Table A 12
Parameter
SOM-VAR
AC-TPC
CAMELOT
seeds
(1001, 1012, 1134, 2475, 6138, 7415, 1863, 7203, 9253, 1782)
F().1)
P(0.0))
F(0.01)
F(0.1)
F(0.01)
2(0.001)
10.1. 0.2 0.0
L(64]
(1288)
L(128)
SOM dim
L"
K(6)
kemell
(DIW", cuclidenn
"polynomial" 10C (10)
Ieestimttoss
[100, 200, 300
depth
61 10)
min-shild-weipht
A.7: Parameter range used for Gric-scarch hyper- parameter optimisation. For each model, the list of parameter values tested is indicated, In bold, the optimum set hyper parameters indicated for each model.
16

---

TSKK Cluster Phanotypes
TSX Cluster 0: N-798
TSRM Cluster 1: 1:N-118
035
a.o
0.25
TSKM Cluster 2: N--112
TSKM Cluster 3: N-349
0.00
02b
0.00
Ni-e9s
N-1188
0.00 Heality
Beath
Gardlct
Healty
Death
tarthes
A.9: plots learnt cluster phenotypes for TSKM with K Each plot represents a cluster its phenotype the corresponding empirical outcome distribution cluster-assigned patient cohort.
Chaisring ing Performace comper ben intowen dillerunt cluseer ansigremants.
TSOM Label Projection
Main Model Label Projections
Chas M/7/
Clar 5603
Clas
20
Tang 0012
-60
-20
20
40
-60
-20
20
40
A. 10: Scatter plot of cluster patient data after projection to dimensions.
Outcome
Healthy
Death
ICU
Cardiac
Cluster 0
149
44
10
7
Cluster
739
28
5
Cluster
579
Cluster 3
93
92
12
5
Cluster 4
373
0
Cluster
288
10
10
A.12: Table with empirical number of outcome admissions observed for each cluster learnt by the proposed model.
17

---

HOWT Aur Pic mape Felett cuaDre
A.11: Plot mean Heart- Rate (HR) trajectory solid line given by TSKM learnt clusters (top) and CAMELOT (bottom). The respective standard errors represented by the dashed lines. We visualised trajec- tories from up to days prior to an outcome event or discharge black lines represent the time window (72 24 hours prior event or discharge) considered for input all models.
We also computed summary statistics for the learnt CAMELOT clusters. For each of the result- ing clusters, median, and quartile values were computed and plotted, except on the case of binary variables, where only the number of positive occurrences (and the corresponding proportion in the cluster) are shown.
Lastly, we plot the mean cluster trajectory evolution for SBP and FIO2 present supportive evi- dence for the personalised attention maps in Figure 6 These features were selected from atten- tion map analysis.
18

---

180 200)
:2001000 (900)
E210 4900
363 1361--00000 553)
LOILO 15)
(66.0.00.. 8
IMC0
31. 1133)
105 1888
61 G1
20003-455
83
10007-15
(. 138-113.
730 10000
625 (266 -PR)
4.1 -1.
-50001t 24000
60
650:26
JANCLI
85S
920 92.0
1701160-
670000 15.0)
20-1200- 21.0
583 38.9
10:3 1:0
730 700 79000
950 138255555
111 12
10 12:9
11200 (4)
11 (1.00-0:)
45:23-76
72:3L
11:39-23
4.:3111 -
2002300 31.0
710,000-500 -000000000 85.0
4.:36-1.5)
1881 (1300
1390))
5133--88
240
A8.0 78.0)
14-24300- 06.0
300/220--440 540)
1641353 3699
16.1 (0000000000 2000
114: (5644444
80139999
51.035559)
11 12.
125 1&5
0000 0.]
0000-00 0.1
08000 3.
10 10000-111 I18
09.05-1)
129 22.
3A2 10 200
72000000 1953
1867 1558 213.7
450
1300050-LCN 1128)
(3.45-115)
201
88.0
1200
1011o
0.05 1300)
55.3004 1550040--000
21.-218-- 22U) (L0)
363 165)
19000 100
1135500 (00)
1790 (7
30 (3.35%
1 12.9
01011 12n
01 (00-113)
25009-50 5-)
69
1 1.00
53132-99 9.1)
12215700-000 920)
17.028- 1215
60136-43 (5)
15TA0,300 1400)
59/42-87
S0L
17-0L68-
6550500-
260122
Cle
277
BL0,220 9..9)
1800660 183)
1500110
mapps 78.)
55034
210310 23.0
36 1361 383
10110 1.0)
130
Mu 18 9y
285000 %g
11.1 (200 1330
10007A 125
01 (1000-833
00-00-00)
25088-6
76154-105
1200-110 1.4
61333 10.0
2002500 520
71003000000 0800-9900 9U.
500071-1880 13631)
1.136-
15000310
5342-85)
180 -8.0
123 (1000.00 1420
65 1400
25 0940 97.00
2L0 11.00
3631990- 167)
10:33 1.00
280 34.0)
30004
460
9: 11:0)
99 11.5)
IU (10 (12)
100 00.00-100 11)
28
16155-108.
0P365-2-0
8.7
25 2000
2100 (LB00) 2000
71.8 01.0--02200 19900
13:33995000 5.00
13600 1410
161 13.0
230
22.0
Calsomial (1-
Cooditr A4B
Innger
Hile
Srealalhsiztte (ipmn))
milg
mmlg
Jer
Fturan lesprd Uraan aociaainoo
Mlt Vitbal Patt Uireppacc Scae
Disa Oloed Frosare
Istinated Orgere Simmiiiv
ware illeel Lal cven (Bood) aasinoppi cres divoog
BAd0 cill court (hlod
Bokiookil-tookiii di makee
NEUtopei wouk (boot
Trlpleepte cont (Mood)
Neatrphhid 4 pmmpheente Tukin
burrrii (eem))
Chostinos teme (ppeeeena)
Coeettrrc towtin lon (ptennal
Polaerm lovd plaeno 500Y levllphauss
coretteett leves
Ele Menire Aeriie-sltiiin Aitiiin Sugiid shinsion
Descipoin
Houtenee
Reginary
Tompenire
Palen age
Micepoiess
Harragide
50e
1102
TEMD
AXPU
genter
blaiiee
Surpial
868
EBR
XBU
LS54
NUR
POC
Eleer Heegiilll
Saik
Senmu
A.13: Descriptive statistics and information of all input data features. Variables are displayed with type, de- scription, units and average statistics. We separate all features according to medical literature, including vital- sign, static, serum and haematological variables. Statistics shown for each cohort learnt by model.
19

---

Moan FID2 Trajectories for CAMPLOT Clusars.
Canteme
THE Ortninnee
R- 222 72
101
100
A.14: Plot of mean Systolic Bload Pressure (SBP) trajectories solid line as given by the clusters learnt by our model (top). the bottom, mean FIO2 uajectories are displayed. The respective standard errors are represented by the dashed lines. We visualised trajectories from up 7 days prior an outcome event or discharge the black lines represent the time window (72 24 hours prior to an event discharge) considered input all models.
20


Reviews:
[{"review_id": "NT9c6B1YTfk", "reviewer": "Reviewer_9DjJ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1) The authors did a good job in explaining and navigating the time-series complexity and challenges posed by EHR data particularly the limitations of previous work and the results section.\n2) The paper provided comprehensive explanation of the proposed deep learning network especially the added model interpretability which plays a crucial role in integrating techniques/algorithms of artificial intelligence in healthcare.\n\nWeakness:\n1) My key concern about the paper is the lack of rigorous experimentation in designing the study cohort. Real-time diagnosis and prediction of clinical interventions is a major challenge especially using EHR data and for this study only the observations within 24 and 72 hours before the outcome were considered. The best practice while studying EHR-based predictions is to design an observation and prediction window, this practice helps understand how long the observations made by the model is valid and helps the clinicians to proactively assist their patients. The analysis presented in the manuscript lacks the experimentation with the sliding window and this further limits the scope for improvement and assessment of the models performance and determining the best possible subtype.\n\n2) Similarly as noted above, the missingness treatment in the study also lacks experimentation. The foundation of the proposed deep learning framework is based on the quality of data and missing data is more prevalent in EHR but the authors failed to elaborate more on the missing data imputation section as in why authors chose to impute missing data with previous time block and median when there are other state-of-the art techniques/algorithms available to impute missing values especially when the quality improvement of the underlying data could increase cluster separability and model predictions. This could possibly help future readers of this paper to approach/reproduce/handle missing data problems in their respective healthcare settings.\n\n3) The clustering separability of the proposed framework(CAMELOT) is significantly low than the benchmark TSKM and also the F-1 score of the outcome prediction scores is low, and AUROC is only marginally better than the other benchmark models. Unfortunately, the novelty in the method proposed by the authors does not reflect in the model outcomes.\n\nMinor comments:\n1) Authors should include what percentage of missingness was in the feature variables before imputation, perhaps in table A.2.\n2) In section 3.6, I do not see citations/references for the libraries/software's used by the authors for this study. Please cite them appropriately.\n3) I suggest authors choose a different color palette/schema to improve the interpretation of the feature-time cluster relevance map and make the heatmap(Figure 6) more visually appealing.\n4) In Discussion section, 'learnt cluster attention maps introduced in Figure 6 introduce yet another layer' -> 'learnt", "summary_of_the_paper": "The authors through this paper propose a deep learning framework to identify phenotypically separable clusters using EHR data and introduce a feature-time attention layer to better represent patient data including optimizing two loss functions to address class imbalance in the datasets. The study aims to make a more precision based clinical observation by leveraging feature dimensions and time-series nature of EHR data.", "summary_of_the_review": "Although authors did a fairly good job in conceptualizing and developing the latter stages of their proposed deep learning framework, as I elaborated in my main review, the prior stages of the framework lack experimentation. Model-centric approach is important but data-centric approach is more crucial especially in healthcare where predictions play a vital role in assessing disease progression and therapeutic intervention.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Lpw87vZTXEv", "reviewer": "Reviewer_nD3g", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n1. The paper introduces time-feature relevance maps, which enable domain experts to look at features-time combinations which are in some important for their predictions.\n2. The paper is written in a manner which makes experiments fairly easy to reproduce. \n3. The proposed method is better than the National Early Warning Score used in the UK hospitals.\n\nWeaknesses/Scope for improvement: \n1. The TSKM baseline performs really well for the clustering problem. The authors claim that the superior results of TSKM are due to the metric bias towards convex clusters. However, given the capacity of deep neural networks and the ease of learning convex clusters, I am not convinced why the deep learning method would be at a disadvantage. The authors also claim that clustering on the input space and latent space cannot be compared. However, my understanding is that learning a smaller dimensional latent representation should instead help clustering, if the latent representations are meaningful. \n2. I believe that in addition to the distribution of likely outcomes, common characteristics of patients in a cluster would be clinically relevant. For example, the could the authors come up with a way to use their proposed feature-time relevance maps to characterize each phenotype? I believe that would be very helpful.\n3. There are no experiments which serve as evidence that the proposed loss functions are (1) needed, and (2) work. I can imagine that the authors can design simple experiments to illustrate the cluster collapse problem etc. which their loss functions are designed to handle. These experiments would both establish the need and serve as a \"proof\" that the loss functions are working as expected. \n4. I am not sure if the baselines to predict outcomes directly from EHR data are strong enough. I suggest using stronger baselines to directly predict outcomes, for example 1-D ResNets have been found to be really performant time series classification models.\n\nQuestions:\n1. Are the cluster representations $\\mathcal{C}$ interpretable? \n2. The authors are phenotyping patients based on unique distributions of outcomes. In this respect, how does one define \"clearer\" and \"more separable\" cluster phenotypes? More generally, how does one define a \"good\" phenotype? I feel the distributions in case of AC-TPC model are \"clear\" too.\n3. I am not sure how does the proposed model handle different features having different sampling frequencies. \n\nSuggestions: \n1. The figures do not seem to have high quality. I recommend saving them as .svg or .pdf to ensure the same.  \n2. Typographical/grammatical errors: \n     -- \"which can has a well-known solution\" => \"which has a well-known solution\"\n     -- \"with a smaller change of death\" => \"with a smaller chance of death\"\n3. Appendix A.10 has the true number of patients per outcome for each cluster. It would be interesting to see how close is the distribution of outcomes to the ground truth, based on a measure of distance between distributions. \n4. Ensure that the colour maps in Fig.6 have the same range i.e. the same mapping between the colour and the relevance value.", "summary_of_the_paper": "The paper presents a way to predict outcomes of patients using EHR records. The authors propose a phenotyping model which clusters similar patients based on their outcome distribution. They also introduce feature-time relevance map which helps explain for a particular patient the feature-time combinations which are important for their prediction.", "summary_of_the_review": "I believe that the work is interesting, solves an important problem and does well. However, further experiments are needed, for instance, to check why the model is not performing as well in the clustering problem. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "F797-iwcfhu", "reviewer": "Reviewer_p3Vb", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper tackles the important problem of phenotyping with an attention-based recurrent model. The neural network predicts an assignment for population-level latent centroids.\n\nWhile the method is novel, it would be valuable to justify further the use of an LSTM, as attention-based mechanisms leverage long time dependencies while accelerating the training time of the model. The use of an LSTM might jeopardize this gain. A comparison with an MLP feature extractor would be valuable.\n\nFurther clarifications are necessary to describe the attention mechanisms (how is D_f chosen and what does it represent ?) and the model training (Why is the model trained iteratively ? It seems it could be done end to end).\nIn the experiment section, it is not clear why one should average performances over multiple seeds as it quantifies the modelling sensitivity to hyper parameter tuning and training, but not the model generalization to external datasets. Bootstrapped or cross-validation performances should be preferred. Additionally, the use of the 'Occam razor' selection of hyperparameters needs to be justified, as it is not clear to me how it is optimized in a large hyperparameter space.\n\nIn the results and discussion, metrics should be further described to highlight what they capture and what reflects a clinically relevant clustering. Figure 6 does not allow a better understanding of the clusters. It would be more interesting to show average attention for patients assigned to the same cluster.\n\nThe authors should also consider similar approaches proposed in the literature. \n- Mechanisms that leverage both temporal and EHR data have been recently presented [1]. \n- Models which both cluster and model an outcome are also used in the statistical literature as profile regression models [2]. \n- Balanced loss is also widely used in the machine learning literature, the authors should not claim novelty on this part but cite papers such as [3]\n\nFinally, a few minor revisions:\n- \"Traditional clustering models such as KMeans [...] have been shown to fail to capture the existing time dependent deature relationships\" would need citations\n- The authors describe a 4 hour moving window. Do they use separate window or overlapping ones, ie what is the frequency of the final data ?\n- Median imputation is using validation and test data, shouldn't be training data ?\n\n\n\n[1] Rocheteau, E., Liò, P. and Hyland, S., 2021, April. Temporal pointwise convolutional networks for length of stay prediction in the intensive care unit. In Proceedings of the Conference on Health, Inference, and Learning (pp. 58-68).\n\n[2] Molitor, J., Papathomas, M., Jerrett, M. and Richardson, S., 2010. Bayesian profile regression with an application to the National Survey of Children's Health. Biostatistics, 11(3), pp.484-498.\n\n[3] Cui, Y., Jia, M., Lin, T.Y., Song, Y. and Belongie, S., 2019. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9268-9277).", "summary_of_the_paper": "This work proposes a supervised approach to phenotype patients given their EHR trajectory and predicted outcome. The authors argue that the lack of interpretability of current deep learning approaches does not allow clinically relevant phenotypes and propose a feature-time attention mechanism to tackle this issue.\n", "summary_of_the_review": "This work addresses an interesting and important problem in medical machine learning. However, it would benefit from additional methodological descriptions and results' interpretations.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9SHeCcG0z7o", "reviewer": "Reviewer_MGut", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \\\nWriting is clear and well organized. \\\nNice figures and appendix content. \\\nGood performance at prediction combined with cluster interpretability. \n\n\nCons: \\\nOnly evaluated on private data. \\\nThe clusters initialization. Is there a specific reason to use K-Means? Was it evaluated a different initialization method? \\\nThe lack of ablation studies. The impact of removing or changing some of the framework's features, like the custom losses and Attention Block, should be evaluated to bring a stronger understanding of its impact on final performance. \\\nHow does your Attention Encoder differ from RETAIN[1]? It looks very similar. \n\n\nMinor comments:\\\nThere is a missing blue dot and Figure 5 \n\n\n\n\\\n[1] Edward Choi, Mohammad Taha Bahadori, Joshua A Kulas, Andy Schuetz, Walter F Stewart, and Jimeng Sun. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. arXiv preprint arXiv:1608.05745, 2016.\n", "summary_of_the_paper": "The paper describes a cluster-based learning method for EHR time series of in-hospital patients aimed to be interpretable and improve outcome prediction.\\\nIt claims to contribute with an interpretable framework and its training process, with a weighted loss to deal with the class imbalance and a custom loss to avoid cluster collapse.", "summary_of_the_review": "It is a good paper with relevant contributions.\n\nI am not sure if the results are well evaluated so I expect to conclude at the discussion period.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper has been independently assessed by four expert reviewers. Two of them recommend acceptance (one straight, one marginal), and two rejection (both marginal). Among the main limitations of the presented work, found by the reviewers, was the limited reproducibility of the results due to the use of private data. The authors attempted to defend their experimental design choice to use only one private dataset by stating difficulty in obtaining public data that would be suitable for their method. I am personally in a strong disagreement with that statement, and with the sub-statement of the authors that some publicly available ICU data may not be suitable. That either signals limited practical utility of the presented approach to non-ICU settings only, or is simply incorrect. The presented approach nonetheless has an intriguing potential and I would be inclined to recommends its acceptance. Alas, I find the lack of reproducibility to be a significant drawback of the way this work is currently presented and this limitation could not be easily resolved. Therefore I am leaning towards recommending a rejection.
Decision: Reject