Original: Dynamic Outcomes-Based Clustering of Disease Trajectory in Mechanically Ventilated Patients
Revised: Dynamic Outcomes-Based Clustering of Disease Trajectory in Mechanically Ventilated Patients

Original: Abstract
Revised: Abstract

Original: The advancement of Electronic Health Records (EHRs) and machine learning have enabled a data-driven and personalised approach to healthcare.
Revised: The advancement of Electronic Health Records (EHRs) and machine learning have enabled a data-driven and personalised approach to healthcare.

Original: One step in this direction is to uncover patient sub-types with similar disease trajectories in a heterogeneous population.
Revised: One step in this direction is to uncover patient sub-types with similar disease trajectories in a heterogeneous population.

Original: This is especially important in the context of mechanical ventilation in intensive care, where mortality is high and there is no consensus on treatment.
Revised: This is especially important in the context of mechanical ventilation in intensive care, where mortality is high and there is no consensus on treatment.

Original: In this work, we present an approach to clustering mechanical ventilation episodes, using a multi-task combination of supervised, self-supervised and unsupervised learning techniques.
Revised: In this work, we present an approach to clustering mechanical ventilation episodes, using a multi-task combination of supervised, self-supervised and unsupervised learning techniques.

Original: Our dynamic clustering assignment is guided to reflect the phenotype , trajectory and outcomes of the patient.
Revised: Our dynamic clustering assignment is guided to reflect the phenotype , trajectory and outcomes of the patient.

Original: Experimentation on a real-world dataset is encouraging and we hope that this could translate into actionable insights in guiding future clinical research
Revised: Experimentation on a real-world dataset is encouraging and we hope that this could translate into actionable insights in guiding future clinical research

Original:  Introduction and Related Work
Revised:  Introduction and Related Work

Original: Patients on mechanical ventilation are a highly heterogeneous group, with widely differing outcomes.
Revised: Patients on mechanical ventilation are a highly heterogeneous group, with widely differing outcomes.

Original: Some have relatively healthy lungs e.g. if they are recovering from surgery on another organ; whereas others have varying degrees of pulmonary failure.
Revised: Some have relatively healthy lungs e.g. if they are recovering from surgery on another organ; whereas others have varying degrees of pulmonary failure.

Original: Pulmonary failure can be acute e.g.
Revised: Pulmonary failure can be acute e.g.

Original: Acute Respiratory Distress Syndrome (ARDS) and deteriorate rapidly, or chronic, typically evolving slowly.
Revised: Acute Respiratory Distress Syndrome (ARDS) and deteriorate rapidly, or chronic, typically evolving slowly.

Original: Unfortunately, patients on ventilators have high mortality [20, 24] and there is no established consensus on optimal treatment strategies from randomised controlled trials [2].
Revised: Unfortunately, patients on ventilators have high mortality [20, 24] and there is no established consensus on optimal treatment strategies from randomised controlled trials [2].

Original: Therefore, there is great potential benefit to be gained from phenotype discovery in order to guide future clinical studies.
Revised: Therefore, there is great potential benefit to be gained from phenotype discovery in order to guide future clinical studies.

Original: To this end, we have developed a dynamic clustering approach for mechanically ventilated patients in the ICU.
Revised: To this end, we have developed a dynamic clustering approach for mechanically ventilated patients in the ICU.

Original: Previous work shows us that simple clustering techniques have revealed actionable sub-phenotypes by secondary analysis of RCT data.
Revised: Previous work shows us that simple clustering techniques have revealed actionable sub-phenotypes by secondary analysis of RCT data.

Original: For example, latent trajectory modelling of inflammatory biomarkers has revealed sub-types of ARDS [9].
Revised: For example, latent trajectory modelling of inflammatory biomarkers has revealed sub-types of ARDS [9].

Original: Clustering of transcriptomic data has revealed patient populations that in which steroid therapy may be beneficial in sepsis [1].
Revised: Clustering of transcriptomic data has revealed patient populations that in which steroid therapy may be beneficial in sepsis [1].

Original: Routinely collected data has also been used to find simple trajectory clustersof for sepsis based on physiological parameters [3] which have been shown to have differential response to fluids.
Revised: Routinely collected data has also been used to find simple trajectory clusters for sepsis based on physiological parameters [3] which have been shown to have differential response to fluids.

Original: We know that temporal neural network architectures can handle the heterogeneous population in the ICU, both using supervised [27, 11] and unsupervised [21, 39] approaches.
Revised: We know that temporal neural network architectures can handle the heterogeneous population in the ICU, both using supervised [27, 11] and unsupervised [21, 39] approaches.

Original: Temporal clustering approaches have been applied successfully to other domains e.g. in Parkinson’s [41], diabetes [28] and cystic fibrosis [17] and increasingly in intensive care as discussed above.
Revised: Temporal clustering approaches have been applied successfully to other domains e.g. in Parkinson’s [41], diabetes [28] and cystic fibrosis [17] and increasingly in intensive care as discussed above.

Original: We have designed our clusters to share similarities in phenotype , trajectory and outcomes .
Revised: We have designed our clusters to share similarities in phenotype , trajectory and outcomes .

Original: We generate a cluster for each hour of a patient’s stay, meaning that if an event happens which alters the predicted trajectory and outcomes, there will be a shift in the cluster assignment.
Revised: We generate a cluster for each hour of a patient’s stay, meaning that if an event happens which alters the predicted trajectory and outcomes, there will be a shift in the cluster assignment.

Original: This is interesting, not only because it can reveal which events are associated with these shifts, but also what might have happened if the ventilation strategy had been different.
Revised: This is interesting, not only because it can reveal which events are associated with these shifts, but also what might have happened if the ventilation strategy had been different.

Original: We hope that our work could someday translate into actionable insights in guiding future clinical research.
Revised: We hope that our work could someday translate into actionable insights in guiding future clinical research.

Original:  Methods
Revised:  Methods

Original: Broadly, our strategy was to train a temporal encoder to embed the patient data at every timestep (this is analogous to returning all the hidden states for an LSTM model).
Revised: Broadly, our strategy was to train a temporal encoder to embed the patient data at every timestep (this is analogous to returning all the hidden states for an LSTM model).

Original: We used a mixture of supervised, unsupervised and self-supervised learning to do this (see Section D below).
Revised: We used a mixture of supervised, unsupervised and self-supervised learning to do this (see Section D below).

Original: Once the encoder training was complete, we used an unsupervised method to cluster the embeddings, so that we get a cluster for every timestep in the patient’s ventilation episode.
Revised: Once the encoder training was complete, we used an unsupervised method to cluster the embeddings, so that we get a cluster for every timestep in the patient’s ventilation episode.

Original: The code can be found at: https://github.com/EmmaRocheteau/Mechanical-Ventilation-Clustering .
Revised: The code can be found at: https://github.com/EmmaRocheteau/Mechanical-Ventilation-Clustering .

Original: The data consisted of both timeseries and static features (Section 3).
Revised: The data consisted of both timeseries and static features (Section 3).

Original: The supervised tasks included two binary tasks: hospital mortality and the risk of receiving a tracheostomy 1 , and two duration tasks: the remaining length of stay (LoS) from timestep t , and their remaining ventilation duration (VD).
Revised: The supervised tasks included two binary tasks: hospital mortality and the risk of receiving a tracheostomy 1 , and two duration tasks: the remaining length of stay (LoS) from timestep t , and their remaining ventilation duration (VD).

Original: This ensured that the patient outcomes are stored within the embedding.
Revised: This ensured that the patient outcomes are stored within the embedding.

Original: In addition, we trained a decoder to reconstruct timestep t and the static data.
Revised: In addition, we trained a decoder to reconstruct timestep t and the static data.

Original: This is an unsupervised approach which encourages the embedding to retain the patient phenotype .
Revised: This is an unsupervised approach which encourages the embedding to retain the patient phenotype .

Original: Finally, we predicted timestep t + 1 , a self-supervised approach designed to embed the patient trajectory .
Revised: Finally, we predicted timestep t + 1 , a self-supervised approach designed to embed the patient trajectory .

Original: See Figure 1 for a schematic.
Revised: See Figure 1 for a schematic.

Original: After training was complete, we performed k-medoids on the embeddings to obtain the clusters.
Revised: After training was complete, we performed k-medoids on the embeddings to obtain the clusters.

Original: Encoder
Revised: Encoder

Original: In recent years, LSTMs have been by far the most popular model for predicting clinical outcomes and have achieved state-of-the-art results [11, 29, 25, 36].
Revised: In recent years, LSTMs have been by far the most popular model for predicting clinical outcomes and have achieved state-of-the-art results [11, 29, 25, 36].

Original: They have also been applied to other patient prediction tasks e.g. forecasting diagnoses and medications [6, 19], and mortality prediction [5, 11, 30].
Revised: They have also been applied to other patient prediction tasks e.g. forecasting diagnoses and medications [6, 19], and mortality prediction [5, 11, 30].

Original: More recently, the Transformer model [38] has marginally outperformed the
Revised: More recently, the Transformer model [38] has marginally outperformed the

Original: LSTM when predicting LoS [31].
Revised: LSTM when predicting LoS [31].

Original: Rocheteau et al.
Revised: Rocheteau et al.

Original: [27] showed that Temporal Pointwise Convolution (TPC) outperformed both the LSTM and Transformer models on mortality and LoS.
Revised: [27] showed that Temporal Pointwise Convolution (TPC) outperformed both the LSTM and Transformer models on mortality and LoS.

Original: Therefore, we chose to investigate these three encoders.
Revised: Therefore, we chose to investigate these three encoders.

Original: Further details on both the encoder implementations and k-medoids clustering are given in Section C.
Revised: Further details on both the encoder implementations and k-medoids clustering are given in Section C.

Original:  Data
Revised:  Data

Original: We used the Amsterdam UMC database version 1.0.2 [35], which contains 23,106 ICU admissions from 20,109 patients admitted between 2003 and 2016.
Revised: We used the Amsterdam UMC database version 1.0.2 [35], which contains 23,106 ICU admissions from 20,109 patients admitted between 2003 and 2016.

Original: We selected all of the mechanical ventilation episodes with a minimum duration of 4 hours, capping the maximum duration after 21 days to reduce computational costs.
Revised: We selected all of the mechanical ventilation episodes with a minimum duration of 4 hours, capping the maximum duration after 21 days to reduce computational costs.

Original: This corresponded to 14,836 episodes which occurred during 13,502 ICU
Revised: This corresponded to 14,836 episodes which occurred during 13,502 ICU

Original: [Table]
Revised: [Table]

Original: admissions from a cohort of 12,597 unique patients.
Revised: admissions from a cohort of 12,597 unique patients.

Original: We selected 31 time series features and 14 static features.
Revised: We selected 31 time series features and 14 static features.

Original: The data were split such that 70%, 15% and 15% were used for training, validation and testing respectively.
Revised: The data were split such that 70%, 15% and 15% were used for training, validation and testing respectively.

Original: These were split by patient , not ventilation episode, to avoid data leakage from the train set.
Revised: These were split by patient , not ventilation episode, to avoid data leakage from the train set.

Original: Further details on the data preprocessing are provided in Section B.
Revised: Further details on the data preprocessing are provided in Section B.

Original:  Results
Revised:  Results

Original: (a) – Full Task Setting The TPC model performs significantly better than the LSTM and Transformer on the outcome tasks (Table 1a), which is in line with previous findings in MIMIC-IV and eICU
Revised: (a) – Full Task Setting The TPC model performs significantly better than the LSTM and Transformer on the outcome tasks (Table 1a), which is in line with previous findings in MIMIC-IV and eICU

Original: The superiority of the TPC model is also evident in the variational and ablation experiments.
Revised: The superiority of the TPC model is also evident in the variational and ablation experiments.

Original: Interestingly, the Transformer performs poorly on the binary tasks but better on the duration tasks with respect to the LSTM.
Revised: Interestingly, the Transformer performs poorly on the binary tasks but better on the duration tasks with respect to the LSTM.

Original: Additionally, the LSTM performs the best on the reconstruction and forecasting tasks (Table 14a).
Revised: Additionally, the LSTM performs the best on the reconstruction and forecasting tasks (Table 14a).

Original: Possible reasons for these findings are discussed in Section 5.
Revised: Possible reasons for these findings are discussed in Section 5.

Original: (b) – Variational Embedding Spaces We experimented with making the embeddings ‘variational’, by representing the embedding as a set of means and standard deviations to allow sampling of embedding coordinates.
Revised: (b) – Variational Embedding Spaces We experimented with making the embeddings ‘variational’, by representing the embedding as a set of means and standard deviations to allow sampling of embedding coordinates.

Original: The rationale was that by forcing the embedding space to be smoother, we might improve the quality of the clustering as
Revised: The rationale was that by forcing the embedding space to be smoother, we might improve the quality of the clustering as

Original: the distances between patients in the embedding space become more reliable.
Revised: the distances between patients in the embedding space become more reliable.

Original: However, this was found to universally hurt performance (Table 1b and Table 14b) and it produced clusters which were more homogeneous in terms of outcomes and features, which was counter to the aim of producing clinically distinct clusters.
Revised: However, this was found to universally hurt performance (Table 1b and Table 14b) and it produced clusters which were more homogeneous in terms of outcomes and features, which was counter to the aim of producing clinically distinct clusters.

Original: Ablation Study We performed an ablation study on the tasks used to train the representation space.
Revised: Ablation Study We performed an ablation study on the tasks used to train the representation space.

Original: The full set of results and analysis are included in Secyion A.1 the Appendix.
Revised: The full set of results and analysis are included in Secyion A.1 the Appendix.

Original: However, the trend is such that the best results for all tasks (except for the duration tasks) are achieved when all tasks are included (Table 4).
Revised: However, the trend is such that the best results for all tasks (except for the duration tasks) are achieved when all tasks are included (Table 4).

Original: The reason for the exception is in the duration only task setting (g), is explored further in the discussion.
Revised: The reason for the exception is in the duration only task setting (g), is explored further in the discussion.

Original: Overall, our ablation study indicates that having multiple competing learning objectives has a stabilising effect on learning the representation.
Revised: Overall, our ablation study indicates that having multiple competing learning objectives has a stabilising effect on learning the representation.

Original: Cluster Analysis As the best performing encoder, we focused on analysing the clusters produced by the TPC model.
Revised: Cluster Analysis As the best performing encoder, we focused on analysing the clusters produced by the TPC model.

Original: The cluster in which each patient spent the majority of their time in was assigned its primary cluster.
Revised: The cluster in which each patient spent the majority of their time in was assigned its primary cluster.

Original: Table 2 and 3 show summary statistics for each cluster.
Revised: Table 2 and 3 show summary statistics for each cluster.

Original: These are described in much more detail in Section A.2, along with visualisations.
Revised: These are described in much more detail in Section A.2, along with visualisations.

Original: Here we will simply say that:
Revised: Here we will simply say that:

Original: • Cluster 1 contains the sickest patients, with an average mortality of 72.0%.
Revised: • Cluster 1 contains the sickest patients, with an average mortality of 72.0%.

Original: They also have signs of severe respiratory distress.
Revised: They also have signs of severe respiratory distress.

Original: We could describe this phenotype as a ‘early, life-threatening pulmonary injury’ patient group.
Revised: We could describe this phenotype as a ‘early, life-threatening pulmonary injury’ patient group.

Original: • Cluster 2 also display substantial mortality and severe pulmonary dysfunction.
Revised: • Cluster 2 also display substantial mortality and severe pulmonary dysfunction.

Original: However they are also characterised by very long LoS and VD, with consequent high rates of tracheostomy.
Revised: However they are also characterised by very long LoS and VD, with consequent high rates of tracheostomy.

Original: This might be described as a ‘pulmonary critical illness’ phenotype.
Revised: This might be described as a ‘pulmonary critical illness’ phenotype.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: • Cluster 3 have the best outcomes, with short LoS and low mortality.
Revised: • Cluster 3 have the best outcomes, with short LoS and low mortality.

Original: This is a ‘short stay’ phenotype of most likely perioperative patients.
Revised: This is a ‘short stay’ phenotype of most likely perioperative patients.

Original: • Cluster 4 have relatively low mortality but high rates of tracheostomy.
Revised: • Cluster 4 have relatively low mortality but high rates of tracheostomy.

Original: We could therefore describe them as a ‘general critical illness’ phenotype.
Revised: We could therefore describe them as a ‘general critical illness’ phenotype.

Original: • Cluster 5 shows a moderate to severe group, who do not display the pulmonary injury of cluster 1.
Revised: • Cluster 5 shows a moderate to severe group, who do not display the pulmonary injury of cluster 1.

Original: Therefore this is an ‘early, life-threatening non-pulmonary injury’ phenotype.
Revised: Therefore this is an ‘early, life-threatening non-pulmonary injury’ phenotype.

Original: Reliability We investigated the reproducibility of these phenotypes.
Revised: Reliability We investigated the reproducibility of these phenotypes.

Original: We chose to analyse the clusters in the following settings: i) alternative encoder models, ii) retraining the TPC model with different random seeds and iii) varying the value of k.
Revised: We chose to analyse the clusters in the following settings: i) alternative encoder models, ii) retraining the TPC model with different random seeds and iii) varying the value of k.

Original: The clusters were found to be surprisingly stable, with key features of the extracted phenotypes remaining similar between models.
Revised: The clusters were found to be surprisingly stable, with key features of the extracted phenotypes remaining similar between models.

Original: With increasing value of k, we noticed that rather than completely rearranging the position of the clusters, increasing k progressively subdivides existing clusters, hinting that the clusters are hierarchically organised (more in the discussion).
Revised: With increasing value of k, we noticed that rather than completely rearranging the position of the clusters, increasing k progressively subdivides existing clusters, hinting that the clusters are hierarchically organised (more in the discussion).

Original: The full analysis is included in Section A.3.
Revised: The full analysis is included in Section A.3.

Original:  Discussion
Revised:  Discussion

Original: We evaluated the use of TPC model, trained using supervised, unsupervised and self-supervised learning techniques, for the purposes of phenotype discovery in mechanically ventilated patients.
Revised: We evaluated the use of TPC model, trained using supervised, unsupervised and self-supervised learning techniques, for the purposes of phenotype discovery in mechanically ventilated patients.

Original: We will discuss the most important findings in turn.
Revised: We will discuss the most important findings in turn.

Original: Firstly, we reaffirmed that the TPC model performs better than alternative encoders on EHR data for patient outcome prediction.
Revised: Firstly, we reaffirmed that the TPC model performs better than alternative encoders on EHR data for patient outcome prediction.

Original: This time on the Amsterdam UMC database [35], and with added tasks.
Revised: This time on the Amsterdam UMC database [35], and with added tasks.

Original: Secondly, we found that the Transformer outperformed LSTM on LoS and VD, but performed much worse on the mortality task, and slightly worse on the tracheostomy task.
Revised: Secondly, we found that the Transformer outperformed LSTM on LoS and VD, but performed much worse on the mortality task, and slightly worse on the tracheostomy task.

Original: This may be because the task weighting was more favourouble to the LSTM and TPC models, whereas the Transformer would have benefited from greater weighting towards the binary tasks.
Revised: This may be because the task weighting was more favourouble to the LSTM and TPC models, whereas the Transformer would have benefited from greater weighting towards the binary tasks.

Original: Another possibility is that the binary tasks particularly benefit from biases in the LSTM and TPC encoders, because these models naturally emphasise recent timepoints (and these are especially important for solving the mortality task).
Revised: Another possibility is that the binary tasks particularly benefit from biases in the LSTM and TPC encoders, because these models naturally emphasise recent timepoints (and these are especially important for solving the mortality task).

Original: As for the reason that the Transformer performs better on tracheostomy than mortality, it could be because there is positive correlation between the LoS, VD and tracheostomy tasks.
Revised: As for the reason that the Transformer performs better on tracheostomy than mortality, it could be because there is positive correlation between the LoS, VD and tracheostomy tasks.

Original: Solving the duration tasks makes the tracheostomy task easier, whereas the relation to mortality is more complex (Figure 2).
Revised: Solving the duration tasks makes the tracheostomy task easier, whereas the relation to mortality is more complex (Figure 2).

Original: To briefly comment on the reconstruction results in Table 14 in the Appendix; it may seem surprising that the LSTM model performs best on the reconstruction and forecasting tasks.
Revised: To briefly comment on the reconstruction results in Table 14 in the Appendix; it may seem surprising that the LSTM model performs best on the reconstruction and forecasting tasks.

Original: However, this could be explained if the LSTM is creating simpler ‘lower level’ representations that are easier to translate back to the original data using two-layer decoder networks.
Revised: However, this could be explained if the LSTM is creating simpler ‘lower level’ representations that are easier to translate back to the original data using two-layer decoder networks.

Original: Thirdly, Table 4 reveals a general trend that the more tasks that are added, the better results across all the tasks, with particular benefits to the tracheostomy task.
Revised: Thirdly, Table 4 reveals a general trend that the more tasks that are added, the better results across all the tasks, with particular benefits to the tracheostomy task.

Original: The exception to this was the duration only setting.
Revised: The exception to this was the duration only setting.

Original: There are two possible explanations for the discrepancy:
Revised: There are two possible explanations for the discrepancy:

Original: The weighting of the duration task was not sufficient.
Revised: The weighting of the duration task was not sufficient.

Original: The tracheostomy task (but not mortality) reduces the performance on the duration tasks.
Revised: The tracheostomy task (but not mortality) reduces the performance on the duration tasks.

Original: The former does not seem likely, because the Transformer is probably over-weighting the duration tasks, and yet, it follows the same trend as the LSTM and TPC.
Revised: The former does not seem likely, because the Transformer is probably over-weighting the duration tasks, and yet, it follows the same trend as the LSTM and TPC.

Original: The latter may appear to be counterintuitive, because the duration tasks are correlated with tracheostomy.
Revised: The latter may appear to be counterintuitive, because the duration tasks are correlated with tracheostomy.

Original: Usually this is an advantage of multitask learning, because it enhances the signal:noise ratio when certain types of noise only apply to one task.
Revised: Usually this is an advantage of multitask learning, because it enhances the signal:noise ratio when certain types of noise only apply to one task.

Original: However, looking closely at Figure 2, we can see that there is an area of patients near the bottom of the figure, in cluster 5.
Revised: However, looking closely at Figure 2, we can see that there is an area of patients near the bottom of the figure, in cluster 5.

Original: These patients have long VD and LoS but have been separated from the other long stay patients in clusters 2 and 4.
Revised: These patients have long VD and LoS but have been separated from the other long stay patients in clusters 2 and 4.

Original: The separation can be attributed to these patients never receiving tracheostomies, therefore the tracheostomy task forces the representation space to separate these groups when they would be otherwise be aligned.
Revised: The separation can be attributed to these patients never receiving tracheostomies, therefore the tracheostomy task forces the representation space to separate these groups when they would be otherwise be aligned.

Original: Given the simple nature of the predictor networks, this may harm the performance on the duration tasks because the predictor cannot effectively map these patients to appropriately long LoS. This theory would be formally tested by accompanying the duration tasks with the mortality task only.
Revised: Given the simple nature of the predictor networks, this may harm the performance on the duration tasks because the predictor cannot effectively map these patients to appropriately long LoS. This theory would be formally tested by accompanying the duration tasks with the mortality task only.

Original: Finally, regarding the repeatability of the clustering, we demonstrated that key aspects of the learned representations (both of different encoders and TPC instances) are consistently recognised.
Revised: Finally, regarding the repeatability of the clustering, we demonstrated that key aspects of the learned representations (both of different encoders and TPC instances) are consistently recognised.

Original: The separation on other traits, especially when distinguishing the sickest patients from the moderately ill, was more malleable.
Revised: The separation on other traits, especially when distinguishing the sickest patients from the moderately ill, was more malleable.

Original: This suggests that perhaps there is not a well defined distinction between these, but rather a scale of deterioration, through which an arbitrary line can be drawn.
Revised: This suggests that perhaps there is not a well defined distinction between these, but rather a scale of deterioration, through which an arbitrary line can be drawn.

Original:  Limitations and Future Work
Revised:  Limitations and Future Work

Original: Hierarchical Clustering It is evident that certain clusters are more related than others.
Revised: Hierarchical Clustering It is evident that certain clusters are more related than others.

Original: A tree based hierarchy of clusters seems more natural than a flat structure.
Revised: A tree based hierarchy of clusters seems more natural than a flat structure.

Original: We are particularly interested in modifying an approach for genetics data [23, 4, 7], and are hoping to apply it to the ICU.
Revised: We are particularly interested in modifying an approach for genetics data [23, 4, 7], and are hoping to apply it to the ICU.

Original: Contrastive Learning We are investigating the use of contrastive learning to regularise the embedding space e.g. in [40].
Revised: Contrastive Learning We are investigating the use of contrastive learning to regularise the embedding space e.g. in [40].

Original: Currently, there is no explicit loss to enforce relative positioning of the embeddings.
Revised: Currently, there is no explicit loss to enforce relative positioning of the embeddings.

Original: Despite this, we have empirically found the clusters presented in this paper to be very stable, both temporally and to encoder type.
Revised: Despite this, we have empirically found the clusters presented in this paper to be very stable, both temporally and to encoder type.

Original: This is likely to be because our predictor and decoder networks are very simple, meaning that the embedding space does not have the freedom to model similar patient trajectories (which should be in the same cluster) in disparate parts of the embedding space.
Revised: This is likely to be because our predictor and decoder networks are very simple, meaning that the embedding space does not have the freedom to model similar patient trajectories (which should be in the same cluster) in disparate parts of the embedding space.

Original: Nevertheless, contrastive learning could provide further regularisation.
Revised: Nevertheless, contrastive learning could provide further regularisation.

Original: Generalisability While we have shown that the clusters are surprisingly stable, repeating the work on another dataset (e.g. MIMIC-IV or eICU) would strengthen this assessment.
Revised: Generalisability While we have shown that the clusters are surprisingly stable, repeating the work on another dataset (e.g. MIMIC-IV or eICU) would strengthen this assessment.

Original:  Summary
Revised:  Summary

Original: While we acknowledge important limitations in our work, we have shown that:
Revised: While we acknowledge important limitations in our work, we have shown that:

Original: The TPC model outperforms alternative encoders on patient outcome prediction tasks.
Revised: The TPC model outperforms alternative encoders on patient outcome prediction tasks.

Original: We can generate clinically meaningful and interpretable clusters using this technique.
Revised: We can generate clinically meaningful and interpretable clusters using this technique.

Original: The phenotypes are similar across choices of encoder and number of clusters.
Revised: The phenotypes are similar across choices of encoder and number of clusters.

Original: The cluster assignment is remarkably stable over time, and membership is determined early on.
Revised: The cluster assignment is remarkably stable over time, and membership is determined early on.

Original: This is particularly encouraging as a substrate for future intervention studies, because they rely on phenotyping before any intervention.
Revised: This is particularly encouraging as a substrate for future intervention studies, because they rely on phenotyping before any intervention.

Original: Stable cluster transitions do occur but they are infrequent.
Revised: Stable cluster transitions do occur but they are infrequent.

Original: Studying these transitions with a view towards understanding the causes is an important avenue for future work.
Revised: Studying these transitions with a view towards understanding the causes is an important avenue for future work.

Original: A Further Analyses A.1 Ablation Study
Revised: A Further Analyses A.1 Ablation Study

Original: We performed an ablation study on the tasks used to train the representation space.
Revised: We performed an ablation study on the tasks used to train the representation space.

Original: The results are shown in Table 4.
Revised: The results are shown in Table 4.

Original: Firstly, we see that the best results for all tasks (except for the duration tasks) are achieved in the full multi-task setting.
Revised: Firstly, we see that the best results for all tasks (except for the duration tasks) are achieved in the full multi-task setting.

Original: Not a single metric improves in the other ablation settings, and yet at least one metric showed a deterioration in performance (the exception in task setting (g) is discussed below).
Revised: Not a single metric improves in the other ablation settings, and yet at least one metric showed a deterioration in performance (the exception in task setting (g) is discussed below).

Original: Overall this indicates that having multiple competing learning objectives has a stabilising effect on learning the representation.
Revised: Overall this indicates that having multiple competing learning objectives has a stabilising effect on learning the representation.

Original: (c) – No Forecasting Experiment (c) included all the tasks except forecasting one timestep ahead.
Revised: (c) – No Forecasting Experiment (c) included all the tasks except forecasting one timestep ahead.

Original: When we compare experiment (c) to (a), we see that the results are mostly similar, but there is a consistent decrease in performance, which is statistically significant at the p<0.05 level on the tracheostomy task (AUPRC in the TPC model and AUROC in the Transformer model).
Revised: When we compare experiment (c) to (a), we see that the results are mostly similar, but there is a consistent decrease in performance, which is statistically significant at the p<0.05 level on the tracheostomy task (AUPRC in the TPC model and AUROC in the Transformer model).

Original: On the reconstruction task, again the performance is similar but statistically worse in the last timestep reconstruction in the LSTM model.
Revised: On the reconstruction task, again the performance is similar but statistically worse in the last timestep reconstruction in the LSTM model.

Original: This means that the forecasting task is contributing slightly to the performance in (a), but the benefit is small.
Revised: This means that the forecasting task is contributing slightly to the performance in (a), but the benefit is small.

Original: (d) – No Reconstruction Experiment (d) removes both the timestep t reconstruction and the static data reconstruction tasks, but keeps the forecasting task.
Revised: (d) – No Reconstruction Experiment (d) removes both the timestep t reconstruction and the static data reconstruction tasks, but keeps the forecasting task.

Original: The effect
Revised: The effect

Original: size
Revised: size

Original: is larger than in (c),
Revised: is larger than in (c),

Original: but again
Revised: but again

Original: is
Revised: is

Original: only
Revised: only

Original: statistically significant on the tracheostomy task.
Revised: statistically significant on the tracheostomy task.

Original: The forecasting task performs
Revised: The forecasting task performs

Original: significantly
Revised: significantly

Original: worse in the Transformer and LSTM models without the reconstruction.
Revised: worse in the Transformer and LSTM models without the reconstruction.

Original: (e) – Prediction Tasks Only Experiment (e) includes the binary and duration prediction tasks, but no reconstruction or forecasting.
Revised: (e) – Prediction Tasks Only Experiment (e) includes the binary and duration prediction tasks, but no reconstruction or forecasting.

Original: The performance again deteriorates, particularly on the tracheostomy task, we also start to see a more noticeable deterioration in the duration tasks, although this is not yet statistically significant.
Revised: The performance again deteriorates, particularly on the tracheostomy task, we also start to see a more noticeable deterioration in the duration tasks, although this is not yet statistically significant.

Original: ) is only the binary tasks, and (g) is only the duration tasks.
Revised: ) is only the binary tasks, and (g) is only the duration tasks.

Original: The colour scheme, metrics and statistical test comparisons are explained in the legend to Table
Revised: The colour scheme, metrics and statistical test comparisons are explained in the legend to Table

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Binary Tasks Only Experiment (
Revised: Binary Tasks Only Experiment (

Original: ) follows the trend of worsening performance as tasks are removed.
Revised: ) follows the trend of worsening performance as tasks are removed.

Original: This means that the mortality and tracheostomy tasks consistently benefit from supplementary tasks which help to distinguish signal from noise
Revised: This means that the mortality and tracheostomy tasks consistently benefit from supplementary tasks which help to distinguish signal from noise

Original: [Table]
Revised: [Table]

Original: (g) – Duration Tasks Only Experiment (g) shows unexpected results; all of the models return better results when only predicting LoS and VD.
Revised: (g) – Duration Tasks Only Experiment (g) shows unexpected results; all of the models return better results when only predicting LoS and VD.

Original: This is not what has been observed previously in multitask settings ([27, 11]).
Revised: This is not what has been observed previously in multitask settings ([27, 11]).

Original: This is discussed further in Section 5.
Revised: This is discussed further in Section 5.

Original: However, overall the trend is such that the more tasks that are included, the better the average results across tasks.
Revised: However, overall the trend is such that the more tasks that are included, the better the average results across tasks.

Original: A.2 Cluster Analysis
Revised: A.2 Cluster Analysis

Original: As the best performing encoder, we have focused on analysing the clusters produced by the TPC model.
Revised: As the best performing encoder, we have focused on analysing the clusters produced by the TPC model.

Original: In order to analyse the average differences between the patients in each cluster, it was necessary to flatten the clustering into one ‘primary’ cluster per patient.
Revised: In order to analyse the average differences between the patients in each cluster, it was necessary to flatten the clustering into one ‘primary’ cluster per patient.

Original: This was to prevent confusion, since patients can enter multiple clusters during their ICU stay (sometimes only for one or two timepoints), and this is disproportionately true of the long stay patients.
Revised: This was to prevent confusion, since patients can enter multiple clusters during their ICU stay (sometimes only for one or two timepoints), and this is disproportionately true of the long stay patients.

Original: The cluster in which each patient spent the majority of their time in was assigned its primary cluster.
Revised: The cluster in which each patient spent the majority of their time in was assigned its primary cluster.

Original: If there were multiple modes, then the mode experienced later in the sequence was chosen.
Revised: If there were multiple modes, then the mode experienced later in the sequence was chosen.

Original: The next two sections A.2.1 and A.2.2 characterise the behaviour of the primary clusters.
Revised: The next two sections A.2.1 and A.2.2 characterise the behaviour of the primary clusters.

Original: Section A.2.3 will then analyse the dynamic aspects of the clustering from multiple perspectives.
Revised: Section A.2.3 will then analyse the dynamic aspects of the clustering from multiple perspectives.

Original: A.2.1 Differences in Phenotype and Outcomes
Revised: A.2.1 Differences in Phenotype and Outcomes

Original: Table 2 shows the mean outcomes for each cluster.
Revised: Table 2 shows the mean outcomes for each cluster.

Original: We also analysed some key features in the original data, to visualise differences in patient phenotype that the model identified.
Revised: We also analysed some key features in the original data, to visualise differences in patient phenotype that the model identified.

Original: The average values of key features in patients divided by primary cluster are shown in Table 3.
Revised: The average values of key features in patients divided by primary cluster are shown in Table 3.

Original: Broadly we can say that:
Revised: Broadly we can say that:

Original: • Cluster 1 contains the sickest patients, with an average mortality of 72.0%.
Revised: • Cluster 1 contains the sickest patients, with an average mortality of 72.0%.

Original: They are short stay patients and unsurprisingly they have the lowest rate of tracheostomy as most do not survive or stay long enough to require complex respiratory weaning.
Revised: They are short stay patients and unsurprisingly they have the lowest rate of tracheostomy as most do not survive or stay long enough to require complex respiratory weaning.

Original: Table
Revised: Table

Original: shows they are primarily ventilated with ‘mandatory’ ventilation settings, meaning the machine is deciding the respiratory rate, and either the tidal volume or pressure differential across the lung.
Revised: shows they are primarily ventilated with ‘mandatory’ ventilation settings, meaning the machine is deciding the respiratory rate, and either the tidal volume or pressure differential across the lung.

Original: Furthermore, they have the least compliant lungs with higher peak inspiratory pressure and higher PEEP.
Revised: Furthermore, they have the least compliant lungs with higher peak inspiratory pressure and higher PEEP.

Original: This means that the physical properties of the lung have been damaged and it is harder to inflate the lung.
Revised: This means that the physical properties of the lung have been damaged and it is harder to inflate the lung.

Original: The P/F ratio is also very low, which indicates that the lung function is also impaired i.e. the patients cannot absorb as much oxygen.
Revised: The P/F ratio is also very low, which indicates that the lung function is also impaired i.e. the patients cannot absorb as much oxygen.

Original: This is in keeping with severe respiratory distress.
Revised: This is in keeping with severe respiratory distress.

Original: We could describe this phenotype as a ‘early, life-threatening pulmonary injury’ patient group.
Revised: We could describe this phenotype as a ‘early, life-threatening pulmonary injury’ patient group.

Original: • Cluster 2 also display substantial mortality and, again, from Table 3 it becomes clear that these patients also represent a group with severe pulmonary dysfunction like cluster 1.
Revised: • Cluster 2 also display substantial mortality and, again, from Table 3 it becomes clear that these patients also represent a group with severe pulmonary dysfunction like cluster 1.

Original: However this phenotype is characterised by very long LoS and VD, with consequent high rates of tracheostomy: this phenotype represents patients who are difficult to wean from mechanical ventilation with resultant long ICU stay.
Revised: However this phenotype is characterised by very long LoS and VD, with consequent high rates of tracheostomy: this phenotype represents patients who are difficult to wean from mechanical ventilation with resultant long ICU stay.

Original: This might be described as a ‘pulmonary critical illness’ phenotype.
Revised: This might be described as a ‘pulmonary critical illness’ phenotype.

Original: • Cluster 3 have the best outcomes, with short LoS and low mortality.
Revised: • Cluster 3 have the best outcomes, with short LoS and low mortality.

Original: They are extubated without tracheostomy, because they have less pulmonary injury.
Revised: They are extubated without tracheostomy, because they have less pulmonary injury.

Original: This appears to be a ‘short stay’ phenotype who require a brief period of organ support, perhaps after significant surgery.
Revised: This appears to be a ‘short stay’ phenotype who require a brief period of organ support, perhaps after significant surgery.

Original: • Cluster 4 have relatively low mortality but high rates of tracheostomy.
Revised: • Cluster 4 have relatively low mortality but high rates of tracheostomy.

Original: Table 3 shows modest levels of respiratory failure and good lung compliance.
Revised: Table 3 shows modest levels of respiratory failure and good lung compliance.

Original: Thus, whilst these patients are difficult to wean from mechanical ventilation (like cluster 2), this is due to factors that are not primarily related to pulmonary pathology: once they receive a tracheostomy they are rapidly liberated from mechanical ventilation.
Revised: Thus, whilst these patients are difficult to wean from mechanical ventilation (like cluster 2), this is due to factors that are not primarily related to pulmonary pathology: once they receive a tracheostomy they are rapidly liberated from mechanical ventilation.

Original: We could therefore describe them as a ‘general critical illness’ phenotype.
Revised: We could therefore describe them as a ‘general critical illness’ phenotype.

Original: • Cluster 5 shows a moderate to severe group, who are not as acutely unwell as cluster 1, but are still high-risk.
Revised: • Cluster 5 shows a moderate to severe group, who are not as acutely unwell as cluster 1, but are still high-risk.

Original: From Table 3 we see that pulmonary injury is not a prominent feature so we could characterise these patients as ‘early, life-threatening non-pulmonary injury’ patients.
Revised: From Table 3 we see that pulmonary injury is not a prominent feature so we could characterise these patients as ‘early, life-threatening non-pulmonary injury’ patients.

Original: Overall the findings from Tables 2 and 3 show that there are statistically significant and clinically meaningful differences between the clusters.
Revised: Overall the findings from Tables 2 and 3 show that there are statistically significant and clinically meaningful differences between the clusters.

Original: These can be visualised in Figure which provides maps of the representation space, showing how the clusters relate to one another as well as the distribution of outcomes and features.
Revised: These can be visualised in Figure which provides maps of the representation space, showing how the clusters relate to one another as well as the distribution of outcomes and features.

Original: A.2.2 Medoid Analysis
Revised: A.2.2 Medoid Analysis

Original: The medoids produced by the clustering algorithm are shown in Figure 3 and give a description of a representative patient in each cluster.
Revised: The medoids produced by the clustering algorithm are shown in Figure 3 and give a description of a representative patient in each cluster.

Original: Note that each medoid corresponds to both a patient and a specific time-point in their ventilation episode 2 .
Revised: Note that each medoid corresponds to both a patient and a specific time-point in their ventilation episode 2 .

Original: • The medoid patient for cluster 1 (female, age 60-69) died 4 hours after the episode shown without a tracheostomy.
Revised: • The medoid patient for cluster 1 (female, age 60-69) died 4 hours after the episode shown without a tracheostomy.

Original: Her deterioration is predictable from multiple parameters in the data but infection (high WBC) and pulmonary dysfunction are particularly noteworthy.
Revised: Her deterioration is predictable from multiple parameters in the data but infection (high WBC) and pulmonary dysfunction are particularly noteworthy.

Original: • The typical medoid patient representing cluster 2 (male, 80+ years old) received a tracheostomy 19 days after the episode shown, and was discharged at 23 days.
Revised: • The typical medoid patient representing cluster 2 (male, 80+ years old) received a tracheostomy 19 days after the episode shown, and was discharged at 23 days.

Original: This patient
Revised: This patient

Original: required late as well as early mandatory ventilation suggesting that he ran into later pulmonary complications – mostly likely infectious since his CRP remains high throughout the stay.
Revised: required late as well as early mandatory ventilation suggesting that he ran into later pulmonary complications – mostly likely infectious since his CRP remains high throughout the stay.

Original: • The medoid patient in cluster 3 (female, age 60-69) was discharged from hospital the day after her brief window of ventilation.
Revised: • The medoid patient in cluster 3 (female, age 60-69) was discharged from hospital the day after her brief window of ventilation.

Original: She does not display substantial physiological derangement.
Revised: She does not display substantial physiological derangement.

Original: • The patient in cluster 4 (female, age 60-69) received a tracheostomy 3 days after the sequence shown.
Revised: • The patient in cluster 4 (female, age 60-69) received a tracheostomy 3 days after the sequence shown.

Original: Her lung compliance and P/F ratio are both high compared to clusters 1 and 2, indicating better lung function.
Revised: Her lung compliance and P/F ratio are both high compared to clusters 1 and 2, indicating better lung function.

Original: Therefore, we can conclude that she needed a tracheostomy for reasons other than lung injury.
Revised: Therefore, we can conclude that she needed a tracheostomy for reasons other than lung injury.

Original: This is also supported her ventilator settings: low peak inspiratory pressure and PEEP and no mandatory ventilation modes.
Revised: This is also supported her ventilator settings: low peak inspiratory pressure and PEEP and no mandatory ventilation modes.

Original: Her end tidal COlevels suggest a degree of hypoventilation which improves as she is slowly weaned from mechanical ventilation.
Revised: Her end tidal COlevels suggest a degree of hypoventilation which improves as she is slowly weaned from mechanical ventilation.

Original: • Lastly, the patient in cluster 5 (female, 80+ years old) stayed for 9 further days in hospital before being discharged.
Revised: • Lastly, the patient in cluster 5 (female, 80+ years old) stayed for 9 further days in hospital before being discharged.

Original: The short duration of ventilation and relatively normal pulmonary physiology is again consistent with a non-pulmonary phenotype.
Revised: The short duration of ventilation and relatively normal pulmonary physiology is again consistent with a non-pulmonary phenotype.

Original: [Figure]
Revised: [Figure]

Original: A.2.3 Temporal Analysis
Revised: A.2.3 Temporal Analysis

Original: Broadly, there are two perspectives when evaluating the dynamic aspects of this clustering.
Revised: Broadly, there are two perspectives when evaluating the dynamic aspects of this clustering.

Original: One is the ‘Markovian’ perspective, where we can examine the transition function between clusters.
Revised: One is the ‘Markovian’ perspective, where we can examine the transition function between clusters.

Original: This is shown in Figure 4.
Revised: This is shown in Figure 4.

Original: Unsurprisingly, this reveals that the patient is always most likely to remain in the same cluster.
Revised: Unsurprisingly, this reveals that the patient is always most likely to remain in the same cluster.

Original: However the most common inter-cluster transitions are from cluster 5 to cluster 4, and cluster 1 to cluster 5.
Revised: However the most common inter-cluster transitions are from cluster 5 to cluster 4, and cluster 1 to cluster 5.

Original: Note that these clusters are next to one another and share lengthy borders in Figure 2.
Revised: Note that these clusters are next to one another and share lengthy borders in Figure 2.

Original: Most of the patients who transition to ‘Died’ come from cluster 1, and most of the ‘Discharged’ patients come from cluster 3.
Revised: Most of the patients who transition to ‘Died’ come from cluster 1, and most of the ‘Discharged’ patients come from cluster 3.

Original: The other perspective is to look at the number of patients in each cluster at different time points after admission, and observe the transitions between them (Figure 5).
Revised: The other perspective is to look at the number of patients in each cluster at different time points after admission, and observe the transitions between them (Figure 5).

Original: Transitions from cluster 3 to ‘extubated’ are very common within the first day, but then they almost disappear by 3 days.
Revised: Transitions from cluster 3 to ‘extubated’ are very common within the first day, but then they almost disappear by 3 days.

Original: This cannot be seen with the Markovian perspective in Figure 4.
Revised: This cannot be seen with the Markovian perspective in Figure 4.

Original: Cluster 2 contains patients with the longest ventilation episodes, which can be seen by its low rate of attrition over time.
Revised: Cluster 2 contains patients with the longest ventilation episodes, which can be seen by its low rate of attrition over time.

Original: Number of clusters per patient Figure 6 shows that most patients remain in only one cluster during their ventilation episode.
Revised: Number of clusters per patient Figure 6 shows that most patients remain in only one cluster during their ventilation episode.

Original: However, when the distribution is broken down by primary cluster, we can see that this is heavily driven by the behaviour of cluster 3 patients, which tend to remain in cluster 3 for their entire ventilation duration (note that they tend to have short VDs so this is not so surprising).
Revised: However, when the distribution is broken down by primary cluster, we can see that this is heavily driven by the behaviour of cluster 3 patients, which tend to remain in cluster 3 for their entire ventilation duration (note that they tend to have short VDs so this is not so surprising).

Original: In contrast, clusters 2 and 5 most commonly appear alongside other clusters during a single ventilation episode.
Revised: In contrast, clusters 2 and 5 most commonly appear alongside other clusters during a single ventilation episode.

Original: This means that for most episodes attributed to cluster 2 or 5, there are transitions either into or out of these clusters.
Revised: This means that for most episodes attributed to cluster 2 or 5, there are transitions either into or out of these clusters.

Original: These are explored next.
Revised: These are explored next.

Original: Cluster transitions The clusters produced by the TPC model are remarkably stable over time, given that there is no explicit loss incentive to constrain the representation to behave in this way.
Revised: Cluster transitions The clusters produced by the TPC model are remarkably stable over time, given that there is no explicit loss incentive to constrain the representation to behave in this way.

Original: Figure 7 shows the distribution of timepoints that the patients first enter their primary cluster.
Revised: Figure 7 shows the distribution of timepoints that the patients first enter their primary cluster.

Original: Clusters 2 and 3 are particularly likely to accurately assigned during the first hour of ventilation (87% and 89% respectively), while cluster 4 is the least likely to be identified early (64%).
Revised: Clusters 2 and 3 are particularly likely to accurately assigned during the first hour of ventilation (87% and 89% respectively), while cluster 4 is the least likely to be identified early (64%).

Original: Next, we investigated what we will refer to as ‘stable’ transitions between clusters.
Revised: Next, we investigated what we will refer to as ‘stable’ transitions between clusters.

Original: In order to be characterised as stable, the origin cluster needed to remain stable in the 5 hours preceding the transition, and the patient was not permitted to re-enter the origin cluster for 5 hours following the transition.
Revised: In order to be characterised as stable, the origin cluster needed to remain stable in the 5 hours preceding the transition, and the patient was not permitted to re-enter the origin cluster for 5 hours following the transition.

Original: This was primarily to screen out patients who were at the boundary between two clusters, continually crossing back and forth but not representing a true transition from one to the other.
Revised: This was primarily to screen out patients who were at the boundary between two clusters, continually crossing back and forth but not representing a true transition from one to the other.

Original: Before screening, there were 22,036 cluster transitions, corresponding to 870 separate ventilation episodes (39% of the total in the test set).
Revised: Before screening, there were 22,036 cluster transitions, corresponding to 870 separate ventilation episodes (39% of the total in the test set).

Original: Of these transitions, only 291 represented stable movement between clusters.
Revised: Of these transitions, only 291 represented stable movement between clusters.

Original: We further removed any transitions between two clusters that had fewer than 15 transition examples, as this would be insufficient to analyse.
Revised: We further removed any transitions between two clusters that had fewer than 15 transition examples, as this would be insufficient to analyse.

Original: The remaining 230 transitions are shown in
Revised: The remaining 230 transitions are shown in

Original: Table 5.
Revised: Table 5.

Original: Firstly, it is noteworthy that the outcomes reflect the destination cluster, not the origin cluster.
Revised: Firstly, it is noteworthy that the outcomes reflect the destination cluster, not the origin cluster.

Original: The exception to this is the ‘urgency’ column, which is not an outcome, but a label assigned at admission and hence is more likely to reflect the origin cluster (although it is worth mentioning that the origin cluster is not necessarily the cluster at admission).
Revised: The exception to this is the ‘urgency’ column, which is not an outcome, but a label assigned at admission and hence is more likely to reflect the origin cluster (although it is worth mentioning that the origin cluster is not necessarily the cluster at admission).

Original: Cluster 5 stands out as being disproportionately involved in inter-cluster transitions.
Revised: Cluster 5 stands out as being disproportionately involved in inter-cluster transitions.

Original: Of these, the most common is 5 → 3, which occurs when the model overestimates the risk to the patient early on in the ventilation episode.
Revised: Of these, the most common is 5 → 3, which occurs when the model overestimates the risk to the patient early on in the ventilation episode.

Original: Not shown in Figure 5, is that the average predicted risk of death drops from 56.4% 5 hours prior to the transition, to 41.7% at the point of transition.
Revised: Not shown in Figure 5, is that the average predicted risk of death drops from 56.4% 5 hours prior to the transition, to 41.7% at the point of transition.

Original: There is also a corresponding reduction in tracheostomy risk (-13%), LoS (-17.1% after adjustment 3 ) and VD (-26.4% after adjustment) as predicted by the model, and dramatic improvements in physiological parameters such as lung compliance (+35%) and P/F ratio (+15%).
Revised: There is also a corresponding reduction in tracheostomy risk (-13%), LoS (-17.1% after adjustment 3 ) and VD (-26.4% after adjustment) as predicted by the model, and dramatic improvements in physiological parameters such as lung compliance (+35%) and P/F ratio (+15%).

Original: Another interesting transition is 3 → 1, which happens when the model initially believes the patient to be relatively healthy, but then quickly re-adjusts to predict poor outcomes.
Revised: Another interesting transition is 3 → 1, which happens when the model initially believes the patient to be relatively healthy, but then quickly re-adjusts to predict poor outcomes.

Original: Looking in more detail at the raw data, we discovered that these patients are younger (only 23.5% are 70+), which could explain why the model was initially optimistic and why the deterioration is so rapid 4 .
Revised: Looking in more detail at the raw data, we discovered that these patients are younger (only 23.5% are 70+), which could explain why the model was initially optimistic and why the deterioration is so rapid 4 .

Original: We also observed a deterioration in the lung compliance (-26.3%) and P/F ratios (-12.8%), and a change in the ventilator
Revised: We also observed a deterioration in the lung compliance (-26.3%) and P/F ratios (-12.8%), and a change in the ventilator

Original: [Table]
Revised: [Table]

Original: settings – namely higher PEEP and peak inspiratory pressure and lower tidal volumes – reflecting a drop in lung compliance of the patients.
Revised: settings – namely higher PEEP and peak inspiratory pressure and lower tidal volumes – reflecting a drop in lung compliance of the patients.

Original: Most of these patients died within 12 hours of the transition to cluster 1.
Revised: Most of these patients died within 12 hours of the transition to cluster 1.

Original: A.3 Cluster Reliability
Revised: A.3 Cluster Reliability

Original: Choice of encoder Figure 8 compares the cluster assignments using different encoder models.
Revised: Choice of encoder Figure 8 compares the cluster assignments using different encoder models.

Original: It is encouraging that there is a strong cohesion between some of the clusters, meaning that the models are picking out genuine and consistent patterns in the data.
Revised: It is encouraging that there is a strong cohesion between some of the clusters, meaning that the models are picking out genuine and consistent patterns in the data.

Original: If we examine the TPC/LSTM comparison (far left in Figure 8) we can see that clusters 2, 3 andin the TPC correspond to 5, 3 and 1 respectively in the LSTM.
Revised: If we examine the TPC/LSTM comparison (far left in Figure 8) we can see that clusters 2, 3 andin the TPC correspond to 5, 3 and 1 respectively in the LSTM.

Original: In addition, clusters 1 and 5 (TPC) imperfectly map to 2 and 4 (LSTM) – the main difference being that some additional patients in cluster 5 in the TPC map to cluster 2 in the LSTM.
Revised: In addition, clusters 1 and 5 (TPC) imperfectly map to 2 and 4 (LSTM) – the main difference being that some additional patients in cluster 5 in the TPC map to cluster 2 in the LSTM.

Original: This means that the k-medoids algorithm has placed a different boundary between these groups in the clustering process.
Revised: This means that the k-medoids algorithm has placed a different boundary between these groups in the clustering process.

Original: Looking back to Figure 2, clusters 1 and 5 are revealed to be neighbours – in fact, cluster 5 appears to envelope cluster 1, suggesting that 1 is a sub-cluster of 5.
Revised: Looking back to Figure 2, clusters 1 and 5 are revealed to be neighbours – in fact, cluster 5 appears to envelope cluster 1, suggesting that 1 is a sub-cluster of 5.

Original: This is also consistent with the clinical picture shown in
Revised: This is also consistent with the clinical picture shown in

Original: Table 2, where the main difference is that cluster 1 appear to have acute pulmonary dysfunction, most likely in addition to other organ failures.
Revised: Table 2, where the main difference is that cluster 1 appear to have acute pulmonary dysfunction, most likely in addition to other organ failures.

Original: Therefore, the explanation for this discrepancy is that the LSTM has a more generous threshold than the TPC for inclusion in its highest risk ‘early, life-threatening pulmonary injury’ category (cluster 2).
Revised: Therefore, the explanation for this discrepancy is that the LSTM has a more generous threshold than the TPC for inclusion in its highest risk ‘early, life-threatening pulmonary injury’ category (cluster 2).

Original: In the TPC/Transformer comparison, the clusters largely correlate, except that cluster 5 patients in the TPC have been placed into cluster 4 in the Transformer.
Revised: In the TPC/Transformer comparison, the clusters largely correlate, except that cluster 5 patients in the TPC have been placed into cluster 4 in the Transformer.

Original: Further investigation revealed these to be patients with increased risk of receiving a tracheostomy (i.e. the patients which lay closest to the decision boundary between the clusters).
Revised: Further investigation revealed these to be patients with increased risk of receiving a tracheostomy (i.e. the patients which lay closest to the decision boundary between the clusters).

Original: The LSTM/Transformer comparison mirrors some of the correlations in TPC/LSTM but the mapping is less precise.
Revised: The LSTM/Transformer comparison mirrors some of the correlations in TPC/LSTM but the mapping is less precise.

Original: This could be because the models do not perform as well, making the representations less reliable.
Revised: This could be because the models do not perform as well, making the representations less reliable.

Original: It could also be because the models have different affinities for the various tasks, creating divergent biases in the representation space.
Revised: It could also be because the models have different affinities for the various tasks, creating divergent biases in the representation space.

Original: It is worth noting that in all three models, cluster 3 is the most distinct.
Revised: It is worth noting that in all three models, cluster 3 is the most distinct.

Original: This is unsurprising because it corresponds to patients whose physiology is closest to ‘normal’.
Revised: This is unsurprising because it corresponds to patients whose physiology is closest to ‘normal’.

Original: Therefore this group is the most homogeneous and can always be identified easily.
Revised: Therefore this group is the most homogeneous and can always be identified easily.

Original: Retraining TPC
Revised: Retraining TPC

Original: We retrained the TPC model 5 times with different initialisation generated by different random seeds and compared the resulting clusters to the original (Figure 9).
Revised: We retrained the TPC model 5 times with different initialisation generated by different random seeds and compared the resulting clusters to the original (Figure 9).

Original: Overall, there is strong cohesion between the models, but sometimes there are shifts in the boundaries between neighbouring clusters in Figure 2.
Revised: Overall, there is strong cohesion between the models, but sometimes there are shifts in the boundaries between neighbouring clusters in Figure 2.

Original: Especially between cluster 5 (moderate-severe) and cluster(severe), where some models (TPC 2, TPC 3, TPC 4) allow patients in cluster 5 to enter their ‘cluster 1’ equivalent.
Revised: Especially between cluster 5 (moderate-severe) and cluster(severe), where some models (TPC 2, TPC 3, TPC 4) allow patients in cluster 5 to enter their ‘cluster 1’ equivalent.

Original: Nevertheless, very distinct phenotypes are almost never mixed e.g. clusters 2 and 3,and 4, or 1 and 3 (as defined by the TPC 1 model).
Revised: Nevertheless, very distinct phenotypes are almost never mixed e.g. clusters 2 and 3,and 4, or 1 and 3 (as defined by the TPC 1 model).

Original: As in the encoder comparisons, cluster 3 is always well characterised.
Revised: As in the encoder comparisons, cluster 3 is always well characterised.

Original: Number of clusters The value of k was determined using the elbow method.
Revised: Number of clusters The value of k was determined using the elbow method.

Original: In Figure 10, we show how the clusters would appear with increasing value of k.
Revised: In Figure 10, we show how the clusters would appear with increasing value of k.

Original: What is most striking is that each time a cluster is added, the new cluster either inserts itself within an existing cluster, or it appears at the intersection between existing clusters.
Revised: What is most striking is that each time a cluster is added, the new cluster either inserts itself within an existing cluster, or it appears at the intersection between existing clusters.

Original: For example, as we move from 2 to 3 clusters, the new cluster 3 is almost completely contained within the old cluster 1.
Revised: For example, as we move from 2 to 3 clusters, the new cluster 3 is almost completely contained within the old cluster 1.

Original: This pattern of sub-dividing an existing cluster generally continues until we reach 6 and 7 clusters, when the new cluster inserts itself at the boundary between two or more old clusters.
Revised: This pattern of sub-dividing an existing cluster generally continues until we reach 6 and 7 clusters, when the new cluster inserts itself at the boundary between two or more old clusters.

Original: In other words, increasing the value of k does not completely shift the position of all of the clusters, but rather it carefully subdivides them.
Revised: In other words, increasing the value of k does not completely shift the position of all of the clusters, but rather it carefully subdivides them.

Original: The importance of this behaviour with increasing value of k is discussed in the next section.
Revised: The importance of this behaviour with increasing value of k is discussed in the next section.

Original: B Data Preprocessing
Revised: B Data Preprocessing

Original: Of the 14,836 episodes extracted from the Amsterdam data, 13,783 ended in extubation or death of the patient, 648 ended with a tracheotomy procedure occurring within 21 days, 399 patients were still on ventilation at 21 days, and 6 patients were converted to a non-invasive ventilation setting.
Revised: Of the 14,836 episodes extracted from the Amsterdam data, 13,783 ended in extubation or death of the patient, 648 ended with a tracheotomy procedure occurring within 21 days, 399 patients were still on ventilation at 21 days, and 6 patients were converted to a non-invasive ventilation setting.

Original: Figureshows a summary of the cohort.
Revised: Figureshows a summary of the cohort.

Original: [Table]
Revised: [Table]

Original: B.1 Static Features
Revised: B.1 Static Features

Original: We extracted 14 static features from the admissions table (Table 7).
Revised: We extracted 14 static features from the admissions table (Table 7).

Original: Discrete and continuous variables were scaled to the interval [-1, 1], using the 5th and 95th percentiles as the boundaries, and absolute cut offs were placed at [-4, 4].
Revised: Discrete and continuous variables were scaled to the interval [-1, 1], using the 5th and 95th percentiles as the boundaries, and absolute cut offs were placed at [-4, 4].

Original: Binary variables were coded as 1 and 0.
Revised: Binary variables were coded as 1 and 0.

Original: Categorical variables were converted to one-hot encodings, with the exception of ‘agegroup’, ‘heightgroup’ and ‘weightgroup’.
Revised: Categorical variables were converted to one-hot encodings, with the exception of ‘agegroup’, ‘heightgroup’ and ‘weightgroup’.

Original: These appear as ordered categories e.g. [18-39, 40-49, 50-59, 60-69, 70-79, 80+] for agegroup.
Revised: These appear as ordered categories e.g. [18-39, 40-49, 50-59, 60-69, 70-79, 80+] for agegroup.

Original: We converted these to an ordered set centred on 0, [-1, -0.6, -0.2, 0.2, 0.6, 1], to preserve the quantitative significance of each category.
Revised: We converted these to an ordered set centred on 0, [-1, -0.6, -0.2, 0.2, 0.6, 1], to preserve the quantitative significance of each category.

Original: Count’ and ‘Ventilation Episode Count’ based on previous admissions and ventilation episodes
Revised: Count’ and ‘Ventilation Episode Count’ based on previous admissions and ventilation episodes

Original: [Table]
Revised: [Table]

Original: B.2 Time Series
Revised: B.2 Time Series

Original: For each ventilation episode, we selected 31 time series variables, mostly from the numericitems table (these are shown in Table 12).
Revised: For each ventilation episode, we selected 31 time series variables, mostly from the numericitems table (these are shown in Table 12).

Original: We used a semi-automatic process for feature selection.
Revised: We used a semi-automatic process for feature selection.

Original: To be included, the variable had to be present in at least 25% of patient stays, and these were further narrowed down with advice from Dr Ari Ercole.
Revised: To be included, the variable had to be present in at least 25% of patient stays, and these were further narrowed down with advice from Dr Ari Ercole.

Original: We extracted ‘diagnosissubgroups’ using a query from the
Revised: We extracted ‘diagnosissubgroups’ using a query from the

Original: AmsterdamUMCdb github repository [34] and ventilator settings from listitems .
Revised: AmsterdamUMCdb github repository [34] and ventilator settings from listitems .

Original: The ventilator settings classification is given in Table 13.
Revised: The ventilator settings classification is given in Table 13.

Original: We engineered the features ‘lung compliance’ and ‘P/F ratio’ because they are clinically important, and we have previously noted that neural networks are unreliable when performing divisions.
Revised: We engineered the features ‘lung compliance’ and ‘P/F ratio’ because they are clinically important, and we have previously noted that neural networks are unreliable when performing divisions.

Original: We calculated lung compliance as:
Revised: We calculated lung compliance as:

Original: [Equation]
Revised: [Equation]

Original: where 0.73556 is a conversion factor to convert lung compliance to its usual unit of ml/cmH 2 O. We calculated the P/F ratio as:
Revised: where 0.73556 is a conversion factor to convert lung compliance to its usual unit of ml/cmH 2 O. We calculated the P/F ratio as:

Original: [Equation]
Revised: [Equation]

Original: where FiO 2 is expressed as a fraction rather than a percentage.
Revised: where FiO 2 is expressed as a fraction rather than a percentage.

Original: The time series variables were standardised in the same manner as the static features.
Revised: The time series variables were standardised in the same manner as the static features.

Original: To help the model cope with this missing data, we re-sampled according to one-hour intervals and forward-filled the data over the gaps.
Revised: To help the model cope with this missing data, we re-sampled according to one-hour intervals and forward-filled the data over the gaps.

Original: Note that this is more realistic than interpolation as the clinician would only have the most recent value.
Revised: Note that this is more realistic than interpolation as the clinician would only have the most recent value.

Original: After forward-filling was complete, any data recorded before the ICU admission was removed.
Revised: After forward-filling was complete, any data recorded before the ICU admission was removed.

Original: Decay Indicators With the forward-filling method alone, the model would not know whether a particular data point was genuine or whether the data had been imputed.
Revised: Decay Indicators With the forward-filling method alone, the model would not know whether a particular data point was genuine or whether the data had been imputed.

Original: This is important because the sampling itself may be informative, for example a deteriorating patient may have more frequent investigations.
Revised: This is important because the sampling itself may be informative, for example a deteriorating patient may have more frequent investigations.

Original: To mitigate for this, we added ‘decay indicators’ to specify where the data had been imputed, and if it had, how long it had been since the genuine measurement was taken.
Revised: To mitigate for this, we added ‘decay indicators’ to specify where the data had been imputed, and if it had, how long it had been since the genuine measurement was taken.

Original: The decay was calculated as 0 .
Revised: The decay was calculated as 0 .

Original: 8 j , where j is the time since the last recording.
Revised: 8 j , where j is the time since the last recording.

Original: This is similar in spirit to the masking used by Rocheteau et al.
Revised: This is similar in spirit to the masking used by Rocheteau et al.

Original: [27], Che et al.
Revised: [27], Che et al.

Original: C Additional Implementation Details D Prediction Tasks D.1 Remaining Length of Stay and Ventilation Duration
Revised: C Additional Implementation Details D Prediction Tasks D.1 Remaining Length of Stay and Ventilation Duration

Original: We assigned a remaining length of stay (LoS) and remaining ventilation duration (VD) target to each hour of the ventilation episode, ending when the patient dies or is extubated.
Revised: We assigned a remaining length of stay (LoS) and remaining ventilation duration (VD) target to each hour of the ventilation episode, ending when the patient dies or is extubated.

Original: The remaining LoS is calculated by subtracting the time elapsed in the ICU from the total LoS.
Revised: The remaining LoS is calculated by subtracting the time elapsed in the ICU from the total LoS.

Original: The remaining VD is calculated by subtracting the time elapsed in the ventilation episode from the total VD.
Revised: The remaining VD is calculated by subtracting the time elapsed in the ventilation episode from the total VD.

Original: We only trained on data from the first 21 days of the ventilation episode to protect against batches becoming overly long and slowing down training.
Revised: We only trained on data from the first 21 days of the ventilation episode to protect against batches becoming overly long and slowing down training.

Original: The remaining LoS and VD each have a significant positive skew which makes the duration tasks more challenging.
Revised: The remaining LoS and VD each have a significant positive skew which makes the duration tasks more challenging.

Original: We partly circumvent this by replacing the commonly used mean squared error (MSE) loss with mean squared log error (MSLE), as in Rocheteau et al.
Revised: We partly circumvent this by replacing the commonly used mean squared error (MSE) loss with mean squared log error (MSLE), as in Rocheteau et al.

Original: D.2 Mortality and Tracheostomy
Revised: D.2 Mortality and Tracheostomy

Original: Unlike the duration tasks, these tasks are static, i.e. the labels do not change during the ventilation episode.
Revised: Unlike the duration tasks, these tasks are static, i.e. the labels do not change during the ventilation episode.

Original: Both tasks have significant class imbalance (only 14.6% and 7.4% of patients died or received a tracheostomy respectively).
Revised: Both tasks have significant class imbalance (only 14.6% and 7.4% of patients died or received a tracheostomy respectively).

Original: In order to encourage the model to prioritise learning these important outcomes, we applied class weighting to the task (where the weight is proportional to the inverse of the frequency of each outcome).
Revised: In order to encourage the model to prioritise learning these important outcomes, we applied class weighting to the task (where the weight is proportional to the inverse of the frequency of each outcome).

Original: We used binary crossentropy as the loss function for both tasks.
Revised: We used binary crossentropy as the loss function for both tasks.

Original: D.3 Reconstruction and Forecasting
Revised: D.3 Reconstruction and Forecasting

Original: As shown in Figure 1, we use the embedding to reconstruct the timestep t , and forecast one timestep ( t + 1 ) ahead.
Revised: As shown in Figure 1, we use the embedding to reconstruct the timestep t , and forecast one timestep ( t + 1 ) ahead.

Original: For the reconstruction of t and forecast of t + 1 , we apply the mean squared error since the data can (very approximately) be assumed to be Gaussian centred at 0 following normalisation.
Revised: For the reconstruction of t and forecast of t + 1 , we apply the mean squared error since the data can (very approximately) be assumed to be Gaussian centred at 0 following normalisation.

Original: We also reconstruct the following static features: sex, urgency of admission, agegroup, weightgroup, and heightgroup.
Revised: We also reconstruct the following static features: sex, urgency of admission, agegroup, weightgroup, and heightgroup.

Original: The first two are binary, and so we apply the binary crossentropy loss function.
Revised: The first two are binary, and so we apply the binary crossentropy loss function.

Original: The other three are ordered categorical (as explained in Section B.1), therefore we use the mean squared error loss function.
Revised: The other three are ordered categorical (as explained in Section B.1), therefore we use the mean squared error loss function.

Original: Since these tasks are auxiliary (we are not interested in the performance as an outcome of the model), we reported the their loss function values (see Section D.3) as ‘metrics’ since they do not need to be interpretable.
Revised: Since these tasks are auxiliary (we are not interested in the performance as an outcome of the model), we reported the their loss function values (see Section D.3) as ‘metrics’ since they do not need to be interpretable.

Original: The relative weightings of all of these tasks are given in Section E.
Revised: The relative weightings of all of these tasks are given in Section E.

Original: D.4 Model Architecture
Revised: D.4 Model Architecture

Original: We tested three different encoder models to generate the embeddings.
Revised: We tested three different encoder models to generate the embeddings.

Original: They were all trained as follows.
Revised: They were all trained as follows.

Original: Firstly, the time series are given to the encoder network which processes and then combines them with the static features.
Revised: Firstly, the time series are given to the encoder network which processes and then combines them with the static features.

Original: These are then passed through a small two-layer pointwise convolution to generate the embeddings (shown in green on Figure 1).
Revised: These are then passed through a small two-layer pointwise convolution to generate the embeddings (shown in green on Figure 1).

Original: These are given to a predictor network, a reconstruction network and a forecasting network.
Revised: These are given to a predictor network, a reconstruction network and a forecasting network.

Original: The predictor network is one layer, with four outputs, corresponding to the four outcome tasks – tracheostomy, mortality, LoS and VD.
Revised: The predictor network is one layer, with four outputs, corresponding to the four outcome tasks – tracheostomy, mortality, LoS and VD.

Original: For the binary predictions, we apply a sigmoid activation function to generate a prediction between 0 and 1 and for the duration predictions we apply an exponential function.
Revised: For the binary predictions, we apply a sigmoid activation function to generate a prediction between 0 and 1 and for the duration predictions we apply an exponential function.

Original: This is intended to help to circumvent a common issue seen in previous models (e.g. Harutyunyan et al.
Revised: This is intended to help to circumvent a common issue seen in previous models (e.g. Harutyunyan et al.

Original: [11], as they struggle to produce predictions over the full range of durations when the data is very skewed) because it effectively allows the upstream network to model log( LoS ) instead of LoS.
Revised: [11], as they struggle to produce predictions over the full range of durations when the data is very skewed) because it effectively allows the upstream network to model log( LoS ) instead of LoS.

Original: The log( LoS ) distribution is much closer to a Gaussian distribution than the remaining
Revised: The log( LoS ) distribution is much closer to a Gaussian distribution than the remaining

Original: LoS. No activation function is placed on the outputs of the forecasting or time series reconstruction networks, because the variables are continuous.
Revised: LoS. No activation function is placed on the outputs of the forecasting or time series reconstruction networks, because the variables are continuous.

Original: Batch normalisation [14] and dropout [32] is used to regularise the model.
Revised: Batch normalisation [14] and dropout [32] is used to regularise the model.

Original: The hyperparameter search methodology is described in Section E.
Revised: The hyperparameter search methodology is described in Section E.

Original: The LSTM [13] is very similar to the one used in a recent eICU benchmark paper including LoS prediction [29].
Revised: The LSTM [13] is very similar to the one used in a recent eICU benchmark paper including LoS prediction [29].

Original: The Transformer [38] is very similar to its original implementation except that we added temporal masking to impose causality 5 (see Section E for their hyperparameters).
Revised: The Transformer [38] is very similar to its original implementation except that we added temporal masking to impose causality 5 (see Section E for their hyperparameters).

Original: D.5 TPC Model Architecture
Revised: D.5 TPC Model Architecture

Original: We use a Temporal Pointwise Convolution (TPC) [26] network as one of our encoders.
Revised: We use a Temporal Pointwise Convolution (TPC) [26] network as one of our encoders.

Original: This is a model that takes advantage of both temporal convolution (to analyse trends) and pointwise convolution (to look for any important variable interactions).
Revised: This is a model that takes advantage of both temporal convolution (to analyse trends) and pointwise convolution (to look for any important variable interactions).

Original: It is inspired by the way that clinicians would approach an assessment of a patient e.g. they might check how the respiratory rate is changing over time, and they may also look at combination features e.g. the PaO 2 /FiO 2 ratio.
Revised: It is inspired by the way that clinicians would approach an assessment of a patient e.g. they might check how the respiratory rate is changing over time, and they may also look at combination features e.g. the PaO 2 /FiO 2 ratio.

Original: The components of the network are briefly explained below.
Revised: The components of the network are briefly explained below.

Original: Temporal Convolution Temporal Convolution Networks (TCNs) [37, 15] are models that convolve over the time dimension.
Revised: Temporal Convolution Temporal Convolution Networks (TCNs) [37, 15] are models that convolve over the time dimension.

Original: The TPC model uses stacked TCNs to extract temporal trends in the data.
Revised: The TPC model uses stacked TCNs to extract temporal trends in the data.

Original: Unlike most implementations, it does not share weights across features i.e. weight sharing is only across time.
Revised: Unlike most implementations, it does not share weights across features i.e. weight sharing is only across time.

Original: This is because the features in a typical EHR differ sufficiently in their temporal characteristics and warrant specialised processing.
Revised: This is because the features in a typical EHR differ sufficiently in their temporal characteristics and warrant specialised processing.

Original: In TCNs, the receptive field sizes 6 are highly adaptable.
Revised: In TCNs, the receptive field sizes 6 are highly adaptable.

Original: They can be increased by using greater dilation, larger kernel sizes or by stacking more layers.
Revised: They can be increased by using greater dilation, larger kernel sizes or by stacking more layers.

Original: By contrast, recurrent neural networks such as LSTM can only process one time step at a time, and Transformers have a weaker sense of temporal structure e.g. periodicity, which is central to understanding time series in the EHR.
Revised: By contrast, recurrent neural networks such as LSTM can only process one time step at a time, and Transformers have a weaker sense of temporal structure e.g. periodicity, which is central to understanding time series in the EHR.

Original: TEMPORAL
Revised: TEMPORAL

Original: POINTWISE add padding
Revised: POINTWISE add padding

Original: T
Revised: T

Original: F + Z tY +
Revised: F + Z tY +

Original: T Z*
Revised: T Z*

Original: F + Z tY +
Revised: F + Z tY +

Original: T
Revised: T

Original: F + Z tY +
Revised: F + Z tY +

Original: T + d(k - 1)
Revised: T + d(k - 1)

Original: T
Revised: T

Original: F + Z tY*
Revised: F + Z tY*

Original: T
Revised: T

Original: F + Z t
Revised: F + Z t

Original: Y* repeat Y* times
Revised: Y* repeat Y* times

Original: T
Revised: T

Original: F + Z t
Revised: F + Z t

Original: Y* +
Revised: Y* +

Original: ReLU, add skip connections add flat features (repeated T times) & decay indicators f
Revised: ReLU, add skip connections add flat features (repeated T times) & decay indicators f

Original: T 1 batchnorm, dropout batchnorm,dropout
Revised: T 1 batchnorm, dropout batchnorm,dropout

Original: F
Revised: F

Original: T T
Revised: T T

Original: Pointwise Convolution Pointwise (or 1x1) convolution [18] is typically used to reduce the dimensions in an input [33].
Revised: Pointwise Convolution Pointwise (or 1x1) convolution [18] is typically used to reduce the dimensions in an input [33].

Original: However in the TPC model it is used to compute interaction features from the existing feature set at each timepoint.
Revised: However in the TPC model it is used to compute interaction features from the existing feature set at each timepoint.

Original: Skip Connections Skip connections [12] allow each layer to see the original data and the pointwise outputs from previous layers.
Revised: Skip Connections Skip connections [12] allow each layer to see the original data and the pointwise outputs from previous layers.

Original: This helps the network to cope with infrequently sampled data.
Revised: This helps the network to cope with infrequently sampled data.

Original: Temporal Pointwise Convolution
Revised: Temporal Pointwise Convolution

Original: The full model combines temporal and pointwise convolution in parallel.
Revised: The full model combines temporal and pointwise convolution in parallel.

Original: Figure 11 shows just one layer, however our implementation has 6 layers stacked sequentially (Table 8).
Revised: Figure 11 shows just one layer, however our implementation has 6 layers stacked sequentially (Table 8).

Original: With each successive TPC layer, the temporal dilation is increased by 1.
Revised: With each successive TPC layer, the temporal dilation is increased by 1.

Original: D.6 K-Medoids Clustering
Revised: D.6 K-Medoids Clustering

Original: We used k-medoids clustering to cluster the learned embeddings.
Revised: We used k-medoids clustering to cluster the learned embeddings.

Original: K-medoids is similar to k-means, except that it operates with medoids rather than centroids.
Revised: K-medoids is similar to k-means, except that it operates with medoids rather than centroids.

Original: This means that the medoids will always be a true observation in the data, while that is not usually the case for centroids.
Revised: This means that the medoids will always be a true observation in the data, while that is not usually the case for centroids.

Original: The main advantage is that k-medoids are less sensitive to outliers than k-means, which is more suitable in this context where the data is noisy and heavily skewed 7 .
Revised: The main advantage is that k-medoids are less sensitive to outliers than k-means, which is more suitable in this context where the data is noisy and heavily skewed 7 .

Original: Both k-means and k-medoids operate on pairwise similarities.
Revised: Both k-means and k-medoids operate on pairwise similarities.

Original: We decided to use Euclidean distance rather than cosine similarity.
Revised: We decided to use Euclidean distance rather than cosine similarity.

Original: This is because intuitively, it is not only the direction that the patient is moving in that matters, but also the distance along that axis.
Revised: This is because intuitively, it is not only the direction that the patient is moving in that matters, but also the distance along that axis.

Original: For example, if a particular ‘direction’ represents acute decompensated heart failure, we also care how severe the decompensation is.
Revised: For example, if a particular ‘direction’ represents acute decompensated heart failure, we also care how severe the decompensation is.

Original: As detailed in Section D. we applied batch normalisation [14] to the embeddings, to ensure that the embedding distribution remained within a reasonable range.
Revised: As detailed in Section D. we applied batch normalisation [14] to the embeddings, to ensure that the embedding distribution remained within a reasonable range.

Original: The value of k (5 for all models) was chosen using the elbow method (see Section E.1 in the Appendix).
Revised: The value of k (5 for all models) was chosen using the elbow method (see Section E.1 in the Appendix).

Original: E Hyperparameter Search Methodology
Revised: E Hyperparameter Search Methodology

Original: All the encoders have hyperparameters that can broadly be split into three categories: time series specific, non-time series specific and global parameters (shown in more detail in Tables 8, 9 and 10).
Revised: All the encoders have hyperparameters that can broadly be split into three categories: time series specific, non-time series specific and global parameters (shown in more detail in Tables 8, 9 and 10).

Original: The hyperparameter search ranges have been included in Table 11.
Revised: The hyperparameter search ranges have been included in Table 11.

Original: We ran 10 hyperparameter trials to optimise the remaining parameters for the TPC, LSTM, and Transformer models.
Revised: We ran 10 hyperparameter trials to optimise the remaining parameters for the TPC, LSTM, and Transformer models.

Original: The number of epochs was determined by selecting the best validation performance from a model trained overepochs (early stopping was then used for each individual model).
Revised: The number of epochs was determined by selecting the best validation performance from a model trained overepochs (early stopping was then used for each individual model).

Original: All deep learning methods were implemented in PyTorch [22] using PyTorch Lightning [8] and were optimised using Adam [16].
Revised: All deep learning methods were implemented in PyTorch [22] using PyTorch Lightning [8] and were optimised using Adam [16].

Original: We also optimised for the weighting between the tasks.
Revised: We also optimised for the weighting between the tasks.

Original: We simply multiplied the loss for each component by a hyperparameter.
Revised: We simply multiplied the loss for each component by a hyperparameter.

Original: The best overall learning curves were found when the task weighting coefficients were: 0.5 for the duration tasks, 1 for the binary tasks, 0.1 for time series reconstruction and forecasting, and 0.002 for binary feature reconstruction.
Revised: The best overall learning curves were found when the task weighting coefficients were: 0.5 for the duration tasks, 1 for the binary tasks, 0.1 for time series reconstruction and forecasting, and 0.002 for binary feature reconstruction.

Original: The reason for the small weighting for binary feature reconstruction was that the task appeared very easy for the models, especially predicting the sex of the patient, and so the representation became dominated with this at the expense of the other tasks.
Revised: The reason for the small weighting for binary feature reconstruction was that the task appeared very easy for the models, especially predicting the sex of the patient, and so the representation became dominated with this at the expense of the other tasks.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: E.1 Number of Clusters
Revised: E.1 Number of Clusters

Original: The value of k was determined using an average value from the elbow method across various encoders.
Revised: The value of k was determined using an average value from the elbow method across various encoders.

Original: Specifically we looked for the point at which the Within Cluster Sum of Squares (WCSS) started to
Revised: Specifically we looked for the point at which the Within Cluster Sum of Squares (WCSS) started to

Original: [Table]
Revised: [Table]

Original: tail off with increasing values of k. Figure 13 shows an example elbow plot.
Revised: tail off with increasing values of k. Figure 13 shows an example elbow plot.

Original: We selected the valueacross all the models.
Revised: We selected the valueacross all the models.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]