Original: P HYSIOLOGICAL S IGNAL E MBEDDINGS (PHASE) VIA I NTERPRETABLE S TACKED M ODELS
Revised: P HYSIOLOGICAL S IGNAL E MBEDDINGS (PHASE) VIA I NTERPRETABLE S TACKED M ODELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: In health, machine learning is increasingly common, yet neural network embedding (representation) learning is arguably under-utilized for physiological signals.
Revised: In health, machine learning is increasingly common, yet neural network embedding (representation) learning is arguably under-utilized for physiological signals.

Original: This inadequacy stands out in stark contrast to more traditional computer science domains, such as computer vision (CV), and natural language processing (NLP).
Revised: This inadequacy stands out in stark contrast to more traditional computer science domains, such as computer vision (CV), and natural language processing (NLP).

Original: For physiological signals, learning feature embeddings is a natural solution to data insufﬁciency caused by patient privacy concerns – rather than share data, researchers may share informative embedding models (i.e., representation models), which map patient data to an output embedding.
Revised: For physiological signals, learning feature embeddings is a natural solution to data insufﬁciency caused by patient privacy concerns – rather than share data, researchers may share informative embedding models (i.e., representation models), which map patient data to an output embedding.

Original: Here, we present the PHASE (PHysiologicAl Signal Embeddings) framework, which consists of three components: i) learning neural network embeddings of physiological signals, ii) predicting outcomes based on the learned embedding, and iii) interpreting the prediction results by estimating feature attributions in the “stacked” models (i.e., feature embedding model followed by prediction model).
Revised: Here, we present the PHASE (PHysiologicAl Signal Embeddings) framework, which consists of three components: i) learning neural network embeddings of physiological signals, ii) predicting outcomes based on the learned embedding, and iii) interpreting the prediction results by estimating feature attributions in the “stacked” models (i.e., feature embedding model followed by prediction model).

Original: PHASE is novel in three ways: 1)
Revised: PHASE is novel in three ways: 1)

Original: To our knowledge, PHASE is the ﬁrst instance of transferal of neural networks to create physiological signal embeddings.
Revised: To our knowledge, PHASE is the ﬁrst instance of transferal of neural networks to create physiological signal embeddings.

Original: We present a tractable method to obtain feature attributions through stacked models.
Revised: We present a tractable method to obtain feature attributions through stacked models.

Original: We prove that our stacked model attributions can approximate Shapley values – attributions known to have desirable properties – for arbitrary sets of models.
Revised: We prove that our stacked model attributions can approximate Shapley values – attributions known to have desirable properties – for arbitrary sets of models.

Original: 3) PHASE was extensively tested in a cross-hospital setting including publicly available data.
Revised: 3) PHASE was extensively tested in a cross-hospital setting including publicly available data.

Original: In our experiments, we show that PHASE signiﬁcantly outperforms alternative embeddings – such as raw, exponential moving average/variance, and autoencoder – currently in use.
Revised: In our experiments, we show that PHASE signiﬁcantly outperforms alternative embeddings – such as raw, exponential moving average/variance, and autoencoder – currently in use.

Original: Furthermore, we provide evidence that transferring neural network embedding/representation learners between distinct hospitals can yield performant embeddings and offer recommendations when transference is ineffective.
Revised: Furthermore, we provide evidence that transferring neural network embedding/representation learners between distinct hospitals can yield performant embeddings and offer recommendations when transference is ineffective.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Learning embeddings (i.e., representation learning) (Bengio et al., 2013) has been applied to medical images and clinical text (Tajbakhsh et al., 2016; Ravishankar et al., 2016;
Revised: Representation learning (i.e., learning embeddings) (Bengio et al., 2013) has been applied to medical images and clinical text (Tajbakhsh et al., 2016; Ravishankar et al., 2016;

Original: Lv et al., 2014)
Revised: Lv et al., 2014)

Original: but has been under-explored for time series physiological signals in electronic health records.
Revised: but has been under-explored for time series physiological signals in electronic health records.

Original: This paper introduces the PHASE (PHysiologicAl Signal Embeddings) framework to learn embeddings of physiological signals (Figure 1a), which can be used for various prediction tasks (Figure 1b), helps interpretation by computing feature attributions of the original features (i.e., not embeddings) for a prediction result in a tricky “stacked” model situation (i.e., embedding model followed by prediction model) (Figure 1c), and has been extensively tested in terms of its transferability using data from multiple hospitals (Figure 1d).
Revised: This paper introduces the PHASE (PHysiologicAl Signal Embeddings) framework to learn embeddings of physiological signals (Figure 1a), which can be used for various prediction tasks (Figure 1b), and has been extensively tested in terms of its transferability using data from multiple hospitals (Figure 1d). In addition, this paper introduces an interpretability method to compute per-sample feature attributions of the original features (i.e., not embeddings) for a prediction result in a tricky “stacked” model situation (i.e., embedding model followed by prediction model) (Figure 1c).

Original: Based on computer vision (CV) and natural language processing (NLP), exemplars of representation learning, physiological signals are well suited to embeddings.
Revised: Based on computer vision (CV) and natural language processing (NLP), exemplars of representation learning, physiological signals are well suited to embeddings.

Original: In particular, CV and NLP share two notable traits with physiological signals.
Revised: In particular, CV and NLP share two notable traits with physiological signals.

Original: The ﬁrst is consistency .
Revised: The ﬁrst is consistency .

Original: For CV, the domain has consistent features: edges, colors, and other visual attributes.
Revised: For CV, the domain has consistent features: edges, colors, and other visual attributes.

Original: For NLP, the domain is a particular language with semantic relationships consistent across bodies of text.
Revised: For NLP, the domain is a particular language with semantic relationships consistent across bodies of text.

Original: For sequential signals, physiological patterns are arguably consistent across individuals.
Revised: For sequential signals, physiological patterns are arguably consistent across individuals.

Original: The second attribute is complexity .
Revised: The second attribute is complexity .

Original: Across these three domains, each particular domain is sufﬁciently complex such that learning embeddings is non-trivial.
Revised: Across these three domains, each particular domain is sufﬁciently complex such that learning embeddings is non-trivial.

Original: Together, consistency and complexity suggest that for a particular domain, every research group independently spends a signiﬁcant time to learn embeddings that may ultimately be
Revised: Together, consistency and complexity suggest that for a particular domain, every research group independently spends a signiﬁcant time to learn embeddings that may ultimately be

Original: LSTM LSTM
Revised: LSTM LSTM

Original: LSTM LSTM
Revised: LSTM LSTM

Original: XGB …
Revised: XGB …

Original: Interpretation …
Revised: Interpretation …

Original: XGB
Revised: XGB

Original: Source Hospital
Revised: Source Hospital

Original: Target Hospital min SAO2 ()*,⋯,()- … LSTM
Revised: Target Hospital min SAO2 ()*,⋯,()- … LSTM

Original: LSTM
Revised: LSTM

Original: SAO2 (.-/,⋯,(
Revised: SAO2 (.-/,⋯,(

Original: TV (.-/,⋯,( min TV ()*,⋯,()-
Revised: TV (.-/,⋯,( min TV ()*,⋯,()-

Original: SAO2 (.-/,⋯,( … LSTM
Revised: SAO2 (.-/,⋯,( … LSTM

Original: LSTM …
Revised: LSTM …

Original: LSTM
Revised: LSTM

Original: XGB
Revised: XGB

Original: Transference
Revised: Transference

Original: IndependentTreeSHAP
Revised: IndependentTreeSHAP

Original: DeepSHAP
Revised: DeepSHAP

Original: Representationmodel transfer
Revised: Representationmodel transfer

Original: Is min SAO2 ()*,⋯,()- ≤ 92?
Revised: Is min SAO2 ()*,⋯,()- ≤ 92?

Original: Learning Representations
Revised: Learning Representations

Original: Prediction
Revised: Prediction

Original: TV (.-/,⋯,(
Revised: TV (.-/,⋯,(

Original: [Figure]
Revised: [Figure]

Original: [Figure] 
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: quite similar.
Revised: quite similar.

Original: In order to avoid this negative externality, NLP and CV have made great progress on standardizing their embeddings; in health, physiological signals are a natural next step.
Revised: In order to avoid this negative externality, NLP and CV have made great progress on standardizing their embeddings; in health, physiological signals are a natural next step.

Original: Furthermore, physiological signals have unique properties that make them arguably better suited to representation learning than traditional CV and NLP applications.
Revised: Furthermore, physiological signals have unique properties that make them arguably better suited to representation learning than traditional CV and NLP applications.

Original: First, physiological signals are typically generated in the health domain, which is constrained by patient privacy concerns .
Revised: First, physiological signals are typically generated in the health domain, which is constrained by patient privacy concerns .

Original: These concerns make sharing data between hospitals next to impossible; however, sharing models between hospitals is intuitively safer and generally accepted.
Revised: These concerns make sharing data between hospitals next to impossible; however, sharing models between hospitals is intuitively safer and generally accepted.

Original: Second, a key component to successful transfer learning is a community of researchers that work on related problems.
Revised: Second, a key component to successful transfer learning is a community of researchers that work on related problems.

Original: According to Faust et al.
Revised: According to Faust et al.

Original: (2018), there were at least ﬁfty-three research publications using deep learning methods for physiological signals in the past ten years.
Revised: (2018), there were at least ﬁfty-three research publications using deep learning methods for physiological signals in the past ten years.

Original: Additionally, we discuss particular examples of neural networks for physiological signals in Section 2.2.
Revised: Additionally, we discuss particular examples of neural networks for physiological signals in Section 2.2.

Original: These varied applications of neural networks imply that there is a large community of machine learning research scientists working on physiological signals, a community that could one day work collaboratively to help patients by sharing models.
Revised: These varied applications of neural networks imply that there is a large community of machine learning research scientists working on physiological signals, a community that could one day work collaboratively to help patients by sharing models.

Original: Although embedding learning has many aforementioned advantages, it makes interpretation more difﬁcult.
Revised: Although embedding learning has many aforementioned advantages, it makes interpretation more difﬁcult.

Original: Existing interpretation methods (Shrikumar et al., 2016; Sundararajan et al., 2017; Lundberg & Lee, 2017; Lundberg et al., 2018) do not work for models trained using learned embeddings, because they will assign attributions to the embeddings.
Revised: Naive applications of existing interpretation methods (Shrikumar et al., 2016; Sundararajan et al., 2017; Lundberg & Lee, 2017; Lundberg et al., 2018) do not work for models trained using learned embeddings, because they will assign attributions to the embeddings.

Original: Feature attributions assigned to embeddings will be meaningless, because the embeddings do not map to any particular input feature.
Revised: Feature attributions assigned to embeddings will be meaningless, because the embeddings do not map to any particular input feature.

Original: Instead, each embedding is a complicated, potentially non-linear combination of the original raw physiological signals.
Revised: Instead, each embedding is a complicated, potentially non-linear combination of the original raw physiological signals.

Original: In a health domain, inability to meaningfully interpret your model is unsatisfactory.
Revised: In a health domain, inability to meaningfully interpret your model is unsatisfactory.

Original: Healthcare providers and patients alike generally want to know the reasoning behind predictions/diagnoses.
Revised: Healthcare providers and patients alike generally want to know the reasoning behind predictions/diagnoses.

Original: Interpretability can enhance both scientiﬁc discovery as well as provide credibility to predictive models.
Revised: Interpretability can enhance both scientiﬁc discovery as well as provide credibility to predictive models.

Original: In order to provide a principled methodology for mapping embedding attributions back into physiological signal attributions, we provide a proof that justiﬁes PHASE’s
Revised: In order to provide a principled methodology for mapping embedding attributions back into physiological signal attributions, we provide a proof that justiﬁes PHASE’s

Original: Shapley value framework in Section 3.3.
Revised: Shapley value framework in Section 3.3.

Original: This framework generalizes across arbitrary stacked models and currently encompasses neural network models (e.g., linear models, neural networks) and tree-based models (e.g., gradient boosting machines and random forests).
Revised: This framework generalizes across arbitrary stacked models and currently encompasses neural network models (e.g., linear models, neural networks) and tree-based models (e.g., gradient boosting machines and random forests).

Original: In the following sections, we discuss previous related work (Section 2) and describe the PHASE framework (Section 3).
Revised: In the following sections, we discuss previous related work (Section 2) and describe the PHASE framework (Section 3).

Original: In Section 4, we ﬁrst evaluate how well our neural network embeddings make accurate predictions (Section 4.2.1).
Revised: In Section 4, we ﬁrst evaluate how well our neural network embeddings make accurate predictions (Section 4.2.1).

Original: Second, we evaluate whether transferring these embedding learners still enables accurate predictions across three different hospitals separated by location and across hospital departments (Section 4.2.2).
Revised: Second, we evaluate whether transferring these embedding learners still enables accurate predictions across three different hospitals separated by location and across hospital departments (Section 4.2.2).

Original: Lastly, we present a visualization of our methodology for providing Shapley value feature attributions through stacked models in Section 4.2.3.
Revised: Lastly, we present a visualization of our methodology for providing Shapley value feature attributions through stacked models in Section 4.2.3.

Original:  R ELATED W ORK 2. R EPRESENTATION LEARNING IN THE HEALTH DOMAIN
Revised:  R ELATED W ORK 2. R EPRESENTATION LEARNING IN THE HEALTH DOMAIN

Original: Representation learning (embedding learning) in health is growing more popular.
Revised: Representation learning (embedding learning) in health is growing more popular.

Original: One particularly natural subdomain is medical image analysis, e.g., mammography analysis, kidney detection in ultrasound images, optical coherence tomography image analysis, diagnosing pneumonia using chest
Revised: One particularly natural subdomain is medical image analysis, e.g., mammography analysis, kidney detection in ultrasound images, optical coherence tomography image analysis, diagnosing pneumonia using chest

Original: X-ray images, lung pattern analysis, otitis media image analysis, and more (Arevalo et al., 2016;
Revised: X-ray images, lung pattern analysis, otitis media image analysis, and more (Arevalo et al., 2016;

Original: Ravishankar et al., 2016; Kermany et al., 2018; Liao et al., 2013; Christodoulidis et al., 2016; Shie et al., 2015).
Revised: Ravishankar et al., 2016; Kermany et al., 2018; Liao et al., 2013; Christodoulidis et al., 2016; Shie et al., 2015).

Original: Outside of image analysis, additional examples of transfer learning in the medical domain include Lv et al.
Revised: Outside of image analysis, additional examples of transfer learning in the medical domain include Lv et al.

Original: (2014), Wiens et al.
Revised: (2014), Wiens et al.

Original: Brisimi et al.
Revised: Brisimi et al.

Original: (2018), Choi et al.
Revised: (2018), Choi et al.

Original: (2017), Choi et al.
Revised: (2017), Choi et al.

Original: (2016), and Che et al.
Revised: (2016), and Che et al.

Original: Even within physiological signals, some examples of embedding learning are beginning to sprout up, including Wu et al.
Revised: Even within physiological signals, some examples of embedding learning are beginning to sprout up, including Wu et al.

Original: (2013), who utilize kNNs to perform transfer learning for brain-computer interaction.
Revised: (2013), who utilize kNNs to perform transfer learning for brain-computer interaction.

Original: Comparatively, PHASE transfers neural networks as embedding functions learned in an partially supervised manner, where the embeddings provide a basis for training a model on any prediction task (as opposed to being tied to the prediction they were trained on).
Revised: Comparatively, PHASE transfers neural networks as embedding functions learned in an partially supervised manner, where the embeddings provide a basis for training a model on any prediction task (as opposed to being tied to the prediction they were trained on).

Original: Here, we denote partially supervised networks to be networks trained with prediction tasks related to the ﬁnal downstream prediction.
Revised: We denote partially supervised networks to be networks trained with prediction tasks related to the ﬁnal downstream prediction.

Original: 2. N EURAL NETWORKS FOR PHYSIOLOGICAL SIGNALS
Revised: 2. N EURAL NETWORKS FOR PHYSIOLOGICAL SIGNALS

Original: To our knowledge, our work is the ﬁrst to transfer deep neural networks for embedding sequential physiological signals, embeddings which are not tied to a particular prediction problem.
Revised: To our knowledge, our work is the ﬁrst to transfer deep neural networks for embedding sequential physiological signals, embeddings which are not tied to a particular prediction problem.

Original: One caveat is that supervised deep learning can be said to inherently learn embeddings.
Revised: One caveat is that supervised deep learning can be said to inherently learn embeddings.

Original: In physiological signals, there are several examples of particular supervised learning tasks with neural networks.
Revised: In physiological signals, there are several examples of particular supervised learning tasks with neural networks.

Original: Srinivasan et al.
Revised: Srinivasan et al.

Original: and Guo et al.
Revised: and Guo et al.

Original: both detect epilepsy from signals, Wilson & Russell (2003) utilize psycho-physiological measurements to assess mental workload, Wagner et al.
Revised: both detect epilepsy from signals, Wilson & Russell (2003) utilize psycho-physiological measurements to assess mental workload, Wagner et al.

Original: and Chanel et al.
Revised: and Chanel et al.

Original: (2006) utilize physiological signals to classify emotions, Koike & Kawato (1995) reconstruct human arm movement from EMG signals, Sullivan et al.
Revised: (2006) utilize physiological signals to classify emotions, Koike & Kawato (1995) reconstruct human arm movement from EMG signals, Sullivan et al.

Original: (2010) reconstruct missing physiological signals, and Yang & Hsieh (2016) use acoustic signals to detect anomalies in heart sound.
Revised: (2010) reconstruct missing physiological signals, and Yang & Hsieh (2016) use acoustic signals to detect anomalies in heart sound.

Original: Based on this substantive community of research scientists working on physiological signals, there is a clear opportunity to unify independent research by appropriately using partially supervised feature embedding learning.
Revised: Based on this substantive community of research scientists working on physiological signals, there is a clear opportunity to unify independent research by appropriately using partially supervised feature embedding learning.

Original: In the vein of embedding learning, Martinez et al.
Revised: In the vein of embedding learning, Martinez et al.

Original: (2013) applied autoencoders to blood volume pulse and skin conductance measured from 36 people playing a video game and used the encodings to predict affective state.
Revised: (2013) applied autoencoders to blood volume pulse and skin conductance measured from 36 people playing a video game and used the encodings to predict affective state.

Original: In their paper, the sample size is fairly small and reﬂects that their primary objective was to perform feature extraction and feature selection.
Revised: In their paper, the sample size is fairly small and reﬂects that their primary objective was to perform feature extraction and feature selection.

Original: In contrast, PHASE evaluates transferring embedding learners (i.e., feature extractors) across multiple hospitals (Table 1).
Revised: In contrast, PHASE evaluates transferring embedding learners (i.e., feature extractors) across multiple hospitals (Table 1).

Original: 2. F ORECASTING FOR OPERATING ROOM DATA
Revised: 2. F ORECASTING FOR OPERATING ROOM DATA

Original: Lundberg et al.
Revised: Lundberg et al.

Original: (2017) proposed an approach, namely Prescience, which achieved state-of-the-art hypoxemia predictions using operating room data, the same data we used to evaluate PHASE.
Revised: (2017) proposed an approach, namely Prescience, which achieved state-of-the-art hypoxemia predictions using operating room data, the same data we used to evaluate PHASE.

Original: Prescience utilizes gradient boosting machines (GBM) applied to features extracted by using traditional time series feature extraction methods – exponential moving average/variance embeddings.
Revised: Prescience utilizes gradient boosting machines (GBM) applied to features extracted by using traditional time series feature extraction methods – exponential moving average/variance embeddings.

Original: Prescience compares prediction models including gradient boosting machines, linear lasso, linear SVM, and a parzen window with the objective of forecasting low blood oxygen in the future.
Revised: Prescience compares prediction models including gradient boosting machines, linear lasso, linear SVM, and a parzen window with the objective of forecasting low blood oxygen in the future.

Original: Ultimately,
Revised: Ultimately,

Original: Lundberg et al.
Revised: Lundberg et al.

Original: ﬁnd the highest performing method to be GBM trees.
Revised: ﬁnd the highest performing method to be GBM trees.

Original: With samples drawn from the same data set, PHASE seeks to substitute the feature extraction used in Prescience with a deep learning approach, which resulted in a better average precision compared to Prescience without the clinical text features (4 is Prescience and 12 is PHASE in Figure 2).
Revised: With samples drawn from the same data set, PHASE seeks to substitute the feature extraction used in Prescience with a deep learning approach, which resulted in a better average precision compared to Prescience without the clinical text features (4 is Prescience and 12 is PHASE in Figure 2).

Original: 2. F EATURE ATTRIBUTIONS FOR INTERPRETABILITY
Revised: 2. F EATURE ATTRIBUTIONS FOR INTERPRETABILITY

Original: Interpretability of models  has been addressed in numerous independent pieces of work (Shrikumar et al., 2016; Sundararajan et al., 2017; Lundberg & Lee, 2017; Lundberg et al., 2018).
Revised: Interpretability of models via local feature attributions has been addressed in numerous independent pieces of work (Shrikumar et al., 2016; Sundararajan et al., 2017; Lundberg & Lee, 2017; Lundberg et al., 2018).

Original: For our evaluation of interpretability, we choose to focus on Shapley values introduced by Lloyd Shapley, originally in the context of game theory (Shapley, 1953).
Revised: For our evaluation of interpretability, we choose to focus on Shapley values introduced by Lloyd Shapley, originally in the context of game theory (Shapley, 1953).

Original: Lundberg & Lee (2017) identify Shapley values as the only additive feature attribution method that satisﬁes the properties of local accuracy, missingness, and consistency.
Revised: Lundberg & Lee (2017) identify Shapley values as the only additive feature attribution method that satisﬁes the properties of local accuracy, missingness, and consistency.

Original: For PHASE, our pipeline includes multiple models – GBM trees anda LSTM networks.
Revised: For PHASE, our pipeline includes multiple models – GBM trees and LSTM networks.

Original: Methods exist for obtaining Shapley values for GBM trees (TreeSHAP) and for neural networks (Deep LIFT/Deep SHAP) (Lundberg et al., 2018; Shrikumar et al., 2016; Lundberg, 2018).
Revised: Methods exist for obtaining Shapley values for GBM trees (Tree SHAP) and for neural networks (Deep LIFT/Deep SHAP) (Lundberg et al., 2018; Shrikumar et al., 2016; Lundberg, 2018).

Original: However, the default version of these methodologies do not beget a theoretically justiﬁed approach for propagating attributions through multiple models.
Revised: However, the default version of these methodologies do not beget a theoretically justiﬁed approach for propagating attributions through multiple models.

Original: In fact, to the authors’ knowledge, a tractable method for obtaining Shapley value attributions for stacked models does not exist.
Revised: In fact, to the authors’ knowledge, a tractable method for obtaining local feature attributions for mixes of neural networks and trees does not exist.

Original: In this paper, we utilize versions of TreeSHAP and DeepLIFT that create single reference attributions that can be composed to address stacked models.
Revised: In this paper, we utilize versions of Tree SHAP and Deep LIFT that create single reference attributions that can be composed to address stacked models.

Original: At the end of obtaining many attributions, the average of these attributions approximates the Shapley value attributions (more details in Section 3.3).
Revised: At the end of obtaining many attributions, the average of these attributions approximates the Shapley value attributions (more details in Section 3.3).

Original:  O UR A PPROACH : PH YSIOLOGIC A L S IGNAL E MBEDDINGS (PHASE)
Revised:  O UR A PPROACH : PH YSIOLOGIC A L S IGNAL E MBEDDINGS (PHASE)

Original: Taking inspiration from sinusoidal waveforms, we name our methodology PHASE.
Revised: Taking inspiration from sinusoidal waveforms, we name our methodology PHASE.

Original: In the PHASE framework, the ﬁrst step is to learn neural network embeddings for physiological signals (Figure 1a).
Revised: In the PHASE framework, the ﬁrst step is to learn neural network embeddings for physiological signals (Figure 1a).

Original: The second step is to predict outcomes based on the learned feature embedding (as in Figure 1b), potentially across multiple hospitals (as in Figure 1d).
Revised: The second step is to predict outcomes based on the learned feature embedding (as in Figure 1b), potentially across multiple hospitals (as in Figure 1d).

Original: Finally, the last step is to interpret the prediction results by estimating feature attributions through the models trained in the ﬁrst two steps (as in Figure 1c).
Revised: Finally, the last step is to interpret the prediction results by estimating feature attributions through the models trained in the ﬁrst two steps (as in Figure 1c).

Original: 3. L EARNING E MBEDDINGS - L ONG S HORT T ERM M EMORY (LSTM) N ETWORKS
Revised: 3. L EARNING E MBEDDINGS - L ONG S HORT T ERM M EMORY (LSTM) N ETWORKS

Original: PHASE uses LSTM networks to learn feature embeddings from time series physiological data.
Revised: PHASE uses LSTM networks to learn feature embeddings from time series physiological data.

Original: LSTMs are a popular variant on recurrent neural networks introduced by Hochreiter & Schmidhuber (1997).
Revised: LSTMs are a popular variant on recurrent neural networks introduced by Hochreiter & Schmidhuber (1997).

Original: They have the capacity to model long term dependencies, while avoiding the vanishing gradient problem (Pascanu et al., 2012).
Revised: They have the capacity to model long term dependencies, while avoiding the vanishing gradient problem (Pascanu et al., 2012).

Original: Details on the model architecture may be found in Section 6.2.
Revised: Details on the model architecture may be found in Section 6.2.

Original: For PHASE, we ﬁrst train a univariate LSTM on each physiological signal P , predicting the minimum of P in the future ﬁve minutes (Figure 1a).
Revised: For PHASE, we ﬁrst train a univariate LSTM on each physiological signal P , predicting the minimum of P in the future ﬁve minutes (Figure 1a).

Original: Note that we choose the minimum of the next ﬁve minutes because we care about forecasting adverse outcomes.
Revised: Note that we choose the minimum of the next ﬁve minutes because we care about forecasting adverse outcomes.

Original: Then, we obtain hidden embeddings of the original physiological signals by passing them through to the hidden layer (the red layer in
Revised: Then, we obtain hidden embeddings of the original physiological signals by passing them through to the hidden layer (the red layer in

Original: Figure 1a).
Revised: Figure 1a).

Original: These embeddings are unsupervised in the sense that training them simply requires the same feature (albeit at different time steps).
Revised: These embeddings are unsupervised in the sense that training them simply requires the same feature (albeit at different time steps).

Original: Yet, they are supervised in that we specify our interest to be forecasting adverse outcomes constituted by low signal values.
Revised: Yet, they are supervised in that we specify our interest to be forecasting adverse outcomes constituted by low signal values.

Original: We choose to focus on the minimum, because adverse outcomes in physiological signals are often tied to too-low signals.
Revised: We choose to focus on the minimum, because adverse outcomes in physiological signals are often tied to too-low signals.

Original: We ﬁnd that the completely unsupervised alternative of an LSTM autoencoder is signiﬁcantly less performant than the LSTM trained to predict the minimum of the next ﬁve minutes (8 is autoencoder and 12 is PHASE in Figure 2).
Revised: We ﬁnd that the completely unsupervised alternative of an LSTM autoencoder is signiﬁcantly less performant than the LSTM trained to predict the minimum of the next ﬁve minutes (8 is autoencoder and 12 is PHASE in Figure 2).

Original: One reason behind having univariate neural networks is for transference.
Revised: One reason behind having univariate neural networks is for transference.

Original: By using univariate networks, the input to the ﬁnal prediction model may be any set of physiological signals with existing embedding learners.
Revised: By using univariate networks, the input to the ﬁnal prediction model may be any set of physiological signals with existing embedding learners.

Original: This is especially useful because hospital departments have substantial variation in the signals they may choose to collect for features.
Revised: This is especially useful because hospital departments have substantial variation in the signals they may choose to collect for features.

Original: Another reason for univariate networks is that data in a single hospital is often collected at different points in time, or new measurement devices may be introduced to data collection systems.
Revised: Another reason for univariate networks is that data in a single hospital is often collected at different points in time, or new measurement devices may be introduced to data collection systems.

Original: For traditional pipelines, it may be necessary to re-train entire machine learning pipelines when new features are introduced.
Revised: For traditional pipelines, it may be necessary to re-train entire machine learning pipelines when new features are introduced.

Original: With univariate networks, the ﬂexibility would mean pre-existing embedding learners would not necessarily need to be re-trained.
Revised: With univariate networks, the ﬂexibility would mean pre-existing embedding learners would not necessarily need to be re-trained.

Original: 3. P REDICTION - G RADIENT B OOSTING M ACHINES (GBM)
Revised: 3. P REDICTION - G RADIENT B OOSTING M ACHINES (GBM)

Original: PHASE can use any prediction model.
Revised: PHASE can use any prediction model.

Original: In this paper, we focus on gradient boosting machine trees because the Prescience method found that they outperform several other models in the operating room data (Lundberg et al., 2017).
Revised: In this paper, we focus on gradient boosting machine trees because the Prescience method found that they outperform several other models in the operating room data (Lundberg et al., 2017).

Original: Gradient boosting machines were introduced by Friedman (2001).
Revised: Gradient boosting machines were introduced by Friedman (2001).

Original: This technique creates an ensemble of weak prediction models in order to perform classiﬁcation/regression tasks in an iterative fashion.
Revised: This technique creates an ensemble of weak prediction models in order to perform classiﬁcation/regression tasks in an iterative fashion.

Original: In particular, we utilize XGBoost, a popular implementation of gradient boosting machines that uses additive regression trees (Chen & Guestrin,2016).
Revised: In particular, we utilize XGBoost, a popular implementation of gradient boosting machines that uses additive regression trees (Chen & Guestrin,

Original: XGBoost often dominates in Kaggle, a platform for predictive modeling competitions.
Revised: XGBoost often dominates in Kaggle, a platform for predictive modeling competitions.

Original: In particular, seventeen out of twenty nine challenge winning solutions used XGBoost in 2015 (Chen &
Revised: In particular, seventeen out of twenty nine challenge winning solutions used XGBoost in 2015 (Chen &

Original: Guestrin, 2016).
Revised: Guestrin, 2016).

Original: For PHASE, we postulate that utilizing embeddings of time series signals provides stronger features for the ultimate prediction with XGB (as visualized in Figure 1b).
Revised: For PHASE, we postulate that utilizing embeddings of time series signals provides stronger features for the ultimate prediction with XGB (as visualized in Figure 1b).

Original: Details on the model architecture may be found in Section 6.2.
Revised: Details on the model architecture may be found in Section 6.2.

Original: 3. I NTERPRETATION - S HAPLEY V ALUES THROUGH S TACKED M ODELS
Revised: 3. I NTERPRETATION - S HAPLEY V ALUES THROUGH S TACKED M ODELS

Original: PHASE addresses an inherent challenge in the interpretation of an embedding model (or feature representation model).
Revised: PHASE addresses an inherent challenge in the interpretation of an embedding model (or feature representation model).

Original: Estimating feature attributions is a common way to make a prediction result interpretable.
Revised: Estimating feature attributions is a common way to make a prediction result interpretable.

Original: At a high level, the goal is to explain how much each feature matters for aparticular model’s prediction.
Revised: At a high level, the goal is to explain how much each feature matters for a model’s prediction.

Original: However, this goal is only meaningful if the model being explained uses features with a natural human interpretation.
Revised: However, this goal is only meaningful if the model being explained uses features with a natural human interpretation.

Original: For example, if we interpret PHASE’s GBM model, which takes the embeddings as input and outputs a prediction, our feature attributions will be assigned to the embeddings, which are not meaningful to doctors or patients.
Revised: For example, if we interpret PHASE’s GBM model, which takes the embeddings as input and outputs a prediction, our feature attributions will be assigned to the embeddings, which are not meaningful to doctors or patients.

Original: The answer is to extend the prediction model (here, a GBM) by combining the feature embedding model (here, a LSTM network), which makes a “stacked” model (Figure 1c).
Revised: The answer is to extend the prediction model (here, a GBM) by combining the feature embedding model (here, a LSTM network), which makes a “stacked” model (Figure 1c).

Original: Since the original features in the embedding stage are meaningful, one solution is to utilize a model agnostic feature attribution method over the “stacked” model.
Revised: Since the original features in the embedding stage are meaningful, one solution is to utilize a model agnostic feature attribution method over the “stacked” model.

Original: For our attributions, we aim to provide Shapley values, but unfortunately the exact model agnostic computation has an exponential computational complexity ( O ( N 2 M ) , where N is the sample size and M is the number of features) (Shapley, 1953).
Revised: For our attributions, we aim to provide Shapley values, for which the exact model agnostic computation has an exponential computational complexity ( O ( N 2 M ) , where N is the sample size and M is the number of features) (Shapley, 1953).

Original: In response, one might want to use a model-speciﬁc method of computing approximate Shapley values – which often gains speed by using knowledge of the model.
Revised: In response, one might want to use a model-speciﬁc method of computing approximate Shapley values to gain speed by knowing the model.

Original: However, to the authors’ knowledge, there was previously noknown model-speciﬁc method to estimate Shapley values for a stack comprised of LSTMs and a GBM.
Revised: However, to the authors’ knowledge, there was previously no model-speciﬁc method to estimate Shapley values for a stack comprised of LSTMs and a GBM (or even local feature attributions for that matter).

Original: Single reference Shapley values: Our new method for estimating Shapley values for the aforementioned stacked model (i.e., LSTMs and GBM), requires adaptations on two existing feature attributions methods.
Revised: Single reference Shapley values: Our new method for estimating Shapley values for the aforementioned stacked model (i.e., LSTMs and GBM), requires adaptations on two existing feature attributions methods.

Original: First is Deep SHAP, a variant on DeepLIFT – a feature attribution method for neural networks (Lundberg, 2018; Shrikumar et al., 2016).
Revised: First is Deep SHAP, a variant on Deep LIFT – a feature attribution method for neural networks (Lundberg, 2018; Shrikumar et al., 2016).

Original: Deep SHAP differs from DeepLIFT in that it can ﬁnd attributions for single references.
Revised: Deep SHAP differs from Deep LIFT in that it can ﬁnd attributions for single references.

Original: Both methods can be written as modiﬁcations to a traditional backward pass through a neural network (Ancona et al., 2018).
Revised: Both methods can be written as modiﬁcations to a traditional backward pass through a neural network (Ancona et al., 2018).

Original: Since the computational complexity of a backward pass is the same as a forward pass through the network, we can consider this cost “low”.
Revised: Since the computational complexity of a backward pass is the same as a forward pass through the network, we can consider this cost “low”.

Original: The second method we utilize is “Independent Tree SHAP”.
Revised: The second method we utilize is “Independent Tree SHAP”.

Original: This method is a variation on normal TreeSHAP (Lundberg et al., 2018), but it can be computed for single references.
Revised: This method is a variation on normal Tree SHAP (Lundberg et al., 2018), but it can be computed for single references.

Original: Independent Tree SHAP has a computational complexity of O ( MLT ) , where L is the maximum number of leaves in any given tree and T is the number of trees in the GBM.
Revised: Independent Tree SHAP has a computational complexity of O ( MLT ) , where L is the maximum number of leaves in any given tree and T is the number of trees in the GBM.

Original: “Stacked” model Shapley values: Combining these two methods amounts to treating the “stacked” model (Figure 1c) as a larger neural network and applying Deep SHAP to pass back attributions as gradients at each layer (Ancona et al., 2018).
Revised: “Stacked” model Shapley values: Combining these two methods amounts to treating the “stacked” model (Figure 1c) as a larger neural network and applying Deep SHAP to pass back attributions as gradients at each layer (Ancona et al., 2018).

Original: However, at the GBM layer we obtain the appropriate gradients by dividing the Independent Tree SHAP Shapley values by the difference between the sample and the references.
Revised: However, at the GBM layer we obtain the appropriate gradients by dividing the Independent Tree SHAP Shapley values by the difference between the sample and the references.

Original: According to Theorem 1, we can then average over these single reference attributions for an approximation to the Shapley values.
Revised: According to Theorem 1, we can then average over these single reference attributions for an approximation to the Shapley values.

Original: Generalizability: Note that Theorem 1 also implies that for any arbitrary set of models in a stack, if single reference Shapley values are obtainable for each model, the Shapley values for the entire stack can be obtained.
Revised: Generalizability: Note that Theorem 1 also implies that for any arbitrary set of models in a stack, if single reference Shapley values are obtainable for each model, the Shapley values for the entire stack can be obtained.

Original: Because the single reference Shapley value methods are known for neural networks and for trees, any “stacked” model composed of these two methods can be explained.
Revised: Because the single reference Shapley value methods are known for neural networks and for trees, any “stacked” model composed of these two methods can be explained.

Original: Worth noting is that many embedding/prediction models can be represented as neural networks, making our framework to attribute “stacked” models fairly general.
Revised: Worth noting is that many embedding/prediction models can be represented as neural networks, making our framework to attribute “stacked” models fairly general.

Original: Theorem 1.
Revised: Theorem 1.

Original: Computing the average over single reference Shapley values approaches the true Shapley values.
Revised: Computing the average over single reference Shapley values approaches the true Shapley values.

Original: Proof.
Revised: Proof.

Original: Starting with the deﬁnition of Shapley values:
Revised: Starting with the deﬁnition of Shapley values:

Original: [Equation]
Revised: [Equation]

Original: where D is the data distribution, F is the set of all features, and f is our model.
Revised: where D is the data distribution, F is the set of all features, and f is our model.

Original: Rewriting the sum over all permutations of F , rather than over all combinations, the weighting term becomes one:
Revised: Rewriting the sum over all permutations of F , rather than over all combinations, the weighting term becomes one:

Original: [Equation]
Revised: [Equation]

Original: where the last step depends on independence between the permutations, independence between the conditional and non-conditional sets, and the data generating mechanism.
Revised: where the last step depends on independence between the permutations, independence between the conditional and non-conditional sets, and the data generating mechanism.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We ﬁrst describe our data sets , evaluation metric, model architectures, and the results of comparisons between PHASE and alternative approaches in various testing scenarios (Section 4.2).
Revised: We ﬁrst describe our data sets, evaluation metric, model architectures, and the results of comparisons between PHASE and alternative approaches in various testing scenarios (Section 4.2).

Original: More details on the model architecture and the models used for each experiment can be found in the Appendix.
Revised: More details on the model architecture and the models used for each experiment can be found in the Appendix.

Original: 4. E XPERIMENTAL S ETUP
Revised: 4. E XPERIMENTAL S ETUP

Original: Data Description Hospital 0/1 data was collected via the Anesthesia Information Management System (AIMS), which records all data measured in the operating room during surgery.
Revised: Data Description Hospital 0/1 data was collected via the Anesthesia Information Management System (AIMS), which records all data measured in the operating room during surgery.

Original: Both medical centers are within the same city (within 10 miles of each other).
Revised: Both medical centers are within the same city (within 10 miles of each other).

Original: Hospital P is a sub-sampled version of the publicly available MIMIC data set from PhysioNet, which contains data obtained from an intensive care unit in Boston, Massachusetts (Johnson et al., 2016).
Revised: Hospital P is a sub-sampled version of the publicly available MIMIC data set from PhysioNet, which contains data obtained from an intensive care unit in Boston, Massachusetts (Johnson et al., 2016).

Original: Hospital P data was collected several thousands of miles from the medical centers associated with hospital 0/1 data.
Revised: Hospital P data was collected several thousands of miles from the medical centers associated with hospital 0/1 data.

Original: More details about these hospitals are in Table 1.
Revised: Some details about these hospitals are in Table 1, with more in Appendix: Section 6.1.

Original: [Table]
Revised: [Table]

Original: The hospital 0/1 data includes static information (height, weight, age, sex, procedure codes), as well as real-time measurements of thirty-ﬁve physiological signals (e.g., SaO 2 , FiO 2 , ETCO 2 , etc.)
Revised: The hospital 0/1 data includes static information (height, weight, age, sex, procedure codes), as well as real-time measurements of thirty-ﬁve physiological signals (e.g., SaO 2 , FiO 2 , ETCO 2 , etc.)

Original: sampled minute by minute.
Revised: sampled minute by minute.

Original: Although the hospital P data contains several physiological signals sampled at a high frequency, we solely use a minute by minute SaO 2 signal.
Revised: Although the hospital P data contains several physiological signals sampled at a high frequency, we solely use a minute by minute SaO 2 signal for our experiments.

Original: Any missing values in the data are imputed by the mean and each feature is normalized to have unit mean and variance.
Revised: Any missing values in the data are imputed by the mean and each feature is standardized to have unit mean and variance.

Original: One important note is that although hospitals 0 and 1 are spatially close, one is an academic medical center and one is a trauma center.
Revised: One important note is that although hospitals 0 and 1 are spatially close, one is an academic medical center and one is a trauma center.

Original: More details on the patients from these hospitals can be found in Section 6.1.
Revised: More details on the patients from these hospitals can be found in Section 6.1.

Original: Evaluation Methodology PHASE and alternative approaches are evaluated based on real-time prediction tasks, for example, whether a certain condition will occur in the next 5 minutes (details in
Revised: Evaluation Methodology PHASE and alternative approaches are evaluated based on real-time prediction tasks, for example, whether a certain condition will occur in the next 5 minutes (details in

Original: Section 4.2).
Revised: Sections 4.2 and 6.1).

Original: Our evaluation metric for prediction performance in binary classiﬁcation is area under the precision-recall curve, otherwise known as average precision (AP).
Revised: Our evaluation metric for prediction performance in binary classiﬁcation is area under the precision-recall curve, otherwise known as average precision (AP).

Original: Rather than ROC curves, PR curves often better highlight imbalanced labels.
Revised: Rather than ROC curves, PR curves often better highlight imbalanced labels.

Original: Precision is deﬁned as tp

Original: [Equation] and recall is [Equation] where tp is true positives, fp false positives, and fn false negatives.
Revised: Precision is deﬁned as tp tp + fp and recall is tp tp + fn , where tp is true positives, fp false positives, and fn false negatives.

Original: The area under the curve provides a summary statistic that balances both precision and recall.
Revised: The area under the curve provides a summary statistic that balances both precision and recall.

Original: 4. R ESULTS
Revised: 4. R ESULTS

Original: In this section, we compare different embeddings of physiological signals as discussed in Table(where Min h represents PHASE).
Revised: In this section, we compare different embeddings of physiological signals as discussed in Table(where Min h represents PHASE).

Original: Based on these comparisons, we see the overall performance of predicting three adverse clinical events in Section 4.2.1 as well as a discussion of how well the embedding learners transfer between hospitals in Section 4.2.2.
Revised: Based on these comparisons, we see the overall performance of predicting three adverse clinical events in Section 4.2.1 as well as a discussion of how well the embedding learners transfer between hospitals in Section 4.2.2.

Original: Lastly, in Section 4.2.3, we depict the attributions from our new model stacking method for obtaining Shapley values.
Revised: Lastly, in Section 4.2.3, we depict the attributions from our new model stacking method for obtaining Shapley values.

Original: In the operating room, there are a few physiological signals that stand out as indicators of adverse outcomes.
Revised: In the operating room, there are a few physiological signals that stand out as indicators of adverse outcomes.

Original: Three of these signals include SaO 2 (blood oxygen), ETCO 2 (end tidal CO2), and NIBPM (noninvasive blood pressure measurement), which are linked to our three adverse outcomes: hypoxemia, hypocapnia, and hypotension, respectively.
Revised: Three of these signals include SaO 2 (blood oxygen), ETCO 2 (end tidal CO2), and NIBPM (noninvasive blood pressure measurement), which are linked to our three adverse outcomes: hypoxemia, hypocapnia, and hypotension, respectively.

Original: Forecasting these outcomes is particularly important because deviations from the norm could spell disaster.
Revised: Forecasting these outcomes is particularly important because deviations from the norm could spell disaster (Lundberg et al., 2017; Curley et al., 2010; Barak et al., 2015).

Original: More details on the labelling are in Section 6.1. 
Revised: More details on labels in Section 6.1. [Table] Prescience [Table]

Revised: Exponential moving averages as well as variances of each signal.

Revised: Computed where weights decay with a half-life of 6 seconds, 1 minute, or 5 minutes.

Original: [Table]
Revised: [Table]

Original: Hidden layer embedding from an LSTM trained to predict the minimum of the current signal ﬁve minutes into the future on hospital h’s data.
Revised: Hidden layer embedding from an LSTM trained to predict the minimum of the current signal ﬁve minutes into the future on hospital h’s data.

Original: [Table]
Revised: [Table]

Original: Hidden layer embedding from an LSTM trained to predict an output signal identical to the input signal on hospital h’s data.
Revised: Hidden layer embedding from an LSTM trained to predict an output signal identical to the input signal on hospital h’s data.

Original: [Table]
Revised: [Table]

Original: Hidden layer embedding from an LSTM trained to predict hypoxemia on hospital h’s data.
Revised: Hidden layer embedding from an LSTM trained to predict hypoxemia on hospital h’s data.

Original: 4.2. P REDICTION P ERFORMANCE
Revised: 4.2. P REDICTION P ERFORMANCE

Original: In this section, we aim to investigate the performance gains  embeddings offer over other methodologies.
Revised: In this section, we aim to investigate the performance gains PHASE embeddings offer over other embeddings.

Original: Figure 2 shows the performance of XGB with different representations of the same signals across three prediction tasks.
Revised: Figure 2 shows the performance of XGB with different representations of the same signals across three prediction tasks.

Original: In terms of pre-training for this experiment, there is none for the Raw and EMA embeddings.
Revised: In terms of pre-training for this experiment, there is none for the Raw and EMA embeddings.

Original: However for Min h and Auto h , we have trained ﬁfteen univariate LSTM networks for both objectives across both hospitals (a total of sixty  networks).
Revised: However for Min h and Auto h , we have trained ﬁfteen univariate LSTM networks for both objectives across both hospitals (a total of sixty networks, not including ﬁne-tuned networks).

Original: We ﬁx these same LSTM networks to generate hidden embeddings of the original signals across the three ﬁnal prediction tasks of hypoxemia, hypocapnia, and hypotension.
Revised: We ﬁx these same LSTM networks to generate hidden embeddings of the original signals across the three ﬁnal prediction tasks of hypoxemia, hypocapnia, and hypotension.

Original: In terms of performance, the average precision points with all signals (in blue) are almost always signiﬁcantly above their associated average precision points using only a single signal (in red).
Revised: In terms of performance, the average precision points with all signals (in blue) are almost always signiﬁcantly above their associated average precision points using only a single signal (in red).

Original: This suggests the outcomes derived from forecasting a single signal are complex and beneﬁt from having access to more signals.
Revised: This suggests the outcomes derived from forecasting a single signal are complex and beneﬁt from having access to more signals.

Original: Most importantly, with the exception of the ﬁne tuned model (14), the Min h (PHASE) models (10 and 12) consistently outperform all other models in Figure 2 by a signiﬁcant margin.
Revised: Most importantly, with the exception of the ﬁne tuned model (14), the Min h (PHASE) models (10 and 12) consistently outperform all other models in Figure 2 by a signiﬁcant margin.

Original: The fact that LSTM autoencoders fail to grasp meaningful representations of the data in comparison to partially supervised LSTMs offers an insight; on our data set, the key to performant embeddings is to have closeness between the LSTM’s prediction task and the downstream prediction task.
Revised: The fact that LSTM autoencoders fail to grasp meaningful representations of the data in comparison to partially supervised LSTMs offers an insight; on our data set, the key to performant embeddings is to have closeness between the LSTM’s prediction task and the downstream prediction task.

Original: In this case, effective embeddings for forecasting adverse outcomes related to too-low signals, required that the embeddings themselves were related to low signals in the future.
Revised: In this case, effective embeddings for forecasting adverse outcomes related to too-low signals, required that the embeddings themselves were related to low signals in the future.

Original: 4.2. T RANSFERENCE BETWEEN H OSPITALS
Revised: 4.2. T RANSFERENCE BETWEEN H OSPITALS

Original: In this section, we have two aims: the ﬁrst aim is to evaluate how well our embedding models transfer.
Revised: In this section, we have two aims: the ﬁrst aim is to evaluate how well our embedding models transfer.

Original: The second aim is to explore methods to repurpose them when there is a large amount of domain shift between hospitals.
Revised: The second aim is to explore methods to repurpose them when there is a large amount of domain shift between hospitals.

Original: First, we can look back to Figure 2.
Revised: First, we can look back to Figure 2.

Original: The feature embeddings learned in a source hospital that differs to the target hospital (12) performs signiﬁcantly better than the EMA (Prescience - Lundberg et al.
Revised: The feature embeddings learned in a source hospital that differs to the target hospital (12) performs signiﬁcantly better than the EMA (Prescience - Lundberg et al.

Original: and Raw embeddings (2 and 4) and generally on par with a matching source and target hospital (10).
Revised: and Raw embeddings (2 and 4) and generally on par with a matching source and target hospital (10).

Original: This is promising, because it suggests that the domain shift between hospitals 0 and 1 does not prevent physiological signal embeddings from transferring well.
Revised: This is promising, because it suggests that the domain shift between hospitals 0 and 1 does not prevent physiological signal embeddings from transferring well.

Original: Although these hospitals do have similar patient distributions, this transference is better than
Revised: Although these hospitals do have similar patient distributions, this transference is better than

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Notation described in Table where T the details experiment in Section might be expected, given that one hospital is an academic medical center and the other is a trauma center. 
Revised: might be expected, given that one hospital is an academic medical center and the other is a trauma center. As further evidence of their differences, we report the top ten diagnoses from each hospital in

Revised: Appendix: Section 6.1 and ﬁnd no overlap apart from CALCULUS OF KIDNEY between hospitals 0 and 1.

Original: Next, in Figure 3, we can see that the Min P embeddings obtained from the publicly available PhysioNet datacreates a worse embedding for SaO 2 (2 p )  in comparison to the model trained with the target hospital’s data (6).
Revised: Next, in Figure 3, we can see that the Min P embeddings created from the publicly available PhysioNet data (2 p ) are worse representations of SaO 2 in comparison to the embeddings trained with the target hospital’s data (6).

Original: This result suggests that the domain shift between hospital P and the OR hospitals is too large - resulting in learned Min embeddings ( 2 P ) that are not as useful for predic- tion.
Revised: This result implies that the domain shift between hospital P and the OR hospitals is too large – resulting in learned Min embeddings ( 2 P ) that are not as useful for prediction.

Original: The improvement of the ﬁne tuned models ( 4 P ) over Min P embeddings, suggests the following insight: that ﬁne tuning serves to recover the performance lost due to transference across distributionally disparate hospitals.
Revised: Next, the improvement of the ﬁne tuned models ( 4 P ) over Min P embeddings, suggests the following insight: that ﬁne tuning serves to recover the performance lost due to transference across distributionally disparate hospitals.

Original: Then, observing the improvement of Hypox P embeddings ( 6 P ) over
Revised: Then, observing the improvement of Hypox P embeddings ( 6 P ) over

Original: Min P embeddings, suggests a natural insight: that closeness between LSTM prediction tasks and the downstream prediction task is beneﬁcial in the face of transference.
Revised: Min P embeddings, conveys a natural insight: that closeness between LSTM prediction tasks and the downstream prediction task is beneﬁcial in the face of transference.

Original: This naturally extends the observation from Section 4.2.1: that closeness between LSTM prediction tasks and the downstream prediction task is beneﬁcial in the face of performance.
Revised: This naturally extends the observation from Section 4.2.1: that closeness between LSTM prediction tasks and the downstream prediction task is beneﬁcial in the face of performance.

Original: These two observations suggest two approaches to transferring across substantially different hospital data sets: 1.) train LSTMs with  prediction tasks identical to the downstream prediction and 2.)
Revised: These two observations suggest two approaches to transferring across substantially different hospital data sets: 1.) train LSTMs with very speciﬁc prediction tasks that match the downstream prediction and 2.)

Original: ﬁne tuneMin LSTM networks to overcome shifts in domain.
Revised: ﬁne tune LSTM networks.

Original: 4.2. Q UANTITATIVE V ALIDATION OF I NTERPRETATION FOR S TACKED M ODELS
Revised: 4.2. Q UANTITATIVE V ALIDATION OF I NTERPRETATION FOR S TACKED M ODELS

Original: In this section, our aim is to evaluate the efﬁcacy of our interpretability method for stacked models.
Revised: In this section, our aim is to evaluate the efﬁcacy of our interpretability method for stacked models.

Original: To do so, we separate interpretability evaluation approaches into  qualitative and quantitative.
Revised: To do so, we separate interpretability evaluation approaches into two categories: qualitative and quantitative.

Original: Although qualitative evaluations are key to ensuring human insight into interpretability methods, we focus on a quantitative approach.
Revised: Qualitative evaluations are important to ensure human insight into interpretability methods.

Original: Our primary goal is to ensure that our novel method to obtain local feature attributions for stacked models are correct in the sense that they estimate Shapley values.
Revised: However, our primary goal is to ensure that our novel method to obtain local feature attributions for stacked models is correct in the sense that the attributions estimate Shapley values.

Original: In terms of a qualitative evaluation of feature attributions, Lundberg et al.
Revised: One qualitative evaluation of feature attributions is Lundberg et al.

Original: (2017)  demonstrate that local feature attributions improve the performance of practicing anesthesiologists in forecasting hypoxemia.
Revised: (2017) who demonstrate that local feature attributions improve the performance of practicing anesthesiologists in forecasting hypoxemia.

Original: Our quantitative validation is a standard ablation/perturbation test, in a similar fashion to other interpretability evaluations:
Revised: Our quantitative validation is a standard ablation/perturbation test, in a similar fashion to other interpretability evaluations:

Original: Arras et al.
Revised: Arras et al.

Original: (2017), Hooker et al.
Revised: (2017), Hooker et al.

Original: (2018), Ancona et al.
Revised: (2018), Ancona et al.

Original: (2018), and Samek et al.
Revised: (2018), and Samek et al.

Original: The test consists of the following.
Revised: The test consists of the following.

Original: For a single sample, we sort the input features according to their attributions, and iteratively impute each feature by the mean of the last two minutes.
Revised: For a single sample, we sort the input features according to their attributions, and iteratively impute each feature by the mean of the last two minutes.

Original: In order to ensure our interpretations generalize, we evaluate on the test set.
Revised: In order to ensure our interpretations generalize, we evaluate on the test set.

Original: Additionally, we use the top 1000 positive samples sorted by the predicted probability of hypoxemia (true positives).
Revised: Additionally, we use the top 1000 positive samples sorted by the predicted probability of hypoxemia (true positives).

Original: Then we evaluate the mean predicted probability across all samples, which will start high (for true positives) and monotonically decrease as we impute features, leading to an overall decrease in the average probability.
Revised: Then we evaluate the mean predicted probability across all samples, which will start high (for true positives) and monotonically decrease as we impute features, leading to an overall decrease in the average probability.

Original: Good interpretability methods should result in an initial steepness because the most ”important” hypoxemic pathology is imputed ﬁrst.
Revised: Good interpretability methods should result in an initial steepness because the most ”important” hypoxemic pathology is imputed ﬁrst.

Original: [Figure]
Revised: [Figure]

Original: The model stack used for the interpretability evaluation (right) is (9) in Figure 2a.
Revised: The model stack used for the interpretability evaluation (right) is (9) in Figure 2a.

Original: More details about the setup for this experiment in Section 6.5.
Revised: More details about the setup for this experiment in Section 6.

Original: Deep SHAP was originally proposed for traditional deep learning models, we extend it to support a mixture of model types in PHASE.
Revised: Deep SHAP was originally proposed for traditional deep learning models, we extend it to support a mixture of model types in PHASE.

Original: As such, our primary aim is to ensure closeness to Shapley values, by evaluating against the pre-existing Deep SHAP methodology.
Revised: As such, our primary aim is to evaluate against the pre-existing Deep SHAP methodology.

Original: We anchor the performance of LSTM embeddings fed into XGB (PHASE) against LSTM embeddings fed into an MLP, because original Deep SHAP supports only traditional neural network architecture (see Figurein the Appendix for more details on the model architectures).
Revised: We compare LSTM embeddings fed into XGB (PHASE) against LSTM embeddings fed into an MLP, because original Deep SHAP supports only traditional neural network architecture (see Figure 11 in the Appendix for more details on the model setups).

Original: In Figure 4, we verify that ordering feature imputation by our interpretability method which combines Deep SHAP with Independent Tree SHAP (LSTM → XGB) does lead to initial steepness within the predicted probability of hypox- emia in a similar fashion to Deep SHAP alone (LSTM → MLP), with both methods outperforming a random ordering.
Revised: In Figure 4, we verify that ordering feature imputation by our interpretability method which combines Deep SHAP with Independent Tree SHAP (LSTM → XGB) does lead to initial steepness of the predicted probability of hypoxemia in a similar fashion to Deep SHAP alone (LSTM → MLP), with both methods outperforming a random ordering.

Original: In fact, it appears that for the Figure 4 (right), imputing our attributions for LSTM → XGB has an even more destructive effect on performance in the early number of imputed features.
Revised: In fact, it appears that for the Figure 4 (right), imputing our attributions for LSTM → XGB has a potentially more destructive effect on performance in the early number of imputed features.

Original: In Appendix Section 6.5.1, we show attributions from both methods in a informal visual evaluation to conﬁrm concordance.
Revised: Lastly, in Appendix Section 6.5.1, we show attributions from both methods in a informal visual evaluation to conﬁrm concordance.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This paper presents PHASE, a new approach to machine learning with physiological signals based on transferring embedding learners.
Revised: This paper presents PHASE, a new approach to machine learning with physiological signals based on transferring embedding learners.

Original: PHASE has potentially far-reaching impacts, because neural networks inherently create an embedding before the ﬁnal output layer.
Revised: PHASE has potentially far-reaching impacts, because neural networks inherently create an embedding before the ﬁnal output layer.

Original: As discussed in Section 2.2, there is a large body of research independently working on neural networks for physiological signals.
Revised: As discussed in Section 2.2, there is a large body of research independently working on neural networks for physiological signals.

Original: PHASE offers a potential method of collaboration by analyzing partially supervised univariate networks as semi-private ways to share meaningful signals without sharing data sets.
Revised: PHASE offers a potential method of collaboration by analyzing partially supervised univariate networks as semi-private ways to share meaningful signals without sharing data sets.

Original: In the results section we offer several insights into transference of univariate LSTM embedding functions.
Revised: In the results section we offer several insights into transference of univariate LSTM embedding functions.

Original: First, closeness of upstream (LSTM) and downstream prediction tasks is indeed important for both predictive performance and transference.
Revised: First, closeness of upstream (LSTM) and downstream prediction tasks is indeed important for both predictive performance and transference.

Original: For performance, we found that predicting the minimum of the future ﬁve minutes was sufﬁcient for the LSTMs to generate good embeddings.
Revised: For performance, we found that predicting the minimum of the future ﬁve minutes was sufﬁcient for the LSTMs to generate good embeddings.

Original: For transference, predicting the minimum of the next ﬁve minutes was sufﬁcient to transfer across similar domains (operating room data from an academic medical center and a trauma center) when predicting hypoxemia.
Revised: For transference, predicting the minimum of the next ﬁve minutes was sufﬁcient to transfer across similar domains (operating room data from an academic medical center and a trauma center) when predicting hypoxemia.

Original: However when attempting to utilize a representation from Hospital P, we found that the difference between operating rooms and intensive care units was likely too large to provide good predictions.
Revised: However when attempting to utilize a representation from Hospital P, we found that the difference between operating rooms and intensive care units was likely too large to provide good predictions.

Original: Two solutions to this include ﬁne tuning the Min LSTM models as well as acknowledging the large amount of domain shift and training speciﬁc LSTM embedding models with a particular downstream prediction in mind.
Revised: Two solutions to this include ﬁne tuning the Min LSTM models as well as acknowledging the large amount of domain shift and training speciﬁc LSTM embedding models with a particular downstream prediction in mind.

Original: Last but not least, this paper introduced a way to obtain feature attributions for stacked models of neural networks and trees.
Revised: Last but not least, this paper introduced a way to obtain feature attributions for stacked models of neural networks and trees.

Original: By showing that Shapley values may be computed as the mean over single reference Shapley values, this model stacking framework generalizes to all models for which single reference Shapley values can be obtained, which was quantitatively veriﬁed in Section 4.2.3.
Revised: By showing that Shapley values may be computed as the mean over single reference Shapley values, this model stacking framework generalizes to all models for which single reference Shapley values can be obtained, which was quantitatively veriﬁed in Section 4.2.3.

Original: We intend to release code pertinent to training the LSTM models, obtaining embeddings, predicting with XGB models, and model stacking feature attributions – submitted as a pull request to the
Revised: We intend to release code pertinent to training the LSTM models, obtaining embeddings, predicting with XGB models, and model stacking feature attributions – submitted as a pull request to the

Original: SHAP github (https://github.com/slundberg/shap).
Revised: SHAP github (https://github.com/slundberg/shap).

Original: Additionally, we intend to release our embedding models, which we primarily recommend for use in forecasting ”hypo” predictions.
Revised: Additionally, we intend to release our embedding models, which we primarily recommend for use in forecasting ”hypo” predictions.

Original: In the direction of future work, it is important to carefully consider representation learning in health – particularly in light of model inversion attacks as discussed in Fredrikson et al. (2015).
Revised: In the direction of future work, it is important to carefully consider representation learning in health – particularly in light of model inversion attacks as discussed in Fredrikson et al. (2015).

Original: To this end, future work in making precise statements about the privacy of models deserves attention, for which one potential avenue may be differential privacy (Dwork, 2008).
Revised: To this end, future work in making precise statements about the privacy of models deserves attention, for which one potential avenue may be differential privacy (Dwork, 2008).

Original: Other important areas to explore include extending these results to higher sampling frequencies.
Revised: Other important areas to explore include extending these results to higher sampling frequencies.

Original: Our data was sampled once per minute, but higher resolution data may beget different neural network architectures.
Revised: Our data was sampled once per minute, but higher resolution data may beget different neural network architectures.

Original: Lastly, further work may include quantifying the relationship between domain shifts in hospitals and PHASE and determining other relevant prediction tasks for which embeddings can be applied (e.g., ”hyper” predictions, doctor action prediction, etc.).
Revised: Lastly, further work may include quantifying the relationship between domain shifts in hospitals and PHASE and determining other relevant prediction tasks for which embeddings can be applied (e.g., ”hyper” predictions, doctor action prediction, etc.).

Original: Special Section: Methods in Clinical Research Informatics

Original:  A PPENDIX 6. E XPERIMENTAL SETUP
Revised:  A PPENDIX 6. E XPERIMENTAL SETUP

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Top ten diagnoses (Hospital 0):
Revised: Top ten diagnoses (Hospital 0):

Original: CATARACT NOS, SUBARACHNOID HEMORRHAGE, CALCULUS OF KIDNEY,
Revised: CATARACT NOS, SUBARACHNOID HEMORRHAGE, CALCULUS OF KIDNEY,

Original: OTHER
Revised: OTHER

Original: COMPLICATIONS DUE TO OTHER INTERNAL ORTHOPEDIC DEVICE IMPLANT
Revised: COMPLICATIONS DUE TO OTHER INTERNAL ORTHOPEDIC DEVICE IMPLANT

Original: AND GRAFT, SENILE CATARACT NOS, SENILE CATARACT UNSPECIFIED,
Revised: AND GRAFT, SENILE CATARACT NOS, SENILE CATARACT UNSPECIFIED,

Original: NECROTIZING FASCIITIS, CATARACT, CARPAL TUNNEL SYNDROME, CMP NEC
Revised: NECROTIZING FASCIITIS, CATARACT, CARPAL TUNNEL SYNDROME, CMP NEC

Original: D/T ORTH DEV NEC
Revised: D/T ORTH DEV NEC

Original: Top ten diagnoses (Hospital 1):
Revised: Top ten diagnoses (Hospital 1):

Original: MALIGNANT NEOPLASM OF BREAST (FEMALE) UNSPECIFIED, MALIGN NEOPL
Revised: MALIGNANT NEOPLASM OF BREAST (FEMALE) UNSPECIFIED, MALIGN NEOPL

Original: BREAST NOS, ATRIAL FIBRILLATION, MORBID OBESITY, CALCULUS OF
Revised: BREAST NOS, ATRIAL FIBRILLATION, MORBID OBESITY, CALCULUS OF

Original: KIDNEY, ESOPHAGEAL REFLUX, MALIGNANT NEOPLASM OF PROSTATE,
Revised: KIDNEY, ESOPHAGEAL REFLUX, MALIGNANT NEOPLASM OF PROSTATE,

Original: MALIGNANT NEOPLASM OF BLADDER PART UNSPECIFIED, PREV C-SECT
Revised: MALIGNANT NEOPLASM OF BLADDER PART UNSPECIFIED, PREV C-SECT

Original: NOS-DELIVER, END STAGE RENAL DISEASE
Revised: NOS-DELIVER, END STAGE RENAL DISEASE

Original: Top ten diagnoses (Hospital P):
Revised: Top ten diagnoses (Hospital P):

Original: NEWBORN, PNEUMONIA, TELEMETRY, SEPSIS, CONGESTIVE HEART FAILURE,
Revised: NEWBORN, PNEUMONIA, TELEMETRY, SEPSIS, CONGESTIVE HEART FAILURE,

Original: CORONARY ARTERY DISEASE, CHEST PAIN, GASTROINTESTINAL BLEED,
Revised: CORONARY ARTERY DISEASE, CHEST PAIN, GASTROINTESTINAL BLEED,

Original: ALTERED MENTAL STATUS, INTRACRANIAL HEMORRHAGE
Revised: ALTERED MENTAL STATUS, INTRACRANIAL HEMORRHAGE

Original: Labels For hypoxemia, a particular time point t is labelled to be one if the minimum of the next ﬁve minutes is hypoxemic ( min( SaO t +1: t +6) ≤ 92 ).
Revised: Labels For hypoxemia, a particular time point t is labelled to be one if the minimum of the next ﬁve minutes is hypoxemic ( min( SaO t +1: t +6) ≤ 92 ).

Original: All points where the current time step is currently hypoxemic are ignored (SaO t 2 ≤ 92 ).
Revised: All points where the current time step is currently hypoxemic are ignored (SaO t 2 ≤ 92 ).

Original: Additionally we ignore time points where the past ten minutes were all missing or the future ﬁve minutes were all missing.
Revised: Additionally we ignore time points where the past ten minutes were all missing or the future ﬁve minutes were all missing.

Original: Hypocapnia and hypotension are only labelled for hospitals 0 and 1.
Revised: Hypocapnia and hypotension are only labelled for hospitals 0 and 1.

Original: Additionally, we have stricter label conditions.
Revised: Additionally, we have stricter label conditions.

Original: We labeled the current time point t to be one if ( min( S t − 10: t ) > T ) and the minimum of the next ﬁve minutes is ”hypo” ( min( S t +1: t +5 ) ≤ T ).
Revised: We labeled the current time point t to be one if ( min( S t − 10: t ) > T ) and the minimum of the next ﬁve minutes is ”hypo” ( min( S t +1: t +5 ) ≤ T ).

Original: We labeled the current time point t to be zero if ( min( S t − 10: t ) > T ) and the minimum of the next ten minutes is not ”hypo” ( min( S t +1: t +10 ) > T ).
Revised: We labeled the current time point t to be zero if ( min( S t − 10: t ) > T ) and the minimum of the next ten minutes is not ”hypo” ( min( S t +1: t +10 ) > T ).

Original: All other time points were not considered.
Revised: All other time points were not considered.

Original: For hypocapnia, the threshold T = 34 and the signal S is ETCO 2 .
Revised: For hypocapnia, the threshold T = 34 and the signal S is ETCO 2 .

Original: For hypotension the threshold is T = 59 and the signal S is NIBPM.
Revised: For hypotension the threshold is T = 59 and the signal S is NIBPM.

Original: Additionally we ignore time points where the past ten minutes were all missing or the future ﬁve minutes were all missing.
Revised: Additionally we ignore time points where the past ten minutes were all missing or the future ﬁve minutes were all missing.

Original: As a result, we have different sample sizes for different prediction tasks (reported in Table 4).
Revised: As a result, we have different sample sizes for different prediction tasks (reported in Table 4).

Original: For Min predictions, the label is the value of min( S t +1: t +5 ) , points without signal for in the future ﬁve minutes are ignored.
Revised: For Min predictions, the label is the value of min( S t +1: t +5 ) , points without signal for in the future ﬁve minutes are ignored.

Original: For Auto predictions, the label is all the time points:
Revised: For Auto predictions, the label is all the time points:

Original: S t − 59: t .
Revised: S t − 59: t .

Original: The sample sizes for Min and Auto are the same and are reported in Table 3.
Revised: The sample sizes for Min and Auto are the same and are reported in Table 3.

Original: [Table]
Revised: [Table]

Original: 77e 3 . 79e

Original: NIBPS 2 . 37e 2 .

Original: 63e 3 .

Original: 34e 2 .

Original: 49e 2 . 77e 3 . 80e

Original: PEAK 3 . 20e 3 .

Original: 56e 4 .

Original: 47e 1 . 77e 1 . 96e 2 .

Original: 63e

Original: PEEP 3 . 17e 3 . 53e 4 . 42e 3 .

Original: 19e 3 .

Original: 54e 4 .

Original: 76e

Original: PIP 3 . 20e 3 .

Original: 56e 4 .

Original: 47e 1 . 77e 1 . 96e 2 .

Original: 63e

Original: RESPRATE 3 . 06e 3 .

Original: 40e 4 . 27e 3 . 65e 4 . 05e 5 . 44e SAO 3 .
Revised: [Table]

Original: 55e 3 .

Original: 94e 4 . 96e 3 . 79e 4 . 21e 5 . 65e TEMP 1 . 58e 1 . 76e 2 . 16e 2 .
Revised: [Table]

Original: 41e 2 . 68e 3 . 62e

Original: TV 3 . 21e 3 .

Original: 57e 4 . 47e 1 . 90e 2 . 11e 2 . 84e [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: M ODEL A RCHITECTURE AND T RAINING
Revised: M ODEL A RCHITECTURE AND T RAINING

Original: LSTM Architecture and Training: We utilize LSTMs with forget gates, introduced by Gers et al.
Revised: LSTM Architecture and Training: We utilize LSTMs with forget gates, introduced by Gers et al.

Original: (2000), implemented in the Keras library with a Tensorﬂow back-end.
Revised: (2000), implemented in the Keras library with a Tensorﬂow back-end.

Original: We train our networks with either regression (Auto and Min embeddings) or classiﬁcation (Hypox) objectives.
Revised: We train our networks with either regression (Auto and Min embeddings) or classiﬁcation (Hypox) objectives.

Original: For regression, we optimize using Adam with an MSE loss function.
Revised: For regression, we optimize using Adam with an MSE loss function.

Original: For classiﬁcation we optimize using RMSProp with a binary cross-entropy loss function (additionally, we upsample to maintain balanced batches during training).
Revised: For classiﬁcation we optimize using RMSProp with a binary cross-entropy loss function (additionally, we upsample to maintain balanced batches during training).

Original: Our model architectures consist of two hidden layers, each with 200 LSTM cells with dense connections between all layers.
Revised: Our model architectures consist of two hidden layers, each with 200 LSTM cells with dense connections between all layers.

Original: We found that important steps in training LSTM networks for our data are to impute missing values by the training mean, standardize data, and to randomize sample ordering prior to training (allowing us to sample data points in order without replacement).
Revised: We found that important steps in training LSTM networks for our data are to impute missing values by the training mean, standardize data, and to randomize sample ordering prior to training (allowing us to sample data points in order without replacement).

Original: To prevent overﬁtting, we utilized dropouts between layers as well as recurrent dropouts for the LSTM nodes.
Revised: To prevent overﬁtting, we utilized dropouts between layers as well as recurrent dropouts for the LSTM nodes.

Original: Using a learning rate of 0.001 gave us the best ﬁnal results.
Revised: Using a learning rate of 0.001 gave us the best ﬁnal results.

Original: The LSTM models were run to convergence (until their validation accuracy did not improve for ﬁve rounds of batch stochastic gradient descent).
Revised: The LSTM models were run to convergence (until their validation accuracy did not improve for ﬁve rounds of batch stochastic gradient descent).

Original: In order to train these models, we utilize three GPUs (GeForce GTX 1080 Ti graphics cards).
Revised: In order to train these models, we utilize three GPUs (GeForce GTX 1080 Ti graphics cards).

Original: GBM Architecture and Training: We train GBM trees in Python using XGBoost, an open source library for gradient boosting trees.
Revised: GBM Architecture and Training: We train GBM trees in Python using XGBoost, an open source library for gradient boosting trees.

Original: XGBoost works well in practice in part due to it’s ease of use and ﬂexibility.
Revised: XGBoost works well in practice in part due to it’s ease of use and ﬂexibility.

Original: Imputing and standardizing are unnecessary because GBM trees are based on splits in the training data, implying that scale does not matter and missing data is informative as is.
Revised: Imputing and standardizing are unnecessary because GBM trees are based on splits in the training data, implying that scale does not matter and missing data is informative as is.

Original: We found that a learning rate of 0.02 for hypoxemia (0.1 for hypotension and hypocapnia), a max tree depth of 6, subsampling rate of 0.5, and a logistic objective gave us good performance.
Revised: We found that a learning rate of 0.02 for hypoxemia (0.1 for hypotension and hypocapnia), a max tree depth of 6, subsampling rate of 0.5, and a logistic objective gave us good performance.

Original: All XGB models were run until their validation accuracy was non-improving for ﬁve rounds of adding estimators (trees).
Revised: All XGB models were run until their validation accuracy was non-improving for ﬁve rounds of adding estimators (trees).

Original: In order to train these models, we utilize 72 CPUs (Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz)
Revised: In order to train these models, we utilize 72 CPUs (Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz)

Original: 6. P REDICTION PERFORMANCE
Revised: 6. P REDICTION PERFORMANCE

Original: [Figure]
Revised: [Figure]

Original: XGB
Revised: XGB

Original: Hypoxemia
Revised: Hypoxemia

Original: XGB
Revised: XGB

Original: Hypoxemia …
Revised: Hypoxemia …

Original: SAO
Revised: SAO

Original: [Equation]
Revised: [Equation]

Original: TV
Revised: TV

Original: [Equation]
Revised: [Equation]

Original: XGB
Revised: XGB

Original: Hypoxemia
Revised: Hypoxemia

Original: SAO
Revised: SAO

Original: [Equation]
Revised: [Equation]

Original: EMA EMA
Revised: EMA EMA

Original: Raw (2)
Revised: Raw (2)

Original: EMA (4)
Revised: EMA (4)

Original: Min (10) (12)
Revised: Min (10) (12)

Original: Auto (6) (8)
Revised: Auto (6) (8)

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: 21e −0 .0 .0 .1 .0 . 0 .0 .0 .
Revised: 21e −0 .0 .0 .1 .0 . 0 .0 .0 .

Original: Fig 2b
Revised: Fig 2b

Original: Fig 2d
Revised: Fig 2d

Original: Fig 2f          0 .0 .0 .0 .0 .0 .0 .0 .0 .
Revised: Fig 2f          0 .0 .0 .0 .0 .0 .0 .0 .0 .

Original: 2e −8 .
Revised: 2e −8 .

Original: 9e −0 .
Revised: 9e −0 .

Original: 0e −
Revised: 0e −

Original: Leave One Signal Out Test:
Revised: Leave One Signal Out Test:

Original: In Figure 9, we create a simulated setting – when predicting each event, we excluded the corresponding physiological signal from our features.
Revised: In Figure 9, we create a simulated setting – when predicting each event, we excluded the corresponding physiological signal from our features.

Original: For example, we assumed that SaO 2 is not recorded when predicting hypoxemia.
Revised: For example, we assumed that SaO 2 is not recorded when predicting hypoxemia.

Original: Under this setting, we must rely on the remaining signals to predict hypoxemia.
Revised: Under this setting, we must rely on the remaining signals to predict hypoxemia.

Original: This setting is a more unsupervised evaluation in the sense that our outcome is not derived from a signal we create an embedding for.
Revised: This setting is a more unsupervised evaluation in the sense that our outcome is not derived from a signal we create an embedding for.

Original: As our results show (Figure 9), PHASEs outperformance is consistent for hypocapnia and hypotension.
Revised: As our results show (Figure 9), PHASEs outperformance is consistent for hypocapnia and hypotension.

Original: For hypoxemia, all representations perform poorly because predicting hypoxemia heavily relies on SaO2, leaving little signal for the remaining features.
Revised: For hypoxemia, all representations perform poorly because predicting hypoxemia heavily relies on SaO2, leaving little signal for the remaining features.

Original: This is likely due in part to the low base rates of hypoxemia: 1 . 06% in hospital 0 and 2 .
Revised: This is likely due in part to the low base rates of hypoxemia: 1 . 06% in hospital 0 and 2 .

Original: 32% in hospital 1.
Revised: 32% in hospital 1.

Original: Investigating further, we found that the log loss of the
Revised: Investigating further, we found that the log loss of the

Original: Min embeddings was lower than other embeddings on the validation set, but not on the test set.
Revised: Min embeddings was lower than other embeddings on the validation set, but not on the test set.

Original: This overﬁtting further suggests that there was little signal to be captured, causing simpler embeddings like EMA to be favored.
Revised: This overﬁtting further suggests that there was little signal to be captured, causing simpler embeddings like EMA to be favored.

Original: [Figure]
Revised: [Figure]

Original: c i s i
Revised: c i s i

Original: [Figure]
Revised: [Figure]

Original: 6. T RANSFERENCE
Revised: 6. T RANSFERENCE

Original: Multiple connections into a single XGB model are simply concatenated.
Revised: Multiple connections into a single XGB model are simply concatenated.

Original: All LSTMs consist of two layers each with 200 LSTM cells, trained in identical manners, as described in Section 6.2.
Revised: All LSTMs consist of two layers each with 200 LSTM cells, trained in identical manners, as described in Section 6.2.

Original: For
Revised: For

Original: XGB, the training is detailed in Section 6.2 as well.
Revised: XGB, the training is detailed in Section 6.2 as well.

Original: The univariate predictions made in Figureare similarly obtained, but only utilize the single feature used to obtain the ﬁnal prediction.
Revised: The univariate predictions made in Figureare similarly obtained, but only utilize the single feature used to obtain the ﬁnal prediction.

Original: Here
Revised: Here

Original: ”Hypoxemia”
Revised: ”Hypoxemia”

Original: ”Is
Revised: ”Is

Original: +
Revised: +

Original: +5) )
Revised: +5) )

Original: ≤ 92 ?”.
Revised: ≤ 92 ?”.

Original: ETCO2 &'(),⋯,& ETCO2 &,-,⋯,&,( ETCO2 &'(),⋯,&
Revised: ETCO2 &'(),⋯,& ETCO2 &,-,⋯,&,( ETCO2 &'(),⋯,&

Original: LSTM min SAO2 &,-,⋯,&,( SAO2 &'(),⋯,&
Revised: LSTM min SAO2 &,-,⋯,&,( SAO2 &'(),⋯,&

Original: Hospital P
Revised: Hospital P

Original: Target Hospital …
Revised: Target Hospital …

Original: LSTM LSTM
Revised: LSTM LSTM

Original: SAO2 &'(),⋯,&
Revised: SAO2 &'(),⋯,&

Original: TV &'(),⋯,&
Revised: TV &'(),⋯,&

Original: Hypoxemia
Revised: Hypoxemia

Original: Min (2 p )
Revised: Min (2 p )

Original: LSTM
Revised: LSTM

Original: LSTM …
Revised: LSTM …

Original: TV &'(),⋯,& TV &'(),⋯,&
Revised: TV &'(),⋯,& TV &'(),⋯,&

Original: LSTM
Revised: LSTM

Original: XGB
Revised: XGB

Original: LSTM SAO2 &'(),⋯,&
Revised: LSTM SAO2 &'(),⋯,&

Original: Hypoxemia
Revised: Hypoxemia

Original: ETCO2 &'(),⋯,& …
Revised: ETCO2 &'(),⋯,& …

Original: LSTM
Revised: LSTM

Original: LSTM
Revised: LSTM

Original: SAO2 &'(),⋯,&
Revised: SAO2 &'(),⋯,&

Original: TV &'(),⋯,&
Revised: TV &'(),⋯,&

Original: Hypoxemia
Revised: Hypoxemia

Original: Hypox (4 p )
Revised: Hypox (4 p )

Original: LSTM
Revised: LSTM

Original: XGB
Revised: XGB

Original: Hospital P
Revised: Hospital P

Original: Target Hospital
Revised: Target Hospital

Original: ETCO2 &,-,⋯,&,( ETCO2 &'(),⋯,&
Revised: ETCO2 &,-,⋯,&,( ETCO2 &'(),⋯,&

Original: LSTM
Revised: LSTM

Original: LSTM …
Revised: LSTM …

Original: TV &'(),⋯,& TV &'(),⋯,&
Revised: TV &'(),⋯,& TV &'(),⋯,&

Original: Target Hospital Target Hospital
Revised: Target Hospital Target Hospital

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 6. I NTERPRETABILITY
Revised: 6. I NTERPRETABILITY

Original: [Figure]
Revised: [Figure]

Original: 6.5. Q UALITATIVE E VALUTION OF I NTERPRETATION FOR S TACKED M ODELS
Revised: 6.5. Q UALITATIVE E VALUTION OF I NTERPRETATION FOR S TACKED M ODELS

Original: In Figure
Revised: In Figure

Original: [Figure]
Revised: [Figure]

Original: we present the local feature attributions for two different model stacks across a random set of examples.
Revised: we present the local feature attributions for two different model stacks across a random set of examples.

Original: We can see that the models  appear togenerally agree on their predictions, although there is more disagreement - which is likely due to the fact that these  twomodels do indeed make different predictions.
Revised: We can see that the models generally appear to agree on their predictions, although there are occasional disagreements - which are likely due to the fact that these attributions are for two different models.

Original: Being able to observe these trends is useful to understanding models and to achieving credibility.
Revised: Being able to observe these trends is useful to understanding models and to achieving credibility.

Original: As our primary aim is to ensure that our model stacking local feature attributions agree with feature attributions on neural networks, we also provide attributions for true positives and true negatives in settings where the models agree.
Revised: As our primary aim is to ensure that our model stacking local feature attributions agree with feature attributions on neural networks, we also provide attributions for true positives and true negatives in settings where the models agree.

Original: In Figure 13, we present local feature attributions for two different model stacks across true positive examples.
Revised: In Figure 13, we present local feature attributions for two different model stacks across true positive examples.

Original: Looking at these true positive examples we can see two consistent trends: high variability and a low absolute value of blood oxygen.
Revised: Looking at these true positive examples we can see two consistent trends: high variability and a low absolute value of blood oxygen.

Original: Looking at the attributions we can discover that the dips in blood oxygen - minute to minute variability was important in both model stacks.
Revised: Looking at the attributions we can discover that the dips in blood oxygen - minute to minute variability was important in both model stacks.

Original: Additionally, the closer the time point is to the actual prediction, the more important it is.
Revised: Additionally, the closer the time point is to the actual prediction, the more important it is.

Original: In Figure 12, we present the local feature attributions for two different model stacks across true negative examples.
Revised: In Figure 14, we present the local feature attributions for two different model stacks across true negative examples.

Original: Here we can see that variability and dips make a much smaller relative impact in the model predictions.
Revised: Here we can see that variability and dips make a much smaller relative impact in the model predictions.

Original: Instead, the most important factor in determining hypoxemia is the high value of SaO 2 closest to the ﬁnal prediction.
Revised: Instead, the most important factor in determining hypoxemia is the high value of SaO 2 closest to the ﬁnal prediction.

Original: Furthermore, the feature attributions reveal interesting trends in the attributions for the MLP model, where there appears to be a consistent trend even though the samples look fairly different.
Revised: Furthermore, the feature attributions reveal interesting trends in the attributions for the MLP model, where there appears to be a consistent trend even though the samples look fairly different.