D EEP S ENSING : A CTIVE S ENSING USING M ULTIDIRECTIONAL R ECURRENT N EURAL N ETWORKS
A BSTRACT
For every prediction we might wish to make, we must decide what to observe (what source of information) and when to observe it.
Because making observations is costly, this decision must trade off the value of information against the cost of observation.
Making observations (sensing) should be an active choice.
To solve the problem of active sensing we develop a novel deep learning architecture:
Deep Sensing .
At training time, Deep Sensing learns how to issue predictions at various cost-performance points.
To do this, it creates a different presentation at each of a variety of different performance levels, each associated with a particular set of measurement rates (costs).
This requires learning how to estimate the value of real measurements vs. inferred measurements, which in turn requires learning how to infer missing (unobserved) measurements.
To infer missing measurements, we develop a Multi-directional Recurrent Neural Network (M-RNN).
An M-RNN differs from a bi-directional RNN in that it sequentially operates across streams in addition to within streams , and because the timing of inputs into the hidden layers is both lagged and advanced.
At runtime, the operator prescribes a performance level or a cost constraint, and Deep Sensing determines what measurements to take and what to infer from those measurements, and then issues predictions.
To demonstrate the power of our method, we apply it to two real-world medical datasets with signiﬁcantly improved performance.
NTRODUCTION
Making observations is costly.
Hence, for every prediction we might wish to make, we must decide what to observe – i.e., what source of information to consult/use – and when to observe it.
This (joint) decision involves a trade-off between the value of the information that will/might be obtained from the observation and the cost of making the observation.
There is little reason to make an observation if the result of that observation can already be conﬁdently estimated on the basis of what is already known or if the result would be of little value in any case; it would be much better to conserve the resources to make a different observation at a different time.
Thus making observations (sensing) should be an active choice (Yu et al.
The problem of active sensing has many applications, from healthcare (the example we use here to illustrate our method) to neuroscience to robotics to wireless communications.
The central point of our approach is that we need to estimate the value of information.
This must be learned at training time.
We learn the estimated value for a speciﬁed set of measurements by ﬁrst predicting on the basis of the information we have, then deleting the speciﬁed set of measurements, inferring what we have deleted on the basis of the data that remains, making a new prediction on the basis of the inferred measurements and the remaining data, and comparing the two predictions.
(Part of our architecture is designed speciﬁcally for these tasks.)
To infer missing data, we develop a novel architecture called a Multi-directional Recurrent Neural
Network (M-RNN).
Like a bi-directional RNN (Bi-RNN) (Graves & Schmidhuber (2005)), an MRNN operates forward and backward in each data stream - in the intra-stream directions .
Unlike a Bi-RNN, an M-RNN also operates across streams – in the inter-stream directions .
And unlike a
Bi-RNN, the timing of inputs into the hidden layers of an M-RNN is lagged in the forward direction and advanced in the backward direction.
(To the best of our knowledge, our architecture is the ﬁrst that operates in this way).
Our M-RNN executes both interpolation (intra-stream) and imputation (inter-stream) to infer missing data.
Because we need to trade off performance against cost, our neural network must learn – at training time – how to issue predictions at various cost-performance points.
To do this, it creates multiple presentations (neural network parameters) at various performance levels associated with different measurement rates (costs).
Each presentation is learned on the basis of a particular set of missing data; these sets are constructed recursively and adaptively.
An important aspect of our solution to the active sensing problem is that there are differences between the operation at training time and runtime.
At training time we can use data at the current time to infer missing data at an earlier time; i.e. we can operate non-causally .
We cannot do so – and do not do so – at runtime.
However, after a new sample is received at runtime, we can and do go back to improve the previous inferences (interpolations and imputations) which will in turn improve both current and future predictions.
To demonstrate the power of our method, we apply it to two real-world medical datasets.
We show that our method yields signiﬁcantly greater predictive power (measured as the Area Under the ROC
Curve (AUC)) per unit cost in comparison to other state-of-the-art methods.
Because our inference methods are of interest in themselves, we compare the root mean squared error (RMSE) and corresponding AUC for our method to that of state-of-the-art imputation methods in statistics such as
White et al.
; Rehfeld et al.
; Garc ´ ıa-Laencina et al. (2010), RNN-based imputation methods such as Choi et al.
; Lipton et al.
; Che et al. (2016); Futoma et al.
(2017), and interpolation methods such as Kreindler & Lumsden (2012)
; Mondal & Percival (2010).
In all cases, we demonstrate large and signiﬁcant improvements.
R ELATED W ORKS
Previous works related to Deep Sensing fall into four areas: Active sensing, missing value inference,
Bayesian optimization and RNN methods.
(Active sensing is related to both active learning and to reinforcement learning, but actually rather different from both of them; see the discussion below.)
Active sensing As discussed in the Introduction, the focus of active sensing (Yu et al.
(2009); Alaa & van der Schaar (2016))
and of screening policies (Ahuja et al.
(2017)) is to determine what and when to measure; this is an important question whenever acquiring measurements is costly.
Yu et al.
(2009) studies the problem of active sensing using a Bayesian approach with Gaussian processes.
This work models the data streams as Gaussian processes, so if the number of data streams is D then the number of parameters is of order D 2 and estimation accuracy decreases dramatically as D grows.
Moreover, this work creates only a single presentation for the entire data set and hence does not effectively trade off predictive gain against measurement cost, and cannot deal with a setting in which there are different costs to sample different variables.
Alaa & van der Schaar (2016) addresses the problem of active sensing for a single data stream.
That work assumes a speciﬁc stochastic process to learn the optimal time to sample the next measurement, given the characteristics of the speciﬁc stochastic process.
Because this work treats only a single data stream and imposes a particular model of the data, this approach cannot be applied to a general data stream and is ineffective in active sensing across multiple data streams.
Ahuja et al.
(2017) proposes a methodology for personalized screening in the medical domain but the procedure for learning presentations is independent of the screening policy.
A particular approach to active sensing ( submodular optimization ; see Iwata et al.
; Schrijver (2000) for example) minimizes a submodular objective function as a proxy for minimizing the true ”cost - information gain”.
Deep Sensing does not use a submodular objective function – or any other particular objective function – as a proxy; instead, Deep Sensing uses a greedy algorithm to ﬁnd the individual measurements that yield a positive ”information gain - cost”, and uses the set of all such individual measurements as the set of measurements that should be performed.
(If this set is empty, Deep Sensing moves on to the next possible measurement date, and so forth.)
The details are discussed in Section 3.
Missing value inference: There are two standard methods to deal with missing information in timeseries data streams: interpolation and imputation.
Interpolation methods (Kreindler & Lumsden (2012); Mondal & Percival (2010)) attempt to capture the temporal relationships within each data stream but not the relationships across streams.
Imputation methods (White et al.
; Rehfeld et al.
(2011); Garc´ıa-Laencina et al.
(2010)) attempt to capture the synchronous relationships across data streams but not the temporal relationships within streams.
(Most of this work assumes a speciﬁc model of the data, rather than learning a presentation from the data, as Deep Sensing does.)
We are not aware of any previous work that attempts to capture both the relationships within stream and the relationships across streams .
Bayesian optimization: The problem of costly measurements has been studied in other areas as well.
Bayesian optimization (Pelikan et al.
; Snoek et al.
(2012)) uses a Gaussian process regression (GPR) to approximate the loss function for a given optimization problem (Seo et al.
This approximation is then used to sequentially evaluate the true loss function at points where the expected decrease in loss is the greatest.
When function evaluations are computationally costly (e.g. for hyper-parameter optimization in complex problems), this approach is a way of identifying good minima given constraints on time.
There are signiﬁcant differences between Bayesian optimization and Deep Sensing.
Firstly, in
Bayesian optimization, “cost” is usually taken to be computation time, and optimization is performed subject to a cost constraint – a maximum number of permissible evaluations.
In traditional settings, this cost is essentially treated as constant, and not explicitly considered in selecting points during the optimization procedure.
Deep Sensing, on the other hand, trades off cost against (predictive) gain.
Secondly, in the active sensing setting we consider, measurements can be taken only forwards in time; in the setting of Bayesian optimization, no restrictions are placed on the location of function evaluations.
Active sensing thus captures the problems in the healthcare setting, in which causal predictions are needed to inform the actions of practitioners in a timely fashion.
Finally, Bayesian optimization uses GPR to approximate loss functions, which places limitations on the types of functions which it can mimic.
Because neural networks are ”universal approximators” (Hornik et al.
, the RNNs used in Deep Sensing allow it to model a richer set of functions (and give rise to more complicated and interesting dynamics).
RNN methods: RNNs have been used successfully for prediction on the basis of time-series data with missing data and irregular sampling.
The approach of Gingras & Bengio (1996) is to ﬁrst replace all the missing information with a mean value and use the feedback loop from the hidden states to update the imputed value while learning the classiﬁcation problem using a standard RNN.
Tresp & Briegel (1998) uses the Expectation-Maximization (EM) algorithm to impute the missing values and uses the reconstructed data streams as inputs to a standard RNN for prediction.
As with standard imputation methods, the imputation depends only on the synchronous relationships across data streams and not on the temporal relationships within streams.
Parveen & Green (2002) use a linear model to estimate missing values from the latest measurement and the hidden state of each stream.
As with standard interpolation methods, the estimate depends only on the temporal relationships within each stream and not on the relationships across streams.
More recent works address both missing values and irregularly sampled time-series data streams (Choi et al.
; Lipton et al.
; Che et al. (2016); Kim et al.
These papers use the sampling times to capture the informative missingness and time interval information to deal with irregular sampling.
They do this by concatenating the measurements, sampling information and time intervals and using the concatenation as the input of an RNN.
These papers differ in the replacements they use for missing values.
Choi et al.
; Lipton et al.
; Kim et al.
(2017) replace the missing values with 0, mean values or latest measurements – all of which are independent of either the intra-stream or inter-stream relationships or both.
Therefore, those methods cannot be extended to our active sensing algorithm.
Che et al.
(2016) imputes the missing values using only the latest measurements, the mean value of each stream, and the time interval.
It is not bi-directional and so cannot use information available at a given time to update estimates of information that is missing at an earlier time.
Futoma et al.
(2017) assume a Gaussian Process in order to learn the latent variables from irregularly sampled longitudinal datasets, and use the outputs of this Gaussian Process as the inputs of an RNN to deal with the irregular sampling of the dataset.
Active learning and reinforcement learning Active learning (e.g. MacKay (1992); Seung et al.
and reinforcement learning both have something in common with active sensing, in that they all have to do with the (possibly costly) acquisition of information.
However active learning focuses on the acquisition of labels , while active sensing focuses on the (costly) acquisition of measurements .
And reinforcement learning focuses on actions (which directly affect the state), while active sensing focuses on observations (which do not affect the state).
B ACKGROUND 3. N OTATION
The training set consists of N arrays of data.
It is convenient to use the language of healthcare and to speak of the array n as the information of patient n , so that there are N patients in the training set.
For each patient n , we have a multivariate time-series data stream of length T (the length T and the other components may depend on the patient n but for the moment we suppress the dependence on n ) that consists of three components: measurements X , labels Y and time stamps S .
Because measurements are not necessarily made at regular intervals, we distinguish between time stamps and actual times.
The time stamp t = 1 , 2 , . . .
simply indexes the sequence of times at which measurements were taken; s t is the actual time at which the measurements x t were taken and the label y t was realized.
For convenience we normalize so that s 1 = 0 ; we assume actual times are strictly increasing: s t +1 > s t for 0 < t ≤ T − 1 .
The label y t represents the outcome realized for patient n at time stamp t (actual time s t ).
Labels may be discrete or continuous.
In the former case we are considering a classiﬁcation problem (e.g. prediction of an event, such as discharge, clinical deterioration, death); in the latter case, we are considering a regression problem (e.g. prediction of value or family of values).
If we are interested explicitly in the estimation of missing data for its own sake, then y t would represent the actual observed data at time stamp t .
Y is the vector of outcomes for this patient.
We normalize so that labels and predictions lie in [0 , 1] .
There are D streams of measurements; each measurement is a real number, but not all measurements may be observed at each time stamp.
Hence we view the set of possible measurements at time stamp t as R ∗ =
R ∪ {∗} .
We interpret x dt = ∗ to mean that the stream d was not measured at time stamp t ; otherwise x dt ∈ R is the measurement of stream d at time stamp t .
X is the array of measurements of all streams at all time stamps for the patient under consideration.
It is convenient to introduce some notation to keep track of what is measured/not measured (observed/not observed).
For each time stamp t and stream d , write m dt =
if x dt = ∗ (not measured) and m dt = 1 if x dt ∈ R (measured).
Let δ dt be the actual amount of time that has elapsed since the stream d was measured last.
δ dt can be deﬁned recursively as follows:
[Equation]
where δ d 1 =
Write δ t for the vector of elapsed times at time stamp t and ∆ =
{ δ 1 , δ 2 , ..., δ T } .
The information available for patient n is the triple ( X n , Y n , S n )
The entire training set therefore is the sets of triples D
= { ( X n , Y n , S n ) } Nn =1 .
We use functional notation to identify information about each patient, so x dt ( n ) is the measurement of stream d at time stamp t for patient n , etc.
3. P ROBLEM F ORMULATION
At time stamp T , we have an array of measurements (which may or may not include the current label y T ); we must decide the next time s T +1 at which to take new measurements and what measurements to conduct at that time.
We measure the information provided by new samples by the effect on the label y T
+1 , so we deﬁne the predictive loss from not sampling as the increase in uncertainty of our prediction of y T +1 and the predictive gain as the decrease in uncertainty of our prediction of y T +1 .
Our approach is to ﬁnd the ﬁrst actual time τ at which the (estimated) predictive gain provided by new samples exceeds the cost of sampling (keeping in mind that the cost of sampling may be different for different streams), and to make the set of measurements at time τ that maximizes the (estimated) predictive gain minus the cost.
Our objective is to ﬁnd the set of measurements that maximize (net) rewards, which we take as ”information gain - cost”; this formulation is common; see Stachniss et al.
(2005); Visser & Slamet (2008)
for instance.
Somewhat more formally, our objective is to solve the maximization problem:
[Equation]
where M is the set of possible measurements.
(For convenience, we assume here that the set of possible measurements is the same at every time but there would be no difﬁculty in allowing for the set of possible measurements to be different at different times.
Note that the term ”measurement” could actually encompass a panel of tests that can be made at the same cost.)
Solving this maximization problem presents two immediate problems.
The ﬁrst problem is that cost is well-deﬁned and observable in our setting, but we need to decide the appropriate measure of information gain.
Information gain is often deﬁned Stachniss et al.
F¨ollmer (1973) as the decrease in entropy.
However, to properly compute the entropy, we should know the distribution of predictions.
Instead, we use the decrease in uncertainty of prediction – measured as the difference between the upper and the lower bounds of the prediction – as our measure of information gain.
The second problem is that maximizing over all possible subsets of measurements presents a potentially intractable problem; instead, we take a greedy approach that yields an approximation to the true optimum.
We discuss both of these issues below.
The actual error in sampling stream d at s T +1 =
τ is the difference between the estimated values and the actual measurement e dT +1 =
| ˆ x dT +1 − x dT +1 | .
We don’t know the actual error so we must construct an estimated error ˆ e dT +1 .
Assuming that the distribution of errors is approximately normal (an assumption that is justiﬁed in the Appendix), the conﬁdence intervals in the measurement of x dT +1 are of the form CI x =
(ˆ x dT +1 − λ ˆ e dT +1 , ˆ x dT +
+ λ ˆ
e dT +1 ) ; e.g. λ = 1 .
96 for the 95% conﬁdence level Rothenberg (1984); Davison & Hinkley (1997); Efron & Tibshirani (1986);
Bartlett (1953).
Note that the conﬁdence intervals depend only on the estimates and not the true values (which are of course unknown).
Each vector of estimates (ˆ x dT +1 ) of measurements, together with previous data (measured and inferred), can be used to produce a prediction ˆ y dT +1 (see below).
The conﬁdence intervals for the stream measurements translate immediately into lower and upper conﬁdence estimates ˆ y d,lT +1 , ˆ y d,uT +1 (respectively) for the label prediction:
[Equation]
where X T and S T are previous measurements and measurement times until time stamp T .
The (estimated) predictive gain in stream d is therefore the difference ˆ y d,uT +1 − ˆ y d,lT +1 .
Note that, because the minimization and maximization problems for each feature are independent each other, equation (1) can be solved by one-dimensional gradient descent.
Having deﬁned gain, we now deﬁne (estimated) predictive gain minus cost as F ( C T +1 , X T , S T ) .
At each time s T +1 , we seek to ﬁnd the subset C T +1 ⊂ M of measurements that maximizes predictive gain minus cost; i.e. we wish to solve:
[Equation]
However, if the number of possible measurements is large (which is typical), and there are complementarities among measurements, then solving the optimization problem (2) requires examining all possible subsets of measurements – which is intractable.
Instead, we follow a greedy procedure: we identify all the individual streams d with the property that the value of measuring that stream (by itself) exceeds the cost c
d of sampling from that stream; we then take C ∗ T +1 to be that set of measurements.
This is a tractable optimization problem that yields an approximation to the actual optimal set of measurements.
(We note again that every set of tests that can be carried out as a single panel at the same cost can be considered as a single test.)
Hence we will actually solve the problem:
[Equation]
𝑻 B l oo d
P re ss u re 𝒕 𝑻 H e a r t r a t e 𝒕 𝑻 M o r t a li t y R
i s k 𝒕 𝑻 B
l oo d
P re ss u re 𝑻 H e a r t r a t e
MeasuredImputed
Risk Prediction
Active Sensing 𝒕+ 𝒕+
Prediction Block Interpolation
[Figure]
Imputation Block
[Figure]
Sensing
[Figure] Estimate without sensing It is important to note that C ∗ T +1 might be empty; i.e., there might be no measurements for which the information gain exceeds the cost. Because of this, Deep Sensing answers both the question ”when to sample?”
as well as the question ”what to sample?”
At each time T , Deep Sensing asks whether there are any measurements to be made at time T +1 for which the beneﬁt outweighs the cost.
If the answer is ”yes” (i.e. C ∗ T +1 (cid:54) = ∅ )
then Deep Sensing recommends that those measurements should be made at time T + 1 .
If the answer is ”no” (i.e. C ∗ T +1 = ∅ )
then Deep Sensing asks whether there are any measurements to be made at time T
+ 2 for which the beneﬁt outweighs the cost, and so forth.
Predicting Labels: Given data (measured and inferred) until any time stamp T (cid:48) , we generate a prediction ˆ y T (cid:48) .
The prediction rule can be learned from training data by any of various machine learning algorithms; we use a standard GRU-based RNN (Chung et al.
(See Prediction in
Section 4.)
Estimating the Values of New Measurements: We view the problem of estimating new measurements as a special case of estimating missing measurements, so we begin by discussing our novel methods for this problem.
Fix data D through time stamp T .
Assume that x dt = ∗ .
There are two standard methods to form an estimate ˆ x dt : interpolation and imputation.
Interpolation uses only the measurements x dt (cid:48) of the ﬁxed data stream d for other time stamps t (cid:48)
(perhaps both before and after t ).
Interpolation ignores the correlation with other data streams.
Imputation uses only the measurements x d (cid:48) t at the ﬁxed time t for other data streams d (cid:48) .
Imputation ignores the correlation with other times.
In principle, we could try to form the estimate ˆ x dt by using all the information in D .
However, this would require learning a vast number of parameters and hence a vast number of training instances, so this is impractical.
Instead, we propose an efﬁcient hierarchical learning framework using a novel
RNN architecture that allows us to capture the correlations both within streams and across streams .
The entire process of Deep Sensing is illustrated in Fig.
D EEP S ENSING : A LGORITHM
In this section, we describe the training and runtime stages of the Deep Sensing algorithm.
Fig.shows block diagrams of the two stages.
4. T RAINING S TAGE
The training stage of Fig.
2 shows the block diagram of the entire 5 blocks of the training stage.
The ﬁrst four blocks train the Interpolation block ( Φ ), the Imputation block ( Ψ ), the Error Estimation block ( Γ ) and the Prediction block ( Ω ).
The Adaptive Sampling block creates multiple presentations based on different sets of missing data.
(The importance of this will be explained below.)
Loss function: The objective of the interpolation and imputation blocks is to minimize the error that would be made in estimating missing measurements.
Evidently, we cannot estimate the error of [Figure] a measurement that is truly missing in the dataset.
Instead we ﬁx a measurement that was actually made, remove that measurement, form an estimate for the measurement using only the data set
D − x dt (i.e. the data set with x dt removed), and then compute the error between the estimate and the actual measurement (that was deleted).
If x dt is an actual measurement and ˆ x dt is the estimate formed when x dt is removed then the loss can be deﬁned as the mean squared error (MSE) l (ˆ x dt , x dt ) =
(ˆ x dt − x dt ) 2 .
The loss for the entire dataset D is deﬁned as
L ( { ˆ x dt , x dt } ) =
N (cid:88) n = (cid:34)(cid:80) T n
[Equation]
Interpolation: The objective of the interpolation block is to construct an interpolation function Φ that operates within a stream.
To emphasize that the estimate for x dt depends on the data with x dt removed, we abuse notation and write ˜ x dt =
Φ ( D − x dt ) .
(Keep in mind that we are actually only using the data from stream d , not the data from other streams.)
We construct the estimation function Φ using a bi-directional recurrent neural network (Bi-RNN) with a Gated Recurrent Unit (GRU).
However, unlike a conventional Bi-RNN (Graves & Schmidhuber (2005)) [Equation] the timing of inputs into the hidden layer is lagged in the forward direction and advanced in the backward direction: at time t , inputs of forward hidden states come from t − 1 and inputs of backward hidden states come from t + 1 .
Mathematically, we have:
[Equation]
[Equation]
where (cid:12) is element-wise multiplication, σ is the sigmoid function, φ is tanh function, and arrows indicate forward/backward direction.
The output o t is the interpolated value ˜ x t .
In this interpolation block, we are only using/capturing the temporal correlation within each stream.
As a consequence, the matrices U, V, W are block-diagonal.
Hence the total number of parameters that must be learned is on the order of the number D of streams.
(Recall that in a standard Bi-RNN, the number of parameters to be learned will be on the order of the square D 2 of the number of streams.)
This
(cid:1872)− (cid:1872) (cid:1872)+
FC Layers
FC Layers FC Layers
Outputs
Inputs (cid:1872)− (cid:1872) (cid:1872)+
Outputs (cid:1856)− (cid:1856) (cid:1856)+ (cid:1856)− (cid:1856) (cid:1856)+
Inputs
Bi-RNN Bi-RNN
Bi-RNN (cid:1872) (cid:1856)
I n t e r po l a ti on
I m pu t a ti on
I n t e r po l a ti on
I m pu t a ti on
Inputs
I n t e r po l a ti on
I m pu t a ti on (cid:1872) (cid:1856) ((cid:1853))
((cid:1854))
((cid:1855))
Outputs avoids overﬁtting and leads to signiﬁcant performance improvements as compared to a standard
Bi-RNN.
(See the Interpolation part of Fig. 3.)
Imputation: The objective of the imputation block is to construct an imputation function Ψ that operates across streams.
Again, we abuse notation and write ˇ x dt = Ψ
( D − x dt ) . Keep in mind that now we are using only data at time stamp t , not data from other time stamps.
We construct the function Ψ to be independent of the time stamp t ; so we construct it using fully connected layers (FC); see Imputation part of Fig 3:
[Equation]
where o t = ˆ x t and the block-diagonal entries of U are zero because we do not use x dt to estimate ˆ x dt .
We use multiple deeply stacked FC layers using linear activation functions.
We jointly learn the functions Φ and Ψ using the stacked networks of Bi-RNN and FC layers.
[Equation]
We refer to the entire structure as a Multi-directional Recurrent Neural Network (M-RNN) ; see Fig.3.
Prediction: Now that we have a procedure to reconstruct (interpolate/impute) missing data, we use the reconstructed data to predict the labels.
We accomplish this in the prediction block.
Note that for prediction, we use actual measurements when available and estimated measurements when actual measurements are missing (not available).
We also use as an input the mask vector (the indicator of missingness), which provides to the network the information as to whether measurements are actual or estimated.
Once again, we construct the function to minimize the prediction error when we predict an observed label.
The loss function is deﬁned as L ( { ˆ y t , y t } )
= 1 N (cid:80) Nn =
[Equation]
(cid:80) Tnt = [Equation] ˆ
[Equation]
n
[Equation]
y
[Equation]
n
[Equation]
2 T n
The prediction block optimizes the function:
[Equation]
Note that we use the time intervals δ dt as inputs to the prediction function in order to deal with the fact that the data streams are irregularly sampled.
This optimization problem is a standard problem for timely prediction so we can use a standard GRU-based RNN (Chung et al.
(2014)) to solve it.
Error Estimation:
At runtime, we have to decide when/what to sample in the active sensing block.
We make this decision on the basis of predictive gain which is determined by the difference between our estimate of a measurement and what the actual value of the measurement would be; the actual error e dt = | ˆ x dt − x dt | .
Of course, we do not know what this will be because we do not know what the actual value of the measurement would be.
Hence we need an estimate ˆ e dt of the actual error.
As before, we construct this estimate on the basis of the actual training data that we have.
For tractability, we posit that this estimate depends on the pattern of missing data and on time intervals of the measurements but not on actual measurements.
We use the same mean square loss function:
[Equation]
Hence we need to solve for the function
[Equation]
Because this involves both inter-stream and intra-stream variables, we again use our M-RNN structure.
However, the inputs and outputs are different: for the interpolation and imputation blocks, the inputs are the measurements { x dτ } , sensing information { m dτ } and the time intervals { δ dt } and the output is the estimated measurement ˆ x d (cid:48) t .
For the error estimation block, the inputs are the sensing information { m dτ } and the time intervals { δ dt } and the output is the estimation error
ˆ e dt .
It is useful to understand the relationship between the mask vector (which indicates missing measurements) and the three different categories of missingness of measurements.
(1) If the measurements in the dataset are Missing Completely At Random (MCAR) then the mask vector is independent of the observable features/measurements.
(2) If the measurements in the dataset are Missing At
Random (MAR) then the correlation between the mask vector can be completely explained by the observable features/measurements.
(3) If the measurements in the dataset are Missing Not At Random (MNAR) then the mask vector (and the values of the missing features) cannot be completely explained by the observable features/measurements.
We have focused on the MCAR and MAR settings because the values of missing measurements are estimated based on the observable features.
However, our approach also has something useful to say in the MNAR setting as well, because we use the mask vector – which depends on both observed and unobserved variables and therefore incorporates ”informative missingness” – as an input for both estimation of missing values and for prediction.
We demonstrate this point in Section 5.4.
Adaptive Sampling:
As pointed out in Section 3, the decision of what/when to sense arises from trading off the cost of measurement against the predictive gain of measurement.
To this point, we have constructed a procedure that achieves a certain performance – predictive gain – at a prescribed cost.
If we are willing to settle for a lower level of performance, we can do so at lower cost by sampling less often.
To know how much less often to sample we need to know how much information would be lost if we sampled less often, which we can determine by carrying out the previous procedure to produce different presentations, each based on a different pattern of missing data.
For each presentation we need to train the functions Φ , Ψ , Ω , Γ on the appropriate training set, which is smaller than the original training set.
To construct these presentations, we begin with the original training set and remove additional measurements.
We should not do this at random, but rather using the informational criteria we use to decide on active sensing at runtime: remove measurements whose predictive gain is below a given threshold.
We call this adaptive sampling .
This will yield a decreasing sequence of data sets
D 0 ⊃ D 1 ⊃ ... ⊃ D R
(where D
= D , the original dataset).
The training procedures for the functions Φ , Ψ , Ω , Γ are as follows.
Fix thresholds u 1 , . . .
, u R > 0 .
(In practice, these would be speciﬁed by the user.)
We begin with D 0 .
For each measurement x dt ( n ) ∈ D
we use the current functions Φ = Φ 0 , Ψ
= Ψ 0 , Ω = Ω 0 , Γ =
Γ 0 to compute the predictive gain from that measurement in the current dataset.
We sequentially delete all the measurements whose predictive gain (”information gain - cost”) is below the prescribed threshold u 1 ; this yields a resampled data set D 1 .
We now retrain on D 1 to obtain new functions Φ 1 , Ψ 1 , Ω 1 , Γ 1 and repeat the same procedure: for each measurement x dt ( n ) ∈ D 1 , we compute the predictive gain from that measurement in the current dataset and sequentially delete those measurements whose predictive gain is below threshold u 2 , etc.
We repeat the above procedures continuing through whatever set of thresholds are chosen.
(It is important to keep in mind that, in the active sampling process, if the actual dataset is not complete, we only consider measurements that are actually recorded in the dataset.
Thus we are never confronted with the need to compare an estimate/prediction against data that is actually missing.)
Note that if we increase the threshold u r , we delete more data and retain fewer samples, so the expenditure on sampling is smaller.
However because we have trained on fewer samples, our predictions will be less accurate.
This creates the cost-performance trade-off.
Fig.
2 illustrates the entire block diagram of Deep Sensing.
Fig 5 in the Appendix details the operation of Deep Sensing in runtime.
Pseudo-codes of Deep Sensing for the training and runtime stages are shown in the
Appendix.
E XPERIMENTS
In this section, we evaluate the performance of Deep Sensing using two real-world medical datasets.
Our experimental results present three sets of comparisons: active sensing, prediction, and missing value inference.
The ﬁrst comparison shows the performance gain of Deep Sensing (in comparison with benchmarks) in sensing the critical measurements for prediction.
The second and third comparisons show the performance gain of the M-RNN algorithm in estimating missing values and the effect on prediction accuracy.
We describe all conﬁgurations of the various algorithms in the
Appendix.
5. D ATA D ESCRIPTION
We conducted our experiments using two real-world medical datasets.
The ﬁrst of these datasets is
MIMIC-III (Johnson et al.
(2016)) which records data on patients in intensive care units (ICU).
We used 22,803 patients who admitted were to ICU after 2008.
We use the 20 vital signs which were most frequently measured and for which missing rates are lowest (e.g. heart rate, respiratory rate) and 20 lab tests (e.g. creatinine, chloride).
Thus we have 40 physiological data streams in all.
Vital signs were taken approximately every 1 hour; lab tests were taken approximately every 24 hours.
For this dataset, the adverse event we predict is death.
The second of these datasets, which we call
Wards, was assembled and described by Alaa et al.
(2017b).
(We are grateful to the authors for sharing this dataset with us.)
The Wards dataset records 37 physiological data streams (vital signs and lab tests) on 6,094 patients who were hospitalized in a major medical center in 2013-2015.
Vital signs were taken approximately every 4 hours; lab tests were taken approximately every 24 hours.
For this dataset, the adverse event we predict is admission to ICU as a result of clinical deterioration.
5. S IMULATION S ETUP
We randomly divided the dataset into a mutually exclusive training set (80%) and testing set (20%).
We conducted 10 independent experiments with different combinations of training/testing sets; we report the mean and variance of the performance in the 10 experiments.
In our experiments we are trying to predict which patients will experience an adverse event (death for the MIMIC-III dataset and ICU admission for the Wards dataset) within 24 hours from the current time.
At each time, we assign the label 1 to patients who experienced the relevant adverse event within 24 hours; for other patients we assigned the label 0.
(Formally: y t = 1 for s t > S T − 24 and y t = 0 for s t ≤ S T −where S T is the time that the adverse event occurred.
Of course the true label y t is not observed at time stamp s t .)
Active sensing:
To evaluate the performance of Deep Sensing, we graph predictive accuracy – area under the ROC curve (AUC) – against cost.
(The cost of each possible measurement is well-deﬁned in the medical domain.
If all measurements were equally costly, we could simply identify the cost with the observation rate.
Because some measurements are most costly than others, we simply weight those measurements more heavily when expressing the cost in terms of the observation rate.
In this case, we take the cost for lab tests to be 5 times the cost for taking vital signs so weight lab tests accordingly.)
We compare the predictive accuracy of Deep Sensing with multiple presentations,
Deep Sensing with a single presentation (using only the original dataset to train), Deep Sensing with [Figure] random sampling, and two benchmarks based on the method of Che et al.
(2016) for prediction with missing data (sampling either using the method of Deep Sensing or randomly).
Prediction: We also evaluate prediction given only the available observations.
The prediction we consider is the adverse event (death or clinical deterioration); we use AUC as the performance metric.
We compare the performance of Deep Sensing with four state-of-the-art RNN timely-prediction models and a GRU-based RNN method using conventional estimation methods for interpolation and imputation.
To make the comparison fair, we use GRU-based RNNs for each benchmark.
Deep
Sensing is compared with the benchmarks in two settings.
In setting A, we sampled 60% of the measurements; for Deep Sensing, we used the Deep Sensing algorithm, for the benchmarks, we use random sampling.
In setting B we sampled 60% of the measurements, using the Deep Sensing algorithm everywhere.
Estimation of missing values: To evaluate the performance of the M-RNN algorithm (the combination of interpolation and imputation block) in estimating missing values, we compare with other standard methods: interpolations (Spline and Cubic Kreindler & Lumsden (2012)), imputations (MICE (White et al.
(2011)), Kernel (Rehfeld et al.
and EM (Garc ´ ıa-Laencina et al.
We randomly remove 30% of the observations and treat them as missing.
We then estimate the missing observations values using our M-RNN algorithm and benchmarks.
We use the root mean square error (RMSE) between estimated values and actual observed values as the performance metric.
Runtime of Deep Sensing:
Deep Sensing is computationally efﬁcient.
For instance, on the MIMICIII dataset (23,200 samples, 40 dimensions, 25 time stamps), Deep Sensing takes less than 1 hour on a machine with i7-6900K CPU (3.2GHz x 16) and 64GB RAM.
By comparison, the most common imputation method, MICE (implemented in R packages) takes 11 hours on the same machine.
5. S IMULATION R ESULTS AND D ISCUSSION
Active Sensing:
As Fig. 4 illustrates, Deep Sensing predicts best for every speciﬁcation of cost; equivalently, Deep Sensing expends the least cost for every speciﬁed prediction accuracy.
Fig.also shows that the performance gains achieved by Deep Sensing come both from active sampling and from better inference.
As seen in Figure 4, if the observation rate were 100% (so there would be no gain from active sensing), the AUC improvement would be limited.
However, as the observation rate decreases the AUC gain increases because Deep Sensing actively decides what to sample and when to sample, thereby providing results that are much superior to random sampling.
Prediction: Tables 1 and 2 provide the mean, standard deviation, and performance gain (%) (in terms of AUC) from Deep Sensing in comparison to the benchmarks for two real-world medical datasets.
Table 1 and 2 show that Deep Sensing provides signiﬁcant gains of the prediction accuracy for both datasets (around 30% in Setting A and 20% in Setting B for all the benchmarks).
The signiﬁcant gains for prediction come from the combination of accurate missing value inference and active sensing as seen in Figure 4. [Table]
for descriptions of Settings A, B).
*: p-value < 0. [Table] Estimation of missing values: Table 3 shows the mean and standard deviation of the RMSE for
M-RNN and benchmarks for both the MIMIC-III and the Wards dataset.
The RMSE of M-RNN is of the best benchmark in MIMIC-III dataset and less than 70% that of the best benchmark in Wards dataset.
All the improvements are statistically signiﬁcant (p-value < 0.05).
[Table]
5. S OURCE OF GAINS
Our M-RNN architecture consists of two components: the interpolation block (with forward/backward connection) and the imputation block).
To understand the source of gains provided by the various components of our approach, we carry out a series of experiments in which we restrict the operation of our architecture in various ways.
In the ﬁrst experiment, we restrict to interpolation only (no imputation), in the second experiment, we restrict to imputation only (no interpolation), in the third experiment we restrict to forward interpolation only (no backward interpolation), and in the fourth experiment we replace the GRU based RNN with a standard RNN.
Table 4 provides the results of these experiments.
As can be seen, both the interpolation and imputation blocks by themselves provide signiﬁcant performance improvements.
(Because the sampling frequencies are high in both datasets, the performance gain of the interpolation block is higher than that of the imputation block.)
The backward connection also improves performance, but only marginally (approximately 10%).
Finally, using the GRU-based RNN signiﬁcantly improves performance by capturing longterm dependencies in an efﬁcient way.
As we have discussed above, and as is illustrated in Section 4, Deep Sensing can be applied if data is missing completely at random (MCAR) or missing at random (MAR), but it can also be applied if data is missing not at random (MNAR).
1 To apply Deep Sensing in the MNAR setting, we incorporate the mask vector (the indicator of the missingness) as an additional input to capture ”informative missingnes”.
As can be seen in the last row in Table 4, doing so leads to a signiﬁcant improvement – approximately 30%.
[Table]
C ONCLUSION
The problem of active sensing is a very important one but has not been thoroughly treated in the literature.
We present here a solution based on a novel deep learning architecture.
As part of the solution, we provide a new method for reconstructing missing data that exploits joint interpolation within data streams and imputation across data streams.
We demonstrate that Deep Sensing makes large and statistically signiﬁcant improvements in comparison with state-of-the-art benchmarks in two real-world datasets. A CKNOWLEDGMENTS
This work was supported by the Ofﬁce of Naval Research (ONR) and the NSF (Grant number:
ECCS1462245, ECCS1533983, and ECCS1407712).
A PPENDIX J USTIFICATION OF THE APPROXIMATE CONFIDENCE INTERVAL IN S ECTION
Deﬁne ˆ x = x + n where x is a measurement and ˆ x is the estimated measurement.
For the moment, assume that n is Gaussian noise and that we can interpret the error e
= | ˆ x − x | as the estimated standard deviation of the Gaussian noise n .
Then, (ˆ x − λe, ˆ x + λe ) is the proper conﬁdence interval for x in the formal statistical sense.
The assumption of Gaussian noise is quite standard and probably needs no further comment.
The interpretation of the error ( | ˆ x − x | ) as the estimated standard deviation of Gaussian noise is not standard but can be justiﬁed in the following way.
Let us assume that our measurement x comes from a Gaussian distribution N ( µ, σ 2 ) .
If our estimate ˆ x is the expected value of x (i.e. µ ), then we will have x = ˆ x + n , where x is the observed measurement from a Gaussian distribution, ˆ x is the expected value of x and n is normal (Gaussian) distribution N (0 , σ 2 ) .
In that case, the expected value of e = | ˆ x − x | = | n | is just the standard deviation of Gaussian noise, which is (cid:112)
E [ n 2 ] = σ .
Hence, we need two assumptions: (1) our estimate ˆ x is the expected value of x ; (2) the observed measurement can be approximated as the sum of the expectation of x and Gaussian noise (approximate normality [Rothenberg (1984); Davison & Hinkley (1997); Efron & Tibshirani (1986); Bartlett (1953)]).
More formally, assume that the measurement x is sampled from an unknown distribution P θ ; i.e. x ∼ P θ .
If P θ is itself normal (Gaussian), then it follows that.
[Equation]
where n ∼ N (0 , σ 2 ) .
(This uses the observation that the expectation of the normal distribution is the mean).
In general, we cannot assume that P θ is normal, but it will be enough if it is approximately normal, which is a common assumption in the literature (see [Rothenberg (1984); Davison & Hinkley (1997); Efron & Tibshirani (1986); Bartlett (1953)] for instance.)
In that case, following the literature we can obtain
[Equation]
where n ∼ N (0 , E [( x − E [ x ]) 2 ]) .
From this we obtain that ˆ x = E
[ x ] .
(In practice, we use ˆ x as the sample mean of x which converges to E [ x ] ).
In that case the distribution of the error e = | ˆ xx | coincides with the distribution of the absolute value of samples generated by the normally distributed noise n :
[Equation]
Thus, estimating e can be interpreted as estimating the standard deviation of the measurements x .
HE OPERATION OF D EEP S ENSING IN RUNTIME
[Figure]
T x x x x x x
T+
[Figure]
(cid:2176)(cid:2878)(cid:2778)
[Figure]
Zoom in
Input: Dataset D = { ( X n , Y n , S n ) } Nn =1 , multiple representations U = { u 1 , u 2 , ..., u R }
For each representation u r ∈ U
Initialization: Φ 1 , Ψ 1 , Ω 1 , Γ 1 ← Xavier Initialization, D 1 ← D Φ r ← Φ r − 1 , Ψ r ← Ψ r − 1 , Ω r ← Ω r − 1 , Γ r ← Γ r − 1 and D r ← D r − 1 Interpolation and Imputation: Using M-RNN and FC layers Φ ∗ r , Ψ ∗ r = arg min Φ , Ψ (cid:34)
L ( { Φ (cid:16) { x dτ , Ψ (cid:16) { x dτ , m dτ , δ dτ } τ =1: T (cid:17) , m dτ } d =1: D (cid:17) , x dτ } ) (cid:35)
Error Estimation: Using M-RNN and FC layers Γ ∗ r = arg min Γ (cid:34)
L ( { Γ (cid:16) { m dτ [Equation] d =1: D , { δ dτ , m dτ } τ =1: T (cid:17) , e dτ } ) (cid:35)
Prediction: Using RNN with GRU Ω ∗ r = arg min Ω (cid:34)
L ( { Ω (cid:16) { x dτ , ˆ x dτ , δ dτ } τ =1: T,d =1: D (cid:17) , y τ } ) (cid:35)
Adaptive Sampling:
For all τ, s, n
If ˆ y u,lτ ( n ) − ˆ y d,lτ ( n ) <
u r
D r ← D r − x sτ ( n )
End
If
End While
End For
Output: Four trained functions { Φ r , Ψ r , Ω r , Γ r } r =1: R =
Input: cost C
=
{ c 1 , c 2 , ..., c D } , corresponding trained functions Φ r , Ψ r , Ω r , Γ r , and λ
For t ∈ { 1 , 2 , ..., T }
Interpolation: ˜ x dt = Φ (cid:16) { x dτ , m dτ , δ dτ } τ =1: t − 1 (cid:17)
Imputation: ˆ x dt = Ψ (cid:16) { x dt , ˜ x dt , m dt } d =1: D (cid:17)
Error Estimation: ˆ e dt = Γ (cid:16) { m dτ } d =1: D , { δ dτ , m dτ } τ =1: t (cid:17)
Compute the conﬁdence interval of imputed value: CI x = (ˆ x dt − λ
ˆ
e dt , ˆ x dt
+ λ ˆ e dt )
Prediction Interval: Compute the conﬁdence interval of predictions: CI y
=
(ˆ y d,lt , ˆ y d,ut )
ˆ y d,l = min ˆ x dτ ∈ CI x Ω (cid:16) { x dτ , ˆ x dτ , δ dτ } τ =1: t,d =1: D (cid:17)
ˆ y d,u = max ˆ x dτ ∈ CI x Ω (cid:16) { x dτ , ˆ x dτ , δ dτ } τ =1: t,d =1: D (cid:17)
Active Sensing: Decision of sensing a dt
= I (ˆ y d,ut − ˆ y d,lt > c d )
Prediction: ˆ y t =
Ω (cid:16) { x dτ , ˆ x dτ , δ dτ } τ =1: t,d =1: D (cid:17)
End For
Output: Active sensing { a dt } t =1: T,d =1: D , Prediction { ˆ y t } t =1: T =
C ONFIGURATIONS OF THE E XPERIMENTS
We used ”interp1” package in MATLAB for interpolation algorithms, and ”mice” and ”amelia” packages in R for imputation algorithms.
Table 5 illustrates the details of the prediction algorithm
[Table] [Table]