Title: Recurrent Neural Networks for Multivariate Time Series with Missing Values
Authors: Zhengping Che
Sanjay Purushotham
Kyunghyun Cho
David Sontag
Yan Liu
Venue: ICLR 2017
Keywords: Deep learning

Abstract: Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Units (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provides useful insights for better understanding and utilization of missing values in time series analysis.


Review discussion:
Interesting work!
Another simple baseline for dealing with missing values would be to add a standard NN layer (instead of a handcrafted one) for preprocessing the input consisting of the data and the mask,  feeding into the GRU layer. It is conceivable that if the preprocessing layer happens to be some sort of gated, recurrent layer (GRU/LSTM), that it would learn to impute the missing values and compute important statistics of the mask.
Therefore, it would important to address the following question:
How does the performance of the GRU-simple baseline change if it is given more hidden layers (with same number of parameters and with increasing number of parameters)?
Thanks for your suggestion on experiments for multi-layer RNNs!
As you mentioned, The methods of handling missing values in the baseline GRU-simple and the proposed GRU-D can be generalized to multi-layer RNNs.
We tested the following 2-layer RNN models with similar number of parameters and with more parameters to validate the generalization and efficacy of our model.
GRU-simple-2: A 2-layer RNN model, where the first RNN layer is GRU-simple, and the second layer is a standard GRU layer.
GRU-D-2: A 2-layer RNN model, where the first RNN layer is GRU-D, and the second layer is a standard GRU layer.
Our GRU-D model consistently outperforms other baselines.
Also a deeper or larger model won't always help, given that the available dataset size is limited.
Here's a comparison on physionet mortality prediction task:
Layer size		# of params	AUC score
GRU-simple			43			18495		0.8156
GRU-simple-2			32-32		18947		0.8159
GRU-simple-2(large)	43-64		39250		0.8208
GRU-D				49			18838		0.8424
GRU-D-2				34-34		18599		0.8420
GRU-D-2(large)		49-64		40739		0.8363
We observed similar result improvements on MIMIC-III dataset, where GRU-D outperforms GRU-simple by AUC score >=0.01 with 2 RNN layers.
We will include the detailed evaluation results on multi-layer RNN models in the appendix of the updated paper.
The authors propose a RNN-method for time-series classification with missing values, that can make use of potential information in missing values. It is based on a simple linear imputation of missing values with learnable parameters.
Furthermore, time-intervals between missing values are computed and used to scale the RNN computation downstream. The authors demonstrate that their method outperforms reasonable baselines on (small to mid-sized) real world datasets. The paper is clearly written.
IMO the authors propose a reasonable approach for dealing with missing values for their intended application domain, where data is not abundant and requires smallish models. I’m somewhat sceptical if the benefits would carry over to big datasets, where more general, less handcrafted multi-layer RNNs are an option.
Thanks for your comments and your pre-review suggestions for us to improve this work.
We'd like to clarify that our model is widely applicable and competitive as well on big datasets.
As shown in the experiments (Section 3.3 and Appendix A.3.5 in the updated draft), the proposed GRU-D model achieves consistent improvement in both one-layer and multi-layer RNN settings on all the datasets of varied size.
As far as we know, the MIMIC-III dataset we used is the largest publicly available one of its kind of tens of thousands of admission records. We agreed that the datasets in healthcare is not as many as in other mature domains such as computer vision or natural language processing, where massive public datasets are available and deep learning models are demonstrated to be powerful. Data collection and sharing in healthcare has it own myriad of challenges and is beyond the scope of this paper.
Our methodologies work irrespective of the dataset size.
Also, it is quite important to make deep learning more applicable in domains with moderate amount of available data such as healthcare. Motivated by this, our work is a reasonable approach to improve the RNN performance in such condition by resorting to the missing patterns in the data.
This paper proposed a way to deal with supervised multivariate time series tasks involving missing values. The high level idea is still using the recurrent neural network (specifically, GRU in this paper) to do sequence supervised learning, e.g., classification, but modifications have been made to the input and hidden layers of RNNs to tackle the missing value problem.
pros:
1) the insight of utilizing missing value is critical. the observation of decaying effect in the healthcare application is also interesting;
2) the experiment seems to be solid; the baseline algorithms and analysis of results are also done properly.
cons:
1) the novelty of this work is not enough. Adding a decaying smooth factor to input and hidden layers seems to be the main modification of the architecture.
2) the datasets used in this paper are small.
3) the decaying effect might not be able to generalize to other domains.
Thanks for your time and thoughtful comments! We would like to address your the questions and concerns below.
Q1: the novelty of this work is not enough. Adding a decaying smooth factor to input and hidden layers seems to be the main modification of the architecture.
A: We would like to clarify our contributions: we develop a reasonable and novel RNN framework for time series classification, which 1) capture missing patterns, 2) automatically imputes or accounts for missing values, and 3) consistently improves classification results. To the best of our knowledge, our work is one of the first of this kind. None of the previous works explicitly modified the RNN structure to capture and utilize missingness information for applications such as healthcare. Experiments on both synthetic and real datasets have demonstrated our proposed GRU-D model is the best among existing state-of-the-art models.
Q2: the datasets used in this paper are small.
A: (This question is also asked by AnonReviewer2 and we thus copied the same answer here.)
As far as we know, the MIMIC-III dataset we used is the largest publicly available one of its kind of tens of thousands of admission records. We agreed that the datasets in healthcare is not as many as in other mature domains such as computer vision or natural language processing, where massive public datasets are available and deep learning models are demonstrated to be powerful. Data collection and sharing in healthcare has it own myriad of challenges and is beyond the scope of this paper.
Our methodologies work irrespective of the dataset size.
Also, it is quite important to make deep learning more applicable in domains with moderate amount of available data such as healthcare. Motivated by this, we believe our work is a reasonable approach to improve the RNN performance in such condition by resorting to the missing patterns in the data.
Q3: the decaying effect might not be able to generalize to other domains.
A: The proposed model works in a broad set of domains where we encounter time series data missing at random or not at random. In this paper we evaluated our model for healthcare applications. Given that we're taking ICU data as example, our decay idea is very well matched to homeostatic systems (e.g., human body) that actively control themselves to revert to mean. We demonstrated it is indeed helpful in practice.
Furthermore, our framework is flexible enough to incorporate decay functions in a variety of forms which can handle different missing mechanisms.
This paper presents a modified gated RNN caled GRU-D that deals with time series which display a lot of missing values in their input. They work on two fronts. The first deals with the missing inputs directly by using a learned convex combination of the previous available value (forward imputation) and the mean value (mean imputation). The second includes dampening the recurrent layer not unlike a second reset gate, but parametrized according to the time elapsed since the last available value of each attributes.
Positives
- Clear definition of the task (handling missing values for classification of time series)
- Many interesting baselines to test the new model against.
- The model presented deals with the missing values in a novel, ML-type way (learn new dampening parameters).
- The extensive tests done on the datasets is probably the greatest asset of this paper.
Negatives
- The paper could use some double checking for typos.
- The Section A.2.3 really belongs in the main article as it deals with important related works. Swap it with the imprecise diagrams of the model if you need space.
- No mention of any methods from the statistics litterature.
Here are the two main points of this review that informs my decision:
1. The results, while promising, are below expectations. The paper hasn’t been able to convince me that GRU-simple (without intervals) isn’t just as well-suited for the task of handling missing inputs as GRU-D. In the main paper, GRU-simple is presented as the main baseline.
Yet, it includes a lot of extraneous parameters (the intervals) that, according to Table 5, probably hurts the model more than it helps it. Having a third of it’s parameters being of dubious value, it brings the question of the fairness of the comparison done in the main paper, especially since in the one table where GRU-simple (without intervals) is present, GRU-D doesn’t significantly outperforms it.
2. My second concern, and biggest, is with some claims that are peppered through the paper. The first is about the relationship with the presence rate of data in the dataset and the diagnostics. I might be wrong, but that only indicates that the doctor in charge of that patient requested the relevant analyses be done according to the patient’s condition. That would mean that an expert system based on this data would always seem to be one step behind.
The second claim is the last sentence of the introduction, which sets huge expectations that were not met by the paper.
Another is that “simply concatenating masking and time interval vectors fails to exploit the temporal structure of missing values” is unsubstantiated and actually disproven later in the paper.
Yet another is the conclusion that since GRU models displayed the best improvement between a subsample of the dataset and the whole of it means that the improvement is going to continue to grow as more data is added. This fails to consider that non-GRU models actually started with much better results than most GRU ones.
Lastly is their claim to capture informative missingness by incorporating masking and time intervals directly inside the GRU architecture.
While the authors did make these changes, the fact that they also concatenate the mask to the input, just like GRU-simple (without intervals), leads me to question the actual improvement made by GRU-D.
Given that, while I find that the work that has been put into the paper is above average, I wouldn’t accept that paper without a reframing of the findings and a better focus on the real contribution of this paper, which I believe is the novel way to parametrize the choice of imputation method.
Thanks for your time and thoughtful comments!
Before addressing detailed comments, we would like to clarify our contribution in this work: we develop a reasonable and novel RNN framework for time series classification, which 1) captures missing patterns, 2) automatically imputes or accounts for missing values, and 3) consistently obtains the state-of-the-art results on the challenging healthcare domain classification problems. To the best of our knowledge, our work is one of the first of this kind. None of the previous works explicitly modify the RNN structure to capture and utilize missingness information for applications such as healthcare. Experiments on synthetic and real datasets have demonstrated our proposed GRU-D model is the best among existing state-of-the-art models.
We agree that GRU-simple (with and without intervals) models are strong baselines, but the proposed GRU-D model is definitely the best among all GRU baselines for handling and utilizing missingness.
GRU-simple models often but not always get marginal improvement against other GRU baselines (i.e., GRU-mean/forward).
Moreover, the masking and/or time interval are fed into the model same as other input values without considering their roles as `missingness indicators`.
Empirically we found GRU-simple baselines (with and without intervals) have quite close performance and are not as good as GRU-D.
Thus we take GRU-simple as the representative in the main paper and show its variants in the appendix. It’s worth noting that while time interval doesn’t help much in GRU-simple, it not only improves the performance of GRU-D but also provides useful interpretations for the prediction tasks (as shown in Fig. 4).
We believe the superiority of GRU-D model over GRU baselines for the following reasons:
1. It achieves consistent performance improvement on the difficult prediction tasks: ~0.02 AUC score in PhysioNet mortality prediction and ~0.01 AUC score in MIMIC-III mortality prediction over the best baselines.
2. GRU-D model for healthcare applications is an example of how domain knowledge can be simply incorporated into the model architecture design to better handle and utilize missing values.
3. GRU-D provides useful insights for raw input features from decay plots/distributions. These results help us understand the underlying variable impact on prediction tasks and also help domain experts understand and apply our model and discover new domain knowledge.
Other detailed comments:
Q: the relationship with the presence rate of data in the dataset and the diagnostics.
A: Figure 1 shows the observations on missingness and prediction tasks and, thus demonstrates the usefulness of missingness. Missingness can be due to several factors, such as medical events, saving costs, anomalies, inconvenience and so on. Healthcare providers' judgments and actions may also lead to some missing values in the data.
However, missing data is widespread in our dataset and it's hard to figure out the reason for the missing values of each variable.
Q: the conclusion that since GRU models displayed the best improvement between a subsample of the dataset and the whole of it means that the improvement is going to continue to grow as more data is added. This fails to consider that non-GRU models actually started with much better results than most GRU ones.
A: As shown in Figure 7, GRU-D has comparable performance on 2k samples and beats non-GRU models with increasing margin on 10k and 19k samples. Other GRU baselines also improve with more training samples. Given these results, we expect this trend will continue if we have even larger healthcare datasets. Especially, the performance gap between the proposed GRU-D and non-RNN baselines is expected to increase.
Q: The Section A.2.3 really belongs in the main article as it deals with important related works. Swap it with the imprecise diagrams of the model if you need space.
A: We have revised and reorganized the contents on related baseline models as you suggested to make it more clear.
Q: No mention of any methods from the statistics literature.
A: The goal of this work is to make predictions on whether a patient has a disease (or what is his/her mortality status) based on the time series observations with missing values from ICU.
In short, it is a time series classification task with utilization of missing patterns but not a data imputation task.
Also, as discussed in the paper (Wells et al., 2013), missing not at random imputation techniques lead to suboptimal performance.
# Major interrogation following a perusing of the revised paper:
- eq 4 and 6 : as the cardinality D = |delta_t| = |x_t| = |lambda| (eq 4), I understand how lambda_x is used in 5.
However, other than constraining |h_t| = |x_t| = D, the definition of lambda_h makes little sense. If you use h_t instead of delta_t, that makes this a reset gate, which is already in the GRU.
# Answer to comments
I do not dispute the claim that GRU-D outperforms other algorithms in the majority of the tests made throughout the article.
However, I still think that the main contribution of this paper is to learn the parameters of a convex combination of two imputation method. Since these parameters are indeed useful to extract some meaning (from figure 4), I am willing to retract part of my comment.
From your answer : "GRU-D model for healthcare applications is an example of how domain knowledge can be simply incorporated into the model architecture design to better handle and utilize missing values."
I fail to see where you incorporated specific domain knowledge in the model. Usually, domain knowledge would be used to choose an imputation method, but since you elegantly side-step that problem by learning which method to choose, I can't find where else you use domain knowledge. If you are referring to the homeostatic properties of the human body, I would point out that Figure 4a establishes that forward imputation is always the prefered method (lambda never below 0.5).
I still maintain that in your specific tests, not-at-random-missingness used for prediction isn't useful for an health care expert.
However, GRU-D does use the presence of features as an additional clue for classification.
After another look at Figure 7, I think I might have based my comment on GRU-simple, which is my mistake.
Finally, I still believe that for the paper to have a greater impact, a review of better imputation methods would have been needed. Not only could GRU-D be based on more advanced methods (instead of just forward and mean imputation), but it would have made your contribution more credible.
From your answer : "it is a time series classification task with utilization of missing patterns but not a data imputation task".
Yet, you beat the model from Lipton et al. (~ GRU-simple masking only) by combining two imputation methods in your model, which clearly suggests that imputation is a very important factor in learning a good model that handles missing values.
While I applaud the proposition of an integrated method that does not require a two-step approach, you still ignore a whole field of study designed to solve the missing value problem.
# Random minor comments from reading the revised paper, in order of appearance:
- 1.0, last paragraph : time interval -> time intervals (and in subsequent paragraphs)
- 2.0, last line of page 2: t=0 -> t=1
- 2.0, last paragraph : remove l_n from \mathcal{D} as it would put your target in the training set
- 2.1 : j, which I believe is the j-th neuron in the hidden layer, is never defined.
- 2.1 : switching the definitions of z_t and r_t would make the formulas read more naturally
- eq 1, 2, ... : I understand that the algorithm uses m_t^d a lot, but the paper would read more fluidly by using cases definitions.
- 2.2 : in health care domain -> either "in the health care domain" or "in health care domains"
- 3.3, last paragraph : both the datasets -> both datasets
- 3.4, first paragraph : perform superior -> perform better
- 3.4, last paragraph : add reference to fig. 7
- 4.0 : "by incorporating masking [...] inside the GRU architecture". With m_t being concatenated to x_t, this claim doesn't stand.
Thanks for your comments and follow-up questions. Please find our reply below.
1) Question about Eq. 4-6.
A: Firstly we just want to make it clear that the `lambda` mentioned in your question refers to the decay rate `gamma` in these equations.
As shown in Eq. 4, in GRU-D, gamma_x = exp(W_x \dot delta + b_x) and gamma_h = exp(W_h \dot delta + b_h), where `delta` is the time interval and W_x/h, b_x/h are model parameters.
Say the numbers of input features and hidden states are |x_t| = D and |h_t| = H, respectively.
W_x is a diagonal matrix with size [D,D], and W_h is a matrix with size [H,D].
By doing this, GRU-D decay the input feature based on its own time interval and decay the hidden states based on the time intervals of all input features.
The gamma_h (which is calculated based on delta_t) is different from the reset gate (which is calculated based on h_{t-1} and x_t) in a standard GRU.
2) The homeostatic properties of the human body and Figure 4a).
A: For some of the input features, simply forward imputation on the missing values can lead to reasonable prediction performance.
However, forward imputation is not optimal for all features, otherwise the input decay in Figure 4a) would stay at 1 for all features at all time.
Following the homeostatic properties, we do find a few variables have different decay rates which contribute to the model performance a lot, and many of them are validated by domain experts.
Also, the plots in Figure 4a) show the input decay for time interval of <= 24 hours, and the decay values will be lower for longer time interval.
3) `I still maintain that in your specific tests, not-at-random-missingness used for prediction isn't useful for an health care expert.
However, GRU-D does use the presence of features as an additional clue for classification.`
A: We agree that not-at-random-missingness may have not been properly used for health care experts. This may due to several reasons, e.g., limited time constraints to analyze the (missing) data, complex and even unknown reasons and sources of missing values, etc.
However, not properly utilizing the missingness does not mean it is useless. Both the correlation validation like Fig. 1 and the performance improvements gained by GRU-D demonstrate that missingness can also be an informative resource rather than only cumbrance.
4) `a review of better imputation methods would have been needed.`
`you still ignore a whole field of study designed to solve the missing value problem.`
A: Thanks for your suggestion on including a broader reviews of imputation methods. We'd like to include imputation literature survey on imputation methods in the upcoming draft.
We apologize for the confusion and we are not ignoring the research progress from the imputation field, but we are proposing a novel way to use missingness informativeness directly in the deep learning models which can help in training end-to-end ML/AI systems. As far as we know, imputation for missing-not-at-random is an open ongoing research problem.
5) by combining two imputation methods in your model, which clearly suggests that imputation is a very important factor in learning a good model that handles missing values.
A: Our model not only combines forward and mean imputations but actually utilizes time intervals and masking. The interval information is never considered in forward/mean imputation methods.
6) `"by incorporating masking [...] inside the GRU architecture". With m_t being concatenated to x_t, this claim doesn't stand.`
A: m_t is not only concatenated to the input and fed to get hidden state H, but also acts as the switch to control the flow of current input x_{t} and decayed input x_{t-1}. We updated figure 3b to make it clearer.
7) Thanks for your detailed comments for the revised paper. We have made corresponding changes in a new version.
Of course, gamma... please excuse my confusion with those upside down lambdas!
Now I understand that you didn't constrain your parameters matrix W to be square, which I assumed would be the case with the comments about not constraining the matrix to be diagonal. Thank you for the precision.
In the revised paper, you switched z_t and r_t in section 2.1 but not in section 2.2.
I look forward to reading the future draft with the inclusion of the survey of the imputation literature!

Meta-review:
role: PC
data: comment: This paper presents a modification of GRU-RNNs to handle missing data explicitly, allowing them to exploit data not missing at random. The method is presented clearly enough, but the reviewers felt that the claims were overreaching. It's also unsatisfying that the method depends on specific modifications of RNN architectures for a particular domain, instead of being a more general approach.

Decision: Reject