Original: G RADIENT I MPORTANCE L EARNING FOR I NCOMPLETE O BSERVATIONS
Revised: G RADIENT I MPORTANCE L EARNING FOR I NCOMPLETE O BSERVATIONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Though recent works have developed methods that can generate estimates (or imputations) of the missing entries in a dataset to facilitate downstream analysis, most depend on assumptions that may not align with real-world applications and could suffer from poor performance in subsequent tasks such as classiﬁcation.
Revised: Though recent works have developed methods that can generate estimates (or imputations) of the missing entries in a dataset to facilitate downstream analysis, most depend on assumptions that may not align with real-world applications and could suffer from poor performance in subsequent tasks such as classiﬁcation.

Original: This is particularly true if the data have large missingness rates or a small sample size.
Revised: This is particularly true if the data have large missingness rates or a small sample size.

Original: More importantly, the imputation error could be propagated into the prediction step that follows, which may constrain the capabilities of the prediction model.
Revised: More importantly, the imputation error could be propagated into the prediction step that follows, which may constrain the capabilities of the prediction model.

Original: In this work, we introduce the gradient importance learning (GIL) method to train multilayer perceptrons (MLPs) and long short-term memories (LSTMs) to directly perform inference from inputs containing missing values without imputation .
Revised: In this work, we introduce the gradient importance learning (GIL) method to train multilayer perceptrons (MLPs) and long short-term memories (LSTMs) to directly perform inference from inputs containing missing values without imputation .

Original: Speciﬁcally, we employ reinforcement learning (RL) to adjust the gradients used to train these models via back-propagation.
Revised: Speciﬁcally, we employ reinforcement learning (RL) to adjust the gradients used to train these models via back-propagation.

Original: This allows the model to exploit the underlying information behind missingness patterns .
Revised: This allows the model to exploit the underlying information behind missingness patterns .

Original: We test the approach on real-world time-series ( i.e. , MIMIC-III), tabular data obtained from an eye clinic, and a standard dataset ( i.e. , MNIST), where our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
Revised: We test the approach on real-world time-series ( i.e. , MIMIC-III), tabular data obtained from an eye clinic, and a standard dataset ( i.e. , MNIST), where our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: We consider learning from incomplete datasets.
Revised: We consider learning from incomplete datasets.

Original: Components of the data could be missing for various reasons, for instance, missing responses from participants, data loss, and restricted access issues.
Revised: Components of the data could be missing for various reasons, for instance, missing responses from participants, data loss, and restricted access issues.

Original: This phenomenon is prevalent in the healthcare domain, mostly in the context of electronic health records (EHRs), which are structured as patient-speciﬁc irregular timelines with attributes, e.g. , diagnosis, laboratory tests, vitals, thus resulting in high missingness across patients for any arbitrary time point.
Revised: This phenomenon is prevalent in the healthcare domain, mostly in the context of electronic health records (EHRs), which are structured as patient-speciﬁc irregular timelines with attributes, e.g. , diagnosis, laboratory tests, vitals, thus resulting in high missingness across patients for any arbitrary time point.

Original: Such missingness introduces difﬁculties when developing models and performing inference in real-world applications such as Kam & Kim (2017); Scherpf et al.
Revised: Such missingness introduces difﬁculties when developing models and performing inference in real-world applications such as Kam & Kim (2017); Scherpf et al.

Original: Existing works tackle this problem by either proposing imputation algorithms (IAs) to explicitly produce estimates of the missing data, or by imposing imputation objectives during inference, e.g. , withholding observed values as being the ground-truth and learning to impute them.
Revised: Existing works tackle this problem by either proposing imputation algorithms (IAs) to explicitly produce estimates of the missing data, or by imposing imputation objectives during inference, e.g. , withholding observed values as being the ground-truth and learning to impute them.

Original: However, some of these either require additional modeling assumptions for the underlying distributions (Bashir & Wei, 2018; Fortuin et al., 2020), or formatting of the data (Yu et al., 2016; Schnabel et al., 2016).
Revised: However, some of these either require additional modeling assumptions for the underlying distributions (Bashir & Wei, 2018; Fortuin et al., 2020), or formatting of the data (Yu et al., 2016; Schnabel et al., 2016).

Original: Other applications depend on domain knowledge for pre-processing and modeling such as Yang et al.
Revised: Other applications depend on domain knowledge for pre-processing and modeling such as Yang et al.

Original: (2018); Kam & Kim (2017), or introduce additional information-based losses (Cao et al., 2018;
Revised: (2018); Kam & Kim (2017), or introduce additional information-based losses (Cao et al., 2018;

Original: Lipton et al., 2016), which are usually intractable with real-world data.
Revised: Lipton et al., 2016), which are usually intractable with real-world data.

Original: Moreover, generative methods (Mattei & Frellsen, 2019;
Revised: Moreover, generative methods (Mattei & Frellsen, 2019;

Original: Yoon et al., 2018) could result in imputations with high variation ( e.g. , low conﬁdence of the output distribution), when data have high missingness rates or small sample sizes.
Revised: Yoon et al., 2018) could result in imputations with high variation ( e.g. , low conﬁdence of the output distribution), when data have high missingness rates or small sample sizes.

Original: Figure 1 illustrates imputations generated by two state-of-the-art deep generative models on the MNIST dataset with 50%, 70% and 90% of the pixels set as missing ( i.e. , masked out).
Revised: Figure 1 illustrates imputations generated by two state-of-the-art deep generative models on the MNIST dataset with 50%, 70% and 90% of the pixels set as missing ( i.e. , masked out).

Original: It is observed that both approaches suffer from inaccurate reconstruction of the original digits as the missingness rate increases, which is manifested as some of the imputed images not being recognizable as digits.
Revised: It is observed that both approaches suffer from inaccurate reconstruction of the original digits as the missingness rate increases, which is manifested as some of the imputed images not being recognizable as digits.

Original: Importantly, the error introduced by the imputation process can be further propagated into downstream inference and prediction stages.
Revised: Importantly, the error introduced by the imputation process can be further propagated into downstream inference and prediction stages.

Original: Imputing the missing data (either explicitly or implicitly) is, in most cases, not necessary, more so, considering that sometimes where and when the data are missing can be intrinsically informative.
Revised: Imputing the missing data (either explicitly or implicitly) is, in most cases, not necessary, more so, considering that sometimes where and when the data are missing can be intrinsically informative.

Original: Consider a scenario in which two patients, A and B, admitted to the intensive care unit (ICU) suffer from bacterial and viral infections, respectively, and assume that the healthcare provider monitors the status of the patients by ordering two (slightly) different blood tests daily, namely, a culture test/panel speciﬁc to bacterial infections and a RT-PCR test/panel speciﬁc to viral infections.
Revised: Consider a scenario in which two patients, A and B, admitted to the intensive care unit (ICU) suffer from bacterial and viral infections, respectively, and assume that the healthcare provider monitors the status of the patients by ordering two (slightly) different blood tests daily, namely, a culture test/panel speciﬁc to bacterial infections and a RT-PCR test/panel speciﬁc to viral infections.

Original: Hence, patient
Revised: Hence, patient

Original: A is likely to have much fewer orders (and results) for viral tests.
Revised: A is likely to have much fewer orders (and results) for viral tests.

Original: In both cases, the missingness patterns are indicative of the underlying condition of both patients.
Revised: In both cases, the missingness patterns are indicative of the underlying condition of both patients.

Original: Moreover, such patterns are more commonly found in incomplete data caused by missing at random (MAR) or missing not at random (MNAR) mechanisms, which usually introduce additional difﬁculties for inference (Little &
Revised: Moreover, such patterns are more commonly found in incomplete data caused by missing at random (MAR) or missing not at random (MNAR) mechanisms, which usually introduce additional difﬁculties for inference (Little &

Original: Rubin, 2019).
Revised: Rubin, 2019).

Original: Inspired by this, we propose the gradient importance learning (GIL) method, which facilitates an imputation-free learning framework for incomplete data by simultaneously leveraging the observed data and the information underlying missingness patterns.
Revised: Inspired by this, we propose the gradient importance learning (GIL) method, which facilitates an imputation-free learning framework for incomplete data by simultaneously leveraging the observed data and the information underlying missingness patterns.

Original: Speciﬁcally, we propose to use an importance matrix to weight the gradients that are used to update the model parameters in the backpropagation process, after the computational graph of the model is settled, which allows the model to exploit the information underlying missingness without imputation.
Revised: Speciﬁcally, we propose to use an importance matrix to weight the gradients that are used to update the model parameters in the backpropagation process, after the computational graph of the model is settled, which allows the model to exploit the information underlying missingness without imputation.

Original: However, these gradients cannot be tracked by automated differentiation tools such as Tensorﬂow (Abadi et al., 2016).
Revised: However, these gradients cannot be tracked by automated differentiation tools such as Tensorﬂow (Abadi et al., 2016).

Original: So motivated, we propose to generate the importance matrix using a reinforcement learning (RL) policy on-the-ﬂy .
Revised: So motivated, we propose to generate the importance matrix using a reinforcement learning (RL) policy on-the-ﬂy .

Original: This is done by conditioning on the status of training procedure characterized by the model parameters, inputs and model performance at the current training step.
Revised: This is done by conditioning on the status of training procedure characterized by the model parameters, inputs and model performance at the current training step.

Original: Concurrently, RL algorithms are used to update the policy by ﬁrst modeling the back-propagation process as an RL environment, or a Markov decision process (MDP).
Revised: Concurrently, RL algorithms are used to update the policy by ﬁrst modeling the back-propagation process as an RL environment, or a Markov decision process (MDP).

Original: Then, by interacting with the environment, the RL algorithm can learn the optimal policy so the importance matrix can aid the training of the prediction models to attain desirable performance.
Revised: Then, by interacting with the environment, the RL algorithm can learn the optimal policy so the importance matrix can aid the training of the prediction models to attain desirable performance.

Original: Moreover, we also show that our framework can be augmented with feature learning techniques, e.g. , contrastive learning as in Chen et al.
Revised: Moreover, we also show that our framework can be augmented with feature learning techniques, e.g. , contrastive learning as in Chen et al.

Original: (2020), which can further improve the performance of the proposed imputation-free models.
Revised: (2020), which can further improve the performance of the proposed imputation-free models.

Original: The technical contributions of this work can be summarized as follows: ( i ) to the best of our knowledge, this is the ﬁrst work that trains deep learning (DL) models to perform accurate imputationfree predictions with missing inputs.
Revised: The technical contributions of this work can be summarized as follows: ( i ) to the best of our knowledge, this is the ﬁrst work that trains deep learning (DL) models to perform accurate imputationfree predictions with missing inputs.

Original: This allows models to effectively handle high missing rates, while signiﬁcantly reducing the prediction error compared to existing imputation-prediction frameworks.
Revised: This allows models to effectively handle high missing rates, while signiﬁcantly reducing the prediction error compared to existing imputation-prediction frameworks.

Original: ( ii ) Unlike existing approaches that require additional modeling assumptions or that rely on methods (or pre-existing knowledge) that intrinsically have advantages over speciﬁc types of data, our method does not require any assumptions or domain expertise.
Revised: ( ii ) Unlike existing approaches that require additional modeling assumptions or that rely on methods (or pre-existing knowledge) that intrinsically have advantages over speciﬁc types of data, our method does not require any assumptions or domain expertise.

Original: Consequently, it can consistently achieve top performance over a variety of data and under different conditions.
Revised: Consequently, it can consistently achieve top performance over a variety of data and under different conditions.

Original: ( iii )
Revised: ( iii )

Original: The proposed framework also facilitates feature learning from incomplete data, as the importance matrix can guide the hidden layers of the model to capture information underlying missingness patterns during training, which results in more expressive features, as illustrated in Figure 3.
Revised: The proposed framework also facilitates feature learning from incomplete data, as the importance matrix can guide the hidden layers of the model to capture information underlying missingness patterns during training, which results in more expressive features, as illustrated in Figure 3.

Original:  G RADIENT I MPORTANCE L EARNING (GIL)
Revised:  G RADIENT I MPORTANCE L EARNING (GIL)

Original: In this section, we introduce GIL; a method that facilitates training of imputation-free prediction models with incomplete data.
Revised: In this section, we introduce GIL; a method that facilitates training of imputation-free prediction models with incomplete data.

Original: In Section 2.2, we show how the gradient descent directions can be adjusted using the importance matrix A .
Revised: In Section 2.2, we show how the gradient descent directions can be adjusted using the importance matrix A .

Original: In
Revised: In

Original: Section 2.3, we introduce an RL framework to generate the elements of A which can leverage the information underlying missingness during training, as illustrated in Figure 2.
Revised: Section 2.3, we introduce an RL framework to generate the elements of A which can leverage the information underlying missingness during training, as illustrated in Figure 2.

Original: 2. P ROBLEM F ORMULATION
Revised: 2. P ROBLEM F ORMULATION

Original: We consider a dataset containing N observations X =
Revised: We consider a dataset containing N observations X =

Original: ( X 1 , X 2 , . . .
Revised: ( X 1 , X 2 , . . .

Original: , X N ) where each X j can be represented by a vector x j ∈ R d (for tabular data) or a matrix X j ∈ R T j × d (for time-series) with
Revised: , X N ) where each X j can be represented by a vector x j ∈ R d (for tabular data) or a matrix X j ∈ R T j × d (for time-series) with

Original: T j denoting the time horizon of the sequence X j .
Revised: T j denoting the time horizon of the sequence X j .

Original: Since the focus of this work is principally on time-series data, recurrent neural networks will receive signiﬁcant attention.
Revised: Since the focus of this work is principally on time-series data, recurrent neural networks will receive signiﬁcant attention.

Original: We also deﬁne the set of missing indicators M =
Revised: We also deﬁne the set of missing indicators M =

Original: ( M 1 , M 2 , . . .
Revised: ( M 1 , M 2 , . . .

Original: , M N ) , where M j = m j ∈ { 0 , 1 } d or M j ∈ { 0 , 1 } T j × d depending on the dimension of X j , and 0’s (or 1’s) correspond to the entries in X j that are missing (or not missing), respectively.
Revised: , M N ) , where M j = m j ∈ { 0 , 1 } d or M j ∈ { 0 , 1 } T j × d depending on the dimension of X j , and 0’s (or 1’s) correspond to the entries in X j that are missing (or not missing), respectively.

Original: Finally, we assume that each X j is associated with some label y j ∈ Y ; thus Y =
Revised: Finally, we assume that each X j is associated with some label y j ∈ Y ; thus Y =

Original: ( y 1 , y 2 , . . .
Revised: ( y 1 , y 2 , . . .

Original: , y N ) denotes the set of labels.
Revised: , y N ) denotes the set of labels.

Original: We deﬁne the problem as learning a model parameterized by W to directly predict y j by generating ˆy j that maximizes the log-likelihood (cid:80)
Revised: We deﬁne the problem as learning a model parameterized by W to directly predict y j by generating ˆy j that maximizes the log-likelihood (cid:80)

Original: Nj =1 log p ( ˆy j | X j , M j , W ) , without imputing the missing values in X j .
Revised: Nj =1 log p ( ˆy j | X j , M j , W ) , without imputing the missing values in X j .

Original: 2. G RADIENT I MPORTANCE
Revised: 2. G RADIENT I MPORTANCE

Original: Missingness is a common issue found in tabular data and time series, where multi-layered perceptron (MLP) (Bishop, 2006) and long short-term memory (LSTM)
Revised: Missingness is a common issue found in tabular data and time series, where multi-layered perceptron (MLP) (Bishop, 2006) and long short-term memory (LSTM)

Original: (Hochreiter & Schmidhuber, 1997) models, respectively, are often considered for predictions.
Revised: (Hochreiter & Schmidhuber, 1997) models, respectively, are often considered for predictions.

Original: Below we illustrate how the importance matrix can be applied to gradients, which are produced by taking regular stochastic gradient descent (SGD) steps, toward training MLP models with incomplete tabular inputs.
Revised: Below we illustrate how the importance matrix can be applied to gradients, which are produced by taking regular stochastic gradient descent (SGD) steps, toward training MLP models with incomplete tabular inputs.

Original: Note that this idea can be easily extended to sequential inputs with LSTM models and the corresponding details can be found in Appendix A. First, the deﬁnitions of the dataset and missing indicators can be reduced to X tab
Revised: Note that this idea can be easily extended to sequential inputs with LSTM models and the corresponding details can be found in Appendix A. First, the deﬁnitions of the dataset and missing indicators can be reduced to X tab

Original: = ( x 1 , x 2 , . . .
Revised: = ( x 1 , x 2 , . . .

Original: , x N ) and M tab = ( m 1 , m 2 , . . .
Revised: , x N ) and M tab = ( m 1 , m 2 , . . .

Original: , m N ) , with x j ∈ R d and m j ∈ R d , respectively.
Revised: , m N ) , with x j ∈ R d and m j ∈ R d , respectively.

Original: Then we deﬁne an MLP with k hidden layers, for k ≥ 2 , as follows
Revised: Then we deﬁne an MLP with k hidden layers, for k ≥ 2 , as follows

Original: [Equation]
Revised: [Equation]

Original: where x ∈ R
Revised: where x ∈ R

Original: d is the input, W i and φ i are the weight matrix and activation functions for the i -th hidden layer, respectively, and we have omitted the bias terms for notational convenience.
Revised: d is the input, W i and φ i are the weight matrix and activation functions for the i -th hidden layer, respectively, and we have omitted the bias terms for notational convenience.

Original: The ﬁrst layer can be interpreted as the encoding layer W enc =
Revised: The ﬁrst layer can be interpreted as the encoding layer W enc =

Original: W 1 that maps inputs to features ζ = f enc ( x | W enc ) , while the rest are the inference layers W inf = { W 2 , . . .
Revised: W 1 that maps inputs to features ζ = f enc ( x | W enc ) , while the rest are the inference layers W inf = { W 2 , . . .

Original: , W k , W out } that map the features to prediction ˆy = f inf ( ζ | W inf ) .
Revised: , W k , W out } that map the features to prediction ˆy = f inf ( ζ | W inf ) .

Original: In the following proposition, we show that the gradients of some commonly used loss functions, E ( ˆy , y ) , such as cross entropy or mean squared errors, w.r.t.
Revised: In the following proposition, we show that the gradients of some commonly used loss functions, E ( ˆy , y ) , such as cross entropy or mean squared errors, w.r.t.

Original: W 1 , can be formulated in the form of an outer product.
Revised: W 1 , can be formulated in the form of an outer product.

Original: The proof and details can be found in Appendix E.
Revised: The proof and details can be found in Appendix E.

Original: Proposition 1 Given a MLP and a smooth loss function E ( ˆy , y ) , the gradients of E w.r.t.
Revised: Proposition 1 Given a MLP and a smooth loss function E ( ˆy , y ) , the gradients of E w.r.t.

Original: the encoding layer can be formulated as ∂E/∂ W 1 = ∆ 1 x (cid:62) , where ∆ 1 contains the gradients propagated from all the inference layers, x ∈ X tab and y ∈ Y .
Revised: the encoding layer can be formulated as ∂E/∂ W 1 = ∆ 1 x (cid:62) , where ∆ 1 contains the gradients propagated from all the inference layers, x ∈ X tab and y ∈ Y .

Original: From this Proposition it can be seen that the gradients that are used to train the encoding layers, following regular SGD solvers, can be formulated as the outer product between the gradients ∆ propagated from the inference layers and the input x as
Revised: From this Proposition it can be seen that the gradients that are used to train the encoding layers, following regular SGD solvers, can be formulated as the outer product between the gradients ∆ propagated from the inference layers and the input x as

Original: [Equation]
Revised: [Equation]

Original: where W enc ∈ R e × d , ∆ ∈ R e , x ∈ R d , e is the dimension of the features, and d is the dimension of the input.
Revised: where W enc ∈ R e × d , ∆ ∈ R e , x ∈ R d , e is the dimension of the features, and d is the dimension of the input.

Original: To simplify notation, note that henceforth we use x to refer to an individual tabular data x j ∈ X tab , regardless of its index.
Revised: To simplify notation, note that henceforth we use x to refer to an individual tabular data x j ∈ X tab , regardless of its index.

Original: The corresponding SGD updates for training W enc , using learning rate α , are W enc ← W enc − α ∆ · x (cid:62) .
Revised: The corresponding SGD updates for training W enc , using learning rate α , are W enc ← W enc − α ∆ · x (cid:62) .

Original: As a result, it is observed from (2) that the j -th ( j ∈ [1 , d ] ) column of ( ∂E/∂ W enc )
Revised: As a result, it is observed from (2) that the j -th ( j ∈ [1 , d ] ) column of ( ∂E/∂ W enc )

Original: SGD is weighted by the j -th element in x given ∆ .
Revised: SGD is weighted by the j -th element in x given ∆ .

Original: However, given that some entries in x could be missing (according to the missing indicator m ) and their values are usually replaced by a placeholder ξ ∈ R , it may not be ideal do directly back-propagate the gradients in the form of (2), as the features ζ captured by the encoding layers may not be expressive enough for the inference layers to make accurate predictions.
Revised: However, given that some entries in x could be missing (according to the missing indicator m ) and their values are usually replaced by a placeholder ξ ∈ R , it may not be ideal do directly back-propagate the gradients in the form of (2), as the features ζ captured by the encoding layers may not be expressive enough for the inference layers to make accurate predictions.

Original: Instead, we consider introducing the importance matrix A ∈ [0 , 1]
Revised: Instead, we consider introducing the importance matrix A ∈ [0 , 1]

Original: e × d to adjust the gradient descent direction as
Revised: e × d to adjust the gradient descent direction as

Original: [Equation]
Revised: [Equation]

Original: which not only trains the model to capture information from the observed inputs that are most useful for making accurate predictions, but also to leverage the information underlying the missingness in the data.
Revised: which not only trains the model to capture information from the observed inputs that are most useful for making accurate predictions, but also to leverage the information underlying the missingness in the data.

Original: The elements of A can be generated using the RL framework introduced in the next section.
Revised: The elements of A can be generated using the RL framework introduced in the next section.

Original: Note that above we have omitted all the bias terms to simplify our presentation and note that: i ) gradients w.r.t.
Revised: Note that above we have omitted all the bias terms to simplify our presentation and note that: i ) gradients w.r.t.

Original: to biases do not conform to the outer product format, and ii ) more importantly, these gradients do not depend on the inputs – thus in practice, the importance matrix A is only applied to the gradients of the weights.
Revised: to biases do not conform to the outer product format, and ii ) more importantly, these gradients do not depend on the inputs – thus in practice, the importance matrix A is only applied to the gradients of the weights.

Original: Moreover, we do not consider convolutional neural network (CNN) models (LeCun et al., 1999) in this work because of i ).
Revised: Moreover, we do not consider convolutional neural network (CNN) models (LeCun et al., 1999) in this work because of i ).

Original: Though the proposed framework could still be applied to CNNs, it may not be as efﬁcient as for MLPs or LSTMs, where the search space for
Revised: Though the proposed framework could still be applied to CNNs, it may not be as efﬁcient as for MLPs or LSTMs, where the search space for

Original: A is signiﬁcantly reduced by taking advantages of the outer product format, as shown in (4) below.
Revised: A is signiﬁcantly reduced by taking advantages of the outer product format, as shown in (4) below.

Original: Importantly, our focus is to address the missingness in tabular data and time-series, in which case
Revised: Importantly, our focus is to address the missingness in tabular data and time-series, in which case

Original: MLPs and LSTMs are appropriate, while the missingness in images is usually related to applications in other domains such as compressed sensing (Wu et al., 2019) or image super-resolution
Revised: MLPs and LSTMs are appropriate, while the missingness in images is usually related to applications in other domains such as compressed sensing (Wu et al., 2019) or image super-resolution

Original: (Dong et al., 2015), which we plan to explore in the future.
Revised: (Dong et al., 2015), which we plan to explore in the future.

Original: 2. RL TO G ENERATE I MPORTANCE M ATRIX
Revised: 2. RL TO G ENERATE I MPORTANCE M ATRIX

Original: Now we show how to use RL to generate the importance matrix A .
Revised: Now we show how to use RL to generate the importance matrix A .

Original: In general, RL aims to learn an optimal policy that maximizes expected rewards, by interacting with an unknown environment (Sutton & Barto, 2018).
Revised: In general, RL aims to learn an optimal policy that maximizes expected rewards, by interacting with an unknown environment (Sutton & Barto, 2018).

Original: Formally, the RL environment is characterized by a Markov decision process (MDP), with details introduced below.
Revised: Formally, the RL environment is characterized by a Markov decision process (MDP), with details introduced below.

Original: Each time after the agent chooses an action a = π ( s ) at state s following some policy π , the environment responds with the next state s (cid:48) , along with an immediate reward r =
Revised: Each time after the agent chooses an action a = π ( s ) at state s following some policy π , the environment responds with the next state s (cid:48) , along with an immediate reward r =

Original: R ( s, a ) given a reward function R ( · , · ) .
Revised: R ( s, a ) given a reward function R ( · , · ) .

Original: The goal is to learn an optimal policy that maximizes the expected total reward deﬁned as J ( π ) =
Revised: The goal is to learn an optimal policy that maximizes the expected total reward deﬁned as J ( π ) =

Original: E ρ π
Revised: E ρ π

Original: [ (cid:80) Ti =

Revised: [ (cid:80) Ti =

Original: γ
Revised: γ

Original: i r ( s i , a i )] , where T is the horizon of the MDP, γ ∈ [0 , 1) is the discounting factor, and ρ π = { ( s 0 , a 0 ) , ( s 1 , a 1 ) , . . .
Revised: i r ( s i , a i )] , where T is the horizon of the MDP, γ ∈ [0 , 1) is the discounting factor, and ρ π = { ( s 0 , a 0 ) , ( s 1 , a 1 ) , . . .

Original: | a i =
Revised: | a i =

Original: π ( s i ) } is the sequence of states and actions drawn from the trajectory distribution determined by π .
Revised: π ( s i ) } is the sequence of states and actions drawn from the trajectory distribution determined by π .

Original: In our case, the elements of A are determined on-the-ﬂy following an RL policy π θ , parameterized by θ , conditioned on some states that characterize the status of the back-propagation at the current time step.
Revised: In our case, the elements of A are determined on-the-ﬂy following an RL policy π θ , parameterized by θ , conditioned on some states that characterize the status of the back-propagation at the current time step.

Original: The policy π θ is updated, concurrently with the weights W enc , by an RL agent that interacts with the back-propagation process modeled as the MDP deﬁned as M = ( S , A , P , R, γ ) , with each of its elements introduced below.
Revised: The policy π θ is updated, concurrently with the weights W enc , by an RL agent that interacts with the back-propagation process modeled as the MDP deﬁned as M = ( S , A , P , R, γ ) , with each of its elements introduced below.

Original: State Space S .
Revised: State Space S .

Original: The state space characterizes the agent’s knowledge of the environment at each time step.
Revised: The state space characterizes the agent’s knowledge of the environment at each time step.

Original: To constitute the states we consider the 4-tuple s = ( x , m , ζ, ˆy ) including current training input x ∈ R d , the missing indicator m ∈ R d , the feature ζ , i.e. , the outputs of the encoding layer of
Revised: To constitute the states we consider the 4-tuple s = ( x , m , ζ, ˆy ) including current training input x ∈ R d , the missing indicator m ∈ R d , the feature ζ , i.e. , the outputs of the encoding layer of

Original: MLPs or the hidden states h t of LSTMs, and the predictions ˆy produced by the inference layers.
Revised: MLPs or the hidden states h t of LSTMs, and the predictions ˆy produced by the inference layers.

Original: Action Space A .
Revised: Action Space A .

Original: We consider combining the importance matrix A ∈ [0 , 1] e × d with the parameter gradients ( ∂E/∂ W enc ) SGD when updating W enc following
Revised: We consider combining the importance matrix A ∈ [0 , 1] e × d with the parameter gradients ( ∂E/∂ W enc ) SGD when updating W enc following

Original: [Equation]
Revised: [Equation]

Original: where here the equality follows from (2) and Proposition 1, such that all rows of A can be set to be equal to the importance a (cid:62) ∈
Revised: where here the equality follows from (2) and Proposition 1, such that all rows of A can be set to be equal to the importance a (cid:62) ∈

Original: [0 , 1] d , which is obtained from the policy following a = π θ ( s ) .
Revised: [0 , 1] d , which is obtained from the policy following a = π θ ( s ) .

Original: Transitions P : S ×
Revised: Transitions P : S ×

Original: A → S .
Revised: A → S .

Original: The transition dynamics determines how to transit from a current state s to the next state s (cid:48) given an action a in the environment.
Revised: The transition dynamics determines how to transit from a current state s to the next state s (cid:48) given an action a in the environment.

Original: In our case, the pair ( x , m ) is sampled from the training dataset at each step, so the transitions x → x (cid:48) and m → m (cid:48) are determined by how training samples are selected from the training batch during back-propagation.
Revised: In our case, the pair ( x , m ) is sampled from the training dataset at each step, so the transitions x → x (cid:48) and m → m (cid:48) are determined by how training samples are selected from the training batch during back-propagation.

Original: The update of W enc → W (cid:48) enc is conditioned on the importance a as captured in (4), which results in the transitions ζ = f enc ( x | W enc )
Revised: The update of W enc → W (cid:48) enc is conditioned on the importance a as captured in (4), which results in the transitions ζ = f enc ( x | W enc )

Original: → ζ (cid:48)
Revised: → ζ (cid:48)

Original: =
Revised: =

Original: f enc ( x (cid:48) | W (cid:48) enc ) .
Revised: f enc ( x (cid:48) | W (cid:48) enc ) .

Original: The update of W inf → W (cid:48) inf follows from the regular SGD updates.
Revised: The update of W inf → W (cid:48) inf follows from the regular SGD updates.

Original: Then, the transition ˆy = f
Revised: Then, the transition ˆy = f

Original: ( x | W ) → ˆy (cid:48)
Revised: ( x | W ) → ˆy (cid:48)

Original: =
Revised: =

Original: f ( x (cid:48) | W (cid:48) ) follows from the other transitions deﬁned above.
Revised: f ( x (cid:48) | W (cid:48) ) follows from the other transitions deﬁned above.

Original: Finally, we can deﬁne the transition between states as s =
Revised: Finally, we can deﬁne the transition between states as s =

Original: ( x , m , ζ, ˆy ) → s (cid:48)
Revised: ( x , m , ζ, ˆy ) → s (cid:48)

Original: =
Revised: =

Original: ( x (cid:48) , m (cid:48) , ζ (cid:48) , ˆy (cid:48) ) .
Revised: ( x (cid:48) , m (cid:48) , ζ (cid:48) , ˆy (cid:48) ) .

Original: Reward Function R .
Revised: Reward Function R .

Original: After the RL agent takes an action a at the state s , an immediate reward r = R ( s , a ) is returned by the environment which provides essential information to update π θ (Silver et al., 2014; Lillicrap et al., 2015).
Revised: After the RL agent takes an action a at the state s , an immediate reward r = R ( s , a ) is returned by the environment which provides essential information to update π θ (Silver et al., 2014; Lillicrap et al., 2015).

Original: We deﬁne the reward function as R ( s , a ) = − E
Revised: We deﬁne the reward function as R ( s , a ) = − E

Original: ( ˆy , y ) .
Revised: ( ˆy , y ) .

Original: We utilize actor-critic RL (Silver et al., 2014; Lillicrap et al., 2015) to update π θ , which outputs the importance a that is used to concurrently update W enc .
Revised: We utilize actor-critic RL (Silver et al., 2014; Lillicrap et al., 2015) to update π θ , which outputs the importance a that is used to concurrently update W enc .

Original: Speciﬁcally, we train the target policy (or actor)
Revised: Speciﬁcally, we train the target policy (or actor)

Original: π θ along with the critic Q ν : S ×
Revised: π θ along with the critic Q ν : S ×

Original: A → R , parameterized by ν , by maximizing J β ( π θ ) =
Revised: A → R , parameterized by ν , by maximizing J β ( π θ ) =

Original: E s ∼ ρ β
Revised: E s ∼ ρ β

Original: [ Q ν ( s, π θ ( s ))] which gives an approximation to the expected total reward J ( π ) .
Revised: [ Q ν ( s, π θ ( s ))] which gives an approximation to the expected total reward J ( π ) .

Original: Speciﬁcally, the trajectories ρ β
Revised: Speciﬁcally, the trajectories ρ β

Original: = { ( s 0 , a 0 ) , ( s 1 , a 1 ) , . . .
Revised: = { ( s 0 , a 0 ) , ( s 1 , a 1 ) , . . .

Original: | a i = β
Revised: | a i = β

Original: ( s i ) } collected under the behavioral policy β : S
Revised: ( s i ) } collected under the behavioral policy β : S

Original: → A are used to update θ and ν jointly following
Revised: → A are used to update θ and ν jointly following

Original: [Equation]
Revised: [Equation]

Original: where β is usually obtained by adding noise to the output of π θ to ensure sufﬁcient exploration of the state and action space,
Revised: where β is usually obtained by adding noise to the output of π θ to ensure sufﬁcient exploration of the state and action space,

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: ( s , a ) is the temporal difference error (residual) in RL, and α θ , α ν are the learning rates for θ, ν , respectively.
Revised: ( s , a ) is the temporal difference error (residual) in RL, and α θ , α ν are the learning rates for θ, ν , respectively.

Original: , W enc ← W (cid:48) enc , W inf ← W (cid:48) inf 16: end for
Revised: , W enc ← W (cid:48) enc , W inf ← W (cid:48) inf 16: end for

Original: We summarize the GIL approach in Algorithm 1 and the detailed descriptions can be found in
Revised: We summarize the GIL approach in Algorithm 1 and the detailed descriptions can be found in

Original: Appendix B. Note that the missing indicator m
Revised: Appendix B. Note that the missing indicator m

Original: ∈ R d would be a good heuristic to replace the importance a as it could prevent the gradients produced by the missing dimensions from propagation.
Revised: ∈ R d would be a good heuristic to replace the importance a as it could prevent the gradients produced by the missing dimensions from propagation.

Original: However, it does not train the model to capture the hidden information underlying the missing data, which results in its performance being dominated by GIL as demonstrated in Section 4.
Revised: However, it does not train the model to capture the hidden information underlying the missing data, which results in its performance being dominated by GIL as demonstrated in Section 4.

Original: 2. E XTENSIONS OF THE F RAMEWORK
Revised: 2. E XTENSIONS OF THE F RAMEWORK

Original: The proposed GIL framework uses RL agents to guide the model toward minimizing its objective
Revised: The proposed GIL framework uses RL agents to guide the model toward minimizing its objective

Original: E (ˆ y , y ) , which is usually captured by general prediction losses such as cross entropy or mean squared error.
Revised: E (ˆ y , y ) , which is usually captured by general prediction losses such as cross entropy or mean squared error.

Original: Below we use an example to show how our method can be extended to adapt ideas from related domains, which can augment training of the imputation-free prediction models by altering the reward function.
Revised: Below we use an example to show how our method can be extended to adapt ideas from related domains, which can augment training of the imputation-free prediction models by altering the reward function.

Original: Recent works in contrastive learning ( e.g. , Chen et al.
Revised: Recent works in contrastive learning ( e.g. , Chen et al.

Original: (2020)) can train
Revised: (2020)) can train

Original: DL models to generate highly expressive features by penalizing (or promoting) the distributional difference D ( ζ + , ζ − ) among the features associated with inputs that are similar to (or different from) each other, where ζ + denotes the set of features corresponding to one set of homogeneous inputs
Revised: DL models to generate highly expressive features by penalizing (or promoting) the distributional difference D ( ζ + , ζ − ) among the features associated with inputs that are similar to (or different from) each other, where ζ + denotes the set of features corresponding to one set of homogeneous inputs

Original: X con , and ζ − are the features generated by data that are signiﬁcantly different from the ones in X con .
Revised: X con , and ζ − are the features generated by data that are signiﬁcantly different from the ones in X con .

Original: However, such methods may not be directly applied to the missing data setting considered in this work, as its loss D is usually designed toward unsupervised training and might need to be carefully re-worked in our case.
Revised: However, such methods may not be directly applied to the missing data setting considered in this work, as its loss D is usually designed toward unsupervised training and might need to be carefully re-worked in our case.

Original: This idea can be adapted to our framework by deﬁning ζ + as the features mapped from the inputs (by the encoding layers W enc ) associated with the same label y ∈ Y and ζ − the features corresponding to some other label y (cid:48)
Revised: This idea can be adapted to our framework by deﬁning ζ + as the features mapped from the inputs (by the encoding layers W enc ) associated with the same label y ∈ Y and ζ − the features corresponding to some other label y (cid:48)

Original: ∈ Y .
Revised: ∈ Y .

Original: Then we can deﬁne the new reward function
Revised: Then we can deﬁne the new reward function

Original: R ( s , a ) = − E (ˆ y , y )
Revised: R ( s , a ) = − E (ˆ y , y )

Original: + c · D ( ζ + , ζ − ) , c > 0 , which does not require D to be carefully crafted, provided that W enc is not trained by directly propagating gradients from it.
Revised: + c · D ( ζ + , ζ − ) , c > 0 , which does not require D to be carefully crafted, provided that W enc is not trained by directly propagating gradients from it.

Original: In Section 4 it will be shown in case studies that by simply deﬁning D as the mean squared error between ζ + and ζ − can improve the prediction performance.
Revised: In Section 4 it will be shown in case studies that by simply deﬁning D as the mean squared error between ζ + and ζ − can improve the prediction performance.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Missing Data Imputation.
Revised: Missing Data Imputation.

Original: Traditional mean/median-ﬁlling and carrying-forward imputation methods are still used widely, as they are straightforward to implement and interpret (Honaker & King, 2010).
Revised: Traditional mean/median-ﬁlling and carrying-forward imputation methods are still used widely, as they are straightforward to implement and interpret (Honaker & King, 2010).

Original: Recently, there have been state-of-the-art imputation algorithms (IAs) proposed to produce smooth imputations with interpretable uncertainty estimates.
Revised: Recently, there have been state-of-the-art imputation algorithms (IAs) proposed to produce smooth imputations with interpretable uncertainty estimates.

Original: Speciﬁcally, some adopt Bayesian methods where the observed data are ﬁt to data-generating models, including Gaussian processes (Wilson et al., 2016; Fortuin & Rätsch, 2019; Fortuin et al., 2018), multivariate imputation by chained equations (MICE) (Azur et al., 2011), random forests (Stekhoven & Bühlmann, 2012), etc ., or statistical optimization methods such as expectation-maximization (EM) (García-Laencina et al., 2010; Bashir & Wei, 2018).
Revised: Speciﬁcally, some adopt Bayesian methods where the observed data are ﬁt to data-generating models, including Gaussian processes (Wilson et al., 2016; Fortuin & Rätsch, 2019; Fortuin et al., 2018), multivariate imputation by chained equations (MICE) (Azur et al., 2011), random forests (Stekhoven & Bühlmann, 2012), etc ., or statistical optimization methods such as expectation-maximization (EM) (García-Laencina et al., 2010; Bashir & Wei, 2018).

Original: However, they often suffer from limited scalability, require assumptions over the underlying distribution, or fail to generalize well when used with datasets containing mixed types of variables, i.e. , when discrete and continuous values exist simultaneously (Yoon et al., 2018; Fortuin et al., 2020).
Revised: However, they often suffer from limited scalability, require assumptions over the underlying distribution, or fail to generalize well when used with datasets containing mixed types of variables, i.e. , when discrete and continuous values exist simultaneously (Yoon et al., 2018; Fortuin et al., 2020).

Original: There also exist matrix-completion methods that usually assume the data are static, i.e. , information does not change over time, and often rely on low-rank assumptions (Wang et al., 2006; Yu et al., 2016; Mazumder et al., 2010; Schnabel et al., 2016).
Revised: There also exist matrix-completion methods that usually assume the data are static, i.e. , information does not change over time, and often rely on low-rank assumptions (Wang et al., 2006; Yu et al., 2016; Mazumder et al., 2010; Schnabel et al., 2016).

Original: More recently, several DL-based
Revised: More recently, several DL-based

Original: IAs have been proposed following advancements in deep generative models such as deep latent variable models (DLVMs) and generative adversarial networks (GANs) (Mattei & Frellsen, 2019;
Revised: IAs have been proposed following advancements in deep generative models such as deep latent variable models (DLVMs) and generative adversarial networks (GANs) (Mattei & Frellsen, 2019;

Original: Yoon et al., 2018; Fortuin et al., 2020; Li et al., 2019; Ma et al., 2018).
Revised: Yoon et al., 2018; Fortuin et al., 2020; Li et al., 2019; Ma et al., 2018).

Original: DLVMs are usually trained to capture the underlying distribution of the data, by maximizing an evidence lower bound (ELBO), which could introduce high variations to the output distributions (Kingma & Welling, 2013) and cause poor performance in practice, if the data have high missingness (as for the example shown in
Revised: DLVMs are usually trained to capture the underlying distribution of the data, by maximizing an evidence lower bound (ELBO), which could introduce high variations to the output distributions (Kingma & Welling, 2013) and cause poor performance in practice, if the data have high missingness (as for the example shown in

Original: Figure 1).
Revised: Figure 1).

Original: There also exist end-to-end methods that withhold observed values from inputs and impose reconstruction losses during training of prediction models (Cao et al., 2018; Ipsen et al., 2020).
Revised: There also exist end-to-end methods that withhold observed values from inputs and impose reconstruction losses during training of prediction models (Cao et al., 2018; Ipsen et al., 2020).

Original: In addition, data-driven IAs are developed speciﬁcally toward medical time series by incorporating prior domain knowledge (Yang et al., 2018; Scherpf et al., 2019; Calvert et al., 2016; Kam & Kim, 2017).
Revised: In addition, data-driven IAs are developed speciﬁcally toward medical time series by incorporating prior domain knowledge (Yang et al., 2018; Scherpf et al., 2019; Calvert et al., 2016; Kam & Kim, 2017).

Original: On the other hand, a few recent works attempt to address missing inputs through an imputation-free manner (Morvan et al., 2020;
Revised: On the other hand, a few recent works attempt to address missing inputs through an imputation-free manner (Morvan et al., 2020;

Original: Sportisse et al., 2020), however, they are limited to linear regression.
Revised: Sportisse et al., 2020), however, they are limited to linear regression.

Original: Attention.
Revised: Attention.

Original: The importance matrix used in the GIL is somewhat reminiscent of visual attention mechanisms in the context of image captioning or multi-object recognition (Mnih et al., 2014;
Revised: The importance matrix used in the GIL is somewhat reminiscent of visual attention mechanisms in the context of image captioning or multi-object recognition (Mnih et al., 2014;

Original: Ba et al., 2014; Xu et al., 2015) as they both require training of the prediction models with RL.
Revised: Ba et al., 2014; Xu et al., 2015) as they both require training of the prediction models with RL.

Original: We brieﬂy discuss the connections and distinctions between them, with full details provided in
Revised: We brieﬂy discuss the connections and distinctions between them, with full details provided in

Original: Appendix C. Visual attentions are commonly used to train CNNs to focus on speciﬁc portions of the inputs that are most helpful for making predictions.
Revised: Appendix C. Visual attentions are commonly used to train CNNs to focus on speciﬁc portions of the inputs that are most helpful for making predictions.

Original: They are determined by maximizing an evidence lower bound (ELBO), which is later proved to be equivalent to the REINFORCE algorithm in the RL literature (Sutton & Barto, 2018; Mnih et al., 2014).
Revised: They are determined by maximizing an evidence lower bound (ELBO), which is later proved to be equivalent to the REINFORCE algorithm in the RL literature (Sutton & Barto, 2018; Mnih et al., 2014).

Original: However, these methods cannot be applied directly to our problem, as they require features to be exclusively associated spatially with speciﬁc parts of the inputs, which is attainable by using convolutional encoders with image inputs but intractable with the type of data considered in our work.
Revised: However, these methods cannot be applied directly to our problem, as they require features to be exclusively associated spatially with speciﬁc parts of the inputs, which is attainable by using convolutional encoders with image inputs but intractable with the type of data considered in our work.

Original: Instead, our approach overcomes this issue by directly applying importance weights, generated by RL, into the gradient space during back-propagation.
Revised: Instead, our approach overcomes this issue by directly applying importance weights, generated by RL, into the gradient space during back-propagation.

Original: Such issues motivate use of the term importance instead of attention .
Revised: Such issues motivate use of the term importance instead of attention .

Original: Lastly, our method does not require one to formulate the learning objective as an ELBO, and as a result GIL can adopt any state-of-the-art RL algorithm, without being limited to REINFORCE as in Mnih et al.
Revised: Lastly, our method does not require one to formulate the learning objective as an ELBO, and as a result GIL can adopt any state-of-the-art RL algorithm, without being limited to REINFORCE as in Mnih et al.

Original: (2014); Ba et al.
Revised: (2014); Ba et al.

Original: Xu et al. (2015).
Revised: Xu et al. (2015).

Original: Besides, other types of attention mechanisms are also proposed toward applications in sequence modeling and natural language processing (NLP) such as Cheng et al.
Revised: Besides, other types of attention mechanisms are also proposed toward applications in sequence modeling and natural language processing (NLP) such as Cheng et al.

Original: ; Vaswani et al.
Revised: ; Vaswani et al.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate the performance of the imputation-free prediction models trained by GIL against the existing imputation-prediction paradigm on both benchmark and real-world datasets, where the imputation stage employs both state-of-the-art and classic imputation algorithms (IAs), with the details introduced in Section 4.1.
Revised: We evaluate the performance of the imputation-free prediction models trained by GIL against the existing imputation-prediction paradigm on both benchmark and real-world datasets, where the imputation stage employs both state-of-the-art and classic imputation algorithms (IAs), with the details introduced in Section 4.1.

Original: We also consider variations of the GIL method proposed in Sectionto illustrate its robustness and ﬂexibility.
Revised: We also consider variations of the GIL method proposed in Sectionto illustrate its robustness and ﬂexibility.

Original: The datasets we use include i ) MIMIC-III (Johnson et al., 2016) that consists of real-world EHRs obtained from intensive care units (ICUs), ii ) a de-identiﬁed ophthalmic patient dataset obtained from an eye center in North America, and iii ) hand-written digits MNIST (LeCun & Cortes, 2010).
Revised: The datasets we use include i ) MIMIC-III (Johnson et al., 2016) that consists of real-world EHRs obtained from intensive care units (ICUs), ii ) a de-identiﬁed ophthalmic patient dataset obtained from an eye center in North America, and iii ) hand-written digits MNIST (LeCun & Cortes, 2010).

Original: We also tested on a smaller scaled ICU time-series from 2012 Physionet challenge (Silva et al., 2012) and these results can be found in Appendix D.4.
Revised: We also tested on a smaller scaled ICU time-series from 2012 Physionet challenge (Silva et al., 2012) and these results can be found in Appendix D.4.

Original: Some of the data are augmented with additional missingness to ensure sufﬁcient missing rates and the datasets we use cover all types of missing data, i.e., missing complete at random (MCAR), MAR and MNAR (Little & Rubin, 2019).
Revised: Some of the data are augmented with additional missingness to ensure sufﬁcient missing rates and the datasets we use cover all types of missing data, i.e., missing complete at random (MCAR), MAR and MNAR (Little & Rubin, 2019).

Original: We found that the proposed method not only outperforms the baselines on all three datasets under various experimental settings, but also offers better feature representations as shown in Figure 3.
Revised: We found that the proposed method not only outperforms the baselines on all three datasets under various experimental settings, but also offers better feature representations as shown in Figure 3.

Original: We start with an overview of the baseline methods, in the following sub-section, and then proceed to present our experimental ﬁndings.
Revised: We start with an overview of the baseline methods, in the following sub-section, and then proceed to present our experimental ﬁndings.

Original: 4. V ARIANTS OF GIL AND B ASELINES
Revised: 4. V ARIANTS OF GIL AND B ASELINES

Original: Our method (GIL) trains MLPs or LSTMs (depending on the type of data) to directly output the predictions given incomplete inputs without imputation.
Revised: Our method (GIL) trains MLPs or LSTMs (depending on the type of data) to directly output the predictions given incomplete inputs without imputation.

Original: Following from Section 2.4, we also test on
Revised: Following from Section 2.4, we also test on

Original: [Table]
Revised: [Table]

Original: binary classiﬁcation tasks using GIL-D which includes the distributional difference term D ( ζ + , ζ − ) , captured by mean squared errors, into the reward function.
Revised: binary classiﬁcation tasks using GIL-D which includes the distributional difference term D ( ζ + , ζ − ) , captured by mean squared errors, into the reward function.

Original: For baselines, we start with another variant of GIL that uses a simple heuristic – the missing indicator m – to replace the importance a in GIL; we denote it as GIL-H.
Revised: For baselines, we start with another variant of GIL that uses a simple heuristic – the missing indicator m – to replace the importance a in GIL; we denote it as GIL-H.

Original: This helps analyze the advantages of training the models using the importance obtained from the RL policy learned by GIL, versus a heuristic that simply discards the gradients produced by the subset of input entries that are missing.
Revised: This helps analyze the advantages of training the models using the importance obtained from the RL policy learned by GIL, versus a heuristic that simply discards the gradients produced by the subset of input entries that are missing.

Original: For other baselines, following the imputation-prediction framework, the imputation stage employs state-of-the-art IAs including MIWAE (Mattei & Frellsen, 2019) which imputes the missing data by training variational auto-encoders (VAEs) to maximize an ELBO, GP-VAE (Fortuin et al., 2020) that couples VAEs with Gaussian processes (GPs) to consider the temporal correlation in time-series, and the generative adversarial network-based method GAIN (Yoon et al., 2018).
Revised: For other baselines, following the imputation-prediction framework, the imputation stage employs state-of-the-art IAs including MIWAE (Mattei & Frellsen, 2019) which imputes the missing data by training variational auto-encoders (VAEs) to maximize an ELBO, GP-VAE (Fortuin et al., 2020) that couples VAEs with Gaussian processes (GPs) to consider the temporal correlation in time-series, and the generative adversarial network-based method GAIN (Yoon et al., 2018).

Original: Further, some classical imputation methods are also considered, including MICE, missForest (MF), k -nearest neighbor (kNN), expectation-maximization (EM), mean-imputation (Mean), zero-imputation (Zero) and carrying-forward (CF).
Revised: Further, some classical imputation methods are also considered, including MICE, missForest (MF), k -nearest neighbor (kNN), expectation-maximization (EM), mean-imputation (Mean), zero-imputation (Zero) and carrying-forward (CF).

Original: The imputed data from these baselines are fed into the prediction models that share the same architecture as the ones trained by GIL, for fair comparison.
Revised: The imputed data from these baselines are fed into the prediction models that share the same architecture as the ones trained by GIL, for fair comparison.

Original: For time-series, we also compare to BRITS (Cao et al., 2018) which trains bidirectional LSTMs end-to-end by masking out additional observed data and jointly imposing imputation and classiﬁcation objectives.
Revised: For time-series, we also compare to BRITS (Cao et al., 2018) which trains bidirectional LSTMs end-to-end by masking out additional observed data and jointly imposing imputation and classiﬁcation objectives.

Original: 4. MIMIC-III
Revised: 4. MIMIC-III

Original: We consider the early prediction of septic shock using the MIMIC-III dataset, following the frameworks provided in recent data-driven works for data pre-processing (Sheetrit et al., 2017; Fleuren et al., 2020; Khoshnevisan et al., 2020).
Revised: We consider the early prediction of septic shock using the MIMIC-III dataset, following the frameworks provided in recent data-driven works for data pre-processing (Sheetrit et al., 2017; Fleuren et al., 2020; Khoshnevisan et al., 2020).

Original: Speciﬁcally, we use 14 commonly-utilized vital signs and lab results over a 2-hour observation window to predict whether a patient will develop septic shock in the next 4-hour window.
Revised: Speciﬁcally, we use 14 commonly-utilized vital signs and lab results over a 2-hour observation window to predict whether a patient will develop septic shock in the next 4-hour window.

Original: The resulting dataset used for training and testing contains 2,166 sequences each of which has length between 3 and 125, and the overall missing rate is 71.63%; the missing data in this dataset can be considered as a mixture of MCAR, MAR and MNAR.
Revised: The resulting dataset used for training and testing contains 2,166 sequences each of which has length between 3 and 125, and the overall missing rate is 71.63%; the missing data in this dataset can be considered as a mixture of MCAR, MAR and MNAR.

Original: More details can be found in Appendix D.1.
Revised: More details can be found in Appendix D.1.

Original: To evaluate performance, we consider the prediction model constituted by a 1024-unit LSTM layer for encoding followed by a dense output layer for inference.
Revised: To evaluate performance, we consider the prediction model constituted by a 1024-unit LSTM layer for encoding followed by a dense output layer for inference.

Original: Considering that some of the baseline methods are not designed to capture the temporal correlations within the sequences which do not follow a ﬁxed horizon, besides testing with the varied-length sequences (Var-l.) obtained in above, we also test with a ﬁxed-length version (Fix-l.) where the maximum time step for each sequence is set to be 8 (see Appendix D.1 for details).
Revised: Considering that some of the baseline methods are not designed to capture the temporal correlations within the sequences which do not follow a ﬁxed horizon, besides testing with the varied-length sequences (Var-l.) obtained in above, we also test with a ﬁxed-length version (Fix-l.) where the maximum time step for each sequence is set to be 8 (see Appendix D.1 for details).

Original: The accuracy 1 and AUC for the two tests are summarized in Table 1.
Revised: The accuracy 1 and AUC for the two tests are summarized in Table 1.

Original: When varied-length sequences are considered, GIL(-D) slightly outperforms GAIN and MICE while it signiﬁcantly dominates the other baseline methods.
Revised: When varied-length sequences are considered, GIL(-D) slightly outperforms GAIN and MICE while it signiﬁcantly dominates the other baseline methods.

Original: However, when ﬁxedlength sequences are considered, GAIN and MICE’s performance decreases dramatically and are signiﬁcantly dominated by that of GIL(-D).
Revised: However, when ﬁxedlength sequences are considered, GAIN and MICE’s performance decreases dramatically and are signiﬁcantly dominated by that of GIL(-D).

Original: This could be caused by omitting the records beyond the 8-th time step as both models are ﬂexible enough to capture such information from that point onwards, while in contrast, the performance increases for kNN, MF and EM which are in general based on much simpler models.
Revised: This could be caused by omitting the records beyond the 8-th time step as both models are ﬂexible enough to capture such information from that point onwards, while in contrast, the performance increases for kNN, MF and EM which are in general based on much simpler models.

Original: On the other hand, the models trained by GIL(-D) was not signiﬁcantly affected by the information loss.
Revised: On the other hand, the models trained by GIL(-D) was not signiﬁcantly affected by the information loss.

Original: In fact, it emphasizes that applying importance to the gradients during training can enable the model to capture the information behind missing data.
Revised: In fact, it emphasizes that applying importance to the gradients during training can enable the model to capture the information behind missing data.

Original: Moreover, the dramatically increased performance from GIL-H to GIL(-D) in both tests underscores the signiﬁcance of training the models using the importance determined by the RL policy learned by GIL(-D), instead of a pre-deﬁned heuristic.
Revised: Moreover, the dramatically increased performance from GIL-H to GIL(-D) in both tests underscores the signiﬁcance of training the models using the importance determined by the RL policy learned by GIL(-D), instead of a pre-deﬁned heuristic.

Original: Note that according to a recent survey of septic shock predictions (Fleuren et al., 2020), the overall maximum AUC attained by existing data-driven methods, which uses domain knowledge to design models speciﬁcally for ICU time-series, is 0.96 and it is comparable to that attained by our method.
Revised: Note that according to a recent survey of septic shock predictions (Fleuren et al., 2020), the overall maximum AUC attained by existing data-driven methods, which uses domain knowledge to design models speciﬁcally for ICU time-series, is 0.96 and it is comparable to that attained by our method.

Original: Finally, GP-VAE and BRITS require the input sequences to have the same horizon, so we only report for the Fix-l. case where they under-perform.
Revised: Finally, GP-VAE and BRITS require the input sequences to have the same horizon, so we only report for the Fix-l. case where they under-perform.

Original: These are possibly due to
Revised: These are possibly due to

Original: [Table]
Revised: [Table]

Original: the high variation in GPVAE’s uncertainty estimation component, and the mechanism of imposing additional missingness in BRITS.
Revised: the high variation in GPVAE’s uncertainty estimation component, and the mechanism of imposing additional missingness in BRITS.

Original: 4. O PHTHALMIC D ATA
Revised: 4. O PHTHALMIC D ATA

Original: We consider identifying diabetic retinopathy (DR), an eye disease caused by diabetes, using the deidentiﬁed data obtained from an eye clinic constituted by a mixture of image feature vectors and tabular information.
Revised: We consider identifying diabetic retinopathy (DR), an eye disease caused by diabetes, using the deidentiﬁed data obtained from an eye clinic constituted by a mixture of image feature vectors and tabular information.

Original: Speciﬁcally, the data collected from each subject is formulated as a vector with 4, dimensions containing two 2,048-length feature vectors from two retinal imaging modalities, optic coherence tomography (OCT) and color fundus photography (CFP), followed by a 5-dimensional data characterizing demographic ( e.g. , age) and diabetic information ( e.g. , years of diabetes).
Revised: Speciﬁcally, the data collected from each subject is formulated as a vector with 4, dimensions containing two 2,048-length feature vectors from two retinal imaging modalities, optic coherence tomography (OCT) and color fundus photography (CFP), followed by a 5-dimensional data characterizing demographic ( e.g. , age) and diabetic information ( e.g. , years of diabetes).

Original: Additional details of the dataset and examples of the OCT and CFP images are provided in the Appendix D.2.
Revised: Additional details of the dataset and examples of the OCT and CFP images are provided in the Appendix D.2.

Original: At most one of the two types of retinal images could be missing due to MAR as sometimes the diagnosis can be performed with a single modality (Cui et al., 2021), while the demographic/diabetic information can be missing due to MCAR.
Revised: At most one of the two types of retinal images could be missing due to MAR as sometimes the diagnosis can be performed with a single modality (Cui et al., 2021), while the demographic/diabetic information can be missing due to MCAR.

Original: A total of 1,148 subjects are included in this dataset with a 17% missing rate (M.R.) 3 .
Revised: A total of 1,148 subjects are included in this dataset with a 17% missing rate (M.R.) 3 .

Original: We apply additional random masks over both the input image features (by assuming one or two entire image feature vectors are missing) and demographic/diabetic information (by assuming some entries are missing) to increase the missing rate to 25% and 35%, where for each case 3 different random masks are applied.
Revised: We apply additional random masks over both the input image features (by assuming one or two entire image feature vectors are missing) and demographic/diabetic information (by assuming some entries are missing) to increase the missing rate to 25% and 35%, where for each case 3 different random masks are applied.

Original: For this experiment we consider an MLP with 2 hidden layers with 1,000 nodes each for prediction.
Revised: For this experiment we consider an MLP with 2 hidden layers with 1,000 nodes each for prediction.

Original: The results are shown in Table 2 where the mean and standard deviations (in subscripts) of accuracy and AUC are reported.
Revised: The results are shown in Table 2 where the mean and standard deviations (in subscripts) of accuracy and AUC are reported.

Original: Although GIL and GP-VAE result in similar performance in the 25% missing rate case, GIL still achieves higher accuracy and AUC; these are signiﬁcantly improved, with signiﬁcantly lower standard deviation over GP-VAE, when the missing rate increases to 35%.
Revised: Although GIL and GP-VAE result in similar performance in the 25% missing rate case, GIL still achieves higher accuracy and AUC; these are signiﬁcantly improved, with signiﬁcantly lower standard deviation over GP-VAE, when the missing rate increases to 35%.

Original: In general, GIL-D outperforms all the other methods, with an exception of the higher standard deviation of accuracy in the 35% M.R. case, which beneﬁts from the ﬂexibility of the framework we proposed.
Revised: In general, GIL-D outperforms all the other methods, with an exception of the higher standard deviation of accuracy in the 35% M.R. case, which beneﬁts from the ﬂexibility of the framework we proposed.

Original: With increased missingness, the performance of most baselines drops signiﬁcantly and are close to the zero-imputation baseline as the image feature vectors occupy most dimensions of the inputs while imputing with zeros would be a reasonable heuristic.
Revised: With increased missingness, the performance of most baselines drops signiﬁcantly and are close to the zero-imputation baseline as the image feature vectors occupy most dimensions of the inputs while imputing with zeros would be a reasonable heuristic.

Original: 4. MNIST
Revised: 4. MNIST

Original: This work focuses mostly on tabular inputs or time-series; however, we test on MNIST images given its simple structure, which could be classiﬁed using MLPs and more importantly, because results from these data are easier to interpret.
Revised: This work focuses mostly on tabular inputs or time-series; however, we test on MNIST images given its simple structure, which could be classiﬁed using MLPs and more importantly, because results from these data are easier to interpret.

Original: Speciﬁcally, we test on the MCAR version of MNIST where a pre-determined portion of pixels ( i.e. , out of 50%, 70% and 90%) are masked off uniformly at random from the original images, and the MAR version, which assumes part of the image is always observable and the positions of missing pixels are determined by a distribution conditioned on features extracted from the observable portion following Mattei & Frellsen (2019).
Revised: Speciﬁcally, we test on the MCAR version of MNIST where a pre-determined portion of pixels ( i.e. , out of 50%, 70% and 90%) are masked off uniformly at random from the original images, and the MAR version, which assumes part of the image is always observable and the positions of missing pixels are determined by a distribution conditioned on features extracted from the observable portion following Mattei & Frellsen (2019).

Original: For each test, the masks are applied
Revised: For each test, the masks are applied

Original: [Table] [Table]
Revised: [Table] [Table]

Original: with 5 different random seeds, resulting in the standard deviations reported in Table 3 which also summarizes obtained results.
Revised: with 5 different random seeds, resulting in the standard deviations reported in Table 3 which also summarizes obtained results.

Original: We consider using MLPs constituted by two hidden dense layers with 500 nodes each as the prediction models 4 .
Revised: We consider using MLPs constituted by two hidden dense layers with 500 nodes each as the prediction models 4 .

Original: Note that in the MCAR setting, GP-VAE achieves slightly higher average accuracy than our method when 50% and 70% of the pixels missing, because it relies on convolutional encoding layers, which intrinsically have advantages on this dataset.
Revised: Note that in the MCAR setting, GP-VAE achieves slightly higher average accuracy than our method when 50% and 70% of the pixels missing, because it relies on convolutional encoding layers, which intrinsically have advantages on this dataset.

Original: However, GP-VAE’s performance is associated with higher standard deviation in both cases and is signiﬁcantly outperformed by our method when 90% pixels are missing.
Revised: However, GP-VAE’s performance is associated with higher standard deviation in both cases and is signiﬁcantly outperformed by our method when 90% pixels are missing.

Original: The models trained by GIL also signiﬁcantly outperforms the baselines following the imputation-prediction framework.
Revised: The models trained by GIL also signiﬁcantly outperforms the baselines following the imputation-prediction framework.

Original: This suggests that most of the IAs fail to reconstruct the input data when the missing rate is high (see Figure 1).
Revised: This suggests that most of the IAs fail to reconstruct the input data when the missing rate is high (see Figure 1).

Original: Moreover, the errors produced during the imputation are then propagated into the prediction step which results in the inferior performance.
Revised: Moreover, the errors produced during the imputation are then propagated into the prediction step which results in the inferior performance.

Original: In the MAR setting, the task becomes more challenging as entire rows of pixels could be missing.
Revised: In the MAR setting, the task becomes more challenging as entire rows of pixels could be missing.

Original: Finally, the model trained by GIL outperforms most of the baselines depending on IAs with an exception of MIWAE, which only slightly outperforms it.
Revised: Finally, the model trained by GIL outperforms most of the baselines depending on IAs with an exception of MIWAE, which only slightly outperforms it.

Original: Note that GIL-H’s performance is close to GIL in most of the settings due to the simple structure of the MNIST digits where the missing indicator m could be a good estimation of the importance a .
Revised: Note that GIL-H’s performance is close to GIL in most of the settings due to the simple structure of the MNIST digits where the missing indicator m could be a good estimation of the importance a .

Original: Feature Space Visualization
Revised: Feature Space Visualization

Original: In Figure 3, we use t -distributed stochastic neighbor embedding ( tSNE) to visualize the features learned by the prediction model trained by GIL compared to MIWAE and GAIN.
Revised: In Figure 3, we use t -distributed stochastic neighbor embedding ( tSNE) to visualize the features learned by the prediction model trained by GIL compared to MIWAE and GAIN.

Original: We observe that GIL results in more expressive features than the others as it generates clusters with clearer boundaries.
Revised: We observe that GIL results in more expressive features than the others as it generates clusters with clearer boundaries.

Original: 4. C ORRELATION BETWEEN I MPUTATION AND P REDICTION P ERFORMANCE
Revised: 4. C ORRELATION BETWEEN I MPUTATION AND P REDICTION P ERFORMANCE

Original: In this section we study if imputation error is correlated with the downstream prediction performance, under the imputation-prediction framework.
Revised: In this section we study if imputation error is correlated with the downstream prediction performance, under the imputation-prediction framework.

Original: Each column of Table 4 is obtained by calculating the
Revised: Each column of Table 4 is obtained by calculating the

Original: Pearson’s correlation coefﬁcient (Freedman et al., 2007) and its p -value, between the imputation mean squared error (MSE) and prediction accuracy, across different imputation methods under the same dataset.
Revised: Pearson’s correlation coefﬁcient (Freedman et al., 2007) and its p -value, between the imputation mean squared error (MSE) and prediction accuracy, across different imputation methods under the same dataset.

Original: It can be observed that the imputation MSE is negatively correlated with prediction performance for most of the datasets and data with higher M.R. tends to have higher degree of negative correlation, which indicates that imputation error could be propagated to downstream tasks.
Revised: It can be observed that the imputation MSE is negatively correlated with prediction performance for most of the datasets and data with higher M.R. tends to have higher degree of negative correlation, which indicates that imputation error could be propagated to downstream tasks.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have developed the GIL method, training DL models (MLPs and LSTMs) to directly perform inference with incomplete data, without the need for imputation.
Revised: We have developed the GIL method, training DL models (MLPs and LSTMs) to directly perform inference with incomplete data, without the need for imputation.

Original: Existing methods addressing the problem of missing data mostly follow the two-step (imputation-prediction) framework.
Revised: Existing methods addressing the problem of missing data mostly follow the two-step (imputation-prediction) framework.

Original: However, the error produced during imputation can be propagated into subsequent tasks, especially when the data have high missingness rates or small sample sizes.
Revised: However, the error produced during imputation can be propagated into subsequent tasks, especially when the data have high missingness rates or small sample sizes.

Original: GIL circumvents these issues by applying importance weighting to the gradients to leverage the information underlying the missingness patterns in the data.
Revised: GIL circumvents these issues by applying importance weighting to the gradients to leverage the information underlying the missingness patterns in the data.

Original: We have evaluated our method by comparing it to the state-of-the-art baselines using IAs on two real-world datasets and one benchmark dataset.
Revised: We have evaluated our method by comparing it to the state-of-the-art baselines using IAs on two real-world datasets and one benchmark dataset.

Original: A E XTENDING GIL TO LSTM S
Revised: A E XTENDING GIL TO LSTM S

Original: LSTM Model.
Revised: LSTM Model.

Original: We also consider the use of an LSTM as the encoder for sequential inputs with varied lengths.
Revised: We also consider the use of an LSTM as the encoder for sequential inputs with varied lengths.

Original: Speciﬁcally, in this case, we can deﬁne X j = ( x (cid:62) j, 1 , x (cid:62) j, 2 , . . .
Revised: Speciﬁcally, in this case, we can deﬁne X j = ( x (cid:62) j, 1 , x (cid:62) j, 2 , . . .

Original: , x (cid:62) j,T j ) (cid:62) ∈ R T j × d along with M j = ( m (cid:62) j, 1 , m (cid:62) j, 2 , . . .
Revised: , x (cid:62) j,T j ) (cid:62) ∈ R T j × d along with M j = ( m (cid:62) j, 1 , m (cid:62) j, 2 , . . .

Original: , m (cid:62) j,T i ) ∈ { 0 , 1 } T j × d such that each x j,t ∈ R d and m j,t ∈ { 0 , 1 } d , where i ∈ [1 , N ] and t ∈ [1 , T j ] .
Revised: , m (cid:62) j,T i ) ∈ { 0 , 1 } T j × d such that each x j,t ∈ R d and m j,t ∈ { 0 , 1 } d , where i ∈ [1 , N ] and t ∈ [1 , T j ] .

Original: Recall that the forward pass of an LSTM follows
Revised: Recall that the forward pass of an LSTM follows

Original: [Equation]
Revised: [Equation]

Original: where x t ∈ R d is the input at time step t , { W o , W i , W g , W f ,
Revised: where x t ∈ R d is the input at time step t , { W o , W i , W g , W f ,

Original: U o , U
Revised: U o , U

Original: i , U g , U f } are the weights, σ ( x )
Revised: i , U g , U f } are the weights, σ ( x )

Original: = 1 / (1 + e − x ) is the sigmoid function, and (cid:12) is the element-wise product.
Revised: = 1 / (1 + e − x ) is the sigmoid function, and (cid:12) is the element-wise product.

Original: The output layer is appended after h t to constitute the inference layer, i.e. ,
Revised: The output layer is appended after h t to constitute the inference layer, i.e. ,

Original: [Equation]
Revised: [Equation]

Original: where φ out is the activation function, W inf =
Revised: where φ out is the activation function, W inf =

Original: W out are the weights and t ∈ [1 , T j ] .
Revised: W out are the weights and t ∈ [1 , T j ] .

Original: The following proposition shows that given a loss function E ( ˆy , y ) , the gradients of E w.r.t the encoding weights
Revised: The following proposition shows that given a loss function E ( ˆy , y ) , the gradients of E w.r.t the encoding weights

Original: W enc =
Revised: W enc =

Original: { W o , W i , W g , W f } can be written in the form of outer products.
Revised: { W o , W i , W g , W f } can be written in the form of outer products.

Original: Note that the gradients for (autoregressive) parameters depending on previous hidden states h t − 1 , i.e., , { U o , U i , U g , U f } , like the bias terms, cannot be written as outer products; thus, these weights are updated following the regular SGD, without importance weighting.
Revised: Note that the gradients for (autoregressive) parameters depending on previous hidden states h t − 1 , i.e., , { U o , U i , U g , U f } , like the bias terms, cannot be written as outer products; thus, these weights are updated following the regular SGD, without importance weighting.

Original: The proof is provided following the proposition.
Revised: The proof is provided following the proposition.

Original: Proposition 2 Given an LSTM as described in (6)-(7) and a smooth loss function E ( ˆy , y ) , the gradients of E w.r.t.
Revised: Proposition 2 Given an LSTM as described in (6)-(7) and a smooth loss function E ( ˆy , y ) , the gradients of E w.r.t.

Original: W enc =
Revised: W enc =

Original: { W o , W i , W g , W f } at time t can be written in outer product forms – i.e., it holds that ∂E∂
Revised: { W o , W i , W g , W f } at time t can be written in outer product forms – i.e., it holds that ∂E∂

Original: [Equation]
Revised: [Equation]

Original: , where ∆ o t , ∆ i t , ∆ g t , ∆ f t are the gradients propagated from the inference layers deﬁned in below, and x t represents the observation obtained at time step t from any sequence X j ∈ X in general.
Revised: , where ∆ o t , ∆ i t , ∆ g t , ∆ f t are the gradients propagated from the inference layers deﬁned in below, and x t represents the observation obtained at time step t from any sequence X j ∈ X in general.

Original: Then it follows that if we deﬁne x as the t -th observation from any sequence X j ∈ R T j × d , regardless of it index, (4) can be directly used for training the LSTM encoding weights W enc =
Revised: Then it follows that if we deﬁne x as the t -th observation from any sequence X j ∈ R T j × d , regardless of it index, (4) can be directly used for training the LSTM encoding weights W enc =

Original: { W o , W i , W g , W f } .
Revised: { W o , W i , W g , W f } .

Original: Now we prove the Proposition above.
Revised: Now we prove the Proposition above.

Original: We ﬁrst deﬁne
Revised: We ﬁrst deﬁne

Original: [Equation]
Revised: [Equation]

Original: Now to prove the proposition we start from the derivative of the smooth loss function E w.r.t.
Revised: Now to prove the proposition we start from the derivative of the smooth loss function E w.r.t.

Original: x , which can be derived as
Revised: x , which can be derived as

Original: [Equation]
Revised: [Equation]

Original: Then the derivatives of E w.r.t.
Revised: Then the derivatives of E w.r.t.

Original: the o t , c t , f t , c t − 1 , i t and g t are
Revised: the o t , c t , f t , c t − 1 , i t and g t are

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Now, the derivatives of E w.r.t the weights W o , W
Revised: Now, the derivatives of E w.r.t the weights W o , W

Original: i , W g and W f are ∂E
Revised: i , W g and W f are ∂E

Original: [Equation]
Revised: [Equation]

Original: o t ∂ o t ∂ W o (21) =
Revised: o t ∂ o t ∂ W o (21) =

Original: (cid:20) ∂E
Revised: (cid:20) ∂E

Original: ∂ h t (cid:12) tanh( c t ) (cid:12)
Revised: ∂ h t (cid:12) tanh( c t ) (cid:12)

Original: o t (cid:12) (1 − o t ) (cid:21) x
Revised: o t (cid:12) (1 − o t ) (cid:21) x

Original: (cid:62) t (22) =
Revised: (cid:62) t (22) =

Original: ∆ o t x (cid:62) t (23) ∂E
Revised: ∆ o t x (cid:62) t (23) ∂E

Original: ∂ W i (cid:12) (cid:12)(cid:12) t =
Revised: ∂ W i (cid:12) (cid:12)(cid:12) t =

Original: ∂E ∂ h t ∂ h t ∂ c t ∂ c t ∂ i t ∂ i t
Revised: ∂E ∂ h t ∂ h t ∂ c t ∂ c t ∂ i t ∂ i t

Original: ∂ W i (24) = (cid:104) ∂E
Revised: ∂ W i (24) = (cid:104) ∂E

Original: ∂ h t (cid:12) o t (cid:12) (1 − tanh 2 ( c t )) (cid:12) g t (cid:12) i t (cid:12) (1 − i t ) (cid:105) x (cid:62) t (25) =
Revised: ∂ h t (cid:12) o t (cid:12) (1 − tanh 2 ( c t )) (cid:12) g t (cid:12) i t (cid:12) (1 − i t ) (cid:105) x (cid:62) t (25) =

Original: ∆ i t x (cid:62) t (26) ∂E
Revised: ∆ i t x (cid:62) t (26) ∂E

Original: ∂ W g (cid:12)(cid:12)(cid:12) t = ∂E
Revised: ∂ W g (cid:12)(cid:12)(cid:12) t = ∂E

Original: ∂ h t ∂ h t ∂ c t ∂ c t ∂
Revised: ∂ h t ∂ h t ∂ c t ∂ c t ∂

Original: g t ∂ g t ∂ W g (27) = (cid:104) ∂E
Revised: g t ∂ g t ∂ W g (27) = (cid:104) ∂E

Original: ∂ h t (cid:12) o t (cid:12) (1 − tanh 2 ( c t )) (cid:12) i t (cid:12) (1 − g 2 t ) (cid:105) x
Revised: ∂ h t (cid:12) o t (cid:12) (1 − tanh 2 ( c t )) (cid:12) i t (cid:12) (1 − g 2 t ) (cid:105) x

Original: (cid:62) t (28) =
Revised: (cid:62) t (28) =

Original: ∆ g t x (cid:62) t (29) ∂E
Revised: ∆ g t x (cid:62) t (29) ∂E

Original: ∂ W f (cid:12)(cid:12)(cid:12) t = ∂E ∂ h t ∂ h t ∂ c t ∂ c t ∂ f t ∂ f t ∂ W f (30) =
Revised: ∂ W f (cid:12)(cid:12)(cid:12) t = ∂E ∂ h t ∂ h t ∂ c t ∂ c t ∂ f t ∂ f t ∂ W f (30) =

Original: (cid:104) ∂E
Revised: (cid:104) ∂E

Original: ∂ h t (cid:12) o t (cid:12) (1 − tanh 2 ( c t )) (cid:12) c
Revised: ∂ h t (cid:12) o t (cid:12) (1 − tanh 2 ( c t )) (cid:12) c

Original: t − 1 (cid:12) f t (cid:12) (1 −
Revised: t − 1 (cid:12) f t (cid:12) (1 −

Original: [Equation]
Revised: [Equation]

Original: Note that since σ (cid:48) ( · ) = σ ( · )(1 − σ ( · )) , in the transition between (21) and (22) it follows that
Revised: Note that since σ (cid:48) ( · ) = σ ( · )(1 − σ ( · )) , in the transition between (21) and (22) it follows that

Original: [Equation]
Revised: [Equation]

Original: Similarly, the transition between (24) and (25) follows
Revised: Similarly, the transition between (24) and (25) follows

Original: [Equation]
Revised: [Equation]

Original: At last, the transition between (30) and (31) follows
Revised: At last, the transition between (30) and (31) follows

Original: [Equation]
Revised: [Equation]

Original: Furthemore, since tanh (cid:48) ( · ) = 1 − tanh 2 ( · ) , the transition between (27) and (28) follows
Revised: Furthemore, since tanh (cid:48) ( · ) = 1 − tanh 2 ( · ) , the transition between (27) and (28) follows

Original: [Equation]
Revised: [Equation]

Original: which concludes the proof.
Revised: which concludes the proof.

Original: B D ESCRIPTION OF A LGORITHM
Revised: B D ESCRIPTION OF A LGORITHM

Original: The algorithm takes as input the training dataset X , the training targets Y , the missing indicators M , the weights of the encoding layers W enc , the weights for the inference layers W inf , the actor π θ , the critic Q ν , learning rates { α, α θ , α ν } and training loss function E .
Revised: The algorithm takes as input the training dataset X , the training targets Y , the missing indicators M , the weights of the encoding layers W enc , the weights for the inference layers W inf , the actor π θ , the critic Q ν , learning rates { α, α θ , α ν } and training loss function E .

Original: Our approach starts by initializing all the parameters W enc , W inf , π θ , Q ν , sampling x ∈ X with the corresponding m ∈ M that will be used for training in the ﬁrst iteration, obtaining the feature ζ and the prediction ˆy , which constitute the initial state as s =
Revised: Our approach starts by initializing all the parameters W enc , W inf , π θ , Q ν , sampling x ∈ X with the corresponding m ∈ M that will be used for training in the ﬁrst iteration, obtaining the feature ζ and the prediction ˆy , which constitute the initial state as s =

Original: ( x , m , ζ, ˆy ) .
Revised: ( x , m , ζ, ˆy ) .

Original: In each iteration, ﬁrst the importance is generated from a behavioral policy β that is conditioned on the target policy π θ , such as the noisy exploration policy proposed in Lillicrap et al. (2015).
Revised: In each iteration, ﬁrst the importance is generated from a behavioral policy β that is conditioned on the target policy π θ , such as the noisy exploration policy proposed in Lillicrap et al. (2015).

Original: Then the encoding layer is trained following (4), while the inference layers are trained following the regular gradient descent.
Revised: Then the encoding layer is trained following (4), while the inference layers are trained following the regular gradient descent.

Original: After training, the new prediction is obtained following the updated weights and its value is assigned to ˆy , which is then used to generate the reward following the reward function R .
Revised: After training, the new prediction is obtained following the updated weights and its value is assigned to ˆy , which is then used to generate the reward following the reward function R .

Original: Then the training sample x (cid:48) for the next iteration is sampled and the corresponding m (cid:48) , ζ (cid:48) , ˆy (cid:48) are obtained to constitute the next state s (cid:48) .
Revised: Then the training sample x (cid:48) for the next iteration is sampled and the corresponding m (cid:48) , ζ (cid:48) , ˆy (cid:48) are obtained to constitute the next state s (cid:48) .

Original: Finally, the actor π θ and critic Q ν are updated following (5).
Revised: Finally, the actor π θ and critic Q ν are updated following (5).

Original: We refer to Silver et al.
Revised: We refer to Silver et al.

Original: ; Lillicrap et al.
Revised: ; Lillicrap et al.

Original: (2015) for more details on actor-critic RL.
Revised: (2015) for more details on actor-critic RL.

Original: C I MPORTANCE VS A TTENTIONS
Revised: C I MPORTANCE VS A TTENTIONS

Original: We now illustrate the connections and distinctions between the importance in the GIL and visual attentions Mnih et al.
Revised: We now illustrate the connections and distinctions between the importance in the GIL and visual attentions Mnih et al.

Original: Ba et al.
Revised: Ba et al.

Original: Xu et al. (2015), which are commonly used to train
Revised: Xu et al. (2015), which are commonly used to train

Original: CNNs to focus on speciﬁc dimensions of the inputs that are most helpful for making predictions in the context of image captioning and multi-object recognition.
Revised: CNNs to focus on speciﬁc dimensions of the inputs that are most helpful for making predictions in the context of image captioning and multi-object recognition.

Original: In visual attentions, an input image is ﬁrst encoded into a set of vectors I = { l 1 , . . .
Revised: In visual attentions, an input image is ﬁrst encoded into a set of vectors I = { l 1 , . . .

Original: , l L } where each l i is a feature vector corresponding to a speciﬁc region in the image as they are retrieved from lower convolutional layers.
Revised: , l L } where each l i is a feature vector corresponding to a speciﬁc region in the image as they are retrieved from lower convolutional layers.

Original: Then, the attention α t,i for time step t and region i ∈ [1 , L ] is generated following Bahdanau et al.
Revised: Then, the attention α t,i for time step t and region i ∈ [1 , L ] is generated following Bahdanau et al.

Original: (2014) as α t,i = exp( e t,i ) / (cid:80)
Revised: (2014) as α t,i = exp( e t,i ) / (cid:80)

Original: Lj =1 exp ( e t,k ) , where e t,i = f att ( l i , h t − 1 ) , f att is usually an MLP (or the socalled attention model) and h t − 1 is the hidden state of a recurrent neural network (RNN) Hochreiter & Schmidhuber (1997) used for prediction.
Revised: Lj =1 exp ( e t,k ) , where e t,i = f att ( l i , h t − 1 ) , f att is usually an MLP (or the socalled attention model) and h t − 1 is the hidden state of a recurrent neural network (RNN) Hochreiter & Schmidhuber (1997) used for prediction.

Original: Then, a weighted average of the feature vectors (cid:80) i l t,i l i is fed into the prediction network for further inference where l t,i is a random variable parametrized by α t,i following p ( l t,i = 1 | l j<t , I ) =
Revised: Then, a weighted average of the feature vectors (cid:80) i l t,i l i is fed into the prediction network for further inference where l t,i is a random variable parametrized by α t,i following p ( l t,i = 1 | l j<t , I ) =

Original: α t,i , and l j<t represents the historical values of l t,i for all i ∈ [1 , L ] .
Revised: α t,i , and l j<t represents the historical values of l t,i for all i ∈ [1 , L ] .

Original: Given the deﬁnition of the multinoulli variable l t,i , the model is not smooth and thus cannot be trained following the regular back-propagation.
Revised: Given the deﬁnition of the multinoulli variable l t,i , the model is not smooth and thus cannot be trained following the regular back-propagation.

Original: So the prediction network is trained to maximize the evidence lower bound (ELBO) which updates the prediction network parameter θ using
Revised: So the prediction network is trained to maximize the evidence lower bound (ELBO) which updates the prediction network parameter θ using

Original: [Equation]
Revised: [Equation]

Original: where ˜ l i are the Monte-Carlo samples drawn from p ( l t,i |
Revised: where ˜ l i are the Monte-Carlo samples drawn from p ( l t,i |

Original: l j<t , I ) and b represents the performance from a baseline E [log p ( y | θ baseline )] .
Revised: l j<t , I ) and b represents the performance from a baseline E [log p ( y | θ baseline )] .

Original: It is notable that (41) is equivalent to the REINFOCE algorithm in RL Williams (1992).
Revised: It is notable that (41) is equivalent to the REINFOCE algorithm in RL Williams (1992).

Original: Speciﬁcally, the search space can be interpreted as an MDP which is usually used to model the environment in RL problems – i.e., the state space is constituted by I , the RNN hidden state h t and the historical visitations
Revised: Speciﬁcally, the search space can be interpreted as an MDP which is usually used to model the environment in RL problems – i.e., the state space is constituted by I , the RNN hidden state h t and the historical visitations

Original: l j<t ; the actions are multinouli variables l t,i ; the reward function is deﬁned as the marginal log-likelihood log p ( y | l t,i , I ) ; and the policy is characterized by p ( l t,i | l j<t , I )
Revised: l j<t ; the actions are multinouli variables l t,i ; the reward function is deﬁned as the marginal log-likelihood log p ( y | l t,i , I ) ; and the policy is characterized by p ( l t,i | l j<t , I )

Original: Xu et al.
Revised: Xu et al.

Original: ; Mnih et al.
Revised: ; Mnih et al.

Original: Ba et al.
Revised: Ba et al.

Original: However, these methods cannot be applied directly to our problem as it requires the features I to exclusively associate with speciﬁc parts of the inputs spatially , which is attainable by using convolutional encoders with image inputs but intractable with tabular inputs considered in our case (see Section 2.1).
Revised: However, these methods cannot be applied directly to our problem as it requires the features I to exclusively associate with speciﬁc parts of the inputs spatially , which is attainable by using convolutional encoders with image inputs but intractable with tabular inputs considered in our case (see Section 2.1).

Original: Instead, our approach overcomes this issue by directly applying importance, generated by RL, into the gradient space during back-propagation.
Revised: Instead, our approach overcomes this issue by directly applying importance, generated by RL, into the gradient space during back-propagation.

Original: Moreover, our method does not require to formulate the learning objective as ELBOs and as a result GIL can adopt any state-of-the-art
Revised: Moreover, our method does not require to formulate the learning objective as ELBOs and as a result GIL can adopt any state-of-the-art

Original: RL algorithm – not limited to REINFORCE only as in Mnih et al.
Revised: RL algorithm – not limited to REINFORCE only as in Mnih et al.

Original: Ba et al.
Revised: Ba et al.

Original: Xu et al.
Revised: Xu et al.

Original: D E XPERIMENTAL D ETAILS AND A DDITIONAL E XPERIMENTS
Revised: D E XPERIMENTAL D ETAILS AND A DDITIONAL E XPERIMENTS

Original: The case studies are run on a work station with three Nvidia Quadro RTX 6000 GPUs with 24GB of memory for each.
Revised: The case studies are run on a work station with three Nvidia Quadro RTX 6000 GPUs with 24GB of memory for each.

Original: We use Tensorﬂow to implement the models and training algorithms.
Revised: We use Tensorﬂow to implement the models and training algorithms.

Original: To train the imputation-free prediction models using GIL, we perform a grid search for the model learning rate α ∈ { 0 . 001 , 0 . 0007 , 0 . 0005 , 0 . 0003 , 0 . 0001 , 0 . 00005 , 0 . 00001 } , the exponential decay step for α is selected from { 1000 , 750 , 500 } and the exponential decay rate for α is selected from { 0 . 95 , 0 . 9 , 0 . 85 , 0 . 8 } .
Revised: To train the imputation-free prediction models using GIL, we perform a grid search for the model learning rate α ∈ { 0 . 001 , 0 . 0007 , 0 . 0005 , 0 . 0003 , 0 . 0001 , 0 . 00005 , 0 . 00001 } , the exponential decay step for α is selected from { 1000 , 750 , 500 } and the exponential decay rate for α is selected from { 0 . 95 , 0 . 9 , 0 . 85 , 0 . 8 } .

Original: The actor π θ and critic Q ν in the GIL (i.e., Alg. 1) are trained using deep deterministic policy gradient (DDPG) Lillicrap et al.
Revised: The actor π θ and critic Q ν in the GIL (i.e., Alg. 1) are trained using deep deterministic policy gradient (DDPG) Lillicrap et al.

Original: where the discounting factor γ = 0 . 99 .
Revised: where the discounting factor γ = 0 . 99 .

Original: We choose the behavioural policy β in GIL as
Revised: We choose the behavioural policy β in GIL as

Original: [Equation]
Revised: [Equation]

Original: probability
Revised: probability

Original: [Equation]
Revised: [Equation]

Original: missing indicator
Revised: missing indicator

Original: [Equation]
Revised: [Equation]

Original: The learning rates of the actor α π and the critic α Q are selected by performing grid search from { 0 . 0005 , 0 . 0001 , 0 . 00005 , 0 . 00001 } and { 0 . 001 , 0 .
Revised: The learning rates of the actor α π and the critic α Q are selected by performing grid search from { 0 . 0005 , 0 . 0001 , 0 . 00005 , 0 . 00001 } and { 0 . 001 , 0 .

Original: respectively.
Revised: respectively.

Original: To train the imputation-free models using GIL-H/GIL-D, we follow the same grid search for α along with its decay steps and decay rates.
Revised: To train the imputation-free models using GIL-H/GIL-D, we follow the same grid search for α along with its decay steps and decay rates.

Original: From the implementation perspective, we replace the missing entries, in the inputs x , with a placeholder value before feeding them into the model.
Revised: From the implementation perspective, we replace the missing entries, in the inputs x , with a placeholder value before feeding them into the model.

Original: This can avoid value errors thrown by Tensorﬂow if the input vectors contain NaN’s.
Revised: This can avoid value errors thrown by Tensorﬂow if the input vectors contain NaN’s.

Original: However, note these values will not be used to update the parameters.
Revised: However, note these values will not be used to update the parameters.

Original: For the state-of-the-art baselines – GAIN 5 , MIWAE 6 , GPVAE 7 and
Revised: For the state-of-the-art baselines – GAIN 5 , MIWAE 6 , GPVAE 7 and

Original: BRITS 8 – we use the implementations published on the Github by the authors.
Revised: BRITS 8 – we use the implementations published on the Github by the authors.

Original: Adam optimizer is used to train all the prediction models for baselines, or to compute ( ∂E/∂ W ) SGD for GIL, GIL-H and GIL-D.
Revised: Adam optimizer is used to train all the prediction models for baselines, or to compute ( ∂E/∂ W ) SGD for GIL, GIL-H and GIL-D.

Original: All the models are trained using a batch size of 128.
Revised: All the models are trained using a batch size of 128.

Original: The details of selecting the other hyper-parameters of each case study can be found in the corresponding sub-section below.
Revised: The details of selecting the other hyper-parameters of each case study can be found in the corresponding sub-section below.

Original: D. MIMIC-III
Revised: D. MIMIC-III

Original: Dataset Formulation The MIMIC-III 9 contains EHRs obtained from roughly 40,000 patients who stayed in the ICUs in the Beth Israel Deaconess Medical Center between 2001 and 2012 (Johnson et al., 2016) and septic shock is a severe type of sepsis that results in above 40% mortality rate.
Revised: Dataset Formulation The MIMIC-III 9 contains EHRs obtained from roughly 40,000 patients who stayed in the ICUs in the Beth Israel Deaconess Medical Center between 2001 and 2012 (Johnson et al., 2016) and septic shock is a severe type of sepsis that results in above 40% mortality rate.

Original: Figure 4 illustrates the 2-hour observation window, 4-hour early prediction window (EPW) and the relation between inputs and targets for predictions.
Revised: Figure 4 illustrates the 2-hour observation window, 4-hour early prediction window (EPW) and the relation between inputs and targets for predictions.

Original: Note that each work related to septic shock predictions adopts a slightly different strategy in terms of selecting lab/vital attributes and the lengths of the observation and EPW (Mao et al., 2018; Darwiche & Mukherjee, 2018; Yee et al., 2019;
Revised: Note that each work related to septic shock predictions adopts a slightly different strategy in terms of selecting lab/vital attributes and the lengths of the observation and EPW (Mao et al., 2018; Darwiche & Mukherjee, 2018; Yee et al., 2019;

Original: Sheetrit et al., 2017; Liu et al., 2019; Khoshnevisan et al., 2020; Fleuren et al., 2020).
Revised: Sheetrit et al., 2017; Liu et al., 2019; Khoshnevisan et al., 2020; Fleuren et al., 2020).

Original: In this work, the 4-hour EPW allows including sufﬁcient number of subjects which can ensure statistical signiﬁcance of the results, while the smaller observation window keeps the task challenging.
Revised: In this work, the 4-hour EPW allows including sufﬁcient number of subjects which can ensure statistical signiﬁcance of the results, while the smaller observation window keeps the task challenging.

Original: To formulate the training and testing dataset, we ﬁrst selected 14 commonly used attributes for predictions as suggested in previous works (Sheetrit et al., 2017; Fleuren et al., 2020; Khoshnevisan et al., 2020), which are constituted by vital signs including temperature, respiratory rate, heart rate, systolic blood pressure,
Revised: To formulate the training and testing dataset, we ﬁrst selected 14 commonly used attributes for predictions as suggested in previous works (Sheetrit et al., 2017; Fleuren et al., 2020; Khoshnevisan et al., 2020), which are constituted by vital signs including temperature, respiratory rate, heart rate, systolic blood pressure,

Original: Data acquired from https mean arterial pressure, peripheral oxygen saturation (or SpO 2 ), fraction of inspired oxygen (or
Revised: Data acquired from https mean arterial pressure, peripheral oxygen saturation (or SpO 2 ), fraction of inspired oxygen (or

Original: FIO 2 ), and lab tests including white blood cell count, serum lactate level, platelet count, creatinine, bilirubin, bandemia of white blood cells, blood urea nitrogen.
Revised: FIO 2 ), and lab tests including white blood cell count, serum lactate level, platelet count, creatinine, bilirubin, bandemia of white blood cells, blood urea nitrogen.

Original: To form the septic shock cohort, we ﬁrst identify the patients who were diagnosed with sepsis by indexing with the International
Revised: To form the septic shock cohort, we ﬁrst identify the patients who were diagnosed with sepsis by indexing with the International

Original: Classiﬁcation of Diseases 9 (ICD-9) code 995.91 and 995.92.
Revised: Classiﬁcation of Diseases 9 (ICD-9) code 995.91 and 995.92.

Original: Then the patients with septic shock history are selected using ICD-9 code 785.52 with their septic shock onset time determined following the third international consensus for sepsis and septic shock (Sepsis-3) standard (Singer et al., 2016).
Revised: Then the patients with septic shock history are selected using ICD-9 code 785.52 with their septic shock onset time determined following the third international consensus for sepsis and septic shock (Sepsis-3) standard (Singer et al., 2016).

Original: Finally, we remove the patients whose septic shock onset time was less than 6 hours after admission to the ICUs since the data is not enough to be formulated as sequences pertaining to the 2-hour observation and 4-hour prediction window.
Revised: Finally, we remove the patients whose septic shock onset time was less than 6 hours after admission to the ICUs since the data is not enough to be formulated as sequences pertaining to the 2-hour observation and 4-hour prediction window.

Original: Consequently, a total of 1,083 septic shock patients are identiﬁed.
Revised: Consequently, a total of 1,083 septic shock patients are identiﬁed.

Original: To form the non-shock (or control) cohort, ﬁrst 1,083 patients are randomly sampled from all admissions who have at least 2 hours of records excluding the ones who are in the septic shock cohort, then a 2-hour time frame is randomly selected to be used as the observation window.
Revised: To form the non-shock (or control) cohort, ﬁrst 1,083 patients are randomly sampled from all admissions who have at least 2 hours of records excluding the ones who are in the septic shock cohort, then a 2-hour time frame is randomly selected to be used as the observation window.

Original: All patients selected following the above procedure are split into 8:2 to formulate the training and testing datasets, which are referred to as the varied-length (Var-l.) sequences in Section 4.2.
Revised: All patients selected following the above procedure are split into 8:2 to formulate the training and testing datasets, which are referred to as the varied-length (Var-l.) sequences in Section 4.2.

Original: In the test set, the number of subjects associated with positive and negative labels, respectively, are selected to be equivalent.
Revised: In the test set, the number of subjects associated with positive and negative labels, respectively, are selected to be equivalent.

Original: Formulation of Fix-l. Sequences To formulate the ﬁxed-length (Fix-l.) sequences, we follow this protocol – if sequence length is less than 8, we pad it with the latest recording available until the length reaches 8, otherwise we truncate the length to 8 by discarding the data from that point on-wards.
Revised: Formulation of Fix-l. Sequences To formulate the ﬁxed-length (Fix-l.) sequences, we follow this protocol – if sequence length is less than 8, we pad it with the latest recording available until the length reaches 8, otherwise we truncate the length to 8 by discarding the data from that point on-wards.

Original: We speciﬁcally choose this threshold because most of the sequences have length ≤ 8, as shown in Figure 5.
Revised: We speciﬁcally choose this threshold because most of the sequences have length ≤ 8, as shown in Figure 5.

Original: Type of Missingness
Revised: Type of Missingness

Original: The missing data in this dataset can be considered as a mixture of MCAR,
Revised: The missing data in this dataset can be considered as a mixture of MCAR,

Original: MAR and MNAR.
Revised: MAR and MNAR.

Original: Speciﬁcally, recordings from human mistakes or malfunctioned equipment manifest as MCAR, the dependency across vital signs and lab results give rise to MAR ( e.g. , speciﬁc lab tests are ordered only if abnormalities found in the related vital readings or other lab results), and the mismatch among the sampling frequency of some periodically recorded attributes such as temperature (obtained hourly) and blood test (conducted daily) can be considered as MNAR (Little &
Revised: Speciﬁcally, recordings from human mistakes or malfunctioned equipment manifest as MCAR, the dependency across vital signs and lab results give rise to MAR ( e.g. , speciﬁc lab tests are ordered only if abnormalities found in the related vital readings or other lab results), and the mismatch among the sampling frequency of some periodically recorded attributes such as temperature (obtained hourly) and blood test (conducted daily) can be considered as MNAR (Little &

Original: Rubin, 2019; Mamandipoor et al., 2019).
Revised: Rubin, 2019; Mamandipoor et al., 2019).

Original: Training Details To train the prediction models for both GIL and baselines, we use maximum training steps of 2,000 and 4,000 for varied-length and ﬁxed-length sequences respectively.
Revised: Training Details To train the prediction models for both GIL and baselines, we use maximum training steps of 2,000 and 4,000 for varied-length and ﬁxed-length sequences respectively.

Original: For imputation of the missing values, we train the imputation method MIWAE using the learning rates { 0 . 001 , 0 . 0001 } with other hyper-parameters provided by the authors in the code.
Revised: For imputation of the missing values, we train the imputation method MIWAE using the learning rates { 0 . 001 , 0 . 0001 } with other hyper-parameters provided by the authors in the code.

Original: To train GAIN, we use the default learning rates provided by the authors to train the generator and discriminator.
Revised: To train GAIN, we use the default learning rates provided by the authors to train the generator and discriminator.

Original: GAIN contains one hyper-parameter that balances between the two losses L G and L M deﬁned in Yoon et al.
Revised: GAIN contains one hyper-parameter that balances between the two losses L G and L M deﬁned in Yoon et al.

Original: (2018) which is selected from { 0 . 1 , 1 , 10 , 100 } .
Revised: (2018) which is selected from { 0 . 1 , 1 , 10 , 100 } .

Original: To train GP-VAE on the Fix-l. case, the set of hyperparameters we use contain latent _ dim = { 6 , 12 , 35 } , encoder _ size = 256 , 256 , decoder _ size = 256 , 256 , 256 , window _ size = { 3 , 4 , 6 } , beta = { 0 . 2 , 0 . 5 , 0 . 8 } , sigma = 1 . 005 , length _ scale =
Revised: To train GP-VAE on the Fix-l. case, the set of hyperparameters we use contain latent _ dim = { 6 , 12 , 35 } , encoder _ size = 256 , 256 , decoder _ size = 256 , 256 , 256 , window _ size = { 3 , 4 , 6 } , beta = { 0 . 2 , 0 . 5 , 0 . 8 } , sigma = 1 . 005 , length _ scale =

Original: To train BRITS we used the recommended impute _ weight = 0 . 3 , label _ weight = 1 . 0 .
Revised: To train BRITS we used the recommended impute _ weight = 0 . 3 , label _ weight = 1 . 0 .

Original: The term D ( ζ + , ζ − ) included in the reward function for GIL-D is deﬁned as in the follows.
Revised: The term D ( ζ + , ζ − ) included in the reward function for GIL-D is deﬁned as in the follows.

Original: Assume that in each training epoch b/ 2 inputs with label 0 and b/ 2 inputs with label 1 are sampled from the dataset, where b is the batch size.
Revised: Assume that in each training epoch b/ 2 inputs with label 0 and b/ 2 inputs with label 1 are sampled from the dataset, where b is the batch size.

Original: We use F + = ( ζ +1 , . . .
Revised: We use F + = ( ζ +1 , . . .

Original: , ζ + b/ 2 ) ∈ R b/ 2 × e to denote the features associated with positive labels ( i.e. , 1) in the current batch and F − =
Revised: , ζ + b/ 2 ) ∈ R b/ 2 × e to denote the features associated with positive labels ( i.e. , 1) in the current batch and F − =

Original: ( ζ − 1 , . . .
Revised: ( ζ − 1 , . . .

Original: , ζ − b/ 2 ) ∈ R b/ 2 × e are , where e is the dimension of each individual feature ζ .
Revised: , ζ − b/ 2 ) ∈ R b/ 2 × e are , where e is the dimension of each individual feature ζ .

Original: Then we deﬁne D as
Revised: Then we deﬁne D as

Original: [Equation]
Revised: [Equation]

Original: (43) where the slicing index follows the syntax from Python, e.g. , F + [0 : b/ 4] corresponds to the 0-th to ( b/ 4 − 1) -th rows of F +
Revised: (43) where the slicing index follows the syntax from Python, e.g. , F + [0 : b/ 4] corresponds to the 0-th to ( b/ 4 − 1) -th rows of F +

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: Dataset
Revised: Dataset

Original: Introduction This dataset is originally constituted by OCT images, CFP images and patient EHRs including demographic information and test results related to diabetes.
Revised: Introduction This dataset is originally constituted by OCT images, CFP images and patient EHRs including demographic information and test results related to diabetes.

Original: A total of 1, subjects are included in this dataset.
Revised: A total of 1, subjects are included in this dataset.

Original: OCT refers to the two- or three-dimensional images capturing the retinal architectures of the eyes that are scanned by low-coherence lights.
Revised: OCT refers to the two- or three-dimensional images capturing the retinal architectures of the eyes that are scanned by low-coherence lights.

Original: The 3D OCT image is usually called the volume scan and it is constituted by 61 or 101 2D image slices depending on the spec of the scanner.
Revised: The 3D OCT image is usually called the volume scan and it is constituted by 61 or 101 2D image slices depending on the spec of the scanner.

Original: Examples of 3D and 2D OCT scans are shown in Figure 6.
Revised: Examples of 3D and 2D OCT scans are shown in Figure 6.

Original: In this experiment the volume scans we use contain the 61 slices, while we only consider the center slice (31th) 2D image.
Revised: In this experiment the volume scans we use contain the 61 slices, while we only consider the center slice (31th) 2D image.

Original: The CFP images are the color images captured by a fundus camera showing the condition of the interior surface of the eye, where an example is shown in Figure 7.
Revised: The CFP images are the color images captured by a fundus camera showing the condition of the interior surface of the eye, where an example is shown in Figure 7.

Original: Dataset Formulation To formulate the data into tabular inputs, the 2D OCT and CFP retinal images are ﬁrst fed in to two inception-v3 (Szegedy et al., 2016)
Revised: Dataset Formulation To formulate the data into tabular inputs, the 2D OCT and CFP retinal images are ﬁrst fed in to two inception-v3 (Szegedy et al., 2016)

Original: CNNs, pre-trained with similar images provided by (Kermany et al., 2018) and (Kaggle, 2015) respectively, and the image feature vectors with dimension of 2,048 each are output by the global average pooling layer.
Revised: CNNs, pre-trained with similar images provided by (Kermany et al., 2018) and (Kaggle, 2015) respectively, and the image feature vectors with dimension of 2,048 each are output by the global average pooling layer.

Original: Then the patient
Revised: Then the patient

Original: EHR information include age (integer), sex (boolean), length of diabetic history (integer), A1C result (integer), which measures blood sugar levels, and if insulin has been used (boolean) constitute a 5-dimensional vector that is concatenated to the end of the OCT and CFP feature vectors.
Revised: EHR information include age (integer), sex (boolean), length of diabetic history (integer), A1C result (integer), which measures blood sugar levels, and if insulin has been used (boolean) constitute a 5-dimensional vector that is concatenated to the end of the OCT and CFP feature vectors.

Original: We split all the subjects into a training cohort and a testing cohort following a ratio of 9:1.
Revised: We split all the subjects into a training cohort and a testing cohort following a ratio of 9:1.

Original: In the test set, the number of subjects associated with positive and negative labels, respectively, are selected to be equivalent.
Revised: In the test set, the number of subjects associated with positive and negative labels, respectively, are selected to be equivalent.

Original: The raw images from the training cohort are augmented through cropping and rotation before feeding into inception-v3.
Revised: The raw images from the training cohort are augmented through cropping and rotation before feeding into inception-v3.

Original: All the data from the testing cohort are not augmented.
Revised: All the data from the testing cohort are not augmented.

Original: Figureillustrates how the training and testing inputs are formulated, as well as the input-output relation of the prediction models.
Revised: Figureillustrates how the training and testing inputs are formulated, as well as the input-output relation of the prediction models.

Original: Training Details To train the prediction models for both GIL and baselines, we use maximum training steps of 2,000.
Revised: Training Details To train the prediction models for both GIL and baselines, we use maximum training steps of 2,000.

Original: For imputation of the missing values, MIWAE is trained using the learning rates { 0 . 001 , 0 . 0001 } with other hyper-parameters provided by the authors in the code.
Revised: For imputation of the missing values, MIWAE is trained using the learning rates { 0 . 001 , 0 . 0001 } with other hyper-parameters provided by the authors in the code.

Original: GAIN is trained using hyper-parameters selected from { 0 . 1 , 1 , 10 , 100 } .
Revised: GAIN is trained using hyper-parameters selected from { 0 . 1 , 1 , 10 , 100 } .

Original: GP-VAE is trained by applying the arguments latent _ dim = 256 , encoder _ size = 256 , 256 , decoder _ size = 256 , 256 , 256 , window _ size = 3 , beta = 0 . 8 , sigma = 1 to its code published in Github ( https://github.com/ratschlab/GP-VAE ).
Revised: GP-VAE is trained by applying the arguments latent _ dim = 256 , encoder _ size = 256 , 256 , decoder _ size = 256 , 256 , 256 , window _ size = 3 , beta = 0 . 8 , sigma = 1 to its code published in Github ( https://github.com/ratschlab/GP-VAE ).

Original: MF, EM and kNN are not included in this experiment due to the high dimensionality of the inputs which makes these algorithms very computational inefﬁcient as imputation results cannot be produced within days.
Revised: MF, EM and kNN are not included in this experiment due to the high dimensionality of the inputs which makes these algorithms very computational inefﬁcient as imputation results cannot be produced within days.

Original: D. MNIST
Revised: D. MNIST

Original: Training Details To train the prediction models for both GIL and baselines, we use maximum training steps of 1,0000.
Revised: Training Details To train the prediction models for both GIL and baselines, we use maximum training steps of 1,0000.

Original: For imputation of the missing values, MIWAE is trained using the learning rates { 0 . 001 , 0 . 0001 } with other hyper-parameters provided by the authors in the code.
Revised: For imputation of the missing values, MIWAE is trained using the learning rates { 0 . 001 , 0 . 0001 } with other hyper-parameters provided by the authors in the code.

Original: GAIN is trained using hyper-parameters selected from { 0 . 1 , 1 , 10 , 100 } .
Revised: GAIN is trained using hyper-parameters selected from { 0 . 1 , 1 , 10 , 100 } .

Original: GP-VAE is trained by applying the arguments latent _ dim = 256 , encoder _ size = 256 , 256 , decoder _ size = 256 , 256 , 256 , window _ size = 3 , beta = 0 . 8 , sigma = 1 to its code published in Github ( https://github.com/ratschlab/GP-VAE ).
Revised: GP-VAE is trained by applying the arguments latent _ dim = 256 , encoder _ size = 256 , 256 , decoder _ size = 256 , 256 , 256 , window _ size = 3 , beta = 0 . 8 , sigma = 1 to its code published in Github ( https://github.com/ratschlab/GP-VAE ).

Original: MF, EM and kNN are not included in this experiment due to the high dimensionality of the inputs which makes these algorithms very computational inefﬁcient as imputation results cannot be produced within days.
Revised: MF, EM and kNN are not included in this experiment due to the high dimensionality of the inputs which makes these algorithms very computational inefﬁcient as imputation results cannot be produced within days.

Original: Additional Results We also trained GIL-D on the MCAR version of MNIST dataset and the performance are shown in Table 5 below.
Revised: Additional Results We also trained GIL-D on the MCAR version of MNIST dataset and the performance are shown in Table 5 below.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Other than MIMIC-III, we also tested on a smaller scaled ICU time-series from 2012 Physionet challenge Silva et al.
Revised: Other than MIMIC-III, we also tested on a smaller scaled ICU time-series from 2012 Physionet challenge Silva et al.

Original: (2012) which contain data obtained from 12,000 patients.
Revised: (2012) which contain data obtained from 12,000 patients.

Original: We use the data pre-processed and open-sourced by Fortuin et al.
Revised: We use the data pre-processed and open-sourced by Fortuin et al.

Original: For each patient the values of 35 different attributes ( e.g., blood pressure, temperature) are recorded over a 48-hour window.
Revised: For each patient the values of 35 different attributes ( e.g., blood pressure, temperature) are recorded over a 48-hour window.

Original: As a result, the data for each patient can be formulated into a matrix in R 48 × 35 , i.e. , the sequence length across patients are the same.
Revised: As a result, the data for each patient can be formulated into a matrix in R 48 × 35 , i.e. , the sequence length across patients are the same.

Original: This dataset has an overall 78.5% missing rate and a binary label is assigned to each patient where 87% of them are 0’s and 13% are 1’s.
Revised: This dataset has an overall 78.5% missing rate and a binary label is assigned to each patient where 87% of them are 0’s and 13% are 1’s.

Original: Therefore we include average precision (AP) which is calculated from the precision-recall curve as an additional metric to evaluate the performance toward imbalanced labels.
Revised: Therefore we include average precision (AP) which is calculated from the precision-recall curve as an additional metric to evaluate the performance toward imbalanced labels.

Original: For this dataset, we consider a 1024-unit LSTM layer for encoding and a dense output layer for inference.
Revised: For this dataset, we consider a 1024-unit LSTM layer for encoding and a dense output layer for inference.

Original: It can be observed from Table 6 that although our method achieves the highest accuracy and 2-nd highest AP, while most methods perform very close to mean- and zeroimputation.
Revised: It can be observed from Table 6 that although our method achieves the highest accuracy and 2-nd highest AP, while most methods perform very close to mean- and zeroimputation.

Original: This could be caused by the its simple structure, as all the sequences share the same time horizon, which signiﬁcantly reduces the difﬁculties for the classiﬁcation task.
Revised: This could be caused by the its simple structure, as all the sequences share the same time horizon, which signiﬁcantly reduces the difﬁculties for the classiﬁcation task.

Original: Moreover, the highly imbalanced labels and the small population result in the performance to be hardly distinguishable across different methods.
Revised: Moreover, the highly imbalanced labels and the small population result in the performance to be hardly distinguishable across different methods.

Original: And this is the reason that we focus on the MIMIC-III dataset to study the strengths and shortcomings of the methods.
Revised: And this is the reason that we focus on the MIMIC-III dataset to study the strengths and shortcomings of the methods.

Original: E P ROOF FOR P ROPOSITION
Revised: E P ROOF FOR P ROPOSITION

Original: Here we prove a generalized version of Proposition 1 as stated in the following proposition, which shows that the gradient of the loss function E w.r.t.
Revised: Here we prove a generalized version of Proposition 1 as stated in the following proposition, which shows that the gradient of the loss function E w.r.t.

Original: any layer in the MLP can be written in the outer product format.
Revised: any layer in the MLP can be written in the outer product format.

Original: Proposition 3 Given an MLP as in (1) and a smooth loss function E ( ˆy | y ) , the gradients for any hidden layer W i , i ∈ [1 , k ] can be represented as
Revised: Proposition 3 Given an MLP as in (1) and a smooth loss function E ( ˆy | y ) , the gradients for any hidden layer W i , i ∈ [1 , k ] can be represented as

Original: [Equation]
Revised: [Equation]

Original: (44) where q i − 1 = φ i − 1 ( W i − 1 q
Revised: (44) where q i − 1 = φ i − 1 ( W i − 1 q

Original: i − 2 ) is the output from the i − 1 -th layer with q 0 = x , and ∆ i =
Revised: i − 2 ) is the output from the i − 1 -th layer with q 0 = x , and ∆ i =

Original: W (cid:62) i +1 ∆ i +1 (cid:12) φ (cid:48)
Revised: W (cid:62) i +1 ∆ i +1 (cid:12) φ (cid:48)

Original: i ( W i q i − 1 ) with ∆ out = ∂E∂ ˆy (cid:12) φ (cid:48) out ( W out q 2 k ) and (cid:12) denotes the element-wise (Hadamard) product.
Revised: i ( W i q i − 1 ) with ∆ out = ∂E∂ ˆy (cid:12) φ (cid:48) out ( W out q 2 k ) and (cid:12) denotes the element-wise (Hadamard) product.

Original: Now we prove Proposition 3.
Revised: Now we prove Proposition 3.

Original: We consider the MLP characterized by
Revised: We consider the MLP characterized by

Original: [Equation]
Revised: [Equation]

Original: 45) where x ∈ R d represents the input to the model, W i is the weight matrix for the i -th hidden layer, and φ i is the activation function of the i -th hidden layer.
Revised: 45) where x ∈ R d represents the input to the model, W i is the weight matrix for the i -th hidden layer, and φ i is the activation function of the i -th hidden layer.

Original: We start with deriving the derivatives for the output layer
Revised: We start with deriving the derivatives for the output layer

Original: [Equation]
Revised: [Equation]

Original: 48) where q k is the output from the k -th hidden layer.
Revised: 48) where q k is the output from the k -th hidden layer.

Original: Now we show the derivative of E w.r.t.
Revised: Now we show the derivative of E w.r.t.

Original: the k -th hidden layer
Revised: the k -th hidden layer

Original: [Equation]
Revised: [Equation]

Original: We now prove Proposition 3 by induction.
Revised: We now prove Proposition 3 by induction.

Original: First assume that for j ∈ [2 , k ] , the derivative of E w.r
Revised: First assume that for j ∈ [2 , k ] , the derivative of E w.r

Original: [Equation]
Revised: [Equation]

Original: t. the j -th hidden layer is
Revised: t. the j -th hidden layer is

Original: [Equation]
Revised: [Equation]

Original: Moreover, let us assume that
Revised: Moreover, let us assume that

Original: [Equation]
Revised: [Equation]

Original: Now we need to show that for j − 1 , it holds that
Revised: Now we need to show that for j − 1 , it holds that

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst prove that (60) holds,
Revised: We ﬁrst prove that (60) holds,

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: e.,
Revised: e.,

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to (60)
Revised: which is equivalent to (60)

Original: [Equation]
Revised: [Equation]

Original: To prove that (58) holds, we start with ∂E
Revised: To prove that (58) holds, we start with ∂E

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to (58).
Revised: which is equivalent to (58).

Original: F A DDITIONAL R ELATED W ORKS
Revised: F A DDITIONAL R ELATED W ORKS

Original: RL for Optimization.
Revised: RL for Optimization.

Original: This work is also related to approaches that use RL to solve optimization problems, or to improve existing solvers.
Revised: This work is also related to approaches that use RL to solve optimization problems, or to improve existing solvers.

Original: For example, Boyan & Moore (2000) proposes a framework to improve the efﬁciency of local search algorithms by predicting the search outcomes and biasing toward directions that provide higher returns, which is approached by estimating value functions of the MDP corresponding to the optimization problem.
Revised: For example, Boyan & Moore (2000) proposes a framework to improve the efﬁciency of local search algorithms by predicting the search outcomes and biasing toward directions that provide higher returns, which is approached by estimating value functions of the MDP corresponding to the optimization problem.

Original: Recently, Li & Malik (2016) trains RL agents to perform gradient descent steps over linear regression and neural network models.
Revised: Recently, Li & Malik (2016) trains RL agents to perform gradient descent steps over linear regression and neural network models.

Original: Similarly, Wei et al.
Revised: Similarly, Wei et al.

Original: (2020) introduces a policy network to automatically determine parameters for the plug-and-play frameworks (Sreehari et al., 2017) which can solve inverse imaging problems.
Revised: (2020) introduces a policy network to automatically determine parameters for the plug-and-play frameworks (Sreehari et al., 2017) which can solve inverse imaging problems.

Original: Note that these methods require complete inputs and cannot be adapted to the problem we consider.
Revised: Note that these methods require complete inputs and cannot be adapted to the problem we consider.

Revised: G A DDITIONAL A BLATION S TUDY

Revised: In this section, we compare GIL against an ablation baseline that applies the importance directly to the inputs x , instead of the gradient space.

Revised: Speciﬁcally, suppose a differential parametric function h ψ ( x ) , parameterized by ψ , is multiplied with x element-wise.

Revised: Then, as opposed to (4), the gradient updates w.r.t.

Revised: the encoding layers can formulated as

Revised: [Equation]

Revised: Consequently, all model elements ( i.e. , h ψ , W enc and W inf ) could be trained by gradient descent, without introducing RL policies.

Revised: [Table]

Revised: In Table 7 it shows the performance of the model described above over the ophthalmic dataset, as well as MNIST digits with 70% and 90% missing rate (M.R.)

Revised: Speciﬁcally, h ψ ( x ) is captured by a neural network with the same architecture as the policy π θ in GIL.

Revised: The training and testing are conducted following the same convention ( e.g. , random seeds that determine the missing entries and hyper-parameter search) as introduced in Sections 4.3, 4.4 and Appendix D.2, D.3.

Revised: It can be observed that the ablation baseline attained similar performance as to GIL-H and zero imputation.

Revised: The reason that leads to these results would be that the gradients for training h ψ ( x ) still follow the outer product format ∆ · x (cid:62) which are left to be accounted for.

Revised: Speciﬁcally, our method is built on top of the idea, and the experiments in Section 4 also support that if part of the inputs are missing they may not provide sufﬁcient information to train the prediction models, given the outer product format of the gradients.

Revised: Similarly, the gradients for h ψ ( x ) also follow this format and h ψ ( x ) may not be properly trained directly on incomplete inputs x , which could in general limit the performance of the prediction model overall.

Revised: Additional rationales for using RL to obtain gradient importance in GIL.

Revised: The ablation baseline above is closely related to visual attention (VA) models as discussed in Appendix C, and could be seen as a preliminary version of them where h ψ ( x ) is used to re-weight elements in the inputs x .

Revised: VA techniques beneﬁt from the fact that features learned by CNNs are spatially correlated to the inputs, so the attentions could be directly applied to learned features.

Revised: To achieve this, VA formulates the objective as an ELBO which is shown equivalent to the objective of a basic policy gradient RL algorithm, REINFORCE (Mnih et al., 2014).

Revised: However, it was not clear how such VA methods could be adapted to the problem we consider, as the features learned by MLPs or LSTMs are not spatially correlated with inputs.

Revised: More on importance versus attentions.

Revised: The other baseline, BRITS (Cao et al., 2018), uses bidirectional RNN to process time-series, with attentions applied to the hidden states of RNNs, followed by optimizing over imputation and prediction objectives jointly during training.

Revised: BRITS is considered as the state-of-the-art method that could predict with incomplete time-series inputs end-to-end .

Revised: However, we showed that BRITS is outperformed by our method on the MIMIC dataset; see Table 1.

Revised: The reason would be that BRITS requires masking off part of the observed inputs to constitute the imputation objective; thus, the information provided for model training is even more limited given the intrinsic > 70% M.R. of MIMIC.