Gradient Importance Learning for Incomplete Observations
Authors: Qitong Gao
Dong Wang
Joshua David Amason
Siyang Yuan
Chenyang Tao
Ricardo Henao
Majda Hadziahmetovic
Lawrence Carin
Miroslav Pajic
Venue: ICLR 2022 Poster (2022)
Keywords: Missing Data
Reinforcement Learning
Representation Learning

Abstract: Though recent works have developed methods that can generate estimates (or imputations) of the missing entries in a dataset to facilitate downstream analysis, most depend on assumptions that may not align with real-world applications and could suffer from poor performance in subsequent tasks such as classification. This is particularly true if the data have large missingness rates or a small sample size. More importantly, the imputation error could be propagated into the prediction step that follows, which may constrain the capabilities of the prediction model. In this work, we introduce the gradient importance learning (GIL) method to train multilayer perceptrons (MLPs) and long short-term memories (LSTMs) to directly perform inference from inputs containing missing values without imputation. Specifically, we employ reinforcement learning (RL) to adjust the gradients used to train these models via back-propagation. This allows the model to exploit the underlying information behind missingness patterns. We test the approach on real-world time-series (i.e., MIMIC-III), tabular data obtained from an eye clinic, and a standard dataset (i.e., MNIST), where our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.

Published as a conference paper at ICLR 2022
GRADIENT IMPORTANCE LEARNING FOR INCOMPLETE OBSERVATIONS
Qitong Gao"
Dong Wang"
Joshua D. Amason"
Siyang Yuan" Chenyang Tao"
Ricardo Henao'
Majda Hadziahmetovic" Lawrence Carin
Miroslav Pajic
ABSTRACT
Though recent works have developed methods that can generate estimates (or im- putations) of the missing entries in a dataset to facilitate downstream analysis, most depend assumptions that may not align with real-world applications and could suffer from poor performance in subsequent tasks such as classification. This is par- ticularly true if the data have large missingness rates or a small sample size. More importantly, the imputation error could be propagated into the prediction step that follows, which may constrain the capabilities of the prediction model. this work, we introduce the gradient importance learning (GIL) method to train multilayer perceptrons (MLPs) and long short-term memories (LSTMs) directly perform inference from inputs containing missing values without imputation. Specifically, we employ reinforcement learning (RI.) adjust the gradients used to train these models via back propaaropag This allows the model to exploit the underlying infor- mation behind missingness patterns. We test the approach on real-world time-series (i.e., MIMIC-III, tabular data obtained from an eye clinic, and standard dataset (i.e., MNIST), where our imputation-free predictions outperform the traditional two-step imputation- predictions using state-of-the of-the-art imputation methods.
INTRODUCTION
We consider learning from incomplete datasets. Components of the data could be missing for various reasons, for instance, missing responses from participants, data loss, and restricted access issues. This phenomenon is prevalent in the healthcare domain, mostly in the context of electronic health records (EHRs), which are structured as patient-specific irregular timelines with attributes, e.g., diagnosis, laboratory tests, vitals, thus resulting high missingness across patients for any arbitrary time point. Such missingness introduces difficulties when developing models and performing inference in real-world applications such as Kam Kim (0117); Scherpf et al. 2019). Existing works tackle this problem by either proposing imputation algorithms (IAs) to explicilly produce estimates of the missing data, or by imposing imputation objectives during inference, e.g., withholding observed values being the moound-truth and learning to impute them. However, some of these either require additional modeling assumptions for the underlying distributions (Bashir & Weil 2018 Fortuin et al. 2020), formatting of the data (Yu al.) 2016 (Schnabel al. 20.66 Other applications depend domain knowledge for pre- pre-processing and modeling such as Yang et al. (2018); Kam & Kim (2017). or introduce additional information-basee losses (Cao al. [2018] Lipton al. | (2016), which are usually intractable with real-world data. Moreover, generative methods (Mattei & Frellsen 2019
Yoon et al. 2018) could result in imputations with high variation (e.g., low confidence the output distribution), when data have high missingness rates or small sample sizes. Figure illustrates imputations generated by two state-of-the-arr deep generative models on the MNIST dataset with 50%, 70% and 90% of the pixels set as missing (i.e., masked out). It is observed that both approaches suffer from inaccurate reconstruction of the original digits as the missingness rate increases, which is manifested some of the imputed images not being recognizable as digits. Importantly, the error introduced by the imputation process can be further propagated into downstream inference and prediction stages.
Imputing the missing data (either explicitly or implicitly) is, in most cases, not necessary, more so, considering that sometimes where and when the data are missing can be intrinsically informative.
Duke University, USA. Contact: {qi toigtnn-a gao, mi ros1 av. pajic] @duke edu.
Kiig Ahdullah University of Seience and Technology, Saudi Arabia.
Code available https: : ://gi thub.
thub. www/gaogi www.o/gttooooooooooooooo http://raddeentn... innet-imppotnnneeee Tearnin

---

Published as a conference paper at ICLR 2022
Consider scenario in which two patients, A and B, admitted to the intensive care unit (ICU) suffer from bacterial and viral infections, respectively, and assume that the healthcare provider monitors the status of the patients by ordering two (slightly) different blood tests periodically, namely, culture test/panel specific to bacterial infections and a RT-PCR test/panel specific to viral infections. Hence, patient A is likely have much fewer orders (and results) for viral tests. In both cases, the missingness patterns are indicative of the underlying condition of both patients. Moreover, such patterns are more commonly found in incomplete data caused by missing at random (MAR) or missing not at random (MNAR) mechanisms, which usually introduce additional difficulties for inference (Little & Rubin 2019). Inspired by this, we propose the gradient importance learning (GIL) method, which facilitates an impulation--rree learning framework for incomplete data by simultaneously leveraging the observed data and the information underlying missingness patterns.
Specifically, we propose to use an importance matrix to weight the gradients that are used to update the model parameters in the back- propagation process, after the computational graph of the model is settled, which allows the model to exploit the information underly- ing missingness without imputation. However, these gradients cannot be tracked by automated differentiation tools such as Tensorflow (Abadi itt al. 2016). So motivated, we propose to gener- ate the importance matrix using a reinforcement learning (RL) policy on-the-ffy. This is done by conditioning on the status of training procedure characterized by the model parameters, inputs and model performance at the current training step. Concurrently, RL algorithms are used to update the policy by first modeling the pack-propagation process an RL environment, environment, the RL algorithm can Iearn the optimal policy so the importance matrix can aid the training of the prediction models to attain desirable performance. Moreover, we also show that framework can be augmented with feature learning techniques, e.g- contrastive learning as in Chen et al. 2020), which can further improve the performance of the proposed imputation-fres models.
50% Missing
70% Micsing
90% Micsing
Original
4
7
9 2
7
6
Masked
4//
3
MIWAE
4
7
3.
3
3
9
GAIN
4/'
7
Figure 1: MNIST digits imputed by statc-of-the-art imputation methods MIWAE (Mattei & Frellsen 2019). GAIN (Yoon al. 2018).
Markov decision process (MDP). Then, by interacting with the
The technical contributions of this work can be summarized as follows: (i) to the best of our knowledge, this is the first work that trains deep learning (DL) models to perform accurate imputation- free predictions with missing inputs. This allows models to effectively handle high missing rates, while significantly reducing the prediction error compared existing aopuutaiio-prediccti (ii) Unlike existing approaches that require additional modeling assumptions or that rely on methods (or pre-existing knowledge) that intrinsically have advantages over specific types of data, our method does not require any assumptions or domain expertise. Consequently, it can consistently achieve top performance over a variety of data and under different conditions. (tii) The proposed framework also facilitates feature learning from incomplete data, as the importance matrix can guide the hidden layers the model to capture information underlying missingness patterns during training, which results in more expressive features, as illustrated in Figure B
frameworks.
2 GRADIENT IMPORTANCE LEARNING (GIL)
In this section, we introduce GIL, a method that facilitatee training of imputation-fre prediction models with incomplete data. In Section 2.2 we show how the gradient descent directions can be adjusted using the importance matrix A. In Section
Environment
Input Data Micsing Indicator
States s.=(mmm..))
Labels
Reward n.
Encoding Layars
Inference Layers
Agent/Policy
2.3 we introduce an RL framework to generate the elements of A which can leverage the information underlying missingness during training. illustrated in Figure
Features
Predictions
Importance Matrix A (Action a)
GIL
300
Figure 2: Overview of the GIL. framework.
2.1 PROBLEM FORMULATION
We consider dataset containing N observations X (X1, X2,..., .,,. where each X, can be represented by vector Xj Rd (for tabular data) matrix X, E RT, xd (for time-series) with
2

---

Published as a conference paper at ICLR 2022
T3 denoting the time horizon of the sequence X,. Since the focus of this work is principally on time-series data, recurrent neural networks will receive significant attention. We also define the set of missing indicators M (M, M2,...... My). where M, m, E {0, M, {0, 1)7;xd depending on the dimension of Xj. and 0's (or I's) correspond to the entries in X, that are missing (or not missing), respectively. Finally, we assume that each X; is associated with some label y, ): thus y = Y2,..,,..... .Nn denotes the set of labels. We define the problem as learning model parameterized by W to directly predict yj by generating 9, that maximizes the log- leleeell Σラ=1 log 18g [X, M,, W), without imputing the missing values in Xj.
2.2 GRADIENT IMPORTANCE
Missingness is common issue found in tabular data and time series, where multi multi-layered perceptron (MLP) (Bishop 2006) and long short-term memory (LSTM) (Hochreiter & Schmidhuber 1997) models, respectively,
often considered for predictions. Below we illustrate how the importance matrix can be applied to gradients, which are produced by taking regular stochastic gradient descent (SGD) steps, toward training MLP models with incomplete tabular inputs. Note that this idea can be easily extended to sequential inputs with LSTM models and the corresponding details can be found in Appendix A First, the definitions of the dataset and missing indicators can be reduced to Heab = (X1,X), X2.......... .x) and Meab = (mj m2...... my), ), with x, E R2d and mj E R
respectively. Then we define an MLP with & hidden layers, for h: 2 2, as follows
Doat (W oott@((W Φ2 (W261 ( (Wxx))));
(1)
where x E IRd is the input, W. and Φ are the weight matrix and activation functions for the i-th hidden layer, respectively, and we have omitted the bias terms for notational convenience. The first layer can be interpreted
the encoding layer W ene Wi that maps inputs to features <= [xne(X|We Wee))),), eno), while the rest are the inference layers W Winf = { (W....., Wk. W ]ur} that map the features to prediction ÿ (SI/(S/Winn inf). In the following proposition, we show that the gradients of some commonly used loss functions, E(f.y y), such as cross entropy or mean squared errors, w.I.. W1, can be formulated in the form of an outer product. The proof and details can be found in Appendix
Proposition 1 Given MLP and smooth loss function E(y,y) y), the gradients of E w.r.. the encod- ing layer can be formulated as 0E/0W: = Aix where A contains the gradients propagated from all the inference layers, x E Maab and y V.
From this Proposition it can be seen that the gradients that are used to train the encoding layers. following regular SGD solvers, can be formulated as the outer product between the gradients A propagated from the inference layers and the input x as
(DE/OW = xT,
A
where W ene E _eedd A R. x RRd, e is the dimension of the features, and d is the dimension of the input. To simplify notation, note that henceforth we use x to refer to an individual tabular data X, E Xaab, regardless of its index.
The corresponding SGD updates for training Wene, using learning rate a, are W enc + W Vene A -x". As a result, is observed from (2) that the j-th (j E [1,d] column of (DE/oW enE)SGD is weighted by the j-th element in x given A. However, given that some entries in x could be missing (according to the missing indicator m) and their values are usually replaced by a placeholder & E R. it may not be ideal do directly back-propagate the gradients in the form of (2) as the features captured by the encoding layers may not be expressive enough for the inference layers to make accurate predictions. Instead, we consider introducing the importance matrix A E [0, 1jexd to adjust the gradient descent direction as
Wene Venc f- Wen a(OE enc)SGD C A,
(3)
which not only trains the model to capture information from the observed inputs that are most useful for making accurate predictions, but also to leverage the information underlying the missingness in the data. The elements of A can be generated using the RL. framework introduced in the next section.
Note that above we have omitted all the bias terms to simplify presentation and note that: i) gradients w.r.t to biases do not conform to the outer product format, and ii) more importantly, these gradients do not depend on the inputs thus in practice, the importance matrix A is only applied
3

---

Published as a conference paper at ICLR 2022
to the gradients of the weights. Moreover, we do not consider convolutional neural network (CNN) models (LeCun et al.) 1999) in this work because of i). Though the proposed framework could still be applied to CNNs, it may not be efficient for MLPs or LSTMs, where the search space for A is significantly reduced by taking advantages of the outer product format, as shown in (4) below. Importantly, our focus is to address the missingness in tabular data and time-series, in which case MLPs and LSTMs are appropriate, while the missingness in images is usually related to applications in other domains such as compressed sensing (Wu al. 2019) or image super-resolution (Dong et al. 2015), which we plan to explore in the future.
2.3 RL TO GENERATE IMPORTANCE MATRIX
Now we show how to use RL to generate the importance matrix A. In general, RL aims to learn an optimal policy that maximizes expected rewards, by interacting with an unknown environment (Sutton
& Barto 2018). Formally, the RL environment is characterized by a Markov decision process (MDP), with details introduced below. Each time after the agent chooses an action a (s)) state following some policy т, the environment responds with the next state s', along with an immediate reward r R(s,a) a) given a reward function R(.-). The goal to learn an optimal policy that
maximizes the expected total reward defined as J(T) = Epp E y'r(s;, a:)]. where T is the horizon of the MDP, 7 [0, 1) is the discounting factor, and Pa {[90, (o): (81, (s),,)).... la; π ( is the sequence of states and actions drawn from the trajectory distribution determined by m.
In our case, the elements of A are determined on-the-fly following an RL. policy na, parameterized by 0, conditioned on some states that characterize the status of the back-propagatin step. The policy Hy updated, concurrently with the weights W encs by an RL agent that interacts with the back-propagation process modeled as the MDP defined as M = (S,A,,, R,, 7), with each of its elements introduced below.
at the current time
State Space S. The state space characterizes the agent's knowledge of the environment at each time step. To constitute the states we consider the 4-tuple (x, m, (.) including current training input x E IR,, the missing indicator m E R&, the feature Ç. i.e., the outputs of the encoding layer of MLPs the hidden states h, of LSTMs, and the predictions ý produced by the inference layers.
Action Space A. We consider combining the importance matrix A [0, with the parameter gradients Fi /2W eGD)SGD when updating Wene following
W'nne + Wene - a0E/ NalE//WWeeeeeee ene)sGD @ AA= Wene - aA (x @ (aa))
(4)
where here the equality follows from Q and Proposition I such that all rows of A can be set to be equal to the importance a E [0. which is obtained from the policy following a = Tt(s).
Transitions P S x A S. The transition dynamics determines how to transit from a current state the next state s' given action in the environment. In our case, the pair (x, m) is sampled from the training dataset at each step, so the transitions x + x' and m -> m' are determined by how training samples are selected from the training batch during pack-propagatio
The update of W Wene- + W is conditioned on the importance a as captured in (4) which results in the transitions Ç = fene(x| ene) + ç' fene W W'ne). The update of Winf -> W follows
('' from the regular SGD updates. Then, the transition ý = [(x[W) + y' = f(x' W") follows from the other transitions defined above. Finally, we can define the transition between states as s = (x, m. (, + s' = (x', m'','',3'
Reward Function R. After the RL agent takes an action a at the state s, an immediate reward R(s a) is returned by the environment which provides essential information to update no (Silver 2014 Lillicrap et al. 2015). We define the reward function as R(s, a) = y).
et al.
We utilize actor-critic RL (Silver et al. 2014: Lillicrap et al. 2013) to update ng. which outputs the importance a that is used to concurrently update W Wene- Specifically, we train the target policy (or actor) ne along with the critic Q. S x A + R. parameterized by v, by maximizing Ja(T)) Eg-ps [Q, Te (s))] which gives an approximation to the expected total reward (TT). Specifically, the trajectories P3 = {(5a, ag). (s1, a1),... |a; = [sx))) collected under the behavioral policy 3 S + A are used to update O and jointly following
V' - v + a (s,
0' + A + og VoTo V Qu a)la=xo(s):
(5)
where 3 is usually obtained by adding noise to the output of स. ensure sufficient exploration of the state and action space, 6 = r + 77 (s', (ss)))) - (s a) is the temporal difference error (residual) in RI, and og Og are the learning rates for 0, v, respectively.
4

---

Published as a conference paper at ICLR 2022
Algorithm 1 Gradient Importance Learning (GIL). Input: X, Y.Y. M, W enc. W inf. TT. Qu. Ogg, ays at E
Begin:
Initialize W enc and W Winf- actor Te and critic Qu
2: Sample x from A' and obtain the corresponding label y from 3
3: Obtain the feature + fene Wene) and prediction y - Senf (C) iin) from the encoding and inference layers, respectively
4: s + (x, m. (.
5: for iter in 1 mar mar_iter do
6: Obtain importance from a behavioral policy a = (s)
7: Train the encoding layer following W ene <- W ene a (x Oa in (4
8:
Train the inference layers following regular gradient descent, i.e.,
Win Win fn a (0E/0W in )
9:
Obtain the prediction following the updated weights ý + f(x W eeeer W Wing) Obtain the reward r <- R(s, a)
11:
Get new sample x' from X' and obtain the corresponding label y' from y
12:
Obtain the feature Ç' + Seno(x' (x'( W en)) and prediction y = [C' kny) from the encod-
ing and inference layers, respectively
13:
s' < (x', m', 3')
14:
Update the actor Te and critic Qu using (s,,, following
15:
s f- s', Wene f- W
enc Winf < W inf
16: end for
We summarize the GIL approach in Algorithm and the detailed descriptions can be found in Appendix B Note that the missing indicator m Edd would be good heuristic replace the importance as it could prevent the gradients produced by the missing dimensions from propagation. However, does not train the model capture the hidden information underlying the missing data, which results in its performance being dominated by GIL as demonstrated in Section 4
2.4 EXTENSIONS OF THE FRAMEWORK
The proposed GIL framework uses RL agents guide the model toward minimizing its objective y), which is usually captured by general prediction losses such as cross entropy or mean squared error. Below we use an example to show how our method can be extended adapt ideas from related domains, which can augment training of the imputation-free prediction models by altering the reward function. Recent works in contrastive learning, e-g. Chen et al. (2020), can train DL models to generate highly expressive features by penalizing (or promoting) the distributional difference D(S ) among the features associated with inputs that are similar to (or different from) one set of homogeneous inputs
each other, where Ç+ denotes the set of features corresponding Xoom. and < are the features generated by data that are significantly different from the ones Xoon However, such methods may not be directly applied to the missing data setting considered in this work, as their loss is usually designed toward unsupervised training and might need to be carefully re-worked in case. This idea be adapted to our framework by defining Ç+ the features mapped from the inputs (by the encoding layers Wene) associated with the same label y E y and < the features corresponding R(s, a) = - E' y) + c D(S+ (-), O, which does not require D to be carefully crafted, provided that W ene not trained by directly propagating gradients from it. In Section it will be shown in case studies that by simply defining D as the mean squared error between Ç+ and Ç can improve the prediction performance.
some other label y' VV Then we can define the new reward function
RELATED WORK
Missing Data Imputation. Traditional mean/median- filling and carrying-forward imputation meth- ods are still used widely, as they are straightforward to implement and interpret (Honaker & King 2010). Recently, there have been state-of-the-an imputation algorithms (IAs) proposed to produce smooth imputations with interpretable uncertainty estimates. Specifically, some adopt Bayesian meth- ods where the observed data are fit to data- mnttaggnnrating models, including Gaussian processes (Wilson cet al. 2016, Fortuin & Rätsch 2019 Fortuin et al., 2018), multivariate imputation by chained equa- tions (MICE) (Aur et al. 201 random forests (Stekhoven & Bühlmann 2012), etc., statistical
5

---

Published as a conference paper at ICLR 2022
optimization methods such as expectation maaitiiaaoaii (EM) Larrnia- al. 2010: Bashir & Wei, 2018). However, they often suffer from limited scalability, require assumptions over the underlying distribution, or fail to generalize well when used with datasets containing mixed types of variables, Le., when discrete and continuous values exist simultaneously (Yoon et al. 2018 Fortuin et al. 2020). There also exist matrix-completion methods that usually assume the data are static, T.e. information does not change over time, and often rely on low-rank assumptions (Wang et al. 2006 Yu et al. 2016; Mazumder et al. 2010; Schnabel et al. 2016). More recently, several Da-based IAs have been proposed following advancements in deep generative models such as deep latent variable models (DLVMs) and generative adversarial networks (GANs) (Mattei & Frellsen 2019]
Yoon et al. 2018: Fortuin et al. 2020 Li et al. 2019; Ma et al. 2018). DLVMs are usually trained to capture the underlying distribution of the data, by maximizing an evidence lower bound (ELBO),
which could introduce high variations to the output distributions (Kingma & Welling 2013) and
cause poor performance in practice, if the data have high missingness (as for the example shown in Figure I. There also exist end-to-end methods that withhold observed values from inputs and impose reconstruction losses during training of prediction models (Cao et al. 2018 [psen et al.) 2020). In addition, data-driven LAs are developed specifically toward medical time series by incorporating prior domain knowledge (Yang et al.) 2018 Scherpf et al., 2019); Calvert et al. 2016,) Gao et al. 2021). On the other hand, a few recent works attempt to address missing inputs through an imputation-free manner (Morvan et al. 20200 Sportisse et al. 2020), however, they are limited to linear regression.
Attention. The importance matrix used in the GIL is somewhat reminiscent of visual attention mechanisms in the context of image captioning or multi-object recognition (Mnih et al. [2014] Ba et al. 2014, Xu et al., 2015) as they both require training of the prediction models with RL. We briefly discuss the connections and distinctions between them, with full details provided in Appendix C Visual attentions are commonly used to train CNNs to focus on specific portions of the inputs that most helpful for making predictions. They determined by maximizing an evidence lower bound (ELBO), which is later proved to be equivalent to the REINFORCE algorithm in the RI. literature (Sutton& & Barto 2018 Mnih al. 2014). However, these methods cannot be applied directly to our problem, as they require features to be exclusively associated spatially with specific parts of the inputs, which is attainable by using convolutional encoders with image inputs but intractable with the type of data considered in our work. Instead, our approach overcomes this issue by directly applying importance weights generated by RL, into the gradient space during pack-propagation Such issues motivate use of the term importance instead of atlention. Lastly, our method does not require one to formulate the learning objective as an ELBO, and as a result GIL can adopt any ofthh-of-the-at RL algorithm, without being limited to REINFORCE as in Mnih et al. 20044;; Ba et al. (2014); Xu et al. (201)). Besides, other types of attention mechanisms are also proposed toward applications in sequence modeling and natural language processing (NLP) such as Cheng et al. 2 Vaswani et al. (2017).
4 EXPERIMENTS
We evaluate the performance of the imputation-free prediction models trained by GIL against the existing imputation imputation-prddicionn prediction paradigm on both benchmark and real-world datasets, where the imputation stage employs both otate-of-the-art and classic imputation algorithms (IAs), with the details introduced in Section[4. We also consider variations of the GILL method proposed in Section 2 to illustrate its robustness and flexibility. The datasets we use include i) MIMIC-II (Johnson et al. 2016) that consists of real-world EHRS obtained from intensive care units (ICUs), id) de-identified ophthalmic patient dataset obtained from an eye center in North America, and iii) hand-written digits MNIST (LeCun& & Cortes). We also tested on a smaller scaled ICU time-series from 2012 Physionet challenge (Silva et al. 2012) and these results can be found in Appendix D.4 Some of the data are augmented with additional missingness to ensure sufficient missing rates and the datasets we use cover all types of missing data, i.e., missing complete at random (MCAR), MAR and MNAR (Liutee Rubin 2019). We found that the proposed method not only outperforms the baselines on all three datasets under various experimental settings, but also offers better feature representations as shown in Figure 3 We start with an overview of the baseline methods, in the following sub-section, and then proceed to present our experimental findings.
4.1
VARIANTS OF GIL AND BASELINES
Our method (GIL) trains MLPs or LSTMs (depending on the type of data) to directly output the predictions given incomplete inputs without imputation. Following from Section 2.4| we also test on
6

---

Published as a conference paper at ICLR 2022
Table I: Accuracy and AUC obtained from the MIMIC-III dataset.
GIL
GAIN
MIWAE
GA-VAE
BRITS
MICE
Mean
CF
KNN
EM
Var-L.
Acc. 93.32
93.09
89.17
90.32
88.71
92.17
88.02
87.32
84.79
75.81
68.20
AUC 96.10
96.79
92.96
95.57
94.28
25.97
92.56
91.78
91.86
81.73
75.23
Fix-I.
Acc. 91.47
91.01
88.25
88.48
86.18
76.50
80.24
90.09
86.41
86.87
85.48
78.11
70.51
AUC 95.29
95.57
92.99
91.94
93.10
81.47
92.13
94.02
91.69
91.98
92.38
84.54
79,97
binary classification tasks using GIL-D which includes the distributional difference term D(C ,(t+,5)). captured by mean squared errors, into the reward function. For baselines, we start with another variant of GIL that uses a simple heuristic - the missing indicator m- to replace the importance a in GIL,; we denote it as GHL.-H. This helps analyze the advantages of training the models using the importance obtained from the RL policy learned by GII,, versus heuristic that simply discards the gradients produced by the subset of input entries that are missing.
For other baselines, following the imputation-prediitioon predictiod framework, the imputation stage employs state-of-the-art IAs including MIWAE (Mattei & Frellsen 2019) which imputes the missing data by training variational auto-encoders (VAEs) to maximize an ELBO, GP-VAE (Fortuin et al.) 2020) that couples VAEs with Gaussian processes (GPs) to consider the temporal correlation in senessries, and the generative adversarial network-basct method GAIN (Yoon et al. 2018). Further, some classical imputation methods are also considered, including MICE, missForest (MF), k-nearest neighbor (kNN), oppectation-maxxiizzaiioo (EM), mean-imputation (Mean), zero-imputation (Zero) and carrying- -orrarrd (CF). The imputed data from these baselines are fed into the prediction models that share the same architecture as the ones trained by GIL, for fair comparison. For time-series, we also compare to BRITS (Cao et al. 2018) which trains bidirectional LSTMs end-to-end by masking out additional observed data and jointly imposing imputation and classification objectives.
4.2 MIMIC-III
We consider the early prediction of septic shock using the MIMIC-III dataset, following the frame- works provided in recent data-driver works for data pre-processing (Sheetrit et al.) 2017 Fleuren vital signs and
al. 2020 Khoshnevisan et al. 2020). Specifically, we use 14 commonly-utin lab results over a 2-hour observation window to predict whether patient will develop septic shock in the next 4-hour window. The resulting dataset used for training and testing contains 2, 166 sequences each of which has length between 3 and 125, and the overall missing rate is 7 71.63%; the missing data in this dataset can be considered as a mixture of MCAR, MAR and MNAR. More details can be found in Appendix D.I
To evaluate performance, we consider the prediction model constituted by a 1024-unit LSTM layer for encoding followed by a dense output layer for inference. Considering that some of the baseline methods are not designed to capture the temporal correlations within the sequences which do not follow fixed horizon, besides testing with the varied-length sequences (Var-l.) obtained in above, we also test with fixed-length version (Fix-I.) where the maximum time step for each sequence is set to be (see Appendix D.1 for details). The accuracy Securay/aand and AUC for the two tests are summarized in Table I When varied-length sequences are considered, GIL(-D) slightly outperforms GAIN and MICE while it significantly dominates the other baseline methods. However, when fixed- length sequences are considered, GAIN and MICE's performance decreases dramatically and are significantly dominated by that of GIL(-D). This could be caused by omitting the records beyond the 8-th time step as both models are flexible enough to capture such information from that point on- wards, while in contrast, the performance increases for KNN, MF and EM which are in general based on much simpler models. On the other hand, the models trained by GIL(-D) was not significantly affected by the information loss. In fact, it emphasizes that applying importance to the gradients during training can enable the model to capture the information behind missing data. Moreover, the dramatically increased performance from GIL-H to GIL(-D) in both tests underscores the significance of training the models using the importance determined by the RL policy learned by GIL(-D), instead of pre-defined heuristic. Note that according to a recent survey of septic shock predictions (Fleuren et al. 2020), the overall maximum AUC attained by existing data-driven methods, which uses domain knowledge to design models specifically for ICU time-series, is 0.96 and it is comparable to that attained by our method. Finally, GP- VAE and BRITS require the input sequences to have the same horizon, we only report for the Fix-l. case where they under-perform. These are possibly due to
'll accuracy values in this work obtained using decision threshold of 0.5.
7

---

Published as a conference paper at ICLR 2022
Table 2: Average Accuracy and AUC obtained from the Ophthalmic dataset over 3 different random masks. Subscripts are standard deviations.
M.R.
GIL
-D
-11
GAIN
GP-VAE
MIWAE
MICE
Zero
Acc.
86.84,.43
87.13,.09
83.630.83
85,38
85.67p
80.99
84.2
84,50
25%
AUC
92.401.33
92.42).44
88.872.16
90.132.0e
91.471.0z
84.041.47
91.351.a3
90.596.a2
Acc.
35%
83.33 0.72
85.09 2.58
80.41 1.49
80.41 0.83
80.41g.s3
39.244.38
80.41
80. 12..09
AUC
88.49 0.68
90.68 2.36
87.02 3.03
88.02
2.35
84.85
85,51 3.47
85.87 a.24
88.02 0.97
[0]
Figure 3: L-SNE visualization of the feature space learned by (a) GIL, (b) MIWAE and (c) GAIN on the MNIST dataset with 90% missing rate.
the high variation in GPVAF's uncertainty estimation component, and the mechanism of imposing additional missingness in BRITS"
4.3 OPHTHALMIC DATA
We consider identifying diabetic retinopathy (DR), an eye disease caused by diabetes, using the de-identified data obtained from Duke Eye Center, constituted by a mixture of image feature vectors and tabular information. Specifically, the data collected from each subject is formulated as vector with 4,101 dimensions containing two 2,048-length feature vectors from two retinal imaging modalities, optic coherence tomography (OCT) and color fundus photography (CFP), followed by a 5-dimensional data characterizing demographic (e.g., age) and diabetic information (e.g., years of diabetes). Additional details of the dataset and examples of the OCT and CFP images are provided in the Appendix [.2] At most one of the two types of retinal images could be missing due to MAR as sometimes the diagnosis can be performed with a single modality (Cui et al. [021]. while the demographic/diabetii information can be missing due to MCAR. A total of 1,148 subjects are We apply additional random masks over both the imput image features (by assuming one two entire image feature vectors are missing) and demographic/diabetic information (by assuming some entries are missing) to increase the missing rate 25% % and 35%, where for each case 3 different random masks are applied.
included in this dataset with a 17% missing rate (M.R. F
For this experiment we consider an MLP with 2 hidden layers with 1,000 nodes each for prediction. The results are shown in Table 2 where the mean and standard deviations (in subseripts) of accuracy and AUC are reported. Although GIL and GP-VAE result in similar performance in the 25% missing rate case, GIL still achieves higher accuracy and AUC; these are significantly improved, with significantly lower standard deviation over GP-VAE, when the missing rate increases to 35%. In general, GIL-D outperforms all the other methods, with an exception of the higher standard deviation of accuracy in the 35% M.R. case, which benefits from the flexibility of the framework we proposed. With increased missingness, the performance of most baselines drops significantly and are close to the zero- pero-imputation baseline as the image feature vectors occupy most dimensions of the inputs while imputing with zeros would be a reasonable heuristic.
4.4 MNIST
This work focuses mostly on tabular inputs or time-series; however, we test on MNIST images given its simple structure, which could be classified using MLPs and more importantly, because results from these data are easier to interpret. Specifically, we test on the MCAR version of MNIST where a pre-determined portion of pixels (i.e., out of 50%, 70% and 90%) are masked off uniformly at random from the original images, and the MAR version, which assumes part of the image is always observable and the positions of missing pixels are determined by distribution conditioned on features extracted from the observable portion following Mattei & Frellsen 2019). For each test, the masks are applied
"The typer-parameters used to train their models can he found in Appendix D.
When calculating missing rates, each feature vector is only counted as single entry regardless of size.
8

---

Published as a conference paper at ICLR 2022
Table 3: Average Accuracy reported for the MNIST dataset over different missing rates. For each missing rate, 5 random masks were used resulting in standard deviations shown as subscripts (x10-)
M.R.
GIL
-H
GAIN
MIWAF
MICE
7ero
GP-VAF
50%
96.29o.7
96.081.2
96.230.7
95.460.8
94.58p.2
95.831.7
96.573.1
MCAR
70%
93.350.9
93.260.9
91.983.8
93.200.s
90.53g
92.80o.s
93.493
90%
78.473.9
78.444.7
72.58g
77.9
73.48s
76.675.3
76.25²
MAR
93.230.3
93.15
92.970.8
93.18o.4
92.621.1
82.502.9
92.621.3
Table 4: Correlation between imputation MSE and prediction accuracy for different MRs.
MNIST
Ophthalmic
MIMIC Ground-truths Unknown
50% M.R.
70% M.R.
90% M.R.
25% M.R.
35% M.R.
Pearson's Coeff. -0.48
-0.88
-0.86
-0.12
-0.54
p-value 0.018
<0.01
<0.01
0,581
<0.01
with 5 different random seeds, resulting in the standard deviations reported in Table 3 which also summarizes obtained results.
We consider using MLPs constituted by two hidden dense layers with 500 nodes each as the prediction model.]] Note that in the MCAR setting. GP-VAE achieves slightly higher average accuracy than our method when 50% and 70% of the pixels missing, because it relies on convolutional encoding layers, which intrinsically have advantages on this dataset. However, GA-'sE's performance is associated with higher standard deviation in both cases and is significantly outperformed by our method when 90% pixels are missing. The models trained by GIL also significantly outperforms the baselines following the imputation-prediccinn framework. This suggests that most of the IAs fail to reconstruct the input data when the missing rate is high (see Figure D. Moreover, the errors produced during the imputation In the MAR setting, the task becomes more challenging as entire rows of pixels could be missing. Finally, the model trained by GIL. outperforms most of the baselines depending exception of MIWAF, which only slightly outperforms it. Note that GIL.-H's performance is close to GIL in most of the settings due the simple structure of the MNIST digits where the missing indicator m could be a good estimation of the importance a.
then propagated into the prediction step which results in the inferior performance.
TAs with an
Feature Space Visualization In Figure we use (-distributed stochastic neighbor embedding (1- SNE) to visualize the features learned by the prediction model trained by GIL compared to MIWAE and GAIN, We observe that GIL results in more expressive features than the others as it generates clusters with clearer boundaries.
4.5 CORRELATION BETWEEN IMPUTATION AND PREDICTION PERFORMANCE
In this section we study if imputation error is correlated with the downstream prediction performance, under the Pearson's 's correlation coefficient (Freedman et al., 2007) and its p-value, between the imputation mean squared error (MSE) and prediction accuracy, across different imputation methods under the same dataset. It can be observed that the imputation MSE is negatively correlated with prediction performance for most of the datasets and data with higher M.R. tends to have higher degree of negative correlation, which indicates that imputation error could be propagated downstream tasks.
prediction framework. Each column of Table 4 is obtained by calculating the
CONCLUSION
We have developed the GIL method, training DL models (MLPs and LSTMs) to directly perform inference with incomplete data, without the need for imputation. Existing methods addressing the problem of missing data mostly follow the two-step iimputation--rediction framework. However, the еттог produced during imputation can be propagated into subsequent tasks, especially when the data have high missingness rates or small sample sizes. GIL circumvents these issues by applying importance weighting to the gradients to leverage the information underlying the missingness patterns in the data. We have evaluated our method by comparing to the tfate-of-the-art baselines using IAs on two real-world datasets and one benchmark dataset.
'The performance of MIWAE reported here is different from Mattei & Frellsen 2019 hecause they used
CNN classifier.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
This research was supported part by the ONR under agreements N00014-17-1-2555 and N00014-20- 1-2745, AFOSR under award number FA9550-19-1 19-1-0199 NIH under NIH/NIDDK R01-DK123062 and NIH/NINDS IR6INS120246 awards, DARPA, as well as the NSF under CNS- award and the National AI Institute for Edge Computing Leveraging Next Generation Wireless Networks, Grant 2N1-2112562.
We would also like to thank Ruiyi Zhang (Adobe Research) for advice that leads to improved paper presentations, and Ge Gao (North Carolina State University) for sharing insights toward processing the MIMIC-III dataset well as the use of some imputation baselines.
REFERENCES
Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. Tensorflow: A system for large-scale machine learning. 2016.
Melissa Azur, Elizabeth A Stuart, Constantine Frangakis, and Philip J Leaf. Multiple imputation by chained equations: what is it and how does it work? International journal of methods in psychiatric research, 20(1):40-49, 2011.
Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. Multiple object recognition with visual attention. arviv preprint arXiv: 1412.7755, 2014.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint 0rX9v:1409.044 2014.
Faraj Bashir and Hua-Liang Wei. Handling missing data multivariate time series using a vector autoregressive model-imputation (var-im) algorithm. Neurocomputing, 276:23-30, 2018
Christopher M Bishop. Pattern recognition and machine learning. Springer, 2006.
Justin Boyan and Andrew W Moore. Learning evaluation functions to improve optimization by local search. Journal of Machine Learning Research, I(Nov):77-112, 2000.
Jacob S Calvert, Daniel A Price, Uli K Chettipally, Christopher Barton, Mitchell D Feldman, Jana Hoffman, Melissa Jay, and Ritankar Das. A computational approach to early sepsis detection. Computers in biology and medicine, 74:69-74 73, 2016.
Wei Cao. Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputation for time series. arXiv preprint 1rXiv:1805.10572 2018.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pp. 1597-1607. PMLR, 2020.
Jianpeng Cheng, Li Dong, and Mirella Lapata. Long short-term memory-networks for machine reading. arXiv preprint 1601 106733, 2016.
Ying Cui, Ying Zhu, Jay C Wang, Yifan Lu, Rebecca Zeng. Raviv Katz, Filippos Vingopoulos, Rongrong Le, Inês Laíns, David M Wu, et al. Comparison of widefield swept-source optical coherence tomography angiography with ultra- titaf-ld colour fundus photography and fluorescein angiography for detection of lesions in diabetic retinopathy. British Journal of Ophthalmology. 105(4):577-581, 2021.
Aiman Darwiche and Sumitra Mukherjee. Machine learning methods for septic shock prediction. In Proceedings of the 2018 International Conference on Artificial Intelligence and Virtual Reality, pp. 104-1 100, 2018
10

---

Published as a conference paper at ICLR 2022
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2): 295-307, 2015.
Lucas M Fleuren, Thomas LT Klausch, Charlotte L Zwager, Linda Schoonmade, Tingjie Guo, Luca F Roggeveen, Eleonora L Swart, Armand RJ Girbes, Patrick Thoral, Ari Ercole, et al. Machine learning for the prediction of sepsis: systematic review and meta-analysis of diagnostic test accuracy. Intensive care medicine, 46(3):383-400) 2020.
Vincent Fortuin and Gunnar Rätsch. Deep mean functions for meta-learning in gaussian processes. arXiv preprint FrXiiv1901.08000 2019.
Vincent Fortuin, Gideon Dresdner, Heiko Strathmann, and Gunnar Rätsch. Scalable gaussian processes discrete domains. arXiv preprint arXiv: 10368, 2018.
Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. Gp-vae: Deep probabilistic time series imputation. In International Conference on Artificial Intelligence and Statistics, pp. 1651-1661. PMLR, 2020.
David Freedman, Robert Pisani, and Roger Purves. Statistics (international student edition). Pisani, R. Purves, 4th edn. WW Norton & Company, New York, 2007.
Qitong Gao, Joshua Amason, Scott Cousins, Miroslav Pajic, and Majda Hadziahmetovic. Automated identification of referable retinal pathology in telcophthalmolggy setting. Translational vision science & technology, 10(6):30-30, 2021.
Pedro CanninaLa José-Luis Bancho-Gómez, and Aníbal Figueiras Vidal. Pattern clas- sification with missing data: review. Neural Computing and Applications, 19(2):233-222 2010.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
James Honaker and Gary King. What to do about missing values in time-series cross-section data. American journal of political science, 2001
Niels Ipsen, Pierre-Alexandre Mattei, and Jes Frellsen. How to deal with missing data in supervised deep learning? In JCML Workshop on the Art of Learning with Missing Values (Artemiss). 2020.
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-Wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, freely accessible critical care database. Scientific data, 3(1):1-9, 2016.
Kaggle. Kaggle diabetic retinopathy detection competition. https: // www www.ka..gg..oo.oo.. diabetic- ret ientiinopattt dww.ctteeoon.oooo - 2015. Accessed: 2021-05-00
Hye Jin Kam and Ha Young Kim. Learning representations for the early detection of sepsis with deep neural networks. Computers in biology and medicine, 89:248-222 225, 2017.
Daniel S Kermany, Michael Goldbaum, Wenjia Cai, Carolina CS Valentim, Huiying Liang, Sally Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, 172(5): 122-1131, 2018.
Farzaneh Khoshnevisan et al. A variational recurrent adversarial multi-souree domain adaptation framework for septic shock early prediction across medical systems. 2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.61 2013.
arXiv preprint
Yann LeCun and Corinna Cortes. MNIST handwritten digit database.
Yann LeCun, Patrick Haffner, Léon Bottou, and Yoshua Bengio. Object recognition with gradient- based learning. In Shape, contour and grouping in computer vision, pp. 319-345. Springer, 1999.
11

---

Published as a conference paper at ICLR 2022
Ke Li and Jitendra Malik. Learning optimize. arXiv preprint arXiv: 1r0x:0:6006018883, 2016.
Steven Cheng-Xian Xian Li, Bo Jiang, and Benjamin Marlin. Misgan: Learning from incomplete data with generative adversarial networks. arXiv preprint arXiv: 1rXiv:1902.09599 2019.
Timothy Lillicrap, Jonathan Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 02971, 2015.
Zachary C Lipton, David Kale, and Randall Wetzel. Directly modeling missing data in sequences with mnss: Improved classification clinical time series. In Machine learning for healthcare conference, pp. 253-270. PMLR, 2016.
Roderick JA Little and Donald Rubin. Statistical analysis with missing data, volume 793. John Wiley & Sons, 2019.
Ran Liu, Joseph L Greenstein, Stephen Granite, James Fackler, Melania Bembea, Sridevi Sarma, and Raimond I. Winslow. Data-driven discovery of novel sepsis pre-shock state predicts impending septic shock in the ICU. Scientific reports, 9(1):1-9, 2019.
Chao Ma, Sebastian Tschiatschek, Konstantina Palla, José Miguel Hernández-Looato, Sebastian Nowozin, and Cheng Zhang. Eddi: Efficient dynamic discovery of high-value information with partial VAE. arXiv preprint arXiv:1809 11142, 2018.
Behrooz Mamandipoor, Mahshid Majd, Monica Moz, and Venet Osmani. Blood lactate concentration prediction in critical care patients: handling missing values. arXiv preprint 1rXiv:1910001473.
2019.
Qingqing Mao, Mclissa Jay, Jana L Hoffman, Jacob Calvert, Christopher Barton, David Shimabukuro, Lisa Shieh, Uli Chettipally, Grant Fletcher, Yaniv Kerem, et al. Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU. BMJ open, 8(1), 2018
Pierre-Alexandre Mattei and Jes Frellsen. Miwae: Deep generative modelling and imputation of incomplete data sets. In International Conference on Machine Learning, pp. 4413 4423. PMLR, 2019.
Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. The Journal of Machine Learning Research, 11:2287-2322. 2010.
Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual attention. Advances in Neural Information Processing Systems, 27:2204-2211 2014.
Marine Le Morvan, Julie Josse, Thomas Moreau, Erwan Scornet, and Gaël Varoquaux. Neumiss networks: differentiable programming for supervised learning with missing values. 2020.
Matthieu Scherpf, Felix Gräßer, Hagen Malberg, and Sebastian Zaunseder. Predicting sepsis with a recurrent neural network using the MIMIC III database. Computers in biology and medicine, 1 113: 103395, 2019.
Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. Recommendations as treatments: Debiasing learning and evaluation. In International Conference on Machine Learning. pp. 1670-1679. PMLR, 2016.
Eitam Sheetrit, Nir Nissim, Denis Klimov, Lior Fuchs, Yuval Elovici, and Yuval Shahar. Temporal pattern discovery for accurate sepsis diagnosis in ICU patients. arXiv preprint 1r9
2017.
Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. Predicting in-hospital mortality of ICU patients: The physionet/computing in cardiology challenge 2012. In 2012 Computing in Cardiology, pp. 245-248 IEEE, 2012.
12

---

Published as a conference paper at ICLR 2022
David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. Deterministic policy gradient algorithms. 2014.
Mervyn Singer, Clifford Deutschman, Christopher Warren Seymour, Manu Shankar-Hari, Djillali Annane, Michael Bauer, Rinaldo Bellomo, Gordon R Bernard, Jean-Daniel Chiche, Craig M Coopersmith, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). Jama, 315(8):801-88))) -100, 2016.
Aude Sportisse, Claire Boyer, Aymeric Dieuleveut, and Julie Josses. Debiasing averaged stochastic gradient descent to handle missing values. Advances in Neural Information Processing Systems, 33, 2020.
Suhas Sreehari, SV Venkatakrishnan, Katherine I. Bouman, Jeffrey P Simmons, Lawrence F Drummy, and Charles Bouman. Multi-resolution data fusion for super-resolution electron microscopy. In Proceedings of the IEEE conference computer vision and pattern recognition workshops, pp. 88-96, 2017.
Daniel Stekhoven and Peter Bühlmann. neneiforest-onn-paaaaa missing value imputation for mixed-type data. Bioinformaties, 28(1):112-118, 2012.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Z.bigniww Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 28 1822826, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, pp. 5998-6008, 2017.
Jun Wang, Arjen P De Vries, and Marcel JT Reinders. Unifying user- and item-based collabo- rative filtering approaches by similarity fusion. In Proceedings of the 29th anmual international ACM SIGIR conference on Research and development in information retrieval, pp. 501-508, 2006.
Kaixuan Wei, Angelica Aviles Rivveo Jingwei Liang. Ying Fu, Carola-Bibiane Schönlicb, and Hua Huang. Tuning-free plug-and-ple proximal algorithm for inverse imaging problems. In International Conference on Machine Learning, pp. 101699 PMLR, 2020.
Ronald J Williams. Simple statistical fruddcen--fllowiing algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229 25665.229-226. 1992.
Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P Xing. Deep kernel learning. In Artificial intelligence and statistics, pp. 370-378. PMLR, 2016.
Yan Wu, Mihaela Rosca, and Timothy Lillicrap. Deep compressed sensing. In International Conference on Machine Learning, pp. (8600. PMLR, 2019.
Kelvin Xu, Jimmy Ba. Ryan Kiros. Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In International conference machine learning, pp. 2048-2057, 2015.
Xi Yang, Yuan Zhang, and Min Chi. Time-aware subgroup matrix decomposition: Imputing missing data using forecasting events. 2018 IEEE International Conference on Big Data (Big Data), pp. 1533. IEEE, 2018
Christopher R Yee, Niven R Narain, Viatcheslav R Akmaev, and Vijetha Vemulapalli. A data-driven approach predicting septic shock in the intensive care unit. Biomedical informatics insights, 11: 1178222619885 2019.
Jinsung Yoon, James Jordon, and Mihaela Schaar. Gain: Missing data imputation using generative adversarial nets. In International Conference Machine Learning, pp. 5698. PMLR, 2018.
Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. Temporal regularized matrix factorization for nigh-dimensions time series prediction. In Advances in Neural Information Processing Systems, pp. 847-855, 2016.
13

---

Published as a conference paper at ICLR 2022
A EXTENDING GIL TO LSTMS
LSTM Model. We also consider the use an LSTM as the encoder for sequential inputs with varied lengths. Specifically, in this case, we can define Xj (,2 x,,,) with M, (m) I j....., ..m,,,m,
E 7 along
f,r:)) ) E {0, 1) such that each x₁, E R and my E [0,
where E [1, N and E [1,Tg]. Recall that the forward pass of an LSTM follows
o, oo j. -oh i = oW 11hy-1), f o ,x Uphe-1), (6)
g: = tanh xXH + -ghh-)), C = O c.- +i 1i0g gg. h, or @ tanh(c;))
where x, E R is the input at time step t. (Wo. Wi, Wg. Wr, U. U, Ugg Ur} are the weights, (x)) = (1 + e=* is the sigmoid function, and is the element-wise product. The output layer is appended after h2 to constitute the inference layer, i.e.,
y: Dout ( out he),
where Pout the activation function, Wini inf proposition shows that given a loss function E(:.),, the gradients of E w.r.t the encoding weights W enc (W.. Wi, W Wr) can written in the form of outer products. Note that the gradients for (autoregressive) parameters depending on previous hidden states h,-1. i.e.. {Ua, U, U Ur). like the bias terms, cannot be written as outer products; thus, these weights are updated following regular SGD, without importance weighting. The proof is provided following the proposition.
W out are the weights and [1, The following
Proposition 2 Given an LSTM as described in - and a smooth loss function E y), the gradients of E w.E.t. W enc { (Wo. Wi, W g. Wr at time be written outer product forms aE A ow, A where
i.e., it holds that ag aW.
A
DE
A|x,
aE
aW;
aW
AP A A A are the gradients propagated from the inference layers defined in below. and x reprexents the observation obtained time step L from any sequence X; X in general.
Then it follows that if we define x as the t-th observation from any sequence X, E RT re- gardless of it index, (4) can be directly used for training the LSTM encoding weights W enn [W., Wi, W g: Wr).
Now prove the Proposition above.
We first define
A -E//hh, O tanh(c:) C (l-
(8)
A -0E/Oh, o, & (1 tanh (c.)) og 0 (1-i),
A
-0/0hh Oo. C
(i
(10)
A -e//bh
O o, @
o ft).
(11)
Now to prove the proposition we start from the derivative of the smooth loss function E w.r.t. which can be derived as
OF
O(W out
h1)
(12)
Oh,
Oh,
ØE
D'(W (ooche))
(13)
0g
=WT
① 0 D' ond Woulhe)
(14)
out
14

---

Published as a conference paper at ICLR 2022
Then the derivatives of E w.r.t the O,, Ct2 f,, Cc-1, and g:
OF
0F Oh,
do: Th do:
aE
@ tanb(c;)
(15)
Oh.
DE
dE Oh:
oc, Oh dc,
0F
: or @ (1 - tan)
(16)
bh.
DE
DE 0h dc.
ar.
Oh, dc, Or,
0F
C o (1 tanh?(()))))) o C-11
(17)
dh,
OE
0E 0h, dc.
Oc,-1
Oh, dc, Oc--1
0L
8 o, ④ tanh? (an))(c,)) ff,
(18)
oh,
0E
aE dh dc:
0i,
Th, dc, oi,
dE
o. (1 tanh (c,) og
(19)
0E
DE dh, dc,
0g.
Th, de, 0g:
O
@ or @ (1 tanh @i.
(20)
Oh,
Now, the derivatives of E w.r. the weights W.. W. W g and Wt
OE
DE Th, do:
(21)
OW.
Oh, do: dW. TE
tanh(c:) @ oi) x,
(22)
Oh,
A
(23)
0E
OE oh, dc, Hi
(24)
ow,
Th, dc, Ri aW;
OE
Oo C (1 -
i.) xT
(25)
Oh,
A|x
(26)
OE
OE Ōh, dc, 0g.
(27)
ow
Th, dc, dg. aW g
0E
O (1 tanh (a)) 83) 83)
(28)
Lh;
-fxT
(29)
0F
O Oh, dc, Of.
(30)
ow,
Th, dc, Of, oWr
OE
o o, @ (1 tanh tanh?(c:)) @ c-1 0 (1 x
(31)
Oh,
A,T.T
(32)
15

---

Published as a conference paper at ICLR 2022
Note that since d') = a0)((1 o()), in the transition between (21 and (22) it follows that
0o:
=0 <' + U.h
(33)
aw.
0 (Wox + -.h (1-- (Wox + Uoh(-1)) x
(34)
(1-))
(35)
Similarly, the transition between (24 and 25 follows
di,
oW o i1)
(36)
At last, the transition between 30 and 31 follows
öf.
(1-f).
(37)
owr
Furthemore, since t-nh'()) tanh
(-), (.),
the transition between 27 and 28 follows
0g.
tanh' (W gx Ugh
(38)
dWg
=(1 - lanni Manli2(Wa,
W xx + U Ugh+-1))x;
(39)
= -11 - gi x
(40)
which concludes the proof.
16

---

Published as a conference paper at ICLR 2022
B DESCRIPTION OF ALGORITHM 1
The algorithm takes as input the training dataset X' the training targets V the missing indicators M, the weights of the encoding layers W enee the weights for the inference layers inf. the actor TT. the critic Qu learning rates {ar Oig, ) and training loss function E. Our approach starts by initializing all the parameters W enc- W fnf= mg. Qu sampling x X with the corresponding m M that will be used for training in the first iteration, obtaining the feature Ç and the prediction y, which constitute the initial state as 8 (x, m, (.)). In each iteration, first the importance generated from behavioral policy / that is conditioned on the target policy æo, such as the noisy exploration policy proposed in Lillicrap et al. al. (2015). Then the encoding layer trained following ( while the inference layers are trained following the regular gradient descent. After training, the new prediction obtained following the updated weights and its value is assigned to y, which is then used to generate the reward following the reward function R. Then the training sample x' for the next iteration is sampled and the corresponding m', 5' <' are obtained to constitute the next state s'. Finally, the actor we and critic Qu are updated following (5) We refer to Silver et al. (2014); Lillicrap al. (2015) for more details on actor-critic RL.
17

---

Published as a conference paper at ICLR 2022
C IMPORTANCE VS ATTENTIONS
We now illustrate the connections and distinctions between the importance in the GIL and visual attentions Mnih et (2014); Ba et al. (2014); Xu al. 2015), which are commonly used train CNNs to focus on specific dimensions of the inputs that are most helpful for making predictions in the context of image captioning and multi-object recognition.
visual attentions, an input image is first encoded into set of vectors I {l..... 1L] where each 1; is feature vector corresponding to a specific region in the image as they are retrieved from lower convolutional layers. Then, the attention oz,i for time step and region i E [1, r.] is generated following BBahdanau et al. (2014) as ot, cxp(et,)) Σラ= exp (et,k), where ct,: face (1,, -1) fau is usually an MLP (or the so- called attention model) and h the hidden state recurrent neural network (RNN) Hochreiter & Schmidhuber (197)) used for prediction. Then, weighted average of the feature vectors 2; 4, fed into the prediction network for further inference where 4,, a random variable parametrized by C, following p(4, = I) = Otti and list represents the historical values of 4,i for all i E L].
Given the definition of the multinoulli variable h.i. the model is not smooth and thus cannot be trained following the regular back-propaal propagattnn,.nn So the prediction network trained to maximize the evidence lower bound (ELBO) which updates the prediction network parameter using
OJ 00
log p(y|ū,, + (log a loog Zl. 00
(41)
n
00
where i, are the Monte-Carlo samples drawn from p(4, jl, and represents the performance from a baseline E log p(y oulyyOoeen (0bsseline) )]. It is notable that (41) is equivalent to the REINFOCE algorithm RI. (illiamss 1992). Specifically, the search space can be interpreted as an MDP which usually used to model the environment in RL problems i.e., the state space is constituted by I, the RNN hidden state h, and the historical visitations ljcc: the actions are multinouli variables 4,,6 the reward function is defined the marginal log-likelihood log (.,p((14...))) I); and the policy is characterized by p(le,il1 I) Xu et al. (2015): Mnih et al. 20014 Ba al. (2014).
However, these methods cannot be applied directly to our problem as it requires the features Z to exclusively associate with specific parts of the imputs spatially, which is attainable by using convolutional encoders with image inputs but intractable with tabular inputs considered in our case (see Section 1. Instead, our approach overcomes this issue by directly applying importance, generated by RL, into the gradient space during back-propagation Moreover, our method does not require to formulate the learning objective as ELBOs and as result GIL can adopt any state-of-the-ar RL algorithm not limited to REINFORCE only in Mnih et al. (2014) Ba et al. (2014); Xu et al. (015))
18

---

Published as a conference paper at ICLR 2022
D EXPERIMENTAL DETAILS AND ADDITIONAL EXPERIMENTS
The case studies are run on work station with three Nvidia Quadro RTX 6000 GPUs with 24GB of memory for each. We use Tensorflow to implement the models and training algorithms. To train the imputation-free prediction models using GIL, we perform a grid search for the model learning rate a {0.001, 0.0007, 0.0005, 0.0003, 0.0001, 0.00005, 0.00001}, the exponential decay step for is selected from {1000, 750, 500) and the exponential decay rate for is selected from {0.95, 0.9, 0.85, 0.8). The actor 2 and critic Q, the GIL (i.e., Alg. 1) are trained using deep deterministic policy gradient (DDPG)[aaa) Lillicrap al. (0155) where the discounting factor ? 0.99. We choose the behavioural policy in GIL as
(
with probability pl: with probability P2:
((s)
missing indicator m
(42)
random action
with probability p3-
The learning rates of the actor as and the critic ag are selected by performing grid search from {0.0005, 0.0001, 0.00005, 0.00001} and {0.001, 0.0005, 0.0001) } respectively. To train the imputation-fre free models using GIL-H/GIL-D, we follow the same grid search for a along with its decay steps and decay rates. From the implementation perspective, in the inputs x, with a placeholder value before feeding them into the model. This can avoid value errors thrown by Tensorflow if the input vectors contain NaN's. However, be used to update the parameters. For the state-of-the-arr baselines - GAIN MIWAE GPVAE and BRITS we use the implementations published on the Github by the authors. Adam optimizer is used to train all the prediction models for baselines, or to compute and GIL-D. All the models are trained using a batch size of 128. The details of selecting the other hyper-parameters of each case study can be found in the corresponding sub-section below.
replace the missing entries,
these values will not
JSGD for GIL,, GIL-H
D.1 MIMIC-III
2hr vital/lab observation
4hr early prediction window
records
Temporal Sequence Inputs
Predict
Septic Shock Not
Figure 4: Graphical depiction of the 2-hour observation window and 4-hour early prediction window (EPW) considered in this case study.
Dataset Formulation The MIMIC-II contains EHRS obtained from roughly 40,000 patients who stayed in the ICUs in the Beth Israel Deaconess Medical Center between 2001 and 2012 (Johnson al., 2016) and septic shock is severe type of sepsis that results in above 40% mortality rate. Figure Tillustrates the 2-hour observation window, 4-hour early prediction window (EPW) and the relation between inputs and targets for predictions. Note that each work related to septic shock predictions adopts a slightly different strategy in terms of selecting lab/vital attributes and the lengths of the observation and EPW (Mo et al.) 2018 Darwiche & Mukherjee 2018 Yee et al., 2019) Shectrit et al. 2017 Liu et al.) 2019 Khoshnevisan al.) Fleuren al. 2020). In this work, the 4-hour EPW allows including sufficient number of subjects which can ensure statistical significance of the results, while the smaller observation window keeps the task challenging. To formulate the training and testing dataset, we first selected 14 commonly used attributes for predictions as suggested in previous works (Sheetrit al. 2017 Fleuren al. 2020 Khoshnevisan al. 2020), which constituted by vital signs including temperature, respiratory rate, heart rate, systolic blood pressure,
Data acquired from httph:httpttmmmii
Access this dataset follows the MIT-CI.P license.
19

---

Published as a conference paper at ICLR 2022
mean arterial pressure, peripheral oxygen saturation (or SpO2), fraction of inspired oxygen (or FIO2), and lab tests including white blood cell count, serum lactate level, platelet count, creatinine, bilirubin, bandemia of white blood cells, blood urea nitrogen. To form the septic shock cohort, we first identify the patients who were diagnosed with sepsis by indexing with the International Classification of Diseases 9 (ICD-9) code 995.91 and 995.92. Then the patients with septic shock history are selected using ICD-9 code 785.52 with their septic shock onset time determined following the third international consensus for sepsis and septic shock (Sepsis-3) standard (Singer et al. 2016). Finally, we remove the patients whose septic shock onset time was less than 6 hours after admission to the ICUs since the data not enough to be formulated as sequences pertaining to the 2-hour observation and 4-hour prediction window. Consequently, a total of 1,083 septic shock patients are identified. To form the non-shock (or control) cohort, first 1,083 patients are randomly sampled from all admissions who have at least 2 hours of records excluding the ones who are in the septic shock cohort, then a 2-hour time frame randomly selected to be used as the observation window. All patients selected following the above procedure split into 8:2 to formulate the training and testing datasets, which are referred to as the varied-length (Var-I.) sequences in Section 4.2 In the test set, the number of subjects associated with positive and negative labels, respectively, are selected to be equivalent.
Histogram showing various sequence lengths
2500
2000
1500
1000
500
7.5
12.5 Sequence Length
10.0
15.0
17.5
20.0
Figure 5: Histogram of the sequences lengths of the MIMIC-III dataset (with maximum length truncated at 20). It can be observed that most of the sequences have length $ 8.
Formulation of Fix-l. Sequences To formulate the fixed-length (Fix-l.) sequences, we follow this protocol sequence length is less than 8, we pad it with the latest recording available until the length reaches 8, otherwise we truncate the length to 8 by discarding the data from that point on-wards. We specifically choose this threshold because most of the sequences have length S 8, as shown in Figure 5
Type of Missingness MAR and MNAR. Specifically, recordings from human mistakes malfunctioned equipment manifest as MCAR, the dependency across vital signs and lab results give rise to MAR (e.g., specific lab tests are ordered only if abnormalities found the related vital readings or other lab results), and the mismatch among the sampling frequency of some periodically recorded attributes such as temperature (obtained hourly) and blood test (conducted daily) can be considered as MNAR (Little
The missing data in this dataset can be considered as a mixture of MCAR,
Rubin 2019 Mamandipoor et al. 2019).
Training Details To train the prediction models for both GIL and baselines, we use maximum training steps of 2,000 and 4,000 for varied-length and fixed-length sequences respectively. For imputation of the missing values, we train the imputation method MIWAE using the learning rates (0.001, 0.00013 with other hyper-parameters provided by the authors in the code. To train GAIN, we use the default learning rates provided by the authors to train the generator and discriminator. GAIN contains one hyper-parament parameter that balances between the two losses LG and L defined in Yoon et al. (2018) which is selected from {0.1, 1, 10, 100}. To train GP-VAE on the Fix-l. case, the set of hyper- parameters we use contain latent dim = {6, 12, 35), encoder size = 256, 256, decoder sizeer__seeeee 256, 256, 256, window _izees {3. 4, 6), beta (0.2, 0.5, 0.8), sigma 1.005, length_scale
sizze
20

---

Published as a conference paper at ICLR 2022
{2, 4, 7). To train BRITS used the recommended impute weight label. weight 1.0. The term D(S", included in the reward function for GIL-D is defined as in the follows. Assume that in each training epoch b/2 imputs with label 0 and 2/2 imputs with label are sampled from the dataset, where b is the batch size. We use F+ (Ct,.... ..... Sh/2) E R5/2xe to denote the features associated with positive labels (i.e., 1) in the current batch and F- (ST :.... (6/2)))6/))) E 2 where is the dimension of each individual feature Ç. Then we define D as
are,
MSE(F* [0: b/ 4), F [0: b/4) + MSE(SSFFFFF (b/4: b/2), F 1b/4 6/2)) MSSE [0: b/4] F+ [6/4: b/2) MSE(F [0: b/4], F [0/4 b/2)),
(43)
where the slicing index follows the syntax from Python, e.g. [= 6/4] corresponds to the 0-th to (b/4 - 1)-th rows of F+.
D.2
OPHTHALMIC DATA
Figure 6: Example of a 3D OCT volume scan and 2D OCT image slice from the volume scan.
Dataset Introduction This dataset is originally constituted by OCT images, CFP images and patient EHRs including demographic information and test results related to diabetes. total of 1,148 subjects are included in this dataset. OCT refers to the two- or three-dimensional images capturing the retinal architectures of the eyes that scanned by low-coherence lights. The 3D OCT image usually called the volume scan and is constituted by 61 or 101 2D image slices depending on the spec of the scanner. Examples of 3D and 2D OCT scans are shown in Figure In this experiment the volume scans we use contain the 61 slices, while we only consider the center slice (31th) 2D image. The CFP images are the color images captured by fundus camera showing the condition of the interior surface of the eye, where an example is shown in Figure 7
Dataset Formulation To formulate the data into tabular inputs, the 2D OCT and CFP retinal images are first fed in two inception-v2 v3 (Szegedy al. 2016) CNNs, pre-trained with similar images provided by (Kermany et al. 2018) and (Kaggle 2015) respectively, and the image feature vectors with dimension of 2,048 each are output by the global average pooling layer. Then the patient EHR information include age (integer), sex (boolean), length of diabetic history (integer), A1C result (integer), which measures blood sugar levels, and if insulin has been used (boolean) constitute a 5-dimensional vector that is concatenated to the end of the OCT and CFP feature vectors. We split all the subjects into a training cohort and testing cohort following ratio of 9:1. the test set, the number of subjects associated with positive and negative labels, respectively, are selected to be equivalent. The raw images from the training cohort are augmented through cropping and rotation augmented. Figure 7
before feeding into inception-v3. All the data from the testing cohort illustrates how the training and testing inputs are formulated, as well as the input-output relation of the prediction models.
To train the prediction models for both GIL and baselines, mum training steps of 2,000. For imputation of the missing values, MIWAE trained using the learning rates {0.001, 0.00011 with other hyper-parameters provided by the authors in the
Training Details
maxi-
21

---

Published as a conference paper at ICLR 2022
OCT
Inception-v3
OCT Feature Vector
2,040 dimension
+
CFP
Inception-v3
Feature Vector
Predict
2,040 dimension
No DR
Age
Sex
Diabetic Length
A1C Result
Insulin Use
patient
int
bool
int
int
bool
Figure 7: Diagram showing the pipline of this experiment.
code. GAIN is trained using hyper-paramnen parameters selected from {0.1, 1, 10, 1000. GP-VAE is trained by applying the arguments latent dim 256, encoder _size__________ 256, 256, 256, window_st siize = 3. beta 0.8, sigma = 1 to its code published in Github (https: /ttp:s://gtthie
256, 256, decoder_ste _issee
www..aatcchha/aa--oa--.......... (AEE) MF, EM and kNN are not included this experiment due to the high dimensionality of the inputs which makes these algorithms very computa- tional inefficient as imputation results cannot be produced within days.
D.3 MNIST
Training Details To train the prediction models for both GIL and baselines, mum training steps of 1,0000. For imputation of the missing values, MIWAE is trained us- ing the learning rates {0.001, 0.0001} with other the code. GAIN is trained using hyper-parameters selected from {0.1, 1, 10, 100). GP-VAE is trained by applying the arguments (atent_ding 256, encoder_ _sizee_eee__ 256, 256, decoder_sis 256, 256, 256, window _ize (https: https:///ttthi
maxi-
provided by the authors in
size = 3, beta 0.8, sigma 1 to its code published in Github www.ooo/aaaeae com/ratscchaa/GG GP-aaa//G--V.. MF, EM and kNN are not included in this experiment due to the high dimensionality of the inputs which makes these algorithms very computa- tional inefficient as imputation results cannot be produced within days.
We also trained GIL-D on the MCAR version of MNIST dataset and the performance are shown Table below.
Additional Results
Table 5: Accuracy for GIL-D on the MCAR version of MNIST dataset. Standard deviations are in subscripts (x 10 -3).
Missing Rate
50%
70%
90%
Acc.
96.29..09
93.35o.9
78.473.9
D.4 PHYSIONET
Table 6: Accuracy, AUC and average precision (AP) obtained from the Physionet dataset
GIL
GIL-H
GAIN
GP-VAE
MIWAE
BRITS
Mean
Zero
Acc.
87.45
87.38
86.23
86.85
87.12
86.20
86.87
87.15
AUC
82.20
82.00
78.02
82.21
83.47
81.30
82.57
81.14
AP
49,19
48.64
39,89
47.05
49.31
43.57
48.30
47.37
22

---

Published as a conference paper at ICLR 2022
Other than MIMIC-III, we also tested smaller scaled ICU time-series from 2012 Physionet challenge Silva et al. (2012) which contain data obtained from 12,000 patients. We use the data pre-processed and open-sourced by Fortuin et al. 20200) 10 For each patient the values of 35 different attributes (e.g., blood pressure, temperature) are recorded over a 48-hour window. As a result, the data for each patient be formulated into matrix in IR 1838 i.e., the sequence length across patients are the same. This dataset has an overall 78.5% missing rate and a binary label is assigned to each patient where 87% of them are O's and 13% are I's. Therefore we include average precision (AP) which is calculated from the precision-recall curve as an additional metric to evaluate the performance toward imbalanced labels. For this dataset, we consider a 1024-unit LSTM layer for encoding and dense output layer for inference. It can be observed from Table 6 that although our method achieves highest accuracy and 2-nd highest AP, while most methods perform very close to mean- and zero- imputation. This could be caused by the its simple structure, as all the sequences share the same time horizon, which significantly reduces the difficulties for the classification task. Moreover, the highly imbalanced labels and the small population result in the performance to be hardly distinguishable across different methods. And this is the reason that we focus on the MIMIC-III dataset to study the strengths and shortcomings of the methods.
23

---

Published as a conference paper at ICLR 2022
E PROOF FOR PROPOSITION 1
Here we prove a generalized version of Proposition as stated in the following proposition, which shows that the gradient of the loss function w.r.t. any layer in the MLP can written in the outer product format.
Proposition 3 Given an MLP as in ( and smooth loss function E y), the gradients for any hidden layer W. i k1 can be represented as
OE
A.9
(44)
OW,
where 9i-1 Φi-1 (W-i(W.-1Q4-2 is the output from the i - 1-th layer with Qo and A @ Nout (W (Wout Q2t) and @ denotes the element-wise
WT [-1 Ait1 & (WW(qq:) with Dout (Hadamard) product.
0E
Now we prove Proposition
We consider the MLP characterized by
ý = @out W Ox W 62(2201 (Wix))))))
(45)
where x E IR represents the input to the model, W, is the weight matrix for the i-th hidden layer, and Φ; is the activation function of the i-th hidden layer.
We start with deriving the derivatives for the output layer
0E
OE
oy
(46)
OW out
ey
OW out
OE
① Dout (W (Woud ql.
(47)
08
-out out 9
(48)
where q is the output from the k-th hidden layer. Now we show the derivative of w.r.t. the k-th hidden layer
DE
OE 09 aqu
(49)
OW
OY dq.
aW
DE
0qk
W out 05
dourl W qz)l
(50)
OW
WT T
(51)
out
out OW
out out) @ d% (W qi_
(52)
(53)
We now prove Proposition the j-th hidden layer is
by induction. First assume that for j E [2, k). the derivative of E w.r.t.
aE aw, [ H o 9
(54)
A, -T
(55)
Moreover, let us assume that
0E WT T
aE
(56)
0q, itl 0q
-WT
/+1 A 1+1-
(57)
24

---

Published as a conference paper at ICLR 2022
Now need to show that for
it holds that
OF
[W] A,) o (W
.
(58)
Ow, j-1
OE
W
о (W
(59)
dq-1 dqi
W A, A,
(60)
We first prove that 60) holds, i.e..
0E
0E dq:
(61)
dqj-1
oq, òqp-1
oqj
=W j+1 A j+1 رq,-1
(62)
=W/ [ o (
(63)
W
(64)
which is equivalent to (60).
To prove that (58) holds, we start with
OE
DE 0q. 0qp-1
(65)
OW j-1
dq Oq: OW j-1
0q, Öq,-1
-W
j+1 1 j+1 0q oW J-1
(66)
8qi-1 WW 5+1 +1 C (
(67)
OW,- ;-1
-WJ A, 0q--1
(68)
dw, ,-1
=[W] A,) oW 47-21
(69)
which is equivalent to (58).
25

---

Published as a conference paper at ICLR 2022
F ADDITIONAL RELATED WORKS
RL for Optimization. This work is also related to approaches that use RL. to solve optimization problems, or improve existing solvers. For example. [oyan & Moore (2000) proposes framework to improve the efficiency of local search algorithms by predicting the search outcomes and biasing toward directions that provide higher returns, which is approached by estimating value functions of the MDP corresponding to the optimization problem. Recently. Li & Malik (2016) trains RL agents to perform gradient descent steps over linear regression and neural network models. Similarly, Wei et al. (2020) introduces policy network to automatically determine parameters for the plug-and-pin frameworks (Sreehari al. (017) which can solve inverse imaging problems. Note that these methods require complete inputs and cannot be adapted to the problem consider.
26

---

Published as a conference paper at ICLR 2022
G ADDITIONAL ABLATION STUDY
In this section, we compare GIL against an ablation baseline that applies the importance directly to the inputs x, instead of the gradient space. Specifically, suppose differential parametric function (x), parameterized by v. is multiplied with x clement-wise. Then, as opposed to 44, the gradient updates w.r.t. the encoding layers can formulated as
W enc
f W enc aA (x C hx)(x)))
(70)
Consequently, all model elements (i.e., hig W enc and W inn) could be trained by gradient descent, without introducing RL policies.
Table 7: Performance of the ablation model over the ophthalmic and MNIST datasets Ophthalmic
MNIST
25% M.R.
35% M.R.
70% M.R.
90% M.R.
Acc.
84.50
80.99
93.22
78.31.8e 18831.88-03
AUC
89.781.68
87.262.53
In Table 7 shows the performance of the model described above over the ophthalmic dataset, as well as MNIST digits with 70% and 90% missing rate (M.R.). Specifically, hyg (x) is captured by neural network with the same architecture the policy To in GIL. The training and testing are conducted following the same convention (e.g., random seeds that determine the missing entries and
hyper-parameter search) as introduced in Sections 4.3 4.4 and Appendix D.2 D.3 can be observed that the ablation baseline attained similar performance as to GIL-H and zero imputation. The reason that leads these results would be that the gradients for training (x) still follow the outer product format A
which are left be accounted for. Specifically, our method is built on top of the idea, and the experiments in Section 4 alsso support that if part of the inputs are missing they may not provide sufficient information train the prediction models, given the outer product format of the gradients. Similarly, the gradients for (x) also follow this format and hy (x) may not be properly trained directly on incomplete inputs x, which could in general limit the overall performance of the prediction model.
Additional rationales for using RL to obtain gradient importance in GIL. The ablation base- line above is closely related visual attention (VA) models as discussed in Appendix [ and could be seen as preliminary version of them where (x) is used to re-weight elements in the inputs VA techniques benefit from the fact that features learned by CNNs are spatially correlated the inputs, so the attentions could be directly applied to learned features. To achieve this, VA formulates the objective as an ELBO which shown equivalent to the objective of a basic policy gradient RL algorithm, REINFORCE (Mnih et al. 2014). However, it was not clear how such VA methods could be adapted to the problem we consider, as the features learned by MIPs or LSTMs are not spatially correlated with inputs.
More on importance versus attentions. The other baseline, BRITS (Cao et al., 2018). uses bidirectional RNN process time-series, with attentions applied to the hidden states of RNNs, followed by optimizing over imputation and prediction objectives jointly during training. BRITS is considered as the state-of-the-art method that could predict with incomplete time-series inputs end-to-end, However, we showed that BRITS is outperformed by our method on the MIMIC dataset; see Table D The reason would be that BRITS requires masking off part of the observed inputs to constitute the imputation objective: thus the information provided for model training is even more limited given the intrinsic 70% M.R. of MIMIC
27


Reviews:
[{"review_id": "qjBnMs02tu4", "reviewer": "Reviewer_azSY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros\n- Proposes novel method\n- Outperforms baselines\n\nQuestions\n- You say you use batches of size 128, but the description of the algorithm describes a scenario of batch size 1 if I understand correctly. How do you handle large batch sizes?\n    - There also seems to be a slight mistake in the algorithm with the index i in $x_i$\n- What are the compute requirements compared to the baselines?\n\nCons\n- Complicated method, compared to mean imputation for example.\nThe method requires two different loops and naturally one wonders whether there wouldn't be a method that does only require one objective.\n- I think the explanation of the method is a little lengthy and overly complicated, as one might assume knowledge of standard back propagation in MLPs.\n", "summary_of_the_paper": "The paper proposes a method to train on data with missing features. It does so with a single model that can handle missing features, not with extra imputation methods. This is done by weighting the gradient update of the first weight matrix of the neural network with a vector a that is produced by an RL agent. This agent's reward in turn is the performance of the model after the update.", "summary_of_the_review": "While I believe the objective of the paper is worth research: a neural network that works well with data missingness out of the box, I believe the method is a little complicated to be a practical tool and for other research to build upon it. Additionally, I want to note that, while I understand the algorithm well, I never worked with data missingness before.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "mQHaeZUBnKO", "reviewer": "Reviewer_4Cb5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper targeted at an important problem in machine learning and proposed a novel idea to address it. The paper is well structured and easy to follow. The experimental results on the MIMIC-III datasets are encouraging.\nMy main concerns about this work are as follows:\n1.\tImputation-based methods will not work well on data with informative missing, as in MIMIC-III, because patients in different conditions or different metrics are measured in different time scales. Thus, the missing patterns may contain some hidden information about the patients, and imputation may lead to a lost of the information. This can also be observed in Table 1, where most of the compared methods are worse than GIL-H (a heuristic that simply discards the gradients produced by the subset of input entries that are missing). Thus, there might be concerns that the proposed method will work well in time series data with informative missing but may not work equally well in other missing patterns, e.g., only random missing. For example, the results on the MNIST dataset seems to confirm that the proposed method cannot significantly outperform the baselines even with simple zero imputation.\n2.\tThere might be an efficiency issue when using RL to weigh the importance of gradients. The training of the RL policies is generally time-consuming, so it will be also important to analyze the efficiency of the proposed method.", "summary_of_the_paper": "This paper targeted at the missing data issue in time series data and proposed a imputation-free method to handle missing data. More specifically, the authors proposed a gradient importance learning method named GIL to weigh the gradients for different parameters using reinforcement learning. Experiments on one tabular dataset and two image datasets demonstrated the effectiveness of the proposed method.", "summary_of_the_review": "Handling missing data is important in real machine learning problems and it is good to see that imputation-free method can also work well under this issue. Since there are some unanswered questions in the paper, I would like to give a slightly positive rating.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jGd5rfgjIB2", "reviewer": "Reviewer_he3p", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The general problem of doing predictions based on incomplete data is important. The ideas are simple and clearly presented. The experiments are comprehensive both in terms of comparing with new and classical methods and in terms of applying to real datasets to show applicability and toy datasets (MNIST) to understand qualitative performance. The limitations (e.g. that this is only applicable to MLPs and LSTMs) are also clearly stated.", "summary_of_the_paper": "This paper proposes a method to do MLP / LSTM-based inferences/predictions based on incomplete data. While typical methods would impute and then predict, the proposed method predicts directly based on incomplete data. The method learns an “importance” matrix which is multiplied with the first weight matrix of the neural network. The learning is done using reinforcement learning, with the negative prediction error being the reward. This method is compared with new and classical prediction methods for incomplete data on a range of datasets and is shown to be the best most of the time.", "summary_of_the_review": "N/A", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "PptLLLazU0X", "reviewer": "Reviewer_3AZM", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I have several concerns as follows:\n\n1. The motivation of the paper is not so clear. Specifically, I'm not fully convinced by the arguments between Eq. (2) and Eq. (3). I understand how the missing values can affect the learning of the parameters in the encoder according to Eq. (2). However, adding an element-wise multiplicative parameter matrix A seems quite arbitrary. \n\n1.1 Why choose this specific form?\n\n1.2 Note that each sample may have a different mask sampled randomly. Can a shared matrix across all data samples solve the problems caused by the missing values with diverse patterns? This is counter-intuitive.\n\n1.3. Why it must be on the gradient level, not the parameter level? If the matrix A is prefixed, using the gradient in Eq. (3) is equivalent to reparameterizing the encoder layer by multiplicating the same A, right? Then why not directly optimize all parameters using SGD without adding an RL procedure. I know the final parameters are different by using different optimization strategies. But why this gradient level modification with RL is preferable for this task?\n\n2. I note the authors discussed the limitation of the proposed models, mainly about CNN. I'm also wondering the same questions about other popular techniques used in deep learning, including but not limited to batch normalization and residual structures.  \n\n3. How does the additional RL training procedure affects the efficiency of the training? It is necessary to compare the time and memory complexities and training stabilities.\n\n4. What is learned in matrix A? Intuitively, different mask distributions will affect matrix A but how? Any intuition behind this?\n\n5. For the experiments, I didn't see how the hyperparameters including the model architectures are selected. For instance, does the MLP structures used in this paper exactly the same as the ones used in SOTA methods? How do the additional hyperparameters set?\n\n6. A very basic \"imputation-free\" baseline is to directly train a model to predict the label given an incomplete input. For instance, on the MNIST dataset, I wonder about the results of widely used CNN models  (e.g. Alexnet, Resnet-26) on the same data. I notice the limitation of the paper on CNNs while it is not the reason to ignore such a natural baseline.\n", "summary_of_the_paper": "The paper proposes gradient importance learning for predicting labels on incomplete data. The problem is: during both training and testing, certain dimensions of the samples are masked randomly and the masks (i.e. the \"missing indicators\") are known for the algorithm. Among existing methods in this problem, the paper claims that it is the first \"imputation-free\" way to solve this problem. The paper focuses on MLP and LSTM and adds an (element-wise) multiplicative parameter matrix on the gradient of the first layer (the one that is mostly closed to the data). The matrix is trained using reinforcement learning. The authors also connect its proposal to visual attention. Experiments are conducted on tabular, time series, and a simple image dataset.\n", "summary_of_the_review": "I raise several questions about the motivation, clarity, and experiments of the paper. I believe the current version should be significantly improved to reach the acceptance of ICLR.\n\n**post discussion with the authors**\n\nI thank the authors for this productive reviewing process. I increased my score because my major concern has been addressed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposed an imputation free method to handle missing data by learning an input encoding matrix using RL with the prediction error as reward/penalty signal. Reviewers appreciate the interesting setup where RL is used to deal with missing data, and the method being imputation free. Three out of four reviewers (reviewer he3p, azSY, and 4Cb5) have raised concerns on the complexity of the proposed method, but it seems like all the reviewers see the strength of the work outweigh the weakness.
Decision: Accept (Poster)