Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a method to do MLP / LSTM-based inferences/predictions based on incomplete data. While typical methods would impute and then predict, the proposed method predicts directly based on incomplete data. The method learns an “importance” matrix which is multiplied with the first weight matrix of the neural network. The learning is done using reinforcement learning, with the negative prediction error being the reward. This method is compared with new and classical prediction methods for incomplete data on a range of datasets and is shown to be the best most of the time.
main_review: The general problem of doing predictions based on incomplete data is important. The ideas are simple and clearly presented. The experiments are comprehensive both in terms of comparing with new and classical methods and in terms of applying to real datasets to show applicability and toy datasets (MNIST) to understand qualitative performance. The limitations (e.g. that this is only applicable to MLPs and LSTMs) are also clearly stated.
summary_of_the_review: N/A
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a method to train on data with missing features. It does so with a single model that can handle missing features, not with extra imputation methods. This is done by weighting the gradient update of the first weight matrix of the neural network with a vector a that is produced by an RL agent. This agent's reward in turn is the performance of the model after the update.
main_review: Pros
- Proposes novel method
- Outperforms baselines

Questions
- You say you use batches of size 128, but the description of the algorithm describes a scenario of batch size 1 if I understand correctly. How do you handle large batch sizes?
    - There also seems to be a slight mistake in the algorithm with the index i in $x_i$
- What are the compute requirements compared to the baselines?

Cons
- Complicated method, compared to mean imputation for example.
The method requires two different loops and naturally one wonders whether there wouldn't be a method that does only require one objective.
- I think the explanation of the method is a little lengthy and overly complicated, as one might assume knowledge of standard back propagation in MLPs.

summary_of_the_review: While I believe the objective of the paper is worth research: a neural network that works well with data missingness out of the box, I believe the method is a little complicated to be a practical tool and for other research to build upon it. Additionally, I want to note that, while I understand the algorithm well, I never worked with data missingness before.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper targeted at the missing data issue in time series data and proposed a imputation-free method to handle missing data. More specifically, the authors proposed a gradient importance learning method named GIL to weigh the gradients for different parameters using reinforcement learning. Experiments on one tabular dataset and two image datasets demonstrated the effectiveness of the proposed method.
main_review: This paper targeted at an important problem in machine learning and proposed a novel idea to address it. The paper is well structured and easy to follow. The experimental results on the MIMIC-III datasets are encouraging.
My main concerns about this work are as follows:
1.	Imputation-based methods will not work well on data with informative missing, as in MIMIC-III, because patients in different conditions or different metrics are measured in different time scales. Thus, the missing patterns may contain some hidden information about the patients, and imputation may lead to a lost of the information. This can also be observed in Table 1, where most of the compared methods are worse than GIL-H (a heuristic that simply discards the gradients produced by the subset of input entries that are missing). Thus, there might be concerns that the proposed method will work well in time series data with informative missing but may not work equally well in other missing patterns, e.g., only random missing. For example, the results on the MNIST dataset seems to confirm that the proposed method cannot significantly outperform the baselines even with simple zero imputation.
2.	There might be an efficiency issue when using RL to weigh the importance of gradients. The training of the RL policies is generally time-consuming, so it will be also important to analyze the efficiency of the proposed method.
summary_of_the_review: Handling missing data is important in real machine learning problems and it is good to see that imputation-free method can also work well under this issue. Since there are some unanswered questions in the paper, I would like to give a slightly positive rating.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes gradient importance learning for predicting labels on incomplete data. The problem is: during both training and testing, certain dimensions of the samples are masked randomly and the masks (i.e. the "missing indicators") are known for the algorithm. Among existing methods in this problem, the paper claims that it is the first "imputation-free" way to solve this problem. The paper focuses on MLP and LSTM and adds an (element-wise) multiplicative parameter matrix on the gradient of the first layer (the one that is mostly closed to the data). The matrix is trained using reinforcement learning. The authors also connect its proposal to visual attention. Experiments are conducted on tabular, time series, and a simple image dataset.

main_review: I have several concerns as follows:

1. The motivation of the paper is not so clear. Specifically, I'm not fully convinced by the arguments between Eq. (2) and Eq. (3). I understand how the missing values can affect the learning of the parameters in the encoder according to Eq. (2). However, adding an element-wise multiplicative parameter matrix A seems quite arbitrary. 

1.1 Why choose this specific form?

1.2 Note that each sample may have a different mask sampled randomly. Can a shared matrix across all data samples solve the problems caused by the missing values with diverse patterns? This is counter-intuitive.

1.3. Why it must be on the gradient level, not the parameter level? If the matrix A is prefixed, using the gradient in Eq. (3) is equivalent to reparameterizing the encoder layer by multiplicating the same A, right? Then why not directly optimize all parameters using SGD without adding an RL procedure. I know the final parameters are different by using different optimization strategies. But why this gradient level modification with RL is preferable for this task?

2. I note the authors discussed the limitation of the proposed models, mainly about CNN. I'm also wondering the same questions about other popular techniques used in deep learning, including but not limited to batch normalization and residual structures.  

3. How does the additional RL training procedure affects the efficiency of the training? It is necessary to compare the time and memory complexities and training stabilities.

4. What is learned in matrix A? Intuitively, different mask distributions will affect matrix A but how? Any intuition behind this?

5. For the experiments, I didn't see how the hyperparameters including the model architectures are selected. For instance, does the MLP structures used in this paper exactly the same as the ones used in SOTA methods? How do the additional hyperparameters set?

6. A very basic "imputation-free" baseline is to directly train a model to predict the label given an incomplete input. For instance, on the MNIST dataset, I wonder about the results of widely used CNN models  (e.g. Alexnet, Resnet-26) on the same data. I notice the limitation of the paper on CNNs while it is not the reason to ignore such a natural baseline.

summary_of_the_review: I raise several questions about the motivation, clarity, and experiments of the paper. I believe the current version should be significantly improved to reach the acceptance of ICLR.

**post discussion with the authors**

I thank the authors for this productive reviewing process. I increased my score because my major concern has been addressed.
review_title: null

Meta-review: The paper proposed an imputation free method to handle missing data by learning an input encoding matrix using RL with the prediction error as reward/penalty signal. Reviewers appreciate the interesting setup where RL is used to deal with missing data, and the method being imputation free. Three out of four reviewers (reviewer he3p, azSY, and 4Cb5) have raised concerns on the complexity of the proposed method, but it seems like all the reviewers see the strength of the work outweigh the weakness.
Decision: Accept (Poster)