Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This submission proposes a new imputation algorithm called Markov-Blanket Miss-Forest (MBMF), which involves two phases: Markov Blanket-based Feature Selection (MBFS), and MissForest (MF) imputation (Stekhoven & Buhlmann, 2012). 

For each partially observed variable V_i, MBFS finds its Markov Blanket within some m-graph which is referred to as the graphical expression of missingness proposed by Mohan et al. (2013).  Then, elements of the Markov Blanket are considered as explanatory features of V_i in the Random Forest regression model used in MF.  

Some experiments are conducted to assess the usefulness of MBMF, in comparison with the original MF, and simpler imputation algorithms (Mean, Mode and KNN algorithms).
strength_and_weaknesses: Strengths
- Markov Blanket-based Feature Selection (MBFS) and imputation is interesting
- Experimental results show that MBMF can provide promising imputation accuracy  

Weaknesses: I think the paper is interesting and I have a few questions. 
- Assumption: Is the assumption 1 "R_i could only be a leaf node in m-graph" made to make the problem of finding Markov Blankets easier? Or is it a theoretical guarantee?  
- M-graphs: For each data set, can MBFS recover the m-graph which informs the missingness mechanisms? For each given incomplete data set, can the output of MBFS suggest which underlying assumption of missingness was used to create that incomplete data set?   
- Scalability: Experiments are done on data sets with small and moderate sizes (at most 30 variables and at most 8124). Can you comment on the scalability of MBFS?
clarity,_quality,_novelty_and_reproducibility: I think the submission is well written.
summary_of_the_review: I think "MBFS" & "imputation algorithm" is an interesting imputation approach and can be implemented with different imputation algorithms. I gave a few questions in "Strength And Weaknesses".
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose MBMF, essentially a feature selection procedure for a second step imputation method. With MBMF the authors aim to recover the m-graph that governs a system's missingness (in the tabular domain). Using the m-graph, the authors recover a variable of interest's markov blanket which are used as features for imputation. The second step uses the selected features to impute a missing variable. 

Their experiments show that MBMF combined with MissForest outperforms the most adopted imputation strategies. 
strength_and_weaknesses: STRENGTHS

* Clearly, causality and structure are important elements in imputation, particularly in the tabular domain. Imputation is an important research area that should receive more attention than it currently does.

* I found the paper quite well written. Also, I find it interesting that the authors base their method on a tried and tested algorithm from 2003. 

WEAKNESSES

I think the paper was interesting and have only a few questions I would like the authors' opinion on.

* What exactly is the motivation for using the Markov blanket instead of the parents of X_i? Kyono et al. (2020) seem to be doing well with the covariate's parents? (Note that an m-graph is _causal_, saying it is recovered allows assuming that the parents of a variable are _actually_ the causes). 

* Would it be useful to try MBMF with other algorithms (beyond MissForest)? If so, why haven't you done this and could you? If not, why not?

* It seems GAIN is not performing very well, despite it being considered the sota, could you elaborate or reason why this is the case?


clarity,_quality,_novelty_and_reproducibility: The paper was quite well written. However, I did not find any instructions or leads to reproduce the authors' results. 
summary_of_the_review: I think the paper is well written, interesting, and touches on some interesting ideas to marry causal discovery and imputation. I was left with a few questions I hope the authors can answer in their rebuttal.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a method for feature selection to be used prior to estimating an imputation model. The method is based on finding the Markov blanket for each partially observed variable. In synthetic and semi-synthetic experiments, the methods slightly outperforms an imputation model trained without feature selection with a lower total run time (feature selection + model estimation).
strength_and_weaknesses: See below.
clarity,_quality,_novelty_and_reproducibility: See below.
summary_of_the_review: Overall, I found this paper exceptionally clear and well-written. The method is interesting and the experiments are reasonable and well-executed. My only overall concern is regards motivation. By and large, ML methods favor general regularization over using feature selection as a preprocessing step, so I was a bit unconvinced by the initial motivation that feature selection is necessary to improve imputation performance. I think this intuition is born out by the experiments which seem to demonstrate only a small improvement over the base method applied without feature selection. I think the intro would be improved with a bit more justification for the approach and, specifically, why the authors believe feature selection is the right approach here when regularization has worked so well in other settings. With that said, I still think the approach is reasonable and merits publication.

Minor comments:
1. Overall, I thought the background provided in the intro was excellent. My only quibble is that, in my experience, the most commonly used approach to missing data in statistical settings is multiple imputation and it might be worth mentioning this approach in the intro. Note that improved imputation models mean improve MI.

2. Intro: Modem --> Mode

3. Figures 1 and 2: I think these figures could be cleaned up by organizing them in a 3x3 grid. If you are concerned about space, I recommend sharing x and y axes within the grid.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a feature selection method based on markov blanket under missing data regime and apply missforest after feature selection. The main contribution is the feature selection algorithm.
strength_and_weaknesses: Strength:
1. The authors propose a markov blanked based feature selection algorithm for missing data. The selected features are then used in MissForest. 

Weakness:
1. The authors should include more recent works to claim that their method is SOTA. MissForest is kind of old (10 years old). More recent SOTA should be included. For example, 
MIRACLE (Kyono et al., 2021)
Sinkhorn (Muzellec et al., 2020)
SoftImpute (Hastie et al., 2015)
HyperImpute (Jarrettet al., 2022)
2. MimMB is similar to MBMF. Empirical comparison should also be included.
3. Unlike MimMB, the authors claim that improves computational efficiency. A comparison of execution time is more persuasive. If it's trivial, the authors can correct me without making this new result of computational efficiency. 
clarity,_quality,_novelty_and_reproducibility: The paper is clear. Markov Blanket + MissForest is novel. 
summary_of_the_review: I think this paper is at the borderline. The authors do not compare to many new SOTA methods, which I believe have better results than MissForest. But it can be the reason that MissForest is not that good so MBMF is not as good as those SOTA methods.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Simple paper about Markov blanket feature selection for imputation with the missforest idea. Empirical results are satisfactory, showing superiority at times to some SOTA. The topic might be seen as niche, but reviewers appreciated the work on this topic, indicating that it requires more attention that it gives.
justification_for_why_not_higher_score: Reviewers considered the work well done, but topic and scope might not attract all the general audience.
justification_for_why_not_lower_score: One could claim that the simplicity and scope are reasons for a lower score. However, all reviewers agreed on the validity of the topic and direction. The only really major concern was regarding expanding the comparison to more SOTA ideas, which the authors have put an effort to improve.
Decision: Accept: poster