Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content:  The paper proposes a neural network architecture to address the problem of estimating a sparse precision matrix from data (and therefore inferring conditional independence if the random variables are gaussian).

The authors base their algorithm in the semidefinite relaxation by Banerjee et al. They add a regularization terms and penalization parameters, which they learn using neural networks. They consider an alternating minimization implementation similar to ADMM and the neural networks are only used to find the regularization parameters.

In order to learn the parameters, the training optimizes the regularization parameters that maximize the recovery objective function (meaning how far is the estimated precision matrices from the true given precision matrices) and doesn’t consider the sparsity. 

Something that is not a priori obvious is the setting of using a family of precision matrices from a family of graphs and trying to learn an underlying precision matrix (by averaging them?). Further explanation of beginning of section 3 would be useful. 

Something else that is not clear to this reviewer is the motivation for the loss (9). If the objective is to find the parameters that maximize the recovery objective without taking the sparsity into consideration then why not choose them that way in (1), why there should be learning involved? And what is the learning exactly pursuing? Is it trying to learn a way to combine the information from the different samples consistently? [I acknowledge this is probably a naive question, but maybe addressing this in section 3.3 will help understanding].

I think the overall idea is interesting. Regularization parameters are usually problematic because it is not obvious how to choose them. Having an automatic, data-driven way to choose them is a useful algorithm design tool. The objective pursued in the choice of the loss function is a key concept of the paper and I believe it is not clearly explained. Explaining this point in a convincing way will improve the paper and my assessment from weak reject to strong accept. I suggest cutting the introduction to half and use that space to justify and explain sections 3 and 3.3 in depth.  

---
Edit: I thank the authors and reviewer 1 for their explanations. I changed my rating to accept. I think it would be useful for the readers to include some of these remarks in the paper.
review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: The authors propose a new method for graph recovery, which is a more data-driven approach by deep learning. It makes an original approach to this problem. In-depth theoretical results are provided in supplementary material. Good attention is also paid to hyper-parameter tuning.

However, some parts can be clarified and improved:

- In the introduction a number of phrases should be clarified: 
it is not entirely clear what the meaning is of the input and output of the problem, input covariance and output precision matrix. In the part of related works, it is difficult to understand in this stage why RNN and deep Q-learning is related to the scope of the paper.

- It is not clear whether section 2 contains new elements or whether the new contribution is entirely in section 3. 

- Please motivate the use of CNN in section 3.1. Why are convolutional layers important within this context?

- eq (6): the methodology of this formulation can be better positioned with respect to the existing literature. It appears to be based on principles of synchronization and consensus (in the term ||Z - Theta||_F). Additional explanation and references are needed at this point.

review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: This paper proposes an approach to data driven edge recovery for sparse gaussian mrfs. The authors propose an AM procedure for solving the l1 regularized maximum likelihood which can be unrolled and parameterized. This method is shown to converge faster at inference time than other methods and it is also far more effective in terms of training time compared to an existing data driven method. The authors provide a theoretical analysis which explains how the AM procedure should succeed and some insights on how it can potentially converge better using an adaptive model (motivating the learning part).

Experiments:
Overall the experiments demonstrate the method is superior. I have a few comments/concerns however:
-Scalability to larger graphs. The graphs used here a relatively small. I would like to see how well this scales to larger graphs at least in principal if not experimentally. Is there any issues that make this difficult? For example more iterations might be needed for convergence and this becomes problematic for learning. Can this already compete  with large scale methods like BigQUIC.
-Are the training graphs always the same distribution as the test graphs (e.g. in terms of the sparsity level)? It would be good to evaluate how well the model works when the training conditions differ to testing, since applying it to real data would require this gap.
-Closely related to the above if I have understood correctly all the experiments including the gene networks are on synthetic data, it would be good however to see if synthetic data can help generalize to real data.
- How many iterations are used to train the model? Is the number of iterations ever more at inference than training? It seems the NMSE is increasing after hitting a bottom I am wondering if that is related to mismatch in the number of iterations in test/train
- (minor) the authors compare wall clock time per iteration in Table 2 however their method converges much faster, it would be good to also see the overall clock time for each method after some reasonable stopping crieria, to show how big the overall gain is. 

Related Work:
The overview of sparse graph recovery for Gaussian random variables is good and concise. However I found the high level motivations given in Introduction/Sec 3 are similar (at times even the wording) to those of Belilovsky et al 2017 which introduced/motivate the data driven approach to this problem. Although this reference is used in the experimental section, it would be appropriate to clarify the difference/contribution compared to this work in the Intro, Sec3, and/or related work as a naive reading of the paper incorrectly suggests it is the first to consider a data driven approach to this problem.


Other comments:
- In equation (9) \Theta^{*(i)} should there be an (i) index there, I suspect this is a typo but  if it is not can the authors explain how the target differs for different (i)
- The data generating process described in Sec 5, how does it assure SPD, the described procedure (sampling off diagonal entries U(-1,1) then randomly setting zeros) does not appear to me to assure SPD without further constraints.
- I appreciated Appendix C10/C11 the overview of other attempts to parametrize the inference procedure

Overall I found this work relevant, the formulation well motivated, and potentially of high impact for the community working on inference of sparse conditional independence structure. I would give the score at the moment between weak accept and accept. There is a few points which I would like the authors to clarify or correct in their rebuttal and I would be happy to increase my score.

-----Post Rebuttal
The authors have addressed my primary concerns and revised the text, I am thus increasing my score. I recommend the authors also itemize the main changes in the text from the initial manuscript as they are not easy to find right now using the openreview revision comparison system (which seems to be broken). For example its not clear if the results shown within text are in the new manuscript.
review_title: Official Blind Review #1

Meta-review:  The paper proposes a neural network architecture to address the problem of estimating a sparse precision matrix from data, which can be used for inferring conditional independence if the random variables are gaussian. The authors propose an Alternating Minimisation procedure for solving the l1 regularized maximum likelihood which can be unrolled and parameterized. This method is shown to converge faster at inference time than other methods and it is also far more effective in terms of training time compared to an existing data driven method.

Reviewers had good initial impressions of this paper, pointing out the significance of the idea and the soundness of the setup. After a productive rebuttal phase the authors significantly improved the readibility and successfully clarified the remaining concerns of the reviewers. This AC thus recommends acceptance. 
Decision: Accept (Poster)