Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: # Post Rebuttal

The authors have partially and satisfactorily addressed my concerns. In line of this I am raising my score to Weak Accept.

This paper proposes a new molecular graph generative model (GraphAF) which fuses the best of two worlds of generative networks - reversible flow and autoregressive mode. Such integration enjoys a) faster training due to parallel computation b) molecular validity checker during inference supported by sequential sampling process and c) exact likelihood maximisation due to invertible encoder. In lieu of such advantages, the model trains two times faster than the existing state-of-the-art and generates 100% valid molecules when trained on ZINC dataset. Further, it also demonstrates that additionally if the chemical properties are optimised during training with reinforcement learning policy then GraphAF outperforms all the prior works.

Although the paper presents an interesting fusion of different generative models, in its current form it leans towards rejection due to the following factors:
1) The empirical validation of GraphAF is contained to single dataset - ZINC with a maximum of 38 atoms. From the table 2, it seems to me every prior method works pretty well on important metrics. There is very little room for improvement. I recommend including results on QM9 and CEPDB datasets. 
2) The model being data-agnostic, it makes sense to evaluate them on generic graph datasets - synthetic and real.
3) The novelty of the model is limited. The flow-based graph generative model is introduced in Graph Normalizing Flow (GNF) (NeurIPS'19, NeurIPS'18 workshop). The reversible flow is extended to whole graph in GraphNVP. Unlike GNF, GraphNVP and GraphAF do away with decoder. The major difference being the sampling process - one-shot to sequential.
I am willing to improve my rating given that some of this points are addressed.

Clarification:
1. What are the inputs edge-mlp's operate on ? Given the generation step is sequential, it is not clear to me why all the node embeddings H_i^L is given as input in eq (8). I also noted that the dimension of H_i^L varies with size of sub-graphs. Also note mismatch in the notation 'f' used in algorithm 1 and 'g' from the main text. 
2. Please compare inference time. 

Other weakness:
1. Due to invertible flow modeling, the latent space is usually restricted to small dimension. In current case it is 9 for node feature and 3 for edge features. This drawback alongside the sequential edge generation prevents GraphAF from scaling to complex and large graphs with many labels.
2. Moreover, GraphAF utilizes only single layer of flow i.e., eq (9). This is clearly not sufficient to model complex graphs. And in its current form it is not clear how one can extend to multi-layer flow.
3. The encoder modeling in GraphAF also shares similarity with Variational graph auto-encoder. Instead of constraining latent distribution using KL divergence, GraphAF maximizes graph likelihood to enforce base distribution.

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper proposes a new invertible-flow based graph generation model. 
The main difference from the previous flow-based generation model (GraphNVP) is the choice of flow mappings (coupling flow --> autoregressive flow). 
The authors formulate conditional probabilities of iterative node/edge generation. Iterative sampling scheme naturally allows incorporating validity checks in each sampling step, assuring 100% validity in graph generation. The paper also proposes an implementation of molecule lead optimization combined with a RL framework. The experimental results show superiority in terms of valid graph generation and property optimization. 


Overall, the paper is written well. I feel no difficulty in understanding the main idea and the equations in the paper.
 
Introduction of the normalizing flows (Sec 3.1) can be expanded to reach non-expert users. Advantages of using invertible flows (against other generative models such as GANs and VAEs) are not described rigorously in the current manuscript. I also suggest citing a nice review for invertible flows appeared recently: 

Ivan Kobyzev, Simon Prince, and Marcus A Brubaker, ``Normalizing Flows: Introduction and Ideas'', arXiv: 1908.09257, 2019.


Explanations of the Sec 4.4 (+ appendix B) is simply insufficient to reproduce the experiments. More descriptions or references are required. 


Experimental results seem promising. A better validity score than the previous flow model illustrates the efficacy of the autoregressive flow against the coupling flow. 
The performance on the property optimization (Table 3) seems brilliant. However, there is no discussion why the combination of the autoregressive flow and the RL performs greatly, compared to baselines. Some discussions will help the community to further improve the optimization tasks in the future. 

+ Overall, a good paper. well written, easy to understand. 
+ A new variant of the invertible-flow based graph generation model.  The novelty lies in the iterative generation process, naturally combined with the autoregressive flow. 
+ Superior to the one-shot flow baseline (GraphNVP) even if additional validity checks are omitted (Table 2)
+ Good performances in property optimizations (Table 3, 4) 
- The explanation for RL process is simply insufficient for reproduction.
-- No discussions about reasons why GraphAF+RL performs great in property optimization. 

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes a generative model architecture for molecular graph generation based on autoregressive flows. The main contribution of this paper is to combine existing techniques (auto-regressive BFS-ordered generation of graphs, normalizing flows, dequantization by Gaussian noise, fine-tuning based on reinforcement learning for molecular property optimization, and validity constrained sampling). Most of these techniques are well-established either for data generation with normalizing flows or for molecular graph generation and the novelty lies in the combination of these building blocks into a framework. Training can be carried out in parallel over the sequential generation process, as no hidden states with sequential dependency are assumed (unlike a regular RNN). Experimental validation is carried out on a standard ZINC molecule generation benchmark (graphs with up to 48 nodes) and the reported metrics are competitive with recent related work.

Overall, the paper is very well written, nicely structured and addresses an important problem. The framework in its entirety is novel, but the building blocks of the proposed framework are established in prior work and the idea of using normalizing flows for graph generation has been proposed in earlier work (see [1] and [2]). Nonetheless, I find the paper relevant for an ICLR audience and the quality of execution and presentation of the paper is good.

I have two major (technical) concerns with the flow-based formulation used in the paper with regards to order-invariance and the utilized de-quantization scheme.
* Order-invariance: The paper states that the “exact density of each molecule can be efficiently computed by the change-of-variables formula”. This seems to be incorrect, as the exact density is a product over all order-specific densities for all possible permutations in which the molecular graph can be represented. The change-of-variables formula does not provide an efficient way to circumvent this order-invariance issue, at least not in the way it is presented in the paper. Even when using BFS-ordered representations, the subspace of possible permutations is still typically too large to allow for efficient evaluation of the exact density. I suspect that the authors assume a canonical ordering of the graph representations, which is a strong assumption, but does not seem to be mentioned in the paper. How is the canonical ordering chosen? How is local structural symmetry broken in a consistent manner?
* De-quantization: The de-quantization scheme used in this paper seems to be ill-suited for categorical variables. What motivates the use of adding Gaussian noise to categorical (one-hot encoded) variables, other than that it seems to work OK in the reported experiments? Adding Gaussian noise in this way can move these variables outside of the probability simplex — is this a valid technique in the framework of normalizing flows? Adding Gaussian noise makes sense if the data represents quantized continuous data, e.g. bit-quantized image data, but I have concerns about the validity of using this method for categorical data (both edge type and node features are categorical in this application). Other comparable generative models for graph-structured data use a relaxed discrete distribution (concrete / Gumbel softmax), e.g. in MolGAN [De Cao & Kipf (2018)], to address this issue — would this also be applicable here?

I think that these two issues will have to be addressed before this paper can be considered for publication, and I recommend a weak reject at this point.

[1] Madhawa et al., GraphNVP: An invertible flow model for generating molecular graphs. (2019)
[2] Liu et al., Graph Normalizing Flows. (2019) — not cited


UPDATE:

My two main technical concerns have been addressed in the rebuttal and I think that the revised version of the paper can be accepted to ICLR (my comment w.r.t. novelty still holds and hence I recommend 'weak accept').
review_title: Official Blind Review #1

Meta-review: All reviewers agreed that this paper is essentially a combination of existing ideas, making it a bit incremental, but is well-executed and a good contribution.  Specifically, to quote R1:

"This paper proposes a generative model architecture for molecular graph generation based on autoregressive flows. The main contribution of this paper is to combine existing techniques (auto-regressive BFS-ordered generation of graphs, normalizing flows, dequantization by Gaussian noise, fine-tuning based on reinforcement learning for molecular property optimization, and validity constrained sampling). Most of these techniques are well-established either for data generation with normalizing flows or for molecular graph generation and the novelty lies in the combination of these building blocks into a framework. ... Overall, the paper is very well written, nicely structured and addresses an important problem. The framework in its entirety is novel, but the building blocks of the proposed framework are established in prior work and the idea of using normalizing flows for graph generation has been proposed in earlier work. Nonetheless, I find the paper relevant for an ICLR audience and the quality of execution and presentation of the paper is good."
Decision: Accept (Poster)