Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper aims at finding the steerable factors of a graph. It probes the latent space, trains an edit function and a distance function.
The objective function of learning the edit function and the distance function consists of three quantities--- the noise contrastive estimation based objective, sparsity prone regularizer and the diversity encouraging similarity regularizer.
Experiments have been conducted to support the efficacy of their model.
strength_and_weaknesses: > Strengths

1. The paper tackles an important problem for controllable graph generation
2. Apparently simple approach seems to work well

> Weakness

1. Lack of clarity in many places
2. Lack of contrast from prior work 
clarity,_quality,_novelty_and_reproducibility: > Clarity

The paper lacks clarity significantly.  My main concern in understanding the paper are two folds--- the exact mode of operation of $z$ and how the effect of editing $z$ is being realized on graph generation.
In my understanding the latent code $z$ is a random variable which is being sampled from some distribution $\mathcal{N}(\mu(G),\Sigma(G))$-- here $\mu$ and $\Sigma$ are neural nets. Is it the case, that we are performing edits on z and then computing the expected objective on the edited z? I suspect there will be a lot of variance for this. In principle, are you not trying to shift the distribution?
Why then do you not modify the encoder of a VAE directly and regularize with respect to the trained encoder?

Eq. (2) is very confusing  to say the least. Is $z$ a deterministic function of the graph data $x$? Then, are the graphs being generated in deterministic manner. What do you mean by $\bar{x} = g(z)$? How, the graph data is being generated in a deterministic manner.

> Quality

Given such lack of clarity, I am not in  a position to comment on the quality of the paper.

> Novelty

Although, the novelty is limited, I am fine as long as this simple method is sound and works well in practice.

> Reproducibility

I tried to find the code for better understanding of the method, unfortunately, I could not find one.

summary_of_the_review: My main concern is clarity (this can impact quality and technical soundness too) of the paper, as I mentioned above. Also, a not-so-minor point is that in ICLR, citations are given in author-year format rather than the citation-number format, whereas the authors follow latter. Note that, changing the current format to citation-number format would change the content significantly.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper:  The paper proposes a method denoted as GraphCG to perform unsupervised discovery of steerable factors in latent space of deep graph generative models. The paper is to answer questions:  (1) is the latent representation space learned from DGMs on graph data disentangled
(2) Given a pretrained DGM with not perfectly disentangled latent space, is there a flexible framework enabling the graph controllable generation in an unsupervised manner?


strength_and_weaknesses: The paper is solving an interesting problem.

Weakness: 
1. Idea: 
The paper combines ideas like graph encoder/decoder, mutual information, contrastive estimation, EBM. I do not see how these ideas help with better understanding of the latent space 

Can the authors give some intuition about "It starts with the assumption that the latent representations edited
with the same semantic direction and step size should possess similar information (corresponding
to the factors) to certain degree, thus by maximizing the mutual information them, we can learn the
most semantic-rich directions." and "we set the editing condition as containing both the semantic directions and step
sizes, and we assume that maximizing the MI between different conditions can maximize the shared
information within each condition. " -- why doing so we can learn meaningful latent representation? Is figure 4 can show some intuition?

2: Experiments: 
Can the paper include comparison with other disentanglement methods? e.g. in Table 1

3 evaluation: 
How do we sure the learned latent representation is really meaningful? The authors give Tanimoto measure for molecular editing. 
clarity,_quality,_novelty_and_reproducibility: the evaluation can be more clear
summary_of_the_review: The paper asked two questions in the introduction; however, I do not feel fully persuaded by the solution proposed in the paper. The evaluation of the paper can be more improved.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper tries to develop a method called GraphCG for the unsupervised discovery of steerable factors in the latent space of deep graph generative models. The authors observe that the learned representation space of some methods is not perfectly disentangled.  Thus, they propose to learn the semantic-rich directions by maximizing the corresponding mutual information for tackling this problem. The experiments on molecular graphs and point clouds show the effectiveness of the proposed GraphCG.
strength_and_weaknesses: The strengths of this paper are as follows:
1. This paper focuses on a fundamental and important research problem which is discovering steerable factors in graphs.
2. This paper is clearly written and the proposed method is easy to understand to me. Figure 1 clearly presents the training phase of the method, which is also clearly listed in Algorithm 1. 
3. The experimental results especially for Figures 3 and 4 well show the effectiveness of the proposed method.

The weaknesses of this paper are as follows:
1. The authors seem to ignore some literature on disentangled graph learning methods. The main difference between disentangled latent factors in disentanglement literature and the steerable factors in the paper is not well discussed. I am not sure whether the disentangled graph learning method is good enough for solving the problem, which raises my concerns about the technical novelty of this paper.
2. The time complexity as well as the empirical efficiency of the proposed method is not well discussed, since relatively complex techniques are proposed in the model.
3. Another concern is about the experiment. I think the experiments should consider more baselines (e.g., disentangled graph learning methods and SSL methods as discussed in section 4.4) for more convincing results.

clarity,_quality,_novelty_and_reproducibility: The clarity and quality of this paper are good, considering its clear expressions and good experiments results, while the novelty and experimental baselines are two of my biggest concerns. It would be good if the authors can address these concerns above during the rebuttal.
summary_of_the_review: In summary, this paper focuses on an important problem and proposes an effective method to tackle this problem. But I still have some concerns above. Overall, I recommend weak reject for this paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose GraphCG, a method for finding steerable factors in the latent space of already-learned deep generative model (DGM) for graphs. The method learns an energy-based model (EBM) in the latent space of a DGM with the goal of finding direction vectors $d_i$ that correspond to meaningful semantic modification of the latent vectors. The authors evaluate their method on molecular graphs and point clouds. 

strength_and_weaknesses: Strengths:
* Steerable generation is a key challenge in generative modeling.
* The paper is generally well-written and the presentation is mostly clear.
* The authors present some interesting experimental results.

Weaknesses:
* There seems to be nothing graph-specific about the proposed method itself.
* The authors only evaluate their method on molecular graphs and point clouds, where the latter are not graphs in the narrow sense.
* Some aspects of the model description are confusing to me (see next section for details).
* No results about ablation or hyperparameter studies are provided, e.g., comparing the linear and non-linear editing functions. It is not clear which version the authors used in each of their experiments. 
clarity,_quality,_novelty_and_reproducibility: **Clarity**: mostly good, with the exception of the model description, which I find vague and confusing in parts. The loss function is not entirely clear to me. First, below Eq. (10) the authors refer to Appendix D for details, but none can be found there. Next, I'm confused about the positive and negative pairs. Equation 10 takes a latent code pair as input and does not distinguish whether the latent code pair is a positive or negative pair. Should negative pairs not be treated differently than positive pairs? Next, the authors say that the noise distribution is to sample uniformly on the input set. However, in Eq. (10) the noisy samples also have $j$ and $\beta$, which are different than $i$ and $\alpha$ of the original data point. Is this also a part of the noise distribution? Also, the noise distribution (take a uniform sample from the empirical data) is the same as the construction of **positive** pairs as described for GraphCG-R. How can this be resolved? This part of the paper should be thoroughly revised to improve clarity.

**Quality**: Technically, the method is relatively simple and not specific to graphs at all. It could equivalently be applied to any other data domain, as it only relies on input samples to be encoded as continuous latent vectors. As a consequence, it should also be evaluated as a general-purpose method, e.g., also on some image datasets. Moreover, even the evaluation focusing on graphs is very narrow, with a heavy focus on molecules plus a small evaluation on point clouds, which are not even graphs in the narrow sense. Overall, the set of datasets and backbone models is quite limited.

**Novelty**: I am not aware of all the latest literature in steerable generation for DGMs, so I cannot judge this aspect adequately.

**Reproducibility**: Given the paper and the supplementary material it is impossible to reproduce the results because the authors do not specify for each of their results which version of the editing function they used (Fig. 4) and which hyperparameter settings performed best. There are two variants of linear editing function in Appendix E, and it is not clear which of those was actually used. The authors did not provide an implementation of their method.
summary_of_the_review: The current manuscript's weaknesses outweigh the merits in my view. The most important drawbacks to me are:
* The vague and (at least to me) not sufficiently clear model description.
* Labeling the method as graph-specific while there is (as far as I can see) nothing graph-specific about it.  
* Insufficient ablation/ hyperparameter studies are provided. The only ablation study is visual in nature and thus not a quantitative comparison of the variants.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The work considers graph generative model, specifically tackling controllable generation. While the tackled task is highly interesting and important, the reviewers agreed about various weaknesses which should be adressed before publication. In particular, the authors are encouraged to improve the writing/clarify of the paper, clarify stronger the different to prior work and add missing related work, improve the experimental comparisons (e.g. with more baselines).
justification_for_why_not_higher_score: All reviewers agreed that the weaknesses outweigh the strenghts, thus, recommending rejection.
Decision: Reject