From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: The work considers graph generative model, specifically tackling controllable generation. While the tackled task is highly interesting and important, the reviewers agreed about various weaknesses which should be adressed before publication. In particular, the authors are encouraged to improve the writing/clarify of the paper, clarify stronger the different to prior work and add missing related work, improve the experimental comparisons (e.g. with more baselines).
justification_for_why_not_higher_score: All reviewers agreed that the weaknesses outweigh the strenghts, thus, recommending rejection.
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper62/Authors
title: Thank you for the follow-ups. More explanations on VAE are added.
comment: 
We thank the reviewer for the follow-ups, and we are glad the reviewer agrees on the discussion on the flow-based models and diffusion models. We address the concern on the VAE models as below.

**Q: The reviewer claims that $q(x|z)$ is a distribution, so given an latent code $z$, the output $x$ should be random.**

1. **In general, a deterministic function can be a distribution**. First, we want to clarify that the above claim may not be valid in general. A deterministic mapping is also a distribution, e.g., the flow-based model or Dirac distribution.
2. **Common VAE is deterministic with reparameterization in practice.** Then let's get back to VAE. In this work, by default, when talking about the VAE, we are actually referring to the common VAE or AEVB using the multivariate Gaussian (Sec 3 in the [VAE paper](https://arxiv.org/pdf/1312.6114.pdf)). Then, we want to claim that common VAE is indeed deterministic with the reparameterization trick. Here the common VAE means the one introduced in the original VAE paper. This is supported by the following references:
    - Kingma's tutorial [here](https://arxiv.org/pdf/1906.02691.pdf), please check Figure 2.3.
    - A relevant work [1] also discusses the common VAE (before proposing their method):
        - Before eq(8), `... by a decoder that is deterministic in practice ...`.
        - In Sec 3, `However, common implementations of VAEs as ... a simpler, deterministic view for this probabilistic mechanism`.
3. **HierVAE is also a common VAE.** The backbone model used in our work, HierVAE, is also a common VAE. Feel free to check Sec 3.2 in the [HierVAE paper](https://arxiv.org/pdf/2002.03230.pdf).
4. **More rigorously, the decoder in common VAE is a Gaussian MLP.** This is a very detailed and more rigorous point. In what follows, we will explain with VAE on the continuous data, and similar conclusions can be obtained for the discrete data. For VAE on the continuous data, the decoding of the common VAE is also a Gaussian: $p(x|z) = \mathcal{N}(f(z); \sigma^2 I)$. In practice, many works are using the mean ($f(z)$) as the reconstruction $x$, as mentioned by [1]. The intuition is that by training VAE on a large dataset, when the loss converges to a small value, then the **mean** of the Gaussian MLP can be used as a good proxy for the data (drawn from the data distribution). To this end, by deterministic decoding, we mean that **the decoding of the multivariate Gaussian mean is deterministic**. In our paper, we wrote this from the common VAE aspect, not from the rigorous definition, since it may distract the readers' attention. We are not sure if the reviewer is asking this, **if so, we appreciate the reviewer for reading this carefully and we are happy to clarify this in the future version**. Meanwhile, we want to highlight that, since we are referring the deterministic decoding to the multivariate Gaussian mean decoding, it won't affect the effectiveness of GraphCG, e.g., with HierVAE as the backbone.

[1] https://openreview.net/forum?id=S1g7tpEYDS

----

Hope this can address your concerns, and we are happy to take more questions.

From: ICLR.cc/2023/Conference/Paper62/Reviewer_F7x5
title: Question re.  the function g
comment: I would like to thank the authors for the response. My key concern about $g$ is not yet addressed. 

The authors mentioned for VAE based models:
 > *the encoding is random with reparameterization and the decoding is deterministic*

I do not think it is correct.  The decoder is a distribution p(x|z) which means that even conditioned upon z, the generation of x is random.
Could the authors please clarify their points here?

From: ICLR.cc/2023/Conference/Paper62/Authors
title: Thank you for the follow-ups. We further clarify the conditions of GraphCG.
comment: 
Hi, thank you for reading our paper carefully and raising the score after the revision. To your remaining concerns, we reiterate them below and hope they can sufficiently address your concerns now.

**Q: The reviewer asks if our proposed model is specific to graphs and if our algorithm can be applied to other data like tabular data and audio. The reviewer also concerns that only two graph data is not sufficient.**

A brief answer is that our GraphCG algorithm is for data/applications satisfying **two conditions**: **(1) the SOTA generative models are not well-disentangled** and **(2) the SOTA generative models are encoder-decoder format with graph-level latent codes**. More details are listed below. Till this end, only the molecular graphs and point clouds are available.

- First, we want to highlight that graph is a very general data structure, and most of the data can be formulated as graph data, including but not limited to molecular graphs, point clouds, images, tabular data, and audio.
- In this work, we are focusing on the **general** setting, where all graph data can be considered. Meanwhile, the **two conditions** need to be satisfied.
- Then we specify the (in)-feasibility of each data structure as follows:
  - **Image.** We are happy to see that reviewer also agrees with us that the image is a special type of graph, as we also listed the related works in our previous reply. But the SOTA image generative model is well-disentangled (please check [5,6,7,8,9] in the previous reply). Thus, image editing becomes quite simple. In other words, the SOTA image generative model is too special and too well explored, and our work wants to study more **general data** other than images, where **a well-disentangled generative model is not available** (**condition 1**). For more details, the reviewer can check our previous reply.
  - **Molecular graphs and point clouds.** Thus, this drives us to study molecular graphs and point clouds.
  - **Tabular.** We are not quite aware of the generative model on tabular data. Maybe the reviewer can help give some references?
  - **Audio and language.** The SOTA models on such sequential data is based on the Transformer (e.g., BART, T5), where the latent is the token-level representation, not sequence-level. Our proposed GraphCG is the graph-level or sequence-level in this context (**condition 2**), thus directly applying GraphCG on such sequential data is infeasible. (The bottleneck is how to transform the edited sequence-level code updates to token-level for decoding.)
- To sum up, we have explained the main reasons why the mainstream data structures are feasible/infeasible. Specifically, we have claimed why image editing is not satisfied under **condition 1**, and why other data structures (audio, NLP, etc) are infeasible under **condition 2**. Meanwhile, we are happy to learn if the reviewer can give the specific references on the general graph data, where **both condition 1 & 2 are satisfied**.

**Q: The reviewer claims that `point clouds are not graphs in the narrow sense`.**

This reply was to the second point of `Weaknesses` in the reviewer's initial comments. We are happy that this point is addressed now.

**Q: The reviewer asks which results in Table 5 & 6 are used in Table 2.**

All such information has been provided in **both the main paper and our previous reply**. The reviewer can check the `Reproducibility Statement`. It's on the 10th page of the main paper. Or the reviewer can check our previous reply (please see reply `(2/2)`). We further put a copy here:

- Table 5: GraphCG-P with eq (27) and GraphCG-R with eq (25) are reported in Table 2.
- Table 6: GraphCG-P with eq (26) and GraphCG-R with eq (26) are reported in Table 2.


**Q: The reviewer asks why Eq(26) with GraphCG-R is always 0.**

- First, we want to highlight that it's only 0 with MoFlow (Table 5). When we use HierVAE as the backbone (Table 6), the values are normal.
- Thus, we want to say that such results, especially the non-linear functions, highly depend on the backbone models. But the analysis on the backbone models is beyond the scope of this work.

----

Hope this can address your concerns, and we are happy to take more questions.

From: ICLR.cc/2023/Conference/Paper62/Reviewer_VdZt
title: Response
comment: Dear authors,

Thank you for your thoughtful response to my review. Regarding my top three concerns, here is my response:

1. model description: the authors have adequately addressed my concerns.
2. referring to the model as graph-specific: the authors have not really addressed the core of my argument and argue that images are also special types of graphs. By that argument, the proposed method should still be evaluated on images because they are also graphs.

More specific responses:
> GraphCG is indeed a general graph-specific method

Can you describe what about the model is specific to graphs? What makes it infeasible to take the framework proposed by the authors and apply it to tabular data, audio, and so on?

> It would be great if the reviewer can give references saying that point clouds are not graphs.

I did not claim this. 

> We want to highlight that in general, graph editing is different from image editing. Image is very specific graph data with well-disentangled generative models.

I agree, but from what I see the method does not perform graph editing but editing of latent vectors, which are then decoded by a fixed pre-trained model. Thus, the method could be applied to arbitrary data domains as long as a data point is encoded as a single vector. 

My (not addressed) concern is the following: the authors refer to their method as graph-specific, yet at the core it is not (see my previous point). Therefore (a) the label seems misplaced and (b) it seems arbitrary to pick only two specific instances of graph data as evaluation domains. 

3. Insufficient ablation/ hyperparameter studies

The authors refer to Tables 5 and 6 in the appendix for the hyperparameter selection. Table 5 raises some questions. Why is the performance of the Eq. (26) version always zero for GraphCG-R? I don't see any explicit mentioning in the main text which versions are used in Table 2 -- how is the reader supposed to know?

All in all, I've increased my score yet am not willing to champion this work because of my remaining concerns mentioned above.

From: ICLR.cc/2023/Conference/Paper62/Authors
title: Reproducible results and updated description for objective function (1/2)
comment: 
We appreciate the reviewer for acknowledging the clarity of this work and the interesting results, and we also thank the reviewer for carefully reading our formulations. The reviewer's main concerns are on the reproducibility and objective function, and hope our reply can address the reviewer's concerns.

___

**Q: The reviewer identifies our work as not graph-specific.**

First we want to clarify that graph is a very general data structure, including the molecular graph and point clouds.
- In this sense, image is a very special case of graph/structured data. It has a very tight spatial correlation, which is in a rigid form. You can see check [1,2,3] for recent explorations on GNNs on images. Such a nice spatial correlation enables disentangled generative model. For example in vision, StyleGAN [4] has proven with nice disentanglement properties [5,6,7,8,9].
- But possessing such a nice generative model is not the case for general graph data. I.e., we typically don’t have such a nice disentangled generative model, as discussed in Sec 1 and 3.
  - If the generative model has some very nice disentanglement properties, like StyleGAN, then the unsupervised steerable exploration can be quite straightforward, like SeFa. SeFa is the SOTA unsupervised editing method on the image, and it only needs a simple PCA operation.
  - But, in reality, and in the general setting, the generative models fail to provide nice properties like disentanglement (for the general graph data). Then we need GraphCG. It is a general framework for less ordered graph data (compared to images).
  - Further, when we compare SeFa with GraphCG, we can observe that GraphCG is better by a large margin. This also verifies that the general graph controllable generation is a more challenging task.
- So to get back to the reviewer’s question, GraphCG is indeed a general graph-specific method, aiming at exploring the steerable factors in the not-well-disentangled generative model setting. And obviously, image generation is not included in this research line because the SOTA image generative model is good enough (well-disentangled [5,6,7,8,9]) with simple yet efficient methods like SeFa.

**The reviewer claims that `Point clouds are not graphs in the narrow sense.`**

It would be great if the reviewer can give references saying that point clouds are not graphs. On our end, we find certain works [10,11,12] are treating the point clouds as graphs and applying GNNs on top of it ([12] is a nice survey on this).


**The reviewer has some questions in the objective function description.**
- `Details in Appendix D`. Thanks for reminding. We removed it because it’s too long but quite well-known as the standard NCE. So in Appendix D, we refer to another work (GraphMVP [38] in the paper references with the exact derivatives). Now in the revised version, we have added them back.
- `confused about the positive and negative pairs and Eq 10`. The latent code pairs themselves are directly related to positive or negative. What defines the positive (negative) is if the latent pairs move along the same (different) directions. These information has been revealed in Eq 10. Eq 10 is essentially the standard binary cross-entropy, and the positive pairs are the positive terms.
- `on the noise distribution is to sample uniformly on the input set`. Sorry for the confusion. It's a typo and we have fixed (also highlighted) in the revised version. This can also solve your following question on the construction of positive pairs. We have highlighted the corresponding part in the latest revision. Hope it is much clear now.
- To sum up, this part is also the core method section of GraphCG. We added the derivations and fixed the noise distribution in the latest version. Meanwhile, we want to highlight that the positive and negative pairs are defined on the direction and step size, which is the biggest difference from previous contrastive learning works (also asked by reviewer CQ2j). This is like the core idea of GraphCG. To better understand it, we provide more details in Figure 1(a) and Algorithm 1. Maybe the reviewer can also check them to get a better understanding.

From: ICLR.cc/2023/Conference/Paper62/Authors
title: Reproducible results and updated description for objective function (2/2)
comment: **The reviewer concerns on which linear and nonlinear functions (in Sec E) are used.**

Actually, we provided these details in Sec F & G. For example, one of the main concerns on the optimal hyperparameters that we report in Table 2, we have such details in Table 5 & 6.
- Table 5: GraphCG-P with eq (27) and GraphCG-R with eq (25) are reported in Table 2.
- Table 6: GraphCG-P with eq (26) and GraphCG-R with eq (26) are reported in Table 2.

**The reviewer also concerns the reproducibility issue.**

We appreciate the reviewer for raising this. We had all these details in our local bash scripts, which will also be released together with the codes.

So for Figure 4, we are using the GraphCG-P with eq (27) on HierVAE. We have all the log files and output files (e.g., the SMILES of output molecules) saved.
- We put the output files in an anonymous link here in case the reviewer is interested: https://anonymous.4open.science/r/GraphCG_outputs-075A/.
- If the reviewer is interested in more details, we list the direction index and data index below. In the csv files that we shared:
  - Direction 0 is the halogen fragment (data 4, 71).
  - Direction 5 is the amide fragment (data 95, 61).
  - Direction 6 is the chain length (data 57, 14).
  - Direction 7 is the alcohol and phenol fragments (data 10, 8).
- We just want to highlight that we have carefully maintained our codes and results for reproducibility.

**The reviewer's last concern is on the limitation of the datasets and models.**

On our end, we have searched the literature carefully and provided very comprehensive results on the molecular graph and point clouds.
- On the molecular graph, we have both the quantitative and qualitative results on two datasets and two generative models.
- On the point clouds, we couldn’t find a good quantitative measure (if the reviewer has any reference for us to check, that would be great), yet the qualitative results still show promising findings (e.g., the engine of airplanes) on three point clouds datasets.
- The issue on the image has been explained above. We want to highlight that in general, graph editing is different from image editing. Image is very specific graph data with well-disentangled generative models. The **highest priority** in this work, as claimed in Sec 1 & 3 & 4, is the general setting where the generative model is not perfectly disentangled.
- To sum up, we included 3 generative models on 5 datasets. This is by far, to the best of our knowledge, the most comprehensive editing experiment along the graph editing research line. It wold be great if reviewer can provide references to related work with more comprehensive results. We are happy to check and learn from those works.


[1] Han, Kai, et al. "Vision GNN: An Image is Worth Graph of Nodes." arXiv preprint arXiv:2206.00272 (2022).

[2] https://graph-neural-networks.github.io/static/file/chapter20.pdf

[3] Corso, Gabriele, et al. "Principal neighbourhood aggregation for graph nets." Advances in Neural Information Processing Systems 33 (2020): 13260-13271.

[4] Karras, Tero, Samuli Laine, and Timo Aila. "A style-based generator architecture for generative adversarial networks." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.

[5] Collins, Edo, et al. "Editing in style: Uncovering the local semantics of gans." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

[6] Shen, Yujun, et al. "Interpreting the latent space of gans for semantic face editing." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

[7] Härkönen, Erik, et al. "Ganspace: Discovering interpretable gan controls." Advances in Neural Information Processing Systems 33 (2020): 9841-9850.

[8] Tewari, Ayush, et al. "Stylerig: Rigging stylegan for 3d control over portrait images." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

[9] Wu, Zongze, Dani Lischinski, and Eli Shechtman. "Stylespace analysis: Disentangled controls for stylegan image generation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

[10] Tailor, Shyam A., et al. "Towards efficient point cloud graph neural networks through architectural simplification." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

[11] Shi, Weijing, and Raj Rajkumar. "Point-gnn: Graph neural network for 3d object detection in a point cloud." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

[12] Guo, Yulan, et al. "Deep learning for 3d point clouds: A survey." IEEE transactions on pattern analysis and machine intelligence 43.12 (2020): 4338-4364.

From: ICLR.cc/2023/Conference/Paper62/Authors
title: GraphCG is different from contrastive self-supervised learning
comment: 
We appreciate the reviewer for acknowledging the importance of our work, the clarity of our writing, as well as the effectiveness of GraphCG. The reviewer's main concerns are on the novelty and experimental baselines, which can be summarized as the comparison between GraphCG and graph contrastive SSL. We were aware of this during the submission and we had a paragraph for detailed discussion in Sec 4.4. In what follows (and also the latest revision), we highlighted the key differences. We hope the following answer can address your concerns.

___

**Q: The reviewer concerns about the literature on disentangled graph learning methods.**

We are aware there are certain contrastive pretraining methods on graph (as discussed in Sec 4.4). GraphCG is fundamentally different from them, and contrastive learning cannot be used for the editing task. Other than these works, it would be great if the reviewer can list the references for us to check.

**Q: The reviewer asks about the time complexity and efficiency.**
- We added the time complexity in the revision (Sec E).
- For the empirical efficiency, we had it in Sec E, together with the detailed hyperparameters. In brevity, all the jobs can be done within 3 hours on a single V100 GPU card.


**Q: The reviewer asks if we should consider contrastive SSL as baselines.**

We would like to check with the reviewer on what disentangled graph learning methods are referred to here. If the reviewer is talking about graph contrastive SSL methods like ContextPred, Infograph, Deep Graph InfoMax, GraphCL, GraphMVP, etc, then the answer is no, GraphCG is fundamentally different from them.
  - The views defined in the contrastive SSL methods are on the graph data (adding nodes, removing edges, etc), and the goal is to learn a better representation.
  - But in GraphCG, the views are defined on the editing direction and step size (in the latent space). The representation (which is obtained from the generative model) is fixed, and the goal is to learn semantically meaningful directions.
  - Therefore, these two are fundamentally different, though both are using contrastive learning. We have a more thorough discussion in Sec 4.4. Feel free to check it out.

Other than these works, we are happy to discuss if the reviewer can provide the referrences of related works.

From: ICLR.cc/2023/Conference/Paper62/Authors
title: More intuitions are explained to better understand the paper
comment: We appreciate the reviewer for acknowledging that graph editing is an interesting problem. This work is a little math-heavy, and may require certain prior knowledge (generative model, MI, EBM, NCE, etc) to read. In the following, we further explain the high-level idea in a more intuitive way to help the reviewer to better understand our work, in which we hope can increase the reviewer's confidence in the significance of our work.

___


**The reviewer raises questions on the high-level idea of how MI can be helpful for steerable factor exploration.**

We list the high-level intuition into the following key steps for the reviewer to better understand it:
  - We have **one assumption** that maximizing the MI on the direction and step size can help learn semantic-rich information.
    - To answer the question of the reviewer on the **intuition** of why this MI can work. MI measures the non-linear dependency between two variables. Here if we maximize the MI between two directions, we are assuming that for latent codes (encoding from different data points) moving along the same direction and step size, they can share more similar semantic information; they share less information if from different directions (or step sizes).
    - We want to further clarify one point with the reviewer is that, here, we are not learning the representation, because the encoder is fixed (with a certain degree of randomness). What we care about is how semantic information is transferred in the latent space.
  - Then in terms of the method (GraphCG), we introduce EBM as the solution. Why EBM? Because it has many promising solutions (we adopt NCE here). In the future, we can consider more advanced solutions like the diffusion model (Sec 6). However, in this work, the first priority is to prove if the assumption holds, so we adopt NCE, the simple yet efficient solution.
  - Finally, with GraphCG, we empirically get good results both qualitatively and quantitatively, and **together they can prove that our assumption holds**. Especially for figure 3, where we can observe that the learned 4 directions do match with the domain/chemistry knowledge. To the best of our knowledge, this is the first work showing such observation in the graph controllable generation research line.
  - To sum up, we first have one assumption on maximizing MI on latent directions can help explore the steerable factors. To solve it, we use EBM and NCE. Finally, the empirical findings can prove that this assumption holds, with results matching the domain knowledge.

**The reviewer asks if the disentanglement measures can be compared with editing methods.**

Table 1 shows the disentanglement measures/metrics, and Table 2 is about the editing methods. So these two tables cannot be straightforwardly merged. Meanwhile, we do provide the disentanglement before and after the editing, details are as followed:
  - Recall that we need to determine the factors before the disentanglement evaluation. So we choose the meaningful GraphCG results (details in `Reproducibility Statement` in the latest revised paper), where we learn 4 directions and each with a piece of semantic information (fragments in the molecular graphs).
  - Then we calculate the disentanglement measure before and after editing. The results are in Table 7 (in the latest revised paper), where the disentanglement measure does increase after editing with GraphCG.


**The reviewer asks if we have qualitative results other than Tanimoto measure.**

The answer is yes. In addition to the quantitative results, we also show the qualitative results on both molecular graphs and point clouds. For example, in Figure 3 and Figure 6, we show that GraphCG is able to learn 4 directions that can match with the domain knowledge, where the steerable factors are halogens, hydroxyls, amides, and chain length, respectively. Similar qualitative results can be observed for the point clouds in Figure 4 and the appendix.

From: ICLR.cc/2023/Conference/Paper62/Authors
title: We clarify the problem setting of graph editing
comment: 
We appreciate the reviewer for acknowledging the importance of this graph editing problem, as well as the effectiveness of GraphCG. For the clarity issue raised by the reviewer, we provide more detailed explanations as below.

___

**The reviewer identifies that the latent code $z$ is sampled from a VAE-like distribution.**

Yes, but the $\mathcal{N}(\mu(G), \Sigma(G))$ (VAE) is just one specific example. Our proposed GraphCG is model/distribution-agnostic and can be applied to arbitrary generative models, like flow-based models, diffusion models, etc. Besides, we want to highlight that the generative model is pretrained and fixed (see `Problem formulation` in Sec 2).


**Q: `Is it the case, that we are performing edits on z and then computing the expected objective on the edited z?`**

Yes, this is what GraphCG is doing.


**Q: the reviewer has some questions on the distribution shift**.

We need to double-check with the reviewer on the distribution of what variables. Do you mean if the model distribution will change? Or if the distribution of the output graph will change compared to the input graph?
  - If you are referring to the first case, then no. Because we are not changing the encoder and decoder (fixed and pretrained). As long as the backbone generation is pretrained on a large dataset and frozen (see Sec 2), then it is expected to cover a large molecule space and the distribution won’t shift.
  - If you are referring to the second case, then yes. Because that’s what the editing is doing: we expect that we can change the input graph with certain common modifications along the same direction.


**Q: The reviewer asks why don't we change the encoder.**

The answer is that, when we change the encoder, the model distribution changes, which is not expected in the context of graph editing. In graph editing, the generative model (encoder and decoder) is pretrained on the large dataset, and it is expected to cover a wide range of distribution modes. Here, we are not changing the model distribution (which aims at estimating the data distribution). But instead, we are modifying the input graphs following the model distribution. This is a very standard setting in the other domain [1].


**Q: on Eq 2 and how the graph is generated in a deterministic manner?**

Wheter or not $z$ is deterministic w.r.t. $x$ depends on the backbone generative model. Yet, Eq 2 holds for both cases.
  - In $\bar x' = g(\bar z)$, $\bar z$ is the edited latent code, $\bar x'$ is the edited output graph, and $g$ is the decoder. Notice that $g$ is needless to be deterministic. We list more detailed discussions below:
    - If the generative model is a flow-based model, then it is deterministic in the auto-encoding pipeline. For the inference time, $z$ is randomly sampled. Related works on graph generation include GraphAF, and MoFlow. The reviewer can check these papers and also the flow-based model materials [2,3].
    - If it’s the standard VAE-style model (like HierVAE used in this paper), the encoding is random with reparameterization and the decoding is deterministic. Again, for inference, $z$ is randomly sampled and $\bar z \to \bar x’$ is deterministic. A nice tutorial can be found here [4].
  - In this work, when GraphCG learns the directions, it’s using the auto-encoding pipeline, as shown in Eq 2.
  - We want to highlight that our proposed GraphCG is model-agnostic, i.e., it can be applied to the arbitrary backbone generative model.


**Q: The reviewer also concerns about the lack of contrast from prior work.**

Maybe the reviewer can give some references to the prior works? We are glad to check them during the rebuttal.


[1] Shen, Yujun, and Bolei Zhou. "Closed-form factorization of latent semantics in gans." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

[2] https://deepgenerativemodels.github.io/notes/flow/

[3] https://lilianweng.github.io/posts/2018-10-13-flow-models/

[4] https://jaan.io/what-is-variational-autoencoder-vae-tutorial/

From: ICLR.cc/2023/Conference/Paper62/Reviewer_F7x5
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper aims at finding the steerable factors of a graph. It probes the latent space, trains an edit function and a distance function.
The objective function of learning the edit function and the distance function consists of three quantities--- the noise contrastive estimation based objective, sparsity prone regularizer and the diversity encouraging similarity regularizer.
Experiments have been conducted to support the efficacy of their model.
strength_and_weaknesses: > Strengths

1. The paper tackles an important problem for controllable graph generation
2. Apparently simple approach seems to work well

> Weakness

1. Lack of clarity in many places
2. Lack of contrast from prior work 
clarity,_quality,_novelty_and_reproducibility: > Clarity

The paper lacks clarity significantly.  My main concern in understanding the paper are two folds--- the exact mode of operation of $z$ and how the effect of editing $z$ is being realized on graph generation.
In my understanding the latent code $z$ is a random variable which is being sampled from some distribution $\mathcal{N}(\mu(G),\Sigma(G))$-- here $\mu$ and $\Sigma$ are neural nets. Is it the case, that we are performing edits on z and then computing the expected objective on the edited z? I suspect there will be a lot of variance for this. In principle, are you not trying to shift the distribution?
Why then do you not modify the encoder of a VAE directly and regularize with respect to the trained encoder?

Eq. (2) is very confusing  to say the least. Is $z$ a deterministic function of the graph data $x$? Then, are the graphs being generated in deterministic manner. What do you mean by $\bar{x} = g(z)$? How, the graph data is being generated in a deterministic manner.

> Quality

Given such lack of clarity, I am not in  a position to comment on the quality of the paper.

> Novelty

Although, the novelty is limited, I am fine as long as this simple method is sound and works well in practice.

> Reproducibility

I tried to find the code for better understanding of the method, unfortunately, I could not find one.

summary_of_the_review: My main concern is clarity (this can impact quality and technical soundness too) of the paper, as I mentioned above. Also, a not-so-minor point is that in ICLR, citations are given in author-year format rather than the citation-number format, whereas the authors follow latter. Note that, changing the current format to citation-number format would change the content significantly.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper62/Reviewer_wzWp
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper:  The paper proposes a method denoted as GraphCG to perform unsupervised discovery of steerable factors in latent space of deep graph generative models. The paper is to answer questions:  (1) is the latent representation space learned from DGMs on graph data disentangled
(2) Given a pretrained DGM with not perfectly disentangled latent space, is there a flexible framework enabling the graph controllable generation in an unsupervised manner?


strength_and_weaknesses: The paper is solving an interesting problem.

Weakness: 
1. Idea: 
The paper combines ideas like graph encoder/decoder, mutual information, contrastive estimation, EBM. I do not see how these ideas help with better understanding of the latent space 

Can the authors give some intuition about "It starts with the assumption that the latent representations edited
with the same semantic direction and step size should possess similar information (corresponding
to the factors) to certain degree, thus by maximizing the mutual information them, we can learn the
most semantic-rich directions." and "we set the editing condition as containing both the semantic directions and step
sizes, and we assume that maximizing the MI between different conditions can maximize the shared
information within each condition. " -- why doing so we can learn meaningful latent representation? Is figure 4 can show some intuition?

2: Experiments: 
Can the paper include comparison with other disentanglement methods? e.g. in Table 1

3 evaluation: 
How do we sure the learned latent representation is really meaningful? The authors give Tanimoto measure for molecular editing. 
clarity,_quality,_novelty_and_reproducibility: the evaluation can be more clear
summary_of_the_review: The paper asked two questions in the introduction; however, I do not feel fully persuaded by the solution proposed in the paper. The evaluation of the paper can be more improved. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper62/Reviewer_CQ2j
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper tries to develop a method called GraphCG for the unsupervised discovery of steerable factors in the latent space of deep graph generative models. The authors observe that the learned representation space of some methods is not perfectly disentangled.  Thus, they propose to learn the semantic-rich directions by maximizing the corresponding mutual information for tackling this problem. The experiments on molecular graphs and point clouds show the effectiveness of the proposed GraphCG.
strength_and_weaknesses: The strengths of this paper are as follows:
1. This paper focuses on a fundamental and important research problem which is discovering steerable factors in graphs.
2. This paper is clearly written and the proposed method is easy to understand to me. Figure 1 clearly presents the training phase of the method, which is also clearly listed in Algorithm 1. 
3. The experimental results especially for Figures 3 and 4 well show the effectiveness of the proposed method.

The weaknesses of this paper are as follows:
1. The authors seem to ignore some literature on disentangled graph learning methods. The main difference between disentangled latent factors in disentanglement literature and the steerable factors in the paper is not well discussed. I am not sure whether the disentangled graph learning method is good enough for solving the problem, which raises my concerns about the technical novelty of this paper.
2. The time complexity as well as the empirical efficiency of the proposed method is not well discussed, since relatively complex techniques are proposed in the model.
3. Another concern is about the experiment. I think the experiments should consider more baselines (e.g., disentangled graph learning methods and SSL methods as discussed in section 4.4) for more convincing results.

clarity,_quality,_novelty_and_reproducibility: The clarity and quality of this paper are good, considering its clear expressions and good experiments results, while the novelty and experimental baselines are two of my biggest concerns. It would be good if the authors can address these concerns above during the rebuttal.
summary_of_the_review: In summary, this paper focuses on an important problem and proposes an effective method to tackle this problem. But I still have some concerns above. Overall, I recommend weak reject for this paper.

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper62/Reviewer_VdZt
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The authors propose GraphCG, a method for finding steerable factors in the latent space of already-learned deep generative model (DGM) for graphs. The method learns an energy-based model (EBM) in the latent space of a DGM with the goal of finding direction vectors $d_i$ that correspond to meaningful semantic modification of the latent vectors. The authors evaluate their method on molecular graphs and point clouds. 

strength_and_weaknesses: Strengths:
* Steerable generation is a key challenge in generative modeling.
* The paper is generally well-written and the presentation is mostly clear.
* The authors present some interesting experimental results.

Weaknesses:
* There seems to be nothing graph-specific about the proposed method itself.
* The authors only evaluate their method on molecular graphs and point clouds, where the latter are not graphs in the narrow sense.
* Some aspects of the model description are confusing to me (see next section for details).
* No results about ablation or hyperparameter studies are provided, e.g., comparing the linear and non-linear editing functions. It is not clear which version the authors used in each of their experiments. 
clarity,_quality,_novelty_and_reproducibility: **Clarity**: mostly good, with the exception of the model description, which I find vague and confusing in parts. The loss function is not entirely clear to me. First, below Eq. (10) the authors refer to Appendix D for details, but none can be found there. Next, I'm confused about the positive and negative pairs. Equation 10 takes a latent code pair as input and does not distinguish whether the latent code pair is a positive or negative pair. Should negative pairs not be treated differently than positive pairs? Next, the authors say that the noise distribution is to sample uniformly on the input set. However, in Eq. (10) the noisy samples also have $j$ and $\beta$, which are different than $i$ and $\alpha$ of the original data point. Is this also a part of the noise distribution? Also, the noise distribution (take a uniform sample from the empirical data) is the same as the construction of **positive** pairs as described for GraphCG-R. How can this be resolved? This part of the paper should be thoroughly revised to improve clarity.

**Quality**: Technically, the method is relatively simple and not specific to graphs at all. It could equivalently be applied to any other data domain, as it only relies on input samples to be encoded as continuous latent vectors. As a consequence, it should also be evaluated as a general-purpose method, e.g., also on some image datasets. Moreover, even the evaluation focusing on graphs is very narrow, with a heavy focus on molecules plus a small evaluation on point clouds, which are not even graphs in the narrow sense. Overall, the set of datasets and backbone models is quite limited.

**Novelty**: I am not aware of all the latest literature in steerable generation for DGMs, so I cannot judge this aspect adequately.

**Reproducibility**: Given the paper and the supplementary material it is impossible to reproduce the results because the authors do not specify for each of their results which version of the editing function they used (Fig. 4) and which hyperparameter settings performed best. There are two variants of linear editing function in Appendix E, and it is not clear which of those was actually used. The authors did not provide an implementation of their method.
summary_of_the_review: The current manuscript's weaknesses outweigh the merits in my view. The most important drawbacks to me are:
* The vague and (at least to me) not sufficiently clear model description.
* Labeling the method as graph-specific while there is (as far as I can see) nothing graph-specific about it.  
* Insufficient ablation/ hyperparameter studies are provided. The only ablation study is visual in nature and thus not a quantitative comparison of the variants.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold