Title: Diffusion Twigs with Loop Guidance for Conditional Graph Generation
Venue: NeurIPS 2024
Accepted: yes
Keywords: graph network

conditional generation

diffusion

generative models

molecule design

molecule optimization
Abstract: We introduce a novel score-based diffusion framework named  Twigs that incorporates multiple co-evolving flows for enriching conditional generation tasks. Specifically, a central or trunk diffusion process is associated with a primary variable (e.g., graph structure), and additional offshoot or  stem processes are dedicated to dependent variables (e.g., graph properties or labels). A new strategy, which we call loop guidance, effectively orchestrates the flow of information between the trunk and the stem processes during sampling. This approach allows us to uncover intricate interactions and dependencies, and unlock new generative capabilities. We provide extensive experiments to demonstrate strong performance gains of the proposed method over contemporary baselines in the context of conditional graph generation, underscoring the potential of Twigs in challenging generative tasks such as inverse molecular design and molecular optimization. 
Code is available at https://github.com/Aalto-QuML/Diffusion_twigs.

Diffusion Twigs with Loop Guidance for Conditional Graph Generation

Abstract

We introduce a novel score-based diffusion framework named Twigs that incorporates multiple co-evolving flows for enriching conditional generation tasks. Specifically, a central or trunk diffusion process is associated with a primary variable (e.g., graph structure), and additional offshoot or stem processes are dedicated to dependent variables (e.g., graph properties or labels). A new strategy, which we call loop guidance , effectively orchestrates the flow of information between the trunk and the stem processes during sampling. This approach allows us to uncover intricate interactions and dependencies, and unlock new generative capabilities. We provide extensive experiments to demonstrate strong performance gains of the proposed method over contemporary baselines in the context of conditional graph generation, underscoring the potential of Twigs in challenging generative tasks such as inverse molecular design and molecular optimization. Code is available at https://github.com/Aalto-QuML/Diffusion_twigs .

1 Introduction

Conditional graph generation is a fundamental problem in scientific domains such as de novo drug design [21, 43, 74] and material design [39]. However, searching for new molecules with desired physicochemical properties poses significant challenges to traditional brute-force methods due to the vast combinatorial spaces [64]. With the advent of neural networks [44], deep generative models have emerged as a powerful tool for learning informative conditional representations of molecules, facilitating the development of in silico methods for chemical design [16, 31, 61, 73].

Score-based diffusion generative models (SGMs) and denoising probabilistic diffusion models (DDPMs) [24, 67] have recently emerged as powerful techniques for training deep networks on graphstructured data, with applications spanning molecular design [37, 53, 36, 81], molecular docking [6], molecular dynamics simulations [78], protein folding [79], and backbone modeling [70]. Notably, diffusion models exhibit superior capabilities for conditional graph generation, excelling in both discrete [26, 75, 49] and continuous [3, 28, 45, 11] settings. The training of the mentioned conditional diffusion models is achieved by two types of diffusion guidance algorithms: classifier-based guidance

Yogesh Verma †

Vikas Garg

[8], which involves training a separate property predictor model alongside the diffusion model; and classifier-free guidance [23], which integrates scores from both unconditional and conditional diffusion models. While these guidance techniques have been found to be effective, the algorithm design is not tailored to encompass the intricate hierarchical or multi-resolution elements inherent in conditional generation. Consequently, it is plausible that this inadequacy may contribute to suboptimal representations, particularly notable in tasks such as conditional graph generation. The recent success of hierarchical diffusion flows in various domains, such as modeling interactions between node and edge features [37], multi-resolution modeling [25], decision-making [47], and conditional image generation [4, 71] underscores the need to integrate hierarchical information beyond the capabilities of classifier-based and classifier-free guidance.

We assert that conditional diffusion models for structured spaces, such as graphs, could be enhanced with hierarchical conditional processes . Specifically, rather than treating heterogeneous structural and label information uniformly within the hierarchy, we advocate for the co-evolution of multiple processes with distinct roles (asymmetric). These roles encompass a primary process governing the structural evolution alongside multiple secondary processes responsible for driving conditional content. We aim to propose an alternative to existing conditional graph diffusion techniques (outlined in Table 1) by bestowing the models with finer control over two key aspects: 1) the evolution of structural graph components, including nodes and edges, and 2) the co-adaptation of the graph structure in conjunction with one or more associated properties.

Towards this objective, we present a novel diffusion framework for conditional generation named Twigs , drawing analogies from the trunk and offshoots of a tree. Concretely, we establish a central trunk process governing a primary variable, which interacts with several stem processes, each associated with a secondary variable. In contrast with classifier-free and classifier-based methodologies, a novel conditional mechanism, termed loop guidance , orchestrates information exchange between the trunk and the stem processes (refer to Figure 1). Our methodology facilitates the acquisition of flexible representations, capitalizing on the disentanglement of intricate interactions and dependencies. We formalize our framework by drawing upon the theory of denoising score matching [67] and leveraging tools derived from stochastic differential equations (SDEs) [1]. The effectiveness of Twigs is substantiated through compelling empirical validation across various conventional constrained generation tasks, utilizing both molecular and generic graph datasets.

1.1 Contributions

In summary, this paper makes the following key contributions:

Method

Conditional

Asymmetric

Multiple flows

GDSS [37]

✗

✗

✓

✓

EEGSDE [3]

✓

✗

✗

✓

MOOD[45]

✓

✗

✗

✓

JODO [28]

✓

✗

✗

✓

EDGE [5]

✗

✗

✓

✗

GraphMaker [46]

✓

✗

✓

✗

✓

✗

✗

✗

Gruver et al. [18]

✓

✗

✗

✓

✓

✓

✗

✗

✓

✓

✓

✓

(Conceptual and methodological) The introduction of a new score-based, end-to-end trainable, non-autoregressive generative model Twigs designed for acquiring conditional representations. Our approach enables precise guidance of multiple property-conditioned diffusion processes.

(Technical) We present a robust mathematical framework, including a novel strategy called loop guidance , that employs tools from Stochastic Differential Equations (SDEs) to derive both the forward diffusion process and its corresponding reverse SDE for conditional generation. This framework is designed to seamlessly integrate additional contexts as conditioning information.

(Empirical) We showcase the versatility of the proposed diffusion mechanism ( Twigs ) through extensive empirical evidence across various challenging conditional graph generation tasks, consistently surpassing contemporary baselines.

2 Related works

In Table 1 we provide an overview of the similarities and differences between Twigs and related methods. We refer the reader to Appendix E for additional related work.

Diffusion guidance is typically applied to regulate the diffusion process for conditional generation. Previous approaches that perform class-conditional generation are divided into classifier-based [8], and classifier-free guidance [23]. While some works model diffusion with multiple flows [5, 37, 46], they treat nodes and edges in a symmetric way; i.e., they associate multiple flows for nodes and edges that have equivalent contributions (in other words, these flows have the same roles). We instead abstract graph properties as secondary processes that branch from, and interact with, the main process that pertains to the graph structure. In addition, while other guidance methods are related [18, 40, 52], they do not leverage multiple diffusion flows. To our knowledge, the proposed method is the first to incorporate multiple diffusion flows in a hierarchical fashion for conditional generation. We formalize in Table 2 how Twigs differs, mathematically, from classifier-free and classifier-based methods.

Conditional Diffusion for Graphs Recent advancements in generative modeling have prominently featured score-based techniques (SGM), utilizing diffusion or stochastic differential equations (SDEs) [19, 32, 35, 37, 48], including for graph generation [3, 5, 13, 14, 15, 18, 26, 40, 45, 46, 52, 72, 75, 82]. Guidance methods have been adopted in conditional molecule generation settings. The works from Hoogeboom et al. [26], Huang et al. [28, 28], Xu et al. [82] are classifier-free approaches, while Bao et al. [3], Vignac et al. [75], Lee et al. [45] focus on classifier-based methods. Diverging from these approaches, we explicitly model the dynamic interaction between primary variables (e.g., graph structure) and dependent variables (e.g., graph properties) using dedicated diffusion processes to achieve more expressive representations and improve performance for conditional generation.

3 Diffusion Twigs

Method overview We extend score-based techniques [67] for training conditional diffusion models over graphs. Differently from current guidance methods, as summarised in Table 2, we leverage a finer control over the structure and graph properties to diffuse multiple hierarchical processes, toward achieving a more robust representation. Our method, Twigs , defines a trunk process over the primary

variable (graph structure) y s , and a stem process over each dependent variable y i ∈ R (e.g., graph property). We achieve the desired flexibility with a variable y s that encompasses both node features and the adjacency matrix as well as the coordinates. The details of the dimensions of y s are given in Section B.1 for the 3D case, and in Section B.2 for the 2D case.

Forward process We define multiple forward processes within a hierarchy that co-evolves data and properties into noise. The trunk forward process for the graph structure y s is defined as

where f s and g s are corresponding diffusion and drift functions, and d w is the Wiener noise. The stem forward process over the k dependent variables y = { y 1 , . . . , y k } is defined as

Here, f p and g p denote the diffusion and drift functions, respectively, for the k stem processes. Collectively, along with the trunk forward process, they constitute Twigs . These operations introduce random Gaussian noise, iteratively, to the data toward a prior (typically Gaussian) distribution.

Reverse Process The Twigs reverse process starts from the prior distribution (Gaussian noise) towards the data distribution. A key difference with Song et al. [67] is that here our variable y t comprises both structure and properties , leading to the following modification of the overall diffusion process:

We derive Equation (3) in Section A.1. The joint distribution over the trunk and stem processes is assumed to factorize as

In turn, the score function simplifies as in Equation (5), leading to the decomposition in Equation (6).

Conditional modeling We expand our proposed approach to enable conditional generation with an external context y C = { y c | c ∈ C } , where C ⊆ { 1 , . . . , k } . The context can be represented as a scalar or vector, describing a particular value associated with a data-dependent variable. For example, in case of molecules, it could represent one or more of the k properties such as the Synthetic Accessibility (SA) score or the Quantitative Estimate of Drug likeness (QED). This extension modifies the joint distribution for the score function in Equation (5).

Reverse SDE under conditioning context

The reverse SDE for y t = { y s,t , { y i,t } k } give an external conditioning context y C is shown below (details in Appendix A.2).

We resort to the following factorization of the distribution, conditioned on the context y C :

As a result, the factorization of the score function ∇ y t log p t ( y s,t , { y i,t } k , y C ) amounts to

The above-factorized score function parameterizes our reverse diffusion process, thus offering a novel approach to integrate external contextual information into conditional generation.

Training We propose to train Twigs by incorporating the factorization from Equation (8) within a score-matching objective function [30, 67]. Algorithm 1 shows the training procedure to learn two types of time-dependent score-based models: s θ,t , which approximates the trunk variable, and s ϕ i ,t which approximates the coupling between the stem variable and the trunk variable. The objective function for optimizing the score networks s θ , s ϕ i , is given as follows:

where E y 0 = E y s, 0 , y i, 0 and E y t = E y s,t , y i,t . It is worth noting that the influence introduced by the variable s ϕ i provides the directions for the diffusion model to converge into distributions with the desired properties. Such property-oriented knowledge operates in conjunction with the structural information provided by s θ , resulting in a novel form of guidance that is orchestrated by a branching diffusion process, named Loop guidance .

Sampling Given a trained conditional Twigs model, our generative process begins by sampling an external context or conditioning value y C , which can also be supplied externally. We then simulate the reverse diffusion process, similar to the one described in Equation 8, but with a modified score function to generate the data. The proposed algorithm for generating new data samples with Twigs is given in Algorithm 2 and involves a loop of updates between processes: the stem score network s ϕ i evolves the property y i , integrating information from the structure y s , and subsequently, the updated property information from s ϕ i is integrated into the main process by the score network s θ .

4 Experiments

We conduct a set of comprehensive experiments to demonstrate that Twigs improves over contemporary conditional generation methods. Benchmarks include: molecule generation conditioned over single (§ 4.1), and multiple (§ 4.2) properties on QM9, as well as molecule optimization on ZINC250K (§ 4.3), and network-graph generation conditioned on desired properties (§ 4.4).

KL:3.0e-04

KL:5.8e-04

KL:9.7e-04

KL:5.4e-02

KL:7.2e-02

KL:2.4e-03

KL:1.3e-02

KL:1.7e-02

KL:9.2e-01

KL:4.1e-01

KL:4.4e-02

50

α

75

100

-40

-20

0

-8

EHOMO

-6

-4

2.5

ELUMO

0.0

2.5

0

μ

10

5

10

4.1 Single Quantum properties on QM9

Setup. We evaluate the effectiveness of Twigs for generating molecules with a single desired quantum property, sourced from the QM9 dataset [58], specifically, we consider C v , µ , α , ∆ ϵ , ϵ LUMO and ϵ HOMO . To ensure consistency and comparability with the baselines, which include JODO [28], EDM [26], EEGSDE [3], GeoLDM [82], TEDMol [49], EquiFM [68], we adhere to the identical dataset preprocessing, training/test data partitions, and evaluation metrics outlined by Huang et al. [28]. Regarding parameterization of Twigs , we follow the attention architecture defined in Section B.1 with a single stem process.

Results. In Table 3, we report the Mean Absolute Error (MAE) results, and in Table 4, the Novelty, Atom stability and Molecule stability. Our method outperforms all the evaluated baselines across the specified properties. In Figure 2, the bottom row provides a Kernel Density Estimation (KDE) visualization which shows that Twigs achieves a more accurate distribution for the property values when compared with JODO, while the top row shows some 3D molecule samples by our model.

NH

OH

HO

OH

HO

4.2 Multiple Quantum properties on QM9

Setup. This experiment evaluates the capability to combine multiple desired properties in the generated molecule. Specifically we follow Huang et al. [29] and consider all possible combinations of properties involving µ : ( C v , µ ), ( ∆ ϵ, µ ), ( α, µ ). Since we model two properties, we test our Twigs with two stem networks within the attention architecture described in Section B.1. We benchmark against several contemporary baselines, including EDM [26], EEGSDE [3] and JODO [28].

Results. In Table 5, we present the Mean Absolute Error (MAE) results obtained from the property predictors introduced by Huang et al. [28] for the various property pairs under consideration. The superior performance of Twigs across all baselines reinforces the findings from the single property experiment (Section 4.1), emphasizing the benefits of learning multiple hierarchical stem processes.

QED 0.84

SA 0.62

QED0.69

SA0.90

QED 0.60

SA 0.60

QED0.68

SA 0.50

QED 0.70

SA 0.50

QED 0.53

SA 0.74

DS -5.6

DS -5.9

Ds -7.7

DS -6.9

DS -7.6

DS -7.5

QED0.66

SA 0.59

QED0.80

SA 0.53

QED0.68

SA 0.67

QED 0.75

SA 0.61

QED 0.66

SA 0.54

QED 0.64

SA 0.46

HN

H,N

HN

DS -7.3

DS -7.6

DS -7.3

DS -8.3

DS -7.3

DS -8.8

4.3 Molecule optimization on ZINC250K

Setup. The goal is to generate molecules from the ZINC250K dataset that exhibit optimal binding affinity, drug-likeness, and synthesizability for the following five target proteins: parp1, fa7, 5ht1b, braf, jak2 . We adhere to the evaluation protocol established by Lee et al. [45], which involves generating 3000 molecules and assessing them using two metrics that constrain the desired properties, including docking score (DS), drug-likeness (QED), and synthetic accessibility (SA).

The first metric, Novel hit ratio (%) , represents the fraction of unique hit molecules that have a maximum Tanimoto similarity of less than 0.4 with the training molecules. Hit molecules are defined as those meeting the criteria: DS < (the median DS of the known active molecules), QED > 0.5, and SA < 5. The second metric, Novel top 5% docking score , is the average DS of the top 5% unique molecules that satisfy QED > 0.5 and SA < 5, with a maximum similarity of less than 0.4 to the training molecules.

Baselines. We consider REINVENT [55]: a reinforcement learning (RL) model that utilizes a prior sequence model, MORLD [33]: a RL model that uses QED and SA scores as intermediate rewards and docking scores as final rewards, HierVAE [34]: a VAE-based model that utilizes hierarchical molecular representation and active learning, GDSS [37]: a score-based diffusion model that evolves nodes and edge information with a system of SDEs, MOOD [45]: a score-based diffusion model based on GDSS that trains an additional property predictor to improve conditional generation. For MOOD we consider the version without the out-of-distribution (OOD) control, to have a fair comparison with our method. For Twigs we follow the GCN-based architecture described in Section B.2, with multiple stem processes (one for each target protein).

Results. In Table 6 we report the results for top 5% docking scores. We observe that Twigs achieves the highest score across all properties, excluding braf, where it achieves the second-best score after MOOD. In Table 7 we report the results for Novel hit ratio. The outcomes confirm that our model is improving the performance substantially over all the considered properties, except for braf, on which Twigs is the second-best performing model after MOOD. In Figure 4, we provide some samples of the molecules obtained by Twigs with the respective QED, SA, and docking score. Additionally, in Table 13 we report the MAE values for generating molecules with a desired target protein property, and in Table 14 we compare the inference cost of Twigs against MOOD.

4.4 Generation of Network graphs with desired properties

Setup. We follow the data processing delineated by Jo et al. [37] and provide results for the Community-small [60] and Enzymes datasets [62]. To test the capabilities to generate conditional graphs, we extract four properties via the NetworkX library [20], including density, clustering, assortativity, and transitivity. Considering a graph G with n nodes and m edges, we have: (1) Density: d = 2 m n ( n -1) , (2) Clustering coefficient: the average C = 1 n ∑ v ∈ G c v . (3) Assortativity: measures the similarity of connections in the graph with respect to the node degree. (4) Transitivity: the fraction of all possible triangles present in G . Possible triangles are identified by the number of "triads" (two edges with a shared vertex). The transitivity is T = 3 # triangles # triads .

Baselines. In terms of baselines, we first consider two versions of MOOD [45] (two OOD coefficients), and we train the property predictors using the codes from the authors. Our second baseline is GDSS [37], which we modify to be equipped with a classifier-free guidance scheme. We also consider the version of GDSS based on transformers, which leverages the graph-multi-head attention [2]. Finally, we consider Digress [75], which is a classifier-based guidance diffusion model based on attention mechanisms. We parameterize our Twigs model with our GCN architecture described in Section B.2, with a single stem process.

Community

Enzymes

Density

Clustering

Assortativity

Transitivity

Community

KL:0.001

KL:0.004

KL:4.810

KL:0.006

KL:0.023

0.25

0.30

0.35

0.40

0.2

0.4

0.6

0.8

0.5

0.0

0.5

0.4

0.6

0.8

Enzymes

KL:0.046

KL:0.061

KL:1.437

KL:0.068

KL:0.193

KL:0.242

KL:3.273

KL:0.234

0.0

0.2

0.4

-0.25 0.000.250.500.751.00

-0.50-0.250.000.250.50

0.75

0.00

0.25

0.50

0.75

1.00

TwigSMOODData

Results. Table 8 reports the MAE average of three runs, demonstrating that Twigs consistently outperforms the considered baselines on all cases across the two datasets. MOOD is the second-best performing model in the majority of the cases. We further strengthen the MAE results by providing in Figure 5 (bottom) the KDE plots of the property distributions of the graph generated by Twigs and MOOD. The Figure demonstrates that Twigs can achieve a higher fidelity to the data, which is also confirmed by the lower KL divergence values. Figure 5 (top) depicts some random graph samples generated by Twigs .

4.5 Ablation study on multiple properties

Setup. Assuming conditional independence among the properties α , ϵ HOMO, ϵ LUMO, ∆ ϵ , µ , and C v given the molecular graph can simplify the modeling process. This assumption leverages the fact that the molecular graph captures the essential structural dependencies, allowing us to treat the properties as independent for computational efficiency and ease of interpretation, even if slight interdependencies exist.

Results. Here we show that such modeling assumption can work practically. Table 9 reports the MAE on molecular graphs for QM9 on three properties, showing that our method consistently achieves lower error on all the properties. Table 10 shows that on generic graphs Twigs can achieve lower MAE on all the considered cases, in the cases of two and three properties.

4.6 Training time

In Table 11 we study the impact of multiple diffusion flows on the community-small and Enzymes datasets. Specifically, we report the average time for the overall training for Twigs with one and three secondary diffusion flows. We observe that our models encounter a small overhead compared to GDSS and Digress, however, we believe it is a good tradeoff because it achieves a lower MAE.

5 Conclusion, Broader Implications, and Limitations

We introduced a novel approach to model conditional information within generative models tailored for graph data. Twigs incorporates the novel mechanism of loop guidance to control the overall generative process by first bifurcating the diffusion flow into multiple stem processes and then re-integrating them into the trunk process, resembling a loop. Our experimental results showcase the performance gains of Twigs when compared to current state-of-the-art baselines across various conditional graph generation tasks.

Conditional generation is fast emerging as one of the most exciting avenues within machine learning and would benefit from techniques beyond classifier-based and classifier-free schemes, making our method applicable to settings beyond this work. Indeed, while the current work has focused on graph settings, Twigs might find use in other domains (e.g., image, text, and audio). However, whether Twigs is effective in such settings needs to be investigated in future works.

Training multiple properties (stem processes) might require training additional parameters, incurring additional computation and training time. Our ablation study on training time due to multiple processes (Section 4.6) suggests that Twigs could provide a good tradeoff (lower MAE compared to some prominent existing methods at the expense of small additional computational overhead).

Finally, assuming factorization of the distribution over stem processes conditioned on the trunk process might not always be realistic. Our experiments in Section 4.5 suggest that Twigs might still be able to achieve a strong performance when considering multiple properties. In case some prior knowledge is available about some properties that violate this assumption, we could, in principle, adapt Twigs by grouping them into a single stem process while factorizing with the remaining ones.

Acknowledgments

YV and VG acknowledge support from the Research Council of Finland for the 'Human-steered next-generation machine learning for reviving drug design' project (grant decision 342077). VG also acknowledges Jane and Aatos Erkko Foundation (grant 7001703) for 'Biodesign: Use of artificial intelligence in enzyme design for synthetic biology'. GM acknowledges support from the Engineering and Physical Sciences Research Council (EPSRC) and the BBC under iCASE. AF is partially funded by the CRUK National Biomarker Centre, by the Manchester Experimental Cancer Medicine Centre and the NIHR Manchester Biomedical Research Centre.

References

Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications , 12(3):313-326, 1982. ISSN 0304-4149. doi: https://doi.org/10.1016/ 0304-4149(82)90051-5. URL https://www.sciencedirect.com/science/article/ pii/0304414982900515 .

Jinheon Baek, Minki Kang, and Sung Ju Hwang. Accurate learning of graph representations with graph multiset pooling. In International Conference on Learning Representations , 2021. URL https://openreview.net/forum?id=JHcqXGaqiGn .

Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, and Jun Zhu. Equivariant energy-guided SDE for inverse molecular design. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=r0otLtOwYW .

Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. arXiv preprint arXiv:2302.08113 , 2023.

Xiaohui Chen, Jiaxing He, Xu Han, and Liping Liu. Efficient and degree-guided graph generation via discrete diffusion modeling. In International Conference on Machine Learning , pages 4585-4610. PMLR, 2023.

Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking. arXiv preprint arXiv:2210.01776 , 2022.

Alex O Davies, Nirav S Ajmeri, et al. Hierarchical gnns for large graph generation. arXiv preprint arXiv:2306.11412 , 2023.

Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021.

Anh-Dung Dinh, Daochang Liu, and Chang Xu. Rethinking conditional diffusion sampling with progressive guidance. In Thirty-seventh Conference on Neural Information Processing Systems , 2023. URL https://openreview.net/forum?id=gThGBHhqcU .

Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc. In International Conference on Machine Learning , pages 8489-8510. PMLR, 2023.

Alexandru Dumitrescu, Dani Korpela, Markus Heinonen, Yogesh Verma, Valerii Iakovlev, Vikas Garg, and Harri Lähdesmäki. Field-based molecule generation. arXiv preprint arXiv:2402.15864 , 2024.

Peter Eckmann, Kunyang Sun, Bo Zhao, Mudong Feng, Michael K Gilson, and Rose Yu. Limo: Latent inceptionism for targeted molecule generation. arXiv preprint arXiv:2206.09010 , 2022.

Niklas Gebauer, Michael Gastegger, and Kristof Schütt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. Advances in neural information processing systems , 32, 2019.

Niklas WA Gebauer, Michael Gastegger, Stefaan SP Hessmann, Klaus-Robert Müller, and Kristof T Schütt. Inverse design of 3d molecular structures with conditional generative neural networks. Nature communications , 13(1):973, 2022.

Zijie Geng, Shufang Xie, Yingce Xia, Lijun Wu, Tao Qin, Jie Wang, Yongdong Zhang, Feng Wu, and Tie-Yan Liu. De novo molecular generation via connection-aware motif mining. In International Conference on Learning Representations , 2023. URL https://openreview. net/forum?id=Q_Jexl8-qDi .

Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science , 4(2):268-276, 2018.

Alexandros Graikos, Nikolay Malkin, Nebojsa Jojic, and Dimitris Samaras. Diffusion models as plug-and-play priors. Advances in Neural Information Processing Systems , 35:14715-14728, 2022.

Nate Gruver, Samuel Stanton, Nathan Frey, Tim GJ Rudner, Isidro Hotzel, Julien LafranceVanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G Wilson. Protein design with guided discrete diffusion. Advances in neural information processing systems , 36, 2023.

Florentin Guth, Simon Coste, Valentin De Bortoli, and Stephane Mallat. Wavelet score-based generative modeling, 2022.

Aric Hagberg, Pieter Swart, and Daniel S Chult. Exploring network structure, dynamics, and function using networkx. Technical report, Los Alamos National Lab.(LANL), Los Alamos, NM(United States), 2008.

Philip J Hajduk and Jonathan Greer. A decade of fragment-based drug design: strategic advances and lessons learned. Nature reviews Drug discovery , 6(3):211-219, 2007.

Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J Zico Kolter, Ruslan Salakhutdinov, et al. Manifold preserving guided diffusion. arXiv preprint arXiv:2311.16424 , 2023.

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. arXiv preprint arxiv:2006.11239 , 2020.

Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. The Journal of Machine Learning Research , 23(1):2249-2281, 2022.

Emiel Hoogeboom, Vıctor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning , pages 8867-8887. PMLR, 2022.

Han Huang, Leilei Sun, Bowen Du, Yanjie Fu, and Weifeng Lv. Graphgdp: Generative diffusion processes for permutation invariant graph generation. In 2022 IEEE International Conference on Data Mining (ICDM) , pages 201-210. IEEE, 2022.

Han Huang, Leilei Sun, Bowen Du, and Weifeng Lv. Learning joint 2d & 3d diffusion models for complete molecule generation. arXiv preprint arXiv:2305.12347 , 2023.

Yinan Huang, Xingang Peng, Jianzhu Ma, and Muhan Zhang. 3dlinker: an e (3) equivariant variational autoencoder for molecular linker design. arXiv preprint arXiv:2205.07309 , 2022.

Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research , 6(4), 2005.

John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph-based protein design. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/ paper/2019/file/f3a4ff4839c56a5f460c88cce3666a2b-Paper.pdf .

John Ingraham, Max Baranov, Zak Costello, Vincent Frappier, Ahmed Ismail, Shan Tie, Wujie Wang, Vincent Xue, Fritz Obermeyer, Andrew Beam, et al. Illuminating protein space with a programmable generative model. BioRxiv , pages 2022-12, 2022.

Woosung Jeon and Dongsup Kim. Autonomous molecule generation using reinforcement learning and docking to develop potential novel inhibitors. Scientific reports , 10(1):22104, 2020.

Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In International conference on machine learning , pages 48394848. PMLR, 2020.

Bowen Jing, Gabriele Corso, Renato Berlinghieri, and Tommi Jaakkola. Subspace diffusion generative models. In Lecture Notes in Computer Science , pages 274-289. Springer Nature Switzerland, 2022. doi: 10.1007/978-3-031-20050-2_17. URL https://doi.org/10.1007% 2F978-3-031-20050-2_17 .

Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. Advances in Neural Information Processing Systems , 35:24240-24253, 2022.

Jaehyeong Jo, Seul Lee, and Sung Ju Hwang. Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference on Machine Learning , pages 10362-10383. PMLR, 2022.

Jaehyeong Jo, Dongki Kim, and Sung Ju Hwang. Graph generation with destination-driven diffusion mixture. arXiv preprint arXiv:2302.03596 , 2023.

Kisuk Kang, Ying Shirley Meng, Julien Breger, Clare P Grey, and Gerbrand Ceder. Electrodes with high power and high capacity for rechargeable lithium batteries. Science , 311(5763): 977-980, 2006.

Leo Klarner, Tim GJ Rudner, Garrett M Morris, Charlotte M Deane, and Yee Whye Teh. Context-guided diffusion for out-of-distribution molecular and protein design. arXiv preprint arXiv:2407.11942 , 2024.

Lingkai Kong, Jiaming Cui, Haotian Sun, Yuchen Zhuang, B Aditya Prakash, and Chao Zhang. Autoregressive diffusion model for graph generation. In International Conference on Machine Learning , pages 17391-17408. PMLR, 2023.

Xiangzhe Kong, Wenbing Huang, Zhixing Tan, and Yang Liu. Molecule generation by principal subgraph mining and assembling. Advances in Neural Information Processing Systems , 35: 2550-2563, 2022.

Najwa Laabid, Severi Rissanen, Markus Heinonen, Arno Solin, and Vikas Garg. Alignment is key for applying diffusion models to retrosynthesis. arXiv preprint arXiv:2405.17656 , 2024.

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature , 521(7553):436-444, 2015.

Seul Lee, Jaehyeong Jo, and Sung Ju Hwang. Exploring chemical space with score-based out-of-distribution generation. Proceedings of the 40th International Conference on Machine Learning , 2023.

Mufei Li, Eleonora Kreaˇ ci´ c, Vamsi K Potluru, and Pan Li. Graphmaker: Can diffusion models generate large attributed graphs? arXiv preprint arXiv:2310.13833 , 2023.

Wenhao Li, Xiangfeng Wang, Bo Jin, and Hongyuan Zha. Hierarchical diffusion for offline decision making. In International Conference on Machine Learning , pages 20035-20064. PMLR, 2023.

Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. Graphebm: Molecular graph generation with energy-based models. arXiv preprint arXiv:2102.00546 , 2021.

Yanchen Luo, Sihang Li, Zhiyuan Liu, Jiancan Wu, Zhengyi Yang, Xiangnan He, Xiang Wang, and Qi Tian. Text-guided diffusion model for 3d molecule generation, 2024. URL https://openreview.net/forum?id=FdUloEgBSE .

Lukasz Maziarka, Agnieszka Pocha, Jan Kaczmarczyk, Krzysztof Rataj, Tomasz Danel, and Michał Warchoł. Mol-cyclegan: a generative model for molecular optimization. Journal of Cheminformatics , 12(1):1-18, 2020.

Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073 , 2021.

Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, and Jennifer Listgarten. Unlocking guidance for discrete state-space diffusion and flow models. arXiv preprint arXiv:2406.01572 , 2024.

Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Permutation invariant graph generation via score-based generative modeling. In International Conference on Artificial Intelligence and Statistics , pages 4474-4484. PMLR, 2020.

Juhwan Noh, Dae-Woong Jeong, Kiyoung Kim, Sehui Han, Moontae Lee, Honglak Lee, and Yousung Jung. Path-aware and structure-preserving generation of synthetically accessible molecules. In International Conference on Machine Learning , pages 16952-16968. PMLR, 2022.

Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics , 9(1):1-14, 2017.

Yidong Ouyang, Liyan Xie, and Guang Cheng. Improving adversarial robustness through the contrastive-guided diffusion process. In International Conference on Machine Learning , pages 26699-26723. PMLR, 2023.

Bo Qiang, Yuxuan Song, Minkai Xu, Jingjing Gong, Bowen Gao, Hao Zhou, Wei-Ying Ma, and Yanyan Lan. Coarse-to-fine: a hierarchical diffusion model for molecule generation in 3d. In International Conference on Machine Learning , pages 28277-28299. PMLR, 2023.

Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data , 1, 2014.

Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, et al. Structure-based drug design with equivariant diffusion models. arXiv preprint arXiv:2210.13695 , 2022.

Ida Schomburg, Antje Chang, Christian Ebeling, Marion Gremse, Christian Heldt, Gregor Huhn, and Dietmar Schomburg. Brenda, the enzyme database: updates and major new developments. Nucleic acids research , 32(suppl_1):D431-D433, 2004.

Daniel Schwalbe-Koda and Rafael Gómez-Bombarelli. Generative models for automatic chemical design. Machine Learning Meets Quantum Physics , pages 445-467, 2020.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina EliassiRad. Collective classification in network data. AI magazine , 29(3):93-93, 2008.

Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. In International conference on machine learning , pages 9558-9568. PMLR, 2021.

Gregory Sliwoski, Sandeepkumar Kothiwale, Jens Meiler, and Edward W Lowe. Computational methods in drug discovery. Pharmacological reviews , 66(1):334-395, 2014.

Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=9_gsMA8MRKQ .

Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming-Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning , pages 32483-32498. PMLR, 2023.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations , 2021.

Yuxuan Song, Jingjing Gong, Minkai Xu, Ziyao Cao, Yanyan Lan, Stefano Ermon, Hao Zhou, and Wei-Ying Ma. Equivariant flow matching with hybrid probability transport for 3d molecule generation. In Thirty-seventh Conference on Neural Information Processing Systems , 2023. URL https://openreview.net/forum?id=hHUZ5V9XFu .

Brian L Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. arXiv preprint arXiv:2206.04119 , 2022.

Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motifscaffolding problem. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=6TxBxqNME1Y .

Alex M Tseng, Tommaso Biancalani, Max Shen, and Gabriele Scalia. Hierarchically branched diffusion models for efficient and interpretable multi-class conditional generation. arXiv preprint arXiv:2212.10777 , 2022.

Alex M Tseng, Nathaniel Diamant, Tommaso Biancalani, and Gabriele Scalia. Graphguide: interpretable and controllable conditional graph generation with discrete bernoulli diffusion. arXiv preprint arXiv:2302.03790 , 2023.

Yogesh Verma, Samuel Kaski, Markus Heinonen, and Vikas Garg. Modular flows: Differential molecular generation. In Advances in Neural Information Processing Systems , volume 35, pages 12409-12421. Curran Associates, Inc., 2022.

Yogesh Verma, Markus Heinonen, and Vikas Garg. Abode: Ab initio antibody design using conjoined odes. In International Conference on Machine Learning , pages 35037-35050. PMLR, 2023.

Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. Digress: Discrete denoising diffusion for graph generation. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/ forum?id=UaAD-Nu86WX .

Wujie Wang, Minkai Xu, Chen Cai, Benjamin Kurt Miller, Tess Smidt, Yusu Wang, Jian Tang, and Rafael Gómez-Bombarelli. Generative coarse-graining of molecular conformations. arXiv preprint arXiv:2201.12176 , 2022.

Zichao Wang, Weili Nie, Zhuoran Qiao, Chaowei Xiao, Richard Baraniuk, and Anima Anandkumar. Retrieval-based controllable molecule generation. arXiv preprint arXiv:2208.11126 , 2022.

Fang Wu and Stan Z Li. Diffmd: a geometric diffusion model for molecular dynamics simulations. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 5321-5329, 2023.

Kevin Eric Wu, Kevin K Yang, Rianne van den Berg, James Zou, Alex Xijie Lu, and Ava P Amini. Protein structure generation via folding diffusion, 2023. URL https://openreview. net/forum?id=Nkd7AS2USRd .

Lemeng Wu, Chengyue Gong, Xingchao Liu, Mao Ye, and Qiang Liu. Diffusion-based molecule generation with informative prior bridges. Advances in Neural Information Processing Systems , 35:36533-36545, 2022.

Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations , 2022. URL https://openreview.net/forum?id= PzcvxEMzvQC .

Minkai Xu, Alexander S Powers, Ron O Dror, Stefano Ermon, and Jure Leskovec. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning , pages 38592-38610. PMLR, 2023.

A Proofs

A.1 Derivation of the reverse SDE

For a Stochastic Differential Equation (SDE) of the form,

where f ( · ) and g ( · ) are diffusion, drift function and d w is the weiner noise. The evolution of the distribution of x t is governed by the Kolmogorov Forward Equation (KFE) as,

Kolmogrov Forward/Backward Equation (KFE/KBE). Essentially KFE describes the evolution of a probability distribution p ( x t ) forward in time. The reverse-time SDE can be derived by solving the Kolmogorov Backward Equation (K.B.E) as derived in Anderson [1]. It can be defined for t 1 ≥ t 0 as,

where x t 0 and x t 1 are distributions at the respective time steps. Specifically, it models how the distribution dynamics at a later point t 1 in time changes as we change t 0 at an earlier time.

In our case, we consider the diffusion over structure y s and properties { y 1 , . . . , y k } . The KFE of the system y = { y s , y 1 , . . . , y k } is given by,

Independence Factorization. We can factorize p ( y t ) based on our assumption that the properties { y 1 ,t , . . . , y k,t } are independent conditioned on the structure y s,t as

Leveraging this factorization, we can define a system of SDEs with KFEs for each variable, leading us to the SDE system defined in Eq. 1 and Eq. 2.

Reverse SDE: In the reverse case, we aim to denoise the full vector y = { y s , y 1 , . . . , y k } where y s denotes the diffusion over structure and { y 1 , . . . , y k } over the k properties via reverse SDE. Expressing in the form of Eq. 12, we note that for t 1 ≥ t 0 ,

Anderson [1] defines a joint distribution over the time-ordered variables y t 1 and y t 0 to derive the reverse SDE. We utilize their analysis and define a joint distribution

We denote p ( y s,t 0 , y 1 ,t 0 , . . . , y k,t 0 ) by p ( y t 0 ) , and note that it can be decomposed similarly as in Eq. 14. Taking the time derivative of Eq. 16, we get

Comparison with KFE/KBE. We observe that ∂ t p ( y s,t 1 , y 1 ,t 1 , . . . , y k,t 1 | y s,t 0 , y 1 ,t 0 , . . . , y k,t 0 ) corresponds to the KBE in Eq. 15 and ∂ t p ( y t 0 ) to the KFE in Eq. 13. Denoting

The derivatives can be handled, by following standard differentiation rules as,

Evaluating the derivative of the products in the forward Kolmogorov equation and substituting the derivatives accordingly we obtain,

Matching the terms of the second-order derivatives with the expansion of the derivative and doing some algebraic manipulations, we obtain

which can be written as

Comparison with KFE. The above result is in the form of a Kolmogorov forward equation with the joint probability distribution p ( y t 1 , y t 0 ) . The time-ordering is t 1 > t 0 and the term -∂ t p ( y t 1 , y t 0 ) describes the change of probability distribution as we move backward in time. We can marginalize over t 1 , using the Leibniz rule, to obtain

This finally gives a stochastic differential equation analogous to the Fokker-Planck/forward Kolmogorov equation that can be solved backward in time:

We keep g 2 ( y t 0 ) independent of y t 0 . Applying the log-derivative trick, the SDE simplifies to

A.2 Conditional score factorization

We extend our method to incorporate an external context or conditional information for conditional generation, similar to classifier-based [8] and classifier-free [23] guidance. Following similar notation, the reverse SDE [67], given an external context y C can be written as

Here y t = { y s,t , y 1 ,t , . . . , y k,t } , and y C = { y c | c ∈ C } is an external context or conditioning variable. This external context can be a scalar or vector describing a property value of the primary variable like QED or plogp in the case of molecules or image labels in the case of images. The ∇ y t log p t ( y t , y C ) term pertains to the score function which guides the process (see table 2 for comparison with both classifier-based and classifier-free guidance). Under our condition independence assumption, the score function factorizes as

B Parameterizations

Here we describe two instances of Twigs based on architecture choices: Attention networks, and graph convolution networks (GCNs). Twigs with attention is used in 4.1 and 4.2, while Twigs with GCNs is used in 4.3 and 4.4.

B.1 Twigs with graph attention

We denote the variable y s as a 3D graph G = ( A , x , h ) , with node coordinates x = ( x 1 , . . . , x N ) ∈ R N × 3 , node features h = ( h 1 , . . . , h N ) ∈ R N × d 1 , and edge information A ∈ R N × N × d 2 . The variable C ∈ R denotes the conditional information, which is obtained by adding the noise level log( α 2 t /σ 2 t ) , the perturbed property y i ∼ N (0 , I ) ∈ R , and the fixed property y C ∈ R . The context C is combined with y s by multilayer perceptions (MLP), after projecting ( h, A , x ) respectively into H , E , P :

where MHA is the multi-head attention, and AdaLN is Adaptive LayerNorm (LN) function. Subsequently, we leverage the Scale function Scale(h , C ) = MLP( C ) · h , and the Feed Forward Network ( FFN ) to obtain the Diffusion Graph Transformer (DGT) block, as defined in [28], which is described by Eq (31)(32). DGT first computes the intermediate representations for the l -th layer as:

then computes the l +1 layer as:

̸

The Twigs trunk process s θ is parameterized as:

where PDGT i resembles the stem process networks s ϕ i , which is obtained by pooling to a onedimensional variable by an MLP operation, over the output of the DGT block. To optimize Eq (9), DGT minimizes the denoising score matching objective from [28] for node, edge and position information ( h, A , x ), while PDGT i for the perturbed property y i .

B.2 Twigs with graph convolutions

In the case of 2D graphs with N nodes we consider the variable y s = ( X , A ) ∈ R N × F × R N × N , where F is the dimension of the node features, X ∈ R N × F are node features, A ∈ R N × N is weighted adjacency matrix. We define the perturbed property y i ∈ R and the (fixed) property y C ∈ R . The stem process network s ϕ i is given as:

where v i and v C are vectors obtained by repeating N times the perturbed property y i and the fixed property y C respectively, and concatenating them into the node features matrix X . The Twigs trunk process s θ is obtained by combining the contributions from the properties y i derived by the stem processes s ϕ i and the structure y s , as follows:

where s θ X is a conditional node feature score network: s θ X = MLP( GNN ( X ∥ y C , A )) . Finally, following [37], A is co-evolved together with the node features, by the adjacency score model s A θ

where GMH is graph multi-head attention [2], which employs higher-order adjacency matrices A p t , and K denotes the number of GMH layers. The optimization for the Twigs objective function (9), is obtained by minimizing the denoising score matching for A , X , P i .

The GMH block employs higher-order adjacency matrices A p t to represent the long-range dependencies and is provided as: s θ A ( G t ) = MLP ([ { GMH ( H i , A p t ) } K,P i =0 ,p =1 ]) .

C Additional experimental results

C.1 QM9 dataset

Further details for generation conditioned on quantum properties from Section 4.1.

Molecular quality. Additional results for molecular stability in 2D and Fréchet ChemNet Distance (FCD) for 2D and 3D are given in Table 12.

C.2 ZINC250K dataset

Conditional generation. The evaluation is performed by measuring the MAE of the pre-trained predictors released from [45], which given a molecule G t are trained to predict

where ̂ DS is the normalized docking score (DS) of the considered target protein, QED is the druglikeness, and ̂ SA is the normalized synthetic accessibility (SA).

In terms of baselines, we consider the MOOD model [45], which leverages a classifier-based guidance scheme, and we also implement a diffusion guidance version of GDSS [37] based on the classifier-free scheme. Our Twigs method is parameterized by the architecture described in B.2, with a single stem process. The models are conditioned on the function in Equation (37).

Results. In Table 13, we report the mean MAE values over multiple runs computed from the generated molecules using the pre-trained classifiers from [45]. We can observe that the Twigs consistently achieves a lower error, demonstrating an improved control over generating molecules with the desired target proteins.

Runtime. We have incorporated the runtime for molecule generation at inference time for a largescale dataset (ZINC250K) as for Section 4.3, in Table 14. A comparison with MOOD [45] indicates that our model incurs a certain overhead, as anticipated. However, it demonstrates improved alignment when generating conditional molecules.

D Experimental details

D.1 Computational resources

All experiments are performed with GPUs, Nvidia A100 or v100.

D.2 Models details

We follow the data splits from Huang et al. [28] for 4.1, 4.2, the ones from Lee et al. [45] for 4.3, and the data splits from Jo et al. [37] for 4.4. We use Adam optimizers on all experiments.

For Sections 4.1 and 4.2 we follow the same hyperparameters from Huang et al. [28]. For Section 4.3 we follow the hyperparameters from Lee et al. [45], for the MOOD baseline, we explore OOD coefficients between 0 . 01 and 0 . 09 . For Section 4.4 we follow the hyperparameters from Jo et al. [37].

E Additional Related Works

This section extends the discussion presented in Section 2 by exploring additional related works in the field. In Table 15 we summarise related methods including score-sdes, hierarchical models (not necessarily conditional), and hierarchical conditional models.

Conditional molecular diffusion. Guidance techniques have also been adopted in conditional molecule generation settings: in the context of classifier-free approaches, Hoogeboom et al. [26] proposes an equivariant approach based on DDPM for 3D molecules; Huang et al. [28] explores attention mechanisms within SGM models; and Xu et al. [82] investigates DDPMs in latent space settings.

In terms of classifier-based guidance, Bao et al. [3] incorporate energy guidance into a diffusion model by leveraging a stochastic differential equation; Vignac et al. [75] provide a DDPM coupled with a classifier over quantum molecular properties; and Lee et al. [45] operate over a pre-trained SGM and train an additional predictor for fine-tuning the desired protein target properties.

Guidance methods. Recent works utilize multiple diffusion processes: cascaded diffusion [25], provides a flow for each resolution, and GDSS [37] has a joint system of diffusion processes one for nodes and the other for edge features, but it does not cover mechanisms for conditional generation. Tseng et al. [71] define a hierarchy of branching points within a single diffusion flow.

Other Diffusion methods for Graphs. Other works related to ours focus on hierarchical diffusion processes [7], diffusion applied to protein backbones [69], geometry-based models [59, 82], and autoregressive models [41]. In the realm of stochastic differential equation (SDE)-based approaches, the literature includes bridge methods [38], permutation invariance [27], torsional modeling [36], and docking [6]. Additionally, [63] introduces the ConfGF approach, estimating gradient fields of atomic coordinates, while [80] proposes a method steering the training of diffusion-based generative models using physical and statistical prior information.

Autoencoder-Based graph models. This category includes works employing autoencoders, such as retrieval-based models [77, 12], scaffold modeling [50], link design [29], and coarse-grain modeling [76]. Notably, [54] proposes a reaction-embedded and structure-conditioned variational autoencoder, while [42] defines the concept of principal subgraphs, relevant to informative patterns within molecules.

Conditional Diffusion. In the realm of diffusion generative models, several noteworthy approaches have been developed to enhance their performance and versatility. Du et al. [10] introduce an energy-based parameterization of diffusion models, allowing the integration of novel compositional operators and Metropolis-corrected samplers. Building on this, He et al. [22] contribute a training-free conditional generation framework, leveraging pretrained diffusion models focusing on the manifold hypothesis to refine guided diffusion steps and introduce a shortcut algorithm. Meanwhile, Meng et al. [51] employ a stochastic differential equation (SDE) in synthesizing realistic images, iterating through denoising steps guided by a pretrained diffusion model.

In a different vein, Song et al. [66] propose guiding denoising diffusion models with general differentiable loss functions in a plug-and-play manner, facilitating controllable generation without additional training. Addressing the challenge of inferring high-dimensional data within the context of diffusion models, Graikos et al. [17] present a model consisting of a prior and an auxiliary differentiable constraint. Dinh et al. [9] tackle diversity and adversarial effects in classifier guidance for diffusion generative models by allowing relevant classes' gradients to contribute to shared information construction during noisy early sampling steps. Furthermore, Song et al. [65] put forth a method for estimating conditional scores without additional training. Lastly, Ouyang et al. [56] propose the Contrastive-Guided Diffusion Process (Contrastive-DP), integrating contrastive loss to guide the diffusion model in data generation. These diverse contributions collectively advance the field by addressing various challenges and expanding the capabilities of diffusion generative models.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We demonstrate with theoretical results and a comprehensive set of experiments.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitations provided in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Proofs provided in Section A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: In Section 4 we provide details to reproduce the results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

The implementation details are in appendix to run the experiments. The used datasets are public and can be accessed with the reference paper.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Described in D.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report mean and standard deviation in our experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Described in D.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We conform with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Provided in Section 5.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Does not apply for our paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The resources that we used are cited, the source code we used is released on open licenses.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve Crowdsourcing.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve IRB.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
The paper presents Twigs, a score-based diffusion framework for enhancing conditional generation tasks. It includes a central trunk diffusion process for graph structure and stem processes for graph properties. The innovative loop guidance strategy manages information flow between trunk and stem processes. Experiments show Twigs significantly outperforms contemporary baselines, especially in tasks like inverse molecular design.

STRENGTHS:
1. A new guidance strategy that explicitly models the coupling effects between condition and data.
2. Impressive experimental results.

WEAKNESSES:
1. The main drawback of the proposed method is the limited flexibility. The loop guidance requires the joint training of $s_{\theta}$ and $s_{\phi}$ and it cannot provide guidance for unseen conditions. 
2. Some of the experimental settings and evaluation pipeline are unclear. Please at lease specify them in the appendix.
3. Minor: Please put back the guidelines of paper checklist.

QUESTIONS:
1. How will the sampling step size affect the generation? Can we use different step sizes when sampling $y_s$ and $y_i$?
2. Classifier guidance helps the diffusion model to generalize to new conditions. Can Twigs provide guidance of unseen conditions by, for example, separating the training of $s_{\theta}$ and $s_{\phi}$?
3. How will the incorporation of multiple $s_{\phi}$ influence the total training time of Twigs? Can you provide the training time comparison as well?

LIMITATIONS:
The limitations mentioned in the manuscript are more like future direction of applying the similar strategy to other domains. Please provide more discussion if possible.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study proposed a conditional generative model with guided diffusion. The authors introduce a new mechanism called loop guidance to include conditions. Empirical analysis includes small molecule generation on a diverse of datasets and properties, with both quantitative evaluation and visualizations.\n\nSTRENGTHS:\n1. The paper presents a well-structured framework for conditional generation with guided diffusion, including a neat summary of relevant methods in this field.\n2. The presentation of the methods and results is clear and easy to follow.\n3. The authors provide extensive experimental comparisons, offering a thorough evaluation of the proposed method against various baselines.\n\nWEAKNESSES:\nThe weaknesses mainly arise from the insufficient empirical evidence:\n\n1. No comparison has been provided regarding the computational cost, making it unclear how efficient the proposed method is relative to others.\n2. Although the authors claimed that conditional generation is a fast-developing research field, the baseline methods used for comparison are generally from before 2023, which may not reflect the current SOTA.\n3. No ablation study or hyperparameter selection/analysis has been reported. \n4. Some terminologies could be introduced and analyzed more carefully (see question 1 below).\n\nQUESTIONS:\n1. What are the meanings of the properties $C_v$, $\\mu$, $\\alpha$, $\\Delta\\epsilon$, $\\epsilon_{\\text{LUMO}}$, and $\\epsilon_{\\text{HOMO}}$? Why are they important impact factors for molecule generation? Are they practically easy to obtain? Since these properties are used as guides when generating small molecules, do the generated molecules actually have properties close to the guide? \n2. What is the relationship and difference between the proposed method and other recent works on guided diffusion, such as https://arxiv.org/pdf/2406.01572, https://arxiv.org/abs/2305.20009, and https://openreview.net/pdf?id=8NfHmzo0Op? (Note: I'm not requiring a quantitative comparison, especially for the first one, which was uploaded after the conference submission deadline. However, for the latter two papers, as they have been published for more than 3 months, I believe they should be included in the paper.)\n3. How should the top row of Figure 2 be interpreted? Are they different samples from the same guidance, or are they the same sample shown from different views?\n4. Similarly, it is not clear how to interpret the results in Figure 3 (and this figure was not referenced in the main text). Are all the samples generated based on the same input properties? How can the legitimacy of these generations be validated? What does the ground-truth molecule look like? To what level they are better than the generation performance of the baseline methods?\n\nLIMITATIONS:\nThe authors listed one limitation of the current work as not generalizing the framework to other types of data (text, image). No additional analysis on the broader impact has been provided.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Twigs, a new score-based diffusion model for graph generation conditioned on graph-level properties. It employs two diffusion processes, one for graph data (trunk process) and one for graph-level properties (stem process). The underlying generation process corresponds to a factorization of the joint distribution into the unconditional distribution of the graph data and conditionally independent distributions of graph properties given the graph data. Empirical studies across diverse benchmarks demonstrate the effectiveness of the proposed approach.\n\nSTRENGTHS:\n**S1:** The overall presentation is clear and easy to follow.\n\n**S2:** Extensive empirical studies demonstrate the effectiveness of the method for conditional molecule and graph structure generation.\n\nWEAKNESSES:\n**W1**: The idea of hierarchical (conditional) diffusion model for graph generation has been explored in previous works. For example, GDSS [1] and EDGE [2] explore the factorization of the joint distribution of graph structure and node attributes for unconditional molecule generation. GraphMaker [3] explores this idea for node-label-conditioned generation of large attributed graphs.\n\n**W2**: The assumption of the conditional independence of the multiple graph properties may be too strong.\n\n**W3**: The empirical studies performed consider at most modeling two graph properties at a time and there is a lack of understanding in how the model performance changes as more graph properties are modeled simultaneously. In addition, there is a lack of understanding in how the model performance varies as the properties get more correlated.\n\n**W4**: Some presentation details can be further clarified. For example:\n- The terms \"hierarchical modeling\" and \"hierarchical conditional diffusion\" are used without a formal definition in table 1.\n- I guess $y_{s,t}$ between L110 and 111 refers to $y_{s}$ at time $t$, but it is not formally defined.\n- L106 says that $y_s$ is the primary variable (graph structure) and takes shape $\\mathbb{R}^{N\\times D}$. Does this consist of both adjacency matrix and node features like atom types?\n\n[1] Jo et al. Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations.\n\n[2] Chen et al. Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling.\n\n[3] Li et al. GraphMaker: Can Diffusion Models Generate Large Attributed Graphs?\n\nQUESTIONS:\nIn addition to the issues mentioned in the \"Weaknesses\" section, \n\n**Q1**: Why did you not include the results of EDM and TEDMol in table 4, as reported in [1]?\n\n[1] Luo et al. Text-guided Diffusion Model for 3D Molecule Generation.\n\nLIMITATIONS:\nThe paper assumes conditional independence between multiple graph properties, which does not necessarily hold in practice.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

comments: SUMMARY:
This paper proposes a novel score-based diffusion framework called Twigs that incorporates multiple co-evolving flows to capture complex interactions and dependencies for enriching conditional generation tasks. It consists of a central trunk process and additional stem processes, coordinated by a loop guidance strategy during sampling. Extensive experiments on conditional graph generation demonstrate Twigs' strong performance gains over baselines, highlighting its potential for challenging generative tasks like inverse molecular design.

STRENGTHS:
1. Based on my knowledge, the idea of incorporating multiple co-evolving diffusion processes proposed in this paper is very novel, especially in the field of graph generation.
2. The paper is well written and provides sufficient background information, such as Table 2, to help understand the differences between the method proposed in this paper and other classifier-based and classifier-free methods.
3. The experiments are very comprehensive and thorough, and the experimental results fully demonstrate the outstanding performance of the proposed method.

WEAKNESSES:
1. The content in Section 3 is written too generally and does not address the specific characteristics of graph data very well. Additionally, the mathematical symbols used appear somewhat disorganized.
2. I am a little bit confused about the the dimension graph structure $y_s \in \mathbb{R}^{N\times D}$ on top of Eq. (1). Does $D$ covers a lot more information, e.g. node coordinates (dim 3), node features (dim $d_1$), and  edge information (dim $N \times d_2$), as indicated in Appendix B.1?

QUESTIONS:
1. Is the external context $y_C$ one of the $k$ dependent graph properties $y_k$ in Eq. 2?

LIMITATIONS:
Not applied.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12