Graph Diffusion Transformers for Multi-Conditional Molecular Generation
Authors: Gang Liu
Jiaxin Xu
Tengfei Luo
Meng Jiang
Venue: NeurIPS 2024 oral (2024)
Keywords: Graph Diffusion Transformers
Inverse Molecular Design
Molecular Generation
Polymer Generation
Primary area: machine_learning_for_physical_sciences
TL;DR: We introduce Graph Diffusion Transformers for multi-conditional polymer and small molecule inverse design. It offers predictor-free guidance by learning the representations of categorical and numerical properties, enabling accurate denoising.

Abstract: Inverse molecular design with diffusion models holds great potential for advancements in material and drug discovery. Despite success in unconditional molecule generation, integrating multiple properties such as synthetic score and gas permeability as condition constraints into diffusion models remains unexplored. We present the Graph Diffusion Transformer (Graph DiT) for multi-conditional molecular generation. Graph DiT has a condition encoder to learn the representation of numerical and categorical properties and utilizes a Transformer-based graph denoiser to achieve molecular graph denoising under conditions. Unlike previous graph diffusion models that add noise separately on the atoms and bonds in the forward diffusion process, we propose a graph-dependent noise model for training Graph DiT, designed to accurately estimate graph-related noise in molecules. We extensively validate the Graph DiT for multi-conditional polymer and small molecule generation. Results demonstrate our superiority across metrics from distribution learning to condition control for molecular properties. A polymer inverse design task for gas separation with feedback from domain experts further demonstrates its practical utility. The code is available at https://github.com/liugangcode/Graph-DiT.

Graph Diffusion Transformers for Multi-Condition Molecular Generation
Gang Liu, Jiaxin Xu, Tengfei Luo, Meng Jiang University of Notre Dame (gliu7, jxu24, tluo, mjiang2) @nd edu
Abstract
Inverse molecular design with diffusion models holds great potential for advance- ments in material and drug discovery. Despite success in unconditional molecular generation, integrating multiple properties such as synthetic score and gas per- meability condition constraints into diffusion models remains unexplored. We present the Graph Diffusion Transformer (Graph DiT) for multi-conditional molec- ular generation. Graph DiT integrates encoder learn numerical and categorical property representations with the Transformer- graph diffusion models that add noise separately the atoms and bonds in the forward diffusion process, Graph DiT is trained with a novel graph-dependent noise model for accurate estimation of graph-related noise in molecules. We ex- tensively validate Graph DiT for multi-conditional polymer and small molecule generation. Results demonstrate the superiority of Graph DiT across nine metrics from distribution learning to condition control for molecular properties. A polymer inverse design task for gas separation with feedback from domain experts further demonstrates its practical utility.
denoiser. Unlike previous
Introduction
Diffusion models for molecular graphs essential for inverse design of materials and drugs by
generating molecules and polymers (macro-molecules) [4), because the models can be effectively trained to predict discrete graph structures and atom/bond types in denoising processes (43). Practical inverse designs consider multiple factors such as molecular synthetic score and various properties [15]
known as the task of multi-conditional graph generation.
Existing work converted multiple conditions into a single one and solved the task as single-condition generation 25). However, multi- property relations may not be properly explicitly defined [5]. First, the properties have diverse scales and units. For example, the synthetic complexity ranges from 1 to 5 R. while the gas permeability varies widely, exceeding 10,000 in Barrier units makes it hard for models to balance the conditions. Second, multi-conditions consist of a mix of categorical and numerical properties. The common practice of addition 47 inadequate for combination.
(4) This gap
multiplication (25 is
Figure ((a) empirically illustrates the challenges in multi-conditional generation, i.e., discovering molecules meeting multiple properties. We used a test set of 100 data points with three properties: syn- thesizability (Synth.) [12], O2 and N₂ permeability (O,Рerm and N₂Perm) [4 A single-conditional diffusion model generated up to 30 graphs for each condition, resulting in a total of 90 graphs for three conditions. We sort the 30 graphs in each set using polymer property Oracle (see appendix [B.3). Then, we check whether a shared polymer structure that meets multi-property constraints can be identified across different condition sets. If we find the polymer, its rank K (where K is between and 30) indicates how high it appears on the lists, considering all condition sets. not, we set K 30. Figure ((a) shows the frequency distribution of K on the 100 test cases. The median K was
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Synth.
Oz Perm
Perm
Synth.
O2 Perm
N2 Perm
Generated
Generated
Generated
Ranking
Ranking
Shared from Top
55
48-
If found: K
else: 30
K
K
32-
24-
Median: 30
Median=4
Median=9
Median=11
10 20 25
(a) Existing work's limitation: A me- dian rank of 30 showed that on fewer than half test polymers. the sets of generated graphs from different single conditions in- tersected, indicating a failure to generate polymers meeting multiple properties.
(b) Proposed work: Our idea, multi- conditional guidance for diffusion models, successfully generated polymers that satis- fied multi-proper constraints. achieved higher rank than 30 in any set single-conditional generated graphs.
Figure 1: Multi-conditional diffusion guidance (b) generates polymers of higher property accuracy than existing work in (a). Explanations in Section and details are in appendix B.3
30, indicating that the multiple properties not over half of the test polymers despite generating a large number of graphs.
To address these challenges, we project multi pulti-properties into representations by learning. thereby guiding the diffusion process for molecule generation. We propose the Graph Diffusion Transformer (Graph DiT) for graph denoising under conditions. Graph DiT has condition encoder for property representation learning and graph denoiser. The condition encoder utilizes novel clustering- method for numerical properties and one-hot encoding for categorical ones to learn multi cellti-propetty representations. The graph denoiser first integrates node and edge features into graph tokens, then denoise these tokens with adaptive layer normalization (AdaLN) in Transformer layers [19 34). AdaLN replaces the molecular statistics (mean and variance) in each hidden layer with those from the condition representation, effectively outperforming other redictor-based and predictor-free conditioning methods [22
as shown in Section 4.4 We observe that existing forward apply noise separately atoms and bonds, which may compromise the accuracy of Graph DiT in noise estimation. Hence, we propose a novel graph-dependent noise model that effectively applies noise tailored the dependencies between atoms and bonds within the graph.
43
34]
diffusion processes 43 22
Results Figure I(b) show that the polymers generated by Graph DiT closely align with multi- property constraints. For each test case, have graph generated from Graph DiT conditional on three properties. The Oracle determines the rank of this graph among 30 single-conditionally generated graphs for each condition. We find the median ranks are 4, 9, and 11, for Synth., O₂ Perm, and N₂ Perm, respectively, all much higher than 30. Note that the ranked set of 30 graphs was very competitive because the model trained on the specific condition dedicatedly.
In experiments, we evaluate model performance on one polymer and three small molecule datasets. The polymer dataset includes four numerical conditions for multi-conditional evaluation. Our model has the lowest average mean absolute error (MAE), significantly reducing the error by 17.86% compared to the best baseline. It also excels in small molecule tasks, achieving over 0.9 accuracy task-related categorical conditions, notably surpassing the baseline accuracy of less than 0.6. We also examine the model's 's utility in inverse polymer designs for O,/N2 gas separation, with domain expert feedback highlighting our model's 's practical utility in multi-conditional molecular design.

---

2 Problem Definition
2.1 Multi-Conditional Inverse Molecular Design
molecular graph G EV, consists of a set of nodes (atoms) V and edges (bonds) E. We follow [33 and define "non-bond" as a type of edge. There are N atoms and each atom has a one-hot encoding, denoting the atom type. We represent as Xv RXXPv where Fy the total number of atom types. Similarly, the bond features are a tensor X E RNNNNFE,, representing both the graph structure and FE bond types.
Let C = te,, C2,.,......... .CM) be set of M numerical and categorical conditions. The task is: q(G C1 C2...... ........ ,.M ((G) :2..... G), where g represents observed probabílity. We use model parameterized by 0 for multi-conditional molecular generation pe (G C). The evaluation involves both distribution learning (G) [35] and condition control q(c1. C2. : CM G). We follow previous work in assuming that there exist different oracle functions O that can independently
evaluate each conditioned property [14]: q(c,, .2.... CM G) II O,(c; G). Note that the oracles are not used in the training of pg.
....
2.2 Diffusion Model Graph Data
Diffusion models consist of forward and reverse diffusion processes We refer to the forward diffusion process as the diffusion process following [17]. The diffusion process (('):T) | GG)
TT=1 G(-1) corrupts molecular graph data (G G) into noisy states G As timesteps
T ((T)) converges stationary distribution क (G). The reverse Markov process po (G
( (G G²). parameterized by neural networks, gradually denoises the latent states toward the desired data distribution.
Diffusion Process One may perturb G in a discrete state-space to capture the structural properties of molecules [43] Two transition matrices Qv E R xFv and QF E R FeexFF are defined for nodes Xv and edges X respectively [43]. Then, each step q(G G'-1,G9 the diffusion process is sampled as follows.
g(G G²-1) in
x
Cat (X k p x Q()
(1)
[X x Cat (X xx=p=xx-000 Q²2)
where Cat(X: p)) denotes sampling from categorical distribution with probability p. We remove the subscript (v/E) when the description applies to both nodes and edges. It is assumed that the noise
Q' (i < f) is independently applied to X in each step i, allowing to rewrite qXx X the probability of the initial state x(X' X Cat (X': p X'Q',, where Q' = II Q¹
Noise Scheduling Transition matrices Qv and QE control the noise applied to atom features and bond features, respectively. Vignac et al. 443 defined TG) (mx E R Ev, m. E R as the marginal distributions of atom types and bond types. The transition matrix at timestep is
Q' = II a for atoms or bonds, where m' denotes the transposed row vector. Therefore, we have Q' = a
1m)mm', where a²= II== a". The cosine schedule
332 is often chosen
for at (1.5 + s) (11++))) +))².
Reverse Process With the initial sampling GT ~ T (G), the reverse process generates G iteratively in reversed steps ( = T, T 1,...,... 0. We use neural network to predict the probability (O G*)
as the product over nodes and edges 43]
(G G
II VEV
po Go) II po(e' --1 G') CEE
(2)
Go GO) could be combined with g Go (O) to estimate the reverse distribution on the graph po G GO). For example, Po (o G*) ) marginalized over predictions of node types
E X.. which applies similarly to edges:
po(u' I
Σ 9(00-1 1 poo( G').
(3)
DER.
The neural network could be trained to minimize the negative log log-likelhhod
43
L Ey())) (G9)
[-ExcGo log pa (x G*)]
(4)
3

---

T
t-1
GT
Multi-tedd
C
Attention
a 10-116% 100 G¹) Po (.
properties
Numerical
Categorical
AdaLN
Condition Encoder
Representations Conditions
Clustering
One-Hot Encoding
Encoding
MLP
"!!!
Adeptive Layer Normalization with Conditions' Statissllss
XWA
Graph
Layer
G
Denoiser
GO
Graph Denolser Transformer Layers)
(b) Condition Encoder Figure 2: Denoising framework and architectures for Graph DiT. Details are in Section 3.2
(a) Denaising Process
where x E Go denotes the node or edge features. Typically, the reverse process in diffusion models does not consider molecular properties conditions. While there have been efforts to introduce property-related guidance using additional predictors, the more promising approach of predictor-fiee free guidance [6], particularly in multi-conditione generation, remains underexplored.
Multi-Conditional Graph Diffusion Transformers
We present the denoising framework of Graph DiT in Figure 2 The condition encoder learns the representation of M conditions. The statistics of this representation like mean and variance are used to replace the ones from the molecular representations [19] (see Section (.2) Besides, introduce new noise model in the diffusion process to better fit graph-structured molecules (see Section 3.1
3.1 Graph-Dependers Noise Models
The transition probability of node or an edge should rely on the joint distribution of nodes and edges in the prior state. However, treat node and edge state transitions as independent, misaligning with the denoising process in Eq. 3 This difference between the sampling distributions of noise in the diffusion and reverse processes introduces unnecessary challenges to multi coltiicoodiiional molecular generations.
example shown in Eq. { current diffusion models [22 43 25
To address this, we use single matrix XG E NN to represent graph tokens for G, with FG = Fv N FF. Token representations are created by concatenating the node feature matrix Xy and the flattened edge connection matrix from Xg. Each row vector in X contains features for both nodes and edges, representing all connections and non-connections. Hence, we could design transition matrix QG considering the joint distribution of nodes and edges. QG R Fo Pa is constructed from four matrices Qv, QEV E RF Qg. QVE IR P denoting the transition probability ("dependent old state" + "target new state") node - edge; node - edge, respectively.
node; edge node; edge
Qy QG 1N & QEV
I'v 2 QVE QE
1Nx
where & denotes the Kronecker product, 1N. I'y, and 1 represent the column vector, row vector, and matrix with all 1 elements, respectively. According to Eq. (5), the first Fy columns in QG determine the node feature transitions based on both node features (first Fy rows) and edge features (remaining N Fg rows). Conversely, the remaining N Fg columns determine the edge feature transitions, depending on the entire graph. We introduce new diffusion noise model:
q(x x
Cat (X:: P = X Q%))
where P the unnormalized probability and Cat denotes categorical sampling: The first Fy columns of are normalized to sample Xt. while the remaining N F dimensions are reshaped and normalized to sample edges X These components are combined to form X completing the Cat sampling.
Choice QVE and QEv Similar the definitions of my and m 43 we leverage the prior knowledge within the training data for the formulation of task-specific matrices, Qgv and Qu We calculate co-occurrence frequencies of atom and bond types training molecular graphs to obtain marginal atom-bond co-occurrence probability distribution. For each bond type, each row in mæ represents the probability of co-occurring atom types. myg the transpose of m mæv and has similar meaning. Subsequently, we define Qgv a a'14 (1 a² a')1m'g and QvE a'I+ +++00 (1- - ā 11 g-

---

3.2 Denoising Models with Multi-Property Conditions
We present Graph DiT as the denoising model generate molecules under multi-conditions C fci, c2...., CM without extra predictors.
Predictor-Free Guidance The predictor-free free reverse process po(G"=1 G", C) aims to generate molecules with high probability q(C GO). This could be achieved by linear combination of the log probability for unconditional and conditional denoising [16]:
Po G = log po (G G*) + (o
po po
G*, I
log po G*))
(7)
where denotes the scale of conditional guidance. Unlike classifier-fre guidance [66] which typically predicts noise, we directly estimate pe pe(G C). We one one denoising model GG*, .C) for both po G²) and (G G C). Here, fo(G", C () computes the unconditional probability by substituting the original conditional embeddings with the null value. During training, we randomly drop the condition with a ratio, i.e., C 0, to learn the embedding of the null value. fo(G" = XG,C) comprises two components: the condition encoder and the graph denoiser. An overview of the architecture presented F
Condition Encoder We treat the timestep as a special condition and follow [31] to obtain D- dimensional representation t with sinusoidal encoding. For property-related numerical categorical condition e E C. we apply distinct encoding operations to get D--dimensional representation. For a categorical condition, we use the one-hot encoding. For a numerical variable, we introduce clustering encoding method. This defines learnable centroids, assigning G, to clusters, and transforming the soft assignment vector of condition values into the representation. It could be implemented using two Linear layers and a Softmax layer in the middle as: Linear (Softmax (Linear(c))), Finally, Σ i=1 encode(c;). where encode is the
we could obtain the representation of the condition specific encoding method based on the condition type. For numerical conditions, we evaluate our proposed clustering-basee based approach against alternatives like direct or interval-basee encodings (28) As noted in Section 4.4 the clustering encoding outperforms the other methods.
Given the noisy graph at timestep f, the graph tokens are first encoded into the hidden space H Linear(X,, where H E R i), We then adapt the standard Transformer layers [42] with self-attention and multi-layer perceptrons (MLP), but replace the normalization with the adaptive layer normalization (AdaLN) controlled by the representations of the conditions
Graph Denoiser:
Transformer Layers
AdaLN(H,, c). For each row h in H:
AdaLN (h, c) (e(c) h- (b) + Bo(c), (h)
(8)
where ya()) and (-) are mean and variance values. @ indicates element-wise product. 70(+) and Bo(-) are neural network modules in fa(-), each of which consists of two linear layers with SiL.U activation [[] in the middle. We have a gated variant AdaLN gate for residuals:
AdaLN gate (h, c) = co(c) @ AdaLN (h,c)
34 There are other options to learn the structure representation from the condition 34: In- In-Context conditioning adds condition representation to the structure representation at the beginning of the structure encoder, and
We apply the zero initialization for the first layer of (-))), (.).) and (.
Cross-Attenile calculates cross-attention between the condition and structure representation. We
observe in Section 4.4 that AdaLN performs best among them.
Graph Denoiser: Final MLP We have the hidden states H after the final Transformer layers, the MLP is used to predict node probabilities X and edge probabilities X t 0:
X AdaLN(MMP(HH) c).
(10)
We split the output X into atom and bond features -0 The first Fy dimensions of X represent node type probabilities, and the remaining N Fg dimensions cover probabilities for N edge types associated with the node, as detailed in Section 3.1
Generation to Molecule Conversion A common way of converting generated graphs molecules selects only the largest connected component [43), denoted as Graph DiT-LCC our model. For Graph DiT, connect all components by randomly selecting atoms. It minimally alters the generated structure to more accurately reflect model performance than Graph DIT-LCC.
5

---

Table 1: Multi- Conditional Generation of 10K Polymers: Results the synthetie score (Synth.) and three numerical properties (gas permeability for O2. N2, CO2). MAE is calculated between the input conditions and the properties of the generated polymers using Oracles. Best results are highlighted.
Model
Validity 1 rwle checking)
Distributonn 1 Lernnng
Condition Centrol
Coverage
Diversity
Simalarity
Distance
Synth
OyPerm
NyPerm
C0pPerm
Avg. MAE
Graph GA
1.0000 (N.A)
11/11
0.8828
0.9299
9.1882
1.3307
1.9840
2,2500
1,9489
1.8884
MARS
1,0000 (N.A.)
11/11
0.8375
0.9283
7.5620
1.1658
1.5761
1,8327
1/6074
1.5455
LSTM-HC
(N.A.)
0.8918
0.7937
18.1562
1.4251
1.1003
1,2365
1,772
JTVAE-RO
1,0000 (N.A.)
10/11
0.7566
0.7294
23.9950
1,0714
1.0781
1,2352
1,0978
1,1206
DiGress
0.9913 (0.2362)
11/11
0.9099
0.2724
22.7237
2.9842
1,7163
2,0630
1,6738
2,1093
DiGress v2
0,9812 (0.3057)
11/11
0,9105
0.2771
21.7311
2.7507
1.7130
2,0332
1,6648
2.0479
GDSS
0.9205 (0.9076)
9/11
0.7510
0.0000
34,2627
1.3701
1.0271
1,0820
1,0083
1.1369
MOOD
(19866 (0.9205)
0.8349
0.0227
39.3081
1,4019
1,4961
1.7603
1,4788
Graph DIT-LCC (Ours)
0,9755 (0.8437)
11/11
0.8875
0.0560
7.0949
3,009
0.8001
0.9562
0.8125
0.9697
Graph DIT (Ours)
(0.8437)
11/11
0.8712
0.9600
6.6443
1.2973
0.7440
0,8857
0.7550
4 Experiment
RQ1: We validate the generative power of Graph DiT compared to baselines from molecular optimization and diffusion models in Section RQ2: We study polymer inverse design for gas RQ3: We conduct further analysis examine Graph DiT in Section 4.4
separation in Section 4.3
4.1
Experimental Setup
We use datasets with over ten types of atoms and up to fifty nodes in a molecular graph. We include both numerical and categorical properties for drugs and materials, offering a benchmark for evaluation across diverse chemical spaces. Model performance is validated across up to nine metrics, including distribution coverage, diversity, and condition control capacity for various properties.
Datasets and Input Conditions We have one polymer dataset [40] for materials, featuring three numerical gas permeability conditions: OyPerm, CO,Perm, and N2Perm. For drug design, we create three Mass-balanced datasets from MoleculeNet [46] HIV, BBBP, and BACE, each with categorical property related to HIV virus replication inhibition, blood-brain barrier permeability, or human _ secretase 1 inhibition, respectively. We have two more numerical conditions for synthesizability from synthetic accessibility (SAS) and complexity scores (SCS) [12 g
Evaluation We randomly split the dataset into training, validation, and testing (reference) sets in a 6:2:2 ratio. Evaluations are conducted on 10,000 generated examples with metrics (35 (1) molecular validity (Validity); (2) heavy atom type coverage (Coverage); (3) internal diversity among the generated examples (Diversity); (4) fragment- baased similarity with the reference set (Similarity); (5) Fréchet ChemNet Distance with the reference set (Distance) (36): MAE between the generated and conditioned (6) synthetic accessibility score [12 (Synth.); (7)~(9) MAE/Accuracy for the numer- ical/categorical task conditions (Property). The evaluation Oracle uses random forest trained on all task-related molecules [4]. Lower MAE or higher accuracy indicates stronger model controllability.
Baselines We select strong and popular molecular optimization baselines from recent studies 14]: Graph-GA (20], MARS (7), JTVAE (21) with Bayesian optimization (JTVAE-BO), LSTM [6] on SMILES with Hill Climbing (LSTM-HC). We include the most recent diffusion models: GDSS
DiGress 43 and their conditional version with extra predictors: MOOD 25 and DiGress v2 43 We train multi-task predictors using the same architecture for MOOD and DiGress v2 models to provide additional guidance for generation. For molecular optimization, we formulate the condition set of each test data point as a combined goal, minimizing the sum of the normalized errors between generated and input properties. We train random forest model for each property using the training data to optimize the molecular structure.
4.2 RQ1: Multi-Condiiional Molecular Generation
We have the observations from Table and Table 2
Chemical Validity High validity may not accurately represent the model's generative performance hard-coded rules are introduced in the algorithm. For example, GraphGA could eliminate nanivalid molecules during mutation and crossover iterations to achieve perfect validity in the final evaluation. step, DiGress, GDSS, and MOOD show a
Without rule checking in the seneration--t-mooeeee marked performance decline, with validity often dropping from 0.99 to below 0.6. In contrast, Graph DiT often maintains over 0.8 validity without any rule-based processing.

---

Table 2: synthesizability score (Synth.) and a categorical task-specific property. MAE/Accuracy is calculated by comparing input conditions and generated properties. The best number per metric is highlighted.
Generation of 10K Small Molecules: Each dataset involves a numerical
Tasks Model
Validity (w/o checking)
Distribution Leuming
Condition Control
Coverage
Diversity 1 Similariy
Distance
Synthe, MAE 1 Property Avc.
Avg. Rank
Graph GA
1,0000 IN.A.)
8/8
0,8585
0,9805
7,4104
0.9633
0.4590
6,5000
MARS
1,0000 IN
8/8
018338
0.8M27
6.7921
1.0123
0.5184
5.0000
LSTM-IC JTVAF-RO
0,9972 (N.A.)
0.8146
0.7482
17,5585
0/9207
0.5816
3,0000
Syneh. BACE
1,0000 (N.A.)
628
0.6682
0.7281
30.4669
0.9923
0.4628
7,5000
DiGress
0.3511 (0.2858)
0.6912
24,6560
2,0681
0.5061
8,0000
DiCless V2
0,3546 (0.2680)
8/8
08812
0.7027
25.3270
2,3365
0.5113
7,5000
GDSS
0.2879 (0.2589)
4/8
0.8756
0,2708
66.7539
1.6122
0.5036
7,5000
MOOD
0.9967 (0,4502)
8/8
0.8902
0,2587
44,2394
1,853
0,5062
7,0000
Graph DIT LCC (Ouns)
0.8646 (0.8495)
8/8
0.8240
0.8157
6.9836
0:4053
0,9050
2,0000
Graph DIT (Ours)
0.8674 (0.8495)
0,2288
0.8722
7,0456
0.3998
0.9135
1,0000
Graph GA
1.0000 IN A.)
9/9
0,8950
0.9509
10.1659
1,20K2
0.3015
7,5000
MARS
100000 (N.A.)
89
0,8637
0.7696
10.9791
1,2250
0.5189
6.0000
LSTM-HC
0,9990 IN.A.)
8/9
0.5883
0.8932
16.3904
0.9969
0.5590
4.0000
Synt. BBBP
JTVAE-BO
100000 (N.A.)
0,7458
0.5821
33,5746
1,1619
0.4958
6,0000
DiCiress
0.6950 (0:4871)
9/9
0,009
0.6805
T8.6921
2.3658
0.6536
6.500n
DiGress
0.6892 (0.4100)
969
0.9107
0.6336
19,4498
2.2694
0.6531
6,5000
GDSS
0.62188 (0.1919)
3/9
08415
0,2672
39.9440
1,3788
0,5087
7,0000
MOOD
0.8008 (0.5789)
9/9
0.9273
0.1715
34,2506
2,0284
0,4903
8,5000
Graph DIT-L (Ouns
0.8657 (0.8505)
9/9
0,857
0.9324
11,8587
0.3717
0,9390
2,00000
Graph DIT (Ouns)
0.8468 (0.8505)
9/9
0,8856
0,9329
11,8519
0.3551
0.9417
1,0000
Graph GA
100000 (N.A.)
28/29
0.8993
0.9661
4,4418
0.9839
0,0038
5,0000
MARS
1,0000 (N A.)
26/29
0,8764
0.6517
7,2893
0.9691
0.6455
4,00000
LSTM-HC JIVAE-BO
0,9994 (N.A.)
13/29
0,9091
0.9145
7,4659
0,9480
0.6736
3,0000
1.0000 IN.A.)
3/29
0.5055
0.4173
41.9771
1,2359
0.4850
7,5000
Synth, & HIV
DiGress
0,4377 (0.3643)
22/29
0,9191
0.8562
13,0809
1.9216
0.5335
7,5000
DiChess v2
0,5050 (0, (0:4222)
24/29
0.9193
0,8476
13.3997
1.5934
0.5331
7,5000
GDSS
0.6926 (0.6757)
4/29
0.7817
0.1032
15,3416
1,2515
0,1530
8,5000
MODD
0.2875 (0.2113)
29129
0.9280
0,1361
32,3523
2,3144
0,5106
9,0000
Graph DIT-LCC (Ours)
0.7635 (0.7415)
28/29
0.8966
0.9535
5,8790
0.3084
0.9966
1,5000
Graph DIT (Ours)
0.7666 (0.7415)
28/29
0.8974
0.9575
6.0216
0,3086
0.9777
1.5000
Distribution Learning GraphGA is simple yet effective baseline for generating in in-distribution molecules, e.g., on BBBP and HIV generation datasets. Diffusson model baselines such as DiGress and MOOD could produce diverse molecules but often fail to capture the original data distribution in multi-conditional tasks. Graph DiT shows the competitive performance of diffusion models in fitting complex molecular data distributions. Using fragment-based similarity and neural network-based distance metrics (36), we achieve the best in the polymer task and rank second in the HIV small molecule task, involving up to 11 and 29 types of heavy atoms, respectively.
Condition Controllability LSTM-HC surpasses many baselines, achieving lower average MAE on polymer properties and higher rankings small molecular properties. However, its control over synthetic scores in polymer tasks is relatively poor. Conversely, MARS effectively manages synthetic scores for polymers but exhibits larger MAE in gas permeability conditions compared to other baselines. GDSS performs well in gas permeability control but underperforms Graph GA and MARS in terms of the synthetic score condition. DiGress v2 and MOOD, although equipped with the predictor guidance, still exhibit limited condition control compared to their unconditional counterparts over polymer and small molecule tasks. These baselines struggle balance and control multiple conditions in generation. In contrast, Graph DiT significantly improves diffusion models and achieves the best multi-conditional performance in all tasks. In polymer tasks, Graph DiT reduces MAE on all gas permeability conditions, averaging + +17.8% improvement over the best baseline LSTM-HC. For small molecule tasks, Graph DiT consistently ranks top-1 in condition controllability with over 0.9 accuracy in categorical conditions. Compared to Graph DiT-LCC, we observe that Graph DiT, which connects all generated graph components, shows better controllability performance due to minimal rule-based post-generation processing.
4.3
RQ2: Polymer Inverse Design for Gas Separation
We aim to design polymers with high O₂ and low N₂ permeability, demonstrating the models' precise control over related properties. Following Robeson [38)ss 's definition of high-performance polymers based on the Oz/N, permeability ratio, we selected 16 polymers meeting this criterion from 609 examples as our test/reference set. The remaining data is used for training and validation. Subsequently, we generated 1,000 polymers conditioned on test set labels.
7

---

(a) MARS
(b) LSTM-HC
(c) DiGress
MOOD
(e) Ours
UtS: 0.16/ AS: 0.77
UIS: 0.2/ AS: 0.97 Rank: (4)
UtS: 0.08/ AS: 0.94 Rank: (12)
UtS: 0.13/AS: 1.00
UIS: 0.75 AS: 1.00 Rank: (1)
Rank: (6)
Rank:
0.19 0.93 Rank:
0.06 0.21 Rank: (14)
UtS: AS: Rank: (11)
UtS: 0.49 AS: 0.97
Rank: (6)
Rank: (2)
UIS: AS: 0.70 Rank: (8)
UIS: 0.11 / 0.96
UtS: 0.04 1.00
UIS: 0.07 AS: 0.98 Rank: (13)
UtS:
Rank: (10)
Rank: (15)
Rank: (3)
Figure 3: Polymer Inverse Design for O2/N2 Gas Separation: Feedback from four domain experts includes an average Utility Score (UtS) for relative usefulness and an Agreement Score (AS) for generated polymers, both ranging [0, 1]. Polymers are generated conditional on (SAS=3.8, SCS=4.3, OyPerm=34.00 NyPemm=5.2). The top-3 polymers, highlighted, are all generated by Graph DiT.
O, Perm
O, Perm
O, Perm
0.76
Distance
0.59
Distance
Distance
9.52
NyParm
2.0
M.Perm.
0.06
N Parm
Cluster Interval
AdaLN en-Context
Depedent Separate
0.48
aise
0.53
0.57
0.46
0.5e
Similant Hy
Similarity
Similarity
s.de
Cross-Attn
DiGress v2
0,8
co.
500
CO.Perm
5.00
CO CO.Pern
$,00
0.35
0.54
0:52
1.03
1.42
Diversity
3.08 1.00
Diversity
1.4
Diversity
(a) Numerical Condition Encodings
(b) Condition Architectures (c) Graph Dependent Noise Model
Figure 4: Relative Performance of Different Model Designs: A higher bar indicates better perfor- mance. We use the performance clustering-basee encoding or AdaLN as the Reference Value and the current option the Current Value. Relative performance is calculated as Carrent Valte for Similarity and Diversity metries, and as
Reference
Reference Value for other metrics.
Current Value
In Figure we present the top three polymers generated by each model for case study with expertise. Initially, a random forest algorithm identifies the top five polymers per method based on average MAE in two gas permeability. These 25 polymers are then shuffled and evaluated by four polymer scientists, who rank them from to 25 using their domain knowledge. Rankings are normalized to Utility Score (UtS) ranging from I, with higher scores indicating greater utility. The variance in UtS is converted into an Agreement Score (AS) for further evaluation. As shown in Figure 3 there is a high consensus among experts that the three polymers generated by Graph DiT are the most promising for successful polymer inverse design tasks. More details are in appendix D By comparing generated examples from different models, we have further observations:
DiGress and MOOD struggle to capture polymerization points, marked with asterisks ("*"), which one of the most important features that distinguish polymers from small molecules. Additionally, the two methods frequently feature excessive carbon atoms and overly large cycles. These molecular configurations with significant distortion from the canonical geometry of stable compounds may lead to poor synthesizability [55 7
LSTM-HC may result in too-long carbon chains with limited diversity. MARS produces examples with asymmetrical graph structures, challenging polymer synthesis [10 2)

---

Graph DiT generates structurally diverse and symmetric polymers with two polymerization points, indicative of more valid and synthesizable polymer structures. The first two, which are polyimides, imply effective gas separation performance [24]
4.4 RQ3: Ablation Studies and Model Analysis
we analyze three components that impact our model's learning in various conditions. Our assessment of relative performance is based on the ratio between our method and comparative approaches. The first component is numerical conditional encoding. Results in Figure 4(a) highlight the superiority of clustering encoding over direct and interval-based encoding, particularly in controlling gas permeability, despite its slightly lower diversity. The second component concerns the neural architecture for conditions. As shown in Figure 4(b) similar to Figure (a)) AdaLN surpasses both In-Context Conditioning and Cross-Attento
Model Components In light of Table |
in learning distribution with better condition controllability. The third component validates the importance of the graph-dependent noise model compared to separately applying noise to atoms and bonds. It also shows the improvement of the predictor-free Graph DiT over the predietor-guided DiGress v2, even without the graph-dependent noise model. More results on model controllability are in appendix L
Oracle Selections We analyze the robustness of Oracles in evaluating six task-related properties (three gas permeability and three small molecule properties) across six conditional generation tasks. Oracles are switched from Random Forest to Gaussian Process or Support Vector Machines for ranking generative model performance. Re- sults in TableB3shor consistent rankings (Graph DIT, LSTM-HC, MARS, JTVAE-BO, MOOD, GDSS, GraphGA), It indicates that while perfectly approximating the truth properties of generated molecules is difficult, we could effectively com- pare the relative performance of various models. Graph DiT consistently ranked first among baselines.
Table 3: Oracles for Generation Evaluation: We consider three Oracles. Generative performance is ranked on average from to 9 across six prop- erties, with various Oracles yielding similar out- comes. We highlight models with the same rank- ing sequence in different Oracle evaluation.
Avg. Rank
Randen Forest
Gaussian Precess
Support Victor Machine
Graph
Gruph
Graph DiT DiGress V2 DiEress LSTM-HC
LSTM-HC
DiGiress v2
'MARS
TiChess
JTVAE-BO
LSTM-HCC
MOOD
MARS
MARS
DiCiress
JTVAE-BO
JTVAF-BO
DiGress x2
MOOD
MOOD
GDSS
Graph
Graph
Graph
5 Related Work
Diffusion Models for Molecules:
Score-based diffusion models applied noise and denoising in
continuous space 33 22 DiGress (43 used discrete noise as transition matrices based on marginal
distributions of atom and bond types. Extra predictor models studied to guide the generation process DiGress and GDSS
25
Diffusion models could also be used for molecular property
prediction 27). for conformation 48 and molecule generation with 3D atomic coordinates [18 (3
We focus on molecular graph generation, considering the high computational cost of accurate 3D coordinates for larger molecules like polymers [23]. We explore predictor-fit free diffusion guidance, instead of the classifier guidance [9] 44,, for generating molecules under categorical and numerical conditions. It can be integrated with diffusion models for atomic coordinates in future research.
Molecular Optimization: Optimization algorithms could optimize molecules towards property
constraints, including genetic algorithms [20], Bayesian optimization [39 50], REINFORCE (45).
and reinforcement learning [30]. Both sequential and graph-based generative models 6 21 30 along
with diverse sampling methods 47 [3] are used in conjunction with these algorithms to produce
desirable molecules. Thess methods have been applied to both single-objective and multi-objectiin optimization, the latter by manually integrating multiple property conditions into single one (5] 25) Several challenges in molecular optimization methods remain underexplored, including the inadequate or unclear definition of multi-property relations when integration into a single objective (5), and the inaccessibility of the oracle function for property- optimization during the training phase [4]
Conclusion
In this work, we solved inverse molecular design using properties as predictor- diffusion guidance. The proposed Graph DiT performed diffusion based on the joint distribution of atoms and bonds in both forward and reverse processes. introduced representation learning for multiple categorical and numerical properties and utilized Transformer-based graph denoiser for conditional graph denois- ing. Results on multi-conditional generations and polymer inverse designs showed the remarkable generative capabilities of Graph DiT, making it suitable for designing promising molecules.

---

Acknowledgements
This work supported by NSF IIS-2142827, IIS-2146761, IIS-2234058, CHET-2332270, CBET- 2102592, and ONR N00014-22-1-2507.
References
[1] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 14:17981-17993, 2021.
[2] Alexandru T Balaban. Symmetry in chemical structures and reactions. In Symmetry, pages
999-1020. Elsevier, 1986.
[3] Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, and Jun Zhu. Equivariant energy-guided SDE for inverse molecular design. In The Eleventh International Conference on Learning Representations, 2023. URL
[4] J Wesley Barnett, Connor R Bilchak, Yiwen Wang, Brian C Benicewicz, Laura A Murdock, Tristan Bereau, and Sanat K Kumar. Designing exceptional gas-separation polymer membranes using machine learning. Science advances, 6(20))eaa74301, 2020.
[5] Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F Jensen. Gener- ative models for molecular discovery: Recent advances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12(5)e 608, 2022.
[6] Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: bench- marking models for de novo molecular design. Journal of chemical information and modeling, 59(3):1096-1 108, 2019.
[7] Robert F Bruns and Ian A Watson. Rules for identifying potentially reactive or promiscuous compounds. Journal of medicinal chemistry, 55(22):963-9772, 2012.
[8] Connor W Coley, Luke Rogers, William H Green, and Klavs F Jensen. Sescore: synthetic complexity learned from reaction corpus. Journal of chemical information and modeling, 58 (2):252-261, 2018.
[9] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-$8 189900-7794, 2021.
[10] Jack D Dunitz. Symmetry arguments in chemistry. Proceedings of the National Academy of Sciences, (2225):12660-4266 1996.
[11] Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3-11, 2018.
[12) Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminfor- matics, 1:1-11, 2009.
[13] Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W Coley, and Jimeng Sun. Differentiable scaffolding tree for molecular optimization. arXiv preprint 109.:26009 2021.
[14] Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley, Sample efficiency matters: benchmark for practical molecular optimization. Advances in Neural Information Processing Systems, 35:21342-21 242221157 2022.
[15) Niklas WA Gebauer, Michael Gastegger, Stefaan SP Hessmann, Klaus-Robert Müller, and Kristof Schütt. Inverse design of 3d molecular structures with conditional generative neural networks. Nature communications, 13(1):973, 2022.
[16] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint
a207 2598, 2022.
[17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[18] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation 3d. International conference on machine learning, pages 8867-8887. PMLR, 2022.
10

---

[19] Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision, pages 1501-1510, 2017.
[20] Jan H Jensen. A graph-hased genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567-3577 2019.
[21] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pages 2323-2333 PMLR, 2018.
[22] Jachyeong Jo, Seul Lee, and Sung Ju Hwang.
Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference Machine Learning, volume 162, pages 10362-10383, PMLR, 2022.
[23] Soumil Joshi and Sanket A Deshmukh. A review of advancements in coarse-grained molecular dynamics simulations. Molecular Simulation, 17(10-11:7766-03, 2021.
[24] Michael Langsam. Polyimides for gas separation. In Polyimides, pages 697-742. CRC Press, 2018.
[25] Seul Lee, Jachyeong Jo, and Sung Ju Hwang. Exploring chemical space with score-based out-of- distribution generation. In International Conference Machine Learning, pages 18872-18892. PMLR, 2023.
[26] Gang Liu, Tong Zhao, Jiaxin Xu, Tengfei Luo, and Meng Jiang. Graph rationalization with mnvironment-basee augmentations. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 1069-1078, 2022.
[27] Gang Liu, Eric Inac, Tong Zhao, Jiaxin Xu, Tengfei Luo, and Meng Jiang. Data-centric learning from unlabeled graphs with diffusion model. In l'hirty-seventh Conference Neural Informa- tion Processing Systems, 2023.
ptttcp::///penrevvie
pest/forummiid-maaavCCll
[28] Gang Liu, Tong Zhao, Eric Inae, Tengfei Luo, and Meng Jiang. Semi-supervised graph imbalanced regression. In 29th SIGKDD Conference Knowledge Discovery and Data Mining, 2023.
[29] Ruimin Ma and Tengfei Luo. Pilm: a benchmark database for polymer informatics. Journal of Chemical Information and Modeling, 60(10):4684-4690, 2020.
[30] Amina Mollaysa, Brooks Paige, and Alexandros Kalousis. Goal-directed generation of discrete structures with conditional generative models. Advances in Neural Information Processing Systems, 33:21923-21933, 2020.
[31] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Iyya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112. 2021.
[32] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[33] Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Permutation invariant graph generation via bcore-based generative modeling. In International Conference on Artificial Intelligence and Statistics, pages 4474-4484, PMLR, 2020.
[34] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
[35] Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez-Lengeling, Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Aleksey Artamonov, Vladimir Aladinskiy, Mark Veselov, al. Molecular sets (moses): benchmarking platform for molecular generation models. Frontiers in pharmacology, 1 1::656644, 2020.
[36] Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, and Gunter Klambauer. Fréchet chemnet distance: metric for generative models for molecules in drug discovery. Journal of chemical information and modeling, 58(9): (889):1736-11 776-17441 2018.
[37] Philipp Renz, Dries Van Rompaey, Jörg Kurt Wegner, Sepp Hochreiter, and Günter Klambauer. On failure modes in molecule generation and optimization. Drug Discovery Today: Technologies,
32:55-66 63, 2019.
11

---

[38] Lloyd M Robeson. The upper bound revisited. Journal of membrane science, 320(1-2):390-400 2008.
[39] Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan Adams, and Nando De Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104 (1):148-175, 2015.
[40] A Thortton, L Robeson, B Freeman, and D Uhlmann. Polymer gas separation membrane database, 2012.
[41] Austin Tripp and José Miguel Hernández-Lobato. Genetic algorithms are strong baselines for molecule generation. arXiv preprint arXiv:2310.0926) 2023.
[42] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[43] Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pas- cal Frossard. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209. 14734, 2022.
[44] Tomer Weiss, Eduardo Mayo Yanes, Sabyasachi Chakraborty, Luca Cosmo, Alex M Bron- stein, and Renana Prrshoni-Poranne Guided diffusion for inverse molecular design. Nature Computational Science, pages 1-10, 2023.
[45] Ronald J Williams. Simple statistical gralient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8:229-256, 1992.
[46] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, 922):113-530 2018.
[47] Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint vrXiv:2103.10432. 2021.
[48] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff:
A geometric diffusion model for molecular conformation generation. In International Con- ference on Learning Representations, 2022. URL httpp::/ppennrerrew.ee...fooo formmii
PzcvxEMzvQC)
[49] Minkai Xu, Alexander S Powers, Ron O Dror, Stefano Ermon, and Jure Leskovec. Geometric latent diffusion models for 3d molecule generation. In International Conference Machine Learning, pages 38592-38610. PMLR, 2023.
[50] Yiheng Zhu, Jialu Wu, Chaowen Hu, Jiahuan Yan, Tingjun Hou, Jian Wu, et al. Sample- efficient oblti-objective molecular optimization with gflownets. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Table 4: Dataset information for all nulti-condiiional generation and inverse polymer design tasks. Oz/COn/NgPema only denotes the data statistics considering only one permeability and the generation results presented in Table 5] The number of task conditions shown in the table does not include the timestep condition in the diffusion model.
Molecule
Heavy Atoe Type
Avg.
Arg. Bunds
Iuput Numerical
Input Categorical
Datnsets
(TainnValilanion/Tee
Triming
Atoms
Atcems
Atomss
Bonds
Bonds
Task Conditions
Task Conditions
Gas Porm
(31/11111)
11
48
27.97
56
32,67
BACE
1332 (798/6672667
8
50
33.67
54
36.44
2
ВВыP
872 (522/175/175)
9
30
24.38
53
26.26
2
2372 (142754775475
60
27.28
O,N,
609 (176/119116)
11
48
27.90
96
32.63
4
Oy only
(672/126/126)
C0 only
584 (350/177/1777)
11
18
27.59
96
32.23
3
NyPenn only
(369/123124)
27.96
A Details on the Denoising Model Component
A.1 Numerical Condition Encoding
We explore several approaches for encoding numerical conditions. addition to the clustering- based method, we consider:
1. The direct encoding approach, which employs a linear layer map a continuous number into high-dimensions space.
2. The interval- approach, as described in (28). divides the label space into Ninntorvll intervals. It then converts the number into an interval index, allowing to apply one-hot encoding for the number.
A.2 Neural Architecture for Conditions
Besides the AdaLN, there are two more options to integrate condition representation into molecular graph representations 34
1. The In-Context conditioning approach adds the condition representation c to each row of the molecular graph representation H after mapping the X into H using the linear layer in the structure encoder.
2. The Cross- Attention approach concatenates the timestep encoding vector with the condition representation from synthesis scores task-related properties into two-length sequence. each Transformer encoder layer, this is followed by a cross-attention layer at the end of the standard multi-head self-attention layer.
B Details on Datasets and Evaluation Methods
All experiments can be run single A6000 GPU card.
B.1 Datasets and Task Conditions
As presented in Table we collect popular datasets in prediction tasks for more challenging molecular generation tasks. We include polymer dataset [40 26 for material design. It consists of conditions of Oz, CO2, and N2, which study the numerical gas permeability for oxygen, carbon dioxide, and nitrogen, respectively. Additionally, we also study the generative performance of different models separately on the polymer data with O2, CO2, or N2, as illustrated Table 4 We also create three class-balanced molecule datasets from [46) for drug design: HIV, BBBP. and BACE, which study categorical properties related to the inhibition of HIV virus replication, blood-brain barrier permeability, and inhibition of human f-secretase respectively. We aim to generate synthesizable
13

---

Trining MCD (durs) DiGress
bidres GpSS M000
Graph GA MARS
STX
1.0
1.0
0.1
0.01
0.1
0.001
0.01
0.0001
0.001
1e:05
0.0001
N 0 S1 S
Ge Br
SINGLE
DOUBLE
TRIPLE AROMATIC
(a) Histogram for Heavy Atom Type Distribution
(b) Histogram for Bond Type Distribution
Figure Histogram of Generated Distribution for Atom and Bond Types in Different Models. Results are calculated based on Table | for the polymer gas permeability tasks. We observe that the atom and bond type distributions from our Graph DiT's generated molecules are closer to those of the training data than other diffusion models. It indicates that Graph DiT has better capacity for learning molecular distributions.
molecules. Therefore, we add two numerical conditions for synthetic complexity scores 12
g for each of the tasks. For the gas separation polymer design task, we consider joint conditions for O2 and N2 and measure the selectivity O2/N₂ as the ratio between two gas permeability scores. All polymer gas permeabilities are scaled in the log space following previous work 29). We focus on experiments for polymers and molecules within 50 nodes.
B.2 Evaluation and Metrics
We randomly split the dataset into training, validation, and testing (reference) in a 6:2:2 ratio. We investigate more than eight metrics to systematically evaluate the generation performance. First, assess generation validity (Validity). Second, evaluate the distribution learning capacity of different models by measuring heavy atom type coverage (Coverage), internal diversity among the generated examples using Tanimoto similarity (Diversity), fragment-based similarity with the reference set (Similarity), and the Fréchet ChemNet Distance with the reference set (Distance). Third, we evaluate the model's controllability by measuring the mean absolute error (MAE) between the generated condition score and the actual condition scores if the condition is numerical; otherwise, we measure the accuracy score. We follow previous work [4] to use the random forest trained on all the available data as the Oracle evaluation function. For molecular optimization algorithms, we train random forest predictors on the training set for conditional generation. For predictor- mreiictorguided diffusion models, DiGress v2 and MOOD, we use the same architecture as their denoising models to train predictors for diffusion guidance. Given the conditions in the test set, we report the generation performance by generating 10,000 examples for the six multi-condiiional generation tasks and 1,000 examples for the polymer inverse design problem focused on selectivity.
B.3 Datasets and Tasks in Figure
Using the same dataset from the Og/N₂ polymer inverse design task, we keep 100 polymers to provide condition sets for testing and split the rest into training and validation sets in a 0.65:0.35 ratio. We apply our proposed Graph DiT for both single-conditional and multi-condiiional approaches, focusing on three properties: (1) Synth. score for synthesizability [12). (2) O2 permeability, and (3) N₂ permeability. The angle-conditional approach generates 30 polymers per condition for each test data point, totaling 9,000 polymers. In contrast, the multi-conditional approach generates 30 polymers for each set of conditions per test data point, resulting 3,000 polymers. We rank these polymers based on the mean absolute error between the generated properties (evaluated by random forest model trained on all the data to simulate the Oracle function) and the conditional property. For each test data point, we also rank the best multi-conditional polymer in different single-conditional
14

---

72
Median: 30
Median: 16
Examplo
56
40
32
30
5
10
(a) Single-Conditiin Generation
(b) Nultii-Conditional Generation
Figure 6: We compare the average ranking of generated polymers with desirable properties. First, generate three sets of polymers using a single-conditional approach for each condition. In (a). as shown in Figure ! we find the ranking of the shared structure for each multi-condition requirement. In (b), polymers are generated using multi-condiiional approach, and for each, identify the highest ranking among the three single-conditional sets. Then, we calculate the median of these maximum ranking positions, which is 16, approximately 2x better than single-conditions generation, which has median value greater than 30.
sets. For the single-conditional approach, we identify a common polymer meeting various properties and visualize the minimum top K value distribution across all 100 test points.
In addition Figure 1 polymers within the single-conditional sets. The results are shown in Figure O Both Figure and Figure G demonstrate the advantages of multi-conditional generation over single-conditional generation.
compute the median rankings of the multi-conditionallyy generated
C Details on Multi-Conditional Generation Results
show results for three polymer generation tasks in Tand molecule generation tasks in Table 2 As complementary results for Table TT we present new results generation using only one gas permeability in T5 We also compare the distributions of atom and bond types between generated and training data in Figure [ Furthermore, Figure visualizes the -wo-dimensional molecular data distribution both training and generated molecules across various generative models.
C.1 Discussion on Diffusion Model Baselines
While diffusion models like GDSS 22 and DiGress 4 show promise in unconditional tasks, their performance in multi-conditional generations needs improvement for fitting training distributions and achieving
controllable results. As indicated by Figure 5(a) the generation of GDSS [22
often collapses to carbon elements with Gaussian noise the continuous diffusion state-space. MOOD (25] improves
type coverage by adding predictor guidance and outto--distributtin
hyper-parameter, but it is hard to the training distribution, as visualized in Figure DiGress and its predictor-guided variant [43] (i.e., DiGress v2), using discrete state-space and transition matrices for diffusion noise, outperform GDSS and MOOD in distribution fitting and internal diversity in Tables I and 5 However, indicated in Figure 7 and Figure 7 these two models still generate too many out-of diitof--dittriuutionn examples without justification of the generalization capacity. While GDSS and MOOD show lower average MAE in polymer conditional generation tasks, their subpar distribution learning performance and the results from Table suggest that this may be due to the carbon element, which may be a confounder and affect the evaluation of the correlation between the model and controllability in polymer tasks.
C.2 Discussion on Molecular Optimization Baselines
Popular molecular optimization baselines are competitive in molecular generation tasks. Earlier studies have noted their strong performance: Gao et al. [4] showed their effectiveness in the standard molecular optimizations with combined optimization target, and Tripp and Hernández-Loofe found that genetic algorithm often outperforms recent methods in unconditional generation. We
15

---

Table 5: Generation of 10K Polymers: Results on numerical synthesizability score (Synth.) and a numerical properties (gas permeability for O2, N2. or CO2). MAE is calculated between input conditions and generated properties. Best results are highlighted.
Validity 1 (wlo nwle checking)
Distribation Learning
Condition Control
Tasks Model
Coverage
Diversity
Similarity
Distance
Syath. 4
Property J Avg. J
Gruph GA
1,0000 (N.A.)
11/11
0.8885
0,9180
8,3925
1,3254
1.8922
4,4521
MARS
1,0000 (N.A.)
10/11
0.2263
0,5170
26.6354
0.8502
18853
3.6472
HC
0,9896 (N.A.)
10/11
0.8898
0,8015
17,5424
1.2727
1.1323
3.8278
Synh. & Og Perm
JTVAF.BO
1,0000 (N.A.)
8/11
0.7672
0.8895
21,3698
0,9703
1,3257
3.2971
DiGress
0,9934 (0.3756)
11/11
0.9156
0.2648
19,9364
2.5093
1.624
5,2492
DiGress v2
0,9842 (0.4237)
11/11
0.9204
0.2311
20,4500
2.3444
1.6445
5.1255
GDSS
0,9910 (0,4482)
1/11
0.8891
0,0058
36.5735
1.6074
1.4803
4.3219
MOOD
0.9952 (0:4764)
9/11
0.8898
0,0072
36,0428
1,5089
1.4595
4.2277
Graph DIT-LC (Ous)
0.9826 (0.8974)
11/11
0.8941
0,9662
5.8940
1.1302
0.8341
3.1345
Graph DIT (Ours)
0.8242 (0.8974)
11/11
0.8788
0.9688
5.4287
1,0672
0.7843
2,9442
Gruph GA
1,0000 (N.A.)
11/11
0,8806
0.8760
9,4945
1.2599
2.3122
4,7050
MARS
1,0000 (N.A.)
10/11
0.1118
0.3269
33.6684
2,1089
2.3316
5,8333
LSTM-HC
0.9901 (N.A.)
10/11
0.8924
0.7804
17,6290
1.4530
1.2798
4,1070
Synt, & N₂ Perm
JTVAF.BO
1.0000 (N.A.)
10/11
0.7741
0.7264
22.5093
0.9414
1,2874
3.2911
DiGress
0,9798 (0.3326)
11/11
0.9150
0,2670
21.1077
2,7562
2.0242
5,9103
DiGress v2
0.9801 (0.3759)
11/11
0.9180
0,1842
20,7820
2.4734
1.9538
5,5614
GDSS
0.9941 (0.8000)
3/11
0.8543
0.0030
33,3815
1,5277
1,5886
4,3365
MOOD
0.9980 (0.4453)
11/11
0,8857
0,0028
34.9385
1,5087
1.7018
4.3261
Graph DIT-LC (Ous)
0,9803 (0.9054)
11/11
0.8894
0,9670
5.9049
1.1908
0.9721
3.3105
Gruph DIT (Ours)
0.8165 (0.9054)
11/11
0,8726
0,9713
5.7943
1,0969
0,9472
3.1124
Graph GA
1,0000 (N.A.)
11/11
0,8889
0,9134
7.1234
1.3427
188588
4.4079
MARS
1,0000 (N.A.)
11/11
0.8460
0,9083
8.9201
1.1623
1,4808
3,8612
LSTM-HC
0,9893 (N.A.)
10/11
0.8938
0.7262
1368
1,4018
1.1436
3.8079
Syath, & CO2 Perm
JTVAE
1.0000 (N.A.)
7/11
0.7671
0.7978
22,9047
1.0550
1.1663
3.2622
DiGress
0,9802 (0.3741)
11/11
0.9100
0.1576
19.61
2.4554
1.5377
5.1926
DiGress v2
0.9868 (0.2486)
11/11
0.9137
0,2686
20.1563
2,8087
1.5590
5.5939
GDSS
0.9876 (0.6987)
1/11
0.8786
0.0026
32.1841
1,4679
1.3584
4.0440
MOOD
0.9881 (0.7880)
11/11
0,8690
0.0025
30,9310
1.5463
1.3443
4.1464
Graph DIT-LC (Ours)
0.9836 (0.8841)
11/11
0.8916
0,9247
5.7776
1,2991
0.8603
3,3394
Graph DIT (Ours)
0.8291 (0.8841)
11/11
0.8743
0,9403
5,6815
1,2225
0.7728
3.1155
Table 6: Complete results on 1,000 generated polymers for the inverse O2/N₂ gas separation polymer design. # UB is the count of generated polymers successfully identified (by Oracle functions) as upper bound instances defined by Robeson (38]
Validity
Distribution Learning
Condition Control
Model
rulo checking
Cuverage
Divensity
Similartt
Distance
Symth 4
O2 L
N;
Avg. MAE
Graph GA
1,0000 (N A.)
10/11
0.8848
0,3734
29.9060
1.6545
1.8720
2,1984
8.7726
57
MARS
L.0000 A.)
11/11
0.7886
0.1922
32.5679
1.4909
1.7940
2.2170
9,3112
51
LSTM
0,9910
10/11
0,8040
0.1758
37.1556
1,3553
1.5066
1.8791
10:5529
45
ITVAE-BO
L00000 (N.A.)
2/11
0,7849
0.2541
33.6430
2.0723
1.7633
2.1998
9,7722
33
DiGress
0,9930 (0.3120)
9/11
09019
0.1156
30.5716
L6892
1.1680
1.3529
8.6061
DiGress v2
0.9940 (0.1760)
11/11
0.9075
0,2793
24.6239
2.1370
1,1847
1,3743
8:4707
85
GDSS
0,9910 (0.9180)
1/11
0.8210
0.0000
11.7199
2.3508
1.4097
1.8328
11.7813
56
MOOD
0.9960 (0.5640)
0.8833
0.0000
46.6095
1.5963
1,3921
1.7360
1288551
Graph DIT-LC
0.975 (0.7170)
10/11
0.8966
0.6401
26.1647
1.5119
0.8388
0.9035
3.2541
90
Graph DVT (Ouns)
0.7800(071100
11/11
0.8838
0.6028
26,3378
1.4081
0.7476
0.8213
2.9700
68
observe their competitive performance in multi-conniiional settings, characterized by high validity. good atom type coverage, and distribution similarity in Tables [ MARS [47] and JTVAE [1] perform well for controlling the synthetic accessibility score [2] but are less effective at controlling specific task properties like gas permeability, BACF, BBBP, and HIV. For example, in generation tasks with the categorical task condition, the generated examples only achieve around 50% accuracy in hitting the input condition.
16

---

Graph GA
(b) MARS
INC
(c) LSTM-HC
(d) ITVAE-BO
THA
(e) DiGress
(f) DiGress v2
(g) GDSS
(h) MOOD
Task
Task
(i) Graph DiT (Ours)
Figure 7: Distribution of training (grey-colored) and generated (orange-colored) molecules. The generated distribution in Figure is from Graph DiT, and the visualization shows that the generated molecular data points fit the training distribution well, with reasonable interpolation and extrapolation the training data space.
C.3 Discussion on Training Dynamics
In Figure 8 we illustrate changes in various indicators on the validation set during model training. We note an increase in generated validity and similarity to the validation reference set, along with a decrease in distance to the reference set and errors between generated properties and conditions, indicating gradual improvement in conditional generation over epochs. However, trade-off between distribution fitting and internal diversity is observed in our current model, suggesting that further work enhancing generation diversity could be promising.

---

Vabilly al Check
Gradarly
231
2.1
202 180
605
100
3070
3738
1200
200 1800
14:10
Sychnorrilyy
10.07
(a) OyPerm Only
(b) COyPerm Only
Vildi na Check
Sumntrr
10l00 m2 Creck
anduury
2.4
673
0.0
1800
600
(c) N₂ Perm Only
(d) BACE
Vilei AJ Chec
1-00
615
618
0.30
6.0
231
12.00
13.00
613-
(e) BBBP
(I) HIV
Figure 8: Change of indicators on the validation set during model training. The generated validity and similarity the validation reference set have increased, accompanied by decrease in distance the reference set and errors between generated properties and conditions. We also observe a trade-off' between distribution fitting and internal diversity over epochs.
C.4 Discussion on Uniqueness, and Novelty
We evaluate the model's performance on Novelty and Uniqueness. Unlike unconditional generation, multi-conditional generation involves generating multiple possible molecules under the same condi- tion. Therefore, we compute these metrics across different sets conditions rather than for generated molecules under the same conditions. Results are presented in Table 7
Graph DiT demonstrates reasonable performance on these metrics. However, higher Novelty and Uniqueness values do not necessarily indicate better performance, as they may not reflect the model's ability design satisfactory molecules with desirable properties. Moreover, these values risk leading to mislcading conclusions. For instance, AddCarbon achieves nearly perfect scores (99. 99.94% Novelty
18

---

Table 7: Comparison of Novelty and Uniqueness across different conditions
Metric
Graph GA
MARS
LSTM-HC
JTVAE-BO
Digress
DiGress v2
GDSS
MOOD
Graph DIT
Novelty
0.9950
1.0000
0,9507
1.0000
0.9908
0,9799
0,9190
0.9867
0.9702
Uniqueness
1.0000
0.7500
0,9550
0.6757
1,0000
0,9730
0.1622
0.9820
0.8829
and 99.86% Uniqueness) according to 37 41 yet it randomly adds carbon atoms existing molecules, resulting in new molecules that are not practically useful [37].
D Details on Polymer Inverse Design
We aim to design polymers with high O₂ and low N₂ permeability, showing refined control of models these properties. This is reflected in the selectivity, defined the O2/N, permeability ratio. Robeson (38) has identified an inherent trade-off between gas permeability and selectivity, known as the upper-bound. Ideally, high bugh-performance polymers should fall in the above-the-bount region, demonstrating an effective combination of permeability and selectivity. We have 609 polymers with annotated permeability values for both gases. 16 above-the-boume polymers are included in the test/reference set and excluded from the training set. We generate 1,000 polymers conditional on test set labels.
D.1 Survey Setup on Generated Polymers
Figure 9: The reference polymer structure in the case study has conditions {SAS=3.8, SCS=4.3, OyPerm=34.0, 5.
We aim to gather expert evaluations the generation performance of various methods. We conduct a study using a test data point from the Oz/N₂ gas separation inverse design task, taking its properties conditions. The structure of the selected data point is presented in Figure We display 25 generated polymers, each with its properties, alongside three real polymers from the training dataset references. The first real polymer serves the test reference, while the other two, similar in properties to the first, also aid experts in assessing the generated polymers. The properties of these generated polymers are predicted using a well-trained random forest model. Experts are asked to rank the generated polymers from I to 25, considering: (I) Structures of real polymers with desirable properties; (2) Predicted properties of generated polymers, displayed beneath each visualization. Here, rank of represents the best example as per domain knowledge, while 25 is the least favorable. Rankings are then converted utility scores (UtS) ranging from 0 using ranking* allowing us to quantify the relative performance different generation methods. The agreement score (AS) could be obtained by exp (-25 x Variance (UtS, UtS UtSg, UtS.)), where UtS, denotes the utility score from the i-th domain expert. (3) Finally, we select top-3 polymers for each generative models and
present them in Figure 3
D.2 Results on Inverse Design
We present the inverse design results of all 1,000 generated polymers validity, distribution learning. and condition control in Table # UB is the count of generated polymers successfully identified as upper bound instances. Higher UB indicates that Graph DiT has a higher likelihood of generating candidates for excellent O₂ and N₂ gas separation. The smallest MAE across most properties and 9.9% average MAE improvement over baselines highlight Graph DiT's superior control in generating examples closely aligned with multiple conditions.
19

---

Validity
Synth.
0.9
1.6
0.8
1.4
0.7
1.2
Value from test set
Value from test set
0.6
1.0
200
400
600
800
1000
200
400
600
800
1000
22 property values
N2 property values
(a) Changes of Validity
(b) Changes of MAE to the target Synthesizability
N
O
1.6
1.4
1.2
1.0
Value from test set
0.8
Value
0.6
200
400
600
800
1000
0
200
400
600
800
1000
N2 property values
N2 property values
(c) Changes of MAE to the target N2
(d) Changes of MAE to the target O2
Figure 10: Analysis of Model Controllability when Varying N2 Values: The true N2 value from the test set is 213.75. We note that the controllability performance (i.e., MAE value) for N₂ and Og is measured in log space.
E Details and More Results on Model Analysis
E.1
Case Studies on Generation Controllability
We conduct new case study on the O2/Nz polymer dataset, studying the controllability on three properties synthesizability score, O2 and N₂ properties with varied N₂ property values. We select a polymer example from the test set and vary its N₂ while keeping other properties fixed. The N₂ property from the test polymer is 213.75, and we vary it from 0 to 1000. We sample 50 values within this range and generate 100 polymer graphs conditioned on multiple properties with each sampled value. We evaluate various metrics, including the chemical validity of the generated polymers.
We visualize results in Figure TO We consistently observe that validity and controllability per- formance improve as the values approach 213.75, derived from a real test polymer. Conversely, performance deteriorates when the sampled N₂ values are closer to the extremes of the sampling range (0 or 1000). This observation underscores the interdependency between conditions, where less frequent combinations of different properties may be more challenging to learn. Moreover, the model performs well across elatively large range from 0 to 1000 in terms of validity, O2, and synthesizability score control. This demonstrates good generalization of the proposed method in capturing complex condition interdependencies.
20

---

O.Perm
1.00
Distance
0.86
2,00
1.00
N;Perm
MLP Linear
Similarity
1.00
000
1.00
CO: Perm
1.00
Diversity
Figure 11: Ablation studies the final MLP layer
Table 8: Training Performance of Oracle Methods: train the models all polymers small molecules in a task to simulate the Oracle. Results from the random forest model are highlighted because has the lowest training MAE and highest training AUC.
OzPerm NgPerm CO2Perm BACE BBBP HIV (MAE)
(MAE)
(MAE)
(AUC) (AUC) (AUC)
Random Forest
0.3662
0.4006
0.3486
0.9895 0.9954 0.9996
Gaussian Process
1.9631
2.3806
1.8543
0.9610
0.9943 0.9511
Support Vector Machine
0.7462
0.9509
0.8594
0.8889
0.9472 0.9304
E.2
Ablation Studies Final MLP
In addition to the three components related to conditioning effectiveness of Graph DiT studied in 4.4 we also examine the importance of the final layer MLP for conditional graph denoising. Results in Figure 11 show that MLP significantly outperforms a linear layer 34
E.3 Details on Oracle Simulation
We train three types of Oracles based on Random Forest, Gaussian Process, and Support Vector Machine on all polymers or molecules in task to evaluate the properties of generated polymers conditional on O,Perm only, NyPerm only, COyPerm only, BACE, BBBP, HIV. The training performance (MAE or AUC) is presented in Table g Our findings show that the random forest achieves the lowest MAE and highest AUC scores, leading us to select it for simulating oracles in our generation evaluation process.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Discussions on the proposed method and empirical support can be found in sections 1, 3, and 4.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Discussions on generation diversity and Oracle evaluations are found in Sections 4.2 and 4.4.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
22

---

Answer:
[NA]
Justification:
The paper focuses empirical studies.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Details are in Section 4, appendix, and supplementary code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily to reproduce that algorithm.
algorithm, the paper should make it clear how
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution either be way to access this model for reproducing the results or way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
new model (e.g., large language model), then there should
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
23

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code is provided in the supplementary materials. Data and code will be on Github after publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
Policy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.ccppbbll/ oobblii/gudddss//ddebbbbbi
hat////oooSueiisseeeeeee weabbbisssiee SttSSubbissoooooiiiioo.. onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details are in Section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: Improvement percentages (> 10%) are discussed in Section 4.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Discussions are in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
The research conforms the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The work has positive societal impacts on designing molecules for drugs and materials, as discussed in Sections I and 6.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
25

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The work does present issues of high-risk misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All data sources and baseline models open-sourced. They have been properly credited and mentioned, as outlined in Section 4.1.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
26

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, datasets.paperssiit wwwppppppttthhhoco thcode www.coo//aaaaoo datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The experiment results are verified through survey conducted with small group of domain experts. However, no crowdsourcing experiments were conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
27

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "030gGRKHWc", "reviewer": "Reviewer_RRe9", "summary": "This work proposes Graph Diffusion Transformer (Graph DiT), which is a a molecular generation model based on multi-condition, and diffusion process. Graph DiT enables multi-conditional molecular generation, integrating multiple properties, eg. synthetic score and gas permeability.Graph DiT employs a graph-dependent noise model (instead of node level or edge level), and is claimed to improve noise estimation accuracy in molecules. Empirical validation across polymer and small molecule generation tasks shows Graph DiT’s better performance in condition control and distribution learning.", "questions": "- Since the model is Graph DiT, the methodology is unclear on how 'graph structure' is modeled within the architecture. If not, it would resemble as not a specific graph transformer based model.", "limitations": "yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The topic of how diffusion models can be further made effective for multi-conditional molecular generation is focused and respective limitation of existing work is illustrated with examples. Although there can be multiple directions to improve the integration of conditions, which are discussed.\n- The use of graph-dependent noise model forms the basis of enhancing noise estimation beyond what is seen in DIGRESS and similar models. While doing this, several challenges can arise. These challenges are discuss upon by the paper to a god extent.\n- Empirical results show improvements in multiple metrics compared to Digress, MOOD and other baselines.", "weaknesses": "- Due to the graph dependent noise, the model may be limited or unscalable to medium or large scale graphs.\n- Minor format comment: the color shades in the charts like in Figure 4 are inconsistent with their labels.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# W1: Scalability of graph dependent noise\n\n## Subgraph-level noise model is possible for larger graphs\n\nThank you for your comment. The transition matrix is manageable for molecules, as we only need to model heavy atoms, which usually number less than 50 for a molecule [1].\n\nFor larger graphs, we could explore building the matrix at the subgraph level in the future, treating subgraphs as nodes and maintaining only inter-subgraph edges. This approach could handle larger-scale graphs more efficiently.\n\n# W2: Color shades\n\nThank you for your comment. The discrepancy is due to different alpha values for transparency. We have updated Figure 4 with consistent alpha values and provided it in the rebuttal PDF.\n\n# Q1: How graph structure is modeled\n\nThank you for your question. Graph DiT differs from vision and language Transformers by using graph tokens. Given a node on the graph, our new graph token concatenates node features with all related edge features, preserving node connectivity.\n\n# Reference\n\n[1] Molecular sets (MOSES): a benchmarking platform for molecular generation models. Frontiers in Pharmacology. 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NZQnHJ3F4D", "reviewer": "Reviewer_hJ6K", "summary": "This research introduces the Graph Diffusion Transformer (Graph DiT) for generating molecules with multiple properties, such as synthetic score and gas permeability. Unlike previous models, Graph DiT uses a new noise model and a Transformer-based denoiser to better handle molecular structures. Experiments show that it performs well in generating both polymers and small molecules.", "questions": "1. How do you determine the $\\text{rank}\\( K \\)$ of molecules under single-condition constraints, and how does this compare to the $\\text{rank}\\( K \\)$ in multi-condition scenarios?\n\n2. Could you clarify the methodology used by the Oracle to rank these graphs, particularly how closely attributes of ranked molecules match the conditional attributes?\n\n3. Does the graph-dependent noise schedule enhance the validity of your method compared to a separate discrete diffusion schedule? Could you provide comparative results?\n\n4. Given the higher similarity and lower distance of your generated molecules to a reference set, how do you ensure that the novelty is not compromised?\n\n5. What advantages does using a learnable dropping embedding offer over simply excluding the condition encoder from the training process in unconditional generation?\n\n6. Have you explored generation with three or more conditions, such as Synth., HIV, and BBBP? If not, what constraints prevent such multi-condition generation?\n\n7. Could you provide examples of true polymers corresponding to the conditions shown in Figure 3, to facilitate a direct comparison and enhance the evaluation of your model’s performance?", "limitations": "NA", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper successfully applies the DiT framework from computer vision.\n\n2. The method jointly applies noise between atoms and bonds,", "weaknesses": "1. **Limited Demonstration of Multi-Condition Capability:** The paper demonstrates conditional generation with only two conditions, such as Synth. and HIV, yet claims capability for multiple conditions. To substantiate this claim, it would be beneficial to test on more complex condition sets such as GSK3β+JNK3+QED+SA, as evaluated in the MARS framework and also many other methods. Or at least three properties like Synth, HIV, and BBBP to validate multi-conditional generative capacity.\n\n2. **Unexplored Texture Conditions:** Given that the model incorporates strategies from DiT, it is imperative to assess how it performs under texture conditions. Testing under these conditions would provide a more comprehensive evaluation of the model's versatility and adherence to the principles derived from DiT.\n\n3. **Incomplete Evaluation Metrics for Unconditional Generation:** The evaluation metrics employed for unconditional generation do not adequately measure key aspects such as uniqueness, novelty, and FCD (Fréchet ChemNet Distance). Including these metrics would offer a more rounded understanding of the model’s performance in generating novel and diverse molecular structures.\n\n4. **Ambiguity in Graph-Dependent Noise Schedule Impact:** While the advantages of a graph-dependent noise schedule are discussed, Figure 4(c) does not provide clear empirical evidence of its impact, particularly in comparison with separate discrete diffusion schedules.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# W1: Limited Demonstration of Multi-Condition Capability\n\n## We evaluate models on up to four conditions, not two\n\nThank you for your comment. In Table 1, we evaluate all models on up to four conditions: $O_2$, $N_2$, and $CO_2$ permeability. \n\nBased on the above points, we respectfully request a reconsideration of the assessment that \"Limited Demonstration of Multi-Condition Capability\".\n\n# W2: Text condition\n\n## Text condition is out of the scope of the paper\n\nWe appreciate your comment. Property conditions align with our research objectives in inverse molecular design.\n\n## The DiT work [1] did not explore text conditions\n\nThe original DiT work [1] used classes from ImageNet rather than text conditions, leaving text conditioning for future work. \n\nBased on the above points, we respectfully request a reconsideration of the assessment that \"it is imperative to assess how it performs under texture condition\" and \"adherence to the principles derived from DiT\".\n\n# W3: Incomplete uniqueness, novelty, and FCD\n\n## FCD was used in the paper\n\nLine 208, Tables 1 and 2, and Figure 4 presented FCD as the distance metric.\n\n## Uniqueness and novelty are provided\n\nHere are results complementing Table 1:\n\n| Method     | Validity | Novelty | Uniqueness | V * N * U |\n|------------|----------|---------|------------|---------------------------------|\n| Graph GA   |   1.0000 |  0.9950 |     1.0000 |                          0.9950 |\n| MARS       |   1.0000 |  1.0000 |     0.7500 |                          0.7500 |\n| LSTM-HC    |   0.9910 |  0.9507 |     0.9550 |                          0.8997 |\n| JTVAE-BO   |   1.0000 |  1.0000 |     0.6847 |                          0.6847 |\n| Digress    |   0.9913 |  0.9908 |     0.9730 |                          0.9556 |\n| DiGress v2 |   0.9812 |  0.9799 |     0.9820 |                          0.9442 |\n| GDSS       |   0.9190 |  0.9190 |     0.1532 |                          0.1293 |\n| MOOD       |   0.9867 |  0.9867 |     0.9730 |                          0.9473 |\n| Graph DiT  |   0.9760 |  0.9702 |     0.8919 |                          0.8445 |\n\nGraph DiT achieves good results. However, we note these metrics alone don't necessarily indicate practical utility.\n\n## Uniqueness and novelty may be flawed metrics\n\nRecent research has shown uniqueness and novelty can be easily flawed [2]: Randomly adding carbons to existing molecules can yield almost 100% novelty and uniqueness. We have opted to use internal diversity, which offers a more nuanced reflection of generation diversity.\n\n## Tables 1 and 2 present up to 9 comprehensive metrics\n\nThe nine metrics (Lines 204-211) are comprehensive, evaluating chemical validity, distribution matching, and multi-condition controllability. They assess diverse and useful molecule generation. Section 4.3's expert case studies provide additional valuable model assessment.\n\n# W4: Ambiguity in graph-dependent noise\n\n## Figure 4$(c)$ shows ~2x improvement on controllability\n\nThank you for your feedback. Figure 4$(c)$ shows the non-dependent noise model has only 49-55% of the graph-dependent model's controllability for gas permeability, demonstrating a significant improvement. We respectfully request a re-examination of the results.\n\n# Q1: Ranking and K value\n\n## Details are in Lines 35-37, 56-57\n\nFor single-condition constraints (Lines 35-37):\n> we check whether a shared polymer structure that meets multi-property constraints can be identified across different condition sets. If we find the polymer, its rank K (where K is between 1 and 30) indicates how high it appears on the lists, considering all condition sets. If not, we set K as 30.\n> \n\nFor multi-condition generated graphs (Lines 56-57)\n> The Oracle determines the rank of this graph among 30 single-conditionally generated graphs for each condition.\n> \n\n# Q2: How oracle rank the molecular graphs\n\n## Details are in Appendix B.3 (Lines 520-522) \n\nFor how Oracle ranks polymers (Lines 520-522):\n> We rank these polymers based on the mean absolute error between the generated properties (evaluated by a random forest model trained on all the data to simulate the Oracle function) and the conditional property.\n\nWe use ranking positions to measure closeness to target conditions. For multi-conditional generated polymers, their median ranks are 4, 9, and 11 for Synth., $O_2$, and $N_2$ permeability (Lines 57-58, Figure 1).\n\n# Q3: Does Graph-dependent noise model enhance validity?\n\n## Yes, it improves validity from 0.4946 to 0.8245\n\n# Q4: Novelty\n\n## Good similarity and distance do not imply bad novelty\n\nNovelty is provided in response to W3.\n\nIn Line 204, the reference set consists of left-out test cases unknown during training. Therefore, good similarity and distance to the reference set do not indicate poor novelty, which is typically defined on the training set.\n\n# Q5: Learnable dropping embeddings\n\nIt is widely used [1,3,4] by DiT [1] and DALL-E 2 [4]. It enhances flexibility for handling missing values and improves training stability by learning representations for null embeddings.\n\n# Q6: Have you explored three or more conditions?\n\n## Yes, we explored and reported results for up to four conditions in Table 1\n\nThe HIV and BBBP datasets contain only one overlapping molecule. We may use learnable dropping embeddings to handle missing condition values. But we cannot obtain enough test cases for multi-conditional evaluations.\n\n# Q7: True polymer in Figure 3\n\nThe SMILES string is:\n```\nNC1=C(*)C=CC(=C1)C1=CC(N)=C(C=C1)N1C(=O)C2=CC=C(C=C2C1=O)C(C1=CC=C2C(=O)N(*)C(=O)C2=C1)(C(F)(F)F)C(F)(F)F\n```\n\nThe figure is in the rebuttal PDF and will be updated in the paper.\n\n# Reference\n[1] Scalable Diffusion Models with Transformers. ICCV 2023. \n\n[2] Genetic Algorithms are Strong Baselines for Molecule Generation. 2023.\n\n[3] Classifier-Free Diffusion Guidance. NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.\n\n[4] Hierarchical Text-Conditional Image Generation with CLIP Latents. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6dveeTmnEk", "reviewer": "Reviewer_N56G", "summary": "This paper presents Graph DiT, which initially learns conditions (including categorical and numerical properties) through clustering and one-hot encodings. Subsequently, Graph DiT utilizes a Transformer architecture during the diffusion denoising phase to refine noisy molecular graphs incorporating these conditions. Experimental results underscore Graph DiT's efficacy in multi-conditional generation and polymer inverse design, emphasizing its capability to innovate in molecule creation.", "questions": "- Please refer to the \"weaknesses\" section above.\n\n- The authors argue in lines 25 to 27 that the diverse scales and units of properties present a significant challenge in multi-property optimization. This diversity can complicate the comparison and combination of different properties, potentially leading to skewed optimization results. In my view, a straightforward solution to this issue could involve scaling the properties to a common range, such as 0 to 1. This approach would normalize the scales and facilitate fair comparisons and effective optimization across different properties, thereby addressing the challenge effectively.\n\n- What does \"LCC\" signify in DiT-LCC?\n\n- What visualization method is used in Figure 6? Is it a linear technique like PCA, or a non-linear method such as UMAP, utilized for dimensionality reduction?", "limitations": "None", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- **This paper makes somewhat novel contributions**: For example, unlike previous studies that treat node and edge state transitions independently, potentially misaligning with the denoising process, this research proposes a graph-dependent noise model. Graph DiT constructs a transition matrix based on the joint distribution of nodes and edges, enhancing the coherence of the denoising process.\n\n- **The experiments address three primary questions**: (1) The authors validated the generative capabilities of Graph DiT against baseline models from molecular optimization and diffusion. (2) The authors explored polymer inverse design for gas separation. (3) They conducted additional analysis to further examine the capabilities of Graph DiT.\n\n- A portion of the experimental results showed superior performance compared to the baselines.", "weaknesses": "- The paper should emphasize the main contributions and acknowledge its limitations.\n\n- The presentation of the paper should be improved as it is difficult to follow.\n\n- **Lack of experiments**: Similar to DiGress, the paper should compute statistics such as uniqueness, novelty, and VUN (Valid, Unique, and Novel graphs) to provide a comprehensive assessment of the method's efficacy and innovation in graph generation and optimization. These metrics are crucial for evaluating the quality and originality of the generated graphs, ensuring a thorough comparison with existing approaches in the field.\n\n- The experimental results in Table 2 did not demonstrate superior performance compared to other baseline methods. The effectiveness of the proposed Graph DiT appears somewhat trivial to some extent. Only the results in conditional control show good performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# W1, W2: Contribution, limitation, and paper presentation\n\nThank you for your comment. We can highlight lines 41-43 and 62-64 using italics to emphasize the contributions and organize them into bullet points. Any further suggestions or discussion are highly appreciated.\n\nWe discussed limitations in Section 4.4 (Lines 281, 288-289) regarding generation diversity and Oracle functions used for evaluation. We revise the main paper to acknowledge these again in the Conclusion.\n\nWe have corrected Figure (4) colors and revised Eq. (6) for clarity. Any further suggestions or discussions are welcome.\n\n# W2: Lack of metrics (Novelty & Uniqueness)\n\n## Results on required metrics \n\nWe show that Graph DiT achieves good values for novelty and uniqueness.\n\n| Method     | Validity | Novelty | Uniqueness | V * N * U |\n|------------|----------|---------|------------|---------------------------------|\n| Graph GA   |   1.0000 |  0.9950 |     1.0000 |                          0.9950 |\n| MARS       |   1.0000 |  1.0000 |     0.7500 |                          0.7500 |\n| LSTM-HC    |   0.9910 |  0.9507 |     0.9550 |                          0.8997 |\n| JTVAE-BO   |   1.0000 |  1.0000 |     0.6847 |                          0.6847 |\n| Digress    |   0.9913 |  0.9908 |     0.9730 |                          0.9556 |\n| DiGress v2 |   0.9812 |  0.9799 |     0.9820 |                          0.9442 |\n| GDSS       |   0.9190 |  0.9190 |     0.1532 |                          0.1293 |\n| MOOD       |   0.9867 |  0.9867 |     0.9730 |                          0.9473 |\n| Graph DiT  |   0.9760 |  0.9702 |     0.8919 |                          0.8445 |\n\n## Flaws in uniqueness and novelty\n\nWe did not choose uniqueness and novelty as major metrics because recent research shows they can be flawed [1]: randomly adding carbons to existing molecules can yield almost 100% novelty and uniqueness. Instead, we use internal diversity to better reflect generation diversity.\n\n## Current evaluation has 9 metrics, enhanced by expertise\n\nWe defend our nine metrics for evaluating molecular quality, practical utility, and diversity. These metrics cover chemical validity, distribution matching (diversity, distance, similarity), and multi-condition controllability.\n\nAdditionally, Section 4.3 includes case studies with domain experts, providing valuable assessments often lacking in previous studies.\n\nBased on the above points, we respectfully request a reconsideration of the assessment if the reviewer finds these metrics sufficient. We welcome further discussion on this matter.\n\n# W3: Results in Table 2\n\n## From 0.6 to 0.9, Graph DiT significantly outperforms the baselines\n\nThank you for your comment. Condition control is a core goal in inverse molecular design, as we need to design drugs/materials that meet human requirements. As shown in Table 2, Graph DiT effectively improves existing baselines from 0.6 to 0.9 in this regard.\n\nBased on the above points, we respectfully request a reconsideration of the assessment that \"... results in Table 2 did not demonstrate superior performance...\"\n\n## More thoughts on effectiveness and practical utility\n\nWe value the reviewer's opinion that an ideal model should excel across all metrics. However, we respectfully argue that defining ground-truth diversity and distribution distance is still debated with many choices (e.g. uniqueness or internal diversity). Therefore, we use up to 9 metrics to comprehensively evaluate model performance from diverse perspectives. Comparing very similar numbers, such as a distance metric of 6.7 vs. 7.0, is less compelling for demonstrating a model's ability to generate practically useful molecules. Significant improvement in multi-conditional controllability is more indicative of practical utility. Additionally, in Section 4.3, we involve domain experts to verify the generated results in real applications.\n\nWe greatly appreciate the reviewer’s comments and invite further discussion on the matter for any remaining concerns.\n\n# Q2: Diverse scales of properties\n\n## Lines 217-218 align with the reviewer’s question\n\nThank you for the excellent question. As detailed in Lines 217-218, our implementation for molecular optimization baselines aligns with your question. Results in Tables 1 and 2 demonstrate that Graph DiT effectively outperforms these baselines.\n\n# Q3 LCC meanings\n\n## Lines 185-188: LCC denotes largest connected component\n\nThank you for your question. Please see Lines 185-188 for more:\n> A common way of converting generated graphs to molecules selects only the largest connected component [42], denoted as Graph DiT-LCC in our model. For Graph DiT, we connect all components by randomly selecting atoms. It minimally alters the generated structure to more accurately reflect model performance than Graph DiT-LCC.\n> \n\n# Q4: Visualization method in Figure 6\n\nWe use PCA (Principal Component Analysis) to reduce the dimensionality of Morgan Fingerprints [2] to two dimensions for visualization.\n\n# Reference\n\n[1] Genetic Algorithms are Strong Baselines for Molecule Generation. 2023.\n\n[2] Extended-Connectivity Fingerprints. JCIM 2010.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ivAFSyOx5S", "reviewer": "Reviewer_rG3q", "summary": "In molecular generation tasks, there is often a desire for the generator to produce molecules that satisfy multiple properties simultaneously. Previous architectural designs did not pay particular attention to the scenario of optimizing multiple constraints in tandem. Therefore, the author has proposed a Multi-Conditional Graph Diffusion Transformer. In the noise addition phase, the author has employed a novel noise addition method, successfully establishing connections between the graph's nodes and edges. At the same time, the author has designed a new condition encoder and a graph diffusion transformer architecture. This architecture is capable of addressing both numerical and categorical conditions simultaneously. Experiments have demonstrated that the proposed method exhibits excellent performance.", "questions": "**Major Issues**\n\n- The comparison presented in Figure 1 is not fair. Molecules that rank low on a certain single property may not necessarily rank low on other properties. Therefore, for a fair comparison, the statistical graph shown in Figure 1(b) should follow the same format as in Figure 1(a). That is, the authors should calculate and visualize the minimal $ K $ at which a molecule can satisfy all the constraints within rank $ K $ for all the generated molecules.\n- All the datasets used for drug design suffer from a severe class imbalance issue, i.e., the ratio of positive to negative samples is less than 1 to 20. Could the authors specify the approximate ratio of positive to negative samples in the input condition (label) when calculating the accuracy in Table 2? If the ratio of positive to negative samples follows the original dataset's proportion, using ROC-AUC seems to be a more reasonable metric.\n- How is the Avg. Rank calculated in Table 2?\n- In section 3.1, if the setting of $\\mathbf{Q}\\_V$ follows the setting of Digress, and $\\mathbf{Q}\\_{EV}$ is derived from the marginal distribution of $V$, then we have, for any $k$, $\\sum\\_{i=1}^{F_V}\\mathbf{P}\\_{k,i}=N+1 \\neq 1$, where $\\mathbf{P}=\\mathbf{X}\\_G^{t-1}\\mathbf{Q}\\_G^t$, $\\mathbf{P}\\_{k,i}$ represents the probability of node $k$ being of category $i$. The edges have the same issue. Therefore, $q(\\mathbf{X}\\_G^t|\\mathbf{X}_{G}^{t-1})$, as defined by the formula in the article, does not seem to be a distribution. However, in the specific implementation process, the graph is sampled from the \"distribution\" instead of directly using the probability as the edge and node features, which means the flaws mentioned above  theoretically should not affect the model's performance. But the authors need to make corrections in the writing.\n\n**Minor Issues**\n\n- Many diffusion-based generation methods for conditional generation only require the introduction of conditions during the generation process, without the need to incorporate conditions during the training process. Can the proposed method achieve the same?\n\n\n**If you can address all the major issues, I would be very willing to raise my score**", "limitations": "I believe the societal impacts and limitations of this work are discussed to a sufficient extent.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The writing is mostly clear and easy to follow\n- The evaluation is comprehensive.", "weaknesses": "- some comparisons are not fair, see the questions for detail.\n- some metrics of evaluation is not so suitable, see questions for detail", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Major 1: Ranking comparison in Figure 1\n\n## Results are updated to show ~$2\\times$ improvement\n\nThank you for your suggestion. We have created a new figure (attached in the rebuttal PDF) based on your feedback. We first identify the maximum ranking position among the three single-conditional generation sets for each multi-condition generated polymer. Then, we calculate the median of these maximum ranking positions, which is 16—approximately $2\\times$ better than single-conditional generation, which has a median value greater than 30.\n\n# Major 2: Data imbalance issue\n\n## The datasets are balanced, and the accuracy is reasonable\n\nThank you for your comment. Please refer to Line 200:\n> For drug design, we create three class-balanced datasets from MoleculeNet [45]: HIV, BBBP, and BACE...\n\nThe positive-to-negative ratio is 1:1, making accuracy a reasonable metric in this case.\n\n# Major 4: How avg. rank calculated\n\nIt is calculated by averaging the ranking positions of model performance for each condition (property).\n\n# Major 5: The issue with $q(\\mathbf{X}_G^{t} | \\mathbf{X}_G^{t-1})$\n\n## We correct this issue\n\nThank you for pointing this out! The term $\\mathbf{X}_G^{t-1}\\mathbf{Q}_G^{t}$ results in unnormalized probabilities. In the implementation, we need to split it into node and edge components and normalize them separately. We have corrected this issue and revised the description of Eq. (6) as follows:\n\nWe introduce a new diffusion noise model. At each forward diffusion step $t$, noise is applied to $\\mathbf{X}_G^{t-1}$, resulting in an unnormalized probability $\\mathbf{\\tilde{p}}= \\mathbf{X}_G^{t-1} \\mathbf{Q}_G^t$. We first separate and normalize the first $F_V$ columns of $\\mathbf{\\tilde{p}}$ to obtain the noisy node states $\\mathbf{X}_V^t$. We then reshape and normalize the remaining $N \\cdot E$ dimensions to obtain the edge states $\\mathbf{X}_E^{t}$. These components are combined to construct $\\mathbf{X}_G^{t}$.\n\n# Minor 1: Can the proposed method achieve no condition during training \n\nYes, here are two strategies: \n\n1. Replace the condition encoder with a molecular encoder that uses a self-supervised task during large-scale pretraining. In this case, the generation conditions should be the molecular structure rather than labels. For property conditions, we can retrieve molecular structures with similar labels.\n2. Learn null embeddings of the conditions during pretraining. Although fine-tuning the condition encoder for specific tasks will still be necessary, pretraining can help reduce costs and label requirements.\n\nAll these strategies are promising directions for extending Graph DiT.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the time and effort of all the reviewers in evaluating our work. In response to the following comments, we have attached a PDF with three figures

1. **RRe9**: "Minor format comment: the color shades in the charts like in Figure 4 are inconsistent with their labels."

2. **hJ6K**: "Could you provide examples of true polymers corresponding to the conditions shown in Figure 3"

3. **rG3q**: "The authors should calculate and visualize the minimal $K$ at which a molecule can satisfy all constraints within rank $K$ for all generated molecules."

We believe that all concerns have been adequately addressed. Should there be any important issues that we may not have fully addressed, we would greatly appreciate the opportunity to discuss them further.

Meta-review: The paper contributes Graph Diffusion Transformer (Graph DiT), a diffusion-based model which learns representations of multiple conditions using an additional encoder. The paper contributes also a novel graph-dependent noise model for molecules. During the rebuttal period, the Authors have performed additional analyses and provided novel results, specifically, values of Validity, Novelty and Uniqueness, as requested by the Reviewers. These values were not the best among the compared models. However, other metrics showing favourable performance have been provided.

The contribution is of interest for the NeurIPS community.
Decision: Accept (oral)